2024-04-02T13:31:31.602 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-04-02T13:31:31.602 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi --owner scheduled_pdonnell@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsx}} --verbose --archive /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635602 -- /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635602/orig.config.yaml 2024-04-02T13:31:31.663 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T13:31:31.756 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635602 branch: wip-batrick-testing-20240402.004512 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsx}} email: pdonnell@redhat.com first_in_suite: false job_id: '7635602' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240402.004512 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240402.004512 sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2763 sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 sleep_before_teardown: 0 subset: 318/512 suite: fs suite_branch: wip-batrick-testing-20240402.004512 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi052.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ4hvD4dzn+gihLLou7tjNJeN+pPC+08Djcuvx2kkrTbIFRynm6+soIhDSKaNFH5WTOXNg0msf8ZUHbDhabFDLM= smithi088.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLEnBstAMH77Vw5uhL3R5WQH+sBW3hvPvnK1HFA2D4RUwPKYRw1e9a4xZzv2/fShvfc5EeYAVjQtNzAYZaTZxHs= smithi189.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPIdCMVgKBubA6WSEoJW9Dl19J6G5iL0VTTqWlyAu+Agou+bdmHcqVF5rdr2e5CSJ5YWxSQVBMBPWbIz1mnjZZM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-04-02_11:52:43 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-02T13:31:31.756 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa; will attempt to use it 2024-04-02T13:31:31.757 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks 2024-04-02T13:31:31.757 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-02T13:31:31.759 INFO:teuthology.task.internal:Checking packages... 2024-04-02T13:31:31.783 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '191d963ac9de7322da8aff20b567b3fa458e2ef6' 2024-04-02T13:31:31.783 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-02T13:31:31.783 INFO:teuthology.packaging:ref: None 2024-04-02T13:31:31.783 INFO:teuthology.packaging:tag: None 2024-04-02T13:31:31.783 INFO:teuthology.packaging:branch: wip-batrick-testing-20240402.004512 2024-04-02T13:31:31.783 INFO:teuthology.packaging:sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:31:31.783 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240402.004512 2024-04-02T13:31:31.995 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2609.g191d963a 2024-04-02T13:31:31.997 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-02T13:31:32.003 INFO:teuthology.task.internal:no buildpackages task found 2024-04-02T13:31:32.003 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-02T13:31:32.009 INFO:teuthology.task.internal:Saving configuration 2024-04-02T13:31:32.030 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-02T13:31:32.036 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T13:31:32.056 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi052.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635602', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 13:25:59.065404', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ4hvD4dzn+gihLLou7tjNJeN+pPC+08Djcuvx2kkrTbIFRynm6+soIhDSKaNFH5WTOXNg0msf8ZUHbDhabFDLM='} 2024-04-02T13:31:32.079 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi088.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635602', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 13:25:59.066622', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLEnBstAMH77Vw5uhL3R5WQH+sBW3hvPvnK1HFA2D4RUwPKYRw1e9a4xZzv2/fShvfc5EeYAVjQtNzAYZaTZxHs='} 2024-04-02T13:31:32.096 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi189.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635602', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 13:25:59.063604', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPIdCMVgKBubA6WSEoJW9Dl19J6G5iL0VTTqWlyAu+Agou+bdmHcqVF5rdr2e5CSJ5YWxSQVBMBPWbIz1mnjZZM='} 2024-04-02T13:31:32.096 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-02T13:31:32.102 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-02T13:31:32.102 INFO:teuthology.task.internal:roles: ubuntu@smithi088.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-02T13:31:32.102 INFO:teuthology.task.internal:roles: ubuntu@smithi189.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-02T13:31:32.102 INFO:teuthology.run_tasks:Running task console_log... 2024-04-02T13:31:32.198 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7feb9a570160>, signals=[15]) 2024-04-02T13:31:32.198 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-02T13:31:32.204 INFO:teuthology.task.internal:Opening connections... 2024-04-02T13:31:32.204 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi052.front.sepia.ceph.com 2024-04-02T13:31:32.206 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:31:32.311 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi088.front.sepia.ceph.com 2024-04-02T13:31:32.312 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:31:32.406 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi189.front.sepia.ceph.com 2024-04-02T13:31:32.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:31:32.494 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-02T13:31:32.500 DEBUG:teuthology.orchestra.run.smithi052:> uname -m 2024-04-02T13:31:32.519 INFO:teuthology.orchestra.run.smithi052.stdout:x86_64 2024-04-02T13:31:32.519 DEBUG:teuthology.orchestra.run.smithi052:> cat /etc/os-release 2024-04-02T13:31:32.578 INFO:teuthology.orchestra.run.smithi052.stdout:NAME="CentOS Stream" 2024-04-02T13:31:32.578 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION="9" 2024-04-02T13:31:32.578 INFO:teuthology.orchestra.run.smithi052.stdout:ID="centos" 2024-04-02T13:31:32.578 INFO:teuthology.orchestra.run.smithi052.stdout:ID_LIKE="rhel fedora" 2024-04-02T13:31:32.578 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION_ID="9" 2024-04-02T13:31:32.578 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM_ID="platform:el9" 2024-04-02T13:31:32.578 INFO:teuthology.orchestra.run.smithi052.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-02T13:31:32.578 INFO:teuthology.orchestra.run.smithi052.stdout:ANSI_COLOR="0;31" 2024-04-02T13:31:32.579 INFO:teuthology.orchestra.run.smithi052.stdout:LOGO="fedora-logo-icon" 2024-04-02T13:31:32.579 INFO:teuthology.orchestra.run.smithi052.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-02T13:31:32.579 INFO:teuthology.orchestra.run.smithi052.stdout:HOME_URL="https://centos.org/" 2024-04-02T13:31:32.579 INFO:teuthology.orchestra.run.smithi052.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T13:31:32.579 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-02T13:31:32.579 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T13:31:32.580 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server 2024-04-02T13:31:32.623 DEBUG:teuthology.orchestra.run.smithi088:> uname -m 2024-04-02T13:31:32.642 INFO:teuthology.orchestra.run.smithi088.stdout:x86_64 2024-04-02T13:31:32.643 DEBUG:teuthology.orchestra.run.smithi088:> cat /etc/os-release 2024-04-02T13:31:32.705 INFO:teuthology.orchestra.run.smithi088.stdout:NAME="CentOS Stream" 2024-04-02T13:31:32.706 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION="9" 2024-04-02T13:31:32.706 INFO:teuthology.orchestra.run.smithi088.stdout:ID="centos" 2024-04-02T13:31:32.706 INFO:teuthology.orchestra.run.smithi088.stdout:ID_LIKE="rhel fedora" 2024-04-02T13:31:32.706 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_ID="9" 2024-04-02T13:31:32.706 INFO:teuthology.orchestra.run.smithi088.stdout:PLATFORM_ID="platform:el9" 2024-04-02T13:31:32.706 INFO:teuthology.orchestra.run.smithi088.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-02T13:31:32.706 INFO:teuthology.orchestra.run.smithi088.stdout:ANSI_COLOR="0;31" 2024-04-02T13:31:32.706 INFO:teuthology.orchestra.run.smithi088.stdout:LOGO="fedora-logo-icon" 2024-04-02T13:31:32.706 INFO:teuthology.orchestra.run.smithi088.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-02T13:31:32.706 INFO:teuthology.orchestra.run.smithi088.stdout:HOME_URL="https://centos.org/" 2024-04-02T13:31:32.706 INFO:teuthology.orchestra.run.smithi088.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T13:31:32.707 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-02T13:31:32.707 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T13:31:32.707 INFO:teuthology.lock.ops:Updating smithi088.front.sepia.ceph.com on lock server 2024-04-02T13:31:32.730 DEBUG:teuthology.orchestra.run.smithi189:> uname -m 2024-04-02T13:31:32.749 INFO:teuthology.orchestra.run.smithi189.stdout:x86_64 2024-04-02T13:31:32.749 DEBUG:teuthology.orchestra.run.smithi189:> cat /etc/os-release 2024-04-02T13:31:32.806 INFO:teuthology.orchestra.run.smithi189.stdout:NAME="CentOS Stream" 2024-04-02T13:31:32.806 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION="9" 2024-04-02T13:31:32.807 INFO:teuthology.orchestra.run.smithi189.stdout:ID="centos" 2024-04-02T13:31:32.807 INFO:teuthology.orchestra.run.smithi189.stdout:ID_LIKE="rhel fedora" 2024-04-02T13:31:32.807 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION_ID="9" 2024-04-02T13:31:32.807 INFO:teuthology.orchestra.run.smithi189.stdout:PLATFORM_ID="platform:el9" 2024-04-02T13:31:32.807 INFO:teuthology.orchestra.run.smithi189.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-02T13:31:32.807 INFO:teuthology.orchestra.run.smithi189.stdout:ANSI_COLOR="0;31" 2024-04-02T13:31:32.807 INFO:teuthology.orchestra.run.smithi189.stdout:LOGO="fedora-logo-icon" 2024-04-02T13:31:32.807 INFO:teuthology.orchestra.run.smithi189.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-02T13:31:32.807 INFO:teuthology.orchestra.run.smithi189.stdout:HOME_URL="https://centos.org/" 2024-04-02T13:31:32.807 INFO:teuthology.orchestra.run.smithi189.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T13:31:32.807 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-02T13:31:32.807 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T13:31:32.808 INFO:teuthology.lock.ops:Updating smithi189.front.sepia.ceph.com on lock server 2024-04-02T13:31:32.830 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-02T13:31:32.851 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-02T13:31:32.862 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-02T13:31:32.863 DEBUG:teuthology.orchestra.run.smithi052:> test '!' -e /home/ubuntu/cephtest 2024-04-02T13:31:32.866 DEBUG:teuthology.orchestra.run.smithi088:> test '!' -e /home/ubuntu/cephtest 2024-04-02T13:31:32.868 DEBUG:teuthology.orchestra.run.smithi189:> test '!' -e /home/ubuntu/cephtest 2024-04-02T13:31:32.884 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-02T13:31:32.914 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-02T13:31:32.915 DEBUG:teuthology.orchestra.run.smithi052:> test -z $(ls -A /var/lib/ceph) 2024-04-02T13:31:32.923 DEBUG:teuthology.orchestra.run.smithi088:> test -z $(ls -A /var/lib/ceph) 2024-04-02T13:31:32.926 DEBUG:teuthology.orchestra.run.smithi189:> test -z $(ls -A /var/lib/ceph) 2024-04-02T13:31:33.025 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-02T13:31:33.178 INFO:teuthology.run_tasks:Running task kernel... 2024-04-02T13:31:33.207 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-02T13:31:33.207 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-02T13:31:33.207 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-02T13:31:33.208 DEBUG:teuthology.orchestra.run.smithi052:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-02T13:31:33.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T13:31:33.227 DEBUG:teuthology.orchestra.run.smithi052:> uname -r 2024-04-02T13:31:33.287 INFO:teuthology.orchestra.run.smithi052.stdout:5.14.0-361.el9.x86_64 2024-04-02T13:31:33.288 INFO:teuthology.task.kernel:Running kernel on smithi052: 5.14.0-361.el9.x86_64 2024-04-02T13:31:33.288 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum install -y kernel 2024-04-02T13:31:35.527 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-04-02T13:31:36.969 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS-9 - BaseOS 7.4 MB/s | 2.0 MB 00:00 2024-04-02T13:31:38.252 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - BaseOS 8.7 MB/s | 8.0 MB 00:00 2024-04-02T13:31:41.203 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-04-02T13:31:54.452 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - CRB 660 kB/s | 6.1 MB 00:09 2024-04-02T13:31:55.953 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 16 kB 00:00 2024-04-02T13:31:59.805 INFO:teuthology.orchestra.run.smithi052.stdout:Extra Packages for Enterprise Linux 5.6 MB/s | 21 MB 00:03 2024-04-02T13:32:05.030 INFO:teuthology.orchestra.run.smithi052.stdout:lab-extras 50 kB/s | 1.7 kB 00:00 2024-04-02T13:32:05.235 INFO:teuthology.orchestra.run.smithi052.stdout:Last metadata expiration check: 0:00:01 ago on Tue 02 Apr 2024 01:32:04 PM UTC. 2024-04-02T13:32:07.512 INFO:teuthology.orchestra.run.smithi052.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-02T13:32:07.512 INFO:teuthology.orchestra.run.smithi052.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-02T13:32:07.727 INFO:teuthology.orchestra.run.smithi052.stdout:Dependencies resolved. 2024-04-02T13:32:07.736 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2024-04-02T13:32:07.736 INFO:teuthology.orchestra.run.smithi052.stdout: Package Arch Version Repository Size 2024-04-02T13:32:07.736 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2024-04-02T13:32:07.736 INFO:teuthology.orchestra.run.smithi052.stdout:Installing: 2024-04-02T13:32:07.736 INFO:teuthology.orchestra.run.smithi052.stdout: kernel x86_64 5.14.0-434.el9 CentOS-BaseOS 192 k 2024-04-02T13:32:07.736 INFO:teuthology.orchestra.run.smithi052.stdout:Installing dependencies: 2024-04-02T13:32:07.736 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-core x86_64 5.14.0-434.el9 CentOS-BaseOS 15 M 2024-04-02T13:32:07.736 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-modules x86_64 5.14.0-434.el9 CentOS-BaseOS 34 M 2024-04-02T13:32:07.736 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-modules-core x86_64 5.14.0-434.el9 CentOS-BaseOS 28 M 2024-04-02T13:32:07.736 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:32:07.736 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction Summary 2024-04-02T13:32:07.737 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2024-04-02T13:32:07.737 INFO:teuthology.orchestra.run.smithi052.stdout:Install 4 Packages 2024-04-02T13:32:07.737 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:32:07.737 INFO:teuthology.orchestra.run.smithi052.stdout:Total download size: 76 M 2024-04-02T13:32:07.737 INFO:teuthology.orchestra.run.smithi052.stdout:Installed size: 123 M 2024-04-02T13:32:07.737 INFO:teuthology.orchestra.run.smithi052.stdout:Downloading Packages: 2024-04-02T13:32:07.923 INFO:teuthology.orchestra.run.smithi052.stdout:(1/4): kernel-5.14.0-434.el9.x86_64.rpm 1.0 MB/s | 192 kB 00:00 2024-04-02T13:32:09.408 INFO:teuthology.orchestra.run.smithi052.stdout:(2/4): kernel-core-5.14.0-434.el9.x86_64.rpm 9.0 MB/s | 15 MB 00:01 2024-04-02T13:32:10.157 INFO:teuthology.orchestra.run.smithi052.stdout:(3/4): kernel-modules-5.14.0-434.el9.x86_64.rpm 14 MB/s | 34 MB 00:02 2024-04-02T13:32:10.407 INFO:teuthology.orchestra.run.smithi052.stdout:(4/4): kernel-modules-core-5.14.0-434.el9.x86_6 11 MB/s | 28 MB 00:02 2024-04-02T13:32:10.410 INFO:teuthology.orchestra.run.smithi052.stdout:-------------------------------------------------------------------------------- 2024-04-02T13:32:10.410 INFO:teuthology.orchestra.run.smithi052.stdout:Total 29 MB/s | 76 MB 00:02 2024-04-02T13:32:10.410 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction check 2024-04-02T13:32:10.850 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction check succeeded. 2024-04-02T13:32:10.851 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction test 2024-04-02T13:32:11.710 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction test succeeded. 2024-04-02T13:32:11.710 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction 2024-04-02T13:32:14.157 INFO:teuthology.orchestra.run.smithi052.stdout: Preparing : 1/1 2024-04-02T13:32:14.772 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : kernel-modules-core-5.14.0-434.el9.x86_64 1/4 2024-04-02T13:32:15.024 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T13:32:17.918 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T13:32:18.053 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T13:32:23.123 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T13:32:23.160 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:32:28.258 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:32:58.907 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:32:58.908 INFO:teuthology.orchestra.run.smithi052.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-02T13:32:58.908 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:32:58.953 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:32:59.345 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:32:59.345 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : kernel-5.14.0-434.el9.x86_64 1/4 2024-04-02T13:32:59.345 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T13:32:59.345 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T13:32:59.933 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:32:59.934 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:32:59.934 INFO:teuthology.orchestra.run.smithi052.stdout:Installed: 2024-04-02T13:32:59.934 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-5.14.0-434.el9.x86_64 2024-04-02T13:32:59.934 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-core-5.14.0-434.el9.x86_64 2024-04-02T13:32:59.934 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-modules-5.14.0-434.el9.x86_64 2024-04-02T13:32:59.934 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-modules-core-5.14.0-434.el9.x86_64 2024-04-02T13:32:59.934 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:32:59.935 INFO:teuthology.orchestra.run.smithi052.stdout:Complete! 2024-04-02T13:33:00.272 DEBUG:teuthology.orchestra.run.smithi052:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T13:33:00.420 INFO:teuthology.orchestra.run.smithi052.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T13:33:00.420 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T13:33:00.420 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-434.el9.x86_64 2024-04-02T13:33:00.420 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-02T13:33:00.421 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-02T13:33:00.421 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-04-02T13:33:00.421 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi052.front.sepia.ceph.com, path=None, version=distro) 2024-04-02T13:33:00.421 DEBUG:teuthology.orchestra.run.smithi052:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T13:33:00.504 INFO:teuthology.orchestra.run.smithi052.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T13:33:00.504 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T13:33:00.505 DEBUG:teuthology.orchestra.run.smithi052:> sudo rpm -qi grub2-tools 2024-04-02T13:33:00.543 INFO:teuthology.orchestra.run.smithi052.stdout:Name : grub2-tools 2024-04-02T13:33:00.543 INFO:teuthology.orchestra.run.smithi052.stdout:Epoch : 1 2024-04-02T13:33:00.543 INFO:teuthology.orchestra.run.smithi052.stdout:Version : 2.06 2024-04-02T13:33:00.543 INFO:teuthology.orchestra.run.smithi052.stdout:Release : 68.el9 2024-04-02T13:33:00.543 INFO:teuthology.orchestra.run.smithi052.stdout:Architecture: x86_64 2024-04-02T13:33:00.543 INFO:teuthology.orchestra.run.smithi052.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-02T13:33:00.543 INFO:teuthology.orchestra.run.smithi052.stdout:Group : Unspecified 2024-04-02T13:33:00.543 INFO:teuthology.orchestra.run.smithi052.stdout:Size : 8264254 2024-04-02T13:33:00.544 INFO:teuthology.orchestra.run.smithi052.stdout:License : GPLv3+ 2024-04-02T13:33:00.544 INFO:teuthology.orchestra.run.smithi052.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-02T13:33:00.544 INFO:teuthology.orchestra.run.smithi052.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-02T13:33:00.544 INFO:teuthology.orchestra.run.smithi052.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-02T13:33:00.544 INFO:teuthology.orchestra.run.smithi052.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-02T13:33:00.544 INFO:teuthology.orchestra.run.smithi052.stdout:Packager : builder@centos.org 2024-04-02T13:33:00.544 INFO:teuthology.orchestra.run.smithi052.stdout:Vendor : CentOS 2024-04-02T13:33:00.544 INFO:teuthology.orchestra.run.smithi052.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-02T13:33:00.544 INFO:teuthology.orchestra.run.smithi052.stdout:Summary : Support tools for GRUB. 2024-04-02T13:33:00.545 INFO:teuthology.orchestra.run.smithi052.stdout:Description : 2024-04-02T13:33:00.545 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:33:00.545 INFO:teuthology.orchestra.run.smithi052.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-02T13:33:00.545 INFO:teuthology.orchestra.run.smithi052.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-02T13:33:00.545 INFO:teuthology.orchestra.run.smithi052.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-02T13:33:00.545 INFO:teuthology.orchestra.run.smithi052.stdout:hardware devices. 2024-04-02T13:33:00.545 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:33:00.545 INFO:teuthology.orchestra.run.smithi052.stdout:This subpackage provides tools for support of all platforms. 2024-04-02T13:33:00.547 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-02T13:33:00.547 INFO:teuthology.task.kernel:Updating grub on smithi052 to boot 5.14.0-434.el9.x86_64 2024-04-02T13:33:00.547 DEBUG:teuthology.orchestra.run.smithi052:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-02T13:33:01.135 INFO:teuthology.orchestra.run.smithi052.stderr:Generating grub configuration file ... 2024-04-02T13:33:02.633 INFO:teuthology.orchestra.run.smithi052.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-02T13:33:02.649 INFO:teuthology.orchestra.run.smithi052.stderr:done 2024-04-02T13:33:02.659 DEBUG:teuthology.orchestra.run.smithi052:> mktemp 2024-04-02T13:33:02.674 INFO:teuthology.orchestra.run.smithi052.stdout:/tmp/tmp.wDBkKJln8b 2024-04-02T13:33:02.675 DEBUG:teuthology.orchestra.run.smithi052:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wDBkKJln8b 2024-04-02T13:33:02.747 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 0666 /tmp/tmp.wDBkKJln8b 2024-04-02T13:33:02.891 DEBUG:teuthology.orchestra.remote:smithi052:/tmp/tmp.wDBkKJln8b is 6KB 2024-04-02T13:33:02.940 DEBUG:teuthology.orchestra.run.smithi052:> rm -fr /tmp/tmp.wDBkKJln8b 2024-04-02T13:33:02.954 DEBUG:teuthology.orchestra.run.smithi052:> sudo /bin/ls /boot/loader/entries || true 2024-04-02T13:33:03.022 INFO:teuthology.orchestra.run.smithi052.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-02T13:33:03.023 INFO:teuthology.orchestra.run.smithi052.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-02T13:33:03.023 INFO:teuthology.orchestra.run.smithi052.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-02T13:33:03.023 INFO:teuthology.orchestra.run.smithi052.stdout:15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64.conf 2024-04-02T13:33:03.024 DEBUG:teuthology.orchestra.run.smithi052:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64 2024-04-02T13:33:03.168 DEBUG:teuthology.orchestra.run.smithi052:> sudo shutdown -r now 2024-04-02T13:33:33.172 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-02T13:33:33.172 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-04-02T13:33:33.173 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:33:51.518 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.52 2024-04-02T13:34:00.519 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-04-02T13:34:00.520 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:35:00.522 DEBUG:teuthology.orchestra.remote:timed out 2024-04-02T13:35:12.524 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-04-02T13:35:12.525 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:35:12.877 DEBUG:teuthology.orchestra.run.smithi052:> true 2024-04-02T13:35:13.258 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-04-02T13:35:13.258 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-434.el9.x86_64"... 2024-04-02T13:35:13.259 DEBUG:teuthology.orchestra.run.smithi052:> uname -r 2024-04-02T13:35:13.274 INFO:teuthology.orchestra.run.smithi052.stdout:5.14.0-434.el9.x86_64 2024-04-02T13:35:13.274 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-434.el9.x86_64 vs 5.14.0-434.el9.x86_64 2024-04-02T13:35:13.274 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-02T13:35:13.274 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-02T13:35:14.276 DEBUG:teuthology.parallel:result is None 2024-04-02T13:35:14.276 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-02T13:35:14.283 INFO:teuthology.task.internal:Creating test directory... 2024-04-02T13:35:14.284 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-02T13:35:14.286 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-02T13:35:14.288 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-02T13:35:14.306 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-02T13:35:14.313 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-02T13:35:14.320 INFO:teuthology.task.internal:Creating archive directory... 2024-04-02T13:35:14.321 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-02T13:35:14.344 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-02T13:35:14.347 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-02T13:35:14.446 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-02T13:35:14.453 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-02T13:35:14.453 DEBUG:teuthology.orchestra.run.smithi088:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-02T13:35:14.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T13:35:14.469 DEBUG:teuthology.orchestra.run.smithi189:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-02T13:35:14.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T13:35:14.486 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-02T13:35:14.488 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-02T13:35:14.513 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-02T13:35:14.561 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T13:35:14.565 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T13:35:14.584 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T13:35:14.587 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T13:35:14.589 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T13:35:14.620 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T13:35:14.622 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-02T13:35:14.629 INFO:teuthology.task.internal:Configuring sudo... 2024-04-02T13:35:14.630 DEBUG:teuthology.orchestra.run.smithi052:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-02T13:35:14.632 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-02T13:35:14.635 DEBUG:teuthology.orchestra.run.smithi189:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-02T13:35:14.664 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-02T13:35:14.673 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-02T13:35:14.674 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-02T13:35:14.702 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-02T13:35:14.704 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-02T13:35:14.720 DEBUG:teuthology.orchestra.run.smithi052:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T13:35:14.803 DEBUG:teuthology.orchestra.run.smithi052:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T13:35:14.854 DEBUG:teuthology.orchestra.run.smithi052:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T13:35:14.937 DEBUG:teuthology.orchestra.run.smithi052:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T13:35:14.979 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-04-02T13:35:14.979 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-02T13:35:15.047 DEBUG:teuthology.orchestra.run.smithi088:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T13:35:15.106 DEBUG:teuthology.orchestra.run.smithi088:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T13:35:15.146 DEBUG:teuthology.orchestra.run.smithi088:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T13:35:15.233 DEBUG:teuthology.orchestra.run.smithi088:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T13:35:15.274 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-02T13:35:15.274 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-02T13:35:15.343 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T13:35:15.388 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T13:35:15.448 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T13:35:15.491 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T13:35:15.536 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-02T13:35:15.536 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-02T13:35:15.605 DEBUG:teuthology.orchestra.run.smithi052:> sudo service rsyslog restart 2024-04-02T13:35:15.607 DEBUG:teuthology.orchestra.run.smithi088:> sudo service rsyslog restart 2024-04-02T13:35:15.610 DEBUG:teuthology.orchestra.run.smithi189:> sudo service rsyslog restart 2024-04-02T13:35:15.645 INFO:teuthology.orchestra.run.smithi088.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T13:35:15.654 INFO:teuthology.orchestra.run.smithi052.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T13:35:15.691 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T13:35:16.126 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-02T13:35:16.134 INFO:teuthology.task.internal:Starting timer... 2024-04-02T13:35:16.134 INFO:teuthology.run_tasks:Running task pcp... 2024-04-02T13:35:16.157 INFO:teuthology.run_tasks:Running task selinux... 2024-04-02T13:35:16.167 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-02T13:35:16.168 DEBUG:teuthology.orchestra.run.smithi052:> sudo service auditd rotate 2024-04-02T13:35:16.222 INFO:teuthology.orchestra.run.smithi052.stdout:Rotating logs: 2024-04-02T13:35:16.224 DEBUG:teuthology.orchestra.run.smithi088:> sudo service auditd rotate 2024-04-02T13:35:16.285 INFO:teuthology.orchestra.run.smithi088.stdout:Rotating logs: 2024-04-02T13:35:16.286 DEBUG:teuthology.orchestra.run.smithi189:> sudo service auditd rotate 2024-04-02T13:35:16.351 INFO:teuthology.orchestra.run.smithi189.stdout:Rotating logs: 2024-04-02T13:35:16.353 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-02T13:35:16.353 DEBUG:teuthology.orchestra.run.smithi052:> /usr/sbin/getenforce 2024-04-02T13:35:16.382 INFO:teuthology.orchestra.run.smithi052.stdout:Permissive 2024-04-02T13:35:16.383 DEBUG:teuthology.orchestra.run.smithi088:> /usr/sbin/getenforce 2024-04-02T13:35:16.407 INFO:teuthology.orchestra.run.smithi088.stdout:Permissive 2024-04-02T13:35:16.407 DEBUG:teuthology.orchestra.run.smithi189:> /usr/sbin/getenforce 2024-04-02T13:35:16.432 INFO:teuthology.orchestra.run.smithi189.stdout:Permissive 2024-04-02T13:35:16.432 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi052.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi088.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi189.front.sepia.ceph.com': 'permissive'} 2024-04-02T13:35:16.432 DEBUG:teuthology.orchestra.run.smithi052:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T13:35:16.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T13:35:16.460 DEBUG:teuthology.orchestra.run.smithi088:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T13:35:16.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T13:35:16.490 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T13:35:16.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T13:35:16.517 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-02T13:35:16.517 DEBUG:teuthology.orchestra.run.smithi052:> sudo /usr/sbin/setenforce permissive 2024-04-02T13:35:16.548 DEBUG:teuthology.orchestra.run.smithi088:> sudo /usr/sbin/setenforce permissive 2024-04-02T13:35:16.584 DEBUG:teuthology.orchestra.run.smithi189:> sudo /usr/sbin/setenforce permissive 2024-04-02T13:35:16.617 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-02T13:35:16.626 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-02T13:35:16.643 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-02T13:35:16.644 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi052.front.sepia.ceph.com,smithi088.front.sepia.ceph.com,smithi189.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-02T13:39:50.456 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi052.front.sepia.ceph.com'), Remote(name='ubuntu@smithi088.front.sepia.ceph.com'), Remote(name='ubuntu@smithi189.front.sepia.ceph.com')] 2024-04-02T13:39:50.457 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-04-02T13:39:50.458 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:39:50.548 DEBUG:teuthology.orchestra.run.smithi052:> true 2024-04-02T13:39:50.629 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-04-02T13:39:50.629 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-04-02T13:39:50.630 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:39:50.718 DEBUG:teuthology.orchestra.run.smithi088:> true 2024-04-02T13:39:50.786 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-04-02T13:39:50.787 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-04-02T13:39:50.787 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:39:50.879 DEBUG:teuthology.orchestra.run.smithi189:> true 2024-04-02T13:39:50.949 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-04-02T13:39:50.950 INFO:teuthology.run_tasks:Running task clock... 2024-04-02T13:39:50.961 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-02T13:39:50.961 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-02T13:39:50.962 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T13:39:50.964 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-02T13:39:50.964 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T13:39:50.966 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-02T13:39:50.967 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T13:39:50.995 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-02T13:39:50.995 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-02T13:39:51.012 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-02T13:39:51.012 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-02T13:39:51.021 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-02T13:39:51.038 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-02T13:39:51.040 INFO:teuthology.orchestra.run.smithi088.stderr:sudo: ntpd: command not found 2024-04-02T13:39:51.043 INFO:teuthology.orchestra.run.smithi052.stderr:sudo: ntpd: command not found 2024-04-02T13:39:51.054 INFO:teuthology.orchestra.run.smithi088.stdout:506 Cannot talk to daemon 2024-04-02T13:39:51.058 INFO:teuthology.orchestra.run.smithi052.stdout:506 Cannot talk to daemon 2024-04-02T13:39:51.067 INFO:teuthology.orchestra.run.smithi189.stderr:sudo: ntpd: command not found 2024-04-02T13:39:51.070 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-02T13:39:51.074 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-02T13:39:51.081 INFO:teuthology.orchestra.run.smithi189.stdout:506 Cannot talk to daemon 2024-04-02T13:39:51.086 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-02T13:39:51.091 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-02T13:39:51.097 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-02T13:39:51.114 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-02T13:39:51.130 INFO:teuthology.orchestra.run.smithi088.stderr:bash: line 1: ntpq: command not found 2024-04-02T13:39:51.134 INFO:teuthology.orchestra.run.smithi088.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T13:39:51.134 INFO:teuthology.orchestra.run.smithi088.stdout:=============================================================================== 2024-04-02T13:39:51.134 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:39:51.134 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:39:51.134 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:39:51.134 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:39:51.152 INFO:teuthology.orchestra.run.smithi052.stderr:bash: line 1: ntpq: command not found 2024-04-02T13:39:51.156 INFO:teuthology.orchestra.run.smithi052.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T13:39:51.156 INFO:teuthology.orchestra.run.smithi052.stdout:=============================================================================== 2024-04-02T13:39:51.156 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:39:51.156 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:39:51.156 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:39:51.156 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:39:51.158 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-04-02T13:39:51.162 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T13:39:51.162 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-04-02T13:39:51.162 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:39:51.162 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:39:51.162 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:39:51.162 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:39:51.163 INFO:teuthology.run_tasks:Running task install... 2024-04-02T13:39:51.171 DEBUG:teuthology.task.install:project ceph 2024-04-02T13:39:51.171 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6'}} 2024-04-02T13:39:51.171 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6'} 2024-04-02T13:39:51.171 INFO:teuthology.task.install:Using flavor: default 2024-04-02T13:39:51.177 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-02T13:39:51.177 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-02T13:39:51.177 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'tag': None, 'wait_for_package': False} 2024-04-02T13:39:51.178 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:39:51.180 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'tag': None, 'wait_for_package': False} 2024-04-02T13:39:51.180 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:39:51.181 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'tag': None, 'wait_for_package': False} 2024-04-02T13:39:51.181 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:39:51.327 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/ 2024-04-02T13:39:51.327 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2609.g191d963a 2024-04-02T13:39:51.331 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/ 2024-04-02T13:39:51.331 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2609.g191d963a 2024-04-02T13:39:51.339 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/ 2024-04-02T13:39:51.339 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2609.g191d963a 2024-04-02T13:39:51.459 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-02T13:39:51.460 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-02T13:39:51.460 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-02T13:39:51.462 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-02T13:39:51.463 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-04-02T13:39:51.463 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-02T13:39:51.489 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-02T13:39:51.490 DEBUG:teuthology.orchestra.run.smithi088:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/191d963ac9de7322da8aff20b567b3fa458e2ef6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-02T13:39:51.491 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-02T13:39:51.491 DEBUG:teuthology.orchestra.run.smithi052:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/191d963ac9de7322da8aff20b567b3fa458e2ef6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-02T13:39:51.511 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-02T13:39:51.511 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-02T13:39:51.511 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-02T13:39:51.539 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-02T13:39:51.539 DEBUG:teuthology.orchestra.run.smithi189:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/191d963ac9de7322da8aff20b567b3fa458e2ef6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-02T13:39:51.556 DEBUG:teuthology.orchestra.run.smithi088:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-02T13:39:51.558 DEBUG:teuthology.orchestra.run.smithi052:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-02T13:39:51.606 DEBUG:teuthology.orchestra.run.smithi189:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-02T13:39:51.636 DEBUG:teuthology.orchestra.run.smithi088:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-02T13:39:51.638 DEBUG:teuthology.orchestra.run.smithi052:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-02T13:39:51.686 DEBUG:teuthology.orchestra.run.smithi189:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-02T13:39:51.702 INFO:teuthology.orchestra.run.smithi088.stdout:check_obsoletes = 1 2024-04-02T13:39:51.704 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum clean all 2024-04-02T13:39:51.704 INFO:teuthology.orchestra.run.smithi052.stdout:check_obsoletes = 1 2024-04-02T13:39:51.706 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum clean all 2024-04-02T13:39:51.753 INFO:teuthology.orchestra.run.smithi189.stdout:check_obsoletes = 1 2024-04-02T13:39:51.754 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum clean all 2024-04-02T13:39:51.941 INFO:teuthology.orchestra.run.smithi088.stdout:74 files removed 2024-04-02T13:39:51.959 INFO:teuthology.orchestra.run.smithi052.stdout:74 files removed 2024-04-02T13:39:51.968 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-02T13:39:51.986 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-02T13:39:51.990 INFO:teuthology.orchestra.run.smithi189.stdout:74 files removed 2024-04-02T13:39:52.017 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-02T13:39:52.768 INFO:teuthology.orchestra.run.smithi088.stdout:ceph packages for x86_64 163 kB/s | 83 kB 00:00 2024-04-02T13:39:52.828 INFO:teuthology.orchestra.run.smithi189.stdout:ceph packages for x86_64 159 kB/s | 83 kB 00:00 2024-04-02T13:39:52.830 INFO:teuthology.orchestra.run.smithi052.stdout:ceph packages for x86_64 155 kB/s | 83 kB 00:00 2024-04-02T13:39:53.218 INFO:teuthology.orchestra.run.smithi088.stdout:ceph noarch packages 28 kB/s | 12 kB 00:00 2024-04-02T13:39:53.331 INFO:teuthology.orchestra.run.smithi052.stdout:ceph noarch packages 26 kB/s | 12 kB 00:00 2024-04-02T13:39:53.346 INFO:teuthology.orchestra.run.smithi189.stdout:ceph noarch packages 24 kB/s | 12 kB 00:00 2024-04-02T13:39:53.677 INFO:teuthology.orchestra.run.smithi088.stdout:ceph source packages 4.4 kB/s | 1.9 kB 00:00 2024-04-02T13:39:53.856 INFO:teuthology.orchestra.run.smithi052.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-04-02T13:39:53.946 INFO:teuthology.orchestra.run.smithi189.stdout:ceph source packages 3.3 kB/s | 1.9 kB 00:00 2024-04-02T13:39:54.072 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-04-02T13:39:54.375 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-04-02T13:39:54.482 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-04-02T13:39:55.530 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS-9 - BaseOS 5.8 MB/s | 2.0 MB 00:00 2024-04-02T13:39:55.842 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS-9 - BaseOS 6.9 MB/s | 2.0 MB 00:00 2024-04-02T13:39:55.857 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS-9 - BaseOS 7.3 MB/s | 2.0 MB 00:00 2024-04-02T13:39:56.575 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - BaseOS 21 MB/s | 8.0 MB 00:00 2024-04-02T13:39:56.707 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-02T13:39:57.279 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - BaseOS 5.6 MB/s | 8.0 MB 00:01 2024-04-02T13:39:58.576 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-02T13:39:58.966 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-04-02T13:39:59.280 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-02T13:40:03.309 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - CRB 7.8 MB/s | 6.1 MB 00:00 2024-04-02T13:40:03.989 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - CRB 5.0 MB/s | 6.1 MB 00:01 2024-04-02T13:40:04.801 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 16 kB 00:00 2024-04-02T13:40:06.036 INFO:teuthology.orchestra.run.smithi189.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-04-02T13:40:06.314 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-04-02T13:40:06.718 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - CRB 1.4 MB/s | 6.1 MB 00:04 2024-04-02T13:40:07.427 INFO:teuthology.orchestra.run.smithi088.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-04-02T13:40:09.077 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-04-02T13:40:10.969 INFO:teuthology.orchestra.run.smithi189.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-04-02T13:40:11.987 INFO:teuthology.orchestra.run.smithi052.stdout:Extra Packages for Enterprise Linux 7.5 MB/s | 21 MB 00:02 2024-04-02T13:40:12.341 INFO:teuthology.orchestra.run.smithi088.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-02T13:40:13.469 INFO:teuthology.orchestra.run.smithi189.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T13:40:13.470 INFO:teuthology.orchestra.run.smithi189.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T13:40:13.484 INFO:teuthology.orchestra.run.smithi189.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-02T13:40:13.485 INFO:teuthology.orchestra.run.smithi189.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-02T13:40:13.486 INFO:teuthology.orchestra.run.smithi189.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-02T13:40:13.487 INFO:teuthology.orchestra.run.smithi189.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-02T13:40:13.489 INFO:teuthology.orchestra.run.smithi189.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-02T13:40:13.492 INFO:teuthology.orchestra.run.smithi189.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-02T13:40:13.493 INFO:teuthology.orchestra.run.smithi189.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-02T13:40:13.573 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-04-02T13:40:13.585 INFO:teuthology.orchestra.run.smithi189.stdout:============================================================================================ 2024-04-02T13:40:13.585 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-04-02T13:40:13.585 INFO:teuthology.orchestra.run.smithi189.stdout:============================================================================================ 2024-04-02T13:40:13.585 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-04-02T13:40:13.585 INFO:teuthology.orchestra.run.smithi189.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-02T13:40:13.585 INFO:teuthology.orchestra.run.smithi189.stdout: ceph x86_64 2:19.0.0-2609.g191d963a.el9 ceph 6.6 k 2024-04-02T13:40:13.585 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.4 M 2024-04-02T13:40:13.585 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 890 k 2024-04-02T13:40:13.585 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2609.g191d963a.el9 ceph 145 k 2024-04-02T13:40:13.585 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr x86_64 2:19.0.0-2609.g191d963a.el9 ceph 1.5 M 2024-04-02T13:40:13.585 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 141 k 2024-04-02T13:40:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 3.5 M 2024-04-02T13:40:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 7.4 M 2024-04-02T13:40:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 49 k 2024-04-02T13:40:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 9.8 M 2024-04-02T13:40:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test x86_64 2:19.0.0-2609.g191d963a.el9 ceph 48 M 2024-04-02T13:40:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 264 k 2024-04-02T13:40:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 762 k 2024-04-02T13:40:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 225 k 2024-04-02T13:40:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-shell x86_64 2:19.0.0-2609.g191d963a.el9 ceph 27 k 2024-04-02T13:40:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-top noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 18 k 2024-04-02T13:40:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-02T13:40:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-02T13:40:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-02T13:40:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 32 k 2024-04-02T13:40:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 747 k 2024-04-02T13:40:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 127 k 2024-04-02T13:40:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-02T13:40:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-02T13:40:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-02T13:40:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-02T13:40:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-02T13:40:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-02T13:40:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs x86_64 2:19.0.0-2609.g191d963a.el9 ceph 161 k 2024-04-02T13:40:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados x86_64 2:19.0.0-2609.g191d963a.el9 ceph 320 k 2024-04-02T13:40:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 299 k 2024-04-02T13:40:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 99 k 2024-04-02T13:40:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 85 k 2024-04-02T13:40:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T13:40:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 170 k 2024-04-02T13:40:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-02T13:40:13.588 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrading: 2024-04-02T13:40:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-04-02T13:40:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-04-02T13:40:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-04-02T13:40:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-04-02T13:40:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-02T13:40:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-02T13:40:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-02T13:40:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-02T13:40:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-02T13:40:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-02T13:40:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-02T13:40:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-02T13:40:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-02T13:40:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: librados2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.4 M 2024-04-02T13:40:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T13:40:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-02T13:40:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-02T13:40:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-02T13:40:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-02T13:40:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-02T13:40:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-02T13:40:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-02T13:40:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-02T13:40:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-02T13:40:13.590 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-04-02T13:40:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-02T13:40:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-02T13:40:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-02T13:40:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 21 M 2024-04-02T13:40:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 34 k 2024-04-02T13:40:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds x86_64 2:19.0.0-2609.g191d963a.el9 ceph 2.4 M 2024-04-02T13:40:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 249 k 2024-04-02T13:40:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon x86_64 2:19.0.0-2609.g191d963a.el9 ceph 4.7 M 2024-04-02T13:40:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 17 M 2024-04-02T13:40:13.591 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 16 k 2024-04-02T13:40:13.591 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux x86_64 2:19.0.0-2609.g191d963a.el9 ceph 25 k 2024-04-02T13:40:13.591 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-02T13:40:13.591 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-02T13:40:13.591 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-02T13:40:13.591 INFO:teuthology.orchestra.run.smithi189.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-02T13:40:13.591 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-02T13:40:13.591 INFO:teuthology.orchestra.run.smithi189.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-02T13:40:13.591 INFO:teuthology.orchestra.run.smithi189.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-02T13:40:13.591 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-02T13:40:13.591 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-02T13:40:13.591 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite x86_64 2:19.0.0-2609.g191d963a.el9 ceph 164 k 2024-04-02T13:40:13.591 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-02T13:40:13.592 INFO:teuthology.orchestra.run.smithi189.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-02T13:40:13.592 INFO:teuthology.orchestra.run.smithi189.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-02T13:40:13.592 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-02T13:40:13.592 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-02T13:40:13.592 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 500 k 2024-04-02T13:40:13.592 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-02T13:40:13.592 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.1 M 2024-04-02T13:40:13.592 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-02T13:40:13.592 INFO:teuthology.orchestra.run.smithi189.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-02T13:40:13.592 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-02T13:40:13.592 INFO:teuthology.orchestra.run.smithi189.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-02T13:40:13.592 INFO:teuthology.orchestra.run.smithi189.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-02T13:40:13.592 INFO:teuthology.orchestra.run.smithi189.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-02T13:40:13.593 INFO:teuthology.orchestra.run.smithi189.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-02T13:40:13.593 INFO:teuthology.orchestra.run.smithi189.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-02T13:40:13.593 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-02T13:40:13.593 INFO:teuthology.orchestra.run.smithi189.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-02T13:40:13.593 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-02T13:40:13.593 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-02T13:40:13.593 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-02T13:40:13.593 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-02T13:40:13.593 INFO:teuthology.orchestra.run.smithi189.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-02T13:40:13.593 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-02T13:40:13.593 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-02T13:40:13.593 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-02T13:40:13.593 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 45 k 2024-04-02T13:40:13.593 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 134 k 2024-04-02T13:40:13.594 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-02T13:40:13.594 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-02T13:40:13.594 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-02T13:40:13.594 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-02T13:40:13.594 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-02T13:40:13.594 INFO:teuthology.orchestra.run.smithi189.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-02T13:40:13.594 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-02T13:40:13.594 INFO:teuthology.orchestra.run.smithi189.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-02T13:40:13.594 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-02T13:40:13.594 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-02T13:40:13.594 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-02T13:40:13.594 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-02T13:40:13.594 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-02T13:40:13.595 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-02T13:40:13.595 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-02T13:40:13.595 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-02T13:40:13.595 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-02T13:40:13.595 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-02T13:40:13.595 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-02T13:40:13.595 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-02T13:40:13.595 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-02T13:40:13.595 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-02T13:40:13.595 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-02T13:40:13.595 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-02T13:40:13.595 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-02T13:40:13.595 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-02T13:40:13.595 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-02T13:40:13.596 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-02T13:40:13.596 INFO:teuthology.orchestra.run.smithi189.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-02T13:40:13.596 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-02T13:40:13.596 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-02T13:40:13.596 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-02T13:40:13.596 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-02T13:40:13.596 INFO:teuthology.orchestra.run.smithi189.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-02T13:40:13.596 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-02T13:40:13.596 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-02T13:40:13.596 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-02T13:40:13.596 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-02T13:40:13.596 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-02T13:40:13.596 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-02T13:40:13.596 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-02T13:40:13.596 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-02T13:40:13.597 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-02T13:40:13.597 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-02T13:40:13.597 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-02T13:40:13.597 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-02T13:40:13.597 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-02T13:40:13.597 INFO:teuthology.orchestra.run.smithi189.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-02T13:40:13.597 INFO:teuthology.orchestra.run.smithi189.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-02T13:40:13.597 INFO:teuthology.orchestra.run.smithi189.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-02T13:40:13.597 INFO:teuthology.orchestra.run.smithi189.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-02T13:40:13.597 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-02T13:40:13.597 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-02T13:40:13.597 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-02T13:40:13.597 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-02T13:40:13.597 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-02T13:40:13.597 INFO:teuthology.orchestra.run.smithi189.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-02T13:40:13.598 INFO:teuthology.orchestra.run.smithi189.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-02T13:40:13.598 INFO:teuthology.orchestra.run.smithi189.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-02T13:40:13.598 INFO:teuthology.orchestra.run.smithi189.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-02T13:40:13.598 INFO:teuthology.orchestra.run.smithi189.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-02T13:40:13.598 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-02T13:40:13.598 INFO:teuthology.orchestra.run.smithi189.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-02T13:40:13.598 INFO:teuthology.orchestra.run.smithi189.stdout:Installing weak dependencies: 2024-04-02T13:40:13.598 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-02T13:40:13.598 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-02T13:40:13.598 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-04-02T13:40:13.598 INFO:teuthology.orchestra.run.smithi189.stdout:============================================================================================ 2024-04-02T13:40:13.598 INFO:teuthology.orchestra.run.smithi189.stdout:Install 150 Packages 2024-04-02T13:40:13.598 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrade 24 Packages 2024-04-02T13:40:13.598 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-02T13:40:13.599 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 226 M 2024-04-02T13:40:13.599 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-04-02T13:40:14.054 INFO:teuthology.orchestra.run.smithi189.stdout:(1/174): ceph-19.0.0-2609.g191d963a.el9.x86_64. 41 kB/s | 6.6 kB 00:00 2024-04-02T13:40:14.329 INFO:teuthology.orchestra.run.smithi189.stdout:(2/174): ceph-fuse-19.0.0-2609.g191d963a.el9.x8 3.2 MB/s | 890 kB 00:00 2024-04-02T13:40:14.421 INFO:teuthology.orchestra.run.smithi189.stdout:(3/174): ceph-immutable-object-cache-19.0.0-260 1.5 MB/s | 145 kB 00:00 2024-04-02T13:40:14.505 INFO:teuthology.orchestra.run.smithi189.stdout:(4/174): ceph-base-19.0.0-2609.g191d963a.el9.x8 8.8 MB/s | 5.4 MB 00:00 2024-04-02T13:40:14.697 INFO:teuthology.orchestra.run.smithi189.stdout:(5/174): ceph-mgr-19.0.0-2609.g191d963a.el9.x86 7.8 MB/s | 1.5 MB 00:00 2024-04-02T13:40:14.897 INFO:teuthology.orchestra.run.smithi189.stdout:(6/174): ceph-mds-19.0.0-2609.g191d963a.el9.x86 5.0 MB/s | 2.4 MB 00:00 2024-04-02T13:40:15.047 INFO:teuthology.orchestra.run.smithi189.stdout:(7/174): ceph-mon-19.0.0-2609.g191d963a.el9.x86 13 MB/s | 4.7 MB 00:00 2024-04-02T13:40:15.166 INFO:teuthology.orchestra.run.smithi088.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T13:40:15.167 INFO:teuthology.orchestra.run.smithi088.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T13:40:15.181 INFO:teuthology.orchestra.run.smithi088.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-02T13:40:15.182 INFO:teuthology.orchestra.run.smithi088.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-02T13:40:15.183 INFO:teuthology.orchestra.run.smithi088.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-02T13:40:15.184 INFO:teuthology.orchestra.run.smithi088.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-02T13:40:15.186 INFO:teuthology.orchestra.run.smithi088.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-02T13:40:15.189 INFO:teuthology.orchestra.run.smithi088.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-02T13:40:15.190 INFO:teuthology.orchestra.run.smithi088.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-02T13:40:15.270 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-04-02T13:40:15.296 INFO:teuthology.orchestra.run.smithi088.stdout:============================================================================================ 2024-04-02T13:40:15.297 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2024-04-02T13:40:15.297 INFO:teuthology.orchestra.run.smithi088.stdout:============================================================================================ 2024-04-02T13:40:15.297 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-04-02T13:40:15.297 INFO:teuthology.orchestra.run.smithi088.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-02T13:40:15.297 INFO:teuthology.orchestra.run.smithi088.stdout: ceph x86_64 2:19.0.0-2609.g191d963a.el9 ceph 6.6 k 2024-04-02T13:40:15.297 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-base x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.4 M 2024-04-02T13:40:15.297 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 890 k 2024-04-02T13:40:15.297 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2609.g191d963a.el9 ceph 145 k 2024-04-02T13:40:15.297 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr x86_64 2:19.0.0-2609.g191d963a.el9 ceph 1.5 M 2024-04-02T13:40:15.297 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 141 k 2024-04-02T13:40:15.297 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 3.5 M 2024-04-02T13:40:15.298 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 7.4 M 2024-04-02T13:40:15.298 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-rook noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 49 k 2024-04-02T13:40:15.298 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-radosgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 9.8 M 2024-04-02T13:40:15.298 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-test x86_64 2:19.0.0-2609.g191d963a.el9 ceph 48 M 2024-04-02T13:40:15.298 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-volume noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 264 k 2024-04-02T13:40:15.298 INFO:teuthology.orchestra.run.smithi088.stdout: cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 762 k 2024-04-02T13:40:15.298 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 225 k 2024-04-02T13:40:15.298 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-shell x86_64 2:19.0.0-2609.g191d963a.el9 ceph 27 k 2024-04-02T13:40:15.298 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-top noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 18 k 2024-04-02T13:40:15.298 INFO:teuthology.orchestra.run.smithi088.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-02T13:40:15.298 INFO:teuthology.orchestra.run.smithi088.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-02T13:40:15.299 INFO:teuthology.orchestra.run.smithi088.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-02T13:40:15.299 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 32 k 2024-04-02T13:40:15.299 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 747 k 2024-04-02T13:40:15.299 INFO:teuthology.orchestra.run.smithi088.stdout: librados-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 127 k 2024-04-02T13:40:15.299 INFO:teuthology.orchestra.run.smithi088.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-02T13:40:15.299 INFO:teuthology.orchestra.run.smithi088.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-02T13:40:15.299 INFO:teuthology.orchestra.run.smithi088.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-02T13:40:15.299 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-02T13:40:15.299 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-02T13:40:15.299 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-02T13:40:15.299 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cephfs x86_64 2:19.0.0-2609.g191d963a.el9 ceph 161 k 2024-04-02T13:40:15.299 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rados x86_64 2:19.0.0-2609.g191d963a.el9 ceph 320 k 2024-04-02T13:40:15.299 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 299 k 2024-04-02T13:40:15.300 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 99 k 2024-04-02T13:40:15.300 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 85 k 2024-04-02T13:40:15.300 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T13:40:15.300 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-nbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 170 k 2024-04-02T13:40:15.300 INFO:teuthology.orchestra.run.smithi088.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-02T13:40:15.300 INFO:teuthology.orchestra.run.smithi088.stdout:Upgrading: 2024-04-02T13:40:15.300 INFO:teuthology.orchestra.run.smithi088.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-04-02T13:40:15.300 INFO:teuthology.orchestra.run.smithi088.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-04-02T13:40:15.300 INFO:teuthology.orchestra.run.smithi088.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-04-02T13:40:15.300 INFO:teuthology.orchestra.run.smithi088.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-04-02T13:40:15.300 INFO:teuthology.orchestra.run.smithi088.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-02T13:40:15.300 INFO:teuthology.orchestra.run.smithi088.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-02T13:40:15.300 INFO:teuthology.orchestra.run.smithi088.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-02T13:40:15.300 INFO:teuthology.orchestra.run.smithi088.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-02T13:40:15.301 INFO:teuthology.orchestra.run.smithi088.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-02T13:40:15.301 INFO:teuthology.orchestra.run.smithi088.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-02T13:40:15.301 INFO:teuthology.orchestra.run.smithi088.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-02T13:40:15.301 INFO:teuthology.orchestra.run.smithi088.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-02T13:40:15.301 INFO:teuthology.orchestra.run.smithi088.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-02T13:40:15.301 INFO:teuthology.orchestra.run.smithi088.stdout: librados2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.4 M 2024-04-02T13:40:15.301 INFO:teuthology.orchestra.run.smithi088.stdout: librbd1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T13:40:15.301 INFO:teuthology.orchestra.run.smithi088.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-02T13:40:15.301 INFO:teuthology.orchestra.run.smithi088.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-02T13:40:15.301 INFO:teuthology.orchestra.run.smithi088.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-02T13:40:15.301 INFO:teuthology.orchestra.run.smithi088.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-02T13:40:15.301 INFO:teuthology.orchestra.run.smithi088.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-02T13:40:15.302 INFO:teuthology.orchestra.run.smithi088.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-02T13:40:15.302 INFO:teuthology.orchestra.run.smithi088.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-02T13:40:15.302 INFO:teuthology.orchestra.run.smithi088.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-02T13:40:15.302 INFO:teuthology.orchestra.run.smithi088.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-02T13:40:15.302 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2024-04-02T13:40:15.302 INFO:teuthology.orchestra.run.smithi088.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-02T13:40:15.302 INFO:teuthology.orchestra.run.smithi088.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-02T13:40:15.302 INFO:teuthology.orchestra.run.smithi088.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-02T13:40:15.302 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 21 M 2024-04-02T13:40:15.302 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 34 k 2024-04-02T13:40:15.302 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mds x86_64 2:19.0.0-2609.g191d963a.el9 ceph 2.4 M 2024-04-02T13:40:15.302 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 249 k 2024-04-02T13:40:15.302 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mon x86_64 2:19.0.0-2609.g191d963a.el9 ceph 4.7 M 2024-04-02T13:40:15.303 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-osd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 17 M 2024-04-02T13:40:15.303 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 16 k 2024-04-02T13:40:15.303 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-selinux x86_64 2:19.0.0-2609.g191d963a.el9 ceph 25 k 2024-04-02T13:40:15.303 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-02T13:40:15.303 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-02T13:40:15.303 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-02T13:40:15.303 INFO:teuthology.orchestra.run.smithi088.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-02T13:40:15.303 INFO:teuthology.orchestra.run.smithi088.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-02T13:40:15.303 INFO:teuthology.orchestra.run.smithi088.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-02T13:40:15.303 INFO:teuthology.orchestra.run.smithi088.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-02T13:40:15.303 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-02T13:40:15.303 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-02T13:40:15.303 INFO:teuthology.orchestra.run.smithi088.stdout: libcephsqlite x86_64 2:19.0.0-2609.g191d963a.el9 ceph 164 k 2024-04-02T13:40:15.303 INFO:teuthology.orchestra.run.smithi088.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-02T13:40:15.304 INFO:teuthology.orchestra.run.smithi088.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-02T13:40:15.304 INFO:teuthology.orchestra.run.smithi088.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-02T13:40:15.304 INFO:teuthology.orchestra.run.smithi088.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-02T13:40:15.304 INFO:teuthology.orchestra.run.smithi088.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-02T13:40:15.304 INFO:teuthology.orchestra.run.smithi088.stdout: libradosstriper1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 500 k 2024-04-02T13:40:15.304 INFO:teuthology.orchestra.run.smithi088.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-02T13:40:15.304 INFO:teuthology.orchestra.run.smithi088.stdout: librgw2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.1 M 2024-04-02T13:40:15.304 INFO:teuthology.orchestra.run.smithi088.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-02T13:40:15.304 INFO:teuthology.orchestra.run.smithi088.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-02T13:40:15.304 INFO:teuthology.orchestra.run.smithi088.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-02T13:40:15.304 INFO:teuthology.orchestra.run.smithi088.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-02T13:40:15.304 INFO:teuthology.orchestra.run.smithi088.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-02T13:40:15.304 INFO:teuthology.orchestra.run.smithi088.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-02T13:40:15.305 INFO:teuthology.orchestra.run.smithi088.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-02T13:40:15.305 INFO:teuthology.orchestra.run.smithi088.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-02T13:40:15.305 INFO:teuthology.orchestra.run.smithi088.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-02T13:40:15.305 INFO:teuthology.orchestra.run.smithi088.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-02T13:40:15.305 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-02T13:40:15.305 INFO:teuthology.orchestra.run.smithi088.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-02T13:40:15.305 INFO:teuthology.orchestra.run.smithi088.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-02T13:40:15.305 INFO:teuthology.orchestra.run.smithi088.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-02T13:40:15.305 INFO:teuthology.orchestra.run.smithi088.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-02T13:40:15.305 INFO:teuthology.orchestra.run.smithi088.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-02T13:40:15.305 INFO:teuthology.orchestra.run.smithi088.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-02T13:40:15.305 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-02T13:40:15.305 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-argparse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 45 k 2024-04-02T13:40:15.305 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 134 k 2024-04-02T13:40:15.306 INFO:teuthology.orchestra.run.smithi088.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-02T13:40:15.306 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-02T13:40:15.306 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-02T13:40:15.306 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-02T13:40:15.306 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-02T13:40:15.306 INFO:teuthology.orchestra.run.smithi088.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-02T13:40:15.306 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-02T13:40:15.306 INFO:teuthology.orchestra.run.smithi088.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-02T13:40:15.306 INFO:teuthology.orchestra.run.smithi088.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-02T13:40:15.306 INFO:teuthology.orchestra.run.smithi088.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-02T13:40:15.306 INFO:teuthology.orchestra.run.smithi088.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-02T13:40:15.306 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-02T13:40:15.306 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-02T13:40:15.307 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-02T13:40:15.307 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-02T13:40:15.307 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-02T13:40:15.307 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-02T13:40:15.307 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-02T13:40:15.307 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-02T13:40:15.307 INFO:teuthology.orchestra.run.smithi088.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-02T13:40:15.307 INFO:teuthology.orchestra.run.smithi088.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-02T13:40:15.307 INFO:teuthology.orchestra.run.smithi088.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-02T13:40:15.307 INFO:teuthology.orchestra.run.smithi088.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-02T13:40:15.307 INFO:teuthology.orchestra.run.smithi088.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-02T13:40:15.307 INFO:teuthology.orchestra.run.smithi088.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-02T13:40:15.307 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-02T13:40:15.308 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-02T13:40:15.308 INFO:teuthology.orchestra.run.smithi088.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-02T13:40:15.308 INFO:teuthology.orchestra.run.smithi088.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-02T13:40:15.308 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-02T13:40:15.308 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-02T13:40:15.308 INFO:teuthology.orchestra.run.smithi088.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-02T13:40:15.308 INFO:teuthology.orchestra.run.smithi088.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-02T13:40:15.308 INFO:teuthology.orchestra.run.smithi088.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-02T13:40:15.308 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-02T13:40:15.308 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-02T13:40:15.308 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-02T13:40:15.308 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-02T13:40:15.308 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-02T13:40:15.308 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-02T13:40:15.309 INFO:teuthology.orchestra.run.smithi088.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-02T13:40:15.309 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-02T13:40:15.309 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-02T13:40:15.309 INFO:teuthology.orchestra.run.smithi088.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-02T13:40:15.309 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-02T13:40:15.309 INFO:teuthology.orchestra.run.smithi088.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-02T13:40:15.309 INFO:teuthology.orchestra.run.smithi088.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-02T13:40:15.309 INFO:teuthology.orchestra.run.smithi088.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-02T13:40:15.309 INFO:teuthology.orchestra.run.smithi088.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-02T13:40:15.309 INFO:teuthology.orchestra.run.smithi088.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-02T13:40:15.309 INFO:teuthology.orchestra.run.smithi088.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-02T13:40:15.309 INFO:teuthology.orchestra.run.smithi088.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-02T13:40:15.309 INFO:teuthology.orchestra.run.smithi088.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-02T13:40:15.309 INFO:teuthology.orchestra.run.smithi088.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-02T13:40:15.310 INFO:teuthology.orchestra.run.smithi088.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-02T13:40:15.310 INFO:teuthology.orchestra.run.smithi088.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-02T13:40:15.310 INFO:teuthology.orchestra.run.smithi088.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-02T13:40:15.310 INFO:teuthology.orchestra.run.smithi088.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-02T13:40:15.310 INFO:teuthology.orchestra.run.smithi088.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-02T13:40:15.310 INFO:teuthology.orchestra.run.smithi088.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-02T13:40:15.310 INFO:teuthology.orchestra.run.smithi088.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-02T13:40:15.310 INFO:teuthology.orchestra.run.smithi088.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-02T13:40:15.310 INFO:teuthology.orchestra.run.smithi088.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-02T13:40:15.310 INFO:teuthology.orchestra.run.smithi088.stdout:Installing weak dependencies: 2024-04-02T13:40:15.310 INFO:teuthology.orchestra.run.smithi088.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-02T13:40:15.310 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-02T13:40:15.310 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-04-02T13:40:15.311 INFO:teuthology.orchestra.run.smithi088.stdout:============================================================================================ 2024-04-02T13:40:15.311 INFO:teuthology.orchestra.run.smithi088.stdout:Install 150 Packages 2024-04-02T13:40:15.311 INFO:teuthology.orchestra.run.smithi088.stdout:Upgrade 24 Packages 2024-04-02T13:40:15.311 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-02T13:40:15.311 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 226 M 2024-04-02T13:40:15.311 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-04-02T13:40:15.781 INFO:teuthology.orchestra.run.smithi189.stdout:(8/174): ceph-radosgw-19.0.0-2609.g191d963a.el9 13 MB/s | 9.8 MB 00:00 2024-04-02T13:40:15.831 INFO:teuthology.orchestra.run.smithi189.stdout:(9/174): ceph-selinux-19.0.0-2609.g191d963a.el9 504 kB/s | 25 kB 00:00 2024-04-02T13:40:16.284 INFO:teuthology.orchestra.run.smithi088.stdout:(1/174): ceph-19.0.0-2609.g191d963a.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-04-02T13:40:16.606 INFO:teuthology.orchestra.run.smithi189.stdout:(10/174): ceph-common-19.0.0-2609.g191d963a.el9 7.6 MB/s | 21 MB 00:02 2024-04-02T13:40:16.706 INFO:teuthology.orchestra.run.smithi189.stdout:(11/174): cephfs-mirror-19.0.0-2609.g191d963a.e 2.2 MB/s | 225 kB 00:00 2024-04-02T13:40:16.767 INFO:teuthology.orchestra.run.smithi088.stdout:(2/174): ceph-fuse-19.0.0-2609.g191d963a.el9.x8 1.8 MB/s | 890 kB 00:00 2024-04-02T13:40:16.790 INFO:teuthology.orchestra.run.smithi189.stdout:(12/174): cephfs-shell-19.0.0-2609.g191d963a.el 325 kB/s | 27 kB 00:00 2024-04-02T13:40:16.850 INFO:teuthology.orchestra.run.smithi088.stdout:(3/174): ceph-immutable-object-cache-19.0.0-260 1.7 MB/s | 145 kB 00:00 2024-04-02T13:40:16.873 INFO:teuthology.orchestra.run.smithi189.stdout:(13/174): libcephfs-devel-19.0.0-2609.g191d963a 381 kB/s | 32 kB 00:00 2024-04-02T13:40:17.057 INFO:teuthology.orchestra.run.smithi189.stdout:(14/174): libcephfs2-19.0.0-2609.g191d963a.el9. 4.0 MB/s | 747 kB 00:00 2024-04-02T13:40:17.113 INFO:teuthology.orchestra.run.smithi052.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-02T13:40:17.158 INFO:teuthology.orchestra.run.smithi189.stdout:(15/174): libcephsqlite-19.0.0-2609.g191d963a.e 1.6 MB/s | 164 kB 00:00 2024-04-02T13:40:17.249 INFO:teuthology.orchestra.run.smithi189.stdout:(16/174): librados-devel-19.0.0-2609.g191d963a. 1.4 MB/s | 127 kB 00:00 2024-04-02T13:40:17.534 INFO:teuthology.orchestra.run.smithi088.stdout:(4/174): ceph-mds-19.0.0-2609.g191d963a.el9.x86 3.5 MB/s | 2.4 MB 00:00 2024-04-02T13:40:17.566 INFO:teuthology.orchestra.run.smithi189.stdout:(17/174): libradosstriper1-19.0.0-2609.g191d963 1.5 MB/s | 500 kB 00:00 2024-04-02T13:40:17.867 INFO:teuthology.orchestra.run.smithi088.stdout:(5/174): ceph-mgr-19.0.0-2609.g191d963a.el9.x86 4.5 MB/s | 1.5 MB 00:00 2024-04-02T13:40:18.608 INFO:teuthology.orchestra.run.smithi189.stdout:(18/174): librgw2-19.0.0-2609.g191d963a.el9.x86 4.9 MB/s | 5.1 MB 00:01 2024-04-02T13:40:18.658 INFO:teuthology.orchestra.run.smithi189.stdout:(19/174): python3-ceph-argparse-19.0.0-2609.g19 905 kB/s | 45 kB 00:00 2024-04-02T13:40:18.659 INFO:teuthology.orchestra.run.smithi088.stdout:(6/174): ceph-mon-19.0.0-2609.g191d963a.el9.x86 5.9 MB/s | 4.7 MB 00:00 2024-04-02T13:40:18.726 INFO:teuthology.orchestra.run.smithi189.stdout:(20/174): python3-ceph-common-19.0.0-2609.g191d 2.0 MB/s | 134 kB 00:00 2024-04-02T13:40:18.825 INFO:teuthology.orchestra.run.smithi189.stdout:(21/174): python3-cephfs-19.0.0-2609.g191d963a. 1.6 MB/s | 161 kB 00:00 2024-04-02T13:40:18.934 INFO:teuthology.orchestra.run.smithi189.stdout:(22/174): python3-rados-19.0.0-2609.g191d963a.e 2.9 MB/s | 320 kB 00:00 2024-04-02T13:40:19.043 INFO:teuthology.orchestra.run.smithi189.stdout:(23/174): python3-rbd-19.0.0-2609.g191d963a.el9 2.7 MB/s | 299 kB 00:00 2024-04-02T13:40:19.137 INFO:teuthology.orchestra.run.smithi189.stdout:(24/174): python3-rgw-19.0.0-2609.g191d963a.el9 1.1 MB/s | 99 kB 00:00 2024-04-02T13:40:19.235 INFO:teuthology.orchestra.run.smithi189.stdout:(25/174): rbd-fuse-19.0.0-2609.g191d963a.el9.x8 855 kB/s | 85 kB 00:00 2024-04-02T13:40:19.411 INFO:teuthology.orchestra.run.smithi189.stdout:(26/174): ceph-osd-19.0.0-2609.g191d963a.el9.x8 3.7 MB/s | 17 MB 00:04 2024-04-02T13:40:19.578 INFO:teuthology.orchestra.run.smithi189.stdout:(27/174): rbd-nbd-19.0.0-2609.g191d963a.el9.x86 1.0 MB/s | 170 kB 00:00 2024-04-02T13:40:19.669 INFO:teuthology.orchestra.run.smithi189.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2609.g 371 kB/s | 34 kB 00:00 2024-04-02T13:40:19.762 INFO:teuthology.orchestra.run.smithi189.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2609.g191d963 1.5 MB/s | 141 kB 00:00 2024-04-02T13:40:19.783 INFO:teuthology.orchestra.run.smithi052.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T13:40:19.784 INFO:teuthology.orchestra.run.smithi052.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T13:40:19.784 INFO:teuthology.orchestra.run.smithi088.stdout:(7/174): ceph-base-19.0.0-2609.g191d963a.el9.x8 1.4 MB/s | 5.4 MB 00:03 2024-04-02T13:40:19.796 INFO:teuthology.orchestra.run.smithi052.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-02T13:40:19.797 INFO:teuthology.orchestra.run.smithi052.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-02T13:40:19.798 INFO:teuthology.orchestra.run.smithi052.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-02T13:40:19.799 INFO:teuthology.orchestra.run.smithi052.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-02T13:40:19.801 INFO:teuthology.orchestra.run.smithi052.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-02T13:40:19.803 INFO:teuthology.orchestra.run.smithi052.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-02T13:40:19.804 INFO:teuthology.orchestra.run.smithi052.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-02T13:40:19.890 INFO:teuthology.orchestra.run.smithi052.stdout:Dependencies resolved. 2024-04-02T13:40:19.917 INFO:teuthology.orchestra.run.smithi052.stdout:============================================================================================ 2024-04-02T13:40:19.917 INFO:teuthology.orchestra.run.smithi052.stdout: Package Arch Version Repository Size 2024-04-02T13:40:19.918 INFO:teuthology.orchestra.run.smithi052.stdout:============================================================================================ 2024-04-02T13:40:19.918 INFO:teuthology.orchestra.run.smithi052.stdout:Installing: 2024-04-02T13:40:19.918 INFO:teuthology.orchestra.run.smithi052.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-02T13:40:19.918 INFO:teuthology.orchestra.run.smithi052.stdout: ceph x86_64 2:19.0.0-2609.g191d963a.el9 ceph 6.6 k 2024-04-02T13:40:19.918 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-base x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.4 M 2024-04-02T13:40:19.918 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 890 k 2024-04-02T13:40:19.918 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2609.g191d963a.el9 ceph 145 k 2024-04-02T13:40:19.918 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr x86_64 2:19.0.0-2609.g191d963a.el9 ceph 1.5 M 2024-04-02T13:40:19.919 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 141 k 2024-04-02T13:40:19.919 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 3.5 M 2024-04-02T13:40:19.919 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 7.4 M 2024-04-02T13:40:19.919 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-rook noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 49 k 2024-04-02T13:40:19.919 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-radosgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 9.8 M 2024-04-02T13:40:19.919 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-test x86_64 2:19.0.0-2609.g191d963a.el9 ceph 48 M 2024-04-02T13:40:19.919 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-volume noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 264 k 2024-04-02T13:40:19.919 INFO:teuthology.orchestra.run.smithi052.stdout: cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 762 k 2024-04-02T13:40:19.919 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 225 k 2024-04-02T13:40:19.919 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-shell x86_64 2:19.0.0-2609.g191d963a.el9 ceph 27 k 2024-04-02T13:40:19.919 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-top noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 18 k 2024-04-02T13:40:19.919 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-02T13:40:19.920 INFO:teuthology.orchestra.run.smithi052.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-02T13:40:19.920 INFO:teuthology.orchestra.run.smithi052.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-02T13:40:19.920 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 32 k 2024-04-02T13:40:19.920 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 747 k 2024-04-02T13:40:19.920 INFO:teuthology.orchestra.run.smithi052.stdout: librados-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 127 k 2024-04-02T13:40:19.920 INFO:teuthology.orchestra.run.smithi052.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-02T13:40:19.920 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-02T13:40:19.920 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-02T13:40:19.920 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-02T13:40:19.920 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-02T13:40:19.920 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-02T13:40:19.920 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cephfs x86_64 2:19.0.0-2609.g191d963a.el9 ceph 161 k 2024-04-02T13:40:19.920 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rados x86_64 2:19.0.0-2609.g191d963a.el9 ceph 320 k 2024-04-02T13:40:19.921 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 299 k 2024-04-02T13:40:19.921 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 99 k 2024-04-02T13:40:19.921 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 85 k 2024-04-02T13:40:19.921 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T13:40:19.921 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-nbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 170 k 2024-04-02T13:40:19.921 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-02T13:40:19.921 INFO:teuthology.orchestra.run.smithi052.stdout:Upgrading: 2024-04-02T13:40:19.921 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-04-02T13:40:19.921 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-04-02T13:40:19.921 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-04-02T13:40:19.921 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-04-02T13:40:19.921 INFO:teuthology.orchestra.run.smithi052.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-02T13:40:19.922 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-02T13:40:19.922 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-02T13:40:19.922 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-02T13:40:19.922 INFO:teuthology.orchestra.run.smithi052.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-02T13:40:19.922 INFO:teuthology.orchestra.run.smithi052.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-02T13:40:19.922 INFO:teuthology.orchestra.run.smithi052.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-02T13:40:19.922 INFO:teuthology.orchestra.run.smithi052.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-02T13:40:19.922 INFO:teuthology.orchestra.run.smithi052.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-02T13:40:19.922 INFO:teuthology.orchestra.run.smithi052.stdout: librados2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.4 M 2024-04-02T13:40:19.922 INFO:teuthology.orchestra.run.smithi052.stdout: librbd1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T13:40:19.922 INFO:teuthology.orchestra.run.smithi052.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-02T13:40:19.922 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-02T13:40:19.923 INFO:teuthology.orchestra.run.smithi052.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-02T13:40:19.923 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-02T13:40:19.923 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-02T13:40:19.923 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-02T13:40:19.923 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-02T13:40:19.923 INFO:teuthology.orchestra.run.smithi052.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-02T13:40:19.923 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-02T13:40:19.923 INFO:teuthology.orchestra.run.smithi052.stdout:Installing dependencies: 2024-04-02T13:40:19.923 INFO:teuthology.orchestra.run.smithi052.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-02T13:40:19.923 INFO:teuthology.orchestra.run.smithi052.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-02T13:40:19.923 INFO:teuthology.orchestra.run.smithi052.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-02T13:40:19.923 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 21 M 2024-04-02T13:40:19.923 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 34 k 2024-04-02T13:40:19.923 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mds x86_64 2:19.0.0-2609.g191d963a.el9 ceph 2.4 M 2024-04-02T13:40:19.924 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 249 k 2024-04-02T13:40:19.924 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mon x86_64 2:19.0.0-2609.g191d963a.el9 ceph 4.7 M 2024-04-02T13:40:19.924 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-osd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 17 M 2024-04-02T13:40:19.924 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 16 k 2024-04-02T13:40:19.924 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-selinux x86_64 2:19.0.0-2609.g191d963a.el9 ceph 25 k 2024-04-02T13:40:19.924 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-02T13:40:19.924 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-02T13:40:19.924 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-02T13:40:19.924 INFO:teuthology.orchestra.run.smithi052.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-02T13:40:19.924 INFO:teuthology.orchestra.run.smithi052.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-02T13:40:19.924 INFO:teuthology.orchestra.run.smithi052.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-02T13:40:19.924 INFO:teuthology.orchestra.run.smithi052.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-02T13:40:19.924 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-02T13:40:19.925 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-02T13:40:19.925 INFO:teuthology.orchestra.run.smithi052.stdout: libcephsqlite x86_64 2:19.0.0-2609.g191d963a.el9 ceph 164 k 2024-04-02T13:40:19.925 INFO:teuthology.orchestra.run.smithi052.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-02T13:40:19.925 INFO:teuthology.orchestra.run.smithi052.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-02T13:40:19.925 INFO:teuthology.orchestra.run.smithi052.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-02T13:40:19.925 INFO:teuthology.orchestra.run.smithi052.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-02T13:40:19.925 INFO:teuthology.orchestra.run.smithi052.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-02T13:40:19.925 INFO:teuthology.orchestra.run.smithi052.stdout: libradosstriper1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 500 k 2024-04-02T13:40:19.925 INFO:teuthology.orchestra.run.smithi052.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-02T13:40:19.925 INFO:teuthology.orchestra.run.smithi052.stdout: librgw2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.1 M 2024-04-02T13:40:19.925 INFO:teuthology.orchestra.run.smithi052.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-02T13:40:19.925 INFO:teuthology.orchestra.run.smithi052.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-02T13:40:19.926 INFO:teuthology.orchestra.run.smithi052.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-02T13:40:19.926 INFO:teuthology.orchestra.run.smithi052.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-02T13:40:19.926 INFO:teuthology.orchestra.run.smithi052.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-02T13:40:19.926 INFO:teuthology.orchestra.run.smithi052.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-02T13:40:19.926 INFO:teuthology.orchestra.run.smithi052.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-02T13:40:19.926 INFO:teuthology.orchestra.run.smithi052.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-02T13:40:19.926 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-02T13:40:19.926 INFO:teuthology.orchestra.run.smithi052.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-02T13:40:19.926 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-02T13:40:19.926 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-02T13:40:19.926 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-02T13:40:19.926 INFO:teuthology.orchestra.run.smithi052.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-02T13:40:19.926 INFO:teuthology.orchestra.run.smithi052.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-02T13:40:19.927 INFO:teuthology.orchestra.run.smithi052.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-02T13:40:19.927 INFO:teuthology.orchestra.run.smithi052.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-02T13:40:19.927 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-02T13:40:19.927 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-argparse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 45 k 2024-04-02T13:40:19.927 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 134 k 2024-04-02T13:40:19.927 INFO:teuthology.orchestra.run.smithi052.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-02T13:40:19.927 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-02T13:40:19.927 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-02T13:40:19.927 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-02T13:40:19.927 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-02T13:40:19.927 INFO:teuthology.orchestra.run.smithi052.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-02T13:40:19.927 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-02T13:40:19.928 INFO:teuthology.orchestra.run.smithi052.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-02T13:40:19.928 INFO:teuthology.orchestra.run.smithi052.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-02T13:40:19.928 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-02T13:40:19.928 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-02T13:40:19.928 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-02T13:40:19.928 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-02T13:40:19.928 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-02T13:40:19.928 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-02T13:40:19.928 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-02T13:40:19.928 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-02T13:40:19.928 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-02T13:40:19.928 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-02T13:40:19.928 INFO:teuthology.orchestra.run.smithi052.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-02T13:40:19.928 INFO:teuthology.orchestra.run.smithi052.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-02T13:40:19.929 INFO:teuthology.orchestra.run.smithi052.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-02T13:40:19.929 INFO:teuthology.orchestra.run.smithi052.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-02T13:40:19.929 INFO:teuthology.orchestra.run.smithi052.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-02T13:40:19.929 INFO:teuthology.orchestra.run.smithi052.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-02T13:40:19.929 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-02T13:40:19.929 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-02T13:40:19.929 INFO:teuthology.orchestra.run.smithi052.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-02T13:40:19.929 INFO:teuthology.orchestra.run.smithi052.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-02T13:40:19.929 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-02T13:40:19.929 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-02T13:40:19.929 INFO:teuthology.orchestra.run.smithi052.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-02T13:40:19.929 INFO:teuthology.orchestra.run.smithi052.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-02T13:40:19.929 INFO:teuthology.orchestra.run.smithi052.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-02T13:40:19.930 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-02T13:40:19.930 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-02T13:40:19.930 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-02T13:40:19.930 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-02T13:40:19.930 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-02T13:40:19.930 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-02T13:40:19.930 INFO:teuthology.orchestra.run.smithi052.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-02T13:40:19.930 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-02T13:40:19.930 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-02T13:40:19.931 INFO:teuthology.orchestra.run.smithi052.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-02T13:40:19.931 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-02T13:40:19.931 INFO:teuthology.orchestra.run.smithi052.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-02T13:40:19.931 INFO:teuthology.orchestra.run.smithi052.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-02T13:40:19.931 INFO:teuthology.orchestra.run.smithi052.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-02T13:40:19.931 INFO:teuthology.orchestra.run.smithi052.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-02T13:40:19.931 INFO:teuthology.orchestra.run.smithi052.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-02T13:40:19.931 INFO:teuthology.orchestra.run.smithi052.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-02T13:40:19.931 INFO:teuthology.orchestra.run.smithi052.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-02T13:40:19.931 INFO:teuthology.orchestra.run.smithi052.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-02T13:40:19.931 INFO:teuthology.orchestra.run.smithi052.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-02T13:40:19.931 INFO:teuthology.orchestra.run.smithi052.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-02T13:40:19.931 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-02T13:40:19.932 INFO:teuthology.orchestra.run.smithi052.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-02T13:40:19.932 INFO:teuthology.orchestra.run.smithi052.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-02T13:40:19.932 INFO:teuthology.orchestra.run.smithi052.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-02T13:40:19.932 INFO:teuthology.orchestra.run.smithi052.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-02T13:40:19.932 INFO:teuthology.orchestra.run.smithi052.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-02T13:40:19.932 INFO:teuthology.orchestra.run.smithi052.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-02T13:40:19.932 INFO:teuthology.orchestra.run.smithi052.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-02T13:40:19.932 INFO:teuthology.orchestra.run.smithi052.stdout:Installing weak dependencies: 2024-04-02T13:40:19.932 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-02T13:40:19.932 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:40:19.932 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction Summary 2024-04-02T13:40:19.932 INFO:teuthology.orchestra.run.smithi052.stdout:============================================================================================ 2024-04-02T13:40:19.932 INFO:teuthology.orchestra.run.smithi052.stdout:Install 150 Packages 2024-04-02T13:40:19.932 INFO:teuthology.orchestra.run.smithi052.stdout:Upgrade 24 Packages 2024-04-02T13:40:19.933 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:40:19.933 INFO:teuthology.orchestra.run.smithi052.stdout:Total download size: 226 M 2024-04-02T13:40:19.933 INFO:teuthology.orchestra.run.smithi052.stdout:Downloading Packages: 2024-04-02T13:40:20.112 INFO:teuthology.orchestra.run.smithi189.stdout:(30/174): rbd-mirror-19.0.0-2609.g191d963a.el9. 3.5 MB/s | 3.1 MB 00:00 2024-04-02T13:40:20.451 INFO:teuthology.orchestra.run.smithi088.stdout:(8/174): ceph-osd-19.0.0-2609.g191d963a.el9.x86 9.4 MB/s | 17 MB 00:01 2024-04-02T13:40:20.493 INFO:teuthology.orchestra.run.smithi088.stdout:(9/174): ceph-selinux-19.0.0-2609.g191d963a.el9 605 kB/s | 25 kB 00:00 2024-04-02T13:40:20.574 INFO:teuthology.orchestra.run.smithi189.stdout:(31/174): ceph-test-19.0.0-2609.g191d963a.el9.x 10 MB/s | 48 MB 00:04 2024-04-02T13:40:20.646 INFO:teuthology.orchestra.run.smithi189.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2609.g19 3.3 MB/s | 249 kB 00:00 2024-04-02T13:40:20.693 INFO:teuthology.orchestra.run.smithi088.stdout:(10/174): ceph-common-19.0.0-2609.g191d963a.el9 4.3 MB/s | 21 MB 00:04 2024-04-02T13:40:20.696 INFO:teuthology.orchestra.run.smithi189.stdout:(33/174): ceph-mgr-rook-19.0.0-2609.g191d963a.e 989 kB/s | 49 kB 00:00 2024-04-02T13:40:20.760 INFO:teuthology.orchestra.run.smithi088.stdout:(11/174): cephfs-mirror-19.0.0-2609.g191d963a.e 3.3 MB/s | 225 kB 00:00 2024-04-02T13:40:20.771 INFO:teuthology.orchestra.run.smithi189.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2609.g1 220 kB/s | 16 kB 00:00 2024-04-02T13:40:20.822 INFO:teuthology.orchestra.run.smithi052.stdout:(1/174): ceph-19.0.0-2609.g191d963a.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-04-02T13:40:20.835 INFO:teuthology.orchestra.run.smithi088.stdout:(12/174): cephfs-shell-19.0.0-2609.g191d963a.el 361 kB/s | 27 kB 00:00 2024-04-02T13:40:20.871 INFO:teuthology.orchestra.run.smithi189.stdout:(35/174): ceph-volume-19.0.0-2609.g191d963a.el9 2.6 MB/s | 264 kB 00:00 2024-04-02T13:40:20.911 INFO:teuthology.orchestra.run.smithi088.stdout:(13/174): libcephfs-devel-19.0.0-2609.g191d963a 422 kB/s | 32 kB 00:00 2024-04-02T13:40:21.038 INFO:teuthology.orchestra.run.smithi189.stdout:(36/174): cephadm-19.0.0-2609.g191d963a.el9.noa 4.5 MB/s | 762 kB 00:00 2024-04-02T13:40:21.147 INFO:teuthology.orchestra.run.smithi189.stdout:(37/174): ceph-mgr-dashboard-19.0.0-2609.g191d9 2.6 MB/s | 3.5 MB 00:01 2024-04-02T13:40:21.161 INFO:teuthology.orchestra.run.smithi088.stdout:(14/174): libcephfs2-19.0.0-2609.g191d963a.el9. 2.9 MB/s | 747 kB 00:00 2024-04-02T13:40:21.172 INFO:teuthology.orchestra.run.smithi189.stdout:(38/174): cephfs-top-19.0.0-2609.g191d963a.el9. 137 kB/s | 18 kB 00:00 2024-04-02T13:40:21.236 INFO:teuthology.orchestra.run.smithi088.stdout:(15/174): libcephsqlite-19.0.0-2609.g191d963a.e 2.1 MB/s | 164 kB 00:00 2024-04-02T13:40:21.312 INFO:teuthology.orchestra.run.smithi088.stdout:(16/174): librados-devel-19.0.0-2609.g191d963a. 1.7 MB/s | 127 kB 00:00 2024-04-02T13:40:21.322 INFO:teuthology.orchestra.run.smithi189.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 662 kB 00:00 2024-04-02T13:40:21.356 INFO:teuthology.orchestra.run.smithi189.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 3.2 MB/s | 106 kB 00:00 2024-04-02T13:40:21.398 INFO:teuthology.orchestra.run.smithi189.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-04-02T13:40:21.431 INFO:teuthology.orchestra.run.smithi189.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 954 kB/s | 32 kB 00:00 2024-04-02T13:40:21.462 INFO:teuthology.orchestra.run.smithi088.stdout:(17/174): libradosstriper1-19.0.0-2609.g191d963 3.3 MB/s | 500 kB 00:00 2024-04-02T13:40:21.490 INFO:teuthology.orchestra.run.smithi189.stdout:(43/174): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 920 kB 00:00 2024-04-02T13:40:21.557 INFO:teuthology.orchestra.run.smithi189.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-02T13:40:21.583 INFO:teuthology.orchestra.run.smithi189.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 189 kB/s | 17 kB 00:00 2024-04-02T13:40:21.674 INFO:teuthology.orchestra.run.smithi189.stdout:(46/174): ceph-mgr-diskprediction-local-19.0.0- 4.7 MB/s | 7.4 MB 00:01 2024-04-02T13:40:21.699 INFO:teuthology.orchestra.run.smithi189.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 71 kB/s | 10 kB 00:00 2024-04-02T13:40:21.725 INFO:teuthology.orchestra.run.smithi189.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 159 kB 00:00 2024-04-02T13:40:21.742 INFO:teuthology.orchestra.run.smithi189.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-04-02T13:40:21.775 INFO:teuthology.orchestra.run.smithi189.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-04-02T13:40:21.789 INFO:teuthology.orchestra.run.smithi052.stdout:(2/174): ceph-fuse-19.0.0-2609.g191d963a.el9.x8 921 kB/s | 890 kB 00:00 2024-04-02T13:40:21.801 INFO:teuthology.orchestra.run.smithi189.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 7.5 MB/s | 578 kB 00:00 2024-04-02T13:40:21.818 INFO:teuthology.orchestra.run.smithi189.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-04-02T13:40:21.843 INFO:teuthology.orchestra.run.smithi189.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.2 MB/s | 282 kB 00:00 2024-04-02T13:40:21.868 INFO:teuthology.orchestra.run.smithi189.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.7 MB/s | 187 kB 00:00 2024-04-02T13:40:21.885 INFO:teuthology.orchestra.run.smithi189.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 831 kB/s | 34 kB 00:00 2024-04-02T13:40:21.910 INFO:teuthology.orchestra.run.smithi189.stdout:(56/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.2 MB/s | 217 kB 00:00 2024-04-02T13:40:21.947 INFO:teuthology.orchestra.run.smithi052.stdout:(3/174): ceph-immutable-object-cache-19.0.0-260 915 kB/s | 145 kB 00:00 2024-04-02T13:40:21.994 INFO:teuthology.orchestra.run.smithi189.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-04-02T13:40:22.028 INFO:teuthology.orchestra.run.smithi189.stdout:(58/174): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.5 MB 00:00 2024-04-02T13:40:22.061 INFO:teuthology.orchestra.run.smithi189.stdout:(59/174): postgresql-contrib-13.14-1.el9.x86_64 5.4 MB/s | 817 kB 00:00 2024-04-02T13:40:22.086 INFO:teuthology.orchestra.run.smithi189.stdout:(60/174): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 132 kB 00:00 2024-04-02T13:40:22.128 INFO:teuthology.orchestra.run.smithi189.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-02T13:40:22.229 INFO:teuthology.orchestra.run.smithi189.stdout:(62/174): postgresql-server-13.14-1.el9.x86_64. 34 MB/s | 5.7 MB 00:00 2024-04-02T13:40:22.253 INFO:teuthology.orchestra.run.smithi189.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 465 kB/s | 78 kB 00:00 2024-04-02T13:40:22.354 INFO:teuthology.orchestra.run.smithi189.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 5.8 MB 00:00 2024-04-02T13:40:22.371 INFO:teuthology.orchestra.run.smithi189.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 174 kB/s | 24 kB 00:00 2024-04-02T13:40:22.388 INFO:teuthology.orchestra.run.smithi189.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 137 kB/s | 18 kB 00:00 2024-04-02T13:40:22.413 INFO:teuthology.orchestra.run.smithi189.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2024-04-02T13:40:22.430 INFO:teuthology.orchestra.run.smithi189.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 761 kB/s | 32 kB 00:00 2024-04-02T13:40:22.531 INFO:teuthology.orchestra.run.smithi189.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 4.8 MB 00:00 2024-04-02T13:40:22.556 INFO:teuthology.orchestra.run.smithi189.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-04-02T13:40:22.562 INFO:teuthology.orchestra.run.smithi088.stdout:(18/174): librgw2-19.0.0-2609.g191d963a.el9.x86 4.7 MB/s | 5.1 MB 00:01 2024-04-02T13:40:22.581 INFO:teuthology.orchestra.run.smithi189.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 174 kB 00:00 2024-04-02T13:40:22.606 INFO:teuthology.orchestra.run.smithi189.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 918 kB/s | 69 kB 00:00 2024-04-02T13:40:22.632 INFO:teuthology.orchestra.run.smithi189.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 824 kB/s | 41 kB 00:00 2024-04-02T13:40:22.646 INFO:teuthology.orchestra.run.smithi088.stdout:(19/174): python3-ceph-argparse-19.0.0-2609.g19 542 kB/s | 45 kB 00:00 2024-04-02T13:40:22.657 INFO:teuthology.orchestra.run.smithi189.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-04-02T13:40:22.682 INFO:teuthology.orchestra.run.smithi189.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-04-02T13:40:22.708 INFO:teuthology.orchestra.run.smithi189.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-04-02T13:40:22.729 INFO:teuthology.orchestra.run.smithi088.stdout:(20/174): python3-ceph-common-19.0.0-2609.g191d 1.6 MB/s | 134 kB 00:00 2024-04-02T13:40:22.732 INFO:teuthology.orchestra.run.smithi189.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 630 kB/s | 47 kB 00:00 2024-04-02T13:40:22.758 INFO:teuthology.orchestra.run.smithi189.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 569 kB/s | 43 kB 00:00 2024-04-02T13:40:22.783 INFO:teuthology.orchestra.run.smithi189.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 774 kB/s | 39 kB 00:00 2024-04-02T13:40:22.804 INFO:teuthology.orchestra.run.smithi088.stdout:(21/174): python3-cephfs-19.0.0-2609.g191d963a. 2.1 MB/s | 161 kB 00:00 2024-04-02T13:40:22.823 INFO:teuthology.orchestra.run.smithi052.stdout:(4/174): ceph-base-19.0.0-2609.g191d963a.el9.x8 2.3 MB/s | 5.4 MB 00:02 2024-04-02T13:40:22.947 INFO:teuthology.orchestra.run.smithi088.stdout:(22/174): python3-rados-19.0.0-2609.g191d963a.e 2.2 MB/s | 320 kB 00:00 2024-04-02T13:40:23.009 INFO:teuthology.orchestra.run.smithi189.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 164 kB/s | 41 kB 00:00 2024-04-02T13:40:23.059 INFO:teuthology.orchestra.run.smithi189.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 805 kB/s | 220 kB 00:00 2024-04-02T13:40:23.097 INFO:teuthology.orchestra.run.smithi088.stdout:(23/174): python3-rbd-19.0.0-2609.g191d963a.el9 1.9 MB/s | 299 kB 00:00 2024-04-02T13:40:23.140 INFO:teuthology.orchestra.run.smithi052.stdout:(5/174): ceph-mgr-19.0.0-2609.g191d963a.el9.x86 4.7 MB/s | 1.5 MB 00:00 2024-04-02T13:40:23.180 INFO:teuthology.orchestra.run.smithi088.stdout:(24/174): python3-rgw-19.0.0-2609.g191d963a.el9 1.2 MB/s | 99 kB 00:00 2024-04-02T13:40:23.209 INFO:teuthology.orchestra.run.smithi189.stdout:(82/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-04-02T13:40:23.234 INFO:teuthology.orchestra.run.smithi189.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 156 kB/s | 35 kB 00:00 2024-04-02T13:40:23.248 INFO:teuthology.orchestra.run.smithi052.stdout:(6/174): ceph-mds-19.0.0-2609.g191d963a.el9.x86 1.8 MB/s | 2.4 MB 00:01 2024-04-02T13:40:23.259 INFO:teuthology.orchestra.run.smithi189.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 232 kB/s | 46 kB 00:00 2024-04-02T13:40:23.264 INFO:teuthology.orchestra.run.smithi088.stdout:(25/174): rbd-fuse-19.0.0-2609.g191d963a.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-02T13:40:23.284 INFO:teuthology.orchestra.run.smithi189.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-04-02T13:40:23.310 INFO:teuthology.orchestra.run.smithi189.stdout:(86/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 300 kB 00:00 2024-04-02T13:40:23.335 INFO:teuthology.orchestra.run.smithi189.stdout:(87/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 832 kB/s | 63 kB 00:00 2024-04-02T13:40:23.360 INFO:teuthology.orchestra.run.smithi189.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 588 kB/s | 44 kB 00:00 2024-04-02T13:40:23.386 INFO:teuthology.orchestra.run.smithi189.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-04-02T13:40:23.411 INFO:teuthology.orchestra.run.smithi189.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-04-02T13:40:23.453 INFO:teuthology.orchestra.run.smithi189.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-04-02T13:40:23.478 INFO:teuthology.orchestra.run.smithi189.stdout:(92/174): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-04-02T13:40:23.503 INFO:teuthology.orchestra.run.smithi189.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-04-02T13:40:23.537 INFO:teuthology.orchestra.run.smithi189.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.2 MB/s | 187 kB 00:00 2024-04-02T13:40:23.662 INFO:teuthology.orchestra.run.smithi189.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-04-02T13:40:23.687 INFO:teuthology.orchestra.run.smithi189.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-04-02T13:40:23.713 INFO:teuthology.orchestra.run.smithi189.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-02T13:40:23.747 INFO:teuthology.orchestra.run.smithi189.stdout:(98/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.2 MB/s | 803 kB 00:00 2024-04-02T13:40:23.788 INFO:teuthology.orchestra.run.smithi189.stdout:(99/174): elfutils-libelf-devel-0.190-2.el9.x86 214 kB/s | 21 kB 00:00 2024-04-02T13:40:23.805 INFO:teuthology.orchestra.run.smithi189.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 276 kB/s | 25 kB 00:00 2024-04-02T13:40:23.906 INFO:teuthology.orchestra.run.smithi189.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 2.0 MB/s | 245 kB 00:00 2024-04-02T13:40:23.931 INFO:teuthology.orchestra.run.smithi189.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.9 MB/s | 248 kB 00:00 2024-04-02T13:40:23.956 INFO:teuthology.orchestra.run.smithi189.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-04-02T13:40:24.048 INFO:teuthology.orchestra.run.smithi189.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 190 kB/s | 22 kB 00:00 2024-04-02T13:40:24.089 INFO:teuthology.orchestra.run.smithi088.stdout:(26/174): rbd-mirror-19.0.0-2609.g191d963a.el9. 3.7 MB/s | 3.1 MB 00:00 2024-04-02T13:40:24.157 INFO:teuthology.orchestra.run.smithi052.stdout:(7/174): ceph-mon-19.0.0-2609.g191d963a.el9.x86 4.6 MB/s | 4.7 MB 00:01 2024-04-02T13:40:24.357 INFO:teuthology.orchestra.run.smithi189.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.1 MB/s | 863 kB 00:00 2024-04-02T13:40:24.499 INFO:teuthology.orchestra.run.smithi189.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-04-02T13:40:24.523 INFO:teuthology.orchestra.run.smithi088.stdout:(27/174): ceph-test-19.0.0-2609.g191d963a.el9.x 12 MB/s | 48 MB 00:04 2024-04-02T13:40:24.548 INFO:teuthology.orchestra.run.smithi088.stdout:(28/174): rbd-nbd-19.0.0-2609.g191d963a.el9.x86 371 kB/s | 170 kB 00:00 2024-04-02T13:40:24.574 INFO:teuthology.orchestra.run.smithi189.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 261 kB/s | 19 kB 00:00 2024-04-02T13:40:24.599 INFO:teuthology.orchestra.run.smithi088.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2609.g 453 kB/s | 34 kB 00:00 2024-04-02T13:40:24.623 INFO:teuthology.orchestra.run.smithi088.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2609.g191d963 1.8 MB/s | 141 kB 00:00 2024-04-02T13:40:24.649 INFO:teuthology.orchestra.run.smithi189.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.0 MB/s | 309 kB 00:00 2024-04-02T13:40:24.724 INFO:teuthology.orchestra.run.smithi189.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 346 kB/s | 26 kB 00:00 2024-04-02T13:40:24.816 INFO:teuthology.orchestra.run.smithi189.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 529 kB/s | 49 kB 00:00 2024-04-02T13:40:24.900 INFO:teuthology.orchestra.run.smithi189.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 810 kB/s | 67 kB 00:00 2024-04-02T13:40:25.017 INFO:teuthology.orchestra.run.smithi189.stdout:(112/174): openssl-devel-3.0.7-27.el9.x86_64.rp 3.3 MB/s | 4.1 MB 00:01 2024-04-02T13:40:25.042 INFO:teuthology.orchestra.run.smithi189.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-04-02T13:40:25.099 INFO:teuthology.orchestra.run.smithi088.stdout:(31/174): ceph-radosgw-19.0.0-2609.g191d963a.el 1.9 MB/s | 9.8 MB 00:05 2024-04-02T13:40:25.101 INFO:teuthology.orchestra.run.smithi189.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 8.4 MB/s | 4.4 MB 00:00 2024-04-02T13:40:25.157 INFO:teuthology.orchestra.run.smithi052.stdout:(8/174): ceph-common-19.0.0-2609.g191d963a.el9. 4.5 MB/s | 21 MB 00:04 2024-04-02T13:40:25.165 INFO:teuthology.orchestra.run.smithi088.stdout:(32/174): ceph-mgr-dashboard-19.0.0-2609.g191d9 6.2 MB/s | 3.5 MB 00:00 2024-04-02T13:40:25.193 INFO:teuthology.orchestra.run.smithi189.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 3.6 MB/s | 548 kB 00:00 2024-04-02T13:40:25.215 INFO:teuthology.orchestra.run.smithi052.stdout:(9/174): ceph-selinux-19.0.0-2609.g191d963a.el9 435 kB/s | 25 kB 00:00 2024-04-02T13:40:25.218 INFO:teuthology.orchestra.run.smithi189.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 371 kB/s | 43 kB 00:00 2024-04-02T13:40:25.224 INFO:teuthology.orchestra.run.smithi088.stdout:(33/174): ceph-mgr-rook-19.0.0-2609.g191d963a.e 846 kB/s | 49 kB 00:00 2024-04-02T13:40:25.249 INFO:teuthology.orchestra.run.smithi088.stdout:(34/174): ceph-mgr-modules-core-19.0.0-2609.g19 1.6 MB/s | 249 kB 00:00 2024-04-02T13:40:25.268 INFO:teuthology.orchestra.run.smithi189.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 428 kB/s | 32 kB 00:00 2024-04-02T13:40:25.292 INFO:teuthology.orchestra.run.smithi088.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2609.g1 246 kB/s | 16 kB 00:00 2024-04-02T13:40:25.304 INFO:teuthology.orchestra.run.smithi189.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 174 kB/s | 15 kB 00:00 2024-04-02T13:40:25.380 INFO:teuthology.orchestra.run.smithi189.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-04-02T13:40:25.413 INFO:teuthology.orchestra.run.smithi189.stdout:(120/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.1 MB/s | 838 kB 00:00 2024-04-02T13:40:25.425 INFO:teuthology.orchestra.run.smithi088.stdout:(36/174): ceph-volume-19.0.0-2609.g191d963a.el9 1.5 MB/s | 264 kB 00:00 2024-04-02T13:40:25.438 INFO:teuthology.orchestra.run.smithi189.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-04-02T13:40:25.469 INFO:teuthology.orchestra.run.smithi088.stdout:(37/174): cephadm-19.0.0-2609.g191d963a.el9.noa 4.2 MB/s | 762 kB 00:00 2024-04-02T13:40:25.514 INFO:teuthology.orchestra.run.smithi189.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 351 kB/s | 35 kB 00:00 2024-04-02T13:40:25.539 INFO:teuthology.orchestra.run.smithi189.stdout:(123/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.9 MB/s | 303 kB 00:00 2024-04-02T13:40:25.542 INFO:teuthology.orchestra.run.smithi088.stdout:(38/174): cephfs-top-19.0.0-2609.g191d963a.el9. 156 kB/s | 18 kB 00:00 2024-04-02T13:40:25.565 INFO:teuthology.orchestra.run.smithi189.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 1.6 MB/s | 203 kB 00:00 2024-04-02T13:40:25.631 INFO:teuthology.orchestra.run.smithi189.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-04-02T13:40:25.690 INFO:teuthology.orchestra.run.smithi189.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 85 kB/s | 11 kB 00:00 2024-04-02T13:40:25.715 INFO:teuthology.orchestra.run.smithi189.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2024-04-02T13:40:25.751 INFO:teuthology.orchestra.run.smithi088.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 662 kB 00:00 2024-04-02T13:40:25.784 INFO:teuthology.orchestra.run.smithi088.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 3.7 MB/s | 920 kB 00:00 2024-04-02T13:40:25.790 INFO:teuthology.orchestra.run.smithi189.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-04-02T13:40:25.799 INFO:teuthology.orchestra.run.smithi052.stdout:(10/174): ceph-radosgw-19.0.0-2609.g191d963a.el 6.0 MB/s | 9.8 MB 00:01 2024-04-02T13:40:25.807 INFO:teuthology.orchestra.run.smithi189.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-04-02T13:40:25.835 INFO:teuthology.orchestra.run.smithi088.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 106 kB 00:00 2024-04-02T13:40:25.860 INFO:teuthology.orchestra.run.smithi088.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 4.0 MB/s | 307 kB 00:00 2024-04-02T13:40:25.866 INFO:teuthology.orchestra.run.smithi052.stdout:(11/174): cephfs-mirror-19.0.0-2609.g191d963a.e 3.3 MB/s | 225 kB 00:00 2024-04-02T13:40:25.866 INFO:teuthology.orchestra.run.smithi189.stdout:(130/174): python3-grpcio-1.46.7-10.el9.x86_64. 5.8 MB/s | 2.0 MB 00:00 2024-04-02T13:40:25.883 INFO:teuthology.orchestra.run.smithi189.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-04-02T13:40:25.885 INFO:teuthology.orchestra.run.smithi088.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 635 kB/s | 32 kB 00:00 2024-04-02T13:40:25.933 INFO:teuthology.orchestra.run.smithi052.stdout:(12/174): cephfs-shell-19.0.0-2609.g191d963a.el 406 kB/s | 27 kB 00:00 2024-04-02T13:40:25.983 INFO:teuthology.orchestra.run.smithi052.stdout:(13/174): libcephfs-devel-19.0.0-2609.g191d963a 635 kB/s | 32 kB 00:00 2024-04-02T13:40:26.000 INFO:teuthology.orchestra.run.smithi189.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-04-02T13:40:26.025 INFO:teuthology.orchestra.run.smithi189.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 555 kB/s | 79 kB 00:00 2024-04-02T13:40:26.036 INFO:teuthology.orchestra.run.smithi088.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-04-02T13:40:26.042 INFO:teuthology.orchestra.run.smithi189.stdout:(134/174): python3-logutils-0.3.5-21.el9.noarch 264 kB/s | 46 kB 00:00 2024-04-02T13:40:26.061 INFO:teuthology.orchestra.run.smithi088.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 98 kB/s | 17 kB 00:00 2024-04-02T13:40:26.103 INFO:teuthology.orchestra.run.smithi088.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 243 kB/s | 10 kB 00:00 2024-04-02T13:40:26.109 INFO:teuthology.orchestra.run.smithi189.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 530 kB/s | 58 kB 00:00 2024-04-02T13:40:26.117 INFO:teuthology.orchestra.run.smithi052.stdout:(14/174): libcephfs2-19.0.0-2609.g191d963a.el9. 5.5 MB/s | 747 kB 00:00 2024-04-02T13:40:26.128 INFO:teuthology.orchestra.run.smithi088.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 159 kB 00:00 2024-04-02T13:40:26.134 INFO:teuthology.orchestra.run.smithi189.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-04-02T13:40:26.153 INFO:teuthology.orchestra.run.smithi088.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 886 kB/s | 44 kB 00:00 2024-04-02T13:40:26.160 INFO:teuthology.orchestra.run.smithi189.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-04-02T13:40:26.179 INFO:teuthology.orchestra.run.smithi088.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-04-02T13:40:26.184 INFO:teuthology.orchestra.run.smithi052.stdout:(15/174): libcephsqlite-19.0.0-2609.g191d963a.e 2.4 MB/s | 164 kB 00:00 2024-04-02T13:40:26.210 INFO:teuthology.orchestra.run.smithi189.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 895 kB/s | 90 kB 00:00 2024-04-02T13:40:26.212 INFO:teuthology.orchestra.run.smithi088.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 9.8 MB/s | 578 kB 00:00 2024-04-02T13:40:26.238 INFO:teuthology.orchestra.run.smithi088.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 809 kB/s | 46 kB 00:00 2024-04-02T13:40:26.244 INFO:teuthology.orchestra.run.smithi189.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2024-04-02T13:40:26.251 INFO:teuthology.orchestra.run.smithi052.stdout:(16/174): librados-devel-19.0.0-2609.g191d963a. 1.9 MB/s | 127 kB 00:00 2024-04-02T13:40:26.263 INFO:teuthology.orchestra.run.smithi088.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-04-02T13:40:26.288 INFO:teuthology.orchestra.run.smithi088.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-04-02T13:40:26.294 INFO:teuthology.orchestra.run.smithi189.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-02T13:40:26.314 INFO:teuthology.orchestra.run.smithi088.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-04-02T13:40:26.319 INFO:teuthology.orchestra.run.smithi189.stdout:(141/174): python3-pyperclip-1.8.2-2.el9.noarch 153 kB/s | 28 kB 00:00 2024-04-02T13:40:26.339 INFO:teuthology.orchestra.run.smithi088.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 688 kB/s | 34 kB 00:00 2024-04-02T13:40:26.344 INFO:teuthology.orchestra.run.smithi189.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 588 kB/s | 59 kB 00:00 2024-04-02T13:40:26.359 INFO:teuthology.orchestra.run.smithi052.stdout:(17/174): libradosstriper1-19.0.0-2609.g191d963 4.5 MB/s | 500 kB 00:00 2024-04-02T13:40:26.378 INFO:teuthology.orchestra.run.smithi189.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 426 kB/s | 36 kB 00:00 2024-04-02T13:40:26.412 INFO:teuthology.orchestra.run.smithi189.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 554 kB/s | 51 kB 00:00 2024-04-02T13:40:26.431 INFO:teuthology.orchestra.run.smithi088.stdout:(56/174): ceph-mgr-diskprediction-local-19.0.0- 4.1 MB/s | 7.4 MB 00:01 2024-04-02T13:40:26.445 INFO:teuthology.orchestra.run.smithi189.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-02T13:40:26.479 INFO:teuthology.orchestra.run.smithi189.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 892 kB/s | 90 kB 00:00 2024-04-02T13:40:26.506 INFO:teuthology.orchestra.run.smithi088.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2024-04-02T13:40:26.521 INFO:teuthology.orchestra.run.smithi189.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-04-02T13:40:26.548 INFO:teuthology.orchestra.run.smithi088.stdout:(58/174): postgresql-13.14-1.el9.x86_64.rpm 7.4 MB/s | 1.5 MB 00:00 2024-04-02T13:40:26.582 INFO:teuthology.orchestra.run.smithi088.stdout:(59/174): postgresql-contrib-13.14-1.el9.x86_64 5.4 MB/s | 817 kB 00:00 2024-04-02T13:40:26.609 INFO:teuthology.orchestra.run.smithi088.stdout:(60/174): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 132 kB 00:00 2024-04-02T13:40:26.621 INFO:teuthology.orchestra.run.smithi189.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 114 kB/s | 20 kB 00:00 2024-04-02T13:40:26.646 INFO:teuthology.orchestra.run.smithi189.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-04-02T13:40:26.668 INFO:teuthology.orchestra.run.smithi052.stdout:(18/174): ceph-osd-19.0.0-2609.g191d963a.el9.x8 4.9 MB/s | 17 MB 00:03 2024-04-02T13:40:26.676 INFO:teuthology.orchestra.run.smithi088.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-04-02T13:40:26.771 INFO:teuthology.orchestra.run.smithi189.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 6.4 MB/s | 1.6 MB 00:00 2024-04-02T13:40:26.785 INFO:teuthology.orchestra.run.smithi052.stdout:(19/174): python3-ceph-argparse-19.0.0-2609.g19 388 kB/s | 45 kB 00:00 2024-04-02T13:40:26.801 INFO:teuthology.orchestra.run.smithi088.stdout:(62/174): postgresql-server-13.14-1.el9.x86_64. 26 MB/s | 5.7 MB 00:00 2024-04-02T13:40:26.826 INFO:teuthology.orchestra.run.smithi088.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 358 kB/s | 78 kB 00:00 2024-04-02T13:40:26.869 INFO:teuthology.orchestra.run.smithi052.stdout:(20/174): python3-ceph-common-19.0.0-2609.g191d 1.6 MB/s | 134 kB 00:00 2024-04-02T13:40:26.927 INFO:teuthology.orchestra.run.smithi088.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 5.8 MB 00:00 2024-04-02T13:40:26.952 INFO:teuthology.orchestra.run.smithi088.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 165 kB/s | 24 kB 00:00 2024-04-02T13:40:26.961 INFO:teuthology.orchestra.run.smithi052.stdout:(21/174): python3-cephfs-19.0.0-2609.g191d963a. 1.7 MB/s | 161 kB 00:00 2024-04-02T13:40:26.977 INFO:teuthology.orchestra.run.smithi088.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 122 kB/s | 18 kB 00:00 2024-04-02T13:40:27.003 INFO:teuthology.orchestra.run.smithi088.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-04-02T13:40:27.013 INFO:teuthology.orchestra.run.smithi189.stdout:(151/174): NetworkManager-1.47.2-1.el9.x86_64.r 9.5 MB/s | 2.3 MB 00:00 2024-04-02T13:40:27.028 INFO:teuthology.orchestra.run.smithi088.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 634 kB/s | 32 kB 00:00 2024-04-02T13:40:27.070 INFO:teuthology.orchestra.run.smithi052.stdout:(22/174): python3-rados-19.0.0-2609.g191d963a.e 2.9 MB/s | 320 kB 00:00 2024-04-02T13:40:27.122 INFO:teuthology.orchestra.run.smithi189.stdout:(152/174): NetworkManager-libnm-1.47.2-1.el9.x8 17 MB/s | 1.8 MB 00:00 2024-04-02T13:40:27.145 INFO:teuthology.orchestra.run.smithi052.stdout:(23/174): librgw2-19.0.0-2609.g191d963a.el9.x86 6.6 MB/s | 5.1 MB 00:00 2024-04-02T13:40:27.156 INFO:teuthology.orchestra.run.smithi189.stdout:(153/174): NetworkManager-team-1.47.2-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-04-02T13:40:27.170 INFO:teuthology.orchestra.run.smithi052.stdout:(24/174): python3-rbd-19.0.0-2609.g191d963a.el9 2.9 MB/s | 299 kB 00:00 2024-04-02T13:40:27.178 INFO:teuthology.orchestra.run.smithi088.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 4.8 MB 00:00 2024-04-02T13:40:27.198 INFO:teuthology.orchestra.run.smithi189.stdout:(154/174): NetworkManager-tui-1.47.2-1.el9.x86_ 5.8 MB/s | 247 kB 00:00 2024-04-02T13:40:27.203 INFO:teuthology.orchestra.run.smithi088.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 217 kB 00:00 2024-04-02T13:40:27.204 INFO:teuthology.orchestra.run.smithi052.stdout:(25/174): python3-rgw-19.0.0-2609.g191d963a.el9 1.7 MB/s | 99 kB 00:00 2024-04-02T13:40:27.229 INFO:teuthology.orchestra.run.smithi088.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 882 kB/s | 174 kB 00:00 2024-04-02T13:40:27.240 INFO:teuthology.orchestra.run.smithi189.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.5 MB/s | 321 kB 00:00 2024-04-02T13:40:27.246 INFO:teuthology.orchestra.run.smithi052.stdout:(26/174): rbd-fuse-19.0.0-2609.g191d963a.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-02T13:40:27.254 INFO:teuthology.orchestra.run.smithi088.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 924 kB/s | 69 kB 00:00 2024-04-02T13:40:27.279 INFO:teuthology.orchestra.run.smithi088.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 827 kB/s | 41 kB 00:00 2024-04-02T13:40:27.304 INFO:teuthology.orchestra.run.smithi088.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-04-02T13:40:27.346 INFO:teuthology.orchestra.run.smithi088.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 132 kB 00:00 2024-04-02T13:40:27.354 INFO:teuthology.orchestra.run.smithi052.stdout:(27/174): rbd-nbd-19.0.0-2609.g191d963a.el9.x86 1.5 MB/s | 170 kB 00:00 2024-04-02T13:40:27.371 INFO:teuthology.orchestra.run.smithi088.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 711 kB/s | 47 kB 00:00 2024-04-02T13:40:27.397 INFO:teuthology.orchestra.run.smithi088.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 211 kB 00:00 2024-04-02T13:40:27.407 INFO:teuthology.orchestra.run.smithi189.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 428 kB/s | 71 kB 00:00 2024-04-02T13:40:27.422 INFO:teuthology.orchestra.run.smithi088.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 567 kB/s | 43 kB 00:00 2024-04-02T13:40:27.449 INFO:teuthology.orchestra.run.smithi189.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 876 kB/s | 36 kB 00:00 2024-04-02T13:40:27.516 INFO:teuthology.orchestra.run.smithi189.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.8 MB/s | 192 kB 00:00 2024-04-02T13:40:27.588 INFO:teuthology.orchestra.run.smithi052.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2609.g 145 kB/s | 34 kB 00:00 2024-04-02T13:40:27.643 INFO:teuthology.orchestra.run.smithi189.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.0 MB/s | 256 kB 00:00 2024-04-02T13:40:27.705 INFO:teuthology.orchestra.run.smithi052.stdout:(29/174): rbd-mirror-19.0.0-2609.g191d963a.el9. 6.2 MB/s | 3.1 MB 00:00 2024-04-02T13:40:27.723 INFO:teuthology.orchestra.run.smithi088.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 119 kB/s | 39 kB 00:00 2024-04-02T13:40:27.731 INFO:teuthology.orchestra.run.smithi052.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2609.g191d963 995 kB/s | 141 kB 00:00 2024-04-02T13:40:27.766 INFO:teuthology.orchestra.run.smithi189.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 187 kB/s | 23 kB 00:00 2024-04-02T13:40:27.842 INFO:teuthology.orchestra.run.smithi189.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 108 kB 00:00 2024-04-02T13:40:27.956 INFO:teuthology.orchestra.run.smithi088.stdout:(80/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-04-02T13:40:27.982 INFO:teuthology.orchestra.run.smithi088.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 73 kB/s | 41 kB 00:00 2024-04-02T13:40:28.007 INFO:teuthology.orchestra.run.smithi088.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 780 kB/s | 220 kB 00:00 2024-04-02T13:40:28.032 INFO:teuthology.orchestra.run.smithi088.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 467 kB/s | 35 kB 00:00 2024-04-02T13:40:28.057 INFO:teuthology.orchestra.run.smithi088.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 928 kB/s | 46 kB 00:00 2024-04-02T13:40:28.082 INFO:teuthology.orchestra.run.smithi088.stdout:(85/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 300 kB 00:00 2024-04-02T13:40:28.108 INFO:teuthology.orchestra.run.smithi088.stdout:(86/174): uuid-1.6.2-55.el9.x86_64.rpm 746 kB/s | 56 kB 00:00 2024-04-02T13:40:28.135 INFO:teuthology.orchestra.run.smithi088.stdout:(87/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 834 kB/s | 63 kB 00:00 2024-04-02T13:40:28.158 INFO:teuthology.orchestra.run.smithi088.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 584 kB/s | 44 kB 00:00 2024-04-02T13:40:28.184 INFO:teuthology.orchestra.run.smithi088.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-04-02T13:40:28.209 INFO:teuthology.orchestra.run.smithi088.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-04-02T13:40:28.225 INFO:teuthology.orchestra.run.smithi189.stdout:(162/174): librbd1-19.0.0-2609.g191d963a.el9.x8 2.0 MB/s | 3.1 MB 00:01 2024-04-02T13:40:28.243 INFO:teuthology.orchestra.run.smithi088.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-04-02T13:40:28.268 INFO:teuthology.orchestra.run.smithi088.stdout:(92/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-04-02T13:40:28.281 INFO:teuthology.orchestra.run.smithi052.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2609.g191d9 6.2 MB/s | 3.5 MB 00:00 2024-04-02T13:40:28.293 INFO:teuthology.orchestra.run.smithi088.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-04-02T13:40:28.300 INFO:teuthology.orchestra.run.smithi189.stdout:(163/174): librados2-19.0.0-2609.g191d963a.el9. 2.0 MB/s | 3.4 MB 00:01 2024-04-02T13:40:28.318 INFO:teuthology.orchestra.run.smithi088.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-04-02T13:40:28.351 INFO:teuthology.orchestra.run.smithi189.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 136 kB 00:00 2024-04-02T13:40:28.356 INFO:teuthology.orchestra.run.smithi052.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2609.g19 3.2 MB/s | 249 kB 00:00 2024-04-02T13:40:28.401 INFO:teuthology.orchestra.run.smithi189.stdout:(165/174): libfdisk-2.37.4-18.el9.x86_64.rpm 277 kB/s | 155 kB 00:00 2024-04-02T13:40:28.415 INFO:teuthology.orchestra.run.smithi052.stdout:(33/174): ceph-mgr-rook-19.0.0-2609.g191d963a.e 847 kB/s | 49 kB 00:00 2024-04-02T13:40:28.426 INFO:teuthology.orchestra.run.smithi189.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 509 kB/s | 64 kB 00:00 2024-04-02T13:40:28.465 INFO:teuthology.orchestra.run.smithi052.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2609.g1 330 kB/s | 16 kB 00:00 2024-04-02T13:40:28.468 INFO:teuthology.orchestra.run.smithi189.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 245 kB/s | 29 kB 00:00 2024-04-02T13:40:28.515 INFO:teuthology.orchestra.run.smithi052.stdout:(35/174): ceph-volume-19.0.0-2609.g191d963a.el9 5.1 MB/s | 264 kB 00:00 2024-04-02T13:40:28.518 INFO:teuthology.orchestra.run.smithi189.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 10 MB/s | 1.2 MB 00:00 2024-04-02T13:40:28.569 INFO:teuthology.orchestra.run.smithi189.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 9.1 MB/s | 465 kB 00:00 2024-04-02T13:40:28.619 INFO:teuthology.orchestra.run.smithi189.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 11 MB/s | 2.2 MB 00:00 2024-04-02T13:40:28.640 INFO:teuthology.orchestra.run.smithi052.stdout:(36/174): cephadm-19.0.0-2609.g191d963a.el9.noa 6.0 MB/s | 762 kB 00:00 2024-04-02T13:40:28.644 INFO:teuthology.orchestra.run.smithi189.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 414 kB/s | 31 kB 00:00 2024-04-02T13:40:28.687 INFO:teuthology.orchestra.run.smithi189.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 16 MB/s | 1.1 MB 00:00 2024-04-02T13:40:28.690 INFO:teuthology.orchestra.run.smithi052.stdout:(37/174): cephfs-top-19.0.0-2609.g191d963a.el9. 366 kB/s | 18 kB 00:00 2024-04-02T13:40:28.762 INFO:teuthology.orchestra.run.smithi189.stdout:(173/174): util-linux-2.37.4-18.el9.x86_64.rpm 7.8 MB/s | 2.3 MB 00:00 2024-04-02T13:40:28.787 INFO:teuthology.orchestra.run.smithi189.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 523 kB/s | 74 kB 00:00 2024-04-02T13:40:28.792 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-04-02T13:40:28.792 INFO:teuthology.orchestra.run.smithi189.stdout:Total 15 MB/s | 226 MB 00:15 2024-04-02T13:40:28.858 INFO:teuthology.orchestra.run.smithi052.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 3.9 MB/s | 662 kB 00:00 2024-04-02T13:40:29.179 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-04-02T13:40:29.185 INFO:teuthology.orchestra.run.smithi088.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 204 kB/s | 187 kB 00:00 2024-04-02T13:40:29.227 INFO:teuthology.orchestra.run.smithi088.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 215 kB/s | 195 kB 00:00 2024-04-02T13:40:29.308 INFO:teuthology.orchestra.run.smithi052.stdout:(39/174): ceph-test-19.0.0-2609.g191d963a.el9.x 12 MB/s | 48 MB 00:04 2024-04-02T13:40:29.333 INFO:teuthology.orchestra.run.smithi052.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 1.9 MB/s | 920 kB 00:00 2024-04-02T13:40:29.358 INFO:teuthology.orchestra.run.smithi052.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 106 kB 00:00 2024-04-02T13:40:29.392 INFO:teuthology.orchestra.run.smithi052.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 307 kB 00:00 2024-04-02T13:40:29.418 INFO:teuthology.orchestra.run.smithi052.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 545 kB/s | 32 kB 00:00 2024-04-02T13:40:29.452 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-04-02T13:40:29.452 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-04-02T13:40:29.485 INFO:teuthology.orchestra.run.smithi052.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-04-02T13:40:29.510 INFO:teuthology.orchestra.run.smithi052.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 193 kB/s | 17 kB 00:00 2024-04-02T13:40:29.535 INFO:teuthology.orchestra.run.smithi052.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 201 kB/s | 10 kB 00:00 2024-04-02T13:40:29.619 INFO:teuthology.orchestra.run.smithi052.stdout:(47/174): ceph-mgr-diskprediction-local-19.0.0- 3.9 MB/s | 7.4 MB 00:01 2024-04-02T13:40:29.644 INFO:teuthology.orchestra.run.smithi052.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 159 kB 00:00 2024-04-02T13:40:29.678 INFO:teuthology.orchestra.run.smithi052.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 311 kB/s | 44 kB 00:00 2024-04-02T13:40:29.711 INFO:teuthology.orchestra.run.smithi052.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 578 kB 00:00 2024-04-02T13:40:29.737 INFO:teuthology.orchestra.run.smithi052.stdout:(51/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 661 kB 00:00 2024-04-02T13:40:29.762 INFO:teuthology.orchestra.run.smithi052.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 561 kB/s | 46 kB 00:00 2024-04-02T13:40:29.796 INFO:teuthology.orchestra.run.smithi052.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.3 MB/s | 282 kB 00:00 2024-04-02T13:40:29.821 INFO:teuthology.orchestra.run.smithi052.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2024-04-02T13:40:29.846 INFO:teuthology.orchestra.run.smithi052.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.5 MB/s | 217 kB 00:00 2024-04-02T13:40:29.852 INFO:teuthology.orchestra.run.smithi088.stdout:(97/174): elfutils-libelf-devel-0.190-2.el9.x86 34 kB/s | 21 kB 00:00 2024-04-02T13:40:29.872 INFO:teuthology.orchestra.run.smithi052.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 453 kB/s | 34 kB 00:00 2024-04-02T13:40:29.930 INFO:teuthology.orchestra.run.smithi052.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 18 MB/s | 1.5 MB 00:00 2024-04-02T13:40:29.997 INFO:teuthology.orchestra.run.smithi052.stdout:(58/174): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-04-02T13:40:30.002 INFO:teuthology.orchestra.run.smithi088.stdout:(98/174): libgfortran-11.4.1-3.el9.x86_64.rpm 470 kB/s | 803 kB 00:01 2024-04-02T13:40:30.028 INFO:teuthology.orchestra.run.smithi088.stdout:(99/174): python3-requests-2.25.1-8.el9.noarch. 148 kB/s | 125 kB 00:00 2024-04-02T13:40:30.039 INFO:teuthology.orchestra.run.smithi052.stdout:(59/174): postgresql-contrib-13.14-1.el9.x86_64 4.8 MB/s | 817 kB 00:00 2024-04-02T13:40:30.064 INFO:teuthology.orchestra.run.smithi052.stdout:(60/174): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 132 kB 00:00 2024-04-02T13:40:30.164 INFO:teuthology.orchestra.run.smithi052.stdout:(61/174): postgresql-server-13.14-1.el9.x86_64. 35 MB/s | 5.7 MB 00:00 2024-04-02T13:40:30.190 INFO:teuthology.orchestra.run.smithi052.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 636 kB/s | 78 kB 00:00 2024-04-02T13:40:30.232 INFO:teuthology.orchestra.run.smithi052.stdout:(63/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.1 MB/s | 1.0 MB 00:00 2024-04-02T13:40:30.315 INFO:teuthology.orchestra.run.smithi052.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 39 MB/s | 5.8 MB 00:00 2024-04-02T13:40:30.340 INFO:teuthology.orchestra.run.smithi052.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 168 kB/s | 24 kB 00:00 2024-04-02T13:40:30.365 INFO:teuthology.orchestra.run.smithi052.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 138 kB/s | 18 kB 00:00 2024-04-02T13:40:30.391 INFO:teuthology.orchestra.run.smithi052.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-04-02T13:40:30.416 INFO:teuthology.orchestra.run.smithi052.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 632 kB/s | 32 kB 00:00 2024-04-02T13:40:30.478 INFO:teuthology.orchestra.run.smithi088.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 41 kB/s | 25 kB 00:00 2024-04-02T13:40:30.516 INFO:teuthology.orchestra.run.smithi052.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 4.8 MB 00:00 2024-04-02T13:40:30.542 INFO:teuthology.orchestra.run.smithi052.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 217 kB 00:00 2024-04-02T13:40:30.567 INFO:teuthology.orchestra.run.smithi052.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 174 kB 00:00 2024-04-02T13:40:30.600 INFO:teuthology.orchestra.run.smithi052.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 829 kB/s | 69 kB 00:00 2024-04-02T13:40:30.617 INFO:teuthology.orchestra.run.smithi052.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 825 kB/s | 41 kB 00:00 2024-04-02T13:40:30.643 INFO:teuthology.orchestra.run.smithi052.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-04-02T13:40:30.668 INFO:teuthology.orchestra.run.smithi052.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-04-02T13:40:30.693 INFO:teuthology.orchestra.run.smithi052.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.8 MB/s | 211 kB 00:00 2024-04-02T13:40:30.719 INFO:teuthology.orchestra.run.smithi052.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 953 kB/s | 47 kB 00:00 2024-04-02T13:40:30.744 INFO:teuthology.orchestra.run.smithi052.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 573 kB/s | 43 kB 00:00 2024-04-02T13:40:30.894 INFO:teuthology.orchestra.run.smithi052.stdout:(79/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 279 kB/s | 41 kB 00:00 2024-04-02T13:40:30.919 INFO:teuthology.orchestra.run.smithi052.stdout:(80/174): python3-toml-0.10.2-6.el9.noarch.rpm 224 kB/s | 39 kB 00:00 2024-04-02T13:40:31.086 INFO:teuthology.orchestra.run.smithi052.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 40 MB/s | 16 MB 00:00 2024-04-02T13:40:31.111 INFO:teuthology.orchestra.run.smithi088.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 226 kB/s | 245 kB 00:01 2024-04-02T13:40:31.112 INFO:teuthology.orchestra.run.smithi052.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 1.0 MB/s | 220 kB 00:00 2024-04-02T13:40:31.137 INFO:teuthology.orchestra.run.smithi052.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 162 kB/s | 35 kB 00:00 2024-04-02T13:40:31.162 INFO:teuthology.orchestra.run.smithi052.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 928 kB/s | 46 kB 00:00 2024-04-02T13:40:31.184 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-04-02T13:40:31.185 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-04-02T13:40:31.196 INFO:teuthology.orchestra.run.smithi052.stdout:(85/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.5 MB/s | 300 kB 00:00 2024-04-02T13:40:31.221 INFO:teuthology.orchestra.run.smithi052.stdout:(86/174): uuid-1.6.2-55.el9.x86_64.rpm 673 kB/s | 56 kB 00:00 2024-04-02T13:40:31.246 INFO:teuthology.orchestra.run.smithi052.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 879 kB/s | 44 kB 00:00 2024-04-02T13:40:31.271 INFO:teuthology.orchestra.run.smithi052.stdout:(88/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 829 kB/s | 63 kB 00:00 2024-04-02T13:40:31.296 INFO:teuthology.orchestra.run.smithi052.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-04-02T13:40:31.322 INFO:teuthology.orchestra.run.smithi052.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-04-02T13:40:31.347 INFO:teuthology.orchestra.run.smithi052.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 103 kB 00:00 2024-04-02T13:40:31.381 INFO:teuthology.orchestra.run.smithi052.stdout:(92/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-04-02T13:40:31.395 INFO:teuthology.orchestra.run.smithi088.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 270 kB/s | 248 kB 00:00 2024-04-02T13:40:31.406 INFO:teuthology.orchestra.run.smithi052.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 124 kB 00:00 2024-04-02T13:40:31.431 INFO:teuthology.orchestra.run.smithi052.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.2 MB/s | 187 kB 00:00 2024-04-02T13:40:31.573 INFO:teuthology.orchestra.run.smithi052.stdout:(95/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-04-02T13:40:31.598 INFO:teuthology.orchestra.run.smithi052.stdout:(96/174): jq-1.6-16.el9.x86_64.rpm 976 kB/s | 187 kB 00:00 2024-04-02T13:40:31.624 INFO:teuthology.orchestra.run.smithi052.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-02T13:40:31.657 INFO:teuthology.orchestra.run.smithi052.stdout:(98/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.1 MB/s | 803 kB 00:00 2024-04-02T13:40:31.758 INFO:teuthology.orchestra.run.smithi052.stdout:(99/174): elfutils-libelf-devel-0.190-2.el9.x86 135 kB/s | 21 kB 00:00 2024-04-02T13:40:31.783 INFO:teuthology.orchestra.run.smithi052.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 160 kB/s | 25 kB 00:00 2024-04-02T13:40:31.822 INFO:teuthology.orchestra.run.smithi088.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 132 kB/s | 94 kB 00:00 2024-04-02T13:40:31.892 INFO:teuthology.orchestra.run.smithi052.stdout:(101/174): python3-jinja2-2.11.3-5.el9.noarch.r 2.2 MB/s | 248 kB 00:00 2024-04-02T13:40:31.917 INFO:teuthology.orchestra.run.smithi052.stdout:(102/174): python3-devel-3.9.17-2.el9.x86_64.rp 1.5 MB/s | 245 kB 00:00 2024-04-02T13:40:31.970 INFO:teuthology.orchestra.run.smithi088.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 39 kB/s | 22 kB 00:00 2024-04-02T13:40:31.975 INFO:teuthology.orchestra.run.smithi052.stdout:(103/174): openssl-devel-3.0.7-27.el9.x86_64.rp 13 MB/s | 4.1 MB 00:00 2024-04-02T13:40:32.001 INFO:teuthology.orchestra.run.smithi052.stdout:(104/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 862 kB/s | 94 kB 00:00 2024-04-02T13:40:32.035 INFO:teuthology.orchestra.run.smithi052.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 190 kB/s | 22 kB 00:00 2024-04-02T13:40:32.185 INFO:teuthology.orchestra.run.smithi052.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 4.0 MB/s | 863 kB 00:00 2024-04-02T13:40:32.412 INFO:teuthology.orchestra.run.smithi088.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-04-02T13:40:32.427 INFO:teuthology.orchestra.run.smithi052.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 80 kB/s | 19 kB 00:00 2024-04-02T13:40:32.452 INFO:teuthology.orchestra.run.smithi052.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 741 kB/s | 309 kB 00:00 2024-04-02T13:40:32.477 INFO:teuthology.orchestra.run.smithi052.stdout:(109/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-02T13:40:32.527 INFO:teuthology.orchestra.run.smithi052.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 345 kB/s | 26 kB 00:00 2024-04-02T13:40:32.553 INFO:teuthology.orchestra.run.smithi052.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 642 kB/s | 49 kB 00:00 2024-04-02T13:40:32.554 INFO:teuthology.orchestra.run.smithi088.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 2.1 MB/s | 309 kB 00:00 2024-04-02T13:40:32.596 INFO:teuthology.orchestra.run.smithi088.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 468 kB/s | 19 kB 00:00 2024-04-02T13:40:32.762 INFO:teuthology.orchestra.run.smithi052.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 288 kB/s | 67 kB 00:00 2024-04-02T13:40:32.787 INFO:teuthology.orchestra.run.smithi052.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 646 kB/s | 151 kB 00:00 2024-04-02T13:40:32.896 INFO:teuthology.orchestra.run.smithi052.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 5.0 MB/s | 548 kB 00:00 2024-04-02T13:40:32.929 INFO:teuthology.orchestra.run.smithi052.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.9 MB/s | 838 kB 00:00 2024-04-02T13:40:32.955 INFO:teuthology.orchestra.run.smithi052.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 742 kB/s | 43 kB 00:00 2024-04-02T13:40:32.988 INFO:teuthology.orchestra.run.smithi052.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 551 kB/s | 32 kB 00:00 2024-04-02T13:40:33.022 INFO:teuthology.orchestra.run.smithi052.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-04-02T13:40:33.064 INFO:teuthology.orchestra.run.smithi052.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-02T13:40:33.114 INFO:teuthology.orchestra.run.smithi052.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-04-02T13:40:33.181 INFO:teuthology.orchestra.run.smithi052.stdout:(121/174): libarrow-9.0.0-10.el9.x86_64.rpm 5.9 MB/s | 4.4 MB 00:00 2024-04-02T13:40:33.206 INFO:teuthology.orchestra.run.smithi052.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 384 kB/s | 35 kB 00:00 2024-04-02T13:40:33.231 INFO:teuthology.orchestra.run.smithi052.stdout:(123/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.8 MB/s | 303 kB 00:00 2024-04-02T13:40:33.265 INFO:teuthology.orchestra.run.smithi052.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-04-02T13:40:33.307 INFO:teuthology.orchestra.run.smithi052.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-02T13:40:33.332 INFO:teuthology.orchestra.run.smithi052.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 159 kB/s | 11 kB 00:00 2024-04-02T13:40:33.338 INFO:teuthology.orchestra.run.smithi088.stdout:(108/174): libarrow-9.0.0-10.el9.x86_64.rpm 6.0 MB/s | 4.4 MB 00:00 2024-04-02T13:40:33.380 INFO:teuthology.orchestra.run.smithi088.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 627 kB/s | 26 kB 00:00 2024-04-02T13:40:33.391 INFO:teuthology.orchestra.run.smithi052.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-04-02T13:40:33.416 INFO:teuthology.orchestra.run.smithi052.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 162 kB/s | 18 kB 00:00 2024-04-02T13:40:33.430 INFO:teuthology.orchestra.run.smithi088.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 970 kB/s | 49 kB 00:00 2024-04-02T13:40:33.441 INFO:teuthology.orchestra.run.smithi052.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 213 kB/s | 23 kB 00:00 2024-04-02T13:40:33.466 INFO:teuthology.orchestra.run.smithi052.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-04-02T13:40:33.472 INFO:teuthology.orchestra.run.smithi088.stdout:(111/174): protobuf-compiler-3.14.0-13.el9.x86_ 523 kB/s | 863 kB 00:01 2024-04-02T13:40:33.498 INFO:teuthology.orchestra.run.smithi088.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-02T13:40:33.509 INFO:teuthology.orchestra.run.smithi052.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 295 kB/s | 20 kB 00:00 2024-04-02T13:40:33.534 INFO:teuthology.orchestra.run.smithi052.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 692 kB/s | 46 kB 00:00 2024-04-02T13:40:33.567 INFO:teuthology.orchestra.run.smithi052.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-04-02T13:40:33.590 INFO:teuthology.orchestra.run.smithi088.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-02T13:40:33.593 INFO:teuthology.orchestra.run.smithi052.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 940 kB/s | 79 kB 00:00 2024-04-02T13:40:33.618 INFO:teuthology.orchestra.run.smithi052.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 687 kB/s | 58 kB 00:00 2024-04-02T13:40:33.652 INFO:teuthology.orchestra.run.smithi052.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-02T13:40:33.660 INFO:teuthology.orchestra.run.smithi088.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.2 MB/s | 838 kB 00:00 2024-04-02T13:40:33.677 INFO:teuthology.orchestra.run.smithi052.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-04-02T13:40:33.695 INFO:teuthology.orchestra.run.smithi088.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.1 MB/s | 43 kB 00:00 2024-04-02T13:40:33.702 INFO:teuthology.orchestra.run.smithi052.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-04-02T13:40:33.727 INFO:teuthology.orchestra.run.smithi052.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-04-02T13:40:33.737 INFO:teuthology.orchestra.run.smithi088.stdout:(116/174): python3-asyncssh-2.13.2-5.el9.noarch 3.6 MB/s | 548 kB 00:00 2024-04-02T13:40:33.753 INFO:teuthology.orchestra.run.smithi052.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 406 kB/s | 31 kB 00:00 2024-04-02T13:40:33.754 INFO:teuthology.orchestra.run.smithi088.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 550 kB/s | 32 kB 00:00 2024-04-02T13:40:33.771 INFO:teuthology.orchestra.run.smithi088.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 447 kB/s | 15 kB 00:00 2024-04-02T13:40:33.778 INFO:teuthology.orchestra.run.smithi052.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-02T13:40:33.796 INFO:teuthology.orchestra.run.smithi088.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-04-02T13:40:33.803 INFO:teuthology.orchestra.run.smithi052.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 781 kB/s | 59 kB 00:00 2024-04-02T13:40:33.829 INFO:teuthology.orchestra.run.smithi052.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-04-02T13:40:33.854 INFO:teuthology.orchestra.run.smithi052.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 678 kB/s | 51 kB 00:00 2024-04-02T13:40:33.855 INFO:teuthology.orchestra.run.smithi088.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-02T13:40:33.888 INFO:teuthology.orchestra.run.smithi052.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-02T13:40:33.888 INFO:teuthology.orchestra.run.smithi088.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-02T13:40:33.913 INFO:teuthology.orchestra.run.smithi052.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-04-02T13:40:33.938 INFO:teuthology.orchestra.run.smithi052.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-04-02T13:40:33.947 INFO:teuthology.orchestra.run.smithi088.stdout:(122/174): python3-google-auth-2.28.2-1.el9.noa 3.4 MB/s | 203 kB 00:00 2024-04-02T13:40:33.963 INFO:teuthology.orchestra.run.smithi052.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-04-02T13:40:33.972 INFO:teuthology.orchestra.run.smithi088.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 301 kB/s | 35 kB 00:00 2024-04-02T13:40:33.989 INFO:teuthology.orchestra.run.smithi052.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-02T13:40:34.014 INFO:teuthology.orchestra.run.smithi088.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-04-02T13:40:34.057 INFO:teuthology.orchestra.run.smithi088.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 257 kB/s | 11 kB 00:00 2024-04-02T13:40:34.097 INFO:teuthology.orchestra.run.smithi052.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-04-02T13:40:34.098 INFO:teuthology.orchestra.run.smithi088.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 426 kB/s | 18 kB 00:00 2024-04-02T13:40:34.140 INFO:teuthology.orchestra.run.smithi088.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 556 kB/s | 23 kB 00:00 2024-04-02T13:40:34.182 INFO:teuthology.orchestra.run.smithi088.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 469 kB/s | 19 kB 00:00 2024-04-02T13:40:34.224 INFO:teuthology.orchestra.run.smithi088.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 475 kB/s | 20 kB 00:00 2024-04-02T13:40:34.273 INFO:teuthology.orchestra.run.smithi052.stdout:(151/174): NetworkManager-1.47.2-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-04-02T13:40:34.340 INFO:teuthology.orchestra.run.smithi052.stdout:(152/174): NetworkManager-libnm-1.47.2-1.el9.x8 27 MB/s | 1.8 MB 00:00 2024-04-02T13:40:34.365 INFO:teuthology.orchestra.run.smithi052.stdout:(153/174): NetworkManager-team-1.47.2-1.el9.x86 1.6 MB/s | 41 kB 00:00 2024-04-02T13:40:34.407 INFO:teuthology.orchestra.run.smithi052.stdout:(154/174): NetworkManager-tui-1.47.2-1.el9.x86_ 5.8 MB/s | 247 kB 00:00 2024-04-02T13:40:34.441 INFO:teuthology.orchestra.run.smithi088.stdout:(130/174): python3-grpcio-1.46.7-10.el9.x86_64. 4.1 MB/s | 2.0 MB 00:00 2024-04-02T13:40:34.449 INFO:teuthology.orchestra.run.smithi052.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-04-02T13:40:34.516 INFO:teuthology.orchestra.run.smithi052.stdout:(156/174): librbd1-19.0.0-2609.g191d963a.el9.x8 6.0 MB/s | 3.1 MB 00:00 2024-04-02T13:40:34.517 INFO:teuthology.orchestra.run.smithi088.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 613 kB/s | 46 kB 00:00 2024-04-02T13:40:34.541 INFO:teuthology.orchestra.run.smithi052.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 775 kB/s | 71 kB 00:00 2024-04-02T13:40:34.558 INFO:teuthology.orchestra.run.smithi088.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 3.1 MB/s | 1.0 MB 00:00 2024-04-02T13:40:34.584 INFO:teuthology.orchestra.run.smithi088.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-04-02T13:40:34.608 INFO:teuthology.orchestra.run.smithi052.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.8 MB/s | 192 kB 00:00 2024-04-02T13:40:34.609 INFO:teuthology.orchestra.run.smithi088.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-02T13:40:34.633 INFO:teuthology.orchestra.run.smithi052.stdout:(159/174): elfutils-debuginfod-client-0.190-2.e 312 kB/s | 36 kB 00:00 2024-04-02T13:40:34.642 INFO:teuthology.orchestra.run.smithi088.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 496 kB/s | 16 kB 00:00 2024-04-02T13:40:34.667 INFO:teuthology.orchestra.run.smithi052.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 4.3 MB/s | 256 kB 00:00 2024-04-02T13:40:34.668 INFO:teuthology.orchestra.run.smithi088.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-02T13:40:34.684 INFO:teuthology.orchestra.run.smithi088.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-04-02T13:40:34.692 INFO:teuthology.orchestra.run.smithi052.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 392 kB/s | 23 kB 00:00 2024-04-02T13:40:34.701 INFO:teuthology.orchestra.run.smithi088.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 840 kB/s | 28 kB 00:00 2024-04-02T13:40:34.717 INFO:teuthology.orchestra.run.smithi052.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-04-02T13:40:34.718 INFO:teuthology.orchestra.run.smithi088.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 918 kB/s | 31 kB 00:00 2024-04-02T13:40:34.752 INFO:teuthology.orchestra.run.smithi088.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-04-02T13:40:34.769 INFO:teuthology.orchestra.run.smithi088.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-02T13:40:34.785 INFO:teuthology.orchestra.run.smithi052.stdout:(163/174): librados2-19.0.0-2609.g191d963a.el9. 4.1 MB/s | 3.4 MB 00:00 2024-04-02T13:40:34.786 INFO:teuthology.orchestra.run.smithi088.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-04-02T13:40:34.811 INFO:teuthology.orchestra.run.smithi088.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 1.2 MB/s | 51 kB 00:00 2024-04-02T13:40:34.818 INFO:teuthology.orchestra.run.smithi052.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 155 kB 00:00 2024-04-02T13:40:34.843 INFO:teuthology.orchestra.run.smithi052.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 136 kB 00:00 2024-04-02T13:40:34.853 INFO:teuthology.orchestra.run.smithi088.stdout:(144/174): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-04-02T13:40:34.869 INFO:teuthology.orchestra.run.smithi052.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 572 kB/s | 29 kB 00:00 2024-04-02T13:40:34.878 INFO:teuthology.orchestra.run.smithi088.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-04-02T13:40:34.911 INFO:teuthology.orchestra.run.smithi052.stdout:(167/174): libsmartcols-2.37.4-18.el9.x86_64.rp 508 kB/s | 64 kB 00:00 2024-04-02T13:40:34.912 INFO:teuthology.orchestra.run.smithi088.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 600 kB/s | 20 kB 00:00 2024-04-02T13:40:34.952 INFO:teuthology.orchestra.run.smithi052.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 11 MB/s | 1.2 MB 00:00 2024-04-02T13:40:34.954 INFO:teuthology.orchestra.run.smithi088.stdout:(147/174): re2-20211101-3.el9.x86_64.rpm 4.5 MB/s | 192 kB 00:00 2024-04-02T13:40:34.988 INFO:teuthology.orchestra.run.smithi088.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-04-02T13:40:35.003 INFO:teuthology.orchestra.run.smithi052.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 9.1 MB/s | 465 kB 00:00 2024-04-02T13:40:35.053 INFO:teuthology.orchestra.run.smithi052.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 622 kB/s | 31 kB 00:00 2024-04-02T13:40:35.120 INFO:teuthology.orchestra.run.smithi052.stdout:(171/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 16 MB/s | 1.1 MB 00:00 2024-04-02T13:40:35.162 INFO:teuthology.orchestra.run.smithi052.stdout:(172/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 7.4 MB/s | 2.2 MB 00:00 2024-04-02T13:40:35.213 INFO:teuthology.orchestra.run.smithi052.stdout:(173/174): util-linux-2.37.4-18.el9.x86_64.rpm 7.6 MB/s | 2.3 MB 00:00 2024-04-02T13:40:35.238 INFO:teuthology.orchestra.run.smithi052.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 635 kB/s | 74 kB 00:00 2024-04-02T13:40:35.245 INFO:teuthology.orchestra.run.smithi052.stdout:-------------------------------------------------------------------------------- 2024-04-02T13:40:35.245 INFO:teuthology.orchestra.run.smithi052.stdout:Total 15 MB/s | 226 MB 00:15 2024-04-02T13:40:35.288 INFO:teuthology.orchestra.run.smithi088.stdout:(149/174): librados2-19.0.0-2609.g191d963a.el9. 11 MB/s | 3.4 MB 00:00 2024-04-02T13:40:35.321 INFO:teuthology.orchestra.run.smithi088.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 4.3 MB/s | 1.6 MB 00:00 2024-04-02T13:40:35.422 INFO:teuthology.orchestra.run.smithi088.stdout:(151/174): NetworkManager-1.47.2-1.el9.x86_64.r 23 MB/s | 2.3 MB 00:00 2024-04-02T13:40:35.497 INFO:teuthology.orchestra.run.smithi088.stdout:(152/174): NetworkManager-libnm-1.47.2-1.el9.x8 24 MB/s | 1.8 MB 00:00 2024-04-02T13:40:35.522 INFO:teuthology.orchestra.run.smithi088.stdout:(153/174): NetworkManager-team-1.47.2-1.el9.x86 1.6 MB/s | 41 kB 00:00 2024-04-02T13:40:35.572 INFO:teuthology.orchestra.run.smithi088.stdout:(154/174): librbd1-19.0.0-2609.g191d963a.el9.x8 11 MB/s | 3.1 MB 00:00 2024-04-02T13:40:35.598 INFO:teuthology.orchestra.run.smithi088.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 3.2 MB/s | 247 kB 00:00 2024-04-02T13:40:35.623 INFO:teuthology.orchestra.run.smithi088.stdout:(156/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-04-02T13:40:35.630 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction check 2024-04-02T13:40:35.927 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction check succeeded. 2024-04-02T13:40:35.927 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction test 2024-04-02T13:40:35.957 INFO:teuthology.orchestra.run.smithi088.stdout:(157/174): openssl-devel-3.0.7-27.el9.x86_64.rp 711 kB/s | 4.1 MB 00:05 2024-04-02T13:40:36.249 INFO:teuthology.orchestra.run.smithi088.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 58 kB/s | 36 kB 00:00 2024-04-02T13:40:36.307 INFO:teuthology.orchestra.run.smithi088.stdout:(159/174): acl-2.3.1-4.el9.x86_64.rpm 101 kB/s | 71 kB 00:00 2024-04-02T13:40:36.782 INFO:teuthology.orchestra.run.smithi088.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 233 kB/s | 192 kB 00:00 2024-04-02T13:40:36.891 INFO:teuthology.orchestra.run.smithi088.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 39 kB/s | 23 kB 00:00 2024-04-02T13:40:37.183 INFO:teuthology.orchestra.run.smithi088.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 274 kB/s | 256 kB 00:00 2024-04-02T13:40:37.550 INFO:teuthology.orchestra.run.smithi088.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 141 kB/s | 108 kB 00:00 2024-04-02T13:40:37.708 INFO:teuthology.orchestra.run.smithi088.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 189 kB/s | 155 kB 00:00 2024-04-02T13:40:37.934 INFO:teuthology.orchestra.run.smithi088.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 181 kB/s | 136 kB 00:00 2024-04-02T13:40:38.066 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction test succeeded. 2024-04-02T13:40:38.066 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction 2024-04-02T13:40:38.182 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-04-02T13:40:38.242 INFO:teuthology.orchestra.run.smithi088.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 92 kB/s | 64 kB 00:00 2024-04-02T13:40:38.290 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-02T13:40:38.326 INFO:teuthology.orchestra.run.smithi088.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 46 kB/s | 29 kB 00:00 2024-04-02T13:40:38.333 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-02T13:40:38.364 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T13:40:38.374 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T13:40:38.421 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-02T13:40:38.475 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-02T13:40:38.509 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-02T13:40:38.554 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-02T13:40:38.593 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-02T13:40:38.915 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-02T13:40:38.977 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:40:39.099 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:40:39.129 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:40:39.219 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:40:39.251 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T13:40:39.290 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-02T13:40:39.477 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-02T13:40:39.524 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-02T13:40:39.552 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:40:39.639 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:40:39.661 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T13:40:39.725 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T13:40:39.756 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-02T13:40:39.796 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T13:40:39.812 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T13:40:40.004 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T13:40:40.035 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T13:40:40.059 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-02T13:40:40.092 INFO:teuthology.orchestra.run.smithi088.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 574 kB/s | 1.2 MB 00:02 2024-04-02T13:40:40.155 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-02T13:40:40.227 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T13:40:40.271 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T13:40:40.343 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:40:40.397 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:40:40.611 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:40:40.725 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-02T13:40:40.816 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-02T13:40:40.860 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-02T13:40:40.896 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-02T13:40:40.939 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-02T13:40:41.015 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-02T13:40:41.068 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-02T13:40:41.105 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-02T13:40:41.171 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-02T13:40:41.204 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-02T13:40:41.248 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-02T13:40:41.280 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-02T13:40:41.308 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-02T13:40:41.309 INFO:teuthology.orchestra.run.smithi088.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 383 kB/s | 465 kB 00:01 2024-04-02T13:40:41.380 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-02T13:40:41.430 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 39/198 2024-04-02T13:40:41.473 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 40/198 2024-04-02T13:40:41.502 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 41/198 2024-04-02T13:40:41.570 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-02T13:40:41.616 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-02T13:40:41.642 INFO:teuthology.orchestra.run.smithi088.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 649 kB/s | 2.2 MB 00:03 2024-04-02T13:40:41.689 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-02T13:40:41.825 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-02T13:40:41.868 INFO:teuthology.orchestra.run.smithi088.stdout:(171/174): util-linux-2.37.4-18.el9.x86_64.rpm 660 kB/s | 2.3 MB 00:03 2024-04-02T13:40:41.877 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-02T13:40:41.918 INFO:teuthology.orchestra.run.smithi088.stdout:(172/174): util-linux-user-2.37.4-18.el9.x86_64 51 kB/s | 31 kB 00:00 2024-04-02T13:40:42.410 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-02T13:40:42.489 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-02T13:40:42.565 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-02T13:40:42.568 INFO:teuthology.orchestra.run.smithi088.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 110 kB/s | 74 kB 00:00 2024-04-02T13:40:42.609 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T13:40:42.618 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T13:40:42.648 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-02T13:40:42.689 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-02T13:40:42.727 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 53/198 2024-04-02T13:40:42.757 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-02T13:40:42.992 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-02T13:40:43.160 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-02T13:40:43.234 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-02T13:40:43.278 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-02T13:40:43.408 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-02T13:40:43.464 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-02T13:40:43.518 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-02T13:40:43.592 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-02T13:40:43.623 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-02T13:40:43.643 INFO:teuthology.orchestra.run.smithi088.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 550 kB/s | 1.1 MB 00:01 2024-04-02T13:40:43.649 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-04-02T13:40:43.649 INFO:teuthology.orchestra.run.smithi088.stdout:Total 8.0 MB/s | 226 MB 00:28 2024-04-02T13:40:43.659 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-02T13:40:43.706 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-02T13:40:43.742 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-02T13:40:43.785 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-02T13:40:43.828 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-02T13:40:43.962 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-02T13:40:44.033 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-04-02T13:40:44.250 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-02T13:40:44.279 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-02T13:40:44.305 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-04-02T13:40:44.306 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-04-02T13:40:44.321 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-02T13:40:44.419 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-02T13:40:44.461 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-02T13:40:44.501 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-02T13:40:44.527 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-02T13:40:44.551 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-02T13:40:44.625 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-02T13:40:44.753 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-02T13:40:44.860 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-02T13:40:44.913 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-02T13:40:44.960 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-02T13:40:45.024 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-02T13:40:45.071 INFO:teuthology.orchestra.run.smithi052.stdout: Preparing : 1/1 2024-04-02T13:40:45.151 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-02T13:40:45.194 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-02T13:40:45.207 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-02T13:40:45.238 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T13:40:45.248 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T13:40:45.255 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-02T13:40:45.304 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-02T13:40:45.370 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-02T13:40:45.384 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-02T13:40:45.418 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-02T13:40:45.420 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-02T13:40:45.452 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-02T13:40:45.452 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-02T13:40:45.492 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T13:40:45.519 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-02T13:40:45.561 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-02T13:40:45.602 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-02T13:40:45.633 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-02T13:40:45.659 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-02T13:40:45.662 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-02T13:40:45.739 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-02T13:40:45.781 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-02T13:40:45.814 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-02T13:40:45.856 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-02T13:40:45.890 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-02T13:40:45.979 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-02T13:40:46.014 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-04-02T13:40:46.015 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-04-02T13:40:46.086 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-02T13:40:46.095 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-02T13:40:46.127 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:40:46.138 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-02T13:40:46.177 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-02T13:40:46.224 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-02T13:40:46.236 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:40:46.262 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:40:46.322 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-02T13:40:46.346 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:40:46.374 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T13:40:46.387 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-02T13:40:46.414 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-02T13:40:46.598 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-02T13:40:46.656 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-02T13:40:46.684 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:40:46.789 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:40:46.818 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T13:40:46.866 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T13:40:46.914 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-02T13:40:46.936 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T13:40:46.952 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T13:40:47.121 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-02T13:40:47.175 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-02T13:40:47.209 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-02T13:40:47.258 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-02T13:40:47.298 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T13:40:47.337 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T13:40:47.345 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-02T13:40:47.376 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-02T13:40:47.377 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-02T13:40:47.555 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-02T13:40:47.583 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-02T13:40:47.653 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-02T13:40:47.660 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-02T13:40:47.903 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T13:40:47.964 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T13:40:48.077 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:40:48.132 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:40:48.309 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-02T13:40:48.350 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:40:48.439 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-02T13:40:48.486 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-02T13:40:48.587 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-02T13:40:48.620 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-02T13:40:48.647 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-02T13:40:48.692 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-02T13:40:48.769 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-02T13:40:48.938 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-02T13:40:48.976 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-02T13:40:49.051 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-02T13:40:49.090 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-02T13:40:49.134 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-02T13:40:49.167 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-02T13:40:49.196 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-02T13:40:49.269 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-02T13:40:49.301 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 39/198 2024-04-02T13:40:49.345 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 40/198 2024-04-02T13:40:49.381 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 41/198 2024-04-02T13:40:49.455 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-02T13:40:49.496 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-02T13:40:49.573 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-02T13:40:49.666 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-02T13:40:49.711 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-02T13:40:49.749 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-02T13:40:49.775 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-02T13:40:49.780 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-02T13:40:49.811 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-02T13:40:49.838 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-02T13:40:50.154 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-02T13:40:50.185 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T13:40:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T13:40:50.307 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 125/198 2024-04-02T13:40:50.332 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-02T13:40:50.343 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-02T13:40:50.414 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-02T13:40:50.501 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-02T13:40:50.546 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T13:40:50.557 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T13:40:50.587 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-02T13:40:50.644 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-02T13:40:50.682 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 53/198 2024-04-02T13:40:50.712 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-02T13:40:50.804 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-02T13:40:50.838 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T13:40:50.893 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T13:40:50.915 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 129/198 2024-04-02T13:40:51.100 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-02T13:40:51.325 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-02T13:40:51.406 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-02T13:40:51.450 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-02T13:40:51.595 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-02T13:40:51.636 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-02T13:40:51.701 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-02T13:40:51.783 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-02T13:40:51.819 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-02T13:40:51.856 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-02T13:40:51.904 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-02T13:40:51.938 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-02T13:40:51.982 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-02T13:40:52.026 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-02T13:40:52.173 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-02T13:40:52.221 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-02T13:40:52.259 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-02T13:40:52.301 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-02T13:40:52.400 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-02T13:40:52.442 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-02T13:40:52.483 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-02T13:40:52.508 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-02T13:40:52.533 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-02T13:40:52.641 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-02T13:40:52.774 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-02T13:40:52.833 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-04-02T13:40:52.898 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-02T13:40:52.946 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-02T13:40:52.950 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-02T13:40:52.980 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-02T13:40:53.017 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-02T13:40:53.019 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T13:40:53.029 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T13:40:53.077 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-02T13:40:53.086 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-02T13:40:53.131 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-02T13:40:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:40:53.165 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-02T13:40:53.177 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:40:53.190 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-02T13:40:53.232 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-02T13:40:53.281 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-02T13:40:53.329 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-02T13:40:53.482 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-02T13:40:53.527 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-02T13:40:53.552 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-02T13:40:53.558 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-02T13:40:53.589 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T13:40:53.608 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:40:53.617 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-02T13:40:53.652 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-02T13:40:53.691 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-02T13:40:53.718 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:40:53.720 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:40:53.722 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-02T13:40:53.747 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T13:40:53.750 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:40:53.752 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-02T13:40:53.828 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T13:40:53.828 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-02T13:40:53.828 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-02T13:40:53.828 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-02T13:40:53.840 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-02T13:40:53.841 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:40:53.857 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T13:40:53.871 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T13:40:53.879 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-02T13:40:53.907 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-02T13:40:53.913 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-02T13:40:53.954 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-02T13:40:53.988 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-02T13:40:54.079 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-02T13:40:54.081 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-02T13:40:54.127 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-02T13:40:54.155 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:40:54.171 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-02T13:40:54.229 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-02T13:40:54.259 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:40:54.268 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-02T13:40:54.281 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T13:40:54.315 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-02T13:40:54.346 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T13:40:54.385 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-02T13:40:54.400 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T13:40:54.416 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T13:40:54.418 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-02T13:40:54.470 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-02T13:40:54.600 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T13:40:54.630 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T13:40:54.662 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-02T13:40:54.758 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-02T13:40:54.848 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T13:40:54.908 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T13:40:54.981 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:40:55.042 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:40:55.240 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-02T13:40:55.254 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:40:55.289 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-02T13:40:55.322 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-02T13:40:55.364 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-02T13:40:55.372 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-02T13:40:55.458 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-02T13:40:55.462 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-02T13:40:55.497 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-02T13:40:55.506 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-02T13:40:55.523 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-02T13:40:55.559 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-02T13:40:55.633 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-02T13:40:55.693 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-02T13:40:55.696 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-02T13:40:55.734 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-02T13:40:55.737 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-02T13:40:55.798 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-02T13:40:55.818 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-02T13:40:55.833 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-02T13:40:55.877 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-02T13:40:55.909 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-02T13:40:55.937 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-02T13:40:56.007 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-02T13:40:56.043 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 39/198 2024-04-02T13:40:56.085 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 40/198 2024-04-02T13:40:56.114 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 41/198 2024-04-02T13:40:56.182 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-02T13:40:56.228 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-02T13:40:56.301 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-02T13:40:56.445 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-02T13:40:56.497 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-02T13:40:56.507 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-02T13:40:56.631 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-02T13:40:57.016 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-02T13:40:57.092 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-02T13:40:57.179 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-02T13:40:57.221 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T13:40:57.230 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T13:40:57.261 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-02T13:40:57.302 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-02T13:40:57.339 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 53/198 2024-04-02T13:40:57.369 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-02T13:40:57.615 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-02T13:40:57.770 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-02T13:40:57.838 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-02T13:40:57.882 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-02T13:40:57.928 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-02T13:40:58.013 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-02T13:40:58.020 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-02T13:40:58.059 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-02T13:40:58.068 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-02T13:40:58.091 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-02T13:40:58.118 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-02T13:40:58.123 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-02T13:40:58.196 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-02T13:40:58.227 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-02T13:40:58.263 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-02T13:40:58.310 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-02T13:40:58.355 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-02T13:40:58.398 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-02T13:40:58.448 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-02T13:40:58.448 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-02T13:40:58.481 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T13:40:58.573 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T13:40:58.590 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-02T13:40:58.602 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 125/198 2024-04-02T13:40:58.637 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-02T13:40:58.640 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-02T13:40:58.667 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-02T13:40:58.708 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-02T13:40:58.802 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-02T13:40:58.833 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-02T13:40:58.873 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-02T13:40:58.907 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-02T13:40:58.932 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-02T13:40:59.003 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-02T13:40:59.127 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-02T13:40:59.131 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-02T13:40:59.159 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T13:40:59.230 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T13:40:59.239 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-02T13:40:59.252 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 129/198 2024-04-02T13:40:59.282 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-02T13:40:59.329 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-02T13:40:59.385 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-02T13:40:59.576 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-02T13:40:59.627 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-02T13:40:59.757 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-02T13:40:59.792 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-02T13:40:59.824 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-02T13:40:59.863 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T13:40:59.891 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-02T13:40:59.926 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-02T13:40:59.964 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-02T13:40:59.996 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-02T13:41:00.026 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-02T13:41:00.102 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-02T13:41:00.136 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-02T13:41:00.170 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-02T13:41:00.211 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-02T13:41:00.254 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-02T13:41:00.343 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-02T13:41:00.432 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-02T13:41:00.476 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-02T13:41:00.508 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-02T13:41:00.555 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-02T13:41:00.650 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-02T13:41:00.701 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-02T13:41:01.418 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-02T13:41:01.456 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-02T13:41:01.490 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-02T13:41:01.535 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:41:01.538 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-02T13:41:01.565 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:41:01.616 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-02T13:41:01.649 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-02T13:41:01.819 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-02T13:41:01.847 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-02T13:41:01.924 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-02T13:41:02.132 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:41:02.168 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T13:41:02.249 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T13:41:02.250 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-02T13:41:02.250 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-02T13:41:02.250 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:41:02.286 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T13:41:02.559 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-02T13:41:02.685 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-02T13:41:03.892 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-02T13:41:03.964 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-02T13:41:03.994 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-02T13:41:04.026 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-02T13:41:04.053 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-02T13:41:04.368 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-02T13:41:04.400 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T13:41:04.490 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T13:41:04.521 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 125/198 2024-04-02T13:41:04.557 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-02T13:41:05.018 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-02T13:41:05.045 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T13:41:05.115 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T13:41:05.138 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 129/198 2024-04-02T13:41:05.751 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T13:41:05.752 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /sys 2024-04-02T13:41:05.752 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /proc 2024-04-02T13:41:05.752 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /mnt 2024-04-02T13:41:05.752 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /var/tmp 2024-04-02T13:41:05.752 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /home 2024-04-02T13:41:05.752 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /root 2024-04-02T13:41:05.752 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /tmp 2024-04-02T13:41:05.752 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-02T13:41:05.925 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T13:41:05.941 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T13:41:05.941 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:05.941 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-02T13:41:05.941 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-02T13:41:07.327 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:41:07.367 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:41:07.421 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T13:41:07.447 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T13:41:07.447 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:07.447 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-02T13:41:07.447 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T13:41:07.447 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T13:41:07.447 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-02T13:41:07.672 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T13:41:07.694 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T13:41:07.695 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:07.695 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-02T13:41:07.695 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T13:41:07.695 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T13:41:07.695 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-02T13:41:07.903 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:41:07.989 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T13:41:08.139 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T13:41:08.162 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T13:41:08.162 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:08.162 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-02T13:41:08.162 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T13:41:08.162 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T13:41:08.162 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-02T13:41:08.250 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-02T13:41:08.251 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T13:41:08.251 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-02T13:41:08.251 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-02T13:41:08.252 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-02T13:41:08.472 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T13:41:08.634 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T13:41:08.664 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T13:41:08.696 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 139/198 2024-04-02T13:41:08.757 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 140/198 2024-04-02T13:41:08.787 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T13:41:09.832 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T13:41:09.872 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T13:41:10.843 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T13:41:10.873 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T13:41:10.974 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T13:41:11.137 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 144/198 2024-04-02T13:41:11.166 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T13:41:11.190 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T13:41:11.190 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:11.190 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-02T13:41:11.190 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T13:41:11.190 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T13:41:11.191 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-02T13:41:11.234 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T13:41:11.245 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T13:41:11.290 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 147/198 2024-04-02T13:41:12.166 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-02T13:41:12.194 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T13:41:12.217 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T13:41:12.217 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:12.217 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-02T13:41:12.217 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T13:41:12.217 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T13:41:12.217 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-02T13:41:12.262 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T13:41:12.283 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T13:41:12.283 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:12.283 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-02T13:41:12.283 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-02T13:41:12.337 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:41:12.359 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:41:12.360 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:12.360 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-02T13:41:12.360 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T13:41:12.360 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T13:41:12.360 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-02T13:41:12.671 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T13:41:12.693 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T13:41:12.693 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:12.694 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-02T13:41:12.694 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T13:41:12.694 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T13:41:12.694 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-02T13:41:14.117 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T13:41:14.117 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /sys 2024-04-02T13:41:14.117 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /proc 2024-04-02T13:41:14.117 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /mnt 2024-04-02T13:41:14.118 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /var/tmp 2024-04-02T13:41:14.118 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /home 2024-04-02T13:41:14.118 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /root 2024-04-02T13:41:14.118 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /tmp 2024-04-02T13:41:14.118 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:41:14.323 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T13:41:14.340 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T13:41:14.341 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:14.341 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-02T13:41:14.341 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:41:15.876 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T13:41:15.901 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T13:41:15.901 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:15.901 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-02T13:41:15.901 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T13:41:15.901 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T13:41:15.901 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:41:16.135 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T13:41:16.157 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T13:41:16.157 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:16.157 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-02T13:41:16.157 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T13:41:16.157 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T13:41:16.158 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:41:16.619 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T13:41:16.641 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T13:41:16.642 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:16.642 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-02T13:41:16.642 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T13:41:16.642 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T13:41:16.642 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:41:16.737 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-02T13:41:17.063 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T13:41:17.094 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T13:41:17.125 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T13:41:17.126 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 139/198 2024-04-02T13:41:17.166 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 154/198 2024-04-02T13:41:17.189 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 140/198 2024-04-02T13:41:17.196 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 155/198 2024-04-02T13:41:17.224 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-02T13:41:17.225 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T13:41:17.402 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 157/198 2024-04-02T13:41:17.514 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-02T13:41:17.554 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 159/198 2024-04-02T13:41:17.596 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-02T13:41:17.648 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-02T13:41:17.692 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-02T13:41:17.724 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 163/198 2024-04-02T13:41:17.756 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-02T13:41:17.786 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-04-02T13:41:17.832 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-04-02T13:41:17.883 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-02T13:41:17.917 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-02T13:41:18.323 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T13:41:18.352 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T13:41:18.497 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:41:18.532 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:41:18.552 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:41:19.369 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T13:41:19.381 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 170/198 2024-04-02T13:41:19.394 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T13:41:19.468 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-02T13:41:19.499 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T13:41:19.509 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-02T13:41:19.591 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-02T13:41:19.630 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-02T13:41:19.655 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-02T13:41:19.664 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 144/198 2024-04-02T13:41:19.681 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-02T13:41:19.696 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T13:41:19.707 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-02T13:41:19.707 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T13:41:19.719 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T13:41:19.719 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:19.719 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-02T13:41:19.719 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T13:41:19.719 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T13:41:19.719 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:41:19.763 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T13:41:19.774 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T13:41:19.778 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T13:41:19.808 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-02T13:41:19.808 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T13:41:19.830 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 147/198 2024-04-02T13:41:19.892 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T13:41:19.919 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-02T13:41:19.951 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-02T13:41:19.977 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-02T13:41:20.002 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-02T13:41:20.035 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-02T13:41:20.075 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-02T13:41:20.107 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-02T13:41:20.132 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-02T13:41:20.153 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T13:41:20.153 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /sys 2024-04-02T13:41:20.154 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /proc 2024-04-02T13:41:20.154 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /mnt 2024-04-02T13:41:20.154 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /var/tmp 2024-04-02T13:41:20.154 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /home 2024-04-02T13:41:20.154 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /root 2024-04-02T13:41:20.154 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /tmp 2024-04-02T13:41:20.154 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-02T13:41:20.157 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-02T13:41:20.182 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-02T13:41:20.208 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-02T13:41:20.214 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:41:20.214 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:41:20.325 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T13:41:20.341 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T13:41:20.341 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:20.341 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-02T13:41:20.341 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-02T13:41:20.394 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:41:20.394 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T13:41:20.477 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T13:41:20.508 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-02T13:41:20.535 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-02T13:41:20.561 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-02T13:41:20.586 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-02T13:41:20.649 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T13:41:20.759 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-02T13:41:20.798 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T13:41:20.822 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T13:41:20.822 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:20.822 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-02T13:41:20.822 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T13:41:20.822 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T13:41:20.822 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:41:20.866 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T13:41:20.888 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T13:41:20.888 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:20.888 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-02T13:41:20.888 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:41:20.933 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:41:20.955 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:41:20.956 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:20.956 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-02T13:41:20.956 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T13:41:20.956 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T13:41:20.956 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:41:21.275 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T13:41:21.298 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T13:41:21.298 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:21.298 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-02T13:41:21.298 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T13:41:21.298 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T13:41:21.298 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:41:21.820 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T13:41:21.846 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T13:41:21.847 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:21.847 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-02T13:41:21.847 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T13:41:21.847 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T13:41:21.847 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-02T13:41:22.070 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T13:41:22.093 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T13:41:22.093 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:22.093 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-02T13:41:22.094 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T13:41:22.094 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T13:41:22.094 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-02T13:41:22.537 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T13:41:22.561 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T13:41:22.561 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:22.561 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-02T13:41:22.561 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T13:41:22.561 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T13:41:22.561 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-02T13:41:22.650 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-02T13:41:23.034 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T13:41:23.065 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T13:41:23.097 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 139/198 2024-04-02T13:41:23.158 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 140/198 2024-04-02T13:41:23.188 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T13:41:24.225 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T13:41:24.256 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T13:41:25.225 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T13:41:25.248 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T13:41:25.348 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T13:41:25.511 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 144/198 2024-04-02T13:41:25.542 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T13:41:25.564 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T13:41:25.565 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:25.565 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-02T13:41:25.565 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T13:41:25.565 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T13:41:25.565 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-02T13:41:25.609 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T13:41:25.620 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T13:41:25.665 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 147/198 2024-04-02T13:41:26.038 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T13:41:26.049 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T13:41:26.049 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 1/198 2024-04-02T13:41:26.049 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2/198 2024-04-02T13:41:26.049 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 3/198 2024-04-02T13:41:26.050 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 4/198 2024-04-02T13:41:26.050 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 5/198 2024-04-02T13:41:26.050 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 6/198 2024-04-02T13:41:26.050 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 7/198 2024-04-02T13:41:26.050 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 8/198 2024-04-02T13:41:26.050 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 9/198 2024-04-02T13:41:26.050 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:41:26.051 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:41:26.051 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T13:41:26.051 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 13/198 2024-04-02T13:41:26.051 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 14/198 2024-04-02T13:41:26.051 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 15/198 2024-04-02T13:41:26.051 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:41:26.052 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 17/198 2024-04-02T13:41:26.052 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 18/198 2024-04-02T13:41:26.052 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 19/198 2024-04-02T13:41:26.052 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 20/198 2024-04-02T13:41:26.052 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 21/198 2024-04-02T13:41:26.052 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 22/198 2024-04-02T13:41:26.052 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 23/198 2024-04-02T13:41:26.052 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 24/198 2024-04-02T13:41:26.053 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 25/198 2024-04-02T13:41:26.053 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 26/198 2024-04-02T13:41:26.053 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 27/198 2024-04-02T13:41:26.053 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 28/198 2024-04-02T13:41:26.053 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 29/198 2024-04-02T13:41:26.053 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 30/198 2024-04-02T13:41:26.053 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 31/198 2024-04-02T13:41:26.054 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 32/198 2024-04-02T13:41:26.054 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 33/198 2024-04-02T13:41:26.054 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 34/198 2024-04-02T13:41:26.054 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 35/198 2024-04-02T13:41:26.054 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 36/198 2024-04-02T13:41:26.054 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 37/198 2024-04-02T13:41:26.054 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 38/198 2024-04-02T13:41:26.055 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 39/198 2024-04-02T13:41:26.055 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-02T13:41:26.055 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-02T13:41:26.055 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-02T13:41:26.055 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-02T13:41:26.055 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-02T13:41:26.055 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-02T13:41:26.055 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-02T13:41:26.056 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-02T13:41:26.056 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-02T13:41:26.056 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-02T13:41:26.056 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-02T13:41:26.056 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-02T13:41:26.056 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-02T13:41:26.056 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-02T13:41:26.056 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-02T13:41:26.057 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-02T13:41:26.057 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-02T13:41:26.057 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-02T13:41:26.057 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-02T13:41:26.057 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-02T13:41:26.057 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-02T13:41:26.057 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-02T13:41:26.057 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-02T13:41:26.058 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-02T13:41:26.058 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-02T13:41:26.058 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-02T13:41:26.058 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-02T13:41:26.058 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-02T13:41:26.058 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-02T13:41:26.058 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-02T13:41:26.058 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-02T13:41:26.059 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-02T13:41:26.059 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-02T13:41:26.059 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-02T13:41:26.059 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-02T13:41:26.059 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-02T13:41:26.059 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-02T13:41:26.059 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-02T13:41:26.059 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-02T13:41:26.060 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-02T13:41:26.060 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-02T13:41:26.060 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-02T13:41:26.060 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-02T13:41:26.060 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-02T13:41:26.060 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-02T13:41:26.060 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-02T13:41:26.060 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-02T13:41:26.061 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-02T13:41:26.061 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-02T13:41:26.061 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T13:41:26.061 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-02T13:41:26.061 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-02T13:41:26.061 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-02T13:41:26.061 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-02T13:41:26.061 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-02T13:41:26.061 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-02T13:41:26.061 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-02T13:41:26.061 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-02T13:41:26.061 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-02T13:41:26.062 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-02T13:41:26.062 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-02T13:41:26.062 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-02T13:41:26.062 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-02T13:41:26.062 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-02T13:41:26.062 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-02T13:41:26.062 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-02T13:41:26.062 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-02T13:41:26.062 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-02T13:41:26.062 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-02T13:41:26.063 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-02T13:41:26.063 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-02T13:41:26.063 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-02T13:41:26.063 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-02T13:41:26.063 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-02T13:41:26.063 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-02T13:41:26.063 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-02T13:41:26.063 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-02T13:41:26.063 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-02T13:41:26.063 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-02T13:41:26.063 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-02T13:41:26.063 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-02T13:41:26.063 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-02T13:41:26.064 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-02T13:41:26.064 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-02T13:41:26.064 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-02T13:41:26.064 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-02T13:41:26.064 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-02T13:41:26.064 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-02T13:41:26.064 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-02T13:41:26.064 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-02T13:41:26.064 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-02T13:41:26.064 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-02T13:41:26.064 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-02T13:41:26.064 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-02T13:41:26.064 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-02T13:41:26.065 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-02T13:41:26.065 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-02T13:41:26.065 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-02T13:41:26.065 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-02T13:41:26.065 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-02T13:41:26.065 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-02T13:41:26.065 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-02T13:41:26.065 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-02T13:41:26.065 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-02T13:41:26.065 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-02T13:41:26.065 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-02T13:41:26.065 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-02T13:41:26.065 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-02T13:41:26.065 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-02T13:41:26.066 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-02T13:41:26.066 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-02T13:41:26.066 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:41:26.066 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-02T13:41:26.066 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T13:41:26.066 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-02T13:41:26.066 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-04-02T13:41:26.066 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-02T13:41:26.066 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-04-02T13:41:26.066 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-02T13:41:26.066 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-04-02T13:41:26.066 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-02T13:41:26.067 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-04-02T13:41:26.067 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-02T13:41:26.067 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-02T13:41:26.067 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-02T13:41:26.067 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-02T13:41:26.067 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-02T13:41:26.067 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-02T13:41:26.067 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-02T13:41:26.067 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-02T13:41:26.067 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-02T13:41:26.067 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-02T13:41:26.067 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-02T13:41:26.067 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-02T13:41:26.067 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-02T13:41:26.068 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-02T13:41:26.068 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-02T13:41:26.068 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-02T13:41:26.068 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-02T13:41:26.068 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-02T13:41:26.068 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-02T13:41:26.068 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-02T13:41:26.068 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-02T13:41:26.068 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-02T13:41:26.068 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-02T13:41:26.068 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-02T13:41:26.068 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-02T13:41:26.068 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-02T13:41:26.069 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-02T13:41:26.069 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-02T13:41:26.069 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-02T13:41:26.069 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-02T13:41:26.069 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-02T13:41:26.069 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-02T13:41:26.069 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-02T13:41:26.069 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-02T13:41:26.069 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-02T13:41:26.069 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-02T13:41:26.080 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 154/198 2024-04-02T13:41:26.110 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 155/198 2024-04-02T13:41:26.137 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-02T13:41:26.336 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 157/198 2024-04-02T13:41:26.457 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-02T13:41:26.502 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 159/198 2024-04-02T13:41:26.533 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-02T13:41:26.552 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-02T13:41:26.569 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T13:41:26.592 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T13:41:26.592 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:26.592 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-02T13:41:26.592 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T13:41:26.593 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T13:41:26.593 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-02T13:41:26.610 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-02T13:41:26.637 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T13:41:26.644 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-02T13:41:26.659 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T13:41:26.659 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:26.659 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-02T13:41:26.659 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-02T13:41:26.677 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 163/198 2024-04-02T13:41:26.704 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:41:26.718 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-02T13:41:26.726 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:41:26.727 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:26.727 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-02T13:41:26.727 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T13:41:26.727 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T13:41:26.727 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-02T13:41:26.764 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-04-02T13:41:26.834 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-04-02T13:41:26.893 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-02T13:41:26.929 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-02T13:41:27.030 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T13:41:27.052 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T13:41:27.053 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:41:27.053 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-02T13:41:27.053 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T13:41:27.053 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T13:41:27.053 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-02T13:41:27.556 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:41:27.593 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:41:27.612 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:41:28.498 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 170/198 2024-04-02T13:41:28.579 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-02T13:41:28.630 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-02T13:41:28.732 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-02T13:41:28.768 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-02T13:41:28.793 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-02T13:41:28.818 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-02T13:41:28.845 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-02T13:41:28.845 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T13:41:28.916 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T13:41:28.945 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-02T13:41:28.946 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T13:41:29.014 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T13:41:29.048 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-02T13:41:29.081 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-02T13:41:29.107 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-02T13:41:29.112 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-02T13:41:29.112 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-02T13:41:29.112 INFO:teuthology.orchestra.run.smithi189.stdout:Upgraded: 2024-04-02T13:41:29.112 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-04-02T13:41:29.112 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-04-02T13:41:29.113 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-04-02T13:41:29.113 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-04-02T13:41:29.113 INFO:teuthology.orchestra.run.smithi189.stdout: acl-2.3.1-4.el9.x86_64 2024-04-02T13:41:29.113 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-02T13:41:29.113 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-02T13:41:29.113 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-02T13:41:29.113 INFO:teuthology.orchestra.run.smithi189.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-02T13:41:29.113 INFO:teuthology.orchestra.run.smithi189.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-02T13:41:29.114 INFO:teuthology.orchestra.run.smithi189.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-02T13:41:29.114 INFO:teuthology.orchestra.run.smithi189.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-02T13:41:29.114 INFO:teuthology.orchestra.run.smithi189.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-02T13:41:29.114 INFO:teuthology.orchestra.run.smithi189.stdout: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.114 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.114 INFO:teuthology.orchestra.run.smithi189.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-02T13:41:29.114 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-02T13:41:29.114 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-02T13:41:29.115 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-02T13:41:29.115 INFO:teuthology.orchestra.run.smithi189.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-02T13:41:29.115 INFO:teuthology.orchestra.run.smithi189.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-02T13:41:29.115 INFO:teuthology.orchestra.run.smithi189.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-02T13:41:29.115 INFO:teuthology.orchestra.run.smithi189.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-02T13:41:29.115 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-02T13:41:29.115 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-04-02T13:41:29.115 INFO:teuthology.orchestra.run.smithi189.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-02T13:41:29.116 INFO:teuthology.orchestra.run.smithi189.stdout: automake-1.16.2-8.el9.noarch 2024-04-02T13:41:29.116 INFO:teuthology.orchestra.run.smithi189.stdout: bison-3.7.4-5.el9.x86_64 2024-04-02T13:41:29.116 INFO:teuthology.orchestra.run.smithi189.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-02T13:41:29.116 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.116 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.116 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.116 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.116 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:29.117 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.117 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.117 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.117 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:29.117 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:29.117 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:29.118 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:29.118 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:29.118 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.118 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.118 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:29.118 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.118 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.118 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.118 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:29.118 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:29.118 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.118 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.118 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:29.119 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-02T13:41:29.119 INFO:teuthology.orchestra.run.smithi189.stdout: flex-2.6.4-9.el9.x86_64 2024-04-02T13:41:29.119 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-02T13:41:29.119 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-02T13:41:29.119 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-02T13:41:29.119 INFO:teuthology.orchestra.run.smithi189.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-02T13:41:29.119 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-02T13:41:29.119 INFO:teuthology.orchestra.run.smithi189.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-02T13:41:29.119 INFO:teuthology.orchestra.run.smithi189.stdout: jq-1.6-16.el9.x86_64 2024-04-02T13:41:29.119 INFO:teuthology.orchestra.run.smithi189.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-02T13:41:29.119 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-02T13:41:29.119 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-02T13:41:29.119 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.120 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.120 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.120 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-02T13:41:29.120 INFO:teuthology.orchestra.run.smithi189.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-02T13:41:29.120 INFO:teuthology.orchestra.run.smithi189.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-02T13:41:29.120 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-02T13:41:29.120 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-02T13:41:29.120 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.120 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.120 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-02T13:41:29.120 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.121 INFO:teuthology.orchestra.run.smithi189.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-02T13:41:29.121 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-02T13:41:29.121 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-02T13:41:29.121 INFO:teuthology.orchestra.run.smithi189.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-02T13:41:29.121 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-02T13:41:29.121 INFO:teuthology.orchestra.run.smithi189.stdout: lua-5.4.4-4.el9.x86_64 2024-04-02T13:41:29.121 INFO:teuthology.orchestra.run.smithi189.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-02T13:41:29.121 INFO:teuthology.orchestra.run.smithi189.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-02T13:41:29.121 INFO:teuthology.orchestra.run.smithi189.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-02T13:41:29.121 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-02T13:41:29.121 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-02T13:41:29.122 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-02T13:41:29.122 INFO:teuthology.orchestra.run.smithi189.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-02T13:41:29.122 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-13.14-1.el9.x86_64 2024-04-02T13:41:29.122 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-02T13:41:29.122 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-02T13:41:29.122 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-02T13:41:29.122 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-02T13:41:29.122 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-02T13:41:29.122 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-02T13:41:29.122 INFO:teuthology.orchestra.run.smithi189.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-02T13:41:29.122 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-02T13:41:29.122 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-02T13:41:29.122 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-02T13:41:29.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-02T13:41:29.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-02T13:41:29.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-02T13:41:29.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-02T13:41:29.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-02T13:41:29.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-02T13:41:29.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-02T13:41:29.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-02T13:41:29.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-02T13:41:29.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-02T13:41:29.123 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-02T13:41:29.124 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-02T13:41:29.124 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-02T13:41:29.124 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-02T13:41:29.124 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-02T13:41:29.124 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-02T13:41:29.124 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-02T13:41:29.124 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-02T13:41:29.124 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-02T13:41:29.124 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-02T13:41:29.124 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-02T13:41:29.124 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-02T13:41:29.124 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-02T13:41:29.124 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-02T13:41:29.125 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-02T13:41:29.125 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-02T13:41:29.125 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-02T13:41:29.125 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-02T13:41:29.126 INFO:teuthology.orchestra.run.smithi189.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-02T13:41:29.126 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-02T13:41:29.126 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ply-3.11-14.el9.noarch 2024-04-02T13:41:29.126 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-02T13:41:29.126 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-02T13:41:29.126 INFO:teuthology.orchestra.run.smithi189.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-02T13:41:29.126 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-02T13:41:29.126 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-02T13:41:29.126 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-02T13:41:29.126 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-02T13:41:29.126 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-02T13:41:29.126 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-02T13:41:29.127 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.127 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.127 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-02T13:41:29.127 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-02T13:41:29.127 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-02T13:41:29.127 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.127 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-02T13:41:29.127 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-02T13:41:29.127 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-02T13:41:29.127 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-02T13:41:29.127 INFO:teuthology.orchestra.run.smithi189.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-02T13:41:29.127 INFO:teuthology.orchestra.run.smithi189.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-02T13:41:29.127 INFO:teuthology.orchestra.run.smithi189.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-02T13:41:29.128 INFO:teuthology.orchestra.run.smithi189.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-02T13:41:29.128 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-02T13:41:29.128 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-02T13:41:29.128 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-02T13:41:29.128 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-02T13:41:29.128 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-02T13:41:29.128 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-02T13:41:29.128 INFO:teuthology.orchestra.run.smithi189.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-02T13:41:29.128 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.128 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.128 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:29.128 INFO:teuthology.orchestra.run.smithi189.stdout: re2-1:20211101-3.el9.x86_64 2024-04-02T13:41:29.128 INFO:teuthology.orchestra.run.smithi189.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-02T13:41:29.129 INFO:teuthology.orchestra.run.smithi189.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-02T13:41:29.129 INFO:teuthology.orchestra.run.smithi189.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-02T13:41:29.129 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-02T13:41:29.129 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-02T13:41:29.129 INFO:teuthology.orchestra.run.smithi189.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-02T13:41:29.129 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-02T13:41:29.129 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-04-02T13:41:29.140 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-02T13:41:29.174 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-02T13:41:29.221 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-02T13:41:29.261 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-02T13:41:29.287 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-02T13:41:29.312 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-02T13:41:29.337 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-02T13:41:29.387 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-02T13:41:29.394 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:41:29.394 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:41:29.535 DEBUG:teuthology.parallel:result is None 2024-04-02T13:41:29.573 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:41:29.574 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T13:41:29.656 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T13:41:29.687 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-02T13:41:29.715 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-02T13:41:29.740 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-02T13:41:29.774 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-02T13:41:29.842 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T13:41:31.467 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T13:41:31.508 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 154/198 2024-04-02T13:41:31.539 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 155/198 2024-04-02T13:41:31.566 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-02T13:41:31.738 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 157/198 2024-04-02T13:41:31.856 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-02T13:41:31.897 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 159/198 2024-04-02T13:41:31.936 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-02T13:41:32.006 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-02T13:41:32.040 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-02T13:41:32.064 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 163/198 2024-04-02T13:41:32.105 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-02T13:41:32.134 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-04-02T13:41:32.181 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-04-02T13:41:32.250 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-02T13:41:32.291 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-02T13:41:32.844 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:41:32.889 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:41:32.909 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:41:33.744 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 170/198 2024-04-02T13:41:33.834 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-02T13:41:33.875 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-02T13:41:33.955 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-02T13:41:33.996 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-02T13:41:34.021 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-02T13:41:34.046 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-02T13:41:34.073 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-02T13:41:34.073 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T13:41:34.161 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T13:41:34.190 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-02T13:41:34.190 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T13:41:34.283 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T13:41:34.318 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-02T13:41:34.351 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-02T13:41:34.377 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-02T13:41:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-02T13:41:34.434 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-02T13:41:34.482 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-02T13:41:34.522 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-02T13:41:34.548 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-02T13:41:34.573 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-02T13:41:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-02T13:41:34.632 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-02T13:41:34.639 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:41:34.639 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:41:34.819 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:41:34.819 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T13:41:34.901 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T13:41:34.933 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-02T13:41:34.960 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-02T13:41:34.985 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-02T13:41:35.011 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-02T13:41:35.075 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T13:41:35.281 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T13:41:35.282 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 1/198 2024-04-02T13:41:35.282 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2/198 2024-04-02T13:41:35.282 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 3/198 2024-04-02T13:41:35.282 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 4/198 2024-04-02T13:41:35.282 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 5/198 2024-04-02T13:41:35.282 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 6/198 2024-04-02T13:41:35.282 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 7/198 2024-04-02T13:41:35.282 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 8/198 2024-04-02T13:41:35.282 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 9/198 2024-04-02T13:41:35.282 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:41:35.282 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:41:35.282 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T13:41:35.283 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 13/198 2024-04-02T13:41:35.283 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 14/198 2024-04-02T13:41:35.283 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 15/198 2024-04-02T13:41:35.283 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:41:35.283 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 17/198 2024-04-02T13:41:35.283 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 18/198 2024-04-02T13:41:35.283 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 19/198 2024-04-02T13:41:35.283 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 20/198 2024-04-02T13:41:35.283 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 21/198 2024-04-02T13:41:35.283 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 22/198 2024-04-02T13:41:35.283 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 23/198 2024-04-02T13:41:35.283 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 24/198 2024-04-02T13:41:35.283 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 25/198 2024-04-02T13:41:35.284 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 26/198 2024-04-02T13:41:35.284 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 27/198 2024-04-02T13:41:35.284 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 28/198 2024-04-02T13:41:35.284 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 29/198 2024-04-02T13:41:35.300 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 30/198 2024-04-02T13:41:35.300 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 31/198 2024-04-02T13:41:35.300 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 32/198 2024-04-02T13:41:35.300 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 33/198 2024-04-02T13:41:35.300 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 34/198 2024-04-02T13:41:35.300 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 35/198 2024-04-02T13:41:35.300 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 36/198 2024-04-02T13:41:35.301 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 37/198 2024-04-02T13:41:35.301 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 38/198 2024-04-02T13:41:35.301 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 39/198 2024-04-02T13:41:35.301 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-02T13:41:35.301 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-02T13:41:35.301 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-02T13:41:35.301 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-02T13:41:35.301 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-02T13:41:35.301 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-02T13:41:35.301 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-02T13:41:35.301 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-02T13:41:35.301 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-02T13:41:35.301 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-02T13:41:35.301 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-02T13:41:35.302 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-02T13:41:35.302 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-02T13:41:35.302 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-02T13:41:35.302 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-02T13:41:35.302 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-02T13:41:35.302 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-02T13:41:35.302 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-02T13:41:35.302 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-02T13:41:35.302 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-02T13:41:35.302 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-02T13:41:35.302 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-02T13:41:35.303 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-02T13:41:35.303 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-02T13:41:35.303 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-02T13:41:35.303 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-02T13:41:35.303 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-02T13:41:35.303 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-02T13:41:35.303 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-02T13:41:35.303 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-02T13:41:35.303 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-02T13:41:35.303 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-02T13:41:35.303 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-02T13:41:35.303 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-02T13:41:35.303 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-02T13:41:35.303 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-02T13:41:35.304 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-02T13:41:35.304 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-02T13:41:35.304 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-02T13:41:35.304 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-02T13:41:35.304 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-02T13:41:35.304 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-02T13:41:35.304 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-02T13:41:35.304 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-02T13:41:35.304 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-02T13:41:35.304 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-02T13:41:35.304 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-02T13:41:35.304 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-02T13:41:35.304 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-02T13:41:35.304 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T13:41:35.305 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-02T13:41:35.305 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-02T13:41:35.305 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-02T13:41:35.305 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-02T13:41:35.305 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-02T13:41:35.305 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-02T13:41:35.305 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-02T13:41:35.305 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-02T13:41:35.305 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-02T13:41:35.305 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-02T13:41:35.305 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-02T13:41:35.305 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-02T13:41:35.305 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-02T13:41:35.305 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-02T13:41:35.305 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-02T13:41:35.306 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-02T13:41:35.306 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-02T13:41:35.306 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-02T13:41:35.306 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-02T13:41:35.306 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-02T13:41:35.306 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-02T13:41:35.306 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-02T13:41:35.306 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-02T13:41:35.306 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-02T13:41:35.306 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-02T13:41:35.306 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-02T13:41:35.306 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-02T13:41:35.306 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-02T13:41:35.306 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-02T13:41:35.307 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-02T13:41:35.307 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-02T13:41:35.307 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-02T13:41:35.307 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-02T13:41:35.307 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-02T13:41:35.307 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-02T13:41:35.307 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-02T13:41:35.307 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-02T13:41:35.307 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-02T13:41:35.307 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-02T13:41:35.307 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-02T13:41:35.307 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-02T13:41:35.307 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-02T13:41:35.307 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-02T13:41:35.308 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-02T13:41:35.308 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-02T13:41:35.308 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-02T13:41:35.308 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-02T13:41:35.308 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-02T13:41:35.308 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-02T13:41:35.308 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-02T13:41:35.308 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-02T13:41:35.308 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-02T13:41:35.308 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-02T13:41:35.308 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-02T13:41:35.308 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-02T13:41:35.308 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-02T13:41:35.308 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-02T13:41:35.309 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-02T13:41:35.309 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-02T13:41:35.309 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-02T13:41:35.309 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-02T13:41:35.309 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:41:35.309 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-02T13:41:35.309 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T13:41:35.309 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-02T13:41:35.309 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-04-02T13:41:35.309 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-02T13:41:35.309 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-04-02T13:41:35.309 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-02T13:41:35.309 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-04-02T13:41:35.309 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-02T13:41:35.309 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-04-02T13:41:35.310 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-02T13:41:35.310 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-02T13:41:35.310 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-02T13:41:35.310 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-02T13:41:35.310 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-02T13:41:35.310 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-02T13:41:35.310 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-02T13:41:35.310 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-02T13:41:35.310 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-02T13:41:35.310 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-02T13:41:35.310 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-02T13:41:35.310 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-02T13:41:35.310 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-02T13:41:35.310 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-02T13:41:35.311 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-02T13:41:35.311 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-02T13:41:35.311 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-02T13:41:35.311 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-02T13:41:35.311 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-02T13:41:35.311 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-02T13:41:35.311 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-02T13:41:35.311 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-02T13:41:35.311 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-02T13:41:35.311 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-02T13:41:35.311 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-02T13:41:35.311 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-02T13:41:35.311 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-02T13:41:35.311 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-02T13:41:35.312 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-02T13:41:35.312 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-02T13:41:35.312 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-02T13:41:35.312 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-02T13:41:35.312 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-02T13:41:35.312 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-02T13:41:35.312 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-02T13:41:35.312 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-02T13:41:38.455 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-02T13:41:38.455 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:41:38.455 INFO:teuthology.orchestra.run.smithi052.stdout:Upgraded: 2024-04-02T13:41:38.455 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-04-02T13:41:38.455 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-04-02T13:41:38.455 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-04-02T13:41:38.456 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-04-02T13:41:38.456 INFO:teuthology.orchestra.run.smithi052.stdout: acl-2.3.1-4.el9.x86_64 2024-04-02T13:41:38.456 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-02T13:41:38.456 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-02T13:41:38.456 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-02T13:41:38.456 INFO:teuthology.orchestra.run.smithi052.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-02T13:41:38.456 INFO:teuthology.orchestra.run.smithi052.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-02T13:41:38.456 INFO:teuthology.orchestra.run.smithi052.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-02T13:41:38.456 INFO:teuthology.orchestra.run.smithi052.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-02T13:41:38.456 INFO:teuthology.orchestra.run.smithi052.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-02T13:41:38.456 INFO:teuthology.orchestra.run.smithi052.stdout: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.456 INFO:teuthology.orchestra.run.smithi052.stdout: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.456 INFO:teuthology.orchestra.run.smithi052.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-02T13:41:38.456 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-02T13:41:38.457 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-02T13:41:38.457 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-02T13:41:38.457 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-02T13:41:38.457 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-02T13:41:38.457 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-02T13:41:38.457 INFO:teuthology.orchestra.run.smithi052.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-02T13:41:38.457 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-02T13:41:38.457 INFO:teuthology.orchestra.run.smithi052.stdout:Installed: 2024-04-02T13:41:38.457 INFO:teuthology.orchestra.run.smithi052.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-02T13:41:38.457 INFO:teuthology.orchestra.run.smithi052.stdout: automake-1.16.2-8.el9.noarch 2024-04-02T13:41:38.457 INFO:teuthology.orchestra.run.smithi052.stdout: bison-3.7.4-5.el9.x86_64 2024-04-02T13:41:38.457 INFO:teuthology.orchestra.run.smithi052.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-02T13:41:38.457 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.458 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.458 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.458 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.458 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-grafana-dashboards-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:38.458 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-immutable-object-cache-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.458 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.458 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.458 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:38.458 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:38.458 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:38.458 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:38.458 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:38.458 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.459 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.459 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:38.459 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.459 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.459 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.459 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:38.459 INFO:teuthology.orchestra.run.smithi052.stdout: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:38.459 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.459 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.459 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:38.459 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-02T13:41:38.459 INFO:teuthology.orchestra.run.smithi052.stdout: flex-2.6.4-9.el9.x86_64 2024-04-02T13:41:38.459 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-02T13:41:38.459 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-02T13:41:38.460 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-02T13:41:38.460 INFO:teuthology.orchestra.run.smithi052.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-02T13:41:38.460 INFO:teuthology.orchestra.run.smithi052.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-02T13:41:38.460 INFO:teuthology.orchestra.run.smithi052.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-02T13:41:38.460 INFO:teuthology.orchestra.run.smithi052.stdout: jq-1.6-16.el9.x86_64 2024-04-02T13:41:38.460 INFO:teuthology.orchestra.run.smithi052.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-02T13:41:38.460 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-02T13:41:38.460 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-02T13:41:38.460 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.460 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.460 INFO:teuthology.orchestra.run.smithi052.stdout: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.460 INFO:teuthology.orchestra.run.smithi052.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-02T13:41:38.460 INFO:teuthology.orchestra.run.smithi052.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-02T13:41:38.461 INFO:teuthology.orchestra.run.smithi052.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-02T13:41:38.461 INFO:teuthology.orchestra.run.smithi052.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-02T13:41:38.461 INFO:teuthology.orchestra.run.smithi052.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-02T13:41:38.461 INFO:teuthology.orchestra.run.smithi052.stdout: librados-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.461 INFO:teuthology.orchestra.run.smithi052.stdout: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.461 INFO:teuthology.orchestra.run.smithi052.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-02T13:41:38.461 INFO:teuthology.orchestra.run.smithi052.stdout: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.461 INFO:teuthology.orchestra.run.smithi052.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-02T13:41:38.461 INFO:teuthology.orchestra.run.smithi052.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-02T13:41:38.461 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-02T13:41:38.461 INFO:teuthology.orchestra.run.smithi052.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-02T13:41:38.461 INFO:teuthology.orchestra.run.smithi052.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-02T13:41:38.461 INFO:teuthology.orchestra.run.smithi052.stdout: lua-5.4.4-4.el9.x86_64 2024-04-02T13:41:38.461 INFO:teuthology.orchestra.run.smithi052.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-02T13:41:38.462 INFO:teuthology.orchestra.run.smithi052.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-02T13:41:38.462 INFO:teuthology.orchestra.run.smithi052.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-02T13:41:38.462 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-02T13:41:38.462 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-02T13:41:38.462 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-02T13:41:38.462 INFO:teuthology.orchestra.run.smithi052.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-02T13:41:38.462 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-13.14-1.el9.x86_64 2024-04-02T13:41:38.462 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-02T13:41:38.462 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-02T13:41:38.462 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-02T13:41:38.462 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-02T13:41:38.462 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-02T13:41:38.462 INFO:teuthology.orchestra.run.smithi052.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-02T13:41:38.463 INFO:teuthology.orchestra.run.smithi052.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-02T13:41:38.463 INFO:teuthology.orchestra.run.smithi052.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-02T13:41:38.463 INFO:teuthology.orchestra.run.smithi052.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-02T13:41:38.463 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-02T13:41:38.463 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-argparse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.463 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.463 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.463 INFO:teuthology.orchestra.run.smithi052.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-02T13:41:38.463 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-02T13:41:38.463 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-02T13:41:38.463 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-02T13:41:38.463 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-02T13:41:38.463 INFO:teuthology.orchestra.run.smithi052.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-02T13:41:38.464 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-02T13:41:38.464 INFO:teuthology.orchestra.run.smithi052.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-02T13:41:38.464 INFO:teuthology.orchestra.run.smithi052.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-02T13:41:38.464 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-02T13:41:38.464 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-02T13:41:38.464 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-02T13:41:38.464 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-02T13:41:38.464 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-02T13:41:38.464 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-02T13:41:38.464 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-02T13:41:38.464 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-02T13:41:38.464 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-02T13:41:38.464 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-02T13:41:38.465 INFO:teuthology.orchestra.run.smithi052.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-02T13:41:38.465 INFO:teuthology.orchestra.run.smithi052.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-02T13:41:38.465 INFO:teuthology.orchestra.run.smithi052.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-02T13:41:38.465 INFO:teuthology.orchestra.run.smithi052.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-02T13:41:38.465 INFO:teuthology.orchestra.run.smithi052.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-02T13:41:38.465 INFO:teuthology.orchestra.run.smithi052.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-02T13:41:38.465 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-02T13:41:38.465 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-02T13:41:38.465 INFO:teuthology.orchestra.run.smithi052.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-02T13:41:38.465 INFO:teuthology.orchestra.run.smithi052.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-02T13:41:38.465 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-02T13:41:38.465 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ply-3.11-14.el9.noarch 2024-04-02T13:41:38.465 INFO:teuthology.orchestra.run.smithi052.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-02T13:41:38.465 INFO:teuthology.orchestra.run.smithi052.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-02T13:41:38.466 INFO:teuthology.orchestra.run.smithi052.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-02T13:41:38.466 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-02T13:41:38.466 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-02T13:41:38.466 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-02T13:41:38.466 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-02T13:41:38.466 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-02T13:41:38.466 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-02T13:41:38.466 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.466 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.466 INFO:teuthology.orchestra.run.smithi052.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-02T13:41:38.466 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-02T13:41:38.466 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-02T13:41:38.466 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.466 INFO:teuthology.orchestra.run.smithi052.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-02T13:41:38.467 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-02T13:41:38.467 INFO:teuthology.orchestra.run.smithi052.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-02T13:41:38.467 INFO:teuthology.orchestra.run.smithi052.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-02T13:41:38.467 INFO:teuthology.orchestra.run.smithi052.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-02T13:41:38.467 INFO:teuthology.orchestra.run.smithi052.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-02T13:41:38.467 INFO:teuthology.orchestra.run.smithi052.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-02T13:41:38.467 INFO:teuthology.orchestra.run.smithi052.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-02T13:41:38.467 INFO:teuthology.orchestra.run.smithi052.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-02T13:41:38.467 INFO:teuthology.orchestra.run.smithi052.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-02T13:41:38.467 INFO:teuthology.orchestra.run.smithi052.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-02T13:41:38.467 INFO:teuthology.orchestra.run.smithi052.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-02T13:41:38.467 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-02T13:41:38.467 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-02T13:41:38.468 INFO:teuthology.orchestra.run.smithi052.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-02T13:41:38.468 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.468 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.468 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:38.468 INFO:teuthology.orchestra.run.smithi052.stdout: re2-1:20211101-3.el9.x86_64 2024-04-02T13:41:38.468 INFO:teuthology.orchestra.run.smithi052.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-02T13:41:38.468 INFO:teuthology.orchestra.run.smithi052.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-02T13:41:38.468 INFO:teuthology.orchestra.run.smithi052.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-02T13:41:38.468 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-02T13:41:38.468 INFO:teuthology.orchestra.run.smithi052.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-02T13:41:38.469 INFO:teuthology.orchestra.run.smithi052.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-02T13:41:38.469 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:41:38.469 INFO:teuthology.orchestra.run.smithi052.stdout:Complete! 2024-04-02T13:41:38.925 DEBUG:teuthology.parallel:result is None 2024-04-02T13:41:40.595 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T13:41:40.595 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 1/198 2024-04-02T13:41:40.596 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2/198 2024-04-02T13:41:40.596 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 3/198 2024-04-02T13:41:40.596 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 4/198 2024-04-02T13:41:40.596 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 5/198 2024-04-02T13:41:40.596 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 6/198 2024-04-02T13:41:40.596 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 7/198 2024-04-02T13:41:40.596 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 8/198 2024-04-02T13:41:40.596 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 9/198 2024-04-02T13:41:40.596 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:41:40.596 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:41:40.596 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T13:41:40.596 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 13/198 2024-04-02T13:41:40.597 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 14/198 2024-04-02T13:41:40.597 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 15/198 2024-04-02T13:41:40.597 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:41:40.597 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 17/198 2024-04-02T13:41:40.597 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 18/198 2024-04-02T13:41:40.597 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 19/198 2024-04-02T13:41:40.597 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 20/198 2024-04-02T13:41:40.597 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 21/198 2024-04-02T13:41:40.597 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 22/198 2024-04-02T13:41:40.597 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 23/198 2024-04-02T13:41:40.597 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 24/198 2024-04-02T13:41:40.597 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 25/198 2024-04-02T13:41:40.597 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 26/198 2024-04-02T13:41:40.597 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 27/198 2024-04-02T13:41:40.615 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 28/198 2024-04-02T13:41:40.615 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 29/198 2024-04-02T13:41:40.615 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 30/198 2024-04-02T13:41:40.615 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 31/198 2024-04-02T13:41:40.615 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 32/198 2024-04-02T13:41:40.615 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 33/198 2024-04-02T13:41:40.615 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 34/198 2024-04-02T13:41:40.615 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 35/198 2024-04-02T13:41:40.615 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 36/198 2024-04-02T13:41:40.616 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 37/198 2024-04-02T13:41:40.616 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 38/198 2024-04-02T13:41:40.616 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 39/198 2024-04-02T13:41:40.616 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-02T13:41:40.616 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-02T13:41:40.616 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-02T13:41:40.616 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-02T13:41:40.616 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-02T13:41:40.616 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-02T13:41:40.616 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-02T13:41:40.616 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-02T13:41:40.616 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-02T13:41:40.617 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-02T13:41:40.617 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-02T13:41:40.617 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-02T13:41:40.617 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-02T13:41:40.617 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-02T13:41:40.617 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-02T13:41:40.617 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-02T13:41:40.617 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-02T13:41:40.617 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-02T13:41:40.617 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-02T13:41:40.617 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-02T13:41:40.617 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-02T13:41:40.617 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-02T13:41:40.618 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-02T13:41:40.618 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-02T13:41:40.618 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-02T13:41:40.618 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-02T13:41:40.618 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-02T13:41:40.618 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-02T13:41:40.618 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-02T13:41:40.618 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-02T13:41:40.618 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-02T13:41:40.618 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-02T13:41:40.618 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-02T13:41:40.618 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-02T13:41:40.618 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-02T13:41:40.619 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-02T13:41:40.619 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-02T13:41:40.619 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-02T13:41:40.619 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-02T13:41:40.619 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-02T13:41:40.619 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-02T13:41:40.619 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-02T13:41:40.619 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-02T13:41:40.619 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-02T13:41:40.619 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-02T13:41:40.619 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-02T13:41:40.619 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-02T13:41:40.619 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-02T13:41:40.620 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-02T13:41:40.620 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T13:41:40.620 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-02T13:41:40.620 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-02T13:41:40.620 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-02T13:41:40.620 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-02T13:41:40.620 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-02T13:41:40.620 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-02T13:41:40.620 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-02T13:41:40.620 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-02T13:41:40.620 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-02T13:41:40.620 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-02T13:41:40.620 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-02T13:41:40.621 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-02T13:41:40.621 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-02T13:41:40.621 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-02T13:41:40.621 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-02T13:41:40.621 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-02T13:41:40.621 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-02T13:41:40.621 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-02T13:41:40.621 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-02T13:41:40.621 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-02T13:41:40.621 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-02T13:41:40.621 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-02T13:41:40.621 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-02T13:41:40.621 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-02T13:41:40.622 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-02T13:41:40.622 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-02T13:41:40.622 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-02T13:41:40.622 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-02T13:41:40.622 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-02T13:41:40.622 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-02T13:41:40.622 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-02T13:41:40.622 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-02T13:41:40.622 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-02T13:41:40.622 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-02T13:41:40.622 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-02T13:41:40.622 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-02T13:41:40.622 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-02T13:41:40.623 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-02T13:41:40.623 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-02T13:41:40.623 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-02T13:41:40.623 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-02T13:41:40.623 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-02T13:41:40.623 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-02T13:41:40.623 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-02T13:41:40.623 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-02T13:41:40.623 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-02T13:41:40.623 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-02T13:41:40.623 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-02T13:41:40.623 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-02T13:41:40.623 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-02T13:41:40.624 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-02T13:41:40.624 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-02T13:41:40.624 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-02T13:41:40.624 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-02T13:41:40.624 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-02T13:41:40.624 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-02T13:41:40.624 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-02T13:41:40.624 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-02T13:41:40.624 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-02T13:41:40.624 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-02T13:41:40.624 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-02T13:41:40.624 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:41:40.624 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-02T13:41:40.624 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T13:41:40.625 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-02T13:41:40.625 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-04-02T13:41:40.625 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-02T13:41:40.625 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-04-02T13:41:40.625 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-02T13:41:40.625 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-04-02T13:41:40.625 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-02T13:41:40.625 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-04-02T13:41:40.625 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-02T13:41:40.625 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-02T13:41:40.625 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-02T13:41:40.625 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-02T13:41:40.625 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-02T13:41:40.625 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-02T13:41:40.626 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-02T13:41:40.626 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-02T13:41:40.626 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-02T13:41:40.626 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-02T13:41:40.626 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-02T13:41:40.626 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-02T13:41:40.626 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-02T13:41:40.626 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-02T13:41:40.626 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-02T13:41:40.626 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-02T13:41:40.626 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-02T13:41:40.626 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-02T13:41:40.626 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-02T13:41:40.627 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-02T13:41:40.627 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-02T13:41:40.627 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-02T13:41:40.627 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-02T13:41:40.627 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-02T13:41:40.627 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-02T13:41:40.627 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-02T13:41:40.627 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-02T13:41:40.627 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-02T13:41:40.628 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-02T13:41:40.628 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-02T13:41:40.628 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-02T13:41:40.628 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-02T13:41:40.628 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-02T13:41:40.628 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-02T13:41:40.628 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-02T13:41:40.628 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-02T13:41:43.924 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-02T13:41:43.925 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-02T13:41:43.925 INFO:teuthology.orchestra.run.smithi088.stdout:Upgraded: 2024-04-02T13:41:43.925 INFO:teuthology.orchestra.run.smithi088.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-04-02T13:41:43.925 INFO:teuthology.orchestra.run.smithi088.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-04-02T13:41:43.925 INFO:teuthology.orchestra.run.smithi088.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-04-02T13:41:43.925 INFO:teuthology.orchestra.run.smithi088.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-04-02T13:41:43.925 INFO:teuthology.orchestra.run.smithi088.stdout: acl-2.3.1-4.el9.x86_64 2024-04-02T13:41:43.925 INFO:teuthology.orchestra.run.smithi088.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-02T13:41:43.925 INFO:teuthology.orchestra.run.smithi088.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-02T13:41:43.925 INFO:teuthology.orchestra.run.smithi088.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-02T13:41:43.925 INFO:teuthology.orchestra.run.smithi088.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-02T13:41:43.926 INFO:teuthology.orchestra.run.smithi088.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-02T13:41:43.926 INFO:teuthology.orchestra.run.smithi088.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-02T13:41:43.926 INFO:teuthology.orchestra.run.smithi088.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-02T13:41:43.926 INFO:teuthology.orchestra.run.smithi088.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-02T13:41:43.926 INFO:teuthology.orchestra.run.smithi088.stdout: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.926 INFO:teuthology.orchestra.run.smithi088.stdout: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.926 INFO:teuthology.orchestra.run.smithi088.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-02T13:41:43.926 INFO:teuthology.orchestra.run.smithi088.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-02T13:41:43.926 INFO:teuthology.orchestra.run.smithi088.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-02T13:41:43.926 INFO:teuthology.orchestra.run.smithi088.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-02T13:41:43.926 INFO:teuthology.orchestra.run.smithi088.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-02T13:41:43.926 INFO:teuthology.orchestra.run.smithi088.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-02T13:41:43.927 INFO:teuthology.orchestra.run.smithi088.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-02T13:41:43.927 INFO:teuthology.orchestra.run.smithi088.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-02T13:41:43.927 INFO:teuthology.orchestra.run.smithi088.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-02T13:41:43.927 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-04-02T13:41:43.927 INFO:teuthology.orchestra.run.smithi088.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-02T13:41:43.927 INFO:teuthology.orchestra.run.smithi088.stdout: automake-1.16.2-8.el9.noarch 2024-04-02T13:41:43.927 INFO:teuthology.orchestra.run.smithi088.stdout: bison-3.7.4-5.el9.x86_64 2024-04-02T13:41:43.927 INFO:teuthology.orchestra.run.smithi088.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-02T13:41:43.927 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.927 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.927 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.927 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.927 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-grafana-dashboards-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:43.928 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-immutable-object-cache-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.928 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.928 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.928 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:43.928 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:43.928 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:43.928 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:43.928 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:43.928 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.928 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.928 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:43.928 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.928 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.929 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.929 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:43.929 INFO:teuthology.orchestra.run.smithi088.stdout: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:43.929 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.929 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.929 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:41:43.929 INFO:teuthology.orchestra.run.smithi088.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-02T13:41:43.929 INFO:teuthology.orchestra.run.smithi088.stdout: flex-2.6.4-9.el9.x86_64 2024-04-02T13:41:43.929 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-02T13:41:43.929 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-02T13:41:43.929 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-02T13:41:43.929 INFO:teuthology.orchestra.run.smithi088.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-02T13:41:43.929 INFO:teuthology.orchestra.run.smithi088.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-02T13:41:43.930 INFO:teuthology.orchestra.run.smithi088.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-02T13:41:43.930 INFO:teuthology.orchestra.run.smithi088.stdout: jq-1.6-16.el9.x86_64 2024-04-02T13:41:43.930 INFO:teuthology.orchestra.run.smithi088.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-02T13:41:43.930 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-02T13:41:43.930 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-02T13:41:43.930 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.930 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.930 INFO:teuthology.orchestra.run.smithi088.stdout: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.930 INFO:teuthology.orchestra.run.smithi088.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-02T13:41:43.930 INFO:teuthology.orchestra.run.smithi088.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-02T13:41:43.931 INFO:teuthology.orchestra.run.smithi088.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-02T13:41:43.931 INFO:teuthology.orchestra.run.smithi088.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-02T13:41:43.931 INFO:teuthology.orchestra.run.smithi088.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-02T13:41:43.931 INFO:teuthology.orchestra.run.smithi088.stdout: librados-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.931 INFO:teuthology.orchestra.run.smithi088.stdout: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.931 INFO:teuthology.orchestra.run.smithi088.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-02T13:41:43.931 INFO:teuthology.orchestra.run.smithi088.stdout: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.931 INFO:teuthology.orchestra.run.smithi088.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-02T13:41:43.931 INFO:teuthology.orchestra.run.smithi088.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-02T13:41:43.931 INFO:teuthology.orchestra.run.smithi088.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-02T13:41:43.931 INFO:teuthology.orchestra.run.smithi088.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-02T13:41:43.931 INFO:teuthology.orchestra.run.smithi088.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-02T13:41:43.931 INFO:teuthology.orchestra.run.smithi088.stdout: lua-5.4.4-4.el9.x86_64 2024-04-02T13:41:43.932 INFO:teuthology.orchestra.run.smithi088.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-02T13:41:43.932 INFO:teuthology.orchestra.run.smithi088.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-02T13:41:43.932 INFO:teuthology.orchestra.run.smithi088.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-02T13:41:43.932 INFO:teuthology.orchestra.run.smithi088.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-02T13:41:43.932 INFO:teuthology.orchestra.run.smithi088.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-02T13:41:43.932 INFO:teuthology.orchestra.run.smithi088.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-02T13:41:43.932 INFO:teuthology.orchestra.run.smithi088.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-02T13:41:43.932 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-13.14-1.el9.x86_64 2024-04-02T13:41:43.932 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-02T13:41:43.932 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-02T13:41:43.932 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-02T13:41:43.932 INFO:teuthology.orchestra.run.smithi088.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-02T13:41:43.933 INFO:teuthology.orchestra.run.smithi088.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-02T13:41:43.933 INFO:teuthology.orchestra.run.smithi088.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-02T13:41:43.933 INFO:teuthology.orchestra.run.smithi088.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-02T13:41:43.933 INFO:teuthology.orchestra.run.smithi088.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-02T13:41:43.933 INFO:teuthology.orchestra.run.smithi088.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-02T13:41:43.933 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-02T13:41:43.933 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-argparse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.933 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.934 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.934 INFO:teuthology.orchestra.run.smithi088.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-02T13:41:43.934 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-02T13:41:43.934 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-02T13:41:43.934 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-02T13:41:43.934 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-02T13:41:43.934 INFO:teuthology.orchestra.run.smithi088.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-02T13:41:43.934 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-02T13:41:43.934 INFO:teuthology.orchestra.run.smithi088.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-02T13:41:43.934 INFO:teuthology.orchestra.run.smithi088.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-02T13:41:43.934 INFO:teuthology.orchestra.run.smithi088.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-02T13:41:43.935 INFO:teuthology.orchestra.run.smithi088.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-02T13:41:43.935 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-02T13:41:43.935 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-02T13:41:43.935 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-02T13:41:43.935 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-02T13:41:43.935 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-02T13:41:43.935 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-02T13:41:43.935 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-02T13:41:43.935 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-02T13:41:43.935 INFO:teuthology.orchestra.run.smithi088.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-02T13:41:43.936 INFO:teuthology.orchestra.run.smithi088.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-02T13:41:43.936 INFO:teuthology.orchestra.run.smithi088.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-02T13:41:43.936 INFO:teuthology.orchestra.run.smithi088.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-02T13:41:43.936 INFO:teuthology.orchestra.run.smithi088.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-02T13:41:43.936 INFO:teuthology.orchestra.run.smithi088.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-02T13:41:43.936 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-02T13:41:43.936 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-02T13:41:43.936 INFO:teuthology.orchestra.run.smithi088.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-02T13:41:43.936 INFO:teuthology.orchestra.run.smithi088.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-02T13:41:43.936 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-02T13:41:43.936 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ply-3.11-14.el9.noarch 2024-04-02T13:41:43.936 INFO:teuthology.orchestra.run.smithi088.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-02T13:41:43.937 INFO:teuthology.orchestra.run.smithi088.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-02T13:41:43.937 INFO:teuthology.orchestra.run.smithi088.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-02T13:41:43.937 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-02T13:41:43.937 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-02T13:41:43.937 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-02T13:41:43.937 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-02T13:41:43.937 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-02T13:41:43.937 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-02T13:41:43.937 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.937 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.937 INFO:teuthology.orchestra.run.smithi088.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-02T13:41:43.938 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-02T13:41:43.938 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-02T13:41:43.938 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.938 INFO:teuthology.orchestra.run.smithi088.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-02T13:41:43.938 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-02T13:41:43.938 INFO:teuthology.orchestra.run.smithi088.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-02T13:41:43.938 INFO:teuthology.orchestra.run.smithi088.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-02T13:41:43.938 INFO:teuthology.orchestra.run.smithi088.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-02T13:41:43.938 INFO:teuthology.orchestra.run.smithi088.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-02T13:41:43.938 INFO:teuthology.orchestra.run.smithi088.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-02T13:41:43.938 INFO:teuthology.orchestra.run.smithi088.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-02T13:41:43.938 INFO:teuthology.orchestra.run.smithi088.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-02T13:41:43.939 INFO:teuthology.orchestra.run.smithi088.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-02T13:41:43.939 INFO:teuthology.orchestra.run.smithi088.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-02T13:41:43.939 INFO:teuthology.orchestra.run.smithi088.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-02T13:41:43.939 INFO:teuthology.orchestra.run.smithi088.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-02T13:41:43.939 INFO:teuthology.orchestra.run.smithi088.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-02T13:41:43.939 INFO:teuthology.orchestra.run.smithi088.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-02T13:41:43.939 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.940 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.940 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:41:43.940 INFO:teuthology.orchestra.run.smithi088.stdout: re2-1:20211101-3.el9.x86_64 2024-04-02T13:41:43.940 INFO:teuthology.orchestra.run.smithi088.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-02T13:41:43.940 INFO:teuthology.orchestra.run.smithi088.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-02T13:41:43.940 INFO:teuthology.orchestra.run.smithi088.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-02T13:41:43.940 INFO:teuthology.orchestra.run.smithi088.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-02T13:41:43.940 INFO:teuthology.orchestra.run.smithi088.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-02T13:41:43.941 INFO:teuthology.orchestra.run.smithi088.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-02T13:41:43.941 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-02T13:41:43.941 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-04-02T13:41:44.427 DEBUG:teuthology.parallel:result is None 2024-04-02T13:41:44.428 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:41:44.720 DEBUG:teuthology.orchestra.run.smithi052:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-02T13:41:44.745 INFO:teuthology.orchestra.run.smithi052.stdout:19.0.0-2609.g191d963a.el9 2024-04-02T13:41:44.746 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2609.g191d963a.el9 2024-04-02T13:41:44.746 INFO:teuthology.task.install:The correct ceph version 19.0.0-2609.g191d963a is installed. 2024-04-02T13:41:44.748 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:41:44.969 DEBUG:teuthology.orchestra.run.smithi088:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-02T13:41:44.991 INFO:teuthology.orchestra.run.smithi088.stdout:19.0.0-2609.g191d963a.el9 2024-04-02T13:41:44.992 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2609.g191d963a.el9 2024-04-02T13:41:44.992 INFO:teuthology.task.install:The correct ceph version 19.0.0-2609.g191d963a is installed. 2024-04-02T13:41:44.994 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:41:45.140 DEBUG:teuthology.orchestra.run.smithi189:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-02T13:41:45.166 INFO:teuthology.orchestra.run.smithi189.stdout:19.0.0-2609.g191d963a.el9 2024-04-02T13:41:45.166 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2609.g191d963a.el9 2024-04-02T13:41:45.167 INFO:teuthology.task.install:The correct ceph version 19.0.0-2609.g191d963a is installed. 2024-04-02T13:41:45.168 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-02T13:41:45.169 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-04-02T13:41:45.169 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-02T13:41:45.203 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-02T13:41:45.204 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-02T13:41:45.235 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-02T13:41:45.235 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-02T13:41:45.269 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-02T13:41:45.269 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-04-02T13:41:45.269 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/daemon-helper 2024-04-02T13:41:45.301 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-02T13:41:45.371 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-02T13:41:45.371 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/daemon-helper 2024-04-02T13:41:45.400 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-02T13:41:45.470 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-02T13:41:45.470 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/daemon-helper 2024-04-02T13:41:45.502 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-02T13:41:45.572 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-02T13:41:45.573 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-04-02T13:41:45.573 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-02T13:41:45.604 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-02T13:41:45.673 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-02T13:41:45.673 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-02T13:41:45.701 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-02T13:41:45.768 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-02T13:41:45.769 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-02T13:41:45.796 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-02T13:41:45.862 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-02T13:41:45.862 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-04-02T13:41:45.863 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/stdin-killer 2024-04-02T13:41:45.894 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-02T13:41:45.965 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-02T13:41:45.965 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/stdin-killer 2024-04-02T13:41:45.993 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-02T13:41:46.060 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-02T13:41:46.061 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/stdin-killer 2024-04-02T13:41:46.088 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-02T13:41:46.155 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-02T13:41:46.261 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-02T13:41:46.261 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:41:46.262 INFO:tasks.cephadm:Cluster fsid is bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:41:46.262 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-02T13:41:46.262 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.52', 'mon.b': '172.21.15.88', 'mon.c': '172.21.15.189'} 2024-04-02T13:41:46.262 INFO:tasks.cephadm:First mon is mon.a on smithi052 2024-04-02T13:41:46.262 INFO:tasks.cephadm:First mgr is x 2024-04-02T13:41:46.263 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-02T13:41:46.263 DEBUG:teuthology.orchestra.run.smithi052:> sudo hostname $(hostname -s) 2024-04-02T13:41:46.295 DEBUG:teuthology.orchestra.run.smithi088:> sudo hostname $(hostname -s) 2024-04-02T13:41:46.323 DEBUG:teuthology.orchestra.run.smithi189:> sudo hostname $(hostname -s) 2024-04-02T13:41:46.352 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-02T13:41:46.352 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:41:46.568 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78154/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2609-g191d963a', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2609.g191d963a'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-02 02:04:04.615852', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240402.004512', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-02T13:41:46.702 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-batrick-testing-20240402.004512, sha1 191d963ac9de7322da8aff20b567b3fa458e2ef6 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:41:46.704 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/x86_64/flavors/default/cephadm 2024-04-02T13:41:46.704 DEBUG:teuthology.orchestra.run.smithi052:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-02T13:41:47.400 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r--. 1 ubuntu ubuntu 781237 Apr 2 13:41 /home/ubuntu/cephtest/cephadm 2024-04-02T13:41:47.400 DEBUG:teuthology.orchestra.run.smithi088:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-02T13:41:47.997 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r--. 1 ubuntu ubuntu 781237 Apr 2 13:41 /home/ubuntu/cephtest/cephadm 2024-04-02T13:41:47.997 DEBUG:teuthology.orchestra.run.smithi189:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-02T13:41:48.609 INFO:teuthology.orchestra.run.smithi189.stdout:-rw-r--r--. 1 ubuntu ubuntu 781237 Apr 2 13:41 /home/ubuntu/cephtest/cephadm 2024-04-02T13:41:48.610 DEBUG:teuthology.orchestra.run.smithi052:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-02T13:41:48.629 DEBUG:teuthology.orchestra.run.smithi088:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-02T13:41:48.647 DEBUG:teuthology.orchestra.run.smithi189:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-02T13:41:48.682 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 on all hosts... 2024-04-02T13:41:48.682 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 pull 2024-04-02T13:41:48.685 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 pull 2024-04-02T13:41:48.690 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 pull 2024-04-02T13:41:49.559 INFO:teuthology.orchestra.run.smithi189.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6... 2024-04-02T13:41:49.585 INFO:teuthology.orchestra.run.smithi088.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6... 2024-04-02T13:41:49.690 INFO:teuthology.orchestra.run.smithi052.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6... 2024-04-02T13:42:23.271 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:42:23.271 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_version": "ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev)", 2024-04-02T13:42:23.271 INFO:teuthology.orchestra.run.smithi052.stdout: "image_id": "f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f", 2024-04-02T13:42:23.272 INFO:teuthology.orchestra.run.smithi052.stdout: "repo_digests": [ 2024-04-02T13:42:23.272 INFO:teuthology.orchestra.run.smithi052.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534" 2024-04-02T13:42:23.272 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-04-02T13:42:23.272 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:42:43.396 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-04-02T13:42:43.396 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_version": "ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev)", 2024-04-02T13:42:43.396 INFO:teuthology.orchestra.run.smithi189.stdout: "image_id": "f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f", 2024-04-02T13:42:43.396 INFO:teuthology.orchestra.run.smithi189.stdout: "repo_digests": [ 2024-04-02T13:42:43.397 INFO:teuthology.orchestra.run.smithi189.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534" 2024-04-02T13:42:43.397 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-04-02T13:42:43.397 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-04-02T13:42:43.850 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-02T13:42:43.850 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_version": "ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev)", 2024-04-02T13:42:43.850 INFO:teuthology.orchestra.run.smithi088.stdout: "image_id": "f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f", 2024-04-02T13:42:43.850 INFO:teuthology.orchestra.run.smithi088.stdout: "repo_digests": [ 2024-04-02T13:42:43.851 INFO:teuthology.orchestra.run.smithi088.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534" 2024-04-02T13:42:43.851 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-04-02T13:42:43.851 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-02T13:42:43.877 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /etc/ceph 2024-04-02T13:42:43.912 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /etc/ceph 2024-04-02T13:42:43.942 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /etc/ceph 2024-04-02T13:42:43.973 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 777 /etc/ceph 2024-04-02T13:42:44.003 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 777 /etc/ceph 2024-04-02T13:42:44.030 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 777 /etc/ceph 2024-04-02T13:42:44.057 INFO:tasks.cephadm:Writing seed config... 2024-04-02T13:42:44.059 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-02T13:42:44.059 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-02T13:42:44.059 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-02T13:42:44.059 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-02T13:42:44.059 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-02T13:42:44.059 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-02T13:42:44.059 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-02T13:42:44.059 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-02T13:42:44.059 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-02T13:42:44.059 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-02T13:42:44.059 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-02T13:42:44.059 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-02T13:42:44.060 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-02T13:42:44.060 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-02T13:42:44.060 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-02T13:42:44.060 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-02T13:42:44.060 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-02T13:42:44.060 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-02T13:42:44.060 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-02T13:42:44.060 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-02T13:42:44.060 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-02T13:42:44.060 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-02T13:42:44.060 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-02T13:42:44.060 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-02T13:42:44.060 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-02T13:42:44.060 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-02T13:42:44.060 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-02T13:42:44.060 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-02T13:42:44.061 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-02T13:42:44.061 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-02T13:42:44.061 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-02T13:42:44.061 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-02T13:42:44.061 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-02T13:42:44.061 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-02T13:42:44.061 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-02T13:42:44.061 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-02T13:42:44.061 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-02T13:42:44.061 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-02T13:42:44.061 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-02T13:42:44.061 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-02T13:42:44.061 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-02T13:42:44.061 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-02T13:42:44.061 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-02T13:42:44.061 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-02T13:42:44.061 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-02T13:42:44.062 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-02T13:42:44.062 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-02T13:42:44.063 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-04-02T13:42:44.063 DEBUG:teuthology.orchestra.run.smithi052:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-02T13:42:44.080 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-02T13:42:44.081 DEBUG:teuthology.orchestra.run.smithi052:mon.a> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mon.a.service 2024-04-02T13:42:44.123 DEBUG:teuthology.orchestra.run.smithi052:mgr.x> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mgr.x.service 2024-04-02T13:42:44.166 INFO:tasks.cephadm:Bootstrapping... 2024-04-02T13:42:44.167 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 -v bootstrap --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.52 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-02T13:42:44.369 INFO:teuthology.orchestra.run.smithi052.stdout:-------------------------------------------------------------------------------- 2024-04-02T13:42:44.369 INFO:teuthology.orchestra.run.smithi052.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6', '-v', 'bootstrap', '--fsid', 'bfa00e6e-f0f6-11ee-b647-cb9ed24678a4', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.52', '--skip-admin-label'] 2024-04-02T13:42:44.369 INFO:teuthology.orchestra.run.smithi052.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-02T13:42:44.370 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying podman|docker is present... 2024-04-02T13:42:44.394 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-02T13:42:44.394 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying lvm2 is present... 2024-04-02T13:42:44.394 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying time synchronization is in place... 2024-04-02T13:42:44.401 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-02T13:42:44.402 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-02T13:42:44.407 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-02T13:42:44.407 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2024-04-02T13:42:44.414 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout enabled 2024-04-02T13:42:44.419 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout active 2024-04-02T13:42:44.420 INFO:teuthology.orchestra.run.smithi052.stdout:Unit chronyd.service is enabled and running 2024-04-02T13:42:44.420 INFO:teuthology.orchestra.run.smithi052.stdout:Repeating the final host check... 2024-04-02T13:42:44.443 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-02T13:42:44.443 INFO:teuthology.orchestra.run.smithi052.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-02T13:42:44.443 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl is present 2024-04-02T13:42:44.443 INFO:teuthology.orchestra.run.smithi052.stdout:lvcreate is present 2024-04-02T13:42:44.449 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-02T13:42:44.450 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-02T13:42:44.455 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-02T13:42:44.455 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2024-04-02T13:42:44.461 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout enabled 2024-04-02T13:42:44.468 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout active 2024-04-02T13:42:44.468 INFO:teuthology.orchestra.run.smithi052.stdout:Unit chronyd.service is enabled and running 2024-04-02T13:42:44.468 INFO:teuthology.orchestra.run.smithi052.stdout:Host looks OK 2024-04-02T13:42:44.468 INFO:teuthology.orchestra.run.smithi052.stdout:Cluster fsid: bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:42:44.468 INFO:teuthology.orchestra.run.smithi052.stdout:Acquiring lock 140655284642480 on /run/cephadm/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4.lock 2024-04-02T13:42:44.468 INFO:teuthology.orchestra.run.smithi052.stdout:Lock 140655284642480 acquired on /run/cephadm/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4.lock 2024-04-02T13:42:44.468 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying IP 172.21.15.52 port 3300 ... 2024-04-02T13:42:44.469 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying IP 172.21.15.52 port 6789 ... 2024-04-02T13:42:44.469 INFO:teuthology.orchestra.run.smithi052.stdout:Base mon IP(s) is [172.21.15.52:3300, 172.21.15.52:6789], mon addrv is [v2:172.21.15.52:3300,v1:172.21.15.52:6789] 2024-04-02T13:42:44.472 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.52 metric 100 2024-04-02T13:42:44.472 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.52 metric 100 2024-04-02T13:42:44.474 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-02T13:42:44.475 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-02T13:42:44.475 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-02T13:42:44.477 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-02T13:42:44.477 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-02T13:42:44.477 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-02T13:42:44.477 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-02T13:42:44.477 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15de/64 scope link noprefixroute 2024-04-02T13:42:44.477 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-02T13:42:44.478 INFO:teuthology.orchestra.run.smithi052.stdout:Mon IP `172.21.15.52` is in CIDR network `172.21.0.0/20` 2024-04-02T13:42:44.478 INFO:teuthology.orchestra.run.smithi052.stdout:Mon IP `172.21.15.52` is in CIDR network `172.21.0.0/20` 2024-04-02T13:42:44.478 INFO:teuthology.orchestra.run.smithi052.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-02T13:42:44.479 INFO:teuthology.orchestra.run.smithi052.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-02T13:42:44.479 INFO:teuthology.orchestra.run.smithi052.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6... 2024-04-02T13:42:47.859 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stdout f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f 2024-04-02T13:42:47.860 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6... 2024-04-02T13:42:47.860 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Getting image source signatures 2024-04-02T13:42:47.860 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Copying blob sha256:db8c11fadded9ebce76f2e720ef87f32e8913958c61c526c89a5e4e2c41a8200 2024-04-02T13:42:47.860 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Copying blob sha256:11da15500eca1f826c328eb3265caa30b0a3730510e2a32023084d6fe9cee20c 2024-04-02T13:42:47.860 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Copying config sha256:f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f 2024-04-02T13:42:47.860 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-02T13:42:49.850 INFO:teuthology.orchestra.run.smithi052.stdout:ceph: stdout ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T13:42:49.850 INFO:teuthology.orchestra.run.smithi052.stdout:Ceph version: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T13:42:49.850 INFO:teuthology.orchestra.run.smithi052.stdout:Extracting ceph user uid/gid from container image... 2024-04-02T13:42:50.747 INFO:teuthology.orchestra.run.smithi052.stdout:stat: stdout 167 167 2024-04-02T13:42:50.747 INFO:teuthology.orchestra.run.smithi052.stdout:Creating initial keys... 2024-04-02T13:42:51.620 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph-authtool: stdout AQBbCwxmDXjiBBAAqfooWB9BzamHWUt68xD5wg== 2024-04-02T13:42:52.610 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph-authtool: stdout AQBbCwxmAYlwOBAAX11I9Tn0BjAdHucc5uKAJA== 2024-04-02T13:42:53.625 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph-authtool: stdout AQBcCwxmMyGrNhAAofoS4GYEuKOGhld/dXr0rg== 2024-04-02T13:42:53.625 INFO:teuthology.orchestra.run.smithi052.stdout:Creating initial monmap... 2024-04-02T13:42:57.117 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-02T13:42:57.117 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-02T13:42:57.117 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:42:57.117 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-02T13:42:57.117 INFO:teuthology.orchestra.run.smithi052.stdout:monmaptool for a [v2:172.21.15.52:3300,v1:172.21.15.52:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-02T13:42:57.117 INFO:teuthology.orchestra.run.smithi052.stdout:setting min_mon_release = quincy 2024-04-02T13:42:57.118 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: set fsid to bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:42:57.118 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-02T13:42:57.118 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:42:57.118 INFO:teuthology.orchestra.run.smithi052.stdout:Creating mon... 2024-04-02T13:42:58.139 INFO:teuthology.orchestra.run.smithi052.stdout:create mon.a on 2024-04-02T13:42:58.348 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-02T13:42:58.490 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-02T13:42:58.676 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4.target → /etc/systemd/system/ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4.target. 2024-04-02T13:42:58.677 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4.target → /etc/systemd/system/ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4.target. 2024-04-02T13:42:58.919 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mon.a 2024-04-02T13:42:58.920 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Failed to reset failed state of unit ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mon.a.service: Unit ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mon.a.service not loaded. 2024-04-02T13:42:59.086 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4.target.wants/ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mon.a.service → /etc/systemd/system/ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@.service. 2024-04-02T13:42:59.687 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T13:42:59.687 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout disabled 2024-04-02T13:42:59.693 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T13:42:59.693 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2024-04-02T13:42:59.693 INFO:teuthology.orchestra.run.smithi052.stdout:firewalld.service is not enabled 2024-04-02T13:42:59.693 INFO:teuthology.orchestra.run.smithi052.stdout:Not possible to enable service . firewalld.service is not available 2024-04-02T13:42:59.694 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mon to start... 2024-04-02T13:42:59.694 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mon... 2024-04-02T13:43:00.724 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout cluster: 2024-04-02T13:43:00.731 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout id: bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:43:00.731 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-02T13:43:00.731 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-04-02T13:43:00.731 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout services: 2024-04-02T13:43:00.731 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.542675s) 2024-04-02T13:43:00.731 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-02T13:43:00.731 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-02T13:43:00.731 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-04-02T13:43:00.731 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout data: 2024-04-02T13:43:00.732 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-02T13:43:00.732 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-02T13:43:00.732 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-02T13:43:00.732 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout pgs: 2024-04-02T13:43:00.732 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-04-02T13:43:00.732 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.099+0000 7f5f0cb49700 1 Processor -- start 2024-04-02T13:43:00.732 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.099+0000 7f5f0cb49700 1 -- start start 2024-04-02T13:43:00.732 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.099+0000 7f5f0cb49700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f08106fd0 0x7f5f081073f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:00.732 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.099+0000 7f5f0cb49700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5f081079c0 con 0x7f5f08106fd0 2024-04-02T13:43:00.732 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.099+0000 7f5f0659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f08106fd0 0x7f5f081073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:00.732 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.100+0000 7f5f0659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f08106fd0 0x7f5f081073f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37108/0 (socket says 172.21.15.52:37108) 2024-04-02T13:43:00.732 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.100+0000 7f5f0659c700 1 -- 172.21.15.52:0/1161931809 learned_addr learned my addr 172.21.15.52:0/1161931809 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:00.732 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.100+0000 7f5f0659c700 1 -- 172.21.15.52:0/1161931809 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f08108180 con 0x7f5f08106fd0 2024-04-02T13:43:00.732 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.100+0000 7f5f0659c700 1 --2- 172.21.15.52:0/1161931809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f08106fd0 0x7f5f081073f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800bb40 tx=0x7f5ef800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2a60eb31dbfbfa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:00.733 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.100+0000 7f5f0559a700 1 -- 172.21.15.52:0/1161931809 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ef800dbe0 con 0x7f5f08106fd0 2024-04-02T13:43:00.733 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.100+0000 7f5f0559a700 1 -- 172.21.15.52:0/1161931809 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef800dd80 con 0x7f5f08106fd0 2024-04-02T13:43:00.733 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.100+0000 7f5f0559a700 1 -- 172.21.15.52:0/1161931809 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ef8004180 con 0x7f5f08106fd0 2024-04-02T13:43:00.733 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.101+0000 7f5f0cb49700 1 -- 172.21.15.52:0/1161931809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f08106fd0 msgr2=0x7f5f081073f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:00.733 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.101+0000 7f5f0cb49700 1 --2- 172.21.15.52:0/1161931809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f08106fd0 0x7f5f081073f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800bb40 tx=0x7f5ef800be50 comp rx=0 tx=0).stop 2024-04-02T13:43:00.733 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.101+0000 7f5f0cb49700 1 -- 172.21.15.52:0/1161931809 shutdown_connections 2024-04-02T13:43:00.733 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.101+0000 7f5f0cb49700 1 --2- 172.21.15.52:0/1161931809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f08106fd0 0x7f5f081073f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:00.733 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.101+0000 7f5f0cb49700 1 -- 172.21.15.52:0/1161931809 >> 172.21.15.52:0/1161931809 conn(0x7f5f081025b0 msgr2=0x7f5f081049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:00.733 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.101+0000 7f5f0cb49700 1 -- 172.21.15.52:0/1161931809 shutdown_connections 2024-04-02T13:43:00.733 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.101+0000 7f5f0cb49700 1 -- 172.21.15.52:0/1161931809 wait complete. 2024-04-02T13:43:00.733 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.102+0000 7f5f0cb49700 1 Processor -- start 2024-04-02T13:43:00.733 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.102+0000 7f5f0cb49700 1 -- start start 2024-04-02T13:43:00.734 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.102+0000 7f5f0cb49700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f08106fd0 0x7f5f081995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:00.734 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.102+0000 7f5f0cb49700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5f081085f0 con 0x7f5f08106fd0 2024-04-02T13:43:00.734 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.102+0000 7f5f0659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f08106fd0 0x7f5f081995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:00.734 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.102+0000 7f5f0659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f08106fd0 0x7f5f081995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37120/0 (socket says 172.21.15.52:37120) 2024-04-02T13:43:00.734 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.102+0000 7f5f0659c700 1 -- 172.21.15.52:0/3476487318 learned_addr learned my addr 172.21.15.52:0/3476487318 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:00.734 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.103+0000 7f5f0659c700 1 -- 172.21.15.52:0/3476487318 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f08199b10 con 0x7f5f08106fd0 2024-04-02T13:43:00.734 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.103+0000 7f5f0659c700 1 --2- 172.21.15.52:0/3476487318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f08106fd0 0x7f5f081995d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800b340 tx=0x7f5ef8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:00.734 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.103+0000 7f5ef77fe700 1 -- 172.21.15.52:0/3476487318 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ef8029070 con 0x7f5f08106fd0 2024-04-02T13:43:00.734 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.103+0000 7f5ef77fe700 1 -- 172.21.15.52:0/3476487318 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef80044e0 con 0x7f5f08106fd0 2024-04-02T13:43:00.734 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.103+0000 7f5ef77fe700 1 -- 172.21.15.52:0/3476487318 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ef80105a0 con 0x7f5f08106fd0 2024-04-02T13:43:00.734 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.103+0000 7f5f0cb49700 1 -- 172.21.15.52:0/3476487318 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f08199da0 con 0x7f5f08106fd0 2024-04-02T13:43:00.735 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.103+0000 7f5f0cb49700 1 -- 172.21.15.52:0/3476487318 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f0819ca40 con 0x7f5f08106fd0 2024-04-02T13:43:00.735 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.103+0000 7f5ef77fe700 1 -- 172.21.15.52:0/3476487318 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5ef802c070 con 0x7f5f08106fd0 2024-04-02T13:43:00.735 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.103+0000 7f5ef77fe700 1 -- 172.21.15.52:0/3476487318 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5ef8021070 con 0x7f5f08106fd0 2024-04-02T13:43:00.735 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.104+0000 7f5f0cb49700 1 -- 172.21.15.52:0/3476487318 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f08063c20 con 0x7f5f08106fd0 2024-04-02T13:43:00.735 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.106+0000 7f5ef77fe700 1 -- 172.21.15.52:0/3476487318 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5ef8015070 con 0x7f5f08106fd0 2024-04-02T13:43:00.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.179+0000 7f5f0cb49700 1 -- 172.21.15.52:0/3476487318 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f5f0802de80 con 0x7f5f08106fd0 2024-04-02T13:43:00.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.180+0000 7f5ef77fe700 1 -- 172.21.15.52:0/3476487318 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f5ef800dce0 con 0x7f5f08106fd0 2024-04-02T13:43:00.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.180+0000 7f5f0cb49700 1 -- 172.21.15.52:0/3476487318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f08106fd0 msgr2=0x7f5f081995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:00.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.180+0000 7f5f0cb49700 1 --2- 172.21.15.52:0/3476487318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f08106fd0 0x7f5f081995d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800b340 tx=0x7f5ef8004750 comp rx=0 tx=0).stop 2024-04-02T13:43:00.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.181+0000 7f5f0cb49700 1 -- 172.21.15.52:0/3476487318 shutdown_connections 2024-04-02T13:43:00.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.181+0000 7f5f0cb49700 1 --2- 172.21.15.52:0/3476487318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f08106fd0 0x7f5f081995d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:00.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.181+0000 7f5f0cb49700 1 -- 172.21.15.52:0/3476487318 >> 172.21.15.52:0/3476487318 conn(0x7f5f081025b0 msgr2=0x7f5f081902d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:00.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.181+0000 7f5f0cb49700 1 -- 172.21.15.52:0/3476487318 shutdown_connections 2024-04-02T13:43:00.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:00.181+0000 7f5f0cb49700 1 -- 172.21.15.52:0/3476487318 wait complete. 2024-04-02T13:43:00.736 INFO:teuthology.orchestra.run.smithi052.stdout:mon is available 2024-04-02T13:43:00.736 INFO:teuthology.orchestra.run.smithi052.stdout:Assimilating anything we can from ceph.conf... 2024-04-02T13:43:01.837 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:01 smithi052 ceph-mon[31728]: from='client.? 172.21.15.52:0/203436388' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-02T13:43:01.846 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-04-02T13:43:01.846 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [global] 2024-04-02T13:43:01.846 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout fsid = bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:43:01.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.52:3300,v1:172.21.15.52:6789] 2024-04-02T13:43:01.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-02T13:43:01.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-02T13:43:01.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-02T13:43:01.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-02T13:43:01.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-04-02T13:43:01.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-02T13:43:01.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-02T13:43:01.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-04-02T13:43:01.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [osd] 2024-04-02T13:43:01.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-02T13:43:01.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-02T13:43:01.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-02T13:43:01.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-02T13:43:01.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-02T13:43:01.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.149+0000 7f7009228700 1 Processor -- start 2024-04-02T13:43:01.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.149+0000 7f7009228700 1 -- start start 2024-04-02T13:43:01.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.149+0000 7f7009228700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7004106960 0x7f7004106d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:01.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.149+0000 7f7009228700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f700407c640 con 0x7f7004106960 2024-04-02T13:43:01.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.149+0000 7f7002d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7004106960 0x7f7004106d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:01.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.149+0000 7f7002d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7004106960 0x7f7004106d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37126/0 (socket says 172.21.15.52:37126) 2024-04-02T13:43:01.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.149+0000 7f7002d9d700 1 -- 172.21.15.52:0/3556963444 learned_addr learned my addr 172.21.15.52:0/3556963444 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:01.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.150+0000 7f7002d9d700 1 -- 172.21.15.52:0/3556963444 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70041072c0 con 0x7f7004106960 2024-04-02T13:43:01.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.150+0000 7f7002d9d700 1 --2- 172.21.15.52:0/3556963444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7004106960 0x7f7004106d80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6ff40098a0 tx=0x7f6ff4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3b3db58db66676 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:01.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.150+0000 7f7001d9b700 1 -- 172.21.15.52:0/3556963444 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ff4004160 con 0x7f7004106960 2024-04-02T13:43:01.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.150+0000 7f7001d9b700 1 -- 172.21.15.52:0/3556963444 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ff4004300 con 0x7f7004106960 2024-04-02T13:43:01.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.150+0000 7f7001d9b700 1 -- 172.21.15.52:0/3556963444 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ff40036a0 con 0x7f7004106960 2024-04-02T13:43:01.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.150+0000 7f7009228700 1 -- 172.21.15.52:0/3556963444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7004106960 msgr2=0x7f7004106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:01.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.150+0000 7f7009228700 1 --2- 172.21.15.52:0/3556963444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7004106960 0x7f7004106d80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6ff40098a0 tx=0x7f6ff4009bb0 comp rx=0 tx=0).stop 2024-04-02T13:43:01.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.151+0000 7f7009228700 1 -- 172.21.15.52:0/3556963444 shutdown_connections 2024-04-02T13:43:01.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.151+0000 7f7009228700 1 --2- 172.21.15.52:0/3556963444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7004106960 0x7f7004106d80 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:01.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.151+0000 7f7009228700 1 -- 172.21.15.52:0/3556963444 >> 172.21.15.52:0/3556963444 conn(0x7f7004102540 msgr2=0x7f7004104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:01.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.151+0000 7f7009228700 1 -- 172.21.15.52:0/3556963444 shutdown_connections 2024-04-02T13:43:01.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.151+0000 7f7009228700 1 -- 172.21.15.52:0/3556963444 wait complete. 2024-04-02T13:43:01.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.151+0000 7f7009228700 1 Processor -- start 2024-04-02T13:43:01.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.151+0000 7f7009228700 1 -- start start 2024-04-02T13:43:01.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.152+0000 7f7009228700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7004106960 0x7f700419d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:01.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.152+0000 7f7009228700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7004107670 con 0x7f7004106960 2024-04-02T13:43:01.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.152+0000 7f7002d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7004106960 0x7f700419d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:01.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.152+0000 7f7002d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7004106960 0x7f700419d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37136/0 (socket says 172.21.15.52:37136) 2024-04-02T13:43:01.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.152+0000 7f7002d9d700 1 -- 172.21.15.52:0/203436388 learned_addr learned my addr 172.21.15.52:0/203436388 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:01.850 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.152+0000 7f7002d9d700 1 -- 172.21.15.52:0/203436388 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700419de80 con 0x7f7004106960 2024-04-02T13:43:01.850 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.152+0000 7f7002d9d700 1 --2- 172.21.15.52:0/203436388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7004106960 0x7f700419d940 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4000c00 tx=0x7f6ff4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:01.850 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.152+0000 7f6febfff700 1 -- 172.21.15.52:0/203436388 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ff4003db0 con 0x7f7004106960 2024-04-02T13:43:01.850 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.152+0000 7f6febfff700 1 -- 172.21.15.52:0/203436388 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ff4011630 con 0x7f7004106960 2024-04-02T13:43:01.850 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.152+0000 7f7009228700 1 -- 172.21.15.52:0/203436388 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f700419e110 con 0x7f7004106960 2024-04-02T13:43:01.850 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.152+0000 7f6febfff700 1 -- 172.21.15.52:0/203436388 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ff4011950 con 0x7f7004106960 2024-04-02T13:43:01.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.152+0000 7f7009228700 1 -- 172.21.15.52:0/203436388 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70041a0e00 con 0x7f7004106960 2024-04-02T13:43:01.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.153+0000 7f6febfff700 1 -- 172.21.15.52:0/203436388 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6ff4011af0 con 0x7f7004106960 2024-04-02T13:43:01.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.153+0000 7f6febfff700 1 -- 172.21.15.52:0/203436388 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6ff401c450 con 0x7f7004106960 2024-04-02T13:43:01.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.153+0000 7f7009228700 1 -- 172.21.15.52:0/203436388 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7004063c20 con 0x7f7004106960 2024-04-02T13:43:01.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.156+0000 7f6febfff700 1 -- 172.21.15.52:0/203436388 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f6ff4032020 con 0x7f7004106960 2024-04-02T13:43:01.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.226+0000 7f7009228700 1 -- 172.21.15.52:0/203436388 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f70041a0fd0 con 0x7f7004106960 2024-04-02T13:43:01.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.229+0000 7f6febfff700 1 -- 172.21.15.52:0/203436388 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f6ff4004260 con 0x7f7004106960 2024-04-02T13:43:01.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.230+0000 7f6febfff700 1 -- 172.21.15.52:0/203436388 <== mon.0 v2:172.21.15.52:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6ff401b450 con 0x7f7004106960 2024-04-02T13:43:01.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.231+0000 7f7009228700 1 -- 172.21.15.52:0/203436388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7004106960 msgr2=0x7f700419d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:01.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.231+0000 7f7009228700 1 --2- 172.21.15.52:0/203436388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7004106960 0x7f700419d940 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4000c00 tx=0x7f6ff4004750 comp rx=0 tx=0).stop 2024-04-02T13:43:01.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.231+0000 7f7009228700 1 -- 172.21.15.52:0/203436388 shutdown_connections 2024-04-02T13:43:01.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.231+0000 7f7009228700 1 --2- 172.21.15.52:0/203436388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7004106960 0x7f700419d940 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:01.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.231+0000 7f7009228700 1 -- 172.21.15.52:0/203436388 >> 172.21.15.52:0/203436388 conn(0x7f7004102540 msgr2=0x7f7004193f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:01.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.231+0000 7f7009228700 1 -- 172.21.15.52:0/203436388 shutdown_connections 2024-04-02T13:43:01.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:01.231+0000 7f7009228700 1 -- 172.21.15.52:0/203436388 wait complete. 2024-04-02T13:43:01.852 INFO:teuthology.orchestra.run.smithi052.stdout:Generating new minimal ceph.conf... 2024-04-02T13:43:03.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:02 smithi052 ceph-mon[31728]: from='client.? 172.21.15.52:0/336085550' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:43:03.461 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.282+0000 7f4274905700 1 Processor -- start 2024-04-02T13:43:03.461 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.282+0000 7f4274905700 1 -- start start 2024-04-02T13:43:03.461 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.282+0000 7f4274905700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4270108b60 0x7f427010af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:03.461 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.282+0000 7f4274905700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f427007c640 con 0x7f4270108b60 2024-04-02T13:43:03.461 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.282+0000 7f426e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4270108b60 0x7f427010af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:03.461 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.283+0000 7f426e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4270108b60 0x7f427010af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44302/0 (socket says 172.21.15.52:44302) 2024-04-02T13:43:03.461 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.283+0000 7f426e59c700 1 -- 172.21.15.52:0/2600354209 learned_addr learned my addr 172.21.15.52:0/2600354209 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:03.461 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.283+0000 7f426e59c700 1 -- 172.21.15.52:0/2600354209 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427010b490 con 0x7f4270108b60 2024-04-02T13:43:03.461 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.283+0000 7f426e59c700 1 --2- 172.21.15.52:0/2600354209 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4270108b60 0x7f427010af50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4268009e80 tx=0x7f426800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba4c717cf854022 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:03.461 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.283+0000 7f426d59a700 1 -- 172.21.15.52:0/2600354209 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4268017070 con 0x7f4270108b60 2024-04-02T13:43:03.461 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.283+0000 7f426d59a700 1 -- 172.21.15.52:0/2600354209 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f426800b8c0 con 0x7f4270108b60 2024-04-02T13:43:03.462 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.283+0000 7f426d59a700 1 -- 172.21.15.52:0/2600354209 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42680045c0 con 0x7f4270108b60 2024-04-02T13:43:03.462 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.284+0000 7f4274905700 1 -- 172.21.15.52:0/2600354209 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4270108b60 msgr2=0x7f427010af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:03.462 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.284+0000 7f4274905700 1 --2- 172.21.15.52:0/2600354209 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4270108b60 0x7f427010af50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4268009e80 tx=0x7f426800b330 comp rx=0 tx=0).stop 2024-04-02T13:43:03.462 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.284+0000 7f4274905700 1 -- 172.21.15.52:0/2600354209 shutdown_connections 2024-04-02T13:43:03.462 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.284+0000 7f4274905700 1 --2- 172.21.15.52:0/2600354209 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4270108b60 0x7f427010af50 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:03.462 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.284+0000 7f4274905700 1 -- 172.21.15.52:0/2600354209 >> 172.21.15.52:0/2600354209 conn(0x7f4270102530 msgr2=0x7f4270104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:03.462 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.284+0000 7f4274905700 1 -- 172.21.15.52:0/2600354209 shutdown_connections 2024-04-02T13:43:03.462 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.284+0000 7f4274905700 1 -- 172.21.15.52:0/2600354209 wait complete. 2024-04-02T13:43:03.462 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.285+0000 7f4274905700 1 Processor -- start 2024-04-02T13:43:03.462 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.285+0000 7f4274905700 1 -- start start 2024-04-02T13:43:03.462 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.285+0000 7f4274905700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4270108b60 0x7f42701083e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:03.462 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.285+0000 7f4274905700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f427010c3f0 con 0x7f4270108b60 2024-04-02T13:43:03.462 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.285+0000 7f426e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4270108b60 0x7f42701083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:03.463 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.285+0000 7f426e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4270108b60 0x7f42701083e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44318/0 (socket says 172.21.15.52:44318) 2024-04-02T13:43:03.463 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.285+0000 7f426e59c700 1 -- 172.21.15.52:0/336085550 learned_addr learned my addr 172.21.15.52:0/336085550 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:03.463 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.285+0000 7f426e59c700 1 -- 172.21.15.52:0/336085550 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4270108920 con 0x7f4270108b60 2024-04-02T13:43:03.463 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.285+0000 7f426e59c700 1 --2- 172.21.15.52:0/336085550 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4270108b60 0x7f42701083e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f42680072a0 tx=0x7f426800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:03.463 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.286+0000 7f42677fe700 1 -- 172.21.15.52:0/336085550 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4268020020 con 0x7f4270108b60 2024-04-02T13:43:03.463 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.286+0000 7f42677fe700 1 -- 172.21.15.52:0/336085550 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4268003b50 con 0x7f4270108b60 2024-04-02T13:43:03.463 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.286+0000 7f4274905700 1 -- 172.21.15.52:0/336085550 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4270106b10 con 0x7f4270108b60 2024-04-02T13:43:03.463 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.286+0000 7f42677fe700 1 -- 172.21.15.52:0/336085550 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4268017040 con 0x7f4270108b60 2024-04-02T13:43:03.463 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.286+0000 7f4274905700 1 -- 172.21.15.52:0/336085550 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4270107050 con 0x7f4270108b60 2024-04-02T13:43:03.463 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.286+0000 7f42677fe700 1 -- 172.21.15.52:0/336085550 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4268023020 con 0x7f4270108b60 2024-04-02T13:43:03.463 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.286+0000 7f42677fe700 1 -- 172.21.15.52:0/336085550 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f426800f050 con 0x7f4270108b60 2024-04-02T13:43:03.463 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.287+0000 7f4274905700 1 -- 172.21.15.52:0/336085550 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4270063c20 con 0x7f4270108b60 2024-04-02T13:43:03.463 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.289+0000 7f42677fe700 1 -- 172.21.15.52:0/336085550 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f426802a020 con 0x7f4270108b60 2024-04-02T13:43:03.464 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.360+0000 7f4274905700 1 -- 172.21.15.52:0/336085550 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f42701072a0 con 0x7f4270108b60 2024-04-02T13:43:03.464 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.360+0000 7f42677fe700 1 -- 172.21.15.52:0/336085550 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f426800b860 con 0x7f4270108b60 2024-04-02T13:43:03.464 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.361+0000 7f4274905700 1 -- 172.21.15.52:0/336085550 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4270108b60 msgr2=0x7f42701083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:03.464 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.361+0000 7f4274905700 1 --2- 172.21.15.52:0/336085550 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4270108b60 0x7f42701083e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f42680072a0 tx=0x7f426800bd50 comp rx=0 tx=0).stop 2024-04-02T13:43:03.464 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.361+0000 7f4274905700 1 -- 172.21.15.52:0/336085550 shutdown_connections 2024-04-02T13:43:03.464 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.361+0000 7f4274905700 1 --2- 172.21.15.52:0/336085550 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4270108b60 0x7f42701083e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:03.464 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.361+0000 7f4274905700 1 -- 172.21.15.52:0/336085550 >> 172.21.15.52:0/336085550 conn(0x7f4270102530 msgr2=0x7f4270103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:03.464 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.361+0000 7f4274905700 1 -- 172.21.15.52:0/336085550 shutdown_connections 2024-04-02T13:43:03.464 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:02.361+0000 7f4274905700 1 -- 172.21.15.52:0/336085550 wait complete. 2024-04-02T13:43:03.464 INFO:teuthology.orchestra.run.smithi052.stdout:Restarting the monitor... 2024-04-02T13:43:03.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:03 smithi052 systemd[1]: Stopping Ceph mon.a for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T13:43:03.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:03 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-a[31724]: 2024-04-02T13:43:03.720+0000 7f64bcb77700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:43:03.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:03 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-a[31724]: 2024-04-02T13:43:03.720+0000 7f64bcb77700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-02T13:43:04.189 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:03 smithi052 podman[31973]: 2024-04-02 13:43:03.935321062 +0000 UTC m=+0.308660296 container died 9ece0d496bf9617f018a5088a0ca45b7ecab8b78132b5a97f56d390b65867b50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-02T13:43:04.548 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:04 smithi052 podman[31973]: 2024-04-02 13:43:04.295480615 +0000 UTC m=+0.668819880 container cleanup 9ece0d496bf9617f018a5088a0ca45b7ecab8b78132b5a97f56d390b65867b50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-a, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-02T13:43:04.548 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:04 smithi052 bash[31973]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-a 2024-04-02T13:43:04.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:04 smithi052 podman[31987]: 2024-04-02 13:43:04.546308839 +0000 UTC m=+0.611019309 container remove 9ece0d496bf9617f018a5088a0ca45b7ecab8b78132b5a97f56d390b65867b50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-a, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:43:04.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:04 smithi052 systemd[1]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mon.a.service: Deactivated successfully. 2024-04-02T13:43:04.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:04 smithi052 systemd[1]: Stopped Ceph mon.a for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4. 2024-04-02T13:43:04.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:04 smithi052 systemd[1]: Starting Ceph mon.a for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T13:43:05.194 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:04 smithi052 podman[32085]: 2024-04-02 13:43:04.827078622 +0000 UTC m=+0.017448006 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:43:05.194 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:04 smithi052 podman[32085]: 2024-04-02 13:43:04.947411295 +0000 UTC m=+0.137780657 container create bcb19f26fc2831378bb2651f24afeb62329f351e7576f203ccbddca409ea7310 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-02T13:43:05.264 INFO:teuthology.orchestra.run.smithi052.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-02T13:43:05.480 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 podman[32085]: 2024-04-02 13:43:05.191542114 +0000 UTC m=+0.381911490 container init bcb19f26fc2831378bb2651f24afeb62329f351e7576f203ccbddca409ea7310 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True) 2024-04-02T13:43:05.480 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 podman[32085]: 2024-04-02 13:43:05.194763842 +0000 UTC m=+0.385133205 container start bcb19f26fc2831378bb2651f24afeb62329f351e7576f203ccbddca409ea7310 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0) 2024-04-02T13:43:05.480 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: set uid:gid to 167:167 (ceph:ceph) 2024-04-02T13:43:05.480 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev), process ceph-mon, pid 2 2024-04-02T13:43:05.480 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: pidfile_write: ignore empty --pid-file 2024-04-02T13:43:05.481 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: load: jerasure load: lrc 2024-04-02T13:43:05.481 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: RocksDB version: 7.9.2 2024-04-02T13:43:05.481 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Git sha 0 2024-04-02T13:43:05.481 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Compile date 2024-04-02 01:02:34 2024-04-02T13:43:05.481 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: DB SUMMARY 2024-04-02T13:43:05.481 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: DB Session ID: 2YPKCDIZG1LCFN7SGKAD 2024-04-02T13:43:05.481 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: CURRENT file: CURRENT 2024-04-02T13:43:05.481 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: IDENTITY file: IDENTITY 2024-04-02T13:43:05.481 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-02T13:43:05.481 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-02T13:43:05.482 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105656 ; 2024-04-02T13:43:05.482 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.error_if_exists: 0 2024-04-02T13:43:05.482 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.create_if_missing: 0 2024-04-02T13:43:05.482 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.paranoid_checks: 1 2024-04-02T13:43:05.482 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-02T13:43:05.482 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-02T13:43:05.482 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-02T13:43:05.482 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.env: 0x55a892e88c60 2024-04-02T13:43:05.483 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.fs: PosixFileSystem 2024-04-02T13:43:05.483 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.info_log: 0x55a895697ae0 2024-04-02T13:43:05.483 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_file_opening_threads: 16 2024-04-02T13:43:05.483 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.statistics: (nil) 2024-04-02T13:43:05.483 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.use_fsync: 0 2024-04-02T13:43:05.483 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_log_file_size: 0 2024-04-02T13:43:05.483 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-02T13:43:05.483 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-02T13:43:05.483 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.keep_log_file_num: 1000 2024-04-02T13:43:05.483 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.recycle_log_file_num: 0 2024-04-02T13:43:05.483 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.allow_fallocate: 1 2024-04-02T13:43:05.483 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.allow_mmap_reads: 0 2024-04-02T13:43:05.484 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.allow_mmap_writes: 0 2024-04-02T13:43:05.485 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.use_direct_reads: 0 2024-04-02T13:43:05.485 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-02T13:43:05.485 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.create_missing_column_families: 0 2024-04-02T13:43:05.485 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.db_log_dir: 2024-04-02T13:43:05.486 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.wal_dir: 2024-04-02T13:43:05.486 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-02T13:43:05.486 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-02T13:43:05.486 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-02T13:43:05.486 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-02T13:43:05.487 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-02T13:43:05.487 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-02T13:43:05.487 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.advise_random_on_open: 1 2024-04-02T13:43:05.487 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.db_write_buffer_size: 0 2024-04-02T13:43:05.487 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.write_buffer_manager: 0x55a8956681e0 2024-04-02T13:43:05.487 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-02T13:43:05.488 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-02T13:43:05.488 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-02T13:43:05.488 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.rate_limiter: (nil) 2024-04-02T13:43:05.488 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-02T13:43:05.488 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.wal_recovery_mode: 2 2024-04-02T13:43:05.488 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.enable_thread_tracking: 0 2024-04-02T13:43:05.489 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.enable_pipelined_write: 0 2024-04-02T13:43:05.489 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.unordered_write: 0 2024-04-02T13:43:05.489 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-02T13:43:05.489 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-02T13:43:05.489 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-02T13:43:05.489 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-02T13:43:05.490 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.row_cache: None 2024-04-02T13:43:05.490 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.wal_filter: None 2024-04-02T13:43:05.494 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-02T13:43:05.494 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.allow_ingest_behind: 0 2024-04-02T13:43:05.494 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.two_write_queues: 0 2024-04-02T13:43:05.494 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.manual_wal_flush: 0 2024-04-02T13:43:05.494 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.wal_compression: 0 2024-04-02T13:43:05.494 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.atomic_flush: 0 2024-04-02T13:43:05.494 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-02T13:43:05.494 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-02T13:43:05.495 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-02T13:43:05.495 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.log_readahead_size: 0 2024-04-02T13:43:05.495 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-02T13:43:05.495 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.best_efforts_recovery: 0 2024-04-02T13:43:05.495 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-02T13:43:05.495 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-02T13:43:05.495 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.allow_data_in_errors: 0 2024-04-02T13:43:05.495 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.db_host_id: __hostname__ 2024-04-02T13:43:05.495 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-02T13:43:05.495 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_background_jobs: 2 2024-04-02T13:43:05.495 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_background_compactions: -1 2024-04-02T13:43:05.495 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_subcompactions: 1 2024-04-02T13:43:05.495 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-02T13:43:05.495 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-02T13:43:05.495 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-02T13:43:05.496 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_total_wal_size: 0 2024-04-02T13:43:05.496 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-02T13:43:05.496 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-02T13:43:05.496 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-02T13:43:05.496 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-02T13:43:05.496 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_open_files: -1 2024-04-02T13:43:05.496 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.bytes_per_sync: 0 2024-04-02T13:43:05.496 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-02T13:43:05.496 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-02T13:43:05.496 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.compaction_readahead_size: 0 2024-04-02T13:43:05.496 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_background_flushes: -1 2024-04-02T13:43:05.496 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Compression algorithms supported: 2024-04-02T13:43:05.496 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-02T13:43:05.496 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: kZSTD supported: 0 2024-04-02T13:43:05.497 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: kXpressCompression supported: 0 2024-04-02T13:43:05.497 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: kLZ4HCCompression supported: 1 2024-04-02T13:43:05.497 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: kZlibCompression supported: 1 2024-04-02T13:43:05.497 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: kSnappyCompression supported: 1 2024-04-02T13:43:05.497 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: kLZ4Compression supported: 1 2024-04-02T13:43:05.497 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: kBZip2Compression supported: 0 2024-04-02T13:43:05.497 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-02T13:43:05.497 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-02T13:43:05.497 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-02T13:43:05.497 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-02T13:43:05.497 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-02T13:43:05.497 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.merge_operator: 2024-04-02T13:43:05.497 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.compaction_filter: None 2024-04-02T13:43:05.497 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.compaction_filter_factory: None 2024-04-02T13:43:05.497 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.sst_partitioner_factory: None 2024-04-02T13:43:05.498 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-02T13:43:05.498 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-02T13:43:05.498 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a895697cc0) 2024-04-02T13:43:05.498 INFO:journalctl@ceph.mon.a.smithi052.stdout: cache_index_and_filter_blocks: 1 2024-04-02T13:43:05.498 INFO:journalctl@ceph.mon.a.smithi052.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-02T13:43:05.498 INFO:journalctl@ceph.mon.a.smithi052.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-02T13:43:05.498 INFO:journalctl@ceph.mon.a.smithi052.stdout: pin_top_level_index_and_filter: 1 2024-04-02T13:43:05.498 INFO:journalctl@ceph.mon.a.smithi052.stdout: index_type: 0 2024-04-02T13:43:05.498 INFO:journalctl@ceph.mon.a.smithi052.stdout: data_block_index_type: 0 2024-04-02T13:43:05.498 INFO:journalctl@ceph.mon.a.smithi052.stdout: index_shortening: 1 2024-04-02T13:43:05.498 INFO:journalctl@ceph.mon.a.smithi052.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-02T13:43:05.498 INFO:journalctl@ceph.mon.a.smithi052.stdout: checksum: 4 2024-04-02T13:43:05.498 INFO:journalctl@ceph.mon.a.smithi052.stdout: no_block_cache: 0 2024-04-02T13:43:05.498 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_cache: 0x55a89569b090 2024-04-02T13:43:05.499 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_cache_name: BinnedLRUCache 2024-04-02T13:43:05.499 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_cache_options: 2024-04-02T13:43:05.499 INFO:journalctl@ceph.mon.a.smithi052.stdout: capacity : 536870912 2024-04-02T13:43:05.499 INFO:journalctl@ceph.mon.a.smithi052.stdout: num_shard_bits : 4 2024-04-02T13:43:05.499 INFO:journalctl@ceph.mon.a.smithi052.stdout: strict_capacity_limit : 0 2024-04-02T13:43:05.499 INFO:journalctl@ceph.mon.a.smithi052.stdout: high_pri_pool_ratio: 0.000 2024-04-02T13:43:05.499 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_cache_compressed: (nil) 2024-04-02T13:43:05.499 INFO:journalctl@ceph.mon.a.smithi052.stdout: persistent_cache: (nil) 2024-04-02T13:43:05.499 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_size: 4096 2024-04-02T13:43:05.499 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_size_deviation: 10 2024-04-02T13:43:05.499 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_restart_interval: 16 2024-04-02T13:43:05.499 INFO:journalctl@ceph.mon.a.smithi052.stdout: index_block_restart_interval: 1 2024-04-02T13:43:05.499 INFO:journalctl@ceph.mon.a.smithi052.stdout: metadata_block_size: 4096 2024-04-02T13:43:05.499 INFO:journalctl@ceph.mon.a.smithi052.stdout: partition_filters: 0 2024-04-02T13:43:05.499 INFO:journalctl@ceph.mon.a.smithi052.stdout: use_delta_encoding: 1 2024-04-02T13:43:05.500 INFO:journalctl@ceph.mon.a.smithi052.stdout: filter_policy: bloomfilter 2024-04-02T13:43:05.500 INFO:journalctl@ceph.mon.a.smithi052.stdout: whole_key_filtering: 1 2024-04-02T13:43:05.500 INFO:journalctl@ceph.mon.a.smithi052.stdout: verify_compression: 0 2024-04-02T13:43:05.500 INFO:journalctl@ceph.mon.a.smithi052.stdout: read_amp_bytes_per_bit: 0 2024-04-02T13:43:05.500 INFO:journalctl@ceph.mon.a.smithi052.stdout: format_version: 5 2024-04-02T13:43:05.500 INFO:journalctl@ceph.mon.a.smithi052.stdout: enable_index_compression: 1 2024-04-02T13:43:05.500 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_align: 0 2024-04-02T13:43:05.500 INFO:journalctl@ceph.mon.a.smithi052.stdout: max_auto_readahead_size: 262144 2024-04-02T13:43:05.500 INFO:journalctl@ceph.mon.a.smithi052.stdout: prepopulate_block_cache: 0 2024-04-02T13:43:05.500 INFO:journalctl@ceph.mon.a.smithi052.stdout: initial_auto_readahead_size: 8192 2024-04-02T13:43:05.500 INFO:journalctl@ceph.mon.a.smithi052.stdout: num_file_reads_for_auto_readahead: 2 2024-04-02T13:43:05.500 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.write_buffer_size: 33554432 2024-04-02T13:43:05.500 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_write_buffer_number: 2 2024-04-02T13:43:05.500 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.compression: NoCompression 2024-04-02T13:43:05.500 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.bottommost_compression: Disabled 2024-04-02T13:43:05.501 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.prefix_extractor: nullptr 2024-04-02T13:43:05.501 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-02T13:43:05.501 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.num_levels: 7 2024-04-02T13:43:05.501 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-02T13:43:05.501 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-02T13:43:05.501 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-02T13:43:05.501 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-02T13:43:05.501 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-02T13:43:05.501 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-02T13:43:05.501 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-02T13:43:05.501 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-02T13:43:05.501 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-02T13:43:05.501 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-02T13:43:05.501 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-02T13:43:05.501 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-02T13:43:05.502 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-02T13:43:05.502 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.compression_opts.level: 32767 2024-04-02T13:43:05.502 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.compression_opts.strategy: 0 2024-04-02T13:43:05.502 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-02T13:43:05.502 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-02T13:43:05.502 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-02T13:43:05.502 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-02T13:43:05.502 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.compression_opts.enabled: false 2024-04-02T13:43:05.502 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-02T13:43:05.502 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-02T13:43:05.502 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-02T13:43:05.502 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-02T13:43:05.502 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.target_file_size_base: 67108864 2024-04-02T13:43:05.502 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-02T13:43:05.503 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-02T13:43:05.503 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-02T13:43:05.503 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-02T13:43:05.503 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-02T13:43:05.503 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-02T13:43:05.503 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-02T13:43:05.503 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-02T13:43:05.503 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-02T13:43:05.503 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-02T13:43:05.503 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-02T13:43:05.503 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-02T13:43:05.503 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-02T13:43:05.503 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-02T13:43:05.503 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.arena_block_size: 1048576 2024-04-02T13:43:05.503 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-02T13:43:05.504 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-02T13:43:05.504 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.disable_auto_compactions: 0 2024-04-02T13:43:05.504 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-02T13:43:05.504 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-02T13:43:05.504 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-02T13:43:05.504 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-02T13:43:05.504 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-02T13:43:05.504 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-02T13:43:05.504 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-02T13:43:05.504 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-02T13:43:05.504 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-02T13:43:05.504 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-02T13:43:05.504 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-02T13:43:05.504 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.inplace_update_support: 0 2024-04-02T13:43:05.504 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-02T13:43:05.505 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-02T13:43:05.505 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-02T13:43:05.505 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-02T13:43:05.505 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.bloom_locality: 0 2024-04-02T13:43:05.505 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.max_successive_merges: 0 2024-04-02T13:43:05.505 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-02T13:43:05.505 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.paranoid_file_checks: 0 2024-04-02T13:43:05.505 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.force_consistency_checks: 1 2024-04-02T13:43:05.505 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.report_bg_io_stats: 0 2024-04-02T13:43:05.505 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.ttl: 2592000 2024-04-02T13:43:05.505 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-02T13:43:05.505 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-02T13:43:05.505 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-02T13:43:05.505 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.enable_blob_files: false 2024-04-02T13:43:05.505 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.min_blob_size: 0 2024-04-02T13:43:05.506 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.blob_file_size: 268435456 2024-04-02T13:43:05.506 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-02T13:43:05.506 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-02T13:43:05.506 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-02T13:43:05.506 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-02T13:43:05.506 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-02T13:43:05.506 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.blob_file_starting_level: 0 2024-04-02T13:43:05.506 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-02T13:43:05.506 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-02T13:43:05.506 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-02T13:43:05.506 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 75d33ee5-e659-4ddb-be68-a479f69e635e 2024-04-02T13:43:05.506 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712065385223385, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-02T13:43:05.506 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-02T13:43:05.506 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712065385224453, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100715, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98609, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14308, "raw_average_key_size": 49, "raw_value_size": 90863, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712065385, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "75d33ee5-e659-4ddb-be68-a479f69e635e", "db_session_id": "2YPKCDIZG1LCFN7SGKAD", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-02T13:43:05.507 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712065385224537, "job": 1, "event": "recovery_finished"} 2024-04-02T13:43:05.507 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-02T13:43:05.507 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-02T13:43:05.507 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a89578a000 2024-04-02T13:43:05.507 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: DB pointer 0x55a895774000 2024-04-02T13:43:05.507 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-02T13:43:05.507 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-02T13:43:05.507 INFO:journalctl@ceph.mon.a.smithi052.stdout: ** DB Stats ** 2024-04-02T13:43:05.507 INFO:journalctl@ceph.mon.a.smithi052.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T13:43:05.507 INFO:journalctl@ceph.mon.a.smithi052.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-02T13:43:05.507 INFO:journalctl@ceph.mon.a.smithi052.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T13:43:05.507 INFO:journalctl@ceph.mon.a.smithi052.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T13:43:05.507 INFO:journalctl@ceph.mon.a.smithi052.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-02T13:43:05.507 INFO:journalctl@ceph.mon.a.smithi052.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T13:43:05.507 INFO:journalctl@ceph.mon.a.smithi052.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T13:43:05.508 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2024-04-02T13:43:05.508 INFO:journalctl@ceph.mon.a.smithi052.stdout: ** Compaction Stats [default] ** 2024-04-02T13:43:05.508 INFO:journalctl@ceph.mon.a.smithi052.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T13:43:05.508 INFO:journalctl@ceph.mon.a.smithi052.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-02T13:43:05.508 INFO:journalctl@ceph.mon.a.smithi052.stdout: L0 2/0 100.21 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 133.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-02T13:43:05.508 INFO:journalctl@ceph.mon.a.smithi052.stdout: Sum 2/0 100.21 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 133.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-02T13:43:05.508 INFO:journalctl@ceph.mon.a.smithi052.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 133.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-02T13:43:05.508 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2024-04-02T13:43:05.508 INFO:journalctl@ceph.mon.a.smithi052.stdout: ** Compaction Stats [default] ** 2024-04-02T13:43:05.508 INFO:journalctl@ceph.mon.a.smithi052.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T13:43:05.508 INFO:journalctl@ceph.mon.a.smithi052.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-02T13:43:05.508 INFO:journalctl@ceph.mon.a.smithi052.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 133.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-02T13:43:05.508 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2024-04-02T13:43:05.508 INFO:journalctl@ceph.mon.a.smithi052.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-02T13:43:05.508 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2024-04-02T13:43:05.509 INFO:journalctl@ceph.mon.a.smithi052.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T13:43:05.509 INFO:journalctl@ceph.mon.a.smithi052.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-02T13:43:05.509 INFO:journalctl@ceph.mon.a.smithi052.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-02T13:43:05.509 INFO:journalctl@ceph.mon.a.smithi052.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-02T13:43:05.509 INFO:journalctl@ceph.mon.a.smithi052.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-02T13:43:05.509 INFO:journalctl@ceph.mon.a.smithi052.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-02T13:43:05.509 INFO:journalctl@ceph.mon.a.smithi052.stdout: Cumulative compaction: 0.00 GB write, 31.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T13:43:05.509 INFO:journalctl@ceph.mon.a.smithi052.stdout: Interval compaction: 0.00 GB write, 31.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T13:43:05.509 INFO:journalctl@ceph.mon.a.smithi052.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-02T13:43:05.509 INFO:journalctl@ceph.mon.a.smithi052.stdout: Block cache BinnedLRUCache@0x55a89569b090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-02T13:43:05.509 INFO:journalctl@ceph.mon.a.smithi052.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-02T13:43:05.509 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2024-04-02T13:43:05.509 INFO:journalctl@ceph.mon.a.smithi052.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-02T13:43:05.509 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: starting mon.a rank 0 at public addrs [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] at bind addrs [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:43:05.509 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: mon.a@-1(???) e1 preinit fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:43:05.510 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: mon.a@-1(???).mds e1 new map 2024-04-02T13:43:05.510 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: mon.a@-1(???).mds e1 print_map 2024-04-02T13:43:05.510 INFO:journalctl@ceph.mon.a.smithi052.stdout: e1 2024-04-02T13:43:05.510 INFO:journalctl@ceph.mon.a.smithi052.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T13:43:05.510 INFO:journalctl@ceph.mon.a.smithi052.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T13:43:05.510 INFO:journalctl@ceph.mon.a.smithi052.stdout: legacy client fscid: -1 2024-04-02T13:43:05.510 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2024-04-02T13:43:05.510 INFO:journalctl@ceph.mon.a.smithi052.stdout: No filesystems configured 2024-04-02T13:43:05.510 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-02T13:43:05.510 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T13:43:05.510 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T13:43:05.510 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T13:43:05.510 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 ceph-mon[32104]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-02T13:43:05.510 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 bash[32085]: bcb19f26fc2831378bb2651f24afeb62329f351e7576f203ccbddca409ea7310 2024-04-02T13:43:05.510 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:05 smithi052 systemd[1]: Started Ceph mon.a for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4. 2024-04-02T13:43:06.344 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.672+0000 7f18c73b2700 1 Processor -- start 2024-04-02T13:43:06.344 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.673+0000 7f18c73b2700 1 -- start start 2024-04-02T13:43:06.344 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.673+0000 7f18c73b2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c007e6b0 0x7f18c007cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:06.344 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.673+0000 7f18c73b2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f18c007d130 con 0x7f18c007e6b0 2024-04-02T13:43:06.344 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.673+0000 7f18c514e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c007e6b0 0x7f18c007cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:06.345 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.673+0000 7f18c514e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c007e6b0 0x7f18c007cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44328/0 (socket says 172.21.15.52:44328) 2024-04-02T13:43:06.345 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.673+0000 7f18c514e700 1 -- 172.21.15.52:0/3043506918 learned_addr learned my addr 172.21.15.52:0/3043506918 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:06.345 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.674+0000 7f18c514e700 1 -- 172.21.15.52:0/3043506918 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c007d2b0 con 0x7f18c007e6b0 2024-04-02T13:43:06.345 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.674+0000 7f18c514e700 1 --2- 172.21.15.52:0/3043506918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c007e6b0 0x7f18c007cb60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0098a0 tx=0x7f18bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9da2a8c5cffb6e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:06.345 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.674+0000 7f18b7fff700 1 -- 172.21.15.52:0/3043506918 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18bc004160 con 0x7f18c007e6b0 2024-04-02T13:43:06.345 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.674+0000 7f18b7fff700 1 -- 172.21.15.52:0/3043506918 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f18bc004300 con 0x7f18c007e6b0 2024-04-02T13:43:06.345 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.674+0000 7f18b7fff700 1 -- 172.21.15.52:0/3043506918 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18bc011b50 con 0x7f18c007e6b0 2024-04-02T13:43:06.346 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.674+0000 7f18c73b2700 1 -- 172.21.15.52:0/3043506918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c007e6b0 msgr2=0x7f18c007cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:06.346 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.674+0000 7f18c73b2700 1 --2- 172.21.15.52:0/3043506918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c007e6b0 0x7f18c007cb60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0098a0 tx=0x7f18bc009bb0 comp rx=0 tx=0).stop 2024-04-02T13:43:06.346 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.674+0000 7f18c73b2700 1 -- 172.21.15.52:0/3043506918 shutdown_connections 2024-04-02T13:43:06.346 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.674+0000 7f18c73b2700 1 --2- 172.21.15.52:0/3043506918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c007e6b0 0x7f18c007cb60 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:06.346 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.674+0000 7f18c73b2700 1 -- 172.21.15.52:0/3043506918 >> 172.21.15.52:0/3043506918 conn(0x7f18c01036b0 msgr2=0x7f18c0105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:06.346 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.675+0000 7f18c73b2700 1 -- 172.21.15.52:0/3043506918 shutdown_connections 2024-04-02T13:43:06.346 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.675+0000 7f18c73b2700 1 -- 172.21.15.52:0/3043506918 wait complete. 2024-04-02T13:43:06.346 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.675+0000 7f18c73b2700 1 Processor -- start 2024-04-02T13:43:06.346 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.675+0000 7f18c73b2700 1 -- start start 2024-04-02T13:43:06.346 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.675+0000 7f18c73b2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c007e6b0 0x7f18c019e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:06.346 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.675+0000 7f18c73b2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f18c010e680 con 0x7f18c007e6b0 2024-04-02T13:43:06.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.676+0000 7f18c514e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c007e6b0 0x7f18c019e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:06.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.676+0000 7f18c514e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c007e6b0 0x7f18c019e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44342/0 (socket says 172.21.15.52:44342) 2024-04-02T13:43:06.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.676+0000 7f18c514e700 1 -- 172.21.15.52:0/239200919 learned_addr learned my addr 172.21.15.52:0/239200919 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:06.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.676+0000 7f18c514e700 1 -- 172.21.15.52:0/239200919 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c019e730 con 0x7f18c007e6b0 2024-04-02T13:43:06.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.676+0000 7f18c514e700 1 --2- 172.21.15.52:0/239200919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c007e6b0 0x7f18c019e1f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f18bc011670 tx=0x7f18bc011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:06.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.676+0000 7f18b67fc700 1 -- 172.21.15.52:0/239200919 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18bc0039c0 con 0x7f18c007e6b0 2024-04-02T13:43:06.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.676+0000 7f18c73b2700 1 -- 172.21.15.52:0/239200919 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c019e9c0 con 0x7f18c007e6b0 2024-04-02T13:43:06.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.676+0000 7f18b67fc700 1 -- 172.21.15.52:0/239200919 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f18bc011e50 con 0x7f18c007e6b0 2024-04-02T13:43:06.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.676+0000 7f18b67fc700 1 -- 172.21.15.52:0/239200919 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18bc01a5c0 con 0x7f18c007e6b0 2024-04-02T13:43:06.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.676+0000 7f18c73b2700 1 -- 172.21.15.52:0/239200919 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c01a16b0 con 0x7f18c007e6b0 2024-04-02T13:43:06.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.676+0000 7f18b67fc700 1 -- 172.21.15.52:0/239200919 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f18bc01ac50 con 0x7f18c007e6b0 2024-04-02T13:43:06.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.676+0000 7f18b67fc700 1 -- 172.21.15.52:0/239200919 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f18bc019700 con 0x7f18c007e6b0 2024-04-02T13:43:06.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.677+0000 7f18c73b2700 1 -- 172.21.15.52:0/239200919 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18c0063c20 con 0x7f18c007e6b0 2024-04-02T13:43:06.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.679+0000 7f18b67fc700 1 -- 172.21.15.52:0/239200919 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f18bc003b60 con 0x7f18c007e6b0 2024-04-02T13:43:06.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.749+0000 7f18c73b2700 1 -- 172.21.15.52:0/239200919 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f18c01a1880 con 0x7f18c007e6b0 2024-04-02T13:43:06.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.751+0000 7f18b67fc700 1 -- 172.21.15.52:0/239200919 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f18bc019900 con 0x7f18c007e6b0 2024-04-02T13:43:06.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.752+0000 7f18c73b2700 1 -- 172.21.15.52:0/239200919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c007e6b0 msgr2=0x7f18c019e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:06.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.752+0000 7f18c73b2700 1 --2- 172.21.15.52:0/239200919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c007e6b0 0x7f18c019e1f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f18bc011670 tx=0x7f18bc011b60 comp rx=0 tx=0).stop 2024-04-02T13:43:06.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.752+0000 7f18c73b2700 1 -- 172.21.15.52:0/239200919 shutdown_connections 2024-04-02T13:43:06.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.752+0000 7f18c73b2700 1 --2- 172.21.15.52:0/239200919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18c007e6b0 0x7f18c019e1f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:06.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.752+0000 7f18c73b2700 1 -- 172.21.15.52:0/239200919 >> 172.21.15.52:0/239200919 conn(0x7f18c01036b0 msgr2=0x7f18c0104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:06.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.752+0000 7f18c73b2700 1 -- 172.21.15.52:0/239200919 shutdown_connections 2024-04-02T13:43:06.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:05.752+0000 7f18c73b2700 1 -- 172.21.15.52:0/239200919 wait complete. 2024-04-02T13:43:06.348 INFO:teuthology.orchestra.run.smithi052.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-02T13:43:06.366 INFO:teuthology.orchestra.run.smithi052.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-02T13:43:06.366 INFO:teuthology.orchestra.run.smithi052.stdout:Creating mgr... 2024-04-02T13:43:06.366 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-02T13:43:06.367 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-02T13:43:06.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:06 smithi052 ceph-mon[32104]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-02T13:43:06.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:06 smithi052 ceph-mon[32104]: monmap epoch 1 2024-04-02T13:43:06.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:06 smithi052 ceph-mon[32104]: fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:43:06.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:06 smithi052 ceph-mon[32104]: last_changed 2024-04-02T13:42:54.002665+0000 2024-04-02T13:43:06.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:06 smithi052 ceph-mon[32104]: created 2024-04-02T13:42:54.002665+0000 2024-04-02T13:43:06.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:06 smithi052 ceph-mon[32104]: min_mon_release 19 (squid) 2024-04-02T13:43:06.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:06 smithi052 ceph-mon[32104]: election_strategy: 1 2024-04-02T13:43:06.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:06 smithi052 ceph-mon[32104]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-04-02T13:43:06.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:06 smithi052 ceph-mon[32104]: fsmap 2024-04-02T13:43:06.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:06 smithi052 ceph-mon[32104]: osdmap e1: 0 total, 0 up, 0 in 2024-04-02T13:43:06.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:06 smithi052 ceph-mon[32104]: mgrmap e1: no daemons active 2024-04-02T13:43:06.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:06 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/239200919' entity='client.admin' 2024-04-02T13:43:06.619 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mgr.x 2024-04-02T13:43:06.619 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Failed to reset failed state of unit ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mgr.x.service: Unit ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mgr.x.service not loaded. 2024-04-02T13:43:06.763 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4.target.wants/ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mgr.x.service → /etc/systemd/system/ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@.service. 2024-04-02T13:43:07.000 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:06 smithi052 systemd[1]: Starting Ceph mgr.x for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T13:43:07.305 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:06 smithi052 podman[32323]: 2024-04-02 13:43:06.900015389 +0000 UTC m=+0.017559568 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:43:07.305 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:07 smithi052 podman[32323]: 2024-04-02 13:43:07.008317838 +0000 UTC m=+0.125862042 container create c88f5d3e94020556dac8920fd178408575b666be184f1106a35aff0e380c6a39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0) 2024-04-02T13:43:07.305 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:07 smithi052 podman[32323]: 2024-04-02 13:43:07.227301338 +0000 UTC m=+0.344845513 container init c88f5d3e94020556dac8920fd178408575b666be184f1106a35aff0e380c6a39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-02T13:43:07.305 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:07 smithi052 podman[32323]: 2024-04-02 13:43:07.229775676 +0000 UTC m=+0.347319837 container start c88f5d3e94020556dac8920fd178408575b666be184f1106a35aff0e380c6a39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0) 2024-04-02T13:43:07.305 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:07 smithi052 bash[32323]: c88f5d3e94020556dac8920fd178408575b666be184f1106a35aff0e380c6a39 2024-04-02T13:43:07.318 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T13:43:07.318 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout disabled 2024-04-02T13:43:07.325 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T13:43:07.325 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2024-04-02T13:43:07.325 INFO:teuthology.orchestra.run.smithi052.stdout:firewalld.service is not enabled 2024-04-02T13:43:07.325 INFO:teuthology.orchestra.run.smithi052.stdout:Not possible to enable service . firewalld.service is not available 2024-04-02T13:43:07.334 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T13:43:07.334 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout disabled 2024-04-02T13:43:07.343 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T13:43:07.343 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2024-04-02T13:43:07.343 INFO:teuthology.orchestra.run.smithi052.stdout:firewalld.service is not enabled 2024-04-02T13:43:07.343 INFO:teuthology.orchestra.run.smithi052.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-02T13:43:07.343 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mgr to start... 2024-04-02T13:43:07.343 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mgr... 2024-04-02T13:43:07.556 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:07 smithi052 systemd[1]: Started Ceph mgr.x for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4. 2024-04-02T13:43:07.832 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:07 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:07.794+0000 7f01d5b9a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T13:43:08.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:07 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3633283548' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T13:43:08.089 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:07 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:07.894+0000 7f01d5b9a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T13:43:08.433 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-04-02T13:43:08.434 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-04-02T13:43:08.434 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsid": "bfa00e6e-f0f6-11ee-b647-cb9ed24678a4", 2024-04-02T13:43:08.434 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T13:43:08.434 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T13:43:08.434 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T13:43:08.434 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T13:43:08.434 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:08.434 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T13:43:08.434 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T13:43:08.434 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 0 2024-04-02T13:43:08.434 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:43:08.434 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T13:43:08.434 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "a" 2024-04-02T13:43:08.435 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:43:08.435 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-02T13:43:08.435 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T13:43:08.435 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:43:08.435 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-02T13:43:08.435 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T13:43:08.435 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:08.435 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T13:43:08.435 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:43:08.435 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T13:43:08.435 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T13:43:08.435 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T13:43:08.435 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T13:43:08.436 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T13:43:08.436 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T13:43:08.436 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:08.436 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T13:43:08.436 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T13:43:08.436 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T13:43:08.436 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T13:43:08.436 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T13:43:08.436 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T13:43:08.437 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T13:43:08.437 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T13:43:08.437 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T13:43:08.437 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:08.437 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T13:43:08.437 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:43:08.437 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T13:43:08.438 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T13:43:08.438 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:08.438 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T13:43:08.438 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-02T13:43:08.438 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T13:43:08.438 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T13:43:08.438 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T13:43:08.438 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T13:43:08.438 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T13:43:08.438 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:43:08.438 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T13:43:08.438 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:08.439 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T13:43:08.439 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:43:08.439 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T13:42:59.639537+0000", 2024-04-02T13:43:08.439 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T13:43:08.439 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:08.439 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T13:43:08.439 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-04-02T13:43:08.439 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.748+0000 7efee4126700 1 Processor -- start 2024-04-02T13:43:08.439 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.748+0000 7efee4126700 1 -- start start 2024-04-02T13:43:08.439 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.748+0000 7efee4126700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efedc1080b0 0x7efedc1084d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:08.440 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.748+0000 7efee4126700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efedc108a10 con 0x7efedc1080b0 2024-04-02T13:43:08.440 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.748+0000 7efee1ec2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efedc1080b0 0x7efedc1084d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:08.440 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.748+0000 7efee1ec2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efedc1080b0 0x7efedc1084d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44366/0 (socket says 172.21.15.52:44366) 2024-04-02T13:43:08.440 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.748+0000 7efee1ec2700 1 -- 172.21.15.52:0/1160222716 learned_addr learned my addr 172.21.15.52:0/1160222716 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:08.440 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.749+0000 7efee1ec2700 1 -- 172.21.15.52:0/1160222716 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efedc108b90 con 0x7efedc1080b0 2024-04-02T13:43:08.440 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.749+0000 7efee1ec2700 1 --2- 172.21.15.52:0/1160222716 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efedc1080b0 0x7efedc1084d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efed80098a0 tx=0x7efed8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f65fcfce43f42 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:08.440 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.749+0000 7efee0ec0700 1 -- 172.21.15.52:0/1160222716 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efed8004160 con 0x7efedc1080b0 2024-04-02T13:43:08.440 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.749+0000 7efee0ec0700 1 -- 172.21.15.52:0/1160222716 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efed8004300 con 0x7efedc1080b0 2024-04-02T13:43:08.440 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.749+0000 7efee0ec0700 1 -- 172.21.15.52:0/1160222716 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efed8011b50 con 0x7efedc1080b0 2024-04-02T13:43:08.441 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.750+0000 7efee4126700 1 -- 172.21.15.52:0/1160222716 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efedc1080b0 msgr2=0x7efedc1084d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:08.441 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.750+0000 7efee4126700 1 --2- 172.21.15.52:0/1160222716 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efedc1080b0 0x7efedc1084d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efed80098a0 tx=0x7efed8009bb0 comp rx=0 tx=0).stop 2024-04-02T13:43:08.441 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.750+0000 7efee4126700 1 -- 172.21.15.52:0/1160222716 shutdown_connections 2024-04-02T13:43:08.441 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.750+0000 7efee4126700 1 --2- 172.21.15.52:0/1160222716 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efedc1080b0 0x7efedc1084d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:08.441 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.750+0000 7efee4126700 1 -- 172.21.15.52:0/1160222716 >> 172.21.15.52:0/1160222716 conn(0x7efedc103730 msgr2=0x7efedc105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:08.441 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.750+0000 7efee4126700 1 -- 172.21.15.52:0/1160222716 shutdown_connections 2024-04-02T13:43:08.441 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.750+0000 7efee4126700 1 -- 172.21.15.52:0/1160222716 wait complete. 2024-04-02T13:43:08.441 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.750+0000 7efee4126700 1 Processor -- start 2024-04-02T13:43:08.441 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.750+0000 7efee4126700 1 -- start start 2024-04-02T13:43:08.442 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.751+0000 7efee4126700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efedc1080b0 0x7efedc19e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:08.442 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.751+0000 7efee4126700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efedc10c520 con 0x7efedc1080b0 2024-04-02T13:43:08.442 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.751+0000 7efee1ec2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efedc1080b0 0x7efedc19e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:08.442 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.751+0000 7efee1ec2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efedc1080b0 0x7efedc19e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44368/0 (socket says 172.21.15.52:44368) 2024-04-02T13:43:08.442 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.751+0000 7efee1ec2700 1 -- 172.21.15.52:0/3633283548 learned_addr learned my addr 172.21.15.52:0/3633283548 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:08.442 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.751+0000 7efee1ec2700 1 -- 172.21.15.52:0/3633283548 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efedc19e750 con 0x7efedc1080b0 2024-04-02T13:43:08.443 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.751+0000 7efee1ec2700 1 --2- 172.21.15.52:0/3633283548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efedc1080b0 0x7efedc19e210 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efed8011b60 tx=0x7efed8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:08.443 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.751+0000 7efed2ffd700 1 -- 172.21.15.52:0/3633283548 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efed8003b20 con 0x7efedc1080b0 2024-04-02T13:43:08.443 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.751+0000 7efed2ffd700 1 -- 172.21.15.52:0/3633283548 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efed8003cc0 con 0x7efedc1080b0 2024-04-02T13:43:08.443 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.751+0000 7efee4126700 1 -- 172.21.15.52:0/3633283548 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efedc19e9e0 con 0x7efedc1080b0 2024-04-02T13:43:08.443 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.751+0000 7efed2ffd700 1 -- 172.21.15.52:0/3633283548 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efed8024840 con 0x7efedc1080b0 2024-04-02T13:43:08.443 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.751+0000 7efee4126700 1 -- 172.21.15.52:0/3633283548 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efedc1a16d0 con 0x7efedc1080b0 2024-04-02T13:43:08.443 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.752+0000 7efed2ffd700 1 -- 172.21.15.52:0/3633283548 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efed8022020 con 0x7efedc1080b0 2024-04-02T13:43:08.443 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.752+0000 7efed2ffd700 1 -- 172.21.15.52:0/3633283548 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efed8011e00 con 0x7efedc1080b0 2024-04-02T13:43:08.443 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.752+0000 7efee4126700 1 -- 172.21.15.52:0/3633283548 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efedc063c20 con 0x7efedc1080b0 2024-04-02T13:43:08.443 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.754+0000 7efed2ffd700 1 -- 172.21.15.52:0/3633283548 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7efed802d3f0 con 0x7efedc1080b0 2024-04-02T13:43:08.443 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.829+0000 7efee4126700 1 -- 172.21.15.52:0/3633283548 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7efedc1a1c10 con 0x7efedc1080b0 2024-04-02T13:43:08.443 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.829+0000 7efed2ffd700 1 -- 172.21.15.52:0/3633283548 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7efed8032030 con 0x7efedc1080b0 2024-04-02T13:43:08.443 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.830+0000 7efee4126700 1 -- 172.21.15.52:0/3633283548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efedc1080b0 msgr2=0x7efedc19e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:08.444 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.830+0000 7efee4126700 1 --2- 172.21.15.52:0/3633283548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efedc1080b0 0x7efedc19e210 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efed8011b60 tx=0x7efed8011c40 comp rx=0 tx=0).stop 2024-04-02T13:43:08.444 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.830+0000 7efee4126700 1 -- 172.21.15.52:0/3633283548 shutdown_connections 2024-04-02T13:43:08.444 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.830+0000 7efee4126700 1 --2- 172.21.15.52:0/3633283548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efedc1080b0 0x7efedc19e210 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:08.444 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.830+0000 7efee4126700 1 -- 172.21.15.52:0/3633283548 >> 172.21.15.52:0/3633283548 conn(0x7efedc103730 msgr2=0x7efedc104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:08.444 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.830+0000 7efee4126700 1 -- 172.21.15.52:0/3633283548 shutdown_connections 2024-04-02T13:43:08.444 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:07.830+0000 7efee4126700 1 -- 172.21.15.52:0/3633283548 wait complete. 2024-04-02T13:43:08.444 INFO:teuthology.orchestra.run.smithi052.stdout:mgr not available, waiting (1/15)... 2024-04-02T13:43:08.800 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:08 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:08.560+0000 7f01d5b9a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T13:43:08.800 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:08 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:08.654+0000 7f01d5b9a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T13:43:09.089 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:08 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:08.797+0000 7f01d5b9a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T13:43:09.089 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:08 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:08.894+0000 7f01d5b9a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T13:43:09.089 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:08 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:08.979+0000 7f01d5b9a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T13:43:09.389 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:09 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:09.273+0000 7f01d5b9a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T13:43:09.750 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:09 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:09.385+0000 7f01d5b9a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T13:43:09.750 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:09 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:09.473+0000 7f01d5b9a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T13:43:09.750 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:09 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:09.569+0000 7f01d5b9a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T13:43:09.751 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:09 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:09.747+0000 7f01d5b9a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T13:43:10.805 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:10 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:10.756+0000 7f01d5b9a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T13:43:11.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:10 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/393265915' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T13:43:11.089 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:10 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:10.893+0000 7f01d5b9a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T13:43:11.517 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:11 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:11.241+0000 7f01d5b9a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T13:43:11.517 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:11 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:11.332+0000 7f01d5b9a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T13:43:11.531 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-04-02T13:43:11.532 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-04-02T13:43:11.532 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsid": "bfa00e6e-f0f6-11ee-b647-cb9ed24678a4", 2024-04-02T13:43:11.532 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T13:43:11.532 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T13:43:11.532 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T13:43:11.532 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T13:43:11.532 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:11.532 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T13:43:11.532 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T13:43:11.532 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 0 2024-04-02T13:43:11.533 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:43:11.533 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T13:43:11.533 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "a" 2024-04-02T13:43:11.533 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:43:11.533 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-02T13:43:11.533 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T13:43:11.533 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:43:11.533 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-02T13:43:11.533 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T13:43:11.533 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:11.533 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T13:43:11.533 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:43:11.533 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T13:43:11.534 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T13:43:11.534 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T13:43:11.534 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T13:43:11.534 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T13:43:11.534 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T13:43:11.534 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:11.534 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T13:43:11.534 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T13:43:11.534 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T13:43:11.534 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T13:43:11.535 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T13:43:11.535 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T13:43:11.535 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T13:43:11.535 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T13:43:11.535 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T13:43:11.535 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:11.535 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T13:43:11.535 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:43:11.535 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T13:43:11.535 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T13:43:11.535 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:11.535 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T13:43:11.536 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-02T13:43:11.536 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T13:43:11.536 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T13:43:11.536 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T13:43:11.536 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T13:43:11.536 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T13:43:11.536 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:43:11.536 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T13:43:11.536 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:11.536 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T13:43:11.536 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:43:11.536 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T13:42:59.639537+0000", 2024-04-02T13:43:11.536 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T13:43:11.537 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:11.537 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T13:43:11.537 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-04-02T13:43:11.537 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.827+0000 7f25f6ede700 1 Processor -- start 2024-04-02T13:43:11.537 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.827+0000 7f25f6ede700 1 -- start start 2024-04-02T13:43:11.537 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.827+0000 7f25f6ede700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25f007e690 0x7f25f007cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:11.537 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.827+0000 7f25f6ede700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f25f007d080 con 0x7f25f007e690 2024-04-02T13:43:11.537 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.828+0000 7f25f4c7a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25f007e690 0x7f25f007cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:11.537 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.828+0000 7f25f4c7a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25f007e690 0x7f25f007cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44370/0 (socket says 172.21.15.52:44370) 2024-04-02T13:43:11.537 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.828+0000 7f25f4c7a700 1 -- 172.21.15.52:0/2977045731 learned_addr learned my addr 172.21.15.52:0/2977045731 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:11.537 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.828+0000 7f25f4c7a700 1 -- 172.21.15.52:0/2977045731 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f007d200 con 0x7f25f007e690 2024-04-02T13:43:11.538 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.828+0000 7f25f4c7a700 1 --2- 172.21.15.52:0/2977045731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25f007e690 0x7f25f007cb40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f25e40098a0 tx=0x7f25e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b6e1d0006ad28da server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:11.538 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.828+0000 7f25ef7fe700 1 -- 172.21.15.52:0/2977045731 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25e4004160 con 0x7f25f007e690 2024-04-02T13:43:11.538 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.828+0000 7f25ef7fe700 1 -- 172.21.15.52:0/2977045731 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f25e4004300 con 0x7f25f007e690 2024-04-02T13:43:11.538 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.828+0000 7f25ef7fe700 1 -- 172.21.15.52:0/2977045731 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25e4011b50 con 0x7f25f007e690 2024-04-02T13:43:11.538 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.829+0000 7f25f6ede700 1 -- 172.21.15.52:0/2977045731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25f007e690 msgr2=0x7f25f007cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:11.538 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.829+0000 7f25f6ede700 1 --2- 172.21.15.52:0/2977045731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25f007e690 0x7f25f007cb40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f25e40098a0 tx=0x7f25e4009bb0 comp rx=0 tx=0).stop 2024-04-02T13:43:11.538 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.829+0000 7f25f6ede700 1 -- 172.21.15.52:0/2977045731 shutdown_connections 2024-04-02T13:43:11.538 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.829+0000 7f25f6ede700 1 --2- 172.21.15.52:0/2977045731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25f007e690 0x7f25f007cb40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:11.538 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.829+0000 7f25f6ede700 1 -- 172.21.15.52:0/2977045731 >> 172.21.15.52:0/2977045731 conn(0x7f25f01036b0 msgr2=0x7f25f0105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:11.538 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.829+0000 7f25f6ede700 1 -- 172.21.15.52:0/2977045731 shutdown_connections 2024-04-02T13:43:11.538 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.829+0000 7f25f6ede700 1 -- 172.21.15.52:0/2977045731 wait complete. 2024-04-02T13:43:11.539 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.830+0000 7f25f6ede700 1 Processor -- start 2024-04-02T13:43:11.539 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.830+0000 7f25f6ede700 1 -- start start 2024-04-02T13:43:11.539 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.830+0000 7f25f6ede700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25f007e690 0x7f25f019e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:11.539 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.830+0000 7f25f6ede700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f25f010e6a0 con 0x7f25f007e690 2024-04-02T13:43:11.539 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.830+0000 7f25f4c7a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25f007e690 0x7f25f019e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:11.539 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.830+0000 7f25f4c7a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25f007e690 0x7f25f019e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44382/0 (socket says 172.21.15.52:44382) 2024-04-02T13:43:11.539 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.830+0000 7f25f4c7a700 1 -- 172.21.15.52:0/393265915 learned_addr learned my addr 172.21.15.52:0/393265915 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:11.539 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.830+0000 7f25f4c7a700 1 -- 172.21.15.52:0/393265915 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f019e770 con 0x7f25f007e690 2024-04-02T13:43:11.539 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.830+0000 7f25f4c7a700 1 --2- 172.21.15.52:0/393265915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25f007e690 0x7f25f019e230 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f25e4011b60 tx=0x7f25e4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:11.539 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.831+0000 7f25edffb700 1 -- 172.21.15.52:0/393265915 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25e4003b20 con 0x7f25f007e690 2024-04-02T13:43:11.539 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.831+0000 7f25edffb700 1 -- 172.21.15.52:0/393265915 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f25e4003cc0 con 0x7f25f007e690 2024-04-02T13:43:11.540 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.831+0000 7f25f6ede700 1 -- 172.21.15.52:0/393265915 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f019ea00 con 0x7f25f007e690 2024-04-02T13:43:11.540 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.831+0000 7f25edffb700 1 -- 172.21.15.52:0/393265915 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25e4024840 con 0x7f25f007e690 2024-04-02T13:43:11.540 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.831+0000 7f25f6ede700 1 -- 172.21.15.52:0/393265915 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f01a16f0 con 0x7f25f007e690 2024-04-02T13:43:11.540 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.831+0000 7f25edffb700 1 -- 172.21.15.52:0/393265915 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f25e4022020 con 0x7f25f007e690 2024-04-02T13:43:11.540 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.831+0000 7f25edffb700 1 -- 172.21.15.52:0/393265915 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f25e4011e00 con 0x7f25f007e690 2024-04-02T13:43:11.540 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.832+0000 7f25f6ede700 1 -- 172.21.15.52:0/393265915 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25f019eee0 con 0x7f25f007e690 2024-04-02T13:43:11.540 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.834+0000 7f25edffb700 1 -- 172.21.15.52:0/393265915 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f25e4043dd0 con 0x7f25f007e690 2024-04-02T13:43:11.540 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.908+0000 7f25f6ede700 1 -- 172.21.15.52:0/393265915 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f25f01a18c0 con 0x7f25f007e690 2024-04-02T13:43:11.540 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.908+0000 7f25edffb700 1 -- 172.21.15.52:0/393265915 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f25e4032030 con 0x7f25f007e690 2024-04-02T13:43:11.540 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.909+0000 7f25f6ede700 1 -- 172.21.15.52:0/393265915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25f007e690 msgr2=0x7f25f019e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:11.540 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.909+0000 7f25f6ede700 1 --2- 172.21.15.52:0/393265915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25f007e690 0x7f25f019e230 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f25e4011b60 tx=0x7f25e4011c40 comp rx=0 tx=0).stop 2024-04-02T13:43:11.540 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.909+0000 7f25f6ede700 1 -- 172.21.15.52:0/393265915 shutdown_connections 2024-04-02T13:43:11.541 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.909+0000 7f25f6ede700 1 --2- 172.21.15.52:0/393265915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25f007e690 0x7f25f019e230 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:11.541 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.909+0000 7f25f6ede700 1 -- 172.21.15.52:0/393265915 >> 172.21.15.52:0/393265915 conn(0x7f25f01036b0 msgr2=0x7f25f01041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:11.541 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.909+0000 7f25f6ede700 1 -- 172.21.15.52:0/393265915 shutdown_connections 2024-04-02T13:43:11.541 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:10.909+0000 7f25f6ede700 1 -- 172.21.15.52:0/393265915 wait complete. 2024-04-02T13:43:11.541 INFO:teuthology.orchestra.run.smithi052.stdout:mgr not available, waiting (2/15)... 2024-04-02T13:43:11.839 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:11 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:11.534+0000 7f01d5b9a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:43:11.839 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:11 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:11.724+0000 7f01d5b9a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T13:43:12.279 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:12 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:12.015+0000 7f01d5b9a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:43:12.279 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:12 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:12.276+0000 7f01d5b9a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T13:43:12.589 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:12 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:12.519+0000 7f01d5b9a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T13:43:13.648 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:13 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:13.351+0000 7f01d5b9a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T13:43:14.270 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:14 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3406107446' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T13:43:14.270 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:13 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:13.937+0000 7f01d5b9a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T13:43:14.270 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:14 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:14.040+0000 7f01d5b9a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T13:43:14.270 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:14 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:14.267+0000 7f01d5b9a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T13:43:14.846 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-04-02T13:43:14.846 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-04-02T13:43:14.846 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsid": "bfa00e6e-f0f6-11ee-b647-cb9ed24678a4", 2024-04-02T13:43:14.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T13:43:14.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T13:43:14.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T13:43:14.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T13:43:14.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:14.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T13:43:14.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T13:43:14.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 0 2024-04-02T13:43:14.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:43:14.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T13:43:14.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "a" 2024-04-02T13:43:14.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:43:14.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-02T13:43:14.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T13:43:14.847 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:43:14.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-02T13:43:14.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T13:43:14.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:14.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T13:43:14.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:43:14.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T13:43:14.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T13:43:14.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T13:43:14.857 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T13:43:14.857 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T13:43:14.857 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T13:43:14.857 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:14.857 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T13:43:14.857 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T13:43:14.857 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T13:43:14.857 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T13:43:14.857 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T13:43:14.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T13:43:14.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T13:43:14.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T13:43:14.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T13:43:14.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:14.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T13:43:14.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:43:14.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T13:43:14.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T13:43:14.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:14.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T13:43:14.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-02T13:43:14.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T13:43:14.859 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T13:43:14.859 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T13:43:14.859 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T13:43:14.859 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T13:43:14.859 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:43:14.859 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T13:43:14.859 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:14.859 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T13:43:14.859 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:43:14.859 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T13:42:59.639537+0000", 2024-04-02T13:43:14.859 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T13:43:14.859 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:14.859 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T13:43:14.859 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-04-02T13:43:14.860 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.974+0000 7f915b9a0700 1 Processor -- start 2024-04-02T13:43:14.860 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.975+0000 7f915b9a0700 1 -- start start 2024-04-02T13:43:14.860 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.975+0000 7f915b9a0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9154106720 0x7f9154106b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.860 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.975+0000 7f915b9a0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9154107110 con 0x7f9154106720 2024-04-02T13:43:14.860 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.975+0000 7f915973c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9154106720 0x7f9154106b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.860 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.975+0000 7f915973c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9154106720 0x7f9154106b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53562/0 (socket says 172.21.15.52:53562) 2024-04-02T13:43:14.860 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.975+0000 7f915973c700 1 -- 172.21.15.52:0/2332822617 learned_addr learned my addr 172.21.15.52:0/2332822617 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:14.860 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.975+0000 7f915973c700 1 -- 172.21.15.52:0/2332822617 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9154107920 con 0x7f9154106720 2024-04-02T13:43:14.861 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.975+0000 7f915973c700 1 --2- 172.21.15.52:0/2332822617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9154106720 0x7f9154106b40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f915000bb40 tx=0x7f915000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9f4bfdddf26644 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:14.861 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.976+0000 7f9147fff700 1 -- 172.21.15.52:0/2332822617 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f915000dad0 con 0x7f9154106720 2024-04-02T13:43:14.861 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.976+0000 7f9147fff700 1 -- 172.21.15.52:0/2332822617 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f91500044f0 con 0x7f9154106720 2024-04-02T13:43:14.861 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.976+0000 7f9147fff700 1 -- 172.21.15.52:0/2332822617 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9150017410 con 0x7f9154106720 2024-04-02T13:43:14.861 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.976+0000 7f915b9a0700 1 -- 172.21.15.52:0/2332822617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9154106720 msgr2=0x7f9154106b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:14.861 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.976+0000 7f915b9a0700 1 --2- 172.21.15.52:0/2332822617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9154106720 0x7f9154106b40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f915000bb40 tx=0x7f915000be50 comp rx=0 tx=0).stop 2024-04-02T13:43:14.861 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.976+0000 7f915b9a0700 1 -- 172.21.15.52:0/2332822617 shutdown_connections 2024-04-02T13:43:14.861 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.976+0000 7f915b9a0700 1 --2- 172.21.15.52:0/2332822617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9154106720 0x7f9154106b40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.862 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.976+0000 7f915b9a0700 1 -- 172.21.15.52:0/2332822617 >> 172.21.15.52:0/2332822617 conn(0x7f9154101ce0 msgr2=0x7f9154104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:14.862 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.976+0000 7f915b9a0700 1 -- 172.21.15.52:0/2332822617 shutdown_connections 2024-04-02T13:43:14.862 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.977+0000 7f915b9a0700 1 -- 172.21.15.52:0/2332822617 wait complete. 2024-04-02T13:43:14.862 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.977+0000 7f915b9a0700 1 Processor -- start 2024-04-02T13:43:14.862 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.977+0000 7f915b9a0700 1 -- start start 2024-04-02T13:43:14.862 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.977+0000 7f915b9a0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9154106720 0x7f915419e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:14.862 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.977+0000 7f915b9a0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f91541080a0 con 0x7f9154106720 2024-04-02T13:43:14.863 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.977+0000 7f915973c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9154106720 0x7f915419e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:14.863 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.977+0000 7f915973c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9154106720 0x7f915419e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53564/0 (socket says 172.21.15.52:53564) 2024-04-02T13:43:14.863 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.977+0000 7f915973c700 1 -- 172.21.15.52:0/3406107446 learned_addr learned my addr 172.21.15.52:0/3406107446 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:14.863 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.978+0000 7f915973c700 1 -- 172.21.15.52:0/3406107446 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f915419e830 con 0x7f9154106720 2024-04-02T13:43:14.863 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.978+0000 7f915973c700 1 --2- 172.21.15.52:0/3406107446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9154106720 0x7f915419e2f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f915000bb10 tx=0x7f91500047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:14.863 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.978+0000 7f91467fc700 1 -- 172.21.15.52:0/3406107446 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9150004110 con 0x7f9154106720 2024-04-02T13:43:14.863 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.978+0000 7f91467fc700 1 -- 172.21.15.52:0/3406107446 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9150024070 con 0x7f9154106720 2024-04-02T13:43:14.863 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.978+0000 7f915b9a0700 1 -- 172.21.15.52:0/3406107446 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f915419eac0 con 0x7f9154106720 2024-04-02T13:43:14.863 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.978+0000 7f91467fc700 1 -- 172.21.15.52:0/3406107446 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f915001fa40 con 0x7f9154106720 2024-04-02T13:43:14.863 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.978+0000 7f915b9a0700 1 -- 172.21.15.52:0/3406107446 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91541a1760 con 0x7f9154106720 2024-04-02T13:43:14.864 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.978+0000 7f91467fc700 1 -- 172.21.15.52:0/3406107446 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f915001fc30 con 0x7f9154106720 2024-04-02T13:43:14.864 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.978+0000 7f91467fc700 1 -- 172.21.15.52:0/3406107446 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f91500283c0 con 0x7f9154106720 2024-04-02T13:43:14.864 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.979+0000 7f915b9a0700 1 -- 172.21.15.52:0/3406107446 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9154063c20 con 0x7f9154106720 2024-04-02T13:43:14.864 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:13.981+0000 7f91467fc700 1 -- 172.21.15.52:0/3406107446 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9150012070 con 0x7f9154106720 2024-04-02T13:43:14.864 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:14.055+0000 7f915b9a0700 1 -- 172.21.15.52:0/3406107446 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f915402df30 con 0x7f9154106720 2024-04-02T13:43:14.864 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:14.056+0000 7f91467fc700 1 -- 172.21.15.52:0/3406107446 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f91500042b0 con 0x7f9154106720 2024-04-02T13:43:14.864 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:14.057+0000 7f915b9a0700 1 -- 172.21.15.52:0/3406107446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9154106720 msgr2=0x7f915419e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:14.864 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:14.057+0000 7f915b9a0700 1 --2- 172.21.15.52:0/3406107446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9154106720 0x7f915419e2f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f915000bb10 tx=0x7f91500047c0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.864 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:14.057+0000 7f915b9a0700 1 -- 172.21.15.52:0/3406107446 shutdown_connections 2024-04-02T13:43:14.864 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:14.057+0000 7f915b9a0700 1 --2- 172.21.15.52:0/3406107446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9154106720 0x7f915419e2f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:14.864 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:14.057+0000 7f915b9a0700 1 -- 172.21.15.52:0/3406107446 >> 172.21.15.52:0/3406107446 conn(0x7f9154101ce0 msgr2=0x7f9154190b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:14.864 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:14.057+0000 7f915b9a0700 1 -- 172.21.15.52:0/3406107446 shutdown_connections 2024-04-02T13:43:14.864 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:14.057+0000 7f915b9a0700 1 -- 172.21.15.52:0/3406107446 wait complete. 2024-04-02T13:43:14.864 INFO:teuthology.orchestra.run.smithi052.stdout:mgr not available, waiting (3/15)... 2024-04-02T13:43:15.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:15 smithi052 ceph-mon[32104]: Activating manager daemon x 2024-04-02T13:43:15.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:15 smithi052 ceph-mon[32104]: mgrmap e2: x(active, starting, since 0.0032787s) 2024-04-02T13:43:15.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:15 smithi052 ceph-mon[32104]: from='mgr.14100 172.21.15.52:0/3428675444' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T13:43:15.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:15 smithi052 ceph-mon[32104]: from='mgr.14100 172.21.15.52:0/3428675444' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T13:43:15.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:15 smithi052 ceph-mon[32104]: from='mgr.14100 172.21.15.52:0/3428675444' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T13:43:15.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:15 smithi052 ceph-mon[32104]: from='mgr.14100 172.21.15.52:0/3428675444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:43:15.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:15 smithi052 ceph-mon[32104]: from='mgr.14100 172.21.15.52:0/3428675444' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T13:43:15.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:15 smithi052 ceph-mon[32104]: Manager daemon x is now available 2024-04-02T13:43:15.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:15 smithi052 ceph-mon[32104]: from='mgr.14100 172.21.15.52:0/3428675444' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T13:43:15.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:15 smithi052 ceph-mon[32104]: from='mgr.14100 172.21.15.52:0/3428675444' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T13:43:15.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:15 smithi052 ceph-mon[32104]: from='mgr.14100 172.21.15.52:0/3428675444' entity='mgr.x' 2024-04-02T13:43:15.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:15 smithi052 ceph-mon[32104]: from='mgr.14100 172.21.15.52:0/3428675444' entity='mgr.x' 2024-04-02T13:43:15.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:15 smithi052 ceph-mon[32104]: from='mgr.14100 172.21.15.52:0/3428675444' entity='mgr.x' 2024-04-02T13:43:16.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:16 smithi052 ceph-mon[32104]: mgrmap e3: x(active, since 1.00719s) 2024-04-02T13:43:17.578 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:17 smithi052 ceph-mon[32104]: mgrmap e4: x(active, since 2s) 2024-04-02T13:43:18.136 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-04-02T13:43:18.136 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-04-02T13:43:18.136 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsid": "bfa00e6e-f0f6-11ee-b647-cb9ed24678a4", 2024-04-02T13:43:18.136 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T13:43:18.137 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T13:43:18.137 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T13:43:18.137 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T13:43:18.137 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:18.137 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T13:43:18.137 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T13:43:18.137 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 0 2024-04-02T13:43:18.137 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:43:18.137 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T13:43:18.137 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "a" 2024-04-02T13:43:18.137 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:43:18.137 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-02T13:43:18.137 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T13:43:18.137 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:43:18.137 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-02T13:43:18.138 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T13:43:18.138 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:18.138 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T13:43:18.138 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:43:18.138 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T13:43:18.138 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T13:43:18.138 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T13:43:18.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T13:43:18.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T13:43:18.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T13:43:18.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:18.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T13:43:18.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T13:43:18.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T13:43:18.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T13:43:18.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T13:43:18.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T13:43:18.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T13:43:18.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T13:43:18.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T13:43:18.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:18.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T13:43:18.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:43:18.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T13:43:18.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T13:43:18.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:18.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T13:43:18.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-02T13:43:18.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T13:43:18.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T13:43:18.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T13:43:18.158 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T13:43:18.158 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T13:43:18.158 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:43:18.158 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T13:43:18.158 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:18.158 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T13:43:18.159 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:43:18.159 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T13:42:59.639537+0000", 2024-04-02T13:43:18.159 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T13:43:18.159 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:43:18.159 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T13:43:18.159 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-04-02T13:43:18.160 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.271+0000 7f37702ab700 1 Processor -- start 2024-04-02T13:43:18.160 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.271+0000 7f37702ab700 1 -- start start 2024-04-02T13:43:18.160 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.271+0000 7f37702ab700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3768108af0 0x7f376810aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:18.160 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.271+0000 7f37702ab700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f376807be20 con 0x7f3768108af0 2024-04-02T13:43:18.160 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.272+0000 7f376e047700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3768108af0 0x7f376810aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:18.160 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.272+0000 7f376e047700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3768108af0 0x7f376810aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53634/0 (socket says 172.21.15.52:53634) 2024-04-02T13:43:18.160 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.272+0000 7f376e047700 1 -- 172.21.15.52:0/3864250248 learned_addr learned my addr 172.21.15.52:0/3864250248 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:18.160 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.272+0000 7f376e047700 1 -- 172.21.15.52:0/3864250248 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376810b420 con 0x7f3768108af0 2024-04-02T13:43:18.160 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.272+0000 7f376e047700 1 --2- 172.21.15.52:0/3864250248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3768108af0 0x7f376810aee0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f37580098a0 tx=0x7f3758009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b208a47fe45d9c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:18.161 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.272+0000 7f376d045700 1 -- 172.21.15.52:0/3864250248 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3758004160 con 0x7f3768108af0 2024-04-02T13:43:18.161 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.272+0000 7f376d045700 1 -- 172.21.15.52:0/3864250248 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3758004300 con 0x7f3768108af0 2024-04-02T13:43:18.161 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.272+0000 7f376d045700 1 -- 172.21.15.52:0/3864250248 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3758011b50 con 0x7f3768108af0 2024-04-02T13:43:18.161 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.273+0000 7f37702ab700 1 -- 172.21.15.52:0/3864250248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3768108af0 msgr2=0x7f376810aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:18.161 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.273+0000 7f37702ab700 1 --2- 172.21.15.52:0/3864250248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3768108af0 0x7f376810aee0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f37580098a0 tx=0x7f3758009bb0 comp rx=0 tx=0).stop 2024-04-02T13:43:18.161 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.273+0000 7f37702ab700 1 -- 172.21.15.52:0/3864250248 shutdown_connections 2024-04-02T13:43:18.161 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.273+0000 7f37702ab700 1 --2- 172.21.15.52:0/3864250248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3768108af0 0x7f376810aee0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:18.161 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.273+0000 7f37702ab700 1 -- 172.21.15.52:0/3864250248 >> 172.21.15.52:0/3864250248 conn(0x7f3768102500 msgr2=0x7f3768104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:18.162 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.273+0000 7f37702ab700 1 -- 172.21.15.52:0/3864250248 shutdown_connections 2024-04-02T13:43:18.162 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.273+0000 7f37702ab700 1 -- 172.21.15.52:0/3864250248 wait complete. 2024-04-02T13:43:18.162 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.274+0000 7f37702ab700 1 Processor -- start 2024-04-02T13:43:18.162 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.274+0000 7f37702ab700 1 -- start start 2024-04-02T13:43:18.162 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.274+0000 7f37702ab700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3768108af0 0x7f376819c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:18.162 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.274+0000 7f37702ab700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f376810c3f0 con 0x7f3768108af0 2024-04-02T13:43:18.162 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.274+0000 7f376e047700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3768108af0 0x7f376819c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:18.162 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.274+0000 7f376e047700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3768108af0 0x7f376819c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53638/0 (socket says 172.21.15.52:53638) 2024-04-02T13:43:18.163 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.274+0000 7f376e047700 1 -- 172.21.15.52:0/844046539 learned_addr learned my addr 172.21.15.52:0/844046539 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:18.163 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.274+0000 7f376e047700 1 -- 172.21.15.52:0/844046539 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376819c560 con 0x7f3768108af0 2024-04-02T13:43:18.163 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.275+0000 7f376e047700 1 --2- 172.21.15.52:0/844046539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3768108af0 0x7f376819c020 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3758011b60 tx=0x7f3758011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:18.163 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.275+0000 7f375f7fe700 1 -- 172.21.15.52:0/844046539 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3758003b20 con 0x7f3768108af0 2024-04-02T13:43:18.163 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.275+0000 7f375f7fe700 1 -- 172.21.15.52:0/844046539 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3758003cc0 con 0x7f3768108af0 2024-04-02T13:43:18.163 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.275+0000 7f37702ab700 1 -- 172.21.15.52:0/844046539 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376819c7f0 con 0x7f3768108af0 2024-04-02T13:43:18.163 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.275+0000 7f375f7fe700 1 -- 172.21.15.52:0/844046539 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3758024840 con 0x7f3768108af0 2024-04-02T13:43:18.163 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.275+0000 7f37702ab700 1 -- 172.21.15.52:0/844046539 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f376819f4e0 con 0x7f3768108af0 2024-04-02T13:43:18.164 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.276+0000 7f375f7fe700 1 -- 172.21.15.52:0/844046539 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f3758022020 con 0x7f3768108af0 2024-04-02T13:43:18.164 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.276+0000 7f375f7fe700 1 --2- 172.21.15.52:0/844046539 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f3754039560 0x7f375403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:18.164 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.276+0000 7f375f7fe700 1 -- 172.21.15.52:0/844046539 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3758011e00 con 0x7f3768108af0 2024-04-02T13:43:18.164 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.276+0000 7f37702ab700 1 -- 172.21.15.52:0/844046539 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3768063c20 con 0x7f3768108af0 2024-04-02T13:43:18.164 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.276+0000 7f376d846700 1 --2- 172.21.15.52:0/844046539 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f3754039560 0x7f375403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:18.164 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.276+0000 7f376d846700 1 --2- 172.21.15.52:0/844046539 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f3754039560 0x7f375403ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3764006fd0 tx=0x7f3764006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:18.165 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.283+0000 7f375f7fe700 1 -- 172.21.15.52:0/844046539 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f37580172b0 con 0x7f3768108af0 2024-04-02T13:43:18.165 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.575+0000 7f37702ab700 1 -- 172.21.15.52:0/844046539 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f376819fa50 con 0x7f3768108af0 2024-04-02T13:43:18.165 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.576+0000 7f375f7fe700 1 -- 172.21.15.52:0/844046539 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f375800b7e0 con 0x7f3768108af0 2024-04-02T13:43:18.165 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.578+0000 7f37702ab700 1 -- 172.21.15.52:0/844046539 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f3754039560 msgr2=0x7f375403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:18.165 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.578+0000 7f37702ab700 1 --2- 172.21.15.52:0/844046539 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f3754039560 0x7f375403ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3764006fd0 tx=0x7f3764006e40 comp rx=0 tx=0).stop 2024-04-02T13:43:18.165 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.578+0000 7f37702ab700 1 -- 172.21.15.52:0/844046539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3768108af0 msgr2=0x7f376819c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:18.165 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.578+0000 7f37702ab700 1 --2- 172.21.15.52:0/844046539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3768108af0 0x7f376819c020 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3758011b60 tx=0x7f3758011c40 comp rx=0 tx=0).stop 2024-04-02T13:43:18.166 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.578+0000 7f37702ab700 1 -- 172.21.15.52:0/844046539 shutdown_connections 2024-04-02T13:43:18.166 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.578+0000 7f37702ab700 1 --2- 172.21.15.52:0/844046539 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f3754039560 0x7f375403ba20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:18.166 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.578+0000 7f37702ab700 1 --2- 172.21.15.52:0/844046539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3768108af0 0x7f376819c020 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:18.166 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.578+0000 7f37702ab700 1 -- 172.21.15.52:0/844046539 >> 172.21.15.52:0/844046539 conn(0x7f3768102500 msgr2=0x7f37681030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:18.166 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.578+0000 7f37702ab700 1 -- 172.21.15.52:0/844046539 shutdown_connections 2024-04-02T13:43:18.166 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:17.578+0000 7f37702ab700 1 -- 172.21.15.52:0/844046539 wait complete. 2024-04-02T13:43:18.167 INFO:teuthology.orchestra.run.smithi052.stdout:mgr is available 2024-04-02T13:43:18.533 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:18 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/844046539' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T13:43:19.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:19 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1515693143' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-02T13:43:19.388 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-04-02T13:43:19.388 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [global] 2024-04-02T13:43:19.388 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout fsid = bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:43:19.388 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-02T13:43:19.388 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-02T13:43:19.388 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-02T13:43:19.388 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-02T13:43:19.388 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-04-02T13:43:19.388 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-02T13:43:19.389 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-02T13:43:19.389 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-04-02T13:43:19.389 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [osd] 2024-04-02T13:43:19.389 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-02T13:43:19.389 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-02T13:43:19.389 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-02T13:43:19.389 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-02T13:43:19.389 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-02T13:43:19.389 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.563+0000 7f275a34e700 1 Processor -- start 2024-04-02T13:43:19.389 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.563+0000 7f275a34e700 1 -- start start 2024-04-02T13:43:19.389 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.563+0000 7f275a34e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27541080d0 0x7f27541084f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:19.389 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.563+0000 7f275a34e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2754108ac0 con 0x7f27541080d0 2024-04-02T13:43:19.389 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.563+0000 7f2753fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27541080d0 0x7f27541084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:19.389 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.564+0000 7f2753fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27541080d0 0x7f27541084f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53640/0 (socket says 172.21.15.52:53640) 2024-04-02T13:43:19.390 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.564+0000 7f2753fff700 1 -- 172.21.15.52:0/4200302266 learned_addr learned my addr 172.21.15.52:0/4200302266 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:19.390 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.564+0000 7f2753fff700 1 -- 172.21.15.52:0/4200302266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2754109280 con 0x7f27541080d0 2024-04-02T13:43:19.390 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.564+0000 7f2753fff700 1 --2- 172.21.15.52:0/4200302266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27541080d0 0x7f27541084f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f27440098a0 tx=0x7f2744009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7244170fdfa86b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:19.390 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.564+0000 7f2752ffd700 1 -- 172.21.15.52:0/4200302266 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2744004160 con 0x7f27541080d0 2024-04-02T13:43:19.390 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.564+0000 7f2752ffd700 1 -- 172.21.15.52:0/4200302266 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2744004300 con 0x7f27541080d0 2024-04-02T13:43:19.390 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.564+0000 7f2752ffd700 1 -- 172.21.15.52:0/4200302266 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2744011b50 con 0x7f27541080d0 2024-04-02T13:43:19.390 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.565+0000 7f275a34e700 1 -- 172.21.15.52:0/4200302266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27541080d0 msgr2=0x7f27541084f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:19.390 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.565+0000 7f275a34e700 1 --2- 172.21.15.52:0/4200302266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27541080d0 0x7f27541084f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f27440098a0 tx=0x7f2744009bb0 comp rx=0 tx=0).stop 2024-04-02T13:43:19.390 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.565+0000 7f275a34e700 1 -- 172.21.15.52:0/4200302266 shutdown_connections 2024-04-02T13:43:19.390 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.565+0000 7f275a34e700 1 --2- 172.21.15.52:0/4200302266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27541080d0 0x7f27541084f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:19.390 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.565+0000 7f275a34e700 1 -- 172.21.15.52:0/4200302266 >> 172.21.15.52:0/4200302266 conn(0x7f27541036b0 msgr2=0x7f2754105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:19.390 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.565+0000 7f275a34e700 1 -- 172.21.15.52:0/4200302266 shutdown_connections 2024-04-02T13:43:19.390 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.565+0000 7f275a34e700 1 -- 172.21.15.52:0/4200302266 wait complete. 2024-04-02T13:43:19.391 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.565+0000 7f275a34e700 1 Processor -- start 2024-04-02T13:43:19.391 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.565+0000 7f275a34e700 1 -- start start 2024-04-02T13:43:19.391 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.566+0000 7f275a34e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27541080d0 0x7f275407e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:19.391 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.566+0000 7f275a34e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f275410c630 con 0x7f27541080d0 2024-04-02T13:43:19.391 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.566+0000 7f2753fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27541080d0 0x7f275407e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:19.391 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.566+0000 7f2753fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27541080d0 0x7f275407e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53646/0 (socket says 172.21.15.52:53646) 2024-04-02T13:43:19.391 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.566+0000 7f2753fff700 1 -- 172.21.15.52:0/1515693143 learned_addr learned my addr 172.21.15.52:0/1515693143 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:19.391 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.566+0000 7f2753fff700 1 -- 172.21.15.52:0/1515693143 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f275407e910 con 0x7f27541080d0 2024-04-02T13:43:19.391 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.566+0000 7f2753fff700 1 --2- 172.21.15.52:0/1515693143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27541080d0 0x7f275407e3d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2744000c00 tx=0x7f2744011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:19.391 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.566+0000 7f27517fa700 1 -- 172.21.15.52:0/1515693143 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2744003b20 con 0x7f27541080d0 2024-04-02T13:43:19.391 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.566+0000 7f27517fa700 1 -- 172.21.15.52:0/1515693143 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2744003cc0 con 0x7f27541080d0 2024-04-02T13:43:19.391 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.566+0000 7f275a34e700 1 -- 172.21.15.52:0/1515693143 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f275407cb00 con 0x7f27541080d0 2024-04-02T13:43:19.391 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.566+0000 7f27517fa700 1 -- 172.21.15.52:0/1515693143 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2744024910 con 0x7f27541080d0 2024-04-02T13:43:19.391 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.566+0000 7f275a34e700 1 -- 172.21.15.52:0/1515693143 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f275407d040 con 0x7f27541080d0 2024-04-02T13:43:19.391 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.567+0000 7f27517fa700 1 -- 172.21.15.52:0/1515693143 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f2744022020 con 0x7f27541080d0 2024-04-02T13:43:19.392 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.567+0000 7f27517fa700 1 --2- 172.21.15.52:0/1515693143 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f273c0395b0 0x7f273c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:19.392 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.567+0000 7f275a34e700 1 -- 172.21.15.52:0/1515693143 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2754063c20 con 0x7f27541080d0 2024-04-02T13:43:19.392 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.567+0000 7f27517fa700 1 -- 172.21.15.52:0/1515693143 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2744011e00 con 0x7f27541080d0 2024-04-02T13:43:19.392 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.568+0000 7f27537fe700 1 --2- 172.21.15.52:0/1515693143 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f273c0395b0 0x7f273c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:19.392 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.568+0000 7f27537fe700 1 --2- 172.21.15.52:0/1515693143 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f273c0395b0 0x7f273c03ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2748006fd0 tx=0x7f2748006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:19.392 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.573+0000 7f27517fa700 1 -- 172.21.15.52:0/1515693143 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f274402cda0 con 0x7f27541080d0 2024-04-02T13:43:19.392 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.777+0000 7f275a34e700 1 -- 172.21.15.52:0/1515693143 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f275407d940 con 0x7f27541080d0 2024-04-02T13:43:19.392 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.778+0000 7f27517fa700 1 -- 172.21.15.52:0/1515693143 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f2744032020 con 0x7f27541080d0 2024-04-02T13:43:19.392 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.780+0000 7f275a34e700 1 -- 172.21.15.52:0/1515693143 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f273c0395b0 msgr2=0x7f273c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:19.393 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.780+0000 7f275a34e700 1 --2- 172.21.15.52:0/1515693143 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f273c0395b0 0x7f273c03ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2748006fd0 tx=0x7f2748006e40 comp rx=0 tx=0).stop 2024-04-02T13:43:19.393 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.780+0000 7f275a34e700 1 -- 172.21.15.52:0/1515693143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27541080d0 msgr2=0x7f275407e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:19.393 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.780+0000 7f275a34e700 1 --2- 172.21.15.52:0/1515693143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27541080d0 0x7f275407e3d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2744000c00 tx=0x7f2744011c10 comp rx=0 tx=0).stop 2024-04-02T13:43:19.393 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.780+0000 7f275a34e700 1 -- 172.21.15.52:0/1515693143 shutdown_connections 2024-04-02T13:43:19.393 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.780+0000 7f275a34e700 1 --2- 172.21.15.52:0/1515693143 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f273c0395b0 0x7f273c03ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:19.393 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.780+0000 7f275a34e700 1 --2- 172.21.15.52:0/1515693143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27541080d0 0x7f275407e3d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:19.393 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.780+0000 7f275a34e700 1 -- 172.21.15.52:0/1515693143 >> 172.21.15.52:0/1515693143 conn(0x7f27541036b0 msgr2=0x7f2754104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:19.393 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.780+0000 7f275a34e700 1 -- 172.21.15.52:0/1515693143 shutdown_connections 2024-04-02T13:43:19.393 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:18.780+0000 7f275a34e700 1 -- 172.21.15.52:0/1515693143 wait complete. 2024-04-02T13:43:19.393 INFO:teuthology.orchestra.run.smithi052.stdout:Enabling cephadm module... 2024-04-02T13:43:20.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:20 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3711842724' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-02T13:43:20.589 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:20 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: ignoring --setuser ceph since I am not root 2024-04-02T13:43:20.589 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:20 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: ignoring --setgroup ceph since I am not root 2024-04-02T13:43:20.951 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.821+0000 7ffa48bcd700 1 Processor -- start 2024-04-02T13:43:20.951 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.821+0000 7ffa48bcd700 1 -- start start 2024-04-02T13:43:20.951 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.821+0000 7ffa48bcd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa44108090 0x7ffa441084b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:20.952 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.821+0000 7ffa48bcd700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffa44108a80 con 0x7ffa44108090 2024-04-02T13:43:20.952 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.822+0000 7ffa4259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa44108090 0x7ffa441084b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:20.952 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.822+0000 7ffa4259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa44108090 0x7ffa441084b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53656/0 (socket says 172.21.15.52:53656) 2024-04-02T13:43:20.952 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.822+0000 7ffa4259c700 1 -- 172.21.15.52:0/1422306561 learned_addr learned my addr 172.21.15.52:0/1422306561 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:20.952 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.822+0000 7ffa4259c700 1 -- 172.21.15.52:0/1422306561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa44109240 con 0x7ffa44108090 2024-04-02T13:43:20.952 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.822+0000 7ffa4259c700 1 --2- 172.21.15.52:0/1422306561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa44108090 0x7ffa441084b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009e80 tx=0x7ffa3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26522dfaa1492484 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:20.952 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.822+0000 7ffa4159a700 1 -- 172.21.15.52:0/1422306561 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa3c017070 con 0x7ffa44108090 2024-04-02T13:43:20.952 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.822+0000 7ffa4159a700 1 -- 172.21.15.52:0/1422306561 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffa3c00b8c0 con 0x7ffa44108090 2024-04-02T13:43:20.952 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.822+0000 7ffa4159a700 1 -- 172.21.15.52:0/1422306561 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa3c0045c0 con 0x7ffa44108090 2024-04-02T13:43:20.952 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.823+0000 7ffa48bcd700 1 -- 172.21.15.52:0/1422306561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa44108090 msgr2=0x7ffa441084b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:20.952 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.823+0000 7ffa48bcd700 1 --2- 172.21.15.52:0/1422306561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa44108090 0x7ffa441084b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009e80 tx=0x7ffa3c00b330 comp rx=0 tx=0).stop 2024-04-02T13:43:20.952 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.823+0000 7ffa48bcd700 1 -- 172.21.15.52:0/1422306561 shutdown_connections 2024-04-02T13:43:20.952 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.823+0000 7ffa48bcd700 1 --2- 172.21.15.52:0/1422306561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa44108090 0x7ffa441084b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:20.953 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.823+0000 7ffa48bcd700 1 -- 172.21.15.52:0/1422306561 >> 172.21.15.52:0/1422306561 conn(0x7ffa44103630 msgr2=0x7ffa44105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:20.953 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.823+0000 7ffa48bcd700 1 -- 172.21.15.52:0/1422306561 shutdown_connections 2024-04-02T13:43:20.953 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.823+0000 7ffa48bcd700 1 -- 172.21.15.52:0/1422306561 wait complete. 2024-04-02T13:43:20.953 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.824+0000 7ffa48bcd700 1 Processor -- start 2024-04-02T13:43:20.953 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.824+0000 7ffa48bcd700 1 -- start start 2024-04-02T13:43:20.953 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.824+0000 7ffa48bcd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa44108090 0x7ffa4419bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:20.953 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.824+0000 7ffa48bcd700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4410c5f0 con 0x7ffa44108090 2024-04-02T13:43:20.953 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.824+0000 7ffa4259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa44108090 0x7ffa4419bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:20.953 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.824+0000 7ffa4259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa44108090 0x7ffa4419bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53670/0 (socket says 172.21.15.52:53670) 2024-04-02T13:43:20.953 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.824+0000 7ffa4259c700 1 -- 172.21.15.52:0/3711842724 learned_addr learned my addr 172.21.15.52:0/3711842724 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:20.953 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.824+0000 7ffa4259c700 1 -- 172.21.15.52:0/3711842724 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4419c4f0 con 0x7ffa44108090 2024-04-02T13:43:20.953 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.824+0000 7ffa4259c700 1 --2- 172.21.15.52:0/3711842724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa44108090 0x7ffa4419bfb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009530 tx=0x7ffa3c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:20.953 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.825+0000 7ffa337fe700 1 -- 172.21.15.52:0/3711842724 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa3c020020 con 0x7ffa44108090 2024-04-02T13:43:20.953 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.825+0000 7ffa337fe700 1 -- 172.21.15.52:0/3711842724 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffa3c003b50 con 0x7ffa44108090 2024-04-02T13:43:20.954 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.825+0000 7ffa48bcd700 1 -- 172.21.15.52:0/3711842724 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4419c780 con 0x7ffa44108090 2024-04-02T13:43:20.954 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.825+0000 7ffa337fe700 1 -- 172.21.15.52:0/3711842724 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa3c017040 con 0x7ffa44108090 2024-04-02T13:43:20.954 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.825+0000 7ffa48bcd700 1 -- 172.21.15.52:0/3711842724 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa4419f470 con 0x7ffa44108090 2024-04-02T13:43:20.954 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.826+0000 7ffa337fe700 1 -- 172.21.15.52:0/3711842724 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7ffa3c023020 con 0x7ffa44108090 2024-04-02T13:43:20.954 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.826+0000 7ffa48bcd700 1 -- 172.21.15.52:0/3711842724 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa44063c20 con 0x7ffa44108090 2024-04-02T13:43:20.954 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.826+0000 7ffa337fe700 1 --2- 172.21.15.52:0/3711842724 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7ffa2c0395b0 0x7ffa2c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:20.954 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.826+0000 7ffa337fe700 1 -- 172.21.15.52:0/3711842724 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffa3c00f050 con 0x7ffa44108090 2024-04-02T13:43:20.954 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.826+0000 7ffa41d9b700 1 --2- 172.21.15.52:0/3711842724 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7ffa2c0395b0 0x7ffa2c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:20.954 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.826+0000 7ffa41d9b700 1 --2- 172.21.15.52:0/3711842724 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7ffa2c0395b0 0x7ffa2c03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffa38006fd0 tx=0x7ffa38006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:20.954 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:19.831+0000 7ffa337fe700 1 -- 172.21.15.52:0/3711842724 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ffa3c02a020 con 0x7ffa44108090 2024-04-02T13:43:20.954 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:20.103+0000 7ffa48bcd700 1 -- 172.21.15.52:0/3711842724 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7ffa4419f640 con 0x7ffa44108090 2024-04-02T13:43:20.954 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:20.279+0000 7ffa337fe700 1 -- 172.21.15.52:0/3711842724 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7ffa4419f640 con 0x7ffa44108090 2024-04-02T13:43:20.954 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:20.281+0000 7ffa337fe700 1 -- 172.21.15.52:0/3711842724 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7ffa3c012430 con 0x7ffa44108090 2024-04-02T13:43:20.955 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:20.282+0000 7ffa48bcd700 1 -- 172.21.15.52:0/3711842724 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7ffa2c0395b0 msgr2=0x7ffa2c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:20.955 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:20.282+0000 7ffa48bcd700 1 --2- 172.21.15.52:0/3711842724 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7ffa2c0395b0 0x7ffa2c03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffa38006fd0 tx=0x7ffa38006e40 comp rx=0 tx=0).stop 2024-04-02T13:43:20.955 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:20.282+0000 7ffa48bcd700 1 -- 172.21.15.52:0/3711842724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa44108090 msgr2=0x7ffa4419bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:20.955 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:20.282+0000 7ffa48bcd700 1 --2- 172.21.15.52:0/3711842724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa44108090 0x7ffa4419bfb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009530 tx=0x7ffa3c00bd50 comp rx=0 tx=0).stop 2024-04-02T13:43:20.955 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:20.282+0000 7ffa48bcd700 1 -- 172.21.15.52:0/3711842724 shutdown_connections 2024-04-02T13:43:20.955 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:20.282+0000 7ffa48bcd700 1 --2- 172.21.15.52:0/3711842724 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7ffa2c0395b0 0x7ffa2c03ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:20.955 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:20.282+0000 7ffa48bcd700 1 --2- 172.21.15.52:0/3711842724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa44108090 0x7ffa4419bfb0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:20.955 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:20.282+0000 7ffa48bcd700 1 -- 172.21.15.52:0/3711842724 >> 172.21.15.52:0/3711842724 conn(0x7ffa44103630 msgr2=0x7ffa44104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:20.955 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:20.283+0000 7ffa48bcd700 1 -- 172.21.15.52:0/3711842724 shutdown_connections 2024-04-02T13:43:20.955 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:20.283+0000 7ffa48bcd700 1 -- 172.21.15.52:0/3711842724 wait complete. 2024-04-02T13:43:21.180 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:20 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:20.856+0000 7f6382a4e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T13:43:21.180 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:20 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:20.954+0000 7f6382a4e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T13:43:21.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:21 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3711842724' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-02T13:43:21.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:21 smithi052 ceph-mon[32104]: mgrmap e5: x(active, since 6s) 2024-04-02T13:43:21.875 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:21 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:21.630+0000 7f6382a4e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T13:43:21.875 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:21 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:21.729+0000 7f6382a4e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T13:43:21.875 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:21 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:21.872+0000 7f6382a4e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T13:43:22.267 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:21 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:21.971+0000 7f6382a4e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T13:43:22.267 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:22 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:22.057+0000 7f6382a4e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T13:43:22.275 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-04-02T13:43:22.275 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-02T13:43:22.275 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-02T13:43:22.275 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-02T13:43:22.275 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-02T13:43:22.275 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-04-02T13:43:22.276 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.385+0000 7fdd40c22700 1 Processor -- start 2024-04-02T13:43:22.276 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.385+0000 7fdd40c22700 1 -- start start 2024-04-02T13:43:22.276 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.385+0000 7fdd40c22700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd3c108b00 0x7fdd3c10aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:22.276 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.385+0000 7fdd40c22700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3c07c680 con 0x7fdd3c108b00 2024-04-02T13:43:22.276 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.385+0000 7fdd3a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd3c108b00 0x7fdd3c10aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:22.276 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.385+0000 7fdd3a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd3c108b00 0x7fdd3c10aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53704/0 (socket says 172.21.15.52:53704) 2024-04-02T13:43:22.276 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.385+0000 7fdd3a59c700 1 -- 172.21.15.52:0/3115743508 learned_addr learned my addr 172.21.15.52:0/3115743508 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:22.276 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.386+0000 7fdd3a59c700 1 -- 172.21.15.52:0/3115743508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd3c10b430 con 0x7fdd3c108b00 2024-04-02T13:43:22.276 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.386+0000 7fdd3a59c700 1 --2- 172.21.15.52:0/3115743508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd3c108b00 0x7fdd3c10aef0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0098a0 tx=0x7fdd2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9601695215780ff server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:22.276 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.386+0000 7fdd3959a700 1 -- 172.21.15.52:0/3115743508 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd2c004160 con 0x7fdd3c108b00 2024-04-02T13:43:22.276 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.386+0000 7fdd3959a700 1 -- 172.21.15.52:0/3115743508 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdd2c004300 con 0x7fdd3c108b00 2024-04-02T13:43:22.276 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.386+0000 7fdd3959a700 1 -- 172.21.15.52:0/3115743508 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd2c011b50 con 0x7fdd3c108b00 2024-04-02T13:43:22.276 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.387+0000 7fdd40c22700 1 -- 172.21.15.52:0/3115743508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd3c108b00 msgr2=0x7fdd3c10aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:22.277 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.387+0000 7fdd40c22700 1 --2- 172.21.15.52:0/3115743508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd3c108b00 0x7fdd3c10aef0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0098a0 tx=0x7fdd2c009bb0 comp rx=0 tx=0).stop 2024-04-02T13:43:22.277 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.387+0000 7fdd40c22700 1 -- 172.21.15.52:0/3115743508 shutdown_connections 2024-04-02T13:43:22.277 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.387+0000 7fdd40c22700 1 --2- 172.21.15.52:0/3115743508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd3c108b00 0x7fdd3c10aef0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:22.277 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.387+0000 7fdd40c22700 1 -- 172.21.15.52:0/3115743508 >> 172.21.15.52:0/3115743508 conn(0x7fdd3c1024d0 msgr2=0x7fdd3c104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:22.277 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.387+0000 7fdd40c22700 1 -- 172.21.15.52:0/3115743508 shutdown_connections 2024-04-02T13:43:22.277 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.387+0000 7fdd40c22700 1 -- 172.21.15.52:0/3115743508 wait complete. 2024-04-02T13:43:22.277 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.388+0000 7fdd40c22700 1 Processor -- start 2024-04-02T13:43:22.277 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.388+0000 7fdd40c22700 1 -- start start 2024-04-02T13:43:22.277 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.388+0000 7fdd40c22700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd3c108b00 0x7fdd3c19bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:22.277 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.388+0000 7fdd40c22700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3c10c3f0 con 0x7fdd3c108b00 2024-04-02T13:43:22.277 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.388+0000 7fdd3a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd3c108b00 0x7fdd3c19bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:22.277 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.388+0000 7fdd3a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd3c108b00 0x7fdd3c19bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53710/0 (socket says 172.21.15.52:53710) 2024-04-02T13:43:22.277 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.388+0000 7fdd3a59c700 1 -- 172.21.15.52:0/3663158620 learned_addr learned my addr 172.21.15.52:0/3663158620 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:22.277 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.388+0000 7fdd3a59c700 1 -- 172.21.15.52:0/3663158620 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd3c19c510 con 0x7fdd3c108b00 2024-04-02T13:43:22.278 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.388+0000 7fdd3a59c700 1 --2- 172.21.15.52:0/3663158620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd3c108b00 0x7fdd3c19bfd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c011b60 tx=0x7fdd2c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:22.278 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.389+0000 7fdd2b7fe700 1 -- 172.21.15.52:0/3663158620 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd2c003b20 con 0x7fdd3c108b00 2024-04-02T13:43:22.278 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.389+0000 7fdd2b7fe700 1 -- 172.21.15.52:0/3663158620 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdd2c003cc0 con 0x7fdd3c108b00 2024-04-02T13:43:22.278 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.389+0000 7fdd2b7fe700 1 -- 172.21.15.52:0/3663158620 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd2c024840 con 0x7fdd3c108b00 2024-04-02T13:43:22.278 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.389+0000 7fdd40c22700 1 -- 172.21.15.52:0/3663158620 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd3c19c7a0 con 0x7fdd3c108b00 2024-04-02T13:43:22.278 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.389+0000 7fdd40c22700 1 -- 172.21.15.52:0/3663158620 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd3c19f490 con 0x7fdd3c108b00 2024-04-02T13:43:22.278 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.390+0000 7fdd2b7fe700 1 -- 172.21.15.52:0/3663158620 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fdd2c022020 con 0x7fdd3c108b00 2024-04-02T13:43:22.278 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.390+0000 7fdd40c22700 1 -- 172.21.15.52:0/3663158620 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd3c063c20 con 0x7fdd3c108b00 2024-04-02T13:43:22.278 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.390+0000 7fdd2b7fe700 1 --2- 172.21.15.52:0/3663158620 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7fdd24039600 0x7fdd2403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:22.278 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.390+0000 7fdd2b7fe700 1 -- 172.21.15.52:0/3663158620 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdd2c011e00 con 0x7fdd3c108b00 2024-04-02T13:43:22.278 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.390+0000 7fdd39d9b700 1 -- 172.21.15.52:0/3663158620 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7fdd24039600 msgr2=0x7fdd2403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2368921152 2024-04-02T13:43:22.278 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.390+0000 7fdd39d9b700 1 --2- 172.21.15.52:0/3663158620 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7fdd24039600 0x7fdd2403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:43:22.278 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.395+0000 7fdd2b7fe700 1 -- 172.21.15.52:0/3663158620 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fdd2c00ba30 con 0x7fdd3c108b00 2024-04-02T13:43:22.279 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.591+0000 7fdd39d9b700 1 -- 172.21.15.52:0/3663158620 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7fdd24039600 msgr2=0x7fdd2403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2368921152 2024-04-02T13:43:22.279 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.591+0000 7fdd39d9b700 1 --2- 172.21.15.52:0/3663158620 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7fdd24039600 0x7fdd2403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T13:43:22.279 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.666+0000 7fdd40c22700 1 -- 172.21.15.52:0/3663158620 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fdd3c19f660 con 0x7fdd3c108b00 2024-04-02T13:43:22.279 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.666+0000 7fdd2b7fe700 1 -- 172.21.15.52:0/3663158620 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fdd3c19f660 con 0x7fdd3c108b00 2024-04-02T13:43:22.279 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.668+0000 7fdd40c22700 1 -- 172.21.15.52:0/3663158620 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7fdd24039600 msgr2=0x7fdd2403bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:43:22.279 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.668+0000 7fdd40c22700 1 --2- 172.21.15.52:0/3663158620 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7fdd24039600 0x7fdd2403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:22.279 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.668+0000 7fdd40c22700 1 -- 172.21.15.52:0/3663158620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd3c108b00 msgr2=0x7fdd3c19bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:22.279 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.668+0000 7fdd40c22700 1 --2- 172.21.15.52:0/3663158620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd3c108b00 0x7fdd3c19bfd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c011b60 tx=0x7fdd2c011c40 comp rx=0 tx=0).stop 2024-04-02T13:43:22.279 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.668+0000 7fdd40c22700 1 -- 172.21.15.52:0/3663158620 shutdown_connections 2024-04-02T13:43:22.279 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.668+0000 7fdd40c22700 1 --2- 172.21.15.52:0/3663158620 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7fdd24039600 0x7fdd2403bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:22.279 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.668+0000 7fdd40c22700 1 --2- 172.21.15.52:0/3663158620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd3c108b00 0x7fdd3c19bfd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:22.279 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.668+0000 7fdd40c22700 1 -- 172.21.15.52:0/3663158620 >> 172.21.15.52:0/3663158620 conn(0x7fdd3c1024d0 msgr2=0x7fdd3c103010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:22.279 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.668+0000 7fdd40c22700 1 -- 172.21.15.52:0/3663158620 shutdown_connections 2024-04-02T13:43:22.279 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:21.668+0000 7fdd40c22700 1 -- 172.21.15.52:0/3663158620 wait complete. 2024-04-02T13:43:22.279 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for the mgr to restart... 2024-04-02T13:43:22.280 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mgr epoch 5... 2024-04-02T13:43:22.540 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:22 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3663158620' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-02T13:43:22.541 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:22 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:22.331+0000 7f6382a4e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T13:43:22.541 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:22 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:22.444+0000 7f6382a4e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T13:43:22.541 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:22 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:22.537+0000 7f6382a4e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T13:43:22.819 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:22 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:22.636+0000 7f6382a4e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T13:43:22.819 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:22 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:22.816+0000 7f6382a4e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T13:43:24.089 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:23 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:23.808+0000 7f6382a4e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T13:43:24.089 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:23 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:23.947+0000 7f6382a4e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T13:43:24.387 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:24 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:24.291+0000 7f6382a4e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T13:43:24.387 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:24 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:24.383+0000 7f6382a4e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T13:43:24.743 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:24 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:24.587+0000 7f6382a4e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:43:25.075 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:24 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:24.780+0000 7f6382a4e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T13:43:25.335 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:25 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:25.072+0000 7f6382a4e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:43:25.335 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:25 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:25.332+0000 7f6382a4e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T13:43:25.589 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:25 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:25.575+0000 7f6382a4e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T13:43:26.839 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:26 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:26.404+0000 7f6382a4e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T13:43:27.310 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:26 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:26.985+0000 7f6382a4e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T13:43:27.310 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:27 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:27.085+0000 7f6382a4e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T13:43:27.310 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:27 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:27.307+0000 7f6382a4e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T13:43:27.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:27 smithi052 ceph-mon[32104]: Active manager daemon x restarted 2024-04-02T13:43:27.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:27 smithi052 ceph-mon[32104]: Activating manager daemon x 2024-04-02T13:43:27.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:27 smithi052 ceph-mon[32104]: osdmap e2: 0 total, 0 up, 0 in 2024-04-02T13:43:27.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:27 smithi052 ceph-mon[32104]: mgrmap e6: x(active, starting, since 0.00314194s) 2024-04-02T13:43:27.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:27 smithi052 ceph-mon[32104]: from='mgr.14120 172.21.15.52:0/3185562986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:43:27.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:27 smithi052 ceph-mon[32104]: from='mgr.14120 172.21.15.52:0/3185562986' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T13:43:27.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:27 smithi052 ceph-mon[32104]: from='mgr.14120 172.21.15.52:0/3185562986' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T13:43:27.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:27 smithi052 ceph-mon[32104]: from='mgr.14120 172.21.15.52:0/3185562986' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T13:43:27.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:27 smithi052 ceph-mon[32104]: from='mgr.14120 172.21.15.52:0/3185562986' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T13:43:27.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:27 smithi052 ceph-mon[32104]: Manager daemon x is now available 2024-04-02T13:43:27.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:27 smithi052 ceph-mon[32104]: from='mgr.14120 172.21.15.52:0/3185562986' entity='mgr.x' 2024-04-02T13:43:27.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:27 smithi052 ceph-mon[32104]: from='mgr.14120 172.21.15.52:0/3185562986' entity='mgr.x' 2024-04-02T13:43:27.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:27 smithi052 ceph-mon[32104]: from='mgr.14120 172.21.15.52:0/3185562986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:43:27.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:27 smithi052 ceph-mon[32104]: from='mgr.14120 172.21.15.52:0/3185562986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:43:27.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:27 smithi052 ceph-mon[32104]: from='mgr.14120 172.21.15.52:0/3185562986' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T13:43:28.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:28 smithi052 ceph-mon[32104]: Found migration_current of "None". Setting to last migration. 2024-04-02T13:43:28.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:28 smithi052 ceph-mon[32104]: from='mgr.14120 172.21.15.52:0/3185562986' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T13:43:28.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:28 smithi052 ceph-mon[32104]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-02T13:43:28.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:28 smithi052 ceph-mon[32104]: mgrmap e7: x(active, since 1.0054s) 2024-04-02T13:43:28.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:28 smithi052 ceph-mon[32104]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-02T13:43:28.969 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-04-02T13:43:28.978 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-02T13:43:28.978 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-02T13:43:28.978 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-04-02T13:43:28.978 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.701+0000 7f975c781700 1 Processor -- start 2024-04-02T13:43:28.978 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.701+0000 7f975c781700 1 -- start start 2024-04-02T13:43:28.978 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.701+0000 7f975c781700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9754106f50 0x7f9754107390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:28.978 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.701+0000 7f975c781700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f97541078d0 con 0x7f9754106f50 2024-04-02T13:43:28.978 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.701+0000 7f975a51d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9754106f50 0x7f9754107390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:28.978 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.701+0000 7f975a51d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9754106f50 0x7f9754107390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39876/0 (socket says 172.21.15.52:39876) 2024-04-02T13:43:28.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.701+0000 7f975a51d700 1 -- 172.21.15.52:0/1323594237 learned_addr learned my addr 172.21.15.52:0/1323594237 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:28.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.702+0000 7f975a51d700 1 -- 172.21.15.52:0/1323594237 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97541080e0 con 0x7f9754106f50 2024-04-02T13:43:28.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.702+0000 7f975a51d700 1 --2- 172.21.15.52:0/1323594237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9754106f50 0x7f9754107390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f974400bb40 tx=0x7f974400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc12cdd23cc5a894 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:28.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.702+0000 7f975951b700 1 -- 172.21.15.52:0/1323594237 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f974400dbe0 con 0x7f9754106f50 2024-04-02T13:43:28.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.702+0000 7f975951b700 1 -- 172.21.15.52:0/1323594237 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f97440044f0 con 0x7f9754106f50 2024-04-02T13:43:28.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.702+0000 7f975951b700 1 -- 172.21.15.52:0/1323594237 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9744010470 con 0x7f9754106f50 2024-04-02T13:43:28.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.703+0000 7f975c781700 1 -- 172.21.15.52:0/1323594237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9754106f50 msgr2=0x7f9754107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:28.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.703+0000 7f975c781700 1 --2- 172.21.15.52:0/1323594237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9754106f50 0x7f9754107390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f974400bb40 tx=0x7f974400be50 comp rx=0 tx=0).stop 2024-04-02T13:43:28.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.703+0000 7f975c781700 1 -- 172.21.15.52:0/1323594237 shutdown_connections 2024-04-02T13:43:28.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.703+0000 7f975c781700 1 --2- 172.21.15.52:0/1323594237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9754106f50 0x7f9754107390 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.703+0000 7f975c781700 1 -- 172.21.15.52:0/1323594237 >> 172.21.15.52:0/1323594237 conn(0x7f9754102560 msgr2=0x7f9754104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:28.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.703+0000 7f975c781700 1 -- 172.21.15.52:0/1323594237 shutdown_connections 2024-04-02T13:43:28.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.703+0000 7f975c781700 1 -- 172.21.15.52:0/1323594237 wait complete. 2024-04-02T13:43:28.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.704+0000 7f975c781700 1 Processor -- start 2024-04-02T13:43:28.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.704+0000 7f975c781700 1 -- start start 2024-04-02T13:43:28.980 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.704+0000 7f975c781700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9754106f50 0x7f9754199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:28.980 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.704+0000 7f975c781700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f975407b830 con 0x7f9754106f50 2024-04-02T13:43:28.980 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.704+0000 7f975a51d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9754106f50 0x7f9754199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:28.980 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.704+0000 7f975a51d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9754106f50 0x7f9754199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39886/0 (socket says 172.21.15.52:39886) 2024-04-02T13:43:28.980 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.704+0000 7f975a51d700 1 -- 172.21.15.52:0/3086625036 learned_addr learned my addr 172.21.15.52:0/3086625036 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:28.980 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.704+0000 7f975a51d700 1 -- 172.21.15.52:0/3086625036 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975419a390 con 0x7f9754106f50 2024-04-02T13:43:28.980 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.705+0000 7f975a51d700 1 --2- 172.21.15.52:0/3086625036 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9754106f50 0x7f9754199e50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f974400b340 tx=0x7f974400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:28.980 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.705+0000 7f974b7fe700 1 -- 172.21.15.52:0/3086625036 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9744027070 con 0x7f9754106f50 2024-04-02T13:43:28.980 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.705+0000 7f974b7fe700 1 -- 172.21.15.52:0/3086625036 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9744010b40 con 0x7f9754106f50 2024-04-02T13:43:28.980 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.705+0000 7f974b7fe700 1 -- 172.21.15.52:0/3086625036 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9744022b00 con 0x7f9754106f50 2024-04-02T13:43:28.980 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.705+0000 7f975c781700 1 -- 172.21.15.52:0/3086625036 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f975419a620 con 0x7f9754106f50 2024-04-02T13:43:28.980 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.705+0000 7f975c781700 1 -- 172.21.15.52:0/3086625036 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f975419d2c0 con 0x7f9754106f50 2024-04-02T13:43:28.980 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.706+0000 7f974b7fe700 1 -- 172.21.15.52:0/3086625036 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f974402a070 con 0x7f9754106f50 2024-04-02T13:43:28.980 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.706+0000 7f974b7fe700 1 --2- 172.21.15.52:0/3086625036 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f9740039600 0x7f974003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:28.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.706+0000 7f974b7fe700 1 -- 172.21.15.52:0/3086625036 --> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f974003c1d0 con 0x7f9740039600 2024-04-02T13:43:28.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.706+0000 7f974b7fe700 1 -- 172.21.15.52:0/3086625036 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f974401f070 con 0x7f9754106f50 2024-04-02T13:43:28.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.706+0000 7f9759d1c700 1 -- 172.21.15.52:0/3086625036 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f9740039600 msgr2=0x7f974003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2368921152 2024-04-02T13:43:28.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.706+0000 7f9759d1c700 1 --2- 172.21.15.52:0/3086625036 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f9740039600 0x7f974003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:43:28.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.907+0000 7f9759d1c700 1 -- 172.21.15.52:0/3086625036 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f9740039600 msgr2=0x7f974003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2368921152 2024-04-02T13:43:28.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:22.907+0000 7f9759d1c700 1 --2- 172.21.15.52:0/3086625036 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f9740039600 0x7f974003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T13:43:28.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:23.308+0000 7f9759d1c700 1 -- 172.21.15.52:0/3086625036 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f9740039600 msgr2=0x7f974003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2368921152 2024-04-02T13:43:28.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:23.308+0000 7f9759d1c700 1 --2- 172.21.15.52:0/3086625036 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f9740039600 0x7f974003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-02T13:43:28.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:24.109+0000 7f9759d1c700 1 -- 172.21.15.52:0/3086625036 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f9740039600 msgr2=0x7f974003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2368921152 2024-04-02T13:43:28.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:24.109+0000 7f9759d1c700 1 --2- 172.21.15.52:0/3086625036 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f9740039600 0x7f974003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-02T13:43:28.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:25.710+0000 7f9759d1c700 1 -- 172.21.15.52:0/3086625036 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f9740039600 msgr2=0x7f974003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2368921152 2024-04-02T13:43:28.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:25.711+0000 7f9759d1c700 1 --2- 172.21.15.52:0/3086625036 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f9740039600 0x7f974003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-02T13:43:28.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:27.310+0000 7f974b7fe700 1 -- 172.21.15.52:0/3086625036 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f974402cce0 con 0x7f9754106f50 2024-04-02T13:43:28.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:27.310+0000 7f974b7fe700 1 -- 172.21.15.52:0/3086625036 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f9740039600 msgr2=0x7f974003bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:43:28.982 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:27.310+0000 7f974b7fe700 1 --2- 172.21.15.52:0/3086625036 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f9740039600 0x7f974003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.982 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:28.311+0000 7f974b7fe700 1 -- 172.21.15.52:0/3086625036 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f974402c440 con 0x7f9754106f50 2024-04-02T13:43:28.982 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:28.311+0000 7f974b7fe700 1 --2- 172.21.15.52:0/3086625036 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f974003cef0 0x7f974003f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:28.982 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:28.311+0000 7f974b7fe700 1 -- 172.21.15.52:0/3086625036 --> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f974402ee70 con 0x7f974003cef0 2024-04-02T13:43:28.982 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:28.312+0000 7f9759d1c700 1 --2- 172.21.15.52:0/3086625036 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f974003cef0 0x7f974003f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:28.982 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:28.313+0000 7f9759d1c700 1 --2- 172.21.15.52:0/3086625036 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f974003cef0 0x7f974003f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9750003a10 tx=0x7f97500092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:28.982 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:28.313+0000 7f974b7fe700 1 -- 172.21.15.52:0/3086625036 <== mgr.14120 v2:172.21.15.52:6800/673843171 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f974402ee70 con 0x7f974003cef0 2024-04-02T13:43:28.982 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:28.320+0000 7f975c781700 1 -- 172.21.15.52:0/3086625036 --> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f9754063c20 con 0x7f974003cef0 2024-04-02T13:43:28.982 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:28.320+0000 7f974b7fe700 1 -- 172.21.15.52:0/3086625036 <== mgr.14120 v2:172.21.15.52:6800/673843171 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f9754063c20 con 0x7f974003cef0 2024-04-02T13:43:28.982 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:28.320+0000 7f975c781700 1 -- 172.21.15.52:0/3086625036 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f974003cef0 msgr2=0x7f974003f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:28.982 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:28.320+0000 7f975c781700 1 --2- 172.21.15.52:0/3086625036 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f974003cef0 0x7f974003f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9750003a10 tx=0x7f97500092b0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.982 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:28.320+0000 7f975c781700 1 -- 172.21.15.52:0/3086625036 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9754106f50 msgr2=0x7f9754199e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:28.982 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:28.320+0000 7f975c781700 1 --2- 172.21.15.52:0/3086625036 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9754106f50 0x7f9754199e50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f974400b340 tx=0x7f974400dd90 comp rx=0 tx=0).stop 2024-04-02T13:43:28.982 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:28.320+0000 7f975c781700 1 -- 172.21.15.52:0/3086625036 shutdown_connections 2024-04-02T13:43:28.983 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:28.320+0000 7f975c781700 1 --2- 172.21.15.52:0/3086625036 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f974003cef0 0x7f974003f2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.983 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:28.320+0000 7f975c781700 1 --2- 172.21.15.52:0/3086625036 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9754106f50 0x7f9754199e50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.983 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:28.320+0000 7f975c781700 1 --2- 172.21.15.52:0/3086625036 >> [v2:172.21.15.52:6800/2368921152,v1:172.21.15.52:6801/2368921152] conn(0x7f9740039600 0x7f974003bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.983 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:28.320+0000 7f975c781700 1 -- 172.21.15.52:0/3086625036 >> 172.21.15.52:0/3086625036 conn(0x7f9754102560 msgr2=0x7f9754190940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:28.983 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:28.321+0000 7f975c781700 1 -- 172.21.15.52:0/3086625036 shutdown_connections 2024-04-02T13:43:28.983 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:28.321+0000 7f975c781700 1 -- 172.21.15.52:0/3086625036 wait complete. 2024-04-02T13:43:28.983 INFO:teuthology.orchestra.run.smithi052.stdout:mgr epoch 5 is available 2024-04-02T13:43:28.983 INFO:teuthology.orchestra.run.smithi052.stdout:Setting orchestrator backend to cephadm... 2024-04-02T13:43:30.249 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:29 smithi052 ceph-mon[32104]: from='mgr.14120 172.21.15.52:0/3185562986' entity='mgr.x' 2024-04-02T13:43:30.249 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:29 smithi052 ceph-mon[32104]: from='mgr.14120 172.21.15.52:0/3185562986' entity='mgr.x' 2024-04-02T13:43:30.249 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:29 smithi052 ceph-mon[32104]: from='mgr.14120 172.21.15.52:0/3185562986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:43:30.249 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:29 smithi052 ceph-mon[32104]: from='mgr.14120 172.21.15.52:0/3185562986' entity='mgr.x' 2024-04-02T13:43:30.249 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:29 smithi052 ceph-mon[32104]: from='mgr.14120 172.21.15.52:0/3185562986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:43:30.262 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.410+0000 7f14c806e700 1 Processor -- start 2024-04-02T13:43:30.262 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.410+0000 7f14c806e700 1 -- start start 2024-04-02T13:43:30.262 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.410+0000 7f14c806e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14c01080d0 0x7f14c01084f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:30.262 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.410+0000 7f14c806e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f14c0108ac0 con 0x7f14c01080d0 2024-04-02T13:43:30.262 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.410+0000 7f14c5e0a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14c01080d0 0x7f14c01084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:30.263 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.410+0000 7f14c5e0a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14c01080d0 0x7f14c01084f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39956/0 (socket says 172.21.15.52:39956) 2024-04-02T13:43:30.263 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.410+0000 7f14c5e0a700 1 -- 172.21.15.52:0/2999241626 learned_addr learned my addr 172.21.15.52:0/2999241626 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:30.263 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.410+0000 7f14c5e0a700 1 -- 172.21.15.52:0/2999241626 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c0109280 con 0x7f14c01080d0 2024-04-02T13:43:30.263 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.411+0000 7f14c5e0a700 1 --2- 172.21.15.52:0/2999241626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14c01080d0 0x7f14c01084f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f14bc0098a0 tx=0x7f14bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae5f77e741fa34f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:30.263 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.411+0000 7f14c4e08700 1 -- 172.21.15.52:0/2999241626 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14bc004160 con 0x7f14c01080d0 2024-04-02T13:43:30.263 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.411+0000 7f14c4e08700 1 -- 172.21.15.52:0/2999241626 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f14bc004300 con 0x7f14c01080d0 2024-04-02T13:43:30.263 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.411+0000 7f14c4e08700 1 -- 172.21.15.52:0/2999241626 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14bc011b50 con 0x7f14c01080d0 2024-04-02T13:43:30.263 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.411+0000 7f14c806e700 1 -- 172.21.15.52:0/2999241626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14c01080d0 msgr2=0x7f14c01084f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:30.264 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.411+0000 7f14c806e700 1 --2- 172.21.15.52:0/2999241626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14c01080d0 0x7f14c01084f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f14bc0098a0 tx=0x7f14bc009bb0 comp rx=0 tx=0).stop 2024-04-02T13:43:30.264 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.411+0000 7f14c806e700 1 -- 172.21.15.52:0/2999241626 shutdown_connections 2024-04-02T13:43:30.264 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.411+0000 7f14c806e700 1 --2- 172.21.15.52:0/2999241626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14c01080d0 0x7f14c01084f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:30.264 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.411+0000 7f14c806e700 1 -- 172.21.15.52:0/2999241626 >> 172.21.15.52:0/2999241626 conn(0x7f14c01036b0 msgr2=0x7f14c0105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:30.264 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.412+0000 7f14c806e700 1 -- 172.21.15.52:0/2999241626 shutdown_connections 2024-04-02T13:43:30.264 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.412+0000 7f14c806e700 1 -- 172.21.15.52:0/2999241626 wait complete. 2024-04-02T13:43:30.264 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.412+0000 7f14c806e700 1 Processor -- start 2024-04-02T13:43:30.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.412+0000 7f14c806e700 1 -- start start 2024-04-02T13:43:30.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.412+0000 7f14c806e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14c01080d0 0x7f14c019c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:30.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.412+0000 7f14c806e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f14c010c630 con 0x7f14c01080d0 2024-04-02T13:43:30.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.412+0000 7f14c5e0a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14c01080d0 0x7f14c019c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:30.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.412+0000 7f14c5e0a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14c01080d0 0x7f14c019c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39970/0 (socket says 172.21.15.52:39970) 2024-04-02T13:43:30.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.412+0000 7f14c5e0a700 1 -- 172.21.15.52:0/3272791052 learned_addr learned my addr 172.21.15.52:0/3272791052 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:30.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.413+0000 7f14c5e0a700 1 -- 172.21.15.52:0/3272791052 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c019c5b0 con 0x7f14c01080d0 2024-04-02T13:43:30.265 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.413+0000 7f14c5e0a700 1 --2- 172.21.15.52:0/3272791052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14c01080d0 0x7f14c019c070 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f14bc011670 tx=0x7f14bc011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:30.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.413+0000 7f14b6ffd700 1 -- 172.21.15.52:0/3272791052 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14bc0039c0 con 0x7f14c01080d0 2024-04-02T13:43:30.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.413+0000 7f14c806e700 1 -- 172.21.15.52:0/3272791052 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c019c840 con 0x7f14c01080d0 2024-04-02T13:43:30.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.413+0000 7f14b6ffd700 1 -- 172.21.15.52:0/3272791052 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f14bc011e50 con 0x7f14c01080d0 2024-04-02T13:43:30.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.413+0000 7f14b6ffd700 1 -- 172.21.15.52:0/3272791052 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14bc01a5c0 con 0x7f14c01080d0 2024-04-02T13:43:30.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.413+0000 7f14c806e700 1 -- 172.21.15.52:0/3272791052 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c019f530 con 0x7f14c01080d0 2024-04-02T13:43:30.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.414+0000 7f14b6ffd700 1 -- 172.21.15.52:0/3272791052 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f14bc024430 con 0x7f14c01080d0 2024-04-02T13:43:30.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.414+0000 7f14c806e700 1 -- 172.21.15.52:0/3272791052 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14c0063c20 con 0x7f14c01080d0 2024-04-02T13:43:30.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.414+0000 7f14b6ffd700 1 --2- 172.21.15.52:0/3272791052 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f14ac039450 0x7f14ac03b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:30.266 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.414+0000 7f14b6ffd700 1 -- 172.21.15.52:0/3272791052 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f14bc04d9c0 con 0x7f14c01080d0 2024-04-02T13:43:30.267 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.414+0000 7f14c5609700 1 --2- 172.21.15.52:0/3272791052 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f14ac039450 0x7f14ac03b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:30.267 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.414+0000 7f14c5609700 1 --2- 172.21.15.52:0/3272791052 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f14ac039450 0x7f14ac03b910 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f14b0006fd0 tx=0x7f14b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:30.267 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.419+0000 7f14b6ffd700 1 -- 172.21.15.52:0/3272791052 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f14bc02d910 con 0x7f14c01080d0 2024-04-02T13:43:30.267 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.647+0000 7f14c806e700 1 -- 172.21.15.52:0/3272791052 --> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f14c0104920 con 0x7f14ac039450 2024-04-02T13:43:30.267 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.652+0000 7f14b6ffd700 1 -- 172.21.15.52:0/3272791052 <== mgr.14120 v2:172.21.15.52:6800/673843171 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f14c0104920 con 0x7f14ac039450 2024-04-02T13:43:30.267 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.655+0000 7f14c806e700 1 -- 172.21.15.52:0/3272791052 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f14ac039450 msgr2=0x7f14ac03b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:30.267 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.655+0000 7f14c806e700 1 --2- 172.21.15.52:0/3272791052 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f14ac039450 0x7f14ac03b910 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f14b0006fd0 tx=0x7f14b0006e40 comp rx=0 tx=0).stop 2024-04-02T13:43:30.268 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.655+0000 7f14c806e700 1 -- 172.21.15.52:0/3272791052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14c01080d0 msgr2=0x7f14c019c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:30.268 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.655+0000 7f14c806e700 1 --2- 172.21.15.52:0/3272791052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14c01080d0 0x7f14c019c070 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f14bc011670 tx=0x7f14bc011b60 comp rx=0 tx=0).stop 2024-04-02T13:43:30.268 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.655+0000 7f14c806e700 1 -- 172.21.15.52:0/3272791052 shutdown_connections 2024-04-02T13:43:30.268 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.655+0000 7f14c806e700 1 --2- 172.21.15.52:0/3272791052 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f14ac039450 0x7f14ac03b910 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:30.268 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.655+0000 7f14c806e700 1 --2- 172.21.15.52:0/3272791052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14c01080d0 0x7f14c019c070 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:30.268 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.655+0000 7f14c806e700 1 -- 172.21.15.52:0/3272791052 >> 172.21.15.52:0/3272791052 conn(0x7f14c01036b0 msgr2=0x7f14c0104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:30.268 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.655+0000 7f14c806e700 1 -- 172.21.15.52:0/3272791052 shutdown_connections 2024-04-02T13:43:30.268 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:29.655+0000 7f14c806e700 1 -- 172.21.15.52:0/3272791052 wait complete. 2024-04-02T13:43:31.212 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:30 smithi052 ceph-mon[32104]: [02/Apr/2024:13:43:29] ENGINE Bus STARTING 2024-04-02T13:43:31.213 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:30 smithi052 ceph-mon[32104]: [02/Apr/2024:13:43:29] ENGINE Serving on http://172.21.15.52:8765 2024-04-02T13:43:31.213 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:30 smithi052 ceph-mon[32104]: [02/Apr/2024:13:43:29] ENGINE Serving on https://172.21.15.52:7150 2024-04-02T13:43:31.213 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:30 smithi052 ceph-mon[32104]: [02/Apr/2024:13:43:29] ENGINE Bus STARTED 2024-04-02T13:43:31.213 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:30 smithi052 ceph-mon[32104]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:43:31.213 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:30 smithi052 ceph-mon[32104]: mgrmap e8: x(active, since 2s) 2024-04-02T13:43:31.504 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-02T13:43:31.504 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.684+0000 7fa3f5737700 1 Processor -- start 2024-04-02T13:43:31.504 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.684+0000 7fa3f5737700 1 -- start start 2024-04-02T13:43:31.504 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.684+0000 7fa3f5737700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3f0107420 0x7fa3f0107840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:31.504 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.684+0000 7fa3f5737700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f0107e10 con 0x7fa3f0107420 2024-04-02T13:43:31.504 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.685+0000 7fa3eeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3f0107420 0x7fa3f0107840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:31.505 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.685+0000 7fa3eeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3f0107420 0x7fa3f0107840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39978/0 (socket says 172.21.15.52:39978) 2024-04-02T13:43:31.505 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.685+0000 7fa3eeffd700 1 -- 172.21.15.52:0/65013024 learned_addr learned my addr 172.21.15.52:0/65013024 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:31.505 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.685+0000 7fa3eeffd700 1 -- 172.21.15.52:0/65013024 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f0108140 con 0x7fa3f0107420 2024-04-02T13:43:31.505 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.685+0000 7fa3eeffd700 1 --2- 172.21.15.52:0/65013024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3f0107420 0x7fa3f0107840 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00098a0 tx=0x7fa3e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c82a19a36af6bb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:31.505 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.685+0000 7fa3edffb700 1 -- 172.21.15.52:0/65013024 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3e0004160 con 0x7fa3f0107420 2024-04-02T13:43:31.505 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.685+0000 7fa3edffb700 1 -- 172.21.15.52:0/65013024 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa3e0004300 con 0x7fa3f0107420 2024-04-02T13:43:31.505 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.685+0000 7fa3edffb700 1 -- 172.21.15.52:0/65013024 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3e0011b50 con 0x7fa3f0107420 2024-04-02T13:43:31.505 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.686+0000 7fa3f5737700 1 -- 172.21.15.52:0/65013024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3f0107420 msgr2=0x7fa3f0107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:31.505 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.686+0000 7fa3f5737700 1 --2- 172.21.15.52:0/65013024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3f0107420 0x7fa3f0107840 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00098a0 tx=0x7fa3e0009bb0 comp rx=0 tx=0).stop 2024-04-02T13:43:31.505 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.686+0000 7fa3f5737700 1 -- 172.21.15.52:0/65013024 shutdown_connections 2024-04-02T13:43:31.505 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.686+0000 7fa3f5737700 1 --2- 172.21.15.52:0/65013024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3f0107420 0x7fa3f0107840 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:31.505 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.686+0000 7fa3f5737700 1 -- 172.21.15.52:0/65013024 >> 172.21.15.52:0/65013024 conn(0x7fa3f0101c80 msgr2=0x7fa3f01040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:31.505 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.686+0000 7fa3f5737700 1 -- 172.21.15.52:0/65013024 shutdown_connections 2024-04-02T13:43:31.505 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.686+0000 7fa3f5737700 1 -- 172.21.15.52:0/65013024 wait complete. 2024-04-02T13:43:31.505 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.687+0000 7fa3f5737700 1 Processor -- start 2024-04-02T13:43:31.506 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.687+0000 7fa3f5737700 1 -- start start 2024-04-02T13:43:31.506 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.687+0000 7fa3f5737700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3f0107420 0x7fa3f01920b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:31.506 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.687+0000 7fa3f5737700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f010b4f0 con 0x7fa3f0107420 2024-04-02T13:43:31.506 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.687+0000 7fa3eeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3f0107420 0x7fa3f01920b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:31.506 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.687+0000 7fa3eeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3f0107420 0x7fa3f01920b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39986/0 (socket says 172.21.15.52:39986) 2024-04-02T13:43:31.506 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.687+0000 7fa3eeffd700 1 -- 172.21.15.52:0/886398562 learned_addr learned my addr 172.21.15.52:0/886398562 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:31.506 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.687+0000 7fa3eeffd700 1 -- 172.21.15.52:0/886398562 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f01925f0 con 0x7fa3f0107420 2024-04-02T13:43:31.506 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.688+0000 7fa3eeffd700 1 --2- 172.21.15.52:0/886398562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3f0107420 0x7fa3f01920b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0011b60 tx=0x7fa3e0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:31.506 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.688+0000 7fa3e7fff700 1 -- 172.21.15.52:0/886398562 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3e0003b20 con 0x7fa3f0107420 2024-04-02T13:43:31.506 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.688+0000 7fa3e7fff700 1 -- 172.21.15.52:0/886398562 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa3e0003cc0 con 0x7fa3f0107420 2024-04-02T13:43:31.506 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.688+0000 7fa3e7fff700 1 -- 172.21.15.52:0/886398562 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3e0024840 con 0x7fa3f0107420 2024-04-02T13:43:31.506 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.688+0000 7fa3f5737700 1 -- 172.21.15.52:0/886398562 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f01907e0 con 0x7fa3f0107420 2024-04-02T13:43:31.506 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.688+0000 7fa3f5737700 1 -- 172.21.15.52:0/886398562 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f0190d20 con 0x7fa3f0107420 2024-04-02T13:43:31.506 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.689+0000 7fa3e7fff700 1 -- 172.21.15.52:0/886398562 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa3e0022020 con 0x7fa3f0107420 2024-04-02T13:43:31.507 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.689+0000 7fa3f5737700 1 -- 172.21.15.52:0/886398562 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f0063c20 con 0x7fa3f0107420 2024-04-02T13:43:31.507 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.689+0000 7fa3e7fff700 1 --2- 172.21.15.52:0/886398562 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7fa3dc039600 0x7fa3dc03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:31.507 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.689+0000 7fa3e7fff700 1 -- 172.21.15.52:0/886398562 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa3e0011e00 con 0x7fa3f0107420 2024-04-02T13:43:31.507 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.689+0000 7fa3ee7fc700 1 --2- 172.21.15.52:0/886398562 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7fa3dc039600 0x7fa3dc03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:31.507 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.689+0000 7fa3ee7fc700 1 --2- 172.21.15.52:0/886398562 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7fa3dc039600 0x7fa3dc03bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8006fd0 tx=0x7fa3d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:31.507 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.694+0000 7fa3e7fff700 1 -- 172.21.15.52:0/886398562 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa3e000ba30 con 0x7fa3f0107420 2024-04-02T13:43:31.507 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.904+0000 7fa3f5737700 1 -- 172.21.15.52:0/886398562 --> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fa3f0195ec0 con 0x7fa3dc039600 2024-04-02T13:43:31.507 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.904+0000 7fa3e7fff700 1 -- 172.21.15.52:0/886398562 <== mgr.14120 v2:172.21.15.52:6800/673843171 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fa3f0195ec0 con 0x7fa3dc039600 2024-04-02T13:43:31.507 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.906+0000 7fa3f5737700 1 -- 172.21.15.52:0/886398562 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7fa3dc039600 msgr2=0x7fa3dc03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:31.509 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.906+0000 7fa3f5737700 1 --2- 172.21.15.52:0/886398562 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7fa3dc039600 0x7fa3dc03bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8006fd0 tx=0x7fa3d8006e40 comp rx=0 tx=0).stop 2024-04-02T13:43:31.509 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.906+0000 7fa3f5737700 1 -- 172.21.15.52:0/886398562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3f0107420 msgr2=0x7fa3f01920b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:31.509 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.906+0000 7fa3f5737700 1 --2- 172.21.15.52:0/886398562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3f0107420 0x7fa3f01920b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0011b60 tx=0x7fa3e0011c40 comp rx=0 tx=0).stop 2024-04-02T13:43:31.510 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.907+0000 7fa3f5737700 1 -- 172.21.15.52:0/886398562 shutdown_connections 2024-04-02T13:43:31.510 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.907+0000 7fa3f5737700 1 --2- 172.21.15.52:0/886398562 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7fa3dc039600 0x7fa3dc03bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:31.510 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.907+0000 7fa3f5737700 1 --2- 172.21.15.52:0/886398562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3f0107420 0x7fa3f01920b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:31.510 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.907+0000 7fa3f5737700 1 -- 172.21.15.52:0/886398562 >> 172.21.15.52:0/886398562 conn(0x7fa3f0101c80 msgr2=0x7fa3f0102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:31.510 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.907+0000 7fa3f5737700 1 -- 172.21.15.52:0/886398562 shutdown_connections 2024-04-02T13:43:31.510 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:30.907+0000 7fa3f5737700 1 -- 172.21.15.52:0/886398562 wait complete. 2024-04-02T13:43:31.510 INFO:teuthology.orchestra.run.smithi052.stdout:Generating ssh key... 2024-04-02T13:43:32.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:31 smithi052 ceph-mon[32104]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:43:32.819 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:32 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: Generating public/private rsa key pair. 2024-04-02T13:43:32.819 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:32 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: Your identification has been saved in /tmp/tmpn5q18v8m/key. 2024-04-02T13:43:32.819 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:32 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: Your public key has been saved in /tmp/tmpn5q18v8m/key.pub. 2024-04-02T13:43:32.819 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:32 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: The key fingerprint is: 2024-04-02T13:43:32.819 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:32 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: SHA256:dNcD1DxK7+8eADtMT+Ec++/AR38RfjwYulTZBAKlGTk ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:43:32.819 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:32 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: The key's randomart image is: 2024-04-02T13:43:32.819 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:32 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: +---[RSA 3072]----+ 2024-04-02T13:43:32.820 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:32 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: | o++o*.. | 2024-04-02T13:43:32.820 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:32 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: | E+ =o@ | 2024-04-02T13:43:32.820 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:32 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: | .oo+.@o+ | 2024-04-02T13:43:32.820 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:32 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: | . .o.O *o.| 2024-04-02T13:43:32.820 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:32 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: | S * = =+| 2024-04-02T13:43:32.820 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:32 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: | . o.o.*| 2024-04-02T13:43:32.820 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:32 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: | . oo=| 2024-04-02T13:43:32.820 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:32 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: | ++| 2024-04-02T13:43:32.820 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:32 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: | o+| 2024-04-02T13:43:32.820 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:32 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: +----[SHA256]-----+ 2024-04-02T13:43:33.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:32 smithi052 ceph-mon[32104]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:43:33.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:32 smithi052 ceph-mon[32104]: Generating ssh key... 2024-04-02T13:43:33.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:32 smithi052 ceph-mon[32104]: from='mgr.14120 172.21.15.52:0/3185562986' entity='mgr.x' 2024-04-02T13:43:33.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:32 smithi052 ceph-mon[32104]: from='mgr.14120 172.21.15.52:0/3185562986' entity='mgr.x' 2024-04-02T13:43:33.117 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.926+0000 7f680fd9b700 1 Processor -- start 2024-04-02T13:43:33.117 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.926+0000 7f680fd9b700 1 -- start start 2024-04-02T13:43:33.117 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.927+0000 7f680fd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68081080d0 0x7f68081084f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:33.117 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.927+0000 7f680fd9b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6808108ac0 con 0x7f68081080d0 2024-04-02T13:43:33.118 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.927+0000 7f680db37700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68081080d0 0x7f68081084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:33.118 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.927+0000 7f680db37700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68081080d0 0x7f68081084f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37690/0 (socket says 172.21.15.52:37690) 2024-04-02T13:43:33.118 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.927+0000 7f680db37700 1 -- 172.21.15.52:0/490649928 learned_addr learned my addr 172.21.15.52:0/490649928 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:33.118 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.927+0000 7f680db37700 1 -- 172.21.15.52:0/490649928 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6808109280 con 0x7f68081080d0 2024-04-02T13:43:33.118 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.927+0000 7f680db37700 1 --2- 172.21.15.52:0/490649928 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68081080d0 0x7f68081084f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f68040098a0 tx=0x7f6804009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2bac2cf3bb04999 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:33.118 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.927+0000 7f680cb35700 1 -- 172.21.15.52:0/490649928 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6804004160 con 0x7f68081080d0 2024-04-02T13:43:33.119 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.927+0000 7f680cb35700 1 -- 172.21.15.52:0/490649928 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6804004300 con 0x7f68081080d0 2024-04-02T13:43:33.119 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.928+0000 7f680cb35700 1 -- 172.21.15.52:0/490649928 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6804011b50 con 0x7f68081080d0 2024-04-02T13:43:33.119 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.928+0000 7f680fd9b700 1 -- 172.21.15.52:0/490649928 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68081080d0 msgr2=0x7f68081084f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:33.119 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.928+0000 7f680fd9b700 1 --2- 172.21.15.52:0/490649928 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68081080d0 0x7f68081084f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f68040098a0 tx=0x7f6804009bb0 comp rx=0 tx=0).stop 2024-04-02T13:43:33.119 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.928+0000 7f680fd9b700 1 -- 172.21.15.52:0/490649928 shutdown_connections 2024-04-02T13:43:33.120 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.928+0000 7f680fd9b700 1 --2- 172.21.15.52:0/490649928 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68081080d0 0x7f68081084f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:33.120 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.928+0000 7f680fd9b700 1 -- 172.21.15.52:0/490649928 >> 172.21.15.52:0/490649928 conn(0x7f68081036b0 msgr2=0x7f6808105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:33.120 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.928+0000 7f680fd9b700 1 -- 172.21.15.52:0/490649928 shutdown_connections 2024-04-02T13:43:33.120 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.928+0000 7f680fd9b700 1 -- 172.21.15.52:0/490649928 wait complete. 2024-04-02T13:43:33.120 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.929+0000 7f680fd9b700 1 Processor -- start 2024-04-02T13:43:33.120 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.929+0000 7f680fd9b700 1 -- start start 2024-04-02T13:43:33.121 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.929+0000 7f680fd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68081080d0 0x7f680819e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:33.121 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.929+0000 7f680fd9b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f680810c630 con 0x7f68081080d0 2024-04-02T13:43:33.121 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.929+0000 7f680db37700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68081080d0 0x7f680819e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:33.121 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.929+0000 7f680db37700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68081080d0 0x7f680819e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37692/0 (socket says 172.21.15.52:37692) 2024-04-02T13:43:33.121 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.929+0000 7f680db37700 1 -- 172.21.15.52:0/3574688161 learned_addr learned my addr 172.21.15.52:0/3574688161 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:33.121 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.929+0000 7f680db37700 1 -- 172.21.15.52:0/3574688161 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680819e7b0 con 0x7f68081080d0 2024-04-02T13:43:33.122 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.929+0000 7f680db37700 1 --2- 172.21.15.52:0/3574688161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68081080d0 0x7f680819e270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6804000c00 tx=0x7f6804011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:33.122 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.930+0000 7f67feffd700 1 -- 172.21.15.52:0/3574688161 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6804003b20 con 0x7f68081080d0 2024-04-02T13:43:33.122 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.930+0000 7f67feffd700 1 -- 172.21.15.52:0/3574688161 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6804003cc0 con 0x7f68081080d0 2024-04-02T13:43:33.122 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.930+0000 7f680fd9b700 1 -- 172.21.15.52:0/3574688161 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f680819ea40 con 0x7f68081080d0 2024-04-02T13:43:33.122 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.930+0000 7f67feffd700 1 -- 172.21.15.52:0/3574688161 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6804024910 con 0x7f68081080d0 2024-04-02T13:43:33.123 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.930+0000 7f680fd9b700 1 -- 172.21.15.52:0/3574688161 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68081a1730 con 0x7f68081080d0 2024-04-02T13:43:33.123 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.931+0000 7f67feffd700 1 -- 172.21.15.52:0/3574688161 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6804022020 con 0x7f68081080d0 2024-04-02T13:43:33.123 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.931+0000 7f680fd9b700 1 -- 172.21.15.52:0/3574688161 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6808063c20 con 0x7f68081080d0 2024-04-02T13:43:33.123 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.931+0000 7f67feffd700 1 --2- 172.21.15.52:0/3574688161 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f67f4039600 0x7f67f403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:33.123 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.931+0000 7f67feffd700 1 -- 172.21.15.52:0/3574688161 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6804011e00 con 0x7f68081080d0 2024-04-02T13:43:33.123 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.931+0000 7f680d336700 1 --2- 172.21.15.52:0/3574688161 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f67f4039600 0x7f67f403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:33.124 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.931+0000 7f680d336700 1 --2- 172.21.15.52:0/3574688161 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f67f4039600 0x7f67f403bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f67f8006fd0 tx=0x7f67f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:33.124 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:31.936+0000 7f67feffd700 1 -- 172.21.15.52:0/3574688161 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f680401ad80 con 0x7f68081080d0 2024-04-02T13:43:33.124 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:32.145+0000 7f680fd9b700 1 -- 172.21.15.52:0/3574688161 --> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f6808104890 con 0x7f67f4039600 2024-04-02T13:43:33.124 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:32.514+0000 7f67feffd700 1 -- 172.21.15.52:0/3574688161 <== mgr.14120 v2:172.21.15.52:6800/673843171 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6808104890 con 0x7f67f4039600 2024-04-02T13:43:33.124 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:32.516+0000 7f680fd9b700 1 -- 172.21.15.52:0/3574688161 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f67f4039600 msgr2=0x7f67f403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:33.124 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:32.516+0000 7f680fd9b700 1 --2- 172.21.15.52:0/3574688161 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f67f4039600 0x7f67f403bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f67f8006fd0 tx=0x7f67f8006e40 comp rx=0 tx=0).stop 2024-04-02T13:43:33.125 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:32.516+0000 7f680fd9b700 1 -- 172.21.15.52:0/3574688161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68081080d0 msgr2=0x7f680819e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:33.125 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:32.516+0000 7f680fd9b700 1 --2- 172.21.15.52:0/3574688161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68081080d0 0x7f680819e270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6804000c00 tx=0x7f6804011c10 comp rx=0 tx=0).stop 2024-04-02T13:43:33.125 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:32.516+0000 7f680fd9b700 1 -- 172.21.15.52:0/3574688161 shutdown_connections 2024-04-02T13:43:33.125 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:32.516+0000 7f680fd9b700 1 --2- 172.21.15.52:0/3574688161 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f67f4039600 0x7f67f403bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:33.125 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:32.516+0000 7f680fd9b700 1 --2- 172.21.15.52:0/3574688161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68081080d0 0x7f680819e270 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:33.125 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:32.516+0000 7f680fd9b700 1 -- 172.21.15.52:0/3574688161 >> 172.21.15.52:0/3574688161 conn(0x7f68081036b0 msgr2=0x7f6808104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:33.126 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:32.516+0000 7f680fd9b700 1 -- 172.21.15.52:0/3574688161 shutdown_connections 2024-04-02T13:43:33.126 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:32.516+0000 7f680fd9b700 1 -- 172.21.15.52:0/3574688161 wait complete. 2024-04-02T13:43:34.294 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDg71LEBNc12A3gZoTZdHgWjUqn9ojL8ipKhZNRcFfgnSdpzo9/dppZ2/F4G7qJSJz694RpqP7dzhv6hJ+rTdWmNslRA7OroYXC69qiQxsSa8peyAQD/WeLjI3YfwmKToPYZkgYtPHAmNoUz5RkZZT7V9+C4To/LOVt/wfHRvpioicZxWIG7A04Bp7dRNhTr9ciVCEt+tXvi+ZHZ+t9Y+kLX1BxLM2nMJU1sYPjAak62k5d4kM9fjF7TsFIGxheh2f7klt7HeaY6WfjVIQyynX3N02E89A5xtWwiqc4wTjJUCfZJYp94Y7ixb3qlrLgw5+Xy19HwutBQbk4EfbEQeGx5082zQsD0qwb3QHmJFcc/CtQUuz5UQtcu9Nd35bD/OqcrS5dsELCtMrVU82oCYTnPOVokXfsZWt7sukceicyIruGhmgK9sRN7uuBx4NcBdUzK5iFxMQ4amo4EAiToF3gQ0j19TEH82penQRAIjbQhj5WebW5nuuAr6peihbvMqc= ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:43:34.294 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.485+0000 7f7419819700 1 Processor -- start 2024-04-02T13:43:34.294 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.485+0000 7f7419819700 1 -- start start 2024-04-02T13:43:34.295 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.486+0000 7f7419819700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74141080d0 0x7f74141084f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:34.295 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.486+0000 7f7419819700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7414108ac0 con 0x7f74141080d0 2024-04-02T13:43:34.295 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.486+0000 7f7412ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74141080d0 0x7f74141084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:34.295 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.486+0000 7f7412ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74141080d0 0x7f74141084f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37700/0 (socket says 172.21.15.52:37700) 2024-04-02T13:43:34.295 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.486+0000 7f7412ffd700 1 -- 172.21.15.52:0/1333758754 learned_addr learned my addr 172.21.15.52:0/1333758754 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:34.295 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.486+0000 7f7412ffd700 1 -- 172.21.15.52:0/1333758754 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7414109280 con 0x7f74141080d0 2024-04-02T13:43:34.295 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.486+0000 7f7412ffd700 1 --2- 172.21.15.52:0/1333758754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74141080d0 0x7f74141084f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f740c009e80 tx=0x7f740c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b0a35c48a8fd25 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:34.295 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.486+0000 7f7411ffb700 1 -- 172.21.15.52:0/1333758754 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f740c017070 con 0x7f74141080d0 2024-04-02T13:43:34.295 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.486+0000 7f7411ffb700 1 -- 172.21.15.52:0/1333758754 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f740c00b8c0 con 0x7f74141080d0 2024-04-02T13:43:34.295 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.486+0000 7f7411ffb700 1 -- 172.21.15.52:0/1333758754 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f740c0045c0 con 0x7f74141080d0 2024-04-02T13:43:34.295 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.487+0000 7f7419819700 1 -- 172.21.15.52:0/1333758754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74141080d0 msgr2=0x7f74141084f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:34.295 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.487+0000 7f7419819700 1 --2- 172.21.15.52:0/1333758754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74141080d0 0x7f74141084f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f740c009e80 tx=0x7f740c00b330 comp rx=0 tx=0).stop 2024-04-02T13:43:34.295 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.487+0000 7f7419819700 1 -- 172.21.15.52:0/1333758754 shutdown_connections 2024-04-02T13:43:34.296 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.487+0000 7f7419819700 1 --2- 172.21.15.52:0/1333758754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74141080d0 0x7f74141084f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:34.296 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.487+0000 7f7419819700 1 -- 172.21.15.52:0/1333758754 >> 172.21.15.52:0/1333758754 conn(0x7f74141036b0 msgr2=0x7f7414105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:34.296 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.487+0000 7f7419819700 1 -- 172.21.15.52:0/1333758754 shutdown_connections 2024-04-02T13:43:34.296 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.487+0000 7f7419819700 1 -- 172.21.15.52:0/1333758754 wait complete. 2024-04-02T13:43:34.296 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.488+0000 7f7419819700 1 Processor -- start 2024-04-02T13:43:34.296 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.488+0000 7f7419819700 1 -- start start 2024-04-02T13:43:34.296 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.488+0000 7f7419819700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74141080d0 0x7f741419bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:34.296 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.488+0000 7f7419819700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f741410c630 con 0x7f74141080d0 2024-04-02T13:43:34.296 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.488+0000 7f7412ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74141080d0 0x7f741419bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:34.296 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.488+0000 7f7412ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74141080d0 0x7f741419bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37704/0 (socket says 172.21.15.52:37704) 2024-04-02T13:43:34.296 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.488+0000 7f7412ffd700 1 -- 172.21.15.52:0/1053475691 learned_addr learned my addr 172.21.15.52:0/1053475691 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:34.296 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.488+0000 7f7412ffd700 1 -- 172.21.15.52:0/1053475691 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f741419c520 con 0x7f74141080d0 2024-04-02T13:43:34.296 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.489+0000 7f7412ffd700 1 --2- 172.21.15.52:0/1053475691 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74141080d0 0x7f741419bfe0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f740c0072a0 tx=0x7f740c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:34.296 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.489+0000 7f7418817700 1 -- 172.21.15.52:0/1053475691 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f740c020020 con 0x7f74141080d0 2024-04-02T13:43:34.296 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.489+0000 7f7418817700 1 -- 172.21.15.52:0/1053475691 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f740c003b50 con 0x7f74141080d0 2024-04-02T13:43:34.297 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.489+0000 7f7419819700 1 -- 172.21.15.52:0/1053475691 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f741419c7b0 con 0x7f74141080d0 2024-04-02T13:43:34.297 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.489+0000 7f7418817700 1 -- 172.21.15.52:0/1053475691 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f740c017040 con 0x7f74141080d0 2024-04-02T13:43:34.297 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.489+0000 7f7419819700 1 -- 172.21.15.52:0/1053475691 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f741419f4a0 con 0x7f74141080d0 2024-04-02T13:43:34.297 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.490+0000 7f7418817700 1 -- 172.21.15.52:0/1053475691 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f740c023020 con 0x7f74141080d0 2024-04-02T13:43:34.297 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.490+0000 7f7419819700 1 -- 172.21.15.52:0/1053475691 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7414063c20 con 0x7f74141080d0 2024-04-02T13:43:34.297 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.490+0000 7f7418817700 1 --2- 172.21.15.52:0/1053475691 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f7400039560 0x7f740003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:34.297 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.490+0000 7f7418817700 1 -- 172.21.15.52:0/1053475691 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f740c00f050 con 0x7f74141080d0 2024-04-02T13:43:34.297 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.490+0000 7f74127fc700 1 --2- 172.21.15.52:0/1053475691 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f7400039560 0x7f740003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:34.297 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.490+0000 7f74127fc700 1 --2- 172.21.15.52:0/1053475691 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f7400039560 0x7f740003ba20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f73fc006fd0 tx=0x7f73fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:34.297 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.495+0000 7f7418817700 1 -- 172.21.15.52:0/1053475691 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f740c02a020 con 0x7f74141080d0 2024-04-02T13:43:34.297 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.706+0000 7f7419819700 1 -- 172.21.15.52:0/1053475691 --> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f74141048b0 con 0x7f7400039560 2024-04-02T13:43:34.297 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.707+0000 7f7418817700 1 -- 172.21.15.52:0/1053475691 <== mgr.14120 v2:172.21.15.52:6800/673843171 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f74141048b0 con 0x7f7400039560 2024-04-02T13:43:34.297 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.709+0000 7f7419819700 1 -- 172.21.15.52:0/1053475691 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f7400039560 msgr2=0x7f740003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:34.298 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.709+0000 7f7419819700 1 --2- 172.21.15.52:0/1053475691 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f7400039560 0x7f740003ba20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f73fc006fd0 tx=0x7f73fc006e40 comp rx=0 tx=0).stop 2024-04-02T13:43:34.298 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.709+0000 7f7419819700 1 -- 172.21.15.52:0/1053475691 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74141080d0 msgr2=0x7f741419bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:34.298 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.709+0000 7f7419819700 1 --2- 172.21.15.52:0/1053475691 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74141080d0 0x7f741419bfe0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f740c0072a0 tx=0x7f740c00bd50 comp rx=0 tx=0).stop 2024-04-02T13:43:34.298 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.709+0000 7f7419819700 1 -- 172.21.15.52:0/1053475691 shutdown_connections 2024-04-02T13:43:34.298 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.709+0000 7f7419819700 1 --2- 172.21.15.52:0/1053475691 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f7400039560 0x7f740003ba20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:34.298 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.709+0000 7f7419819700 1 --2- 172.21.15.52:0/1053475691 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74141080d0 0x7f741419bfe0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:34.298 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.709+0000 7f7419819700 1 -- 172.21.15.52:0/1053475691 >> 172.21.15.52:0/1053475691 conn(0x7f74141036b0 msgr2=0x7f74141041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:34.298 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.709+0000 7f7419819700 1 -- 172.21.15.52:0/1053475691 shutdown_connections 2024-04-02T13:43:34.298 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:33.709+0000 7f7419819700 1 -- 172.21.15.52:0/1053475691 wait complete. 2024-04-02T13:43:34.298 INFO:teuthology.orchestra.run.smithi052.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-02T13:43:34.298 INFO:teuthology.orchestra.run.smithi052.stdout:Adding key to root@localhost authorized_keys... 2024-04-02T13:43:34.298 INFO:teuthology.orchestra.run.smithi052.stdout:Adding host smithi052... 2024-04-02T13:43:34.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:34 smithi052 ceph-mon[32104]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:43:35.534 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:35 smithi052 ceph-mon[32104]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi052", "addr": "172.21.15.52", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:43:36.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:36 smithi052 ceph-mon[32104]: Deploying cephadm binary to smithi052 2024-04-02T13:43:39.942 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout Added host 'smithi052' with addr '172.21.15.52' 2024-04-02T13:43:39.942 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.712+0000 7fd0a452f700 1 Processor -- start 2024-04-02T13:43:39.942 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.712+0000 7fd0a452f700 1 -- start start 2024-04-02T13:43:39.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.712+0000 7fd0a452f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd09c1080d0 0x7fd09c1084f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:39.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.712+0000 7fd0a452f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c108ac0 con 0x7fd09c1080d0 2024-04-02T13:43:39.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.712+0000 7fd0a22cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd09c1080d0 0x7fd09c1084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:39.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.712+0000 7fd0a22cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd09c1080d0 0x7fd09c1084f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37720/0 (socket says 172.21.15.52:37720) 2024-04-02T13:43:39.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.712+0000 7fd0a22cb700 1 -- 172.21.15.52:0/1723517120 learned_addr learned my addr 172.21.15.52:0/1723517120 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:39.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.713+0000 7fd0a22cb700 1 -- 172.21.15.52:0/1723517120 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09c109280 con 0x7fd09c1080d0 2024-04-02T13:43:39.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.713+0000 7fd0a22cb700 1 --2- 172.21.15.52:0/1723517120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd09c1080d0 0x7fd09c1084f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd0980098a0 tx=0x7fd098009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a456f2397f1826e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:39.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.713+0000 7fd0a12c9700 1 -- 172.21.15.52:0/1723517120 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd098004160 con 0x7fd09c1080d0 2024-04-02T13:43:39.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.713+0000 7fd0a12c9700 1 -- 172.21.15.52:0/1723517120 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd098004300 con 0x7fd09c1080d0 2024-04-02T13:43:39.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.713+0000 7fd0a12c9700 1 -- 172.21.15.52:0/1723517120 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd098011b50 con 0x7fd09c1080d0 2024-04-02T13:43:39.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.713+0000 7fd0a452f700 1 -- 172.21.15.52:0/1723517120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd09c1080d0 msgr2=0x7fd09c1084f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:39.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.713+0000 7fd0a452f700 1 --2- 172.21.15.52:0/1723517120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd09c1080d0 0x7fd09c1084f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd0980098a0 tx=0x7fd098009bb0 comp rx=0 tx=0).stop 2024-04-02T13:43:39.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.714+0000 7fd0a452f700 1 -- 172.21.15.52:0/1723517120 shutdown_connections 2024-04-02T13:43:39.944 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.714+0000 7fd0a452f700 1 --2- 172.21.15.52:0/1723517120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd09c1080d0 0x7fd09c1084f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:39.944 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.714+0000 7fd0a452f700 1 -- 172.21.15.52:0/1723517120 >> 172.21.15.52:0/1723517120 conn(0x7fd09c1036b0 msgr2=0x7fd09c105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:39.944 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.714+0000 7fd0a452f700 1 -- 172.21.15.52:0/1723517120 shutdown_connections 2024-04-02T13:43:39.944 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.714+0000 7fd0a452f700 1 -- 172.21.15.52:0/1723517120 wait complete. 2024-04-02T13:43:39.944 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.714+0000 7fd0a452f700 1 Processor -- start 2024-04-02T13:43:39.944 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.714+0000 7fd0a452f700 1 -- start start 2024-04-02T13:43:39.944 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.714+0000 7fd0a452f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd09c1080d0 0x7fd09c19c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:39.944 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.714+0000 7fd0a452f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c10c630 con 0x7fd09c1080d0 2024-04-02T13:43:39.944 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.715+0000 7fd0a22cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd09c1080d0 0x7fd09c19c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:39.944 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.715+0000 7fd0a22cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd09c1080d0 0x7fd09c19c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37732/0 (socket says 172.21.15.52:37732) 2024-04-02T13:43:39.944 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.715+0000 7fd0a22cb700 1 -- 172.21.15.52:0/2647039495 learned_addr learned my addr 172.21.15.52:0/2647039495 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:39.944 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.715+0000 7fd0a22cb700 1 -- 172.21.15.52:0/2647039495 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09c19c5b0 con 0x7fd09c1080d0 2024-04-02T13:43:39.944 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.715+0000 7fd0a22cb700 1 --2- 172.21.15.52:0/2647039495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd09c1080d0 0x7fd09c19c070 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd098011670 tx=0x7fd098011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:39.944 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.715+0000 7fd0937fe700 1 -- 172.21.15.52:0/2647039495 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0980039c0 con 0x7fd09c1080d0 2024-04-02T13:43:39.945 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.715+0000 7fd0937fe700 1 -- 172.21.15.52:0/2647039495 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd098011e50 con 0x7fd09c1080d0 2024-04-02T13:43:39.945 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.715+0000 7fd0a452f700 1 -- 172.21.15.52:0/2647039495 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd09c19c840 con 0x7fd09c1080d0 2024-04-02T13:43:39.945 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.715+0000 7fd0937fe700 1 -- 172.21.15.52:0/2647039495 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd09801a5c0 con 0x7fd09c1080d0 2024-04-02T13:43:39.945 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.715+0000 7fd0a452f700 1 -- 172.21.15.52:0/2647039495 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd09c19f530 con 0x7fd09c1080d0 2024-04-02T13:43:39.945 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.716+0000 7fd0937fe700 1 -- 172.21.15.52:0/2647039495 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fd09801a870 con 0x7fd09c1080d0 2024-04-02T13:43:39.945 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.716+0000 7fd0a452f700 1 -- 172.21.15.52:0/2647039495 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd09c063c20 con 0x7fd09c1080d0 2024-04-02T13:43:39.945 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.716+0000 7fd0937fe700 1 --2- 172.21.15.52:0/2647039495 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7fd0880395b0 0x7fd08803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:39.945 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.716+0000 7fd0937fe700 1 -- 172.21.15.52:0/2647039495 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd09804dc20 con 0x7fd09c1080d0 2024-04-02T13:43:39.945 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.716+0000 7fd0a1aca700 1 --2- 172.21.15.52:0/2647039495 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7fd0880395b0 0x7fd08803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:39.945 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.717+0000 7fd0a1aca700 1 --2- 172.21.15.52:0/2647039495 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7fd0880395b0 0x7fd08803ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd08c006fd0 tx=0x7fd08c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:39.945 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.721+0000 7fd0937fe700 1 -- 172.21.15.52:0/2647039495 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd09800be50 con 0x7fd09c1080d0 2024-04-02T13:43:39.945 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:34.935+0000 7fd0a452f700 1 -- 172.21.15.52:0/2647039495 --> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi052", "addr": "172.21.15.52", "target": ["mon-mgr", ""]}) -- 0x7fd09c104920 con 0x7fd0880395b0 2024-04-02T13:43:39.945 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:39.342+0000 7fd0937fe700 1 -- 172.21.15.52:0/2647039495 <== mgr.14120 v2:172.21.15.52:6800/673843171 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fd09c104920 con 0x7fd0880395b0 2024-04-02T13:43:39.946 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:39.345+0000 7fd0a452f700 1 -- 172.21.15.52:0/2647039495 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7fd0880395b0 msgr2=0x7fd08803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:39.946 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:39.345+0000 7fd0a452f700 1 --2- 172.21.15.52:0/2647039495 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7fd0880395b0 0x7fd08803ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd08c006fd0 tx=0x7fd08c006e40 comp rx=0 tx=0).stop 2024-04-02T13:43:39.946 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:39.345+0000 7fd0a452f700 1 -- 172.21.15.52:0/2647039495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd09c1080d0 msgr2=0x7fd09c19c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:39.946 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:39.345+0000 7fd0a452f700 1 --2- 172.21.15.52:0/2647039495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd09c1080d0 0x7fd09c19c070 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd098011670 tx=0x7fd098011b60 comp rx=0 tx=0).stop 2024-04-02T13:43:39.946 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:39.345+0000 7fd0a452f700 1 -- 172.21.15.52:0/2647039495 shutdown_connections 2024-04-02T13:43:39.946 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:39.345+0000 7fd0a452f700 1 --2- 172.21.15.52:0/2647039495 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7fd0880395b0 0x7fd08803ba70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:39.946 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:39.345+0000 7fd0a452f700 1 --2- 172.21.15.52:0/2647039495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd09c1080d0 0x7fd09c19c070 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:39.946 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:39.345+0000 7fd0a452f700 1 -- 172.21.15.52:0/2647039495 >> 172.21.15.52:0/2647039495 conn(0x7fd09c1036b0 msgr2=0x7fd09c104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:39.946 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:39.345+0000 7fd0a452f700 1 -- 172.21.15.52:0/2647039495 shutdown_connections 2024-04-02T13:43:39.946 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:39.345+0000 7fd0a452f700 1 -- 172.21.15.52:0/2647039495 wait complete. 2024-04-02T13:43:39.946 INFO:teuthology.orchestra.run.smithi052.stdout:Deploying unmanaged mon service... 2024-04-02T13:43:40.509 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:40 smithi052 ceph-mon[32104]: from='mgr.14120 172.21.15.52:0/3185562986' entity='mgr.x' 2024-04-02T13:43:40.509 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:40 smithi052 ceph-mon[32104]: Added host smithi052 2024-04-02T13:43:40.509 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:40 smithi052 ceph-mon[32104]: from='mgr.14120 172.21.15.52:0/3185562986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:43:41.957 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-02T13:43:41.957 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.506+0000 7f8827bbf700 1 Processor -- start 2024-04-02T13:43:41.957 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.506+0000 7f8827bbf700 1 -- start start 2024-04-02T13:43:41.957 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.506+0000 7f8827bbf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f882007e690 0x7f882007cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:41.957 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.506+0000 7f8827bbf700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f882007d080 con 0x7f882007e690 2024-04-02T13:43:41.957 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.506+0000 7f882595b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f882007e690 0x7f882007cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:41.957 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.506+0000 7f882595b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f882007e690 0x7f882007cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37736/0 (socket says 172.21.15.52:37736) 2024-04-02T13:43:41.958 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.506+0000 7f882595b700 1 -- 172.21.15.52:0/4062980127 learned_addr learned my addr 172.21.15.52:0/4062980127 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:41.958 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.507+0000 7f882595b700 1 -- 172.21.15.52:0/4062980127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f882007d200 con 0x7f882007e690 2024-04-02T13:43:41.958 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.507+0000 7f882595b700 1 --2- 172.21.15.52:0/4062980127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f882007e690 0x7f882007cb40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f881c0098a0 tx=0x7f881c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c223d5dc74d4618 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:41.958 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.507+0000 7f8824959700 1 -- 172.21.15.52:0/4062980127 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f881c004160 con 0x7f882007e690 2024-04-02T13:43:41.958 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.507+0000 7f8824959700 1 -- 172.21.15.52:0/4062980127 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f881c004300 con 0x7f882007e690 2024-04-02T13:43:41.958 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.507+0000 7f8824959700 1 -- 172.21.15.52:0/4062980127 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f881c011b50 con 0x7f882007e690 2024-04-02T13:43:41.958 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.508+0000 7f8827bbf700 1 -- 172.21.15.52:0/4062980127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f882007e690 msgr2=0x7f882007cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:41.958 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.508+0000 7f8827bbf700 1 --2- 172.21.15.52:0/4062980127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f882007e690 0x7f882007cb40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f881c0098a0 tx=0x7f881c009bb0 comp rx=0 tx=0).stop 2024-04-02T13:43:41.958 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.508+0000 7f8827bbf700 1 -- 172.21.15.52:0/4062980127 shutdown_connections 2024-04-02T13:43:41.958 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.508+0000 7f8827bbf700 1 --2- 172.21.15.52:0/4062980127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f882007e690 0x7f882007cb40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:41.958 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.508+0000 7f8827bbf700 1 -- 172.21.15.52:0/4062980127 >> 172.21.15.52:0/4062980127 conn(0x7f88201036b0 msgr2=0x7f8820105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:41.959 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.508+0000 7f8827bbf700 1 -- 172.21.15.52:0/4062980127 shutdown_connections 2024-04-02T13:43:41.959 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.508+0000 7f8827bbf700 1 -- 172.21.15.52:0/4062980127 wait complete. 2024-04-02T13:43:41.959 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.508+0000 7f8827bbf700 1 Processor -- start 2024-04-02T13:43:41.959 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.508+0000 7f8827bbf700 1 -- start start 2024-04-02T13:43:41.959 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.508+0000 7f8827bbf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f882007e690 0x7f882019e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:41.959 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.508+0000 7f8827bbf700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f882010e6a0 con 0x7f882007e690 2024-04-02T13:43:41.959 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.509+0000 7f882595b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f882007e690 0x7f882019e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:41.959 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.509+0000 7f882595b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f882007e690 0x7f882019e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37746/0 (socket says 172.21.15.52:37746) 2024-04-02T13:43:41.959 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.509+0000 7f882595b700 1 -- 172.21.15.52:0/695855011 learned_addr learned my addr 172.21.15.52:0/695855011 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:41.959 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.509+0000 7f882595b700 1 -- 172.21.15.52:0/695855011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f882019e790 con 0x7f882007e690 2024-04-02T13:43:41.959 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.509+0000 7f882595b700 1 --2- 172.21.15.52:0/695855011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f882007e690 0x7f882019e250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f881c011670 tx=0x7f881c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:41.959 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.509+0000 7f8816ffd700 1 -- 172.21.15.52:0/695855011 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f881c0039c0 con 0x7f882007e690 2024-04-02T13:43:41.959 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.509+0000 7f8816ffd700 1 -- 172.21.15.52:0/695855011 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f881c011e50 con 0x7f882007e690 2024-04-02T13:43:41.960 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.509+0000 7f8816ffd700 1 -- 172.21.15.52:0/695855011 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f881c01a5c0 con 0x7f882007e690 2024-04-02T13:43:41.960 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.509+0000 7f8827bbf700 1 -- 172.21.15.52:0/695855011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f882019ea20 con 0x7f882007e690 2024-04-02T13:43:41.960 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.509+0000 7f8827bbf700 1 -- 172.21.15.52:0/695855011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88201a1710 con 0x7f882007e690 2024-04-02T13:43:41.960 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.510+0000 7f8816ffd700 1 -- 172.21.15.52:0/695855011 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f881c024430 con 0x7f882007e690 2024-04-02T13:43:41.960 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.510+0000 7f8827bbf700 1 -- 172.21.15.52:0/695855011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8820063c20 con 0x7f882007e690 2024-04-02T13:43:41.960 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.510+0000 7f8816ffd700 1 --2- 172.21.15.52:0/695855011 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f880c0395b0 0x7f880c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:41.960 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.510+0000 7f8816ffd700 1 -- 172.21.15.52:0/695855011 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f881c04db50 con 0x7f882007e690 2024-04-02T13:43:41.960 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.510+0000 7f882515a700 1 --2- 172.21.15.52:0/695855011 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f880c0395b0 0x7f880c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:41.960 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.511+0000 7f882515a700 1 --2- 172.21.15.52:0/695855011 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f880c0395b0 0x7f880c03ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8810006fd0 tx=0x7f8810006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:41.960 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.515+0000 7f8816ffd700 1 -- 172.21.15.52:0/695855011 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f881c02d910 con 0x7f882007e690 2024-04-02T13:43:41.960 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.746+0000 7f8827bbf700 1 -- 172.21.15.52:0/695855011 --> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f8820104920 con 0x7f880c0395b0 2024-04-02T13:43:41.960 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.751+0000 7f8816ffd700 1 -- 172.21.15.52:0/695855011 <== mgr.14120 v2:172.21.15.52:6800/673843171 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8820104920 con 0x7f880c0395b0 2024-04-02T13:43:41.960 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.753+0000 7f8827bbf700 1 -- 172.21.15.52:0/695855011 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f880c0395b0 msgr2=0x7f880c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:41.961 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.753+0000 7f8827bbf700 1 --2- 172.21.15.52:0/695855011 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f880c0395b0 0x7f880c03ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8810006fd0 tx=0x7f8810006e40 comp rx=0 tx=0).stop 2024-04-02T13:43:41.961 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.753+0000 7f8827bbf700 1 -- 172.21.15.52:0/695855011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f882007e690 msgr2=0x7f882019e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:41.961 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.753+0000 7f8827bbf700 1 --2- 172.21.15.52:0/695855011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f882007e690 0x7f882019e250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f881c011670 tx=0x7f881c011b60 comp rx=0 tx=0).stop 2024-04-02T13:43:41.961 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.753+0000 7f8827bbf700 1 -- 172.21.15.52:0/695855011 shutdown_connections 2024-04-02T13:43:41.961 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.753+0000 7f8827bbf700 1 --2- 172.21.15.52:0/695855011 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f880c0395b0 0x7f880c03ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:41.961 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.753+0000 7f8827bbf700 1 --2- 172.21.15.52:0/695855011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f882007e690 0x7f882019e250 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:41.961 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.753+0000 7f8827bbf700 1 -- 172.21.15.52:0/695855011 >> 172.21.15.52:0/695855011 conn(0x7f88201036b0 msgr2=0x7f8820104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:41.961 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.753+0000 7f8827bbf700 1 -- 172.21.15.52:0/695855011 shutdown_connections 2024-04-02T13:43:41.961 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:40.753+0000 7f8827bbf700 1 -- 172.21.15.52:0/695855011 wait complete. 2024-04-02T13:43:41.961 INFO:teuthology.orchestra.run.smithi052.stdout:Deploying unmanaged mgr service... 2024-04-02T13:43:42.080 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:41 smithi052 ceph-mon[32104]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:43:42.081 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:41 smithi052 ceph-mon[32104]: Saving service mon spec with placement count:5 2024-04-02T13:43:42.081 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:41 smithi052 ceph-mon[32104]: from='mgr.14120 172.21.15.52:0/3185562986' entity='mgr.x' 2024-04-02T13:43:43.465 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-02T13:43:43.466 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.393+0000 7f6bfc919700 1 Processor -- start 2024-04-02T13:43:43.466 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.393+0000 7f6bfc919700 1 -- start start 2024-04-02T13:43:43.466 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.393+0000 7f6bfc919700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bf81080d0 0x7f6bf81084f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:43.466 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.393+0000 7f6bfc919700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf8108ac0 con 0x7f6bf81080d0 2024-04-02T13:43:43.466 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.393+0000 7f6bf659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bf81080d0 0x7f6bf81084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:43.466 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.393+0000 7f6bf659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bf81080d0 0x7f6bf81084f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42444/0 (socket says 172.21.15.52:42444) 2024-04-02T13:43:43.466 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.393+0000 7f6bf659c700 1 -- 172.21.15.52:0/473569089 learned_addr learned my addr 172.21.15.52:0/473569089 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:43.466 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.394+0000 7f6bf659c700 1 -- 172.21.15.52:0/473569089 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf8109280 con 0x7f6bf81080d0 2024-04-02T13:43:43.466 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.394+0000 7f6bf659c700 1 --2- 172.21.15.52:0/473569089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bf81080d0 0x7f6bf81084f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6be80098a0 tx=0x7f6be8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2409e5d17b774b53 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:43.466 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.394+0000 7f6bf5d9b700 1 -- 172.21.15.52:0/473569089 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6be8004160 con 0x7f6bf81080d0 2024-04-02T13:43:43.466 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.394+0000 7f6bf5d9b700 1 -- 172.21.15.52:0/473569089 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6be8004300 con 0x7f6bf81080d0 2024-04-02T13:43:43.466 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.394+0000 7f6bf5d9b700 1 -- 172.21.15.52:0/473569089 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6be8011b50 con 0x7f6bf81080d0 2024-04-02T13:43:43.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.395+0000 7f6bfc919700 1 -- 172.21.15.52:0/473569089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bf81080d0 msgr2=0x7f6bf81084f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:43.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.395+0000 7f6bfc919700 1 --2- 172.21.15.52:0/473569089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bf81080d0 0x7f6bf81084f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6be80098a0 tx=0x7f6be8009bb0 comp rx=0 tx=0).stop 2024-04-02T13:43:43.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.395+0000 7f6bfc919700 1 -- 172.21.15.52:0/473569089 shutdown_connections 2024-04-02T13:43:43.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.395+0000 7f6bfc919700 1 --2- 172.21.15.52:0/473569089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bf81080d0 0x7f6bf81084f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:43.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.395+0000 7f6bfc919700 1 -- 172.21.15.52:0/473569089 >> 172.21.15.52:0/473569089 conn(0x7f6bf81036b0 msgr2=0x7f6bf8105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:43.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.395+0000 7f6bfc919700 1 -- 172.21.15.52:0/473569089 shutdown_connections 2024-04-02T13:43:43.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.395+0000 7f6bfc919700 1 -- 172.21.15.52:0/473569089 wait complete. 2024-04-02T13:43:43.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.396+0000 7f6bfc919700 1 Processor -- start 2024-04-02T13:43:43.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.396+0000 7f6bfc919700 1 -- start start 2024-04-02T13:43:43.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.396+0000 7f6bfc919700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bf81080d0 0x7f6bf819c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:43.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.396+0000 7f6bfc919700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf810c630 con 0x7f6bf81080d0 2024-04-02T13:43:43.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.396+0000 7f6bf659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bf81080d0 0x7f6bf819c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:43.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.396+0000 7f6bf659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bf81080d0 0x7f6bf819c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42452/0 (socket says 172.21.15.52:42452) 2024-04-02T13:43:43.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.396+0000 7f6bf659c700 1 -- 172.21.15.52:0/686868445 learned_addr learned my addr 172.21.15.52:0/686868445 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:43.468 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.396+0000 7f6bf659c700 1 -- 172.21.15.52:0/686868445 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf819c5b0 con 0x7f6bf81080d0 2024-04-02T13:43:43.468 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.396+0000 7f6bf659c700 1 --2- 172.21.15.52:0/686868445 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bf81080d0 0x7f6bf819c070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6be8011670 tx=0x7f6be8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:43.468 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.396+0000 7f6bef7fe700 1 -- 172.21.15.52:0/686868445 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6be80039c0 con 0x7f6bf81080d0 2024-04-02T13:43:43.468 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.396+0000 7f6bef7fe700 1 -- 172.21.15.52:0/686868445 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6be8011e50 con 0x7f6bf81080d0 2024-04-02T13:43:43.468 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.396+0000 7f6bfc919700 1 -- 172.21.15.52:0/686868445 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bf819c840 con 0x7f6bf81080d0 2024-04-02T13:43:43.468 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.397+0000 7f6bef7fe700 1 -- 172.21.15.52:0/686868445 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6be801a5c0 con 0x7f6bf81080d0 2024-04-02T13:43:43.468 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.397+0000 7f6bfc919700 1 -- 172.21.15.52:0/686868445 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bf819f530 con 0x7f6bf81080d0 2024-04-02T13:43:43.468 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.397+0000 7f6bef7fe700 1 -- 172.21.15.52:0/686868445 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6be801a8c0 con 0x7f6bf81080d0 2024-04-02T13:43:43.468 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.397+0000 7f6bfc919700 1 -- 172.21.15.52:0/686868445 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bf8063c20 con 0x7f6bf81080d0 2024-04-02T13:43:43.468 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.398+0000 7f6bef7fe700 1 --2- 172.21.15.52:0/686868445 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f6be4039560 0x7f6be403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:43.468 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.398+0000 7f6bef7fe700 1 -- 172.21.15.52:0/686868445 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6be804dc40 con 0x7f6bf81080d0 2024-04-02T13:43:43.468 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.398+0000 7f6beffff700 1 --2- 172.21.15.52:0/686868445 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f6be4039560 0x7f6be403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:43.468 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.398+0000 7f6beffff700 1 --2- 172.21.15.52:0/686868445 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f6be4039560 0x7f6be403ba20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6be0006fd0 tx=0x7f6be0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:43.469 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.403+0000 7f6bef7fe700 1 -- 172.21.15.52:0/686868445 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6be8019c70 con 0x7f6bf81080d0 2024-04-02T13:43:43.469 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.629+0000 7f6bfc919700 1 -- 172.21.15.52:0/686868445 --> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f6bf8104920 con 0x7f6be4039560 2024-04-02T13:43:43.469 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.633+0000 7f6bef7fe700 1 -- 172.21.15.52:0/686868445 <== mgr.14120 v2:172.21.15.52:6800/673843171 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6bf8104920 con 0x7f6be4039560 2024-04-02T13:43:43.469 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.635+0000 7f6bfc919700 1 -- 172.21.15.52:0/686868445 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f6be4039560 msgr2=0x7f6be403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:43.469 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.635+0000 7f6bfc919700 1 --2- 172.21.15.52:0/686868445 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f6be4039560 0x7f6be403ba20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6be0006fd0 tx=0x7f6be0006e40 comp rx=0 tx=0).stop 2024-04-02T13:43:43.469 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.635+0000 7f6bfc919700 1 -- 172.21.15.52:0/686868445 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bf81080d0 msgr2=0x7f6bf819c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:43.469 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.635+0000 7f6bfc919700 1 --2- 172.21.15.52:0/686868445 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bf81080d0 0x7f6bf819c070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6be8011670 tx=0x7f6be8011b60 comp rx=0 tx=0).stop 2024-04-02T13:43:43.469 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.635+0000 7f6bfc919700 1 -- 172.21.15.52:0/686868445 shutdown_connections 2024-04-02T13:43:43.469 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.635+0000 7f6bfc919700 1 --2- 172.21.15.52:0/686868445 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f6be4039560 0x7f6be403ba20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:43.469 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.635+0000 7f6bfc919700 1 --2- 172.21.15.52:0/686868445 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bf81080d0 0x7f6bf819c070 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:43.469 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.635+0000 7f6bfc919700 1 -- 172.21.15.52:0/686868445 >> 172.21.15.52:0/686868445 conn(0x7f6bf81036b0 msgr2=0x7f6bf8104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:43.469 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.635+0000 7f6bfc919700 1 -- 172.21.15.52:0/686868445 shutdown_connections 2024-04-02T13:43:43.469 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:42.636+0000 7f6bfc919700 1 -- 172.21.15.52:0/686868445 wait complete. 2024-04-02T13:43:43.714 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:43 smithi052 ceph-mon[32104]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:43:43.714 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:43 smithi052 ceph-mon[32104]: Saving service mgr spec with placement count:2 2024-04-02T13:43:43.714 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:43 smithi052 ceph-mon[32104]: from='mgr.14120 172.21.15.52:0/3185562986' entity='mgr.x' 2024-04-02T13:43:43.714 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:43 smithi052 ceph-mon[32104]: from='mgr.14120 172.21.15.52:0/3185562986' entity='mgr.x' 2024-04-02T13:43:43.714 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:43 smithi052 ceph-mon[32104]: from='mgr.14120 172.21.15.52:0/3185562986' entity='mgr.x' 2024-04-02T13:43:44.713 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.896+0000 7f771b105700 1 Processor -- start 2024-04-02T13:43:44.713 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.896+0000 7f771b105700 1 -- start start 2024-04-02T13:43:44.714 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.896+0000 7f771b105700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7714108b10 0x7f771410af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:44.714 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.896+0000 7f771b105700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f771407c6c0 con 0x7f7714108b10 2024-04-02T13:43:44.714 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.897+0000 7f7718ea1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7714108b10 0x7f771410af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:44.714 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.897+0000 7f7718ea1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7714108b10 0x7f771410af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42466/0 (socket says 172.21.15.52:42466) 2024-04-02T13:43:44.714 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.897+0000 7f7718ea1700 1 -- 172.21.15.52:0/582495798 learned_addr learned my addr 172.21.15.52:0/582495798 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:44.714 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.897+0000 7f7718ea1700 1 -- 172.21.15.52:0/582495798 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771410b440 con 0x7f7714108b10 2024-04-02T13:43:44.714 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.897+0000 7f7718ea1700 1 --2- 172.21.15.52:0/582495798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7714108b10 0x7f771410af00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f77080098a0 tx=0x7f7708009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248e0deac22fae14 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:44.714 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.897+0000 7f77137fe700 1 -- 172.21.15.52:0/582495798 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7708004160 con 0x7f7714108b10 2024-04-02T13:43:44.714 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.897+0000 7f77137fe700 1 -- 172.21.15.52:0/582495798 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7708004300 con 0x7f7714108b10 2024-04-02T13:43:44.714 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.897+0000 7f77137fe700 1 -- 172.21.15.52:0/582495798 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7708011b20 con 0x7f7714108b10 2024-04-02T13:43:44.714 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.898+0000 7f771b105700 1 -- 172.21.15.52:0/582495798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7714108b10 msgr2=0x7f771410af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:44.714 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.898+0000 7f771b105700 1 --2- 172.21.15.52:0/582495798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7714108b10 0x7f771410af00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f77080098a0 tx=0x7f7708009bb0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.714 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.898+0000 7f771b105700 1 -- 172.21.15.52:0/582495798 shutdown_connections 2024-04-02T13:43:44.715 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.898+0000 7f771b105700 1 --2- 172.21.15.52:0/582495798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7714108b10 0x7f771410af00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.715 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.898+0000 7f771b105700 1 -- 172.21.15.52:0/582495798 >> 172.21.15.52:0/582495798 conn(0x7f7714102560 msgr2=0x7f7714104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:44.715 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.898+0000 7f771b105700 1 -- 172.21.15.52:0/582495798 shutdown_connections 2024-04-02T13:43:44.715 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.898+0000 7f771b105700 1 -- 172.21.15.52:0/582495798 wait complete. 2024-04-02T13:43:44.715 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.899+0000 7f771b105700 1 Processor -- start 2024-04-02T13:43:44.715 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.899+0000 7f771b105700 1 -- start start 2024-04-02T13:43:44.715 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.899+0000 7f771b105700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7714108b10 0x7f771419bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:44.715 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.899+0000 7f771b105700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f771410c4a0 con 0x7f7714108b10 2024-04-02T13:43:44.715 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.899+0000 7f7718ea1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7714108b10 0x7f771419bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:44.715 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.899+0000 7f7718ea1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7714108b10 0x7f771419bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42480/0 (socket says 172.21.15.52:42480) 2024-04-02T13:43:44.715 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.899+0000 7f7718ea1700 1 -- 172.21.15.52:0/3894213454 learned_addr learned my addr 172.21.15.52:0/3894213454 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:44.715 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.899+0000 7f7718ea1700 1 -- 172.21.15.52:0/3894213454 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771419c4f0 con 0x7f7714108b10 2024-04-02T13:43:44.715 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.899+0000 7f7718ea1700 1 --2- 172.21.15.52:0/3894213454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7714108b10 0x7f771419bfb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f77080117e0 tx=0x7f7708011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:44.715 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.899+0000 7f7711ffb700 1 -- 172.21.15.52:0/3894213454 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7708003bf0 con 0x7f7714108b10 2024-04-02T13:43:44.716 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.900+0000 7f771b105700 1 -- 172.21.15.52:0/3894213454 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f771419c780 con 0x7f7714108b10 2024-04-02T13:43:44.716 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.900+0000 7f7711ffb700 1 -- 172.21.15.52:0/3894213454 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f770801a910 con 0x7f7714108b10 2024-04-02T13:43:44.716 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.900+0000 7f7711ffb700 1 -- 172.21.15.52:0/3894213454 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7708024970 con 0x7f7714108b10 2024-04-02T13:43:44.716 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.900+0000 7f771b105700 1 -- 172.21.15.52:0/3894213454 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f771419cbe0 con 0x7f7714108b10 2024-04-02T13:43:44.716 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.901+0000 7f7711ffb700 1 -- 172.21.15.52:0/3894213454 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f7708022020 con 0x7f7714108b10 2024-04-02T13:43:44.716 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.901+0000 7f771b105700 1 -- 172.21.15.52:0/3894213454 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7714063c20 con 0x7f7714108b10 2024-04-02T13:43:44.716 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.901+0000 7f7711ffb700 1 --2- 172.21.15.52:0/3894213454 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f7704039560 0x7f770403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:44.716 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.901+0000 7f7711ffb700 1 -- 172.21.15.52:0/3894213454 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f770804da40 con 0x7f7714108b10 2024-04-02T13:43:44.716 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.901+0000 7f7713fff700 1 --2- 172.21.15.52:0/3894213454 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f7704039560 0x7f770403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:44.716 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.901+0000 7f7713fff700 1 --2- 172.21.15.52:0/3894213454 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f7704039560 0x7f770403ba20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7700006fd0 tx=0x7f7700006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:44.716 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:43.906+0000 7f7711ffb700 1 -- 172.21.15.52:0/3894213454 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7708011e20 con 0x7f7714108b10 2024-04-02T13:43:44.716 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:44.123+0000 7f771b105700 1 -- 172.21.15.52:0/3894213454 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f771419f530 con 0x7f7714108b10 2024-04-02T13:43:44.717 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:44.126+0000 7f7711ffb700 1 -- 172.21.15.52:0/3894213454 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f770804de20 con 0x7f7714108b10 2024-04-02T13:43:44.717 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:44.128+0000 7f771b105700 1 -- 172.21.15.52:0/3894213454 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f7704039560 msgr2=0x7f770403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:44.717 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:44.129+0000 7f771b105700 1 --2- 172.21.15.52:0/3894213454 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f7704039560 0x7f770403ba20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7700006fd0 tx=0x7f7700006e40 comp rx=0 tx=0).stop 2024-04-02T13:43:44.717 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:44.129+0000 7f771b105700 1 -- 172.21.15.52:0/3894213454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7714108b10 msgr2=0x7f771419bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:44.717 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:44.129+0000 7f771b105700 1 --2- 172.21.15.52:0/3894213454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7714108b10 0x7f771419bfb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f77080117e0 tx=0x7f7708011c10 comp rx=0 tx=0).stop 2024-04-02T13:43:44.717 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:44.129+0000 7f771b105700 1 -- 172.21.15.52:0/3894213454 shutdown_connections 2024-04-02T13:43:44.717 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:44.129+0000 7f771b105700 1 --2- 172.21.15.52:0/3894213454 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f7704039560 0x7f770403ba20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.717 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:44.129+0000 7f771b105700 1 --2- 172.21.15.52:0/3894213454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7714108b10 0x7f771419bfb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:44.717 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:44.129+0000 7f771b105700 1 -- 172.21.15.52:0/3894213454 >> 172.21.15.52:0/3894213454 conn(0x7f7714102560 msgr2=0x7f77141030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:44.717 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:44.129+0000 7f771b105700 1 -- 172.21.15.52:0/3894213454 shutdown_connections 2024-04-02T13:43:44.717 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:44.129+0000 7f771b105700 1 -- 172.21.15.52:0/3894213454 wait complete. 2024-04-02T13:43:45.327 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:45 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3894213454' entity='client.admin' 2024-04-02T13:43:46.013 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.127+0000 7f860fc0a700 1 Processor -- start 2024-04-02T13:43:46.013 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.127+0000 7f860fc0a700 1 -- start start 2024-04-02T13:43:46.013 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.127+0000 7f860fc0a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f860807e690 0x7f860807cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:46.013 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.127+0000 7f860fc0a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f860807d080 con 0x7f860807e690 2024-04-02T13:43:46.014 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.127+0000 7f860d9a6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f860807e690 0x7f860807cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:46.014 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.127+0000 7f860d9a6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f860807e690 0x7f860807cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42492/0 (socket says 172.21.15.52:42492) 2024-04-02T13:43:46.014 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.127+0000 7f860d9a6700 1 -- 172.21.15.52:0/1942588775 learned_addr learned my addr 172.21.15.52:0/1942588775 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:46.014 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.127+0000 7f860d9a6700 1 -- 172.21.15.52:0/1942588775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860807d200 con 0x7f860807e690 2024-04-02T13:43:46.014 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.128+0000 7f860d9a6700 1 --2- 172.21.15.52:0/1942588775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f860807e690 0x7f860807cb40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8600009e80 tx=0x7f860000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f53ed86e104e9a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:46.014 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.128+0000 7f860c9a4700 1 -- 172.21.15.52:0/1942588775 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8600017070 con 0x7f860807e690 2024-04-02T13:43:46.014 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.128+0000 7f860c9a4700 1 -- 172.21.15.52:0/1942588775 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f860000b8c0 con 0x7f860807e690 2024-04-02T13:43:46.014 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.128+0000 7f860c9a4700 1 -- 172.21.15.52:0/1942588775 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86000045c0 con 0x7f860807e690 2024-04-02T13:43:46.014 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.128+0000 7f860fc0a700 1 -- 172.21.15.52:0/1942588775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f860807e690 msgr2=0x7f860807cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:46.014 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.128+0000 7f860fc0a700 1 --2- 172.21.15.52:0/1942588775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f860807e690 0x7f860807cb40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8600009e80 tx=0x7f860000b330 comp rx=0 tx=0).stop 2024-04-02T13:43:46.014 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.128+0000 7f860fc0a700 1 -- 172.21.15.52:0/1942588775 shutdown_connections 2024-04-02T13:43:46.014 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.128+0000 7f860fc0a700 1 --2- 172.21.15.52:0/1942588775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f860807e690 0x7f860807cb40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.014 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.128+0000 7f860fc0a700 1 -- 172.21.15.52:0/1942588775 >> 172.21.15.52:0/1942588775 conn(0x7f8608103610 msgr2=0x7f8608105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:46.015 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.129+0000 7f860fc0a700 1 -- 172.21.15.52:0/1942588775 shutdown_connections 2024-04-02T13:43:46.015 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.129+0000 7f860fc0a700 1 -- 172.21.15.52:0/1942588775 wait complete. 2024-04-02T13:43:46.015 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.129+0000 7f860fc0a700 1 Processor -- start 2024-04-02T13:43:46.015 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.129+0000 7f860fc0a700 1 -- start start 2024-04-02T13:43:46.015 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.129+0000 7f860fc0a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f860807e690 0x7f860819e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:46.015 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.129+0000 7f860fc0a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f860810e630 con 0x7f860807e690 2024-04-02T13:43:46.015 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.129+0000 7f860d9a6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f860807e690 0x7f860819e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:46.015 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.129+0000 7f860d9a6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f860807e690 0x7f860819e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42508/0 (socket says 172.21.15.52:42508) 2024-04-02T13:43:46.015 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.129+0000 7f860d9a6700 1 -- 172.21.15.52:0/4208024156 learned_addr learned my addr 172.21.15.52:0/4208024156 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:46.015 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.130+0000 7f860d9a6700 1 -- 172.21.15.52:0/4208024156 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860819e720 con 0x7f860807e690 2024-04-02T13:43:46.015 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.130+0000 7f860d9a6700 1 --2- 172.21.15.52:0/4208024156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f860807e690 0x7f860819e1e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8600009fd0 tx=0x7f8600009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:46.015 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.130+0000 7f85feffd700 1 -- 172.21.15.52:0/4208024156 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8600020020 con 0x7f860807e690 2024-04-02T13:43:46.015 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.130+0000 7f85feffd700 1 -- 172.21.15.52:0/4208024156 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86000045e0 con 0x7f860807e690 2024-04-02T13:43:46.015 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.130+0000 7f860fc0a700 1 -- 172.21.15.52:0/4208024156 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f860819e9b0 con 0x7f860807e690 2024-04-02T13:43:46.016 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.130+0000 7f85feffd700 1 -- 172.21.15.52:0/4208024156 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8600017040 con 0x7f860807e690 2024-04-02T13:43:46.016 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.130+0000 7f860fc0a700 1 -- 172.21.15.52:0/4208024156 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86081a16a0 con 0x7f860807e690 2024-04-02T13:43:46.016 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.131+0000 7f85feffd700 1 -- 172.21.15.52:0/4208024156 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8600023020 con 0x7f860807e690 2024-04-02T13:43:46.016 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.131+0000 7f860fc0a700 1 -- 172.21.15.52:0/4208024156 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8608063c20 con 0x7f860807e690 2024-04-02T13:43:46.016 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.131+0000 7f85feffd700 1 --2- 172.21.15.52:0/4208024156 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f85f40395b0 0x7f85f403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:46.016 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.131+0000 7f85feffd700 1 -- 172.21.15.52:0/4208024156 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f860000f050 con 0x7f860807e690 2024-04-02T13:43:46.016 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.131+0000 7f860d1a5700 1 --2- 172.21.15.52:0/4208024156 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f85f40395b0 0x7f85f403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:46.016 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.132+0000 7f860d1a5700 1 --2- 172.21.15.52:0/4208024156 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f85f40395b0 0x7f85f403ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f85f8006fd0 tx=0x7f85f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:46.016 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.136+0000 7f85feffd700 1 -- 172.21.15.52:0/4208024156 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f860002a020 con 0x7f860807e690 2024-04-02T13:43:46.016 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.345+0000 7f860fc0a700 1 -- 172.21.15.52:0/4208024156 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f86081a1870 con 0x7f860807e690 2024-04-02T13:43:46.016 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.347+0000 7f85feffd700 1 -- 172.21.15.52:0/4208024156 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f86081a1870 con 0x7f860807e690 2024-04-02T13:43:46.016 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.350+0000 7f860fc0a700 1 -- 172.21.15.52:0/4208024156 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f85f40395b0 msgr2=0x7f85f403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:46.016 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.350+0000 7f860fc0a700 1 --2- 172.21.15.52:0/4208024156 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f85f40395b0 0x7f85f403ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f85f8006fd0 tx=0x7f85f8006e40 comp rx=0 tx=0).stop 2024-04-02T13:43:46.016 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.350+0000 7f860fc0a700 1 -- 172.21.15.52:0/4208024156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f860807e690 msgr2=0x7f860819e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:46.017 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.350+0000 7f860fc0a700 1 --2- 172.21.15.52:0/4208024156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f860807e690 0x7f860819e1e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8600009fd0 tx=0x7f8600009390 comp rx=0 tx=0).stop 2024-04-02T13:43:46.017 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.350+0000 7f860fc0a700 1 -- 172.21.15.52:0/4208024156 shutdown_connections 2024-04-02T13:43:46.017 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.350+0000 7f860fc0a700 1 --2- 172.21.15.52:0/4208024156 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f85f40395b0 0x7f85f403ba70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.017 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.350+0000 7f860fc0a700 1 --2- 172.21.15.52:0/4208024156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f860807e690 0x7f860819e1e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:46.017 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.350+0000 7f860fc0a700 1 -- 172.21.15.52:0/4208024156 >> 172.21.15.52:0/4208024156 conn(0x7f8608103610 msgr2=0x7f8608104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:46.017 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.350+0000 7f860fc0a700 1 -- 172.21.15.52:0/4208024156 shutdown_connections 2024-04-02T13:43:46.017 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:45.350+0000 7f860fc0a700 1 -- 172.21.15.52:0/4208024156 wait complete. 2024-04-02T13:43:46.017 INFO:teuthology.orchestra.run.smithi052.stdout:Enabling the dashboard module... 2024-04-02T13:43:46.699 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:46 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/4208024156' entity='client.admin' 2024-04-02T13:43:47.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:47 smithi052 ceph-mon[32104]: from='mgr.14120 172.21.15.52:0/3185562986' entity='mgr.x' 2024-04-02T13:43:47.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:47 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/811983070' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-02T13:43:47.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:47 smithi052 ceph-mon[32104]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:43:47.840 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:47 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: ignoring --setuser ceph since I am not root 2024-04-02T13:43:47.840 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:47 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: ignoring --setgroup ceph since I am not root 2024-04-02T13:43:48.314 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:48 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:48.128+0000 7feb8cc3f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T13:43:48.314 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:48 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:48.224+0000 7feb8cc3f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T13:43:48.322 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.696+0000 7f6993095700 1 Processor -- start 2024-04-02T13:43:48.322 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.696+0000 7f6993095700 1 -- start start 2024-04-02T13:43:48.322 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.696+0000 7f6993095700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f698c108090 0x7f698c1084b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:48.322 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.696+0000 7f6993095700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f698c108a80 con 0x7f698c108090 2024-04-02T13:43:48.322 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.697+0000 7f6990e31700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f698c108090 0x7f698c1084b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:48.322 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.697+0000 7f6990e31700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f698c108090 0x7f698c1084b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42516/0 (socket says 172.21.15.52:42516) 2024-04-02T13:43:48.322 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.697+0000 7f6990e31700 1 -- 172.21.15.52:0/1985982786 learned_addr learned my addr 172.21.15.52:0/1985982786 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:48.322 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.697+0000 7f6990e31700 1 -- 172.21.15.52:0/1985982786 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698c109240 con 0x7f698c108090 2024-04-02T13:43:48.322 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.697+0000 7f6990e31700 1 --2- 172.21.15.52:0/1985982786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f698c108090 0x7f698c1084b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f69800098a0 tx=0x7f6980009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a45ccfeb014cfd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:48.323 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.697+0000 7f698b7fe700 1 -- 172.21.15.52:0/1985982786 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6980004160 con 0x7f698c108090 2024-04-02T13:43:48.323 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.697+0000 7f698b7fe700 1 -- 172.21.15.52:0/1985982786 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6980004300 con 0x7f698c108090 2024-04-02T13:43:48.323 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.697+0000 7f698b7fe700 1 -- 172.21.15.52:0/1985982786 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6980011b20 con 0x7f698c108090 2024-04-02T13:43:48.323 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.698+0000 7f6993095700 1 -- 172.21.15.52:0/1985982786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f698c108090 msgr2=0x7f698c1084b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:48.323 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.698+0000 7f6993095700 1 --2- 172.21.15.52:0/1985982786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f698c108090 0x7f698c1084b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f69800098a0 tx=0x7f6980009bb0 comp rx=0 tx=0).stop 2024-04-02T13:43:48.323 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.698+0000 7f6993095700 1 -- 172.21.15.52:0/1985982786 shutdown_connections 2024-04-02T13:43:48.323 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.698+0000 7f6993095700 1 --2- 172.21.15.52:0/1985982786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f698c108090 0x7f698c1084b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:48.323 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.698+0000 7f6993095700 1 -- 172.21.15.52:0/1985982786 >> 172.21.15.52:0/1985982786 conn(0x7f698c103630 msgr2=0x7f698c105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:48.323 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.698+0000 7f6993095700 1 -- 172.21.15.52:0/1985982786 shutdown_connections 2024-04-02T13:43:48.323 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.698+0000 7f6993095700 1 -- 172.21.15.52:0/1985982786 wait complete. 2024-04-02T13:43:48.323 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.699+0000 7f6993095700 1 Processor -- start 2024-04-02T13:43:48.323 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.699+0000 7f6993095700 1 -- start start 2024-04-02T13:43:48.323 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.699+0000 7f6993095700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f698c108090 0x7f698c19bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:48.324 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.699+0000 7f6993095700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f698c10c5f0 con 0x7f698c108090 2024-04-02T13:43:48.324 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.699+0000 7f6990e31700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f698c108090 0x7f698c19bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:48.324 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.699+0000 7f6990e31700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f698c108090 0x7f698c19bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42524/0 (socket says 172.21.15.52:42524) 2024-04-02T13:43:48.324 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.699+0000 7f6990e31700 1 -- 172.21.15.52:0/811983070 learned_addr learned my addr 172.21.15.52:0/811983070 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:48.324 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.699+0000 7f6990e31700 1 -- 172.21.15.52:0/811983070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698c19c510 con 0x7f698c108090 2024-04-02T13:43:48.324 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.699+0000 7f6990e31700 1 --2- 172.21.15.52:0/811983070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f698c108090 0x7f698c19bfd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f69800117e0 tx=0x7f6980011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:48.324 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.700+0000 7f6989ffb700 1 -- 172.21.15.52:0/811983070 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6980003bf0 con 0x7f698c108090 2024-04-02T13:43:48.324 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.700+0000 7f6993095700 1 -- 172.21.15.52:0/811983070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698c19c7a0 con 0x7f698c108090 2024-04-02T13:43:48.324 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.700+0000 7f6989ffb700 1 -- 172.21.15.52:0/811983070 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f698001a910 con 0x7f698c108090 2024-04-02T13:43:48.324 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.700+0000 7f6989ffb700 1 -- 172.21.15.52:0/811983070 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6980024970 con 0x7f698c108090 2024-04-02T13:43:48.324 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.700+0000 7f6993095700 1 -- 172.21.15.52:0/811983070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f698c19cc00 con 0x7f698c108090 2024-04-02T13:43:48.324 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.701+0000 7f6989ffb700 1 -- 172.21.15.52:0/811983070 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6980022020 con 0x7f698c108090 2024-04-02T13:43:48.324 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.701+0000 7f6993095700 1 -- 172.21.15.52:0/811983070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f698c063c20 con 0x7f698c108090 2024-04-02T13:43:48.324 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.701+0000 7f6989ffb700 1 --2- 172.21.15.52:0/811983070 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f697c0395b0 0x7f697c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:48.325 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.701+0000 7f6989ffb700 1 -- 172.21.15.52:0/811983070 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f698004ded0 con 0x7f698c108090 2024-04-02T13:43:48.325 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.701+0000 7f698bfff700 1 --2- 172.21.15.52:0/811983070 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f697c0395b0 0x7f697c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:48.325 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.701+0000 7f698bfff700 1 --2- 172.21.15.52:0/811983070 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f697c0395b0 0x7f697c03ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6978006fd0 tx=0x7f6978006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:48.325 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.706+0000 7f6989ffb700 1 -- 172.21.15.52:0/811983070 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f698002cdf0 con 0x7f698c108090 2024-04-02T13:43:48.325 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:46.979+0000 7f6993095700 1 -- 172.21.15.52:0/811983070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f698c19f690 con 0x7f698c108090 2024-04-02T13:43:48.325 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:47.541+0000 7f6989ffb700 1 -- 172.21.15.52:0/811983070 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f6980049090 con 0x7f698c108090 2024-04-02T13:43:48.325 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:47.543+0000 7f6989ffb700 1 -- 172.21.15.52:0/811983070 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f698002dc90 con 0x7f698c108090 2024-04-02T13:43:48.325 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:47.544+0000 7f6993095700 1 -- 172.21.15.52:0/811983070 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f697c0395b0 msgr2=0x7f697c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:48.325 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:47.544+0000 7f6993095700 1 --2- 172.21.15.52:0/811983070 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f697c0395b0 0x7f697c03ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6978006fd0 tx=0x7f6978006e40 comp rx=0 tx=0).stop 2024-04-02T13:43:48.325 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:47.544+0000 7f6993095700 1 -- 172.21.15.52:0/811983070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f698c108090 msgr2=0x7f698c19bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:48.325 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:47.544+0000 7f6993095700 1 --2- 172.21.15.52:0/811983070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f698c108090 0x7f698c19bfd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f69800117e0 tx=0x7f6980011ba0 comp rx=0 tx=0).stop 2024-04-02T13:43:48.325 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:47.544+0000 7f6993095700 1 -- 172.21.15.52:0/811983070 shutdown_connections 2024-04-02T13:43:48.325 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:47.544+0000 7f6993095700 1 --2- 172.21.15.52:0/811983070 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f697c0395b0 0x7f697c03ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:48.325 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:47.544+0000 7f6993095700 1 --2- 172.21.15.52:0/811983070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f698c108090 0x7f698c19bfd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:48.325 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:47.544+0000 7f6993095700 1 -- 172.21.15.52:0/811983070 >> 172.21.15.52:0/811983070 conn(0x7f698c103630 msgr2=0x7f698c104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:48.326 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:47.544+0000 7f6993095700 1 -- 172.21.15.52:0/811983070 shutdown_connections 2024-04-02T13:43:48.326 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:47.544+0000 7f6993095700 1 -- 172.21.15.52:0/811983070 wait complete. 2024-04-02T13:43:48.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:48 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/811983070' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-02T13:43:48.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:48 smithi052 ceph-mon[32104]: mgrmap e9: x(active, since 20s) 2024-04-02T13:43:49.157 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:48 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:48.902+0000 7feb8cc3f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T13:43:49.157 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:49 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:48.998+0000 7feb8cc3f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T13:43:49.157 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:49 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:49.153+0000 7feb8cc3f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T13:43:49.451 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:49 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:49.251+0000 7feb8cc3f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T13:43:49.451 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:49 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:49.338+0000 7feb8cc3f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T13:43:49.726 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:49 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/766978272' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-02T13:43:49.726 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:49 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:49.609+0000 7feb8cc3f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T13:43:49.726 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:49 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:49.722+0000 7feb8cc3f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T13:43:49.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-04-02T13:43:49.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-02T13:43:49.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-02T13:43:49.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-02T13:43:49.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-02T13:43:49.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-04-02T13:43:49.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.750+0000 7fd25d355700 1 Processor -- start 2024-04-02T13:43:49.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.750+0000 7fd25d355700 1 -- start start 2024-04-02T13:43:49.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.750+0000 7fd25d355700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd258106f80 0x7fd2581073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:49.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.750+0000 7fd25d355700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd2581078e0 con 0x7fd258106f80 2024-04-02T13:43:49.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.750+0000 7fd256ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd258106f80 0x7fd2581073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:49.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.750+0000 7fd256ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd258106f80 0x7fd2581073a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42532/0 (socket says 172.21.15.52:42532) 2024-04-02T13:43:49.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.751+0000 7fd256ffd700 1 -- 172.21.15.52:0/965807227 learned_addr learned my addr 172.21.15.52:0/965807227 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:49.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.751+0000 7fd256ffd700 1 -- 172.21.15.52:0/965807227 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd258108090 con 0x7fd258106f80 2024-04-02T13:43:49.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.751+0000 7fd256ffd700 1 --2- 172.21.15.52:0/965807227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd258106f80 0x7fd2581073a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0098a0 tx=0x7fd24c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3556eec4e7508df5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:49.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.751+0000 7fd255ffb700 1 -- 172.21.15.52:0/965807227 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd24c004160 con 0x7fd258106f80 2024-04-02T13:43:49.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.751+0000 7fd255ffb700 1 -- 172.21.15.52:0/965807227 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd24c004300 con 0x7fd258106f80 2024-04-02T13:43:49.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.751+0000 7fd255ffb700 1 -- 172.21.15.52:0/965807227 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd24c011b20 con 0x7fd258106f80 2024-04-02T13:43:49.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.752+0000 7fd25d355700 1 -- 172.21.15.52:0/965807227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd258106f80 msgr2=0x7fd2581073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:49.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.752+0000 7fd25d355700 1 --2- 172.21.15.52:0/965807227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd258106f80 0x7fd2581073a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0098a0 tx=0x7fd24c009bb0 comp rx=0 tx=0).stop 2024-04-02T13:43:49.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.752+0000 7fd25d355700 1 -- 172.21.15.52:0/965807227 shutdown_connections 2024-04-02T13:43:49.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.752+0000 7fd25d355700 1 --2- 172.21.15.52:0/965807227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd258106f80 0x7fd2581073a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:49.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.752+0000 7fd25d355700 1 -- 172.21.15.52:0/965807227 >> 172.21.15.52:0/965807227 conn(0x7fd258102540 msgr2=0x7fd258104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:49.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.752+0000 7fd25d355700 1 -- 172.21.15.52:0/965807227 shutdown_connections 2024-04-02T13:43:49.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.752+0000 7fd25d355700 1 -- 172.21.15.52:0/965807227 wait complete. 2024-04-02T13:43:49.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.752+0000 7fd25d355700 1 Processor -- start 2024-04-02T13:43:49.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.753+0000 7fd25d355700 1 -- start start 2024-04-02T13:43:49.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.753+0000 7fd25d355700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd258106f80 0x7fd25819bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:49.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.753+0000 7fd25d355700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd25810b4d0 con 0x7fd258106f80 2024-04-02T13:43:49.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.753+0000 7fd256ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd258106f80 0x7fd25819bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:49.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.753+0000 7fd256ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd258106f80 0x7fd25819bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42540/0 (socket says 172.21.15.52:42540) 2024-04-02T13:43:49.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.753+0000 7fd256ffd700 1 -- 172.21.15.52:0/766978272 learned_addr learned my addr 172.21.15.52:0/766978272 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:49.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.753+0000 7fd256ffd700 1 -- 172.21.15.52:0/766978272 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25819c4a0 con 0x7fd258106f80 2024-04-02T13:43:49.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.753+0000 7fd256ffd700 1 --2- 172.21.15.52:0/766978272 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd258106f80 0x7fd25819bf60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0117e0 tx=0x7fd24c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:49.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.754+0000 7fd23ffff700 1 -- 172.21.15.52:0/766978272 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd24c003bf0 con 0x7fd258106f80 2024-04-02T13:43:49.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.754+0000 7fd25d355700 1 -- 172.21.15.52:0/766978272 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd25819c730 con 0x7fd258106f80 2024-04-02T13:43:49.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.754+0000 7fd23ffff700 1 -- 172.21.15.52:0/766978272 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd24c01a910 con 0x7fd258106f80 2024-04-02T13:43:49.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.754+0000 7fd23ffff700 1 -- 172.21.15.52:0/766978272 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd24c024970 con 0x7fd258106f80 2024-04-02T13:43:49.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.754+0000 7fd25d355700 1 -- 172.21.15.52:0/766978272 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd25819cb90 con 0x7fd258106f80 2024-04-02T13:43:49.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.754+0000 7fd23ffff700 1 -- 172.21.15.52:0/766978272 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fd24c022020 con 0x7fd258106f80 2024-04-02T13:43:49.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.755+0000 7fd23ffff700 1 --2- 172.21.15.52:0/766978272 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7fd240039610 0x7fd24003bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:49.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.755+0000 7fd23ffff700 1 -- 172.21.15.52:0/766978272 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd24c04da50 con 0x7fd258106f80 2024-04-02T13:43:49.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.755+0000 7fd25d355700 1 -- 172.21.15.52:0/766978272 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd258063c20 con 0x7fd258106f80 2024-04-02T13:43:49.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.755+0000 7fd2567fc700 1 -- 172.21.15.52:0/766978272 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7fd240039610 msgr2=0x7fd24003bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/673843171 2024-04-02T13:43:49.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.755+0000 7fd2567fc700 1 --2- 172.21.15.52:0/766978272 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7fd240039610 0x7fd24003bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:43:49.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.760+0000 7fd23ffff700 1 -- 172.21.15.52:0/766978272 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd24c011e20 con 0x7fd258106f80 2024-04-02T13:43:49.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.956+0000 7fd2567fc700 1 -- 172.21.15.52:0/766978272 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7fd240039610 msgr2=0x7fd24003bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/673843171 2024-04-02T13:43:49.741 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:48.956+0000 7fd2567fc700 1 --2- 172.21.15.52:0/766978272 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7fd240039610 0x7fd24003bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T13:43:49.741 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:49.025+0000 7fd25d355700 1 -- 172.21.15.52:0/766978272 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fd258063200 con 0x7fd258106f80 2024-04-02T13:43:49.741 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:49.025+0000 7fd23ffff700 1 -- 172.21.15.52:0/766978272 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fd24c00b9d0 con 0x7fd258106f80 2024-04-02T13:43:49.741 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:49.028+0000 7fd25d355700 1 -- 172.21.15.52:0/766978272 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7fd240039610 msgr2=0x7fd24003bad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:43:49.741 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:49.028+0000 7fd25d355700 1 --2- 172.21.15.52:0/766978272 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7fd240039610 0x7fd24003bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:49.741 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:49.028+0000 7fd25d355700 1 -- 172.21.15.52:0/766978272 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd258106f80 msgr2=0x7fd25819bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:49.741 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:49.028+0000 7fd25d355700 1 --2- 172.21.15.52:0/766978272 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd258106f80 0x7fd25819bf60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0117e0 tx=0x7fd24c011c10 comp rx=0 tx=0).stop 2024-04-02T13:43:49.741 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:49.028+0000 7fd25d355700 1 -- 172.21.15.52:0/766978272 shutdown_connections 2024-04-02T13:43:49.741 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:49.028+0000 7fd25d355700 1 --2- 172.21.15.52:0/766978272 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7fd240039610 0x7fd24003bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:49.741 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:49.028+0000 7fd25d355700 1 --2- 172.21.15.52:0/766978272 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd258106f80 0x7fd25819bf60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:49.741 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:49.028+0000 7fd25d355700 1 -- 172.21.15.52:0/766978272 >> 172.21.15.52:0/766978272 conn(0x7fd258102540 msgr2=0x7fd2581032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:49.741 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:49.028+0000 7fd25d355700 1 -- 172.21.15.52:0/766978272 shutdown_connections 2024-04-02T13:43:49.741 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:49.028+0000 7fd25d355700 1 -- 172.21.15.52:0/766978272 wait complete. 2024-04-02T13:43:49.741 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for the mgr to restart... 2024-04-02T13:43:49.741 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mgr epoch 9... 2024-04-02T13:43:49.980 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:49 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:49.815+0000 7feb8cc3f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T13:43:49.980 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:49 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:49.915+0000 7feb8cc3f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T13:43:50.339 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:50 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:50.098+0000 7feb8cc3f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T13:43:51.589 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:51 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:51.091+0000 7feb8cc3f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T13:43:51.589 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:51 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:51.226+0000 7feb8cc3f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T13:43:51.886 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:51 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:51.587+0000 7feb8cc3f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T13:43:51.886 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:51 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:51.680+0000 7feb8cc3f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T13:43:51.886 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:51 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:51.883+0000 7feb8cc3f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:43:52.339 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:52 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:52.075+0000 7feb8cc3f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T13:43:52.641 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:52 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:52.376+0000 7feb8cc3f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:43:53.089 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:52 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:52.637+0000 7feb8cc3f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T13:43:53.089 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:52 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:52.883+0000 7feb8cc3f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T13:43:54.089 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:53 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:53.730+0000 7feb8cc3f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T13:43:54.585 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:54 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:54.322+0000 7feb8cc3f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T13:43:54.585 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:54 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:54.424+0000 7feb8cc3f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T13:43:54.839 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:43:54 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:43:54.651+0000 7feb8cc3f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T13:43:54.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:54 smithi052 ceph-mon[32104]: Active manager daemon x restarted 2024-04-02T13:43:54.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:54 smithi052 ceph-mon[32104]: Activating manager daemon x 2024-04-02T13:43:54.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:54 smithi052 ceph-mon[32104]: osdmap e3: 0 total, 0 up, 0 in 2024-04-02T13:43:54.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:54 smithi052 ceph-mon[32104]: mgrmap e10: x(active, starting, since 0.00342393s) 2024-04-02T13:43:54.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:54 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:43:54.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:54 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T13:43:54.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:54 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T13:43:54.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:54 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T13:43:54.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:54 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T13:43:54.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:54 smithi052 ceph-mon[32104]: Manager daemon x is now available 2024-04-02T13:43:54.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:54 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:43:54.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:54 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T13:43:56.071 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:55 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T13:43:56.071 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:55 smithi052 ceph-mon[32104]: mgrmap e11: x(active, since 1.00612s) 2024-04-02T13:43:56.674 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-04-02T13:43:56.674 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-02T13:43:56.674 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-02T13:43:56.674 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-04-02T13:43:56.674 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.189+0000 7f9549273700 1 Processor -- start 2024-04-02T13:43:56.674 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.189+0000 7f9549273700 1 -- start start 2024-04-02T13:43:56.675 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.189+0000 7f9549273700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9544108b50 0x7f954410af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:56.675 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.189+0000 7f9549273700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f954407bdc0 con 0x7f9544108b50 2024-04-02T13:43:56.675 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.189+0000 7f9542ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9544108b50 0x7f954410af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:56.675 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.189+0000 7f9542ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9544108b50 0x7f954410af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42548/0 (socket says 172.21.15.52:42548) 2024-04-02T13:43:56.675 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.189+0000 7f9542ffd700 1 -- 172.21.15.52:0/2551081806 learned_addr learned my addr 172.21.15.52:0/2551081806 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:56.675 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.190+0000 7f9542ffd700 1 -- 172.21.15.52:0/2551081806 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954410b480 con 0x7f9544108b50 2024-04-02T13:43:56.675 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.190+0000 7f9542ffd700 1 --2- 172.21.15.52:0/2551081806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9544108b50 0x7f954410af40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f95340098a0 tx=0x7f9534009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c48bf6826fb9570 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:56.675 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.190+0000 7f9541ffb700 1 -- 172.21.15.52:0/2551081806 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9534004160 con 0x7f9544108b50 2024-04-02T13:43:56.675 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.190+0000 7f9541ffb700 1 -- 172.21.15.52:0/2551081806 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9534004300 con 0x7f9544108b50 2024-04-02T13:43:56.675 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.190+0000 7f9541ffb700 1 -- 172.21.15.52:0/2551081806 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9534011b20 con 0x7f9544108b50 2024-04-02T13:43:56.675 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.191+0000 7f9549273700 1 -- 172.21.15.52:0/2551081806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9544108b50 msgr2=0x7f954410af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:56.675 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.191+0000 7f9549273700 1 --2- 172.21.15.52:0/2551081806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9544108b50 0x7f954410af40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f95340098a0 tx=0x7f9534009bb0 comp rx=0 tx=0).stop 2024-04-02T13:43:56.675 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.191+0000 7f9549273700 1 -- 172.21.15.52:0/2551081806 shutdown_connections 2024-04-02T13:43:56.675 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.191+0000 7f9549273700 1 --2- 172.21.15.52:0/2551081806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9544108b50 0x7f954410af40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:56.676 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.191+0000 7f9549273700 1 -- 172.21.15.52:0/2551081806 >> 172.21.15.52:0/2551081806 conn(0x7f9544102500 msgr2=0x7f9544104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:56.676 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.191+0000 7f9549273700 1 -- 172.21.15.52:0/2551081806 shutdown_connections 2024-04-02T13:43:56.676 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.191+0000 7f9549273700 1 -- 172.21.15.52:0/2551081806 wait complete. 2024-04-02T13:43:56.676 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.191+0000 7f9549273700 1 Processor -- start 2024-04-02T13:43:56.676 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.192+0000 7f9549273700 1 -- start start 2024-04-02T13:43:56.676 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.192+0000 7f9549273700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9544108b50 0x7f954419bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:56.676 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.192+0000 7f9549273700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f954410c430 con 0x7f9544108b50 2024-04-02T13:43:56.676 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.192+0000 7f9542ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9544108b50 0x7f954419bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:56.676 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.192+0000 7f9542ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9544108b50 0x7f954419bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42552/0 (socket says 172.21.15.52:42552) 2024-04-02T13:43:56.676 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.192+0000 7f9542ffd700 1 -- 172.21.15.52:0/3654375681 learned_addr learned my addr 172.21.15.52:0/3654375681 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:56.676 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.192+0000 7f9542ffd700 1 -- 172.21.15.52:0/3654375681 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954419c4f0 con 0x7f9544108b50 2024-04-02T13:43:56.676 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.192+0000 7f9542ffd700 1 --2- 172.21.15.52:0/3654375681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9544108b50 0x7f954419bfb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f95340117e0 tx=0x7f9534011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:56.676 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.193+0000 7f953bfff700 1 -- 172.21.15.52:0/3654375681 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9534003bf0 con 0x7f9544108b50 2024-04-02T13:43:56.676 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.193+0000 7f9549273700 1 -- 172.21.15.52:0/3654375681 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f954419c780 con 0x7f9544108b50 2024-04-02T13:43:56.677 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.193+0000 7f953bfff700 1 -- 172.21.15.52:0/3654375681 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f953401a910 con 0x7f9544108b50 2024-04-02T13:43:56.677 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.193+0000 7f953bfff700 1 -- 172.21.15.52:0/3654375681 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9534024970 con 0x7f9544108b50 2024-04-02T13:43:56.677 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.193+0000 7f9549273700 1 -- 172.21.15.52:0/3654375681 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f954419cbe0 con 0x7f9544108b50 2024-04-02T13:43:56.677 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.193+0000 7f953bfff700 1 -- 172.21.15.52:0/3654375681 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f9534022020 con 0x7f9544108b50 2024-04-02T13:43:56.677 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.194+0000 7f953bfff700 1 --2- 172.21.15.52:0/3654375681 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f95300395b0 0x7f953003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:56.677 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.194+0000 7f9549273700 1 -- 172.21.15.52:0/3654375681 --> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9544063c20 con 0x7f95300395b0 2024-04-02T13:43:56.677 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.194+0000 7f953bfff700 1 -- 172.21.15.52:0/3654375681 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f953404da90 con 0x7f9544108b50 2024-04-02T13:43:56.677 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.194+0000 7f95427fc700 1 -- 172.21.15.52:0/3654375681 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f95300395b0 msgr2=0x7f953003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/673843171 2024-04-02T13:43:56.677 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.194+0000 7f95427fc700 1 --2- 172.21.15.52:0/3654375681 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f95300395b0 0x7f953003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:43:56.677 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.395+0000 7f95427fc700 1 -- 172.21.15.52:0/3654375681 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f95300395b0 msgr2=0x7f953003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/673843171 2024-04-02T13:43:56.677 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.395+0000 7f95427fc700 1 --2- 172.21.15.52:0/3654375681 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f95300395b0 0x7f953003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T13:43:56.677 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.796+0000 7f95427fc700 1 -- 172.21.15.52:0/3654375681 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f95300395b0 msgr2=0x7f953003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/673843171 2024-04-02T13:43:56.677 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:50.796+0000 7f95427fc700 1 --2- 172.21.15.52:0/3654375681 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f95300395b0 0x7f953003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-02T13:43:56.677 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:51.597+0000 7f95427fc700 1 -- 172.21.15.52:0/3654375681 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f95300395b0 msgr2=0x7f953003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/673843171 2024-04-02T13:43:56.678 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:51.597+0000 7f95427fc700 1 --2- 172.21.15.52:0/3654375681 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f95300395b0 0x7f953003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-02T13:43:56.678 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:53.198+0000 7f95427fc700 1 -- 172.21.15.52:0/3654375681 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f95300395b0 msgr2=0x7f953003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/673843171 2024-04-02T13:43:56.678 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:53.198+0000 7f95427fc700 1 --2- 172.21.15.52:0/3654375681 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f95300395b0 0x7f953003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-02T13:43:56.678 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:54.654+0000 7f953bfff700 1 -- 172.21.15.52:0/3654375681 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f953404d050 con 0x7f9544108b50 2024-04-02T13:43:56.678 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:54.654+0000 7f953bfff700 1 -- 172.21.15.52:0/3654375681 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f95300395b0 msgr2=0x7f953003ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:43:56.678 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:54.654+0000 7f953bfff700 1 --2- 172.21.15.52:0/3654375681 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f95300395b0 0x7f953003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:56.678 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:55.656+0000 7f953bfff700 1 -- 172.21.15.52:0/3654375681 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f953404f9c0 con 0x7f9544108b50 2024-04-02T13:43:56.678 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:55.656+0000 7f953bfff700 1 --2- 172.21.15.52:0/3654375681 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f953003ce40 0x7f953003f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:56.678 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:55.656+0000 7f953bfff700 1 -- 172.21.15.52:0/3654375681 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f953401ac60 con 0x7f953003ce40 2024-04-02T13:43:56.678 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:55.656+0000 7f95427fc700 1 --2- 172.21.15.52:0/3654375681 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f953003ce40 0x7f953003f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:56.678 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:55.657+0000 7f95427fc700 1 --2- 172.21.15.52:0/3654375681 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f953003ce40 0x7f953003f230 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f952c003a10 tx=0x7f952c0092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:56.678 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:55.658+0000 7f953bfff700 1 -- 172.21.15.52:0/3654375681 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f953401ac60 con 0x7f953003ce40 2024-04-02T13:43:56.678 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:55.665+0000 7f9549273700 1 -- 172.21.15.52:0/3654375681 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f9544103720 con 0x7f953003ce40 2024-04-02T13:43:56.678 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:55.665+0000 7f953bfff700 1 -- 172.21.15.52:0/3654375681 <== mgr.14152 v2:172.21.15.52:6800/3836085837 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f9544103720 con 0x7f953003ce40 2024-04-02T13:43:56.679 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:55.665+0000 7f9549273700 1 -- 172.21.15.52:0/3654375681 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f953003ce40 msgr2=0x7f953003f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:56.679 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:55.665+0000 7f9549273700 1 --2- 172.21.15.52:0/3654375681 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f953003ce40 0x7f953003f230 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f952c003a10 tx=0x7f952c0092b0 comp rx=0 tx=0).stop 2024-04-02T13:43:56.679 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:55.665+0000 7f9549273700 1 -- 172.21.15.52:0/3654375681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9544108b50 msgr2=0x7f954419bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:56.679 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:55.665+0000 7f9549273700 1 --2- 172.21.15.52:0/3654375681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9544108b50 0x7f954419bfb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f95340117e0 tx=0x7f9534011c10 comp rx=0 tx=0).stop 2024-04-02T13:43:56.679 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:55.665+0000 7f9549273700 1 -- 172.21.15.52:0/3654375681 shutdown_connections 2024-04-02T13:43:56.679 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:55.665+0000 7f9549273700 1 --2- 172.21.15.52:0/3654375681 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f953003ce40 0x7f953003f230 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:56.679 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:55.665+0000 7f9549273700 1 --2- 172.21.15.52:0/3654375681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9544108b50 0x7f954419bfb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:56.679 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:55.665+0000 7f9549273700 1 --2- 172.21.15.52:0/3654375681 >> [v2:172.21.15.52:6800/673843171,v1:172.21.15.52:6801/673843171] conn(0x7f95300395b0 0x7f953003ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:56.679 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:55.665+0000 7f9549273700 1 -- 172.21.15.52:0/3654375681 >> 172.21.15.52:0/3654375681 conn(0x7f9544102500 msgr2=0x7f9544103010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:56.679 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:55.666+0000 7f9549273700 1 -- 172.21.15.52:0/3654375681 shutdown_connections 2024-04-02T13:43:56.679 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:55.666+0000 7f9549273700 1 -- 172.21.15.52:0/3654375681 wait complete. 2024-04-02T13:43:56.679 INFO:teuthology.orchestra.run.smithi052.stdout:mgr epoch 9 is available 2024-04-02T13:43:56.679 INFO:teuthology.orchestra.run.smithi052.stdout:Generating a dashboard self-signed certificate... 2024-04-02T13:43:56.924 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:56 smithi052 ceph-mon[32104]: [02/Apr/2024:13:43:56] ENGINE Bus STARTING 2024-04-02T13:43:56.924 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:56 smithi052 ceph-mon[32104]: [02/Apr/2024:13:43:56] ENGINE Serving on https://172.21.15.52:7150 2024-04-02T13:43:56.924 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:56 smithi052 ceph-mon[32104]: [02/Apr/2024:13:43:56] ENGINE Serving on http://172.21.15.52:8765 2024-04-02T13:43:56.924 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:56 smithi052 ceph-mon[32104]: [02/Apr/2024:13:43:56] ENGINE Bus STARTED 2024-04-02T13:43:58.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:57 smithi052 ceph-mon[32104]: mgrmap e12: x(active, since 2s) 2024-04-02T13:43:58.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:57 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:43:58.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:57 smithi052 ceph-mon[32104]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:43:58.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:57 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:43:58.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:57 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:43:58.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-02T13:43:58.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.116+0000 7fb94d881700 1 Processor -- start 2024-04-02T13:43:58.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.116+0000 7fb94d881700 1 -- start start 2024-04-02T13:43:58.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.116+0000 7fb94d881700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb948108ad0 0x7fb94810aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.116+0000 7fb94d881700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb94807be40 con 0x7fb948108ad0 2024-04-02T13:43:58.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.117+0000 7fb946ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb948108ad0 0x7fb94810aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.117+0000 7fb946ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb948108ad0 0x7fb94810aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57314/0 (socket says 172.21.15.52:57314) 2024-04-02T13:43:58.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.117+0000 7fb946ffd700 1 -- 172.21.15.52:0/1694182668 learned_addr learned my addr 172.21.15.52:0/1694182668 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:58.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.117+0000 7fb946ffd700 1 -- 172.21.15.52:0/1694182668 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb94810b400 con 0x7fb948108ad0 2024-04-02T13:43:58.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.117+0000 7fb946ffd700 1 --2- 172.21.15.52:0/1694182668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb948108ad0 0x7fb94810aec0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb9380098a0 tx=0x7fb938009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ca153be94b973d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:58.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.117+0000 7fb945ffb700 1 -- 172.21.15.52:0/1694182668 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb938004160 con 0x7fb948108ad0 2024-04-02T13:43:58.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.117+0000 7fb945ffb700 1 -- 172.21.15.52:0/1694182668 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb938004300 con 0x7fb948108ad0 2024-04-02T13:43:58.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.117+0000 7fb945ffb700 1 -- 172.21.15.52:0/1694182668 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb938011b20 con 0x7fb948108ad0 2024-04-02T13:43:58.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.118+0000 7fb94d881700 1 -- 172.21.15.52:0/1694182668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb948108ad0 msgr2=0x7fb94810aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.118+0000 7fb94d881700 1 --2- 172.21.15.52:0/1694182668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb948108ad0 0x7fb94810aec0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb9380098a0 tx=0x7fb938009bb0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.118+0000 7fb94d881700 1 -- 172.21.15.52:0/1694182668 shutdown_connections 2024-04-02T13:43:58.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.118+0000 7fb94d881700 1 --2- 172.21.15.52:0/1694182668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb948108ad0 0x7fb94810aec0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.118+0000 7fb94d881700 1 -- 172.21.15.52:0/1694182668 >> 172.21.15.52:0/1694182668 conn(0x7fb948102500 msgr2=0x7fb948104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:58.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.118+0000 7fb94d881700 1 -- 172.21.15.52:0/1694182668 shutdown_connections 2024-04-02T13:43:58.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.118+0000 7fb94d881700 1 -- 172.21.15.52:0/1694182668 wait complete. 2024-04-02T13:43:58.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.119+0000 7fb94d881700 1 Processor -- start 2024-04-02T13:43:58.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.119+0000 7fb94d881700 1 -- start start 2024-04-02T13:43:58.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.119+0000 7fb94d881700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb948108ad0 0x7fb94819bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.119+0000 7fb94d881700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb94810c460 con 0x7fb948108ad0 2024-04-02T13:43:58.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.119+0000 7fb946ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb948108ad0 0x7fb94819bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.119+0000 7fb946ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb948108ad0 0x7fb94819bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57328/0 (socket says 172.21.15.52:57328) 2024-04-02T13:43:58.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.119+0000 7fb946ffd700 1 -- 172.21.15.52:0/1807104609 learned_addr learned my addr 172.21.15.52:0/1807104609 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:58.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.120+0000 7fb946ffd700 1 -- 172.21.15.52:0/1807104609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb94819c510 con 0x7fb948108ad0 2024-04-02T13:43:58.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.120+0000 7fb946ffd700 1 --2- 172.21.15.52:0/1807104609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb948108ad0 0x7fb94819bfd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb9380117e0 tx=0x7fb938011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:58.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.120+0000 7fb94c87f700 1 -- 172.21.15.52:0/1807104609 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb938003ba0 con 0x7fb948108ad0 2024-04-02T13:43:58.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.120+0000 7fb94c87f700 1 -- 172.21.15.52:0/1807104609 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb93801a910 con 0x7fb948108ad0 2024-04-02T13:43:58.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.120+0000 7fb94d881700 1 -- 172.21.15.52:0/1807104609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb94819c7a0 con 0x7fb948108ad0 2024-04-02T13:43:58.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.120+0000 7fb94c87f700 1 -- 172.21.15.52:0/1807104609 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9380248f0 con 0x7fb948108ad0 2024-04-02T13:43:58.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.120+0000 7fb94d881700 1 -- 172.21.15.52:0/1807104609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb94819cc00 con 0x7fb948108ad0 2024-04-02T13:43:58.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.121+0000 7fb94c87f700 1 -- 172.21.15.52:0/1807104609 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fb938022020 con 0x7fb948108ad0 2024-04-02T13:43:58.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.121+0000 7fb94c87f700 1 --2- 172.21.15.52:0/1807104609 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb930039660 0x7fb93003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.121+0000 7fb94c87f700 1 -- 172.21.15.52:0/1807104609 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb93804e1d0 con 0x7fb948108ad0 2024-04-02T13:43:58.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.121+0000 7fb94d881700 1 -- 172.21.15.52:0/1807104609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb948063c20 con 0x7fb948108ad0 2024-04-02T13:43:58.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.121+0000 7fb9467fc700 1 --2- 172.21.15.52:0/1807104609 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb930039660 0x7fb93003bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.122+0000 7fb9467fc700 1 --2- 172.21.15.52:0/1807104609 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb930039660 0x7fb93003bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb93c006fd0 tx=0x7fb93c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:58.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.127+0000 7fb94c87f700 1 -- 172.21.15.52:0/1807104609 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb93800b5a0 con 0x7fb948108ad0 2024-04-02T13:43:58.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.347+0000 7fb94d881700 1 -- 172.21.15.52:0/1807104609 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fb9481036e0 con 0x7fb930039660 2024-04-02T13:43:58.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.526+0000 7fb94c87f700 1 -- 172.21.15.52:0/1807104609 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fb9481036e0 con 0x7fb930039660 2024-04-02T13:43:58.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.528+0000 7fb94d881700 1 -- 172.21.15.52:0/1807104609 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb930039660 msgr2=0x7fb93003bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.528+0000 7fb94d881700 1 --2- 172.21.15.52:0/1807104609 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb930039660 0x7fb93003bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb93c006fd0 tx=0x7fb93c006e40 comp rx=0 tx=0).stop 2024-04-02T13:43:58.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.528+0000 7fb94d881700 1 -- 172.21.15.52:0/1807104609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb948108ad0 msgr2=0x7fb94819bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.528+0000 7fb94d881700 1 --2- 172.21.15.52:0/1807104609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb948108ad0 0x7fb94819bfd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb9380117e0 tx=0x7fb938011ba0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.528+0000 7fb94d881700 1 -- 172.21.15.52:0/1807104609 shutdown_connections 2024-04-02T13:43:58.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.528+0000 7fb94d881700 1 --2- 172.21.15.52:0/1807104609 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb930039660 0x7fb93003bb20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.528+0000 7fb94d881700 1 --2- 172.21.15.52:0/1807104609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb948108ad0 0x7fb94819bfd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.528+0000 7fb94d881700 1 -- 172.21.15.52:0/1807104609 >> 172.21.15.52:0/1807104609 conn(0x7fb948102500 msgr2=0x7fb948102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:58.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.528+0000 7fb94d881700 1 -- 172.21.15.52:0/1807104609 shutdown_connections 2024-04-02T13:43:58.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:57.528+0000 7fb94d881700 1 -- 172.21.15.52:0/1807104609 wait complete. 2024-04-02T13:43:58.107 INFO:teuthology.orchestra.run.smithi052.stdout:Creating initial admin user... 2024-04-02T13:43:59.544 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$M0h90cw4Eh4mbz0rLmG59OOxKq2MX0yj547rRRrZl9YdYqi19cyq6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1712065439, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-02T13:43:59.544 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.518+0000 7f3a830fa700 1 Processor -- start 2024-04-02T13:43:59.544 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.518+0000 7f3a830fa700 1 -- start start 2024-04-02T13:43:59.544 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.518+0000 7f3a830fa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7c106f80 0x7f3a7c1073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.544 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.518+0000 7f3a830fa700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1078e0 con 0x7f3a7c106f80 2024-04-02T13:43:59.544 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.518+0000 7f3a80e96700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7c106f80 0x7f3a7c1073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.544 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.518+0000 7f3a80e96700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7c106f80 0x7f3a7c1073a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57330/0 (socket says 172.21.15.52:57330) 2024-04-02T13:43:59.545 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.518+0000 7f3a80e96700 1 -- 172.21.15.52:0/1315244927 learned_addr learned my addr 172.21.15.52:0/1315244927 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:59.545 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.518+0000 7f3a80e96700 1 -- 172.21.15.52:0/1315244927 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7c108090 con 0x7f3a7c106f80 2024-04-02T13:43:59.545 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.519+0000 7f3a80e96700 1 --2- 172.21.15.52:0/1315244927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7c106f80 0x7f3a7c1073a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3a700098a0 tx=0x7f3a70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4612dd6ce8c416fc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:59.545 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.519+0000 7f3a7b7fe700 1 -- 172.21.15.52:0/1315244927 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a70004160 con 0x7f3a7c106f80 2024-04-02T13:43:59.545 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.519+0000 7f3a7b7fe700 1 -- 172.21.15.52:0/1315244927 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a70004300 con 0x7f3a7c106f80 2024-04-02T13:43:59.545 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.519+0000 7f3a7b7fe700 1 -- 172.21.15.52:0/1315244927 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a70011b20 con 0x7f3a7c106f80 2024-04-02T13:43:59.545 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.519+0000 7f3a830fa700 1 -- 172.21.15.52:0/1315244927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7c106f80 msgr2=0x7f3a7c1073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.545 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.519+0000 7f3a830fa700 1 --2- 172.21.15.52:0/1315244927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7c106f80 0x7f3a7c1073a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3a700098a0 tx=0x7f3a70009bb0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.545 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.519+0000 7f3a830fa700 1 -- 172.21.15.52:0/1315244927 shutdown_connections 2024-04-02T13:43:59.545 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.519+0000 7f3a830fa700 1 --2- 172.21.15.52:0/1315244927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7c106f80 0x7f3a7c1073a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.545 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.519+0000 7f3a830fa700 1 -- 172.21.15.52:0/1315244927 >> 172.21.15.52:0/1315244927 conn(0x7f3a7c102540 msgr2=0x7f3a7c104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:59.545 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.519+0000 7f3a830fa700 1 -- 172.21.15.52:0/1315244927 shutdown_connections 2024-04-02T13:43:59.545 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.519+0000 7f3a830fa700 1 -- 172.21.15.52:0/1315244927 wait complete. 2024-04-02T13:43:59.545 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.520+0000 7f3a830fa700 1 Processor -- start 2024-04-02T13:43:59.546 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.520+0000 7f3a830fa700 1 -- start start 2024-04-02T13:43:59.546 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.520+0000 7f3a830fa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7c106f80 0x7f3a7c19bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.546 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.520+0000 7f3a830fa700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c10b4d0 con 0x7f3a7c106f80 2024-04-02T13:43:59.546 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.520+0000 7f3a80e96700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7c106f80 0x7f3a7c19bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.546 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.520+0000 7f3a80e96700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7c106f80 0x7f3a7c19bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57332/0 (socket says 172.21.15.52:57332) 2024-04-02T13:43:59.546 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.520+0000 7f3a80e96700 1 -- 172.21.15.52:0/3703712634 learned_addr learned my addr 172.21.15.52:0/3703712634 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:43:59.546 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.521+0000 7f3a80e96700 1 -- 172.21.15.52:0/3703712634 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7c19c4d0 con 0x7f3a7c106f80 2024-04-02T13:43:59.546 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.521+0000 7f3a80e96700 1 --2- 172.21.15.52:0/3703712634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7c106f80 0x7f3a7c19bf90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3a700117e0 tx=0x7f3a70011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:59.546 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.521+0000 7f3a79ffb700 1 -- 172.21.15.52:0/3703712634 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a70003ba0 con 0x7f3a7c106f80 2024-04-02T13:43:59.546 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.521+0000 7f3a79ffb700 1 -- 172.21.15.52:0/3703712634 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a7001a910 con 0x7f3a7c106f80 2024-04-02T13:43:59.546 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.521+0000 7f3a830fa700 1 -- 172.21.15.52:0/3703712634 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a7c19c760 con 0x7f3a7c106f80 2024-04-02T13:43:59.546 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.521+0000 7f3a79ffb700 1 -- 172.21.15.52:0/3703712634 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a700248f0 con 0x7f3a7c106f80 2024-04-02T13:43:59.546 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.521+0000 7f3a830fa700 1 -- 172.21.15.52:0/3703712634 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a7c19cbc0 con 0x7f3a7c106f80 2024-04-02T13:43:59.547 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.522+0000 7f3a79ffb700 1 -- 172.21.15.52:0/3703712634 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f3a70022020 con 0x7f3a7c106f80 2024-04-02T13:43:59.547 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.522+0000 7f3a79ffb700 1 --2- 172.21.15.52:0/3703712634 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3a6c039600 0x7f3a6c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.547 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.522+0000 7f3a79ffb700 1 -- 172.21.15.52:0/3703712634 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3a7004e1e0 con 0x7f3a7c106f80 2024-04-02T13:43:59.547 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.522+0000 7f3a830fa700 1 -- 172.21.15.52:0/3703712634 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a7c063c20 con 0x7f3a7c106f80 2024-04-02T13:43:59.548 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.522+0000 7f3a7bfff700 1 --2- 172.21.15.52:0/3703712634 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3a6c039600 0x7f3a6c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.548 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.523+0000 7f3a7bfff700 1 --2- 172.21.15.52:0/3703712634 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3a6c039600 0x7f3a6c03bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3a68006fd0 tx=0x7f3a68006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:59.548 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.527+0000 7f3a79ffb700 1 -- 172.21.15.52:0/3703712634 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3a70018350 con 0x7f3a7c106f80 2024-04-02T13:43:59.548 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:58.761+0000 7f3a830fa700 1 -- 172.21.15.52:0/3703712634 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f3a7c1036a0 con 0x7f3a6c039600 2024-04-02T13:43:59.548 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:59.007+0000 7f3a79ffb700 1 -- 172.21.15.52:0/3703712634 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f3a7c1036a0 con 0x7f3a6c039600 2024-04-02T13:43:59.548 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:59.010+0000 7f3a830fa700 1 -- 172.21.15.52:0/3703712634 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3a6c039600 msgr2=0x7f3a6c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.548 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:59.010+0000 7f3a830fa700 1 --2- 172.21.15.52:0/3703712634 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3a6c039600 0x7f3a6c03bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3a68006fd0 tx=0x7f3a68006e40 comp rx=0 tx=0).stop 2024-04-02T13:43:59.548 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:59.010+0000 7f3a830fa700 1 -- 172.21.15.52:0/3703712634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7c106f80 msgr2=0x7f3a7c19bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.548 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:59.010+0000 7f3a830fa700 1 --2- 172.21.15.52:0/3703712634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7c106f80 0x7f3a7c19bf90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3a700117e0 tx=0x7f3a70011ba0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.548 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:59.010+0000 7f3a830fa700 1 -- 172.21.15.52:0/3703712634 shutdown_connections 2024-04-02T13:43:59.548 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:59.010+0000 7f3a830fa700 1 --2- 172.21.15.52:0/3703712634 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3a6c039600 0x7f3a6c03bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.548 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:59.010+0000 7f3a830fa700 1 --2- 172.21.15.52:0/3703712634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7c106f80 0x7f3a7c19bf90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.549 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:59.010+0000 7f3a830fa700 1 -- 172.21.15.52:0/3703712634 >> 172.21.15.52:0/3703712634 conn(0x7f3a7c102540 msgr2=0x7f3a7c102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:59.549 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:59.010+0000 7f3a830fa700 1 -- 172.21.15.52:0/3703712634 shutdown_connections 2024-04-02T13:43:59.549 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:43:59.010+0000 7f3a830fa700 1 -- 172.21.15.52:0/3703712634 wait complete. 2024-04-02T13:43:59.549 INFO:teuthology.orchestra.run.smithi052.stdout:Fetching dashboard port number... 2024-04-02T13:43:59.964 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:59 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:43:59.964 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:43:59 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:00.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:00 smithi052 ceph-mon[32104]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:44:00.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:00 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/604185471' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-02T13:44:01.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 8443 2024-04-02T13:44:01.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.065+0000 7f616112a700 1 Processor -- start 2024-04-02T13:44:01.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.065+0000 7f616112a700 1 -- start start 2024-04-02T13:44:01.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.065+0000 7f616112a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f615c108a90 0x7f615c10ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:01.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.065+0000 7f616112a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f615c07be80 con 0x7f615c108a90 2024-04-02T13:44:01.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.066+0000 7f615ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f615c108a90 0x7f615c10ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:01.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.066+0000 7f615ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f615c108a90 0x7f615c10ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57348/0 (socket says 172.21.15.52:57348) 2024-04-02T13:44:01.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.066+0000 7f615ad9d700 1 -- 172.21.15.52:0/2465818121 learned_addr learned my addr 172.21.15.52:0/2465818121 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:44:01.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.066+0000 7f615ad9d700 1 -- 172.21.15.52:0/2465818121 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615c10b3c0 con 0x7f615c108a90 2024-04-02T13:44:01.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.066+0000 7f615ad9d700 1 --2- 172.21.15.52:0/2465818121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f615c108a90 0x7f615c10ae80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f61500098a0 tx=0x7f6150009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf1a99291a5141d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:01.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.066+0000 7f6159d9b700 1 -- 172.21.15.52:0/2465818121 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6150004160 con 0x7f615c108a90 2024-04-02T13:44:01.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.067+0000 7f6159d9b700 1 -- 172.21.15.52:0/2465818121 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6150004300 con 0x7f615c108a90 2024-04-02T13:44:01.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.067+0000 7f6159d9b700 1 -- 172.21.15.52:0/2465818121 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6150011b20 con 0x7f615c108a90 2024-04-02T13:44:01.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.067+0000 7f616112a700 1 -- 172.21.15.52:0/2465818121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f615c108a90 msgr2=0x7f615c10ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:01.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.067+0000 7f616112a700 1 --2- 172.21.15.52:0/2465818121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f615c108a90 0x7f615c10ae80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f61500098a0 tx=0x7f6150009bb0 comp rx=0 tx=0).stop 2024-04-02T13:44:01.728 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.067+0000 7f616112a700 1 -- 172.21.15.52:0/2465818121 shutdown_connections 2024-04-02T13:44:01.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.067+0000 7f616112a700 1 --2- 172.21.15.52:0/2465818121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f615c108a90 0x7f615c10ae80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:01.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.067+0000 7f616112a700 1 -- 172.21.15.52:0/2465818121 >> 172.21.15.52:0/2465818121 conn(0x7f615c102500 msgr2=0x7f615c104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:01.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.067+0000 7f616112a700 1 -- 172.21.15.52:0/2465818121 shutdown_connections 2024-04-02T13:44:01.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.067+0000 7f616112a700 1 -- 172.21.15.52:0/2465818121 wait complete. 2024-04-02T13:44:01.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.068+0000 7f616112a700 1 Processor -- start 2024-04-02T13:44:01.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.068+0000 7f616112a700 1 -- start start 2024-04-02T13:44:01.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.068+0000 7f616112a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f615c108a90 0x7f615c19bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:01.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.068+0000 7f616112a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f615c10c430 con 0x7f615c108a90 2024-04-02T13:44:01.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.069+0000 7f615ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f615c108a90 0x7f615c19bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:01.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.069+0000 7f615ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f615c108a90 0x7f615c19bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57360/0 (socket says 172.21.15.52:57360) 2024-04-02T13:44:01.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.069+0000 7f615ad9d700 1 -- 172.21.15.52:0/604185471 learned_addr learned my addr 172.21.15.52:0/604185471 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:44:01.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.069+0000 7f615ad9d700 1 -- 172.21.15.52:0/604185471 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615c19c4c0 con 0x7f615c108a90 2024-04-02T13:44:01.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.069+0000 7f615ad9d700 1 --2- 172.21.15.52:0/604185471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f615c108a90 0x7f615c19bf80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f61500117e0 tx=0x7f6150011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:01.729 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.069+0000 7f6143fff700 1 -- 172.21.15.52:0/604185471 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6150003bf0 con 0x7f615c108a90 2024-04-02T13:44:01.730 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.069+0000 7f616112a700 1 -- 172.21.15.52:0/604185471 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f615c19c750 con 0x7f615c108a90 2024-04-02T13:44:01.730 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.069+0000 7f616112a700 1 -- 172.21.15.52:0/604185471 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f615c19cbb0 con 0x7f615c108a90 2024-04-02T13:44:01.730 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.069+0000 7f6143fff700 1 -- 172.21.15.52:0/604185471 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f615001a910 con 0x7f615c108a90 2024-04-02T13:44:01.730 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.069+0000 7f6143fff700 1 -- 172.21.15.52:0/604185471 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6150024970 con 0x7f615c108a90 2024-04-02T13:44:01.730 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.070+0000 7f6143fff700 1 -- 172.21.15.52:0/604185471 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f6150022020 con 0x7f615c108a90 2024-04-02T13:44:01.730 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.070+0000 7f6143fff700 1 --2- 172.21.15.52:0/604185471 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f6144039610 0x7f614403bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:01.730 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.070+0000 7f616112a700 1 -- 172.21.15.52:0/604185471 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f615c063c20 con 0x7f615c108a90 2024-04-02T13:44:01.730 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.070+0000 7f6143fff700 1 -- 172.21.15.52:0/604185471 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f615004eb10 con 0x7f615c108a90 2024-04-02T13:44:01.730 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.071+0000 7f615a59c700 1 --2- 172.21.15.52:0/604185471 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f6144039610 0x7f614403bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:01.730 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.071+0000 7f615a59c700 1 --2- 172.21.15.52:0/604185471 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f6144039610 0x7f614403bad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f614c006fd0 tx=0x7f614c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:01.730 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.076+0000 7f6143fff700 1 -- 172.21.15.52:0/604185471 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6150011e20 con 0x7f615c108a90 2024-04-02T13:44:01.730 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.292+0000 7f616112a700 1 -- 172.21.15.52:0/604185471 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f615c19f580 con 0x7f615c108a90 2024-04-02T13:44:01.730 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.292+0000 7f6143fff700 1 -- 172.21.15.52:0/604185471 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f6150010380 con 0x7f615c108a90 2024-04-02T13:44:01.731 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.294+0000 7f616112a700 1 -- 172.21.15.52:0/604185471 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f6144039610 msgr2=0x7f614403bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:01.731 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.294+0000 7f616112a700 1 --2- 172.21.15.52:0/604185471 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f6144039610 0x7f614403bad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f614c006fd0 tx=0x7f614c006e40 comp rx=0 tx=0).stop 2024-04-02T13:44:01.731 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.294+0000 7f616112a700 1 -- 172.21.15.52:0/604185471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f615c108a90 msgr2=0x7f615c19bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:01.731 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.294+0000 7f616112a700 1 --2- 172.21.15.52:0/604185471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f615c108a90 0x7f615c19bf80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f61500117e0 tx=0x7f6150011c10 comp rx=0 tx=0).stop 2024-04-02T13:44:01.731 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.294+0000 7f616112a700 1 -- 172.21.15.52:0/604185471 shutdown_connections 2024-04-02T13:44:01.731 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.294+0000 7f616112a700 1 --2- 172.21.15.52:0/604185471 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f6144039610 0x7f614403bad0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:01.731 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.294+0000 7f616112a700 1 --2- 172.21.15.52:0/604185471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f615c108a90 0x7f615c19bf80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:01.731 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.294+0000 7f616112a700 1 -- 172.21.15.52:0/604185471 >> 172.21.15.52:0/604185471 conn(0x7f615c102500 msgr2=0x7f615c102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:01.731 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.295+0000 7f616112a700 1 -- 172.21.15.52:0/604185471 shutdown_connections 2024-04-02T13:44:01.731 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:00.295+0000 7f616112a700 1 -- 172.21.15.52:0/604185471 wait complete. 2024-04-02T13:44:01.734 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T13:44:01.734 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout disabled 2024-04-02T13:44:01.742 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T13:44:01.742 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2024-04-02T13:44:01.742 INFO:teuthology.orchestra.run.smithi052.stdout:firewalld.service is not enabled 2024-04-02T13:44:01.743 INFO:teuthology.orchestra.run.smithi052.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-02T13:44:01.744 INFO:teuthology.orchestra.run.smithi052.stdout:Ceph Dashboard is now available at: 2024-04-02T13:44:01.744 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:44:01.744 INFO:teuthology.orchestra.run.smithi052.stdout: URL: https://smithi052.front.sepia.ceph.com:8443/ 2024-04-02T13:44:01.744 INFO:teuthology.orchestra.run.smithi052.stdout: User: admin 2024-04-02T13:44:01.744 INFO:teuthology.orchestra.run.smithi052.stdout: Password: u9ja69gl5t 2024-04-02T13:44:01.744 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:44:01.745 INFO:teuthology.orchestra.run.smithi052.stdout:Saving cluster configuration to /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config directory 2024-04-02T13:44:01.745 INFO:teuthology.orchestra.run.smithi052.stdout:Enabling autotune for osd_memory_target 2024-04-02T13:44:02.697 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:02 smithi052 ceph-mon[32104]: mgrmap e13: x(active, since 6s) 2024-04-02T13:44:03.000 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.187+0000 7f3525ce1700 1 Processor -- start 2024-04-02T13:44:03.000 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.187+0000 7f3525ce1700 1 -- start start 2024-04-02T13:44:03.000 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.187+0000 7f3525ce1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f352007de20 0x7f352007c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:03.000 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.187+0000 7f3525ce1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f352007c8a0 con 0x7f352007de20 2024-04-02T13:44:03.000 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.188+0000 7f351f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f352007de20 0x7f352007c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:03.000 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.188+0000 7f351f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f352007de20 0x7f352007c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53478/0 (socket says 172.21.15.52:53478) 2024-04-02T13:44:03.000 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.188+0000 7f351f7fe700 1 -- 172.21.15.52:0/3331201795 learned_addr learned my addr 172.21.15.52:0/3331201795 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:44:03.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.188+0000 7f351f7fe700 1 -- 172.21.15.52:0/3331201795 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352007ca20 con 0x7f352007de20 2024-04-02T13:44:03.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.188+0000 7f351f7fe700 1 --2- 172.21.15.52:0/3331201795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f352007de20 0x7f352007c2d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f35100098a0 tx=0x7f3510009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccffa8eb955ab957 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:03.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.188+0000 7f351e7fc700 1 -- 172.21.15.52:0/3331201795 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3510004160 con 0x7f352007de20 2024-04-02T13:44:03.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.188+0000 7f351e7fc700 1 -- 172.21.15.52:0/3331201795 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3510004300 con 0x7f352007de20 2024-04-02T13:44:03.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.188+0000 7f351e7fc700 1 -- 172.21.15.52:0/3331201795 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3510011b20 con 0x7f352007de20 2024-04-02T13:44:03.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.189+0000 7f3525ce1700 1 -- 172.21.15.52:0/3331201795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f352007de20 msgr2=0x7f352007c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:03.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.189+0000 7f3525ce1700 1 --2- 172.21.15.52:0/3331201795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f352007de20 0x7f352007c2d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f35100098a0 tx=0x7f3510009bb0 comp rx=0 tx=0).stop 2024-04-02T13:44:03.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.189+0000 7f3525ce1700 1 -- 172.21.15.52:0/3331201795 shutdown_connections 2024-04-02T13:44:03.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.189+0000 7f3525ce1700 1 --2- 172.21.15.52:0/3331201795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f352007de20 0x7f352007c2d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:03.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.189+0000 7f3525ce1700 1 -- 172.21.15.52:0/3331201795 >> 172.21.15.52:0/3331201795 conn(0x7f3520103660 msgr2=0x7f3520105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:03.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.189+0000 7f3525ce1700 1 -- 172.21.15.52:0/3331201795 shutdown_connections 2024-04-02T13:44:03.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.189+0000 7f3525ce1700 1 -- 172.21.15.52:0/3331201795 wait complete. 2024-04-02T13:44:03.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.190+0000 7f3525ce1700 1 Processor -- start 2024-04-02T13:44:03.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.190+0000 7f3525ce1700 1 -- start start 2024-04-02T13:44:03.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.190+0000 7f3525ce1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f352007de20 0x7f352019e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:03.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.190+0000 7f3525ce1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f352010e660 con 0x7f352007de20 2024-04-02T13:44:03.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.190+0000 7f351f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f352007de20 0x7f352019e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:03.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.190+0000 7f351f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f352007de20 0x7f352019e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53488/0 (socket says 172.21.15.52:53488) 2024-04-02T13:44:03.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.190+0000 7f351f7fe700 1 -- 172.21.15.52:0/1112316807 learned_addr learned my addr 172.21.15.52:0/1112316807 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:44:03.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.191+0000 7f351f7fe700 1 -- 172.21.15.52:0/1112316807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352019e730 con 0x7f352007de20 2024-04-02T13:44:03.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.191+0000 7f351f7fe700 1 --2- 172.21.15.52:0/1112316807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f352007de20 0x7f352019e1f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3510011b60 tx=0x7f3510011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:03.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.191+0000 7f351cff9700 1 -- 172.21.15.52:0/1112316807 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3510003b60 con 0x7f352007de20 2024-04-02T13:44:03.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.191+0000 7f3525ce1700 1 -- 172.21.15.52:0/1112316807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f352019e9c0 con 0x7f352007de20 2024-04-02T13:44:03.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.191+0000 7f351cff9700 1 -- 172.21.15.52:0/1112316807 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f351001a910 con 0x7f352007de20 2024-04-02T13:44:03.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.191+0000 7f351cff9700 1 -- 172.21.15.52:0/1112316807 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3510024930 con 0x7f352007de20 2024-04-02T13:44:03.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.191+0000 7f3525ce1700 1 -- 172.21.15.52:0/1112316807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f352019ee20 con 0x7f352007de20 2024-04-02T13:44:03.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.192+0000 7f351cff9700 1 -- 172.21.15.52:0/1112316807 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3510022020 con 0x7f352007de20 2024-04-02T13:44:03.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.192+0000 7f3525ce1700 1 -- 172.21.15.52:0/1112316807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3520063c20 con 0x7f352007de20 2024-04-02T13:44:03.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.192+0000 7f351cff9700 1 --2- 172.21.15.52:0/1112316807 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3508039700 0x7f350803bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:03.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.192+0000 7f351cff9700 1 -- 172.21.15.52:0/1112316807 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f351004db10 con 0x7f352007de20 2024-04-02T13:44:03.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.192+0000 7f351effd700 1 --2- 172.21.15.52:0/1112316807 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3508039700 0x7f350803bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:03.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.193+0000 7f351effd700 1 --2- 172.21.15.52:0/1112316807 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3508039700 0x7f350803bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3514006fd0 tx=0x7f3514006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:03.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.197+0000 7f351cff9700 1 -- 172.21.15.52:0/1112316807 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f351001a4e0 con 0x7f352007de20 2024-04-02T13:44:03.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.403+0000 7f3525ce1700 1 -- 172.21.15.52:0/1112316807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f35201a18b0 con 0x7f352007de20 2024-04-02T13:44:03.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.404+0000 7f351cff9700 1 -- 172.21.15.52:0/1112316807 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f351002d3f0 con 0x7f352007de20 2024-04-02T13:44:03.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.406+0000 7f3525ce1700 1 -- 172.21.15.52:0/1112316807 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3508039700 msgr2=0x7f350803bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:03.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.406+0000 7f3525ce1700 1 --2- 172.21.15.52:0/1112316807 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3508039700 0x7f350803bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3514006fd0 tx=0x7f3514006e40 comp rx=0 tx=0).stop 2024-04-02T13:44:03.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.406+0000 7f3525ce1700 1 -- 172.21.15.52:0/1112316807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f352007de20 msgr2=0x7f352019e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:03.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.406+0000 7f3525ce1700 1 --2- 172.21.15.52:0/1112316807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f352007de20 0x7f352019e1f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3510011b60 tx=0x7f3510011c40 comp rx=0 tx=0).stop 2024-04-02T13:44:03.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.406+0000 7f3525ce1700 1 -- 172.21.15.52:0/1112316807 shutdown_connections 2024-04-02T13:44:03.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.406+0000 7f3525ce1700 1 --2- 172.21.15.52:0/1112316807 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3508039700 0x7f350803bbc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:03.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.406+0000 7f3525ce1700 1 --2- 172.21.15.52:0/1112316807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f352007de20 0x7f352019e1f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:03.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.406+0000 7f3525ce1700 1 -- 172.21.15.52:0/1112316807 >> 172.21.15.52:0/1112316807 conn(0x7f3520103660 msgr2=0x7f3520104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:03.004 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.406+0000 7f3525ce1700 1 -- 172.21.15.52:0/1112316807 shutdown_connections 2024-04-02T13:44:03.004 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:02.406+0000 7f3525ce1700 1 -- 172.21.15.52:0/1112316807 wait complete. 2024-04-02T13:44:04.346 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.419+0000 7fe0b1e55700 1 Processor -- start 2024-04-02T13:44:04.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.419+0000 7fe0b1e55700 1 -- start start 2024-04-02T13:44:04.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.419+0000 7fe0b1e55700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0ac07e6b0 0x7fe0ac07cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:04.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.419+0000 7fe0b1e55700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac07d130 con 0x7fe0ac07e6b0 2024-04-02T13:44:04.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.419+0000 7fe0ab7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0ac07e6b0 0x7fe0ac07cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:04.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.419+0000 7fe0ab7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0ac07e6b0 0x7fe0ac07cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53504/0 (socket says 172.21.15.52:53504) 2024-04-02T13:44:04.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.419+0000 7fe0ab7fe700 1 -- 172.21.15.52:0/2989429795 learned_addr learned my addr 172.21.15.52:0/2989429795 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:44:04.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.419+0000 7fe0ab7fe700 1 -- 172.21.15.52:0/2989429795 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ac07d2b0 con 0x7fe0ac07e6b0 2024-04-02T13:44:04.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.419+0000 7fe0ab7fe700 1 --2- 172.21.15.52:0/2989429795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0ac07e6b0 0x7fe0ac07cb60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0098a0 tx=0x7fe09c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a358c6672532c384 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:04.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.420+0000 7fe0aa7fc700 1 -- 172.21.15.52:0/2989429795 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe09c004160 con 0x7fe0ac07e6b0 2024-04-02T13:44:04.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.420+0000 7fe0aa7fc700 1 -- 172.21.15.52:0/2989429795 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe09c004300 con 0x7fe0ac07e6b0 2024-04-02T13:44:04.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.420+0000 7fe0aa7fc700 1 -- 172.21.15.52:0/2989429795 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe09c011b20 con 0x7fe0ac07e6b0 2024-04-02T13:44:04.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.420+0000 7fe0b1e55700 1 -- 172.21.15.52:0/2989429795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0ac07e6b0 msgr2=0x7fe0ac07cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:04.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.420+0000 7fe0b1e55700 1 --2- 172.21.15.52:0/2989429795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0ac07e6b0 0x7fe0ac07cb60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0098a0 tx=0x7fe09c009bb0 comp rx=0 tx=0).stop 2024-04-02T13:44:04.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.420+0000 7fe0b1e55700 1 -- 172.21.15.52:0/2989429795 shutdown_connections 2024-04-02T13:44:04.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.420+0000 7fe0b1e55700 1 --2- 172.21.15.52:0/2989429795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0ac07e6b0 0x7fe0ac07cb60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:04.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.420+0000 7fe0b1e55700 1 -- 172.21.15.52:0/2989429795 >> 172.21.15.52:0/2989429795 conn(0x7fe0ac103670 msgr2=0x7fe0ac105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:04.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.420+0000 7fe0b1e55700 1 -- 172.21.15.52:0/2989429795 shutdown_connections 2024-04-02T13:44:04.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.420+0000 7fe0b1e55700 1 -- 172.21.15.52:0/2989429795 wait complete. 2024-04-02T13:44:04.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.421+0000 7fe0b1e55700 1 Processor -- start 2024-04-02T13:44:04.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.421+0000 7fe0b1e55700 1 -- start start 2024-04-02T13:44:04.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.421+0000 7fe0b1e55700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0ac07e6b0 0x7fe0ac19e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:04.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.421+0000 7fe0b1e55700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac10e690 con 0x7fe0ac07e6b0 2024-04-02T13:44:04.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.422+0000 7fe0ab7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0ac07e6b0 0x7fe0ac19e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:04.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.422+0000 7fe0ab7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0ac07e6b0 0x7fe0ac19e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53508/0 (socket says 172.21.15.52:53508) 2024-04-02T13:44:04.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.422+0000 7fe0ab7fe700 1 -- 172.21.15.52:0/2354344950 learned_addr learned my addr 172.21.15.52:0/2354344950 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:44:04.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.422+0000 7fe0ab7fe700 1 -- 172.21.15.52:0/2354344950 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ac19e6e0 con 0x7fe0ac07e6b0 2024-04-02T13:44:04.349 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.422+0000 7fe0ab7fe700 1 --2- 172.21.15.52:0/2354344950 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0ac07e6b0 0x7fe0ac19e1a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0117e0 tx=0x7fe09c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:04.349 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.422+0000 7fe0a8ff9700 1 -- 172.21.15.52:0/2354344950 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe09c003bf0 con 0x7fe0ac07e6b0 2024-04-02T13:44:04.349 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.422+0000 7fe0a8ff9700 1 -- 172.21.15.52:0/2354344950 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe09c01a910 con 0x7fe0ac07e6b0 2024-04-02T13:44:04.349 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.422+0000 7fe0b1e55700 1 -- 172.21.15.52:0/2354344950 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0ac19e970 con 0x7fe0ac07e6b0 2024-04-02T13:44:04.349 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.422+0000 7fe0a8ff9700 1 -- 172.21.15.52:0/2354344950 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe09c024970 con 0x7fe0ac07e6b0 2024-04-02T13:44:04.349 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.422+0000 7fe0b1e55700 1 -- 172.21.15.52:0/2354344950 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0ac19edd0 con 0x7fe0ac07e6b0 2024-04-02T13:44:04.349 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.423+0000 7fe0a8ff9700 1 -- 172.21.15.52:0/2354344950 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe09c022020 con 0x7fe0ac07e6b0 2024-04-02T13:44:04.349 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.423+0000 7fe0a8ff9700 1 --2- 172.21.15.52:0/2354344950 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe0940396b0 0x7fe09403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:04.349 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.423+0000 7fe0a8ff9700 1 -- 172.21.15.52:0/2354344950 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe09c04dbe0 con 0x7fe0ac07e6b0 2024-04-02T13:44:04.349 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.423+0000 7fe0b1e55700 1 -- 172.21.15.52:0/2354344950 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0ac063c20 con 0x7fe0ac07e6b0 2024-04-02T13:44:04.349 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.423+0000 7fe0aaffd700 1 --2- 172.21.15.52:0/2354344950 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe0940396b0 0x7fe09403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:04.349 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.424+0000 7fe0aaffd700 1 --2- 172.21.15.52:0/2354344950 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe0940396b0 0x7fe09403bb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0006fd0 tx=0x7fe0a0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:04.349 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.428+0000 7fe0a8ff9700 1 -- 172.21.15.52:0/2354344950 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe09c011e20 con 0x7fe0ac07e6b0 2024-04-02T13:44:04.350 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.749+0000 7fe0b1e55700 1 -- 172.21.15.52:0/2354344950 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fe0ac1a1a50 con 0x7fe0ac07e6b0 2024-04-02T13:44:04.350 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.751+0000 7fe0a8ff9700 1 -- 172.21.15.52:0/2354344950 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7fe09c032030 con 0x7fe0ac07e6b0 2024-04-02T13:44:04.350 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.753+0000 7fe0b1e55700 1 -- 172.21.15.52:0/2354344950 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe0940396b0 msgr2=0x7fe09403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:04.350 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.753+0000 7fe0b1e55700 1 --2- 172.21.15.52:0/2354344950 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe0940396b0 0x7fe09403bb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0006fd0 tx=0x7fe0a0006e40 comp rx=0 tx=0).stop 2024-04-02T13:44:04.350 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.753+0000 7fe0b1e55700 1 -- 172.21.15.52:0/2354344950 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0ac07e6b0 msgr2=0x7fe0ac19e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:04.350 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.753+0000 7fe0b1e55700 1 --2- 172.21.15.52:0/2354344950 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0ac07e6b0 0x7fe0ac19e1a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0117e0 tx=0x7fe09c011c10 comp rx=0 tx=0).stop 2024-04-02T13:44:04.350 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.753+0000 7fe0b1e55700 1 -- 172.21.15.52:0/2354344950 shutdown_connections 2024-04-02T13:44:04.350 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.753+0000 7fe0b1e55700 1 --2- 172.21.15.52:0/2354344950 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe0940396b0 0x7fe09403bb70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:04.350 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.753+0000 7fe0b1e55700 1 --2- 172.21.15.52:0/2354344950 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0ac07e6b0 0x7fe0ac19e1a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:04.350 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.753+0000 7fe0b1e55700 1 -- 172.21.15.52:0/2354344950 >> 172.21.15.52:0/2354344950 conn(0x7fe0ac103670 msgr2=0x7fe0ac104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:04.350 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.754+0000 7fe0b1e55700 1 -- 172.21.15.52:0/2354344950 shutdown_connections 2024-04-02T13:44:04.350 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-04-02T13:44:03.754+0000 7fe0b1e55700 1 -- 172.21.15.52:0/2354344950 wait complete. 2024-04-02T13:44:04.350 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-02T13:44:04.350 INFO:teuthology.orchestra.run.smithi052.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-02T13:44:04.351 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:44:04.351 INFO:teuthology.orchestra.run.smithi052.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-02T13:44:04.351 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:44:04.351 INFO:teuthology.orchestra.run.smithi052.stdout:Or, if you are only running a single cluster on this host: 2024-04-02T13:44:04.351 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:44:04.351 INFO:teuthology.orchestra.run.smithi052.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-02T13:44:04.351 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:44:04.351 INFO:teuthology.orchestra.run.smithi052.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-02T13:44:04.351 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:44:04.351 INFO:teuthology.orchestra.run.smithi052.stdout: ceph telemetry on 2024-04-02T13:44:04.351 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:44:04.351 INFO:teuthology.orchestra.run.smithi052.stdout:For more information see: 2024-04-02T13:44:04.351 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:44:04.351 INFO:teuthology.orchestra.run.smithi052.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-02T13:44:04.351 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:44:04.352 INFO:teuthology.orchestra.run.smithi052.stdout:Bootstrap complete. 2024-04-02T13:44:04.386 INFO:tasks.cephadm:Fetching config... 2024-04-02T13:44:04.386 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-04-02T13:44:04.386 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-02T13:44:04.401 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-02T13:44:04.402 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-04-02T13:44:04.402 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-02T13:44:04.457 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-02T13:44:04.457 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-04-02T13:44:04.457 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/keyring of=/dev/stdout 2024-04-02T13:44:04.524 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-02T13:44:04.524 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-04-02T13:44:04.524 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-02T13:44:04.580 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-02T13:44:04.580 DEBUG:teuthology.orchestra.run.smithi052:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDg71LEBNc12A3gZoTZdHgWjUqn9ojL8ipKhZNRcFfgnSdpzo9/dppZ2/F4G7qJSJz694RpqP7dzhv6hJ+rTdWmNslRA7OroYXC69qiQxsSa8peyAQD/WeLjI3YfwmKToPYZkgYtPHAmNoUz5RkZZT7V9+C4To/LOVt/wfHRvpioicZxWIG7A04Bp7dRNhTr9ciVCEt+tXvi+ZHZ+t9Y+kLX1BxLM2nMJU1sYPjAak62k5d4kM9fjF7TsFIGxheh2f7klt7HeaY6WfjVIQyynX3N02E89A5xtWwiqc4wTjJUCfZJYp94Y7ixb3qlrLgw5+Xy19HwutBQbk4EfbEQeGx5082zQsD0qwb3QHmJFcc/CtQUuz5UQtcu9Nd35bD/OqcrS5dsELCtMrVU82oCYTnPOVokXfsZWt7sukceicyIruGhmgK9sRN7uuBx4NcBdUzK5iFxMQ4amo4EAiToF3gQ0j19TEH82penQRAIjbQhj5WebW5nuuAr6peihbvMqc= ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-02T13:44:04.661 INFO:teuthology.orchestra.run.smithi052.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDg71LEBNc12A3gZoTZdHgWjUqn9ojL8ipKhZNRcFfgnSdpzo9/dppZ2/F4G7qJSJz694RpqP7dzhv6hJ+rTdWmNslRA7OroYXC69qiQxsSa8peyAQD/WeLjI3YfwmKToPYZkgYtPHAmNoUz5RkZZT7V9+C4To/LOVt/wfHRvpioicZxWIG7A04Bp7dRNhTr9ciVCEt+tXvi+ZHZ+t9Y+kLX1BxLM2nMJU1sYPjAak62k5d4kM9fjF7TsFIGxheh2f7klt7HeaY6WfjVIQyynX3N02E89A5xtWwiqc4wTjJUCfZJYp94Y7ixb3qlrLgw5+Xy19HwutBQbk4EfbEQeGx5082zQsD0qwb3QHmJFcc/CtQUuz5UQtcu9Nd35bD/OqcrS5dsELCtMrVU82oCYTnPOVokXfsZWt7sukceicyIruGhmgK9sRN7uuBx4NcBdUzK5iFxMQ4amo4EAiToF3gQ0j19TEH82penQRAIjbQhj5WebW5nuuAr6peihbvMqc= ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:44:04.675 DEBUG:teuthology.orchestra.run.smithi088:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDg71LEBNc12A3gZoTZdHgWjUqn9ojL8ipKhZNRcFfgnSdpzo9/dppZ2/F4G7qJSJz694RpqP7dzhv6hJ+rTdWmNslRA7OroYXC69qiQxsSa8peyAQD/WeLjI3YfwmKToPYZkgYtPHAmNoUz5RkZZT7V9+C4To/LOVt/wfHRvpioicZxWIG7A04Bp7dRNhTr9ciVCEt+tXvi+ZHZ+t9Y+kLX1BxLM2nMJU1sYPjAak62k5d4kM9fjF7TsFIGxheh2f7klt7HeaY6WfjVIQyynX3N02E89A5xtWwiqc4wTjJUCfZJYp94Y7ixb3qlrLgw5+Xy19HwutBQbk4EfbEQeGx5082zQsD0qwb3QHmJFcc/CtQUuz5UQtcu9Nd35bD/OqcrS5dsELCtMrVU82oCYTnPOVokXfsZWt7sukceicyIruGhmgK9sRN7uuBx4NcBdUzK5iFxMQ4amo4EAiToF3gQ0j19TEH82penQRAIjbQhj5WebW5nuuAr6peihbvMqc= ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-02T13:44:04.720 INFO:teuthology.orchestra.run.smithi088.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDg71LEBNc12A3gZoTZdHgWjUqn9ojL8ipKhZNRcFfgnSdpzo9/dppZ2/F4G7qJSJz694RpqP7dzhv6hJ+rTdWmNslRA7OroYXC69qiQxsSa8peyAQD/WeLjI3YfwmKToPYZkgYtPHAmNoUz5RkZZT7V9+C4To/LOVt/wfHRvpioicZxWIG7A04Bp7dRNhTr9ciVCEt+tXvi+ZHZ+t9Y+kLX1BxLM2nMJU1sYPjAak62k5d4kM9fjF7TsFIGxheh2f7klt7HeaY6WfjVIQyynX3N02E89A5xtWwiqc4wTjJUCfZJYp94Y7ixb3qlrLgw5+Xy19HwutBQbk4EfbEQeGx5082zQsD0qwb3QHmJFcc/CtQUuz5UQtcu9Nd35bD/OqcrS5dsELCtMrVU82oCYTnPOVokXfsZWt7sukceicyIruGhmgK9sRN7uuBx4NcBdUzK5iFxMQ4amo4EAiToF3gQ0j19TEH82penQRAIjbQhj5WebW5nuuAr6peihbvMqc= ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:44:04.735 DEBUG:teuthology.orchestra.run.smithi189:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDg71LEBNc12A3gZoTZdHgWjUqn9ojL8ipKhZNRcFfgnSdpzo9/dppZ2/F4G7qJSJz694RpqP7dzhv6hJ+rTdWmNslRA7OroYXC69qiQxsSa8peyAQD/WeLjI3YfwmKToPYZkgYtPHAmNoUz5RkZZT7V9+C4To/LOVt/wfHRvpioicZxWIG7A04Bp7dRNhTr9ciVCEt+tXvi+ZHZ+t9Y+kLX1BxLM2nMJU1sYPjAak62k5d4kM9fjF7TsFIGxheh2f7klt7HeaY6WfjVIQyynX3N02E89A5xtWwiqc4wTjJUCfZJYp94Y7ixb3qlrLgw5+Xy19HwutBQbk4EfbEQeGx5082zQsD0qwb3QHmJFcc/CtQUuz5UQtcu9Nd35bD/OqcrS5dsELCtMrVU82oCYTnPOVokXfsZWt7sukceicyIruGhmgK9sRN7uuBx4NcBdUzK5iFxMQ4amo4EAiToF3gQ0j19TEH82penQRAIjbQhj5WebW5nuuAr6peihbvMqc= ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-02T13:44:04.782 INFO:teuthology.orchestra.run.smithi189.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDg71LEBNc12A3gZoTZdHgWjUqn9ojL8ipKhZNRcFfgnSdpzo9/dppZ2/F4G7qJSJz694RpqP7dzhv6hJ+rTdWmNslRA7OroYXC69qiQxsSa8peyAQD/WeLjI3YfwmKToPYZkgYtPHAmNoUz5RkZZT7V9+C4To/LOVt/wfHRvpioicZxWIG7A04Bp7dRNhTr9ciVCEt+tXvi+ZHZ+t9Y+kLX1BxLM2nMJU1sYPjAak62k5d4kM9fjF7TsFIGxheh2f7klt7HeaY6WfjVIQyynX3N02E89A5xtWwiqc4wTjJUCfZJYp94Y7ixb3qlrLgw5+Xy19HwutBQbk4EfbEQeGx5082zQsD0qwb3QHmJFcc/CtQUuz5UQtcu9Nd35bD/OqcrS5dsELCtMrVU82oCYTnPOVokXfsZWt7sukceicyIruGhmgK9sRN7uuBx4NcBdUzK5iFxMQ4amo4EAiToF3gQ0j19TEH82penQRAIjbQhj5WebW5nuuAr6peihbvMqc= ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:44:04.797 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-02T13:44:04.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:04 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2354344950' entity='client.admin' 2024-04-02T13:44:05.039 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:44:07.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.225+0000 7f50bd36d700 1 -- 172.21.15.52:0/1070466978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50b8102af0 msgr2=0x7f50b8104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:07.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.225+0000 7f50bd36d700 1 --2- 172.21.15.52:0/1070466978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50b8102af0 0x7f50b8104ee0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f50a8009910 tx=0x7f50a8009c20 comp rx=0 tx=0).stop 2024-04-02T13:44:07.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.225+0000 7f50bd36d700 1 -- 172.21.15.52:0/1070466978 shutdown_connections 2024-04-02T13:44:07.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.225+0000 7f50bd36d700 1 --2- 172.21.15.52:0/1070466978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50b8102af0 0x7f50b8104ee0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:07.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.225+0000 7f50bd36d700 1 -- 172.21.15.52:0/1070466978 >> 172.21.15.52:0/1070466978 conn(0x7f50b80fc4c0 msgr2=0x7f50b80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:07.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.225+0000 7f50bd36d700 1 -- 172.21.15.52:0/1070466978 shutdown_connections 2024-04-02T13:44:07.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.225+0000 7f50bd36d700 1 -- 172.21.15.52:0/1070466978 wait complete. 2024-04-02T13:44:07.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.226+0000 7f50bd36d700 1 Processor -- start 2024-04-02T13:44:07.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.226+0000 7f50bd36d700 1 -- start start 2024-04-02T13:44:07.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.226+0000 7f50bd36d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50b8102af0 0x7f50b8102370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:07.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.226+0000 7f50bd36d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f50b8108090 con 0x7f50b8102af0 2024-04-02T13:44:07.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.226+0000 7f50b6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50b8102af0 0x7f50b8102370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:07.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.226+0000 7f50b6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50b8102af0 0x7f50b8102370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53518/0 (socket says 172.21.15.52:53518) 2024-04-02T13:44:07.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.226+0000 7f50b6ffd700 1 -- 172.21.15.52:0/2646470888 learned_addr learned my addr 172.21.15.52:0/2646470888 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:44:07.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.227+0000 7f50b6ffd700 1 -- 172.21.15.52:0/2646470888 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b81028b0 con 0x7f50b8102af0 2024-04-02T13:44:07.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.227+0000 7f50b6ffd700 1 --2- 172.21.15.52:0/2646470888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50b8102af0 0x7f50b8102370 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f50a8004fa0 tx=0x7f50a8005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:07.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.227+0000 7f509ffff700 1 -- 172.21.15.52:0/2646470888 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50a8027070 con 0x7f50b8102af0 2024-04-02T13:44:07.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.227+0000 7f509ffff700 1 -- 172.21.15.52:0/2646470888 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50a800be40 con 0x7f50b8102af0 2024-04-02T13:44:07.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.227+0000 7f50bd36d700 1 -- 172.21.15.52:0/2646470888 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b8100aa0 con 0x7f50b8102af0 2024-04-02T13:44:07.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.227+0000 7f509ffff700 1 -- 172.21.15.52:0/2646470888 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50a800f460 con 0x7f50b8102af0 2024-04-02T13:44:07.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.227+0000 7f50bd36d700 1 -- 172.21.15.52:0/2646470888 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b8100fe0 con 0x7f50b8102af0 2024-04-02T13:44:07.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.228+0000 7f509ffff700 1 -- 172.21.15.52:0/2646470888 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f50a802a070 con 0x7f50b8102af0 2024-04-02T13:44:07.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.228+0000 7f50bd36d700 1 -- 172.21.15.52:0/2646470888 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50b8108810 con 0x7f50b8102af0 2024-04-02T13:44:07.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.228+0000 7f509ffff700 1 --2- 172.21.15.52:0/2646470888 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f50a0039650 0x7f50a003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:07.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.228+0000 7f509ffff700 1 -- 172.21.15.52:0/2646470888 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f50a801e030 con 0x7f50b8102af0 2024-04-02T13:44:07.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.228+0000 7f50b67fc700 1 --2- 172.21.15.52:0/2646470888 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f50a0039650 0x7f50a003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:07.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.229+0000 7f50b67fc700 1 --2- 172.21.15.52:0/2646470888 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f50a0039650 0x7f50a003bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f50ac006fd0 tx=0x7f50ac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:07.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.233+0000 7f509ffff700 1 -- 172.21.15.52:0/2646470888 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f50a8013070 con 0x7f50b8102af0 2024-04-02T13:44:07.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.439+0000 7f50bd36d700 1 -- 172.21.15.52:0/2646470888 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f50b8190bd0 con 0x7f50b8102af0 2024-04-02T13:44:07.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.441+0000 7f509ffff700 1 -- 172.21.15.52:0/2646470888 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f50a802b3b0 con 0x7f50b8102af0 2024-04-02T13:44:07.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.443+0000 7f50bd36d700 1 -- 172.21.15.52:0/2646470888 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f50a0039650 msgr2=0x7f50a003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:07.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.443+0000 7f50bd36d700 1 --2- 172.21.15.52:0/2646470888 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f50a0039650 0x7f50a003bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f50ac006fd0 tx=0x7f50ac006e40 comp rx=0 tx=0).stop 2024-04-02T13:44:07.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.443+0000 7f50bd36d700 1 -- 172.21.15.52:0/2646470888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50b8102af0 msgr2=0x7f50b8102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:07.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.443+0000 7f50bd36d700 1 --2- 172.21.15.52:0/2646470888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50b8102af0 0x7f50b8102370 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f50a8004fa0 tx=0x7f50a8005080 comp rx=0 tx=0).stop 2024-04-02T13:44:07.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.443+0000 7f50bd36d700 1 -- 172.21.15.52:0/2646470888 shutdown_connections 2024-04-02T13:44:07.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.443+0000 7f50bd36d700 1 --2- 172.21.15.52:0/2646470888 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f50a0039650 0x7f50a003bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:07.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.443+0000 7f50bd36d700 1 --2- 172.21.15.52:0/2646470888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50b8102af0 0x7f50b8102370 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:07.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.443+0000 7f50bd36d700 1 -- 172.21.15.52:0/2646470888 >> 172.21.15.52:0/2646470888 conn(0x7f50b80fc4c0 msgr2=0x7f50b80fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:07.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.443+0000 7f50bd36d700 1 -- 172.21.15.52:0/2646470888 shutdown_connections 2024-04-02T13:44:07.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:07.443+0000 7f50bd36d700 1 -- 172.21.15.52:0/2646470888 wait complete. 2024-04-02T13:44:07.459 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:07 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:07.459 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:07 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:07.459 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:07 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi052", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:44:07.459 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:07 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:07.459 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:07 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:44:07.459 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:07 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:08.080 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-02T13:44:08.080 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-02T13:44:08.321 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:44:08.746 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:08 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2646470888' entity='client.admin' 2024-04-02T13:44:08.746 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:08 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:44:08.746 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:08 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:09.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.861+0000 7f6f6dd27700 1 -- 172.21.15.52:0/4017120743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f68100f10 msgr2=0x7f6f68101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:09.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.861+0000 7f6f6dd27700 1 --2- 172.21.15.52:0/4017120743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f68100f10 0x7f6f68101350 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009910 tx=0x7f6f58009c20 comp rx=0 tx=0).stop 2024-04-02T13:44:09.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.862+0000 7f6f6dd27700 1 -- 172.21.15.52:0/4017120743 shutdown_connections 2024-04-02T13:44:09.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.862+0000 7f6f6dd27700 1 --2- 172.21.15.52:0/4017120743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f68100f10 0x7f6f68101350 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:09.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.862+0000 7f6f6dd27700 1 -- 172.21.15.52:0/4017120743 >> 172.21.15.52:0/4017120743 conn(0x7f6f680fc500 msgr2=0x7f6f680fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:09.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.862+0000 7f6f6dd27700 1 -- 172.21.15.52:0/4017120743 shutdown_connections 2024-04-02T13:44:09.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.862+0000 7f6f6dd27700 1 -- 172.21.15.52:0/4017120743 wait complete. 2024-04-02T13:44:09.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.862+0000 7f6f6dd27700 1 Processor -- start 2024-04-02T13:44:09.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.862+0000 7f6f6dd27700 1 -- start start 2024-04-02T13:44:09.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.863+0000 7f6f6dd27700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f68100f10 0x7f6f6806aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:09.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.863+0000 7f6f6dd27700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6f68106f50 con 0x7f6f68100f10 2024-04-02T13:44:09.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.863+0000 7f6f677fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f68100f10 0x7f6f6806aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:09.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.863+0000 7f6f677fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f68100f10 0x7f6f6806aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53540/0 (socket says 172.21.15.52:53540) 2024-04-02T13:44:09.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.863+0000 7f6f677fe700 1 -- 172.21.15.52:0/2145782254 learned_addr learned my addr 172.21.15.52:0/2145782254 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:44:09.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.863+0000 7f6f677fe700 1 -- 172.21.15.52:0/2145782254 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6819a570 con 0x7f6f68100f10 2024-04-02T13:44:09.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.863+0000 7f6f677fe700 1 --2- 172.21.15.52:0/2145782254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f68100f10 0x7f6f6806aea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009fd0 tx=0x7f6f58005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:09.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.863+0000 7f6f64ff9700 1 -- 172.21.15.52:0/2145782254 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f58027070 con 0x7f6f68100f10 2024-04-02T13:44:09.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.863+0000 7f6f64ff9700 1 -- 172.21.15.52:0/2145782254 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f5800be40 con 0x7f6f68100f10 2024-04-02T13:44:09.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.863+0000 7f6f6dd27700 1 -- 172.21.15.52:0/2145782254 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f6819a740 con 0x7f6f68100f10 2024-04-02T13:44:09.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.863+0000 7f6f64ff9700 1 -- 172.21.15.52:0/2145782254 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f5800f460 con 0x7f6f68100f10 2024-04-02T13:44:09.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.863+0000 7f6f6dd27700 1 -- 172.21.15.52:0/2145782254 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f6819aae0 con 0x7f6f68100f10 2024-04-02T13:44:09.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.864+0000 7f6f64ff9700 1 -- 172.21.15.52:0/2145782254 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6f5802a070 con 0x7f6f68100f10 2024-04-02T13:44:09.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.864+0000 7f6f6dd27700 1 -- 172.21.15.52:0/2145782254 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f681076d0 con 0x7f6f68100f10 2024-04-02T13:44:09.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.865+0000 7f6f64ff9700 1 --2- 172.21.15.52:0/2145782254 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f6f500396b0 0x7f6f5003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:09.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.865+0000 7f6f64ff9700 1 -- 172.21.15.52:0/2145782254 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6f5801e030 con 0x7f6f68100f10 2024-04-02T13:44:09.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.865+0000 7f6f66ffd700 1 --2- 172.21.15.52:0/2145782254 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f6f500396b0 0x7f6f5003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:09.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.865+0000 7f6f66ffd700 1 --2- 172.21.15.52:0/2145782254 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f6f500396b0 0x7f6f5003bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c006fd0 tx=0x7f6f5c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:09.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:09.870+0000 7f6f64ff9700 1 -- 172.21.15.52:0/2145782254 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6f58013070 con 0x7f6f68100f10 2024-04-02T13:44:10.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:10.080+0000 7f6f6dd27700 1 -- 172.21.15.52:0/2145782254 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f6f6802dee0 con 0x7f6f500396b0 2024-04-02T13:44:10.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:10.083+0000 7f6f64ff9700 1 -- 172.21.15.52:0/2145782254 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6f6802dee0 con 0x7f6f500396b0 2024-04-02T13:44:10.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:10.085+0000 7f6f6dd27700 1 -- 172.21.15.52:0/2145782254 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f6f500396b0 msgr2=0x7f6f5003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:10.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:10.085+0000 7f6f6dd27700 1 --2- 172.21.15.52:0/2145782254 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f6f500396b0 0x7f6f5003bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c006fd0 tx=0x7f6f5c006e40 comp rx=0 tx=0).stop 2024-04-02T13:44:10.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:10.085+0000 7f6f6dd27700 1 -- 172.21.15.52:0/2145782254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f68100f10 msgr2=0x7f6f6806aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:10.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:10.085+0000 7f6f6dd27700 1 --2- 172.21.15.52:0/2145782254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f68100f10 0x7f6f6806aea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009fd0 tx=0x7f6f58005050 comp rx=0 tx=0).stop 2024-04-02T13:44:10.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:10.085+0000 7f6f6dd27700 1 -- 172.21.15.52:0/2145782254 shutdown_connections 2024-04-02T13:44:10.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:10.085+0000 7f6f6dd27700 1 --2- 172.21.15.52:0/2145782254 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f6f500396b0 0x7f6f5003bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:10.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:10.085+0000 7f6f6dd27700 1 --2- 172.21.15.52:0/2145782254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f68100f10 0x7f6f6806aea0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:10.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:10.085+0000 7f6f6dd27700 1 -- 172.21.15.52:0/2145782254 >> 172.21.15.52:0/2145782254 conn(0x7f6f680fc500 msgr2=0x7f6f680fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:10.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:10.086+0000 7f6f6dd27700 1 -- 172.21.15.52:0/2145782254 shutdown_connections 2024-04-02T13:44:10.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:10.086+0000 7f6f6dd27700 1 -- 172.21.15.52:0/2145782254 wait complete. 2024-04-02T13:44:10.876 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi088 2024-04-02T13:44:10.877 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-02T13:44:10.877 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/etc/ceph/ceph.conf 2024-04-02T13:44:10.895 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-02T13:44:10.895 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:44:10.952 INFO:tasks.cephadm:Adding host smithi088 to orchestrator... 2024-04-02T13:44:10.953 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph orch host add smithi088 2024-04-02T13:44:11.190 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:44:11.311 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:11 smithi052 ceph-mon[32104]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:44:11.311 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:11.311 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:44:11.311 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:11.311 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:44:11.311 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:11 smithi052 ceph-mon[32104]: Updating smithi052:/etc/ceph/ceph.conf 2024-04-02T13:44:11.311 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:11 smithi052 ceph-mon[32104]: Updating smithi052:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:44:11.312 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:11 smithi052 ceph-mon[32104]: Updating smithi052:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:44:11.312 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:11.312 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:11.312 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:12.258 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:12 smithi052 ceph-mon[32104]: Updating smithi052:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:44:12.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.705+0000 7f2dedb0d700 1 -- 172.21.15.52:0/3066610626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2de8102230 msgr2=0x7f2de8102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:12.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.705+0000 7f2dedb0d700 1 --2- 172.21.15.52:0/3066610626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2de8102230 0x7f2de8102610 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800b0a0 tx=0x7f2dd800b3b0 comp rx=0 tx=0).stop 2024-04-02T13:44:12.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.705+0000 7f2dedb0d700 1 -- 172.21.15.52:0/3066610626 shutdown_connections 2024-04-02T13:44:12.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.705+0000 7f2dedb0d700 1 --2- 172.21.15.52:0/3066610626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2de8102230 0x7f2de8102610 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:12.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.705+0000 7f2dedb0d700 1 -- 172.21.15.52:0/3066610626 >> 172.21.15.52:0/3066610626 conn(0x7f2de80fdb00 msgr2=0x7f2de80fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:12.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.705+0000 7f2dedb0d700 1 -- 172.21.15.52:0/3066610626 shutdown_connections 2024-04-02T13:44:12.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.705+0000 7f2dedb0d700 1 -- 172.21.15.52:0/3066610626 wait complete. 2024-04-02T13:44:12.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.705+0000 7f2dedb0d700 1 Processor -- start 2024-04-02T13:44:12.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.706+0000 7f2dedb0d700 1 -- start start 2024-04-02T13:44:12.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.706+0000 7f2dedb0d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2de8102230 0x7f2de80748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:12.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.706+0000 7f2dedb0d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2de810c600 con 0x7f2de8102230 2024-04-02T13:44:12.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.706+0000 7f2de77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2de8102230 0x7f2de80748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:12.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.706+0000 7f2de77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2de8102230 0x7f2de80748f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52866/0 (socket says 172.21.15.52:52866) 2024-04-02T13:44:12.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.706+0000 7f2de77fe700 1 -- 172.21.15.52:0/1752787513 learned_addr learned my addr 172.21.15.52:0/1752787513 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:44:12.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.706+0000 7f2de77fe700 1 -- 172.21.15.52:0/1752787513 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de80767e0 con 0x7f2de8102230 2024-04-02T13:44:12.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.706+0000 7f2de77fe700 1 --2- 172.21.15.52:0/1752787513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2de8102230 0x7f2de80748f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2dd80093e0 tx=0x7f2dd8003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:12.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.706+0000 7f2de4ff9700 1 -- 172.21.15.52:0/1752787513 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dd8017040 con 0x7f2de8102230 2024-04-02T13:44:12.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.707+0000 7f2de4ff9700 1 -- 172.21.15.52:0/1752787513 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dd8022070 con 0x7f2de8102230 2024-04-02T13:44:12.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.707+0000 7f2de4ff9700 1 -- 172.21.15.52:0/1752787513 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dd801d440 con 0x7f2de8102230 2024-04-02T13:44:12.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.707+0000 7f2dedb0d700 1 -- 172.21.15.52:0/1752787513 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de8074e90 con 0x7f2de8102230 2024-04-02T13:44:12.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.707+0000 7f2dedb0d700 1 -- 172.21.15.52:0/1752787513 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de8075350 con 0x7f2de8102230 2024-04-02T13:44:12.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.707+0000 7f2de4ff9700 1 -- 172.21.15.52:0/1752787513 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2dd8004000 con 0x7f2de8102230 2024-04-02T13:44:12.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.708+0000 7f2dedb0d700 1 -- 172.21.15.52:0/1752787513 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2de8103710 con 0x7f2de8102230 2024-04-02T13:44:12.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.708+0000 7f2de4ff9700 1 --2- 172.21.15.52:0/1752787513 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2dd40396a0 0x7f2dd403bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:12.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.708+0000 7f2de4ff9700 1 -- 172.21.15.52:0/1752787513 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2dd800f050 con 0x7f2de8102230 2024-04-02T13:44:12.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.708+0000 7f2de6ffd700 1 --2- 172.21.15.52:0/1752787513 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2dd40396a0 0x7f2dd403bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:12.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.708+0000 7f2de6ffd700 1 --2- 172.21.15.52:0/1752787513 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2dd40396a0 0x7f2dd403bb60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0006fd0 tx=0x7f2dd0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:12.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.713+0000 7f2de4ff9700 1 -- 172.21.15.52:0/1752787513 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2dd8007e20 con 0x7f2de8102230 2024-04-02T13:44:12.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:12.923+0000 7f2dedb0d700 1 -- 172.21.15.52:0/1752787513 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi088", "target": ["mon-mgr", ""]}) -- 0x7f2de8075e20 con 0x7f2dd40396a0 2024-04-02T13:44:14.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:13 smithi052 ceph-mon[32104]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi088", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:44:15.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:14 smithi052 ceph-mon[32104]: Deploying cephadm binary to smithi088 2024-04-02T13:44:16.044 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:15 smithi052 ceph-mon[32104]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:17.331 INFO:teuthology.orchestra.run.smithi052.stdout:Added host 'smithi088' with addr '172.21.15.88' 2024-04-02T13:44:17.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:17.327+0000 7f2de4ff9700 1 -- 172.21.15.52:0/1752787513 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f2de8075e20 con 0x7f2dd40396a0 2024-04-02T13:44:17.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:17.330+0000 7f2dedb0d700 1 -- 172.21.15.52:0/1752787513 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2dd40396a0 msgr2=0x7f2dd403bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:17.330+0000 7f2dedb0d700 1 --2- 172.21.15.52:0/1752787513 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2dd40396a0 0x7f2dd403bb60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0006fd0 tx=0x7f2dd0006e40 comp rx=0 tx=0).stop 2024-04-02T13:44:17.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:17.330+0000 7f2dedb0d700 1 -- 172.21.15.52:0/1752787513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2de8102230 msgr2=0x7f2de80748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:17.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:17.330+0000 7f2dedb0d700 1 --2- 172.21.15.52:0/1752787513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2de8102230 0x7f2de80748f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2dd80093e0 tx=0x7f2dd8003ce0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:17.330+0000 7f2dedb0d700 1 -- 172.21.15.52:0/1752787513 shutdown_connections 2024-04-02T13:44:17.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:17.330+0000 7f2dedb0d700 1 --2- 172.21.15.52:0/1752787513 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2dd40396a0 0x7f2dd403bb60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:17.330+0000 7f2dedb0d700 1 --2- 172.21.15.52:0/1752787513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2de8102230 0x7f2de80748f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:17.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:17.330+0000 7f2dedb0d700 1 -- 172.21.15.52:0/1752787513 >> 172.21.15.52:0/1752787513 conn(0x7f2de80fdb00 msgr2=0x7f2de8103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:17.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:17.330+0000 7f2dedb0d700 1 -- 172.21.15.52:0/1752787513 shutdown_connections 2024-04-02T13:44:17.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:17.330+0000 7f2dedb0d700 1 -- 172.21.15.52:0/1752787513 wait complete. 2024-04-02T13:44:17.929 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph orch host ls --format=json 2024-04-02T13:44:18.171 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:44:18.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:18 smithi052 ceph-mon[32104]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:18.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:18 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:18.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:18 smithi052 ceph-mon[32104]: Added host smithi088 2024-04-02T13:44:18.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:18 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:44:18.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:18 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:19.500 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:19 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:19.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.724+0000 7f05fb70e700 1 -- 172.21.15.52:0/2824065482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05f4101160 msgr2=0x7f05f4101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:19.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.724+0000 7f05fb70e700 1 --2- 172.21.15.52:0/2824065482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05f4101160 0x7f05f4101540 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009910 tx=0x7f05e4009c20 comp rx=0 tx=0).stop 2024-04-02T13:44:19.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.724+0000 7f05fb70e700 1 -- 172.21.15.52:0/2824065482 shutdown_connections 2024-04-02T13:44:19.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.724+0000 7f05fb70e700 1 --2- 172.21.15.52:0/2824065482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05f4101160 0x7f05f4101540 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.724+0000 7f05fb70e700 1 -- 172.21.15.52:0/2824065482 >> 172.21.15.52:0/2824065482 conn(0x7f05f40fca30 msgr2=0x7f05f40fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:19.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.724+0000 7f05fb70e700 1 -- 172.21.15.52:0/2824065482 shutdown_connections 2024-04-02T13:44:19.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.724+0000 7f05fb70e700 1 -- 172.21.15.52:0/2824065482 wait complete. 2024-04-02T13:44:19.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.725+0000 7f05fb70e700 1 Processor -- start 2024-04-02T13:44:19.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.725+0000 7f05fb70e700 1 -- start start 2024-04-02T13:44:19.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.725+0000 7f05fb70e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05f4101160 0x7f05f419d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.725+0000 7f05fb70e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f05f410b530 con 0x7f05f4101160 2024-04-02T13:44:19.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.725+0000 7f05f94aa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05f4101160 0x7f05f419d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.725+0000 7f05f94aa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05f4101160 0x7f05f419d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52884/0 (socket says 172.21.15.52:52884) 2024-04-02T13:44:19.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.725+0000 7f05f94aa700 1 -- 172.21.15.52:0/2679580701 learned_addr learned my addr 172.21.15.52:0/2679580701 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:44:19.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.726+0000 7f05f94aa700 1 -- 172.21.15.52:0/2679580701 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f419dc80 con 0x7f05f4101160 2024-04-02T13:44:19.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.726+0000 7f05f94aa700 1 --2- 172.21.15.52:0/2679580701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05f4101160 0x7f05f419d740 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f05e4005850 tx=0x7f05e4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:19.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.726+0000 7f05ea7fc700 1 -- 172.21.15.52:0/2679580701 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e4026070 con 0x7f05f4101160 2024-04-02T13:44:19.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.726+0000 7f05fb70e700 1 -- 172.21.15.52:0/2679580701 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f419df70 con 0x7f05f4101160 2024-04-02T13:44:19.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.726+0000 7f05ea7fc700 1 -- 172.21.15.52:0/2679580701 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05e400bbd0 con 0x7f05f4101160 2024-04-02T13:44:19.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.726+0000 7f05ea7fc700 1 -- 172.21.15.52:0/2679580701 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05e4003970 con 0x7f05f4101160 2024-04-02T13:44:19.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.726+0000 7f05fb70e700 1 -- 172.21.15.52:0/2679580701 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f4197900 con 0x7f05f4101160 2024-04-02T13:44:19.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.727+0000 7f05ea7fc700 1 -- 172.21.15.52:0/2679580701 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f05e4029070 con 0x7f05f4101160 2024-04-02T13:44:19.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.727+0000 7f05ea7fc700 1 --2- 172.21.15.52:0/2679580701 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f05e0039650 0x7f05e003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:19.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.727+0000 7f05ea7fc700 1 -- 172.21.15.52:0/2679580701 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f05e401e070 con 0x7f05f4101160 2024-04-02T13:44:19.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.727+0000 7f05fb70e700 1 -- 172.21.15.52:0/2679580701 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f4102640 con 0x7f05f4101160 2024-04-02T13:44:19.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.727+0000 7f05f8ca9700 1 --2- 172.21.15.52:0/2679580701 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f05e0039650 0x7f05e003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:19.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.727+0000 7f05f8ca9700 1 --2- 172.21.15.52:0/2679580701 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f05e0039650 0x7f05e003bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f05f0006fd0 tx=0x7f05f0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:19.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.732+0000 7f05ea7fc700 1 -- 172.21.15.52:0/2679580701 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f05e4030070 con 0x7f05f4101160 2024-04-02T13:44:19.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.950+0000 7f05fb70e700 1 -- 172.21.15.52:0/2679580701 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f05f4198e30 con 0x7f05e0039650 2024-04-02T13:44:19.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.951+0000 7f05ea7fc700 1 -- 172.21.15.52:0/2679580701 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f05f4198e30 con 0x7f05e0039650 2024-04-02T13:44:19.954 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:44:19.954 INFO:teuthology.orchestra.run.smithi052.stdout:[{"addr": "172.21.15.52", "hostname": "smithi052", "labels": [], "status": ""}, {"addr": "172.21.15.88", "hostname": "smithi088", "labels": [], "status": ""}] 2024-04-02T13:44:19.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.953+0000 7f05fb70e700 1 -- 172.21.15.52:0/2679580701 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f05e0039650 msgr2=0x7f05e003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:19.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.953+0000 7f05fb70e700 1 --2- 172.21.15.52:0/2679580701 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f05e0039650 0x7f05e003bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f05f0006fd0 tx=0x7f05f0006e40 comp rx=0 tx=0).stop 2024-04-02T13:44:19.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.953+0000 7f05fb70e700 1 -- 172.21.15.52:0/2679580701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05f4101160 msgr2=0x7f05f419d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:19.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.953+0000 7f05fb70e700 1 --2- 172.21.15.52:0/2679580701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05f4101160 0x7f05f419d740 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f05e4005850 tx=0x7f05e4005030 comp rx=0 tx=0).stop 2024-04-02T13:44:19.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.953+0000 7f05fb70e700 1 -- 172.21.15.52:0/2679580701 shutdown_connections 2024-04-02T13:44:19.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.953+0000 7f05fb70e700 1 --2- 172.21.15.52:0/2679580701 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f05e0039650 0x7f05e003bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.953+0000 7f05fb70e700 1 --2- 172.21.15.52:0/2679580701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05f4101160 0x7f05f419d740 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:19.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.953+0000 7f05fb70e700 1 -- 172.21.15.52:0/2679580701 >> 172.21.15.52:0/2679580701 conn(0x7f05f40fca30 msgr2=0x7f05f406a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:19.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.953+0000 7f05fb70e700 1 -- 172.21.15.52:0/2679580701 shutdown_connections 2024-04-02T13:44:19.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:19.953+0000 7f05fb70e700 1 -- 172.21.15.52:0/2679580701 wait complete. 2024-04-02T13:44:20.543 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:20 smithi052 ceph-mon[32104]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:20.544 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:20 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:20.573 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi189 2024-04-02T13:44:20.573 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-02T13:44:20.574 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.conf 2024-04-02T13:44:20.592 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-02T13:44:20.593 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:44:20.650 INFO:tasks.cephadm:Adding host smithi189 to orchestrator... 2024-04-02T13:44:20.650 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph orch host add smithi189 2024-04-02T13:44:20.897 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:44:21.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:21 smithi052 ceph-mon[32104]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:44:22.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.375+0000 7faf56dbf700 1 -- 172.21.15.52:0/2660949140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf50102230 msgr2=0x7faf50102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:22.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.375+0000 7faf56dbf700 1 --2- 172.21.15.52:0/2660949140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf50102230 0x7faf50102610 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faf4800b0a0 tx=0x7faf4800b3b0 comp rx=0 tx=0).stop 2024-04-02T13:44:22.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.375+0000 7faf56dbf700 1 -- 172.21.15.52:0/2660949140 shutdown_connections 2024-04-02T13:44:22.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.375+0000 7faf56dbf700 1 --2- 172.21.15.52:0/2660949140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf50102230 0x7faf50102610 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:22.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.375+0000 7faf56dbf700 1 -- 172.21.15.52:0/2660949140 >> 172.21.15.52:0/2660949140 conn(0x7faf500fdb00 msgr2=0x7faf500fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:22.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.375+0000 7faf56dbf700 1 -- 172.21.15.52:0/2660949140 shutdown_connections 2024-04-02T13:44:22.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.375+0000 7faf56dbf700 1 -- 172.21.15.52:0/2660949140 wait complete. 2024-04-02T13:44:22.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.375+0000 7faf56dbf700 1 Processor -- start 2024-04-02T13:44:22.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.375+0000 7faf56dbf700 1 -- start start 2024-04-02T13:44:22.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.376+0000 7faf56dbf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf50102230 0x7faf500748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:22.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.376+0000 7faf56dbf700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faf5010c600 con 0x7faf50102230 2024-04-02T13:44:22.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.376+0000 7faf54b5b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf50102230 0x7faf500748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:22.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.376+0000 7faf54b5b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf50102230 0x7faf500748f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33804/0 (socket says 172.21.15.52:33804) 2024-04-02T13:44:22.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.376+0000 7faf54b5b700 1 -- 172.21.15.52:0/3641750169 learned_addr learned my addr 172.21.15.52:0/3641750169 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:44:22.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.376+0000 7faf54b5b700 1 -- 172.21.15.52:0/3641750169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf500767e0 con 0x7faf50102230 2024-04-02T13:44:22.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.376+0000 7faf54b5b700 1 --2- 172.21.15.52:0/3641750169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf50102230 0x7faf500748f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faf48015040 tx=0x7faf48003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:22.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.376+0000 7faf4dffb700 1 -- 172.21.15.52:0/3641750169 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf48017040 con 0x7faf50102230 2024-04-02T13:44:22.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.376+0000 7faf4dffb700 1 -- 172.21.15.52:0/3641750169 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf48022070 con 0x7faf50102230 2024-04-02T13:44:22.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.376+0000 7faf56dbf700 1 -- 172.21.15.52:0/3641750169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf50074e90 con 0x7faf50102230 2024-04-02T13:44:22.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.376+0000 7faf4dffb700 1 -- 172.21.15.52:0/3641750169 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf4801d5d0 con 0x7faf50102230 2024-04-02T13:44:22.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.377+0000 7faf56dbf700 1 -- 172.21.15.52:0/3641750169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf50075350 con 0x7faf50102230 2024-04-02T13:44:22.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.378+0000 7faf4dffb700 1 -- 172.21.15.52:0/3641750169 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7faf4801d880 con 0x7faf50102230 2024-04-02T13:44:22.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.378+0000 7faf56dbf700 1 -- 172.21.15.52:0/3641750169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf50103710 con 0x7faf50102230 2024-04-02T13:44:22.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.378+0000 7faf4dffb700 1 --2- 172.21.15.52:0/3641750169 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faf380396b0 0x7faf3803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:22.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.378+0000 7faf4dffb700 1 -- 172.21.15.52:0/3641750169 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faf4800f050 con 0x7faf50102230 2024-04-02T13:44:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.378+0000 7faf4ffff700 1 --2- 172.21.15.52:0/3641750169 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faf380396b0 0x7faf3803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.378+0000 7faf4ffff700 1 --2- 172.21.15.52:0/3641750169 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faf380396b0 0x7faf3803bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faf44006fd0 tx=0x7faf44006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.383+0000 7faf4dffb700 1 -- 172.21.15.52:0/3641750169 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7faf48007e20 con 0x7faf50102230 2024-04-02T13:44:22.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:22 smithi052 ceph-mon[32104]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:22.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:22.594+0000 7faf56dbf700 1 -- 172.21.15.52:0/3641750169 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}) -- 0x7faf50075e20 con 0x7faf380396b0 2024-04-02T13:44:23.440 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:23 smithi052 ceph-mon[32104]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:44:23.441 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:23 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:23.441 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:23 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:23.441 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:23 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:23.441 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:23 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:23.441 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:23 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi088", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:44:23.441 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:23 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:23.441 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:23 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:44:24.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:24 smithi052 ceph-mon[32104]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:24.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:24 smithi052 ceph-mon[32104]: Updating smithi088:/etc/ceph/ceph.conf 2024-04-02T13:44:24.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:24 smithi052 ceph-mon[32104]: Deploying cephadm binary to smithi189 2024-04-02T13:44:24.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:24 smithi052 ceph-mon[32104]: Updating smithi088:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:44:25.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:25 smithi052 ceph-mon[32104]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:44:26.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:26 smithi052 ceph-mon[32104]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:26.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:26.890+0000 7faf4dffb700 1 -- 172.21.15.52:0/3641750169 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7faf50075e20 con 0x7faf380396b0 2024-04-02T13:44:26.895 INFO:teuthology.orchestra.run.smithi052.stdout:Added host 'smithi189' with addr '172.21.15.189' 2024-04-02T13:44:26.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:26.893+0000 7faf56dbf700 1 -- 172.21.15.52:0/3641750169 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faf380396b0 msgr2=0x7faf3803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:26.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:26.893+0000 7faf56dbf700 1 --2- 172.21.15.52:0/3641750169 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faf380396b0 0x7faf3803bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faf44006fd0 tx=0x7faf44006e40 comp rx=0 tx=0).stop 2024-04-02T13:44:26.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:26.893+0000 7faf56dbf700 1 -- 172.21.15.52:0/3641750169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf50102230 msgr2=0x7faf500748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:26.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:26.893+0000 7faf56dbf700 1 --2- 172.21.15.52:0/3641750169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf50102230 0x7faf500748f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faf48015040 tx=0x7faf48003d50 comp rx=0 tx=0).stop 2024-04-02T13:44:26.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:26.893+0000 7faf56dbf700 1 -- 172.21.15.52:0/3641750169 shutdown_connections 2024-04-02T13:44:26.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:26.893+0000 7faf56dbf700 1 --2- 172.21.15.52:0/3641750169 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faf380396b0 0x7faf3803bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:26.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:26.893+0000 7faf56dbf700 1 --2- 172.21.15.52:0/3641750169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf50102230 0x7faf500748f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:26.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:26.893+0000 7faf56dbf700 1 -- 172.21.15.52:0/3641750169 >> 172.21.15.52:0/3641750169 conn(0x7faf500fdb00 msgr2=0x7faf50103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:26.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:26.893+0000 7faf56dbf700 1 -- 172.21.15.52:0/3641750169 shutdown_connections 2024-04-02T13:44:26.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:26.893+0000 7faf56dbf700 1 -- 172.21.15.52:0/3641750169 wait complete. 2024-04-02T13:44:27.493 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph orch host ls --format=json 2024-04-02T13:44:27.737 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:44:27.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:27 smithi052 ceph-mon[32104]: Updating smithi088:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:44:27.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:27.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:27.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:27.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:27.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:44:27.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:28.812 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:28 smithi052 ceph-mon[32104]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:28.812 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:28 smithi052 ceph-mon[32104]: Added host smithi189 2024-04-02T13:44:28.812 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:28 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:29.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.300+0000 7f5f6ca80700 1 -- 172.21.15.52:0/55111027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f68106fd0 msgr2=0x7f5f681073b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:29.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.300+0000 7f5f6ca80700 1 --2- 172.21.15.52:0/55111027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f68106fd0 0x7f5f681073b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009910 tx=0x7f5f58009c20 comp rx=0 tx=0).stop 2024-04-02T13:44:29.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.300+0000 7f5f6ca80700 1 -- 172.21.15.52:0/55111027 shutdown_connections 2024-04-02T13:44:29.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.300+0000 7f5f6ca80700 1 --2- 172.21.15.52:0/55111027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f68106fd0 0x7f5f681073b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:29.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.300+0000 7f5f6ca80700 1 -- 172.21.15.52:0/55111027 >> 172.21.15.52:0/55111027 conn(0x7f5f680fc140 msgr2=0x7f5f680fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:29.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.300+0000 7f5f6ca80700 1 -- 172.21.15.52:0/55111027 shutdown_connections 2024-04-02T13:44:29.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.300+0000 7f5f6ca80700 1 -- 172.21.15.52:0/55111027 wait complete. 2024-04-02T13:44:29.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.301+0000 7f5f6ca80700 1 Processor -- start 2024-04-02T13:44:29.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.301+0000 7f5f6ca80700 1 -- start start 2024-04-02T13:44:29.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.301+0000 7f5f6ca80700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f68106fd0 0x7f5f681016e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:29.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.301+0000 7f5f6ca80700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6810c750 con 0x7f5f68106fd0 2024-04-02T13:44:29.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.301+0000 7f5f6659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f68106fd0 0x7f5f681016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:29.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.301+0000 7f5f6659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f68106fd0 0x7f5f681016e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33824/0 (socket says 172.21.15.52:33824) 2024-04-02T13:44:29.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.301+0000 7f5f6659c700 1 -- 172.21.15.52:0/3295974158 learned_addr learned my addr 172.21.15.52:0/3295974158 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:44:29.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.301+0000 7f5f6659c700 1 -- 172.21.15.52:0/3295974158 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f681035d0 con 0x7f5f68106fd0 2024-04-02T13:44:29.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.302+0000 7f5f6659c700 1 --2- 172.21.15.52:0/3295974158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f68106fd0 0x7f5f681016e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5f58005850 tx=0x7f5f58005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:29.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.302+0000 7f5f5f7fe700 1 -- 172.21.15.52:0/3295974158 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f58026070 con 0x7f5f68106fd0 2024-04-02T13:44:29.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.302+0000 7f5f6ca80700 1 -- 172.21.15.52:0/3295974158 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f68101c20 con 0x7f5f68106fd0 2024-04-02T13:44:29.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.302+0000 7f5f5f7fe700 1 -- 172.21.15.52:0/3295974158 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f5800bc30 con 0x7f5f68106fd0 2024-04-02T13:44:29.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.302+0000 7f5f5f7fe700 1 -- 172.21.15.52:0/3295974158 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f58003a20 con 0x7f5f68106fd0 2024-04-02T13:44:29.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.302+0000 7f5f6ca80700 1 -- 172.21.15.52:0/3295974158 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f681020e0 con 0x7f5f68106fd0 2024-04-02T13:44:29.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.303+0000 7f5f5f7fe700 1 -- 172.21.15.52:0/3295974158 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5f58029070 con 0x7f5f68106fd0 2024-04-02T13:44:29.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.303+0000 7f5f5f7fe700 1 --2- 172.21.15.52:0/3295974158 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f5f54039650 0x7f5f5403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:29.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.303+0000 7f5f5f7fe700 1 -- 172.21.15.52:0/3295974158 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5f5801e070 con 0x7f5f68106fd0 2024-04-02T13:44:29.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.303+0000 7f5f65d9b700 1 --2- 172.21.15.52:0/3295974158 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f5f54039650 0x7f5f5403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:29.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.303+0000 7f5f6ca80700 1 -- 172.21.15.52:0/3295974158 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f68063c20 con 0x7f5f68106fd0 2024-04-02T13:44:29.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.304+0000 7f5f65d9b700 1 --2- 172.21.15.52:0/3295974158 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f5f54039650 0x7f5f5403bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5f50006fd0 tx=0x7f5f50006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:29.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.308+0000 7f5f5f7fe700 1 -- 172.21.15.52:0/3295974158 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5f58030020 con 0x7f5f68106fd0 2024-04-02T13:44:29.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.526+0000 7f5f6ca80700 1 -- 172.21.15.52:0/3295974158 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5f68102c10 con 0x7f5f54039650 2024-04-02T13:44:29.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.527+0000 7f5f5f7fe700 1 -- 172.21.15.52:0/3295974158 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f5f68102c10 con 0x7f5f54039650 2024-04-02T13:44:29.530 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:44:29.530 INFO:teuthology.orchestra.run.smithi052.stdout:[{"addr": "172.21.15.52", "hostname": "smithi052", "labels": [], "status": ""}, {"addr": "172.21.15.88", "hostname": "smithi088", "labels": [], "status": ""}, {"addr": "172.21.15.189", "hostname": "smithi189", "labels": [], "status": ""}] 2024-04-02T13:44:29.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.529+0000 7f5f6ca80700 1 -- 172.21.15.52:0/3295974158 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f5f54039650 msgr2=0x7f5f5403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:29.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.529+0000 7f5f6ca80700 1 --2- 172.21.15.52:0/3295974158 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f5f54039650 0x7f5f5403bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5f50006fd0 tx=0x7f5f50006e40 comp rx=0 tx=0).stop 2024-04-02T13:44:29.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.529+0000 7f5f6ca80700 1 -- 172.21.15.52:0/3295974158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f68106fd0 msgr2=0x7f5f681016e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:29.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.529+0000 7f5f6ca80700 1 --2- 172.21.15.52:0/3295974158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f68106fd0 0x7f5f681016e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5f58005850 tx=0x7f5f58005030 comp rx=0 tx=0).stop 2024-04-02T13:44:29.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.529+0000 7f5f6ca80700 1 -- 172.21.15.52:0/3295974158 shutdown_connections 2024-04-02T13:44:29.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.529+0000 7f5f6ca80700 1 --2- 172.21.15.52:0/3295974158 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f5f54039650 0x7f5f5403bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:29.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.529+0000 7f5f6ca80700 1 --2- 172.21.15.52:0/3295974158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f68106fd0 0x7f5f681016e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:29.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.529+0000 7f5f6ca80700 1 -- 172.21.15.52:0/3295974158 >> 172.21.15.52:0/3295974158 conn(0x7f5f680fc140 msgr2=0x7f5f68109ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:29.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.529+0000 7f5f6ca80700 1 -- 172.21.15.52:0/3295974158 shutdown_connections 2024-04-02T13:44:29.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:29.529+0000 7f5f6ca80700 1 -- 172.21.15.52:0/3295974158 wait complete. 2024-04-02T13:44:30.183 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-02T13:44:30.184 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph osd crush tunables default 2024-04-02T13:44:30.425 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:44:30.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:30 smithi052 ceph-mon[32104]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:30.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:30 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:30.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:30 smithi052 ceph-mon[32104]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:44:31.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.955+0000 7f8dfb3a3700 1 -- 172.21.15.52:0/2968767235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8df4102230 msgr2=0x7f8df4102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.955+0000 7f8dfb3a3700 1 --2- 172.21.15.52:0/2968767235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8df4102230 0x7f8df4102610 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8df0009910 tx=0x7f8df0009c20 comp rx=0 tx=0).stop 2024-04-02T13:44:31.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.955+0000 7f8dfb3a3700 1 -- 172.21.15.52:0/2968767235 shutdown_connections 2024-04-02T13:44:31.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.955+0000 7f8dfb3a3700 1 --2- 172.21.15.52:0/2968767235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8df4102230 0x7f8df4102610 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.955+0000 7f8dfb3a3700 1 -- 172.21.15.52:0/2968767235 >> 172.21.15.52:0/2968767235 conn(0x7f8df40fdb00 msgr2=0x7f8df40fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.956+0000 7f8dfb3a3700 1 -- 172.21.15.52:0/2968767235 shutdown_connections 2024-04-02T13:44:31.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.956+0000 7f8dfb3a3700 1 -- 172.21.15.52:0/2968767235 wait complete. 2024-04-02T13:44:31.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.956+0000 7f8dfb3a3700 1 Processor -- start 2024-04-02T13:44:31.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.956+0000 7f8dfb3a3700 1 -- start start 2024-04-02T13:44:31.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.956+0000 7f8dfb3a3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8df4102230 0x7f8df419d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.956+0000 7f8dfb3a3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8df410c600 con 0x7f8df4102230 2024-04-02T13:44:31.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.956+0000 7f8df913f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8df4102230 0x7f8df419d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.956+0000 7f8df913f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8df4102230 0x7f8df419d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33672/0 (socket says 172.21.15.52:33672) 2024-04-02T13:44:31.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.957+0000 7f8df913f700 1 -- 172.21.15.52:0/2269823775 learned_addr learned my addr 172.21.15.52:0/2269823775 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:44:31.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.957+0000 7f8df913f700 1 -- 172.21.15.52:0/2269823775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df419dd50 con 0x7f8df4102230 2024-04-02T13:44:31.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.957+0000 7f8df913f700 1 --2- 172.21.15.52:0/2269823775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8df4102230 0x7f8df419d810 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8df0005850 tx=0x7f8df0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.957+0000 7f8de67fc700 1 -- 172.21.15.52:0/2269823775 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8df0026070 con 0x7f8df4102230 2024-04-02T13:44:31.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.957+0000 7f8de67fc700 1 -- 172.21.15.52:0/2269823775 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8df000bbd0 con 0x7f8df4102230 2024-04-02T13:44:31.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.957+0000 7f8dfb3a3700 1 -- 172.21.15.52:0/2269823775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8df419e040 con 0x7f8df4102230 2024-04-02T13:44:31.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.957+0000 7f8de67fc700 1 -- 172.21.15.52:0/2269823775 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8df0003970 con 0x7f8df4102230 2024-04-02T13:44:31.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.957+0000 7f8dfb3a3700 1 -- 172.21.15.52:0/2269823775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8df41979d0 con 0x7f8df4102230 2024-04-02T13:44:31.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.958+0000 7f8de67fc700 1 -- 172.21.15.52:0/2269823775 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8df0029070 con 0x7f8df4102230 2024-04-02T13:44:31.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.958+0000 7f8dfb3a3700 1 -- 172.21.15.52:0/2269823775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8df4103710 con 0x7f8df4102230 2024-04-02T13:44:31.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.958+0000 7f8de67fc700 1 --2- 172.21.15.52:0/2269823775 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f8de0039650 0x7f8de003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.958+0000 7f8de67fc700 1 -- 172.21.15.52:0/2269823775 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8df001e070 con 0x7f8df4102230 2024-04-02T13:44:31.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.958+0000 7f8df893e700 1 --2- 172.21.15.52:0/2269823775 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f8de0039650 0x7f8de003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.959+0000 7f8df893e700 1 --2- 172.21.15.52:0/2269823775 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f8de0039650 0x7f8de003bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8de8006fd0 tx=0x7f8de8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:31.963+0000 7f8de67fc700 1 -- 172.21.15.52:0/2269823775 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8df0013070 con 0x7f8df4102230 2024-04-02T13:44:32.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:32.177+0000 7f8dfb3a3700 1 -- 172.21.15.52:0/2269823775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f8df4198f00 con 0x7f8df4102230 2024-04-02T13:44:32.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:32.512+0000 7f8de67fc700 1 -- 172.21.15.52:0/2269823775 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f8df0030090 con 0x7f8df4102230 2024-04-02T13:44:32.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:32.515+0000 7f8dfb3a3700 1 -- 172.21.15.52:0/2269823775 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f8de0039650 msgr2=0x7f8de003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:32.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:32.515+0000 7f8dfb3a3700 1 --2- 172.21.15.52:0/2269823775 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f8de0039650 0x7f8de003bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8de8006fd0 tx=0x7f8de8006e40 comp rx=0 tx=0).stop 2024-04-02T13:44:32.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:32.515+0000 7f8dfb3a3700 1 -- 172.21.15.52:0/2269823775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8df4102230 msgr2=0x7f8df419d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:32.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:32.515+0000 7f8dfb3a3700 1 --2- 172.21.15.52:0/2269823775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8df4102230 0x7f8df419d810 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8df0005850 tx=0x7f8df0005030 comp rx=0 tx=0).stop 2024-04-02T13:44:32.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:32.516+0000 7f8dfb3a3700 1 -- 172.21.15.52:0/2269823775 shutdown_connections 2024-04-02T13:44:32.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:32.516+0000 7f8dfb3a3700 1 --2- 172.21.15.52:0/2269823775 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f8de0039650 0x7f8de003bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:32.516+0000 7f8dfb3a3700 1 --2- 172.21.15.52:0/2269823775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8df4102230 0x7f8df419d810 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:32.516+0000 7f8dfb3a3700 1 -- 172.21.15.52:0/2269823775 >> 172.21.15.52:0/2269823775 conn(0x7f8df40fdb00 msgr2=0x7f8df4105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:32.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:32.516+0000 7f8dfb3a3700 1 -- 172.21.15.52:0/2269823775 shutdown_connections 2024-04-02T13:44:32.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:32.516+0000 7f8dfb3a3700 1 -- 172.21.15.52:0/2269823775 wait complete. 2024-04-02T13:44:32.522 INFO:teuthology.orchestra.run.smithi052.stderr:adjusted tunables profile to default 2024-04-02T13:44:32.830 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:32 smithi052 ceph-mon[32104]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:32.830 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:32 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2269823775' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-02T13:44:33.151 INFO:tasks.cephadm:Adding mon.a on smithi052 2024-04-02T13:44:33.151 INFO:tasks.cephadm:Adding mon.b on smithi088 2024-04-02T13:44:33.151 INFO:tasks.cephadm:Adding mon.c on smithi189 2024-04-02T13:44:33.152 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph orch apply mon '3;smithi052:172.21.15.52=a;smithi088:172.21.15.88=b;smithi189:172.21.15.189=c' 2024-04-02T13:44:33.388 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:44:33.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:33 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2269823775' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-02T13:44:33.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:33 smithi052 ceph-mon[32104]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:44:33.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:33 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:33.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:33 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:33.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:33 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:33.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:33 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:33.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:33 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:44:33.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:33 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:33.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:33 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:44:34.834 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.830+0000 7fe342e5f700 1 -- 172.21.15.189:0/2979600996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe33c102230 msgr2=0x7fe33c102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:34.835 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.830+0000 7fe342e5f700 1 --2- 172.21.15.189:0/2979600996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe33c102230 0x7fe33c102610 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe338009910 tx=0x7fe338009c20 comp rx=0 tx=0).stop 2024-04-02T13:44:34.835 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.830+0000 7fe342e5f700 1 -- 172.21.15.189:0/2979600996 shutdown_connections 2024-04-02T13:44:34.835 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.830+0000 7fe342e5f700 1 --2- 172.21.15.189:0/2979600996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe33c102230 0x7fe33c102610 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:34.835 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.830+0000 7fe342e5f700 1 -- 172.21.15.189:0/2979600996 >> 172.21.15.189:0/2979600996 conn(0x7fe33c0fdb00 msgr2=0x7fe33c0fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:34.835 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.830+0000 7fe342e5f700 1 -- 172.21.15.189:0/2979600996 shutdown_connections 2024-04-02T13:44:34.835 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.830+0000 7fe342e5f700 1 -- 172.21.15.189:0/2979600996 wait complete. 2024-04-02T13:44:34.835 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.831+0000 7fe342e5f700 1 Processor -- start 2024-04-02T13:44:34.835 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.831+0000 7fe342e5f700 1 -- start start 2024-04-02T13:44:34.835 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.831+0000 7fe342e5f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe33c102230 0x7fe33c0748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:34.835 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.831+0000 7fe342e5f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c10c600 con 0x7fe33c102230 2024-04-02T13:44:34.835 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.832+0000 7fe340bfb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe33c102230 0x7fe33c0748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:34.835 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.832+0000 7fe340bfb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe33c102230 0x7fe33c0748f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.189:60312/0 (socket says 172.21.15.189:60312) 2024-04-02T13:44:34.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.832+0000 7fe340bfb700 1 -- 172.21.15.189:0/4294555623 learned_addr learned my addr 172.21.15.189:0/4294555623 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-02T13:44:34.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.832+0000 7fe340bfb700 1 -- 172.21.15.189:0/4294555623 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe33c0767e0 con 0x7fe33c102230 2024-04-02T13:44:34.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.832+0000 7fe340bfb700 1 --2- 172.21.15.189:0/4294555623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe33c102230 0x7fe33c0748f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe338005850 tx=0x7fe338005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:34.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.832+0000 7fe331ffb700 1 -- 172.21.15.189:0/4294555623 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe338026070 con 0x7fe33c102230 2024-04-02T13:44:34.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.832+0000 7fe342e5f700 1 -- 172.21.15.189:0/4294555623 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe33c074e90 con 0x7fe33c102230 2024-04-02T13:44:34.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.832+0000 7fe331ffb700 1 -- 172.21.15.189:0/4294555623 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe33800bc30 con 0x7fe33c102230 2024-04-02T13:44:34.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.832+0000 7fe331ffb700 1 -- 172.21.15.189:0/4294555623 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe338003a20 con 0x7fe33c102230 2024-04-02T13:44:34.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.833+0000 7fe342e5f700 1 -- 172.21.15.189:0/4294555623 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe33c075350 con 0x7fe33c102230 2024-04-02T13:44:34.838 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.833+0000 7fe342e5f700 1 -- 172.21.15.189:0/4294555623 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe33c103710 con 0x7fe33c102230 2024-04-02T13:44:34.838 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.834+0000 7fe331ffb700 1 -- 172.21.15.189:0/4294555623 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe338029070 con 0x7fe33c102230 2024-04-02T13:44:34.838 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.834+0000 7fe331ffb700 1 --2- 172.21.15.189:0/4294555623 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe324039650 0x7fe32403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:34.838 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.834+0000 7fe331ffb700 1 -- 172.21.15.189:0/4294555623 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe33801e070 con 0x7fe33c102230 2024-04-02T13:44:34.838 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.834+0000 7fe333fff700 1 --2- 172.21.15.189:0/4294555623 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe324039650 0x7fe32403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:34.838 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.835+0000 7fe333fff700 1 --2- 172.21.15.189:0/4294555623 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe324039650 0x7fe32403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe32c006fd0 tx=0x7fe32c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:34.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:34 smithi052 ceph-mon[32104]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:34.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:34 smithi052 ceph-mon[32104]: Updating smithi189:/etc/ceph/ceph.conf 2024-04-02T13:44:34.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:34 smithi052 ceph-mon[32104]: Updating smithi189:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:44:34.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:34 smithi052 ceph-mon[32104]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:44:34.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:34 smithi052 ceph-mon[32104]: Updating smithi189:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:44:34.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:34 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:34.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:34 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:34.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:34 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:34.842 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:34.839+0000 7fe331ffb700 1 -- 172.21.15.189:0/4294555623 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe338013070 con 0x7fe33c102230 2024-04-02T13:44:35.053 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:35.050+0000 7fe342e5f700 1 -- 172.21.15.189:0/4294555623 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi052:172.21.15.52=a;smithi088:172.21.15.88=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}) -- 0x7fe33c075e20 con 0x7fe324039650 2024-04-02T13:44:35.059 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:35.057+0000 7fe331ffb700 1 -- 172.21.15.189:0/4294555623 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe33c075e20 con 0x7fe324039650 2024-04-02T13:44:35.060 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mon update... 2024-04-02T13:44:35.062 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:35.059+0000 7fe342e5f700 1 -- 172.21.15.189:0/4294555623 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe324039650 msgr2=0x7fe32403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:35.062 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:35.059+0000 7fe342e5f700 1 --2- 172.21.15.189:0/4294555623 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe324039650 0x7fe32403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe32c006fd0 tx=0x7fe32c006e40 comp rx=0 tx=0).stop 2024-04-02T13:44:35.062 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:35.059+0000 7fe342e5f700 1 -- 172.21.15.189:0/4294555623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe33c102230 msgr2=0x7fe33c0748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:35.062 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:35.059+0000 7fe342e5f700 1 --2- 172.21.15.189:0/4294555623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe33c102230 0x7fe33c0748f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe338005850 tx=0x7fe338005030 comp rx=0 tx=0).stop 2024-04-02T13:44:35.062 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:35.059+0000 7fe342e5f700 1 -- 172.21.15.189:0/4294555623 shutdown_connections 2024-04-02T13:44:35.062 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:35.059+0000 7fe342e5f700 1 --2- 172.21.15.189:0/4294555623 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe324039650 0x7fe32403bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:35.062 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:35.059+0000 7fe342e5f700 1 --2- 172.21.15.189:0/4294555623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe33c102230 0x7fe33c0748f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:35.063 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:35.059+0000 7fe342e5f700 1 -- 172.21.15.189:0/4294555623 >> 172.21.15.189:0/4294555623 conn(0x7fe33c0fdb00 msgr2=0x7fe33c103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:35.063 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:35.059+0000 7fe342e5f700 1 -- 172.21.15.189:0/4294555623 shutdown_connections 2024-04-02T13:44:35.063 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:35.059+0000 7fe342e5f700 1 -- 172.21.15.189:0/4294555623 wait complete. 2024-04-02T13:44:35.629 DEBUG:teuthology.orchestra.run.smithi088:mon.b> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mon.b.service 2024-04-02T13:44:35.632 DEBUG:teuthology.orchestra.run.smithi189:mon.c> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mon.c.service 2024-04-02T13:44:35.635 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-02T13:44:35.635 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T13:44:36.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:36 smithi052 ceph-mon[32104]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:36.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:36 smithi052 ceph-mon[32104]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi052:172.21.15.52=a;smithi088:172.21.15.88=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:44:36.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:36 smithi052 ceph-mon[32104]: Saving service mon spec with placement smithi052:172.21.15.52=a;smithi088:172.21.15.88=b;smithi189:172.21.15.189=c;count:3 2024-04-02T13:44:36.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:36 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:36.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:36 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:44:36.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:36 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:36.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:36 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:44:36.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:36 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:36.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:36 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:44:36.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:36 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:36.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:36 smithi052 ceph-mon[32104]: Deploying daemon mon.c on smithi189 2024-04-02T13:44:37.406 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:44:38.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:38 smithi052 ceph-mon[32104]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:39.544 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.539+0000 7fe084033700 1 -- 172.21.15.189:0/3831184565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe07c102230 msgr2=0x7fe07c102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:39.545 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.539+0000 7fe084033700 1 --2- 172.21.15.189:0/3831184565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe07c102230 0x7fe07c102610 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe07400b0a0 tx=0x7fe07400b3b0 comp rx=0 tx=0).stop 2024-04-02T13:44:39.545 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.539+0000 7fe084033700 1 -- 172.21.15.189:0/3831184565 shutdown_connections 2024-04-02T13:44:39.545 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.539+0000 7fe084033700 1 --2- 172.21.15.189:0/3831184565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe07c102230 0x7fe07c102610 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:39.545 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.539+0000 7fe084033700 1 -- 172.21.15.189:0/3831184565 >> 172.21.15.189:0/3831184565 conn(0x7fe07c0fdae0 msgr2=0x7fe07c0fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:39.545 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.539+0000 7fe084033700 1 -- 172.21.15.189:0/3831184565 shutdown_connections 2024-04-02T13:44:39.545 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.539+0000 7fe084033700 1 -- 172.21.15.189:0/3831184565 wait complete. 2024-04-02T13:44:39.545 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.540+0000 7fe084033700 1 Processor -- start 2024-04-02T13:44:39.545 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.540+0000 7fe084033700 1 -- start start 2024-04-02T13:44:39.545 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.540+0000 7fe084033700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe07c102230 0x7fe07c19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:39.545 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.540+0000 7fe084033700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe07c10c600 con 0x7fe07c102230 2024-04-02T13:44:39.545 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.540+0000 7fe081dcf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe07c102230 0x7fe07c19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:39.546 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.540+0000 7fe081dcf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe07c102230 0x7fe07c19d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.189:36746/0 (socket says 172.21.15.189:36746) 2024-04-02T13:44:39.546 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.540+0000 7fe081dcf700 1 -- 172.21.15.189:0/2773224711 learned_addr learned my addr 172.21.15.189:0/2773224711 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-02T13:44:39.546 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.541+0000 7fe081dcf700 1 -- 172.21.15.189:0/2773224711 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe07c19dbe0 con 0x7fe07c102230 2024-04-02T13:44:39.546 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.541+0000 7fe081dcf700 1 --2- 172.21.15.189:0/2773224711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe07c102230 0x7fe07c19d6a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe074015040 tx=0x7fe074003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:39.546 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.541+0000 7fe07affd700 1 -- 172.21.15.189:0/2773224711 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe074017040 con 0x7fe07c102230 2024-04-02T13:44:39.546 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.541+0000 7fe084033700 1 -- 172.21.15.189:0/2773224711 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe07c19de70 con 0x7fe07c102230 2024-04-02T13:44:39.546 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.541+0000 7fe07affd700 1 -- 172.21.15.189:0/2773224711 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe074022070 con 0x7fe07c102230 2024-04-02T13:44:39.546 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.541+0000 7fe07affd700 1 -- 172.21.15.189:0/2773224711 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe07401d5d0 con 0x7fe07c102230 2024-04-02T13:44:39.546 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.541+0000 7fe084033700 1 -- 172.21.15.189:0/2773224711 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe07c197860 con 0x7fe07c102230 2024-04-02T13:44:39.547 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.542+0000 7fe07affd700 1 -- 172.21.15.189:0/2773224711 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe074004000 con 0x7fe07c102230 2024-04-02T13:44:39.547 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.542+0000 7fe084033700 1 -- 172.21.15.189:0/2773224711 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe07c103710 con 0x7fe07c102230 2024-04-02T13:44:39.547 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.542+0000 7fe07affd700 1 --2- 172.21.15.189:0/2773224711 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe064039650 0x7fe06403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:39.547 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.542+0000 7fe07affd700 1 -- 172.21.15.189:0/2773224711 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe07400f050 con 0x7fe07c102230 2024-04-02T13:44:39.547 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.542+0000 7fe0815ce700 1 --2- 172.21.15.189:0/2773224711 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe064039650 0x7fe06403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:39.547 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.543+0000 7fe0815ce700 1 --2- 172.21.15.189:0/2773224711 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe064039650 0x7fe06403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe070006fd0 tx=0x7fe070006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:39.550 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.548+0000 7fe07affd700 1 -- 172.21.15.189:0/2773224711 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe074054080 con 0x7fe07c102230 2024-04-02T13:44:39.844 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.841+0000 7fe084033700 1 -- 172.21.15.189:0/2773224711 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe07c198d90 con 0x7fe07c102230 2024-04-02T13:44:39.845 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.842+0000 7fe07affd700 1 -- 172.21.15.189:0/2773224711 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fe074007e20 con 0x7fe07c102230 2024-04-02T13:44:39.845 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-02T13:44:39.845 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":1,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","modified":"2024-04-02T13:42:54.002665Z","created":"2024-04-02T13:42:54.002665Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:3300","nonce":0},{"type":"v1","addr":"172.21.15.52:6789","nonce":0}]},"addr":"172.21.15.52:6789/0","public_addr":"172.21.15.52:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T13:44:39.847 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.844+0000 7fe084033700 1 -- 172.21.15.189:0/2773224711 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe064039650 msgr2=0x7fe06403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:39.847 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.844+0000 7fe084033700 1 --2- 172.21.15.189:0/2773224711 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe064039650 0x7fe06403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe070006fd0 tx=0x7fe070006e40 comp rx=0 tx=0).stop 2024-04-02T13:44:39.847 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.844+0000 7fe084033700 1 -- 172.21.15.189:0/2773224711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe07c102230 msgr2=0x7fe07c19d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:39.847 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.844+0000 7fe084033700 1 --2- 172.21.15.189:0/2773224711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe07c102230 0x7fe07c19d6a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe074015040 tx=0x7fe074003d50 comp rx=0 tx=0).stop 2024-04-02T13:44:39.847 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.844+0000 7fe084033700 1 -- 172.21.15.189:0/2773224711 shutdown_connections 2024-04-02T13:44:39.847 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.844+0000 7fe084033700 1 --2- 172.21.15.189:0/2773224711 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe064039650 0x7fe06403bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:39.847 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.844+0000 7fe084033700 1 --2- 172.21.15.189:0/2773224711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe07c102230 0x7fe07c19d6a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:39.847 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.844+0000 7fe084033700 1 -- 172.21.15.189:0/2773224711 >> 172.21.15.189:0/2773224711 conn(0x7fe07c0fdae0 msgr2=0x7fe07c103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:39.847 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.844+0000 7fe084033700 1 -- 172.21.15.189:0/2773224711 shutdown_connections 2024-04-02T13:44:39.847 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:39.844+0000 7fe084033700 1 -- 172.21.15.189:0/2773224711 wait complete. 2024-04-02T13:44:39.848 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 1 2024-04-02T13:44:41.550 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-02T13:44:41.551 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T13:44:41.787 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:44:45.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:45 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:45.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:44:45.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:44:45.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: mon.a calling monitor election 2024-04-02T13:44:45.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:45.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:44:45.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:44:45.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: mon.c calling monitor election 2024-04-02T13:44:45.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:45.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:44:45.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:44:45.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:44:45.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-02T13:44:45.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: monmap epoch 2 2024-04-02T13:44:45.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:44:45.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: last_changed 2024-04-02T13:44:39.933747+0000 2024-04-02T13:44:45.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: created 2024-04-02T13:42:54.002665+0000 2024-04-02T13:44:45.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: min_mon_release 19 (squid) 2024-04-02T13:44:45.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: election_strategy: 1 2024-04-02T13:44:45.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-04-02T13:44:45.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-04-02T13:44:45.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: fsmap 2024-04-02T13:44:45.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:44:45.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: mgrmap e13: x(active, since 50s) 2024-04-02T13:44:45.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: overall HEALTH_OK 2024-04-02T13:44:45.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:45.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:45.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:45.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:44:45.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:45 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:46.252 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.248+0000 7f5b33fcd700 1 -- 172.21.15.189:0/579949336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b180035c0 msgr2=0x7f5b18005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:46.252 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.248+0000 7f5b33fcd700 1 --2- 172.21.15.189:0/579949336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b180035c0 0x7f5b18005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c102000 tx=0x7f5b1c00feb0 comp rx=0 tx=0).stop 2024-04-02T13:44:46.252 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.249+0000 7f5b33fcd700 1 -- 172.21.15.189:0/579949336 shutdown_connections 2024-04-02T13:44:46.252 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.249+0000 7f5b33fcd700 1 --2- 172.21.15.189:0/579949336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b180035c0 0x7f5b18005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:46.252 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.249+0000 7f5b33fcd700 1 -- 172.21.15.189:0/579949336 >> 172.21.15.189:0/579949336 conn(0x7f5b2c0fdbc0 msgr2=0x7f5b2c0fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:46.252 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:46 smithi189 ceph-mon[36683]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:46.252 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:46 smithi189 ceph-mon[36683]: Deploying daemon mon.b on smithi088 2024-04-02T13:44:46.252 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:46 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:44:46.255 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.249+0000 7f5b33fcd700 1 -- 172.21.15.189:0/579949336 shutdown_connections 2024-04-02T13:44:46.255 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.249+0000 7f5b33fcd700 1 -- 172.21.15.189:0/579949336 wait complete. 2024-04-02T13:44:46.255 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.250+0000 7f5b33fcd700 1 Processor -- start 2024-04-02T13:44:46.255 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.250+0000 7f5b33fcd700 1 -- start start 2024-04-02T13:44:46.255 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.250+0000 7f5b33fcd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b180035c0 0x7f5b2c19d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:46.255 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.250+0000 7f5b33fcd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b2c19de40 0x7f5b2c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:46.255 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.250+0000 7f5b33fcd700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c10a160 con 0x7f5b180035c0 2024-04-02T13:44:46.255 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.250+0000 7f5b33fcd700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c109fe0 con 0x7f5b2c19de40 2024-04-02T13:44:46.255 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.250+0000 7f5b31568700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b2c19de40 0x7f5b2c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:46.255 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.250+0000 7f5b31568700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b2c19de40 0x7f5b2c197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:40802/0 (socket says 172.21.15.189:40802) 2024-04-02T13:44:46.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.250+0000 7f5b31568700 1 -- 172.21.15.189:0/2805904445 learned_addr learned my addr 172.21.15.189:0/2805904445 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-02T13:44:46.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.250+0000 7f5b31d69700 1 --2- 172.21.15.189:0/2805904445 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b180035c0 0x7f5b2c19d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:46.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.250+0000 7f5b31568700 1 -- 172.21.15.189:0/2805904445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b2c19de40 msgr2=0x7f5b2c197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-02T13:44:46.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.250+0000 7f5b31568700 1 -- 172.21.15.189:0/2805904445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b2c19de40 msgr2=0x7f5b2c197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-02T13:44:46.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.250+0000 7f5b31568700 1 --2- 172.21.15.189:0/2805904445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b2c19de40 0x7f5b2c197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T13:44:46.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.250+0000 7f5b31568700 1 --2- 172.21.15.189:0/2805904445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b2c19de40 0x7f5b2c197ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:44:46.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.251+0000 7f5b31d69700 1 -- 172.21.15.189:0/2805904445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b2c19de40 msgr2=0x7f5b2c197ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:44:46.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.251+0000 7f5b31d69700 1 --2- 172.21.15.189:0/2805904445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b2c19de40 0x7f5b2c197ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:46.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.251+0000 7f5b31d69700 1 -- 172.21.15.189:0/2805904445 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b2c19e480 con 0x7f5b180035c0 2024-04-02T13:44:46.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.251+0000 7f5b31d69700 1 --2- 172.21.15.189:0/2805904445 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b180035c0 0x7f5b2c19d900 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c0059d0 tx=0x7f5b1c0057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:46.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.251+0000 7f5b22ffd700 1 -- 172.21.15.189:0/2805904445 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f5b1c004b20 con 0x7f5b180035c0 2024-04-02T13:44:46.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.251+0000 7f5b22ffd700 1 -- 172.21.15.189:0/2805904445 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b1c013e40 con 0x7f5b180035c0 2024-04-02T13:44:46.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.251+0000 7f5b33fcd700 1 -- 172.21.15.189:0/2805904445 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b2c198000 con 0x7f5b180035c0 2024-04-02T13:44:46.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.251+0000 7f5b22ffd700 1 -- 172.21.15.189:0/2805904445 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f5b1c011cd0 con 0x7f5b180035c0 2024-04-02T13:44:46.257 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.251+0000 7f5b33fcd700 1 -- 172.21.15.189:0/2805904445 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b2c198590 con 0x7f5b180035c0 2024-04-02T13:44:46.257 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.252+0000 7f5b22ffd700 1 -- 172.21.15.189:0/2805904445 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5b1c023070 con 0x7f5b180035c0 2024-04-02T13:44:46.257 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.252+0000 7f5b22ffd700 1 --2- 172.21.15.189:0/2805904445 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f5b100393f0 0x7f5b1003b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:46.258 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.252+0000 7f5b33fcd700 1 -- 172.21.15.189:0/2805904445 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b2c103810 con 0x7f5b180035c0 2024-04-02T13:44:46.258 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.253+0000 7f5b22ffd700 1 -- 172.21.15.189:0/2805904445 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5b1c020050 con 0x7f5b180035c0 2024-04-02T13:44:46.258 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.253+0000 7f5b31568700 1 --2- 172.21.15.189:0/2805904445 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f5b100393f0 0x7f5b1003b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:46.258 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.253+0000 7f5b31568700 1 --2- 172.21.15.189:0/2805904445 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f5b100393f0 0x7f5b1003b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5b28009180 tx=0x7f5b28008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:46.261 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.258+0000 7f5b22ffd700 1 -- 172.21.15.189:0/2805904445 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5b1c031020 con 0x7f5b180035c0 2024-04-02T13:44:46.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:46 smithi052 ceph-mon[32104]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:46.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:46 smithi052 ceph-mon[32104]: Deploying daemon mon.b on smithi088 2024-04-02T13:44:46.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:46 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:44:46.551 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.548+0000 7f5b33fcd700 1 -- 172.21.15.189:0/2805904445 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f5b2c198ff0 con 0x7f5b180035c0 2024-04-02T13:44:46.552 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.549+0000 7f5b22ffd700 1 -- 172.21.15.189:0/2805904445 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f5b1c028030 con 0x7f5b180035c0 2024-04-02T13:44:46.553 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-02T13:44:46.553 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":2,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","modified":"2024-04-02T13:44:39.933747Z","created":"2024-04-02T13:42:54.002665Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:3300","nonce":0},{"type":"v1","addr":"172.21.15.52:6789","nonce":0}]},"addr":"172.21.15.52:6789/0","public_addr":"172.21.15.52:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-02T13:44:46.554 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.552+0000 7f5b33fcd700 1 -- 172.21.15.189:0/2805904445 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f5b100393f0 msgr2=0x7f5b1003b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:46.555 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.552+0000 7f5b33fcd700 1 --2- 172.21.15.189:0/2805904445 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f5b100393f0 0x7f5b1003b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5b28009180 tx=0x7f5b28008040 comp rx=0 tx=0).stop 2024-04-02T13:44:46.555 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.552+0000 7f5b33fcd700 1 -- 172.21.15.189:0/2805904445 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b180035c0 msgr2=0x7f5b2c19d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:46.555 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.552+0000 7f5b33fcd700 1 --2- 172.21.15.189:0/2805904445 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b180035c0 0x7f5b2c19d900 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c0059d0 tx=0x7f5b1c0057d0 comp rx=0 tx=0).stop 2024-04-02T13:44:46.555 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.552+0000 7f5b33fcd700 1 -- 172.21.15.189:0/2805904445 shutdown_connections 2024-04-02T13:44:46.555 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.552+0000 7f5b33fcd700 1 --2- 172.21.15.189:0/2805904445 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f5b100393f0 0x7f5b1003b8b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:46.555 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.552+0000 7f5b33fcd700 1 --2- 172.21.15.189:0/2805904445 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b180035c0 0x7f5b2c19d900 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:46.555 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.552+0000 7f5b33fcd700 1 --2- 172.21.15.189:0/2805904445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b2c19de40 0x7f5b2c197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:46.555 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.552+0000 7f5b33fcd700 1 -- 172.21.15.189:0/2805904445 >> 172.21.15.189:0/2805904445 conn(0x7f5b2c0fdbc0 msgr2=0x7f5b2c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:46.555 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.552+0000 7f5b33fcd700 1 -- 172.21.15.189:0/2805904445 shutdown_connections 2024-04-02T13:44:46.555 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:46.552+0000 7f5b33fcd700 1 -- 172.21.15.189:0/2805904445 wait complete. 2024-04-02T13:44:46.556 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 2 2024-04-02T13:44:47.119 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:47 smithi189 ceph-mon[36683]: from='client.? 172.21.15.189:0/2805904445' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T13:44:47.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:47 smithi052 ceph-mon[32104]: from='client.? 172.21.15.189:0/2805904445' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T13:44:48.121 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-02T13:44:48.122 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T13:44:48.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:48 smithi189 ceph-mon[36683]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:48.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:48 smithi052 ceph-mon[32104]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:48.373 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:44:48.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 podman[36509]: 2024-04-02 13:44:48.242997461 +0000 UTC m=+0.017508012 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:44:48.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 podman[36509]: 2024-04-02 13:44:48.549577727 +0000 UTC m=+0.324088273 container init 9e66ff83e3f91ec2ad55240248270b19efa411f900cf66bcffde5c481ea87ea5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:44:48.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 podman[36509]: 2024-04-02 13:44:48.55263284 +0000 UTC m=+0.327143382 container start 9e66ff83e3f91ec2ad55240248270b19efa411f900cf66bcffde5c481ea87ea5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-b, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T13:44:48.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: set uid:gid to 167:167 (ceph:ceph) 2024-04-02T13:44:48.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev), process ceph-mon, pid 2 2024-04-02T13:44:48.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: pidfile_write: ignore empty --pid-file 2024-04-02T13:44:48.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: load: jerasure load: lrc 2024-04-02T13:44:48.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: RocksDB version: 7.9.2 2024-04-02T13:44:48.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Git sha 0 2024-04-02T13:44:48.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Compile date 2024-04-02 01:02:34 2024-04-02T13:44:48.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: DB SUMMARY 2024-04-02T13:44:48.923 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: DB Session ID: LFSVZN0OKJK89RGHZRJC 2024-04-02T13:44:48.923 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: CURRENT file: CURRENT 2024-04-02T13:44:48.923 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: IDENTITY file: IDENTITY 2024-04-02T13:44:48.923 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-02T13:44:48.923 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-02T13:44:48.923 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-02T13:44:48.923 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.error_if_exists: 0 2024-04-02T13:44:48.924 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.create_if_missing: 0 2024-04-02T13:44:48.924 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.paranoid_checks: 1 2024-04-02T13:44:48.924 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-02T13:44:48.924 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-02T13:44:48.924 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-02T13:44:48.924 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.env: 0x5591c9594c60 2024-04-02T13:44:48.925 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.fs: PosixFileSystem 2024-04-02T13:44:48.925 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.info_log: 0x5591cabadae0 2024-04-02T13:44:48.925 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_file_opening_threads: 16 2024-04-02T13:44:48.925 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.statistics: (nil) 2024-04-02T13:44:48.925 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.use_fsync: 0 2024-04-02T13:44:48.925 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_log_file_size: 0 2024-04-02T13:44:48.926 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-02T13:44:48.926 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-02T13:44:48.926 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.keep_log_file_num: 1000 2024-04-02T13:44:48.926 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.recycle_log_file_num: 0 2024-04-02T13:44:48.926 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.allow_fallocate: 1 2024-04-02T13:44:48.927 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.allow_mmap_reads: 0 2024-04-02T13:44:48.927 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.allow_mmap_writes: 0 2024-04-02T13:44:48.927 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.use_direct_reads: 0 2024-04-02T13:44:48.927 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-02T13:44:48.927 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.create_missing_column_families: 0 2024-04-02T13:44:48.927 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.db_log_dir: 2024-04-02T13:44:48.927 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.wal_dir: 2024-04-02T13:44:48.928 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-02T13:44:48.928 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-02T13:44:48.928 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-02T13:44:48.928 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-02T13:44:48.928 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-02T13:44:48.928 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-02T13:44:48.929 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.advise_random_on_open: 1 2024-04-02T13:44:48.929 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.db_write_buffer_size: 0 2024-04-02T13:44:48.929 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.write_buffer_manager: 0x5591cab7e1e0 2024-04-02T13:44:48.929 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-02T13:44:48.929 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-02T13:44:48.929 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-02T13:44:48.930 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.rate_limiter: (nil) 2024-04-02T13:44:48.930 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-02T13:44:48.930 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.wal_recovery_mode: 2 2024-04-02T13:44:48.930 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.enable_thread_tracking: 0 2024-04-02T13:44:48.930 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.enable_pipelined_write: 0 2024-04-02T13:44:48.930 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.unordered_write: 0 2024-04-02T13:44:48.931 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-02T13:44:48.931 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-02T13:44:48.931 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-02T13:44:48.931 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-02T13:44:48.931 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.row_cache: None 2024-04-02T13:44:48.931 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.wal_filter: None 2024-04-02T13:44:48.932 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-02T13:44:48.934 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.allow_ingest_behind: 0 2024-04-02T13:44:48.934 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.two_write_queues: 0 2024-04-02T13:44:48.934 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.manual_wal_flush: 0 2024-04-02T13:44:48.934 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.wal_compression: 0 2024-04-02T13:44:48.934 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.atomic_flush: 0 2024-04-02T13:44:48.935 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-02T13:44:48.935 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-02T13:44:48.935 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-02T13:44:48.935 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.log_readahead_size: 0 2024-04-02T13:44:48.935 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-02T13:44:48.935 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.best_efforts_recovery: 0 2024-04-02T13:44:48.935 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-02T13:44:48.935 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-02T13:44:48.935 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.allow_data_in_errors: 0 2024-04-02T13:44:48.935 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.db_host_id: __hostname__ 2024-04-02T13:44:48.935 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-02T13:44:48.935 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_background_jobs: 2 2024-04-02T13:44:48.935 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_background_compactions: -1 2024-04-02T13:44:48.936 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_subcompactions: 1 2024-04-02T13:44:48.936 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-02T13:44:48.936 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-02T13:44:48.936 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-02T13:44:48.936 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_total_wal_size: 0 2024-04-02T13:44:48.936 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-02T13:44:48.936 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-02T13:44:48.936 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-02T13:44:48.936 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-02T13:44:48.936 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_open_files: -1 2024-04-02T13:44:48.936 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.bytes_per_sync: 0 2024-04-02T13:44:48.936 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-02T13:44:48.936 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-02T13:44:48.937 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.compaction_readahead_size: 0 2024-04-02T13:44:48.937 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_background_flushes: -1 2024-04-02T13:44:48.937 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Compression algorithms supported: 2024-04-02T13:44:48.937 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-02T13:44:48.937 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: kZSTD supported: 0 2024-04-02T13:44:48.937 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: kXpressCompression supported: 0 2024-04-02T13:44:48.937 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: kLZ4HCCompression supported: 1 2024-04-02T13:44:48.937 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: kZlibCompression supported: 1 2024-04-02T13:44:48.937 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: kSnappyCompression supported: 1 2024-04-02T13:44:48.937 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: kLZ4Compression supported: 1 2024-04-02T13:44:48.937 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: kBZip2Compression supported: 0 2024-04-02T13:44:48.937 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-02T13:44:48.937 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-02T13:44:48.938 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-02T13:44:48.938 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-02T13:44:48.938 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-02T13:44:48.938 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.merge_operator: 2024-04-02T13:44:48.938 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.compaction_filter: None 2024-04-02T13:44:48.938 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.compaction_filter_factory: None 2024-04-02T13:44:48.938 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.sst_partitioner_factory: None 2024-04-02T13:44:48.938 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-02T13:44:48.938 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-02T13:44:48.938 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5591cabadcc0) 2024-04-02T13:44:48.938 INFO:journalctl@ceph.mon.b.smithi088.stdout: cache_index_and_filter_blocks: 1 2024-04-02T13:44:48.938 INFO:journalctl@ceph.mon.b.smithi088.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-02T13:44:48.939 INFO:journalctl@ceph.mon.b.smithi088.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-02T13:44:48.939 INFO:journalctl@ceph.mon.b.smithi088.stdout: pin_top_level_index_and_filter: 1 2024-04-02T13:44:48.939 INFO:journalctl@ceph.mon.b.smithi088.stdout: index_type: 0 2024-04-02T13:44:48.939 INFO:journalctl@ceph.mon.b.smithi088.stdout: data_block_index_type: 0 2024-04-02T13:44:48.939 INFO:journalctl@ceph.mon.b.smithi088.stdout: index_shortening: 1 2024-04-02T13:44:48.939 INFO:journalctl@ceph.mon.b.smithi088.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-02T13:44:48.939 INFO:journalctl@ceph.mon.b.smithi088.stdout: checksum: 4 2024-04-02T13:44:48.939 INFO:journalctl@ceph.mon.b.smithi088.stdout: no_block_cache: 0 2024-04-02T13:44:48.939 INFO:journalctl@ceph.mon.b.smithi088.stdout: block_cache: 0x5591cabb1090 2024-04-02T13:44:48.939 INFO:journalctl@ceph.mon.b.smithi088.stdout: block_cache_name: BinnedLRUCache 2024-04-02T13:44:48.939 INFO:journalctl@ceph.mon.b.smithi088.stdout: block_cache_options: 2024-04-02T13:44:48.939 INFO:journalctl@ceph.mon.b.smithi088.stdout: capacity : 536870912 2024-04-02T13:44:48.940 INFO:journalctl@ceph.mon.b.smithi088.stdout: num_shard_bits : 4 2024-04-02T13:44:48.940 INFO:journalctl@ceph.mon.b.smithi088.stdout: strict_capacity_limit : 0 2024-04-02T13:44:48.940 INFO:journalctl@ceph.mon.b.smithi088.stdout: high_pri_pool_ratio: 0.000 2024-04-02T13:44:48.940 INFO:journalctl@ceph.mon.b.smithi088.stdout: block_cache_compressed: (nil) 2024-04-02T13:44:48.940 INFO:journalctl@ceph.mon.b.smithi088.stdout: persistent_cache: (nil) 2024-04-02T13:44:48.940 INFO:journalctl@ceph.mon.b.smithi088.stdout: block_size: 4096 2024-04-02T13:44:48.940 INFO:journalctl@ceph.mon.b.smithi088.stdout: block_size_deviation: 10 2024-04-02T13:44:48.940 INFO:journalctl@ceph.mon.b.smithi088.stdout: block_restart_interval: 16 2024-04-02T13:44:48.940 INFO:journalctl@ceph.mon.b.smithi088.stdout: index_block_restart_interval: 1 2024-04-02T13:44:48.940 INFO:journalctl@ceph.mon.b.smithi088.stdout: metadata_block_size: 4096 2024-04-02T13:44:48.940 INFO:journalctl@ceph.mon.b.smithi088.stdout: partition_filters: 0 2024-04-02T13:44:48.940 INFO:journalctl@ceph.mon.b.smithi088.stdout: use_delta_encoding: 1 2024-04-02T13:44:48.941 INFO:journalctl@ceph.mon.b.smithi088.stdout: filter_policy: bloomfilter 2024-04-02T13:44:48.941 INFO:journalctl@ceph.mon.b.smithi088.stdout: whole_key_filtering: 1 2024-04-02T13:44:48.941 INFO:journalctl@ceph.mon.b.smithi088.stdout: verify_compression: 0 2024-04-02T13:44:48.941 INFO:journalctl@ceph.mon.b.smithi088.stdout: read_amp_bytes_per_bit: 0 2024-04-02T13:44:48.941 INFO:journalctl@ceph.mon.b.smithi088.stdout: format_version: 5 2024-04-02T13:44:48.941 INFO:journalctl@ceph.mon.b.smithi088.stdout: enable_index_compression: 1 2024-04-02T13:44:48.941 INFO:journalctl@ceph.mon.b.smithi088.stdout: block_align: 0 2024-04-02T13:44:48.941 INFO:journalctl@ceph.mon.b.smithi088.stdout: max_auto_readahead_size: 262144 2024-04-02T13:44:48.941 INFO:journalctl@ceph.mon.b.smithi088.stdout: prepopulate_block_cache: 0 2024-04-02T13:44:48.941 INFO:journalctl@ceph.mon.b.smithi088.stdout: initial_auto_readahead_size: 8192 2024-04-02T13:44:48.943 INFO:journalctl@ceph.mon.b.smithi088.stdout: num_file_reads_for_auto_readahead: 2 2024-04-02T13:44:48.943 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.write_buffer_size: 33554432 2024-04-02T13:44:48.943 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_write_buffer_number: 2 2024-04-02T13:44:48.943 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.compression: NoCompression 2024-04-02T13:44:48.943 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.bottommost_compression: Disabled 2024-04-02T13:44:48.943 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.prefix_extractor: nullptr 2024-04-02T13:44:48.943 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-02T13:44:48.943 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.num_levels: 7 2024-04-02T13:44:48.943 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-02T13:44:48.943 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-02T13:44:48.944 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-02T13:44:48.944 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-02T13:44:48.944 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-02T13:44:48.944 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-02T13:44:48.944 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-02T13:44:48.944 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-02T13:44:48.944 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-02T13:44:48.944 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-02T13:44:48.944 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-02T13:44:48.944 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-02T13:44:48.944 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-02T13:44:48.944 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.compression_opts.level: 32767 2024-04-02T13:44:48.944 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.compression_opts.strategy: 0 2024-04-02T13:44:48.944 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-02T13:44:48.945 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-02T13:44:48.945 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-02T13:44:48.945 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-02T13:44:48.945 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.compression_opts.enabled: false 2024-04-02T13:44:48.945 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-02T13:44:48.945 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-02T13:44:48.945 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-02T13:44:48.945 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-02T13:44:48.945 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.target_file_size_base: 67108864 2024-04-02T13:44:48.945 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-02T13:44:48.945 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-02T13:44:48.945 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-02T13:44:48.945 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-02T13:44:48.945 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-02T13:44:48.945 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-02T13:44:48.946 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-02T13:44:48.946 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-02T13:44:48.946 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-02T13:44:48.946 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-02T13:44:48.946 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-02T13:44:48.946 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-02T13:44:48.946 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-02T13:44:48.946 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-02T13:44:48.946 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.arena_block_size: 1048576 2024-04-02T13:44:48.946 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-02T13:44:48.946 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-02T13:44:48.946 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.disable_auto_compactions: 0 2024-04-02T13:44:48.946 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-02T13:44:48.946 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-02T13:44:48.946 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-02T13:44:48.947 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-02T13:44:48.947 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-02T13:44:48.947 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-02T13:44:48.947 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-02T13:44:48.947 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-02T13:44:48.947 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-02T13:44:48.947 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-02T13:44:48.947 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-02T13:44:48.947 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.inplace_update_support: 0 2024-04-02T13:44:48.947 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-02T13:44:48.947 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-02T13:44:48.947 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-02T13:44:48.947 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-02T13:44:48.947 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.bloom_locality: 0 2024-04-02T13:44:48.947 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.max_successive_merges: 0 2024-04-02T13:44:48.948 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-02T13:44:48.948 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.paranoid_file_checks: 0 2024-04-02T13:44:48.948 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.force_consistency_checks: 1 2024-04-02T13:44:48.948 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.report_bg_io_stats: 0 2024-04-02T13:44:48.948 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.ttl: 2592000 2024-04-02T13:44:48.948 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-02T13:44:48.948 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-02T13:44:48.948 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-02T13:44:48.948 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.enable_blob_files: false 2024-04-02T13:44:48.948 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.min_blob_size: 0 2024-04-02T13:44:48.948 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.blob_file_size: 268435456 2024-04-02T13:44:48.948 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-02T13:44:48.948 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-02T13:44:48.949 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-02T13:44:48.949 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-02T13:44:48.949 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-02T13:44:48.949 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.blob_file_starting_level: 0 2024-04-02T13:44:48.949 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-02T13:44:48.949 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-02T13:44:48.949 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-02T13:44:48.949 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 257539ac-d0ec-4ef0-9eab-a9cf66fa78e4 2024-04-02T13:44:48.949 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712065488581604, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-02T13:44:48.949 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-02T13:44:48.949 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712065488582030, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712065488, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "257539ac-d0ec-4ef0-9eab-a9cf66fa78e4", "db_session_id": "LFSVZN0OKJK89RGHZRJC", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-02T13:44:48.949 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712065488582109, "job": 1, "event": "recovery_finished"} 2024-04-02T13:44:48.949 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-02T13:44:48.949 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-02T13:44:48.949 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5591caca0000 2024-04-02T13:44:48.950 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: DB pointer 0x5591cac8a000 2024-04-02T13:44:48.950 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-02T13:44:48.950 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-02T13:44:48.950 INFO:journalctl@ceph.mon.b.smithi088.stdout: ** DB Stats ** 2024-04-02T13:44:48.950 INFO:journalctl@ceph.mon.b.smithi088.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T13:44:48.950 INFO:journalctl@ceph.mon.b.smithi088.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-02T13:44:48.950 INFO:journalctl@ceph.mon.b.smithi088.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T13:44:48.950 INFO:journalctl@ceph.mon.b.smithi088.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T13:44:48.950 INFO:journalctl@ceph.mon.b.smithi088.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-02T13:44:48.950 INFO:journalctl@ceph.mon.b.smithi088.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T13:44:48.950 INFO:journalctl@ceph.mon.b.smithi088.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T13:44:48.950 INFO:journalctl@ceph.mon.b.smithi088.stdout: 2024-04-02T13:44:48.950 INFO:journalctl@ceph.mon.b.smithi088.stdout: ** Compaction Stats [default] ** 2024-04-02T13:44:48.950 INFO:journalctl@ceph.mon.b.smithi088.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T13:44:48.951 INFO:journalctl@ceph.mon.b.smithi088.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-02T13:44:48.951 INFO:journalctl@ceph.mon.b.smithi088.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-02T13:44:48.951 INFO:journalctl@ceph.mon.b.smithi088.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-02T13:44:48.951 INFO:journalctl@ceph.mon.b.smithi088.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-02T13:44:48.951 INFO:journalctl@ceph.mon.b.smithi088.stdout: 2024-04-02T13:44:48.951 INFO:journalctl@ceph.mon.b.smithi088.stdout: ** Compaction Stats [default] ** 2024-04-02T13:44:48.951 INFO:journalctl@ceph.mon.b.smithi088.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T13:44:48.951 INFO:journalctl@ceph.mon.b.smithi088.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-02T13:44:48.951 INFO:journalctl@ceph.mon.b.smithi088.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-02T13:44:48.951 INFO:journalctl@ceph.mon.b.smithi088.stdout: 2024-04-02T13:44:48.951 INFO:journalctl@ceph.mon.b.smithi088.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-02T13:44:48.951 INFO:journalctl@ceph.mon.b.smithi088.stdout: 2024-04-02T13:44:48.951 INFO:journalctl@ceph.mon.b.smithi088.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T13:44:48.952 INFO:journalctl@ceph.mon.b.smithi088.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-02T13:44:48.952 INFO:journalctl@ceph.mon.b.smithi088.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-02T13:44:48.952 INFO:journalctl@ceph.mon.b.smithi088.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-02T13:44:48.952 INFO:journalctl@ceph.mon.b.smithi088.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-02T13:44:48.952 INFO:journalctl@ceph.mon.b.smithi088.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-02T13:44:48.952 INFO:journalctl@ceph.mon.b.smithi088.stdout: Cumulative compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T13:44:48.952 INFO:journalctl@ceph.mon.b.smithi088.stdout: Interval compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T13:44:48.952 INFO:journalctl@ceph.mon.b.smithi088.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-02T13:44:48.952 INFO:journalctl@ceph.mon.b.smithi088.stdout: Block cache BinnedLRUCache@0x5591cabb1090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-02T13:44:48.952 INFO:journalctl@ceph.mon.b.smithi088.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-02T13:44:48.952 INFO:journalctl@ceph.mon.b.smithi088.stdout: 2024-04-02T13:44:48.952 INFO:journalctl@ceph.mon.b.smithi088.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-02T13:44:48.952 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-02T13:44:48.952 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: using public_addr v2:172.21.15.88:0/0 -> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] 2024-04-02T13:44:48.952 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: starting mon.b rank -1 at public addrs [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] at bind addrs [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:44:48.953 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: mon.b@-1(???) e0 preinit fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:44:48.953 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: mon.b@-1(synchronizing).mds e1 new map 2024-04-02T13:44:48.953 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-02T13:44:48.953 INFO:journalctl@ceph.mon.b.smithi088.stdout: e1 2024-04-02T13:44:48.953 INFO:journalctl@ceph.mon.b.smithi088.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T13:44:48.953 INFO:journalctl@ceph.mon.b.smithi088.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T13:44:48.953 INFO:journalctl@ceph.mon.b.smithi088.stdout: legacy client fscid: -1 2024-04-02T13:44:48.953 INFO:journalctl@ceph.mon.b.smithi088.stdout: 2024-04-02T13:44:48.953 INFO:journalctl@ceph.mon.b.smithi088.stdout: No filesystems configured 2024-04-02T13:44:48.953 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-02T13:44:48.953 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-02T13:44:48.953 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-02T13:44:48.953 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-02T13:44:48.953 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-02T13:44:48.954 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-02T13:44:48.954 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-02T13:44:48.954 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T13:44:48.954 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T13:44:48.954 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T13:44:48.954 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:48.954 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:48.954 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:44:48.954 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:48.954 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:44:48.954 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:48.954 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:48.954 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:48.954 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:48.955 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi088", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:44:48.955 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:48.955 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:44:48.955 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:48.955 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: Updating smithi088:/etc/ceph/ceph.conf 2024-04-02T13:44:48.955 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: Deploying cephadm binary to smithi189 2024-04-02T13:44:48.955 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: Updating smithi088:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:44:48.955 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:44:48.955 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:48.955 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: Updating smithi088:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:44:48.955 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:48.955 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:48.955 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:48.956 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:48.956 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:44:48.956 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:48.956 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:48.956 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: Added host smithi189 2024-04-02T13:44:48.956 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:48.956 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:48.956 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:48.956 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:44:48.956 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:48.956 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2269823775' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-02T13:44:48.956 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2269823775' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-02T13:44:48.957 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:44:48.957 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:48.957 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:48.957 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:48.957 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:48.957 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:44:48.957 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:48.957 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:44:48.957 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:48.957 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: Updating smithi189:/etc/ceph/ceph.conf 2024-04-02T13:44:48.957 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: Updating smithi189:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:44:48.957 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:44:48.957 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: Updating smithi189:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:44:48.957 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:48.958 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:48.958 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:48.958 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:48.958 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi052:172.21.15.52=a;smithi088:172.21.15.88=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:44:48.958 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: Saving service mon spec with placement smithi052:172.21.15.52=a;smithi088:172.21.15.88=b;smithi189:172.21.15.189=c;count:3 2024-04-02T13:44:48.958 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:48.958 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:44:48.958 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:48.958 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:44:48.958 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:48.959 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:44:48.959 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:48.959 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: Deploying daemon mon.c on smithi189 2024-04-02T13:44:48.959 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:48.959 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:44:48.959 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:44:48.959 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: mon.a calling monitor election 2024-04-02T13:44:48.959 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:48.959 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:44:48.959 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:44:48.959 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: mon.c calling monitor election 2024-04-02T13:44:48.959 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:48.959 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:44:48.959 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:44:48.960 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:44:48.960 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-02T13:44:48.960 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: monmap epoch 2 2024-04-02T13:44:48.960 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:44:48.960 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: last_changed 2024-04-02T13:44:39.933747+0000 2024-04-02T13:44:48.960 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: created 2024-04-02T13:42:54.002665+0000 2024-04-02T13:44:48.960 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: min_mon_release 19 (squid) 2024-04-02T13:44:48.960 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: election_strategy: 1 2024-04-02T13:44:48.960 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-04-02T13:44:48.960 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-04-02T13:44:48.961 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: fsmap 2024-04-02T13:44:48.961 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:44:48.961 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: mgrmap e13: x(active, since 50s) 2024-04-02T13:44:48.961 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: overall HEALTH_OK 2024-04-02T13:44:48.961 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:48.961 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:48.961 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:48.961 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:44:48.961 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:48.961 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:48.961 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: Deploying daemon mon.b on smithi088 2024-04-02T13:44:48.961 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:44:48.961 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: from='client.? 172.21.15.189:0/2805904445' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T13:44:48.961 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:48.961 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 ceph-mon[36528]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-02T13:44:48.962 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 bash[36509]: 9e66ff83e3f91ec2ad55240248270b19efa411f900cf66bcffde5c481ea87ea5 2024-04-02T13:44:48.962 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:48 smithi088 systemd[1]: Started Ceph mon.b for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4. 2024-04-02T13:44:54.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:44:54.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: mon.c calling monitor election 2024-04-02T13:44:54.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: mon.a calling monitor election 2024-04-02T13:44:54.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:44:54.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:44:54.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:54.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:44:54.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:44:54.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:54.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:44:54.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:44:54.040 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:54.040 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:44:54.040 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-02T13:44:54.040 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: monmap epoch 3 2024-04-02T13:44:54.040 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:44:54.040 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: last_changed 2024-04-02T13:44:48.605500+0000 2024-04-02T13:44:54.040 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: created 2024-04-02T13:42:54.002665+0000 2024-04-02T13:44:54.040 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: min_mon_release 19 (squid) 2024-04-02T13:44:54.040 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: election_strategy: 1 2024-04-02T13:44:54.040 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-04-02T13:44:54.040 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-04-02T13:44:54.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: 2: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.b 2024-04-02T13:44:54.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: fsmap 2024-04-02T13:44:54.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:44:54.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: mgrmap e13: x(active, since 58s) 2024-04-02T13:44:54.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: overall HEALTH_OK 2024-04-02T13:44:54.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:54.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:53 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:54.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:44:54.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: mon.c calling monitor election 2024-04-02T13:44:54.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: mon.a calling monitor election 2024-04-02T13:44:54.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:44:54.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:44:54.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:54.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:44:54.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:44:54.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:54.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:44:54.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:44:54.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:54.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:44:54.091 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-02T13:44:54.091 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: monmap epoch 3 2024-04-02T13:44:54.091 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:44:54.091 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: last_changed 2024-04-02T13:44:48.605500+0000 2024-04-02T13:44:54.091 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: created 2024-04-02T13:42:54.002665+0000 2024-04-02T13:44:54.091 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: min_mon_release 19 (squid) 2024-04-02T13:44:54.091 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: election_strategy: 1 2024-04-02T13:44:54.091 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-04-02T13:44:54.091 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-04-02T13:44:54.091 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: 2: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.b 2024-04-02T13:44:54.091 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: fsmap 2024-04-02T13:44:54.091 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:44:54.092 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: mgrmap e13: x(active, since 58s) 2024-04-02T13:44:54.092 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: overall HEALTH_OK 2024-04-02T13:44:54.092 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:54.092 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:53 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:54.157 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.154+0000 7fa9a8280700 1 -- 172.21.15.189:0/4094014913 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9880035c0 msgr2=0x7fa988005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:54.158 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.154+0000 7fa9a8280700 1 --2- 172.21.15.189:0/4094014913 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9880035c0 0x7fa988005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa9a006a810 tx=0x7fa99000fe60 comp rx=0 tx=0).stop 2024-04-02T13:44:54.158 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.154+0000 7fa9a8280700 1 -- 172.21.15.189:0/4094014913 shutdown_connections 2024-04-02T13:44:54.158 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.154+0000 7fa9a8280700 1 --2- 172.21.15.189:0/4094014913 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9880035c0 0x7fa988005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:54.158 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.154+0000 7fa9a8280700 1 -- 172.21.15.189:0/4094014913 >> 172.21.15.189:0/4094014913 conn(0x7fa9a00fcac0 msgr2=0x7fa9a00feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:54.158 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.154+0000 7fa9a8280700 1 -- 172.21.15.189:0/4094014913 shutdown_connections 2024-04-02T13:44:54.158 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.154+0000 7fa9a8280700 1 -- 172.21.15.189:0/4094014913 wait complete. 2024-04-02T13:44:54.161 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.155+0000 7fa9a8280700 1 Processor -- start 2024-04-02T13:44:54.161 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.155+0000 7fa9a8280700 1 -- start start 2024-04-02T13:44:54.161 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.155+0000 7fa9a8280700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9880035c0 0x7fa9a0197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:54.161 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.155+0000 7fa9a8280700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9a01982b0 0x7fa9a019c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:54.161 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.155+0000 7fa9a8280700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9a019cb80 0x7fa9a019d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:54.161 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.155+0000 7fa9a8280700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a010a100 con 0x7fa9880035c0 2024-04-02T13:44:54.161 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.155+0000 7fa9a8280700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a0109f80 con 0x7fa9a01982b0 2024-04-02T13:44:54.161 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.155+0000 7fa9a8280700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a010a280 con 0x7fa9a019cb80 2024-04-02T13:44:54.161 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.156+0000 7fa9a681d700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9a019cb80 0x7fa9a019d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:54.161 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.156+0000 7fa9a681d700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9a019cb80 0x7fa9a019d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:58862/0 (socket says 172.21.15.189:58862) 2024-04-02T13:44:54.161 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.156+0000 7fa9a681d700 1 -- 172.21.15.189:0/304556055 learned_addr learned my addr 172.21.15.189:0/304556055 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-02T13:44:54.162 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.156+0000 7fa9a601c700 1 --2- 172.21.15.189:0/304556055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9880035c0 0x7fa9a0197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:54.162 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.156+0000 7fa9a581b700 1 --2- 172.21.15.189:0/304556055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9a01982b0 0x7fa9a019c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:54.162 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.156+0000 7fa9a681d700 1 -- 172.21.15.189:0/304556055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9a019cb80 msgr2=0x7fa9a019d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-02T13:44:54.162 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.156+0000 7fa9a681d700 1 -- 172.21.15.189:0/304556055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9a019cb80 msgr2=0x7fa9a019d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-02T13:44:54.162 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.156+0000 7fa9a681d700 1 --2- 172.21.15.189:0/304556055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9a019cb80 0x7fa9a019d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T13:44:54.162 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.156+0000 7fa9a681d700 1 --2- 172.21.15.189:0/304556055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9a019cb80 0x7fa9a019d030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:44:54.162 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.156+0000 7fa9a581b700 1 -- 172.21.15.189:0/304556055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9a01982b0 msgr2=0x7fa9a019c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-02T13:44:54.162 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.156+0000 7fa9a581b700 1 -- 172.21.15.189:0/304556055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9a01982b0 msgr2=0x7fa9a019c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-02T13:44:54.162 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.156+0000 7fa9a581b700 1 --2- 172.21.15.189:0/304556055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9a01982b0 0x7fa9a019c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T13:44:54.162 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.156+0000 7fa9a581b700 1 --2- 172.21.15.189:0/304556055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9a01982b0 0x7fa9a019c640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:44:54.162 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.156+0000 7fa9a601c700 1 -- 172.21.15.189:0/304556055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9a019cb80 msgr2=0x7fa9a019d030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:44:54.162 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.156+0000 7fa9a601c700 1 --2- 172.21.15.189:0/304556055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9a019cb80 0x7fa9a019d030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:54.162 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.156+0000 7fa9a601c700 1 -- 172.21.15.189:0/304556055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9a01982b0 msgr2=0x7fa9a019c640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:44:54.163 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.156+0000 7fa9a601c700 1 --2- 172.21.15.189:0/304556055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9a01982b0 0x7fa9a019c640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:54.163 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.156+0000 7fa9a601c700 1 -- 172.21.15.189:0/304556055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a01a48f0 con 0x7fa9880035c0 2024-04-02T13:44:54.163 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.156+0000 7fa9a601c700 1 --2- 172.21.15.189:0/304556055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9880035c0 0x7fa9a0197d70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa9a006a810 tx=0x7fa990014fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:54.163 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.156+0000 7fa99f7fe700 1 -- 172.21.15.189:0/304556055 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa990022040 con 0x7fa9880035c0 2024-04-02T13:44:54.163 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.157+0000 7fa9a8280700 1 -- 172.21.15.189:0/304556055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a01a4ac0 con 0x7fa9880035c0 2024-04-02T13:44:54.163 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.157+0000 7fa99f7fe700 1 -- 172.21.15.189:0/304556055 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa99002c070 con 0x7fa9880035c0 2024-04-02T13:44:54.163 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.157+0000 7fa99f7fe700 1 -- 172.21.15.189:0/304556055 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa990004190 con 0x7fa9880035c0 2024-04-02T13:44:54.163 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.157+0000 7fa9a8280700 1 -- 172.21.15.189:0/304556055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a01a5000 con 0x7fa9880035c0 2024-04-02T13:44:54.163 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.157+0000 7fa99f7fe700 1 -- 172.21.15.189:0/304556055 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa990004330 con 0x7fa9880035c0 2024-04-02T13:44:54.163 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.158+0000 7fa99f7fe700 1 --2- 172.21.15.189:0/304556055 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa98c039ae0 0x7fa98c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:54.163 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.158+0000 7fa99f7fe700 1 -- 172.21.15.189:0/304556055 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa99004efe0 con 0x7fa9880035c0 2024-04-02T13:44:54.163 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.158+0000 7fa9a8280700 1 -- 172.21.15.189:0/304556055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a0063c20 con 0x7fa9880035c0 2024-04-02T13:44:54.163 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.158+0000 7fa9a581b700 1 --2- 172.21.15.189:0/304556055 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa98c039ae0 0x7fa98c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:54.164 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.158+0000 7fa9a581b700 1 --2- 172.21.15.189:0/304556055 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa98c039ae0 0x7fa98c03bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa994009850 tx=0x7fa994008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:54.166 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.163+0000 7fa99f7fe700 1 -- 172.21.15.189:0/304556055 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa99000d340 con 0x7fa9880035c0 2024-04-02T13:44:54.479 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.476+0000 7fa9a8280700 1 -- 172.21.15.189:0/304556055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa9a01a51d0 con 0x7fa9880035c0 2024-04-02T13:44:54.480 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.477+0000 7fa99f7fe700 1 -- 172.21.15.189:0/304556055 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7fa9900048f0 con 0x7fa9880035c0 2024-04-02T13:44:54.480 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-02T13:44:54.480 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":3,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","modified":"2024-04-02T13:44:48.605500Z","created":"2024-04-02T13:42:54.002665Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:3300","nonce":0},{"type":"v1","addr":"172.21.15.52:6789","nonce":0}]},"addr":"172.21.15.52:6789/0","public_addr":"172.21.15.52:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-02T13:44:54.482 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.479+0000 7fa9a8280700 1 -- 172.21.15.189:0/304556055 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa98c039ae0 msgr2=0x7fa98c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:54.483 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.479+0000 7fa9a8280700 1 --2- 172.21.15.189:0/304556055 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa98c039ae0 0x7fa98c03bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa994009850 tx=0x7fa994008040 comp rx=0 tx=0).stop 2024-04-02T13:44:54.483 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.479+0000 7fa9a8280700 1 -- 172.21.15.189:0/304556055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9880035c0 msgr2=0x7fa9a0197d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:54.483 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.479+0000 7fa9a8280700 1 --2- 172.21.15.189:0/304556055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9880035c0 0x7fa9a0197d70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa9a006a810 tx=0x7fa990014fe0 comp rx=0 tx=0).stop 2024-04-02T13:44:54.483 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.479+0000 7fa9a8280700 1 -- 172.21.15.189:0/304556055 shutdown_connections 2024-04-02T13:44:54.483 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.479+0000 7fa9a8280700 1 --2- 172.21.15.189:0/304556055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9a019cb80 0x7fa9a019d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:54.483 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.479+0000 7fa9a8280700 1 --2- 172.21.15.189:0/304556055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9880035c0 0x7fa9a0197d70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:54.483 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.479+0000 7fa9a8280700 1 --2- 172.21.15.189:0/304556055 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa98c039ae0 0x7fa98c03bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:54.483 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.479+0000 7fa9a8280700 1 --2- 172.21.15.189:0/304556055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9a01982b0 0x7fa9a019c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:54.483 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.479+0000 7fa9a8280700 1 -- 172.21.15.189:0/304556055 >> 172.21.15.189:0/304556055 conn(0x7fa9a00fcac0 msgr2=0x7fa9a00fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:54.483 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.480+0000 7fa9a8280700 1 -- 172.21.15.189:0/304556055 shutdown_connections 2024-04-02T13:44:54.483 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:44:54.480+0000 7fa9a8280700 1 -- 172.21.15.189:0/304556055 wait complete. 2024-04-02T13:44:54.484 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 3 2024-04-02T13:44:54.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:54 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:54.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:54 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:54.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:54 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:44:54.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:54 smithi189 ceph-mon[36683]: from='client.? 172.21.15.189:0/304556055' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T13:44:54.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:54 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:44:55.027 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:54 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:55.027 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:54 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:55.027 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:54 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:44:55.027 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:54 smithi052 ceph-mon[32104]: from='client.? 172.21.15.189:0/304556055' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T13:44:55.027 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:54 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:44:55.782 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-02T13:44:55.782 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph config generate-minimal-conf 2024-04-02T13:44:55.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:44:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: mon.c calling monitor election 2024-04-02T13:44:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: mon.a calling monitor election 2024-04-02T13:44:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:44:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:44:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:44:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:44:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:44:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:44:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:44:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-02T13:44:55.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: monmap epoch 3 2024-04-02T13:44:55.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:44:55.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: last_changed 2024-04-02T13:44:48.605500+0000 2024-04-02T13:44:55.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: created 2024-04-02T13:42:54.002665+0000 2024-04-02T13:44:55.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: min_mon_release 19 (squid) 2024-04-02T13:44:55.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: election_strategy: 1 2024-04-02T13:44:55.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-04-02T13:44:55.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-04-02T13:44:55.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: 2: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.b 2024-04-02T13:44:55.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: fsmap 2024-04-02T13:44:55.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:44:55.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: mgrmap e13: x(active, since 58s) 2024-04-02T13:44:55.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: overall HEALTH_OK 2024-04-02T13:44:55.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:55.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:55.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:55.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:55.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:44:55.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: from='client.? 172.21.15.189:0/304556055' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T13:44:55.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:44:56.047 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:44:56.405 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: mon.b calling monitor election 2024-04-02T13:44:56.405 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: mon.b calling monitor election 2024-04-02T13:44:56.405 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: mon.a calling monitor election 2024-04-02T13:44:56.405 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: mon.c calling monitor election 2024-04-02T13:44:56.405 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-02T13:44:56.405 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: monmap epoch 3 2024-04-02T13:44:56.405 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:44:56.405 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: last_changed 2024-04-02T13:44:48.605500+0000 2024-04-02T13:44:56.405 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: created 2024-04-02T13:42:54.002665+0000 2024-04-02T13:44:56.405 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: min_mon_release 19 (squid) 2024-04-02T13:44:56.406 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: election_strategy: 1 2024-04-02T13:44:56.406 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-04-02T13:44:56.406 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-04-02T13:44:56.406 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: 2: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.b 2024-04-02T13:44:56.406 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: fsmap 2024-04-02T13:44:56.406 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:44:56.406 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: mgrmap e13: x(active, since 60s) 2024-04-02T13:44:56.406 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: overall HEALTH_OK 2024-04-02T13:44:56.406 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.406 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.406 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.406 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.406 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.406 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.407 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.407 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.407 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.407 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.407 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.407 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:44:56.407 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:44:56.407 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:56.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: mon.b calling monitor election 2024-04-02T13:44:56.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: mon.b calling monitor election 2024-04-02T13:44:56.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: mon.a calling monitor election 2024-04-02T13:44:56.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: mon.c calling monitor election 2024-04-02T13:44:56.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-02T13:44:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: monmap epoch 3 2024-04-02T13:44:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:44:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: last_changed 2024-04-02T13:44:48.605500+0000 2024-04-02T13:44:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: created 2024-04-02T13:42:54.002665+0000 2024-04-02T13:44:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: min_mon_release 19 (squid) 2024-04-02T13:44:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: election_strategy: 1 2024-04-02T13:44:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-04-02T13:44:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-04-02T13:44:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: 2: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.b 2024-04-02T13:44:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: fsmap 2024-04-02T13:44:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:44:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: mgrmap e13: x(active, since 60s) 2024-04-02T13:44:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: overall HEALTH_OK 2024-04-02T13:44:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:44:56.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:44:56.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:56.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: mon.b calling monitor election 2024-04-02T13:44:56.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: mon.b calling monitor election 2024-04-02T13:44:56.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: mon.a calling monitor election 2024-04-02T13:44:56.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: mon.c calling monitor election 2024-04-02T13:44:56.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-02T13:44:56.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: monmap epoch 3 2024-04-02T13:44:56.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:44:56.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: last_changed 2024-04-02T13:44:48.605500+0000 2024-04-02T13:44:56.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: created 2024-04-02T13:42:54.002665+0000 2024-04-02T13:44:56.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: min_mon_release 19 (squid) 2024-04-02T13:44:56.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: election_strategy: 1 2024-04-02T13:44:56.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-04-02T13:44:56.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-04-02T13:44:56.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: 2: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.b 2024-04-02T13:44:56.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: fsmap 2024-04-02T13:44:56.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:44:56.540 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: mgrmap e13: x(active, since 60s) 2024-04-02T13:44:56.540 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: overall HEALTH_OK 2024-04-02T13:44:56.540 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.540 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.540 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.540 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.540 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.540 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.540 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.540 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.540 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.540 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.540 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:56.540 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:44:56.540 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:44:56.541 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:57.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:57 smithi088 ceph-mon[36528]: Reconfiguring mon.a (unknown last config time)... 2024-04-02T13:44:57.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:57 smithi088 ceph-mon[36528]: Reconfiguring daemon mon.a on smithi052 2024-04-02T13:44:57.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:57 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:44:57.458 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:57 smithi052 ceph-mon[32104]: Reconfiguring mon.a (unknown last config time)... 2024-04-02T13:44:57.458 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:57 smithi052 ceph-mon[32104]: Reconfiguring daemon mon.a on smithi052 2024-04-02T13:44:57.458 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:57 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:44:57.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:57 smithi189 ceph-mon[36683]: Reconfiguring mon.a (unknown last config time)... 2024-04-02T13:44:57.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:57 smithi189 ceph-mon[36683]: Reconfiguring daemon mon.a on smithi052 2024-04-02T13:44:57.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:57 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:44:58.397 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:58 smithi052 ceph-mon[32104]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:58.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:58 smithi088 ceph-mon[36528]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:58.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:58 smithi189 ceph-mon[36683]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:44:58.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.632+0000 7f65fa196700 1 -- 172.21.15.52:0/3042137123 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65f406ab30 msgr2=0x7f65f406afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:58.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.632+0000 7f65fa196700 1 --2- 172.21.15.52:0/3042137123 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65f406ab30 0x7f65f406afb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0099c0 tx=0x7f65dc009cd0 comp rx=0 tx=0).stop 2024-04-02T13:44:58.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.632+0000 7f65fa196700 1 -- 172.21.15.52:0/3042137123 shutdown_connections 2024-04-02T13:44:58.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.632+0000 7f65fa196700 1 --2- 172.21.15.52:0/3042137123 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65f406b4f0 0x7f65f410f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:58.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.632+0000 7f65fa196700 1 --2- 172.21.15.52:0/3042137123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65f4108630 0x7f65f4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:58.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.632+0000 7f65fa196700 1 --2- 172.21.15.52:0/3042137123 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65f406ab30 0x7f65f406afb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:58.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.632+0000 7f65fa196700 1 -- 172.21.15.52:0/3042137123 >> 172.21.15.52:0/3042137123 conn(0x7f65f4077540 msgr2=0x7f65f4077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:58.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.632+0000 7f65fa196700 1 -- 172.21.15.52:0/3042137123 shutdown_connections 2024-04-02T13:44:58.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.632+0000 7f65fa196700 1 -- 172.21.15.52:0/3042137123 wait complete. 2024-04-02T13:44:58.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.633+0000 7f65fa196700 1 Processor -- start 2024-04-02T13:44:58.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.633+0000 7f65fa196700 1 -- start start 2024-04-02T13:44:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.633+0000 7f65fa196700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65f406ab30 0x7f65f41015f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.633+0000 7f65fa196700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65f406b4f0 0x7f65f4101b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.633+0000 7f65fa196700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65f4108630 0x7f65f4102070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.633+0000 7f65fa196700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f65f4112930 con 0x7f65f406b4f0 2024-04-02T13:44:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.633+0000 7f65fa196700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f65f41127b0 con 0x7f65f406ab30 2024-04-02T13:44:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.633+0000 7f65fa196700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f65f4112ab0 con 0x7f65f4108630 2024-04-02T13:44:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.633+0000 7f65f2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65f406b4f0 0x7f65f4101b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.633+0000 7f65f2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65f406b4f0 0x7f65f4101b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35610/0 (socket says 172.21.15.52:35610) 2024-04-02T13:44:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.633+0000 7f65f2ffd700 1 -- 172.21.15.52:0/1496042716 learned_addr learned my addr 172.21.15.52:0/1496042716 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:44:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.633+0000 7f65f37fe700 1 --2- 172.21.15.52:0/1496042716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65f406ab30 0x7f65f41015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.634+0000 7f65f3fff700 1 --2- 172.21.15.52:0/1496042716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65f4108630 0x7f65f4102070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.634+0000 7f65f2ffd700 1 -- 172.21.15.52:0/1496042716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65f4108630 msgr2=0x7f65f4102070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.634+0000 7f65f2ffd700 1 --2- 172.21.15.52:0/1496042716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65f4108630 0x7f65f4102070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.634+0000 7f65f2ffd700 1 -- 172.21.15.52:0/1496042716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65f406ab30 msgr2=0x7f65f41015f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.634+0000 7f65f2ffd700 1 --2- 172.21.15.52:0/1496042716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65f406ab30 0x7f65f41015f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.634+0000 7f65f2ffd700 1 -- 172.21.15.52:0/1496042716 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f41a7310 con 0x7f65f406b4f0 2024-04-02T13:44:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.634+0000 7f65f37fe700 1 --2- 172.21.15.52:0/1496042716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65f406ab30 0x7f65f41015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.634+0000 7f65f2ffd700 1 --2- 172.21.15.52:0/1496042716 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65f406b4f0 0x7f65f4101b30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0059d0 tx=0x7f65dc00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.634+0000 7f65f0ff9700 1 -- 172.21.15.52:0/1496042716 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc027070 con 0x7f65f406b4f0 2024-04-02T13:44:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.634+0000 7f65f0ff9700 1 -- 172.21.15.52:0/1496042716 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65dc003680 con 0x7f65f406b4f0 2024-04-02T13:44:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.634+0000 7f65fa196700 1 -- 172.21.15.52:0/1496042716 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f41a75a0 con 0x7f65f406b4f0 2024-04-02T13:44:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.634+0000 7f65f0ff9700 1 -- 172.21.15.52:0/1496042716 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc017650 con 0x7f65f406b4f0 2024-04-02T13:44:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.634+0000 7f65fa196700 1 -- 172.21.15.52:0/1496042716 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f41a79f0 con 0x7f65f406b4f0 2024-04-02T13:44:58.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.635+0000 7f65f0ff9700 1 -- 172.21.15.52:0/1496042716 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f65dc02a070 con 0x7f65f406b4f0 2024-04-02T13:44:58.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.635+0000 7f65fa196700 1 -- 172.21.15.52:0/1496042716 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65f40692b0 con 0x7f65f406b4f0 2024-04-02T13:44:58.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.635+0000 7f65f0ff9700 1 --2- 172.21.15.52:0/1496042716 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f65e0039890 0x7f65e003bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:58.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.635+0000 7f65f0ff9700 1 -- 172.21.15.52:0/1496042716 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f65dc04d9e0 con 0x7f65f406b4f0 2024-04-02T13:44:58.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.635+0000 7f65f37fe700 1 --2- 172.21.15.52:0/1496042716 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f65e0039890 0x7f65e003bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:58.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.636+0000 7f65f37fe700 1 --2- 172.21.15.52:0/1496042716 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f65e0039890 0x7f65e003bd50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f65e400a7b0 tx=0x7f65e400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:58.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.640+0000 7f65f0ff9700 1 -- 172.21.15.52:0/1496042716 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f65dc013070 con 0x7f65f406b4f0 2024-04-02T13:44:58.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.852+0000 7f65fa196700 1 -- 172.21.15.52:0/1496042716 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f65f402dd30 con 0x7f65f406b4f0 2024-04-02T13:44:58.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.853+0000 7f65f0ff9700 1 -- 172.21.15.52:0/1496042716 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f65dc031090 con 0x7f65f406b4f0 2024-04-02T13:44:58.856 INFO:teuthology.orchestra.run.smithi052.stdout:# minimal ceph.conf for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:44:58.856 INFO:teuthology.orchestra.run.smithi052.stdout:[global] 2024-04-02T13:44:58.856 INFO:teuthology.orchestra.run.smithi052.stdout: fsid = bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:44:58.856 INFO:teuthology.orchestra.run.smithi052.stdout: mon_host = [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] 2024-04-02T13:44:58.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.855+0000 7f65fa196700 1 -- 172.21.15.52:0/1496042716 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f65e0039890 msgr2=0x7f65e003bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:58.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.855+0000 7f65fa196700 1 --2- 172.21.15.52:0/1496042716 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f65e0039890 0x7f65e003bd50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f65e400a7b0 tx=0x7f65e400d040 comp rx=0 tx=0).stop 2024-04-02T13:44:58.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.855+0000 7f65fa196700 1 -- 172.21.15.52:0/1496042716 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65f406b4f0 msgr2=0x7f65f4101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:58.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.855+0000 7f65fa196700 1 --2- 172.21.15.52:0/1496042716 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65f406b4f0 0x7f65f4101b30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0059d0 tx=0x7f65dc00fa00 comp rx=0 tx=0).stop 2024-04-02T13:44:58.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.855+0000 7f65fa196700 1 -- 172.21.15.52:0/1496042716 shutdown_connections 2024-04-02T13:44:58.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.855+0000 7f65fa196700 1 --2- 172.21.15.52:0/1496042716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65f4108630 0x7f65f4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:58.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.855+0000 7f65fa196700 1 --2- 172.21.15.52:0/1496042716 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f65e0039890 0x7f65e003bd50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:58.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.855+0000 7f65fa196700 1 --2- 172.21.15.52:0/1496042716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65f406ab30 0x7f65f41015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:58.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.855+0000 7f65fa196700 1 --2- 172.21.15.52:0/1496042716 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65f406b4f0 0x7f65f4101b30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:58.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.855+0000 7f65fa196700 1 -- 172.21.15.52:0/1496042716 >> 172.21.15.52:0/1496042716 conn(0x7f65f4077540 msgr2=0x7f65f4100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:58.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.855+0000 7f65fa196700 1 -- 172.21.15.52:0/1496042716 shutdown_connections 2024-04-02T13:44:58.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:44:58.855+0000 7f65fa196700 1 -- 172.21.15.52:0/1496042716 wait complete. 2024-04-02T13:44:59.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:59 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:59.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:59 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:59.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:59 smithi088 ceph-mon[36528]: Reconfiguring mon.b (monmap changed)... 2024-04-02T13:44:59.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:59 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:44:59.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:59 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:44:59.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:59 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:59.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:59 smithi088 ceph-mon[36528]: Reconfiguring daemon mon.b on smithi088 2024-04-02T13:44:59.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:44:59 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1496042716' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:59.442 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:59 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:59.442 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:59 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:59.442 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:59 smithi052 ceph-mon[32104]: Reconfiguring mon.b (monmap changed)... 2024-04-02T13:44:59.442 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:59 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:44:59.442 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:59 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:44:59.442 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:59 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:59.442 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:59 smithi052 ceph-mon[32104]: Reconfiguring daemon mon.b on smithi088 2024-04-02T13:44:59.443 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:44:59 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1496042716' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:59.469 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-02T13:44:59.470 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-04-02T13:44:59.470 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/ceph/ceph.conf 2024-04-02T13:44:59.500 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-04-02T13:44:59.500 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:44:59.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:59 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:59.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:59 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:44:59.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:59 smithi189 ceph-mon[36683]: Reconfiguring mon.b (monmap changed)... 2024-04-02T13:44:59.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:59 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:44:59.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:59 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:44:59.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:59 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:59.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:59 smithi189 ceph-mon[36683]: Reconfiguring daemon mon.b on smithi088 2024-04-02T13:44:59.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:44:59 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1496042716' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:59.568 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-02T13:44:59.568 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/ceph/ceph.conf 2024-04-02T13:44:59.598 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-02T13:44:59.598 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:44:59.665 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-02T13:44:59.665 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.conf 2024-04-02T13:44:59.699 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-02T13:44:59.700 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:44:59.771 INFO:tasks.cephadm:Adding mgr.x on smithi052 2024-04-02T13:44:59.771 INFO:tasks.cephadm:Adding mgr.y on smithi088 2024-04-02T13:44:59.771 INFO:tasks.cephadm:Adding mgr.z on smithi189 2024-04-02T13:44:59.771 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph orch apply mgr '3;smithi052=x;smithi088=y;smithi189=z' 2024-04-02T13:45:00.055 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:45:00.411 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:00 smithi189 ceph-mon[36683]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:00.412 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:00 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:00.412 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:00 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:00.412 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:00 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:45:00.412 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:00 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:45:00.412 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:00 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:00.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:00 smithi052 ceph-mon[32104]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:00.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:00 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:00.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:00 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:00.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:00 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:45:00.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:00 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:45:00.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:00 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:00.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:00 smithi088 ceph-mon[36528]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:00.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:00 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:00.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:00 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:00.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:00 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:45:00.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:00 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:45:00.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:00 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:01.227 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:01 smithi189 ceph-mon[36683]: Reconfiguring mon.c (monmap changed)... 2024-04-02T13:45:01.228 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:01 smithi189 ceph-mon[36683]: Reconfiguring daemon mon.c on smithi189 2024-04-02T13:45:01.443 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.438+0000 7f8691c9e700 1 -- 172.21.15.189:0/3873697884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f868c074a80 msgr2=0x7f868c074ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:01.443 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.438+0000 7f8691c9e700 1 --2- 172.21.15.189:0/3873697884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f868c074a80 0x7f868c074ea0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f867c009910 tx=0x7f867c009c20 comp rx=0 tx=0).stop 2024-04-02T13:45:01.443 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.438+0000 7f8691c9e700 1 -- 172.21.15.189:0/3873697884 shutdown_connections 2024-04-02T13:45:01.443 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.438+0000 7f8691c9e700 1 --2- 172.21.15.189:0/3873697884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f868c074a80 0x7f868c074ea0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:01.443 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.438+0000 7f8691c9e700 1 -- 172.21.15.189:0/3873697884 >> 172.21.15.189:0/3873697884 conn(0x7f868c0fd6e0 msgr2=0x7f868c0ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:01.443 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.438+0000 7f8691c9e700 1 -- 172.21.15.189:0/3873697884 shutdown_connections 2024-04-02T13:45:01.443 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.438+0000 7f8691c9e700 1 -- 172.21.15.189:0/3873697884 wait complete. 2024-04-02T13:45:01.444 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.438+0000 7f8691c9e700 1 Processor -- start 2024-04-02T13:45:01.444 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.439+0000 7f8691c9e700 1 -- start start 2024-04-02T13:45:01.444 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.439+0000 7f8691c9e700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f868c074a80 0x7f868c19e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:01.444 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.439+0000 7f8691c9e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f868c19e570 0x7f868c1a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:01.444 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.439+0000 7f8691c9e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f868c19e9f0 0x7f868c1a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:01.444 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.439+0000 7f8691c9e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f868c10e910 con 0x7f868c19e570 2024-04-02T13:45:01.444 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.439+0000 7f8691c9e700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f868c10e790 con 0x7f868c19e9f0 2024-04-02T13:45:01.444 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.439+0000 7f8691c9e700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f868c10ea90 con 0x7f868c074a80 2024-04-02T13:45:01.444 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.439+0000 7f868b7fe700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f868c074a80 0x7f868c19e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:01.444 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.439+0000 7f868b7fe700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f868c074a80 0x7f868c19e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:55144/0 (socket says 172.21.15.189:55144) 2024-04-02T13:45:01.444 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.439+0000 7f868b7fe700 1 -- 172.21.15.189:0/1511006349 learned_addr learned my addr 172.21.15.189:0/1511006349 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-02T13:45:01.444 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.439+0000 7f868affd700 1 --2- 172.21.15.189:0/1511006349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f868c19e570 0x7f868c1a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:01.444 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.439+0000 7f868bfff700 1 --2- 172.21.15.189:0/1511006349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f868c19e9f0 0x7f868c1a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:01.444 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.439+0000 7f868b7fe700 1 -- 172.21.15.189:0/1511006349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f868c19e9f0 msgr2=0x7f868c1a5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:01.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.439+0000 7f868b7fe700 1 --2- 172.21.15.189:0/1511006349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f868c19e9f0 0x7f868c1a5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:01.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.439+0000 7f868b7fe700 1 -- 172.21.15.189:0/1511006349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f868c19e570 msgr2=0x7f868c1a35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:01.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.439+0000 7f868b7fe700 1 --2- 172.21.15.189:0/1511006349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f868c19e570 0x7f868c1a35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:01.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.439+0000 7f868b7fe700 1 -- 172.21.15.189:0/1511006349 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868c1a6440 con 0x7f868c074a80 2024-04-02T13:45:01.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.439+0000 7f868bfff700 1 --2- 172.21.15.189:0/1511006349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f868c19e9f0 0x7f868c1a5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:01.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.440+0000 7f868b7fe700 1 --2- 172.21.15.189:0/1511006349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f868c074a80 0x7f868c19e030 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f867c005950 tx=0x7f867c004f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:01.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.440+0000 7f8688ff9700 1 -- 172.21.15.189:0/1511006349 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c027070 con 0x7f868c074a80 2024-04-02T13:45:01.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.440+0000 7f8691c9e700 1 -- 172.21.15.189:0/1511006349 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f868c1a66d0 con 0x7f868c074a80 2024-04-02T13:45:01.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.440+0000 7f8688ff9700 1 -- 172.21.15.189:0/1511006349 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f867c00be40 con 0x7f868c074a80 2024-04-02T13:45:01.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.440+0000 7f8688ff9700 1 -- 172.21.15.189:0/1511006349 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c00f460 con 0x7f868c074a80 2024-04-02T13:45:01.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.440+0000 7f8691c9e700 1 -- 172.21.15.189:0/1511006349 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f868c1a6c80 con 0x7f868c074a80 2024-04-02T13:45:01.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.441+0000 7f8688ff9700 1 -- 172.21.15.189:0/1511006349 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f867c02a070 con 0x7f868c074a80 2024-04-02T13:45:01.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.441+0000 7f8691c9e700 1 -- 172.21.15.189:0/1511006349 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f868c063c20 con 0x7f868c074a80 2024-04-02T13:45:01.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.441+0000 7f8688ff9700 1 --2- 172.21.15.189:0/1511006349 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f8674039ae0 0x7f867403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:01.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.441+0000 7f8688ff9700 1 -- 172.21.15.189:0/1511006349 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f867c04dc90 con 0x7f868c074a80 2024-04-02T13:45:01.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.441+0000 7f868affd700 1 --2- 172.21.15.189:0/1511006349 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f8674039ae0 0x7f867403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:01.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.442+0000 7f868affd700 1 --2- 172.21.15.189:0/1511006349 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f8674039ae0 0x7f867403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8680009ad0 tx=0x7f8680009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:01.449 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.446+0000 7f8688ff9700 1 -- 172.21.15.189:0/1511006349 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f867c013070 con 0x7f868c074a80 2024-04-02T13:45:01.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:01 smithi052 ceph-mon[32104]: Reconfiguring mon.c (monmap changed)... 2024-04-02T13:45:01.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:01 smithi052 ceph-mon[32104]: Reconfiguring daemon mon.c on smithi189 2024-04-02T13:45:01.668 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.665+0000 7f8691c9e700 1 -- 172.21.15.189:0/1511006349 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi052=x;smithi088=y;smithi189=z", "target": ["mon-mgr", ""]}) -- 0x7f868c02e0d0 con 0x7f8674039ae0 2024-04-02T13:45:01.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:01 smithi088 ceph-mon[36528]: Reconfiguring mon.c (monmap changed)... 2024-04-02T13:45:01.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:01 smithi088 ceph-mon[36528]: Reconfiguring daemon mon.c on smithi189 2024-04-02T13:45:01.674 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.671+0000 7f8688ff9700 1 -- 172.21.15.189:0/1511006349 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f868c02e0d0 con 0x7f8674039ae0 2024-04-02T13:45:01.674 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mgr update... 2024-04-02T13:45:01.676 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.673+0000 7f8691c9e700 1 -- 172.21.15.189:0/1511006349 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f8674039ae0 msgr2=0x7f867403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:01.677 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.673+0000 7f8691c9e700 1 --2- 172.21.15.189:0/1511006349 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f8674039ae0 0x7f867403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8680009ad0 tx=0x7f8680009210 comp rx=0 tx=0).stop 2024-04-02T13:45:01.677 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.673+0000 7f8691c9e700 1 -- 172.21.15.189:0/1511006349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f868c074a80 msgr2=0x7f868c19e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:01.677 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.673+0000 7f8691c9e700 1 --2- 172.21.15.189:0/1511006349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f868c074a80 0x7f868c19e030 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f867c005950 tx=0x7f867c004f80 comp rx=0 tx=0).stop 2024-04-02T13:45:01.677 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.673+0000 7f8691c9e700 1 -- 172.21.15.189:0/1511006349 shutdown_connections 2024-04-02T13:45:01.677 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.673+0000 7f8691c9e700 1 --2- 172.21.15.189:0/1511006349 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f8674039ae0 0x7f867403bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:01.677 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.673+0000 7f8691c9e700 1 --2- 172.21.15.189:0/1511006349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f868c19e9f0 0x7f868c1a5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:01.677 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.673+0000 7f8691c9e700 1 --2- 172.21.15.189:0/1511006349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f868c074a80 0x7f868c19e030 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:01.677 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.673+0000 7f8691c9e700 1 --2- 172.21.15.189:0/1511006349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f868c19e570 0x7f868c1a35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:01.677 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.673+0000 7f8691c9e700 1 -- 172.21.15.189:0/1511006349 >> 172.21.15.189:0/1511006349 conn(0x7f868c0fd6e0 msgr2=0x7f868c0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:01.677 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.674+0000 7f8691c9e700 1 -- 172.21.15.189:0/1511006349 shutdown_connections 2024-04-02T13:45:01.677 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:01.674+0000 7f8691c9e700 1 -- 172.21.15.189:0/1511006349 wait complete. 2024-04-02T13:45:02.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:02 smithi189 ceph-mon[36683]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:02.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:02 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:02.587 DEBUG:teuthology.orchestra.run.smithi088:mgr.y> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mgr.y.service 2024-04-02T13:45:02.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:02 smithi052 ceph-mon[32104]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:02.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:02 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:02.590 DEBUG:teuthology.orchestra.run.smithi189:mgr.z> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mgr.z.service 2024-04-02T13:45:02.592 INFO:tasks.cephadm:Deploying OSDs... 2024-04-02T13:45:02.592 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-04-02T13:45:02.592 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/scratch_devs of=/dev/stdout 2024-04-02T13:45:02.611 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-02T13:45:02.612 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_1 2024-04-02T13:45:02.615 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:02 smithi088 ceph-mon[36528]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:02.615 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:02 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:02.668 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-02T13:45:02.668 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:45:02.668 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-02T13:45:02.668 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:45:02.668 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:45:02.668 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-04-02 13:44:05.402778821 +0000 2024-04-02T13:45:02.669 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-04-02 13:39:21.270104630 +0000 2024-04-02T13:45:02.669 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-04-02 13:39:21.270104630 +0000 2024-04-02T13:45:02.669 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-04-02 13:39:21.270104630 +0000 2024-04-02T13:45:02.669 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-02T13:45:02.737 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-04-02T13:45:02.737 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-04-02T13:45:02.738 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000116774 s, 4.4 MB/s 2024-04-02T13:45:02.739 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-02T13:45:02.796 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_2 2024-04-02T13:45:02.852 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-02T13:45:02.852 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:45:02.852 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-02T13:45:02.852 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:45:02.852 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:45:02.852 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-04-02 13:44:05.442778322 +0000 2024-04-02T13:45:02.852 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-04-02 13:39:21.605100363 +0000 2024-04-02T13:45:02.853 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-04-02 13:39:21.605100363 +0000 2024-04-02T13:45:02.853 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-04-02 13:39:21.605100363 +0000 2024-04-02T13:45:02.853 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-02T13:45:02.919 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-04-02T13:45:02.920 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-04-02T13:45:02.920 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 9.8553e-05 s, 5.2 MB/s 2024-04-02T13:45:02.921 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-02T13:45:02.977 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_3 2024-04-02T13:45:03.031 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-02T13:45:03.032 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:45:03.032 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-02T13:45:03.032 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:45:03.032 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:45:03.032 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-04-02 13:44:05.480777848 +0000 2024-04-02T13:45:03.032 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-04-02 13:39:21.939096107 +0000 2024-04-02T13:45:03.032 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-04-02 13:39:21.939096107 +0000 2024-04-02T13:45:03.032 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-04-02 13:39:21.939096107 +0000 2024-04-02T13:45:03.032 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-02T13:45:03.097 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-04-02T13:45:03.097 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-04-02T13:45:03.097 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000107432 s, 4.8 MB/s 2024-04-02T13:45:03.098 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-02T13:45:03.154 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_4 2024-04-02T13:45:03.209 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-02T13:45:03.210 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:45:03.210 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-02T13:45:03.210 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:45:03.210 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:45:03.210 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-04-02 13:44:05.517777386 +0000 2024-04-02T13:45:03.210 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-04-02 13:39:22.272091865 +0000 2024-04-02T13:45:03.210 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-04-02 13:39:22.272091865 +0000 2024-04-02T13:45:03.210 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-04-02 13:39:22.272091865 +0000 2024-04-02T13:45:03.210 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-02T13:45:03.275 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-04-02T13:45:03.275 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-04-02T13:45:03.275 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 9.17e-05 s, 5.6 MB/s 2024-04-02T13:45:03.276 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-02T13:45:03.332 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-02T13:45:03.332 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/scratch_devs of=/dev/stdout 2024-04-02T13:45:03.351 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-02T13:45:03.351 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_1 2024-04-02T13:45:03.409 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-02T13:45:03.409 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:45:03.409 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-04-02T13:45:03.409 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:45:03.409 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:45:03.409 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-04-02 13:44:48.055631179 +0000 2024-04-02T13:45:03.409 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-04-02 13:39:28.502898719 +0000 2024-04-02T13:45:03.409 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-04-02 13:39:28.502898719 +0000 2024-04-02T13:45:03.409 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-04-02 13:39:28.502898719 +0000 2024-04-02T13:45:03.410 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-02T13:45:03.475 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:03 smithi088 ceph-mon[36528]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi052=x;smithi088=y;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:45:03.475 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:03 smithi088 ceph-mon[36528]: Saving service mgr spec with placement smithi052=x;smithi088=y;smithi189=z;count:3 2024-04-02T13:45:03.475 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:03 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:03.475 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:03 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:03.475 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:03 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:03.478 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-04-02T13:45:03.478 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-04-02T13:45:03.478 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.00027777 s, 1.8 MB/s 2024-04-02T13:45:03.480 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-02T13:45:03.537 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_2 2024-04-02T13:45:03.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:03 smithi189 ceph-mon[36683]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi052=x;smithi088=y;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:45:03.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:03 smithi189 ceph-mon[36683]: Saving service mgr spec with placement smithi052=x;smithi088=y;smithi189=z;count:3 2024-04-02T13:45:03.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:03 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:03.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:03 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:03.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:03 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:03.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:03 smithi052 ceph-mon[32104]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi052=x;smithi088=y;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:45:03.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:03 smithi052 ceph-mon[32104]: Saving service mgr spec with placement smithi052=x;smithi088=y;smithi189=z;count:3 2024-04-02T13:45:03.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:03 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:03.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:03 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:03.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:03 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:03.594 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-02T13:45:03.594 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:45:03.595 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-04-02T13:45:03.595 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:45:03.595 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:45:03.595 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-04-02 13:44:48.055631179 +0000 2024-04-02T13:45:03.595 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-04-02 13:39:28.866890893 +0000 2024-04-02T13:45:03.595 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-04-02 13:39:28.866890893 +0000 2024-04-02T13:45:03.595 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-04-02 13:39:28.866890893 +0000 2024-04-02T13:45:03.595 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-02T13:45:03.663 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-04-02T13:45:03.664 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-04-02T13:45:03.664 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000264411 s, 1.9 MB/s 2024-04-02T13:45:03.665 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-02T13:45:03.723 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_3 2024-04-02T13:45:03.778 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-02T13:45:03.779 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:45:03.779 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-04-02T13:45:03.779 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:45:03.779 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:45:03.779 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-04-02 13:44:48.056631159 +0000 2024-04-02T13:45:03.779 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-04-02 13:39:29.207883561 +0000 2024-04-02T13:45:03.779 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-04-02 13:39:29.207883561 +0000 2024-04-02T13:45:03.779 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-04-02 13:39:29.207883561 +0000 2024-04-02T13:45:03.779 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-02T13:45:03.845 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-04-02T13:45:03.846 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-04-02T13:45:03.846 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000210511 s, 2.4 MB/s 2024-04-02T13:45:03.847 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-02T13:45:03.903 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_4 2024-04-02T13:45:03.958 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-02T13:45:03.958 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:45:03.959 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-04-02T13:45:03.959 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:45:03.959 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:45:03.959 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-04-02 13:44:48.056631159 +0000 2024-04-02T13:45:03.959 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-04-02 13:39:29.572875713 +0000 2024-04-02T13:45:03.959 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-04-02 13:39:29.572875713 +0000 2024-04-02T13:45:03.959 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-04-02 13:39:29.572875713 +0000 2024-04-02T13:45:03.959 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-02T13:45:04.025 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-04-02T13:45:04.025 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-04-02T13:45:04.025 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000234133 s, 2.2 MB/s 2024-04-02T13:45:04.026 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-02T13:45:04.083 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-02T13:45:04.083 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/scratch_devs of=/dev/stdout 2024-04-02T13:45:04.100 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-02T13:45:04.100 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_1 2024-04-02T13:45:04.155 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-02T13:45:04.156 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:45:04.156 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-02T13:45:04.156 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:45:04.156 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:45:04.156 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-04-02 13:44:38.854681408 +0000 2024-04-02T13:45:04.156 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-04-02 13:39:21.290382144 +0000 2024-04-02T13:45:04.156 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-04-02 13:39:21.290382144 +0000 2024-04-02T13:45:04.156 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-04-02 13:39:21.290382144 +0000 2024-04-02T13:45:04.156 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-02T13:45:04.221 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-04-02T13:45:04.221 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-04-02T13:45:04.221 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.00028896 s, 1.8 MB/s 2024-04-02T13:45:04.222 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-02T13:45:04.279 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_2 2024-04-02T13:45:04.333 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-02T13:45:04.334 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:45:04.334 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-04-02T13:45:04.334 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:45:04.334 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:45:04.334 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-04-02 13:44:38.854681408 +0000 2024-04-02T13:45:04.334 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-04-02 13:39:21.631375085 +0000 2024-04-02T13:45:04.334 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-04-02 13:39:21.631375085 +0000 2024-04-02T13:45:04.334 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-04-02 13:39:21.631375085 +0000 2024-04-02T13:45:04.334 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-02T13:45:04.400 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-04-02T13:45:04.400 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-04-02T13:45:04.400 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000213435 s, 2.4 MB/s 2024-04-02T13:45:04.401 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-02T13:45:04.457 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_3 2024-04-02T13:45:04.474 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:04 smithi189 ceph-mon[36683]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:04.474 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:04 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:04.474 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:04 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:04.474 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:04 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:04.474 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:04 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:45:04.475 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:04 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T13:45:04.475 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:04 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:45:04.475 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:04 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:04.489 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-02T13:45:04.489 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:45:04.489 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-04-02T13:45:04.489 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:45:04.489 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:45:04.489 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-04-02 13:44:38.854681408 +0000 2024-04-02T13:45:04.489 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-04-02 13:39:21.981367840 +0000 2024-04-02T13:45:04.489 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-04-02 13:39:21.981367840 +0000 2024-04-02T13:45:04.489 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-04-02 13:39:21.981367840 +0000 2024-04-02T13:45:04.490 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-02T13:45:04.555 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-04-02T13:45:04.556 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-04-02T13:45:04.556 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000202451 s, 2.5 MB/s 2024-04-02T13:45:04.557 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-02T13:45:04.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:04 smithi052 ceph-mon[32104]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:04.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:04 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:04.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:04 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:04.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:04 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:04.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:04 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:45:04.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:04 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T13:45:04.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:04 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:45:04.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:04 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:04.614 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_4 2024-04-02T13:45:04.669 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-02T13:45:04.669 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:45:04.669 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-04-02T13:45:04.669 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:45:04.670 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:45:04.670 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-04-02 13:44:38.855681390 +0000 2024-04-02T13:45:04.670 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-04-02 13:39:22.329360636 +0000 2024-04-02T13:45:04.670 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-04-02 13:39:22.329360636 +0000 2024-04-02T13:45:04.670 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-04-02 13:39:22.329360636 +0000 2024-04-02T13:45:04.670 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-02T13:45:04.671 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:04 smithi088 ceph-mon[36528]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:04.671 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:04 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:04.671 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:04 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:04.671 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:04 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:04.671 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:04 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:45:04.671 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:04 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T13:45:04.671 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:04 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:45:04.671 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:04 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:04.735 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-04-02T13:45:04.736 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-04-02T13:45:04.736 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000213157 s, 2.4 MB/s 2024-04-02T13:45:04.737 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-02T13:45:04.792 INFO:tasks.cephadm:Deploying osd.0 on smithi052 with /dev/vg_nvme/lv_4... 2024-04-02T13:45:04.792 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-02T13:45:05.039 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:45:05.301 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:05 smithi189 ceph-mon[36683]: Deploying daemon mgr.z on smithi189 2024-04-02T13:45:05.504 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:05 smithi052 ceph-mon[32104]: Deploying daemon mgr.z on smithi189 2024-04-02T13:45:05.571 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:05 smithi088 ceph-mon[36528]: Deploying daemon mgr.z on smithi189 2024-04-02T13:45:06.254 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:06 smithi189 ceph-mon[36683]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:06.254 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:06 smithi189 systemd[1]: Started Ceph mgr.z for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4. 2024-04-02T13:45:06.337 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:06 smithi052 ceph-mon[32104]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:06.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:06 smithi088 ceph-mon[36528]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:06.788 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:06 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:45:06.696+0000 7ff9c8064200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T13:45:06.930 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-02T13:45:06.931 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-02T13:45:06.931 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2024-04-02T13:45:06.931 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2024-04-02T13:45:06.931 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00852511 s, 1.2 GB/s 2024-04-02T13:45:06.931 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2024-04-02T13:45:07.287 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:06 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:45:06.792+0000 7ff9c8064200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T13:45:07.539 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:07 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:07.539 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:07 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:07.539 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:07 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:07.539 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:07 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:45:07.539 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:07 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T13:45:07.539 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:07 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:45:07.539 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:07 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:07.539 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:07 smithi052 ceph-mon[32104]: Deploying daemon mgr.y on smithi088 2024-04-02T13:45:07.540 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi052:vg_nvme/lv_4 2024-04-02T13:45:07.588 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:07 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:07.588 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:07 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:07.588 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:07 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:07.588 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:07 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:45:07.589 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:07 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T13:45:07.589 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:07 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:45:07.589 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:07 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:07.589 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:07 smithi088 ceph-mon[36528]: Deploying daemon mgr.y on smithi088 2024-04-02T13:45:07.689 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:07 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:45:07.459+0000 7ff9c8064200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T13:45:07.689 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:07 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:45:07.552+0000 7ff9c8064200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T13:45:07.689 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:07 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:07.689 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:07 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:07.689 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:07 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:07.689 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:07 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:45:07.689 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:07 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T13:45:07.689 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:07 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:45:07.689 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:07 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:07.689 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:07 smithi189 ceph-mon[36683]: Deploying daemon mgr.y on smithi088 2024-04-02T13:45:07.781 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:45:08.038 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:07 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:45:07.694+0000 7ff9c8064200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T13:45:08.038 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:07 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:45:07.788+0000 7ff9c8064200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T13:45:08.038 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:07 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:45:07.871+0000 7ff9c8064200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T13:45:08.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:08 smithi088 ceph-mon[36528]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:08.426 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:08 smithi189 ceph-mon[36683]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:08.426 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:08 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:45:08.132+0000 7ff9c8064200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T13:45:08.426 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:08 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:45:08.242+0000 7ff9c8064200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T13:45:08.426 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:08 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:45:08.328+0000 7ff9c8064200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T13:45:08.426 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:08 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:45:08.423+0000 7ff9c8064200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T13:45:08.519 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:08 smithi052 ceph-mon[32104]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:08.788 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:08 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:45:08.598+0000 7ff9c8064200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T13:45:08.801 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:08 smithi088 systemd[1]: Starting Ceph mgr.y for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T13:45:09.104 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:08 smithi088 podman[37902]: 2024-04-02 13:45:08.799211592 +0000 UTC m=+0.112818412 container create 671f18422f67908b2704866b3611d4dae0df462e1c0469ad1ad1b20f461c4406 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0) 2024-04-02T13:45:09.104 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:08 smithi088 podman[37902]: 2024-04-02 13:45:08.704020172 +0000 UTC m=+0.017626978 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:45:09.104 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:09 smithi088 podman[37902]: 2024-04-02 13:45:09.018571136 +0000 UTC m=+0.332177936 container init 671f18422f67908b2704866b3611d4dae0df462e1c0469ad1ad1b20f461c4406 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True) 2024-04-02T13:45:09.104 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:09 smithi088 podman[37902]: 2024-04-02 13:45:09.021025725 +0000 UTC m=+0.334632529 container start 671f18422f67908b2704866b3611d4dae0df462e1c0469ad1ad1b20f461c4406 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:45:09.104 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:09 smithi088 ceph-mgr[37924]: -- 172.21.15.88:0/2533106093 <== mon.2 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55acfb2765a0 con 0x55acfb24ac00 2024-04-02T13:45:09.104 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:09 smithi088 bash[37902]: 671f18422f67908b2704866b3611d4dae0df462e1c0469ad1ad1b20f461c4406 2024-04-02T13:45:09.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.322+0000 7f5abbfa3700 1 -- 172.21.15.52:0/95308563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ab4106400 msgr2=0x7f5ab410ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:09.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.322+0000 7f5abbfa3700 1 --2- 172.21.15.52:0/95308563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ab4106400 0x7f5ab410ccb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00b0d0 tx=0x7f5aac00b3e0 comp rx=0 tx=0).stop 2024-04-02T13:45:09.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.322+0000 7f5abbfa3700 1 -- 172.21.15.52:0/95308563 shutdown_connections 2024-04-02T13:45:09.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.322+0000 7f5abbfa3700 1 --2- 172.21.15.52:0/95308563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ab4106400 0x7f5ab410ccb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:09.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.322+0000 7f5abbfa3700 1 --2- 172.21.15.52:0/95308563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ab4104860 0x7f5ab4104c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:09.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.322+0000 7f5abbfa3700 1 --2- 172.21.15.52:0/95308563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ab4105a60 0x7f5ab4105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:09.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.322+0000 7f5abbfa3700 1 -- 172.21.15.52:0/95308563 >> 172.21.15.52:0/95308563 conn(0x7f5ab40ffe00 msgr2=0x7f5ab4102240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:09.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.322+0000 7f5abbfa3700 1 -- 172.21.15.52:0/95308563 shutdown_connections 2024-04-02T13:45:09.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.322+0000 7f5abbfa3700 1 -- 172.21.15.52:0/95308563 wait complete. 2024-04-02T13:45:09.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.322+0000 7f5abbfa3700 1 Processor -- start 2024-04-02T13:45:09.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.323+0000 7f5abbfa3700 1 -- start start 2024-04-02T13:45:09.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.323+0000 7f5abbfa3700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ab4104860 0x7f5ab419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:09.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.323+0000 7f5abbfa3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ab4105a60 0x7f5ab419aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:09.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.323+0000 7f5abbfa3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ab4106400 0x7f5ab41a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:09.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.323+0000 7f5abbfa3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab4110450 con 0x7f5ab4106400 2024-04-02T13:45:09.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.323+0000 7f5abbfa3700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab41102d0 con 0x7f5ab4105a60 2024-04-02T13:45:09.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.323+0000 7f5abbfa3700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab41105d0 con 0x7f5ab4104860 2024-04-02T13:45:09.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.323+0000 7f5aba540700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ab4106400 0x7f5ab41a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:09.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.323+0000 7f5aba540700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ab4106400 0x7f5ab41a1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36032/0 (socket says 172.21.15.52:36032) 2024-04-02T13:45:09.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.323+0000 7f5aba540700 1 -- 172.21.15.52:0/1354399547 learned_addr learned my addr 172.21.15.52:0/1354399547 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:45:09.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.323+0000 7f5ab9d3f700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ab4104860 0x7f5ab419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:09.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.323+0000 7f5ab953e700 1 --2- 172.21.15.52:0/1354399547 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ab4105a60 0x7f5ab419aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:09.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.323+0000 7f5aba540700 1 -- 172.21.15.52:0/1354399547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ab4104860 msgr2=0x7f5ab419a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:09.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.323+0000 7f5aba540700 1 --2- 172.21.15.52:0/1354399547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ab4104860 0x7f5ab419a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:09.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.323+0000 7f5aba540700 1 -- 172.21.15.52:0/1354399547 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ab4105a60 msgr2=0x7f5ab419aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:09.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.323+0000 7f5aba540700 1 --2- 172.21.15.52:0/1354399547 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ab4105a60 0x7f5ab419aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:09.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.323+0000 7f5aba540700 1 -- 172.21.15.52:0/1354399547 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab41a25d0 con 0x7f5ab4106400 2024-04-02T13:45:09.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.323+0000 7f5ab953e700 1 --2- 172.21.15.52:0/1354399547 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ab4105a60 0x7f5ab419aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:45:09.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.323+0000 7f5aba540700 1 --2- 172.21.15.52:0/1354399547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ab4106400 0x7f5ab41a1f30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00b770 tx=0x7f5aac007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:09.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.323+0000 7f5aaaffd700 1 -- 172.21.15.52:0/1354399547 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac003c00 con 0x7f5ab4106400 2024-04-02T13:45:09.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.324+0000 7f5aaaffd700 1 -- 172.21.15.52:0/1354399547 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aac003da0 con 0x7f5ab4106400 2024-04-02T13:45:09.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.324+0000 7f5abbfa3700 1 -- 172.21.15.52:0/1354399547 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab41a2860 con 0x7f5ab4106400 2024-04-02T13:45:09.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.324+0000 7f5aaaffd700 1 -- 172.21.15.52:0/1354399547 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac01b9d0 con 0x7f5ab4106400 2024-04-02T13:45:09.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.324+0000 7f5abbfa3700 1 -- 172.21.15.52:0/1354399547 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab41a2d10 con 0x7f5ab4106400 2024-04-02T13:45:09.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.324+0000 7f5aaaffd700 1 -- 172.21.15.52:0/1354399547 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5aac01bb70 con 0x7f5ab4106400 2024-04-02T13:45:09.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.325+0000 7f5aaaffd700 1 --2- 172.21.15.52:0/1354399547 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f5aa0039890 0x7f5aa003bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:09.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.325+0000 7f5aaaffd700 1 -- 172.21.15.52:0/1354399547 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5aac00f020 con 0x7f5ab4106400 2024-04-02T13:45:09.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.325+0000 7f5abbfa3700 1 -- 172.21.15.52:0/1354399547 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab40692b0 con 0x7f5ab4106400 2024-04-02T13:45:09.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.325+0000 7f5ab9d3f700 1 --2- 172.21.15.52:0/1354399547 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f5aa0039890 0x7f5aa003bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:09.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.325+0000 7f5ab9d3f700 1 --2- 172.21.15.52:0/1354399547 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f5aa0039890 0x7f5aa003bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5ab41058c0 tx=0x7f5ab0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:09.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.331+0000 7f5aaaffd700 1 -- 172.21.15.52:0/1354399547 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5aac011c50 con 0x7f5ab4106400 2024-04-02T13:45:09.419 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:09 smithi088 systemd[1]: Started Ceph mgr.y for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4. 2024-04-02T13:45:09.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:09.565+0000 7f5abbfa3700 1 -- 172.21.15.52:0/1354399547 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5ab4062a30 con 0x7f5aa0039890 2024-04-02T13:45:09.919 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:09 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:45:09.578+0000 7f9dd40e4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T13:45:09.919 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:09 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:45:09.673+0000 7f9dd40e4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T13:45:10.037 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:09 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:45:09.578+0000 7ff9c8064200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T13:45:10.038 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:09 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:45:09.715+0000 7ff9c8064200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T13:45:10.375 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:10 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:45:10.076+0000 7ff9c8064200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T13:45:10.375 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:10 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:45:10.164+0000 7ff9c8064200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T13:45:10.375 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:10 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:45:10.372+0000 7ff9c8064200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:45:10.376 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:10 smithi189 ceph-mon[36683]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:10.376 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:10 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:10.376 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:10 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:10.376 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:10 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:10.376 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:10 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:10.376 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:10 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:10.376 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:10 smithi189 ceph-mon[36683]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:45:10.376 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:10 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:45:10.376 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:10 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:45:10.376 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:10 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:10.376 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:10 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:10.419 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:10 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:45:10.351+0000 7f9dd40e4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T13:45:10.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:10 smithi088 ceph-mon[36528]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:10.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:10 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:10.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:10 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:10.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:10 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:10.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:10 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:10.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:10 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:10.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:10 smithi088 ceph-mon[36528]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:45:10.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:10 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:45:10.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:10 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:45:10.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:10 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:10.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:10 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:10.459 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:10 smithi052 ceph-mon[32104]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:10.459 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:10 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:10.459 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:10 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:10.459 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:10 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:10.459 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:10 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:10.459 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:10 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:10.459 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:10 smithi052 ceph-mon[32104]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:45:10.459 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:10 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:45:10.460 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:10 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:45:10.460 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:10 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:10.460 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:10 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:10.703 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:10 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:45:10.443+0000 7f9dd40e4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T13:45:10.703 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:10 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:45:10.592+0000 7f9dd40e4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T13:45:10.703 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:10 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:45:10.700+0000 7f9dd40e4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T13:45:10.787 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:10 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:45:10.558+0000 7ff9c8064200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T13:45:11.045 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:10 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:45:10.785+0000 7f9dd40e4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T13:45:11.045 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:11 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:45:11.042+0000 7f9dd40e4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T13:45:11.113 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:10 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:45:10.850+0000 7ff9c8064200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:45:11.113 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:11 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:45:11.110+0000 7ff9c8064200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T13:45:11.325 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:11 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:45:11.149+0000 7f9dd40e4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T13:45:11.325 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:11 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:45:11.231+0000 7f9dd40e4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T13:45:11.325 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:11 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:45:11.323+0000 7f9dd40e4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T13:45:11.537 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:11 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:45:11.343+0000 7ff9c8064200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T13:45:11.656 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:11 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:45:11.494+0000 7f9dd40e4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T13:45:11.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:11 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:11.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:11 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:11 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:11 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:11 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:11 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:11 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:11 smithi088 ceph-mon[36528]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:11 smithi088 ceph-mon[36528]: Reconfiguring mgr.x (unknown last config time)... 2024-04-02T13:45:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:11 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:45:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:11 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:45:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:11 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:11 smithi088 ceph-mon[36528]: Reconfiguring daemon mgr.x on smithi052 2024-04-02T13:45:12.037 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:11 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:12.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:11 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:12.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:11 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:12.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:11 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:12.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:11 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:12.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:11 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:12.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:11 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:12.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:11 smithi189 ceph-mon[36683]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:12.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:11 smithi189 ceph-mon[36683]: Reconfiguring mgr.x (unknown last config time)... 2024-04-02T13:45:12.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:11 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:45:12.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:11 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:45:12.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:11 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:12.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:11 smithi189 ceph-mon[36683]: Reconfiguring daemon mgr.x on smithi052 2024-04-02T13:45:12.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:12.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:12.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:12.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:12.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:12.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:12.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:12.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:11 smithi052 ceph-mon[32104]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:12.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:11 smithi052 ceph-mon[32104]: Reconfiguring mgr.x (unknown last config time)... 2024-04-02T13:45:12.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:45:12.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:45:12.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:12.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:11 smithi052 ceph-mon[32104]: Reconfiguring daemon mgr.x on smithi052 2024-04-02T13:45:12.537 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:12 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:45:12.144+0000 7ff9c8064200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T13:45:12.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:12 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/462191411' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fe9c5ef-4edb-4b75-8e4f-5615bae725c3"}]: dispatch 2024-04-02T13:45:12.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:12 smithi052 ceph-mon[32104]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fe9c5ef-4edb-4b75-8e4f-5615bae725c3"}]: dispatch 2024-04-02T13:45:12.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:12 smithi052 ceph-mon[32104]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4fe9c5ef-4edb-4b75-8e4f-5615bae725c3"}]': finished 2024-04-02T13:45:12.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:12 smithi052 ceph-mon[32104]: osdmap e5: 1 total, 0 up, 1 in 2024-04-02T13:45:12.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:12 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:45:12.898 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:12 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/462191411' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fe9c5ef-4edb-4b75-8e4f-5615bae725c3"}]: dispatch 2024-04-02T13:45:12.898 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:12 smithi088 ceph-mon[36528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fe9c5ef-4edb-4b75-8e4f-5615bae725c3"}]: dispatch 2024-04-02T13:45:12.898 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:12 smithi088 ceph-mon[36528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4fe9c5ef-4edb-4b75-8e4f-5615bae725c3"}]': finished 2024-04-02T13:45:12.898 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:12 smithi088 ceph-mon[36528]: osdmap e5: 1 total, 0 up, 1 in 2024-04-02T13:45:12.899 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:12 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:45:12.899 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:12 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:45:12.443+0000 7f9dd40e4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T13:45:12.899 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:12 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:45:12.569+0000 7f9dd40e4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T13:45:12.899 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:12 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:45:12.896+0000 7f9dd40e4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T13:45:13.030 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:12 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/462191411' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fe9c5ef-4edb-4b75-8e4f-5615bae725c3"}]: dispatch 2024-04-02T13:45:13.030 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:12 smithi189 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fe9c5ef-4edb-4b75-8e4f-5615bae725c3"}]: dispatch 2024-04-02T13:45:13.030 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:12 smithi189 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4fe9c5ef-4edb-4b75-8e4f-5615bae725c3"}]': finished 2024-04-02T13:45:13.030 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:12 smithi189 ceph-mon[36683]: osdmap e5: 1 total, 0 up, 1 in 2024-04-02T13:45:13.031 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:12 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:45:13.031 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:12 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:45:12.715+0000 7ff9c8064200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T13:45:13.031 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:12 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:45:12.812+0000 7ff9c8064200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T13:45:13.031 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:45:13 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:45:13.027+0000 7ff9c8064200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T13:45:13.169 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:12 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:45:12.982+0000 7f9dd40e4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T13:45:13.627 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:13 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:45:13.171+0000 7f9dd40e4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:45:13.627 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:13 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:45:13.350+0000 7f9dd40e4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T13:45:13.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:13.676+0000 7f5aaaffd700 1 -- 172.21.15.52:0/1354399547 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f5aac022d20 con 0x7f5ab4106400 2024-04-02T13:45:13.880 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:13 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:45:13.624+0000 7f9dd40e4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:45:13.880 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:13 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:45:13.877+0000 7f9dd40e4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T13:45:13.880 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:13 smithi088 ceph-mon[36528]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:13.880 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:13 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1288360126' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:45:13.880 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:13 smithi088 ceph-mon[36528]: Standby manager daemon z started 2024-04-02T13:45:13.880 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:13 smithi088 ceph-mon[36528]: from='mgr.? 172.21.15.189:0/3089847764' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T13:45:13.880 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:13 smithi088 ceph-mon[36528]: from='mgr.? 172.21.15.189:0/3089847764' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:45:13.881 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:13 smithi088 ceph-mon[36528]: from='mgr.? 172.21.15.189:0/3089847764' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T13:45:13.881 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:13 smithi088 ceph-mon[36528]: from='mgr.? 172.21.15.189:0/3089847764' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:45:14.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:13 smithi189 ceph-mon[36683]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:14.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:13 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1288360126' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:45:14.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:13 smithi189 ceph-mon[36683]: Standby manager daemon z started 2024-04-02T13:45:14.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:13 smithi189 ceph-mon[36683]: from='mgr.? 172.21.15.189:0/3089847764' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T13:45:14.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:13 smithi189 ceph-mon[36683]: from='mgr.? 172.21.15.189:0/3089847764' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:45:14.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:13 smithi189 ceph-mon[36683]: from='mgr.? 172.21.15.189:0/3089847764' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T13:45:14.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:13 smithi189 ceph-mon[36683]: from='mgr.? 172.21.15.189:0/3089847764' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:45:14.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:13 smithi052 ceph-mon[32104]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:14.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:13 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1288360126' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:45:14.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:13 smithi052 ceph-mon[32104]: Standby manager daemon z started 2024-04-02T13:45:14.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:13 smithi052 ceph-mon[32104]: from='mgr.? 172.21.15.189:0/3089847764' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T13:45:14.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:13 smithi052 ceph-mon[32104]: from='mgr.? 172.21.15.189:0/3089847764' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:45:14.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:13 smithi052 ceph-mon[32104]: from='mgr.? 172.21.15.189:0/3089847764' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T13:45:14.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:13 smithi052 ceph-mon[32104]: from='mgr.? 172.21.15.189:0/3089847764' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:45:14.169 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:14 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:45:14.106+0000 7f9dd40e4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T13:45:15.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:14 smithi189 ceph-mon[36683]: mgrmap e14: x(active, since 79s), standbys: z 2024-04-02T13:45:15.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:14 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T13:45:15.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:14 smithi052 ceph-mon[32104]: mgrmap e14: x(active, since 79s), standbys: z 2024-04-02T13:45:15.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:14 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T13:45:15.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:14 smithi088 ceph-mon[36528]: mgrmap e14: x(active, since 79s), standbys: z 2024-04-02T13:45:15.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:14 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T13:45:15.169 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:14 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:45:14.884+0000 7f9dd40e4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T13:45:15.743 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:15 smithi088 ceph-mon[36528]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:15.744 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:15 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:45:15.439+0000 7f9dd40e4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T13:45:15.744 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:15 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:45:15.532+0000 7f9dd40e4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T13:45:15.744 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:45:15 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:45:15.741+0000 7f9dd40e4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T13:45:16.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:15 smithi189 ceph-mon[36683]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:16.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:15 smithi052 ceph-mon[32104]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:16.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:16.686+0000 7f5aaaffd700 1 -- 172.21.15.52:0/1354399547 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f5aac0253b0 con 0x7f5ab4106400 2024-04-02T13:45:16.948 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:16 smithi052 ceph-mon[32104]: Standby manager daemon y started 2024-04-02T13:45:16.949 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:16 smithi052 ceph-mon[32104]: from='mgr.? 172.21.15.88:0/2719207248' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T13:45:16.949 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:16 smithi052 ceph-mon[32104]: from='mgr.? 172.21.15.88:0/2719207248' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:45:16.949 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:16 smithi052 ceph-mon[32104]: from='mgr.? 172.21.15.88:0/2719207248' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T13:45:16.949 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:16 smithi052 ceph-mon[32104]: from='mgr.? 172.21.15.88:0/2719207248' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:45:17.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:16 smithi189 ceph-mon[36683]: Standby manager daemon y started 2024-04-02T13:45:17.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:16 smithi189 ceph-mon[36683]: from='mgr.? 172.21.15.88:0/2719207248' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T13:45:17.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:16 smithi189 ceph-mon[36683]: from='mgr.? 172.21.15.88:0/2719207248' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:45:17.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:16 smithi189 ceph-mon[36683]: from='mgr.? 172.21.15.88:0/2719207248' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T13:45:17.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:16 smithi189 ceph-mon[36683]: from='mgr.? 172.21.15.88:0/2719207248' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:45:17.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:16 smithi088 ceph-mon[36528]: Standby manager daemon y started 2024-04-02T13:45:17.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:16 smithi088 ceph-mon[36528]: from='mgr.? 172.21.15.88:0/2719207248' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T13:45:17.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:16 smithi088 ceph-mon[36528]: from='mgr.? 172.21.15.88:0/2719207248' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:45:17.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:16 smithi088 ceph-mon[36528]: from='mgr.? 172.21.15.88:0/2719207248' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T13:45:17.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:16 smithi088 ceph-mon[36528]: from='mgr.? 172.21.15.88:0/2719207248' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:45:17.807 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:17 smithi052 ceph-mon[32104]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:17.807 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:17 smithi052 ceph-mon[32104]: mgrmap e15: x(active, since 82s), standbys: z, y 2024-04-02T13:45:17.807 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:17 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T13:45:17.807 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:17 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:17.807 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:17 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:17.807 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:17 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:17.807 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:17 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:17.807 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:17 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:17.808 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:17 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:18.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:17 smithi189 ceph-mon[36683]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:18.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:17 smithi189 ceph-mon[36683]: mgrmap e15: x(active, since 82s), standbys: z, y 2024-04-02T13:45:18.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:17 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T13:45:18.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:17 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:18.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:17 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:18.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:17 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:18.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:17 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:18.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:17 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:18.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:17 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:18.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:17 smithi088 ceph-mon[36528]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:18.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:17 smithi088 ceph-mon[36528]: mgrmap e15: x(active, since 82s), standbys: z, y 2024-04-02T13:45:18.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:17 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T13:45:18.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:17 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:18.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:17 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:18.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:17 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:18.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:17 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:18.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:17 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:18.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:17 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:19.907 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:19 smithi052 ceph-mon[32104]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:20.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:19 smithi189 ceph-mon[36683]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:20.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:19 smithi088 ceph-mon[36528]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:20.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:20 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T13:45:20.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:20 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:20.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:20 smithi052 ceph-mon[32104]: Deploying daemon osd.0 on smithi052 2024-04-02T13:45:21.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:20 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T13:45:21.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:20 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:21.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:20 smithi189 ceph-mon[36683]: Deploying daemon osd.0 on smithi052 2024-04-02T13:45:21.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:20 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T13:45:21.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:20 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:21.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:20 smithi088 ceph-mon[36528]: Deploying daemon osd.0 on smithi052 2024-04-02T13:45:21.832 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:21 smithi052 ceph-mon[32104]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:22.037 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:21 smithi189 ceph-mon[36683]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:22.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:21 smithi088 ceph-mon[36528]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:24.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:23 smithi189 ceph-mon[36683]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:24.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:23 smithi052 ceph-mon[32104]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:24.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:23 smithi088 ceph-mon[36528]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:26.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:26 smithi088 ceph-mon[36528]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:26.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:26 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:26.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:26 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:26.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:26 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:26.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:26 smithi189 ceph-mon[36683]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:26.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:26 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:26.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:26 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:26.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:26 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:26.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:26 smithi052 ceph-mon[32104]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:26.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:26 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:26.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:26 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:26.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:26 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:28.403 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:28 smithi052 ceph-mon[32104]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:28.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:28.476+0000 7f5aaaffd700 1 -- 172.21.15.52:0/1354399547 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5ab4062a30 con 0x7f5aa0039890 2024-04-02T13:45:28.480 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 0 on host 'smithi052' 2024-04-02T13:45:28.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:28.479+0000 7f5abbfa3700 1 -- 172.21.15.52:0/1354399547 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f5aa0039890 msgr2=0x7f5aa003bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:28.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:28.479+0000 7f5abbfa3700 1 --2- 172.21.15.52:0/1354399547 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f5aa0039890 0x7f5aa003bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5ab41058c0 tx=0x7f5ab0009260 comp rx=0 tx=0).stop 2024-04-02T13:45:28.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:28.479+0000 7f5abbfa3700 1 -- 172.21.15.52:0/1354399547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ab4106400 msgr2=0x7f5ab41a1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:28.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:28.479+0000 7f5abbfa3700 1 --2- 172.21.15.52:0/1354399547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ab4106400 0x7f5ab41a1f30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00b770 tx=0x7f5aac007a20 comp rx=0 tx=0).stop 2024-04-02T13:45:28.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:28.479+0000 7f5abbfa3700 1 -- 172.21.15.52:0/1354399547 shutdown_connections 2024-04-02T13:45:28.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:28.479+0000 7f5abbfa3700 1 --2- 172.21.15.52:0/1354399547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ab4106400 0x7f5ab41a1f30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:28.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:28.479+0000 7f5abbfa3700 1 --2- 172.21.15.52:0/1354399547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ab4104860 0x7f5ab419a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:28.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:28.479+0000 7f5abbfa3700 1 --2- 172.21.15.52:0/1354399547 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f5aa0039890 0x7f5aa003bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:28.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:28.479+0000 7f5abbfa3700 1 --2- 172.21.15.52:0/1354399547 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ab4105a60 0x7f5ab419aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:28.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:28.479+0000 7f5abbfa3700 1 -- 172.21.15.52:0/1354399547 >> 172.21.15.52:0/1354399547 conn(0x7f5ab40ffe00 msgr2=0x7f5ab4108c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:28.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:28.479+0000 7f5abbfa3700 1 -- 172.21.15.52:0/1354399547 shutdown_connections 2024-04-02T13:45:28.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:45:28.479+0000 7f5abbfa3700 1 -- 172.21.15.52:0/1354399547 wait complete. 2024-04-02T13:45:28.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:28 smithi088 ceph-mon[36528]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:28.787 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:28 smithi189 ceph-mon[36683]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:29.447 DEBUG:teuthology.orchestra.run.smithi052:osd.0> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.0.service 2024-04-02T13:45:29.455 INFO:tasks.cephadm:Deploying osd.1 on smithi088 with /dev/vg_nvme/lv_4... 2024-04-02T13:45:29.456 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-02T13:45:29.695 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:45:29.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:29 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:29.791 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:29 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:29.791 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:29 smithi189 ceph-mon[36683]: from='osd.0 [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T13:45:29.820 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:29 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:29.820 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:29 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:29.820 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:29 smithi088 ceph-mon[36528]: from='osd.0 [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T13:45:29.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:29 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:29.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:29 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:29.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:29 smithi052 ceph-mon[32104]: from='osd.0 [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T13:45:30.752 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:30 smithi088 ceph-mon[36528]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:30.752 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:30 smithi088 ceph-mon[36528]: from='osd.0 [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-02T13:45:30.752 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:30 smithi088 ceph-mon[36528]: osdmap e6: 1 total, 0 up, 1 in 2024-04-02T13:45:30.752 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:30 smithi088 ceph-mon[36528]: from='osd.0 [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-04-02T13:45:30.752 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:30 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:45:30.752 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:30 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:30.752 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:30 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:30.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:30 smithi189 ceph-mon[36683]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:30.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:30 smithi189 ceph-mon[36683]: from='osd.0 [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-02T13:45:30.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:30 smithi189 ceph-mon[36683]: osdmap e6: 1 total, 0 up, 1 in 2024-04-02T13:45:30.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:30 smithi189 ceph-mon[36683]: from='osd.0 [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-04-02T13:45:30.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:30 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:45:30.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:30 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:30.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:30 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:30.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:30 smithi052 ceph-mon[32104]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:30.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:30 smithi052 ceph-mon[32104]: from='osd.0 [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-02T13:45:30.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:30 smithi052 ceph-mon[32104]: osdmap e6: 1 total, 0 up, 1 in 2024-04-02T13:45:30.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:30 smithi052 ceph-mon[32104]: from='osd.0 [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-04-02T13:45:30.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:30 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:45:30.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:30 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:30.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:30 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:31.339 INFO:journalctl@ceph.osd.0.smithi052.stdout:Apr 02 13:45:30 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-0[44682]: 2024-04-02T13:45:30.963+0000 7fdf42a51700 -1 osd.0 0 waiting for initial osdmap 2024-04-02T13:45:31.339 INFO:journalctl@ceph.osd.0.smithi052.stdout:Apr 02 13:45:30 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-0[44682]: 2024-04-02T13:45:30.967+0000 7fdf3a83c700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:45:31.574 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-02T13:45:31.574 INFO:teuthology.orchestra.run.smithi088.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-02T13:45:31.574 INFO:teuthology.orchestra.run.smithi088.stderr: stderr: 10+0 records in 2024-04-02T13:45:31.574 INFO:teuthology.orchestra.run.smithi088.stderr:10+0 records out 2024-04-02T13:45:31.574 INFO:teuthology.orchestra.run.smithi088.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110137 s, 952 MB/s 2024-04-02T13:45:31.574 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping successful for: 2024-04-02T13:45:31.653 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:31 smithi052 ceph-mon[32104]: from='osd.0 [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-04-02T13:45:31.653 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:31 smithi052 ceph-mon[32104]: osdmap e7: 1 total, 0 up, 1 in 2024-04-02T13:45:31.653 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:31 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:45:31.653 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:31 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:45:31.653 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:31 smithi052 ceph-mon[32104]: from='osd.0 [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472]' entity='osd.0' 2024-04-02T13:45:31.654 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:31 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:45:31.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:31 smithi189 ceph-mon[36683]: from='osd.0 [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-04-02T13:45:31.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:31 smithi189 ceph-mon[36683]: osdmap e7: 1 total, 0 up, 1 in 2024-04-02T13:45:31.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:31 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:45:31.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:31 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:45:31.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:31 smithi189 ceph-mon[36683]: from='osd.0 [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472]' entity='osd.0' 2024-04-02T13:45:31.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:31 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:45:31.891 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:31 smithi088 ceph-mon[36528]: from='osd.0 [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-04-02T13:45:31.891 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:31 smithi088 ceph-mon[36528]: osdmap e7: 1 total, 0 up, 1 in 2024-04-02T13:45:31.891 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:31 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:45:31.891 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:31 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:45:31.891 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:31 smithi088 ceph-mon[36528]: from='osd.0 [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472]' entity='osd.0' 2024-04-02T13:45:31.891 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:31 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:45:32.205 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi088:vg_nvme/lv_4 2024-04-02T13:45:32.467 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:45:32.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:32 smithi052 ceph-mon[32104]: purged_snaps scrub starts 2024-04-02T13:45:32.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:32 smithi052 ceph-mon[32104]: purged_snaps scrub ok 2024-04-02T13:45:32.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:32 smithi052 ceph-mon[32104]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:32.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:32 smithi052 ceph-mon[32104]: osd.0 [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472] boot 2024-04-02T13:45:32.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:32 smithi052 ceph-mon[32104]: osdmap e8: 1 total, 1 up, 1 in 2024-04-02T13:45:32.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:32 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:45:32.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:32 smithi189 ceph-mon[36683]: purged_snaps scrub starts 2024-04-02T13:45:32.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:32 smithi189 ceph-mon[36683]: purged_snaps scrub ok 2024-04-02T13:45:32.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:32 smithi189 ceph-mon[36683]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:32.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:32 smithi189 ceph-mon[36683]: osd.0 [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472] boot 2024-04-02T13:45:32.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:32 smithi189 ceph-mon[36683]: osdmap e8: 1 total, 1 up, 1 in 2024-04-02T13:45:32.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:32 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:45:32.811 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:32 smithi088 ceph-mon[36528]: purged_snaps scrub starts 2024-04-02T13:45:32.811 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:32 smithi088 ceph-mon[36528]: purged_snaps scrub ok 2024-04-02T13:45:32.811 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:32 smithi088 ceph-mon[36528]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:45:32.811 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:32 smithi088 ceph-mon[36528]: osd.0 [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472] boot 2024-04-02T13:45:32.811 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:32 smithi088 ceph-mon[36528]: osdmap e8: 1 total, 1 up, 1 in 2024-04-02T13:45:32.811 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:32 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:45:33.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.992+0000 7fa2893d8700 1 -- 172.21.15.88:0/847629382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa284105a80 msgr2=0x7fa284107e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:33.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.992+0000 7fa2893d8700 1 --2- 172.21.15.88:0/847629382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa284105a80 0x7fa284107e70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa26c009990 tx=0x7fa26c009ca0 comp rx=0 tx=0).stop 2024-04-02T13:45:33.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.992+0000 7fa2893d8700 1 -- 172.21.15.88:0/847629382 shutdown_connections 2024-04-02T13:45:33.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.992+0000 7fa2893d8700 1 --2- 172.21.15.88:0/847629382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2841083b0 0x7fa28410a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:33.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.992+0000 7fa2893d8700 1 --2- 172.21.15.88:0/847629382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa284103150 0x7fa284105540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:33.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.992+0000 7fa2893d8700 1 --2- 172.21.15.88:0/847629382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa284105a80 0x7fa284107e70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:33.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.992+0000 7fa2893d8700 1 -- 172.21.15.88:0/847629382 >> 172.21.15.88:0/847629382 conn(0x7fa2840fcb80 msgr2=0x7fa2840fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:33.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.992+0000 7fa2893d8700 1 -- 172.21.15.88:0/847629382 shutdown_connections 2024-04-02T13:45:33.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.992+0000 7fa2893d8700 1 -- 172.21.15.88:0/847629382 wait complete. 2024-04-02T13:45:33.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.992+0000 7fa2893d8700 1 Processor -- start 2024-04-02T13:45:33.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.992+0000 7fa2893d8700 1 -- start start 2024-04-02T13:45:33.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.993+0000 7fa2893d8700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa284103150 0x7fa2841988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:33.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.993+0000 7fa2893d8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa284105a80 0x7fa284198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:33.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.993+0000 7fa2893d8700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2841083b0 0x7fa28419fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:33.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.993+0000 7fa2893d8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa28410e270 con 0x7fa284105a80 2024-04-02T13:45:33.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.993+0000 7fa2893d8700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa28410e0f0 con 0x7fa284103150 2024-04-02T13:45:33.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.993+0000 7fa2893d8700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa28410e3f0 con 0x7fa2841083b0 2024-04-02T13:45:33.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.993+0000 7fa282ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa284103150 0x7fa2841988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:33.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.993+0000 7fa282ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa284103150 0x7fa2841988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50968/0 (socket says 172.21.15.88:50968) 2024-04-02T13:45:33.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.993+0000 7fa282ffd700 1 -- 172.21.15.88:0/2193162635 learned_addr learned my addr 172.21.15.88:0/2193162635 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-02T13:45:33.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.993+0000 7fa2837fe700 1 --2- 172.21.15.88:0/2193162635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2841083b0 0x7fa28419fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:33.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.993+0000 7fa27a7fc700 1 --2- 172.21.15.88:0/2193162635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa284105a80 0x7fa284198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:33.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.993+0000 7fa282ffd700 1 -- 172.21.15.88:0/2193162635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2841083b0 msgr2=0x7fa28419fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:33.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.993+0000 7fa282ffd700 1 --2- 172.21.15.88:0/2193162635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2841083b0 0x7fa28419fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:33.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.993+0000 7fa282ffd700 1 -- 172.21.15.88:0/2193162635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa284105a80 msgr2=0x7fa284198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:33.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.993+0000 7fa282ffd700 1 --2- 172.21.15.88:0/2193162635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa284105a80 0x7fa284198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:33.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.993+0000 7fa282ffd700 1 -- 172.21.15.88:0/2193162635 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2841a04f0 con 0x7fa284103150 2024-04-02T13:45:33.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.993+0000 7fa27a7fc700 1 --2- 172.21.15.88:0/2193162635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa284105a80 0x7fa284198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:45:33.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.993+0000 7fa282ffd700 1 --2- 172.21.15.88:0/2193162635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa284103150 0x7fa2841988e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa27400c920 tx=0x7fa27400cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:33.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.993+0000 7fa280ff9700 1 -- 172.21.15.88:0/2193162635 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa274007b00 con 0x7fa284103150 2024-04-02T13:45:33.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.993+0000 7fa2893d8700 1 -- 172.21.15.88:0/2193162635 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2841a07e0 con 0x7fa284103150 2024-04-02T13:45:33.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.993+0000 7fa280ff9700 1 -- 172.21.15.88:0/2193162635 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa274004d10 con 0x7fa284103150 2024-04-02T13:45:33.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.993+0000 7fa280ff9700 1 -- 172.21.15.88:0/2193162635 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa274005680 con 0x7fa284103150 2024-04-02T13:45:33.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.994+0000 7fa2893d8700 1 -- 172.21.15.88:0/2193162635 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2841a0d70 con 0x7fa284103150 2024-04-02T13:45:34.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.995+0000 7fa2893d8700 1 -- 172.21.15.88:0/2193162635 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2840692b0 con 0x7fa284103150 2024-04-02T13:45:34.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.996+0000 7fa280ff9700 1 -- 172.21.15.88:0/2193162635 <== mon.2 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fa274004750 con 0x7fa284103150 2024-04-02T13:45:34.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.996+0000 7fa280ff9700 1 --2- 172.21.15.88:0/2193162635 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa2700a3bb0 0x7fa2700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:34.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.996+0000 7fa280ff9700 1 -- 172.21.15.88:0/2193162635 <== mon.2 v2:172.21.15.88:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fa2740ce4e0 con 0x7fa284103150 2024-04-02T13:45:34.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.996+0000 7fa27a7fc700 1 --2- 172.21.15.88:0/2193162635 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa2700a3bb0 0x7fa2700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:34.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:33.997+0000 7fa27a7fc700 1 --2- 172.21.15.88:0/2193162635 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa2700a3bb0 0x7fa2700a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa26c005cd0 tx=0x7fa26c005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:34.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:34.001+0000 7fa280ff9700 1 -- 172.21.15.88:0/2193162635 <== mon.2 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa274014030 con 0x7fa284103150 2024-04-02T13:45:34.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:33 smithi088 ceph-mon[36528]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:34.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:33 smithi088 ceph-mon[36528]: osdmap e9: 1 total, 1 up, 1 in 2024-04-02T13:45:34.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:33 smithi088 ceph-mon[36528]: Detected new or changed devices on smithi052 2024-04-02T13:45:34.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:33 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:34.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:33 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:34.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:33 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:34.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:33 smithi088 ceph-mon[36528]: Adjusting osd_memory_target on smithi052 to 17093M 2024-04-02T13:45:34.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:33 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:34.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:33 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:34.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:33 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:34.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:33 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:34.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:33 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:34.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:33 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:34.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:33 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:34.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:33 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:34.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:34.209+0000 7fa2893d8700 1 -- 172.21.15.88:0/2193162635 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa284062a30 con 0x7fa2700a3bb0 2024-04-02T13:45:34.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:33 smithi189 ceph-mon[36683]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:34.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:33 smithi189 ceph-mon[36683]: osdmap e9: 1 total, 1 up, 1 in 2024-04-02T13:45:34.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:33 smithi189 ceph-mon[36683]: Detected new or changed devices on smithi052 2024-04-02T13:45:34.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:33 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:34.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:33 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:34.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:33 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:34.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:33 smithi189 ceph-mon[36683]: Adjusting osd_memory_target on smithi052 to 17093M 2024-04-02T13:45:34.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:33 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:34.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:33 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:34.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:33 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:34.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:33 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:34.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:33 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:34.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:33 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:34.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:33 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:34.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:33 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:34.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:33 smithi052 ceph-mon[32104]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:34.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:33 smithi052 ceph-mon[32104]: osdmap e9: 1 total, 1 up, 1 in 2024-04-02T13:45:34.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:33 smithi052 ceph-mon[32104]: Detected new or changed devices on smithi052 2024-04-02T13:45:34.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:33 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:34.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:33 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:34.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:33 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:34.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:33 smithi052 ceph-mon[32104]: Adjusting osd_memory_target on smithi052 to 17093M 2024-04-02T13:45:34.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:33 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:34.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:33 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:34.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:33 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:34.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:33 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:34.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:33 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:34.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:33 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:34.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:33 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:34.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:33 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:35.054 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:34 smithi088 ceph-mon[36528]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:45:35.054 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:34 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:45:35.055 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:34 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:45:35.055 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:34 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:35.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:34 smithi189 ceph-mon[36683]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:45:35.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:34 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:45:35.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:34 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:45:35.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:34 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:35.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:34 smithi052 ceph-mon[32104]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:45:35.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:34 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:45:35.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:34 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:45:35.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:34 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:36.052 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:35 smithi088 ceph-mon[36528]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:36.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:35 smithi189 ceph-mon[36683]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:36.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:35 smithi052 ceph-mon[32104]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:37.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:36 smithi088 ceph-mon[36528]: from='client.? 172.21.15.88:0/4052752308' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6"}]: dispatch 2024-04-02T13:45:37.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:36 smithi088 ceph-mon[36528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6"}]: dispatch 2024-04-02T13:45:37.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:36 smithi088 ceph-mon[36528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6"}]': finished 2024-04-02T13:45:37.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:36 smithi088 ceph-mon[36528]: osdmap e10: 2 total, 1 up, 2 in 2024-04-02T13:45:37.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:36 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:45:37.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:36 smithi189 ceph-mon[36683]: from='client.? 172.21.15.88:0/4052752308' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6"}]: dispatch 2024-04-02T13:45:37.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:36 smithi189 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6"}]: dispatch 2024-04-02T13:45:37.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:36 smithi189 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6"}]': finished 2024-04-02T13:45:37.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:36 smithi189 ceph-mon[36683]: osdmap e10: 2 total, 1 up, 2 in 2024-04-02T13:45:37.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:36 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:45:37.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:36 smithi052 ceph-mon[32104]: from='client.? 172.21.15.88:0/4052752308' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6"}]: dispatch 2024-04-02T13:45:37.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:36 smithi052 ceph-mon[32104]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6"}]: dispatch 2024-04-02T13:45:37.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:36 smithi052 ceph-mon[32104]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6"}]': finished 2024-04-02T13:45:37.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:36 smithi052 ceph-mon[32104]: osdmap e10: 2 total, 1 up, 2 in 2024-04-02T13:45:37.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:36 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:45:38.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:37 smithi189 ceph-mon[36683]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:38.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:37 smithi189 ceph-mon[36683]: from='client.? 172.21.15.88:0/2979526992' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:45:38.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:37 smithi052 ceph-mon[32104]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:38.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:37 smithi052 ceph-mon[32104]: from='client.? 172.21.15.88:0/2979526992' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:45:38.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:37 smithi088 ceph-mon[36528]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:38.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:37 smithi088 ceph-mon[36528]: from='client.? 172.21.15.88:0/2979526992' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:45:40.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:39 smithi189 ceph-mon[36683]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:40.325 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:39 smithi088 ceph-mon[36528]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:40.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:39 smithi052 ceph-mon[32104]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:42.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:41 smithi189 ceph-mon[36683]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:42.296 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:41 smithi088 ceph-mon[36528]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:42.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:41 smithi052 ceph-mon[32104]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:44.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:43 smithi189 ceph-mon[36683]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:44.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:43 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T13:45:44.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:43 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:44.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:43 smithi189 ceph-mon[36683]: Deploying daemon osd.1 on smithi088 2024-04-02T13:45:44.337 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:43 smithi088 ceph-mon[36528]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:44.337 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:43 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T13:45:44.337 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:43 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:44.337 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:43 smithi088 ceph-mon[36528]: Deploying daemon osd.1 on smithi088 2024-04-02T13:45:44.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:43 smithi052 ceph-mon[32104]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:44.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:43 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T13:45:44.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:43 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:44.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:43 smithi052 ceph-mon[32104]: Deploying daemon osd.1 on smithi088 2024-04-02T13:45:46.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:45 smithi189 ceph-mon[36683]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:46.331 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:45 smithi088 ceph-mon[36528]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:46.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:45 smithi052 ceph-mon[32104]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:48.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:47 smithi088 ceph-mon[36528]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:48.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:47 smithi189 ceph-mon[36683]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:48.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:47 smithi052 ceph-mon[32104]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:49.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:49 smithi088 ceph-mon[36528]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:49.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:49 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:49.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:49 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:49.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:49 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:50.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:49 smithi189 ceph-mon[36683]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:50.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:49 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:50.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:49 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:50.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:49 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:50.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:49 smithi052 ceph-mon[32104]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:50.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:49 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:50.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:49 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:50.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:49 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:51.811 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:51 smithi088 ceph-mon[36528]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:51.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:51.871+0000 7fa280ff9700 1 -- 172.21.15.88:0/2193162635 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa284062a30 con 0x7fa2700a3bb0 2024-04-02T13:45:51.876 INFO:teuthology.orchestra.run.smithi088.stdout:Created osd(s) 1 on host 'smithi088' 2024-04-02T13:45:51.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:51.874+0000 7fa2893d8700 1 -- 172.21.15.88:0/2193162635 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa2700a3bb0 msgr2=0x7fa2700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:51.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:51.874+0000 7fa2893d8700 1 --2- 172.21.15.88:0/2193162635 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa2700a3bb0 0x7fa2700a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa26c005cd0 tx=0x7fa26c005c60 comp rx=0 tx=0).stop 2024-04-02T13:45:51.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:51.874+0000 7fa2893d8700 1 -- 172.21.15.88:0/2193162635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa284103150 msgr2=0x7fa2841988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:51.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:51.874+0000 7fa2893d8700 1 --2- 172.21.15.88:0/2193162635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa284103150 0x7fa2841988e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa27400c920 tx=0x7fa27400cc30 comp rx=0 tx=0).stop 2024-04-02T13:45:51.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:51.874+0000 7fa2893d8700 1 -- 172.21.15.88:0/2193162635 shutdown_connections 2024-04-02T13:45:51.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:51.874+0000 7fa2893d8700 1 --2- 172.21.15.88:0/2193162635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2841083b0 0x7fa28419fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:51.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:51.874+0000 7fa2893d8700 1 --2- 172.21.15.88:0/2193162635 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa2700a3bb0 0x7fa2700a6070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:51.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:51.874+0000 7fa2893d8700 1 --2- 172.21.15.88:0/2193162635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa284103150 0x7fa2841988e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:51.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:51.874+0000 7fa2893d8700 1 --2- 172.21.15.88:0/2193162635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa284105a80 0x7fa284198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:51.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:51.874+0000 7fa2893d8700 1 -- 172.21.15.88:0/2193162635 >> 172.21.15.88:0/2193162635 conn(0x7fa2840fcb80 msgr2=0x7fa284103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:51.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:51.874+0000 7fa2893d8700 1 -- 172.21.15.88:0/2193162635 shutdown_connections 2024-04-02T13:45:51.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:45:51.874+0000 7fa2893d8700 1 -- 172.21.15.88:0/2193162635 wait complete. 2024-04-02T13:45:52.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:51 smithi189 ceph-mon[36683]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:52.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:51 smithi052 ceph-mon[32104]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:52.517 DEBUG:teuthology.orchestra.run.smithi088:osd.1> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.1.service 2024-04-02T13:45:52.519 INFO:tasks.cephadm:Deploying osd.2 on smithi189 with /dev/vg_nvme/lv_4... 2024-04-02T13:45:52.520 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-02T13:45:52.759 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 02 13:45:52 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-1[41137]: 2024-04-02T13:45:52.603+0000 7fcbdf2e47c0 -1 osd.1 0 log_to_monitors true 2024-04-02T13:45:52.767 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:45:53.076 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:52 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:53.076 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:52 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:53.076 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:52 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:53.076 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:52 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:53.076 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:52 smithi088 ceph-mon[36528]: from='osd.1 [v2:172.21.15.88:6800/288986438,v1:172.21.15.88:6801/288986438]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T13:45:53.076 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:52 smithi088 ceph-mon[36528]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T13:45:53.099 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:52 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:53.100 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:52 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:53.100 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:52 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:53.100 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:52 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:53.100 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:52 smithi189 ceph-mon[36683]: from='osd.1 [v2:172.21.15.88:6800/288986438,v1:172.21.15.88:6801/288986438]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T13:45:53.100 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:52 smithi189 ceph-mon[36683]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T13:45:53.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:52 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:53.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:52 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:53.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:52 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:53.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:52 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:53.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:52 smithi052 ceph-mon[32104]: from='osd.1 [v2:172.21.15.88:6800/288986438,v1:172.21.15.88:6801/288986438]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T13:45:53.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:52 smithi052 ceph-mon[32104]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T13:45:54.185 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:54 smithi189 ceph-mon[36683]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:54.185 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:54 smithi189 ceph-mon[36683]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-02T13:45:54.185 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:54 smithi189 ceph-mon[36683]: from='osd.1 [v2:172.21.15.88:6800/288986438,v1:172.21.15.88:6801/288986438]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-02T13:45:54.185 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:54 smithi189 ceph-mon[36683]: osdmap e11: 2 total, 1 up, 2 in 2024-04-02T13:45:54.185 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:54 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:45:54.185 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:54 smithi189 ceph-mon[36683]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-02T13:45:54.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:54 smithi088 ceph-mon[36528]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:54.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:54 smithi088 ceph-mon[36528]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-02T13:45:54.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:54 smithi088 ceph-mon[36528]: from='osd.1 [v2:172.21.15.88:6800/288986438,v1:172.21.15.88:6801/288986438]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-02T13:45:54.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:54 smithi088 ceph-mon[36528]: osdmap e11: 2 total, 1 up, 2 in 2024-04-02T13:45:54.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:54 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:45:54.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:54 smithi088 ceph-mon[36528]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-02T13:45:54.526 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:54 smithi052 ceph-mon[32104]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:54.526 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:54 smithi052 ceph-mon[32104]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-02T13:45:54.526 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:54 smithi052 ceph-mon[32104]: from='osd.1 [v2:172.21.15.88:6800/288986438,v1:172.21.15.88:6801/288986438]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-02T13:45:54.526 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:54 smithi052 ceph-mon[32104]: osdmap e11: 2 total, 1 up, 2 in 2024-04-02T13:45:54.526 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:54 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:45:54.526 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:54 smithi052 ceph-mon[32104]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-02T13:45:54.756 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-02T13:45:54.756 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-02T13:45:54.756 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-04-02T13:45:54.757 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-04-02T13:45:54.757 INFO:teuthology.orchestra.run.smithi189.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113633 s, 923 MB/s 2024-04-02T13:45:54.757 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-04-02T13:45:55.156 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 02 13:45:54 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-1[41137]: 2024-04-02T13:45:54.900+0000 7fcbd4949700 -1 osd.1 0 waiting for initial osdmap 2024-04-02T13:45:55.156 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 02 13:45:54 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-1[41137]: 2024-04-02T13:45:54.904+0000 7fcbcc734700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:45:55.362 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:55 smithi189 ceph-mon[36683]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-04-02T13:45:55.362 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:55 smithi189 ceph-mon[36683]: osdmap e12: 2 total, 1 up, 2 in 2024-04-02T13:45:55.363 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:55 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:45:55.363 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:55 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:45:55.363 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:55 smithi189 ceph-mon[36683]: from='osd.1 ' entity='osd.1' 2024-04-02T13:45:55.390 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi189:vg_nvme/lv_4 2024-04-02T13:45:55.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:55 smithi088 ceph-mon[36528]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-04-02T13:45:55.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:55 smithi088 ceph-mon[36528]: osdmap e12: 2 total, 1 up, 2 in 2024-04-02T13:45:55.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:45:55.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:45:55.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:55 smithi088 ceph-mon[36528]: from='osd.1 ' entity='osd.1' 2024-04-02T13:45:55.490 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:55 smithi052 ceph-mon[32104]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-04-02T13:45:55.491 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:55 smithi052 ceph-mon[32104]: osdmap e12: 2 total, 1 up, 2 in 2024-04-02T13:45:55.491 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:55 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:45:55.491 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:55 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:45:55.491 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:55 smithi052 ceph-mon[32104]: from='osd.1 ' entity='osd.1' 2024-04-02T13:45:55.633 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:45:56.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:56 smithi189 ceph-mon[36683]: purged_snaps scrub starts 2024-04-02T13:45:56.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:56 smithi189 ceph-mon[36683]: purged_snaps scrub ok 2024-04-02T13:45:56.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:56 smithi189 ceph-mon[36683]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:56.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:45:56.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:56 smithi189 ceph-mon[36683]: Detected new or changed devices on smithi088 2024-04-02T13:45:56.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:56.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:56.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:56.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:56 smithi189 ceph-mon[36683]: Adjusting osd_memory_target on smithi088 to 17080M 2024-04-02T13:45:56.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:56.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:56.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:56.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:56.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:56.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:56.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:56.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:56.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:56 smithi189 ceph-mon[36683]: osd.1 [v2:172.21.15.88:6800/288986438,v1:172.21.15.88:6801/288986438] boot 2024-04-02T13:45:56.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:56 smithi189 ceph-mon[36683]: osdmap e13: 2 total, 2 up, 2 in 2024-04-02T13:45:56.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:45:56.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:56 smithi088 ceph-mon[36528]: purged_snaps scrub starts 2024-04-02T13:45:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:56 smithi088 ceph-mon[36528]: purged_snaps scrub ok 2024-04-02T13:45:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:56 smithi088 ceph-mon[36528]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:45:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:56 smithi088 ceph-mon[36528]: Detected new or changed devices on smithi088 2024-04-02T13:45:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:56 smithi088 ceph-mon[36528]: Adjusting osd_memory_target on smithi088 to 17080M 2024-04-02T13:45:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:56.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:56.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:56.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:56.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:56.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:56 smithi088 ceph-mon[36528]: osd.1 [v2:172.21.15.88:6800/288986438,v1:172.21.15.88:6801/288986438] boot 2024-04-02T13:45:56.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:56 smithi088 ceph-mon[36528]: osdmap e13: 2 total, 2 up, 2 in 2024-04-02T13:45:56.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:45:56.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:56 smithi052 ceph-mon[32104]: purged_snaps scrub starts 2024-04-02T13:45:56.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:56 smithi052 ceph-mon[32104]: purged_snaps scrub ok 2024-04-02T13:45:56.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:56 smithi052 ceph-mon[32104]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:45:56.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:45:56.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:56 smithi052 ceph-mon[32104]: Detected new or changed devices on smithi088 2024-04-02T13:45:56.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:56.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:56.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:56.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:56 smithi052 ceph-mon[32104]: Adjusting osd_memory_target on smithi088 to 17080M 2024-04-02T13:45:56.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:56.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:56.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:56.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:56.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:56.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:56.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:56.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:45:56.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:56 smithi052 ceph-mon[32104]: osd.1 [v2:172.21.15.88:6800/288986438,v1:172.21.15.88:6801/288986438] boot 2024-04-02T13:45:56.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:56 smithi052 ceph-mon[32104]: osdmap e13: 2 total, 2 up, 2 in 2024-04-02T13:45:56.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:45:57.177 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.171+0000 7fba02b52700 1 -- 172.21.15.189:0/1610339405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9fc1010a0 msgr2=0x7fb9fc10ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:57.177 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.171+0000 7fba02b52700 1 --2- 172.21.15.189:0/1610339405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9fc1010a0 0x7fb9fc10ac50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb9f400b0a0 tx=0x7fb9f400b3b0 comp rx=0 tx=0).stop 2024-04-02T13:45:57.177 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.171+0000 7fba02b52700 1 -- 172.21.15.189:0/1610339405 shutdown_connections 2024-04-02T13:45:57.177 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.171+0000 7fba02b52700 1 --2- 172.21.15.189:0/1610339405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9fc1010a0 0x7fb9fc10ac50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:57.177 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.171+0000 7fba02b52700 1 --2- 172.21.15.189:0/1610339405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9fc103c60 0x7fb9fc104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:57.177 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.171+0000 7fba02b52700 1 --2- 172.21.15.189:0/1610339405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9fc1006e0 0x7fb9fc100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:57.177 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.171+0000 7fba02b52700 1 -- 172.21.15.189:0/1610339405 >> 172.21.15.189:0/1610339405 conn(0x7fb9fc0fc280 msgr2=0x7fb9fc0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:57.177 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.172+0000 7fba02b52700 1 -- 172.21.15.189:0/1610339405 shutdown_connections 2024-04-02T13:45:57.177 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.172+0000 7fba02b52700 1 -- 172.21.15.189:0/1610339405 wait complete. 2024-04-02T13:45:57.177 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.172+0000 7fba02b52700 1 Processor -- start 2024-04-02T13:45:57.177 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.172+0000 7fba02b52700 1 -- start start 2024-04-02T13:45:57.177 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.172+0000 7fba02b52700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9fc1006e0 0x7fb9fc198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:57.178 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.172+0000 7fba02b52700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9fc1010a0 0x7fb9fc198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:57.178 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.172+0000 7fba02b52700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9fc103c60 0x7fb9fc19ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:57.178 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.172+0000 7fba02b52700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb9fc10e3d0 con 0x7fb9fc1010a0 2024-04-02T13:45:57.178 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.172+0000 7fba02b52700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb9fc10e250 con 0x7fb9fc1006e0 2024-04-02T13:45:57.178 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.172+0000 7fba02b52700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb9fc10e550 con 0x7fb9fc103c60 2024-04-02T13:45:57.178 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.172+0000 7fba010ef700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9fc103c60 0x7fb9fc19ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:57.178 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.173+0000 7fba010ef700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9fc103c60 0x7fb9fc19ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:47300/0 (socket says 172.21.15.189:47300) 2024-04-02T13:45:57.178 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.173+0000 7fba010ef700 1 -- 172.21.15.189:0/2486646924 learned_addr learned my addr 172.21.15.189:0/2486646924 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-02T13:45:57.178 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.173+0000 7fb9fbfff700 1 --2- 172.21.15.189:0/2486646924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9fc1010a0 0x7fb9fc198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:57.178 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.173+0000 7fba008ee700 1 --2- 172.21.15.189:0/2486646924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9fc1006e0 0x7fb9fc198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:57.178 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.173+0000 7fba010ef700 1 -- 172.21.15.189:0/2486646924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9fc1006e0 msgr2=0x7fb9fc198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:57.178 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.173+0000 7fba010ef700 1 --2- 172.21.15.189:0/2486646924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9fc1006e0 0x7fb9fc198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:57.178 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.173+0000 7fba010ef700 1 -- 172.21.15.189:0/2486646924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9fc1010a0 msgr2=0x7fb9fc198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:57.179 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.173+0000 7fba010ef700 1 --2- 172.21.15.189:0/2486646924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9fc1010a0 0x7fb9fc198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:57.179 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.173+0000 7fba010ef700 1 -- 172.21.15.189:0/2486646924 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9fc1a05c0 con 0x7fb9fc103c60 2024-04-02T13:45:57.179 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.173+0000 7fb9fbfff700 1 --2- 172.21.15.189:0/2486646924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9fc1010a0 0x7fb9fc198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:45:57.179 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.173+0000 7fba008ee700 1 --2- 172.21.15.189:0/2486646924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9fc1006e0 0x7fb9fc198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:45:57.179 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.173+0000 7fba010ef700 1 --2- 172.21.15.189:0/2486646924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9fc103c60 0x7fb9fc19ff20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb9f40095e0 tx=0x7fb9f4007770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:57.179 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.173+0000 7fb9f9ffb700 1 -- 172.21.15.189:0/2486646924 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4021070 con 0x7fb9fc103c60 2024-04-02T13:45:57.179 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.173+0000 7fb9f9ffb700 1 -- 172.21.15.189:0/2486646924 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9f4004020 con 0x7fb9fc103c60 2024-04-02T13:45:57.179 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.173+0000 7fb9f9ffb700 1 -- 172.21.15.189:0/2486646924 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4005280 con 0x7fb9fc103c60 2024-04-02T13:45:57.179 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.173+0000 7fba02b52700 1 -- 172.21.15.189:0/2486646924 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9fc1a0850 con 0x7fb9fc103c60 2024-04-02T13:45:57.179 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.173+0000 7fba02b52700 1 -- 172.21.15.189:0/2486646924 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9fc1a0dd0 con 0x7fb9fc103c60 2024-04-02T13:45:57.180 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.174+0000 7fba02b52700 1 -- 172.21.15.189:0/2486646924 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9fc0692b0 con 0x7fb9fc103c60 2024-04-02T13:45:57.180 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.176+0000 7fb9f9ffb700 1 -- 172.21.15.189:0/2486646924 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fb9f40041d0 con 0x7fb9fc103c60 2024-04-02T13:45:57.181 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.176+0000 7fb9f9ffb700 1 --2- 172.21.15.189:0/2486646924 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb9ec0a3c30 0x7fb9ec0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:57.181 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.177+0000 7fb9f9ffb700 1 -- 172.21.15.189:0/2486646924 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fb9f400f020 con 0x7fb9fc103c60 2024-04-02T13:45:57.181 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.177+0000 7fba008ee700 1 --2- 172.21.15.189:0/2486646924 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb9ec0a3c30 0x7fb9ec0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:57.181 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.177+0000 7fba008ee700 1 --2- 172.21.15.189:0/2486646924 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb9ec0a3c30 0x7fb9ec0a60f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc06aa30 tx=0x7fb9f0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:57.183 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.180+0000 7fb9f9ffb700 1 -- 172.21.15.189:0/2486646924 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb9f409db00 con 0x7fb9fc103c60 2024-04-02T13:45:57.391 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:45:57.388+0000 7fba02b52700 1 -- 172.21.15.189:0/2486646924 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb9fc062a30 con 0x7fb9ec0a3c30 2024-04-02T13:45:58.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:58 smithi189 ceph-mon[36683]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:45:58.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:58 smithi189 ceph-mon[36683]: osdmap e14: 2 total, 2 up, 2 in 2024-04-02T13:45:58.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:58 smithi189 ceph-mon[36683]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:45:58.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:58 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:45:58.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:58 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:45:58.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:45:58 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:58.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:58 smithi088 ceph-mon[36528]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:45:58.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:58 smithi088 ceph-mon[36528]: osdmap e14: 2 total, 2 up, 2 in 2024-04-02T13:45:58.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:58 smithi088 ceph-mon[36528]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:45:58.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:58 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:45:58.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:58 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:45:58.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:45:58 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:58.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:58 smithi052 ceph-mon[32104]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:45:58.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:58 smithi052 ceph-mon[32104]: osdmap e14: 2 total, 2 up, 2 in 2024-04-02T13:45:58.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:58 smithi052 ceph-mon[32104]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:45:58.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:58 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:45:58.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:58 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:45:58.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:45:58 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:00.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:00 smithi088 ceph-mon[36528]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:00.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:00 smithi088 ceph-mon[36528]: from='client.? 172.21.15.189:0/1264173091' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "142d5d34-737f-44bf-a17c-c2ad6d0be723"}]: dispatch 2024-04-02T13:46:00.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:00 smithi088 ceph-mon[36528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "142d5d34-737f-44bf-a17c-c2ad6d0be723"}]: dispatch 2024-04-02T13:46:00.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:00 smithi088 ceph-mon[36528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "142d5d34-737f-44bf-a17c-c2ad6d0be723"}]': finished 2024-04-02T13:46:00.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:00 smithi088 ceph-mon[36528]: osdmap e15: 3 total, 2 up, 3 in 2024-04-02T13:46:00.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:00 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:46:00.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:00 smithi189 ceph-mon[36683]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:00.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:00 smithi189 ceph-mon[36683]: from='client.? 172.21.15.189:0/1264173091' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "142d5d34-737f-44bf-a17c-c2ad6d0be723"}]: dispatch 2024-04-02T13:46:00.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:00 smithi189 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "142d5d34-737f-44bf-a17c-c2ad6d0be723"}]: dispatch 2024-04-02T13:46:00.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:00 smithi189 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "142d5d34-737f-44bf-a17c-c2ad6d0be723"}]': finished 2024-04-02T13:46:00.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:00 smithi189 ceph-mon[36683]: osdmap e15: 3 total, 2 up, 3 in 2024-04-02T13:46:00.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:00 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:46:00.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:00 smithi052 ceph-mon[32104]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:00.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:00 smithi052 ceph-mon[32104]: from='client.? 172.21.15.189:0/1264173091' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "142d5d34-737f-44bf-a17c-c2ad6d0be723"}]: dispatch 2024-04-02T13:46:00.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:00 smithi052 ceph-mon[32104]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "142d5d34-737f-44bf-a17c-c2ad6d0be723"}]: dispatch 2024-04-02T13:46:00.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:00 smithi052 ceph-mon[32104]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "142d5d34-737f-44bf-a17c-c2ad6d0be723"}]': finished 2024-04-02T13:46:00.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:00 smithi052 ceph-mon[32104]: osdmap e15: 3 total, 2 up, 3 in 2024-04-02T13:46:00.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:00 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:46:01.537 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:01 smithi189 ceph-mon[36683]: from='client.? 172.21.15.189:0/1566595945' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:46:01.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:01 smithi052 ceph-mon[32104]: from='client.? 172.21.15.189:0/1566595945' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:46:01.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:01 smithi088 ceph-mon[36528]: from='client.? 172.21.15.189:0/1566595945' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:46:02.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:02 smithi189 ceph-mon[36683]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:02.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:02 smithi052 ceph-mon[32104]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:02.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:02 smithi088 ceph-mon[36528]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:04.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:04 smithi189 ceph-mon[36683]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:04.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:04 smithi052 ceph-mon[32104]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:04.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:04 smithi088 ceph-mon[36528]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:06.471 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:06 smithi088 ceph-mon[36528]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:06.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:06 smithi189 ceph-mon[36683]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:06.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:06 smithi052 ceph-mon[32104]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:07.279 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:07 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T13:46:07.279 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:07 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:07.489 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:07 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T13:46:07.490 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:07 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:07.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:07 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T13:46:07.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:07 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:08.489 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:08 smithi189 ceph-mon[36683]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:08.489 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:08 smithi189 ceph-mon[36683]: Deploying daemon osd.2 on smithi189 2024-04-02T13:46:08.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:08 smithi052 ceph-mon[32104]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:08.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:08 smithi052 ceph-mon[32104]: Deploying daemon osd.2 on smithi189 2024-04-02T13:46:08.660 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:08 smithi088 ceph-mon[36528]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:08.660 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:08 smithi088 ceph-mon[36528]: Deploying daemon osd.2 on smithi189 2024-04-02T13:46:10.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:10 smithi189 ceph-mon[36683]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:10.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:10 smithi052 ceph-mon[32104]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:10.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:10 smithi088 ceph-mon[36528]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:12.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:12 smithi189 ceph-mon[36683]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:12.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:12 smithi052 ceph-mon[32104]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:12.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:12 smithi088 ceph-mon[36528]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:13.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:13 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:13.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:13 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:13.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:13 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:46:13.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:13 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:13.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:13 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:13.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:13 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:46:13.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:13 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:13.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:13 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:13.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:13 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:46:14.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:14 smithi189 ceph-mon[36683]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:14.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:14 smithi052 ceph-mon[32104]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:14.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:14 smithi088 ceph-mon[36528]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:16.007 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:46:16.004+0000 7fb9f9ffb700 1 -- 172.21.15.189:0/2486646924 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb9fc062a30 con 0x7fb9ec0a3c30 2024-04-02T13:46:16.007 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 2 on host 'smithi189' 2024-04-02T13:46:16.009 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:46:16.006+0000 7fba02b52700 1 -- 172.21.15.189:0/2486646924 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb9ec0a3c30 msgr2=0x7fb9ec0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:16.009 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:46:16.006+0000 7fba02b52700 1 --2- 172.21.15.189:0/2486646924 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb9ec0a3c30 0x7fb9ec0a60f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc06aa30 tx=0x7fb9f0008040 comp rx=0 tx=0).stop 2024-04-02T13:46:16.010 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:46:16.006+0000 7fba02b52700 1 -- 172.21.15.189:0/2486646924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9fc103c60 msgr2=0x7fb9fc19ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:16.010 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:46:16.006+0000 7fba02b52700 1 --2- 172.21.15.189:0/2486646924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9fc103c60 0x7fb9fc19ff20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb9f40095e0 tx=0x7fb9f4007770 comp rx=0 tx=0).stop 2024-04-02T13:46:16.010 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:46:16.006+0000 7fba02b52700 1 -- 172.21.15.189:0/2486646924 shutdown_connections 2024-04-02T13:46:16.010 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:46:16.006+0000 7fba02b52700 1 --2- 172.21.15.189:0/2486646924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9fc103c60 0x7fb9fc19ff20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:16.010 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:46:16.006+0000 7fba02b52700 1 --2- 172.21.15.189:0/2486646924 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb9ec0a3c30 0x7fb9ec0a60f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:16.010 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:46:16.006+0000 7fba02b52700 1 --2- 172.21.15.189:0/2486646924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9fc1006e0 0x7fb9fc198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:16.010 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:46:16.006+0000 7fba02b52700 1 --2- 172.21.15.189:0/2486646924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9fc1010a0 0x7fb9fc198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:16.010 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:46:16.006+0000 7fba02b52700 1 -- 172.21.15.189:0/2486646924 >> 172.21.15.189:0/2486646924 conn(0x7fb9fc0fc280 msgr2=0x7fb9fc106c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:16.010 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:46:16.006+0000 7fba02b52700 1 -- 172.21.15.189:0/2486646924 shutdown_connections 2024-04-02T13:46:16.010 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:46:16.006+0000 7fba02b52700 1 -- 172.21.15.189:0/2486646924 wait complete. 2024-04-02T13:46:16.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:16 smithi189 ceph-mon[36683]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:16.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:16 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:16.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:16 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:16.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:16 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:16.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:16 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:46:16.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:16 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:16.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:16 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:16.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:16 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:16.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:16 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:46:16.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:16 smithi052 ceph-mon[32104]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:16.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:16 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:16.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:16 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:16.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:16 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:16.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:16 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:46:16.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:16 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:16.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:16 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:16.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:16 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:16.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:16 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:46:16.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:16 smithi088 ceph-mon[36528]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:16.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:16 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:16.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:16 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:16.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:16 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:16.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:16 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:46:16.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:16 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:16.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:16 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:16.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:16 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:16.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:16 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:46:16.668 DEBUG:teuthology.orchestra.run.smithi189:osd.2> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.2.service 2024-04-02T13:46:16.670 INFO:tasks.cephadm:Deploying osd.3 on smithi052 with /dev/vg_nvme/lv_3... 2024-04-02T13:46:16.670 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-02T13:46:16.923 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:46:17.287 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:17 smithi189 ceph-mon[36683]: from='osd.2 [v2:172.21.15.189:6800/2416423625,v1:172.21.15.189:6801/2416423625]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T13:46:17.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:17 smithi189 ceph-mon[36683]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T13:46:17.303 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:17 smithi052 ceph-mon[32104]: from='osd.2 [v2:172.21.15.189:6800/2416423625,v1:172.21.15.189:6801/2416423625]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T13:46:17.303 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:17 smithi052 ceph-mon[32104]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T13:46:17.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:17 smithi088 ceph-mon[36528]: from='osd.2 [v2:172.21.15.189:6800/2416423625,v1:172.21.15.189:6801/2416423625]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T13:46:17.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:17 smithi088 ceph-mon[36528]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T13:46:18.165 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:18 smithi189 ceph-mon[36683]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:18.166 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:18 smithi189 ceph-mon[36683]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-02T13:46:18.166 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:18 smithi189 ceph-mon[36683]: osdmap e16: 3 total, 2 up, 3 in 2024-04-02T13:46:18.166 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:18 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:46:18.166 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:18 smithi189 ceph-mon[36683]: from='osd.2 [v2:172.21.15.189:6800/2416423625,v1:172.21.15.189:6801/2416423625]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-02T13:46:18.166 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:18 smithi189 ceph-mon[36683]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-02T13:46:18.347 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:18 smithi052 ceph-mon[32104]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:18.347 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:18 smithi052 ceph-mon[32104]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-02T13:46:18.347 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:18 smithi052 ceph-mon[32104]: osdmap e16: 3 total, 2 up, 3 in 2024-04-02T13:46:18.347 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:18 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:46:18.347 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:18 smithi052 ceph-mon[32104]: from='osd.2 [v2:172.21.15.189:6800/2416423625,v1:172.21.15.189:6801/2416423625]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-02T13:46:18.347 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:18 smithi052 ceph-mon[32104]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-02T13:46:18.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:18 smithi088 ceph-mon[36528]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:18.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:18 smithi088 ceph-mon[36528]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-02T13:46:18.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:18 smithi088 ceph-mon[36528]: osdmap e16: 3 total, 2 up, 3 in 2024-04-02T13:46:18.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:18 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:46:18.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:18 smithi088 ceph-mon[36528]: from='osd.2 [v2:172.21.15.189:6800/2416423625,v1:172.21.15.189:6801/2416423625]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-02T13:46:18.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:18 smithi088 ceph-mon[36528]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-02T13:46:19.102 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 02 13:46:18 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-2[41830]: 2024-04-02T13:46:18.844+0000 7f89cc362700 -1 osd.2 0 waiting for initial osdmap 2024-04-02T13:46:19.102 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 02 13:46:18 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-2[41830]: 2024-04-02T13:46:18.849+0000 7f89c414d700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:46:19.155 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-02T13:46:19.155 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-02T13:46:19.155 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2024-04-02T13:46:19.155 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2024-04-02T13:46:19.155 INFO:teuthology.orchestra.run.smithi052.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00829781 s, 1.3 GB/s 2024-04-02T13:46:19.155 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2024-04-02T13:46:19.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:19 smithi088 ceph-mon[36528]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-02T13:46:19.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:19 smithi088 ceph-mon[36528]: osdmap e17: 3 total, 2 up, 3 in 2024-04-02T13:46:19.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:19 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:46:19.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:19 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:46:19.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:19 smithi088 ceph-mon[36528]: from='osd.2 ' entity='osd.2' 2024-04-02T13:46:19.460 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:19 smithi052 ceph-mon[32104]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-02T13:46:19.460 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:19 smithi052 ceph-mon[32104]: osdmap e17: 3 total, 2 up, 3 in 2024-04-02T13:46:19.460 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:19 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:46:19.460 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:19 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:46:19.460 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:19 smithi052 ceph-mon[32104]: from='osd.2 ' entity='osd.2' 2024-04-02T13:46:19.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:19 smithi189 ceph-mon[36683]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-02T13:46:19.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:19 smithi189 ceph-mon[36683]: osdmap e17: 3 total, 2 up, 3 in 2024-04-02T13:46:19.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:19 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:46:19.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:19 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:46:19.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:19 smithi189 ceph-mon[36683]: from='osd.2 ' entity='osd.2' 2024-04-02T13:46:19.776 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi052:vg_nvme/lv_3 2024-04-02T13:46:20.025 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:46:20.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:20 smithi088 ceph-mon[36528]: purged_snaps scrub starts 2024-04-02T13:46:20.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:20 smithi088 ceph-mon[36528]: purged_snaps scrub ok 2024-04-02T13:46:20.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:20 smithi088 ceph-mon[36528]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:20.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:20 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:46:20.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:20 smithi088 ceph-mon[36528]: osd.2 [v2:172.21.15.189:6800/2416423625,v1:172.21.15.189:6801/2416423625] boot 2024-04-02T13:46:20.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:20 smithi088 ceph-mon[36528]: osdmap e18: 3 total, 3 up, 3 in 2024-04-02T13:46:20.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:20 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:46:20.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:20 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:20.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:20 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:20.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:20 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:46:20.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:20 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:20.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:20 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:20.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:20 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:46:20.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:20 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:20.435 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:20 smithi052 ceph-mon[32104]: purged_snaps scrub starts 2024-04-02T13:46:20.435 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:20 smithi052 ceph-mon[32104]: purged_snaps scrub ok 2024-04-02T13:46:20.435 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:20 smithi052 ceph-mon[32104]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:20.436 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:20 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:46:20.436 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:20 smithi052 ceph-mon[32104]: osd.2 [v2:172.21.15.189:6800/2416423625,v1:172.21.15.189:6801/2416423625] boot 2024-04-02T13:46:20.436 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:20 smithi052 ceph-mon[32104]: osdmap e18: 3 total, 3 up, 3 in 2024-04-02T13:46:20.436 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:20 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:46:20.436 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:20 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:20.436 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:20 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:20.436 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:20 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:46:20.436 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:20 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:20.436 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:20 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:20.436 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:20 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:46:20.436 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:20 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:20.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:20 smithi189 ceph-mon[36683]: purged_snaps scrub starts 2024-04-02T13:46:20.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:20 smithi189 ceph-mon[36683]: purged_snaps scrub ok 2024-04-02T13:46:20.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:20 smithi189 ceph-mon[36683]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:46:20.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:20 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:46:20.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:20 smithi189 ceph-mon[36683]: osd.2 [v2:172.21.15.189:6800/2416423625,v1:172.21.15.189:6801/2416423625] boot 2024-04-02T13:46:20.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:20 smithi189 ceph-mon[36683]: osdmap e18: 3 total, 3 up, 3 in 2024-04-02T13:46:20.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:20 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:46:20.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:20 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:20.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:20 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:20.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:20 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:46:20.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:20 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:20.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:20 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:20.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:20 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:46:20.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:20 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:21.212 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:21 smithi052 ceph-mon[32104]: Detected new or changed devices on smithi189 2024-04-02T13:46:21.212 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:21 smithi052 ceph-mon[32104]: Adjusting osd_memory_target on smithi189 to 17080M 2024-04-02T13:46:21.213 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:21 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-02T13:46:21.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:21 smithi088 ceph-mon[36528]: Detected new or changed devices on smithi189 2024-04-02T13:46:21.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:21 smithi088 ceph-mon[36528]: Adjusting osd_memory_target on smithi189 to 17080M 2024-04-02T13:46:21.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:21 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-02T13:46:21.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:21 smithi189 ceph-mon[36683]: Detected new or changed devices on smithi189 2024-04-02T13:46:21.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:21 smithi189 ceph-mon[36683]: Adjusting osd_memory_target on smithi189 to 17080M 2024-04-02T13:46:21.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:21 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-02T13:46:21.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.737+0000 7f89d6f3f700 1 -- 172.21.15.52:0/112740276 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89d0101810 msgr2=0x7f89d0101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:21.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.737+0000 7f89d6f3f700 1 --2- 172.21.15.52:0/112740276 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89d0101810 0x7f89d0101c70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f89c40099c0 tx=0x7f89c4009cd0 comp rx=0 tx=0).stop 2024-04-02T13:46:21.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.737+0000 7f89d6f3f700 1 -- 172.21.15.52:0/112740276 shutdown_connections 2024-04-02T13:46:21.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.737+0000 7f89d6f3f700 1 --2- 172.21.15.52:0/112740276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89d01021b0 0x7f89d0109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:21.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.737+0000 7f89d6f3f700 1 --2- 172.21.15.52:0/112740276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89d0102ad0 0x7f89d0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:21.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.737+0000 7f89d6f3f700 1 --2- 172.21.15.52:0/112740276 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89d0101810 0x7f89d0101c70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:21.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.737+0000 7f89d6f3f700 1 -- 172.21.15.52:0/112740276 >> 172.21.15.52:0/112740276 conn(0x7f89d00fc280 msgr2=0x7f89d00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:21.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.737+0000 7f89d6f3f700 1 -- 172.21.15.52:0/112740276 shutdown_connections 2024-04-02T13:46:21.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.737+0000 7f89d6f3f700 1 -- 172.21.15.52:0/112740276 wait complete. 2024-04-02T13:46:21.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.738+0000 7f89d6f3f700 1 Processor -- start 2024-04-02T13:46:21.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.738+0000 7f89d6f3f700 1 -- start start 2024-04-02T13:46:21.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.738+0000 7f89d6f3f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89d0101810 0x7f89d0105620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:21.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.738+0000 7f89d6f3f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89d01021b0 0x7f89d0103c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:21.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.738+0000 7f89d6f3f700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89d0102ad0 0x7f89d01041b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:21.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.738+0000 7f89d6f3f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f89d010d120 con 0x7f89d0101810 2024-04-02T13:46:21.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.738+0000 7f89d6f3f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f89d010cfa0 con 0x7f89d01021b0 2024-04-02T13:46:21.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.738+0000 7f89d6f3f700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f89d010d2a0 con 0x7f89d0102ad0 2024-04-02T13:46:21.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.738+0000 7f89d4cdb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89d0101810 0x7f89d0105620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:21.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.738+0000 7f89d4cdb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89d0101810 0x7f89d0105620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:32804/0 (socket says 172.21.15.52:32804) 2024-04-02T13:46:21.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.738+0000 7f89d4cdb700 1 -- 172.21.15.52:0/1703621038 learned_addr learned my addr 172.21.15.52:0/1703621038 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:46:21.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.738+0000 7f89cffff700 1 --2- 172.21.15.52:0/1703621038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89d01021b0 0x7f89d0103c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:21.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.738+0000 7f89d54dc700 1 --2- 172.21.15.52:0/1703621038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89d0102ad0 0x7f89d01041b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:21.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.738+0000 7f89d4cdb700 1 -- 172.21.15.52:0/1703621038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89d0102ad0 msgr2=0x7f89d01041b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:21.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.738+0000 7f89d4cdb700 1 --2- 172.21.15.52:0/1703621038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89d0102ad0 0x7f89d01041b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:21.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.738+0000 7f89d4cdb700 1 -- 172.21.15.52:0/1703621038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89d01021b0 msgr2=0x7f89d0103c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:21.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.738+0000 7f89d4cdb700 1 --2- 172.21.15.52:0/1703621038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89d01021b0 0x7f89d0103c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:21.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.738+0000 7f89d4cdb700 1 -- 172.21.15.52:0/1703621038 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d0104a60 con 0x7f89d0101810 2024-04-02T13:46:21.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.738+0000 7f89d54dc700 1 --2- 172.21.15.52:0/1703621038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89d0102ad0 0x7f89d01041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:46:21.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.739+0000 7f89d4cdb700 1 --2- 172.21.15.52:0/1703621038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89d0101810 0x7f89d0105620 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f89c000d980 tx=0x7f89c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:21.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.739+0000 7f89cdffb700 1 -- 172.21.15.52:0/1703621038 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0017070 con 0x7f89d0101810 2024-04-02T13:46:21.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.739+0000 7f89d6f3f700 1 -- 172.21.15.52:0/1703621038 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d01a2930 con 0x7f89d0101810 2024-04-02T13:46:21.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.739+0000 7f89cdffb700 1 -- 172.21.15.52:0/1703621038 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c0009e10 con 0x7f89d0101810 2024-04-02T13:46:21.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.739+0000 7f89cdffb700 1 -- 172.21.15.52:0/1703621038 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c000f620 con 0x7f89d0101810 2024-04-02T13:46:21.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.739+0000 7f89d6f3f700 1 -- 172.21.15.52:0/1703621038 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d01a2df0 con 0x7f89d0101810 2024-04-02T13:46:21.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.740+0000 7f89d6f3f700 1 -- 172.21.15.52:0/1703621038 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89d00692b0 con 0x7f89d0101810 2024-04-02T13:46:21.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.741+0000 7f89cdffb700 1 -- 172.21.15.52:0/1703621038 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f89c0009930 con 0x7f89d0101810 2024-04-02T13:46:21.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.742+0000 7f89cdffb700 1 --2- 172.21.15.52:0/1703621038 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f89b80a3bb0 0x7f89b80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:21.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.742+0000 7f89cffff700 1 --2- 172.21.15.52:0/1703621038 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f89b80a3bb0 0x7f89b80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:21.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.742+0000 7f89cdffb700 1 -- 172.21.15.52:0/1703621038 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7f89c00ceb20 con 0x7f89d0101810 2024-04-02T13:46:21.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.742+0000 7f89cffff700 1 --2- 172.21.15.52:0/1703621038 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f89b80a3bb0 0x7f89b80a6070 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f89c4000c00 tx=0x7f89c4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:21.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.746+0000 7f89cdffb700 1 -- 172.21.15.52:0/1703621038 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f89c009c920 con 0x7f89d0101810 2024-04-02T13:46:21.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:21.965+0000 7f89d6f3f700 1 -- 172.21.15.52:0/1703621038 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f89d01a3080 con 0x7f89b80a3bb0 2024-04-02T13:46:22.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:22 smithi052 ceph-mon[32104]: pgmap v81: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:22.228 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:22 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-02T13:46:22.228 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:22 smithi052 ceph-mon[32104]: osdmap e19: 3 total, 3 up, 3 in 2024-04-02T13:46:22.228 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:22 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T13:46:22.228 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:22 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:46:22.228 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:22 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:46:22.228 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:22 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:22.369 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 02 13:46:22 smithi189 sudo[45475]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T13:46:22.369 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 02 13:46:22 smithi189 sudo[45475]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:46:22.369 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 02 13:46:22 smithi189 sudo[45475]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:46:22.369 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 02 13:46:22 smithi189 sudo[45475]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:46:22.370 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:22 smithi189 ceph-mon[36683]: pgmap v81: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:22.370 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:22 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-02T13:46:22.370 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:22 smithi189 ceph-mon[36683]: osdmap e19: 3 total, 3 up, 3 in 2024-04-02T13:46:22.370 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:22 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T13:46:22.370 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:22 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:46:22.370 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:22 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:46:22.370 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:22 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:22.419 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 02 13:46:22 smithi088 sudo[44853]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T13:46:22.419 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 02 13:46:22 smithi088 sudo[44853]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:46:22.419 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 02 13:46:22 smithi088 sudo[44853]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:46:22.420 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 02 13:46:22 smithi088 sudo[44853]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:46:22.420 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 02 13:46:22 smithi088 sudo[44856]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T13:46:22.420 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 02 13:46:22 smithi088 sudo[44856]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:46:22.420 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 02 13:46:22 smithi088 sudo[44856]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:46:22.420 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 02 13:46:22 smithi088 sudo[44856]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:46:22.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:22 smithi088 ceph-mon[36528]: pgmap v81: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:22.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:22 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-02T13:46:22.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:22 smithi088 ceph-mon[36528]: osdmap e19: 3 total, 3 up, 3 in 2024-04-02T13:46:22.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:22 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T13:46:22.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:22 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:46:22.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:22 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:46:22.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:22 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:22.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:22 smithi052 sudo[49147]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T13:46:22.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:22 smithi052 sudo[49147]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:46:22.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:22 smithi052 sudo[49147]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:46:22.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:22 smithi052 sudo[49147]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:46:22.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:22 smithi052 sudo[49162]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T13:46:22.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:22 smithi052 sudo[49162]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:46:22.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:22 smithi052 sudo[49162]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:46:22.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:22 smithi052 sudo[49162]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:46:22.591 INFO:journalctl@ceph.osd.0.smithi052.stdout:Apr 02 13:46:22 smithi052 sudo[49141]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T13:46:22.591 INFO:journalctl@ceph.osd.0.smithi052.stdout:Apr 02 13:46:22 smithi052 sudo[49141]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:46:22.591 INFO:journalctl@ceph.osd.0.smithi052.stdout:Apr 02 13:46:22 smithi052 sudo[49141]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:46:22.591 INFO:journalctl@ceph.osd.0.smithi052.stdout:Apr 02 13:46:22 smithi052 sudo[49141]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:46:22.591 INFO:journalctl@ceph.osd.0.smithi052.stdout:Apr 02 13:46:22 smithi052 sudo[49144]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T13:46:22.591 INFO:journalctl@ceph.osd.0.smithi052.stdout:Apr 02 13:46:22 smithi052 sudo[49144]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:46:22.591 INFO:journalctl@ceph.osd.0.smithi052.stdout:Apr 02 13:46:22 smithi052 sudo[49144]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:46:22.591 INFO:journalctl@ceph.osd.0.smithi052.stdout:Apr 02 13:46:22 smithi052 sudo[49144]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:46:22.788 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 02 13:46:22 smithi189 sudo[45478]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T13:46:22.788 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 02 13:46:22 smithi189 sudo[45478]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:46:22.788 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 02 13:46:22 smithi189 sudo[45478]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:46:22.788 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 02 13:46:22 smithi189 sudo[45478]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:46:22.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:22 smithi189 sudo[45481]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T13:46:22.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:22 smithi189 sudo[45481]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:46:22.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:22 smithi189 sudo[45481]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:46:22.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:22 smithi189 sudo[45481]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:46:22.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:22 smithi189 sudo[45484]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T13:46:22.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:22 smithi189 sudo[45484]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:46:22.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:22 smithi189 sudo[45484]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:46:22.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:22 smithi189 sudo[45484]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:46:22.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:22 smithi088 sudo[44859]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T13:46:22.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:22 smithi088 sudo[44859]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:46:22.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:22 smithi088 sudo[44859]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:46:22.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:22 smithi088 sudo[44859]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:46:22.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:22 smithi088 sudo[44862]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T13:46:22.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:22 smithi088 sudo[44862]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:46:22.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:22 smithi088 sudo[44862]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:46:22.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:22 smithi088 sudo[44862]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:46:23.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:23 smithi052 ceph-mon[32104]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:46:23.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:23 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-02T13:46:23.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:23 smithi052 ceph-mon[32104]: osdmap e20: 3 total, 3 up, 3 in 2024-04-02T13:46:23.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:23 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:23.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:23 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:23.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:23 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:23.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:23 smithi052 ceph-mon[32104]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:46:23.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:23 smithi052 ceph-mon[32104]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:46:23.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:23 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:46:23.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:23 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:46:23.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:23 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:46:23.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:23 smithi052 ceph-mon[32104]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:46:23.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:23 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:46:23.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:23 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:46:23.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:23 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:46:23.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:23 smithi052 ceph-mon[32104]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:46:23.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:23 smithi052 ceph-mon[32104]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:46:23.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:23 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:46:23.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:23 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:46:23.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:23 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:46:23.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:23 smithi052 ceph-mon[32104]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:46:23.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:23.414+0000 7f89cdffb700 1 -- 172.21.15.52:0/1703621038 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f89c0098890 con 0x7f89d0101810 2024-04-02T13:46:23.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:23 smithi088 ceph-mon[36528]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:46:23.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:23 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-02T13:46:23.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:23 smithi088 ceph-mon[36528]: osdmap e20: 3 total, 3 up, 3 in 2024-04-02T13:46:23.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:23 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:23.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:23 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:23.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:23 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:23.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:23 smithi088 ceph-mon[36528]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:46:23.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:23 smithi088 ceph-mon[36528]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:46:23.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:23 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:46:23.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:23 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:46:23.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:23 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:46:23.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:23 smithi088 ceph-mon[36528]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:46:23.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:23 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:46:23.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:23 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:46:23.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:23 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:46:23.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:23 smithi088 ceph-mon[36528]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:46:23.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:23 smithi088 ceph-mon[36528]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:46:23.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:23 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:46:23.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:23 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:46:23.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:23 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:46:23.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:23 smithi088 ceph-mon[36528]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:46:23.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:23 smithi189 ceph-mon[36683]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:46:23.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:23 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-02T13:46:23.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:23 smithi189 ceph-mon[36683]: osdmap e20: 3 total, 3 up, 3 in 2024-04-02T13:46:23.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:23 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:23.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:23 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:23.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:23 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:23.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:23 smithi189 ceph-mon[36683]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:46:23.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:23 smithi189 ceph-mon[36683]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:46:23.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:23 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:46:23.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:23 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:46:23.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:23 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:46:23.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:23 smithi189 ceph-mon[36683]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:46:23.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:23 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:46:23.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:23 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:46:23.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:23 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:46:23.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:23 smithi189 ceph-mon[36683]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:46:23.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:23 smithi189 ceph-mon[36683]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:46:23.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:23 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:46:23.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:23 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:46:23.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:23 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:46:23.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:23 smithi189 ceph-mon[36683]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:46:24.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:24 smithi052 ceph-mon[32104]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:24.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:24 smithi052 ceph-mon[32104]: osdmap e21: 3 total, 3 up, 3 in 2024-04-02T13:46:24.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:24 smithi052 ceph-mon[32104]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-02T13:46:24.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:24 smithi088 ceph-mon[36528]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:24.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:24 smithi088 ceph-mon[36528]: osdmap e21: 3 total, 3 up, 3 in 2024-04-02T13:46:24.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:24 smithi088 ceph-mon[36528]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-02T13:46:24.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:24 smithi189 ceph-mon[36683]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:24.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:24 smithi189 ceph-mon[36683]: osdmap e21: 3 total, 3 up, 3 in 2024-04-02T13:46:24.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:24 smithi189 ceph-mon[36683]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-02T13:46:25.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:25 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3112510245' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4"}]: dispatch 2024-04-02T13:46:25.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:25 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3112510245' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4"}]': finished 2024-04-02T13:46:25.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:25 smithi088 ceph-mon[36528]: osdmap e22: 4 total, 3 up, 4 in 2024-04-02T13:46:25.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:25 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:46:25.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:25 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3112510245' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4"}]: dispatch 2024-04-02T13:46:25.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:25 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3112510245' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4"}]': finished 2024-04-02T13:46:25.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:25 smithi189 ceph-mon[36683]: osdmap e22: 4 total, 3 up, 4 in 2024-04-02T13:46:25.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:25 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:46:25.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:25 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3112510245' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4"}]: dispatch 2024-04-02T13:46:25.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:25 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3112510245' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4"}]': finished 2024-04-02T13:46:25.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:25 smithi052 ceph-mon[32104]: osdmap e22: 4 total, 3 up, 4 in 2024-04-02T13:46:25.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:25 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:46:26.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:26 smithi088 ceph-mon[36528]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:26.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:26 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/219437755' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:46:26.537 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:26 smithi189 ceph-mon[36683]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:26.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:26 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/219437755' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:46:26.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:26 smithi052 ceph-mon[32104]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:26.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:26 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/219437755' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:46:28.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:28 smithi088 ceph-mon[36528]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:28.535 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:28 smithi052 ceph-mon[32104]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:28.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:28 smithi189 ceph-mon[36683]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:30.219 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:30 smithi052 ceph-mon[32104]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:30.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:30 smithi088 ceph-mon[36528]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:30.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:30 smithi189 ceph-mon[36683]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:32.280 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:32 smithi052 ceph-mon[32104]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:32.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:32 smithi088 ceph-mon[36528]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:32.537 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:32 smithi189 ceph-mon[36683]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:33.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:33 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T13:46:33.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:33 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:33.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:33 smithi052 ceph-mon[32104]: Deploying daemon osd.3 on smithi052 2024-04-02T13:46:33.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:33 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T13:46:33.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:33 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:33.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:33 smithi088 ceph-mon[36528]: Deploying daemon osd.3 on smithi052 2024-04-02T13:46:33.540 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:33 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T13:46:33.540 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:33 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:33.540 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:33 smithi189 ceph-mon[36683]: Deploying daemon osd.3 on smithi052 2024-04-02T13:46:34.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:34 smithi088 ceph-mon[36528]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:34.433 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:34 smithi052 ceph-mon[32104]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:34.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:34 smithi189 ceph-mon[36683]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:36.283 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:36 smithi052 ceph-mon[32104]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:36.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:36 smithi088 ceph-mon[36528]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:36.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:36 smithi189 ceph-mon[36683]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:38.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:38 smithi088 ceph-mon[36528]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:38.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:38 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:38.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:38 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:38.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:38 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:46:38.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:38 smithi189 ceph-mon[36683]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:38.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:38 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:38.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:38 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:38.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:38 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:46:38.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:38 smithi052 ceph-mon[32104]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:38.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:38 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:38.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:38 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:38.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:38 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:46:40.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:40 smithi088 ceph-mon[36528]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:40.428 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:40 smithi052 ceph-mon[32104]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:40.537 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:40 smithi189 ceph-mon[36683]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:41.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:41.721+0000 7f89cdffb700 1 -- 172.21.15.52:0/1703621038 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f89d01a3080 con 0x7f89b80a3bb0 2024-04-02T13:46:41.724 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 3 on host 'smithi052' 2024-04-02T13:46:41.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:41.723+0000 7f89d6f3f700 1 -- 172.21.15.52:0/1703621038 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f89b80a3bb0 msgr2=0x7f89b80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:41.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:41.723+0000 7f89d6f3f700 1 --2- 172.21.15.52:0/1703621038 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f89b80a3bb0 0x7f89b80a6070 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f89c4000c00 tx=0x7f89c4005fb0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:41.723+0000 7f89d6f3f700 1 -- 172.21.15.52:0/1703621038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89d0101810 msgr2=0x7f89d0105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:41.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:41.723+0000 7f89d6f3f700 1 --2- 172.21.15.52:0/1703621038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89d0101810 0x7f89d0105620 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f89c000d980 tx=0x7f89c000dc90 comp rx=0 tx=0).stop 2024-04-02T13:46:41.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:41.723+0000 7f89d6f3f700 1 -- 172.21.15.52:0/1703621038 shutdown_connections 2024-04-02T13:46:41.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:41.723+0000 7f89d6f3f700 1 --2- 172.21.15.52:0/1703621038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89d0102ad0 0x7f89d01041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:41.723+0000 7f89d6f3f700 1 --2- 172.21.15.52:0/1703621038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89d0101810 0x7f89d0105620 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:41.723+0000 7f89d6f3f700 1 --2- 172.21.15.52:0/1703621038 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f89b80a3bb0 0x7f89b80a6070 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:41.723+0000 7f89d6f3f700 1 --2- 172.21.15.52:0/1703621038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89d01021b0 0x7f89d0103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:41.723+0000 7f89d6f3f700 1 -- 172.21.15.52:0/1703621038 >> 172.21.15.52:0/1703621038 conn(0x7f89d00fc280 msgr2=0x7f89d0100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:41.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:41.723+0000 7f89d6f3f700 1 -- 172.21.15.52:0/1703621038 shutdown_connections 2024-04-02T13:46:41.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:46:41.723+0000 7f89d6f3f700 1 -- 172.21.15.52:0/1703621038 wait complete. 2024-04-02T13:46:42.078 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:41 smithi052 ceph-mon[32104]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:42.079 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:41 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:42.079 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:41 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:42.079 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:41 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:42.079 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:41 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:46:42.079 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:41 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:42.079 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:41 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:42.079 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:41 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:42.079 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:41 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:46:42.079 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:41 smithi052 ceph-mon[32104]: from='osd.3 [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T13:46:42.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:41 smithi088 ceph-mon[36528]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:42.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:41 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:42.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:41 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:42.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:41 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:42.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:41 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:46:42.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:41 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:42.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:41 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:42.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:41 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:42.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:41 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:46:42.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:41 smithi088 ceph-mon[36528]: from='osd.3 [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T13:46:42.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:41 smithi189 ceph-mon[36683]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:42.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:41 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:42.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:41 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:42.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:41 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:42.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:41 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:46:42.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:41 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:42.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:41 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:42.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:41 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:42.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:41 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:46:42.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:41 smithi189 ceph-mon[36683]: from='osd.3 [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T13:46:42.416 DEBUG:teuthology.orchestra.run.smithi052:osd.3> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.3.service 2024-04-02T13:46:42.418 INFO:tasks.cephadm:Deploying osd.4 on smithi088 with /dev/vg_nvme/lv_3... 2024-04-02T13:46:42.419 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-02T13:46:42.662 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:46:43.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:43 smithi052 ceph-mon[32104]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:43.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:43 smithi052 ceph-mon[32104]: from='osd.3 [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-02T13:46:43.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:43 smithi052 ceph-mon[32104]: osdmap e23: 4 total, 3 up, 4 in 2024-04-02T13:46:43.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:43 smithi052 ceph-mon[32104]: from='osd.3 [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-04-02T13:46:43.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:43 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:46:44.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:43 smithi189 ceph-mon[36683]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:44.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:43 smithi189 ceph-mon[36683]: from='osd.3 [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-02T13:46:44.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:43 smithi189 ceph-mon[36683]: osdmap e23: 4 total, 3 up, 4 in 2024-04-02T13:46:44.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:43 smithi189 ceph-mon[36683]: from='osd.3 [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-04-02T13:46:44.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:43 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:46:44.041 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:43 smithi088 ceph-mon[36528]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:44.041 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:43 smithi088 ceph-mon[36528]: from='osd.3 [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-02T13:46:44.041 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:43 smithi088 ceph-mon[36528]: osdmap e23: 4 total, 3 up, 4 in 2024-04-02T13:46:44.041 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:43 smithi088 ceph-mon[36528]: from='osd.3 [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-04-02T13:46:44.042 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:43 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:46:44.567 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-02T13:46:44.567 INFO:teuthology.orchestra.run.smithi088.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-02T13:46:44.567 INFO:teuthology.orchestra.run.smithi088.stderr: stderr: 10+0 records in 2024-04-02T13:46:44.567 INFO:teuthology.orchestra.run.smithi088.stderr:10+0 records out 2024-04-02T13:46:44.567 INFO:teuthology.orchestra.run.smithi088.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110757 s, 947 MB/s 2024-04-02T13:46:44.567 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping successful for: 2024-04-02T13:46:44.589 INFO:journalctl@ceph.osd.3.smithi052.stdout:Apr 02 13:46:44 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-3[51356]: 2024-04-02T13:46:44.208+0000 7f382fb22700 -1 osd.3 0 waiting for initial osdmap 2024-04-02T13:46:44.590 INFO:journalctl@ceph.osd.3.smithi052.stdout:Apr 02 13:46:44 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-3[51356]: 2024-04-02T13:46:44.213+0000 7f382710c700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:46:45.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:44 smithi189 ceph-mon[36683]: from='osd.3 [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-04-02T13:46:45.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:44 smithi189 ceph-mon[36683]: osdmap e24: 4 total, 3 up, 4 in 2024-04-02T13:46:45.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:44 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:46:45.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:44 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:46:45.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:44 smithi189 ceph-mon[36683]: from='osd.3 [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980]' entity='osd.3' 2024-04-02T13:46:45.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:44 smithi189 ceph-mon[36683]: osd.3 [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980] boot 2024-04-02T13:46:45.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:44 smithi189 ceph-mon[36683]: osdmap e25: 4 total, 4 up, 4 in 2024-04-02T13:46:45.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:44 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:46:45.052 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:44 smithi052 ceph-mon[32104]: from='osd.3 [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-04-02T13:46:45.052 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:44 smithi052 ceph-mon[32104]: osdmap e24: 4 total, 3 up, 4 in 2024-04-02T13:46:45.052 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:44 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:46:45.052 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:44 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:46:45.053 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:44 smithi052 ceph-mon[32104]: from='osd.3 [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980]' entity='osd.3' 2024-04-02T13:46:45.053 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:44 smithi052 ceph-mon[32104]: osd.3 [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980] boot 2024-04-02T13:46:45.053 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:44 smithi052 ceph-mon[32104]: osdmap e25: 4 total, 4 up, 4 in 2024-04-02T13:46:45.053 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:44 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:46:45.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:44 smithi088 ceph-mon[36528]: from='osd.3 [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-04-02T13:46:45.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:44 smithi088 ceph-mon[36528]: osdmap e24: 4 total, 3 up, 4 in 2024-04-02T13:46:45.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:44 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:46:45.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:44 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:46:45.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:44 smithi088 ceph-mon[36528]: from='osd.3 [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980]' entity='osd.3' 2024-04-02T13:46:45.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:44 smithi088 ceph-mon[36528]: osd.3 [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980] boot 2024-04-02T13:46:45.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:44 smithi088 ceph-mon[36528]: osdmap e25: 4 total, 4 up, 4 in 2024-04-02T13:46:45.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:44 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:46:45.204 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi088:vg_nvme/lv_3 2024-04-02T13:46:45.453 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:46:45.797 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:45 smithi088 ceph-mon[36528]: purged_snaps scrub starts 2024-04-02T13:46:45.798 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:45 smithi088 ceph-mon[36528]: purged_snaps scrub ok 2024-04-02T13:46:45.798 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:45 smithi088 ceph-mon[36528]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:45.798 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:45 smithi088 ceph-mon[36528]: osdmap e26: 4 total, 4 up, 4 in 2024-04-02T13:46:45.798 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:45 smithi088 ceph-mon[36528]: Detected new or changed devices on smithi052 2024-04-02T13:46:45.798 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:45 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:45.798 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:45 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:45.798 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:45 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:46:45.798 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:45 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:46:45.798 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:45 smithi088 ceph-mon[36528]: Adjusting osd_memory_target on smithi052 to 8546M 2024-04-02T13:46:45.798 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:45 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:45.798 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:45 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:45.798 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:45 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:46:45.799 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:45 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:46.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:45 smithi189 ceph-mon[36683]: purged_snaps scrub starts 2024-04-02T13:46:46.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:45 smithi189 ceph-mon[36683]: purged_snaps scrub ok 2024-04-02T13:46:46.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:45 smithi189 ceph-mon[36683]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:46.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:45 smithi189 ceph-mon[36683]: osdmap e26: 4 total, 4 up, 4 in 2024-04-02T13:46:46.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:45 smithi189 ceph-mon[36683]: Detected new or changed devices on smithi052 2024-04-02T13:46:46.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:45 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:46.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:45 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:46.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:45 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:46:46.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:45 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:46:46.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:45 smithi189 ceph-mon[36683]: Adjusting osd_memory_target on smithi052 to 8546M 2024-04-02T13:46:46.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:45 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:46.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:45 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:46.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:45 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:46:46.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:45 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:46.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:45 smithi052 ceph-mon[32104]: purged_snaps scrub starts 2024-04-02T13:46:46.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:45 smithi052 ceph-mon[32104]: purged_snaps scrub ok 2024-04-02T13:46:46.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:45 smithi052 ceph-mon[32104]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:46:46.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:45 smithi052 ceph-mon[32104]: osdmap e26: 4 total, 4 up, 4 in 2024-04-02T13:46:46.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:45 smithi052 ceph-mon[32104]: Detected new or changed devices on smithi052 2024-04-02T13:46:46.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:45 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:46.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:45 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:46.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:45 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:46:46.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:45 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:46:46.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:45 smithi052 ceph-mon[32104]: Adjusting osd_memory_target on smithi052 to 8546M 2024-04-02T13:46:46.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:45 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:46.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:45 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:46.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:45 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:46:46.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:45 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:46:47.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.071+0000 7f9e56d31700 1 -- 172.21.15.88:0/3152961590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e50108310 msgr2=0x7f9e5010a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:47.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.071+0000 7f9e56d31700 1 --2- 172.21.15.88:0/3152961590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e50108310 0x7f9e5010a780 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800b0a0 tx=0x7f9e4800b3b0 comp rx=0 tx=0).stop 2024-04-02T13:46:47.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.071+0000 7f9e56d31700 1 -- 172.21.15.88:0/3152961590 shutdown_connections 2024-04-02T13:46:47.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.071+0000 7f9e56d31700 1 --2- 172.21.15.88:0/3152961590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e50108310 0x7f9e5010a780 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:47.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.071+0000 7f9e56d31700 1 --2- 172.21.15.88:0/3152961590 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e501030b0 0x7f9e501054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:47.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.071+0000 7f9e56d31700 1 --2- 172.21.15.88:0/3152961590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e501059e0 0x7f9e50107dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:47.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.071+0000 7f9e56d31700 1 -- 172.21.15.88:0/3152961590 >> 172.21.15.88:0/3152961590 conn(0x7f9e500fcac0 msgr2=0x7f9e500fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:47.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.071+0000 7f9e56d31700 1 -- 172.21.15.88:0/3152961590 shutdown_connections 2024-04-02T13:46:47.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.071+0000 7f9e56d31700 1 -- 172.21.15.88:0/3152961590 wait complete. 2024-04-02T13:46:47.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.072+0000 7f9e56d31700 1 Processor -- start 2024-04-02T13:46:47.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.072+0000 7f9e56d31700 1 -- start start 2024-04-02T13:46:47.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.072+0000 7f9e56d31700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e501030b0 0x7f9e50198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:47.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.072+0000 7f9e56d31700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e501059e0 0x7f9e50198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:47.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.072+0000 7f9e56d31700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e50108310 0x7f9e5019fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:47.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.072+0000 7f9e56d31700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5010e230 con 0x7f9e50108310 2024-04-02T13:46:47.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.072+0000 7f9e56d31700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5010e0b0 con 0x7f9e501059e0 2024-04-02T13:46:47.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.072+0000 7f9e56d31700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5010e3b0 con 0x7f9e501030b0 2024-04-02T13:46:47.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.072+0000 7f9e47fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e501059e0 0x7f9e50198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:47.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.072+0000 7f9e47fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e501059e0 0x7f9e50198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36056/0 (socket says 172.21.15.88:36056) 2024-04-02T13:46:47.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.072+0000 7f9e47fff700 1 -- 172.21.15.88:0/2175938653 learned_addr learned my addr 172.21.15.88:0/2175938653 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-02T13:46:47.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.072+0000 7f9e552ce700 1 --2- 172.21.15.88:0/2175938653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e50108310 0x7f9e5019fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:47.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.072+0000 7f9e54acd700 1 --2- 172.21.15.88:0/2175938653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e501030b0 0x7f9e50198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:47.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.072+0000 7f9e47fff700 1 -- 172.21.15.88:0/2175938653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e501030b0 msgr2=0x7f9e50198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:47.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.072+0000 7f9e47fff700 1 --2- 172.21.15.88:0/2175938653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e501030b0 0x7f9e50198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:47.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.072+0000 7f9e47fff700 1 -- 172.21.15.88:0/2175938653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e50108310 msgr2=0x7f9e5019fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:47.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.072+0000 7f9e47fff700 1 --2- 172.21.15.88:0/2175938653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e50108310 0x7f9e5019fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:47.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.072+0000 7f9e47fff700 1 -- 172.21.15.88:0/2175938653 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e501a04a0 con 0x7f9e501059e0 2024-04-02T13:46:47.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.072+0000 7f9e54acd700 1 --2- 172.21.15.88:0/2175938653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e501030b0 0x7f9e50198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:46:47.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.073+0000 7f9e47fff700 1 --2- 172.21.15.88:0/2175938653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e501059e0 0x7f9e50198d80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000d950 tx=0x7f9e4000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:47.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.073+0000 7f9e45ffb700 1 -- 172.21.15.88:0/2175938653 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e40016070 con 0x7f9e501059e0 2024-04-02T13:46:47.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.073+0000 7f9e56d31700 1 -- 172.21.15.88:0/2175938653 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e501a0790 con 0x7f9e501059e0 2024-04-02T13:46:47.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.073+0000 7f9e45ffb700 1 -- 172.21.15.88:0/2175938653 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e40009e20 con 0x7f9e501059e0 2024-04-02T13:46:47.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.073+0000 7f9e45ffb700 1 -- 172.21.15.88:0/2175938653 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4000f620 con 0x7f9e501059e0 2024-04-02T13:46:47.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.073+0000 7f9e56d31700 1 -- 172.21.15.88:0/2175938653 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e501a0d20 con 0x7f9e501059e0 2024-04-02T13:46:47.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.074+0000 7f9e56d31700 1 -- 172.21.15.88:0/2175938653 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e500692b0 con 0x7f9e501059e0 2024-04-02T13:46:47.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.075+0000 7f9e45ffb700 1 -- 172.21.15.88:0/2175938653 <== mon.2 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9e40019070 con 0x7f9e501059e0 2024-04-02T13:46:47.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.076+0000 7f9e45ffb700 1 --2- 172.21.15.88:0/2175938653 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f9e380a3c40 0x7f9e380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:47.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.076+0000 7f9e45ffb700 1 -- 172.21.15.88:0/2175938653 <== mon.2 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7f9e400ced80 con 0x7f9e501059e0 2024-04-02T13:46:47.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.076+0000 7f9e54acd700 1 --2- 172.21.15.88:0/2175938653 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f9e380a3c40 0x7f9e380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:47.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.077+0000 7f9e54acd700 1 --2- 172.21.15.88:0/2175938653 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f9e380a3c40 0x7f9e380a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00bcf0 tx=0x7f9e4c00b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:47.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.080+0000 7f9e45ffb700 1 -- 172.21.15.88:0/2175938653 <== mon.2 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9e4009c960 con 0x7f9e501059e0 2024-04-02T13:46:47.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:46:47.289+0000 7f9e56d31700 1 -- 172.21.15.88:0/2175938653 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9e50062af0 con 0x7f9e380a3c40 2024-04-02T13:46:47.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:47 smithi189 ceph-mon[36683]: osdmap e27: 4 total, 4 up, 4 in 2024-04-02T13:46:47.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:47 smithi052 ceph-mon[32104]: osdmap e27: 4 total, 4 up, 4 in 2024-04-02T13:46:47.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:47 smithi088 ceph-mon[36528]: osdmap e27: 4 total, 4 up, 4 in 2024-04-02T13:46:48.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:48 smithi088 ceph-mon[36528]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 384 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:46:48.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:48 smithi088 ceph-mon[36528]: osdmap e28: 4 total, 4 up, 4 in 2024-04-02T13:46:48.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:48 smithi088 ceph-mon[36528]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:46:48.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:46:48.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:46:48.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:48.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:48 smithi189 ceph-mon[36683]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 384 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:46:48.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:48 smithi189 ceph-mon[36683]: osdmap e28: 4 total, 4 up, 4 in 2024-04-02T13:46:48.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:48 smithi189 ceph-mon[36683]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:46:48.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:48 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:46:48.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:48 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:46:48.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:48 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:48.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:48 smithi052 ceph-mon[32104]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 384 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:46:48.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:48 smithi052 ceph-mon[32104]: osdmap e28: 4 total, 4 up, 4 in 2024-04-02T13:46:48.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:48 smithi052 ceph-mon[32104]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:46:48.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:48 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:46:48.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:48 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:46:48.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:48 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:49.537 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:49 smithi189 ceph-mon[36683]: osdmap e29: 4 total, 4 up, 4 in 2024-04-02T13:46:49.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:49 smithi052 ceph-mon[32104]: osdmap e29: 4 total, 4 up, 4 in 2024-04-02T13:46:49.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:49 smithi088 ceph-mon[36528]: osdmap e29: 4 total, 4 up, 4 in 2024-04-02T13:46:50.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:50 smithi189 ceph-mon[36683]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 384 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:46:50.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:50 smithi189 ceph-mon[36683]: from='client.? 172.21.15.88:0/287277406' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fbfe997-a0e7-4b89-8e6b-754a9bee10a1"}]: dispatch 2024-04-02T13:46:50.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:50 smithi189 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fbfe997-a0e7-4b89-8e6b-754a9bee10a1"}]: dispatch 2024-04-02T13:46:50.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:50 smithi189 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fbfe997-a0e7-4b89-8e6b-754a9bee10a1"}]': finished 2024-04-02T13:46:50.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:50 smithi189 ceph-mon[36683]: osdmap e30: 5 total, 4 up, 5 in 2024-04-02T13:46:50.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:50 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:46:50.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:50 smithi052 ceph-mon[32104]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 384 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:46:50.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:50 smithi052 ceph-mon[32104]: from='client.? 172.21.15.88:0/287277406' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fbfe997-a0e7-4b89-8e6b-754a9bee10a1"}]: dispatch 2024-04-02T13:46:50.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:50 smithi052 ceph-mon[32104]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fbfe997-a0e7-4b89-8e6b-754a9bee10a1"}]: dispatch 2024-04-02T13:46:50.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:50 smithi052 ceph-mon[32104]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fbfe997-a0e7-4b89-8e6b-754a9bee10a1"}]': finished 2024-04-02T13:46:50.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:50 smithi052 ceph-mon[32104]: osdmap e30: 5 total, 4 up, 5 in 2024-04-02T13:46:50.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:50 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:46:50.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:50 smithi088 ceph-mon[36528]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 384 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:46:50.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:50 smithi088 ceph-mon[36528]: from='client.? 172.21.15.88:0/287277406' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fbfe997-a0e7-4b89-8e6b-754a9bee10a1"}]: dispatch 2024-04-02T13:46:50.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:50 smithi088 ceph-mon[36528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fbfe997-a0e7-4b89-8e6b-754a9bee10a1"}]: dispatch 2024-04-02T13:46:50.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:50 smithi088 ceph-mon[36528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fbfe997-a0e7-4b89-8e6b-754a9bee10a1"}]': finished 2024-04-02T13:46:50.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:50 smithi088 ceph-mon[36528]: osdmap e30: 5 total, 4 up, 5 in 2024-04-02T13:46:50.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:50 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:46:51.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:51 smithi052 ceph-mon[32104]: from='client.? 172.21.15.88:0/2138414704' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:46:51.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:51 smithi088 ceph-mon[36528]: from='client.? 172.21.15.88:0/2138414704' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:46:51.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:51 smithi189 ceph-mon[36683]: from='client.? 172.21.15.88:0/2138414704' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:46:52.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:52 smithi052 ceph-mon[32104]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:46:52.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:52 smithi088 ceph-mon[36528]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:46:52.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:52 smithi189 ceph-mon[36683]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:46:54.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:54 smithi052 ceph-mon[32104]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T13:46:54.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:54 smithi088 ceph-mon[36528]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T13:46:54.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:54 smithi189 ceph-mon[36683]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T13:46:56.491 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:56 smithi088 ceph-mon[36528]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-02T13:46:56.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:56 smithi052 ceph-mon[32104]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-02T13:46:56.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:56 smithi189 ceph-mon[36683]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-02T13:46:58.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:58 smithi052 ceph-mon[32104]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-02T13:46:58.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:58 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T13:46:58.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:58 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:58.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:46:58 smithi052 ceph-mon[32104]: Deploying daemon osd.4 on smithi088 2024-04-02T13:46:58.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:58 smithi088 ceph-mon[36528]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-02T13:46:58.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:58 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T13:46:58.671 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:58 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:58.671 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:46:58 smithi088 ceph-mon[36528]: Deploying daemon osd.4 on smithi088 2024-04-02T13:46:58.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:58 smithi189 ceph-mon[36683]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-02T13:46:58.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:58 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T13:46:58.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:58 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:58.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:46:58 smithi189 ceph-mon[36683]: Deploying daemon osd.4 on smithi088 2024-04-02T13:47:00.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:00 smithi052 ceph-mon[32104]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T13:47:00.639 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:00 smithi088 ceph-mon[36528]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T13:47:00.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:00 smithi189 ceph-mon[36683]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T13:47:02.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:02 smithi052 ceph-mon[32104]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-02T13:47:02.660 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:02 smithi088 ceph-mon[36528]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-02T13:47:02.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:02 smithi189 ceph-mon[36683]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-02T13:47:04.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:03 smithi088 ceph-mon[36528]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-02T13:47:04.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:03 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:04.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:03 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:04.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:03 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:47:04.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:03 smithi189 ceph-mon[36683]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-02T13:47:04.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:03 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:04.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:03 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:04.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:03 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:47:04.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:03 smithi052 ceph-mon[32104]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-02T13:47:04.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:03 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:04.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:03 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:04.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:03 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:47:06.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:05 smithi088 ceph-mon[36528]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:47:06.257 INFO:teuthology.orchestra.run.smithi088.stdout:Created osd(s) 4 on host 'smithi088' 2024-04-02T13:47:06.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:47:06.253+0000 7f9e45ffb700 1 -- 172.21.15.88:0/2175938653 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9e50062af0 con 0x7f9e380a3c40 2024-04-02T13:47:06.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:47:06.255+0000 7f9e56d31700 1 -- 172.21.15.88:0/2175938653 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f9e380a3c40 msgr2=0x7f9e380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:06.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:47:06.255+0000 7f9e56d31700 1 --2- 172.21.15.88:0/2175938653 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f9e380a3c40 0x7f9e380a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00bcf0 tx=0x7f9e4c00b2c0 comp rx=0 tx=0).stop 2024-04-02T13:47:06.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:47:06.255+0000 7f9e56d31700 1 -- 172.21.15.88:0/2175938653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e501059e0 msgr2=0x7f9e50198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:06.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:47:06.255+0000 7f9e56d31700 1 --2- 172.21.15.88:0/2175938653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e501059e0 0x7f9e50198d80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000d950 tx=0x7f9e4000dc60 comp rx=0 tx=0).stop 2024-04-02T13:47:06.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:47:06.255+0000 7f9e56d31700 1 -- 172.21.15.88:0/2175938653 shutdown_connections 2024-04-02T13:47:06.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:47:06.255+0000 7f9e56d31700 1 --2- 172.21.15.88:0/2175938653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e50108310 0x7f9e5019fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:06.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:47:06.255+0000 7f9e56d31700 1 --2- 172.21.15.88:0/2175938653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e501030b0 0x7f9e50198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:06.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:47:06.255+0000 7f9e56d31700 1 --2- 172.21.15.88:0/2175938653 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f9e380a3c40 0x7f9e380a6100 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:06.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:47:06.255+0000 7f9e56d31700 1 --2- 172.21.15.88:0/2175938653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e501059e0 0x7f9e50198d80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:06.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:47:06.255+0000 7f9e56d31700 1 -- 172.21.15.88:0/2175938653 >> 172.21.15.88:0/2175938653 conn(0x7f9e500fcac0 msgr2=0x7f9e50103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:06.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:47:06.255+0000 7f9e56d31700 1 -- 172.21.15.88:0/2175938653 shutdown_connections 2024-04-02T13:47:06.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:47:06.255+0000 7f9e56d31700 1 -- 172.21.15.88:0/2175938653 wait complete. 2024-04-02T13:47:06.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:05 smithi189 ceph-mon[36683]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:47:06.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:05 smithi052 ceph-mon[32104]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:47:07.007 DEBUG:teuthology.orchestra.run.smithi088:osd.4> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.4.service 2024-04-02T13:47:07.009 INFO:tasks.cephadm:Deploying osd.5 on smithi189 with /dev/vg_nvme/lv_3... 2024-04-02T13:47:07.009 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-02T13:47:07.265 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:47:07.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:07 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:07.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:07 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:07.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:07 smithi189 ceph-mon[36683]: from='osd.4 [v2:172.21.15.88:6808/2077759625,v1:172.21.15.88:6809/2077759625]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T13:47:07.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:07 smithi189 ceph-mon[36683]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T13:47:07.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:07 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:07.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:07 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:07.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:07 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:07.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:07 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:07.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:07 smithi052 ceph-mon[32104]: from='osd.4 [v2:172.21.15.88:6808/2077759625,v1:172.21.15.88:6809/2077759625]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T13:47:07.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:07 smithi052 ceph-mon[32104]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T13:47:07.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:07 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:07.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:07 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:07.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:07 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:07.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:07 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:07.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:07 smithi088 ceph-mon[36528]: from='osd.4 [v2:172.21.15.88:6808/2077759625,v1:172.21.15.88:6809/2077759625]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T13:47:07.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:07 smithi088 ceph-mon[36528]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T13:47:07.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:07 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:07.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:07 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:08.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:08 smithi088 ceph-mon[36528]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:47:08.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:08 smithi088 ceph-mon[36528]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-02T13:47:08.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:08 smithi088 ceph-mon[36528]: osdmap e31: 5 total, 4 up, 5 in 2024-04-02T13:47:08.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:08 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:47:08.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:08 smithi088 ceph-mon[36528]: from='osd.4 [v2:172.21.15.88:6808/2077759625,v1:172.21.15.88:6809/2077759625]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-02T13:47:08.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:08 smithi088 ceph-mon[36528]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-02T13:47:08.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:08 smithi189 ceph-mon[36683]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:47:08.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:08 smithi189 ceph-mon[36683]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-02T13:47:08.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:08 smithi189 ceph-mon[36683]: osdmap e31: 5 total, 4 up, 5 in 2024-04-02T13:47:08.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:08 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:47:08.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:08 smithi189 ceph-mon[36683]: from='osd.4 [v2:172.21.15.88:6808/2077759625,v1:172.21.15.88:6809/2077759625]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-02T13:47:08.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:08 smithi189 ceph-mon[36683]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-02T13:47:08.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:08 smithi052 ceph-mon[32104]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:47:08.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:08 smithi052 ceph-mon[32104]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-02T13:47:08.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:08 smithi052 ceph-mon[32104]: osdmap e31: 5 total, 4 up, 5 in 2024-04-02T13:47:08.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:08 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:47:08.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:08 smithi052 ceph-mon[32104]: from='osd.4 [v2:172.21.15.88:6808/2077759625,v1:172.21.15.88:6809/2077759625]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-02T13:47:08.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:08 smithi052 ceph-mon[32104]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-02T13:47:09.058 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 02 13:47:08 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-4[47689]: 2024-04-02T13:47:08.921+0000 7f8654f12700 -1 osd.4 0 waiting for initial osdmap 2024-04-02T13:47:09.058 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 02 13:47:08 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-4[47689]: 2024-04-02T13:47:08.926+0000 7f864ccfd700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:47:09.165 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-02T13:47:09.165 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-02T13:47:09.165 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-04-02T13:47:09.165 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-04-02T13:47:09.165 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130466 s, 804 MB/s 2024-04-02T13:47:09.165 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-04-02T13:47:09.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:09 smithi088 ceph-mon[36528]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-04-02T13:47:09.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:09 smithi088 ceph-mon[36528]: osdmap e32: 5 total, 4 up, 5 in 2024-04-02T13:47:09.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:09 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:47:09.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:09 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:47:09.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:09 smithi088 ceph-mon[36528]: from='osd.4 ' entity='osd.4' 2024-04-02T13:47:09.468 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:09 smithi189 ceph-mon[36683]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-04-02T13:47:09.468 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:09 smithi189 ceph-mon[36683]: osdmap e32: 5 total, 4 up, 5 in 2024-04-02T13:47:09.468 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:09 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:47:09.468 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:09 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:47:09.468 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:09 smithi189 ceph-mon[36683]: from='osd.4 ' entity='osd.4' 2024-04-02T13:47:09.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:09 smithi052 ceph-mon[32104]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-04-02T13:47:09.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:09 smithi052 ceph-mon[32104]: osdmap e32: 5 total, 4 up, 5 in 2024-04-02T13:47:09.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:09 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:47:09.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:09 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:47:09.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:09 smithi052 ceph-mon[32104]: from='osd.4 ' entity='osd.4' 2024-04-02T13:47:09.780 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi189:vg_nvme/lv_3 2024-04-02T13:47:10.017 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:47:10.467 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:10 smithi088 ceph-mon[36528]: purged_snaps scrub starts 2024-04-02T13:47:10.467 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:10 smithi088 ceph-mon[36528]: purged_snaps scrub ok 2024-04-02T13:47:10.467 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:10 smithi088 ceph-mon[36528]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:47:10.468 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:10 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:47:10.468 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:10 smithi088 ceph-mon[36528]: osd.4 [v2:172.21.15.88:6808/2077759625,v1:172.21.15.88:6809/2077759625] boot 2024-04-02T13:47:10.468 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:10 smithi088 ceph-mon[36528]: osdmap e33: 5 total, 5 up, 5 in 2024-04-02T13:47:10.468 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:10 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:47:10.480 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:10 smithi189 ceph-mon[36683]: purged_snaps scrub starts 2024-04-02T13:47:10.480 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:10 smithi189 ceph-mon[36683]: purged_snaps scrub ok 2024-04-02T13:47:10.481 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:10 smithi189 ceph-mon[36683]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:47:10.481 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:10 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:47:10.481 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:10 smithi189 ceph-mon[36683]: osd.4 [v2:172.21.15.88:6808/2077759625,v1:172.21.15.88:6809/2077759625] boot 2024-04-02T13:47:10.481 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:10 smithi189 ceph-mon[36683]: osdmap e33: 5 total, 5 up, 5 in 2024-04-02T13:47:10.481 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:10 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:47:10.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:10 smithi052 ceph-mon[32104]: purged_snaps scrub starts 2024-04-02T13:47:10.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:10 smithi052 ceph-mon[32104]: purged_snaps scrub ok 2024-04-02T13:47:10.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:10 smithi052 ceph-mon[32104]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:47:10.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:10 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:47:10.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:10 smithi052 ceph-mon[32104]: osd.4 [v2:172.21.15.88:6808/2077759625,v1:172.21.15.88:6809/2077759625] boot 2024-04-02T13:47:10.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:10 smithi052 ceph-mon[32104]: osdmap e33: 5 total, 5 up, 5 in 2024-04-02T13:47:10.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:10 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:47:11.612 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.605+0000 7fc5adb13700 1 -- 172.21.15.189:0/3301458231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5a8105a10 msgr2=0x7fc5a8107e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:11.613 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.605+0000 7fc5adb13700 1 --2- 172.21.15.189:0/3301458231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5a8105a10 0x7fc5a8107e00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009990 tx=0x7fc59c009ca0 comp rx=0 tx=0).stop 2024-04-02T13:47:11.613 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.605+0000 7fc5adb13700 1 -- 172.21.15.189:0/3301458231 shutdown_connections 2024-04-02T13:47:11.613 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.605+0000 7fc5adb13700 1 --2- 172.21.15.189:0/3301458231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5a8108340 0x7fc5a810a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:11.613 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.605+0000 7fc5adb13700 1 --2- 172.21.15.189:0/3301458231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5a81030e0 0x7fc5a81054d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:11.613 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.605+0000 7fc5adb13700 1 --2- 172.21.15.189:0/3301458231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5a8105a10 0x7fc5a8107e00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:11.613 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.605+0000 7fc5adb13700 1 -- 172.21.15.189:0/3301458231 >> 172.21.15.189:0/3301458231 conn(0x7fc5a80fcb10 msgr2=0x7fc5a80fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:11.613 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.606+0000 7fc5adb13700 1 -- 172.21.15.189:0/3301458231 shutdown_connections 2024-04-02T13:47:11.613 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.606+0000 7fc5adb13700 1 -- 172.21.15.189:0/3301458231 wait complete. 2024-04-02T13:47:11.613 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.606+0000 7fc5adb13700 1 Processor -- start 2024-04-02T13:47:11.613 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.606+0000 7fc5adb13700 1 -- start start 2024-04-02T13:47:11.613 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.606+0000 7fc5adb13700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5a81030e0 0x7fc5a8102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:11.613 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.606+0000 7fc5adb13700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5a8105a10 0x7fc5a8101090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:11.613 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.606+0000 7fc5adb13700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5a8108340 0x7fc5a81015d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:11.614 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.606+0000 7fc5adb13700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a810e2a0 con 0x7fc5a8105a10 2024-04-02T13:47:11.614 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.606+0000 7fc5adb13700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a810e120 con 0x7fc5a81030e0 2024-04-02T13:47:11.614 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.606+0000 7fc5adb13700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a810e420 con 0x7fc5a8108340 2024-04-02T13:47:11.614 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.606+0000 7fc5a7fff700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5a8108340 0x7fc5a81015d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:11.614 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.606+0000 7fc5a7fff700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5a8108340 0x7fc5a81015d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:57858/0 (socket says 172.21.15.189:57858) 2024-04-02T13:47:11.614 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.607+0000 7fc5a7fff700 1 -- 172.21.15.189:0/2847086891 learned_addr learned my addr 172.21.15.189:0/2847086891 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-02T13:47:11.614 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.607+0000 7fc5a77fe700 1 --2- 172.21.15.189:0/2847086891 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5a81030e0 0x7fc5a8102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:11.614 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.607+0000 7fc5a6ffd700 1 --2- 172.21.15.189:0/2847086891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5a8105a10 0x7fc5a8101090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:11.614 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.607+0000 7fc5a7fff700 1 -- 172.21.15.189:0/2847086891 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5a81030e0 msgr2=0x7fc5a8102a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:11.614 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.607+0000 7fc5a7fff700 1 --2- 172.21.15.189:0/2847086891 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5a81030e0 0x7fc5a8102a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:11.614 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.607+0000 7fc5a7fff700 1 -- 172.21.15.189:0/2847086891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5a8105a10 msgr2=0x7fc5a8101090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:11.614 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.607+0000 7fc5a7fff700 1 --2- 172.21.15.189:0/2847086891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5a8105a10 0x7fc5a8101090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:11.615 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.607+0000 7fc5a7fff700 1 -- 172.21.15.189:0/2847086891 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a8101e80 con 0x7fc5a8108340 2024-04-02T13:47:11.615 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.607+0000 7fc5a7fff700 1 --2- 172.21.15.189:0/2847086891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5a8108340 0x7fc5a81015d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00094f0 tx=0x7fc5a000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:11.615 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.607+0000 7fc5a4ff9700 1 -- 172.21.15.189:0/2847086891 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a000ddd0 con 0x7fc5a8108340 2024-04-02T13:47:11.615 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.607+0000 7fc5a4ff9700 1 -- 172.21.15.189:0/2847086891 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5a0009e20 con 0x7fc5a8108340 2024-04-02T13:47:11.615 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.607+0000 7fc5adb13700 1 -- 172.21.15.189:0/2847086891 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a81a2a40 con 0x7fc5a8108340 2024-04-02T13:47:11.615 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.607+0000 7fc5a4ff9700 1 -- 172.21.15.189:0/2847086891 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0015660 con 0x7fc5a8108340 2024-04-02T13:47:11.615 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.607+0000 7fc5adb13700 1 -- 172.21.15.189:0/2847086891 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a81a2f80 con 0x7fc5a8108340 2024-04-02T13:47:11.615 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.608+0000 7fc5adb13700 1 -- 172.21.15.189:0/2847086891 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5a80692b0 con 0x7fc5a8108340 2024-04-02T13:47:11.615 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.609+0000 7fc5a4ff9700 1 -- 172.21.15.189:0/2847086891 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc5a0004ad0 con 0x7fc5a8108340 2024-04-02T13:47:11.615 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.610+0000 7fc5a4ff9700 1 --2- 172.21.15.189:0/2847086891 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc5900a3c40 0x7fc5900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:11.615 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.610+0000 7fc5a4ff9700 1 -- 172.21.15.189:0/2847086891 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fc5a00cf690 con 0x7fc5a8108340 2024-04-02T13:47:11.616 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.610+0000 7fc5a77fe700 1 --2- 172.21.15.189:0/2847086891 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc5900a3c40 0x7fc5900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:11.616 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.611+0000 7fc5a77fe700 1 --2- 172.21.15.189:0/2847086891 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc5900a3c40 0x7fc5900a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc598009a10 tx=0x7fc598009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:11.616 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.614+0000 7fc5a4ff9700 1 -- 172.21.15.189:0/2847086891 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc5a0010070 con 0x7fc5a8108340 2024-04-02T13:47:11.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:11 smithi189 ceph-mon[36683]: Detected new or changed devices on smithi088 2024-04-02T13:47:11.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:11 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:11.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:11 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:11.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:11 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:47:11.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:11 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:47:11.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:11 smithi189 ceph-mon[36683]: Adjusting osd_memory_target on smithi088 to 8540M 2024-04-02T13:47:11.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:11 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:11.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:11 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:11.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:11 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:47:11.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:11 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:11.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:11 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:47:11.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:11 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:11.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:11 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:47:11.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:11 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:11.825 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:11.822+0000 7fc5adb13700 1 -- 172.21.15.189:0/2847086891 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc5a8062a30 con 0x7fc5900a3c40 2024-04-02T13:47:11.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:11 smithi052 ceph-mon[32104]: Detected new or changed devices on smithi088 2024-04-02T13:47:11.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:11.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:11.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:47:11.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:47:11.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:11 smithi052 ceph-mon[32104]: Adjusting osd_memory_target on smithi088 to 8540M 2024-04-02T13:47:11.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:11.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:11.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:47:11.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:11.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:47:11.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:11.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:47:11.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:11.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:11 smithi088 ceph-mon[36528]: Detected new or changed devices on smithi088 2024-04-02T13:47:11.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:11 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:11.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:11 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:11 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:47:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:11 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:47:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:11 smithi088 ceph-mon[36528]: Adjusting osd_memory_target on smithi088 to 8540M 2024-04-02T13:47:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:11 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:11 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:11 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:47:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:11 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:11 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:47:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:11 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:11 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:47:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:11 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:12.748 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:12 smithi189 ceph-mon[36683]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 167 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:12.749 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:12 smithi189 ceph-mon[36683]: osdmap e34: 5 total, 5 up, 5 in 2024-04-02T13:47:12.749 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:12 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:47:12.749 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:12 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:47:12.749 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:12 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:12.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:12 smithi052 ceph-mon[32104]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 167 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:12.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:12 smithi052 ceph-mon[32104]: osdmap e34: 5 total, 5 up, 5 in 2024-04-02T13:47:12.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:12 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:47:12.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:12 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:47:12.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:12 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:12.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:12 smithi088 ceph-mon[36528]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 167 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:12.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:12 smithi088 ceph-mon[36528]: osdmap e34: 5 total, 5 up, 5 in 2024-04-02T13:47:12.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:12 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:47:12.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:12 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:47:12.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:12 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:13.744 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:13 smithi189 ceph-mon[36683]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:47:13.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:13 smithi052 ceph-mon[32104]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:47:13.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:13 smithi088 ceph-mon[36528]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:47:14.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:14 smithi052 ceph-mon[32104]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 167 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:14.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:14 smithi088 ceph-mon[36528]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 167 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:15.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:14 smithi189 ceph-mon[36683]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 167 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:15.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:15 smithi052 ceph-mon[32104]: from='client.? 172.21.15.189:0/1954310680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77eeb2e8-2420-4dcc-8774-beed04c6d827"}]: dispatch 2024-04-02T13:47:15.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:15 smithi052 ceph-mon[32104]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77eeb2e8-2420-4dcc-8774-beed04c6d827"}]: dispatch 2024-04-02T13:47:15.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:15 smithi052 ceph-mon[32104]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77eeb2e8-2420-4dcc-8774-beed04c6d827"}]': finished 2024-04-02T13:47:15.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:15 smithi052 ceph-mon[32104]: osdmap e35: 6 total, 5 up, 6 in 2024-04-02T13:47:15.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:15 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:47:15.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:15 smithi052 ceph-mon[32104]: from='client.? 172.21.15.189:0/3399537067' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:47:15.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:15 smithi088 ceph-mon[36528]: from='client.? 172.21.15.189:0/1954310680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77eeb2e8-2420-4dcc-8774-beed04c6d827"}]: dispatch 2024-04-02T13:47:15.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:15 smithi088 ceph-mon[36528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77eeb2e8-2420-4dcc-8774-beed04c6d827"}]: dispatch 2024-04-02T13:47:15.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:15 smithi088 ceph-mon[36528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77eeb2e8-2420-4dcc-8774-beed04c6d827"}]': finished 2024-04-02T13:47:15.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:15 smithi088 ceph-mon[36528]: osdmap e35: 6 total, 5 up, 6 in 2024-04-02T13:47:15.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:15 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:47:15.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:15 smithi088 ceph-mon[36528]: from='client.? 172.21.15.189:0/3399537067' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:47:16.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:15 smithi189 ceph-mon[36683]: from='client.? 172.21.15.189:0/1954310680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77eeb2e8-2420-4dcc-8774-beed04c6d827"}]: dispatch 2024-04-02T13:47:16.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:15 smithi189 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77eeb2e8-2420-4dcc-8774-beed04c6d827"}]: dispatch 2024-04-02T13:47:16.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:15 smithi189 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77eeb2e8-2420-4dcc-8774-beed04c6d827"}]': finished 2024-04-02T13:47:16.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:15 smithi189 ceph-mon[36683]: osdmap e35: 6 total, 5 up, 6 in 2024-04-02T13:47:16.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:15 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:47:16.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:15 smithi189 ceph-mon[36683]: from='client.? 172.21.15.189:0/3399537067' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:47:16.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:16 smithi052 ceph-mon[32104]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 167 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:16.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:16 smithi088 ceph-mon[36528]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 167 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:17.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:16 smithi189 ceph-mon[36683]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 167 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:18.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:18 smithi189 ceph-mon[36683]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:18.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:18 smithi052 ceph-mon[32104]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:18.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:18 smithi088 ceph-mon[36528]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:20.835 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:20 smithi189 ceph-mon[36683]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:20.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:20 smithi052 ceph-mon[32104]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:20.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:20 smithi088 ceph-mon[36528]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:22.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:22 smithi052 ceph-mon[32104]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:22.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:22 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T13:47:22.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:22 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:22.885 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:22 smithi189 ceph-mon[36683]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:22.885 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:22 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T13:47:22.885 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:22 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:22.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:22 smithi088 ceph-mon[36528]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:22.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:22 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T13:47:22.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:22 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:23.846 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:23 smithi189 ceph-mon[36683]: Deploying daemon osd.5 on smithi189 2024-04-02T13:47:23.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:23 smithi088 ceph-mon[36528]: Deploying daemon osd.5 on smithi189 2024-04-02T13:47:24.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:23 smithi052 ceph-mon[32104]: Deploying daemon osd.5 on smithi189 2024-04-02T13:47:24.858 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:24 smithi189 ceph-mon[36683]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:24.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:24 smithi088 ceph-mon[36528]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:24.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:24 smithi052 ceph-mon[32104]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:26.795 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:26 smithi189 ceph-mon[36683]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:26.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:26 smithi088 ceph-mon[36528]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:27.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:26 smithi052 ceph-mon[32104]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:28.727 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:28 smithi189 ceph-mon[36683]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:28.727 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:28 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:28.727 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:28 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:28.727 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:28 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:47:28.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:28 smithi088 ceph-mon[36528]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:28.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:28 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:28.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:28 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:28.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:28 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:47:29.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:28 smithi052 ceph-mon[32104]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:29.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:28 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:29.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:28 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:29.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:28 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:47:30.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:30 smithi189 ceph-mon[36683]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:30.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:30 smithi088 ceph-mon[36528]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:31.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:30 smithi052 ceph-mon[32104]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:31.349 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:31.344+0000 7fc5a4ff9700 1 -- 172.21.15.189:0/2847086891 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc5a8062a30 con 0x7fc5900a3c40 2024-04-02T13:47:31.350 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 5 on host 'smithi189' 2024-04-02T13:47:31.350 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:31.347+0000 7fc5adb13700 1 -- 172.21.15.189:0/2847086891 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc5900a3c40 msgr2=0x7fc5900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.351 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:31.347+0000 7fc5adb13700 1 --2- 172.21.15.189:0/2847086891 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc5900a3c40 0x7fc5900a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc598009a10 tx=0x7fc598009190 comp rx=0 tx=0).stop 2024-04-02T13:47:31.351 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:31.347+0000 7fc5adb13700 1 -- 172.21.15.189:0/2847086891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5a8108340 msgr2=0x7fc5a81015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:31.351 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:31.347+0000 7fc5adb13700 1 --2- 172.21.15.189:0/2847086891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5a8108340 0x7fc5a81015d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00094f0 tx=0x7fc5a000d3b0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.351 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:31.347+0000 7fc5adb13700 1 -- 172.21.15.189:0/2847086891 shutdown_connections 2024-04-02T13:47:31.351 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:31.347+0000 7fc5adb13700 1 --2- 172.21.15.189:0/2847086891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5a8108340 0x7fc5a81015d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.351 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:31.347+0000 7fc5adb13700 1 --2- 172.21.15.189:0/2847086891 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc5900a3c40 0x7fc5900a6100 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.351 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:31.347+0000 7fc5adb13700 1 --2- 172.21.15.189:0/2847086891 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5a81030e0 0x7fc5a8102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.351 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:31.347+0000 7fc5adb13700 1 --2- 172.21.15.189:0/2847086891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5a8105a10 0x7fc5a8101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:31.351 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:31.347+0000 7fc5adb13700 1 -- 172.21.15.189:0/2847086891 >> 172.21.15.189:0/2847086891 conn(0x7fc5a80fcb10 msgr2=0x7fc5a8103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:31.351 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:31.347+0000 7fc5adb13700 1 -- 172.21.15.189:0/2847086891 shutdown_connections 2024-04-02T13:47:31.351 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:47:31.347+0000 7fc5adb13700 1 -- 172.21.15.189:0/2847086891 wait complete. 2024-04-02T13:47:32.061 DEBUG:teuthology.orchestra.run.smithi189:osd.5> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.5.service 2024-04-02T13:47:32.063 INFO:tasks.cephadm:Deploying osd.6 on smithi052 with /dev/vg_nvme/lv_2... 2024-04-02T13:47:32.063 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-02T13:47:32.317 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:47:32.438 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:32 smithi052 ceph-mon[32104]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:32.438 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:32 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:32.438 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:32 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:32.438 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:32 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:32.438 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:32 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:32.439 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:32 smithi052 ceph-mon[32104]: from='osd.5 [v2:172.21.15.189:6808/3106153812,v1:172.21.15.189:6809/3106153812]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T13:47:32.439 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:32 smithi052 ceph-mon[32104]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T13:47:32.566 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:32 smithi189 ceph-mon[36683]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:32.566 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:32 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:32.566 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:32 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:32.566 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:32 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:32.566 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:32 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:32.566 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:32 smithi189 ceph-mon[36683]: from='osd.5 [v2:172.21.15.189:6808/3106153812,v1:172.21.15.189:6809/3106153812]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T13:47:32.566 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:32 smithi189 ceph-mon[36683]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T13:47:32.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:32 smithi088 ceph-mon[36528]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:32.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:32 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:32.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:32 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:32.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:32 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:32.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:32 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:32.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:32 smithi088 ceph-mon[36528]: from='osd.5 [v2:172.21.15.189:6808/3106153812,v1:172.21.15.189:6809/3106153812]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T13:47:32.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:32 smithi088 ceph-mon[36528]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T13:47:33.787 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:33 smithi189 ceph-mon[36683]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-02T13:47:33.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:33 smithi189 ceph-mon[36683]: osdmap e36: 6 total, 5 up, 6 in 2024-04-02T13:47:33.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:33 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:47:33.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:33 smithi189 ceph-mon[36683]: from='osd.5 [v2:172.21.15.189:6808/3106153812,v1:172.21.15.189:6809/3106153812]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-02T13:47:33.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:33 smithi189 ceph-mon[36683]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-02T13:47:33.788 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 13:47:33 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T13:47:33.609+0000 7f2e41010700 -1 osd.5 0 waiting for initial osdmap 2024-04-02T13:47:33.788 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 13:47:33 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T13:47:33.615+0000 7f2e3ce03700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:47:33.795 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:33 smithi052 ceph-mon[32104]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-02T13:47:33.795 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:33 smithi052 ceph-mon[32104]: osdmap e36: 6 total, 5 up, 6 in 2024-04-02T13:47:33.795 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:33 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:47:33.795 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:33 smithi052 ceph-mon[32104]: from='osd.5 [v2:172.21.15.189:6808/3106153812,v1:172.21.15.189:6809/3106153812]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-02T13:47:33.795 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:33 smithi052 ceph-mon[32104]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-02T13:47:33.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:33 smithi088 ceph-mon[36528]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-02T13:47:33.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:33 smithi088 ceph-mon[36528]: osdmap e36: 6 total, 5 up, 6 in 2024-04-02T13:47:33.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:33 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:47:33.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:33 smithi088 ceph-mon[36528]: from='osd.5 [v2:172.21.15.189:6808/3106153812,v1:172.21.15.189:6809/3106153812]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-02T13:47:33.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:33 smithi088 ceph-mon[36528]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-02T13:47:34.370 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-02T13:47:34.371 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-02T13:47:34.371 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2024-04-02T13:47:34.371 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2024-04-02T13:47:34.371 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00875387 s, 1.2 GB/s 2024-04-02T13:47:34.371 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2024-04-02T13:47:34.750 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:34 smithi052 ceph-mon[32104]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:34.750 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:34 smithi052 ceph-mon[32104]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-02T13:47:34.750 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:34 smithi052 ceph-mon[32104]: osdmap e37: 6 total, 5 up, 6 in 2024-04-02T13:47:34.750 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:34 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:47:34.750 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:34 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:47:34.770 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:34 smithi189 ceph-mon[36683]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:34.770 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:34 smithi189 ceph-mon[36683]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-02T13:47:34.770 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:34 smithi189 ceph-mon[36683]: osdmap e37: 6 total, 5 up, 6 in 2024-04-02T13:47:34.770 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:34 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:47:34.770 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:34 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:47:34.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:34 smithi088 ceph-mon[36528]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:34.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:34 smithi088 ceph-mon[36528]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-02T13:47:34.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:34 smithi088 ceph-mon[36528]: osdmap e37: 6 total, 5 up, 6 in 2024-04-02T13:47:34.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:34 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:47:34.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:34 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:47:35.053 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi052:vg_nvme/lv_2 2024-04-02T13:47:35.312 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:47:35.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:35 smithi088 ceph-mon[36528]: purged_snaps scrub starts 2024-04-02T13:47:35.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:35 smithi088 ceph-mon[36528]: purged_snaps scrub ok 2024-04-02T13:47:35.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:35 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:47:35.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:35 smithi088 ceph-mon[36528]: osd.5 [v2:172.21.15.189:6808/3106153812,v1:172.21.15.189:6809/3106153812] boot 2024-04-02T13:47:35.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:35 smithi088 ceph-mon[36528]: osdmap e38: 6 total, 6 up, 6 in 2024-04-02T13:47:35.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:35 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:47:35.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:35 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:35.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:35 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:35.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:35 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:47:35.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:35 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:47:35.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:35 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:35.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:35 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:35.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:35 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:47:35.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:35 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:35.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:35 smithi088 ceph-mon[36528]: osdmap e39: 6 total, 6 up, 6 in 2024-04-02T13:47:35.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:35 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:47:35.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:35 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:35.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:35 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:47:35.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:35 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:36.025 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:35 smithi052 ceph-mon[32104]: purged_snaps scrub starts 2024-04-02T13:47:36.025 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:35 smithi052 ceph-mon[32104]: purged_snaps scrub ok 2024-04-02T13:47:36.025 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:35 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:47:36.025 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:35 smithi052 ceph-mon[32104]: osd.5 [v2:172.21.15.189:6808/3106153812,v1:172.21.15.189:6809/3106153812] boot 2024-04-02T13:47:36.025 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:35 smithi052 ceph-mon[32104]: osdmap e38: 6 total, 6 up, 6 in 2024-04-02T13:47:36.025 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:35 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:47:36.025 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:35 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:36.025 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:35 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:36.025 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:35 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:47:36.025 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:35 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:47:36.025 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:35 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:36.025 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:35 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:36.026 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:35 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:47:36.026 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:35 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:36.026 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:35 smithi052 ceph-mon[32104]: osdmap e39: 6 total, 6 up, 6 in 2024-04-02T13:47:36.026 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:35 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:47:36.026 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:35 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:36.026 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:35 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:47:36.026 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:35 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:36.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:35 smithi189 ceph-mon[36683]: purged_snaps scrub starts 2024-04-02T13:47:36.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:35 smithi189 ceph-mon[36683]: purged_snaps scrub ok 2024-04-02T13:47:36.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:35 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:47:36.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:35 smithi189 ceph-mon[36683]: osd.5 [v2:172.21.15.189:6808/3106153812,v1:172.21.15.189:6809/3106153812] boot 2024-04-02T13:47:36.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:35 smithi189 ceph-mon[36683]: osdmap e38: 6 total, 6 up, 6 in 2024-04-02T13:47:36.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:35 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:47:36.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:35 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:36.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:35 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:36.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:35 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:47:36.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:35 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:47:36.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:35 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:36.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:35 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:36.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:35 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:47:36.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:35 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:36.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:35 smithi189 ceph-mon[36683]: osdmap e39: 6 total, 6 up, 6 in 2024-04-02T13:47:36.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:35 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:47:36.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:35 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:36.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:35 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:47:36.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:35 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:36.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.845+0000 7fc8c8325700 1 -- 172.21.15.52:0/2292629839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8c0101520 msgr2=0x7fc8c0101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:36.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.845+0000 7fc8c8325700 1 --2- 172.21.15.52:0/2292629839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8c0101520 0x7fc8c0101940 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc0099c0 tx=0x7fc8bc009cd0 comp rx=0 tx=0).stop 2024-04-02T13:47:36.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.845+0000 7fc8c8325700 1 -- 172.21.15.52:0/2292629839 shutdown_connections 2024-04-02T13:47:36.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.845+0000 7fc8c8325700 1 --2- 172.21.15.52:0/2292629839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8c01030c0 0x7fc8c0109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:36.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.845+0000 7fc8c8325700 1 --2- 172.21.15.52:0/2292629839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8c0101520 0x7fc8c0101940 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:36.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.845+0000 7fc8c8325700 1 --2- 172.21.15.52:0/2292629839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8c0102720 0x7fc8c0102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:36.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.845+0000 7fc8c8325700 1 -- 172.21.15.52:0/2292629839 >> 172.21.15.52:0/2292629839 conn(0x7fc8c00fcac0 msgr2=0x7fc8c00fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:36.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.845+0000 7fc8c8325700 1 -- 172.21.15.52:0/2292629839 shutdown_connections 2024-04-02T13:47:36.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.845+0000 7fc8c8325700 1 -- 172.21.15.52:0/2292629839 wait complete. 2024-04-02T13:47:36.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.846+0000 7fc8c8325700 1 Processor -- start 2024-04-02T13:47:36.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.846+0000 7fc8c8325700 1 -- start start 2024-04-02T13:47:36.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.846+0000 7fc8c8325700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8c0101520 0x7fc8c01977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:36.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.846+0000 7fc8c8325700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8c0102720 0x7fc8c0197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:36.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.846+0000 7fc8c8325700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8c01030c0 0x7fc8c019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:36.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.846+0000 7fc8c8325700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c010d170 con 0x7fc8c0101520 2024-04-02T13:47:36.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.846+0000 7fc8c8325700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c010cff0 con 0x7fc8c01030c0 2024-04-02T13:47:36.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.846+0000 7fc8c8325700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c010d2f0 con 0x7fc8c0102720 2024-04-02T13:47:36.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.846+0000 7fc8c60c1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8c0101520 0x7fc8c01977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:36.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.846+0000 7fc8c60c1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8c0101520 0x7fc8c01977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38022/0 (socket says 172.21.15.52:38022) 2024-04-02T13:47:36.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.846+0000 7fc8c60c1700 1 -- 172.21.15.52:0/3421086729 learned_addr learned my addr 172.21.15.52:0/3421086729 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:47:36.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.847+0000 7fc8c68c2700 1 --2- 172.21.15.52:0/3421086729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8c01030c0 0x7fc8c019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:36.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.847+0000 7fc8c58c0700 1 --2- 172.21.15.52:0/3421086729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8c0102720 0x7fc8c0197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:36.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.847+0000 7fc8c60c1700 1 -- 172.21.15.52:0/3421086729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8c0102720 msgr2=0x7fc8c0197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:36.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.847+0000 7fc8c60c1700 1 --2- 172.21.15.52:0/3421086729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8c0102720 0x7fc8c0197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:36.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.847+0000 7fc8c60c1700 1 -- 172.21.15.52:0/3421086729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8c01030c0 msgr2=0x7fc8c019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:36.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.847+0000 7fc8c60c1700 1 --2- 172.21.15.52:0/3421086729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8c01030c0 0x7fc8c019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:36.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.847+0000 7fc8c60c1700 1 -- 172.21.15.52:0/3421086729 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c019f440 con 0x7fc8c0101520 2024-04-02T13:47:36.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.847+0000 7fc8c58c0700 1 --2- 172.21.15.52:0/3421086729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8c0102720 0x7fc8c0197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:47:36.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.847+0000 7fc8c60c1700 1 --2- 172.21.15.52:0/3421086729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8c0101520 0x7fc8c01977e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009f90 tx=0x7fc8bc009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:36.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.847+0000 7fc8b77fe700 1 -- 172.21.15.52:0/3421086729 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc027070 con 0x7fc8c0101520 2024-04-02T13:47:36.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.847+0000 7fc8b77fe700 1 -- 172.21.15.52:0/3421086729 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8bc003f40 con 0x7fc8c0101520 2024-04-02T13:47:36.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.847+0000 7fc8c8325700 1 -- 172.21.15.52:0/3421086729 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c019f6d0 con 0x7fc8c0101520 2024-04-02T13:47:36.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.847+0000 7fc8b77fe700 1 -- 172.21.15.52:0/3421086729 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc022ca0 con 0x7fc8c0101520 2024-04-02T13:47:36.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.847+0000 7fc8c8325700 1 -- 172.21.15.52:0/3421086729 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c019fc80 con 0x7fc8c0101520 2024-04-02T13:47:36.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.848+0000 7fc8c8325700 1 -- 172.21.15.52:0/3421086729 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8c00692b0 con 0x7fc8c0101520 2024-04-02T13:47:36.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.849+0000 7fc8b77fe700 1 -- 172.21.15.52:0/3421086729 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc8bc02a070 con 0x7fc8c0101520 2024-04-02T13:47:36.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.850+0000 7fc8b77fe700 1 --2- 172.21.15.52:0/3421086729 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc8ac0a3cc0 0x7fc8ac0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:36.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.850+0000 7fc8c58c0700 1 --2- 172.21.15.52:0/3421086729 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc8ac0a3cc0 0x7fc8ac0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:36.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.850+0000 7fc8b77fe700 1 -- 172.21.15.52:0/3421086729 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7fc8bc0d0280 con 0x7fc8c0101520 2024-04-02T13:47:36.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.850+0000 7fc8c58c0700 1 --2- 172.21.15.52:0/3421086729 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc8ac0a3cc0 0x7fc8ac0a6180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0005fd0 tx=0x7fc8b0005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:36.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:36.854+0000 7fc8b77fe700 1 -- 172.21.15.52:0/3421086729 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc8bc05ef20 con 0x7fc8c0101520 2024-04-02T13:47:36.857 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:36 smithi052 ceph-mon[32104]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:36.857 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:36 smithi052 ceph-mon[32104]: Detected new or changed devices on smithi189 2024-04-02T13:47:36.858 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:36 smithi052 ceph-mon[32104]: Adjusting osd_memory_target on smithi189 to 8540M 2024-04-02T13:47:36.858 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:36 smithi052 ceph-mon[32104]: osdmap e40: 6 total, 6 up, 6 in 2024-04-02T13:47:36.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:36 smithi088 ceph-mon[36528]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:36.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:36 smithi088 ceph-mon[36528]: Detected new or changed devices on smithi189 2024-04-02T13:47:36.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:36 smithi088 ceph-mon[36528]: Adjusting osd_memory_target on smithi189 to 8540M 2024-04-02T13:47:36.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:36 smithi088 ceph-mon[36528]: osdmap e40: 6 total, 6 up, 6 in 2024-04-02T13:47:37.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:36 smithi189 ceph-mon[36683]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:47:37.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:36 smithi189 ceph-mon[36683]: Detected new or changed devices on smithi189 2024-04-02T13:47:37.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:36 smithi189 ceph-mon[36683]: Adjusting osd_memory_target on smithi189 to 8540M 2024-04-02T13:47:37.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:36 smithi189 ceph-mon[36683]: osdmap e40: 6 total, 6 up, 6 in 2024-04-02T13:47:37.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:37.073+0000 7fc8c8325700 1 -- 172.21.15.52:0/3421086729 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc8c0062a30 con 0x7fc8ac0a3cc0 2024-04-02T13:47:37.917 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:37 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:47:37.917 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:37 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:47:37.917 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:37 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:37.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:37 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:47:37.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:37 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:47:37.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:37 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:38.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:37 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:47:38.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:37 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:47:38.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:37 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:38.696 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:38 smithi052 ceph-mon[32104]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:47:38.696 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:38 smithi052 ceph-mon[32104]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:47:38.696 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:38 smithi052 ceph-mon[32104]: osdmap e41: 6 total, 6 up, 6 in 2024-04-02T13:47:38.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:38 smithi088 ceph-mon[36528]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:47:38.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:38 smithi088 ceph-mon[36528]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:47:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:38 smithi088 ceph-mon[36528]: osdmap e41: 6 total, 6 up, 6 in 2024-04-02T13:47:39.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:38 smithi189 ceph-mon[36683]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:47:39.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:38 smithi189 ceph-mon[36683]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:47:39.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:38 smithi189 ceph-mon[36683]: osdmap e41: 6 total, 6 up, 6 in 2024-04-02T13:47:39.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:39 smithi088 ceph-mon[36528]: osdmap e42: 6 total, 6 up, 6 in 2024-04-02T13:47:40.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:39 smithi189 ceph-mon[36683]: osdmap e42: 6 total, 6 up, 6 in 2024-04-02T13:47:40.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:39 smithi052 ceph-mon[32104]: osdmap e42: 6 total, 6 up, 6 in 2024-04-02T13:47:40.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:40 smithi088 ceph-mon[36528]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:47:40.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:40 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2430825505' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17099b55-abab-4f7b-8cab-c9573cd9b2f4"}]: dispatch 2024-04-02T13:47:40.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:40 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2430825505' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17099b55-abab-4f7b-8cab-c9573cd9b2f4"}]': finished 2024-04-02T13:47:40.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:40 smithi088 ceph-mon[36528]: osdmap e43: 7 total, 6 up, 7 in 2024-04-02T13:47:40.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:40 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:47:40.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:40 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2044767482' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:47:41.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:40 smithi189 ceph-mon[36683]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:47:41.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:40 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2430825505' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17099b55-abab-4f7b-8cab-c9573cd9b2f4"}]: dispatch 2024-04-02T13:47:41.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:40 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2430825505' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17099b55-abab-4f7b-8cab-c9573cd9b2f4"}]': finished 2024-04-02T13:47:41.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:40 smithi189 ceph-mon[36683]: osdmap e43: 7 total, 6 up, 7 in 2024-04-02T13:47:41.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:40 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:47:41.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:40 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2044767482' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:47:41.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:40 smithi052 ceph-mon[32104]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 194 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:47:41.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:40 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2430825505' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17099b55-abab-4f7b-8cab-c9573cd9b2f4"}]: dispatch 2024-04-02T13:47:41.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:40 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2430825505' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17099b55-abab-4f7b-8cab-c9573cd9b2f4"}]': finished 2024-04-02T13:47:41.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:40 smithi052 ceph-mon[32104]: osdmap e43: 7 total, 6 up, 7 in 2024-04-02T13:47:41.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:40 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:47:41.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:40 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2044767482' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:47:42.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:42 smithi088 ceph-mon[36528]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:47:43.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:42 smithi189 ceph-mon[36683]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:47:43.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:42 smithi052 ceph-mon[32104]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:47:44.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:44 smithi052 ceph-mon[32104]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T13:47:44.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:44 smithi088 ceph-mon[36528]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T13:47:45.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:44 smithi189 ceph-mon[36683]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T13:47:46.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:46 smithi088 ceph-mon[36528]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-02T13:47:46.970 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:46 smithi052 ceph-mon[32104]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-02T13:47:47.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:46 smithi189 ceph-mon[36683]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-02T13:47:47.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:47 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-02T13:47:47.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:47 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:47.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:47 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-02T13:47:47.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:47 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:48.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:47 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-02T13:47:48.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:47 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:48.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:48 smithi088 ceph-mon[36528]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-02T13:47:48.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:48 smithi088 ceph-mon[36528]: Deploying daemon osd.6 on smithi052 2024-04-02T13:47:48.983 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:48 smithi052 ceph-mon[32104]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-02T13:47:48.983 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:48 smithi052 ceph-mon[32104]: Deploying daemon osd.6 on smithi052 2024-04-02T13:47:49.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:48 smithi189 ceph-mon[36683]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-02T13:47:49.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:48 smithi189 ceph-mon[36683]: Deploying daemon osd.6 on smithi052 2024-04-02T13:47:49.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:49 smithi052 ceph-mon[32104]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T13:47:49.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:49 smithi088 ceph-mon[36528]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T13:47:50.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:49 smithi189 ceph-mon[36683]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T13:47:52.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:51 smithi189 ceph-mon[36683]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-04-02T13:47:52.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:51 smithi052 ceph-mon[32104]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-04-02T13:47:52.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:51 smithi088 ceph-mon[36528]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-04-02T13:47:54.535 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:54 smithi052 ceph-mon[32104]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-02T13:47:54.536 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:54 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:54.536 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:54 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:54.536 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:54 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:47:54.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:54 smithi088 ceph-mon[36528]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-02T13:47:54.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:54 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:54.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:54 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:54.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:54 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:47:54.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:54 smithi189 ceph-mon[36683]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-02T13:47:54.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:54 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:54.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:54 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:54.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:54 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:47:56.663 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:56 smithi052 ceph-mon[32104]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:47:56.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:56 smithi088 ceph-mon[36528]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:47:56.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:56 smithi189 ceph-mon[36683]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:47:57.050 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 6 on host 'smithi052' 2024-04-02T13:47:57.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:57.046+0000 7fc8b77fe700 1 -- 172.21.15.52:0/3421086729 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc8c0062a30 con 0x7fc8ac0a3cc0 2024-04-02T13:47:57.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:57.048+0000 7fc8c8325700 1 -- 172.21.15.52:0/3421086729 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc8ac0a3cc0 msgr2=0x7fc8ac0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:57.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:57.048+0000 7fc8c8325700 1 --2- 172.21.15.52:0/3421086729 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc8ac0a3cc0 0x7fc8ac0a6180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0005fd0 tx=0x7fc8b0005ee0 comp rx=0 tx=0).stop 2024-04-02T13:47:57.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:57.049+0000 7fc8c8325700 1 -- 172.21.15.52:0/3421086729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8c0101520 msgr2=0x7fc8c01977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:57.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:57.049+0000 7fc8c8325700 1 --2- 172.21.15.52:0/3421086729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8c0101520 0x7fc8c01977e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009f90 tx=0x7fc8bc009580 comp rx=0 tx=0).stop 2024-04-02T13:47:57.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:57.049+0000 7fc8c8325700 1 -- 172.21.15.52:0/3421086729 shutdown_connections 2024-04-02T13:47:57.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:57.049+0000 7fc8c8325700 1 --2- 172.21.15.52:0/3421086729 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc8ac0a3cc0 0x7fc8ac0a6180 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:57.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:57.049+0000 7fc8c8325700 1 --2- 172.21.15.52:0/3421086729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8c01030c0 0x7fc8c019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:57.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:57.049+0000 7fc8c8325700 1 --2- 172.21.15.52:0/3421086729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8c0101520 0x7fc8c01977e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:57.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:57.049+0000 7fc8c8325700 1 --2- 172.21.15.52:0/3421086729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8c0102720 0x7fc8c0197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:57.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:57.049+0000 7fc8c8325700 1 -- 172.21.15.52:0/3421086729 >> 172.21.15.52:0/3421086729 conn(0x7fc8c00fcac0 msgr2=0x7fc8c0105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:57.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:57.049+0000 7fc8c8325700 1 -- 172.21.15.52:0/3421086729 shutdown_connections 2024-04-02T13:47:57.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:47:57.049+0000 7fc8c8325700 1 -- 172.21.15.52:0/3421086729 wait complete. 2024-04-02T13:47:57.884 DEBUG:teuthology.orchestra.run.smithi052:osd.6> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.6.service 2024-04-02T13:47:57.887 INFO:tasks.cephadm:Deploying osd.7 on smithi088 with /dev/vg_nvme/lv_2... 2024-04-02T13:47:57.887 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-02T13:47:58.135 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:47:58.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:58 smithi088 ceph-mon[36528]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:47:58.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:58 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:58.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:58 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:58.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:58 smithi088 ceph-mon[36528]: from='osd.6 [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-02T13:47:58.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:58 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:58.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:58 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:58.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:58 smithi189 ceph-mon[36683]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:47:58.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:58 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:58.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:58 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:58.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:58 smithi189 ceph-mon[36683]: from='osd.6 [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-02T13:47:58.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:58 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:58.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:58 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:58.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:58 smithi052 ceph-mon[32104]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:47:58.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:58 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:58.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:58 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:58.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:58 smithi052 ceph-mon[32104]: from='osd.6 [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-02T13:47:58.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:58 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:58.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:58 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:47:59.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:59 smithi189 ceph-mon[36683]: from='osd.6 [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-02T13:47:59.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:59 smithi189 ceph-mon[36683]: osdmap e44: 7 total, 6 up, 7 in 2024-04-02T13:47:59.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:59 smithi189 ceph-mon[36683]: from='osd.6 [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-04-02T13:47:59.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:47:59 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:47:59.301 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:59 smithi088 ceph-mon[36528]: from='osd.6 [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-02T13:47:59.301 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:59 smithi088 ceph-mon[36528]: osdmap e44: 7 total, 6 up, 7 in 2024-04-02T13:47:59.301 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:59 smithi088 ceph-mon[36528]: from='osd.6 [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-04-02T13:47:59.301 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:47:59 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:47:59.342 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:59 smithi052 ceph-mon[32104]: from='osd.6 [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-02T13:47:59.343 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:59 smithi052 ceph-mon[32104]: osdmap e44: 7 total, 6 up, 7 in 2024-04-02T13:47:59.343 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:59 smithi052 ceph-mon[32104]: from='osd.6 [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-04-02T13:47:59.343 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:47:59 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:48:00.016 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 13:47:59 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T13:47:59.679+0000 7f50a34f5700 -1 osd.6 0 waiting for initial osdmap 2024-04-02T13:48:00.017 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 13:47:59 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T13:47:59.688+0000 7f509b2e0700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:48:00.164 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-02T13:48:00.165 INFO:teuthology.orchestra.run.smithi088.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-02T13:48:00.165 INFO:teuthology.orchestra.run.smithi088.stderr: stderr: 10+0 records in 2024-04-02T13:48:00.165 INFO:teuthology.orchestra.run.smithi088.stderr:10+0 records out 2024-04-02T13:48:00.165 INFO:teuthology.orchestra.run.smithi088.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109997 s, 953 MB/s 2024-04-02T13:48:00.165 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping successful for: 2024-04-02T13:48:00.189 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:00 smithi088 ceph-mon[36528]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:48:00.189 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:00 smithi088 ceph-mon[36528]: from='osd.6 [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-04-02T13:48:00.189 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:00 smithi088 ceph-mon[36528]: osdmap e45: 7 total, 6 up, 7 in 2024-04-02T13:48:00.190 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:00 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:48:00.190 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:00 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:48:00.190 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:00 smithi088 ceph-mon[36528]: from='osd.6 [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509]' entity='osd.6' 2024-04-02T13:48:00.274 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:00 smithi052 ceph-mon[32104]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:48:00.274 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:00 smithi052 ceph-mon[32104]: from='osd.6 [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-04-02T13:48:00.275 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:00 smithi052 ceph-mon[32104]: osdmap e45: 7 total, 6 up, 7 in 2024-04-02T13:48:00.275 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:00 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:48:00.275 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:00 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:48:00.275 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:00 smithi052 ceph-mon[32104]: from='osd.6 [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509]' entity='osd.6' 2024-04-02T13:48:00.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:00 smithi189 ceph-mon[36683]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:48:00.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:00 smithi189 ceph-mon[36683]: from='osd.6 [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-04-02T13:48:00.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:00 smithi189 ceph-mon[36683]: osdmap e45: 7 total, 6 up, 7 in 2024-04-02T13:48:00.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:00 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:48:00.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:00 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:48:00.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:00 smithi189 ceph-mon[36683]: from='osd.6 [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509]' entity='osd.6' 2024-04-02T13:48:00.812 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi088:vg_nvme/lv_2 2024-04-02T13:48:01.076 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:48:01.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:01 smithi088 ceph-mon[36528]: purged_snaps scrub starts 2024-04-02T13:48:01.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:01 smithi088 ceph-mon[36528]: purged_snaps scrub ok 2024-04-02T13:48:01.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:01 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:48:01.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:01 smithi088 ceph-mon[36528]: osd.6 [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509] boot 2024-04-02T13:48:01.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:01 smithi088 ceph-mon[36528]: osdmap e46: 7 total, 7 up, 7 in 2024-04-02T13:48:01.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:01 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:48:01.277 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:01 smithi052 ceph-mon[32104]: purged_snaps scrub starts 2024-04-02T13:48:01.278 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:01 smithi052 ceph-mon[32104]: purged_snaps scrub ok 2024-04-02T13:48:01.278 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:01 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:48:01.278 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:01 smithi052 ceph-mon[32104]: osd.6 [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509] boot 2024-04-02T13:48:01.278 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:01 smithi052 ceph-mon[32104]: osdmap e46: 7 total, 7 up, 7 in 2024-04-02T13:48:01.278 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:01 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:48:01.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:01 smithi189 ceph-mon[36683]: purged_snaps scrub starts 2024-04-02T13:48:01.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:01 smithi189 ceph-mon[36683]: purged_snaps scrub ok 2024-04-02T13:48:01.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:01 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:48:01.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:01 smithi189 ceph-mon[36683]: osd.6 [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509] boot 2024-04-02T13:48:01.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:01 smithi189 ceph-mon[36683]: osdmap e46: 7 total, 7 up, 7 in 2024-04-02T13:48:01.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:01 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:48:02.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:02 smithi088 ceph-mon[36528]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:48:02.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:02 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:02.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:02 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:02.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:02 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:02.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:02 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:02.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:02 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:02.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:02 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:02.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:02 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:02.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:02 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:48:02.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:02 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:02.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:02 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:48:02.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:02 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:02.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:02 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:48:02.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:02 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:02.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:02 smithi052 ceph-mon[32104]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:48:02.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:02 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:02.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:02 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:02.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:02 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:02.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:02 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:02.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:02 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:02.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:02 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:02.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:02 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:02.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:02 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:48:02.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:02 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:02.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:02 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:48:02.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:02 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:02.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:02 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:48:02.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:02 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:02.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:02 smithi189 ceph-mon[36683]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:48:02.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:02 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:02.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:02 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:02.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:02 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:02.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:02 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:02.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:02 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:02.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:02 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:02.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:02 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:02.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:02 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:48:02.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:02 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:02.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:02 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:48:02.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:02 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:02.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:02 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:48:02.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:02 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:02.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.719+0000 7f81b5af1700 1 -- 172.21.15.88:0/2164486021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b0102660 msgr2=0x7f81b0102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:02.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.719+0000 7f81b5af1700 1 --2- 172.21.15.88:0/2164486021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b0102660 0x7f81b0102a80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f81a00099c0 tx=0x7f81a0009cd0 comp rx=0 tx=0).stop 2024-04-02T13:48:02.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.719+0000 7f81b5af1700 1 -- 172.21.15.88:0/2164486021 shutdown_connections 2024-04-02T13:48:02.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.719+0000 7f81b5af1700 1 --2- 172.21.15.88:0/2164486021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81b0104200 0x7f81b010aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:02.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.719+0000 7f81b5af1700 1 --2- 172.21.15.88:0/2164486021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b0102660 0x7f81b0102a80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:02.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.719+0000 7f81b5af1700 1 --2- 172.21.15.88:0/2164486021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81b0103860 0x7f81b0103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:02.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.719+0000 7f81b5af1700 1 -- 172.21.15.88:0/2164486021 >> 172.21.15.88:0/2164486021 conn(0x7f81b00fdc00 msgr2=0x7f81b0100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:02.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.719+0000 7f81b5af1700 1 -- 172.21.15.88:0/2164486021 shutdown_connections 2024-04-02T13:48:02.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.719+0000 7f81b5af1700 1 -- 172.21.15.88:0/2164486021 wait complete. 2024-04-02T13:48:02.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.719+0000 7f81b5af1700 1 Processor -- start 2024-04-02T13:48:02.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.719+0000 7f81b5af1700 1 -- start start 2024-04-02T13:48:02.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.720+0000 7f81b5af1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b0102660 0x7f81b0198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:02.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.720+0000 7f81b5af1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81b0103860 0x7f81b0198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:02.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.720+0000 7f81b5af1700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81b0104200 0x7f81b019fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:02.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.720+0000 7f81b5af1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f81b010e240 con 0x7f81b0103860 2024-04-02T13:48:02.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.720+0000 7f81b5af1700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f81b010e0c0 con 0x7f81b0102660 2024-04-02T13:48:02.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.720+0000 7f81b5af1700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f81b010e3c0 con 0x7f81b0104200 2024-04-02T13:48:02.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.720+0000 7f81af7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b0102660 0x7f81b0198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:02.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.720+0000 7f81af7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b0102660 0x7f81b0198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35700/0 (socket says 172.21.15.88:35700) 2024-04-02T13:48:02.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.720+0000 7f81af7fe700 1 -- 172.21.15.88:0/1564006410 learned_addr learned my addr 172.21.15.88:0/1564006410 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-02T13:48:02.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.720+0000 7f81affff700 1 --2- 172.21.15.88:0/1564006410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81b0104200 0x7f81b019fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:02.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.720+0000 7f81a6ffd700 1 --2- 172.21.15.88:0/1564006410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81b0103860 0x7f81b0198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:02.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.720+0000 7f81af7fe700 1 -- 172.21.15.88:0/1564006410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81b0104200 msgr2=0x7f81b019fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:02.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.720+0000 7f81af7fe700 1 --2- 172.21.15.88:0/1564006410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81b0104200 0x7f81b019fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:02.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.720+0000 7f81af7fe700 1 -- 172.21.15.88:0/1564006410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81b0103860 msgr2=0x7f81b0198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:02.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.720+0000 7f81af7fe700 1 --2- 172.21.15.88:0/1564006410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81b0103860 0x7f81b0198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:02.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.720+0000 7f81af7fe700 1 -- 172.21.15.88:0/1564006410 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b01a04d0 con 0x7f81b0102660 2024-04-02T13:48:02.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.720+0000 7f81affff700 1 --2- 172.21.15.88:0/1564006410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81b0104200 0x7f81b019fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:48:02.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.720+0000 7f81a6ffd700 1 --2- 172.21.15.88:0/1564006410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81b0103860 0x7f81b0198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:48:02.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.720+0000 7f81af7fe700 1 --2- 172.21.15.88:0/1564006410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b0102660 0x7f81b0198870 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f81a000bff0 tx=0x7f81a0005b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:02.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.720+0000 7f81ad7fa700 1 -- 172.21.15.88:0/1564006410 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0027070 con 0x7f81b0102660 2024-04-02T13:48:02.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.721+0000 7f81ad7fa700 1 -- 172.21.15.88:0/1564006410 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81a0004150 con 0x7f81b0102660 2024-04-02T13:48:02.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.721+0000 7f81b5af1700 1 -- 172.21.15.88:0/1564006410 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b01a0760 con 0x7f81b0102660 2024-04-02T13:48:02.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.721+0000 7f81ad7fa700 1 -- 172.21.15.88:0/1564006410 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0022db0 con 0x7f81b0102660 2024-04-02T13:48:02.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.721+0000 7f81b5af1700 1 -- 172.21.15.88:0/1564006410 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b01a0c40 con 0x7f81b0102660 2024-04-02T13:48:02.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.721+0000 7f81b5af1700 1 -- 172.21.15.88:0/1564006410 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b00692b0 con 0x7f81b0102660 2024-04-02T13:48:02.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.723+0000 7f81ad7fa700 1 -- 172.21.15.88:0/1564006410 <== mon.2 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f81a002a070 con 0x7f81b0102660 2024-04-02T13:48:02.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.724+0000 7f81ad7fa700 1 --2- 172.21.15.88:0/1564006410 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f819c0a3c40 0x7f819c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:02.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.724+0000 7f81ad7fa700 1 -- 172.21.15.88:0/1564006410 <== mon.2 v2:172.21.15.88:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f81a00d0c30 con 0x7f81b0102660 2024-04-02T13:48:02.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.724+0000 7f81a6ffd700 1 --2- 172.21.15.88:0/1564006410 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f819c0a3c40 0x7f819c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:02.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.724+0000 7f81a6ffd700 1 --2- 172.21.15.88:0/1564006410 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f819c0a3c40 0x7f819c0a6100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8198005fd0 tx=0x7f8198005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:02.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.727+0000 7f81ad7fa700 1 -- 172.21.15.88:0/1564006410 <== mon.2 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f81a0031070 con 0x7f81b0102660 2024-04-02T13:48:02.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:02.935+0000 7f81b5af1700 1 -- 172.21.15.88:0/1564006410 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f81b0062a30 con 0x7f819c0a3c40 2024-04-02T13:48:03.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:03 smithi052 ceph-mon[32104]: Detected new or changed devices on smithi052 2024-04-02T13:48:03.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:03 smithi052 ceph-mon[32104]: Adjusting osd_memory_target on smithi052 to 5697M 2024-04-02T13:48:03.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:03 smithi052 ceph-mon[32104]: osdmap e47: 7 total, 7 up, 7 in 2024-04-02T13:48:03.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:03 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:48:03.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:03 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:48:03.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:03 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:03.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:03 smithi088 ceph-mon[36528]: Detected new or changed devices on smithi052 2024-04-02T13:48:03.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:03 smithi088 ceph-mon[36528]: Adjusting osd_memory_target on smithi052 to 5697M 2024-04-02T13:48:03.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:03 smithi088 ceph-mon[36528]: osdmap e47: 7 total, 7 up, 7 in 2024-04-02T13:48:03.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:03 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:48:03.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:03 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:48:03.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:03 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:03.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:03 smithi189 ceph-mon[36683]: Detected new or changed devices on smithi052 2024-04-02T13:48:03.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:03 smithi189 ceph-mon[36683]: Adjusting osd_memory_target on smithi052 to 5697M 2024-04-02T13:48:03.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:03 smithi189 ceph-mon[36683]: osdmap e47: 7 total, 7 up, 7 in 2024-04-02T13:48:03.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:03 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:48:03.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:03 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:48:03.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:03 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:04.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:04 smithi088 ceph-mon[36528]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:04.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:04 smithi088 ceph-mon[36528]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:48:04.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:04 smithi052 ceph-mon[32104]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:04.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:04 smithi052 ceph-mon[32104]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:48:04.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:04 smithi189 ceph-mon[36683]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:04.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:04 smithi189 ceph-mon[36683]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:48:06.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:06 smithi052 ceph-mon[32104]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:06.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:06 smithi052 ceph-mon[32104]: from='client.? 172.21.15.88:0/1981436085' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fe3487a-3c01-483b-95ca-71ce42e74bcc"}]: dispatch 2024-04-02T13:48:06.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:06 smithi052 ceph-mon[32104]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fe3487a-3c01-483b-95ca-71ce42e74bcc"}]: dispatch 2024-04-02T13:48:06.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:06 smithi052 ceph-mon[32104]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fe3487a-3c01-483b-95ca-71ce42e74bcc"}]': finished 2024-04-02T13:48:06.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:06 smithi052 ceph-mon[32104]: osdmap e48: 8 total, 7 up, 8 in 2024-04-02T13:48:06.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:06 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:48:06.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:06 smithi189 ceph-mon[36683]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:06.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:06 smithi189 ceph-mon[36683]: from='client.? 172.21.15.88:0/1981436085' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fe3487a-3c01-483b-95ca-71ce42e74bcc"}]: dispatch 2024-04-02T13:48:06.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:06 smithi189 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fe3487a-3c01-483b-95ca-71ce42e74bcc"}]: dispatch 2024-04-02T13:48:06.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:06 smithi189 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fe3487a-3c01-483b-95ca-71ce42e74bcc"}]': finished 2024-04-02T13:48:06.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:06 smithi189 ceph-mon[36683]: osdmap e48: 8 total, 7 up, 8 in 2024-04-02T13:48:06.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:06 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:48:06.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:06 smithi088 ceph-mon[36528]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:06.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:06 smithi088 ceph-mon[36528]: from='client.? 172.21.15.88:0/1981436085' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fe3487a-3c01-483b-95ca-71ce42e74bcc"}]: dispatch 2024-04-02T13:48:06.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:06 smithi088 ceph-mon[36528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fe3487a-3c01-483b-95ca-71ce42e74bcc"}]: dispatch 2024-04-02T13:48:06.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:06 smithi088 ceph-mon[36528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fe3487a-3c01-483b-95ca-71ce42e74bcc"}]': finished 2024-04-02T13:48:06.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:06 smithi088 ceph-mon[36528]: osdmap e48: 8 total, 7 up, 8 in 2024-04-02T13:48:06.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:06 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:48:07.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:07 smithi052 ceph-mon[32104]: from='client.? 172.21.15.88:0/1523899462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:48:07.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:07 smithi088 ceph-mon[36528]: from='client.? 172.21.15.88:0/1523899462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:48:07.537 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:07 smithi189 ceph-mon[36683]: from='client.? 172.21.15.88:0/1523899462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:48:08.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:08 smithi052 ceph-mon[32104]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:08.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:08 smithi088 ceph-mon[36528]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:08.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:08 smithi189 ceph-mon[36683]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:10.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:10 smithi052 ceph-mon[32104]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:10.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:10 smithi088 ceph-mon[36528]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:10.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:10 smithi189 ceph-mon[36683]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:12.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:12 smithi052 ceph-mon[32104]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:12.363 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:12 smithi088 ceph-mon[36528]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:12.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:12 smithi189 ceph-mon[36683]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:14.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:14 smithi052 ceph-mon[32104]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:14.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:14 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-02T13:48:14.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:14 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:14.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:14 smithi052 ceph-mon[32104]: Deploying daemon osd.7 on smithi088 2024-04-02T13:48:14.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:14 smithi088 ceph-mon[36528]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:14.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:14 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-02T13:48:14.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:14 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:14.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:14 smithi088 ceph-mon[36528]: Deploying daemon osd.7 on smithi088 2024-04-02T13:48:14.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:14 smithi189 ceph-mon[36683]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:14.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:14 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-02T13:48:14.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:14 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:14.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:14 smithi189 ceph-mon[36683]: Deploying daemon osd.7 on smithi088 2024-04-02T13:48:16.328 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:16 smithi088 ceph-mon[36528]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:16.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:16 smithi052 ceph-mon[32104]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:16.537 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:16 smithi189 ceph-mon[36683]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:18.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:18 smithi052 ceph-mon[32104]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:18.384 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:18 smithi088 ceph-mon[36528]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:18.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:18 smithi189 ceph-mon[36683]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:19.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:19 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:19.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:19 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:19.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:19 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:48:20.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:19 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:20.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:19 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:20.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:19 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:48:20.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:19 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:20.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:19 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:20.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:19 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:48:20.793 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:20 smithi088 ceph-mon[36528]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:21.037 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:20 smithi189 ceph-mon[36683]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:21.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:20 smithi052 ceph-mon[32104]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:21.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:21.954+0000 7f81ad7fa700 1 -- 172.21.15.88:0/1564006410 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f81b0062a30 con 0x7f819c0a3c40 2024-04-02T13:48:21.960 INFO:teuthology.orchestra.run.smithi088.stdout:Created osd(s) 7 on host 'smithi088' 2024-04-02T13:48:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:21.956+0000 7f81b5af1700 1 -- 172.21.15.88:0/1564006410 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f819c0a3c40 msgr2=0x7f819c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:21.956+0000 7f81b5af1700 1 --2- 172.21.15.88:0/1564006410 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f819c0a3c40 0x7f819c0a6100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8198005fd0 tx=0x7f8198005dc0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:21.956+0000 7f81b5af1700 1 -- 172.21.15.88:0/1564006410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b0102660 msgr2=0x7f81b0198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:21.956+0000 7f81b5af1700 1 --2- 172.21.15.88:0/1564006410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b0102660 0x7f81b0198870 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f81a000bff0 tx=0x7f81a0005b00 comp rx=0 tx=0).stop 2024-04-02T13:48:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:21.956+0000 7f81b5af1700 1 -- 172.21.15.88:0/1564006410 shutdown_connections 2024-04-02T13:48:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:21.956+0000 7f81b5af1700 1 --2- 172.21.15.88:0/1564006410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81b0104200 0x7f81b019fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:21.956+0000 7f81b5af1700 1 --2- 172.21.15.88:0/1564006410 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f819c0a3c40 0x7f819c0a6100 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:21.956+0000 7f81b5af1700 1 --2- 172.21.15.88:0/1564006410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b0102660 0x7f81b0198870 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:21.956+0000 7f81b5af1700 1 --2- 172.21.15.88:0/1564006410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81b0103860 0x7f81b0198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:21.956+0000 7f81b5af1700 1 -- 172.21.15.88:0/1564006410 >> 172.21.15.88:0/1564006410 conn(0x7f81b00fdc00 msgr2=0x7f81b0106b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:21.956+0000 7f81b5af1700 1 -- 172.21.15.88:0/1564006410 shutdown_connections 2024-04-02T13:48:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:48:21.956+0000 7f81b5af1700 1 -- 172.21.15.88:0/1564006410 wait complete. 2024-04-02T13:48:22.562 DEBUG:teuthology.orchestra.run.smithi088:osd.7> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.7.service 2024-04-02T13:48:22.564 INFO:tasks.cephadm:Deploying osd.8 on smithi189 with /dev/vg_nvme/lv_2... 2024-04-02T13:48:22.564 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-02T13:48:22.814 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:48:22.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:22 smithi088 ceph-mon[36528]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:22.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:22 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:22.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:22 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:22.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:22 smithi088 ceph-mon[36528]: from='osd.7 [v2:172.21.15.88:6816/714628313,v1:172.21.15.88:6817/714628313]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T13:48:22.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:22 smithi088 ceph-mon[36528]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T13:48:22.936 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:22 smithi189 ceph-mon[36683]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:22.936 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:22 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:22.936 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:22 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:22.936 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:22 smithi189 ceph-mon[36683]: from='osd.7 [v2:172.21.15.88:6816/714628313,v1:172.21.15.88:6817/714628313]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T13:48:22.936 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:22 smithi189 ceph-mon[36683]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T13:48:23.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:22 smithi052 ceph-mon[32104]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:23.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:22 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:23.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:22 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:23.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:22 smithi052 ceph-mon[32104]: from='osd.7 [v2:172.21.15.88:6816/714628313,v1:172.21.15.88:6817/714628313]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T13:48:23.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:22 smithi052 ceph-mon[32104]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T13:48:24.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:23 smithi189 ceph-mon[36683]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:24.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:23 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:24.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:23 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:24.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:23 smithi189 ceph-mon[36683]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-02T13:48:24.042 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:23 smithi189 ceph-mon[36683]: from='osd.7 [v2:172.21.15.88:6816/714628313,v1:172.21.15.88:6817/714628313]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-02T13:48:24.042 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:23 smithi189 ceph-mon[36683]: osdmap e49: 8 total, 7 up, 8 in 2024-04-02T13:48:24.042 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:23 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:48:24.042 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:23 smithi189 ceph-mon[36683]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-02T13:48:24.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:23 smithi052 ceph-mon[32104]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:24.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:23 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:24.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:23 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:24.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:23 smithi052 ceph-mon[32104]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-02T13:48:24.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:23 smithi052 ceph-mon[32104]: from='osd.7 [v2:172.21.15.88:6816/714628313,v1:172.21.15.88:6817/714628313]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-02T13:48:24.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:23 smithi052 ceph-mon[32104]: osdmap e49: 8 total, 7 up, 8 in 2024-04-02T13:48:24.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:23 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:48:24.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:23 smithi052 ceph-mon[32104]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-02T13:48:24.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:23 smithi088 ceph-mon[36528]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:24.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:23 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:24.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:23 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:24.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:23 smithi088 ceph-mon[36528]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-02T13:48:24.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:23 smithi088 ceph-mon[36528]: from='osd.7 [v2:172.21.15.88:6816/714628313,v1:172.21.15.88:6817/714628313]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-02T13:48:24.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:23 smithi088 ceph-mon[36528]: osdmap e49: 8 total, 7 up, 8 in 2024-04-02T13:48:24.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:23 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:48:24.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:23 smithi088 ceph-mon[36528]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-02T13:48:24.915 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-02T13:48:24.915 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-02T13:48:24.915 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-04-02T13:48:24.915 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-04-02T13:48:24.915 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127882 s, 820 MB/s 2024-04-02T13:48:24.916 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-04-02T13:48:24.948 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 13:48:24 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T13:48:24.672+0000 7f9d43c0b700 -1 osd.7 0 waiting for initial osdmap 2024-04-02T13:48:24.949 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 13:48:24 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T13:48:24.680+0000 7f9d3b9f6700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:48:25.254 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:24 smithi189 ceph-mon[36683]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-04-02T13:48:25.254 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:24 smithi189 ceph-mon[36683]: osdmap e50: 8 total, 7 up, 8 in 2024-04-02T13:48:25.255 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:24 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:48:25.255 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:24 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:48:25.255 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:24 smithi189 ceph-mon[36683]: from='osd.7 ' entity='osd.7' 2024-04-02T13:48:25.279 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:24 smithi052 ceph-mon[32104]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-04-02T13:48:25.280 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:24 smithi052 ceph-mon[32104]: osdmap e50: 8 total, 7 up, 8 in 2024-04-02T13:48:25.280 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:24 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:48:25.280 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:24 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:48:25.280 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:24 smithi052 ceph-mon[32104]: from='osd.7 ' entity='osd.7' 2024-04-02T13:48:25.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:24 smithi088 ceph-mon[36528]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-04-02T13:48:25.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:24 smithi088 ceph-mon[36528]: osdmap e50: 8 total, 7 up, 8 in 2024-04-02T13:48:25.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:24 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:48:25.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:24 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:48:25.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:24 smithi088 ceph-mon[36528]: from='osd.7 ' entity='osd.7' 2024-04-02T13:48:25.582 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi189:vg_nvme/lv_2 2024-04-02T13:48:25.830 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:48:26.168 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:25 smithi088 ceph-mon[36528]: purged_snaps scrub starts 2024-04-02T13:48:26.168 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:25 smithi088 ceph-mon[36528]: purged_snaps scrub ok 2024-04-02T13:48:26.168 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:25 smithi088 ceph-mon[36528]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:26.168 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:25 smithi088 ceph-mon[36528]: osd.7 [v2:172.21.15.88:6816/714628313,v1:172.21.15.88:6817/714628313] boot 2024-04-02T13:48:26.168 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:25 smithi088 ceph-mon[36528]: osdmap e51: 8 total, 8 up, 8 in 2024-04-02T13:48:26.168 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:25 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:48:26.168 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:25 smithi088 ceph-mon[36528]: osdmap e52: 8 total, 8 up, 8 in 2024-04-02T13:48:26.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:25 smithi189 ceph-mon[36683]: purged_snaps scrub starts 2024-04-02T13:48:26.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:25 smithi189 ceph-mon[36683]: purged_snaps scrub ok 2024-04-02T13:48:26.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:25 smithi189 ceph-mon[36683]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:26.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:25 smithi189 ceph-mon[36683]: osd.7 [v2:172.21.15.88:6816/714628313,v1:172.21.15.88:6817/714628313] boot 2024-04-02T13:48:26.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:25 smithi189 ceph-mon[36683]: osdmap e51: 8 total, 8 up, 8 in 2024-04-02T13:48:26.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:25 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:48:26.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:25 smithi189 ceph-mon[36683]: osdmap e52: 8 total, 8 up, 8 in 2024-04-02T13:48:26.244 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:25 smithi052 ceph-mon[32104]: purged_snaps scrub starts 2024-04-02T13:48:26.244 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:25 smithi052 ceph-mon[32104]: purged_snaps scrub ok 2024-04-02T13:48:26.244 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:25 smithi052 ceph-mon[32104]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:48:26.244 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:25 smithi052 ceph-mon[32104]: osd.7 [v2:172.21.15.88:6816/714628313,v1:172.21.15.88:6817/714628313] boot 2024-04-02T13:48:26.244 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:25 smithi052 ceph-mon[32104]: osdmap e51: 8 total, 8 up, 8 in 2024-04-02T13:48:26.244 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:25 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:48:26.244 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:25 smithi052 ceph-mon[32104]: osdmap e52: 8 total, 8 up, 8 in 2024-04-02T13:48:27.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:27 smithi189 ceph-mon[36683]: Detected new or changed devices on smithi088 2024-04-02T13:48:27.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:27 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:27.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:27 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:27.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:27 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:27.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:27 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:27.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:27 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:27.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:27 smithi189 ceph-mon[36683]: Adjusting osd_memory_target on smithi088 to 5693M 2024-04-02T13:48:27.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:27 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:27.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:27 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:27.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:27 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:48:27.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:27 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:27.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:27 smithi189 ceph-mon[36683]: osdmap e53: 8 total, 8 up, 8 in 2024-04-02T13:48:27.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:27 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:48:27.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:27 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:27.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:27 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:48:27.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:27 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:27.546 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.541+0000 7f399f096700 1 -- 172.21.15.189:0/2585394073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3998101810 msgr2=0x7f3998101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:27.546 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.541+0000 7f399f096700 1 --2- 172.21.15.189:0/2585394073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3998101810 0x7f3998101c70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f39840099c0 tx=0x7f3984009cd0 comp rx=0 tx=0).stop 2024-04-02T13:48:27.546 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.541+0000 7f399f096700 1 -- 172.21.15.189:0/2585394073 shutdown_connections 2024-04-02T13:48:27.546 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.541+0000 7f399f096700 1 --2- 172.21.15.189:0/2585394073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39981021b0 0x7f3998109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:27.546 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.541+0000 7f399f096700 1 --2- 172.21.15.189:0/2585394073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3998102ad0 0x7f3998102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:27.546 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.541+0000 7f399f096700 1 --2- 172.21.15.189:0/2585394073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3998101810 0x7f3998101c70 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:27.546 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.541+0000 7f399f096700 1 -- 172.21.15.189:0/2585394073 >> 172.21.15.189:0/2585394073 conn(0x7f39980fc240 msgr2=0x7f39980fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:27.546 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.541+0000 7f399f096700 1 -- 172.21.15.189:0/2585394073 shutdown_connections 2024-04-02T13:48:27.546 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.541+0000 7f399f096700 1 -- 172.21.15.189:0/2585394073 wait complete. 2024-04-02T13:48:27.546 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.542+0000 7f399f096700 1 Processor -- start 2024-04-02T13:48:27.547 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.542+0000 7f399f096700 1 -- start start 2024-04-02T13:48:27.547 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.542+0000 7f399f096700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3998101810 0x7f3998198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:27.547 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.542+0000 7f399f096700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39981021b0 0x7f3998198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:27.547 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.542+0000 7f399f096700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3998102ad0 0x7f399819fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:27.547 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.542+0000 7f399f096700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f399810d130 con 0x7f39981021b0 2024-04-02T13:48:27.547 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.542+0000 7f399f096700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f399810cfb0 con 0x7f3998101810 2024-04-02T13:48:27.547 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.542+0000 7f399f096700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f399810d2b0 con 0x7f3998102ad0 2024-04-02T13:48:27.547 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.542+0000 7f399d633700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3998102ad0 0x7f399819fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:27.547 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.542+0000 7f399d633700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3998102ad0 0x7f399819fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:38410/0 (socket says 172.21.15.189:38410) 2024-04-02T13:48:27.547 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.542+0000 7f399d633700 1 -- 172.21.15.189:0/3141833846 learned_addr learned my addr 172.21.15.189:0/3141833846 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-02T13:48:27.549 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.542+0000 7f399ce32700 1 --2- 172.21.15.189:0/3141833846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3998101810 0x7f3998198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:27.549 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.542+0000 7f3997fff700 1 --2- 172.21.15.189:0/3141833846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39981021b0 0x7f3998198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:27.550 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.543+0000 7f399d633700 1 -- 172.21.15.189:0/3141833846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3998101810 msgr2=0x7f3998198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:27.550 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.543+0000 7f399d633700 1 --2- 172.21.15.189:0/3141833846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3998101810 0x7f3998198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:27.550 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.543+0000 7f399d633700 1 -- 172.21.15.189:0/3141833846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39981021b0 msgr2=0x7f3998198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:27.550 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.543+0000 7f399d633700 1 --2- 172.21.15.189:0/3141833846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39981021b0 0x7f3998198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:27.550 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.543+0000 7f399d633700 1 -- 172.21.15.189:0/3141833846 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3998103a40 con 0x7f3998102ad0 2024-04-02T13:48:27.550 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.543+0000 7f399d633700 1 --2- 172.21.15.189:0/3141833846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3998102ad0 0x7f399819fe50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f399000add0 tx=0x7f399000c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:27.550 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.543+0000 7f3995ffb700 1 -- 172.21.15.189:0/3141833846 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3990017020 con 0x7f3998102ad0 2024-04-02T13:48:27.550 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.543+0000 7f399f096700 1 -- 172.21.15.189:0/3141833846 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3998103d30 con 0x7f3998102ad0 2024-04-02T13:48:27.550 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.543+0000 7f3995ffb700 1 -- 172.21.15.189:0/3141833846 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f399000ae00 con 0x7f3998102ad0 2024-04-02T13:48:27.550 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.543+0000 7f3995ffb700 1 -- 172.21.15.189:0/3141833846 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39900053e0 con 0x7f3998102ad0 2024-04-02T13:48:27.550 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.543+0000 7f399f096700 1 -- 172.21.15.189:0/3141833846 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39981042c0 con 0x7f3998102ad0 2024-04-02T13:48:27.550 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.544+0000 7f399f096700 1 -- 172.21.15.189:0/3141833846 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39980692b0 con 0x7f3998102ad0 2024-04-02T13:48:27.550 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.545+0000 7f3995ffb700 1 -- 172.21.15.189:0/3141833846 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f399000c890 con 0x7f3998102ad0 2024-04-02T13:48:27.551 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.546+0000 7f3995ffb700 1 --2- 172.21.15.189:0/3141833846 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f39880a3c40 0x7f39880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:27.551 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.546+0000 7f3995ffb700 1 -- 172.21.15.189:0/3141833846 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5397+0+0 (secure 0 0 0) 0x7f39900cefd0 con 0x7f3998102ad0 2024-04-02T13:48:27.551 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.546+0000 7f399ce32700 1 --2- 172.21.15.189:0/3141833846 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f39880a3c40 0x7f39880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:27.551 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.546+0000 7f399ce32700 1 --2- 172.21.15.189:0/3141833846 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f39880a3c40 0x7f39880a6100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f399806aaf0 tx=0x7f398c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:27.553 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.550+0000 7f3995ffb700 1 -- 172.21.15.189:0/3141833846 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3990021020 con 0x7f3998102ad0 2024-04-02T13:48:27.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:27 smithi052 ceph-mon[32104]: Detected new or changed devices on smithi088 2024-04-02T13:48:27.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:27.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:27.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:27.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:27.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:27.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:27 smithi052 ceph-mon[32104]: Adjusting osd_memory_target on smithi088 to 5693M 2024-04-02T13:48:27.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:27.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:27.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:48:27.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:27.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:27 smithi052 ceph-mon[32104]: osdmap e53: 8 total, 8 up, 8 in 2024-04-02T13:48:27.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:48:27.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:27.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:48:27.592 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:27.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:27 smithi088 ceph-mon[36528]: Detected new or changed devices on smithi088 2024-04-02T13:48:27.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:27 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:27.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:27 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:27.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:27 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:27.671 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:27 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:27.671 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:27 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:27.671 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:27 smithi088 ceph-mon[36528]: Adjusting osd_memory_target on smithi088 to 5693M 2024-04-02T13:48:27.671 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:27 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:27.671 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:27 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:27.671 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:27 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:48:27.671 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:27 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:27.671 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:27 smithi088 ceph-mon[36528]: osdmap e53: 8 total, 8 up, 8 in 2024-04-02T13:48:27.671 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:27 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:48:27.671 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:27 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:27.671 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:27 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:48:27.671 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:27 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:27.761 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:27.757+0000 7f399f096700 1 -- 172.21.15.189:0/3141833846 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f39980fe3e0 con 0x7f39880a3c40 2024-04-02T13:48:28.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:28 smithi088 ceph-mon[36528]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:28.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:28 smithi088 ceph-mon[36528]: osdmap e54: 8 total, 8 up, 8 in 2024-04-02T13:48:28.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:28 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:48:28.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:28 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:48:28.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:28 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:28.703 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:28 smithi189 ceph-mon[36683]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:28.703 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:28 smithi189 ceph-mon[36683]: osdmap e54: 8 total, 8 up, 8 in 2024-04-02T13:48:28.703 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:28 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:48:28.703 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:28 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:48:28.703 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:28 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:28.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:28 smithi052 ceph-mon[32104]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:28.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:28 smithi052 ceph-mon[32104]: osdmap e54: 8 total, 8 up, 8 in 2024-04-02T13:48:28.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:28 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:48:28.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:28 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:48:28.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:28 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:29.667 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:29 smithi189 ceph-mon[36683]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:48:29.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:29 smithi189 ceph-mon[36683]: osdmap e55: 8 total, 8 up, 8 in 2024-04-02T13:48:29.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:29 smithi088 ceph-mon[36528]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:48:29.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:29 smithi088 ceph-mon[36528]: osdmap e55: 8 total, 8 up, 8 in 2024-04-02T13:48:29.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:29 smithi052 ceph-mon[32104]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:48:29.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:29 smithi052 ceph-mon[32104]: osdmap e55: 8 total, 8 up, 8 in 2024-04-02T13:48:30.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:30 smithi088 ceph-mon[36528]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:30.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:30 smithi189 ceph-mon[36683]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:30.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:30 smithi052 ceph-mon[32104]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:31.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:31 smithi088 ceph-mon[36528]: from='client.? 172.21.15.189:0/1823681876' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f93ee6a0-9a7b-4090-9035-a169686c4b9b"}]: dispatch 2024-04-02T13:48:31.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:31 smithi088 ceph-mon[36528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f93ee6a0-9a7b-4090-9035-a169686c4b9b"}]: dispatch 2024-04-02T13:48:31.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:31 smithi088 ceph-mon[36528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f93ee6a0-9a7b-4090-9035-a169686c4b9b"}]': finished 2024-04-02T13:48:31.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:31 smithi088 ceph-mon[36528]: osdmap e56: 9 total, 8 up, 9 in 2024-04-02T13:48:31.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:31 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:48:31.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:31 smithi088 ceph-mon[36528]: from='client.? 172.21.15.189:0/2783112636' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:48:31.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:31 smithi189 ceph-mon[36683]: from='client.? 172.21.15.189:0/1823681876' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f93ee6a0-9a7b-4090-9035-a169686c4b9b"}]: dispatch 2024-04-02T13:48:31.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:31 smithi189 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f93ee6a0-9a7b-4090-9035-a169686c4b9b"}]: dispatch 2024-04-02T13:48:31.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:31 smithi189 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f93ee6a0-9a7b-4090-9035-a169686c4b9b"}]': finished 2024-04-02T13:48:31.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:31 smithi189 ceph-mon[36683]: osdmap e56: 9 total, 8 up, 9 in 2024-04-02T13:48:31.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:31 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:48:31.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:31 smithi189 ceph-mon[36683]: from='client.? 172.21.15.189:0/2783112636' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:48:31.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:31 smithi052 ceph-mon[32104]: from='client.? 172.21.15.189:0/1823681876' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f93ee6a0-9a7b-4090-9035-a169686c4b9b"}]: dispatch 2024-04-02T13:48:31.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:31 smithi052 ceph-mon[32104]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f93ee6a0-9a7b-4090-9035-a169686c4b9b"}]: dispatch 2024-04-02T13:48:31.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:31 smithi052 ceph-mon[32104]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f93ee6a0-9a7b-4090-9035-a169686c4b9b"}]': finished 2024-04-02T13:48:31.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:31 smithi052 ceph-mon[32104]: osdmap e56: 9 total, 8 up, 9 in 2024-04-02T13:48:31.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:31 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:48:31.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:31 smithi052 ceph-mon[32104]: from='client.? 172.21.15.189:0/2783112636' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:48:32.787 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:32 smithi189 ceph-mon[36683]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:32.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:32 smithi052 ceph-mon[32104]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:32.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:32 smithi088 ceph-mon[36528]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:34.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:34 smithi189 ceph-mon[36683]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:34.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:34 smithi052 ceph-mon[32104]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:34.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:34 smithi088 ceph-mon[36528]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:36.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:36 smithi052 ceph-mon[32104]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:36.690 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:36 smithi189 ceph-mon[36683]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:36.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:36 smithi088 ceph-mon[36528]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:38.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:38 smithi189 ceph-mon[36683]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:38.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:38 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-02T13:48:38.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:38 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:38.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:38 smithi052 ceph-mon[32104]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:38.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:38 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-02T13:48:38.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:38 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:38 smithi088 ceph-mon[36528]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:38 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-02T13:48:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:38 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:39.823 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:39 smithi189 ceph-mon[36683]: Deploying daemon osd.8 on smithi189 2024-04-02T13:48:39.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:39 smithi052 ceph-mon[32104]: Deploying daemon osd.8 on smithi189 2024-04-02T13:48:39.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:39 smithi088 ceph-mon[36528]: Deploying daemon osd.8 on smithi189 2024-04-02T13:48:40.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:40 smithi052 ceph-mon[32104]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:40.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:40 smithi088 ceph-mon[36528]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:41.028 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:40 smithi189 ceph-mon[36683]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:42.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:42 smithi189 ceph-mon[36683]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:42.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:42 smithi052 ceph-mon[32104]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:42.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:42 smithi088 ceph-mon[36528]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:44.599 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:44 smithi189 ceph-mon[36683]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:44.599 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:44 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:44.600 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:44 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:44.600 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:44 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:48:44.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:44 smithi052 ceph-mon[32104]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:44.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:44 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:44.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:44 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:44.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:44 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:48:44.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:44 smithi088 ceph-mon[36528]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:44.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:44 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:44.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:44 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:44.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:44 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:48:46.674 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:46 smithi189 ceph-mon[36683]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:46.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:46 smithi052 ceph-mon[32104]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:46.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:46 smithi088 ceph-mon[36528]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:47.973 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:47 smithi189 ceph-mon[36683]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:47.973 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:47 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:47.973 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:47 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:47.974 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:47 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:47.974 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:47 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:48:47.974 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:47 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:47.980 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:47.976+0000 7f3995ffb700 1 -- 172.21.15.189:0/3141833846 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f39980fe3e0 con 0x7f39880a3c40 2024-04-02T13:48:47.980 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 8 on host 'smithi189' 2024-04-02T13:48:47.983 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:47.978+0000 7f399f096700 1 -- 172.21.15.189:0/3141833846 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f39880a3c40 msgr2=0x7f39880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:47.983 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:47.978+0000 7f399f096700 1 --2- 172.21.15.189:0/3141833846 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f39880a3c40 0x7f39880a6100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f399806aaf0 tx=0x7f398c00a380 comp rx=0 tx=0).stop 2024-04-02T13:48:47.983 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:47.978+0000 7f399f096700 1 -- 172.21.15.189:0/3141833846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3998102ad0 msgr2=0x7f399819fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:47.983 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:47.978+0000 7f399f096700 1 --2- 172.21.15.189:0/3141833846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3998102ad0 0x7f399819fe50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f399000add0 tx=0x7f399000c330 comp rx=0 tx=0).stop 2024-04-02T13:48:47.983 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:47.978+0000 7f399f096700 1 -- 172.21.15.189:0/3141833846 shutdown_connections 2024-04-02T13:48:47.983 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:47.978+0000 7f399f096700 1 --2- 172.21.15.189:0/3141833846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3998102ad0 0x7f399819fe50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:47.983 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:47.978+0000 7f399f096700 1 --2- 172.21.15.189:0/3141833846 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f39880a3c40 0x7f39880a6100 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:47.983 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:47.978+0000 7f399f096700 1 --2- 172.21.15.189:0/3141833846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3998101810 0x7f3998198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:47.983 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:47.978+0000 7f399f096700 1 --2- 172.21.15.189:0/3141833846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39981021b0 0x7f3998198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:47.983 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:47.978+0000 7f399f096700 1 -- 172.21.15.189:0/3141833846 >> 172.21.15.189:0/3141833846 conn(0x7f39980fc240 msgr2=0x7f39980fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:47.983 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:47.978+0000 7f399f096700 1 -- 172.21.15.189:0/3141833846 shutdown_connections 2024-04-02T13:48:47.984 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:48:47.978+0000 7f399f096700 1 -- 172.21.15.189:0/3141833846 wait complete. 2024-04-02T13:48:48.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:47 smithi052 ceph-mon[32104]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:48.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:47 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:48.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:47 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:48.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:47 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:48.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:47 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:48:48.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:47 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:48.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:47 smithi088 ceph-mon[36528]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:48.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:47 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:48.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:47 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:48.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:47 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:48.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:47 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:48:48.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:47 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:48.649 DEBUG:teuthology.orchestra.run.smithi189:osd.8> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.8.service 2024-04-02T13:48:48.652 INFO:tasks.cephadm:Deploying osd.9 on smithi052 with /dev/vg_nvme/lv_1... 2024-04-02T13:48:48.652 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-02T13:48:48.826 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:48 smithi189 ceph-mon[36683]: from='osd.8 [v2:172.21.15.189:6816/1602944902,v1:172.21.15.189:6817/1602944902]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T13:48:48.826 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:48 smithi189 ceph-mon[36683]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T13:48:48.826 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:48 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:48.826 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:48 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:48.827 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:48 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:48:48.908 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:48:49.030 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:48 smithi052 ceph-mon[32104]: from='osd.8 [v2:172.21.15.189:6816/1602944902,v1:172.21.15.189:6817/1602944902]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T13:48:49.030 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:48 smithi052 ceph-mon[32104]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T13:48:49.030 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:48 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:49.030 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:48 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:49.030 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:48 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:48:49.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:48 smithi088 ceph-mon[36528]: from='osd.8 [v2:172.21.15.189:6816/1602944902,v1:172.21.15.189:6817/1602944902]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T13:48:49.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:48 smithi088 ceph-mon[36528]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T13:48:49.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:49.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:49.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:48 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:48:50.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:49 smithi189 ceph-mon[36683]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:50.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:49 smithi189 ceph-mon[36683]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-02T13:48:50.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:49 smithi189 ceph-mon[36683]: osdmap e57: 9 total, 8 up, 9 in 2024-04-02T13:48:50.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:49 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:48:50.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:49 smithi189 ceph-mon[36683]: from='osd.8 [v2:172.21.15.189:6816/1602944902,v1:172.21.15.189:6817/1602944902]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-02T13:48:50.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:49 smithi189 ceph-mon[36683]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-02T13:48:50.051 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:49 smithi052 ceph-mon[32104]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:50.051 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:49 smithi052 ceph-mon[32104]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-02T13:48:50.051 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:49 smithi052 ceph-mon[32104]: osdmap e57: 9 total, 8 up, 9 in 2024-04-02T13:48:50.051 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:49 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:48:50.051 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:49 smithi052 ceph-mon[32104]: from='osd.8 [v2:172.21.15.189:6816/1602944902,v1:172.21.15.189:6817/1602944902]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-02T13:48:50.051 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:49 smithi052 ceph-mon[32104]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-02T13:48:50.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:49 smithi088 ceph-mon[36528]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:50.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:49 smithi088 ceph-mon[36528]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-02T13:48:50.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:49 smithi088 ceph-mon[36528]: osdmap e57: 9 total, 8 up, 9 in 2024-04-02T13:48:50.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:49 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:48:50.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:49 smithi088 ceph-mon[36528]: from='osd.8 [v2:172.21.15.189:6816/1602944902,v1:172.21.15.189:6817/1602944902]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-02T13:48:50.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:49 smithi088 ceph-mon[36528]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-02T13:48:50.773 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 13:48:50 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T13:48:50.497+0000 7f32c4b2f700 -1 osd.8 0 waiting for initial osdmap 2024-04-02T13:48:50.773 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 13:48:50 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T13:48:50.512+0000 7f32bc91a700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:48:51.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:50 smithi189 ceph-mon[36683]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-02T13:48:51.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:50 smithi189 ceph-mon[36683]: osdmap e58: 9 total, 8 up, 9 in 2024-04-02T13:48:51.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:50 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:48:51.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:50 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:48:51.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:50 smithi189 ceph-mon[36683]: osdmap e59: 9 total, 8 up, 9 in 2024-04-02T13:48:51.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:50 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:48:51.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:50 smithi189 ceph-mon[36683]: from='osd.8 ' entity='osd.8' 2024-04-02T13:48:51.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:50 smithi052 ceph-mon[32104]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-02T13:48:51.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:50 smithi052 ceph-mon[32104]: osdmap e58: 9 total, 8 up, 9 in 2024-04-02T13:48:51.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:50 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:48:51.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:50 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:48:51.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:50 smithi052 ceph-mon[32104]: osdmap e59: 9 total, 8 up, 9 in 2024-04-02T13:48:51.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:50 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:48:51.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:50 smithi052 ceph-mon[32104]: from='osd.8 ' entity='osd.8' 2024-04-02T13:48:51.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:50 smithi088 ceph-mon[36528]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-02T13:48:51.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:50 smithi088 ceph-mon[36528]: osdmap e58: 9 total, 8 up, 9 in 2024-04-02T13:48:51.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:50 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:48:51.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:50 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:48:51.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:50 smithi088 ceph-mon[36528]: osdmap e59: 9 total, 8 up, 9 in 2024-04-02T13:48:51.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:50 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:48:51.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:50 smithi088 ceph-mon[36528]: from='osd.8 ' entity='osd.8' 2024-04-02T13:48:51.225 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-02T13:48:51.225 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-02T13:48:51.226 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2024-04-02T13:48:51.226 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2024-04-02T13:48:51.226 INFO:teuthology.orchestra.run.smithi052.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00866173 s, 1.2 GB/s 2024-04-02T13:48:51.226 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2024-04-02T13:48:51.935 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi052:vg_nvme/lv_1 2024-04-02T13:48:52.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:51 smithi189 ceph-mon[36683]: purged_snaps scrub starts 2024-04-02T13:48:52.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:51 smithi189 ceph-mon[36683]: purged_snaps scrub ok 2024-04-02T13:48:52.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:51 smithi189 ceph-mon[36683]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:52.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:51 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:48:52.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:51 smithi189 ceph-mon[36683]: osd.8 [v2:172.21.15.189:6816/1602944902,v1:172.21.15.189:6817/1602944902] boot 2024-04-02T13:48:52.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:51 smithi189 ceph-mon[36683]: osdmap e60: 9 total, 9 up, 9 in 2024-04-02T13:48:52.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:51 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:48:52.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:51 smithi052 ceph-mon[32104]: purged_snaps scrub starts 2024-04-02T13:48:52.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:51 smithi052 ceph-mon[32104]: purged_snaps scrub ok 2024-04-02T13:48:52.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:51 smithi052 ceph-mon[32104]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:52.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:51 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:48:52.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:51 smithi052 ceph-mon[32104]: osd.8 [v2:172.21.15.189:6816/1602944902,v1:172.21.15.189:6817/1602944902] boot 2024-04-02T13:48:52.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:51 smithi052 ceph-mon[32104]: osdmap e60: 9 total, 9 up, 9 in 2024-04-02T13:48:52.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:51 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:48:52.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:51 smithi088 ceph-mon[36528]: purged_snaps scrub starts 2024-04-02T13:48:52.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:51 smithi088 ceph-mon[36528]: purged_snaps scrub ok 2024-04-02T13:48:52.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:51 smithi088 ceph-mon[36528]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:48:52.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:51 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:48:52.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:51 smithi088 ceph-mon[36528]: osd.8 [v2:172.21.15.189:6816/1602944902,v1:172.21.15.189:6817/1602944902] boot 2024-04-02T13:48:52.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:51 smithi088 ceph-mon[36528]: osdmap e60: 9 total, 9 up, 9 in 2024-04-02T13:48:52.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:51 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:48:52.186 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:48:53.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:53 smithi088 ceph-mon[36528]: Detected new or changed devices on smithi189 2024-04-02T13:48:53.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:53 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:53.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:53 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:53.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:53 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:53.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:53 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:53.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:53 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:53.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:53 smithi088 ceph-mon[36528]: Adjusting osd_memory_target on smithi189 to 5693M 2024-04-02T13:48:53.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:53 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:53.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:53 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:53.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:53 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:48:53.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:53 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:53.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:53 smithi088 ceph-mon[36528]: osdmap e61: 9 total, 9 up, 9 in 2024-04-02T13:48:53.423 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:53 smithi052 ceph-mon[32104]: Detected new or changed devices on smithi189 2024-04-02T13:48:53.423 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:53 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:53.423 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:53 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:53.423 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:53 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:53.423 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:53 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:53.424 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:53 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:53.424 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:53 smithi052 ceph-mon[32104]: Adjusting osd_memory_target on smithi189 to 5693M 2024-04-02T13:48:53.424 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:53 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:53.424 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:53 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:53.424 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:53 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:48:53.424 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:53 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:53.424 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:53 smithi052 ceph-mon[32104]: osdmap e61: 9 total, 9 up, 9 in 2024-04-02T13:48:53.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:53 smithi189 ceph-mon[36683]: Detected new or changed devices on smithi189 2024-04-02T13:48:53.674 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:53 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:53.674 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:53 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:53.674 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:53 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:53.675 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:53 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:53.675 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:53 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:48:53.675 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:53 smithi189 ceph-mon[36683]: Adjusting osd_memory_target on smithi189 to 5693M 2024-04-02T13:48:53.675 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:53 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:53.675 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:53 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:53.675 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:53 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:48:53.675 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:53 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:48:53.675 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:53 smithi189 ceph-mon[36683]: osdmap e61: 9 total, 9 up, 9 in 2024-04-02T13:48:53.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.965+0000 7f76b9a2f700 1 -- 172.21.15.52:0/3676947847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76b4102620 msgr2=0x7f76b4102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.965+0000 7f76b9a2f700 1 --2- 172.21.15.52:0/3676947847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76b4102620 0x7f76b4102a40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f76a40099c0 tx=0x7f76a4009cd0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.965+0000 7f76b9a2f700 1 -- 172.21.15.52:0/3676947847 shutdown_connections 2024-04-02T13:48:53.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.965+0000 7f76b9a2f700 1 --2- 172.21.15.52:0/3676947847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76b41041c0 0x7f76b410aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.965+0000 7f76b9a2f700 1 --2- 172.21.15.52:0/3676947847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76b4102620 0x7f76b4102a40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.965+0000 7f76b9a2f700 1 --2- 172.21.15.52:0/3676947847 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76b4103820 0x7f76b4103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.965+0000 7f76b9a2f700 1 -- 172.21.15.52:0/3676947847 >> 172.21.15.52:0/3676947847 conn(0x7f76b40fdc00 msgr2=0x7f76b4100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.965+0000 7f76b9a2f700 1 -- 172.21.15.52:0/3676947847 shutdown_connections 2024-04-02T13:48:53.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.965+0000 7f76b9a2f700 1 -- 172.21.15.52:0/3676947847 wait complete. 2024-04-02T13:48:53.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.966+0000 7f76b9a2f700 1 Processor -- start 2024-04-02T13:48:53.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.966+0000 7f76b9a2f700 1 -- start start 2024-04-02T13:48:53.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.966+0000 7f76b9a2f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76b4102620 0x7f76b419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.966+0000 7f76b9a2f700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76b4103820 0x7f76b419afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.966+0000 7f76b9a2f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76b41041c0 0x7f76b41a2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.966+0000 7f76b9a2f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f76b410e1f0 con 0x7f76b4102620 2024-04-02T13:48:53.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.966+0000 7f76b9a2f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f76b410e070 con 0x7f76b41041c0 2024-04-02T13:48:53.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.966+0000 7f76b9a2f700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f76b410e370 con 0x7f76b4103820 2024-04-02T13:48:53.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.966+0000 7f76b2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76b4102620 0x7f76b419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.966+0000 7f76b2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76b4102620 0x7f76b419aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38776/0 (socket says 172.21.15.52:38776) 2024-04-02T13:48:53.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.966+0000 7f76b2ffd700 1 -- 172.21.15.52:0/2776028296 learned_addr learned my addr 172.21.15.52:0/2776028296 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:48:53.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.966+0000 7f76b37fe700 1 --2- 172.21.15.52:0/2776028296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76b41041c0 0x7f76b41a2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.967+0000 7f76b27fc700 1 --2- 172.21.15.52:0/2776028296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76b4103820 0x7f76b419afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.967+0000 7f76b2ffd700 1 -- 172.21.15.52:0/2776028296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76b4103820 msgr2=0x7f76b419afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.967+0000 7f76b2ffd700 1 --2- 172.21.15.52:0/2776028296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76b4103820 0x7f76b419afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.967+0000 7f76b2ffd700 1 -- 172.21.15.52:0/2776028296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76b41041c0 msgr2=0x7f76b41a2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.967+0000 7f76b2ffd700 1 --2- 172.21.15.52:0/2776028296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76b41041c0 0x7f76b41a2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.967+0000 7f76b2ffd700 1 -- 172.21.15.52:0/2776028296 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b41a26e0 con 0x7f76b4102620 2024-04-02T13:48:53.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.967+0000 7f76b37fe700 1 --2- 172.21.15.52:0/2776028296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76b41041c0 0x7f76b41a2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:48:53.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.967+0000 7f76b2ffd700 1 --2- 172.21.15.52:0/2776028296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76b4102620 0x7f76b419aa80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f76a4005b30 tx=0x7f76a4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.967+0000 7f76b8a2d700 1 -- 172.21.15.52:0/2776028296 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a4026070 con 0x7f76b4102620 2024-04-02T13:48:53.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.967+0000 7f76b9a2f700 1 -- 172.21.15.52:0/2776028296 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b41a2970 con 0x7f76b4102620 2024-04-02T13:48:53.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.967+0000 7f76b8a2d700 1 -- 172.21.15.52:0/2776028296 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76a4004150 con 0x7f76b4102620 2024-04-02T13:48:53.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.967+0000 7f76b8a2d700 1 -- 172.21.15.52:0/2776028296 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a4021410 con 0x7f76b4102620 2024-04-02T13:48:53.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.967+0000 7f76b9a2f700 1 -- 172.21.15.52:0/2776028296 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b41a2e50 con 0x7f76b4102620 2024-04-02T13:48:53.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.968+0000 7f76b9a2f700 1 -- 172.21.15.52:0/2776028296 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b40692b0 con 0x7f76b4102620 2024-04-02T13:48:53.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.969+0000 7f76b8a2d700 1 -- 172.21.15.52:0/2776028296 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f76a4029070 con 0x7f76b4102620 2024-04-02T13:48:53.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.970+0000 7f76b8a2d700 1 --2- 172.21.15.52:0/2776028296 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f769c0a3c40 0x7f769c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.970+0000 7f76b27fc700 1 --2- 172.21.15.52:0/2776028296 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f769c0a3c40 0x7f769c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.970+0000 7f76b8a2d700 1 -- 172.21.15.52:0/2776028296 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5858+0+0 (secure 0 0 0) 0x7f76a40d0610 con 0x7f76b4102620 2024-04-02T13:48:53.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.970+0000 7f76b27fc700 1 --2- 172.21.15.52:0/2776028296 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f769c0a3c40 0x7f769c0a6100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f76a8005950 tx=0x7f76a80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:53.975+0000 7f76b8a2d700 1 -- 172.21.15.52:0/2776028296 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f76a405df20 con 0x7f76b4102620 2024-04-02T13:48:54.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:48:54.193+0000 7f76b9a2f700 1 -- 172.21.15.52:0/2776028296 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f76b4075f40 con 0x7f769c0a3c40 2024-04-02T13:48:54.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:54 smithi052 ceph-mon[32104]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:48:54.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:54 smithi052 ceph-mon[32104]: osdmap e62: 9 total, 9 up, 9 in 2024-04-02T13:48:54.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:54 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:48:54.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:54 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:48:54.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:54 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:54.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:54 smithi088 ceph-mon[36528]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:48:54.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:54 smithi088 ceph-mon[36528]: osdmap e62: 9 total, 9 up, 9 in 2024-04-02T13:48:54.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:54 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:48:54.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:54 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:48:54.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:54 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:54.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:54 smithi189 ceph-mon[36683]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:48:54.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:54 smithi189 ceph-mon[36683]: osdmap e62: 9 total, 9 up, 9 in 2024-04-02T13:48:54.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:54 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:48:54.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:54 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:48:54.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:54 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:55.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:55 smithi052 ceph-mon[32104]: from='client.14397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:48:55.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:55 smithi052 ceph-mon[32104]: osdmap e63: 9 total, 9 up, 9 in 2024-04-02T13:48:55.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:55 smithi088 ceph-mon[36528]: from='client.14397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:48:55.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:55 smithi088 ceph-mon[36528]: osdmap e63: 9 total, 9 up, 9 in 2024-04-02T13:48:55.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:55 smithi189 ceph-mon[36683]: from='client.14397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:48:55.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:55 smithi189 ceph-mon[36683]: osdmap e63: 9 total, 9 up, 9 in 2024-04-02T13:48:56.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:56 smithi052 ceph-mon[32104]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:48:56.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:56 smithi052 ceph-mon[32104]: osdmap e64: 9 total, 9 up, 9 in 2024-04-02T13:48:56.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:56 smithi088 ceph-mon[36528]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:48:56.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:56 smithi088 ceph-mon[36528]: osdmap e64: 9 total, 9 up, 9 in 2024-04-02T13:48:56.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:56 smithi189 ceph-mon[36683]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:48:56.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:56 smithi189 ceph-mon[36683]: osdmap e64: 9 total, 9 up, 9 in 2024-04-02T13:48:57.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:57 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3176593071' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8aaad86a-c313-467b-8998-13f154bda92a"}]: dispatch 2024-04-02T13:48:57.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:57 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3176593071' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8aaad86a-c313-467b-8998-13f154bda92a"}]': finished 2024-04-02T13:48:57.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:57 smithi052 ceph-mon[32104]: osdmap e65: 10 total, 9 up, 10 in 2024-04-02T13:48:57.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:57 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:48:57.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:57 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3176593071' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8aaad86a-c313-467b-8998-13f154bda92a"}]: dispatch 2024-04-02T13:48:57.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:57 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3176593071' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8aaad86a-c313-467b-8998-13f154bda92a"}]': finished 2024-04-02T13:48:57.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:57 smithi088 ceph-mon[36528]: osdmap e65: 10 total, 9 up, 10 in 2024-04-02T13:48:57.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:57 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:48:57.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:57 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3176593071' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8aaad86a-c313-467b-8998-13f154bda92a"}]: dispatch 2024-04-02T13:48:57.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:57 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3176593071' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8aaad86a-c313-467b-8998-13f154bda92a"}]': finished 2024-04-02T13:48:57.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:57 smithi189 ceph-mon[36683]: osdmap e65: 10 total, 9 up, 10 in 2024-04-02T13:48:57.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:57 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:48:58.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:58 smithi052 ceph-mon[32104]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-04-02T13:48:58.592 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:48:58 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1740903854' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:48:58.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:58 smithi088 ceph-mon[36528]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-04-02T13:48:58.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:48:58 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1740903854' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:48:58.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:58 smithi189 ceph-mon[36683]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-04-02T13:48:58.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:48:58 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1740903854' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:49:00.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:00 smithi052 ceph-mon[32104]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-02T13:49:00.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:00 smithi088 ceph-mon[36528]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-02T13:49:00.787 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:00 smithi189 ceph-mon[36683]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-02T13:49:02.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:02 smithi052 ceph-mon[32104]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-02T13:49:02.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:02 smithi088 ceph-mon[36528]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-02T13:49:02.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:02 smithi189 ceph-mon[36683]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-02T13:49:04.386 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:04 smithi052 ceph-mon[32104]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-02T13:49:04.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:04 smithi088 ceph-mon[36528]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-02T13:49:04.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:04 smithi189 ceph-mon[36683]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-02T13:49:05.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:05 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-02T13:49:05.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:05 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:05.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:05 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-02T13:49:05.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:05 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:05.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:05 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-02T13:49:05.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:05 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:06.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:06 smithi052 ceph-mon[32104]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-02T13:49:06.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:06 smithi052 ceph-mon[32104]: Deploying daemon osd.9 on smithi052 2024-04-02T13:49:06.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:06 smithi088 ceph-mon[36528]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-02T13:49:06.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:06 smithi088 ceph-mon[36528]: Deploying daemon osd.9 on smithi052 2024-04-02T13:49:06.689 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:06 smithi189 ceph-mon[36683]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-02T13:49:06.689 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:06 smithi189 ceph-mon[36683]: Deploying daemon osd.9 on smithi052 2024-04-02T13:49:08.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:08 smithi052 ceph-mon[32104]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:49:08.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:08 smithi088 ceph-mon[36528]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:49:08.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:08 smithi189 ceph-mon[36683]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:49:10.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:10 smithi052 ceph-mon[32104]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:49:10.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:10 smithi088 ceph-mon[36528]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:49:10.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:10 smithi189 ceph-mon[36683]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:49:12.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:12 smithi052 ceph-mon[32104]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:49:12.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:12 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:12.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:12 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:12.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:12 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:49:12.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:12 smithi088 ceph-mon[36528]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:49:12.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:12 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:12.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:12 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:12.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:12 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:49:12.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:12 smithi189 ceph-mon[36683]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:49:12.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:12 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:12.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:12 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:12.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:12 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:49:14.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:14 smithi052 ceph-mon[32104]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:49:14.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:14 smithi088 ceph-mon[36528]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:49:14.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:14 smithi189 ceph-mon[36683]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:49:15.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:49:15.103+0000 7f76b8a2d700 1 -- 172.21.15.52:0/2776028296 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f76b4075f40 con 0x7f769c0a3c40 2024-04-02T13:49:15.107 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 9 on host 'smithi052' 2024-04-02T13:49:15.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:49:15.105+0000 7f76b9a2f700 1 -- 172.21.15.52:0/2776028296 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f769c0a3c40 msgr2=0x7f769c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:15.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:49:15.105+0000 7f76b9a2f700 1 --2- 172.21.15.52:0/2776028296 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f769c0a3c40 0x7f769c0a6100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f76a8005950 tx=0x7f76a80058e0 comp rx=0 tx=0).stop 2024-04-02T13:49:15.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:49:15.105+0000 7f76b9a2f700 1 -- 172.21.15.52:0/2776028296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76b4102620 msgr2=0x7f76b419aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:15.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:49:15.105+0000 7f76b9a2f700 1 --2- 172.21.15.52:0/2776028296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76b4102620 0x7f76b419aa80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f76a4005b30 tx=0x7f76a4009580 comp rx=0 tx=0).stop 2024-04-02T13:49:15.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:49:15.105+0000 7f76b9a2f700 1 -- 172.21.15.52:0/2776028296 shutdown_connections 2024-04-02T13:49:15.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:49:15.105+0000 7f76b9a2f700 1 --2- 172.21.15.52:0/2776028296 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f769c0a3c40 0x7f769c0a6100 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:15.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:49:15.105+0000 7f76b9a2f700 1 --2- 172.21.15.52:0/2776028296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76b41041c0 0x7f76b41a2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:15.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:49:15.105+0000 7f76b9a2f700 1 --2- 172.21.15.52:0/2776028296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76b4102620 0x7f76b419aa80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:15.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:49:15.105+0000 7f76b9a2f700 1 --2- 172.21.15.52:0/2776028296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76b4103820 0x7f76b419afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:15.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:49:15.105+0000 7f76b9a2f700 1 -- 172.21.15.52:0/2776028296 >> 172.21.15.52:0/2776028296 conn(0x7f76b40fdc00 msgr2=0x7f76b4074820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:15.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:49:15.105+0000 7f76b9a2f700 1 -- 172.21.15.52:0/2776028296 shutdown_connections 2024-04-02T13:49:15.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:49:15.106+0000 7f76b9a2f700 1 -- 172.21.15.52:0/2776028296 wait complete. 2024-04-02T13:49:15.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:15 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:15.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:15 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:15.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:15 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:15.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:15 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:49:15.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:15 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:15.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:15 smithi052 ceph-mon[32104]: from='osd.9 [v2:172.21.15.52:6826/2350236811,v1:172.21.15.52:6827/2350236811]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-02T13:49:15.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:15 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:15.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:15 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:49:15.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:15 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:15.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:15 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:15.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:15 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:15.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:15 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:15.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:15 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:49:15.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:15 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:15.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:15 smithi088 ceph-mon[36528]: from='osd.9 [v2:172.21.15.52:6826/2350236811,v1:172.21.15.52:6827/2350236811]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-02T13:49:15.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:15 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:15.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:15 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:49:15.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:15 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:15.756 DEBUG:teuthology.orchestra.run.smithi052:osd.9> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.9.service 2024-04-02T13:49:15.758 INFO:tasks.cephadm:Deploying osd.10 on smithi088 with /dev/vg_nvme/lv_1... 2024-04-02T13:49:15.758 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-02T13:49:15.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:15 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:15.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:15 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:15.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:15 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:15.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:15 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:49:15.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:15 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:15.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:15 smithi189 ceph-mon[36683]: from='osd.9 [v2:172.21.15.52:6826/2350236811,v1:172.21.15.52:6827/2350236811]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-02T13:49:15.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:15 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:15.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:15 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:49:15.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:15 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:16.007 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:49:16.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:16 smithi052 ceph-mon[32104]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:49:16.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:16 smithi052 ceph-mon[32104]: from='osd.9 [v2:172.21.15.52:6826/2350236811,v1:172.21.15.52:6827/2350236811]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-02T13:49:16.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:16 smithi052 ceph-mon[32104]: osdmap e66: 10 total, 9 up, 10 in 2024-04-02T13:49:16.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:16 smithi052 ceph-mon[32104]: from='osd.9 [v2:172.21.15.52:6826/2350236811,v1:172.21.15.52:6827/2350236811]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-04-02T13:49:16.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:16 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:49:16.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 13:49:16 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T13:49:16.323+0000 7fb16da31700 -1 osd.9 0 waiting for initial osdmap 2024-04-02T13:49:16.398 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:16 smithi088 ceph-mon[36528]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:49:16.399 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:16 smithi088 ceph-mon[36528]: from='osd.9 [v2:172.21.15.52:6826/2350236811,v1:172.21.15.52:6827/2350236811]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-02T13:49:16.399 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:16 smithi088 ceph-mon[36528]: osdmap e66: 10 total, 9 up, 10 in 2024-04-02T13:49:16.399 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:16 smithi088 ceph-mon[36528]: from='osd.9 [v2:172.21.15.52:6826/2350236811,v1:172.21.15.52:6827/2350236811]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-04-02T13:49:16.399 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:16 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:49:16.639 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 13:49:16 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T13:49:16.336+0000 7fb169824700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:49:16.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:16 smithi189 ceph-mon[36683]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:49:16.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:16 smithi189 ceph-mon[36683]: from='osd.9 [v2:172.21.15.52:6826/2350236811,v1:172.21.15.52:6827/2350236811]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-02T13:49:16.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:16 smithi189 ceph-mon[36683]: osdmap e66: 10 total, 9 up, 10 in 2024-04-02T13:49:16.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:16 smithi189 ceph-mon[36683]: from='osd.9 [v2:172.21.15.52:6826/2350236811,v1:172.21.15.52:6827/2350236811]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-04-02T13:49:16.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:16 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:49:17.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:17 smithi088 ceph-mon[36528]: from='osd.9 [v2:172.21.15.52:6826/2350236811,v1:172.21.15.52:6827/2350236811]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-04-02T13:49:17.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:17 smithi088 ceph-mon[36528]: osdmap e67: 10 total, 9 up, 10 in 2024-04-02T13:49:17.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:17 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:49:17.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:17 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:49:17.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:17 smithi052 ceph-mon[32104]: from='osd.9 [v2:172.21.15.52:6826/2350236811,v1:172.21.15.52:6827/2350236811]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-04-02T13:49:17.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:17 smithi052 ceph-mon[32104]: osdmap e67: 10 total, 9 up, 10 in 2024-04-02T13:49:17.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:17 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:49:17.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:17 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:49:17.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:17 smithi189 ceph-mon[36683]: from='osd.9 [v2:172.21.15.52:6826/2350236811,v1:172.21.15.52:6827/2350236811]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-04-02T13:49:17.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:17 smithi189 ceph-mon[36683]: osdmap e67: 10 total, 9 up, 10 in 2024-04-02T13:49:17.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:17 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:49:17.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:17 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:49:18.094 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-02T13:49:18.094 INFO:teuthology.orchestra.run.smithi088.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-02T13:49:18.095 INFO:teuthology.orchestra.run.smithi088.stderr: stderr: 10+0 records in 2024-04-02T13:49:18.095 INFO:teuthology.orchestra.run.smithi088.stderr:10+0 records out 2024-04-02T13:49:18.095 INFO:teuthology.orchestra.run.smithi088.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0121316 s, 864 MB/s 2024-04-02T13:49:18.095 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping successful for: 2024-04-02T13:49:18.491 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:18 smithi052 ceph-mon[32104]: purged_snaps scrub starts 2024-04-02T13:49:18.491 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:18 smithi052 ceph-mon[32104]: purged_snaps scrub ok 2024-04-02T13:49:18.491 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:18 smithi052 ceph-mon[32104]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:49:18.492 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:18 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:49:18.492 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:18 smithi052 ceph-mon[32104]: osd.9 [v2:172.21.15.52:6826/2350236811,v1:172.21.15.52:6827/2350236811] boot 2024-04-02T13:49:18.492 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:18 smithi052 ceph-mon[32104]: osdmap e68: 10 total, 10 up, 10 in 2024-04-02T13:49:18.492 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:18 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:49:18.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:18 smithi088 ceph-mon[36528]: purged_snaps scrub starts 2024-04-02T13:49:18.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:18 smithi088 ceph-mon[36528]: purged_snaps scrub ok 2024-04-02T13:49:18.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:18 smithi088 ceph-mon[36528]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:49:18.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:18 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:49:18.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:18 smithi088 ceph-mon[36528]: osd.9 [v2:172.21.15.52:6826/2350236811,v1:172.21.15.52:6827/2350236811] boot 2024-04-02T13:49:18.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:18 smithi088 ceph-mon[36528]: osdmap e68: 10 total, 10 up, 10 in 2024-04-02T13:49:18.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:18 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:49:18.746 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi088:vg_nvme/lv_1 2024-04-02T13:49:18.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:18 smithi189 ceph-mon[36683]: purged_snaps scrub starts 2024-04-02T13:49:18.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:18 smithi189 ceph-mon[36683]: purged_snaps scrub ok 2024-04-02T13:49:18.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:18 smithi189 ceph-mon[36683]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:49:18.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:18 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:49:18.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:18 smithi189 ceph-mon[36683]: osd.9 [v2:172.21.15.52:6826/2350236811,v1:172.21.15.52:6827/2350236811] boot 2024-04-02T13:49:18.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:18 smithi189 ceph-mon[36683]: osdmap e68: 10 total, 10 up, 10 in 2024-04-02T13:49:18.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:18 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:49:18.987 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:49:19.397 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:19 smithi088 ceph-mon[36528]: osdmap e69: 10 total, 10 up, 10 in 2024-04-02T13:49:19.397 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:19 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:19.397 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:19 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:19.397 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:19 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:19.397 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:19 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:19.397 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:19 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:19.397 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:19 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:19.398 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:19 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:19.398 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:19 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:19.398 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:19 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:49:19.398 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:19 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:19.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:19 smithi052 ceph-mon[32104]: osdmap e69: 10 total, 10 up, 10 in 2024-04-02T13:49:19.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:19 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:19.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:19 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:19.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:19 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:19.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:19 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:19.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:19 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:19.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:19 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:19.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:19 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:19.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:19 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:19.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:19 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:49:19.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:19 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:19.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:19 smithi189 ceph-mon[36683]: osdmap e69: 10 total, 10 up, 10 in 2024-04-02T13:49:19.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:19 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:19.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:19 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:19.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:19 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:19.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:19 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:19.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:19 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:19.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:19 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:19.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:19 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:19.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:19 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:19.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:19 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:49:19.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:19 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:20.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:20 smithi088 ceph-mon[36528]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:20.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:20 smithi088 ceph-mon[36528]: Detected new or changed devices on smithi052 2024-04-02T13:49:20.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:20 smithi088 ceph-mon[36528]: Adjusting osd_memory_target on smithi052 to 4273M 2024-04-02T13:49:20.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.680+0000 7f6333972700 1 -- 172.21.15.88:0/4250796526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f632c103910 msgr2=0x7f632c103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:20.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.680+0000 7f6333972700 1 --2- 172.21.15.88:0/4250796526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f632c103910 0x7f632c103d70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f631c009990 tx=0x7f631c009ca0 comp rx=0 tx=0).stop 2024-04-02T13:49:20.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.681+0000 7f6333972700 1 -- 172.21.15.88:0/4250796526 shutdown_connections 2024-04-02T13:49:20.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.681+0000 7f6333972700 1 --2- 172.21.15.88:0/4250796526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f632c1042b0 0x7f632c10ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:20.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.681+0000 7f6333972700 1 --2- 172.21.15.88:0/4250796526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f632c100930 0x7f632c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:20.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.681+0000 7f6333972700 1 --2- 172.21.15.88:0/4250796526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f632c103910 0x7f632c103d70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:20.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.681+0000 7f6333972700 1 -- 172.21.15.88:0/4250796526 >> 172.21.15.88:0/4250796526 conn(0x7f632c0fc290 msgr2=0x7f632c0fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:20.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.681+0000 7f6333972700 1 -- 172.21.15.88:0/4250796526 shutdown_connections 2024-04-02T13:49:20.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.681+0000 7f6333972700 1 -- 172.21.15.88:0/4250796526 wait complete. 2024-04-02T13:49:20.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.681+0000 7f6333972700 1 Processor -- start 2024-04-02T13:49:20.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.681+0000 7f6333972700 1 -- start start 2024-04-02T13:49:20.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.681+0000 7f6333972700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f632c100930 0x7f632c198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:20.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.681+0000 7f6333972700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f632c103910 0x7f632c198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:20.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.681+0000 7f6333972700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f632c1042b0 0x7f632c19fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:20.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.681+0000 7f6333972700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f632c10e250 con 0x7f632c1042b0 2024-04-02T13:49:20.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.681+0000 7f6333972700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f632c10e0d0 con 0x7f632c100930 2024-04-02T13:49:20.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.681+0000 7f6333972700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f632c10e3d0 con 0x7f632c103910 2024-04-02T13:49:20.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.681+0000 7f633170e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f632c100930 0x7f632c198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:20.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.682+0000 7f633170e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f632c100930 0x7f632c198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44884/0 (socket says 172.21.15.88:44884) 2024-04-02T13:49:20.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.682+0000 7f633170e700 1 -- 172.21.15.88:0/210080900 learned_addr learned my addr 172.21.15.88:0/210080900 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-02T13:49:20.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.682+0000 7f6331f0f700 1 --2- 172.21.15.88:0/210080900 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f632c1042b0 0x7f632c19fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:20.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.682+0000 7f6330f0d700 1 --2- 172.21.15.88:0/210080900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f632c103910 0x7f632c198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:20.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.682+0000 7f633170e700 1 -- 172.21.15.88:0/210080900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f632c103910 msgr2=0x7f632c198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:20.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.682+0000 7f633170e700 1 --2- 172.21.15.88:0/210080900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f632c103910 0x7f632c198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:20.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.682+0000 7f633170e700 1 -- 172.21.15.88:0/210080900 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f632c1042b0 msgr2=0x7f632c19fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:20.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.682+0000 7f633170e700 1 --2- 172.21.15.88:0/210080900 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f632c1042b0 0x7f632c19fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:20.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.682+0000 7f633170e700 1 -- 172.21.15.88:0/210080900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f632c1a0490 con 0x7f632c100930 2024-04-02T13:49:20.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.682+0000 7f633170e700 1 --2- 172.21.15.88:0/210080900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f632c100930 0x7f632c198830 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f632800d980 tx=0x7f632800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:20.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.682+0000 7f63227fc700 1 -- 172.21.15.88:0/210080900 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6328017070 con 0x7f632c100930 2024-04-02T13:49:20.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.682+0000 7f6333972700 1 -- 172.21.15.88:0/210080900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f632c1a0780 con 0x7f632c100930 2024-04-02T13:49:20.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.682+0000 7f63227fc700 1 -- 172.21.15.88:0/210080900 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f632800fd70 con 0x7f632c100930 2024-04-02T13:49:20.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.682+0000 7f63227fc700 1 -- 172.21.15.88:0/210080900 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632800e3b0 con 0x7f632c100930 2024-04-02T13:49:20.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.682+0000 7f6333972700 1 -- 172.21.15.88:0/210080900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f632c1a0c70 con 0x7f632c100930 2024-04-02T13:49:20.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.683+0000 7f6333972700 1 -- 172.21.15.88:0/210080900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f632c0692b0 con 0x7f632c100930 2024-04-02T13:49:20.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.684+0000 7f63227fc700 1 -- 172.21.15.88:0/210080900 <== mon.2 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6328010660 con 0x7f632c100930 2024-04-02T13:49:20.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.685+0000 7f63227fc700 1 --2- 172.21.15.88:0/210080900 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f63180a3c40 0x7f63180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:20.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.685+0000 7f63227fc700 1 -- 172.21.15.88:0/210080900 <== mon.2 v2:172.21.15.88:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7f63280cf690 con 0x7f632c100930 2024-04-02T13:49:20.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.685+0000 7f6330f0d700 1 --2- 172.21.15.88:0/210080900 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f63180a3c40 0x7f63180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:20.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.685+0000 7f6330f0d700 1 --2- 172.21.15.88:0/210080900 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f63180a3c40 0x7f63180a6100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f631c00b5c0 tx=0x7f631c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:20.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.689+0000 7f63227fc700 1 -- 172.21.15.88:0/210080900 <== mon.2 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6328010460 con 0x7f632c100930 2024-04-02T13:49:20.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:20 smithi189 ceph-mon[36683]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:20.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:20 smithi189 ceph-mon[36683]: Detected new or changed devices on smithi052 2024-04-02T13:49:20.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:20 smithi189 ceph-mon[36683]: Adjusting osd_memory_target on smithi052 to 4273M 2024-04-02T13:49:20.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:20 smithi052 ceph-mon[32104]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:20.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:20 smithi052 ceph-mon[32104]: Detected new or changed devices on smithi052 2024-04-02T13:49:20.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:20 smithi052 ceph-mon[32104]: Adjusting osd_memory_target on smithi052 to 4273M 2024-04-02T13:49:20.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:20.897+0000 7f6333972700 1 -- 172.21.15.88:0/210080900 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f632c062a30 con 0x7f63180a3c40 2024-04-02T13:49:21.653 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:21 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:49:21.653 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:21 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:49:21.653 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:21 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:21.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:21 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:49:21.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:21 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:49:21.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:21 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:21.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:21 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:49:21.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:21 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:49:21.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:21 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:22.653 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:22 smithi088 ceph-mon[36528]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:22.653 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:22 smithi088 ceph-mon[36528]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:49:22.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:22 smithi189 ceph-mon[36683]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:22.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:22 smithi189 ceph-mon[36683]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:49:22.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:22 smithi052 ceph-mon[32104]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:22.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:22 smithi052 ceph-mon[32104]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:49:24.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:24 smithi088 ceph-mon[36528]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:24.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:24 smithi088 ceph-mon[36528]: from='client.? 172.21.15.88:0/3754141755' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d33f6f2-3a01-4753-b2ef-b8c42454198e"}]: dispatch 2024-04-02T13:49:24.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:24 smithi088 ceph-mon[36528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d33f6f2-3a01-4753-b2ef-b8c42454198e"}]: dispatch 2024-04-02T13:49:24.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:24 smithi088 ceph-mon[36528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d33f6f2-3a01-4753-b2ef-b8c42454198e"}]': finished 2024-04-02T13:49:24.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:24 smithi088 ceph-mon[36528]: osdmap e70: 11 total, 10 up, 11 in 2024-04-02T13:49:24.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:24 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:49:24.642 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:24 smithi052 ceph-mon[32104]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:24.642 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:24 smithi052 ceph-mon[32104]: from='client.? 172.21.15.88:0/3754141755' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d33f6f2-3a01-4753-b2ef-b8c42454198e"}]: dispatch 2024-04-02T13:49:24.642 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:24 smithi052 ceph-mon[32104]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d33f6f2-3a01-4753-b2ef-b8c42454198e"}]: dispatch 2024-04-02T13:49:24.642 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:24 smithi052 ceph-mon[32104]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d33f6f2-3a01-4753-b2ef-b8c42454198e"}]': finished 2024-04-02T13:49:24.642 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:24 smithi052 ceph-mon[32104]: osdmap e70: 11 total, 10 up, 11 in 2024-04-02T13:49:24.642 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:24 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:49:24.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:24 smithi189 ceph-mon[36683]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:24.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:24 smithi189 ceph-mon[36683]: from='client.? 172.21.15.88:0/3754141755' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d33f6f2-3a01-4753-b2ef-b8c42454198e"}]: dispatch 2024-04-02T13:49:24.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:24 smithi189 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d33f6f2-3a01-4753-b2ef-b8c42454198e"}]: dispatch 2024-04-02T13:49:24.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:24 smithi189 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d33f6f2-3a01-4753-b2ef-b8c42454198e"}]': finished 2024-04-02T13:49:24.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:24 smithi189 ceph-mon[36683]: osdmap e70: 11 total, 10 up, 11 in 2024-04-02T13:49:24.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:24 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:49:25.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:25 smithi088 ceph-mon[36528]: from='client.? 172.21.15.88:0/2919217306' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:49:25.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:25 smithi189 ceph-mon[36683]: from='client.? 172.21.15.88:0/2919217306' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:49:25.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:25 smithi052 ceph-mon[32104]: from='client.? 172.21.15.88:0/2919217306' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:49:26.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:26 smithi088 ceph-mon[36528]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:26.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:26 smithi189 ceph-mon[36683]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:26.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:26 smithi052 ceph-mon[32104]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:28.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:28 smithi088 ceph-mon[36528]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:28.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:28 smithi189 ceph-mon[36683]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:28.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:28 smithi052 ceph-mon[32104]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:30.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:30 smithi088 ceph-mon[36528]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:30.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:30 smithi189 ceph-mon[36683]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:30.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:30 smithi052 ceph-mon[32104]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:31.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:31 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-02T13:49:31.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:31 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:31.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:31 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-02T13:49:31.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:31 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:31.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:31 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-02T13:49:31.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:31 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:32.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:32 smithi088 ceph-mon[36528]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:32.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:32 smithi088 ceph-mon[36528]: Deploying daemon osd.10 on smithi088 2024-04-02T13:49:32.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:32 smithi189 ceph-mon[36683]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:32.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:32 smithi189 ceph-mon[36683]: Deploying daemon osd.10 on smithi088 2024-04-02T13:49:32.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:32 smithi052 ceph-mon[32104]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:32.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:32 smithi052 ceph-mon[32104]: Deploying daemon osd.10 on smithi088 2024-04-02T13:49:34.581 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:34 smithi088 ceph-mon[36528]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:34.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:34 smithi189 ceph-mon[36683]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:34.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:34 smithi052 ceph-mon[32104]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:36.625 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:36 smithi088 ceph-mon[36528]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:36.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:36 smithi189 ceph-mon[36683]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:36.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:36 smithi052 ceph-mon[32104]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:37.628 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:37 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:49:37.628 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:37 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:37.628 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:37 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:37.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:37 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:49:37.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:37 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:37.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:37 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:37.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:37 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:49:37.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:37 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:37.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:37 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:38.621 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:38 smithi088 ceph-mon[36528]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:38.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:38 smithi189 ceph-mon[36683]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:38.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:38 smithi052 ceph-mon[32104]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:40.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:40 smithi088 ceph-mon[36528]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:40.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:40 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:40.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:40 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:40.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:40 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:40.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:40 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:49:40.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:40 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:40.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:40 smithi189 ceph-mon[36683]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:40.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:40 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:40.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:40 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:40.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:40 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:40.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:40 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:49:40.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:40 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:40.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:40.813+0000 7f63227fc700 1 -- 172.21.15.88:0/210080900 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f632c062a30 con 0x7f63180a3c40 2024-04-02T13:49:40.817 INFO:teuthology.orchestra.run.smithi088.stdout:Created osd(s) 10 on host 'smithi088' 2024-04-02T13:49:40.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:40.816+0000 7f6333972700 1 -- 172.21.15.88:0/210080900 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f63180a3c40 msgr2=0x7f63180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:40.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:40.816+0000 7f6333972700 1 --2- 172.21.15.88:0/210080900 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f63180a3c40 0x7f63180a6100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f631c00b5c0 tx=0x7f631c005fd0 comp rx=0 tx=0).stop 2024-04-02T13:49:40.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:40.816+0000 7f6333972700 1 -- 172.21.15.88:0/210080900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f632c100930 msgr2=0x7f632c198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:40.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:40.816+0000 7f6333972700 1 --2- 172.21.15.88:0/210080900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f632c100930 0x7f632c198830 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f632800d980 tx=0x7f632800dc90 comp rx=0 tx=0).stop 2024-04-02T13:49:40.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:40.816+0000 7f6333972700 1 -- 172.21.15.88:0/210080900 shutdown_connections 2024-04-02T13:49:40.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:40.816+0000 7f6333972700 1 --2- 172.21.15.88:0/210080900 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f632c1042b0 0x7f632c19fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:40.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:40.816+0000 7f6333972700 1 --2- 172.21.15.88:0/210080900 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f63180a3c40 0x7f63180a6100 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:40.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:40.816+0000 7f6333972700 1 --2- 172.21.15.88:0/210080900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f632c100930 0x7f632c198830 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:40.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:40.816+0000 7f6333972700 1 --2- 172.21.15.88:0/210080900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f632c103910 0x7f632c198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:40.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:40.816+0000 7f6333972700 1 -- 172.21.15.88:0/210080900 >> 172.21.15.88:0/210080900 conn(0x7f632c0fc290 msgr2=0x7f632c106bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:40.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:40.816+0000 7f6333972700 1 -- 172.21.15.88:0/210080900 shutdown_connections 2024-04-02T13:49:40.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-02T13:49:40.816+0000 7f6333972700 1 -- 172.21.15.88:0/210080900 wait complete. 2024-04-02T13:49:40.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:40 smithi052 ceph-mon[32104]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:40.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:40 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:40.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:40 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:40.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:40 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:40.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:40 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:49:40.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:40 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:41.512 DEBUG:teuthology.orchestra.run.smithi088:osd.10> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.10.service 2024-04-02T13:49:41.514 INFO:tasks.cephadm:Deploying osd.11 on smithi189 with /dev/vg_nvme/lv_1... 2024-04-02T13:49:41.515 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-02T13:49:41.763 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:49:42.016 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:41 smithi088 ceph-mon[36528]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:42.016 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:41 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:42.016 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:41 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:49:42.016 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:41 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:42.016 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:41 smithi088 ceph-mon[36528]: from='osd.10 [v2:172.21.15.88:6824/3425831447,v1:172.21.15.88:6825/3425831447]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T13:49:42.017 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:41 smithi088 ceph-mon[36528]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T13:49:42.062 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:41 smithi189 ceph-mon[36683]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:42.062 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:41 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:42.062 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:41 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:49:42.062 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:41 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:42.062 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:41 smithi189 ceph-mon[36683]: from='osd.10 [v2:172.21.15.88:6824/3425831447,v1:172.21.15.88:6825/3425831447]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T13:49:42.063 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:41 smithi189 ceph-mon[36683]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T13:49:42.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:41 smithi052 ceph-mon[32104]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:42.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:41 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:42.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:41 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:49:42.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:41 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:42.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:41 smithi052 ceph-mon[32104]: from='osd.10 [v2:172.21.15.88:6824/3425831447,v1:172.21.15.88:6825/3425831447]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T13:49:42.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:41 smithi052 ceph-mon[32104]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T13:49:43.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:42 smithi189 ceph-mon[36683]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-02T13:49:43.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:42 smithi189 ceph-mon[36683]: from='osd.10 [v2:172.21.15.88:6824/3425831447,v1:172.21.15.88:6825/3425831447]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-02T13:49:43.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:42 smithi189 ceph-mon[36683]: osdmap e71: 11 total, 10 up, 11 in 2024-04-02T13:49:43.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:42 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:49:43.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:42 smithi189 ceph-mon[36683]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-02T13:49:43.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:42 smithi052 ceph-mon[32104]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-02T13:49:43.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:42 smithi052 ceph-mon[32104]: from='osd.10 [v2:172.21.15.88:6824/3425831447,v1:172.21.15.88:6825/3425831447]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-02T13:49:43.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:42 smithi052 ceph-mon[32104]: osdmap e71: 11 total, 10 up, 11 in 2024-04-02T13:49:43.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:42 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:49:43.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:42 smithi052 ceph-mon[32104]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-02T13:49:43.108 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:42 smithi088 ceph-mon[36528]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-02T13:49:43.108 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:42 smithi088 ceph-mon[36528]: from='osd.10 [v2:172.21.15.88:6824/3425831447,v1:172.21.15.88:6825/3425831447]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-02T13:49:43.109 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:42 smithi088 ceph-mon[36528]: osdmap e71: 11 total, 10 up, 11 in 2024-04-02T13:49:43.109 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:42 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:49:43.109 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:42 smithi088 ceph-mon[36528]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-02T13:49:43.109 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 13:49:42 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T13:49:42.810+0000 7f492ca84700 -1 osd.10 0 waiting for initial osdmap 2024-04-02T13:49:43.109 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 13:49:42 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T13:49:42.822+0000 7f4928877700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:49:44.082 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-02T13:49:44.083 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-02T13:49:44.083 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-04-02T13:49:44.083 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-04-02T13:49:44.083 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0200107 s, 524 MB/s 2024-04-02T13:49:44.083 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-04-02T13:49:44.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:43 smithi189 ceph-mon[36683]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:44.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:43 smithi189 ceph-mon[36683]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-04-02T13:49:44.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:43 smithi189 ceph-mon[36683]: osdmap e72: 11 total, 10 up, 11 in 2024-04-02T13:49:44.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:43 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:49:44.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:43 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:49:44.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:43 smithi052 ceph-mon[32104]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:44.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:43 smithi052 ceph-mon[32104]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-04-02T13:49:44.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:43 smithi052 ceph-mon[32104]: osdmap e72: 11 total, 10 up, 11 in 2024-04-02T13:49:44.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:43 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:49:44.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:43 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:49:44.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:43 smithi088 ceph-mon[36528]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:44.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:43 smithi088 ceph-mon[36528]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-04-02T13:49:44.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:43 smithi088 ceph-mon[36528]: osdmap e72: 11 total, 10 up, 11 in 2024-04-02T13:49:44.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:43 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:49:44.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:43 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:49:44.825 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi189:vg_nvme/lv_1 2024-04-02T13:49:44.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:44 smithi088 ceph-mon[36528]: purged_snaps scrub starts 2024-04-02T13:49:44.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:44 smithi088 ceph-mon[36528]: purged_snaps scrub ok 2024-04-02T13:49:44.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:44 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:49:44.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:44 smithi088 ceph-mon[36528]: osd.10 [v2:172.21.15.88:6824/3425831447,v1:172.21.15.88:6825/3425831447] boot 2024-04-02T13:49:44.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:44 smithi088 ceph-mon[36528]: osdmap e73: 11 total, 11 up, 11 in 2024-04-02T13:49:44.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:44 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:49:44.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:44 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:44.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:44 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:44.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:44 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:44.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:44 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:44.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:44 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:44.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:44 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:44.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:44 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:44.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:44 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:44.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:44 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:49:44.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:44 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:45.069 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:49:45.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:44 smithi052 ceph-mon[32104]: purged_snaps scrub starts 2024-04-02T13:49:45.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:44 smithi052 ceph-mon[32104]: purged_snaps scrub ok 2024-04-02T13:49:45.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:44 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:49:45.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:44 smithi052 ceph-mon[32104]: osd.10 [v2:172.21.15.88:6824/3425831447,v1:172.21.15.88:6825/3425831447] boot 2024-04-02T13:49:45.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:44 smithi052 ceph-mon[32104]: osdmap e73: 11 total, 11 up, 11 in 2024-04-02T13:49:45.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:44 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:49:45.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:44 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:45.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:44 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:45.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:44 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:45.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:44 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:45.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:44 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:45.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:44 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:45.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:44 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:45.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:44 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:45.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:44 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:49:45.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:44 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:45.192 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:44 smithi189 ceph-mon[36683]: purged_snaps scrub starts 2024-04-02T13:49:45.192 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:44 smithi189 ceph-mon[36683]: purged_snaps scrub ok 2024-04-02T13:49:45.192 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:44 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:49:45.192 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:44 smithi189 ceph-mon[36683]: osd.10 [v2:172.21.15.88:6824/3425831447,v1:172.21.15.88:6825/3425831447] boot 2024-04-02T13:49:45.192 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:44 smithi189 ceph-mon[36683]: osdmap e73: 11 total, 11 up, 11 in 2024-04-02T13:49:45.192 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:44 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:49:45.192 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:44 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:45.192 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:44 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:45.192 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:44 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:45.192 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:44 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:45.192 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:44 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:45.193 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:44 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:49:45.193 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:44 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:45.193 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:44 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:45.193 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:44 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:49:45.193 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:44 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:49:46.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:45 smithi052 ceph-mon[32104]: Detected new or changed devices on smithi088 2024-04-02T13:49:46.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:45 smithi052 ceph-mon[32104]: Adjusting osd_memory_target on smithi088 to 4270M 2024-04-02T13:49:46.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:45 smithi052 ceph-mon[32104]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:46.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:45 smithi052 ceph-mon[32104]: osdmap e74: 11 total, 11 up, 11 in 2024-04-02T13:49:46.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:45 smithi088 ceph-mon[36528]: Detected new or changed devices on smithi088 2024-04-02T13:49:46.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:45 smithi088 ceph-mon[36528]: Adjusting osd_memory_target on smithi088 to 4270M 2024-04-02T13:49:46.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:45 smithi088 ceph-mon[36528]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:46.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:45 smithi088 ceph-mon[36528]: osdmap e74: 11 total, 11 up, 11 in 2024-04-02T13:49:46.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:45 smithi189 ceph-mon[36683]: Detected new or changed devices on smithi088 2024-04-02T13:49:46.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:45 smithi189 ceph-mon[36683]: Adjusting osd_memory_target on smithi088 to 4270M 2024-04-02T13:49:46.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:45 smithi189 ceph-mon[36683]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:49:46.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:45 smithi189 ceph-mon[36683]: osdmap e74: 11 total, 11 up, 11 in 2024-04-02T13:49:46.898 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.893+0000 7f926ba29700 1 -- 172.21.15.189:0/1500175127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9264100ea0 msgr2=0x7f92641032d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:46.898 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.893+0000 7f926ba29700 1 --2- 172.21.15.189:0/1500175127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9264100ea0 0x7f92641032d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9260009990 tx=0x7f9260009ca0 comp rx=0 tx=0).stop 2024-04-02T13:49:46.898 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.893+0000 7f926ba29700 1 -- 172.21.15.189:0/1500175127 shutdown_connections 2024-04-02T13:49:46.898 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.893+0000 7f926ba29700 1 --2- 172.21.15.189:0/1500175127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92641061e0 0x7f9264106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:46.899 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.893+0000 7f926ba29700 1 --2- 172.21.15.189:0/1500175127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9264100ea0 0x7f92641032d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:46.899 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.893+0000 7f926ba29700 1 --2- 172.21.15.189:0/1500175127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9264103810 0x7f9264105ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:46.899 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.893+0000 7f926ba29700 1 -- 172.21.15.189:0/1500175127 >> 172.21.15.189:0/1500175127 conn(0x7f92640fcac0 msgr2=0x7f92640fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:46.899 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.893+0000 7f926ba29700 1 -- 172.21.15.189:0/1500175127 shutdown_connections 2024-04-02T13:49:46.899 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.893+0000 7f926ba29700 1 -- 172.21.15.189:0/1500175127 wait complete. 2024-04-02T13:49:46.899 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.894+0000 7f926ba29700 1 Processor -- start 2024-04-02T13:49:46.899 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.894+0000 7f926ba29700 1 -- start start 2024-04-02T13:49:46.899 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.894+0000 7f926ba29700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9264100ea0 0x7f926419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:46.899 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.894+0000 7f926ba29700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9264103810 0x7f926419f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:46.899 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.894+0000 7f926ba29700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92641061e0 0x7f92641a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:46.899 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.894+0000 7f926ba29700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9264114860 con 0x7f9264100ea0 2024-04-02T13:49:46.899 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.894+0000 7f926ba29700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f92641146e0 con 0x7f9264103810 2024-04-02T13:49:46.899 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.894+0000 7f926ba29700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f92641149e0 con 0x7f92641061e0 2024-04-02T13:49:46.900 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.894+0000 7f9269fc6700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92641061e0 0x7f92641a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:46.900 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.894+0000 7f9269fc6700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92641061e0 0x7f92641a64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:43028/0 (socket says 172.21.15.189:43028) 2024-04-02T13:49:46.900 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.894+0000 7f9269fc6700 1 -- 172.21.15.189:0/1304242806 learned_addr learned my addr 172.21.15.189:0/1304242806 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-02T13:49:46.900 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.894+0000 7f92697c5700 1 --2- 172.21.15.189:0/1304242806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9264100ea0 0x7f926419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:46.900 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.894+0000 7f9268fc4700 1 --2- 172.21.15.189:0/1304242806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9264103810 0x7f926419f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:46.900 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.894+0000 7f9269fc6700 1 -- 172.21.15.189:0/1304242806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9264103810 msgr2=0x7f926419f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:46.900 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.894+0000 7f9269fc6700 1 --2- 172.21.15.189:0/1304242806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9264103810 0x7f926419f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:46.900 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.894+0000 7f9269fc6700 1 -- 172.21.15.189:0/1304242806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9264100ea0 msgr2=0x7f926419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:46.900 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.894+0000 7f9269fc6700 1 --2- 172.21.15.189:0/1304242806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9264100ea0 0x7f926419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:46.900 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.894+0000 7f9269fc6700 1 -- 172.21.15.189:0/1304242806 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92641a6b90 con 0x7f92641061e0 2024-04-02T13:49:46.900 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.894+0000 7f9269fc6700 1 --2- 172.21.15.189:0/1304242806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92641061e0 0x7f92641a64f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f925c00b9a0 tx=0x7f925c00bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:46.900 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.895+0000 7f925a7fc700 1 -- 172.21.15.189:0/1304242806 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c019070 con 0x7f92641061e0 2024-04-02T13:49:46.900 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.895+0000 7f926ba29700 1 -- 172.21.15.189:0/1304242806 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92641a6e80 con 0x7f92641061e0 2024-04-02T13:49:46.901 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.895+0000 7f925a7fc700 1 -- 172.21.15.189:0/1304242806 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f925c014440 con 0x7f92641061e0 2024-04-02T13:49:46.901 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.895+0000 7f925a7fc700 1 -- 172.21.15.189:0/1304242806 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c0075d0 con 0x7f92641061e0 2024-04-02T13:49:46.901 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.895+0000 7f926ba29700 1 -- 172.21.15.189:0/1304242806 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92641a73b0 con 0x7f92641061e0 2024-04-02T13:49:46.902 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.896+0000 7f926ba29700 1 -- 172.21.15.189:0/1304242806 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92640692b0 con 0x7f92641061e0 2024-04-02T13:49:46.902 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.898+0000 7f925a7fc700 1 -- 172.21.15.189:0/1304242806 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f925c01c070 con 0x7f92641061e0 2024-04-02T13:49:46.902 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.898+0000 7f925a7fc700 1 --2- 172.21.15.189:0/1304242806 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f92500a3c40 0x7f92500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:46.902 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.898+0000 7f925a7fc700 1 -- 172.21.15.189:0/1304242806 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7f925c0cf620 con 0x7f92641061e0 2024-04-02T13:49:46.902 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.898+0000 7f92697c5700 1 --2- 172.21.15.189:0/1304242806 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f92500a3c40 0x7f92500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:46.902 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.899+0000 7f92697c5700 1 --2- 172.21.15.189:0/1304242806 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f92500a3c40 0x7f92500a6100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f926000b5c0 tx=0x7f9260005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:46.905 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:46.902+0000 7f925a7fc700 1 -- 172.21.15.189:0/1304242806 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f925c09c6e0 con 0x7f92641061e0 2024-04-02T13:49:47.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:46 smithi189 ceph-mon[36683]: osdmap e75: 11 total, 11 up, 11 in 2024-04-02T13:49:47.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:46 smithi052 ceph-mon[32104]: osdmap e75: 11 total, 11 up, 11 in 2024-04-02T13:49:47.115 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:49:47.112+0000 7f926ba29700 1 -- 172.21.15.189:0/1304242806 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9264062a30 con 0x7f92500a3c40 2024-04-02T13:49:47.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:46 smithi088 ceph-mon[36528]: osdmap e75: 11 total, 11 up, 11 in 2024-04-02T13:49:48.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:47 smithi189 ceph-mon[36683]: pgmap v241: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T13:49:48.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:47 smithi189 ceph-mon[36683]: osdmap e76: 11 total, 11 up, 11 in 2024-04-02T13:49:48.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:47 smithi189 ceph-mon[36683]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:49:48.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:47 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:49:48.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:47 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:49:48.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:47 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:48.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:47 smithi052 ceph-mon[32104]: pgmap v241: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T13:49:48.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:47 smithi052 ceph-mon[32104]: osdmap e76: 11 total, 11 up, 11 in 2024-04-02T13:49:48.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:47 smithi052 ceph-mon[32104]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:49:48.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:47 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:49:48.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:47 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:49:48.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:47 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:48.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:47 smithi088 ceph-mon[36528]: pgmap v241: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T13:49:48.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:47 smithi088 ceph-mon[36528]: osdmap e76: 11 total, 11 up, 11 in 2024-04-02T13:49:48.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:47 smithi088 ceph-mon[36528]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:49:48.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:47 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:49:48.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:47 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:49:48.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:47 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:49.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:48 smithi088 ceph-mon[36528]: osdmap e77: 11 total, 11 up, 11 in 2024-04-02T13:49:49.219 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:48 smithi189 ceph-mon[36683]: osdmap e77: 11 total, 11 up, 11 in 2024-04-02T13:49:49.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:48 smithi052 ceph-mon[32104]: osdmap e77: 11 total, 11 up, 11 in 2024-04-02T13:49:50.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:49 smithi088 ceph-mon[36528]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T13:49:50.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:49 smithi189 ceph-mon[36683]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T13:49:50.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:49 smithi052 ceph-mon[32104]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T13:49:51.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:50 smithi088 ceph-mon[36528]: from='client.? 172.21.15.189:0/3882041710' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4369f21d-d344-4e2f-b556-010faefd182c"}]: dispatch 2024-04-02T13:49:51.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:50 smithi088 ceph-mon[36528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4369f21d-d344-4e2f-b556-010faefd182c"}]: dispatch 2024-04-02T13:49:51.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:50 smithi088 ceph-mon[36528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4369f21d-d344-4e2f-b556-010faefd182c"}]': finished 2024-04-02T13:49:51.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:50 smithi088 ceph-mon[36528]: osdmap e78: 12 total, 11 up, 12 in 2024-04-02T13:49:51.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:50 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:49:51.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:50 smithi088 ceph-mon[36528]: from='client.? 172.21.15.189:0/346809893' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:49:51.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:50 smithi189 ceph-mon[36683]: from='client.? 172.21.15.189:0/3882041710' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4369f21d-d344-4e2f-b556-010faefd182c"}]: dispatch 2024-04-02T13:49:51.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:50 smithi189 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4369f21d-d344-4e2f-b556-010faefd182c"}]: dispatch 2024-04-02T13:49:51.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:50 smithi189 ceph-mon[36683]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4369f21d-d344-4e2f-b556-010faefd182c"}]': finished 2024-04-02T13:49:51.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:50 smithi189 ceph-mon[36683]: osdmap e78: 12 total, 11 up, 12 in 2024-04-02T13:49:51.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:50 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:49:51.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:50 smithi189 ceph-mon[36683]: from='client.? 172.21.15.189:0/346809893' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:49:51.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:50 smithi052 ceph-mon[32104]: from='client.? 172.21.15.189:0/3882041710' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4369f21d-d344-4e2f-b556-010faefd182c"}]: dispatch 2024-04-02T13:49:51.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:50 smithi052 ceph-mon[32104]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4369f21d-d344-4e2f-b556-010faefd182c"}]: dispatch 2024-04-02T13:49:51.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:50 smithi052 ceph-mon[32104]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4369f21d-d344-4e2f-b556-010faefd182c"}]': finished 2024-04-02T13:49:51.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:50 smithi052 ceph-mon[32104]: osdmap e78: 12 total, 11 up, 12 in 2024-04-02T13:49:51.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:50 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:49:51.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:50 smithi052 ceph-mon[32104]: from='client.? 172.21.15.189:0/346809893' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:49:52.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:52 smithi052 ceph-mon[32104]: pgmap v246: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-04-02T13:49:52.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:52 smithi088 ceph-mon[36528]: pgmap v246: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-04-02T13:49:52.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:52 smithi189 ceph-mon[36683]: pgmap v246: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-04-02T13:49:54.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:54 smithi052 ceph-mon[32104]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:49:54.415 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:54 smithi189 ceph-mon[36683]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:49:54.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:54 smithi088 ceph-mon[36528]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:49:56.335 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:56 smithi189 ceph-mon[36683]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:49:56.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:56 smithi052 ceph-mon[32104]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:49:56.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:56 smithi088 ceph-mon[36528]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:49:58.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:58 smithi052 ceph-mon[32104]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:49:58.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:58 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-02T13:49:58.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:58 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:58.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:49:58 smithi052 ceph-mon[32104]: Deploying daemon osd.11 on smithi189 2024-04-02T13:49:58.360 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:58 smithi189 ceph-mon[36683]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:49:58.360 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:58 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-02T13:49:58.361 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:58 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:58.361 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:49:58 smithi189 ceph-mon[36683]: Deploying daemon osd.11 on smithi189 2024-04-02T13:49:58.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:58 smithi088 ceph-mon[36528]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:49:58.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:58 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-02T13:49:58.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:58 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:58.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:49:58 smithi088 ceph-mon[36528]: Deploying daemon osd.11 on smithi189 2024-04-02T13:50:00.104 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:00 smithi189 ceph-mon[36683]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:50:00.104 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:00 smithi189 ceph-mon[36683]: overall HEALTH_OK 2024-04-02T13:50:00.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:00 smithi052 ceph-mon[32104]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:50:00.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:00 smithi052 ceph-mon[32104]: overall HEALTH_OK 2024-04-02T13:50:00.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:00 smithi088 ceph-mon[36528]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:50:00.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:00 smithi088 ceph-mon[36528]: overall HEALTH_OK 2024-04-02T13:50:02.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:02 smithi189 ceph-mon[36683]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:50:02.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:02 smithi088 ceph-mon[36528]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:50:02.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:02 smithi052 ceph-mon[32104]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:50:04.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:04 smithi088 ceph-mon[36528]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:50:04.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:04 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:04.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:04 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:04.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:04 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:50:04.440 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:04 smithi189 ceph-mon[36683]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:50:04.440 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:04 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:04.440 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:04 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:04.440 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:04 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:50:04.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:04 smithi052 ceph-mon[32104]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:50:04.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:04 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:04.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:04 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:04.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:04 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:50:06.352 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:06 smithi189 ceph-mon[36683]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:50:06.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:06 smithi088 ceph-mon[36528]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:50:06.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:06 smithi052 ceph-mon[32104]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:50:07.842 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:07.839+0000 7f925a7fc700 1 -- 172.21.15.189:0/1304242806 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f9264062a30 con 0x7f92500a3c40 2024-04-02T13:50:07.843 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 11 on host 'smithi189' 2024-04-02T13:50:07.845 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:07.842+0000 7f926ba29700 1 -- 172.21.15.189:0/1304242806 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f92500a3c40 msgr2=0x7f92500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:07.845 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:07.842+0000 7f926ba29700 1 --2- 172.21.15.189:0/1304242806 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f92500a3c40 0x7f92500a6100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f926000b5c0 tx=0x7f9260005fb0 comp rx=0 tx=0).stop 2024-04-02T13:50:07.845 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:07.842+0000 7f926ba29700 1 -- 172.21.15.189:0/1304242806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92641061e0 msgr2=0x7f92641a64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:07.845 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:07.842+0000 7f926ba29700 1 --2- 172.21.15.189:0/1304242806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92641061e0 0x7f92641a64f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f925c00b9a0 tx=0x7f925c00bd60 comp rx=0 tx=0).stop 2024-04-02T13:50:07.845 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:07.842+0000 7f926ba29700 1 -- 172.21.15.189:0/1304242806 shutdown_connections 2024-04-02T13:50:07.846 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:07.842+0000 7f926ba29700 1 --2- 172.21.15.189:0/1304242806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92641061e0 0x7f92641a64f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:07.846 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:07.842+0000 7f926ba29700 1 --2- 172.21.15.189:0/1304242806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9264100ea0 0x7f926419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:07.846 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:07.842+0000 7f926ba29700 1 --2- 172.21.15.189:0/1304242806 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f92500a3c40 0x7f92500a6100 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:07.846 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:07.842+0000 7f926ba29700 1 --2- 172.21.15.189:0/1304242806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9264103810 0x7f926419f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:07.846 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:07.842+0000 7f926ba29700 1 -- 172.21.15.189:0/1304242806 >> 172.21.15.189:0/1304242806 conn(0x7f92640fcac0 msgr2=0x7f92641042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:07.846 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:07.842+0000 7f926ba29700 1 -- 172.21.15.189:0/1304242806 shutdown_connections 2024-04-02T13:50:07.846 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:07.842+0000 7f926ba29700 1 -- 172.21.15.189:0/1304242806 wait complete. 2024-04-02T13:50:08.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:08 smithi189 ceph-mon[36683]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:50:08.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:08 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:08.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:08 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:08.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:08 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:08.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:08 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:50:08.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:08 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:08.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:08 smithi189 ceph-mon[36683]: from='osd.11 [v2:172.21.15.189:6824/383422941,v1:172.21.15.189:6825/383422941]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T13:50:08.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:08 smithi189 ceph-mon[36683]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T13:50:08.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:08 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:50:08.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:08 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:08.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:08 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:08.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:08 smithi052 ceph-mon[32104]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:50:08.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:08 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:08.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:08 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:08.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:08 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:08.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:08 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:50:08.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:08 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:08.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:08 smithi052 ceph-mon[32104]: from='osd.11 [v2:172.21.15.189:6824/383422941,v1:172.21.15.189:6825/383422941]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T13:50:08.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:08 smithi052 ceph-mon[32104]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T13:50:08.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:08 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:50:08.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:08 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:08.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:08 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:08.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:08 smithi088 ceph-mon[36528]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:50:08.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:08 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:08.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:08 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:08.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:08 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:08.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:08 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:50:08.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:08 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:08.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:08 smithi088 ceph-mon[36528]: from='osd.11 [v2:172.21.15.189:6824/383422941,v1:172.21.15.189:6825/383422941]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T13:50:08.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:08 smithi088 ceph-mon[36528]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T13:50:08.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:08 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:50:08.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:08 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:08.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:08 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:08.547 DEBUG:teuthology.orchestra.run.smithi189:osd.11> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.11.service 2024-04-02T13:50:08.550 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-02T13:50:08.550 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph osd stat -f json 2024-04-02T13:50:08.808 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:50:09.188 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:09 smithi052 ceph-mon[32104]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-02T13:50:09.189 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:09 smithi052 ceph-mon[32104]: osdmap e79: 12 total, 11 up, 12 in 2024-04-02T13:50:09.189 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:09 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:50:09.189 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:09 smithi052 ceph-mon[32104]: from='osd.11 [v2:172.21.15.189:6824/383422941,v1:172.21.15.189:6825/383422941]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-02T13:50:09.189 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:09 smithi052 ceph-mon[32104]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-02T13:50:09.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:09 smithi088 ceph-mon[36528]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-02T13:50:09.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:09 smithi088 ceph-mon[36528]: osdmap e79: 12 total, 11 up, 12 in 2024-04-02T13:50:09.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:09 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:50:09.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:09 smithi088 ceph-mon[36528]: from='osd.11 [v2:172.21.15.189:6824/383422941,v1:172.21.15.189:6825/383422941]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-02T13:50:09.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:09 smithi088 ceph-mon[36528]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-02T13:50:09.505 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:09 smithi189 ceph-mon[36683]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-02T13:50:09.506 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:09 smithi189 ceph-mon[36683]: osdmap e79: 12 total, 11 up, 12 in 2024-04-02T13:50:09.506 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:09 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:50:09.506 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:09 smithi189 ceph-mon[36683]: from='osd.11 [v2:172.21.15.189:6824/383422941,v1:172.21.15.189:6825/383422941]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-02T13:50:09.506 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:09 smithi189 ceph-mon[36683]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-02T13:50:10.065 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 13:50:09 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T13:50:09.801+0000 7fa7a11b9700 -1 osd.11 0 waiting for initial osdmap 2024-04-02T13:50:10.065 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 13:50:09 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T13:50:09.816+0000 7fa798fa4700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:50:10.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:10 smithi052 ceph-mon[32104]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:50:10.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:10 smithi052 ceph-mon[32104]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-02T13:50:10.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:10 smithi052 ceph-mon[32104]: osdmap e80: 12 total, 11 up, 12 in 2024-04-02T13:50:10.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:10 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:50:10.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:10 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:50:10.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:10 smithi052 ceph-mon[32104]: from='osd.11 ' entity='osd.11' 2024-04-02T13:50:10.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:10 smithi088 ceph-mon[36528]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:50:10.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:10 smithi088 ceph-mon[36528]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-02T13:50:10.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:10 smithi088 ceph-mon[36528]: osdmap e80: 12 total, 11 up, 12 in 2024-04-02T13:50:10.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:10 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:50:10.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:10 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:50:10.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:10 smithi088 ceph-mon[36528]: from='osd.11 ' entity='osd.11' 2024-04-02T13:50:10.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:10 smithi189 ceph-mon[36683]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:50:10.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:10 smithi189 ceph-mon[36683]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-02T13:50:10.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:10 smithi189 ceph-mon[36683]: osdmap e80: 12 total, 11 up, 12 in 2024-04-02T13:50:10.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:10 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:50:10.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:10 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:50:10.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:10 smithi189 ceph-mon[36683]: from='osd.11 ' entity='osd.11' 2024-04-02T13:50:10.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.646+0000 7f2865926700 1 -- 172.21.15.52:0/3804208369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28601041e0 msgr2=0x7f286010aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:10.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.646+0000 7f2865926700 1 --2- 172.21.15.52:0/3804208369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28601041e0 0x7f286010aa90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f28580099c0 tx=0x7f2858009cd0 comp rx=0 tx=0).stop 2024-04-02T13:50:10.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.646+0000 7f2865926700 1 -- 172.21.15.52:0/3804208369 shutdown_connections 2024-04-02T13:50:10.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.646+0000 7f2865926700 1 --2- 172.21.15.52:0/3804208369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28601041e0 0x7f286010aa90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:10.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.646+0000 7f2865926700 1 --2- 172.21.15.52:0/3804208369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2860102640 0x7f2860102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:10.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.646+0000 7f2865926700 1 --2- 172.21.15.52:0/3804208369 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2860103840 0x7f2860103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:10.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.646+0000 7f2865926700 1 -- 172.21.15.52:0/3804208369 >> 172.21.15.52:0/3804208369 conn(0x7f28600fdc00 msgr2=0x7f2860100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:10.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.646+0000 7f2865926700 1 -- 172.21.15.52:0/3804208369 shutdown_connections 2024-04-02T13:50:10.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.646+0000 7f2865926700 1 -- 172.21.15.52:0/3804208369 wait complete. 2024-04-02T13:50:10.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.647+0000 7f2865926700 1 Processor -- start 2024-04-02T13:50:10.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.647+0000 7f2865926700 1 -- start start 2024-04-02T13:50:10.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.647+0000 7f2865926700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2860102640 0x7f2860198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:10.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.647+0000 7f2865926700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2860103840 0x7f2860198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:10.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.647+0000 7f2865926700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28601041e0 0x7f286019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:10.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.647+0000 7f2865926700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f286010e220 con 0x7f2860102640 2024-04-02T13:50:10.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.647+0000 7f2865926700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f286010e0a0 con 0x7f28601041e0 2024-04-02T13:50:10.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.647+0000 7f2865926700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f286010e3a0 con 0x7f2860103840 2024-04-02T13:50:10.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.647+0000 7f285effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2860102640 0x7f2860198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:10.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.647+0000 7f285effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2860102640 0x7f2860198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49382/0 (socket says 172.21.15.52:49382) 2024-04-02T13:50:10.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.647+0000 7f285effd700 1 -- 172.21.15.52:0/1490530487 learned_addr learned my addr 172.21.15.52:0/1490530487 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:10.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.647+0000 7f285e7fc700 1 --2- 172.21.15.52:0/1490530487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2860103840 0x7f2860198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:10.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.647+0000 7f285f7fe700 1 --2- 172.21.15.52:0/1490530487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28601041e0 0x7f286019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:10.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.647+0000 7f285effd700 1 -- 172.21.15.52:0/1490530487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2860103840 msgr2=0x7f2860198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:10.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.647+0000 7f285effd700 1 --2- 172.21.15.52:0/1490530487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2860103840 0x7f2860198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:10.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.647+0000 7f285effd700 1 -- 172.21.15.52:0/1490530487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28601041e0 msgr2=0x7f286019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:10.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.647+0000 7f285effd700 1 --2- 172.21.15.52:0/1490530487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28601041e0 0x7f286019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:10.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.647+0000 7f285effd700 1 -- 172.21.15.52:0/1490530487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28601a03a0 con 0x7f2860102640 2024-04-02T13:50:10.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.648+0000 7f285effd700 1 --2- 172.21.15.52:0/1490530487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2860102640 0x7f2860198740 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2850007e30 tx=0x7f285000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:10.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.648+0000 7f2864924700 1 -- 172.21.15.52:0/1490530487 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850016020 con 0x7f2860102640 2024-04-02T13:50:10.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.648+0000 7f2865926700 1 -- 172.21.15.52:0/1490530487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28601a0690 con 0x7f2860102640 2024-04-02T13:50:10.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.648+0000 7f2864924700 1 -- 172.21.15.52:0/1490530487 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f285000ac50 con 0x7f2860102640 2024-04-02T13:50:10.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.648+0000 7f2864924700 1 -- 172.21.15.52:0/1490530487 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28500115e0 con 0x7f2860102640 2024-04-02T13:50:10.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.648+0000 7f2865926700 1 -- 172.21.15.52:0/1490530487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28601a0bd0 con 0x7f2860102640 2024-04-02T13:50:10.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.649+0000 7f2865926700 1 -- 172.21.15.52:0/1490530487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28600692b0 con 0x7f2860102640 2024-04-02T13:50:10.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.651+0000 7f2864924700 1 -- 172.21.15.52:0/1490530487 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2850019020 con 0x7f2860102640 2024-04-02T13:50:10.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.651+0000 7f2864924700 1 --2- 172.21.15.52:0/1490530487 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f284c0a3c40 0x7f284c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:10.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.652+0000 7f285e7fc700 1 --2- 172.21.15.52:0/1490530487 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f284c0a3c40 0x7f284c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:10.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.652+0000 7f2864924700 1 -- 172.21.15.52:0/1490530487 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6847+0+0 (secure 0 0 0) 0x7f28500cff30 con 0x7f2860102640 2024-04-02T13:50:10.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.652+0000 7f285e7fc700 1 --2- 172.21.15.52:0/1490530487 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f284c0a3c40 0x7f284c0a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2848004630 tx=0x7f284800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:10.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.655+0000 7f2864924700 1 -- 172.21.15.52:0/1490530487 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2850020030 con 0x7f2860102640 2024-04-02T13:50:10.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.866+0000 7f2865926700 1 -- 172.21.15.52:0/1490530487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f286002ddf0 con 0x7f2860102640 2024-04-02T13:50:10.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.866+0000 7f2864924700 1 -- 172.21.15.52:0/1490530487 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f2850023e50 con 0x7f2860102640 2024-04-02T13:50:10.870 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:50:10.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.868+0000 7f2865926700 1 -- 172.21.15.52:0/1490530487 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f284c0a3c40 msgr2=0x7f284c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:10.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.868+0000 7f2865926700 1 --2- 172.21.15.52:0/1490530487 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f284c0a3c40 0x7f284c0a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2848004630 tx=0x7f284800b3c0 comp rx=0 tx=0).stop 2024-04-02T13:50:10.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.868+0000 7f2865926700 1 -- 172.21.15.52:0/1490530487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2860102640 msgr2=0x7f2860198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:10.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.868+0000 7f2865926700 1 --2- 172.21.15.52:0/1490530487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2860102640 0x7f2860198740 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2850007e30 tx=0x7f285000a340 comp rx=0 tx=0).stop 2024-04-02T13:50:10.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.868+0000 7f2865926700 1 -- 172.21.15.52:0/1490530487 shutdown_connections 2024-04-02T13:50:10.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.868+0000 7f2865926700 1 --2- 172.21.15.52:0/1490530487 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f284c0a3c40 0x7f284c0a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:10.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.868+0000 7f2865926700 1 --2- 172.21.15.52:0/1490530487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28601041e0 0x7f286019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.868+0000 7f2865926700 1 --2- 172.21.15.52:0/1490530487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2860102640 0x7f2860198740 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.868+0000 7f2865926700 1 --2- 172.21.15.52:0/1490530487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2860103840 0x7f2860198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.868+0000 7f2865926700 1 -- 172.21.15.52:0/1490530487 >> 172.21.15.52:0/1490530487 conn(0x7f28600fdc00 msgr2=0x7f2860106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.868+0000 7f2865926700 1 -- 172.21.15.52:0/1490530487 shutdown_connections 2024-04-02T13:50:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:10.868+0000 7f2865926700 1 -- 172.21.15.52:0/1490530487 wait complete. 2024-04-02T13:50:11.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:11 smithi088 ceph-mon[36528]: purged_snaps scrub starts 2024-04-02T13:50:11.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:11 smithi088 ceph-mon[36528]: purged_snaps scrub ok 2024-04-02T13:50:11.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:11 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:50:11.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:11 smithi088 ceph-mon[36528]: osd.11 [v2:172.21.15.189:6824/383422941,v1:172.21.15.189:6825/383422941] boot 2024-04-02T13:50:11.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:11 smithi088 ceph-mon[36528]: osdmap e81: 12 total, 12 up, 12 in 2024-04-02T13:50:11.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:11 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:50:11.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:11 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1490530487' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T13:50:11.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:11 smithi189 ceph-mon[36683]: purged_snaps scrub starts 2024-04-02T13:50:11.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:11 smithi189 ceph-mon[36683]: purged_snaps scrub ok 2024-04-02T13:50:11.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:11 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:50:11.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:11 smithi189 ceph-mon[36683]: osd.11 [v2:172.21.15.189:6824/383422941,v1:172.21.15.189:6825/383422941] boot 2024-04-02T13:50:11.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:11 smithi189 ceph-mon[36683]: osdmap e81: 12 total, 12 up, 12 in 2024-04-02T13:50:11.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:11 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:50:11.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:11 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1490530487' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T13:50:11.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:11 smithi052 ceph-mon[32104]: purged_snaps scrub starts 2024-04-02T13:50:11.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:11 smithi052 ceph-mon[32104]: purged_snaps scrub ok 2024-04-02T13:50:11.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:50:11.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:11 smithi052 ceph-mon[32104]: osd.11 [v2:172.21.15.189:6824/383422941,v1:172.21.15.189:6825/383422941] boot 2024-04-02T13:50:11.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:11 smithi052 ceph-mon[32104]: osdmap e81: 12 total, 12 up, 12 in 2024-04-02T13:50:11.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:11 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:50:11.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:11 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1490530487' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T13:50:11.663 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1712065810,"num_in_osds":12,"osd_in_since":1712065790,"num_remapped_pgs":0} 2024-04-02T13:50:11.663 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-02T13:50:11.914 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:50:12.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:12 smithi189 ceph-mon[36683]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:50:12.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:12 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:12.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:12 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:12.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:12 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:50:12.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:12 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:50:12.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:12 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:50:12.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:12 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:50:12.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:12 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:12.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:12 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:12.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:12 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:50:12.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:12 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:12.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:12 smithi052 ceph-mon[32104]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:50:12.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:12 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:12.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:12 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:12.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:12 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:50:12.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:12 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:50:12.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:12 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:50:12.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:12 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:50:12.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:12 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:12.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:12 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:12.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:12 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:50:12.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:12 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:12.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:12 smithi088 ceph-mon[36528]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:50:12.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:12 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:12.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:12 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:12.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:12 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:50:12.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:12 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:50:12.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:12 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:50:12.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:12 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:50:12.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:12 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:12.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:12 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:12.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:12 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:50:12.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:12 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:13.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:13 smithi052 ceph-mon[32104]: Detected new or changed devices on smithi189 2024-04-02T13:50:13.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:13 smithi052 ceph-mon[32104]: Adjusting osd_memory_target on smithi189 to 4270M 2024-04-02T13:50:13.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:13 smithi052 ceph-mon[32104]: osdmap e82: 12 total, 12 up, 12 in 2024-04-02T13:50:13.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:13 smithi088 ceph-mon[36528]: Detected new or changed devices on smithi189 2024-04-02T13:50:13.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:13 smithi088 ceph-mon[36528]: Adjusting osd_memory_target on smithi189 to 4270M 2024-04-02T13:50:13.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:13 smithi088 ceph-mon[36528]: osdmap e82: 12 total, 12 up, 12 in 2024-04-02T13:50:13.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:13 smithi189 ceph-mon[36683]: Detected new or changed devices on smithi189 2024-04-02T13:50:13.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:13 smithi189 ceph-mon[36683]: Adjusting osd_memory_target on smithi189 to 4270M 2024-04-02T13:50:13.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:13 smithi189 ceph-mon[36683]: osdmap e82: 12 total, 12 up, 12 in 2024-04-02T13:50:13.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.596+0000 7fb2b932e700 1 -- 172.21.15.52:0/2093654513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2b4107500 msgr2=0x7fb2b41078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:13.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.596+0000 7fb2b932e700 1 --2- 172.21.15.52:0/2093654513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2b4107500 0x7fb2b41078e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb2a400b0a0 tx=0x7fb2a400b3b0 comp rx=0 tx=0).stop 2024-04-02T13:50:13.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.596+0000 7fb2b932e700 1 -- 172.21.15.52:0/2093654513 shutdown_connections 2024-04-02T13:50:13.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.596+0000 7fb2b932e700 1 --2- 172.21.15.52:0/2093654513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2b4101eb0 0x7fb2b410e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:13.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.596+0000 7fb2b932e700 1 --2- 172.21.15.52:0/2093654513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2b4107500 0x7fb2b41078e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:13.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.596+0000 7fb2b932e700 1 --2- 172.21.15.52:0/2093654513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2b41014f0 0x7fb2b4101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:13.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.596+0000 7fb2b932e700 1 -- 172.21.15.52:0/2093654513 >> 172.21.15.52:0/2093654513 conn(0x7fb2b4078420 msgr2=0x7fb2b4078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:13.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.596+0000 7fb2b932e700 1 -- 172.21.15.52:0/2093654513 shutdown_connections 2024-04-02T13:50:13.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.596+0000 7fb2b932e700 1 -- 172.21.15.52:0/2093654513 wait complete. 2024-04-02T13:50:13.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.597+0000 7fb2b932e700 1 Processor -- start 2024-04-02T13:50:13.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.597+0000 7fb2b932e700 1 -- start start 2024-04-02T13:50:13.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.597+0000 7fb2b932e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2b41014f0 0x7fb2b419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:13.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.597+0000 7fb2b932e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2b4101eb0 0x7fb2b419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:13.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.597+0000 7fb2b932e700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2b4107500 0x7fb2b4198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:13.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.597+0000 7fb2b932e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b4111770 con 0x7fb2b4101eb0 2024-04-02T13:50:13.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.597+0000 7fb2b932e700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b41115f0 con 0x7fb2b41014f0 2024-04-02T13:50:13.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.597+0000 7fb2b932e700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b41118f0 con 0x7fb2b4107500 2024-04-02T13:50:13.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.597+0000 7fb2b27fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2b4101eb0 0x7fb2b419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:13.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.597+0000 7fb2b27fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2b4101eb0 0x7fb2b419edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33364/0 (socket says 172.21.15.52:33364) 2024-04-02T13:50:13.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.597+0000 7fb2b27fc700 1 -- 172.21.15.52:0/2849320924 learned_addr learned my addr 172.21.15.52:0/2849320924 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:13.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.597+0000 7fb2b2ffd700 1 --2- 172.21.15.52:0/2849320924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2b41014f0 0x7fb2b419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:13.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.597+0000 7fb2b37fe700 1 --2- 172.21.15.52:0/2849320924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2b4107500 0x7fb2b4198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:13.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.597+0000 7fb2b27fc700 1 -- 172.21.15.52:0/2849320924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2b4107500 msgr2=0x7fb2b4198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:13.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.597+0000 7fb2b27fc700 1 --2- 172.21.15.52:0/2849320924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2b4107500 0x7fb2b4198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:13.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.597+0000 7fb2b27fc700 1 -- 172.21.15.52:0/2849320924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2b41014f0 msgr2=0x7fb2b419e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:13.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.597+0000 7fb2b27fc700 1 --2- 172.21.15.52:0/2849320924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2b41014f0 0x7fb2b419e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:13.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.598+0000 7fb2b27fc700 1 -- 172.21.15.52:0/2849320924 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b41991f0 con 0x7fb2b4101eb0 2024-04-02T13:50:13.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.598+0000 7fb2b37fe700 1 --2- 172.21.15.52:0/2849320924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2b4107500 0x7fb2b4198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:50:13.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.598+0000 7fb2b2ffd700 1 --2- 172.21.15.52:0/2849320924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2b41014f0 0x7fb2b419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:50:13.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.598+0000 7fb2b27fc700 1 --2- 172.21.15.52:0/2849320924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2b4101eb0 0x7fb2b419edc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb29c00c9b0 tx=0x7fb29c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:13.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.598+0000 7fb2abfff700 1 -- 172.21.15.52:0/2849320924 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c007b00 con 0x7fb2b4101eb0 2024-04-02T13:50:13.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.598+0000 7fb2abfff700 1 -- 172.21.15.52:0/2849320924 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb29c007ca0 con 0x7fb2b4101eb0 2024-04-02T13:50:13.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.598+0000 7fb2abfff700 1 -- 172.21.15.52:0/2849320924 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c018730 con 0x7fb2b4101eb0 2024-04-02T13:50:13.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.598+0000 7fb2b932e700 1 -- 172.21.15.52:0/2849320924 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b41994e0 con 0x7fb2b4101eb0 2024-04-02T13:50:13.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.598+0000 7fb2b932e700 1 -- 172.21.15.52:0/2849320924 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b41a5750 con 0x7fb2b4101eb0 2024-04-02T13:50:13.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.599+0000 7fb2b932e700 1 -- 172.21.15.52:0/2849320924 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2b40692b0 con 0x7fb2b4101eb0 2024-04-02T13:50:13.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.600+0000 7fb2abfff700 1 -- 172.21.15.52:0/2849320924 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb29c0188d0 con 0x7fb2b4101eb0 2024-04-02T13:50:13.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.601+0000 7fb2abfff700 1 --2- 172.21.15.52:0/2849320924 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb2a00a3cc0 0x7fb2a00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:13.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.601+0000 7fb2b2ffd700 1 --2- 172.21.15.52:0/2849320924 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb2a00a3cc0 0x7fb2a00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:13.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.601+0000 7fb2abfff700 1 -- 172.21.15.52:0/2849320924 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb29c0d0220 con 0x7fb2b4101eb0 2024-04-02T13:50:13.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.601+0000 7fb2b2ffd700 1 --2- 172.21.15.52:0/2849320924 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb2a00a3cc0 0x7fb2a00a6180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009fd0 tx=0x7fb2a400b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:13.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.604+0000 7fb2abfff700 1 -- 172.21.15.52:0/2849320924 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb29c014030 con 0x7fb2b4101eb0 2024-04-02T13:50:13.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.811+0000 7fb2b932e700 1 -- 172.21.15.52:0/2849320924 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb2b4199f70 con 0x7fb2b4101eb0 2024-04-02T13:50:13.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.812+0000 7fb2abfff700 1 -- 172.21.15.52:0/2849320924 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18405 (secure 0 0 0) 0x7fb2b4199f70 con 0x7fb2b4101eb0 2024-04-02T13:50:13.815 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:50:13.815 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":82,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T13:50:12.112448+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:50:13.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.814+0000 7fb2b932e700 1 -- 172.21.15.52:0/2849320924 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb2a00a3cc0 msgr2=0x7fb2a00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:13.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.814+0000 7fb2b932e700 1 --2- 172.21.15.52:0/2849320924 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb2a00a3cc0 0x7fb2a00a6180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009fd0 tx=0x7fb2a400b7b0 comp rx=0 tx=0).stop 2024-04-02T13:50:13.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.814+0000 7fb2b932e700 1 -- 172.21.15.52:0/2849320924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2b4101eb0 msgr2=0x7fb2b419edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:13.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.814+0000 7fb2b932e700 1 --2- 172.21.15.52:0/2849320924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2b4101eb0 0x7fb2b419edc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb29c00c9b0 tx=0x7fb29c00ccc0 comp rx=0 tx=0).stop 2024-04-02T13:50:13.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.814+0000 7fb2b932e700 1 -- 172.21.15.52:0/2849320924 shutdown_connections 2024-04-02T13:50:13.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.814+0000 7fb2b932e700 1 --2- 172.21.15.52:0/2849320924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2b4107500 0x7fb2b4198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:13.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.814+0000 7fb2b932e700 1 --2- 172.21.15.52:0/2849320924 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb2a00a3cc0 0x7fb2a00a6180 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:13.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.814+0000 7fb2b932e700 1 --2- 172.21.15.52:0/2849320924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2b41014f0 0x7fb2b419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:13.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.814+0000 7fb2b932e700 1 --2- 172.21.15.52:0/2849320924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2b4101eb0 0x7fb2b419edc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:13.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.814+0000 7fb2b932e700 1 -- 172.21.15.52:0/2849320924 >> 172.21.15.52:0/2849320924 conn(0x7fb2b4078420 msgr2=0x7fb2b4100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:13.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.814+0000 7fb2b932e700 1 -- 172.21.15.52:0/2849320924 shutdown_connections 2024-04-02T13:50:13.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:13.814+0000 7fb2b932e700 1 -- 172.21.15.52:0/2849320924 wait complete. 2024-04-02T13:50:14.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:14 smithi088 ceph-mon[36528]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:14.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:14 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2849320924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:50:14.448 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:14 smithi052 ceph-mon[32104]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:14.448 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:14 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2849320924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:50:14.477 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:46:20.746053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T13:50:14.477 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph osd pool get .mgr pg_num 2024-04-02T13:50:14.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:14 smithi189 ceph-mon[36683]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:14.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:14 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2849320924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:50:14.719 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:50:16.371 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:16 smithi052 ceph-mon[32104]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:16.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:16 smithi088 ceph-mon[36528]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:16.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.475+0000 7fede36b9700 1 -- 172.21.15.52:0/2848694140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feddc06a2a0 msgr2=0x7feddc06a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:16.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.475+0000 7fede36b9700 1 --2- 172.21.15.52:0/2848694140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feddc06a2a0 0x7feddc06a720 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fedcc009990 tx=0x7fedcc009ca0 comp rx=0 tx=0).stop 2024-04-02T13:50:16.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.475+0000 7fede36b9700 1 -- 172.21.15.52:0/2848694140 shutdown_connections 2024-04-02T13:50:16.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.475+0000 7fede36b9700 1 --2- 172.21.15.52:0/2848694140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feddc06ac60 0x7feddc10f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:16.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.475+0000 7fede36b9700 1 --2- 172.21.15.52:0/2848694140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feddc108650 0x7feddc108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:16.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.475+0000 7fede36b9700 1 --2- 172.21.15.52:0/2848694140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feddc06a2a0 0x7feddc06a720 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:16.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.475+0000 7fede36b9700 1 -- 172.21.15.52:0/2848694140 >> 172.21.15.52:0/2848694140 conn(0x7feddc078420 msgr2=0x7feddc078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:16.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.475+0000 7fede36b9700 1 -- 172.21.15.52:0/2848694140 shutdown_connections 2024-04-02T13:50:16.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.476+0000 7fede36b9700 1 -- 172.21.15.52:0/2848694140 wait complete. 2024-04-02T13:50:16.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.476+0000 7fede36b9700 1 Processor -- start 2024-04-02T13:50:16.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.476+0000 7fede36b9700 1 -- start start 2024-04-02T13:50:16.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.476+0000 7fede36b9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feddc06a2a0 0x7feddc198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:16.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.476+0000 7fede36b9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feddc06ac60 0x7feddc199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:16.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.476+0000 7fede36b9700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feddc108650 0x7feddc19d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:16.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.476+0000 7fede36b9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feddc112940 con 0x7feddc06a2a0 2024-04-02T13:50:16.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.476+0000 7fede36b9700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feddc1127c0 con 0x7feddc06ac60 2024-04-02T13:50:16.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.476+0000 7fede36b9700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feddc112ac0 con 0x7feddc108650 2024-04-02T13:50:16.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.476+0000 7fede1455700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feddc06a2a0 0x7feddc198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:16.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.476+0000 7fede1455700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feddc06a2a0 0x7feddc198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33392/0 (socket says 172.21.15.52:33392) 2024-04-02T13:50:16.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.476+0000 7fede1455700 1 -- 172.21.15.52:0/3739400178 learned_addr learned my addr 172.21.15.52:0/3739400178 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:16.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.477+0000 7fede0c54700 1 --2- 172.21.15.52:0/3739400178 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feddc06ac60 0x7feddc199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:16.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.477+0000 7fede1c56700 1 --2- 172.21.15.52:0/3739400178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feddc108650 0x7feddc19d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:16.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.477+0000 7fede1455700 1 -- 172.21.15.52:0/3739400178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feddc108650 msgr2=0x7feddc19d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:16.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.477+0000 7fede1455700 1 --2- 172.21.15.52:0/3739400178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feddc108650 0x7feddc19d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:16.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.477+0000 7fede1455700 1 -- 172.21.15.52:0/3739400178 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feddc06ac60 msgr2=0x7feddc199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:16.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.477+0000 7fede1455700 1 --2- 172.21.15.52:0/3739400178 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feddc06ac60 0x7feddc199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:16.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.477+0000 7fede1455700 1 -- 172.21.15.52:0/3739400178 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feddc19dd60 con 0x7feddc06a2a0 2024-04-02T13:50:16.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.477+0000 7fede1455700 1 --2- 172.21.15.52:0/3739400178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feddc06a2a0 0x7feddc198d50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fedd800d980 tx=0x7fedd800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:16.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.477+0000 7fedd27fc700 1 -- 172.21.15.52:0/3739400178 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8017070 con 0x7feddc06a2a0 2024-04-02T13:50:16.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.477+0000 7fede36b9700 1 -- 172.21.15.52:0/3739400178 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feddc19e050 con 0x7feddc06a2a0 2024-04-02T13:50:16.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.477+0000 7fedd27fc700 1 -- 172.21.15.52:0/3739400178 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedd8005020 con 0x7feddc06a2a0 2024-04-02T13:50:16.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.477+0000 7fedd27fc700 1 -- 172.21.15.52:0/3739400178 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8009bc0 con 0x7feddc06a2a0 2024-04-02T13:50:16.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.477+0000 7fede36b9700 1 -- 172.21.15.52:0/3739400178 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feddc1a58d0 con 0x7feddc06a2a0 2024-04-02T13:50:16.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.478+0000 7fede36b9700 1 -- 172.21.15.52:0/3739400178 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feddc0692b0 con 0x7feddc06a2a0 2024-04-02T13:50:16.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.479+0000 7fedd27fc700 1 -- 172.21.15.52:0/3739400178 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fedd80051d0 con 0x7feddc06a2a0 2024-04-02T13:50:16.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.480+0000 7fedd27fc700 1 --2- 172.21.15.52:0/3739400178 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fedc80a3c40 0x7fedc80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:16.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.480+0000 7fede0c54700 1 --2- 172.21.15.52:0/3739400178 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fedc80a3c40 0x7fedc80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:16.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.480+0000 7fedd27fc700 1 -- 172.21.15.52:0/3739400178 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fedd80cff00 con 0x7feddc06a2a0 2024-04-02T13:50:16.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.480+0000 7fede0c54700 1 --2- 172.21.15.52:0/3739400178 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fedc80a3c40 0x7fedc80a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feddc078b00 tx=0x7fedcc00b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:16.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.485+0000 7fedd27fc700 1 -- 172.21.15.52:0/3739400178 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fedd8020020 con 0x7feddc06a2a0 2024-04-02T13:50:16.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:16 smithi189 ceph-mon[36683]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:16.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.690+0000 7fede36b9700 1 -- 172.21.15.52:0/3739400178 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feddc02dd30 con 0x7feddc06a2a0 2024-04-02T13:50:16.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.690+0000 7fedd27fc700 1 -- 172.21.15.52:0/3739400178 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fedd809cce0 con 0x7feddc06a2a0 2024-04-02T13:50:16.693 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-04-02T13:50:16.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.692+0000 7fede36b9700 1 -- 172.21.15.52:0/3739400178 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fedc80a3c40 msgr2=0x7fedc80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:16.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.692+0000 7fede36b9700 1 --2- 172.21.15.52:0/3739400178 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fedc80a3c40 0x7fedc80a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feddc078b00 tx=0x7fedcc00b560 comp rx=0 tx=0).stop 2024-04-02T13:50:16.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.692+0000 7fede36b9700 1 -- 172.21.15.52:0/3739400178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feddc06a2a0 msgr2=0x7feddc198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:16.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.692+0000 7fede36b9700 1 --2- 172.21.15.52:0/3739400178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feddc06a2a0 0x7feddc198d50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fedd800d980 tx=0x7fedd800dc90 comp rx=0 tx=0).stop 2024-04-02T13:50:16.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.692+0000 7fede36b9700 1 -- 172.21.15.52:0/3739400178 shutdown_connections 2024-04-02T13:50:16.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.692+0000 7fede36b9700 1 --2- 172.21.15.52:0/3739400178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feddc108650 0x7feddc19d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:16.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.692+0000 7fede36b9700 1 --2- 172.21.15.52:0/3739400178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feddc06a2a0 0x7feddc198d50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:16.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.692+0000 7fede36b9700 1 --2- 172.21.15.52:0/3739400178 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fedc80a3c40 0x7fedc80a6100 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:16.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.692+0000 7fede36b9700 1 --2- 172.21.15.52:0/3739400178 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feddc06ac60 0x7feddc199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:16.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.692+0000 7fede36b9700 1 -- 172.21.15.52:0/3739400178 >> 172.21.15.52:0/3739400178 conn(0x7feddc078420 msgr2=0x7feddc100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:16.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.693+0000 7fede36b9700 1 -- 172.21.15.52:0/3739400178 shutdown_connections 2024-04-02T13:50:16.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:16.693+0000 7fede36b9700 1 -- 172.21.15.52:0/3739400178 wait complete. 2024-04-02T13:50:17.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:17 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3739400178' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:50:17.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:17 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3739400178' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:50:17.488 INFO:tasks.cephadm:Adding mds.a on smithi052 2024-04-02T13:50:17.489 INFO:tasks.cephadm:Adding mds.d on smithi052 2024-04-02T13:50:17.489 INFO:tasks.cephadm:Adding mds.g on smithi052 2024-04-02T13:50:17.489 INFO:tasks.cephadm:Adding mds.j on smithi052 2024-04-02T13:50:17.489 INFO:tasks.cephadm:Adding mds.b on smithi088 2024-04-02T13:50:17.489 INFO:tasks.cephadm:Adding mds.e on smithi088 2024-04-02T13:50:17.489 INFO:tasks.cephadm:Adding mds.h on smithi088 2024-04-02T13:50:17.489 INFO:tasks.cephadm:Adding mds.k on smithi088 2024-04-02T13:50:17.489 INFO:tasks.cephadm:Adding mds.c on smithi189 2024-04-02T13:50:17.489 INFO:tasks.cephadm:Adding mds.f on smithi189 2024-04-02T13:50:17.489 INFO:tasks.cephadm:Adding mds.i on smithi189 2024-04-02T13:50:17.489 INFO:tasks.cephadm:Adding mds.l on smithi189 2024-04-02T13:50:17.490 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph orch apply mds all '12;smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi088=b;smithi088=e;smithi088=h;smithi088=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l' 2024-04-02T13:50:17.521 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:17 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3739400178' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:50:17.741 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:50:18.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:18 smithi088 ceph-mon[36528]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:18.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:18 smithi189 ceph-mon[36683]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:18.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:18 smithi052 ceph-mon[32104]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:19.410 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.405+0000 7efd24457700 1 -- 172.21.15.189:0/865234248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd1c1041c0 msgr2=0x7efd1c10aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:19.410 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.405+0000 7efd24457700 1 --2- 172.21.15.189:0/865234248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd1c1041c0 0x7efd1c10aa70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efd0c009990 tx=0x7efd0c009ca0 comp rx=0 tx=0).stop 2024-04-02T13:50:19.410 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.405+0000 7efd24457700 1 -- 172.21.15.189:0/865234248 shutdown_connections 2024-04-02T13:50:19.410 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.405+0000 7efd24457700 1 --2- 172.21.15.189:0/865234248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd1c1041c0 0x7efd1c10aa70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:19.410 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.405+0000 7efd24457700 1 --2- 172.21.15.189:0/865234248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd1c102620 0x7efd1c102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:19.410 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.405+0000 7efd24457700 1 --2- 172.21.15.189:0/865234248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd1c103820 0x7efd1c103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:19.410 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.405+0000 7efd24457700 1 -- 172.21.15.189:0/865234248 >> 172.21.15.189:0/865234248 conn(0x7efd1c0fdc00 msgr2=0x7efd1c100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:19.411 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.405+0000 7efd24457700 1 -- 172.21.15.189:0/865234248 shutdown_connections 2024-04-02T13:50:19.411 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.405+0000 7efd24457700 1 -- 172.21.15.189:0/865234248 wait complete. 2024-04-02T13:50:19.411 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.406+0000 7efd24457700 1 Processor -- start 2024-04-02T13:50:19.411 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.406+0000 7efd24457700 1 -- start start 2024-04-02T13:50:19.411 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.406+0000 7efd24457700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd1c102620 0x7efd1c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:19.411 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.406+0000 7efd24457700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd1c103820 0x7efd1c198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:19.411 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.406+0000 7efd24457700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd1c1041c0 0x7efd1c19fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:19.411 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.406+0000 7efd24457700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c10e200 con 0x7efd1c1041c0 2024-04-02T13:50:19.411 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.406+0000 7efd24457700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c10e080 con 0x7efd1c103820 2024-04-02T13:50:19.411 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.406+0000 7efd24457700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c10e380 con 0x7efd1c102620 2024-04-02T13:50:19.411 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.406+0000 7efd221f3700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd1c102620 0x7efd1c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:19.411 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.406+0000 7efd221f3700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd1c102620 0x7efd1c1987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:55506/0 (socket says 172.21.15.189:55506) 2024-04-02T13:50:19.411 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.406+0000 7efd221f3700 1 -- 172.21.15.189:0/2578237422 learned_addr learned my addr 172.21.15.189:0/2578237422 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-02T13:50:19.411 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.406+0000 7efd229f4700 1 --2- 172.21.15.189:0/2578237422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd1c1041c0 0x7efd1c19fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:19.412 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.406+0000 7efd219f2700 1 --2- 172.21.15.189:0/2578237422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd1c103820 0x7efd1c198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:19.412 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.406+0000 7efd221f3700 1 -- 172.21.15.189:0/2578237422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd1c103820 msgr2=0x7efd1c198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:19.412 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.406+0000 7efd221f3700 1 --2- 172.21.15.189:0/2578237422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd1c103820 0x7efd1c198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:19.412 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.406+0000 7efd221f3700 1 -- 172.21.15.189:0/2578237422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd1c1041c0 msgr2=0x7efd1c19fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:19.412 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.406+0000 7efd221f3700 1 --2- 172.21.15.189:0/2578237422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd1c1041c0 0x7efd1c19fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:19.412 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.406+0000 7efd221f3700 1 -- 172.21.15.189:0/2578237422 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1c1a0420 con 0x7efd1c102620 2024-04-02T13:50:19.412 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.406+0000 7efd221f3700 1 --2- 172.21.15.189:0/2578237422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd1c102620 0x7efd1c1987c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efd1400add0 tx=0x7efd1400c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:19.412 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.407+0000 7efd1b7fe700 1 -- 172.21.15.189:0/2578237422 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14017070 con 0x7efd1c102620 2024-04-02T13:50:19.412 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.407+0000 7efd1b7fe700 1 -- 172.21.15.189:0/2578237422 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd14004bb0 con 0x7efd1c102620 2024-04-02T13:50:19.412 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.407+0000 7efd24457700 1 -- 172.21.15.189:0/2578237422 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1c1a0710 con 0x7efd1c102620 2024-04-02T13:50:19.412 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.407+0000 7efd1b7fe700 1 -- 172.21.15.189:0/2578237422 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14004020 con 0x7efd1c102620 2024-04-02T13:50:19.412 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.407+0000 7efd24457700 1 -- 172.21.15.189:0/2578237422 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd1c1a0c50 con 0x7efd1c102620 2024-04-02T13:50:19.413 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.408+0000 7efd24457700 1 -- 172.21.15.189:0/2578237422 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd1c0692b0 con 0x7efd1c102620 2024-04-02T13:50:19.413 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.409+0000 7efd1b7fe700 1 -- 172.21.15.189:0/2578237422 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efd1401a070 con 0x7efd1c102620 2024-04-02T13:50:19.413 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.409+0000 7efd1b7fe700 1 --2- 172.21.15.189:0/2578237422 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7efd040a3cc0 0x7efd040a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:19.413 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.409+0000 7efd1b7fe700 1 -- 172.21.15.189:0/2578237422 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7efd140cf140 con 0x7efd1c102620 2024-04-02T13:50:19.414 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.410+0000 7efd219f2700 1 --2- 172.21.15.189:0/2578237422 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7efd040a3cc0 0x7efd040a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:19.414 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.410+0000 7efd219f2700 1 --2- 172.21.15.189:0/2578237422 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7efd040a3cc0 0x7efd040a6180 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efd10005fd0 tx=0x7efd10005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:19.416 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.413+0000 7efd1b7fe700 1 -- 172.21.15.189:0/2578237422 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7efd1409bfc0 con 0x7efd1c102620 2024-04-02T13:50:19.625 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.621+0000 7efd24457700 1 -- 172.21.15.189:0/2578237422 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi088=b;smithi088=e;smithi088=h;smithi088=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}) -- 0x7efd1c062cf0 con 0x7efd040a3cc0 2024-04-02T13:50:19.632 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.629+0000 7efd1b7fe700 1 -- 172.21.15.189:0/2578237422 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7efd1c062cf0 con 0x7efd040a3cc0 2024-04-02T13:50:19.632 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mds.all update... 2024-04-02T13:50:19.635 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.632+0000 7efd24457700 1 -- 172.21.15.189:0/2578237422 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7efd040a3cc0 msgr2=0x7efd040a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:19.635 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.632+0000 7efd24457700 1 --2- 172.21.15.189:0/2578237422 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7efd040a3cc0 0x7efd040a6180 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efd10005fd0 tx=0x7efd10005f60 comp rx=0 tx=0).stop 2024-04-02T13:50:19.635 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.632+0000 7efd24457700 1 -- 172.21.15.189:0/2578237422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd1c102620 msgr2=0x7efd1c1987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:19.635 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.632+0000 7efd24457700 1 --2- 172.21.15.189:0/2578237422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd1c102620 0x7efd1c1987c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efd1400add0 tx=0x7efd1400c330 comp rx=0 tx=0).stop 2024-04-02T13:50:19.635 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.632+0000 7efd24457700 1 -- 172.21.15.189:0/2578237422 shutdown_connections 2024-04-02T13:50:19.635 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.632+0000 7efd24457700 1 --2- 172.21.15.189:0/2578237422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd1c1041c0 0x7efd1c19fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:19.635 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.632+0000 7efd24457700 1 --2- 172.21.15.189:0/2578237422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd1c102620 0x7efd1c1987c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:19.635 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.632+0000 7efd24457700 1 --2- 172.21.15.189:0/2578237422 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7efd040a3cc0 0x7efd040a6180 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:19.635 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.632+0000 7efd24457700 1 --2- 172.21.15.189:0/2578237422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd1c103820 0x7efd1c198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:19.635 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.632+0000 7efd24457700 1 -- 172.21.15.189:0/2578237422 >> 172.21.15.189:0/2578237422 conn(0x7efd1c0fdc00 msgr2=0x7efd1c106ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:19.635 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.632+0000 7efd24457700 1 -- 172.21.15.189:0/2578237422 shutdown_connections 2024-04-02T13:50:19.635 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-02T13:50:19.632+0000 7efd24457700 1 -- 172.21.15.189:0/2578237422 wait complete. 2024-04-02T13:50:20.369 DEBUG:teuthology.orchestra.run.smithi052:mds.a> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.a.service 2024-04-02T13:50:20.373 DEBUG:teuthology.orchestra.run.smithi052:mds.d> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.d.service 2024-04-02T13:50:20.416 DEBUG:teuthology.orchestra.run.smithi052:mds.g> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.g.service 2024-04-02T13:50:20.459 DEBUG:teuthology.orchestra.run.smithi052:mds.j> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.j.service 2024-04-02T13:50:20.502 DEBUG:teuthology.orchestra.run.smithi088:mds.b> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.b.service 2024-04-02T13:50:20.504 DEBUG:teuthology.orchestra.run.smithi088:mds.e> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.e.service 2024-04-02T13:50:20.547 DEBUG:teuthology.orchestra.run.smithi088:mds.h> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.h.service 2024-04-02T13:50:20.590 DEBUG:teuthology.orchestra.run.smithi088:mds.k> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.k.service 2024-04-02T13:50:20.634 DEBUG:teuthology.orchestra.run.smithi189:mds.c> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.c.service 2024-04-02T13:50:20.637 DEBUG:teuthology.orchestra.run.smithi189:mds.f> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.f.service 2024-04-02T13:50:20.680 DEBUG:teuthology.orchestra.run.smithi189:mds.i> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.i.service 2024-04-02T13:50:20.723 DEBUG:teuthology.orchestra.run.smithi189:mds.l> sudo journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.l.service 2024-04-02T13:50:20.765 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-02T13:50:20.766 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:50:20.791 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:20 smithi052 ceph-mon[32104]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:20.791 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:20 smithi052 ceph-mon[32104]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi088=b;smithi088=e;smithi088=h;smithi088=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:50:20.792 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:20 smithi052 ceph-mon[32104]: Saving service mds.all spec with placement smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi088=b;smithi088=e;smithi088=h;smithi088=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-04-02T13:50:20.792 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:20 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:20.792 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:20 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:50:20.792 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:20 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:20.792 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:20 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:50:20.792 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:20 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:20.792 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:20 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:20.792 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:20 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:20.792 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:20 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:20.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.881+0000 7f58d382a640 1 -- 172.21.15.52:0/2218731164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58cc1078c0 msgr2=0x7f58cc109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:20.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.881+0000 7f58d382a640 1 --2- 172.21.15.52:0/2218731164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58cc1078c0 0x7f58cc109cb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f58c4009ca0 tx=0x7f58c402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:50:20.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.881+0000 7f58d382a640 1 -- 172.21.15.52:0/2218731164 shutdown_connections 2024-04-02T13:50:20.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.881+0000 7f58d382a640 1 --2- 172.21.15.52:0/2218731164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58cc1078c0 0x7f58cc109cb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:20.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.881+0000 7f58d382a640 1 --2- 172.21.15.52:0/2218731164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58cc104f90 0x7f58cc107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:20.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.881+0000 7f58d382a640 1 --2- 172.21.15.52:0/2218731164 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58cc102660 0x7f58cc104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:20.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.881+0000 7f58d382a640 1 -- 172.21.15.52:0/2218731164 >> 172.21.15.52:0/2218731164 conn(0x7f58cc0fbfb0 msgr2=0x7f58cc0fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:20.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.881+0000 7f58d382a640 1 -- 172.21.15.52:0/2218731164 shutdown_connections 2024-04-02T13:50:20.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.881+0000 7f58d382a640 1 -- 172.21.15.52:0/2218731164 wait complete. 2024-04-02T13:50:20.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.882+0000 7f58d382a640 1 Processor -- start 2024-04-02T13:50:20.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.882+0000 7f58d382a640 1 -- start start 2024-04-02T13:50:20.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.882+0000 7f58d382a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58cc102660 0x7f58cc197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:20.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.882+0000 7f58d382a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58cc104f90 0x7f58cc198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:20.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.882+0000 7f58d382a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58cc1078c0 0x7f58cc19f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:20.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.882+0000 7f58d382a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc10d7e0 con 0x7f58cc1078c0 2024-04-02T13:50:20.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.882+0000 7f58d382a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc10d660 con 0x7f58cc102660 2024-04-02T13:50:20.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.882+0000 7f58d382a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc10d960 con 0x7f58cc104f90 2024-04-02T13:50:20.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.882+0000 7f58d1dc7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58cc1078c0 0x7f58cc19f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:20.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.882+0000 7f58d1dc7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58cc1078c0 0x7f58cc19f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33408/0 (socket says 172.21.15.52:33408) 2024-04-02T13:50:20.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.882+0000 7f58d1dc7640 1 -- 172.21.15.52:0/3135125554 learned_addr learned my addr 172.21.15.52:0/3135125554 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:20.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.882+0000 7f58d15c6640 1 --2- 172.21.15.52:0/3135125554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58cc102660 0x7f58cc197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:20.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.882+0000 7f58d0dc5640 1 --2- 172.21.15.52:0/3135125554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58cc104f90 0x7f58cc198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:20.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.883+0000 7f58d1dc7640 1 -- 172.21.15.52:0/3135125554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58cc104f90 msgr2=0x7f58cc198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:20.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.883+0000 7f58d1dc7640 1 --2- 172.21.15.52:0/3135125554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58cc104f90 0x7f58cc198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:20.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.883+0000 7f58d1dc7640 1 -- 172.21.15.52:0/3135125554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58cc102660 msgr2=0x7f58cc197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:20.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.883+0000 7f58d1dc7640 1 --2- 172.21.15.52:0/3135125554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58cc102660 0x7f58cc197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:20.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.883+0000 7f58d1dc7640 1 -- 172.21.15.52:0/3135125554 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58cc19f9e0 con 0x7f58cc1078c0 2024-04-02T13:50:20.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.883+0000 7f58d1dc7640 1 --2- 172.21.15.52:0/3135125554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58cc1078c0 0x7f58cc19f370 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f58c4037040 tx=0x7f58c4032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:20.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.883+0000 7f58c27fc640 1 -- 172.21.15.52:0/3135125554 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4041020 con 0x7f58cc1078c0 2024-04-02T13:50:20.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.883+0000 7f58d382a640 1 -- 172.21.15.52:0/3135125554 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58cc19fc70 con 0x7f58cc1078c0 2024-04-02T13:50:20.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.883+0000 7f58c27fc640 1 -- 172.21.15.52:0/3135125554 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58c4034cd0 con 0x7f58cc1078c0 2024-04-02T13:50:20.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.883+0000 7f58c27fc640 1 -- 172.21.15.52:0/3135125554 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4039040 con 0x7f58cc1078c0 2024-04-02T13:50:20.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.883+0000 7f58d382a640 1 -- 172.21.15.52:0/3135125554 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58cc1a0160 con 0x7f58cc1078c0 2024-04-02T13:50:20.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.884+0000 7f58d382a640 1 -- 172.21.15.52:0/3135125554 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58cc0fdaa0 con 0x7f58cc1078c0 2024-04-02T13:50:20.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.885+0000 7f58c27fc640 1 -- 172.21.15.52:0/3135125554 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f58c4044020 con 0x7f58cc1078c0 2024-04-02T13:50:20.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.886+0000 7f58c27fc640 1 --2- 172.21.15.52:0/3135125554 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f58b80a3bb0 0x7f58b80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:20.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.886+0000 7f58d15c6640 1 --2- 172.21.15.52:0/3135125554 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f58b80a3bb0 0x7f58b80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:20.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.886+0000 7f58c27fc640 1 -- 172.21.15.52:0/3135125554 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f58c40f1e30 con 0x7f58cc1078c0 2024-04-02T13:50:20.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.886+0000 7f58d15c6640 1 --2- 172.21.15.52:0/3135125554 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f58b80a3bb0 0x7f58b80a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f58c800abe0 tx=0x7f58c800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:20.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:20.891+0000 7f58c27fc640 1 -- 172.21.15.52:0/3135125554 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f58c404b020 con 0x7f58cc1078c0 2024-04-02T13:50:20.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:20 smithi088 ceph-mon[36528]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:20.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:20 smithi088 ceph-mon[36528]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi088=b;smithi088=e;smithi088=h;smithi088=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:50:20.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:20 smithi088 ceph-mon[36528]: Saving service mds.all spec with placement smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi088=b;smithi088=e;smithi088=h;smithi088=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-04-02T13:50:20.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:20 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:20.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:20 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:50:20.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:20 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:20.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:20 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:50:20.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:20 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:20.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:20 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:20.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:20 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:20.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:20 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:21.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:20 smithi189 ceph-mon[36683]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:21.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:20 smithi189 ceph-mon[36683]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi088=b;smithi088=e;smithi088=h;smithi088=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:50:21.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:20 smithi189 ceph-mon[36683]: Saving service mds.all spec with placement smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi088=b;smithi088=e;smithi088=h;smithi088=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-04-02T13:50:21.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:20 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:21.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:20 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:50:21.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:20 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:21.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:20 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:50:21.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:20 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:21.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:20 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:21.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:20 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:21.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:20 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:21.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.076+0000 7f58d382a640 1 -- 172.21.15.52:0/3135125554 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58cc02d5c0 con 0x7f58cc1078c0 2024-04-02T13:50:21.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.077+0000 7f58c27fc640 1 -- 172.21.15.52:0/3135125554 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18405 (secure 0 0 0) 0x7f58c40becb0 con 0x7f58cc1078c0 2024-04-02T13:50:21.081 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:50:21.081 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":82,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T13:50:12.112448+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:50:21.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.079+0000 7f58d382a640 1 -- 172.21.15.52:0/3135125554 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f58b80a3bb0 msgr2=0x7f58b80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:21.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.079+0000 7f58d382a640 1 --2- 172.21.15.52:0/3135125554 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f58b80a3bb0 0x7f58b80a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f58c800abe0 tx=0x7f58c800a2d0 comp rx=0 tx=0).stop 2024-04-02T13:50:21.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.079+0000 7f58d382a640 1 -- 172.21.15.52:0/3135125554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58cc1078c0 msgr2=0x7f58cc19f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:21.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.079+0000 7f58d382a640 1 --2- 172.21.15.52:0/3135125554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58cc1078c0 0x7f58cc19f370 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f58c4037040 tx=0x7f58c4032f20 comp rx=0 tx=0).stop 2024-04-02T13:50:21.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.079+0000 7f58d382a640 1 -- 172.21.15.52:0/3135125554 shutdown_connections 2024-04-02T13:50:21.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.079+0000 7f58d382a640 1 --2- 172.21.15.52:0/3135125554 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f58b80a3bb0 0x7f58b80a6070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:21.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.079+0000 7f58d382a640 1 --2- 172.21.15.52:0/3135125554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58cc1078c0 0x7f58cc19f370 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:21.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.079+0000 7f58d382a640 1 --2- 172.21.15.52:0/3135125554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58cc104f90 0x7f58cc198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:21.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.079+0000 7f58d382a640 1 --2- 172.21.15.52:0/3135125554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58cc102660 0x7f58cc197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:21.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.079+0000 7f58d382a640 1 -- 172.21.15.52:0/3135125554 >> 172.21.15.52:0/3135125554 conn(0x7f58cc0fbfb0 msgr2=0x7f58cc102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:21.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.079+0000 7f58d382a640 1 -- 172.21.15.52:0/3135125554 shutdown_connections 2024-04-02T13:50:21.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.079+0000 7f58d382a640 1 -- 172.21.15.52:0/3135125554 wait complete. 2024-04-02T13:50:21.094 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:46:20.746053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T13:50:21.094 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:50:21.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.237+0000 7fb6f3aeb640 1 -- 172.21.15.52:0/513854253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6ec1025a0 msgr2=0x7fb6ec104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:21.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.237+0000 7fb6f3aeb640 1 --2- 172.21.15.52:0/513854253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6ec1025a0 0x7fb6ec104990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009900 tx=0x7fb6e802bc30 comp rx=0 tx=0).stop 2024-04-02T13:50:21.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.237+0000 7fb6f3aeb640 1 -- 172.21.15.52:0/513854253 shutdown_connections 2024-04-02T13:50:21.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.237+0000 7fb6f3aeb640 1 --2- 172.21.15.52:0/513854253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6ec107800 0x7fb6ec109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:21.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.237+0000 7fb6f3aeb640 1 --2- 172.21.15.52:0/513854253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6ec104ed0 0x7fb6ec1072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:21.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.237+0000 7fb6f3aeb640 1 --2- 172.21.15.52:0/513854253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6ec1025a0 0x7fb6ec104990 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:21.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.237+0000 7fb6f3aeb640 1 -- 172.21.15.52:0/513854253 >> 172.21.15.52:0/513854253 conn(0x7fb6ec0fbfb0 msgr2=0x7fb6ec0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:21.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.237+0000 7fb6f3aeb640 1 -- 172.21.15.52:0/513854253 shutdown_connections 2024-04-02T13:50:21.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.237+0000 7fb6f3aeb640 1 -- 172.21.15.52:0/513854253 wait complete. 2024-04-02T13:50:21.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.237+0000 7fb6f3aeb640 1 Processor -- start 2024-04-02T13:50:21.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.237+0000 7fb6f3aeb640 1 -- start start 2024-04-02T13:50:21.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.237+0000 7fb6f3aeb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6ec1025a0 0x7fb6ec101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:21.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.237+0000 7fb6f3aeb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6ec104ed0 0x7fb6ec100550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:21.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.237+0000 7fb6f3aeb640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6ec107800 0x7fb6ec100a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:21.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.237+0000 7fb6f3aeb640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec10d7c0 con 0x7fb6ec104ed0 2024-04-02T13:50:21.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.237+0000 7fb6f3aeb640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec10d640 con 0x7fb6ec107800 2024-04-02T13:50:21.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.237+0000 7fb6f3aeb640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec10d940 con 0x7fb6ec1025a0 2024-04-02T13:50:21.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.237+0000 7fb6f1086640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6ec104ed0 0x7fb6ec100550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:21.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.238+0000 7fb6f1086640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6ec104ed0 0x7fb6ec100550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33426/0 (socket says 172.21.15.52:33426) 2024-04-02T13:50:21.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.238+0000 7fb6f1086640 1 -- 172.21.15.52:0/1237989816 learned_addr learned my addr 172.21.15.52:0/1237989816 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:21.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.238+0000 7fb6f1887640 1 --2- 172.21.15.52:0/1237989816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6ec1025a0 0x7fb6ec101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:21.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.238+0000 7fb6f1086640 1 -- 172.21.15.52:0/1237989816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6ec1025a0 msgr2=0x7fb6ec101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:21.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.238+0000 7fb6f1086640 1 --2- 172.21.15.52:0/1237989816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6ec1025a0 0x7fb6ec101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:21.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.238+0000 7fb6f2088640 1 --2- 172.21.15.52:0/1237989816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6ec107800 0x7fb6ec100a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:21.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.238+0000 7fb6f1086640 1 -- 172.21.15.52:0/1237989816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6ec107800 msgr2=0x7fb6ec100a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:21.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.238+0000 7fb6f1086640 1 --2- 172.21.15.52:0/1237989816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6ec107800 0x7fb6ec100a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:21.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.238+0000 7fb6f1086640 1 -- 172.21.15.52:0/1237989816 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6ec101320 con 0x7fb6ec104ed0 2024-04-02T13:50:21.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.238+0000 7fb6f1086640 1 --2- 172.21.15.52:0/1237989816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6ec104ed0 0x7fb6ec100550 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00c970 tx=0x7fb6dc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:21.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.238+0000 7fb6e2ffd640 1 -- 172.21.15.52:0/1237989816 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc007bb0 con 0x7fb6ec104ed0 2024-04-02T13:50:21.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.238+0000 7fb6e2ffd640 1 -- 172.21.15.52:0/1237989816 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6dc007d50 con 0x7fb6ec104ed0 2024-04-02T13:50:21.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.238+0000 7fb6f3aeb640 1 -- 172.21.15.52:0/1237989816 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6ec1a1f80 con 0x7fb6ec104ed0 2024-04-02T13:50:21.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.238+0000 7fb6e2ffd640 1 -- 172.21.15.52:0/1237989816 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc00f450 con 0x7fb6ec104ed0 2024-04-02T13:50:21.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.238+0000 7fb6f3aeb640 1 -- 172.21.15.52:0/1237989816 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6ec1a23e0 con 0x7fb6ec104ed0 2024-04-02T13:50:21.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.239+0000 7fb6f3aeb640 1 -- 172.21.15.52:0/1237989816 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6ec0fdaa0 con 0x7fb6ec104ed0 2024-04-02T13:50:21.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.240+0000 7fb6e2ffd640 1 -- 172.21.15.52:0/1237989816 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb6dc0046c0 con 0x7fb6ec104ed0 2024-04-02T13:50:21.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.241+0000 7fb6e2ffd640 1 --2- 172.21.15.52:0/1237989816 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb6d80a3bb0 0x7fb6d80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:21.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.241+0000 7fb6f1887640 1 --2- 172.21.15.52:0/1237989816 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb6d80a3bb0 0x7fb6d80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:21.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.241+0000 7fb6e2ffd640 1 -- 172.21.15.52:0/1237989816 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb6dc0cfff0 con 0x7fb6ec104ed0 2024-04-02T13:50:21.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.241+0000 7fb6f1887640 1 --2- 172.21.15.52:0/1237989816 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb6d80a3bb0 0x7fb6d80a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb6e802c7b0 tx=0x7fb6e8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:21.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.245+0000 7fb6e2ffd640 1 -- 172.21.15.52:0/1237989816 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb6dc014030 con 0x7fb6ec104ed0 2024-04-02T13:50:21.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.429+0000 7fb6f3aeb640 1 -- 172.21.15.52:0/1237989816 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb6ec062840 con 0x7fb6ec104ed0 2024-04-02T13:50:21.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.430+0000 7fb6e2ffd640 1 -- 172.21.15.52:0/1237989816 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fb6dc09cdd0 con 0x7fb6ec104ed0 2024-04-02T13:50:21.433 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-04-02T13:50:21.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.431+0000 7fb6f3aeb640 1 -- 172.21.15.52:0/1237989816 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb6d80a3bb0 msgr2=0x7fb6d80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:21.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.431+0000 7fb6f3aeb640 1 --2- 172.21.15.52:0/1237989816 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb6d80a3bb0 0x7fb6d80a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb6e802c7b0 tx=0x7fb6e8005fd0 comp rx=0 tx=0).stop 2024-04-02T13:50:21.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.431+0000 7fb6f3aeb640 1 -- 172.21.15.52:0/1237989816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6ec104ed0 msgr2=0x7fb6ec100550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:21.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.431+0000 7fb6f3aeb640 1 --2- 172.21.15.52:0/1237989816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6ec104ed0 0x7fb6ec100550 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00c970 tx=0x7fb6dc00ce30 comp rx=0 tx=0).stop 2024-04-02T13:50:21.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.432+0000 7fb6f3aeb640 1 -- 172.21.15.52:0/1237989816 shutdown_connections 2024-04-02T13:50:21.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.432+0000 7fb6f3aeb640 1 --2- 172.21.15.52:0/1237989816 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb6d80a3bb0 0x7fb6d80a6070 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:21.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.432+0000 7fb6f3aeb640 1 --2- 172.21.15.52:0/1237989816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6ec107800 0x7fb6ec100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:21.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.432+0000 7fb6f3aeb640 1 --2- 172.21.15.52:0/1237989816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6ec104ed0 0x7fb6ec100550 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:21.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.432+0000 7fb6f3aeb640 1 --2- 172.21.15.52:0/1237989816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6ec1025a0 0x7fb6ec101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:21.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.432+0000 7fb6f3aeb640 1 -- 172.21.15.52:0/1237989816 >> 172.21.15.52:0/1237989816 conn(0x7fb6ec0fbfb0 msgr2=0x7fb6ec102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:21.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.432+0000 7fb6f3aeb640 1 -- 172.21.15.52:0/1237989816 shutdown_connections 2024-04-02T13:50:21.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.432+0000 7fb6f3aeb640 1 -- 172.21.15.52:0/1237989816 wait complete. 2024-04-02T13:50:21.446 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:50:21.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.545+0000 7f311bde9640 1 -- 172.21.15.52:0/565143632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3114101c90 msgr2=0x7f3114102110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:21.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.545+0000 7f311bde9640 1 --2- 172.21.15.52:0/565143632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3114101c90 0x7f3114102110 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3104009900 tx=0x7f310402bc30 comp rx=0 tx=0).stop 2024-04-02T13:50:21.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.545+0000 7f311bde9640 1 -- 172.21.15.52:0/565143632 shutdown_connections 2024-04-02T13:50:21.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.545+0000 7f311bde9640 1 --2- 172.21.15.52:0/565143632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3114102650 0x7f3114108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:21.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.545+0000 7f311bde9640 1 --2- 172.21.15.52:0/565143632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3114101c90 0x7f3114102110 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:21.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.545+0000 7f311bde9640 1 --2- 172.21.15.52:0/565143632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3114100aa0 0x7f3114100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:21.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.545+0000 7f311bde9640 1 -- 172.21.15.52:0/565143632 >> 172.21.15.52:0/565143632 conn(0x7f31140fc030 msgr2=0x7f31140fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:21.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.545+0000 7f311bde9640 1 -- 172.21.15.52:0/565143632 shutdown_connections 2024-04-02T13:50:21.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.545+0000 7f311bde9640 1 -- 172.21.15.52:0/565143632 wait complete. 2024-04-02T13:50:21.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.546+0000 7f311bde9640 1 Processor -- start 2024-04-02T13:50:21.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.546+0000 7f311bde9640 1 -- start start 2024-04-02T13:50:21.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.546+0000 7f311bde9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3114100aa0 0x7f3114104a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:21.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.546+0000 7f311bde9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3114101c90 0x7f3114103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:21.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.546+0000 7f311bde9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3114102650 0x7f31141035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:21.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.546+0000 7f311bde9640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f311410c690 con 0x7f3114100aa0 2024-04-02T13:50:21.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.546+0000 7f311bde9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f311410c510 con 0x7f3114102650 2024-04-02T13:50:21.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.546+0000 7f311bde9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f311410c810 con 0x7f3114101c90 2024-04-02T13:50:21.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.546+0000 7f3119b85640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3114100aa0 0x7f3114104a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:21.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.546+0000 7f3119b85640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3114100aa0 0x7f3114104a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58172/0 (socket says 172.21.15.52:58172) 2024-04-02T13:50:21.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.546+0000 7f3119b85640 1 -- 172.21.15.52:0/833708920 learned_addr learned my addr 172.21.15.52:0/833708920 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:21.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.546+0000 7f3119384640 1 --2- 172.21.15.52:0/833708920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3114101c90 0x7f3114103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:21.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.546+0000 7f3119b85640 1 -- 172.21.15.52:0/833708920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3114101c90 msgr2=0x7f3114103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:21.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.546+0000 7f311a386640 1 --2- 172.21.15.52:0/833708920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3114102650 0x7f31141035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:21.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.546+0000 7f3119b85640 1 --2- 172.21.15.52:0/833708920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3114101c90 0x7f3114103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:21.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.546+0000 7f3119b85640 1 -- 172.21.15.52:0/833708920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3114102650 msgr2=0x7f31141035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:21.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.546+0000 7f3119b85640 1 --2- 172.21.15.52:0/833708920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3114102650 0x7f31141035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:21.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.546+0000 7f3119b85640 1 -- 172.21.15.52:0/833708920 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3114103e40 con 0x7f3114100aa0 2024-04-02T13:50:21.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.546+0000 7f3119b85640 1 --2- 172.21.15.52:0/833708920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3114100aa0 0x7f3114104a30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f311000e9c0 tx=0x7f311000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:21.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.546+0000 7f310affd640 1 -- 172.21.15.52:0/833708920 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311000cd80 con 0x7f3114100aa0 2024-04-02T13:50:21.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.546+0000 7f310affd640 1 -- 172.21.15.52:0/833708920 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31100035f0 con 0x7f3114100aa0 2024-04-02T13:50:21.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.546+0000 7f311bde9640 1 -- 172.21.15.52:0/833708920 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31141a1f50 con 0x7f3114100aa0 2024-04-02T13:50:21.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.547+0000 7f310affd640 1 -- 172.21.15.52:0/833708920 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3110010690 con 0x7f3114100aa0 2024-04-02T13:50:21.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.547+0000 7f311bde9640 1 -- 172.21.15.52:0/833708920 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31141a2350 con 0x7f3114100aa0 2024-04-02T13:50:21.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.547+0000 7f311bde9640 1 -- 172.21.15.52:0/833708920 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31141a2690 con 0x7f3114100aa0 2024-04-02T13:50:21.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.549+0000 7f310affd640 1 -- 172.21.15.52:0/833708920 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f31100037a0 con 0x7f3114100aa0 2024-04-02T13:50:21.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.549+0000 7f310affd640 1 --2- 172.21.15.52:0/833708920 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f31000a3bb0 0x7f31000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:21.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.549+0000 7f3119384640 1 --2- 172.21.15.52:0/833708920 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f31000a3bb0 0x7f31000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:21.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.549+0000 7f310affd640 1 -- 172.21.15.52:0/833708920 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f31100d0430 con 0x7f3114100aa0 2024-04-02T13:50:21.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.550+0000 7f3119384640 1 --2- 172.21.15.52:0/833708920 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f31000a3bb0 0x7f31000a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f310402c7b0 tx=0x7f3104005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:21.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.553+0000 7f310affd640 1 -- 172.21.15.52:0/833708920 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f311009d2b0 con 0x7f3114100aa0 2024-04-02T13:50:21.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.784+0000 7f311bde9640 1 -- 172.21.15.52:0/833708920 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31141a2860 con 0x7f3114100aa0 2024-04-02T13:50:21.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.784+0000 7f310affd640 1 -- 172.21.15.52:0/833708920 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f311009cbb0 con 0x7f3114100aa0 2024-04-02T13:50:21.788 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:50:21.788 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-02T13:50:21.788 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 1 2024-04-02T13:50:21.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:21 smithi189 ceph-mon[36683]: Deploying daemon mds.l on smithi189 2024-04-02T13:50:21.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:21 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3135125554' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:50:21.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:21 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1237989816' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:50:21.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.786+0000 7f311bde9640 1 -- 172.21.15.52:0/833708920 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f31000a3bb0 msgr2=0x7f31000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:21.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.786+0000 7f311bde9640 1 --2- 172.21.15.52:0/833708920 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f31000a3bb0 0x7f31000a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f310402c7b0 tx=0x7f3104005fd0 comp rx=0 tx=0).stop 2024-04-02T13:50:21.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.786+0000 7f311bde9640 1 -- 172.21.15.52:0/833708920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3114100aa0 msgr2=0x7f3114104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:21.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.786+0000 7f311bde9640 1 --2- 172.21.15.52:0/833708920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3114100aa0 0x7f3114104a30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f311000e9c0 tx=0x7f311000ee80 comp rx=0 tx=0).stop 2024-04-02T13:50:21.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.787+0000 7f311bde9640 1 -- 172.21.15.52:0/833708920 shutdown_connections 2024-04-02T13:50:21.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.787+0000 7f311bde9640 1 --2- 172.21.15.52:0/833708920 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f31000a3bb0 0x7f31000a6070 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:21.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.787+0000 7f311bde9640 1 --2- 172.21.15.52:0/833708920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3114102650 0x7f31141035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:21.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.787+0000 7f311bde9640 1 --2- 172.21.15.52:0/833708920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3114101c90 0x7f3114103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:21.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.787+0000 7f311bde9640 1 --2- 172.21.15.52:0/833708920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3114100aa0 0x7f3114104a30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:21.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.787+0000 7f311bde9640 1 -- 172.21.15.52:0/833708920 >> 172.21.15.52:0/833708920 conn(0x7f31140fc030 msgr2=0x7f31140fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:21.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.787+0000 7f311bde9640 1 -- 172.21.15.52:0/833708920 shutdown_connections 2024-04-02T13:50:21.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:21.787+0000 7f311bde9640 1 -- 172.21.15.52:0/833708920 wait complete. 2024-04-02T13:50:21.799 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:21 smithi052 ceph-mon[32104]: Deploying daemon mds.l on smithi189 2024-04-02T13:50:21.799 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:21 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3135125554' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:50:21.799 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:21 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1237989816' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:50:21.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:21 smithi088 ceph-mon[36528]: Deploying daemon mds.l on smithi189 2024-04-02T13:50:21.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:21 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3135125554' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:50:21.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:21 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1237989816' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:50:22.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:22 smithi189 ceph-mon[36683]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:22.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:22 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/833708920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:22.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:22 smithi088 ceph-mon[36528]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:22.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:22 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/833708920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:23.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:22 smithi052 ceph-mon[32104]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:23.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:22 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/833708920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:23.288 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 02 13:50:23 smithi189 podman[66545]: 2024-04-02 13:50:23.132651999 +0000 UTC m=+0.132904076 container create 82ee271879cb8553682581db677f3947819ec73532c8ba08bc5d49896b5831c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-l, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-02T13:50:23.788 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 02 13:50:23 smithi189 podman[66545]: 2024-04-02 13:50:23.451888711 +0000 UTC m=+0.452140752 container init 82ee271879cb8553682581db677f3947819ec73532c8ba08bc5d49896b5831c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-l, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, ceph=True) 2024-04-02T13:50:23.788 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 02 13:50:23 smithi189 podman[66545]: 2024-04-02 13:50:23.45447424 +0000 UTC m=+0.454726267 container start 82ee271879cb8553682581db677f3947819ec73532c8ba08bc5d49896b5831c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-l, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-02T13:50:23.788 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 02 13:50:23 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-l[66560]: starting mds.l at 2024-04-02T13:50:23.788 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 02 13:50:23 smithi189 bash[66545]: 82ee271879cb8553682581db677f3947819ec73532c8ba08bc5d49896b5831c0 2024-04-02T13:50:23.788 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 02 13:50:23 smithi189 systemd[1]: Started Ceph mds.l for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4. 2024-04-02T13:50:23.801 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:50:23.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.913+0000 7fe110e60640 1 -- 172.21.15.52:0/1224599100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe10c100a40 msgr2=0x7fe10c100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:23.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.913+0000 7fe110e60640 1 --2- 172.21.15.52:0/1224599100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe10c100a40 0x7fe10c100e60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009900 tx=0x7fe0fc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:50:23.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.913+0000 7fe110e60640 1 -- 172.21.15.52:0/1224599100 shutdown_connections 2024-04-02T13:50:23.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.913+0000 7fe110e60640 1 --2- 172.21.15.52:0/1224599100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe10c1025f0 0x7fe10c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:23.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.913+0000 7fe110e60640 1 --2- 172.21.15.52:0/1224599100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe10c101c30 0x7fe10c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:23.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.913+0000 7fe110e60640 1 --2- 172.21.15.52:0/1224599100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe10c100a40 0x7fe10c100e60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:23.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.913+0000 7fe110e60640 1 -- 172.21.15.52:0/1224599100 >> 172.21.15.52:0/1224599100 conn(0x7fe10c0fbfb0 msgr2=0x7fe10c0fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.913+0000 7fe110e60640 1 -- 172.21.15.52:0/1224599100 shutdown_connections 2024-04-02T13:50:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.913+0000 7fe110e60640 1 -- 172.21.15.52:0/1224599100 wait complete. 2024-04-02T13:50:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.914+0000 7fe110e60640 1 Processor -- start 2024-04-02T13:50:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.914+0000 7fe110e60640 1 -- start start 2024-04-02T13:50:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.914+0000 7fe110e60640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe10c100a40 0x7fe10c197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.914+0000 7fe110e60640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe10c101c30 0x7fe10c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.914+0000 7fe110e60640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe10c1025f0 0x7fe10c19f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.914+0000 7fe110e60640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c10d7a0 con 0x7fe10c100a40 2024-04-02T13:50:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.914+0000 7fe110e60640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c10d620 con 0x7fe10c1025f0 2024-04-02T13:50:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.914+0000 7fe110e60640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c10d920 con 0x7fe10c101c30 2024-04-02T13:50:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.914+0000 7fe10a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe10c100a40 0x7fe10c197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.914+0000 7fe10a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe10c100a40 0x7fe10c197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58188/0 (socket says 172.21.15.52:58188) 2024-04-02T13:50:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.914+0000 7fe10a59c640 1 -- 172.21.15.52:0/3682504571 learned_addr learned my addr 172.21.15.52:0/3682504571 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.914+0000 7fe10ad9d640 1 --2- 172.21.15.52:0/3682504571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe10c1025f0 0x7fe10c19f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.914+0000 7fe103fff640 1 --2- 172.21.15.52:0/3682504571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe10c101c30 0x7fe10c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:23.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.915+0000 7fe10a59c640 1 -- 172.21.15.52:0/3682504571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe10c101c30 msgr2=0x7fe10c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:23.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.915+0000 7fe10a59c640 1 --2- 172.21.15.52:0/3682504571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe10c101c30 0x7fe10c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:23.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.915+0000 7fe10a59c640 1 -- 172.21.15.52:0/3682504571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe10c1025f0 msgr2=0x7fe10c19f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:23.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.915+0000 7fe10a59c640 1 --2- 172.21.15.52:0/3682504571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe10c1025f0 0x7fe10c19f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:23.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.915+0000 7fe10a59c640 1 -- 172.21.15.52:0/3682504571 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe10c19fa90 con 0x7fe10c100a40 2024-04-02T13:50:23.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.915+0000 7fe103fff640 1 --2- 172.21.15.52:0/3682504571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe10c101c30 0x7fe10c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:50:23.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.915+0000 7fe10a59c640 1 --2- 172.21.15.52:0/3682504571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe10c100a40 0x7fe10c197e70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc000c00 tx=0x7fe0fc005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:23.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.915+0000 7fe1037fe640 1 -- 172.21.15.52:0/3682504571 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc041070 con 0x7fe10c100a40 2024-04-02T13:50:23.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.915+0000 7fe1037fe640 1 -- 172.21.15.52:0/3682504571 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0fc0044d0 con 0x7fe10c100a40 2024-04-02T13:50:23.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.915+0000 7fe110e60640 1 -- 172.21.15.52:0/3682504571 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe10c19fd20 con 0x7fe10c100a40 2024-04-02T13:50:23.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.915+0000 7fe1037fe640 1 -- 172.21.15.52:0/3682504571 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc034440 con 0x7fe10c100a40 2024-04-02T13:50:23.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.915+0000 7fe110e60640 1 -- 172.21.15.52:0/3682504571 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe10c1a01c0 con 0x7fe10c100a40 2024-04-02T13:50:23.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.916+0000 7fe110e60640 1 -- 172.21.15.52:0/3682504571 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe10c107d60 con 0x7fe10c100a40 2024-04-02T13:50:23.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.917+0000 7fe1037fe640 1 -- 172.21.15.52:0/3682504571 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe0fc044070 con 0x7fe10c100a40 2024-04-02T13:50:23.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.918+0000 7fe1037fe640 1 --2- 172.21.15.52:0/3682504571 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe0f80a3bb0 0x7fe0f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:23.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.918+0000 7fe103fff640 1 --2- 172.21.15.52:0/3682504571 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe0f80a3bb0 0x7fe0f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:23.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.918+0000 7fe1037fe640 1 -- 172.21.15.52:0/3682504571 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe0fc0f1af0 con 0x7fe10c100a40 2024-04-02T13:50:23.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.918+0000 7fe103fff640 1 --2- 172.21.15.52:0/3682504571 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe0f80a3bb0 0x7fe0f80a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe10c199380 tx=0x7fe0f4005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:23.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:23.922+0000 7fe1037fe640 1 -- 172.21.15.52:0/3682504571 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe0fc0be970 con 0x7fe10c100a40 2024-04-02T13:50:24.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:24.146+0000 7fe110e60640 1 -- 172.21.15.52:0/3682504571 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe10c02d5e0 con 0x7fe10c100a40 2024-04-02T13:50:24.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:24.147+0000 7fe1037fe640 1 -- 172.21.15.52:0/3682504571 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fe0fc0be270 con 0x7fe10c100a40 2024-04-02T13:50:24.150 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:50:24.150 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-02T13:50:24.150 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 2 2024-04-02T13:50:24.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:24.149+0000 7fe110e60640 1 -- 172.21.15.52:0/3682504571 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe0f80a3bb0 msgr2=0x7fe0f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:24.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:24.149+0000 7fe110e60640 1 --2- 172.21.15.52:0/3682504571 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe0f80a3bb0 0x7fe0f80a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe10c199380 tx=0x7fe0f4005ec0 comp rx=0 tx=0).stop 2024-04-02T13:50:24.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:24.149+0000 7fe110e60640 1 -- 172.21.15.52:0/3682504571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe10c100a40 msgr2=0x7fe10c197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:24.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:24.149+0000 7fe110e60640 1 --2- 172.21.15.52:0/3682504571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe10c100a40 0x7fe10c197e70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc000c00 tx=0x7fe0fc005d50 comp rx=0 tx=0).stop 2024-04-02T13:50:24.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:24.149+0000 7fe110e60640 1 -- 172.21.15.52:0/3682504571 shutdown_connections 2024-04-02T13:50:24.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:24.149+0000 7fe110e60640 1 --2- 172.21.15.52:0/3682504571 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe0f80a3bb0 0x7fe0f80a6070 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:24.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:24.149+0000 7fe110e60640 1 --2- 172.21.15.52:0/3682504571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe10c1025f0 0x7fe10c19f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:24.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:24.149+0000 7fe110e60640 1 --2- 172.21.15.52:0/3682504571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe10c101c30 0x7fe10c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:24.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:24.149+0000 7fe110e60640 1 --2- 172.21.15.52:0/3682504571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe10c100a40 0x7fe10c197e70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:24.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:24.149+0000 7fe110e60640 1 -- 172.21.15.52:0/3682504571 >> 172.21.15.52:0/3682504571 conn(0x7fe10c0fbfb0 msgr2=0x7fe10c104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:24.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:24.149+0000 7fe110e60640 1 -- 172.21.15.52:0/3682504571 shutdown_connections 2024-04-02T13:50:24.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:24.149+0000 7fe110e60640 1 -- 172.21.15.52:0/3682504571 wait complete. 2024-04-02T13:50:24.619 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:24 smithi088 ceph-mon[36528]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:24.619 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:24 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.189:6832/3157031038,v1:172.21.15.189:6833/3157031038] up:boot 2024-04-02T13:50:24.619 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:24 smithi088 ceph-mon[36528]: fsmap 1 up:standby 2024-04-02T13:50:24.619 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:24 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T13:50:24.619 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:24 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:24.619 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:24 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:24.619 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:24 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:24.619 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:24 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:24.619 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:24 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:24.619 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:24 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:24.619 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:24 smithi088 ceph-mon[36528]: Deploying daemon mds.k on smithi088 2024-04-02T13:50:24.620 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:24 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3682504571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:24.776 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:24 smithi052 ceph-mon[32104]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:24.776 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:24 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.189:6832/3157031038,v1:172.21.15.189:6833/3157031038] up:boot 2024-04-02T13:50:24.776 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:24 smithi052 ceph-mon[32104]: fsmap 1 up:standby 2024-04-02T13:50:24.776 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:24 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T13:50:24.776 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:24 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:24.776 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:24 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:24.776 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:24 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:24.776 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:24 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:24.777 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:24 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:24.777 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:24 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:24.777 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:24 smithi052 ceph-mon[32104]: Deploying daemon mds.k on smithi088 2024-04-02T13:50:24.777 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:24 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3682504571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:24.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:24 smithi189 ceph-mon[36683]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:24.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:24 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.189:6832/3157031038,v1:172.21.15.189:6833/3157031038] up:boot 2024-04-02T13:50:24.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:24 smithi189 ceph-mon[36683]: fsmap 1 up:standby 2024-04-02T13:50:24.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:24 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T13:50:24.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:24 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:24.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:24 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:24.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:24 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:24.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:24 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:24.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:24 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:24.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:24 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:24.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:24 smithi189 ceph-mon[36683]: Deploying daemon mds.k on smithi088 2024-04-02T13:50:24.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:24 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3682504571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:26.120 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 02 13:50:25 smithi088 systemd[1]: Starting Ceph mds.k for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T13:50:26.164 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:50:26.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.269+0000 7fb9b6222640 1 -- 172.21.15.52:0/668222243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9b01025d0 msgr2=0x7fb9b0108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:26.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.269+0000 7fb9b6222640 1 --2- 172.21.15.52:0/668222243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9b01025d0 0x7fb9b0108e60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009ca0 tx=0x7fb9a802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:50:26.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.269+0000 7fb9b6222640 1 -- 172.21.15.52:0/668222243 shutdown_connections 2024-04-02T13:50:26.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.269+0000 7fb9b6222640 1 --2- 172.21.15.52:0/668222243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9b01025d0 0x7fb9b0108e60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:26.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.270+0000 7fb9b6222640 1 --2- 172.21.15.52:0/668222243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9b0101c10 0x7fb9b0102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:26.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.270+0000 7fb9b6222640 1 --2- 172.21.15.52:0/668222243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9b0100a20 0x7fb9b0100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:26.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.270+0000 7fb9b6222640 1 -- 172.21.15.52:0/668222243 >> 172.21.15.52:0/668222243 conn(0x7fb9b00fbf90 msgr2=0x7fb9b00fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:26.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.270+0000 7fb9b6222640 1 -- 172.21.15.52:0/668222243 shutdown_connections 2024-04-02T13:50:26.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.270+0000 7fb9b6222640 1 -- 172.21.15.52:0/668222243 wait complete. 2024-04-02T13:50:26.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.270+0000 7fb9b6222640 1 Processor -- start 2024-04-02T13:50:26.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.270+0000 7fb9b6222640 1 -- start start 2024-04-02T13:50:26.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.270+0000 7fb9b6222640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9b0100a20 0x7fb9b01049b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:26.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.270+0000 7fb9b6222640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9b0101c10 0x7fb9b0102ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:26.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.270+0000 7fb9b6222640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9b01025d0 0x7fb9b0103530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:26.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.270+0000 7fb9b6222640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b010c6a0 con 0x7fb9b0101c10 2024-04-02T13:50:26.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.270+0000 7fb9b6222640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b010c520 con 0x7fb9b0100a20 2024-04-02T13:50:26.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.270+0000 7fb9b6222640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b010c820 con 0x7fb9b01025d0 2024-04-02T13:50:26.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.271+0000 7fb9aeffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9b0101c10 0x7fb9b0102ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:26.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.271+0000 7fb9aeffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9b0101c10 0x7fb9b0102ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58204/0 (socket says 172.21.15.52:58204) 2024-04-02T13:50:26.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.271+0000 7fb9aeffd640 1 -- 172.21.15.52:0/4024016840 learned_addr learned my addr 172.21.15.52:0/4024016840 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:26.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.271+0000 7fb9af7fe640 1 --2- 172.21.15.52:0/4024016840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9b0100a20 0x7fb9b01049b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.271+0000 7fb9affff640 1 --2- 172.21.15.52:0/4024016840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9b01025d0 0x7fb9b0103530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.271+0000 7fb9aeffd640 1 -- 172.21.15.52:0/4024016840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9b01025d0 msgr2=0x7fb9b0103530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.271+0000 7fb9aeffd640 1 --2- 172.21.15.52:0/4024016840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9b01025d0 0x7fb9b0103530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.271+0000 7fb9aeffd640 1 -- 172.21.15.52:0/4024016840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9b0100a20 msgr2=0x7fb9b01049b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.271+0000 7fb9aeffd640 1 --2- 172.21.15.52:0/4024016840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9b0100a20 0x7fb9b01049b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.271+0000 7fb9aeffd640 1 -- 172.21.15.52:0/4024016840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b0103dc0 con 0x7fb9b0101c10 2024-04-02T13:50:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.271+0000 7fb9af7fe640 1 --2- 172.21.15.52:0/4024016840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9b0100a20 0x7fb9b01049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:50:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.271+0000 7fb9affff640 1 --2- 172.21.15.52:0/4024016840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9b01025d0 0x7fb9b0103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:50:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.271+0000 7fb9aeffd640 1 --2- 172.21.15.52:0/4024016840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9b0101c10 0x7fb9b0102ff0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400c970 tx=0x7fb9a400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.271+0000 7fb9acff9640 1 -- 172.21.15.52:0/4024016840 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4007bb0 con 0x7fb9b0101c10 2024-04-02T13:50:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.271+0000 7fb9acff9640 1 -- 172.21.15.52:0/4024016840 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a4007d50 con 0x7fb9b0101c10 2024-04-02T13:50:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.271+0000 7fb9b6222640 1 -- 172.21.15.52:0/4024016840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b01a1f00 con 0x7fb9b0101c10 2024-04-02T13:50:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.271+0000 7fb9acff9640 1 -- 172.21.15.52:0/4024016840 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a400f450 con 0x7fb9b0101c10 2024-04-02T13:50:26.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.271+0000 7fb9b6222640 1 -- 172.21.15.52:0/4024016840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b01a2310 con 0x7fb9b0101c10 2024-04-02T13:50:26.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.272+0000 7fb9b6222640 1 -- 172.21.15.52:0/4024016840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9b01a2600 con 0x7fb9b0101c10 2024-04-02T13:50:26.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.273+0000 7fb9acff9640 1 -- 172.21.15.52:0/4024016840 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb9a4005020 con 0x7fb9b0101c10 2024-04-02T13:50:26.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.274+0000 7fb9acff9640 1 --2- 172.21.15.52:0/4024016840 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb9980a3bb0 0x7fb9980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:26.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.274+0000 7fb9af7fe640 1 --2- 172.21.15.52:0/4024016840 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb9980a3bb0 0x7fb9980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:26.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.274+0000 7fb9acff9640 1 -- 172.21.15.52:0/4024016840 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb9a40cfd40 con 0x7fb9b0101c10 2024-04-02T13:50:26.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.275+0000 7fb9af7fe640 1 --2- 172.21.15.52:0/4024016840 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb9980a3bb0 0x7fb9980a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0101a70 tx=0x7fb9a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:26.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.279+0000 7fb9acff9640 1 -- 172.21.15.52:0/4024016840 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb9a4014030 con 0x7fb9b0101c10 2024-04-02T13:50:26.374 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 02 13:50:26 smithi088 podman[65702]: 2024-04-02 13:50:26.021134517 +0000 UTC m=+0.017651828 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:50:26.374 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 02 13:50:26 smithi088 podman[65702]: 2024-04-02 13:50:26.140181673 +0000 UTC m=+0.136699008 container create 0f8f26af25837ecb1c9d02660d2e7da65c239b5d208c8d454b993b823e1841af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-k, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-02T13:50:26.374 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 02 13:50:26 smithi088 podman[65702]: 2024-04-02 13:50:26.367800031 +0000 UTC m=+0.364317334 container init 0f8f26af25837ecb1c9d02660d2e7da65c239b5d208c8d454b993b823e1841af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-k, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:50:26.374 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 02 13:50:26 smithi088 podman[65702]: 2024-04-02 13:50:26.370294763 +0000 UTC m=+0.366812048 container start 0f8f26af25837ecb1c9d02660d2e7da65c239b5d208c8d454b993b823e1841af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-k, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-02T13:50:26.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.511+0000 7fb9b6222640 1 -- 172.21.15.52:0/4024016840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9b01a29a0 con 0x7fb9b0101c10 2024-04-02T13:50:26.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.512+0000 7fb9acff9640 1 -- 172.21.15.52:0/4024016840 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fb9a409cbc0 con 0x7fb9b0101c10 2024-04-02T13:50:26.515 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:50:26.515 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-02T13:50:26.515 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 3 2024-04-02T13:50:26.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.513+0000 7fb9b6222640 1 -- 172.21.15.52:0/4024016840 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb9980a3bb0 msgr2=0x7fb9980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:26.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.513+0000 7fb9b6222640 1 --2- 172.21.15.52:0/4024016840 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb9980a3bb0 0x7fb9980a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0101a70 tx=0x7fb9a0005f90 comp rx=0 tx=0).stop 2024-04-02T13:50:26.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.513+0000 7fb9b6222640 1 -- 172.21.15.52:0/4024016840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9b0101c10 msgr2=0x7fb9b0102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:26.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.513+0000 7fb9b6222640 1 --2- 172.21.15.52:0/4024016840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9b0101c10 0x7fb9b0102ff0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400c970 tx=0x7fb9a400ce30 comp rx=0 tx=0).stop 2024-04-02T13:50:26.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.514+0000 7fb9b6222640 1 -- 172.21.15.52:0/4024016840 shutdown_connections 2024-04-02T13:50:26.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.514+0000 7fb9b6222640 1 --2- 172.21.15.52:0/4024016840 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb9980a3bb0 0x7fb9980a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:26.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.514+0000 7fb9b6222640 1 --2- 172.21.15.52:0/4024016840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9b01025d0 0x7fb9b0103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:26.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.514+0000 7fb9b6222640 1 --2- 172.21.15.52:0/4024016840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9b0101c10 0x7fb9b0102ff0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:26.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.514+0000 7fb9b6222640 1 --2- 172.21.15.52:0/4024016840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9b0100a20 0x7fb9b01049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:26.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.514+0000 7fb9b6222640 1 -- 172.21.15.52:0/4024016840 >> 172.21.15.52:0/4024016840 conn(0x7fb9b00fbf90 msgr2=0x7fb9b00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:26.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.514+0000 7fb9b6222640 1 -- 172.21.15.52:0/4024016840 shutdown_connections 2024-04-02T13:50:26.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:26.514+0000 7fb9b6222640 1 -- 172.21.15.52:0/4024016840 wait complete. 2024-04-02T13:50:26.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:26 smithi088 ceph-mon[36528]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:26.670 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 02 13:50:26 smithi088 ceph-mds[65721]: -- 172.21.15.88:0/2683192382 <== mon.2 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bb2e3fda40 con 0x55bb2e3d0c00 2024-04-02T13:50:26.670 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 02 13:50:26 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-k[65717]: starting mds.k at 2024-04-02T13:50:26.670 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 02 13:50:26 smithi088 bash[65702]: 0f8f26af25837ecb1c9d02660d2e7da65c239b5d208c8d454b993b823e1841af 2024-04-02T13:50:26.670 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 02 13:50:26 smithi088 systemd[1]: Started Ceph mds.k for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4. 2024-04-02T13:50:26.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:26 smithi189 ceph-mon[36683]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:26.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:26 smithi052 ceph-mon[32104]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:27.783 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:27 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:27.783 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:27 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.88:6832/4248342942,v1:172.21.15.88:6833/4248342942] up:boot 2024-04-02T13:50:27.783 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:27 smithi088 ceph-mon[36528]: fsmap 2 up:standby 2024-04-02T13:50:27.783 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:27 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T13:50:27.783 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:27 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:27.783 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:27 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/4024016840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:27.783 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:27 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:27.783 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:27 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:27.783 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:27 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:27.783 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:27 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:27.784 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:27 smithi088 ceph-mon[36528]: Deploying daemon mds.j on smithi052 2024-04-02T13:50:27.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:27 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:27.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:27 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.88:6832/4248342942,v1:172.21.15.88:6833/4248342942] up:boot 2024-04-02T13:50:27.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:27 smithi189 ceph-mon[36683]: fsmap 2 up:standby 2024-04-02T13:50:27.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:27 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T13:50:27.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:27 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:27.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:27 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/4024016840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:27.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:27 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:27.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:27 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:27.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:27 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:27.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:27 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:27.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:27 smithi189 ceph-mon[36683]: Deploying daemon mds.j on smithi052 2024-04-02T13:50:27.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:27.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:27 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.88:6832/4248342942,v1:172.21.15.88:6833/4248342942] up:boot 2024-04-02T13:50:27.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:27 smithi052 ceph-mon[32104]: fsmap 2 up:standby 2024-04-02T13:50:27.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T13:50:27.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:27.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:27 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/4024016840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:27.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:27.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:27.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:27.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:27 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:27.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:27 smithi052 ceph-mon[32104]: Deploying daemon mds.j on smithi052 2024-04-02T13:50:28.527 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:50:28.700 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:28 smithi052 ceph-mon[32104]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:28.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.782+0000 7fbaf77fe640 1 -- 172.21.15.52:0/3591161362 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec02cc80 con 0x7fbaf8102600 2024-04-02T13:50:28.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.783+0000 7fbafebc8640 1 -- 172.21.15.52:0/3591161362 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaf8102600 msgr2=0x7fbaf81049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:28.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.783+0000 7fbafebc8640 1 --2- 172.21.15.52:0/3591161362 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaf8102600 0x7fbaf81049f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbaec009900 tx=0x7fbaec02bc30 comp rx=0 tx=0).stop 2024-04-02T13:50:28.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.783+0000 7fbafebc8640 1 -- 172.21.15.52:0/3591161362 shutdown_connections 2024-04-02T13:50:28.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.783+0000 7fbafebc8640 1 --2- 172.21.15.52:0/3591161362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbaf8107860 0x7fbaf8109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:28.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.783+0000 7fbafebc8640 1 --2- 172.21.15.52:0/3591161362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbaf8104f30 0x7fbaf8107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:28.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.783+0000 7fbafebc8640 1 --2- 172.21.15.52:0/3591161362 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaf8102600 0x7fbaf81049f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:28.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.783+0000 7fbafebc8640 1 -- 172.21.15.52:0/3591161362 >> 172.21.15.52:0/3591161362 conn(0x7fbaf80fbf90 msgr2=0x7fbaf80fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:28.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.783+0000 7fbafebc8640 1 -- 172.21.15.52:0/3591161362 shutdown_connections 2024-04-02T13:50:28.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.783+0000 7fbafebc8640 1 -- 172.21.15.52:0/3591161362 wait complete. 2024-04-02T13:50:28.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.784+0000 7fbafebc8640 1 Processor -- start 2024-04-02T13:50:28.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.784+0000 7fbafebc8640 1 -- start start 2024-04-02T13:50:28.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.784+0000 7fbafebc8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbaf8102600 0x7fbaf819a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:28.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.784+0000 7fbafebc8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaf8104f30 0x7fbaf819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:28.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.784+0000 7fbafebc8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbaf8107860 0x7fbaf81a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:28.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.784+0000 7fbafebc8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf8101c20 con 0x7fbaf8104f30 2024-04-02T13:50:28.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.784+0000 7fbafebc8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf8101aa0 con 0x7fbaf8102600 2024-04-02T13:50:28.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.784+0000 7fbafebc8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf8101da0 con 0x7fbaf8107860 2024-04-02T13:50:28.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.784+0000 7fbaf7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaf8104f30 0x7fbaf819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:28.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.784+0000 7fbaf7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaf8104f30 0x7fbaf819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58226/0 (socket says 172.21.15.52:58226) 2024-04-02T13:50:28.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.784+0000 7fbaf7fff640 1 -- 172.21.15.52:0/880231750 learned_addr learned my addr 172.21.15.52:0/880231750 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:28.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.784+0000 7fbafc964640 1 --2- 172.21.15.52:0/880231750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbaf8102600 0x7fbaf819a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:28.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.784+0000 7fbafd165640 1 --2- 172.21.15.52:0/880231750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbaf8107860 0x7fbaf81a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:28.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.784+0000 7fbaf7fff640 1 -- 172.21.15.52:0/880231750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbaf8107860 msgr2=0x7fbaf81a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:28.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.784+0000 7fbaf7fff640 1 --2- 172.21.15.52:0/880231750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbaf8107860 0x7fbaf81a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:28.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.784+0000 7fbaf7fff640 1 -- 172.21.15.52:0/880231750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbaf8102600 msgr2=0x7fbaf819a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:28.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.784+0000 7fbaf7fff640 1 --2- 172.21.15.52:0/880231750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbaf8102600 0x7fbaf819a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:28.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.784+0000 7fbaf7fff640 1 -- 172.21.15.52:0/880231750 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf81a1d00 con 0x7fbaf8104f30 2024-04-02T13:50:28.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.784+0000 7fbafd165640 1 --2- 172.21.15.52:0/880231750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbaf8107860 0x7fbaf81a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:50:28.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.784+0000 7fbaf7fff640 1 --2- 172.21.15.52:0/880231750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaf8104f30 0x7fbaf819a670 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbae400d770 tx=0x7fbae400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:28.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.785+0000 7fbaf5ffb640 1 -- 172.21.15.52:0/880231750 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae4017070 con 0x7fbaf8104f30 2024-04-02T13:50:28.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.785+0000 7fbaf5ffb640 1 -- 172.21.15.52:0/880231750 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae4009c30 con 0x7fbaf8104f30 2024-04-02T13:50:28.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.785+0000 7fbaf5ffb640 1 -- 172.21.15.52:0/880231750 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae400f620 con 0x7fbaf8104f30 2024-04-02T13:50:28.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.785+0000 7fbafebc8640 1 -- 172.21.15.52:0/880231750 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf81a1ff0 con 0x7fbaf8104f30 2024-04-02T13:50:28.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.785+0000 7fbafebc8640 1 -- 172.21.15.52:0/880231750 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf81a2500 con 0x7fbaf8104f30 2024-04-02T13:50:28.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.786+0000 7fbafebc8640 1 -- 172.21.15.52:0/880231750 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaf80fda80 con 0x7fbaf8104f30 2024-04-02T13:50:28.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.787+0000 7fbaf5ffb640 1 -- 172.21.15.52:0/880231750 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbae401a070 con 0x7fbaf8104f30 2024-04-02T13:50:28.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.787+0000 7fbaf5ffb640 1 --2- 172.21.15.52:0/880231750 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fbae80a3c30 0x7fbae80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:28.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.788+0000 7fbafc964640 1 --2- 172.21.15.52:0/880231750 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fbae80a3c30 0x7fbae80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:28.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.788+0000 7fbaf5ffb640 1 -- 172.21.15.52:0/880231750 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fbae40cf9f0 con 0x7fbaf8104f30 2024-04-02T13:50:28.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.788+0000 7fbafc964640 1 --2- 172.21.15.52:0/880231750 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fbae80a3c30 0x7fbae80a60f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbaec02c7b0 tx=0x7fbaec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:28.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:28.795+0000 7fbaf5ffb640 1 -- 172.21.15.52:0/880231750 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fbae4009de0 con 0x7fbaf8104f30 2024-04-02T13:50:28.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:28 smithi088 ceph-mon[36528]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:29.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:28 smithi189 ceph-mon[36683]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:29.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:29.034+0000 7fbafebc8640 1 -- 172.21.15.52:0/880231750 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbaf802d370 con 0x7fbaf8104f30 2024-04-02T13:50:29.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:29.035+0000 7fbaf5ffb640 1 -- 172.21.15.52:0/880231750 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fbae409c850 con 0x7fbaf8104f30 2024-04-02T13:50:29.039 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:50:29.039 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-02T13:50:29.039 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 3 2024-04-02T13:50:29.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:29.037+0000 7fbafebc8640 1 -- 172.21.15.52:0/880231750 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fbae80a3c30 msgr2=0x7fbae80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:29.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:29.037+0000 7fbafebc8640 1 --2- 172.21.15.52:0/880231750 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fbae80a3c30 0x7fbae80a60f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbaec02c7b0 tx=0x7fbaec005fd0 comp rx=0 tx=0).stop 2024-04-02T13:50:29.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:29.037+0000 7fbafebc8640 1 -- 172.21.15.52:0/880231750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaf8104f30 msgr2=0x7fbaf819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:29.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:29.037+0000 7fbafebc8640 1 --2- 172.21.15.52:0/880231750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaf8104f30 0x7fbaf819a670 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbae400d770 tx=0x7fbae400dc30 comp rx=0 tx=0).stop 2024-04-02T13:50:29.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:29.037+0000 7fbafebc8640 1 -- 172.21.15.52:0/880231750 shutdown_connections 2024-04-02T13:50:29.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:29.037+0000 7fbafebc8640 1 --2- 172.21.15.52:0/880231750 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fbae80a3c30 0x7fbae80a60f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:29.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:29.037+0000 7fbafebc8640 1 --2- 172.21.15.52:0/880231750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbaf8107860 0x7fbaf81a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:29.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:29.037+0000 7fbafebc8640 1 --2- 172.21.15.52:0/880231750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaf8104f30 0x7fbaf819a670 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:29.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:29.037+0000 7fbafebc8640 1 --2- 172.21.15.52:0/880231750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbaf8102600 0x7fbaf819a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:29.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:29.037+0000 7fbafebc8640 1 -- 172.21.15.52:0/880231750 >> 172.21.15.52:0/880231750 conn(0x7fbaf80fbf90 msgr2=0x7fbaf8102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:29.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:29.037+0000 7fbafebc8640 1 -- 172.21.15.52:0/880231750 shutdown_connections 2024-04-02T13:50:29.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:29.037+0000 7fbafebc8640 1 -- 172.21.15.52:0/880231750 wait complete. 2024-04-02T13:50:29.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:29 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/880231750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:29.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:29 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:29.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:29 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:29.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:29 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/880231750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:29.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:29 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:29.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:29 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:30.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:29 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/880231750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:30.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:29 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:30.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:29 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:30.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:30 smithi052 ceph-mon[32104]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:30.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:30 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.52:6834/1683593044,v1:172.21.15.52:6835/1683593044] up:boot 2024-04-02T13:50:30.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:30 smithi052 ceph-mon[32104]: fsmap 3 up:standby 2024-04-02T13:50:30.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:30 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T13:50:30.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:30 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:30.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:30 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:30.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:30 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:30.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:30 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:30.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:30 smithi052 ceph-mon[32104]: Deploying daemon mds.d on smithi052 2024-04-02T13:50:30.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:30 smithi088 ceph-mon[36528]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:30 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.52:6834/1683593044,v1:172.21.15.52:6835/1683593044] up:boot 2024-04-02T13:50:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:30 smithi088 ceph-mon[36528]: fsmap 3 up:standby 2024-04-02T13:50:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:30 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T13:50:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:30 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:30 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:30 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:30 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:30 smithi088 ceph-mon[36528]: Deploying daemon mds.d on smithi052 2024-04-02T13:50:31.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:30 smithi189 ceph-mon[36683]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:31.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:30 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.52:6834/1683593044,v1:172.21.15.52:6835/1683593044] up:boot 2024-04-02T13:50:31.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:30 smithi189 ceph-mon[36683]: fsmap 3 up:standby 2024-04-02T13:50:31.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:30 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T13:50:31.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:30 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:31.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:30 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:31.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:30 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:31.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:30 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:31.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:30 smithi189 ceph-mon[36683]: Deploying daemon mds.d on smithi052 2024-04-02T13:50:31.053 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:50:31.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.164+0000 7f786ca9a640 1 -- 172.21.15.52:0/3967949609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7868101c10 msgr2=0x7f7868102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:31.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.164+0000 7f786ca9a640 1 --2- 172.21.15.52:0/3967949609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7868101c10 0x7f7868102090 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7850009900 tx=0x7f785002bc30 comp rx=0 tx=0).stop 2024-04-02T13:50:31.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.164+0000 7f786ca9a640 1 -- 172.21.15.52:0/3967949609 shutdown_connections 2024-04-02T13:50:31.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.164+0000 7f786ca9a640 1 --2- 172.21.15.52:0/3967949609 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78681025d0 0x7f7868108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:31.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.164+0000 7f786ca9a640 1 --2- 172.21.15.52:0/3967949609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7868101c10 0x7f7868102090 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:31.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.164+0000 7f786ca9a640 1 --2- 172.21.15.52:0/3967949609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7868100a20 0x7f7868100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:31.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.164+0000 7f786ca9a640 1 -- 172.21.15.52:0/3967949609 >> 172.21.15.52:0/3967949609 conn(0x7f78680fbf90 msgr2=0x7f78680fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:31.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.164+0000 7f786ca9a640 1 -- 172.21.15.52:0/3967949609 shutdown_connections 2024-04-02T13:50:31.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.164+0000 7f786ca9a640 1 -- 172.21.15.52:0/3967949609 wait complete. 2024-04-02T13:50:31.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.165+0000 7f786ca9a640 1 Processor -- start 2024-04-02T13:50:31.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.165+0000 7f786ca9a640 1 -- start start 2024-04-02T13:50:31.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.165+0000 7f786ca9a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7868100a20 0x7f78680697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:31.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.165+0000 7f786ca9a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7868101c10 0x7f786819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:31.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.165+0000 7f786ca9a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78681025d0 0x7f786819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:31.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.165+0000 7f786ca9a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f786810c6a0 con 0x7f78681025d0 2024-04-02T13:50:31.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.165+0000 7f786ca9a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f786810c520 con 0x7f7868101c10 2024-04-02T13:50:31.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.165+0000 7f786ca9a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f786810c820 con 0x7f7868100a20 2024-04-02T13:50:31.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.165+0000 7f7866d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78681025d0 0x7f786819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:31.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.165+0000 7f7866d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78681025d0 0x7f786819f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58268/0 (socket says 172.21.15.52:58268) 2024-04-02T13:50:31.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.165+0000 7f7866d9d640 1 -- 172.21.15.52:0/919168719 learned_addr learned my addr 172.21.15.52:0/919168719 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:31.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.165+0000 7f786659c640 1 --2- 172.21.15.52:0/919168719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7868100a20 0x7f78680697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:31.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.165+0000 7f785ffff640 1 --2- 172.21.15.52:0/919168719 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7868101c10 0x7f786819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:31.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.165+0000 7f7866d9d640 1 -- 172.21.15.52:0/919168719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7868100a20 msgr2=0x7f78680697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:31.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.165+0000 7f7866d9d640 1 --2- 172.21.15.52:0/919168719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7868100a20 0x7f78680697d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:31.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.165+0000 7f7866d9d640 1 -- 172.21.15.52:0/919168719 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7868101c10 msgr2=0x7f786819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:31.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.165+0000 7f7866d9d640 1 --2- 172.21.15.52:0/919168719 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7868101c10 0x7f786819cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:31.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.165+0000 7f7866d9d640 1 -- 172.21.15.52:0/919168719 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f786819fac0 con 0x7f78681025d0 2024-04-02T13:50:31.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.165+0000 7f785ffff640 1 --2- 172.21.15.52:0/919168719 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7868101c10 0x7f786819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:50:31.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.165+0000 7f7866d9d640 1 --2- 172.21.15.52:0/919168719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78681025d0 0x7f786819f1b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f786000eb10 tx=0x7f786000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:31.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.166+0000 7f785f7fe640 1 -- 172.21.15.52:0/919168719 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7860004b20 con 0x7f78681025d0 2024-04-02T13:50:31.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.166+0000 7f786ca9a640 1 -- 172.21.15.52:0/919168719 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f786819fd50 con 0x7f78681025d0 2024-04-02T13:50:31.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.166+0000 7f785f7fe640 1 -- 172.21.15.52:0/919168719 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7860004cc0 con 0x7f78681025d0 2024-04-02T13:50:31.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.166+0000 7f785f7fe640 1 -- 172.21.15.52:0/919168719 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7860010760 con 0x7f78681025d0 2024-04-02T13:50:31.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.166+0000 7f786ca9a640 1 -- 172.21.15.52:0/919168719 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78681a01c0 con 0x7f78681025d0 2024-04-02T13:50:31.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.167+0000 7f786ca9a640 1 -- 172.21.15.52:0/919168719 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7868107dc0 con 0x7f78681025d0 2024-04-02T13:50:31.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.168+0000 7f785f7fe640 1 -- 172.21.15.52:0/919168719 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7860005020 con 0x7f78681025d0 2024-04-02T13:50:31.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.168+0000 7f785f7fe640 1 --2- 172.21.15.52:0/919168719 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f78540a3bb0 0x7f78540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:31.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.168+0000 7f786659c640 1 --2- 172.21.15.52:0/919168719 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f78540a3bb0 0x7f78540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:31.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.168+0000 7f785f7fe640 1 -- 172.21.15.52:0/919168719 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f78600d0580 con 0x7f78681025d0 2024-04-02T13:50:31.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.169+0000 7f786659c640 1 --2- 172.21.15.52:0/919168719 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f78540a3bb0 0x7f78540a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7858005fd0 tx=0x7f785800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:31.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.173+0000 7f785f7fe640 1 -- 172.21.15.52:0/919168719 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f786009d400 con 0x7f78681025d0 2024-04-02T13:50:31.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.400+0000 7f786ca9a640 1 -- 172.21.15.52:0/919168719 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f786802d540 con 0x7f78681025d0 2024-04-02T13:50:31.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.400+0000 7f785f7fe640 1 -- 172.21.15.52:0/919168719 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f786009cd00 con 0x7f78681025d0 2024-04-02T13:50:31.404 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:50:31.404 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-02T13:50:31.404 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 4 2024-04-02T13:50:31.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.402+0000 7f786ca9a640 1 -- 172.21.15.52:0/919168719 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f78540a3bb0 msgr2=0x7f78540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:31.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.402+0000 7f786ca9a640 1 --2- 172.21.15.52:0/919168719 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f78540a3bb0 0x7f78540a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7858005fd0 tx=0x7f785800b390 comp rx=0 tx=0).stop 2024-04-02T13:50:31.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.402+0000 7f786ca9a640 1 -- 172.21.15.52:0/919168719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78681025d0 msgr2=0x7f786819f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:31.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.402+0000 7f786ca9a640 1 --2- 172.21.15.52:0/919168719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78681025d0 0x7f786819f1b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f786000eb10 tx=0x7f786000c2d0 comp rx=0 tx=0).stop 2024-04-02T13:50:31.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.402+0000 7f786ca9a640 1 -- 172.21.15.52:0/919168719 shutdown_connections 2024-04-02T13:50:31.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.402+0000 7f786ca9a640 1 --2- 172.21.15.52:0/919168719 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f78540a3bb0 0x7f78540a6070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:31.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.402+0000 7f786ca9a640 1 --2- 172.21.15.52:0/919168719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78681025d0 0x7f786819f1b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:31.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.402+0000 7f786ca9a640 1 --2- 172.21.15.52:0/919168719 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7868101c10 0x7f786819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:31.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.402+0000 7f786ca9a640 1 --2- 172.21.15.52:0/919168719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7868100a20 0x7f78680697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:31.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.402+0000 7f786ca9a640 1 -- 172.21.15.52:0/919168719 >> 172.21.15.52:0/919168719 conn(0x7f78680fbf90 msgr2=0x7f7868104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:31.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.403+0000 7f786ca9a640 1 -- 172.21.15.52:0/919168719 shutdown_connections 2024-04-02T13:50:31.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:31.403+0000 7f786ca9a640 1 -- 172.21.15.52:0/919168719 wait complete. 2024-04-02T13:50:31.734 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:31 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/919168719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:31.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:31 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/919168719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:32.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:31 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/919168719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:32.089 INFO:journalctl@ceph.mds.d.smithi052.stdout:Apr 02 13:50:31 smithi052 systemd[1]: Starting Ceph mds.d for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T13:50:32.383 INFO:journalctl@ceph.mds.d.smithi052.stdout:Apr 02 13:50:32 smithi052 podman[70374]: 2024-04-02 13:50:32.030924402 +0000 UTC m=+0.017466534 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:50:32.384 INFO:journalctl@ceph.mds.d.smithi052.stdout:Apr 02 13:50:32 smithi052 podman[70374]: 2024-04-02 13:50:32.128535321 +0000 UTC m=+0.115077440 container create ef988063f2ad61f1a4c1276706137e26765b5aedc40deac23f94a83fb3ba2df2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-02T13:50:32.384 INFO:journalctl@ceph.mds.d.smithi052.stdout:Apr 02 13:50:32 smithi052 podman[70374]: 2024-04-02 13:50:32.381432305 +0000 UTC m=+0.367974424 container init ef988063f2ad61f1a4c1276706137e26765b5aedc40deac23f94a83fb3ba2df2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240326, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-02T13:50:32.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:32 smithi052 ceph-mon[32104]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:32.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:32 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:32.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:32 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:32.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:32 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:32.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:32 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:32.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:32 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:32.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:32 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:32.840 INFO:journalctl@ceph.mds.d.smithi052.stdout:Apr 02 13:50:32 smithi052 podman[70374]: 2024-04-02 13:50:32.384208536 +0000 UTC m=+0.370750662 container start ef988063f2ad61f1a4c1276706137e26765b5aedc40deac23f94a83fb3ba2df2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-02T13:50:32.840 INFO:journalctl@ceph.mds.d.smithi052.stdout:Apr 02 13:50:32 smithi052 ceph-mds[70393]: -- 172.21.15.52:0/2186745253 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560108787a40 con 0x56010875b000 2024-04-02T13:50:32.840 INFO:journalctl@ceph.mds.d.smithi052.stdout:Apr 02 13:50:32 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-d[70389]: starting mds.d at 2024-04-02T13:50:32.840 INFO:journalctl@ceph.mds.d.smithi052.stdout:Apr 02 13:50:32 smithi052 bash[70374]: ef988063f2ad61f1a4c1276706137e26765b5aedc40deac23f94a83fb3ba2df2 2024-04-02T13:50:32.840 INFO:journalctl@ceph.mds.d.smithi052.stdout:Apr 02 13:50:32 smithi052 systemd[1]: Started Ceph mds.d for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4. 2024-04-02T13:50:32.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:32 smithi088 ceph-mon[36528]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:32.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:32 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:32.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:32 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:32.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:32 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:32.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:32 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:32.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:32 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:32.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:32 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:33.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:32 smithi189 ceph-mon[36683]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:33.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:32 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:33.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:32 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:33.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:32 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:33.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:32 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:33.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:32 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:33.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:32 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:33.417 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:50:33.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.534+0000 7f081547b640 1 -- 172.21.15.52:0/1787287538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0810103780 msgr2=0x7f081010a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.534+0000 7f081547b640 1 --2- 172.21.15.52:0/1787287538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0810103780 0x7f081010a010 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0804009930 tx=0x7f080402bc30 comp rx=0 tx=0).stop 2024-04-02T13:50:33.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.534+0000 7f081547b640 1 -- 172.21.15.52:0/1787287538 shutdown_connections 2024-04-02T13:50:33.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.534+0000 7f081547b640 1 --2- 172.21.15.52:0/1787287538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0810103780 0x7f081010a010 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.534+0000 7f081547b640 1 --2- 172.21.15.52:0/1787287538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0810102dc0 0x7f0810103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.534+0000 7f081547b640 1 --2- 172.21.15.52:0/1787287538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08100ffdf0 0x7f0810100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.534+0000 7f081547b640 1 -- 172.21.15.52:0/1787287538 >> 172.21.15.52:0/1787287538 conn(0x7f08100fb720 msgr2=0x7f08100fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.534+0000 7f081547b640 1 -- 172.21.15.52:0/1787287538 shutdown_connections 2024-04-02T13:50:33.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.534+0000 7f081547b640 1 -- 172.21.15.52:0/1787287538 wait complete. 2024-04-02T13:50:33.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.535+0000 7f081547b640 1 Processor -- start 2024-04-02T13:50:33.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.535+0000 7f081547b640 1 -- start start 2024-04-02T13:50:33.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.535+0000 7f081547b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08100ffdf0 0x7f0810102730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.535+0000 7f081547b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0810102dc0 0x7f0810100d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.535+0000 7f081547b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0810103780 0x7f08101012b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.535+0000 7f081547b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f081010d7a0 con 0x7f08100ffdf0 2024-04-02T13:50:33.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.535+0000 7f081547b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f081010d620 con 0x7f0810103780 2024-04-02T13:50:33.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.535+0000 7f081547b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f081010d920 con 0x7f0810102dc0 2024-04-02T13:50:33.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.536+0000 7f080effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08100ffdf0 0x7f0810102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.536+0000 7f080effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08100ffdf0 0x7f0810102730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57526/0 (socket says 172.21.15.52:57526) 2024-04-02T13:50:33.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.536+0000 7f080effd640 1 -- 172.21.15.52:0/3064852556 learned_addr learned my addr 172.21.15.52:0/3064852556 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:33.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.536+0000 7f080e7fc640 1 --2- 172.21.15.52:0/3064852556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0810102dc0 0x7f0810100d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.536+0000 7f080f7fe640 1 --2- 172.21.15.52:0/3064852556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0810103780 0x7f08101012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.536+0000 7f080effd640 1 -- 172.21.15.52:0/3064852556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0810102dc0 msgr2=0x7f0810100d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.536+0000 7f080effd640 1 --2- 172.21.15.52:0/3064852556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0810102dc0 0x7f0810100d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.536+0000 7f080effd640 1 -- 172.21.15.52:0/3064852556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0810103780 msgr2=0x7f08101012b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.536+0000 7f080effd640 1 --2- 172.21.15.52:0/3064852556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0810103780 0x7f08101012b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.536+0000 7f080effd640 1 -- 172.21.15.52:0/3064852556 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0810101b40 con 0x7f08100ffdf0 2024-04-02T13:50:33.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.536+0000 7f080f7fe640 1 --2- 172.21.15.52:0/3064852556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0810103780 0x7f08101012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:50:33.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.536+0000 7f080e7fc640 1 --2- 172.21.15.52:0/3064852556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0810102dc0 0x7f0810100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:50:33.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.536+0000 7f080effd640 1 --2- 172.21.15.52:0/3064852556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08100ffdf0 0x7f0810102730 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f080800ebc0 tx=0x7f080800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.536+0000 7f07f7fff640 1 -- 172.21.15.52:0/3064852556 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808003b70 con 0x7f08100ffdf0 2024-04-02T13:50:33.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.536+0000 7f07f7fff640 1 -- 172.21.15.52:0/3064852556 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0808003d10 con 0x7f08100ffdf0 2024-04-02T13:50:33.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.536+0000 7f081547b640 1 -- 172.21.15.52:0/3064852556 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08101a1f30 con 0x7f08100ffdf0 2024-04-02T13:50:33.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.536+0000 7f07f7fff640 1 -- 172.21.15.52:0/3064852556 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808019980 con 0x7f08100ffdf0 2024-04-02T13:50:33.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.536+0000 7f081547b640 1 -- 172.21.15.52:0/3064852556 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08101a2340 con 0x7f08100ffdf0 2024-04-02T13:50:33.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.537+0000 7f081547b640 1 -- 172.21.15.52:0/3064852556 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0810108ef0 con 0x7f08100ffdf0 2024-04-02T13:50:33.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.538+0000 7f07f7fff640 1 -- 172.21.15.52:0/3064852556 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0808019b20 con 0x7f08100ffdf0 2024-04-02T13:50:33.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.539+0000 7f07f7fff640 1 --2- 172.21.15.52:0/3064852556 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f07f80a3bb0 0x7f07f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.539+0000 7f080e7fc640 1 --2- 172.21.15.52:0/3064852556 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f07f80a3bb0 0x7f07f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.539+0000 7f07f7fff640 1 -- 172.21.15.52:0/3064852556 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f08080d07e0 con 0x7f08100ffdf0 2024-04-02T13:50:33.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.540+0000 7f080e7fc640 1 --2- 172.21.15.52:0/3064852556 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f07f80a3bb0 0x7f07f80a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f08101024a0 tx=0x7f080000b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.544+0000 7f07f7fff640 1 -- 172.21.15.52:0/3064852556 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f080809d660 con 0x7f08100ffdf0 2024-04-02T13:50:33.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.772+0000 7f081547b640 1 -- 172.21.15.52:0/3064852556 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f081002d370 con 0x7f08100ffdf0 2024-04-02T13:50:33.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.772+0000 7f07f7fff640 1 -- 172.21.15.52:0/3064852556 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7f080809cf60 con 0x7f08100ffdf0 2024-04-02T13:50:33.776 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:50:33.776 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-02T13:50:33.776 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 5 2024-04-02T13:50:33.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.774+0000 7f081547b640 1 -- 172.21.15.52:0/3064852556 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f07f80a3bb0 msgr2=0x7f07f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.774+0000 7f081547b640 1 --2- 172.21.15.52:0/3064852556 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f07f80a3bb0 0x7f07f80a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f08101024a0 tx=0x7f080000b5f0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.774+0000 7f081547b640 1 -- 172.21.15.52:0/3064852556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08100ffdf0 msgr2=0x7f0810102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.774+0000 7f081547b640 1 --2- 172.21.15.52:0/3064852556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08100ffdf0 0x7f0810102730 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f080800ebc0 tx=0x7f080800c6a0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.774+0000 7f081547b640 1 -- 172.21.15.52:0/3064852556 shutdown_connections 2024-04-02T13:50:33.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.774+0000 7f081547b640 1 --2- 172.21.15.52:0/3064852556 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f07f80a3bb0 0x7f07f80a6070 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.774+0000 7f081547b640 1 --2- 172.21.15.52:0/3064852556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0810103780 0x7f08101012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.774+0000 7f081547b640 1 --2- 172.21.15.52:0/3064852556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0810102dc0 0x7f0810100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.774+0000 7f081547b640 1 --2- 172.21.15.52:0/3064852556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08100ffdf0 0x7f0810102730 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.774+0000 7f081547b640 1 -- 172.21.15.52:0/3064852556 >> 172.21.15.52:0/3064852556 conn(0x7f08100fb720 msgr2=0x7f08101060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.775+0000 7f081547b640 1 -- 172.21.15.52:0/3064852556 shutdown_connections 2024-04-02T13:50:33.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:33.775+0000 7f081547b640 1 -- 172.21.15.52:0/3064852556 wait complete. 2024-04-02T13:50:33.787 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:33 smithi052 ceph-mon[32104]: Deploying daemon mds.e on smithi088 2024-04-02T13:50:33.787 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:33 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.52:6836/361596788,v1:172.21.15.52:6837/361596788] up:boot 2024-04-02T13:50:33.787 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:33 smithi052 ceph-mon[32104]: fsmap 4 up:standby 2024-04-02T13:50:33.787 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:33 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T13:50:33.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:33 smithi088 ceph-mon[36528]: Deploying daemon mds.e on smithi088 2024-04-02T13:50:33.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:33 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.52:6836/361596788,v1:172.21.15.52:6837/361596788] up:boot 2024-04-02T13:50:33.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:33 smithi088 ceph-mon[36528]: fsmap 4 up:standby 2024-04-02T13:50:33.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:33 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T13:50:34.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:33 smithi189 ceph-mon[36683]: Deploying daemon mds.e on smithi088 2024-04-02T13:50:34.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:33 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.52:6836/361596788,v1:172.21.15.52:6837/361596788] up:boot 2024-04-02T13:50:34.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:33 smithi189 ceph-mon[36683]: fsmap 4 up:standby 2024-04-02T13:50:34.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:33 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T13:50:34.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:34 smithi052 ceph-mon[32104]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:34.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:34 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3064852556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:34.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:34 smithi088 ceph-mon[36528]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:34.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:34 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3064852556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:34.920 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 02 13:50:34 smithi088 systemd[1]: Starting Ceph mds.e for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T13:50:35.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:34 smithi189 ceph-mon[36683]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:35.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:34 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3064852556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:35.292 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 02 13:50:35 smithi088 podman[66026]: 2024-04-02 13:50:34.922132368 +0000 UTC m=+0.017605453 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:50:35.292 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 02 13:50:35 smithi088 podman[66026]: 2024-04-02 13:50:35.044948771 +0000 UTC m=+0.140421840 container create ee6638eee1800dafee1270fe768726f9206f2a2fc3271aed2dcb507fa9462e4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-04-02T13:50:35.669 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 02 13:50:35 smithi088 podman[66026]: 2024-04-02 13:50:35.289514369 +0000 UTC m=+0.384987457 container init ee6638eee1800dafee1270fe768726f9206f2a2fc3271aed2dcb507fa9462e4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True) 2024-04-02T13:50:35.670 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 02 13:50:35 smithi088 podman[66026]: 2024-04-02 13:50:35.29266993 +0000 UTC m=+0.388143019 container start ee6638eee1800dafee1270fe768726f9206f2a2fc3271aed2dcb507fa9462e4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-e, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2) 2024-04-02T13:50:35.670 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 02 13:50:35 smithi088 ceph-mds[66046]: -- 172.21.15.88:0/1277629690 <== mon.2 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cc2bd01a40 con 0x55cc2bcd4800 2024-04-02T13:50:35.670 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 02 13:50:35 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-e[66042]: starting mds.e at 2024-04-02T13:50:35.670 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 02 13:50:35 smithi088 bash[66026]: ee6638eee1800dafee1270fe768726f9206f2a2fc3271aed2dcb507fa9462e4b 2024-04-02T13:50:35.670 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 02 13:50:35 smithi088 systemd[1]: Started Ceph mds.e for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4. 2024-04-02T13:50:35.789 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:50:35.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.897+0000 7fb6c6e60640 1 -- 172.21.15.52:0/3020252688 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6c01036d0 msgr2=0x7fb6c0109f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:35.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.897+0000 7fb6c6e60640 1 --2- 172.21.15.52:0/3020252688 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6c01036d0 0x7fb6c0109f60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009900 tx=0x7fb6b002bc30 comp rx=0 tx=0).stop 2024-04-02T13:50:35.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.897+0000 7fb6c6e60640 1 -- 172.21.15.52:0/3020252688 shutdown_connections 2024-04-02T13:50:35.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.897+0000 7fb6c6e60640 1 --2- 172.21.15.52:0/3020252688 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6c01036d0 0x7fb6c0109f60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.897+0000 7fb6c6e60640 1 --2- 172.21.15.52:0/3020252688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6c0102d10 0x7fb6c0103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.897+0000 7fb6c6e60640 1 --2- 172.21.15.52:0/3020252688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c0101b20 0x7fb6c0101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.897+0000 7fb6c6e60640 1 -- 172.21.15.52:0/3020252688 >> 172.21.15.52:0/3020252688 conn(0x7fb6c00fd0b0 msgr2=0x7fb6c00ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:35.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.897+0000 7fb6c6e60640 1 -- 172.21.15.52:0/3020252688 shutdown_connections 2024-04-02T13:50:35.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.897+0000 7fb6c6e60640 1 -- 172.21.15.52:0/3020252688 wait complete. 2024-04-02T13:50:35.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.897+0000 7fb6c6e60640 1 Processor -- start 2024-04-02T13:50:35.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.897+0000 7fb6c6e60640 1 -- start start 2024-04-02T13:50:35.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.897+0000 7fb6c6e60640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6c0101b20 0x7fb6c0197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.897+0000 7fb6c6e60640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c0102d10 0x7fb6c01982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.897+0000 7fb6c6e60640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6c01036d0 0x7fb6c019f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.897+0000 7fb6c6e60640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c010d7a0 con 0x7fb6c0101b20 2024-04-02T13:50:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.897+0000 7fb6c6e60640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c010d620 con 0x7fb6c01036d0 2024-04-02T13:50:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.897+0000 7fb6c6e60640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c010d920 con 0x7fb6c0102d10 2024-04-02T13:50:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.897+0000 7fb6c4bfc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6c0101b20 0x7fb6c0197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.897+0000 7fb6c4bfc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6c0101b20 0x7fb6c0197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57548/0 (socket says 172.21.15.52:57548) 2024-04-02T13:50:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.897+0000 7fb6c4bfc640 1 -- 172.21.15.52:0/588703739 learned_addr learned my addr 172.21.15.52:0/588703739 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.898+0000 7fb6bffff640 1 --2- 172.21.15.52:0/588703739 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c0102d10 0x7fb6c01982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.898+0000 7fb6c53fd640 1 --2- 172.21.15.52:0/588703739 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6c01036d0 0x7fb6c019f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.898+0000 7fb6c4bfc640 1 -- 172.21.15.52:0/588703739 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c0102d10 msgr2=0x7fb6c01982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.898+0000 7fb6c4bfc640 1 --2- 172.21.15.52:0/588703739 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c0102d10 0x7fb6c01982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.898+0000 7fb6c4bfc640 1 -- 172.21.15.52:0/588703739 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6c01036d0 msgr2=0x7fb6c019f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.898+0000 7fb6c4bfc640 1 --2- 172.21.15.52:0/588703739 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6c01036d0 0x7fb6c019f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.898+0000 7fb6c4bfc640 1 -- 172.21.15.52:0/588703739 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c019f9d0 con 0x7fb6c0101b20 2024-04-02T13:50:35.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.898+0000 7fb6c53fd640 1 --2- 172.21.15.52:0/588703739 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6c01036d0 0x7fb6c019f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:50:35.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.898+0000 7fb6c4bfc640 1 --2- 172.21.15.52:0/588703739 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6c0101b20 0x7fb6c0197db0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800eb70 tx=0x7fb6b800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:35.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.898+0000 7fb6bdffb640 1 -- 172.21.15.52:0/588703739 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8003b70 con 0x7fb6c0101b20 2024-04-02T13:50:35.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.898+0000 7fb6bdffb640 1 -- 172.21.15.52:0/588703739 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6b800ee70 con 0x7fb6c0101b20 2024-04-02T13:50:35.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.898+0000 7fb6c6e60640 1 -- 172.21.15.52:0/588703739 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c019fcc0 con 0x7fb6c0101b20 2024-04-02T13:50:35.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.898+0000 7fb6bdffb640 1 -- 172.21.15.52:0/588703739 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8010830 con 0x7fb6c0101b20 2024-04-02T13:50:35.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.898+0000 7fb6c6e60640 1 -- 172.21.15.52:0/588703739 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c01a01c0 con 0x7fb6c0101b20 2024-04-02T13:50:35.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.899+0000 7fb6c6e60640 1 -- 172.21.15.52:0/588703739 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c0108e40 con 0x7fb6c0101b20 2024-04-02T13:50:35.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.900+0000 7fb6bdffb640 1 -- 172.21.15.52:0/588703739 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb6b8004020 con 0x7fb6c0101b20 2024-04-02T13:50:35.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.901+0000 7fb6bdffb640 1 --2- 172.21.15.52:0/588703739 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb6a80a3c30 0x7fb6a80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:35.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.901+0000 7fb6bffff640 1 --2- 172.21.15.52:0/588703739 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb6a80a3c30 0x7fb6a80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:35.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.901+0000 7fb6bdffb640 1 -- 172.21.15.52:0/588703739 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb6b80d06b0 con 0x7fb6c0101b20 2024-04-02T13:50:35.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.901+0000 7fb6bffff640 1 --2- 172.21.15.52:0/588703739 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb6a80a3c30 0x7fb6a80a60f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb6c01992c0 tx=0x7fb6b400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:35.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:35.905+0000 7fb6bdffb640 1 -- 172.21.15.52:0/588703739 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb6b809d490 con 0x7fb6c0101b20 2024-04-02T13:50:36.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:36.131+0000 7fb6c6e60640 1 -- 172.21.15.52:0/588703739 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6c002d370 con 0x7fb6c0101b20 2024-04-02T13:50:36.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:36.132+0000 7fb6bdffb640 1 -- 172.21.15.52:0/588703739 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4684 (secure 0 0 0) 0x7fb6c002d370 con 0x7fb6c0101b20 2024-04-02T13:50:36.135 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:50:36.135 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-02T13:50:36.135 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 6 2024-04-02T13:50:36.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:36.134+0000 7fb6c6e60640 1 -- 172.21.15.52:0/588703739 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb6a80a3c30 msgr2=0x7fb6a80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:36.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:36.134+0000 7fb6c6e60640 1 --2- 172.21.15.52:0/588703739 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb6a80a3c30 0x7fb6a80a60f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb6c01992c0 tx=0x7fb6b400b390 comp rx=0 tx=0).stop 2024-04-02T13:50:36.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:36.134+0000 7fb6c6e60640 1 -- 172.21.15.52:0/588703739 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6c0101b20 msgr2=0x7fb6c0197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:36.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:36.134+0000 7fb6c6e60640 1 --2- 172.21.15.52:0/588703739 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6c0101b20 0x7fb6c0197db0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800eb70 tx=0x7fb6b800c6a0 comp rx=0 tx=0).stop 2024-04-02T13:50:36.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:36.134+0000 7fb6c6e60640 1 -- 172.21.15.52:0/588703739 shutdown_connections 2024-04-02T13:50:36.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:36.134+0000 7fb6c6e60640 1 --2- 172.21.15.52:0/588703739 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb6a80a3c30 0x7fb6a80a60f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:36.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:36.134+0000 7fb6c6e60640 1 --2- 172.21.15.52:0/588703739 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6c01036d0 0x7fb6c019f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:36.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:36.134+0000 7fb6c6e60640 1 --2- 172.21.15.52:0/588703739 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c0102d10 0x7fb6c01982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:36.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:36.134+0000 7fb6c6e60640 1 --2- 172.21.15.52:0/588703739 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6c0101b20 0x7fb6c0197db0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:36.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:36.134+0000 7fb6c6e60640 1 -- 172.21.15.52:0/588703739 >> 172.21.15.52:0/588703739 conn(0x7fb6c00fd0b0 msgr2=0x7fb6c0106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:36.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:36.134+0000 7fb6c6e60640 1 -- 172.21.15.52:0/588703739 shutdown_connections 2024-04-02T13:50:36.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:36.134+0000 7fb6c6e60640 1 -- 172.21.15.52:0/588703739 wait complete. 2024-04-02T13:50:36.677 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:36 smithi189 ceph-mon[36683]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:36.677 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:36 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:36.677 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:36 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:36.677 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:36 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:36.677 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:36 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:36.677 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:36 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:36.677 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:36 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:36.677 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:36 smithi189 ceph-mon[36683]: Deploying daemon mds.f on smithi189 2024-04-02T13:50:36.677 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:36 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.88:6834/5019107,v1:172.21.15.88:6835/5019107] up:boot 2024-04-02T13:50:36.677 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:36 smithi189 ceph-mon[36683]: fsmap 5 up:standby 2024-04-02T13:50:36.677 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:36 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T13:50:36.678 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:36 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/588703739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:36.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:36 smithi052 ceph-mon[32104]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:36.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:36 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:36.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:36 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:36.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:36 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:36.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:36 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:36.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:36 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:36.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:36 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:36.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:36 smithi052 ceph-mon[32104]: Deploying daemon mds.f on smithi189 2024-04-02T13:50:36.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:36 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.88:6834/5019107,v1:172.21.15.88:6835/5019107] up:boot 2024-04-02T13:50:36.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:36 smithi052 ceph-mon[32104]: fsmap 5 up:standby 2024-04-02T13:50:36.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:36 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T13:50:36.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:36 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/588703739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:36.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:36 smithi088 ceph-mon[36528]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:36.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:36 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:36.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:36 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:36.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:36 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:36.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:36 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:36.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:36 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:36.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:36 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:36.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:36 smithi088 ceph-mon[36528]: Deploying daemon mds.f on smithi189 2024-04-02T13:50:36.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:36 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.88:6834/5019107,v1:172.21.15.88:6835/5019107] up:boot 2024-04-02T13:50:36.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:36 smithi088 ceph-mon[36528]: fsmap 5 up:standby 2024-04-02T13:50:36.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:36 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T13:50:36.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:36 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/588703739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:37.788 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 02 13:50:37 smithi189 systemd[1]: Starting Ceph mds.f for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T13:50:38.149 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:50:38.176 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 02 13:50:37 smithi189 podman[66866]: 2024-04-02 13:50:37.801668192 +0000 UTC m=+0.017759428 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:50:38.176 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 02 13:50:37 smithi189 podman[66866]: 2024-04-02 13:50:37.928781883 +0000 UTC m=+0.144873100 container create 67fc9458e9f5e1b2a3bee3c98f549973deb66238ef10de0c3f48f3187bb8d9c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-f, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-02T13:50:38.176 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 02 13:50:38 smithi189 podman[66866]: 2024-04-02 13:50:38.173490527 +0000 UTC m=+0.389581741 container init 67fc9458e9f5e1b2a3bee3c98f549973deb66238ef10de0c3f48f3187bb8d9c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, GIT_CLEAN=True) 2024-04-02T13:50:38.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.261+0000 7f010edcd640 1 -- 172.21.15.52:0/182710678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01081036d0 msgr2=0x7f0108109f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:38.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.261+0000 7f010edcd640 1 --2- 172.21.15.52:0/182710678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01081036d0 0x7f0108109f60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0100009ca0 tx=0x7f010002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:50:38.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.261+0000 7f010edcd640 1 -- 172.21.15.52:0/182710678 shutdown_connections 2024-04-02T13:50:38.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.261+0000 7f010edcd640 1 --2- 172.21.15.52:0/182710678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01081036d0 0x7f0108109f60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:38.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.261+0000 7f010edcd640 1 --2- 172.21.15.52:0/182710678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0108102d10 0x7f0108103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:38.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.261+0000 7f010edcd640 1 --2- 172.21.15.52:0/182710678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0108101b20 0x7f0108101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:38.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.261+0000 7f010edcd640 1 -- 172.21.15.52:0/182710678 >> 172.21.15.52:0/182710678 conn(0x7f01080fd0b0 msgr2=0x7f01080ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:38.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.261+0000 7f010edcd640 1 -- 172.21.15.52:0/182710678 shutdown_connections 2024-04-02T13:50:38.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.261+0000 7f010edcd640 1 -- 172.21.15.52:0/182710678 wait complete. 2024-04-02T13:50:38.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.262+0000 7f010edcd640 1 Processor -- start 2024-04-02T13:50:38.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.262+0000 7f010edcd640 1 -- start start 2024-04-02T13:50:38.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.262+0000 7f010edcd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0108101b20 0x7f0108197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:38.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.262+0000 7f010edcd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0108102d10 0x7f01081982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:38.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.262+0000 7f010edcd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01081036d0 0x7f010819f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:38.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.262+0000 7f010edcd640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f010810d780 con 0x7f0108101b20 2024-04-02T13:50:38.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.262+0000 7f010edcd640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f010810d600 con 0x7f01081036d0 2024-04-02T13:50:38.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.262+0000 7f010edcd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f010810d900 con 0x7f0108102d10 2024-04-02T13:50:38.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.262+0000 7f010cb69640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0108101b20 0x7f0108197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:38.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.262+0000 7f010cb69640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0108101b20 0x7f0108197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57562/0 (socket says 172.21.15.52:57562) 2024-04-02T13:50:38.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.262+0000 7f010cb69640 1 -- 172.21.15.52:0/1923702497 learned_addr learned my addr 172.21.15.52:0/1923702497 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:38.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.262+0000 7f010d36a640 1 --2- 172.21.15.52:0/1923702497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01081036d0 0x7f010819f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:38.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.262+0000 7f0107fff640 1 --2- 172.21.15.52:0/1923702497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0108102d10 0x7f01081982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:38.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.263+0000 7f010cb69640 1 -- 172.21.15.52:0/1923702497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0108102d10 msgr2=0x7f01081982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:38.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.263+0000 7f010cb69640 1 --2- 172.21.15.52:0/1923702497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0108102d10 0x7f01081982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:38.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.263+0000 7f010cb69640 1 -- 172.21.15.52:0/1923702497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01081036d0 msgr2=0x7f010819f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:38.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.263+0000 7f010cb69640 1 --2- 172.21.15.52:0/1923702497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01081036d0 0x7f010819f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:38.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.263+0000 7f010cb69640 1 -- 172.21.15.52:0/1923702497 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010819f9c0 con 0x7f0108101b20 2024-04-02T13:50:38.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.263+0000 7f010cb69640 1 --2- 172.21.15.52:0/1923702497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0108101b20 0x7f0108197db0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f00f800e9c0 tx=0x7f00f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:38.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.263+0000 7f0105ffb640 1 -- 172.21.15.52:0/1923702497 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f800cde0 con 0x7f0108101b20 2024-04-02T13:50:38.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.263+0000 7f0105ffb640 1 -- 172.21.15.52:0/1923702497 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00f8010910 con 0x7f0108101b20 2024-04-02T13:50:38.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.263+0000 7f010edcd640 1 -- 172.21.15.52:0/1923702497 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f010819fcb0 con 0x7f0108101b20 2024-04-02T13:50:38.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.263+0000 7f0105ffb640 1 -- 172.21.15.52:0/1923702497 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f8018a00 con 0x7f0108101b20 2024-04-02T13:50:38.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.263+0000 7f010edcd640 1 -- 172.21.15.52:0/1923702497 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01081a01b0 con 0x7f0108101b20 2024-04-02T13:50:38.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.264+0000 7f010edcd640 1 -- 172.21.15.52:0/1923702497 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0108108de0 con 0x7f0108101b20 2024-04-02T13:50:38.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.265+0000 7f0105ffb640 1 -- 172.21.15.52:0/1923702497 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f00f801f020 con 0x7f0108101b20 2024-04-02T13:50:38.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.266+0000 7f0105ffb640 1 --2- 172.21.15.52:0/1923702497 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f00f00a3bb0 0x7f00f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:38.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.266+0000 7f0105ffb640 1 -- 172.21.15.52:0/1923702497 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f00f80d01c0 con 0x7f0108101b20 2024-04-02T13:50:38.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.266+0000 7f0107fff640 1 --2- 172.21.15.52:0/1923702497 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f00f00a3bb0 0x7f00f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:38.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.266+0000 7f0107fff640 1 --2- 172.21.15.52:0/1923702497 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f00f00a3bb0 0x7f00f00a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00bbe0 tx=0x7f00fc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:38.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.270+0000 7f0105ffb640 1 -- 172.21.15.52:0/1923702497 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f00f809d040 con 0x7f0108101b20 2024-04-02T13:50:38.460 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 02 13:50:38 smithi189 podman[66866]: 2024-04-02 13:50:38.176295074 +0000 UTC m=+0.392386305 container start 67fc9458e9f5e1b2a3bee3c98f549973deb66238ef10de0c3f48f3187bb8d9c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:50:38.460 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 02 13:50:38 smithi189 ceph-mds[66885]: -- 172.21.15.189:0/3537990577 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5603bb53fa40 con 0x5603bb512800 2024-04-02T13:50:38.460 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 02 13:50:38 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-f[66881]: starting mds.f at 2024-04-02T13:50:38.460 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 02 13:50:38 smithi189 bash[66866]: 67fc9458e9f5e1b2a3bee3c98f549973deb66238ef10de0c3f48f3187bb8d9c0 2024-04-02T13:50:38.460 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 02 13:50:38 smithi189 systemd[1]: Started Ceph mds.f for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4. 2024-04-02T13:50:38.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.500+0000 7f010edcd640 1 -- 172.21.15.52:0/1923702497 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f010802d370 con 0x7f0108101b20 2024-04-02T13:50:38.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.500+0000 7f0105ffb640 1 -- 172.21.15.52:0/1923702497 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5512 (secure 0 0 0) 0x7f010802d370 con 0x7f0108101b20 2024-04-02T13:50:38.503 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:50:38.503 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-02T13:50:38.504 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 7 2024-04-02T13:50:38.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.502+0000 7f010edcd640 1 -- 172.21.15.52:0/1923702497 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f00f00a3bb0 msgr2=0x7f00f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:38.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.502+0000 7f010edcd640 1 --2- 172.21.15.52:0/1923702497 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f00f00a3bb0 0x7f00f00a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00bbe0 tx=0x7f00fc00b390 comp rx=0 tx=0).stop 2024-04-02T13:50:38.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.502+0000 7f010edcd640 1 -- 172.21.15.52:0/1923702497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0108101b20 msgr2=0x7f0108197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:38.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.502+0000 7f010edcd640 1 --2- 172.21.15.52:0/1923702497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0108101b20 0x7f0108197db0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f00f800e9c0 tx=0x7f00f800ee80 comp rx=0 tx=0).stop 2024-04-02T13:50:38.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.502+0000 7f010edcd640 1 -- 172.21.15.52:0/1923702497 shutdown_connections 2024-04-02T13:50:38.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.502+0000 7f010edcd640 1 --2- 172.21.15.52:0/1923702497 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f00f00a3bb0 0x7f00f00a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:38.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.502+0000 7f010edcd640 1 --2- 172.21.15.52:0/1923702497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01081036d0 0x7f010819f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:38.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.502+0000 7f010edcd640 1 --2- 172.21.15.52:0/1923702497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0108102d10 0x7f01081982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:38.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.502+0000 7f010edcd640 1 --2- 172.21.15.52:0/1923702497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0108101b20 0x7f0108197db0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:38.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.502+0000 7f010edcd640 1 -- 172.21.15.52:0/1923702497 >> 172.21.15.52:0/1923702497 conn(0x7f01080fd0b0 msgr2=0x7f0108105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:38.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.502+0000 7f010edcd640 1 -- 172.21.15.52:0/1923702497 shutdown_connections 2024-04-02T13:50:38.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:38.502+0000 7f010edcd640 1 -- 172.21.15.52:0/1923702497 wait complete. 2024-04-02T13:50:38.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:38 smithi189 ceph-mon[36683]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:38.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:38 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:38.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:38 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:38.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:38 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:38.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:38 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:38.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:38 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:38.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:38 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:38.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:38 smithi052 ceph-mon[32104]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:38.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:38 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:38.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:38 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:38.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:38 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:38.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:38 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:38.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:38 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:38.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:38 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:38.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:38 smithi088 ceph-mon[36528]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:38 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:38 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:38 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:38 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:38 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:38 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:39.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:39 smithi189 ceph-mon[36683]: Deploying daemon mds.c on smithi189 2024-04-02T13:50:39.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:39 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] up:boot 2024-04-02T13:50:39.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:39 smithi189 ceph-mon[36683]: fsmap 6 up:standby 2024-04-02T13:50:39.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:39 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T13:50:39.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:39 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1923702497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:39.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:39 smithi052 ceph-mon[32104]: Deploying daemon mds.c on smithi189 2024-04-02T13:50:39.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:39 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] up:boot 2024-04-02T13:50:39.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:39 smithi052 ceph-mon[32104]: fsmap 6 up:standby 2024-04-02T13:50:39.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:39 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T13:50:39.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:39 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1923702497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:39.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:39 smithi088 ceph-mon[36528]: Deploying daemon mds.c on smithi189 2024-04-02T13:50:39.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:39 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] up:boot 2024-04-02T13:50:39.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:39 smithi088 ceph-mon[36528]: fsmap 6 up:standby 2024-04-02T13:50:39.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:39 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T13:50:39.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:39 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1923702497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:40.517 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:50:40.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:40 smithi189 ceph-mon[36683]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:40.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.629+0000 7f1a56b04640 1 -- 172.21.15.52:0/3120080471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a50102d10 msgr2=0x7f1a50103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:40.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.629+0000 7f1a56b04640 1 --2- 172.21.15.52:0/3120080471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a50102d10 0x7f1a50103190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009900 tx=0x7f1a4402bc30 comp rx=0 tx=0).stop 2024-04-02T13:50:40.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.629+0000 7f1a56b04640 1 -- 172.21.15.52:0/3120080471 shutdown_connections 2024-04-02T13:50:40.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.629+0000 7f1a56b04640 1 --2- 172.21.15.52:0/3120080471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a501036d0 0x7f1a50109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:40.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.629+0000 7f1a56b04640 1 --2- 172.21.15.52:0/3120080471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a50102d10 0x7f1a50103190 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:40.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.629+0000 7f1a56b04640 1 --2- 172.21.15.52:0/3120080471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a50101b20 0x7f1a50101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:40.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.629+0000 7f1a56b04640 1 -- 172.21.15.52:0/3120080471 >> 172.21.15.52:0/3120080471 conn(0x7f1a500fd0b0 msgr2=0x7f1a500ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:40.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.629+0000 7f1a56b04640 1 -- 172.21.15.52:0/3120080471 shutdown_connections 2024-04-02T13:50:40.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.629+0000 7f1a56b04640 1 -- 172.21.15.52:0/3120080471 wait complete. 2024-04-02T13:50:40.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.629+0000 7f1a56b04640 1 Processor -- start 2024-04-02T13:50:40.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.629+0000 7f1a56b04640 1 -- start start 2024-04-02T13:50:40.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.630+0000 7f1a56b04640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a50101b20 0x7f1a50197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:40.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.630+0000 7f1a56b04640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a50102d10 0x7f1a50198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:40.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.630+0000 7f1a56b04640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a501036d0 0x7f1a5019f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:40.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.630+0000 7f1a56b04640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5010d7a0 con 0x7f1a50102d10 2024-04-02T13:50:40.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.630+0000 7f1a56b04640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5010d620 con 0x7f1a501036d0 2024-04-02T13:50:40.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.630+0000 7f1a56b04640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5010d920 con 0x7f1a50101b20 2024-04-02T13:50:40.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.630+0000 7f1a4ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a50102d10 0x7f1a50198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:40.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.630+0000 7f1a4ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a50102d10 0x7f1a50198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57588/0 (socket says 172.21.15.52:57588) 2024-04-02T13:50:40.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.630+0000 7f1a4ffff640 1 -- 172.21.15.52:0/2690211795 learned_addr learned my addr 172.21.15.52:0/2690211795 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:40.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.630+0000 7f1a548a0640 1 --2- 172.21.15.52:0/2690211795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a50101b20 0x7f1a50197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:40.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.630+0000 7f1a550a1640 1 --2- 172.21.15.52:0/2690211795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a501036d0 0x7f1a5019f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:40.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.630+0000 7f1a4ffff640 1 -- 172.21.15.52:0/2690211795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a50101b20 msgr2=0x7f1a50197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:40.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.630+0000 7f1a4ffff640 1 --2- 172.21.15.52:0/2690211795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a50101b20 0x7f1a50197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:40.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.630+0000 7f1a4ffff640 1 -- 172.21.15.52:0/2690211795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a501036d0 msgr2=0x7f1a5019f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:40.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.630+0000 7f1a4ffff640 1 --2- 172.21.15.52:0/2690211795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a501036d0 0x7f1a5019f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:40.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.630+0000 7f1a4ffff640 1 -- 172.21.15.52:0/2690211795 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5019fa30 con 0x7f1a50102d10 2024-04-02T13:50:40.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.630+0000 7f1a4ffff640 1 --2- 172.21.15.52:0/2690211795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a50102d10 0x7f1a50198350 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009230 tx=0x7f1a44005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:40.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.630+0000 7f1a4dffb640 1 -- 172.21.15.52:0/2690211795 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44041070 con 0x7f1a50102d10 2024-04-02T13:50:40.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.630+0000 7f1a4dffb640 1 -- 172.21.15.52:0/2690211795 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a440044d0 con 0x7f1a50102d10 2024-04-02T13:50:40.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.630+0000 7f1a56b04640 1 -- 172.21.15.52:0/2690211795 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a5019fcc0 con 0x7f1a50102d10 2024-04-02T13:50:40.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.630+0000 7f1a4dffb640 1 -- 172.21.15.52:0/2690211795 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44034440 con 0x7f1a50102d10 2024-04-02T13:50:40.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.631+0000 7f1a56b04640 1 -- 172.21.15.52:0/2690211795 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a501a01b0 con 0x7f1a50102d10 2024-04-02T13:50:40.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.632+0000 7f1a56b04640 1 -- 172.21.15.52:0/2690211795 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a50108e40 con 0x7f1a50102d10 2024-04-02T13:50:40.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.633+0000 7f1a4dffb640 1 -- 172.21.15.52:0/2690211795 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1a44044070 con 0x7f1a50102d10 2024-04-02T13:50:40.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.633+0000 7f1a4dffb640 1 --2- 172.21.15.52:0/2690211795 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1a380a3bb0 0x7f1a380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:40.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.633+0000 7f1a548a0640 1 --2- 172.21.15.52:0/2690211795 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1a380a3bb0 0x7f1a380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:40.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.633+0000 7f1a4dffb640 1 -- 172.21.15.52:0/2690211795 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f1a440f1bb0 con 0x7f1a50102d10 2024-04-02T13:50:40.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.634+0000 7f1a548a0640 1 --2- 172.21.15.52:0/2690211795 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1a380a3bb0 0x7f1a380a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1a50102b70 tx=0x7f1a48009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:40.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.638+0000 7f1a4dffb640 1 -- 172.21.15.52:0/2690211795 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1a440bea30 con 0x7f1a50102d10 2024-04-02T13:50:40.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:40 smithi052 ceph-mon[32104]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:40.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.868+0000 7f1a56b04640 1 -- 172.21.15.52:0/2690211795 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a5002d580 con 0x7f1a50102d10 2024-04-02T13:50:40.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.868+0000 7f1a4dffb640 1 -- 172.21.15.52:0/2690211795 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5512 (secure 0 0 0) 0x7f1a440be330 con 0x7f1a50102d10 2024-04-02T13:50:40.871 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:50:40.871 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-02T13:50:40.872 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 7 2024-04-02T13:50:40.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.870+0000 7f1a56b04640 1 -- 172.21.15.52:0/2690211795 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1a380a3bb0 msgr2=0x7f1a380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:40.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.870+0000 7f1a56b04640 1 --2- 172.21.15.52:0/2690211795 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1a380a3bb0 0x7f1a380a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1a50102b70 tx=0x7f1a48009500 comp rx=0 tx=0).stop 2024-04-02T13:50:40.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.871+0000 7f1a56b04640 1 -- 172.21.15.52:0/2690211795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a50102d10 msgr2=0x7f1a50198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:40.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.871+0000 7f1a56b04640 1 --2- 172.21.15.52:0/2690211795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a50102d10 0x7f1a50198350 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009230 tx=0x7f1a44005d50 comp rx=0 tx=0).stop 2024-04-02T13:50:40.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.871+0000 7f1a56b04640 1 -- 172.21.15.52:0/2690211795 shutdown_connections 2024-04-02T13:50:40.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.871+0000 7f1a56b04640 1 --2- 172.21.15.52:0/2690211795 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1a380a3bb0 0x7f1a380a6070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:40.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.871+0000 7f1a56b04640 1 --2- 172.21.15.52:0/2690211795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a501036d0 0x7f1a5019f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:40.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.871+0000 7f1a56b04640 1 --2- 172.21.15.52:0/2690211795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a50102d10 0x7f1a50198350 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:40.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.871+0000 7f1a56b04640 1 --2- 172.21.15.52:0/2690211795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a50101b20 0x7f1a50197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:40.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.871+0000 7f1a56b04640 1 -- 172.21.15.52:0/2690211795 >> 172.21.15.52:0/2690211795 conn(0x7f1a500fd0b0 msgr2=0x7f1a50106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:40.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.871+0000 7f1a56b04640 1 -- 172.21.15.52:0/2690211795 shutdown_connections 2024-04-02T13:50:40.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:40.871+0000 7f1a56b04640 1 -- 172.21.15.52:0/2690211795 wait complete. 2024-04-02T13:50:40.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:40 smithi088 ceph-mon[36528]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:41.288 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 02 13:50:41 smithi189 systemd[1]: Started Ceph mds.c for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4. 2024-04-02T13:50:41.776 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:41 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2690211795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:41.776 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:41 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:41.776 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:41 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:41.776 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:41 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:41.776 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:41 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:41.776 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:41 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:41.776 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:41 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:41.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:41 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2690211795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:41.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:41 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:41.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:41 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:41.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:41 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:41.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:41 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:41.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:41 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:41.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:41 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:41.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:41 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2690211795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:41.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:41 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:41.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:41 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:41.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:41 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:41.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:41 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:41.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:41 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:41.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:41 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:42.779 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:42 smithi088 ceph-mon[36528]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:42.779 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:42 smithi088 ceph-mon[36528]: Deploying daemon mds.h on smithi088 2024-04-02T13:50:42.779 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:42 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] up:boot 2024-04-02T13:50:42.779 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:42 smithi088 ceph-mon[36528]: fsmap 7 up:standby 2024-04-02T13:50:42.779 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:42 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T13:50:42.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:42 smithi189 ceph-mon[36683]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:42.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:42 smithi189 ceph-mon[36683]: Deploying daemon mds.h on smithi088 2024-04-02T13:50:42.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:42 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] up:boot 2024-04-02T13:50:42.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:42 smithi189 ceph-mon[36683]: fsmap 7 up:standby 2024-04-02T13:50:42.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:42 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T13:50:42.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:42 smithi052 ceph-mon[32104]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:42.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:42 smithi052 ceph-mon[32104]: Deploying daemon mds.h on smithi088 2024-04-02T13:50:42.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:42 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] up:boot 2024-04-02T13:50:42.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:42 smithi052 ceph-mon[32104]: fsmap 7 up:standby 2024-04-02T13:50:42.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:42 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T13:50:42.886 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:50:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.997+0000 7f8551b3d640 1 -- 172.21.15.52:0/3635088986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f854c104ed0 msgr2=0x7f854c1072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.997+0000 7f8551b3d640 1 --2- 172.21.15.52:0/3635088986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f854c104ed0 0x7f854c1072c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8540009900 tx=0x7f854002bc30 comp rx=0 tx=0).stop 2024-04-02T13:50:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.998+0000 7f8551b3d640 1 -- 172.21.15.52:0/3635088986 shutdown_connections 2024-04-02T13:50:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.998+0000 7f8551b3d640 1 --2- 172.21.15.52:0/3635088986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f854c107800 0x7f854c109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.998+0000 7f8551b3d640 1 --2- 172.21.15.52:0/3635088986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f854c104ed0 0x7f854c1072c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.998+0000 7f8551b3d640 1 --2- 172.21.15.52:0/3635088986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f854c1025a0 0x7f854c104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.998+0000 7f8551b3d640 1 -- 172.21.15.52:0/3635088986 >> 172.21.15.52:0/3635088986 conn(0x7f854c0fbf90 msgr2=0x7f854c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.998+0000 7f8551b3d640 1 -- 172.21.15.52:0/3635088986 shutdown_connections 2024-04-02T13:50:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.998+0000 7f8551b3d640 1 -- 172.21.15.52:0/3635088986 wait complete. 2024-04-02T13:50:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.998+0000 7f8551b3d640 1 Processor -- start 2024-04-02T13:50:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.998+0000 7f8551b3d640 1 -- start start 2024-04-02T13:50:43.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.999+0000 7f8551b3d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f854c1025a0 0x7f854c197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:43.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.999+0000 7f8551b3d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f854c104ed0 0x7f854c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:43.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.999+0000 7f8551b3d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f854c107800 0x7f854c19f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:43.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.999+0000 7f8551b3d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f854c10d7b0 con 0x7f854c107800 2024-04-02T13:50:43.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.999+0000 7f8551b3d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f854c10d630 con 0x7f854c104ed0 2024-04-02T13:50:43.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.999+0000 7f8551b3d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f854c10d930 con 0x7f854c1025a0 2024-04-02T13:50:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.999+0000 7f854bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f854c107800 0x7f854c19f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.999+0000 7f854bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f854c107800 0x7f854c19f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33030/0 (socket says 172.21.15.52:33030) 2024-04-02T13:50:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.999+0000 7f854bfff640 1 -- 172.21.15.52:0/3825590440 learned_addr learned my addr 172.21.15.52:0/3825590440 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.999+0000 7f854affd640 1 --2- 172.21.15.52:0/3825590440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f854c104ed0 0x7f854c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.999+0000 7f854b7fe640 1 --2- 172.21.15.52:0/3825590440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f854c1025a0 0x7f854c197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.999+0000 7f854bfff640 1 -- 172.21.15.52:0/3825590440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f854c1025a0 msgr2=0x7f854c197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.999+0000 7f854bfff640 1 --2- 172.21.15.52:0/3825590440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f854c1025a0 0x7f854c197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.999+0000 7f854bfff640 1 -- 172.21.15.52:0/3825590440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f854c104ed0 msgr2=0x7f854c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.999+0000 7f854bfff640 1 --2- 172.21.15.52:0/3825590440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f854c104ed0 0x7f854c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.999+0000 7f854bfff640 1 -- 172.21.15.52:0/3825590440 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854c19faa0 con 0x7f854c107800 2024-04-02T13:50:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.999+0000 7f854b7fe640 1 --2- 172.21.15.52:0/3825590440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f854c1025a0 0x7f854c197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:50:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.999+0000 7f854bfff640 1 --2- 172.21.15.52:0/3825590440 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f854c107800 0x7f854c19f430 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f854400eaa0 tx=0x7f854400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.999+0000 7f8548ff9640 1 -- 172.21.15.52:0/3825590440 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854400cdb0 con 0x7f854c107800 2024-04-02T13:50:43.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.999+0000 7f8548ff9640 1 -- 172.21.15.52:0/3825590440 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8544005020 con 0x7f854c107800 2024-04-02T13:50:43.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:42.999+0000 7f8551b3d640 1 -- 172.21.15.52:0/3825590440 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f854c19fd90 con 0x7f854c107800 2024-04-02T13:50:43.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:43.000+0000 7f8548ff9640 1 -- 172.21.15.52:0/3825590440 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85440116b0 con 0x7f854c107800 2024-04-02T13:50:43.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:43.000+0000 7f8551b3d640 1 -- 172.21.15.52:0/3825590440 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f854c1a0250 con 0x7f854c107800 2024-04-02T13:50:43.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:43.001+0000 7f8551b3d640 1 -- 172.21.15.52:0/3825590440 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f854c0fda80 con 0x7f854c107800 2024-04-02T13:50:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:43.002+0000 7f8548ff9640 1 -- 172.21.15.52:0/3825590440 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8544005560 con 0x7f854c107800 2024-04-02T13:50:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:43.002+0000 7f8548ff9640 1 --2- 172.21.15.52:0/3825590440 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f85340a3c30 0x7f85340a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:43.003+0000 7f854b7fe640 1 --2- 172.21.15.52:0/3825590440 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f85340a3c30 0x7f85340a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:43.003+0000 7f8548ff9640 1 -- 172.21.15.52:0/3825590440 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f85440d01d0 con 0x7f854c107800 2024-04-02T13:50:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:43.003+0000 7f854b7fe640 1 --2- 172.21.15.52:0/3825590440 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f85340a3c30 0x7f85340a60f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f853c005fd0 tx=0x7f853c005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:43.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:43.007+0000 7f8548ff9640 1 -- 172.21.15.52:0/3825590440 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8544016030 con 0x7f854c107800 2024-04-02T13:50:43.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:43.235+0000 7f8551b3d640 1 -- 172.21.15.52:0/3825590440 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f854c062700 con 0x7f854c107800 2024-04-02T13:50:43.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:43.236+0000 7f8548ff9640 1 -- 172.21.15.52:0/3825590440 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6340 (secure 0 0 0) 0x7f854409cfb0 con 0x7f854c107800 2024-04-02T13:50:43.239 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:50:43.239 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-02T13:50:43.240 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 8 2024-04-02T13:50:43.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:43.238+0000 7f8551b3d640 1 -- 172.21.15.52:0/3825590440 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f85340a3c30 msgr2=0x7f85340a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:43.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:43.238+0000 7f8551b3d640 1 --2- 172.21.15.52:0/3825590440 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f85340a3c30 0x7f85340a60f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f853c005fd0 tx=0x7f853c005ea0 comp rx=0 tx=0).stop 2024-04-02T13:50:43.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:43.238+0000 7f8551b3d640 1 -- 172.21.15.52:0/3825590440 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f854c107800 msgr2=0x7f854c19f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:43.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:43.238+0000 7f8551b3d640 1 --2- 172.21.15.52:0/3825590440 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f854c107800 0x7f854c19f430 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f854400eaa0 tx=0x7f854400ef60 comp rx=0 tx=0).stop 2024-04-02T13:50:43.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:43.239+0000 7f8551b3d640 1 -- 172.21.15.52:0/3825590440 shutdown_connections 2024-04-02T13:50:43.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:43.239+0000 7f8551b3d640 1 --2- 172.21.15.52:0/3825590440 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f85340a3c30 0x7f85340a60f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:43.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:43.239+0000 7f8551b3d640 1 --2- 172.21.15.52:0/3825590440 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f854c107800 0x7f854c19f430 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:43.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:43.239+0000 7f8551b3d640 1 --2- 172.21.15.52:0/3825590440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f854c104ed0 0x7f854c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:43.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:43.239+0000 7f8551b3d640 1 --2- 172.21.15.52:0/3825590440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f854c1025a0 0x7f854c197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:43.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:43.239+0000 7f8551b3d640 1 -- 172.21.15.52:0/3825590440 >> 172.21.15.52:0/3825590440 conn(0x7f854c0fbf90 msgr2=0x7f854c103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:43.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:43.239+0000 7f8551b3d640 1 -- 172.21.15.52:0/3825590440 shutdown_connections 2024-04-02T13:50:43.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:43.239+0000 7f8551b3d640 1 -- 172.21.15.52:0/3825590440 wait complete. 2024-04-02T13:50:43.419 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 02 13:50:43 smithi088 systemd[1]: Starting Ceph mds.h for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T13:50:43.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:43 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3825590440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:43.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:43 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3825590440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:43.812 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:43 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3825590440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:43.813 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 02 13:50:43 smithi088 podman[66334]: 2024-04-02 13:50:43.432660417 +0000 UTC m=+0.017744439 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:50:43.813 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 02 13:50:43 smithi088 podman[66334]: 2024-04-02 13:50:43.557904853 +0000 UTC m=+0.142988858 container create db0696c6a3761338e4df3142be444c25e55f75cfce89b076402d6daf086e02fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-h, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-02T13:50:44.169 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 02 13:50:43 smithi088 podman[66334]: 2024-04-02 13:50:43.810596347 +0000 UTC m=+0.395680353 container init db0696c6a3761338e4df3142be444c25e55f75cfce89b076402d6daf086e02fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-h, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-02T13:50:44.170 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 02 13:50:43 smithi088 podman[66334]: 2024-04-02 13:50:43.813361379 +0000 UTC m=+0.398445398 container start db0696c6a3761338e4df3142be444c25e55f75cfce89b076402d6daf086e02fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-h, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux ) 2024-04-02T13:50:44.170 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 02 13:50:43 smithi088 ceph-mds[66353]: -- 172.21.15.88:0/1312776869 <== mon.2 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558917cb9a40 con 0x558917c8cc00 2024-04-02T13:50:44.170 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 02 13:50:43 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-h[66349]: starting mds.h at 2024-04-02T13:50:44.170 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 02 13:50:43 smithi088 bash[66334]: db0696c6a3761338e4df3142be444c25e55f75cfce89b076402d6daf086e02fe 2024-04-02T13:50:44.170 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 02 13:50:43 smithi088 systemd[1]: Started Ceph mds.h for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4. 2024-04-02T13:50:44.749 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:44 smithi189 ceph-mon[36683]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:44.749 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:44 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:44.749 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:44 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:44.749 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:44 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:44.749 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:44 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:44.749 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:44 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:44.749 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:44 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:44.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:44 smithi052 ceph-mon[32104]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:44.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:44 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:44.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:44 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:44.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:44 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:44.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:44 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:44.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:44 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:44.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:44 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:44.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:44 smithi088 ceph-mon[36528]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:44.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:44 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:44.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:44 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:44.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:44 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:44.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:44 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:44.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:44 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:44.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:44 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:45.254 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:50:45.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.366+0000 7fea0271c640 1 -- 172.21.15.52:0/1949785958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9fc101c30 msgr2=0x7fe9fc1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:45.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.366+0000 7fea0271c640 1 --2- 172.21.15.52:0/1949785958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9fc101c30 0x7fe9fc1020b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009900 tx=0x7fe9e402bc30 comp rx=0 tx=0).stop 2024-04-02T13:50:45.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.366+0000 7fea0271c640 1 -- 172.21.15.52:0/1949785958 shutdown_connections 2024-04-02T13:50:45.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.366+0000 7fea0271c640 1 --2- 172.21.15.52:0/1949785958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9fc1025f0 0x7fe9fc108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:45.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.366+0000 7fea0271c640 1 --2- 172.21.15.52:0/1949785958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9fc101c30 0x7fe9fc1020b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:45.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.366+0000 7fea0271c640 1 --2- 172.21.15.52:0/1949785958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc100a40 0x7fe9fc100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:45.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.366+0000 7fea0271c640 1 -- 172.21.15.52:0/1949785958 >> 172.21.15.52:0/1949785958 conn(0x7fe9fc0fbfb0 msgr2=0x7fe9fc0fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:45.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.366+0000 7fea0271c640 1 -- 172.21.15.52:0/1949785958 shutdown_connections 2024-04-02T13:50:45.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.366+0000 7fea0271c640 1 -- 172.21.15.52:0/1949785958 wait complete. 2024-04-02T13:50:45.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.366+0000 7fea0271c640 1 Processor -- start 2024-04-02T13:50:45.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.366+0000 7fea0271c640 1 -- start start 2024-04-02T13:50:45.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.367+0000 7fea0271c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9fc100a40 0x7fe9fc197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:45.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.367+0000 7fea0271c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9fc101c30 0x7fe9fc1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:45.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.367+0000 7fea0271c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc1025f0 0x7fe9fc19f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:45.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.367+0000 7fea0271c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc10c6c0 con 0x7fe9fc101c30 2024-04-02T13:50:45.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.367+0000 7fea0271c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc10c540 con 0x7fe9fc1025f0 2024-04-02T13:50:45.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.367+0000 7fea0271c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc10c840 con 0x7fe9fc100a40 2024-04-02T13:50:45.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.367+0000 7fe9fb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9fc101c30 0x7fe9fc1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:45.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.367+0000 7fe9fb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9fc101c30 0x7fe9fc1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33050/0 (socket says 172.21.15.52:33050) 2024-04-02T13:50:45.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.367+0000 7fe9fb7fe640 1 -- 172.21.15.52:0/966127782 learned_addr learned my addr 172.21.15.52:0/966127782 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:45.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.367+0000 7fe9fbfff640 1 --2- 172.21.15.52:0/966127782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9fc100a40 0x7fe9fc197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:45.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.367+0000 7fea00cb9640 1 --2- 172.21.15.52:0/966127782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc1025f0 0x7fe9fc19f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:45.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.367+0000 7fe9fb7fe640 1 -- 172.21.15.52:0/966127782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9fc100a40 msgr2=0x7fe9fc197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:45.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.367+0000 7fe9fb7fe640 1 --2- 172.21.15.52:0/966127782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9fc100a40 0x7fe9fc197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:45.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.367+0000 7fe9fb7fe640 1 -- 172.21.15.52:0/966127782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc1025f0 msgr2=0x7fe9fc19f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:45.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.367+0000 7fe9fb7fe640 1 --2- 172.21.15.52:0/966127782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc1025f0 0x7fe9fc19f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:45.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.367+0000 7fe9fb7fe640 1 -- 172.21.15.52:0/966127782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9fc19fa80 con 0x7fe9fc101c30 2024-04-02T13:50:45.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.367+0000 7fe9fb7fe640 1 --2- 172.21.15.52:0/966127782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9fc101c30 0x7fe9fc1983a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009230 tx=0x7fe9e4005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:45.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.367+0000 7fe9f97fa640 1 -- 172.21.15.52:0/966127782 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4041070 con 0x7fe9fc101c30 2024-04-02T13:50:45.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.367+0000 7fe9f97fa640 1 -- 172.21.15.52:0/966127782 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9e40054e0 con 0x7fe9fc101c30 2024-04-02T13:50:45.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.367+0000 7fea0271c640 1 -- 172.21.15.52:0/966127782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9fc19fd10 con 0x7fe9fc101c30 2024-04-02T13:50:45.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.367+0000 7fe9f97fa640 1 -- 172.21.15.52:0/966127782 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4034440 con 0x7fe9fc101c30 2024-04-02T13:50:45.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.367+0000 7fea0271c640 1 -- 172.21.15.52:0/966127782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9fc1a01b0 con 0x7fe9fc101c30 2024-04-02T13:50:45.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.368+0000 7fea0271c640 1 -- 172.21.15.52:0/966127782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9fc1a03e0 con 0x7fe9fc101c30 2024-04-02T13:50:45.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.370+0000 7fe9f97fa640 1 -- 172.21.15.52:0/966127782 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe9e4044070 con 0x7fe9fc101c30 2024-04-02T13:50:45.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.370+0000 7fe9f97fa640 1 --2- 172.21.15.52:0/966127782 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe9e80a3bb0 0x7fe9e80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:45.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.370+0000 7fe9fbfff640 1 --2- 172.21.15.52:0/966127782 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe9e80a3bb0 0x7fe9e80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:45.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.370+0000 7fe9f97fa640 1 -- 172.21.15.52:0/966127782 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe9e40f2020 con 0x7fe9fc101c30 2024-04-02T13:50:45.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.371+0000 7fe9fbfff640 1 --2- 172.21.15.52:0/966127782 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe9e80a3bb0 0x7fe9e80a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc101a90 tx=0x7fe9ec00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:45.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.374+0000 7fe9f97fa640 1 -- 172.21.15.52:0/966127782 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe9e40beea0 con 0x7fe9fc101c30 2024-04-02T13:50:45.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:45 smithi052 ceph-mon[32104]: Deploying daemon mds.i on smithi189 2024-04-02T13:50:45.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:45 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] up:boot 2024-04-02T13:50:45.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:45 smithi052 ceph-mon[32104]: fsmap 8 up:standby 2024-04-02T13:50:45.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:45 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T13:50:45.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:45 smithi189 ceph-mon[36683]: Deploying daemon mds.i on smithi189 2024-04-02T13:50:45.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:45 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] up:boot 2024-04-02T13:50:45.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:45 smithi189 ceph-mon[36683]: fsmap 8 up:standby 2024-04-02T13:50:45.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:45 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T13:50:45.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.599+0000 7fea0271c640 1 -- 172.21.15.52:0/966127782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9fc1a0770 con 0x7fe9fc101c30 2024-04-02T13:50:45.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.600+0000 7fe9f97fa640 1 -- 172.21.15.52:0/966127782 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7165 (secure 0 0 0) 0x7fe9e40be7a0 con 0x7fe9fc101c30 2024-04-02T13:50:45.603 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:50:45.603 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-02T13:50:45.603 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 9 2024-04-02T13:50:45.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.602+0000 7fea0271c640 1 -- 172.21.15.52:0/966127782 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe9e80a3bb0 msgr2=0x7fe9e80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:45.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.602+0000 7fea0271c640 1 --2- 172.21.15.52:0/966127782 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe9e80a3bb0 0x7fe9e80a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc101a90 tx=0x7fe9ec00a2d0 comp rx=0 tx=0).stop 2024-04-02T13:50:45.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.602+0000 7fea0271c640 1 -- 172.21.15.52:0/966127782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9fc101c30 msgr2=0x7fe9fc1983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:45.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.602+0000 7fea0271c640 1 --2- 172.21.15.52:0/966127782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9fc101c30 0x7fe9fc1983a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009230 tx=0x7fe9e4005dd0 comp rx=0 tx=0).stop 2024-04-02T13:50:45.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.602+0000 7fea0271c640 1 -- 172.21.15.52:0/966127782 shutdown_connections 2024-04-02T13:50:45.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.602+0000 7fea0271c640 1 --2- 172.21.15.52:0/966127782 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe9e80a3bb0 0x7fe9e80a6070 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:45.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.602+0000 7fea0271c640 1 --2- 172.21.15.52:0/966127782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc1025f0 0x7fe9fc19f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:45.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.602+0000 7fea0271c640 1 --2- 172.21.15.52:0/966127782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9fc101c30 0x7fe9fc1983a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:45.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.602+0000 7fea0271c640 1 --2- 172.21.15.52:0/966127782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9fc100a40 0x7fe9fc197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:45.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.602+0000 7fea0271c640 1 -- 172.21.15.52:0/966127782 >> 172.21.15.52:0/966127782 conn(0x7fe9fc0fbfb0 msgr2=0x7fe9fc0fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:45.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.602+0000 7fea0271c640 1 -- 172.21.15.52:0/966127782 shutdown_connections 2024-04-02T13:50:45.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:45.602+0000 7fea0271c640 1 -- 172.21.15.52:0/966127782 wait complete. 2024-04-02T13:50:45.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:45 smithi088 ceph-mon[36528]: Deploying daemon mds.i on smithi189 2024-04-02T13:50:45.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:45 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] up:boot 2024-04-02T13:50:45.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:45 smithi088 ceph-mon[36528]: fsmap 8 up:standby 2024-04-02T13:50:45.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:45 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T13:50:46.288 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 02 13:50:46 smithi189 systemd[1]: Starting Ceph mds.i for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T13:50:46.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:46 smithi189 ceph-mon[36683]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:46.582 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:46 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/966127782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:46.583 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 02 13:50:46 smithi189 podman[67483]: 2024-04-02 13:50:46.207763566 +0000 UTC m=+0.017823257 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:50:46.583 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 02 13:50:46 smithi189 podman[67483]: 2024-04-02 13:50:46.314151223 +0000 UTC m=+0.124210897 container create ad384f043f4af8f88ad183060e5f3b8fcf81b84313bbcce02973f4cb455e0f01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-i, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-02T13:50:46.583 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 02 13:50:46 smithi189 podman[67483]: 2024-04-02 13:50:46.580224095 +0000 UTC m=+0.390283790 container init ad384f043f4af8f88ad183060e5f3b8fcf81b84313bbcce02973f4cb455e0f01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, GIT_CLEAN=True) 2024-04-02T13:50:46.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:46 smithi052 ceph-mon[32104]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:46.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:46 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/966127782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:46.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:46 smithi088 ceph-mon[36528]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:46.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:46 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/966127782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:47.038 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 02 13:50:46 smithi189 podman[67483]: 2024-04-02 13:50:46.582822913 +0000 UTC m=+0.392882587 container start ad384f043f4af8f88ad183060e5f3b8fcf81b84313bbcce02973f4cb455e0f01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-i, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:50:47.038 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 02 13:50:46 smithi189 ceph-mds[67503]: -- 172.21.15.189:0/2819305529 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558722565a40 con 0x558722539000 2024-04-02T13:50:47.038 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 02 13:50:46 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-i[67499]: starting mds.i at 2024-04-02T13:50:47.038 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 02 13:50:46 smithi189 bash[67483]: ad384f043f4af8f88ad183060e5f3b8fcf81b84313bbcce02973f4cb455e0f01 2024-04-02T13:50:47.038 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 02 13:50:46 smithi189 systemd[1]: Started Ceph mds.i for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4. 2024-04-02T13:50:47.617 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:50:47.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.721+0000 7fba4a8d3640 1 -- 172.21.15.52:0/1339914314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba44101b20 msgr2=0x7fba44101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:47.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.721+0000 7fba4a8d3640 1 --2- 172.21.15.52:0/1339914314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba44101b20 0x7fba44101f40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fba34009900 tx=0x7fba3402bc30 comp rx=0 tx=0).stop 2024-04-02T13:50:47.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.721+0000 7fba4a8d3640 1 -- 172.21.15.52:0/1339914314 shutdown_connections 2024-04-02T13:50:47.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.721+0000 7fba4a8d3640 1 --2- 172.21.15.52:0/1339914314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba441036d0 0x7fba44109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:47.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.721+0000 7fba4a8d3640 1 --2- 172.21.15.52:0/1339914314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba44102d10 0x7fba44103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:47.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.721+0000 7fba4a8d3640 1 --2- 172.21.15.52:0/1339914314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba44101b20 0x7fba44101f40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:47.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.721+0000 7fba4a8d3640 1 -- 172.21.15.52:0/1339914314 >> 172.21.15.52:0/1339914314 conn(0x7fba440fd0b0 msgr2=0x7fba440ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:47.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.721+0000 7fba4a8d3640 1 -- 172.21.15.52:0/1339914314 shutdown_connections 2024-04-02T13:50:47.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.721+0000 7fba4a8d3640 1 -- 172.21.15.52:0/1339914314 wait complete. 2024-04-02T13:50:47.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.722+0000 7fba4a8d3640 1 Processor -- start 2024-04-02T13:50:47.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.722+0000 7fba4a8d3640 1 -- start start 2024-04-02T13:50:47.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.722+0000 7fba4a8d3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba44101b20 0x7fba44197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:47.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.722+0000 7fba4a8d3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba44102d10 0x7fba44198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:47.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.722+0000 7fba4a8d3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba441036d0 0x7fba4419f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:47.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.722+0000 7fba4a8d3640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fba4410d7f0 con 0x7fba44101b20 2024-04-02T13:50:47.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.722+0000 7fba4a8d3640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fba4410d670 con 0x7fba44102d10 2024-04-02T13:50:47.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.722+0000 7fba4a8d3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fba4410d970 con 0x7fba441036d0 2024-04-02T13:50:47.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.722+0000 7fba43fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba44101b20 0x7fba44197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:47.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.722+0000 7fba43fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba44101b20 0x7fba44197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33072/0 (socket says 172.21.15.52:33072) 2024-04-02T13:50:47.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.722+0000 7fba43fff640 1 -- 172.21.15.52:0/3552946567 learned_addr learned my addr 172.21.15.52:0/3552946567 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:47.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.722+0000 7fba48e70640 1 --2- 172.21.15.52:0/3552946567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba441036d0 0x7fba4419f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:47.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.722+0000 7fba437fe640 1 --2- 172.21.15.52:0/3552946567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba44102d10 0x7fba44198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:47.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.722+0000 7fba43fff640 1 -- 172.21.15.52:0/3552946567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba441036d0 msgr2=0x7fba4419f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:47.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.722+0000 7fba43fff640 1 --2- 172.21.15.52:0/3552946567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba441036d0 0x7fba4419f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:47.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.722+0000 7fba43fff640 1 -- 172.21.15.52:0/3552946567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba44102d10 msgr2=0x7fba44198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:47.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.722+0000 7fba43fff640 1 --2- 172.21.15.52:0/3552946567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba44102d10 0x7fba44198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:47.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.722+0000 7fba43fff640 1 -- 172.21.15.52:0/3552946567 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba4419faf0 con 0x7fba44101b20 2024-04-02T13:50:47.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.722+0000 7fba437fe640 1 --2- 172.21.15.52:0/3552946567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba44102d10 0x7fba44198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:50:47.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.723+0000 7fba43fff640 1 --2- 172.21.15.52:0/3552946567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba44101b20 0x7fba44197ed0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fba34005ab0 tx=0x7fba34005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:47.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.723+0000 7fba417fa640 1 -- 172.21.15.52:0/3552946567 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba34041070 con 0x7fba44101b20 2024-04-02T13:50:47.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.723+0000 7fba417fa640 1 -- 172.21.15.52:0/3552946567 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba340054e0 con 0x7fba44101b20 2024-04-02T13:50:47.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.723+0000 7fba4a8d3640 1 -- 172.21.15.52:0/3552946567 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba4419fd80 con 0x7fba44101b20 2024-04-02T13:50:47.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.723+0000 7fba417fa640 1 -- 172.21.15.52:0/3552946567 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba34034440 con 0x7fba44101b20 2024-04-02T13:50:47.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.723+0000 7fba4a8d3640 1 -- 172.21.15.52:0/3552946567 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba441a01c0 con 0x7fba44101b20 2024-04-02T13:50:47.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.724+0000 7fba4a8d3640 1 -- 172.21.15.52:0/3552946567 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba44108ec0 con 0x7fba44101b20 2024-04-02T13:50:47.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.725+0000 7fba417fa640 1 -- 172.21.15.52:0/3552946567 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fba34044070 con 0x7fba44101b20 2024-04-02T13:50:47.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.725+0000 7fba417fa640 1 --2- 172.21.15.52:0/3552946567 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fba2c0a3bb0 0x7fba2c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:47.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.726+0000 7fba437fe640 1 --2- 172.21.15.52:0/3552946567 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fba2c0a3bb0 0x7fba2c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:47.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.726+0000 7fba417fa640 1 -- 172.21.15.52:0/3552946567 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fba340f1ef0 con 0x7fba44101b20 2024-04-02T13:50:47.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.726+0000 7fba437fe640 1 --2- 172.21.15.52:0/3552946567 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fba2c0a3bb0 0x7fba2c0a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fba441993e0 tx=0x7fba38008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:47.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.729+0000 7fba417fa640 1 -- 172.21.15.52:0/3552946567 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fba340becd0 con 0x7fba44101b20 2024-04-02T13:50:47.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.955+0000 7fba4a8d3640 1 -- 172.21.15.52:0/3552946567 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba4402d370 con 0x7fba44101b20 2024-04-02T13:50:47.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.956+0000 7fba417fa640 1 -- 172.21.15.52:0/3552946567 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7992 (secure 0 0 0) 0x7fba340be5d0 con 0x7fba44101b20 2024-04-02T13:50:47.959 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:50:47.959 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-02T13:50:47.960 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 10 2024-04-02T13:50:47.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.958+0000 7fba4a8d3640 1 -- 172.21.15.52:0/3552946567 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fba2c0a3bb0 msgr2=0x7fba2c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:47.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.958+0000 7fba4a8d3640 1 --2- 172.21.15.52:0/3552946567 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fba2c0a3bb0 0x7fba2c0a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fba441993e0 tx=0x7fba38008040 comp rx=0 tx=0).stop 2024-04-02T13:50:47.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.958+0000 7fba4a8d3640 1 -- 172.21.15.52:0/3552946567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba44101b20 msgr2=0x7fba44197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:47.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.958+0000 7fba4a8d3640 1 --2- 172.21.15.52:0/3552946567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba44101b20 0x7fba44197ed0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fba34005ab0 tx=0x7fba34005dc0 comp rx=0 tx=0).stop 2024-04-02T13:50:47.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.958+0000 7fba4a8d3640 1 -- 172.21.15.52:0/3552946567 shutdown_connections 2024-04-02T13:50:47.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.958+0000 7fba4a8d3640 1 --2- 172.21.15.52:0/3552946567 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fba2c0a3bb0 0x7fba2c0a6070 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:47.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.958+0000 7fba4a8d3640 1 --2- 172.21.15.52:0/3552946567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba441036d0 0x7fba4419f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:47.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.958+0000 7fba4a8d3640 1 --2- 172.21.15.52:0/3552946567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba44102d10 0x7fba44198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:47.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.958+0000 7fba4a8d3640 1 --2- 172.21.15.52:0/3552946567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba44101b20 0x7fba44197ed0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:47.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.958+0000 7fba4a8d3640 1 -- 172.21.15.52:0/3552946567 >> 172.21.15.52:0/3552946567 conn(0x7fba440fd0b0 msgr2=0x7fba44106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:47.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.958+0000 7fba4a8d3640 1 -- 172.21.15.52:0/3552946567 shutdown_connections 2024-04-02T13:50:47.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:47.958+0000 7fba4a8d3640 1 -- 172.21.15.52:0/3552946567 wait complete. 2024-04-02T13:50:47.971 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:47 smithi052 ceph-mon[32104]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:47.971 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:47 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:47.971 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:47 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:47.971 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:47 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:47.971 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:47 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:47.971 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:47 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:47.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:47 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:47.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:47 smithi052 ceph-mon[32104]: Deploying daemon mds.g on smithi052 2024-04-02T13:50:47.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:47 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] up:boot 2024-04-02T13:50:47.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:47 smithi052 ceph-mon[32104]: fsmap 9 up:standby 2024-04-02T13:50:47.972 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:47 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T13:50:48.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:47 smithi189 ceph-mon[36683]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:48.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:47 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:48.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:47 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:48.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:47 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:48.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:47 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:48.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:47 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:48.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:47 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:48.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:47 smithi189 ceph-mon[36683]: Deploying daemon mds.g on smithi052 2024-04-02T13:50:48.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:47 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] up:boot 2024-04-02T13:50:48.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:47 smithi189 ceph-mon[36683]: fsmap 9 up:standby 2024-04-02T13:50:48.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:47 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T13:50:48.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:47 smithi088 ceph-mon[36528]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:48.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:47 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:48.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:47 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:48.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:47 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:48.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:47 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:48.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:47 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:48.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:47 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:48.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:47 smithi088 ceph-mon[36528]: Deploying daemon mds.g on smithi052 2024-04-02T13:50:48.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:47 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] up:boot 2024-04-02T13:50:48.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:47 smithi088 ceph-mon[36528]: fsmap 9 up:standby 2024-04-02T13:50:48.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:47 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T13:50:49.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:48 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3552946567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:49.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:48 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3552946567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:49.090 INFO:journalctl@ceph.mds.g.smithi052.stdout:Apr 02 13:50:48 smithi052 systemd[1]: Starting Ceph mds.g for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T13:50:49.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:48 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3552946567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:49.475 INFO:journalctl@ceph.mds.g.smithi052.stdout:Apr 02 13:50:49 smithi052 podman[70972]: 2024-04-02 13:50:49.086227725 +0000 UTC m=+0.017374792 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:50:49.475 INFO:journalctl@ceph.mds.g.smithi052.stdout:Apr 02 13:50:49 smithi052 podman[70972]: 2024-04-02 13:50:49.220382537 +0000 UTC m=+0.151529649 container create 111cee9802159fd512aa896a7153636618f6a8545ecbed040746a9b19daf2d5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-g, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS) 2024-04-02T13:50:49.475 INFO:journalctl@ceph.mds.g.smithi052.stdout:Apr 02 13:50:49 smithi052 podman[70972]: 2024-04-02 13:50:49.473235919 +0000 UTC m=+0.404382987 container init 111cee9802159fd512aa896a7153636618f6a8545ecbed040746a9b19daf2d5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-g, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux ) 2024-04-02T13:50:49.839 INFO:journalctl@ceph.mds.g.smithi052.stdout:Apr 02 13:50:49 smithi052 podman[70972]: 2024-04-02 13:50:49.47595465 +0000 UTC m=+0.407101718 container start 111cee9802159fd512aa896a7153636618f6a8545ecbed040746a9b19daf2d5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-g, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-02T13:50:49.839 INFO:journalctl@ceph.mds.g.smithi052.stdout:Apr 02 13:50:49 smithi052 ceph-mds[70991]: -- 172.21.15.52:0/2410430203 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bc01a49a40 con 0x55bc01a1c800 2024-04-02T13:50:49.839 INFO:journalctl@ceph.mds.g.smithi052.stdout:Apr 02 13:50:49 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-g[70987]: starting mds.g at 2024-04-02T13:50:49.839 INFO:journalctl@ceph.mds.g.smithi052.stdout:Apr 02 13:50:49 smithi052 bash[70972]: 111cee9802159fd512aa896a7153636618f6a8545ecbed040746a9b19daf2d5d 2024-04-02T13:50:49.839 INFO:journalctl@ceph.mds.g.smithi052.stdout:Apr 02 13:50:49 smithi052 systemd[1]: Started Ceph mds.g for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4. 2024-04-02T13:50:49.974 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:50:50.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.078+0000 7f8e3ada8640 1 -- 172.21.15.52:0/1836700508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e34101c10 msgr2=0x7f8e34102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:50.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.078+0000 7f8e3ada8640 1 --2- 172.21.15.52:0/1836700508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e34101c10 0x7f8e34102090 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009900 tx=0x7f8e2002bc30 comp rx=0 tx=0).stop 2024-04-02T13:50:50.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.078+0000 7f8e3ada8640 1 -- 172.21.15.52:0/1836700508 shutdown_connections 2024-04-02T13:50:50.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.078+0000 7f8e3ada8640 1 --2- 172.21.15.52:0/1836700508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e341025d0 0x7f8e34108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:50.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.078+0000 7f8e3ada8640 1 --2- 172.21.15.52:0/1836700508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e34101c10 0x7f8e34102090 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:50.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.078+0000 7f8e3ada8640 1 --2- 172.21.15.52:0/1836700508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e34100a20 0x7f8e34100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:50.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.078+0000 7f8e3ada8640 1 -- 172.21.15.52:0/1836700508 >> 172.21.15.52:0/1836700508 conn(0x7f8e340fbf90 msgr2=0x7f8e340fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:50.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.078+0000 7f8e3ada8640 1 -- 172.21.15.52:0/1836700508 shutdown_connections 2024-04-02T13:50:50.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.078+0000 7f8e3ada8640 1 -- 172.21.15.52:0/1836700508 wait complete. 2024-04-02T13:50:50.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.078+0000 7f8e3ada8640 1 Processor -- start 2024-04-02T13:50:50.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.078+0000 7f8e3ada8640 1 -- start start 2024-04-02T13:50:50.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.078+0000 7f8e3ada8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e34100a20 0x7f8e34197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:50.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.078+0000 7f8e3ada8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e34101c10 0x7f8e34198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:50.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.078+0000 7f8e3ada8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e341025d0 0x7f8e3419f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:50.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.078+0000 7f8e3ada8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3410c680 con 0x7f8e34100a20 2024-04-02T13:50:50.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.078+0000 7f8e3ada8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3410c500 con 0x7f8e341025d0 2024-04-02T13:50:50.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.078+0000 7f8e3ada8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3410c800 con 0x7f8e34101c10 2024-04-02T13:50:50.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.079+0000 7f8e38b44640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e34100a20 0x7f8e34197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:50.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.079+0000 7f8e38b44640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e34100a20 0x7f8e34197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33106/0 (socket says 172.21.15.52:33106) 2024-04-02T13:50:50.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.079+0000 7f8e38b44640 1 -- 172.21.15.52:0/3413402221 learned_addr learned my addr 172.21.15.52:0/3413402221 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:50.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.079+0000 7f8e39345640 1 --2- 172.21.15.52:0/3413402221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e341025d0 0x7f8e3419f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:50.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.079+0000 7f8e33fff640 1 --2- 172.21.15.52:0/3413402221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e34101c10 0x7f8e34198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:50.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.079+0000 7f8e38b44640 1 -- 172.21.15.52:0/3413402221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e34101c10 msgr2=0x7f8e34198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:50.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.079+0000 7f8e38b44640 1 --2- 172.21.15.52:0/3413402221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e34101c10 0x7f8e34198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:50.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.079+0000 7f8e38b44640 1 -- 172.21.15.52:0/3413402221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e341025d0 msgr2=0x7f8e3419f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:50.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.079+0000 7f8e38b44640 1 --2- 172.21.15.52:0/3413402221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e341025d0 0x7f8e3419f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:50.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.079+0000 7f8e38b44640 1 -- 172.21.15.52:0/3413402221 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3419fa00 con 0x7f8e34100a20 2024-04-02T13:50:50.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.079+0000 7f8e33fff640 1 --2- 172.21.15.52:0/3413402221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e34101c10 0x7f8e34198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:50:50.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.079+0000 7f8e38b44640 1 --2- 172.21.15.52:0/3413402221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e34100a20 0x7f8e34197de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800c970 tx=0x7f8e2800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:50.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.079+0000 7f8e31ffb640 1 -- 172.21.15.52:0/3413402221 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e28007c10 con 0x7f8e34100a20 2024-04-02T13:50:50.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.079+0000 7f8e31ffb640 1 -- 172.21.15.52:0/3413402221 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e28007db0 con 0x7f8e34100a20 2024-04-02T13:50:50.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.079+0000 7f8e3ada8640 1 -- 172.21.15.52:0/3413402221 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e3419fcf0 con 0x7f8e34100a20 2024-04-02T13:50:50.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.079+0000 7f8e31ffb640 1 -- 172.21.15.52:0/3413402221 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2800f8a0 con 0x7f8e34100a20 2024-04-02T13:50:50.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.079+0000 7f8e3ada8640 1 -- 172.21.15.52:0/3413402221 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e341a0190 con 0x7f8e34100a20 2024-04-02T13:50:50.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.080+0000 7f8e3ada8640 1 -- 172.21.15.52:0/3413402221 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e341a0480 con 0x7f8e34100a20 2024-04-02T13:50:50.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.081+0000 7f8e31ffb640 1 -- 172.21.15.52:0/3413402221 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8e2800fa60 con 0x7f8e34100a20 2024-04-02T13:50:50.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.082+0000 7f8e31ffb640 1 --2- 172.21.15.52:0/3413402221 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f8e240a3bb0 0x7f8e240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:50.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.082+0000 7f8e31ffb640 1 -- 172.21.15.52:0/3413402221 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8e280d0280 con 0x7f8e34100a20 2024-04-02T13:50:50.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.082+0000 7f8e33fff640 1 --2- 172.21.15.52:0/3413402221 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f8e240a3bb0 0x7f8e240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:50.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.083+0000 7f8e33fff640 1 --2- 172.21.15.52:0/3413402221 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f8e240a3bb0 0x7f8e240a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8e20036040 tx=0x7f8e20000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:50.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.090+0000 7f8e31ffb640 1 -- 172.21.15.52:0/3413402221 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8e2809d190 con 0x7f8e34100a20 2024-04-02T13:50:50.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.333+0000 7f8e3ada8640 1 -- 172.21.15.52:0/3413402221 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e341a0650 con 0x7f8e34100a20 2024-04-02T13:50:50.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.334+0000 7f8e31ffb640 1 -- 172.21.15.52:0/3413402221 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8818 (secure 0 0 0) 0x7f8e2809ca90 con 0x7f8e34100a20 2024-04-02T13:50:50.337 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:50:50.337 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-02T13:50:50.338 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 11 2024-04-02T13:50:50.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.337+0000 7f8e3ada8640 1 -- 172.21.15.52:0/3413402221 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f8e240a3bb0 msgr2=0x7f8e240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:50.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.337+0000 7f8e3ada8640 1 --2- 172.21.15.52:0/3413402221 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f8e240a3bb0 0x7f8e240a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8e20036040 tx=0x7f8e20000bc0 comp rx=0 tx=0).stop 2024-04-02T13:50:50.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.337+0000 7f8e3ada8640 1 -- 172.21.15.52:0/3413402221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e34100a20 msgr2=0x7f8e34197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:50.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.337+0000 7f8e3ada8640 1 --2- 172.21.15.52:0/3413402221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e34100a20 0x7f8e34197de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800c970 tx=0x7f8e2800ce30 comp rx=0 tx=0).stop 2024-04-02T13:50:50.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.337+0000 7f8e3ada8640 1 -- 172.21.15.52:0/3413402221 shutdown_connections 2024-04-02T13:50:50.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.337+0000 7f8e3ada8640 1 --2- 172.21.15.52:0/3413402221 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f8e240a3bb0 0x7f8e240a6070 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:50.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.337+0000 7f8e3ada8640 1 --2- 172.21.15.52:0/3413402221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e341025d0 0x7f8e3419f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:50.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.337+0000 7f8e3ada8640 1 --2- 172.21.15.52:0/3413402221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e34101c10 0x7f8e34198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:50.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.337+0000 7f8e3ada8640 1 --2- 172.21.15.52:0/3413402221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e34100a20 0x7f8e34197de0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:50.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.337+0000 7f8e3ada8640 1 -- 172.21.15.52:0/3413402221 >> 172.21.15.52:0/3413402221 conn(0x7f8e340fbf90 msgr2=0x7f8e340fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:50.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.337+0000 7f8e3ada8640 1 -- 172.21.15.52:0/3413402221 shutdown_connections 2024-04-02T13:50:50.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:50.337+0000 7f8e3ada8640 1 -- 172.21.15.52:0/3413402221 wait complete. 2024-04-02T13:50:50.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:50 smithi052 ceph-mon[32104]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:50.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:50 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:50.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:50 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:50.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:50 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:50.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:50 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:50.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:50 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:50.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:50 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:50.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:50 smithi052 ceph-mon[32104]: Deploying daemon mds.a on smithi052 2024-04-02T13:50:50.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:50 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.52:6838/2213112521,v1:172.21.15.52:6839/2213112521] up:boot 2024-04-02T13:50:50.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:50 smithi052 ceph-mon[32104]: fsmap 10 up:standby 2024-04-02T13:50:50.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:50 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T13:50:50.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:50 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3413402221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:50.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:50 smithi088 ceph-mon[36528]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:50.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:50 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:50.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:50 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:50.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:50 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:50.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:50 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:50.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:50 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:50.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:50 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:50.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:50 smithi088 ceph-mon[36528]: Deploying daemon mds.a on smithi052 2024-04-02T13:50:50.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:50 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.52:6838/2213112521,v1:172.21.15.52:6839/2213112521] up:boot 2024-04-02T13:50:50.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:50 smithi088 ceph-mon[36528]: fsmap 10 up:standby 2024-04-02T13:50:50.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:50 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T13:50:50.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:50 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3413402221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:51.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:50 smithi189 ceph-mon[36683]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:51.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:50 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:51.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:50 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:51.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:50 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:51.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:50 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:51.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:50 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:51.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:50 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:51.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:50 smithi189 ceph-mon[36683]: Deploying daemon mds.a on smithi052 2024-04-02T13:50:51.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:50 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.52:6838/2213112521,v1:172.21.15.52:6839/2213112521] up:boot 2024-04-02T13:50:51.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:50 smithi189 ceph-mon[36683]: fsmap 10 up:standby 2024-04-02T13:50:51.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:50 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T13:50:51.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:50 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3413402221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:52.028 INFO:journalctl@ceph.mds.a.smithi052.stdout:Apr 02 13:50:51 smithi052 systemd[1]: Starting Ceph mds.a for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T13:50:52.291 INFO:journalctl@ceph.mds.a.smithi052.stdout:Apr 02 13:50:52 smithi052 podman[71327]: 2024-04-02 13:50:51.927820954 +0000 UTC m=+0.018255150 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:50:52.291 INFO:journalctl@ceph.mds.a.smithi052.stdout:Apr 02 13:50:52 smithi052 podman[71327]: 2024-04-02 13:50:52.044155661 +0000 UTC m=+0.134589837 container create c4bf472e5cd1e0f975093bad60c016da28e48deca3d36752d6fb22e8cda9bb32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-02T13:50:52.291 INFO:journalctl@ceph.mds.a.smithi052.stdout:Apr 02 13:50:52 smithi052 podman[71327]: 2024-04-02 13:50:52.288549358 +0000 UTC m=+0.378983531 container init c4bf472e5cd1e0f975093bad60c016da28e48deca3d36752d6fb22e8cda9bb32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True) 2024-04-02T13:50:52.352 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:50:52.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.459+0000 7f3f8c9e9640 1 -- 172.21.15.52:0/3981199959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f881025a0 msgr2=0x7f3f88104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:52.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.459+0000 7f3f8c9e9640 1 --2- 172.21.15.52:0/3981199959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f881025a0 0x7f3f88104990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3f78009900 tx=0x7f3f7802bc30 comp rx=0 tx=0).stop 2024-04-02T13:50:52.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.459+0000 7f3f8c9e9640 1 -- 172.21.15.52:0/3981199959 shutdown_connections 2024-04-02T13:50:52.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.459+0000 7f3f8c9e9640 1 --2- 172.21.15.52:0/3981199959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3f88107800 0x7f3f88109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:52.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.459+0000 7f3f8c9e9640 1 --2- 172.21.15.52:0/3981199959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f88104ed0 0x7f3f881072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:52.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.459+0000 7f3f8c9e9640 1 --2- 172.21.15.52:0/3981199959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f881025a0 0x7f3f88104990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:52.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.459+0000 7f3f8c9e9640 1 -- 172.21.15.52:0/3981199959 >> 172.21.15.52:0/3981199959 conn(0x7f3f880fbf90 msgr2=0x7f3f880fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:52.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.459+0000 7f3f8c9e9640 1 -- 172.21.15.52:0/3981199959 shutdown_connections 2024-04-02T13:50:52.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.459+0000 7f3f8c9e9640 1 -- 172.21.15.52:0/3981199959 wait complete. 2024-04-02T13:50:52.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.460+0000 7f3f8c9e9640 1 Processor -- start 2024-04-02T13:50:52.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.460+0000 7f3f8c9e9640 1 -- start start 2024-04-02T13:50:52.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.460+0000 7f3f8c9e9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f881025a0 0x7f3f88197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:52.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.460+0000 7f3f8c9e9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3f88104ed0 0x7f3f88198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:52.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.460+0000 7f3f8c9e9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f88107800 0x7f3f8819f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:52.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.460+0000 7f3f8c9e9640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8810d7a0 con 0x7f3f881025a0 2024-04-02T13:50:52.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.460+0000 7f3f8c9e9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8810d620 con 0x7f3f88107800 2024-04-02T13:50:52.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.460+0000 7f3f8c9e9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8810d920 con 0x7f3f88104ed0 2024-04-02T13:50:52.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.460+0000 7f3f8659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f881025a0 0x7f3f88197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:52.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.460+0000 7f3f8659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f881025a0 0x7f3f88197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52956/0 (socket says 172.21.15.52:52956) 2024-04-02T13:50:52.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.460+0000 7f3f8659c640 1 -- 172.21.15.52:0/1547534244 learned_addr learned my addr 172.21.15.52:0/1547534244 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:52.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.460+0000 7f3f86d9d640 1 --2- 172.21.15.52:0/1547534244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f88107800 0x7f3f8819f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:52.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.460+0000 7f3f8659c640 1 -- 172.21.15.52:0/1547534244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3f88104ed0 msgr2=0x7f3f88198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:52.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.460+0000 7f3f85d9b640 1 --2- 172.21.15.52:0/1547534244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3f88104ed0 0x7f3f88198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:52.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.460+0000 7f3f8659c640 1 --2- 172.21.15.52:0/1547534244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3f88104ed0 0x7f3f88198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:52.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.460+0000 7f3f8659c640 1 -- 172.21.15.52:0/1547534244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f88107800 msgr2=0x7f3f8819f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:52.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.460+0000 7f3f8659c640 1 --2- 172.21.15.52:0/1547534244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f88107800 0x7f3f8819f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:52.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.460+0000 7f3f8659c640 1 -- 172.21.15.52:0/1547534244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8819fa40 con 0x7f3f881025a0 2024-04-02T13:50:52.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.460+0000 7f3f86d9d640 1 --2- 172.21.15.52:0/1547534244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f88107800 0x7f3f8819f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:50:52.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.461+0000 7f3f8659c640 1 --2- 172.21.15.52:0/1547534244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f881025a0 0x7f3f88197e20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3f78009230 tx=0x7f3f78005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:52.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.461+0000 7f3f777fe640 1 -- 172.21.15.52:0/1547534244 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78041070 con 0x7f3f881025a0 2024-04-02T13:50:52.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.461+0000 7f3f8c9e9640 1 -- 172.21.15.52:0/1547534244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8819fcd0 con 0x7f3f881025a0 2024-04-02T13:50:52.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.461+0000 7f3f777fe640 1 -- 172.21.15.52:0/1547534244 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f780054e0 con 0x7f3f881025a0 2024-04-02T13:50:52.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.461+0000 7f3f777fe640 1 -- 172.21.15.52:0/1547534244 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78034440 con 0x7f3f881025a0 2024-04-02T13:50:52.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.461+0000 7f3f8c9e9640 1 -- 172.21.15.52:0/1547534244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f881a01c0 con 0x7f3f881025a0 2024-04-02T13:50:52.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.462+0000 7f3f8c9e9640 1 -- 172.21.15.52:0/1547534244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f880fda80 con 0x7f3f881025a0 2024-04-02T13:50:52.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.463+0000 7f3f777fe640 1 -- 172.21.15.52:0/1547534244 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3f78044070 con 0x7f3f881025a0 2024-04-02T13:50:52.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.463+0000 7f3f777fe640 1 --2- 172.21.15.52:0/1547534244 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3f700a3bb0 0x7f3f700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:52.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.464+0000 7f3f777fe640 1 -- 172.21.15.52:0/1547534244 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f3f780f1d30 con 0x7f3f881025a0 2024-04-02T13:50:52.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.464+0000 7f3f85d9b640 1 --2- 172.21.15.52:0/1547534244 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3f700a3bb0 0x7f3f700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:52.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.464+0000 7f3f85d9b640 1 --2- 172.21.15.52:0/1547534244 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3f700a3bb0 0x7f3f700a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3f88199330 tx=0x7f3f7c005ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:52.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.468+0000 7f3f777fe640 1 -- 172.21.15.52:0/1547534244 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3f780bebb0 con 0x7f3f881025a0 2024-04-02T13:50:52.589 INFO:journalctl@ceph.mds.a.smithi052.stdout:Apr 02 13:50:52 smithi052 podman[71327]: 2024-04-02 13:50:52.291414534 +0000 UTC m=+0.381848709 container start c4bf472e5cd1e0f975093bad60c016da28e48deca3d36752d6fb22e8cda9bb32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-02T13:50:52.589 INFO:journalctl@ceph.mds.a.smithi052.stdout:Apr 02 13:50:52 smithi052 ceph-mds[71346]: -- 172.21.15.52:0/1019669853 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5645aac09a40 con 0x5645aabdcc00 2024-04-02T13:50:52.589 INFO:journalctl@ceph.mds.a.smithi052.stdout:Apr 02 13:50:52 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-a[71342]: starting mds.a at 2024-04-02T13:50:52.589 INFO:journalctl@ceph.mds.a.smithi052.stdout:Apr 02 13:50:52 smithi052 bash[71327]: c4bf472e5cd1e0f975093bad60c016da28e48deca3d36752d6fb22e8cda9bb32 2024-04-02T13:50:52.589 INFO:journalctl@ceph.mds.a.smithi052.stdout:Apr 02 13:50:52 smithi052 systemd[1]: Started Ceph mds.a for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4. 2024-04-02T13:50:52.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.697+0000 7f3f8c9e9640 1 -- 172.21.15.52:0/1547534244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f8802d370 con 0x7f3f881025a0 2024-04-02T13:50:52.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.698+0000 7f3f777fe640 1 -- 172.21.15.52:0/1547534244 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9644 (secure 0 0 0) 0x7f3f780be4b0 con 0x7f3f881025a0 2024-04-02T13:50:52.701 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:50:52.701 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-02T13:50:52.701 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 12 2024-04-02T13:50:52.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.700+0000 7f3f8c9e9640 1 -- 172.21.15.52:0/1547534244 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3f700a3bb0 msgr2=0x7f3f700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:52.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.700+0000 7f3f8c9e9640 1 --2- 172.21.15.52:0/1547534244 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3f700a3bb0 0x7f3f700a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3f88199330 tx=0x7f3f7c005ca0 comp rx=0 tx=0).stop 2024-04-02T13:50:52.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.700+0000 7f3f8c9e9640 1 -- 172.21.15.52:0/1547534244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f881025a0 msgr2=0x7f3f88197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:52.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.700+0000 7f3f8c9e9640 1 --2- 172.21.15.52:0/1547534244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f881025a0 0x7f3f88197e20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3f78009230 tx=0x7f3f78005dd0 comp rx=0 tx=0).stop 2024-04-02T13:50:52.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.700+0000 7f3f8c9e9640 1 -- 172.21.15.52:0/1547534244 shutdown_connections 2024-04-02T13:50:52.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.700+0000 7f3f8c9e9640 1 --2- 172.21.15.52:0/1547534244 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3f700a3bb0 0x7f3f700a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:52.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.700+0000 7f3f8c9e9640 1 --2- 172.21.15.52:0/1547534244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f88107800 0x7f3f8819f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:52.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.700+0000 7f3f8c9e9640 1 --2- 172.21.15.52:0/1547534244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3f88104ed0 0x7f3f88198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:52.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.700+0000 7f3f8c9e9640 1 --2- 172.21.15.52:0/1547534244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f881025a0 0x7f3f88197e20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:52.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.700+0000 7f3f8c9e9640 1 -- 172.21.15.52:0/1547534244 >> 172.21.15.52:0/1547534244 conn(0x7f3f880fbf90 msgr2=0x7f3f88102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:52.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.700+0000 7f3f8c9e9640 1 -- 172.21.15.52:0/1547534244 shutdown_connections 2024-04-02T13:50:52.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:52.700+0000 7f3f8c9e9640 1 -- 172.21.15.52:0/1547534244 wait complete. 2024-04-02T13:50:52.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:52 smithi088 ceph-mon[36528]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:52.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:52 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:52.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:52 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:52.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:52 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:52.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:52 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:52.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:52 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:52.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:52 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:53.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:52 smithi189 ceph-mon[36683]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:53.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:52 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:53.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:52 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:53.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:52 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:53.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:52 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:53.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:52 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:53.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:52 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:53.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:52 smithi052 ceph-mon[32104]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:53.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:52 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:53.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:52 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:53.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:52 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:53.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:52 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:50:53.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:52 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:50:53.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:52 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:53.836 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:53 smithi088 ceph-mon[36528]: Deploying daemon mds.b on smithi088 2024-04-02T13:50:53.836 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:53 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.52:6840/2689882080,v1:172.21.15.52:6841/2689882080] up:boot 2024-04-02T13:50:53.837 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:53 smithi088 ceph-mon[36528]: fsmap 11 up:standby 2024-04-02T13:50:53.837 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:53 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T13:50:53.837 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:53 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1547534244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:54.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:53 smithi189 ceph-mon[36683]: Deploying daemon mds.b on smithi088 2024-04-02T13:50:54.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:53 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.52:6840/2689882080,v1:172.21.15.52:6841/2689882080] up:boot 2024-04-02T13:50:54.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:53 smithi189 ceph-mon[36683]: fsmap 11 up:standby 2024-04-02T13:50:54.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:53 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T13:50:54.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:53 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1547534244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:54.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:53 smithi052 ceph-mon[32104]: Deploying daemon mds.b on smithi088 2024-04-02T13:50:54.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:53 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.52:6840/2689882080,v1:172.21.15.52:6841/2689882080] up:boot 2024-04-02T13:50:54.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:53 smithi052 ceph-mon[32104]: fsmap 11 up:standby 2024-04-02T13:50:54.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:53 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T13:50:54.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:53 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1547534244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:54.715 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:50:54.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.828+0000 7f65cd0ba640 1 -- 172.21.15.52:0/4025302604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65c8101c90 msgr2=0x7f65c8102110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:54.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.828+0000 7f65cd0ba640 1 --2- 172.21.15.52:0/4025302604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65c8101c90 0x7f65c8102110 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009900 tx=0x7f65b002bc30 comp rx=0 tx=0).stop 2024-04-02T13:50:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.828+0000 7f65cd0ba640 1 -- 172.21.15.52:0/4025302604 shutdown_connections 2024-04-02T13:50:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.828+0000 7f65cd0ba640 1 --2- 172.21.15.52:0/4025302604 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65c8102650 0x7f65c8108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.828+0000 7f65cd0ba640 1 --2- 172.21.15.52:0/4025302604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65c8101c90 0x7f65c8102110 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.828+0000 7f65cd0ba640 1 --2- 172.21.15.52:0/4025302604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65c8100aa0 0x7f65c8100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.828+0000 7f65cd0ba640 1 -- 172.21.15.52:0/4025302604 >> 172.21.15.52:0/4025302604 conn(0x7f65c80fc030 msgr2=0x7f65c80fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.828+0000 7f65cd0ba640 1 -- 172.21.15.52:0/4025302604 shutdown_connections 2024-04-02T13:50:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.828+0000 7f65cd0ba640 1 -- 172.21.15.52:0/4025302604 wait complete. 2024-04-02T13:50:54.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.829+0000 7f65cd0ba640 1 Processor -- start 2024-04-02T13:50:54.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.829+0000 7f65cd0ba640 1 -- start start 2024-04-02T13:50:54.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.829+0000 7f65cd0ba640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65c8100aa0 0x7f65c806acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:54.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.829+0000 7f65cd0ba640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65c8101c90 0x7f65c819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.829+0000 7f65cd0ba640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65c8102650 0x7f65c819f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.829+0000 7f65cd0ba640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f65c810c6b0 con 0x7f65c8101c90 2024-04-02T13:50:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.829+0000 7f65cd0ba640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f65c810c530 con 0x7f65c8100aa0 2024-04-02T13:50:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.829+0000 7f65cd0ba640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f65c810c830 con 0x7f65c8102650 2024-04-02T13:50:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.829+0000 7f65bffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65c8101c90 0x7f65c819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.829+0000 7f65bffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65c8101c90 0x7f65c819cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52974/0 (socket says 172.21.15.52:52974) 2024-04-02T13:50:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.829+0000 7f65bffff640 1 -- 172.21.15.52:0/1863123623 learned_addr learned my addr 172.21.15.52:0/1863123623 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.829+0000 7f65c6d9d640 1 --2- 172.21.15.52:0/1863123623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65c8100aa0 0x7f65c806acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.829+0000 7f65c759e640 1 --2- 172.21.15.52:0/1863123623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65c8102650 0x7f65c819f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.829+0000 7f65bffff640 1 -- 172.21.15.52:0/1863123623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65c8102650 msgr2=0x7f65c819f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.829+0000 7f65bffff640 1 --2- 172.21.15.52:0/1863123623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65c8102650 0x7f65c819f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.829+0000 7f65bffff640 1 -- 172.21.15.52:0/1863123623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65c8100aa0 msgr2=0x7f65c806acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.829+0000 7f65bffff640 1 --2- 172.21.15.52:0/1863123623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65c8100aa0 0x7f65c806acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:54.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.829+0000 7f65bffff640 1 -- 172.21.15.52:0/1863123623 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c819fa80 con 0x7f65c8101c90 2024-04-02T13:50:54.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.829+0000 7f65c6d9d640 1 --2- 172.21.15.52:0/1863123623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65c8100aa0 0x7f65c806acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:50:54.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.830+0000 7f65bffff640 1 --2- 172.21.15.52:0/1863123623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65c8101c90 0x7f65c819cb80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009230 tx=0x7f65b002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:54.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.830+0000 7f65c4d99640 1 -- 172.21.15.52:0/1863123623 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0041070 con 0x7f65c8101c90 2024-04-02T13:50:54.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.830+0000 7f65c4d99640 1 -- 172.21.15.52:0/1863123623 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b0004450 con 0x7f65c8101c90 2024-04-02T13:50:54.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.830+0000 7f65cd0ba640 1 -- 172.21.15.52:0/1863123623 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c819fd10 con 0x7f65c8101c90 2024-04-02T13:50:54.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.830+0000 7f65c4d99640 1 -- 172.21.15.52:0/1863123623 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0003e20 con 0x7f65c8101c90 2024-04-02T13:50:54.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.830+0000 7f65cd0ba640 1 -- 172.21.15.52:0/1863123623 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c81a01b0 con 0x7f65c8101c90 2024-04-02T13:50:54.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.831+0000 7f65cd0ba640 1 -- 172.21.15.52:0/1863123623 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c8107dc0 con 0x7f65c8101c90 2024-04-02T13:50:54.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.832+0000 7f65c4d99640 1 -- 172.21.15.52:0/1863123623 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f65b0044070 con 0x7f65c8101c90 2024-04-02T13:50:54.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.833+0000 7f65c4d99640 1 --2- 172.21.15.52:0/1863123623 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f65b40a3bb0 0x7f65b40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:54.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.833+0000 7f65c6d9d640 1 --2- 172.21.15.52:0/1863123623 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f65b40a3bb0 0x7f65b40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:54.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.833+0000 7f65c4d99640 1 -- 172.21.15.52:0/1863123623 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f65b00f1b30 con 0x7f65c8101c90 2024-04-02T13:50:54.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.833+0000 7f65c6d9d640 1 --2- 172.21.15.52:0/1863123623 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f65b40a3bb0 0x7f65b40a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f65c8101af0 tx=0x7f65b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:54.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:54.837+0000 7f65c4d99640 1 -- 172.21.15.52:0/1863123623 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f65b00be9b0 con 0x7f65c8101c90 2024-04-02T13:50:54.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:54 smithi088 ceph-mon[36528]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:54.920 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 02 13:50:54 smithi088 systemd[1]: Starting Ceph mds.b for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T13:50:54.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:54 smithi052 ceph-mon[32104]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:55.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:54 smithi189 ceph-mon[36683]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:55.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:55.064+0000 7f65cd0ba640 1 -- 172.21.15.52:0/1863123623 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65c802d570 con 0x7f65c8101c90 2024-04-02T13:50:55.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:55.064+0000 7f65c4d99640 1 -- 172.21.15.52:0/1863123623 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9644 (secure 0 0 0) 0x7f65b00b8e20 con 0x7f65c8101c90 2024-04-02T13:50:55.067 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:50:55.068 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-02T13:50:55.068 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 12 2024-04-02T13:50:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:55.066+0000 7f65cd0ba640 1 -- 172.21.15.52:0/1863123623 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f65b40a3bb0 msgr2=0x7f65b40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:55.066+0000 7f65cd0ba640 1 --2- 172.21.15.52:0/1863123623 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f65b40a3bb0 0x7f65b40a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f65c8101af0 tx=0x7f65b8005f90 comp rx=0 tx=0).stop 2024-04-02T13:50:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:55.066+0000 7f65cd0ba640 1 -- 172.21.15.52:0/1863123623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65c8101c90 msgr2=0x7f65c819cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:55.066+0000 7f65cd0ba640 1 --2- 172.21.15.52:0/1863123623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65c8101c90 0x7f65c819cb80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009230 tx=0x7f65b002c650 comp rx=0 tx=0).stop 2024-04-02T13:50:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:55.066+0000 7f65cd0ba640 1 -- 172.21.15.52:0/1863123623 shutdown_connections 2024-04-02T13:50:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:55.066+0000 7f65cd0ba640 1 --2- 172.21.15.52:0/1863123623 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f65b40a3bb0 0x7f65b40a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:55.066+0000 7f65cd0ba640 1 --2- 172.21.15.52:0/1863123623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65c8102650 0x7f65c819f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:55.066+0000 7f65cd0ba640 1 --2- 172.21.15.52:0/1863123623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65c8101c90 0x7f65c819cb80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:55.066+0000 7f65cd0ba640 1 --2- 172.21.15.52:0/1863123623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65c8100aa0 0x7f65c806acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:55.066+0000 7f65cd0ba640 1 -- 172.21.15.52:0/1863123623 >> 172.21.15.52:0/1863123623 conn(0x7f65c80fc030 msgr2=0x7f65c8104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:55.066+0000 7f65cd0ba640 1 -- 172.21.15.52:0/1863123623 shutdown_connections 2024-04-02T13:50:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:55.066+0000 7f65cd0ba640 1 -- 172.21.15.52:0/1863123623 wait complete. 2024-04-02T13:50:55.226 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 02 13:50:54 smithi088 podman[66638]: 2024-04-02 13:50:54.970647596 +0000 UTC m=+0.114651398 container create 3189802bb9564724045d58037a77c863dc732ee697d952000a5dd65597afaf40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:50:55.226 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 02 13:50:54 smithi088 podman[66638]: 2024-04-02 13:50:54.873491705 +0000 UTC m=+0.017495515 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:50:55.226 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 02 13:50:55 smithi088 podman[66638]: 2024-04-02 13:50:55.198584504 +0000 UTC m=+0.342588318 container init 3189802bb9564724045d58037a77c863dc732ee697d952000a5dd65597afaf40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-02T13:50:55.226 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 02 13:50:55 smithi088 podman[66638]: 2024-04-02 13:50:55.201247867 +0000 UTC m=+0.345251679 container start 3189802bb9564724045d58037a77c863dc732ee697d952000a5dd65597afaf40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-b, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS) 2024-04-02T13:50:55.226 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 02 13:50:55 smithi088 ceph-mds[66657]: -- 172.21.15.88:0/3674267999 <== mon.2 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56453d33da40 con 0x56453d310c00 2024-04-02T13:50:55.645 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 02 13:50:55 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-b[66653]: starting mds.b at 2024-04-02T13:50:55.645 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 02 13:50:55 smithi088 bash[66638]: 3189802bb9564724045d58037a77c863dc732ee697d952000a5dd65597afaf40 2024-04-02T13:50:55.645 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 02 13:50:55 smithi088 systemd[1]: Started Ceph mds.b for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4. 2024-04-02T13:50:55.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:55 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1863123623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:50:56.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:55 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1863123623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:56.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:55 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:56.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:55 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:56.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:55 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:56.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:55 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:56.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:55 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:56.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:55 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:50:56.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:55 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1863123623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:56.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:55 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:56.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:55 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:56.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:55 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:56.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:55 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:56.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:55 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:56.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:55 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:50:56.678 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:56 smithi052 ceph-mon[32104]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:56.678 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:56 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] up:boot 2024-04-02T13:50:56.678 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:56 smithi052 ceph-mon[32104]: fsmap 12 up:standby 2024-04-02T13:50:56.678 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T13:50:56.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:56 smithi189 ceph-mon[36683]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:56.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:56 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] up:boot 2024-04-02T13:50:56.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:56 smithi189 ceph-mon[36683]: fsmap 12 up:standby 2024-04-02T13:50:56.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T13:50:56.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:56 smithi088 ceph-mon[36528]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:56.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:56 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] up:boot 2024-04-02T13:50:56.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:56 smithi088 ceph-mon[36528]: fsmap 12 up:standby 2024-04-02T13:50:56.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T13:50:57.081 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:50:57.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.203+0000 7f768947b640 1 -- 172.21.15.52:0/532769556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7684101c30 msgr2=0x7f76841020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:57.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.203+0000 7f768947b640 1 --2- 172.21.15.52:0/532769556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7684101c30 0x7f76841020b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7674009900 tx=0x7f767402bc30 comp rx=0 tx=0).stop 2024-04-02T13:50:57.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.203+0000 7f768947b640 1 -- 172.21.15.52:0/532769556 shutdown_connections 2024-04-02T13:50:57.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.203+0000 7f768947b640 1 --2- 172.21.15.52:0/532769556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76841025f0 0x7f7684108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:57.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.203+0000 7f768947b640 1 --2- 172.21.15.52:0/532769556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7684101c30 0x7f76841020b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:57.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.203+0000 7f768947b640 1 --2- 172.21.15.52:0/532769556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7684100a40 0x7f7684100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:57.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.203+0000 7f768947b640 1 -- 172.21.15.52:0/532769556 >> 172.21.15.52:0/532769556 conn(0x7f76840fbfb0 msgr2=0x7f76840fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:57.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.203+0000 7f768947b640 1 -- 172.21.15.52:0/532769556 shutdown_connections 2024-04-02T13:50:57.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.203+0000 7f768947b640 1 -- 172.21.15.52:0/532769556 wait complete. 2024-04-02T13:50:57.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.203+0000 7f768947b640 1 Processor -- start 2024-04-02T13:50:57.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.204+0000 7f768947b640 1 -- start start 2024-04-02T13:50:57.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.204+0000 7f768947b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7684100a40 0x7f76841049d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:57.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.204+0000 7f768947b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7684101c30 0x7f7684103010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:57.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.204+0000 7f768947b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76841025f0 0x7f7684103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:57.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.204+0000 7f768947b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f768410c6f0 con 0x7f76841025f0 2024-04-02T13:50:57.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.204+0000 7f768947b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f768410c570 con 0x7f7684100a40 2024-04-02T13:50:57.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.204+0000 7f768947b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f768410c870 con 0x7f7684101c30 2024-04-02T13:50:57.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.204+0000 7f76837fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76841025f0 0x7f7684103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:57.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.204+0000 7f76837fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76841025f0 0x7f7684103550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52980/0 (socket says 172.21.15.52:52980) 2024-04-02T13:50:57.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.204+0000 7f7682ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7684100a40 0x7f76841049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:57.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.204+0000 7f76837fe640 1 -- 172.21.15.52:0/417074851 learned_addr learned my addr 172.21.15.52:0/417074851 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:57.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.204+0000 7f76827fc640 1 --2- 172.21.15.52:0/417074851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7684101c30 0x7f7684103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:57.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.204+0000 7f7682ffd640 1 -- 172.21.15.52:0/417074851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7684101c30 msgr2=0x7f7684103010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:57.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.204+0000 7f7682ffd640 1 --2- 172.21.15.52:0/417074851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7684101c30 0x7f7684103010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:57.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.204+0000 7f7682ffd640 1 -- 172.21.15.52:0/417074851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76841025f0 msgr2=0x7f7684103550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:57.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.204+0000 7f7682ffd640 1 --2- 172.21.15.52:0/417074851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76841025f0 0x7f7684103550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:57.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.204+0000 7f7682ffd640 1 -- 172.21.15.52:0/417074851 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7684103de0 con 0x7f7684100a40 2024-04-02T13:50:57.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.204+0000 7f76837fe640 1 --2- 172.21.15.52:0/417074851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76841025f0 0x7f7684103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:50:57.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.204+0000 7f7682ffd640 1 --2- 172.21.15.52:0/417074851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7684100a40 0x7f76841049d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f767800d6a0 tx=0x7f767800db60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:57.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.205+0000 7f766bfff640 1 -- 172.21.15.52:0/417074851 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7678017070 con 0x7f7684100a40 2024-04-02T13:50:57.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.205+0000 7f768947b640 1 -- 172.21.15.52:0/417074851 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76841a1fb0 con 0x7f7684100a40 2024-04-02T13:50:57.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.205+0000 7f766bfff640 1 -- 172.21.15.52:0/417074851 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7678009d00 con 0x7f7684100a40 2024-04-02T13:50:57.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.205+0000 7f766bfff640 1 -- 172.21.15.52:0/417074851 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7678003c60 con 0x7f7684100a40 2024-04-02T13:50:57.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.205+0000 7f768947b640 1 -- 172.21.15.52:0/417074851 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76841a2410 con 0x7f7684100a40 2024-04-02T13:50:57.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.206+0000 7f768947b640 1 -- 172.21.15.52:0/417074851 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76841a2700 con 0x7f7684100a40 2024-04-02T13:50:57.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.207+0000 7f766bfff640 1 -- 172.21.15.52:0/417074851 <== mon.2 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7678009820 con 0x7f7684100a40 2024-04-02T13:50:57.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.207+0000 7f766bfff640 1 --2- 172.21.15.52:0/417074851 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f766c0a3bb0 0x7f766c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:57.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.207+0000 7f76827fc640 1 --2- 172.21.15.52:0/417074851 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f766c0a3bb0 0x7f766c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:57.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.207+0000 7f766bfff640 1 -- 172.21.15.52:0/417074851 <== mon.2 v2:172.21.15.88:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f76780cf970 con 0x7f7684100a40 2024-04-02T13:50:57.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.208+0000 7f76827fc640 1 --2- 172.21.15.52:0/417074851 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f766c0a3bb0 0x7f766c0a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f767402c240 tx=0x7f767402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:57.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.211+0000 7f766bfff640 1 -- 172.21.15.52:0/417074851 <== mon.2 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f767809c1a0 con 0x7f7684100a40 2024-04-02T13:50:57.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.487+0000 7f768947b640 1 -- 172.21.15.52:0/417074851 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76841a28d0 con 0x7f7684100a40 2024-04-02T13:50:57.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.488+0000 7f766bfff640 1 -- 172.21.15.52:0/417074851 <== mon.2 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10470 (secure 0 0 0) 0x7f76780a04a0 con 0x7f7684100a40 2024-04-02T13:50:57.491 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:50:57.491 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24503,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-02T13:50:57.492 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 13 2024-04-02T13:50:57.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.490+0000 7f768947b640 1 -- 172.21.15.52:0/417074851 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f766c0a3bb0 msgr2=0x7f766c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:57.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.490+0000 7f768947b640 1 --2- 172.21.15.52:0/417074851 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f766c0a3bb0 0x7f766c0a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f767402c240 tx=0x7f767402df90 comp rx=0 tx=0).stop 2024-04-02T13:50:57.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.490+0000 7f768947b640 1 -- 172.21.15.52:0/417074851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7684100a40 msgr2=0x7f76841049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:57.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.490+0000 7f768947b640 1 --2- 172.21.15.52:0/417074851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7684100a40 0x7f76841049d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f767800d6a0 tx=0x7f767800db60 comp rx=0 tx=0).stop 2024-04-02T13:50:57.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.490+0000 7f768947b640 1 -- 172.21.15.52:0/417074851 shutdown_connections 2024-04-02T13:50:57.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.490+0000 7f768947b640 1 --2- 172.21.15.52:0/417074851 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f766c0a3bb0 0x7f766c0a6070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:57.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.490+0000 7f768947b640 1 --2- 172.21.15.52:0/417074851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76841025f0 0x7f7684103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:57.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.490+0000 7f768947b640 1 --2- 172.21.15.52:0/417074851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7684101c30 0x7f7684103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:57.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.490+0000 7f768947b640 1 --2- 172.21.15.52:0/417074851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7684100a40 0x7f76841049d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:57.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.490+0000 7f768947b640 1 -- 172.21.15.52:0/417074851 >> 172.21.15.52:0/417074851 conn(0x7f76840fbfb0 msgr2=0x7f76840fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:57.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.490+0000 7f768947b640 1 -- 172.21.15.52:0/417074851 shutdown_connections 2024-04-02T13:50:57.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.490+0000 7f768947b640 1 -- 172.21.15.52:0/417074851 wait complete. 2024-04-02T13:50:57.505 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:50:57.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.607+0000 7f253e3af640 1 -- 172.21.15.52:0/4237350148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2538101c10 msgr2=0x7f2538102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:57.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.607+0000 7f253e3af640 1 --2- 172.21.15.52:0/4237350148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2538101c10 0x7f2538102090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f252c009900 tx=0x7f252c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:50:57.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.607+0000 7f253e3af640 1 -- 172.21.15.52:0/4237350148 shutdown_connections 2024-04-02T13:50:57.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.607+0000 7f253e3af640 1 --2- 172.21.15.52:0/4237350148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25381025d0 0x7f2538108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:57.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.607+0000 7f253e3af640 1 --2- 172.21.15.52:0/4237350148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2538101c10 0x7f2538102090 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:57.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.607+0000 7f253e3af640 1 --2- 172.21.15.52:0/4237350148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2538100a20 0x7f2538100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:57.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.607+0000 7f253e3af640 1 -- 172.21.15.52:0/4237350148 >> 172.21.15.52:0/4237350148 conn(0x7f25380fbfb0 msgr2=0x7f25380fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:57.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.607+0000 7f253e3af640 1 -- 172.21.15.52:0/4237350148 shutdown_connections 2024-04-02T13:50:57.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.608+0000 7f253e3af640 1 -- 172.21.15.52:0/4237350148 wait complete. 2024-04-02T13:50:57.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.608+0000 7f253e3af640 1 Processor -- start 2024-04-02T13:50:57.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.608+0000 7f253e3af640 1 -- start start 2024-04-02T13:50:57.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.609+0000 7f253e3af640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2538100a20 0x7f2538196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:57.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.609+0000 7f253e3af640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2538101c10 0x7f25381972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:57.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.609+0000 7f253e3af640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25381025d0 0x7f253819e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:57.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.609+0000 7f253e3af640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f253810c680 con 0x7f25381025d0 2024-04-02T13:50:57.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.609+0000 7f253e3af640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f253810c500 con 0x7f2538100a20 2024-04-02T13:50:57.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.609+0000 7f253e3af640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f253810c800 con 0x7f2538101c10 2024-04-02T13:50:57.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.609+0000 7f253c94c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25381025d0 0x7f253819e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:57.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.609+0000 7f2537fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2538100a20 0x7f2538196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:57.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.609+0000 7f253c94c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25381025d0 0x7f253819e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52990/0 (socket says 172.21.15.52:52990) 2024-04-02T13:50:57.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.609+0000 7f2537fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2538100a20 0x7f2538196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:41948/0 (socket says 172.21.15.52:41948) 2024-04-02T13:50:57.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.609+0000 7f253c94c640 1 -- 172.21.15.52:0/3087151138 learned_addr learned my addr 172.21.15.52:0/3087151138 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:57.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.609+0000 7f25377fe640 1 --2- 172.21.15.52:0/3087151138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2538101c10 0x7f25381972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:57.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.609+0000 7f253c94c640 1 -- 172.21.15.52:0/3087151138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2538101c10 msgr2=0x7f25381972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:57.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.609+0000 7f253c94c640 1 --2- 172.21.15.52:0/3087151138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2538101c10 0x7f25381972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:57.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.609+0000 7f253c94c640 1 -- 172.21.15.52:0/3087151138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2538100a20 msgr2=0x7f2538196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:57.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.609+0000 7f253c94c640 1 --2- 172.21.15.52:0/3087151138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2538100a20 0x7f2538196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:57.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.609+0000 7f253c94c640 1 -- 172.21.15.52:0/3087151138 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253819e9b0 con 0x7f25381025d0 2024-04-02T13:50:57.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.609+0000 7f2537fff640 1 --2- 172.21.15.52:0/3087151138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2538100a20 0x7f2538196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:50:57.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.609+0000 7f253c94c640 1 --2- 172.21.15.52:0/3087151138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25381025d0 0x7f253819e310 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2530009fd0 tx=0x7f253000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:57.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.610+0000 7f25357fa640 1 -- 172.21.15.52:0/3087151138 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2530017070 con 0x7f25381025d0 2024-04-02T13:50:57.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.610+0000 7f25357fa640 1 -- 172.21.15.52:0/3087151138 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2530003e50 con 0x7f25381025d0 2024-04-02T13:50:57.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.610+0000 7f253e3af640 1 -- 172.21.15.52:0/3087151138 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f253819eca0 con 0x7f25381025d0 2024-04-02T13:50:57.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.610+0000 7f25357fa640 1 -- 172.21.15.52:0/3087151138 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25300106c0 con 0x7f25381025d0 2024-04-02T13:50:57.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.610+0000 7f253e3af640 1 -- 172.21.15.52:0/3087151138 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f253819f1a0 con 0x7f25381025d0 2024-04-02T13:50:57.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.611+0000 7f253e3af640 1 -- 172.21.15.52:0/3087151138 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2538107dc0 con 0x7f25381025d0 2024-04-02T13:50:57.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.613+0000 7f25357fa640 1 -- 172.21.15.52:0/3087151138 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2530003970 con 0x7f25381025d0 2024-04-02T13:50:57.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.613+0000 7f25357fa640 1 --2- 172.21.15.52:0/3087151138 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f25200a3bb0 0x7f25200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:57.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.613+0000 7f2537fff640 1 --2- 172.21.15.52:0/3087151138 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f25200a3bb0 0x7f25200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:57.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.613+0000 7f25357fa640 1 -- 172.21.15.52:0/3087151138 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f25300cfce0 con 0x7f25381025d0 2024-04-02T13:50:57.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.614+0000 7f2537fff640 1 --2- 172.21.15.52:0/3087151138 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f25200a3bb0 0x7f25200a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2538198270 tx=0x7f2528005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:57.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.616+0000 7f25357fa640 1 -- 172.21.15.52:0/3087151138 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f253009c510 con 0x7f25381025d0 2024-04-02T13:50:57.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.806+0000 7f253e3af640 1 -- 172.21.15.52:0/3087151138 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f253802d540 con 0x7f25381025d0 2024-04-02T13:50:57.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.808+0000 7f25357fa640 1 -- 172.21.15.52:0/3087151138 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18405 (secure 0 0 0) 0x7f25300a0810 con 0x7f25381025d0 2024-04-02T13:50:57.811 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:50:57.811 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":82,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T13:50:12.112448+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:50:57.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.810+0000 7f253e3af640 1 -- 172.21.15.52:0/3087151138 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f25200a3bb0 msgr2=0x7f25200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:57.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.810+0000 7f253e3af640 1 --2- 172.21.15.52:0/3087151138 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f25200a3bb0 0x7f25200a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2538198270 tx=0x7f2528005f90 comp rx=0 tx=0).stop 2024-04-02T13:50:57.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.810+0000 7f253e3af640 1 -- 172.21.15.52:0/3087151138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25381025d0 msgr2=0x7f253819e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:57.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.810+0000 7f253e3af640 1 --2- 172.21.15.52:0/3087151138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25381025d0 0x7f253819e310 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2530009fd0 tx=0x7f253000b260 comp rx=0 tx=0).stop 2024-04-02T13:50:57.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.810+0000 7f253e3af640 1 -- 172.21.15.52:0/3087151138 shutdown_connections 2024-04-02T13:50:57.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.810+0000 7f253e3af640 1 --2- 172.21.15.52:0/3087151138 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f25200a3bb0 0x7f25200a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:57.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.810+0000 7f253e3af640 1 --2- 172.21.15.52:0/3087151138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25381025d0 0x7f253819e310 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:57.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.810+0000 7f253e3af640 1 --2- 172.21.15.52:0/3087151138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2538101c10 0x7f25381972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:57.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.810+0000 7f253e3af640 1 --2- 172.21.15.52:0/3087151138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2538100a20 0x7f2538196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:57.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.810+0000 7f253e3af640 1 -- 172.21.15.52:0/3087151138 >> 172.21.15.52:0/3087151138 conn(0x7f25380fbfb0 msgr2=0x7f2538104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:57.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.810+0000 7f253e3af640 1 -- 172.21.15.52:0/3087151138 shutdown_connections 2024-04-02T13:50:57.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.810+0000 7f253e3af640 1 -- 172.21.15.52:0/3087151138 wait complete. 2024-04-02T13:50:57.824 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:46:20.746053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T13:50:57.825 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:50:57.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.928+0000 7fc197606640 1 -- 172.21.15.52:0/535533415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc190100a00 msgr2=0x7fc190100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:57.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.928+0000 7fc197606640 1 --2- 172.21.15.52:0/535533415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc190100a00 0x7fc190100e20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc188009900 tx=0x7fc18802bc30 comp rx=0 tx=0).stop 2024-04-02T13:50:57.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.929+0000 7fc197606640 1 -- 172.21.15.52:0/535533415 shutdown_connections 2024-04-02T13:50:57.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.929+0000 7fc197606640 1 --2- 172.21.15.52:0/535533415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1901025b0 0x7fc190108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:57.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.929+0000 7fc197606640 1 --2- 172.21.15.52:0/535533415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc190101bf0 0x7fc190102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:57.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.929+0000 7fc197606640 1 --2- 172.21.15.52:0/535533415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc190100a00 0x7fc190100e20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:57.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.929+0000 7fc197606640 1 -- 172.21.15.52:0/535533415 >> 172.21.15.52:0/535533415 conn(0x7fc1900fbf90 msgr2=0x7fc1900fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:57.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.929+0000 7fc197606640 1 -- 172.21.15.52:0/535533415 shutdown_connections 2024-04-02T13:50:57.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.929+0000 7fc197606640 1 -- 172.21.15.52:0/535533415 wait complete. 2024-04-02T13:50:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.929+0000 7fc197606640 1 Processor -- start 2024-04-02T13:50:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.929+0000 7fc197606640 1 -- start start 2024-04-02T13:50:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.929+0000 7fc197606640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc190100a00 0x7fc190104b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.929+0000 7fc197606640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc190101bf0 0x7fc190103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.929+0000 7fc197606640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1901025b0 0x7fc1901036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.929+0000 7fc197606640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc19010c670 con 0x7fc190101bf0 2024-04-02T13:50:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.929+0000 7fc197606640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc19010c4f0 con 0x7fc1901025b0 2024-04-02T13:50:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.929+0000 7fc197606640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc19010c7f0 con 0x7fc190100a00 2024-04-02T13:50:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.929+0000 7fc194ba1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc190101bf0 0x7fc190103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.929+0000 7fc194ba1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc190101bf0 0x7fc190103170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52998/0 (socket says 172.21.15.52:52998) 2024-04-02T13:50:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.930+0000 7fc194ba1640 1 -- 172.21.15.52:0/2262695093 learned_addr learned my addr 172.21.15.52:0/2262695093 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.930+0000 7fc1953a2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc190100a00 0x7fc190104b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.930+0000 7fc195ba3640 1 --2- 172.21.15.52:0/2262695093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1901025b0 0x7fc1901036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:57.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.930+0000 7fc194ba1640 1 -- 172.21.15.52:0/2262695093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc190100a00 msgr2=0x7fc190104b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:57.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.930+0000 7fc194ba1640 1 --2- 172.21.15.52:0/2262695093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc190100a00 0x7fc190104b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:57.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.930+0000 7fc194ba1640 1 -- 172.21.15.52:0/2262695093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1901025b0 msgr2=0x7fc1901036b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:57.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.930+0000 7fc194ba1640 1 --2- 172.21.15.52:0/2262695093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1901025b0 0x7fc1901036b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:57.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.930+0000 7fc194ba1640 1 -- 172.21.15.52:0/2262695093 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc190103f40 con 0x7fc190101bf0 2024-04-02T13:50:57.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.930+0000 7fc1953a2640 1 --2- 172.21.15.52:0/2262695093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc190100a00 0x7fc190104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:50:57.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.930+0000 7fc195ba3640 1 --2- 172.21.15.52:0/2262695093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1901025b0 0x7fc1901036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:50:57.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.930+0000 7fc194ba1640 1 --2- 172.21.15.52:0/2262695093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc190101bf0 0x7fc190103170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc18000e9f0 tx=0x7fc18000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:57.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.930+0000 7fc1867fc640 1 -- 172.21.15.52:0/2262695093 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18000cd80 con 0x7fc190101bf0 2024-04-02T13:50:57.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.930+0000 7fc1867fc640 1 -- 172.21.15.52:0/2262695093 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc180005500 con 0x7fc190101bf0 2024-04-02T13:50:57.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.930+0000 7fc197606640 1 -- 172.21.15.52:0/2262695093 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1901a1f70 con 0x7fc190101bf0 2024-04-02T13:50:57.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.930+0000 7fc1867fc640 1 -- 172.21.15.52:0/2262695093 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180010690 con 0x7fc190101bf0 2024-04-02T13:50:57.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.930+0000 7fc197606640 1 -- 172.21.15.52:0/2262695093 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1901a23d0 con 0x7fc190101bf0 2024-04-02T13:50:57.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.931+0000 7fc197606640 1 -- 172.21.15.52:0/2262695093 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1901a26c0 con 0x7fc190101bf0 2024-04-02T13:50:57.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.932+0000 7fc1867fc640 1 -- 172.21.15.52:0/2262695093 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc1800046c0 con 0x7fc190101bf0 2024-04-02T13:50:57.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.933+0000 7fc1867fc640 1 --2- 172.21.15.52:0/2262695093 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc17c0a3c30 0x7fc17c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:57.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.933+0000 7fc1953a2640 1 --2- 172.21.15.52:0/2262695093 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc17c0a3c30 0x7fc17c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:57.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.933+0000 7fc1867fc640 1 -- 172.21.15.52:0/2262695093 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc1800d0270 con 0x7fc190101bf0 2024-04-02T13:50:57.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.933+0000 7fc1953a2640 1 --2- 172.21.15.52:0/2262695093 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc17c0a3c30 0x7fc17c0a60f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc188000c00 tx=0x7fc188005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:57.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:57.938+0000 7fc1867fc640 1 -- 172.21.15.52:0/2262695093 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc18009d0f0 con 0x7fc190101bf0 2024-04-02T13:50:58.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.137+0000 7fc197606640 1 -- 172.21.15.52:0/2262695093 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc1901a2a60 con 0x7fc190101bf0 2024-04-02T13:50:58.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.137+0000 7fc1867fc640 1 -- 172.21.15.52:0/2262695093 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fc18009c9f0 con 0x7fc190101bf0 2024-04-02T13:50:58.140 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-04-02T13:50:58.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.139+0000 7fc197606640 1 -- 172.21.15.52:0/2262695093 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc17c0a3c30 msgr2=0x7fc17c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:58.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.139+0000 7fc197606640 1 --2- 172.21.15.52:0/2262695093 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc17c0a3c30 0x7fc17c0a60f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc188000c00 tx=0x7fc188005fd0 comp rx=0 tx=0).stop 2024-04-02T13:50:58.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.139+0000 7fc197606640 1 -- 172.21.15.52:0/2262695093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc190101bf0 msgr2=0x7fc190103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:58.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.139+0000 7fc197606640 1 --2- 172.21.15.52:0/2262695093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc190101bf0 0x7fc190103170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc18000e9f0 tx=0x7fc18000edf0 comp rx=0 tx=0).stop 2024-04-02T13:50:58.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.139+0000 7fc197606640 1 -- 172.21.15.52:0/2262695093 shutdown_connections 2024-04-02T13:50:58.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.139+0000 7fc197606640 1 --2- 172.21.15.52:0/2262695093 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc17c0a3c30 0x7fc17c0a60f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:58.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.139+0000 7fc197606640 1 --2- 172.21.15.52:0/2262695093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1901025b0 0x7fc1901036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:58.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.139+0000 7fc197606640 1 --2- 172.21.15.52:0/2262695093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc190101bf0 0x7fc190103170 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:58.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.139+0000 7fc197606640 1 --2- 172.21.15.52:0/2262695093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc190100a00 0x7fc190104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:58.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.139+0000 7fc197606640 1 -- 172.21.15.52:0/2262695093 >> 172.21.15.52:0/2262695093 conn(0x7fc1900fbf90 msgr2=0x7fc190068c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:58.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.140+0000 7fc197606640 1 -- 172.21.15.52:0/2262695093 shutdown_connections 2024-04-02T13:50:58.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.140+0000 7fc197606640 1 -- 172.21.15.52:0/2262695093 wait complete. 2024-04-02T13:50:58.154 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-02T13:50:58.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.255+0000 7f354f12b640 1 -- 172.21.15.52:0/1354107449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3548100a20 msgr2=0x7f3548100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:58.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.255+0000 7f354f12b640 1 --2- 172.21.15.52:0/1354107449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3548100a20 0x7f3548100e40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3540009ca0 tx=0x7f354002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:50:58.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.255+0000 7f354f12b640 1 -- 172.21.15.52:0/1354107449 shutdown_connections 2024-04-02T13:50:58.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.255+0000 7f354f12b640 1 --2- 172.21.15.52:0/1354107449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35481025d0 0x7f3548108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:58.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.255+0000 7f354f12b640 1 --2- 172.21.15.52:0/1354107449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3548101c10 0x7f3548102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:58.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.255+0000 7f354f12b640 1 --2- 172.21.15.52:0/1354107449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3548100a20 0x7f3548100e40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:58.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.255+0000 7f354f12b640 1 -- 172.21.15.52:0/1354107449 >> 172.21.15.52:0/1354107449 conn(0x7f35480fbf90 msgr2=0x7f35480fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:58.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.255+0000 7f354f12b640 1 -- 172.21.15.52:0/1354107449 shutdown_connections 2024-04-02T13:50:58.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.255+0000 7f354f12b640 1 -- 172.21.15.52:0/1354107449 wait complete. 2024-04-02T13:50:58.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.256+0000 7f354f12b640 1 Processor -- start 2024-04-02T13:50:58.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.256+0000 7f354f12b640 1 -- start start 2024-04-02T13:50:58.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.256+0000 7f354f12b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3548100a20 0x7f3548196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:58.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.256+0000 7f354f12b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3548101c10 0x7f3548197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:58.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.256+0000 7f354f12b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35481025d0 0x7f354819e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:58.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.256+0000 7f354f12b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f354810c680 con 0x7f3548101c10 2024-04-02T13:50:58.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.256+0000 7f354f12b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f354810c500 con 0x7f3548100a20 2024-04-02T13:50:58.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.256+0000 7f354f12b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f354810c800 con 0x7f35481025d0 2024-04-02T13:50:58.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.256+0000 7f3547fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3548101c10 0x7f3548197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:58.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.256+0000 7f3547fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3548101c10 0x7f3548197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53020/0 (socket says 172.21.15.52:53020) 2024-04-02T13:50:58.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.256+0000 7f354cec7640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3548100a20 0x7f3548196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:58.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.256+0000 7f3547fff640 1 -- 172.21.15.52:0/2501105733 learned_addr learned my addr 172.21.15.52:0/2501105733 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:58.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.256+0000 7f354d6c8640 1 --2- 172.21.15.52:0/2501105733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35481025d0 0x7f354819e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:58.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.257+0000 7f354cec7640 1 -- 172.21.15.52:0/2501105733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35481025d0 msgr2=0x7f354819e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:58.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.257+0000 7f354cec7640 1 --2- 172.21.15.52:0/2501105733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35481025d0 0x7f354819e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:58.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.257+0000 7f354cec7640 1 -- 172.21.15.52:0/2501105733 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3548101c10 msgr2=0x7f3548197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:58.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.257+0000 7f354cec7640 1 --2- 172.21.15.52:0/2501105733 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3548101c10 0x7f3548197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:58.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.257+0000 7f354cec7640 1 -- 172.21.15.52:0/2501105733 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354819e950 con 0x7f3548100a20 2024-04-02T13:50:58.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.257+0000 7f3547fff640 1 --2- 172.21.15.52:0/2501105733 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3548101c10 0x7f3548197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:50:58.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.257+0000 7f354cec7640 1 --2- 172.21.15.52:0/2501105733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3548100a20 0x7f3548196d40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f354002c630 tx=0x7f3540032f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:58.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.257+0000 7f3545ffb640 1 -- 172.21.15.52:0/2501105733 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3540039040 con 0x7f3548100a20 2024-04-02T13:50:58.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.257+0000 7f3545ffb640 1 -- 172.21.15.52:0/2501105733 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3540035c20 con 0x7f3548100a20 2024-04-02T13:50:58.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.257+0000 7f354f12b640 1 -- 172.21.15.52:0/2501105733 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354819ebe0 con 0x7f3548100a20 2024-04-02T13:50:58.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.257+0000 7f3545ffb640 1 -- 172.21.15.52:0/2501105733 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3540004280 con 0x7f3548100a20 2024-04-02T13:50:58.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.257+0000 7f354f12b640 1 -- 172.21.15.52:0/2501105733 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f354819f0d0 con 0x7f3548100a20 2024-04-02T13:50:58.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.258+0000 7f354f12b640 1 -- 172.21.15.52:0/2501105733 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3548107d40 con 0x7f3548100a20 2024-04-02T13:50:58.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.259+0000 7f3545ffb640 1 -- 172.21.15.52:0/2501105733 <== mon.2 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3540046030 con 0x7f3548100a20 2024-04-02T13:50:58.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.259+0000 7f3545ffb640 1 --2- 172.21.15.52:0/2501105733 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f35300a3bb0 0x7f35300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:58.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.260+0000 7f3547fff640 1 --2- 172.21.15.52:0/2501105733 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f35300a3bb0 0x7f35300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:58.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.260+0000 7f3545ffb640 1 -- 172.21.15.52:0/2501105733 <== mon.2 v2:172.21.15.88:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f35400f1960 con 0x7f3548100a20 2024-04-02T13:50:58.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.260+0000 7f3547fff640 1 --2- 172.21.15.52:0/2501105733 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f35300a3bb0 0x7f35300a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f353c0104e0 tx=0x7f353c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:58.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.264+0000 7f3545ffb640 1 -- 172.21.15.52:0/2501105733 <== mon.2 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f354004d060 con 0x7f3548100a20 2024-04-02T13:50:58.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.454+0000 7f354f12b640 1 -- 172.21.15.52:0/2501105733 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f354802d5c0 con 0x7f3548100a20 2024-04-02T13:50:58.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.455+0000 7f3545ffb640 1 -- 172.21.15.52:0/2501105733 <== mon.2 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f35400be870 con 0x7f3548100a20 2024-04-02T13:50:58.458 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:50:58.459 INFO:teuthology.orchestra.run.smithi052.stdout:[ 2024-04-02T13:50:58.459 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T13:50:58.459 INFO:teuthology.orchestra.run.smithi052.stdout: "poolnum": 1, 2024-04-02T13:50:58.459 INFO:teuthology.orchestra.run.smithi052.stdout: "poolname": ".mgr" 2024-04-02T13:50:58.459 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:50:58.459 INFO:teuthology.orchestra.run.smithi052.stdout:] 2024-04-02T13:50:58.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.457+0000 7f354f12b640 1 -- 172.21.15.52:0/2501105733 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f35300a3bb0 msgr2=0x7f35300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:58.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.457+0000 7f354f12b640 1 --2- 172.21.15.52:0/2501105733 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f35300a3bb0 0x7f35300a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f353c0104e0 tx=0x7f353c00b410 comp rx=0 tx=0).stop 2024-04-02T13:50:58.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.457+0000 7f354f12b640 1 -- 172.21.15.52:0/2501105733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3548100a20 msgr2=0x7f3548196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:58.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.457+0000 7f354f12b640 1 --2- 172.21.15.52:0/2501105733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3548100a20 0x7f3548196d40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f354002c630 tx=0x7f3540032f20 comp rx=0 tx=0).stop 2024-04-02T13:50:58.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.457+0000 7f354f12b640 1 -- 172.21.15.52:0/2501105733 shutdown_connections 2024-04-02T13:50:58.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.457+0000 7f354f12b640 1 --2- 172.21.15.52:0/2501105733 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f35300a3bb0 0x7f35300a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:58.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.457+0000 7f354f12b640 1 --2- 172.21.15.52:0/2501105733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35481025d0 0x7f354819e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:58.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.457+0000 7f354f12b640 1 --2- 172.21.15.52:0/2501105733 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3548101c10 0x7f3548197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:58.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.457+0000 7f354f12b640 1 --2- 172.21.15.52:0/2501105733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3548100a20 0x7f3548196d40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:58.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.457+0000 7f354f12b640 1 -- 172.21.15.52:0/2501105733 >> 172.21.15.52:0/2501105733 conn(0x7f35480fbf90 msgr2=0x7f3548104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:58.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.457+0000 7f354f12b640 1 -- 172.21.15.52:0/2501105733 shutdown_connections 2024-04-02T13:50:58.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.457+0000 7f354f12b640 1 -- 172.21.15.52:0/2501105733 wait complete. 2024-04-02T13:50:58.471 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:58 smithi052 ceph-mon[32104]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:58.471 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:58 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:58.471 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:58 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:58.471 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:58 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:58.471 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:58 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:58.471 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:58 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:58.471 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:58 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:58.471 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:58 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:58.471 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:58 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:50:58.471 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:58 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:58.471 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:58 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/417074851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:58.472 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:58 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:50:58.472 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:58 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3087151138' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:50:58.472 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:58 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2262695093' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:50:58.473 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-02T13:50:58.473 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-02T13:50:58.510 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:58 smithi088 ceph-mon[36528]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:58.510 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:58 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:58.510 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:58 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:58.511 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:58 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:58.511 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:58 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:58.511 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:58 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:58.511 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:58 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:58.511 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:58 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:58.511 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:58 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:50:58.511 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:58 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:58.511 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:58 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/417074851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:58.511 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:58 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:50:58.511 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:58 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3087151138' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:50:58.511 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:58 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2262695093' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:50:58.518 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:58 smithi189 ceph-mon[36683]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:50:58.518 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:58 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:58.518 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:58 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:58.518 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:58 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:58.518 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:58 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:58.518 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:58 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:58.518 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:58 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:58.518 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:58 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:58.518 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:58 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:50:58.518 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:58 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:50:58.518 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:58 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/417074851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:58.518 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:58 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:50:58.518 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:58 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3087151138' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:50:58.518 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:58 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2262695093' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:50:58.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.579+0000 7fb99b9bc640 1 -- 172.21.15.52:0/2165013962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb994102d20 msgr2=0x7fb9941031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:58.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.579+0000 7fb99b9bc640 1 --2- 172.21.15.52:0/2165013962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb994102d20 0x7fb9941031a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb984009900 tx=0x7fb98402bc30 comp rx=0 tx=0).stop 2024-04-02T13:50:58.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.579+0000 7fb99b9bc640 1 -- 172.21.15.52:0/2165013962 shutdown_connections 2024-04-02T13:50:58.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.579+0000 7fb99b9bc640 1 --2- 172.21.15.52:0/2165013962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9941036e0 0x7fb994109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:58.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.579+0000 7fb99b9bc640 1 --2- 172.21.15.52:0/2165013962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb994102d20 0x7fb9941031a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:58.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.579+0000 7fb99b9bc640 1 --2- 172.21.15.52:0/2165013962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb994101b30 0x7fb994101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:58.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.579+0000 7fb99b9bc640 1 -- 172.21.15.52:0/2165013962 >> 172.21.15.52:0/2165013962 conn(0x7fb9940fd0a0 msgr2=0x7fb9940ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:58.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.579+0000 7fb99b9bc640 1 -- 172.21.15.52:0/2165013962 shutdown_connections 2024-04-02T13:50:58.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.579+0000 7fb99b9bc640 1 -- 172.21.15.52:0/2165013962 wait complete. 2024-04-02T13:50:58.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.580+0000 7fb99b9bc640 1 Processor -- start 2024-04-02T13:50:58.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.580+0000 7fb99b9bc640 1 -- start start 2024-04-02T13:50:58.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.580+0000 7fb99b9bc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb994101b30 0x7fb994197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:58.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.580+0000 7fb99b9bc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb994102d20 0x7fb994198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:58.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.580+0000 7fb99b9bc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9941036e0 0x7fb99419f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:58.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.580+0000 7fb99b9bc640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb99410d7b0 con 0x7fb994102d20 2024-04-02T13:50:58.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.580+0000 7fb99b9bc640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb99410d630 con 0x7fb9941036e0 2024-04-02T13:50:58.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.580+0000 7fb99b9bc640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb99410d930 con 0x7fb994101b30 2024-04-02T13:50:58.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.580+0000 7fb998f57640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb994102d20 0x7fb994198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:58.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.580+0000 7fb998f57640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb994102d20 0x7fb994198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53036/0 (socket says 172.21.15.52:53036) 2024-04-02T13:50:58.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.580+0000 7fb998f57640 1 -- 172.21.15.52:0/3822786089 learned_addr learned my addr 172.21.15.52:0/3822786089 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:58.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.580+0000 7fb999f59640 1 --2- 172.21.15.52:0/3822786089 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9941036e0 0x7fb99419f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:58.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.580+0000 7fb999758640 1 --2- 172.21.15.52:0/3822786089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb994101b30 0x7fb994197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:58.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.580+0000 7fb998f57640 1 -- 172.21.15.52:0/3822786089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb994101b30 msgr2=0x7fb994197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:58.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.580+0000 7fb998f57640 1 --2- 172.21.15.52:0/3822786089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb994101b30 0x7fb994197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:58.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.580+0000 7fb998f57640 1 -- 172.21.15.52:0/3822786089 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9941036e0 msgr2=0x7fb99419f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:58.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.580+0000 7fb998f57640 1 --2- 172.21.15.52:0/3822786089 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9941036e0 0x7fb99419f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:58.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.580+0000 7fb998f57640 1 -- 172.21.15.52:0/3822786089 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb99419fa50 con 0x7fb994102d20 2024-04-02T13:50:58.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.581+0000 7fb998f57640 1 --2- 172.21.15.52:0/3822786089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb994102d20 0x7fb994198370 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb984009230 tx=0x7fb984005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:58.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.581+0000 7fb98a7fc640 1 -- 172.21.15.52:0/3822786089 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb984041070 con 0x7fb994102d20 2024-04-02T13:50:58.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.581+0000 7fb98a7fc640 1 -- 172.21.15.52:0/3822786089 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9840054e0 con 0x7fb994102d20 2024-04-02T13:50:58.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.581+0000 7fb99b9bc640 1 -- 172.21.15.52:0/3822786089 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb99419fce0 con 0x7fb994102d20 2024-04-02T13:50:58.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.581+0000 7fb98a7fc640 1 -- 172.21.15.52:0/3822786089 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb984034440 con 0x7fb994102d20 2024-04-02T13:50:58.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.581+0000 7fb99b9bc640 1 -- 172.21.15.52:0/3822786089 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9941a01d0 con 0x7fb994102d20 2024-04-02T13:50:58.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.582+0000 7fb99b9bc640 1 -- 172.21.15.52:0/3822786089 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb994108e50 con 0x7fb994102d20 2024-04-02T13:50:58.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.583+0000 7fb98a7fc640 1 -- 172.21.15.52:0/3822786089 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb984044070 con 0x7fb994102d20 2024-04-02T13:50:58.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.584+0000 7fb98a7fc640 1 --2- 172.21.15.52:0/3822786089 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb9800a3c30 0x7fb9800a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:58.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.584+0000 7fb999758640 1 --2- 172.21.15.52:0/3822786089 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb9800a3c30 0x7fb9800a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:58.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.584+0000 7fb98a7fc640 1 -- 172.21.15.52:0/3822786089 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb9840f2040 con 0x7fb994102d20 2024-04-02T13:50:58.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.584+0000 7fb999758640 1 --2- 172.21.15.52:0/3822786089 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb9800a3c30 0x7fb9800a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb990005d00 tx=0x7fb990005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:58.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.587+0000 7fb98a7fc640 1 -- 172.21.15.52:0/3822786089 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb9840bee20 con 0x7fb994102d20 2024-04-02T13:50:58.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:58.774+0000 7fb99b9bc640 1 -- 172.21.15.52:0/3822786089 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb99402d590 con 0x7fb994102d20 2024-04-02T13:50:59.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.218+0000 7fb98a7fc640 1 -- 172.21.15.52:0/3822786089 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7fb9840be720 con 0x7fb994102d20 2024-04-02T13:50:59.221 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_metadata' created 2024-04-02T13:50:59.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.221+0000 7fb99b9bc640 1 -- 172.21.15.52:0/3822786089 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb9800a3c30 msgr2=0x7fb9800a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:59.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.221+0000 7fb99b9bc640 1 --2- 172.21.15.52:0/3822786089 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb9800a3c30 0x7fb9800a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb990005d00 tx=0x7fb990005c50 comp rx=0 tx=0).stop 2024-04-02T13:50:59.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.221+0000 7fb99b9bc640 1 -- 172.21.15.52:0/3822786089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb994102d20 msgr2=0x7fb994198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:59.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.221+0000 7fb99b9bc640 1 --2- 172.21.15.52:0/3822786089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb994102d20 0x7fb994198370 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb984009230 tx=0x7fb984005dd0 comp rx=0 tx=0).stop 2024-04-02T13:50:59.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.222+0000 7fb99b9bc640 1 -- 172.21.15.52:0/3822786089 shutdown_connections 2024-04-02T13:50:59.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.222+0000 7fb99b9bc640 1 --2- 172.21.15.52:0/3822786089 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb9800a3c30 0x7fb9800a60f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:59.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.222+0000 7fb99b9bc640 1 --2- 172.21.15.52:0/3822786089 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9941036e0 0x7fb99419f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:59.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.222+0000 7fb99b9bc640 1 --2- 172.21.15.52:0/3822786089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb994102d20 0x7fb994198370 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:59.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.222+0000 7fb99b9bc640 1 --2- 172.21.15.52:0/3822786089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb994101b30 0x7fb994197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:59.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.222+0000 7fb99b9bc640 1 -- 172.21.15.52:0/3822786089 >> 172.21.15.52:0/3822786089 conn(0x7fb9940fd0a0 msgr2=0x7fb994106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:59.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.222+0000 7fb99b9bc640 1 -- 172.21.15.52:0/3822786089 shutdown_connections 2024-04-02T13:50:59.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.222+0000 7fb99b9bc640 1 -- 172.21.15.52:0/3822786089 wait complete. 2024-04-02T13:50:59.240 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-02T13:50:59.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.358+0000 7f70ccb1c640 1 -- 172.21.15.52:0/253020102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70c8107840 msgr2=0x7f70c8109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:59.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.358+0000 7f70ccb1c640 1 --2- 172.21.15.52:0/253020102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70c8107840 0x7f70c8109c30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009ca0 tx=0x7f70c002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:50:59.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.358+0000 7f70ccb1c640 1 -- 172.21.15.52:0/253020102 shutdown_connections 2024-04-02T13:50:59.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.358+0000 7f70ccb1c640 1 --2- 172.21.15.52:0/253020102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70c8107840 0x7f70c8109c30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:59.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.358+0000 7f70ccb1c640 1 --2- 172.21.15.52:0/253020102 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70c8104f10 0x7f70c8107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:59.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.358+0000 7f70ccb1c640 1 --2- 172.21.15.52:0/253020102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70c81025e0 0x7f70c81049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:59.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.358+0000 7f70ccb1c640 1 -- 172.21.15.52:0/253020102 >> 172.21.15.52:0/253020102 conn(0x7f70c80fbf90 msgr2=0x7f70c80fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:59.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.358+0000 7f70ccb1c640 1 -- 172.21.15.52:0/253020102 shutdown_connections 2024-04-02T13:50:59.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.358+0000 7f70ccb1c640 1 -- 172.21.15.52:0/253020102 wait complete. 2024-04-02T13:50:59.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.359+0000 7f70ccb1c640 1 Processor -- start 2024-04-02T13:50:59.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.359+0000 7f70ccb1c640 1 -- start start 2024-04-02T13:50:59.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.359+0000 7f70ccb1c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70c81025e0 0x7f70c8197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:59.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.359+0000 7f70ccb1c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70c8104f10 0x7f70c8198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:59.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.359+0000 7f70ccb1c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70c8107840 0x7f70c819f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:59.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.359+0000 7f70ccb1c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f70c810d7a0 con 0x7f70c8107840 2024-04-02T13:50:59.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.359+0000 7f70ccb1c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f70c810d620 con 0x7f70c81025e0 2024-04-02T13:50:59.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.359+0000 7f70ccb1c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f70c810d920 con 0x7f70c8104f10 2024-04-02T13:50:59.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.359+0000 7f70c6d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70c8107840 0x7f70c819f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:59.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.360+0000 7f70c6d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70c8107840 0x7f70c819f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53058/0 (socket says 172.21.15.52:53058) 2024-04-02T13:50:59.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.360+0000 7f70c6d9d640 1 -- 172.21.15.52:0/343378864 learned_addr learned my addr 172.21.15.52:0/343378864 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:50:59.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.360+0000 7f70c659c640 1 --2- 172.21.15.52:0/343378864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70c81025e0 0x7f70c8197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:59.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.360+0000 7f70bdd9b640 1 --2- 172.21.15.52:0/343378864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70c8104f10 0x7f70c8198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:59.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.360+0000 7f70c6d9d640 1 -- 172.21.15.52:0/343378864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70c8104f10 msgr2=0x7f70c8198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:59.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.360+0000 7f70c6d9d640 1 --2- 172.21.15.52:0/343378864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70c8104f10 0x7f70c8198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:59.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.360+0000 7f70c6d9d640 1 -- 172.21.15.52:0/343378864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70c81025e0 msgr2=0x7f70c8197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:59.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.360+0000 7f70c6d9d640 1 --2- 172.21.15.52:0/343378864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70c81025e0 0x7f70c8197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:59.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.360+0000 7f70c6d9d640 1 -- 172.21.15.52:0/343378864 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c819fa40 con 0x7f70c8107840 2024-04-02T13:50:59.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.360+0000 7f70c6d9d640 1 --2- 172.21.15.52:0/343378864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70c8107840 0x7f70c819f3d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f70c0037040 tx=0x7f70c0032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:59.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.360+0000 7f70bffff640 1 -- 172.21.15.52:0/343378864 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c0041020 con 0x7f70c8107840 2024-04-02T13:50:59.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.360+0000 7f70bffff640 1 -- 172.21.15.52:0/343378864 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70c0034d90 con 0x7f70c8107840 2024-04-02T13:50:59.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.360+0000 7f70ccb1c640 1 -- 172.21.15.52:0/343378864 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c819fcd0 con 0x7f70c8107840 2024-04-02T13:50:59.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.360+0000 7f70bffff640 1 -- 172.21.15.52:0/343378864 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c0039040 con 0x7f70c8107840 2024-04-02T13:50:59.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.360+0000 7f70ccb1c640 1 -- 172.21.15.52:0/343378864 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c81a01c0 con 0x7f70c8107840 2024-04-02T13:50:59.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.361+0000 7f70ccb1c640 1 -- 172.21.15.52:0/343378864 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70c80fda80 con 0x7f70c8107840 2024-04-02T13:50:59.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.363+0000 7f70bffff640 1 -- 172.21.15.52:0/343378864 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f70c0044020 con 0x7f70c8107840 2024-04-02T13:50:59.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.364+0000 7f70bffff640 1 --2- 172.21.15.52:0/343378864 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f70b40a3bb0 0x7f70b40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:59.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.364+0000 7f70c659c640 1 --2- 172.21.15.52:0/343378864 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f70b40a3bb0 0x7f70b40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:59.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.364+0000 7f70bffff640 1 -- 172.21.15.52:0/343378864 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f70c00f2440 con 0x7f70c8107840 2024-04-02T13:50:59.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.364+0000 7f70c659c640 1 --2- 172.21.15.52:0/343378864 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f70b40a3bb0 0x7f70b40a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009bd0 tx=0x7f70b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:59.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.367+0000 7f70bffff640 1 -- 172.21.15.52:0/343378864 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f70c004b020 con 0x7f70c8107840 2024-04-02T13:50:59.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:59 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2501105733' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-02T13:50:59.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:50:59 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3822786089' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-02T13:50:59.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:50:59.557+0000 7f70ccb1c640 1 -- 172.21.15.52:0/343378864 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f70c802d370 con 0x7f70c8107840 2024-04-02T13:50:59.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:59 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2501105733' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-02T13:50:59.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:50:59 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3822786089' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-02T13:50:59.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:59 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2501105733' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-02T13:50:59.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:50:59 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3822786089' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-02T13:51:00.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.220+0000 7f70bffff640 1 -- 172.21.15.52:0/343378864 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) ==== 171+0+0 (secure 0 0 0) 0x7f70c00bf0b0 con 0x7f70c8107840 2024-04-02T13:51:00.223 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data' created 2024-04-02T13:51:00.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.224+0000 7f70ccb1c640 1 -- 172.21.15.52:0/343378864 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f70b40a3bb0 msgr2=0x7f70b40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:00.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.224+0000 7f70ccb1c640 1 --2- 172.21.15.52:0/343378864 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f70b40a3bb0 0x7f70b40a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009bd0 tx=0x7f70b8005f90 comp rx=0 tx=0).stop 2024-04-02T13:51:00.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.224+0000 7f70ccb1c640 1 -- 172.21.15.52:0/343378864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70c8107840 msgr2=0x7f70c819f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:00.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.224+0000 7f70ccb1c640 1 --2- 172.21.15.52:0/343378864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70c8107840 0x7f70c819f3d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f70c0037040 tx=0x7f70c0032f20 comp rx=0 tx=0).stop 2024-04-02T13:51:00.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.226+0000 7f70ccb1c640 1 -- 172.21.15.52:0/343378864 shutdown_connections 2024-04-02T13:51:00.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.226+0000 7f70ccb1c640 1 --2- 172.21.15.52:0/343378864 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f70b40a3bb0 0x7f70b40a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:00.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.226+0000 7f70ccb1c640 1 --2- 172.21.15.52:0/343378864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70c8107840 0x7f70c819f3d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:00.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.226+0000 7f70ccb1c640 1 --2- 172.21.15.52:0/343378864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70c8104f10 0x7f70c8198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:00.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.226+0000 7f70ccb1c640 1 --2- 172.21.15.52:0/343378864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70c81025e0 0x7f70c8197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:00.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.226+0000 7f70ccb1c640 1 -- 172.21.15.52:0/343378864 >> 172.21.15.52:0/343378864 conn(0x7f70c80fbf90 msgr2=0x7f70c8102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:00.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.226+0000 7f70ccb1c640 1 -- 172.21.15.52:0/343378864 shutdown_connections 2024-04-02T13:51:00.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.226+0000 7f70ccb1c640 1 -- 172.21.15.52:0/343378864 wait complete. 2024-04-02T13:51:00.254 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-02T13:51:00.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.368+0000 7f2c5f8fb640 1 -- 172.21.15.52:0/1115301573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c581036d0 msgr2=0x7f2c58109f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:00.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.368+0000 7f2c5f8fb640 1 --2- 172.21.15.52:0/1115301573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c581036d0 0x7f2c58109f60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009ca0 tx=0x7f2c5002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:51:00.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.368+0000 7f2c5f8fb640 1 -- 172.21.15.52:0/1115301573 shutdown_connections 2024-04-02T13:51:00.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.368+0000 7f2c5f8fb640 1 --2- 172.21.15.52:0/1115301573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c581036d0 0x7f2c58109f60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:00.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.368+0000 7f2c5f8fb640 1 --2- 172.21.15.52:0/1115301573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c58102d10 0x7f2c58103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:00.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.368+0000 7f2c5f8fb640 1 --2- 172.21.15.52:0/1115301573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c58101b20 0x7f2c58101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:00.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.368+0000 7f2c5f8fb640 1 -- 172.21.15.52:0/1115301573 >> 172.21.15.52:0/1115301573 conn(0x7f2c580fd070 msgr2=0x7f2c580ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:00.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.368+0000 7f2c5f8fb640 1 -- 172.21.15.52:0/1115301573 shutdown_connections 2024-04-02T13:51:00.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.368+0000 7f2c5f8fb640 1 -- 172.21.15.52:0/1115301573 wait complete. 2024-04-02T13:51:00.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.369+0000 7f2c5f8fb640 1 Processor -- start 2024-04-02T13:51:00.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.369+0000 7f2c5f8fb640 1 -- start start 2024-04-02T13:51:00.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.369+0000 7f2c5f8fb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c58101b20 0x7f2c58197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:00.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.369+0000 7f2c5f8fb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c58102d10 0x7f2c58198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:00.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.369+0000 7f2c5f8fb640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c581036d0 0x7f2c5819f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:00.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.369+0000 7f2c5f8fb640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5810d780 con 0x7f2c58101b20 2024-04-02T13:51:00.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.369+0000 7f2c5f8fb640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5810d600 con 0x7f2c581036d0 2024-04-02T13:51:00.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.369+0000 7f2c5f8fb640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5810d900 con 0x7f2c58102d10 2024-04-02T13:51:00.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.369+0000 7f2c5d697640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c58101b20 0x7f2c58197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:00.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.369+0000 7f2c5d697640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c58101b20 0x7f2c58197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53086/0 (socket says 172.21.15.52:53086) 2024-04-02T13:51:00.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.369+0000 7f2c5d697640 1 -- 172.21.15.52:0/1788462746 learned_addr learned my addr 172.21.15.52:0/1788462746 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:00.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.369+0000 7f2c5de98640 1 --2- 172.21.15.52:0/1788462746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c581036d0 0x7f2c5819f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:00.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.369+0000 7f2c5d697640 1 -- 172.21.15.52:0/1788462746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c58102d10 msgr2=0x7f2c58198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:00.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.369+0000 7f2c5d697640 1 --2- 172.21.15.52:0/1788462746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c58102d10 0x7f2c58198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:00.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.369+0000 7f2c5d697640 1 -- 172.21.15.52:0/1788462746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c581036d0 msgr2=0x7f2c5819f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:00.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.369+0000 7f2c5d697640 1 --2- 172.21.15.52:0/1788462746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c581036d0 0x7f2c5819f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:00.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.369+0000 7f2c5d697640 1 -- 172.21.15.52:0/1788462746 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c5819f9f0 con 0x7f2c58101b20 2024-04-02T13:51:00.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.369+0000 7f2c5d697640 1 --2- 172.21.15.52:0/1788462746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c58101b20 0x7f2c58197dd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2c5400d770 tx=0x7f2c5400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:00.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.370+0000 7f2c4e7fc640 1 -- 172.21.15.52:0/1788462746 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c54017070 con 0x7f2c58101b20 2024-04-02T13:51:00.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.370+0000 7f2c4e7fc640 1 -- 172.21.15.52:0/1788462746 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c54009c30 con 0x7f2c58101b20 2024-04-02T13:51:00.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.370+0000 7f2c5f8fb640 1 -- 172.21.15.52:0/1788462746 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c5819fce0 con 0x7f2c58101b20 2024-04-02T13:51:00.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.370+0000 7f2c4e7fc640 1 -- 172.21.15.52:0/1788462746 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c54003dd0 con 0x7f2c58101b20 2024-04-02T13:51:00.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.370+0000 7f2c5f8fb640 1 -- 172.21.15.52:0/1788462746 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c581a0220 con 0x7f2c58101b20 2024-04-02T13:51:00.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.371+0000 7f2c5f8fb640 1 -- 172.21.15.52:0/1788462746 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c58108e40 con 0x7f2c58101b20 2024-04-02T13:51:00.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.372+0000 7f2c4e7fc640 1 -- 172.21.15.52:0/1788462746 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2c54003f90 con 0x7f2c58101b20 2024-04-02T13:51:00.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.372+0000 7f2c4e7fc640 1 --2- 172.21.15.52:0/1788462746 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2c440a3c30 0x7f2c440a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:00.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.373+0000 7f2c5ce96640 1 --2- 172.21.15.52:0/1788462746 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2c440a3c30 0x7f2c440a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:00.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.373+0000 7f2c4e7fc640 1 -- 172.21.15.52:0/1788462746 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7f2c540d0060 con 0x7f2c58101b20 2024-04-02T13:51:00.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.373+0000 7f2c5ce96640 1 --2- 172.21.15.52:0/1788462746 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2c440a3c30 0x7f2c440a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2c48005fd0 tx=0x7f2c48005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:00.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.376+0000 7f2c4e7fc640 1 -- 172.21.15.52:0/1788462746 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2c54009de0 con 0x7f2c58101b20 2024-04-02T13:51:00.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:00 smithi189 ceph-mon[36683]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:51:00.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:00 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3822786089' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-02T13:51:00.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:00 smithi189 ceph-mon[36683]: osdmap e83: 12 total, 12 up, 12 in 2024-04-02T13:51:00.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:00 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:51:00.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:00 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:51:00.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:00 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:51:00.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:00 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:51:00.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:00 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:51:00.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:00 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:51:00.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:00 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:51:00.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:00 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:51:00.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:00 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:51:00.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:00 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/343378864' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-02T13:51:00.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:00 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:51:00.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:00 smithi052 ceph-mon[32104]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:51:00.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:00 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3822786089' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-02T13:51:00.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:00 smithi052 ceph-mon[32104]: osdmap e83: 12 total, 12 up, 12 in 2024-04-02T13:51:00.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:00 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:51:00.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:00 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:51:00.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:00 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:51:00.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:00 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:51:00.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:00 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:51:00.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:00 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:51:00.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:00 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:51:00.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:00 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:51:00.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:00 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:51:00.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:00 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/343378864' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-02T13:51:00.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:00 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:51:00.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:00.601+0000 7f2c5f8fb640 1 -- 172.21.15.52:0/1788462746 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2c5802d370 con 0x7f2c58101b20 2024-04-02T13:51:00.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:00 smithi088 ceph-mon[36528]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:51:00.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:00 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3822786089' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-02T13:51:00.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:00 smithi088 ceph-mon[36528]: osdmap e83: 12 total, 12 up, 12 in 2024-04-02T13:51:00.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:00 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:51:00.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:00 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:51:00.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:00 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:51:00.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:00 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:51:00.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:00 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:51:00.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:00 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:51:00.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:00 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:51:00.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:00 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:51:00.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:00 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:51:00.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:00 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/343378864' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-02T13:51:00.671 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:00 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:51:01.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.322+0000 7f2c4e7fc640 1 -- 172.21.15.52:0/1788462746 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f2c5409cbd0 con 0x7f2c58101b20 2024-04-02T13:51:01.326 INFO:teuthology.orchestra.run.smithi052.stderr:new fs with metadata pool 2 and data pool 3 2024-04-02T13:51:01.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.325+0000 7f2c5f8fb640 1 -- 172.21.15.52:0/1788462746 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2c440a3c30 msgr2=0x7f2c440a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:01.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.325+0000 7f2c5f8fb640 1 --2- 172.21.15.52:0/1788462746 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2c440a3c30 0x7f2c440a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2c48005fd0 tx=0x7f2c48005850 comp rx=0 tx=0).stop 2024-04-02T13:51:01.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.325+0000 7f2c5f8fb640 1 -- 172.21.15.52:0/1788462746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c58101b20 msgr2=0x7f2c58197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:01.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.325+0000 7f2c5f8fb640 1 --2- 172.21.15.52:0/1788462746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c58101b20 0x7f2c58197dd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2c5400d770 tx=0x7f2c5400dc30 comp rx=0 tx=0).stop 2024-04-02T13:51:01.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.325+0000 7f2c5f8fb640 1 -- 172.21.15.52:0/1788462746 shutdown_connections 2024-04-02T13:51:01.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.325+0000 7f2c5f8fb640 1 --2- 172.21.15.52:0/1788462746 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2c440a3c30 0x7f2c440a60f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:01.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.325+0000 7f2c5f8fb640 1 --2- 172.21.15.52:0/1788462746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c581036d0 0x7f2c5819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:01.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.325+0000 7f2c5f8fb640 1 --2- 172.21.15.52:0/1788462746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c58102d10 0x7f2c58198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:01.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.325+0000 7f2c5f8fb640 1 --2- 172.21.15.52:0/1788462746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c58101b20 0x7f2c58197dd0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:01.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.325+0000 7f2c5f8fb640 1 -- 172.21.15.52:0/1788462746 >> 172.21.15.52:0/1788462746 conn(0x7f2c580fd070 msgr2=0x7f2c58106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:01.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.325+0000 7f2c5f8fb640 1 -- 172.21.15.52:0/1788462746 shutdown_connections 2024-04-02T13:51:01.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.325+0000 7f2c5f8fb640 1 -- 172.21.15.52:0/1788462746 wait complete. 2024-04-02T13:51:01.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:01 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/343378864' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-02T13:51:01.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:01 smithi052 ceph-mon[32104]: osdmap e84: 12 total, 12 up, 12 in 2024-04-02T13:51:01.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:01 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1788462746' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-02T13:51:01.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:01 smithi052 ceph-mon[32104]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-02T13:51:01.343 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-02T13:51:01.343 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-02T13:51:01.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.450+0000 7f5c5c2d3640 1 -- 172.21.15.52:0/2243933846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c54104ed0 msgr2=0x7f5c541072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:01.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.450+0000 7f5c5c2d3640 1 --2- 172.21.15.52:0/2243933846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c54104ed0 0x7f5c541072c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009900 tx=0x7f5c4402bc30 comp rx=0 tx=0).stop 2024-04-02T13:51:01.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.451+0000 7f5c5c2d3640 1 -- 172.21.15.52:0/2243933846 shutdown_connections 2024-04-02T13:51:01.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.451+0000 7f5c5c2d3640 1 --2- 172.21.15.52:0/2243933846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c54107800 0x7f5c54109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:01.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.451+0000 7f5c5c2d3640 1 --2- 172.21.15.52:0/2243933846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c54104ed0 0x7f5c541072c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:01.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.451+0000 7f5c5c2d3640 1 --2- 172.21.15.52:0/2243933846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c541025a0 0x7f5c54104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.451+0000 7f5c5c2d3640 1 -- 172.21.15.52:0/2243933846 >> 172.21.15.52:0/2243933846 conn(0x7f5c540fbfb0 msgr2=0x7f5c540fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.451+0000 7f5c5c2d3640 1 -- 172.21.15.52:0/2243933846 shutdown_connections 2024-04-02T13:51:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.451+0000 7f5c5c2d3640 1 -- 172.21.15.52:0/2243933846 wait complete. 2024-04-02T13:51:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.451+0000 7f5c5c2d3640 1 Processor -- start 2024-04-02T13:51:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.451+0000 7f5c5c2d3640 1 -- start start 2024-04-02T13:51:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.451+0000 7f5c5c2d3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c541025a0 0x7f5c54101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.451+0000 7f5c5c2d3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c54104ed0 0x7f5c54100550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.451+0000 7f5c5c2d3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c54107800 0x7f5c54100a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.451+0000 7f5c5c2d3640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5410d870 con 0x7f5c541025a0 2024-04-02T13:51:01.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.451+0000 7f5c5c2d3640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5410d6f0 con 0x7f5c54104ed0 2024-04-02T13:51:01.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.451+0000 7f5c5c2d3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5410d9f0 con 0x7f5c54107800 2024-04-02T13:51:01.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.451+0000 7f5c5a06f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c541025a0 0x7f5c54101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:01.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.451+0000 7f5c5a06f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c541025a0 0x7f5c54101f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53102/0 (socket says 172.21.15.52:53102) 2024-04-02T13:51:01.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.451+0000 7f5c5a06f640 1 -- 172.21.15.52:0/911293511 learned_addr learned my addr 172.21.15.52:0/911293511 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:01.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.452+0000 7f5c5a870640 1 --2- 172.21.15.52:0/911293511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c54107800 0x7f5c54100a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:01.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.452+0000 7f5c5a06f640 1 -- 172.21.15.52:0/911293511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c54107800 msgr2=0x7f5c54100a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:01.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.452+0000 7f5c5986e640 1 --2- 172.21.15.52:0/911293511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c54104ed0 0x7f5c54100550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:01.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.452+0000 7f5c5a06f640 1 --2- 172.21.15.52:0/911293511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c54107800 0x7f5c54100a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:01.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.452+0000 7f5c5a06f640 1 -- 172.21.15.52:0/911293511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c54104ed0 msgr2=0x7f5c54100550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:01.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.452+0000 7f5c5a06f640 1 --2- 172.21.15.52:0/911293511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c54104ed0 0x7f5c54100550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:01.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.452+0000 7f5c5a06f640 1 -- 172.21.15.52:0/911293511 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c54101320 con 0x7f5c541025a0 2024-04-02T13:51:01.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.452+0000 7f5c5a06f640 1 --2- 172.21.15.52:0/911293511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c541025a0 0x7f5c54101f10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000b4a0 tx=0x7f5c5000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:01.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.452+0000 7f5c4b7fe640 1 -- 172.21.15.52:0/911293511 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50014070 con 0x7f5c541025a0 2024-04-02T13:51:01.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.452+0000 7f5c5c2d3640 1 -- 172.21.15.52:0/911293511 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c541a2060 con 0x7f5c541025a0 2024-04-02T13:51:01.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.452+0000 7f5c4b7fe640 1 -- 172.21.15.52:0/911293511 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c50010af0 con 0x7f5c541025a0 2024-04-02T13:51:01.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.452+0000 7f5c4b7fe640 1 -- 172.21.15.52:0/911293511 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50017d10 con 0x7f5c541025a0 2024-04-02T13:51:01.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.452+0000 7f5c5c2d3640 1 -- 172.21.15.52:0/911293511 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c541a25b0 con 0x7f5c541025a0 2024-04-02T13:51:01.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.453+0000 7f5c5c2d3640 1 -- 172.21.15.52:0/911293511 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c540fdaa0 con 0x7f5c541025a0 2024-04-02T13:51:01.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.454+0000 7f5c4b7fe640 1 -- 172.21.15.52:0/911293511 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5c50017070 con 0x7f5c541025a0 2024-04-02T13:51:01.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.455+0000 7f5c4b7fe640 1 --2- 172.21.15.52:0/911293511 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f5c400a3bb0 0x7f5c400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:01.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.455+0000 7f5c5986e640 1 --2- 172.21.15.52:0/911293511 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f5c400a3bb0 0x7f5c400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:01.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.455+0000 7f5c4b7fe640 1 -- 172.21.15.52:0/911293511 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f5c500cfb10 con 0x7f5c541025a0 2024-04-02T13:51:01.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.455+0000 7f5c5986e640 1 --2- 172.21.15.52:0/911293511 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f5c400a3bb0 0x7f5c400a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009230 tx=0x7f5c44005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:01.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.459+0000 7f5c4b7fe640 1 -- 172.21.15.52:0/911293511 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5c5009c640 con 0x7f5c541025a0 2024-04-02T13:51:01.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:01 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/343378864' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-02T13:51:01.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:01 smithi189 ceph-mon[36683]: osdmap e84: 12 total, 12 up, 12 in 2024-04-02T13:51:01.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:01 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1788462746' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-02T13:51:01.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:01 smithi189 ceph-mon[36683]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-02T13:51:01.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:01.654+0000 7f5c5c2d3640 1 -- 172.21.15.52:0/911293511 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f5c5402d370 con 0x7f5c541025a0 2024-04-02T13:51:01.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:01 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/343378864' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-02T13:51:01.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:01 smithi088 ceph-mon[36528]: osdmap e84: 12 total, 12 up, 12 in 2024-04-02T13:51:01.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:01 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1788462746' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-02T13:51:01.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:01 smithi088 ceph-mon[36528]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-02T13:51:02.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.236+0000 7f5c4b7fe640 1 -- 172.21.15.52:0/911293511 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) ==== 157+0+0 (secure 0 0 0) 0x7f5c5009bf40 con 0x7f5c541025a0 2024-04-02T13:51:02.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.239+0000 7f5c5c2d3640 1 -- 172.21.15.52:0/911293511 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f5c400a3bb0 msgr2=0x7f5c400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:02.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.239+0000 7f5c5c2d3640 1 --2- 172.21.15.52:0/911293511 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f5c400a3bb0 0x7f5c400a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009230 tx=0x7f5c44005fd0 comp rx=0 tx=0).stop 2024-04-02T13:51:02.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.239+0000 7f5c5c2d3640 1 -- 172.21.15.52:0/911293511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c541025a0 msgr2=0x7f5c54101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:02.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.239+0000 7f5c5c2d3640 1 --2- 172.21.15.52:0/911293511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c541025a0 0x7f5c54101f10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000b4a0 tx=0x7f5c5000b960 comp rx=0 tx=0).stop 2024-04-02T13:51:02.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.239+0000 7f5c5c2d3640 1 -- 172.21.15.52:0/911293511 shutdown_connections 2024-04-02T13:51:02.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.239+0000 7f5c5c2d3640 1 --2- 172.21.15.52:0/911293511 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f5c400a3bb0 0x7f5c400a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:02.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.239+0000 7f5c5c2d3640 1 --2- 172.21.15.52:0/911293511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c54107800 0x7f5c54100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:02.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.240+0000 7f5c5c2d3640 1 --2- 172.21.15.52:0/911293511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c54104ed0 0x7f5c54100550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:02.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.240+0000 7f5c5c2d3640 1 --2- 172.21.15.52:0/911293511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c541025a0 0x7f5c54101f10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:02.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.240+0000 7f5c5c2d3640 1 -- 172.21.15.52:0/911293511 >> 172.21.15.52:0/911293511 conn(0x7f5c540fbfb0 msgr2=0x7f5c54102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:02.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.240+0000 7f5c5c2d3640 1 -- 172.21.15.52:0/911293511 shutdown_connections 2024-04-02T13:51:02.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.240+0000 7f5c5c2d3640 1 -- 172.21.15.52:0/911293511 wait complete. 2024-04-02T13:51:02.257 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-02T13:51:02.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.373+0000 7f9aea1cd640 1 -- 172.21.15.52:0/2751633790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ae4101630 msgr2=0x7f9ae4108ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:02.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.373+0000 7f9aea1cd640 1 --2- 172.21.15.52:0/2751633790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ae4101630 0x7f9ae4108ed0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9adc009900 tx=0x7f9adc02bc80 comp rx=0 tx=0).stop 2024-04-02T13:51:02.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.373+0000 7f9aea1cd640 1 -- 172.21.15.52:0/2751633790 shutdown_connections 2024-04-02T13:51:02.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.373+0000 7f9aea1cd640 1 --2- 172.21.15.52:0/2751633790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ae4101630 0x7f9ae4108ed0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:02.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.373+0000 7f9aea1cd640 1 --2- 172.21.15.52:0/2751633790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ae4100c70 0x7f9ae41010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:02.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.373+0000 7f9aea1cd640 1 --2- 172.21.15.52:0/2751633790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae41020b0 0x7f9ae41024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:02.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.373+0000 7f9aea1cd640 1 -- 172.21.15.52:0/2751633790 >> 172.21.15.52:0/2751633790 conn(0x7f9ae40fb720 msgr2=0x7f9ae40fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:02.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.373+0000 7f9aea1cd640 1 -- 172.21.15.52:0/2751633790 shutdown_connections 2024-04-02T13:51:02.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.373+0000 7f9aea1cd640 1 -- 172.21.15.52:0/2751633790 wait complete. 2024-04-02T13:51:02.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.373+0000 7f9aea1cd640 1 Processor -- start 2024-04-02T13:51:02.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.373+0000 7f9aea1cd640 1 -- start start 2024-04-02T13:51:02.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.374+0000 7f9aea1cd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae4100c70 0x7f9ae41006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:02.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.374+0000 7f9aea1cd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ae4101630 0x7f9ae419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:02.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.374+0000 7f9aea1cd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ae41020b0 0x7f9ae419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:02.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.374+0000 7f9aea1cd640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae410c6a0 con 0x7f9ae4101630 2024-04-02T13:51:02.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.374+0000 7f9aea1cd640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae410c520 con 0x7f9ae4100c70 2024-04-02T13:51:02.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.374+0000 7f9aea1cd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae410c820 con 0x7f9ae41020b0 2024-04-02T13:51:02.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.374+0000 7f9ae2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ae4101630 0x7f9ae419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:02.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.374+0000 7f9ae2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ae4101630 0x7f9ae419caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53910/0 (socket says 172.21.15.52:53910) 2024-04-02T13:51:02.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.374+0000 7f9ae2ffd640 1 -- 172.21.15.52:0/2597458067 learned_addr learned my addr 172.21.15.52:0/2597458067 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:02.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.374+0000 7f9ae3fff640 1 --2- 172.21.15.52:0/2597458067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ae41020b0 0x7f9ae419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:02.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.374+0000 7f9ae37fe640 1 --2- 172.21.15.52:0/2597458067 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae4100c70 0x7f9ae41006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:02.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.374+0000 7f9ae2ffd640 1 -- 172.21.15.52:0/2597458067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ae41020b0 msgr2=0x7f9ae419f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:02.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.374+0000 7f9ae2ffd640 1 --2- 172.21.15.52:0/2597458067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ae41020b0 0x7f9ae419f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:02.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.374+0000 7f9ae2ffd640 1 -- 172.21.15.52:0/2597458067 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae4100c70 msgr2=0x7f9ae41006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:02.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.374+0000 7f9ae2ffd640 1 --2- 172.21.15.52:0/2597458067 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae4100c70 0x7f9ae41006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:02.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.374+0000 7f9ae2ffd640 1 -- 172.21.15.52:0/2597458067 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae419f9d0 con 0x7f9ae4101630 2024-04-02T13:51:02.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.374+0000 7f9ae37fe640 1 --2- 172.21.15.52:0/2597458067 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae4100c70 0x7f9ae41006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:51:02.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.374+0000 7f9ae2ffd640 1 --2- 172.21.15.52:0/2597458067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ae4101630 0x7f9ae419caa0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800d770 tx=0x7f9ad800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:02.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.374+0000 7f9ae0ff9640 1 -- 172.21.15.52:0/2597458067 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8017070 con 0x7f9ae4101630 2024-04-02T13:51:02.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.374+0000 7f9ae0ff9640 1 -- 172.21.15.52:0/2597458067 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ad8009c30 con 0x7f9ae4101630 2024-04-02T13:51:02.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.374+0000 7f9aea1cd640 1 -- 172.21.15.52:0/2597458067 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ae419fc60 con 0x7f9ae4101630 2024-04-02T13:51:02.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.374+0000 7f9ae0ff9640 1 -- 172.21.15.52:0/2597458067 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8003dd0 con 0x7f9ae4101630 2024-04-02T13:51:02.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.375+0000 7f9aea1cd640 1 -- 172.21.15.52:0/2597458067 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ae41a01b0 con 0x7f9ae4101630 2024-04-02T13:51:02.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.376+0000 7f9aea1cd640 1 -- 172.21.15.52:0/2597458067 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ae4107d50 con 0x7f9ae4101630 2024-04-02T13:51:02.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.377+0000 7f9ae0ff9640 1 -- 172.21.15.52:0/2597458067 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9ad8003fb0 con 0x7f9ae4101630 2024-04-02T13:51:02.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.377+0000 7f9ae0ff9640 1 --2- 172.21.15.52:0/2597458067 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f9acc0a3bb0 0x7f9acc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:02.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.377+0000 7f9ae37fe640 1 --2- 172.21.15.52:0/2597458067 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f9acc0a3bb0 0x7f9acc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:02.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.377+0000 7f9ae0ff9640 1 -- 172.21.15.52:0/2597458067 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8277+0+0 (secure 0 0 0) 0x7f9ad80cfda0 con 0x7f9ae4101630 2024-04-02T13:51:02.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.378+0000 7f9ae37fe640 1 --2- 172.21.15.52:0/2597458067 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f9acc0a3bb0 0x7f9acc0a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4007e70 tx=0x7f9ad4007850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:02.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.382+0000 7f9ae0ff9640 1 -- 172.21.15.52:0/2597458067 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9ad809c750 con 0x7f9ae4101630 2024-04-02T13:51:02.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:02 smithi189 ceph-mon[36683]: pgmap v287: 97 pgs: 15 creating+peering, 9 active+clean, 73 unknown; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:51:02.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:02 smithi189 ceph-mon[36683]: osdmap e85: 12 total, 12 up, 12 in 2024-04-02T13:51:02.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:02 smithi189 ceph-mon[36683]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T13:51:02.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:02 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1788462746' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-02T13:51:02.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:02 smithi189 ceph-mon[36683]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-02T13:51:02.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:02 smithi189 ceph-mon[36683]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-02T13:51:02.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:02 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/911293511' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-02T13:51:02.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:02.568+0000 7f9aea1cd640 1 -- 172.21.15.52:0/2597458067 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9ae40627a0 con 0x7f9ae4101630 2024-04-02T13:51:02.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:02 smithi052 ceph-mon[32104]: pgmap v287: 97 pgs: 15 creating+peering, 9 active+clean, 73 unknown; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:51:02.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:02 smithi052 ceph-mon[32104]: osdmap e85: 12 total, 12 up, 12 in 2024-04-02T13:51:02.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:02 smithi052 ceph-mon[32104]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T13:51:02.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:02 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1788462746' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-02T13:51:02.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:02 smithi052 ceph-mon[32104]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-02T13:51:02.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:02 smithi052 ceph-mon[32104]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-02T13:51:02.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:02 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/911293511' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-02T13:51:02.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:02 smithi088 ceph-mon[36528]: pgmap v287: 97 pgs: 15 creating+peering, 9 active+clean, 73 unknown; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:51:02.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:02 smithi088 ceph-mon[36528]: osdmap e85: 12 total, 12 up, 12 in 2024-04-02T13:51:02.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:02 smithi088 ceph-mon[36528]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T13:51:02.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:02 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1788462746' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-02T13:51:02.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:02 smithi088 ceph-mon[36528]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-02T13:51:02.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:02 smithi088 ceph-mon[36528]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-02T13:51:02.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:02 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/911293511' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-02T13:51:03.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:03 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/911293511' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-02T13:51:03.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:03 smithi189 ceph-mon[36683]: osdmap e86: 12 total, 12 up, 12 in 2024-04-02T13:51:03.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:03 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] up:active 2024-04-02T13:51:03.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:03 smithi189 ceph-mon[36683]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:51:03.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:03 smithi189 ceph-mon[36683]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:51:03.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:03 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2597458067' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-02T13:51:03.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:03 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/911293511' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-02T13:51:03.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:03 smithi052 ceph-mon[32104]: osdmap e86: 12 total, 12 up, 12 in 2024-04-02T13:51:03.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:03 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] up:active 2024-04-02T13:51:03.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:03 smithi052 ceph-mon[32104]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:51:03.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:03 smithi052 ceph-mon[32104]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:51:03.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:03 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2597458067' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-02T13:51:03.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:03 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/911293511' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-02T13:51:03.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:03 smithi088 ceph-mon[36528]: osdmap e86: 12 total, 12 up, 12 in 2024-04-02T13:51:03.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:03 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] up:active 2024-04-02T13:51:03.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:03 smithi088 ceph-mon[36528]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:51:03.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:03 smithi088 ceph-mon[36528]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:51:03.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:03 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2597458067' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-02T13:51:04.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.336+0000 7f9ae0ff9640 1 -- 172.21.15.52:0/2597458067 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) ==== 228+0+0 (secure 0 0 0) 0x7f9ad8009de0 con 0x7f9ae4101630 2024-04-02T13:51:04.340 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data_ec' created 2024-04-02T13:51:04.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.340+0000 7f9aea1cd640 1 -- 172.21.15.52:0/2597458067 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f9acc0a3bb0 msgr2=0x7f9acc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:04.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.340+0000 7f9aea1cd640 1 --2- 172.21.15.52:0/2597458067 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f9acc0a3bb0 0x7f9acc0a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4007e70 tx=0x7f9ad4007850 comp rx=0 tx=0).stop 2024-04-02T13:51:04.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.340+0000 7f9aea1cd640 1 -- 172.21.15.52:0/2597458067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ae4101630 msgr2=0x7f9ae419caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:04.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.340+0000 7f9aea1cd640 1 --2- 172.21.15.52:0/2597458067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ae4101630 0x7f9ae419caa0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800d770 tx=0x7f9ad800dc30 comp rx=0 tx=0).stop 2024-04-02T13:51:04.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.340+0000 7f9aea1cd640 1 -- 172.21.15.52:0/2597458067 shutdown_connections 2024-04-02T13:51:04.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.340+0000 7f9aea1cd640 1 --2- 172.21.15.52:0/2597458067 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f9acc0a3bb0 0x7f9acc0a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:04.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.340+0000 7f9aea1cd640 1 --2- 172.21.15.52:0/2597458067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ae41020b0 0x7f9ae419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:04.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.340+0000 7f9aea1cd640 1 --2- 172.21.15.52:0/2597458067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ae4101630 0x7f9ae419caa0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:04.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.340+0000 7f9aea1cd640 1 --2- 172.21.15.52:0/2597458067 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae4100c70 0x7f9ae41006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:04.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.340+0000 7f9aea1cd640 1 -- 172.21.15.52:0/2597458067 >> 172.21.15.52:0/2597458067 conn(0x7f9ae40fb720 msgr2=0x7f9ae4104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:04.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.340+0000 7f9aea1cd640 1 -- 172.21.15.52:0/2597458067 shutdown_connections 2024-04-02T13:51:04.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.340+0000 7f9aea1cd640 1 -- 172.21.15.52:0/2597458067 wait complete. 2024-04-02T13:51:04.362 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-02T13:51:04.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.472+0000 7fcdc72b7640 1 -- 172.21.15.52:0/992860260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdc01077e0 msgr2=0x7fcdc0109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:04.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.472+0000 7fcdc72b7640 1 --2- 172.21.15.52:0/992860260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdc01077e0 0x7fcdc0109c30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009ca0 tx=0x7fcdb802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:51:04.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.472+0000 7fcdc72b7640 1 -- 172.21.15.52:0/992860260 shutdown_connections 2024-04-02T13:51:04.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.472+0000 7fcdc72b7640 1 --2- 172.21.15.52:0/992860260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdc01077e0 0x7fcdc0109c30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:04.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.472+0000 7fcdc72b7640 1 --2- 172.21.15.52:0/992860260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdc0104eb0 0x7fcdc01072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:04.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.472+0000 7fcdc72b7640 1 --2- 172.21.15.52:0/992860260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcdc0102580 0x7fcdc0104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:04.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.472+0000 7fcdc72b7640 1 -- 172.21.15.52:0/992860260 >> 172.21.15.52:0/992860260 conn(0x7fcdc00fbf90 msgr2=0x7fcdc00fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:04.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.472+0000 7fcdc72b7640 1 -- 172.21.15.52:0/992860260 shutdown_connections 2024-04-02T13:51:04.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.472+0000 7fcdc72b7640 1 -- 172.21.15.52:0/992860260 wait complete. 2024-04-02T13:51:04.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.473+0000 7fcdc72b7640 1 Processor -- start 2024-04-02T13:51:04.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.473+0000 7fcdc72b7640 1 -- start start 2024-04-02T13:51:04.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.473+0000 7fcdc72b7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdc0102580 0x7fcdc0197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:04.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.473+0000 7fcdc72b7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcdc0104eb0 0x7fcdc0198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:04.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.473+0000 7fcdc72b7640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdc01077e0 0x7fcdc019f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:04.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.473+0000 7fcdc72b7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc010d7a0 con 0x7fcdc0102580 2024-04-02T13:51:04.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.473+0000 7fcdc72b7640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc010d620 con 0x7fcdc01077e0 2024-04-02T13:51:04.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.473+0000 7fcdc72b7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc010d920 con 0x7fcdc0104eb0 2024-04-02T13:51:04.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.473+0000 7fcdc5053640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdc0102580 0x7fcdc0197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:04.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.473+0000 7fcdc5053640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdc0102580 0x7fcdc0197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53938/0 (socket says 172.21.15.52:53938) 2024-04-02T13:51:04.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.473+0000 7fcdc5053640 1 -- 172.21.15.52:0/254074231 learned_addr learned my addr 172.21.15.52:0/254074231 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:04.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.473+0000 7fcdc5854640 1 --2- 172.21.15.52:0/254074231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdc01077e0 0x7fcdc019f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:04.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.473+0000 7fcdc5053640 1 -- 172.21.15.52:0/254074231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcdc0104eb0 msgr2=0x7fcdc0198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:04.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.473+0000 7fcdc5053640 1 --2- 172.21.15.52:0/254074231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcdc0104eb0 0x7fcdc0198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:04.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.474+0000 7fcdc5053640 1 -- 172.21.15.52:0/254074231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdc01077e0 msgr2=0x7fcdc019f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:04.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.474+0000 7fcdc5053640 1 --2- 172.21.15.52:0/254074231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdc01077e0 0x7fcdc019f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:04.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.474+0000 7fcdc5053640 1 -- 172.21.15.52:0/254074231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc019fa00 con 0x7fcdc0102580 2024-04-02T13:51:04.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.474+0000 7fcdc5854640 1 --2- 172.21.15.52:0/254074231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdc01077e0 0x7fcdc019f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:51:04.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.474+0000 7fcdc5053640 1 --2- 172.21.15.52:0/254074231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdc0102580 0x7fcdc0197de0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc00c970 tx=0x7fcdbc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:04.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.474+0000 7fcdb67fc640 1 -- 172.21.15.52:0/254074231 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc007bb0 con 0x7fcdc0102580 2024-04-02T13:51:04.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.474+0000 7fcdb67fc640 1 -- 172.21.15.52:0/254074231 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdbc007d50 con 0x7fcdc0102580 2024-04-02T13:51:04.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.474+0000 7fcdc72b7640 1 -- 172.21.15.52:0/254074231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc019fcf0 con 0x7fcdc0102580 2024-04-02T13:51:04.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.474+0000 7fcdb67fc640 1 -- 172.21.15.52:0/254074231 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc00f450 con 0x7fcdc0102580 2024-04-02T13:51:04.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.474+0000 7fcdc72b7640 1 -- 172.21.15.52:0/254074231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc01a0190 con 0x7fcdc0102580 2024-04-02T13:51:04.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.475+0000 7fcdc72b7640 1 -- 172.21.15.52:0/254074231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdc00fda80 con 0x7fcdc0102580 2024-04-02T13:51:04.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.476+0000 7fcdb67fc640 1 -- 172.21.15.52:0/254074231 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcdbc004fe0 con 0x7fcdc0102580 2024-04-02T13:51:04.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.477+0000 7fcdb67fc640 1 --2- 172.21.15.52:0/254074231 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fcdac0a3bb0 0x7fcdac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:04.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.477+0000 7fcdc4852640 1 --2- 172.21.15.52:0/254074231 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fcdac0a3bb0 0x7fcdac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:04.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.477+0000 7fcdb67fc640 1 -- 172.21.15.52:0/254074231 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7fcdbc0d0820 con 0x7fcdc0102580 2024-04-02T13:51:04.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.477+0000 7fcdc4852640 1 --2- 172.21.15.52:0/254074231 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fcdac0a3bb0 0x7fcdac0a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcdc01992f0 tx=0x7fcdb0005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:04.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.480+0000 7fcdb67fc640 1 -- 172.21.15.52:0/254074231 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fcdbc014030 con 0x7fcdc0102580 2024-04-02T13:51:04.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:04 smithi189 ceph-mon[36683]: pgmap v290: 97 pgs: 42 creating+peering, 34 active+clean, 21 unknown; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-04-02T13:51:04.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:04 smithi189 ceph-mon[36683]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-02T13:51:04.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:04 smithi189 ceph-mon[36683]: Cluster is now healthy 2024-04-02T13:51:04.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:04 smithi189 ceph-mon[36683]: osdmap e87: 12 total, 12 up, 12 in 2024-04-02T13:51:04.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:04 smithi052 ceph-mon[32104]: pgmap v290: 97 pgs: 42 creating+peering, 34 active+clean, 21 unknown; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-04-02T13:51:04.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:04 smithi052 ceph-mon[32104]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-02T13:51:04.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:04 smithi052 ceph-mon[32104]: Cluster is now healthy 2024-04-02T13:51:04.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:04 smithi052 ceph-mon[32104]: osdmap e87: 12 total, 12 up, 12 in 2024-04-02T13:51:04.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:04 smithi088 ceph-mon[36528]: pgmap v290: 97 pgs: 42 creating+peering, 34 active+clean, 21 unknown; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-04-02T13:51:04.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:04 smithi088 ceph-mon[36528]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-02T13:51:04.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:04 smithi088 ceph-mon[36528]: Cluster is now healthy 2024-04-02T13:51:04.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:04 smithi088 ceph-mon[36528]: osdmap e87: 12 total, 12 up, 12 in 2024-04-02T13:51:04.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:04.667+0000 7fcdc72b7640 1 -- 172.21.15.52:0/254074231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fcdc002d370 con 0x7fcdc0102580 2024-04-02T13:51:05.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.341+0000 7fcdb67fc640 1 -- 172.21.15.52:0/254074231 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) ==== 169+0+0 (secure 0 0 0) 0x7fcdbc09cfe0 con 0x7fcdc0102580 2024-04-02T13:51:05.345 INFO:teuthology.orchestra.run.smithi052.stderr:set pool 4 allow_ec_overwrites to true 2024-04-02T13:51:05.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.345+0000 7fcdc72b7640 1 -- 172.21.15.52:0/254074231 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fcdac0a3bb0 msgr2=0x7fcdac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:05.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.345+0000 7fcdc72b7640 1 --2- 172.21.15.52:0/254074231 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fcdac0a3bb0 0x7fcdac0a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcdc01992f0 tx=0x7fcdb0005bf0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.345+0000 7fcdc72b7640 1 -- 172.21.15.52:0/254074231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdc0102580 msgr2=0x7fcdc0197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.345+0000 7fcdc72b7640 1 --2- 172.21.15.52:0/254074231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdc0102580 0x7fcdc0197de0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc00c970 tx=0x7fcdbc00ce30 comp rx=0 tx=0).stop 2024-04-02T13:51:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.345+0000 7fcdc72b7640 1 -- 172.21.15.52:0/254074231 shutdown_connections 2024-04-02T13:51:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.345+0000 7fcdc72b7640 1 --2- 172.21.15.52:0/254074231 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fcdac0a3bb0 0x7fcdac0a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.345+0000 7fcdc72b7640 1 --2- 172.21.15.52:0/254074231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdc01077e0 0x7fcdc019f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.345+0000 7fcdc72b7640 1 --2- 172.21.15.52:0/254074231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcdc0104eb0 0x7fcdc0198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.345+0000 7fcdc72b7640 1 --2- 172.21.15.52:0/254074231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdc0102580 0x7fcdc0197de0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.345+0000 7fcdc72b7640 1 -- 172.21.15.52:0/254074231 >> 172.21.15.52:0/254074231 conn(0x7fcdc00fbf90 msgr2=0x7fcdc0102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.346+0000 7fcdc72b7640 1 -- 172.21.15.52:0/254074231 shutdown_connections 2024-04-02T13:51:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.346+0000 7fcdc72b7640 1 -- 172.21.15.52:0/254074231 wait complete. 2024-04-02T13:51:05.371 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-02T13:51:05.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.479+0000 7f52187a8640 1 -- 172.21.15.52:0/9977925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5210101c00 msgr2=0x7f5210102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:05.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.479+0000 7f52187a8640 1 --2- 172.21.15.52:0/9977925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5210101c00 0x7f5210102080 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5200009900 tx=0x7f520002bc30 comp rx=0 tx=0).stop 2024-04-02T13:51:05.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.479+0000 7f52187a8640 1 -- 172.21.15.52:0/9977925 shutdown_connections 2024-04-02T13:51:05.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.479+0000 7f52187a8640 1 --2- 172.21.15.52:0/9977925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52101025c0 0x7f5210108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.479+0000 7f52187a8640 1 --2- 172.21.15.52:0/9977925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5210101c00 0x7f5210102080 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.479+0000 7f52187a8640 1 --2- 172.21.15.52:0/9977925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5210100a10 0x7f5210100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.479+0000 7f52187a8640 1 -- 172.21.15.52:0/9977925 >> 172.21.15.52:0/9977925 conn(0x7f52100fbfa0 msgr2=0x7f52100fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:05.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.479+0000 7f52187a8640 1 -- 172.21.15.52:0/9977925 shutdown_connections 2024-04-02T13:51:05.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.479+0000 7f52187a8640 1 -- 172.21.15.52:0/9977925 wait complete. 2024-04-02T13:51:05.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.479+0000 7f52187a8640 1 Processor -- start 2024-04-02T13:51:05.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.479+0000 7f52187a8640 1 -- start start 2024-04-02T13:51:05.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.480+0000 7f52187a8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5210100a10 0x7f521006ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.480+0000 7f52187a8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5210101c00 0x7f521019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.480+0000 7f52187a8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52101025c0 0x7f521019f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.480+0000 7f52187a8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f521010c690 con 0x7f52101025c0 2024-04-02T13:51:05.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.480+0000 7f52187a8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f521010c510 con 0x7f5210101c00 2024-04-02T13:51:05.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.480+0000 7f52187a8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f521010c810 con 0x7f5210100a10 2024-04-02T13:51:05.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.480+0000 7f5216d45640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52101025c0 0x7f521019f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:05.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.480+0000 7f5216d45640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52101025c0 0x7f521019f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53948/0 (socket says 172.21.15.52:53948) 2024-04-02T13:51:05.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.480+0000 7f5216d45640 1 -- 172.21.15.52:0/1408724299 learned_addr learned my addr 172.21.15.52:0/1408724299 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:05.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.480+0000 7f5216544640 1 --2- 172.21.15.52:0/1408724299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5210100a10 0x7f521006ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:05.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.480+0000 7f5215d43640 1 --2- 172.21.15.52:0/1408724299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5210101c00 0x7f521019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:05.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.480+0000 7f5216d45640 1 -- 172.21.15.52:0/1408724299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5210100a10 msgr2=0x7f521006ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:05.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.480+0000 7f5216d45640 1 --2- 172.21.15.52:0/1408724299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5210100a10 0x7f521006ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.480+0000 7f5216d45640 1 -- 172.21.15.52:0/1408724299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5210101c00 msgr2=0x7f521019cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:05.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.480+0000 7f5216d45640 1 --2- 172.21.15.52:0/1408724299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5210101c00 0x7f521019cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.480+0000 7f5216d45640 1 -- 172.21.15.52:0/1408724299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f521019fab0 con 0x7f52101025c0 2024-04-02T13:51:05.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.480+0000 7f5216d45640 1 --2- 172.21.15.52:0/1408724299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52101025c0 0x7f521019f1a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f520800eaa0 tx=0x7f520800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:05.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.480+0000 7f52077fe640 1 -- 172.21.15.52:0/1408724299 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520800cdb0 con 0x7f52101025c0 2024-04-02T13:51:05.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.480+0000 7f52077fe640 1 -- 172.21.15.52:0/1408724299 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5208005020 con 0x7f52101025c0 2024-04-02T13:51:05.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.480+0000 7f52187a8640 1 -- 172.21.15.52:0/1408724299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f521019fd40 con 0x7f52101025c0 2024-04-02T13:51:05.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.480+0000 7f52077fe640 1 -- 172.21.15.52:0/1408724299 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52080116b0 con 0x7f52101025c0 2024-04-02T13:51:05.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.480+0000 7f52187a8640 1 -- 172.21.15.52:0/1408724299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52101a01b0 con 0x7f52101025c0 2024-04-02T13:51:05.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.481+0000 7f52187a8640 1 -- 172.21.15.52:0/1408724299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5210107db0 con 0x7f52101025c0 2024-04-02T13:51:05.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.483+0000 7f52077fe640 1 -- 172.21.15.52:0/1408724299 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5208005560 con 0x7f52101025c0 2024-04-02T13:51:05.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.484+0000 7f52077fe640 1 --2- 172.21.15.52:0/1408724299 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f51fc0a3bb0 0x7f51fc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.484+0000 7f5216544640 1 --2- 172.21.15.52:0/1408724299 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f51fc0a3bb0 0x7f51fc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:05.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.484+0000 7f52077fe640 1 -- 172.21.15.52:0/1408724299 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7f52080d0bf0 con 0x7f52101025c0 2024-04-02T13:51:05.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.484+0000 7f5216544640 1 --2- 172.21.15.52:0/1408724299 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f51fc0a3bb0 0x7f51fc0a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f520c005fd0 tx=0x7f520c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:05.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.488+0000 7f52077fe640 1 -- 172.21.15.52:0/1408724299 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5208016030 con 0x7f52101025c0 2024-04-02T13:51:05.648 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:05 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2597458067' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-02T13:51:05.648 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:05 smithi088 ceph-mon[36528]: osdmap e88: 12 total, 12 up, 12 in 2024-04-02T13:51:05.648 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:05 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/254074231' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-02T13:51:05.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:05.720+0000 7f52187a8640 1 -- 172.21.15.52:0/1408724299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f521002d410 con 0x7f52101025c0 2024-04-02T13:51:05.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:05 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2597458067' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-02T13:51:05.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:05 smithi189 ceph-mon[36683]: osdmap e88: 12 total, 12 up, 12 in 2024-04-02T13:51:05.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:05 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/254074231' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-02T13:51:05.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:05 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2597458067' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-02T13:51:05.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:05 smithi052 ceph-mon[32104]: osdmap e88: 12 total, 12 up, 12 in 2024-04-02T13:51:05.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:05 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/254074231' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-02T13:51:05.919 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 02 13:51:05 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-b[66653]: 2024-04-02T13:51:05.645+0000 7fed29007700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-02T13:51:06.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.343+0000 7f52077fe640 1 -- 172.21.15.52:0/1408724299 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f520809d3b0 con 0x7f52101025c0 2024-04-02T13:51:06.347 INFO:teuthology.orchestra.run.smithi052.stderr:added data pool 4 to fsmap 2024-04-02T13:51:06.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.347+0000 7f52187a8640 1 -- 172.21.15.52:0/1408724299 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f51fc0a3bb0 msgr2=0x7f51fc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.347+0000 7f52187a8640 1 --2- 172.21.15.52:0/1408724299 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f51fc0a3bb0 0x7f51fc0a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f520c005fd0 tx=0x7f520c00b390 comp rx=0 tx=0).stop 2024-04-02T13:51:06.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.347+0000 7f52187a8640 1 -- 172.21.15.52:0/1408724299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52101025c0 msgr2=0x7f521019f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.347+0000 7f52187a8640 1 --2- 172.21.15.52:0/1408724299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52101025c0 0x7f521019f1a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f520800eaa0 tx=0x7f520800ef60 comp rx=0 tx=0).stop 2024-04-02T13:51:06.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.347+0000 7f52187a8640 1 -- 172.21.15.52:0/1408724299 shutdown_connections 2024-04-02T13:51:06.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.347+0000 7f52187a8640 1 --2- 172.21.15.52:0/1408724299 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f51fc0a3bb0 0x7f51fc0a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.347+0000 7f52187a8640 1 --2- 172.21.15.52:0/1408724299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52101025c0 0x7f521019f1a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.347+0000 7f52187a8640 1 --2- 172.21.15.52:0/1408724299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5210101c00 0x7f521019cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.347+0000 7f52187a8640 1 --2- 172.21.15.52:0/1408724299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5210100a10 0x7f521006ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.347+0000 7f52187a8640 1 -- 172.21.15.52:0/1408724299 >> 172.21.15.52:0/1408724299 conn(0x7f52100fbfa0 msgr2=0x7f5210104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:06.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.347+0000 7f52187a8640 1 -- 172.21.15.52:0/1408724299 shutdown_connections 2024-04-02T13:51:06.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.347+0000 7f52187a8640 1 -- 172.21.15.52:0/1408724299 wait complete. 2024-04-02T13:51:06.365 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:51:06.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.471+0000 7f70c0255640 1 -- 172.21.15.52:0/624959714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70b81025d0 msgr2=0x7f70b8108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.471+0000 7f70c0255640 1 --2- 172.21.15.52:0/624959714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70b81025d0 0x7f70b8108e60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009ca0 tx=0x7f70b002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.471+0000 7f70c0255640 1 -- 172.21.15.52:0/624959714 shutdown_connections 2024-04-02T13:51:06.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.471+0000 7f70c0255640 1 --2- 172.21.15.52:0/624959714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70b81025d0 0x7f70b8108e60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.471+0000 7f70c0255640 1 --2- 172.21.15.52:0/624959714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70b8101c10 0x7f70b8102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.471+0000 7f70c0255640 1 --2- 172.21.15.52:0/624959714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70b8100a20 0x7f70b8100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.471+0000 7f70c0255640 1 -- 172.21.15.52:0/624959714 >> 172.21.15.52:0/624959714 conn(0x7f70b80fbf90 msgr2=0x7f70b80fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:06.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.471+0000 7f70c0255640 1 -- 172.21.15.52:0/624959714 shutdown_connections 2024-04-02T13:51:06.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.471+0000 7f70c0255640 1 -- 172.21.15.52:0/624959714 wait complete. 2024-04-02T13:51:06.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.472+0000 7f70c0255640 1 Processor -- start 2024-04-02T13:51:06.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.472+0000 7f70c0255640 1 -- start start 2024-04-02T13:51:06.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.472+0000 7f70c0255640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70b8100a20 0x7f70b810b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.472+0000 7f70c0255640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70b8101c10 0x7f70b81099e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.472+0000 7f70c0255640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70b81025d0 0x7f70b8109f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.472+0000 7f70c0255640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f70b810d7c0 con 0x7f70b8101c10 2024-04-02T13:51:06.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.472+0000 7f70c0255640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f70b810d640 con 0x7f70b8100a20 2024-04-02T13:51:06.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.472+0000 7f70c0255640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f70b810d940 con 0x7f70b81025d0 2024-04-02T13:51:06.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.472+0000 7f70bd7f0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70b8101c10 0x7f70b81099e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.472+0000 7f70bd7f0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70b8101c10 0x7f70b81099e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53968/0 (socket says 172.21.15.52:53968) 2024-04-02T13:51:06.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.472+0000 7f70bd7f0640 1 -- 172.21.15.52:0/2696032410 learned_addr learned my addr 172.21.15.52:0/2696032410 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:06.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.472+0000 7f70bdff1640 1 --2- 172.21.15.52:0/2696032410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70b8100a20 0x7f70b810b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.472+0000 7f70be7f2640 1 --2- 172.21.15.52:0/2696032410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70b81025d0 0x7f70b8109f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.473+0000 7f70bd7f0640 1 -- 172.21.15.52:0/2696032410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70b81025d0 msgr2=0x7f70b8109f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.473+0000 7f70bd7f0640 1 --2- 172.21.15.52:0/2696032410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70b81025d0 0x7f70b8109f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.473+0000 7f70bd7f0640 1 -- 172.21.15.52:0/2696032410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70b8100a20 msgr2=0x7f70b810b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.473+0000 7f70bd7f0640 1 --2- 172.21.15.52:0/2696032410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70b8100a20 0x7f70b810b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.473+0000 7f70bd7f0640 1 -- 172.21.15.52:0/2696032410 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b810a7b0 con 0x7f70b8101c10 2024-04-02T13:51:06.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.473+0000 7f70bdff1640 1 --2- 172.21.15.52:0/2696032410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70b8100a20 0x7f70b810b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:51:06.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.473+0000 7f70bd7f0640 1 --2- 172.21.15.52:0/2696032410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70b8101c10 0x7f70b81099e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f70a800c970 tx=0x7f70a800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:06.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.473+0000 7f70aeffd640 1 -- 172.21.15.52:0/2696032410 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8007bb0 con 0x7f70b8101c10 2024-04-02T13:51:06.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.473+0000 7f70aeffd640 1 -- 172.21.15.52:0/2696032410 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a8007d50 con 0x7f70b8101c10 2024-04-02T13:51:06.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.473+0000 7f70c0255640 1 -- 172.21.15.52:0/2696032410 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70b81a1f70 con 0x7f70b8101c10 2024-04-02T13:51:06.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.473+0000 7f70aeffd640 1 -- 172.21.15.52:0/2696032410 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a800f450 con 0x7f70b8101c10 2024-04-02T13:51:06.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.473+0000 7f70c0255640 1 -- 172.21.15.52:0/2696032410 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70b81a2380 con 0x7f70b8101c10 2024-04-02T13:51:06.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.474+0000 7f70c0255640 1 -- 172.21.15.52:0/2696032410 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70b8107ce0 con 0x7f70b8101c10 2024-04-02T13:51:06.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.475+0000 7f70aeffd640 1 -- 172.21.15.52:0/2696032410 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f70a8004fe0 con 0x7f70b8101c10 2024-04-02T13:51:06.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.475+0000 7f70aeffd640 1 --2- 172.21.15.52:0/2696032410 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f70a40a3bb0 0x7f70a40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.476+0000 7f70bdff1640 1 --2- 172.21.15.52:0/2696032410 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f70a40a3bb0 0x7f70a40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.476+0000 7f70aeffd640 1 -- 172.21.15.52:0/2696032410 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f70a80d0860 con 0x7f70b8101c10 2024-04-02T13:51:06.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.476+0000 7f70bdff1640 1 --2- 172.21.15.52:0/2696032410 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f70a40a3bb0 0x7f70a40a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f70b8101a70 tx=0x7f70b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:06.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.479+0000 7f70aeffd640 1 -- 172.21.15.52:0/2696032410 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f70a8014030 con 0x7f70b8101c10 2024-04-02T13:51:06.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:06 smithi088 ceph-mon[36528]: pgmap v293: 129 pgs: 42 creating+peering, 49 active+clean, 38 unknown; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 6 op/s 2024-04-02T13:51:06.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:06 smithi088 ceph-mon[36528]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T13:51:06.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:06 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/254074231' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-02T13:51:06.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:06 smithi088 ceph-mon[36528]: osdmap e89: 12 total, 12 up, 12 in 2024-04-02T13:51:06.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:06 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1408724299' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-02T13:51:06.689 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:06 smithi189 ceph-mon[36683]: pgmap v293: 129 pgs: 42 creating+peering, 49 active+clean, 38 unknown; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 6 op/s 2024-04-02T13:51:06.689 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:06 smithi189 ceph-mon[36683]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T13:51:06.690 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:06 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/254074231' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-02T13:51:06.690 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:06 smithi189 ceph-mon[36683]: osdmap e89: 12 total, 12 up, 12 in 2024-04-02T13:51:06.690 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:06 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1408724299' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-02T13:51:06.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.711+0000 7f70c0255640 1 -- 172.21.15.52:0/2696032410 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70b802d370 con 0x7f70b8101c10 2024-04-02T13:51:06.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.712+0000 7f70aeffd640 1 -- 172.21.15.52:0/2696032410 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11788 (secure 0 0 0) 0x7f70a809cfa0 con 0x7f70b8101c10 2024-04-02T13:51:06.715 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:51:06.716 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:51:05.722477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24503,"qdb_cluster":[24503]},"id":1}]} 2024-04-02T13:51:06.716 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 17 2024-04-02T13:51:06.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.714+0000 7f70c0255640 1 -- 172.21.15.52:0/2696032410 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f70a40a3bb0 msgr2=0x7f70a40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.714+0000 7f70c0255640 1 --2- 172.21.15.52:0/2696032410 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f70a40a3bb0 0x7f70a40a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f70b8101a70 tx=0x7f70b4005f90 comp rx=0 tx=0).stop 2024-04-02T13:51:06.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.714+0000 7f70c0255640 1 -- 172.21.15.52:0/2696032410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70b8101c10 msgr2=0x7f70b81099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.714+0000 7f70c0255640 1 --2- 172.21.15.52:0/2696032410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70b8101c10 0x7f70b81099e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f70a800c970 tx=0x7f70a800ce30 comp rx=0 tx=0).stop 2024-04-02T13:51:06.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.715+0000 7f70c0255640 1 -- 172.21.15.52:0/2696032410 shutdown_connections 2024-04-02T13:51:06.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.715+0000 7f70c0255640 1 --2- 172.21.15.52:0/2696032410 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f70a40a3bb0 0x7f70a40a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.715+0000 7f70c0255640 1 --2- 172.21.15.52:0/2696032410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70b81025d0 0x7f70b8109f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.715+0000 7f70c0255640 1 --2- 172.21.15.52:0/2696032410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70b8101c10 0x7f70b81099e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.715+0000 7f70c0255640 1 --2- 172.21.15.52:0/2696032410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70b8100a20 0x7f70b810b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.715+0000 7f70c0255640 1 -- 172.21.15.52:0/2696032410 >> 172.21.15.52:0/2696032410 conn(0x7f70b80fbf90 msgr2=0x7f70b8104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:06.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.715+0000 7f70c0255640 1 -- 172.21.15.52:0/2696032410 shutdown_connections 2024-04-02T13:51:06.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.715+0000 7f70c0255640 1 -- 172.21.15.52:0/2696032410 wait complete. 2024-04-02T13:51:06.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:06 smithi052 ceph-mon[32104]: pgmap v293: 129 pgs: 42 creating+peering, 49 active+clean, 38 unknown; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 6 op/s 2024-04-02T13:51:06.728 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:06 smithi052 ceph-mon[32104]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T13:51:06.728 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:06 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/254074231' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-02T13:51:06.728 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:06 smithi052 ceph-mon[32104]: osdmap e89: 12 total, 12 up, 12 in 2024-04-02T13:51:06.728 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:06 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1408724299' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-02T13:51:06.729 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:51:06.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.829+0000 7fa2a79d6640 1 -- 172.21.15.52:0/3128962610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2a01025e0 msgr2=0x7fa2a0108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.829+0000 7fa2a79d6640 1 --2- 172.21.15.52:0/3128962610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2a01025e0 0x7fa2a0108e70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa298009ca0 tx=0x7fa29802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.829+0000 7fa2a79d6640 1 -- 172.21.15.52:0/3128962610 shutdown_connections 2024-04-02T13:51:06.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.829+0000 7fa2a79d6640 1 --2- 172.21.15.52:0/3128962610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2a01025e0 0x7fa2a0108e70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.829+0000 7fa2a79d6640 1 --2- 172.21.15.52:0/3128962610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2a0101c20 0x7fa2a01020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.829+0000 7fa2a79d6640 1 --2- 172.21.15.52:0/3128962610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2a0100a30 0x7fa2a0100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.829+0000 7fa2a79d6640 1 -- 172.21.15.52:0/3128962610 >> 172.21.15.52:0/3128962610 conn(0x7fa2a00fbfe0 msgr2=0x7fa2a00fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:06.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.829+0000 7fa2a79d6640 1 -- 172.21.15.52:0/3128962610 shutdown_connections 2024-04-02T13:51:06.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.829+0000 7fa2a79d6640 1 -- 172.21.15.52:0/3128962610 wait complete. 2024-04-02T13:51:06.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.829+0000 7fa2a79d6640 1 Processor -- start 2024-04-02T13:51:06.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.829+0000 7fa2a79d6640 1 -- start start 2024-04-02T13:51:06.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.830+0000 7fa2a79d6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2a0100a30 0x7fa2a006ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.830+0000 7fa2a79d6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2a0101c20 0x7fa2a019cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.830+0000 7fa2a79d6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2a01025e0 0x7fa2a019f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.830+0000 7fa2a79d6640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a010c6b0 con 0x7fa2a01025e0 2024-04-02T13:51:06.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.830+0000 7fa2a79d6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a010c530 con 0x7fa2a0100a30 2024-04-02T13:51:06.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.830+0000 7fa2a79d6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a010c830 con 0x7fa2a0101c20 2024-04-02T13:51:06.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.830+0000 7fa2a5772640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2a0100a30 0x7fa2a006ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.830+0000 7fa2a5f73640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2a01025e0 0x7fa2a019f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.830+0000 7fa2a5772640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2a0100a30 0x7fa2a006ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:42526/0 (socket says 172.21.15.52:42526) 2024-04-02T13:51:06.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.830+0000 7fa2a5772640 1 -- 172.21.15.52:0/1570201955 learned_addr learned my addr 172.21.15.52:0/1570201955 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:06.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.830+0000 7fa2a5f73640 1 --2- 172.21.15.52:0/1570201955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2a01025e0 0x7fa2a019f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53980/0 (socket says 172.21.15.52:53980) 2024-04-02T13:51:06.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.830+0000 7fa2a4f71640 1 --2- 172.21.15.52:0/1570201955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2a0101c20 0x7fa2a019cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.830+0000 7fa2a5f73640 1 -- 172.21.15.52:0/1570201955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2a0101c20 msgr2=0x7fa2a019cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.830+0000 7fa2a5f73640 1 --2- 172.21.15.52:0/1570201955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2a0101c20 0x7fa2a019cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.830+0000 7fa2a5f73640 1 -- 172.21.15.52:0/1570201955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2a0100a30 msgr2=0x7fa2a006ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.830+0000 7fa2a5f73640 1 --2- 172.21.15.52:0/1570201955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2a0100a30 0x7fa2a006ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.830+0000 7fa2a5f73640 1 -- 172.21.15.52:0/1570201955 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a019fa30 con 0x7fa2a01025e0 2024-04-02T13:51:06.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.830+0000 7fa2a5772640 1 --2- 172.21.15.52:0/1570201955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2a0100a30 0x7fa2a006ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:06.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.830+0000 7fa2a5f73640 1 --2- 172.21.15.52:0/1570201955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2a01025e0 0x7fa2a019f170 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa29802c630 tx=0x7fa298032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:06.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.830+0000 7fa2967fc640 1 -- 172.21.15.52:0/1570201955 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298039040 con 0x7fa2a01025e0 2024-04-02T13:51:06.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.830+0000 7fa2967fc640 1 -- 172.21.15.52:0/1570201955 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa298044070 con 0x7fa2a01025e0 2024-04-02T13:51:06.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.831+0000 7fa2a79d6640 1 -- 172.21.15.52:0/1570201955 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a019fcc0 con 0x7fa2a01025e0 2024-04-02T13:51:06.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.831+0000 7fa2967fc640 1 -- 172.21.15.52:0/1570201955 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2980042a0 con 0x7fa2a01025e0 2024-04-02T13:51:06.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.831+0000 7fa2a79d6640 1 -- 172.21.15.52:0/1570201955 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a01a01d0 con 0x7fa2a01025e0 2024-04-02T13:51:06.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.831+0000 7fa2a79d6640 1 -- 172.21.15.52:0/1570201955 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a0107d50 con 0x7fa2a01025e0 2024-04-02T13:51:06.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.832+0000 7fa2967fc640 1 -- 172.21.15.52:0/1570201955 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa298032690 con 0x7fa2a01025e0 2024-04-02T13:51:06.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.833+0000 7fa2967fc640 1 --2- 172.21.15.52:0/1570201955 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa28c0a3c30 0x7fa28c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.833+0000 7fa2a5772640 1 --2- 172.21.15.52:0/1570201955 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa28c0a3c30 0x7fa28c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.833+0000 7fa2967fc640 1 -- 172.21.15.52:0/1570201955 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa2980f2930 con 0x7fa2a01025e0 2024-04-02T13:51:06.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.834+0000 7fa2a5772640 1 --2- 172.21.15.52:0/1570201955 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa28c0a3c30 0x7fa28c0a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa29c0058c0 tx=0x7fa29c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:06.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:06.838+0000 7fa2967fc640 1 -- 172.21.15.52:0/1570201955 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa2980bf160 con 0x7fa2a01025e0 2024-04-02T13:51:07.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.025+0000 7fa2a79d6640 1 -- 172.21.15.52:0/1570201955 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2a0062700 con 0x7fa2a01025e0 2024-04-02T13:51:07.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.026+0000 7fa2967fc640 1 -- 172.21.15.52:0/1570201955 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24077 (secure 0 0 0) 0x7fa2980bea60 con 0x7fa2a01025e0 2024-04-02T13:51:07.030 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:51:07.030 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":90,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T13:51:05.722301+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:50:58.778690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:50:59.561527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:51:03.343487+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:51:07.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.028+0000 7fa2a79d6640 1 -- 172.21.15.52:0/1570201955 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa28c0a3c30 msgr2=0x7fa28c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.028+0000 7fa2a79d6640 1 --2- 172.21.15.52:0/1570201955 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa28c0a3c30 0x7fa28c0a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa29c0058c0 tx=0x7fa29c00a300 comp rx=0 tx=0).stop 2024-04-02T13:51:07.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.028+0000 7fa2a79d6640 1 -- 172.21.15.52:0/1570201955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2a01025e0 msgr2=0x7fa2a019f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.028+0000 7fa2a79d6640 1 --2- 172.21.15.52:0/1570201955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2a01025e0 0x7fa2a019f170 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa29802c630 tx=0x7fa298032f20 comp rx=0 tx=0).stop 2024-04-02T13:51:07.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.029+0000 7fa2a79d6640 1 -- 172.21.15.52:0/1570201955 shutdown_connections 2024-04-02T13:51:07.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.029+0000 7fa2a79d6640 1 --2- 172.21.15.52:0/1570201955 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa28c0a3c30 0x7fa28c0a60f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.029+0000 7fa2a79d6640 1 --2- 172.21.15.52:0/1570201955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2a01025e0 0x7fa2a019f170 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.029+0000 7fa2a79d6640 1 --2- 172.21.15.52:0/1570201955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2a0101c20 0x7fa2a019cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.029+0000 7fa2a79d6640 1 --2- 172.21.15.52:0/1570201955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2a0100a30 0x7fa2a006ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.029+0000 7fa2a79d6640 1 -- 172.21.15.52:0/1570201955 >> 172.21.15.52:0/1570201955 conn(0x7fa2a00fbfe0 msgr2=0x7fa2a0104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:07.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.029+0000 7fa2a79d6640 1 -- 172.21.15.52:0/1570201955 shutdown_connections 2024-04-02T13:51:07.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.029+0000 7fa2a79d6640 1 -- 172.21.15.52:0/1570201955 wait complete. 2024-04-02T13:51:07.043 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:51:07.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.143+0000 7f940aaa0640 1 -- 172.21.15.52:0/1938885301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9404100540 msgr2=0x7f940410a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.143+0000 7f940aaa0640 1 --2- 172.21.15.52:0/1938885301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9404100540 0x7f940410a0f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009900 tx=0x7f93ec02bc30 comp rx=0 tx=0).stop 2024-04-02T13:51:07.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.143+0000 7f940aaa0640 1 -- 172.21.15.52:0/1938885301 shutdown_connections 2024-04-02T13:51:07.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.143+0000 7f940aaa0640 1 --2- 172.21.15.52:0/1938885301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9404100540 0x7f940410a0f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.143+0000 7f940aaa0640 1 --2- 172.21.15.52:0/1938885301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94040ffb80 0x7f9404100000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.143+0000 7f940aaa0640 1 --2- 172.21.15.52:0/1938885301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9404103110 0x7f9404103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.143+0000 7f940aaa0640 1 -- 172.21.15.52:0/1938885301 >> 172.21.15.52:0/1938885301 conn(0x7f94040fb720 msgr2=0x7f94040fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:07.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.143+0000 7f940aaa0640 1 -- 172.21.15.52:0/1938885301 shutdown_connections 2024-04-02T13:51:07.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.143+0000 7f940aaa0640 1 -- 172.21.15.52:0/1938885301 wait complete. 2024-04-02T13:51:07.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.143+0000 7f940aaa0640 1 Processor -- start 2024-04-02T13:51:07.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.143+0000 7f940aaa0640 1 -- start start 2024-04-02T13:51:07.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.143+0000 7f940aaa0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94040ffb80 0x7f9404102810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.143+0000 7f940aaa0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9404100540 0x7f9404100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.143+0000 7f940aaa0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9404103110 0x7f9404101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.143+0000 7f940aaa0640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f940410d880 con 0x7f9404100540 2024-04-02T13:51:07.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.143+0000 7f940aaa0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f940410d700 con 0x7f94040ffb80 2024-04-02T13:51:07.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.143+0000 7f940aaa0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f940410da00 con 0x7f9404103110 2024-04-02T13:51:07.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.143+0000 7f9403fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9404100540 0x7f9404100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.144+0000 7f9403fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9404100540 0x7f9404100e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54002/0 (socket says 172.21.15.52:54002) 2024-04-02T13:51:07.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.144+0000 7f9403fff640 1 -- 172.21.15.52:0/59457488 learned_addr learned my addr 172.21.15.52:0/59457488 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:07.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.144+0000 7f940883c640 1 --2- 172.21.15.52:0/59457488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94040ffb80 0x7f9404102810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.144+0000 7f940903d640 1 --2- 172.21.15.52:0/59457488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9404103110 0x7f9404101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.144+0000 7f9403fff640 1 -- 172.21.15.52:0/59457488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9404103110 msgr2=0x7f9404101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.144+0000 7f9403fff640 1 --2- 172.21.15.52:0/59457488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9404103110 0x7f9404101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.144+0000 7f9403fff640 1 -- 172.21.15.52:0/59457488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94040ffb80 msgr2=0x7f9404102810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.144+0000 7f9403fff640 1 --2- 172.21.15.52:0/59457488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94040ffb80 0x7f9404102810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.144+0000 7f9403fff640 1 -- 172.21.15.52:0/59457488 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9404101c20 con 0x7f9404100540 2024-04-02T13:51:07.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.144+0000 7f940883c640 1 --2- 172.21.15.52:0/59457488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94040ffb80 0x7f9404102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:51:07.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.144+0000 7f9403fff640 1 --2- 172.21.15.52:0/59457488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9404100540 0x7f9404100e50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f93f400ba50 tx=0x7f93f400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:07.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.144+0000 7f9401ffb640 1 -- 172.21.15.52:0/59457488 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4019070 con 0x7f9404100540 2024-04-02T13:51:07.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.144+0000 7f9401ffb640 1 -- 172.21.15.52:0/59457488 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93f400cbe0 con 0x7f9404100540 2024-04-02T13:51:07.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.144+0000 7f940aaa0640 1 -- 172.21.15.52:0/59457488 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94041a1f50 con 0x7f9404100540 2024-04-02T13:51:07.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.144+0000 7f9401ffb640 1 -- 172.21.15.52:0/59457488 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f40076e0 con 0x7f9404100540 2024-04-02T13:51:07.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.144+0000 7f940aaa0640 1 -- 172.21.15.52:0/59457488 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94041a23b0 con 0x7f9404100540 2024-04-02T13:51:07.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.145+0000 7f940aaa0640 1 -- 172.21.15.52:0/59457488 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9404108f70 con 0x7f9404100540 2024-04-02T13:51:07.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.146+0000 7f9401ffb640 1 -- 172.21.15.52:0/59457488 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f93f401c070 con 0x7f9404100540 2024-04-02T13:51:07.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.147+0000 7f9401ffb640 1 --2- 172.21.15.52:0/59457488 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f93f00a3bb0 0x7f93f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.147+0000 7f940883c640 1 --2- 172.21.15.52:0/59457488 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f93f00a3bb0 0x7f93f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.147+0000 7f9401ffb640 1 -- 172.21.15.52:0/59457488 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f93f40cfc50 con 0x7f9404100540 2024-04-02T13:51:07.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.147+0000 7f940883c640 1 --2- 172.21.15.52:0/59457488 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f93f00a3bb0 0x7f93f00a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0058c0 tx=0x7f93fc005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:07.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.151+0000 7f9401ffb640 1 -- 172.21.15.52:0/59457488 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f93f40d5020 con 0x7f9404100540 2024-04-02T13:51:07.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.339+0000 7f940aaa0640 1 -- 172.21.15.52:0/59457488 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94040627a0 con 0x7f9404100540 2024-04-02T13:51:07.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.340+0000 7f9401ffb640 1 -- 172.21.15.52:0/59457488 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24077 (secure 0 0 0) 0x7f93f409c3f0 con 0x7f9404100540 2024-04-02T13:51:07.344 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:51:07.344 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":90,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T13:51:05.722301+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:50:58.778690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:50:59.561527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:51:03.343487+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:51:07.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.342+0000 7f940aaa0640 1 -- 172.21.15.52:0/59457488 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f93f00a3bb0 msgr2=0x7f93f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.342+0000 7f940aaa0640 1 --2- 172.21.15.52:0/59457488 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f93f00a3bb0 0x7f93f00a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0058c0 tx=0x7f93fc005ec0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.342+0000 7f940aaa0640 1 -- 172.21.15.52:0/59457488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9404100540 msgr2=0x7f9404100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.342+0000 7f940aaa0640 1 --2- 172.21.15.52:0/59457488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9404100540 0x7f9404100e50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f93f400ba50 tx=0x7f93f400bf10 comp rx=0 tx=0).stop 2024-04-02T13:51:07.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.342+0000 7f940aaa0640 1 -- 172.21.15.52:0/59457488 shutdown_connections 2024-04-02T13:51:07.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.342+0000 7f940aaa0640 1 --2- 172.21.15.52:0/59457488 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f93f00a3bb0 0x7f93f00a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.342+0000 7f940aaa0640 1 --2- 172.21.15.52:0/59457488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9404103110 0x7f9404101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.342+0000 7f940aaa0640 1 --2- 172.21.15.52:0/59457488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9404100540 0x7f9404100e50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.342+0000 7f940aaa0640 1 --2- 172.21.15.52:0/59457488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94040ffb80 0x7f9404102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.342+0000 7f940aaa0640 1 -- 172.21.15.52:0/59457488 >> 172.21.15.52:0/59457488 conn(0x7f94040fb720 msgr2=0x7f9404106140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:07.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.342+0000 7f940aaa0640 1 -- 172.21.15.52:0/59457488 shutdown_connections 2024-04-02T13:51:07.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.342+0000 7f940aaa0640 1 -- 172.21.15.52:0/59457488 wait complete. 2024-04-02T13:51:07.373 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:51:07.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.473+0000 7fd977493640 1 -- 172.21.15.52:0/3881550813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd970100390 msgr2=0x7fd9701007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.473+0000 7fd977493640 1 --2- 172.21.15.52:0/3881550813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd970100390 0x7fd9701007b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009900 tx=0x7fd96c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:51:07.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.473+0000 7fd977493640 1 -- 172.21.15.52:0/3881550813 shutdown_connections 2024-04-02T13:51:07.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.473+0000 7fd977493640 1 --2- 172.21.15.52:0/3881550813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd970101690 0x7fd970101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.473+0000 7fd977493640 1 --2- 172.21.15.52:0/3881550813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd970100cf0 0x7fd970101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.473+0000 7fd977493640 1 --2- 172.21.15.52:0/3881550813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd970100390 0x7fd9701007b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.473+0000 7fd977493640 1 -- 172.21.15.52:0/3881550813 >> 172.21.15.52:0/3881550813 conn(0x7fd9700fbfb0 msgr2=0x7fd9700fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:07.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.473+0000 7fd977493640 1 -- 172.21.15.52:0/3881550813 shutdown_connections 2024-04-02T13:51:07.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.473+0000 7fd977493640 1 -- 172.21.15.52:0/3881550813 wait complete. 2024-04-02T13:51:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.474+0000 7fd977493640 1 Processor -- start 2024-04-02T13:51:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.474+0000 7fd977493640 1 -- start start 2024-04-02T13:51:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.474+0000 7fd977493640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd970100390 0x7fd97019a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.474+0000 7fd977493640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd970100cf0 0x7fd97019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.474+0000 7fd977493640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd970101690 0x7fd9701a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.474+0000 7fd977493640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd97010f960 con 0x7fd970101690 2024-04-02T13:51:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.474+0000 7fd977493640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd97010f7e0 con 0x7fd970100cf0 2024-04-02T13:51:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.474+0000 7fd977493640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd97010fae0 con 0x7fd970100390 2024-04-02T13:51:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.474+0000 7fd975a30640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd970101690 0x7fd9701a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.474+0000 7fd975a30640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd970101690 0x7fd9701a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54032/0 (socket says 172.21.15.52:54032) 2024-04-02T13:51:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.474+0000 7fd975a30640 1 -- 172.21.15.52:0/3448846514 learned_addr learned my addr 172.21.15.52:0/3448846514 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.474+0000 7fd974a2e640 1 --2- 172.21.15.52:0/3448846514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd970100cf0 0x7fd97019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.474+0000 7fd97522f640 1 --2- 172.21.15.52:0/3448846514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd970100390 0x7fd97019a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.474+0000 7fd975a30640 1 -- 172.21.15.52:0/3448846514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd970100390 msgr2=0x7fd97019a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.474+0000 7fd975a30640 1 --2- 172.21.15.52:0/3448846514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd970100390 0x7fd97019a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.474+0000 7fd975a30640 1 -- 172.21.15.52:0/3448846514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd970100cf0 msgr2=0x7fd97019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.474+0000 7fd975a30640 1 --2- 172.21.15.52:0/3448846514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd970100cf0 0x7fd97019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.474+0000 7fd975a30640 1 -- 172.21.15.52:0/3448846514 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9701a1c50 con 0x7fd970101690 2024-04-02T13:51:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.474+0000 7fd974a2e640 1 --2- 172.21.15.52:0/3448846514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd970100cf0 0x7fd97019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.475+0000 7fd975a30640 1 --2- 172.21.15.52:0/3448846514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd970101690 0x7fd9701a15e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd968009fd0 tx=0x7fd96800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.475+0000 7fd9667fc640 1 -- 172.21.15.52:0/3448846514 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd968017070 con 0x7fd970101690 2024-04-02T13:51:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.475+0000 7fd9667fc640 1 -- 172.21.15.52:0/3448846514 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd968003e50 con 0x7fd970101690 2024-04-02T13:51:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.475+0000 7fd977493640 1 -- 172.21.15.52:0/3448846514 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9701a1f40 con 0x7fd970101690 2024-04-02T13:51:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.475+0000 7fd9667fc640 1 -- 172.21.15.52:0/3448846514 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9680106c0 con 0x7fd970101690 2024-04-02T13:51:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.475+0000 7fd977493640 1 -- 172.21.15.52:0/3448846514 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9701a2400 con 0x7fd970101690 2024-04-02T13:51:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.476+0000 7fd977493640 1 -- 172.21.15.52:0/3448846514 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd97010b2e0 con 0x7fd970101690 2024-04-02T13:51:07.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.477+0000 7fd9667fc640 1 -- 172.21.15.52:0/3448846514 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd968003970 con 0x7fd970101690 2024-04-02T13:51:07.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.477+0000 7fd9667fc640 1 --2- 172.21.15.52:0/3448846514 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fd95c0a3c30 0x7fd95c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.478+0000 7fd97522f640 1 --2- 172.21.15.52:0/3448846514 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fd95c0a3c30 0x7fd95c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.478+0000 7fd9667fc640 1 -- 172.21.15.52:0/3448846514 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fd9680d0360 con 0x7fd970101690 2024-04-02T13:51:07.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.478+0000 7fd97522f640 1 --2- 172.21.15.52:0/3448846514 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fd95c0a3c30 0x7fd95c0a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd96c006010 tx=0x7fd96c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:07.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.482+0000 7fd9667fc640 1 -- 172.21.15.52:0/3448846514 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd96809cb00 con 0x7fd970101690 2024-04-02T13:51:07.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:07 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1408724299' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-02T13:51:07.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:07 smithi088 ceph-mon[36528]: osdmap e90: 12 total, 12 up, 12 in 2024-04-02T13:51:07.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:07 smithi088 ceph-mon[36528]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:51:07.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:07 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2696032410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:51:07.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:07 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1570201955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:07.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:07 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/59457488' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:07.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:07 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1408724299' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-02T13:51:07.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:07 smithi189 ceph-mon[36683]: osdmap e90: 12 total, 12 up, 12 in 2024-04-02T13:51:07.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:07 smithi189 ceph-mon[36683]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:51:07.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:07 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2696032410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:51:07.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:07 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1570201955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:07.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:07 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/59457488' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:07.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.711+0000 7fd977493640 1 -- 172.21.15.52:0/3448846514 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd97002d370 con 0x7fd970101690 2024-04-02T13:51:07.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.711+0000 7fd9667fc640 1 -- 172.21.15.52:0/3448846514 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11788 (secure 0 0 0) 0x7fd96809c400 con 0x7fd970101690 2024-04-02T13:51:07.715 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:51:07.715 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:51:05.722477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24503,"qdb_cluster":[24503]},"id":1}]} 2024-04-02T13:51:07.715 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 17 2024-04-02T13:51:07.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.713+0000 7fd977493640 1 -- 172.21.15.52:0/3448846514 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fd95c0a3c30 msgr2=0x7fd95c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.713+0000 7fd977493640 1 --2- 172.21.15.52:0/3448846514 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fd95c0a3c30 0x7fd95c0a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd96c006010 tx=0x7fd96c000bc0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.714+0000 7fd977493640 1 -- 172.21.15.52:0/3448846514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd970101690 msgr2=0x7fd9701a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.714+0000 7fd977493640 1 --2- 172.21.15.52:0/3448846514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd970101690 0x7fd9701a15e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd968009fd0 tx=0x7fd96800b260 comp rx=0 tx=0).stop 2024-04-02T13:51:07.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.714+0000 7fd977493640 1 -- 172.21.15.52:0/3448846514 shutdown_connections 2024-04-02T13:51:07.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.714+0000 7fd977493640 1 --2- 172.21.15.52:0/3448846514 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fd95c0a3c30 0x7fd95c0a60f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.714+0000 7fd977493640 1 --2- 172.21.15.52:0/3448846514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd970101690 0x7fd9701a15e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.714+0000 7fd977493640 1 --2- 172.21.15.52:0/3448846514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd970100cf0 0x7fd97019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.714+0000 7fd977493640 1 --2- 172.21.15.52:0/3448846514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd970100390 0x7fd97019a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.714+0000 7fd977493640 1 -- 172.21.15.52:0/3448846514 >> 172.21.15.52:0/3448846514 conn(0x7fd9700fbfb0 msgr2=0x7fd9700fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:07.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.714+0000 7fd977493640 1 -- 172.21.15.52:0/3448846514 shutdown_connections 2024-04-02T13:51:07.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.714+0000 7fd977493640 1 -- 172.21.15.52:0/3448846514 wait complete. 2024-04-02T13:51:07.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:07 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1408724299' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-02T13:51:07.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:07 smithi052 ceph-mon[32104]: osdmap e90: 12 total, 12 up, 12 in 2024-04-02T13:51:07.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:07 smithi052 ceph-mon[32104]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:51:07.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:07 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2696032410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:51:07.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:07 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1570201955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:07.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:07 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/59457488' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:07.728 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:51:07.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.828+0000 7fbdb6a42640 1 -- 172.21.15.52:0/887850652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdb0101c90 msgr2=0x7fbdb0102110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.828+0000 7fbdb6a42640 1 --2- 172.21.15.52:0/887850652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdb0101c90 0x7fbdb0102110 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbda4009900 tx=0x7fbda402bc30 comp rx=0 tx=0).stop 2024-04-02T13:51:07.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.828+0000 7fbdb6a42640 1 -- 172.21.15.52:0/887850652 shutdown_connections 2024-04-02T13:51:07.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.828+0000 7fbdb6a42640 1 --2- 172.21.15.52:0/887850652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdb0102650 0x7fbdb0108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.828+0000 7fbdb6a42640 1 --2- 172.21.15.52:0/887850652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdb0101c90 0x7fbdb0102110 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.828+0000 7fbdb6a42640 1 --2- 172.21.15.52:0/887850652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbdb0100aa0 0x7fbdb0100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.828+0000 7fbdb6a42640 1 -- 172.21.15.52:0/887850652 >> 172.21.15.52:0/887850652 conn(0x7fbdb00fc050 msgr2=0x7fbdb00fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:07.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.828+0000 7fbdb6a42640 1 -- 172.21.15.52:0/887850652 shutdown_connections 2024-04-02T13:51:07.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.828+0000 7fbdb6a42640 1 -- 172.21.15.52:0/887850652 wait complete. 2024-04-02T13:51:07.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.829+0000 7fbdb6a42640 1 Processor -- start 2024-04-02T13:51:07.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.829+0000 7fbdb6a42640 1 -- start start 2024-04-02T13:51:07.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.829+0000 7fbdb6a42640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbdb0100aa0 0x7fbdb0197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.829+0000 7fbdb6a42640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdb0101c90 0x7fbdb0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.829+0000 7fbdb6a42640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdb0102650 0x7fbdb019f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.829+0000 7fbdb6a42640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb010d7b0 con 0x7fbdb0102650 2024-04-02T13:51:07.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.829+0000 7fbdb6a42640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb010d630 con 0x7fbdb0101c90 2024-04-02T13:51:07.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.829+0000 7fbdb6a42640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb010d930 con 0x7fbdb0100aa0 2024-04-02T13:51:07.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.829+0000 7fbdb4fdf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdb0102650 0x7fbdb019f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.829+0000 7fbdb4fdf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdb0102650 0x7fbdb019f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54054/0 (socket says 172.21.15.52:54054) 2024-04-02T13:51:07.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.829+0000 7fbdb4fdf640 1 -- 172.21.15.52:0/3074978382 learned_addr learned my addr 172.21.15.52:0/3074978382 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:07.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.829+0000 7fbdaf7fe640 1 --2- 172.21.15.52:0/3074978382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdb0101c90 0x7fbdb0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.829+0000 7fbdaffff640 1 --2- 172.21.15.52:0/3074978382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbdb0100aa0 0x7fbdb0197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.829+0000 7fbdb4fdf640 1 -- 172.21.15.52:0/3074978382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbdb0100aa0 msgr2=0x7fbdb0197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.829+0000 7fbdb4fdf640 1 --2- 172.21.15.52:0/3074978382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbdb0100aa0 0x7fbdb0197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.829+0000 7fbdb4fdf640 1 -- 172.21.15.52:0/3074978382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdb0101c90 msgr2=0x7fbdb0198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.829+0000 7fbdb4fdf640 1 --2- 172.21.15.52:0/3074978382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdb0101c90 0x7fbdb0198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.829+0000 7fbdb4fdf640 1 -- 172.21.15.52:0/3074978382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb019fa10 con 0x7fbdb0102650 2024-04-02T13:51:07.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.829+0000 7fbdb4fdf640 1 --2- 172.21.15.52:0/3074978382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdb0102650 0x7fbdb019f3a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbda800eb10 tx=0x7fbda800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:07.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.830+0000 7fbdad7fa640 1 -- 172.21.15.52:0/3074978382 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda8004b20 con 0x7fbdb0102650 2024-04-02T13:51:07.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.830+0000 7fbdb6a42640 1 -- 172.21.15.52:0/3074978382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb019fd00 con 0x7fbdb0102650 2024-04-02T13:51:07.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.830+0000 7fbdad7fa640 1 -- 172.21.15.52:0/3074978382 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbda8004cc0 con 0x7fbdb0102650 2024-04-02T13:51:07.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.830+0000 7fbdad7fa640 1 -- 172.21.15.52:0/3074978382 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda8010760 con 0x7fbdb0102650 2024-04-02T13:51:07.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.830+0000 7fbdb6a42640 1 -- 172.21.15.52:0/3074978382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb01a01d0 con 0x7fbdb0102650 2024-04-02T13:51:07.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.831+0000 7fbdb6a42640 1 -- 172.21.15.52:0/3074978382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdb0107dc0 con 0x7fbdb0102650 2024-04-02T13:51:07.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.832+0000 7fbdad7fa640 1 -- 172.21.15.52:0/3074978382 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbda8005000 con 0x7fbdb0102650 2024-04-02T13:51:07.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.832+0000 7fbdad7fa640 1 --2- 172.21.15.52:0/3074978382 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fbd980a3bb0 0x7fbd980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.832+0000 7fbdaffff640 1 --2- 172.21.15.52:0/3074978382 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fbd980a3bb0 0x7fbd980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.833+0000 7fbdad7fa640 1 -- 172.21.15.52:0/3074978382 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fbda80d0e30 con 0x7fbdb0102650 2024-04-02T13:51:07.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.833+0000 7fbdaffff640 1 --2- 172.21.15.52:0/3074978382 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fbd980a3bb0 0x7fbd980a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0101af0 tx=0x7fbda0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:07.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:07.836+0000 7fbdad7fa640 1 -- 172.21.15.52:0/3074978382 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fbda8053f20 con 0x7fbdb0102650 2024-04-02T13:51:08.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.025+0000 7fbdb6a42640 1 -- 172.21.15.52:0/3074978382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbdb002d370 con 0x7fbdb0102650 2024-04-02T13:51:08.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.027+0000 7fbdad7fa640 1 -- 172.21.15.52:0/3074978382 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24077 (secure 0 0 0) 0x7fbda809d570 con 0x7fbdb0102650 2024-04-02T13:51:08.030 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:51:08.030 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":90,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T13:51:05.722301+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:50:58.778690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:50:59.561527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:51:03.343487+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:51:08.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.029+0000 7fbdb6a42640 1 -- 172.21.15.52:0/3074978382 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fbd980a3bb0 msgr2=0x7fbd980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.029+0000 7fbdb6a42640 1 --2- 172.21.15.52:0/3074978382 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fbd980a3bb0 0x7fbd980a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0101af0 tx=0x7fbda0005f90 comp rx=0 tx=0).stop 2024-04-02T13:51:08.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.029+0000 7fbdb6a42640 1 -- 172.21.15.52:0/3074978382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdb0102650 msgr2=0x7fbdb019f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.029+0000 7fbdb6a42640 1 --2- 172.21.15.52:0/3074978382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdb0102650 0x7fbdb019f3a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbda800eb10 tx=0x7fbda800c2d0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.029+0000 7fbdb6a42640 1 -- 172.21.15.52:0/3074978382 shutdown_connections 2024-04-02T13:51:08.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.029+0000 7fbdb6a42640 1 --2- 172.21.15.52:0/3074978382 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fbd980a3bb0 0x7fbd980a6070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.029+0000 7fbdb6a42640 1 --2- 172.21.15.52:0/3074978382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdb0102650 0x7fbdb019f3a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.029+0000 7fbdb6a42640 1 --2- 172.21.15.52:0/3074978382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdb0101c90 0x7fbdb0198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.029+0000 7fbdb6a42640 1 --2- 172.21.15.52:0/3074978382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbdb0100aa0 0x7fbdb0197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.029+0000 7fbdb6a42640 1 -- 172.21.15.52:0/3074978382 >> 172.21.15.52:0/3074978382 conn(0x7fbdb00fc050 msgr2=0x7fbdb0104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:08.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.029+0000 7fbdb6a42640 1 -- 172.21.15.52:0/3074978382 shutdown_connections 2024-04-02T13:51:08.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.029+0000 7fbdb6a42640 1 -- 172.21.15.52:0/3074978382 wait complete. 2024-04-02T13:51:08.045 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-02T13:51:00.603246+0000', 'modified': '2024-04-02T13:51:05.722477+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24503}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24503': {'gid': 24503, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.88:6839/2513527493', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6838', 'nonce': 2513527493}, {'type': 'v1', 'addr': '172.21.15.88:6839', 'nonce': 2513527493}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24503, 'qdb_cluster': [24503]} 2024-04-02T13:51:08.045 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-02T13:51:08.045 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-02T13:51:08.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.145+0000 7f24f7070640 1 -- 172.21.15.52:0/389514587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24f01025c0 msgr2=0x7f24f0108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.145+0000 7f24f7070640 1 --2- 172.21.15.52:0/389514587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24f01025c0 0x7f24f0108e50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009ca0 tx=0x7f24e802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.145+0000 7f24f7070640 1 -- 172.21.15.52:0/389514587 shutdown_connections 2024-04-02T13:51:08.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.145+0000 7f24f7070640 1 --2- 172.21.15.52:0/389514587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24f01025c0 0x7f24f0108e50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.145+0000 7f24f7070640 1 --2- 172.21.15.52:0/389514587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24f0101c00 0x7f24f0102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.145+0000 7f24f7070640 1 --2- 172.21.15.52:0/389514587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24f0100a10 0x7f24f0100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.145+0000 7f24f7070640 1 -- 172.21.15.52:0/389514587 >> 172.21.15.52:0/389514587 conn(0x7f24f00fbfa0 msgr2=0x7f24f00fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:08.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.146+0000 7f24f7070640 1 -- 172.21.15.52:0/389514587 shutdown_connections 2024-04-02T13:51:08.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.146+0000 7f24f7070640 1 -- 172.21.15.52:0/389514587 wait complete. 2024-04-02T13:51:08.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.146+0000 7f24f7070640 1 Processor -- start 2024-04-02T13:51:08.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.146+0000 7f24f7070640 1 -- start start 2024-04-02T13:51:08.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.146+0000 7f24f7070640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24f0100a10 0x7f24f0197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.146+0000 7f24f7070640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24f0101c00 0x7f24f0198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.146+0000 7f24f7070640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24f01025c0 0x7f24f019f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.146+0000 7f24f7070640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f24f010c690 con 0x7f24f0101c00 2024-04-02T13:51:08.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.146+0000 7f24f7070640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f24f010c510 con 0x7f24f0100a10 2024-04-02T13:51:08.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.146+0000 7f24f7070640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f24f010c810 con 0x7f24f01025c0 2024-04-02T13:51:08.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.146+0000 7f24effff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24f0101c00 0x7f24f0198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.146+0000 7f24effff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24f0101c00 0x7f24f0198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54082/0 (socket says 172.21.15.52:54082) 2024-04-02T13:51:08.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.146+0000 7f24effff640 1 -- 172.21.15.52:0/3934098834 learned_addr learned my addr 172.21.15.52:0/3934098834 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:08.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.146+0000 7f24f4e0c640 1 --2- 172.21.15.52:0/3934098834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24f0100a10 0x7f24f0197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.146+0000 7f24f560d640 1 --2- 172.21.15.52:0/3934098834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24f01025c0 0x7f24f019f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.147+0000 7f24effff640 1 -- 172.21.15.52:0/3934098834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24f01025c0 msgr2=0x7f24f019f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.147+0000 7f24effff640 1 --2- 172.21.15.52:0/3934098834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24f01025c0 0x7f24f019f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.147+0000 7f24effff640 1 -- 172.21.15.52:0/3934098834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24f0100a10 msgr2=0x7f24f0197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.147+0000 7f24effff640 1 --2- 172.21.15.52:0/3934098834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24f0100a10 0x7f24f0197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.147+0000 7f24effff640 1 -- 172.21.15.52:0/3934098834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f019fa00 con 0x7f24f0101c00 2024-04-02T13:51:08.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.147+0000 7f24f560d640 1 --2- 172.21.15.52:0/3934098834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24f01025c0 0x7f24f019f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:51:08.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.147+0000 7f24effff640 1 --2- 172.21.15.52:0/3934098834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24f0101c00 0x7f24f0198320 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f24e400e9f0 tx=0x7f24e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:08.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.147+0000 7f24edffb640 1 -- 172.21.15.52:0/3934098834 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e400cd80 con 0x7f24f0101c00 2024-04-02T13:51:08.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.147+0000 7f24edffb640 1 -- 172.21.15.52:0/3934098834 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24e4010910 con 0x7f24f0101c00 2024-04-02T13:51:08.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.147+0000 7f24edffb640 1 -- 172.21.15.52:0/3934098834 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e4018990 con 0x7f24f0101c00 2024-04-02T13:51:08.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.147+0000 7f24f7070640 1 -- 172.21.15.52:0/3934098834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f019fcf0 con 0x7f24f0101c00 2024-04-02T13:51:08.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.147+0000 7f24f7070640 1 -- 172.21.15.52:0/3934098834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f01a0190 con 0x7f24f0101c00 2024-04-02T13:51:08.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.150+0000 7f24edffb640 1 -- 172.21.15.52:0/3934098834 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f24e4010ac0 con 0x7f24f0101c00 2024-04-02T13:51:08.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.150+0000 7f24f9c56740 1 Processor -- start 2024-04-02T13:51:08.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.150+0000 7f24f9c56740 1 -- start start 2024-04-02T13:51:08.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.150+0000 7f24f9c56740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e07d422ae0 0x55e07d422f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.150+0000 7f24f9c56740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07d4234d0 0x55e07d427fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.150+0000 7f24f9c56740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d428640 0x55e07d42aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.150+0000 7f24f9c56740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e07d387060 con 0x55e07d428640 2024-04-02T13:51:08.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.150+0000 7f24f9c56740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e07d32f650 con 0x55e07d422ae0 2024-04-02T13:51:08.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.150+0000 7f24f9c56740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e07d383010 con 0x55e07d4234d0 2024-04-02T13:51:08.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.150+0000 7f24effff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d428640 0x55e07d42aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.150+0000 7f24effff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d428640 0x55e07d42aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54094/0 (socket says 172.21.15.52:54094) 2024-04-02T13:51:08.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.150+0000 7f24effff640 1 -- 172.21.15.52:0/3971506072 learned_addr learned my addr 172.21.15.52:0/3971506072 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:08.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.151+0000 7f24f4e0c640 1 --2- 172.21.15.52:0/3971506072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07d4234d0 0x55e07d427fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.151+0000 7f24f560d640 1 --2- 172.21.15.52:0/3971506072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e07d422ae0 0x55e07d422f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.151+0000 7f24edffb640 1 --2- 172.21.15.52:0/3934098834 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f24d80a3bb0 0x7f24d80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.151+0000 7f24f560d640 1 --2- 172.21.15.52:0/3934098834 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f24d80a3bb0 0x7f24d80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.151+0000 7f24effff640 1 -- 172.21.15.52:0/3971506072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07d4234d0 msgr2=0x55e07d427fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:08.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.151+0000 7f24effff640 1 --2- 172.21.15.52:0/3971506072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07d4234d0 0x55e07d427fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.151+0000 7f24effff640 1 -- 172.21.15.52:0/3971506072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e07d422ae0 msgr2=0x55e07d422f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:08.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.151+0000 7f24effff640 1 --2- 172.21.15.52:0/3971506072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e07d422ae0 0x55e07d422f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.151+0000 7f24effff640 1 -- 172.21.15.52:0/3971506072 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e07d09b2d0 con 0x55e07d428640 2024-04-02T13:51:08.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.151+0000 7f24f4e0c640 1 --2- 172.21.15.52:0/3971506072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07d4234d0 0x55e07d427fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:08.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.151+0000 7f24edffb640 1 -- 172.21.15.52:0/3934098834 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f24e40d0a60 con 0x7f24f0101c00 2024-04-02T13:51:08.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.151+0000 7f24effff640 1 --2- 172.21.15.52:0/3971506072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d428640 0x55e07d42aad0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f24e4099470 tx=0x7f24e4018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852de21c79fa45e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:08.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.151+0000 7f24d6ffd640 1 -- 172.21.15.52:0/3971506072 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e409c5e0 con 0x55e07d428640 2024-04-02T13:51:08.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.151+0000 7f24f560d640 1 --2- 172.21.15.52:0/3934098834 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f24d80a3bb0 0x7f24d80a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f24e8006010 tx=0x7f24e802c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:08.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.151+0000 7f24d6ffd640 1 -- 172.21.15.52:0/3971506072 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24e40d0e50 con 0x55e07d428640 2024-04-02T13:51:08.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.151+0000 7f24d6ffd640 1 -- 172.21.15.52:0/3971506072 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e409c400 con 0x55e07d428640 2024-04-02T13:51:08.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.152+0000 7f24f9c56740 1 -- 172.21.15.52:0/3971506072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d428640 msgr2=0x55e07d42aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.152+0000 7f24f9c56740 1 --2- 172.21.15.52:0/3971506072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d428640 0x55e07d42aad0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f24e4099470 tx=0x7f24e4018420 comp rx=0 tx=0).stop 2024-04-02T13:51:08.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.152+0000 7f24f9c56740 1 -- 172.21.15.52:0/3971506072 shutdown_connections 2024-04-02T13:51:08.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.152+0000 7f24f9c56740 1 --2- 172.21.15.52:0/3971506072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d428640 0x55e07d42aad0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.152+0000 7f24f9c56740 1 --2- 172.21.15.52:0/3971506072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07d4234d0 0x55e07d427fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.152+0000 7f24f9c56740 1 --2- 172.21.15.52:0/3971506072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e07d422ae0 0x55e07d422f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.152+0000 7f24f9c56740 1 -- 172.21.15.52:0/3971506072 >> 172.21.15.52:0/3971506072 conn(0x55e07d41e050 msgr2=0x55e07d41e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:08.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.152+0000 7f24f9c56740 1 -- 172.21.15.52:0/3971506072 shutdown_connections 2024-04-02T13:51:08.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.152+0000 7f24f9c56740 1 -- 172.21.15.52:0/3971506072 wait complete. 2024-04-02T13:51:08.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.152+0000 7f24f9c56740 1 -- start start 2024-04-02T13:51:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.152+0000 7f24f9c56740 1 Processor -- start 2024-04-02T13:51:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.153+0000 7f24f9c56740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d422ae0 0x55e07d42cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.153+0000 7f24f9c56740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e07d4234d0 0x55e07d42d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.153+0000 7f24f9c56740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07d428640 0x55e07d434540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.153+0000 7f24f9c56740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e07d42b590 con 0x55e07d422ae0 2024-04-02T13:51:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.153+0000 7f24f9c56740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e07d342a10 con 0x55e07d4234d0 2024-04-02T13:51:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.153+0000 7f24f9c56740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e07d42b710 con 0x55e07d428640 2024-04-02T13:51:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.153+0000 7f24f9c56740 10 client.0 _ll_register_callbacks cb 0x55e07d37f0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.153+0000 7f24f4e0c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d422ae0 0x55e07d42cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.153+0000 7f24f4e0c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d422ae0 0x55e07d42cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54108/0 (socket says 172.21.15.52:54108) 2024-04-02T13:51:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.153+0000 7f24f4e0c640 1 -- 172.21.15.52:0/189511680 learned_addr learned my addr 172.21.15.52:0/189511680 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.153+0000 7f24f560d640 1 --2- 172.21.15.52:0/189511680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07d428640 0x55e07d434540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.153+0000 7f24effff640 1 --2- 172.21.15.52:0/189511680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e07d4234d0 0x55e07d42d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.153+0000 7f24f4e0c640 1 -- 172.21.15.52:0/189511680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07d428640 msgr2=0x55e07d434540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.153+0000 7f24f4e0c640 1 --2- 172.21.15.52:0/189511680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07d428640 0x55e07d434540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.153+0000 7f24f4e0c640 1 -- 172.21.15.52:0/189511680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e07d4234d0 msgr2=0x55e07d42d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:08.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.153+0000 7f24f4e0c640 1 --2- 172.21.15.52:0/189511680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e07d4234d0 0x55e07d42d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.153+0000 7f24f4e0c640 1 -- 172.21.15.52:0/189511680 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e07d1656b0 con 0x55e07d422ae0 2024-04-02T13:51:08.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.153+0000 7f24effff640 1 --2- 172.21.15.52:0/189511680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e07d4234d0 0x55e07d42d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:51:08.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.153+0000 7f24f4e0c640 1 --2- 172.21.15.52:0/189511680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d422ae0 0x55e07d42cfc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f24e00126e0 tx=0x7f24e0012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42fe54a7831233c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:08.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.153+0000 7f24cffff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:51:08.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.153+0000 7f24cffff640 1 -- 172.21.15.52:0/189511680 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e0004030 con 0x55e07d422ae0 2024-04-02T13:51:08.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.153+0000 7f24cffff640 1 -- 172.21.15.52:0/189511680 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f24e00041d0 con 0x55e07d422ae0 2024-04-02T13:51:08.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.153+0000 7f24f9c56740 1 -- 172.21.15.52:0/189511680 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24e40d0e50 con 0x55e07d422ae0 2024-04-02T13:51:08.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.154+0000 7f24cffff640 1 -- 172.21.15.52:0/189511680 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f24e00043d0 con 0x55e07d422ae0 2024-04-02T13:51:08.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.154+0000 7f24f9c56740 10 client.14742 fetch_fsmap learned FSMap version 17 2024-04-02T13:51:08.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.154+0000 7f24f9c56740 1 -- 172.21.15.52:0/189511680 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55e07d4379d0 con 0x55e07d422ae0 2024-04-02T13:51:08.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.154+0000 7f24cffff640 1 -- 172.21.15.52:0/189511680 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24e40d0e50 con 0x55e07d422ae0 2024-04-02T13:51:08.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.154+0000 7f24cffff640 1 -- 172.21.15.52:0/189511680 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f24e0003ab0 con 0x55e07d422ae0 2024-04-02T13:51:08.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.154+0000 7f24f9c56740 10 client.14742 fetch_fsmap finished waiting for FSMap version 17 2024-04-02T13:51:08.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.154+0000 7f24f9c56740 10 client.14742 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T13:51:08.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.154+0000 7f24f9c56740 20 client.14742 populate_metadata read hostname 'smithi052' 2024-04-02T13:51:08.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.154+0000 7f24f9c56740 1 --2- 172.21.15.52:0/189511680 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55e07d43a0e0 0x55e07d43b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.154+0000 7f24f9c56740 4 client.14742 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:51:08.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.154+0000 7f24f9c56740 1 -- 172.21.15.52:0/189511680 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e07d32ecf0 con 0x55e07d43a0e0 2024-04-02T13:51:08.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.154+0000 7f24effff640 1 --2- 172.21.15.52:0/189511680 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55e07d43a0e0 0x55e07d43b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.155+0000 7f24effff640 1 --2- 172.21.15.52:0/189511680 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55e07d43a0e0 0x55e07d43b550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8153b225365cbde5 server_cookie=d89ade9adcb1187a in_seq=0 out_seq=0 2024-04-02T13:51:08.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.155+0000 7f24cffff640 10 client.14742 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:51:08.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.155+0000 7f24cffff640 1 -- 172.21.15.52:0/189511680 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e07d32ecf0 con 0x55e07d43a0e0 2024-04-02T13:51:08.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.155+0000 7f24cffff640 10 client.14742 handle_command_reply: tid=0 2024-04-02T13:51:08.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.155+0000 7f24f9c56740 1 client.14742 shutdown 2024-04-02T13:51:08.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.155+0000 7f24f9c56740 1 -- 172.21.15.52:0/189511680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d422ae0 msgr2=0x55e07d42cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.156+0000 7f24f9c56740 1 --2- 172.21.15.52:0/189511680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d422ae0 0x55e07d42cfc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f24e00126e0 tx=0x7f24e0012ba0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.156+0000 7f24f9c56740 1 -- 172.21.15.52:0/189511680 shutdown_connections 2024-04-02T13:51:08.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.156+0000 7f24f9c56740 1 --2- 172.21.15.52:0/189511680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07d428640 0x55e07d434540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.156+0000 7f24f9c56740 1 --2- 172.21.15.52:0/189511680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e07d4234d0 0x55e07d42d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.156+0000 7f24f9c56740 1 --2- 172.21.15.52:0/189511680 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55e07d43a0e0 0x55e07d43b550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.156+0000 7f24f9c56740 1 --2- 172.21.15.52:0/189511680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d422ae0 0x55e07d42cfc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.156+0000 7f24cffff640 0 client.14742 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:51:08.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.156+0000 7f24f9c56740 1 -- 172.21.15.52:0/189511680 >> 172.21.15.52:0/189511680 conn(0x55e07d41e050 msgr2=0x55e07d41d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:08.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.156+0000 7f24f9c56740 1 -- 172.21.15.52:0/189511680 shutdown_connections 2024-04-02T13:51:08.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.156+0000 7f24f9c56740 1 -- 172.21.15.52:0/189511680 wait complete. 2024-04-02T13:51:08.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.156+0000 7f24f9c56740 20 client.14742 trim_cache size 0 max 16384 2024-04-02T13:51:08.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.174+0000 7f24f9c56740 1 Processor -- start 2024-04-02T13:51:08.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.174+0000 7f24f9c56740 1 -- start start 2024-04-02T13:51:08.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.174+0000 7f24f9c56740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d422ae0 0x55e07d42aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.174+0000 7f24f9c56740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07d4234d0 0x55e07d434500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.174+0000 7f24f9c56740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e07d428640 0x55e07d41c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.174+0000 7f24f9c56740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e07d439890 con 0x55e07d422ae0 2024-04-02T13:51:08.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.174+0000 7f24f9c56740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e07d439f60 con 0x55e07d428640 2024-04-02T13:51:08.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.174+0000 7f24f9c56740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e07d43a630 con 0x55e07d4234d0 2024-04-02T13:51:08.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.174+0000 7f24f4e0c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d422ae0 0x55e07d42aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.174+0000 7f24f4e0c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d422ae0 0x55e07d42aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54118/0 (socket says 172.21.15.52:54118) 2024-04-02T13:51:08.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.174+0000 7f24f4e0c640 1 -- 172.21.15.52:0/2728413875 learned_addr learned my addr 172.21.15.52:0/2728413875 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:08.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.174+0000 7f24f560d640 1 --2- 172.21.15.52:0/2728413875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e07d428640 0x55e07d41c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.174+0000 7f24effff640 1 --2- 172.21.15.52:0/2728413875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07d4234d0 0x55e07d434500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.174+0000 7f24f4e0c640 1 -- 172.21.15.52:0/2728413875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07d4234d0 msgr2=0x55e07d434500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:08.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.174+0000 7f24f4e0c640 1 --2- 172.21.15.52:0/2728413875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07d4234d0 0x55e07d434500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.174+0000 7f24f4e0c640 1 -- 172.21.15.52:0/2728413875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e07d428640 msgr2=0x55e07d41c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:08.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.174+0000 7f24f4e0c640 1 --2- 172.21.15.52:0/2728413875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e07d428640 0x55e07d41c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.174+0000 7f24f4e0c640 1 -- 172.21.15.52:0/2728413875 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e07d4377f0 con 0x55e07d422ae0 2024-04-02T13:51:08.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.174+0000 7f24f560d640 1 --2- 172.21.15.52:0/2728413875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e07d428640 0x55e07d41c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:51:08.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.175+0000 7f24f4e0c640 1 --2- 172.21.15.52:0/2728413875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d422ae0 0x55e07d42aa30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f24e0000c00 tx=0x7f24e00096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce862543d7dbf5fd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:08.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.175+0000 7f24d67fc640 1 -- 172.21.15.52:0/2728413875 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e001c620 con 0x55e07d422ae0 2024-04-02T13:51:08.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.175+0000 7f24d67fc640 1 -- 172.21.15.52:0/2728413875 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24e001cca0 con 0x55e07d422ae0 2024-04-02T13:51:08.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.175+0000 7f24d67fc640 1 -- 172.21.15.52:0/2728413875 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e0003c20 con 0x55e07d422ae0 2024-04-02T13:51:08.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.175+0000 7f24f9c56740 1 -- 172.21.15.52:0/2728413875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d422ae0 msgr2=0x55e07d42aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.175+0000 7f24f9c56740 1 --2- 172.21.15.52:0/2728413875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d422ae0 0x55e07d42aa30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f24e0000c00 tx=0x7f24e00096c0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.175+0000 7f24f9c56740 1 -- 172.21.15.52:0/2728413875 shutdown_connections 2024-04-02T13:51:08.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.175+0000 7f24f9c56740 1 --2- 172.21.15.52:0/2728413875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e07d428640 0x55e07d41c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.175+0000 7f24f9c56740 1 --2- 172.21.15.52:0/2728413875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07d4234d0 0x55e07d434500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.175+0000 7f24f9c56740 1 --2- 172.21.15.52:0/2728413875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d422ae0 0x55e07d42aa30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.175+0000 7f24f9c56740 1 -- 172.21.15.52:0/2728413875 >> 172.21.15.52:0/2728413875 conn(0x55e07d41e050 msgr2=0x55e07d4249d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:08.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.175+0000 7f24f9c56740 1 -- 172.21.15.52:0/2728413875 shutdown_connections 2024-04-02T13:51:08.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.175+0000 7f24f9c56740 1 -- 172.21.15.52:0/2728413875 wait complete. 2024-04-02T13:51:08.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.176+0000 7f24f9c56740 1 -- start start 2024-04-02T13:51:08.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.176+0000 7f24f9c56740 1 Processor -- start 2024-04-02T13:51:08.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.176+0000 7f24f9c56740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d422ae0 0x55e07d42fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.176+0000 7f24f9c56740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e07d4234d0 0x55e07d4301c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.176+0000 7f24f9c56740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07d428640 0x55e07d4331e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.176+0000 7f24f9c56740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e07d43b3d0 con 0x55e07d422ae0 2024-04-02T13:51:08.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.176+0000 7f24f9c56740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e07d43ad00 con 0x55e07d4234d0 2024-04-02T13:51:08.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.176+0000 7f24f9c56740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e07d43baa0 con 0x55e07d428640 2024-04-02T13:51:08.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.176+0000 7f24f9c56740 10 client.0 _ll_register_callbacks cb 0x7f24e0009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:08.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.176+0000 7f24f4e0c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d422ae0 0x55e07d42fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.176+0000 7f24f4e0c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d422ae0 0x55e07d42fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54134/0 (socket says 172.21.15.52:54134) 2024-04-02T13:51:08.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.176+0000 7f24effff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e07d4234d0 0x55e07d4301c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.176+0000 7f24f4e0c640 1 -- 172.21.15.52:0/2219446357 learned_addr learned my addr 172.21.15.52:0/2219446357 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:08.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.176+0000 7f24f560d640 1 --2- 172.21.15.52:0/2219446357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07d428640 0x55e07d4331e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.176+0000 7f24f4e0c640 1 -- 172.21.15.52:0/2219446357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07d428640 msgr2=0x55e07d4331e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:08.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.176+0000 7f24f4e0c640 1 --2- 172.21.15.52:0/2219446357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07d428640 0x55e07d4331e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.176+0000 7f24f4e0c640 1 -- 172.21.15.52:0/2219446357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e07d4234d0 msgr2=0x55e07d4301c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:08.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.176+0000 7f24f4e0c640 1 --2- 172.21.15.52:0/2219446357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e07d4234d0 0x55e07d4301c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.176+0000 7f24f4e0c640 1 -- 172.21.15.52:0/2219446357 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e07d4337d0 con 0x55e07d422ae0 2024-04-02T13:51:08.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.176+0000 7f24effff640 1 --2- 172.21.15.52:0/2219446357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e07d4234d0 0x55e07d4301c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:08.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.177+0000 7f24f4e0c640 1 --2- 172.21.15.52:0/2219446357 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d422ae0 0x55e07d42fc80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f24e0003ed0 tx=0x7f24e0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce93c95956e5fcc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:08.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.177+0000 7f24cffff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:51:08.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.177+0000 7f24cffff640 1 -- 172.21.15.52:0/2219446357 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e001ce50 con 0x55e07d422ae0 2024-04-02T13:51:08.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.177+0000 7f24cffff640 1 -- 172.21.15.52:0/2219446357 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f24e001c830 con 0x55e07d422ae0 2024-04-02T13:51:08.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.177+0000 7f24f9c56740 1 -- 172.21.15.52:0/2219446357 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24e001cca0 con 0x55e07d422ae0 2024-04-02T13:51:08.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.177+0000 7f24cffff640 1 -- 172.21.15.52:0/2219446357 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f24e001ca70 con 0x55e07d422ae0 2024-04-02T13:51:08.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.177+0000 7f24f9c56740 10 client.14748 fetch_fsmap learned FSMap version 17 2024-04-02T13:51:08.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.177+0000 7f24f9c56740 1 -- 172.21.15.52:0/2219446357 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55e07d4a00f0 con 0x55e07d422ae0 2024-04-02T13:51:08.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.177+0000 7f24cffff640 1 -- 172.21.15.52:0/2219446357 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24e001cca0 con 0x55e07d422ae0 2024-04-02T13:51:08.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.177+0000 7f24cffff640 1 -- 172.21.15.52:0/2219446357 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f24e0004570 con 0x55e07d422ae0 2024-04-02T13:51:08.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.177+0000 7f24f9c56740 10 client.14748 fetch_fsmap finished waiting for FSMap version 17 2024-04-02T13:51:08.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.177+0000 7f24f9c56740 10 client.14748 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T13:51:08.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.177+0000 7f24f9c56740 20 client.14748 populate_metadata read hostname 'smithi052' 2024-04-02T13:51:08.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.177+0000 7f24f9c56740 1 --2- 172.21.15.52:0/2219446357 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55e07d4a48d0 0x55e07d4a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.177+0000 7f24f9c56740 4 client.14748 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:51:08.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.177+0000 7f24f9c56740 1 -- 172.21.15.52:0/2219446357 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "status"}) -- 0x55e07d3945a0 con 0x55e07d4a48d0 2024-04-02T13:51:08.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.177+0000 7f24effff640 1 --2- 172.21.15.52:0/2219446357 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55e07d4a48d0 0x55e07d4a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.178+0000 7f24cffff640 10 client.14748 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:51:08.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.178+0000 7f24effff640 1 --2- 172.21.15.52:0/2219446357 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55e07d4a48d0 0x55e07d4a35e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2348bf64ecf8680a server_cookie=43cfd555f43ff904 in_seq=0 out_seq=0 2024-04-02T13:51:08.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.178+0000 7f24cffff640 1 -- 172.21.15.52:0/2219446357 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55e07d3945a0 con 0x55e07d4a48d0 2024-04-02T13:51:08.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.178+0000 7f24cffff640 10 client.14748 handle_command_reply: tid=0 2024-04-02T13:51:08.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.178+0000 7f24f9c56740 1 client.14748 shutdown 2024-04-02T13:51:08.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.178+0000 7f24f9c56740 1 -- 172.21.15.52:0/2219446357 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d422ae0 msgr2=0x55e07d42fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.178+0000 7f24f9c56740 1 --2- 172.21.15.52:0/2219446357 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d422ae0 0x55e07d42fc80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f24e0003ed0 tx=0x7f24e0003b90 comp rx=0 tx=0).stop 2024-04-02T13:51:08.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.178+0000 7f24f9c56740 1 -- 172.21.15.52:0/2219446357 shutdown_connections 2024-04-02T13:51:08.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.178+0000 7f24f9c56740 1 --2- 172.21.15.52:0/2219446357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e07d428640 0x55e07d4331e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.178+0000 7f24f9c56740 1 --2- 172.21.15.52:0/2219446357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e07d4234d0 0x55e07d4301c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.178+0000 7f24f9c56740 1 --2- 172.21.15.52:0/2219446357 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55e07d4a48d0 0x55e07d4a35e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.178+0000 7f24f9c56740 1 --2- 172.21.15.52:0/2219446357 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e07d422ae0 0x55e07d42fc80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.178+0000 7f24cffff640 0 client.14748 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:51:08.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.178+0000 7f24f9c56740 1 -- 172.21.15.52:0/2219446357 >> 172.21.15.52:0/2219446357 conn(0x55e07d41e050 msgr2=0x55e07d41c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:08.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.178+0000 7f24f9c56740 1 -- 172.21.15.52:0/2219446357 shutdown_connections 2024-04-02T13:51:08.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.178+0000 7f24f9c56740 1 -- 172.21.15.52:0/2219446357 wait complete. 2024-04-02T13:51:08.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.178+0000 7f24f9c56740 20 client.14748 trim_cache size 0 max 16384 2024-04-02T13:51:08.191 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:51:08.191 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "bfa00e6e-f0f6-11ee-b647-cb9ed24678a4", 2024-04-02T13:51:08.191 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 0, 2024-04-02T13:51:08.191 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24503, 2024-04-02T13:51:08.191 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-04-02T13:51:08.191 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-04-02T13:51:08.191 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-04-02T13:51:08.191 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 6.8548532140000002, 2024-04-02T13:51:08.191 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 17, 2024-04-02T13:51:08.191 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 90, 2024-04-02T13:51:08.191 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 89, 2024-04-02T13:51:08.191 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 12.95271228 2024-04-02T13:51:08.191 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:51:08.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.179+0000 7f24f7070640 1 -- 172.21.15.52:0/3934098834 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f24d80a3bb0 msgr2=0x7f24d80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.179+0000 7f24f7070640 1 --2- 172.21.15.52:0/3934098834 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f24d80a3bb0 0x7f24d80a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f24e8006010 tx=0x7f24e802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.179+0000 7f24f7070640 1 -- 172.21.15.52:0/3934098834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24f0101c00 msgr2=0x7f24f0198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.179+0000 7f24f7070640 1 --2- 172.21.15.52:0/3934098834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24f0101c00 0x7f24f0198320 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f24e400e9f0 tx=0x7f24e400eeb0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.179+0000 7f24f7070640 1 -- 172.21.15.52:0/3934098834 shutdown_connections 2024-04-02T13:51:08.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.179+0000 7f24f7070640 1 --2- 172.21.15.52:0/3934098834 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f24d80a3bb0 0x7f24d80a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.179+0000 7f24f7070640 1 --2- 172.21.15.52:0/3934098834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24f01025c0 0x7f24f019f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.179+0000 7f24f7070640 1 --2- 172.21.15.52:0/3934098834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24f0101c00 0x7f24f0198320 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.179+0000 7f24f7070640 1 --2- 172.21.15.52:0/3934098834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24f0100a10 0x7f24f0197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.179+0000 7f24f7070640 1 -- 172.21.15.52:0/3934098834 >> 172.21.15.52:0/3934098834 conn(0x7f24f00fbfa0 msgr2=0x7f24f006a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:08.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.179+0000 7f24f7070640 1 -- 172.21.15.52:0/3934098834 shutdown_connections 2024-04-02T13:51:08.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:08.179+0000 7f24f7070640 1 -- 172.21.15.52:0/3934098834 wait complete. 2024-04-02T13:51:08.195 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-02T13:51:08.195 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-02T13:51:08.195 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph fs ls 2024-04-02T13:51:08.439 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:51:08.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:08 smithi088 ceph-mon[36528]: pgmap v296: 129 pgs: 13 creating+peering, 109 active+clean, 7 unknown; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 6 op/s 2024-04-02T13:51:08.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:08 smithi088 ceph-mon[36528]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-02T13:51:08.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:08 smithi088 ceph-mon[36528]: Cluster is now healthy 2024-04-02T13:51:08.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:08 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3448846514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:51:08.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:08 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3074978382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:08.688 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:08 smithi052 ceph-mon[32104]: pgmap v296: 129 pgs: 13 creating+peering, 109 active+clean, 7 unknown; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 6 op/s 2024-04-02T13:51:08.688 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:08 smithi052 ceph-mon[32104]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-02T13:51:08.688 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:08 smithi052 ceph-mon[32104]: Cluster is now healthy 2024-04-02T13:51:08.688 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:08 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3448846514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:51:08.688 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:08 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3074978382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:08.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:08 smithi189 ceph-mon[36683]: pgmap v296: 129 pgs: 13 creating+peering, 109 active+clean, 7 unknown; 579 KiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 6 op/s 2024-04-02T13:51:08.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:08 smithi189 ceph-mon[36683]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-02T13:51:08.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:08 smithi189 ceph-mon[36683]: Cluster is now healthy 2024-04-02T13:51:08.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:08 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3448846514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:51:08.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:08 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3074978382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.241+0000 7faef12b5700 1 -- 172.21.15.52:0/1577866213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faeec100b40 msgr2=0x7faeec100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.241+0000 7faef12b5700 1 --2- 172.21.15.52:0/1577866213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faeec100b40 0x7faeec100fc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faed4009990 tx=0x7faed4009ca0 comp rx=0 tx=0).stop 2024-04-02T13:51:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.241+0000 7faef12b5700 1 -- 172.21.15.52:0/1577866213 shutdown_connections 2024-04-02T13:51:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.241+0000 7faef12b5700 1 --2- 172.21.15.52:0/1577866213 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faeec101500 0x7faeec10f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.241+0000 7faef12b5700 1 --2- 172.21.15.52:0/1577866213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faeec108630 0x7faeec108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.241+0000 7faef12b5700 1 --2- 172.21.15.52:0/1577866213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faeec100b40 0x7faeec100fc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.241+0000 7faef12b5700 1 -- 172.21.15.52:0/1577866213 >> 172.21.15.52:0/1577866213 conn(0x7faeec077ac0 msgr2=0x7faeec0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.241+0000 7faef12b5700 1 -- 172.21.15.52:0/1577866213 shutdown_connections 2024-04-02T13:51:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.241+0000 7faef12b5700 1 -- 172.21.15.52:0/1577866213 wait complete. 2024-04-02T13:51:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.242+0000 7faef12b5700 1 Processor -- start 2024-04-02T13:51:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.242+0000 7faef12b5700 1 -- start start 2024-04-02T13:51:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.242+0000 7faef12b5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faeec100b40 0x7faeec19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.242+0000 7faef12b5700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faeec101500 0x7faeec19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.242+0000 7faef12b5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faeec108630 0x7faeec198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.242+0000 7faef12b5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faeec1128d0 con 0x7faeec100b40 2024-04-02T13:51:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.242+0000 7faef12b5700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faeec112750 con 0x7faeec108630 2024-04-02T13:51:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.242+0000 7faef12b5700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faeec112a50 con 0x7faeec101500 2024-04-02T13:51:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.242+0000 7faeeaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faeec100b40 0x7faeec19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.242+0000 7faeeaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faeec100b40 0x7faeec19ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54156/0 (socket says 172.21.15.52:54156) 2024-04-02T13:51:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.242+0000 7faeeaffd700 1 -- 172.21.15.52:0/1451800782 learned_addr learned my addr 172.21.15.52:0/1451800782 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.243+0000 7faeeb7fe700 1 --2- 172.21.15.52:0/1451800782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faeec108630 0x7faeec198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.243+0000 7faeea7fc700 1 --2- 172.21.15.52:0/1451800782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faeec101500 0x7faeec19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.243+0000 7faeeaffd700 1 -- 172.21.15.52:0/1451800782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faeec101500 msgr2=0x7faeec19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:10.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.243+0000 7faeeaffd700 1 --2- 172.21.15.52:0/1451800782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faeec101500 0x7faeec19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:10.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.243+0000 7faeeaffd700 1 -- 172.21.15.52:0/1451800782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faeec108630 msgr2=0x7faeec198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:10.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.243+0000 7faeeaffd700 1 --2- 172.21.15.52:0/1451800782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faeec108630 0x7faeec198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:10.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.243+0000 7faeeaffd700 1 -- 172.21.15.52:0/1451800782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeec199410 con 0x7faeec100b40 2024-04-02T13:51:10.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.243+0000 7faeea7fc700 1 --2- 172.21.15.52:0/1451800782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faeec101500 0x7faeec19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:51:10.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.243+0000 7faeeb7fe700 1 --2- 172.21.15.52:0/1451800782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faeec108630 0x7faeec198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:51:10.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.243+0000 7faeeaffd700 1 --2- 172.21.15.52:0/1451800782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faeec100b40 0x7faeec19ea60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7faedc00ada0 tx=0x7faedc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:10.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.243+0000 7faee3fff700 1 -- 172.21.15.52:0/1451800782 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc017020 con 0x7faeec100b40 2024-04-02T13:51:10.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.243+0000 7faef12b5700 1 -- 172.21.15.52:0/1451800782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeec199700 con 0x7faeec100b40 2024-04-02T13:51:10.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.243+0000 7faee3fff700 1 -- 172.21.15.52:0/1451800782 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faedc004bb0 con 0x7faeec100b40 2024-04-02T13:51:10.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.243+0000 7faee3fff700 1 -- 172.21.15.52:0/1451800782 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc0055d0 con 0x7faeec100b40 2024-04-02T13:51:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.243+0000 7faef12b5700 1 -- 172.21.15.52:0/1451800782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeec1a59e0 con 0x7faeec100b40 2024-04-02T13:51:10.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.244+0000 7faef12b5700 1 -- 172.21.15.52:0/1451800782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeec103e60 con 0x7faeec100b40 2024-04-02T13:51:10.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.245+0000 7faee3fff700 1 -- 172.21.15.52:0/1451800782 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faedc0057b0 con 0x7faeec100b40 2024-04-02T13:51:10.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.246+0000 7faee3fff700 1 --2- 172.21.15.52:0/1451800782 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faed80a3c40 0x7faed80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:10.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.246+0000 7faeea7fc700 1 --2- 172.21.15.52:0/1451800782 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faed80a3c40 0x7faed80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:10.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.246+0000 7faee3fff700 1 -- 172.21.15.52:0/1451800782 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7faedc0cfeb0 con 0x7faeec100b40 2024-04-02T13:51:10.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.246+0000 7faeea7fc700 1 --2- 172.21.15.52:0/1451800782 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faed80a3c40 0x7faed80a6100 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faed400b5c0 tx=0x7faed4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:10.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.250+0000 7faee3fff700 1 -- 172.21.15.52:0/1451800782 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7faedc021020 con 0x7faeec100b40 2024-04-02T13:51:10.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.508+0000 7faef12b5700 1 -- 172.21.15.52:0/1451800782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7faeec199ff0 con 0x7faeec100b40 2024-04-02T13:51:10.511 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:10 smithi052 ceph-mon[32104]: pgmap v297: 129 pgs: 13 creating+peering, 116 active+clean; 579 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-04-02T13:51:10.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.508+0000 7faee3fff700 1 -- 172.21.15.52:0/1451800782 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7faedc09c6e0 con 0x7faeec100b40 2024-04-02T13:51:10.512 INFO:teuthology.orchestra.run.smithi052.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-02T13:51:10.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.510+0000 7faef12b5700 1 -- 172.21.15.52:0/1451800782 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faed80a3c40 msgr2=0x7faed80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:10.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.510+0000 7faef12b5700 1 --2- 172.21.15.52:0/1451800782 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faed80a3c40 0x7faed80a6100 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faed400b5c0 tx=0x7faed4005fb0 comp rx=0 tx=0).stop 2024-04-02T13:51:10.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.510+0000 7faef12b5700 1 -- 172.21.15.52:0/1451800782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faeec100b40 msgr2=0x7faeec19ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:10.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.510+0000 7faef12b5700 1 --2- 172.21.15.52:0/1451800782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faeec100b40 0x7faeec19ea60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7faedc00ada0 tx=0x7faedc00c330 comp rx=0 tx=0).stop 2024-04-02T13:51:10.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.510+0000 7faef12b5700 1 -- 172.21.15.52:0/1451800782 shutdown_connections 2024-04-02T13:51:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.510+0000 7faef12b5700 1 --2- 172.21.15.52:0/1451800782 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faed80a3c40 0x7faed80a6100 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.510+0000 7faef12b5700 1 --2- 172.21.15.52:0/1451800782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faeec108630 0x7faeec198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.510+0000 7faef12b5700 1 --2- 172.21.15.52:0/1451800782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faeec100b40 0x7faeec19ea60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.510+0000 7faef12b5700 1 --2- 172.21.15.52:0/1451800782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faeec101500 0x7faeec19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.510+0000 7faef12b5700 1 -- 172.21.15.52:0/1451800782 >> 172.21.15.52:0/1451800782 conn(0x7faeec077ac0 msgr2=0x7faeec0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.510+0000 7faef12b5700 1 -- 172.21.15.52:0/1451800782 shutdown_connections 2024-04-02T13:51:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:10.510+0000 7faef12b5700 1 -- 172.21.15.52:0/1451800782 wait complete. 2024-04-02T13:51:10.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:10 smithi088 ceph-mon[36528]: pgmap v297: 129 pgs: 13 creating+peering, 116 active+clean; 579 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-04-02T13:51:10.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:10 smithi189 ceph-mon[36683]: pgmap v297: 129 pgs: 13 creating+peering, 116 active+clean; 579 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-04-02T13:51:11.228 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-02T13:51:11.228 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-02T13:51:11.228 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-04-02T13:51:11.228 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-02T13:51:11.228 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-02T13:51:11.228 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-02T13:51:11.228 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-02T13:51:11.228 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-02T13:51:11.228 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-02T13:51:11.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:51:11.229 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-04-02T13:51:11.245 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-02T13:51:11.246 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-02T13:51:11.246 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-04-02T13:51:11.246 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-04-02T13:51:11.246 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-04-02T13:51:11.246 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-04-02T13:51:11.246 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T13:51:11.246 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-04-02T13:51:11.246 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-04-02T13:51:11.246 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T13:51:11.246 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-04-02T13:51:11.246 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-04-02T13:51:11.247 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-02T13:51:11.247 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-04-02T13:51:11.247 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-04-02T13:51:11.247 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-02T13:51:11.247 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 42207sec preferred_lft 42207sec 2024-04-02T13:51:11.247 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link noprefixroute 2024-04-02T13:51:11.247 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-04-02T13:51:11.247 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T13:51:11.247 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-04-02T13:51:11.247 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-04-02T13:51:11.248 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-02T13:51:11.248 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:51:11.248 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-04-02T13:51:11.248 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add name ceph-brx type bridge 2024-04-02T13:51:11.248 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr flush dev ceph-brx 2024-04-02T13:51:11.248 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set ceph-brx up 2024-04-02T13:51:11.248 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-02T13:51:11.248 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-04-02T13:51:11.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:51:11.453 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:11 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1451800782' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T13:51:11.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:51:11.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:51:11.463 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-02T13:51:11.490 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-04-02T13:51:11.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:51:11.491 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-04-02T13:51:11.560 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-04-02T13:51:11.560 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-02T13:51:11.560 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-02T13:51:11.560 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-02T13:51:11.561 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-02T13:51:11.561 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:51:11.561 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-04-02T13:51:11.561 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-02T13:51:11.561 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-02T13:51:11.561 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-02T13:51:11.561 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-04-02T13:51:11.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:51:11.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:11 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1451800782' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T13:51:11.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:11 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1451800782' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T13:51:11.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:51:11.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:51:11.866 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-04-02T13:51:11.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:51:11.881 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list-id 2024-04-02T13:51:11.935 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:51:11.936 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-04-02T13:51:11.936 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-02T13:51:11.936 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-02T13:51:11.936 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-04-02T13:51:12.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:51:12.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:51:12.059 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-02T13:51:12.060 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:51:12.060 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-04-02T13:51:12.060 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-02T13:51:12.060 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-02T13:51:12.060 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-02T13:51:12.060 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-02T13:51:12.060 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-02T13:51:12.060 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-04-02T13:51:12.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:51:12.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:51:12.192 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:51:12.192 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-04-02T13:51:12.192 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-04-02T13:51:12.192 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set dev brx.0 master ceph-brx 2024-04-02T13:51:12.192 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-04-02T13:51:12.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:51:12.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:51:12.274 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-02T13:51:12.274 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T13:51:12.274 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-02T13:51:12.288 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-02T13:51:12.288 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T13:51:12.288 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-02T13:51:12.343 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-04-02T13:51:12.410 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-04-02T13:51:12.465 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-04-02T13:51:12.465 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-04-02T13:51:12.465 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-04-02T13:51:12.465 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-04-02T13:51:12.465 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-04-02T13:51:12.465 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-04-02T13:51:12.465 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-04-02T13:51:12.465 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-04-02T13:51:12.465 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-04-02T13:51:12.465 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-04-02T13:51:12.465 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-04-02T13:51:12.465 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/selinux 2024-04-02T13:51:12.466 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T13:51:12.466 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-04-02T13:51:12.466 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-04-02T13:51:12.466 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-04-02T13:51:12.466 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-04-02T13:51:12.466 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-02T13:51:12.466 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-04-02T13:51:12.466 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysctl.service 2024-04-02T13:51:12.466 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-04-02T13:51:12.466 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-02T13:51:12.466 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/nfs/rpc_pipefs 2024-04-02T13:51:12.466 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-04-02T13:51:12.466 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-04-02T13:51:12.467 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay 2024-04-02T13:51:12.467 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/e2a934be67d7732519bc745f972402f3370cfc104fd791936ca578b77d7e091c/merged 2024-04-02T13:51:12.467 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/9882874cd0b3836de653bb7508150f4d46543cbe47a7889fe9df909f297c8a16/merged 2024-04-02T13:51:12.467 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/0 2024-04-02T13:51:12.467 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T13:51:12.467 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/8ebc0aaf83d4d47717fa8c6bc6c27f4640faae62c299b85f5fb73c3bd6b6e872/merged 2024-04-02T13:51:12.467 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/b3b9506d1a7d8a0c3a663e3a69e39c9b826dd8b5f22fcc531601524a832e8716/merged 2024-04-02T13:51:12.467 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/58a78e4edc29bdf9af91d2f14d43ffb3d2e922da4a80bbddd4a06e8f8dba91ba/merged 2024-04-02T13:51:12.467 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/954748c01b81d59daf7ef9274c1470971a068c13bbc6734837bf93acdc4f8760/merged 2024-04-02T13:51:12.467 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/add3d42523b97f114060936a5faa8d7a5d800f5bff5c8a62b51d1ae9c1d3cb8a/merged 2024-04-02T13:51:12.467 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/43be3792b1e8e7feb237c60dde1887984483ef2e4017fdc7f61e95aa6a90a28e/merged 2024-04-02T13:51:12.467 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/bff41a7c743f5e4bae87eb697f1abff216dccac7dfad762a4df3f8a7d1d3be50/merged 2024-04-02T13:51:12.467 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/7a6bb4a3068c19b8d4fd7867460f4e11208ecc4a4428db4f48c52b1b999515f4/merged 2024-04-02T13:51:12.467 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-04-02T13:51:12.467 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-02T13:51:12.468 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-02T13:51:12.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:51:12.468 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-04-02T13:51:12.519 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-02T13:51:12.519 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-02T13:51:12.561 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-04-02T13:51:12.587 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-04-02T13:51:12.642 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-04-02T13:51:12.642 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-04-02T13:51:12.642 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-04-02T13:51:12.642 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-04-02T13:51:12.643 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-04-02T13:51:12.643 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-04-02T13:51:12.643 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-04-02T13:51:12.643 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-04-02T13:51:12.643 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-04-02T13:51:12.643 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-04-02T13:51:12.643 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-04-02T13:51:12.643 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/selinux 2024-04-02T13:51:12.643 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T13:51:12.643 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-04-02T13:51:12.643 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-04-02T13:51:12.643 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-04-02T13:51:12.643 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-04-02T13:51:12.643 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-02T13:51:12.643 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-04-02T13:51:12.644 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysctl.service 2024-04-02T13:51:12.644 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-04-02T13:51:12.644 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-02T13:51:12.644 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/nfs/rpc_pipefs 2024-04-02T13:51:12.644 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-04-02T13:51:12.644 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-04-02T13:51:12.644 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay 2024-04-02T13:51:12.644 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/e2a934be67d7732519bc745f972402f3370cfc104fd791936ca578b77d7e091c/merged 2024-04-02T13:51:12.644 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/9882874cd0b3836de653bb7508150f4d46543cbe47a7889fe9df909f297c8a16/merged 2024-04-02T13:51:12.644 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/0 2024-04-02T13:51:12.644 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T13:51:12.644 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/8ebc0aaf83d4d47717fa8c6bc6c27f4640faae62c299b85f5fb73c3bd6b6e872/merged 2024-04-02T13:51:12.644 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/b3b9506d1a7d8a0c3a663e3a69e39c9b826dd8b5f22fcc531601524a832e8716/merged 2024-04-02T13:51:12.644 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/58a78e4edc29bdf9af91d2f14d43ffb3d2e922da4a80bbddd4a06e8f8dba91ba/merged 2024-04-02T13:51:12.644 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/954748c01b81d59daf7ef9274c1470971a068c13bbc6734837bf93acdc4f8760/merged 2024-04-02T13:51:12.645 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/add3d42523b97f114060936a5faa8d7a5d800f5bff5c8a62b51d1ae9c1d3cb8a/merged 2024-04-02T13:51:12.645 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/43be3792b1e8e7feb237c60dde1887984483ef2e4017fdc7f61e95aa6a90a28e/merged 2024-04-02T13:51:12.645 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/bff41a7c743f5e4bae87eb697f1abff216dccac7dfad762a4df3f8a7d1d3be50/merged 2024-04-02T13:51:12.645 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/7a6bb4a3068c19b8d4fd7867460f4e11208ecc4a4428db4f48c52b1b999515f4/merged 2024-04-02T13:51:12.645 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-04-02T13:51:12.645 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-02T13:51:12.645 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-02T13:51:12.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:51:12.645 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-04-02T13:51:12.659 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi052.stderr:2024-04-02T13:51:12.656+0000 7f9df0573480 -1 init, newargv = 0x55d08a3ef580 newargc=15 2024-04-02T13:51:12.659 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi052.stderr:ceph-fuse[73495]: starting ceph client 2024-04-02T13:51:12.665 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi052.stderr:ceph-fuse[73495]: starting fuse 2024-04-02T13:51:12.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:12 smithi088 ceph-mon[36528]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 3 op/s 2024-04-02T13:51:12.674 INFO:teuthology.orchestra.run.smithi052.stdout:119 2024-04-02T13:51:12.674 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-04-02T13:51:12.674 DEBUG:teuthology.orchestra.run.smithi052:> sudo stdin-killer -- python3 -c ' 2024-04-02T13:51:12.674 DEBUG:teuthology.orchestra.run.smithi052:> import glob 2024-04-02T13:51:12.674 DEBUG:teuthology.orchestra.run.smithi052:> import re 2024-04-02T13:51:12.674 DEBUG:teuthology.orchestra.run.smithi052:> import os 2024-04-02T13:51:12.674 DEBUG:teuthology.orchestra.run.smithi052:> import subprocess 2024-04-02T13:51:12.675 DEBUG:teuthology.orchestra.run.smithi052:> 2024-04-02T13:51:12.675 DEBUG:teuthology.orchestra.run.smithi052:> def _find_admin_socket(client_name): 2024-04-02T13:51:12.675 DEBUG:teuthology.orchestra.run.smithi052:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-02T13:51:12.675 DEBUG:teuthology.orchestra.run.smithi052:> files = glob.glob(asok_path) 2024-04-02T13:51:12.675 DEBUG:teuthology.orchestra.run.smithi052:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-02T13:51:12.675 DEBUG:teuthology.orchestra.run.smithi052:> 2024-04-02T13:51:12.675 DEBUG:teuthology.orchestra.run.smithi052:> # Given a non-glob path, it better be there 2024-04-02T13:51:12.675 DEBUG:teuthology.orchestra.run.smithi052:> if "*" not in asok_path: 2024-04-02T13:51:12.675 DEBUG:teuthology.orchestra.run.smithi052:> assert(len(files) == 1) 2024-04-02T13:51:12.675 DEBUG:teuthology.orchestra.run.smithi052:> return files[0] 2024-04-02T13:51:12.675 DEBUG:teuthology.orchestra.run.smithi052:> 2024-04-02T13:51:12.675 DEBUG:teuthology.orchestra.run.smithi052:> for f in files: 2024-04-02T13:51:12.675 DEBUG:teuthology.orchestra.run.smithi052:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-02T13:51:12.675 DEBUG:teuthology.orchestra.run.smithi052:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-02T13:51:12.675 DEBUG:teuthology.orchestra.run.smithi052:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-02T13:51:12.675 DEBUG:teuthology.orchestra.run.smithi052:> contents = proc_f.read() 2024-04-02T13:51:12.676 DEBUG:teuthology.orchestra.run.smithi052:> if mountpoint in contents: 2024-04-02T13:51:12.676 DEBUG:teuthology.orchestra.run.smithi052:> return f 2024-04-02T13:51:12.676 DEBUG:teuthology.orchestra.run.smithi052:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-02T13:51:12.676 DEBUG:teuthology.orchestra.run.smithi052:> 2024-04-02T13:51:12.676 DEBUG:teuthology.orchestra.run.smithi052:> print(_find_admin_socket("client.admin")) 2024-04-02T13:51:12.676 DEBUG:teuthology.orchestra.run.smithi052:> ' 2024-04-02T13:51:12.738 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:12 smithi052 ceph-mon[32104]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 3 op/s 2024-04-02T13:51:12.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:12 smithi189 ceph-mon[36683]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 3 op/s 2024-04-02T13:51:12.795 INFO:teuthology.orchestra.run.smithi052.stdout:/var/run/ceph/ceph-client.admin.73495.asok 2024-04-02T13:51:12.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:51:12.805 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.73495.asok 2024-04-02T13:51:12.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:51:12.806 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.73495.asok status 2024-04-02T13:51:12.943 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:51:12.943 INFO:teuthology.orchestra.run.smithi052.stdout: "metadata": { 2024-04-02T13:51:12.943 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_sha1": "191d963ac9de7322da8aff20b567b3fa458e2ef6", 2024-04-02T13:51:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_version": "ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev)", 2024-04-02T13:51:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "entity_id": "admin", 2024-04-02T13:51:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": "smithi052", 2024-04-02T13:51:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-02T13:51:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "pid": "73495", 2024-04-02T13:51:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "root": "/" 2024-04-02T13:51:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T13:51:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "dentry_count": 0, 2024-04-02T13:51:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "dentry_pinned_count": 0, 2024-04-02T13:51:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 14760, 2024-04-02T13:51:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "inst": { 2024-04-02T13:51:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "name": { 2024-04-02T13:51:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "client", 2024-04-02T13:51:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "num": 14760 2024-04-02T13:51:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T13:51:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": { 2024-04-02T13:51:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-04-02T13:51:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "192.168.0.1:0", 2024-04-02T13:51:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 385341264 2024-04-02T13:51:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:51:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T13:51:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": { 2024-04-02T13:51:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-04-02T13:51:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "192.168.0.1:0", 2024-04-02T13:51:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 385341264 2024-04-02T13:51:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T13:51:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "inst_str": "client.14760 192.168.0.1:0/385341264", 2024-04-02T13:51:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "addr_str": "192.168.0.1:0/385341264", 2024-04-02T13:51:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "inode_count": 1, 2024-04-02T13:51:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_epoch": 17, 2024-04-02T13:51:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_epoch": 90, 2024-04-02T13:51:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_epoch_barrier": 0, 2024-04-02T13:51:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: "blocklisted": false, 2024-04-02T13:51:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs" 2024-04-02T13:51:12.946 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:51:12.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:51:12.952 DEBUG:teuthology.orchestra.run.smithi052:> stat --file-system '--printf=%T 2024-04-02T13:51:12.952 DEBUG:teuthology.orchestra.run.smithi052:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-02T13:51:12.967 INFO:teuthology.orchestra.run.smithi052.stdout:fuseblk 2024-04-02T13:51:12.967 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-02T13:51:12.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:51:12.967 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-02T13:51:13.035 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-02T13:51:13.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T13:51:13.144 INFO:teuthology.orchestra.run.smithi052.stdout:# file: . 2024-04-02T13:51:13.144 INFO:teuthology.orchestra.run.smithi052.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-02T13:51:13.144 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:51:13.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:51:13.150 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-04-02T13:51:13.150 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T13:51:13.166 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:51:13.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.306+0000 7f520359e640 1 -- 172.21.15.52:0/3134566088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5204100850 msgr2=0x7f5204100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:13.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.306+0000 7f520359e640 1 --2- 172.21.15.52:0/3134566088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5204100850 0x7f5204100c70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009900 tx=0x7f51fc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:51:13.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.306+0000 7f520359e640 1 -- 172.21.15.52:0/3134566088 shutdown_connections 2024-04-02T13:51:13.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.306+0000 7f520359e640 1 --2- 172.21.15.52:0/3134566088 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5204102400 0x7f5204108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:13.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.306+0000 7f520359e640 1 --2- 172.21.15.52:0/3134566088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5204101a40 0x7f5204101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:13.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.306+0000 7f520359e640 1 --2- 172.21.15.52:0/3134566088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5204100850 0x7f5204100c70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:13.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.306+0000 7f520359e640 1 -- 172.21.15.52:0/3134566088 >> 172.21.15.52:0/3134566088 conn(0x7f52040fbde0 msgr2=0x7f52040fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:13.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.307+0000 7f520359e640 1 -- 172.21.15.52:0/3134566088 shutdown_connections 2024-04-02T13:51:13.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.307+0000 7f520359e640 1 -- 172.21.15.52:0/3134566088 wait complete. 2024-04-02T13:51:13.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.307+0000 7f520359e640 1 Processor -- start 2024-04-02T13:51:13.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.307+0000 7f520359e640 1 -- start start 2024-04-02T13:51:13.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.307+0000 7f520359e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5204100850 0x7f5204197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:13.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.307+0000 7f520359e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5204101a40 0x7f5204198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:13.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.307+0000 7f520359e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5204102400 0x7f520419f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:13.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.307+0000 7f520359e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f520410d5b0 con 0x7f5204100850 2024-04-02T13:51:13.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.307+0000 7f520359e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f520410d430 con 0x7f5204102400 2024-04-02T13:51:13.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.307+0000 7f520359e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f520410d730 con 0x7f5204101a40 2024-04-02T13:51:13.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.308+0000 7f520259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5204100850 0x7f5204197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:13.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.308+0000 7f520259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5204100850 0x7f5204197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44240/0 (socket says 172.21.15.52:44240) 2024-04-02T13:51:13.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.308+0000 7f520259c640 1 -- 172.21.15.52:0/3431783911 learned_addr learned my addr 172.21.15.52:0/3431783911 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:13.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.308+0000 7f5201d9b640 1 --2- 172.21.15.52:0/3431783911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5204101a40 0x7f5204198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:13.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.308+0000 7f5202d9d640 1 --2- 172.21.15.52:0/3431783911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5204102400 0x7f520419f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:13.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.308+0000 7f520259c640 1 -- 172.21.15.52:0/3431783911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5204101a40 msgr2=0x7f5204198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:13.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.308+0000 7f520259c640 1 --2- 172.21.15.52:0/3431783911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5204101a40 0x7f5204198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:13.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.308+0000 7f520259c640 1 -- 172.21.15.52:0/3431783911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5204102400 msgr2=0x7f520419f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:13.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.308+0000 7f520259c640 1 --2- 172.21.15.52:0/3431783911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5204102400 0x7f520419f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:13.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.308+0000 7f520259c640 1 -- 172.21.15.52:0/3431783911 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520419f860 con 0x7f5204100850 2024-04-02T13:51:13.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.308+0000 7f5202d9d640 1 --2- 172.21.15.52:0/3431783911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5204102400 0x7f520419f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:51:13.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.308+0000 7f520259c640 1 --2- 172.21.15.52:0/3431783911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5204100850 0x7f5204197c50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f51fc005ab0 tx=0x7f51fc005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:13.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.308+0000 7f51fb7fe640 1 -- 172.21.15.52:0/3431783911 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc041070 con 0x7f5204100850 2024-04-02T13:51:13.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.308+0000 7f51fb7fe640 1 -- 172.21.15.52:0/3431783911 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51fc0054e0 con 0x7f5204100850 2024-04-02T13:51:13.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.308+0000 7f520359e640 1 -- 172.21.15.52:0/3431783911 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520419faf0 con 0x7f5204100850 2024-04-02T13:51:13.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.308+0000 7f51fb7fe640 1 -- 172.21.15.52:0/3431783911 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc034440 con 0x7f5204100850 2024-04-02T13:51:13.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.308+0000 7f520359e640 1 -- 172.21.15.52:0/3431783911 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f520419ffe0 con 0x7f5204100850 2024-04-02T13:51:13.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.309+0000 7f520359e640 1 -- 172.21.15.52:0/3431783911 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5204107bf0 con 0x7f5204100850 2024-04-02T13:51:13.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.310+0000 7f51fb7fe640 1 -- 172.21.15.52:0/3431783911 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f51fc044070 con 0x7f5204100850 2024-04-02T13:51:13.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.311+0000 7f51fb7fe640 1 --2- 172.21.15.52:0/3431783911 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f51f00a3bb0 0x7f51f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:13.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.311+0000 7f51fb7fe640 1 -- 172.21.15.52:0/3431783911 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f51fc0f25c0 con 0x7f5204100850 2024-04-02T13:51:13.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.311+0000 7f5201d9b640 1 --2- 172.21.15.52:0/3431783911 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f51f00a3bb0 0x7f51f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:13.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.311+0000 7f5201d9b640 1 --2- 172.21.15.52:0/3431783911 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f51f00a3bb0 0x7f51f00a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f51ec005fd0 tx=0x7f51ec005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:13.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.315+0000 7f51fb7fe640 1 -- 172.21.15.52:0/3431783911 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f51fc0bed60 con 0x7f5204100850 2024-04-02T13:51:13.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.508+0000 7f520359e640 1 -- 172.21.15.52:0/3431783911 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f520402d600 con 0x7f5204100850 2024-04-02T13:51:13.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.509+0000 7f51fb7fe640 1 -- 172.21.15.52:0/3431783911 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24077 (secure 0 0 0) 0x7f51fc0be660 con 0x7f5204100850 2024-04-02T13:51:13.513 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:51:13.513 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":90,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T13:51:05.722301+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:50:58.778690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:50:59.561527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:51:03.343487+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:51:13.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.511+0000 7f520359e640 1 -- 172.21.15.52:0/3431783911 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f51f00a3bb0 msgr2=0x7f51f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:13.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.511+0000 7f520359e640 1 --2- 172.21.15.52:0/3431783911 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f51f00a3bb0 0x7f51f00a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f51ec005fd0 tx=0x7f51ec005850 comp rx=0 tx=0).stop 2024-04-02T13:51:13.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.511+0000 7f520359e640 1 -- 172.21.15.52:0/3431783911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5204100850 msgr2=0x7f5204197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:13.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.511+0000 7f520359e640 1 --2- 172.21.15.52:0/3431783911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5204100850 0x7f5204197c50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f51fc005ab0 tx=0x7f51fc005dc0 comp rx=0 tx=0).stop 2024-04-02T13:51:13.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.511+0000 7f520359e640 1 -- 172.21.15.52:0/3431783911 shutdown_connections 2024-04-02T13:51:13.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.511+0000 7f520359e640 1 --2- 172.21.15.52:0/3431783911 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f51f00a3bb0 0x7f51f00a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:13.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.511+0000 7f520359e640 1 --2- 172.21.15.52:0/3431783911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5204102400 0x7f520419f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:13.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.511+0000 7f520359e640 1 --2- 172.21.15.52:0/3431783911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5204101a40 0x7f5204198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:13.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.511+0000 7f520359e640 1 --2- 172.21.15.52:0/3431783911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5204100850 0x7f5204197c50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:13.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.511+0000 7f520359e640 1 -- 172.21.15.52:0/3431783911 >> 172.21.15.52:0/3431783911 conn(0x7f52040fbde0 msgr2=0x7f5204104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:13.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.511+0000 7f520359e640 1 -- 172.21.15.52:0/3431783911 shutdown_connections 2024-04-02T13:51:13.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.511+0000 7f520359e640 1 -- 172.21.15.52:0/3431783911 wait complete. 2024-04-02T13:51:13.525 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:46:20.746053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:50:58.778690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:50:59.561527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:51:03.343487+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:51:13.526 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:51:13.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.629+0000 7f090e03a640 1 -- 172.21.15.52:0/2103522555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0908101b20 msgr2=0x7f0908101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:13.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.629+0000 7f090e03a640 1 --2- 172.21.15.52:0/2103522555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0908101b20 0x7f0908101f40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009900 tx=0x7f08f802bc30 comp rx=0 tx=0).stop 2024-04-02T13:51:13.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.629+0000 7f090e03a640 1 -- 172.21.15.52:0/2103522555 shutdown_connections 2024-04-02T13:51:13.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.629+0000 7f090e03a640 1 --2- 172.21.15.52:0/2103522555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09081036d0 0x7f0908109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:13.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.629+0000 7f090e03a640 1 --2- 172.21.15.52:0/2103522555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0908102d10 0x7f0908103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:13.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.629+0000 7f090e03a640 1 --2- 172.21.15.52:0/2103522555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0908101b20 0x7f0908101f40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:13.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.629+0000 7f090e03a640 1 -- 172.21.15.52:0/2103522555 >> 172.21.15.52:0/2103522555 conn(0x7f09080fd090 msgr2=0x7f09080ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:13.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.629+0000 7f090e03a640 1 -- 172.21.15.52:0/2103522555 shutdown_connections 2024-04-02T13:51:13.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.629+0000 7f090e03a640 1 -- 172.21.15.52:0/2103522555 wait complete. 2024-04-02T13:51:13.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.630+0000 7f090e03a640 1 Processor -- start 2024-04-02T13:51:13.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.630+0000 7f090e03a640 1 -- start start 2024-04-02T13:51:13.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.630+0000 7f090e03a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0908101b20 0x7f0908197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:13.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.630+0000 7f090e03a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0908102d10 0x7f0908198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:13.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.630+0000 7f090e03a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09081036d0 0x7f090819f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:13.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.630+0000 7f090e03a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f090810d770 con 0x7f0908102d10 2024-04-02T13:51:13.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.630+0000 7f090e03a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f090810d5f0 con 0x7f09081036d0 2024-04-02T13:51:13.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.630+0000 7f090e03a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f090810d8f0 con 0x7f0908101b20 2024-04-02T13:51:13.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.630+0000 7f0906ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0908102d10 0x7f0908198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:13.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.630+0000 7f0906ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0908102d10 0x7f0908198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44258/0 (socket says 172.21.15.52:44258) 2024-04-02T13:51:13.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.630+0000 7f0906ffd640 1 -- 172.21.15.52:0/3443782003 learned_addr learned my addr 172.21.15.52:0/3443782003 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:13.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.630+0000 7f0907fff640 1 --2- 172.21.15.52:0/3443782003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09081036d0 0x7f090819f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:13.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.630+0000 7f09077fe640 1 --2- 172.21.15.52:0/3443782003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0908101b20 0x7f0908197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:13.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.630+0000 7f0906ffd640 1 -- 172.21.15.52:0/3443782003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0908101b20 msgr2=0x7f0908197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:13.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.630+0000 7f0906ffd640 1 --2- 172.21.15.52:0/3443782003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0908101b20 0x7f0908197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:13.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.630+0000 7f0906ffd640 1 -- 172.21.15.52:0/3443782003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09081036d0 msgr2=0x7f090819f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:13.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.630+0000 7f0906ffd640 1 --2- 172.21.15.52:0/3443782003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09081036d0 0x7f090819f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:13.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.630+0000 7f0906ffd640 1 -- 172.21.15.52:0/3443782003 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f090819fa30 con 0x7f0908102d10 2024-04-02T13:51:13.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.630+0000 7f0906ffd640 1 --2- 172.21.15.52:0/3443782003 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0908102d10 0x7f0908198350 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00d950 tx=0x7f08fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:13.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.631+0000 7f0904ff9640 1 -- 172.21.15.52:0/3443782003 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc017030 con 0x7f0908102d10 2024-04-02T13:51:13.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.631+0000 7f090e03a640 1 -- 172.21.15.52:0/3443782003 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f090819fd20 con 0x7f0908102d10 2024-04-02T13:51:13.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.631+0000 7f0904ff9640 1 -- 172.21.15.52:0/3443782003 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08fc004020 con 0x7f0908102d10 2024-04-02T13:51:13.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.631+0000 7f0904ff9640 1 -- 172.21.15.52:0/3443782003 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc0047a0 con 0x7f0908102d10 2024-04-02T13:51:13.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.631+0000 7f090e03a640 1 -- 172.21.15.52:0/3443782003 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09081a0210 con 0x7f0908102d10 2024-04-02T13:51:13.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.632+0000 7f090e03a640 1 -- 172.21.15.52:0/3443782003 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0908108f10 con 0x7f0908102d10 2024-04-02T13:51:13.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.633+0000 7f0904ff9640 1 -- 172.21.15.52:0/3443782003 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f08fc0041d0 con 0x7f0908102d10 2024-04-02T13:51:13.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.633+0000 7f0904ff9640 1 --2- 172.21.15.52:0/3443782003 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f08f00a3bb0 0x7f08f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:13.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.633+0000 7f09077fe640 1 --2- 172.21.15.52:0/3443782003 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f08f00a3bb0 0x7f08f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:13.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.633+0000 7f0904ff9640 1 -- 172.21.15.52:0/3443782003 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f08fc0d00b0 con 0x7f0908102d10 2024-04-02T13:51:13.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.634+0000 7f09077fe640 1 --2- 172.21.15.52:0/3443782003 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f08f00a3bb0 0x7f08f00a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f08f8036040 tx=0x7f08f8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:13.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.638+0000 7f0904ff9640 1 -- 172.21.15.52:0/3443782003 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f08fc09c850 con 0x7f0908102d10 2024-04-02T13:51:13.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.822+0000 7f090e03a640 1 -- 172.21.15.52:0/3443782003 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f09080627a0 con 0x7f0908102d10 2024-04-02T13:51:13.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.822+0000 7f0904ff9640 1 -- 172.21.15.52:0/3443782003 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f08fc09c150 con 0x7f0908102d10 2024-04-02T13:51:13.826 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-04-02T13:51:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.824+0000 7f090e03a640 1 -- 172.21.15.52:0/3443782003 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f08f00a3bb0 msgr2=0x7f08f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.824+0000 7f090e03a640 1 --2- 172.21.15.52:0/3443782003 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f08f00a3bb0 0x7f08f00a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f08f8036040 tx=0x7f08f8005fd0 comp rx=0 tx=0).stop 2024-04-02T13:51:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.824+0000 7f090e03a640 1 -- 172.21.15.52:0/3443782003 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0908102d10 msgr2=0x7f0908198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.824+0000 7f090e03a640 1 --2- 172.21.15.52:0/3443782003 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0908102d10 0x7f0908198350 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00d950 tx=0x7f08fc00de10 comp rx=0 tx=0).stop 2024-04-02T13:51:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.824+0000 7f090e03a640 1 -- 172.21.15.52:0/3443782003 shutdown_connections 2024-04-02T13:51:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.824+0000 7f090e03a640 1 --2- 172.21.15.52:0/3443782003 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f08f00a3bb0 0x7f08f00a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.824+0000 7f090e03a640 1 --2- 172.21.15.52:0/3443782003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09081036d0 0x7f090819f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.824+0000 7f090e03a640 1 --2- 172.21.15.52:0/3443782003 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0908102d10 0x7f0908198350 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.824+0000 7f090e03a640 1 --2- 172.21.15.52:0/3443782003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0908101b20 0x7f0908197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.824+0000 7f090e03a640 1 -- 172.21.15.52:0/3443782003 >> 172.21.15.52:0/3443782003 conn(0x7f09080fd090 msgr2=0x7f09081060e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:13.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.824+0000 7f090e03a640 1 -- 172.21.15.52:0/3443782003 shutdown_connections 2024-04-02T13:51:13.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.824+0000 7f090e03a640 1 -- 172.21.15.52:0/3443782003 wait complete. 2024-04-02T13:51:13.838 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:51:13.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.942+0000 7f391480d640 1 -- 172.21.15.52:0/1923451740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3910100aa0 msgr2=0x7f3910100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:13.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.942+0000 7f391480d640 1 --2- 172.21.15.52:0/1923451740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3910100aa0 0x7f3910100ec0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3900009900 tx=0x7f390002bc30 comp rx=0 tx=0).stop 2024-04-02T13:51:13.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.942+0000 7f391480d640 1 -- 172.21.15.52:0/1923451740 shutdown_connections 2024-04-02T13:51:13.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.942+0000 7f391480d640 1 --2- 172.21.15.52:0/1923451740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3910102650 0x7f3910108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:13.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.942+0000 7f391480d640 1 --2- 172.21.15.52:0/1923451740 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3910101c90 0x7f3910102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:13.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.942+0000 7f391480d640 1 --2- 172.21.15.52:0/1923451740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3910100aa0 0x7f3910100ec0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:13.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.942+0000 7f391480d640 1 -- 172.21.15.52:0/1923451740 >> 172.21.15.52:0/1923451740 conn(0x7f39100fc010 msgr2=0x7f39100fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:13.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.942+0000 7f391480d640 1 -- 172.21.15.52:0/1923451740 shutdown_connections 2024-04-02T13:51:13.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.942+0000 7f391480d640 1 -- 172.21.15.52:0/1923451740 wait complete. 2024-04-02T13:51:13.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.942+0000 7f391480d640 1 Processor -- start 2024-04-02T13:51:13.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.942+0000 7f391480d640 1 -- start start 2024-04-02T13:51:13.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.943+0000 7f391480d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3910100aa0 0x7f3910197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:13.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.943+0000 7f391480d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3910101c90 0x7f3910198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:13.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.943+0000 7f391480d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3910102650 0x7f391019f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:13.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.943+0000 7f391480d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f391010d840 con 0x7f3910100aa0 2024-04-02T13:51:13.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.943+0000 7f391480d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f391010d6c0 con 0x7f3910102650 2024-04-02T13:51:13.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.943+0000 7f391480d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f391010d9c0 con 0x7f3910101c90 2024-04-02T13:51:13.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.943+0000 7f390e59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3910100aa0 0x7f3910197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:13.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.943+0000 7f390e59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3910100aa0 0x7f3910197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44288/0 (socket says 172.21.15.52:44288) 2024-04-02T13:51:13.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.943+0000 7f390e59c640 1 -- 172.21.15.52:0/3140654707 learned_addr learned my addr 172.21.15.52:0/3140654707 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:13.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.943+0000 7f390dd9b640 1 --2- 172.21.15.52:0/3140654707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3910101c90 0x7f3910198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:13.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.943+0000 7f390e59c640 1 -- 172.21.15.52:0/3140654707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3910101c90 msgr2=0x7f3910198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:13.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.943+0000 7f390ed9d640 1 --2- 172.21.15.52:0/3140654707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3910102650 0x7f391019f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:13.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.943+0000 7f390e59c640 1 --2- 172.21.15.52:0/3140654707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3910101c90 0x7f3910198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:13.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.943+0000 7f390e59c640 1 -- 172.21.15.52:0/3140654707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3910102650 msgr2=0x7f391019f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:13.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.943+0000 7f390e59c640 1 --2- 172.21.15.52:0/3140654707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3910102650 0x7f391019f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:13.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.943+0000 7f390e59c640 1 -- 172.21.15.52:0/3140654707 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391019fb20 con 0x7f3910100aa0 2024-04-02T13:51:13.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.943+0000 7f390e59c640 1 --2- 172.21.15.52:0/3140654707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3910100aa0 0x7f3910197f00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3900000c00 tx=0x7f3900005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:13.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.943+0000 7f38ff7fe640 1 -- 172.21.15.52:0/3140654707 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900041070 con 0x7f3910100aa0 2024-04-02T13:51:13.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.943+0000 7f391480d640 1 -- 172.21.15.52:0/3140654707 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f391019fdb0 con 0x7f3910100aa0 2024-04-02T13:51:13.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.943+0000 7f38ff7fe640 1 -- 172.21.15.52:0/3140654707 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39000044d0 con 0x7f3910100aa0 2024-04-02T13:51:13.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.943+0000 7f391480d640 1 -- 172.21.15.52:0/3140654707 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39101a0210 con 0x7f3910100aa0 2024-04-02T13:51:13.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.943+0000 7f38ff7fe640 1 -- 172.21.15.52:0/3140654707 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390004b060 con 0x7f3910100aa0 2024-04-02T13:51:13.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.944+0000 7f391480d640 1 -- 172.21.15.52:0/3140654707 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3910107de0 con 0x7f3910100aa0 2024-04-02T13:51:13.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.946+0000 7f38ff7fe640 1 -- 172.21.15.52:0/3140654707 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3900044070 con 0x7f3910100aa0 2024-04-02T13:51:13.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.946+0000 7f38ff7fe640 1 --2- 172.21.15.52:0/3140654707 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f38f80a3bb0 0x7f38f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:13.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.946+0000 7f390dd9b640 1 --2- 172.21.15.52:0/3140654707 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f38f80a3bb0 0x7f38f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:13.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.946+0000 7f38ff7fe640 1 -- 172.21.15.52:0/3140654707 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f39000f2760 con 0x7f3910100aa0 2024-04-02T13:51:13.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.947+0000 7f390dd9b640 1 --2- 172.21.15.52:0/3140654707 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f38f80a3bb0 0x7f38f80a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3910199410 tx=0x7f390400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:13.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:13.950+0000 7f38ff7fe640 1 -- 172.21.15.52:0/3140654707 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f39000bef00 con 0x7f3910100aa0 2024-04-02T13:51:14.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.133+0000 7f391480d640 1 -- 172.21.15.52:0/3140654707 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3910062800 con 0x7f3910100aa0 2024-04-02T13:51:14.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.134+0000 7f38ff7fe640 1 -- 172.21.15.52:0/3140654707 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f39000be800 con 0x7f3910100aa0 2024-04-02T13:51:14.137 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-04-02T13:51:14.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.136+0000 7f391480d640 1 -- 172.21.15.52:0/3140654707 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f38f80a3bb0 msgr2=0x7f38f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:14.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.136+0000 7f391480d640 1 --2- 172.21.15.52:0/3140654707 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f38f80a3bb0 0x7f38f80a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3910199410 tx=0x7f390400b390 comp rx=0 tx=0).stop 2024-04-02T13:51:14.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.136+0000 7f391480d640 1 -- 172.21.15.52:0/3140654707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3910100aa0 msgr2=0x7f3910197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:14.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.136+0000 7f391480d640 1 --2- 172.21.15.52:0/3140654707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3910100aa0 0x7f3910197f00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3900000c00 tx=0x7f3900005d50 comp rx=0 tx=0).stop 2024-04-02T13:51:14.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.136+0000 7f391480d640 1 -- 172.21.15.52:0/3140654707 shutdown_connections 2024-04-02T13:51:14.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.136+0000 7f391480d640 1 --2- 172.21.15.52:0/3140654707 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f38f80a3bb0 0x7f38f80a6070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.136+0000 7f391480d640 1 --2- 172.21.15.52:0/3140654707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3910102650 0x7f391019f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.136+0000 7f391480d640 1 --2- 172.21.15.52:0/3140654707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3910101c90 0x7f3910198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.136+0000 7f391480d640 1 --2- 172.21.15.52:0/3140654707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3910100aa0 0x7f3910197f00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.136+0000 7f391480d640 1 -- 172.21.15.52:0/3140654707 >> 172.21.15.52:0/3140654707 conn(0x7f39100fc010 msgr2=0x7f3910104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:14.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.136+0000 7f391480d640 1 -- 172.21.15.52:0/3140654707 shutdown_connections 2024-04-02T13:51:14.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.136+0000 7f391480d640 1 -- 172.21.15.52:0/3140654707 wait complete. 2024-04-02T13:51:14.149 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:51:14.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.252+0000 7fa5ade9a640 1 -- 172.21.15.52:0/2703650449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5a81025c0 msgr2=0x7fa5a8108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:14.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.252+0000 7fa5ade9a640 1 --2- 172.21.15.52:0/2703650449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5a81025c0 0x7fa5a8108e50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0009ca0 tx=0x7fa5a002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.252+0000 7fa5ade9a640 1 -- 172.21.15.52:0/2703650449 shutdown_connections 2024-04-02T13:51:14.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.252+0000 7fa5ade9a640 1 --2- 172.21.15.52:0/2703650449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5a81025c0 0x7fa5a8108e50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.252+0000 7fa5ade9a640 1 --2- 172.21.15.52:0/2703650449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5a8101c00 0x7fa5a8102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.252+0000 7fa5ade9a640 1 --2- 172.21.15.52:0/2703650449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5a8100a10 0x7fa5a8100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.252+0000 7fa5ade9a640 1 -- 172.21.15.52:0/2703650449 >> 172.21.15.52:0/2703650449 conn(0x7fa5a80fbfa0 msgr2=0x7fa5a80fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:14.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.252+0000 7fa5ade9a640 1 -- 172.21.15.52:0/2703650449 shutdown_connections 2024-04-02T13:51:14.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.252+0000 7fa5ade9a640 1 -- 172.21.15.52:0/2703650449 wait complete. 2024-04-02T13:51:14.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.253+0000 7fa5ade9a640 1 Processor -- start 2024-04-02T13:51:14.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.253+0000 7fa5ade9a640 1 -- start start 2024-04-02T13:51:14.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.253+0000 7fa5ade9a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5a8100a10 0x7fa5a806ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:14.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.253+0000 7fa5ade9a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5a8101c00 0x7fa5a819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:14.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.253+0000 7fa5ade9a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5a81025c0 0x7fa5a819f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:14.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.253+0000 7fa5ade9a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a810c670 con 0x7fa5a8100a10 2024-04-02T13:51:14.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.253+0000 7fa5ade9a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a810c4f0 con 0x7fa5a8101c00 2024-04-02T13:51:14.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.253+0000 7fa5ade9a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a810c7f0 con 0x7fa5a81025c0 2024-04-02T13:51:14.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.253+0000 7fa5a77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5a8100a10 0x7fa5a806ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:14.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.253+0000 7fa5a77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5a8100a10 0x7fa5a806ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44312/0 (socket says 172.21.15.52:44312) 2024-04-02T13:51:14.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.253+0000 7fa5a77fe640 1 -- 172.21.15.52:0/1411409711 learned_addr learned my addr 172.21.15.52:0/1411409711 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:14.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.254+0000 7fa5a77fe640 1 -- 172.21.15.52:0/1411409711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5a81025c0 msgr2=0x7fa5a819f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:14.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.254+0000 7fa5a7fff640 1 --2- 172.21.15.52:0/1411409711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5a81025c0 0x7fa5a819f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:14.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.254+0000 7fa5a6ffd640 1 --2- 172.21.15.52:0/1411409711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5a8101c00 0x7fa5a819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:14.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.254+0000 7fa5a77fe640 1 --2- 172.21.15.52:0/1411409711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5a81025c0 0x7fa5a819f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.254+0000 7fa5a77fe640 1 -- 172.21.15.52:0/1411409711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5a8101c00 msgr2=0x7fa5a819cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:14.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.254+0000 7fa5a77fe640 1 --2- 172.21.15.52:0/1411409711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5a8101c00 0x7fa5a819cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.254+0000 7fa5a77fe640 1 -- 172.21.15.52:0/1411409711 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a819f9e0 con 0x7fa5a8100a10 2024-04-02T13:51:14.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.254+0000 7fa5a7fff640 1 --2- 172.21.15.52:0/1411409711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5a81025c0 0x7fa5a819f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:51:14.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.254+0000 7fa5a77fe640 1 --2- 172.21.15.52:0/1411409711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5a8100a10 0x7fa5a806ac40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa59800e9c0 tx=0x7fa59800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:14.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.254+0000 7fa5a4ff9640 1 -- 172.21.15.52:0/1411409711 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59800cd80 con 0x7fa5a8100a10 2024-04-02T13:51:14.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.254+0000 7fa5a4ff9640 1 -- 172.21.15.52:0/1411409711 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5980035f0 con 0x7fa5a8100a10 2024-04-02T13:51:14.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.254+0000 7fa5ade9a640 1 -- 172.21.15.52:0/1411409711 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a819fc70 con 0x7fa5a8100a10 2024-04-02T13:51:14.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.254+0000 7fa5a4ff9640 1 -- 172.21.15.52:0/1411409711 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598010690 con 0x7fa5a8100a10 2024-04-02T13:51:14.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.254+0000 7fa5ade9a640 1 -- 172.21.15.52:0/1411409711 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a81a01c0 con 0x7fa5a8100a10 2024-04-02T13:51:14.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.255+0000 7fa5ade9a640 1 -- 172.21.15.52:0/1411409711 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a8107cd0 con 0x7fa5a8100a10 2024-04-02T13:51:14.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.256+0000 7fa5a4ff9640 1 -- 172.21.15.52:0/1411409711 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa5980037a0 con 0x7fa5a8100a10 2024-04-02T13:51:14.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.257+0000 7fa5a4ff9640 1 --2- 172.21.15.52:0/1411409711 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa5900a3bb0 0x7fa5900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:14.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.257+0000 7fa5a4ff9640 1 -- 172.21.15.52:0/1411409711 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa5980d0b90 con 0x7fa5a8100a10 2024-04-02T13:51:14.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.257+0000 7fa5a6ffd640 1 --2- 172.21.15.52:0/1411409711 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa5900a3bb0 0x7fa5900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:14.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.257+0000 7fa5a6ffd640 1 --2- 172.21.15.52:0/1411409711 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa5900a3bb0 0x7fa5900a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa5a806a9b0 tx=0x7fa59c005cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:14.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.261+0000 7fa5a4ff9640 1 -- 172.21.15.52:0/1411409711 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa59809d2d0 con 0x7fa5a8100a10 2024-04-02T13:51:14.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.450+0000 7fa5ade9a640 1 -- 172.21.15.52:0/1411409711 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa5a8062800 con 0x7fa5a8100a10 2024-04-02T13:51:14.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.450+0000 7fa5a4ff9640 1 -- 172.21.15.52:0/1411409711 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fa59809cbd0 con 0x7fa5a8100a10 2024-04-02T13:51:14.453 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-04-02T13:51:14.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.452+0000 7fa5ade9a640 1 -- 172.21.15.52:0/1411409711 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa5900a3bb0 msgr2=0x7fa5900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:14.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.452+0000 7fa5ade9a640 1 --2- 172.21.15.52:0/1411409711 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa5900a3bb0 0x7fa5900a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa5a806a9b0 tx=0x7fa59c005cc0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.452+0000 7fa5ade9a640 1 -- 172.21.15.52:0/1411409711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5a8100a10 msgr2=0x7fa5a806ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:14.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.452+0000 7fa5ade9a640 1 --2- 172.21.15.52:0/1411409711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5a8100a10 0x7fa5a806ac40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa59800e9c0 tx=0x7fa59800ee80 comp rx=0 tx=0).stop 2024-04-02T13:51:14.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.452+0000 7fa5ade9a640 1 -- 172.21.15.52:0/1411409711 shutdown_connections 2024-04-02T13:51:14.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.452+0000 7fa5ade9a640 1 --2- 172.21.15.52:0/1411409711 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa5900a3bb0 0x7fa5900a6070 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.452+0000 7fa5ade9a640 1 --2- 172.21.15.52:0/1411409711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5a81025c0 0x7fa5a819f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.452+0000 7fa5ade9a640 1 --2- 172.21.15.52:0/1411409711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5a8101c00 0x7fa5a819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.452+0000 7fa5ade9a640 1 --2- 172.21.15.52:0/1411409711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5a8100a10 0x7fa5a806ac40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.452+0000 7fa5ade9a640 1 -- 172.21.15.52:0/1411409711 >> 172.21.15.52:0/1411409711 conn(0x7fa5a80fbfa0 msgr2=0x7fa5a8104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:14.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.453+0000 7fa5ade9a640 1 -- 172.21.15.52:0/1411409711 shutdown_connections 2024-04-02T13:51:14.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.453+0000 7fa5ade9a640 1 -- 172.21.15.52:0/1411409711 wait complete. 2024-04-02T13:51:14.465 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:14 smithi052 ceph-mon[32104]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 2 op/s 2024-04-02T13:51:14.465 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:14 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3431783911' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:14.465 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:14 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3443782003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:51:14.465 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:14 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3140654707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:51:14.467 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:51:14.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.569+0000 7fdda33f6640 1 -- 172.21.15.52:0/4147972886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd9c1036d0 msgr2=0x7fdd9c109f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:14.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.569+0000 7fdda33f6640 1 --2- 172.21.15.52:0/4147972886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd9c1036d0 0x7fdd9c109f60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdd94009ca0 tx=0x7fdd9402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.569+0000 7fdda33f6640 1 -- 172.21.15.52:0/4147972886 shutdown_connections 2024-04-02T13:51:14.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.569+0000 7fdda33f6640 1 --2- 172.21.15.52:0/4147972886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd9c1036d0 0x7fdd9c109f60 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.569+0000 7fdda33f6640 1 --2- 172.21.15.52:0/4147972886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd9c102d10 0x7fdd9c103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.569+0000 7fdda33f6640 1 --2- 172.21.15.52:0/4147972886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd9c101b20 0x7fdd9c101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.569+0000 7fdda33f6640 1 -- 172.21.15.52:0/4147972886 >> 172.21.15.52:0/4147972886 conn(0x7fdd9c0fd090 msgr2=0x7fdd9c0ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:14.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.569+0000 7fdda33f6640 1 -- 172.21.15.52:0/4147972886 shutdown_connections 2024-04-02T13:51:14.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.569+0000 7fdda33f6640 1 -- 172.21.15.52:0/4147972886 wait complete. 2024-04-02T13:51:14.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.570+0000 7fdda33f6640 1 Processor -- start 2024-04-02T13:51:14.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.570+0000 7fdda33f6640 1 -- start start 2024-04-02T13:51:14.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.570+0000 7fdda33f6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd9c101b20 0x7fdd9c199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:14.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.570+0000 7fdda33f6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd9c102d10 0x7fdd9c19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:14.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.570+0000 7fdda33f6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd9c1036d0 0x7fdd9c1a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:14.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.570+0000 7fdda33f6640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c10d7a0 con 0x7fdd9c102d10 2024-04-02T13:51:14.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.570+0000 7fdda33f6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c10d620 con 0x7fdd9c1036d0 2024-04-02T13:51:14.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.570+0000 7fdda33f6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c10d920 con 0x7fdd9c101b20 2024-04-02T13:51:14.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.570+0000 7fdda0991640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd9c102d10 0x7fdd9c19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:14.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.570+0000 7fdda0991640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd9c102d10 0x7fdd9c19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44326/0 (socket says 172.21.15.52:44326) 2024-04-02T13:51:14.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.570+0000 7fdda0991640 1 -- 172.21.15.52:0/3635807017 learned_addr learned my addr 172.21.15.52:0/3635807017 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:14.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.570+0000 7fdda1993640 1 --2- 172.21.15.52:0/3635807017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd9c1036d0 0x7fdd9c1a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:14.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.570+0000 7fdda1192640 1 --2- 172.21.15.52:0/3635807017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd9c101b20 0x7fdd9c199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:14.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.570+0000 7fdda0991640 1 -- 172.21.15.52:0/3635807017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd9c101b20 msgr2=0x7fdd9c199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:14.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.570+0000 7fdda0991640 1 --2- 172.21.15.52:0/3635807017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd9c101b20 0x7fdd9c199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.570+0000 7fdda0991640 1 -- 172.21.15.52:0/3635807017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd9c1036d0 msgr2=0x7fdd9c1a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:14.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.570+0000 7fdda0991640 1 --2- 172.21.15.52:0/3635807017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd9c1036d0 0x7fdd9c1a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.570+0000 7fdda0991640 1 -- 172.21.15.52:0/3635807017 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9c1a1bd0 con 0x7fdd9c102d10 2024-04-02T13:51:14.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.571+0000 7fdda0991640 1 --2- 172.21.15.52:0/3635807017 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd9c102d10 0x7fdd9c19a4f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00d770 tx=0x7fdd8c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:14.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.571+0000 7fdd927fc640 1 -- 172.21.15.52:0/3635807017 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c017070 con 0x7fdd9c102d10 2024-04-02T13:51:14.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.571+0000 7fdd927fc640 1 -- 172.21.15.52:0/3635807017 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd8c005500 con 0x7fdd9c102d10 2024-04-02T13:51:14.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.571+0000 7fdda33f6640 1 -- 172.21.15.52:0/3635807017 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd9c1a1ec0 con 0x7fdd9c102d10 2024-04-02T13:51:14.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.571+0000 7fdd927fc640 1 -- 172.21.15.52:0/3635807017 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c004e00 con 0x7fdd9c102d10 2024-04-02T13:51:14.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.571+0000 7fdda33f6640 1 -- 172.21.15.52:0/3635807017 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd9c1a2330 con 0x7fdd9c102d10 2024-04-02T13:51:14.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.572+0000 7fdda33f6640 1 -- 172.21.15.52:0/3635807017 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd9c1a25c0 con 0x7fdd9c102d10 2024-04-02T13:51:14.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.573+0000 7fdd927fc640 1 -- 172.21.15.52:0/3635807017 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdd8c004fa0 con 0x7fdd9c102d10 2024-04-02T13:51:14.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.573+0000 7fdd927fc640 1 --2- 172.21.15.52:0/3635807017 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdd880a3bb0 0x7fdd880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:14.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.573+0000 7fdda1192640 1 --2- 172.21.15.52:0/3635807017 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdd880a3bb0 0x7fdd880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:14.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.573+0000 7fdd927fc640 1 -- 172.21.15.52:0/3635807017 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fdd8c0d0480 con 0x7fdd9c102d10 2024-04-02T13:51:14.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.574+0000 7fdda1192640 1 --2- 172.21.15.52:0/3635807017 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdd880a3bb0 0x7fdd880a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c102b70 tx=0x7fdd98005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:14.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.577+0000 7fdd927fc640 1 -- 172.21.15.52:0/3635807017 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fdd8c09cbc0 con 0x7fdd9c102d10 2024-04-02T13:51:14.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:14 smithi088 ceph-mon[36528]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 2 op/s 2024-04-02T13:51:14.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:14 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3431783911' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:14.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:14 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3443782003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:51:14.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:14 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3140654707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:51:14.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.764+0000 7fdda33f6640 1 -- 172.21.15.52:0/3635807017 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fdd9c1a2790 con 0x7fdd9c102d10 2024-04-02T13:51:14.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.764+0000 7fdd927fc640 1 -- 172.21.15.52:0/3635807017 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7fdd8c09c4c0 con 0x7fdd9c102d10 2024-04-02T13:51:14.768 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-04-02T13:51:14.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.766+0000 7fdda33f6640 1 -- 172.21.15.52:0/3635807017 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdd880a3bb0 msgr2=0x7fdd880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:14.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.766+0000 7fdda33f6640 1 --2- 172.21.15.52:0/3635807017 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdd880a3bb0 0x7fdd880a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c102b70 tx=0x7fdd98005f90 comp rx=0 tx=0).stop 2024-04-02T13:51:14.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.766+0000 7fdda33f6640 1 -- 172.21.15.52:0/3635807017 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd9c102d10 msgr2=0x7fdd9c19a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:14.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.766+0000 7fdda33f6640 1 --2- 172.21.15.52:0/3635807017 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd9c102d10 0x7fdd9c19a4f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00d770 tx=0x7fdd8c00dc30 comp rx=0 tx=0).stop 2024-04-02T13:51:14.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.766+0000 7fdda33f6640 1 -- 172.21.15.52:0/3635807017 shutdown_connections 2024-04-02T13:51:14.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.766+0000 7fdda33f6640 1 --2- 172.21.15.52:0/3635807017 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdd880a3bb0 0x7fdd880a6070 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.766+0000 7fdda33f6640 1 --2- 172.21.15.52:0/3635807017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd9c1036d0 0x7fdd9c1a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.766+0000 7fdda33f6640 1 --2- 172.21.15.52:0/3635807017 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd9c102d10 0x7fdd9c19a4f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.766+0000 7fdda33f6640 1 --2- 172.21.15.52:0/3635807017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd9c101b20 0x7fdd9c199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.766+0000 7fdda33f6640 1 -- 172.21.15.52:0/3635807017 >> 172.21.15.52:0/3635807017 conn(0x7fdd9c0fd090 msgr2=0x7fdd9c073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:14.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.766+0000 7fdda33f6640 1 -- 172.21.15.52:0/3635807017 shutdown_connections 2024-04-02T13:51:14.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.766+0000 7fdda33f6640 1 -- 172.21.15.52:0/3635807017 wait complete. 2024-04-02T13:51:14.780 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T13:51:14.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:14 smithi189 ceph-mon[36683]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 2 op/s 2024-04-02T13:51:14.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:14 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3431783911' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:14.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:14 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3443782003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:51:14.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:14 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3140654707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:51:14.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.881+0000 7f1f87cba640 1 -- 172.21.15.52:0/2321192637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f80102590 msgr2=0x7f1f80104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:14.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.881+0000 7f1f87cba640 1 --2- 172.21.15.52:0/2321192637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f80102590 0x7f1f80104980 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009ca0 tx=0x7f1f7402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.881+0000 7f1f87cba640 1 -- 172.21.15.52:0/2321192637 shutdown_connections 2024-04-02T13:51:14.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.881+0000 7f1f87cba640 1 --2- 172.21.15.52:0/2321192637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f801077f0 0x7f1f80109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.881+0000 7f1f87cba640 1 --2- 172.21.15.52:0/2321192637 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f80104ec0 0x7f1f801072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.881+0000 7f1f87cba640 1 --2- 172.21.15.52:0/2321192637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f80102590 0x7f1f80104980 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.881+0000 7f1f87cba640 1 -- 172.21.15.52:0/2321192637 >> 172.21.15.52:0/2321192637 conn(0x7f1f800fbfa0 msgr2=0x7f1f800fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:14.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.881+0000 7f1f87cba640 1 -- 172.21.15.52:0/2321192637 shutdown_connections 2024-04-02T13:51:14.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.881+0000 7f1f87cba640 1 -- 172.21.15.52:0/2321192637 wait complete. 2024-04-02T13:51:14.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.882+0000 7f1f87cba640 1 Processor -- start 2024-04-02T13:51:14.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.882+0000 7f1f87cba640 1 -- start start 2024-04-02T13:51:14.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.882+0000 7f1f87cba640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f80102590 0x7f1f80197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:14.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.882+0000 7f1f87cba640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f80104ec0 0x7f1f80198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:14.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.882+0000 7f1f87cba640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f801077f0 0x7f1f8019f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:14.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.882+0000 7f1f87cba640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8010d7b0 con 0x7f1f80104ec0 2024-04-02T13:51:14.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.882+0000 7f1f87cba640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8010d630 con 0x7f1f801077f0 2024-04-02T13:51:14.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.882+0000 7f1f87cba640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8010d930 con 0x7f1f80102590 2024-04-02T13:51:14.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.882+0000 7f1f85255640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f80104ec0 0x7f1f80198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:14.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.882+0000 7f1f85255640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f80104ec0 0x7f1f80198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44350/0 (socket says 172.21.15.52:44350) 2024-04-02T13:51:14.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.882+0000 7f1f85255640 1 -- 172.21.15.52:0/833837314 learned_addr learned my addr 172.21.15.52:0/833837314 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:14.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.882+0000 7f1f85a56640 1 --2- 172.21.15.52:0/833837314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f80102590 0x7f1f80197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:14.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.882+0000 7f1f86257640 1 --2- 172.21.15.52:0/833837314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f801077f0 0x7f1f8019f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:14.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.882+0000 7f1f85255640 1 -- 172.21.15.52:0/833837314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f80102590 msgr2=0x7f1f80197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:14.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.882+0000 7f1f85255640 1 --2- 172.21.15.52:0/833837314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f80102590 0x7f1f80197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.882+0000 7f1f85255640 1 -- 172.21.15.52:0/833837314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f801077f0 msgr2=0x7f1f8019f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:14.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.882+0000 7f1f85255640 1 --2- 172.21.15.52:0/833837314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f801077f0 0x7f1f8019f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:14.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.882+0000 7f1f85255640 1 -- 172.21.15.52:0/833837314 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8019fb00 con 0x7f1f80104ec0 2024-04-02T13:51:14.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.882+0000 7f1f86257640 1 --2- 172.21.15.52:0/833837314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f801077f0 0x7f1f8019f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:51:14.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.882+0000 7f1f85255640 1 --2- 172.21.15.52:0/833837314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f80104ec0 0x7f1f80198420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00e9c0 tx=0x7f1f7c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:14.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.883+0000 7f1f72ffd640 1 -- 172.21.15.52:0/833837314 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c00cd80 con 0x7f1f80104ec0 2024-04-02T13:51:14.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.883+0000 7f1f87cba640 1 -- 172.21.15.52:0/833837314 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8019fdf0 con 0x7f1f80104ec0 2024-04-02T13:51:14.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.883+0000 7f1f72ffd640 1 -- 172.21.15.52:0/833837314 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f7c005500 con 0x7f1f80104ec0 2024-04-02T13:51:14.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.883+0000 7f1f72ffd640 1 -- 172.21.15.52:0/833837314 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c010690 con 0x7f1f80104ec0 2024-04-02T13:51:14.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.883+0000 7f1f87cba640 1 -- 172.21.15.52:0/833837314 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f801a02b0 con 0x7f1f80104ec0 2024-04-02T13:51:14.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.884+0000 7f1f87cba640 1 -- 172.21.15.52:0/833837314 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f800fda90 con 0x7f1f80104ec0 2024-04-02T13:51:14.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.885+0000 7f1f72ffd640 1 -- 172.21.15.52:0/833837314 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1f7c0046c0 con 0x7f1f80104ec0 2024-04-02T13:51:14.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.885+0000 7f1f72ffd640 1 --2- 172.21.15.52:0/833837314 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1f6c0a3bb0 0x7f1f6c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:14.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.885+0000 7f1f85a56640 1 --2- 172.21.15.52:0/833837314 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1f6c0a3bb0 0x7f1f6c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:14.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.885+0000 7f1f72ffd640 1 -- 172.21.15.52:0/833837314 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f1f7c0d0a50 con 0x7f1f80104ec0 2024-04-02T13:51:14.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.886+0000 7f1f85a56640 1 --2- 172.21.15.52:0/833837314 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1f6c0a3bb0 0x7f1f6c0a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1f74006010 tx=0x7f1f74032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:14.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:14.890+0000 7f1f72ffd640 1 -- 172.21.15.52:0/833837314 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1f7c09d1f0 con 0x7f1f80104ec0 2024-04-02T13:51:15.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.089+0000 7f1f87cba640 1 -- 172.21.15.52:0/833837314 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1f8002d370 con 0x7f1f80104ec0 2024-04-02T13:51:15.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.089+0000 7f1f72ffd640 1 -- 172.21.15.52:0/833837314 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+587 (secure 0 0 0) 0x7f1f7c09caf0 con 0x7f1f80104ec0 2024-04-02T13:51:15.092 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2475099387 2024-04-03T13:43:54.653361+0000 2024-04-02T13:51:15.093 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/2368921152 2024-04-03T13:43:27.309404+0000 2024-04-02T13:51:15.093 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/673843171 2024-04-03T13:43:54.653361+0000 2024-04-02T13:51:15.093 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/2368921152 2024-04-03T13:43:27.309404+0000 2024-04-02T13:51:15.093 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2609300748 2024-04-03T13:43:27.309404+0000 2024-04-02T13:51:15.093 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/85571395 2024-04-03T13:43:27.309404+0000 2024-04-02T13:51:15.093 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/293852652 2024-04-03T13:43:54.653361+0000 2024-04-02T13:51:15.093 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/1829428070 2024-04-03T13:43:27.309404+0000 2024-04-02T13:51:15.093 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/673843171 2024-04-03T13:43:54.653361+0000 2024-04-02T13:51:15.093 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/3257395293 2024-04-03T13:43:54.653361+0000 2024-04-02T13:51:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:listed 10 entries 2024-04-02T13:51:15.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.091+0000 7f1f87cba640 1 -- 172.21.15.52:0/833837314 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1f6c0a3bb0 msgr2=0x7f1f6c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:15.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.091+0000 7f1f87cba640 1 --2- 172.21.15.52:0/833837314 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1f6c0a3bb0 0x7f1f6c0a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1f74006010 tx=0x7f1f74032590 comp rx=0 tx=0).stop 2024-04-02T13:51:15.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.091+0000 7f1f87cba640 1 -- 172.21.15.52:0/833837314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f80104ec0 msgr2=0x7f1f80198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:15.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.091+0000 7f1f87cba640 1 --2- 172.21.15.52:0/833837314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f80104ec0 0x7f1f80198420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00e9c0 tx=0x7f1f7c00ee80 comp rx=0 tx=0).stop 2024-04-02T13:51:15.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.091+0000 7f1f87cba640 1 -- 172.21.15.52:0/833837314 shutdown_connections 2024-04-02T13:51:15.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.091+0000 7f1f87cba640 1 --2- 172.21.15.52:0/833837314 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1f6c0a3bb0 0x7f1f6c0a6070 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.091+0000 7f1f87cba640 1 --2- 172.21.15.52:0/833837314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f801077f0 0x7f1f8019f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.091+0000 7f1f87cba640 1 --2- 172.21.15.52:0/833837314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f80104ec0 0x7f1f80198420 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.091+0000 7f1f87cba640 1 --2- 172.21.15.52:0/833837314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f80102590 0x7f1f80197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.091+0000 7f1f87cba640 1 -- 172.21.15.52:0/833837314 >> 172.21.15.52:0/833837314 conn(0x7f1f800fbfa0 msgr2=0x7f1f80102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:15.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.091+0000 7f1f87cba640 1 -- 172.21.15.52:0/833837314 shutdown_connections 2024-04-02T13:51:15.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.091+0000 7f1f87cba640 1 -- 172.21.15.52:0/833837314 wait complete. 2024-04-02T13:51:15.105 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-04-02T13:51:15.105 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T13:51:15.120 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:51:15.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.259+0000 7f0d8a22a640 1 -- 172.21.15.52:0/2580012309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d84101b30 msgr2=0x7f0d84101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:15.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.259+0000 7f0d8a22a640 1 --2- 172.21.15.52:0/2580012309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d84101b30 0x7f0d84101f50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009ca0 tx=0x7f0d7402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.259+0000 7f0d8a22a640 1 -- 172.21.15.52:0/2580012309 shutdown_connections 2024-04-02T13:51:15.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.259+0000 7f0d8a22a640 1 --2- 172.21.15.52:0/2580012309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d841036e0 0x7f0d84109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.259+0000 7f0d8a22a640 1 --2- 172.21.15.52:0/2580012309 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d84102d20 0x7f0d841031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.259+0000 7f0d8a22a640 1 --2- 172.21.15.52:0/2580012309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d84101b30 0x7f0d84101f50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.259+0000 7f0d8a22a640 1 -- 172.21.15.52:0/2580012309 >> 172.21.15.52:0/2580012309 conn(0x7f0d840fd0e0 msgr2=0x7f0d840ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:15.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.259+0000 7f0d8a22a640 1 -- 172.21.15.52:0/2580012309 shutdown_connections 2024-04-02T13:51:15.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.259+0000 7f0d8a22a640 1 -- 172.21.15.52:0/2580012309 wait complete. 2024-04-02T13:51:15.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.260+0000 7f0d8a22a640 1 Processor -- start 2024-04-02T13:51:15.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.260+0000 7f0d8a22a640 1 -- start start 2024-04-02T13:51:15.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.260+0000 7f0d8a22a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d84101b30 0x7f0d840758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:15.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.260+0000 7f0d8a22a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d84102d20 0x7f0d84073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:15.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.260+0000 7f0d8a22a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d841036e0 0x7f0d84074430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:15.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.260+0000 7f0d8a22a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8410d780 con 0x7f0d84101b30 2024-04-02T13:51:15.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.260+0000 7f0d8a22a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8410d600 con 0x7f0d841036e0 2024-04-02T13:51:15.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.260+0000 7f0d8a22a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8410d900 con 0x7f0d84102d20 2024-04-02T13:51:15.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.260+0000 7f0d837fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d84101b30 0x7f0d840758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:15.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.260+0000 7f0d837fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d84101b30 0x7f0d840758b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44364/0 (socket says 172.21.15.52:44364) 2024-04-02T13:51:15.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.260+0000 7f0d837fe640 1 -- 172.21.15.52:0/3645220523 learned_addr learned my addr 172.21.15.52:0/3645220523 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:15.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.261+0000 7f0d83fff640 1 --2- 172.21.15.52:0/3645220523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d841036e0 0x7f0d84074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:15.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.261+0000 7f0d82ffd640 1 --2- 172.21.15.52:0/3645220523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d84102d20 0x7f0d84073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:15.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.261+0000 7f0d837fe640 1 -- 172.21.15.52:0/3645220523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d84102d20 msgr2=0x7f0d84073ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:15.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.261+0000 7f0d837fe640 1 --2- 172.21.15.52:0/3645220523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d84102d20 0x7f0d84073ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.261+0000 7f0d837fe640 1 -- 172.21.15.52:0/3645220523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d841036e0 msgr2=0x7f0d84074430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:15.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.261+0000 7f0d837fe640 1 --2- 172.21.15.52:0/3645220523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d841036e0 0x7f0d84074430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.261+0000 7f0d837fe640 1 -- 172.21.15.52:0/3645220523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d84074cc0 con 0x7f0d84101b30 2024-04-02T13:51:15.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.261+0000 7f0d83fff640 1 --2- 172.21.15.52:0/3645220523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d841036e0 0x7f0d84074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:51:15.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.261+0000 7f0d837fe640 1 --2- 172.21.15.52:0/3645220523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d84101b30 0x7f0d840758b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0d74000f80 tx=0x7f0d74032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:15.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.261+0000 7f0d80ff9640 1 -- 172.21.15.52:0/3645220523 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74039040 con 0x7f0d84101b30 2024-04-02T13:51:15.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.261+0000 7f0d80ff9640 1 -- 172.21.15.52:0/3645220523 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d74044070 con 0x7f0d84101b30 2024-04-02T13:51:15.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.261+0000 7f0d8a22a640 1 -- 172.21.15.52:0/3645220523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d841a1e80 con 0x7f0d84101b30 2024-04-02T13:51:15.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.261+0000 7f0d80ff9640 1 -- 172.21.15.52:0/3645220523 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d740042a0 con 0x7f0d84101b30 2024-04-02T13:51:15.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.261+0000 7f0d8a22a640 1 -- 172.21.15.52:0/3645220523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d841a2280 con 0x7f0d84101b30 2024-04-02T13:51:15.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.262+0000 7f0d8a22a640 1 -- 172.21.15.52:0/3645220523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d84108df0 con 0x7f0d84101b30 2024-04-02T13:51:15.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.263+0000 7f0d80ff9640 1 -- 172.21.15.52:0/3645220523 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0d74003970 con 0x7f0d84101b30 2024-04-02T13:51:15.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.264+0000 7f0d80ff9640 1 --2- 172.21.15.52:0/3645220523 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f0d6c0a3bb0 0x7f0d6c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:15.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.264+0000 7f0d82ffd640 1 --2- 172.21.15.52:0/3645220523 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f0d6c0a3bb0 0x7f0d6c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:15.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.264+0000 7f0d80ff9640 1 -- 172.21.15.52:0/3645220523 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f0d740f3520 con 0x7f0d84101b30 2024-04-02T13:51:15.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.264+0000 7f0d82ffd640 1 --2- 172.21.15.52:0/3645220523 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f0d6c0a3bb0 0x7f0d6c0a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0d84075620 tx=0x7f0d78005cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:15.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.267+0000 7f0d80ff9640 1 -- 172.21.15.52:0/3645220523 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0d740bfcc0 con 0x7f0d84101b30 2024-04-02T13:51:15.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.455+0000 7f0d8a22a640 1 -- 172.21.15.52:0/3645220523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d84062800 con 0x7f0d84101b30 2024-04-02T13:51:15.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.457+0000 7f0d80ff9640 1 -- 172.21.15.52:0/3645220523 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24077 (secure 0 0 0) 0x7f0d740bf5c0 con 0x7f0d84101b30 2024-04-02T13:51:15.460 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:51:15.461 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":90,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T13:51:05.722301+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:50:58.778690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:50:59.561527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:51:03.343487+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:51:15.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.459+0000 7f0d8a22a640 1 -- 172.21.15.52:0/3645220523 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f0d6c0a3bb0 msgr2=0x7f0d6c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:15.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.459+0000 7f0d8a22a640 1 --2- 172.21.15.52:0/3645220523 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f0d6c0a3bb0 0x7f0d6c0a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0d84075620 tx=0x7f0d78005cc0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.459+0000 7f0d8a22a640 1 -- 172.21.15.52:0/3645220523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d84101b30 msgr2=0x7f0d840758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:15.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.459+0000 7f0d8a22a640 1 --2- 172.21.15.52:0/3645220523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d84101b30 0x7f0d840758b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0d74000f80 tx=0x7f0d74032ec0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.459+0000 7f0d8a22a640 1 -- 172.21.15.52:0/3645220523 shutdown_connections 2024-04-02T13:51:15.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.459+0000 7f0d8a22a640 1 --2- 172.21.15.52:0/3645220523 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f0d6c0a3bb0 0x7f0d6c0a6070 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.459+0000 7f0d8a22a640 1 --2- 172.21.15.52:0/3645220523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d841036e0 0x7f0d84074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.459+0000 7f0d8a22a640 1 --2- 172.21.15.52:0/3645220523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d84102d20 0x7f0d84073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.459+0000 7f0d8a22a640 1 --2- 172.21.15.52:0/3645220523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d84101b30 0x7f0d840758b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.459+0000 7f0d8a22a640 1 -- 172.21.15.52:0/3645220523 >> 172.21.15.52:0/3645220523 conn(0x7f0d840fd0e0 msgr2=0x7f0d84105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:15.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.459+0000 7f0d8a22a640 1 -- 172.21.15.52:0/3645220523 shutdown_connections 2024-04-02T13:51:15.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.459+0000 7f0d8a22a640 1 -- 172.21.15.52:0/3645220523 wait complete. 2024-04-02T13:51:15.473 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:46:20.746053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:50:58.778690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:50:59.561527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:51:03.343487+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:51:15.473 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:51:15.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.577+0000 7f73497a8640 1 -- 172.21.15.52:0/1003396021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7344107800 msgr2=0x7f7344109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:15.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.577+0000 7f73497a8640 1 --2- 172.21.15.52:0/1003396021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7344107800 0x7f7344109c50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f733c009ca0 tx=0x7f733c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.577+0000 7f73497a8640 1 -- 172.21.15.52:0/1003396021 shutdown_connections 2024-04-02T13:51:15.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.577+0000 7f73497a8640 1 --2- 172.21.15.52:0/1003396021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7344107800 0x7f7344109c50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.577+0000 7f73497a8640 1 --2- 172.21.15.52:0/1003396021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7344104ed0 0x7f73441072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.577+0000 7f73497a8640 1 --2- 172.21.15.52:0/1003396021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73441025a0 0x7f7344104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.577+0000 7f73497a8640 1 -- 172.21.15.52:0/1003396021 >> 172.21.15.52:0/1003396021 conn(0x7f73440fbfb0 msgr2=0x7f73440fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:15.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.577+0000 7f73497a8640 1 -- 172.21.15.52:0/1003396021 shutdown_connections 2024-04-02T13:51:15.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.577+0000 7f73497a8640 1 -- 172.21.15.52:0/1003396021 wait complete. 2024-04-02T13:51:15.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.578+0000 7f73497a8640 1 Processor -- start 2024-04-02T13:51:15.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.578+0000 7f73497a8640 1 -- start start 2024-04-02T13:51:15.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.578+0000 7f73497a8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73441025a0 0x7f7344197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:15.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.578+0000 7f73497a8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7344104ed0 0x7f7344198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:15.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.578+0000 7f73497a8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7344107800 0x7f734419f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:15.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.578+0000 7f73497a8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f734410d7c0 con 0x7f7344107800 2024-04-02T13:51:15.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.578+0000 7f73497a8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f734410d640 con 0x7f73441025a0 2024-04-02T13:51:15.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.578+0000 7f73497a8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f734410d940 con 0x7f7344104ed0 2024-04-02T13:51:15.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.578+0000 7f73437fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7344107800 0x7f734419f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:15.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.578+0000 7f73437fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7344107800 0x7f734419f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44390/0 (socket says 172.21.15.52:44390) 2024-04-02T13:51:15.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.578+0000 7f73437fe640 1 -- 172.21.15.52:0/340257665 learned_addr learned my addr 172.21.15.52:0/340257665 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:15.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.578+0000 7f7342ffd640 1 --2- 172.21.15.52:0/340257665 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73441025a0 0x7f7344197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:15.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.578+0000 7f733bfff640 1 --2- 172.21.15.52:0/340257665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7344104ed0 0x7f7344198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:15.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.578+0000 7f73437fe640 1 -- 172.21.15.52:0/340257665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7344104ed0 msgr2=0x7f7344198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:15.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.578+0000 7f73437fe640 1 --2- 172.21.15.52:0/340257665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7344104ed0 0x7f7344198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.578+0000 7f73437fe640 1 -- 172.21.15.52:0/340257665 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73441025a0 msgr2=0x7f7344197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:15.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.578+0000 7f73437fe640 1 --2- 172.21.15.52:0/340257665 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73441025a0 0x7f7344197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.578+0000 7f73437fe640 1 -- 172.21.15.52:0/340257665 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734419fa20 con 0x7f7344107800 2024-04-02T13:51:15.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.578+0000 7f733bfff640 1 --2- 172.21.15.52:0/340257665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7344104ed0 0x7f7344198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:51:15.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.578+0000 7f73437fe640 1 --2- 172.21.15.52:0/340257665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7344107800 0x7f734419f3b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f733c02c700 tx=0x7f733c032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:15.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.579+0000 7f7340ff9640 1 -- 172.21.15.52:0/340257665 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c039040 con 0x7f7344107800 2024-04-02T13:51:15.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.579+0000 7f7340ff9640 1 -- 172.21.15.52:0/340257665 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f733c044070 con 0x7f7344107800 2024-04-02T13:51:15.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.579+0000 7f73497a8640 1 -- 172.21.15.52:0/340257665 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f734419fcb0 con 0x7f7344107800 2024-04-02T13:51:15.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.579+0000 7f7340ff9640 1 -- 172.21.15.52:0/340257665 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c0042a0 con 0x7f7344107800 2024-04-02T13:51:15.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.579+0000 7f73497a8640 1 -- 172.21.15.52:0/340257665 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73441a01a0 con 0x7f7344107800 2024-04-02T13:51:15.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.580+0000 7f73497a8640 1 -- 172.21.15.52:0/340257665 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73440fdaa0 con 0x7f7344107800 2024-04-02T13:51:15.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.581+0000 7f7340ff9640 1 -- 172.21.15.52:0/340257665 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f733c003970 con 0x7f7344107800 2024-04-02T13:51:15.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.581+0000 7f7340ff9640 1 --2- 172.21.15.52:0/340257665 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f73300a3bb0 0x7f73300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:15.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.582+0000 7f7342ffd640 1 --2- 172.21.15.52:0/340257665 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f73300a3bb0 0x7f73300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:15.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.582+0000 7f7340ff9640 1 -- 172.21.15.52:0/340257665 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f733c0f3200 con 0x7f7344107800 2024-04-02T13:51:15.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.582+0000 7f7342ffd640 1 --2- 172.21.15.52:0/340257665 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f73300a3bb0 0x7f73300a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7334005fd0 tx=0x7f7334005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:15.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.585+0000 7f7340ff9640 1 -- 172.21.15.52:0/340257665 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f733c0bfa30 con 0x7f7344107800 2024-04-02T13:51:15.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:15 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1411409711' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:51:15.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:15 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3635807017' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:51:15.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:15 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/833837314' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:51:15.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.770+0000 7f73497a8640 1 -- 172.21.15.52:0/340257665 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f734402d5c0 con 0x7f7344107800 2024-04-02T13:51:15.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.770+0000 7f7340ff9640 1 -- 172.21.15.52:0/340257665 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f733c047e40 con 0x7f7344107800 2024-04-02T13:51:15.773 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-04-02T13:51:15.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.772+0000 7f73497a8640 1 -- 172.21.15.52:0/340257665 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f73300a3bb0 msgr2=0x7f73300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:15.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.772+0000 7f73497a8640 1 --2- 172.21.15.52:0/340257665 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f73300a3bb0 0x7f73300a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7334005fd0 tx=0x7f7334005850 comp rx=0 tx=0).stop 2024-04-02T13:51:15.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.772+0000 7f73497a8640 1 -- 172.21.15.52:0/340257665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7344107800 msgr2=0x7f734419f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:15.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.772+0000 7f73497a8640 1 --2- 172.21.15.52:0/340257665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7344107800 0x7f734419f3b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f733c02c700 tx=0x7f733c032f20 comp rx=0 tx=0).stop 2024-04-02T13:51:15.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.772+0000 7f73497a8640 1 -- 172.21.15.52:0/340257665 shutdown_connections 2024-04-02T13:51:15.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.772+0000 7f73497a8640 1 --2- 172.21.15.52:0/340257665 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f73300a3bb0 0x7f73300a6070 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.772+0000 7f73497a8640 1 --2- 172.21.15.52:0/340257665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7344107800 0x7f734419f3b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.772+0000 7f73497a8640 1 --2- 172.21.15.52:0/340257665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7344104ed0 0x7f7344198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.772+0000 7f73497a8640 1 --2- 172.21.15.52:0/340257665 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73441025a0 0x7f7344197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.772+0000 7f73497a8640 1 -- 172.21.15.52:0/340257665 >> 172.21.15.52:0/340257665 conn(0x7f73440fbfb0 msgr2=0x7f7344102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:15.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.772+0000 7f73497a8640 1 -- 172.21.15.52:0/340257665 shutdown_connections 2024-04-02T13:51:15.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.772+0000 7f73497a8640 1 -- 172.21.15.52:0/340257665 wait complete. 2024-04-02T13:51:15.785 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:15 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1411409711' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:51:15.785 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:15 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3635807017' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:51:15.785 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:15 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/833837314' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:51:15.786 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:51:15.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:15 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1411409711' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:51:15.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:15 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3635807017' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:51:15.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:15 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/833837314' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:51:15.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.889+0000 7f7702532640 1 -- 172.21.15.52:0/4111619256 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76fc101b10 msgr2=0x7f76fc101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:15.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.889+0000 7f7702532640 1 --2- 172.21.15.52:0/4111619256 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76fc101b10 0x7f76fc101f30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009900 tx=0x7f76ec02bc30 comp rx=0 tx=0).stop 2024-04-02T13:51:15.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.889+0000 7f7702532640 1 -- 172.21.15.52:0/4111619256 shutdown_connections 2024-04-02T13:51:15.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.889+0000 7f7702532640 1 --2- 172.21.15.52:0/4111619256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76fc1036c0 0x7f76fc109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.889+0000 7f7702532640 1 --2- 172.21.15.52:0/4111619256 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76fc102d00 0x7f76fc103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.889+0000 7f7702532640 1 --2- 172.21.15.52:0/4111619256 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76fc101b10 0x7f76fc101f30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.889+0000 7f7702532640 1 -- 172.21.15.52:0/4111619256 >> 172.21.15.52:0/4111619256 conn(0x7f76fc0fd0a0 msgr2=0x7f76fc0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:15.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.889+0000 7f7702532640 1 -- 172.21.15.52:0/4111619256 shutdown_connections 2024-04-02T13:51:15.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.889+0000 7f7702532640 1 -- 172.21.15.52:0/4111619256 wait complete. 2024-04-02T13:51:15.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.890+0000 7f7702532640 1 Processor -- start 2024-04-02T13:51:15.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.890+0000 7f7702532640 1 -- start start 2024-04-02T13:51:15.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.890+0000 7f7702532640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76fc101b10 0x7f76fc197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:15.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.890+0000 7f7702532640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76fc102d00 0x7f76fc198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:15.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.890+0000 7f7702532640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76fc1036c0 0x7f76fc19f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:15.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.890+0000 7f7702532640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc10d790 con 0x7f76fc1036c0 2024-04-02T13:51:15.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.890+0000 7f7702532640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc10d610 con 0x7f76fc102d00 2024-04-02T13:51:15.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.890+0000 7f7702532640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc10d910 con 0x7f76fc101b10 2024-04-02T13:51:15.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.890+0000 7f7700acf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76fc1036c0 0x7f76fc19f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:15.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.890+0000 7f7700acf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76fc1036c0 0x7f76fc19f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44408/0 (socket says 172.21.15.52:44408) 2024-04-02T13:51:15.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.890+0000 7f7700acf640 1 -- 172.21.15.52:0/964991035 learned_addr learned my addr 172.21.15.52:0/964991035 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:15.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.890+0000 7f76fb7fe640 1 --2- 172.21.15.52:0/964991035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76fc102d00 0x7f76fc198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:15.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.890+0000 7f76fbfff640 1 --2- 172.21.15.52:0/964991035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76fc101b10 0x7f76fc197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:15.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.890+0000 7f7700acf640 1 -- 172.21.15.52:0/964991035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76fc101b10 msgr2=0x7f76fc197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:15.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.890+0000 7f7700acf640 1 --2- 172.21.15.52:0/964991035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76fc101b10 0x7f76fc197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.890+0000 7f7700acf640 1 -- 172.21.15.52:0/964991035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76fc102d00 msgr2=0x7f76fc198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:15.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.890+0000 7f7700acf640 1 --2- 172.21.15.52:0/964991035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76fc102d00 0x7f76fc198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:15.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.891+0000 7f7700acf640 1 -- 172.21.15.52:0/964991035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76fc19fa00 con 0x7f76fc1036c0 2024-04-02T13:51:15.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.891+0000 7f76fbfff640 1 --2- 172.21.15.52:0/964991035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76fc101b10 0x7f76fc197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:51:15.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.891+0000 7f76fb7fe640 1 --2- 172.21.15.52:0/964991035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76fc102d00 0x7f76fc198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:51:15.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.891+0000 7f7700acf640 1 --2- 172.21.15.52:0/964991035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76fc1036c0 0x7f76fc19f390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f76f400eb10 tx=0x7f76f400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:15.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.891+0000 7f76f97fa640 1 -- 172.21.15.52:0/964991035 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f4004b20 con 0x7f76fc1036c0 2024-04-02T13:51:15.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.891+0000 7f76f97fa640 1 -- 172.21.15.52:0/964991035 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76f4004cc0 con 0x7f76fc1036c0 2024-04-02T13:51:15.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.891+0000 7f7702532640 1 -- 172.21.15.52:0/964991035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76fc19fcf0 con 0x7f76fc1036c0 2024-04-02T13:51:15.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.891+0000 7f76f97fa640 1 -- 172.21.15.52:0/964991035 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f4010760 con 0x7f76fc1036c0 2024-04-02T13:51:15.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.891+0000 7f7702532640 1 -- 172.21.15.52:0/964991035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76fc1a01e0 con 0x7f76fc1036c0 2024-04-02T13:51:15.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.892+0000 7f7702532640 1 -- 172.21.15.52:0/964991035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76fc108e30 con 0x7f76fc1036c0 2024-04-02T13:51:15.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.893+0000 7f76f97fa640 1 -- 172.21.15.52:0/964991035 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f76f4005020 con 0x7f76fc1036c0 2024-04-02T13:51:15.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.893+0000 7f76f97fa640 1 --2- 172.21.15.52:0/964991035 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f76e40a3bb0 0x7f76e40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:15.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.894+0000 7f76f97fa640 1 -- 172.21.15.52:0/964991035 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f76f40d0e70 con 0x7f76fc1036c0 2024-04-02T13:51:15.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.894+0000 7f76fbfff640 1 --2- 172.21.15.52:0/964991035 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f76e40a3bb0 0x7f76e40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:15.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.894+0000 7f76fbfff640 1 --2- 172.21.15.52:0/964991035 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f76e40a3bb0 0x7f76e40a6070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f76ec036040 tx=0x7f76ec000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:15.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:15.898+0000 7f76f97fa640 1 -- 172.21.15.52:0/964991035 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f76f409d5b0 con 0x7f76fc1036c0 2024-04-02T13:51:16.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.083+0000 7f7702532640 1 -- 172.21.15.52:0/964991035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f76fc0627a0 con 0x7f76fc1036c0 2024-04-02T13:51:16.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.083+0000 7f76f97fa640 1 -- 172.21.15.52:0/964991035 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f76f409ceb0 con 0x7f76fc1036c0 2024-04-02T13:51:16.087 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-04-02T13:51:16.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.085+0000 7f7702532640 1 -- 172.21.15.52:0/964991035 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f76e40a3bb0 msgr2=0x7f76e40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:16.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.085+0000 7f7702532640 1 --2- 172.21.15.52:0/964991035 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f76e40a3bb0 0x7f76e40a6070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f76ec036040 tx=0x7f76ec000bc0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.085+0000 7f7702532640 1 -- 172.21.15.52:0/964991035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76fc1036c0 msgr2=0x7f76fc19f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:16.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.085+0000 7f7702532640 1 --2- 172.21.15.52:0/964991035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76fc1036c0 0x7f76fc19f390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f76f400eb10 tx=0x7f76f400c2d0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.085+0000 7f7702532640 1 -- 172.21.15.52:0/964991035 shutdown_connections 2024-04-02T13:51:16.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.085+0000 7f7702532640 1 --2- 172.21.15.52:0/964991035 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f76e40a3bb0 0x7f76e40a6070 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.085+0000 7f7702532640 1 --2- 172.21.15.52:0/964991035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76fc1036c0 0x7f76fc19f390 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.085+0000 7f7702532640 1 --2- 172.21.15.52:0/964991035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76fc102d00 0x7f76fc198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.085+0000 7f7702532640 1 --2- 172.21.15.52:0/964991035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76fc101b10 0x7f76fc197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.085+0000 7f7702532640 1 -- 172.21.15.52:0/964991035 >> 172.21.15.52:0/964991035 conn(0x7f76fc0fd0a0 msgr2=0x7f76fc106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:16.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.085+0000 7f7702532640 1 -- 172.21.15.52:0/964991035 shutdown_connections 2024-04-02T13:51:16.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.085+0000 7f7702532640 1 -- 172.21.15.52:0/964991035 wait complete. 2024-04-02T13:51:16.099 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:51:16.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.201+0000 7f10c16f2640 1 -- 172.21.15.52:0/2033085770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10bc100a00 msgr2=0x7f10bc100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:16.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.201+0000 7f10c16f2640 1 --2- 172.21.15.52:0/2033085770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10bc100a00 0x7f10bc100e20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009900 tx=0x7f10b402bc30 comp rx=0 tx=0).stop 2024-04-02T13:51:16.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.201+0000 7f10c16f2640 1 -- 172.21.15.52:0/2033085770 shutdown_connections 2024-04-02T13:51:16.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.201+0000 7f10c16f2640 1 --2- 172.21.15.52:0/2033085770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10bc1025b0 0x7f10bc108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.201+0000 7f10c16f2640 1 --2- 172.21.15.52:0/2033085770 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10bc101bf0 0x7f10bc102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.201+0000 7f10c16f2640 1 --2- 172.21.15.52:0/2033085770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10bc100a00 0x7f10bc100e20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.201+0000 7f10c16f2640 1 -- 172.21.15.52:0/2033085770 >> 172.21.15.52:0/2033085770 conn(0x7f10bc0fbf90 msgr2=0x7f10bc0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:16.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.201+0000 7f10c16f2640 1 -- 172.21.15.52:0/2033085770 shutdown_connections 2024-04-02T13:51:16.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.201+0000 7f10c16f2640 1 -- 172.21.15.52:0/2033085770 wait complete. 2024-04-02T13:51:16.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.202+0000 7f10c16f2640 1 Processor -- start 2024-04-02T13:51:16.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.202+0000 7f10c16f2640 1 -- start start 2024-04-02T13:51:16.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.202+0000 7f10c16f2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10bc100a00 0x7f10bc10b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:16.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.202+0000 7f10c16f2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10bc101bf0 0x7f10bc1099c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:16.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.202+0000 7f10c16f2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10bc1025b0 0x7f10bc109f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:16.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.202+0000 7f10c16f2640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc10d7a0 con 0x7f10bc1025b0 2024-04-02T13:51:16.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.202+0000 7f10c16f2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc10d620 con 0x7f10bc101bf0 2024-04-02T13:51:16.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.202+0000 7f10c16f2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc10d920 con 0x7f10bc100a00 2024-04-02T13:51:16.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.202+0000 7f10bb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10bc1025b0 0x7f10bc109f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:16.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.202+0000 7f10bb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10bc1025b0 0x7f10bc109f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44436/0 (socket says 172.21.15.52:44436) 2024-04-02T13:51:16.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.202+0000 7f10bb7fe640 1 -- 172.21.15.52:0/51907382 learned_addr learned my addr 172.21.15.52:0/51907382 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:16.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.202+0000 7f10baffd640 1 --2- 172.21.15.52:0/51907382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10bc100a00 0x7f10bc10b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:16.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.202+0000 7f10ba7fc640 1 --2- 172.21.15.52:0/51907382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10bc101bf0 0x7f10bc1099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:16.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.203+0000 7f10bb7fe640 1 -- 172.21.15.52:0/51907382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10bc100a00 msgr2=0x7f10bc10b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:16.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.203+0000 7f10bb7fe640 1 --2- 172.21.15.52:0/51907382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10bc100a00 0x7f10bc10b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.203+0000 7f10bb7fe640 1 -- 172.21.15.52:0/51907382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10bc101bf0 msgr2=0x7f10bc1099c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:16.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.203+0000 7f10bb7fe640 1 --2- 172.21.15.52:0/51907382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10bc101bf0 0x7f10bc1099c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.203+0000 7f10bb7fe640 1 -- 172.21.15.52:0/51907382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10bc10a790 con 0x7f10bc1025b0 2024-04-02T13:51:16.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.203+0000 7f10ba7fc640 1 --2- 172.21.15.52:0/51907382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10bc101bf0 0x7f10bc1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:51:16.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.203+0000 7f10bb7fe640 1 --2- 172.21.15.52:0/51907382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10bc1025b0 0x7f10bc109f00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00eaa0 tx=0x7f10ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:16.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.203+0000 7f10b3fff640 1 -- 172.21.15.52:0/51907382 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac00cdb0 con 0x7f10bc1025b0 2024-04-02T13:51:16.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.203+0000 7f10b3fff640 1 -- 172.21.15.52:0/51907382 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10ac005020 con 0x7f10bc1025b0 2024-04-02T13:51:16.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.203+0000 7f10c16f2640 1 -- 172.21.15.52:0/51907382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10bc1a1ee0 con 0x7f10bc1025b0 2024-04-02T13:51:16.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.203+0000 7f10b3fff640 1 -- 172.21.15.52:0/51907382 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac0116b0 con 0x7f10bc1025b0 2024-04-02T13:51:16.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.203+0000 7f10c16f2640 1 -- 172.21.15.52:0/51907382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10bc1a2340 con 0x7f10bc1025b0 2024-04-02T13:51:16.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.204+0000 7f10c16f2640 1 -- 172.21.15.52:0/51907382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10bc107d20 con 0x7f10bc1025b0 2024-04-02T13:51:16.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.205+0000 7f10b3fff640 1 -- 172.21.15.52:0/51907382 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f10ac005560 con 0x7f10bc1025b0 2024-04-02T13:51:16.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.205+0000 7f10b3fff640 1 --2- 172.21.15.52:0/51907382 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f10a80a3bb0 0x7f10a80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:16.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.206+0000 7f10baffd640 1 --2- 172.21.15.52:0/51907382 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f10a80a3bb0 0x7f10a80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:16.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.206+0000 7f10b3fff640 1 -- 172.21.15.52:0/51907382 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f10ac0d08c0 con 0x7f10bc1025b0 2024-04-02T13:51:16.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.206+0000 7f10baffd640 1 --2- 172.21.15.52:0/51907382 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f10a80a3bb0 0x7f10a80a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f10b4000c00 tx=0x7f10b4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:16.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.210+0000 7f10b3fff640 1 -- 172.21.15.52:0/51907382 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f10ac016030 con 0x7f10bc1025b0 2024-04-02T13:51:16.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.399+0000 7f10c16f2640 1 -- 172.21.15.52:0/51907382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f10bc0627a0 con 0x7f10bc1025b0 2024-04-02T13:51:16.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.399+0000 7f10b3fff640 1 -- 172.21.15.52:0/51907382 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f10ac09d000 con 0x7f10bc1025b0 2024-04-02T13:51:16.402 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-04-02T13:51:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.401+0000 7f10c16f2640 1 -- 172.21.15.52:0/51907382 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f10a80a3bb0 msgr2=0x7f10a80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.401+0000 7f10c16f2640 1 --2- 172.21.15.52:0/51907382 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f10a80a3bb0 0x7f10a80a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f10b4000c00 tx=0x7f10b4005fd0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.401+0000 7f10c16f2640 1 -- 172.21.15.52:0/51907382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10bc1025b0 msgr2=0x7f10bc109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.401+0000 7f10c16f2640 1 --2- 172.21.15.52:0/51907382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10bc1025b0 0x7f10bc109f00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00eaa0 tx=0x7f10ac00ef60 comp rx=0 tx=0).stop 2024-04-02T13:51:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.401+0000 7f10c16f2640 1 -- 172.21.15.52:0/51907382 shutdown_connections 2024-04-02T13:51:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.402+0000 7f10c16f2640 1 --2- 172.21.15.52:0/51907382 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f10a80a3bb0 0x7f10a80a6070 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.402+0000 7f10c16f2640 1 --2- 172.21.15.52:0/51907382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10bc1025b0 0x7f10bc109f00 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.402+0000 7f10c16f2640 1 --2- 172.21.15.52:0/51907382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10bc101bf0 0x7f10bc1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.402+0000 7f10c16f2640 1 --2- 172.21.15.52:0/51907382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f10bc100a00 0x7f10bc10b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.402+0000 7f10c16f2640 1 -- 172.21.15.52:0/51907382 >> 172.21.15.52:0/51907382 conn(0x7f10bc0fbf90 msgr2=0x7f10bc104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:16.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.402+0000 7f10c16f2640 1 -- 172.21.15.52:0/51907382 shutdown_connections 2024-04-02T13:51:16.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.402+0000 7f10c16f2640 1 -- 172.21.15.52:0/51907382 wait complete. 2024-04-02T13:51:16.415 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:51:16.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.518+0000 7fe71d5f2640 1 -- 172.21.15.52:0/1688846664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7181036d0 msgr2=0x7fe718109f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:16.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.518+0000 7fe71d5f2640 1 --2- 172.21.15.52:0/1688846664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7181036d0 0x7fe718109f60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe710009ca0 tx=0x7fe71002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.518+0000 7fe71d5f2640 1 -- 172.21.15.52:0/1688846664 shutdown_connections 2024-04-02T13:51:16.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.518+0000 7fe71d5f2640 1 --2- 172.21.15.52:0/1688846664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7181036d0 0x7fe718109f60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.518+0000 7fe71d5f2640 1 --2- 172.21.15.52:0/1688846664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe718102d10 0x7fe718103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.518+0000 7fe71d5f2640 1 --2- 172.21.15.52:0/1688846664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe718101b20 0x7fe718101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.518+0000 7fe71d5f2640 1 -- 172.21.15.52:0/1688846664 >> 172.21.15.52:0/1688846664 conn(0x7fe7180fd090 msgr2=0x7fe7180ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:16.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.518+0000 7fe71d5f2640 1 -- 172.21.15.52:0/1688846664 shutdown_connections 2024-04-02T13:51:16.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.518+0000 7fe71d5f2640 1 -- 172.21.15.52:0/1688846664 wait complete. 2024-04-02T13:51:16.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.518+0000 7fe71d5f2640 1 Processor -- start 2024-04-02T13:51:16.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.518+0000 7fe71d5f2640 1 -- start start 2024-04-02T13:51:16.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.519+0000 7fe71d5f2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe718101b20 0x7fe718197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:16.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.519+0000 7fe71d5f2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe718102d10 0x7fe7181982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:16.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.519+0000 7fe71d5f2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7181036d0 0x7fe71819f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:16.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.519+0000 7fe71d5f2640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe71810d780 con 0x7fe718101b20 2024-04-02T13:51:16.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.519+0000 7fe71d5f2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe71810d600 con 0x7fe718102d10 2024-04-02T13:51:16.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.519+0000 7fe71d5f2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe71810d900 con 0x7fe7181036d0 2024-04-02T13:51:16.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.519+0000 7fe716ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe718101b20 0x7fe718197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:16.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.519+0000 7fe716ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe718101b20 0x7fe718197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44466/0 (socket says 172.21.15.52:44466) 2024-04-02T13:51:16.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.519+0000 7fe716ffd640 1 -- 172.21.15.52:0/2386396612 learned_addr learned my addr 172.21.15.52:0/2386396612 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:16.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.519+0000 7fe7177fe640 1 --2- 172.21.15.52:0/2386396612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7181036d0 0x7fe71819f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:16.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.519+0000 7fe70ffff640 1 --2- 172.21.15.52:0/2386396612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe718102d10 0x7fe7181982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:16.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.519+0000 7fe716ffd640 1 -- 172.21.15.52:0/2386396612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7181036d0 msgr2=0x7fe71819f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:16.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.519+0000 7fe716ffd640 1 --2- 172.21.15.52:0/2386396612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7181036d0 0x7fe71819f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.519+0000 7fe716ffd640 1 -- 172.21.15.52:0/2386396612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe718102d10 msgr2=0x7fe7181982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:16.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.519+0000 7fe716ffd640 1 --2- 172.21.15.52:0/2386396612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe718102d10 0x7fe7181982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.519+0000 7fe716ffd640 1 -- 172.21.15.52:0/2386396612 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71819f9c0 con 0x7fe718101b20 2024-04-02T13:51:16.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.519+0000 7fe716ffd640 1 --2- 172.21.15.52:0/2386396612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe718101b20 0x7fe718197db0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe70800d770 tx=0x7fe70800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:16.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.519+0000 7fe714ff9640 1 -- 172.21.15.52:0/2386396612 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708017070 con 0x7fe718101b20 2024-04-02T13:51:16.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.519+0000 7fe714ff9640 1 -- 172.21.15.52:0/2386396612 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe708009c30 con 0x7fe718101b20 2024-04-02T13:51:16.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.519+0000 7fe71d5f2640 1 -- 172.21.15.52:0/2386396612 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe71819fcb0 con 0x7fe718101b20 2024-04-02T13:51:16.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.519+0000 7fe714ff9640 1 -- 172.21.15.52:0/2386396612 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708003dd0 con 0x7fe718101b20 2024-04-02T13:51:16.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.519+0000 7fe71d5f2640 1 -- 172.21.15.52:0/2386396612 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7181a01b0 con 0x7fe718101b20 2024-04-02T13:51:16.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.520+0000 7fe71d5f2640 1 -- 172.21.15.52:0/2386396612 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe718108e40 con 0x7fe718101b20 2024-04-02T13:51:16.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.521+0000 7fe714ff9640 1 -- 172.21.15.52:0/2386396612 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe708003fb0 con 0x7fe718101b20 2024-04-02T13:51:16.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.522+0000 7fe714ff9640 1 --2- 172.21.15.52:0/2386396612 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe7040a3bb0 0x7fe7040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:16.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.522+0000 7fe70ffff640 1 --2- 172.21.15.52:0/2386396612 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe7040a3bb0 0x7fe7040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:16.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.522+0000 7fe714ff9640 1 -- 172.21.15.52:0/2386396612 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe7080d00f0 con 0x7fe718101b20 2024-04-02T13:51:16.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.522+0000 7fe70ffff640 1 --2- 172.21.15.52:0/2386396612 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe7040a3bb0 0x7fe7040a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe7000058c0 tx=0x7fe70000a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:16.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.526+0000 7fe714ff9640 1 -- 172.21.15.52:0/2386396612 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe70809c890 con 0x7fe718101b20 2024-04-02T13:51:16.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:16 smithi088 ceph-mon[36528]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 218 B/s wr, 0 op/s 2024-04-02T13:51:16.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:16 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3645220523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:16.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:16 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/340257665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:51:16.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:16 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/964991035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:51:16.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.712+0000 7fe71d5f2640 1 -- 172.21.15.52:0/2386396612 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe71802d370 con 0x7fe718101b20 2024-04-02T13:51:16.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.712+0000 7fe714ff9640 1 -- 172.21.15.52:0/2386396612 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7fe71802d370 con 0x7fe718101b20 2024-04-02T13:51:16.715 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-04-02T13:51:16.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.714+0000 7fe71d5f2640 1 -- 172.21.15.52:0/2386396612 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe7040a3bb0 msgr2=0x7fe7040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:16.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.714+0000 7fe71d5f2640 1 --2- 172.21.15.52:0/2386396612 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe7040a3bb0 0x7fe7040a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe7000058c0 tx=0x7fe70000a4c0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.714+0000 7fe71d5f2640 1 -- 172.21.15.52:0/2386396612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe718101b20 msgr2=0x7fe718197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:16.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.714+0000 7fe71d5f2640 1 --2- 172.21.15.52:0/2386396612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe718101b20 0x7fe718197db0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe70800d770 tx=0x7fe70800dc30 comp rx=0 tx=0).stop 2024-04-02T13:51:16.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.714+0000 7fe71d5f2640 1 -- 172.21.15.52:0/2386396612 shutdown_connections 2024-04-02T13:51:16.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.714+0000 7fe71d5f2640 1 --2- 172.21.15.52:0/2386396612 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe7040a3bb0 0x7fe7040a6070 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.714+0000 7fe71d5f2640 1 --2- 172.21.15.52:0/2386396612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7181036d0 0x7fe71819f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.714+0000 7fe71d5f2640 1 --2- 172.21.15.52:0/2386396612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe718102d10 0x7fe7181982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.714+0000 7fe71d5f2640 1 --2- 172.21.15.52:0/2386396612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe718101b20 0x7fe718197db0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.714+0000 7fe71d5f2640 1 -- 172.21.15.52:0/2386396612 >> 172.21.15.52:0/2386396612 conn(0x7fe7180fd090 msgr2=0x7fe718106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:16.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.714+0000 7fe71d5f2640 1 -- 172.21.15.52:0/2386396612 shutdown_connections 2024-04-02T13:51:16.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.714+0000 7fe71d5f2640 1 -- 172.21.15.52:0/2386396612 wait complete. 2024-04-02T13:51:16.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:16 smithi052 ceph-mon[32104]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 218 B/s wr, 0 op/s 2024-04-02T13:51:16.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:16 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3645220523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:16.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:16 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/340257665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:51:16.727 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:16 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/964991035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:51:16.729 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T13:51:16.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:16 smithi189 ceph-mon[36683]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 218 B/s wr, 0 op/s 2024-04-02T13:51:16.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:16 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3645220523' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:16.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:16 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/340257665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:51:16.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:16 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/964991035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:51:16.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.830+0000 7f3afc532640 1 -- 172.21.15.52:0/1030636938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3af4100a10 msgr2=0x7f3af4100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:16.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.830+0000 7f3afc532640 1 --2- 172.21.15.52:0/1030636938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3af4100a10 0x7f3af4100e30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3af0009900 tx=0x7f3af002bc30 comp rx=0 tx=0).stop 2024-04-02T13:51:16.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.830+0000 7f3afc532640 1 -- 172.21.15.52:0/1030636938 shutdown_connections 2024-04-02T13:51:16.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.830+0000 7f3afc532640 1 --2- 172.21.15.52:0/1030636938 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3af41025c0 0x7f3af4108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.830+0000 7f3afc532640 1 --2- 172.21.15.52:0/1030636938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3af4101c00 0x7f3af4102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.830+0000 7f3afc532640 1 --2- 172.21.15.52:0/1030636938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3af4100a10 0x7f3af4100e30 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.830+0000 7f3afc532640 1 -- 172.21.15.52:0/1030636938 >> 172.21.15.52:0/1030636938 conn(0x7f3af40fbfa0 msgr2=0x7f3af40fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:16.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.830+0000 7f3afc532640 1 -- 172.21.15.52:0/1030636938 shutdown_connections 2024-04-02T13:51:16.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.830+0000 7f3afc532640 1 -- 172.21.15.52:0/1030636938 wait complete. 2024-04-02T13:51:16.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.830+0000 7f3afc532640 1 Processor -- start 2024-04-02T13:51:16.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.830+0000 7f3afc532640 1 -- start start 2024-04-02T13:51:16.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.831+0000 7f3afc532640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3af4100a10 0x7f3af4197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:16.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.831+0000 7f3afc532640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3af4101c00 0x7f3af4198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:16.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.831+0000 7f3afc532640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3af41025c0 0x7f3af419f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:16.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.831+0000 7f3afc532640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3af410d770 con 0x7f3af4101c00 2024-04-02T13:51:16.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.831+0000 7f3afc532640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3af410d5f0 con 0x7f3af41025c0 2024-04-02T13:51:16.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.831+0000 7f3afc532640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3af410d8f0 con 0x7f3af4100a10 2024-04-02T13:51:16.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.831+0000 7f3af9acd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3af4101c00 0x7f3af4198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:16.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.831+0000 7f3af9acd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3af4101c00 0x7f3af4198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44490/0 (socket says 172.21.15.52:44490) 2024-04-02T13:51:16.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.831+0000 7f3af9acd640 1 -- 172.21.15.52:0/3872223511 learned_addr learned my addr 172.21.15.52:0/3872223511 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:16.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.831+0000 7f3afa2ce640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3af4100a10 0x7f3af4197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:16.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.831+0000 7f3afaacf640 1 --2- 172.21.15.52:0/3872223511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3af41025c0 0x7f3af419f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:16.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.831+0000 7f3af9acd640 1 -- 172.21.15.52:0/3872223511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3af4100a10 msgr2=0x7f3af4197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:16.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.831+0000 7f3af9acd640 1 --2- 172.21.15.52:0/3872223511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3af4100a10 0x7f3af4197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.831+0000 7f3af9acd640 1 -- 172.21.15.52:0/3872223511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3af41025c0 msgr2=0x7f3af419f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:16.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.831+0000 7f3af9acd640 1 --2- 172.21.15.52:0/3872223511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3af41025c0 0x7f3af419f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:16.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.831+0000 7f3af9acd640 1 -- 172.21.15.52:0/3872223511 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af419fa00 con 0x7f3af4101c00 2024-04-02T13:51:16.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.831+0000 7f3af9acd640 1 --2- 172.21.15.52:0/3872223511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3af4101c00 0x7f3af4198320 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400c920 tx=0x7f3ae400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:16.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.831+0000 7f3aeb7fe640 1 -- 172.21.15.52:0/3872223511 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4007c70 con 0x7f3af4101c00 2024-04-02T13:51:16.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.831+0000 7f3aeb7fe640 1 -- 172.21.15.52:0/3872223511 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae4007e10 con 0x7f3af4101c00 2024-04-02T13:51:16.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.831+0000 7f3afc532640 1 -- 172.21.15.52:0/3872223511 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af419fcf0 con 0x7f3af4101c00 2024-04-02T13:51:16.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.831+0000 7f3aeb7fe640 1 -- 172.21.15.52:0/3872223511 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4018890 con 0x7f3af4101c00 2024-04-02T13:51:16.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.832+0000 7f3afc532640 1 -- 172.21.15.52:0/3872223511 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af41a01e0 con 0x7f3af4101c00 2024-04-02T13:51:16.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.832+0000 7f3afc532640 1 -- 172.21.15.52:0/3872223511 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3af4107d80 con 0x7f3af4101c00 2024-04-02T13:51:16.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.834+0000 7f3aeb7fe640 1 -- 172.21.15.52:0/3872223511 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3ae4018a50 con 0x7f3af4101c00 2024-04-02T13:51:16.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.834+0000 7f3aeb7fe640 1 --2- 172.21.15.52:0/3872223511 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ae00a3bb0 0x7f3ae00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:16.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.834+0000 7f3afa2ce640 1 --2- 172.21.15.52:0/3872223511 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ae00a3bb0 0x7f3ae00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:16.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.834+0000 7f3aeb7fe640 1 -- 172.21.15.52:0/3872223511 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f3ae40d0950 con 0x7f3af4101c00 2024-04-02T13:51:16.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.835+0000 7f3afa2ce640 1 --2- 172.21.15.52:0/3872223511 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ae00a3bb0 0x7f3ae00a6070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3af0000c00 tx=0x7f3af0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:16.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:16.838+0000 7f3aeb7fe640 1 -- 172.21.15.52:0/3872223511 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3ae4014030 con 0x7f3af4101c00 2024-04-02T13:51:17.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:17.036+0000 7f3afc532640 1 -- 172.21.15.52:0/3872223511 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3af402d370 con 0x7f3af4101c00 2024-04-02T13:51:17.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:17.037+0000 7f3aeb7fe640 1 -- 172.21.15.52:0/3872223511 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+587 (secure 0 0 0) 0x7f3ae409d180 con 0x7f3af4101c00 2024-04-02T13:51:17.040 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2475099387 2024-04-03T13:43:54.653361+0000 2024-04-02T13:51:17.040 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/2368921152 2024-04-03T13:43:27.309404+0000 2024-04-02T13:51:17.040 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/673843171 2024-04-03T13:43:54.653361+0000 2024-04-02T13:51:17.040 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/2368921152 2024-04-03T13:43:27.309404+0000 2024-04-02T13:51:17.040 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2609300748 2024-04-03T13:43:27.309404+0000 2024-04-02T13:51:17.041 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/85571395 2024-04-03T13:43:27.309404+0000 2024-04-02T13:51:17.041 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/293852652 2024-04-03T13:43:54.653361+0000 2024-04-02T13:51:17.041 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/1829428070 2024-04-03T13:43:27.309404+0000 2024-04-02T13:51:17.041 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/673843171 2024-04-03T13:43:54.653361+0000 2024-04-02T13:51:17.041 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/3257395293 2024-04-03T13:43:54.653361+0000 2024-04-02T13:51:17.041 INFO:teuthology.orchestra.run.smithi052.stderr:listed 10 entries 2024-04-02T13:51:17.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:17.039+0000 7f3afc532640 1 -- 172.21.15.52:0/3872223511 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ae00a3bb0 msgr2=0x7f3ae00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:17.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:17.039+0000 7f3afc532640 1 --2- 172.21.15.52:0/3872223511 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ae00a3bb0 0x7f3ae00a6070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3af0000c00 tx=0x7f3af0005fd0 comp rx=0 tx=0).stop 2024-04-02T13:51:17.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:17.039+0000 7f3afc532640 1 -- 172.21.15.52:0/3872223511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3af4101c00 msgr2=0x7f3af4198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:17.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:17.039+0000 7f3afc532640 1 --2- 172.21.15.52:0/3872223511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3af4101c00 0x7f3af4198320 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400c920 tx=0x7f3ae400cde0 comp rx=0 tx=0).stop 2024-04-02T13:51:17.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:17.039+0000 7f3afc532640 1 -- 172.21.15.52:0/3872223511 shutdown_connections 2024-04-02T13:51:17.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:17.039+0000 7f3afc532640 1 --2- 172.21.15.52:0/3872223511 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ae00a3bb0 0x7f3ae00a6070 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:17.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:17.039+0000 7f3afc532640 1 --2- 172.21.15.52:0/3872223511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3af41025c0 0x7f3af419f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:17.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:17.039+0000 7f3afc532640 1 --2- 172.21.15.52:0/3872223511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3af4101c00 0x7f3af4198320 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:17.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:17.039+0000 7f3afc532640 1 --2- 172.21.15.52:0/3872223511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3af4100a10 0x7f3af4197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:17.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:17.039+0000 7f3afc532640 1 -- 172.21.15.52:0/3872223511 >> 172.21.15.52:0/3872223511 conn(0x7f3af40fbfa0 msgr2=0x7f3af4104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:17.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:17.039+0000 7f3afc532640 1 -- 172.21.15.52:0/3872223511 shutdown_connections 2024-04-02T13:51:17.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:17.039+0000 7f3afc532640 1 -- 172.21.15.52:0/3872223511 wait complete. 2024-04-02T13:51:17.053 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi052.front.sepia.ceph.com... 2024-04-02T13:51:17.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:51:17.053 DEBUG:teuthology.orchestra.run.smithi052:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-02T13:51:17.078 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi052.stderr:ceph-fuse[73495]: fuse finished with error 0 and tester_r 0 2024-04-02T13:51:17.080 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T13:51:17.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:17 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/51907382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:51:17.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:17 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2386396612' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:51:17.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:17 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3872223511' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:51:17.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:17 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/51907382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:51:17.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:17 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2386396612' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:51:17.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:17 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3872223511' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:51:17.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:17 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/51907382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:51:17.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:17 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2386396612' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:51:17.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:17 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3872223511' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:51:18.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:18 smithi088 ceph-mon[36528]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 295 B/s wr, 0 op/s 2024-04-02T13:51:18.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:18 smithi189 ceph-mon[36683]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 295 B/s wr, 0 op/s 2024-04-02T13:51:18.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:18 smithi052 ceph-mon[32104]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 295 B/s wr, 0 op/s 2024-04-02T13:51:20.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:20 smithi088 ceph-mon[36528]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-02T13:51:20.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:20 smithi189 ceph-mon[36683]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-02T13:51:20.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:20 smithi052 ceph-mon[32104]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-02T13:51:22.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:22 smithi088 ceph-mon[36528]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-02T13:51:22.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:22 smithi189 ceph-mon[36683]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-02T13:51:22.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:22 smithi052 ceph-mon[32104]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-02T13:51:23.182 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-04-02T13:51:23.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:51:23.182 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-02T13:51:23.206 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:51:23.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.357+0000 7f3170929640 1 -- 172.21.15.52:0/4004442956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f316c100cb0 msgr2=0x7f316c101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:23.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.357+0000 7f3170929640 1 --2- 172.21.15.52:0/4004442956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f316c100cb0 0x7f316c101130 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3160009900 tx=0x7f316002bc30 comp rx=0 tx=0).stop 2024-04-02T13:51:23.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.357+0000 7f3170929640 1 -- 172.21.15.52:0/4004442956 shutdown_connections 2024-04-02T13:51:23.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.357+0000 7f3170929640 1 --2- 172.21.15.52:0/4004442956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f316c101670 0x7f316c108f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:23.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.357+0000 7f3170929640 1 --2- 172.21.15.52:0/4004442956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f316c100cb0 0x7f316c101130 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:23.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.357+0000 7f3170929640 1 --2- 172.21.15.52:0/4004442956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f316c102060 0x7f316c102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:23.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.357+0000 7f3170929640 1 -- 172.21.15.52:0/4004442956 >> 172.21.15.52:0/4004442956 conn(0x7f316c0fb740 msgr2=0x7f316c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:23.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.357+0000 7f3170929640 1 -- 172.21.15.52:0/4004442956 shutdown_connections 2024-04-02T13:51:23.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.357+0000 7f3170929640 1 -- 172.21.15.52:0/4004442956 wait complete. 2024-04-02T13:51:23.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.358+0000 7f3170929640 1 Processor -- start 2024-04-02T13:51:23.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.358+0000 7f3170929640 1 -- start start 2024-04-02T13:51:23.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.358+0000 7f3170929640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f316c100cb0 0x7f316c196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:23.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.358+0000 7f3170929640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f316c101670 0x7f316c1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:23.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.358+0000 7f3170929640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f316c102060 0x7f316c19e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:23.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.358+0000 7f3170929640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f316c10c6f0 con 0x7f316c100cb0 2024-04-02T13:51:23.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.358+0000 7f3170929640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f316c10c570 con 0x7f316c101670 2024-04-02T13:51:23.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.358+0000 7f3170929640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f316c10c870 con 0x7f316c102060 2024-04-02T13:51:23.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.358+0000 7f316a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f316c100cb0 0x7f316c196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:23.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.358+0000 7f316a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f316c100cb0 0x7f316c196d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53382/0 (socket says 172.21.15.52:53382) 2024-04-02T13:51:23.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.358+0000 7f316a59c640 1 -- 172.21.15.52:0/3582760199 learned_addr learned my addr 172.21.15.52:0/3582760199 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:23.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.358+0000 7f316ad9d640 1 --2- 172.21.15.52:0/3582760199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f316c102060 0x7f316c19e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:23.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.358+0000 7f3169d9b640 1 --2- 172.21.15.52:0/3582760199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f316c101670 0x7f316c1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:23.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.359+0000 7f316a59c640 1 -- 172.21.15.52:0/3582760199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f316c102060 msgr2=0x7f316c19e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:23.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.359+0000 7f316a59c640 1 --2- 172.21.15.52:0/3582760199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f316c102060 0x7f316c19e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:23.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.359+0000 7f316a59c640 1 -- 172.21.15.52:0/3582760199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f316c101670 msgr2=0x7f316c1972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:23.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.359+0000 7f316a59c640 1 --2- 172.21.15.52:0/3582760199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f316c101670 0x7f316c1972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:23.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.359+0000 7f316a59c640 1 -- 172.21.15.52:0/3582760199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f316c19e980 con 0x7f316c100cb0 2024-04-02T13:51:23.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.359+0000 7f316a59c640 1 --2- 172.21.15.52:0/3582760199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f316c100cb0 0x7f316c196d60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f316400e9c0 tx=0x7f316400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:23.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.359+0000 7f315b7fe640 1 -- 172.21.15.52:0/3582760199 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316400cd80 con 0x7f316c100cb0 2024-04-02T13:51:23.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.359+0000 7f315b7fe640 1 -- 172.21.15.52:0/3582760199 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31640035f0 con 0x7f316c100cb0 2024-04-02T13:51:23.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.359+0000 7f3170929640 1 -- 172.21.15.52:0/3582760199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f316c19ec70 con 0x7f316c100cb0 2024-04-02T13:51:23.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.359+0000 7f315b7fe640 1 -- 172.21.15.52:0/3582760199 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164010690 con 0x7f316c100cb0 2024-04-02T13:51:23.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.359+0000 7f3170929640 1 -- 172.21.15.52:0/3582760199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f316c0ffba0 con 0x7f316c100cb0 2024-04-02T13:51:23.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.360+0000 7f3170929640 1 -- 172.21.15.52:0/3582760199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f316c100620 con 0x7f316c100cb0 2024-04-02T13:51:23.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.361+0000 7f315b7fe640 1 -- 172.21.15.52:0/3582760199 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f31640037a0 con 0x7f316c100cb0 2024-04-02T13:51:23.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.362+0000 7f315b7fe640 1 --2- 172.21.15.52:0/3582760199 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f31540a5dc0 0x7f31540a8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:23.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.362+0000 7f3169d9b640 1 --2- 172.21.15.52:0/3582760199 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f31540a5dc0 0x7f31540a8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:23.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.362+0000 7f315b7fe640 1 -- 172.21.15.52:0/3582760199 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f31640d0b40 con 0x7f316c100cb0 2024-04-02T13:51:23.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.362+0000 7f3169d9b640 1 --2- 172.21.15.52:0/3582760199 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f31540a5dc0 0x7f31540a8280 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f316002efd0 tx=0x7f3160000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:23.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.366+0000 7f315b7fe640 1 -- 172.21.15.52:0/3582760199 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f316409d310 con 0x7f316c100cb0 2024-04-02T13:51:23.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.554+0000 7f3170929640 1 -- 172.21.15.52:0/3582760199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f316c100840 con 0x7f316c100cb0 2024-04-02T13:51:23.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.555+0000 7f315b7fe640 1 -- 172.21.15.52:0/3582760199 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24077 (secure 0 0 0) 0x7f316409cc10 con 0x7f316c100cb0 2024-04-02T13:51:23.559 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:51:23.559 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":90,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T13:51:05.722301+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:50:58.778690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:50:59.561527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:51:03.343487+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:51:23.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.558+0000 7f3170929640 1 -- 172.21.15.52:0/3582760199 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f31540a5dc0 msgr2=0x7f31540a8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:23.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.558+0000 7f3170929640 1 --2- 172.21.15.52:0/3582760199 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f31540a5dc0 0x7f31540a8280 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f316002efd0 tx=0x7f3160000bc0 comp rx=0 tx=0).stop 2024-04-02T13:51:23.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.558+0000 7f3170929640 1 -- 172.21.15.52:0/3582760199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f316c100cb0 msgr2=0x7f316c196d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:23.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.558+0000 7f3170929640 1 --2- 172.21.15.52:0/3582760199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f316c100cb0 0x7f316c196d60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f316400e9c0 tx=0x7f316400ee80 comp rx=0 tx=0).stop 2024-04-02T13:51:23.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.558+0000 7f3170929640 1 -- 172.21.15.52:0/3582760199 shutdown_connections 2024-04-02T13:51:23.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.558+0000 7f3170929640 1 --2- 172.21.15.52:0/3582760199 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f31540a5dc0 0x7f31540a8280 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:23.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.558+0000 7f3170929640 1 --2- 172.21.15.52:0/3582760199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f316c102060 0x7f316c19e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:23.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.558+0000 7f3170929640 1 --2- 172.21.15.52:0/3582760199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f316c101670 0x7f316c1972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:23.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.558+0000 7f3170929640 1 --2- 172.21.15.52:0/3582760199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f316c100cb0 0x7f316c196d60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:23.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.558+0000 7f3170929640 1 -- 172.21.15.52:0/3582760199 >> 172.21.15.52:0/3582760199 conn(0x7f316c0fb740 msgr2=0x7f316c104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:23.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.558+0000 7f3170929640 1 -- 172.21.15.52:0/3582760199 shutdown_connections 2024-04-02T13:51:23.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.558+0000 7f3170929640 1 -- 172.21.15.52:0/3582760199 wait complete. 2024-04-02T13:51:23.572 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:51:23.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.672+0000 7f7ba7642640 1 -- 172.21.15.52:0/905611442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ba0102d40 msgr2=0x7f7ba01031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:23.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.672+0000 7f7ba7642640 1 --2- 172.21.15.52:0/905611442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ba0102d40 0x7f7ba01031c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009900 tx=0x7f7b9402bc30 comp rx=0 tx=0).stop 2024-04-02T13:51:23.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.672+0000 7f7ba7642640 1 -- 172.21.15.52:0/905611442 shutdown_connections 2024-04-02T13:51:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.672+0000 7f7ba7642640 1 --2- 172.21.15.52:0/905611442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ba0103700 0x7f7ba0109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.672+0000 7f7ba7642640 1 --2- 172.21.15.52:0/905611442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ba0102d40 0x7f7ba01031c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.672+0000 7f7ba7642640 1 --2- 172.21.15.52:0/905611442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ba0101b50 0x7f7ba0101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.672+0000 7f7ba7642640 1 -- 172.21.15.52:0/905611442 >> 172.21.15.52:0/905611442 conn(0x7f7ba00fd0a0 msgr2=0x7f7ba00ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.672+0000 7f7ba7642640 1 -- 172.21.15.52:0/905611442 shutdown_connections 2024-04-02T13:51:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.672+0000 7f7ba7642640 1 -- 172.21.15.52:0/905611442 wait complete. 2024-04-02T13:51:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.672+0000 7f7ba7642640 1 Processor -- start 2024-04-02T13:51:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.672+0000 7f7ba7642640 1 -- start start 2024-04-02T13:51:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.672+0000 7f7ba7642640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ba0101b50 0x7f7ba0197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.672+0000 7f7ba7642640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ba0102d40 0x7f7ba01982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.672+0000 7f7ba7642640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ba0103700 0x7f7ba019f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.672+0000 7f7ba7642640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba010d7b0 con 0x7f7ba0102d40 2024-04-02T13:51:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.673+0000 7f7ba7642640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba010d630 con 0x7f7ba0101b50 2024-04-02T13:51:23.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.673+0000 7f7ba7642640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba010d930 con 0x7f7ba0103700 2024-04-02T13:51:23.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.673+0000 7f7ba4bdd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ba0102d40 0x7f7ba01982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:23.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.673+0000 7f7ba4bdd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ba0102d40 0x7f7ba01982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53402/0 (socket says 172.21.15.52:53402) 2024-04-02T13:51:23.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.673+0000 7f7ba4bdd640 1 -- 172.21.15.52:0/272046221 learned_addr learned my addr 172.21.15.52:0/272046221 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:23.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.673+0000 7f7ba5bdf640 1 --2- 172.21.15.52:0/272046221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ba0103700 0x7f7ba019f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:23.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.673+0000 7f7ba4bdd640 1 -- 172.21.15.52:0/272046221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ba0103700 msgr2=0x7f7ba019f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:23.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.673+0000 7f7ba4bdd640 1 --2- 172.21.15.52:0/272046221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ba0103700 0x7f7ba019f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:23.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.673+0000 7f7ba4bdd640 1 -- 172.21.15.52:0/272046221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ba0101b50 msgr2=0x7f7ba0197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:23.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.673+0000 7f7ba53de640 1 --2- 172.21.15.52:0/272046221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ba0101b50 0x7f7ba0197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:23.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.673+0000 7f7ba4bdd640 1 --2- 172.21.15.52:0/272046221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ba0101b50 0x7f7ba0197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:23.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.673+0000 7f7ba4bdd640 1 -- 172.21.15.52:0/272046221 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba019f980 con 0x7f7ba0102d40 2024-04-02T13:51:23.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.673+0000 7f7ba5bdf640 1 --2- 172.21.15.52:0/272046221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ba0103700 0x7f7ba019f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:51:23.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.673+0000 7f7ba4bdd640 1 --2- 172.21.15.52:0/272046221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ba0102d40 0x7f7ba01982a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009230 tx=0x7f7b94004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:23.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.673+0000 7f7b9e7fc640 1 -- 172.21.15.52:0/272046221 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b94041070 con 0x7f7ba0102d40 2024-04-02T13:51:23.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.673+0000 7f7b9e7fc640 1 -- 172.21.15.52:0/272046221 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b940054e0 con 0x7f7ba0102d40 2024-04-02T13:51:23.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.673+0000 7f7ba7642640 1 -- 172.21.15.52:0/272046221 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba019fc10 con 0x7f7ba0102d40 2024-04-02T13:51:23.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.673+0000 7f7b9e7fc640 1 -- 172.21.15.52:0/272046221 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b940346a0 con 0x7f7ba0102d40 2024-04-02T13:51:23.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.673+0000 7f7ba7642640 1 -- 172.21.15.52:0/272046221 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba01a0150 con 0x7f7ba0102d40 2024-04-02T13:51:23.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.674+0000 7f7ba7642640 1 -- 172.21.15.52:0/272046221 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ba0108da0 con 0x7f7ba0102d40 2024-04-02T13:51:23.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.675+0000 7f7b9e7fc640 1 -- 172.21.15.52:0/272046221 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7b94044070 con 0x7f7ba0102d40 2024-04-02T13:51:23.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.676+0000 7f7b9e7fc640 1 --2- 172.21.15.52:0/272046221 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f7b880a3c30 0x7f7b880a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:23.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.676+0000 7f7ba53de640 1 --2- 172.21.15.52:0/272046221 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f7b880a3c30 0x7f7b880a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:23.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.676+0000 7f7b9e7fc640 1 -- 172.21.15.52:0/272046221 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f7b940f2750 con 0x7f7ba0102d40 2024-04-02T13:51:23.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.676+0000 7f7ba53de640 1 --2- 172.21.15.52:0/272046221 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f7b880a3c30 0x7f7b880a60f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7b98005fd0 tx=0x7f7b98005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:23.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.680+0000 7f7b9e7fc640 1 -- 172.21.15.52:0/272046221 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7b940bee90 con 0x7f7ba0102d40 2024-04-02T13:51:23.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.869+0000 7f7ba7642640 1 -- 172.21.15.52:0/272046221 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ba002d370 con 0x7f7ba0102d40 2024-04-02T13:51:23.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.870+0000 7f7b9e7fc640 1 -- 172.21.15.52:0/272046221 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24077 (secure 0 0 0) 0x7f7b940be790 con 0x7f7ba0102d40 2024-04-02T13:51:23.874 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:51:23.874 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":90,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T13:51:05.722301+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:50:58.778690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:50:59.561527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:51:03.343487+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:51:23.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.873+0000 7f7ba7642640 1 -- 172.21.15.52:0/272046221 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f7b880a3c30 msgr2=0x7f7b880a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:23.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.873+0000 7f7ba7642640 1 --2- 172.21.15.52:0/272046221 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f7b880a3c30 0x7f7b880a60f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7b98005fd0 tx=0x7f7b98005ce0 comp rx=0 tx=0).stop 2024-04-02T13:51:23.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.873+0000 7f7ba7642640 1 -- 172.21.15.52:0/272046221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ba0102d40 msgr2=0x7f7ba01982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:23.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.873+0000 7f7ba7642640 1 --2- 172.21.15.52:0/272046221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ba0102d40 0x7f7ba01982a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009230 tx=0x7f7b94004040 comp rx=0 tx=0).stop 2024-04-02T13:51:23.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.873+0000 7f7ba7642640 1 -- 172.21.15.52:0/272046221 shutdown_connections 2024-04-02T13:51:23.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.873+0000 7f7ba7642640 1 --2- 172.21.15.52:0/272046221 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f7b880a3c30 0x7f7b880a60f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:23.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.873+0000 7f7ba7642640 1 --2- 172.21.15.52:0/272046221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ba0103700 0x7f7ba019f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:23.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.873+0000 7f7ba7642640 1 --2- 172.21.15.52:0/272046221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ba0102d40 0x7f7ba01982a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:23.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.873+0000 7f7ba7642640 1 --2- 172.21.15.52:0/272046221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ba0101b50 0x7f7ba0197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:23.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.873+0000 7f7ba7642640 1 -- 172.21.15.52:0/272046221 >> 172.21.15.52:0/272046221 conn(0x7f7ba00fd0a0 msgr2=0x7f7ba0105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:23.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.873+0000 7f7ba7642640 1 -- 172.21.15.52:0/272046221 shutdown_connections 2024-04-02T13:51:23.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.873+0000 7f7ba7642640 1 -- 172.21.15.52:0/272046221 wait complete. 2024-04-02T13:51:23.887 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-02T13:51:23.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.990+0000 7fe79ccb2640 1 -- 172.21.15.52:0/2532232529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe798100a00 msgr2=0x7fe798100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:23.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.990+0000 7fe79ccb2640 1 --2- 172.21.15.52:0/2532232529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe798100a00 0x7fe798100e20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe788009900 tx=0x7fe78802bc30 comp rx=0 tx=0).stop 2024-04-02T13:51:23.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.990+0000 7fe79ccb2640 1 -- 172.21.15.52:0/2532232529 shutdown_connections 2024-04-02T13:51:23.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.990+0000 7fe79ccb2640 1 --2- 172.21.15.52:0/2532232529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7981025b0 0x7fe798108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:23.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.990+0000 7fe79ccb2640 1 --2- 172.21.15.52:0/2532232529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe798101bf0 0x7fe798102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:23.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.990+0000 7fe79ccb2640 1 --2- 172.21.15.52:0/2532232529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe798100a00 0x7fe798100e20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:23.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.990+0000 7fe79ccb2640 1 -- 172.21.15.52:0/2532232529 >> 172.21.15.52:0/2532232529 conn(0x7fe7980fbf70 msgr2=0x7fe7980fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:23.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.990+0000 7fe79ccb2640 1 -- 172.21.15.52:0/2532232529 shutdown_connections 2024-04-02T13:51:23.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.990+0000 7fe79ccb2640 1 -- 172.21.15.52:0/2532232529 wait complete. 2024-04-02T13:51:23.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.990+0000 7fe79ccb2640 1 Processor -- start 2024-04-02T13:51:23.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.991+0000 7fe79ccb2640 1 -- start start 2024-04-02T13:51:23.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.991+0000 7fe79ccb2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe798100a00 0x7fe7980697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:23.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.991+0000 7fe79ccb2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe798101bf0 0x7fe79819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:23.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.991+0000 7fe79ccb2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7981025b0 0x7fe79819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:23.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.991+0000 7fe79ccb2640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe79810c680 con 0x7fe7981025b0 2024-04-02T13:51:23.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.991+0000 7fe79ccb2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe79810c500 con 0x7fe798101bf0 2024-04-02T13:51:23.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.991+0000 7fe79ccb2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe79810c800 con 0x7fe798100a00 2024-04-02T13:51:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.991+0000 7fe796d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7981025b0 0x7fe79819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.991+0000 7fe796d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7981025b0 0x7fe79819f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53420/0 (socket says 172.21.15.52:53420) 2024-04-02T13:51:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.991+0000 7fe796d9d640 1 -- 172.21.15.52:0/3036878804 learned_addr learned my addr 172.21.15.52:0/3036878804 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.991+0000 7fe78dd9b640 1 --2- 172.21.15.52:0/3036878804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe798101bf0 0x7fe79819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.991+0000 7fe79659c640 1 --2- 172.21.15.52:0/3036878804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe798100a00 0x7fe7980697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.991+0000 7fe796d9d640 1 -- 172.21.15.52:0/3036878804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe798100a00 msgr2=0x7fe7980697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.991+0000 7fe796d9d640 1 --2- 172.21.15.52:0/3036878804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe798100a00 0x7fe7980697d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.991+0000 7fe796d9d640 1 -- 172.21.15.52:0/3036878804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe798101bf0 msgr2=0x7fe79819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.991+0000 7fe796d9d640 1 --2- 172.21.15.52:0/3036878804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe798101bf0 0x7fe79819cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.991+0000 7fe796d9d640 1 -- 172.21.15.52:0/3036878804 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79819fab0 con 0x7fe7981025b0 2024-04-02T13:51:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.991+0000 7fe796d9d640 1 --2- 172.21.15.52:0/3036878804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7981025b0 0x7fe79819f1a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe79000cad0 tx=0x7fe79000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.992+0000 7fe78ffff640 1 -- 172.21.15.52:0/3036878804 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79000ad60 con 0x7fe7981025b0 2024-04-02T13:51:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.992+0000 7fe79ccb2640 1 -- 172.21.15.52:0/3036878804 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79819fd40 con 0x7fe7981025b0 2024-04-02T13:51:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.992+0000 7fe78ffff640 1 -- 172.21.15.52:0/3036878804 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe790005020 con 0x7fe7981025b0 2024-04-02T13:51:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.992+0000 7fe78ffff640 1 -- 172.21.15.52:0/3036878804 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79000f6b0 con 0x7fe7981025b0 2024-04-02T13:51:23.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.992+0000 7fe79ccb2640 1 -- 172.21.15.52:0/3036878804 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7981a0280 con 0x7fe7981025b0 2024-04-02T13:51:23.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.993+0000 7fe79ccb2640 1 -- 172.21.15.52:0/3036878804 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe798107cc0 con 0x7fe7981025b0 2024-04-02T13:51:23.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.994+0000 7fe78ffff640 1 -- 172.21.15.52:0/3036878804 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe790005560 con 0x7fe7981025b0 2024-04-02T13:51:23.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.994+0000 7fe78ffff640 1 --2- 172.21.15.52:0/3036878804 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe7840a3bb0 0x7fe7840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:23.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.994+0000 7fe78ffff640 1 -- 172.21.15.52:0/3036878804 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe7900d0640 con 0x7fe7981025b0 2024-04-02T13:51:23.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.994+0000 7fe79659c640 1 --2- 172.21.15.52:0/3036878804 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe7840a3bb0 0x7fe7840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:23.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.995+0000 7fe79659c640 1 --2- 172.21.15.52:0/3036878804 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe7840a3bb0 0x7fe7840a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe78802c710 tx=0x7fe788005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:24.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:23.999+0000 7fe78ffff640 1 -- 172.21.15.52:0/3036878804 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe790014030 con 0x7fe7981025b0 2024-04-02T13:51:24.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.225+0000 7fe79ccb2640 1 -- 172.21.15.52:0/3036878804 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe7980627a0 con 0x7fe7981025b0 2024-04-02T13:51:24.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.399+0000 7fe78ffff640 1 -- 172.21.15.52:0/3036878804 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7fe79009cde0 con 0x7fe7981025b0 2024-04-02T13:51:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.401+0000 7fe79ccb2640 1 -- 172.21.15.52:0/3036878804 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe7840a3bb0 msgr2=0x7fe7840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.401+0000 7fe79ccb2640 1 --2- 172.21.15.52:0/3036878804 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe7840a3bb0 0x7fe7840a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe78802c710 tx=0x7fe788005fd0 comp rx=0 tx=0).stop 2024-04-02T13:51:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.401+0000 7fe79ccb2640 1 -- 172.21.15.52:0/3036878804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7981025b0 msgr2=0x7fe79819f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.401+0000 7fe79ccb2640 1 --2- 172.21.15.52:0/3036878804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7981025b0 0x7fe79819f1a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe79000cad0 tx=0x7fe79000ced0 comp rx=0 tx=0).stop 2024-04-02T13:51:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.401+0000 7fe79ccb2640 1 -- 172.21.15.52:0/3036878804 shutdown_connections 2024-04-02T13:51:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.401+0000 7fe79ccb2640 1 --2- 172.21.15.52:0/3036878804 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe7840a3bb0 0x7fe7840a6070 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.401+0000 7fe79ccb2640 1 --2- 172.21.15.52:0/3036878804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7981025b0 0x7fe79819f1a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.401+0000 7fe79ccb2640 1 --2- 172.21.15.52:0/3036878804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe798101bf0 0x7fe79819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.401+0000 7fe79ccb2640 1 --2- 172.21.15.52:0/3036878804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe798100a00 0x7fe7980697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.401+0000 7fe79ccb2640 1 -- 172.21.15.52:0/3036878804 >> 172.21.15.52:0/3036878804 conn(0x7fe7980fbf70 msgr2=0x7fe798104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.402+0000 7fe79ccb2640 1 -- 172.21.15.52:0/3036878804 shutdown_connections 2024-04-02T13:51:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.402+0000 7fe79ccb2640 1 -- 172.21.15.52:0/3036878804 wait complete. 2024-04-02T13:51:24.416 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-02T13:51:24.416 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-02T13:51:24.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.519+0000 7ffba0d7c640 1 -- 172.21.15.52:0/360128958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb9c100550 msgr2=0x7ffb9c10a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:24.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.519+0000 7ffba0d7c640 1 --2- 172.21.15.52:0/360128958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb9c100550 0x7ffb9c10a100 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ffb94009ca0 tx=0x7ffb9402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:51:24.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.519+0000 7ffba0d7c640 1 -- 172.21.15.52:0/360128958 shutdown_connections 2024-04-02T13:51:24.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.519+0000 7ffba0d7c640 1 --2- 172.21.15.52:0/360128958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb9c100550 0x7ffb9c10a100 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:24.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.519+0000 7ffba0d7c640 1 --2- 172.21.15.52:0/360128958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb9c0ffb90 0x7ffb9c100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:24.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.519+0000 7ffba0d7c640 1 --2- 172.21.15.52:0/360128958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb9c103120 0x7ffb9c103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:24.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.519+0000 7ffba0d7c640 1 -- 172.21.15.52:0/360128958 >> 172.21.15.52:0/360128958 conn(0x7ffb9c0fb710 msgr2=0x7ffb9c0fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:24.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.519+0000 7ffba0d7c640 1 -- 172.21.15.52:0/360128958 shutdown_connections 2024-04-02T13:51:24.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.519+0000 7ffba0d7c640 1 -- 172.21.15.52:0/360128958 wait complete. 2024-04-02T13:51:24.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.520+0000 7ffba0d7c640 1 Processor -- start 2024-04-02T13:51:24.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.520+0000 7ffba0d7c640 1 -- start start 2024-04-02T13:51:24.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.520+0000 7ffba0d7c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb9c0ffb90 0x7ffb9c197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:24.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.520+0000 7ffba0d7c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb9c100550 0x7ffb9c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:24.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.520+0000 7ffba0d7c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb9c103120 0x7ffb9c19f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:24.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.520+0000 7ffba0d7c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c10d7b0 con 0x7ffb9c100550 2024-04-02T13:51:24.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.520+0000 7ffba0d7c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c10d630 con 0x7ffb9c0ffb90 2024-04-02T13:51:24.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.520+0000 7ffba0d7c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c10d930 con 0x7ffb9c103120 2024-04-02T13:51:24.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.520+0000 7ffb93fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb9c100550 0x7ffb9c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:24.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.520+0000 7ffb93fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb9c100550 0x7ffb9c198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53446/0 (socket says 172.21.15.52:53446) 2024-04-02T13:51:24.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.520+0000 7ffb93fff640 1 -- 172.21.15.52:0/3260916125 learned_addr learned my addr 172.21.15.52:0/3260916125 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:24.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.520+0000 7ffb9a59c640 1 --2- 172.21.15.52:0/3260916125 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb9c0ffb90 0x7ffb9c197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:24.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.520+0000 7ffb9ad9d640 1 --2- 172.21.15.52:0/3260916125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb9c103120 0x7ffb9c19f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:24.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.520+0000 7ffb93fff640 1 -- 172.21.15.52:0/3260916125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb9c103120 msgr2=0x7ffb9c19f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:24.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.520+0000 7ffb93fff640 1 --2- 172.21.15.52:0/3260916125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb9c103120 0x7ffb9c19f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:24.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.520+0000 7ffb93fff640 1 -- 172.21.15.52:0/3260916125 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb9c0ffb90 msgr2=0x7ffb9c197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:24.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.520+0000 7ffb93fff640 1 --2- 172.21.15.52:0/3260916125 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb9c0ffb90 0x7ffb9c197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:24.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.520+0000 7ffb93fff640 1 -- 172.21.15.52:0/3260916125 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb9c19f9e0 con 0x7ffb9c100550 2024-04-02T13:51:24.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.521+0000 7ffb93fff640 1 --2- 172.21.15.52:0/3260916125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb9c100550 0x7ffb9c198300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400d770 tx=0x7ffb8400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:24.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.521+0000 7ffb937fe640 1 -- 172.21.15.52:0/3260916125 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb84017070 con 0x7ffb9c100550 2024-04-02T13:51:24.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.521+0000 7ffb937fe640 1 -- 172.21.15.52:0/3260916125 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb84005500 con 0x7ffb9c100550 2024-04-02T13:51:24.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.521+0000 7ffba0d7c640 1 -- 172.21.15.52:0/3260916125 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb9c19fcd0 con 0x7ffb9c100550 2024-04-02T13:51:24.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.521+0000 7ffb937fe640 1 -- 172.21.15.52:0/3260916125 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb84004e00 con 0x7ffb9c100550 2024-04-02T13:51:24.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.521+0000 7ffba0d7c640 1 -- 172.21.15.52:0/3260916125 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb9c1a01d0 con 0x7ffb9c100550 2024-04-02T13:51:24.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.522+0000 7ffba0d7c640 1 -- 172.21.15.52:0/3260916125 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb9c108f80 con 0x7ffb9c100550 2024-04-02T13:51:24.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.523+0000 7ffb937fe640 1 -- 172.21.15.52:0/3260916125 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ffb84004fa0 con 0x7ffb9c100550 2024-04-02T13:51:24.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.524+0000 7ffb937fe640 1 --2- 172.21.15.52:0/3260916125 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ffb880a3bb0 0x7ffb880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:24.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.524+0000 7ffb937fe640 1 -- 172.21.15.52:0/3260916125 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ffb840d0480 con 0x7ffb9c100550 2024-04-02T13:51:24.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.524+0000 7ffb9a59c640 1 --2- 172.21.15.52:0/3260916125 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ffb880a3bb0 0x7ffb880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:24.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.524+0000 7ffb9a59c640 1 --2- 172.21.15.52:0/3260916125 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ffb880a3bb0 0x7ffb880a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c069eb0 tx=0x7ffb8c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:24.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.527+0000 7ffb937fe640 1 -- 172.21.15.52:0/3260916125 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ffb8409cc20 con 0x7ffb9c100550 2024-04-02T13:51:24.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:24 smithi088 ceph-mon[36528]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-02T13:51:24.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:24 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3582760199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:24.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:24 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/272046221' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:24.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:24 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3036878804' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-02T13:51:24.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:24.756+0000 7ffba0d7c640 1 -- 172.21.15.52:0/3260916125 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7ffb9c02d500 con 0x7ffb9c100550 2024-04-02T13:51:24.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:24 smithi189 ceph-mon[36683]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-02T13:51:24.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:24 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3582760199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:24.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:24 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/272046221' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:24.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:24 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3036878804' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-02T13:51:24.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:24 smithi052 ceph-mon[32104]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-02T13:51:24.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:24 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3582760199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:24.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:24 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/272046221' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:24.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:24 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3036878804' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-02T13:51:25.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.319+0000 7ffb937fe640 1 -- 172.21.15.52:0/3260916125 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7ffb8409c520 con 0x7ffb9c100550 2024-04-02T13:51:25.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.323+0000 7ffba0d7c640 1 -- 172.21.15.52:0/3260916125 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ffb880a3bb0 msgr2=0x7ffb880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:25.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.323+0000 7ffba0d7c640 1 --2- 172.21.15.52:0/3260916125 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ffb880a3bb0 0x7ffb880a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c069eb0 tx=0x7ffb8c00b390 comp rx=0 tx=0).stop 2024-04-02T13:51:25.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.323+0000 7ffba0d7c640 1 -- 172.21.15.52:0/3260916125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb9c100550 msgr2=0x7ffb9c198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:25.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.323+0000 7ffba0d7c640 1 --2- 172.21.15.52:0/3260916125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb9c100550 0x7ffb9c198300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400d770 tx=0x7ffb8400dc30 comp rx=0 tx=0).stop 2024-04-02T13:51:25.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.323+0000 7ffba0d7c640 1 -- 172.21.15.52:0/3260916125 shutdown_connections 2024-04-02T13:51:25.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.323+0000 7ffba0d7c640 1 --2- 172.21.15.52:0/3260916125 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ffb880a3bb0 0x7ffb880a6070 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:25.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.323+0000 7ffba0d7c640 1 --2- 172.21.15.52:0/3260916125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb9c103120 0x7ffb9c19f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:25.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.323+0000 7ffba0d7c640 1 --2- 172.21.15.52:0/3260916125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb9c100550 0x7ffb9c198300 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:25.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.323+0000 7ffba0d7c640 1 --2- 172.21.15.52:0/3260916125 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb9c0ffb90 0x7ffb9c197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:25.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.323+0000 7ffba0d7c640 1 -- 172.21.15.52:0/3260916125 >> 172.21.15.52:0/3260916125 conn(0x7ffb9c0fb710 msgr2=0x7ffb9c106150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:25.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.323+0000 7ffba0d7c640 1 -- 172.21.15.52:0/3260916125 shutdown_connections 2024-04-02T13:51:25.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.323+0000 7ffba0d7c640 1 -- 172.21.15.52:0/3260916125 wait complete. 2024-04-02T13:51:25.341 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-02T13:51:25.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.449+0000 7fea0d2ba640 1 -- 172.21.15.52:0/3628319065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea080ffdc0 msgr2=0x7fea081001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:25.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.449+0000 7fea0d2ba640 1 --2- 172.21.15.52:0/3628319065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea080ffdc0 0x7fea081001e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fea00009930 tx=0x7fea0002bc20 comp rx=0 tx=0).stop 2024-04-02T13:51:25.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.449+0000 7fea0d2ba640 1 -- 172.21.15.52:0/3628319065 shutdown_connections 2024-04-02T13:51:25.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.449+0000 7fea0d2ba640 1 --2- 172.21.15.52:0/3628319065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea08103750 0x7fea08109fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:25.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.449+0000 7fea0d2ba640 1 --2- 172.21.15.52:0/3628319065 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea08102d90 0x7fea08103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:25.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.449+0000 7fea0d2ba640 1 --2- 172.21.15.52:0/3628319065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea080ffdc0 0x7fea081001e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:25.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.449+0000 7fea0d2ba640 1 -- 172.21.15.52:0/3628319065 >> 172.21.15.52:0/3628319065 conn(0x7fea080fb710 msgr2=0x7fea080fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:25.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.449+0000 7fea0d2ba640 1 -- 172.21.15.52:0/3628319065 shutdown_connections 2024-04-02T13:51:25.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.449+0000 7fea0d2ba640 1 -- 172.21.15.52:0/3628319065 wait complete. 2024-04-02T13:51:25.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.450+0000 7fea0d2ba640 1 Processor -- start 2024-04-02T13:51:25.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.450+0000 7fea0d2ba640 1 -- start start 2024-04-02T13:51:25.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.450+0000 7fea0d2ba640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea080ffdc0 0x7fea08102700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:25.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.450+0000 7fea0d2ba640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea08102d90 0x7fea08100d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:25.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.450+0000 7fea0d2ba640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea08103750 0x7fea08101280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:25.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.450+0000 7fea0d2ba640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fea0810d770 con 0x7fea080ffdc0 2024-04-02T13:51:25.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.450+0000 7fea0d2ba640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fea0810d5f0 con 0x7fea08102d90 2024-04-02T13:51:25.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.450+0000 7fea0d2ba640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fea0810d8f0 con 0x7fea08103750 2024-04-02T13:51:25.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.450+0000 7fea06ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea080ffdc0 0x7fea08102700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:25.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.450+0000 7fea06ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea080ffdc0 0x7fea08102700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53470/0 (socket says 172.21.15.52:53470) 2024-04-02T13:51:25.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.450+0000 7fea06ffd640 1 -- 172.21.15.52:0/1824320085 learned_addr learned my addr 172.21.15.52:0/1824320085 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:25.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.451+0000 7fea077fe640 1 --2- 172.21.15.52:0/1824320085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea08103750 0x7fea08101280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:25.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.451+0000 7fea06ffd640 1 -- 172.21.15.52:0/1824320085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea08103750 msgr2=0x7fea08101280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:25.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.451+0000 7fea067fc640 1 --2- 172.21.15.52:0/1824320085 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea08102d90 0x7fea08100d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:25.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.451+0000 7fea06ffd640 1 --2- 172.21.15.52:0/1824320085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea08103750 0x7fea08101280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:25.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.451+0000 7fea06ffd640 1 -- 172.21.15.52:0/1824320085 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea08102d90 msgr2=0x7fea08100d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:25.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.451+0000 7fea06ffd640 1 --2- 172.21.15.52:0/1824320085 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea08102d90 0x7fea08100d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:25.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.451+0000 7fea06ffd640 1 -- 172.21.15.52:0/1824320085 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea08101b10 con 0x7fea080ffdc0 2024-04-02T13:51:25.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.451+0000 7fea077fe640 1 --2- 172.21.15.52:0/1824320085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea08103750 0x7fea08101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:51:25.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.451+0000 7fea06ffd640 1 --2- 172.21.15.52:0/1824320085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea080ffdc0 0x7fea08102700 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fea00000c00 tx=0x7fea00004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:25.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.451+0000 7fe9fffff640 1 -- 172.21.15.52:0/1824320085 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00043020 con 0x7fea080ffdc0 2024-04-02T13:51:25.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.451+0000 7fea0d2ba640 1 -- 172.21.15.52:0/1824320085 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea081a1e50 con 0x7fea080ffdc0 2024-04-02T13:51:25.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.451+0000 7fe9fffff640 1 -- 172.21.15.52:0/1824320085 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea00004440 con 0x7fea080ffdc0 2024-04-02T13:51:25.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.451+0000 7fea0d2ba640 1 -- 172.21.15.52:0/1824320085 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea081a2260 con 0x7fea080ffdc0 2024-04-02T13:51:25.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.451+0000 7fe9fffff640 1 -- 172.21.15.52:0/1824320085 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00003e00 con 0x7fea080ffdc0 2024-04-02T13:51:25.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.452+0000 7fea0d2ba640 1 -- 172.21.15.52:0/1824320085 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea08108e60 con 0x7fea080ffdc0 2024-04-02T13:51:25.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.453+0000 7fe9fffff640 1 -- 172.21.15.52:0/1824320085 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fea00046020 con 0x7fea080ffdc0 2024-04-02T13:51:25.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.454+0000 7fe9fffff640 1 --2- 172.21.15.52:0/1824320085 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe9f40a3bb0 0x7fe9f40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:25.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.454+0000 7fea067fc640 1 --2- 172.21.15.52:0/1824320085 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe9f40a3bb0 0x7fe9f40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:25.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.454+0000 7fe9fffff640 1 -- 172.21.15.52:0/1824320085 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fea000f2640 con 0x7fea080ffdc0 2024-04-02T13:51:25.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.454+0000 7fea067fc640 1 --2- 172.21.15.52:0/1824320085 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe9f40a3bb0 0x7fe9f40a6070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fea08102470 tx=0x7fe9f0005cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:25.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.458+0000 7fe9fffff640 1 -- 172.21.15.52:0/1824320085 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fea000bed80 con 0x7fea080ffdc0 2024-04-02T13:51:25.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:25 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3036878804' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-02T13:51:25.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:25 smithi088 ceph-mon[36528]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:51:25.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:25 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3260916125' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-02T13:51:25.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:25 smithi088 ceph-mon[36528]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-02T13:51:25.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:25 smithi088 ceph-mon[36528]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-02T13:51:25.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:25 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3260916125' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-02T13:51:25.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:25 smithi088 ceph-mon[36528]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-02T13:51:25.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:25 smithi088 ceph-mon[36528]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-02T13:51:25.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:25.686+0000 7fea0d2ba640 1 -- 172.21.15.52:0/1824320085 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fea08061f40 con 0x7fea080ffdc0 2024-04-02T13:51:25.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:25 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3036878804' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-02T13:51:25.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:25 smithi189 ceph-mon[36683]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:51:25.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:25 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3260916125' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-02T13:51:25.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:25 smithi189 ceph-mon[36683]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-02T13:51:25.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:25 smithi189 ceph-mon[36683]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-02T13:51:25.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:25 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3260916125' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-02T13:51:25.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:25 smithi189 ceph-mon[36683]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-02T13:51:25.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:25 smithi189 ceph-mon[36683]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-02T13:51:25.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:25 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3036878804' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-02T13:51:25.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:25 smithi052 ceph-mon[32104]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:51:25.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:25 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3260916125' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-02T13:51:25.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:25 smithi052 ceph-mon[32104]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-02T13:51:25.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:25 smithi052 ceph-mon[32104]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-02T13:51:25.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:25 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3260916125' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-02T13:51:25.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:25 smithi052 ceph-mon[32104]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-02T13:51:25.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:25 smithi052 ceph-mon[32104]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-02T13:51:26.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.321+0000 7fe9fffff640 1 -- 172.21.15.52:0/1824320085 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7fea000be680 con 0x7fea080ffdc0 2024-04-02T13:51:26.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.324+0000 7fea0d2ba640 1 -- 172.21.15.52:0/1824320085 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe9f40a3bb0 msgr2=0x7fe9f40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:26.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.324+0000 7fea0d2ba640 1 --2- 172.21.15.52:0/1824320085 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe9f40a3bb0 0x7fe9f40a6070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fea08102470 tx=0x7fe9f0005cc0 comp rx=0 tx=0).stop 2024-04-02T13:51:26.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.324+0000 7fea0d2ba640 1 -- 172.21.15.52:0/1824320085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea080ffdc0 msgr2=0x7fea08102700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:26.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.324+0000 7fea0d2ba640 1 --2- 172.21.15.52:0/1824320085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea080ffdc0 0x7fea08102700 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fea00000c00 tx=0x7fea00004040 comp rx=0 tx=0).stop 2024-04-02T13:51:26.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.324+0000 7fea0d2ba640 1 -- 172.21.15.52:0/1824320085 shutdown_connections 2024-04-02T13:51:26.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.324+0000 7fea0d2ba640 1 --2- 172.21.15.52:0/1824320085 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe9f40a3bb0 0x7fe9f40a6070 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:26.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.324+0000 7fea0d2ba640 1 --2- 172.21.15.52:0/1824320085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea08103750 0x7fea08101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:26.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.324+0000 7fea0d2ba640 1 --2- 172.21.15.52:0/1824320085 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea08102d90 0x7fea08100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:26.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.324+0000 7fea0d2ba640 1 --2- 172.21.15.52:0/1824320085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea080ffdc0 0x7fea08102700 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:26.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.324+0000 7fea0d2ba640 1 -- 172.21.15.52:0/1824320085 >> 172.21.15.52:0/1824320085 conn(0x7fea080fb710 msgr2=0x7fea08106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:26.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.324+0000 7fea0d2ba640 1 -- 172.21.15.52:0/1824320085 shutdown_connections 2024-04-02T13:51:26.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.324+0000 7fea0d2ba640 1 -- 172.21.15.52:0/1824320085 wait complete. 2024-04-02T13:51:26.344 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-02T13:51:26.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.454+0000 7f3ef323c640 1 -- 172.21.15.52:0/1329379146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3eec100a00 msgr2=0x7f3eec100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:26.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.454+0000 7f3ef323c640 1 --2- 172.21.15.52:0/1329379146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3eec100a00 0x7f3eec100e20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009be0 tx=0x7f3ee402bef0 comp rx=0 tx=0).stop 2024-04-02T13:51:26.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.454+0000 7f3ef323c640 1 -- 172.21.15.52:0/1329379146 shutdown_connections 2024-04-02T13:51:26.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.454+0000 7f3ef323c640 1 --2- 172.21.15.52:0/1329379146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3eec1025b0 0x7f3eec108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:26.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.454+0000 7f3ef323c640 1 --2- 172.21.15.52:0/1329379146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eec101bf0 0x7f3eec102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:26.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.454+0000 7f3ef323c640 1 --2- 172.21.15.52:0/1329379146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3eec100a00 0x7f3eec100e20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:26.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.454+0000 7f3ef323c640 1 -- 172.21.15.52:0/1329379146 >> 172.21.15.52:0/1329379146 conn(0x7f3eec0fbf90 msgr2=0x7f3eec0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:26.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.454+0000 7f3ef323c640 1 -- 172.21.15.52:0/1329379146 shutdown_connections 2024-04-02T13:51:26.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.454+0000 7f3ef323c640 1 -- 172.21.15.52:0/1329379146 wait complete. 2024-04-02T13:51:26.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.454+0000 7f3ef323c640 1 Processor -- start 2024-04-02T13:51:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.454+0000 7f3ef323c640 1 -- start start 2024-04-02T13:51:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.455+0000 7f3ef323c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3eec100a00 0x7f3eec104990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.455+0000 7f3ef323c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eec101bf0 0x7f3eec102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.455+0000 7f3ef323c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3eec1025b0 0x7f3eec103510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.455+0000 7f3ef323c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec10c6a0 con 0x7f3eec101bf0 2024-04-02T13:51:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.455+0000 7f3ef323c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec10c520 con 0x7f3eec100a00 2024-04-02T13:51:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.455+0000 7f3ef323c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec10c820 con 0x7f3eec1025b0 2024-04-02T13:51:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.455+0000 7f3eebfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eec101bf0 0x7f3eec102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.455+0000 7f3eebfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eec101bf0 0x7f3eec102fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53480/0 (socket says 172.21.15.52:53480) 2024-04-02T13:51:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.455+0000 7f3eebfff640 1 -- 172.21.15.52:0/3052599642 learned_addr learned my addr 172.21.15.52:0/3052599642 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.455+0000 7f3ef0fd8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3eec100a00 0x7f3eec104990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.455+0000 7f3ef17d9640 1 --2- 172.21.15.52:0/3052599642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3eec1025b0 0x7f3eec103510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.455+0000 7f3eebfff640 1 -- 172.21.15.52:0/3052599642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3eec1025b0 msgr2=0x7f3eec103510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.455+0000 7f3eebfff640 1 --2- 172.21.15.52:0/3052599642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3eec1025b0 0x7f3eec103510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.455+0000 7f3eebfff640 1 -- 172.21.15.52:0/3052599642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3eec100a00 msgr2=0x7f3eec104990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:26.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.455+0000 7f3eebfff640 1 --2- 172.21.15.52:0/3052599642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3eec100a00 0x7f3eec104990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:26.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.455+0000 7f3eebfff640 1 -- 172.21.15.52:0/3052599642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eec103da0 con 0x7f3eec101bf0 2024-04-02T13:51:26.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.455+0000 7f3ef0fd8640 1 --2- 172.21.15.52:0/3052599642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3eec100a00 0x7f3eec104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:26.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.455+0000 7f3eebfff640 1 --2- 172.21.15.52:0/3052599642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eec101bf0 0x7f3eec102fd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000e9c0 tx=0x7f3ee000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:26.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.455+0000 7f3ee9ffb640 1 -- 172.21.15.52:0/3052599642 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee000cde0 con 0x7f3eec101bf0 2024-04-02T13:51:26.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.455+0000 7f3ee9ffb640 1 -- 172.21.15.52:0/3052599642 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ee00035f0 con 0x7f3eec101bf0 2024-04-02T13:51:26.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.455+0000 7f3ef323c640 1 -- 172.21.15.52:0/3052599642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eec1a1fe0 con 0x7f3eec101bf0 2024-04-02T13:51:26.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.455+0000 7f3ee9ffb640 1 -- 172.21.15.52:0/3052599642 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0010650 con 0x7f3eec101bf0 2024-04-02T13:51:26.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.456+0000 7f3ef323c640 1 -- 172.21.15.52:0/3052599642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eec1a2440 con 0x7f3eec101bf0 2024-04-02T13:51:26.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.457+0000 7f3ef323c640 1 -- 172.21.15.52:0/3052599642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eec1a2730 con 0x7f3eec101bf0 2024-04-02T13:51:26.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.458+0000 7f3ee9ffb640 1 -- 172.21.15.52:0/3052599642 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3ee00037a0 con 0x7f3eec101bf0 2024-04-02T13:51:26.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.458+0000 7f3ee9ffb640 1 --2- 172.21.15.52:0/3052599642 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ed40a3bb0 0x7f3ed40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:26.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.458+0000 7f3ee9ffb640 1 -- 172.21.15.52:0/3052599642 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f3ee00d08e0 con 0x7f3eec101bf0 2024-04-02T13:51:26.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.458+0000 7f3ef0fd8640 1 --2- 172.21.15.52:0/3052599642 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ed40a3bb0 0x7f3ed40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:26.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.459+0000 7f3ef0fd8640 1 --2- 172.21.15.52:0/3052599642 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ed40a3bb0 0x7f3ed40a6070 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4034fd0 tx=0x7f3ee4000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:26.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.463+0000 7f3ee9ffb640 1 -- 172.21.15.52:0/3052599642 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3ee009d080 con 0x7f3eec101bf0 2024-04-02T13:51:26.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:26 smithi088 ceph-mon[36528]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-02T13:51:26.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:26 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1824320085' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-02T13:51:26.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:26 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1824320085' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-02T13:51:26.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:26 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] up:active 2024-04-02T13:51:26.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:26 smithi088 ceph-mon[36528]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-02T13:51:26.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:26 smithi088 ceph-mon[36528]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-02T13:51:26.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:26 smithi088 ceph-mon[36528]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-02T13:51:26.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:26 smithi088 ceph-mon[36528]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-02T13:51:26.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:26.689+0000 7f3ef323c640 1 -- 172.21.15.52:0/3052599642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f3eec1a2900 con 0x7f3eec101bf0 2024-04-02T13:51:26.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:26 smithi189 ceph-mon[36683]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-02T13:51:26.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:26 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1824320085' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-02T13:51:26.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:26 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1824320085' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-02T13:51:26.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:26 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] up:active 2024-04-02T13:51:26.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:26 smithi189 ceph-mon[36683]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-02T13:51:26.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:26 smithi189 ceph-mon[36683]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-02T13:51:26.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:26 smithi189 ceph-mon[36683]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-02T13:51:26.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:26 smithi189 ceph-mon[36683]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-02T13:51:26.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:26 smithi052 ceph-mon[32104]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-02T13:51:26.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:26 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1824320085' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-02T13:51:26.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:26 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1824320085' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-02T13:51:26.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:26 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] up:active 2024-04-02T13:51:26.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:26 smithi052 ceph-mon[32104]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-02T13:51:26.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:26 smithi052 ceph-mon[32104]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-02T13:51:26.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:26 smithi052 ceph-mon[32104]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-02T13:51:26.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:26 smithi052 ceph-mon[32104]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-02T13:51:27.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.328+0000 7f3ee9ffb640 1 -- 172.21.15.52:0/3052599642 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7f3ee009c980 con 0x7f3eec101bf0 2024-04-02T13:51:27.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.331+0000 7f3ef323c640 1 -- 172.21.15.52:0/3052599642 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ed40a3bb0 msgr2=0x7f3ed40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:27.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.331+0000 7f3ef323c640 1 --2- 172.21.15.52:0/3052599642 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ed40a3bb0 0x7f3ed40a6070 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4034fd0 tx=0x7f3ee4000f40 comp rx=0 tx=0).stop 2024-04-02T13:51:27.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.331+0000 7f3ef323c640 1 -- 172.21.15.52:0/3052599642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eec101bf0 msgr2=0x7f3eec102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:27.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.331+0000 7f3ef323c640 1 --2- 172.21.15.52:0/3052599642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eec101bf0 0x7f3eec102fd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000e9c0 tx=0x7f3ee000ee80 comp rx=0 tx=0).stop 2024-04-02T13:51:27.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.331+0000 7f3ef323c640 1 -- 172.21.15.52:0/3052599642 shutdown_connections 2024-04-02T13:51:27.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.331+0000 7f3ef323c640 1 --2- 172.21.15.52:0/3052599642 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ed40a3bb0 0x7f3ed40a6070 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:27.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.331+0000 7f3ef323c640 1 --2- 172.21.15.52:0/3052599642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3eec1025b0 0x7f3eec103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:27.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.331+0000 7f3ef323c640 1 --2- 172.21.15.52:0/3052599642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3eec101bf0 0x7f3eec102fd0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:27.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.331+0000 7f3ef323c640 1 --2- 172.21.15.52:0/3052599642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3eec100a00 0x7f3eec104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:27.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.331+0000 7f3ef323c640 1 -- 172.21.15.52:0/3052599642 >> 172.21.15.52:0/3052599642 conn(0x7f3eec0fbf90 msgr2=0x7f3eec0fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:27.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.331+0000 7f3ef323c640 1 -- 172.21.15.52:0/3052599642 shutdown_connections 2024-04-02T13:51:27.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.331+0000 7f3ef323c640 1 -- 172.21.15.52:0/3052599642 wait complete. 2024-04-02T13:51:27.352 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-04-02T13:51:27.352 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-02T13:51:27.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.463+0000 7fae44f1c640 1 -- 172.21.15.52:0/3792861876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae40102d00 msgr2=0x7fae40103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:27.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.463+0000 7fae44f1c640 1 --2- 172.21.15.52:0/3792861876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae40102d00 0x7fae40103180 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fae28009900 tx=0x7fae2802bc30 comp rx=0 tx=0).stop 2024-04-02T13:51:27.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.463+0000 7fae44f1c640 1 -- 172.21.15.52:0/3792861876 shutdown_connections 2024-04-02T13:51:27.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.463+0000 7fae44f1c640 1 --2- 172.21.15.52:0/3792861876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae401036c0 0x7fae40109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:27.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.463+0000 7fae44f1c640 1 --2- 172.21.15.52:0/3792861876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae40102d00 0x7fae40103180 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:27.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.463+0000 7fae44f1c640 1 --2- 172.21.15.52:0/3792861876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae40101b10 0x7fae40101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:27.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.463+0000 7fae44f1c640 1 -- 172.21.15.52:0/3792861876 >> 172.21.15.52:0/3792861876 conn(0x7fae400fd0a0 msgr2=0x7fae400ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:27.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.463+0000 7fae44f1c640 1 -- 172.21.15.52:0/3792861876 shutdown_connections 2024-04-02T13:51:27.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.463+0000 7fae44f1c640 1 -- 172.21.15.52:0/3792861876 wait complete. 2024-04-02T13:51:27.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.464+0000 7fae44f1c640 1 Processor -- start 2024-04-02T13:51:27.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.464+0000 7fae44f1c640 1 -- start start 2024-04-02T13:51:27.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.464+0000 7fae44f1c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae40101b10 0x7fae400758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:27.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.464+0000 7fae44f1c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae40102d00 0x7fae40073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:27.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.464+0000 7fae44f1c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae401036c0 0x7fae40074430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:27.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.464+0000 7fae44f1c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fae4010d790 con 0x7fae40102d00 2024-04-02T13:51:27.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.464+0000 7fae44f1c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fae4010d610 con 0x7fae401036c0 2024-04-02T13:51:27.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.464+0000 7fae44f1c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae4010d910 con 0x7fae40101b10 2024-04-02T13:51:27.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.464+0000 7fae3dd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae40102d00 0x7fae40073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:27.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.464+0000 7fae3dd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae40102d00 0x7fae40073ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53506/0 (socket says 172.21.15.52:53506) 2024-04-02T13:51:27.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.464+0000 7fae3dd9b640 1 -- 172.21.15.52:0/1265774545 learned_addr learned my addr 172.21.15.52:0/1265774545 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:27.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.464+0000 7fae3e59c640 1 --2- 172.21.15.52:0/1265774545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae40101b10 0x7fae400758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:27.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.464+0000 7fae3ed9d640 1 --2- 172.21.15.52:0/1265774545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae401036c0 0x7fae40074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:27.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.465+0000 7fae3dd9b640 1 -- 172.21.15.52:0/1265774545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae40101b10 msgr2=0x7fae400758b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:27.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.465+0000 7fae3dd9b640 1 --2- 172.21.15.52:0/1265774545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae40101b10 0x7fae400758b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:27.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.465+0000 7fae3dd9b640 1 -- 172.21.15.52:0/1265774545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae401036c0 msgr2=0x7fae40074430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:27.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.465+0000 7fae3dd9b640 1 --2- 172.21.15.52:0/1265774545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae401036c0 0x7fae40074430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:27.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.465+0000 7fae3dd9b640 1 -- 172.21.15.52:0/1265774545 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae40074cc0 con 0x7fae40102d00 2024-04-02T13:51:27.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.465+0000 7fae3dd9b640 1 --2- 172.21.15.52:0/1265774545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae40102d00 0x7fae40073ef0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fae28009230 tx=0x7fae28005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:27.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.465+0000 7fae377fe640 1 -- 172.21.15.52:0/1265774545 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae28041070 con 0x7fae40102d00 2024-04-02T13:51:27.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.465+0000 7fae377fe640 1 -- 172.21.15.52:0/1265774545 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae280044d0 con 0x7fae40102d00 2024-04-02T13:51:27.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.465+0000 7fae44f1c640 1 -- 172.21.15.52:0/1265774545 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae401a1f00 con 0x7fae40102d00 2024-04-02T13:51:27.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.465+0000 7fae377fe640 1 -- 172.21.15.52:0/1265774545 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae28034440 con 0x7fae40102d00 2024-04-02T13:51:27.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.465+0000 7fae44f1c640 1 -- 172.21.15.52:0/1265774545 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae401a2360 con 0x7fae40102d00 2024-04-02T13:51:27.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.466+0000 7fae44f1c640 1 -- 172.21.15.52:0/1265774545 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae40108eb0 con 0x7fae40102d00 2024-04-02T13:51:27.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.467+0000 7fae377fe640 1 -- 172.21.15.52:0/1265774545 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fae28044070 con 0x7fae40102d00 2024-04-02T13:51:27.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.468+0000 7fae377fe640 1 --2- 172.21.15.52:0/1265774545 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fae2c0a3bb0 0x7fae2c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:27.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.468+0000 7fae3e59c640 1 --2- 172.21.15.52:0/1265774545 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fae2c0a3bb0 0x7fae2c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:27.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.468+0000 7fae377fe640 1 -- 172.21.15.52:0/1265774545 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fae280f2170 con 0x7fae40102d00 2024-04-02T13:51:27.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.468+0000 7fae3e59c640 1 --2- 172.21.15.52:0/1265774545 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fae2c0a3bb0 0x7fae2c0a6070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fae300099a0 tx=0x7fae30009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:27.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.472+0000 7fae377fe640 1 -- 172.21.15.52:0/1265774545 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fae280be2c0 con 0x7fae40102d00 2024-04-02T13:51:27.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:27 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3052599642' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-02T13:51:27.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:27 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3052599642' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-02T13:51:27.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:27 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] up:active 2024-04-02T13:51:27.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:27 smithi088 ceph-mon[36528]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-02T13:51:27.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:27 smithi088 ceph-mon[36528]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-02T13:51:27.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:27 smithi088 ceph-mon[36528]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-04-02T13:51:27.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:27 smithi088 ceph-mon[36528]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-04-02T13:51:27.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:27.683+0000 7fae44f1c640 1 -- 172.21.15.52:0/1265774545 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fae40061f40 con 0x7fae2c0a3bb0 2024-04-02T13:51:27.692 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:27 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3052599642' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-02T13:51:27.692 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:27 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3052599642' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-02T13:51:27.692 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:27 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] up:active 2024-04-02T13:51:27.692 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:27 smithi052 ceph-mon[32104]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-02T13:51:27.692 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:27 smithi052 ceph-mon[32104]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-02T13:51:27.692 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:27 smithi052 ceph-mon[32104]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-04-02T13:51:27.692 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:27 smithi052 ceph-mon[32104]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-04-02T13:51:27.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:27 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3052599642' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-02T13:51:27.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:27 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3052599642' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-02T13:51:27.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:27 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] up:active 2024-04-02T13:51:27.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:27 smithi189 ceph-mon[36683]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-02T13:51:27.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:27 smithi189 ceph-mon[36683]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-02T13:51:27.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:27 smithi189 ceph-mon[36683]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-04-02T13:51:27.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:27 smithi189 ceph-mon[36683]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-04-02T13:51:28.089 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:51:27 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:51:27.688+0000 7feaef0d6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:51:28.089 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:51:27 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:51:27.688+0000 7feaef0d6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:51:28.089 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:51:27 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:51:27.688+0000 7feaef0d6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:51:28.089 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:51:27 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:51:27.688+0000 7feaef0d6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:51:28.089 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:51:27 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:51:27.688+0000 7feaef0d6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:51:28.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.357+0000 7fae377fe640 1 -- 172.21.15.52:0/1265774545 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fae40061f40 con 0x7fae2c0a3bb0 2024-04-02T13:51:28.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.360+0000 7fae44f1c640 1 -- 172.21.15.52:0/1265774545 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fae2c0a3bb0 msgr2=0x7fae2c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:28.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.360+0000 7fae44f1c640 1 --2- 172.21.15.52:0/1265774545 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fae2c0a3bb0 0x7fae2c0a6070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fae300099a0 tx=0x7fae30009500 comp rx=0 tx=0).stop 2024-04-02T13:51:28.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.360+0000 7fae44f1c640 1 -- 172.21.15.52:0/1265774545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae40102d00 msgr2=0x7fae40073ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:28.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.360+0000 7fae44f1c640 1 --2- 172.21.15.52:0/1265774545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae40102d00 0x7fae40073ef0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fae28009230 tx=0x7fae28005d50 comp rx=0 tx=0).stop 2024-04-02T13:51:28.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.360+0000 7fae44f1c640 1 -- 172.21.15.52:0/1265774545 shutdown_connections 2024-04-02T13:51:28.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.360+0000 7fae44f1c640 1 --2- 172.21.15.52:0/1265774545 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fae2c0a3bb0 0x7fae2c0a6070 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:28.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.360+0000 7fae44f1c640 1 --2- 172.21.15.52:0/1265774545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae401036c0 0x7fae40074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:28.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.360+0000 7fae44f1c640 1 --2- 172.21.15.52:0/1265774545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae40102d00 0x7fae40073ef0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:28.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.360+0000 7fae44f1c640 1 --2- 172.21.15.52:0/1265774545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae40101b10 0x7fae400758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:28.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.360+0000 7fae44f1c640 1 -- 172.21.15.52:0/1265774545 >> 172.21.15.52:0/1265774545 conn(0x7fae400fd0a0 msgr2=0x7fae40106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:28.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.360+0000 7fae44f1c640 1 -- 172.21.15.52:0/1265774545 shutdown_connections 2024-04-02T13:51:28.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.360+0000 7fae44f1c640 1 -- 172.21.15.52:0/1265774545 wait complete. 2024-04-02T13:51:28.377 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-02T13:51:28.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.488+0000 7f424eec5640 1 -- 172.21.15.52:0/1384444624 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42481025b0 msgr2=0x7f4248108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:28.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.488+0000 7f424eec5640 1 --2- 172.21.15.52:0/1384444624 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42481025b0 0x7f4248108e40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4238009900 tx=0x7f423802bc30 comp rx=0 tx=0).stop 2024-04-02T13:51:28.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.488+0000 7f424eec5640 1 -- 172.21.15.52:0/1384444624 shutdown_connections 2024-04-02T13:51:28.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.488+0000 7f424eec5640 1 --2- 172.21.15.52:0/1384444624 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42481025b0 0x7f4248108e40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:28.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.489+0000 7f424eec5640 1 --2- 172.21.15.52:0/1384444624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4248101bf0 0x7f4248102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:28.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.489+0000 7f424eec5640 1 --2- 172.21.15.52:0/1384444624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4248100a00 0x7f4248100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:28.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.489+0000 7f424eec5640 1 -- 172.21.15.52:0/1384444624 >> 172.21.15.52:0/1384444624 conn(0x7f42480fbf90 msgr2=0x7f42480fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:28.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.489+0000 7f424eec5640 1 -- 172.21.15.52:0/1384444624 shutdown_connections 2024-04-02T13:51:28.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.489+0000 7f424eec5640 1 -- 172.21.15.52:0/1384444624 wait complete. 2024-04-02T13:51:28.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.489+0000 7f424eec5640 1 Processor -- start 2024-04-02T13:51:28.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.489+0000 7f424eec5640 1 -- start start 2024-04-02T13:51:28.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.489+0000 7f424eec5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4248100a00 0x7f4248197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:28.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.489+0000 7f424eec5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4248101bf0 0x7f4248198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:28.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.489+0000 7f424eec5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42481025b0 0x7f424819f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:28.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.489+0000 7f424eec5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f424810c660 con 0x7f42481025b0 2024-04-02T13:51:28.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.489+0000 7f424eec5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f424810c4e0 con 0x7f4248100a00 2024-04-02T13:51:28.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.489+0000 7f424eec5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f424810c7e0 con 0x7f4248101bf0 2024-04-02T13:51:28.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.489+0000 7f424d462640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42481025b0 0x7f424819f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:28.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.489+0000 7f424d462640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42481025b0 0x7f424819f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53548/0 (socket says 172.21.15.52:53548) 2024-04-02T13:51:28.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.489+0000 7f424d462640 1 -- 172.21.15.52:0/847630139 learned_addr learned my addr 172.21.15.52:0/847630139 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:28.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.489+0000 7f424cc61640 1 --2- 172.21.15.52:0/847630139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4248100a00 0x7f4248197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:28.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.490+0000 7f4247fff640 1 --2- 172.21.15.52:0/847630139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4248101bf0 0x7f4248198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:28.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.490+0000 7f424d462640 1 -- 172.21.15.52:0/847630139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4248101bf0 msgr2=0x7f4248198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:28.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.490+0000 7f424d462640 1 --2- 172.21.15.52:0/847630139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4248101bf0 0x7f4248198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:28.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.490+0000 7f424d462640 1 -- 172.21.15.52:0/847630139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4248100a00 msgr2=0x7f4248197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:28.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.490+0000 7f424d462640 1 --2- 172.21.15.52:0/847630139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4248100a00 0x7f4248197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:28.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.490+0000 7f424d462640 1 -- 172.21.15.52:0/847630139 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f424819f9d0 con 0x7f42481025b0 2024-04-02T13:51:28.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.490+0000 7f4247fff640 1 --2- 172.21.15.52:0/847630139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4248101bf0 0x7f4248198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:51:28.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.490+0000 7f424cc61640 1 --2- 172.21.15.52:0/847630139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4248100a00 0x7f4248197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:28.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.490+0000 7f424d462640 1 --2- 172.21.15.52:0/847630139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42481025b0 0x7f424819f330 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f423802c7b0 tx=0x7f4238005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:28.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.490+0000 7f4245ffb640 1 -- 172.21.15.52:0/847630139 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238041070 con 0x7f42481025b0 2024-04-02T13:51:28.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.490+0000 7f424eec5640 1 -- 172.21.15.52:0/847630139 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f424819fc60 con 0x7f42481025b0 2024-04-02T13:51:28.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.490+0000 7f4245ffb640 1 -- 172.21.15.52:0/847630139 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42380054e0 con 0x7f42481025b0 2024-04-02T13:51:28.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.490+0000 7f4245ffb640 1 -- 172.21.15.52:0/847630139 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238034440 con 0x7f42481025b0 2024-04-02T13:51:28.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.490+0000 7f424eec5640 1 -- 172.21.15.52:0/847630139 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42481a0150 con 0x7f42481025b0 2024-04-02T13:51:28.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.491+0000 7f424eec5640 1 -- 172.21.15.52:0/847630139 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42481a0320 con 0x7f42481025b0 2024-04-02T13:51:28.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.492+0000 7f4245ffb640 1 -- 172.21.15.52:0/847630139 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4238044070 con 0x7f42481025b0 2024-04-02T13:51:28.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.493+0000 7f4245ffb640 1 --2- 172.21.15.52:0/847630139 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f42300a3bb0 0x7f42300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:28.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.493+0000 7f4245ffb640 1 -- 172.21.15.52:0/847630139 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f42380f2450 con 0x7f42481025b0 2024-04-02T13:51:28.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.493+0000 7f424cc61640 1 --2- 172.21.15.52:0/847630139 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f42300a3bb0 0x7f42300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:28.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.493+0000 7f424cc61640 1 --2- 172.21.15.52:0/847630139 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f42300a3bb0 0x7f42300a6070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4240009fd0 tx=0x7f4240009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:28.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.498+0000 7f4245ffb640 1 -- 172.21.15.52:0/847630139 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f42380beb90 con 0x7f42481025b0 2024-04-02T13:51:28.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:28 smithi088 ceph-mon[36528]: pgmap v306: 129 pgs: 129 active+clean; 582 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 1 op/s 2024-04-02T13:51:28.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:28 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] up:active 2024-04-02T13:51:28.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:28 smithi088 ceph-mon[36528]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-02T13:51:28.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:28 smithi088 ceph-mon[36528]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-02T13:51:28.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:28 smithi088 ceph-mon[36528]: Cluster is now healthy 2024-04-02T13:51:28.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:28 smithi088 ceph-mon[36528]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-04-02T13:51:28.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:28 smithi088 ceph-mon[36528]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-02T13:51:28.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:28 smithi088 ceph-mon[36528]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-04-02T13:51:28.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.712+0000 7f424eec5640 1 -- 172.21.15.52:0/847630139 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f42480fe0b0 con 0x7f42300a3bb0 2024-04-02T13:51:28.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.736+0000 7f4245ffb640 1 -- 172.21.15.52:0/847630139 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f42480fe0b0 con 0x7f42300a3bb0 2024-04-02T13:51:28.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.738+0000 7f424eec5640 1 -- 172.21.15.52:0/847630139 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f42300a3bb0 msgr2=0x7f42300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:28.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.738+0000 7f424eec5640 1 --2- 172.21.15.52:0/847630139 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f42300a3bb0 0x7f42300a6070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4240009fd0 tx=0x7f4240009500 comp rx=0 tx=0).stop 2024-04-02T13:51:28.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.738+0000 7f424eec5640 1 -- 172.21.15.52:0/847630139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42481025b0 msgr2=0x7f424819f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:28.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.738+0000 7f424eec5640 1 --2- 172.21.15.52:0/847630139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42481025b0 0x7f424819f330 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f423802c7b0 tx=0x7f4238005dd0 comp rx=0 tx=0).stop 2024-04-02T13:51:28.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.738+0000 7f424eec5640 1 -- 172.21.15.52:0/847630139 shutdown_connections 2024-04-02T13:51:28.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.738+0000 7f424eec5640 1 --2- 172.21.15.52:0/847630139 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f42300a3bb0 0x7f42300a6070 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:28.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.738+0000 7f424eec5640 1 --2- 172.21.15.52:0/847630139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42481025b0 0x7f424819f330 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:28.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.738+0000 7f424eec5640 1 --2- 172.21.15.52:0/847630139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4248101bf0 0x7f4248198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:28.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.738+0000 7f424eec5640 1 --2- 172.21.15.52:0/847630139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4248100a00 0x7f4248197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:28.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.738+0000 7f424eec5640 1 -- 172.21.15.52:0/847630139 >> 172.21.15.52:0/847630139 conn(0x7f42480fbf90 msgr2=0x7f42480fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:28.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.738+0000 7f424eec5640 1 -- 172.21.15.52:0/847630139 shutdown_connections 2024-04-02T13:51:28.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.738+0000 7f424eec5640 1 -- 172.21.15.52:0/847630139 wait complete. 2024-04-02T13:51:28.751 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:28 smithi052 ceph-mon[32104]: pgmap v306: 129 pgs: 129 active+clean; 582 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 1 op/s 2024-04-02T13:51:28.751 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:28 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] up:active 2024-04-02T13:51:28.751 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:28 smithi052 ceph-mon[32104]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-02T13:51:28.751 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:28 smithi052 ceph-mon[32104]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-02T13:51:28.751 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:28 smithi052 ceph-mon[32104]: Cluster is now healthy 2024-04-02T13:51:28.751 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:28 smithi052 ceph-mon[32104]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-04-02T13:51:28.751 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:28 smithi052 ceph-mon[32104]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-02T13:51:28.751 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:28 smithi052 ceph-mon[32104]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-04-02T13:51:28.752 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-02T13:51:28.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:28 smithi189 ceph-mon[36683]: pgmap v306: 129 pgs: 129 active+clean; 582 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 1 op/s 2024-04-02T13:51:28.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:28 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] up:active 2024-04-02T13:51:28.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:28 smithi189 ceph-mon[36683]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-02T13:51:28.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:28 smithi189 ceph-mon[36683]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-02T13:51:28.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:28 smithi189 ceph-mon[36683]: Cluster is now healthy 2024-04-02T13:51:28.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:28 smithi189 ceph-mon[36683]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-04-02T13:51:28.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:28 smithi189 ceph-mon[36683]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-02T13:51:28.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:28 smithi189 ceph-mon[36683]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-04-02T13:51:28.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.856+0000 7fe06a5a1640 1 -- 172.21.15.52:0/877380161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0641008e0 msgr2=0x7fe064100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:28.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.856+0000 7fe06a5a1640 1 --2- 172.21.15.52:0/877380161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0641008e0 0x7fe064100d00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe060009900 tx=0x7fe06002bc30 comp rx=0 tx=0).stop 2024-04-02T13:51:28.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.856+0000 7fe06a5a1640 1 -- 172.21.15.52:0/877380161 shutdown_connections 2024-04-02T13:51:28.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.856+0000 7fe06a5a1640 1 --2- 172.21.15.52:0/877380161 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe064102490 0x7fe064108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:28.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.856+0000 7fe06a5a1640 1 --2- 172.21.15.52:0/877380161 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe064101ad0 0x7fe064101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:28.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.856+0000 7fe06a5a1640 1 --2- 172.21.15.52:0/877380161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0641008e0 0x7fe064100d00 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:28.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.856+0000 7fe06a5a1640 1 -- 172.21.15.52:0/877380161 >> 172.21.15.52:0/877380161 conn(0x7fe0640fbe70 msgr2=0x7fe0640fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:28.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.856+0000 7fe06a5a1640 1 -- 172.21.15.52:0/877380161 shutdown_connections 2024-04-02T13:51:28.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.856+0000 7fe06a5a1640 1 -- 172.21.15.52:0/877380161 wait complete. 2024-04-02T13:51:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.857+0000 7fe06a5a1640 1 Processor -- start 2024-04-02T13:51:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.857+0000 7fe06a5a1640 1 -- start start 2024-04-02T13:51:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.857+0000 7fe06a5a1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0641008e0 0x7fe064104870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.857+0000 7fe06a5a1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe064101ad0 0x7fe064102eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.857+0000 7fe06a5a1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe064102490 0x7fe0641033f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.857+0000 7fe06a5a1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe06410c550 con 0x7fe0641008e0 2024-04-02T13:51:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.857+0000 7fe06a5a1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe06410c3d0 con 0x7fe064101ad0 2024-04-02T13:51:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.857+0000 7fe06a5a1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe06410c6d0 con 0x7fe064102490 2024-04-02T13:51:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.857+0000 7fe06959f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0641008e0 0x7fe064104870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.857+0000 7fe06959f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0641008e0 0x7fe064104870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53572/0 (socket says 172.21.15.52:53572) 2024-04-02T13:51:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.857+0000 7fe06959f640 1 -- 172.21.15.52:0/2089757485 learned_addr learned my addr 172.21.15.52:0/2089757485 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.857+0000 7fe068d9e640 1 --2- 172.21.15.52:0/2089757485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe064101ad0 0x7fe064102eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.857+0000 7fe06959f640 1 -- 172.21.15.52:0/2089757485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe064102490 msgr2=0x7fe0641033f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:28.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.857+0000 7fe069da0640 1 --2- 172.21.15.52:0/2089757485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe064102490 0x7fe0641033f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:28.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.857+0000 7fe06959f640 1 --2- 172.21.15.52:0/2089757485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe064102490 0x7fe0641033f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:28.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.857+0000 7fe06959f640 1 -- 172.21.15.52:0/2089757485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe064101ad0 msgr2=0x7fe064102eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:28.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.857+0000 7fe06959f640 1 --2- 172.21.15.52:0/2089757485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe064101ad0 0x7fe064102eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:28.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.857+0000 7fe06959f640 1 -- 172.21.15.52:0/2089757485 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe064103c80 con 0x7fe0641008e0 2024-04-02T13:51:28.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.857+0000 7fe069da0640 1 --2- 172.21.15.52:0/2089757485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe064102490 0x7fe0641033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:51:28.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.858+0000 7fe06959f640 1 --2- 172.21.15.52:0/2089757485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0641008e0 0x7fe064104870 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe060000c00 tx=0x7fe060005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:28.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.858+0000 7fe05a7fc640 1 -- 172.21.15.52:0/2089757485 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060041070 con 0x7fe0641008e0 2024-04-02T13:51:28.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.858+0000 7fe05a7fc640 1 -- 172.21.15.52:0/2089757485 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0600044d0 con 0x7fe0641008e0 2024-04-02T13:51:28.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.858+0000 7fe06a5a1640 1 -- 172.21.15.52:0/2089757485 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0641a1e30 con 0x7fe0641008e0 2024-04-02T13:51:28.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.858+0000 7fe05a7fc640 1 -- 172.21.15.52:0/2089757485 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060034440 con 0x7fe0641008e0 2024-04-02T13:51:28.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.858+0000 7fe06a5a1640 1 -- 172.21.15.52:0/2089757485 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0641a2290 con 0x7fe0641008e0 2024-04-02T13:51:28.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.859+0000 7fe06a5a1640 1 -- 172.21.15.52:0/2089757485 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0641a2520 con 0x7fe0641008e0 2024-04-02T13:51:28.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.861+0000 7fe05a7fc640 1 -- 172.21.15.52:0/2089757485 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe060044070 con 0x7fe0641008e0 2024-04-02T13:51:28.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.861+0000 7fe05a7fc640 1 --2- 172.21.15.52:0/2089757485 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe0500a3c30 0x7fe0500a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:28.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.861+0000 7fe05a7fc640 1 -- 172.21.15.52:0/2089757485 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe0600f2310 con 0x7fe0641008e0 2024-04-02T13:51:28.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.861+0000 7fe068d9e640 1 --2- 172.21.15.52:0/2089757485 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe0500a3c30 0x7fe0500a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:28.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.862+0000 7fe068d9e640 1 --2- 172.21.15.52:0/2089757485 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe0500a3c30 0x7fe0500a60f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe0641045e0 tx=0x7fe05400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:28.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:28.865+0000 7fe05a7fc640 1 -- 172.21.15.52:0/2089757485 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe0600bea50 con 0x7fe0641008e0 2024-04-02T13:51:29.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.081+0000 7fe06a5a1640 1 -- 172.21.15.52:0/2089757485 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fe0640fe070 con 0x7fe0500a3c30 2024-04-02T13:51:29.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.087+0000 7fe05a7fc640 1 -- 172.21.15.52:0/2089757485 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fe0640fe070 con 0x7fe0500a3c30 2024-04-02T13:51:29.090 INFO:teuthology.orchestra.run.smithi052.stdout:/volumes/qa/sv_0/e6d42db3-0e75-4a80-88a8-278ffdf21489 2024-04-02T13:51:29.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.089+0000 7fe06a5a1640 1 -- 172.21.15.52:0/2089757485 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe0500a3c30 msgr2=0x7fe0500a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:29.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.089+0000 7fe06a5a1640 1 --2- 172.21.15.52:0/2089757485 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe0500a3c30 0x7fe0500a60f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe0641045e0 tx=0x7fe05400b390 comp rx=0 tx=0).stop 2024-04-02T13:51:29.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.089+0000 7fe06a5a1640 1 -- 172.21.15.52:0/2089757485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0641008e0 msgr2=0x7fe064104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:29.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.089+0000 7fe06a5a1640 1 --2- 172.21.15.52:0/2089757485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0641008e0 0x7fe064104870 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe060000c00 tx=0x7fe060005d50 comp rx=0 tx=0).stop 2024-04-02T13:51:29.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.089+0000 7fe06a5a1640 1 -- 172.21.15.52:0/2089757485 shutdown_connections 2024-04-02T13:51:29.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.089+0000 7fe06a5a1640 1 --2- 172.21.15.52:0/2089757485 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe0500a3c30 0x7fe0500a60f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.089+0000 7fe06a5a1640 1 --2- 172.21.15.52:0/2089757485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe064102490 0x7fe0641033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.089+0000 7fe06a5a1640 1 --2- 172.21.15.52:0/2089757485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe064101ad0 0x7fe064102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.089+0000 7fe06a5a1640 1 --2- 172.21.15.52:0/2089757485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0641008e0 0x7fe064104870 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.089+0000 7fe06a5a1640 1 -- 172.21.15.52:0/2089757485 >> 172.21.15.52:0/2089757485 conn(0x7fe0640fbe70 msgr2=0x7fe0640fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:29.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.089+0000 7fe06a5a1640 1 -- 172.21.15.52:0/2089757485 shutdown_connections 2024-04-02T13:51:29.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.089+0000 7fe06a5a1640 1 -- 172.21.15.52:0/2089757485 wait complete. 2024-04-02T13:51:29.103 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-02T13:51:29.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.207+0000 7f9537647640 1 -- 172.21.15.52:0/1758035507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9530101c00 msgr2=0x7f9530102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:29.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.207+0000 7f9537647640 1 --2- 172.21.15.52:0/1758035507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9530101c00 0x7f9530102080 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9520009900 tx=0x7f952002bc30 comp rx=0 tx=0).stop 2024-04-02T13:51:29.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.207+0000 7f9537647640 1 -- 172.21.15.52:0/1758035507 shutdown_connections 2024-04-02T13:51:29.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.207+0000 7f9537647640 1 --2- 172.21.15.52:0/1758035507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95301025c0 0x7f9530108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.207+0000 7f9537647640 1 --2- 172.21.15.52:0/1758035507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9530101c00 0x7f9530102080 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.207+0000 7f9537647640 1 --2- 172.21.15.52:0/1758035507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9530100a10 0x7f9530100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.207+0000 7f9537647640 1 -- 172.21.15.52:0/1758035507 >> 172.21.15.52:0/1758035507 conn(0x7f95300fbfa0 msgr2=0x7f95300fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:29.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.207+0000 7f9537647640 1 -- 172.21.15.52:0/1758035507 shutdown_connections 2024-04-02T13:51:29.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.207+0000 7f9537647640 1 -- 172.21.15.52:0/1758035507 wait complete. 2024-04-02T13:51:29.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.208+0000 7f9537647640 1 Processor -- start 2024-04-02T13:51:29.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.208+0000 7f9537647640 1 -- start start 2024-04-02T13:51:29.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.208+0000 7f9537647640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9530100a10 0x7f9530196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:29.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.208+0000 7f9537647640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9530101c00 0x7f9530197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:29.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.208+0000 7f9537647640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95301025c0 0x7f953019e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:29.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.208+0000 7f9537647640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f953010c670 con 0x7f9530101c00 2024-04-02T13:51:29.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.208+0000 7f9537647640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f953010c4f0 con 0x7f9530100a10 2024-04-02T13:51:29.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.208+0000 7f9537647640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f953010c7f0 con 0x7f95301025c0 2024-04-02T13:51:29.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.208+0000 7f9534be2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9530101c00 0x7f9530197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:29.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.208+0000 7f9534be2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9530101c00 0x7f9530197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53592/0 (socket says 172.21.15.52:53592) 2024-04-02T13:51:29.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.208+0000 7f9534be2640 1 -- 172.21.15.52:0/1925424602 learned_addr learned my addr 172.21.15.52:0/1925424602 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:29.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.208+0000 7f95353e3640 1 --2- 172.21.15.52:0/1925424602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9530100a10 0x7f9530196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:29.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.208+0000 7f9535be4640 1 --2- 172.21.15.52:0/1925424602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95301025c0 0x7f953019e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:29.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.208+0000 7f95353e3640 1 -- 172.21.15.52:0/1925424602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95301025c0 msgr2=0x7f953019e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:29.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.208+0000 7f95353e3640 1 --2- 172.21.15.52:0/1925424602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95301025c0 0x7f953019e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.208+0000 7f95353e3640 1 -- 172.21.15.52:0/1925424602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9530101c00 msgr2=0x7f9530197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:29.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.208+0000 7f95353e3640 1 --2- 172.21.15.52:0/1925424602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9530101c00 0x7f9530197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.208+0000 7f95353e3640 1 -- 172.21.15.52:0/1925424602 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953019e960 con 0x7f9530100a10 2024-04-02T13:51:29.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.208+0000 7f9535be4640 1 --2- 172.21.15.52:0/1925424602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95301025c0 0x7f953019e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:51:29.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.208+0000 7f9534be2640 1 --2- 172.21.15.52:0/1925424602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9530101c00 0x7f9530197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:29.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.209+0000 7f95353e3640 1 --2- 172.21.15.52:0/1925424602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9530100a10 0x7f9530196d50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f952c00e9c0 tx=0x7f952c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:29.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.209+0000 7f95267fc640 1 -- 172.21.15.52:0/1925424602 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c00cd80 con 0x7f9530100a10 2024-04-02T13:51:29.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.209+0000 7f9537647640 1 -- 172.21.15.52:0/1925424602 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f953019ec50 con 0x7f9530100a10 2024-04-02T13:51:29.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.209+0000 7f95267fc640 1 -- 172.21.15.52:0/1925424602 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f952c010910 con 0x7f9530100a10 2024-04-02T13:51:29.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.209+0000 7f95267fc640 1 -- 172.21.15.52:0/1925424602 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c018980 con 0x7f9530100a10 2024-04-02T13:51:29.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.209+0000 7f9537647640 1 -- 172.21.15.52:0/1925424602 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f953019f1a0 con 0x7f9530100a10 2024-04-02T13:51:29.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.210+0000 7f9537647640 1 -- 172.21.15.52:0/1925424602 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9530107e00 con 0x7f9530100a10 2024-04-02T13:51:29.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.211+0000 7f95267fc640 1 -- 172.21.15.52:0/1925424602 <== mon.2 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f952c01f020 con 0x7f9530100a10 2024-04-02T13:51:29.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.211+0000 7f95267fc640 1 --2- 172.21.15.52:0/1925424602 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f951c0a3bb0 0x7f951c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:29.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.211+0000 7f9534be2640 1 --2- 172.21.15.52:0/1925424602 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f951c0a3bb0 0x7f951c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:29.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.211+0000 7f95267fc640 1 -- 172.21.15.52:0/1925424602 <== mon.2 v2:172.21.15.88:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f952c0d0950 con 0x7f9530100a10 2024-04-02T13:51:29.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.212+0000 7f9534be2640 1 --2- 172.21.15.52:0/1925424602 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f951c0a3bb0 0x7f951c0a6070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9520000c00 tx=0x7f952002c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:29.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.215+0000 7f95267fc640 1 -- 172.21.15.52:0/1925424602 <== mon.2 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f952c09d0f0 con 0x7f9530100a10 2024-04-02T13:51:29.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.426+0000 7f95267fc640 1 -- 172.21.15.52:0/1925424602 <== mon.2 v2:172.21.15.88:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f952c018b20 con 0x7f9530100a10 2024-04-02T13:51:29.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.442+0000 7f9537647640 1 -- 172.21.15.52:0/1925424602 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9530061fd0 con 0x7f951c0a3bb0 2024-04-02T13:51:29.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.471+0000 7f95267fc640 1 -- 172.21.15.52:0/1925424602 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9530061fd0 con 0x7f951c0a3bb0 2024-04-02T13:51:29.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.473+0000 7f9537647640 1 -- 172.21.15.52:0/1925424602 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f951c0a3bb0 msgr2=0x7f951c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:29.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.473+0000 7f9537647640 1 --2- 172.21.15.52:0/1925424602 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f951c0a3bb0 0x7f951c0a6070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9520000c00 tx=0x7f952002c880 comp rx=0 tx=0).stop 2024-04-02T13:51:29.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.473+0000 7f9537647640 1 -- 172.21.15.52:0/1925424602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9530100a10 msgr2=0x7f9530196d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:29.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.473+0000 7f9537647640 1 --2- 172.21.15.52:0/1925424602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9530100a10 0x7f9530196d50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f952c00e9c0 tx=0x7f952c00ee80 comp rx=0 tx=0).stop 2024-04-02T13:51:29.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.473+0000 7f9537647640 1 -- 172.21.15.52:0/1925424602 shutdown_connections 2024-04-02T13:51:29.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.473+0000 7f9537647640 1 --2- 172.21.15.52:0/1925424602 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f951c0a3bb0 0x7f951c0a6070 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.473+0000 7f9537647640 1 --2- 172.21.15.52:0/1925424602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95301025c0 0x7f953019e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.473+0000 7f9537647640 1 --2- 172.21.15.52:0/1925424602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9530101c00 0x7f9530197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.473+0000 7f9537647640 1 --2- 172.21.15.52:0/1925424602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9530100a10 0x7f9530196d50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.473+0000 7f9537647640 1 -- 172.21.15.52:0/1925424602 >> 172.21.15.52:0/1925424602 conn(0x7f95300fbfa0 msgr2=0x7f9530104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:29.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.473+0000 7f9537647640 1 -- 172.21.15.52:0/1925424602 shutdown_connections 2024-04-02T13:51:29.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.473+0000 7f9537647640 1 -- 172.21.15.52:0/1925424602 wait complete. 2024-04-02T13:51:29.487 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-02T13:51:29.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.592+0000 7fe004f5f640 1 -- 172.21.15.52:0/1226882152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0001025c0 msgr2=0x7fe000108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:29.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.592+0000 7fe004f5f640 1 --2- 172.21.15.52:0/1226882152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0001025c0 0x7fe000108e50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdff0009ca0 tx=0x7fdff002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.592+0000 7fe004f5f640 1 -- 172.21.15.52:0/1226882152 shutdown_connections 2024-04-02T13:51:29.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.592+0000 7fe004f5f640 1 --2- 172.21.15.52:0/1226882152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe0001025c0 0x7fe000108e50 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.592+0000 7fe004f5f640 1 --2- 172.21.15.52:0/1226882152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe000101c00 0x7fe000102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.592+0000 7fe004f5f640 1 --2- 172.21.15.52:0/1226882152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe000100a10 0x7fe000100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.592+0000 7fe004f5f640 1 -- 172.21.15.52:0/1226882152 >> 172.21.15.52:0/1226882152 conn(0x7fe0000fbfa0 msgr2=0x7fe0000fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:29.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.593+0000 7fe004f5f640 1 -- 172.21.15.52:0/1226882152 shutdown_connections 2024-04-02T13:51:29.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.593+0000 7fe004f5f640 1 -- 172.21.15.52:0/1226882152 wait complete. 2024-04-02T13:51:29.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.593+0000 7fe004f5f640 1 Processor -- start 2024-04-02T13:51:29.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.593+0000 7fe004f5f640 1 -- start start 2024-04-02T13:51:29.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.593+0000 7fe004f5f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe000100a10 0x7fe000197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:29.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.593+0000 7fe004f5f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe000101c00 0x7fe0001982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:29.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.593+0000 7fe004f5f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0001025c0 0x7fe00019f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:29.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.593+0000 7fe004f5f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe00010c670 con 0x7fe000101c00 2024-04-02T13:51:29.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.593+0000 7fe004f5f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe00010c4f0 con 0x7fe0001025c0 2024-04-02T13:51:29.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.593+0000 7fe004f5f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe00010c7f0 con 0x7fe000100a10 2024-04-02T13:51:29.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.593+0000 7fdffdd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe000101c00 0x7fe0001982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:29.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.594+0000 7fdffdd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe000101c00 0x7fe0001982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53610/0 (socket says 172.21.15.52:53610) 2024-04-02T13:51:29.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.594+0000 7fdffdd9b640 1 -- 172.21.15.52:0/3104482230 learned_addr learned my addr 172.21.15.52:0/3104482230 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:29.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.594+0000 7fdffed9d640 1 --2- 172.21.15.52:0/3104482230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0001025c0 0x7fe00019f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:29.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.594+0000 7fdffe59c640 1 --2- 172.21.15.52:0/3104482230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe000100a10 0x7fe000197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:29.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.594+0000 7fdffdd9b640 1 -- 172.21.15.52:0/3104482230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe000100a10 msgr2=0x7fe000197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:29.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.594+0000 7fdffdd9b640 1 --2- 172.21.15.52:0/3104482230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe000100a10 0x7fe000197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.594+0000 7fdffdd9b640 1 -- 172.21.15.52:0/3104482230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0001025c0 msgr2=0x7fe00019f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:29.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.594+0000 7fdffdd9b640 1 --2- 172.21.15.52:0/3104482230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0001025c0 0x7fe00019f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.594+0000 7fdffdd9b640 1 -- 172.21.15.52:0/3104482230 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00019f9d0 con 0x7fe000101c00 2024-04-02T13:51:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.594+0000 7fdffdd9b640 1 --2- 172.21.15.52:0/3104482230 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe000101c00 0x7fe0001982f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdff400c890 tx=0x7fdff400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.594+0000 7fdfef7fe640 1 -- 172.21.15.52:0/3104482230 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4004220 con 0x7fe000101c00 2024-04-02T13:51:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.594+0000 7fdfef7fe640 1 -- 172.21.15.52:0/3104482230 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdff40043c0 con 0x7fe000101c00 2024-04-02T13:51:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.594+0000 7fe004f5f640 1 -- 172.21.15.52:0/3104482230 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe00019fcc0 con 0x7fe000101c00 2024-04-02T13:51:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.594+0000 7fdfef7fe640 1 -- 172.21.15.52:0/3104482230 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff40049e0 con 0x7fe000101c00 2024-04-02T13:51:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.594+0000 7fe004f5f640 1 -- 172.21.15.52:0/3104482230 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0001a01c0 con 0x7fe000101c00 2024-04-02T13:51:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.595+0000 7fe004f5f640 1 -- 172.21.15.52:0/3104482230 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0001a0450 con 0x7fe000101c00 2024-04-02T13:51:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.596+0000 7fdfef7fe640 1 -- 172.21.15.52:0/3104482230 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdff40079b0 con 0x7fe000101c00 2024-04-02T13:51:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.597+0000 7fdfef7fe640 1 --2- 172.21.15.52:0/3104482230 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdfe80a3c40 0x7fdfe80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.597+0000 7fdffe59c640 1 --2- 172.21.15.52:0/3104482230 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdfe80a3c40 0x7fdfe80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.597+0000 7fdfef7fe640 1 -- 172.21.15.52:0/3104482230 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fdff40d07f0 con 0x7fe000101c00 2024-04-02T13:51:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.597+0000 7fdffe59c640 1 --2- 172.21.15.52:0/3104482230 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdfe80a3c40 0x7fdfe80a6100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe000101a60 tx=0x7fdff8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:29.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.601+0000 7fdfef7fe640 1 -- 172.21.15.52:0/3104482230 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fdff4014030 con 0x7fe000101c00 2024-04-02T13:51:29.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:29 smithi189 ceph-mon[36683]: from='client.14874 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:51:29.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:29 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] up:active 2024-04-02T13:51:29.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:29 smithi189 ceph-mon[36683]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-02T13:51:29.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:29 smithi189 ceph-mon[36683]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:29.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.819+0000 7fe004f5f640 1 -- 172.21.15.52:0/3104482230 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fe0000fe130 con 0x7fdfe80a3c40 2024-04-02T13:51:29.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.824+0000 7fdfef7fe640 1 -- 172.21.15.52:0/3104482230 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fe0000fe130 con 0x7fdfe80a3c40 2024-04-02T13:51:29.827 INFO:teuthology.orchestra.run.smithi052.stdout:/volumes/qa/sv_1/e3c6433a-5ae3-4174-8392-2dc149574911 2024-04-02T13:51:29.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.826+0000 7fe004f5f640 1 -- 172.21.15.52:0/3104482230 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdfe80a3c40 msgr2=0x7fdfe80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:29.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.826+0000 7fe004f5f640 1 --2- 172.21.15.52:0/3104482230 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdfe80a3c40 0x7fdfe80a6100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe000101a60 tx=0x7fdff8005f90 comp rx=0 tx=0).stop 2024-04-02T13:51:29.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.826+0000 7fe004f5f640 1 -- 172.21.15.52:0/3104482230 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe000101c00 msgr2=0x7fe0001982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:29.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.826+0000 7fe004f5f640 1 --2- 172.21.15.52:0/3104482230 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe000101c00 0x7fe0001982f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdff400c890 tx=0x7fdff400cd50 comp rx=0 tx=0).stop 2024-04-02T13:51:29.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.826+0000 7fe004f5f640 1 -- 172.21.15.52:0/3104482230 shutdown_connections 2024-04-02T13:51:29.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.826+0000 7fe004f5f640 1 --2- 172.21.15.52:0/3104482230 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdfe80a3c40 0x7fdfe80a6100 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.826+0000 7fe004f5f640 1 --2- 172.21.15.52:0/3104482230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0001025c0 0x7fe00019f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.826+0000 7fe004f5f640 1 --2- 172.21.15.52:0/3104482230 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe000101c00 0x7fe0001982f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.826+0000 7fe004f5f640 1 --2- 172.21.15.52:0/3104482230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe000100a10 0x7fe000197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.826+0000 7fe004f5f640 1 -- 172.21.15.52:0/3104482230 >> 172.21.15.52:0/3104482230 conn(0x7fe0000fbfa0 msgr2=0x7fe0000fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:29.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.826+0000 7fe004f5f640 1 -- 172.21.15.52:0/3104482230 shutdown_connections 2024-04-02T13:51:29.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.826+0000 7fe004f5f640 1 -- 172.21.15.52:0/3104482230 wait complete. 2024-04-02T13:51:29.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:29 smithi052 ceph-mon[32104]: from='client.14874 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:51:29.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:29 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] up:active 2024-04-02T13:51:29.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:29 smithi052 ceph-mon[32104]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-02T13:51:29.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:29 smithi052 ceph-mon[32104]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:29.840 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:51:29.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:29 smithi088 ceph-mon[36528]: from='client.14874 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:51:29.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:29 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] up:active 2024-04-02T13:51:29.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:29 smithi088 ceph-mon[36528]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-02T13:51:29.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:29 smithi088 ceph-mon[36528]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:29.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.940+0000 7feac1504640 1 -- 172.21.15.52:0/953567789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feabc1013c0 msgr2=0x7feabc101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:29.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.940+0000 7feac1504640 1 --2- 172.21.15.52:0/953567789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feabc1013c0 0x7feabc101840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feaa4009860 tx=0x7feaa402bb70 comp rx=0 tx=0).stop 2024-04-02T13:51:29.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.940+0000 7feac1504640 1 -- 172.21.15.52:0/953567789 shutdown_connections 2024-04-02T13:51:29.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.940+0000 7feac1504640 1 --2- 172.21.15.52:0/953567789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feabc101d80 0x7feabc108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.940+0000 7feac1504640 1 --2- 172.21.15.52:0/953567789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feabc1013c0 0x7feabc101840 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.940+0000 7feac1504640 1 --2- 172.21.15.52:0/953567789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feabc1001d0 0x7feabc1005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.940+0000 7feac1504640 1 -- 172.21.15.52:0/953567789 >> 172.21.15.52:0/953567789 conn(0x7feabc0fb740 msgr2=0x7feabc0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:29.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.940+0000 7feac1504640 1 -- 172.21.15.52:0/953567789 shutdown_connections 2024-04-02T13:51:29.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.940+0000 7feac1504640 1 -- 172.21.15.52:0/953567789 wait complete. 2024-04-02T13:51:29.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.940+0000 7feac1504640 1 Processor -- start 2024-04-02T13:51:29.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.940+0000 7feac1504640 1 -- start start 2024-04-02T13:51:29.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.941+0000 7feac1504640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feabc1001d0 0x7feabc197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:29.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.941+0000 7feac1504640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feabc1013c0 0x7feabc1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:29.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.941+0000 7feac1504640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feabc101d80 0x7feabc19f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:29.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.941+0000 7feac1504640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feabc10be60 con 0x7feabc101d80 2024-04-02T13:51:29.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.941+0000 7feac1504640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feabc10bce0 con 0x7feabc1013c0 2024-04-02T13:51:29.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.941+0000 7feac1504640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feabc10bfe0 con 0x7feabc1001d0 2024-04-02T13:51:29.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.941+0000 7feabb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feabc101d80 0x7feabc19f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:29.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.941+0000 7feabb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feabc101d80 0x7feabc19f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53624/0 (socket says 172.21.15.52:53624) 2024-04-02T13:51:29.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.941+0000 7feabb7fe640 1 -- 172.21.15.52:0/280229263 learned_addr learned my addr 172.21.15.52:0/280229263 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:29.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.941+0000 7feaba7fc640 1 --2- 172.21.15.52:0/280229263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feabc1013c0 0x7feabc1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:29.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.941+0000 7feabaffd640 1 --2- 172.21.15.52:0/280229263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feabc1001d0 0x7feabc197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:29.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.941+0000 7feabb7fe640 1 -- 172.21.15.52:0/280229263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feabc1001d0 msgr2=0x7feabc197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:29.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.941+0000 7feabb7fe640 1 --2- 172.21.15.52:0/280229263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feabc1001d0 0x7feabc197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.941+0000 7feabb7fe640 1 -- 172.21.15.52:0/280229263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feabc1013c0 msgr2=0x7feabc1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:29.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.941+0000 7feabb7fe640 1 --2- 172.21.15.52:0/280229263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feabc1013c0 0x7feabc1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:29.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.941+0000 7feabb7fe640 1 -- 172.21.15.52:0/280229263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feabc19fa70 con 0x7feabc101d80 2024-04-02T13:51:29.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.941+0000 7feaba7fc640 1 --2- 172.21.15.52:0/280229263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feabc1013c0 0x7feabc1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:29.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.941+0000 7feabb7fe640 1 --2- 172.21.15.52:0/280229263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feabc101d80 0x7feabc19f400 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feaac00eaa0 tx=0x7feaac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:29.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.941+0000 7feab3fff640 1 -- 172.21.15.52:0/280229263 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac00cdb0 con 0x7feabc101d80 2024-04-02T13:51:29.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.941+0000 7feac1504640 1 -- 172.21.15.52:0/280229263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feabc19fd60 con 0x7feabc101d80 2024-04-02T13:51:29.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.941+0000 7feab3fff640 1 -- 172.21.15.52:0/280229263 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaac005020 con 0x7feabc101d80 2024-04-02T13:51:29.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.941+0000 7feab3fff640 1 -- 172.21.15.52:0/280229263 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac0116b0 con 0x7feabc101d80 2024-04-02T13:51:29.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.941+0000 7feac1504640 1 -- 172.21.15.52:0/280229263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feabc1a0220 con 0x7feabc101d80 2024-04-02T13:51:29.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.943+0000 7feac1504640 1 -- 172.21.15.52:0/280229263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feabc1a04b0 con 0x7feabc101d80 2024-04-02T13:51:29.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.944+0000 7feab3fff640 1 -- 172.21.15.52:0/280229263 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7feaac005560 con 0x7feabc101d80 2024-04-02T13:51:29.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.944+0000 7feab3fff640 1 --2- 172.21.15.52:0/280229263 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7feaa80a3c40 0x7feaa80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:29.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.944+0000 7feab3fff640 1 -- 172.21.15.52:0/280229263 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7feaac0d0960 con 0x7feabc101d80 2024-04-02T13:51:29.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.944+0000 7feabaffd640 1 --2- 172.21.15.52:0/280229263 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7feaa80a3c40 0x7feaa80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:29.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.945+0000 7feabaffd640 1 --2- 172.21.15.52:0/280229263 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7feaa80a3c40 0x7feaa80a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7feab4009530 tx=0x7feab4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:29.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:29.949+0000 7feab3fff640 1 -- 172.21.15.52:0/280229263 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7feaac016030 con 0x7feabc101d80 2024-04-02T13:51:30.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.175+0000 7feac1504640 1 -- 172.21.15.52:0/280229263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feabc1a0680 con 0x7feabc101d80 2024-04-02T13:51:30.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.176+0000 7feab3fff640 1 -- 172.21.15.52:0/280229263 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7feaac09c950 con 0x7feabc101d80 2024-04-02T13:51:30.179 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:51:30.179 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:51:29.345060+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24503,24475,24490,24467,24457]},"id":1}]} 2024-04-02T13:51:30.179 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 27 2024-04-02T13:51:30.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.178+0000 7feac1504640 1 -- 172.21.15.52:0/280229263 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7feaa80a3c40 msgr2=0x7feaa80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:30.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.178+0000 7feac1504640 1 --2- 172.21.15.52:0/280229263 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7feaa80a3c40 0x7feaa80a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7feab4009530 tx=0x7feab4009260 comp rx=0 tx=0).stop 2024-04-02T13:51:30.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.178+0000 7feac1504640 1 -- 172.21.15.52:0/280229263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feabc101d80 msgr2=0x7feabc19f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:30.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.178+0000 7feac1504640 1 --2- 172.21.15.52:0/280229263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feabc101d80 0x7feabc19f400 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feaac00eaa0 tx=0x7feaac00ef60 comp rx=0 tx=0).stop 2024-04-02T13:51:30.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.178+0000 7feac1504640 1 -- 172.21.15.52:0/280229263 shutdown_connections 2024-04-02T13:51:30.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.178+0000 7feac1504640 1 --2- 172.21.15.52:0/280229263 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7feaa80a3c40 0x7feaa80a6100 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.178+0000 7feac1504640 1 --2- 172.21.15.52:0/280229263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feabc101d80 0x7feabc19f400 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.178+0000 7feac1504640 1 --2- 172.21.15.52:0/280229263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feabc1013c0 0x7feabc1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.178+0000 7feac1504640 1 --2- 172.21.15.52:0/280229263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feabc1001d0 0x7feabc197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.178+0000 7feac1504640 1 -- 172.21.15.52:0/280229263 >> 172.21.15.52:0/280229263 conn(0x7feabc0fb740 msgr2=0x7feabc0fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:30.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.178+0000 7feac1504640 1 -- 172.21.15.52:0/280229263 shutdown_connections 2024-04-02T13:51:30.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.178+0000 7feac1504640 1 -- 172.21.15.52:0/280229263 wait complete. 2024-04-02T13:51:30.192 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:51:30.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.291+0000 7f47a2b3d640 1 -- 172.21.15.52:0/4254648110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f479c101c10 msgr2=0x7f479c102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:30.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.291+0000 7f47a2b3d640 1 --2- 172.21.15.52:0/4254648110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f479c101c10 0x7f479c102090 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f478c009ca0 tx=0x7f478c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.292+0000 7f47a2b3d640 1 -- 172.21.15.52:0/4254648110 shutdown_connections 2024-04-02T13:51:30.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.292+0000 7f47a2b3d640 1 --2- 172.21.15.52:0/4254648110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f479c1025d0 0x7f479c108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.292+0000 7f47a2b3d640 1 --2- 172.21.15.52:0/4254648110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f479c101c10 0x7f479c102090 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.292+0000 7f47a2b3d640 1 --2- 172.21.15.52:0/4254648110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f479c100a20 0x7f479c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.292+0000 7f47a2b3d640 1 -- 172.21.15.52:0/4254648110 >> 172.21.15.52:0/4254648110 conn(0x7f479c0fbf90 msgr2=0x7f479c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:30.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.292+0000 7f47a2b3d640 1 -- 172.21.15.52:0/4254648110 shutdown_connections 2024-04-02T13:51:30.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.292+0000 7f47a2b3d640 1 -- 172.21.15.52:0/4254648110 wait complete. 2024-04-02T13:51:30.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.292+0000 7f47a2b3d640 1 Processor -- start 2024-04-02T13:51:30.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.292+0000 7f47a2b3d640 1 -- start start 2024-04-02T13:51:30.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.292+0000 7f47a2b3d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f479c100a20 0x7f479c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:30.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.292+0000 7f47a2b3d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f479c101c10 0x7f479c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:30.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.292+0000 7f47a2b3d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f479c1025d0 0x7f479c19f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:30.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.292+0000 7f47a2b3d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f479c10c680 con 0x7f479c100a20 2024-04-02T13:51:30.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.292+0000 7f47a2b3d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f479c10c500 con 0x7f479c101c10 2024-04-02T13:51:30.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.292+0000 7f47a2b3d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f479c10c800 con 0x7f479c1025d0 2024-04-02T13:51:30.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.292+0000 7f47a08d9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f479c100a20 0x7f479c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:30.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.292+0000 7f47a08d9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f479c100a20 0x7f479c0697d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53636/0 (socket says 172.21.15.52:53636) 2024-04-02T13:51:30.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.292+0000 7f47a08d9640 1 -- 172.21.15.52:0/1857932961 learned_addr learned my addr 172.21.15.52:0/1857932961 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:30.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.292+0000 7f479bfff640 1 --2- 172.21.15.52:0/1857932961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f479c101c10 0x7f479c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:30.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.293+0000 7f47a10da640 1 --2- 172.21.15.52:0/1857932961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f479c1025d0 0x7f479c19f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:30.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.293+0000 7f47a08d9640 1 -- 172.21.15.52:0/1857932961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f479c1025d0 msgr2=0x7f479c19f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:30.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.293+0000 7f47a08d9640 1 --2- 172.21.15.52:0/1857932961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f479c1025d0 0x7f479c19f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.293+0000 7f47a08d9640 1 -- 172.21.15.52:0/1857932961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f479c101c10 msgr2=0x7f479c19cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:30.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.293+0000 7f47a08d9640 1 --2- 172.21.15.52:0/1857932961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f479c101c10 0x7f479c19cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.293+0000 7f47a08d9640 1 -- 172.21.15.52:0/1857932961 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479c19faf0 con 0x7f479c100a20 2024-04-02T13:51:30.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.293+0000 7f47a10da640 1 --2- 172.21.15.52:0/1857932961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f479c1025d0 0x7f479c19f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:51:30.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.293+0000 7f479bfff640 1 --2- 172.21.15.52:0/1857932961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f479c101c10 0x7f479c19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:30.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.293+0000 7f47a08d9640 1 --2- 172.21.15.52:0/1857932961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f479c100a20 0x7f479c0697d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f479400d770 tx=0x7f479400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:30.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.293+0000 7f4799ffb640 1 -- 172.21.15.52:0/1857932961 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4794017070 con 0x7f479c100a20 2024-04-02T13:51:30.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.293+0000 7f4799ffb640 1 -- 172.21.15.52:0/1857932961 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4794009c30 con 0x7f479c100a20 2024-04-02T13:51:30.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.293+0000 7f47a2b3d640 1 -- 172.21.15.52:0/1857932961 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f479c19fd80 con 0x7f479c100a20 2024-04-02T13:51:30.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.293+0000 7f4799ffb640 1 -- 172.21.15.52:0/1857932961 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4794003dd0 con 0x7f479c100a20 2024-04-02T13:51:30.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.293+0000 7f47a2b3d640 1 -- 172.21.15.52:0/1857932961 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f479c1a01e0 con 0x7f479c100a20 2024-04-02T13:51:30.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.294+0000 7f47a2b3d640 1 -- 172.21.15.52:0/1857932961 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f479c107d40 con 0x7f479c100a20 2024-04-02T13:51:30.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.295+0000 7f4799ffb640 1 -- 172.21.15.52:0/1857932961 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4794003fb0 con 0x7f479c100a20 2024-04-02T13:51:30.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.296+0000 7f4799ffb640 1 --2- 172.21.15.52:0/1857932961 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f47880a3c40 0x7f47880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:30.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.296+0000 7f479bfff640 1 --2- 172.21.15.52:0/1857932961 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f47880a3c40 0x7f47880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:30.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.296+0000 7f4799ffb640 1 -- 172.21.15.52:0/1857932961 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f47940d0140 con 0x7f479c100a20 2024-04-02T13:51:30.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.296+0000 7f479bfff640 1 --2- 172.21.15.52:0/1857932961 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f47880a3c40 0x7f47880a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f478c02c700 tx=0x7f478c005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:30.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.301+0000 7f4799ffb640 1 -- 172.21.15.52:0/1857932961 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f479409c780 con 0x7f479c100a20 2024-04-02T13:51:30.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.490+0000 7f47a2b3d640 1 -- 172.21.15.52:0/1857932961 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f479c02d5a0 con 0x7f479c100a20 2024-04-02T13:51:30.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.492+0000 7f4799ffb640 1 -- 172.21.15.52:0/1857932961 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24077 (secure 0 0 0) 0x7f4794009de0 con 0x7f479c100a20 2024-04-02T13:51:30.495 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:51:30.495 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":90,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T13:51:05.722301+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:50:58.778690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:50:59.561527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:51:03.343487+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:51:30.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.494+0000 7f47a2b3d640 1 -- 172.21.15.52:0/1857932961 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f47880a3c40 msgr2=0x7f47880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:30.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.494+0000 7f47a2b3d640 1 --2- 172.21.15.52:0/1857932961 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f47880a3c40 0x7f47880a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f478c02c700 tx=0x7f478c005ef0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.494+0000 7f47a2b3d640 1 -- 172.21.15.52:0/1857932961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f479c100a20 msgr2=0x7f479c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:30.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.494+0000 7f47a2b3d640 1 --2- 172.21.15.52:0/1857932961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f479c100a20 0x7f479c0697d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f479400d770 tx=0x7f479400dc30 comp rx=0 tx=0).stop 2024-04-02T13:51:30.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.494+0000 7f47a2b3d640 1 -- 172.21.15.52:0/1857932961 shutdown_connections 2024-04-02T13:51:30.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.494+0000 7f47a2b3d640 1 --2- 172.21.15.52:0/1857932961 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f47880a3c40 0x7f47880a6100 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.494+0000 7f47a2b3d640 1 --2- 172.21.15.52:0/1857932961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f479c1025d0 0x7f479c19f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.494+0000 7f47a2b3d640 1 --2- 172.21.15.52:0/1857932961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f479c101c10 0x7f479c19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.494+0000 7f47a2b3d640 1 --2- 172.21.15.52:0/1857932961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f479c100a20 0x7f479c0697d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.494+0000 7f47a2b3d640 1 -- 172.21.15.52:0/1857932961 >> 172.21.15.52:0/1857932961 conn(0x7f479c0fbf90 msgr2=0x7f479c104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:30.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.494+0000 7f47a2b3d640 1 -- 172.21.15.52:0/1857932961 shutdown_connections 2024-04-02T13:51:30.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.494+0000 7f47a2b3d640 1 -- 172.21.15.52:0/1857932961 wait complete. 2024-04-02T13:51:30.507 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:30 smithi052 ceph-mon[32104]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:51:30.507 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:30 smithi052 ceph-mon[32104]: pgmap v307: 129 pgs: 129 active+clean; 582 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-02T13:51:30.507 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:30 smithi052 ceph-mon[32104]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:51:30.507 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:30 smithi052 ceph-mon[32104]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-02T13:51:30.507 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:30 smithi052 ceph-mon[32104]: from='client.24692 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:51:30.507 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:30 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/280229263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:51:30.508 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-02T13:51:30.509 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T13:51:30.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.607+0000 7f7f966d3640 1 -- 172.21.15.52:0/2099627138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f900744e0 msgr2=0x7f7f90074940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:30.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.607+0000 7f7f966d3640 1 --2- 172.21.15.52:0/2099627138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f900744e0 0x7f7f90074940 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7f84009900 tx=0x7f7f8402bcd0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.607+0000 7f7f966d3640 1 -- 172.21.15.52:0/2099627138 shutdown_connections 2024-04-02T13:51:30.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.607+0000 7f7f966d3640 1 --2- 172.21.15.52:0/2099627138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f90109a00 0x7f7f9010be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.607+0000 7f7f966d3640 1 --2- 172.21.15.52:0/2099627138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f900744e0 0x7f7f90074940 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.607+0000 7f7f966d3640 1 --2- 172.21.15.52:0/2099627138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f90075ac0 0x7f7f90073f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.607+0000 7f7f966d3640 1 -- 172.21.15.52:0/2099627138 >> 172.21.15.52:0/2099627138 conn(0x7f7f900fd0a0 msgr2=0x7f7f900ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:30.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.607+0000 7f7f966d3640 1 -- 172.21.15.52:0/2099627138 shutdown_connections 2024-04-02T13:51:30.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.608+0000 7f7f966d3640 1 -- 172.21.15.52:0/2099627138 wait complete. 2024-04-02T13:51:30.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.608+0000 7f7f966d3640 1 Processor -- start 2024-04-02T13:51:30.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.608+0000 7f7f966d3640 1 -- start start 2024-04-02T13:51:30.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.608+0000 7f7f966d3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f900744e0 0x7f7f9019a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:30.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.608+0000 7f7f966d3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f90075ac0 0x7f7f9019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:30.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.608+0000 7f7f966d3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f90109a00 0x7f7f901a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:30.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.608+0000 7f7f966d3640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9010f9e0 con 0x7f7f90075ac0 2024-04-02T13:51:30.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.608+0000 7f7f966d3640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9010f860 con 0x7f7f900744e0 2024-04-02T13:51:30.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.608+0000 7f7f966d3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9010fb60 con 0x7f7f90109a00 2024-04-02T13:51:30.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.609+0000 7f7f8f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f90075ac0 0x7f7f9019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:30.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.609+0000 7f7f8f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f90075ac0 0x7f7f9019a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53646/0 (socket says 172.21.15.52:53646) 2024-04-02T13:51:30.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.609+0000 7f7f8f7fe640 1 -- 172.21.15.52:0/1784794118 learned_addr learned my addr 172.21.15.52:0/1784794118 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:30.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.609+0000 7f7f8ffff640 1 --2- 172.21.15.52:0/1784794118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f900744e0 0x7f7f9019a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:30.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.609+0000 7f7f94c70640 1 --2- 172.21.15.52:0/1784794118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f90109a00 0x7f7f901a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:30.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.609+0000 7f7f8f7fe640 1 -- 172.21.15.52:0/1784794118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f90109a00 msgr2=0x7f7f901a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:30.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.609+0000 7f7f8f7fe640 1 --2- 172.21.15.52:0/1784794118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f90109a00 0x7f7f901a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.609+0000 7f7f8f7fe640 1 -- 172.21.15.52:0/1784794118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f900744e0 msgr2=0x7f7f9019a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:30.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.609+0000 7f7f8f7fe640 1 --2- 172.21.15.52:0/1784794118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f900744e0 0x7f7f9019a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.609+0000 7f7f8f7fe640 1 -- 172.21.15.52:0/1784794118 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f901a1c50 con 0x7f7f90075ac0 2024-04-02T13:51:30.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.609+0000 7f7f94c70640 1 --2- 172.21.15.52:0/1784794118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f90109a00 0x7f7f901a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:51:30.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.609+0000 7f7f8f7fe640 1 --2- 172.21.15.52:0/1784794118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f90075ac0 0x7f7f9019a570 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7f8402efd0 tx=0x7f7f84005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:30.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.609+0000 7f7f8d7fa640 1 -- 172.21.15.52:0/1784794118 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f84041070 con 0x7f7f90075ac0 2024-04-02T13:51:30.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.609+0000 7f7f8d7fa640 1 -- 172.21.15.52:0/1784794118 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f840044d0 con 0x7f7f90075ac0 2024-04-02T13:51:30.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.609+0000 7f7f8d7fa640 1 -- 172.21.15.52:0/1784794118 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f84034440 con 0x7f7f90075ac0 2024-04-02T13:51:30.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.609+0000 7f7f966d3640 1 -- 172.21.15.52:0/1784794118 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f901a1ee0 con 0x7f7f90075ac0 2024-04-02T13:51:30.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.609+0000 7f7f966d3640 1 -- 172.21.15.52:0/1784794118 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f901a22f0 con 0x7f7f90075ac0 2024-04-02T13:51:30.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.610+0000 7f7f966d3640 1 -- 172.21.15.52:0/1784794118 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f901087c0 con 0x7f7f90075ac0 2024-04-02T13:51:30.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.611+0000 7f7f8d7fa640 1 -- 172.21.15.52:0/1784794118 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7f84044070 con 0x7f7f90075ac0 2024-04-02T13:51:30.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.612+0000 7f7f8d7fa640 1 --2- 172.21.15.52:0/1784794118 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f7f780a3c40 0x7f7f780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:30.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.612+0000 7f7f8ffff640 1 --2- 172.21.15.52:0/1784794118 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f7f780a3c40 0x7f7f780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:30.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.612+0000 7f7f8d7fa640 1 -- 172.21.15.52:0/1784794118 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f7f840f22a0 con 0x7f7f90075ac0 2024-04-02T13:51:30.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.612+0000 7f7f8ffff640 1 --2- 172.21.15.52:0/1784794118 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f7f780a3c40 0x7f7f780a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009bd0 tx=0x7f7f80005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:30.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.616+0000 7f7f8d7fa640 1 -- 172.21.15.52:0/1784794118 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7f840be990 con 0x7f7f90075ac0 2024-04-02T13:51:30.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:30 smithi189 ceph-mon[36683]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:51:30.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:30 smithi189 ceph-mon[36683]: pgmap v307: 129 pgs: 129 active+clean; 582 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-02T13:51:30.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:30 smithi189 ceph-mon[36683]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:51:30.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:30 smithi189 ceph-mon[36683]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-02T13:51:30.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:30 smithi189 ceph-mon[36683]: from='client.24692 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:51:30.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:30 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/280229263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:51:30.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.800+0000 7f7f966d3640 1 -- 172.21.15.52:0/1784794118 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7f90061f40 con 0x7f7f780a3c40 2024-04-02T13:51:30.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.810+0000 7f7f8d7fa640 1 -- 172.21.15.52:0/1784794118 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340348 (secure 0 0 0) 0x7f7f90061f40 con 0x7f7f780a3c40 2024-04-02T13:51:30.817 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:51:30.819 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-04-02T13:51:30.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.814+0000 7f7f966d3640 1 -- 172.21.15.52:0/1784794118 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f7f780a3c40 msgr2=0x7f7f780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:30.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.814+0000 7f7f966d3640 1 --2- 172.21.15.52:0/1784794118 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f7f780a3c40 0x7f7f780a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009bd0 tx=0x7f7f80005f90 comp rx=0 tx=0).stop 2024-04-02T13:51:30.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.814+0000 7f7f966d3640 1 -- 172.21.15.52:0/1784794118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f90075ac0 msgr2=0x7f7f9019a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:30.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.814+0000 7f7f966d3640 1 --2- 172.21.15.52:0/1784794118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f90075ac0 0x7f7f9019a570 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7f8402efd0 tx=0x7f7f84005d50 comp rx=0 tx=0).stop 2024-04-02T13:51:30.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.814+0000 7f7f966d3640 1 -- 172.21.15.52:0/1784794118 shutdown_connections 2024-04-02T13:51:30.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.814+0000 7f7f966d3640 1 --2- 172.21.15.52:0/1784794118 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f7f780a3c40 0x7f7f780a6100 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.814+0000 7f7f966d3640 1 --2- 172.21.15.52:0/1784794118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f90109a00 0x7f7f901a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.814+0000 7f7f966d3640 1 --2- 172.21.15.52:0/1784794118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f90075ac0 0x7f7f9019a570 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.814+0000 7f7f966d3640 1 --2- 172.21.15.52:0/1784794118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f900744e0 0x7f7f9019a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.814+0000 7f7f966d3640 1 -- 172.21.15.52:0/1784794118 >> 172.21.15.52:0/1784794118 conn(0x7f7f900fd0a0 msgr2=0x7f7f90105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:30.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.814+0000 7f7f966d3640 1 -- 172.21.15.52:0/1784794118 shutdown_connections 2024-04-02T13:51:30.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.814+0000 7f7f966d3640 1 -- 172.21.15.52:0/1784794118 wait complete. 2024-04-02T13:51:30.828 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-04-02T13:51:30.740660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636307,"num_objects":80,"num_object_clones":0,"num_object_copies":241,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":165,"num_read_kb":164,"num_write":359,"num_write_kb":3035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":294,"ondisk_log_size":294,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404500,"kb_used_data":13736,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537324,"statfs":{"total":1151940427776,"available":1151526219776,"internally_reserved":0,"allocated":14065664,"data_stored":7650357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":400012602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":41440,"num_objects":56,"num_object_clones":0,"num_object_copies":169,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":29,"num_read_kb":45,"num_write":131,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003325"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.698018+0000","last_change":"2024-04-02T13:51:01.253563+0000","last_active":"2024-04-02T13:51:06.698018+0000","last_peered":"2024-04-02T13:51:06.698018+0000","last_clean":"2024-04-02T13:51:06.698018+0000","last_became_active":"2024-04-02T13:51:01.253185+0000","last_became_peered":"2024-04-02T13:51:01.253185+0000","last_unstale":"2024-04-02T13:51:06.698018+0000","last_undegraded":"2024-04-02T13:51:06.698018+0000","last_fullsized":"2024-04-02T13:51:06.698018+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:51:05.301880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617002+0000","last_change":"2024-04-02T13:51:01.256406+0000","last_active":"2024-04-02T13:51:06.617002+0000","last_peered":"2024-04-02T13:51:06.617002+0000","last_clean":"2024-04-02T13:51:06.617002+0000","last_became_active":"2024-04-02T13:51:01.256068+0000","last_became_peered":"2024-04-02T13:51:01.256068+0000","last_unstale":"2024-04-02T13:51:06.617002+0000","last_undegraded":"2024-04-02T13:51:06.617002+0000","last_fullsized":"2024-04-02T13:51:06.617002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:56:51.184932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617399+0000","last_change":"2024-04-02T13:51:01.265925+0000","last_active":"2024-04-02T13:51:06.617399+0000","last_peered":"2024-04-02T13:51:06.617399+0000","last_clean":"2024-04-02T13:51:06.617399+0000","last_became_active":"2024-04-02T13:51:01.265587+0000","last_became_peered":"2024-04-02T13:51:01.265587+0000","last_unstale":"2024-04-02T13:51:06.617399+0000","last_undegraded":"2024-04-02T13:51:06.617399+0000","last_fullsized":"2024-04-02T13:51:06.617399+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:40:59.426886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.618392+0000","last_change":"2024-04-02T13:51:01.251521+0000","last_active":"2024-04-02T13:51:06.618392+0000","last_peered":"2024-04-02T13:51:06.618392+0000","last_clean":"2024-04-02T13:51:06.618392+0000","last_became_active":"2024-04-02T13:51:01.251130+0000","last_became_peered":"2024-04-02T13:51:01.251130+0000","last_unstale":"2024-04-02T13:51:06.618392+0000","last_undegraded":"2024-04-02T13:51:06.618392+0000","last_fullsized":"2024-04-02T13:51:06.618392+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:04:16.836381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616240+0000","last_change":"2024-04-02T13:51:01.250934+0000","last_active":"2024-04-02T13:51:06.616240+0000","last_peered":"2024-04-02T13:51:06.616240+0000","last_clean":"2024-04-02T13:51:06.616240+0000","last_became_active":"2024-04-02T13:51:01.250546+0000","last_became_peered":"2024-04-02T13:51:01.250546+0000","last_unstale":"2024-04-02T13:51:06.616240+0000","last_undegraded":"2024-04-02T13:51:06.616240+0000","last_fullsized":"2024-04-02T13:51:06.616240+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:34:17.839207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617366+0000","last_change":"2024-04-02T13:51:01.256160+0000","last_active":"2024-04-02T13:51:06.617366+0000","last_peered":"2024-04-02T13:51:06.617366+0000","last_clean":"2024-04-02T13:51:06.617366+0000","last_became_active":"2024-04-02T13:51:01.255776+0000","last_became_peered":"2024-04-02T13:51:01.255776+0000","last_unstale":"2024-04-02T13:51:06.617366+0000","last_undegraded":"2024-04-02T13:51:06.617366+0000","last_fullsized":"2024-04-02T13:51:06.617366+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:53:16.724662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616681+0000","last_change":"2024-04-02T13:51:01.263049+0000","last_active":"2024-04-02T13:51:06.616681+0000","last_peered":"2024-04-02T13:51:06.616681+0000","last_clean":"2024-04-02T13:51:06.616681+0000","last_became_active":"2024-04-02T13:51:01.262660+0000","last_became_peered":"2024-04-02T13:51:01.262660+0000","last_unstale":"2024-04-02T13:51:06.616681+0000","last_undegraded":"2024-04-02T13:51:06.616681+0000","last_fullsized":"2024-04-02T13:51:06.616681+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:15:32.069188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616185+0000","last_change":"2024-04-02T13:51:01.263064+0000","last_active":"2024-04-02T13:51:06.616185+0000","last_peered":"2024-04-02T13:51:06.616185+0000","last_clean":"2024-04-02T13:51:06.616185+0000","last_became_active":"2024-04-02T13:51:01.262706+0000","last_became_peered":"2024-04-02T13:51:01.262706+0000","last_unstale":"2024-04-02T13:51:06.616185+0000","last_undegraded":"2024-04-02T13:51:06.616185+0000","last_fullsized":"2024-04-02T13:51:06.616185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:46.680723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.349914+0000","last_change":"2024-04-02T13:51:01.257347+0000","last_active":"2024-04-02T13:51:06.349914+0000","last_peered":"2024-04-02T13:51:06.349914+0000","last_clean":"2024-04-02T13:51:06.349914+0000","last_became_active":"2024-04-02T13:51:01.257001+0000","last_became_peered":"2024-04-02T13:51:01.257001+0000","last_unstale":"2024-04-02T13:51:06.349914+0000","last_undegraded":"2024-04-02T13:51:06.349914+0000","last_fullsized":"2024-04-02T13:51:06.349914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:31:22.628436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617067+0000","last_change":"2024-04-02T13:51:01.260533+0000","last_active":"2024-04-02T13:51:06.617067+0000","last_peered":"2024-04-02T13:51:06.617067+0000","last_clean":"2024-04-02T13:51:06.617067+0000","last_became_active":"2024-04-02T13:51:01.260203+0000","last_became_peered":"2024-04-02T13:51:01.260203+0000","last_unstale":"2024-04-02T13:51:06.617067+0000","last_undegraded":"2024-04-02T13:51:06.617067+0000","last_fullsized":"2024-04-02T13:51:06.617067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:00:06.008243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617658+0000","last_change":"2024-04-02T13:51:01.267733+0000","last_active":"2024-04-02T13:51:06.617658+0000","last_peered":"2024-04-02T13:51:06.617658+0000","last_clean":"2024-04-02T13:51:06.617658+0000","last_became_active":"2024-04-02T13:51:01.266642+0000","last_became_peered":"2024-04-02T13:51:01.266642+0000","last_unstale":"2024-04-02T13:51:06.617658+0000","last_undegraded":"2024-04-02T13:51:06.617658+0000","last_fullsized":"2024-04-02T13:51:06.617658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:39:25.696112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.618369+0000","last_change":"2024-04-02T13:51:01.255933+0000","last_active":"2024-04-02T13:51:06.618369+0000","last_peered":"2024-04-02T13:51:06.618369+0000","last_clean":"2024-04-02T13:51:06.618369+0000","last_became_active":"2024-04-02T13:51:01.255558+0000","last_became_peered":"2024-04-02T13:51:01.255558+0000","last_unstale":"2024-04-02T13:51:06.618369+0000","last_undegraded":"2024-04-02T13:51:06.618369+0000","last_fullsized":"2024-04-02T13:51:06.618369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:13:11.537415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347761+0000","last_change":"2024-04-02T13:51:01.256242+0000","last_active":"2024-04-02T13:51:06.347761+0000","last_peered":"2024-04-02T13:51:06.347761+0000","last_clean":"2024-04-02T13:51:06.347761+0000","last_became_active":"2024-04-02T13:51:01.255830+0000","last_became_peered":"2024-04-02T13:51:01.255830+0000","last_unstale":"2024-04-02T13:51:06.347761+0000","last_undegraded":"2024-04-02T13:51:06.347761+0000","last_fullsized":"2024-04-02T13:51:06.347761+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:39.130894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616082+0000","last_change":"2024-04-02T13:51:01.259738+0000","last_active":"2024-04-02T13:51:06.616082+0000","last_peered":"2024-04-02T13:51:06.616082+0000","last_clean":"2024-04-02T13:51:06.616082+0000","last_became_active":"2024-04-02T13:51:01.258630+0000","last_became_peered":"2024-04-02T13:51:01.258630+0000","last_unstale":"2024-04-02T13:51:06.616082+0000","last_undegraded":"2024-04-02T13:51:06.616082+0000","last_fullsized":"2024-04-02T13:51:06.616082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:12:05.401171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616233+0000","last_change":"2024-04-02T13:51:01.257684+0000","last_active":"2024-04-02T13:51:06.616233+0000","last_peered":"2024-04-02T13:51:06.616233+0000","last_clean":"2024-04-02T13:51:06.616233+0000","last_became_active":"2024-04-02T13:51:01.257267+0000","last_became_peered":"2024-04-02T13:51:01.257267+0000","last_unstale":"2024-04-02T13:51:06.616233+0000","last_undegraded":"2024-04-02T13:51:06.616233+0000","last_fullsized":"2024-04-02T13:51:06.616233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:43:45.728233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.349177+0000","last_change":"2024-04-02T13:51:01.261720+0000","last_active":"2024-04-02T13:51:06.349177+0000","last_peered":"2024-04-02T13:51:06.349177+0000","last_clean":"2024-04-02T13:51:06.349177+0000","last_became_active":"2024-04-02T13:51:01.261407+0000","last_became_peered":"2024-04-02T13:51:01.261407+0000","last_unstale":"2024-04-02T13:51:06.349177+0000","last_undegraded":"2024-04-02T13:51:06.349177+0000","last_fullsized":"2024-04-02T13:51:06.349177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:54:58.807606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.697829+0000","last_change":"2024-04-02T13:51:01.254488+0000","last_active":"2024-04-02T13:51:06.697829+0000","last_peered":"2024-04-02T13:51:06.697829+0000","last_clean":"2024-04-02T13:51:06.697829+0000","last_became_active":"2024-04-02T13:51:01.254139+0000","last_became_peered":"2024-04-02T13:51:01.254139+0000","last_unstale":"2024-04-02T13:51:06.697829+0000","last_undegraded":"2024-04-02T13:51:06.697829+0000","last_fullsized":"2024-04-02T13:51:06.697829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:02:41.075727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616657+0000","last_change":"2024-04-02T13:51:01.256107+0000","last_active":"2024-04-02T13:51:06.616657+0000","last_peered":"2024-04-02T13:51:06.616657+0000","last_clean":"2024-04-02T13:51:06.616657+0000","last_became_active":"2024-04-02T13:51:01.255754+0000","last_became_peered":"2024-04-02T13:51:01.255754+0000","last_unstale":"2024-04-02T13:51:06.616657+0000","last_undegraded":"2024-04-02T13:51:06.616657+0000","last_fullsized":"2024-04-02T13:51:06.616657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:36:59.793519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.354581+0000","last_change":"2024-04-02T13:51:01.247550+0000","last_active":"2024-04-02T13:51:06.354581+0000","last_peered":"2024-04-02T13:51:06.354581+0000","last_clean":"2024-04-02T13:51:06.354581+0000","last_became_active":"2024-04-02T13:51:01.247236+0000","last_became_peered":"2024-04-02T13:51:01.247236+0000","last_unstale":"2024-04-02T13:51:06.354581+0000","last_undegraded":"2024-04-02T13:51:06.354581+0000","last_fullsized":"2024-04-02T13:51:06.354581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:16:02.940700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.354234+0000","last_change":"2024-04-02T13:51:05.365182+0000","last_active":"2024-04-02T13:51:06.354234+0000","last_peered":"2024-04-02T13:51:06.354234+0000","last_clean":"2024-04-02T13:51:06.354234+0000","last_became_active":"2024-04-02T13:51:05.364713+0000","last_became_peered":"2024-04-02T13:51:05.364713+0000","last_unstale":"2024-04-02T13:51:06.354234+0000","last_undegraded":"2024-04-02T13:51:06.354234+0000","last_fullsized":"2024-04-02T13:51:06.354234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:14:34.763192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'11","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:27.345883+0000","last_change":"2024-04-02T13:51:00.243244+0000","last_active":"2024-04-02T13:51:27.345883+0000","last_peered":"2024-04-02T13:51:27.345883+0000","last_clean":"2024-04-02T13:51:27.345883+0000","last_became_active":"2024-04-02T13:51:00.242892+0000","last_became_peered":"2024-04-02T13:51:00.242892+0000","last_unstale":"2024-04-02T13:51:27.345883+0000","last_undegraded":"2024-04-02T13:51:27.345883+0000","last_fullsized":"2024-04-02T13:51:27.345883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:49:23.338215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617419+0000","last_change":"2024-04-02T13:51:01.264640+0000","last_active":"2024-04-02T13:51:06.617419+0000","last_peered":"2024-04-02T13:51:06.617419+0000","last_clean":"2024-04-02T13:51:06.617419+0000","last_became_active":"2024-04-02T13:51:01.264161+0000","last_became_peered":"2024-04-02T13:51:01.264161+0000","last_unstale":"2024-04-02T13:51:06.617419+0000","last_undegraded":"2024-04-02T13:51:06.617419+0000","last_fullsized":"2024-04-02T13:51:06.617419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:30:47.245647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617116+0000","last_change":"2024-04-02T13:51:05.362888+0000","last_active":"2024-04-02T13:51:06.617116+0000","last_peered":"2024-04-02T13:51:06.617116+0000","last_clean":"2024-04-02T13:51:06.617116+0000","last_became_active":"2024-04-02T13:51:05.362478+0000","last_became_peered":"2024-04-02T13:51:05.362478+0000","last_unstale":"2024-04-02T13:51:06.617116+0000","last_undegraded":"2024-04-02T13:51:06.617116+0000","last_fullsized":"2024-04-02T13:51:06.617116+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:05:54.304458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617581+0000","last_change":"2024-04-02T13:51:00.247800+0000","last_active":"2024-04-02T13:51:06.617581+0000","last_peered":"2024-04-02T13:51:06.617581+0000","last_clean":"2024-04-02T13:51:06.617581+0000","last_became_active":"2024-04-02T13:51:00.247485+0000","last_became_peered":"2024-04-02T13:51:00.247485+0000","last_unstale":"2024-04-02T13:51:06.617581+0000","last_undegraded":"2024-04-02T13:51:06.617581+0000","last_fullsized":"2024-04-02T13:51:06.617581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:17:24.767558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616088+0000","last_change":"2024-04-02T13:51:01.261416+0000","last_active":"2024-04-02T13:51:06.616088+0000","last_peered":"2024-04-02T13:51:06.616088+0000","last_clean":"2024-04-02T13:51:06.616088+0000","last_became_active":"2024-04-02T13:51:01.261099+0000","last_became_peered":"2024-04-02T13:51:01.261099+0000","last_unstale":"2024-04-02T13:51:06.616088+0000","last_undegraded":"2024-04-02T13:51:06.616088+0000","last_fullsized":"2024-04-02T13:51:06.616088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:10:04.999402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617900+0000","last_change":"2024-04-02T13:51:05.361747+0000","last_active":"2024-04-02T13:51:06.617900+0000","last_peered":"2024-04-02T13:51:06.617900+0000","last_clean":"2024-04-02T13:51:06.617900+0000","last_became_active":"2024-04-02T13:51:05.361098+0000","last_became_peered":"2024-04-02T13:51:05.361098+0000","last_unstale":"2024-04-02T13:51:06.617900+0000","last_undegraded":"2024-04-02T13:51:06.617900+0000","last_fullsized":"2024-04-02T13:51:06.617900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:45:35.048870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:26.336320+0000","last_change":"2024-04-02T13:51:00.244048+0000","last_active":"2024-04-02T13:51:26.336320+0000","last_peered":"2024-04-02T13:51:26.336320+0000","last_clean":"2024-04-02T13:51:26.336320+0000","last_became_active":"2024-04-02T13:51:00.243510+0000","last_became_peered":"2024-04-02T13:51:00.243510+0000","last_unstale":"2024-04-02T13:51:26.336320+0000","last_undegraded":"2024-04-02T13:51:26.336320+0000","last_fullsized":"2024-04-02T13:51:26.336320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:53:48.364137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.349854+0000","last_change":"2024-04-02T13:51:01.264711+0000","last_active":"2024-04-02T13:51:06.349854+0000","last_peered":"2024-04-02T13:51:06.349854+0000","last_clean":"2024-04-02T13:51:06.349854+0000","last_became_active":"2024-04-02T13:51:01.264289+0000","last_became_peered":"2024-04-02T13:51:01.264289+0000","last_unstale":"2024-04-02T13:51:06.349854+0000","last_undegraded":"2024-04-02T13:51:06.349854+0000","last_fullsized":"2024-04-02T13:51:06.349854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:34:41.590071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617854+0000","last_change":"2024-04-02T13:51:05.363121+0000","last_active":"2024-04-02T13:51:06.617854+0000","last_peered":"2024-04-02T13:51:06.617854+0000","last_clean":"2024-04-02T13:51:06.617854+0000","last_became_active":"2024-04-02T13:51:05.362642+0000","last_became_peered":"2024-04-02T13:51:05.362642+0000","last_unstale":"2024-04-02T13:51:06.617854+0000","last_undegraded":"2024-04-02T13:51:06.617854+0000","last_fullsized":"2024-04-02T13:51:06.617854+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:55:40.848731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616416+0000","last_change":"2024-04-02T13:51:00.236519+0000","last_active":"2024-04-02T13:51:06.616416+0000","last_peered":"2024-04-02T13:51:06.616416+0000","last_clean":"2024-04-02T13:51:06.616416+0000","last_became_active":"2024-04-02T13:51:00.235985+0000","last_became_peered":"2024-04-02T13:51:00.235985+0000","last_unstale":"2024-04-02T13:51:06.616416+0000","last_undegraded":"2024-04-02T13:51:06.616416+0000","last_fullsized":"2024-04-02T13:51:06.616416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:08.321765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347657+0000","last_change":"2024-04-02T13:51:01.260333+0000","last_active":"2024-04-02T13:51:06.347657+0000","last_peered":"2024-04-02T13:51:06.347657+0000","last_clean":"2024-04-02T13:51:06.347657+0000","last_became_active":"2024-04-02T13:51:01.259995+0000","last_became_peered":"2024-04-02T13:51:01.259995+0000","last_unstale":"2024-04-02T13:51:06.347657+0000","last_undegraded":"2024-04-02T13:51:06.347657+0000","last_fullsized":"2024-04-02T13:51:06.347657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:03:58.088653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.354511+0000","last_change":"2024-04-02T13:51:05.359281+0000","last_active":"2024-04-02T13:51:06.354511+0000","last_peered":"2024-04-02T13:51:06.354511+0000","last_clean":"2024-04-02T13:51:06.354511+0000","last_became_active":"2024-04-02T13:51:05.358831+0000","last_became_peered":"2024-04-02T13:51:05.358831+0000","last_unstale":"2024-04-02T13:51:06.354511+0000","last_undegraded":"2024-04-02T13:51:06.354511+0000","last_fullsized":"2024-04-02T13:51:06.354511+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:48.461298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:29.355832+0000","last_change":"2024-04-02T13:51:00.241707+0000","last_active":"2024-04-02T13:51:29.355832+0000","last_peered":"2024-04-02T13:51:29.355832+0000","last_clean":"2024-04-02T13:51:29.355832+0000","last_became_active":"2024-04-02T13:51:00.241225+0000","last_became_peered":"2024-04-02T13:51:00.241225+0000","last_unstale":"2024-04-02T13:51:29.355832+0000","last_undegraded":"2024-04-02T13:51:29.355832+0000","last_fullsized":"2024-04-02T13:51:29.355832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:59:44.537503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616279+0000","last_change":"2024-04-02T13:51:01.252942+0000","last_active":"2024-04-02T13:51:06.616279+0000","last_peered":"2024-04-02T13:51:06.616279+0000","last_clean":"2024-04-02T13:51:06.616279+0000","last_became_active":"2024-04-02T13:51:01.252631+0000","last_became_peered":"2024-04-02T13:51:01.252631+0000","last_unstale":"2024-04-02T13:51:06.616279+0000","last_undegraded":"2024-04-02T13:51:06.616279+0000","last_fullsized":"2024-04-02T13:51:06.616279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:43:24.725688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.354285+0000","last_change":"2024-04-02T13:51:05.359208+0000","last_active":"2024-04-02T13:51:06.354285+0000","last_peered":"2024-04-02T13:51:06.354285+0000","last_clean":"2024-04-02T13:51:06.354285+0000","last_became_active":"2024-04-02T13:51:05.358738+0000","last_became_peered":"2024-04-02T13:51:05.358738+0000","last_unstale":"2024-04-02T13:51:06.354285+0000","last_undegraded":"2024-04-02T13:51:06.354285+0000","last_fullsized":"2024-04-02T13:51:06.354285+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:47:46.034562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:27.343299+0000","last_change":"2024-04-02T13:51:00.236413+0000","last_active":"2024-04-02T13:51:27.343299+0000","last_peered":"2024-04-02T13:51:27.343299+0000","last_clean":"2024-04-02T13:51:27.343299+0000","last_became_active":"2024-04-02T13:51:00.236046+0000","last_became_peered":"2024-04-02T13:51:00.236046+0000","last_unstale":"2024-04-02T13:51:27.343299+0000","last_undegraded":"2024-04-02T13:51:27.343299+0000","last_fullsized":"2024-04-02T13:51:27.343299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:25:15.000726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347640+0000","last_change":"2024-04-02T13:51:01.265471+0000","last_active":"2024-04-02T13:51:06.347640+0000","last_peered":"2024-04-02T13:51:06.347640+0000","last_clean":"2024-04-02T13:51:06.347640+0000","last_became_active":"2024-04-02T13:51:01.265114+0000","last_became_peered":"2024-04-02T13:51:01.265114+0000","last_unstale":"2024-04-02T13:51:06.347640+0000","last_undegraded":"2024-04-02T13:51:06.347640+0000","last_fullsized":"2024-04-02T13:51:06.347640+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:01:01.675686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.697886+0000","last_change":"2024-04-02T13:51:05.366813+0000","last_active":"2024-04-02T13:51:06.697886+0000","last_peered":"2024-04-02T13:51:06.697886+0000","last_clean":"2024-04-02T13:51:06.697886+0000","last_became_active":"2024-04-02T13:51:05.366317+0000","last_became_peered":"2024-04-02T13:51:05.366317+0000","last_unstale":"2024-04-02T13:51:06.697886+0000","last_undegraded":"2024-04-02T13:51:06.697886+0000","last_fullsized":"2024-04-02T13:51:06.697886+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:03:54.156620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347405+0000","last_change":"2024-04-02T13:51:00.237204+0000","last_active":"2024-04-02T13:51:06.347405+0000","last_peered":"2024-04-02T13:51:06.347405+0000","last_clean":"2024-04-02T13:51:06.347405+0000","last_became_active":"2024-04-02T13:51:00.236707+0000","last_became_peered":"2024-04-02T13:51:00.236707+0000","last_unstale":"2024-04-02T13:51:06.347405+0000","last_undegraded":"2024-04-02T13:51:06.347405+0000","last_fullsized":"2024-04-02T13:51:06.347405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:00:17.771428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616471+0000","last_change":"2024-04-02T13:51:01.250152+0000","last_active":"2024-04-02T13:51:06.616471+0000","last_peered":"2024-04-02T13:51:06.616471+0000","last_clean":"2024-04-02T13:51:06.616471+0000","last_became_active":"2024-04-02T13:51:01.249817+0000","last_became_peered":"2024-04-02T13:51:01.249817+0000","last_unstale":"2024-04-02T13:51:06.616471+0000","last_undegraded":"2024-04-02T13:51:06.616471+0000","last_fullsized":"2024-04-02T13:51:06.616471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:13:31.719380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617106+0000","last_change":"2024-04-02T13:51:05.366551+0000","last_active":"2024-04-02T13:51:06.617106+0000","last_peered":"2024-04-02T13:51:06.617106+0000","last_clean":"2024-04-02T13:51:06.617106+0000","last_became_active":"2024-04-02T13:51:05.366175+0000","last_became_peered":"2024-04-02T13:51:05.366175+0000","last_unstale":"2024-04-02T13:51:06.617106+0000","last_undegraded":"2024-04-02T13:51:06.617106+0000","last_fullsized":"2024-04-02T13:51:06.617106+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:47:49.822918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:28.352356+0000","last_change":"2024-04-02T13:51:00.239521+0000","last_active":"2024-04-02T13:51:28.352356+0000","last_peered":"2024-04-02T13:51:28.352356+0000","last_clean":"2024-04-02T13:51:28.352356+0000","last_became_active":"2024-04-02T13:51:00.238608+0000","last_became_peered":"2024-04-02T13:51:00.238608+0000","last_unstale":"2024-04-02T13:51:28.352356+0000","last_undegraded":"2024-04-02T13:51:28.352356+0000","last_fullsized":"2024-04-02T13:51:28.352356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:00.983006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617394+0000","last_change":"2024-04-02T13:51:01.255021+0000","last_active":"2024-04-02T13:51:06.617394+0000","last_peered":"2024-04-02T13:51:06.617394+0000","last_clean":"2024-04-02T13:51:06.617394+0000","last_became_active":"2024-04-02T13:51:01.254630+0000","last_became_peered":"2024-04-02T13:51:01.254630+0000","last_unstale":"2024-04-02T13:51:06.617394+0000","last_undegraded":"2024-04-02T13:51:06.617394+0000","last_fullsized":"2024-04-02T13:51:06.617394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:36:08.773431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616756+0000","last_change":"2024-04-02T13:51:05.362004+0000","last_active":"2024-04-02T13:51:06.616756+0000","last_peered":"2024-04-02T13:51:06.616756+0000","last_clean":"2024-04-02T13:51:06.616756+0000","last_became_active":"2024-04-02T13:51:05.361538+0000","last_became_peered":"2024-04-02T13:51:05.361538+0000","last_unstale":"2024-04-02T13:51:06.616756+0000","last_undegraded":"2024-04-02T13:51:06.616756+0000","last_fullsized":"2024-04-02T13:51:06.616756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:27:21.728695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:29.352274+0000","last_change":"2024-04-02T13:51:00.234814+0000","last_active":"2024-04-02T13:51:29.352274+0000","last_peered":"2024-04-02T13:51:29.352274+0000","last_clean":"2024-04-02T13:51:29.352274+0000","last_became_active":"2024-04-02T13:51:00.234481+0000","last_became_peered":"2024-04-02T13:51:00.234481+0000","last_unstale":"2024-04-02T13:51:29.352274+0000","last_undegraded":"2024-04-02T13:51:29.352274+0000","last_fullsized":"2024-04-02T13:51:29.352274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:45:34.901226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.349799+0000","last_change":"2024-04-02T13:51:01.264093+0000","last_active":"2024-04-02T13:51:06.349799+0000","last_peered":"2024-04-02T13:51:06.349799+0000","last_clean":"2024-04-02T13:51:06.349799+0000","last_became_active":"2024-04-02T13:51:01.263754+0000","last_became_peered":"2024-04-02T13:51:01.263754+0000","last_unstale":"2024-04-02T13:51:06.349799+0000","last_undegraded":"2024-04-02T13:51:06.349799+0000","last_fullsized":"2024-04-02T13:51:06.349799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:49:18.630195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617911+0000","last_change":"2024-04-02T13:51:05.370930+0000","last_active":"2024-04-02T13:51:06.617911+0000","last_peered":"2024-04-02T13:51:06.617911+0000","last_clean":"2024-04-02T13:51:06.617911+0000","last_became_active":"2024-04-02T13:51:05.370443+0000","last_became_peered":"2024-04-02T13:51:05.370443+0000","last_unstale":"2024-04-02T13:51:06.617911+0000","last_undegraded":"2024-04-02T13:51:06.617911+0000","last_fullsized":"2024-04-02T13:51:06.617911+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:24:27.055494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:25.332528+0000","last_change":"2024-04-02T13:51:00.241206+0000","last_active":"2024-04-02T13:51:25.332528+0000","last_peered":"2024-04-02T13:51:25.332528+0000","last_clean":"2024-04-02T13:51:25.332528+0000","last_became_active":"2024-04-02T13:51:00.240832+0000","last_became_peered":"2024-04-02T13:51:00.240832+0000","last_unstale":"2024-04-02T13:51:25.332528+0000","last_undegraded":"2024-04-02T13:51:25.332528+0000","last_fullsized":"2024-04-02T13:51:25.332528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:44:11.626435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617673+0000","last_change":"2024-04-02T13:51:01.259315+0000","last_active":"2024-04-02T13:51:06.617673+0000","last_peered":"2024-04-02T13:51:06.617673+0000","last_clean":"2024-04-02T13:51:06.617673+0000","last_became_active":"2024-04-02T13:51:01.258951+0000","last_became_peered":"2024-04-02T13:51:01.258951+0000","last_unstale":"2024-04-02T13:51:06.617673+0000","last_undegraded":"2024-04-02T13:51:06.617673+0000","last_fullsized":"2024-04-02T13:51:06.617673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:59:43.714044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.618115+0000","last_change":"2024-04-02T13:51:05.361495+0000","last_active":"2024-04-02T13:51:06.618115+0000","last_peered":"2024-04-02T13:51:06.618115+0000","last_clean":"2024-04-02T13:51:06.618115+0000","last_became_active":"2024-04-02T13:51:05.360546+0000","last_became_peered":"2024-04-02T13:51:05.360546+0000","last_unstale":"2024-04-02T13:51:06.618115+0000","last_undegraded":"2024-04-02T13:51:06.618115+0000","last_fullsized":"2024-04-02T13:51:06.618115+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:23:59.853497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:27.341944+0000","last_change":"2024-04-02T13:51:00.230938+0000","last_active":"2024-04-02T13:51:27.341944+0000","last_peered":"2024-04-02T13:51:27.341944+0000","last_clean":"2024-04-02T13:51:27.341944+0000","last_became_active":"2024-04-02T13:51:00.230585+0000","last_became_peered":"2024-04-02T13:51:00.230585+0000","last_unstale":"2024-04-02T13:51:27.341944+0000","last_undegraded":"2024-04-02T13:51:27.341944+0000","last_fullsized":"2024-04-02T13:51:27.341944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:51:37.463876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.350041+0000","last_change":"2024-04-02T13:51:01.264216+0000","last_active":"2024-04-02T13:51:06.350041+0000","last_peered":"2024-04-02T13:51:06.350041+0000","last_clean":"2024-04-02T13:51:06.350041+0000","last_became_active":"2024-04-02T13:51:01.263824+0000","last_became_peered":"2024-04-02T13:51:01.263824+0000","last_unstale":"2024-04-02T13:51:06.350041+0000","last_undegraded":"2024-04-02T13:51:06.350041+0000","last_fullsized":"2024-04-02T13:51:06.350041+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:16:26.391163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617882+0000","last_change":"2024-04-02T13:51:05.366002+0000","last_active":"2024-04-02T13:51:06.617882+0000","last_peered":"2024-04-02T13:51:06.617882+0000","last_clean":"2024-04-02T13:51:06.617882+0000","last_became_active":"2024-04-02T13:51:05.365547+0000","last_became_peered":"2024-04-02T13:51:05.365547+0000","last_unstale":"2024-04-02T13:51:06.617882+0000","last_undegraded":"2024-04-02T13:51:06.617882+0000","last_fullsized":"2024-04-02T13:51:06.617882+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:31:38.990617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'25","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:29.460868+0000","last_change":"2024-04-02T13:51:00.233925+0000","last_active":"2024-04-02T13:51:29.460868+0000","last_peered":"2024-04-02T13:51:29.460868+0000","last_clean":"2024-04-02T13:51:29.460868+0000","last_became_active":"2024-04-02T13:51:00.233574+0000","last_became_peered":"2024-04-02T13:51:00.233574+0000","last_unstale":"2024-04-02T13:51:29.460868+0000","last_undegraded":"2024-04-02T13:51:29.460868+0000","last_fullsized":"2024-04-02T13:51:29.460868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:33:32.758874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40471,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":24,"num_write":27,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616529+0000","last_change":"2024-04-02T13:51:01.258825+0000","last_active":"2024-04-02T13:51:06.616529+0000","last_peered":"2024-04-02T13:51:06.616529+0000","last_clean":"2024-04-02T13:51:06.616529+0000","last_became_active":"2024-04-02T13:51:01.258500+0000","last_became_peered":"2024-04-02T13:51:01.258500+0000","last_unstale":"2024-04-02T13:51:06.616529+0000","last_undegraded":"2024-04-02T13:51:06.616529+0000","last_fullsized":"2024-04-02T13:51:06.616529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:33:15.409765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.348101+0000","last_change":"2024-04-02T13:51:05.365010+0000","last_active":"2024-04-02T13:51:06.348101+0000","last_peered":"2024-04-02T13:51:06.348101+0000","last_clean":"2024-04-02T13:51:06.348101+0000","last_became_active":"2024-04-02T13:51:05.364499+0000","last_became_peered":"2024-04-02T13:51:05.364499+0000","last_unstale":"2024-04-02T13:51:06.348101+0000","last_undegraded":"2024-04-02T13:51:06.348101+0000","last_fullsized":"2024-04-02T13:51:06.348101+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:07:03.500017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:29.351762+0000","last_change":"2024-04-02T13:51:00.233459+0000","last_active":"2024-04-02T13:51:29.351762+0000","last_peered":"2024-04-02T13:51:29.351762+0000","last_clean":"2024-04-02T13:51:29.351762+0000","last_became_active":"2024-04-02T13:51:00.232424+0000","last_became_peered":"2024-04-02T13:51:00.232424+0000","last_unstale":"2024-04-02T13:51:29.351762+0000","last_undegraded":"2024-04-02T13:51:29.351762+0000","last_fullsized":"2024-04-02T13:51:29.351762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:26:20.877553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347581+0000","last_change":"2024-04-02T13:51:01.256334+0000","last_active":"2024-04-02T13:51:06.347581+0000","last_peered":"2024-04-02T13:51:06.347581+0000","last_clean":"2024-04-02T13:51:06.347581+0000","last_became_active":"2024-04-02T13:51:01.255964+0000","last_became_peered":"2024-04-02T13:51:01.255964+0000","last_unstale":"2024-04-02T13:51:06.347581+0000","last_undegraded":"2024-04-02T13:51:06.347581+0000","last_fullsized":"2024-04-02T13:51:06.347581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:43:04.842812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.697541+0000","last_change":"2024-04-02T13:51:05.362696+0000","last_active":"2024-04-02T13:51:06.697541+0000","last_peered":"2024-04-02T13:51:06.697541+0000","last_clean":"2024-04-02T13:51:06.697541+0000","last_became_active":"2024-04-02T13:51:05.362225+0000","last_became_peered":"2024-04-02T13:51:05.362225+0000","last_unstale":"2024-04-02T13:51:06.697541+0000","last_undegraded":"2024-04-02T13:51:06.697541+0000","last_fullsized":"2024-04-02T13:51:06.697541+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:23:56.308286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:28.350739+0000","last_change":"2024-04-02T13:51:00.239673+0000","last_active":"2024-04-02T13:51:28.350739+0000","last_peered":"2024-04-02T13:51:28.350739+0000","last_clean":"2024-04-02T13:51:28.350739+0000","last_became_active":"2024-04-02T13:51:00.239233+0000","last_became_peered":"2024-04-02T13:51:00.239233+0000","last_unstale":"2024-04-02T13:51:28.350739+0000","last_undegraded":"2024-04-02T13:51:28.350739+0000","last_fullsized":"2024-04-02T13:51:28.350739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:05:48.119953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617153+0000","last_change":"2024-04-02T13:51:01.257474+0000","last_active":"2024-04-02T13:51:06.617153+0000","last_peered":"2024-04-02T13:51:06.617153+0000","last_clean":"2024-04-02T13:51:06.617153+0000","last_became_active":"2024-04-02T13:51:01.254977+0000","last_became_peered":"2024-04-02T13:51:01.254977+0000","last_unstale":"2024-04-02T13:51:06.617153+0000","last_undegraded":"2024-04-02T13:51:06.617153+0000","last_fullsized":"2024-04-02T13:51:06.617153+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:13:07.446520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617920+0000","last_change":"2024-04-02T13:51:05.371531+0000","last_active":"2024-04-02T13:51:06.617920+0000","last_peered":"2024-04-02T13:51:06.617920+0000","last_clean":"2024-04-02T13:51:06.617920+0000","last_became_active":"2024-04-02T13:51:05.370551+0000","last_became_peered":"2024-04-02T13:51:05.370551+0000","last_unstale":"2024-04-02T13:51:06.617920+0000","last_undegraded":"2024-04-02T13:51:06.617920+0000","last_fullsized":"2024-04-02T13:51:06.617920+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:43:48.796009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:29.353719+0000","last_change":"2024-04-02T13:51:00.237715+0000","last_active":"2024-04-02T13:51:29.353719+0000","last_peered":"2024-04-02T13:51:29.353719+0000","last_clean":"2024-04-02T13:51:29.353719+0000","last_became_active":"2024-04-02T13:51:00.237349+0000","last_became_peered":"2024-04-02T13:51:00.237349+0000","last_unstale":"2024-04-02T13:51:29.353719+0000","last_undegraded":"2024-04-02T13:51:29.353719+0000","last_fullsized":"2024-04-02T13:51:29.353719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:33:20.651562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616896+0000","last_change":"2024-04-02T13:51:01.259447+0000","last_active":"2024-04-02T13:51:06.616896+0000","last_peered":"2024-04-02T13:51:06.616896+0000","last_clean":"2024-04-02T13:51:06.616896+0000","last_became_active":"2024-04-02T13:51:01.259101+0000","last_became_peered":"2024-04-02T13:51:01.259101+0000","last_unstale":"2024-04-02T13:51:06.616896+0000","last_undegraded":"2024-04-02T13:51:06.616896+0000","last_fullsized":"2024-04-02T13:51:06.616896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:51:44.311440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.348037+0000","last_change":"2024-04-02T13:51:05.372446+0000","last_active":"2024-04-02T13:51:06.348037+0000","last_peered":"2024-04-02T13:51:06.348037+0000","last_clean":"2024-04-02T13:51:06.348037+0000","last_became_active":"2024-04-02T13:51:05.372007+0000","last_became_peered":"2024-04-02T13:51:05.372007+0000","last_unstale":"2024-04-02T13:51:06.348037+0000","last_undegraded":"2024-04-02T13:51:06.348037+0000","last_fullsized":"2024-04-02T13:51:06.348037+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:54:23.463179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:26.338319+0000","last_change":"2024-04-02T13:51:00.241492+0000","last_active":"2024-04-02T13:51:26.338319+0000","last_peered":"2024-04-02T13:51:26.338319+0000","last_clean":"2024-04-02T13:51:26.338319+0000","last_became_active":"2024-04-02T13:51:00.241167+0000","last_became_peered":"2024-04-02T13:51:00.241167+0000","last_unstale":"2024-04-02T13:51:26.338319+0000","last_undegraded":"2024-04-02T13:51:26.338319+0000","last_fullsized":"2024-04-02T13:51:26.338319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:44:44.070187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347487+0000","last_change":"2024-04-02T13:51:01.263200+0000","last_active":"2024-04-02T13:51:06.347487+0000","last_peered":"2024-04-02T13:51:06.347487+0000","last_clean":"2024-04-02T13:51:06.347487+0000","last_became_active":"2024-04-02T13:51:01.262812+0000","last_became_peered":"2024-04-02T13:51:01.262812+0000","last_unstale":"2024-04-02T13:51:06.347487+0000","last_undegraded":"2024-04-02T13:51:06.347487+0000","last_fullsized":"2024-04-02T13:51:06.347487+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:33:35.898047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347331+0000","last_change":"2024-04-02T13:51:05.366729+0000","last_active":"2024-04-02T13:51:06.347331+0000","last_peered":"2024-04-02T13:51:06.347331+0000","last_clean":"2024-04-02T13:51:06.347331+0000","last_became_active":"2024-04-02T13:51:05.366257+0000","last_became_peered":"2024-04-02T13:51:05.366257+0000","last_unstale":"2024-04-02T13:51:06.347331+0000","last_undegraded":"2024-04-02T13:51:06.347331+0000","last_fullsized":"2024-04-02T13:51:06.347331+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:46:03.400300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616677+0000","last_change":"2024-04-02T13:51:00.236004+0000","last_active":"2024-04-02T13:51:06.616677+0000","last_peered":"2024-04-02T13:51:06.616677+0000","last_clean":"2024-04-02T13:51:06.616677+0000","last_became_active":"2024-04-02T13:51:00.235160+0000","last_became_peered":"2024-04-02T13:51:00.235160+0000","last_unstale":"2024-04-02T13:51:06.616677+0000","last_undegraded":"2024-04-02T13:51:06.616677+0000","last_fullsized":"2024-04-02T13:51:06.616677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:16:35.617346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.697743+0000","last_change":"2024-04-02T13:51:01.259565+0000","last_active":"2024-04-02T13:51:06.697743+0000","last_peered":"2024-04-02T13:51:06.697743+0000","last_clean":"2024-04-02T13:51:06.697743+0000","last_became_active":"2024-04-02T13:51:01.259225+0000","last_became_peered":"2024-04-02T13:51:01.259225+0000","last_unstale":"2024-04-02T13:51:06.697743+0000","last_undegraded":"2024-04-02T13:51:06.697743+0000","last_fullsized":"2024-04-02T13:51:06.697743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:22:18.878029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616330+0000","last_change":"2024-04-02T13:51:05.518750+0000","last_active":"2024-04-02T13:51:06.616330+0000","last_peered":"2024-04-02T13:51:06.616330+0000","last_clean":"2024-04-02T13:51:06.616330+0000","last_became_active":"2024-04-02T13:51:05.518247+0000","last_became_peered":"2024-04-02T13:51:05.518247+0000","last_unstale":"2024-04-02T13:51:06.616330+0000","last_undegraded":"2024-04-02T13:51:06.616330+0000","last_fullsized":"2024-04-02T13:51:06.616330+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:08:10.232509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616695+0000","last_change":"2024-04-02T13:51:00.241476+0000","last_active":"2024-04-02T13:51:06.616695+0000","last_peered":"2024-04-02T13:51:06.616695+0000","last_clean":"2024-04-02T13:51:06.616695+0000","last_became_active":"2024-04-02T13:51:00.241085+0000","last_became_peered":"2024-04-02T13:51:00.241085+0000","last_unstale":"2024-04-02T13:51:06.616695+0000","last_undegraded":"2024-04-02T13:51:06.616695+0000","last_fullsized":"2024-04-02T13:51:06.616695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:29:01.503901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617647+0000","last_change":"2024-04-02T13:51:01.259791+0000","last_active":"2024-04-02T13:51:06.617647+0000","last_peered":"2024-04-02T13:51:06.617647+0000","last_clean":"2024-04-02T13:51:06.617647+0000","last_became_active":"2024-04-02T13:51:01.259440+0000","last_became_peered":"2024-04-02T13:51:01.259440+0000","last_unstale":"2024-04-02T13:51:06.617647+0000","last_undegraded":"2024-04-02T13:51:06.617647+0000","last_fullsized":"2024-04-02T13:51:06.617647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:43:52.422263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347773+0000","last_change":"2024-04-02T13:51:05.366243+0000","last_active":"2024-04-02T13:51:06.347773+0000","last_peered":"2024-04-02T13:51:06.347773+0000","last_clean":"2024-04-02T13:51:06.347773+0000","last_became_active":"2024-04-02T13:51:05.365801+0000","last_became_peered":"2024-04-02T13:51:05.365801+0000","last_unstale":"2024-04-02T13:51:06.347773+0000","last_undegraded":"2024-04-02T13:51:06.347773+0000","last_fullsized":"2024-04-02T13:51:06.347773+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:33:56.853434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616923+0000","last_change":"2024-04-02T13:51:00.248695+0000","last_active":"2024-04-02T13:51:06.616923+0000","last_peered":"2024-04-02T13:51:06.616923+0000","last_clean":"2024-04-02T13:51:06.616923+0000","last_became_active":"2024-04-02T13:51:00.248313+0000","last_became_peered":"2024-04-02T13:51:00.248313+0000","last_unstale":"2024-04-02T13:51:06.616923+0000","last_undegraded":"2024-04-02T13:51:06.616923+0000","last_fullsized":"2024-04-02T13:51:06.616923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:43:21.410823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617434+0000","last_change":"2024-04-02T13:51:01.260993+0000","last_active":"2024-04-02T13:51:06.617434+0000","last_peered":"2024-04-02T13:51:06.617434+0000","last_clean":"2024-04-02T13:51:06.617434+0000","last_became_active":"2024-04-02T13:51:01.260510+0000","last_became_peered":"2024-04-02T13:51:01.260510+0000","last_unstale":"2024-04-02T13:51:06.617434+0000","last_undegraded":"2024-04-02T13:51:06.617434+0000","last_fullsized":"2024-04-02T13:51:06.617434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:17:04.941822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347875+0000","last_change":"2024-04-02T13:51:05.518455+0000","last_active":"2024-04-02T13:51:06.347875+0000","last_peered":"2024-04-02T13:51:06.347875+0000","last_clean":"2024-04-02T13:51:06.347875+0000","last_became_active":"2024-04-02T13:51:05.517962+0000","last_became_peered":"2024-04-02T13:51:05.517962+0000","last_unstale":"2024-04-02T13:51:06.347875+0000","last_undegraded":"2024-04-02T13:51:06.347875+0000","last_fullsized":"2024-04-02T13:51:06.347875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:19:47.734233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347388+0000","last_change":"2024-04-02T13:51:01.259220+0000","last_active":"2024-04-02T13:51:06.347388+0000","last_peered":"2024-04-02T13:51:06.347388+0000","last_clean":"2024-04-02T13:51:06.347388+0000","last_became_active":"2024-04-02T13:51:01.258890+0000","last_became_peered":"2024-04-02T13:51:01.258890+0000","last_unstale":"2024-04-02T13:51:06.347388+0000","last_undegraded":"2024-04-02T13:51:06.347388+0000","last_fullsized":"2024-04-02T13:51:06.347388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:56:15.444736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347333+0000","last_change":"2024-04-02T13:51:00.246727+0000","last_active":"2024-04-02T13:51:06.347333+0000","last_peered":"2024-04-02T13:51:06.347333+0000","last_clean":"2024-04-02T13:51:06.347333+0000","last_became_active":"2024-04-02T13:51:00.246364+0000","last_became_peered":"2024-04-02T13:51:00.246364+0000","last_unstale":"2024-04-02T13:51:06.347333+0000","last_undegraded":"2024-04-02T13:51:06.347333+0000","last_fullsized":"2024-04-02T13:51:06.347333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:35:13.249972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617351+0000","last_change":"2024-04-02T13:51:05.358506+0000","last_active":"2024-04-02T13:51:06.617351+0000","last_peered":"2024-04-02T13:51:06.617351+0000","last_clean":"2024-04-02T13:51:06.617351+0000","last_became_active":"2024-04-02T13:51:05.358157+0000","last_became_peered":"2024-04-02T13:51:05.358157+0000","last_unstale":"2024-04-02T13:51:06.617351+0000","last_undegraded":"2024-04-02T13:51:06.617351+0000","last_fullsized":"2024-04-02T13:51:06.617351+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:24:15.585124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616819+0000","last_change":"2024-04-02T13:51:01.261939+0000","last_active":"2024-04-02T13:51:06.616819+0000","last_peered":"2024-04-02T13:51:06.616819+0000","last_clean":"2024-04-02T13:51:06.616819+0000","last_became_active":"2024-04-02T13:51:01.261580+0000","last_became_peered":"2024-04-02T13:51:01.261580+0000","last_unstale":"2024-04-02T13:51:06.616819+0000","last_undegraded":"2024-04-02T13:51:06.616819+0000","last_fullsized":"2024-04-02T13:51:06.616819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:22:41.790555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:29.351482+0000","last_change":"2024-04-02T13:51:00.238962+0000","last_active":"2024-04-02T13:51:29.351482+0000","last_peered":"2024-04-02T13:51:29.351482+0000","last_clean":"2024-04-02T13:51:29.351482+0000","last_became_active":"2024-04-02T13:51:00.238475+0000","last_became_peered":"2024-04-02T13:51:00.238475+0000","last_unstale":"2024-04-02T13:51:29.351482+0000","last_undegraded":"2024-04-02T13:51:29.351482+0000","last_fullsized":"2024-04-02T13:51:29.351482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:41:07.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":357,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616515+0000","last_change":"2024-04-02T13:49:47.847355+0000","last_active":"2024-04-02T13:51:06.616515+0000","last_peered":"2024-04-02T13:51:06.616515+0000","last_clean":"2024-04-02T13:51:06.616515+0000","last_became_active":"2024-04-02T13:49:47.846768+0000","last_became_peered":"2024-04-02T13:49:47.846768+0000","last_unstale":"2024-04-02T13:51:06.616515+0000","last_undegraded":"2024-04-02T13:51:06.616515+0000","last_fullsized":"2024-04-02T13:51:06.616515+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:46:21.098780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:46:21.098780+0000","last_clean_scrub_stamp":"2024-04-02T13:46:21.098780+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:03:52.323921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347965+0000","last_change":"2024-04-02T13:51:05.518859+0000","last_active":"2024-04-02T13:51:06.347965+0000","last_peered":"2024-04-02T13:51:06.347965+0000","last_clean":"2024-04-02T13:51:06.347965+0000","last_became_active":"2024-04-02T13:51:05.518364+0000","last_became_peered":"2024-04-02T13:51:05.518364+0000","last_unstale":"2024-04-02T13:51:06.347965+0000","last_undegraded":"2024-04-02T13:51:06.347965+0000","last_fullsized":"2024-04-02T13:51:06.347965+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:10:45.877853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347231+0000","last_change":"2024-04-02T13:51:01.254795+0000","last_active":"2024-04-02T13:51:06.347231+0000","last_peered":"2024-04-02T13:51:06.347231+0000","last_clean":"2024-04-02T13:51:06.347231+0000","last_became_active":"2024-04-02T13:51:01.254452+0000","last_became_peered":"2024-04-02T13:51:01.254452+0000","last_unstale":"2024-04-02T13:51:06.347231+0000","last_undegraded":"2024-04-02T13:51:06.347231+0000","last_fullsized":"2024-04-02T13:51:06.347231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:13:53.747308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616801+0000","last_change":"2024-04-02T13:51:00.241323+0000","last_active":"2024-04-02T13:51:06.616801+0000","last_peered":"2024-04-02T13:51:06.616801+0000","last_clean":"2024-04-02T13:51:06.616801+0000","last_became_active":"2024-04-02T13:51:00.240989+0000","last_became_peered":"2024-04-02T13:51:00.240989+0000","last_unstale":"2024-04-02T13:51:06.616801+0000","last_undegraded":"2024-04-02T13:51:06.616801+0000","last_fullsized":"2024-04-02T13:51:06.616801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:47:52.601517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.697528+0000","last_change":"2024-04-02T13:51:05.360641+0000","last_active":"2024-04-02T13:51:06.697528+0000","last_peered":"2024-04-02T13:51:06.697528+0000","last_clean":"2024-04-02T13:51:06.697528+0000","last_became_active":"2024-04-02T13:51:05.360089+0000","last_became_peered":"2024-04-02T13:51:05.360089+0000","last_unstale":"2024-04-02T13:51:06.697528+0000","last_undegraded":"2024-04-02T13:51:06.697528+0000","last_fullsized":"2024-04-02T13:51:06.697528+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:04:38.393274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616669+0000","last_change":"2024-04-02T13:51:00.241416+0000","last_active":"2024-04-02T13:51:06.616669+0000","last_peered":"2024-04-02T13:51:06.616669+0000","last_clean":"2024-04-02T13:51:06.616669+0000","last_became_active":"2024-04-02T13:51:00.241071+0000","last_became_peered":"2024-04-02T13:51:00.241071+0000","last_unstale":"2024-04-02T13:51:06.616669+0000","last_undegraded":"2024-04-02T13:51:06.616669+0000","last_fullsized":"2024-04-02T13:51:06.616669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:24:16.243737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347985+0000","last_change":"2024-04-02T13:51:01.252156+0000","last_active":"2024-04-02T13:51:06.347985+0000","last_peered":"2024-04-02T13:51:06.347985+0000","last_clean":"2024-04-02T13:51:06.347985+0000","last_became_active":"2024-04-02T13:51:01.251733+0000","last_became_peered":"2024-04-02T13:51:01.251733+0000","last_unstale":"2024-04-02T13:51:06.347985+0000","last_undegraded":"2024-04-02T13:51:06.347985+0000","last_fullsized":"2024-04-02T13:51:06.347985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:54:24.082643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616346+0000","last_change":"2024-04-02T13:51:05.518367+0000","last_active":"2024-04-02T13:51:06.616346+0000","last_peered":"2024-04-02T13:51:06.616346+0000","last_clean":"2024-04-02T13:51:06.616346+0000","last_became_active":"2024-04-02T13:51:05.517783+0000","last_became_peered":"2024-04-02T13:51:05.517783+0000","last_unstale":"2024-04-02T13:51:06.616346+0000","last_undegraded":"2024-04-02T13:51:06.616346+0000","last_fullsized":"2024-04-02T13:51:06.616346+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:10:53.880909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:27.344903+0000","last_change":"2024-04-02T13:51:00.248998+0000","last_active":"2024-04-02T13:51:27.344903+0000","last_peered":"2024-04-02T13:51:27.344903+0000","last_clean":"2024-04-02T13:51:27.344903+0000","last_became_active":"2024-04-02T13:51:00.248613+0000","last_became_peered":"2024-04-02T13:51:00.248613+0000","last_unstale":"2024-04-02T13:51:27.344903+0000","last_undegraded":"2024-04-02T13:51:27.344903+0000","last_fullsized":"2024-04-02T13:51:27.344903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:53.948645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617187+0000","last_change":"2024-04-02T13:51:01.257244+0000","last_active":"2024-04-02T13:51:06.617187+0000","last_peered":"2024-04-02T13:51:06.617187+0000","last_clean":"2024-04-02T13:51:06.617187+0000","last_became_active":"2024-04-02T13:51:01.256739+0000","last_became_peered":"2024-04-02T13:51:01.256739+0000","last_unstale":"2024-04-02T13:51:06.617187+0000","last_undegraded":"2024-04-02T13:51:06.617187+0000","last_fullsized":"2024-04-02T13:51:06.617187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:38:18.786553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.353918+0000","last_change":"2024-04-02T13:51:05.356177+0000","last_active":"2024-04-02T13:51:06.353918+0000","last_peered":"2024-04-02T13:51:06.353918+0000","last_clean":"2024-04-02T13:51:06.353918+0000","last_became_active":"2024-04-02T13:51:05.354704+0000","last_became_peered":"2024-04-02T13:51:05.354704+0000","last_unstale":"2024-04-02T13:51:06.353918+0000","last_undegraded":"2024-04-02T13:51:06.353918+0000","last_fullsized":"2024-04-02T13:51:06.353918+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:27:55.761490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:28.350569+0000","last_change":"2024-04-02T13:51:00.245945+0000","last_active":"2024-04-02T13:51:28.350569+0000","last_peered":"2024-04-02T13:51:28.350569+0000","last_clean":"2024-04-02T13:51:28.350569+0000","last_became_active":"2024-04-02T13:51:00.245528+0000","last_became_peered":"2024-04-02T13:51:00.245528+0000","last_unstale":"2024-04-02T13:51:28.350569+0000","last_undegraded":"2024-04-02T13:51:28.350569+0000","last_fullsized":"2024-04-02T13:51:28.350569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:37:49.263614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617097+0000","last_change":"2024-04-02T13:51:01.260268+0000","last_active":"2024-04-02T13:51:06.617097+0000","last_peered":"2024-04-02T13:51:06.617097+0000","last_clean":"2024-04-02T13:51:06.617097+0000","last_became_active":"2024-04-02T13:51:01.258182+0000","last_became_peered":"2024-04-02T13:51:01.258182+0000","last_unstale":"2024-04-02T13:51:06.617097+0000","last_undegraded":"2024-04-02T13:51:06.617097+0000","last_fullsized":"2024-04-02T13:51:06.617097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:31:31.423988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.618242+0000","last_change":"2024-04-02T13:51:05.518597+0000","last_active":"2024-04-02T13:51:06.618242+0000","last_peered":"2024-04-02T13:51:06.618242+0000","last_clean":"2024-04-02T13:51:06.618242+0000","last_became_active":"2024-04-02T13:51:05.518134+0000","last_became_peered":"2024-04-02T13:51:05.518134+0000","last_unstale":"2024-04-02T13:51:06.618242+0000","last_undegraded":"2024-04-02T13:51:06.618242+0000","last_fullsized":"2024-04-02T13:51:06.618242+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:00:38.632939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:25.333094+0000","last_change":"2024-04-02T13:51:00.241680+0000","last_active":"2024-04-02T13:51:25.333094+0000","last_peered":"2024-04-02T13:51:25.333094+0000","last_clean":"2024-04-02T13:51:25.333094+0000","last_became_active":"2024-04-02T13:51:00.241301+0000","last_became_peered":"2024-04-02T13:51:00.241301+0000","last_unstale":"2024-04-02T13:51:25.333094+0000","last_undegraded":"2024-04-02T13:51:25.333094+0000","last_fullsized":"2024-04-02T13:51:25.333094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:12:10.606973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616114+0000","last_change":"2024-04-02T13:51:01.254795+0000","last_active":"2024-04-02T13:51:06.616114+0000","last_peered":"2024-04-02T13:51:06.616114+0000","last_clean":"2024-04-02T13:51:06.616114+0000","last_became_active":"2024-04-02T13:51:01.254451+0000","last_became_peered":"2024-04-02T13:51:01.254451+0000","last_unstale":"2024-04-02T13:51:06.616114+0000","last_undegraded":"2024-04-02T13:51:06.616114+0000","last_fullsized":"2024-04-02T13:51:06.616114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:58:04.202996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617888+0000","last_change":"2024-04-02T13:51:05.363592+0000","last_active":"2024-04-02T13:51:06.617888+0000","last_peered":"2024-04-02T13:51:06.617888+0000","last_clean":"2024-04-02T13:51:06.617888+0000","last_became_active":"2024-04-02T13:51:05.363211+0000","last_became_peered":"2024-04-02T13:51:05.363211+0000","last_unstale":"2024-04-02T13:51:06.617888+0000","last_undegraded":"2024-04-02T13:51:06.617888+0000","last_fullsized":"2024-04-02T13:51:06.617888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:31:56.561505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:28.351341+0000","last_change":"2024-04-02T13:51:00.243681+0000","last_active":"2024-04-02T13:51:28.351341+0000","last_peered":"2024-04-02T13:51:28.351341+0000","last_clean":"2024-04-02T13:51:28.351341+0000","last_became_active":"2024-04-02T13:51:00.241615+0000","last_became_peered":"2024-04-02T13:51:00.241615+0000","last_unstale":"2024-04-02T13:51:28.351341+0000","last_undegraded":"2024-04-02T13:51:28.351341+0000","last_fullsized":"2024-04-02T13:51:28.351341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:54:30.112735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.697363+0000","last_change":"2024-04-02T13:51:01.252429+0000","last_active":"2024-04-02T13:51:06.697363+0000","last_peered":"2024-04-02T13:51:06.697363+0000","last_clean":"2024-04-02T13:51:06.697363+0000","last_became_active":"2024-04-02T13:51:01.252017+0000","last_became_peered":"2024-04-02T13:51:01.252017+0000","last_unstale":"2024-04-02T13:51:06.697363+0000","last_undegraded":"2024-04-02T13:51:06.697363+0000","last_fullsized":"2024-04-02T13:51:06.697363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:32:02.152578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.618197+0000","last_change":"2024-04-02T13:51:05.361297+0000","last_active":"2024-04-02T13:51:06.618197+0000","last_peered":"2024-04-02T13:51:06.618197+0000","last_clean":"2024-04-02T13:51:06.618197+0000","last_became_active":"2024-04-02T13:51:05.360916+0000","last_became_peered":"2024-04-02T13:51:05.360916+0000","last_unstale":"2024-04-02T13:51:06.618197+0000","last_undegraded":"2024-04-02T13:51:06.618197+0000","last_fullsized":"2024-04-02T13:51:06.618197+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:52:48.608587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:27.343296+0000","last_change":"2024-04-02T13:51:00.249141+0000","last_active":"2024-04-02T13:51:27.343296+0000","last_peered":"2024-04-02T13:51:27.343296+0000","last_clean":"2024-04-02T13:51:27.343296+0000","last_became_active":"2024-04-02T13:51:00.248760+0000","last_became_peered":"2024-04-02T13:51:00.248760+0000","last_unstale":"2024-04-02T13:51:27.343296+0000","last_undegraded":"2024-04-02T13:51:27.343296+0000","last_fullsized":"2024-04-02T13:51:27.343296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:37:12.710427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617427+0000","last_change":"2024-04-02T13:51:01.255557+0000","last_active":"2024-04-02T13:51:06.617427+0000","last_peered":"2024-04-02T13:51:06.617427+0000","last_clean":"2024-04-02T13:51:06.617427+0000","last_became_active":"2024-04-02T13:51:01.255011+0000","last_became_peered":"2024-04-02T13:51:01.255011+0000","last_unstale":"2024-04-02T13:51:06.617427+0000","last_undegraded":"2024-04-02T13:51:06.617427+0000","last_fullsized":"2024-04-02T13:51:06.617427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:01:52.105965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.618366+0000","last_change":"2024-04-02T13:51:05.518764+0000","last_active":"2024-04-02T13:51:06.618366+0000","last_peered":"2024-04-02T13:51:06.618366+0000","last_clean":"2024-04-02T13:51:06.618366+0000","last_became_active":"2024-04-02T13:51:05.518088+0000","last_became_peered":"2024-04-02T13:51:05.518088+0000","last_unstale":"2024-04-02T13:51:06.618366+0000","last_undegraded":"2024-04-02T13:51:06.618366+0000","last_fullsized":"2024-04-02T13:51:06.618366+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:03:02.901384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:25.331268+0000","last_change":"2024-04-02T13:51:00.244147+0000","last_active":"2024-04-02T13:51:25.331268+0000","last_peered":"2024-04-02T13:51:25.331268+0000","last_clean":"2024-04-02T13:51:25.331268+0000","last_became_active":"2024-04-02T13:51:00.243763+0000","last_became_peered":"2024-04-02T13:51:00.243763+0000","last_unstale":"2024-04-02T13:51:25.331268+0000","last_undegraded":"2024-04-02T13:51:25.331268+0000","last_fullsized":"2024-04-02T13:51:25.331268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:34:27.738746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617357+0000","last_change":"2024-04-02T13:51:01.268121+0000","last_active":"2024-04-02T13:51:06.617357+0000","last_peered":"2024-04-02T13:51:06.617357+0000","last_clean":"2024-04-02T13:51:06.617357+0000","last_became_active":"2024-04-02T13:51:01.266363+0000","last_became_peered":"2024-04-02T13:51:01.266363+0000","last_unstale":"2024-04-02T13:51:06.617357+0000","last_undegraded":"2024-04-02T13:51:06.617357+0000","last_fullsized":"2024-04-02T13:51:06.617357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:30:50.254916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.349429+0000","last_change":"2024-04-02T13:51:05.361285+0000","last_active":"2024-04-02T13:51:06.349429+0000","last_peered":"2024-04-02T13:51:06.349429+0000","last_clean":"2024-04-02T13:51:06.349429+0000","last_became_active":"2024-04-02T13:51:05.360122+0000","last_became_peered":"2024-04-02T13:51:05.360122+0000","last_unstale":"2024-04-02T13:51:06.349429+0000","last_undegraded":"2024-04-02T13:51:06.349429+0000","last_fullsized":"2024-04-02T13:51:06.349429+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:30:59.935538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'6","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:29.354823+0000","last_change":"2024-04-02T13:51:00.247235+0000","last_active":"2024-04-02T13:51:29.354823+0000","last_peered":"2024-04-02T13:51:29.354823+0000","last_clean":"2024-04-02T13:51:29.354823+0000","last_became_active":"2024-04-02T13:51:00.246856+0000","last_became_peered":"2024-04-02T13:51:00.246856+0000","last_unstale":"2024-04-02T13:51:29.354823+0000","last_undegraded":"2024-04-02T13:51:29.354823+0000","last_fullsized":"2024-04-02T13:51:29.354823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:17:51.875952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617713+0000","last_change":"2024-04-02T13:51:01.263866+0000","last_active":"2024-04-02T13:51:06.617713+0000","last_peered":"2024-04-02T13:51:06.617713+0000","last_clean":"2024-04-02T13:51:06.617713+0000","last_became_active":"2024-04-02T13:51:01.263564+0000","last_became_peered":"2024-04-02T13:51:01.263564+0000","last_unstale":"2024-04-02T13:51:06.617713+0000","last_undegraded":"2024-04-02T13:51:06.617713+0000","last_fullsized":"2024-04-02T13:51:06.617713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:59:33.663277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616388+0000","last_change":"2024-04-02T13:51:05.363079+0000","last_active":"2024-04-02T13:51:06.616388+0000","last_peered":"2024-04-02T13:51:06.616388+0000","last_clean":"2024-04-02T13:51:06.616388+0000","last_became_active":"2024-04-02T13:51:05.362631+0000","last_became_peered":"2024-04-02T13:51:05.362631+0000","last_unstale":"2024-04-02T13:51:06.616388+0000","last_undegraded":"2024-04-02T13:51:06.616388+0000","last_fullsized":"2024-04-02T13:51:06.616388+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:23.656269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:28.351460+0000","last_change":"2024-04-02T13:51:00.242820+0000","last_active":"2024-04-02T13:51:28.351460+0000","last_peered":"2024-04-02T13:51:28.351460+0000","last_clean":"2024-04-02T13:51:28.351460+0000","last_became_active":"2024-04-02T13:51:00.242262+0000","last_became_peered":"2024-04-02T13:51:00.242262+0000","last_unstale":"2024-04-02T13:51:28.351460+0000","last_undegraded":"2024-04-02T13:51:28.351460+0000","last_fullsized":"2024-04-02T13:51:28.351460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:12:22.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617060+0000","last_change":"2024-04-02T13:51:01.266548+0000","last_active":"2024-04-02T13:51:06.617060+0000","last_peered":"2024-04-02T13:51:06.617060+0000","last_clean":"2024-04-02T13:51:06.617060+0000","last_became_active":"2024-04-02T13:51:01.259003+0000","last_became_peered":"2024-04-02T13:51:01.259003+0000","last_unstale":"2024-04-02T13:51:06.617060+0000","last_undegraded":"2024-04-02T13:51:06.617060+0000","last_fullsized":"2024-04-02T13:51:06.617060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:38:53.692231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347053+0000","last_change":"2024-04-02T13:51:05.359994+0000","last_active":"2024-04-02T13:51:06.347053+0000","last_peered":"2024-04-02T13:51:06.347053+0000","last_clean":"2024-04-02T13:51:06.347053+0000","last_became_active":"2024-04-02T13:51:05.359438+0000","last_became_peered":"2024-04-02T13:51:05.359438+0000","last_unstale":"2024-04-02T13:51:06.347053+0000","last_undegraded":"2024-04-02T13:51:06.347053+0000","last_fullsized":"2024-04-02T13:51:06.347053+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:09:43.929446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347779+0000","last_change":"2024-04-02T13:51:00.237889+0000","last_active":"2024-04-02T13:51:06.347779+0000","last_peered":"2024-04-02T13:51:06.347779+0000","last_clean":"2024-04-02T13:51:06.347779+0000","last_became_active":"2024-04-02T13:51:00.237428+0000","last_became_peered":"2024-04-02T13:51:00.237428+0000","last_unstale":"2024-04-02T13:51:06.347779+0000","last_undegraded":"2024-04-02T13:51:06.347779+0000","last_fullsized":"2024-04-02T13:51:06.347779+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:21:52.483656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616840+0000","last_change":"2024-04-02T13:51:01.262093+0000","last_active":"2024-04-02T13:51:06.616840+0000","last_peered":"2024-04-02T13:51:06.616840+0000","last_clean":"2024-04-02T13:51:06.616840+0000","last_became_active":"2024-04-02T13:51:01.258357+0000","last_became_peered":"2024-04-02T13:51:01.258357+0000","last_unstale":"2024-04-02T13:51:06.616840+0000","last_undegraded":"2024-04-02T13:51:06.616840+0000","last_fullsized":"2024-04-02T13:51:06.616840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:00:06.317307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.697453+0000","last_change":"2024-04-02T13:51:01.261015+0000","last_active":"2024-04-02T13:51:06.697453+0000","last_peered":"2024-04-02T13:51:06.697453+0000","last_clean":"2024-04-02T13:51:06.697453+0000","last_became_active":"2024-04-02T13:51:01.260697+0000","last_became_peered":"2024-04-02T13:51:01.260697+0000","last_unstale":"2024-04-02T13:51:06.697453+0000","last_undegraded":"2024-04-02T13:51:06.697453+0000","last_fullsized":"2024-04-02T13:51:06.697453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:49:02.248995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.354015+0000","last_change":"2024-04-02T13:51:01.257205+0000","last_active":"2024-04-02T13:51:06.354015+0000","last_peered":"2024-04-02T13:51:06.354015+0000","last_clean":"2024-04-02T13:51:06.354015+0000","last_became_active":"2024-04-02T13:51:01.256866+0000","last_became_peered":"2024-04-02T13:51:01.256866+0000","last_unstale":"2024-04-02T13:51:06.354015+0000","last_undegraded":"2024-04-02T13:51:06.354015+0000","last_fullsized":"2024-04-02T13:51:06.354015+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:53:23.211580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.353913+0000","last_change":"2024-04-02T13:51:01.255563+0000","last_active":"2024-04-02T13:51:06.353913+0000","last_peered":"2024-04-02T13:51:06.353913+0000","last_clean":"2024-04-02T13:51:06.353913+0000","last_became_active":"2024-04-02T13:51:01.255228+0000","last_became_peered":"2024-04-02T13:51:01.255228+0000","last_unstale":"2024-04-02T13:51:06.353913+0000","last_undegraded":"2024-04-02T13:51:06.353913+0000","last_fullsized":"2024-04-02T13:51:06.353913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:48:28.200776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616474+0000","last_change":"2024-04-02T13:51:01.258153+0000","last_active":"2024-04-02T13:51:06.616474+0000","last_peered":"2024-04-02T13:51:06.616474+0000","last_clean":"2024-04-02T13:51:06.616474+0000","last_became_active":"2024-04-02T13:51:01.257656+0000","last_became_peered":"2024-04-02T13:51:01.257656+0000","last_unstale":"2024-04-02T13:51:06.616474+0000","last_undegraded":"2024-04-02T13:51:06.616474+0000","last_fullsized":"2024-04-02T13:51:06.616474+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:37.887427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.348019+0000","last_change":"2024-04-02T13:51:01.254675+0000","last_active":"2024-04-02T13:51:06.348019+0000","last_peered":"2024-04-02T13:51:06.348019+0000","last_clean":"2024-04-02T13:51:06.348019+0000","last_became_active":"2024-04-02T13:51:01.254336+0000","last_became_peered":"2024-04-02T13:51:01.254336+0000","last_unstale":"2024-04-02T13:51:06.348019+0000","last_undegraded":"2024-04-02T13:51:06.348019+0000","last_fullsized":"2024-04-02T13:51:06.348019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:36.171961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.697843+0000","last_change":"2024-04-02T13:51:01.259457+0000","last_active":"2024-04-02T13:51:06.697843+0000","last_peered":"2024-04-02T13:51:06.697843+0000","last_clean":"2024-04-02T13:51:06.697843+0000","last_became_active":"2024-04-02T13:51:01.259099+0000","last_became_peered":"2024-04-02T13:51:01.259099+0000","last_unstale":"2024-04-02T13:51:06.697843+0000","last_undegraded":"2024-04-02T13:51:06.697843+0000","last_fullsized":"2024-04-02T13:51:06.697843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:13:06.594132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.354595+0000","last_change":"2024-04-02T13:51:01.248177+0000","last_active":"2024-04-02T13:51:06.354595+0000","last_peered":"2024-04-02T13:51:06.354595+0000","last_clean":"2024-04-02T13:51:06.354595+0000","last_became_active":"2024-04-02T13:51:01.247843+0000","last_became_peered":"2024-04-02T13:51:01.247843+0000","last_unstale":"2024-04-02T13:51:06.354595+0000","last_undegraded":"2024-04-02T13:51:06.354595+0000","last_fullsized":"2024-04-02T13:51:06.354595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:38:05.805382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617548+0000","last_change":"2024-04-02T13:51:01.254822+0000","last_active":"2024-04-02T13:51:06.617548+0000","last_peered":"2024-04-02T13:51:06.617548+0000","last_clean":"2024-04-02T13:51:06.617548+0000","last_became_active":"2024-04-02T13:51:01.254451+0000","last_became_peered":"2024-04-02T13:51:01.254451+0000","last_unstale":"2024-04-02T13:51:06.617548+0000","last_undegraded":"2024-04-02T13:51:06.617548+0000","last_fullsized":"2024-04-02T13:51:06.617548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:15:35.790371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347295+0000","last_change":"2024-04-02T13:51:01.265467+0000","last_active":"2024-04-02T13:51:06.347295+0000","last_peered":"2024-04-02T13:51:06.347295+0000","last_clean":"2024-04-02T13:51:06.347295+0000","last_became_active":"2024-04-02T13:51:01.265072+0000","last_became_peered":"2024-04-02T13:51:01.265072+0000","last_unstale":"2024-04-02T13:51:06.347295+0000","last_undegraded":"2024-04-02T13:51:06.347295+0000","last_fullsized":"2024-04-02T13:51:06.347295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:12:07.702222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616954+0000","last_change":"2024-04-02T13:51:01.252844+0000","last_active":"2024-04-02T13:51:06.616954+0000","last_peered":"2024-04-02T13:51:06.616954+0000","last_clean":"2024-04-02T13:51:06.616954+0000","last_became_active":"2024-04-02T13:51:01.252515+0000","last_became_peered":"2024-04-02T13:51:01.252515+0000","last_unstale":"2024-04-02T13:51:06.616954+0000","last_undegraded":"2024-04-02T13:51:06.616954+0000","last_fullsized":"2024-04-02T13:51:06.616954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:20:16.216418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.349785+0000","last_change":"2024-04-02T13:51:01.260141+0000","last_active":"2024-04-02T13:51:06.349785+0000","last_peered":"2024-04-02T13:51:06.349785+0000","last_clean":"2024-04-02T13:51:06.349785+0000","last_became_active":"2024-04-02T13:51:01.259530+0000","last_became_peered":"2024-04-02T13:51:01.259530+0000","last_unstale":"2024-04-02T13:51:06.349785+0000","last_undegraded":"2024-04-02T13:51:06.349785+0000","last_fullsized":"2024-04-02T13:51:06.349785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:00:03.579414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347874+0000","last_change":"2024-04-02T13:51:01.259981+0000","last_active":"2024-04-02T13:51:06.347874+0000","last_peered":"2024-04-02T13:51:06.347874+0000","last_clean":"2024-04-02T13:51:06.347874+0000","last_became_active":"2024-04-02T13:51:01.259647+0000","last_became_peered":"2024-04-02T13:51:01.259647+0000","last_unstale":"2024-04-02T13:51:06.347874+0000","last_undegraded":"2024-04-02T13:51:06.347874+0000","last_fullsized":"2024-04-02T13:51:06.347874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:10:22.753767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.697580+0000","last_change":"2024-04-02T13:51:01.255342+0000","last_active":"2024-04-02T13:51:06.697580+0000","last_peered":"2024-04-02T13:51:06.697580+0000","last_clean":"2024-04-02T13:51:06.697580+0000","last_became_active":"2024-04-02T13:51:01.254679+0000","last_became_peered":"2024-04-02T13:51:01.254679+0000","last_unstale":"2024-04-02T13:51:06.697580+0000","last_undegraded":"2024-04-02T13:51:06.697580+0000","last_fullsized":"2024-04-02T13:51:06.697580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:25:12.198270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":45822,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":29,"num_read_kb":45,"num_write":177,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":372736,"data_stored":97749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":483005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612632,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1098250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776157,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":483913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037794,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":483631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332135,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36904,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708248,"statfs":{"total":95995035648,"available":95957245952,"internally_reserved":0,"allocated":1593344,"data_stored":1073955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":483005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":486847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":484483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1077248,"data_stored":525839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":483435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":483705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1638400,"data_stored":1080289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":42834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":42207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T13:51:30.834 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T13:51:30.920 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 02 13:51:30 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-b[66653]: 2024-04-02T13:51:30.646+0000 7fed29007700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-02T13:51:30.920 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 02 13:51:30 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-b[66653]: 2024-04-02T13:51:30.646+0000 7fed29007700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-02T13:51:30.920 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 02 13:51:30 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-b[66653]: 2024-04-02T13:51:30.646+0000 7fed29007700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-02T13:51:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:30 smithi088 ceph-mon[36528]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:51:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:30 smithi088 ceph-mon[36528]: pgmap v307: 129 pgs: 129 active+clean; 582 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-02T13:51:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:30 smithi088 ceph-mon[36528]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:51:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:30 smithi088 ceph-mon[36528]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-02T13:51:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:30 smithi088 ceph-mon[36528]: from='client.24692 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:51:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:30 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/280229263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:51:30.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.967+0000 7f47304dc640 1 -- 172.21.15.52:0/332805343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47281025d0 msgr2=0x7f4728108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:30.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.967+0000 7f47304dc640 1 --2- 172.21.15.52:0/332805343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47281025d0 0x7f4728108e60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4718009ca0 tx=0x7f471802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.967+0000 7f47304dc640 1 -- 172.21.15.52:0/332805343 shutdown_connections 2024-04-02T13:51:30.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.967+0000 7f47304dc640 1 --2- 172.21.15.52:0/332805343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47281025d0 0x7f4728108e60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.967+0000 7f47304dc640 1 --2- 172.21.15.52:0/332805343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4728101c10 0x7f4728102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.967+0000 7f47304dc640 1 --2- 172.21.15.52:0/332805343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4728100a20 0x7f4728100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.967+0000 7f47304dc640 1 -- 172.21.15.52:0/332805343 >> 172.21.15.52:0/332805343 conn(0x7f47280fbf90 msgr2=0x7f47280fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:30.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.968+0000 7f47304dc640 1 -- 172.21.15.52:0/332805343 shutdown_connections 2024-04-02T13:51:30.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.968+0000 7f47304dc640 1 -- 172.21.15.52:0/332805343 wait complete. 2024-04-02T13:51:30.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.968+0000 7f47304dc640 1 Processor -- start 2024-04-02T13:51:30.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.968+0000 7f47304dc640 1 -- start start 2024-04-02T13:51:30.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.968+0000 7f47304dc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4728100a20 0x7f4728197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:30.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.968+0000 7f47304dc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4728101c10 0x7f47281982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:30.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.968+0000 7f47304dc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47281025d0 0x7f472819f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:30.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.968+0000 7f47304dc640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f472810d7c0 con 0x7f4728100a20 2024-04-02T13:51:30.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.968+0000 7f47304dc640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f472810d640 con 0x7f47281025d0 2024-04-02T13:51:30.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.968+0000 7f47304dc640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f472810d940 con 0x7f4728101c10 2024-04-02T13:51:30.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.968+0000 7f472e278640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4728100a20 0x7f4728197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:30.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.968+0000 7f472e278640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4728100a20 0x7f4728197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53666/0 (socket says 172.21.15.52:53666) 2024-04-02T13:51:30.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.968+0000 7f472e278640 1 -- 172.21.15.52:0/1867645764 learned_addr learned my addr 172.21.15.52:0/1867645764 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:30.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.969+0000 7f472ea79640 1 --2- 172.21.15.52:0/1867645764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47281025d0 0x7f472819f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:30.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.969+0000 7f472da77640 1 --2- 172.21.15.52:0/1867645764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4728101c10 0x7f47281982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:30.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.969+0000 7f472e278640 1 -- 172.21.15.52:0/1867645764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4728101c10 msgr2=0x7f47281982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:30.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.969+0000 7f472e278640 1 --2- 172.21.15.52:0/1867645764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4728101c10 0x7f47281982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.969+0000 7f472e278640 1 -- 172.21.15.52:0/1867645764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47281025d0 msgr2=0x7f472819f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:30.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.969+0000 7f472e278640 1 --2- 172.21.15.52:0/1867645764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47281025d0 0x7f472819f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:30.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.969+0000 7f472e278640 1 -- 172.21.15.52:0/1867645764 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472819f9d0 con 0x7f4728100a20 2024-04-02T13:51:30.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.969+0000 7f472da77640 1 --2- 172.21.15.52:0/1867645764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4728101c10 0x7f47281982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:51:30.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.969+0000 7f472e278640 1 --2- 172.21.15.52:0/1867645764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4728100a20 0x7f4728197db0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f472000d950 tx=0x7f472000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:30.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.969+0000 7f47277fe640 1 -- 172.21.15.52:0/1867645764 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4720017070 con 0x7f4728100a20 2024-04-02T13:51:30.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.969+0000 7f47277fe640 1 -- 172.21.15.52:0/1867645764 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4720009c30 con 0x7f4728100a20 2024-04-02T13:51:30.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.969+0000 7f47304dc640 1 -- 172.21.15.52:0/1867645764 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f472819fcc0 con 0x7f4728100a20 2024-04-02T13:51:30.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.969+0000 7f47277fe640 1 -- 172.21.15.52:0/1867645764 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472000f600 con 0x7f4728100a20 2024-04-02T13:51:30.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.969+0000 7f47304dc640 1 -- 172.21.15.52:0/1867645764 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47281a01c0 con 0x7f4728100a20 2024-04-02T13:51:30.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.970+0000 7f47304dc640 1 -- 172.21.15.52:0/1867645764 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4728107d40 con 0x7f4728100a20 2024-04-02T13:51:30.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.971+0000 7f47277fe640 1 -- 172.21.15.52:0/1867645764 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f472001a070 con 0x7f4728100a20 2024-04-02T13:51:30.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.972+0000 7f47277fe640 1 --2- 172.21.15.52:0/1867645764 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f47100a3cc0 0x7f47100a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:30.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.972+0000 7f472da77640 1 --2- 172.21.15.52:0/1867645764 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f47100a3cc0 0x7f47100a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:30.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.972+0000 7f47277fe640 1 -- 172.21.15.52:0/1867645764 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f47200d0250 con 0x7f4728100a20 2024-04-02T13:51:30.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.972+0000 7f472da77640 1 --2- 172.21.15.52:0/1867645764 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f47100a3cc0 0x7f47100a6180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f47281992c0 tx=0x7f471c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:30.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:30.976+0000 7f47277fe640 1 -- 172.21.15.52:0/1867645764 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4720009de0 con 0x7f4728100a20 2024-04-02T13:51:31.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:31.159+0000 7f47304dc640 1 -- 172.21.15.52:0/1867645764 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4728061f40 con 0x7f47100a3cc0 2024-04-02T13:51:31.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:31.168+0000 7f47277fe640 1 -- 172.21.15.52:0/1867645764 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340348 (secure 0 0 0) 0x7f4728061f40 con 0x7f47100a3cc0 2024-04-02T13:51:31.176 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:51:31.178 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-04-02T13:51:31.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:31.171+0000 7f47304dc640 1 -- 172.21.15.52:0/1867645764 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f47100a3cc0 msgr2=0x7f47100a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:31.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:31.171+0000 7f47304dc640 1 --2- 172.21.15.52:0/1867645764 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f47100a3cc0 0x7f47100a6180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f47281992c0 tx=0x7f471c005f90 comp rx=0 tx=0).stop 2024-04-02T13:51:31.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:31.171+0000 7f47304dc640 1 -- 172.21.15.52:0/1867645764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4728100a20 msgr2=0x7f4728197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:31.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:31.171+0000 7f47304dc640 1 --2- 172.21.15.52:0/1867645764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4728100a20 0x7f4728197db0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f472000d950 tx=0x7f472000de10 comp rx=0 tx=0).stop 2024-04-02T13:51:31.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:31.171+0000 7f47304dc640 1 -- 172.21.15.52:0/1867645764 shutdown_connections 2024-04-02T13:51:31.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:31.171+0000 7f47304dc640 1 --2- 172.21.15.52:0/1867645764 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f47100a3cc0 0x7f47100a6180 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:31.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:31.171+0000 7f47304dc640 1 --2- 172.21.15.52:0/1867645764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47281025d0 0x7f472819f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:31.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:31.171+0000 7f47304dc640 1 --2- 172.21.15.52:0/1867645764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4728101c10 0x7f47281982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:31.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:31.171+0000 7f47304dc640 1 --2- 172.21.15.52:0/1867645764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4728100a20 0x7f4728197db0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:31.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:31.171+0000 7f47304dc640 1 -- 172.21.15.52:0/1867645764 >> 172.21.15.52:0/1867645764 conn(0x7f47280fbf90 msgr2=0x7f4728104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:31.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:31.172+0000 7f47304dc640 1 -- 172.21.15.52:0/1867645764 shutdown_connections 2024-04-02T13:51:31.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:31.172+0000 7f47304dc640 1 -- 172.21.15.52:0/1867645764 wait complete. 2024-04-02T13:51:31.186 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-04-02T13:51:30.740660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636307,"num_objects":80,"num_object_clones":0,"num_object_copies":241,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":165,"num_read_kb":164,"num_write":359,"num_write_kb":3035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":294,"ondisk_log_size":294,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404500,"kb_used_data":13736,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124537324,"statfs":{"total":1151940427776,"available":1151526219776,"internally_reserved":0,"allocated":14065664,"data_stored":7650357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":400012602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":41440,"num_objects":56,"num_object_clones":0,"num_object_copies":169,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":29,"num_read_kb":45,"num_write":131,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003325"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.698018+0000","last_change":"2024-04-02T13:51:01.253563+0000","last_active":"2024-04-02T13:51:06.698018+0000","last_peered":"2024-04-02T13:51:06.698018+0000","last_clean":"2024-04-02T13:51:06.698018+0000","last_became_active":"2024-04-02T13:51:01.253185+0000","last_became_peered":"2024-04-02T13:51:01.253185+0000","last_unstale":"2024-04-02T13:51:06.698018+0000","last_undegraded":"2024-04-02T13:51:06.698018+0000","last_fullsized":"2024-04-02T13:51:06.698018+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:51:05.301880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617002+0000","last_change":"2024-04-02T13:51:01.256406+0000","last_active":"2024-04-02T13:51:06.617002+0000","last_peered":"2024-04-02T13:51:06.617002+0000","last_clean":"2024-04-02T13:51:06.617002+0000","last_became_active":"2024-04-02T13:51:01.256068+0000","last_became_peered":"2024-04-02T13:51:01.256068+0000","last_unstale":"2024-04-02T13:51:06.617002+0000","last_undegraded":"2024-04-02T13:51:06.617002+0000","last_fullsized":"2024-04-02T13:51:06.617002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:56:51.184932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617399+0000","last_change":"2024-04-02T13:51:01.265925+0000","last_active":"2024-04-02T13:51:06.617399+0000","last_peered":"2024-04-02T13:51:06.617399+0000","last_clean":"2024-04-02T13:51:06.617399+0000","last_became_active":"2024-04-02T13:51:01.265587+0000","last_became_peered":"2024-04-02T13:51:01.265587+0000","last_unstale":"2024-04-02T13:51:06.617399+0000","last_undegraded":"2024-04-02T13:51:06.617399+0000","last_fullsized":"2024-04-02T13:51:06.617399+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:40:59.426886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.618392+0000","last_change":"2024-04-02T13:51:01.251521+0000","last_active":"2024-04-02T13:51:06.618392+0000","last_peered":"2024-04-02T13:51:06.618392+0000","last_clean":"2024-04-02T13:51:06.618392+0000","last_became_active":"2024-04-02T13:51:01.251130+0000","last_became_peered":"2024-04-02T13:51:01.251130+0000","last_unstale":"2024-04-02T13:51:06.618392+0000","last_undegraded":"2024-04-02T13:51:06.618392+0000","last_fullsized":"2024-04-02T13:51:06.618392+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:04:16.836381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616240+0000","last_change":"2024-04-02T13:51:01.250934+0000","last_active":"2024-04-02T13:51:06.616240+0000","last_peered":"2024-04-02T13:51:06.616240+0000","last_clean":"2024-04-02T13:51:06.616240+0000","last_became_active":"2024-04-02T13:51:01.250546+0000","last_became_peered":"2024-04-02T13:51:01.250546+0000","last_unstale":"2024-04-02T13:51:06.616240+0000","last_undegraded":"2024-04-02T13:51:06.616240+0000","last_fullsized":"2024-04-02T13:51:06.616240+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:34:17.839207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617366+0000","last_change":"2024-04-02T13:51:01.256160+0000","last_active":"2024-04-02T13:51:06.617366+0000","last_peered":"2024-04-02T13:51:06.617366+0000","last_clean":"2024-04-02T13:51:06.617366+0000","last_became_active":"2024-04-02T13:51:01.255776+0000","last_became_peered":"2024-04-02T13:51:01.255776+0000","last_unstale":"2024-04-02T13:51:06.617366+0000","last_undegraded":"2024-04-02T13:51:06.617366+0000","last_fullsized":"2024-04-02T13:51:06.617366+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:53:16.724662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616681+0000","last_change":"2024-04-02T13:51:01.263049+0000","last_active":"2024-04-02T13:51:06.616681+0000","last_peered":"2024-04-02T13:51:06.616681+0000","last_clean":"2024-04-02T13:51:06.616681+0000","last_became_active":"2024-04-02T13:51:01.262660+0000","last_became_peered":"2024-04-02T13:51:01.262660+0000","last_unstale":"2024-04-02T13:51:06.616681+0000","last_undegraded":"2024-04-02T13:51:06.616681+0000","last_fullsized":"2024-04-02T13:51:06.616681+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:15:32.069188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616185+0000","last_change":"2024-04-02T13:51:01.263064+0000","last_active":"2024-04-02T13:51:06.616185+0000","last_peered":"2024-04-02T13:51:06.616185+0000","last_clean":"2024-04-02T13:51:06.616185+0000","last_became_active":"2024-04-02T13:51:01.262706+0000","last_became_peered":"2024-04-02T13:51:01.262706+0000","last_unstale":"2024-04-02T13:51:06.616185+0000","last_undegraded":"2024-04-02T13:51:06.616185+0000","last_fullsized":"2024-04-02T13:51:06.616185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:46.680723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.349914+0000","last_change":"2024-04-02T13:51:01.257347+0000","last_active":"2024-04-02T13:51:06.349914+0000","last_peered":"2024-04-02T13:51:06.349914+0000","last_clean":"2024-04-02T13:51:06.349914+0000","last_became_active":"2024-04-02T13:51:01.257001+0000","last_became_peered":"2024-04-02T13:51:01.257001+0000","last_unstale":"2024-04-02T13:51:06.349914+0000","last_undegraded":"2024-04-02T13:51:06.349914+0000","last_fullsized":"2024-04-02T13:51:06.349914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:31:22.628436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617067+0000","last_change":"2024-04-02T13:51:01.260533+0000","last_active":"2024-04-02T13:51:06.617067+0000","last_peered":"2024-04-02T13:51:06.617067+0000","last_clean":"2024-04-02T13:51:06.617067+0000","last_became_active":"2024-04-02T13:51:01.260203+0000","last_became_peered":"2024-04-02T13:51:01.260203+0000","last_unstale":"2024-04-02T13:51:06.617067+0000","last_undegraded":"2024-04-02T13:51:06.617067+0000","last_fullsized":"2024-04-02T13:51:06.617067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:00:06.008243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617658+0000","last_change":"2024-04-02T13:51:01.267733+0000","last_active":"2024-04-02T13:51:06.617658+0000","last_peered":"2024-04-02T13:51:06.617658+0000","last_clean":"2024-04-02T13:51:06.617658+0000","last_became_active":"2024-04-02T13:51:01.266642+0000","last_became_peered":"2024-04-02T13:51:01.266642+0000","last_unstale":"2024-04-02T13:51:06.617658+0000","last_undegraded":"2024-04-02T13:51:06.617658+0000","last_fullsized":"2024-04-02T13:51:06.617658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:39:25.696112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.618369+0000","last_change":"2024-04-02T13:51:01.255933+0000","last_active":"2024-04-02T13:51:06.618369+0000","last_peered":"2024-04-02T13:51:06.618369+0000","last_clean":"2024-04-02T13:51:06.618369+0000","last_became_active":"2024-04-02T13:51:01.255558+0000","last_became_peered":"2024-04-02T13:51:01.255558+0000","last_unstale":"2024-04-02T13:51:06.618369+0000","last_undegraded":"2024-04-02T13:51:06.618369+0000","last_fullsized":"2024-04-02T13:51:06.618369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:13:11.537415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347761+0000","last_change":"2024-04-02T13:51:01.256242+0000","last_active":"2024-04-02T13:51:06.347761+0000","last_peered":"2024-04-02T13:51:06.347761+0000","last_clean":"2024-04-02T13:51:06.347761+0000","last_became_active":"2024-04-02T13:51:01.255830+0000","last_became_peered":"2024-04-02T13:51:01.255830+0000","last_unstale":"2024-04-02T13:51:06.347761+0000","last_undegraded":"2024-04-02T13:51:06.347761+0000","last_fullsized":"2024-04-02T13:51:06.347761+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:39.130894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616082+0000","last_change":"2024-04-02T13:51:01.259738+0000","last_active":"2024-04-02T13:51:06.616082+0000","last_peered":"2024-04-02T13:51:06.616082+0000","last_clean":"2024-04-02T13:51:06.616082+0000","last_became_active":"2024-04-02T13:51:01.258630+0000","last_became_peered":"2024-04-02T13:51:01.258630+0000","last_unstale":"2024-04-02T13:51:06.616082+0000","last_undegraded":"2024-04-02T13:51:06.616082+0000","last_fullsized":"2024-04-02T13:51:06.616082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:12:05.401171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616233+0000","last_change":"2024-04-02T13:51:01.257684+0000","last_active":"2024-04-02T13:51:06.616233+0000","last_peered":"2024-04-02T13:51:06.616233+0000","last_clean":"2024-04-02T13:51:06.616233+0000","last_became_active":"2024-04-02T13:51:01.257267+0000","last_became_peered":"2024-04-02T13:51:01.257267+0000","last_unstale":"2024-04-02T13:51:06.616233+0000","last_undegraded":"2024-04-02T13:51:06.616233+0000","last_fullsized":"2024-04-02T13:51:06.616233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:43:45.728233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.349177+0000","last_change":"2024-04-02T13:51:01.261720+0000","last_active":"2024-04-02T13:51:06.349177+0000","last_peered":"2024-04-02T13:51:06.349177+0000","last_clean":"2024-04-02T13:51:06.349177+0000","last_became_active":"2024-04-02T13:51:01.261407+0000","last_became_peered":"2024-04-02T13:51:01.261407+0000","last_unstale":"2024-04-02T13:51:06.349177+0000","last_undegraded":"2024-04-02T13:51:06.349177+0000","last_fullsized":"2024-04-02T13:51:06.349177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:54:58.807606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.697829+0000","last_change":"2024-04-02T13:51:01.254488+0000","last_active":"2024-04-02T13:51:06.697829+0000","last_peered":"2024-04-02T13:51:06.697829+0000","last_clean":"2024-04-02T13:51:06.697829+0000","last_became_active":"2024-04-02T13:51:01.254139+0000","last_became_peered":"2024-04-02T13:51:01.254139+0000","last_unstale":"2024-04-02T13:51:06.697829+0000","last_undegraded":"2024-04-02T13:51:06.697829+0000","last_fullsized":"2024-04-02T13:51:06.697829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:02:41.075727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616657+0000","last_change":"2024-04-02T13:51:01.256107+0000","last_active":"2024-04-02T13:51:06.616657+0000","last_peered":"2024-04-02T13:51:06.616657+0000","last_clean":"2024-04-02T13:51:06.616657+0000","last_became_active":"2024-04-02T13:51:01.255754+0000","last_became_peered":"2024-04-02T13:51:01.255754+0000","last_unstale":"2024-04-02T13:51:06.616657+0000","last_undegraded":"2024-04-02T13:51:06.616657+0000","last_fullsized":"2024-04-02T13:51:06.616657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:36:59.793519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.354581+0000","last_change":"2024-04-02T13:51:01.247550+0000","last_active":"2024-04-02T13:51:06.354581+0000","last_peered":"2024-04-02T13:51:06.354581+0000","last_clean":"2024-04-02T13:51:06.354581+0000","last_became_active":"2024-04-02T13:51:01.247236+0000","last_became_peered":"2024-04-02T13:51:01.247236+0000","last_unstale":"2024-04-02T13:51:06.354581+0000","last_undegraded":"2024-04-02T13:51:06.354581+0000","last_fullsized":"2024-04-02T13:51:06.354581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:16:02.940700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.354234+0000","last_change":"2024-04-02T13:51:05.365182+0000","last_active":"2024-04-02T13:51:06.354234+0000","last_peered":"2024-04-02T13:51:06.354234+0000","last_clean":"2024-04-02T13:51:06.354234+0000","last_became_active":"2024-04-02T13:51:05.364713+0000","last_became_peered":"2024-04-02T13:51:05.364713+0000","last_unstale":"2024-04-02T13:51:06.354234+0000","last_undegraded":"2024-04-02T13:51:06.354234+0000","last_fullsized":"2024-04-02T13:51:06.354234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:14:34.763192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'11","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:27.345883+0000","last_change":"2024-04-02T13:51:00.243244+0000","last_active":"2024-04-02T13:51:27.345883+0000","last_peered":"2024-04-02T13:51:27.345883+0000","last_clean":"2024-04-02T13:51:27.345883+0000","last_became_active":"2024-04-02T13:51:00.242892+0000","last_became_peered":"2024-04-02T13:51:00.242892+0000","last_unstale":"2024-04-02T13:51:27.345883+0000","last_undegraded":"2024-04-02T13:51:27.345883+0000","last_fullsized":"2024-04-02T13:51:27.345883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:49:23.338215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617419+0000","last_change":"2024-04-02T13:51:01.264640+0000","last_active":"2024-04-02T13:51:06.617419+0000","last_peered":"2024-04-02T13:51:06.617419+0000","last_clean":"2024-04-02T13:51:06.617419+0000","last_became_active":"2024-04-02T13:51:01.264161+0000","last_became_peered":"2024-04-02T13:51:01.264161+0000","last_unstale":"2024-04-02T13:51:06.617419+0000","last_undegraded":"2024-04-02T13:51:06.617419+0000","last_fullsized":"2024-04-02T13:51:06.617419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:30:47.245647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617116+0000","last_change":"2024-04-02T13:51:05.362888+0000","last_active":"2024-04-02T13:51:06.617116+0000","last_peered":"2024-04-02T13:51:06.617116+0000","last_clean":"2024-04-02T13:51:06.617116+0000","last_became_active":"2024-04-02T13:51:05.362478+0000","last_became_peered":"2024-04-02T13:51:05.362478+0000","last_unstale":"2024-04-02T13:51:06.617116+0000","last_undegraded":"2024-04-02T13:51:06.617116+0000","last_fullsized":"2024-04-02T13:51:06.617116+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:05:54.304458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617581+0000","last_change":"2024-04-02T13:51:00.247800+0000","last_active":"2024-04-02T13:51:06.617581+0000","last_peered":"2024-04-02T13:51:06.617581+0000","last_clean":"2024-04-02T13:51:06.617581+0000","last_became_active":"2024-04-02T13:51:00.247485+0000","last_became_peered":"2024-04-02T13:51:00.247485+0000","last_unstale":"2024-04-02T13:51:06.617581+0000","last_undegraded":"2024-04-02T13:51:06.617581+0000","last_fullsized":"2024-04-02T13:51:06.617581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:17:24.767558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616088+0000","last_change":"2024-04-02T13:51:01.261416+0000","last_active":"2024-04-02T13:51:06.616088+0000","last_peered":"2024-04-02T13:51:06.616088+0000","last_clean":"2024-04-02T13:51:06.616088+0000","last_became_active":"2024-04-02T13:51:01.261099+0000","last_became_peered":"2024-04-02T13:51:01.261099+0000","last_unstale":"2024-04-02T13:51:06.616088+0000","last_undegraded":"2024-04-02T13:51:06.616088+0000","last_fullsized":"2024-04-02T13:51:06.616088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:10:04.999402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617900+0000","last_change":"2024-04-02T13:51:05.361747+0000","last_active":"2024-04-02T13:51:06.617900+0000","last_peered":"2024-04-02T13:51:06.617900+0000","last_clean":"2024-04-02T13:51:06.617900+0000","last_became_active":"2024-04-02T13:51:05.361098+0000","last_became_peered":"2024-04-02T13:51:05.361098+0000","last_unstale":"2024-04-02T13:51:06.617900+0000","last_undegraded":"2024-04-02T13:51:06.617900+0000","last_fullsized":"2024-04-02T13:51:06.617900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:45:35.048870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:26.336320+0000","last_change":"2024-04-02T13:51:00.244048+0000","last_active":"2024-04-02T13:51:26.336320+0000","last_peered":"2024-04-02T13:51:26.336320+0000","last_clean":"2024-04-02T13:51:26.336320+0000","last_became_active":"2024-04-02T13:51:00.243510+0000","last_became_peered":"2024-04-02T13:51:00.243510+0000","last_unstale":"2024-04-02T13:51:26.336320+0000","last_undegraded":"2024-04-02T13:51:26.336320+0000","last_fullsized":"2024-04-02T13:51:26.336320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:53:48.364137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.349854+0000","last_change":"2024-04-02T13:51:01.264711+0000","last_active":"2024-04-02T13:51:06.349854+0000","last_peered":"2024-04-02T13:51:06.349854+0000","last_clean":"2024-04-02T13:51:06.349854+0000","last_became_active":"2024-04-02T13:51:01.264289+0000","last_became_peered":"2024-04-02T13:51:01.264289+0000","last_unstale":"2024-04-02T13:51:06.349854+0000","last_undegraded":"2024-04-02T13:51:06.349854+0000","last_fullsized":"2024-04-02T13:51:06.349854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:34:41.590071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617854+0000","last_change":"2024-04-02T13:51:05.363121+0000","last_active":"2024-04-02T13:51:06.617854+0000","last_peered":"2024-04-02T13:51:06.617854+0000","last_clean":"2024-04-02T13:51:06.617854+0000","last_became_active":"2024-04-02T13:51:05.362642+0000","last_became_peered":"2024-04-02T13:51:05.362642+0000","last_unstale":"2024-04-02T13:51:06.617854+0000","last_undegraded":"2024-04-02T13:51:06.617854+0000","last_fullsized":"2024-04-02T13:51:06.617854+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:55:40.848731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616416+0000","last_change":"2024-04-02T13:51:00.236519+0000","last_active":"2024-04-02T13:51:06.616416+0000","last_peered":"2024-04-02T13:51:06.616416+0000","last_clean":"2024-04-02T13:51:06.616416+0000","last_became_active":"2024-04-02T13:51:00.235985+0000","last_became_peered":"2024-04-02T13:51:00.235985+0000","last_unstale":"2024-04-02T13:51:06.616416+0000","last_undegraded":"2024-04-02T13:51:06.616416+0000","last_fullsized":"2024-04-02T13:51:06.616416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:08.321765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347657+0000","last_change":"2024-04-02T13:51:01.260333+0000","last_active":"2024-04-02T13:51:06.347657+0000","last_peered":"2024-04-02T13:51:06.347657+0000","last_clean":"2024-04-02T13:51:06.347657+0000","last_became_active":"2024-04-02T13:51:01.259995+0000","last_became_peered":"2024-04-02T13:51:01.259995+0000","last_unstale":"2024-04-02T13:51:06.347657+0000","last_undegraded":"2024-04-02T13:51:06.347657+0000","last_fullsized":"2024-04-02T13:51:06.347657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:03:58.088653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.354511+0000","last_change":"2024-04-02T13:51:05.359281+0000","last_active":"2024-04-02T13:51:06.354511+0000","last_peered":"2024-04-02T13:51:06.354511+0000","last_clean":"2024-04-02T13:51:06.354511+0000","last_became_active":"2024-04-02T13:51:05.358831+0000","last_became_peered":"2024-04-02T13:51:05.358831+0000","last_unstale":"2024-04-02T13:51:06.354511+0000","last_undegraded":"2024-04-02T13:51:06.354511+0000","last_fullsized":"2024-04-02T13:51:06.354511+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:48.461298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:29.355832+0000","last_change":"2024-04-02T13:51:00.241707+0000","last_active":"2024-04-02T13:51:29.355832+0000","last_peered":"2024-04-02T13:51:29.355832+0000","last_clean":"2024-04-02T13:51:29.355832+0000","last_became_active":"2024-04-02T13:51:00.241225+0000","last_became_peered":"2024-04-02T13:51:00.241225+0000","last_unstale":"2024-04-02T13:51:29.355832+0000","last_undegraded":"2024-04-02T13:51:29.355832+0000","last_fullsized":"2024-04-02T13:51:29.355832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:59:44.537503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616279+0000","last_change":"2024-04-02T13:51:01.252942+0000","last_active":"2024-04-02T13:51:06.616279+0000","last_peered":"2024-04-02T13:51:06.616279+0000","last_clean":"2024-04-02T13:51:06.616279+0000","last_became_active":"2024-04-02T13:51:01.252631+0000","last_became_peered":"2024-04-02T13:51:01.252631+0000","last_unstale":"2024-04-02T13:51:06.616279+0000","last_undegraded":"2024-04-02T13:51:06.616279+0000","last_fullsized":"2024-04-02T13:51:06.616279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:43:24.725688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.354285+0000","last_change":"2024-04-02T13:51:05.359208+0000","last_active":"2024-04-02T13:51:06.354285+0000","last_peered":"2024-04-02T13:51:06.354285+0000","last_clean":"2024-04-02T13:51:06.354285+0000","last_became_active":"2024-04-02T13:51:05.358738+0000","last_became_peered":"2024-04-02T13:51:05.358738+0000","last_unstale":"2024-04-02T13:51:06.354285+0000","last_undegraded":"2024-04-02T13:51:06.354285+0000","last_fullsized":"2024-04-02T13:51:06.354285+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:47:46.034562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:27.343299+0000","last_change":"2024-04-02T13:51:00.236413+0000","last_active":"2024-04-02T13:51:27.343299+0000","last_peered":"2024-04-02T13:51:27.343299+0000","last_clean":"2024-04-02T13:51:27.343299+0000","last_became_active":"2024-04-02T13:51:00.236046+0000","last_became_peered":"2024-04-02T13:51:00.236046+0000","last_unstale":"2024-04-02T13:51:27.343299+0000","last_undegraded":"2024-04-02T13:51:27.343299+0000","last_fullsized":"2024-04-02T13:51:27.343299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:25:15.000726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347640+0000","last_change":"2024-04-02T13:51:01.265471+0000","last_active":"2024-04-02T13:51:06.347640+0000","last_peered":"2024-04-02T13:51:06.347640+0000","last_clean":"2024-04-02T13:51:06.347640+0000","last_became_active":"2024-04-02T13:51:01.265114+0000","last_became_peered":"2024-04-02T13:51:01.265114+0000","last_unstale":"2024-04-02T13:51:06.347640+0000","last_undegraded":"2024-04-02T13:51:06.347640+0000","last_fullsized":"2024-04-02T13:51:06.347640+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:01:01.675686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.697886+0000","last_change":"2024-04-02T13:51:05.366813+0000","last_active":"2024-04-02T13:51:06.697886+0000","last_peered":"2024-04-02T13:51:06.697886+0000","last_clean":"2024-04-02T13:51:06.697886+0000","last_became_active":"2024-04-02T13:51:05.366317+0000","last_became_peered":"2024-04-02T13:51:05.366317+0000","last_unstale":"2024-04-02T13:51:06.697886+0000","last_undegraded":"2024-04-02T13:51:06.697886+0000","last_fullsized":"2024-04-02T13:51:06.697886+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:03:54.156620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347405+0000","last_change":"2024-04-02T13:51:00.237204+0000","last_active":"2024-04-02T13:51:06.347405+0000","last_peered":"2024-04-02T13:51:06.347405+0000","last_clean":"2024-04-02T13:51:06.347405+0000","last_became_active":"2024-04-02T13:51:00.236707+0000","last_became_peered":"2024-04-02T13:51:00.236707+0000","last_unstale":"2024-04-02T13:51:06.347405+0000","last_undegraded":"2024-04-02T13:51:06.347405+0000","last_fullsized":"2024-04-02T13:51:06.347405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:00:17.771428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616471+0000","last_change":"2024-04-02T13:51:01.250152+0000","last_active":"2024-04-02T13:51:06.616471+0000","last_peered":"2024-04-02T13:51:06.616471+0000","last_clean":"2024-04-02T13:51:06.616471+0000","last_became_active":"2024-04-02T13:51:01.249817+0000","last_became_peered":"2024-04-02T13:51:01.249817+0000","last_unstale":"2024-04-02T13:51:06.616471+0000","last_undegraded":"2024-04-02T13:51:06.616471+0000","last_fullsized":"2024-04-02T13:51:06.616471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:13:31.719380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617106+0000","last_change":"2024-04-02T13:51:05.366551+0000","last_active":"2024-04-02T13:51:06.617106+0000","last_peered":"2024-04-02T13:51:06.617106+0000","last_clean":"2024-04-02T13:51:06.617106+0000","last_became_active":"2024-04-02T13:51:05.366175+0000","last_became_peered":"2024-04-02T13:51:05.366175+0000","last_unstale":"2024-04-02T13:51:06.617106+0000","last_undegraded":"2024-04-02T13:51:06.617106+0000","last_fullsized":"2024-04-02T13:51:06.617106+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:47:49.822918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:28.352356+0000","last_change":"2024-04-02T13:51:00.239521+0000","last_active":"2024-04-02T13:51:28.352356+0000","last_peered":"2024-04-02T13:51:28.352356+0000","last_clean":"2024-04-02T13:51:28.352356+0000","last_became_active":"2024-04-02T13:51:00.238608+0000","last_became_peered":"2024-04-02T13:51:00.238608+0000","last_unstale":"2024-04-02T13:51:28.352356+0000","last_undegraded":"2024-04-02T13:51:28.352356+0000","last_fullsized":"2024-04-02T13:51:28.352356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:00.983006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617394+0000","last_change":"2024-04-02T13:51:01.255021+0000","last_active":"2024-04-02T13:51:06.617394+0000","last_peered":"2024-04-02T13:51:06.617394+0000","last_clean":"2024-04-02T13:51:06.617394+0000","last_became_active":"2024-04-02T13:51:01.254630+0000","last_became_peered":"2024-04-02T13:51:01.254630+0000","last_unstale":"2024-04-02T13:51:06.617394+0000","last_undegraded":"2024-04-02T13:51:06.617394+0000","last_fullsized":"2024-04-02T13:51:06.617394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:36:08.773431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616756+0000","last_change":"2024-04-02T13:51:05.362004+0000","last_active":"2024-04-02T13:51:06.616756+0000","last_peered":"2024-04-02T13:51:06.616756+0000","last_clean":"2024-04-02T13:51:06.616756+0000","last_became_active":"2024-04-02T13:51:05.361538+0000","last_became_peered":"2024-04-02T13:51:05.361538+0000","last_unstale":"2024-04-02T13:51:06.616756+0000","last_undegraded":"2024-04-02T13:51:06.616756+0000","last_fullsized":"2024-04-02T13:51:06.616756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:27:21.728695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:29.352274+0000","last_change":"2024-04-02T13:51:00.234814+0000","last_active":"2024-04-02T13:51:29.352274+0000","last_peered":"2024-04-02T13:51:29.352274+0000","last_clean":"2024-04-02T13:51:29.352274+0000","last_became_active":"2024-04-02T13:51:00.234481+0000","last_became_peered":"2024-04-02T13:51:00.234481+0000","last_unstale":"2024-04-02T13:51:29.352274+0000","last_undegraded":"2024-04-02T13:51:29.352274+0000","last_fullsized":"2024-04-02T13:51:29.352274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:45:34.901226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.349799+0000","last_change":"2024-04-02T13:51:01.264093+0000","last_active":"2024-04-02T13:51:06.349799+0000","last_peered":"2024-04-02T13:51:06.349799+0000","last_clean":"2024-04-02T13:51:06.349799+0000","last_became_active":"2024-04-02T13:51:01.263754+0000","last_became_peered":"2024-04-02T13:51:01.263754+0000","last_unstale":"2024-04-02T13:51:06.349799+0000","last_undegraded":"2024-04-02T13:51:06.349799+0000","last_fullsized":"2024-04-02T13:51:06.349799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:49:18.630195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617911+0000","last_change":"2024-04-02T13:51:05.370930+0000","last_active":"2024-04-02T13:51:06.617911+0000","last_peered":"2024-04-02T13:51:06.617911+0000","last_clean":"2024-04-02T13:51:06.617911+0000","last_became_active":"2024-04-02T13:51:05.370443+0000","last_became_peered":"2024-04-02T13:51:05.370443+0000","last_unstale":"2024-04-02T13:51:06.617911+0000","last_undegraded":"2024-04-02T13:51:06.617911+0000","last_fullsized":"2024-04-02T13:51:06.617911+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:24:27.055494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:25.332528+0000","last_change":"2024-04-02T13:51:00.241206+0000","last_active":"2024-04-02T13:51:25.332528+0000","last_peered":"2024-04-02T13:51:25.332528+0000","last_clean":"2024-04-02T13:51:25.332528+0000","last_became_active":"2024-04-02T13:51:00.240832+0000","last_became_peered":"2024-04-02T13:51:00.240832+0000","last_unstale":"2024-04-02T13:51:25.332528+0000","last_undegraded":"2024-04-02T13:51:25.332528+0000","last_fullsized":"2024-04-02T13:51:25.332528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:44:11.626435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617673+0000","last_change":"2024-04-02T13:51:01.259315+0000","last_active":"2024-04-02T13:51:06.617673+0000","last_peered":"2024-04-02T13:51:06.617673+0000","last_clean":"2024-04-02T13:51:06.617673+0000","last_became_active":"2024-04-02T13:51:01.258951+0000","last_became_peered":"2024-04-02T13:51:01.258951+0000","last_unstale":"2024-04-02T13:51:06.617673+0000","last_undegraded":"2024-04-02T13:51:06.617673+0000","last_fullsized":"2024-04-02T13:51:06.617673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:59:43.714044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.618115+0000","last_change":"2024-04-02T13:51:05.361495+0000","last_active":"2024-04-02T13:51:06.618115+0000","last_peered":"2024-04-02T13:51:06.618115+0000","last_clean":"2024-04-02T13:51:06.618115+0000","last_became_active":"2024-04-02T13:51:05.360546+0000","last_became_peered":"2024-04-02T13:51:05.360546+0000","last_unstale":"2024-04-02T13:51:06.618115+0000","last_undegraded":"2024-04-02T13:51:06.618115+0000","last_fullsized":"2024-04-02T13:51:06.618115+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:23:59.853497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:27.341944+0000","last_change":"2024-04-02T13:51:00.230938+0000","last_active":"2024-04-02T13:51:27.341944+0000","last_peered":"2024-04-02T13:51:27.341944+0000","last_clean":"2024-04-02T13:51:27.341944+0000","last_became_active":"2024-04-02T13:51:00.230585+0000","last_became_peered":"2024-04-02T13:51:00.230585+0000","last_unstale":"2024-04-02T13:51:27.341944+0000","last_undegraded":"2024-04-02T13:51:27.341944+0000","last_fullsized":"2024-04-02T13:51:27.341944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:51:37.463876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.350041+0000","last_change":"2024-04-02T13:51:01.264216+0000","last_active":"2024-04-02T13:51:06.350041+0000","last_peered":"2024-04-02T13:51:06.350041+0000","last_clean":"2024-04-02T13:51:06.350041+0000","last_became_active":"2024-04-02T13:51:01.263824+0000","last_became_peered":"2024-04-02T13:51:01.263824+0000","last_unstale":"2024-04-02T13:51:06.350041+0000","last_undegraded":"2024-04-02T13:51:06.350041+0000","last_fullsized":"2024-04-02T13:51:06.350041+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:16:26.391163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617882+0000","last_change":"2024-04-02T13:51:05.366002+0000","last_active":"2024-04-02T13:51:06.617882+0000","last_peered":"2024-04-02T13:51:06.617882+0000","last_clean":"2024-04-02T13:51:06.617882+0000","last_became_active":"2024-04-02T13:51:05.365547+0000","last_became_peered":"2024-04-02T13:51:05.365547+0000","last_unstale":"2024-04-02T13:51:06.617882+0000","last_undegraded":"2024-04-02T13:51:06.617882+0000","last_fullsized":"2024-04-02T13:51:06.617882+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:31:38.990617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'25","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:29.460868+0000","last_change":"2024-04-02T13:51:00.233925+0000","last_active":"2024-04-02T13:51:29.460868+0000","last_peered":"2024-04-02T13:51:29.460868+0000","last_clean":"2024-04-02T13:51:29.460868+0000","last_became_active":"2024-04-02T13:51:00.233574+0000","last_became_peered":"2024-04-02T13:51:00.233574+0000","last_unstale":"2024-04-02T13:51:29.460868+0000","last_undegraded":"2024-04-02T13:51:29.460868+0000","last_fullsized":"2024-04-02T13:51:29.460868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:33:32.758874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40471,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":24,"num_write":27,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616529+0000","last_change":"2024-04-02T13:51:01.258825+0000","last_active":"2024-04-02T13:51:06.616529+0000","last_peered":"2024-04-02T13:51:06.616529+0000","last_clean":"2024-04-02T13:51:06.616529+0000","last_became_active":"2024-04-02T13:51:01.258500+0000","last_became_peered":"2024-04-02T13:51:01.258500+0000","last_unstale":"2024-04-02T13:51:06.616529+0000","last_undegraded":"2024-04-02T13:51:06.616529+0000","last_fullsized":"2024-04-02T13:51:06.616529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:33:15.409765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.348101+0000","last_change":"2024-04-02T13:51:05.365010+0000","last_active":"2024-04-02T13:51:06.348101+0000","last_peered":"2024-04-02T13:51:06.348101+0000","last_clean":"2024-04-02T13:51:06.348101+0000","last_became_active":"2024-04-02T13:51:05.364499+0000","last_became_peered":"2024-04-02T13:51:05.364499+0000","last_unstale":"2024-04-02T13:51:06.348101+0000","last_undegraded":"2024-04-02T13:51:06.348101+0000","last_fullsized":"2024-04-02T13:51:06.348101+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:07:03.500017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:29.351762+0000","last_change":"2024-04-02T13:51:00.233459+0000","last_active":"2024-04-02T13:51:29.351762+0000","last_peered":"2024-04-02T13:51:29.351762+0000","last_clean":"2024-04-02T13:51:29.351762+0000","last_became_active":"2024-04-02T13:51:00.232424+0000","last_became_peered":"2024-04-02T13:51:00.232424+0000","last_unstale":"2024-04-02T13:51:29.351762+0000","last_undegraded":"2024-04-02T13:51:29.351762+0000","last_fullsized":"2024-04-02T13:51:29.351762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:26:20.877553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347581+0000","last_change":"2024-04-02T13:51:01.256334+0000","last_active":"2024-04-02T13:51:06.347581+0000","last_peered":"2024-04-02T13:51:06.347581+0000","last_clean":"2024-04-02T13:51:06.347581+0000","last_became_active":"2024-04-02T13:51:01.255964+0000","last_became_peered":"2024-04-02T13:51:01.255964+0000","last_unstale":"2024-04-02T13:51:06.347581+0000","last_undegraded":"2024-04-02T13:51:06.347581+0000","last_fullsized":"2024-04-02T13:51:06.347581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:43:04.842812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.697541+0000","last_change":"2024-04-02T13:51:05.362696+0000","last_active":"2024-04-02T13:51:06.697541+0000","last_peered":"2024-04-02T13:51:06.697541+0000","last_clean":"2024-04-02T13:51:06.697541+0000","last_became_active":"2024-04-02T13:51:05.362225+0000","last_became_peered":"2024-04-02T13:51:05.362225+0000","last_unstale":"2024-04-02T13:51:06.697541+0000","last_undegraded":"2024-04-02T13:51:06.697541+0000","last_fullsized":"2024-04-02T13:51:06.697541+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:23:56.308286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:28.350739+0000","last_change":"2024-04-02T13:51:00.239673+0000","last_active":"2024-04-02T13:51:28.350739+0000","last_peered":"2024-04-02T13:51:28.350739+0000","last_clean":"2024-04-02T13:51:28.350739+0000","last_became_active":"2024-04-02T13:51:00.239233+0000","last_became_peered":"2024-04-02T13:51:00.239233+0000","last_unstale":"2024-04-02T13:51:28.350739+0000","last_undegraded":"2024-04-02T13:51:28.350739+0000","last_fullsized":"2024-04-02T13:51:28.350739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:05:48.119953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617153+0000","last_change":"2024-04-02T13:51:01.257474+0000","last_active":"2024-04-02T13:51:06.617153+0000","last_peered":"2024-04-02T13:51:06.617153+0000","last_clean":"2024-04-02T13:51:06.617153+0000","last_became_active":"2024-04-02T13:51:01.254977+0000","last_became_peered":"2024-04-02T13:51:01.254977+0000","last_unstale":"2024-04-02T13:51:06.617153+0000","last_undegraded":"2024-04-02T13:51:06.617153+0000","last_fullsized":"2024-04-02T13:51:06.617153+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:13:07.446520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617920+0000","last_change":"2024-04-02T13:51:05.371531+0000","last_active":"2024-04-02T13:51:06.617920+0000","last_peered":"2024-04-02T13:51:06.617920+0000","last_clean":"2024-04-02T13:51:06.617920+0000","last_became_active":"2024-04-02T13:51:05.370551+0000","last_became_peered":"2024-04-02T13:51:05.370551+0000","last_unstale":"2024-04-02T13:51:06.617920+0000","last_undegraded":"2024-04-02T13:51:06.617920+0000","last_fullsized":"2024-04-02T13:51:06.617920+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:43:48.796009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:29.353719+0000","last_change":"2024-04-02T13:51:00.237715+0000","last_active":"2024-04-02T13:51:29.353719+0000","last_peered":"2024-04-02T13:51:29.353719+0000","last_clean":"2024-04-02T13:51:29.353719+0000","last_became_active":"2024-04-02T13:51:00.237349+0000","last_became_peered":"2024-04-02T13:51:00.237349+0000","last_unstale":"2024-04-02T13:51:29.353719+0000","last_undegraded":"2024-04-02T13:51:29.353719+0000","last_fullsized":"2024-04-02T13:51:29.353719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:33:20.651562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616896+0000","last_change":"2024-04-02T13:51:01.259447+0000","last_active":"2024-04-02T13:51:06.616896+0000","last_peered":"2024-04-02T13:51:06.616896+0000","last_clean":"2024-04-02T13:51:06.616896+0000","last_became_active":"2024-04-02T13:51:01.259101+0000","last_became_peered":"2024-04-02T13:51:01.259101+0000","last_unstale":"2024-04-02T13:51:06.616896+0000","last_undegraded":"2024-04-02T13:51:06.616896+0000","last_fullsized":"2024-04-02T13:51:06.616896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:51:44.311440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.348037+0000","last_change":"2024-04-02T13:51:05.372446+0000","last_active":"2024-04-02T13:51:06.348037+0000","last_peered":"2024-04-02T13:51:06.348037+0000","last_clean":"2024-04-02T13:51:06.348037+0000","last_became_active":"2024-04-02T13:51:05.372007+0000","last_became_peered":"2024-04-02T13:51:05.372007+0000","last_unstale":"2024-04-02T13:51:06.348037+0000","last_undegraded":"2024-04-02T13:51:06.348037+0000","last_fullsized":"2024-04-02T13:51:06.348037+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:54:23.463179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:26.338319+0000","last_change":"2024-04-02T13:51:00.241492+0000","last_active":"2024-04-02T13:51:26.338319+0000","last_peered":"2024-04-02T13:51:26.338319+0000","last_clean":"2024-04-02T13:51:26.338319+0000","last_became_active":"2024-04-02T13:51:00.241167+0000","last_became_peered":"2024-04-02T13:51:00.241167+0000","last_unstale":"2024-04-02T13:51:26.338319+0000","last_undegraded":"2024-04-02T13:51:26.338319+0000","last_fullsized":"2024-04-02T13:51:26.338319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:44:44.070187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347487+0000","last_change":"2024-04-02T13:51:01.263200+0000","last_active":"2024-04-02T13:51:06.347487+0000","last_peered":"2024-04-02T13:51:06.347487+0000","last_clean":"2024-04-02T13:51:06.347487+0000","last_became_active":"2024-04-02T13:51:01.262812+0000","last_became_peered":"2024-04-02T13:51:01.262812+0000","last_unstale":"2024-04-02T13:51:06.347487+0000","last_undegraded":"2024-04-02T13:51:06.347487+0000","last_fullsized":"2024-04-02T13:51:06.347487+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:33:35.898047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347331+0000","last_change":"2024-04-02T13:51:05.366729+0000","last_active":"2024-04-02T13:51:06.347331+0000","last_peered":"2024-04-02T13:51:06.347331+0000","last_clean":"2024-04-02T13:51:06.347331+0000","last_became_active":"2024-04-02T13:51:05.366257+0000","last_became_peered":"2024-04-02T13:51:05.366257+0000","last_unstale":"2024-04-02T13:51:06.347331+0000","last_undegraded":"2024-04-02T13:51:06.347331+0000","last_fullsized":"2024-04-02T13:51:06.347331+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:46:03.400300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616677+0000","last_change":"2024-04-02T13:51:00.236004+0000","last_active":"2024-04-02T13:51:06.616677+0000","last_peered":"2024-04-02T13:51:06.616677+0000","last_clean":"2024-04-02T13:51:06.616677+0000","last_became_active":"2024-04-02T13:51:00.235160+0000","last_became_peered":"2024-04-02T13:51:00.235160+0000","last_unstale":"2024-04-02T13:51:06.616677+0000","last_undegraded":"2024-04-02T13:51:06.616677+0000","last_fullsized":"2024-04-02T13:51:06.616677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:16:35.617346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.697743+0000","last_change":"2024-04-02T13:51:01.259565+0000","last_active":"2024-04-02T13:51:06.697743+0000","last_peered":"2024-04-02T13:51:06.697743+0000","last_clean":"2024-04-02T13:51:06.697743+0000","last_became_active":"2024-04-02T13:51:01.259225+0000","last_became_peered":"2024-04-02T13:51:01.259225+0000","last_unstale":"2024-04-02T13:51:06.697743+0000","last_undegraded":"2024-04-02T13:51:06.697743+0000","last_fullsized":"2024-04-02T13:51:06.697743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:22:18.878029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616330+0000","last_change":"2024-04-02T13:51:05.518750+0000","last_active":"2024-04-02T13:51:06.616330+0000","last_peered":"2024-04-02T13:51:06.616330+0000","last_clean":"2024-04-02T13:51:06.616330+0000","last_became_active":"2024-04-02T13:51:05.518247+0000","last_became_peered":"2024-04-02T13:51:05.518247+0000","last_unstale":"2024-04-02T13:51:06.616330+0000","last_undegraded":"2024-04-02T13:51:06.616330+0000","last_fullsized":"2024-04-02T13:51:06.616330+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:08:10.232509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616695+0000","last_change":"2024-04-02T13:51:00.241476+0000","last_active":"2024-04-02T13:51:06.616695+0000","last_peered":"2024-04-02T13:51:06.616695+0000","last_clean":"2024-04-02T13:51:06.616695+0000","last_became_active":"2024-04-02T13:51:00.241085+0000","last_became_peered":"2024-04-02T13:51:00.241085+0000","last_unstale":"2024-04-02T13:51:06.616695+0000","last_undegraded":"2024-04-02T13:51:06.616695+0000","last_fullsized":"2024-04-02T13:51:06.616695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:29:01.503901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617647+0000","last_change":"2024-04-02T13:51:01.259791+0000","last_active":"2024-04-02T13:51:06.617647+0000","last_peered":"2024-04-02T13:51:06.617647+0000","last_clean":"2024-04-02T13:51:06.617647+0000","last_became_active":"2024-04-02T13:51:01.259440+0000","last_became_peered":"2024-04-02T13:51:01.259440+0000","last_unstale":"2024-04-02T13:51:06.617647+0000","last_undegraded":"2024-04-02T13:51:06.617647+0000","last_fullsized":"2024-04-02T13:51:06.617647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:43:52.422263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347773+0000","last_change":"2024-04-02T13:51:05.366243+0000","last_active":"2024-04-02T13:51:06.347773+0000","last_peered":"2024-04-02T13:51:06.347773+0000","last_clean":"2024-04-02T13:51:06.347773+0000","last_became_active":"2024-04-02T13:51:05.365801+0000","last_became_peered":"2024-04-02T13:51:05.365801+0000","last_unstale":"2024-04-02T13:51:06.347773+0000","last_undegraded":"2024-04-02T13:51:06.347773+0000","last_fullsized":"2024-04-02T13:51:06.347773+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:33:56.853434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616923+0000","last_change":"2024-04-02T13:51:00.248695+0000","last_active":"2024-04-02T13:51:06.616923+0000","last_peered":"2024-04-02T13:51:06.616923+0000","last_clean":"2024-04-02T13:51:06.616923+0000","last_became_active":"2024-04-02T13:51:00.248313+0000","last_became_peered":"2024-04-02T13:51:00.248313+0000","last_unstale":"2024-04-02T13:51:06.616923+0000","last_undegraded":"2024-04-02T13:51:06.616923+0000","last_fullsized":"2024-04-02T13:51:06.616923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:43:21.410823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617434+0000","last_change":"2024-04-02T13:51:01.260993+0000","last_active":"2024-04-02T13:51:06.617434+0000","last_peered":"2024-04-02T13:51:06.617434+0000","last_clean":"2024-04-02T13:51:06.617434+0000","last_became_active":"2024-04-02T13:51:01.260510+0000","last_became_peered":"2024-04-02T13:51:01.260510+0000","last_unstale":"2024-04-02T13:51:06.617434+0000","last_undegraded":"2024-04-02T13:51:06.617434+0000","last_fullsized":"2024-04-02T13:51:06.617434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:17:04.941822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347875+0000","last_change":"2024-04-02T13:51:05.518455+0000","last_active":"2024-04-02T13:51:06.347875+0000","last_peered":"2024-04-02T13:51:06.347875+0000","last_clean":"2024-04-02T13:51:06.347875+0000","last_became_active":"2024-04-02T13:51:05.517962+0000","last_became_peered":"2024-04-02T13:51:05.517962+0000","last_unstale":"2024-04-02T13:51:06.347875+0000","last_undegraded":"2024-04-02T13:51:06.347875+0000","last_fullsized":"2024-04-02T13:51:06.347875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:19:47.734233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347388+0000","last_change":"2024-04-02T13:51:01.259220+0000","last_active":"2024-04-02T13:51:06.347388+0000","last_peered":"2024-04-02T13:51:06.347388+0000","last_clean":"2024-04-02T13:51:06.347388+0000","last_became_active":"2024-04-02T13:51:01.258890+0000","last_became_peered":"2024-04-02T13:51:01.258890+0000","last_unstale":"2024-04-02T13:51:06.347388+0000","last_undegraded":"2024-04-02T13:51:06.347388+0000","last_fullsized":"2024-04-02T13:51:06.347388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:56:15.444736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347333+0000","last_change":"2024-04-02T13:51:00.246727+0000","last_active":"2024-04-02T13:51:06.347333+0000","last_peered":"2024-04-02T13:51:06.347333+0000","last_clean":"2024-04-02T13:51:06.347333+0000","last_became_active":"2024-04-02T13:51:00.246364+0000","last_became_peered":"2024-04-02T13:51:00.246364+0000","last_unstale":"2024-04-02T13:51:06.347333+0000","last_undegraded":"2024-04-02T13:51:06.347333+0000","last_fullsized":"2024-04-02T13:51:06.347333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:35:13.249972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617351+0000","last_change":"2024-04-02T13:51:05.358506+0000","last_active":"2024-04-02T13:51:06.617351+0000","last_peered":"2024-04-02T13:51:06.617351+0000","last_clean":"2024-04-02T13:51:06.617351+0000","last_became_active":"2024-04-02T13:51:05.358157+0000","last_became_peered":"2024-04-02T13:51:05.358157+0000","last_unstale":"2024-04-02T13:51:06.617351+0000","last_undegraded":"2024-04-02T13:51:06.617351+0000","last_fullsized":"2024-04-02T13:51:06.617351+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:24:15.585124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616819+0000","last_change":"2024-04-02T13:51:01.261939+0000","last_active":"2024-04-02T13:51:06.616819+0000","last_peered":"2024-04-02T13:51:06.616819+0000","last_clean":"2024-04-02T13:51:06.616819+0000","last_became_active":"2024-04-02T13:51:01.261580+0000","last_became_peered":"2024-04-02T13:51:01.261580+0000","last_unstale":"2024-04-02T13:51:06.616819+0000","last_undegraded":"2024-04-02T13:51:06.616819+0000","last_fullsized":"2024-04-02T13:51:06.616819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:22:41.790555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:29.351482+0000","last_change":"2024-04-02T13:51:00.238962+0000","last_active":"2024-04-02T13:51:29.351482+0000","last_peered":"2024-04-02T13:51:29.351482+0000","last_clean":"2024-04-02T13:51:29.351482+0000","last_became_active":"2024-04-02T13:51:00.238475+0000","last_became_peered":"2024-04-02T13:51:00.238475+0000","last_unstale":"2024-04-02T13:51:29.351482+0000","last_undegraded":"2024-04-02T13:51:29.351482+0000","last_fullsized":"2024-04-02T13:51:29.351482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:41:07.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":357,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616515+0000","last_change":"2024-04-02T13:49:47.847355+0000","last_active":"2024-04-02T13:51:06.616515+0000","last_peered":"2024-04-02T13:51:06.616515+0000","last_clean":"2024-04-02T13:51:06.616515+0000","last_became_active":"2024-04-02T13:49:47.846768+0000","last_became_peered":"2024-04-02T13:49:47.846768+0000","last_unstale":"2024-04-02T13:51:06.616515+0000","last_undegraded":"2024-04-02T13:51:06.616515+0000","last_fullsized":"2024-04-02T13:51:06.616515+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:46:21.098780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:46:21.098780+0000","last_clean_scrub_stamp":"2024-04-02T13:46:21.098780+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:03:52.323921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347965+0000","last_change":"2024-04-02T13:51:05.518859+0000","last_active":"2024-04-02T13:51:06.347965+0000","last_peered":"2024-04-02T13:51:06.347965+0000","last_clean":"2024-04-02T13:51:06.347965+0000","last_became_active":"2024-04-02T13:51:05.518364+0000","last_became_peered":"2024-04-02T13:51:05.518364+0000","last_unstale":"2024-04-02T13:51:06.347965+0000","last_undegraded":"2024-04-02T13:51:06.347965+0000","last_fullsized":"2024-04-02T13:51:06.347965+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:10:45.877853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347231+0000","last_change":"2024-04-02T13:51:01.254795+0000","last_active":"2024-04-02T13:51:06.347231+0000","last_peered":"2024-04-02T13:51:06.347231+0000","last_clean":"2024-04-02T13:51:06.347231+0000","last_became_active":"2024-04-02T13:51:01.254452+0000","last_became_peered":"2024-04-02T13:51:01.254452+0000","last_unstale":"2024-04-02T13:51:06.347231+0000","last_undegraded":"2024-04-02T13:51:06.347231+0000","last_fullsized":"2024-04-02T13:51:06.347231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:13:53.747308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616801+0000","last_change":"2024-04-02T13:51:00.241323+0000","last_active":"2024-04-02T13:51:06.616801+0000","last_peered":"2024-04-02T13:51:06.616801+0000","last_clean":"2024-04-02T13:51:06.616801+0000","last_became_active":"2024-04-02T13:51:00.240989+0000","last_became_peered":"2024-04-02T13:51:00.240989+0000","last_unstale":"2024-04-02T13:51:06.616801+0000","last_undegraded":"2024-04-02T13:51:06.616801+0000","last_fullsized":"2024-04-02T13:51:06.616801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:47:52.601517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.697528+0000","last_change":"2024-04-02T13:51:05.360641+0000","last_active":"2024-04-02T13:51:06.697528+0000","last_peered":"2024-04-02T13:51:06.697528+0000","last_clean":"2024-04-02T13:51:06.697528+0000","last_became_active":"2024-04-02T13:51:05.360089+0000","last_became_peered":"2024-04-02T13:51:05.360089+0000","last_unstale":"2024-04-02T13:51:06.697528+0000","last_undegraded":"2024-04-02T13:51:06.697528+0000","last_fullsized":"2024-04-02T13:51:06.697528+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:04:38.393274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616669+0000","last_change":"2024-04-02T13:51:00.241416+0000","last_active":"2024-04-02T13:51:06.616669+0000","last_peered":"2024-04-02T13:51:06.616669+0000","last_clean":"2024-04-02T13:51:06.616669+0000","last_became_active":"2024-04-02T13:51:00.241071+0000","last_became_peered":"2024-04-02T13:51:00.241071+0000","last_unstale":"2024-04-02T13:51:06.616669+0000","last_undegraded":"2024-04-02T13:51:06.616669+0000","last_fullsized":"2024-04-02T13:51:06.616669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:24:16.243737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347985+0000","last_change":"2024-04-02T13:51:01.252156+0000","last_active":"2024-04-02T13:51:06.347985+0000","last_peered":"2024-04-02T13:51:06.347985+0000","last_clean":"2024-04-02T13:51:06.347985+0000","last_became_active":"2024-04-02T13:51:01.251733+0000","last_became_peered":"2024-04-02T13:51:01.251733+0000","last_unstale":"2024-04-02T13:51:06.347985+0000","last_undegraded":"2024-04-02T13:51:06.347985+0000","last_fullsized":"2024-04-02T13:51:06.347985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:54:24.082643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616346+0000","last_change":"2024-04-02T13:51:05.518367+0000","last_active":"2024-04-02T13:51:06.616346+0000","last_peered":"2024-04-02T13:51:06.616346+0000","last_clean":"2024-04-02T13:51:06.616346+0000","last_became_active":"2024-04-02T13:51:05.517783+0000","last_became_peered":"2024-04-02T13:51:05.517783+0000","last_unstale":"2024-04-02T13:51:06.616346+0000","last_undegraded":"2024-04-02T13:51:06.616346+0000","last_fullsized":"2024-04-02T13:51:06.616346+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:10:53.880909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:27.344903+0000","last_change":"2024-04-02T13:51:00.248998+0000","last_active":"2024-04-02T13:51:27.344903+0000","last_peered":"2024-04-02T13:51:27.344903+0000","last_clean":"2024-04-02T13:51:27.344903+0000","last_became_active":"2024-04-02T13:51:00.248613+0000","last_became_peered":"2024-04-02T13:51:00.248613+0000","last_unstale":"2024-04-02T13:51:27.344903+0000","last_undegraded":"2024-04-02T13:51:27.344903+0000","last_fullsized":"2024-04-02T13:51:27.344903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:53.948645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617187+0000","last_change":"2024-04-02T13:51:01.257244+0000","last_active":"2024-04-02T13:51:06.617187+0000","last_peered":"2024-04-02T13:51:06.617187+0000","last_clean":"2024-04-02T13:51:06.617187+0000","last_became_active":"2024-04-02T13:51:01.256739+0000","last_became_peered":"2024-04-02T13:51:01.256739+0000","last_unstale":"2024-04-02T13:51:06.617187+0000","last_undegraded":"2024-04-02T13:51:06.617187+0000","last_fullsized":"2024-04-02T13:51:06.617187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:38:18.786553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.353918+0000","last_change":"2024-04-02T13:51:05.356177+0000","last_active":"2024-04-02T13:51:06.353918+0000","last_peered":"2024-04-02T13:51:06.353918+0000","last_clean":"2024-04-02T13:51:06.353918+0000","last_became_active":"2024-04-02T13:51:05.354704+0000","last_became_peered":"2024-04-02T13:51:05.354704+0000","last_unstale":"2024-04-02T13:51:06.353918+0000","last_undegraded":"2024-04-02T13:51:06.353918+0000","last_fullsized":"2024-04-02T13:51:06.353918+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:27:55.761490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:28.350569+0000","last_change":"2024-04-02T13:51:00.245945+0000","last_active":"2024-04-02T13:51:28.350569+0000","last_peered":"2024-04-02T13:51:28.350569+0000","last_clean":"2024-04-02T13:51:28.350569+0000","last_became_active":"2024-04-02T13:51:00.245528+0000","last_became_peered":"2024-04-02T13:51:00.245528+0000","last_unstale":"2024-04-02T13:51:28.350569+0000","last_undegraded":"2024-04-02T13:51:28.350569+0000","last_fullsized":"2024-04-02T13:51:28.350569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:37:49.263614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617097+0000","last_change":"2024-04-02T13:51:01.260268+0000","last_active":"2024-04-02T13:51:06.617097+0000","last_peered":"2024-04-02T13:51:06.617097+0000","last_clean":"2024-04-02T13:51:06.617097+0000","last_became_active":"2024-04-02T13:51:01.258182+0000","last_became_peered":"2024-04-02T13:51:01.258182+0000","last_unstale":"2024-04-02T13:51:06.617097+0000","last_undegraded":"2024-04-02T13:51:06.617097+0000","last_fullsized":"2024-04-02T13:51:06.617097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:31:31.423988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.618242+0000","last_change":"2024-04-02T13:51:05.518597+0000","last_active":"2024-04-02T13:51:06.618242+0000","last_peered":"2024-04-02T13:51:06.618242+0000","last_clean":"2024-04-02T13:51:06.618242+0000","last_became_active":"2024-04-02T13:51:05.518134+0000","last_became_peered":"2024-04-02T13:51:05.518134+0000","last_unstale":"2024-04-02T13:51:06.618242+0000","last_undegraded":"2024-04-02T13:51:06.618242+0000","last_fullsized":"2024-04-02T13:51:06.618242+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:00:38.632939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:25.333094+0000","last_change":"2024-04-02T13:51:00.241680+0000","last_active":"2024-04-02T13:51:25.333094+0000","last_peered":"2024-04-02T13:51:25.333094+0000","last_clean":"2024-04-02T13:51:25.333094+0000","last_became_active":"2024-04-02T13:51:00.241301+0000","last_became_peered":"2024-04-02T13:51:00.241301+0000","last_unstale":"2024-04-02T13:51:25.333094+0000","last_undegraded":"2024-04-02T13:51:25.333094+0000","last_fullsized":"2024-04-02T13:51:25.333094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:12:10.606973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616114+0000","last_change":"2024-04-02T13:51:01.254795+0000","last_active":"2024-04-02T13:51:06.616114+0000","last_peered":"2024-04-02T13:51:06.616114+0000","last_clean":"2024-04-02T13:51:06.616114+0000","last_became_active":"2024-04-02T13:51:01.254451+0000","last_became_peered":"2024-04-02T13:51:01.254451+0000","last_unstale":"2024-04-02T13:51:06.616114+0000","last_undegraded":"2024-04-02T13:51:06.616114+0000","last_fullsized":"2024-04-02T13:51:06.616114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:58:04.202996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617888+0000","last_change":"2024-04-02T13:51:05.363592+0000","last_active":"2024-04-02T13:51:06.617888+0000","last_peered":"2024-04-02T13:51:06.617888+0000","last_clean":"2024-04-02T13:51:06.617888+0000","last_became_active":"2024-04-02T13:51:05.363211+0000","last_became_peered":"2024-04-02T13:51:05.363211+0000","last_unstale":"2024-04-02T13:51:06.617888+0000","last_undegraded":"2024-04-02T13:51:06.617888+0000","last_fullsized":"2024-04-02T13:51:06.617888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:31:56.561505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:28.351341+0000","last_change":"2024-04-02T13:51:00.243681+0000","last_active":"2024-04-02T13:51:28.351341+0000","last_peered":"2024-04-02T13:51:28.351341+0000","last_clean":"2024-04-02T13:51:28.351341+0000","last_became_active":"2024-04-02T13:51:00.241615+0000","last_became_peered":"2024-04-02T13:51:00.241615+0000","last_unstale":"2024-04-02T13:51:28.351341+0000","last_undegraded":"2024-04-02T13:51:28.351341+0000","last_fullsized":"2024-04-02T13:51:28.351341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:54:30.112735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.697363+0000","last_change":"2024-04-02T13:51:01.252429+0000","last_active":"2024-04-02T13:51:06.697363+0000","last_peered":"2024-04-02T13:51:06.697363+0000","last_clean":"2024-04-02T13:51:06.697363+0000","last_became_active":"2024-04-02T13:51:01.252017+0000","last_became_peered":"2024-04-02T13:51:01.252017+0000","last_unstale":"2024-04-02T13:51:06.697363+0000","last_undegraded":"2024-04-02T13:51:06.697363+0000","last_fullsized":"2024-04-02T13:51:06.697363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:32:02.152578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.618197+0000","last_change":"2024-04-02T13:51:05.361297+0000","last_active":"2024-04-02T13:51:06.618197+0000","last_peered":"2024-04-02T13:51:06.618197+0000","last_clean":"2024-04-02T13:51:06.618197+0000","last_became_active":"2024-04-02T13:51:05.360916+0000","last_became_peered":"2024-04-02T13:51:05.360916+0000","last_unstale":"2024-04-02T13:51:06.618197+0000","last_undegraded":"2024-04-02T13:51:06.618197+0000","last_fullsized":"2024-04-02T13:51:06.618197+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:52:48.608587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:27.343296+0000","last_change":"2024-04-02T13:51:00.249141+0000","last_active":"2024-04-02T13:51:27.343296+0000","last_peered":"2024-04-02T13:51:27.343296+0000","last_clean":"2024-04-02T13:51:27.343296+0000","last_became_active":"2024-04-02T13:51:00.248760+0000","last_became_peered":"2024-04-02T13:51:00.248760+0000","last_unstale":"2024-04-02T13:51:27.343296+0000","last_undegraded":"2024-04-02T13:51:27.343296+0000","last_fullsized":"2024-04-02T13:51:27.343296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:37:12.710427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617427+0000","last_change":"2024-04-02T13:51:01.255557+0000","last_active":"2024-04-02T13:51:06.617427+0000","last_peered":"2024-04-02T13:51:06.617427+0000","last_clean":"2024-04-02T13:51:06.617427+0000","last_became_active":"2024-04-02T13:51:01.255011+0000","last_became_peered":"2024-04-02T13:51:01.255011+0000","last_unstale":"2024-04-02T13:51:06.617427+0000","last_undegraded":"2024-04-02T13:51:06.617427+0000","last_fullsized":"2024-04-02T13:51:06.617427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:01:52.105965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.618366+0000","last_change":"2024-04-02T13:51:05.518764+0000","last_active":"2024-04-02T13:51:06.618366+0000","last_peered":"2024-04-02T13:51:06.618366+0000","last_clean":"2024-04-02T13:51:06.618366+0000","last_became_active":"2024-04-02T13:51:05.518088+0000","last_became_peered":"2024-04-02T13:51:05.518088+0000","last_unstale":"2024-04-02T13:51:06.618366+0000","last_undegraded":"2024-04-02T13:51:06.618366+0000","last_fullsized":"2024-04-02T13:51:06.618366+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:03:02.901384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:25.331268+0000","last_change":"2024-04-02T13:51:00.244147+0000","last_active":"2024-04-02T13:51:25.331268+0000","last_peered":"2024-04-02T13:51:25.331268+0000","last_clean":"2024-04-02T13:51:25.331268+0000","last_became_active":"2024-04-02T13:51:00.243763+0000","last_became_peered":"2024-04-02T13:51:00.243763+0000","last_unstale":"2024-04-02T13:51:25.331268+0000","last_undegraded":"2024-04-02T13:51:25.331268+0000","last_fullsized":"2024-04-02T13:51:25.331268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:34:27.738746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617357+0000","last_change":"2024-04-02T13:51:01.268121+0000","last_active":"2024-04-02T13:51:06.617357+0000","last_peered":"2024-04-02T13:51:06.617357+0000","last_clean":"2024-04-02T13:51:06.617357+0000","last_became_active":"2024-04-02T13:51:01.266363+0000","last_became_peered":"2024-04-02T13:51:01.266363+0000","last_unstale":"2024-04-02T13:51:06.617357+0000","last_undegraded":"2024-04-02T13:51:06.617357+0000","last_fullsized":"2024-04-02T13:51:06.617357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:30:50.254916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.349429+0000","last_change":"2024-04-02T13:51:05.361285+0000","last_active":"2024-04-02T13:51:06.349429+0000","last_peered":"2024-04-02T13:51:06.349429+0000","last_clean":"2024-04-02T13:51:06.349429+0000","last_became_active":"2024-04-02T13:51:05.360122+0000","last_became_peered":"2024-04-02T13:51:05.360122+0000","last_unstale":"2024-04-02T13:51:06.349429+0000","last_undegraded":"2024-04-02T13:51:06.349429+0000","last_fullsized":"2024-04-02T13:51:06.349429+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:30:59.935538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'6","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:29.354823+0000","last_change":"2024-04-02T13:51:00.247235+0000","last_active":"2024-04-02T13:51:29.354823+0000","last_peered":"2024-04-02T13:51:29.354823+0000","last_clean":"2024-04-02T13:51:29.354823+0000","last_became_active":"2024-04-02T13:51:00.246856+0000","last_became_peered":"2024-04-02T13:51:00.246856+0000","last_unstale":"2024-04-02T13:51:29.354823+0000","last_undegraded":"2024-04-02T13:51:29.354823+0000","last_fullsized":"2024-04-02T13:51:29.354823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:17:51.875952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617713+0000","last_change":"2024-04-02T13:51:01.263866+0000","last_active":"2024-04-02T13:51:06.617713+0000","last_peered":"2024-04-02T13:51:06.617713+0000","last_clean":"2024-04-02T13:51:06.617713+0000","last_became_active":"2024-04-02T13:51:01.263564+0000","last_became_peered":"2024-04-02T13:51:01.263564+0000","last_unstale":"2024-04-02T13:51:06.617713+0000","last_undegraded":"2024-04-02T13:51:06.617713+0000","last_fullsized":"2024-04-02T13:51:06.617713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:59:33.663277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616388+0000","last_change":"2024-04-02T13:51:05.363079+0000","last_active":"2024-04-02T13:51:06.616388+0000","last_peered":"2024-04-02T13:51:06.616388+0000","last_clean":"2024-04-02T13:51:06.616388+0000","last_became_active":"2024-04-02T13:51:05.362631+0000","last_became_peered":"2024-04-02T13:51:05.362631+0000","last_unstale":"2024-04-02T13:51:06.616388+0000","last_undegraded":"2024-04-02T13:51:06.616388+0000","last_fullsized":"2024-04-02T13:51:06.616388+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:23.656269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:28.351460+0000","last_change":"2024-04-02T13:51:00.242820+0000","last_active":"2024-04-02T13:51:28.351460+0000","last_peered":"2024-04-02T13:51:28.351460+0000","last_clean":"2024-04-02T13:51:28.351460+0000","last_became_active":"2024-04-02T13:51:00.242262+0000","last_became_peered":"2024-04-02T13:51:00.242262+0000","last_unstale":"2024-04-02T13:51:28.351460+0000","last_undegraded":"2024-04-02T13:51:28.351460+0000","last_fullsized":"2024-04-02T13:51:28.351460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:12:22.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617060+0000","last_change":"2024-04-02T13:51:01.266548+0000","last_active":"2024-04-02T13:51:06.617060+0000","last_peered":"2024-04-02T13:51:06.617060+0000","last_clean":"2024-04-02T13:51:06.617060+0000","last_became_active":"2024-04-02T13:51:01.259003+0000","last_became_peered":"2024-04-02T13:51:01.259003+0000","last_unstale":"2024-04-02T13:51:06.617060+0000","last_undegraded":"2024-04-02T13:51:06.617060+0000","last_fullsized":"2024-04-02T13:51:06.617060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:38:53.692231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347053+0000","last_change":"2024-04-02T13:51:05.359994+0000","last_active":"2024-04-02T13:51:06.347053+0000","last_peered":"2024-04-02T13:51:06.347053+0000","last_clean":"2024-04-02T13:51:06.347053+0000","last_became_active":"2024-04-02T13:51:05.359438+0000","last_became_peered":"2024-04-02T13:51:05.359438+0000","last_unstale":"2024-04-02T13:51:06.347053+0000","last_undegraded":"2024-04-02T13:51:06.347053+0000","last_fullsized":"2024-04-02T13:51:06.347053+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:09:43.929446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347779+0000","last_change":"2024-04-02T13:51:00.237889+0000","last_active":"2024-04-02T13:51:06.347779+0000","last_peered":"2024-04-02T13:51:06.347779+0000","last_clean":"2024-04-02T13:51:06.347779+0000","last_became_active":"2024-04-02T13:51:00.237428+0000","last_became_peered":"2024-04-02T13:51:00.237428+0000","last_unstale":"2024-04-02T13:51:06.347779+0000","last_undegraded":"2024-04-02T13:51:06.347779+0000","last_fullsized":"2024-04-02T13:51:06.347779+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:21:52.483656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616840+0000","last_change":"2024-04-02T13:51:01.262093+0000","last_active":"2024-04-02T13:51:06.616840+0000","last_peered":"2024-04-02T13:51:06.616840+0000","last_clean":"2024-04-02T13:51:06.616840+0000","last_became_active":"2024-04-02T13:51:01.258357+0000","last_became_peered":"2024-04-02T13:51:01.258357+0000","last_unstale":"2024-04-02T13:51:06.616840+0000","last_undegraded":"2024-04-02T13:51:06.616840+0000","last_fullsized":"2024-04-02T13:51:06.616840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:00:06.317307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.697453+0000","last_change":"2024-04-02T13:51:01.261015+0000","last_active":"2024-04-02T13:51:06.697453+0000","last_peered":"2024-04-02T13:51:06.697453+0000","last_clean":"2024-04-02T13:51:06.697453+0000","last_became_active":"2024-04-02T13:51:01.260697+0000","last_became_peered":"2024-04-02T13:51:01.260697+0000","last_unstale":"2024-04-02T13:51:06.697453+0000","last_undegraded":"2024-04-02T13:51:06.697453+0000","last_fullsized":"2024-04-02T13:51:06.697453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:49:02.248995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.354015+0000","last_change":"2024-04-02T13:51:01.257205+0000","last_active":"2024-04-02T13:51:06.354015+0000","last_peered":"2024-04-02T13:51:06.354015+0000","last_clean":"2024-04-02T13:51:06.354015+0000","last_became_active":"2024-04-02T13:51:01.256866+0000","last_became_peered":"2024-04-02T13:51:01.256866+0000","last_unstale":"2024-04-02T13:51:06.354015+0000","last_undegraded":"2024-04-02T13:51:06.354015+0000","last_fullsized":"2024-04-02T13:51:06.354015+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:53:23.211580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.353913+0000","last_change":"2024-04-02T13:51:01.255563+0000","last_active":"2024-04-02T13:51:06.353913+0000","last_peered":"2024-04-02T13:51:06.353913+0000","last_clean":"2024-04-02T13:51:06.353913+0000","last_became_active":"2024-04-02T13:51:01.255228+0000","last_became_peered":"2024-04-02T13:51:01.255228+0000","last_unstale":"2024-04-02T13:51:06.353913+0000","last_undegraded":"2024-04-02T13:51:06.353913+0000","last_fullsized":"2024-04-02T13:51:06.353913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:48:28.200776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616474+0000","last_change":"2024-04-02T13:51:01.258153+0000","last_active":"2024-04-02T13:51:06.616474+0000","last_peered":"2024-04-02T13:51:06.616474+0000","last_clean":"2024-04-02T13:51:06.616474+0000","last_became_active":"2024-04-02T13:51:01.257656+0000","last_became_peered":"2024-04-02T13:51:01.257656+0000","last_unstale":"2024-04-02T13:51:06.616474+0000","last_undegraded":"2024-04-02T13:51:06.616474+0000","last_fullsized":"2024-04-02T13:51:06.616474+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:37.887427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.348019+0000","last_change":"2024-04-02T13:51:01.254675+0000","last_active":"2024-04-02T13:51:06.348019+0000","last_peered":"2024-04-02T13:51:06.348019+0000","last_clean":"2024-04-02T13:51:06.348019+0000","last_became_active":"2024-04-02T13:51:01.254336+0000","last_became_peered":"2024-04-02T13:51:01.254336+0000","last_unstale":"2024-04-02T13:51:06.348019+0000","last_undegraded":"2024-04-02T13:51:06.348019+0000","last_fullsized":"2024-04-02T13:51:06.348019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:36.171961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.697843+0000","last_change":"2024-04-02T13:51:01.259457+0000","last_active":"2024-04-02T13:51:06.697843+0000","last_peered":"2024-04-02T13:51:06.697843+0000","last_clean":"2024-04-02T13:51:06.697843+0000","last_became_active":"2024-04-02T13:51:01.259099+0000","last_became_peered":"2024-04-02T13:51:01.259099+0000","last_unstale":"2024-04-02T13:51:06.697843+0000","last_undegraded":"2024-04-02T13:51:06.697843+0000","last_fullsized":"2024-04-02T13:51:06.697843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:13:06.594132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.354595+0000","last_change":"2024-04-02T13:51:01.248177+0000","last_active":"2024-04-02T13:51:06.354595+0000","last_peered":"2024-04-02T13:51:06.354595+0000","last_clean":"2024-04-02T13:51:06.354595+0000","last_became_active":"2024-04-02T13:51:01.247843+0000","last_became_peered":"2024-04-02T13:51:01.247843+0000","last_unstale":"2024-04-02T13:51:06.354595+0000","last_undegraded":"2024-04-02T13:51:06.354595+0000","last_fullsized":"2024-04-02T13:51:06.354595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:38:05.805382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.617548+0000","last_change":"2024-04-02T13:51:01.254822+0000","last_active":"2024-04-02T13:51:06.617548+0000","last_peered":"2024-04-02T13:51:06.617548+0000","last_clean":"2024-04-02T13:51:06.617548+0000","last_became_active":"2024-04-02T13:51:01.254451+0000","last_became_peered":"2024-04-02T13:51:01.254451+0000","last_unstale":"2024-04-02T13:51:06.617548+0000","last_undegraded":"2024-04-02T13:51:06.617548+0000","last_fullsized":"2024-04-02T13:51:06.617548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:15:35.790371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347295+0000","last_change":"2024-04-02T13:51:01.265467+0000","last_active":"2024-04-02T13:51:06.347295+0000","last_peered":"2024-04-02T13:51:06.347295+0000","last_clean":"2024-04-02T13:51:06.347295+0000","last_became_active":"2024-04-02T13:51:01.265072+0000","last_became_peered":"2024-04-02T13:51:01.265072+0000","last_unstale":"2024-04-02T13:51:06.347295+0000","last_undegraded":"2024-04-02T13:51:06.347295+0000","last_fullsized":"2024-04-02T13:51:06.347295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:12:07.702222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.616954+0000","last_change":"2024-04-02T13:51:01.252844+0000","last_active":"2024-04-02T13:51:06.616954+0000","last_peered":"2024-04-02T13:51:06.616954+0000","last_clean":"2024-04-02T13:51:06.616954+0000","last_became_active":"2024-04-02T13:51:01.252515+0000","last_became_peered":"2024-04-02T13:51:01.252515+0000","last_unstale":"2024-04-02T13:51:06.616954+0000","last_undegraded":"2024-04-02T13:51:06.616954+0000","last_fullsized":"2024-04-02T13:51:06.616954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:20:16.216418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.349785+0000","last_change":"2024-04-02T13:51:01.260141+0000","last_active":"2024-04-02T13:51:06.349785+0000","last_peered":"2024-04-02T13:51:06.349785+0000","last_clean":"2024-04-02T13:51:06.349785+0000","last_became_active":"2024-04-02T13:51:01.259530+0000","last_became_peered":"2024-04-02T13:51:01.259530+0000","last_unstale":"2024-04-02T13:51:06.349785+0000","last_undegraded":"2024-04-02T13:51:06.349785+0000","last_fullsized":"2024-04-02T13:51:06.349785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:00:03.579414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.347874+0000","last_change":"2024-04-02T13:51:01.259981+0000","last_active":"2024-04-02T13:51:06.347874+0000","last_peered":"2024-04-02T13:51:06.347874+0000","last_clean":"2024-04-02T13:51:06.347874+0000","last_became_active":"2024-04-02T13:51:01.259647+0000","last_became_peered":"2024-04-02T13:51:01.259647+0000","last_unstale":"2024-04-02T13:51:06.347874+0000","last_undegraded":"2024-04-02T13:51:06.347874+0000","last_fullsized":"2024-04-02T13:51:06.347874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:10:22.753767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:51:06.697580+0000","last_change":"2024-04-02T13:51:01.255342+0000","last_active":"2024-04-02T13:51:06.697580+0000","last_peered":"2024-04-02T13:51:06.697580+0000","last_clean":"2024-04-02T13:51:06.697580+0000","last_became_active":"2024-04-02T13:51:01.254679+0000","last_became_peered":"2024-04-02T13:51:01.254679+0000","last_unstale":"2024-04-02T13:51:06.697580+0000","last_undegraded":"2024-04-02T13:51:06.697580+0000","last_fullsized":"2024-04-02T13:51:06.697580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:25:12.198270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":45822,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":29,"num_read_kb":45,"num_write":177,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":372736,"data_stored":97749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":483005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612632,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1098250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776157,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":483913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037794,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":483631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332135,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36904,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708248,"statfs":{"total":95995035648,"available":95957245952,"internally_reserved":0,"allocated":1593344,"data_stored":1073955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":995328,"data_stored":483005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":486847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":484483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1077248,"data_stored":525839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":483435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":483705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1638400,"data_stored":1080289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":42834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":42207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T13:51:31.191 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-02T13:51:31.191 INFO:tasks.cephadm:Setting up client nodes... 2024-04-02T13:51:31.192 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-02T13:51:31.473 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:51:31.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:31 smithi052 ceph-mon[32104]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:51:31.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:31 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1857932961' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:31.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:31 smithi189 ceph-mon[36683]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:51:31.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:31 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1857932961' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:31.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:31 smithi088 ceph-mon[36528]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:51:31.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:31 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1857932961' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:32.764 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:32 smithi052 ceph-mon[32104]: pgmap v308: 129 pgs: 129 active+clean; 621 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-04-02T13:51:32.764 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:32 smithi052 ceph-mon[32104]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:51:32.764 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:32 smithi052 ceph-mon[32104]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:51:32.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:32 smithi189 ceph-mon[36683]: pgmap v308: 129 pgs: 129 active+clean; 621 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-04-02T13:51:32.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:32 smithi189 ceph-mon[36683]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:51:32.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:32 smithi189 ceph-mon[36683]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:51:32.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:32 smithi088 ceph-mon[36528]: pgmap v308: 129 pgs: 129 active+clean; 621 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-04-02T13:51:32.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:32 smithi088 ceph-mon[36528]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:51:32.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:32 smithi088 ceph-mon[36528]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:51:33.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.335+0000 7f7725063700 1 -- 172.21.15.52:0/2509627083 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7720102640 msgr2=0x7f7720102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:33.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.335+0000 7f7725063700 1 --2- 172.21.15.52:0/2509627083 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7720102640 0x7f7720102a60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f77100099c0 tx=0x7f7710009cd0 comp rx=0 tx=0).stop 2024-04-02T13:51:33.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.335+0000 7f7725063700 1 -- 172.21.15.52:0/2509627083 shutdown_connections 2024-04-02T13:51:33.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.335+0000 7f7725063700 1 --2- 172.21.15.52:0/2509627083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77201041e0 0x7f772010aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:33.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.335+0000 7f7725063700 1 --2- 172.21.15.52:0/2509627083 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7720102640 0x7f7720102a60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:33.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.336+0000 7f7725063700 1 --2- 172.21.15.52:0/2509627083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7720103840 0x7f7720103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:33.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.336+0000 7f7725063700 1 -- 172.21.15.52:0/2509627083 >> 172.21.15.52:0/2509627083 conn(0x7f77200fdc00 msgr2=0x7f7720100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:33.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.336+0000 7f7725063700 1 -- 172.21.15.52:0/2509627083 shutdown_connections 2024-04-02T13:51:33.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.336+0000 7f7725063700 1 -- 172.21.15.52:0/2509627083 wait complete. 2024-04-02T13:51:33.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.336+0000 7f7725063700 1 Processor -- start 2024-04-02T13:51:33.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.337+0000 7f7725063700 1 -- start start 2024-04-02T13:51:33.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.337+0000 7f7725063700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7720102640 0x7f772019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:33.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.337+0000 7f7725063700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7720103840 0x7f772019f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:33.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.337+0000 7f7725063700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77201041e0 0x7f77201a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:33.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.337+0000 7f7725063700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f772010e200 con 0x7f77201041e0 2024-04-02T13:51:33.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.337+0000 7f7725063700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f772010e080 con 0x7f7720103840 2024-04-02T13:51:33.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.337+0000 7f7725063700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f772010e380 con 0x7f7720102640 2024-04-02T13:51:33.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.337+0000 7f771f59e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77201041e0 0x7f77201a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:33.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.337+0000 7f771f59e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77201041e0 0x7f77201a6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44458/0 (socket says 172.21.15.52:44458) 2024-04-02T13:51:33.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.337+0000 7f771f59e700 1 -- 172.21.15.52:0/2046900133 learned_addr learned my addr 172.21.15.52:0/2046900133 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:33.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.337+0000 7f771e59c700 1 --2- 172.21.15.52:0/2046900133 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7720103840 0x7f772019f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:33.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.337+0000 7f771ed9d700 1 --2- 172.21.15.52:0/2046900133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7720102640 0x7f772019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:33.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.338+0000 7f771f59e700 1 -- 172.21.15.52:0/2046900133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7720102640 msgr2=0x7f772019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:33.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.338+0000 7f771f59e700 1 --2- 172.21.15.52:0/2046900133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7720102640 0x7f772019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:33.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.338+0000 7f771f59e700 1 -- 172.21.15.52:0/2046900133 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7720103840 msgr2=0x7f772019f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:33.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.338+0000 7f771f59e700 1 --2- 172.21.15.52:0/2046900133 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7720103840 0x7f772019f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:33.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.338+0000 7f771f59e700 1 -- 172.21.15.52:0/2046900133 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77201a69c0 con 0x7f77201041e0 2024-04-02T13:51:33.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.338+0000 7f771f59e700 1 --2- 172.21.15.52:0/2046900133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77201041e0 0x7f77201a6320 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f771800ef60 tx=0x7f771800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:33.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.338+0000 7f7707fff700 1 -- 172.21.15.52:0/2046900133 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77180092e0 con 0x7f77201041e0 2024-04-02T13:51:33.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.338+0000 7f7725063700 1 -- 172.21.15.52:0/2046900133 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77201a6c50 con 0x7f77201041e0 2024-04-02T13:51:33.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.338+0000 7f7707fff700 1 -- 172.21.15.52:0/2046900133 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7718004bb0 con 0x7f77201041e0 2024-04-02T13:51:33.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.338+0000 7f7707fff700 1 -- 172.21.15.52:0/2046900133 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7718004280 con 0x7f77201041e0 2024-04-02T13:51:33.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.338+0000 7f7725063700 1 -- 172.21.15.52:0/2046900133 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77201a7140 con 0x7f77201041e0 2024-04-02T13:51:33.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.339+0000 7f7725063700 1 -- 172.21.15.52:0/2046900133 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7720050b00 con 0x7f77201041e0 2024-04-02T13:51:33.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.340+0000 7f7707fff700 1 -- 172.21.15.52:0/2046900133 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f771801b030 con 0x7f77201041e0 2024-04-02T13:51:33.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.341+0000 7f7707fff700 1 --2- 172.21.15.52:0/2046900133 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f77080a3d80 0x7f77080a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:33.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.341+0000 7f7707fff700 1 -- 172.21.15.52:0/2046900133 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f77180d0150 con 0x7f77201041e0 2024-04-02T13:51:33.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.341+0000 7f771ed9d700 1 --2- 172.21.15.52:0/2046900133 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f77080a3d80 0x7f77080a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:33.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.341+0000 7f771ed9d700 1 --2- 172.21.15.52:0/2046900133 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f77080a3d80 0x7f77080a6240 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f771000b580 tx=0x7f7710005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:33.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.346+0000 7f7707fff700 1 -- 172.21.15.52:0/2046900133 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7718022020 con 0x7f77201041e0 2024-04-02T13:51:33.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.643+0000 7f7725063700 1 -- 172.21.15.52:0/2046900133 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f7720063200 con 0x7f77201041e0 2024-04-02T13:51:33.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.645+0000 7f7707fff700 1 -- 172.21.15.52:0/2046900133 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f7718010040 con 0x7f77201041e0 2024-04-02T13:51:33.648 INFO:teuthology.orchestra.run.smithi052.stdout:[client.0] 2024-04-02T13:51:33.648 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBlDQxmN0ttJhAAXDyICluaGrsej/NW/2DX4g== 2024-04-02T13:51:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.648+0000 7f7725063700 1 -- 172.21.15.52:0/2046900133 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f77080a3d80 msgr2=0x7f77080a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.648+0000 7f7725063700 1 --2- 172.21.15.52:0/2046900133 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f77080a3d80 0x7f77080a6240 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f771000b580 tx=0x7f7710005fb0 comp rx=0 tx=0).stop 2024-04-02T13:51:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.648+0000 7f7725063700 1 -- 172.21.15.52:0/2046900133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77201041e0 msgr2=0x7f77201a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.648+0000 7f7725063700 1 --2- 172.21.15.52:0/2046900133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77201041e0 0x7f77201a6320 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f771800ef60 tx=0x7f771800ef90 comp rx=0 tx=0).stop 2024-04-02T13:51:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.648+0000 7f7725063700 1 -- 172.21.15.52:0/2046900133 shutdown_connections 2024-04-02T13:51:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.648+0000 7f7725063700 1 --2- 172.21.15.52:0/2046900133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77201041e0 0x7f77201a6320 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.648+0000 7f7725063700 1 --2- 172.21.15.52:0/2046900133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7720102640 0x7f772019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.648+0000 7f7725063700 1 --2- 172.21.15.52:0/2046900133 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f77080a3d80 0x7f77080a6240 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.648+0000 7f7725063700 1 --2- 172.21.15.52:0/2046900133 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7720103840 0x7f772019f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.648+0000 7f7725063700 1 -- 172.21.15.52:0/2046900133 >> 172.21.15.52:0/2046900133 conn(0x7f77200fdc00 msgr2=0x7f7720104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.649+0000 7f7725063700 1 -- 172.21.15.52:0/2046900133 shutdown_connections 2024-04-02T13:51:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:33.649+0000 7f7725063700 1 -- 172.21.15.52:0/2046900133 wait complete. 2024-04-02T13:51:34.467 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-04-02T13:51:34.467 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-02T13:51:34.468 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-02T13:51:34.510 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-02T13:51:34.511 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-02T13:51:34.511 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph mgr dump --format=json 2024-04-02T13:51:34.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:34 smithi189 ceph-mon[36683]: pgmap v309: 129 pgs: 129 active+clean; 623 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7.5 KiB/s wr, 17 op/s 2024-04-02T13:51:34.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:34 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2046900133' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T13:51:34.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:34 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2046900133' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-02T13:51:34.796 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:51:34.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:34 smithi052 ceph-mon[32104]: pgmap v309: 129 pgs: 129 active+clean; 623 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7.5 KiB/s wr, 17 op/s 2024-04-02T13:51:34.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:34 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2046900133' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T13:51:34.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:34 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2046900133' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-02T13:51:34.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:34 smithi088 ceph-mon[36528]: pgmap v309: 129 pgs: 129 active+clean; 623 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7.5 KiB/s wr, 17 op/s 2024-04-02T13:51:34.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:34 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2046900133' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T13:51:34.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:34 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2046900133' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-02T13:51:36.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:36 smithi052 ceph-mon[32104]: pgmap v310: 129 pgs: 129 active+clean; 623 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.7 KiB/s wr, 18 op/s 2024-04-02T13:51:36.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.715+0000 7f3bfbda5700 1 -- 172.21.15.52:0/2522556881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3bf4102640 msgr2=0x7f3bf4102aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:36.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.715+0000 7f3bfbda5700 1 --2- 172.21.15.52:0/2522556881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3bf4102640 0x7f3bf4102aa0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3be4009990 tx=0x7f3be4009ca0 comp rx=0 tx=0).stop 2024-04-02T13:51:36.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.715+0000 7f3bfbda5700 1 -- 172.21.15.52:0/2522556881 shutdown_connections 2024-04-02T13:51:36.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.715+0000 7f3bfbda5700 1 --2- 172.21.15.52:0/2522556881 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3bf4102fe0 0x7f3bf410f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:36.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.715+0000 7f3bfbda5700 1 --2- 172.21.15.52:0/2522556881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bf4107520 0x7f3bf4107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:36.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.715+0000 7f3bfbda5700 1 --2- 172.21.15.52:0/2522556881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3bf4102640 0x7f3bf4102aa0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:36.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.715+0000 7f3bfbda5700 1 -- 172.21.15.52:0/2522556881 >> 172.21.15.52:0/2522556881 conn(0x7f3bf4079720 msgr2=0x7f3bf4079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:36.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.716+0000 7f3bfbda5700 1 -- 172.21.15.52:0/2522556881 shutdown_connections 2024-04-02T13:51:36.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.716+0000 7f3bfbda5700 1 -- 172.21.15.52:0/2522556881 wait complete. 2024-04-02T13:51:36.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.716+0000 7f3bfbda5700 1 Processor -- start 2024-04-02T13:51:36.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.716+0000 7f3bfbda5700 1 -- start start 2024-04-02T13:51:36.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.716+0000 7f3bfbda5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3bf4102640 0x7f3bf4198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:36.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.716+0000 7f3bfbda5700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bf4102fe0 0x7f3bf41992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:36.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.716+0000 7f3bfbda5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3bf4107520 0x7f3bf419d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:36.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.716+0000 7f3bfbda5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf41128b0 con 0x7f3bf4102640 2024-04-02T13:51:36.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.716+0000 7f3bfbda5700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf4112730 con 0x7f3bf4107520 2024-04-02T13:51:36.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.716+0000 7f3bfbda5700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf4112a30 con 0x7f3bf4102fe0 2024-04-02T13:51:36.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.716+0000 7f3bf9b41700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3bf4102640 0x7f3bf4198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:36.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.717+0000 7f3bf9b41700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3bf4102640 0x7f3bf4198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44478/0 (socket says 172.21.15.52:44478) 2024-04-02T13:51:36.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.717+0000 7f3bf9b41700 1 -- 172.21.15.52:0/1719880588 learned_addr learned my addr 172.21.15.52:0/1719880588 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:36.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.717+0000 7f3bf9340700 1 --2- 172.21.15.52:0/1719880588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bf4102fe0 0x7f3bf41992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:36.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.717+0000 7f3bfa342700 1 --2- 172.21.15.52:0/1719880588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3bf4107520 0x7f3bf419d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:36.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.717+0000 7f3bf9b41700 1 -- 172.21.15.52:0/1719880588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bf4102fe0 msgr2=0x7f3bf41992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:36.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.717+0000 7f3bf9b41700 1 --2- 172.21.15.52:0/1719880588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bf4102fe0 0x7f3bf41992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:36.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.717+0000 7f3bf9b41700 1 -- 172.21.15.52:0/1719880588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3bf4107520 msgr2=0x7f3bf419d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:36.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.717+0000 7f3bf9b41700 1 --2- 172.21.15.52:0/1719880588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3bf4107520 0x7f3bf419d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:36.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.717+0000 7f3bf9b41700 1 -- 172.21.15.52:0/1719880588 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf419ddb0 con 0x7f3bf4102640 2024-04-02T13:51:36.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.717+0000 7f3bfa342700 1 --2- 172.21.15.52:0/1719880588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3bf4107520 0x7f3bf419d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:51:36.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.717+0000 7f3bf9b41700 1 --2- 172.21.15.52:0/1719880588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3bf4102640 0x7f3bf4198da0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000d980 tx=0x7f3bf000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:36.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.717+0000 7f3beaffd700 1 -- 172.21.15.52:0/1719880588 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf0017070 con 0x7f3bf4102640 2024-04-02T13:51:36.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.717+0000 7f3beaffd700 1 -- 172.21.15.52:0/1719880588 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bf0009e10 con 0x7f3bf4102640 2024-04-02T13:51:36.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.717+0000 7f3bfbda5700 1 -- 172.21.15.52:0/1719880588 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf419e0a0 con 0x7f3bf4102640 2024-04-02T13:51:36.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.717+0000 7f3beaffd700 1 -- 172.21.15.52:0/1719880588 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf000f620 con 0x7f3bf4102640 2024-04-02T13:51:36.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.717+0000 7f3bfbda5700 1 -- 172.21.15.52:0/1719880588 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf41a5920 con 0x7f3bf4102640 2024-04-02T13:51:36.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.718+0000 7f3bfbda5700 1 -- 172.21.15.52:0/1719880588 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bf410ab50 con 0x7f3bf4102640 2024-04-02T13:51:36.724 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:36 smithi088 ceph-mon[36528]: pgmap v310: 129 pgs: 129 active+clean; 623 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.7 KiB/s wr, 18 op/s 2024-04-02T13:51:36.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.719+0000 7f3beaffd700 1 -- 172.21.15.52:0/1719880588 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3bf0009930 con 0x7f3bf4102640 2024-04-02T13:51:36.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.720+0000 7f3beaffd700 1 --2- 172.21.15.52:0/1719880588 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3be00a3cd0 0x7f3be00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:36.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.720+0000 7f3bf9340700 1 --2- 172.21.15.52:0/1719880588 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3be00a3cd0 0x7f3be00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:36.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.720+0000 7f3beaffd700 1 -- 172.21.15.52:0/1719880588 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f3bf00d02e0 con 0x7f3bf4102640 2024-04-02T13:51:36.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.721+0000 7f3bf9340700 1 --2- 172.21.15.52:0/1719880588 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3be00a3cd0 0x7f3be00a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3be400b5c0 tx=0x7f3be4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:36.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.724+0000 7f3beaffd700 1 -- 172.21.15.52:0/1719880588 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3bf009c920 con 0x7f3bf4102640 2024-04-02T13:51:36.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:36 smithi189 ceph-mon[36683]: pgmap v310: 129 pgs: 129 active+clean; 623 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.7 KiB/s wr, 18 op/s 2024-04-02T13:51:36.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:36.995+0000 7f3bfbda5700 1 -- 172.21.15.52:0/1719880588 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f3bf419a100 con 0x7f3bf4102640 2024-04-02T13:51:37.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:37.000+0000 7f3beaffd700 1 -- 172.21.15.52:0/1719880588 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264997 (secure 0 0 0) 0x7f3bf009c220 con 0x7f3bf4102640 2024-04-02T13:51:37.006 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:51:37.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:37.003+0000 7f3bfbda5700 1 -- 172.21.15.52:0/1719880588 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3be00a3cd0 msgr2=0x7f3be00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:37.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:37.003+0000 7f3bfbda5700 1 --2- 172.21.15.52:0/1719880588 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3be00a3cd0 0x7f3be00a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3be400b5c0 tx=0x7f3be4005960 comp rx=0 tx=0).stop 2024-04-02T13:51:37.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:37.003+0000 7f3bfbda5700 1 -- 172.21.15.52:0/1719880588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3bf4102640 msgr2=0x7f3bf4198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:37.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:37.003+0000 7f3bfbda5700 1 --2- 172.21.15.52:0/1719880588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3bf4102640 0x7f3bf4198da0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000d980 tx=0x7f3bf000dc90 comp rx=0 tx=0).stop 2024-04-02T13:51:37.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:37.003+0000 7f3bfbda5700 1 -- 172.21.15.52:0/1719880588 shutdown_connections 2024-04-02T13:51:37.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:37.003+0000 7f3bfbda5700 1 --2- 172.21.15.52:0/1719880588 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3be00a3cd0 0x7f3be00a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:37.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:37.003+0000 7f3bfbda5700 1 --2- 172.21.15.52:0/1719880588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3bf4107520 0x7f3bf419d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:37.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:37.003+0000 7f3bfbda5700 1 --2- 172.21.15.52:0/1719880588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3bf4102640 0x7f3bf4198da0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:37.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:37.003+0000 7f3bfbda5700 1 --2- 172.21.15.52:0/1719880588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bf4102fe0 0x7f3bf41992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:37.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:37.003+0000 7f3bfbda5700 1 -- 172.21.15.52:0/1719880588 >> 172.21.15.52:0/1719880588 conn(0x7f3bf4079720 msgr2=0x7f3bf4100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:37.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:37.003+0000 7f3bfbda5700 1 -- 172.21.15.52:0/1719880588 shutdown_connections 2024-04-02T13:51:37.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:37.003+0000 7f3bfbda5700 1 -- 172.21.15.52:0/1719880588 wait complete. 2024-04-02T13:51:37.735 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:37 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1719880588' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-02T13:51:37.774 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:37 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1719880588' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-02T13:51:37.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:37 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1719880588' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-02T13:51:37.809 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3836085837},{"type":"v1","addr":"172.21.15.52:6801","nonce":3836085837}]},"active_addr":"172.21.15.52:6801/3836085837","active_change":"2024-04-02T13:43:54.653473+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.52:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":713755826}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":1937276046}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":2985814329}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":2957262850}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":2184786455}]}]} 2024-04-02T13:51:37.814 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-02T13:51:37.815 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-02T13:51:37.815 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-02T13:51:38.066 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:51:38.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:38 smithi189 ceph-mon[36683]: pgmap v311: 129 pgs: 129 active+clean; 628 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.2 KiB/s wr, 22 op/s 2024-04-02T13:51:38.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:38 smithi052 ceph-mon[32104]: pgmap v311: 129 pgs: 129 active+clean; 628 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.2 KiB/s wr, 22 op/s 2024-04-02T13:51:38.914 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:38 smithi088 ceph-mon[36528]: pgmap v311: 129 pgs: 129 active+clean; 628 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.2 KiB/s wr, 22 op/s 2024-04-02T13:51:39.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.847+0000 7fc5a9e3a700 1 -- 172.21.15.52:0/2773169831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5a406b6b0 msgr2=0x7fc5a410f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:39.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.847+0000 7fc5a9e3a700 1 --2- 172.21.15.52:0/2773169831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5a406b6b0 0x7fc5a410f490 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00b0a0 tx=0x7fc59c00b3b0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.847+0000 7fc5a9e3a700 1 -- 172.21.15.52:0/2773169831 shutdown_connections 2024-04-02T13:51:39.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.847+0000 7fc5a9e3a700 1 --2- 172.21.15.52:0/2773169831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5a406b6b0 0x7fc5a410f490 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.847+0000 7fc5a9e3a700 1 --2- 172.21.15.52:0/2773169831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5a4107520 0x7fc5a4107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.847+0000 7fc5a9e3a700 1 --2- 172.21.15.52:0/2773169831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5a406ad10 0x7fc5a406b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.847+0000 7fc5a9e3a700 1 -- 172.21.15.52:0/2773169831 >> 172.21.15.52:0/2773169831 conn(0x7fc5a4078500 msgr2=0x7fc5a4078910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:39.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.847+0000 7fc5a9e3a700 1 -- 172.21.15.52:0/2773169831 shutdown_connections 2024-04-02T13:51:39.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.847+0000 7fc5a9e3a700 1 -- 172.21.15.52:0/2773169831 wait complete. 2024-04-02T13:51:39.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.847+0000 7fc5a9e3a700 1 Processor -- start 2024-04-02T13:51:39.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.847+0000 7fc5a9e3a700 1 -- start start 2024-04-02T13:51:39.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.848+0000 7fc5a9e3a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5a406ad10 0x7fc5a4198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.848+0000 7fc5a9e3a700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5a406b6b0 0x7fc5a41992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.848+0000 7fc5a9e3a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5a4107520 0x7fc5a419d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.848+0000 7fc5a9e3a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41128b0 con 0x7fc5a406ad10 2024-04-02T13:51:39.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.848+0000 7fc5a9e3a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a4112730 con 0x7fc5a4107520 2024-04-02T13:51:39.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.848+0000 7fc5a9e3a700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a4112a30 con 0x7fc5a406b6b0 2024-04-02T13:51:39.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.848+0000 7fc5a37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5a406ad10 0x7fc5a4198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.848+0000 7fc5a37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5a406ad10 0x7fc5a4198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44492/0 (socket says 172.21.15.52:44492) 2024-04-02T13:51:39.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.848+0000 7fc5a37fe700 1 -- 172.21.15.52:0/1600162299 learned_addr learned my addr 172.21.15.52:0/1600162299 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:39.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.848+0000 7fc5a3fff700 1 --2- 172.21.15.52:0/1600162299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5a4107520 0x7fc5a419d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.848+0000 7fc5a2ffd700 1 --2- 172.21.15.52:0/1600162299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5a406b6b0 0x7fc5a41992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.848+0000 7fc5a37fe700 1 -- 172.21.15.52:0/1600162299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5a406b6b0 msgr2=0x7fc5a41992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:39.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.848+0000 7fc5a37fe700 1 --2- 172.21.15.52:0/1600162299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5a406b6b0 0x7fc5a41992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.848+0000 7fc5a37fe700 1 -- 172.21.15.52:0/1600162299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5a4107520 msgr2=0x7fc5a419d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:39.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.848+0000 7fc5a37fe700 1 --2- 172.21.15.52:0/1600162299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5a4107520 0x7fc5a419d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.848+0000 7fc5a37fe700 1 -- 172.21.15.52:0/1600162299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a419dd20 con 0x7fc5a406ad10 2024-04-02T13:51:39.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.848+0000 7fc5a37fe700 1 --2- 172.21.15.52:0/1600162299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5a406ad10 0x7fc5a4198d60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc59400e970 tx=0x7fc59400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:39.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.849+0000 7fc5a0ff9700 1 -- 172.21.15.52:0/1600162299 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594004220 con 0x7fc5a406ad10 2024-04-02T13:51:39.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.849+0000 7fc5a0ff9700 1 -- 172.21.15.52:0/1600162299 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5940043c0 con 0x7fc5a406ad10 2024-04-02T13:51:39.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.849+0000 7fc5a9e3a700 1 -- 172.21.15.52:0/1600162299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a419e010 con 0x7fc5a406ad10 2024-04-02T13:51:39.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.849+0000 7fc5a0ff9700 1 -- 172.21.15.52:0/1600162299 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594003dc0 con 0x7fc5a406ad10 2024-04-02T13:51:39.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.849+0000 7fc5a9e3a700 1 -- 172.21.15.52:0/1600162299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a41a5890 con 0x7fc5a406ad10 2024-04-02T13:51:39.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.850+0000 7fc5a9e3a700 1 -- 172.21.15.52:0/1600162299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5a40692b0 con 0x7fc5a406ad10 2024-04-02T13:51:39.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.851+0000 7fc5a0ff9700 1 -- 172.21.15.52:0/1600162299 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc59400c7e0 con 0x7fc5a406ad10 2024-04-02T13:51:39.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.851+0000 7fc5a0ff9700 1 --2- 172.21.15.52:0/1600162299 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc5900a3cd0 0x7fc5900a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.852+0000 7fc5a2ffd700 1 --2- 172.21.15.52:0/1600162299 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc5900a3cd0 0x7fc5900a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.852+0000 7fc5a0ff9700 1 -- 172.21.15.52:0/1600162299 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc5940d0a20 con 0x7fc5a406ad10 2024-04-02T13:51:39.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.852+0000 7fc5a2ffd700 1 --2- 172.21.15.52:0/1600162299 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc5900a3cd0 0x7fc5900a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4078ae0 tx=0x7fc58c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:39.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:39.855+0000 7fc5a0ff9700 1 -- 172.21.15.52:0/1600162299 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc59409d0f0 con 0x7fc5a406ad10 2024-04-02T13:51:40.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:40.066+0000 7fc5a9e3a700 1 -- 172.21.15.52:0/1600162299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5a419a0a0 con 0x7fc5a406ad10 2024-04-02T13:51:40.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:40.067+0000 7fc5a0ff9700 1 -- 172.21.15.52:0/1600162299 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24077 (secure 0 0 0) 0x7fc5a419a0a0 con 0x7fc5a406ad10 2024-04-02T13:51:40.071 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:51:40.071 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":90,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T13:51:05.722301+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:50:58.778690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:50:59.561527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:51:03.343487+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:51:40.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:40.070+0000 7fc5a9e3a700 1 -- 172.21.15.52:0/1600162299 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc5900a3cd0 msgr2=0x7fc5900a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:40.070+0000 7fc5a9e3a700 1 --2- 172.21.15.52:0/1600162299 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc5900a3cd0 0x7fc5900a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4078ae0 tx=0x7fc58c00a300 comp rx=0 tx=0).stop 2024-04-02T13:51:40.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:40.070+0000 7fc5a9e3a700 1 -- 172.21.15.52:0/1600162299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5a406ad10 msgr2=0x7fc5a4198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:40.070+0000 7fc5a9e3a700 1 --2- 172.21.15.52:0/1600162299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5a406ad10 0x7fc5a4198d60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc59400e970 tx=0x7fc59400ec80 comp rx=0 tx=0).stop 2024-04-02T13:51:40.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:40.070+0000 7fc5a9e3a700 1 -- 172.21.15.52:0/1600162299 shutdown_connections 2024-04-02T13:51:40.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:40.070+0000 7fc5a9e3a700 1 --2- 172.21.15.52:0/1600162299 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc5900a3cd0 0x7fc5900a6190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:40.070+0000 7fc5a9e3a700 1 --2- 172.21.15.52:0/1600162299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5a4107520 0x7fc5a419d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:40.070+0000 7fc5a9e3a700 1 --2- 172.21.15.52:0/1600162299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5a406ad10 0x7fc5a4198d60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:40.070+0000 7fc5a9e3a700 1 --2- 172.21.15.52:0/1600162299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5a406b6b0 0x7fc5a41992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:40.070+0000 7fc5a9e3a700 1 -- 172.21.15.52:0/1600162299 >> 172.21.15.52:0/1600162299 conn(0x7fc5a4078500 msgr2=0x7fc5a40ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:40.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:40.070+0000 7fc5a9e3a700 1 -- 172.21.15.52:0/1600162299 shutdown_connections 2024-04-02T13:51:40.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:40.070+0000 7fc5a9e3a700 1 -- 172.21.15.52:0/1600162299 wait complete. 2024-04-02T13:51:40.734 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:40 smithi052 ceph-mon[32104]: pgmap v312: 129 pgs: 129 active+clean; 628 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.0 KiB/s wr, 22 op/s 2024-04-02T13:51:40.734 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:40 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1600162299' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:40.762 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-02T13:51:40.763 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-02T13:51:40.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:40 smithi189 ceph-mon[36683]: pgmap v312: 129 pgs: 129 active+clean; 628 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.0 KiB/s wr, 22 op/s 2024-04-02T13:51:40.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:40 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1600162299' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:40.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:40 smithi088 ceph-mon[36528]: pgmap v312: 129 pgs: 129 active+clean; 628 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.0 KiB/s wr, 22 op/s 2024-04-02T13:51:40.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:40 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1600162299' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:41.008 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:51:42.705 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:42 smithi052 ceph-mon[32104]: pgmap v313: 129 pgs: 129 active+clean; 636 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 KiB/s wr, 23 op/s 2024-04-02T13:51:42.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:42 smithi189 ceph-mon[36683]: pgmap v313: 129 pgs: 129 active+clean; 636 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 KiB/s wr, 23 op/s 2024-04-02T13:51:42.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.804+0000 7fe3cb72d700 1 -- 172.21.15.52:0/4171211953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3c4104ff0 msgr2=0x7fe3c4105480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:42.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.804+0000 7fe3cb72d700 1 --2- 172.21.15.52:0/4171211953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3c4104ff0 0x7fe3c4105480 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc009990 tx=0x7fe3bc009ca0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.804+0000 7fe3cb72d700 1 -- 172.21.15.52:0/4171211953 shutdown_connections 2024-04-02T13:51:42.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.804+0000 7fe3cb72d700 1 --2- 172.21.15.52:0/4171211953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3c4104ff0 0x7fe3c4105480 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.804+0000 7fe3cb72d700 1 --2- 172.21.15.52:0/4171211953 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3c41074e0 0x7fe3c406a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.804+0000 7fe3cb72d700 1 --2- 172.21.15.52:0/4171211953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3c406a7e0 0x7fe3c406ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.804+0000 7fe3cb72d700 1 -- 172.21.15.52:0/4171211953 >> 172.21.15.52:0/4171211953 conn(0x7fe3c4078420 msgr2=0x7fe3c4078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:42.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.804+0000 7fe3cb72d700 1 -- 172.21.15.52:0/4171211953 shutdown_connections 2024-04-02T13:51:42.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.804+0000 7fe3cb72d700 1 -- 172.21.15.52:0/4171211953 wait complete. 2024-04-02T13:51:42.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.805+0000 7fe3cb72d700 1 Processor -- start 2024-04-02T13:51:42.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.805+0000 7fe3cb72d700 1 -- start start 2024-04-02T13:51:42.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.805+0000 7fe3cb72d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3c406a7e0 0x7fe3c419f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.805+0000 7fe3cb72d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3c4104ff0 0x7fe3c419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.805+0000 7fe3cb72d700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3c41074e0 0x7fe3c41a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.805+0000 7fe3cb72d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4118e70 con 0x7fe3c4104ff0 2024-04-02T13:51:42.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.805+0000 7fe3cb72d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4118cf0 con 0x7fe3c406a7e0 2024-04-02T13:51:42.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.805+0000 7fe3cb72d700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4118ff0 con 0x7fe3c41074e0 2024-04-02T13:51:42.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.805+0000 7fe3c8cc8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3c4104ff0 0x7fe3c419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.805+0000 7fe3c8cc8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3c4104ff0 0x7fe3c419f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41526/0 (socket says 172.21.15.52:41526) 2024-04-02T13:51:42.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.805+0000 7fe3c8cc8700 1 -- 172.21.15.52:0/3533676228 learned_addr learned my addr 172.21.15.52:0/3533676228 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:42.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.805+0000 7fe3c94c9700 1 --2- 172.21.15.52:0/3533676228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3c406a7e0 0x7fe3c419f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.805+0000 7fe3c9cca700 1 --2- 172.21.15.52:0/3533676228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3c41074e0 0x7fe3c41a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.805+0000 7fe3c8cc8700 1 -- 172.21.15.52:0/3533676228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3c41074e0 msgr2=0x7fe3c41a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:42.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.805+0000 7fe3c8cc8700 1 --2- 172.21.15.52:0/3533676228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3c41074e0 0x7fe3c41a3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.805+0000 7fe3c8cc8700 1 -- 172.21.15.52:0/3533676228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3c406a7e0 msgr2=0x7fe3c419f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:42.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.805+0000 7fe3c8cc8700 1 --2- 172.21.15.52:0/3533676228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3c406a7e0 0x7fe3c419f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.805+0000 7fe3c8cc8700 1 -- 172.21.15.52:0/3533676228 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c41a42c0 con 0x7fe3c4104ff0 2024-04-02T13:51:42.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.805+0000 7fe3c9cca700 1 --2- 172.21.15.52:0/3533676228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3c41074e0 0x7fe3c41a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:51:42.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.806+0000 7fe3c8cc8700 1 --2- 172.21.15.52:0/3533676228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3c4104ff0 0x7fe3c419f840 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe3b400d950 tx=0x7fe3b400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:42.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.806+0000 7fe3ba7fc700 1 -- 172.21.15.52:0/3533676228 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4016070 con 0x7fe3c4104ff0 2024-04-02T13:51:42.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.806+0000 7fe3ba7fc700 1 -- 172.21.15.52:0/3533676228 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3b4009e20 con 0x7fe3c4104ff0 2024-04-02T13:51:42.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.806+0000 7fe3cb72d700 1 -- 172.21.15.52:0/3533676228 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c41a45b0 con 0x7fe3c4104ff0 2024-04-02T13:51:42.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.806+0000 7fe3ba7fc700 1 -- 172.21.15.52:0/3533676228 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b400f620 con 0x7fe3c4104ff0 2024-04-02T13:51:42.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.806+0000 7fe3cb72d700 1 -- 172.21.15.52:0/3533676228 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c41abe30 con 0x7fe3c4104ff0 2024-04-02T13:51:42.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.807+0000 7fe3cb72d700 1 -- 172.21.15.52:0/3533676228 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c40692b0 con 0x7fe3c4104ff0 2024-04-02T13:51:42.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.808+0000 7fe3ba7fc700 1 -- 172.21.15.52:0/3533676228 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe3b4019070 con 0x7fe3c4104ff0 2024-04-02T13:51:42.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.808+0000 7fe3ba7fc700 1 --2- 172.21.15.52:0/3533676228 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe3b00a3cd0 0x7fe3b00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.808+0000 7fe3c94c9700 1 --2- 172.21.15.52:0/3533676228 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe3b00a3cd0 0x7fe3b00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.809+0000 7fe3ba7fc700 1 -- 172.21.15.52:0/3533676228 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe3b40d0380 con 0x7fe3c4104ff0 2024-04-02T13:51:42.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.809+0000 7fe3c94c9700 1 --2- 172.21.15.52:0/3533676228 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe3b00a3cd0 0x7fe3b00a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009550 tx=0x7fe3c00091f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:42.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:42.812+0000 7fe3ba7fc700 1 -- 172.21.15.52:0/3533676228 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe3b409c9c0 con 0x7fe3c4104ff0 2024-04-02T13:51:42.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:42 smithi088 ceph-mon[36528]: pgmap v313: 129 pgs: 129 active+clean; 636 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 KiB/s wr, 23 op/s 2024-04-02T13:51:43.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:43.018+0000 7fe3cb72d700 1 -- 172.21.15.52:0/3533676228 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe3c41a0610 con 0x7fe3c4104ff0 2024-04-02T13:51:43.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:43.020+0000 7fe3ba7fc700 1 -- 172.21.15.52:0/3533676228 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24077 (secure 0 0 0) 0x7fe3c41a0610 con 0x7fe3c4104ff0 2024-04-02T13:51:43.024 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:51:43.024 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":90,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T13:51:05.722301+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:50:58.778690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:50:59.561527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:51:03.343487+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:51:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:43.022+0000 7fe3cb72d700 1 -- 172.21.15.52:0/3533676228 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe3b00a3cd0 msgr2=0x7fe3b00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:43.022+0000 7fe3cb72d700 1 --2- 172.21.15.52:0/3533676228 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe3b00a3cd0 0x7fe3b00a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009550 tx=0x7fe3c00091f0 comp rx=0 tx=0).stop 2024-04-02T13:51:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:43.022+0000 7fe3cb72d700 1 -- 172.21.15.52:0/3533676228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3c4104ff0 msgr2=0x7fe3c419f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:43.022+0000 7fe3cb72d700 1 --2- 172.21.15.52:0/3533676228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3c4104ff0 0x7fe3c419f840 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe3b400d950 tx=0x7fe3b400dc60 comp rx=0 tx=0).stop 2024-04-02T13:51:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:43.023+0000 7fe3cb72d700 1 -- 172.21.15.52:0/3533676228 shutdown_connections 2024-04-02T13:51:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:43.023+0000 7fe3cb72d700 1 --2- 172.21.15.52:0/3533676228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3c41074e0 0x7fe3c41a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:43.023+0000 7fe3cb72d700 1 --2- 172.21.15.52:0/3533676228 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe3b00a3cd0 0x7fe3b00a6190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:43.023+0000 7fe3cb72d700 1 --2- 172.21.15.52:0/3533676228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3c406a7e0 0x7fe3c419f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:43.023+0000 7fe3cb72d700 1 --2- 172.21.15.52:0/3533676228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3c4104ff0 0x7fe3c419f840 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:43.023+0000 7fe3cb72d700 1 -- 172.21.15.52:0/3533676228 >> 172.21.15.52:0/3533676228 conn(0x7fe3c4078420 msgr2=0x7fe3c4103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:43.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:43.023+0000 7fe3cb72d700 1 -- 172.21.15.52:0/3533676228 shutdown_connections 2024-04-02T13:51:43.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:43.023+0000 7fe3cb72d700 1 -- 172.21.15.52:0/3533676228 wait complete. 2024-04-02T13:51:43.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:43 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3533676228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:43.794 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:43 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3533676228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:43.828 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph tell osd.0 flush_pg_stats 2024-04-02T13:51:43.828 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph tell osd.1 flush_pg_stats 2024-04-02T13:51:43.828 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph tell osd.2 flush_pg_stats 2024-04-02T13:51:43.829 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph tell osd.3 flush_pg_stats 2024-04-02T13:51:43.829 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph tell osd.4 flush_pg_stats 2024-04-02T13:51:43.830 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph tell osd.5 flush_pg_stats 2024-04-02T13:51:43.830 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph tell osd.6 flush_pg_stats 2024-04-02T13:51:43.830 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph tell osd.7 flush_pg_stats 2024-04-02T13:51:43.831 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph tell osd.8 flush_pg_stats 2024-04-02T13:51:43.831 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph tell osd.9 flush_pg_stats 2024-04-02T13:51:43.832 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph tell osd.10 flush_pg_stats 2024-04-02T13:51:43.832 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph tell osd.11 flush_pg_stats 2024-04-02T13:51:43.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:43 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3533676228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:51:44.441 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:51:44.736 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:44 smithi052 ceph-mon[32104]: pgmap v314: 129 pgs: 129 active+clean; 636 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-04-02T13:51:44.752 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:51:44.752 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:51:44.753 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:51:44.753 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:51:44.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:44 smithi189 ceph-mon[36683]: pgmap v314: 129 pgs: 129 active+clean; 636 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-04-02T13:51:44.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:44 smithi088 ceph-mon[36528]: pgmap v314: 129 pgs: 129 active+clean; 636 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-04-02T13:51:46.008 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:51:46.009 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:51:46.009 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:51:46.009 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:51:46.009 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:51:46.465 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:51:46.569 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:46 smithi052 ceph-mon[32104]: pgmap v315: 129 pgs: 129 active+clean; 636 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-02T13:51:46.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:46 smithi189 ceph-mon[36683]: pgmap v315: 129 pgs: 129 active+clean; 636 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-02T13:51:46.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:46 smithi088 ceph-mon[36528]: pgmap v315: 129 pgs: 129 active+clean; 636 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-02T13:51:48.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:48 smithi052 ceph-mon[32104]: pgmap v316: 129 pgs: 129 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-04-02T13:51:48.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:48 smithi088 ceph-mon[36528]: pgmap v316: 129 pgs: 129 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-04-02T13:51:48.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:48 smithi189 ceph-mon[36683]: pgmap v316: 129 pgs: 129 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-04-02T13:51:50.718 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:50 smithi052 ceph-mon[32104]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-02T13:51:50.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:50 smithi189 ceph-mon[36683]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-02T13:51:50.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:50 smithi088 ceph-mon[36528]: pgmap v317: 129 pgs: 129 active+clean; 640 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-02T13:51:51.589 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:51:52.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.628+0000 7f2bed0d1700 1 -- 172.21.15.52:0/3815263315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2be8107520 msgr2=0x7f2be8107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:52.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.628+0000 7f2bed0d1700 1 --2- 172.21.15.52:0/3815263315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2be8107520 0x7f2be8107900 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800b0a0 tx=0x7f2bd800b3b0 comp rx=0 tx=0).stop 2024-04-02T13:51:52.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.628+0000 7f2bed0d1700 1 -- 172.21.15.52:0/3815263315 shutdown_connections 2024-04-02T13:51:52.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.628+0000 7f2bed0d1700 1 --2- 172.21.15.52:0/3815263315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2be8102fe0 0x7f2be810f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:52.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.628+0000 7f2bed0d1700 1 --2- 172.21.15.52:0/3815263315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2be8107520 0x7f2be8107900 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:52.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.628+0000 7f2bed0d1700 1 --2- 172.21.15.52:0/3815263315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2be8102640 0x7f2be8102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:52.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.628+0000 7f2bed0d1700 1 -- 172.21.15.52:0/3815263315 >> 172.21.15.52:0/3815263315 conn(0x7f2be8079720 msgr2=0x7f2be8079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:52.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.628+0000 7f2bed0d1700 1 -- 172.21.15.52:0/3815263315 shutdown_connections 2024-04-02T13:51:52.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.628+0000 7f2bed0d1700 1 -- 172.21.15.52:0/3815263315 wait complete. 2024-04-02T13:51:52.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.628+0000 7f2bed0d1700 1 Processor -- start 2024-04-02T13:51:52.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.629+0000 7f2bed0d1700 1 -- start start 2024-04-02T13:51:52.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.629+0000 7f2bed0d1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2be8102640 0x7f2be8198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:52.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.629+0000 7f2bed0d1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2be8102fe0 0x7f2be8199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:52.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.629+0000 7f2bed0d1700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2be8107520 0x7f2be819d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:52.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.629+0000 7f2bed0d1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81128b0 con 0x7f2be8102fe0 2024-04-02T13:51:52.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.629+0000 7f2bed0d1700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2be8112730 con 0x7f2be8102640 2024-04-02T13:51:52.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.629+0000 7f2bed0d1700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2be8112a30 con 0x7f2be8107520 2024-04-02T13:51:52.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.629+0000 7f2be659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2be8102fe0 0x7f2be8199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.629+0000 7f2be659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2be8102fe0 0x7f2be8199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40716/0 (socket says 172.21.15.52:40716) 2024-04-02T13:51:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.629+0000 7f2be659c700 1 -- 172.21.15.52:0/1789739349 learned_addr learned my addr 172.21.15.52:0/1789739349 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.629+0000 7f2be759e700 1 --2- 172.21.15.52:0/1789739349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2be8107520 0x7f2be819d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.629+0000 7f2be6d9d700 1 --2- 172.21.15.52:0/1789739349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2be8102640 0x7f2be8198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.629+0000 7f2be659c700 1 -- 172.21.15.52:0/1789739349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2be8107520 msgr2=0x7f2be819d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.629+0000 7f2be659c700 1 --2- 172.21.15.52:0/1789739349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2be8107520 0x7f2be819d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.629+0000 7f2be659c700 1 -- 172.21.15.52:0/1789739349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2be8102640 msgr2=0x7f2be8198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.629+0000 7f2be659c700 1 --2- 172.21.15.52:0/1789739349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2be8102640 0x7f2be8198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.629+0000 7f2be659c700 1 -- 172.21.15.52:0/1789739349 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be819de50 con 0x7f2be8102fe0 2024-04-02T13:51:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.630+0000 7f2be6d9d700 1 --2- 172.21.15.52:0/1789739349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2be8102640 0x7f2be8198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:51:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.630+0000 7f2be659c700 1 --2- 172.21.15.52:0/1789739349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2be8102fe0 0x7f2be8199420 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000d980 tx=0x7f2bd000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.630+0000 7f2bdffff700 1 -- 172.21.15.52:0/1789739349 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0017070 con 0x7f2be8102fe0 2024-04-02T13:51:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.630+0000 7f2bed0d1700 1 -- 172.21.15.52:0/1789739349 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be819e140 con 0x7f2be8102fe0 2024-04-02T13:51:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.630+0000 7f2bdffff700 1 -- 172.21.15.52:0/1789739349 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd0005020 con 0x7f2be8102fe0 2024-04-02T13:51:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.630+0000 7f2bdffff700 1 -- 172.21.15.52:0/1789739349 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0009bc0 con 0x7f2be8102fe0 2024-04-02T13:51:52.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.630+0000 7f2bed0d1700 1 -- 172.21.15.52:0/1789739349 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be81a59c0 con 0x7f2be8102fe0 2024-04-02T13:51:52.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.631+0000 7f2bed0d1700 1 -- 172.21.15.52:0/1789739349 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2be8062d00 con 0x7f2be8102fe0 2024-04-02T13:51:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.632+0000 7f2bdffff700 1 -- 172.21.15.52:0/1789739349 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2bd00051d0 con 0x7f2be8102fe0 2024-04-02T13:51:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.633+0000 7f2bdffff700 1 --2- 172.21.15.52:0/1789739349 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2bd40a3cd0 0x7f2bd40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.633+0000 7f2be6d9d700 1 --2- 172.21.15.52:0/1789739349 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2bd40a3cd0 0x7f2bd40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.633+0000 7f2bdffff700 1 -- 172.21.15.52:0/1789739349 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f2bd00d0c90 con 0x7f2be8102fe0 2024-04-02T13:51:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.633+0000 7f2bdffff700 1 --2- 172.21.15.52:0/1789739349 >> [v2:172.21.15.189:6824/383422941,v1:172.21.15.189:6825/383422941] conn(0x7f2bd40ae990 0x7f2bd40b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.633+0000 7f2bdffff700 1 -- 172.21.15.52:0/1789739349 --> [v2:172.21.15.189:6824/383422941,v1:172.21.15.189:6825/383422941] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2bd40b14a0 con 0x7f2bd40ae990 2024-04-02T13:51:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.633+0000 7f2bdffff700 1 -- 172.21.15.52:0/1789739349 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f2bd00d10e0 con 0x7f2be8102fe0 2024-04-02T13:51:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.633+0000 7f2be6d9d700 1 --2- 172.21.15.52:0/1789739349 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2bd40a3cd0 0x7f2bd40a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800b070 tx=0x7f2bd8009370 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.633+0000 7f2be759e700 1 --2- 172.21.15.52:0/1789739349 >> [v2:172.21.15.189:6824/383422941,v1:172.21.15.189:6825/383422941] conn(0x7f2bd40ae990 0x7f2bd40b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.634+0000 7f2be759e700 1 --2- 172.21.15.52:0/1789739349 >> [v2:172.21.15.189:6824/383422941,v1:172.21.15.189:6825/383422941] conn(0x7f2bd40ae990 0x7f2bd40b0dd0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:52.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.635+0000 7f2bdffff700 1 -- 172.21.15.52:0/1789739349 <== osd.11 v2:172.21.15.189:6824/383422941 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2bd40b14a0 con 0x7f2bd40ae990 2024-04-02T13:51:52.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.652+0000 7f2bed0d1700 1 -- 172.21.15.52:0/1789739349 --> [v2:172.21.15.189:6824/383422941,v1:172.21.15.189:6825/383422941] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2be8063180 con 0x7f2bd40ae990 2024-04-02T13:51:52.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.653+0000 7f2bdffff700 1 -- 172.21.15.52:0/1789739349 <== osd.11 v2:172.21.15.189:6824/383422941 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2be8063180 con 0x7f2bd40ae990 2024-04-02T13:51:52.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.653+0000 7f2bed0d1700 1 -- 172.21.15.52:0/1789739349 >> [v2:172.21.15.189:6824/383422941,v1:172.21.15.189:6825/383422941] conn(0x7f2bd40ae990 msgr2=0x7f2bd40b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:52.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.654+0000 7f2bed0d1700 1 --2- 172.21.15.52:0/1789739349 >> [v2:172.21.15.189:6824/383422941,v1:172.21.15.189:6825/383422941] conn(0x7f2bd40ae990 0x7f2bd40b0dd0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:52.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.654+0000 7f2bed0d1700 1 -- 172.21.15.52:0/1789739349 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2bd40a3cd0 msgr2=0x7f2bd40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:52.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.654+0000 7f2bed0d1700 1 --2- 172.21.15.52:0/1789739349 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2bd40a3cd0 0x7f2bd40a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800b070 tx=0x7f2bd8009370 comp rx=0 tx=0).stop 2024-04-02T13:51:52.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.654+0000 7f2bed0d1700 1 -- 172.21.15.52:0/1789739349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2be8102fe0 msgr2=0x7f2be8199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:52.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.654+0000 7f2bed0d1700 1 --2- 172.21.15.52:0/1789739349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2be8102fe0 0x7f2be8199420 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000d980 tx=0x7f2bd000dc90 comp rx=0 tx=0).stop 2024-04-02T13:51:52.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.654+0000 7f2be759e700 1 -- 172.21.15.52:0/1789739349 reap_dead start 2024-04-02T13:51:52.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.654+0000 7f2bed0d1700 1 -- 172.21.15.52:0/1789739349 shutdown_connections 2024-04-02T13:51:52.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.654+0000 7f2bed0d1700 1 -- 172.21.15.52:0/1789739349 >> 172.21.15.52:0/1789739349 conn(0x7f2be8079720 msgr2=0x7f2be8100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:52.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.654+0000 7f2bed0d1700 1 -- 172.21.15.52:0/1789739349 shutdown_connections 2024-04-02T13:51:52.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:52.654+0000 7f2bed0d1700 1 -- 172.21.15.52:0/1789739349 wait complete. 2024-04-02T13:51:52.667 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:52 smithi052 ceph-mon[32104]: pgmap v318: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-02T13:51:52.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:52 smithi189 ceph-mon[36683]: pgmap v318: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-02T13:51:52.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:52 smithi088 ceph-mon[36528]: pgmap v318: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-02T13:51:54.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.385+0000 7f1c9259f700 1 -- 172.21.15.52:0/3868620511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c8c06a2a0 msgr2=0x7f1c8c06a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:54.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.385+0000 7f1c9259f700 1 --2- 172.21.15.52:0/3868620511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c8c06a2a0 0x7f1c8c06a720 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009990 tx=0x7f1c80009ca0 comp rx=0 tx=0).stop 2024-04-02T13:51:54.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.385+0000 7f1c9259f700 1 -- 172.21.15.52:0/3868620511 shutdown_connections 2024-04-02T13:51:54.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.385+0000 7f1c9259f700 1 --2- 172.21.15.52:0/3868620511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c8c06ac60 0x7f1c8c10f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:54.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.385+0000 7f1c9259f700 1 --2- 172.21.15.52:0/3868620511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c8c108650 0x7f1c8c108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:54.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.385+0000 7f1c9259f700 1 --2- 172.21.15.52:0/3868620511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c8c06a2a0 0x7f1c8c06a720 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:54.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.385+0000 7f1c9259f700 1 -- 172.21.15.52:0/3868620511 >> 172.21.15.52:0/3868620511 conn(0x7f1c8c078420 msgr2=0x7f1c8c078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:54.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.386+0000 7f1c9259f700 1 -- 172.21.15.52:0/3868620511 shutdown_connections 2024-04-02T13:51:54.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.386+0000 7f1c9259f700 1 -- 172.21.15.52:0/3868620511 wait complete. 2024-04-02T13:51:54.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.386+0000 7f1c9259f700 1 Processor -- start 2024-04-02T13:51:54.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.386+0000 7f1c9259f700 1 -- start start 2024-04-02T13:51:54.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.387+0000 7f1c9259f700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c8c06a2a0 0x7f1c8c1015c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:54.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.387+0000 7f1c9259f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c8c06ac60 0x7f1c8c101b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:54.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.387+0000 7f1c9259f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c8c108650 0x7f1c8c102040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:54.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.387+0000 7f1c9259f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c112940 con 0x7f1c8c06ac60 2024-04-02T13:51:54.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.387+0000 7f1c9259f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c1127c0 con 0x7f1c8c108650 2024-04-02T13:51:54.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.387+0000 7f1c9259f700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c112ac0 con 0x7f1c8c06a2a0 2024-04-02T13:51:54.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.387+0000 7f1c8b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c8c06ac60 0x7f1c8c101b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:54.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.387+0000 7f1c8b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c8c06ac60 0x7f1c8c101b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40746/0 (socket says 172.21.15.52:40746) 2024-04-02T13:51:54.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.387+0000 7f1c8b7fe700 1 -- 172.21.15.52:0/3687763314 learned_addr learned my addr 172.21.15.52:0/3687763314 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:54.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.387+0000 7f1c90b3c700 1 --2- 172.21.15.52:0/3687763314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c8c108650 0x7f1c8c102040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:54.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.387+0000 7f1c8bfff700 1 --2- 172.21.15.52:0/3687763314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c8c06a2a0 0x7f1c8c1015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:54.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.387+0000 7f1c8b7fe700 1 -- 172.21.15.52:0/3687763314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c8c06a2a0 msgr2=0x7f1c8c1015c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:54.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.387+0000 7f1c8b7fe700 1 --2- 172.21.15.52:0/3687763314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c8c06a2a0 0x7f1c8c1015c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:54.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.387+0000 7f1c8b7fe700 1 -- 172.21.15.52:0/3687763314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c8c108650 msgr2=0x7f1c8c102040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:54.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.387+0000 7f1c8b7fe700 1 --2- 172.21.15.52:0/3687763314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c8c108650 0x7f1c8c102040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:54.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.387+0000 7f1c8b7fe700 1 -- 172.21.15.52:0/3687763314 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c8c1a72a0 con 0x7f1c8c06ac60 2024-04-02T13:51:54.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.387+0000 7f1c8bfff700 1 --2- 172.21.15.52:0/3687763314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c8c06a2a0 0x7f1c8c1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:51:54.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.388+0000 7f1c8b7fe700 1 --2- 172.21.15.52:0/3687763314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c8c06ac60 0x7f1c8c101b00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1c80000c00 tx=0x7f1c800047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:54.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.388+0000 7f1c897fa700 1 -- 172.21.15.52:0/3687763314 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80027070 con 0x7f1c8c06ac60 2024-04-02T13:51:54.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.388+0000 7f1c897fa700 1 -- 172.21.15.52:0/3687763314 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c80003780 con 0x7f1c8c06ac60 2024-04-02T13:51:54.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.388+0000 7f1c897fa700 1 -- 172.21.15.52:0/3687763314 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80022dd0 con 0x7f1c8c06ac60 2024-04-02T13:51:54.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.388+0000 7f1c9259f700 1 -- 172.21.15.52:0/3687763314 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c8c1a7530 con 0x7f1c8c06ac60 2024-04-02T13:51:54.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.388+0000 7f1c9259f700 1 -- 172.21.15.52:0/3687763314 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c8c1a7930 con 0x7f1c8c06ac60 2024-04-02T13:51:54.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.389+0000 7f1c9259f700 1 -- 172.21.15.52:0/3687763314 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1c8c062d00 con 0x7f1c8c06ac60 2024-04-02T13:51:54.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.391+0000 7f1c897fa700 1 -- 172.21.15.52:0/3687763314 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1c8002a070 con 0x7f1c8c06ac60 2024-04-02T13:51:54.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.392+0000 7f1c897fa700 1 --2- 172.21.15.52:0/3687763314 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1c740a3cd0 0x7f1c740a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:54.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.392+0000 7f1c897fa700 1 -- 172.21.15.52:0/3687763314 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f1c800d1ac0 con 0x7f1c8c06ac60 2024-04-02T13:51:54.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.392+0000 7f1c8bfff700 1 --2- 172.21.15.52:0/3687763314 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1c740a3cd0 0x7f1c740a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:54.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.392+0000 7f1c897fa700 1 --2- 172.21.15.52:0/3687763314 >> [v2:172.21.15.88:6816/714628313,v1:172.21.15.88:6817/714628313] conn(0x7f1c740ae990 0x7f1c740b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:54.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.392+0000 7f1c897fa700 1 -- 172.21.15.52:0/3687763314 --> [v2:172.21.15.88:6816/714628313,v1:172.21.15.88:6817/714628313] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1c800056d0 con 0x7f1c740ae990 2024-04-02T13:51:54.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.392+0000 7f1c897fa700 1 -- 172.21.15.52:0/3687763314 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f1c800d1f10 con 0x7f1c8c06ac60 2024-04-02T13:51:54.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.392+0000 7f1c90b3c700 1 --2- 172.21.15.52:0/3687763314 >> [v2:172.21.15.88:6816/714628313,v1:172.21.15.88:6817/714628313] conn(0x7f1c740ae990 0x7f1c740b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:54.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.392+0000 7f1c8bfff700 1 --2- 172.21.15.52:0/3687763314 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1c740a3cd0 0x7f1c740a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c005e10 tx=0x7f1c7c005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:54.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.393+0000 7f1c90b3c700 1 --2- 172.21.15.52:0/3687763314 >> [v2:172.21.15.88:6816/714628313,v1:172.21.15.88:6817/714628313] conn(0x7f1c740ae990 0x7f1c740b0dd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:54.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.394+0000 7f1c897fa700 1 -- 172.21.15.52:0/3687763314 <== osd.7 v2:172.21.15.88:6816/714628313 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1c800056d0 con 0x7f1c740ae990 2024-04-02T13:51:54.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.411+0000 7f1c9259f700 1 -- 172.21.15.52:0/3687763314 --> [v2:172.21.15.88:6816/714628313,v1:172.21.15.88:6817/714628313] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1c8c063180 con 0x7f1c740ae990 2024-04-02T13:51:54.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.412+0000 7f1c897fa700 1 -- 172.21.15.52:0/3687763314 <== osd.7 v2:172.21.15.88:6816/714628313 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1c8c063180 con 0x7f1c740ae990 2024-04-02T13:51:54.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.413+0000 7f1c9259f700 1 -- 172.21.15.52:0/3687763314 >> [v2:172.21.15.88:6816/714628313,v1:172.21.15.88:6817/714628313] conn(0x7f1c740ae990 msgr2=0x7f1c740b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:54.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.413+0000 7f1c9259f700 1 --2- 172.21.15.52:0/3687763314 >> [v2:172.21.15.88:6816/714628313,v1:172.21.15.88:6817/714628313] conn(0x7f1c740ae990 0x7f1c740b0dd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:54.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.413+0000 7f1c9259f700 1 -- 172.21.15.52:0/3687763314 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1c740a3cd0 msgr2=0x7f1c740a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:54.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.413+0000 7f1c9259f700 1 --2- 172.21.15.52:0/3687763314 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1c740a3cd0 0x7f1c740a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c005e10 tx=0x7f1c7c005d80 comp rx=0 tx=0).stop 2024-04-02T13:51:54.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.413+0000 7f1c9259f700 1 -- 172.21.15.52:0/3687763314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c8c06ac60 msgr2=0x7f1c8c101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:54.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.413+0000 7f1c9259f700 1 --2- 172.21.15.52:0/3687763314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c8c06ac60 0x7f1c8c101b00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1c80000c00 tx=0x7f1c800047b0 comp rx=0 tx=0).stop 2024-04-02T13:51:54.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.413+0000 7f1c90b3c700 1 -- 172.21.15.52:0/3687763314 reap_dead start 2024-04-02T13:51:54.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.413+0000 7f1c9259f700 1 -- 172.21.15.52:0/3687763314 shutdown_connections 2024-04-02T13:51:54.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.413+0000 7f1c9259f700 1 -- 172.21.15.52:0/3687763314 >> 172.21.15.52:0/3687763314 conn(0x7f1c8c078420 msgr2=0x7f1c8c1002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:54.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.413+0000 7f1c9259f700 1 -- 172.21.15.52:0/3687763314 shutdown_connections 2024-04-02T13:51:54.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:54.413+0000 7f1c9259f700 1 -- 172.21.15.52:0/3687763314 wait complete. 2024-04-02T13:51:54.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:54 smithi189 ceph-mon[36683]: pgmap v319: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-02T13:51:54.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:54 smithi052 ceph-mon[32104]: pgmap v319: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-02T13:51:54.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:54 smithi088 ceph-mon[36528]: pgmap v319: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-02T13:51:55.719 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:55 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-02T13:51:55.719 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:55 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-02T13:51:55.719 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:55 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-04-02T13:51:55.719 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:55 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-04-02T13:51:55.719 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:55 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-04-02T13:51:55.719 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:55 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-02T13:51:55.719 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:55 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]: dispatch 2024-04-02T13:51:55.719 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:55 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]: dispatch 2024-04-02T13:51:55.719 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:55 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-04-02T13:51:55.719 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:55 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-04-02T13:51:55.763 INFO:teuthology.orchestra.run.smithi052.stdout:347892351000 2024-04-02T13:51:55.763 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.11 2024-04-02T13:51:55.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:55 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-02T13:51:55.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:55 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-02T13:51:55.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:55 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-04-02T13:51:55.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:55 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-04-02T13:51:55.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:55 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-04-02T13:51:55.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:55 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-02T13:51:55.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:55 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]: dispatch 2024-04-02T13:51:55.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:55 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]: dispatch 2024-04-02T13:51:55.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:55 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-04-02T13:51:55.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:55 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-04-02T13:51:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-02T13:51:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-02T13:51:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-04-02T13:51:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-04-02T13:51:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-04-02T13:51:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-02T13:51:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]: dispatch 2024-04-02T13:51:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]: dispatch 2024-04-02T13:51:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-04-02T13:51:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:55 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-04-02T13:51:56.645 INFO:teuthology.orchestra.run.smithi052.stdout:219043332141 2024-04-02T13:51:56.645 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.7 2024-04-02T13:51:56.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:56 smithi189 ceph-mon[36683]: pgmap v320: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-02T13:51:56.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-02T13:51:56.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-02T13:51:56.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-04-02T13:51:56.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-04-02T13:51:56.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-04-02T13:51:56.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-02T13:51:56.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]': finished 2024-04-02T13:51:56.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2024-04-02T13:51:56.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-04-02T13:51:56.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:56 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-04-02T13:51:56.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:56 smithi189 ceph-mon[36683]: osdmap e91: 12 total, 12 up, 12 in 2024-04-02T13:51:56.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:56 smithi052 ceph-mon[32104]: pgmap v320: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-02T13:51:56.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-02T13:51:56.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-02T13:51:56.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-04-02T13:51:56.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-04-02T13:51:56.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-04-02T13:51:56.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-02T13:51:56.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]': finished 2024-04-02T13:51:56.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2024-04-02T13:51:56.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-04-02T13:51:56.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:56 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-04-02T13:51:56.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:56 smithi052 ceph-mon[32104]: osdmap e91: 12 total, 12 up, 12 in 2024-04-02T13:51:56.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:56 smithi088 ceph-mon[36528]: pgmap v320: 129 pgs: 129 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-02T13:51:56.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-02T13:51:56.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-02T13:51:56.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-04-02T13:51:56.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-04-02T13:51:56.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-04-02T13:51:56.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-02T13:51:56.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]': finished 2024-04-02T13:51:56.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2024-04-02T13:51:56.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-04-02T13:51:56.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:56 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-04-02T13:51:56.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:56 smithi088 ceph-mon[36528]: osdmap e91: 12 total, 12 up, 12 in 2024-04-02T13:51:57.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:57 smithi189 ceph-mon[36683]: osdmap e92: 12 total, 12 up, 12 in 2024-04-02T13:51:57.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:57 smithi052 ceph-mon[32104]: osdmap e92: 12 total, 12 up, 12 in 2024-04-02T13:51:57.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:57 smithi088 ceph-mon[36528]: osdmap e92: 12 total, 12 up, 12 in 2024-04-02T13:51:58.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.227+0000 7facc0c34700 1 -- 172.21.15.52:0/3630017348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facbc1075f0 msgr2=0x7facbc1079d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:58.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.227+0000 7facc0c34700 1 --2- 172.21.15.52:0/3630017348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facbc1075f0 0x7facbc1079d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7facac00b0a0 tx=0x7facac00b3b0 comp rx=0 tx=0).stop 2024-04-02T13:51:58.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.227+0000 7facc0c34700 1 -- 172.21.15.52:0/3630017348 shutdown_connections 2024-04-02T13:51:58.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.227+0000 7facc0c34700 1 --2- 172.21.15.52:0/3630017348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facbc101ea0 0x7facbc10e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:58.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.227+0000 7facc0c34700 1 --2- 172.21.15.52:0/3630017348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facbc1075f0 0x7facbc1079d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:58.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.227+0000 7facc0c34700 1 --2- 172.21.15.52:0/3630017348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facbc1014e0 0x7facbc101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:58.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.227+0000 7facc0c34700 1 -- 172.21.15.52:0/3630017348 >> 172.21.15.52:0/3630017348 conn(0x7facbc078460 msgr2=0x7facbc078870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:58.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.227+0000 7facc0c34700 1 -- 172.21.15.52:0/3630017348 shutdown_connections 2024-04-02T13:51:58.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.227+0000 7facc0c34700 1 -- 172.21.15.52:0/3630017348 wait complete. 2024-04-02T13:51:58.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.228+0000 7facc0c34700 1 Processor -- start 2024-04-02T13:51:58.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.228+0000 7facc0c34700 1 -- start start 2024-04-02T13:51:58.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.229+0000 7facc0c34700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facbc1014e0 0x7facbc19d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:58.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.229+0000 7facc0c34700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facbc101ea0 0x7facbc19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:58.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.229+0000 7facc0c34700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facbc1075f0 0x7facbc197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:58.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.229+0000 7facba59c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facbc1014e0 0x7facbc19d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:58.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.229+0000 7facbad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facbc1075f0 0x7facbc197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:58.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.229+0000 7facc0c34700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7facbc1117b0 con 0x7facbc1075f0 2024-04-02T13:51:58.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.229+0000 7facc0c34700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7facbc111630 con 0x7facbc101ea0 2024-04-02T13:51:58.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.229+0000 7facc0c34700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7facbc111930 con 0x7facbc1014e0 2024-04-02T13:51:58.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.229+0000 7facbad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facbc1075f0 0x7facbc197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40766/0 (socket says 172.21.15.52:40766) 2024-04-02T13:51:58.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.229+0000 7facbad9d700 1 -- 172.21.15.52:0/3312029873 learned_addr learned my addr 172.21.15.52:0/3312029873 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:51:58.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.229+0000 7facb9d9b700 1 --2- 172.21.15.52:0/3312029873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facbc101ea0 0x7facbc19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:58.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.229+0000 7facba59c700 1 -- 172.21.15.52:0/3312029873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facbc101ea0 msgr2=0x7facbc19de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:58.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.229+0000 7facba59c700 1 --2- 172.21.15.52:0/3312029873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facbc101ea0 0x7facbc19de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:58.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.229+0000 7facba59c700 1 -- 172.21.15.52:0/3312029873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facbc1075f0 msgr2=0x7facbc197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:58.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.229+0000 7facba59c700 1 --2- 172.21.15.52:0/3312029873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facbc1075f0 0x7facbc197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:58.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.229+0000 7facba59c700 1 -- 172.21.15.52:0/3312029873 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facbc198060 con 0x7facbc1014e0 2024-04-02T13:51:58.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.230+0000 7facba59c700 1 --2- 172.21.15.52:0/3312029873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facbc1014e0 0x7facbc19d930 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7facac00b070 tx=0x7facac004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:58.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.230+0000 7facb37fe700 1 -- 172.21.15.52:0/3312029873 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac021070 con 0x7facbc1014e0 2024-04-02T13:51:58.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.230+0000 7facc0c34700 1 -- 172.21.15.52:0/3312029873 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facbc1982f0 con 0x7facbc1014e0 2024-04-02T13:51:58.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.230+0000 7facb37fe700 1 -- 172.21.15.52:0/3312029873 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facac007b80 con 0x7facbc1014e0 2024-04-02T13:51:58.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.230+0000 7facb37fe700 1 -- 172.21.15.52:0/3312029873 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac0052a0 con 0x7facbc1014e0 2024-04-02T13:51:58.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.230+0000 7facc0c34700 1 -- 172.21.15.52:0/3312029873 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facbc198820 con 0x7facbc1014e0 2024-04-02T13:51:58.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.231+0000 7facc0c34700 1 -- 172.21.15.52:0/3312029873 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7facbc06afe0 con 0x7facbc1014e0 2024-04-02T13:51:58.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.233+0000 7facb37fe700 1 -- 172.21.15.52:0/3312029873 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7facac024070 con 0x7facbc1014e0 2024-04-02T13:51:58.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.233+0000 7facb37fe700 1 --2- 172.21.15.52:0/3312029873 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faca80a3cd0 0x7faca80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:58.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.233+0000 7facb37fe700 1 -- 172.21.15.52:0/3312029873 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7facac00f020 con 0x7facbc1014e0 2024-04-02T13:51:58.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.233+0000 7facb9d9b700 1 --2- 172.21.15.52:0/3312029873 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faca80a3cd0 0x7faca80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:58.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.233+0000 7facb37fe700 1 --2- 172.21.15.52:0/3312029873 >> [v2:172.21.15.88:6824/3425831447,v1:172.21.15.88:6825/3425831447] conn(0x7faca80aecf0 0x7faca80b1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:58.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.233+0000 7facb37fe700 1 -- 172.21.15.52:0/3312029873 --> [v2:172.21.15.88:6824/3425831447,v1:172.21.15.88:6825/3425831447] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7facac013e70 con 0x7faca80aecf0 2024-04-02T13:51:58.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.233+0000 7facb37fe700 1 -- 172.21.15.52:0/3312029873 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7facac0d2500 con 0x7facbc1014e0 2024-04-02T13:51:58.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.234+0000 7facbad9d700 1 --2- 172.21.15.52:0/3312029873 >> [v2:172.21.15.88:6824/3425831447,v1:172.21.15.88:6825/3425831447] conn(0x7faca80aecf0 0x7faca80b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:58.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.234+0000 7facb9d9b700 1 --2- 172.21.15.52:0/3312029873 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faca80a3cd0 0x7faca80a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7facbc1063f0 tx=0x7faca400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:58.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.234+0000 7facbad9d700 1 --2- 172.21.15.52:0/3312029873 >> [v2:172.21.15.88:6824/3425831447,v1:172.21.15.88:6825/3425831447] conn(0x7faca80aecf0 0x7faca80b1190 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:58.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.235+0000 7facb37fe700 1 -- 172.21.15.52:0/3312029873 <== osd.10 v2:172.21.15.88:6824/3425831447 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7facac013e70 con 0x7faca80aecf0 2024-04-02T13:51:58.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.251+0000 7facc0c34700 1 -- 172.21.15.52:0/3312029873 --> [v2:172.21.15.88:6824/3425831447,v1:172.21.15.88:6825/3425831447] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7facbc06b410 con 0x7faca80aecf0 2024-04-02T13:51:58.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.252+0000 7facb37fe700 1 -- 172.21.15.52:0/3312029873 <== osd.10 v2:172.21.15.88:6824/3425831447 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7facbc06b410 con 0x7faca80aecf0 2024-04-02T13:51:58.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.252+0000 7facc0c34700 1 -- 172.21.15.52:0/3312029873 >> [v2:172.21.15.88:6824/3425831447,v1:172.21.15.88:6825/3425831447] conn(0x7faca80aecf0 msgr2=0x7faca80b1190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:58.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.252+0000 7facc0c34700 1 --2- 172.21.15.52:0/3312029873 >> [v2:172.21.15.88:6824/3425831447,v1:172.21.15.88:6825/3425831447] conn(0x7faca80aecf0 0x7faca80b1190 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:58.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.252+0000 7facc0c34700 1 -- 172.21.15.52:0/3312029873 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faca80a3cd0 msgr2=0x7faca80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:58.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.252+0000 7facc0c34700 1 --2- 172.21.15.52:0/3312029873 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faca80a3cd0 0x7faca80a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7facbc1063f0 tx=0x7faca400b410 comp rx=0 tx=0).stop 2024-04-02T13:51:58.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.252+0000 7facc0c34700 1 -- 172.21.15.52:0/3312029873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facbc1014e0 msgr2=0x7facbc19d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:58.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.252+0000 7facc0c34700 1 --2- 172.21.15.52:0/3312029873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facbc1014e0 0x7facbc19d930 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7facac00b070 tx=0x7facac004d00 comp rx=0 tx=0).stop 2024-04-02T13:51:58.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.252+0000 7facbad9d700 1 -- 172.21.15.52:0/3312029873 reap_dead start 2024-04-02T13:51:58.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.252+0000 7facc0c34700 1 -- 172.21.15.52:0/3312029873 shutdown_connections 2024-04-02T13:51:58.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.252+0000 7facc0c34700 1 -- 172.21.15.52:0/3312029873 >> 172.21.15.52:0/3312029873 conn(0x7facbc078460 msgr2=0x7facbc100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:58.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.252+0000 7facc0c34700 1 -- 172.21.15.52:0/3312029873 shutdown_connections 2024-04-02T13:51:58.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:51:58.252+0000 7facc0c34700 1 -- 172.21.15.52:0/3312029873 wait complete. 2024-04-02T13:51:58.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:58 smithi052 ceph-mon[32104]: pgmap v323: 129 pgs: 5 peering, 124 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-02T13:51:58.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:58 smithi189 ceph-mon[36683]: pgmap v323: 129 pgs: 5 peering, 124 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-02T13:51:58.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:58 smithi088 ceph-mon[36528]: pgmap v323: 129 pgs: 5 peering, 124 active+clean; 643 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-02T13:51:59.351 INFO:teuthology.orchestra.run.smithi052.stdout:313532612638 2024-04-02T13:51:59.351 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.10 2024-04-02T13:51:59.575 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:51:59 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:51:59.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:51:59 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:51:59.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:51:59 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:52:00.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.019+0000 7fdb0e542700 1 -- 172.21.15.52:0/3217426575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb08102fe0 msgr2=0x7fdb0810f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:00.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.019+0000 7fdb0e542700 1 --2- 172.21.15.52:0/3217426575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb08102fe0 0x7fdb0810f490 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000b0a0 tx=0x7fdb0000b3b0 comp rx=0 tx=0).stop 2024-04-02T13:52:00.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.019+0000 7fdb0e542700 1 -- 172.21.15.52:0/3217426575 shutdown_connections 2024-04-02T13:52:00.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.019+0000 7fdb0e542700 1 --2- 172.21.15.52:0/3217426575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb08102fe0 0x7fdb0810f490 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:00.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.019+0000 7fdb0e542700 1 --2- 172.21.15.52:0/3217426575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb08107520 0x7fdb08107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:00.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.019+0000 7fdb0e542700 1 --2- 172.21.15.52:0/3217426575 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb08102640 0x7fdb08102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:00.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.019+0000 7fdb0e542700 1 -- 172.21.15.52:0/3217426575 >> 172.21.15.52:0/3217426575 conn(0x7fdb08079720 msgr2=0x7fdb08079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:00.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.019+0000 7fdb0e542700 1 -- 172.21.15.52:0/3217426575 shutdown_connections 2024-04-02T13:52:00.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.019+0000 7fdb0e542700 1 -- 172.21.15.52:0/3217426575 wait complete. 2024-04-02T13:52:00.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.020+0000 7fdb0e542700 1 Processor -- start 2024-04-02T13:52:00.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.020+0000 7fdb0e542700 1 -- start start 2024-04-02T13:52:00.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.020+0000 7fdb0e542700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb08102640 0x7fdb080748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:00.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.020+0000 7fdb0e542700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb08102fe0 0x7fdb08074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:00.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.020+0000 7fdb0e542700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb08107520 0x7fdb08075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:00.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.020+0000 7fdb0e542700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdb08112930 con 0x7fdb08102640 2024-04-02T13:52:00.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.020+0000 7fdb0e542700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdb081127b0 con 0x7fdb08107520 2024-04-02T13:52:00.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.020+0000 7fdb0e542700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb08112ab0 con 0x7fdb08102fe0 2024-04-02T13:52:00.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.020+0000 7fdb07fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb08102640 0x7fdb080748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:00.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.020+0000 7fdb07fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb08102640 0x7fdb080748f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40780/0 (socket says 172.21.15.52:40780) 2024-04-02T13:52:00.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.020+0000 7fdb07fff700 1 -- 172.21.15.52:0/3742299620 learned_addr learned my addr 172.21.15.52:0/3742299620 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:00.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.020+0000 7fdb0cadf700 1 --2- 172.21.15.52:0/3742299620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb08107520 0x7fdb08075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:00.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.020+0000 7fdb077fe700 1 --2- 172.21.15.52:0/3742299620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb08102fe0 0x7fdb08074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:00.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.021+0000 7fdb07fff700 1 -- 172.21.15.52:0/3742299620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb08102fe0 msgr2=0x7fdb08074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:00.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.021+0000 7fdb07fff700 1 --2- 172.21.15.52:0/3742299620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb08102fe0 0x7fdb08074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:00.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.021+0000 7fdb07fff700 1 -- 172.21.15.52:0/3742299620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb08107520 msgr2=0x7fdb08075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:00.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.021+0000 7fdb07fff700 1 --2- 172.21.15.52:0/3742299620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb08107520 0x7fdb08075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:00.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.021+0000 7fdb07fff700 1 -- 172.21.15.52:0/3742299620 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb081a72b0 con 0x7fdb08102640 2024-04-02T13:52:00.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.021+0000 7fdb07fff700 1 --2- 172.21.15.52:0/3742299620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb08102640 0x7fdb080748f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800b500 tx=0x7fdaf800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:00.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.021+0000 7fdb057fa700 1 -- 172.21.15.52:0/3742299620 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8015020 con 0x7fdb08102640 2024-04-02T13:52:00.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.021+0000 7fdb0e542700 1 -- 172.21.15.52:0/3742299620 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb081a75a0 con 0x7fdb08102640 2024-04-02T13:52:00.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.021+0000 7fdb057fa700 1 -- 172.21.15.52:0/3742299620 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaf8011d10 con 0x7fdb08102640 2024-04-02T13:52:00.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.021+0000 7fdb057fa700 1 -- 172.21.15.52:0/3742299620 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8010db0 con 0x7fdb08102640 2024-04-02T13:52:00.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.021+0000 7fdb0e542700 1 -- 172.21.15.52:0/3742299620 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb081a7b30 con 0x7fdb08102640 2024-04-02T13:52:00.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.022+0000 7fdb0e542700 1 -- 172.21.15.52:0/3742299620 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdb08062d00 con 0x7fdb08102640 2024-04-02T13:52:00.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.023+0000 7fdb057fa700 1 -- 172.21.15.52:0/3742299620 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdaf8018020 con 0x7fdb08102640 2024-04-02T13:52:00.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.024+0000 7fdb057fa700 1 --2- 172.21.15.52:0/3742299620 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdaf00a3cd0 0x7fdaf00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:00.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.024+0000 7fdb077fe700 1 --2- 172.21.15.52:0/3742299620 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdaf00a3cd0 0x7fdaf00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:00.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.024+0000 7fdb057fa700 1 -- 172.21.15.52:0/3742299620 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdaf80d0530 con 0x7fdb08102640 2024-04-02T13:52:00.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.024+0000 7fdb057fa700 1 --2- 172.21.15.52:0/3742299620 >> [v2:172.21.15.52:6826/2350236811,v1:172.21.15.52:6827/2350236811] conn(0x7fdaf00aee30 0x7fdaf00b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:00.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.024+0000 7fdb057fa700 1 -- 172.21.15.52:0/3742299620 --> [v2:172.21.15.52:6826/2350236811,v1:172.21.15.52:6827/2350236811] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdaf8018ec0 con 0x7fdaf00aee30 2024-04-02T13:52:00.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.024+0000 7fdb057fa700 1 -- 172.21.15.52:0/3742299620 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fdaf80d08e0 con 0x7fdb08102640 2024-04-02T13:52:00.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.024+0000 7fdb0cadf700 1 --2- 172.21.15.52:0/3742299620 >> [v2:172.21.15.52:6826/2350236811,v1:172.21.15.52:6827/2350236811] conn(0x7fdaf00aee30 0x7fdaf00b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:00.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.024+0000 7fdb077fe700 1 --2- 172.21.15.52:0/3742299620 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdaf00a3cd0 0x7fdaf00a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdb08079d00 tx=0x7fdafc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:00.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.024+0000 7fdb0cadf700 1 --2- 172.21.15.52:0/3742299620 >> [v2:172.21.15.52:6826/2350236811,v1:172.21.15.52:6827/2350236811] conn(0x7fdaf00aee30 0x7fdaf00b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:00.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.025+0000 7fdb057fa700 1 -- 172.21.15.52:0/3742299620 <== osd.9 v2:172.21.15.52:6826/2350236811 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdaf8018ec0 con 0x7fdaf00aee30 2024-04-02T13:52:00.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.041+0000 7fdb0e542700 1 -- 172.21.15.52:0/3742299620 --> [v2:172.21.15.52:6826/2350236811,v1:172.21.15.52:6827/2350236811] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdb08063100 con 0x7fdaf00aee30 2024-04-02T13:52:00.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.042+0000 7fdb057fa700 1 -- 172.21.15.52:0/3742299620 <== osd.9 v2:172.21.15.52:6826/2350236811 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdb08063100 con 0x7fdaf00aee30 2024-04-02T13:52:00.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.042+0000 7fdb0e542700 1 -- 172.21.15.52:0/3742299620 >> [v2:172.21.15.52:6826/2350236811,v1:172.21.15.52:6827/2350236811] conn(0x7fdaf00aee30 msgr2=0x7fdaf00b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:00.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.042+0000 7fdb0e542700 1 --2- 172.21.15.52:0/3742299620 >> [v2:172.21.15.52:6826/2350236811,v1:172.21.15.52:6827/2350236811] conn(0x7fdaf00aee30 0x7fdaf00b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:00.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.042+0000 7fdb0e542700 1 -- 172.21.15.52:0/3742299620 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdaf00a3cd0 msgr2=0x7fdaf00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:00.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.042+0000 7fdb0e542700 1 --2- 172.21.15.52:0/3742299620 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdaf00a3cd0 0x7fdaf00a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdb08079d00 tx=0x7fdafc00b410 comp rx=0 tx=0).stop 2024-04-02T13:52:00.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.042+0000 7fdb0e542700 1 -- 172.21.15.52:0/3742299620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb08102640 msgr2=0x7fdb080748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:00.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.042+0000 7fdb0e542700 1 --2- 172.21.15.52:0/3742299620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb08102640 0x7fdb080748f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800b500 tx=0x7fdaf800b810 comp rx=0 tx=0).stop 2024-04-02T13:52:00.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.042+0000 7fdb0cadf700 1 -- 172.21.15.52:0/3742299620 reap_dead start 2024-04-02T13:52:00.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.042+0000 7fdb0e542700 1 -- 172.21.15.52:0/3742299620 shutdown_connections 2024-04-02T13:52:00.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.042+0000 7fdb0e542700 1 -- 172.21.15.52:0/3742299620 >> 172.21.15.52:0/3742299620 conn(0x7fdb08079720 msgr2=0x7fdb081006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:00.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.042+0000 7fdb0e542700 1 -- 172.21.15.52:0/3742299620 shutdown_connections 2024-04-02T13:52:00.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:00.042+0000 7fdb0e542700 1 -- 172.21.15.52:0/3742299620 wait complete. 2024-04-02T13:52:00.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:00 smithi189 ceph-mon[36683]: pgmap v324: 129 pgs: 5 peering, 124 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 383 B/s wr, 9 op/s 2024-04-02T13:52:00.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:00 smithi088 ceph-mon[36528]: pgmap v324: 129 pgs: 5 peering, 124 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 383 B/s wr, 9 op/s 2024-04-02T13:52:00.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:00 smithi052 ceph-mon[32104]: pgmap v324: 129 pgs: 5 peering, 124 active+clean; 644 KiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 383 B/s wr, 9 op/s 2024-04-02T13:52:01.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.625+0000 7f0f4d71d700 1 -- 172.21.15.52:0/4187391196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f48107520 msgr2=0x7f0f48107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:01.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.625+0000 7f0f4d71d700 1 --2- 172.21.15.52:0/4187391196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f48107520 0x7f0f48107900 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009990 tx=0x7f0f38009ca0 comp rx=0 tx=0).stop 2024-04-02T13:52:01.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.625+0000 7f0f4d71d700 1 -- 172.21.15.52:0/4187391196 shutdown_connections 2024-04-02T13:52:01.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.625+0000 7f0f4d71d700 1 --2- 172.21.15.52:0/4187391196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f48102fe0 0x7f0f4810f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:01.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.625+0000 7f0f4d71d700 1 --2- 172.21.15.52:0/4187391196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f48107520 0x7f0f48107900 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:01.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.625+0000 7f0f4d71d700 1 --2- 172.21.15.52:0/4187391196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f48102640 0x7f0f48102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:01.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.625+0000 7f0f4d71d700 1 -- 172.21.15.52:0/4187391196 >> 172.21.15.52:0/4187391196 conn(0x7f0f48079720 msgr2=0x7f0f48079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:01.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.625+0000 7f0f4d71d700 1 -- 172.21.15.52:0/4187391196 shutdown_connections 2024-04-02T13:52:01.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.625+0000 7f0f4d71d700 1 -- 172.21.15.52:0/4187391196 wait complete. 2024-04-02T13:52:01.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.626+0000 7f0f4d71d700 1 Processor -- start 2024-04-02T13:52:01.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.626+0000 7f0f4d71d700 1 -- start start 2024-04-02T13:52:01.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.627+0000 7f0f4d71d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f48102640 0x7f0f48198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:01.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.627+0000 7f0f4d71d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f48102fe0 0x7f0f481992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:01.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.627+0000 7f0f4d71d700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f48107520 0x7f0f4819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:01.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.627+0000 7f0f4d71d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481128b0 con 0x7f0f48102640 2024-04-02T13:52:01.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.627+0000 7f0f4d71d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0f48112730 con 0x7f0f48102fe0 2024-04-02T13:52:01.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.627+0000 7f0f4d71d700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0f48112a30 con 0x7f0f48107520 2024-04-02T13:52:01.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.627+0000 7f0f46ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f48102640 0x7f0f48198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:01.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.627+0000 7f0f46ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f48102640 0x7f0f48198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58572/0 (socket says 172.21.15.52:58572) 2024-04-02T13:52:01.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.627+0000 7f0f46ffd700 1 -- 172.21.15.52:0/3694832335 learned_addr learned my addr 172.21.15.52:0/3694832335 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:01.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.627+0000 7f0f477fe700 1 --2- 172.21.15.52:0/3694832335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f48107520 0x7f0f4819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:01.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.627+0000 7f0f467fc700 1 --2- 172.21.15.52:0/3694832335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f48102fe0 0x7f0f481992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:01.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.627+0000 7f0f477fe700 1 -- 172.21.15.52:0/3694832335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f48102fe0 msgr2=0x7f0f481992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:01.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.627+0000 7f0f477fe700 1 --2- 172.21.15.52:0/3694832335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f48102fe0 0x7f0f481992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:01.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.627+0000 7f0f477fe700 1 -- 172.21.15.52:0/3694832335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f48102640 msgr2=0x7f0f48198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:01.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.627+0000 7f0f477fe700 1 --2- 172.21.15.52:0/3694832335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f48102640 0x7f0f48198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:01.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.627+0000 7f0f477fe700 1 -- 172.21.15.52:0/3694832335 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4819ddb0 con 0x7f0f48107520 2024-04-02T13:52:01.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.627+0000 7f0f46ffd700 1 --2- 172.21.15.52:0/3694832335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f48102640 0x7f0f48198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:01.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.627+0000 7f0f477fe700 1 --2- 172.21.15.52:0/3694832335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f48107520 0x7f0f4819d670 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0f40007f10 tx=0x7f0f4000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:01.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.628+0000 7f0f2ffff700 1 -- 172.21.15.52:0/3694832335 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f40004db0 con 0x7f0f48107520 2024-04-02T13:52:01.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.628+0000 7f0f2ffff700 1 -- 172.21.15.52:0/3694832335 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f40009e20 con 0x7f0f48107520 2024-04-02T13:52:01.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.628+0000 7f0f4d71d700 1 -- 172.21.15.52:0/3694832335 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f4819e0a0 con 0x7f0f48107520 2024-04-02T13:52:01.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.628+0000 7f0f2ffff700 1 -- 172.21.15.52:0/3694832335 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f40015400 con 0x7f0f48107520 2024-04-02T13:52:01.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.628+0000 7f0f4d71d700 1 -- 172.21.15.52:0/3694832335 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f481a5920 con 0x7f0f48107520 2024-04-02T13:52:01.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.628+0000 7f0f4d71d700 1 -- 172.21.15.52:0/3694832335 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0f48062d80 con 0x7f0f48107520 2024-04-02T13:52:01.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.630+0000 7f0f2ffff700 1 -- 172.21.15.52:0/3694832335 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0f4000db20 con 0x7f0f48107520 2024-04-02T13:52:01.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.630+0000 7f0f2ffff700 1 --2- 172.21.15.52:0/3694832335 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f0f300a3cd0 0x7f0f300a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:01.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.630+0000 7f0f46ffd700 1 --2- 172.21.15.52:0/3694832335 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f0f300a3cd0 0x7f0f300a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:01.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.630+0000 7f0f2ffff700 1 -- 172.21.15.52:0/3694832335 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f0f400d0dd0 con 0x7f0f48107520 2024-04-02T13:52:01.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.630+0000 7f0f2ffff700 1 --2- 172.21.15.52:0/3694832335 >> [v2:172.21.15.88:6800/288986438,v1:172.21.15.88:6801/288986438] conn(0x7f0f300aee30 0x7f0f300b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:01.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.630+0000 7f0f2ffff700 1 -- 172.21.15.52:0/3694832335 --> [v2:172.21.15.88:6800/288986438,v1:172.21.15.88:6801/288986438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0f400053a0 con 0x7f0f300aee30 2024-04-02T13:52:01.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.630+0000 7f0f2ffff700 1 -- 172.21.15.52:0/3694832335 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f0f400d1180 con 0x7f0f48107520 2024-04-02T13:52:01.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.631+0000 7f0f467fc700 1 --2- 172.21.15.52:0/3694832335 >> [v2:172.21.15.88:6800/288986438,v1:172.21.15.88:6801/288986438] conn(0x7f0f300aee30 0x7f0f300b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:01.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.631+0000 7f0f46ffd700 1 --2- 172.21.15.52:0/3694832335 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f0f300a3cd0 0x7f0f300a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800b2e0 tx=0x7f0f38009590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:01.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.631+0000 7f0f467fc700 1 --2- 172.21.15.52:0/3694832335 >> [v2:172.21.15.88:6800/288986438,v1:172.21.15.88:6801/288986438] conn(0x7f0f300aee30 0x7f0f300b12d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:01.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.632+0000 7f0f2ffff700 1 -- 172.21.15.52:0/3694832335 <== osd.1 v2:172.21.15.88:6800/288986438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0f400053a0 con 0x7f0f300aee30 2024-04-02T13:52:01.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.650+0000 7f0f4d71d700 1 -- 172.21.15.52:0/3694832335 --> [v2:172.21.15.88:6800/288986438,v1:172.21.15.88:6801/288986438] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0f48063200 con 0x7f0f300aee30 2024-04-02T13:52:01.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.651+0000 7f0f2ffff700 1 -- 172.21.15.52:0/3694832335 <== osd.1 v2:172.21.15.88:6800/288986438 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0f48063200 con 0x7f0f300aee30 2024-04-02T13:52:01.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.651+0000 7f0f4d71d700 1 -- 172.21.15.52:0/3694832335 >> [v2:172.21.15.88:6800/288986438,v1:172.21.15.88:6801/288986438] conn(0x7f0f300aee30 msgr2=0x7f0f300b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:01.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.651+0000 7f0f4d71d700 1 --2- 172.21.15.52:0/3694832335 >> [v2:172.21.15.88:6800/288986438,v1:172.21.15.88:6801/288986438] conn(0x7f0f300aee30 0x7f0f300b12d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:01.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.651+0000 7f0f4d71d700 1 -- 172.21.15.52:0/3694832335 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f0f300a3cd0 msgr2=0x7f0f300a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:01.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.651+0000 7f0f4d71d700 1 --2- 172.21.15.52:0/3694832335 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f0f300a3cd0 0x7f0f300a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800b2e0 tx=0x7f0f38009590 comp rx=0 tx=0).stop 2024-04-02T13:52:01.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.651+0000 7f0f4d71d700 1 -- 172.21.15.52:0/3694832335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f48107520 msgr2=0x7f0f4819d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:01.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.651+0000 7f0f4d71d700 1 --2- 172.21.15.52:0/3694832335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f48107520 0x7f0f4819d670 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0f40007f10 tx=0x7f0f4000d3b0 comp rx=0 tx=0).stop 2024-04-02T13:52:01.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.651+0000 7f0f477fe700 1 -- 172.21.15.52:0/3694832335 reap_dead start 2024-04-02T13:52:01.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.651+0000 7f0f4d71d700 1 -- 172.21.15.52:0/3694832335 shutdown_connections 2024-04-02T13:52:01.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.651+0000 7f0f4d71d700 1 -- 172.21.15.52:0/3694832335 >> 172.21.15.52:0/3694832335 conn(0x7f0f48079720 msgr2=0x7f0f48100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:01.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.651+0000 7f0f4d71d700 1 -- 172.21.15.52:0/3694832335 shutdown_connections 2024-04-02T13:52:01.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.651+0000 7f0f4d71d700 1 -- 172.21.15.52:0/3694832335 wait complete. 2024-04-02T13:52:02.170 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:02 smithi052 ceph-mon[32104]: pgmap v325: 129 pgs: 5 peering, 124 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 383 B/s wr, 11 op/s 2024-04-02T13:52:02.170 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:02 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:52:02.170 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:02 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:52:02.170 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:02 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:52:02.170 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:02 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:52:02.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:02 smithi088 ceph-mon[36528]: pgmap v325: 129 pgs: 5 peering, 124 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 383 B/s wr, 11 op/s 2024-04-02T13:52:02.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:02 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:52:02.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:02 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:52:02.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:02 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:52:02.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:02 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:52:02.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.892+0000 7f479c0aa700 1 -- 172.21.15.52:0/728863566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4794107660 msgr2=0x7f4794107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:02.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.892+0000 7f479c0aa700 1 --2- 172.21.15.52:0/728863566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4794107660 0x7f4794107a40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f478c00b0d0 tx=0x7f478c00b3e0 comp rx=0 tx=0).stop 2024-04-02T13:52:02.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.892+0000 7f479c0aa700 1 -- 172.21.15.52:0/728863566 shutdown_connections 2024-04-02T13:52:02.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.892+0000 7f479c0aa700 1 --2- 172.21.15.52:0/728863566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f479406b6f0 0x7f479410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:02.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.892+0000 7f479c0aa700 1 --2- 172.21.15.52:0/728863566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4794107660 0x7f4794107a40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:02.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.892+0000 7f479c0aa700 1 --2- 172.21.15.52:0/728863566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f479406ad30 0x7f479406b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:02.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.892+0000 7f479c0aa700 1 -- 172.21.15.52:0/728863566 >> 172.21.15.52:0/728863566 conn(0x7f4794078350 msgr2=0x7f4794078760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:02.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.892+0000 7f479c0aa700 1 -- 172.21.15.52:0/728863566 shutdown_connections 2024-04-02T13:52:02.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.892+0000 7f479c0aa700 1 -- 172.21.15.52:0/728863566 wait complete. 2024-04-02T13:52:02.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.893+0000 7f479c0aa700 1 Processor -- start 2024-04-02T13:52:02.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.893+0000 7f479c0aa700 1 -- start start 2024-04-02T13:52:02.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.893+0000 7f479c0aa700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f479406ad30 0x7f47941018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:02.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.893+0000 7f479c0aa700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f479406b6f0 0x7f4794101de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:02.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.893+0000 7f479c0aa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4794107660 0x7f4794102320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:02.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.893+0000 7f479c0aa700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4794112970 con 0x7f4794107660 2024-04-02T13:52:02.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.893+0000 7f479c0aa700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f47941127f0 con 0x7f479406b6f0 2024-04-02T13:52:02.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.893+0000 7f479c0aa700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4794112af0 con 0x7f479406ad30 2024-04-02T13:52:02.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.893+0000 7f479a647700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4794107660 0x7f4794102320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:02.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.893+0000 7f479a647700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4794107660 0x7f4794102320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58578/0 (socket says 172.21.15.52:58578) 2024-04-02T13:52:02.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.893+0000 7f479a647700 1 -- 172.21.15.52:0/3867149207 learned_addr learned my addr 172.21.15.52:0/3867149207 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:02.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.893+0000 7f4799645700 1 --2- 172.21.15.52:0/3867149207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f479406b6f0 0x7f4794101de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:02.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.893+0000 7f4799e46700 1 --2- 172.21.15.52:0/3867149207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f479406ad30 0x7f47941018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:02.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.893+0000 7f479a647700 1 -- 172.21.15.52:0/3867149207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f479406ad30 msgr2=0x7f47941018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:02.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.893+0000 7f479a647700 1 --2- 172.21.15.52:0/3867149207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f479406ad30 0x7f47941018a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:02.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.893+0000 7f479a647700 1 -- 172.21.15.52:0/3867149207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f479406b6f0 msgr2=0x7f4794101de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:02.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.893+0000 7f479a647700 1 --2- 172.21.15.52:0/3867149207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f479406b6f0 0x7f4794101de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:02.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.893+0000 7f479a647700 1 -- 172.21.15.52:0/3867149207 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47941a7290 con 0x7f4794107660 2024-04-02T13:52:02.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.893+0000 7f4799e46700 1 --2- 172.21.15.52:0/3867149207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f479406ad30 0x7f47941018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:52:02.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.893+0000 7f479a647700 1 --2- 172.21.15.52:0/3867149207 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4794107660 0x7f4794102320 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f478400e970 tx=0x7f478400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:02.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.894+0000 7f4792ffd700 1 -- 172.21.15.52:0/3867149207 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4784004a30 con 0x7f4794107660 2024-04-02T13:52:02.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.894+0000 7f479c0aa700 1 -- 172.21.15.52:0/3867149207 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47941a7580 con 0x7f4794107660 2024-04-02T13:52:02.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.894+0000 7f4792ffd700 1 -- 172.21.15.52:0/3867149207 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4784005500 con 0x7f4794107660 2024-04-02T13:52:02.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.894+0000 7f4792ffd700 1 -- 172.21.15.52:0/3867149207 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478400cda0 con 0x7f4794107660 2024-04-02T13:52:02.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.894+0000 7f479c0aa700 1 -- 172.21.15.52:0/3867149207 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47941a7ac0 con 0x7f4794107660 2024-04-02T13:52:02.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.894+0000 7f479c0aa700 1 -- 172.21.15.52:0/3867149207 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4794062d00 con 0x7f4794107660 2024-04-02T13:52:02.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.896+0000 7f4792ffd700 1 -- 172.21.15.52:0/3867149207 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4784005000 con 0x7f4794107660 2024-04-02T13:52:02.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.896+0000 7f4792ffd700 1 --2- 172.21.15.52:0/3867149207 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f477c0a3cd0 0x7f477c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:02.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.896+0000 7f4799e46700 1 --2- 172.21.15.52:0/3867149207 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f477c0a3cd0 0x7f477c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:02.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.896+0000 7f4792ffd700 1 -- 172.21.15.52:0/3867149207 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f47840d11e0 con 0x7f4794107660 2024-04-02T13:52:02.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.897+0000 7f4792ffd700 1 --2- 172.21.15.52:0/3867149207 >> [v2:172.21.15.88:6808/2077759625,v1:172.21.15.88:6809/2077759625] conn(0x7f477c0aedb0 0x7f477c0b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:02.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.897+0000 7f4792ffd700 1 -- 172.21.15.52:0/3867149207 --> [v2:172.21.15.88:6808/2077759625,v1:172.21.15.88:6809/2077759625] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f478401ee50 con 0x7f477c0aedb0 2024-04-02T13:52:02.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.897+0000 7f4792ffd700 1 -- 172.21.15.52:0/3867149207 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f47840d1630 con 0x7f4794107660 2024-04-02T13:52:02.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.897+0000 7f4799e46700 1 --2- 172.21.15.52:0/3867149207 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f477c0a3cd0 0x7f477c0a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f478c009e60 tx=0x7f478c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:02.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.897+0000 7f4799645700 1 --2- 172.21.15.52:0/3867149207 >> [v2:172.21.15.88:6808/2077759625,v1:172.21.15.88:6809/2077759625] conn(0x7f477c0aedb0 0x7f477c0b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:02.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.897+0000 7f4799645700 1 --2- 172.21.15.52:0/3867149207 >> [v2:172.21.15.88:6808/2077759625,v1:172.21.15.88:6809/2077759625] conn(0x7f477c0aedb0 0x7f477c0b1250 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:02.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.898+0000 7f4792ffd700 1 -- 172.21.15.52:0/3867149207 <== osd.4 v2:172.21.15.88:6808/2077759625 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f478401ee50 con 0x7f477c0aedb0 2024-04-02T13:52:02.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.914+0000 7f479c0aa700 1 -- 172.21.15.52:0/3867149207 --> [v2:172.21.15.88:6808/2077759625,v1:172.21.15.88:6809/2077759625] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4794063180 con 0x7f477c0aedb0 2024-04-02T13:52:02.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.915+0000 7f4792ffd700 1 -- 172.21.15.52:0/3867149207 <== osd.4 v2:172.21.15.88:6808/2077759625 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4794063180 con 0x7f477c0aedb0 2024-04-02T13:52:02.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.915+0000 7f479c0aa700 1 -- 172.21.15.52:0/3867149207 >> [v2:172.21.15.88:6808/2077759625,v1:172.21.15.88:6809/2077759625] conn(0x7f477c0aedb0 msgr2=0x7f477c0b1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:02.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.915+0000 7f479c0aa700 1 --2- 172.21.15.52:0/3867149207 >> [v2:172.21.15.88:6808/2077759625,v1:172.21.15.88:6809/2077759625] conn(0x7f477c0aedb0 0x7f477c0b1250 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:02.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.915+0000 7f479c0aa700 1 -- 172.21.15.52:0/3867149207 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f477c0a3cd0 msgr2=0x7f477c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:02.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.915+0000 7f479c0aa700 1 --2- 172.21.15.52:0/3867149207 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f477c0a3cd0 0x7f477c0a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f478c009e60 tx=0x7f478c00b040 comp rx=0 tx=0).stop 2024-04-02T13:52:02.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.915+0000 7f479c0aa700 1 -- 172.21.15.52:0/3867149207 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4794107660 msgr2=0x7f4794102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:02.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.915+0000 7f479c0aa700 1 --2- 172.21.15.52:0/3867149207 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4794107660 0x7f4794102320 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f478400e970 tx=0x7f478400ec80 comp rx=0 tx=0).stop 2024-04-02T13:52:02.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.915+0000 7f479a647700 1 -- 172.21.15.52:0/3867149207 reap_dead start 2024-04-02T13:52:02.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.916+0000 7f479c0aa700 1 -- 172.21.15.52:0/3867149207 shutdown_connections 2024-04-02T13:52:02.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.916+0000 7f479c0aa700 1 -- 172.21.15.52:0/3867149207 >> 172.21.15.52:0/3867149207 conn(0x7f4794078350 msgr2=0x7f47941003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:02.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.916+0000 7f479c0aa700 1 -- 172.21.15.52:0/3867149207 shutdown_connections 2024-04-02T13:52:02.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:01.916+0000 7f479c0aa700 1 -- 172.21.15.52:0/3867149207 wait complete. 2024-04-02T13:52:02.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:02 smithi189 ceph-mon[36683]: pgmap v325: 129 pgs: 5 peering, 124 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 383 B/s wr, 11 op/s 2024-04-02T13:52:02.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:02 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:52:02.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:02 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:52:02.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:02 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:52:02.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:02 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:52:03.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.383+0000 7f83f7fd8700 1 -- 172.21.15.52:0/2919289995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f006ab30 msgr2=0x7f83f006afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:03.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.383+0000 7f83f7fd8700 1 --2- 172.21.15.52:0/2919289995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f006ab30 0x7f83f006afb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f83ec009990 tx=0x7f83ec009ca0 comp rx=0 tx=0).stop 2024-04-02T13:52:03.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.383+0000 7f83f7fd8700 1 -- 172.21.15.52:0/2919289995 shutdown_connections 2024-04-02T13:52:03.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.383+0000 7f83f7fd8700 1 --2- 172.21.15.52:0/2919289995 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83f006b4f0 0x7f83f010f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:03.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.383+0000 7f83f7fd8700 1 --2- 172.21.15.52:0/2919289995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83f0108630 0x7f83f0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:03.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.383+0000 7f83f7fd8700 1 --2- 172.21.15.52:0/2919289995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f006ab30 0x7f83f006afb0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:03.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.383+0000 7f83f7fd8700 1 -- 172.21.15.52:0/2919289995 >> 172.21.15.52:0/2919289995 conn(0x7f83f0077540 msgr2=0x7f83f0077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:03.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.383+0000 7f83f7fd8700 1 -- 172.21.15.52:0/2919289995 shutdown_connections 2024-04-02T13:52:03.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.383+0000 7f83f7fd8700 1 -- 172.21.15.52:0/2919289995 wait complete. 2024-04-02T13:52:03.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.383+0000 7f83f7fd8700 1 Processor -- start 2024-04-02T13:52:03.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.383+0000 7f83f7fd8700 1 -- start start 2024-04-02T13:52:03.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.384+0000 7f83f7fd8700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83f006ab30 0x7f83f01015f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:03.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.384+0000 7f83f7fd8700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83f006b4f0 0x7f83f0101b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:03.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.384+0000 7f83f7fd8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f0108630 0x7f83f0102070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:03.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.384+0000 7f83f7fd8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f83f0112930 con 0x7f83f0108630 2024-04-02T13:52:03.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.384+0000 7f83f7fd8700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f83f01127b0 con 0x7f83f006ab30 2024-04-02T13:52:03.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.384+0000 7f83f7fd8700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f83f0112ab0 con 0x7f83f006b4f0 2024-04-02T13:52:03.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.384+0000 7f83f6575700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f0108630 0x7f83f0102070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:03.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.384+0000 7f83f6575700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f0108630 0x7f83f0102070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58604/0 (socket says 172.21.15.52:58604) 2024-04-02T13:52:03.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.384+0000 7f83f6575700 1 -- 172.21.15.52:0/4216529785 learned_addr learned my addr 172.21.15.52:0/4216529785 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:03.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.384+0000 7f83f5d74700 1 --2- 172.21.15.52:0/4216529785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83f006ab30 0x7f83f01015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:03.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.384+0000 7f83f5573700 1 --2- 172.21.15.52:0/4216529785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83f006b4f0 0x7f83f0101b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:03.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.384+0000 7f83f6575700 1 -- 172.21.15.52:0/4216529785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83f006b4f0 msgr2=0x7f83f0101b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:03.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.384+0000 7f83f6575700 1 --2- 172.21.15.52:0/4216529785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83f006b4f0 0x7f83f0101b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:03.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.384+0000 7f83f6575700 1 -- 172.21.15.52:0/4216529785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83f006ab30 msgr2=0x7f83f01015f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:03.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.384+0000 7f83f6575700 1 --2- 172.21.15.52:0/4216529785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83f006ab30 0x7f83f01015f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:03.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.384+0000 7f83f6575700 1 -- 172.21.15.52:0/4216529785 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f01a72b0 con 0x7f83f0108630 2024-04-02T13:52:03.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.384+0000 7f83f6575700 1 --2- 172.21.15.52:0/4216529785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f0108630 0x7f83f0102070 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f83e800ada0 tx=0x7f83e800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:03.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.385+0000 7f83e6ffd700 1 -- 172.21.15.52:0/4216529785 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8017020 con 0x7f83f0108630 2024-04-02T13:52:03.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.385+0000 7f83e6ffd700 1 -- 172.21.15.52:0/4216529785 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e800cdb0 con 0x7f83f0108630 2024-04-02T13:52:03.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.385+0000 7f83f7fd8700 1 -- 172.21.15.52:0/4216529785 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f01a75a0 con 0x7f83f0108630 2024-04-02T13:52:03.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.385+0000 7f83e6ffd700 1 -- 172.21.15.52:0/4216529785 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e80053e0 con 0x7f83f0108630 2024-04-02T13:52:03.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.385+0000 7f83f7fd8700 1 -- 172.21.15.52:0/4216529785 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f01a7ab0 con 0x7f83f0108630 2024-04-02T13:52:03.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.385+0000 7f83f7fd8700 1 -- 172.21.15.52:0/4216529785 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f83f01009f0 con 0x7f83f0108630 2024-04-02T13:52:03.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.387+0000 7f83e6ffd700 1 -- 172.21.15.52:0/4216529785 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f83e800c8d0 con 0x7f83f0108630 2024-04-02T13:52:03.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.387+0000 7f83e6ffd700 1 --2- 172.21.15.52:0/4216529785 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f83dc0a3d50 0x7f83dc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:03.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.387+0000 7f83f5d74700 1 --2- 172.21.15.52:0/4216529785 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f83dc0a3d50 0x7f83dc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:03.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.388+0000 7f83e6ffd700 1 -- 172.21.15.52:0/4216529785 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f83e80d0990 con 0x7f83f0108630 2024-04-02T13:52:03.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.388+0000 7f83e6ffd700 1 --2- 172.21.15.52:0/4216529785 >> [v2:172.21.15.189:6800/2416423625,v1:172.21.15.189:6801/2416423625] conn(0x7f83dc0aeeb0 0x7f83dc0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:03.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.388+0000 7f83e6ffd700 1 -- 172.21.15.52:0/4216529785 --> [v2:172.21.15.189:6800/2416423625,v1:172.21.15.189:6801/2416423625] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f83e800fe50 con 0x7f83dc0aeeb0 2024-04-02T13:52:03.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.388+0000 7f83e6ffd700 1 -- 172.21.15.52:0/4216529785 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f83e80d0de0 con 0x7f83f0108630 2024-04-02T13:52:03.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.388+0000 7f83f5d74700 1 --2- 172.21.15.52:0/4216529785 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f83dc0a3d50 0x7f83dc0a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f83f0103a50 tx=0x7f83e0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:03.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.388+0000 7f83f5573700 1 --2- 172.21.15.52:0/4216529785 >> [v2:172.21.15.189:6800/2416423625,v1:172.21.15.189:6801/2416423625] conn(0x7f83dc0aeeb0 0x7f83dc0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:03.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.388+0000 7f83f5573700 1 --2- 172.21.15.52:0/4216529785 >> [v2:172.21.15.189:6800/2416423625,v1:172.21.15.189:6801/2416423625] conn(0x7f83dc0aeeb0 0x7f83dc0b1350 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:03.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.389+0000 7f83e6ffd700 1 -- 172.21.15.52:0/4216529785 <== osd.2 v2:172.21.15.189:6800/2416423625 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f83e800fe50 con 0x7f83dc0aeeb0 2024-04-02T13:52:03.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.406+0000 7f83f7fd8700 1 -- 172.21.15.52:0/4216529785 --> [v2:172.21.15.189:6800/2416423625,v1:172.21.15.189:6801/2416423625] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f83f0103d00 con 0x7f83dc0aeeb0 2024-04-02T13:52:03.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.406+0000 7f83e6ffd700 1 -- 172.21.15.52:0/4216529785 <== osd.2 v2:172.21.15.189:6800/2416423625 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f83f0103d00 con 0x7f83dc0aeeb0 2024-04-02T13:52:03.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.407+0000 7f83f7fd8700 1 -- 172.21.15.52:0/4216529785 >> [v2:172.21.15.189:6800/2416423625,v1:172.21.15.189:6801/2416423625] conn(0x7f83dc0aeeb0 msgr2=0x7f83dc0b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:03.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.407+0000 7f83f7fd8700 1 --2- 172.21.15.52:0/4216529785 >> [v2:172.21.15.189:6800/2416423625,v1:172.21.15.189:6801/2416423625] conn(0x7f83dc0aeeb0 0x7f83dc0b1350 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:03.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.407+0000 7f83f7fd8700 1 -- 172.21.15.52:0/4216529785 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f83dc0a3d50 msgr2=0x7f83dc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:03.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.407+0000 7f83f7fd8700 1 --2- 172.21.15.52:0/4216529785 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f83dc0a3d50 0x7f83dc0a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f83f0103a50 tx=0x7f83e0009190 comp rx=0 tx=0).stop 2024-04-02T13:52:03.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.407+0000 7f83f7fd8700 1 -- 172.21.15.52:0/4216529785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f0108630 msgr2=0x7f83f0102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:03.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.407+0000 7f83f7fd8700 1 --2- 172.21.15.52:0/4216529785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f0108630 0x7f83f0102070 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f83e800ada0 tx=0x7f83e800c330 comp rx=0 tx=0).stop 2024-04-02T13:52:03.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.407+0000 7f83f6575700 1 -- 172.21.15.52:0/4216529785 reap_dead start 2024-04-02T13:52:03.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.407+0000 7f83f7fd8700 1 -- 172.21.15.52:0/4216529785 shutdown_connections 2024-04-02T13:52:03.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.407+0000 7f83f7fd8700 1 -- 172.21.15.52:0/4216529785 >> 172.21.15.52:0/4216529785 conn(0x7f83f0077540 msgr2=0x7f83f00fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:03.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.407+0000 7f83f7fd8700 1 -- 172.21.15.52:0/4216529785 shutdown_connections 2024-04-02T13:52:03.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:03.407+0000 7f83f7fd8700 1 -- 172.21.15.52:0/4216529785 wait complete. 2024-04-02T13:52:03.718 INFO:teuthology.orchestra.run.smithi052.stdout:141733920830 2024-04-02T13:52:03.718 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.4 2024-04-02T13:52:04.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:04 smithi052 ceph-mon[32104]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 383 B/s wr, 12 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-04-02T13:52:04.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:04 smithi088 ceph-mon[36528]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 383 B/s wr, 12 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-04-02T13:52:04.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:04 smithi189 ceph-mon[36683]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 383 B/s wr, 12 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-04-02T13:52:04.768 INFO:teuthology.orchestra.run.smithi052.stdout:292057776164 2024-04-02T13:52:04.769 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.9 2024-04-02T13:52:05.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.728+0000 7f3676616700 1 -- 172.21.15.52:0/309829117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3670103900 msgr2=0x7f3670103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:05.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.728+0000 7f3676616700 1 --2- 172.21.15.52:0/309829117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3670103900 0x7f3670103ce0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3660009990 tx=0x7f3660009ca0 comp rx=0 tx=0).stop 2024-04-02T13:52:05.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.728+0000 7f3676616700 1 -- 172.21.15.52:0/309829117 shutdown_connections 2024-04-02T13:52:05.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.728+0000 7f3676616700 1 --2- 172.21.15.52:0/309829117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f367010cce0 0x7f367010f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:05.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.728+0000 7f3676616700 1 --2- 172.21.15.52:0/309829117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3670103900 0x7f3670103ce0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:05.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.728+0000 7f3676616700 1 --2- 172.21.15.52:0/309829117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3670104220 0x7f367010c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:05.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.728+0000 7f3676616700 1 -- 172.21.15.52:0/309829117 >> 172.21.15.52:0/309829117 conn(0x7f3670078620 msgr2=0x7f3670078a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:05.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.728+0000 7f3676616700 1 -- 172.21.15.52:0/309829117 shutdown_connections 2024-04-02T13:52:05.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.728+0000 7f3676616700 1 -- 172.21.15.52:0/309829117 wait complete. 2024-04-02T13:52:05.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.728+0000 7f3676616700 1 Processor -- start 2024-04-02T13:52:05.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.728+0000 7f3676616700 1 -- start start 2024-04-02T13:52:05.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.729+0000 7f3676616700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3670103900 0x7f3670198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:05.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.729+0000 7f3676616700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3670104220 0x7f3670199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:05.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.729+0000 7f3676616700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f367010cce0 0x7f367019d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:05.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.729+0000 7f3676616700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3670112a40 con 0x7f367010cce0 2024-04-02T13:52:05.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.729+0000 7f3676616700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f36701128c0 con 0x7f3670103900 2024-04-02T13:52:05.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.729+0000 7f3676616700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3670112bc0 con 0x7f3670104220 2024-04-02T13:52:05.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.729+0000 7f3674bb3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f367010cce0 0x7f367019d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:05.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.729+0000 7f3674bb3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f367010cce0 0x7f367019d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58620/0 (socket says 172.21.15.52:58620) 2024-04-02T13:52:05.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.729+0000 7f3674bb3700 1 -- 172.21.15.52:0/424585732 learned_addr learned my addr 172.21.15.52:0/424585732 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:05.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.729+0000 7f366f7fe700 1 --2- 172.21.15.52:0/424585732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3670104220 0x7f3670199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:05.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.729+0000 7f366ffff700 1 --2- 172.21.15.52:0/424585732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3670103900 0x7f3670198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:05.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.729+0000 7f3674bb3700 1 -- 172.21.15.52:0/424585732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3670104220 msgr2=0x7f3670199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:05.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.729+0000 7f3674bb3700 1 --2- 172.21.15.52:0/424585732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3670104220 0x7f3670199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:05.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.729+0000 7f3674bb3700 1 -- 172.21.15.52:0/424585732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3670103900 msgr2=0x7f3670198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:05.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.729+0000 7f3674bb3700 1 --2- 172.21.15.52:0/424585732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3670103900 0x7f3670198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:05.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.729+0000 7f3674bb3700 1 -- 172.21.15.52:0/424585732 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367019de90 con 0x7f367010cce0 2024-04-02T13:52:05.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.729+0000 7f3674bb3700 1 --2- 172.21.15.52:0/424585732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f367010cce0 0x7f367019d750 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f366800b9e0 tx=0x7f366800bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:05.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.730+0000 7f366d7fa700 1 -- 172.21.15.52:0/424585732 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3668019070 con 0x7f367010cce0 2024-04-02T13:52:05.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.730+0000 7f3676616700 1 -- 172.21.15.52:0/424585732 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f367019e180 con 0x7f367010cce0 2024-04-02T13:52:05.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.730+0000 7f366d7fa700 1 -- 172.21.15.52:0/424585732 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f366800ce10 con 0x7f367010cce0 2024-04-02T13:52:05.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.730+0000 7f366d7fa700 1 -- 172.21.15.52:0/424585732 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3668012360 con 0x7f367010cce0 2024-04-02T13:52:05.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.730+0000 7f3676616700 1 -- 172.21.15.52:0/424585732 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36701a5a00 con 0x7f367010cce0 2024-04-02T13:52:05.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.731+0000 7f3676616700 1 -- 172.21.15.52:0/424585732 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f36700fef90 con 0x7f367010cce0 2024-04-02T13:52:05.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.732+0000 7f366d7fa700 1 -- 172.21.15.52:0/424585732 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f366801c070 con 0x7f367010cce0 2024-04-02T13:52:05.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.733+0000 7f366d7fa700 1 --2- 172.21.15.52:0/424585732 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f36580a3d50 0x7f36580a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:05.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.733+0000 7f366ffff700 1 --2- 172.21.15.52:0/424585732 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f36580a3d50 0x7f36580a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:05.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.733+0000 7f366d7fa700 1 -- 172.21.15.52:0/424585732 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f36680cff60 con 0x7f367010cce0 2024-04-02T13:52:05.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.733+0000 7f366d7fa700 1 --2- 172.21.15.52:0/424585732 >> [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980] conn(0x7f36580aeeb0 0x7f36580b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:05.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.733+0000 7f366d7fa700 1 -- 172.21.15.52:0/424585732 --> [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f36580b1a20 con 0x7f36580aeeb0 2024-04-02T13:52:05.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.733+0000 7f366d7fa700 1 -- 172.21.15.52:0/424585732 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f36680d03b0 con 0x7f367010cce0 2024-04-02T13:52:05.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.734+0000 7f366f7fe700 1 --2- 172.21.15.52:0/424585732 >> [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980] conn(0x7f36580aeeb0 0x7f36580b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:05.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.734+0000 7f366ffff700 1 --2- 172.21.15.52:0/424585732 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f36580a3d50 0x7f36580a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3660006010 tx=0x7f366000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:05.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.734+0000 7f366f7fe700 1 --2- 172.21.15.52:0/424585732 >> [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980] conn(0x7f36580aeeb0 0x7f36580b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:05.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.735+0000 7f366d7fa700 1 -- 172.21.15.52:0/424585732 <== osd.3 v2:172.21.15.52:6810/1799783980 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f36580b1a20 con 0x7f36580aeeb0 2024-04-02T13:52:05.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.766+0000 7f3676616700 1 -- 172.21.15.52:0/424585732 --> [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f367006b530 con 0x7f36580aeeb0 2024-04-02T13:52:05.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.766+0000 7f366d7fa700 1 -- 172.21.15.52:0/424585732 <== osd.3 v2:172.21.15.52:6810/1799783980 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f367006b530 con 0x7f36580aeeb0 2024-04-02T13:52:05.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.766+0000 7f3676616700 1 -- 172.21.15.52:0/424585732 >> [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980] conn(0x7f36580aeeb0 msgr2=0x7f36580b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:05.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.766+0000 7f3676616700 1 --2- 172.21.15.52:0/424585732 >> [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980] conn(0x7f36580aeeb0 0x7f36580b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:05.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.766+0000 7f3676616700 1 -- 172.21.15.52:0/424585732 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f36580a3d50 msgr2=0x7f36580a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:05.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.766+0000 7f3676616700 1 --2- 172.21.15.52:0/424585732 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f36580a3d50 0x7f36580a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3660006010 tx=0x7f366000b540 comp rx=0 tx=0).stop 2024-04-02T13:52:05.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.766+0000 7f3676616700 1 -- 172.21.15.52:0/424585732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f367010cce0 msgr2=0x7f367019d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:05.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.766+0000 7f3676616700 1 --2- 172.21.15.52:0/424585732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f367010cce0 0x7f367019d750 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f366800b9e0 tx=0x7f366800bcf0 comp rx=0 tx=0).stop 2024-04-02T13:52:05.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.766+0000 7f3674bb3700 1 -- 172.21.15.52:0/424585732 reap_dead start 2024-04-02T13:52:05.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.767+0000 7f3676616700 1 -- 172.21.15.52:0/424585732 shutdown_connections 2024-04-02T13:52:05.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.767+0000 7f3676616700 1 -- 172.21.15.52:0/424585732 >> 172.21.15.52:0/424585732 conn(0x7f3670078620 msgr2=0x7f36700fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:05.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.767+0000 7f3676616700 1 -- 172.21.15.52:0/424585732 shutdown_connections 2024-04-02T13:52:05.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:04.767+0000 7f3676616700 1 -- 172.21.15.52:0/424585732 wait complete. 2024-04-02T13:52:05.836 INFO:teuthology.orchestra.run.smithi052.stdout:107374182467 2024-04-02T13:52:05.836 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.3 2024-04-02T13:52:06.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:06 smithi088 ceph-mon[36528]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 331 B/s wr, 8 op/s; 7 B/s, 1 keys/s, 0 objects/s recovering 2024-04-02T13:52:06.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:06 smithi189 ceph-mon[36683]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 331 B/s wr, 8 op/s; 7 B/s, 1 keys/s, 0 objects/s recovering 2024-04-02T13:52:06.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:06 smithi052 ceph-mon[32104]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 331 B/s wr, 8 op/s; 7 B/s, 1 keys/s, 0 objects/s recovering 2024-04-02T13:52:07.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.111+0000 7f4f2af38700 1 -- 172.21.15.52:0/106019603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f24102fe0 msgr2=0x7f4f2410f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:07.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.111+0000 7f4f2af38700 1 --2- 172.21.15.52:0/106019603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f24102fe0 0x7f4f2410f490 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00b0a0 tx=0x7f4f1c00b3b0 comp rx=0 tx=0).stop 2024-04-02T13:52:07.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.111+0000 7f4f2af38700 1 -- 172.21.15.52:0/106019603 shutdown_connections 2024-04-02T13:52:07.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.111+0000 7f4f2af38700 1 --2- 172.21.15.52:0/106019603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f24102fe0 0x7f4f2410f490 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:07.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.111+0000 7f4f2af38700 1 --2- 172.21.15.52:0/106019603 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f24107520 0x7f4f24107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:07.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.111+0000 7f4f2af38700 1 --2- 172.21.15.52:0/106019603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f24102640 0x7f4f24102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:07.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.111+0000 7f4f2af38700 1 -- 172.21.15.52:0/106019603 >> 172.21.15.52:0/106019603 conn(0x7f4f24079720 msgr2=0x7f4f24079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.111+0000 7f4f2af38700 1 -- 172.21.15.52:0/106019603 shutdown_connections 2024-04-02T13:52:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.111+0000 7f4f2af38700 1 -- 172.21.15.52:0/106019603 wait complete. 2024-04-02T13:52:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.111+0000 7f4f2af38700 1 Processor -- start 2024-04-02T13:52:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.112+0000 7f4f2af38700 1 -- start start 2024-04-02T13:52:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.112+0000 7f4f2af38700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f24102640 0x7f4f2419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.112+0000 7f4f2af38700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f24102fe0 0x7f4f2419b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.112+0000 7f4f2af38700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f24107520 0x7f4f2419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.112+0000 7f4f2af38700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4f241128b0 con 0x7f4f24102fe0 2024-04-02T13:52:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.112+0000 7f4f2af38700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4f24112730 con 0x7f4f24107520 2024-04-02T13:52:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.112+0000 7f4f2af38700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4f24112a30 con 0x7f4f24102640 2024-04-02T13:52:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.112+0000 7f4f23fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f24102fe0 0x7f4f2419b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.112+0000 7f4f28cd4700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f24102640 0x7f4f2419af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.112+0000 7f4f23fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f24102fe0 0x7f4f2419b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58652/0 (socket says 172.21.15.52:58652) 2024-04-02T13:52:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.112+0000 7f4f23fff700 1 -- 172.21.15.52:0/1562833999 learned_addr learned my addr 172.21.15.52:0/1562833999 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:07.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.112+0000 7f4f23fff700 1 -- 172.21.15.52:0/1562833999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f24102640 msgr2=0x7f4f2419af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:07.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.112+0000 7f4f23fff700 1 --2- 172.21.15.52:0/1562833999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f24102640 0x7f4f2419af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:07.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.112+0000 7f4f294d5700 1 --2- 172.21.15.52:0/1562833999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f24107520 0x7f4f2419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:07.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.112+0000 7f4f23fff700 1 -- 172.21.15.52:0/1562833999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f24107520 msgr2=0x7f4f2419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:07.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.112+0000 7f4f23fff700 1 --2- 172.21.15.52:0/1562833999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f24107520 0x7f4f2419f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:07.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.112+0000 7f4f23fff700 1 -- 172.21.15.52:0/1562833999 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f2419ff50 con 0x7f4f24102fe0 2024-04-02T13:52:07.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.112+0000 7f4f294d5700 1 --2- 172.21.15.52:0/1562833999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f24107520 0x7f4f2419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:52:07.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.112+0000 7f4f28cd4700 1 --2- 172.21.15.52:0/1562833999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f24102640 0x7f4f2419af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:07.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.113+0000 7f4f23fff700 1 --2- 172.21.15.52:0/1562833999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f24102fe0 0x7f4f2419b480 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000e9c0 tx=0x7f4f1000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:07.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.116+0000 7f4f21ffb700 1 -- 172.21.15.52:0/1562833999 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1000cbd0 con 0x7f4f24102fe0 2024-04-02T13:52:07.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.116+0000 7f4f21ffb700 1 -- 172.21.15.52:0/1562833999 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f1000cd70 con 0x7f4f24102fe0 2024-04-02T13:52:07.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.117+0000 7f4f21ffb700 1 -- 172.21.15.52:0/1562833999 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10010430 con 0x7f4f24102fe0 2024-04-02T13:52:07.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.117+0000 7f4f2af38700 1 -- 172.21.15.52:0/1562833999 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f241a0240 con 0x7f4f24102fe0 2024-04-02T13:52:07.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.117+0000 7f4f2af38700 1 -- 172.21.15.52:0/1562833999 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f241a7ac0 con 0x7f4f24102fe0 2024-04-02T13:52:07.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.118+0000 7f4f2af38700 1 -- 172.21.15.52:0/1562833999 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4f24100e90 con 0x7f4f24102fe0 2024-04-02T13:52:07.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.120+0000 7f4f21ffb700 1 -- 172.21.15.52:0/1562833999 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4f10004710 con 0x7f4f24102fe0 2024-04-02T13:52:07.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.120+0000 7f4f21ffb700 1 --2- 172.21.15.52:0/1562833999 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f4f140a3d00 0x7f4f140a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:07.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.120+0000 7f4f28cd4700 1 --2- 172.21.15.52:0/1562833999 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f4f140a3d00 0x7f4f140a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:07.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.120+0000 7f4f21ffb700 1 -- 172.21.15.52:0/1562833999 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f4f100d10f0 con 0x7f4f24102fe0 2024-04-02T13:52:07.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.120+0000 7f4f21ffb700 1 --2- 172.21.15.52:0/1562833999 >> [v2:172.21.15.189:6808/3106153812,v1:172.21.15.189:6809/3106153812] conn(0x7f4f140aee60 0x7f4f140b1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:07.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.121+0000 7f4f21ffb700 1 -- 172.21.15.52:0/1562833999 --> [v2:172.21.15.189:6808/3106153812,v1:172.21.15.189:6809/3106153812] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4f1001aec0 con 0x7f4f140aee60 2024-04-02T13:52:07.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.121+0000 7f4f21ffb700 1 -- 172.21.15.52:0/1562833999 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f4f100d14a0 con 0x7f4f24102fe0 2024-04-02T13:52:07.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.121+0000 7f4f294d5700 1 --2- 172.21.15.52:0/1562833999 >> [v2:172.21.15.189:6808/3106153812,v1:172.21.15.189:6809/3106153812] conn(0x7f4f140aee60 0x7f4f140b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:07.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.121+0000 7f4f28cd4700 1 --2- 172.21.15.52:0/1562833999 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f4f140a3d00 0x7f4f140a61c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4f241036d0 tx=0x7f4f1800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:07.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.121+0000 7f4f294d5700 1 --2- 172.21.15.52:0/1562833999 >> [v2:172.21.15.189:6808/3106153812,v1:172.21.15.189:6809/3106153812] conn(0x7f4f140aee60 0x7f4f140b1300 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:07.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.122+0000 7f4f21ffb700 1 -- 172.21.15.52:0/1562833999 <== osd.5 v2:172.21.15.189:6808/3106153812 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4f1001aec0 con 0x7f4f140aee60 2024-04-02T13:52:07.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.145+0000 7f4f2af38700 1 -- 172.21.15.52:0/1562833999 --> [v2:172.21.15.189:6808/3106153812,v1:172.21.15.189:6809/3106153812] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4f24103cd0 con 0x7f4f140aee60 2024-04-02T13:52:07.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.145+0000 7f4f21ffb700 1 -- 172.21.15.52:0/1562833999 <== osd.5 v2:172.21.15.189:6808/3106153812 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4f24103cd0 con 0x7f4f140aee60 2024-04-02T13:52:07.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.146+0000 7f4f2af38700 1 -- 172.21.15.52:0/1562833999 >> [v2:172.21.15.189:6808/3106153812,v1:172.21.15.189:6809/3106153812] conn(0x7f4f140aee60 msgr2=0x7f4f140b1300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:07.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.146+0000 7f4f2af38700 1 --2- 172.21.15.52:0/1562833999 >> [v2:172.21.15.189:6808/3106153812,v1:172.21.15.189:6809/3106153812] conn(0x7f4f140aee60 0x7f4f140b1300 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:07.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.146+0000 7f4f2af38700 1 -- 172.21.15.52:0/1562833999 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f4f140a3d00 msgr2=0x7f4f140a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:07.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.146+0000 7f4f2af38700 1 --2- 172.21.15.52:0/1562833999 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f4f140a3d00 0x7f4f140a61c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4f241036d0 tx=0x7f4f1800a3b0 comp rx=0 tx=0).stop 2024-04-02T13:52:07.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.146+0000 7f4f2af38700 1 -- 172.21.15.52:0/1562833999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f24102fe0 msgr2=0x7f4f2419b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:07.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.146+0000 7f4f2af38700 1 --2- 172.21.15.52:0/1562833999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f24102fe0 0x7f4f2419b480 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000e9c0 tx=0x7f4f1000ecd0 comp rx=0 tx=0).stop 2024-04-02T13:52:07.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.146+0000 7f4f294d5700 1 -- 172.21.15.52:0/1562833999 reap_dead start 2024-04-02T13:52:07.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.146+0000 7f4f2af38700 1 -- 172.21.15.52:0/1562833999 shutdown_connections 2024-04-02T13:52:07.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.146+0000 7f4f2af38700 1 -- 172.21.15.52:0/1562833999 >> 172.21.15.52:0/1562833999 conn(0x7f4f24079720 msgr2=0x7f4f241004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:07.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.146+0000 7f4f2af38700 1 -- 172.21.15.52:0/1562833999 shutdown_connections 2024-04-02T13:52:07.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:07.146+0000 7f4f2af38700 1 -- 172.21.15.52:0/1562833999 wait complete. 2024-04-02T13:52:08.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.339+0000 7fcaebfff700 1 -- 172.21.15.52:0/2746415159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaec1036f0 msgr2=0x7fcaec103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:08.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.339+0000 7fcaebfff700 1 --2- 172.21.15.52:0/2746415159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaec1036f0 0x7fcaec103ad0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fcadc009990 tx=0x7fcadc009ca0 comp rx=0 tx=0).stop 2024-04-02T13:52:08.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.339+0000 7fcaebfff700 1 -- 172.21.15.52:0/2746415159 shutdown_connections 2024-04-02T13:52:08.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.339+0000 7fcaebfff700 1 --2- 172.21.15.52:0/2746415159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcaec10cad0 0x7fcaec10ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:08.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.339+0000 7fcaebfff700 1 --2- 172.21.15.52:0/2746415159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaec1036f0 0x7fcaec103ad0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:08.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.339+0000 7fcaebfff700 1 --2- 172.21.15.52:0/2746415159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaec104010 0x7fcaec10c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:08.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.339+0000 7fcaebfff700 1 -- 172.21.15.52:0/2746415159 >> 172.21.15.52:0/2746415159 conn(0x7fcaec0fcec0 msgr2=0x7fcaec0ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:08.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.339+0000 7fcaebfff700 1 -- 172.21.15.52:0/2746415159 shutdown_connections 2024-04-02T13:52:08.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.339+0000 7fcaebfff700 1 -- 172.21.15.52:0/2746415159 wait complete. 2024-04-02T13:52:08.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.339+0000 7fcaebfff700 1 Processor -- start 2024-04-02T13:52:08.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.339+0000 7fcaebfff700 1 -- start start 2024-04-02T13:52:08.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.340+0000 7fcaebfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaec1036f0 0x7fcaec19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:08.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.340+0000 7fcaebfff700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcaec104010 0x7fcaec19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:08.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.340+0000 7fcaebfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaec10cad0 0x7fcaec198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:08.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.340+0000 7fcaebfff700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec112810 con 0x7fcaec10cad0 2024-04-02T13:52:08.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.340+0000 7fcaebfff700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec112690 con 0x7fcaec1036f0 2024-04-02T13:52:08.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.340+0000 7fcaebfff700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec112990 con 0x7fcaec104010 2024-04-02T13:52:08.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.340+0000 7fcaeb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaec10cad0 0x7fcaec198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:08.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.340+0000 7fcaeb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaec10cad0 0x7fcaec198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58676/0 (socket says 172.21.15.52:58676) 2024-04-02T13:52:08.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.340+0000 7fcaeb7fe700 1 -- 172.21.15.52:0/3262121943 learned_addr learned my addr 172.21.15.52:0/3262121943 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:08.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.340+0000 7fcaea7fc700 1 --2- 172.21.15.52:0/3262121943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcaec104010 0x7fcaec19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:08.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.340+0000 7fcaeaffd700 1 --2- 172.21.15.52:0/3262121943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaec1036f0 0x7fcaec19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:08.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.340+0000 7fcaeb7fe700 1 -- 172.21.15.52:0/3262121943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcaec104010 msgr2=0x7fcaec19ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:08.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.340+0000 7fcaeb7fe700 1 --2- 172.21.15.52:0/3262121943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcaec104010 0x7fcaec19ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:08.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.340+0000 7fcaeb7fe700 1 -- 172.21.15.52:0/3262121943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaec1036f0 msgr2=0x7fcaec19e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:08.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.340+0000 7fcaeb7fe700 1 --2- 172.21.15.52:0/3262121943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaec1036f0 0x7fcaec19e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:08.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.340+0000 7fcaeb7fe700 1 -- 172.21.15.52:0/3262121943 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaec199210 con 0x7fcaec10cad0 2024-04-02T13:52:08.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.340+0000 7fcaeaffd700 1 --2- 172.21.15.52:0/3262121943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaec1036f0 0x7fcaec19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:52:08.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.340+0000 7fcaeb7fe700 1 --2- 172.21.15.52:0/3262121943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaec10cad0 0x7fcaec198ab0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcae4007ee0 tx=0x7fcae400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:08.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.340+0000 7fcaf09fd700 1 -- 172.21.15.52:0/3262121943 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae400dbd0 con 0x7fcaec10cad0 2024-04-02T13:52:08.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.340+0000 7fcaebfff700 1 -- 172.21.15.52:0/3262121943 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaec199500 con 0x7fcaec10cad0 2024-04-02T13:52:08.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.340+0000 7fcaf09fd700 1 -- 172.21.15.52:0/3262121943 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcae400dd70 con 0x7fcaec10cad0 2024-04-02T13:52:08.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.341+0000 7fcaf09fd700 1 -- 172.21.15.52:0/3262121943 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae4005a60 con 0x7fcaec10cad0 2024-04-02T13:52:08.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.341+0000 7fcaebfff700 1 -- 172.21.15.52:0/3262121943 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaec1a57c0 con 0x7fcaec10cad0 2024-04-02T13:52:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.341+0000 7fcaebfff700 1 -- 172.21.15.52:0/3262121943 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcaec062d40 con 0x7fcaec10cad0 2024-04-02T13:52:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.343+0000 7fcaf09fd700 1 -- 172.21.15.52:0/3262121943 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcae401a020 con 0x7fcaec10cad0 2024-04-02T13:52:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.343+0000 7fcaf09fd700 1 --2- 172.21.15.52:0/3262121943 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fcad40a3cd0 0x7fcad40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.343+0000 7fcaeaffd700 1 --2- 172.21.15.52:0/3262121943 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fcad40a3cd0 0x7fcad40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.343+0000 7fcaf09fd700 1 -- 172.21.15.52:0/3262121943 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fcae40d0670 con 0x7fcaec10cad0 2024-04-02T13:52:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.344+0000 7fcaf09fd700 1 --2- 172.21.15.52:0/3262121943 >> [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472] conn(0x7fcad40aee30 0x7fcad40b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.344+0000 7fcaf09fd700 1 -- 172.21.15.52:0/3262121943 --> [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcad40b19a0 con 0x7fcad40aee30 2024-04-02T13:52:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.344+0000 7fcaf09fd700 1 -- 172.21.15.52:0/3262121943 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fcae40d0ac0 con 0x7fcaec10cad0 2024-04-02T13:52:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.344+0000 7fcaea7fc700 1 --2- 172.21.15.52:0/3262121943 >> [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472] conn(0x7fcad40aee30 0x7fcad40b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.344+0000 7fcaeaffd700 1 --2- 172.21.15.52:0/3262121943 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fcad40a3cd0 0x7fcad40a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcadc000c00 tx=0x7fcadc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.344+0000 7fcaea7fc700 1 --2- 172.21.15.52:0/3262121943 >> [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472] conn(0x7fcad40aee30 0x7fcad40b12d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:08.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.345+0000 7fcaf09fd700 1 -- 172.21.15.52:0/3262121943 <== osd.0 v2:172.21.15.52:6802/1314806472 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcad40b19a0 con 0x7fcad40aee30 2024-04-02T13:52:08.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.361+0000 7fcaebfff700 1 -- 172.21.15.52:0/3262121943 --> [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcaec0631c0 con 0x7fcad40aee30 2024-04-02T13:52:08.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.362+0000 7fcaf09fd700 1 -- 172.21.15.52:0/3262121943 <== osd.0 v2:172.21.15.52:6802/1314806472 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcaec0631c0 con 0x7fcad40aee30 2024-04-02T13:52:08.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.362+0000 7fcaebfff700 1 -- 172.21.15.52:0/3262121943 >> [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472] conn(0x7fcad40aee30 msgr2=0x7fcad40b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:08.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.362+0000 7fcaebfff700 1 --2- 172.21.15.52:0/3262121943 >> [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472] conn(0x7fcad40aee30 0x7fcad40b12d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:08.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.362+0000 7fcaebfff700 1 -- 172.21.15.52:0/3262121943 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fcad40a3cd0 msgr2=0x7fcad40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:08.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.362+0000 7fcaebfff700 1 --2- 172.21.15.52:0/3262121943 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fcad40a3cd0 0x7fcad40a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcadc000c00 tx=0x7fcadc005fb0 comp rx=0 tx=0).stop 2024-04-02T13:52:08.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.362+0000 7fcaebfff700 1 -- 172.21.15.52:0/3262121943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaec10cad0 msgr2=0x7fcaec198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:08.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.362+0000 7fcaebfff700 1 --2- 172.21.15.52:0/3262121943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaec10cad0 0x7fcaec198ab0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcae4007ee0 tx=0x7fcae400d3b0 comp rx=0 tx=0).stop 2024-04-02T13:52:08.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.362+0000 7fcaeb7fe700 1 -- 172.21.15.52:0/3262121943 reap_dead start 2024-04-02T13:52:08.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.362+0000 7fcaebfff700 1 -- 172.21.15.52:0/3262121943 shutdown_connections 2024-04-02T13:52:08.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.362+0000 7fcaebfff700 1 -- 172.21.15.52:0/3262121943 >> 172.21.15.52:0/3262121943 conn(0x7fcaec0fcec0 msgr2=0x7fcaec0fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:08.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.362+0000 7fcaebfff700 1 -- 172.21.15.52:0/3262121943 shutdown_connections 2024-04-02T13:52:08.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:08.362+0000 7fcaebfff700 1 -- 172.21.15.52:0/3262121943 wait complete. 2024-04-02T13:52:08.375 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:08 smithi052 ceph-mon[32104]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 298 B/s wr, 10 op/s; 6 B/s, 0 keys/s, 0 objects/s recovering 2024-04-02T13:52:08.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:08 smithi088 ceph-mon[36528]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 298 B/s wr, 10 op/s; 6 B/s, 0 keys/s, 0 objects/s recovering 2024-04-02T13:52:08.488 INFO:teuthology.orchestra.run.smithi052.stdout:55834574925 2024-04-02T13:52:08.488 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.1 2024-04-02T13:52:08.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:08 smithi189 ceph-mon[36683]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 298 B/s wr, 10 op/s; 6 B/s, 0 keys/s, 0 objects/s recovering 2024-04-02T13:52:09.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.284+0000 7f3ae62fe700 1 -- 172.21.15.52:0/1621362579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ae0102fe0 msgr2=0x7f3ae010f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:09.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.284+0000 7f3ae62fe700 1 --2- 172.21.15.52:0/1621362579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ae0102fe0 0x7f3ae010f490 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800b0a0 tx=0x7f3ad800b3b0 comp rx=0 tx=0).stop 2024-04-02T13:52:09.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.284+0000 7f3ae62fe700 1 -- 172.21.15.52:0/1621362579 shutdown_connections 2024-04-02T13:52:09.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.284+0000 7f3ae62fe700 1 --2- 172.21.15.52:0/1621362579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ae0102fe0 0x7f3ae010f490 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:09.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.284+0000 7f3ae62fe700 1 --2- 172.21.15.52:0/1621362579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ae0107520 0x7f3ae0107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:09.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.284+0000 7f3ae62fe700 1 --2- 172.21.15.52:0/1621362579 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ae0102640 0x7f3ae0102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:09.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.284+0000 7f3ae62fe700 1 -- 172.21.15.52:0/1621362579 >> 172.21.15.52:0/1621362579 conn(0x7f3ae0079720 msgr2=0x7f3ae0079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:09.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.284+0000 7f3ae62fe700 1 -- 172.21.15.52:0/1621362579 shutdown_connections 2024-04-02T13:52:09.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.284+0000 7f3ae62fe700 1 -- 172.21.15.52:0/1621362579 wait complete. 2024-04-02T13:52:09.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.285+0000 7f3ae62fe700 1 Processor -- start 2024-04-02T13:52:09.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.285+0000 7f3ae62fe700 1 -- start start 2024-04-02T13:52:09.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.285+0000 7f3ae62fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ae0102640 0x7f3ae00748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:09.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.285+0000 7f3ae62fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ae0102fe0 0x7f3ae0074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:09.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.285+0000 7f3ae62fe700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ae0107520 0x7f3ae0075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.285+0000 7f3ae62fe700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae01128b0 con 0x7f3ae0102fe0 2024-04-02T13:52:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.285+0000 7f3ae62fe700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae0112730 con 0x7f3ae0102640 2024-04-02T13:52:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.285+0000 7f3ae62fe700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae0112a30 con 0x7f3ae0107520 2024-04-02T13:52:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.285+0000 7f3adf7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ae0102fe0 0x7f3ae0074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.286+0000 7f3adf7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ae0102fe0 0x7f3ae0074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58702/0 (socket says 172.21.15.52:58702) 2024-04-02T13:52:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.286+0000 7f3adf7fe700 1 -- 172.21.15.52:0/739189209 learned_addr learned my addr 172.21.15.52:0/739189209 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.286+0000 7f3ae489b700 1 --2- 172.21.15.52:0/739189209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ae0107520 0x7f3ae0075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.286+0000 7f3adffff700 1 --2- 172.21.15.52:0/739189209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ae0102640 0x7f3ae00748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.286+0000 7f3adf7fe700 1 -- 172.21.15.52:0/739189209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ae0107520 msgr2=0x7f3ae0075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.286+0000 7f3adf7fe700 1 --2- 172.21.15.52:0/739189209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ae0107520 0x7f3ae0075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.286+0000 7f3adf7fe700 1 -- 172.21.15.52:0/739189209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ae0102640 msgr2=0x7f3ae00748f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.286+0000 7f3adf7fe700 1 --2- 172.21.15.52:0/739189209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ae0102640 0x7f3ae00748f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.286+0000 7f3adf7fe700 1 -- 172.21.15.52:0/739189209 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae01a7100 con 0x7f3ae0102fe0 2024-04-02T13:52:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.286+0000 7f3adf7fe700 1 --2- 172.21.15.52:0/739189209 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ae0102fe0 0x7f3ae0074e30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400c9b0 tx=0x7f3ad400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:09.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.286+0000 7f3add7fa700 1 -- 172.21.15.52:0/739189209 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4007b00 con 0x7f3ae0102fe0 2024-04-02T13:52:09.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.286+0000 7f3ae62fe700 1 -- 172.21.15.52:0/739189209 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae01a73f0 con 0x7f3ae0102fe0 2024-04-02T13:52:09.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.286+0000 7f3add7fa700 1 -- 172.21.15.52:0/739189209 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ad4007ca0 con 0x7f3ae0102fe0 2024-04-02T13:52:09.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.286+0000 7f3add7fa700 1 -- 172.21.15.52:0/739189209 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4018780 con 0x7f3ae0102fe0 2024-04-02T13:52:09.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.286+0000 7f3ae62fe700 1 -- 172.21.15.52:0/739189209 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae01a7980 con 0x7f3ae0102fe0 2024-04-02T13:52:09.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.287+0000 7f3ae62fe700 1 -- 172.21.15.52:0/739189209 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3ae0100fd0 con 0x7f3ae0102fe0 2024-04-02T13:52:09.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.288+0000 7f3add7fa700 1 -- 172.21.15.52:0/739189209 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3ad400f450 con 0x7f3ae0102fe0 2024-04-02T13:52:09.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.289+0000 7f3add7fa700 1 --2- 172.21.15.52:0/739189209 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ac80a3cd0 0x7f3ac80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:09.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.289+0000 7f3adffff700 1 --2- 172.21.15.52:0/739189209 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ac80a3cd0 0x7f3ac80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:09.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.289+0000 7f3add7fa700 1 -- 172.21.15.52:0/739189209 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f3ad40d0e00 con 0x7f3ae0102fe0 2024-04-02T13:52:09.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.289+0000 7f3add7fa700 1 --2- 172.21.15.52:0/739189209 >> [v2:172.21.15.189:6816/1602944902,v1:172.21.15.189:6817/1602944902] conn(0x7f3ac80aee30 0x7f3ac80b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:09.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.289+0000 7f3add7fa700 1 -- 172.21.15.52:0/739189209 --> [v2:172.21.15.189:6816/1602944902,v1:172.21.15.189:6817/1602944902] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3ad4011ec0 con 0x7f3ac80aee30 2024-04-02T13:52:09.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.289+0000 7f3add7fa700 1 -- 172.21.15.52:0/739189209 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f3ad40d1250 con 0x7f3ae0102fe0 2024-04-02T13:52:09.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.290+0000 7f3adffff700 1 --2- 172.21.15.52:0/739189209 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ac80a3cd0 0x7f3ac80a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0005ec0 tx=0x7f3ad0005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:09.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.290+0000 7f3ae489b700 1 --2- 172.21.15.52:0/739189209 >> [v2:172.21.15.189:6816/1602944902,v1:172.21.15.189:6817/1602944902] conn(0x7f3ac80aee30 0x7f3ac80b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:09.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.290+0000 7f3ae489b700 1 --2- 172.21.15.52:0/739189209 >> [v2:172.21.15.189:6816/1602944902,v1:172.21.15.189:6817/1602944902] conn(0x7f3ac80aee30 0x7f3ac80b12d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:09.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.291+0000 7f3add7fa700 1 -- 172.21.15.52:0/739189209 <== osd.8 v2:172.21.15.189:6816/1602944902 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3ad4011ec0 con 0x7f3ac80aee30 2024-04-02T13:52:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.308+0000 7f3ae62fe700 1 -- 172.21.15.52:0/739189209 --> [v2:172.21.15.189:6816/1602944902,v1:172.21.15.189:6817/1602944902] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3ae0103cd0 con 0x7f3ac80aee30 2024-04-02T13:52:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.308+0000 7f3add7fa700 1 -- 172.21.15.52:0/739189209 <== osd.8 v2:172.21.15.189:6816/1602944902 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3ae0103cd0 con 0x7f3ac80aee30 2024-04-02T13:52:09.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.309+0000 7f3ae62fe700 1 -- 172.21.15.52:0/739189209 >> [v2:172.21.15.189:6816/1602944902,v1:172.21.15.189:6817/1602944902] conn(0x7f3ac80aee30 msgr2=0x7f3ac80b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:09.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.309+0000 7f3ae62fe700 1 --2- 172.21.15.52:0/739189209 >> [v2:172.21.15.189:6816/1602944902,v1:172.21.15.189:6817/1602944902] conn(0x7f3ac80aee30 0x7f3ac80b12d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:09.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.309+0000 7f3ae62fe700 1 -- 172.21.15.52:0/739189209 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ac80a3cd0 msgr2=0x7f3ac80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:09.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.309+0000 7f3ae62fe700 1 --2- 172.21.15.52:0/739189209 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ac80a3cd0 0x7f3ac80a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0005ec0 tx=0x7f3ad0005e50 comp rx=0 tx=0).stop 2024-04-02T13:52:09.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.309+0000 7f3ae62fe700 1 -- 172.21.15.52:0/739189209 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ae0102fe0 msgr2=0x7f3ae0074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:09.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.309+0000 7f3ae62fe700 1 --2- 172.21.15.52:0/739189209 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ae0102fe0 0x7f3ae0074e30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400c9b0 tx=0x7f3ad400cd70 comp rx=0 tx=0).stop 2024-04-02T13:52:09.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.309+0000 7f3ae489b700 1 -- 172.21.15.52:0/739189209 reap_dead start 2024-04-02T13:52:09.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.309+0000 7f3ae62fe700 1 -- 172.21.15.52:0/739189209 shutdown_connections 2024-04-02T13:52:09.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.309+0000 7f3ae62fe700 1 -- 172.21.15.52:0/739189209 >> 172.21.15.52:0/739189209 conn(0x7f3ae0079720 msgr2=0x7f3ae01005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:09.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.309+0000 7f3ae62fe700 1 -- 172.21.15.52:0/739189209 shutdown_connections 2024-04-02T13:52:09.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:09.309+0000 7f3ae62fe700 1 -- 172.21.15.52:0/739189209 wait complete. 2024-04-02T13:52:09.721 INFO:teuthology.orchestra.run.smithi052.stdout:163208757306 2024-04-02T13:52:09.722 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.5 2024-04-02T13:52:09.747 INFO:teuthology.orchestra.run.smithi052.stdout:34359738451 2024-04-02T13:52:09.747 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.0 2024-04-02T13:52:10.267 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:10 smithi052 ceph-mon[32104]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-04-02T13:52:10.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.367+0000 7fe536bcd700 1 -- 172.21.15.52:0/3918252608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe530101ee0 msgr2=0x7fe53010e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:10.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.367+0000 7fe536bcd700 1 --2- 172.21.15.52:0/3918252608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe530101ee0 0x7fe53010e330 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe52800b0d0 tx=0x7fe52800b3e0 comp rx=0 tx=0).stop 2024-04-02T13:52:10.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.367+0000 7fe536bcd700 1 -- 172.21.15.52:0/3918252608 shutdown_connections 2024-04-02T13:52:10.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.367+0000 7fe536bcd700 1 --2- 172.21.15.52:0/3918252608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe530101ee0 0x7fe53010e330 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:10.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.367+0000 7fe536bcd700 1 --2- 172.21.15.52:0/3918252608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe530107560 0x7fe530107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:10.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.367+0000 7fe536bcd700 1 --2- 172.21.15.52:0/3918252608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe530101520 0x7fe5301019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:10.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.367+0000 7fe536bcd700 1 -- 172.21.15.52:0/3918252608 >> 172.21.15.52:0/3918252608 conn(0x7fe530077540 msgr2=0x7fe530077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:10.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.367+0000 7fe536bcd700 1 -- 172.21.15.52:0/3918252608 shutdown_connections 2024-04-02T13:52:10.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.367+0000 7fe536bcd700 1 -- 172.21.15.52:0/3918252608 wait complete. 2024-04-02T13:52:10.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.368+0000 7fe536bcd700 1 Processor -- start 2024-04-02T13:52:10.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.368+0000 7fe536bcd700 1 -- start start 2024-04-02T13:52:10.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.368+0000 7fe536bcd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe530101520 0x7fe530199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:10.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.368+0000 7fe536bcd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe530101ee0 0x7fe530199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:10.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.368+0000 7fe536bcd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe530107560 0x7fe5301a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:10.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.368+0000 7fe536bcd700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe5301117b0 con 0x7fe530101ee0 2024-04-02T13:52:10.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.368+0000 7fe536bcd700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe530111630 con 0x7fe530107560 2024-04-02T13:52:10.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.368+0000 7fe536bcd700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe530111930 con 0x7fe530101520 2024-04-02T13:52:10.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.368+0000 7fe52ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe530101ee0 0x7fe530199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:10.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.368+0000 7fe52ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe530101ee0 0x7fe530199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58722/0 (socket says 172.21.15.52:58722) 2024-04-02T13:52:10.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.368+0000 7fe52ffff700 1 -- 172.21.15.52:0/519917316 learned_addr learned my addr 172.21.15.52:0/519917316 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:10.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.369+0000 7fe53516a700 1 --2- 172.21.15.52:0/519917316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe530107560 0x7fe5301a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:10.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.369+0000 7fe534969700 1 --2- 172.21.15.52:0/519917316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe530101520 0x7fe530199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:10.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.369+0000 7fe52ffff700 1 -- 172.21.15.52:0/519917316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe530101520 msgr2=0x7fe530199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:10.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.369+0000 7fe52ffff700 1 --2- 172.21.15.52:0/519917316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe530101520 0x7fe530199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:10.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.369+0000 7fe52ffff700 1 -- 172.21.15.52:0/519917316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe530107560 msgr2=0x7fe5301a4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:10.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.369+0000 7fe52ffff700 1 --2- 172.21.15.52:0/519917316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe530107560 0x7fe5301a4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:10.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.369+0000 7fe52ffff700 1 -- 172.21.15.52:0/519917316 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5301a5050 con 0x7fe530101ee0 2024-04-02T13:52:10.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.369+0000 7fe534969700 1 --2- 172.21.15.52:0/519917316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe530101520 0x7fe530199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:52:10.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.369+0000 7fe52ffff700 1 --2- 172.21.15.52:0/519917316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe530101ee0 0x7fe530199fd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00e9a0 tx=0x7fe51c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:10.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.369+0000 7fe52dffb700 1 -- 172.21.15.52:0/519917316 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c00cbd0 con 0x7fe530101ee0 2024-04-02T13:52:10.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.369+0000 7fe536bcd700 1 -- 172.21.15.52:0/519917316 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5301a52e0 con 0x7fe530101ee0 2024-04-02T13:52:10.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.369+0000 7fe52dffb700 1 -- 172.21.15.52:0/519917316 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe51c00cd70 con 0x7fe530101ee0 2024-04-02T13:52:10.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.369+0000 7fe52dffb700 1 -- 172.21.15.52:0/519917316 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c0187c0 con 0x7fe530101ee0 2024-04-02T13:52:10.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.369+0000 7fe536bcd700 1 -- 172.21.15.52:0/519917316 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5301a5840 con 0x7fe530101ee0 2024-04-02T13:52:10.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.370+0000 7fe536bcd700 1 -- 172.21.15.52:0/519917316 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe530062d80 con 0x7fe530101ee0 2024-04-02T13:52:10.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.371+0000 7fe52dffb700 1 -- 172.21.15.52:0/519917316 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe51c010430 con 0x7fe530101ee0 2024-04-02T13:52:10.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.372+0000 7fe52dffb700 1 --2- 172.21.15.52:0/519917316 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe5200a3cd0 0x7fe5200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:10.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.372+0000 7fe534969700 1 --2- 172.21.15.52:0/519917316 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe5200a3cd0 0x7fe5200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:10.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.372+0000 7fe52dffb700 1 -- 172.21.15.52:0/519917316 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fe51c0d1420 con 0x7fe530101ee0 2024-04-02T13:52:10.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.372+0000 7fe52dffb700 1 --2- 172.21.15.52:0/519917316 >> [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509] conn(0x7fe5200aee30 0x7fe5200b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:10.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.372+0000 7fe52dffb700 1 -- 172.21.15.52:0/519917316 --> [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe51c00e1a0 con 0x7fe5200aee30 2024-04-02T13:52:10.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.372+0000 7fe52dffb700 1 -- 172.21.15.52:0/519917316 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fe51c0d17d0 con 0x7fe530101ee0 2024-04-02T13:52:10.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.372+0000 7fe53516a700 1 --2- 172.21.15.52:0/519917316 >> [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509] conn(0x7fe5200aee30 0x7fe5200b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:10.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.372+0000 7fe534969700 1 --2- 172.21.15.52:0/519917316 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe5200a3cd0 0x7fe5200a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe524009b50 tx=0x7fe524009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:10.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.373+0000 7fe53516a700 1 --2- 172.21.15.52:0/519917316 >> [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509] conn(0x7fe5200aee30 0x7fe5200b12d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:10.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.373+0000 7fe52dffb700 1 -- 172.21.15.52:0/519917316 <== osd.6 v2:172.21.15.52:6818/2542655509 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe51c00e1a0 con 0x7fe5200aee30 2024-04-02T13:52:10.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.399+0000 7fe536bcd700 1 -- 172.21.15.52:0/519917316 --> [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe530063200 con 0x7fe5200aee30 2024-04-02T13:52:10.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.399+0000 7fe52dffb700 1 -- 172.21.15.52:0/519917316 <== osd.6 v2:172.21.15.52:6818/2542655509 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe530063200 con 0x7fe5200aee30 2024-04-02T13:52:10.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.400+0000 7fe536bcd700 1 -- 172.21.15.52:0/519917316 >> [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509] conn(0x7fe5200aee30 msgr2=0x7fe5200b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:10.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.400+0000 7fe536bcd700 1 --2- 172.21.15.52:0/519917316 >> [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509] conn(0x7fe5200aee30 0x7fe5200b12d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:10.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.400+0000 7fe536bcd700 1 -- 172.21.15.52:0/519917316 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe5200a3cd0 msgr2=0x7fe5200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:10.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.400+0000 7fe536bcd700 1 --2- 172.21.15.52:0/519917316 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe5200a3cd0 0x7fe5200a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe524009b50 tx=0x7fe524009550 comp rx=0 tx=0).stop 2024-04-02T13:52:10.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.400+0000 7fe536bcd700 1 -- 172.21.15.52:0/519917316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe530101ee0 msgr2=0x7fe530199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:10.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.400+0000 7fe536bcd700 1 --2- 172.21.15.52:0/519917316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe530101ee0 0x7fe530199fd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00e9a0 tx=0x7fe51c00ed60 comp rx=0 tx=0).stop 2024-04-02T13:52:10.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.400+0000 7fe53516a700 1 -- 172.21.15.52:0/519917316 reap_dead start 2024-04-02T13:52:10.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.400+0000 7fe536bcd700 1 -- 172.21.15.52:0/519917316 shutdown_connections 2024-04-02T13:52:10.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.400+0000 7fe536bcd700 1 -- 172.21.15.52:0/519917316 >> 172.21.15.52:0/519917316 conn(0x7fe530077540 msgr2=0x7fe5300fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:10.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.400+0000 7fe536bcd700 1 -- 172.21.15.52:0/519917316 shutdown_connections 2024-04-02T13:52:10.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:10.400+0000 7fe536bcd700 1 -- 172.21.15.52:0/519917316 wait complete. 2024-04-02T13:52:10.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:10 smithi088 ceph-mon[36528]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-04-02T13:52:10.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:10 smithi189 ceph-mon[36683]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-04-02T13:52:10.839 INFO:teuthology.orchestra.run.smithi052.stdout:257698037803 2024-04-02T13:52:10.839 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.8 2024-04-02T13:52:11.441 INFO:teuthology.orchestra.run.smithi052.stdout:77309411400 2024-04-02T13:52:11.442 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.2 2024-04-02T13:52:11.691 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:52:11.707 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:52:12.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:12 smithi088 ceph-mon[36528]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-04-02T13:52:12.422 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:12 smithi052 ceph-mon[32104]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-04-02T13:52:12.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:12 smithi189 ceph-mon[36683]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-04-02T13:52:14.153 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:14 smithi052 ceph-mon[32104]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-04-02T13:52:14.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:14 smithi088 ceph-mon[36528]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-04-02T13:52:14.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:14 smithi189 ceph-mon[36683]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-04-02T13:52:15.084 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:52:15.085 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:52:15.085 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:52:15.086 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:52:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:52:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:52:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:52:15.132 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:52:15.146 INFO:teuthology.orchestra.run.smithi052.stdout:197568495668 2024-04-02T13:52:15.147 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.6 2024-04-02T13:52:15.403 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:52:15.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.956+0000 7ff8ddc72700 1 -- 172.21.15.52:0/2632021811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8d8107520 msgr2=0x7ff8d8107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:15.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.956+0000 7ff8ddc72700 1 --2- 172.21.15.52:0/2632021811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8d8107520 0x7ff8d8107900 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff8c800b0a0 tx=0x7ff8c800b3b0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.956+0000 7ff8ddc72700 1 -- 172.21.15.52:0/2632021811 shutdown_connections 2024-04-02T13:52:15.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.956+0000 7ff8ddc72700 1 --2- 172.21.15.52:0/2632021811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8d8102fe0 0x7ff8d810f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.956+0000 7ff8ddc72700 1 --2- 172.21.15.52:0/2632021811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8d8107520 0x7ff8d8107900 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.956+0000 7ff8ddc72700 1 --2- 172.21.15.52:0/2632021811 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8d8102640 0x7ff8d8102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.956+0000 7ff8ddc72700 1 -- 172.21.15.52:0/2632021811 >> 172.21.15.52:0/2632021811 conn(0x7ff8d8079720 msgr2=0x7ff8d8079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:15.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.956+0000 7ff8ddc72700 1 -- 172.21.15.52:0/2632021811 shutdown_connections 2024-04-02T13:52:15.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.956+0000 7ff8ddc72700 1 -- 172.21.15.52:0/2632021811 wait complete. 2024-04-02T13:52:15.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.957+0000 7ff8ddc72700 1 Processor -- start 2024-04-02T13:52:15.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.957+0000 7ff8ddc72700 1 -- start start 2024-04-02T13:52:15.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.957+0000 7ff8ddc72700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8d8102640 0x7ff8d8198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.957+0000 7ff8ddc72700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8d8102fe0 0x7ff8d8199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.957+0000 7ff8ddc72700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8d8107520 0x7ff8d819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.957+0000 7ff8ddc72700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d81128b0 con 0x7ff8d8107520 2024-04-02T13:52:15.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.957+0000 7ff8ddc72700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d8112730 con 0x7ff8d8102640 2024-04-02T13:52:15.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.957+0000 7ff8ddc72700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d8112a30 con 0x7ff8d8102fe0 2024-04-02T13:52:15.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.957+0000 7ff8d7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8d8107520 0x7ff8d819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.957+0000 7ff8d77fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8d8102640 0x7ff8d8198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.957+0000 7ff8d7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8d8107520 0x7ff8d819d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38900/0 (socket says 172.21.15.52:38900) 2024-04-02T13:52:15.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.957+0000 7ff8d77fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8d8102640 0x7ff8d8198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:48004/0 (socket says 172.21.15.52:48004) 2024-04-02T13:52:15.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.957+0000 7ff8d7fff700 1 -- 172.21.15.52:0/4180152820 learned_addr learned my addr 172.21.15.52:0/4180152820 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:15.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.957+0000 7ff8d6ffd700 1 --2- 172.21.15.52:0/4180152820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8d8102fe0 0x7ff8d8199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.958+0000 7ff8d7fff700 1 -- 172.21.15.52:0/4180152820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8d8102fe0 msgr2=0x7ff8d8199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:15.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.958+0000 7ff8d7fff700 1 --2- 172.21.15.52:0/4180152820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8d8102fe0 0x7ff8d8199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.958+0000 7ff8d7fff700 1 -- 172.21.15.52:0/4180152820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8d8102640 msgr2=0x7ff8d8198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:15.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.958+0000 7ff8d7fff700 1 --2- 172.21.15.52:0/4180152820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8d8102640 0x7ff8d8198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.958+0000 7ff8d7fff700 1 -- 172.21.15.52:0/4180152820 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d819dda0 con 0x7ff8d8107520 2024-04-02T13:52:15.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.958+0000 7ff8d6ffd700 1 --2- 172.21.15.52:0/4180152820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8d8102fe0 0x7ff8d8199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:52:15.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.958+0000 7ff8d77fe700 1 --2- 172.21.15.52:0/4180152820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8d8102640 0x7ff8d8198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:15.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.958+0000 7ff8d7fff700 1 --2- 172.21.15.52:0/4180152820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8d8107520 0x7ff8d819d660 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff8d000c970 tx=0x7ff8d000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:15.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.958+0000 7ff8d4ff9700 1 -- 172.21.15.52:0/4180152820 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d000ce60 con 0x7ff8d8107520 2024-04-02T13:52:15.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.958+0000 7ff8ddc72700 1 -- 172.21.15.52:0/4180152820 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8d819e090 con 0x7ff8d8107520 2024-04-02T13:52:15.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.958+0000 7ff8d4ff9700 1 -- 172.21.15.52:0/4180152820 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8d0004830 con 0x7ff8d8107520 2024-04-02T13:52:15.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.958+0000 7ff8d4ff9700 1 -- 172.21.15.52:0/4180152820 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d00055a0 con 0x7ff8d8107520 2024-04-02T13:52:15.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.958+0000 7ff8ddc72700 1 -- 172.21.15.52:0/4180152820 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8d81a5910 con 0x7ff8d8107520 2024-04-02T13:52:15.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.959+0000 7ff8ddc72700 1 -- 172.21.15.52:0/4180152820 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b8005360 con 0x7ff8d8107520 2024-04-02T13:52:15.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.961+0000 7ff8d4ff9700 1 -- 172.21.15.52:0/4180152820 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff8d0007b00 con 0x7ff8d8107520 2024-04-02T13:52:15.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.961+0000 7ff8d4ff9700 1 --2- 172.21.15.52:0/4180152820 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ff8c40a3cd0 0x7ff8c40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.961+0000 7ff8d4ff9700 1 -- 172.21.15.52:0/4180152820 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff8d00d07e0 con 0x7ff8d8107520 2024-04-02T13:52:15.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.961+0000 7ff8d77fe700 1 --2- 172.21.15.52:0/4180152820 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ff8c40a3cd0 0x7ff8c40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.962+0000 7ff8d77fe700 1 --2- 172.21.15.52:0/4180152820 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ff8c40a3cd0 0x7ff8c40a6190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009fd0 tx=0x7ff8c800b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:15.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:15.965+0000 7ff8d4ff9700 1 -- 172.21.15.52:0/4180152820 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff8d0014030 con 0x7ff8d8107520 2024-04-02T13:52:16.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:16.183+0000 7ff8ddc72700 1 -- 172.21.15.52:0/4180152820 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7ff8b8005920 con 0x7ff8d8107520 2024-04-02T13:52:16.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:16.184+0000 7ff8d4ff9700 1 -- 172.21.15.52:0/4180152820 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff8d009cd00 con 0x7ff8d8107520 2024-04-02T13:52:16.187 INFO:teuthology.orchestra.run.smithi052.stdout:347892351004 2024-04-02T13:52:16.187 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:16 smithi052 ceph-mon[32104]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-02T13:52:16.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:16.186+0000 7ff8ddc72700 1 -- 172.21.15.52:0/4180152820 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ff8c40a3cd0 msgr2=0x7ff8c40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:16.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:16.186+0000 7ff8ddc72700 1 --2- 172.21.15.52:0/4180152820 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ff8c40a3cd0 0x7ff8c40a6190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009fd0 tx=0x7ff8c800b7b0 comp rx=0 tx=0).stop 2024-04-02T13:52:16.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:16.186+0000 7ff8ddc72700 1 -- 172.21.15.52:0/4180152820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8d8107520 msgr2=0x7ff8d819d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:16.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:16.186+0000 7ff8ddc72700 1 --2- 172.21.15.52:0/4180152820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8d8107520 0x7ff8d819d660 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff8d000c970 tx=0x7ff8d000cc80 comp rx=0 tx=0).stop 2024-04-02T13:52:16.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:16.186+0000 7ff8ddc72700 1 -- 172.21.15.52:0/4180152820 shutdown_connections 2024-04-02T13:52:16.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:16.186+0000 7ff8ddc72700 1 --2- 172.21.15.52:0/4180152820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8d8107520 0x7ff8d819d660 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:16.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:16.186+0000 7ff8ddc72700 1 --2- 172.21.15.52:0/4180152820 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ff8c40a3cd0 0x7ff8c40a6190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:16.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:16.186+0000 7ff8ddc72700 1 --2- 172.21.15.52:0/4180152820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8d8102640 0x7ff8d8198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:16.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:16.186+0000 7ff8ddc72700 1 --2- 172.21.15.52:0/4180152820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8d8102fe0 0x7ff8d8199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:16.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:16.186+0000 7ff8ddc72700 1 -- 172.21.15.52:0/4180152820 >> 172.21.15.52:0/4180152820 conn(0x7ff8d8079720 msgr2=0x7ff8d8100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:16.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:16.186+0000 7ff8ddc72700 1 -- 172.21.15.52:0/4180152820 shutdown_connections 2024-04-02T13:52:16.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:16.186+0000 7ff8ddc72700 1 -- 172.21.15.52:0/4180152820 wait complete. 2024-04-02T13:52:16.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:16 smithi088 ceph-mon[36528]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-02T13:52:16.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:16 smithi189 ceph-mon[36683]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-02T13:52:17.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:17 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/4180152820' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-02T13:52:17.361 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351000 got 347892351004 for osd.11 2024-04-02T13:52:17.361 DEBUG:teuthology.parallel:result is None 2024-04-02T13:52:17.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:17 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/4180152820' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-02T13:52:17.437 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:17 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/4180152820' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-02T13:52:18.387 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:18 smithi052 ceph-mon[32104]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-02T13:52:18.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:18 smithi088 ceph-mon[36528]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-02T13:52:18.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:18 smithi189 ceph-mon[36683]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-02T13:52:20.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:20 smithi088 ceph-mon[36528]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:52:20.487 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:20 smithi052 ceph-mon[32104]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:52:20.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:20 smithi189 ceph-mon[36683]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:52:20.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.798+0000 7faaddc83700 1 -- 172.21.15.52:0/1327426507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faad8074820 msgr2=0x7faad8074c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:20.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.798+0000 7faaddc83700 1 --2- 172.21.15.52:0/1327426507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faad8074820 0x7faad8074c00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7faac0009990 tx=0x7faac0009ca0 comp rx=0 tx=0).stop 2024-04-02T13:52:20.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.798+0000 7faaddc83700 1 -- 172.21.15.52:0/1327426507 shutdown_connections 2024-04-02T13:52:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.798+0000 7faaddc83700 1 --2- 172.21.15.52:0/1327426507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faad810eff0 0x7faad81113e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.798+0000 7faaddc83700 1 --2- 172.21.15.52:0/1327426507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faad8074820 0x7faad8074c00 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.798+0000 7faaddc83700 1 --2- 172.21.15.52:0/1327426507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faad8075140 0x7faad810eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.798+0000 7faaddc83700 1 -- 172.21.15.52:0/1327426507 >> 172.21.15.52:0/1327426507 conn(0x7faad8079720 msgr2=0x7faad8079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.798+0000 7faaddc83700 1 -- 172.21.15.52:0/1327426507 shutdown_connections 2024-04-02T13:52:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.798+0000 7faaddc83700 1 -- 172.21.15.52:0/1327426507 wait complete. 2024-04-02T13:52:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.798+0000 7faaddc83700 1 Processor -- start 2024-04-02T13:52:20.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.799+0000 7faaddc83700 1 -- start start 2024-04-02T13:52:20.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.799+0000 7faaddc83700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faad8074820 0x7faad8102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:20.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.799+0000 7faaddc83700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faad8075140 0x7faad8102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:20.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.799+0000 7faaddc83700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faad810eff0 0x7faad8103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:20.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.799+0000 7faaddc83700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faad8114ab0 con 0x7faad8074820 2024-04-02T13:52:20.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.799+0000 7faaddc83700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faad8114930 con 0x7faad810eff0 2024-04-02T13:52:20.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.799+0000 7faaddc83700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faad8114c30 con 0x7faad8075140 2024-04-02T13:52:20.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.799+0000 7faad77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faad8074820 0x7faad8102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:20.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.799+0000 7faad77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faad8074820 0x7faad8102710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38916/0 (socket says 172.21.15.52:38916) 2024-04-02T13:52:20.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.799+0000 7faad77fe700 1 -- 172.21.15.52:0/1614675 learned_addr learned my addr 172.21.15.52:0/1614675 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:20.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.799+0000 7faad6ffd700 1 --2- 172.21.15.52:0/1614675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faad8075140 0x7faad8102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:20.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.799+0000 7faad7fff700 1 --2- 172.21.15.52:0/1614675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faad810eff0 0x7faad8103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.799+0000 7faad77fe700 1 -- 172.21.15.52:0/1614675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faad8075140 msgr2=0x7faad8102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.799+0000 7faad77fe700 1 --2- 172.21.15.52:0/1614675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faad8075140 0x7faad8102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.799+0000 7faad77fe700 1 -- 172.21.15.52:0/1614675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faad810eff0 msgr2=0x7faad8103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.799+0000 7faad77fe700 1 --2- 172.21.15.52:0/1614675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faad810eff0 0x7faad8103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.799+0000 7faad77fe700 1 -- 172.21.15.52:0/1614675 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad81ab650 con 0x7faad8074820 2024-04-02T13:52:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.799+0000 7faad7fff700 1 --2- 172.21.15.52:0/1614675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faad810eff0 0x7faad8103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:52:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.799+0000 7faad77fe700 1 --2- 172.21.15.52:0/1614675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faad8074820 0x7faad8102710 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7faac0005f50 tx=0x7faac00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.800+0000 7faad4ff9700 1 -- 172.21.15.52:0/1614675 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0027070 con 0x7faad8074820 2024-04-02T13:52:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.800+0000 7faaddc83700 1 -- 172.21.15.52:0/1614675 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad81ab8e0 con 0x7faad8074820 2024-04-02T13:52:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.800+0000 7faad4ff9700 1 -- 172.21.15.52:0/1614675 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faac0003780 con 0x7faad8074820 2024-04-02T13:52:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.800+0000 7faad4ff9700 1 -- 172.21.15.52:0/1614675 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0022d70 con 0x7faad8074820 2024-04-02T13:52:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.800+0000 7faaddc83700 1 -- 172.21.15.52:0/1614675 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad81abd40 con 0x7faad8074820 2024-04-02T13:52:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.801+0000 7faaddc83700 1 -- 172.21.15.52:0/1614675 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faad8075ec0 con 0x7faad8074820 2024-04-02T13:52:20.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.802+0000 7faad4ff9700 1 -- 172.21.15.52:0/1614675 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7faac002a070 con 0x7faad8074820 2024-04-02T13:52:20.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.802+0000 7faad4ff9700 1 --2- 172.21.15.52:0/1614675 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faac40a3d00 0x7faac40a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:20.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.803+0000 7faad4ff9700 1 -- 172.21.15.52:0/1614675 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7faac00d1610 con 0x7faad8074820 2024-04-02T13:52:20.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.803+0000 7faad6ffd700 1 --2- 172.21.15.52:0/1614675 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faac40a3d00 0x7faac40a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:20.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.803+0000 7faad6ffd700 1 --2- 172.21.15.52:0/1614675 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faac40a3d00 0x7faac40a61c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7faad8103e80 tx=0x7faac800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:20.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:20.807+0000 7faad4ff9700 1 -- 172.21.15.52:0/1614675 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7faac005ef20 con 0x7faad8074820 2024-04-02T13:52:21.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:21.015+0000 7faaddc83700 1 -- 172.21.15.52:0/1614675 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7faad802dd30 con 0x7faad8074820 2024-04-02T13:52:21.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:21.015+0000 7faad4ff9700 1 -- 172.21.15.52:0/1614675 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7faac0013070 con 0x7faad8074820 2024-04-02T13:52:21.018 INFO:teuthology.orchestra.run.smithi052.stdout:219043332146 2024-04-02T13:52:21.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:21.017+0000 7faaddc83700 1 -- 172.21.15.52:0/1614675 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faac40a3d00 msgr2=0x7faac40a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:21.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:21.017+0000 7faaddc83700 1 --2- 172.21.15.52:0/1614675 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faac40a3d00 0x7faac40a61c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7faad8103e80 tx=0x7faac800b3c0 comp rx=0 tx=0).stop 2024-04-02T13:52:21.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:21.017+0000 7faaddc83700 1 -- 172.21.15.52:0/1614675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faad8074820 msgr2=0x7faad8102710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:21.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:21.017+0000 7faaddc83700 1 --2- 172.21.15.52:0/1614675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faad8074820 0x7faad8102710 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7faac0005f50 tx=0x7faac00047b0 comp rx=0 tx=0).stop 2024-04-02T13:52:21.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:21.017+0000 7faaddc83700 1 -- 172.21.15.52:0/1614675 shutdown_connections 2024-04-02T13:52:21.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:21.017+0000 7faaddc83700 1 --2- 172.21.15.52:0/1614675 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faac40a3d00 0x7faac40a61c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:21.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:21.017+0000 7faaddc83700 1 --2- 172.21.15.52:0/1614675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faad810eff0 0x7faad8103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:21.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:21.017+0000 7faaddc83700 1 --2- 172.21.15.52:0/1614675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faad8074820 0x7faad8102710 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:21.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:21.017+0000 7faaddc83700 1 --2- 172.21.15.52:0/1614675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faad8075140 0x7faad8102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:21.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:21.017+0000 7faaddc83700 1 -- 172.21.15.52:0/1614675 >> 172.21.15.52:0/1614675 conn(0x7faad8079720 msgr2=0x7faad80ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:21.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:21.017+0000 7faaddc83700 1 -- 172.21.15.52:0/1614675 shutdown_connections 2024-04-02T13:52:21.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:21.017+0000 7faaddc83700 1 -- 172.21.15.52:0/1614675 wait complete. 2024-04-02T13:52:21.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:21 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1614675' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-02T13:52:21.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:21 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1614675' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-02T13:52:21.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:21 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1614675' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-02T13:52:22.333 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:22 smithi052 ceph-mon[32104]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-02T13:52:22.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:22 smithi088 ceph-mon[36528]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-02T13:52:22.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:22 smithi189 ceph-mon[36683]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-02T13:52:23.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.990+0000 7f04c4680700 1 -- 172.21.15.52:0/1954221975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04bc101520 msgr2=0x7f04bc1019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:23.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.990+0000 7f04c4680700 1 --2- 172.21.15.52:0/1954221975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04bc101520 0x7f04bc1019a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f04b80099c0 tx=0x7f04b8009cd0 comp rx=0 tx=0).stop 2024-04-02T13:52:23.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.991+0000 7f04c4680700 1 -- 172.21.15.52:0/1954221975 shutdown_connections 2024-04-02T13:52:23.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.991+0000 7f04c4680700 1 --2- 172.21.15.52:0/1954221975 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04bc101ee0 0x7f04bc10e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:23.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.991+0000 7f04c4680700 1 --2- 172.21.15.52:0/1954221975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04bc107560 0x7f04bc107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:23.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.991+0000 7f04c4680700 1 --2- 172.21.15.52:0/1954221975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04bc101520 0x7f04bc1019a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:23.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.991+0000 7f04c4680700 1 -- 172.21.15.52:0/1954221975 >> 172.21.15.52:0/1954221975 conn(0x7f04bc077540 msgr2=0x7f04bc077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:23.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.991+0000 7f04c4680700 1 -- 172.21.15.52:0/1954221975 shutdown_connections 2024-04-02T13:52:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.991+0000 7f04c4680700 1 -- 172.21.15.52:0/1954221975 wait complete. 2024-04-02T13:52:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.991+0000 7f04c4680700 1 Processor -- start 2024-04-02T13:52:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.991+0000 7f04c4680700 1 -- start start 2024-04-02T13:52:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.991+0000 7f04c4680700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04bc101520 0x7f04bc198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.991+0000 7f04c4680700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04bc101ee0 0x7f04bc199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.992+0000 7f04c4680700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04bc107560 0x7f04bc19d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.992+0000 7f04c4680700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc111810 con 0x7f04bc107560 2024-04-02T13:52:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.992+0000 7f04c4680700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc111690 con 0x7f04bc101520 2024-04-02T13:52:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.992+0000 7f04c4680700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc111990 con 0x7f04bc101ee0 2024-04-02T13:52:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.992+0000 7f04c2c1d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04bc107560 0x7f04bc19d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.992+0000 7f04c2c1d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04bc107560 0x7f04bc19d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38220/0 (socket says 172.21.15.52:38220) 2024-04-02T13:52:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.992+0000 7f04c2c1d700 1 -- 172.21.15.52:0/1704400224 learned_addr learned my addr 172.21.15.52:0/1704400224 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:23.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.992+0000 7f04c1c1b700 1 --2- 172.21.15.52:0/1704400224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04bc101ee0 0x7f04bc199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:23.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.992+0000 7f04c241c700 1 --2- 172.21.15.52:0/1704400224 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04bc101520 0x7f04bc198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:23.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.992+0000 7f04c2c1d700 1 -- 172.21.15.52:0/1704400224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04bc101ee0 msgr2=0x7f04bc199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:23.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.992+0000 7f04c2c1d700 1 --2- 172.21.15.52:0/1704400224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04bc101ee0 0x7f04bc199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:23.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.992+0000 7f04c2c1d700 1 -- 172.21.15.52:0/1704400224 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04bc101520 msgr2=0x7f04bc198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:23.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.992+0000 7f04c2c1d700 1 --2- 172.21.15.52:0/1704400224 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04bc101520 0x7f04bc198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:23.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.992+0000 7f04c2c1d700 1 -- 172.21.15.52:0/1704400224 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04bc19dd20 con 0x7f04bc107560 2024-04-02T13:52:23.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.992+0000 7f04c1c1b700 1 --2- 172.21.15.52:0/1704400224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04bc101ee0 0x7f04bc199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:52:23.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.992+0000 7f04c2c1d700 1 --2- 172.21.15.52:0/1704400224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04bc107560 0x7f04bc19d5e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f04b4007ee0 tx=0x7f04b400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:23.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.992+0000 7f04b37fe700 1 -- 172.21.15.52:0/1704400224 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b400dbd0 con 0x7f04bc107560 2024-04-02T13:52:23.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.992+0000 7f04c4680700 1 -- 172.21.15.52:0/1704400224 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04bc19e010 con 0x7f04bc107560 2024-04-02T13:52:23.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.992+0000 7f04b37fe700 1 -- 172.21.15.52:0/1704400224 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04b400dd70 con 0x7f04bc107560 2024-04-02T13:52:23.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.992+0000 7f04b37fe700 1 -- 172.21.15.52:0/1704400224 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b40133f0 con 0x7f04bc107560 2024-04-02T13:52:23.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.992+0000 7f04c4680700 1 -- 172.21.15.52:0/1704400224 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04bc1a5910 con 0x7f04bc107560 2024-04-02T13:52:24.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.993+0000 7f04c4680700 1 -- 172.21.15.52:0/1704400224 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04bc0692b0 con 0x7f04bc107560 2024-04-02T13:52:24.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.995+0000 7f04b37fe700 1 -- 172.21.15.52:0/1704400224 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f04b401a020 con 0x7f04bc107560 2024-04-02T13:52:24.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.995+0000 7f04b37fe700 1 --2- 172.21.15.52:0/1704400224 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f04a80a3cd0 0x7f04a80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:24.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.996+0000 7f04c241c700 1 --2- 172.21.15.52:0/1704400224 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f04a80a3cd0 0x7f04a80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:24.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.996+0000 7f04b37fe700 1 -- 172.21.15.52:0/1704400224 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f04b40d0770 con 0x7f04bc107560 2024-04-02T13:52:24.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.996+0000 7f04c241c700 1 --2- 172.21.15.52:0/1704400224 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f04a80a3cd0 0x7f04a80a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f04ac006fd0 tx=0x7f04ac008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:24.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:23.999+0000 7f04b37fe700 1 -- 172.21.15.52:0/1704400224 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f04b4010070 con 0x7f04bc107560 2024-04-02T13:52:24.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:24.212+0000 7f04c4680700 1 -- 172.21.15.52:0/1704400224 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f04bc1a5c00 con 0x7f04bc107560 2024-04-02T13:52:24.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:24.213+0000 7f04b37fe700 1 -- 172.21.15.52:0/1704400224 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f04b409cc90 con 0x7f04bc107560 2024-04-02T13:52:24.216 INFO:teuthology.orchestra.run.smithi052.stdout:55834574929 2024-04-02T13:52:24.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:24.215+0000 7f04c4680700 1 -- 172.21.15.52:0/1704400224 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f04a80a3cd0 msgr2=0x7f04a80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:24.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:24.215+0000 7f04c4680700 1 --2- 172.21.15.52:0/1704400224 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f04a80a3cd0 0x7f04a80a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f04ac006fd0 tx=0x7f04ac008040 comp rx=0 tx=0).stop 2024-04-02T13:52:24.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:24.215+0000 7f04c4680700 1 -- 172.21.15.52:0/1704400224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04bc107560 msgr2=0x7f04bc19d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:24.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:24.215+0000 7f04c4680700 1 --2- 172.21.15.52:0/1704400224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04bc107560 0x7f04bc19d5e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f04b4007ee0 tx=0x7f04b400d3b0 comp rx=0 tx=0).stop 2024-04-02T13:52:24.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:24.215+0000 7f04c4680700 1 -- 172.21.15.52:0/1704400224 shutdown_connections 2024-04-02T13:52:24.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:24.215+0000 7f04c4680700 1 --2- 172.21.15.52:0/1704400224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04bc107560 0x7f04bc19d5e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:24.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:24.215+0000 7f04c4680700 1 --2- 172.21.15.52:0/1704400224 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f04a80a3cd0 0x7f04a80a6190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:24.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:24.215+0000 7f04c4680700 1 --2- 172.21.15.52:0/1704400224 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04bc101520 0x7f04bc198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:24.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:24.215+0000 7f04c4680700 1 --2- 172.21.15.52:0/1704400224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04bc101ee0 0x7f04bc199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:24.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:24.215+0000 7f04c4680700 1 -- 172.21.15.52:0/1704400224 >> 172.21.15.52:0/1704400224 conn(0x7f04bc077540 msgr2=0x7f04bc100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:24.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:24.215+0000 7f04c4680700 1 -- 172.21.15.52:0/1704400224 shutdown_connections 2024-04-02T13:52:24.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:24.215+0000 7f04c4680700 1 -- 172.21.15.52:0/1704400224 wait complete. 2024-04-02T13:52:24.236 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:24 smithi052 ceph-mon[32104]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:52:24.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:24 smithi088 ceph-mon[36528]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:52:24.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:24 smithi189 ceph-mon[36683]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:52:25.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.236+0000 7ff397b6f700 1 -- 172.21.15.52:0/3192162078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff390107560 msgr2=0x7ff390107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:25.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.236+0000 7ff397b6f700 1 --2- 172.21.15.52:0/3192162078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff390107560 0x7ff390107940 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff3880099c0 tx=0x7ff388009cd0 comp rx=0 tx=0).stop 2024-04-02T13:52:25.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.236+0000 7ff397b6f700 1 -- 172.21.15.52:0/3192162078 shutdown_connections 2024-04-02T13:52:25.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.236+0000 7ff397b6f700 1 --2- 172.21.15.52:0/3192162078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff390101ee0 0x7ff39010e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:25.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.236+0000 7ff397b6f700 1 --2- 172.21.15.52:0/3192162078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff390107560 0x7ff390107940 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:25.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.236+0000 7ff397b6f700 1 --2- 172.21.15.52:0/3192162078 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff390101520 0x7ff3901019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:25.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.236+0000 7ff397b6f700 1 -- 172.21.15.52:0/3192162078 >> 172.21.15.52:0/3192162078 conn(0x7ff390077540 msgr2=0x7ff390077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:25.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.236+0000 7ff397b6f700 1 -- 172.21.15.52:0/3192162078 shutdown_connections 2024-04-02T13:52:25.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.236+0000 7ff397b6f700 1 -- 172.21.15.52:0/3192162078 wait complete. 2024-04-02T13:52:25.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.237+0000 7ff397b6f700 1 Processor -- start 2024-04-02T13:52:25.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.237+0000 7ff397b6f700 1 -- start start 2024-04-02T13:52:25.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.237+0000 7ff397b6f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff390101520 0x7ff390199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:25.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.237+0000 7ff397b6f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff390101ee0 0x7ff39019a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:25.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.237+0000 7ff397b6f700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff390107560 0x7ff3901a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:25.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.237+0000 7ff397b6f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff390111790 con 0x7ff390101ee0 2024-04-02T13:52:25.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.237+0000 7ff397b6f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff390111610 con 0x7ff390101520 2024-04-02T13:52:25.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.237+0000 7ff397b6f700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff390111910 con 0x7ff390107560 2024-04-02T13:52:25.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.237+0000 7ff39510a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff390101ee0 0x7ff39019a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:25.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.237+0000 7ff39590b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff390101520 0x7ff390199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:25.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.237+0000 7ff39510a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff390101ee0 0x7ff39019a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38236/0 (socket says 172.21.15.52:38236) 2024-04-02T13:52:25.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.237+0000 7ff39590b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff390101520 0x7ff390199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:53532/0 (socket says 172.21.15.52:53532) 2024-04-02T13:52:25.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.237+0000 7ff39510a700 1 -- 172.21.15.52:0/318423052 learned_addr learned my addr 172.21.15.52:0/318423052 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:25.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.237+0000 7ff39610c700 1 --2- 172.21.15.52:0/318423052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff390107560 0x7ff3901a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:25.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.238+0000 7ff39510a700 1 -- 172.21.15.52:0/318423052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff390107560 msgr2=0x7ff3901a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:25.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.238+0000 7ff39510a700 1 --2- 172.21.15.52:0/318423052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff390107560 0x7ff3901a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:25.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.238+0000 7ff39510a700 1 -- 172.21.15.52:0/318423052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff390101520 msgr2=0x7ff390199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:25.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.238+0000 7ff39510a700 1 --2- 172.21.15.52:0/318423052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff390101520 0x7ff390199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:25.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.238+0000 7ff39510a700 1 -- 172.21.15.52:0/318423052 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3901a50f0 con 0x7ff390101ee0 2024-04-02T13:52:25.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.238+0000 7ff39590b700 1 --2- 172.21.15.52:0/318423052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff390101520 0x7ff390199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:25.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.238+0000 7ff39510a700 1 --2- 172.21.15.52:0/318423052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff390101ee0 0x7ff39019a070 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff38400d950 tx=0x7ff38400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:25.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.238+0000 7ff38effd700 1 -- 172.21.15.52:0/318423052 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384016070 con 0x7ff390101ee0 2024-04-02T13:52:25.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.238+0000 7ff38effd700 1 -- 172.21.15.52:0/318423052 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff38400fd70 con 0x7ff390101ee0 2024-04-02T13:52:25.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.238+0000 7ff397b6f700 1 -- 172.21.15.52:0/318423052 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff390197980 con 0x7ff390101ee0 2024-04-02T13:52:25.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.238+0000 7ff38effd700 1 -- 172.21.15.52:0/318423052 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38400e610 con 0x7ff390101ee0 2024-04-02T13:52:25.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.238+0000 7ff397b6f700 1 -- 172.21.15.52:0/318423052 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff390197f10 con 0x7ff390101ee0 2024-04-02T13:52:25.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.239+0000 7ff397b6f700 1 -- 172.21.15.52:0/318423052 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3901099f0 con 0x7ff390101ee0 2024-04-02T13:52:25.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.240+0000 7ff38effd700 1 -- 172.21.15.52:0/318423052 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff384019070 con 0x7ff390101ee0 2024-04-02T13:52:25.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.241+0000 7ff38effd700 1 --2- 172.21.15.52:0/318423052 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ff3780a3d50 0x7ff3780a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:25.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.241+0000 7ff38effd700 1 -- 172.21.15.52:0/318423052 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff3840d0300 con 0x7ff390101ee0 2024-04-02T13:52:25.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.241+0000 7ff39590b700 1 --2- 172.21.15.52:0/318423052 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ff3780a3d50 0x7ff3780a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:25.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.241+0000 7ff39590b700 1 --2- 172.21.15.52:0/318423052 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ff3780a3d50 0x7ff3780a6210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff388000c00 tx=0x7ff388005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:25.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.245+0000 7ff38effd700 1 -- 172.21.15.52:0/318423052 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff38409c820 con 0x7ff390101ee0 2024-04-02T13:52:25.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:25 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1704400224' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-02T13:52:25.452 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:25 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1704400224' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-02T13:52:25.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.471+0000 7ff397b6f700 1 -- 172.21.15.52:0/318423052 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7ff3901093d0 con 0x7ff390101ee0 2024-04-02T13:52:25.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.471+0000 7ff38effd700 1 -- 172.21.15.52:0/318423052 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff38409c120 con 0x7ff390101ee0 2024-04-02T13:52:25.475 INFO:teuthology.orchestra.run.smithi052.stdout:313532612643 2024-04-02T13:52:25.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.473+0000 7ff397b6f700 1 -- 172.21.15.52:0/318423052 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ff3780a3d50 msgr2=0x7ff3780a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:25.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.473+0000 7ff397b6f700 1 --2- 172.21.15.52:0/318423052 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ff3780a3d50 0x7ff3780a6210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff388000c00 tx=0x7ff388005fb0 comp rx=0 tx=0).stop 2024-04-02T13:52:25.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.473+0000 7ff397b6f700 1 -- 172.21.15.52:0/318423052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff390101ee0 msgr2=0x7ff39019a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:25.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.473+0000 7ff397b6f700 1 --2- 172.21.15.52:0/318423052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff390101ee0 0x7ff39019a070 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff38400d950 tx=0x7ff38400dc60 comp rx=0 tx=0).stop 2024-04-02T13:52:25.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.474+0000 7ff397b6f700 1 -- 172.21.15.52:0/318423052 shutdown_connections 2024-04-02T13:52:25.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.474+0000 7ff397b6f700 1 --2- 172.21.15.52:0/318423052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff390107560 0x7ff3901a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:25.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.474+0000 7ff397b6f700 1 --2- 172.21.15.52:0/318423052 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ff3780a3d50 0x7ff3780a6210 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:25.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.474+0000 7ff397b6f700 1 --2- 172.21.15.52:0/318423052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff390101520 0x7ff390199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:25.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.474+0000 7ff397b6f700 1 --2- 172.21.15.52:0/318423052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff390101ee0 0x7ff39019a070 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:25.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.474+0000 7ff397b6f700 1 -- 172.21.15.52:0/318423052 >> 172.21.15.52:0/318423052 conn(0x7ff390077540 msgr2=0x7ff3901001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:25.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.474+0000 7ff397b6f700 1 -- 172.21.15.52:0/318423052 shutdown_connections 2024-04-02T13:52:25.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:25.474+0000 7ff397b6f700 1 -- 172.21.15.52:0/318423052 wait complete. 2024-04-02T13:52:25.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:25 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1704400224' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-02T13:52:26.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.079+0000 7f94d7afa700 1 -- 172.21.15.52:0/1077590222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94d0102640 msgr2=0x7f94d0102aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:26.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.079+0000 7f94d7afa700 1 --2- 172.21.15.52:0/1077590222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94d0102640 0x7f94d0102aa0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009990 tx=0x7f94cc009ca0 comp rx=0 tx=0).stop 2024-04-02T13:52:26.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.079+0000 7f94d7afa700 1 -- 172.21.15.52:0/1077590222 shutdown_connections 2024-04-02T13:52:26.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.079+0000 7f94d7afa700 1 --2- 172.21.15.52:0/1077590222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94d0102fe0 0x7f94d010f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:26.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.079+0000 7f94d7afa700 1 --2- 172.21.15.52:0/1077590222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94d0107520 0x7f94d0107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:26.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.079+0000 7f94d7afa700 1 --2- 172.21.15.52:0/1077590222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94d0102640 0x7f94d0102aa0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:26.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.079+0000 7f94d7afa700 1 -- 172.21.15.52:0/1077590222 >> 172.21.15.52:0/1077590222 conn(0x7f94d0079720 msgr2=0x7f94d0079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:26.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.079+0000 7f94d7afa700 1 -- 172.21.15.52:0/1077590222 shutdown_connections 2024-04-02T13:52:26.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.079+0000 7f94d7afa700 1 -- 172.21.15.52:0/1077590222 wait complete. 2024-04-02T13:52:26.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.080+0000 7f94d7afa700 1 Processor -- start 2024-04-02T13:52:26.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.080+0000 7f94d7afa700 1 -- start start 2024-04-02T13:52:26.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.080+0000 7f94d7afa700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94d0102640 0x7f94d0198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:26.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.080+0000 7f94d7afa700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94d0102fe0 0x7f94d0199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:26.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.080+0000 7f94d7afa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94d0107520 0x7f94d019d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:26.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.080+0000 7f94d7afa700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f94d0112910 con 0x7f94d0107520 2024-04-02T13:52:26.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.080+0000 7f94d7afa700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f94d0112790 con 0x7f94d0102640 2024-04-02T13:52:26.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.080+0000 7f94d7afa700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f94d0112a90 con 0x7f94d0102fe0 2024-04-02T13:52:26.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.080+0000 7f94d6097700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94d0107520 0x7f94d019d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:26.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.080+0000 7f94d6097700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94d0107520 0x7f94d019d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38246/0 (socket says 172.21.15.52:38246) 2024-04-02T13:52:26.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.080+0000 7f94d6097700 1 -- 172.21.15.52:0/2384782258 learned_addr learned my addr 172.21.15.52:0/2384782258 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:26.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.080+0000 7f94d5896700 1 --2- 172.21.15.52:0/2384782258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94d0102640 0x7f94d0198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:26.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.080+0000 7f94d5095700 1 --2- 172.21.15.52:0/2384782258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94d0102fe0 0x7f94d0199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:26.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.081+0000 7f94d6097700 1 -- 172.21.15.52:0/2384782258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94d0102fe0 msgr2=0x7f94d0199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:26.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.081+0000 7f94d6097700 1 --2- 172.21.15.52:0/2384782258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94d0102fe0 0x7f94d0199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:26.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.081+0000 7f94d6097700 1 -- 172.21.15.52:0/2384782258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94d0102640 msgr2=0x7f94d0198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:26.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.081+0000 7f94d6097700 1 --2- 172.21.15.52:0/2384782258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94d0102640 0x7f94d0198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:26.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.081+0000 7f94d6097700 1 -- 172.21.15.52:0/2384782258 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d019dd50 con 0x7f94d0107520 2024-04-02T13:52:26.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.081+0000 7f94d6097700 1 --2- 172.21.15.52:0/2384782258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94d0107520 0x7f94d019d610 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f94c800b970 tx=0x7f94c800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:26.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.081+0000 7f94c6ffd700 1 -- 172.21.15.52:0/2384782258 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c8019070 con 0x7f94d0107520 2024-04-02T13:52:26.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.081+0000 7f94c6ffd700 1 -- 172.21.15.52:0/2384782258 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94c800ce10 con 0x7f94d0107520 2024-04-02T13:52:26.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.081+0000 7f94d7afa700 1 -- 172.21.15.52:0/2384782258 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94d019e040 con 0x7f94d0107520 2024-04-02T13:52:26.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.081+0000 7f94c6ffd700 1 -- 172.21.15.52:0/2384782258 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c8007370 con 0x7f94d0107520 2024-04-02T13:52:26.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.081+0000 7f94d7afa700 1 -- 172.21.15.52:0/2384782258 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94d01a58c0 con 0x7f94d0107520 2024-04-02T13:52:26.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.082+0000 7f94d7afa700 1 -- 172.21.15.52:0/2384782258 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94d00692b0 con 0x7f94d0107520 2024-04-02T13:52:26.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.083+0000 7f94c6ffd700 1 -- 172.21.15.52:0/2384782258 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f94c801c070 con 0x7f94d0107520 2024-04-02T13:52:26.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.084+0000 7f94c6ffd700 1 --2- 172.21.15.52:0/2384782258 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f94bc0a3d50 0x7f94bc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:26.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.084+0000 7f94c6ffd700 1 -- 172.21.15.52:0/2384782258 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f94c80cff40 con 0x7f94d0107520 2024-04-02T13:52:26.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.084+0000 7f94d5896700 1 --2- 172.21.15.52:0/2384782258 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f94bc0a3d50 0x7f94bc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:26.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.084+0000 7f94d5896700 1 --2- 172.21.15.52:0/2384782258 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f94bc0a3d50 0x7f94bc0a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f94d01036d0 tx=0x7f94c000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:26.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.091+0000 7f94c6ffd700 1 -- 172.21.15.52:0/2384782258 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f94c809c570 con 0x7f94d0107520 2024-04-02T13:52:26.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.302+0000 7f94d7afa700 1 -- 172.21.15.52:0/2384782258 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f94d019a0a0 con 0x7f94d0107520 2024-04-02T13:52:26.307 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:26 smithi052 ceph-mon[32104]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:52:26.307 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:26 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/318423052' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-02T13:52:26.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.303+0000 7f94c6ffd700 1 -- 172.21.15.52:0/2384782258 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f94c809be70 con 0x7f94d0107520 2024-04-02T13:52:26.308 INFO:teuthology.orchestra.run.smithi052.stdout:292057776169 2024-04-02T13:52:26.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.305+0000 7f94d7afa700 1 -- 172.21.15.52:0/2384782258 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f94bc0a3d50 msgr2=0x7f94bc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:26.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.305+0000 7f94d7afa700 1 --2- 172.21.15.52:0/2384782258 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f94bc0a3d50 0x7f94bc0a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f94d01036d0 tx=0x7f94c000a3b0 comp rx=0 tx=0).stop 2024-04-02T13:52:26.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.305+0000 7f94d7afa700 1 -- 172.21.15.52:0/2384782258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94d0107520 msgr2=0x7f94d019d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:26.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.305+0000 7f94d7afa700 1 --2- 172.21.15.52:0/2384782258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94d0107520 0x7f94d019d610 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f94c800b970 tx=0x7f94c800bc80 comp rx=0 tx=0).stop 2024-04-02T13:52:26.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.305+0000 7f94d7afa700 1 -- 172.21.15.52:0/2384782258 shutdown_connections 2024-04-02T13:52:26.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.305+0000 7f94d7afa700 1 --2- 172.21.15.52:0/2384782258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94d0107520 0x7f94d019d610 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:26.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.305+0000 7f94d7afa700 1 --2- 172.21.15.52:0/2384782258 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f94bc0a3d50 0x7f94bc0a6210 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:26.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.305+0000 7f94d7afa700 1 --2- 172.21.15.52:0/2384782258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94d0102640 0x7f94d0198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:26.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.305+0000 7f94d7afa700 1 --2- 172.21.15.52:0/2384782258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94d0102fe0 0x7f94d0199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:26.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.305+0000 7f94d7afa700 1 -- 172.21.15.52:0/2384782258 >> 172.21.15.52:0/2384782258 conn(0x7f94d0079720 msgr2=0x7f94d01006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:26.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.305+0000 7f94d7afa700 1 -- 172.21.15.52:0/2384782258 shutdown_connections 2024-04-02T13:52:26.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:26.305+0000 7f94d7afa700 1 -- 172.21.15.52:0/2384782258 wait complete. 2024-04-02T13:52:26.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:26 smithi088 ceph-mon[36528]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:52:26.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:26 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/318423052' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-02T13:52:26.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:26 smithi189 ceph-mon[36683]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:52:26.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:26 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/318423052' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-02T13:52:27.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:27 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2384782258' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-02T13:52:27.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:27 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2384782258' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-02T13:52:27.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:27 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2384782258' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-02T13:52:27.761 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776164 got 292057776169 for osd.9 2024-04-02T13:52:27.762 DEBUG:teuthology.parallel:result is None 2024-04-02T13:52:28.209 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574925 got 55834574929 for osd.1 2024-04-02T13:52:28.209 DEBUG:teuthology.parallel:result is None 2024-04-02T13:52:28.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:28 smithi088 ceph-mon[36528]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-02T13:52:28.420 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:28 smithi052 ceph-mon[32104]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-02T13:52:28.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:28 smithi189 ceph-mon[36683]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-02T13:52:29.081 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612638 got 313532612643 for osd.10 2024-04-02T13:52:29.081 DEBUG:teuthology.parallel:result is None 2024-04-02T13:52:29.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.686+0000 7f1ee3adf700 1 -- 172.21.15.52:0/1386420218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1edc06a2a0 msgr2=0x7f1edc06a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:29.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.686+0000 7f1ee3adf700 1 --2- 172.21.15.52:0/1386420218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1edc06a2a0 0x7f1edc06a720 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0009990 tx=0x7f1ed0009ca0 comp rx=0 tx=0).stop 2024-04-02T13:52:29.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.686+0000 7f1ee3adf700 1 -- 172.21.15.52:0/1386420218 shutdown_connections 2024-04-02T13:52:29.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.686+0000 7f1ee3adf700 1 --2- 172.21.15.52:0/1386420218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1edc06ac60 0x7f1edc10f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:29.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.686+0000 7f1ee3adf700 1 --2- 172.21.15.52:0/1386420218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1edc108650 0x7f1edc108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:29.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.686+0000 7f1ee3adf700 1 --2- 172.21.15.52:0/1386420218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1edc06a2a0 0x7f1edc06a720 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:29.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.686+0000 7f1ee3adf700 1 -- 172.21.15.52:0/1386420218 >> 172.21.15.52:0/1386420218 conn(0x7f1edc078420 msgr2=0x7f1edc078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:29.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.686+0000 7f1ee3adf700 1 -- 172.21.15.52:0/1386420218 shutdown_connections 2024-04-02T13:52:29.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.686+0000 7f1ee3adf700 1 -- 172.21.15.52:0/1386420218 wait complete. 2024-04-02T13:52:29.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.687+0000 7f1ee3adf700 1 Processor -- start 2024-04-02T13:52:29.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.687+0000 7f1ee3adf700 1 -- start start 2024-04-02T13:52:29.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.687+0000 7f1ee3adf700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1edc06a2a0 0x7f1edc1015c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:29.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.687+0000 7f1ee3adf700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1edc06ac60 0x7f1edc101b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:29.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.687+0000 7f1ee3adf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1edc108650 0x7f1edc102040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:29.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.687+0000 7f1ee3adf700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc1128f0 con 0x7f1edc108650 2024-04-02T13:52:29.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.687+0000 7f1ee3adf700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc112770 con 0x7f1edc06ac60 2024-04-02T13:52:29.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.687+0000 7f1ee3adf700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1edc112a70 con 0x7f1edc06a2a0 2024-04-02T13:52:29.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.687+0000 7f1ee207c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1edc108650 0x7f1edc102040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.687+0000 7f1ee207c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1edc108650 0x7f1edc102040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38286/0 (socket says 172.21.15.52:38286) 2024-04-02T13:52:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.687+0000 7f1ee207c700 1 -- 172.21.15.52:0/2165495808 learned_addr learned my addr 172.21.15.52:0/2165495808 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.687+0000 7f1ee107a700 1 --2- 172.21.15.52:0/2165495808 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1edc06ac60 0x7f1edc101b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.687+0000 7f1ee187b700 1 --2- 172.21.15.52:0/2165495808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1edc06a2a0 0x7f1edc1015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.688+0000 7f1ee107a700 1 -- 172.21.15.52:0/2165495808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1edc06a2a0 msgr2=0x7f1edc1015c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.688+0000 7f1ee107a700 1 --2- 172.21.15.52:0/2165495808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1edc06a2a0 0x7f1edc1015c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.688+0000 7f1ee107a700 1 -- 172.21.15.52:0/2165495808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1edc108650 msgr2=0x7f1edc102040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.688+0000 7f1ee107a700 1 --2- 172.21.15.52:0/2165495808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1edc108650 0x7f1edc102040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.688+0000 7f1ee107a700 1 -- 172.21.15.52:0/2165495808 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1edc1a7210 con 0x7f1edc06ac60 2024-04-02T13:52:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.688+0000 7f1ee207c700 1 --2- 172.21.15.52:0/2165495808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1edc108650 0x7f1edc102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.688+0000 7f1ee107a700 1 --2- 172.21.15.52:0/2165495808 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1edc06ac60 0x7f1edc101b00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0006010 tx=0x7f1ed0003890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.688+0000 7f1eceffd700 1 -- 172.21.15.52:0/2165495808 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0027070 con 0x7f1edc06ac60 2024-04-02T13:52:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.688+0000 7f1ee3adf700 1 -- 172.21.15.52:0/2165495808 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1edc1a74a0 con 0x7f1edc06ac60 2024-04-02T13:52:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.688+0000 7f1eceffd700 1 -- 172.21.15.52:0/2165495808 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ed00056d0 con 0x7f1edc06ac60 2024-04-02T13:52:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.688+0000 7f1ee3adf700 1 -- 172.21.15.52:0/2165495808 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1edc1a7950 con 0x7f1edc06ac60 2024-04-02T13:52:29.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.688+0000 7f1eceffd700 1 -- 172.21.15.52:0/2165495808 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0022bd0 con 0x7f1edc06ac60 2024-04-02T13:52:29.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.689+0000 7f1ee3adf700 1 -- 172.21.15.52:0/2165495808 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1edc0692b0 con 0x7f1edc06ac60 2024-04-02T13:52:29.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.690+0000 7f1eceffd700 1 -- 172.21.15.52:0/2165495808 <== mon.2 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1ed002a070 con 0x7f1edc06ac60 2024-04-02T13:52:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.691+0000 7f1eceffd700 1 --2- 172.21.15.52:0/2165495808 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1ec80a3cd0 0x7f1ec80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.691+0000 7f1ee187b700 1 --2- 172.21.15.52:0/2165495808 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1ec80a3cd0 0x7f1ec80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.691+0000 7f1eceffd700 1 -- 172.21.15.52:0/2165495808 <== mon.2 v2:172.21.15.88:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f1ed00d1fe0 con 0x7f1edc06ac60 2024-04-02T13:52:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.691+0000 7f1ee187b700 1 --2- 172.21.15.52:0/2165495808 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1ec80a3cd0 0x7f1ec80a6190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1ed800abd0 tx=0x7f1ed800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:29.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.695+0000 7f1eceffd700 1 -- 172.21.15.52:0/2165495808 <== mon.2 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1ed0031020 con 0x7f1edc06ac60 2024-04-02T13:52:29.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.910+0000 7f1ee3adf700 1 -- 172.21.15.52:0/2165495808 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f1edc02dd30 con 0x7f1edc06ac60 2024-04-02T13:52:29.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.911+0000 7f1eceffd700 1 -- 172.21.15.52:0/2165495808 <== mon.2 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1ed0013070 con 0x7f1edc06ac60 2024-04-02T13:52:29.915 INFO:teuthology.orchestra.run.smithi052.stdout:163208757310 2024-04-02T13:52:29.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.913+0000 7f1ee3adf700 1 -- 172.21.15.52:0/2165495808 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1ec80a3cd0 msgr2=0x7f1ec80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:29.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.913+0000 7f1ee3adf700 1 --2- 172.21.15.52:0/2165495808 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1ec80a3cd0 0x7f1ec80a6190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1ed800abd0 tx=0x7f1ed800a380 comp rx=0 tx=0).stop 2024-04-02T13:52:29.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.913+0000 7f1ee3adf700 1 -- 172.21.15.52:0/2165495808 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1edc06ac60 msgr2=0x7f1edc101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:29.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.913+0000 7f1ee3adf700 1 --2- 172.21.15.52:0/2165495808 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1edc06ac60 0x7f1edc101b00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0006010 tx=0x7f1ed0003890 comp rx=0 tx=0).stop 2024-04-02T13:52:29.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.913+0000 7f1ee3adf700 1 -- 172.21.15.52:0/2165495808 shutdown_connections 2024-04-02T13:52:29.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.913+0000 7f1ee3adf700 1 --2- 172.21.15.52:0/2165495808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1edc108650 0x7f1edc102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:29.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.913+0000 7f1ee3adf700 1 --2- 172.21.15.52:0/2165495808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1edc06a2a0 0x7f1edc1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:29.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.913+0000 7f1ee3adf700 1 --2- 172.21.15.52:0/2165495808 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1ec80a3cd0 0x7f1ec80a6190 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:29.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.913+0000 7f1ee3adf700 1 --2- 172.21.15.52:0/2165495808 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1edc06ac60 0x7f1edc101b00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:29.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.913+0000 7f1ee3adf700 1 -- 172.21.15.52:0/2165495808 >> 172.21.15.52:0/2165495808 conn(0x7f1edc078420 msgr2=0x7f1edc1001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:29.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.913+0000 7f1ee3adf700 1 -- 172.21.15.52:0/2165495808 shutdown_connections 2024-04-02T13:52:29.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.913+0000 7f1ee3adf700 1 -- 172.21.15.52:0/2165495808 wait complete. 2024-04-02T13:52:30.027 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332141 got 219043332146 for osd.7 2024-04-02T13:52:30.027 DEBUG:teuthology.parallel:result is None 2024-04-02T13:52:30.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:30 smithi052 ceph-mon[32104]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:52:30.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:30 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1055815038' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-02T13:52:30.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:30 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2165495808' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-02T13:52:30.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:30 smithi088 ceph-mon[36528]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:52:30.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:30 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1055815038' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-02T13:52:30.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:30 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2165495808' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-02T13:52:30.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:30 smithi189 ceph-mon[36683]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:52:30.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:30 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1055815038' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-02T13:52:30.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:30 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2165495808' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-02T13:52:31.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.584+0000 7f1e0929f700 1 -- 172.21.15.52:0/2188802338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e041025f0 msgr2=0x7f1e04102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:31.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.584+0000 7f1e0929f700 1 --2- 172.21.15.52:0/2188802338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e041025f0 0x7f1e04102a70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009990 tx=0x7f1df8009ca0 comp rx=0 tx=0).stop 2024-04-02T13:52:31.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.584+0000 7f1e0929f700 1 -- 172.21.15.52:0/2188802338 shutdown_connections 2024-04-02T13:52:31.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.584+0000 7f1e0929f700 1 --2- 172.21.15.52:0/2188802338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e04102fb0 0x7f1e0410f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:31.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.584+0000 7f1e0929f700 1 --2- 172.21.15.52:0/2188802338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e04108680 0x7f1e04108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:31.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.584+0000 7f1e0929f700 1 --2- 172.21.15.52:0/2188802338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e041025f0 0x7f1e04102a70 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:31.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.584+0000 7f1e0929f700 1 -- 172.21.15.52:0/2188802338 >> 172.21.15.52:0/2188802338 conn(0x7f1e040fe0c0 msgr2=0x7f1e041004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:31.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.584+0000 7f1e0929f700 1 -- 172.21.15.52:0/2188802338 shutdown_connections 2024-04-02T13:52:31.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.584+0000 7f1e0929f700 1 -- 172.21.15.52:0/2188802338 wait complete. 2024-04-02T13:52:31.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.584+0000 7f1e0929f700 1 Processor -- start 2024-04-02T13:52:31.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.585+0000 7f1e0929f700 1 -- start start 2024-04-02T13:52:31.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.585+0000 7f1e0929f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e041025f0 0x7f1e040748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:31.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.585+0000 7f1e0929f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e04102fb0 0x7f1e04074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:31.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.585+0000 7f1e0929f700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e04108680 0x7f1e04075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:31.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.585+0000 7f1e0929f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1e04112990 con 0x7f1e04102fb0 2024-04-02T13:52:31.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.585+0000 7f1e0929f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1e04112810 con 0x7f1e041025f0 2024-04-02T13:52:31.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.585+0000 7f1e0929f700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1e04112b10 con 0x7f1e04108680 2024-04-02T13:52:31.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.585+0000 7f1e027fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e04102fb0 0x7f1e04074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:31.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.585+0000 7f1e027fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e04102fb0 0x7f1e04074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38274/0 (socket says 172.21.15.52:38274) 2024-04-02T13:52:31.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.585+0000 7f1e027fc700 1 -- 172.21.15.52:0/1055815038 learned_addr learned my addr 172.21.15.52:0/1055815038 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:31.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.585+0000 7f1e02ffd700 1 --2- 172.21.15.52:0/1055815038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e041025f0 0x7f1e040748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:31.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.585+0000 7f1e037fe700 1 --2- 172.21.15.52:0/1055815038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e04108680 0x7f1e04075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:31.224 INFO:teuthology.orchestra.run.smithi052.stdout:257698037807 2024-04-02T13:52:31.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.585+0000 7f1e027fc700 1 -- 172.21.15.52:0/1055815038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e04108680 msgr2=0x7f1e04075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:31.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.585+0000 7f1e027fc700 1 --2- 172.21.15.52:0/1055815038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e04108680 0x7f1e04075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:31.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.585+0000 7f1e027fc700 1 -- 172.21.15.52:0/1055815038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e041025f0 msgr2=0x7f1e040748f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:31.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.585+0000 7f1e027fc700 1 --2- 172.21.15.52:0/1055815038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e041025f0 0x7f1e040748f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:31.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.585+0000 7f1e027fc700 1 -- 172.21.15.52:0/1055815038 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e041a72d0 con 0x7f1e04102fb0 2024-04-02T13:52:31.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.586+0000 7f1e027fc700 1 --2- 172.21.15.52:0/1055815038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e04102fb0 0x7f1e04074e30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1df8000c00 tx=0x7f1df8003890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:31.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.586+0000 7f1debfff700 1 -- 172.21.15.52:0/1055815038 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8027070 con 0x7f1e04102fb0 2024-04-02T13:52:31.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.586+0000 7f1e0929f700 1 -- 172.21.15.52:0/1055815038 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e041a7560 con 0x7f1e04102fb0 2024-04-02T13:52:31.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.586+0000 7f1debfff700 1 -- 172.21.15.52:0/1055815038 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1df8005690 con 0x7f1e04102fb0 2024-04-02T13:52:31.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.586+0000 7f1debfff700 1 -- 172.21.15.52:0/1055815038 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8022c60 con 0x7f1e04102fb0 2024-04-02T13:52:31.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.586+0000 7f1e0929f700 1 -- 172.21.15.52:0/1055815038 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e041a7a10 con 0x7f1e04102fb0 2024-04-02T13:52:31.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.587+0000 7f1e0929f700 1 -- 172.21.15.52:0/1055815038 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e040692b0 con 0x7f1e04102fb0 2024-04-02T13:52:31.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.589+0000 7f1debfff700 1 -- 172.21.15.52:0/1055815038 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1df802a070 con 0x7f1e04102fb0 2024-04-02T13:52:31.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.590+0000 7f1debfff700 1 --2- 172.21.15.52:0/1055815038 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1dec0a3cd0 0x7f1dec0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:31.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.590+0000 7f1e02ffd700 1 --2- 172.21.15.52:0/1055815038 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1dec0a3cd0 0x7f1dec0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:31.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.590+0000 7f1debfff700 1 -- 172.21.15.52:0/1055815038 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f1df8022e00 con 0x7f1e04102fb0 2024-04-02T13:52:31.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.590+0000 7f1e02ffd700 1 --2- 172.21.15.52:0/1055815038 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1dec0a3cd0 0x7f1dec0a6190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1e04103730 tx=0x7f1df4007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:31.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.593+0000 7f1debfff700 1 -- 172.21.15.52:0/1055815038 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1df8031020 con 0x7f1e04102fb0 2024-04-02T13:52:31.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.822+0000 7f1e0929f700 1 -- 172.21.15.52:0/1055815038 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f1e0402dd30 con 0x7f1e04102fb0 2024-04-02T13:52:31.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.822+0000 7f1debfff700 1 -- 172.21.15.52:0/1055815038 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1df8013070 con 0x7f1e04102fb0 2024-04-02T13:52:31.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.824+0000 7f1e0929f700 1 -- 172.21.15.52:0/1055815038 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1dec0a3cd0 msgr2=0x7f1dec0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:31.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.824+0000 7f1e0929f700 1 --2- 172.21.15.52:0/1055815038 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1dec0a3cd0 0x7f1dec0a6190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1e04103730 tx=0x7f1df4007400 comp rx=0 tx=0).stop 2024-04-02T13:52:31.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.824+0000 7f1e0929f700 1 -- 172.21.15.52:0/1055815038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e04102fb0 msgr2=0x7f1e04074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:31.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.824+0000 7f1e0929f700 1 --2- 172.21.15.52:0/1055815038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e04102fb0 0x7f1e04074e30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1df8000c00 tx=0x7f1df8003890 comp rx=0 tx=0).stop 2024-04-02T13:52:31.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.824+0000 7f1e0929f700 1 -- 172.21.15.52:0/1055815038 shutdown_connections 2024-04-02T13:52:31.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.824+0000 7f1e0929f700 1 --2- 172.21.15.52:0/1055815038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e04108680 0x7f1e04075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:31.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.824+0000 7f1e0929f700 1 --2- 172.21.15.52:0/1055815038 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f1dec0a3cd0 0x7f1dec0a6190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:31.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.824+0000 7f1e0929f700 1 --2- 172.21.15.52:0/1055815038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e041025f0 0x7f1e040748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:31.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.824+0000 7f1e0929f700 1 --2- 172.21.15.52:0/1055815038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e04102fb0 0x7f1e04074e30 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:31.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.824+0000 7f1e0929f700 1 -- 172.21.15.52:0/1055815038 >> 172.21.15.52:0/1055815038 conn(0x7f1e040fe0c0 msgr2=0x7f1e040ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:31.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.824+0000 7f1e0929f700 1 -- 172.21.15.52:0/1055815038 shutdown_connections 2024-04-02T13:52:31.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:29.824+0000 7f1e0929f700 1 -- 172.21.15.52:0/1055815038 wait complete. 2024-04-02T13:52:31.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.946+0000 7f3e99f9c700 1 -- 172.21.15.52:0/2455896978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e94102640 msgr2=0x7f3e94102aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:31.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.946+0000 7f3e99f9c700 1 --2- 172.21.15.52:0/2455896978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e94102640 0x7f3e94102aa0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009990 tx=0x7f3e7c009ca0 comp rx=0 tx=0).stop 2024-04-02T13:52:31.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.946+0000 7f3e99f9c700 1 -- 172.21.15.52:0/2455896978 shutdown_connections 2024-04-02T13:52:31.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.946+0000 7f3e99f9c700 1 --2- 172.21.15.52:0/2455896978 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e94102fe0 0x7f3e9410f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:31.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.946+0000 7f3e99f9c700 1 --2- 172.21.15.52:0/2455896978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e94107520 0x7f3e94107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:31.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.946+0000 7f3e99f9c700 1 --2- 172.21.15.52:0/2455896978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e94102640 0x7f3e94102aa0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:31.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.946+0000 7f3e99f9c700 1 -- 172.21.15.52:0/2455896978 >> 172.21.15.52:0/2455896978 conn(0x7f3e94079720 msgr2=0x7f3e94079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:31.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.946+0000 7f3e99f9c700 1 -- 172.21.15.52:0/2455896978 shutdown_connections 2024-04-02T13:52:31.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.946+0000 7f3e99f9c700 1 -- 172.21.15.52:0/2455896978 wait complete. 2024-04-02T13:52:31.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.947+0000 7f3e99f9c700 1 Processor -- start 2024-04-02T13:52:31.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.947+0000 7f3e99f9c700 1 -- start start 2024-04-02T13:52:31.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.947+0000 7f3e99f9c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e94102640 0x7f3e94198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:31.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.947+0000 7f3e99f9c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e94102fe0 0x7f3e94199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:31.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.947+0000 7f3e99f9c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e94107520 0x7f3e9419d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:31.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.947+0000 7f3e99f9c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3e941128a0 con 0x7f3e94107520 2024-04-02T13:52:31.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.947+0000 7f3e99f9c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3e94112720 con 0x7f3e94102640 2024-04-02T13:52:31.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.947+0000 7f3e99f9c700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3e94112a20 con 0x7f3e94102fe0 2024-04-02T13:52:31.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.947+0000 7f3e93fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e94107520 0x7f3e9419d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:31.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.947+0000 7f3e93fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e94107520 0x7f3e9419d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56860/0 (socket says 172.21.15.52:56860) 2024-04-02T13:52:31.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.947+0000 7f3e93fff700 1 -- 172.21.15.52:0/2310299489 learned_addr learned my addr 172.21.15.52:0/2310299489 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:31.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.947+0000 7f3e92ffd700 1 --2- 172.21.15.52:0/2310299489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e94102fe0 0x7f3e94199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:31.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.948+0000 7f3e937fe700 1 --2- 172.21.15.52:0/2310299489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e94102640 0x7f3e94198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:31.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.948+0000 7f3e93fff700 1 -- 172.21.15.52:0/2310299489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e94102fe0 msgr2=0x7f3e94199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:31.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.948+0000 7f3e93fff700 1 --2- 172.21.15.52:0/2310299489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e94102fe0 0x7f3e94199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:31.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.948+0000 7f3e93fff700 1 -- 172.21.15.52:0/2310299489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e94102640 msgr2=0x7f3e94198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:31.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.948+0000 7f3e93fff700 1 --2- 172.21.15.52:0/2310299489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e94102640 0x7f3e94198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:31.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.948+0000 7f3e93fff700 1 -- 172.21.15.52:0/2310299489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e9419dd40 con 0x7f3e94107520 2024-04-02T13:52:31.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.948+0000 7f3e93fff700 1 --2- 172.21.15.52:0/2310299489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e94107520 0x7f3e9419d600 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c00d980 tx=0x7f3e8c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:31.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.948+0000 7f3e90ff9700 1 -- 172.21.15.52:0/2310299489 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c017070 con 0x7f3e94107520 2024-04-02T13:52:31.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.948+0000 7f3e90ff9700 1 -- 172.21.15.52:0/2310299489 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e8c005020 con 0x7f3e94107520 2024-04-02T13:52:31.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.948+0000 7f3e99f9c700 1 -- 172.21.15.52:0/2310299489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e9419e030 con 0x7f3e94107520 2024-04-02T13:52:31.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.948+0000 7f3e90ff9700 1 -- 172.21.15.52:0/2310299489 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c009b50 con 0x7f3e94107520 2024-04-02T13:52:31.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.948+0000 7f3e99f9c700 1 -- 172.21.15.52:0/2310299489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e941a58b0 con 0x7f3e94107520 2024-04-02T13:52:31.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.949+0000 7f3e99f9c700 1 -- 172.21.15.52:0/2310299489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e940692b0 con 0x7f3e94107520 2024-04-02T13:52:31.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.950+0000 7f3e90ff9700 1 -- 172.21.15.52:0/2310299489 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3e8c009cf0 con 0x7f3e94107520 2024-04-02T13:52:31.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.951+0000 7f3e90ff9700 1 --2- 172.21.15.52:0/2310299489 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3e800a3cd0 0x7f3e800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:31.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.951+0000 7f3e937fe700 1 --2- 172.21.15.52:0/2310299489 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3e800a3cd0 0x7f3e800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:31.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.951+0000 7f3e90ff9700 1 -- 172.21.15.52:0/2310299489 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f3e8c0d0790 con 0x7f3e94107520 2024-04-02T13:52:31.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.951+0000 7f3e937fe700 1 --2- 172.21.15.52:0/2310299489 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3e800a3cd0 0x7f3e800a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3e941036d0 tx=0x7f3e84007ae0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:31.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:31.956+0000 7f3e90ff9700 1 -- 172.21.15.52:0/2310299489 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3e8c020020 con 0x7f3e94107520 2024-04-02T13:52:32.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:32.163+0000 7f3e99f9c700 1 -- 172.21.15.52:0/2310299489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3e9402dd30 con 0x7f3e94107520 2024-04-02T13:52:32.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:32.165+0000 7f3e90ff9700 1 -- 172.21.15.52:0/2310299489 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3e8c09cc50 con 0x7f3e94107520 2024-04-02T13:52:32.169 INFO:teuthology.orchestra.run.smithi052.stdout:107374182472 2024-04-02T13:52:32.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:32.168+0000 7f3e99f9c700 1 -- 172.21.15.52:0/2310299489 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3e800a3cd0 msgr2=0x7f3e800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:32.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:32.168+0000 7f3e99f9c700 1 --2- 172.21.15.52:0/2310299489 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3e800a3cd0 0x7f3e800a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3e941036d0 tx=0x7f3e84007ae0 comp rx=0 tx=0).stop 2024-04-02T13:52:32.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:32.168+0000 7f3e99f9c700 1 -- 172.21.15.52:0/2310299489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e94107520 msgr2=0x7f3e9419d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:32.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:32.168+0000 7f3e99f9c700 1 --2- 172.21.15.52:0/2310299489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e94107520 0x7f3e9419d600 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c00d980 tx=0x7f3e8c00dc90 comp rx=0 tx=0).stop 2024-04-02T13:52:32.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:32.168+0000 7f3e99f9c700 1 -- 172.21.15.52:0/2310299489 shutdown_connections 2024-04-02T13:52:32.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:32.168+0000 7f3e99f9c700 1 --2- 172.21.15.52:0/2310299489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e94107520 0x7f3e9419d600 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:32.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:32.168+0000 7f3e99f9c700 1 --2- 172.21.15.52:0/2310299489 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3e800a3cd0 0x7f3e800a6190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:32.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:32.168+0000 7f3e99f9c700 1 --2- 172.21.15.52:0/2310299489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e94102640 0x7f3e94198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:32.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:32.168+0000 7f3e99f9c700 1 --2- 172.21.15.52:0/2310299489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e94102fe0 0x7f3e94199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:32.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:32.168+0000 7f3e99f9c700 1 -- 172.21.15.52:0/2310299489 >> 172.21.15.52:0/2310299489 conn(0x7f3e94079720 msgr2=0x7f3e941006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:32.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:32.168+0000 7f3e99f9c700 1 -- 172.21.15.52:0/2310299489 shutdown_connections 2024-04-02T13:52:32.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:32.168+0000 7f3e99f9c700 1 -- 172.21.15.52:0/2310299489 wait complete. 2024-04-02T13:52:32.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:32 smithi088 ceph-mon[36528]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-02T13:52:32.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:32 smithi052 ceph-mon[32104]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-02T13:52:32.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:32 smithi189 ceph-mon[36683]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-02T13:52:33.280 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037803 got 257698037807 for osd.8 2024-04-02T13:52:33.281 DEBUG:teuthology.parallel:result is None 2024-04-02T13:52:33.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:33 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2310299489' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-02T13:52:33.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:33 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2310299489' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-02T13:52:33.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:33 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2310299489' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-02T13:52:33.955 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757306 got 163208757310 for osd.5 2024-04-02T13:52:33.955 DEBUG:teuthology.parallel:result is None 2024-04-02T13:52:34.488 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:34 smithi052 ceph-mon[32104]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T13:52:34.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:34 smithi189 ceph-mon[36683]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T13:52:34.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:34 smithi088 ceph-mon[36528]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T13:52:34.674 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:52:34.703 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182467 got 107374182472 for osd.3 2024-04-02T13:52:34.703 DEBUG:teuthology.parallel:result is None 2024-04-02T13:52:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.157+0000 7fd2dca57700 1 -- 172.21.15.52:0/609162333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd2d8101e90 msgr2=0x7fd2d810e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.157+0000 7fd2dca57700 1 --2- 172.21.15.52:0/609162333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd2d8101e90 0x7fd2d810e380 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000b0d0 tx=0x7fd2d000b3e0 comp rx=0 tx=0).stop 2024-04-02T13:52:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.157+0000 7fd2dca57700 1 -- 172.21.15.52:0/609162333 shutdown_connections 2024-04-02T13:52:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.157+0000 7fd2dca57700 1 --2- 172.21.15.52:0/609162333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd2d8101e90 0x7fd2d810e380 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.157+0000 7fd2dca57700 1 --2- 172.21.15.52:0/609162333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2d8106480 0x7fd2d8106860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.157+0000 7fd2dca57700 1 --2- 172.21.15.52:0/609162333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2d81014d0 0x7fd2d8101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.157+0000 7fd2dca57700 1 -- 172.21.15.52:0/609162333 >> 172.21.15.52:0/609162333 conn(0x7fd2d8078500 msgr2=0x7fd2d8078910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.157+0000 7fd2dca57700 1 -- 172.21.15.52:0/609162333 shutdown_connections 2024-04-02T13:52:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.157+0000 7fd2dca57700 1 -- 172.21.15.52:0/609162333 wait complete. 2024-04-02T13:52:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.157+0000 7fd2dca57700 1 Processor -- start 2024-04-02T13:52:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.158+0000 7fd2dca57700 1 -- start start 2024-04-02T13:52:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.158+0000 7fd2dca57700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd2d81014d0 0x7fd2d8198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.158+0000 7fd2dca57700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2d8101e90 0x7fd2d81992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.158+0000 7fd2dca57700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2d8106480 0x7fd2d819d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.158+0000 7fd2dca57700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d8111810 con 0x7fd2d81014d0 2024-04-02T13:52:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.158+0000 7fd2dca57700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d8111690 con 0x7fd2d8101e90 2024-04-02T13:52:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.158+0000 7fd2dca57700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d8111990 con 0x7fd2d8106480 2024-04-02T13:52:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.158+0000 7fd2d659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd2d81014d0 0x7fd2d8198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.158+0000 7fd2d659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd2d81014d0 0x7fd2d8198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56886/0 (socket says 172.21.15.52:56886) 2024-04-02T13:52:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.158+0000 7fd2d659c700 1 -- 172.21.15.52:0/1058831306 learned_addr learned my addr 172.21.15.52:0/1058831306 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.158+0000 7fd2d6d9d700 1 --2- 172.21.15.52:0/1058831306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2d8106480 0x7fd2d819d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.158+0000 7fd2cffff700 1 --2- 172.21.15.52:0/1058831306 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2d8101e90 0x7fd2d81992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.158+0000 7fd2d659c700 1 -- 172.21.15.52:0/1058831306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2d8106480 msgr2=0x7fd2d819d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.158+0000 7fd2d659c700 1 --2- 172.21.15.52:0/1058831306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2d8106480 0x7fd2d819d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.158+0000 7fd2d659c700 1 -- 172.21.15.52:0/1058831306 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2d8101e90 msgr2=0x7fd2d81992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.158+0000 7fd2d659c700 1 --2- 172.21.15.52:0/1058831306 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2d8101e90 0x7fd2d81992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.158+0000 7fd2d659c700 1 -- 172.21.15.52:0/1058831306 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d819dd90 con 0x7fd2d81014d0 2024-04-02T13:52:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.158+0000 7fd2d6d9d700 1 --2- 172.21.15.52:0/1058831306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2d8106480 0x7fd2d819d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:52:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.158+0000 7fd2cffff700 1 --2- 172.21.15.52:0/1058831306 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2d8101e90 0x7fd2d81992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:52:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.158+0000 7fd2d659c700 1 --2- 172.21.15.52:0/1058831306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd2d81014d0 0x7fd2d8198d80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800d980 tx=0x7fd2c800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.159+0000 7fd2cf7fe700 1 -- 172.21.15.52:0/1058831306 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8017030 con 0x7fd2d81014d0 2024-04-02T13:52:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.159+0000 7fd2dca57700 1 -- 172.21.15.52:0/1058831306 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d819e080 con 0x7fd2d81014d0 2024-04-02T13:52:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.159+0000 7fd2cf7fe700 1 -- 172.21.15.52:0/1058831306 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2c8004500 con 0x7fd2d81014d0 2024-04-02T13:52:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.159+0000 7fd2cf7fe700 1 -- 172.21.15.52:0/1058831306 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8003b20 con 0x7fd2d81014d0 2024-04-02T13:52:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.159+0000 7fd2dca57700 1 -- 172.21.15.52:0/1058831306 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d81a5900 con 0x7fd2d81014d0 2024-04-02T13:52:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.160+0000 7fd2dca57700 1 -- 172.21.15.52:0/1058831306 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2d80692b0 con 0x7fd2d81014d0 2024-04-02T13:52:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.161+0000 7fd2cf7fe700 1 -- 172.21.15.52:0/1058831306 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd2c801a070 con 0x7fd2d81014d0 2024-04-02T13:52:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.161+0000 7fd2cf7fe700 1 --2- 172.21.15.52:0/1058831306 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fd2c40a3cd0 0x7fd2c40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.162+0000 7fd2cffff700 1 --2- 172.21.15.52:0/1058831306 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fd2c40a3cd0 0x7fd2c40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.162+0000 7fd2cf7fe700 1 -- 172.21.15.52:0/1058831306 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd2c80d02b0 con 0x7fd2d81014d0 2024-04-02T13:52:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.162+0000 7fd2cffff700 1 --2- 172.21.15.52:0/1058831306 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fd2c40a3cd0 0x7fd2c40a6190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0005fd0 tx=0x7fd2c0005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.166+0000 7fd2cf7fe700 1 -- 172.21.15.52:0/1058831306 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd2c8009940 con 0x7fd2d81014d0 2024-04-02T13:52:35.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.377+0000 7fd2dca57700 1 -- 172.21.15.52:0/1058831306 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fd2d819a0e0 con 0x7fd2d81014d0 2024-04-02T13:52:35.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.377+0000 7fd2cf7fe700 1 -- 172.21.15.52:0/1058831306 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd2c809c7d0 con 0x7fd2d81014d0 2024-04-02T13:52:35.381 INFO:teuthology.orchestra.run.smithi052.stdout:34359738456 2024-04-02T13:52:35.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.380+0000 7fd2dca57700 1 -- 172.21.15.52:0/1058831306 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fd2c40a3cd0 msgr2=0x7fd2c40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:35.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.380+0000 7fd2dca57700 1 --2- 172.21.15.52:0/1058831306 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fd2c40a3cd0 0x7fd2c40a6190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0005fd0 tx=0x7fd2c0005ee0 comp rx=0 tx=0).stop 2024-04-02T13:52:35.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.380+0000 7fd2dca57700 1 -- 172.21.15.52:0/1058831306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd2d81014d0 msgr2=0x7fd2d8198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:35.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.380+0000 7fd2dca57700 1 --2- 172.21.15.52:0/1058831306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd2d81014d0 0x7fd2d8198d80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800d980 tx=0x7fd2c800dd40 comp rx=0 tx=0).stop 2024-04-02T13:52:35.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.380+0000 7fd2dca57700 1 -- 172.21.15.52:0/1058831306 shutdown_connections 2024-04-02T13:52:35.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.380+0000 7fd2dca57700 1 --2- 172.21.15.52:0/1058831306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2d8106480 0x7fd2d819d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:35.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.380+0000 7fd2dca57700 1 --2- 172.21.15.52:0/1058831306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd2d81014d0 0x7fd2d8198d80 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:35.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.380+0000 7fd2dca57700 1 --2- 172.21.15.52:0/1058831306 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fd2c40a3cd0 0x7fd2c40a6190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:35.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.380+0000 7fd2dca57700 1 --2- 172.21.15.52:0/1058831306 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2d8101e90 0x7fd2d81992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:35.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.380+0000 7fd2dca57700 1 -- 172.21.15.52:0/1058831306 >> 172.21.15.52:0/1058831306 conn(0x7fd2d8078500 msgr2=0x7fd2d81053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:35.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.380+0000 7fd2dca57700 1 -- 172.21.15.52:0/1058831306 shutdown_connections 2024-04-02T13:52:35.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.380+0000 7fd2dca57700 1 -- 172.21.15.52:0/1058831306 wait complete. 2024-04-02T13:52:35.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.401+0000 7f8239c12700 1 -- 172.21.15.52:0/2822833748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8234102fe0 msgr2=0x7f823410f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:35.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.401+0000 7f8239c12700 1 --2- 172.21.15.52:0/2822833748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8234102fe0 0x7f823410f490 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f822c00b0a0 tx=0x7f822c00b3b0 comp rx=0 tx=0).stop 2024-04-02T13:52:35.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.401+0000 7f8239c12700 1 -- 172.21.15.52:0/2822833748 shutdown_connections 2024-04-02T13:52:35.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.401+0000 7f8239c12700 1 --2- 172.21.15.52:0/2822833748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8234102fe0 0x7f823410f490 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:35.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.401+0000 7f8239c12700 1 --2- 172.21.15.52:0/2822833748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8234107520 0x7f8234107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:35.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.401+0000 7f8239c12700 1 --2- 172.21.15.52:0/2822833748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8234102640 0x7f8234102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:35.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.401+0000 7f8239c12700 1 -- 172.21.15.52:0/2822833748 >> 172.21.15.52:0/2822833748 conn(0x7f8234079720 msgr2=0x7f8234079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:35.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.401+0000 7f8239c12700 1 -- 172.21.15.52:0/2822833748 shutdown_connections 2024-04-02T13:52:35.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.401+0000 7f8239c12700 1 -- 172.21.15.52:0/2822833748 wait complete. 2024-04-02T13:52:35.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.402+0000 7f8239c12700 1 Processor -- start 2024-04-02T13:52:35.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.402+0000 7f8239c12700 1 -- start start 2024-04-02T13:52:35.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.402+0000 7f8239c12700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8234102640 0x7f823419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:35.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.402+0000 7f8239c12700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8234102fe0 0x7f823419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:35.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.402+0000 7f8239c12700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8234107520 0x7f8234198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:35.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.402+0000 7f8239c12700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f82341128b0 con 0x7f8234107520 2024-04-02T13:52:35.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.402+0000 7f8239c12700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8234112730 con 0x7f8234102fe0 2024-04-02T13:52:35.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.402+0000 7f8239c12700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8234112a30 con 0x7f8234102640 2024-04-02T13:52:35.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.402+0000 7f8233fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8234107520 0x7f8234198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:35.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.403+0000 7f8233fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8234107520 0x7f8234198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56910/0 (socket says 172.21.15.52:56910) 2024-04-02T13:52:35.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.403+0000 7f8233fff700 1 -- 172.21.15.52:0/845544347 learned_addr learned my addr 172.21.15.52:0/845544347 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:35.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.403+0000 7f82337fe700 1 --2- 172.21.15.52:0/845544347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8234102640 0x7f823419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:35.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.403+0000 7f8233fff700 1 -- 172.21.15.52:0/845544347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8234102640 msgr2=0x7f823419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:35.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.403+0000 7f8233fff700 1 --2- 172.21.15.52:0/845544347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8234102640 0x7f823419e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:35.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.403+0000 7f8232ffd700 1 --2- 172.21.15.52:0/845544347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8234102fe0 0x7f823419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:35.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.403+0000 7f8233fff700 1 -- 172.21.15.52:0/845544347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8234102fe0 msgr2=0x7f823419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:35.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.403+0000 7f8233fff700 1 --2- 172.21.15.52:0/845544347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8234102fe0 0x7f823419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:35.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.403+0000 7f8233fff700 1 -- 172.21.15.52:0/845544347 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8234199250 con 0x7f8234107520 2024-04-02T13:52:35.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.403+0000 7f82337fe700 1 --2- 172.21.15.52:0/845544347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8234102640 0x7f823419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:52:35.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.403+0000 7f8232ffd700 1 --2- 172.21.15.52:0/845544347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8234102fe0 0x7f823419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:52:35.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.403+0000 7f8233fff700 1 --2- 172.21.15.52:0/845544347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8234107520 0x7f8234198af0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f822c0062a0 tx=0x7f822c004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:35.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.403+0000 7f8230ff9700 1 -- 172.21.15.52:0/845544347 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c021070 con 0x7f8234107520 2024-04-02T13:52:35.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.403+0000 7f8230ff9700 1 -- 172.21.15.52:0/845544347 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f822c007c80 con 0x7f8234107520 2024-04-02T13:52:35.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.403+0000 7f8239c12700 1 -- 172.21.15.52:0/845544347 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82341994e0 con 0x7f8234107520 2024-04-02T13:52:35.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.403+0000 7f8230ff9700 1 -- 172.21.15.52:0/845544347 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c01d450 con 0x7f8234107520 2024-04-02T13:52:35.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.403+0000 7f8239c12700 1 -- 172.21.15.52:0/845544347 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82341a5800 con 0x7f8234107520 2024-04-02T13:52:35.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.404+0000 7f8239c12700 1 -- 172.21.15.52:0/845544347 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82340692b0 con 0x7f8234107520 2024-04-02T13:52:35.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.406+0000 7f8230ff9700 1 -- 172.21.15.52:0/845544347 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f822c004000 con 0x7f8234107520 2024-04-02T13:52:35.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.406+0000 7f8230ff9700 1 --2- 172.21.15.52:0/845544347 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f821c0a3cd0 0x7f821c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:35.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.406+0000 7f8230ff9700 1 -- 172.21.15.52:0/845544347 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f822c00f020 con 0x7f8234107520 2024-04-02T13:52:35.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.406+0000 7f82337fe700 1 --2- 172.21.15.52:0/845544347 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f821c0a3cd0 0x7f821c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:35.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.407+0000 7f82337fe700 1 --2- 172.21.15.52:0/845544347 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f821c0a3cd0 0x7f821c0a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8224005e10 tx=0x7f8224005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:35.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.410+0000 7f8230ff9700 1 -- 172.21.15.52:0/845544347 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f822c02a030 con 0x7f8234107520 2024-04-02T13:52:35.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.623+0000 7f8239c12700 1 -- 172.21.15.52:0/845544347 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f823402dd30 con 0x7f8234107520 2024-04-02T13:52:35.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.623+0000 7f8230ff9700 1 -- 172.21.15.52:0/845544347 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f822c09e500 con 0x7f8234107520 2024-04-02T13:52:35.627 INFO:teuthology.orchestra.run.smithi052.stdout:141733920836 2024-04-02T13:52:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.625+0000 7f8239c12700 1 -- 172.21.15.52:0/845544347 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f821c0a3cd0 msgr2=0x7f821c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.625+0000 7f8239c12700 1 --2- 172.21.15.52:0/845544347 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f821c0a3cd0 0x7f821c0a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8224005e10 tx=0x7f8224005d20 comp rx=0 tx=0).stop 2024-04-02T13:52:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.625+0000 7f8239c12700 1 -- 172.21.15.52:0/845544347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8234107520 msgr2=0x7f8234198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.625+0000 7f8239c12700 1 --2- 172.21.15.52:0/845544347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8234107520 0x7f8234198af0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f822c0062a0 tx=0x7f822c004e20 comp rx=0 tx=0).stop 2024-04-02T13:52:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.625+0000 7f8239c12700 1 -- 172.21.15.52:0/845544347 shutdown_connections 2024-04-02T13:52:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.625+0000 7f8239c12700 1 --2- 172.21.15.52:0/845544347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8234107520 0x7f8234198af0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.625+0000 7f8239c12700 1 --2- 172.21.15.52:0/845544347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8234102640 0x7f823419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:35.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.625+0000 7f8239c12700 1 --2- 172.21.15.52:0/845544347 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f821c0a3cd0 0x7f821c0a6190 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:35.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.625+0000 7f8239c12700 1 --2- 172.21.15.52:0/845544347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8234102fe0 0x7f823419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:35.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.625+0000 7f8239c12700 1 -- 172.21.15.52:0/845544347 >> 172.21.15.52:0/845544347 conn(0x7f8234079720 msgr2=0x7f82341006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:35.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.625+0000 7f8239c12700 1 -- 172.21.15.52:0/845544347 shutdown_connections 2024-04-02T13:52:35.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:35.625+0000 7f8239c12700 1 -- 172.21.15.52:0/845544347 wait complete. 2024-04-02T13:52:36.472 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:36 smithi088 ceph-mon[36528]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:52:36.472 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:36 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1058831306' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-02T13:52:36.472 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:36 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/845544347' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-02T13:52:36.488 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:36 smithi052 ceph-mon[32104]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:52:36.488 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:36 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1058831306' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-02T13:52:36.488 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:36 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/845544347' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-02T13:52:36.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:36 smithi189 ceph-mon[36683]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:52:36.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:36 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1058831306' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-02T13:52:36.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:36 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/845544347' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-02T13:52:36.640 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920830 got 141733920836 for osd.4 2024-04-02T13:52:36.640 DEBUG:teuthology.parallel:result is None 2024-04-02T13:52:37.990 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738451 got 34359738456 for osd.0 2024-04-02T13:52:37.990 DEBUG:teuthology.parallel:result is None 2024-04-02T13:52:38.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:38 smithi052 ceph-mon[32104]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-02T13:52:38.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:38 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:52:38.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:38 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:52:38.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:38 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2070725338' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-02T13:52:38.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:38 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:52:38.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:38 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:52:38.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:38 smithi052 ceph-mon[32104]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:52:38.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:38 smithi189 ceph-mon[36683]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-02T13:52:38.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:38 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:52:38.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:38 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:52:38.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:38 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2070725338' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-02T13:52:38.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:38 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:52:38.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:38 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:52:38.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:38 smithi189 ceph-mon[36683]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:52:38.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:38 smithi088 ceph-mon[36528]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-02T13:52:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:38 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:52:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:38 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:52:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:38 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2070725338' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-02T13:52:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:38 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:52:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:38 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:52:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:38 smithi088 ceph-mon[36528]: from='mgr.14152 172.21.15.52:0/1577593514' entity='mgr.x' 2024-04-02T13:52:39.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.621+0000 7fc4b4f93700 1 -- 172.21.15.52:0/3271924721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4b006b4f0 msgr2=0x7fc4b010f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:39.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.621+0000 7fc4b4f93700 1 --2- 172.21.15.52:0/3271924721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4b006b4f0 0x7fc4b010f450 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800b0d0 tx=0x7fc4a800b3e0 comp rx=0 tx=0).stop 2024-04-02T13:52:39.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.621+0000 7fc4b4f93700 1 -- 172.21.15.52:0/3271924721 shutdown_connections 2024-04-02T13:52:39.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.621+0000 7fc4b4f93700 1 --2- 172.21.15.52:0/3271924721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4b006b4f0 0x7fc4b010f450 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:39.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.621+0000 7fc4b4f93700 1 --2- 172.21.15.52:0/3271924721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4b0108630 0x7fc4b0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:39.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.621+0000 7fc4b4f93700 1 --2- 172.21.15.52:0/3271924721 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4b006ab30 0x7fc4b006afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:39.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.621+0000 7fc4b4f93700 1 -- 172.21.15.52:0/3271924721 >> 172.21.15.52:0/3271924721 conn(0x7fc4b0077540 msgr2=0x7fc4b0077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:39.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.621+0000 7fc4b4f93700 1 -- 172.21.15.52:0/3271924721 shutdown_connections 2024-04-02T13:52:39.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.621+0000 7fc4b4f93700 1 -- 172.21.15.52:0/3271924721 wait complete. 2024-04-02T13:52:39.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.622+0000 7fc4b4f93700 1 Processor -- start 2024-04-02T13:52:39.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.622+0000 7fc4b4f93700 1 -- start start 2024-04-02T13:52:39.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.622+0000 7fc4b4f93700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4b006ab30 0x7fc4b0198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:39.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.622+0000 7fc4b4f93700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4b006b4f0 0x7fc4b01992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:39.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.622+0000 7fc4b4f93700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4b0108630 0x7fc4b019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:39.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.622+0000 7fc4b4f93700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b01128b0 con 0x7fc4b006ab30 2024-04-02T13:52:39.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.622+0000 7fc4b4f93700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b0112730 con 0x7fc4b006b4f0 2024-04-02T13:52:39.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.622+0000 7fc4b4f93700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b0112a30 con 0x7fc4b0108630 2024-04-02T13:52:39.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.623+0000 7fc4ae59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4b006ab30 0x7fc4b0198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:39.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.623+0000 7fc4ae59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4b006ab30 0x7fc4b0198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56928/0 (socket says 172.21.15.52:56928) 2024-04-02T13:52:39.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.623+0000 7fc4ae59c700 1 -- 172.21.15.52:0/2070725338 learned_addr learned my addr 172.21.15.52:0/2070725338 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:39.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.623+0000 7fc4add9b700 1 --2- 172.21.15.52:0/2070725338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4b006b4f0 0x7fc4b01992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:39.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.623+0000 7fc4aed9d700 1 --2- 172.21.15.52:0/2070725338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4b0108630 0x7fc4b019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:39.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.623+0000 7fc4ae59c700 1 -- 172.21.15.52:0/2070725338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4b0108630 msgr2=0x7fc4b019d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:39.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.623+0000 7fc4ae59c700 1 --2- 172.21.15.52:0/2070725338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4b0108630 0x7fc4b019d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:39.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.623+0000 7fc4ae59c700 1 -- 172.21.15.52:0/2070725338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4b006b4f0 msgr2=0x7fc4b01992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:39.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.623+0000 7fc4ae59c700 1 --2- 172.21.15.52:0/2070725338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4b006b4f0 0x7fc4b01992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:39.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.623+0000 7fc4ae59c700 1 -- 172.21.15.52:0/2070725338 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b019dd90 con 0x7fc4b006ab30 2024-04-02T13:52:39.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.623+0000 7fc4ae59c700 1 --2- 172.21.15.52:0/2070725338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4b006ab30 0x7fc4b0198d80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000d950 tx=0x7fc4a000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:39.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.623+0000 7fc4a77fe700 1 -- 172.21.15.52:0/2070725338 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0016070 con 0x7fc4b006ab30 2024-04-02T13:52:39.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.623+0000 7fc4a77fe700 1 -- 172.21.15.52:0/2070725338 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4a000fd70 con 0x7fc4b006ab30 2024-04-02T13:52:39.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.623+0000 7fc4b4f93700 1 -- 172.21.15.52:0/2070725338 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b019e080 con 0x7fc4b006ab30 2024-04-02T13:52:39.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.623+0000 7fc4a77fe700 1 -- 172.21.15.52:0/2070725338 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a000e610 con 0x7fc4b006ab30 2024-04-02T13:52:39.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.623+0000 7fc4b4f93700 1 -- 172.21.15.52:0/2070725338 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b01a5900 con 0x7fc4b006ab30 2024-04-02T13:52:39.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.624+0000 7fc4b4f93700 1 -- 172.21.15.52:0/2070725338 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4b00692b0 con 0x7fc4b006ab30 2024-04-02T13:52:39.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.626+0000 7fc4a77fe700 1 -- 172.21.15.52:0/2070725338 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc4a0019070 con 0x7fc4b006ab30 2024-04-02T13:52:39.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.626+0000 7fc4a77fe700 1 --2- 172.21.15.52:0/2070725338 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc49c0a3d50 0x7fc49c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:39.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.626+0000 7fc4a77fe700 1 -- 172.21.15.52:0/2070725338 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc4a00d0270 con 0x7fc4b006ab30 2024-04-02T13:52:39.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.626+0000 7fc4add9b700 1 --2- 172.21.15.52:0/2070725338 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc49c0a3d50 0x7fc49c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:39.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.627+0000 7fc4add9b700 1 --2- 172.21.15.52:0/2070725338 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc49c0a3d50 0x7fc49c0a6210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0077c00 tx=0x7fc49800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:39.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.630+0000 7fc4a77fe700 1 -- 172.21.15.52:0/2070725338 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc4a009c790 con 0x7fc4b006ab30 2024-04-02T13:52:39.066 INFO:teuthology.orchestra.run.smithi052.stdout:77309411407 2024-04-02T13:52:39.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.839+0000 7fc4b4f93700 1 -- 172.21.15.52:0/2070725338 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc4b019a0e0 con 0x7fc4b006ab30 2024-04-02T13:52:39.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.840+0000 7fc4a77fe700 1 -- 172.21.15.52:0/2070725338 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc4a009c090 con 0x7fc4b006ab30 2024-04-02T13:52:39.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.842+0000 7fc4b4f93700 1 -- 172.21.15.52:0/2070725338 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc49c0a3d50 msgr2=0x7fc49c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:39.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.842+0000 7fc4b4f93700 1 --2- 172.21.15.52:0/2070725338 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc49c0a3d50 0x7fc49c0a6210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0077c00 tx=0x7fc49800b410 comp rx=0 tx=0).stop 2024-04-02T13:52:39.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.842+0000 7fc4b4f93700 1 -- 172.21.15.52:0/2070725338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4b006ab30 msgr2=0x7fc4b0198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:39.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.842+0000 7fc4b4f93700 1 --2- 172.21.15.52:0/2070725338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4b006ab30 0x7fc4b0198d80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000d950 tx=0x7fc4a000dc60 comp rx=0 tx=0).stop 2024-04-02T13:52:39.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.842+0000 7fc4b4f93700 1 -- 172.21.15.52:0/2070725338 shutdown_connections 2024-04-02T13:52:39.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.842+0000 7fc4b4f93700 1 --2- 172.21.15.52:0/2070725338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4b0108630 0x7fc4b019d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:39.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.842+0000 7fc4b4f93700 1 --2- 172.21.15.52:0/2070725338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4b006ab30 0x7fc4b0198d80 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:39.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.842+0000 7fc4b4f93700 1 --2- 172.21.15.52:0/2070725338 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc49c0a3d50 0x7fc49c0a6210 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:39.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.842+0000 7fc4b4f93700 1 --2- 172.21.15.52:0/2070725338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4b006b4f0 0x7fc4b01992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:39.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.842+0000 7fc4b4f93700 1 -- 172.21.15.52:0/2070725338 >> 172.21.15.52:0/2070725338 conn(0x7fc4b0077540 msgr2=0x7fc4b01002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:39.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.843+0000 7fc4b4f93700 1 -- 172.21.15.52:0/2070725338 shutdown_connections 2024-04-02T13:52:39.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:37.843+0000 7fc4b4f93700 1 -- 172.21.15.52:0/2070725338 wait complete. 2024-04-02T13:52:39.723 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411400 got 77309411407 for osd.2 2024-04-02T13:52:39.723 DEBUG:teuthology.parallel:result is None 2024-04-02T13:52:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.951+0000 7efed604e700 1 -- 172.21.15.52:0/861992714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efed01014f0 msgr2=0x7efed0101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.951+0000 7efed604e700 1 --2- 172.21.15.52:0/861992714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efed01014f0 0x7efed0101970 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efec4009990 tx=0x7efec4009ca0 comp rx=0 tx=0).stop 2024-04-02T13:52:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.951+0000 7efed604e700 1 -- 172.21.15.52:0/861992714 shutdown_connections 2024-04-02T13:52:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.951+0000 7efed604e700 1 --2- 172.21.15.52:0/861992714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efed0101eb0 0x7efed010e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.951+0000 7efed604e700 1 --2- 172.21.15.52:0/861992714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efed0107500 0x7efed01078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.951+0000 7efed604e700 1 --2- 172.21.15.52:0/861992714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efed01014f0 0x7efed0101970 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.951+0000 7efed604e700 1 -- 172.21.15.52:0/861992714 >> 172.21.15.52:0/861992714 conn(0x7efed0078420 msgr2=0x7efed0078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.951+0000 7efed604e700 1 -- 172.21.15.52:0/861992714 shutdown_connections 2024-04-02T13:52:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.951+0000 7efed604e700 1 -- 172.21.15.52:0/861992714 wait complete. 2024-04-02T13:52:39.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.952+0000 7efed604e700 1 Processor -- start 2024-04-02T13:52:39.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.952+0000 7efed604e700 1 -- start start 2024-04-02T13:52:39.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.953+0000 7efed604e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efed01014f0 0x7efed0199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:39.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.953+0000 7efed604e700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efed0101eb0 0x7efed019a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:39.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.953+0000 7efed604e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efed0107500 0x7efed01a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:39.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.953+0000 7efed604e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efed0111770 con 0x7efed01014f0 2024-04-02T13:52:39.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.953+0000 7efed604e700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efed01115f0 con 0x7efed0107500 2024-04-02T13:52:39.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.953+0000 7efed604e700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efed01118f0 con 0x7efed0101eb0 2024-04-02T13:52:39.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.953+0000 7efecf7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efed01014f0 0x7efed0199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:39.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.953+0000 7efecf7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efed01014f0 0x7efed0199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56940/0 (socket says 172.21.15.52:56940) 2024-04-02T13:52:39.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.953+0000 7efecf7fe700 1 -- 172.21.15.52:0/2951567771 learned_addr learned my addr 172.21.15.52:0/2951567771 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:39.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.953+0000 7efeceffd700 1 --2- 172.21.15.52:0/2951567771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efed0101eb0 0x7efed019a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:39.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.953+0000 7efecffff700 1 --2- 172.21.15.52:0/2951567771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efed0107500 0x7efed01a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:39.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.953+0000 7efecf7fe700 1 -- 172.21.15.52:0/2951567771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efed0101eb0 msgr2=0x7efed019a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:39.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.953+0000 7efecf7fe700 1 --2- 172.21.15.52:0/2951567771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efed0101eb0 0x7efed019a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:39.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.953+0000 7efecf7fe700 1 -- 172.21.15.52:0/2951567771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efed0107500 msgr2=0x7efed01a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:39.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.953+0000 7efecf7fe700 1 --2- 172.21.15.52:0/2951567771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efed0107500 0x7efed01a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:39.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.953+0000 7efecf7fe700 1 -- 172.21.15.52:0/2951567771 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed01a50f0 con 0x7efed01014f0 2024-04-02T13:52:39.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.953+0000 7efecffff700 1 --2- 172.21.15.52:0/2951567771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efed0107500 0x7efed01a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:52:39.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.953+0000 7efecf7fe700 1 --2- 172.21.15.52:0/2951567771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efed01014f0 0x7efed0199b30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efec000d950 tx=0x7efec000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:39.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.953+0000 7efeccff9700 1 -- 172.21.15.52:0/2951567771 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0016070 con 0x7efed01014f0 2024-04-02T13:52:39.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.953+0000 7efeccff9700 1 -- 172.21.15.52:0/2951567771 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efec0009e20 con 0x7efed01014f0 2024-04-02T13:52:39.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.953+0000 7efed604e700 1 -- 172.21.15.52:0/2951567771 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed01a5320 con 0x7efed01014f0 2024-04-02T13:52:39.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.953+0000 7efeccff9700 1 -- 172.21.15.52:0/2951567771 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec000f620 con 0x7efed01014f0 2024-04-02T13:52:39.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.954+0000 7efed604e700 1 -- 172.21.15.52:0/2951567771 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed01a5880 con 0x7efed01014f0 2024-04-02T13:52:39.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.954+0000 7efed604e700 1 -- 172.21.15.52:0/2951567771 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed00692b0 con 0x7efed01014f0 2024-04-02T13:52:39.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.956+0000 7efeccff9700 1 -- 172.21.15.52:0/2951567771 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efec0019070 con 0x7efed01014f0 2024-04-02T13:52:39.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.956+0000 7efeccff9700 1 --2- 172.21.15.52:0/2951567771 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7efeb80a3d50 0x7efeb80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:39.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.956+0000 7efeceffd700 1 --2- 172.21.15.52:0/2951567771 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7efeb80a3d50 0x7efeb80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:39.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.956+0000 7efeccff9700 1 -- 172.21.15.52:0/2951567771 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7efec00d0320 con 0x7efed01014f0 2024-04-02T13:52:39.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.957+0000 7efeceffd700 1 --2- 172.21.15.52:0/2951567771 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7efeb80a3d50 0x7efeb80a6210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efec400b5c0 tx=0x7efec4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:39.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:39.961+0000 7efeccff9700 1 -- 172.21.15.52:0/2951567771 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7efec009c8e0 con 0x7efed01014f0 2024-04-02T13:52:40.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:40.170+0000 7efed604e700 1 -- 172.21.15.52:0/2951567771 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7efed002dd30 con 0x7efed01014f0 2024-04-02T13:52:40.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:40.170+0000 7efeccff9700 1 -- 172.21.15.52:0/2951567771 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7efec009c1e0 con 0x7efed01014f0 2024-04-02T13:52:40.174 INFO:teuthology.orchestra.run.smithi052.stdout:197568495674 2024-04-02T13:52:40.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:40.172+0000 7efed604e700 1 -- 172.21.15.52:0/2951567771 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7efeb80a3d50 msgr2=0x7efeb80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:40.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:40.172+0000 7efed604e700 1 --2- 172.21.15.52:0/2951567771 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7efeb80a3d50 0x7efeb80a6210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efec400b5c0 tx=0x7efec4005960 comp rx=0 tx=0).stop 2024-04-02T13:52:40.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:40.172+0000 7efed604e700 1 -- 172.21.15.52:0/2951567771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efed01014f0 msgr2=0x7efed0199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:40.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:40.172+0000 7efed604e700 1 --2- 172.21.15.52:0/2951567771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efed01014f0 0x7efed0199b30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efec000d950 tx=0x7efec000dc60 comp rx=0 tx=0).stop 2024-04-02T13:52:40.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:40.172+0000 7efed604e700 1 -- 172.21.15.52:0/2951567771 shutdown_connections 2024-04-02T13:52:40.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:40.172+0000 7efed604e700 1 --2- 172.21.15.52:0/2951567771 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7efeb80a3d50 0x7efeb80a6210 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:40.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:40.172+0000 7efed604e700 1 --2- 172.21.15.52:0/2951567771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efed0107500 0x7efed01a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:40.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:40.172+0000 7efed604e700 1 --2- 172.21.15.52:0/2951567771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efed01014f0 0x7efed0199b30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:40.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:40.172+0000 7efed604e700 1 --2- 172.21.15.52:0/2951567771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efed0101eb0 0x7efed019a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:40.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:40.172+0000 7efed604e700 1 -- 172.21.15.52:0/2951567771 >> 172.21.15.52:0/2951567771 conn(0x7efed0078420 msgr2=0x7efed0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:40.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:40.173+0000 7efed604e700 1 -- 172.21.15.52:0/2951567771 shutdown_connections 2024-04-02T13:52:40.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:40.173+0000 7efed604e700 1 -- 172.21.15.52:0/2951567771 wait complete. 2024-04-02T13:52:40.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:40 smithi189 ceph-mon[36683]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:52:40.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:40 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2951567771' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-02T13:52:40.833 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:40 smithi052 ceph-mon[32104]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:52:40.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:40 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2951567771' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-02T13:52:40.862 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495668 got 197568495674 for osd.6 2024-04-02T13:52:40.862 DEBUG:teuthology.parallel:result is None 2024-04-02T13:52:40.863 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-02T13:52:40.863 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph pg dump --format=json 2024-04-02T13:52:40.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:40 smithi088 ceph-mon[36528]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:52:40.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:40 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2951567771' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-02T13:52:41.113 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:52:42.580 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:42 smithi052 ceph-mon[32104]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-02T13:52:42.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:42 smithi189 ceph-mon[36683]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-02T13:52:42.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.815+0000 7fb751527700 1 -- 172.21.15.52:0/1416090903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb74c102130 msgr2=0x7fb74c106150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:42.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.815+0000 7fb751527700 1 --2- 172.21.15.52:0/1416090903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb74c102130 0x7fb74c106150 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb73c0099c0 tx=0x7fb73c009cd0 comp rx=0 tx=0).stop 2024-04-02T13:52:42.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.815+0000 7fb751527700 1 -- 172.21.15.52:0/1416090903 shutdown_connections 2024-04-02T13:52:42.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.815+0000 7fb751527700 1 --2- 172.21.15.52:0/1416090903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb74c106880 0x7fb74c10e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:42.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.815+0000 7fb751527700 1 --2- 172.21.15.52:0/1416090903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb74c101780 0x7fb74c101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:42.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.815+0000 7fb751527700 1 --2- 172.21.15.52:0/1416090903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb74c102130 0x7fb74c106150 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:42.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.815+0000 7fb751527700 1 -- 172.21.15.52:0/1416090903 >> 172.21.15.52:0/1416090903 conn(0x7fb74c077510 msgr2=0x7fb74c077920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:42.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.815+0000 7fb751527700 1 -- 172.21.15.52:0/1416090903 shutdown_connections 2024-04-02T13:52:42.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.816+0000 7fb751527700 1 -- 172.21.15.52:0/1416090903 wait complete. 2024-04-02T13:52:42.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.816+0000 7fb751527700 1 Processor -- start 2024-04-02T13:52:42.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.816+0000 7fb751527700 1 -- start start 2024-04-02T13:52:42.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.816+0000 7fb751527700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb74c101780 0x7fb74c103160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:42.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.816+0000 7fb751527700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb74c102130 0x7fb74c1036a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:42.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.816+0000 7fb751527700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb74c106880 0x7fb74c103be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:42.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.817+0000 7fb751527700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb74c111810 con 0x7fb74c106880 2024-04-02T13:52:42.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.817+0000 7fb751527700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb74c111690 con 0x7fb74c102130 2024-04-02T13:52:42.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.817+0000 7fb751527700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb74c111990 con 0x7fb74c101780 2024-04-02T13:52:42.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.817+0000 7fb74b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb74c106880 0x7fb74c103be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:42.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.817+0000 7fb74b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb74c106880 0x7fb74c103be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59912/0 (socket says 172.21.15.52:59912) 2024-04-02T13:52:42.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.817+0000 7fb74b7fe700 1 -- 172.21.15.52:0/1259377684 learned_addr learned my addr 172.21.15.52:0/1259377684 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:42.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.817+0000 7fb74a7fc700 1 --2- 172.21.15.52:0/1259377684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb74c102130 0x7fb74c1036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:42.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.817+0000 7fb74affd700 1 --2- 172.21.15.52:0/1259377684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb74c101780 0x7fb74c103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:42.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.817+0000 7fb74b7fe700 1 -- 172.21.15.52:0/1259377684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb74c101780 msgr2=0x7fb74c103160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:42.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.817+0000 7fb74b7fe700 1 --2- 172.21.15.52:0/1259377684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb74c101780 0x7fb74c103160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:42.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.817+0000 7fb74b7fe700 1 -- 172.21.15.52:0/1259377684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb74c102130 msgr2=0x7fb74c1036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:42.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.817+0000 7fb74b7fe700 1 --2- 172.21.15.52:0/1259377684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb74c102130 0x7fb74c1036a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:42.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.817+0000 7fb74b7fe700 1 -- 172.21.15.52:0/1259377684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb74c1a7280 con 0x7fb74c106880 2024-04-02T13:52:42.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.817+0000 7fb74affd700 1 --2- 172.21.15.52:0/1259377684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb74c101780 0x7fb74c103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:52:42.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.817+0000 7fb74b7fe700 1 --2- 172.21.15.52:0/1259377684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb74c106880 0x7fb74c103be0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb744007ee0 tx=0x7fb74400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:42.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.817+0000 7fb743fff700 1 -- 172.21.15.52:0/1259377684 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74400dbd0 con 0x7fb74c106880 2024-04-02T13:52:42.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.817+0000 7fb751527700 1 -- 172.21.15.52:0/1259377684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb74c1a7570 con 0x7fb74c106880 2024-04-02T13:52:42.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.817+0000 7fb743fff700 1 -- 172.21.15.52:0/1259377684 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb74400dd70 con 0x7fb74c106880 2024-04-02T13:52:42.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.817+0000 7fb743fff700 1 -- 172.21.15.52:0/1259377684 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb744005a60 con 0x7fb74c106880 2024-04-02T13:52:42.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.817+0000 7fb751527700 1 -- 172.21.15.52:0/1259377684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb74c1a7a30 con 0x7fb74c106880 2024-04-02T13:52:42.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.818+0000 7fb751527700 1 -- 172.21.15.52:0/1259377684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb74c102c10 con 0x7fb74c106880 2024-04-02T13:52:42.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.819+0000 7fb743fff700 1 -- 172.21.15.52:0/1259377684 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb74401a020 con 0x7fb74c106880 2024-04-02T13:52:42.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.820+0000 7fb743fff700 1 --2- 172.21.15.52:0/1259377684 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb7380a3cd0 0x7fb7380a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:42.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.820+0000 7fb74affd700 1 --2- 172.21.15.52:0/1259377684 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb7380a3cd0 0x7fb7380a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:42.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.820+0000 7fb743fff700 1 -- 172.21.15.52:0/1259377684 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb7440d0060 con 0x7fb74c106880 2024-04-02T13:52:42.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.821+0000 7fb74affd700 1 --2- 172.21.15.52:0/1259377684 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb7380a3cd0 0x7fb7380a6190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb7340099d0 tx=0x7fb734008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:42.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:42.825+0000 7fb743fff700 1 -- 172.21.15.52:0/1259377684 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb744010070 con 0x7fb74c106880 2024-04-02T13:52:42.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:42 smithi088 ceph-mon[36528]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-02T13:52:43.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:43.029+0000 7fb751527700 1 -- 172.21.15.52:0/1259377684 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb74c062a30 con 0x7fb7380a3cd0 2024-04-02T13:52:43.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:43.039+0000 7fb743fff700 1 -- 172.21.15.52:0/1259377684 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340419 (secure 0 0 0) 0x7fb74c062a30 con 0x7fb7380a3cd0 2024-04-02T13:52:43.047 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:52:43.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:43.042+0000 7fb751527700 1 -- 172.21.15.52:0/1259377684 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb7380a3cd0 msgr2=0x7fb7380a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:43.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:43.042+0000 7fb751527700 1 --2- 172.21.15.52:0/1259377684 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb7380a3cd0 0x7fb7380a6190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb7340099d0 tx=0x7fb734008040 comp rx=0 tx=0).stop 2024-04-02T13:52:43.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:43.042+0000 7fb751527700 1 -- 172.21.15.52:0/1259377684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb74c106880 msgr2=0x7fb74c103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:43.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:43.042+0000 7fb751527700 1 --2- 172.21.15.52:0/1259377684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb74c106880 0x7fb74c103be0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb744007ee0 tx=0x7fb74400d3b0 comp rx=0 tx=0).stop 2024-04-02T13:52:43.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:43.042+0000 7fb751527700 1 -- 172.21.15.52:0/1259377684 shutdown_connections 2024-04-02T13:52:43.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:43.042+0000 7fb751527700 1 --2- 172.21.15.52:0/1259377684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb74c106880 0x7fb74c103be0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:43.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:43.042+0000 7fb751527700 1 --2- 172.21.15.52:0/1259377684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb74c101780 0x7fb74c103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:43.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:43.042+0000 7fb751527700 1 --2- 172.21.15.52:0/1259377684 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb7380a3cd0 0x7fb7380a6190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:43.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:43.042+0000 7fb751527700 1 --2- 172.21.15.52:0/1259377684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb74c102130 0x7fb74c1036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:43.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:43.042+0000 7fb751527700 1 -- 172.21.15.52:0/1259377684 >> 172.21.15.52:0/1259377684 conn(0x7fb74c077510 msgr2=0x7fb74c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:43.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:43.042+0000 7fb751527700 1 -- 172.21.15.52:0/1259377684 shutdown_connections 2024-04-02T13:52:43.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:43.042+0000 7fb751527700 1 -- 172.21.15.52:0/1259377684 wait complete. 2024-04-02T13:52:43.051 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-04-02T13:52:43.898 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-04-02T13:52:42.762677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659748,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":884,"num_read_kb":568,"num_write":423,"num_write_kb":3080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":405216,"kb_used_data":14384,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124536608,"statfs":{"total":1151940427776,"available":1151525486592,"internally_reserved":0,"allocated":14729216,"data_stored":8034242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":400012602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003810"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.478629+0000","last_change":"2024-04-02T13:51:01.253563+0000","last_active":"2024-04-02T13:51:56.478629+0000","last_peered":"2024-04-02T13:51:56.478629+0000","last_clean":"2024-04-02T13:51:56.478629+0000","last_became_active":"2024-04-02T13:51:01.253185+0000","last_became_peered":"2024-04-02T13:51:01.253185+0000","last_unstale":"2024-04-02T13:51:56.478629+0000","last_undegraded":"2024-04-02T13:51:56.478629+0000","last_fullsized":"2024-04-02T13:51:56.478629+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:51:05.301880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662449+0000","last_change":"2024-04-02T13:51:01.256406+0000","last_active":"2024-04-02T13:51:56.662449+0000","last_peered":"2024-04-02T13:51:56.662449+0000","last_clean":"2024-04-02T13:51:56.662449+0000","last_became_active":"2024-04-02T13:51:01.256068+0000","last_became_peered":"2024-04-02T13:51:01.256068+0000","last_unstale":"2024-04-02T13:51:56.662449+0000","last_undegraded":"2024-04-02T13:51:56.662449+0000","last_fullsized":"2024-04-02T13:51:56.662449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:56:51.184932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662208+0000","last_change":"2024-04-02T13:51:01.265925+0000","last_active":"2024-04-02T13:51:56.662208+0000","last_peered":"2024-04-02T13:51:56.662208+0000","last_clean":"2024-04-02T13:51:56.662208+0000","last_became_active":"2024-04-02T13:51:01.265587+0000","last_became_peered":"2024-04-02T13:51:01.265587+0000","last_unstale":"2024-04-02T13:51:56.662208+0000","last_undegraded":"2024-04-02T13:51:56.662208+0000","last_fullsized":"2024-04-02T13:51:56.662208+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:40:59.426886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.483007+0000","last_change":"2024-04-02T13:51:56.482512+0000","last_active":"2024-04-02T13:51:56.483007+0000","last_peered":"2024-04-02T13:51:56.483007+0000","last_clean":"2024-04-02T13:51:56.483007+0000","last_became_active":"2024-04-02T13:51:56.481439+0000","last_became_peered":"2024-04-02T13:51:56.481439+0000","last_unstale":"2024-04-02T13:51:56.483007+0000","last_undegraded":"2024-04-02T13:51:56.483007+0000","last_fullsized":"2024-04-02T13:51:56.483007+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:23:16.969641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477134+0000","last_change":"2024-04-02T13:51:01.250934+0000","last_active":"2024-04-02T13:51:56.477134+0000","last_peered":"2024-04-02T13:51:56.477134+0000","last_clean":"2024-04-02T13:51:56.477134+0000","last_became_active":"2024-04-02T13:51:01.250546+0000","last_became_peered":"2024-04-02T13:51:01.250546+0000","last_unstale":"2024-04-02T13:51:56.477134+0000","last_undegraded":"2024-04-02T13:51:56.477134+0000","last_fullsized":"2024-04-02T13:51:56.477134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:34:17.839207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.476637+0000","last_change":"2024-04-02T13:51:01.256160+0000","last_active":"2024-04-02T13:51:56.476637+0000","last_peered":"2024-04-02T13:51:56.476637+0000","last_clean":"2024-04-02T13:51:56.476637+0000","last_became_active":"2024-04-02T13:51:01.255776+0000","last_became_peered":"2024-04-02T13:51:01.255776+0000","last_unstale":"2024-04-02T13:51:56.476637+0000","last_undegraded":"2024-04-02T13:51:56.476637+0000","last_fullsized":"2024-04-02T13:51:56.476637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:53:16.724662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663631+0000","last_change":"2024-04-02T13:51:01.263049+0000","last_active":"2024-04-02T13:51:56.663631+0000","last_peered":"2024-04-02T13:51:56.663631+0000","last_clean":"2024-04-02T13:51:56.663631+0000","last_became_active":"2024-04-02T13:51:01.262660+0000","last_became_peered":"2024-04-02T13:51:01.262660+0000","last_unstale":"2024-04-02T13:51:56.663631+0000","last_undegraded":"2024-04-02T13:51:56.663631+0000","last_fullsized":"2024-04-02T13:51:56.663631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:15:32.069188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.481755+0000","last_change":"2024-04-02T13:51:01.263064+0000","last_active":"2024-04-02T13:51:56.481755+0000","last_peered":"2024-04-02T13:51:56.481755+0000","last_clean":"2024-04-02T13:51:56.481755+0000","last_became_active":"2024-04-02T13:51:01.262706+0000","last_became_peered":"2024-04-02T13:51:01.262706+0000","last_unstale":"2024-04-02T13:51:56.481755+0000","last_undegraded":"2024-04-02T13:51:56.481755+0000","last_fullsized":"2024-04-02T13:51:56.481755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:46.680723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663840+0000","last_change":"2024-04-02T13:51:01.257347+0000","last_active":"2024-04-02T13:51:56.663840+0000","last_peered":"2024-04-02T13:51:56.663840+0000","last_clean":"2024-04-02T13:51:56.663840+0000","last_became_active":"2024-04-02T13:51:01.257001+0000","last_became_peered":"2024-04-02T13:51:01.257001+0000","last_unstale":"2024-04-02T13:51:56.663840+0000","last_undegraded":"2024-04-02T13:51:56.663840+0000","last_fullsized":"2024-04-02T13:51:56.663840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:31:22.628436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477954+0000","last_change":"2024-04-02T13:51:01.260533+0000","last_active":"2024-04-02T13:51:56.477954+0000","last_peered":"2024-04-02T13:51:56.477954+0000","last_clean":"2024-04-02T13:51:56.477954+0000","last_became_active":"2024-04-02T13:51:01.260203+0000","last_became_peered":"2024-04-02T13:51:01.260203+0000","last_unstale":"2024-04-02T13:51:56.477954+0000","last_undegraded":"2024-04-02T13:51:56.477954+0000","last_fullsized":"2024-04-02T13:51:56.477954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:00:06.008243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663416+0000","last_change":"2024-04-02T13:51:01.267733+0000","last_active":"2024-04-02T13:51:56.663416+0000","last_peered":"2024-04-02T13:51:56.663416+0000","last_clean":"2024-04-02T13:51:56.663416+0000","last_became_active":"2024-04-02T13:51:01.266642+0000","last_became_peered":"2024-04-02T13:51:01.266642+0000","last_unstale":"2024-04-02T13:51:56.663416+0000","last_undegraded":"2024-04-02T13:51:56.663416+0000","last_fullsized":"2024-04-02T13:51:56.663416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:39:25.696112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477899+0000","last_change":"2024-04-02T13:51:01.255933+0000","last_active":"2024-04-02T13:51:56.477899+0000","last_peered":"2024-04-02T13:51:56.477899+0000","last_clean":"2024-04-02T13:51:56.477899+0000","last_became_active":"2024-04-02T13:51:01.255558+0000","last_became_peered":"2024-04-02T13:51:01.255558+0000","last_unstale":"2024-04-02T13:51:56.477899+0000","last_undegraded":"2024-04-02T13:51:56.477899+0000","last_fullsized":"2024-04-02T13:51:56.477899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:13:11.537415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.555999+0000","last_change":"2024-04-02T13:51:01.256242+0000","last_active":"2024-04-02T13:51:56.555999+0000","last_peered":"2024-04-02T13:51:56.555999+0000","last_clean":"2024-04-02T13:51:56.555999+0000","last_became_active":"2024-04-02T13:51:01.255830+0000","last_became_peered":"2024-04-02T13:51:01.255830+0000","last_unstale":"2024-04-02T13:51:56.555999+0000","last_undegraded":"2024-04-02T13:51:56.555999+0000","last_fullsized":"2024-04-02T13:51:56.555999+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:39.130894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662031+0000","last_change":"2024-04-02T13:51:01.259738+0000","last_active":"2024-04-02T13:51:56.662031+0000","last_peered":"2024-04-02T13:51:56.662031+0000","last_clean":"2024-04-02T13:51:56.662031+0000","last_became_active":"2024-04-02T13:51:01.258630+0000","last_became_peered":"2024-04-02T13:51:01.258630+0000","last_unstale":"2024-04-02T13:51:56.662031+0000","last_undegraded":"2024-04-02T13:51:56.662031+0000","last_fullsized":"2024-04-02T13:51:56.662031+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:12:05.401171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662138+0000","last_change":"2024-04-02T13:51:01.257684+0000","last_active":"2024-04-02T13:51:56.662138+0000","last_peered":"2024-04-02T13:51:56.662138+0000","last_clean":"2024-04-02T13:51:56.662138+0000","last_became_active":"2024-04-02T13:51:01.257267+0000","last_became_peered":"2024-04-02T13:51:01.257267+0000","last_unstale":"2024-04-02T13:51:56.662138+0000","last_undegraded":"2024-04-02T13:51:56.662138+0000","last_fullsized":"2024-04-02T13:51:56.662138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:43:45.728233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663370+0000","last_change":"2024-04-02T13:51:01.261720+0000","last_active":"2024-04-02T13:51:56.663370+0000","last_peered":"2024-04-02T13:51:56.663370+0000","last_clean":"2024-04-02T13:51:56.663370+0000","last_became_active":"2024-04-02T13:51:01.261407+0000","last_became_peered":"2024-04-02T13:51:01.261407+0000","last_unstale":"2024-04-02T13:51:56.663370+0000","last_undegraded":"2024-04-02T13:51:56.663370+0000","last_fullsized":"2024-04-02T13:51:56.663370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:54:58.807606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.478367+0000","last_change":"2024-04-02T13:51:01.254488+0000","last_active":"2024-04-02T13:51:56.478367+0000","last_peered":"2024-04-02T13:51:56.478367+0000","last_clean":"2024-04-02T13:51:56.478367+0000","last_became_active":"2024-04-02T13:51:01.254139+0000","last_became_peered":"2024-04-02T13:51:01.254139+0000","last_unstale":"2024-04-02T13:51:56.478367+0000","last_undegraded":"2024-04-02T13:51:56.478367+0000","last_fullsized":"2024-04-02T13:51:56.478367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:02:41.075727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.475629+0000","last_change":"2024-04-02T13:51:01.256107+0000","last_active":"2024-04-02T13:51:56.475629+0000","last_peered":"2024-04-02T13:51:56.475629+0000","last_clean":"2024-04-02T13:51:56.475629+0000","last_became_active":"2024-04-02T13:51:01.255754+0000","last_became_peered":"2024-04-02T13:51:01.255754+0000","last_unstale":"2024-04-02T13:51:56.475629+0000","last_undegraded":"2024-04-02T13:51:56.475629+0000","last_fullsized":"2024-04-02T13:51:56.475629+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:36:59.793519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477869+0000","last_change":"2024-04-02T13:51:01.247550+0000","last_active":"2024-04-02T13:51:56.477869+0000","last_peered":"2024-04-02T13:51:56.477869+0000","last_clean":"2024-04-02T13:51:56.477869+0000","last_became_active":"2024-04-02T13:51:01.247236+0000","last_became_peered":"2024-04-02T13:51:01.247236+0000","last_unstale":"2024-04-02T13:51:56.477869+0000","last_undegraded":"2024-04-02T13:51:56.477869+0000","last_fullsized":"2024-04-02T13:51:56.477869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:16:02.940700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663779+0000","last_change":"2024-04-02T13:51:56.663779+0000","last_active":"2024-04-02T13:51:56.663779+0000","last_peered":"2024-04-02T13:51:56.663779+0000","last_clean":"2024-04-02T13:51:56.663779+0000","last_became_active":"2024-04-02T13:51:56.663327+0000","last_became_peered":"2024-04-02T13:51:56.663327+0000","last_unstale":"2024-04-02T13:51:56.663779+0000","last_undegraded":"2024-04-02T13:51:56.663779+0000","last_fullsized":"2024-04-02T13:51:56.663779+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:35:34.186278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":110,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:52:36.438008+0000","last_change":"2024-04-02T13:51:00.243244+0000","last_active":"2024-04-02T13:52:36.438008+0000","last_peered":"2024-04-02T13:52:36.438008+0000","last_clean":"2024-04-02T13:52:36.438008+0000","last_became_active":"2024-04-02T13:51:00.242892+0000","last_became_peered":"2024-04-02T13:51:00.242892+0000","last_unstale":"2024-04-02T13:52:36.438008+0000","last_undegraded":"2024-04-02T13:52:36.438008+0000","last_fullsized":"2024-04-02T13:52:36.438008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:49:23.338215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.482338+0000","last_change":"2024-04-02T13:51:01.264640+0000","last_active":"2024-04-02T13:51:56.482338+0000","last_peered":"2024-04-02T13:51:56.482338+0000","last_clean":"2024-04-02T13:51:56.482338+0000","last_became_active":"2024-04-02T13:51:01.264161+0000","last_became_peered":"2024-04-02T13:51:01.264161+0000","last_unstale":"2024-04-02T13:51:56.482338+0000","last_undegraded":"2024-04-02T13:51:56.482338+0000","last_fullsized":"2024-04-02T13:51:56.482338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:30:47.245647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477397+0000","last_change":"2024-04-02T13:51:05.362888+0000","last_active":"2024-04-02T13:51:56.477397+0000","last_peered":"2024-04-02T13:51:56.477397+0000","last_clean":"2024-04-02T13:51:56.477397+0000","last_became_active":"2024-04-02T13:51:05.362478+0000","last_became_peered":"2024-04-02T13:51:05.362478+0000","last_unstale":"2024-04-02T13:51:56.477397+0000","last_undegraded":"2024-04-02T13:51:56.477397+0000","last_fullsized":"2024-04-02T13:51:56.477397+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:05:54.304458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.482423+0000","last_change":"2024-04-02T13:51:00.247800+0000","last_active":"2024-04-02T13:51:56.482423+0000","last_peered":"2024-04-02T13:51:56.482423+0000","last_clean":"2024-04-02T13:51:56.482423+0000","last_became_active":"2024-04-02T13:51:00.247485+0000","last_became_peered":"2024-04-02T13:51:00.247485+0000","last_unstale":"2024-04-02T13:51:56.482423+0000","last_undegraded":"2024-04-02T13:51:56.482423+0000","last_fullsized":"2024-04-02T13:51:56.482423+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:17:24.767558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662205+0000","last_change":"2024-04-02T13:51:01.261416+0000","last_active":"2024-04-02T13:51:56.662205+0000","last_peered":"2024-04-02T13:51:56.662205+0000","last_clean":"2024-04-02T13:51:56.662205+0000","last_became_active":"2024-04-02T13:51:01.261099+0000","last_became_peered":"2024-04-02T13:51:01.261099+0000","last_unstale":"2024-04-02T13:51:56.662205+0000","last_undegraded":"2024-04-02T13:51:56.662205+0000","last_fullsized":"2024-04-02T13:51:56.662205+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:10:04.999402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477503+0000","last_change":"2024-04-02T13:51:05.361747+0000","last_active":"2024-04-02T13:51:56.477503+0000","last_peered":"2024-04-02T13:51:56.477503+0000","last_clean":"2024-04-02T13:51:56.477503+0000","last_became_active":"2024-04-02T13:51:05.361098+0000","last_became_peered":"2024-04-02T13:51:05.361098+0000","last_unstale":"2024-04-02T13:51:56.477503+0000","last_undegraded":"2024-04-02T13:51:56.477503+0000","last_fullsized":"2024-04-02T13:51:56.477503+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:45:35.048870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477099+0000","last_change":"2024-04-02T13:51:00.244048+0000","last_active":"2024-04-02T13:51:56.477099+0000","last_peered":"2024-04-02T13:51:56.477099+0000","last_clean":"2024-04-02T13:51:56.477099+0000","last_became_active":"2024-04-02T13:51:00.243510+0000","last_became_peered":"2024-04-02T13:51:00.243510+0000","last_unstale":"2024-04-02T13:51:56.477099+0000","last_undegraded":"2024-04-02T13:51:56.477099+0000","last_fullsized":"2024-04-02T13:51:56.477099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:53:48.364137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662811+0000","last_change":"2024-04-02T13:51:01.264711+0000","last_active":"2024-04-02T13:51:56.662811+0000","last_peered":"2024-04-02T13:51:56.662811+0000","last_clean":"2024-04-02T13:51:56.662811+0000","last_became_active":"2024-04-02T13:51:01.264289+0000","last_became_peered":"2024-04-02T13:51:01.264289+0000","last_unstale":"2024-04-02T13:51:56.662811+0000","last_undegraded":"2024-04-02T13:51:56.662811+0000","last_fullsized":"2024-04-02T13:51:56.662811+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:34:41.590071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477027+0000","last_change":"2024-04-02T13:51:05.363121+0000","last_active":"2024-04-02T13:51:56.477027+0000","last_peered":"2024-04-02T13:51:56.477027+0000","last_clean":"2024-04-02T13:51:56.477027+0000","last_became_active":"2024-04-02T13:51:05.362642+0000","last_became_peered":"2024-04-02T13:51:05.362642+0000","last_unstale":"2024-04-02T13:51:56.477027+0000","last_undegraded":"2024-04-02T13:51:56.477027+0000","last_fullsized":"2024-04-02T13:51:56.477027+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:55:40.848731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.476657+0000","last_change":"2024-04-02T13:51:00.236519+0000","last_active":"2024-04-02T13:51:56.476657+0000","last_peered":"2024-04-02T13:51:56.476657+0000","last_clean":"2024-04-02T13:51:56.476657+0000","last_became_active":"2024-04-02T13:51:00.235985+0000","last_became_peered":"2024-04-02T13:51:00.235985+0000","last_unstale":"2024-04-02T13:51:56.476657+0000","last_undegraded":"2024-04-02T13:51:56.476657+0000","last_fullsized":"2024-04-02T13:51:56.476657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:08.321765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662909+0000","last_change":"2024-04-02T13:51:01.260333+0000","last_active":"2024-04-02T13:51:56.662909+0000","last_peered":"2024-04-02T13:51:56.662909+0000","last_clean":"2024-04-02T13:51:56.662909+0000","last_became_active":"2024-04-02T13:51:01.259995+0000","last_became_peered":"2024-04-02T13:51:01.259995+0000","last_unstale":"2024-04-02T13:51:56.662909+0000","last_undegraded":"2024-04-02T13:51:56.662909+0000","last_fullsized":"2024-04-02T13:51:56.662909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:03:58.088653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477680+0000","last_change":"2024-04-02T13:51:05.359281+0000","last_active":"2024-04-02T13:51:56.477680+0000","last_peered":"2024-04-02T13:51:56.477680+0000","last_clean":"2024-04-02T13:51:56.477680+0000","last_became_active":"2024-04-02T13:51:05.358831+0000","last_became_peered":"2024-04-02T13:51:05.358831+0000","last_unstale":"2024-04-02T13:51:56.477680+0000","last_undegraded":"2024-04-02T13:51:56.477680+0000","last_fullsized":"2024-04-02T13:51:56.477680+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:48.461298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":106,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:52:39.443548+0000","last_change":"2024-04-02T13:51:00.241707+0000","last_active":"2024-04-02T13:52:39.443548+0000","last_peered":"2024-04-02T13:52:39.443548+0000","last_clean":"2024-04-02T13:52:39.443548+0000","last_became_active":"2024-04-02T13:51:00.241225+0000","last_became_peered":"2024-04-02T13:51:00.241225+0000","last_unstale":"2024-04-02T13:52:39.443548+0000","last_undegraded":"2024-04-02T13:52:39.443548+0000","last_fullsized":"2024-04-02T13:52:39.443548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:59:44.537503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.476384+0000","last_change":"2024-04-02T13:51:01.252942+0000","last_active":"2024-04-02T13:51:56.476384+0000","last_peered":"2024-04-02T13:51:56.476384+0000","last_clean":"2024-04-02T13:51:56.476384+0000","last_became_active":"2024-04-02T13:51:01.252631+0000","last_became_peered":"2024-04-02T13:51:01.252631+0000","last_unstale":"2024-04-02T13:51:56.476384+0000","last_undegraded":"2024-04-02T13:51:56.476384+0000","last_fullsized":"2024-04-02T13:51:56.476384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:43:24.725688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.478285+0000","last_change":"2024-04-02T13:51:05.359208+0000","last_active":"2024-04-02T13:51:56.478285+0000","last_peered":"2024-04-02T13:51:56.478285+0000","last_clean":"2024-04-02T13:51:56.478285+0000","last_became_active":"2024-04-02T13:51:05.358738+0000","last_became_peered":"2024-04-02T13:51:05.358738+0000","last_unstale":"2024-04-02T13:51:56.478285+0000","last_undegraded":"2024-04-02T13:51:56.478285+0000","last_fullsized":"2024-04-02T13:51:56.478285+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:47:46.034562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.476633+0000","last_change":"2024-04-02T13:51:00.236413+0000","last_active":"2024-04-02T13:51:56.476633+0000","last_peered":"2024-04-02T13:51:56.476633+0000","last_clean":"2024-04-02T13:51:56.476633+0000","last_became_active":"2024-04-02T13:51:00.236046+0000","last_became_peered":"2024-04-02T13:51:00.236046+0000","last_unstale":"2024-04-02T13:51:56.476633+0000","last_undegraded":"2024-04-02T13:51:56.476633+0000","last_fullsized":"2024-04-02T13:51:56.476633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:25:15.000726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662699+0000","last_change":"2024-04-02T13:51:01.265471+0000","last_active":"2024-04-02T13:51:56.662699+0000","last_peered":"2024-04-02T13:51:56.662699+0000","last_clean":"2024-04-02T13:51:56.662699+0000","last_became_active":"2024-04-02T13:51:01.265114+0000","last_became_peered":"2024-04-02T13:51:01.265114+0000","last_unstale":"2024-04-02T13:51:56.662699+0000","last_undegraded":"2024-04-02T13:51:56.662699+0000","last_fullsized":"2024-04-02T13:51:56.662699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:01:01.675686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.664063+0000","last_change":"2024-04-02T13:51:56.664063+0000","last_active":"2024-04-02T13:51:56.664063+0000","last_peered":"2024-04-02T13:51:56.664063+0000","last_clean":"2024-04-02T13:51:56.664063+0000","last_became_active":"2024-04-02T13:51:56.663590+0000","last_became_peered":"2024-04-02T13:51:56.663590+0000","last_unstale":"2024-04-02T13:51:56.664063+0000","last_undegraded":"2024-04-02T13:51:56.664063+0000","last_fullsized":"2024-04-02T13:51:56.664063+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:43:51.735851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.555709+0000","last_change":"2024-04-02T13:51:00.237204+0000","last_active":"2024-04-02T13:51:56.555709+0000","last_peered":"2024-04-02T13:51:56.555709+0000","last_clean":"2024-04-02T13:51:56.555709+0000","last_became_active":"2024-04-02T13:51:00.236707+0000","last_became_peered":"2024-04-02T13:51:00.236707+0000","last_unstale":"2024-04-02T13:51:56.555709+0000","last_undegraded":"2024-04-02T13:51:56.555709+0000","last_fullsized":"2024-04-02T13:51:56.555709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:00:17.771428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.481887+0000","last_change":"2024-04-02T13:51:01.250152+0000","last_active":"2024-04-02T13:51:56.481887+0000","last_peered":"2024-04-02T13:51:56.481887+0000","last_clean":"2024-04-02T13:51:56.481887+0000","last_became_active":"2024-04-02T13:51:01.249817+0000","last_became_peered":"2024-04-02T13:51:01.249817+0000","last_unstale":"2024-04-02T13:51:56.481887+0000","last_undegraded":"2024-04-02T13:51:56.481887+0000","last_fullsized":"2024-04-02T13:51:56.481887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:13:31.719380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662339+0000","last_change":"2024-04-02T13:51:05.366551+0000","last_active":"2024-04-02T13:51:56.662339+0000","last_peered":"2024-04-02T13:51:56.662339+0000","last_clean":"2024-04-02T13:51:56.662339+0000","last_became_active":"2024-04-02T13:51:05.366175+0000","last_became_peered":"2024-04-02T13:51:05.366175+0000","last_unstale":"2024-04-02T13:51:56.662339+0000","last_undegraded":"2024-04-02T13:51:56.662339+0000","last_fullsized":"2024-04-02T13:51:56.662339+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:47:49.822918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662285+0000","last_change":"2024-04-02T13:51:00.239521+0000","last_active":"2024-04-02T13:51:56.662285+0000","last_peered":"2024-04-02T13:51:56.662285+0000","last_clean":"2024-04-02T13:51:56.662285+0000","last_became_active":"2024-04-02T13:51:00.238608+0000","last_became_peered":"2024-04-02T13:51:00.238608+0000","last_unstale":"2024-04-02T13:51:56.662285+0000","last_undegraded":"2024-04-02T13:51:56.662285+0000","last_fullsized":"2024-04-02T13:51:56.662285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:00.983006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.475126+0000","last_change":"2024-04-02T13:51:01.255021+0000","last_active":"2024-04-02T13:51:56.475126+0000","last_peered":"2024-04-02T13:51:56.475126+0000","last_clean":"2024-04-02T13:51:56.475126+0000","last_became_active":"2024-04-02T13:51:01.254630+0000","last_became_peered":"2024-04-02T13:51:01.254630+0000","last_unstale":"2024-04-02T13:51:56.475126+0000","last_undegraded":"2024-04-02T13:51:56.475126+0000","last_fullsized":"2024-04-02T13:51:56.475126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:36:08.773431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.481757+0000","last_change":"2024-04-02T13:51:05.362004+0000","last_active":"2024-04-02T13:51:56.481757+0000","last_peered":"2024-04-02T13:51:56.481757+0000","last_clean":"2024-04-02T13:51:56.481757+0000","last_became_active":"2024-04-02T13:51:05.361538+0000","last_became_peered":"2024-04-02T13:51:05.361538+0000","last_unstale":"2024-04-02T13:51:56.481757+0000","last_undegraded":"2024-04-02T13:51:56.481757+0000","last_fullsized":"2024-04-02T13:51:56.481757+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:27:21.728695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":173,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:52:39.442618+0000","last_change":"2024-04-02T13:51:00.234814+0000","last_active":"2024-04-02T13:52:39.442618+0000","last_peered":"2024-04-02T13:52:39.442618+0000","last_clean":"2024-04-02T13:52:39.442618+0000","last_became_active":"2024-04-02T13:51:00.234481+0000","last_became_peered":"2024-04-02T13:51:00.234481+0000","last_unstale":"2024-04-02T13:52:39.442618+0000","last_undegraded":"2024-04-02T13:52:39.442618+0000","last_fullsized":"2024-04-02T13:52:39.442618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:45:34.901226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":142,"num_read_kb":142,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663585+0000","last_change":"2024-04-02T13:51:01.264093+0000","last_active":"2024-04-02T13:51:56.663585+0000","last_peered":"2024-04-02T13:51:56.663585+0000","last_clean":"2024-04-02T13:51:56.663585+0000","last_became_active":"2024-04-02T13:51:01.263754+0000","last_became_peered":"2024-04-02T13:51:01.263754+0000","last_unstale":"2024-04-02T13:51:56.663585+0000","last_undegraded":"2024-04-02T13:51:56.663585+0000","last_fullsized":"2024-04-02T13:51:56.663585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:49:18.630195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.475111+0000","last_change":"2024-04-02T13:51:05.370930+0000","last_active":"2024-04-02T13:51:56.475111+0000","last_peered":"2024-04-02T13:51:56.475111+0000","last_clean":"2024-04-02T13:51:56.475111+0000","last_became_active":"2024-04-02T13:51:05.370443+0000","last_became_peered":"2024-04-02T13:51:05.370443+0000","last_unstale":"2024-04-02T13:51:56.475111+0000","last_undegraded":"2024-04-02T13:51:56.475111+0000","last_fullsized":"2024-04-02T13:51:56.475111+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:24:27.055494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663027+0000","last_change":"2024-04-02T13:51:00.241206+0000","last_active":"2024-04-02T13:51:56.663027+0000","last_peered":"2024-04-02T13:51:56.663027+0000","last_clean":"2024-04-02T13:51:56.663027+0000","last_became_active":"2024-04-02T13:51:00.240832+0000","last_became_peered":"2024-04-02T13:51:00.240832+0000","last_unstale":"2024-04-02T13:51:56.663027+0000","last_undegraded":"2024-04-02T13:51:56.663027+0000","last_fullsized":"2024-04-02T13:51:56.663027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:44:11.626435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662489+0000","last_change":"2024-04-02T13:51:01.259315+0000","last_active":"2024-04-02T13:51:56.662489+0000","last_peered":"2024-04-02T13:51:56.662489+0000","last_clean":"2024-04-02T13:51:56.662489+0000","last_became_active":"2024-04-02T13:51:01.258951+0000","last_became_peered":"2024-04-02T13:51:01.258951+0000","last_unstale":"2024-04-02T13:51:56.662489+0000","last_undegraded":"2024-04-02T13:51:56.662489+0000","last_fullsized":"2024-04-02T13:51:56.662489+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:59:43.714044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.665826+0000","last_change":"2024-04-02T13:51:56.665826+0000","last_active":"2024-04-02T13:51:56.665826+0000","last_peered":"2024-04-02T13:51:56.665826+0000","last_clean":"2024-04-02T13:51:56.665826+0000","last_became_active":"2024-04-02T13:51:56.665213+0000","last_became_peered":"2024-04-02T13:51:56.665213+0000","last_unstale":"2024-04-02T13:51:56.665826+0000","last_undegraded":"2024-04-02T13:51:56.665826+0000","last_fullsized":"2024-04-02T13:51:56.665826+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:45:46.625063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662860+0000","last_change":"2024-04-02T13:51:00.230938+0000","last_active":"2024-04-02T13:51:56.662860+0000","last_peered":"2024-04-02T13:51:56.662860+0000","last_clean":"2024-04-02T13:51:56.662860+0000","last_became_active":"2024-04-02T13:51:00.230585+0000","last_became_peered":"2024-04-02T13:51:00.230585+0000","last_unstale":"2024-04-02T13:51:56.662860+0000","last_undegraded":"2024-04-02T13:51:56.662860+0000","last_fullsized":"2024-04-02T13:51:56.662860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:51:37.463876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663209+0000","last_change":"2024-04-02T13:51:01.264216+0000","last_active":"2024-04-02T13:51:56.663209+0000","last_peered":"2024-04-02T13:51:56.663209+0000","last_clean":"2024-04-02T13:51:56.663209+0000","last_became_active":"2024-04-02T13:51:01.263824+0000","last_became_peered":"2024-04-02T13:51:01.263824+0000","last_unstale":"2024-04-02T13:51:56.663209+0000","last_undegraded":"2024-04-02T13:51:56.663209+0000","last_fullsized":"2024-04-02T13:51:56.663209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:16:26.391163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.475438+0000","last_change":"2024-04-02T13:51:05.366002+0000","last_active":"2024-04-02T13:51:56.475438+0000","last_peered":"2024-04-02T13:51:56.475438+0000","last_clean":"2024-04-02T13:51:56.475438+0000","last_became_active":"2024-04-02T13:51:05.365547+0000","last_became_peered":"2024-04-02T13:51:05.365547+0000","last_unstale":"2024-04-02T13:51:56.475438+0000","last_undegraded":"2024-04-02T13:51:56.475438+0000","last_fullsized":"2024-04-02T13:51:56.475438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:31:38.990617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'36","reported_seq":212,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:52:39.442607+0000","last_change":"2024-04-02T13:51:00.233925+0000","last_active":"2024-04-02T13:52:39.442607+0000","last_peered":"2024-04-02T13:52:39.442607+0000","last_clean":"2024-04-02T13:52:39.442607+0000","last_became_active":"2024-04-02T13:51:00.233574+0000","last_became_peered":"2024-04-02T13:51:00.233574+0000","last_unstale":"2024-04-02T13:52:39.442607+0000","last_undegraded":"2024-04-02T13:52:39.442607+0000","last_fullsized":"2024-04-02T13:52:39.442607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:33:32.758874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62333,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":149,"num_read_kb":136,"num_write":38,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477298+0000","last_change":"2024-04-02T13:51:01.258825+0000","last_active":"2024-04-02T13:51:56.477298+0000","last_peered":"2024-04-02T13:51:56.477298+0000","last_clean":"2024-04-02T13:51:56.477298+0000","last_became_active":"2024-04-02T13:51:01.258500+0000","last_became_peered":"2024-04-02T13:51:01.258500+0000","last_unstale":"2024-04-02T13:51:56.477298+0000","last_undegraded":"2024-04-02T13:51:56.477298+0000","last_fullsized":"2024-04-02T13:51:56.477298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:33:15.409765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663136+0000","last_change":"2024-04-02T13:51:05.365010+0000","last_active":"2024-04-02T13:51:56.663136+0000","last_peered":"2024-04-02T13:51:56.663136+0000","last_clean":"2024-04-02T13:51:56.663136+0000","last_became_active":"2024-04-02T13:51:05.364499+0000","last_became_peered":"2024-04-02T13:51:05.364499+0000","last_unstale":"2024-04-02T13:51:56.663136+0000","last_undegraded":"2024-04-02T13:51:56.663136+0000","last_fullsized":"2024-04-02T13:51:56.663136+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:07:03.500017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.496876+0000","last_change":"2024-04-02T13:51:56.496876+0000","last_active":"2024-04-02T13:51:56.496876+0000","last_peered":"2024-04-02T13:51:56.496876+0000","last_clean":"2024-04-02T13:51:56.496876+0000","last_became_active":"2024-04-02T13:51:56.484672+0000","last_became_peered":"2024-04-02T13:51:56.484672+0000","last_unstale":"2024-04-02T13:51:56.496876+0000","last_undegraded":"2024-04-02T13:51:56.496876+0000","last_fullsized":"2024-04-02T13:51:56.496876+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:31:40.027658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.555868+0000","last_change":"2024-04-02T13:51:01.256334+0000","last_active":"2024-04-02T13:51:56.555868+0000","last_peered":"2024-04-02T13:51:56.555868+0000","last_clean":"2024-04-02T13:51:56.555868+0000","last_became_active":"2024-04-02T13:51:01.255964+0000","last_became_peered":"2024-04-02T13:51:01.255964+0000","last_unstale":"2024-04-02T13:51:56.555868+0000","last_undegraded":"2024-04-02T13:51:56.555868+0000","last_fullsized":"2024-04-02T13:51:56.555868+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:43:04.842812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.479357+0000","last_change":"2024-04-02T13:51:05.362696+0000","last_active":"2024-04-02T13:51:56.479357+0000","last_peered":"2024-04-02T13:51:56.479357+0000","last_clean":"2024-04-02T13:51:56.479357+0000","last_became_active":"2024-04-02T13:51:05.362225+0000","last_became_peered":"2024-04-02T13:51:05.362225+0000","last_unstale":"2024-04-02T13:51:56.479357+0000","last_undegraded":"2024-04-02T13:51:56.479357+0000","last_fullsized":"2024-04-02T13:51:56.479357+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:23:56.308286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662545+0000","last_change":"2024-04-02T13:51:00.239673+0000","last_active":"2024-04-02T13:51:56.662545+0000","last_peered":"2024-04-02T13:51:56.662545+0000","last_clean":"2024-04-02T13:51:56.662545+0000","last_became_active":"2024-04-02T13:51:00.239233+0000","last_became_peered":"2024-04-02T13:51:00.239233+0000","last_unstale":"2024-04-02T13:51:56.662545+0000","last_undegraded":"2024-04-02T13:51:56.662545+0000","last_fullsized":"2024-04-02T13:51:56.662545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:05:48.119953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.664125+0000","last_change":"2024-04-02T13:51:01.257474+0000","last_active":"2024-04-02T13:51:56.664125+0000","last_peered":"2024-04-02T13:51:56.664125+0000","last_clean":"2024-04-02T13:51:56.664125+0000","last_became_active":"2024-04-02T13:51:01.254977+0000","last_became_peered":"2024-04-02T13:51:01.254977+0000","last_unstale":"2024-04-02T13:51:56.664125+0000","last_undegraded":"2024-04-02T13:51:56.664125+0000","last_fullsized":"2024-04-02T13:51:56.664125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:13:07.446520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.475753+0000","last_change":"2024-04-02T13:51:05.371531+0000","last_active":"2024-04-02T13:51:56.475753+0000","last_peered":"2024-04-02T13:51:56.475753+0000","last_clean":"2024-04-02T13:51:56.475753+0000","last_became_active":"2024-04-02T13:51:05.370551+0000","last_became_peered":"2024-04-02T13:51:05.370551+0000","last_unstale":"2024-04-02T13:51:56.475753+0000","last_undegraded":"2024-04-02T13:51:56.475753+0000","last_fullsized":"2024-04-02T13:51:56.475753+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:43:48.796009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'5","reported_seq":105,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:52:38.441548+0000","last_change":"2024-04-02T13:51:00.237715+0000","last_active":"2024-04-02T13:52:38.441548+0000","last_peered":"2024-04-02T13:52:38.441548+0000","last_clean":"2024-04-02T13:52:38.441548+0000","last_became_active":"2024-04-02T13:51:00.237349+0000","last_became_peered":"2024-04-02T13:51:00.237349+0000","last_unstale":"2024-04-02T13:52:38.441548+0000","last_undegraded":"2024-04-02T13:52:38.441548+0000","last_fullsized":"2024-04-02T13:52:38.441548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:33:20.651562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663012+0000","last_change":"2024-04-02T13:51:01.259447+0000","last_active":"2024-04-02T13:51:56.663012+0000","last_peered":"2024-04-02T13:51:56.663012+0000","last_clean":"2024-04-02T13:51:56.663012+0000","last_became_active":"2024-04-02T13:51:01.259101+0000","last_became_peered":"2024-04-02T13:51:01.259101+0000","last_unstale":"2024-04-02T13:51:56.663012+0000","last_undegraded":"2024-04-02T13:51:56.663012+0000","last_fullsized":"2024-04-02T13:51:56.663012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:51:44.311440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.556068+0000","last_change":"2024-04-02T13:51:05.372446+0000","last_active":"2024-04-02T13:51:56.556068+0000","last_peered":"2024-04-02T13:51:56.556068+0000","last_clean":"2024-04-02T13:51:56.556068+0000","last_became_active":"2024-04-02T13:51:05.372007+0000","last_became_peered":"2024-04-02T13:51:05.372007+0000","last_unstale":"2024-04-02T13:51:56.556068+0000","last_undegraded":"2024-04-02T13:51:56.556068+0000","last_fullsized":"2024-04-02T13:51:56.556068+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:54:23.463179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.476245+0000","last_change":"2024-04-02T13:51:00.241492+0000","last_active":"2024-04-02T13:51:56.476245+0000","last_peered":"2024-04-02T13:51:56.476245+0000","last_clean":"2024-04-02T13:51:56.476245+0000","last_became_active":"2024-04-02T13:51:00.241167+0000","last_became_peered":"2024-04-02T13:51:00.241167+0000","last_unstale":"2024-04-02T13:51:56.476245+0000","last_undegraded":"2024-04-02T13:51:56.476245+0000","last_fullsized":"2024-04-02T13:51:56.476245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:44:44.070187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662409+0000","last_change":"2024-04-02T13:51:01.263200+0000","last_active":"2024-04-02T13:51:56.662409+0000","last_peered":"2024-04-02T13:51:56.662409+0000","last_clean":"2024-04-02T13:51:56.662409+0000","last_became_active":"2024-04-02T13:51:01.262812+0000","last_became_peered":"2024-04-02T13:51:01.262812+0000","last_unstale":"2024-04-02T13:51:56.662409+0000","last_undegraded":"2024-04-02T13:51:56.662409+0000","last_fullsized":"2024-04-02T13:51:56.662409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:33:35.898047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662275+0000","last_change":"2024-04-02T13:51:05.366729+0000","last_active":"2024-04-02T13:51:56.662275+0000","last_peered":"2024-04-02T13:51:56.662275+0000","last_clean":"2024-04-02T13:51:56.662275+0000","last_became_active":"2024-04-02T13:51:05.366257+0000","last_became_peered":"2024-04-02T13:51:05.366257+0000","last_unstale":"2024-04-02T13:51:56.662275+0000","last_undegraded":"2024-04-02T13:51:56.662275+0000","last_fullsized":"2024-04-02T13:51:56.662275+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:46:03.400300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.476916+0000","last_change":"2024-04-02T13:51:00.236004+0000","last_active":"2024-04-02T13:51:56.476916+0000","last_peered":"2024-04-02T13:51:56.476916+0000","last_clean":"2024-04-02T13:51:56.476916+0000","last_became_active":"2024-04-02T13:51:00.235160+0000","last_became_peered":"2024-04-02T13:51:00.235160+0000","last_unstale":"2024-04-02T13:51:56.476916+0000","last_undegraded":"2024-04-02T13:51:56.476916+0000","last_fullsized":"2024-04-02T13:51:56.476916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:16:35.617346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.479288+0000","last_change":"2024-04-02T13:51:01.259565+0000","last_active":"2024-04-02T13:51:56.479288+0000","last_peered":"2024-04-02T13:51:56.479288+0000","last_clean":"2024-04-02T13:51:56.479288+0000","last_became_active":"2024-04-02T13:51:01.259225+0000","last_became_peered":"2024-04-02T13:51:01.259225+0000","last_unstale":"2024-04-02T13:51:56.479288+0000","last_undegraded":"2024-04-02T13:51:56.479288+0000","last_fullsized":"2024-04-02T13:51:56.479288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:22:18.878029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.476269+0000","last_change":"2024-04-02T13:51:05.518750+0000","last_active":"2024-04-02T13:51:56.476269+0000","last_peered":"2024-04-02T13:51:56.476269+0000","last_clean":"2024-04-02T13:51:56.476269+0000","last_became_active":"2024-04-02T13:51:05.518247+0000","last_became_peered":"2024-04-02T13:51:05.518247+0000","last_unstale":"2024-04-02T13:51:56.476269+0000","last_undegraded":"2024-04-02T13:51:56.476269+0000","last_fullsized":"2024-04-02T13:51:56.476269+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:08:10.232509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.475385+0000","last_change":"2024-04-02T13:51:00.241476+0000","last_active":"2024-04-02T13:51:56.475385+0000","last_peered":"2024-04-02T13:51:56.475385+0000","last_clean":"2024-04-02T13:51:56.475385+0000","last_became_active":"2024-04-02T13:51:00.241085+0000","last_became_peered":"2024-04-02T13:51:00.241085+0000","last_unstale":"2024-04-02T13:51:56.475385+0000","last_undegraded":"2024-04-02T13:51:56.475385+0000","last_fullsized":"2024-04-02T13:51:56.475385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:29:01.503901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477254+0000","last_change":"2024-04-02T13:51:01.259791+0000","last_active":"2024-04-02T13:51:56.477254+0000","last_peered":"2024-04-02T13:51:56.477254+0000","last_clean":"2024-04-02T13:51:56.477254+0000","last_became_active":"2024-04-02T13:51:01.259440+0000","last_became_peered":"2024-04-02T13:51:01.259440+0000","last_unstale":"2024-04-02T13:51:56.477254+0000","last_undegraded":"2024-04-02T13:51:56.477254+0000","last_fullsized":"2024-04-02T13:51:56.477254+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:43:52.422263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.555849+0000","last_change":"2024-04-02T13:51:05.366243+0000","last_active":"2024-04-02T13:51:56.555849+0000","last_peered":"2024-04-02T13:51:56.555849+0000","last_clean":"2024-04-02T13:51:56.555849+0000","last_became_active":"2024-04-02T13:51:05.365801+0000","last_became_peered":"2024-04-02T13:51:05.365801+0000","last_unstale":"2024-04-02T13:51:56.555849+0000","last_undegraded":"2024-04-02T13:51:56.555849+0000","last_fullsized":"2024-04-02T13:51:56.555849+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:33:56.853434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":105,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:52:39.442336+0000","last_change":"2024-04-02T13:51:00.248695+0000","last_active":"2024-04-02T13:52:39.442336+0000","last_peered":"2024-04-02T13:52:39.442336+0000","last_clean":"2024-04-02T13:52:39.442336+0000","last_became_active":"2024-04-02T13:51:00.248313+0000","last_became_peered":"2024-04-02T13:51:00.248313+0000","last_unstale":"2024-04-02T13:52:39.442336+0000","last_undegraded":"2024-04-02T13:52:39.442336+0000","last_fullsized":"2024-04-02T13:52:39.442336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:43:21.410823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477001+0000","last_change":"2024-04-02T13:51:01.260993+0000","last_active":"2024-04-02T13:51:56.477001+0000","last_peered":"2024-04-02T13:51:56.477001+0000","last_clean":"2024-04-02T13:51:56.477001+0000","last_became_active":"2024-04-02T13:51:01.260510+0000","last_became_peered":"2024-04-02T13:51:01.260510+0000","last_unstale":"2024-04-02T13:51:56.477001+0000","last_undegraded":"2024-04-02T13:51:56.477001+0000","last_fullsized":"2024-04-02T13:51:56.477001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:17:04.941822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.556012+0000","last_change":"2024-04-02T13:51:05.518455+0000","last_active":"2024-04-02T13:51:56.556012+0000","last_peered":"2024-04-02T13:51:56.556012+0000","last_clean":"2024-04-02T13:51:56.556012+0000","last_became_active":"2024-04-02T13:51:05.517962+0000","last_became_peered":"2024-04-02T13:51:05.517962+0000","last_unstale":"2024-04-02T13:51:56.556012+0000","last_undegraded":"2024-04-02T13:51:56.556012+0000","last_fullsized":"2024-04-02T13:51:56.556012+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:19:47.734233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662479+0000","last_change":"2024-04-02T13:51:01.259220+0000","last_active":"2024-04-02T13:51:56.662479+0000","last_peered":"2024-04-02T13:51:56.662479+0000","last_clean":"2024-04-02T13:51:56.662479+0000","last_became_active":"2024-04-02T13:51:01.258890+0000","last_became_peered":"2024-04-02T13:51:01.258890+0000","last_unstale":"2024-04-02T13:51:56.662479+0000","last_undegraded":"2024-04-02T13:51:56.662479+0000","last_fullsized":"2024-04-02T13:51:56.662479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:56:15.444736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":105,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:52:39.442219+0000","last_change":"2024-04-02T13:51:00.246727+0000","last_active":"2024-04-02T13:52:39.442219+0000","last_peered":"2024-04-02T13:52:39.442219+0000","last_clean":"2024-04-02T13:52:39.442219+0000","last_became_active":"2024-04-02T13:51:00.246364+0000","last_became_peered":"2024-04-02T13:51:00.246364+0000","last_unstale":"2024-04-02T13:52:39.442219+0000","last_undegraded":"2024-04-02T13:52:39.442219+0000","last_fullsized":"2024-04-02T13:52:39.442219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:35:13.249972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663127+0000","last_change":"2024-04-02T13:51:05.358506+0000","last_active":"2024-04-02T13:51:56.663127+0000","last_peered":"2024-04-02T13:51:56.663127+0000","last_clean":"2024-04-02T13:51:56.663127+0000","last_became_active":"2024-04-02T13:51:05.358157+0000","last_became_peered":"2024-04-02T13:51:05.358157+0000","last_unstale":"2024-04-02T13:51:56.663127+0000","last_undegraded":"2024-04-02T13:51:56.663127+0000","last_fullsized":"2024-04-02T13:51:56.663127+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:24:15.585124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.665735+0000","last_change":"2024-04-02T13:51:56.665307+0000","last_active":"2024-04-02T13:51:56.665735+0000","last_peered":"2024-04-02T13:51:56.665735+0000","last_clean":"2024-04-02T13:51:56.665735+0000","last_became_active":"2024-04-02T13:51:56.665003+0000","last_became_peered":"2024-04-02T13:51:56.665003+0000","last_unstale":"2024-04-02T13:51:56.665735+0000","last_undegraded":"2024-04-02T13:51:56.665735+0000","last_fullsized":"2024-04-02T13:51:56.665735+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:02:46.421524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.478223+0000","last_change":"2024-04-02T13:51:00.238962+0000","last_active":"2024-04-02T13:51:56.478223+0000","last_peered":"2024-04-02T13:51:56.478223+0000","last_clean":"2024-04-02T13:51:56.478223+0000","last_became_active":"2024-04-02T13:51:00.238475+0000","last_became_peered":"2024-04-02T13:51:00.238475+0000","last_unstale":"2024-04-02T13:51:56.478223+0000","last_undegraded":"2024-04-02T13:51:56.478223+0000","last_fullsized":"2024-04-02T13:51:56.478223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:41:07.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":362,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662789+0000","last_change":"2024-04-02T13:49:47.847355+0000","last_active":"2024-04-02T13:51:56.662789+0000","last_peered":"2024-04-02T13:51:56.662789+0000","last_clean":"2024-04-02T13:51:56.662789+0000","last_became_active":"2024-04-02T13:49:47.846768+0000","last_became_peered":"2024-04-02T13:49:47.846768+0000","last_unstale":"2024-04-02T13:51:56.662789+0000","last_undegraded":"2024-04-02T13:51:56.662789+0000","last_fullsized":"2024-04-02T13:51:56.662789+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:46:21.098780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:46:21.098780+0000","last_clean_scrub_stamp":"2024-04-02T13:46:21.098780+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:03:52.323921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663467+0000","last_change":"2024-04-02T13:51:05.518859+0000","last_active":"2024-04-02T13:51:56.663467+0000","last_peered":"2024-04-02T13:51:56.663467+0000","last_clean":"2024-04-02T13:51:56.663467+0000","last_became_active":"2024-04-02T13:51:05.518364+0000","last_became_peered":"2024-04-02T13:51:05.518364+0000","last_unstale":"2024-04-02T13:51:56.663467+0000","last_undegraded":"2024-04-02T13:51:56.663467+0000","last_fullsized":"2024-04-02T13:51:56.663467+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:10:45.877853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.555609+0000","last_change":"2024-04-02T13:51:01.254795+0000","last_active":"2024-04-02T13:51:56.555609+0000","last_peered":"2024-04-02T13:51:56.555609+0000","last_clean":"2024-04-02T13:51:56.555609+0000","last_became_active":"2024-04-02T13:51:01.254452+0000","last_became_peered":"2024-04-02T13:51:01.254452+0000","last_unstale":"2024-04-02T13:51:56.555609+0000","last_undegraded":"2024-04-02T13:51:56.555609+0000","last_fullsized":"2024-04-02T13:51:56.555609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:13:53.747308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662875+0000","last_change":"2024-04-02T13:51:00.241323+0000","last_active":"2024-04-02T13:51:56.662875+0000","last_peered":"2024-04-02T13:51:56.662875+0000","last_clean":"2024-04-02T13:51:56.662875+0000","last_became_active":"2024-04-02T13:51:00.240989+0000","last_became_peered":"2024-04-02T13:51:00.240989+0000","last_unstale":"2024-04-02T13:51:56.662875+0000","last_undegraded":"2024-04-02T13:51:56.662875+0000","last_fullsized":"2024-04-02T13:51:56.662875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:47:52.601517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.665905+0000","last_change":"2024-04-02T13:51:56.665905+0000","last_active":"2024-04-02T13:51:56.665905+0000","last_peered":"2024-04-02T13:51:56.665905+0000","last_clean":"2024-04-02T13:51:56.665905+0000","last_became_active":"2024-04-02T13:51:56.665315+0000","last_became_peered":"2024-04-02T13:51:56.665315+0000","last_unstale":"2024-04-02T13:51:56.665905+0000","last_undegraded":"2024-04-02T13:51:56.665905+0000","last_fullsized":"2024-04-02T13:51:56.665905+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:44:34.548102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.476884+0000","last_change":"2024-04-02T13:51:00.241416+0000","last_active":"2024-04-02T13:51:56.476884+0000","last_peered":"2024-04-02T13:51:56.476884+0000","last_clean":"2024-04-02T13:51:56.476884+0000","last_became_active":"2024-04-02T13:51:00.241071+0000","last_became_peered":"2024-04-02T13:51:00.241071+0000","last_unstale":"2024-04-02T13:51:56.476884+0000","last_undegraded":"2024-04-02T13:51:56.476884+0000","last_fullsized":"2024-04-02T13:51:56.476884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:24:16.243737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.556027+0000","last_change":"2024-04-02T13:51:01.252156+0000","last_active":"2024-04-02T13:51:56.556027+0000","last_peered":"2024-04-02T13:51:56.556027+0000","last_clean":"2024-04-02T13:51:56.556027+0000","last_became_active":"2024-04-02T13:51:01.251733+0000","last_became_peered":"2024-04-02T13:51:01.251733+0000","last_unstale":"2024-04-02T13:51:56.556027+0000","last_undegraded":"2024-04-02T13:51:56.556027+0000","last_fullsized":"2024-04-02T13:51:56.556027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:54:24.082643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.481850+0000","last_change":"2024-04-02T13:51:05.518367+0000","last_active":"2024-04-02T13:51:56.481850+0000","last_peered":"2024-04-02T13:51:56.481850+0000","last_clean":"2024-04-02T13:51:56.481850+0000","last_became_active":"2024-04-02T13:51:05.517783+0000","last_became_peered":"2024-04-02T13:51:05.517783+0000","last_unstale":"2024-04-02T13:51:56.481850+0000","last_undegraded":"2024-04-02T13:51:56.481850+0000","last_fullsized":"2024-04-02T13:51:56.481850+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:10:53.880909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.475665+0000","last_change":"2024-04-02T13:51:00.248998+0000","last_active":"2024-04-02T13:51:56.475665+0000","last_peered":"2024-04-02T13:51:56.475665+0000","last_clean":"2024-04-02T13:51:56.475665+0000","last_became_active":"2024-04-02T13:51:00.248613+0000","last_became_peered":"2024-04-02T13:51:00.248613+0000","last_unstale":"2024-04-02T13:51:56.475665+0000","last_undegraded":"2024-04-02T13:51:56.475665+0000","last_fullsized":"2024-04-02T13:51:56.475665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:53.948645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.482134+0000","last_change":"2024-04-02T13:51:01.257244+0000","last_active":"2024-04-02T13:51:56.482134+0000","last_peered":"2024-04-02T13:51:56.482134+0000","last_clean":"2024-04-02T13:51:56.482134+0000","last_became_active":"2024-04-02T13:51:01.256739+0000","last_became_peered":"2024-04-02T13:51:01.256739+0000","last_unstale":"2024-04-02T13:51:56.482134+0000","last_undegraded":"2024-04-02T13:51:56.482134+0000","last_fullsized":"2024-04-02T13:51:56.482134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:38:18.786553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.478152+0000","last_change":"2024-04-02T13:51:05.356177+0000","last_active":"2024-04-02T13:51:56.478152+0000","last_peered":"2024-04-02T13:51:56.478152+0000","last_clean":"2024-04-02T13:51:56.478152+0000","last_became_active":"2024-04-02T13:51:05.354704+0000","last_became_peered":"2024-04-02T13:51:05.354704+0000","last_unstale":"2024-04-02T13:51:56.478152+0000","last_undegraded":"2024-04-02T13:51:56.478152+0000","last_fullsized":"2024-04-02T13:51:56.478152+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:27:55.761490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.478405+0000","last_change":"2024-04-02T13:51:00.245945+0000","last_active":"2024-04-02T13:51:56.478405+0000","last_peered":"2024-04-02T13:51:56.478405+0000","last_clean":"2024-04-02T13:51:56.478405+0000","last_became_active":"2024-04-02T13:51:00.245528+0000","last_became_peered":"2024-04-02T13:51:00.245528+0000","last_unstale":"2024-04-02T13:51:56.478405+0000","last_undegraded":"2024-04-02T13:51:56.478405+0000","last_fullsized":"2024-04-02T13:51:56.478405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:37:49.263614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662471+0000","last_change":"2024-04-02T13:51:01.260268+0000","last_active":"2024-04-02T13:51:56.662471+0000","last_peered":"2024-04-02T13:51:56.662471+0000","last_clean":"2024-04-02T13:51:56.662471+0000","last_became_active":"2024-04-02T13:51:01.258182+0000","last_became_peered":"2024-04-02T13:51:01.258182+0000","last_unstale":"2024-04-02T13:51:56.662471+0000","last_undegraded":"2024-04-02T13:51:56.662471+0000","last_fullsized":"2024-04-02T13:51:56.662471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:31:31.423988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477836+0000","last_change":"2024-04-02T13:51:05.518597+0000","last_active":"2024-04-02T13:51:56.477836+0000","last_peered":"2024-04-02T13:51:56.477836+0000","last_clean":"2024-04-02T13:51:56.477836+0000","last_became_active":"2024-04-02T13:51:05.518134+0000","last_became_peered":"2024-04-02T13:51:05.518134+0000","last_unstale":"2024-04-02T13:51:56.477836+0000","last_undegraded":"2024-04-02T13:51:56.477836+0000","last_fullsized":"2024-04-02T13:51:56.477836+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:00:38.632939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663486+0000","last_change":"2024-04-02T13:51:00.241680+0000","last_active":"2024-04-02T13:51:56.663486+0000","last_peered":"2024-04-02T13:51:56.663486+0000","last_clean":"2024-04-02T13:51:56.663486+0000","last_became_active":"2024-04-02T13:51:00.241301+0000","last_became_peered":"2024-04-02T13:51:00.241301+0000","last_unstale":"2024-04-02T13:51:56.663486+0000","last_undegraded":"2024-04-02T13:51:56.663486+0000","last_fullsized":"2024-04-02T13:51:56.663486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:12:10.606973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.476132+0000","last_change":"2024-04-02T13:51:01.254795+0000","last_active":"2024-04-02T13:51:56.476132+0000","last_peered":"2024-04-02T13:51:56.476132+0000","last_clean":"2024-04-02T13:51:56.476132+0000","last_became_active":"2024-04-02T13:51:01.254451+0000","last_became_peered":"2024-04-02T13:51:01.254451+0000","last_unstale":"2024-04-02T13:51:56.476132+0000","last_undegraded":"2024-04-02T13:51:56.476132+0000","last_fullsized":"2024-04-02T13:51:56.476132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:58:04.202996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.475885+0000","last_change":"2024-04-02T13:51:05.363592+0000","last_active":"2024-04-02T13:51:56.475885+0000","last_peered":"2024-04-02T13:51:56.475885+0000","last_clean":"2024-04-02T13:51:56.475885+0000","last_became_active":"2024-04-02T13:51:05.363211+0000","last_became_peered":"2024-04-02T13:51:05.363211+0000","last_unstale":"2024-04-02T13:51:56.475885+0000","last_undegraded":"2024-04-02T13:51:56.475885+0000","last_fullsized":"2024-04-02T13:51:56.475885+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:31:56.561505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.481783+0000","last_change":"2024-04-02T13:51:00.243681+0000","last_active":"2024-04-02T13:51:56.481783+0000","last_peered":"2024-04-02T13:51:56.481783+0000","last_clean":"2024-04-02T13:51:56.481783+0000","last_became_active":"2024-04-02T13:51:00.241615+0000","last_became_peered":"2024-04-02T13:51:00.241615+0000","last_unstale":"2024-04-02T13:51:56.481783+0000","last_undegraded":"2024-04-02T13:51:56.481783+0000","last_fullsized":"2024-04-02T13:51:56.481783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:54:30.112735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.478600+0000","last_change":"2024-04-02T13:51:01.252429+0000","last_active":"2024-04-02T13:51:56.478600+0000","last_peered":"2024-04-02T13:51:56.478600+0000","last_clean":"2024-04-02T13:51:56.478600+0000","last_became_active":"2024-04-02T13:51:01.252017+0000","last_became_peered":"2024-04-02T13:51:01.252017+0000","last_unstale":"2024-04-02T13:51:56.478600+0000","last_undegraded":"2024-04-02T13:51:56.478600+0000","last_fullsized":"2024-04-02T13:51:56.478600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:32:02.152578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477700+0000","last_change":"2024-04-02T13:51:05.361297+0000","last_active":"2024-04-02T13:51:56.477700+0000","last_peered":"2024-04-02T13:51:56.477700+0000","last_clean":"2024-04-02T13:51:56.477700+0000","last_became_active":"2024-04-02T13:51:05.360916+0000","last_became_peered":"2024-04-02T13:51:05.360916+0000","last_unstale":"2024-04-02T13:51:56.477700+0000","last_undegraded":"2024-04-02T13:51:56.477700+0000","last_fullsized":"2024-04-02T13:51:56.477700+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:52:48.608587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.488312+0000","last_change":"2024-04-02T13:51:56.488312+0000","last_active":"2024-04-02T13:51:56.488312+0000","last_peered":"2024-04-02T13:51:56.488312+0000","last_clean":"2024-04-02T13:51:56.488312+0000","last_became_active":"2024-04-02T13:51:56.483987+0000","last_became_peered":"2024-04-02T13:51:56.483987+0000","last_unstale":"2024-04-02T13:51:56.488312+0000","last_undegraded":"2024-04-02T13:51:56.488312+0000","last_fullsized":"2024-04-02T13:51:56.488312+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:46:27.396997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.482196+0000","last_change":"2024-04-02T13:51:01.255557+0000","last_active":"2024-04-02T13:51:56.482196+0000","last_peered":"2024-04-02T13:51:56.482196+0000","last_clean":"2024-04-02T13:51:56.482196+0000","last_became_active":"2024-04-02T13:51:01.255011+0000","last_became_peered":"2024-04-02T13:51:01.255011+0000","last_unstale":"2024-04-02T13:51:56.482196+0000","last_undegraded":"2024-04-02T13:51:56.482196+0000","last_fullsized":"2024-04-02T13:51:56.482196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:01:52.105965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477282+0000","last_change":"2024-04-02T13:51:05.518764+0000","last_active":"2024-04-02T13:51:56.477282+0000","last_peered":"2024-04-02T13:51:56.477282+0000","last_clean":"2024-04-02T13:51:56.477282+0000","last_became_active":"2024-04-02T13:51:05.518088+0000","last_became_peered":"2024-04-02T13:51:05.518088+0000","last_unstale":"2024-04-02T13:51:56.477282+0000","last_undegraded":"2024-04-02T13:51:56.477282+0000","last_fullsized":"2024-04-02T13:51:56.477282+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:03:02.901384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.476603+0000","last_change":"2024-04-02T13:51:00.244147+0000","last_active":"2024-04-02T13:51:56.476603+0000","last_peered":"2024-04-02T13:51:56.476603+0000","last_clean":"2024-04-02T13:51:56.476603+0000","last_became_active":"2024-04-02T13:51:00.243763+0000","last_became_peered":"2024-04-02T13:51:00.243763+0000","last_unstale":"2024-04-02T13:51:56.476603+0000","last_undegraded":"2024-04-02T13:51:56.476603+0000","last_fullsized":"2024-04-02T13:51:56.476603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:34:27.738746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663978+0000","last_change":"2024-04-02T13:51:01.268121+0000","last_active":"2024-04-02T13:51:56.663978+0000","last_peered":"2024-04-02T13:51:56.663978+0000","last_clean":"2024-04-02T13:51:56.663978+0000","last_became_active":"2024-04-02T13:51:01.266363+0000","last_became_peered":"2024-04-02T13:51:01.266363+0000","last_unstale":"2024-04-02T13:51:56.663978+0000","last_undegraded":"2024-04-02T13:51:56.663978+0000","last_fullsized":"2024-04-02T13:51:56.663978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:30:50.254916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662794+0000","last_change":"2024-04-02T13:51:05.361285+0000","last_active":"2024-04-02T13:51:56.662794+0000","last_peered":"2024-04-02T13:51:56.662794+0000","last_clean":"2024-04-02T13:51:56.662794+0000","last_became_active":"2024-04-02T13:51:05.360122+0000","last_became_peered":"2024-04-02T13:51:05.360122+0000","last_unstale":"2024-04-02T13:51:56.662794+0000","last_undegraded":"2024-04-02T13:51:56.662794+0000","last_fullsized":"2024-04-02T13:51:56.662794+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:30:59.935538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'6","reported_seq":106,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:52:38.441941+0000","last_change":"2024-04-02T13:51:00.247235+0000","last_active":"2024-04-02T13:52:38.441941+0000","last_peered":"2024-04-02T13:52:38.441941+0000","last_clean":"2024-04-02T13:52:38.441941+0000","last_became_active":"2024-04-02T13:51:00.246856+0000","last_became_peered":"2024-04-02T13:51:00.246856+0000","last_unstale":"2024-04-02T13:52:38.441941+0000","last_undegraded":"2024-04-02T13:52:38.441941+0000","last_fullsized":"2024-04-02T13:52:38.441941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:17:51.875952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663524+0000","last_change":"2024-04-02T13:51:01.263866+0000","last_active":"2024-04-02T13:51:56.663524+0000","last_peered":"2024-04-02T13:51:56.663524+0000","last_clean":"2024-04-02T13:51:56.663524+0000","last_became_active":"2024-04-02T13:51:01.263564+0000","last_became_peered":"2024-04-02T13:51:01.263564+0000","last_unstale":"2024-04-02T13:51:56.663524+0000","last_undegraded":"2024-04-02T13:51:56.663524+0000","last_fullsized":"2024-04-02T13:51:56.663524+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:59:33.663277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662098+0000","last_change":"2024-04-02T13:51:05.363079+0000","last_active":"2024-04-02T13:51:56.662098+0000","last_peered":"2024-04-02T13:51:56.662098+0000","last_clean":"2024-04-02T13:51:56.662098+0000","last_became_active":"2024-04-02T13:51:05.362631+0000","last_became_peered":"2024-04-02T13:51:05.362631+0000","last_unstale":"2024-04-02T13:51:56.662098+0000","last_undegraded":"2024-04-02T13:51:56.662098+0000","last_fullsized":"2024-04-02T13:51:56.662098+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:23.656269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662986+0000","last_change":"2024-04-02T13:51:00.242820+0000","last_active":"2024-04-02T13:51:56.662986+0000","last_peered":"2024-04-02T13:51:56.662986+0000","last_clean":"2024-04-02T13:51:56.662986+0000","last_became_active":"2024-04-02T13:51:00.242262+0000","last_became_peered":"2024-04-02T13:51:00.242262+0000","last_unstale":"2024-04-02T13:51:56.662986+0000","last_undegraded":"2024-04-02T13:51:56.662986+0000","last_fullsized":"2024-04-02T13:51:56.662986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:12:22.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663311+0000","last_change":"2024-04-02T13:51:01.266548+0000","last_active":"2024-04-02T13:51:56.663311+0000","last_peered":"2024-04-02T13:51:56.663311+0000","last_clean":"2024-04-02T13:51:56.663311+0000","last_became_active":"2024-04-02T13:51:01.259003+0000","last_became_peered":"2024-04-02T13:51:01.259003+0000","last_unstale":"2024-04-02T13:51:56.663311+0000","last_undegraded":"2024-04-02T13:51:56.663311+0000","last_fullsized":"2024-04-02T13:51:56.663311+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:38:53.692231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.661976+0000","last_change":"2024-04-02T13:51:05.359994+0000","last_active":"2024-04-02T13:51:56.661976+0000","last_peered":"2024-04-02T13:51:56.661976+0000","last_clean":"2024-04-02T13:51:56.661976+0000","last_became_active":"2024-04-02T13:51:05.359438+0000","last_became_peered":"2024-04-02T13:51:05.359438+0000","last_unstale":"2024-04-02T13:51:56.661976+0000","last_undegraded":"2024-04-02T13:51:56.661976+0000","last_fullsized":"2024-04-02T13:51:56.661976+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:09:43.929446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.555936+0000","last_change":"2024-04-02T13:51:00.237889+0000","last_active":"2024-04-02T13:51:56.555936+0000","last_peered":"2024-04-02T13:51:56.555936+0000","last_clean":"2024-04-02T13:51:56.555936+0000","last_became_active":"2024-04-02T13:51:00.237428+0000","last_became_peered":"2024-04-02T13:51:00.237428+0000","last_unstale":"2024-04-02T13:51:56.555936+0000","last_undegraded":"2024-04-02T13:51:56.555936+0000","last_fullsized":"2024-04-02T13:51:56.555936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:21:52.483656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663377+0000","last_change":"2024-04-02T13:51:01.262093+0000","last_active":"2024-04-02T13:51:56.663377+0000","last_peered":"2024-04-02T13:51:56.663377+0000","last_clean":"2024-04-02T13:51:56.663377+0000","last_became_active":"2024-04-02T13:51:01.258357+0000","last_became_peered":"2024-04-02T13:51:01.258357+0000","last_unstale":"2024-04-02T13:51:56.663377+0000","last_undegraded":"2024-04-02T13:51:56.663377+0000","last_fullsized":"2024-04-02T13:51:56.663377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:00:06.317307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.483560+0000","last_change":"2024-04-02T13:51:56.483560+0000","last_active":"2024-04-02T13:51:56.483560+0000","last_peered":"2024-04-02T13:51:56.483560+0000","last_clean":"2024-04-02T13:51:56.483560+0000","last_became_active":"2024-04-02T13:51:56.483090+0000","last_became_peered":"2024-04-02T13:51:56.483090+0000","last_unstale":"2024-04-02T13:51:56.483560+0000","last_undegraded":"2024-04-02T13:51:56.483560+0000","last_fullsized":"2024-04-02T13:51:56.483560+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:09:35.640492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663817+0000","last_change":"2024-04-02T13:51:56.663817+0000","last_active":"2024-04-02T13:51:56.663817+0000","last_peered":"2024-04-02T13:51:56.663817+0000","last_clean":"2024-04-02T13:51:56.663817+0000","last_became_active":"2024-04-02T13:51:56.663473+0000","last_became_peered":"2024-04-02T13:51:56.663473+0000","last_unstale":"2024-04-02T13:51:56.663817+0000","last_undegraded":"2024-04-02T13:51:56.663817+0000","last_fullsized":"2024-04-02T13:51:56.663817+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:26:15.566030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477283+0000","last_change":"2024-04-02T13:51:01.255563+0000","last_active":"2024-04-02T13:51:56.477283+0000","last_peered":"2024-04-02T13:51:56.477283+0000","last_clean":"2024-04-02T13:51:56.477283+0000","last_became_active":"2024-04-02T13:51:01.255228+0000","last_became_peered":"2024-04-02T13:51:01.255228+0000","last_unstale":"2024-04-02T13:51:56.477283+0000","last_undegraded":"2024-04-02T13:51:56.477283+0000","last_fullsized":"2024-04-02T13:51:56.477283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:48:28.200776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662423+0000","last_change":"2024-04-02T13:51:01.258153+0000","last_active":"2024-04-02T13:51:56.662423+0000","last_peered":"2024-04-02T13:51:56.662423+0000","last_clean":"2024-04-02T13:51:56.662423+0000","last_became_active":"2024-04-02T13:51:01.257656+0000","last_became_peered":"2024-04-02T13:51:01.257656+0000","last_unstale":"2024-04-02T13:51:56.662423+0000","last_undegraded":"2024-04-02T13:51:56.662423+0000","last_fullsized":"2024-04-02T13:51:56.662423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:37.887427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.556030+0000","last_change":"2024-04-02T13:51:01.254675+0000","last_active":"2024-04-02T13:51:56.556030+0000","last_peered":"2024-04-02T13:51:56.556030+0000","last_clean":"2024-04-02T13:51:56.556030+0000","last_became_active":"2024-04-02T13:51:01.254336+0000","last_became_peered":"2024-04-02T13:51:01.254336+0000","last_unstale":"2024-04-02T13:51:56.556030+0000","last_undegraded":"2024-04-02T13:51:56.556030+0000","last_fullsized":"2024-04-02T13:51:56.556030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:36.171961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.479161+0000","last_change":"2024-04-02T13:51:01.259457+0000","last_active":"2024-04-02T13:51:56.479161+0000","last_peered":"2024-04-02T13:51:56.479161+0000","last_clean":"2024-04-02T13:51:56.479161+0000","last_became_active":"2024-04-02T13:51:01.259099+0000","last_became_peered":"2024-04-02T13:51:01.259099+0000","last_unstale":"2024-04-02T13:51:56.479161+0000","last_undegraded":"2024-04-02T13:51:56.479161+0000","last_fullsized":"2024-04-02T13:51:56.479161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:13:06.594132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.478351+0000","last_change":"2024-04-02T13:51:01.248177+0000","last_active":"2024-04-02T13:51:56.478351+0000","last_peered":"2024-04-02T13:51:56.478351+0000","last_clean":"2024-04-02T13:51:56.478351+0000","last_became_active":"2024-04-02T13:51:01.247843+0000","last_became_peered":"2024-04-02T13:51:01.247843+0000","last_unstale":"2024-04-02T13:51:56.478351+0000","last_undegraded":"2024-04-02T13:51:56.478351+0000","last_fullsized":"2024-04-02T13:51:56.478351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:38:05.805382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.482303+0000","last_change":"2024-04-02T13:51:01.254822+0000","last_active":"2024-04-02T13:51:56.482303+0000","last_peered":"2024-04-02T13:51:56.482303+0000","last_clean":"2024-04-02T13:51:56.482303+0000","last_became_active":"2024-04-02T13:51:01.254451+0000","last_became_peered":"2024-04-02T13:51:01.254451+0000","last_unstale":"2024-04-02T13:51:56.482303+0000","last_undegraded":"2024-04-02T13:51:56.482303+0000","last_fullsized":"2024-04-02T13:51:56.482303+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:15:35.790371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662337+0000","last_change":"2024-04-02T13:51:01.265467+0000","last_active":"2024-04-02T13:51:56.662337+0000","last_peered":"2024-04-02T13:51:56.662337+0000","last_clean":"2024-04-02T13:51:56.662337+0000","last_became_active":"2024-04-02T13:51:01.265072+0000","last_became_peered":"2024-04-02T13:51:01.265072+0000","last_unstale":"2024-04-02T13:51:56.662337+0000","last_undegraded":"2024-04-02T13:51:56.662337+0000","last_fullsized":"2024-04-02T13:51:56.662337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:12:07.702222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.481983+0000","last_change":"2024-04-02T13:51:01.252844+0000","last_active":"2024-04-02T13:51:56.481983+0000","last_peered":"2024-04-02T13:51:56.481983+0000","last_clean":"2024-04-02T13:51:56.481983+0000","last_became_active":"2024-04-02T13:51:01.252515+0000","last_became_peered":"2024-04-02T13:51:01.252515+0000","last_unstale":"2024-04-02T13:51:56.481983+0000","last_undegraded":"2024-04-02T13:51:56.481983+0000","last_fullsized":"2024-04-02T13:51:56.481983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:20:16.216418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.664080+0000","last_change":"2024-04-02T13:51:01.260141+0000","last_active":"2024-04-02T13:51:56.664080+0000","last_peered":"2024-04-02T13:51:56.664080+0000","last_clean":"2024-04-02T13:51:56.664080+0000","last_became_active":"2024-04-02T13:51:01.259530+0000","last_became_peered":"2024-04-02T13:51:01.259530+0000","last_unstale":"2024-04-02T13:51:56.664080+0000","last_undegraded":"2024-04-02T13:51:56.664080+0000","last_fullsized":"2024-04-02T13:51:56.664080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:00:03.579414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.556020+0000","last_change":"2024-04-02T13:51:01.259981+0000","last_active":"2024-04-02T13:51:56.556020+0000","last_peered":"2024-04-02T13:51:56.556020+0000","last_clean":"2024-04-02T13:51:56.556020+0000","last_became_active":"2024-04-02T13:51:01.259647+0000","last_became_peered":"2024-04-02T13:51:01.259647+0000","last_unstale":"2024-04-02T13:51:56.556020+0000","last_undegraded":"2024-04-02T13:51:56.556020+0000","last_fullsized":"2024-04-02T13:51:56.556020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:10:22.753767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.479479+0000","last_change":"2024-04-02T13:51:01.255342+0000","last_active":"2024-04-02T13:51:56.479479+0000","last_peered":"2024-04-02T13:51:56.479479+0000","last_clean":"2024-04-02T13:51:56.479479+0000","last_became_active":"2024-04-02T13:51:01.254679+0000","last_became_peered":"2024-04-02T13:51:01.254679+0000","last_unstale":"2024-04-02T13:51:56.479479+0000","last_undegraded":"2024-04-02T13:51:56.479479+0000","last_fullsized":"2024-04-02T13:51:56.479479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:25:12.198270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69146,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":748,"num_read_kb":449,"num_write":240,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":207438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351010,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":507658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612647,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28720,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716432,"statfs":{"total":95995035648,"available":95965626368,"internally_reserved":0,"allocated":1740800,"data_stored":1156348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776172,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":511225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037810,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":507507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332151,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36968,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708184,"statfs":{"total":95995035648,"available":95957180416,"internally_reserved":0,"allocated":1662976,"data_stored":1103613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495675,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":502633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1114112,"data_stored":565996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":504480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36440,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708712,"statfs":{"total":95995035648,"available":95957721088,"internally_reserved":0,"allocated":1126400,"data_stored":566539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1056768,"data_stored":502341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":506707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1671168,"data_stored":1099195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T13:52:43.904 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph pg dump --format=json 2024-04-02T13:52:44.152 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:52:44.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:44 smithi052 ceph-mon[32104]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T13:52:44.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:44 smithi052 ceph-mon[32104]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:52:44.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:44 smithi189 ceph-mon[36683]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T13:52:44.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:44 smithi189 ceph-mon[36683]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:52:44.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:44 smithi088 ceph-mon[36528]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T13:52:44.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:44 smithi088 ceph-mon[36528]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:52:45.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.942+0000 7f59600d7700 1 -- 172.21.15.52:0/2363554033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5958102670 msgr2=0x7f5958102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:45.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.942+0000 7f59600d7700 1 --2- 172.21.15.52:0/2363554033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5958102670 0x7f5958102af0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f59480099c0 tx=0x7f5948009cd0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.942+0000 7f59600d7700 1 -- 172.21.15.52:0/2363554033 shutdown_connections 2024-04-02T13:52:45.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.942+0000 7f59600d7700 1 --2- 172.21.15.52:0/2363554033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5958103030 0x7f595810f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.942+0000 7f59600d7700 1 --2- 172.21.15.52:0/2363554033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5958108690 0x7f5958108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.942+0000 7f59600d7700 1 --2- 172.21.15.52:0/2363554033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5958102670 0x7f5958102af0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.942+0000 7f59600d7700 1 -- 172.21.15.52:0/2363554033 >> 172.21.15.52:0/2363554033 conn(0x7f59580796b0 msgr2=0x7f5958079ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:45.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.942+0000 7f59600d7700 1 -- 172.21.15.52:0/2363554033 shutdown_connections 2024-04-02T13:52:45.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.942+0000 7f59600d7700 1 -- 172.21.15.52:0/2363554033 wait complete. 2024-04-02T13:52:45.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.943+0000 7f59600d7700 1 Processor -- start 2024-04-02T13:52:45.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.943+0000 7f59600d7700 1 -- start start 2024-04-02T13:52:45.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.943+0000 7f59600d7700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5958102670 0x7f59580748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:45.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.943+0000 7f59600d7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5958103030 0x7f5958074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:45.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.943+0000 7f59600d7700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5958108690 0x7f5958075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:45.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.943+0000 7f59600d7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5958112930 con 0x7f5958103030 2024-04-02T13:52:45.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.943+0000 7f59600d7700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f59581127b0 con 0x7f5958108690 2024-04-02T13:52:45.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.943+0000 7f59600d7700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5958112ab0 con 0x7f5958102670 2024-04-02T13:52:45.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.943+0000 7f595d672700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5958103030 0x7f5958074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:45.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.943+0000 7f595d672700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5958103030 0x7f5958074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59930/0 (socket says 172.21.15.52:59930) 2024-04-02T13:52:45.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.943+0000 7f595d672700 1 -- 172.21.15.52:0/2831836558 learned_addr learned my addr 172.21.15.52:0/2831836558 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:45.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.943+0000 7f595e674700 1 --2- 172.21.15.52:0/2831836558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5958108690 0x7f5958075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:45.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.943+0000 7f595de73700 1 --2- 172.21.15.52:0/2831836558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5958102670 0x7f59580748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:45.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.943+0000 7f595d672700 1 -- 172.21.15.52:0/2831836558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5958102670 msgr2=0x7f59580748f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:45.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.943+0000 7f595d672700 1 --2- 172.21.15.52:0/2831836558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5958102670 0x7f59580748f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.943+0000 7f595d672700 1 -- 172.21.15.52:0/2831836558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5958108690 msgr2=0x7f5958075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:45.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.943+0000 7f595d672700 1 --2- 172.21.15.52:0/2831836558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5958108690 0x7f5958075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.943+0000 7f595d672700 1 -- 172.21.15.52:0/2831836558 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59581a7210 con 0x7f5958103030 2024-04-02T13:52:45.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.943+0000 7f595d672700 1 --2- 172.21.15.52:0/2831836558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5958103030 0x7f5958074e30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f59480059d0 tx=0x7f594800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:45.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.944+0000 7f594effd700 1 -- 172.21.15.52:0/2831836558 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5948027070 con 0x7f5958103030 2024-04-02T13:52:45.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.944+0000 7f59600d7700 1 -- 172.21.15.52:0/2831836558 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59581a74a0 con 0x7f5958103030 2024-04-02T13:52:45.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.944+0000 7f594effd700 1 -- 172.21.15.52:0/2831836558 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5948003680 con 0x7f5958103030 2024-04-02T13:52:45.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.944+0000 7f594effd700 1 -- 172.21.15.52:0/2831836558 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59480173f0 con 0x7f5958103030 2024-04-02T13:52:45.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.944+0000 7f59600d7700 1 -- 172.21.15.52:0/2831836558 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59581a79d0 con 0x7f5958103030 2024-04-02T13:52:45.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.945+0000 7f59600d7700 1 -- 172.21.15.52:0/2831836558 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59580692b0 con 0x7f5958103030 2024-04-02T13:52:45.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.946+0000 7f594effd700 1 -- 172.21.15.52:0/2831836558 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f594802a070 con 0x7f5958103030 2024-04-02T13:52:45.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.946+0000 7f594effd700 1 --2- 172.21.15.52:0/2831836558 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f59440a3cd0 0x7f59440a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:45.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.947+0000 7f595de73700 1 --2- 172.21.15.52:0/2831836558 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f59440a3cd0 0x7f59440a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:45.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.947+0000 7f594effd700 1 -- 172.21.15.52:0/2831836558 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f59480d1450 con 0x7f5958103030 2024-04-02T13:52:45.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.947+0000 7f595de73700 1 --2- 172.21.15.52:0/2831836558 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f59440a3cd0 0x7f59440a6190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5954009be0 tx=0x7f5954009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:45.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:45.950+0000 7f594effd700 1 -- 172.21.15.52:0/2831836558 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5948013070 con 0x7f5958103030 2024-04-02T13:52:46.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:46.150+0000 7f59600d7700 1 -- 172.21.15.52:0/2831836558 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5958062a30 con 0x7f59440a3cd0 2024-04-02T13:52:46.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:46.159+0000 7f594effd700 1 -- 172.21.15.52:0/2831836558 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340419 (secure 0 0 0) 0x7f5958062a30 con 0x7f59440a3cd0 2024-04-02T13:52:46.168 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:52:46.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:46.163+0000 7f59600d7700 1 -- 172.21.15.52:0/2831836558 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f59440a3cd0 msgr2=0x7f59440a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:46.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:46.163+0000 7f59600d7700 1 --2- 172.21.15.52:0/2831836558 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f59440a3cd0 0x7f59440a6190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5954009be0 tx=0x7f5954009260 comp rx=0 tx=0).stop 2024-04-02T13:52:46.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:46.163+0000 7f59600d7700 1 -- 172.21.15.52:0/2831836558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5958103030 msgr2=0x7f5958074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:46.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:46.163+0000 7f59600d7700 1 --2- 172.21.15.52:0/2831836558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5958103030 0x7f5958074e30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f59480059d0 tx=0x7f594800fa00 comp rx=0 tx=0).stop 2024-04-02T13:52:46.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:46.164+0000 7f59600d7700 1 -- 172.21.15.52:0/2831836558 shutdown_connections 2024-04-02T13:52:46.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:46.164+0000 7f59600d7700 1 --2- 172.21.15.52:0/2831836558 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f59440a3cd0 0x7f59440a6190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:46.164+0000 7f59600d7700 1 --2- 172.21.15.52:0/2831836558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5958108690 0x7f5958075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:46.164+0000 7f59600d7700 1 --2- 172.21.15.52:0/2831836558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5958102670 0x7f59580748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:46.164+0000 7f59600d7700 1 --2- 172.21.15.52:0/2831836558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5958103030 0x7f5958074e30 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:46.164+0000 7f59600d7700 1 -- 172.21.15.52:0/2831836558 >> 172.21.15.52:0/2831836558 conn(0x7f59580796b0 msgr2=0x7f59581012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:46.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:46.164+0000 7f59600d7700 1 -- 172.21.15.52:0/2831836558 shutdown_connections 2024-04-02T13:52:46.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:46.164+0000 7f59600d7700 1 -- 172.21.15.52:0/2831836558 wait complete. 2024-04-02T13:52:46.175 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-04-02T13:52:46.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:46 smithi189 ceph-mon[36683]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:52:46.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:46 smithi052 ceph-mon[32104]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:52:46.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:46 smithi088 ceph-mon[36528]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:52:46.980 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-04-02T13:52:44.763210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659748,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":889,"num_read_kb":573,"num_write":423,"num_write_kb":3080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":405216,"kb_used_data":14384,"kb_used_omap":18,"kb_used_meta":390637,"kb_avail":1124536608,"statfs":{"total":1151940427776,"available":1151525486592,"internally_reserved":0,"allocated":14729216,"data_stored":8034242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":400012602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003602"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.478629+0000","last_change":"2024-04-02T13:51:01.253563+0000","last_active":"2024-04-02T13:51:56.478629+0000","last_peered":"2024-04-02T13:51:56.478629+0000","last_clean":"2024-04-02T13:51:56.478629+0000","last_became_active":"2024-04-02T13:51:01.253185+0000","last_became_peered":"2024-04-02T13:51:01.253185+0000","last_unstale":"2024-04-02T13:51:56.478629+0000","last_undegraded":"2024-04-02T13:51:56.478629+0000","last_fullsized":"2024-04-02T13:51:56.478629+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:51:05.301880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662449+0000","last_change":"2024-04-02T13:51:01.256406+0000","last_active":"2024-04-02T13:51:56.662449+0000","last_peered":"2024-04-02T13:51:56.662449+0000","last_clean":"2024-04-02T13:51:56.662449+0000","last_became_active":"2024-04-02T13:51:01.256068+0000","last_became_peered":"2024-04-02T13:51:01.256068+0000","last_unstale":"2024-04-02T13:51:56.662449+0000","last_undegraded":"2024-04-02T13:51:56.662449+0000","last_fullsized":"2024-04-02T13:51:56.662449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:56:51.184932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662208+0000","last_change":"2024-04-02T13:51:01.265925+0000","last_active":"2024-04-02T13:51:56.662208+0000","last_peered":"2024-04-02T13:51:56.662208+0000","last_clean":"2024-04-02T13:51:56.662208+0000","last_became_active":"2024-04-02T13:51:01.265587+0000","last_became_peered":"2024-04-02T13:51:01.265587+0000","last_unstale":"2024-04-02T13:51:56.662208+0000","last_undegraded":"2024-04-02T13:51:56.662208+0000","last_fullsized":"2024-04-02T13:51:56.662208+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:40:59.426886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.483007+0000","last_change":"2024-04-02T13:51:56.482512+0000","last_active":"2024-04-02T13:51:56.483007+0000","last_peered":"2024-04-02T13:51:56.483007+0000","last_clean":"2024-04-02T13:51:56.483007+0000","last_became_active":"2024-04-02T13:51:56.481439+0000","last_became_peered":"2024-04-02T13:51:56.481439+0000","last_unstale":"2024-04-02T13:51:56.483007+0000","last_undegraded":"2024-04-02T13:51:56.483007+0000","last_fullsized":"2024-04-02T13:51:56.483007+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:23:16.969641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477134+0000","last_change":"2024-04-02T13:51:01.250934+0000","last_active":"2024-04-02T13:51:56.477134+0000","last_peered":"2024-04-02T13:51:56.477134+0000","last_clean":"2024-04-02T13:51:56.477134+0000","last_became_active":"2024-04-02T13:51:01.250546+0000","last_became_peered":"2024-04-02T13:51:01.250546+0000","last_unstale":"2024-04-02T13:51:56.477134+0000","last_undegraded":"2024-04-02T13:51:56.477134+0000","last_fullsized":"2024-04-02T13:51:56.477134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:34:17.839207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.476637+0000","last_change":"2024-04-02T13:51:01.256160+0000","last_active":"2024-04-02T13:51:56.476637+0000","last_peered":"2024-04-02T13:51:56.476637+0000","last_clean":"2024-04-02T13:51:56.476637+0000","last_became_active":"2024-04-02T13:51:01.255776+0000","last_became_peered":"2024-04-02T13:51:01.255776+0000","last_unstale":"2024-04-02T13:51:56.476637+0000","last_undegraded":"2024-04-02T13:51:56.476637+0000","last_fullsized":"2024-04-02T13:51:56.476637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:53:16.724662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663631+0000","last_change":"2024-04-02T13:51:01.263049+0000","last_active":"2024-04-02T13:51:56.663631+0000","last_peered":"2024-04-02T13:51:56.663631+0000","last_clean":"2024-04-02T13:51:56.663631+0000","last_became_active":"2024-04-02T13:51:01.262660+0000","last_became_peered":"2024-04-02T13:51:01.262660+0000","last_unstale":"2024-04-02T13:51:56.663631+0000","last_undegraded":"2024-04-02T13:51:56.663631+0000","last_fullsized":"2024-04-02T13:51:56.663631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:15:32.069188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.481755+0000","last_change":"2024-04-02T13:51:01.263064+0000","last_active":"2024-04-02T13:51:56.481755+0000","last_peered":"2024-04-02T13:51:56.481755+0000","last_clean":"2024-04-02T13:51:56.481755+0000","last_became_active":"2024-04-02T13:51:01.262706+0000","last_became_peered":"2024-04-02T13:51:01.262706+0000","last_unstale":"2024-04-02T13:51:56.481755+0000","last_undegraded":"2024-04-02T13:51:56.481755+0000","last_fullsized":"2024-04-02T13:51:56.481755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:46.680723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663840+0000","last_change":"2024-04-02T13:51:01.257347+0000","last_active":"2024-04-02T13:51:56.663840+0000","last_peered":"2024-04-02T13:51:56.663840+0000","last_clean":"2024-04-02T13:51:56.663840+0000","last_became_active":"2024-04-02T13:51:01.257001+0000","last_became_peered":"2024-04-02T13:51:01.257001+0000","last_unstale":"2024-04-02T13:51:56.663840+0000","last_undegraded":"2024-04-02T13:51:56.663840+0000","last_fullsized":"2024-04-02T13:51:56.663840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:31:22.628436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477954+0000","last_change":"2024-04-02T13:51:01.260533+0000","last_active":"2024-04-02T13:51:56.477954+0000","last_peered":"2024-04-02T13:51:56.477954+0000","last_clean":"2024-04-02T13:51:56.477954+0000","last_became_active":"2024-04-02T13:51:01.260203+0000","last_became_peered":"2024-04-02T13:51:01.260203+0000","last_unstale":"2024-04-02T13:51:56.477954+0000","last_undegraded":"2024-04-02T13:51:56.477954+0000","last_fullsized":"2024-04-02T13:51:56.477954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:00:06.008243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663416+0000","last_change":"2024-04-02T13:51:01.267733+0000","last_active":"2024-04-02T13:51:56.663416+0000","last_peered":"2024-04-02T13:51:56.663416+0000","last_clean":"2024-04-02T13:51:56.663416+0000","last_became_active":"2024-04-02T13:51:01.266642+0000","last_became_peered":"2024-04-02T13:51:01.266642+0000","last_unstale":"2024-04-02T13:51:56.663416+0000","last_undegraded":"2024-04-02T13:51:56.663416+0000","last_fullsized":"2024-04-02T13:51:56.663416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:39:25.696112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477899+0000","last_change":"2024-04-02T13:51:01.255933+0000","last_active":"2024-04-02T13:51:56.477899+0000","last_peered":"2024-04-02T13:51:56.477899+0000","last_clean":"2024-04-02T13:51:56.477899+0000","last_became_active":"2024-04-02T13:51:01.255558+0000","last_became_peered":"2024-04-02T13:51:01.255558+0000","last_unstale":"2024-04-02T13:51:56.477899+0000","last_undegraded":"2024-04-02T13:51:56.477899+0000","last_fullsized":"2024-04-02T13:51:56.477899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:13:11.537415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.555999+0000","last_change":"2024-04-02T13:51:01.256242+0000","last_active":"2024-04-02T13:51:56.555999+0000","last_peered":"2024-04-02T13:51:56.555999+0000","last_clean":"2024-04-02T13:51:56.555999+0000","last_became_active":"2024-04-02T13:51:01.255830+0000","last_became_peered":"2024-04-02T13:51:01.255830+0000","last_unstale":"2024-04-02T13:51:56.555999+0000","last_undegraded":"2024-04-02T13:51:56.555999+0000","last_fullsized":"2024-04-02T13:51:56.555999+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:39.130894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662031+0000","last_change":"2024-04-02T13:51:01.259738+0000","last_active":"2024-04-02T13:51:56.662031+0000","last_peered":"2024-04-02T13:51:56.662031+0000","last_clean":"2024-04-02T13:51:56.662031+0000","last_became_active":"2024-04-02T13:51:01.258630+0000","last_became_peered":"2024-04-02T13:51:01.258630+0000","last_unstale":"2024-04-02T13:51:56.662031+0000","last_undegraded":"2024-04-02T13:51:56.662031+0000","last_fullsized":"2024-04-02T13:51:56.662031+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:12:05.401171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662138+0000","last_change":"2024-04-02T13:51:01.257684+0000","last_active":"2024-04-02T13:51:56.662138+0000","last_peered":"2024-04-02T13:51:56.662138+0000","last_clean":"2024-04-02T13:51:56.662138+0000","last_became_active":"2024-04-02T13:51:01.257267+0000","last_became_peered":"2024-04-02T13:51:01.257267+0000","last_unstale":"2024-04-02T13:51:56.662138+0000","last_undegraded":"2024-04-02T13:51:56.662138+0000","last_fullsized":"2024-04-02T13:51:56.662138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:43:45.728233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663370+0000","last_change":"2024-04-02T13:51:01.261720+0000","last_active":"2024-04-02T13:51:56.663370+0000","last_peered":"2024-04-02T13:51:56.663370+0000","last_clean":"2024-04-02T13:51:56.663370+0000","last_became_active":"2024-04-02T13:51:01.261407+0000","last_became_peered":"2024-04-02T13:51:01.261407+0000","last_unstale":"2024-04-02T13:51:56.663370+0000","last_undegraded":"2024-04-02T13:51:56.663370+0000","last_fullsized":"2024-04-02T13:51:56.663370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:54:58.807606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.478367+0000","last_change":"2024-04-02T13:51:01.254488+0000","last_active":"2024-04-02T13:51:56.478367+0000","last_peered":"2024-04-02T13:51:56.478367+0000","last_clean":"2024-04-02T13:51:56.478367+0000","last_became_active":"2024-04-02T13:51:01.254139+0000","last_became_peered":"2024-04-02T13:51:01.254139+0000","last_unstale":"2024-04-02T13:51:56.478367+0000","last_undegraded":"2024-04-02T13:51:56.478367+0000","last_fullsized":"2024-04-02T13:51:56.478367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:02:41.075727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.475629+0000","last_change":"2024-04-02T13:51:01.256107+0000","last_active":"2024-04-02T13:51:56.475629+0000","last_peered":"2024-04-02T13:51:56.475629+0000","last_clean":"2024-04-02T13:51:56.475629+0000","last_became_active":"2024-04-02T13:51:01.255754+0000","last_became_peered":"2024-04-02T13:51:01.255754+0000","last_unstale":"2024-04-02T13:51:56.475629+0000","last_undegraded":"2024-04-02T13:51:56.475629+0000","last_fullsized":"2024-04-02T13:51:56.475629+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:36:59.793519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477869+0000","last_change":"2024-04-02T13:51:01.247550+0000","last_active":"2024-04-02T13:51:56.477869+0000","last_peered":"2024-04-02T13:51:56.477869+0000","last_clean":"2024-04-02T13:51:56.477869+0000","last_became_active":"2024-04-02T13:51:01.247236+0000","last_became_peered":"2024-04-02T13:51:01.247236+0000","last_unstale":"2024-04-02T13:51:56.477869+0000","last_undegraded":"2024-04-02T13:51:56.477869+0000","last_fullsized":"2024-04-02T13:51:56.477869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:16:02.940700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663779+0000","last_change":"2024-04-02T13:51:56.663779+0000","last_active":"2024-04-02T13:51:56.663779+0000","last_peered":"2024-04-02T13:51:56.663779+0000","last_clean":"2024-04-02T13:51:56.663779+0000","last_became_active":"2024-04-02T13:51:56.663327+0000","last_became_peered":"2024-04-02T13:51:56.663327+0000","last_unstale":"2024-04-02T13:51:56.663779+0000","last_undegraded":"2024-04-02T13:51:56.663779+0000","last_fullsized":"2024-04-02T13:51:56.663779+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:35:34.186278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":115,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:52:41.444215+0000","last_change":"2024-04-02T13:51:00.243244+0000","last_active":"2024-04-02T13:52:41.444215+0000","last_peered":"2024-04-02T13:52:41.444215+0000","last_clean":"2024-04-02T13:52:41.444215+0000","last_became_active":"2024-04-02T13:51:00.242892+0000","last_became_peered":"2024-04-02T13:51:00.242892+0000","last_unstale":"2024-04-02T13:52:41.444215+0000","last_undegraded":"2024-04-02T13:52:41.444215+0000","last_fullsized":"2024-04-02T13:52:41.444215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:49:23.338215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.482338+0000","last_change":"2024-04-02T13:51:01.264640+0000","last_active":"2024-04-02T13:51:56.482338+0000","last_peered":"2024-04-02T13:51:56.482338+0000","last_clean":"2024-04-02T13:51:56.482338+0000","last_became_active":"2024-04-02T13:51:01.264161+0000","last_became_peered":"2024-04-02T13:51:01.264161+0000","last_unstale":"2024-04-02T13:51:56.482338+0000","last_undegraded":"2024-04-02T13:51:56.482338+0000","last_fullsized":"2024-04-02T13:51:56.482338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:30:47.245647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477397+0000","last_change":"2024-04-02T13:51:05.362888+0000","last_active":"2024-04-02T13:51:56.477397+0000","last_peered":"2024-04-02T13:51:56.477397+0000","last_clean":"2024-04-02T13:51:56.477397+0000","last_became_active":"2024-04-02T13:51:05.362478+0000","last_became_peered":"2024-04-02T13:51:05.362478+0000","last_unstale":"2024-04-02T13:51:56.477397+0000","last_undegraded":"2024-04-02T13:51:56.477397+0000","last_fullsized":"2024-04-02T13:51:56.477397+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:05:54.304458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.482423+0000","last_change":"2024-04-02T13:51:00.247800+0000","last_active":"2024-04-02T13:51:56.482423+0000","last_peered":"2024-04-02T13:51:56.482423+0000","last_clean":"2024-04-02T13:51:56.482423+0000","last_became_active":"2024-04-02T13:51:00.247485+0000","last_became_peered":"2024-04-02T13:51:00.247485+0000","last_unstale":"2024-04-02T13:51:56.482423+0000","last_undegraded":"2024-04-02T13:51:56.482423+0000","last_fullsized":"2024-04-02T13:51:56.482423+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:17:24.767558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662205+0000","last_change":"2024-04-02T13:51:01.261416+0000","last_active":"2024-04-02T13:51:56.662205+0000","last_peered":"2024-04-02T13:51:56.662205+0000","last_clean":"2024-04-02T13:51:56.662205+0000","last_became_active":"2024-04-02T13:51:01.261099+0000","last_became_peered":"2024-04-02T13:51:01.261099+0000","last_unstale":"2024-04-02T13:51:56.662205+0000","last_undegraded":"2024-04-02T13:51:56.662205+0000","last_fullsized":"2024-04-02T13:51:56.662205+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:10:04.999402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477503+0000","last_change":"2024-04-02T13:51:05.361747+0000","last_active":"2024-04-02T13:51:56.477503+0000","last_peered":"2024-04-02T13:51:56.477503+0000","last_clean":"2024-04-02T13:51:56.477503+0000","last_became_active":"2024-04-02T13:51:05.361098+0000","last_became_peered":"2024-04-02T13:51:05.361098+0000","last_unstale":"2024-04-02T13:51:56.477503+0000","last_undegraded":"2024-04-02T13:51:56.477503+0000","last_fullsized":"2024-04-02T13:51:56.477503+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:45:35.048870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477099+0000","last_change":"2024-04-02T13:51:00.244048+0000","last_active":"2024-04-02T13:51:56.477099+0000","last_peered":"2024-04-02T13:51:56.477099+0000","last_clean":"2024-04-02T13:51:56.477099+0000","last_became_active":"2024-04-02T13:51:00.243510+0000","last_became_peered":"2024-04-02T13:51:00.243510+0000","last_unstale":"2024-04-02T13:51:56.477099+0000","last_undegraded":"2024-04-02T13:51:56.477099+0000","last_fullsized":"2024-04-02T13:51:56.477099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:53:48.364137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662811+0000","last_change":"2024-04-02T13:51:01.264711+0000","last_active":"2024-04-02T13:51:56.662811+0000","last_peered":"2024-04-02T13:51:56.662811+0000","last_clean":"2024-04-02T13:51:56.662811+0000","last_became_active":"2024-04-02T13:51:01.264289+0000","last_became_peered":"2024-04-02T13:51:01.264289+0000","last_unstale":"2024-04-02T13:51:56.662811+0000","last_undegraded":"2024-04-02T13:51:56.662811+0000","last_fullsized":"2024-04-02T13:51:56.662811+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:34:41.590071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477027+0000","last_change":"2024-04-02T13:51:05.363121+0000","last_active":"2024-04-02T13:51:56.477027+0000","last_peered":"2024-04-02T13:51:56.477027+0000","last_clean":"2024-04-02T13:51:56.477027+0000","last_became_active":"2024-04-02T13:51:05.362642+0000","last_became_peered":"2024-04-02T13:51:05.362642+0000","last_unstale":"2024-04-02T13:51:56.477027+0000","last_undegraded":"2024-04-02T13:51:56.477027+0000","last_fullsized":"2024-04-02T13:51:56.477027+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:55:40.848731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.476657+0000","last_change":"2024-04-02T13:51:00.236519+0000","last_active":"2024-04-02T13:51:56.476657+0000","last_peered":"2024-04-02T13:51:56.476657+0000","last_clean":"2024-04-02T13:51:56.476657+0000","last_became_active":"2024-04-02T13:51:00.235985+0000","last_became_peered":"2024-04-02T13:51:00.235985+0000","last_unstale":"2024-04-02T13:51:56.476657+0000","last_undegraded":"2024-04-02T13:51:56.476657+0000","last_fullsized":"2024-04-02T13:51:56.476657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:08.321765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662909+0000","last_change":"2024-04-02T13:51:01.260333+0000","last_active":"2024-04-02T13:51:56.662909+0000","last_peered":"2024-04-02T13:51:56.662909+0000","last_clean":"2024-04-02T13:51:56.662909+0000","last_became_active":"2024-04-02T13:51:01.259995+0000","last_became_peered":"2024-04-02T13:51:01.259995+0000","last_unstale":"2024-04-02T13:51:56.662909+0000","last_undegraded":"2024-04-02T13:51:56.662909+0000","last_fullsized":"2024-04-02T13:51:56.662909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:03:58.088653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477680+0000","last_change":"2024-04-02T13:51:05.359281+0000","last_active":"2024-04-02T13:51:56.477680+0000","last_peered":"2024-04-02T13:51:56.477680+0000","last_clean":"2024-04-02T13:51:56.477680+0000","last_became_active":"2024-04-02T13:51:05.358831+0000","last_became_peered":"2024-04-02T13:51:05.358831+0000","last_unstale":"2024-04-02T13:51:56.477680+0000","last_undegraded":"2024-04-02T13:51:56.477680+0000","last_fullsized":"2024-04-02T13:51:56.477680+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:48.461298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":106,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:52:39.443548+0000","last_change":"2024-04-02T13:51:00.241707+0000","last_active":"2024-04-02T13:52:39.443548+0000","last_peered":"2024-04-02T13:52:39.443548+0000","last_clean":"2024-04-02T13:52:39.443548+0000","last_became_active":"2024-04-02T13:51:00.241225+0000","last_became_peered":"2024-04-02T13:51:00.241225+0000","last_unstale":"2024-04-02T13:52:39.443548+0000","last_undegraded":"2024-04-02T13:52:39.443548+0000","last_fullsized":"2024-04-02T13:52:39.443548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:59:44.537503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.476384+0000","last_change":"2024-04-02T13:51:01.252942+0000","last_active":"2024-04-02T13:51:56.476384+0000","last_peered":"2024-04-02T13:51:56.476384+0000","last_clean":"2024-04-02T13:51:56.476384+0000","last_became_active":"2024-04-02T13:51:01.252631+0000","last_became_peered":"2024-04-02T13:51:01.252631+0000","last_unstale":"2024-04-02T13:51:56.476384+0000","last_undegraded":"2024-04-02T13:51:56.476384+0000","last_fullsized":"2024-04-02T13:51:56.476384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:43:24.725688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.478285+0000","last_change":"2024-04-02T13:51:05.359208+0000","last_active":"2024-04-02T13:51:56.478285+0000","last_peered":"2024-04-02T13:51:56.478285+0000","last_clean":"2024-04-02T13:51:56.478285+0000","last_became_active":"2024-04-02T13:51:05.358738+0000","last_became_peered":"2024-04-02T13:51:05.358738+0000","last_unstale":"2024-04-02T13:51:56.478285+0000","last_undegraded":"2024-04-02T13:51:56.478285+0000","last_fullsized":"2024-04-02T13:51:56.478285+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:47:46.034562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.476633+0000","last_change":"2024-04-02T13:51:00.236413+0000","last_active":"2024-04-02T13:51:56.476633+0000","last_peered":"2024-04-02T13:51:56.476633+0000","last_clean":"2024-04-02T13:51:56.476633+0000","last_became_active":"2024-04-02T13:51:00.236046+0000","last_became_peered":"2024-04-02T13:51:00.236046+0000","last_unstale":"2024-04-02T13:51:56.476633+0000","last_undegraded":"2024-04-02T13:51:56.476633+0000","last_fullsized":"2024-04-02T13:51:56.476633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:25:15.000726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662699+0000","last_change":"2024-04-02T13:51:01.265471+0000","last_active":"2024-04-02T13:51:56.662699+0000","last_peered":"2024-04-02T13:51:56.662699+0000","last_clean":"2024-04-02T13:51:56.662699+0000","last_became_active":"2024-04-02T13:51:01.265114+0000","last_became_peered":"2024-04-02T13:51:01.265114+0000","last_unstale":"2024-04-02T13:51:56.662699+0000","last_undegraded":"2024-04-02T13:51:56.662699+0000","last_fullsized":"2024-04-02T13:51:56.662699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:01:01.675686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.664063+0000","last_change":"2024-04-02T13:51:56.664063+0000","last_active":"2024-04-02T13:51:56.664063+0000","last_peered":"2024-04-02T13:51:56.664063+0000","last_clean":"2024-04-02T13:51:56.664063+0000","last_became_active":"2024-04-02T13:51:56.663590+0000","last_became_peered":"2024-04-02T13:51:56.663590+0000","last_unstale":"2024-04-02T13:51:56.664063+0000","last_undegraded":"2024-04-02T13:51:56.664063+0000","last_fullsized":"2024-04-02T13:51:56.664063+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:43:51.735851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.555709+0000","last_change":"2024-04-02T13:51:00.237204+0000","last_active":"2024-04-02T13:51:56.555709+0000","last_peered":"2024-04-02T13:51:56.555709+0000","last_clean":"2024-04-02T13:51:56.555709+0000","last_became_active":"2024-04-02T13:51:00.236707+0000","last_became_peered":"2024-04-02T13:51:00.236707+0000","last_unstale":"2024-04-02T13:51:56.555709+0000","last_undegraded":"2024-04-02T13:51:56.555709+0000","last_fullsized":"2024-04-02T13:51:56.555709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:00:17.771428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.481887+0000","last_change":"2024-04-02T13:51:01.250152+0000","last_active":"2024-04-02T13:51:56.481887+0000","last_peered":"2024-04-02T13:51:56.481887+0000","last_clean":"2024-04-02T13:51:56.481887+0000","last_became_active":"2024-04-02T13:51:01.249817+0000","last_became_peered":"2024-04-02T13:51:01.249817+0000","last_unstale":"2024-04-02T13:51:56.481887+0000","last_undegraded":"2024-04-02T13:51:56.481887+0000","last_fullsized":"2024-04-02T13:51:56.481887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:13:31.719380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662339+0000","last_change":"2024-04-02T13:51:05.366551+0000","last_active":"2024-04-02T13:51:56.662339+0000","last_peered":"2024-04-02T13:51:56.662339+0000","last_clean":"2024-04-02T13:51:56.662339+0000","last_became_active":"2024-04-02T13:51:05.366175+0000","last_became_peered":"2024-04-02T13:51:05.366175+0000","last_unstale":"2024-04-02T13:51:56.662339+0000","last_undegraded":"2024-04-02T13:51:56.662339+0000","last_fullsized":"2024-04-02T13:51:56.662339+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:47:49.822918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662285+0000","last_change":"2024-04-02T13:51:00.239521+0000","last_active":"2024-04-02T13:51:56.662285+0000","last_peered":"2024-04-02T13:51:56.662285+0000","last_clean":"2024-04-02T13:51:56.662285+0000","last_became_active":"2024-04-02T13:51:00.238608+0000","last_became_peered":"2024-04-02T13:51:00.238608+0000","last_unstale":"2024-04-02T13:51:56.662285+0000","last_undegraded":"2024-04-02T13:51:56.662285+0000","last_fullsized":"2024-04-02T13:51:56.662285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:00.983006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.475126+0000","last_change":"2024-04-02T13:51:01.255021+0000","last_active":"2024-04-02T13:51:56.475126+0000","last_peered":"2024-04-02T13:51:56.475126+0000","last_clean":"2024-04-02T13:51:56.475126+0000","last_became_active":"2024-04-02T13:51:01.254630+0000","last_became_peered":"2024-04-02T13:51:01.254630+0000","last_unstale":"2024-04-02T13:51:56.475126+0000","last_undegraded":"2024-04-02T13:51:56.475126+0000","last_fullsized":"2024-04-02T13:51:56.475126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:36:08.773431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.481757+0000","last_change":"2024-04-02T13:51:05.362004+0000","last_active":"2024-04-02T13:51:56.481757+0000","last_peered":"2024-04-02T13:51:56.481757+0000","last_clean":"2024-04-02T13:51:56.481757+0000","last_became_active":"2024-04-02T13:51:05.361538+0000","last_became_peered":"2024-04-02T13:51:05.361538+0000","last_unstale":"2024-04-02T13:51:56.481757+0000","last_undegraded":"2024-04-02T13:51:56.481757+0000","last_fullsized":"2024-04-02T13:51:56.481757+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:27:21.728695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":173,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:52:39.442618+0000","last_change":"2024-04-02T13:51:00.234814+0000","last_active":"2024-04-02T13:52:39.442618+0000","last_peered":"2024-04-02T13:52:39.442618+0000","last_clean":"2024-04-02T13:52:39.442618+0000","last_became_active":"2024-04-02T13:51:00.234481+0000","last_became_peered":"2024-04-02T13:51:00.234481+0000","last_unstale":"2024-04-02T13:52:39.442618+0000","last_undegraded":"2024-04-02T13:52:39.442618+0000","last_fullsized":"2024-04-02T13:52:39.442618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:45:34.901226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":142,"num_read_kb":142,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663585+0000","last_change":"2024-04-02T13:51:01.264093+0000","last_active":"2024-04-02T13:51:56.663585+0000","last_peered":"2024-04-02T13:51:56.663585+0000","last_clean":"2024-04-02T13:51:56.663585+0000","last_became_active":"2024-04-02T13:51:01.263754+0000","last_became_peered":"2024-04-02T13:51:01.263754+0000","last_unstale":"2024-04-02T13:51:56.663585+0000","last_undegraded":"2024-04-02T13:51:56.663585+0000","last_fullsized":"2024-04-02T13:51:56.663585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:49:18.630195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.475111+0000","last_change":"2024-04-02T13:51:05.370930+0000","last_active":"2024-04-02T13:51:56.475111+0000","last_peered":"2024-04-02T13:51:56.475111+0000","last_clean":"2024-04-02T13:51:56.475111+0000","last_became_active":"2024-04-02T13:51:05.370443+0000","last_became_peered":"2024-04-02T13:51:05.370443+0000","last_unstale":"2024-04-02T13:51:56.475111+0000","last_undegraded":"2024-04-02T13:51:56.475111+0000","last_fullsized":"2024-04-02T13:51:56.475111+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:24:27.055494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663027+0000","last_change":"2024-04-02T13:51:00.241206+0000","last_active":"2024-04-02T13:51:56.663027+0000","last_peered":"2024-04-02T13:51:56.663027+0000","last_clean":"2024-04-02T13:51:56.663027+0000","last_became_active":"2024-04-02T13:51:00.240832+0000","last_became_peered":"2024-04-02T13:51:00.240832+0000","last_unstale":"2024-04-02T13:51:56.663027+0000","last_undegraded":"2024-04-02T13:51:56.663027+0000","last_fullsized":"2024-04-02T13:51:56.663027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:44:11.626435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662489+0000","last_change":"2024-04-02T13:51:01.259315+0000","last_active":"2024-04-02T13:51:56.662489+0000","last_peered":"2024-04-02T13:51:56.662489+0000","last_clean":"2024-04-02T13:51:56.662489+0000","last_became_active":"2024-04-02T13:51:01.258951+0000","last_became_peered":"2024-04-02T13:51:01.258951+0000","last_unstale":"2024-04-02T13:51:56.662489+0000","last_undegraded":"2024-04-02T13:51:56.662489+0000","last_fullsized":"2024-04-02T13:51:56.662489+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:59:43.714044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.665826+0000","last_change":"2024-04-02T13:51:56.665826+0000","last_active":"2024-04-02T13:51:56.665826+0000","last_peered":"2024-04-02T13:51:56.665826+0000","last_clean":"2024-04-02T13:51:56.665826+0000","last_became_active":"2024-04-02T13:51:56.665213+0000","last_became_peered":"2024-04-02T13:51:56.665213+0000","last_unstale":"2024-04-02T13:51:56.665826+0000","last_undegraded":"2024-04-02T13:51:56.665826+0000","last_fullsized":"2024-04-02T13:51:56.665826+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:45:46.625063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662860+0000","last_change":"2024-04-02T13:51:00.230938+0000","last_active":"2024-04-02T13:51:56.662860+0000","last_peered":"2024-04-02T13:51:56.662860+0000","last_clean":"2024-04-02T13:51:56.662860+0000","last_became_active":"2024-04-02T13:51:00.230585+0000","last_became_peered":"2024-04-02T13:51:00.230585+0000","last_unstale":"2024-04-02T13:51:56.662860+0000","last_undegraded":"2024-04-02T13:51:56.662860+0000","last_fullsized":"2024-04-02T13:51:56.662860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:51:37.463876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663209+0000","last_change":"2024-04-02T13:51:01.264216+0000","last_active":"2024-04-02T13:51:56.663209+0000","last_peered":"2024-04-02T13:51:56.663209+0000","last_clean":"2024-04-02T13:51:56.663209+0000","last_became_active":"2024-04-02T13:51:01.263824+0000","last_became_peered":"2024-04-02T13:51:01.263824+0000","last_unstale":"2024-04-02T13:51:56.663209+0000","last_undegraded":"2024-04-02T13:51:56.663209+0000","last_fullsized":"2024-04-02T13:51:56.663209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:16:26.391163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.475438+0000","last_change":"2024-04-02T13:51:05.366002+0000","last_active":"2024-04-02T13:51:56.475438+0000","last_peered":"2024-04-02T13:51:56.475438+0000","last_clean":"2024-04-02T13:51:56.475438+0000","last_became_active":"2024-04-02T13:51:05.365547+0000","last_became_peered":"2024-04-02T13:51:05.365547+0000","last_unstale":"2024-04-02T13:51:56.475438+0000","last_undegraded":"2024-04-02T13:51:56.475438+0000","last_fullsized":"2024-04-02T13:51:56.475438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:31:38.990617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'36","reported_seq":212,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:52:39.442607+0000","last_change":"2024-04-02T13:51:00.233925+0000","last_active":"2024-04-02T13:52:39.442607+0000","last_peered":"2024-04-02T13:52:39.442607+0000","last_clean":"2024-04-02T13:52:39.442607+0000","last_became_active":"2024-04-02T13:51:00.233574+0000","last_became_peered":"2024-04-02T13:51:00.233574+0000","last_unstale":"2024-04-02T13:52:39.442607+0000","last_undegraded":"2024-04-02T13:52:39.442607+0000","last_fullsized":"2024-04-02T13:52:39.442607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:33:32.758874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62333,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":149,"num_read_kb":136,"num_write":38,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477298+0000","last_change":"2024-04-02T13:51:01.258825+0000","last_active":"2024-04-02T13:51:56.477298+0000","last_peered":"2024-04-02T13:51:56.477298+0000","last_clean":"2024-04-02T13:51:56.477298+0000","last_became_active":"2024-04-02T13:51:01.258500+0000","last_became_peered":"2024-04-02T13:51:01.258500+0000","last_unstale":"2024-04-02T13:51:56.477298+0000","last_undegraded":"2024-04-02T13:51:56.477298+0000","last_fullsized":"2024-04-02T13:51:56.477298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:33:15.409765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663136+0000","last_change":"2024-04-02T13:51:05.365010+0000","last_active":"2024-04-02T13:51:56.663136+0000","last_peered":"2024-04-02T13:51:56.663136+0000","last_clean":"2024-04-02T13:51:56.663136+0000","last_became_active":"2024-04-02T13:51:05.364499+0000","last_became_peered":"2024-04-02T13:51:05.364499+0000","last_unstale":"2024-04-02T13:51:56.663136+0000","last_undegraded":"2024-04-02T13:51:56.663136+0000","last_fullsized":"2024-04-02T13:51:56.663136+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:07:03.500017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.496876+0000","last_change":"2024-04-02T13:51:56.496876+0000","last_active":"2024-04-02T13:51:56.496876+0000","last_peered":"2024-04-02T13:51:56.496876+0000","last_clean":"2024-04-02T13:51:56.496876+0000","last_became_active":"2024-04-02T13:51:56.484672+0000","last_became_peered":"2024-04-02T13:51:56.484672+0000","last_unstale":"2024-04-02T13:51:56.496876+0000","last_undegraded":"2024-04-02T13:51:56.496876+0000","last_fullsized":"2024-04-02T13:51:56.496876+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:31:40.027658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.555868+0000","last_change":"2024-04-02T13:51:01.256334+0000","last_active":"2024-04-02T13:51:56.555868+0000","last_peered":"2024-04-02T13:51:56.555868+0000","last_clean":"2024-04-02T13:51:56.555868+0000","last_became_active":"2024-04-02T13:51:01.255964+0000","last_became_peered":"2024-04-02T13:51:01.255964+0000","last_unstale":"2024-04-02T13:51:56.555868+0000","last_undegraded":"2024-04-02T13:51:56.555868+0000","last_fullsized":"2024-04-02T13:51:56.555868+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:43:04.842812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.479357+0000","last_change":"2024-04-02T13:51:05.362696+0000","last_active":"2024-04-02T13:51:56.479357+0000","last_peered":"2024-04-02T13:51:56.479357+0000","last_clean":"2024-04-02T13:51:56.479357+0000","last_became_active":"2024-04-02T13:51:05.362225+0000","last_became_peered":"2024-04-02T13:51:05.362225+0000","last_unstale":"2024-04-02T13:51:56.479357+0000","last_undegraded":"2024-04-02T13:51:56.479357+0000","last_fullsized":"2024-04-02T13:51:56.479357+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:23:56.308286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662545+0000","last_change":"2024-04-02T13:51:00.239673+0000","last_active":"2024-04-02T13:51:56.662545+0000","last_peered":"2024-04-02T13:51:56.662545+0000","last_clean":"2024-04-02T13:51:56.662545+0000","last_became_active":"2024-04-02T13:51:00.239233+0000","last_became_peered":"2024-04-02T13:51:00.239233+0000","last_unstale":"2024-04-02T13:51:56.662545+0000","last_undegraded":"2024-04-02T13:51:56.662545+0000","last_fullsized":"2024-04-02T13:51:56.662545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:05:48.119953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.664125+0000","last_change":"2024-04-02T13:51:01.257474+0000","last_active":"2024-04-02T13:51:56.664125+0000","last_peered":"2024-04-02T13:51:56.664125+0000","last_clean":"2024-04-02T13:51:56.664125+0000","last_became_active":"2024-04-02T13:51:01.254977+0000","last_became_peered":"2024-04-02T13:51:01.254977+0000","last_unstale":"2024-04-02T13:51:56.664125+0000","last_undegraded":"2024-04-02T13:51:56.664125+0000","last_fullsized":"2024-04-02T13:51:56.664125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:13:07.446520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.475753+0000","last_change":"2024-04-02T13:51:05.371531+0000","last_active":"2024-04-02T13:51:56.475753+0000","last_peered":"2024-04-02T13:51:56.475753+0000","last_clean":"2024-04-02T13:51:56.475753+0000","last_became_active":"2024-04-02T13:51:05.370551+0000","last_became_peered":"2024-04-02T13:51:05.370551+0000","last_unstale":"2024-04-02T13:51:56.475753+0000","last_undegraded":"2024-04-02T13:51:56.475753+0000","last_fullsized":"2024-04-02T13:51:56.475753+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:43:48.796009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'5","reported_seq":105,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:52:38.441548+0000","last_change":"2024-04-02T13:51:00.237715+0000","last_active":"2024-04-02T13:52:38.441548+0000","last_peered":"2024-04-02T13:52:38.441548+0000","last_clean":"2024-04-02T13:52:38.441548+0000","last_became_active":"2024-04-02T13:51:00.237349+0000","last_became_peered":"2024-04-02T13:51:00.237349+0000","last_unstale":"2024-04-02T13:52:38.441548+0000","last_undegraded":"2024-04-02T13:52:38.441548+0000","last_fullsized":"2024-04-02T13:52:38.441548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:33:20.651562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663012+0000","last_change":"2024-04-02T13:51:01.259447+0000","last_active":"2024-04-02T13:51:56.663012+0000","last_peered":"2024-04-02T13:51:56.663012+0000","last_clean":"2024-04-02T13:51:56.663012+0000","last_became_active":"2024-04-02T13:51:01.259101+0000","last_became_peered":"2024-04-02T13:51:01.259101+0000","last_unstale":"2024-04-02T13:51:56.663012+0000","last_undegraded":"2024-04-02T13:51:56.663012+0000","last_fullsized":"2024-04-02T13:51:56.663012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:51:44.311440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.556068+0000","last_change":"2024-04-02T13:51:05.372446+0000","last_active":"2024-04-02T13:51:56.556068+0000","last_peered":"2024-04-02T13:51:56.556068+0000","last_clean":"2024-04-02T13:51:56.556068+0000","last_became_active":"2024-04-02T13:51:05.372007+0000","last_became_peered":"2024-04-02T13:51:05.372007+0000","last_unstale":"2024-04-02T13:51:56.556068+0000","last_undegraded":"2024-04-02T13:51:56.556068+0000","last_fullsized":"2024-04-02T13:51:56.556068+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:54:23.463179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.476245+0000","last_change":"2024-04-02T13:51:00.241492+0000","last_active":"2024-04-02T13:51:56.476245+0000","last_peered":"2024-04-02T13:51:56.476245+0000","last_clean":"2024-04-02T13:51:56.476245+0000","last_became_active":"2024-04-02T13:51:00.241167+0000","last_became_peered":"2024-04-02T13:51:00.241167+0000","last_unstale":"2024-04-02T13:51:56.476245+0000","last_undegraded":"2024-04-02T13:51:56.476245+0000","last_fullsized":"2024-04-02T13:51:56.476245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:44:44.070187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662409+0000","last_change":"2024-04-02T13:51:01.263200+0000","last_active":"2024-04-02T13:51:56.662409+0000","last_peered":"2024-04-02T13:51:56.662409+0000","last_clean":"2024-04-02T13:51:56.662409+0000","last_became_active":"2024-04-02T13:51:01.262812+0000","last_became_peered":"2024-04-02T13:51:01.262812+0000","last_unstale":"2024-04-02T13:51:56.662409+0000","last_undegraded":"2024-04-02T13:51:56.662409+0000","last_fullsized":"2024-04-02T13:51:56.662409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:33:35.898047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662275+0000","last_change":"2024-04-02T13:51:05.366729+0000","last_active":"2024-04-02T13:51:56.662275+0000","last_peered":"2024-04-02T13:51:56.662275+0000","last_clean":"2024-04-02T13:51:56.662275+0000","last_became_active":"2024-04-02T13:51:05.366257+0000","last_became_peered":"2024-04-02T13:51:05.366257+0000","last_unstale":"2024-04-02T13:51:56.662275+0000","last_undegraded":"2024-04-02T13:51:56.662275+0000","last_fullsized":"2024-04-02T13:51:56.662275+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:46:03.400300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.476916+0000","last_change":"2024-04-02T13:51:00.236004+0000","last_active":"2024-04-02T13:51:56.476916+0000","last_peered":"2024-04-02T13:51:56.476916+0000","last_clean":"2024-04-02T13:51:56.476916+0000","last_became_active":"2024-04-02T13:51:00.235160+0000","last_became_peered":"2024-04-02T13:51:00.235160+0000","last_unstale":"2024-04-02T13:51:56.476916+0000","last_undegraded":"2024-04-02T13:51:56.476916+0000","last_fullsized":"2024-04-02T13:51:56.476916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:16:35.617346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.479288+0000","last_change":"2024-04-02T13:51:01.259565+0000","last_active":"2024-04-02T13:51:56.479288+0000","last_peered":"2024-04-02T13:51:56.479288+0000","last_clean":"2024-04-02T13:51:56.479288+0000","last_became_active":"2024-04-02T13:51:01.259225+0000","last_became_peered":"2024-04-02T13:51:01.259225+0000","last_unstale":"2024-04-02T13:51:56.479288+0000","last_undegraded":"2024-04-02T13:51:56.479288+0000","last_fullsized":"2024-04-02T13:51:56.479288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:22:18.878029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.476269+0000","last_change":"2024-04-02T13:51:05.518750+0000","last_active":"2024-04-02T13:51:56.476269+0000","last_peered":"2024-04-02T13:51:56.476269+0000","last_clean":"2024-04-02T13:51:56.476269+0000","last_became_active":"2024-04-02T13:51:05.518247+0000","last_became_peered":"2024-04-02T13:51:05.518247+0000","last_unstale":"2024-04-02T13:51:56.476269+0000","last_undegraded":"2024-04-02T13:51:56.476269+0000","last_fullsized":"2024-04-02T13:51:56.476269+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:08:10.232509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.475385+0000","last_change":"2024-04-02T13:51:00.241476+0000","last_active":"2024-04-02T13:51:56.475385+0000","last_peered":"2024-04-02T13:51:56.475385+0000","last_clean":"2024-04-02T13:51:56.475385+0000","last_became_active":"2024-04-02T13:51:00.241085+0000","last_became_peered":"2024-04-02T13:51:00.241085+0000","last_unstale":"2024-04-02T13:51:56.475385+0000","last_undegraded":"2024-04-02T13:51:56.475385+0000","last_fullsized":"2024-04-02T13:51:56.475385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:29:01.503901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477254+0000","last_change":"2024-04-02T13:51:01.259791+0000","last_active":"2024-04-02T13:51:56.477254+0000","last_peered":"2024-04-02T13:51:56.477254+0000","last_clean":"2024-04-02T13:51:56.477254+0000","last_became_active":"2024-04-02T13:51:01.259440+0000","last_became_peered":"2024-04-02T13:51:01.259440+0000","last_unstale":"2024-04-02T13:51:56.477254+0000","last_undegraded":"2024-04-02T13:51:56.477254+0000","last_fullsized":"2024-04-02T13:51:56.477254+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:43:52.422263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.555849+0000","last_change":"2024-04-02T13:51:05.366243+0000","last_active":"2024-04-02T13:51:56.555849+0000","last_peered":"2024-04-02T13:51:56.555849+0000","last_clean":"2024-04-02T13:51:56.555849+0000","last_became_active":"2024-04-02T13:51:05.365801+0000","last_became_peered":"2024-04-02T13:51:05.365801+0000","last_unstale":"2024-04-02T13:51:56.555849+0000","last_undegraded":"2024-04-02T13:51:56.555849+0000","last_fullsized":"2024-04-02T13:51:56.555849+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:33:56.853434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":105,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:52:39.442336+0000","last_change":"2024-04-02T13:51:00.248695+0000","last_active":"2024-04-02T13:52:39.442336+0000","last_peered":"2024-04-02T13:52:39.442336+0000","last_clean":"2024-04-02T13:52:39.442336+0000","last_became_active":"2024-04-02T13:51:00.248313+0000","last_became_peered":"2024-04-02T13:51:00.248313+0000","last_unstale":"2024-04-02T13:52:39.442336+0000","last_undegraded":"2024-04-02T13:52:39.442336+0000","last_fullsized":"2024-04-02T13:52:39.442336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:43:21.410823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477001+0000","last_change":"2024-04-02T13:51:01.260993+0000","last_active":"2024-04-02T13:51:56.477001+0000","last_peered":"2024-04-02T13:51:56.477001+0000","last_clean":"2024-04-02T13:51:56.477001+0000","last_became_active":"2024-04-02T13:51:01.260510+0000","last_became_peered":"2024-04-02T13:51:01.260510+0000","last_unstale":"2024-04-02T13:51:56.477001+0000","last_undegraded":"2024-04-02T13:51:56.477001+0000","last_fullsized":"2024-04-02T13:51:56.477001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:17:04.941822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.556012+0000","last_change":"2024-04-02T13:51:05.518455+0000","last_active":"2024-04-02T13:51:56.556012+0000","last_peered":"2024-04-02T13:51:56.556012+0000","last_clean":"2024-04-02T13:51:56.556012+0000","last_became_active":"2024-04-02T13:51:05.517962+0000","last_became_peered":"2024-04-02T13:51:05.517962+0000","last_unstale":"2024-04-02T13:51:56.556012+0000","last_undegraded":"2024-04-02T13:51:56.556012+0000","last_fullsized":"2024-04-02T13:51:56.556012+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:19:47.734233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662479+0000","last_change":"2024-04-02T13:51:01.259220+0000","last_active":"2024-04-02T13:51:56.662479+0000","last_peered":"2024-04-02T13:51:56.662479+0000","last_clean":"2024-04-02T13:51:56.662479+0000","last_became_active":"2024-04-02T13:51:01.258890+0000","last_became_peered":"2024-04-02T13:51:01.258890+0000","last_unstale":"2024-04-02T13:51:56.662479+0000","last_undegraded":"2024-04-02T13:51:56.662479+0000","last_fullsized":"2024-04-02T13:51:56.662479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:56:15.444736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":105,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:52:39.442219+0000","last_change":"2024-04-02T13:51:00.246727+0000","last_active":"2024-04-02T13:52:39.442219+0000","last_peered":"2024-04-02T13:52:39.442219+0000","last_clean":"2024-04-02T13:52:39.442219+0000","last_became_active":"2024-04-02T13:51:00.246364+0000","last_became_peered":"2024-04-02T13:51:00.246364+0000","last_unstale":"2024-04-02T13:52:39.442219+0000","last_undegraded":"2024-04-02T13:52:39.442219+0000","last_fullsized":"2024-04-02T13:52:39.442219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:35:13.249972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663127+0000","last_change":"2024-04-02T13:51:05.358506+0000","last_active":"2024-04-02T13:51:56.663127+0000","last_peered":"2024-04-02T13:51:56.663127+0000","last_clean":"2024-04-02T13:51:56.663127+0000","last_became_active":"2024-04-02T13:51:05.358157+0000","last_became_peered":"2024-04-02T13:51:05.358157+0000","last_unstale":"2024-04-02T13:51:56.663127+0000","last_undegraded":"2024-04-02T13:51:56.663127+0000","last_fullsized":"2024-04-02T13:51:56.663127+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:24:15.585124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.665735+0000","last_change":"2024-04-02T13:51:56.665307+0000","last_active":"2024-04-02T13:51:56.665735+0000","last_peered":"2024-04-02T13:51:56.665735+0000","last_clean":"2024-04-02T13:51:56.665735+0000","last_became_active":"2024-04-02T13:51:56.665003+0000","last_became_peered":"2024-04-02T13:51:56.665003+0000","last_unstale":"2024-04-02T13:51:56.665735+0000","last_undegraded":"2024-04-02T13:51:56.665735+0000","last_fullsized":"2024-04-02T13:51:56.665735+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:02:46.421524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.478223+0000","last_change":"2024-04-02T13:51:00.238962+0000","last_active":"2024-04-02T13:51:56.478223+0000","last_peered":"2024-04-02T13:51:56.478223+0000","last_clean":"2024-04-02T13:51:56.478223+0000","last_became_active":"2024-04-02T13:51:00.238475+0000","last_became_peered":"2024-04-02T13:51:00.238475+0000","last_unstale":"2024-04-02T13:51:56.478223+0000","last_undegraded":"2024-04-02T13:51:56.478223+0000","last_fullsized":"2024-04-02T13:51:56.478223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:41:07.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":362,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662789+0000","last_change":"2024-04-02T13:49:47.847355+0000","last_active":"2024-04-02T13:51:56.662789+0000","last_peered":"2024-04-02T13:51:56.662789+0000","last_clean":"2024-04-02T13:51:56.662789+0000","last_became_active":"2024-04-02T13:49:47.846768+0000","last_became_peered":"2024-04-02T13:49:47.846768+0000","last_unstale":"2024-04-02T13:51:56.662789+0000","last_undegraded":"2024-04-02T13:51:56.662789+0000","last_fullsized":"2024-04-02T13:51:56.662789+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:46:21.098780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:46:21.098780+0000","last_clean_scrub_stamp":"2024-04-02T13:46:21.098780+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:03:52.323921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663467+0000","last_change":"2024-04-02T13:51:05.518859+0000","last_active":"2024-04-02T13:51:56.663467+0000","last_peered":"2024-04-02T13:51:56.663467+0000","last_clean":"2024-04-02T13:51:56.663467+0000","last_became_active":"2024-04-02T13:51:05.518364+0000","last_became_peered":"2024-04-02T13:51:05.518364+0000","last_unstale":"2024-04-02T13:51:56.663467+0000","last_undegraded":"2024-04-02T13:51:56.663467+0000","last_fullsized":"2024-04-02T13:51:56.663467+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:10:45.877853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.555609+0000","last_change":"2024-04-02T13:51:01.254795+0000","last_active":"2024-04-02T13:51:56.555609+0000","last_peered":"2024-04-02T13:51:56.555609+0000","last_clean":"2024-04-02T13:51:56.555609+0000","last_became_active":"2024-04-02T13:51:01.254452+0000","last_became_peered":"2024-04-02T13:51:01.254452+0000","last_unstale":"2024-04-02T13:51:56.555609+0000","last_undegraded":"2024-04-02T13:51:56.555609+0000","last_fullsized":"2024-04-02T13:51:56.555609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:13:53.747308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662875+0000","last_change":"2024-04-02T13:51:00.241323+0000","last_active":"2024-04-02T13:51:56.662875+0000","last_peered":"2024-04-02T13:51:56.662875+0000","last_clean":"2024-04-02T13:51:56.662875+0000","last_became_active":"2024-04-02T13:51:00.240989+0000","last_became_peered":"2024-04-02T13:51:00.240989+0000","last_unstale":"2024-04-02T13:51:56.662875+0000","last_undegraded":"2024-04-02T13:51:56.662875+0000","last_fullsized":"2024-04-02T13:51:56.662875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:47:52.601517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.665905+0000","last_change":"2024-04-02T13:51:56.665905+0000","last_active":"2024-04-02T13:51:56.665905+0000","last_peered":"2024-04-02T13:51:56.665905+0000","last_clean":"2024-04-02T13:51:56.665905+0000","last_became_active":"2024-04-02T13:51:56.665315+0000","last_became_peered":"2024-04-02T13:51:56.665315+0000","last_unstale":"2024-04-02T13:51:56.665905+0000","last_undegraded":"2024-04-02T13:51:56.665905+0000","last_fullsized":"2024-04-02T13:51:56.665905+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:44:34.548102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.476884+0000","last_change":"2024-04-02T13:51:00.241416+0000","last_active":"2024-04-02T13:51:56.476884+0000","last_peered":"2024-04-02T13:51:56.476884+0000","last_clean":"2024-04-02T13:51:56.476884+0000","last_became_active":"2024-04-02T13:51:00.241071+0000","last_became_peered":"2024-04-02T13:51:00.241071+0000","last_unstale":"2024-04-02T13:51:56.476884+0000","last_undegraded":"2024-04-02T13:51:56.476884+0000","last_fullsized":"2024-04-02T13:51:56.476884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:24:16.243737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.556027+0000","last_change":"2024-04-02T13:51:01.252156+0000","last_active":"2024-04-02T13:51:56.556027+0000","last_peered":"2024-04-02T13:51:56.556027+0000","last_clean":"2024-04-02T13:51:56.556027+0000","last_became_active":"2024-04-02T13:51:01.251733+0000","last_became_peered":"2024-04-02T13:51:01.251733+0000","last_unstale":"2024-04-02T13:51:56.556027+0000","last_undegraded":"2024-04-02T13:51:56.556027+0000","last_fullsized":"2024-04-02T13:51:56.556027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:54:24.082643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.481850+0000","last_change":"2024-04-02T13:51:05.518367+0000","last_active":"2024-04-02T13:51:56.481850+0000","last_peered":"2024-04-02T13:51:56.481850+0000","last_clean":"2024-04-02T13:51:56.481850+0000","last_became_active":"2024-04-02T13:51:05.517783+0000","last_became_peered":"2024-04-02T13:51:05.517783+0000","last_unstale":"2024-04-02T13:51:56.481850+0000","last_undegraded":"2024-04-02T13:51:56.481850+0000","last_fullsized":"2024-04-02T13:51:56.481850+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:10:53.880909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'11","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.475665+0000","last_change":"2024-04-02T13:51:00.248998+0000","last_active":"2024-04-02T13:51:56.475665+0000","last_peered":"2024-04-02T13:51:56.475665+0000","last_clean":"2024-04-02T13:51:56.475665+0000","last_became_active":"2024-04-02T13:51:00.248613+0000","last_became_peered":"2024-04-02T13:51:00.248613+0000","last_unstale":"2024-04-02T13:51:56.475665+0000","last_undegraded":"2024-04-02T13:51:56.475665+0000","last_fullsized":"2024-04-02T13:51:56.475665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:53.948645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.482134+0000","last_change":"2024-04-02T13:51:01.257244+0000","last_active":"2024-04-02T13:51:56.482134+0000","last_peered":"2024-04-02T13:51:56.482134+0000","last_clean":"2024-04-02T13:51:56.482134+0000","last_became_active":"2024-04-02T13:51:01.256739+0000","last_became_peered":"2024-04-02T13:51:01.256739+0000","last_unstale":"2024-04-02T13:51:56.482134+0000","last_undegraded":"2024-04-02T13:51:56.482134+0000","last_fullsized":"2024-04-02T13:51:56.482134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:38:18.786553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.478152+0000","last_change":"2024-04-02T13:51:05.356177+0000","last_active":"2024-04-02T13:51:56.478152+0000","last_peered":"2024-04-02T13:51:56.478152+0000","last_clean":"2024-04-02T13:51:56.478152+0000","last_became_active":"2024-04-02T13:51:05.354704+0000","last_became_peered":"2024-04-02T13:51:05.354704+0000","last_unstale":"2024-04-02T13:51:56.478152+0000","last_undegraded":"2024-04-02T13:51:56.478152+0000","last_fullsized":"2024-04-02T13:51:56.478152+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:27:55.761490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.478405+0000","last_change":"2024-04-02T13:51:00.245945+0000","last_active":"2024-04-02T13:51:56.478405+0000","last_peered":"2024-04-02T13:51:56.478405+0000","last_clean":"2024-04-02T13:51:56.478405+0000","last_became_active":"2024-04-02T13:51:00.245528+0000","last_became_peered":"2024-04-02T13:51:00.245528+0000","last_unstale":"2024-04-02T13:51:56.478405+0000","last_undegraded":"2024-04-02T13:51:56.478405+0000","last_fullsized":"2024-04-02T13:51:56.478405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:37:49.263614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662471+0000","last_change":"2024-04-02T13:51:01.260268+0000","last_active":"2024-04-02T13:51:56.662471+0000","last_peered":"2024-04-02T13:51:56.662471+0000","last_clean":"2024-04-02T13:51:56.662471+0000","last_became_active":"2024-04-02T13:51:01.258182+0000","last_became_peered":"2024-04-02T13:51:01.258182+0000","last_unstale":"2024-04-02T13:51:56.662471+0000","last_undegraded":"2024-04-02T13:51:56.662471+0000","last_fullsized":"2024-04-02T13:51:56.662471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:31:31.423988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477836+0000","last_change":"2024-04-02T13:51:05.518597+0000","last_active":"2024-04-02T13:51:56.477836+0000","last_peered":"2024-04-02T13:51:56.477836+0000","last_clean":"2024-04-02T13:51:56.477836+0000","last_became_active":"2024-04-02T13:51:05.518134+0000","last_became_peered":"2024-04-02T13:51:05.518134+0000","last_unstale":"2024-04-02T13:51:56.477836+0000","last_undegraded":"2024-04-02T13:51:56.477836+0000","last_fullsized":"2024-04-02T13:51:56.477836+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:00:38.632939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663486+0000","last_change":"2024-04-02T13:51:00.241680+0000","last_active":"2024-04-02T13:51:56.663486+0000","last_peered":"2024-04-02T13:51:56.663486+0000","last_clean":"2024-04-02T13:51:56.663486+0000","last_became_active":"2024-04-02T13:51:00.241301+0000","last_became_peered":"2024-04-02T13:51:00.241301+0000","last_unstale":"2024-04-02T13:51:56.663486+0000","last_undegraded":"2024-04-02T13:51:56.663486+0000","last_fullsized":"2024-04-02T13:51:56.663486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:12:10.606973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.476132+0000","last_change":"2024-04-02T13:51:01.254795+0000","last_active":"2024-04-02T13:51:56.476132+0000","last_peered":"2024-04-02T13:51:56.476132+0000","last_clean":"2024-04-02T13:51:56.476132+0000","last_became_active":"2024-04-02T13:51:01.254451+0000","last_became_peered":"2024-04-02T13:51:01.254451+0000","last_unstale":"2024-04-02T13:51:56.476132+0000","last_undegraded":"2024-04-02T13:51:56.476132+0000","last_fullsized":"2024-04-02T13:51:56.476132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:58:04.202996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.475885+0000","last_change":"2024-04-02T13:51:05.363592+0000","last_active":"2024-04-02T13:51:56.475885+0000","last_peered":"2024-04-02T13:51:56.475885+0000","last_clean":"2024-04-02T13:51:56.475885+0000","last_became_active":"2024-04-02T13:51:05.363211+0000","last_became_peered":"2024-04-02T13:51:05.363211+0000","last_unstale":"2024-04-02T13:51:56.475885+0000","last_undegraded":"2024-04-02T13:51:56.475885+0000","last_fullsized":"2024-04-02T13:51:56.475885+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:31:56.561505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.481783+0000","last_change":"2024-04-02T13:51:00.243681+0000","last_active":"2024-04-02T13:51:56.481783+0000","last_peered":"2024-04-02T13:51:56.481783+0000","last_clean":"2024-04-02T13:51:56.481783+0000","last_became_active":"2024-04-02T13:51:00.241615+0000","last_became_peered":"2024-04-02T13:51:00.241615+0000","last_unstale":"2024-04-02T13:51:56.481783+0000","last_undegraded":"2024-04-02T13:51:56.481783+0000","last_fullsized":"2024-04-02T13:51:56.481783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:54:30.112735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.478600+0000","last_change":"2024-04-02T13:51:01.252429+0000","last_active":"2024-04-02T13:51:56.478600+0000","last_peered":"2024-04-02T13:51:56.478600+0000","last_clean":"2024-04-02T13:51:56.478600+0000","last_became_active":"2024-04-02T13:51:01.252017+0000","last_became_peered":"2024-04-02T13:51:01.252017+0000","last_unstale":"2024-04-02T13:51:56.478600+0000","last_undegraded":"2024-04-02T13:51:56.478600+0000","last_fullsized":"2024-04-02T13:51:56.478600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:32:02.152578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477700+0000","last_change":"2024-04-02T13:51:05.361297+0000","last_active":"2024-04-02T13:51:56.477700+0000","last_peered":"2024-04-02T13:51:56.477700+0000","last_clean":"2024-04-02T13:51:56.477700+0000","last_became_active":"2024-04-02T13:51:05.360916+0000","last_became_peered":"2024-04-02T13:51:05.360916+0000","last_unstale":"2024-04-02T13:51:56.477700+0000","last_undegraded":"2024-04-02T13:51:56.477700+0000","last_fullsized":"2024-04-02T13:51:56.477700+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:52:48.608587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.488312+0000","last_change":"2024-04-02T13:51:56.488312+0000","last_active":"2024-04-02T13:51:56.488312+0000","last_peered":"2024-04-02T13:51:56.488312+0000","last_clean":"2024-04-02T13:51:56.488312+0000","last_became_active":"2024-04-02T13:51:56.483987+0000","last_became_peered":"2024-04-02T13:51:56.483987+0000","last_unstale":"2024-04-02T13:51:56.488312+0000","last_undegraded":"2024-04-02T13:51:56.488312+0000","last_fullsized":"2024-04-02T13:51:56.488312+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:46:27.396997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.482196+0000","last_change":"2024-04-02T13:51:01.255557+0000","last_active":"2024-04-02T13:51:56.482196+0000","last_peered":"2024-04-02T13:51:56.482196+0000","last_clean":"2024-04-02T13:51:56.482196+0000","last_became_active":"2024-04-02T13:51:01.255011+0000","last_became_peered":"2024-04-02T13:51:01.255011+0000","last_unstale":"2024-04-02T13:51:56.482196+0000","last_undegraded":"2024-04-02T13:51:56.482196+0000","last_fullsized":"2024-04-02T13:51:56.482196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:01:52.105965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477282+0000","last_change":"2024-04-02T13:51:05.518764+0000","last_active":"2024-04-02T13:51:56.477282+0000","last_peered":"2024-04-02T13:51:56.477282+0000","last_clean":"2024-04-02T13:51:56.477282+0000","last_became_active":"2024-04-02T13:51:05.518088+0000","last_became_peered":"2024-04-02T13:51:05.518088+0000","last_unstale":"2024-04-02T13:51:56.477282+0000","last_undegraded":"2024-04-02T13:51:56.477282+0000","last_fullsized":"2024-04-02T13:51:56.477282+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:03:02.901384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.476603+0000","last_change":"2024-04-02T13:51:00.244147+0000","last_active":"2024-04-02T13:51:56.476603+0000","last_peered":"2024-04-02T13:51:56.476603+0000","last_clean":"2024-04-02T13:51:56.476603+0000","last_became_active":"2024-04-02T13:51:00.243763+0000","last_became_peered":"2024-04-02T13:51:00.243763+0000","last_unstale":"2024-04-02T13:51:56.476603+0000","last_undegraded":"2024-04-02T13:51:56.476603+0000","last_fullsized":"2024-04-02T13:51:56.476603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:34:27.738746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663978+0000","last_change":"2024-04-02T13:51:01.268121+0000","last_active":"2024-04-02T13:51:56.663978+0000","last_peered":"2024-04-02T13:51:56.663978+0000","last_clean":"2024-04-02T13:51:56.663978+0000","last_became_active":"2024-04-02T13:51:01.266363+0000","last_became_peered":"2024-04-02T13:51:01.266363+0000","last_unstale":"2024-04-02T13:51:56.663978+0000","last_undegraded":"2024-04-02T13:51:56.663978+0000","last_fullsized":"2024-04-02T13:51:56.663978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:30:50.254916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662794+0000","last_change":"2024-04-02T13:51:05.361285+0000","last_active":"2024-04-02T13:51:56.662794+0000","last_peered":"2024-04-02T13:51:56.662794+0000","last_clean":"2024-04-02T13:51:56.662794+0000","last_became_active":"2024-04-02T13:51:05.360122+0000","last_became_peered":"2024-04-02T13:51:05.360122+0000","last_unstale":"2024-04-02T13:51:56.662794+0000","last_undegraded":"2024-04-02T13:51:56.662794+0000","last_fullsized":"2024-04-02T13:51:56.662794+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:30:59.935538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'6","reported_seq":106,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:52:38.441941+0000","last_change":"2024-04-02T13:51:00.247235+0000","last_active":"2024-04-02T13:52:38.441941+0000","last_peered":"2024-04-02T13:52:38.441941+0000","last_clean":"2024-04-02T13:52:38.441941+0000","last_became_active":"2024-04-02T13:51:00.246856+0000","last_became_peered":"2024-04-02T13:51:00.246856+0000","last_unstale":"2024-04-02T13:52:38.441941+0000","last_undegraded":"2024-04-02T13:52:38.441941+0000","last_fullsized":"2024-04-02T13:52:38.441941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:17:51.875952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663524+0000","last_change":"2024-04-02T13:51:01.263866+0000","last_active":"2024-04-02T13:51:56.663524+0000","last_peered":"2024-04-02T13:51:56.663524+0000","last_clean":"2024-04-02T13:51:56.663524+0000","last_became_active":"2024-04-02T13:51:01.263564+0000","last_became_peered":"2024-04-02T13:51:01.263564+0000","last_unstale":"2024-04-02T13:51:56.663524+0000","last_undegraded":"2024-04-02T13:51:56.663524+0000","last_fullsized":"2024-04-02T13:51:56.663524+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:59:33.663277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662098+0000","last_change":"2024-04-02T13:51:05.363079+0000","last_active":"2024-04-02T13:51:56.662098+0000","last_peered":"2024-04-02T13:51:56.662098+0000","last_clean":"2024-04-02T13:51:56.662098+0000","last_became_active":"2024-04-02T13:51:05.362631+0000","last_became_peered":"2024-04-02T13:51:05.362631+0000","last_unstale":"2024-04-02T13:51:56.662098+0000","last_undegraded":"2024-04-02T13:51:56.662098+0000","last_fullsized":"2024-04-02T13:51:56.662098+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:23.656269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662986+0000","last_change":"2024-04-02T13:51:00.242820+0000","last_active":"2024-04-02T13:51:56.662986+0000","last_peered":"2024-04-02T13:51:56.662986+0000","last_clean":"2024-04-02T13:51:56.662986+0000","last_became_active":"2024-04-02T13:51:00.242262+0000","last_became_peered":"2024-04-02T13:51:00.242262+0000","last_unstale":"2024-04-02T13:51:56.662986+0000","last_undegraded":"2024-04-02T13:51:56.662986+0000","last_fullsized":"2024-04-02T13:51:56.662986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:12:22.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663311+0000","last_change":"2024-04-02T13:51:01.266548+0000","last_active":"2024-04-02T13:51:56.663311+0000","last_peered":"2024-04-02T13:51:56.663311+0000","last_clean":"2024-04-02T13:51:56.663311+0000","last_became_active":"2024-04-02T13:51:01.259003+0000","last_became_peered":"2024-04-02T13:51:01.259003+0000","last_unstale":"2024-04-02T13:51:56.663311+0000","last_undegraded":"2024-04-02T13:51:56.663311+0000","last_fullsized":"2024-04-02T13:51:56.663311+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:38:53.692231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.661976+0000","last_change":"2024-04-02T13:51:05.359994+0000","last_active":"2024-04-02T13:51:56.661976+0000","last_peered":"2024-04-02T13:51:56.661976+0000","last_clean":"2024-04-02T13:51:56.661976+0000","last_became_active":"2024-04-02T13:51:05.359438+0000","last_became_peered":"2024-04-02T13:51:05.359438+0000","last_unstale":"2024-04-02T13:51:56.661976+0000","last_undegraded":"2024-04-02T13:51:56.661976+0000","last_fullsized":"2024-04-02T13:51:56.661976+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:09:43.929446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.555936+0000","last_change":"2024-04-02T13:51:00.237889+0000","last_active":"2024-04-02T13:51:56.555936+0000","last_peered":"2024-04-02T13:51:56.555936+0000","last_clean":"2024-04-02T13:51:56.555936+0000","last_became_active":"2024-04-02T13:51:00.237428+0000","last_became_peered":"2024-04-02T13:51:00.237428+0000","last_unstale":"2024-04-02T13:51:56.555936+0000","last_undegraded":"2024-04-02T13:51:56.555936+0000","last_fullsized":"2024-04-02T13:51:56.555936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:21:52.483656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663377+0000","last_change":"2024-04-02T13:51:01.262093+0000","last_active":"2024-04-02T13:51:56.663377+0000","last_peered":"2024-04-02T13:51:56.663377+0000","last_clean":"2024-04-02T13:51:56.663377+0000","last_became_active":"2024-04-02T13:51:01.258357+0000","last_became_peered":"2024-04-02T13:51:01.258357+0000","last_unstale":"2024-04-02T13:51:56.663377+0000","last_undegraded":"2024-04-02T13:51:56.663377+0000","last_fullsized":"2024-04-02T13:51:56.663377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:00:06.317307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.483560+0000","last_change":"2024-04-02T13:51:56.483560+0000","last_active":"2024-04-02T13:51:56.483560+0000","last_peered":"2024-04-02T13:51:56.483560+0000","last_clean":"2024-04-02T13:51:56.483560+0000","last_became_active":"2024-04-02T13:51:56.483090+0000","last_became_peered":"2024-04-02T13:51:56.483090+0000","last_unstale":"2024-04-02T13:51:56.483560+0000","last_undegraded":"2024-04-02T13:51:56.483560+0000","last_fullsized":"2024-04-02T13:51:56.483560+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:09:35.640492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.663817+0000","last_change":"2024-04-02T13:51:56.663817+0000","last_active":"2024-04-02T13:51:56.663817+0000","last_peered":"2024-04-02T13:51:56.663817+0000","last_clean":"2024-04-02T13:51:56.663817+0000","last_became_active":"2024-04-02T13:51:56.663473+0000","last_became_peered":"2024-04-02T13:51:56.663473+0000","last_unstale":"2024-04-02T13:51:56.663817+0000","last_undegraded":"2024-04-02T13:51:56.663817+0000","last_fullsized":"2024-04-02T13:51:56.663817+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:26:15.566030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.477283+0000","last_change":"2024-04-02T13:51:01.255563+0000","last_active":"2024-04-02T13:51:56.477283+0000","last_peered":"2024-04-02T13:51:56.477283+0000","last_clean":"2024-04-02T13:51:56.477283+0000","last_became_active":"2024-04-02T13:51:01.255228+0000","last_became_peered":"2024-04-02T13:51:01.255228+0000","last_unstale":"2024-04-02T13:51:56.477283+0000","last_undegraded":"2024-04-02T13:51:56.477283+0000","last_fullsized":"2024-04-02T13:51:56.477283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:48:28.200776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662423+0000","last_change":"2024-04-02T13:51:01.258153+0000","last_active":"2024-04-02T13:51:56.662423+0000","last_peered":"2024-04-02T13:51:56.662423+0000","last_clean":"2024-04-02T13:51:56.662423+0000","last_became_active":"2024-04-02T13:51:01.257656+0000","last_became_peered":"2024-04-02T13:51:01.257656+0000","last_unstale":"2024-04-02T13:51:56.662423+0000","last_undegraded":"2024-04-02T13:51:56.662423+0000","last_fullsized":"2024-04-02T13:51:56.662423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:37.887427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.556030+0000","last_change":"2024-04-02T13:51:01.254675+0000","last_active":"2024-04-02T13:51:56.556030+0000","last_peered":"2024-04-02T13:51:56.556030+0000","last_clean":"2024-04-02T13:51:56.556030+0000","last_became_active":"2024-04-02T13:51:01.254336+0000","last_became_peered":"2024-04-02T13:51:01.254336+0000","last_unstale":"2024-04-02T13:51:56.556030+0000","last_undegraded":"2024-04-02T13:51:56.556030+0000","last_fullsized":"2024-04-02T13:51:56.556030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:36.171961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.479161+0000","last_change":"2024-04-02T13:51:01.259457+0000","last_active":"2024-04-02T13:51:56.479161+0000","last_peered":"2024-04-02T13:51:56.479161+0000","last_clean":"2024-04-02T13:51:56.479161+0000","last_became_active":"2024-04-02T13:51:01.259099+0000","last_became_peered":"2024-04-02T13:51:01.259099+0000","last_unstale":"2024-04-02T13:51:56.479161+0000","last_undegraded":"2024-04-02T13:51:56.479161+0000","last_fullsized":"2024-04-02T13:51:56.479161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:13:06.594132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.478351+0000","last_change":"2024-04-02T13:51:01.248177+0000","last_active":"2024-04-02T13:51:56.478351+0000","last_peered":"2024-04-02T13:51:56.478351+0000","last_clean":"2024-04-02T13:51:56.478351+0000","last_became_active":"2024-04-02T13:51:01.247843+0000","last_became_peered":"2024-04-02T13:51:01.247843+0000","last_unstale":"2024-04-02T13:51:56.478351+0000","last_undegraded":"2024-04-02T13:51:56.478351+0000","last_fullsized":"2024-04-02T13:51:56.478351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:38:05.805382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.482303+0000","last_change":"2024-04-02T13:51:01.254822+0000","last_active":"2024-04-02T13:51:56.482303+0000","last_peered":"2024-04-02T13:51:56.482303+0000","last_clean":"2024-04-02T13:51:56.482303+0000","last_became_active":"2024-04-02T13:51:01.254451+0000","last_became_peered":"2024-04-02T13:51:01.254451+0000","last_unstale":"2024-04-02T13:51:56.482303+0000","last_undegraded":"2024-04-02T13:51:56.482303+0000","last_fullsized":"2024-04-02T13:51:56.482303+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:15:35.790371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.662337+0000","last_change":"2024-04-02T13:51:01.265467+0000","last_active":"2024-04-02T13:51:56.662337+0000","last_peered":"2024-04-02T13:51:56.662337+0000","last_clean":"2024-04-02T13:51:56.662337+0000","last_became_active":"2024-04-02T13:51:01.265072+0000","last_became_peered":"2024-04-02T13:51:01.265072+0000","last_unstale":"2024-04-02T13:51:56.662337+0000","last_undegraded":"2024-04-02T13:51:56.662337+0000","last_fullsized":"2024-04-02T13:51:56.662337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:12:07.702222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.481983+0000","last_change":"2024-04-02T13:51:01.252844+0000","last_active":"2024-04-02T13:51:56.481983+0000","last_peered":"2024-04-02T13:51:56.481983+0000","last_clean":"2024-04-02T13:51:56.481983+0000","last_became_active":"2024-04-02T13:51:01.252515+0000","last_became_peered":"2024-04-02T13:51:01.252515+0000","last_unstale":"2024-04-02T13:51:56.481983+0000","last_undegraded":"2024-04-02T13:51:56.481983+0000","last_fullsized":"2024-04-02T13:51:56.481983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:20:16.216418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.664080+0000","last_change":"2024-04-02T13:51:01.260141+0000","last_active":"2024-04-02T13:51:56.664080+0000","last_peered":"2024-04-02T13:51:56.664080+0000","last_clean":"2024-04-02T13:51:56.664080+0000","last_became_active":"2024-04-02T13:51:01.259530+0000","last_became_peered":"2024-04-02T13:51:01.259530+0000","last_unstale":"2024-04-02T13:51:56.664080+0000","last_undegraded":"2024-04-02T13:51:56.664080+0000","last_fullsized":"2024-04-02T13:51:56.664080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:00:03.579414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.556020+0000","last_change":"2024-04-02T13:51:01.259981+0000","last_active":"2024-04-02T13:51:56.556020+0000","last_peered":"2024-04-02T13:51:56.556020+0000","last_clean":"2024-04-02T13:51:56.556020+0000","last_became_active":"2024-04-02T13:51:01.259647+0000","last_became_peered":"2024-04-02T13:51:01.259647+0000","last_unstale":"2024-04-02T13:51:56.556020+0000","last_undegraded":"2024-04-02T13:51:56.556020+0000","last_fullsized":"2024-04-02T13:51:56.556020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:10:22.753767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:51:56.479479+0000","last_change":"2024-04-02T13:51:01.255342+0000","last_active":"2024-04-02T13:51:56.479479+0000","last_peered":"2024-04-02T13:51:56.479479+0000","last_clean":"2024-04-02T13:51:56.479479+0000","last_became_active":"2024-04-02T13:51:01.254679+0000","last_became_peered":"2024-04-02T13:51:01.254679+0000","last_unstale":"2024-04-02T13:51:56.479479+0000","last_undegraded":"2024-04-02T13:51:56.479479+0000","last_fullsized":"2024-04-02T13:51:56.479479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:25:12.198270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69146,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":753,"num_read_kb":454,"num_write":240,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":207438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351010,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":507658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612647,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28720,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716432,"statfs":{"total":95995035648,"available":95965626368,"internally_reserved":0,"allocated":1740800,"data_stored":1156348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776173,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":511225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037810,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":507507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332151,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36968,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708184,"statfs":{"total":95995035648,"available":95957180416,"internally_reserved":0,"allocated":1662976,"data_stored":1103613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495675,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":502633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1114112,"data_stored":565996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":504480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36440,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708712,"statfs":{"total":95995035648,"available":95957721088,"internally_reserved":0,"allocated":1126400,"data_stored":566539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1056768,"data_stored":502341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":506707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1671168,"data_stored":1099195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T13:52:46.985 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-02T13:52:46.986 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-02T13:52:46.986 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-02T13:52:46.986 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph health --format=json 2024-04-02T13:52:47.233 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:52:47.627 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:47 smithi052 ceph-mon[32104]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:52:47.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:47 smithi189 ceph-mon[36683]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:52:47.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:47 smithi088 ceph-mon[36528]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:52:48.765 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:48 smithi052 ceph-mon[32104]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-02T13:52:48.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:48 smithi189 ceph-mon[36683]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-02T13:52:48.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:48 smithi088 ceph-mon[36528]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-02T13:52:49.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.175+0000 7f2943118700 1 -- 172.21.15.52:0/1788465790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f293c101590 msgr2=0x7f293c101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:49.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.175+0000 7f2943118700 1 --2- 172.21.15.52:0/1788465790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f293c101590 0x7f293c101a10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2928009990 tx=0x7f2928009ca0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.175+0000 7f2943118700 1 -- 172.21.15.52:0/1788465790 shutdown_connections 2024-04-02T13:52:49.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.175+0000 7f2943118700 1 --2- 172.21.15.52:0/1788465790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f293c101f50 0x7f293c10e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.175+0000 7f2943118700 1 --2- 172.21.15.52:0/1788465790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f293c103470 0x7f293c103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.175+0000 7f2943118700 1 --2- 172.21.15.52:0/1788465790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f293c101590 0x7f293c101a10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.175+0000 7f2943118700 1 -- 172.21.15.52:0/1788465790 >> 172.21.15.52:0/1788465790 conn(0x7f293c077540 msgr2=0x7f293c077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:49.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.175+0000 7f2943118700 1 -- 172.21.15.52:0/1788465790 shutdown_connections 2024-04-02T13:52:49.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.175+0000 7f2943118700 1 -- 172.21.15.52:0/1788465790 wait complete. 2024-04-02T13:52:49.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.176+0000 7f2943118700 1 Processor -- start 2024-04-02T13:52:49.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.176+0000 7f2943118700 1 -- start start 2024-04-02T13:52:49.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.176+0000 7f2943118700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f293c101590 0x7f293c199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.176+0000 7f2943118700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f293c101f50 0x7f293c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.176+0000 7f2943118700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f293c103470 0x7f293c1a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.176+0000 7f2943118700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f293c111a10 con 0x7f293c101f50 2024-04-02T13:52:49.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.176+0000 7f2943118700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f293c111890 con 0x7f293c101590 2024-04-02T13:52:49.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.176+0000 7f2943118700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f293c111b90 con 0x7f293c103470 2024-04-02T13:52:49.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.177+0000 7f293bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f293c101f50 0x7f293c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.177+0000 7f293bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f293c101f50 0x7f293c19a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59942/0 (socket says 172.21.15.52:59942) 2024-04-02T13:52:49.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.177+0000 7f293bfff700 1 -- 172.21.15.52:0/2097090784 learned_addr learned my addr 172.21.15.52:0/2097090784 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:49.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.177+0000 7f2940eb4700 1 --2- 172.21.15.52:0/2097090784 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f293c101590 0x7f293c199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.177+0000 7f29416b5700 1 --2- 172.21.15.52:0/2097090784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f293c103470 0x7f293c1a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.177+0000 7f293bfff700 1 -- 172.21.15.52:0/2097090784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f293c103470 msgr2=0x7f293c1a4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:49.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.177+0000 7f293bfff700 1 --2- 172.21.15.52:0/2097090784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f293c103470 0x7f293c1a4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.177+0000 7f293bfff700 1 -- 172.21.15.52:0/2097090784 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f293c101590 msgr2=0x7f293c199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:49.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.177+0000 7f293bfff700 1 --2- 172.21.15.52:0/2097090784 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f293c101590 0x7f293c199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.177+0000 7f293bfff700 1 -- 172.21.15.52:0/2097090784 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293c1a5370 con 0x7f293c101f50 2024-04-02T13:52:49.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.177+0000 7f293bfff700 1 --2- 172.21.15.52:0/2097090784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f293c101f50 0x7f293c19a2f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f292800bff0 tx=0x7f2928005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:49.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.177+0000 7f2939ffb700 1 -- 172.21.15.52:0/2097090784 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928027070 con 0x7f293c101f50 2024-04-02T13:52:49.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.177+0000 7f2939ffb700 1 -- 172.21.15.52:0/2097090784 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29280040d0 con 0x7f293c101f50 2024-04-02T13:52:49.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.177+0000 7f2943118700 1 -- 172.21.15.52:0/2097090784 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f293c1a5540 con 0x7f293c101f50 2024-04-02T13:52:49.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.177+0000 7f2939ffb700 1 -- 172.21.15.52:0/2097090784 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928022e00 con 0x7f293c101f50 2024-04-02T13:52:49.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.178+0000 7f2943118700 1 -- 172.21.15.52:0/2097090784 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f293c1a5a70 con 0x7f293c101f50 2024-04-02T13:52:49.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.179+0000 7f2943118700 1 -- 172.21.15.52:0/2097090784 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f293c0692b0 con 0x7f293c101f50 2024-04-02T13:52:49.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.180+0000 7f2939ffb700 1 -- 172.21.15.52:0/2097090784 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f292802a070 con 0x7f293c101f50 2024-04-02T13:52:49.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.180+0000 7f2939ffb700 1 --2- 172.21.15.52:0/2097090784 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f292c0a3d50 0x7f292c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.180+0000 7f2940eb4700 1 --2- 172.21.15.52:0/2097090784 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f292c0a3d50 0x7f292c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.180+0000 7f2939ffb700 1 -- 172.21.15.52:0/2097090784 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f29280d1630 con 0x7f293c101f50 2024-04-02T13:52:49.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.181+0000 7f2940eb4700 1 --2- 172.21.15.52:0/2097090784 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f292c0a3d50 0x7f292c0a6210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2930009a10 tx=0x7f2930009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:49.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.185+0000 7f2939ffb700 1 -- 172.21.15.52:0/2097090784 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f292805ef20 con 0x7f293c101f50 2024-04-02T13:52:49.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.445+0000 7f2943118700 1 -- 172.21.15.52:0/2097090784 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f293c02dd30 con 0x7f293c101f50 2024-04-02T13:52:49.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.445+0000 7f2939ffb700 1 -- 172.21.15.52:0/2097090784 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f2928013070 con 0x7f293c101f50 2024-04-02T13:52:49.449 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:52:49.449 INFO:teuthology.orchestra.run.smithi052.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-02T13:52:49.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.447+0000 7f2943118700 1 -- 172.21.15.52:0/2097090784 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f292c0a3d50 msgr2=0x7f292c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:49.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.447+0000 7f2943118700 1 --2- 172.21.15.52:0/2097090784 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f292c0a3d50 0x7f292c0a6210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2930009a10 tx=0x7f2930009190 comp rx=0 tx=0).stop 2024-04-02T13:52:49.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.447+0000 7f2943118700 1 -- 172.21.15.52:0/2097090784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f293c101f50 msgr2=0x7f293c19a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:49.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.447+0000 7f2943118700 1 --2- 172.21.15.52:0/2097090784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f293c101f50 0x7f293c19a2f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f292800bff0 tx=0x7f2928005590 comp rx=0 tx=0).stop 2024-04-02T13:52:49.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.447+0000 7f2943118700 1 -- 172.21.15.52:0/2097090784 shutdown_connections 2024-04-02T13:52:49.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.447+0000 7f2943118700 1 --2- 172.21.15.52:0/2097090784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f293c103470 0x7f293c1a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.447+0000 7f2943118700 1 --2- 172.21.15.52:0/2097090784 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f292c0a3d50 0x7f292c0a6210 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.447+0000 7f2943118700 1 --2- 172.21.15.52:0/2097090784 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f293c101590 0x7f293c199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.447+0000 7f2943118700 1 --2- 172.21.15.52:0/2097090784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f293c101f50 0x7f293c19a2f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.447+0000 7f2943118700 1 -- 172.21.15.52:0/2097090784 >> 172.21.15.52:0/2097090784 conn(0x7f293c077540 msgr2=0x7f293c105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:49.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.448+0000 7f2943118700 1 -- 172.21.15.52:0/2097090784 shutdown_connections 2024-04-02T13:52:49.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:49.448+0000 7f2943118700 1 -- 172.21.15.52:0/2097090784 wait complete. 2024-04-02T13:52:49.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:49 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2097090784' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-02T13:52:49.804 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:49 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2097090784' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-02T13:52:49.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:49 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2097090784' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-02T13:52:50.247 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-02T13:52:50.248 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:52:50.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.359+0000 7fb557a22640 1 -- 172.21.15.52:0/1416734335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5501025f0 msgr2=0x7fb550108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:50.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.359+0000 7fb557a22640 1 --2- 172.21.15.52:0/1416734335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5501025f0 0x7fb550108e80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb548009ca0 tx=0x7fb54802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:52:50.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.359+0000 7fb557a22640 1 -- 172.21.15.52:0/1416734335 shutdown_connections 2024-04-02T13:52:50.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.359+0000 7fb557a22640 1 --2- 172.21.15.52:0/1416734335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5501025f0 0x7fb550108e80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:50.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.359+0000 7fb557a22640 1 --2- 172.21.15.52:0/1416734335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb550101c30 0x7fb5501020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:50.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.359+0000 7fb557a22640 1 --2- 172.21.15.52:0/1416734335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb550100a40 0x7fb550100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:50.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.359+0000 7fb557a22640 1 -- 172.21.15.52:0/1416734335 >> 172.21.15.52:0/1416734335 conn(0x7fb5500fbfb0 msgr2=0x7fb5500fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:50.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.359+0000 7fb557a22640 1 -- 172.21.15.52:0/1416734335 shutdown_connections 2024-04-02T13:52:50.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.359+0000 7fb557a22640 1 -- 172.21.15.52:0/1416734335 wait complete. 2024-04-02T13:52:50.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.360+0000 7fb557a22640 1 Processor -- start 2024-04-02T13:52:50.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.360+0000 7fb557a22640 1 -- start start 2024-04-02T13:52:50.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.360+0000 7fb557a22640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb550100a40 0x7fb550197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:50.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.360+0000 7fb557a22640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb550101c30 0x7fb5501983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:50.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.360+0000 7fb557a22640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5501025f0 0x7fb55019f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:50.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.360+0000 7fb557a22640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb55010c6a0 con 0x7fb550100a40 2024-04-02T13:52:50.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.360+0000 7fb557a22640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb55010c520 con 0x7fb5501025f0 2024-04-02T13:52:50.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.360+0000 7fb557a22640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb55010c820 con 0x7fb550101c30 2024-04-02T13:52:50.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.360+0000 7fb5557be640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb550100a40 0x7fb550197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:50.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.360+0000 7fb5557be640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb550100a40 0x7fb550197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59972/0 (socket says 172.21.15.52:59972) 2024-04-02T13:52:50.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.360+0000 7fb5557be640 1 -- 172.21.15.52:0/3453826422 learned_addr learned my addr 172.21.15.52:0/3453826422 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:50.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.360+0000 7fb555fbf640 1 --2- 172.21.15.52:0/3453826422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5501025f0 0x7fb55019f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:50.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.360+0000 7fb554fbd640 1 --2- 172.21.15.52:0/3453826422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb550101c30 0x7fb5501983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:50.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.361+0000 7fb5557be640 1 -- 172.21.15.52:0/3453826422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb550101c30 msgr2=0x7fb5501983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:50.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.361+0000 7fb5557be640 1 --2- 172.21.15.52:0/3453826422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb550101c30 0x7fb5501983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:50.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.361+0000 7fb5557be640 1 -- 172.21.15.52:0/3453826422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5501025f0 msgr2=0x7fb55019f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:50.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.361+0000 7fb5557be640 1 --2- 172.21.15.52:0/3453826422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5501025f0 0x7fb55019f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:50.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.361+0000 7fb5557be640 1 -- 172.21.15.52:0/3453826422 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb55019fa80 con 0x7fb550100a40 2024-04-02T13:52:50.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.361+0000 7fb554fbd640 1 --2- 172.21.15.52:0/3453826422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb550101c30 0x7fb5501983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:52:50.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.361+0000 7fb5557be640 1 --2- 172.21.15.52:0/3453826422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb550100a40 0x7fb550197e60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00d950 tx=0x7fb54c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:50.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.361+0000 7fb5467fc640 1 -- 172.21.15.52:0/3453826422 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c017020 con 0x7fb550100a40 2024-04-02T13:52:50.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.361+0000 7fb557a22640 1 -- 172.21.15.52:0/3453826422 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb55019fd70 con 0x7fb550100a40 2024-04-02T13:52:50.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.361+0000 7fb5467fc640 1 -- 172.21.15.52:0/3453826422 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb54c009c30 con 0x7fb550100a40 2024-04-02T13:52:50.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.361+0000 7fb5467fc640 1 -- 172.21.15.52:0/3453826422 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c00f740 con 0x7fb550100a40 2024-04-02T13:52:50.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.361+0000 7fb557a22640 1 -- 172.21.15.52:0/3453826422 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5501a0200 con 0x7fb550100a40 2024-04-02T13:52:50.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.362+0000 7fb557a22640 1 -- 172.21.15.52:0/3453826422 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5501a0490 con 0x7fb550100a40 2024-04-02T13:52:50.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.363+0000 7fb5467fc640 1 -- 172.21.15.52:0/3453826422 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb54c01a020 con 0x7fb550100a40 2024-04-02T13:52:50.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.364+0000 7fb5467fc640 1 --2- 172.21.15.52:0/3453826422 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb53c0a3c40 0x7fb53c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:50.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.364+0000 7fb554fbd640 1 --2- 172.21.15.52:0/3453826422 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb53c0a3c40 0x7fb53c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:50.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.364+0000 7fb5467fc640 1 -- 172.21.15.52:0/3453826422 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb54c0d0660 con 0x7fb550100a40 2024-04-02T13:52:50.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.365+0000 7fb554fbd640 1 --2- 172.21.15.52:0/3453826422 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb53c0a3c40 0x7fb53c0a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb540005fd0 tx=0x7fb540005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:50.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.368+0000 7fb5467fc640 1 -- 172.21.15.52:0/3453826422 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb54c021070 con 0x7fb550100a40 2024-04-02T13:52:50.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.556+0000 7fb557a22640 1 -- 172.21.15.52:0/3453826422 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5501a0820 con 0x7fb550100a40 2024-04-02T13:52:50.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.558+0000 7fb5467fc640 1 -- 172.21.15.52:0/3453826422 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24549 (secure 0 0 0) 0x7fb54c009de0 con 0x7fb550100a40 2024-04-02T13:52:50.561 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:52:50.561 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":92,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T13:51:56.472064+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:50:58.778690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:50:59.561527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:51:03.343487+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:52:50.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.560+0000 7fb557a22640 1 -- 172.21.15.52:0/3453826422 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb53c0a3c40 msgr2=0x7fb53c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:50.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.560+0000 7fb557a22640 1 --2- 172.21.15.52:0/3453826422 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb53c0a3c40 0x7fb53c0a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb540005fd0 tx=0x7fb540005850 comp rx=0 tx=0).stop 2024-04-02T13:52:50.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.560+0000 7fb557a22640 1 -- 172.21.15.52:0/3453826422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb550100a40 msgr2=0x7fb550197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:50.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.560+0000 7fb557a22640 1 --2- 172.21.15.52:0/3453826422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb550100a40 0x7fb550197e60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00d950 tx=0x7fb54c00de10 comp rx=0 tx=0).stop 2024-04-02T13:52:50.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.560+0000 7fb557a22640 1 -- 172.21.15.52:0/3453826422 shutdown_connections 2024-04-02T13:52:50.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.560+0000 7fb557a22640 1 --2- 172.21.15.52:0/3453826422 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb53c0a3c40 0x7fb53c0a6100 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:50.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.560+0000 7fb557a22640 1 --2- 172.21.15.52:0/3453826422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5501025f0 0x7fb55019f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:50.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.560+0000 7fb557a22640 1 --2- 172.21.15.52:0/3453826422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb550101c30 0x7fb5501983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:50.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.560+0000 7fb557a22640 1 --2- 172.21.15.52:0/3453826422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb550100a40 0x7fb550197e60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:50.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.560+0000 7fb557a22640 1 -- 172.21.15.52:0/3453826422 >> 172.21.15.52:0/3453826422 conn(0x7fb5500fbfb0 msgr2=0x7fb550069e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:50.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.560+0000 7fb557a22640 1 -- 172.21.15.52:0/3453826422 shutdown_connections 2024-04-02T13:52:50.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.560+0000 7fb557a22640 1 -- 172.21.15.52:0/3453826422 wait complete. 2024-04-02T13:52:50.574 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:50 smithi052 ceph-mon[32104]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:52:50.575 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:46:20.746053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:50:58.778690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:50:59.561527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:51:03.343487+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:52:50.576 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:52:50.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.679+0000 7f612c677640 1 -- 172.21.15.52:0/2257658083 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6124102650 msgr2=0x7f6124108ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:50.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.679+0000 7f612c677640 1 --2- 172.21.15.52:0/2257658083 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6124102650 0x7f6124108ee0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f611c009ca0 tx=0x7f611c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:52:50.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.679+0000 7f612c677640 1 -- 172.21.15.52:0/2257658083 shutdown_connections 2024-04-02T13:52:50.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.679+0000 7f612c677640 1 --2- 172.21.15.52:0/2257658083 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6124102650 0x7f6124108ee0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:50.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.679+0000 7f612c677640 1 --2- 172.21.15.52:0/2257658083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6124101c90 0x7f6124102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:50.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.679+0000 7f612c677640 1 --2- 172.21.15.52:0/2257658083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6124100aa0 0x7f6124100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:50.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.679+0000 7f612c677640 1 -- 172.21.15.52:0/2257658083 >> 172.21.15.52:0/2257658083 conn(0x7f61240fc010 msgr2=0x7f61240fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:50.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.679+0000 7f612c677640 1 -- 172.21.15.52:0/2257658083 shutdown_connections 2024-04-02T13:52:50.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.679+0000 7f612c677640 1 -- 172.21.15.52:0/2257658083 wait complete. 2024-04-02T13:52:50.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.680+0000 7f612c677640 1 Processor -- start 2024-04-02T13:52:50.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.680+0000 7f612c677640 1 -- start start 2024-04-02T13:52:50.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.680+0000 7f612c677640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6124100aa0 0x7f6124197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:50.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.680+0000 7f612c677640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6124101c90 0x7f6124198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:50.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.680+0000 7f612c677640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6124102650 0x7f612419f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:50.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.680+0000 7f612c677640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f612410c6b0 con 0x7f6124100aa0 2024-04-02T13:52:50.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.680+0000 7f612c677640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f612410c530 con 0x7f6124102650 2024-04-02T13:52:50.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.680+0000 7f612c677640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f612410c830 con 0x7f6124101c90 2024-04-02T13:52:50.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.680+0000 7f612a413640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6124100aa0 0x7f6124197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:50.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.680+0000 7f612a413640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6124100aa0 0x7f6124197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60004/0 (socket says 172.21.15.52:60004) 2024-04-02T13:52:50.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.680+0000 7f612a413640 1 -- 172.21.15.52:0/1359274606 learned_addr learned my addr 172.21.15.52:0/1359274606 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:50.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.680+0000 7f612ac14640 1 --2- 172.21.15.52:0/1359274606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6124102650 0x7f612419f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:50.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.680+0000 7f6129c12640 1 --2- 172.21.15.52:0/1359274606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6124101c90 0x7f6124198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.680+0000 7f612a413640 1 -- 172.21.15.52:0/1359274606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6124101c90 msgr2=0x7f6124198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.680+0000 7f612a413640 1 --2- 172.21.15.52:0/1359274606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6124101c90 0x7f6124198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.680+0000 7f612a413640 1 -- 172.21.15.52:0/1359274606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6124102650 msgr2=0x7f612419f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.680+0000 7f612a413640 1 --2- 172.21.15.52:0/1359274606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6124102650 0x7f612419f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.680+0000 7f612a413640 1 -- 172.21.15.52:0/1359274606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612419fa60 con 0x7f6124100aa0 2024-04-02T13:52:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.680+0000 7f6129c12640 1 --2- 172.21.15.52:0/1359274606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6124101c90 0x7f6124198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:52:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.681+0000 7f612a413640 1 --2- 172.21.15.52:0/1359274606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6124100aa0 0x7f6124197e40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f612000d950 tx=0x7f612000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.681+0000 7f611b7fe640 1 -- 172.21.15.52:0/1359274606 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120017020 con 0x7f6124100aa0 2024-04-02T13:52:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.681+0000 7f611b7fe640 1 -- 172.21.15.52:0/1359274606 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6120009c30 con 0x7f6124100aa0 2024-04-02T13:52:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.681+0000 7f612c677640 1 -- 172.21.15.52:0/1359274606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612419fd50 con 0x7f6124100aa0 2024-04-02T13:52:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.681+0000 7f611b7fe640 1 -- 172.21.15.52:0/1359274606 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612000f740 con 0x7f6124100aa0 2024-04-02T13:52:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.681+0000 7f612c677640 1 -- 172.21.15.52:0/1359274606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61241a01e0 con 0x7f6124100aa0 2024-04-02T13:52:50.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.682+0000 7f612c677640 1 -- 172.21.15.52:0/1359274606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61241a0470 con 0x7f6124100aa0 2024-04-02T13:52:50.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.683+0000 7f611b7fe640 1 -- 172.21.15.52:0/1359274606 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f612001a020 con 0x7f6124100aa0 2024-04-02T13:52:50.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.684+0000 7f611b7fe640 1 --2- 172.21.15.52:0/1359274606 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f61100a3c40 0x7f61100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:50.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.684+0000 7f6129c12640 1 --2- 172.21.15.52:0/1359274606 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f61100a3c40 0x7f61100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:50.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.684+0000 7f611b7fe640 1 -- 172.21.15.52:0/1359274606 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f61200d0660 con 0x7f6124100aa0 2024-04-02T13:52:50.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.684+0000 7f6129c12640 1 --2- 172.21.15.52:0/1359274606 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f61100a3c40 0x7f61100a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6114005d70 tx=0x7f6114005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:50.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.688+0000 7f611b7fe640 1 -- 172.21.15.52:0/1359274606 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6120021070 con 0x7f6124100aa0 2024-04-02T13:52:50.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:50 smithi189 ceph-mon[36683]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:52:50.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.873+0000 7f612c677640 1 -- 172.21.15.52:0/1359274606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f61241a0800 con 0x7f6124100aa0 2024-04-02T13:52:50.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.873+0000 7f611b7fe640 1 -- 172.21.15.52:0/1359274606 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f6120009de0 con 0x7f6124100aa0 2024-04-02T13:52:50.877 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-04-02T13:52:50.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.875+0000 7f612c677640 1 -- 172.21.15.52:0/1359274606 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f61100a3c40 msgr2=0x7f61100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:50.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.875+0000 7f612c677640 1 --2- 172.21.15.52:0/1359274606 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f61100a3c40 0x7f61100a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6114005d70 tx=0x7f6114005d00 comp rx=0 tx=0).stop 2024-04-02T13:52:50.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.876+0000 7f612c677640 1 -- 172.21.15.52:0/1359274606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6124100aa0 msgr2=0x7f6124197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:50.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.876+0000 7f612c677640 1 --2- 172.21.15.52:0/1359274606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6124100aa0 0x7f6124197e40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f612000d950 tx=0x7f612000de10 comp rx=0 tx=0).stop 2024-04-02T13:52:50.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.876+0000 7f612c677640 1 -- 172.21.15.52:0/1359274606 shutdown_connections 2024-04-02T13:52:50.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.876+0000 7f612c677640 1 --2- 172.21.15.52:0/1359274606 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f61100a3c40 0x7f61100a6100 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:50.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.876+0000 7f612c677640 1 --2- 172.21.15.52:0/1359274606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6124102650 0x7f612419f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:50.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.876+0000 7f612c677640 1 --2- 172.21.15.52:0/1359274606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6124101c90 0x7f6124198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:50.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.876+0000 7f612c677640 1 --2- 172.21.15.52:0/1359274606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6124100aa0 0x7f6124197e40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:50.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.876+0000 7f612c677640 1 -- 172.21.15.52:0/1359274606 >> 172.21.15.52:0/1359274606 conn(0x7f61240fc010 msgr2=0x7f61240fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:50.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.876+0000 7f612c677640 1 -- 172.21.15.52:0/1359274606 shutdown_connections 2024-04-02T13:52:50.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.876+0000 7f612c677640 1 -- 172.21.15.52:0/1359274606 wait complete. 2024-04-02T13:52:50.890 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:52:50.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:50 smithi088 ceph-mon[36528]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:52:50.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.992+0000 7fc091932640 1 -- 172.21.15.52:0/2765591718 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc08c104eb0 msgr2=0x7fc08c1072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:50.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.992+0000 7fc091932640 1 --2- 172.21.15.52:0/2765591718 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc08c104eb0 0x7fc08c1072a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc074009900 tx=0x7fc07402bc30 comp rx=0 tx=0).stop 2024-04-02T13:52:50.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.992+0000 7fc091932640 1 -- 172.21.15.52:0/2765591718 shutdown_connections 2024-04-02T13:52:50.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.992+0000 7fc091932640 1 --2- 172.21.15.52:0/2765591718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc08c1077e0 0x7fc08c109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:50.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.992+0000 7fc091932640 1 --2- 172.21.15.52:0/2765591718 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc08c104eb0 0x7fc08c1072a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:50.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.992+0000 7fc091932640 1 --2- 172.21.15.52:0/2765591718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc08c102580 0x7fc08c104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:50.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.992+0000 7fc091932640 1 -- 172.21.15.52:0/2765591718 >> 172.21.15.52:0/2765591718 conn(0x7fc08c0fbf90 msgr2=0x7fc08c0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:50.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.992+0000 7fc091932640 1 -- 172.21.15.52:0/2765591718 shutdown_connections 2024-04-02T13:52:50.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.992+0000 7fc091932640 1 -- 172.21.15.52:0/2765591718 wait complete. 2024-04-02T13:52:50.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.993+0000 7fc091932640 1 Processor -- start 2024-04-02T13:52:50.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.993+0000 7fc091932640 1 -- start start 2024-04-02T13:52:50.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.993+0000 7fc091932640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc08c102580 0x7fc08c197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:50.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.993+0000 7fc091932640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc08c104eb0 0x7fc08c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:50.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.993+0000 7fc091932640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc08c1077e0 0x7fc08c19f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:50.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.993+0000 7fc091932640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c10d780 con 0x7fc08c1077e0 2024-04-02T13:52:50.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.993+0000 7fc091932640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c10d600 con 0x7fc08c102580 2024-04-02T13:52:50.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.993+0000 7fc091932640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c10d900 con 0x7fc08c104eb0 2024-04-02T13:52:50.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.993+0000 7fc08b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc08c1077e0 0x7fc08c19f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:50.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.993+0000 7fc08b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc08c1077e0 0x7fc08c19f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60028/0 (socket says 172.21.15.52:60028) 2024-04-02T13:52:50.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.993+0000 7fc08b7fe640 1 -- 172.21.15.52:0/1873656955 learned_addr learned my addr 172.21.15.52:0/1873656955 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:50.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.994+0000 7fc08a7fc640 1 --2- 172.21.15.52:0/1873656955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc08c104eb0 0x7fc08c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:50.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.994+0000 7fc08affd640 1 --2- 172.21.15.52:0/1873656955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc08c102580 0x7fc08c197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:50.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.994+0000 7fc08b7fe640 1 -- 172.21.15.52:0/1873656955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc08c104eb0 msgr2=0x7fc08c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:50.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.994+0000 7fc08b7fe640 1 --2- 172.21.15.52:0/1873656955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc08c104eb0 0x7fc08c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:50.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.994+0000 7fc08b7fe640 1 -- 172.21.15.52:0/1873656955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc08c102580 msgr2=0x7fc08c197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:50.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.994+0000 7fc08b7fe640 1 --2- 172.21.15.52:0/1873656955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc08c102580 0x7fc08c197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.994+0000 7fc08b7fe640 1 -- 172.21.15.52:0/1873656955 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08c19faa0 con 0x7fc08c1077e0 2024-04-02T13:52:51.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.994+0000 7fc08a7fc640 1 --2- 172.21.15.52:0/1873656955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc08c104eb0 0x7fc08c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:52:51.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.994+0000 7fc08b7fe640 1 --2- 172.21.15.52:0/1873656955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc08c1077e0 0x7fc08c19f430 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00eaa0 tx=0x7fc07c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:51.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.994+0000 7fc090930640 1 -- 172.21.15.52:0/1873656955 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c00cdb0 con 0x7fc08c1077e0 2024-04-02T13:52:51.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.994+0000 7fc090930640 1 -- 172.21.15.52:0/1873656955 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc07c005020 con 0x7fc08c1077e0 2024-04-02T13:52:51.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.994+0000 7fc091932640 1 -- 172.21.15.52:0/1873656955 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08c19fd90 con 0x7fc08c1077e0 2024-04-02T13:52:51.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.994+0000 7fc090930640 1 -- 172.21.15.52:0/1873656955 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07c0116b0 con 0x7fc08c1077e0 2024-04-02T13:52:51.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.994+0000 7fc091932640 1 -- 172.21.15.52:0/1873656955 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc08c1a02b0 con 0x7fc08c1077e0 2024-04-02T13:52:51.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.995+0000 7fc091932640 1 -- 172.21.15.52:0/1873656955 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc08c0fda80 con 0x7fc08c1077e0 2024-04-02T13:52:51.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.996+0000 7fc090930640 1 -- 172.21.15.52:0/1873656955 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc07c005560 con 0x7fc08c1077e0 2024-04-02T13:52:51.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.997+0000 7fc090930640 1 --2- 172.21.15.52:0/1873656955 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc0780a3cc0 0x7fc0780a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:51.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.997+0000 7fc08affd640 1 --2- 172.21.15.52:0/1873656955 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc0780a3cc0 0x7fc0780a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:51.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.997+0000 7fc090930640 1 -- 172.21.15.52:0/1873656955 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc07c0d0f10 con 0x7fc08c1077e0 2024-04-02T13:52:51.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:50.997+0000 7fc08affd640 1 --2- 172.21.15.52:0/1873656955 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc0780a3cc0 0x7fc0780a6180 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc084009aa0 tx=0x7fc084005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:51.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.000+0000 7fc090930640 1 -- 172.21.15.52:0/1873656955 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc07c016030 con 0x7fc08c1077e0 2024-04-02T13:52:51.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.186+0000 7fc091932640 1 -- 172.21.15.52:0/1873656955 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc08c02d370 con 0x7fc08c1077e0 2024-04-02T13:52:51.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.186+0000 7fc090930640 1 -- 172.21.15.52:0/1873656955 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7fc07c09d4b0 con 0x7fc08c1077e0 2024-04-02T13:52:51.189 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-04-02T13:52:51.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.188+0000 7fc091932640 1 -- 172.21.15.52:0/1873656955 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc0780a3cc0 msgr2=0x7fc0780a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:51.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.188+0000 7fc091932640 1 --2- 172.21.15.52:0/1873656955 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc0780a3cc0 0x7fc0780a6180 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc084009aa0 tx=0x7fc084005f90 comp rx=0 tx=0).stop 2024-04-02T13:52:51.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.188+0000 7fc091932640 1 -- 172.21.15.52:0/1873656955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc08c1077e0 msgr2=0x7fc08c19f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:51.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.188+0000 7fc091932640 1 --2- 172.21.15.52:0/1873656955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc08c1077e0 0x7fc08c19f430 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00eaa0 tx=0x7fc07c00ef60 comp rx=0 tx=0).stop 2024-04-02T13:52:51.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.188+0000 7fc091932640 1 -- 172.21.15.52:0/1873656955 shutdown_connections 2024-04-02T13:52:51.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.188+0000 7fc091932640 1 --2- 172.21.15.52:0/1873656955 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc0780a3cc0 0x7fc0780a6180 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.188+0000 7fc091932640 1 --2- 172.21.15.52:0/1873656955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc08c1077e0 0x7fc08c19f430 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.188+0000 7fc091932640 1 --2- 172.21.15.52:0/1873656955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc08c104eb0 0x7fc08c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.188+0000 7fc091932640 1 --2- 172.21.15.52:0/1873656955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc08c102580 0x7fc08c197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.188+0000 7fc091932640 1 -- 172.21.15.52:0/1873656955 >> 172.21.15.52:0/1873656955 conn(0x7fc08c0fbf90 msgr2=0x7fc08c102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:51.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.188+0000 7fc091932640 1 -- 172.21.15.52:0/1873656955 shutdown_connections 2024-04-02T13:52:51.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.188+0000 7fc091932640 1 -- 172.21.15.52:0/1873656955 wait complete. 2024-04-02T13:52:51.203 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:52:51.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.305+0000 7fe6fcaba640 1 -- 172.21.15.52:0/3417463629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6f8101c90 msgr2=0x7fe6f8102110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:51.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.305+0000 7fe6fcaba640 1 --2- 172.21.15.52:0/3417463629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6f8101c90 0x7fe6f8102110 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009900 tx=0x7fe6ec02bc30 comp rx=0 tx=0).stop 2024-04-02T13:52:51.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.305+0000 7fe6fcaba640 1 -- 172.21.15.52:0/3417463629 shutdown_connections 2024-04-02T13:52:51.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.305+0000 7fe6fcaba640 1 --2- 172.21.15.52:0/3417463629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6f8102650 0x7fe6f8108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.305+0000 7fe6fcaba640 1 --2- 172.21.15.52:0/3417463629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6f8101c90 0x7fe6f8102110 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.305+0000 7fe6fcaba640 1 --2- 172.21.15.52:0/3417463629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6f8100aa0 0x7fe6f8100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.305+0000 7fe6fcaba640 1 -- 172.21.15.52:0/3417463629 >> 172.21.15.52:0/3417463629 conn(0x7fe6f80fc010 msgr2=0x7fe6f80fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:51.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.305+0000 7fe6fcaba640 1 -- 172.21.15.52:0/3417463629 shutdown_connections 2024-04-02T13:52:51.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.305+0000 7fe6fcaba640 1 -- 172.21.15.52:0/3417463629 wait complete. 2024-04-02T13:52:51.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.305+0000 7fe6fcaba640 1 Processor -- start 2024-04-02T13:52:51.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.306+0000 7fe6fcaba640 1 -- start start 2024-04-02T13:52:51.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.306+0000 7fe6fcaba640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6f8100aa0 0x7fe6f806ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:51.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.306+0000 7fe6fcaba640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6f8101c90 0x7fe6f819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:51.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.306+0000 7fe6fcaba640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6f8102650 0x7fe6f819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:51.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.306+0000 7fe6fcaba640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f810c690 con 0x7fe6f8101c90 2024-04-02T13:52:51.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.306+0000 7fe6fcaba640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f810c510 con 0x7fe6f8102650 2024-04-02T13:52:51.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.306+0000 7fe6fcaba640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f810c810 con 0x7fe6f8100aa0 2024-04-02T13:52:51.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.306+0000 7fe6f5d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6f8101c90 0x7fe6f819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:51.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.306+0000 7fe6f5d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6f8101c90 0x7fe6f819ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60052/0 (socket says 172.21.15.52:60052) 2024-04-02T13:52:51.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.306+0000 7fe6f5d9b640 1 -- 172.21.15.52:0/3575756846 learned_addr learned my addr 172.21.15.52:0/3575756846 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:51.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.306+0000 7fe6f6d9d640 1 --2- 172.21.15.52:0/3575756846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6f8102650 0x7fe6f819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:51.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.306+0000 7fe6f659c640 1 --2- 172.21.15.52:0/3575756846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6f8100aa0 0x7fe6f806ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:51.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.306+0000 7fe6f5d9b640 1 -- 172.21.15.52:0/3575756846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6f8100aa0 msgr2=0x7fe6f806ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:51.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.306+0000 7fe6f5d9b640 1 --2- 172.21.15.52:0/3575756846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6f8100aa0 0x7fe6f806ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.306+0000 7fe6f5d9b640 1 -- 172.21.15.52:0/3575756846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6f8102650 msgr2=0x7fe6f819f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:51.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.306+0000 7fe6f5d9b640 1 --2- 172.21.15.52:0/3575756846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6f8102650 0x7fe6f819f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.306+0000 7fe6f5d9b640 1 -- 172.21.15.52:0/3575756846 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f819f9c0 con 0x7fe6f8101c90 2024-04-02T13:52:51.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.306+0000 7fe6f5d9b640 1 --2- 172.21.15.52:0/3575756846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6f8101c90 0x7fe6f819ca90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009230 tx=0x7fe6ec005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:51.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.307+0000 7fe6e77fe640 1 -- 172.21.15.52:0/3575756846 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec041070 con 0x7fe6f8101c90 2024-04-02T13:52:51.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.307+0000 7fe6e77fe640 1 -- 172.21.15.52:0/3575756846 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6ec0044d0 con 0x7fe6f8101c90 2024-04-02T13:52:51.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.307+0000 7fe6e77fe640 1 -- 172.21.15.52:0/3575756846 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec034440 con 0x7fe6f8101c90 2024-04-02T13:52:51.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.307+0000 7fe6fcaba640 1 -- 172.21.15.52:0/3575756846 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6f819fc50 con 0x7fe6f8101c90 2024-04-02T13:52:51.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.307+0000 7fe6fcaba640 1 -- 172.21.15.52:0/3575756846 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6f81a0190 con 0x7fe6f8101c90 2024-04-02T13:52:51.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.308+0000 7fe6fcaba640 1 -- 172.21.15.52:0/3575756846 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6f8107dc0 con 0x7fe6f8101c90 2024-04-02T13:52:51.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.309+0000 7fe6e77fe640 1 -- 172.21.15.52:0/3575756846 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe6ec044070 con 0x7fe6f8101c90 2024-04-02T13:52:51.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.309+0000 7fe6e77fe640 1 --2- 172.21.15.52:0/3575756846 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe6e00a3cc0 0x7fe6e00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:51.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.309+0000 7fe6f659c640 1 --2- 172.21.15.52:0/3575756846 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe6e00a3cc0 0x7fe6e00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:51.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.309+0000 7fe6e77fe640 1 -- 172.21.15.52:0/3575756846 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fe6ec0f2300 con 0x7fe6f8101c90 2024-04-02T13:52:51.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.310+0000 7fe6f659c640 1 --2- 172.21.15.52:0/3575756846 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe6e00a3cc0 0x7fe6e00a6180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8101af0 tx=0x7fe6f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:51.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.314+0000 7fe6e77fe640 1 -- 172.21.15.52:0/3575756846 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe6ec0be2e0 con 0x7fe6f8101c90 2024-04-02T13:52:51.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.501+0000 7fe6fcaba640 1 -- 172.21.15.52:0/3575756846 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe6f802d500 con 0x7fe6f8101c90 2024-04-02T13:52:51.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.501+0000 7fe6e77fe640 1 -- 172.21.15.52:0/3575756846 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7fe6ec0c25e0 con 0x7fe6f8101c90 2024-04-02T13:52:51.505 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-04-02T13:52:51.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.503+0000 7fe6fcaba640 1 -- 172.21.15.52:0/3575756846 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe6e00a3cc0 msgr2=0x7fe6e00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:51.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.503+0000 7fe6fcaba640 1 --2- 172.21.15.52:0/3575756846 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe6e00a3cc0 0x7fe6e00a6180 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8101af0 tx=0x7fe6f0005f90 comp rx=0 tx=0).stop 2024-04-02T13:52:51.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.503+0000 7fe6fcaba640 1 -- 172.21.15.52:0/3575756846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6f8101c90 msgr2=0x7fe6f819ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:51.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.503+0000 7fe6fcaba640 1 --2- 172.21.15.52:0/3575756846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6f8101c90 0x7fe6f819ca90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009230 tx=0x7fe6ec005d50 comp rx=0 tx=0).stop 2024-04-02T13:52:51.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.503+0000 7fe6fcaba640 1 -- 172.21.15.52:0/3575756846 shutdown_connections 2024-04-02T13:52:51.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.503+0000 7fe6fcaba640 1 --2- 172.21.15.52:0/3575756846 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe6e00a3cc0 0x7fe6e00a6180 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.503+0000 7fe6fcaba640 1 --2- 172.21.15.52:0/3575756846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6f8102650 0x7fe6f819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.503+0000 7fe6fcaba640 1 --2- 172.21.15.52:0/3575756846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6f8101c90 0x7fe6f819ca90 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.503+0000 7fe6fcaba640 1 --2- 172.21.15.52:0/3575756846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6f8100aa0 0x7fe6f806ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.503+0000 7fe6fcaba640 1 -- 172.21.15.52:0/3575756846 >> 172.21.15.52:0/3575756846 conn(0x7fe6f80fc010 msgr2=0x7fe6f8104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:51.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.503+0000 7fe6fcaba640 1 -- 172.21.15.52:0/3575756846 shutdown_connections 2024-04-02T13:52:51.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.503+0000 7fe6fcaba640 1 -- 172.21.15.52:0/3575756846 wait complete. 2024-04-02T13:52:51.518 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:52:51.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.620+0000 7f0218909640 1 -- 172.21.15.52:0/1355323042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0214101c10 msgr2=0x7f0214102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:51.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.620+0000 7f0218909640 1 --2- 172.21.15.52:0/1355323042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0214101c10 0x7f0214102090 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f01fc009900 tx=0x7f01fc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:52:51.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.620+0000 7f0218909640 1 -- 172.21.15.52:0/1355323042 shutdown_connections 2024-04-02T13:52:51.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.620+0000 7f0218909640 1 --2- 172.21.15.52:0/1355323042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02141025d0 0x7f0214108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.620+0000 7f0218909640 1 --2- 172.21.15.52:0/1355323042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0214101c10 0x7f0214102090 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.620+0000 7f0218909640 1 --2- 172.21.15.52:0/1355323042 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0214100a20 0x7f0214100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.620+0000 7f0218909640 1 -- 172.21.15.52:0/1355323042 >> 172.21.15.52:0/1355323042 conn(0x7f02140fbfb0 msgr2=0x7f02140fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:51.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.620+0000 7f0218909640 1 -- 172.21.15.52:0/1355323042 shutdown_connections 2024-04-02T13:52:51.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.620+0000 7f0218909640 1 -- 172.21.15.52:0/1355323042 wait complete. 2024-04-02T13:52:51.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.621+0000 7f0218909640 1 Processor -- start 2024-04-02T13:52:51.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.621+0000 7f0218909640 1 -- start start 2024-04-02T13:52:51.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.621+0000 7f0218909640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0214100a20 0x7f021406ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:51.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.621+0000 7f0218909640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0214101c10 0x7f021419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:51.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.621+0000 7f0218909640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02141025d0 0x7f02141a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:51.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.621+0000 7f0218909640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f021410c680 con 0x7f02141025d0 2024-04-02T13:52:51.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.621+0000 7f0218909640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f021410c500 con 0x7f0214101c10 2024-04-02T13:52:51.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.621+0000 7f0218909640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f021410c800 con 0x7f0214100a20 2024-04-02T13:52:51.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.621+0000 7f0212d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02141025d0 0x7f02141a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:51.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.621+0000 7f0212d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02141025d0 0x7f02141a1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48390/0 (socket says 172.21.15.52:48390) 2024-04-02T13:52:51.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.621+0000 7f0212d9d640 1 -- 172.21.15.52:0/2829421180 learned_addr learned my addr 172.21.15.52:0/2829421180 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:51.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.621+0000 7f021259c640 1 --2- 172.21.15.52:0/2829421180 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0214100a20 0x7f021406ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:51.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.621+0000 7f020bfff640 1 --2- 172.21.15.52:0/2829421180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0214101c10 0x7f021419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:51.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.622+0000 7f0212d9d640 1 -- 172.21.15.52:0/2829421180 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0214100a20 msgr2=0x7f021406ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:51.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.622+0000 7f0212d9d640 1 --2- 172.21.15.52:0/2829421180 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0214100a20 0x7f021406ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.622+0000 7f0212d9d640 1 -- 172.21.15.52:0/2829421180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0214101c10 msgr2=0x7f021419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:51.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.622+0000 7f0212d9d640 1 --2- 172.21.15.52:0/2829421180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0214101c10 0x7f021419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.622+0000 7f0212d9d640 1 -- 172.21.15.52:0/2829421180 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02141a1c60 con 0x7f02141025d0 2024-04-02T13:52:51.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.622+0000 7f020bfff640 1 --2- 172.21.15.52:0/2829421180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0214101c10 0x7f021419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:52:51.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.622+0000 7f0212d9d640 1 --2- 172.21.15.52:0/2829421180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02141025d0 0x7f02141a1470 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f020c00cce0 tx=0x7f020c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:51.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.622+0000 7f020b7fe640 1 -- 172.21.15.52:0/2829421180 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c00f450 con 0x7f02141025d0 2024-04-02T13:52:51.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.622+0000 7f020b7fe640 1 -- 172.21.15.52:0/2829421180 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f020c00fad0 con 0x7f02141025d0 2024-04-02T13:52:51.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.622+0000 7f0218909640 1 -- 172.21.15.52:0/2829421180 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02141a1ef0 con 0x7f02141025d0 2024-04-02T13:52:51.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.622+0000 7f020b7fe640 1 -- 172.21.15.52:0/2829421180 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c018c10 con 0x7f02141025d0 2024-04-02T13:52:51.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.622+0000 7f0218909640 1 -- 172.21.15.52:0/2829421180 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02141a2360 con 0x7f02141025d0 2024-04-02T13:52:51.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.623+0000 7f0218909640 1 -- 172.21.15.52:0/2829421180 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0214107d40 con 0x7f02141025d0 2024-04-02T13:52:51.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.624+0000 7f020b7fe640 1 -- 172.21.15.52:0/2829421180 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f020c00f5f0 con 0x7f02141025d0 2024-04-02T13:52:51.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.625+0000 7f020b7fe640 1 --2- 172.21.15.52:0/2829421180 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f02000a3c70 0x7f02000a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:51.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.625+0000 7f021259c640 1 --2- 172.21.15.52:0/2829421180 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f02000a3c70 0x7f02000a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:51.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.625+0000 7f020b7fe640 1 -- 172.21.15.52:0/2829421180 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f020c0d1070 con 0x7f02141025d0 2024-04-02T13:52:51.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.625+0000 7f021259c640 1 --2- 172.21.15.52:0/2829421180 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f02000a3c70 0x7f02000a6130 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0204005fd0 tx=0x7f0204005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:51.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.629+0000 7f020b7fe640 1 -- 172.21.15.52:0/2829421180 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f020c014030 con 0x7f02141025d0 2024-04-02T13:52:51.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:51 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3453826422' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:52:51.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:51 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1359274606' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:52:51.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:51 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1873656955' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:52:51.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.815+0000 7f0218909640 1 -- 172.21.15.52:0/2829421180 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f021402d540 con 0x7f02141025d0 2024-04-02T13:52:51.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.815+0000 7f020b7fe640 1 -- 172.21.15.52:0/2829421180 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7f020c09d530 con 0x7f02141025d0 2024-04-02T13:52:51.818 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-04-02T13:52:51.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.817+0000 7f0218909640 1 -- 172.21.15.52:0/2829421180 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f02000a3c70 msgr2=0x7f02000a6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:51.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.817+0000 7f0218909640 1 --2- 172.21.15.52:0/2829421180 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f02000a3c70 0x7f02000a6130 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0204005fd0 tx=0x7f0204005850 comp rx=0 tx=0).stop 2024-04-02T13:52:51.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.817+0000 7f0218909640 1 -- 172.21.15.52:0/2829421180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02141025d0 msgr2=0x7f02141a1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:51.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.817+0000 7f0218909640 1 --2- 172.21.15.52:0/2829421180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02141025d0 0x7f02141a1470 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f020c00cce0 tx=0x7f020c00a620 comp rx=0 tx=0).stop 2024-04-02T13:52:51.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.817+0000 7f0218909640 1 -- 172.21.15.52:0/2829421180 shutdown_connections 2024-04-02T13:52:51.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.817+0000 7f0218909640 1 --2- 172.21.15.52:0/2829421180 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f02000a3c70 0x7f02000a6130 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.817+0000 7f0218909640 1 --2- 172.21.15.52:0/2829421180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02141025d0 0x7f02141a1470 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.817+0000 7f0218909640 1 --2- 172.21.15.52:0/2829421180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0214101c10 0x7f021419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.817+0000 7f0218909640 1 --2- 172.21.15.52:0/2829421180 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0214100a20 0x7f021406ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.817+0000 7f0218909640 1 -- 172.21.15.52:0/2829421180 >> 172.21.15.52:0/2829421180 conn(0x7f02140fbfb0 msgr2=0x7f0214104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:51.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.817+0000 7f0218909640 1 -- 172.21.15.52:0/2829421180 shutdown_connections 2024-04-02T13:52:51.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.817+0000 7f0218909640 1 -- 172.21.15.52:0/2829421180 wait complete. 2024-04-02T13:52:51.830 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:51 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3453826422' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:52:51.831 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:51 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1359274606' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:52:51.831 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:51 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1873656955' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:52:51.832 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:52:51.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:51 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3453826422' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:52:51.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:51 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1359274606' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:52:51.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:51 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1873656955' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:52:51.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.931+0000 7f0bd31cd640 1 -- 172.21.15.52:0/4075602994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bcc102d20 msgr2=0x7f0bcc1031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:51.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.931+0000 7f0bd31cd640 1 --2- 172.21.15.52:0/4075602994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bcc102d20 0x7f0bcc1031a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009900 tx=0x7f0bbc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:52:51.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.931+0000 7f0bd31cd640 1 -- 172.21.15.52:0/4075602994 shutdown_connections 2024-04-02T13:52:51.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.931+0000 7f0bd31cd640 1 --2- 172.21.15.52:0/4075602994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bcc1036e0 0x7f0bcc109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.931+0000 7f0bd31cd640 1 --2- 172.21.15.52:0/4075602994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bcc102d20 0x7f0bcc1031a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.931+0000 7f0bd31cd640 1 --2- 172.21.15.52:0/4075602994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bcc101b30 0x7f0bcc101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.931+0000 7f0bd31cd640 1 -- 172.21.15.52:0/4075602994 >> 172.21.15.52:0/4075602994 conn(0x7f0bcc0fd0a0 msgr2=0x7f0bcc0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:51.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.931+0000 7f0bd31cd640 1 -- 172.21.15.52:0/4075602994 shutdown_connections 2024-04-02T13:52:51.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.931+0000 7f0bd31cd640 1 -- 172.21.15.52:0/4075602994 wait complete. 2024-04-02T13:52:51.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.932+0000 7f0bd31cd640 1 Processor -- start 2024-04-02T13:52:51.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.932+0000 7f0bd31cd640 1 -- start start 2024-04-02T13:52:51.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.932+0000 7f0bd31cd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bcc101b30 0x7f0bcc0758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:51.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.932+0000 7f0bd31cd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bcc102d20 0x7f0bcc073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:51.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.932+0000 7f0bd31cd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bcc1036e0 0x7f0bcc074430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:51.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.932+0000 7f0bd31cd640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc10d790 con 0x7f0bcc101b30 2024-04-02T13:52:51.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.932+0000 7f0bd31cd640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc10d610 con 0x7f0bcc1036e0 2024-04-02T13:52:51.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.932+0000 7f0bd31cd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc10d910 con 0x7f0bcc102d20 2024-04-02T13:52:51.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.932+0000 7f0bd0f69640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bcc101b30 0x7f0bcc0758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:51.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.932+0000 7f0bd0f69640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bcc101b30 0x7f0bcc0758b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48414/0 (socket says 172.21.15.52:48414) 2024-04-02T13:52:51.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.932+0000 7f0bd0f69640 1 -- 172.21.15.52:0/854678705 learned_addr learned my addr 172.21.15.52:0/854678705 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:51.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.932+0000 7f0bd176a640 1 --2- 172.21.15.52:0/854678705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bcc1036e0 0x7f0bcc074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:51.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.932+0000 7f0bc3fff640 1 --2- 172.21.15.52:0/854678705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bcc102d20 0x7f0bcc073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:51.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.933+0000 7f0bd0f69640 1 -- 172.21.15.52:0/854678705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bcc102d20 msgr2=0x7f0bcc073ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:51.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.933+0000 7f0bd0f69640 1 --2- 172.21.15.52:0/854678705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bcc102d20 0x7f0bcc073ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.933+0000 7f0bd0f69640 1 -- 172.21.15.52:0/854678705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bcc1036e0 msgr2=0x7f0bcc074430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:51.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.933+0000 7f0bd0f69640 1 --2- 172.21.15.52:0/854678705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bcc1036e0 0x7f0bcc074430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:51.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.933+0000 7f0bd0f69640 1 -- 172.21.15.52:0/854678705 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bcc074cc0 con 0x7f0bcc101b30 2024-04-02T13:52:51.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.933+0000 7f0bd0f69640 1 --2- 172.21.15.52:0/854678705 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bcc101b30 0x7f0bcc0758b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800d950 tx=0x7f0bc800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:51.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.933+0000 7f0bc1ffb640 1 -- 172.21.15.52:0/854678705 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8017020 con 0x7f0bcc101b30 2024-04-02T13:52:51.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.933+0000 7f0bc1ffb640 1 -- 172.21.15.52:0/854678705 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc8009c30 con 0x7f0bcc101b30 2024-04-02T13:52:51.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.933+0000 7f0bd31cd640 1 -- 172.21.15.52:0/854678705 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bcc1a1f50 con 0x7f0bcc101b30 2024-04-02T13:52:51.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.933+0000 7f0bc1ffb640 1 -- 172.21.15.52:0/854678705 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc800f740 con 0x7f0bcc101b30 2024-04-02T13:52:51.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.933+0000 7f0bd31cd640 1 -- 172.21.15.52:0/854678705 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bcc1a23b0 con 0x7f0bcc101b30 2024-04-02T13:52:51.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.934+0000 7f0bd31cd640 1 -- 172.21.15.52:0/854678705 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bcc108ed0 con 0x7f0bcc101b30 2024-04-02T13:52:51.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.935+0000 7f0bc1ffb640 1 -- 172.21.15.52:0/854678705 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0bc801a020 con 0x7f0bcc101b30 2024-04-02T13:52:51.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.936+0000 7f0bc1ffb640 1 --2- 172.21.15.52:0/854678705 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f0bb40a3cc0 0x7f0bb40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:51.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.936+0000 7f0bc3fff640 1 --2- 172.21.15.52:0/854678705 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f0bb40a3cc0 0x7f0bb40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:51.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.936+0000 7f0bc1ffb640 1 -- 172.21.15.52:0/854678705 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f0bc80d0240 con 0x7f0bcc101b30 2024-04-02T13:52:51.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.936+0000 7f0bc3fff640 1 --2- 172.21.15.52:0/854678705 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f0bb40a3cc0 0x7f0bb40a6180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc006010 tx=0x7f0bbc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:51.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:51.940+0000 7f0bc1ffb640 1 -- 172.21.15.52:0/854678705 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0bc8021070 con 0x7f0bcc101b30 2024-04-02T13:52:52.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.167+0000 7f0bd31cd640 1 -- 172.21.15.52:0/854678705 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0bcc02d370 con 0x7f0bcc101b30 2024-04-02T13:52:52.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.168+0000 7f0bc1ffb640 1 -- 172.21.15.52:0/854678705 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f0bc8009de0 con 0x7f0bcc101b30 2024-04-02T13:52:52.171 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:52:52.171 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:51:29.345060+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24503,24475,24490,24467,24457]},"id":1}]} 2024-04-02T13:52:52.172 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 27 2024-04-02T13:52:52.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.170+0000 7f0bd31cd640 1 -- 172.21.15.52:0/854678705 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f0bb40a3cc0 msgr2=0x7f0bb40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:52.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.170+0000 7f0bd31cd640 1 --2- 172.21.15.52:0/854678705 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f0bb40a3cc0 0x7f0bb40a6180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc006010 tx=0x7f0bbc000bc0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.170+0000 7f0bd31cd640 1 -- 172.21.15.52:0/854678705 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bcc101b30 msgr2=0x7f0bcc0758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:52.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.170+0000 7f0bd31cd640 1 --2- 172.21.15.52:0/854678705 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bcc101b30 0x7f0bcc0758b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0bc800d950 tx=0x7f0bc800de10 comp rx=0 tx=0).stop 2024-04-02T13:52:52.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.170+0000 7f0bd31cd640 1 -- 172.21.15.52:0/854678705 shutdown_connections 2024-04-02T13:52:52.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.170+0000 7f0bd31cd640 1 --2- 172.21.15.52:0/854678705 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f0bb40a3cc0 0x7f0bb40a6180 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.170+0000 7f0bd31cd640 1 --2- 172.21.15.52:0/854678705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bcc1036e0 0x7f0bcc074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.170+0000 7f0bd31cd640 1 --2- 172.21.15.52:0/854678705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0bcc102d20 0x7f0bcc073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.170+0000 7f0bd31cd640 1 --2- 172.21.15.52:0/854678705 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bcc101b30 0x7f0bcc0758b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.170+0000 7f0bd31cd640 1 -- 172.21.15.52:0/854678705 >> 172.21.15.52:0/854678705 conn(0x7f0bcc0fd0a0 msgr2=0x7f0bcc1060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:52.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.170+0000 7f0bd31cd640 1 -- 172.21.15.52:0/854678705 shutdown_connections 2024-04-02T13:52:52.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.170+0000 7f0bd31cd640 1 -- 172.21.15.52:0/854678705 wait complete. 2024-04-02T13:52:52.185 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:52:52.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.286+0000 7f199738f640 1 -- 172.21.15.52:0/217039615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1990101c10 msgr2=0x7f1990102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:52.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.286+0000 7f199738f640 1 --2- 172.21.15.52:0/217039615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1990101c10 0x7f1990102090 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1980009900 tx=0x7f198002bc30 comp rx=0 tx=0).stop 2024-04-02T13:52:52.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.286+0000 7f199738f640 1 -- 172.21.15.52:0/217039615 shutdown_connections 2024-04-02T13:52:52.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.286+0000 7f199738f640 1 --2- 172.21.15.52:0/217039615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19901025d0 0x7f1990108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.286+0000 7f199738f640 1 --2- 172.21.15.52:0/217039615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1990101c10 0x7f1990102090 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.286+0000 7f199738f640 1 --2- 172.21.15.52:0/217039615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1990100a20 0x7f1990100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.286+0000 7f199738f640 1 -- 172.21.15.52:0/217039615 >> 172.21.15.52:0/217039615 conn(0x7f19900fbf90 msgr2=0x7f19900fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:52.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.286+0000 7f199738f640 1 -- 172.21.15.52:0/217039615 shutdown_connections 2024-04-02T13:52:52.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.286+0000 7f199738f640 1 -- 172.21.15.52:0/217039615 wait complete. 2024-04-02T13:52:52.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.287+0000 7f199738f640 1 Processor -- start 2024-04-02T13:52:52.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.287+0000 7f199738f640 1 -- start start 2024-04-02T13:52:52.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.287+0000 7f199738f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1990100a20 0x7f1990196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:52.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.287+0000 7f199738f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1990101c10 0x7f1990197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:52.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.287+0000 7f199738f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19901025d0 0x7f199019e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:52.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.287+0000 7f199738f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f199010c6a0 con 0x7f19901025d0 2024-04-02T13:52:52.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.287+0000 7f199738f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f199010c520 con 0x7f1990100a20 2024-04-02T13:52:52.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.287+0000 7f199738f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f199010c820 con 0x7f1990101c10 2024-04-02T13:52:52.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.287+0000 7f199592c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19901025d0 0x7f199019e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:52.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.287+0000 7f199592c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19901025d0 0x7f199019e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48420/0 (socket says 172.21.15.52:48420) 2024-04-02T13:52:52.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.287+0000 7f199592c640 1 -- 172.21.15.52:0/2505204439 learned_addr learned my addr 172.21.15.52:0/2505204439 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:52.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.287+0000 7f199512b640 1 --2- 172.21.15.52:0/2505204439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1990100a20 0x7f1990196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:52.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.287+0000 7f199492a640 1 --2- 172.21.15.52:0/2505204439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1990101c10 0x7f1990197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:52.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.287+0000 7f199592c640 1 -- 172.21.15.52:0/2505204439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1990101c10 msgr2=0x7f1990197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:52.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.287+0000 7f199592c640 1 --2- 172.21.15.52:0/2505204439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1990101c10 0x7f1990197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.287+0000 7f199592c640 1 -- 172.21.15.52:0/2505204439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1990100a20 msgr2=0x7f1990196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:52.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.287+0000 7f199592c640 1 --2- 172.21.15.52:0/2505204439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1990100a20 0x7f1990196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.287+0000 7f199592c640 1 -- 172.21.15.52:0/2505204439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199019e970 con 0x7f19901025d0 2024-04-02T13:52:52.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.288+0000 7f199592c640 1 --2- 172.21.15.52:0/2505204439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19901025d0 0x7f199019e300 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f198800eaa0 tx=0x7f198800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:52.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.288+0000 7f19867fc640 1 -- 172.21.15.52:0/2505204439 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198800cdb0 con 0x7f19901025d0 2024-04-02T13:52:52.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.288+0000 7f19867fc640 1 -- 172.21.15.52:0/2505204439 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1988005020 con 0x7f19901025d0 2024-04-02T13:52:52.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.288+0000 7f199738f640 1 -- 172.21.15.52:0/2505204439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f199019ec60 con 0x7f19901025d0 2024-04-02T13:52:52.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.288+0000 7f19867fc640 1 -- 172.21.15.52:0/2505204439 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19880116b0 con 0x7f19901025d0 2024-04-02T13:52:52.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.288+0000 7f199738f640 1 -- 172.21.15.52:0/2505204439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f199019f1d0 con 0x7f19901025d0 2024-04-02T13:52:52.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.289+0000 7f199738f640 1 -- 172.21.15.52:0/2505204439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1990107ce0 con 0x7f19901025d0 2024-04-02T13:52:52.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.290+0000 7f19867fc640 1 -- 172.21.15.52:0/2505204439 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1988005560 con 0x7f19901025d0 2024-04-02T13:52:52.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.290+0000 7f19867fc640 1 --2- 172.21.15.52:0/2505204439 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f197c0a3cc0 0x7f197c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:52.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.290+0000 7f199512b640 1 --2- 172.21.15.52:0/2505204439 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f197c0a3cc0 0x7f197c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:52.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.290+0000 7f19867fc640 1 -- 172.21.15.52:0/2505204439 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f19880d0950 con 0x7f19901025d0 2024-04-02T13:52:52.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.291+0000 7f199512b640 1 --2- 172.21.15.52:0/2505204439 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f197c0a3cc0 0x7f197c0a6180 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f198c005fd0 tx=0x7f198c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:52.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.295+0000 7f19867fc640 1 -- 172.21.15.52:0/2505204439 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1988016030 con 0x7f19901025d0 2024-04-02T13:52:52.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.482+0000 7f199738f640 1 -- 172.21.15.52:0/2505204439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f199002d5b0 con 0x7f19901025d0 2024-04-02T13:52:52.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.483+0000 7f19867fc640 1 -- 172.21.15.52:0/2505204439 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24549 (secure 0 0 0) 0x7f198809c820 con 0x7f19901025d0 2024-04-02T13:52:52.487 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:52:52.487 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":92,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T13:51:56.472064+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:50:58.778690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:50:59.561527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:51:03.343487+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:52:52.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.486+0000 7f199738f640 1 -- 172.21.15.52:0/2505204439 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f197c0a3cc0 msgr2=0x7f197c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:52.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.486+0000 7f199738f640 1 --2- 172.21.15.52:0/2505204439 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f197c0a3cc0 0x7f197c0a6180 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f198c005fd0 tx=0x7f198c00b410 comp rx=0 tx=0).stop 2024-04-02T13:52:52.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.486+0000 7f199738f640 1 -- 172.21.15.52:0/2505204439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19901025d0 msgr2=0x7f199019e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:52.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.486+0000 7f199738f640 1 --2- 172.21.15.52:0/2505204439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19901025d0 0x7f199019e300 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f198800eaa0 tx=0x7f198800ef60 comp rx=0 tx=0).stop 2024-04-02T13:52:52.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.486+0000 7f199738f640 1 -- 172.21.15.52:0/2505204439 shutdown_connections 2024-04-02T13:52:52.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.486+0000 7f199738f640 1 --2- 172.21.15.52:0/2505204439 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f197c0a3cc0 0x7f197c0a6180 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.486+0000 7f199738f640 1 --2- 172.21.15.52:0/2505204439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19901025d0 0x7f199019e300 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.486+0000 7f199738f640 1 --2- 172.21.15.52:0/2505204439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1990101c10 0x7f1990197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.486+0000 7f199738f640 1 --2- 172.21.15.52:0/2505204439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1990100a20 0x7f1990196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.486+0000 7f199738f640 1 -- 172.21.15.52:0/2505204439 >> 172.21.15.52:0/2505204439 conn(0x7f19900fbf90 msgr2=0x7f1990104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:52.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.486+0000 7f199738f640 1 -- 172.21.15.52:0/2505204439 shutdown_connections 2024-04-02T13:52:52.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.486+0000 7f199738f640 1 -- 172.21.15.52:0/2505204439 wait complete. 2024-04-02T13:52:52.500 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:46:20.746053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:50:58.778690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:50:59.561527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:51:03.343487+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:52:52.501 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:52:52.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.603+0000 7fdf5823c640 1 -- 172.21.15.52:0/439952701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf500ffe00 msgr2=0x7fdf50100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:52.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.603+0000 7fdf5823c640 1 --2- 172.21.15.52:0/439952701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf500ffe00 0x7fdf50100220 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009930 tx=0x7fdf4c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:52:52.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.603+0000 7fdf5823c640 1 -- 172.21.15.52:0/439952701 shutdown_connections 2024-04-02T13:52:52.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.603+0000 7fdf5823c640 1 --2- 172.21.15.52:0/439952701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf50103790 0x7fdf5010a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.603+0000 7fdf5823c640 1 --2- 172.21.15.52:0/439952701 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf50102dd0 0x7fdf50103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.603+0000 7fdf5823c640 1 --2- 172.21.15.52:0/439952701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf500ffe00 0x7fdf50100220 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.603+0000 7fdf5823c640 1 -- 172.21.15.52:0/439952701 >> 172.21.15.52:0/439952701 conn(0x7fdf500fb730 msgr2=0x7fdf500fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:52.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.604+0000 7fdf5823c640 1 -- 172.21.15.52:0/439952701 shutdown_connections 2024-04-02T13:52:52.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.604+0000 7fdf5823c640 1 -- 172.21.15.52:0/439952701 wait complete. 2024-04-02T13:52:52.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.604+0000 7fdf5823c640 1 Processor -- start 2024-04-02T13:52:52.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.604+0000 7fdf5823c640 1 -- start start 2024-04-02T13:52:52.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.604+0000 7fdf5823c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf500ffe00 0x7fdf50197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:52.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.604+0000 7fdf5823c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf50102dd0 0x7fdf501983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:52.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.604+0000 7fdf5823c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf50103790 0x7fdf5019f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:52.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.604+0000 7fdf5823c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5010d7d0 con 0x7fdf50102dd0 2024-04-02T13:52:52.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.604+0000 7fdf5823c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5010d650 con 0x7fdf50103790 2024-04-02T13:52:52.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.604+0000 7fdf5823c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5010d950 con 0x7fdf500ffe00 2024-04-02T13:52:52.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.604+0000 7fdf557d7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf50102dd0 0x7fdf501983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:52.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.604+0000 7fdf557d7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf50102dd0 0x7fdf501983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48446/0 (socket says 172.21.15.52:48446) 2024-04-02T13:52:52.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.604+0000 7fdf557d7640 1 -- 172.21.15.52:0/1666704427 learned_addr learned my addr 172.21.15.52:0/1666704427 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:52.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.604+0000 7fdf55fd8640 1 --2- 172.21.15.52:0/1666704427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf500ffe00 0x7fdf50197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:52.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.605+0000 7fdf567d9640 1 --2- 172.21.15.52:0/1666704427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf50103790 0x7fdf5019f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:52.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.605+0000 7fdf557d7640 1 -- 172.21.15.52:0/1666704427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf500ffe00 msgr2=0x7fdf50197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:52.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.605+0000 7fdf557d7640 1 --2- 172.21.15.52:0/1666704427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf500ffe00 0x7fdf50197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.605+0000 7fdf557d7640 1 -- 172.21.15.52:0/1666704427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf50103790 msgr2=0x7fdf5019f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:52.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.605+0000 7fdf557d7640 1 --2- 172.21.15.52:0/1666704427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf50103790 0x7fdf5019f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.605+0000 7fdf557d7640 1 -- 172.21.15.52:0/1666704427 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5019fa90 con 0x7fdf50102dd0 2024-04-02T13:52:52.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.605+0000 7fdf55fd8640 1 --2- 172.21.15.52:0/1666704427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf500ffe00 0x7fdf50197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:52:52.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.605+0000 7fdf557d7640 1 --2- 172.21.15.52:0/1666704427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf50102dd0 0x7fdf501983b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000e9f0 tx=0x7fdf4000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:52.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.605+0000 7fdf46ffd640 1 -- 172.21.15.52:0/1666704427 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4000cd80 con 0x7fdf50102dd0 2024-04-02T13:52:52.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.605+0000 7fdf46ffd640 1 -- 172.21.15.52:0/1666704427 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf40010910 con 0x7fdf50102dd0 2024-04-02T13:52:52.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.605+0000 7fdf5823c640 1 -- 172.21.15.52:0/1666704427 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5019fd80 con 0x7fdf50102dd0 2024-04-02T13:52:52.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.605+0000 7fdf46ffd640 1 -- 172.21.15.52:0/1666704427 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf40018990 con 0x7fdf50102dd0 2024-04-02T13:52:52.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.605+0000 7fdf5823c640 1 -- 172.21.15.52:0/1666704427 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf501a0270 con 0x7fdf50102dd0 2024-04-02T13:52:52.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.606+0000 7fdf5823c640 1 -- 172.21.15.52:0/1666704427 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf50108f80 con 0x7fdf50102dd0 2024-04-02T13:52:52.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.607+0000 7fdf46ffd640 1 -- 172.21.15.52:0/1666704427 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdf40010ac0 con 0x7fdf50102dd0 2024-04-02T13:52:52.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.608+0000 7fdf46ffd640 1 --2- 172.21.15.52:0/1666704427 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdf3c0a3c40 0x7fdf3c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:52.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.608+0000 7fdf46ffd640 1 -- 172.21.15.52:0/1666704427 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdf400d1130 con 0x7fdf50102dd0 2024-04-02T13:52:52.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.608+0000 7fdf55fd8640 1 --2- 172.21.15.52:0/1666704427 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdf3c0a3c40 0x7fdf3c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:52.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.608+0000 7fdf55fd8640 1 --2- 172.21.15.52:0/1666704427 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdf3c0a3c40 0x7fdf3c0a6100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c02c7b0 tx=0x7fdf4c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:52.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.612+0000 7fdf46ffd640 1 -- 172.21.15.52:0/1666704427 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fdf4009d650 con 0x7fdf50102dd0 2024-04-02T13:52:52.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:52 smithi189 ceph-mon[36683]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-02T13:52:52.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:52 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3575756846' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:52:52.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:52 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2829421180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:52:52.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:52 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/854678705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:52:52.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.797+0000 7fdf5823c640 1 -- 172.21.15.52:0/1666704427 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdf5002d5c0 con 0x7fdf50102dd0 2024-04-02T13:52:52.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.797+0000 7fdf46ffd640 1 -- 172.21.15.52:0/1666704427 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fdf4009cf50 con 0x7fdf50102dd0 2024-04-02T13:52:52.800 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-04-02T13:52:52.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.799+0000 7fdf5823c640 1 -- 172.21.15.52:0/1666704427 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdf3c0a3c40 msgr2=0x7fdf3c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:52.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.799+0000 7fdf5823c640 1 --2- 172.21.15.52:0/1666704427 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdf3c0a3c40 0x7fdf3c0a6100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c02c7b0 tx=0x7fdf4c005fd0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.799+0000 7fdf5823c640 1 -- 172.21.15.52:0/1666704427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf50102dd0 msgr2=0x7fdf501983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:52.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.799+0000 7fdf5823c640 1 --2- 172.21.15.52:0/1666704427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf50102dd0 0x7fdf501983b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000e9f0 tx=0x7fdf4000eeb0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.799+0000 7fdf5823c640 1 -- 172.21.15.52:0/1666704427 shutdown_connections 2024-04-02T13:52:52.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.799+0000 7fdf5823c640 1 --2- 172.21.15.52:0/1666704427 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdf3c0a3c40 0x7fdf3c0a6100 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.799+0000 7fdf5823c640 1 --2- 172.21.15.52:0/1666704427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf50103790 0x7fdf5019f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.799+0000 7fdf5823c640 1 --2- 172.21.15.52:0/1666704427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf50102dd0 0x7fdf501983b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.799+0000 7fdf5823c640 1 --2- 172.21.15.52:0/1666704427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf500ffe00 0x7fdf50197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.799+0000 7fdf5823c640 1 -- 172.21.15.52:0/1666704427 >> 172.21.15.52:0/1666704427 conn(0x7fdf500fb730 msgr2=0x7fdf50106150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:52.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.799+0000 7fdf5823c640 1 -- 172.21.15.52:0/1666704427 shutdown_connections 2024-04-02T13:52:52.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.799+0000 7fdf5823c640 1 -- 172.21.15.52:0/1666704427 wait complete. 2024-04-02T13:52:52.812 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:52 smithi052 ceph-mon[32104]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-02T13:52:52.812 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:52 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3575756846' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:52:52.812 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:52 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2829421180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:52:52.813 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:52 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/854678705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:52:52.813 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:52:52.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:52 smithi088 ceph-mon[36528]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-02T13:52:52.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:52 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3575756846' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:52:52.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:52 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2829421180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:52:52.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:52 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/854678705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:52:52.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.916+0000 7f7919f3d640 1 -- 172.21.15.52:0/3518482127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7914103770 msgr2=0x7f791410a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:52.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.916+0000 7f7919f3d640 1 --2- 172.21.15.52:0/3518482127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7914103770 0x7f791410a000 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f790c009cd0 tx=0x7f790c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.916+0000 7f7919f3d640 1 -- 172.21.15.52:0/3518482127 shutdown_connections 2024-04-02T13:52:52.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.916+0000 7f7919f3d640 1 --2- 172.21.15.52:0/3518482127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7914103770 0x7f791410a000 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.916+0000 7f7919f3d640 1 --2- 172.21.15.52:0/3518482127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7914102db0 0x7f7914103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.916+0000 7f7919f3d640 1 --2- 172.21.15.52:0/3518482127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79140ffde0 0x7f7914100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.916+0000 7f7919f3d640 1 -- 172.21.15.52:0/3518482127 >> 172.21.15.52:0/3518482127 conn(0x7f79140fb710 msgr2=0x7f79140fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:52.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.916+0000 7f7919f3d640 1 -- 172.21.15.52:0/3518482127 shutdown_connections 2024-04-02T13:52:52.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.916+0000 7f7919f3d640 1 -- 172.21.15.52:0/3518482127 wait complete. 2024-04-02T13:52:52.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.917+0000 7f7919f3d640 1 Processor -- start 2024-04-02T13:52:52.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.917+0000 7f7919f3d640 1 -- start start 2024-04-02T13:52:52.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.917+0000 7f7919f3d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79140ffde0 0x7f7914102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.917+0000 7f7919f3d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7914102db0 0x7f7914100d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.917+0000 7f7919f3d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7914103770 0x7f79141012a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.917+0000 7f7919f3d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f791410d810 con 0x7f7914102db0 2024-04-02T13:52:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.917+0000 7f7919f3d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f791410d690 con 0x7f7914103770 2024-04-02T13:52:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.917+0000 7f7919f3d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f791410d990 con 0x7f79140ffde0 2024-04-02T13:52:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.917+0000 7f7912ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7914102db0 0x7f7914100d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.917+0000 7f7912ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7914102db0 0x7f7914100d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48472/0 (socket says 172.21.15.52:48472) 2024-04-02T13:52:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.917+0000 7f7912ffd640 1 -- 172.21.15.52:0/3550322188 learned_addr learned my addr 172.21.15.52:0/3550322188 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:52.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.918+0000 7f79137fe640 1 --2- 172.21.15.52:0/3550322188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79140ffde0 0x7f7914102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:52.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.918+0000 7f7913fff640 1 --2- 172.21.15.52:0/3550322188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7914103770 0x7f79141012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:52.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.918+0000 7f7912ffd640 1 -- 172.21.15.52:0/3550322188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79140ffde0 msgr2=0x7f7914102720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:52.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.918+0000 7f7912ffd640 1 --2- 172.21.15.52:0/3550322188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79140ffde0 0x7f7914102720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.918+0000 7f7912ffd640 1 -- 172.21.15.52:0/3550322188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7914103770 msgr2=0x7f79141012a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:52.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.918+0000 7f7912ffd640 1 --2- 172.21.15.52:0/3550322188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7914103770 0x7f79141012a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:52.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.918+0000 7f7912ffd640 1 -- 172.21.15.52:0/3550322188 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7914101b30 con 0x7f7914102db0 2024-04-02T13:52:52.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.918+0000 7f7912ffd640 1 --2- 172.21.15.52:0/3550322188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7914102db0 0x7f7914100d60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f790800b4a0 tx=0x7f790800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:52.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.918+0000 7f7910ff9640 1 -- 172.21.15.52:0/3550322188 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908014070 con 0x7f7914102db0 2024-04-02T13:52:52.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.918+0000 7f7910ff9640 1 -- 172.21.15.52:0/3550322188 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7908010af0 con 0x7f7914102db0 2024-04-02T13:52:52.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.918+0000 7f7919f3d640 1 -- 172.21.15.52:0/3550322188 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79141a1fa0 con 0x7f7914102db0 2024-04-02T13:52:52.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.918+0000 7f7910ff9640 1 -- 172.21.15.52:0/3550322188 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908017d10 con 0x7f7914102db0 2024-04-02T13:52:52.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.918+0000 7f7919f3d640 1 -- 172.21.15.52:0/3550322188 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79141a2400 con 0x7f7914102db0 2024-04-02T13:52:52.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.919+0000 7f7919f3d640 1 -- 172.21.15.52:0/3550322188 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7914108ee0 con 0x7f7914102db0 2024-04-02T13:52:52.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.920+0000 7f7910ff9640 1 -- 172.21.15.52:0/3550322188 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7908017070 con 0x7f7914102db0 2024-04-02T13:52:52.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.921+0000 7f7910ff9640 1 --2- 172.21.15.52:0/3550322188 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f78fc0a3c40 0x7f78fc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:52.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.921+0000 7f79137fe640 1 --2- 172.21.15.52:0/3550322188 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f78fc0a3c40 0x7f78fc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:52.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.921+0000 7f7910ff9640 1 -- 172.21.15.52:0/3550322188 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f79080cfb10 con 0x7f7914102db0 2024-04-02T13:52:52.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.921+0000 7f79137fe640 1 --2- 172.21.15.52:0/3550322188 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f78fc0a3c40 0x7f78fc0a6100 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7914069ef0 tx=0x7f790400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:52.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:52.926+0000 7f7910ff9640 1 -- 172.21.15.52:0/3550322188 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f790809c140 con 0x7f7914102db0 2024-04-02T13:52:53.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.111+0000 7f7919f3d640 1 -- 172.21.15.52:0/3550322188 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f791402d5e0 con 0x7f7914102db0 2024-04-02T13:52:53.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.111+0000 7f7910ff9640 1 -- 172.21.15.52:0/3550322188 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f790809ba40 con 0x7f7914102db0 2024-04-02T13:52:53.115 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-04-02T13:52:53.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.113+0000 7f7919f3d640 1 -- 172.21.15.52:0/3550322188 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f78fc0a3c40 msgr2=0x7f78fc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:53.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.113+0000 7f7919f3d640 1 --2- 172.21.15.52:0/3550322188 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f78fc0a3c40 0x7f78fc0a6100 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7914069ef0 tx=0x7f790400a2d0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.113+0000 7f7919f3d640 1 -- 172.21.15.52:0/3550322188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7914102db0 msgr2=0x7f7914100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:53.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.113+0000 7f7919f3d640 1 --2- 172.21.15.52:0/3550322188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7914102db0 0x7f7914100d60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f790800b4a0 tx=0x7f790800b960 comp rx=0 tx=0).stop 2024-04-02T13:52:53.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.113+0000 7f7919f3d640 1 -- 172.21.15.52:0/3550322188 shutdown_connections 2024-04-02T13:52:53.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.113+0000 7f7919f3d640 1 --2- 172.21.15.52:0/3550322188 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f78fc0a3c40 0x7f78fc0a6100 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.113+0000 7f7919f3d640 1 --2- 172.21.15.52:0/3550322188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7914103770 0x7f79141012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.113+0000 7f7919f3d640 1 --2- 172.21.15.52:0/3550322188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7914102db0 0x7f7914100d60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.113+0000 7f7919f3d640 1 --2- 172.21.15.52:0/3550322188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79140ffde0 0x7f7914102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.113+0000 7f7919f3d640 1 -- 172.21.15.52:0/3550322188 >> 172.21.15.52:0/3550322188 conn(0x7f79140fb710 msgr2=0x7f79141060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:53.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.113+0000 7f7919f3d640 1 -- 172.21.15.52:0/3550322188 shutdown_connections 2024-04-02T13:52:53.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.113+0000 7f7919f3d640 1 -- 172.21.15.52:0/3550322188 wait complete. 2024-04-02T13:52:53.129 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:52:53.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.231+0000 7fc4ad82a640 1 -- 172.21.15.52:0/912566096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4a8102db0 msgr2=0x7fc4a8103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:53.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.231+0000 7fc4ad82a640 1 --2- 172.21.15.52:0/912566096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4a8102db0 0x7fc4a8103230 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009930 tx=0x7fc49c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:52:53.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.231+0000 7fc4ad82a640 1 -- 172.21.15.52:0/912566096 shutdown_connections 2024-04-02T13:52:53.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.231+0000 7fc4ad82a640 1 --2- 172.21.15.52:0/912566096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4a8103770 0x7fc4a810a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.231+0000 7fc4ad82a640 1 --2- 172.21.15.52:0/912566096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4a8102db0 0x7fc4a8103230 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.231+0000 7fc4ad82a640 1 --2- 172.21.15.52:0/912566096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4a80ffde0 0x7fc4a8100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.231+0000 7fc4ad82a640 1 -- 172.21.15.52:0/912566096 >> 172.21.15.52:0/912566096 conn(0x7fc4a80fb710 msgr2=0x7fc4a80fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:53.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.231+0000 7fc4ad82a640 1 -- 172.21.15.52:0/912566096 shutdown_connections 2024-04-02T13:52:53.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.231+0000 7fc4ad82a640 1 -- 172.21.15.52:0/912566096 wait complete. 2024-04-02T13:52:53.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.232+0000 7fc4ad82a640 1 Processor -- start 2024-04-02T13:52:53.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.232+0000 7fc4ad82a640 1 -- start start 2024-04-02T13:52:53.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.232+0000 7fc4ad82a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4a80ffde0 0x7fc4a8197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:53.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.232+0000 7fc4ad82a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4a8102db0 0x7fc4a8198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:53.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.232+0000 7fc4ad82a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4a8103770 0x7fc4a819f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:53.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.232+0000 7fc4ad82a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a810d810 con 0x7fc4a80ffde0 2024-04-02T13:52:53.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.232+0000 7fc4ad82a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a810d690 con 0x7fc4a8102db0 2024-04-02T13:52:53.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.232+0000 7fc4ad82a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a810d990 con 0x7fc4a8103770 2024-04-02T13:52:53.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.232+0000 7fc4a6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4a80ffde0 0x7fc4a8197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:53.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.232+0000 7fc4a6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4a80ffde0 0x7fc4a8197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48496/0 (socket says 172.21.15.52:48496) 2024-04-02T13:52:53.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.232+0000 7fc4a6ffd640 1 -- 172.21.15.52:0/4145545070 learned_addr learned my addr 172.21.15.52:0/4145545070 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:53.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.232+0000 7fc4a77fe640 1 --2- 172.21.15.52:0/4145545070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4a8103770 0x7fc4a819f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:53.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.232+0000 7fc4a67fc640 1 --2- 172.21.15.52:0/4145545070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4a8102db0 0x7fc4a8198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:53.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.232+0000 7fc4a6ffd640 1 -- 172.21.15.52:0/4145545070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4a8103770 msgr2=0x7fc4a819f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:53.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.232+0000 7fc4a6ffd640 1 --2- 172.21.15.52:0/4145545070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4a8103770 0x7fc4a819f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.232+0000 7fc4a6ffd640 1 -- 172.21.15.52:0/4145545070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4a8102db0 msgr2=0x7fc4a8198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:53.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.232+0000 7fc4a6ffd640 1 --2- 172.21.15.52:0/4145545070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4a8102db0 0x7fc4a8198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.233+0000 7fc4a6ffd640 1 -- 172.21.15.52:0/4145545070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a819f9f0 con 0x7fc4a80ffde0 2024-04-02T13:52:53.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.233+0000 7fc4a77fe640 1 --2- 172.21.15.52:0/4145545070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4a8103770 0x7fc4a819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:52:53.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.233+0000 7fc4a6ffd640 1 --2- 172.21.15.52:0/4145545070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4a80ffde0 0x7fc4a8197dd0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc49800c970 tx=0x7fc49800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:53.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.233+0000 7fc4ac828640 1 -- 172.21.15.52:0/4145545070 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498007c10 con 0x7fc4a80ffde0 2024-04-02T13:52:53.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.233+0000 7fc4ac828640 1 -- 172.21.15.52:0/4145545070 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc498007db0 con 0x7fc4a80ffde0 2024-04-02T13:52:53.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.233+0000 7fc4ac828640 1 -- 172.21.15.52:0/4145545070 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49800f940 con 0x7fc4a80ffde0 2024-04-02T13:52:53.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.233+0000 7fc4ad82a640 1 -- 172.21.15.52:0/4145545070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a819fce0 con 0x7fc4a80ffde0 2024-04-02T13:52:53.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.233+0000 7fc4ad82a640 1 -- 172.21.15.52:0/4145545070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a81a0180 con 0x7fc4a80ffde0 2024-04-02T13:52:53.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.234+0000 7fc4ad82a640 1 -- 172.21.15.52:0/4145545070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4a8108e80 con 0x7fc4a80ffde0 2024-04-02T13:52:53.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.235+0000 7fc4ac828640 1 -- 172.21.15.52:0/4145545070 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc49800fae0 con 0x7fc4a80ffde0 2024-04-02T13:52:53.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.235+0000 7fc4ac828640 1 --2- 172.21.15.52:0/4145545070 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc4900a3c40 0x7fc4900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:53.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.236+0000 7fc4ac828640 1 -- 172.21.15.52:0/4145545070 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc4980d0d50 con 0x7fc4a80ffde0 2024-04-02T13:52:53.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.236+0000 7fc4a67fc640 1 --2- 172.21.15.52:0/4145545070 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc4900a3c40 0x7fc4900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:53.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.236+0000 7fc4a67fc640 1 --2- 172.21.15.52:0/4145545070 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc4900a3c40 0x7fc4900a6100 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0096c0 tx=0x7fc49c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:53.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.240+0000 7fc4ac828640 1 -- 172.21.15.52:0/4145545070 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc49809d300 con 0x7fc4a80ffde0 2024-04-02T13:52:53.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.428+0000 7fc4ad82a640 1 -- 172.21.15.52:0/4145545070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc4a802d370 con 0x7fc4a80ffde0 2024-04-02T13:52:53.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.429+0000 7fc4ac828640 1 -- 172.21.15.52:0/4145545070 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7fc4a802d370 con 0x7fc4a80ffde0 2024-04-02T13:52:53.432 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-04-02T13:52:53.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.431+0000 7fc4ad82a640 1 -- 172.21.15.52:0/4145545070 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc4900a3c40 msgr2=0x7fc4900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:53.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.431+0000 7fc4ad82a640 1 --2- 172.21.15.52:0/4145545070 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc4900a3c40 0x7fc4900a6100 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0096c0 tx=0x7fc49c02df90 comp rx=0 tx=0).stop 2024-04-02T13:52:53.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.431+0000 7fc4ad82a640 1 -- 172.21.15.52:0/4145545070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4a80ffde0 msgr2=0x7fc4a8197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:53.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.431+0000 7fc4ad82a640 1 --2- 172.21.15.52:0/4145545070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4a80ffde0 0x7fc4a8197dd0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc49800c970 tx=0x7fc49800ce30 comp rx=0 tx=0).stop 2024-04-02T13:52:53.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.431+0000 7fc4ad82a640 1 -- 172.21.15.52:0/4145545070 shutdown_connections 2024-04-02T13:52:53.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.431+0000 7fc4ad82a640 1 --2- 172.21.15.52:0/4145545070 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fc4900a3c40 0x7fc4900a6100 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.431+0000 7fc4ad82a640 1 --2- 172.21.15.52:0/4145545070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4a8103770 0x7fc4a819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.431+0000 7fc4ad82a640 1 --2- 172.21.15.52:0/4145545070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4a8102db0 0x7fc4a8198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.431+0000 7fc4ad82a640 1 --2- 172.21.15.52:0/4145545070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4a80ffde0 0x7fc4a8197dd0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.431+0000 7fc4ad82a640 1 -- 172.21.15.52:0/4145545070 >> 172.21.15.52:0/4145545070 conn(0x7fc4a80fb710 msgr2=0x7fc4a8106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:53.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.431+0000 7fc4ad82a640 1 -- 172.21.15.52:0/4145545070 shutdown_connections 2024-04-02T13:52:53.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.431+0000 7fc4ad82a640 1 -- 172.21.15.52:0/4145545070 wait complete. 2024-04-02T13:52:53.445 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:52:53.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.548+0000 7fe4e7f38640 1 -- 172.21.15.52:0/4188363789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4e0101bf0 msgr2=0x7fe4e0102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:53.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.548+0000 7fe4e7f38640 1 --2- 172.21.15.52:0/4188363789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4e0101bf0 0x7fe4e0102070 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009900 tx=0x7fe4d002bc30 comp rx=0 tx=0).stop 2024-04-02T13:52:53.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.549+0000 7fe4e7f38640 1 -- 172.21.15.52:0/4188363789 shutdown_connections 2024-04-02T13:52:53.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.549+0000 7fe4e7f38640 1 --2- 172.21.15.52:0/4188363789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4e01025b0 0x7fe4e0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.549+0000 7fe4e7f38640 1 --2- 172.21.15.52:0/4188363789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4e0101bf0 0x7fe4e0102070 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.549+0000 7fe4e7f38640 1 --2- 172.21.15.52:0/4188363789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4e0100a00 0x7fe4e0100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.549+0000 7fe4e7f38640 1 -- 172.21.15.52:0/4188363789 >> 172.21.15.52:0/4188363789 conn(0x7fe4e00fbf90 msgr2=0x7fe4e00fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:53.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.549+0000 7fe4e7f38640 1 -- 172.21.15.52:0/4188363789 shutdown_connections 2024-04-02T13:52:53.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.549+0000 7fe4e7f38640 1 -- 172.21.15.52:0/4188363789 wait complete. 2024-04-02T13:52:53.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.549+0000 7fe4e7f38640 1 Processor -- start 2024-04-02T13:52:53.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.549+0000 7fe4e7f38640 1 -- start start 2024-04-02T13:52:53.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.549+0000 7fe4e7f38640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4e0100a00 0x7fe4e00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:53.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.549+0000 7fe4e7f38640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4e0101bf0 0x7fe4e019cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:53.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.549+0000 7fe4e7f38640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4e01025b0 0x7fe4e019f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:53.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.549+0000 7fe4e7f38640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e010c680 con 0x7fe4e0100a00 2024-04-02T13:52:53.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.549+0000 7fe4e7f38640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e010c500 con 0x7fe4e01025b0 2024-04-02T13:52:53.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.549+0000 7fe4e7f38640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e010c800 con 0x7fe4e0101bf0 2024-04-02T13:52:53.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.549+0000 7fe4e5cd4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4e0100a00 0x7fe4e00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:53.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.549+0000 7fe4e5cd4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4e0100a00 0x7fe4e00697d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48526/0 (socket says 172.21.15.52:48526) 2024-04-02T13:52:53.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.549+0000 7fe4e5cd4640 1 -- 172.21.15.52:0/1982718677 learned_addr learned my addr 172.21.15.52:0/1982718677 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:53.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.550+0000 7fe4e64d5640 1 --2- 172.21.15.52:0/1982718677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4e01025b0 0x7fe4e019f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:53.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.550+0000 7fe4e54d3640 1 --2- 172.21.15.52:0/1982718677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4e0101bf0 0x7fe4e019cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:53.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.550+0000 7fe4e5cd4640 1 -- 172.21.15.52:0/1982718677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4e0101bf0 msgr2=0x7fe4e019cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:53.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.550+0000 7fe4e5cd4640 1 --2- 172.21.15.52:0/1982718677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4e0101bf0 0x7fe4e019cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.550+0000 7fe4e5cd4640 1 -- 172.21.15.52:0/1982718677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4e01025b0 msgr2=0x7fe4e019f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:53.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.550+0000 7fe4e5cd4640 1 --2- 172.21.15.52:0/1982718677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4e01025b0 0x7fe4e019f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.550+0000 7fe4e5cd4640 1 -- 172.21.15.52:0/1982718677 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e019fa70 con 0x7fe4e0100a00 2024-04-02T13:52:53.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.550+0000 7fe4e5cd4640 1 --2- 172.21.15.52:0/1982718677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4e0100a00 0x7fe4e00697d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00c970 tx=0x7fe4dc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:53.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.550+0000 7fe4d6ffd640 1 -- 172.21.15.52:0/1982718677 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc004220 con 0x7fe4e0100a00 2024-04-02T13:52:53.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.550+0000 7fe4e7f38640 1 -- 172.21.15.52:0/1982718677 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e019fd00 con 0x7fe4e0100a00 2024-04-02T13:52:53.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.550+0000 7fe4d6ffd640 1 -- 172.21.15.52:0/1982718677 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4dc0043c0 con 0x7fe4e0100a00 2024-04-02T13:52:53.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.550+0000 7fe4d6ffd640 1 -- 172.21.15.52:0/1982718677 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc004a00 con 0x7fe4e0100a00 2024-04-02T13:52:53.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.550+0000 7fe4e7f38640 1 -- 172.21.15.52:0/1982718677 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e01a01d0 con 0x7fe4e0100a00 2024-04-02T13:52:53.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.551+0000 7fe4e7f38640 1 -- 172.21.15.52:0/1982718677 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4e0107d40 con 0x7fe4e0100a00 2024-04-02T13:52:53.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.552+0000 7fe4d6ffd640 1 -- 172.21.15.52:0/1982718677 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe4dc0079b0 con 0x7fe4e0100a00 2024-04-02T13:52:53.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.553+0000 7fe4d6ffd640 1 --2- 172.21.15.52:0/1982718677 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe4cc0a3c40 0x7fe4cc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:53.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.553+0000 7fe4e54d3640 1 --2- 172.21.15.52:0/1982718677 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe4cc0a3c40 0x7fe4cc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:53.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.553+0000 7fe4d6ffd640 1 -- 172.21.15.52:0/1982718677 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fe4dc0d0580 con 0x7fe4e0100a00 2024-04-02T13:52:53.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.553+0000 7fe4e54d3640 1 --2- 172.21.15.52:0/1982718677 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe4cc0a3c40 0x7fe4cc0a6100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0005ab0 tx=0x7fe4d0004f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:53.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.557+0000 7fe4d6ffd640 1 -- 172.21.15.52:0/1982718677 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe4dc014030 con 0x7fe4e0100a00 2024-04-02T13:52:53.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.741+0000 7fe4e7f38640 1 -- 172.21.15.52:0/1982718677 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe4e002d5d0 con 0x7fe4e0100a00 2024-04-02T13:52:53.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.742+0000 7fe4d6ffd640 1 -- 172.21.15.52:0/1982718677 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7fe4dc09c450 con 0x7fe4e0100a00 2024-04-02T13:52:53.745 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-04-02T13:52:53.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.744+0000 7fe4e7f38640 1 -- 172.21.15.52:0/1982718677 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe4cc0a3c40 msgr2=0x7fe4cc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:53.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.744+0000 7fe4e7f38640 1 --2- 172.21.15.52:0/1982718677 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe4cc0a3c40 0x7fe4cc0a6100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0005ab0 tx=0x7fe4d0004f10 comp rx=0 tx=0).stop 2024-04-02T13:52:53.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.744+0000 7fe4e7f38640 1 -- 172.21.15.52:0/1982718677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4e0100a00 msgr2=0x7fe4e00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:53.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.744+0000 7fe4e7f38640 1 --2- 172.21.15.52:0/1982718677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4e0100a00 0x7fe4e00697d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00c970 tx=0x7fe4dc00ce30 comp rx=0 tx=0).stop 2024-04-02T13:52:53.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.744+0000 7fe4e7f38640 1 -- 172.21.15.52:0/1982718677 shutdown_connections 2024-04-02T13:52:53.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.744+0000 7fe4e7f38640 1 --2- 172.21.15.52:0/1982718677 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fe4cc0a3c40 0x7fe4cc0a6100 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.744+0000 7fe4e7f38640 1 --2- 172.21.15.52:0/1982718677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4e01025b0 0x7fe4e019f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.744+0000 7fe4e7f38640 1 --2- 172.21.15.52:0/1982718677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4e0101bf0 0x7fe4e019cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.744+0000 7fe4e7f38640 1 --2- 172.21.15.52:0/1982718677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4e0100a00 0x7fe4e00697d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.744+0000 7fe4e7f38640 1 -- 172.21.15.52:0/1982718677 >> 172.21.15.52:0/1982718677 conn(0x7fe4e00fbf90 msgr2=0x7fe4e0104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:53.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.744+0000 7fe4e7f38640 1 -- 172.21.15.52:0/1982718677 shutdown_connections 2024-04-02T13:52:53.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.744+0000 7fe4e7f38640 1 -- 172.21.15.52:0/1982718677 wait complete. 2024-04-02T13:52:53.757 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:53 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2505204439' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:52:53.757 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:53 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1666704427' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:52:53.757 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:53 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3550322188' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:52:53.757 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:53 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/4145545070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:52:53.758 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:52:53.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:53 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2505204439' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:52:53.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:53 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1666704427' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:52:53.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:53 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3550322188' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:52:53.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:53 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/4145545070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:52:53.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.858+0000 7f9b6dce4640 1 -- 172.21.15.52:0/2652643711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b681025b0 msgr2=0x7f9b681049a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:53.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.858+0000 7f9b6dce4640 1 --2- 172.21.15.52:0/2652643711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b681025b0 0x7f9b681049a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9b58009900 tx=0x7f9b5802bc30 comp rx=0 tx=0).stop 2024-04-02T13:52:53.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.858+0000 7f9b6dce4640 1 -- 172.21.15.52:0/2652643711 shutdown_connections 2024-04-02T13:52:53.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.858+0000 7f9b6dce4640 1 --2- 172.21.15.52:0/2652643711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b68107810 0x7f9b68109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.858+0000 7f9b6dce4640 1 --2- 172.21.15.52:0/2652643711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b68104ee0 0x7f9b681072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.858+0000 7f9b6dce4640 1 --2- 172.21.15.52:0/2652643711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b681025b0 0x7f9b681049a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.858+0000 7f9b6dce4640 1 -- 172.21.15.52:0/2652643711 >> 172.21.15.52:0/2652643711 conn(0x7f9b680fbfe0 msgr2=0x7f9b680fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:53.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.858+0000 7f9b6dce4640 1 -- 172.21.15.52:0/2652643711 shutdown_connections 2024-04-02T13:52:53.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.858+0000 7f9b6dce4640 1 -- 172.21.15.52:0/2652643711 wait complete. 2024-04-02T13:52:53.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.858+0000 7f9b6dce4640 1 Processor -- start 2024-04-02T13:52:53.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.858+0000 7f9b6dce4640 1 -- start start 2024-04-02T13:52:53.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.859+0000 7f9b6dce4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b681025b0 0x7f9b68197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:53.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.859+0000 7f9b6dce4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b68104ee0 0x7f9b681983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:53.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.859+0000 7f9b6dce4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b68107810 0x7f9b6819f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:53.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.859+0000 7f9b6dce4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6810d870 con 0x7f9b68104ee0 2024-04-02T13:52:53.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.859+0000 7f9b6dce4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6810d6f0 con 0x7f9b681025b0 2024-04-02T13:52:53.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.859+0000 7f9b6dce4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6810d9f0 con 0x7f9b68107810 2024-04-02T13:52:53.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.859+0000 7f9b66ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b68104ee0 0x7f9b681983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:53.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.859+0000 7f9b66ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b68104ee0 0x7f9b681983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48544/0 (socket says 172.21.15.52:48544) 2024-04-02T13:52:53.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.859+0000 7f9b66ffd640 1 -- 172.21.15.52:0/1400089852 learned_addr learned my addr 172.21.15.52:0/1400089852 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:53.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.859+0000 7f9b677fe640 1 --2- 172.21.15.52:0/1400089852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b681025b0 0x7f9b68197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:53.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.859+0000 7f9b67fff640 1 --2- 172.21.15.52:0/1400089852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b68107810 0x7f9b6819f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:53.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.859+0000 7f9b66ffd640 1 -- 172.21.15.52:0/1400089852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b68107810 msgr2=0x7f9b6819f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:53.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.859+0000 7f9b66ffd640 1 --2- 172.21.15.52:0/1400089852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b68107810 0x7f9b6819f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.859+0000 7f9b66ffd640 1 -- 172.21.15.52:0/1400089852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b681025b0 msgr2=0x7f9b68197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:53.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.859+0000 7f9b66ffd640 1 --2- 172.21.15.52:0/1400089852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b681025b0 0x7f9b68197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:53.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.859+0000 7f9b66ffd640 1 -- 172.21.15.52:0/1400089852 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6819fad0 con 0x7f9b68104ee0 2024-04-02T13:52:53.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.859+0000 7f9b66ffd640 1 --2- 172.21.15.52:0/1400089852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b68104ee0 0x7f9b681983f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c00d770 tx=0x7f9b5c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:53.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.859+0000 7f9b64ff9640 1 -- 172.21.15.52:0/1400089852 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c017070 con 0x7f9b68104ee0 2024-04-02T13:52:53.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.859+0000 7f9b6dce4640 1 -- 172.21.15.52:0/1400089852 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b6819fdc0 con 0x7f9b68104ee0 2024-04-02T13:52:53.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.859+0000 7f9b64ff9640 1 -- 172.21.15.52:0/1400089852 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b5c009c30 con 0x7f9b68104ee0 2024-04-02T13:52:53.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.859+0000 7f9b64ff9640 1 -- 172.21.15.52:0/1400089852 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c00f620 con 0x7f9b68104ee0 2024-04-02T13:52:53.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.859+0000 7f9b6dce4640 1 -- 172.21.15.52:0/1400089852 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b681a0280 con 0x7f9b68104ee0 2024-04-02T13:52:53.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.861+0000 7f9b6dce4640 1 -- 172.21.15.52:0/1400089852 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b680fdad0 con 0x7f9b68104ee0 2024-04-02T13:52:53.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.862+0000 7f9b64ff9640 1 -- 172.21.15.52:0/1400089852 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9b5c01a070 con 0x7f9b68104ee0 2024-04-02T13:52:53.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.862+0000 7f9b64ff9640 1 --2- 172.21.15.52:0/1400089852 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f9b500a3c40 0x7f9b500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:53.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.862+0000 7f9b64ff9640 1 -- 172.21.15.52:0/1400089852 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f9b5c0d0760 con 0x7f9b68104ee0 2024-04-02T13:52:53.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.862+0000 7f9b677fe640 1 --2- 172.21.15.52:0/1400089852 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f9b500a3c40 0x7f9b500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:53.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.863+0000 7f9b677fe640 1 --2- 172.21.15.52:0/1400089852 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f9b500a3c40 0x7f9b500a6100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9b5802c240 tx=0x7f9b5802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:53.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:53.867+0000 7f9b64ff9640 1 -- 172.21.15.52:0/1400089852 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9b5c009de0 con 0x7f9b68104ee0 2024-04-02T13:52:53.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:53 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2505204439' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:52:53.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:53 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1666704427' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:52:53.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:53 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3550322188' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:52:53.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:53 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/4145545070' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:52:54.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.092+0000 7f9b6dce4640 1 -- 172.21.15.52:0/1400089852 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b6802d370 con 0x7f9b68104ee0 2024-04-02T13:52:54.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.093+0000 7f9b64ff9640 1 -- 172.21.15.52:0/1400089852 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f9b5c09cc80 con 0x7f9b68104ee0 2024-04-02T13:52:54.096 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:52:54.096 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:51:29.345060+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24503,24475,24490,24467,24457]},"id":1}]} 2024-04-02T13:52:54.097 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 27 2024-04-02T13:52:54.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.095+0000 7f9b6dce4640 1 -- 172.21.15.52:0/1400089852 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f9b500a3c40 msgr2=0x7f9b500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:54.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.095+0000 7f9b6dce4640 1 --2- 172.21.15.52:0/1400089852 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f9b500a3c40 0x7f9b500a6100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9b5802c240 tx=0x7f9b5802df90 comp rx=0 tx=0).stop 2024-04-02T13:52:54.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.095+0000 7f9b6dce4640 1 -- 172.21.15.52:0/1400089852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b68104ee0 msgr2=0x7f9b681983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:54.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.095+0000 7f9b6dce4640 1 --2- 172.21.15.52:0/1400089852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b68104ee0 0x7f9b681983f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c00d770 tx=0x7f9b5c00dc30 comp rx=0 tx=0).stop 2024-04-02T13:52:54.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.095+0000 7f9b6dce4640 1 -- 172.21.15.52:0/1400089852 shutdown_connections 2024-04-02T13:52:54.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.095+0000 7f9b6dce4640 1 --2- 172.21.15.52:0/1400089852 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f9b500a3c40 0x7f9b500a6100 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.095+0000 7f9b6dce4640 1 --2- 172.21.15.52:0/1400089852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b68107810 0x7f9b6819f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.095+0000 7f9b6dce4640 1 --2- 172.21.15.52:0/1400089852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b68104ee0 0x7f9b681983f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.095+0000 7f9b6dce4640 1 --2- 172.21.15.52:0/1400089852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b681025b0 0x7f9b68197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.095+0000 7f9b6dce4640 1 -- 172.21.15.52:0/1400089852 >> 172.21.15.52:0/1400089852 conn(0x7f9b680fbfe0 msgr2=0x7f9b68102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:54.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.095+0000 7f9b6dce4640 1 -- 172.21.15.52:0/1400089852 shutdown_connections 2024-04-02T13:52:54.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.095+0000 7f9b6dce4640 1 -- 172.21.15.52:0/1400089852 wait complete. 2024-04-02T13:52:54.111 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:52:54.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.213+0000 7efdf937b640 1 -- 172.21.15.52:0/3695027730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf4107900 msgr2=0x7efdf4109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:54.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.213+0000 7efdf937b640 1 --2- 172.21.15.52:0/3695027730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf4107900 0x7efdf4109cf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7efdec009ca0 tx=0x7efdec02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.213+0000 7efdf937b640 1 -- 172.21.15.52:0/3695027730 shutdown_connections 2024-04-02T13:52:54.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.213+0000 7efdf937b640 1 --2- 172.21.15.52:0/3695027730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf4107900 0x7efdf4109cf0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.213+0000 7efdf937b640 1 --2- 172.21.15.52:0/3695027730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdf4104fd0 0x7efdf41073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.213+0000 7efdf937b640 1 --2- 172.21.15.52:0/3695027730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efdf41026a0 0x7efdf4104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.213+0000 7efdf937b640 1 -- 172.21.15.52:0/3695027730 >> 172.21.15.52:0/3695027730 conn(0x7efdf40fc030 msgr2=0x7efdf40fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:54.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.213+0000 7efdf937b640 1 -- 172.21.15.52:0/3695027730 shutdown_connections 2024-04-02T13:52:54.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.213+0000 7efdf937b640 1 -- 172.21.15.52:0/3695027730 wait complete. 2024-04-02T13:52:54.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.213+0000 7efdf937b640 1 Processor -- start 2024-04-02T13:52:54.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.213+0000 7efdf937b640 1 -- start start 2024-04-02T13:52:54.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.214+0000 7efdf937b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efdf41026a0 0x7efdf4197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.214+0000 7efdf937b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf4104fd0 0x7efdf4198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.214+0000 7efdf937b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdf4107900 0x7efdf419f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.214+0000 7efdf937b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efdf410d7d0 con 0x7efdf4104fd0 2024-04-02T13:52:54.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.214+0000 7efdf937b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efdf410d650 con 0x7efdf4107900 2024-04-02T13:52:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.214+0000 7efdf937b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efdf410d950 con 0x7efdf41026a0 2024-04-02T13:52:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.214+0000 7efdf27fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf4104fd0 0x7efdf4198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.214+0000 7efdf27fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf4104fd0 0x7efdf4198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48562/0 (socket says 172.21.15.52:48562) 2024-04-02T13:52:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.214+0000 7efdf27fc640 1 -- 172.21.15.52:0/2456910384 learned_addr learned my addr 172.21.15.52:0/2456910384 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.214+0000 7efdf2ffd640 1 --2- 172.21.15.52:0/2456910384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efdf41026a0 0x7efdf4197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.214+0000 7efdf37fe640 1 --2- 172.21.15.52:0/2456910384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdf4107900 0x7efdf419f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.214+0000 7efdf27fc640 1 -- 172.21.15.52:0/2456910384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efdf41026a0 msgr2=0x7efdf4197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.214+0000 7efdf27fc640 1 --2- 172.21.15.52:0/2456910384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efdf41026a0 0x7efdf4197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.214+0000 7efdf27fc640 1 -- 172.21.15.52:0/2456910384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdf4107900 msgr2=0x7efdf419f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.214+0000 7efdf27fc640 1 --2- 172.21.15.52:0/2456910384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdf4107900 0x7efdf419f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.214+0000 7efdf27fc640 1 -- 172.21.15.52:0/2456910384 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf419f9c0 con 0x7efdf4104fd0 2024-04-02T13:52:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.214+0000 7efdf37fe640 1 --2- 172.21.15.52:0/2456910384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdf4107900 0x7efdf419f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:52:54.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.214+0000 7efdf27fc640 1 --2- 172.21.15.52:0/2456910384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf4104fd0 0x7efdf4198350 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efddc00c970 tx=0x7efddc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:54.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.214+0000 7efdebfff640 1 -- 172.21.15.52:0/2456910384 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc014070 con 0x7efdf4104fd0 2024-04-02T13:52:54.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.214+0000 7efdebfff640 1 -- 172.21.15.52:0/2456910384 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efddc004440 con 0x7efdf4104fd0 2024-04-02T13:52:54.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.214+0000 7efdebfff640 1 -- 172.21.15.52:0/2456910384 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc00f450 con 0x7efdf4104fd0 2024-04-02T13:52:54.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.215+0000 7efdf937b640 1 -- 172.21.15.52:0/2456910384 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf419fcb0 con 0x7efdf4104fd0 2024-04-02T13:52:54.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.215+0000 7efdf937b640 1 -- 172.21.15.52:0/2456910384 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf41a01b0 con 0x7efdf4104fd0 2024-04-02T13:52:54.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.215+0000 7efdf937b640 1 -- 172.21.15.52:0/2456910384 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf40fdb20 con 0x7efdf4104fd0 2024-04-02T13:52:54.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.217+0000 7efdebfff640 1 -- 172.21.15.52:0/2456910384 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efddc017070 con 0x7efdf4104fd0 2024-04-02T13:52:54.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.218+0000 7efdebfff640 1 --2- 172.21.15.52:0/2456910384 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7efde00a3c40 0x7efde00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.218+0000 7efdf2ffd640 1 --2- 172.21.15.52:0/2456910384 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7efde00a3c40 0x7efde00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.218+0000 7efdebfff640 1 -- 172.21.15.52:0/2456910384 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7efddc0d0b10 con 0x7efdf4104fd0 2024-04-02T13:52:54.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.218+0000 7efdf2ffd640 1 --2- 172.21.15.52:0/2456910384 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7efde00a3c40 0x7efde00a6100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efde40058c0 tx=0x7efde400a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:54.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.221+0000 7efdebfff640 1 -- 172.21.15.52:0/2456910384 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7efddc01e020 con 0x7efdf4104fd0 2024-04-02T13:52:54.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.410+0000 7efdf937b640 1 -- 172.21.15.52:0/2456910384 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efdf402d4f0 con 0x7efdf4104fd0 2024-04-02T13:52:54.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.411+0000 7efdebfff640 1 -- 172.21.15.52:0/2456910384 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24549 (secure 0 0 0) 0x7efddc09d030 con 0x7efdf4104fd0 2024-04-02T13:52:54.415 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:52:54.415 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":92,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T13:51:56.472064+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:50:58.778690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:50:59.561527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:51:03.343487+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:52:54.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.413+0000 7efdf937b640 1 -- 172.21.15.52:0/2456910384 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7efde00a3c40 msgr2=0x7efde00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:54.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.413+0000 7efdf937b640 1 --2- 172.21.15.52:0/2456910384 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7efde00a3c40 0x7efde00a6100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efde40058c0 tx=0x7efde400a250 comp rx=0 tx=0).stop 2024-04-02T13:52:54.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.413+0000 7efdf937b640 1 -- 172.21.15.52:0/2456910384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf4104fd0 msgr2=0x7efdf4198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:54.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.413+0000 7efdf937b640 1 --2- 172.21.15.52:0/2456910384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf4104fd0 0x7efdf4198350 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efddc00c970 tx=0x7efddc00ce30 comp rx=0 tx=0).stop 2024-04-02T13:52:54.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.413+0000 7efdf937b640 1 -- 172.21.15.52:0/2456910384 shutdown_connections 2024-04-02T13:52:54.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.413+0000 7efdf937b640 1 --2- 172.21.15.52:0/2456910384 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7efde00a3c40 0x7efde00a6100 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.413+0000 7efdf937b640 1 --2- 172.21.15.52:0/2456910384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdf4107900 0x7efdf419f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.413+0000 7efdf937b640 1 --2- 172.21.15.52:0/2456910384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf4104fd0 0x7efdf4198350 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.413+0000 7efdf937b640 1 --2- 172.21.15.52:0/2456910384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efdf41026a0 0x7efdf4197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.413+0000 7efdf937b640 1 -- 172.21.15.52:0/2456910384 >> 172.21.15.52:0/2456910384 conn(0x7efdf40fc030 msgr2=0x7efdf4103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:54.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.413+0000 7efdf937b640 1 -- 172.21.15.52:0/2456910384 shutdown_connections 2024-04-02T13:52:54.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.413+0000 7efdf937b640 1 -- 172.21.15.52:0/2456910384 wait complete. 2024-04-02T13:52:54.428 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:52:54.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.532+0000 7f69ed932640 1 -- 172.21.15.52:0/813569732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69e8102d10 msgr2=0x7f69e8103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:54.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.532+0000 7f69ed932640 1 --2- 172.21.15.52:0/813569732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69e8102d10 0x7f69e8103190 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009900 tx=0x7f69d802bc30 comp rx=0 tx=0).stop 2024-04-02T13:52:54.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.533+0000 7f69ed932640 1 -- 172.21.15.52:0/813569732 shutdown_connections 2024-04-02T13:52:54.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.533+0000 7f69ed932640 1 --2- 172.21.15.52:0/813569732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69e81036d0 0x7f69e8109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.533+0000 7f69ed932640 1 --2- 172.21.15.52:0/813569732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69e8102d10 0x7f69e8103190 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.533+0000 7f69ed932640 1 --2- 172.21.15.52:0/813569732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69e8101b20 0x7f69e8101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.533+0000 7f69ed932640 1 -- 172.21.15.52:0/813569732 >> 172.21.15.52:0/813569732 conn(0x7f69e80fd0b0 msgr2=0x7f69e80ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:54.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.533+0000 7f69ed932640 1 -- 172.21.15.52:0/813569732 shutdown_connections 2024-04-02T13:52:54.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.533+0000 7f69ed932640 1 -- 172.21.15.52:0/813569732 wait complete. 2024-04-02T13:52:54.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.533+0000 7f69ed932640 1 Processor -- start 2024-04-02T13:52:54.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.533+0000 7f69ed932640 1 -- start start 2024-04-02T13:52:54.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.533+0000 7f69ed932640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69e8101b20 0x7f69e819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.533+0000 7f69ed932640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69e8102d10 0x7f69e819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.533+0000 7f69ed932640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69e81036d0 0x7f69e81a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.533+0000 7f69ed932640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f69e810d800 con 0x7f69e8102d10 2024-04-02T13:52:54.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.533+0000 7f69ed932640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f69e810d680 con 0x7f69e8101b20 2024-04-02T13:52:54.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.533+0000 7f69ed932640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f69e810d980 con 0x7f69e81036d0 2024-04-02T13:52:54.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.534+0000 7f69e67fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69e8102d10 0x7f69e819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.534+0000 7f69e67fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69e8102d10 0x7f69e819a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48580/0 (socket says 172.21.15.52:48580) 2024-04-02T13:52:54.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.534+0000 7f69e67fc640 1 -- 172.21.15.52:0/1450708826 learned_addr learned my addr 172.21.15.52:0/1450708826 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:54.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.534+0000 7f69e77fe640 1 --2- 172.21.15.52:0/1450708826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69e81036d0 0x7f69e81a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.534+0000 7f69e6ffd640 1 --2- 172.21.15.52:0/1450708826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69e8101b20 0x7f69e819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.534+0000 7f69e67fc640 1 -- 172.21.15.52:0/1450708826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69e81036d0 msgr2=0x7f69e81a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:54.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.534+0000 7f69e67fc640 1 --2- 172.21.15.52:0/1450708826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69e81036d0 0x7f69e81a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.534+0000 7f69e67fc640 1 -- 172.21.15.52:0/1450708826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69e8101b20 msgr2=0x7f69e819a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:54.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.534+0000 7f69e67fc640 1 --2- 172.21.15.52:0/1450708826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69e8101b20 0x7f69e819a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.534+0000 7f69e67fc640 1 -- 172.21.15.52:0/1450708826 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e81a1c50 con 0x7f69e8102d10 2024-04-02T13:52:54.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.534+0000 7f69e67fc640 1 --2- 172.21.15.52:0/1450708826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69e8102d10 0x7f69e819a570 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009230 tx=0x7f69d8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:54.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.534+0000 7f69ec930640 1 -- 172.21.15.52:0/1450708826 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d8041070 con 0x7f69e8102d10 2024-04-02T13:52:54.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.534+0000 7f69ed932640 1 -- 172.21.15.52:0/1450708826 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e81a1ee0 con 0x7f69e8102d10 2024-04-02T13:52:54.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.534+0000 7f69ec930640 1 -- 172.21.15.52:0/1450708826 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69d80054e0 con 0x7f69e8102d10 2024-04-02T13:52:54.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.534+0000 7f69ec930640 1 -- 172.21.15.52:0/1450708826 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d80346a0 con 0x7f69e8102d10 2024-04-02T13:52:54.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.534+0000 7f69ed932640 1 -- 172.21.15.52:0/1450708826 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e81a2340 con 0x7f69e8102d10 2024-04-02T13:52:54.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.535+0000 7f69ed932640 1 -- 172.21.15.52:0/1450708826 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e81a2570 con 0x7f69e8102d10 2024-04-02T13:52:54.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.536+0000 7f69ec930640 1 -- 172.21.15.52:0/1450708826 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f69d8044070 con 0x7f69e8102d10 2024-04-02T13:52:54.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.537+0000 7f69ec930640 1 --2- 172.21.15.52:0/1450708826 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f69d00a3c40 0x7f69d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.537+0000 7f69ec930640 1 -- 172.21.15.52:0/1450708826 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f69d80f25d0 con 0x7f69e8102d10 2024-04-02T13:52:54.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.537+0000 7f69e6ffd640 1 --2- 172.21.15.52:0/1450708826 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f69d00a3c40 0x7f69d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.538+0000 7f69e6ffd640 1 --2- 172.21.15.52:0/1450708826 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f69d00a3c40 0x7f69d00a6100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f69dc005fd0 tx=0x7f69dc005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:54.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.542+0000 7f69ec930640 1 -- 172.21.15.52:0/1450708826 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f69d80bec00 con 0x7f69e8102d10 2024-04-02T13:52:54.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:54 smithi189 ceph-mon[36683]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T13:52:54.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:54 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1982718677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:52:54.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:54 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1400089852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:52:54.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:54 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2456910384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:52:54.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.801+0000 7f69ed932640 1 -- 172.21.15.52:0/1450708826 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69e81a2740 con 0x7f69e8102d10 2024-04-02T13:52:54.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.802+0000 7f69ec930640 1 -- 172.21.15.52:0/1450708826 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f69d80be500 con 0x7f69e8102d10 2024-04-02T13:52:54.805 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:52:54.805 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:51:29.345060+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24503,24475,24490,24467,24457]},"id":1}]} 2024-04-02T13:52:54.805 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 27 2024-04-02T13:52:54.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.804+0000 7f69ed932640 1 -- 172.21.15.52:0/1450708826 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f69d00a3c40 msgr2=0x7f69d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:54.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.804+0000 7f69ed932640 1 --2- 172.21.15.52:0/1450708826 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f69d00a3c40 0x7f69d00a6100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f69dc005fd0 tx=0x7f69dc005ce0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.804+0000 7f69ed932640 1 -- 172.21.15.52:0/1450708826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69e8102d10 msgr2=0x7f69e819a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:54.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.804+0000 7f69ed932640 1 --2- 172.21.15.52:0/1450708826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69e8102d10 0x7f69e819a570 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009230 tx=0x7f69d8004040 comp rx=0 tx=0).stop 2024-04-02T13:52:54.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.804+0000 7f69ed932640 1 -- 172.21.15.52:0/1450708826 shutdown_connections 2024-04-02T13:52:54.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.804+0000 7f69ed932640 1 --2- 172.21.15.52:0/1450708826 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f69d00a3c40 0x7f69d00a6100 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.804+0000 7f69ed932640 1 --2- 172.21.15.52:0/1450708826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69e81036d0 0x7f69e81a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.804+0000 7f69ed932640 1 --2- 172.21.15.52:0/1450708826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69e8102d10 0x7f69e819a570 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.804+0000 7f69ed932640 1 --2- 172.21.15.52:0/1450708826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69e8101b20 0x7f69e819a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.804+0000 7f69ed932640 1 -- 172.21.15.52:0/1450708826 >> 172.21.15.52:0/1450708826 conn(0x7f69e80fd0b0 msgr2=0x7f69e8073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:54.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.804+0000 7f69ed932640 1 -- 172.21.15.52:0/1450708826 shutdown_connections 2024-04-02T13:52:54.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.804+0000 7f69ed932640 1 -- 172.21.15.52:0/1450708826 wait complete. 2024-04-02T13:52:54.818 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:54 smithi052 ceph-mon[32104]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T13:52:54.818 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:54 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1982718677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:52:54.818 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:54 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1400089852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:52:54.818 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:54 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2456910384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:52:54.819 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-02T13:51:00.603246+0000', 'modified': '2024-04-02T13:51:29.345060+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24503, 'mds_1': 24490, 'mds_2': 24475, 'mds_3': 24467, 'mds_4': 24457}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14604': {'gid': 14604, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.52:6839/2213112521', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6838', 'nonce': 2213112521}, {'type': 'v1', 'addr': '172.21.15.52:6839', 'nonce': 2213112521}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14616': {'gid': 14616, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.52:6841/2689882080', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6840', 'nonce': 2689882080}, {'type': 'v1', 'addr': '172.21.15.52:6841', 'nonce': 2689882080}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24410': {'gid': 24410, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.88:6833/4248342942', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6832', 'nonce': 4248342942}, {'type': 'v1', 'addr': '172.21.15.88:6833', 'nonce': 4248342942}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6833/3157031038', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6832', 'nonce': 3157031038}, {'type': 'v1', 'addr': '172.21.15.189:6833', 'nonce': 3157031038}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24437': {'gid': 24437, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.88:6835/5019107', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6834', 'nonce': 5019107}, {'type': 'v1', 'addr': '172.21.15.88:6835', 'nonce': 5019107}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.189:6835/1647960400', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6834', 'nonce': 1647960400}, {'type': 'v1', 'addr': '172.21.15.189:6835', 'nonce': 1647960400}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.88:6837/1091039373', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6836', 'nonce': 1091039373}, {'type': 'v1', 'addr': '172.21.15.88:6837', 'nonce': 1091039373}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.189:6837/2295541152', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6836', 'nonce': 2295541152}, {'type': 'v1', 'addr': '172.21.15.189:6837', 'nonce': 2295541152}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.189:6839/729001189', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6838', 'nonce': 729001189}, {'type': 'v1', 'addr': '172.21.15.189:6839', 'nonce': 729001189}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24503': {'gid': 24503, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.88:6839/2513527493', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6838', 'nonce': 2513527493}, {'type': 'v1', 'addr': '172.21.15.88:6839', 'nonce': 2513527493}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24503, 'qdb_cluster': [24503, 24475, 24490, 24467, 24457]} 2024-04-02T13:52:54.819 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-02T13:52:54.820 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-02T13:52:54.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:54 smithi088 ceph-mon[36528]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T13:52:54.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:54 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1982718677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:52:54.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:54 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1400089852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:52:54.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:54 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2456910384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:52:54.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.922+0000 7f38cc6a0640 1 -- 172.21.15.52:0/609231660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38c4102620 msgr2=0x7f38c4104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:54.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.922+0000 7f38cc6a0640 1 --2- 172.21.15.52:0/609231660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38c4102620 0x7f38c4104a10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f38c0009900 tx=0x7f38c002bc30 comp rx=0 tx=0).stop 2024-04-02T13:52:54.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.922+0000 7f38cc6a0640 1 -- 172.21.15.52:0/609231660 shutdown_connections 2024-04-02T13:52:54.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.922+0000 7f38cc6a0640 1 --2- 172.21.15.52:0/609231660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38c4107880 0x7f38c4109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.922+0000 7f38cc6a0640 1 --2- 172.21.15.52:0/609231660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c4104f50 0x7f38c4107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.922+0000 7f38cc6a0640 1 --2- 172.21.15.52:0/609231660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38c4102620 0x7f38c4104a10 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.922+0000 7f38cc6a0640 1 -- 172.21.15.52:0/609231660 >> 172.21.15.52:0/609231660 conn(0x7f38c40fbf90 msgr2=0x7f38c40fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:54.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.922+0000 7f38cc6a0640 1 -- 172.21.15.52:0/609231660 shutdown_connections 2024-04-02T13:52:54.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.922+0000 7f38cc6a0640 1 -- 172.21.15.52:0/609231660 wait complete. 2024-04-02T13:52:54.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.922+0000 7f38cc6a0640 1 Processor -- start 2024-04-02T13:52:54.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.922+0000 7f38cc6a0640 1 -- start start 2024-04-02T13:52:54.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.922+0000 7f38cc6a0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c4102620 0x7f38c4197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.922+0000 7f38cc6a0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38c4104f50 0x7f38c41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.922+0000 7f38cc6a0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38c4107880 0x7f38c419f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.922+0000 7f38cc6a0640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f38c410d770 con 0x7f38c4107880 2024-04-02T13:52:54.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.922+0000 7f38cc6a0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f38c410d5f0 con 0x7f38c4102620 2024-04-02T13:52:54.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.923+0000 7f38cc6a0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f38c410d8f0 con 0x7f38c4104f50 2024-04-02T13:52:54.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.923+0000 7f38cac3d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38c4107880 0x7f38c419f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.923+0000 7f38cac3d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38c4107880 0x7f38c419f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48598/0 (socket says 172.21.15.52:48598) 2024-04-02T13:52:54.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.923+0000 7f38cac3d640 1 -- 172.21.15.52:0/2674962623 learned_addr learned my addr 172.21.15.52:0/2674962623 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:54.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.923+0000 7f38c9c3b640 1 --2- 172.21.15.52:0/2674962623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38c4104f50 0x7f38c41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.923+0000 7f38ca43c640 1 --2- 172.21.15.52:0/2674962623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c4102620 0x7f38c4197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.923+0000 7f38cac3d640 1 -- 172.21.15.52:0/2674962623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38c4104f50 msgr2=0x7f38c41983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:54.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.923+0000 7f38cac3d640 1 --2- 172.21.15.52:0/2674962623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38c4104f50 0x7f38c41983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.923+0000 7f38cac3d640 1 -- 172.21.15.52:0/2674962623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c4102620 msgr2=0x7f38c4197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:54.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.923+0000 7f38cac3d640 1 --2- 172.21.15.52:0/2674962623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c4102620 0x7f38c4197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.923+0000 7f38cac3d640 1 -- 172.21.15.52:0/2674962623 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c419fac0 con 0x7f38c4107880 2024-04-02T13:52:54.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.923+0000 7f38cac3d640 1 --2- 172.21.15.52:0/2674962623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38c4107880 0x7f38c419f450 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009fd0 tx=0x7f38bc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:54.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.923+0000 7f38bb7fe640 1 -- 172.21.15.52:0/2674962623 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc017070 con 0x7f38c4107880 2024-04-02T13:52:54.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.923+0000 7f38cc6a0640 1 -- 172.21.15.52:0/2674962623 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c419fdb0 con 0x7f38c4107880 2024-04-02T13:52:54.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.923+0000 7f38bb7fe640 1 -- 172.21.15.52:0/2674962623 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38bc003e50 con 0x7f38c4107880 2024-04-02T13:52:54.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.923+0000 7f38bb7fe640 1 -- 172.21.15.52:0/2674962623 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc0106c0 con 0x7f38c4107880 2024-04-02T13:52:54.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.923+0000 7f38cc6a0640 1 -- 172.21.15.52:0/2674962623 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c41a0220 con 0x7f38c4107880 2024-04-02T13:52:54.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.925+0000 7f38bb7fe640 1 -- 172.21.15.52:0/2674962623 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f38bc003970 con 0x7f38c4107880 2024-04-02T13:52:54.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.926+0000 7f38cf285740 1 Processor -- start 2024-04-02T13:52:54.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.926+0000 7f38cf285740 1 -- start start 2024-04-02T13:52:54.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.926+0000 7f38cf285740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559ab2f0aae0 0x559ab2f0af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.926+0000 7f38cf285740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ab2f0b4d0 0x559ab2f0ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.926+0000 7f38cf285740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f10640 0x559ab2f12ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.926+0000 7f38cf285740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559ab2e6f060 con 0x559ab2f10640 2024-04-02T13:52:54.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.926+0000 7f38cf285740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559ab2e17650 con 0x559ab2f0aae0 2024-04-02T13:52:54.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.926+0000 7f38cf285740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559ab2e6b010 con 0x559ab2f0b4d0 2024-04-02T13:52:54.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.926+0000 7f38bb7fe640 1 --2- 172.21.15.52:0/2674962623 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f38b00a3c40 0x7f38b00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.927+0000 7f38c9c3b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f10640 0x559ab2f12ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.927+0000 7f38cac3d640 1 --2- 172.21.15.52:0/2674962623 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f38b00a3c40 0x7f38b00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.927+0000 7f38c9c3b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f10640 0x559ab2f12ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48610/0 (socket says 172.21.15.52:48610) 2024-04-02T13:52:54.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.927+0000 7f38c9c3b640 1 -- 172.21.15.52:0/2186756741 learned_addr learned my addr 172.21.15.52:0/2186756741 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:54.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.927+0000 7f38ca43c640 1 --2- 172.21.15.52:0/2186756741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ab2f0b4d0 0x559ab2f0ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.927+0000 7f38c9c3b640 1 --2- 172.21.15.52:0/2186756741 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559ab2f0aae0 0x559ab2f0af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.927+0000 7f38bb7fe640 1 -- 172.21.15.52:0/2674962623 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f38bc0d0610 con 0x7f38c4107880 2024-04-02T13:52:54.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.927+0000 7f38c9c3b640 1 -- 172.21.15.52:0/2186756741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ab2f0b4d0 msgr2=0x559ab2f0ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:54.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.927+0000 7f38c9c3b640 1 --2- 172.21.15.52:0/2186756741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ab2f0b4d0 0x559ab2f0ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.927+0000 7f38c9c3b640 1 -- 172.21.15.52:0/2186756741 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559ab2f0aae0 msgr2=0x559ab2f0af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:54.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.927+0000 7f38c9c3b640 1 --2- 172.21.15.52:0/2186756741 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559ab2f0aae0 0x559ab2f0af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.927+0000 7f38c9c3b640 1 -- 172.21.15.52:0/2186756741 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ab2b832d0 con 0x559ab2f10640 2024-04-02T13:52:54.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.927+0000 7f38ca43c640 1 --2- 172.21.15.52:0/2186756741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ab2f0b4d0 0x559ab2f0ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:54.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.927+0000 7f38cac3d640 1 --2- 172.21.15.52:0/2674962623 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f38b00a3c40 0x7f38b00a6100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0a4b10 tx=0x7f38bc06df10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:54.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.927+0000 7f38c9c3b640 1 --2- 172.21.15.52:0/2186756741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f10640 0x559ab2f12ad0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f38b4019980 tx=0x7f38b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d22c09874d637b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:54.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.927+0000 7f38b8ff9640 1 -- 172.21.15.52:0/2186756741 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b40192d0 con 0x559ab2f10640 2024-04-02T13:52:54.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.927+0000 7f38b8ff9640 1 -- 172.21.15.52:0/2186756741 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38b4010770 con 0x559ab2f10640 2024-04-02T13:52:54.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.927+0000 7f38b8ff9640 1 -- 172.21.15.52:0/2186756741 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b4003850 con 0x559ab2f10640 2024-04-02T13:52:54.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.928+0000 7f38cf285740 1 -- 172.21.15.52:0/2186756741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f10640 msgr2=0x559ab2f12ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:54.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.928+0000 7f38cf285740 1 --2- 172.21.15.52:0/2186756741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f10640 0x559ab2f12ad0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f38b4019980 tx=0x7f38b4004730 comp rx=0 tx=0).stop 2024-04-02T13:52:54.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.928+0000 7f38cf285740 1 -- 172.21.15.52:0/2186756741 shutdown_connections 2024-04-02T13:52:54.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.928+0000 7f38cf285740 1 --2- 172.21.15.52:0/2186756741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f10640 0x559ab2f12ad0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.928+0000 7f38cf285740 1 --2- 172.21.15.52:0/2186756741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ab2f0b4d0 0x559ab2f0ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.928+0000 7f38cf285740 1 --2- 172.21.15.52:0/2186756741 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559ab2f0aae0 0x559ab2f0af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.928+0000 7f38cf285740 1 -- 172.21.15.52:0/2186756741 >> 172.21.15.52:0/2186756741 conn(0x559ab2f06050 msgr2=0x559ab2f06450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:54.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.928+0000 7f38cf285740 1 -- 172.21.15.52:0/2186756741 shutdown_connections 2024-04-02T13:52:54.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.928+0000 7f38cf285740 1 -- 172.21.15.52:0/2186756741 wait complete. 2024-04-02T13:52:54.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.929+0000 7f38cf285740 1 -- start start 2024-04-02T13:52:54.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.929+0000 7f38cf285740 1 Processor -- start 2024-04-02T13:52:54.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.929+0000 7f38cf285740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559ab2f0aae0 0x559ab2f14ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.929+0000 7f38cf285740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f0b4d0 0x559ab2f15530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.929+0000 7f38cf285740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ab2f10640 0x559ab2f1c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.929+0000 7f38cf285740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559ab2f135f0 con 0x559ab2f0b4d0 2024-04-02T13:52:54.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.929+0000 7f38cf285740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559ab2e2aa10 con 0x559ab2f0aae0 2024-04-02T13:52:54.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.929+0000 7f38cf285740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559ab2f13770 con 0x559ab2f10640 2024-04-02T13:52:54.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.929+0000 7f38cf285740 10 client.0 _ll_register_callbacks cb 0x559ab2e670b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:54.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.929+0000 7f38ca43c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f0b4d0 0x559ab2f15530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.929+0000 7f38ca43c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f0b4d0 0x559ab2f15530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48618/0 (socket says 172.21.15.52:48618) 2024-04-02T13:52:54.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.929+0000 7f38ca43c640 1 -- 172.21.15.52:0/1146342412 learned_addr learned my addr 172.21.15.52:0/1146342412 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:54.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.929+0000 7f38c9c3b640 1 --2- 172.21.15.52:0/1146342412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559ab2f0aae0 0x559ab2f14ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.929+0000 7f38c9c3b640 1 --2- 172.21.15.52:0/1146342412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ab2f10640 0x559ab2f1c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.929+0000 7f38ca43c640 1 -- 172.21.15.52:0/1146342412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ab2f10640 msgr2=0x559ab2f1c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:54.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.929+0000 7f38ca43c640 1 --2- 172.21.15.52:0/1146342412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ab2f10640 0x559ab2f1c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.929+0000 7f38ca43c640 1 -- 172.21.15.52:0/1146342412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559ab2f0aae0 msgr2=0x559ab2f14ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:54.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.929+0000 7f38ca43c640 1 --2- 172.21.15.52:0/1146342412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559ab2f0aae0 0x559ab2f14ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.929+0000 7f38ca43c640 1 -- 172.21.15.52:0/1146342412 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ab2c4d6b0 con 0x559ab2f0b4d0 2024-04-02T13:52:54.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.929+0000 7f38c9c3b640 1 --2- 172.21.15.52:0/1146342412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559ab2f0aae0 0x559ab2f14ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:52:54.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.930+0000 7f38ca43c640 1 --2- 172.21.15.52:0/1146342412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f0b4d0 0x559ab2f15530 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f38c0036040 tx=0x7f38c0009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c3755bc3911c58 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:54.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.930+0000 7f38adffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:52:54.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.930+0000 7f38adffb640 1 -- 172.21.15.52:0/1146342412 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c0005230 con 0x559ab2f0b4d0 2024-04-02T13:52:54.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.930+0000 7f38cf285740 1 -- 172.21.15.52:0/1146342412 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38b4010770 con 0x559ab2f0b4d0 2024-04-02T13:52:54.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.930+0000 7f38adffb640 1 -- 172.21.15.52:0/1146342412 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f38c00053d0 con 0x559ab2f0b4d0 2024-04-02T13:52:54.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.930+0000 7f38adffb640 1 -- 172.21.15.52:0/1146342412 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f38c004e050 con 0x559ab2f0b4d0 2024-04-02T13:52:54.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.930+0000 7f38cf285740 10 client.15210 fetch_fsmap learned FSMap version 27 2024-04-02T13:52:54.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.930+0000 7f38cf285740 1 -- 172.21.15.52:0/1146342412 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559ab2f1f980 con 0x559ab2f0b4d0 2024-04-02T13:52:54.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.930+0000 7f38adffb640 1 -- 172.21.15.52:0/1146342412 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38b4010770 con 0x559ab2f0b4d0 2024-04-02T13:52:54.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.930+0000 7f38adffb640 1 -- 172.21.15.52:0/1146342412 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f38c00036f0 con 0x559ab2f0b4d0 2024-04-02T13:52:54.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.930+0000 7f38cf285740 10 client.15210 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:52:54.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.930+0000 7f38cf285740 10 client.15210 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-02T13:52:54.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.930+0000 7f38cf285740 20 client.15210 populate_metadata read hostname 'smithi052' 2024-04-02T13:52:54.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.930+0000 7f38cf285740 1 --2- 172.21.15.52:0/1146342412 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559ab2f21fa0 0x559ab2f23410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.930+0000 7f38cf285740 4 client.15210 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:52:54.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.930+0000 7f38cf285740 1 -- 172.21.15.52:0/1146342412 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ab2e16cf0 con 0x559ab2f21fa0 2024-04-02T13:52:54.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.931+0000 7f38c9c3b640 1 --2- 172.21.15.52:0/1146342412 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559ab2f21fa0 0x559ab2f23410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.931+0000 7f38c9c3b640 1 --2- 172.21.15.52:0/1146342412 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559ab2f21fa0 0x559ab2f23410 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6704bc85b3d88101 server_cookie=5a611a4c508822eb in_seq=0 out_seq=0 2024-04-02T13:52:54.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.931+0000 7f38adffb640 10 client.15210 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:52:54.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.932+0000 7f38adffb640 1 -- 172.21.15.52:0/1146342412 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559ab2e16cf0 con 0x559ab2f21fa0 2024-04-02T13:52:54.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.932+0000 7f38adffb640 10 client.15210 handle_command_reply: tid=0 2024-04-02T13:52:54.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.932+0000 7f38cf285740 1 client.15210 shutdown 2024-04-02T13:52:54.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.932+0000 7f38cf285740 1 -- 172.21.15.52:0/1146342412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f0b4d0 msgr2=0x559ab2f15530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:54.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.932+0000 7f38cf285740 1 --2- 172.21.15.52:0/1146342412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f0b4d0 0x559ab2f15530 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f38c0036040 tx=0x7f38c0009680 comp rx=0 tx=0).stop 2024-04-02T13:52:54.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.932+0000 7f38cf285740 1 -- 172.21.15.52:0/1146342412 shutdown_connections 2024-04-02T13:52:54.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.932+0000 7f38cf285740 1 --2- 172.21.15.52:0/1146342412 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559ab2f21fa0 0x559ab2f23410 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.932+0000 7f38cf285740 1 --2- 172.21.15.52:0/1146342412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ab2f10640 0x559ab2f1c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.932+0000 7f38cf285740 1 --2- 172.21.15.52:0/1146342412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f0b4d0 0x559ab2f15530 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.932+0000 7f38adffb640 0 client.15210 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:52:54.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.932+0000 7f38cf285740 1 --2- 172.21.15.52:0/1146342412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559ab2f0aae0 0x559ab2f14ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.932+0000 7f38cf285740 1 -- 172.21.15.52:0/1146342412 >> 172.21.15.52:0/1146342412 conn(0x559ab2f06050 msgr2=0x559ab2f05190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:54.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.932+0000 7f38cf285740 1 -- 172.21.15.52:0/1146342412 shutdown_connections 2024-04-02T13:52:54.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.932+0000 7f38cf285740 1 -- 172.21.15.52:0/1146342412 wait complete. 2024-04-02T13:52:54.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.932+0000 7f38cf285740 20 client.15210 trim_cache size 0 max 16384 2024-04-02T13:52:54.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.950+0000 7f38cf285740 1 Processor -- start 2024-04-02T13:52:54.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.950+0000 7f38cf285740 1 -- start start 2024-04-02T13:52:54.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.950+0000 7f38cf285740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559ab2f0aae0 0x559ab2f0c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.950+0000 7f38cf285740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f0b4d0 0x559ab2f1c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.950+0000 7f38cf285740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ab2f10640 0x559ab2f0cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.950+0000 7f38cf285740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559ab2f21750 con 0x559ab2f0b4d0 2024-04-02T13:52:54.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.950+0000 7f38cf285740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559ab2f21e20 con 0x559ab2f0aae0 2024-04-02T13:52:54.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.950+0000 7f38cf285740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559ab2f224f0 con 0x559ab2f10640 2024-04-02T13:52:54.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.950+0000 7f38ca43c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f0b4d0 0x559ab2f1c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.950+0000 7f38ca43c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f0b4d0 0x559ab2f1c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48624/0 (socket says 172.21.15.52:48624) 2024-04-02T13:52:54.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.950+0000 7f38ca43c640 1 -- 172.21.15.52:0/2788025544 learned_addr learned my addr 172.21.15.52:0/2788025544 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:54.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.951+0000 7f38c9c3b640 1 --2- 172.21.15.52:0/2788025544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ab2f10640 0x559ab2f0cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.951+0000 7f38c9c3b640 1 --2- 172.21.15.52:0/2788025544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559ab2f0aae0 0x559ab2f0c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.951+0000 7f38ca43c640 1 -- 172.21.15.52:0/2788025544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ab2f10640 msgr2=0x559ab2f0cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:54.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.951+0000 7f38ca43c640 1 --2- 172.21.15.52:0/2788025544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ab2f10640 0x559ab2f0cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.951+0000 7f38ca43c640 1 -- 172.21.15.52:0/2788025544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559ab2f0aae0 msgr2=0x559ab2f0c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:54.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.951+0000 7f38ca43c640 1 --2- 172.21.15.52:0/2788025544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559ab2f0aae0 0x559ab2f0c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.951+0000 7f38ca43c640 1 -- 172.21.15.52:0/2788025544 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ab2f229a0 con 0x559ab2f0b4d0 2024-04-02T13:52:54.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.951+0000 7f38c9c3b640 1 --2- 172.21.15.52:0/2788025544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559ab2f0aae0 0x559ab2f0c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:52:54.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.951+0000 7f38ca43c640 1 --2- 172.21.15.52:0/2788025544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f0b4d0 0x559ab2f1c530 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f38c002c7b0 tx=0x7f38c0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca854a42bfc7a5a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:54.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.951+0000 7f38affff640 1 -- 172.21.15.52:0/2788025544 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c004b040 con 0x559ab2f0b4d0 2024-04-02T13:52:54.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.951+0000 7f38affff640 1 -- 172.21.15.52:0/2788025544 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38c0041070 con 0x559ab2f0b4d0 2024-04-02T13:52:54.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.951+0000 7f38affff640 1 -- 172.21.15.52:0/2788025544 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c0003850 con 0x559ab2f0b4d0 2024-04-02T13:52:54.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.952+0000 7f38cf285740 1 -- 172.21.15.52:0/2788025544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f0b4d0 msgr2=0x559ab2f1c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:54.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.952+0000 7f38cf285740 1 --2- 172.21.15.52:0/2788025544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f0b4d0 0x559ab2f1c530 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f38c002c7b0 tx=0x7f38c0005e50 comp rx=0 tx=0).stop 2024-04-02T13:52:54.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.952+0000 7f38cf285740 1 -- 172.21.15.52:0/2788025544 shutdown_connections 2024-04-02T13:52:54.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.952+0000 7f38cf285740 1 --2- 172.21.15.52:0/2788025544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ab2f10640 0x559ab2f0cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.952+0000 7f38cf285740 1 --2- 172.21.15.52:0/2788025544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f0b4d0 0x559ab2f1c530 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.952+0000 7f38cf285740 1 --2- 172.21.15.52:0/2788025544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559ab2f0aae0 0x559ab2f0c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.952+0000 7f38cf285740 1 -- 172.21.15.52:0/2788025544 >> 172.21.15.52:0/2788025544 conn(0x559ab2f06050 msgr2=0x559ab2f06430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:54.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.952+0000 7f38cf285740 1 -- 172.21.15.52:0/2788025544 shutdown_connections 2024-04-02T13:52:54.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.952+0000 7f38cf285740 1 -- 172.21.15.52:0/2788025544 wait complete. 2024-04-02T13:52:54.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.952+0000 7f38cf285740 1 -- start start 2024-04-02T13:52:54.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.952+0000 7f38cf285740 1 Processor -- start 2024-04-02T13:52:54.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.952+0000 7f38cf285740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ab2f0aae0 0x559ab2f18740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.952+0000 7f38cf285740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559ab2f0b4d0 0x559ab2f18c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.952+0000 7f38cf285740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f10640 0x559ab2f1bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.952+0000 7f38cf285740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559ab2f23290 con 0x559ab2f10640 2024-04-02T13:52:54.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.952+0000 7f38cf285740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559ab2f22bc0 con 0x559ab2f0b4d0 2024-04-02T13:52:54.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.952+0000 7f38cf285740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559ab2f23960 con 0x559ab2f0aae0 2024-04-02T13:52:54.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.952+0000 7f38cf285740 10 client.0 _ll_register_callbacks cb 0x7f38c0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:54.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.952+0000 7f38c9c3b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f10640 0x559ab2f1bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.953+0000 7f38c9c3b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f10640 0x559ab2f1bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48628/0 (socket says 172.21.15.52:48628) 2024-04-02T13:52:54.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.953+0000 7f38c9c3b640 1 -- 172.21.15.52:0/4123021898 learned_addr learned my addr 172.21.15.52:0/4123021898 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:54.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.953+0000 7f38ca43c640 1 --2- 172.21.15.52:0/4123021898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559ab2f0b4d0 0x559ab2f18c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.953+0000 7f38c9c3b640 1 --2- 172.21.15.52:0/4123021898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ab2f0aae0 0x559ab2f18740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.953+0000 7f38ca43c640 1 -- 172.21.15.52:0/4123021898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ab2f0aae0 msgr2=0x559ab2f18740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:54.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.953+0000 7f38ca43c640 1 --2- 172.21.15.52:0/4123021898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ab2f0aae0 0x559ab2f18740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.953+0000 7f38ca43c640 1 -- 172.21.15.52:0/4123021898 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f10640 msgr2=0x559ab2f1bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:54.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.953+0000 7f38ca43c640 1 --2- 172.21.15.52:0/4123021898 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f10640 0x559ab2f1bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.953+0000 7f38ca43c640 1 -- 172.21.15.52:0/4123021898 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ab2f1c290 con 0x559ab2f0b4d0 2024-04-02T13:52:54.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.953+0000 7f38c9c3b640 1 --2- 172.21.15.52:0/4123021898 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f10640 0x559ab2f1bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:54.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.953+0000 7f38ca43c640 1 --2- 172.21.15.52:0/4123021898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559ab2f0b4d0 0x559ab2f18c80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x559ab2f22fd0 tx=0x7f38c002c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e94312dcc02396 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:54.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.953+0000 7f38adffb640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-02T13:52:54.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.953+0000 7f38adffb640 1 -- 172.21.15.52:0/4123021898 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c005f020 con 0x559ab2f0b4d0 2024-04-02T13:52:54.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.953+0000 7f38adffb640 1 -- 172.21.15.52:0/4123021898 <== mon.2 v2:172.21.15.88:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f38c00037f0 con 0x559ab2f0b4d0 2024-04-02T13:52:54.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.953+0000 7f38cf285740 1 -- 172.21.15.52:0/4123021898 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38c0041070 con 0x559ab2f0b4d0 2024-04-02T13:52:54.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.953+0000 7f38adffb640 1 -- 172.21.15.52:0/4123021898 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f38c0034650 con 0x559ab2f0b4d0 2024-04-02T13:52:54.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.954+0000 7f38cf285740 10 client.24950 fetch_fsmap learned FSMap version 27 2024-04-02T13:52:54.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.954+0000 7f38cf285740 1 -- 172.21.15.52:0/4123021898 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559ab2f88980 con 0x559ab2f0b4d0 2024-04-02T13:52:54.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.954+0000 7f38adffb640 1 -- 172.21.15.52:0/4123021898 <== mon.2 v2:172.21.15.88:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38c0041070 con 0x559ab2f0b4d0 2024-04-02T13:52:54.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.954+0000 7f38adffb640 1 -- 172.21.15.52:0/4123021898 <== mon.2 v2:172.21.15.88:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f38c0046700 con 0x559ab2f0b4d0 2024-04-02T13:52:54.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.954+0000 7f38cf285740 10 client.24950 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:52:54.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.954+0000 7f38cf285740 10 client.24950 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-02T13:52:54.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.954+0000 7f38cf285740 20 client.24950 populate_metadata read hostname 'smithi052' 2024-04-02T13:52:54.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.954+0000 7f38cf285740 1 --2- 172.21.15.52:0/4123021898 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559ab2f8d170 0x559ab2f8be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:54.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.954+0000 7f38cf285740 4 client.24950 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:52:54.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.954+0000 7f38cf285740 1 -- 172.21.15.52:0/4123021898 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "status"}) -- 0x559ab2e7c5a0 con 0x559ab2f8d170 2024-04-02T13:52:54.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.954+0000 7f38c9c3b640 1 --2- 172.21.15.52:0/4123021898 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559ab2f8d170 0x559ab2f8be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:54.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.955+0000 7f38c9c3b640 1 --2- 172.21.15.52:0/4123021898 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559ab2f8d170 0x559ab2f8be80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ddf2b085d87f0ce3 server_cookie=1c640f2468ac6cd3 in_seq=0 out_seq=0 2024-04-02T13:52:54.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.955+0000 7f38adffb640 10 client.24950 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:52:54.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.955+0000 7f38adffb640 1 -- 172.21.15.52:0/4123021898 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559ab2e7c5a0 con 0x559ab2f8d170 2024-04-02T13:52:54.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.955+0000 7f38adffb640 10 client.24950 handle_command_reply: tid=0 2024-04-02T13:52:54.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.955+0000 7f38cf285740 1 client.24950 shutdown 2024-04-02T13:52:54.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.955+0000 7f38cf285740 1 -- 172.21.15.52:0/4123021898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559ab2f0b4d0 msgr2=0x559ab2f18c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:54.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.955+0000 7f38cf285740 1 --2- 172.21.15.52:0/4123021898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559ab2f0b4d0 0x559ab2f18c80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x559ab2f22fd0 tx=0x7f38c002c5d0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.955+0000 7f38cf285740 1 -- 172.21.15.52:0/4123021898 shutdown_connections 2024-04-02T13:52:54.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.955+0000 7f38cf285740 1 --2- 172.21.15.52:0/4123021898 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559ab2f10640 0x559ab2f1bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.955+0000 7f38cf285740 1 --2- 172.21.15.52:0/4123021898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559ab2f0b4d0 0x559ab2f18c80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.955+0000 7f38cf285740 1 --2- 172.21.15.52:0/4123021898 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559ab2f8d170 0x559ab2f8be80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.955+0000 7f38cf285740 1 --2- 172.21.15.52:0/4123021898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ab2f0aae0 0x559ab2f18740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.955+0000 7f38adffb640 0 client.24950 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:52:54.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.955+0000 7f38cf285740 1 -- 172.21.15.52:0/4123021898 >> 172.21.15.52:0/4123021898 conn(0x559ab2f06050 msgr2=0x559ab2f0bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:54.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.955+0000 7f38cf285740 1 -- 172.21.15.52:0/4123021898 shutdown_connections 2024-04-02T13:52:54.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.955+0000 7f38cf285740 1 -- 172.21.15.52:0/4123021898 wait complete. 2024-04-02T13:52:54.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.955+0000 7f38cf285740 20 client.24950 trim_cache size 0 max 16384 2024-04-02T13:52:54.965 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:52:54.966 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "bfa00e6e-f0f6-11ee-b647-cb9ed24678a4", 2024-04-02T13:52:54.966 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 4, 2024-04-02T13:52:54.966 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24457, 2024-04-02T13:52:54.966 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-04-02T13:52:54.966 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-04-02T13:52:54.966 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-04-02T13:52:54.966 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 86.614324568000001, 2024-04-02T13:52:54.966 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 27, 2024-04-02T13:52:54.966 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 92, 2024-04-02T13:52:54.966 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:52:54.966 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 136.754514132 2024-04-02T13:52:54.966 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:52:54.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.956+0000 7f38cc6a0640 1 -- 172.21.15.52:0/2674962623 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f38b00a3c40 msgr2=0x7f38b00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:54.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.956+0000 7f38cc6a0640 1 --2- 172.21.15.52:0/2674962623 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f38b00a3c40 0x7f38b00a6100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0a4b10 tx=0x7f38bc06df10 comp rx=0 tx=0).stop 2024-04-02T13:52:54.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.956+0000 7f38cc6a0640 1 -- 172.21.15.52:0/2674962623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38c4107880 msgr2=0x7f38c419f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:54.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.956+0000 7f38cc6a0640 1 --2- 172.21.15.52:0/2674962623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38c4107880 0x7f38c419f450 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009fd0 tx=0x7f38bc00b260 comp rx=0 tx=0).stop 2024-04-02T13:52:54.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.956+0000 7f38cc6a0640 1 -- 172.21.15.52:0/2674962623 shutdown_connections 2024-04-02T13:52:54.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.956+0000 7f38cc6a0640 1 --2- 172.21.15.52:0/2674962623 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f38b00a3c40 0x7f38b00a6100 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.956+0000 7f38cc6a0640 1 --2- 172.21.15.52:0/2674962623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38c4107880 0x7f38c419f450 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.956+0000 7f38cc6a0640 1 --2- 172.21.15.52:0/2674962623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38c4104f50 0x7f38c41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.956+0000 7f38cc6a0640 1 --2- 172.21.15.52:0/2674962623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c4102620 0x7f38c4197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:54.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.956+0000 7f38cc6a0640 1 -- 172.21.15.52:0/2674962623 >> 172.21.15.52:0/2674962623 conn(0x7f38c40fbf90 msgr2=0x7f38c4103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:54.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.956+0000 7f38cc6a0640 1 -- 172.21.15.52:0/2674962623 shutdown_connections 2024-04-02T13:52:54.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:54.956+0000 7f38cc6a0640 1 -- 172.21.15.52:0/2674962623 wait complete. 2024-04-02T13:52:54.970 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-02T13:52:55.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.071+0000 7fd36c160640 1 -- 172.21.15.52:0/2736827876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3641025b0 msgr2=0x7fd364108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.071+0000 7fd36c160640 1 --2- 172.21.15.52:0/2736827876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3641025b0 0x7fd364108e40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009ca0 tx=0x7fd35c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.071+0000 7fd36c160640 1 -- 172.21.15.52:0/2736827876 shutdown_connections 2024-04-02T13:52:55.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.071+0000 7fd36c160640 1 --2- 172.21.15.52:0/2736827876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3641025b0 0x7fd364108e40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.071+0000 7fd36c160640 1 --2- 172.21.15.52:0/2736827876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd364101bf0 0x7fd364102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.071+0000 7fd36c160640 1 --2- 172.21.15.52:0/2736827876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd364100a00 0x7fd364100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.071+0000 7fd36c160640 1 -- 172.21.15.52:0/2736827876 >> 172.21.15.52:0/2736827876 conn(0x7fd3640fbf90 msgr2=0x7fd3640fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:55.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.071+0000 7fd36c160640 1 -- 172.21.15.52:0/2736827876 shutdown_connections 2024-04-02T13:52:55.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.071+0000 7fd36c160640 1 -- 172.21.15.52:0/2736827876 wait complete. 2024-04-02T13:52:55.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.072+0000 7fd36c160640 1 Processor -- start 2024-04-02T13:52:55.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.072+0000 7fd36c160640 1 -- start start 2024-04-02T13:52:55.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.072+0000 7fd36c160640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd364100a00 0x7fd364104990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.072+0000 7fd36c160640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd364101bf0 0x7fd364102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.072+0000 7fd36c160640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3641025b0 0x7fd364103510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.072+0000 7fd36c160640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd36410c680 con 0x7fd3641025b0 2024-04-02T13:52:55.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.072+0000 7fd36c160640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd36410c500 con 0x7fd364100a00 2024-04-02T13:52:55.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.072+0000 7fd36c160640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd36410c800 con 0x7fd364101bf0 2024-04-02T13:52:55.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.072+0000 7fd36a6fd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3641025b0 0x7fd364103510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.072+0000 7fd36a6fd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3641025b0 0x7fd364103510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48640/0 (socket says 172.21.15.52:48640) 2024-04-02T13:52:55.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.072+0000 7fd36a6fd640 1 -- 172.21.15.52:0/3802146609 learned_addr learned my addr 172.21.15.52:0/3802146609 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:55.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.072+0000 7fd3696fb640 1 --2- 172.21.15.52:0/3802146609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd364101bf0 0x7fd364102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.072+0000 7fd369efc640 1 --2- 172.21.15.52:0/3802146609 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd364100a00 0x7fd364104990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.072+0000 7fd36a6fd640 1 -- 172.21.15.52:0/3802146609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd364101bf0 msgr2=0x7fd364102fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.072+0000 7fd36a6fd640 1 --2- 172.21.15.52:0/3802146609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd364101bf0 0x7fd364102fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.072+0000 7fd36a6fd640 1 -- 172.21.15.52:0/3802146609 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd364100a00 msgr2=0x7fd364104990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.072+0000 7fd36a6fd640 1 --2- 172.21.15.52:0/3802146609 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd364100a00 0x7fd364104990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.072+0000 7fd36a6fd640 1 -- 172.21.15.52:0/3802146609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd364103da0 con 0x7fd3641025b0 2024-04-02T13:52:55.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.072+0000 7fd369efc640 1 --2- 172.21.15.52:0/3802146609 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd364100a00 0x7fd364104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:52:55.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.073+0000 7fd36a6fd640 1 --2- 172.21.15.52:0/3802146609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3641025b0 0x7fd364103510 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd35c02c700 tx=0x7fd35c032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:55.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.073+0000 7fd356ffd640 1 -- 172.21.15.52:0/3802146609 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c039040 con 0x7fd3641025b0 2024-04-02T13:52:55.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.073+0000 7fd36c160640 1 -- 172.21.15.52:0/3802146609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3641a1ee0 con 0x7fd3641025b0 2024-04-02T13:52:55.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.073+0000 7fd356ffd640 1 -- 172.21.15.52:0/3802146609 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd35c044070 con 0x7fd3641025b0 2024-04-02T13:52:55.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.073+0000 7fd356ffd640 1 -- 172.21.15.52:0/3802146609 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c0042a0 con 0x7fd3641025b0 2024-04-02T13:52:55.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.073+0000 7fd36c160640 1 -- 172.21.15.52:0/3802146609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3641a2340 con 0x7fd3641025b0 2024-04-02T13:52:55.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.076+0000 7fd356ffd640 1 -- 172.21.15.52:0/3802146609 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd35c003970 con 0x7fd3641025b0 2024-04-02T13:52:55.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.076+0000 7fd36ee27740 1 Processor -- start 2024-04-02T13:52:55.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.076+0000 7fd36ee27740 1 -- start start 2024-04-02T13:52:55.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.076+0000 7fd36ee27740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564020bb0ae0 0x564020bb0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.076+0000 7fd36ee27740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564020bb14d0 0x564020bb5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.076+0000 7fd36ee27740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb6640 0x564020bb8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.076+0000 7fd36ee27740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564020b15060 con 0x564020bb6640 2024-04-02T13:52:55.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.076+0000 7fd36ee27740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564020abd650 con 0x564020bb0ae0 2024-04-02T13:52:55.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.076+0000 7fd36ee27740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564020b11010 con 0x564020bb14d0 2024-04-02T13:52:55.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.076+0000 7fd3696fb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb6640 0x564020bb8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.076+0000 7fd3696fb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb6640 0x564020bb8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48654/0 (socket says 172.21.15.52:48654) 2024-04-02T13:52:55.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.076+0000 7fd3696fb640 1 -- 172.21.15.52:0/2375034 learned_addr learned my addr 172.21.15.52:0/2375034 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:55.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.076+0000 7fd369efc640 1 --2- 172.21.15.52:0/2375034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564020bb14d0 0x564020bb5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.076+0000 7fd356ffd640 1 --2- 172.21.15.52:0/3802146609 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fd3500a3c40 0x7fd3500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.076+0000 7fd3696fb640 1 --2- 172.21.15.52:0/2375034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564020bb0ae0 0x564020bb0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.077+0000 7fd3696fb640 1 -- 172.21.15.52:0/2375034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564020bb14d0 msgr2=0x564020bb5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.077+0000 7fd36a6fd640 1 --2- 172.21.15.52:0/3802146609 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fd3500a3c40 0x7fd3500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.077+0000 7fd3696fb640 1 --2- 172.21.15.52:0/2375034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564020bb14d0 0x564020bb5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.077+0000 7fd3696fb640 1 -- 172.21.15.52:0/2375034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564020bb0ae0 msgr2=0x564020bb0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.077+0000 7fd3696fb640 1 --2- 172.21.15.52:0/2375034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564020bb0ae0 0x564020bb0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.077+0000 7fd3696fb640 1 -- 172.21.15.52:0/2375034 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640208292d0 con 0x564020bb6640 2024-04-02T13:52:55.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.077+0000 7fd356ffd640 1 -- 172.21.15.52:0/3802146609 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd35c0f2c90 con 0x7fd3641025b0 2024-04-02T13:52:55.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.077+0000 7fd3696fb640 1 --2- 172.21.15.52:0/2375034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb6640 0x564020bb8ad0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd36000b6c0 tx=0x7fd36000b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcada5fb66258285 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:55.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.077+0000 7fd34ffff640 1 -- 172.21.15.52:0/2375034 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd360004220 con 0x564020bb6640 2024-04-02T13:52:55.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.077+0000 7fd36a6fd640 1 --2- 172.21.15.52:0/3802146609 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fd3500a3c40 0x7fd3500a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0be950 tx=0x7fd35c089740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:55.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.077+0000 7fd34ffff640 1 -- 172.21.15.52:0/2375034 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3600043c0 con 0x564020bb6640 2024-04-02T13:52:55.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.077+0000 7fd34ffff640 1 -- 172.21.15.52:0/2375034 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd360010860 con 0x564020bb6640 2024-04-02T13:52:55.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.078+0000 7fd36ee27740 1 -- 172.21.15.52:0/2375034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb6640 msgr2=0x564020bb8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.078+0000 7fd36ee27740 1 --2- 172.21.15.52:0/2375034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb6640 0x564020bb8ad0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd36000b6c0 tx=0x7fd36000b650 comp rx=0 tx=0).stop 2024-04-02T13:52:55.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.078+0000 7fd36ee27740 1 -- 172.21.15.52:0/2375034 shutdown_connections 2024-04-02T13:52:55.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.078+0000 7fd36ee27740 1 --2- 172.21.15.52:0/2375034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb6640 0x564020bb8ad0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.078+0000 7fd36ee27740 1 --2- 172.21.15.52:0/2375034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564020bb14d0 0x564020bb5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.078+0000 7fd36ee27740 1 --2- 172.21.15.52:0/2375034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564020bb0ae0 0x564020bb0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.078+0000 7fd36ee27740 1 -- 172.21.15.52:0/2375034 >> 172.21.15.52:0/2375034 conn(0x564020bac050 msgr2=0x564020bac450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:55.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.078+0000 7fd36ee27740 1 -- 172.21.15.52:0/2375034 shutdown_connections 2024-04-02T13:52:55.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.078+0000 7fd36ee27740 1 -- 172.21.15.52:0/2375034 wait complete. 2024-04-02T13:52:55.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.078+0000 7fd36ee27740 1 -- start start 2024-04-02T13:52:55.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.079+0000 7fd36ee27740 1 Processor -- start 2024-04-02T13:52:55.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.079+0000 7fd36ee27740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564020bb0ae0 0x564020bbb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.079+0000 7fd36ee27740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb14d0 0x564020bbb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.079+0000 7fd36ee27740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564020bb6640 0x564020bc25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.079+0000 7fd36ee27740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564020bb95f0 con 0x564020bb14d0 2024-04-02T13:52:55.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.079+0000 7fd36ee27740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564020ad0a10 con 0x564020bb6640 2024-04-02T13:52:55.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.079+0000 7fd36ee27740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564020bb9770 con 0x564020bb0ae0 2024-04-02T13:52:55.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.079+0000 7fd36ee27740 10 client.0 _ll_register_callbacks cb 0x564020b0d0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:55.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.079+0000 7fd369efc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb14d0 0x564020bbb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.079+0000 7fd369efc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb14d0 0x564020bbb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48656/0 (socket says 172.21.15.52:48656) 2024-04-02T13:52:55.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.079+0000 7fd369efc640 1 -- 172.21.15.52:0/1615594152 learned_addr learned my addr 172.21.15.52:0/1615594152 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:55.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.079+0000 7fd3696fb640 1 --2- 172.21.15.52:0/1615594152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564020bb0ae0 0x564020bbb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.079+0000 7fd3696fb640 1 --2- 172.21.15.52:0/1615594152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564020bb6640 0x564020bc25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.079+0000 7fd369efc640 1 -- 172.21.15.52:0/1615594152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564020bb0ae0 msgr2=0x564020bbb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.079+0000 7fd369efc640 1 --2- 172.21.15.52:0/1615594152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564020bb0ae0 0x564020bbb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.079+0000 7fd369efc640 1 -- 172.21.15.52:0/1615594152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564020bb6640 msgr2=0x564020bc25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.079+0000 7fd369efc640 1 --2- 172.21.15.52:0/1615594152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564020bb6640 0x564020bc25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.079+0000 7fd369efc640 1 -- 172.21.15.52:0/1615594152 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640208f36b0 con 0x564020bb14d0 2024-04-02T13:52:55.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.079+0000 7fd3696fb640 1 --2- 172.21.15.52:0/1615594152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564020bb6640 0x564020bc25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:52:55.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.080+0000 7fd369efc640 1 --2- 172.21.15.52:0/1615594152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb14d0 0x564020bbb560 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd35800bf00 tx=0x7fd358012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8957365b5440c8c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:55.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.080+0000 7fd34d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:52:55.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.080+0000 7fd34d7fa640 1 -- 172.21.15.52:0/1615594152 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3580035f0 con 0x564020bb14d0 2024-04-02T13:52:55.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.080+0000 7fd34d7fa640 1 -- 172.21.15.52:0/1615594152 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd358003790 con 0x564020bb14d0 2024-04-02T13:52:55.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.080+0000 7fd36ee27740 1 -- 172.21.15.52:0/1615594152 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3600043c0 con 0x564020bb14d0 2024-04-02T13:52:55.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.080+0000 7fd34d7fa640 1 -- 172.21.15.52:0/1615594152 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd358003990 con 0x564020bb14d0 2024-04-02T13:52:55.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.080+0000 7fd36ee27740 10 client.15228 fetch_fsmap learned FSMap version 27 2024-04-02T13:52:55.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.080+0000 7fd36ee27740 1 -- 172.21.15.52:0/1615594152 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564020bc59e0 con 0x564020bb14d0 2024-04-02T13:52:55.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.080+0000 7fd34d7fa640 1 -- 172.21.15.52:0/1615594152 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3600043c0 con 0x564020bb14d0 2024-04-02T13:52:55.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.080+0000 7fd34d7fa640 1 -- 172.21.15.52:0/1615594152 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd358019070 con 0x564020bb14d0 2024-04-02T13:52:55.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.080+0000 7fd36ee27740 10 client.15228 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:52:55.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.080+0000 7fd36ee27740 10 client.15228 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-02T13:52:55.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.080+0000 7fd36ee27740 20 client.15228 populate_metadata read hostname 'smithi052' 2024-04-02T13:52:55.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.080+0000 7fd36ee27740 1 --2- 172.21.15.52:0/1615594152 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x564020bc8100 0x564020bc9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.080+0000 7fd36ee27740 4 client.15228 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:52:55.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.080+0000 7fd36ee27740 1 -- 172.21.15.52:0/1615594152 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564020abccf0 con 0x564020bc8100 2024-04-02T13:52:55.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.081+0000 7fd3696fb640 1 --2- 172.21.15.52:0/1615594152 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x564020bc8100 0x564020bc9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.081+0000 7fd3696fb640 1 --2- 172.21.15.52:0/1615594152 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x564020bc8100 0x564020bc9570 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e45e34cd3843e6a4 server_cookie=2a8f3b2998ae0573 in_seq=0 out_seq=0 2024-04-02T13:52:55.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.081+0000 7fd34d7fa640 10 client.15228 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:52:55.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.082+0000 7fd34d7fa640 1 -- 172.21.15.52:0/1615594152 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564020abccf0 con 0x564020bc8100 2024-04-02T13:52:55.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.082+0000 7fd34d7fa640 10 client.15228 handle_command_reply: tid=0 2024-04-02T13:52:55.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.082+0000 7fd36ee27740 1 client.15228 shutdown 2024-04-02T13:52:55.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.082+0000 7fd36ee27740 1 -- 172.21.15.52:0/1615594152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb14d0 msgr2=0x564020bbb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.082+0000 7fd36ee27740 1 --2- 172.21.15.52:0/1615594152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb14d0 0x564020bbb560 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd35800bf00 tx=0x7fd358012820 comp rx=0 tx=0).stop 2024-04-02T13:52:55.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.082+0000 7fd36ee27740 1 -- 172.21.15.52:0/1615594152 shutdown_connections 2024-04-02T13:52:55.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.082+0000 7fd36ee27740 1 --2- 172.21.15.52:0/1615594152 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x564020bc8100 0x564020bc9570 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.082+0000 7fd36ee27740 1 --2- 172.21.15.52:0/1615594152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564020bb6640 0x564020bc25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.082+0000 7fd36ee27740 1 --2- 172.21.15.52:0/1615594152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb14d0 0x564020bbb560 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.082+0000 7fd34d7fa640 0 client.15228 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:52:55.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.082+0000 7fd36ee27740 1 --2- 172.21.15.52:0/1615594152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564020bb0ae0 0x564020bbb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.082+0000 7fd36ee27740 1 -- 172.21.15.52:0/1615594152 >> 172.21.15.52:0/1615594152 conn(0x564020bac050 msgr2=0x564020bab190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:55.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.082+0000 7fd36ee27740 1 -- 172.21.15.52:0/1615594152 shutdown_connections 2024-04-02T13:52:55.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.082+0000 7fd36ee27740 1 -- 172.21.15.52:0/1615594152 wait complete. 2024-04-02T13:52:55.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.082+0000 7fd36ee27740 20 client.15228 trim_cache size 0 max 16384 2024-04-02T13:52:55.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.119+0000 7fd36ee27740 1 Processor -- start 2024-04-02T13:52:55.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.119+0000 7fd36ee27740 1 -- start start 2024-04-02T13:52:55.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.119+0000 7fd36ee27740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564020bb0ae0 0x564020bc2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.119+0000 7fd36ee27740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb14d0 0x564020bc2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.119+0000 7fd36ee27740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564020bb6640 0x564020baa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.119+0000 7fd36ee27740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564020bc78b0 con 0x564020bb14d0 2024-04-02T13:52:55.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.119+0000 7fd36ee27740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564020bc7f80 con 0x564020bb6640 2024-04-02T13:52:55.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.119+0000 7fd36ee27740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564020bc8650 con 0x564020bb0ae0 2024-04-02T13:52:55.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.119+0000 7fd369efc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb14d0 0x564020bc2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.119+0000 7fd369efc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb14d0 0x564020bc2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48660/0 (socket says 172.21.15.52:48660) 2024-04-02T13:52:55.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.119+0000 7fd369efc640 1 -- 172.21.15.52:0/600987474 learned_addr learned my addr 172.21.15.52:0/600987474 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:55.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.119+0000 7fd3696fb640 1 --2- 172.21.15.52:0/600987474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564020bb0ae0 0x564020bc2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.119+0000 7fd369efc640 1 -- 172.21.15.52:0/600987474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564020bb0ae0 msgr2=0x564020bc2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.119+0000 7fd369efc640 1 --2- 172.21.15.52:0/600987474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564020bb0ae0 0x564020bc2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.119+0000 7fd369efc640 1 -- 172.21.15.52:0/600987474 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564020bb6640 msgr2=0x564020baa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.119+0000 7fd369efc640 1 --2- 172.21.15.52:0/600987474 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564020bb6640 0x564020baa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.119+0000 7fd369efc640 1 -- 172.21.15.52:0/600987474 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564020bb18b0 con 0x564020bb14d0 2024-04-02T13:52:55.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.119+0000 7fd369efc640 1 --2- 172.21.15.52:0/600987474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb14d0 0x564020bc2560 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd358006010 tx=0x7fd35800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47a6f5c9263f0ad server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:55.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.120+0000 7fd34f7fe640 1 -- 172.21.15.52:0/600987474 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3580044c0 con 0x564020bb14d0 2024-04-02T13:52:55.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.120+0000 7fd34f7fe640 1 -- 172.21.15.52:0/600987474 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd35802e070 con 0x564020bb14d0 2024-04-02T13:52:55.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.120+0000 7fd34f7fe640 1 -- 172.21.15.52:0/600987474 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358005030 con 0x564020bb14d0 2024-04-02T13:52:55.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.121+0000 7fd36ee27740 1 -- 172.21.15.52:0/600987474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb14d0 msgr2=0x564020bc2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.121+0000 7fd36ee27740 1 --2- 172.21.15.52:0/600987474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb14d0 0x564020bc2560 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd358006010 tx=0x7fd35800b840 comp rx=0 tx=0).stop 2024-04-02T13:52:55.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.121+0000 7fd36ee27740 1 -- 172.21.15.52:0/600987474 shutdown_connections 2024-04-02T13:52:55.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.121+0000 7fd36ee27740 1 --2- 172.21.15.52:0/600987474 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564020bb6640 0x564020baa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.121+0000 7fd36ee27740 1 --2- 172.21.15.52:0/600987474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb14d0 0x564020bc2560 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.121+0000 7fd36ee27740 1 --2- 172.21.15.52:0/600987474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564020bb0ae0 0x564020bc2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.121+0000 7fd36ee27740 1 -- 172.21.15.52:0/600987474 >> 172.21.15.52:0/600987474 conn(0x564020bac050 msgr2=0x564020bb29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:55.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.121+0000 7fd36ee27740 1 -- 172.21.15.52:0/600987474 shutdown_connections 2024-04-02T13:52:55.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.121+0000 7fd36ee27740 1 -- 172.21.15.52:0/600987474 wait complete. 2024-04-02T13:52:55.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.121+0000 7fd36ee27740 1 -- start start 2024-04-02T13:52:55.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.121+0000 7fd36ee27740 1 Processor -- start 2024-04-02T13:52:55.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.121+0000 7fd36ee27740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564020bb0ae0 0x564020bbdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.121+0000 7fd36ee27740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb14d0 0x564020bbe500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.122+0000 7fd36ee27740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564020bb6640 0x564020bc14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.122+0000 7fd36ee27740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564020bc93f0 con 0x564020bb14d0 2024-04-02T13:52:55.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.122+0000 7fd36ee27740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564020bc8d20 con 0x564020bb6640 2024-04-02T13:52:55.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.122+0000 7fd36ee27740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564020bc9ac0 con 0x564020bb0ae0 2024-04-02T13:52:55.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.122+0000 7fd36ee27740 10 client.0 _ll_register_callbacks cb 0x7fd35800b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:55.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.122+0000 7fd369efc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb14d0 0x564020bbe500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.122+0000 7fd369efc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb14d0 0x564020bbe500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48676/0 (socket says 172.21.15.52:48676) 2024-04-02T13:52:55.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.122+0000 7fd369efc640 1 -- 172.21.15.52:0/223510034 learned_addr learned my addr 172.21.15.52:0/223510034 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:55.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.122+0000 7fd3696fb640 1 --2- 172.21.15.52:0/223510034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564020bb0ae0 0x564020bbdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.122+0000 7fd369efc640 1 -- 172.21.15.52:0/223510034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564020bb0ae0 msgr2=0x564020bbdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.122+0000 7fd369efc640 1 --2- 172.21.15.52:0/223510034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564020bb0ae0 0x564020bbdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.122+0000 7fd369efc640 1 -- 172.21.15.52:0/223510034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564020bb6640 msgr2=0x564020bc14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.122+0000 7fd3696fb640 1 --2- 172.21.15.52:0/223510034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564020bb6640 0x564020bc14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.122+0000 7fd369efc640 1 --2- 172.21.15.52:0/223510034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564020bb6640 0x564020bc14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.122+0000 7fd369efc640 1 -- 172.21.15.52:0/223510034 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564020bc1aa0 con 0x564020bb14d0 2024-04-02T13:52:55.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.122+0000 7fd369efc640 1 --2- 172.21.15.52:0/223510034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb14d0 0x564020bbe500 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd358005fe0 tx=0x7fd358003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774f347004bc3b00 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:55.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.122+0000 7fd34d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:52:55.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.122+0000 7fd34d7fa640 1 -- 172.21.15.52:0/223510034 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35801b8a0 con 0x564020bb14d0 2024-04-02T13:52:55.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.122+0000 7fd34d7fa640 1 -- 172.21.15.52:0/223510034 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd358010490 con 0x564020bb14d0 2024-04-02T13:52:55.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.122+0000 7fd36ee27740 1 -- 172.21.15.52:0/223510034 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd35802e070 con 0x564020bb14d0 2024-04-02T13:52:55.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.122+0000 7fd34d7fa640 1 -- 172.21.15.52:0/223510034 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd358040050 con 0x564020bb14d0 2024-04-02T13:52:55.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.122+0000 7fd36ee27740 10 client.15234 fetch_fsmap learned FSMap version 27 2024-04-02T13:52:55.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.122+0000 7fd36ee27740 1 -- 172.21.15.52:0/223510034 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564020c2e2c0 con 0x564020bb14d0 2024-04-02T13:52:55.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.123+0000 7fd34d7fa640 1 -- 172.21.15.52:0/223510034 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd35802e070 con 0x564020bb14d0 2024-04-02T13:52:55.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.123+0000 7fd34d7fa640 1 -- 172.21.15.52:0/223510034 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd358019070 con 0x564020bb14d0 2024-04-02T13:52:55.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.123+0000 7fd36ee27740 10 client.15234 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:52:55.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.123+0000 7fd36ee27740 10 client.15234 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-02T13:52:55.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.123+0000 7fd36ee27740 20 client.15234 populate_metadata read hostname 'smithi052' 2024-04-02T13:52:55.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.123+0000 7fd36ee27740 1 --2- 172.21.15.52:0/223510034 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x564020c32ab0 0x564020c317c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.123+0000 7fd36ee27740 4 client.15234 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:52:55.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.123+0000 7fd36ee27740 1 -- 172.21.15.52:0/223510034 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "status"}) -- 0x564020b225a0 con 0x564020c32ab0 2024-04-02T13:52:55.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.123+0000 7fd3696fb640 1 --2- 172.21.15.52:0/223510034 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x564020c32ab0 0x564020c317c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.123+0000 7fd3696fb640 1 --2- 172.21.15.52:0/223510034 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x564020c32ab0 0x564020c317c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=57845bbf8736e439 server_cookie=eab3b39eccdfca32 in_seq=0 out_seq=0 2024-04-02T13:52:55.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.123+0000 7fd34d7fa640 10 client.15234 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:52:55.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.124+0000 7fd34d7fa640 1 -- 172.21.15.52:0/223510034 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x564020b225a0 con 0x564020c32ab0 2024-04-02T13:52:55.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.124+0000 7fd34d7fa640 10 client.15234 handle_command_reply: tid=0 2024-04-02T13:52:55.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.124+0000 7fd36ee27740 1 client.15234 shutdown 2024-04-02T13:52:55.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.124+0000 7fd36ee27740 1 -- 172.21.15.52:0/223510034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb14d0 msgr2=0x564020bbe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.124+0000 7fd36ee27740 1 --2- 172.21.15.52:0/223510034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb14d0 0x564020bbe500 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd358005fe0 tx=0x7fd358003de0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.124+0000 7fd36ee27740 1 -- 172.21.15.52:0/223510034 shutdown_connections 2024-04-02T13:52:55.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.124+0000 7fd36ee27740 1 --2- 172.21.15.52:0/223510034 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x564020c32ab0 0x564020c317c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.124+0000 7fd36ee27740 1 --2- 172.21.15.52:0/223510034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564020bb6640 0x564020bc14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.124+0000 7fd36ee27740 1 --2- 172.21.15.52:0/223510034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564020bb14d0 0x564020bbe500 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.124+0000 7fd36ee27740 1 --2- 172.21.15.52:0/223510034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564020bb0ae0 0x564020bbdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.124+0000 7fd34d7fa640 0 client.15234 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:52:55.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.124+0000 7fd36ee27740 1 -- 172.21.15.52:0/223510034 >> 172.21.15.52:0/223510034 conn(0x564020bac050 msgr2=0x564020baf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:55.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.124+0000 7fd36ee27740 1 -- 172.21.15.52:0/223510034 shutdown_connections 2024-04-02T13:52:55.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.124+0000 7fd36ee27740 1 -- 172.21.15.52:0/223510034 wait complete. 2024-04-02T13:52:55.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.124+0000 7fd36ee27740 20 client.15234 trim_cache size 0 max 16384 2024-04-02T13:52:55.134 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:52:55.134 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "bfa00e6e-f0f6-11ee-b647-cb9ed24678a4", 2024-04-02T13:52:55.134 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 3, 2024-04-02T13:52:55.134 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24467, 2024-04-02T13:52:55.134 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-04-02T13:52:55.134 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-04-02T13:52:55.134 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-04-02T13:52:55.134 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 87.790136851, 2024-04-02T13:52:55.135 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 27, 2024-04-02T13:52:55.135 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 92, 2024-04-02T13:52:55.135 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:52:55.135 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 131.286010291 2024-04-02T13:52:55.135 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:52:55.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.124+0000 7fd36c160640 1 -- 172.21.15.52:0/3802146609 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fd3500a3c40 msgr2=0x7fd3500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.124+0000 7fd36c160640 1 --2- 172.21.15.52:0/3802146609 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fd3500a3c40 0x7fd3500a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0be950 tx=0x7fd35c089740 comp rx=0 tx=0).stop 2024-04-02T13:52:55.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.124+0000 7fd36c160640 1 -- 172.21.15.52:0/3802146609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3641025b0 msgr2=0x7fd364103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.124+0000 7fd36c160640 1 --2- 172.21.15.52:0/3802146609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3641025b0 0x7fd364103510 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd35c02c700 tx=0x7fd35c032f20 comp rx=0 tx=0).stop 2024-04-02T13:52:55.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.125+0000 7fd36c160640 1 -- 172.21.15.52:0/3802146609 shutdown_connections 2024-04-02T13:52:55.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.125+0000 7fd36c160640 1 --2- 172.21.15.52:0/3802146609 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fd3500a3c40 0x7fd3500a6100 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.125+0000 7fd36c160640 1 --2- 172.21.15.52:0/3802146609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3641025b0 0x7fd364103510 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.125+0000 7fd36c160640 1 --2- 172.21.15.52:0/3802146609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd364101bf0 0x7fd364102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.125+0000 7fd36c160640 1 --2- 172.21.15.52:0/3802146609 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd364100a00 0x7fd364104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.125+0000 7fd36c160640 1 -- 172.21.15.52:0/3802146609 >> 172.21.15.52:0/3802146609 conn(0x7fd3640fbf90 msgr2=0x7fd3640fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:55.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.125+0000 7fd36c160640 1 -- 172.21.15.52:0/3802146609 shutdown_connections 2024-04-02T13:52:55.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.125+0000 7fd36c160640 1 -- 172.21.15.52:0/3802146609 wait complete. 2024-04-02T13:52:55.138 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-02T13:52:55.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.239+0000 7fab0d5ba640 1 -- 172.21.15.52:0/2099477121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab08107880 msgr2=0x7fab08109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.239+0000 7fab0d5ba640 1 --2- 172.21.15.52:0/2099477121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab08107880 0x7fab08109c70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fab00009ca0 tx=0x7fab0002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.239+0000 7fab0d5ba640 1 -- 172.21.15.52:0/2099477121 shutdown_connections 2024-04-02T13:52:55.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.239+0000 7fab0d5ba640 1 --2- 172.21.15.52:0/2099477121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab08107880 0x7fab08109c70 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.239+0000 7fab0d5ba640 1 --2- 172.21.15.52:0/2099477121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab08104f50 0x7fab08107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.239+0000 7fab0d5ba640 1 --2- 172.21.15.52:0/2099477121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab08102620 0x7fab08104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.239+0000 7fab0d5ba640 1 -- 172.21.15.52:0/2099477121 >> 172.21.15.52:0/2099477121 conn(0x7fab080fbf90 msgr2=0x7fab080fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:55.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.239+0000 7fab0d5ba640 1 -- 172.21.15.52:0/2099477121 shutdown_connections 2024-04-02T13:52:55.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.239+0000 7fab0d5ba640 1 -- 172.21.15.52:0/2099477121 wait complete. 2024-04-02T13:52:55.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.240+0000 7fab0d5ba640 1 Processor -- start 2024-04-02T13:52:55.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.240+0000 7fab0d5ba640 1 -- start start 2024-04-02T13:52:55.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.240+0000 7fab0d5ba640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab08102620 0x7fab08197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.240+0000 7fab0d5ba640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab08104f50 0x7fab08198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.240+0000 7fab0d5ba640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab08107880 0x7fab0819f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.240+0000 7fab0d5ba640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fab0810d780 con 0x7fab08104f50 2024-04-02T13:52:55.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.240+0000 7fab0d5ba640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fab0810d600 con 0x7fab08107880 2024-04-02T13:52:55.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.240+0000 7fab0d5ba640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fab0810d900 con 0x7fab08102620 2024-04-02T13:52:55.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.240+0000 7fab067fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab08104f50 0x7fab08198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.240+0000 7fab067fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab08104f50 0x7fab08198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48706/0 (socket says 172.21.15.52:48706) 2024-04-02T13:52:55.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.240+0000 7fab067fc640 1 -- 172.21.15.52:0/402631271 learned_addr learned my addr 172.21.15.52:0/402631271 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:55.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.240+0000 7fab077fe640 1 --2- 172.21.15.52:0/402631271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab08107880 0x7fab0819f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.240+0000 7fab06ffd640 1 --2- 172.21.15.52:0/402631271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab08102620 0x7fab08197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.240+0000 7fab067fc640 1 -- 172.21.15.52:0/402631271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab08102620 msgr2=0x7fab08197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.240+0000 7fab067fc640 1 --2- 172.21.15.52:0/402631271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab08102620 0x7fab08197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.240+0000 7fab067fc640 1 -- 172.21.15.52:0/402631271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab08107880 msgr2=0x7fab0819f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.240+0000 7fab067fc640 1 --2- 172.21.15.52:0/402631271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab08107880 0x7fab0819f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.240+0000 7fab067fc640 1 -- 172.21.15.52:0/402631271 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0819f9e0 con 0x7fab08104f50 2024-04-02T13:52:55.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.241+0000 7fab067fc640 1 --2- 172.21.15.52:0/402631271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab08104f50 0x7fab08198300 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7faafc005fd0 tx=0x7faafc00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:55.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.241+0000 7faaeffff640 1 -- 172.21.15.52:0/402631271 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc017070 con 0x7fab08104f50 2024-04-02T13:52:55.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.241+0000 7faaeffff640 1 -- 172.21.15.52:0/402631271 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faafc009c30 con 0x7fab08104f50 2024-04-02T13:52:55.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.241+0000 7fab0d5ba640 1 -- 172.21.15.52:0/402631271 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0819fcd0 con 0x7fab08104f50 2024-04-02T13:52:55.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.241+0000 7faaeffff640 1 -- 172.21.15.52:0/402631271 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc010460 con 0x7fab08104f50 2024-04-02T13:52:55.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.241+0000 7fab0d5ba640 1 -- 172.21.15.52:0/402631271 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab081a01d0 con 0x7fab08104f50 2024-04-02T13:52:55.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.243+0000 7faaeffff640 1 -- 172.21.15.52:0/402631271 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7faafc010600 con 0x7fab08104f50 2024-04-02T13:52:55.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.243+0000 7fab101a6740 1 Processor -- start 2024-04-02T13:52:55.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.243+0000 7fab101a6740 1 -- start start 2024-04-02T13:52:55.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.243+0000 7fab101a6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff494ef0 0x5570ff495310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.243+0000 7fab101a6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570ff4958e0 0x5570ff49a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.243+0000 7fab101a6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5570ff49aa50 0x5570ff49cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.243+0000 7fab101a6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5570ff3f5060 con 0x5570ff494ef0 2024-04-02T13:52:55.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.243+0000 7fab101a6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5570ff39d650 con 0x5570ff49aa50 2024-04-02T13:52:55.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.243+0000 7fab101a6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5570ff3f1010 con 0x5570ff4958e0 2024-04-02T13:52:55.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.244+0000 7fab077fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff494ef0 0x5570ff495310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.244+0000 7faaeffff640 1 --2- 172.21.15.52:0/402631271 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faaf00a3c40 0x7faaf00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.244+0000 7fab077fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff494ef0 0x5570ff495310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48718/0 (socket says 172.21.15.52:48718) 2024-04-02T13:52:55.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.244+0000 7fab077fe640 1 -- 172.21.15.52:0/3408638888 learned_addr learned my addr 172.21.15.52:0/3408638888 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:55.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.244+0000 7fab06ffd640 1 --2- 172.21.15.52:0/3408638888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570ff4958e0 0x5570ff49a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.244+0000 7fab067fc640 1 --2- 172.21.15.52:0/3408638888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5570ff49aa50 0x5570ff49cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.244+0000 7faaeffff640 1 -- 172.21.15.52:0/402631271 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7faafc0d0660 con 0x7fab08104f50 2024-04-02T13:52:55.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.244+0000 7fab077fe640 1 --2- 172.21.15.52:0/402631271 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faaf00a3c40 0x7faaf00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.244+0000 7fab077fe640 1 -- 172.21.15.52:0/3408638888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570ff4958e0 msgr2=0x5570ff49a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.244+0000 7fab077fe640 1 --2- 172.21.15.52:0/3408638888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570ff4958e0 0x5570ff49a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.244+0000 7fab077fe640 1 -- 172.21.15.52:0/3408638888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5570ff49aa50 msgr2=0x5570ff49cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.244+0000 7fab077fe640 1 --2- 172.21.15.52:0/3408638888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5570ff49aa50 0x5570ff49cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.244+0000 7fab077fe640 1 -- 172.21.15.52:0/3408638888 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570ff1092d0 con 0x5570ff494ef0 2024-04-02T13:52:55.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.244+0000 7fab077fe640 1 --2- 172.21.15.52:0/3408638888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff494ef0 0x5570ff495310 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fab000324c0 tx=0x7fab00032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5504516548e7d5a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:55.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.244+0000 7faaed7fa640 1 -- 172.21.15.52:0/3408638888 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00039040 con 0x5570ff494ef0 2024-04-02T13:52:55.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.244+0000 7faaed7fa640 1 -- 172.21.15.52:0/3408638888 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab00032870 con 0x5570ff494ef0 2024-04-02T13:52:55.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.244+0000 7faaed7fa640 1 -- 172.21.15.52:0/3408638888 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab000346a0 con 0x5570ff494ef0 2024-04-02T13:52:55.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.245+0000 7fab101a6740 1 -- 172.21.15.52:0/3408638888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff494ef0 msgr2=0x5570ff495310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.245+0000 7fab101a6740 1 --2- 172.21.15.52:0/3408638888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff494ef0 0x5570ff495310 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fab000324c0 tx=0x7fab00032e20 comp rx=0 tx=0).stop 2024-04-02T13:52:55.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.245+0000 7fab077fe640 1 --2- 172.21.15.52:0/402631271 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faaf00a3c40 0x7faaf00a6100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fab0004c000 tx=0x7fab00048000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:55.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.245+0000 7fab101a6740 1 -- 172.21.15.52:0/3408638888 shutdown_connections 2024-04-02T13:52:55.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.245+0000 7fab101a6740 1 --2- 172.21.15.52:0/3408638888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5570ff49aa50 0x5570ff49cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.245+0000 7fab101a6740 1 --2- 172.21.15.52:0/3408638888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570ff4958e0 0x5570ff49a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.245+0000 7fab101a6740 1 --2- 172.21.15.52:0/3408638888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff494ef0 0x5570ff495310 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.245+0000 7fab101a6740 1 -- 172.21.15.52:0/3408638888 >> 172.21.15.52:0/3408638888 conn(0x5570ff48c050 msgr2=0x5570ff48c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:55.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.245+0000 7fab101a6740 1 -- 172.21.15.52:0/3408638888 shutdown_connections 2024-04-02T13:52:55.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.245+0000 7fab101a6740 1 -- 172.21.15.52:0/3408638888 wait complete. 2024-04-02T13:52:55.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.245+0000 7fab101a6740 1 -- start start 2024-04-02T13:52:55.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.246+0000 7fab101a6740 1 Processor -- start 2024-04-02T13:52:55.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.246+0000 7fab101a6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5570ff494ef0 0x5570ff49f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.246+0000 7fab101a6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff4958e0 0x5570ff49fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.246+0000 7fab101a6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570ff49aa50 0x5570ff4a69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.246+0000 7fab101a6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5570ff49da00 con 0x5570ff4958e0 2024-04-02T13:52:55.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.246+0000 7fab101a6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5570ff3b0a10 con 0x5570ff494ef0 2024-04-02T13:52:55.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.246+0000 7fab101a6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5570ff49db80 con 0x5570ff49aa50 2024-04-02T13:52:55.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.246+0000 7fab101a6740 10 client.0 _ll_register_callbacks cb 0x5570ff3ed0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:55.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.246+0000 7fab067fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff4958e0 0x5570ff49fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.246+0000 7fab067fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff4958e0 0x5570ff49fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48724/0 (socket says 172.21.15.52:48724) 2024-04-02T13:52:55.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.246+0000 7fab067fc640 1 -- 172.21.15.52:0/1211995842 learned_addr learned my addr 172.21.15.52:0/1211995842 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:55.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.246+0000 7fab06ffd640 1 --2- 172.21.15.52:0/1211995842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5570ff494ef0 0x5570ff49f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.246+0000 7fab067fc640 1 -- 172.21.15.52:0/1211995842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570ff49aa50 msgr2=0x5570ff4a69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.246+0000 7fab067fc640 1 --2- 172.21.15.52:0/1211995842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570ff49aa50 0x5570ff4a69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.246+0000 7fab067fc640 1 -- 172.21.15.52:0/1211995842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5570ff494ef0 msgr2=0x5570ff49f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.246+0000 7fab067fc640 1 --2- 172.21.15.52:0/1211995842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5570ff494ef0 0x5570ff49f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.246+0000 7fab067fc640 1 -- 172.21.15.52:0/1211995842 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570ff1d36b0 con 0x5570ff4958e0 2024-04-02T13:52:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.246+0000 7fab06ffd640 1 --2- 172.21.15.52:0/1211995842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5570ff494ef0 0x5570ff49f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:52:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.246+0000 7fab067fc640 1 --2- 172.21.15.52:0/1211995842 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff4958e0 0x5570ff49fa00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faafc052640 tx=0x7faafc017c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea92b30be00aa838 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.246+0000 7faaeaff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:52:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.246+0000 7faaeaff5640 1 -- 172.21.15.52:0/1211995842 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc0b5060 con 0x5570ff4958e0 2024-04-02T13:52:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.247+0000 7faaeaff5640 1 -- 172.21.15.52:0/1211995842 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7faafc098af0 con 0x5570ff4958e0 2024-04-02T13:52:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.247+0000 7fab101a6740 1 -- 172.21.15.52:0/1211995842 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab00032870 con 0x5570ff4958e0 2024-04-02T13:52:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.247+0000 7faaeaff5640 1 -- 172.21.15.52:0/1211995842 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faafc09cb80 con 0x5570ff4958e0 2024-04-02T13:52:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.247+0000 7fab101a6740 10 client.15246 fetch_fsmap learned FSMap version 27 2024-04-02T13:52:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.247+0000 7fab101a6740 1 -- 172.21.15.52:0/1211995842 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5570ff4a9e60 con 0x5570ff4958e0 2024-04-02T13:52:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.247+0000 7faaeaff5640 1 -- 172.21.15.52:0/1211995842 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab00032870 con 0x5570ff4958e0 2024-04-02T13:52:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.247+0000 7faaeaff5640 1 -- 172.21.15.52:0/1211995842 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7faafc017cd0 con 0x5570ff4958e0 2024-04-02T13:52:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.247+0000 7fab101a6740 10 client.15246 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:52:55.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.247+0000 7fab101a6740 10 client.15246 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-02T13:52:55.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.247+0000 7fab101a6740 20 client.15246 populate_metadata read hostname 'smithi052' 2024-04-02T13:52:55.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.247+0000 7fab101a6740 1 --2- 172.21.15.52:0/1211995842 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5570ff4ac570 0x5570ff4ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.247+0000 7fab101a6740 4 client.15246 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:52:55.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.247+0000 7fab101a6740 1 -- 172.21.15.52:0/1211995842 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570ff39ccf0 con 0x5570ff4ac570 2024-04-02T13:52:55.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.247+0000 7fab06ffd640 1 --2- 172.21.15.52:0/1211995842 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5570ff4ac570 0x5570ff4ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.248+0000 7fab06ffd640 1 --2- 172.21.15.52:0/1211995842 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5570ff4ac570 0x5570ff4ad9e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=648f1a7e4bedd1d0 server_cookie=801bf8aad8576fbc in_seq=0 out_seq=0 2024-04-02T13:52:55.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.248+0000 7faaeaff5640 10 client.15246 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:52:55.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.249+0000 7faaeaff5640 1 -- 172.21.15.52:0/1211995842 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5570ff39ccf0 con 0x5570ff4ac570 2024-04-02T13:52:55.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.249+0000 7faaeaff5640 10 client.15246 handle_command_reply: tid=0 2024-04-02T13:52:55.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.249+0000 7fab101a6740 1 client.15246 shutdown 2024-04-02T13:52:55.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.249+0000 7fab101a6740 1 -- 172.21.15.52:0/1211995842 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff4958e0 msgr2=0x5570ff49fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.249+0000 7fab101a6740 1 --2- 172.21.15.52:0/1211995842 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff4958e0 0x5570ff49fa00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faafc052640 tx=0x7faafc017c60 comp rx=0 tx=0).stop 2024-04-02T13:52:55.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.249+0000 7fab101a6740 1 -- 172.21.15.52:0/1211995842 shutdown_connections 2024-04-02T13:52:55.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.249+0000 7fab101a6740 1 --2- 172.21.15.52:0/1211995842 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5570ff4ac570 0x5570ff4ad9e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.249+0000 7fab101a6740 1 --2- 172.21.15.52:0/1211995842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570ff49aa50 0x5570ff4a69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.249+0000 7fab101a6740 1 --2- 172.21.15.52:0/1211995842 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff4958e0 0x5570ff49fa00 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.249+0000 7faaeaff5640 0 client.15246 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:52:55.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.249+0000 7fab101a6740 1 --2- 172.21.15.52:0/1211995842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5570ff494ef0 0x5570ff49f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.249+0000 7fab101a6740 1 -- 172.21.15.52:0/1211995842 >> 172.21.15.52:0/1211995842 conn(0x5570ff48c050 msgr2=0x5570ff48b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:55.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.249+0000 7fab101a6740 1 -- 172.21.15.52:0/1211995842 shutdown_connections 2024-04-02T13:52:55.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.249+0000 7fab101a6740 1 -- 172.21.15.52:0/1211995842 wait complete. 2024-04-02T13:52:55.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.249+0000 7fab101a6740 20 client.15246 trim_cache size 0 max 16384 2024-04-02T13:52:55.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.267+0000 7fab101a6740 1 Processor -- start 2024-04-02T13:52:55.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.267+0000 7fab101a6740 1 -- start start 2024-04-02T13:52:55.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.267+0000 7fab101a6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff494ef0 0x5570ff49f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.267+0000 7fab101a6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570ff4958e0 0x5570ff4a6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.267+0000 7fab101a6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5570ff49aa50 0x5570ff49dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.267+0000 7fab101a6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5570ff4abd20 con 0x5570ff494ef0 2024-04-02T13:52:55.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.267+0000 7fab101a6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5570ff4ac3f0 con 0x5570ff49aa50 2024-04-02T13:52:55.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.267+0000 7fab101a6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5570ff4acac0 con 0x5570ff4958e0 2024-04-02T13:52:55.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.267+0000 7fab06ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff494ef0 0x5570ff49f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.267+0000 7fab06ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff494ef0 0x5570ff49f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48738/0 (socket says 172.21.15.52:48738) 2024-04-02T13:52:55.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.267+0000 7fab06ffd640 1 -- 172.21.15.52:0/1917708677 learned_addr learned my addr 172.21.15.52:0/1917708677 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:55.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.267+0000 7fab077fe640 1 --2- 172.21.15.52:0/1917708677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5570ff49aa50 0x5570ff49dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.267+0000 7fab067fc640 1 --2- 172.21.15.52:0/1917708677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570ff4958e0 0x5570ff4a6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.267+0000 7fab06ffd640 1 -- 172.21.15.52:0/1917708677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570ff4958e0 msgr2=0x5570ff4a6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.267+0000 7fab06ffd640 1 --2- 172.21.15.52:0/1917708677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570ff4958e0 0x5570ff4a6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.267+0000 7fab06ffd640 1 -- 172.21.15.52:0/1917708677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5570ff49aa50 msgr2=0x5570ff49dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.267+0000 7fab06ffd640 1 --2- 172.21.15.52:0/1917708677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5570ff49aa50 0x5570ff49dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.267+0000 7fab06ffd640 1 -- 172.21.15.52:0/1917708677 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570ff49dd40 con 0x5570ff494ef0 2024-04-02T13:52:55.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.268+0000 7fab06ffd640 1 --2- 172.21.15.52:0/1917708677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff494ef0 0x5570ff49f7e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x5570ff4ae570 tx=0x7faaf800c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f65d1c4cdc725dd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:55.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.268+0000 7faaecff9640 1 -- 172.21.15.52:0/1917708677 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8003aa0 con 0x5570ff494ef0 2024-04-02T13:52:55.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.268+0000 7faaecff9640 1 -- 172.21.15.52:0/1917708677 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf801a810 con 0x5570ff494ef0 2024-04-02T13:52:55.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.268+0000 7faaecff9640 1 -- 172.21.15.52:0/1917708677 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8011830 con 0x5570ff494ef0 2024-04-02T13:52:55.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.268+0000 7fab101a6740 1 -- 172.21.15.52:0/1917708677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff494ef0 msgr2=0x5570ff49f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.268+0000 7fab101a6740 1 --2- 172.21.15.52:0/1917708677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff494ef0 0x5570ff49f7e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x5570ff4ae570 tx=0x7faaf800c4b0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.269+0000 7fab101a6740 1 -- 172.21.15.52:0/1917708677 shutdown_connections 2024-04-02T13:52:55.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.269+0000 7fab101a6740 1 --2- 172.21.15.52:0/1917708677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5570ff49aa50 0x5570ff49dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.269+0000 7fab101a6740 1 --2- 172.21.15.52:0/1917708677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570ff4958e0 0x5570ff4a6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.269+0000 7fab101a6740 1 --2- 172.21.15.52:0/1917708677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff494ef0 0x5570ff49f7e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.269+0000 7fab101a6740 1 -- 172.21.15.52:0/1917708677 >> 172.21.15.52:0/1917708677 conn(0x5570ff48c050 msgr2=0x5570ff49f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:55.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.269+0000 7fab101a6740 1 -- 172.21.15.52:0/1917708677 shutdown_connections 2024-04-02T13:52:55.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.269+0000 7fab101a6740 1 -- 172.21.15.52:0/1917708677 wait complete. 2024-04-02T13:52:55.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.269+0000 7fab101a6740 1 -- start start 2024-04-02T13:52:55.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.269+0000 7fab101a6740 1 Processor -- start 2024-04-02T13:52:55.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.269+0000 7fab101a6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff494ef0 0x5570ff4a2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.269+0000 7fab101a6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5570ff4958e0 0x5570ff4a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.269+0000 7fab101a6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570ff49aa50 0x5570ff4a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.269+0000 7fab101a6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5570ff4ad860 con 0x5570ff494ef0 2024-04-02T13:52:55.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.269+0000 7fab101a6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5570ff4ad190 con 0x5570ff4958e0 2024-04-02T13:52:55.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.269+0000 7fab101a6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5570ff4adf30 con 0x5570ff49aa50 2024-04-02T13:52:55.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.269+0000 7fab101a6740 10 client.0 _ll_register_callbacks cb 0x7faafc018f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:55.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.269+0000 7fab06ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff494ef0 0x5570ff4a2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.269+0000 7fab06ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff494ef0 0x5570ff4a2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48752/0 (socket says 172.21.15.52:48752) 2024-04-02T13:52:55.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.269+0000 7fab06ffd640 1 -- 172.21.15.52:0/32485799 learned_addr learned my addr 172.21.15.52:0/32485799 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:55.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.270+0000 7fab067fc640 1 --2- 172.21.15.52:0/32485799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5570ff4958e0 0x5570ff4a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.270+0000 7fab077fe640 1 --2- 172.21.15.52:0/32485799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570ff49aa50 0x5570ff4a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.270+0000 7fab06ffd640 1 -- 172.21.15.52:0/32485799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570ff49aa50 msgr2=0x5570ff4a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.270+0000 7fab06ffd640 1 --2- 172.21.15.52:0/32485799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570ff49aa50 0x5570ff4a5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.270+0000 7fab06ffd640 1 -- 172.21.15.52:0/32485799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5570ff4958e0 msgr2=0x5570ff4a2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.270+0000 7fab06ffd640 1 --2- 172.21.15.52:0/32485799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5570ff4958e0 0x5570ff4a2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.270+0000 7fab06ffd640 1 -- 172.21.15.52:0/32485799 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570ff4a6270 con 0x5570ff494ef0 2024-04-02T13:52:55.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.270+0000 7fab06ffd640 1 --2- 172.21.15.52:0/32485799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff494ef0 0x5570ff4a2720 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7faaf8000c00 tx=0x7faaf8005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9334e5609ee1624 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:55.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.270+0000 7faaeaff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:52:55.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.270+0000 7faaeaff5640 1 -- 172.21.15.52:0/32485799 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf801a330 con 0x5570ff494ef0 2024-04-02T13:52:55.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.270+0000 7faaeaff5640 1 -- 172.21.15.52:0/32485799 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7faaf801a4d0 con 0x5570ff494ef0 2024-04-02T13:52:55.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.270+0000 7fab101a6740 1 -- 172.21.15.52:0/32485799 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaf801a810 con 0x5570ff494ef0 2024-04-02T13:52:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.270+0000 7faaeaff5640 1 -- 172.21.15.52:0/32485799 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faaf8036050 con 0x5570ff494ef0 2024-04-02T13:52:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.270+0000 7fab101a6740 10 client.15252 fetch_fsmap learned FSMap version 27 2024-04-02T13:52:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.270+0000 7fab101a6740 1 -- 172.21.15.52:0/32485799 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5570ff512b20 con 0x5570ff494ef0 2024-04-02T13:52:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.270+0000 7faaeaff5640 1 -- 172.21.15.52:0/32485799 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf801a810 con 0x5570ff494ef0 2024-04-02T13:52:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.271+0000 7faaeaff5640 1 -- 172.21.15.52:0/32485799 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7faaf802a030 con 0x5570ff494ef0 2024-04-02T13:52:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.271+0000 7fab101a6740 10 client.15252 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:52:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.271+0000 7fab101a6740 10 client.15252 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-02T13:52:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.271+0000 7fab101a6740 20 client.15252 populate_metadata read hostname 'smithi052' 2024-04-02T13:52:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.271+0000 7fab101a6740 1 --2- 172.21.15.52:0/32485799 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5570ff5175e0 0x5570ff5162f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.271+0000 7fab101a6740 4 client.15252 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:52:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.271+0000 7fab101a6740 1 -- 172.21.15.52:0/32485799 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "status"}) -- 0x5570ff4025a0 con 0x5570ff5175e0 2024-04-02T13:52:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.271+0000 7fab067fc640 1 --2- 172.21.15.52:0/32485799 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5570ff5175e0 0x5570ff5162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.271+0000 7fab067fc640 1 --2- 172.21.15.52:0/32485799 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5570ff5175e0 0x5570ff5162f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4de31894393a2d45 server_cookie=7f982fcfc279df7 in_seq=0 out_seq=0 2024-04-02T13:52:55.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.271+0000 7faaeaff5640 10 client.15252 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:52:55.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.272+0000 7faaeaff5640 1 -- 172.21.15.52:0/32485799 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5570ff4025a0 con 0x5570ff5175e0 2024-04-02T13:52:55.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.272+0000 7faaeaff5640 10 client.15252 handle_command_reply: tid=0 2024-04-02T13:52:55.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.272+0000 7fab101a6740 1 client.15252 shutdown 2024-04-02T13:52:55.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.272+0000 7fab101a6740 1 -- 172.21.15.52:0/32485799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff494ef0 msgr2=0x5570ff4a2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.272+0000 7fab101a6740 1 --2- 172.21.15.52:0/32485799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff494ef0 0x5570ff4a2720 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7faaf8000c00 tx=0x7faaf8005d00 comp rx=0 tx=0).stop 2024-04-02T13:52:55.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.272+0000 7fab101a6740 1 -- 172.21.15.52:0/32485799 shutdown_connections 2024-04-02T13:52:55.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.272+0000 7fab101a6740 1 --2- 172.21.15.52:0/32485799 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5570ff5175e0 0x5570ff5162f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.272+0000 7fab101a6740 1 --2- 172.21.15.52:0/32485799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570ff49aa50 0x5570ff4a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.272+0000 7fab101a6740 1 --2- 172.21.15.52:0/32485799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5570ff4958e0 0x5570ff4a2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.272+0000 7fab101a6740 1 --2- 172.21.15.52:0/32485799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570ff494ef0 0x5570ff4a2720 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.272+0000 7faaeaff5640 0 client.15252 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:52:55.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.272+0000 7fab101a6740 1 -- 172.21.15.52:0/32485799 >> 172.21.15.52:0/32485799 conn(0x5570ff48c050 msgr2=0x5570ff48f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:55.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.272+0000 7fab101a6740 1 -- 172.21.15.52:0/32485799 shutdown_connections 2024-04-02T13:52:55.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.272+0000 7fab101a6740 1 -- 172.21.15.52:0/32485799 wait complete. 2024-04-02T13:52:55.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.272+0000 7fab101a6740 20 client.15252 trim_cache size 0 max 16384 2024-04-02T13:52:55.283 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:52:55.283 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "bfa00e6e-f0f6-11ee-b647-cb9ed24678a4", 2024-04-02T13:52:55.284 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 2, 2024-04-02T13:52:55.284 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24475, 2024-04-02T13:52:55.284 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-04-02T13:52:55.284 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-04-02T13:52:55.284 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-04-02T13:52:55.284 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 88.944710947999994, 2024-04-02T13:52:55.284 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 27, 2024-04-02T13:52:55.284 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 92, 2024-04-02T13:52:55.284 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:52:55.284 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 134.28283585099999 2024-04-02T13:52:55.284 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:52:55.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.272+0000 7fab0d5ba640 1 -- 172.21.15.52:0/402631271 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faaf00a3c40 msgr2=0x7faaf00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.272+0000 7fab0d5ba640 1 --2- 172.21.15.52:0/402631271 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faaf00a3c40 0x7faaf00a6100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fab0004c000 tx=0x7fab00048000 comp rx=0 tx=0).stop 2024-04-02T13:52:55.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.272+0000 7fab0d5ba640 1 -- 172.21.15.52:0/402631271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab08104f50 msgr2=0x7fab08198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.272+0000 7fab0d5ba640 1 --2- 172.21.15.52:0/402631271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab08104f50 0x7fab08198300 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7faafc005fd0 tx=0x7faafc00dde0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.272+0000 7fab0d5ba640 1 -- 172.21.15.52:0/402631271 shutdown_connections 2024-04-02T13:52:55.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.272+0000 7fab0d5ba640 1 --2- 172.21.15.52:0/402631271 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7faaf00a3c40 0x7faaf00a6100 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.272+0000 7fab0d5ba640 1 --2- 172.21.15.52:0/402631271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab08107880 0x7fab0819f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.272+0000 7fab0d5ba640 1 --2- 172.21.15.52:0/402631271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab08104f50 0x7fab08198300 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.272+0000 7fab0d5ba640 1 --2- 172.21.15.52:0/402631271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab08102620 0x7fab08197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.272+0000 7fab0d5ba640 1 -- 172.21.15.52:0/402631271 >> 172.21.15.52:0/402631271 conn(0x7fab080fbf90 msgr2=0x7fab08102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:55.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.273+0000 7fab0d5ba640 1 -- 172.21.15.52:0/402631271 shutdown_connections 2024-04-02T13:52:55.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.273+0000 7fab0d5ba640 1 -- 172.21.15.52:0/402631271 wait complete. 2024-04-02T13:52:55.287 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-02T13:52:55.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.389+0000 7f4cd0263640 1 -- 172.21.15.52:0/3646680614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cc81025b0 msgr2=0x7f4cc8108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.389+0000 7f4cd0263640 1 --2- 172.21.15.52:0/3646680614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cc81025b0 0x7f4cc8108e40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8009900 tx=0x7f4cb802bc30 comp rx=0 tx=0).stop 2024-04-02T13:52:55.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.389+0000 7f4cd0263640 1 -- 172.21.15.52:0/3646680614 shutdown_connections 2024-04-02T13:52:55.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.389+0000 7f4cd0263640 1 --2- 172.21.15.52:0/3646680614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cc81025b0 0x7f4cc8108e40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.389+0000 7f4cd0263640 1 --2- 172.21.15.52:0/3646680614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cc8101bf0 0x7f4cc8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.389+0000 7f4cd0263640 1 --2- 172.21.15.52:0/3646680614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc8100a00 0x7f4cc8100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.389+0000 7f4cd0263640 1 -- 172.21.15.52:0/3646680614 >> 172.21.15.52:0/3646680614 conn(0x7f4cc80fbf90 msgr2=0x7f4cc80fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:55.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.389+0000 7f4cd0263640 1 -- 172.21.15.52:0/3646680614 shutdown_connections 2024-04-02T13:52:55.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.390+0000 7f4cd0263640 1 -- 172.21.15.52:0/3646680614 wait complete. 2024-04-02T13:52:55.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.390+0000 7f4cd0263640 1 Processor -- start 2024-04-02T13:52:55.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.390+0000 7f4cd0263640 1 -- start start 2024-04-02T13:52:55.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.390+0000 7f4cd0263640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc8100a00 0x7f4cc8104920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.390+0000 7f4cd0263640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cc8101bf0 0x7f4cc8102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.390+0000 7f4cd0263640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cc81025b0 0x7f4cc8103510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.390+0000 7f4cd0263640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc810c680 con 0x7f4cc81025b0 2024-04-02T13:52:55.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.390+0000 7f4cd0263640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc810c500 con 0x7f4cc8100a00 2024-04-02T13:52:55.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.390+0000 7f4cd0263640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc810c800 con 0x7f4cc8101bf0 2024-04-02T13:52:55.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.390+0000 7f4cce800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cc81025b0 0x7f4cc8103510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.390+0000 7f4cce800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cc81025b0 0x7f4cc8103510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48768/0 (socket says 172.21.15.52:48768) 2024-04-02T13:52:55.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.390+0000 7f4cce800640 1 -- 172.21.15.52:0/2087768692 learned_addr learned my addr 172.21.15.52:0/2087768692 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:55.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.390+0000 7f4ccdfff640 1 --2- 172.21.15.52:0/2087768692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc8100a00 0x7f4cc8104920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.391+0000 7f4ccd7fe640 1 --2- 172.21.15.52:0/2087768692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cc8101bf0 0x7f4cc8102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.391+0000 7f4cce800640 1 -- 172.21.15.52:0/2087768692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cc8101bf0 msgr2=0x7f4cc8102fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.391+0000 7f4cce800640 1 --2- 172.21.15.52:0/2087768692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cc8101bf0 0x7f4cc8102fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.391+0000 7f4cce800640 1 -- 172.21.15.52:0/2087768692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc8100a00 msgr2=0x7f4cc8104920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.391+0000 7f4cce800640 1 --2- 172.21.15.52:0/2087768692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc8100a00 0x7f4cc8104920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.391+0000 7f4cce800640 1 -- 172.21.15.52:0/2087768692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc8103d30 con 0x7f4cc81025b0 2024-04-02T13:52:55.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.391+0000 7f4ccd7fe640 1 --2- 172.21.15.52:0/2087768692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cc8101bf0 0x7f4cc8102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:52:55.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.391+0000 7f4cce800640 1 --2- 172.21.15.52:0/2087768692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cc81025b0 0x7f4cc8103510 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4cb802c7b0 tx=0x7f4cb8005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:55.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.391+0000 7f4cc6ffd640 1 -- 172.21.15.52:0/2087768692 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8041070 con 0x7f4cc81025b0 2024-04-02T13:52:55.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.391+0000 7f4cc6ffd640 1 -- 172.21.15.52:0/2087768692 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cb80054e0 con 0x7f4cc81025b0 2024-04-02T13:52:55.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.391+0000 7f4cd0263640 1 -- 172.21.15.52:0/2087768692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc81a1f00 con 0x7f4cc81025b0 2024-04-02T13:52:55.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.391+0000 7f4cc6ffd640 1 -- 172.21.15.52:0/2087768692 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8034440 con 0x7f4cc81025b0 2024-04-02T13:52:55.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.391+0000 7f4cd0263640 1 -- 172.21.15.52:0/2087768692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc81a2360 con 0x7f4cc81025b0 2024-04-02T13:52:55.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.393+0000 7f4cc6ffd640 1 -- 172.21.15.52:0/2087768692 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4cb8044070 con 0x7f4cc81025b0 2024-04-02T13:52:55.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.394+0000 7f4cd2e50740 1 Processor -- start 2024-04-02T13:52:55.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.394+0000 7f4cd2e50740 1 -- start start 2024-04-02T13:52:55.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.394+0000 7f4cd2e50740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558bc340ae0 0x5558bc340f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.394+0000 7f4cd2e50740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558bc3414d0 0x5558bc345fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.394+0000 7f4cd2e50740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc346640 0x5558bc348ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.394+0000 7f4cd2e50740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5558bc2a5060 con 0x5558bc346640 2024-04-02T13:52:55.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.394+0000 7f4cd2e50740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5558bc24d650 con 0x5558bc3414d0 2024-04-02T13:52:55.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.394+0000 7f4cd2e50740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5558bc2a1010 con 0x5558bc340ae0 2024-04-02T13:52:55.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.394+0000 7f4ccd7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc346640 0x5558bc348ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.394+0000 7f4ccdfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558bc3414d0 0x5558bc345fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.394+0000 7f4cc6ffd640 1 --2- 172.21.15.52:0/2087768692 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f4cb00a3c40 0x7f4cb00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.394+0000 7f4ccd7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc346640 0x5558bc348ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48776/0 (socket says 172.21.15.52:48776) 2024-04-02T13:52:55.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.394+0000 7f4ccd7fe640 1 -- 172.21.15.52:0/104934142 learned_addr learned my addr 172.21.15.52:0/104934142 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:55.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.394+0000 7f4ccdfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558bc3414d0 0x5558bc345fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:36476/0 (socket says 172.21.15.52:36476) 2024-04-02T13:52:55.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.394+0000 7f4ccd7fe640 1 --2- 172.21.15.52:0/104934142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558bc340ae0 0x5558bc340f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.394+0000 7f4cce800640 1 --2- 172.21.15.52:0/2087768692 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f4cb00a3c40 0x7f4cb00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.394+0000 7f4ccdfff640 1 -- 172.21.15.52:0/104934142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558bc340ae0 msgr2=0x5558bc340f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.394+0000 7f4ccdfff640 1 --2- 172.21.15.52:0/104934142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558bc340ae0 0x5558bc340f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.394+0000 7f4ccdfff640 1 -- 172.21.15.52:0/104934142 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc346640 msgr2=0x5558bc348ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.394+0000 7f4ccdfff640 1 --2- 172.21.15.52:0/104934142 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc346640 0x5558bc348ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.394+0000 7f4cc6ffd640 1 -- 172.21.15.52:0/2087768692 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f4cb80f24c0 con 0x7f4cc81025b0 2024-04-02T13:52:55.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.395+0000 7f4ccdfff640 1 -- 172.21.15.52:0/104934142 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558bbfb92d0 con 0x5558bc3414d0 2024-04-02T13:52:55.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.395+0000 7f4ccd7fe640 1 --2- 172.21.15.52:0/104934142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558bc340ae0 0x5558bc340f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:52:55.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.395+0000 7f4cce800640 1 --2- 172.21.15.52:0/2087768692 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f4cb00a3c40 0x7f4cb00a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80badc0 tx=0x7f4cb80ba7c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:55.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.395+0000 7f4ccdfff640 1 --2- 172.21.15.52:0/104934142 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558bc3414d0 0x5558bc345fd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0005fd0 tx=0x7f4cc00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3f453b5eb79ceec server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:55.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.395+0000 7f4caffff640 1 -- 172.21.15.52:0/104934142 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0003c30 con 0x5558bc3414d0 2024-04-02T13:52:55.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.395+0000 7f4caffff640 1 -- 172.21.15.52:0/104934142 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cc0003dd0 con 0x5558bc3414d0 2024-04-02T13:52:55.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.395+0000 7f4caffff640 1 -- 172.21.15.52:0/104934142 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc000fac0 con 0x5558bc3414d0 2024-04-02T13:52:55.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.396+0000 7f4cd2e50740 1 -- 172.21.15.52:0/104934142 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558bc3414d0 msgr2=0x5558bc345fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.396+0000 7f4cd2e50740 1 --2- 172.21.15.52:0/104934142 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558bc3414d0 0x5558bc345fd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0005fd0 tx=0x7f4cc00118d0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.396+0000 7f4cd2e50740 1 -- 172.21.15.52:0/104934142 shutdown_connections 2024-04-02T13:52:55.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.396+0000 7f4cd2e50740 1 --2- 172.21.15.52:0/104934142 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc346640 0x5558bc348ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.396+0000 7f4cd2e50740 1 --2- 172.21.15.52:0/104934142 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558bc3414d0 0x5558bc345fd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.396+0000 7f4cd2e50740 1 --2- 172.21.15.52:0/104934142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558bc340ae0 0x5558bc340f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.396+0000 7f4cd2e50740 1 -- 172.21.15.52:0/104934142 >> 172.21.15.52:0/104934142 conn(0x5558bc33c050 msgr2=0x5558bc33c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:55.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.396+0000 7f4cd2e50740 1 -- 172.21.15.52:0/104934142 shutdown_connections 2024-04-02T13:52:55.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.396+0000 7f4cd2e50740 1 -- 172.21.15.52:0/104934142 wait complete. 2024-04-02T13:52:55.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.396+0000 7f4cd2e50740 1 -- start start 2024-04-02T13:52:55.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.396+0000 7f4cd2e50740 1 Processor -- start 2024-04-02T13:52:55.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.397+0000 7f4cd2e50740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558bc340ae0 0x5558bc34b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.397+0000 7f4cd2e50740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558bc3414d0 0x5558bc34b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.397+0000 7f4cd2e50740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc346640 0x5558bc352580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.397+0000 7f4cd2e50740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5558bc3495f0 con 0x5558bc346640 2024-04-02T13:52:55.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.397+0000 7f4cd2e50740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5558bc260a10 con 0x5558bc3414d0 2024-04-02T13:52:55.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.397+0000 7f4cd2e50740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5558bc349770 con 0x5558bc340ae0 2024-04-02T13:52:55.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.397+0000 7f4cd2e50740 10 client.0 _ll_register_callbacks cb 0x5558bc29d0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:55.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.397+0000 7f4ccd7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc346640 0x5558bc352580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.397+0000 7f4ccd7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc346640 0x5558bc352580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48782/0 (socket says 172.21.15.52:48782) 2024-04-02T13:52:55.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.397+0000 7f4ccd7fe640 1 -- 172.21.15.52:0/901006941 learned_addr learned my addr 172.21.15.52:0/901006941 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:55.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.397+0000 7f4ccd7fe640 1 --2- 172.21.15.52:0/901006941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558bc340ae0 0x5558bc34b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.397+0000 7f4ccdfff640 1 --2- 172.21.15.52:0/901006941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558bc3414d0 0x5558bc34b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.397+0000 7f4ccd7fe640 1 -- 172.21.15.52:0/901006941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558bc340ae0 msgr2=0x5558bc34b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.397+0000 7f4ccd7fe640 1 --2- 172.21.15.52:0/901006941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558bc340ae0 0x5558bc34b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.397+0000 7f4ccd7fe640 1 -- 172.21.15.52:0/901006941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558bc3414d0 msgr2=0x5558bc34b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.397+0000 7f4ccd7fe640 1 --2- 172.21.15.52:0/901006941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558bc3414d0 0x5558bc34b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.397+0000 7f4ccd7fe640 1 -- 172.21.15.52:0/901006941 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558bc0836b0 con 0x5558bc346640 2024-04-02T13:52:55.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.397+0000 7f4ccd7fe640 1 --2- 172.21.15.52:0/901006941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc346640 0x5558bc352580 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00ca60 tx=0x7f4cbc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50dcf17e6faf270 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:55.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.397+0000 7f4cad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:52:55.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.397+0000 7f4cad7fa640 1 -- 172.21.15.52:0/901006941 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc014070 con 0x5558bc346640 2024-04-02T13:52:55.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.397+0000 7f4cad7fa640 1 -- 172.21.15.52:0/901006941 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f4cbc014210 con 0x5558bc346640 2024-04-02T13:52:55.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.397+0000 7f4cd2e50740 1 -- 172.21.15.52:0/901006941 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cc0003dd0 con 0x5558bc346640 2024-04-02T13:52:55.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.398+0000 7f4cad7fa640 1 -- 172.21.15.52:0/901006941 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4cbc02a050 con 0x5558bc346640 2024-04-02T13:52:55.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.398+0000 7f4cd2e50740 10 client.15264 fetch_fsmap learned FSMap version 27 2024-04-02T13:52:55.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.398+0000 7f4cd2e50740 1 -- 172.21.15.52:0/901006941 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5558bc355960 con 0x5558bc346640 2024-04-02T13:52:55.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.398+0000 7f4cad7fa640 1 -- 172.21.15.52:0/901006941 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cc0003dd0 con 0x5558bc346640 2024-04-02T13:52:55.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.398+0000 7f4cad7fa640 1 -- 172.21.15.52:0/901006941 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4cbc0125c0 con 0x5558bc346640 2024-04-02T13:52:55.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.398+0000 7f4cd2e50740 10 client.15264 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:52:55.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.398+0000 7f4cd2e50740 10 client.15264 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-02T13:52:55.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.398+0000 7f4cd2e50740 20 client.15264 populate_metadata read hostname 'smithi052' 2024-04-02T13:52:55.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.398+0000 7f4cd2e50740 1 --2- 172.21.15.52:0/901006941 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5558bc357fa0 0x5558bc359410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.398+0000 7f4cd2e50740 4 client.15264 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:52:55.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.398+0000 7f4cd2e50740 1 -- 172.21.15.52:0/901006941 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558bc24ccf0 con 0x5558bc357fa0 2024-04-02T13:52:55.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.398+0000 7f4ccdfff640 1 --2- 172.21.15.52:0/901006941 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5558bc357fa0 0x5558bc359410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.399+0000 7f4ccdfff640 1 --2- 172.21.15.52:0/901006941 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5558bc357fa0 0x5558bc359410 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=427a4f7b4e083557 server_cookie=49dd1246fa42efe3 in_seq=0 out_seq=0 2024-04-02T13:52:55.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.399+0000 7f4cad7fa640 10 client.15264 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:52:55.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.400+0000 7f4cad7fa640 1 -- 172.21.15.52:0/901006941 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5558bc24ccf0 con 0x5558bc357fa0 2024-04-02T13:52:55.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.400+0000 7f4cad7fa640 10 client.15264 handle_command_reply: tid=0 2024-04-02T13:52:55.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.400+0000 7f4cd2e50740 1 client.15264 shutdown 2024-04-02T13:52:55.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.400+0000 7f4cd2e50740 1 -- 172.21.15.52:0/901006941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc346640 msgr2=0x5558bc352580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.400+0000 7f4cd2e50740 1 --2- 172.21.15.52:0/901006941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc346640 0x5558bc352580 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00ca60 tx=0x7f4cbc004730 comp rx=0 tx=0).stop 2024-04-02T13:52:55.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.400+0000 7f4cd2e50740 1 -- 172.21.15.52:0/901006941 shutdown_connections 2024-04-02T13:52:55.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.400+0000 7f4cd2e50740 1 --2- 172.21.15.52:0/901006941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc346640 0x5558bc352580 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.400+0000 7f4cd2e50740 1 --2- 172.21.15.52:0/901006941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558bc3414d0 0x5558bc34b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.400+0000 7f4cd2e50740 1 --2- 172.21.15.52:0/901006941 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5558bc357fa0 0x5558bc359410 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.400+0000 7f4cd2e50740 1 --2- 172.21.15.52:0/901006941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558bc340ae0 0x5558bc34b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.400+0000 7f4cad7fa640 0 client.15264 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:52:55.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.400+0000 7f4cd2e50740 1 -- 172.21.15.52:0/901006941 >> 172.21.15.52:0/901006941 conn(0x5558bc33c050 msgr2=0x5558bc33b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:55.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.400+0000 7f4cd2e50740 1 -- 172.21.15.52:0/901006941 shutdown_connections 2024-04-02T13:52:55.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.400+0000 7f4cd2e50740 1 -- 172.21.15.52:0/901006941 wait complete. 2024-04-02T13:52:55.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.400+0000 7f4cd2e50740 20 client.15264 trim_cache size 0 max 16384 2024-04-02T13:52:55.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.418+0000 7f4cd2e50740 1 Processor -- start 2024-04-02T13:52:55.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.418+0000 7f4cd2e50740 1 -- start start 2024-04-02T13:52:55.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.419+0000 7f4cd2e50740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558bc340ae0 0x5558bc3494d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.419+0000 7f4cd2e50740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558bc3414d0 0x5558bc3524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.419+0000 7f4cd2e50740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc346640 0x5558bc34ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.419+0000 7f4cd2e50740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5558bc357750 con 0x5558bc346640 2024-04-02T13:52:55.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.419+0000 7f4cd2e50740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5558bc357e20 con 0x5558bc340ae0 2024-04-02T13:52:55.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.419+0000 7f4cd2e50740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5558bc3584f0 con 0x5558bc3414d0 2024-04-02T13:52:55.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.419+0000 7f4ccd7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc346640 0x5558bc34ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.419+0000 7f4ccd7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc346640 0x5558bc34ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48792/0 (socket says 172.21.15.52:48792) 2024-04-02T13:52:55.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.419+0000 7f4ccd7fe640 1 -- 172.21.15.52:0/2429656792 learned_addr learned my addr 172.21.15.52:0/2429656792 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:55.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.419+0000 7f4ccd7fe640 1 --2- 172.21.15.52:0/2429656792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558bc340ae0 0x5558bc3494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.419+0000 7f4ccdfff640 1 --2- 172.21.15.52:0/2429656792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558bc3414d0 0x5558bc3524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.419+0000 7f4ccd7fe640 1 -- 172.21.15.52:0/2429656792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558bc3414d0 msgr2=0x5558bc3524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.419+0000 7f4ccd7fe640 1 --2- 172.21.15.52:0/2429656792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558bc3414d0 0x5558bc3524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.419+0000 7f4ccd7fe640 1 -- 172.21.15.52:0/2429656792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558bc340ae0 msgr2=0x5558bc3494d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.419+0000 7f4ccd7fe640 1 --2- 172.21.15.52:0/2429656792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558bc340ae0 0x5558bc3494d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.419+0000 7f4ccd7fe640 1 -- 172.21.15.52:0/2429656792 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558bc3589a0 con 0x5558bc346640 2024-04-02T13:52:55.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.419+0000 7f4ccd7fe640 1 --2- 172.21.15.52:0/2429656792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc346640 0x5558bc34ac10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc012990 tx=0x7f4cbc00cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8212d76b3f83d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:55.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.419+0000 7f4caf7fe640 1 -- 172.21.15.52:0/2429656792 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc012e00 con 0x5558bc346640 2024-04-02T13:52:55.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.419+0000 7f4caf7fe640 1 -- 172.21.15.52:0/2429656792 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cbc0140b0 con 0x5558bc346640 2024-04-02T13:52:55.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.419+0000 7f4caf7fe640 1 -- 172.21.15.52:0/2429656792 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc0047a0 con 0x5558bc346640 2024-04-02T13:52:55.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.420+0000 7f4cd2e50740 1 -- 172.21.15.52:0/2429656792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc346640 msgr2=0x5558bc34ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.420+0000 7f4cd2e50740 1 --2- 172.21.15.52:0/2429656792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc346640 0x5558bc34ac10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc012990 tx=0x7f4cbc00cfd0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.420+0000 7f4cd2e50740 1 -- 172.21.15.52:0/2429656792 shutdown_connections 2024-04-02T13:52:55.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.420+0000 7f4cd2e50740 1 --2- 172.21.15.52:0/2429656792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc346640 0x5558bc34ac10 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.420+0000 7f4cd2e50740 1 --2- 172.21.15.52:0/2429656792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558bc3414d0 0x5558bc3524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.420+0000 7f4cd2e50740 1 --2- 172.21.15.52:0/2429656792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558bc340ae0 0x5558bc3494d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.420+0000 7f4cd2e50740 1 -- 172.21.15.52:0/2429656792 >> 172.21.15.52:0/2429656792 conn(0x5558bc33c050 msgr2=0x5558bc33c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:55.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.420+0000 7f4cd2e50740 1 -- 172.21.15.52:0/2429656792 shutdown_connections 2024-04-02T13:52:55.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.420+0000 7f4cd2e50740 1 -- 172.21.15.52:0/2429656792 wait complete. 2024-04-02T13:52:55.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.420+0000 7f4cd2e50740 1 -- start start 2024-04-02T13:52:55.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.421+0000 7f4cd2e50740 1 Processor -- start 2024-04-02T13:52:55.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.421+0000 7f4cd2e50740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558bc340ae0 0x5558bc34e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.421+0000 7f4cd2e50740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc3414d0 0x5558bc34ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.421+0000 7f4cd2e50740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558bc346640 0x5558bc351ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.421+0000 7f4cd2e50740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5558bc359290 con 0x5558bc3414d0 2024-04-02T13:52:55.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.421+0000 7f4cd2e50740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5558bc358bc0 con 0x5558bc346640 2024-04-02T13:52:55.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.421+0000 7f4cd2e50740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5558bc359960 con 0x5558bc340ae0 2024-04-02T13:52:55.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.421+0000 7f4cd2e50740 10 client.0 _ll_register_callbacks cb 0x7f4cbc00c9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:55.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.421+0000 7f4ccdfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc3414d0 0x5558bc34ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.421+0000 7f4ccdfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc3414d0 0x5558bc34ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48808/0 (socket says 172.21.15.52:48808) 2024-04-02T13:52:55.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.421+0000 7f4ccdfff640 1 -- 172.21.15.52:0/3931725344 learned_addr learned my addr 172.21.15.52:0/3931725344 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:55.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.421+0000 7f4ccd7fe640 1 --2- 172.21.15.52:0/3931725344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558bc346640 0x5558bc351ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.421+0000 7f4ccd7fe640 1 --2- 172.21.15.52:0/3931725344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558bc340ae0 0x5558bc34e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.421+0000 7f4ccdfff640 1 -- 172.21.15.52:0/3931725344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558bc340ae0 msgr2=0x5558bc34e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.421+0000 7f4ccdfff640 1 --2- 172.21.15.52:0/3931725344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558bc340ae0 0x5558bc34e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.421+0000 7f4ccdfff640 1 -- 172.21.15.52:0/3931725344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558bc346640 msgr2=0x5558bc351ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.421+0000 7f4ccdfff640 1 --2- 172.21.15.52:0/3931725344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558bc346640 0x5558bc351ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.421+0000 7f4ccdfff640 1 -- 172.21.15.52:0/3931725344 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558bc352150 con 0x5558bc3414d0 2024-04-02T13:52:55.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.421+0000 7f4ccd7fe640 1 --2- 172.21.15.52:0/3931725344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558bc340ae0 0x5558bc34e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:52:55.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.422+0000 7f4ccdfff640 1 --2- 172.21.15.52:0/3931725344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc3414d0 0x5558bc34ec10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0009560 tx=0x7f4cc0009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c818646ad272e405 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:55.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.422+0000 7f4cad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:52:55.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.422+0000 7f4cad7fa640 1 -- 172.21.15.52:0/3931725344 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc000f4f0 con 0x5558bc3414d0 2024-04-02T13:52:55.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.422+0000 7f4cad7fa640 1 -- 172.21.15.52:0/3931725344 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f4cc000f690 con 0x5558bc3414d0 2024-04-02T13:52:55.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.422+0000 7f4cd2e50740 1 -- 172.21.15.52:0/3931725344 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cbc0140b0 con 0x5558bc3414d0 2024-04-02T13:52:55.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.422+0000 7f4cad7fa640 1 -- 172.21.15.52:0/3931725344 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4cc000f890 con 0x5558bc3414d0 2024-04-02T13:52:55.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.422+0000 7f4cd2e50740 10 client.15270 fetch_fsmap learned FSMap version 27 2024-04-02T13:52:55.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.422+0000 7f4cd2e50740 1 -- 172.21.15.52:0/3931725344 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5558bc3be9b0 con 0x5558bc3414d0 2024-04-02T13:52:55.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.422+0000 7f4cad7fa640 1 -- 172.21.15.52:0/3931725344 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cbc0140b0 con 0x5558bc3414d0 2024-04-02T13:52:55.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.422+0000 7f4cad7fa640 1 -- 172.21.15.52:0/3931725344 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4cc002d070 con 0x5558bc3414d0 2024-04-02T13:52:55.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.422+0000 7f4cd2e50740 10 client.15270 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:52:55.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.422+0000 7f4cd2e50740 10 client.15270 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-02T13:52:55.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.422+0000 7f4cd2e50740 20 client.15270 populate_metadata read hostname 'smithi052' 2024-04-02T13:52:55.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.422+0000 7f4cd2e50740 1 --2- 172.21.15.52:0/3931725344 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5558bc3c3400 0x5558bc3c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.422+0000 7f4cd2e50740 4 client.15270 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:52:55.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.422+0000 7f4cd2e50740 1 -- 172.21.15.52:0/3931725344 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "status"}) -- 0x5558bc2b25a0 con 0x5558bc3c3400 2024-04-02T13:52:55.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.423+0000 7f4ccd7fe640 1 --2- 172.21.15.52:0/3931725344 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5558bc3c3400 0x5558bc3c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.423+0000 7f4ccd7fe640 1 --2- 172.21.15.52:0/3931725344 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5558bc3c3400 0x5558bc3c2110 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f755a3ccd1971019 server_cookie=eac5765ee8ac630 in_seq=0 out_seq=0 2024-04-02T13:52:55.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.423+0000 7f4cad7fa640 10 client.15270 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:52:55.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.423+0000 7f4cad7fa640 1 -- 172.21.15.52:0/3931725344 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5558bc2b25a0 con 0x5558bc3c3400 2024-04-02T13:52:55.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.423+0000 7f4cad7fa640 10 client.15270 handle_command_reply: tid=0 2024-04-02T13:52:55.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.423+0000 7f4cd2e50740 1 client.15270 shutdown 2024-04-02T13:52:55.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.423+0000 7f4cd2e50740 1 -- 172.21.15.52:0/3931725344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc3414d0 msgr2=0x5558bc34ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.423+0000 7f4cd2e50740 1 --2- 172.21.15.52:0/3931725344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc3414d0 0x5558bc34ec10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0009560 tx=0x7f4cc0009c90 comp rx=0 tx=0).stop 2024-04-02T13:52:55.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.423+0000 7f4cd2e50740 1 -- 172.21.15.52:0/3931725344 shutdown_connections 2024-04-02T13:52:55.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.423+0000 7f4cd2e50740 1 --2- 172.21.15.52:0/3931725344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558bc346640 0x5558bc351ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.423+0000 7f4cd2e50740 1 --2- 172.21.15.52:0/3931725344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558bc3414d0 0x5558bc34ec10 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.423+0000 7f4cd2e50740 1 --2- 172.21.15.52:0/3931725344 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5558bc3c3400 0x5558bc3c2110 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.423+0000 7f4cd2e50740 1 --2- 172.21.15.52:0/3931725344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558bc340ae0 0x5558bc34e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.423+0000 7f4cd2e50740 1 -- 172.21.15.52:0/3931725344 >> 172.21.15.52:0/3931725344 conn(0x5558bc33c050 msgr2=0x5558bc34ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:55.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.423+0000 7f4cad7fa640 0 client.15270 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:52:55.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.423+0000 7f4cd2e50740 1 -- 172.21.15.52:0/3931725344 shutdown_connections 2024-04-02T13:52:55.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.423+0000 7f4cd2e50740 1 -- 172.21.15.52:0/3931725344 wait complete. 2024-04-02T13:52:55.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.423+0000 7f4cd2e50740 20 client.15270 trim_cache size 0 max 16384 2024-04-02T13:52:55.445 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:52:55.445 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "bfa00e6e-f0f6-11ee-b647-cb9ed24678a4", 2024-04-02T13:52:55.445 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 1, 2024-04-02T13:52:55.446 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24490, 2024-04-02T13:52:55.446 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-04-02T13:52:55.446 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-04-02T13:52:55.446 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-04-02T13:52:55.446 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 90.101968709000005, 2024-04-02T13:52:55.446 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 27, 2024-04-02T13:52:55.446 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 92, 2024-04-02T13:52:55.446 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:52:55.446 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 128.816471713 2024-04-02T13:52:55.446 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:52:55.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.424+0000 7f4cd0263640 1 -- 172.21.15.52:0/2087768692 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f4cb00a3c40 msgr2=0x7f4cb00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.424+0000 7f4cd0263640 1 --2- 172.21.15.52:0/2087768692 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f4cb00a3c40 0x7f4cb00a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80badc0 tx=0x7f4cb80ba7c0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.424+0000 7f4cd0263640 1 -- 172.21.15.52:0/2087768692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cc81025b0 msgr2=0x7f4cc8103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.424+0000 7f4cd0263640 1 --2- 172.21.15.52:0/2087768692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cc81025b0 0x7f4cc8103510 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4cb802c7b0 tx=0x7f4cb8005dd0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.424+0000 7f4cd0263640 1 -- 172.21.15.52:0/2087768692 shutdown_connections 2024-04-02T13:52:55.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.424+0000 7f4cd0263640 1 --2- 172.21.15.52:0/2087768692 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f4cb00a3c40 0x7f4cb00a6100 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.424+0000 7f4cd0263640 1 --2- 172.21.15.52:0/2087768692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cc81025b0 0x7f4cc8103510 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.424+0000 7f4cd0263640 1 --2- 172.21.15.52:0/2087768692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cc8101bf0 0x7f4cc8102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.424+0000 7f4cd0263640 1 --2- 172.21.15.52:0/2087768692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc8100a00 0x7f4cc8104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.424+0000 7f4cd0263640 1 -- 172.21.15.52:0/2087768692 >> 172.21.15.52:0/2087768692 conn(0x7f4cc80fbf90 msgr2=0x7f4cc80fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:55.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.424+0000 7f4cd0263640 1 -- 172.21.15.52:0/2087768692 shutdown_connections 2024-04-02T13:52:55.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.424+0000 7f4cd0263640 1 -- 172.21.15.52:0/2087768692 wait complete. 2024-04-02T13:52:55.450 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-02T13:52:55.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.556+0000 7ff1db635640 1 -- 172.21.15.52:0/4153888261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1d41077e0 msgr2=0x7ff1d4109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.556+0000 7ff1db635640 1 --2- 172.21.15.52:0/4153888261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1d41077e0 0x7ff1d4109c30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4009ca0 tx=0x7ff1c402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.556+0000 7ff1db635640 1 -- 172.21.15.52:0/4153888261 shutdown_connections 2024-04-02T13:52:55.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.556+0000 7ff1db635640 1 --2- 172.21.15.52:0/4153888261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1d41077e0 0x7ff1d4109c30 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.556+0000 7ff1db635640 1 --2- 172.21.15.52:0/4153888261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1d4104eb0 0x7ff1d41072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.556+0000 7ff1db635640 1 --2- 172.21.15.52:0/4153888261 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1d4102580 0x7ff1d4104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.556+0000 7ff1db635640 1 -- 172.21.15.52:0/4153888261 >> 172.21.15.52:0/4153888261 conn(0x7ff1d40fbf90 msgr2=0x7ff1d40fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:55.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.556+0000 7ff1db635640 1 -- 172.21.15.52:0/4153888261 shutdown_connections 2024-04-02T13:52:55.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.556+0000 7ff1db635640 1 -- 172.21.15.52:0/4153888261 wait complete. 2024-04-02T13:52:55.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.556+0000 7ff1db635640 1 Processor -- start 2024-04-02T13:52:55.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.556+0000 7ff1db635640 1 -- start start 2024-04-02T13:52:55.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.556+0000 7ff1db635640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1d4102580 0x7ff1d4197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.556+0000 7ff1db635640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1d4104eb0 0x7ff1d4198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.556+0000 7ff1db635640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1d41077e0 0x7ff1d419f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.556+0000 7ff1db635640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d410d7a0 con 0x7ff1d4104eb0 2024-04-02T13:52:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.556+0000 7ff1db635640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d410d620 con 0x7ff1d41077e0 2024-04-02T13:52:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.556+0000 7ff1db635640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d410d920 con 0x7ff1d4102580 2024-04-02T13:52:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.556+0000 7ff1d8bd0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1d4104eb0 0x7ff1d4198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.557+0000 7ff1d8bd0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1d4104eb0 0x7ff1d4198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48820/0 (socket says 172.21.15.52:48820) 2024-04-02T13:52:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.557+0000 7ff1d8bd0640 1 -- 172.21.15.52:0/3180404905 learned_addr learned my addr 172.21.15.52:0/3180404905 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.557+0000 7ff1d93d1640 1 --2- 172.21.15.52:0/3180404905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1d4102580 0x7ff1d4197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.557+0000 7ff1d9bd2640 1 --2- 172.21.15.52:0/3180404905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1d41077e0 0x7ff1d419f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.557+0000 7ff1d8bd0640 1 -- 172.21.15.52:0/3180404905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1d4102580 msgr2=0x7ff1d4197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.557+0000 7ff1d8bd0640 1 --2- 172.21.15.52:0/3180404905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1d4102580 0x7ff1d4197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.557+0000 7ff1d8bd0640 1 -- 172.21.15.52:0/3180404905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1d41077e0 msgr2=0x7ff1d419f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.557+0000 7ff1d8bd0640 1 --2- 172.21.15.52:0/3180404905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1d41077e0 0x7ff1d419f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.557+0000 7ff1d8bd0640 1 -- 172.21.15.52:0/3180404905 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d419fa70 con 0x7ff1d4104eb0 2024-04-02T13:52:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.557+0000 7ff1d9bd2640 1 --2- 172.21.15.52:0/3180404905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1d41077e0 0x7ff1d419f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:52:55.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.557+0000 7ff1d8bd0640 1 --2- 172.21.15.52:0/3180404905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1d4104eb0 0x7ff1d4198390 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800c970 tx=0x7ff1c800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:55.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.557+0000 7ff1d27fc640 1 -- 172.21.15.52:0/3180404905 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8007bb0 con 0x7ff1d4104eb0 2024-04-02T13:52:55.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.557+0000 7ff1d27fc640 1 -- 172.21.15.52:0/3180404905 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1c8007d50 con 0x7ff1d4104eb0 2024-04-02T13:52:55.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.557+0000 7ff1db635640 1 -- 172.21.15.52:0/3180404905 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d419fd60 con 0x7ff1d4104eb0 2024-04-02T13:52:55.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.557+0000 7ff1d27fc640 1 -- 172.21.15.52:0/3180404905 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c80187a0 con 0x7ff1d4104eb0 2024-04-02T13:52:55.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.557+0000 7ff1db635640 1 -- 172.21.15.52:0/3180404905 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d41a0250 con 0x7ff1d4104eb0 2024-04-02T13:52:55.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.560+0000 7ff1d27fc640 1 -- 172.21.15.52:0/3180404905 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff1c8018a60 con 0x7ff1d4104eb0 2024-04-02T13:52:55.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.560+0000 7ff1de21e740 1 Processor -- start 2024-04-02T13:52:55.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.560+0000 7ff1de21e740 1 -- start start 2024-04-02T13:52:55.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.560+0000 7ff1de21e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd17931ae0 0x55cd17931f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.560+0000 7ff1de21e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd179324d0 0x55cd17936fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.560+0000 7ff1de21e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd17937640 0x55cd17939ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.560+0000 7ff1de21e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cd17896060 con 0x55cd179324d0 2024-04-02T13:52:55.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.560+0000 7ff1de21e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cd1783e650 con 0x55cd17931ae0 2024-04-02T13:52:55.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.560+0000 7ff1de21e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cd17892010 con 0x55cd17937640 2024-04-02T13:52:55.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.561+0000 7ff1d93d1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd179324d0 0x55cd17936fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.561+0000 7ff1d27fc640 1 --2- 172.21.15.52:0/3180404905 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ff1bc0a3c30 0x7ff1bc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.561+0000 7ff1d93d1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd179324d0 0x55cd17936fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48824/0 (socket says 172.21.15.52:48824) 2024-04-02T13:52:55.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.561+0000 7ff1d93d1640 1 -- 172.21.15.52:0/2936167782 learned_addr learned my addr 172.21.15.52:0/2936167782 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:55.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.561+0000 7ff1d9bd2640 1 --2- 172.21.15.52:0/3180404905 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ff1bc0a3c30 0x7ff1bc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.561+0000 7ff1d9bd2640 1 --2- 172.21.15.52:0/2936167782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd17931ae0 0x55cd17931f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.561+0000 7ff1d27fc640 1 -- 172.21.15.52:0/3180404905 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff1c80d0830 con 0x7ff1d4104eb0 2024-04-02T13:52:55.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.561+0000 7ff1d8bd0640 1 --2- 172.21.15.52:0/2936167782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd17937640 0x55cd17939ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.561+0000 7ff1d93d1640 1 -- 172.21.15.52:0/2936167782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd17937640 msgr2=0x55cd17939ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.561+0000 7ff1d93d1640 1 --2- 172.21.15.52:0/2936167782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd17937640 0x55cd17939ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.561+0000 7ff1d93d1640 1 -- 172.21.15.52:0/2936167782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd17931ae0 msgr2=0x55cd17931f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.561+0000 7ff1d93d1640 1 --2- 172.21.15.52:0/2936167782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd17931ae0 0x55cd17931f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.561+0000 7ff1d93d1640 1 -- 172.21.15.52:0/2936167782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd175aa2d0 con 0x55cd179324d0 2024-04-02T13:52:55.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.561+0000 7ff1d8bd0640 1 --2- 172.21.15.52:0/2936167782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd17937640 0x55cd17939ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:52:55.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.561+0000 7ff1d93d1640 1 --2- 172.21.15.52:0/2936167782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd179324d0 0x55cd17936fd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00bd10 tx=0x7ff1cc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ad397246788d90 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:55.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.561+0000 7ff1d9bd2640 1 --2- 172.21.15.52:0/3180404905 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ff1bc0a3c30 0x7ff1bc0a60f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55cd17932d30 tx=0x7ff1c4005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:55.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.561+0000 7ff1bb7fe640 1 -- 172.21.15.52:0/2936167782 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc015070 con 0x55cd179324d0 2024-04-02T13:52:55.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.561+0000 7ff1bb7fe640 1 -- 172.21.15.52:0/2936167782 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1cc01a940 con 0x55cd179324d0 2024-04-02T13:52:55.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.561+0000 7ff1bb7fe640 1 -- 172.21.15.52:0/2936167782 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc011a30 con 0x55cd179324d0 2024-04-02T13:52:55.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.562+0000 7ff1de21e740 1 -- 172.21.15.52:0/2936167782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd179324d0 msgr2=0x55cd17936fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.562+0000 7ff1de21e740 1 --2- 172.21.15.52:0/2936167782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd179324d0 0x55cd17936fd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00bd10 tx=0x7ff1cc00f730 comp rx=0 tx=0).stop 2024-04-02T13:52:55.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.562+0000 7ff1de21e740 1 -- 172.21.15.52:0/2936167782 shutdown_connections 2024-04-02T13:52:55.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.562+0000 7ff1de21e740 1 --2- 172.21.15.52:0/2936167782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd17937640 0x55cd17939ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.562+0000 7ff1de21e740 1 --2- 172.21.15.52:0/2936167782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd179324d0 0x55cd17936fd0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.562+0000 7ff1de21e740 1 --2- 172.21.15.52:0/2936167782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd17931ae0 0x55cd17931f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.562+0000 7ff1de21e740 1 -- 172.21.15.52:0/2936167782 >> 172.21.15.52:0/2936167782 conn(0x55cd1792d050 msgr2=0x55cd1792d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:55.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.562+0000 7ff1de21e740 1 -- 172.21.15.52:0/2936167782 shutdown_connections 2024-04-02T13:52:55.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.562+0000 7ff1de21e740 1 -- 172.21.15.52:0/2936167782 wait complete. 2024-04-02T13:52:55.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.562+0000 7ff1de21e740 1 -- start start 2024-04-02T13:52:55.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.562+0000 7ff1de21e740 1 Processor -- start 2024-04-02T13:52:55.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.563+0000 7ff1de21e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd17931ae0 0x55cd1793c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.563+0000 7ff1de21e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd179324d0 0x55cd1793c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.563+0000 7ff1de21e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd17937640 0x55cd17943590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.563+0000 7ff1de21e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cd1793a5f0 con 0x55cd17937640 2024-04-02T13:52:55.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.563+0000 7ff1de21e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cd17851a10 con 0x55cd179324d0 2024-04-02T13:52:55.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.563+0000 7ff1de21e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cd1793a770 con 0x55cd17931ae0 2024-04-02T13:52:55.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.563+0000 7ff1de21e740 10 client.0 _ll_register_callbacks cb 0x55cd1788e0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:55.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.563+0000 7ff1d9bd2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd17937640 0x55cd17943590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.563+0000 7ff1d9bd2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd17937640 0x55cd17943590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48840/0 (socket says 172.21.15.52:48840) 2024-04-02T13:52:55.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.563+0000 7ff1d9bd2640 1 -- 172.21.15.52:0/2916951899 learned_addr learned my addr 172.21.15.52:0/2916951899 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:55.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.563+0000 7ff1d8bd0640 1 --2- 172.21.15.52:0/2916951899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd179324d0 0x55cd1793c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.563+0000 7ff1d93d1640 1 --2- 172.21.15.52:0/2916951899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd17931ae0 0x55cd1793c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.563+0000 7ff1d9bd2640 1 -- 172.21.15.52:0/2916951899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd17931ae0 msgr2=0x55cd1793c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.563+0000 7ff1d9bd2640 1 --2- 172.21.15.52:0/2916951899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd17931ae0 0x55cd1793c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.563+0000 7ff1d9bd2640 1 -- 172.21.15.52:0/2916951899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd179324d0 msgr2=0x55cd1793c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.563+0000 7ff1d9bd2640 1 --2- 172.21.15.52:0/2916951899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd179324d0 0x55cd1793c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.563+0000 7ff1d9bd2640 1 -- 172.21.15.52:0/2916951899 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd176746b0 con 0x55cd17937640 2024-04-02T13:52:55.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.563+0000 7ff1d9bd2640 1 --2- 172.21.15.52:0/2916951899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd17937640 0x55cd17943590 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4009ca0 tx=0x7ff1c40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77ba1e1ed96ce70 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:55.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.563+0000 7ff1b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:52:55.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.564+0000 7ff1b8ff9640 1 -- 172.21.15.52:0/2916951899 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4050020 con 0x55cd17937640 2024-04-02T13:52:55.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.564+0000 7ff1b8ff9640 1 -- 172.21.15.52:0/2916951899 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff1c4004240 con 0x55cd17937640 2024-04-02T13:52:55.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.564+0000 7ff1de21e740 1 -- 172.21.15.52:0/2916951899 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1cc01a940 con 0x55cd17937640 2024-04-02T13:52:55.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.564+0000 7ff1b8ff9640 1 -- 172.21.15.52:0/2916951899 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff1c405b050 con 0x55cd17937640 2024-04-02T13:52:55.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.564+0000 7ff1de21e740 10 client.15282 fetch_fsmap learned FSMap version 27 2024-04-02T13:52:55.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.564+0000 7ff1de21e740 1 -- 172.21.15.52:0/2916951899 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cd179468e0 con 0x55cd17937640 2024-04-02T13:52:55.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.564+0000 7ff1b8ff9640 1 -- 172.21.15.52:0/2916951899 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1cc01a940 con 0x55cd17937640 2024-04-02T13:52:55.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.564+0000 7ff1b8ff9640 1 -- 172.21.15.52:0/2916951899 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff1c404b030 con 0x55cd17937640 2024-04-02T13:52:55.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.564+0000 7ff1de21e740 10 client.15282 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:52:55.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.564+0000 7ff1de21e740 10 client.15282 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T13:52:55.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.564+0000 7ff1de21e740 20 client.15282 populate_metadata read hostname 'smithi052' 2024-04-02T13:52:55.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.564+0000 7ff1de21e740 1 --2- 172.21.15.52:0/2916951899 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cd17949010 0x55cd1794a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.564+0000 7ff1de21e740 4 client.15282 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:52:55.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.564+0000 7ff1de21e740 1 -- 172.21.15.52:0/2916951899 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd1783dcf0 con 0x55cd17949010 2024-04-02T13:52:55.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.564+0000 7ff1d93d1640 1 --2- 172.21.15.52:0/2916951899 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cd17949010 0x55cd1794a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.565+0000 7ff1b8ff9640 10 client.15282 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:52:55.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.565+0000 7ff1d93d1640 1 --2- 172.21.15.52:0/2916951899 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cd17949010 0x55cd1794a480 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b430fd96a7f01706 server_cookie=741e78fd28ba713 in_seq=0 out_seq=0 2024-04-02T13:52:55.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.566+0000 7ff1b8ff9640 1 -- 172.21.15.52:0/2916951899 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cd1783dcf0 con 0x55cd17949010 2024-04-02T13:52:55.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.566+0000 7ff1b8ff9640 10 client.15282 handle_command_reply: tid=0 2024-04-02T13:52:55.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.566+0000 7ff1de21e740 1 client.15282 shutdown 2024-04-02T13:52:55.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.566+0000 7ff1de21e740 1 -- 172.21.15.52:0/2916951899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd17937640 msgr2=0x55cd17943590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.566+0000 7ff1de21e740 1 --2- 172.21.15.52:0/2916951899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd17937640 0x55cd17943590 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4009ca0 tx=0x7ff1c40095c0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.566+0000 7ff1de21e740 1 -- 172.21.15.52:0/2916951899 shutdown_connections 2024-04-02T13:52:55.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.566+0000 7ff1de21e740 1 --2- 172.21.15.52:0/2916951899 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cd17949010 0x55cd1794a480 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.566+0000 7ff1de21e740 1 --2- 172.21.15.52:0/2916951899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd17937640 0x55cd17943590 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.566+0000 7ff1de21e740 1 --2- 172.21.15.52:0/2916951899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd179324d0 0x55cd1793c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.566+0000 7ff1b8ff9640 0 client.15282 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:52:55.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.566+0000 7ff1de21e740 1 --2- 172.21.15.52:0/2916951899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd17931ae0 0x55cd1793c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.566+0000 7ff1de21e740 1 -- 172.21.15.52:0/2916951899 >> 172.21.15.52:0/2916951899 conn(0x55cd1792d050 msgr2=0x55cd1792c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:55.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.566+0000 7ff1de21e740 1 -- 172.21.15.52:0/2916951899 shutdown_connections 2024-04-02T13:52:55.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.566+0000 7ff1de21e740 1 -- 172.21.15.52:0/2916951899 wait complete. 2024-04-02T13:52:55.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.566+0000 7ff1de21e740 20 client.15282 trim_cache size 0 max 16384 2024-04-02T13:52:55.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.584+0000 7ff1de21e740 1 Processor -- start 2024-04-02T13:52:55.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.584+0000 7ff1de21e740 1 -- start start 2024-04-02T13:52:55.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.584+0000 7ff1de21e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd17931ae0 0x55cd17939a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.584+0000 7ff1de21e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd179324d0 0x55cd179434b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.584+0000 7ff1de21e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd17937640 0x55cd17933a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.584+0000 7ff1de21e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cd179487c0 con 0x55cd17931ae0 2024-04-02T13:52:55.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.584+0000 7ff1de21e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cd17948e90 con 0x55cd17937640 2024-04-02T13:52:55.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.584+0000 7ff1de21e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cd17949560 con 0x55cd179324d0 2024-04-02T13:52:55.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.584+0000 7ff1d93d1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd17931ae0 0x55cd17939a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.584+0000 7ff1d93d1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd17931ae0 0x55cd17939a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48852/0 (socket says 172.21.15.52:48852) 2024-04-02T13:52:55.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.584+0000 7ff1d93d1640 1 -- 172.21.15.52:0/3015164016 learned_addr learned my addr 172.21.15.52:0/3015164016 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:55.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.585+0000 7ff1d93d1640 1 -- 172.21.15.52:0/3015164016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd179324d0 msgr2=0x55cd179434b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.585+0000 7ff1d93d1640 1 --2- 172.21.15.52:0/3015164016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd179324d0 0x55cd179434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.585+0000 7ff1d93d1640 1 -- 172.21.15.52:0/3015164016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd17937640 msgr2=0x55cd17933a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.585+0000 7ff1d93d1640 1 --2- 172.21.15.52:0/3015164016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd17937640 0x55cd17933a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.585+0000 7ff1d93d1640 1 -- 172.21.15.52:0/3015164016 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd1792ddc0 con 0x55cd17931ae0 2024-04-02T13:52:55.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.585+0000 7ff1d93d1640 1 --2- 172.21.15.52:0/3015164016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd17931ae0 0x55cd17939a30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00bc30 tx=0x7ff1cc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99bd4aac4e27a8b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:55.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.585+0000 7ff1baffd640 1 -- 172.21.15.52:0/3015164016 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc015040 con 0x55cd17931ae0 2024-04-02T13:52:55.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.585+0000 7ff1baffd640 1 -- 172.21.15.52:0/3015164016 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1cc005520 con 0x55cd17931ae0 2024-04-02T13:52:55.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.585+0000 7ff1baffd640 1 -- 172.21.15.52:0/3015164016 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc011b90 con 0x55cd17931ae0 2024-04-02T13:52:55.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.585+0000 7ff1de21e740 1 -- 172.21.15.52:0/3015164016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd17931ae0 msgr2=0x55cd17939a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.585+0000 7ff1de21e740 1 --2- 172.21.15.52:0/3015164016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd17931ae0 0x55cd17939a30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00bc30 tx=0x7ff1cc005fd0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.586+0000 7ff1de21e740 1 -- 172.21.15.52:0/3015164016 shutdown_connections 2024-04-02T13:52:55.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.586+0000 7ff1de21e740 1 --2- 172.21.15.52:0/3015164016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd17937640 0x55cd17933a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.586+0000 7ff1de21e740 1 --2- 172.21.15.52:0/3015164016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd179324d0 0x55cd179434b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.586+0000 7ff1de21e740 1 --2- 172.21.15.52:0/3015164016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd17931ae0 0x55cd17939a30 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.586+0000 7ff1de21e740 1 -- 172.21.15.52:0/3015164016 >> 172.21.15.52:0/3015164016 conn(0x55cd1792d050 msgr2=0x55cd1792d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:55.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.586+0000 7ff1de21e740 1 -- 172.21.15.52:0/3015164016 shutdown_connections 2024-04-02T13:52:55.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.586+0000 7ff1de21e740 1 -- 172.21.15.52:0/3015164016 wait complete. 2024-04-02T13:52:55.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.586+0000 7ff1de21e740 1 -- start start 2024-04-02T13:52:55.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.586+0000 7ff1de21e740 1 Processor -- start 2024-04-02T13:52:55.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.586+0000 7ff1de21e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd17931ae0 0x55cd179414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.586+0000 7ff1de21e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd179324d0 0x55cd179aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.586+0000 7ff1de21e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd17937640 0x55cd179ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.586+0000 7ff1de21e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cd1794a300 con 0x55cd179324d0 2024-04-02T13:52:55.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.586+0000 7ff1de21e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cd17949c30 con 0x55cd17931ae0 2024-04-02T13:52:55.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.586+0000 7ff1de21e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cd1794a9d0 con 0x55cd17937640 2024-04-02T13:52:55.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.586+0000 7ff1de21e740 10 client.0 _ll_register_callbacks cb 0x7ff1c4037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:55.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.586+0000 7ff1d8bd0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd179324d0 0x55cd179aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.586+0000 7ff1d8bd0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd179324d0 0x55cd179aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48854/0 (socket says 172.21.15.52:48854) 2024-04-02T13:52:55.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.586+0000 7ff1d8bd0640 1 -- 172.21.15.52:0/1579920358 learned_addr learned my addr 172.21.15.52:0/1579920358 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:55.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.587+0000 7ff1d9bd2640 1 --2- 172.21.15.52:0/1579920358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd17937640 0x55cd179ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.587+0000 7ff1d8bd0640 1 -- 172.21.15.52:0/1579920358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd17937640 msgr2=0x55cd179ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.587+0000 7ff1d8bd0640 1 --2- 172.21.15.52:0/1579920358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd17937640 0x55cd179ad4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.587+0000 7ff1d8bd0640 1 -- 172.21.15.52:0/1579920358 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd17931ae0 msgr2=0x55cd179414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:55.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.587+0000 7ff1d93d1640 1 --2- 172.21.15.52:0/1579920358 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd17931ae0 0x55cd179414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.587+0000 7ff1d8bd0640 1 --2- 172.21.15.52:0/1579920358 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd17931ae0 0x55cd179414a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.587+0000 7ff1d8bd0640 1 -- 172.21.15.52:0/1579920358 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd179adcd0 con 0x55cd179324d0 2024-04-02T13:52:55.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.587+0000 7ff1d9bd2640 1 --2- 172.21.15.52:0/1579920358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd17937640 0x55cd179ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:52:55.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.587+0000 7ff1d8bd0640 1 --2- 172.21.15.52:0/1579920358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd179324d0 0x55cd179aafa0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8017000 tx=0x7ff1c8098b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a04ff38d40cb04a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:55.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.587+0000 7ff1b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:52:55.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.587+0000 7ff1b8ff9640 1 -- 172.21.15.52:0/1579920358 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c809c800 con 0x55cd179324d0 2024-04-02T13:52:55.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.587+0000 7ff1b8ff9640 1 -- 172.21.15.52:0/1579920358 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff1c8018d50 con 0x55cd179324d0 2024-04-02T13:52:55.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.587+0000 7ff1de21e740 1 -- 172.21.15.52:0/1579920358 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1cc005520 con 0x55cd179324d0 2024-04-02T13:52:55.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.587+0000 7ff1b8ff9640 1 -- 172.21.15.52:0/1579920358 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff1c80a0b00 con 0x55cd179324d0 2024-04-02T13:52:55.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.587+0000 7ff1de21e740 10 client.15288 fetch_fsmap learned FSMap version 27 2024-04-02T13:52:55.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.587+0000 7ff1de21e740 1 -- 172.21.15.52:0/1579920358 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cd179af040 con 0x55cd179324d0 2024-04-02T13:52:55.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.587+0000 7ff1b8ff9640 1 -- 172.21.15.52:0/1579920358 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1cc005520 con 0x55cd179324d0 2024-04-02T13:52:55.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.587+0000 7ff1b8ff9640 1 -- 172.21.15.52:0/1579920358 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff1c80b4070 con 0x55cd179324d0 2024-04-02T13:52:55.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.587+0000 7ff1de21e740 10 client.15288 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:52:55.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.588+0000 7ff1de21e740 10 client.15288 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T13:52:55.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.588+0000 7ff1de21e740 20 client.15288 populate_metadata read hostname 'smithi052' 2024-04-02T13:52:55.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.588+0000 7ff1de21e740 1 --2- 172.21.15.52:0/1579920358 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cd179b3c10 0x55cd179b2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:55.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.588+0000 7ff1de21e740 4 client.15288 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:52:55.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.588+0000 7ff1de21e740 1 -- 172.21.15.52:0/1579920358 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "status"}) -- 0x55cd178a35a0 con 0x55cd179b3c10 2024-04-02T13:52:55.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.588+0000 7ff1d93d1640 1 --2- 172.21.15.52:0/1579920358 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cd179b3c10 0x55cd179b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:55.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.588+0000 7ff1d93d1640 1 --2- 172.21.15.52:0/1579920358 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cd179b3c10 0x55cd179b2920 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b251a0ef50f5d7d5 server_cookie=7bdde0be582639de in_seq=0 out_seq=0 2024-04-02T13:52:55.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.588+0000 7ff1b8ff9640 10 client.15288 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:52:55.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.588+0000 7ff1b8ff9640 1 -- 172.21.15.52:0/1579920358 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55cd178a35a0 con 0x55cd179b3c10 2024-04-02T13:52:55.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.588+0000 7ff1b8ff9640 10 client.15288 handle_command_reply: tid=0 2024-04-02T13:52:55.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.588+0000 7ff1de21e740 1 client.15288 shutdown 2024-04-02T13:52:55.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.588+0000 7ff1de21e740 1 -- 172.21.15.52:0/1579920358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd179324d0 msgr2=0x55cd179aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.588+0000 7ff1de21e740 1 --2- 172.21.15.52:0/1579920358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd179324d0 0x55cd179aafa0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8017000 tx=0x7ff1c8098b60 comp rx=0 tx=0).stop 2024-04-02T13:52:55.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.588+0000 7ff1de21e740 1 -- 172.21.15.52:0/1579920358 shutdown_connections 2024-04-02T13:52:55.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.588+0000 7ff1de21e740 1 --2- 172.21.15.52:0/1579920358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd17937640 0x55cd179ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.588+0000 7ff1de21e740 1 --2- 172.21.15.52:0/1579920358 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cd179b3c10 0x55cd179b2920 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.588+0000 7ff1de21e740 1 --2- 172.21.15.52:0/1579920358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd179324d0 0x55cd179aafa0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.588+0000 7ff1de21e740 1 --2- 172.21.15.52:0/1579920358 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd17931ae0 0x55cd179414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.588+0000 7ff1b8ff9640 0 client.15288 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:52:55.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.588+0000 7ff1de21e740 1 -- 172.21.15.52:0/1579920358 >> 172.21.15.52:0/1579920358 conn(0x55cd1792d050 msgr2=0x55cd17933a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:55.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.589+0000 7ff1de21e740 1 -- 172.21.15.52:0/1579920358 shutdown_connections 2024-04-02T13:52:55.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.589+0000 7ff1de21e740 1 -- 172.21.15.52:0/1579920358 wait complete. 2024-04-02T13:52:55.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.589+0000 7ff1de21e740 20 client.15288 trim_cache size 0 max 16384 2024-04-02T13:52:55.600 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:52:55.600 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "bfa00e6e-f0f6-11ee-b647-cb9ed24678a4", 2024-04-02T13:52:55.600 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 0, 2024-04-02T13:52:55.600 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24503, 2024-04-02T13:52:55.600 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-04-02T13:52:55.600 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-04-02T13:52:55.600 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-04-02T13:52:55.600 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 114.265241119, 2024-04-02T13:52:55.600 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 27, 2024-04-02T13:52:55.600 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 92, 2024-04-02T13:52:55.600 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:52:55.600 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 120.363103618 2024-04-02T13:52:55.600 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:52:55.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.589+0000 7ff1db635640 1 -- 172.21.15.52:0/3180404905 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ff1bc0a3c30 msgr2=0x7ff1bc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.589+0000 7ff1db635640 1 --2- 172.21.15.52:0/3180404905 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ff1bc0a3c30 0x7ff1bc0a60f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55cd17932d30 tx=0x7ff1c4005da0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.589+0000 7ff1db635640 1 -- 172.21.15.52:0/3180404905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1d4104eb0 msgr2=0x7ff1d4198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:55.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.589+0000 7ff1db635640 1 --2- 172.21.15.52:0/3180404905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1d4104eb0 0x7ff1d4198390 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800c970 tx=0x7ff1c800ce30 comp rx=0 tx=0).stop 2024-04-02T13:52:55.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.589+0000 7ff1db635640 1 -- 172.21.15.52:0/3180404905 shutdown_connections 2024-04-02T13:52:55.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.589+0000 7ff1db635640 1 --2- 172.21.15.52:0/3180404905 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ff1bc0a3c30 0x7ff1bc0a60f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.589+0000 7ff1db635640 1 --2- 172.21.15.52:0/3180404905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1d41077e0 0x7ff1d419f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.589+0000 7ff1db635640 1 --2- 172.21.15.52:0/3180404905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1d4104eb0 0x7ff1d4198390 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.589+0000 7ff1db635640 1 --2- 172.21.15.52:0/3180404905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1d4102580 0x7ff1d4197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:55.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.589+0000 7ff1db635640 1 -- 172.21.15.52:0/3180404905 >> 172.21.15.52:0/3180404905 conn(0x7ff1d40fbf90 msgr2=0x7ff1d4102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:55.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.589+0000 7ff1db635640 1 -- 172.21.15.52:0/3180404905 shutdown_connections 2024-04-02T13:52:55.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:55.589+0000 7ff1db635640 1 -- 172.21.15.52:0/3180404905 wait complete. 2024-04-02T13:52:55.603 INFO:tasks.cephadm:Setup complete, yielding 2024-04-02T13:52:55.604 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T13:52:55.613 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi052.front.sepia.ceph.com 2024-04-02T13:52:55.613 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch status' 2024-04-02T13:52:55.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:55 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1450708826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:52:55.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:55 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1450708826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:52:55.857 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:52:55.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:55 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1450708826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:52:56.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:56 smithi052 ceph-mon[32104]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:52:56.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:56 smithi052 ceph-mon[32104]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-02T13:52:56.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:56 smithi189 ceph-mon[36683]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:52:56.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:56 smithi189 ceph-mon[36683]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-02T13:52:56.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:56 smithi088 ceph-mon[36528]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:52:56.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:56 smithi088 ceph-mon[36528]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-02T13:52:57.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.670+0000 7f3eebb56700 1 -- 172.21.15.52:0/2727555477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee4103150 msgr2=0x7f3ee4105540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:57.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.670+0000 7f3eebb56700 1 --2- 172.21.15.52:0/2727555477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee4103150 0x7f3ee4105540 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3edc009990 tx=0x7f3edc009ca0 comp rx=0 tx=0).stop 2024-04-02T13:52:57.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.670+0000 7f3eebb56700 1 -- 172.21.15.52:0/2727555477 shutdown_connections 2024-04-02T13:52:57.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.670+0000 7f3eebb56700 1 --2- 172.21.15.52:0/2727555477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ee41083b0 0x7f3ee410a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:57.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.670+0000 7f3eebb56700 1 --2- 172.21.15.52:0/2727555477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee4103150 0x7f3ee4105540 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:57.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.670+0000 7f3eebb56700 1 --2- 172.21.15.52:0/2727555477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ee4105a80 0x7f3ee4107e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:57.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.670+0000 7f3eebb56700 1 -- 172.21.15.52:0/2727555477 >> 172.21.15.52:0/2727555477 conn(0x7f3ee40fcac0 msgr2=0x7f3ee40fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:57.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.670+0000 7f3eebb56700 1 -- 172.21.15.52:0/2727555477 shutdown_connections 2024-04-02T13:52:57.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.670+0000 7f3eebb56700 1 -- 172.21.15.52:0/2727555477 wait complete. 2024-04-02T13:52:57.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.671+0000 7f3eebb56700 1 Processor -- start 2024-04-02T13:52:57.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.671+0000 7f3eebb56700 1 -- start start 2024-04-02T13:52:57.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.671+0000 7f3eebb56700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee4103150 0x7f3ee4102ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:57.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.671+0000 7f3eebb56700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ee4105a80 0x7f3ee4101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:57.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.671+0000 7f3eebb56700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ee41083b0 0x7f3ee4101640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:57.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.671+0000 7f3eebb56700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee410e1f0 con 0x7f3ee4103150 2024-04-02T13:52:57.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.671+0000 7f3eebb56700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee410e070 con 0x7f3ee41083b0 2024-04-02T13:52:57.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.671+0000 7f3eebb56700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee410e370 con 0x7f3ee4105a80 2024-04-02T13:52:57.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.671+0000 7f3ee98f2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee4103150 0x7f3ee4102ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:57.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.671+0000 7f3ee98f2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee4103150 0x7f3ee4102ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48864/0 (socket says 172.21.15.52:48864) 2024-04-02T13:52:57.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.671+0000 7f3ee98f2700 1 -- 172.21.15.52:0/1533679750 learned_addr learned my addr 172.21.15.52:0/1533679750 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:52:57.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.671+0000 7f3eea0f3700 1 --2- 172.21.15.52:0/1533679750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ee41083b0 0x7f3ee4101640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:57.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.671+0000 7f3ee90f1700 1 --2- 172.21.15.52:0/1533679750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ee4105a80 0x7f3ee4101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:57.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.672+0000 7f3ee98f2700 1 -- 172.21.15.52:0/1533679750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ee4105a80 msgr2=0x7f3ee4101100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:57.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.672+0000 7f3ee98f2700 1 --2- 172.21.15.52:0/1533679750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ee4105a80 0x7f3ee4101100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:57.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.672+0000 7f3ee98f2700 1 -- 172.21.15.52:0/1533679750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ee41083b0 msgr2=0x7f3ee4101640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:57.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.672+0000 7f3ee98f2700 1 --2- 172.21.15.52:0/1533679750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ee41083b0 0x7f3ee4101640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:57.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.672+0000 7f3ee98f2700 1 -- 172.21.15.52:0/1533679750 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee4101ef0 con 0x7f3ee4103150 2024-04-02T13:52:57.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.672+0000 7f3ee90f1700 1 --2- 172.21.15.52:0/1533679750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ee4105a80 0x7f3ee4101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:52:57.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.672+0000 7f3ee98f2700 1 --2- 172.21.15.52:0/1533679750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee4103150 0x7f3ee4102ab0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0098e0 tx=0x7f3edc009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:57.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.672+0000 7f3ee2ffd700 1 -- 172.21.15.52:0/1533679750 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc027070 con 0x7f3ee4103150 2024-04-02T13:52:57.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.672+0000 7f3eebb56700 1 -- 172.21.15.52:0/1533679750 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee41a29a0 con 0x7f3ee4103150 2024-04-02T13:52:57.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.672+0000 7f3ee2ffd700 1 -- 172.21.15.52:0/1533679750 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3edc004150 con 0x7f3ee4103150 2024-04-02T13:52:57.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.672+0000 7f3ee2ffd700 1 -- 172.21.15.52:0/1533679750 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc00f460 con 0x7f3ee4103150 2024-04-02T13:52:57.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.672+0000 7f3eebb56700 1 -- 172.21.15.52:0/1533679750 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee41a2df0 con 0x7f3ee4103150 2024-04-02T13:52:57.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.673+0000 7f3eebb56700 1 -- 172.21.15.52:0/1533679750 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee40692b0 con 0x7f3ee4103150 2024-04-02T13:52:57.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.674+0000 7f3ee2ffd700 1 -- 172.21.15.52:0/1533679750 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3edc02a070 con 0x7f3ee4103150 2024-04-02T13:52:57.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.674+0000 7f3ee2ffd700 1 --2- 172.21.15.52:0/1533679750 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ecc0a3c40 0x7f3ecc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:57.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.675+0000 7f3ee90f1700 1 --2- 172.21.15.52:0/1533679750 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ecc0a3c40 0x7f3ecc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:57.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.675+0000 7f3ee2ffd700 1 -- 172.21.15.52:0/1533679750 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f3edc0d1550 con 0x7f3ee4103150 2024-04-02T13:52:57.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.675+0000 7f3ee90f1700 1 --2- 172.21.15.52:0/1533679750 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ecc0a3c40 0x7f3ecc0a6100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8005fd0 tx=0x7f3ed8005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:57.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.679+0000 7f3ee2ffd700 1 -- 172.21.15.52:0/1533679750 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3edc013070 con 0x7f3ee4103150 2024-04-02T13:52:57.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.907+0000 7f3eebb56700 1 -- 172.21.15.52:0/1533679750 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f3ee4062a30 con 0x7f3ecc0a3c40 2024-04-02T13:52:57.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.908+0000 7f3ee2ffd700 1 -- 172.21.15.52:0/1533679750 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f3ee4062a30 con 0x7f3ecc0a3c40 2024-04-02T13:52:57.912 INFO:teuthology.orchestra.run.smithi052.stdout:Backend: cephadm 2024-04-02T13:52:57.912 INFO:teuthology.orchestra.run.smithi052.stdout:Available: Yes 2024-04-02T13:52:57.912 INFO:teuthology.orchestra.run.smithi052.stdout:Paused: No 2024-04-02T13:52:57.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.911+0000 7f3eebb56700 1 -- 172.21.15.52:0/1533679750 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ecc0a3c40 msgr2=0x7f3ecc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:57.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.911+0000 7f3eebb56700 1 --2- 172.21.15.52:0/1533679750 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ecc0a3c40 0x7f3ecc0a6100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8005fd0 tx=0x7f3ed8005ee0 comp rx=0 tx=0).stop 2024-04-02T13:52:57.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.911+0000 7f3eebb56700 1 -- 172.21.15.52:0/1533679750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee4103150 msgr2=0x7f3ee4102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:57.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.911+0000 7f3eebb56700 1 --2- 172.21.15.52:0/1533679750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee4103150 0x7f3ee4102ab0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0098e0 tx=0x7f3edc009ee0 comp rx=0 tx=0).stop 2024-04-02T13:52:57.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.911+0000 7f3eebb56700 1 -- 172.21.15.52:0/1533679750 shutdown_connections 2024-04-02T13:52:57.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.911+0000 7f3eebb56700 1 --2- 172.21.15.52:0/1533679750 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ecc0a3c40 0x7f3ecc0a6100 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:57.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.911+0000 7f3eebb56700 1 --2- 172.21.15.52:0/1533679750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ee41083b0 0x7f3ee4101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:57.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.911+0000 7f3eebb56700 1 --2- 172.21.15.52:0/1533679750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee4103150 0x7f3ee4102ab0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:57.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.911+0000 7f3eebb56700 1 --2- 172.21.15.52:0/1533679750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ee4105a80 0x7f3ee4101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:57.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.911+0000 7f3eebb56700 1 -- 172.21.15.52:0/1533679750 >> 172.21.15.52:0/1533679750 conn(0x7f3ee40fcac0 msgr2=0x7f3ee4103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:57.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.911+0000 7f3eebb56700 1 -- 172.21.15.52:0/1533679750 shutdown_connections 2024-04-02T13:52:57.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:52:57.911+0000 7f3eebb56700 1 -- 172.21.15.52:0/1533679750 wait complete. 2024-04-02T13:52:58.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:58 smithi052 ceph-mon[32104]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-02T13:52:58.713 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch ps' 2024-04-02T13:52:58.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:58 smithi189 ceph-mon[36683]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-02T13:52:58.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:58 smithi088 ceph-mon[36528]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-02T13:52:58.963 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:52:59.782 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:52:59 smithi052 ceph-mon[32104]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:52:59.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:52:59 smithi189 ceph-mon[36683]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:52:59.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:52:59 smithi088 ceph-mon[36528]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:00.643 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:00 smithi052 ceph-mon[32104]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-02T13:53:00.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.631+0000 7fbdbaa55700 1 -- 172.21.15.52:0/1984807645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdb4102640 msgr2=0x7fbdb4102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:00.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.631+0000 7fbdbaa55700 1 --2- 172.21.15.52:0/1984807645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdb4102640 0x7fbdb4102a60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbda40099c0 tx=0x7fbda4009cd0 comp rx=0 tx=0).stop 2024-04-02T13:53:00.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.631+0000 7fbdbaa55700 1 -- 172.21.15.52:0/1984807645 shutdown_connections 2024-04-02T13:53:00.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.631+0000 7fbdbaa55700 1 --2- 172.21.15.52:0/1984807645 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdb41041e0 0x7fbdb410aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:00.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.631+0000 7fbdbaa55700 1 --2- 172.21.15.52:0/1984807645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdb4102640 0x7fbdb4102a60 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:00.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.631+0000 7fbdbaa55700 1 --2- 172.21.15.52:0/1984807645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbdb4103840 0x7fbdb4103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:00.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.631+0000 7fbdbaa55700 1 -- 172.21.15.52:0/1984807645 >> 172.21.15.52:0/1984807645 conn(0x7fbdb40fdba0 msgr2=0x7fbdb4100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:00.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.631+0000 7fbdbaa55700 1 -- 172.21.15.52:0/1984807645 shutdown_connections 2024-04-02T13:53:00.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.631+0000 7fbdbaa55700 1 -- 172.21.15.52:0/1984807645 wait complete. 2024-04-02T13:53:00.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.631+0000 7fbdbaa55700 1 Processor -- start 2024-04-02T13:53:00.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.632+0000 7fbdbaa55700 1 -- start start 2024-04-02T13:53:00.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.632+0000 7fbdbaa55700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbdb4102640 0x7fbdb4076390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:00.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.632+0000 7fbdbaa55700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdb4103840 0x7fbdb40749e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:00.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.632+0000 7fbdbaa55700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdb41041e0 0x7fbdb4074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:00.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.632+0000 7fbdbaa55700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb410e200 con 0x7fbdb41041e0 2024-04-02T13:53:00.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.632+0000 7fbdbaa55700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb410e080 con 0x7fbdb4103840 2024-04-02T13:53:00.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.632+0000 7fbdbaa55700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb410e380 con 0x7fbdb4102640 2024-04-02T13:53:00.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.632+0000 7fbdb8ff2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdb41041e0 0x7fbdb4074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:00.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.632+0000 7fbdb8ff2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdb41041e0 0x7fbdb4074f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48882/0 (socket says 172.21.15.52:48882) 2024-04-02T13:53:00.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.632+0000 7fbdb8ff2700 1 -- 172.21.15.52:0/3770270495 learned_addr learned my addr 172.21.15.52:0/3770270495 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:00.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.632+0000 7fbdb37fe700 1 --2- 172.21.15.52:0/3770270495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdb4103840 0x7fbdb40749e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:00.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.632+0000 7fbdb3fff700 1 --2- 172.21.15.52:0/3770270495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbdb4102640 0x7fbdb4076390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:00.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.632+0000 7fbdb8ff2700 1 -- 172.21.15.52:0/3770270495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbdb4102640 msgr2=0x7fbdb4076390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:00.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.632+0000 7fbdb8ff2700 1 --2- 172.21.15.52:0/3770270495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbdb4102640 0x7fbdb4076390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:00.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.632+0000 7fbdb8ff2700 1 -- 172.21.15.52:0/3770270495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdb4103840 msgr2=0x7fbdb40749e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:00.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.632+0000 7fbdb8ff2700 1 --2- 172.21.15.52:0/3770270495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdb4103840 0x7fbdb40749e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:00.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.632+0000 7fbdb8ff2700 1 -- 172.21.15.52:0/3770270495 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb40757d0 con 0x7fbdb41041e0 2024-04-02T13:53:00.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.632+0000 7fbdb8ff2700 1 --2- 172.21.15.52:0/3770270495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdb41041e0 0x7fbdb4074f20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbdac007d70 tx=0x7fbdac00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:00.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.633+0000 7fbdb17fa700 1 -- 172.21.15.52:0/3770270495 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdac016020 con 0x7fbdb41041e0 2024-04-02T13:53:00.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.633+0000 7fbdb17fa700 1 -- 172.21.15.52:0/3770270495 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdac00ad40 con 0x7fbdb41041e0 2024-04-02T13:53:00.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.633+0000 7fbdbaa55700 1 -- 172.21.15.52:0/3770270495 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb41a2950 con 0x7fbdb41041e0 2024-04-02T13:53:00.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.633+0000 7fbdb17fa700 1 -- 172.21.15.52:0/3770270495 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdac011600 con 0x7fbdb41041e0 2024-04-02T13:53:00.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.633+0000 7fbdbaa55700 1 -- 172.21.15.52:0/3770270495 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb41a2e10 con 0x7fbdb41041e0 2024-04-02T13:53:00.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.634+0000 7fbdbaa55700 1 -- 172.21.15.52:0/3770270495 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdb40692b0 con 0x7fbdb41041e0 2024-04-02T13:53:00.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.635+0000 7fbdb17fa700 1 -- 172.21.15.52:0/3770270495 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbdac019020 con 0x7fbdb41041e0 2024-04-02T13:53:00.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.635+0000 7fbdb17fa700 1 --2- 172.21.15.52:0/3770270495 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fbd9c0a3c40 0x7fbd9c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:00.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.635+0000 7fbdb3fff700 1 --2- 172.21.15.52:0/3770270495 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fbd9c0a3c40 0x7fbd9c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:00.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.635+0000 7fbdb17fa700 1 -- 172.21.15.52:0/3770270495 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fbdac0d06f0 con 0x7fbdb41041e0 2024-04-02T13:53:00.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.636+0000 7fbdb3fff700 1 --2- 172.21.15.52:0/3770270495 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fbd9c0a3c40 0x7fbd9c0a6100 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbda400b580 tx=0x7fbda4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:00.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.640+0000 7fbdb17fa700 1 -- 172.21.15.52:0/3770270495 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fbdac020020 con 0x7fbdb41041e0 2024-04-02T13:53:00.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:00 smithi189 ceph-mon[36683]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-02T13:53:00.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.860+0000 7fbdbaa55700 1 -- 172.21.15.52:0/3770270495 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fbdb4062a30 con 0x7fbd9c0a3c40 2024-04-02T13:53:00.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.874+0000 7fbdb17fa700 1 -- 172.21.15.52:0/3770270495 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7fbdb4062a30 con 0x7fbd9c0a3c40 2024-04-02T13:53:00.878 INFO:teuthology.orchestra.run.smithi052.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T13:53:00.878 INFO:teuthology.orchestra.run.smithi052.stdout:mds.a smithi052 running (2m) 23s ago 2m 18.7M - 19.0.0-2609-g191d963a f7301e070545 c4bf472e5cd1 2024-04-02T13:53:00.878 INFO:teuthology.orchestra.run.smithi052.stdout:mds.b smithi088 running (2m) 59s ago 2m 26.3M - 19.0.0-2609-g191d963a f7301e070545 3189802bb956 2024-04-02T13:53:00.878 INFO:teuthology.orchestra.run.smithi052.stdout:mds.c smithi189 running (2m) 59s ago 2m 17.5M - 19.0.0-2609-g191d963a f7301e070545 da47ce00bf48 2024-04-02T13:53:00.878 INFO:teuthology.orchestra.run.smithi052.stdout:mds.d smithi052 running (2m) 23s ago 2m 14.0M - 19.0.0-2609-g191d963a f7301e070545 ef988063f2ad 2024-04-02T13:53:00.878 INFO:teuthology.orchestra.run.smithi052.stdout:mds.e smithi088 running (2m) 59s ago 2m 16.2M - 19.0.0-2609-g191d963a f7301e070545 ee6638eee180 2024-04-02T13:53:00.878 INFO:teuthology.orchestra.run.smithi052.stdout:mds.f smithi189 running (2m) 59s ago 2m 17.2M - 19.0.0-2609-g191d963a f7301e070545 67fc9458e9f5 2024-04-02T13:53:00.878 INFO:teuthology.orchestra.run.smithi052.stdout:mds.g smithi052 running (2m) 23s ago 2m 16.2M - 19.0.0-2609-g191d963a f7301e070545 111cee980215 2024-04-02T13:53:00.878 INFO:teuthology.orchestra.run.smithi052.stdout:mds.h smithi088 running (2m) 59s ago 2m 18.7M - 19.0.0-2609-g191d963a f7301e070545 db0696c6a376 2024-04-02T13:53:00.878 INFO:teuthology.orchestra.run.smithi052.stdout:mds.i smithi189 running (2m) 59s ago 2m 17.6M - 19.0.0-2609-g191d963a f7301e070545 ad384f043f4a 2024-04-02T13:53:00.878 INFO:teuthology.orchestra.run.smithi052.stdout:mds.j smithi052 running (2m) 23s ago 2m 15.4M - 19.0.0-2609-g191d963a f7301e070545 566dd4642c00 2024-04-02T13:53:00.879 INFO:teuthology.orchestra.run.smithi052.stdout:mds.k smithi088 running (2m) 59s ago 2m 18.4M - 19.0.0-2609-g191d963a f7301e070545 0f8f26af2583 2024-04-02T13:53:00.879 INFO:teuthology.orchestra.run.smithi052.stdout:mds.l smithi189 running (2m) 59s ago 2m 18.5M - 19.0.0-2609-g191d963a f7301e070545 82ee271879cb 2024-04-02T13:53:00.879 INFO:teuthology.orchestra.run.smithi052.stdout:mgr.x smithi052 *:9283,8765 running (9m) 23s ago 9m 515M - 19.0.0-2609-g191d963a f7301e070545 c88f5d3e9402 2024-04-02T13:53:00.879 INFO:teuthology.orchestra.run.smithi052.stdout:mgr.y smithi088 *:8443,8765 running (7m) 59s ago 7m 453M - 19.0.0-2609-g191d963a f7301e070545 671f18422f67 2024-04-02T13:53:00.879 INFO:teuthology.orchestra.run.smithi052.stdout:mgr.z smithi189 *:8443,8765 running (7m) 59s ago 7m 452M - 19.0.0-2609-g191d963a f7301e070545 a5b65e91ec36 2024-04-02T13:53:00.879 INFO:teuthology.orchestra.run.smithi052.stdout:mon.a smithi052 running (9m) 23s ago 10m 50.5M 2048M 19.0.0-2609-g191d963a f7301e070545 bcb19f26fc28 2024-04-02T13:53:00.879 INFO:teuthology.orchestra.run.smithi052.stdout:mon.b smithi088 running (8m) 59s ago 8m 40.8M 2048M 19.0.0-2609-g191d963a f7301e070545 9e66ff83e3f9 2024-04-02T13:53:00.879 INFO:teuthology.orchestra.run.smithi052.stdout:mon.c smithi189 running (8m) 59s ago 8m 43.4M 2048M 19.0.0-2609-g191d963a f7301e070545 7cca3de29637 2024-04-02T13:53:00.879 INFO:teuthology.orchestra.run.smithi052.stdout:osd.0 smithi052 running (7m) 23s ago 7m 53.5M 4273M 19.0.0-2609-g191d963a f7301e070545 ca8f74146fef 2024-04-02T13:53:00.879 INFO:teuthology.orchestra.run.smithi052.stdout:osd.1 smithi088 running (7m) 59s ago 7m 54.4M 4270M 19.0.0-2609-g191d963a f7301e070545 a59bbc6312dd 2024-04-02T13:53:00.879 INFO:teuthology.orchestra.run.smithi052.stdout:osd.2 smithi189 running (6m) 59s ago 6m 55.0M 4270M 19.0.0-2609-g191d963a f7301e070545 1825481ddb97 2024-04-02T13:53:00.879 INFO:teuthology.orchestra.run.smithi052.stdout:osd.3 smithi052 running (6m) 23s ago 6m 51.9M 4273M 19.0.0-2609-g191d963a f7301e070545 dc6b29dc4c94 2024-04-02T13:53:00.879 INFO:teuthology.orchestra.run.smithi052.stdout:osd.4 smithi088 running (5m) 59s ago 5m 50.3M 4270M 19.0.0-2609-g191d963a f7301e070545 9fbce6bec939 2024-04-02T13:53:00.879 INFO:teuthology.orchestra.run.smithi052.stdout:osd.5 smithi189 running (5m) 59s ago 5m 47.6M 4270M 19.0.0-2609-g191d963a f7301e070545 68e83f11dec8 2024-04-02T13:53:00.880 INFO:teuthology.orchestra.run.smithi052.stdout:osd.6 smithi052 running (5m) 23s ago 5m 49.0M 4273M 19.0.0-2609-g191d963a f7301e070545 569101ebe81f 2024-04-02T13:53:00.880 INFO:teuthology.orchestra.run.smithi052.stdout:osd.7 smithi088 running (4m) 59s ago 4m 52.6M 4270M 19.0.0-2609-g191d963a f7301e070545 37961c3b4317 2024-04-02T13:53:00.880 INFO:teuthology.orchestra.run.smithi052.stdout:osd.8 smithi189 running (4m) 59s ago 4m 51.7M 4270M 19.0.0-2609-g191d963a f7301e070545 34c22a605081 2024-04-02T13:53:00.880 INFO:teuthology.orchestra.run.smithi052.stdout:osd.9 smithi052 running (3m) 23s ago 3m 45.0M 4273M 19.0.0-2609-g191d963a f7301e070545 3f40d09a3764 2024-04-02T13:53:00.880 INFO:teuthology.orchestra.run.smithi052.stdout:osd.10 smithi088 running (3m) 59s ago 3m 44.1M 4270M 19.0.0-2609-g191d963a f7301e070545 aced5a2df980 2024-04-02T13:53:00.880 INFO:teuthology.orchestra.run.smithi052.stdout:osd.11 smithi189 running (2m) 59s ago 2m 50.0M 4270M 19.0.0-2609-g191d963a f7301e070545 d07d621d0db0 2024-04-02T13:53:00.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.877+0000 7fbdbaa55700 1 -- 172.21.15.52:0/3770270495 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fbd9c0a3c40 msgr2=0x7fbd9c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:00.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.877+0000 7fbdbaa55700 1 --2- 172.21.15.52:0/3770270495 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fbd9c0a3c40 0x7fbd9c0a6100 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbda400b580 tx=0x7fbda4005fb0 comp rx=0 tx=0).stop 2024-04-02T13:53:00.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.877+0000 7fbdbaa55700 1 -- 172.21.15.52:0/3770270495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdb41041e0 msgr2=0x7fbdb4074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:00.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.877+0000 7fbdbaa55700 1 --2- 172.21.15.52:0/3770270495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdb41041e0 0x7fbdb4074f20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbdac007d70 tx=0x7fbdac00a340 comp rx=0 tx=0).stop 2024-04-02T13:53:00.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.877+0000 7fbdbaa55700 1 -- 172.21.15.52:0/3770270495 shutdown_connections 2024-04-02T13:53:00.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.877+0000 7fbdbaa55700 1 --2- 172.21.15.52:0/3770270495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdb41041e0 0x7fbdb4074f20 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:00.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.877+0000 7fbdbaa55700 1 --2- 172.21.15.52:0/3770270495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbdb4102640 0x7fbdb4076390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:00.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.877+0000 7fbdbaa55700 1 --2- 172.21.15.52:0/3770270495 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fbd9c0a3c40 0x7fbd9c0a6100 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:00.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.877+0000 7fbdbaa55700 1 --2- 172.21.15.52:0/3770270495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdb4103840 0x7fbdb40749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:00.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.877+0000 7fbdbaa55700 1 -- 172.21.15.52:0/3770270495 >> 172.21.15.52:0/3770270495 conn(0x7fbdb40fdba0 msgr2=0x7fbdb4106ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:00.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.877+0000 7fbdbaa55700 1 -- 172.21.15.52:0/3770270495 shutdown_connections 2024-04-02T13:53:00.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:00.877+0000 7fbdbaa55700 1 -- 172.21.15.52:0/3770270495 wait complete. 2024-04-02T13:53:00.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:00 smithi088 ceph-mon[36528]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-02T13:53:01.639 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch ls' 2024-04-02T13:53:01.888 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:53:02.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:02 smithi052 ceph-mon[32104]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-02T13:53:02.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:02 smithi052 ceph-mon[32104]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:02.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:02 smithi189 ceph-mon[36683]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-02T13:53:02.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:02 smithi189 ceph-mon[36683]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:02.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:02 smithi088 ceph-mon[36528]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-02T13:53:02.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:02 smithi088 ceph-mon[36528]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:03.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.601+0000 7f659eb19700 1 -- 172.21.15.52:0/2970942918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6598102720 msgr2=0x7f6598102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:03.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.601+0000 7f659eb19700 1 --2- 172.21.15.52:0/2970942918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6598102720 0x7f6598102b80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f65840099c0 tx=0x7f6584009cd0 comp rx=0 tx=0).stop 2024-04-02T13:53:03.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.601+0000 7f659eb19700 1 -- 172.21.15.52:0/2970942918 shutdown_connections 2024-04-02T13:53:03.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.601+0000 7f659eb19700 1 --2- 172.21.15.52:0/2970942918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65981030c0 0x7f6598109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:03.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.601+0000 7f659eb19700 1 --2- 172.21.15.52:0/2970942918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6598101520 0x7f6598101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:03.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.601+0000 7f659eb19700 1 --2- 172.21.15.52:0/2970942918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6598102720 0x7f6598102b80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:03.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.601+0000 7f659eb19700 1 -- 172.21.15.52:0/2970942918 >> 172.21.15.52:0/2970942918 conn(0x7f65980fcac0 msgr2=0x7f65980fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:03.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.601+0000 7f659eb19700 1 -- 172.21.15.52:0/2970942918 shutdown_connections 2024-04-02T13:53:03.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.601+0000 7f659eb19700 1 -- 172.21.15.52:0/2970942918 wait complete. 2024-04-02T13:53:03.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.602+0000 7f659eb19700 1 Processor -- start 2024-04-02T13:53:03.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.602+0000 7f659eb19700 1 -- start start 2024-04-02T13:53:03.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.602+0000 7f659eb19700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6598101520 0x7f65981988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:03.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.602+0000 7f659eb19700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6598102720 0x7f6598198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:03.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.602+0000 7f659eb19700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65981030c0 0x7f659819fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:03.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.602+0000 7f659eb19700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f659810d100 con 0x7f6598102720 2024-04-02T13:53:03.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.602+0000 7f659eb19700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f659810cf80 con 0x7f6598101520 2024-04-02T13:53:03.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.602+0000 7f659eb19700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f659810d280 con 0x7f65981030c0 2024-04-02T13:53:03.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.602+0000 7f6597fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6598102720 0x7f6598198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:03.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.602+0000 7f6597fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6598102720 0x7f6598198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57710/0 (socket says 172.21.15.52:57710) 2024-04-02T13:53:03.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.602+0000 7f6597fff700 1 -- 172.21.15.52:0/2673209654 learned_addr learned my addr 172.21.15.52:0/2673209654 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:03.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.602+0000 7f659c8b5700 1 --2- 172.21.15.52:0/2673209654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6598101520 0x7f65981988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:03.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.602+0000 7f659d0b6700 1 --2- 172.21.15.52:0/2673209654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65981030c0 0x7f659819fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:03.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.603+0000 7f6597fff700 1 -- 172.21.15.52:0/2673209654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65981030c0 msgr2=0x7f659819fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:03.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.603+0000 7f6597fff700 1 --2- 172.21.15.52:0/2673209654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65981030c0 0x7f659819fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:03.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.603+0000 7f6597fff700 1 -- 172.21.15.52:0/2673209654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6598101520 msgr2=0x7f65981988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:03.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.603+0000 7f6597fff700 1 --2- 172.21.15.52:0/2673209654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6598101520 0x7f65981988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:03.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.603+0000 7f6597fff700 1 -- 172.21.15.52:0/2673209654 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65981a04b0 con 0x7f6598102720 2024-04-02T13:53:03.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.603+0000 7f6597fff700 1 --2- 172.21.15.52:0/2673209654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6598102720 0x7f6598198de0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6584000c00 tx=0x7f6584009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:03.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.603+0000 7f6595ffb700 1 -- 172.21.15.52:0/2673209654 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6584026070 con 0x7f6598102720 2024-04-02T13:53:03.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.603+0000 7f659eb19700 1 -- 172.21.15.52:0/2673209654 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65981a0740 con 0x7f6598102720 2024-04-02T13:53:03.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.603+0000 7f6595ffb700 1 -- 172.21.15.52:0/2673209654 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6584005510 con 0x7f6598102720 2024-04-02T13:53:03.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.603+0000 7f6595ffb700 1 -- 172.21.15.52:0/2673209654 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6584003cc0 con 0x7f6598102720 2024-04-02T13:53:03.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.603+0000 7f659eb19700 1 -- 172.21.15.52:0/2673209654 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65981a0bf0 con 0x7f6598102720 2024-04-02T13:53:03.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.604+0000 7f659eb19700 1 -- 172.21.15.52:0/2673209654 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65980692b0 con 0x7f6598102720 2024-04-02T13:53:03.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.605+0000 7f6595ffb700 1 -- 172.21.15.52:0/2673209654 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6584029070 con 0x7f6598102720 2024-04-02T13:53:03.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.606+0000 7f6595ffb700 1 --2- 172.21.15.52:0/2673209654 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f65880a3c40 0x7f65880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:03.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.606+0000 7f659c8b5700 1 --2- 172.21.15.52:0/2673209654 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f65880a3c40 0x7f65880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:03.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.606+0000 7f6595ffb700 1 -- 172.21.15.52:0/2673209654 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f65840d14d0 con 0x7f6598102720 2024-04-02T13:53:03.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.607+0000 7f659c8b5700 1 --2- 172.21.15.52:0/2673209654 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f65880a3c40 0x7f65880a6100 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6598102580 tx=0x7f658c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:03.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.610+0000 7f6595ffb700 1 -- 172.21.15.52:0/2673209654 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6584030020 con 0x7f6598102720 2024-04-02T13:53:03.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.827+0000 7f659eb19700 1 -- 172.21.15.52:0/2673209654 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f65980fecb0 con 0x7f65880a3c40 2024-04-02T13:53:03.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.833+0000 7f6595ffb700 1 -- 172.21.15.52:0/2673209654 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f65980fecb0 con 0x7f65880a3c40 2024-04-02T13:53:03.836 INFO:teuthology.orchestra.run.smithi052.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-02T13:53:03.836 INFO:teuthology.orchestra.run.smithi052.stdout:mds.all 12/12 62s ago 2m smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi088=b;smithi088=e;smithi088=h;smithi088=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-04-02T13:53:03.836 INFO:teuthology.orchestra.run.smithi052.stdout:mgr 3/3 62s ago 8m smithi052=x;smithi088=y;smithi189=z;count:3 2024-04-02T13:53:03.836 INFO:teuthology.orchestra.run.smithi052.stdout:mon 3/3 62s ago 8m smithi052:172.21.15.52=a;smithi088:172.21.15.88=b;smithi189:172.21.15.189=c;count:3 2024-04-02T13:53:03.837 INFO:teuthology.orchestra.run.smithi052.stdout:osd 12 62s ago - 2024-04-02T13:53:03.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.835+0000 7f659eb19700 1 -- 172.21.15.52:0/2673209654 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f65880a3c40 msgr2=0x7f65880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:03.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.835+0000 7f659eb19700 1 --2- 172.21.15.52:0/2673209654 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f65880a3c40 0x7f65880a6100 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6598102580 tx=0x7f658c00b410 comp rx=0 tx=0).stop 2024-04-02T13:53:03.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.835+0000 7f659eb19700 1 -- 172.21.15.52:0/2673209654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6598102720 msgr2=0x7f6598198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:03.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.835+0000 7f659eb19700 1 --2- 172.21.15.52:0/2673209654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6598102720 0x7f6598198de0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6584000c00 tx=0x7f6584009f10 comp rx=0 tx=0).stop 2024-04-02T13:53:03.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.835+0000 7f659eb19700 1 -- 172.21.15.52:0/2673209654 shutdown_connections 2024-04-02T13:53:03.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.835+0000 7f659eb19700 1 --2- 172.21.15.52:0/2673209654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65981030c0 0x7f659819fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:03.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.835+0000 7f659eb19700 1 --2- 172.21.15.52:0/2673209654 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f65880a3c40 0x7f65880a6100 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:03.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.835+0000 7f659eb19700 1 --2- 172.21.15.52:0/2673209654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6598101520 0x7f65981988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:03.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.835+0000 7f659eb19700 1 --2- 172.21.15.52:0/2673209654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6598102720 0x7f6598198de0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:03.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.835+0000 7f659eb19700 1 -- 172.21.15.52:0/2673209654 >> 172.21.15.52:0/2673209654 conn(0x7f65980fcac0 msgr2=0x7f65980fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:03.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.835+0000 7f659eb19700 1 -- 172.21.15.52:0/2673209654 shutdown_connections 2024-04-02T13:53:03.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:03.835+0000 7f659eb19700 1 -- 172.21.15.52:0/2673209654 wait complete. 2024-04-02T13:53:04.483 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch host ls' 2024-04-02T13:53:04.728 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:53:04.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:04 smithi189 ceph-mon[36683]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-02T13:53:04.838 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:04 smithi052 ceph-mon[32104]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-02T13:53:04.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:04 smithi088 ceph-mon[36528]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-02T13:53:05.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:05 smithi189 ceph-mon[36683]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:05.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:05 smithi052 ceph-mon[32104]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:05.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:05 smithi088 ceph-mon[36528]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:06.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.512+0000 7fdbbb46c700 1 -- 172.21.15.52:0/2765962406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbb4101520 msgr2=0x7fdbb4101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:06.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.512+0000 7fdbbb46c700 1 --2- 172.21.15.52:0/2765962406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbb4101520 0x7fdbb4101940 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdbb00099c0 tx=0x7fdbb0009cd0 comp rx=0 tx=0).stop 2024-04-02T13:53:06.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.512+0000 7fdbbb46c700 1 -- 172.21.15.52:0/2765962406 shutdown_connections 2024-04-02T13:53:06.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.512+0000 7fdbbb46c700 1 --2- 172.21.15.52:0/2765962406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbb41030c0 0x7fdbb4109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:06.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.512+0000 7fdbbb46c700 1 --2- 172.21.15.52:0/2765962406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbb4101520 0x7fdbb4101940 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:06.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.512+0000 7fdbbb46c700 1 --2- 172.21.15.52:0/2765962406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbb4102720 0x7fdbb4102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:06.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.512+0000 7fdbbb46c700 1 -- 172.21.15.52:0/2765962406 >> 172.21.15.52:0/2765962406 conn(0x7fdbb40fcae0 msgr2=0x7fdbb40fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:06.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.512+0000 7fdbbb46c700 1 -- 172.21.15.52:0/2765962406 shutdown_connections 2024-04-02T13:53:06.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.512+0000 7fdbbb46c700 1 -- 172.21.15.52:0/2765962406 wait complete. 2024-04-02T13:53:06.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.513+0000 7fdbbb46c700 1 Processor -- start 2024-04-02T13:53:06.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.513+0000 7fdbbb46c700 1 -- start start 2024-04-02T13:53:06.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.513+0000 7fdbbb46c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbb4101520 0x7fdbb4197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:06.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.513+0000 7fdbbb46c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbb4102720 0x7fdbb4197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:06.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.513+0000 7fdbbb46c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbb41030c0 0x7fdbb419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:06.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.513+0000 7fdbbb46c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb410d100 con 0x7fdbb41030c0 2024-04-02T13:53:06.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.513+0000 7fdbbb46c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb410cf80 con 0x7fdbb4102720 2024-04-02T13:53:06.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.513+0000 7fdbbb46c700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb410d280 con 0x7fdbb4101520 2024-04-02T13:53:06.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.513+0000 7fdbb9a09700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbb41030c0 0x7fdbb419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:06.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.513+0000 7fdbb9a09700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbb41030c0 0x7fdbb419ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57732/0 (socket says 172.21.15.52:57732) 2024-04-02T13:53:06.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.513+0000 7fdbb9a09700 1 -- 172.21.15.52:0/1079893118 learned_addr learned my addr 172.21.15.52:0/1079893118 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:06.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.513+0000 7fdbb8a07700 1 --2- 172.21.15.52:0/1079893118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbb4102720 0x7fdbb4197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:06.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.513+0000 7fdbb9208700 1 --2- 172.21.15.52:0/1079893118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbb4101520 0x7fdbb4197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:06.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.514+0000 7fdbb9a09700 1 -- 172.21.15.52:0/1079893118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbb4101520 msgr2=0x7fdbb4197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:06.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.514+0000 7fdbb9a09700 1 --2- 172.21.15.52:0/1079893118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbb4101520 0x7fdbb4197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:06.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.514+0000 7fdbb9a09700 1 -- 172.21.15.52:0/1079893118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbb4102720 msgr2=0x7fdbb4197cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:06.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.514+0000 7fdbb9a09700 1 --2- 172.21.15.52:0/1079893118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbb4102720 0x7fdbb4197cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:06.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.514+0000 7fdbb9a09700 1 -- 172.21.15.52:0/1079893118 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb419f3d0 con 0x7fdbb41030c0 2024-04-02T13:53:06.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.514+0000 7fdbb9a09700 1 --2- 172.21.15.52:0/1079893118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbb41030c0 0x7fdbb419ed30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdbac007f10 tx=0x7fdbac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:06.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.514+0000 7fdbaa7fc700 1 -- 172.21.15.52:0/1079893118 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac004bb0 con 0x7fdbb41030c0 2024-04-02T13:53:06.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.514+0000 7fdbaa7fc700 1 -- 172.21.15.52:0/1079893118 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbac004d50 con 0x7fdbb41030c0 2024-04-02T13:53:06.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.514+0000 7fdbbb46c700 1 -- 172.21.15.52:0/1079893118 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb419f6c0 con 0x7fdbb41030c0 2024-04-02T13:53:06.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.514+0000 7fdbaa7fc700 1 -- 172.21.15.52:0/1079893118 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac0059f0 con 0x7fdbb41030c0 2024-04-02T13:53:06.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.514+0000 7fdbbb46c700 1 -- 172.21.15.52:0/1079893118 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb419fc00 con 0x7fdbb41030c0 2024-04-02T13:53:06.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.515+0000 7fdbbb46c700 1 -- 172.21.15.52:0/1079893118 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb40692b0 con 0x7fdbb41030c0 2024-04-02T13:53:06.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.516+0000 7fdbaa7fc700 1 -- 172.21.15.52:0/1079893118 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdbac01a020 con 0x7fdbb41030c0 2024-04-02T13:53:06.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.517+0000 7fdbaa7fc700 1 --2- 172.21.15.52:0/1079893118 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdba00a3cc0 0x7fdba00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:06.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.517+0000 7fdbb9208700 1 --2- 172.21.15.52:0/1079893118 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdba00a3cc0 0x7fdba00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:06.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.517+0000 7fdbaa7fc700 1 -- 172.21.15.52:0/1079893118 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdbac0cfe50 con 0x7fdbb41030c0 2024-04-02T13:53:06.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.517+0000 7fdbb9208700 1 --2- 172.21.15.52:0/1079893118 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdba00a3cc0 0x7fdba00a6180 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000b580 tx=0x7fdbb0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:06.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.522+0000 7fdbaa7fc700 1 -- 172.21.15.52:0/1079893118 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fdbac010070 con 0x7fdbb41030c0 2024-04-02T13:53:06.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.735+0000 7fdbbb46c700 1 -- 172.21.15.52:0/1079893118 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fdbb4062a30 con 0x7fdba00a3cc0 2024-04-02T13:53:06.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.736+0000 7fdbaa7fc700 1 -- 172.21.15.52:0/1079893118 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fdbb4062a30 con 0x7fdba00a3cc0 2024-04-02T13:53:06.740 INFO:teuthology.orchestra.run.smithi052.stdout:HOST ADDR LABELS STATUS 2024-04-02T13:53:06.740 INFO:teuthology.orchestra.run.smithi052.stdout:smithi052 172.21.15.52 2024-04-02T13:53:06.740 INFO:teuthology.orchestra.run.smithi052.stdout:smithi088 172.21.15.88 2024-04-02T13:53:06.740 INFO:teuthology.orchestra.run.smithi052.stdout:smithi189 172.21.15.189 2024-04-02T13:53:06.740 INFO:teuthology.orchestra.run.smithi052.stdout:3 hosts in cluster 2024-04-02T13:53:06.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.739+0000 7fdbbb46c700 1 -- 172.21.15.52:0/1079893118 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdba00a3cc0 msgr2=0x7fdba00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:06.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.739+0000 7fdbbb46c700 1 --2- 172.21.15.52:0/1079893118 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdba00a3cc0 0x7fdba00a6180 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000b580 tx=0x7fdbb0005fb0 comp rx=0 tx=0).stop 2024-04-02T13:53:06.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.739+0000 7fdbbb46c700 1 -- 172.21.15.52:0/1079893118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbb41030c0 msgr2=0x7fdbb419ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:06.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.739+0000 7fdbbb46c700 1 --2- 172.21.15.52:0/1079893118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbb41030c0 0x7fdbb419ed30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdbac007f10 tx=0x7fdbac00d3b0 comp rx=0 tx=0).stop 2024-04-02T13:53:06.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.739+0000 7fdbbb46c700 1 -- 172.21.15.52:0/1079893118 shutdown_connections 2024-04-02T13:53:06.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.739+0000 7fdbbb46c700 1 --2- 172.21.15.52:0/1079893118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbb41030c0 0x7fdbb419ed30 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:06.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.739+0000 7fdbbb46c700 1 --2- 172.21.15.52:0/1079893118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbb4101520 0x7fdbb4197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:06.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.739+0000 7fdbbb46c700 1 --2- 172.21.15.52:0/1079893118 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdba00a3cc0 0x7fdba00a6180 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:06.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.739+0000 7fdbbb46c700 1 --2- 172.21.15.52:0/1079893118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbb4102720 0x7fdbb4197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:06.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.739+0000 7fdbbb46c700 1 -- 172.21.15.52:0/1079893118 >> 172.21.15.52:0/1079893118 conn(0x7fdbb40fcae0 msgr2=0x7fdbb4105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:06.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.739+0000 7fdbbb46c700 1 -- 172.21.15.52:0/1079893118 shutdown_connections 2024-04-02T13:53:06.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:06.739+0000 7fdbbb46c700 1 -- 172.21.15.52:0/1079893118 wait complete. 2024-04-02T13:53:06.753 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:06 smithi052 ceph-mon[32104]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-02T13:53:06.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:06 smithi189 ceph-mon[36683]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-02T13:53:06.821 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:06 smithi088 ceph-mon[36528]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-02T13:53:07.502 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch device ls' 2024-04-02T13:53:07.760 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:53:08.776 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:08 smithi052 ceph-mon[32104]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:08.776 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:08 smithi052 ceph-mon[32104]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-02T13:53:08.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:08 smithi189 ceph-mon[36683]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:08.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:08 smithi189 ceph-mon[36683]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-02T13:53:08.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:08 smithi088 ceph-mon[36528]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:08.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:08 smithi088 ceph-mon[36528]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-04-02T13:53:09.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.598+0000 7f52c9bf4700 1 -- 172.21.15.52:0/2655351971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52c4103840 msgr2=0x7f52c4103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:09.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.598+0000 7f52c9bf4700 1 --2- 172.21.15.52:0/2655351971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52c4103840 0x7f52c4103ca0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f52b80099c0 tx=0x7f52b8009cd0 comp rx=0 tx=0).stop 2024-04-02T13:53:09.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.598+0000 7f52c9bf4700 1 -- 172.21.15.52:0/2655351971 shutdown_connections 2024-04-02T13:53:09.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.598+0000 7f52c9bf4700 1 --2- 172.21.15.52:0/2655351971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52c41041e0 0x7f52c410aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:09.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.598+0000 7f52c9bf4700 1 --2- 172.21.15.52:0/2655351971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52c4102640 0x7f52c4102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:09.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.598+0000 7f52c9bf4700 1 --2- 172.21.15.52:0/2655351971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52c4103840 0x7f52c4103ca0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:09.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.598+0000 7f52c9bf4700 1 -- 172.21.15.52:0/2655351971 >> 172.21.15.52:0/2655351971 conn(0x7f52c40fdc00 msgr2=0x7f52c4100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:09.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.598+0000 7f52c9bf4700 1 -- 172.21.15.52:0/2655351971 shutdown_connections 2024-04-02T13:53:09.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.598+0000 7f52c9bf4700 1 -- 172.21.15.52:0/2655351971 wait complete. 2024-04-02T13:53:09.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.598+0000 7f52c9bf4700 1 Processor -- start 2024-04-02T13:53:09.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.598+0000 7f52c9bf4700 1 -- start start 2024-04-02T13:53:09.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.599+0000 7f52c9bf4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52c4102640 0x7f52c4076390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:09.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.599+0000 7f52c9bf4700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52c4103840 0x7f52c40749e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:09.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.599+0000 7f52c9bf4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52c41041e0 0x7f52c4074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:09.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.599+0000 7f52c9bf4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f52c410e230 con 0x7f52c41041e0 2024-04-02T13:53:09.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.599+0000 7f52c9bf4700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f52c410e0b0 con 0x7f52c4102640 2024-04-02T13:53:09.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.599+0000 7f52c9bf4700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f52c410e3b0 con 0x7f52c4103840 2024-04-02T13:53:09.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.599+0000 7f52c3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52c41041e0 0x7f52c4074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:09.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.599+0000 7f52c3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52c41041e0 0x7f52c4074f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57750/0 (socket says 172.21.15.52:57750) 2024-04-02T13:53:09.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.599+0000 7f52c3fff700 1 -- 172.21.15.52:0/21897933 learned_addr learned my addr 172.21.15.52:0/21897933 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:09.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.599+0000 7f52c2ffd700 1 --2- 172.21.15.52:0/21897933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52c4103840 0x7f52c40749e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:09.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.599+0000 7f52c37fe700 1 --2- 172.21.15.52:0/21897933 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52c4102640 0x7f52c4076390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:09.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.599+0000 7f52c3fff700 1 -- 172.21.15.52:0/21897933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52c4103840 msgr2=0x7f52c40749e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:09.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.599+0000 7f52c3fff700 1 --2- 172.21.15.52:0/21897933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52c4103840 0x7f52c40749e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:09.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.599+0000 7f52c3fff700 1 -- 172.21.15.52:0/21897933 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52c4102640 msgr2=0x7f52c4076390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:09.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.599+0000 7f52c3fff700 1 --2- 172.21.15.52:0/21897933 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52c4102640 0x7f52c4076390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:09.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.599+0000 7f52c3fff700 1 -- 172.21.15.52:0/21897933 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c40757d0 con 0x7f52c41041e0 2024-04-02T13:53:09.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.599+0000 7f52c2ffd700 1 --2- 172.21.15.52:0/21897933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52c4103840 0x7f52c40749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:09.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.599+0000 7f52c3fff700 1 --2- 172.21.15.52:0/21897933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52c41041e0 0x7f52c4074f20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f52bc007ee0 tx=0x7f52bc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:09.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.599+0000 7f52c0ff9700 1 -- 172.21.15.52:0/21897933 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc00dbd0 con 0x7f52c41041e0 2024-04-02T13:53:09.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.599+0000 7f52c9bf4700 1 -- 172.21.15.52:0/21897933 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c41a29a0 con 0x7f52c41041e0 2024-04-02T13:53:09.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.600+0000 7f52c0ff9700 1 -- 172.21.15.52:0/21897933 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52bc00dd70 con 0x7f52c41041e0 2024-04-02T13:53:09.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.600+0000 7f52c0ff9700 1 -- 172.21.15.52:0/21897933 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc0133f0 con 0x7f52c41041e0 2024-04-02T13:53:09.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.600+0000 7f52c9bf4700 1 -- 172.21.15.52:0/21897933 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c41a2e60 con 0x7f52c41041e0 2024-04-02T13:53:09.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.600+0000 7f52c9bf4700 1 -- 172.21.15.52:0/21897933 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c40692b0 con 0x7f52c41041e0 2024-04-02T13:53:09.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.602+0000 7f52c0ff9700 1 -- 172.21.15.52:0/21897933 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f52bc01a020 con 0x7f52c41041e0 2024-04-02T13:53:09.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.602+0000 7f52c0ff9700 1 --2- 172.21.15.52:0/21897933 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f52ac0a3c40 0x7f52ac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:09.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.602+0000 7f52c0ff9700 1 -- 172.21.15.52:0/21897933 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f52bc0d05c0 con 0x7f52c41041e0 2024-04-02T13:53:09.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.602+0000 7f52c37fe700 1 --2- 172.21.15.52:0/21897933 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f52ac0a3c40 0x7f52ac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:09.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.603+0000 7f52c37fe700 1 --2- 172.21.15.52:0/21897933 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f52ac0a3c40 0x7f52ac0a6100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f52b4005950 tx=0x7f52b40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:09.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.606+0000 7f52c0ff9700 1 -- 172.21.15.52:0/21897933 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f52bc010070 con 0x7f52c41041e0 2024-04-02T13:53:09.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.823+0000 7f52c9bf4700 1 -- 172.21.15.52:0/21897933 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f52c4062a30 con 0x7f52ac0a3c40 2024-04-02T13:53:09.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.825+0000 7f52c0ff9700 1 -- 172.21.15.52:0/21897933 <== mgr.14152 v2:172.21.15.52:6800/3836085837 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f52c4062a30 con 0x7f52ac0a3c40 2024-04-02T13:53:09.829 INFO:teuthology.orchestra.run.smithi052.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-02T13:53:09.829 INFO:teuthology.orchestra.run.smithi052.stdout:smithi052 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513010N480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-02T13:53:09.829 INFO:teuthology.orchestra.run.smithi052.stdout:smithi088 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400XD400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-02T13:53:09.829 INFO:teuthology.orchestra.run.smithi052.stdout:smithi189 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6232004L400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-02T13:53:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.827+0000 7f52c9bf4700 1 -- 172.21.15.52:0/21897933 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f52ac0a3c40 msgr2=0x7f52ac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.827+0000 7f52c9bf4700 1 --2- 172.21.15.52:0/21897933 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f52ac0a3c40 0x7f52ac0a6100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f52b4005950 tx=0x7f52b40058e0 comp rx=0 tx=0).stop 2024-04-02T13:53:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.827+0000 7f52c9bf4700 1 -- 172.21.15.52:0/21897933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52c41041e0 msgr2=0x7f52c4074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.827+0000 7f52c9bf4700 1 --2- 172.21.15.52:0/21897933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52c41041e0 0x7f52c4074f20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f52bc007ee0 tx=0x7f52bc00d3b0 comp rx=0 tx=0).stop 2024-04-02T13:53:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.827+0000 7f52c9bf4700 1 -- 172.21.15.52:0/21897933 shutdown_connections 2024-04-02T13:53:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.827+0000 7f52c9bf4700 1 --2- 172.21.15.52:0/21897933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52c41041e0 0x7f52c4074f20 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.827+0000 7f52c9bf4700 1 --2- 172.21.15.52:0/21897933 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f52ac0a3c40 0x7f52ac0a6100 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:09.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.827+0000 7f52c9bf4700 1 --2- 172.21.15.52:0/21897933 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52c4102640 0x7f52c4076390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:09.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.827+0000 7f52c9bf4700 1 --2- 172.21.15.52:0/21897933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52c4103840 0x7f52c40749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:09.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.827+0000 7f52c9bf4700 1 -- 172.21.15.52:0/21897933 >> 172.21.15.52:0/21897933 conn(0x7f52c40fdc00 msgr2=0x7f52c4106ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:09.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.827+0000 7f52c9bf4700 1 -- 172.21.15.52:0/21897933 shutdown_connections 2024-04-02T13:53:09.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:09.827+0000 7f52c9bf4700 1 -- 172.21.15.52:0/21897933 wait complete. 2024-04-02T13:53:10.501 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T13:53:10.509 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi052.front.sepia.ceph.com 2024-04-02T13:53:10.510 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs dump' 2024-04-02T13:53:10.762 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:53:10.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:10 smithi189 ceph-mon[36683]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-02T13:53:10.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:10 smithi052 ceph-mon[32104]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-02T13:53:10.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:10 smithi088 ceph-mon[36528]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-02T13:53:11.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:11 smithi189 ceph-mon[36683]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:11.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:11 smithi052 ceph-mon[32104]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:11.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:11 smithi088 ceph-mon[36528]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:12.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.466+0000 7f3f00d07700 1 -- 172.21.15.52:0/2601423109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3efc101520 msgr2=0x7f3efc101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:12.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.466+0000 7f3f00d07700 1 --2- 172.21.15.52:0/2601423109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3efc101520 0x7f3efc101940 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0099c0 tx=0x7f3eec009cd0 comp rx=0 tx=0).stop 2024-04-02T13:53:12.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.467+0000 7f3f00d07700 1 -- 172.21.15.52:0/2601423109 shutdown_connections 2024-04-02T13:53:12.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.467+0000 7f3f00d07700 1 --2- 172.21.15.52:0/2601423109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3efc1030c0 0x7f3efc109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:12.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.467+0000 7f3f00d07700 1 --2- 172.21.15.52:0/2601423109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3efc101520 0x7f3efc101940 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:12.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.467+0000 7f3f00d07700 1 --2- 172.21.15.52:0/2601423109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3efc102720 0x7f3efc102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:12.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.467+0000 7f3f00d07700 1 -- 172.21.15.52:0/2601423109 >> 172.21.15.52:0/2601423109 conn(0x7f3efc0fcac0 msgr2=0x7f3efc0fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:12.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.467+0000 7f3f00d07700 1 -- 172.21.15.52:0/2601423109 shutdown_connections 2024-04-02T13:53:12.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.467+0000 7f3f00d07700 1 -- 172.21.15.52:0/2601423109 wait complete. 2024-04-02T13:53:12.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.467+0000 7f3f00d07700 1 Processor -- start 2024-04-02T13:53:12.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.467+0000 7f3f00d07700 1 -- start start 2024-04-02T13:53:12.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.468+0000 7f3f00d07700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3efc101520 0x7f3efc198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:12.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.468+0000 7f3f00d07700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3efc102720 0x7f3efc198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:12.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.468+0000 7f3f00d07700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3efc1030c0 0x7f3efc19fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:12.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.468+0000 7f3f00d07700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc10d170 con 0x7f3efc1030c0 2024-04-02T13:53:12.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.468+0000 7f3f00d07700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc10cff0 con 0x7f3efc102720 2024-04-02T13:53:12.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.468+0000 7f3f00d07700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc10d2f0 con 0x7f3efc101520 2024-04-02T13:53:12.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.468+0000 7f3efad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3efc1030c0 0x7f3efc19fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:12.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.468+0000 7f3efad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3efc1030c0 0x7f3efc19fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40714/0 (socket says 172.21.15.52:40714) 2024-04-02T13:53:12.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.468+0000 7f3efad9d700 1 -- 172.21.15.52:0/230790530 learned_addr learned my addr 172.21.15.52:0/230790530 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:12.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.468+0000 7f3ef9d9b700 1 --2- 172.21.15.52:0/230790530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3efc102720 0x7f3efc198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:12.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.468+0000 7f3efa59c700 1 --2- 172.21.15.52:0/230790530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3efc101520 0x7f3efc198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:12.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.468+0000 7f3efad9d700 1 -- 172.21.15.52:0/230790530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3efc101520 msgr2=0x7f3efc198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:12.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.468+0000 7f3efad9d700 1 --2- 172.21.15.52:0/230790530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3efc101520 0x7f3efc198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:12.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.468+0000 7f3efad9d700 1 -- 172.21.15.52:0/230790530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3efc102720 msgr2=0x7f3efc198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:12.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.468+0000 7f3efad9d700 1 --2- 172.21.15.52:0/230790530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3efc102720 0x7f3efc198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:12.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.468+0000 7f3efad9d700 1 -- 172.21.15.52:0/230790530 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc1a0560 con 0x7f3efc1030c0 2024-04-02T13:53:12.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.468+0000 7f3efa59c700 1 --2- 172.21.15.52:0/230790530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3efc101520 0x7f3efc198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:53:12.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.468+0000 7f3efad9d700 1 --2- 172.21.15.52:0/230790530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3efc1030c0 0x7f3efc19fec0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400b9a0 tx=0x7f3ef400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:12.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.468+0000 7f3eeb7fe700 1 -- 172.21.15.52:0/230790530 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4019070 con 0x7f3efc1030c0 2024-04-02T13:53:12.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.468+0000 7f3eeb7fe700 1 -- 172.21.15.52:0/230790530 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ef4014440 con 0x7f3efc1030c0 2024-04-02T13:53:12.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.468+0000 7f3f00d07700 1 -- 172.21.15.52:0/230790530 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3efc1a0850 con 0x7f3efc1030c0 2024-04-02T13:53:12.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.468+0000 7f3eeb7fe700 1 -- 172.21.15.52:0/230790530 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40075d0 con 0x7f3efc1030c0 2024-04-02T13:53:12.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.468+0000 7f3f00d07700 1 -- 172.21.15.52:0/230790530 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3efc1a0e10 con 0x7f3efc1030c0 2024-04-02T13:53:12.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.469+0000 7f3f00d07700 1 -- 172.21.15.52:0/230790530 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3efc0692b0 con 0x7f3efc1030c0 2024-04-02T13:53:12.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.470+0000 7f3eeb7fe700 1 -- 172.21.15.52:0/230790530 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3ef401c070 con 0x7f3efc1030c0 2024-04-02T13:53:12.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.471+0000 7f3eeb7fe700 1 --2- 172.21.15.52:0/230790530 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ee40a3c40 0x7f3ee40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:12.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.471+0000 7f3efa59c700 1 --2- 172.21.15.52:0/230790530 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ee40a3c40 0x7f3ee40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:12.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.471+0000 7f3eeb7fe700 1 -- 172.21.15.52:0/230790530 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f3ef40cfb40 con 0x7f3efc1030c0 2024-04-02T13:53:12.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.472+0000 7f3efa59c700 1 --2- 172.21.15.52:0/230790530 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ee40a3c40 0x7f3ee40a6100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3eec006010 tx=0x7f3eec005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:12.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.476+0000 7f3eeb7fe700 1 -- 172.21.15.52:0/230790530 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3ef409c250 con 0x7f3efc1030c0 2024-04-02T13:53:12.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.730+0000 7f3f00d07700 1 -- 172.21.15.52:0/230790530 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f3efc1a17f0 con 0x7f3efc1030c0 2024-04-02T13:53:12.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.731+0000 7f3eeb7fe700 1 -- 172.21.15.52:0/230790530 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3086 (secure 0 0 0) 0x7f3ef409bb50 con 0x7f3efc1030c0 2024-04-02T13:53:12.734 INFO:teuthology.orchestra.run.smithi052.stdout:e27 2024-04-02T13:53:12.734 INFO:teuthology.orchestra.run.smithi052.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T13:53:12.735 INFO:teuthology.orchestra.run.smithi052.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T13:53:12.735 INFO:teuthology.orchestra.run.smithi052.stdout:legacy client fscid: 1 2024-04-02T13:53:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:53:12.735 INFO:teuthology.orchestra.run.smithi052.stdout:Filesystem 'cephfs' (1) 2024-04-02T13:53:12.735 INFO:teuthology.orchestra.run.smithi052.stdout:fs_name cephfs 2024-04-02T13:53:12.735 INFO:teuthology.orchestra.run.smithi052.stdout:epoch 27 2024-04-02T13:53:12.735 INFO:teuthology.orchestra.run.smithi052.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-02T13:53:12.735 INFO:teuthology.orchestra.run.smithi052.stdout:created 2024-04-02T13:51:00.603246+0000 2024-04-02T13:53:12.735 INFO:teuthology.orchestra.run.smithi052.stdout:modified 2024-04-02T13:51:29.345060+0000 2024-04-02T13:53:12.736 INFO:teuthology.orchestra.run.smithi052.stdout:tableserver 0 2024-04-02T13:53:12.736 INFO:teuthology.orchestra.run.smithi052.stdout:root 0 2024-04-02T13:53:12.736 INFO:teuthology.orchestra.run.smithi052.stdout:session_timeout 300 2024-04-02T13:53:12.736 INFO:teuthology.orchestra.run.smithi052.stdout:session_autoclose 300 2024-04-02T13:53:12.736 INFO:teuthology.orchestra.run.smithi052.stdout:max_file_size 1099511627776 2024-04-02T13:53:12.736 INFO:teuthology.orchestra.run.smithi052.stdout:max_xattr_size 65536 2024-04-02T13:53:12.736 INFO:teuthology.orchestra.run.smithi052.stdout:required_client_features {} 2024-04-02T13:53:12.736 INFO:teuthology.orchestra.run.smithi052.stdout:last_failure 0 2024-04-02T13:53:12.736 INFO:teuthology.orchestra.run.smithi052.stdout:last_failure_osd_epoch 0 2024-04-02T13:53:12.737 INFO:teuthology.orchestra.run.smithi052.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T13:53:12.737 INFO:teuthology.orchestra.run.smithi052.stdout:max_mds 5 2024-04-02T13:53:12.737 INFO:teuthology.orchestra.run.smithi052.stdout:in 0,1,2,3,4 2024-04-02T13:53:12.737 INFO:teuthology.orchestra.run.smithi052.stdout:up {0=24503,1=24490,2=24475,3=24467,4=24457} 2024-04-02T13:53:12.737 INFO:teuthology.orchestra.run.smithi052.stdout:failed 2024-04-02T13:53:12.737 INFO:teuthology.orchestra.run.smithi052.stdout:damaged 2024-04-02T13:53:12.737 INFO:teuthology.orchestra.run.smithi052.stdout:stopped 2024-04-02T13:53:12.737 INFO:teuthology.orchestra.run.smithi052.stdout:data_pools [3,4] 2024-04-02T13:53:12.737 INFO:teuthology.orchestra.run.smithi052.stdout:metadata_pool 2 2024-04-02T13:53:12.737 INFO:teuthology.orchestra.run.smithi052.stdout:inline_data disabled 2024-04-02T13:53:12.737 INFO:teuthology.orchestra.run.smithi052.stdout:balancer 2024-04-02T13:53:12.737 INFO:teuthology.orchestra.run.smithi052.stdout:bal_rank_mask -1 2024-04-02T13:53:12.738 INFO:teuthology.orchestra.run.smithi052.stdout:standby_count_wanted 0 2024-04-02T13:53:12.738 INFO:teuthology.orchestra.run.smithi052.stdout:qdb_cluster leader: 24503 members: 24503,24475,24490,24467,24457 2024-04-02T13:53:12.738 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.b{0:24503} state up:active seq 3 addr [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:53:12.738 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.e{0:24437} state up:standby-replay seq 1 addr [v2:172.21.15.88:6834/5019107,v1:172.21.15.88:6835/5019107] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:53:12.738 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.i{1:24490} state up:active seq 11 addr [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:53:12.738 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.l{1:24418} state up:standby-replay seq 1 addr [v2:172.21.15.189:6832/3157031038,v1:172.21.15.189:6833/3157031038] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:53:12.738 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.c{2:24475} state up:active seq 13 addr [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:53:12.738 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.k{2:24410} state up:standby-replay seq 1 addr [v2:172.21.15.88:6832/4248342942,v1:172.21.15.88:6833/4248342942] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:53:12.738 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.h{3:24467} state up:active seq 12 addr [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:53:12.738 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.a{3:14616} state up:standby-replay seq 1 addr [v2:172.21.15.52:6840/2689882080,v1:172.21.15.52:6841/2689882080] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:53:12.738 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.f{4:24457} state up:active seq 14 addr [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:53:12.738 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.g{4:14604} state up:standby-replay seq 1 addr [v2:172.21.15.52:6838/2213112521,v1:172.21.15.52:6839/2213112521] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:53:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:53:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:53:12.739 INFO:teuthology.orchestra.run.smithi052.stdout:Standby daemons: 2024-04-02T13:53:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:53:12.739 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.j{-1:14523} state up:standby seq 1 addr [v2:172.21.15.52:6834/1683593044,v1:172.21.15.52:6835/1683593044] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:53:12.739 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.d{-1:14535} state up:standby seq 1 addr [v2:172.21.15.52:6836/361596788,v1:172.21.15.52:6837/361596788] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:53:12.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.734+0000 7f3f00d07700 1 -- 172.21.15.52:0/230790530 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ee40a3c40 msgr2=0x7f3ee40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:12.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.734+0000 7f3f00d07700 1 --2- 172.21.15.52:0/230790530 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ee40a3c40 0x7f3ee40a6100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3eec006010 tx=0x7f3eec005c40 comp rx=0 tx=0).stop 2024-04-02T13:53:12.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.734+0000 7f3f00d07700 1 -- 172.21.15.52:0/230790530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3efc1030c0 msgr2=0x7f3efc19fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:12.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.734+0000 7f3f00d07700 1 --2- 172.21.15.52:0/230790530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3efc1030c0 0x7f3efc19fec0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400b9a0 tx=0x7f3ef400bd60 comp rx=0 tx=0).stop 2024-04-02T13:53:12.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.734+0000 7f3f00d07700 1 -- 172.21.15.52:0/230790530 shutdown_connections 2024-04-02T13:53:12.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.734+0000 7f3f00d07700 1 --2- 172.21.15.52:0/230790530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3efc1030c0 0x7f3efc19fec0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:12.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.734+0000 7f3f00d07700 1 --2- 172.21.15.52:0/230790530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3efc101520 0x7f3efc198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:12.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.734+0000 7f3f00d07700 1 --2- 172.21.15.52:0/230790530 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3ee40a3c40 0x7f3ee40a6100 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:12.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.734+0000 7f3f00d07700 1 --2- 172.21.15.52:0/230790530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3efc102720 0x7f3efc198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:12.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.734+0000 7f3f00d07700 1 -- 172.21.15.52:0/230790530 >> 172.21.15.52:0/230790530 conn(0x7f3efc0fcac0 msgr2=0x7f3efc06a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:12.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.734+0000 7f3f00d07700 1 -- 172.21.15.52:0/230790530 shutdown_connections 2024-04-02T13:53:12.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:12.734+0000 7f3f00d07700 1 -- 172.21.15.52:0/230790530 wait complete. 2024-04-02T13:53:12.740 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 27 2024-04-02T13:53:12.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:12 smithi189 ceph-mon[36683]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-02T13:53:12.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:12 smithi052 ceph-mon[32104]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-02T13:53:12.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:12 smithi088 ceph-mon[36528]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-02T13:53:13.424 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- bash -c 'ceph osd dump' 2024-04-02T13:53:13.671 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:53:13.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:13 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/230790530' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T13:53:13.792 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:13 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/230790530' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T13:53:13.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:13 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/230790530' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T13:53:14.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:14 smithi189 ceph-mon[36683]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T13:53:14.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:14 smithi052 ceph-mon[32104]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T13:53:14.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:14 smithi088 ceph-mon[36528]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T13:53:15.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.415+0000 7fb9c9d30700 1 -- 172.21.15.52:0/3839390838 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9c4102620 msgr2=0x7fb9c4102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:15.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.415+0000 7fb9c9d30700 1 --2- 172.21.15.52:0/3839390838 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9c4102620 0x7fb9c4102a40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40099c0 tx=0x7fb9b4009cd0 comp rx=0 tx=0).stop 2024-04-02T13:53:15.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.415+0000 7fb9c9d30700 1 -- 172.21.15.52:0/3839390838 shutdown_connections 2024-04-02T13:53:15.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.415+0000 7fb9c9d30700 1 --2- 172.21.15.52:0/3839390838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9c41041c0 0x7fb9c410aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:15.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.415+0000 7fb9c9d30700 1 --2- 172.21.15.52:0/3839390838 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9c4102620 0x7fb9c4102a40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:15.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.415+0000 7fb9c9d30700 1 --2- 172.21.15.52:0/3839390838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9c4103820 0x7fb9c4103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:15.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.415+0000 7fb9c9d30700 1 -- 172.21.15.52:0/3839390838 >> 172.21.15.52:0/3839390838 conn(0x7fb9c40fdc00 msgr2=0x7fb9c4100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:15.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.415+0000 7fb9c9d30700 1 -- 172.21.15.52:0/3839390838 shutdown_connections 2024-04-02T13:53:15.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.415+0000 7fb9c9d30700 1 -- 172.21.15.52:0/3839390838 wait complete. 2024-04-02T13:53:15.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.416+0000 7fb9c9d30700 1 Processor -- start 2024-04-02T13:53:15.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.416+0000 7fb9c9d30700 1 -- start start 2024-04-02T13:53:15.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.416+0000 7fb9c9d30700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9c4102620 0x7fb9c41988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:15.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.416+0000 7fb9c9d30700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9c4103820 0x7fb9c4198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:15.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.416+0000 7fb9c9d30700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9c41041c0 0x7fb9c419fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:15.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.416+0000 7fb9c9d30700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c410e260 con 0x7fb9c4103820 2024-04-02T13:53:15.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.416+0000 7fb9c9d30700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c410e0e0 con 0x7fb9c41041c0 2024-04-02T13:53:15.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.416+0000 7fb9c9d30700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c410e3e0 con 0x7fb9c4102620 2024-04-02T13:53:15.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.417+0000 7fb9c2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9c4103820 0x7fb9c4198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:15.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.417+0000 7fb9c2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9c4103820 0x7fb9c4198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40732/0 (socket says 172.21.15.52:40732) 2024-04-02T13:53:15.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.417+0000 7fb9c2ffd700 1 -- 172.21.15.52:0/4011478152 learned_addr learned my addr 172.21.15.52:0/4011478152 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:15.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.417+0000 7fb9c37fe700 1 --2- 172.21.15.52:0/4011478152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9c4102620 0x7fb9c41988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:15.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.417+0000 7fb9c3fff700 1 --2- 172.21.15.52:0/4011478152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9c41041c0 0x7fb9c419fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:15.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.417+0000 7fb9c2ffd700 1 -- 172.21.15.52:0/4011478152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9c4102620 msgr2=0x7fb9c41988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:15.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.417+0000 7fb9c2ffd700 1 --2- 172.21.15.52:0/4011478152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9c4102620 0x7fb9c41988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:15.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.417+0000 7fb9c2ffd700 1 -- 172.21.15.52:0/4011478152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9c41041c0 msgr2=0x7fb9c419fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:15.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.417+0000 7fb9c2ffd700 1 --2- 172.21.15.52:0/4011478152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9c41041c0 0x7fb9c419fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:15.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.417+0000 7fb9c2ffd700 1 -- 172.21.15.52:0/4011478152 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c41a0530 con 0x7fb9c4103820 2024-04-02T13:53:15.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.417+0000 7fb9c3fff700 1 --2- 172.21.15.52:0/4011478152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9c41041c0 0x7fb9c419fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:53:15.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.417+0000 7fb9c2ffd700 1 --2- 172.21.15.52:0/4011478152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9c4103820 0x7fb9c4198e10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800b560 tx=0x7fb9b800b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:15.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.417+0000 7fb9c0ff9700 1 -- 172.21.15.52:0/4011478152 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8015020 con 0x7fb9c4103820 2024-04-02T13:53:15.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.417+0000 7fb9c0ff9700 1 -- 172.21.15.52:0/4011478152 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9b8011d10 con 0x7fb9c4103820 2024-04-02T13:53:15.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.417+0000 7fb9c9d30700 1 -- 172.21.15.52:0/4011478152 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c41a0820 con 0x7fb9c4103820 2024-04-02T13:53:15.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.417+0000 7fb9c0ff9700 1 -- 172.21.15.52:0/4011478152 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b800f2e0 con 0x7fb9c4103820 2024-04-02T13:53:15.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.417+0000 7fb9c9d30700 1 -- 172.21.15.52:0/4011478152 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c41a0d90 con 0x7fb9c4103820 2024-04-02T13:53:15.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.418+0000 7fb9c9d30700 1 -- 172.21.15.52:0/4011478152 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9c40692b0 con 0x7fb9c4103820 2024-04-02T13:53:15.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.420+0000 7fb9c0ff9700 1 -- 172.21.15.52:0/4011478152 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb9b8018020 con 0x7fb9c4103820 2024-04-02T13:53:15.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.421+0000 7fb9c0ff9700 1 --2- 172.21.15.52:0/4011478152 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb9ac0a3c40 0x7fb9ac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:15.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.421+0000 7fb9c37fe700 1 --2- 172.21.15.52:0/4011478152 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb9ac0a3c40 0x7fb9ac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:15.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.421+0000 7fb9c0ff9700 1 -- 172.21.15.52:0/4011478152 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb9b80d0430 con 0x7fb9c4103820 2024-04-02T13:53:15.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.421+0000 7fb9c37fe700 1 --2- 172.21.15.52:0/4011478152 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb9ac0a3c40 0x7fb9ac0a6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400b580 tx=0x7fb9b4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:15.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.424+0000 7fb9c0ff9700 1 -- 172.21.15.52:0/4011478152 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb9b801f020 con 0x7fb9c4103820 2024-04-02T13:53:15.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.636+0000 7fb9c9d30700 1 -- 172.21.15.52:0/4011478152 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fb9c402dd60 con 0x7fb9c4103820 2024-04-02T13:53:15.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.637+0000 7fb9c0ff9700 1 -- 172.21.15.52:0/4011478152 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v92) ==== 56+0+5584 (secure 0 0 0) 0x7fb9b809ca50 con 0x7fb9c4103820 2024-04-02T13:53:15.640 INFO:teuthology.orchestra.run.smithi052.stdout:epoch 92 2024-04-02T13:53:15.640 INFO:teuthology.orchestra.run.smithi052.stdout:fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T13:53:15.641 INFO:teuthology.orchestra.run.smithi052.stdout:created 2024-04-02T13:42:59.639077+0000 2024-04-02T13:53:15.641 INFO:teuthology.orchestra.run.smithi052.stdout:modified 2024-04-02T13:51:56.472064+0000 2024-04-02T13:53:15.641 INFO:teuthology.orchestra.run.smithi052.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-02T13:53:15.641 INFO:teuthology.orchestra.run.smithi052.stdout:crush_version 27 2024-04-02T13:53:15.641 INFO:teuthology.orchestra.run.smithi052.stdout:full_ratio 0.95 2024-04-02T13:53:15.641 INFO:teuthology.orchestra.run.smithi052.stdout:backfillfull_ratio 0.9 2024-04-02T13:53:15.641 INFO:teuthology.orchestra.run.smithi052.stdout:nearfull_ratio 0.85 2024-04-02T13:53:15.641 INFO:teuthology.orchestra.run.smithi052.stdout:require_min_compat_client luminous 2024-04-02T13:53:15.641 INFO:teuthology.orchestra.run.smithi052.stdout:min_compat_client luminous 2024-04-02T13:53:15.641 INFO:teuthology.orchestra.run.smithi052.stdout:require_osd_release squid 2024-04-02T13:53:15.641 INFO:teuthology.orchestra.run.smithi052.stdout:stretch_mode_enabled false 2024-04-02T13:53:15.641 INFO:teuthology.orchestra.run.smithi052.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-02T13:53:15.641 INFO:teuthology.orchestra.run.smithi052.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-04-02T13:53:15.641 INFO:teuthology.orchestra.run.smithi052.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-02T13:53:15.642 INFO:teuthology.orchestra.run.smithi052.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-02T13:53:15.642 INFO:teuthology.orchestra.run.smithi052.stdout:max_osd 12 2024-04-02T13:53:15.642 INFO:teuthology.orchestra.run.smithi052.stdout:osd.0 up in weight 1 up_from 8 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6802/1314806472,v1:172.21.15.52:6803/1314806472] [v2:172.21.15.52:6804/1314806472,v1:172.21.15.52:6805/1314806472] exists,up 4fe9c5ef-4edb-4b75-8e4f-5615bae725c3 2024-04-02T13:53:15.642 INFO:teuthology.orchestra.run.smithi052.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.88:6800/288986438,v1:172.21.15.88:6801/288986438] [v2:172.21.15.88:6802/288986438,v1:172.21.15.88:6803/288986438] exists,up 2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6 2024-04-02T13:53:15.642 INFO:teuthology.orchestra.run.smithi052.stdout:osd.2 up in weight 1 up_from 18 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6800/2416423625,v1:172.21.15.189:6801/2416423625] [v2:172.21.15.189:6802/2416423625,v1:172.21.15.189:6803/2416423625] exists,up 142d5d34-737f-44bf-a17c-c2ad6d0be723 2024-04-02T13:53:15.642 INFO:teuthology.orchestra.run.smithi052.stdout:osd.3 up in weight 1 up_from 25 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6810/1799783980,v1:172.21.15.52:6811/1799783980] [v2:172.21.15.52:6812/1799783980,v1:172.21.15.52:6813/1799783980] exists,up 0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4 2024-04-02T13:53:15.642 INFO:teuthology.orchestra.run.smithi052.stdout:osd.4 up in weight 1 up_from 33 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.88:6808/2077759625,v1:172.21.15.88:6809/2077759625] [v2:172.21.15.88:6810/2077759625,v1:172.21.15.88:6811/2077759625] exists,up 1fbfe997-a0e7-4b89-8e6b-754a9bee10a1 2024-04-02T13:53:15.642 INFO:teuthology.orchestra.run.smithi052.stdout:osd.5 up in weight 1 up_from 38 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6808/3106153812,v1:172.21.15.189:6809/3106153812] [v2:172.21.15.189:6810/3106153812,v1:172.21.15.189:6811/3106153812] exists,up 77eeb2e8-2420-4dcc-8774-beed04c6d827 2024-04-02T13:53:15.642 INFO:teuthology.orchestra.run.smithi052.stdout:osd.6 up in weight 1 up_from 46 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6818/2542655509,v1:172.21.15.52:6819/2542655509] [v2:172.21.15.52:6820/2542655509,v1:172.21.15.52:6821/2542655509] exists,up 17099b55-abab-4f7b-8cab-c9573cd9b2f4 2024-04-02T13:53:15.642 INFO:teuthology.orchestra.run.smithi052.stdout:osd.7 up in weight 1 up_from 51 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.88:6816/714628313,v1:172.21.15.88:6817/714628313] [v2:172.21.15.88:6818/714628313,v1:172.21.15.88:6819/714628313] exists,up 0fe3487a-3c01-483b-95ca-71ce42e74bcc 2024-04-02T13:53:15.642 INFO:teuthology.orchestra.run.smithi052.stdout:osd.8 up in weight 1 up_from 60 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6816/1602944902,v1:172.21.15.189:6817/1602944902] [v2:172.21.15.189:6818/1602944902,v1:172.21.15.189:6819/1602944902] exists,up f93ee6a0-9a7b-4090-9035-a169686c4b9b 2024-04-02T13:53:15.642 INFO:teuthology.orchestra.run.smithi052.stdout:osd.9 up in weight 1 up_from 68 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6826/2350236811,v1:172.21.15.52:6827/2350236811] [v2:172.21.15.52:6828/2350236811,v1:172.21.15.52:6829/2350236811] exists,up 8aaad86a-c313-467b-8998-13f154bda92a 2024-04-02T13:53:15.642 INFO:teuthology.orchestra.run.smithi052.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.88:6824/3425831447,v1:172.21.15.88:6825/3425831447] [v2:172.21.15.88:6826/3425831447,v1:172.21.15.88:6827/3425831447] exists,up 0d33f6f2-3a01-4753-b2ef-b8c42454198e 2024-04-02T13:53:15.642 INFO:teuthology.orchestra.run.smithi052.stdout:osd.11 up in weight 1 up_from 81 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6824/383422941,v1:172.21.15.189:6825/383422941] [v2:172.21.15.189:6826/383422941,v1:172.21.15.189:6827/383422941] exists,up 4369f21d-d344-4e2f-b556-010faefd182c 2024-04-02T13:53:15.643 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 2.a [2,11] 2024-04-02T13:53:15.643 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 2.13 [2,11] 2024-04-02T13:53:15.643 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 3.2 [3,9] 2024-04-02T13:53:15.643 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 3.20 [5,9] 2024-04-02T13:53:15.643 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 3.21 [4,8] 2024-04-02T13:53:15.643 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 3.3c [4,9] 2024-04-02T13:53:15.643 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 4.3 [5,8] 2024-04-02T13:53:15.643 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 4.13 [9,10] 2024-04-02T13:53:15.643 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 4.19 [9,8] 2024-04-02T13:53:15.643 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 4.1f [9,8] 2024-04-02T13:53:15.643 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/2475099387 expires 2024-04-03T13:43:54.653361+0000 2024-04-02T13:53:15.643 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:6800/2368921152 expires 2024-04-03T13:43:27.309404+0000 2024-04-02T13:53:15.643 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:6801/673843171 expires 2024-04-03T13:43:54.653361+0000 2024-04-02T13:53:15.643 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:6801/2368921152 expires 2024-04-03T13:43:27.309404+0000 2024-04-02T13:53:15.643 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/2609300748 expires 2024-04-03T13:43:27.309404+0000 2024-04-02T13:53:15.644 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/85571395 expires 2024-04-03T13:43:27.309404+0000 2024-04-02T13:53:15.644 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/293852652 expires 2024-04-03T13:43:54.653361+0000 2024-04-02T13:53:15.644 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/1829428070 expires 2024-04-03T13:43:27.309404+0000 2024-04-02T13:53:15.644 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:6800/673843171 expires 2024-04-03T13:43:54.653361+0000 2024-04-02T13:53:15.644 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/3257395293 expires 2024-04-03T13:43:54.653361+0000 2024-04-02T13:53:15.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.639+0000 7fb9c9d30700 1 -- 172.21.15.52:0/4011478152 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb9ac0a3c40 msgr2=0x7fb9ac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:15.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.639+0000 7fb9c9d30700 1 --2- 172.21.15.52:0/4011478152 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb9ac0a3c40 0x7fb9ac0a6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400b580 tx=0x7fb9b4005fb0 comp rx=0 tx=0).stop 2024-04-02T13:53:15.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.639+0000 7fb9c9d30700 1 -- 172.21.15.52:0/4011478152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9c4103820 msgr2=0x7fb9c4198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:15.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.639+0000 7fb9c9d30700 1 --2- 172.21.15.52:0/4011478152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9c4103820 0x7fb9c4198e10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800b560 tx=0x7fb9b800b920 comp rx=0 tx=0).stop 2024-04-02T13:53:15.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.639+0000 7fb9c9d30700 1 -- 172.21.15.52:0/4011478152 shutdown_connections 2024-04-02T13:53:15.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.639+0000 7fb9c9d30700 1 --2- 172.21.15.52:0/4011478152 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb9ac0a3c40 0x7fb9ac0a6100 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:15.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.639+0000 7fb9c9d30700 1 --2- 172.21.15.52:0/4011478152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9c41041c0 0x7fb9c419fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:15.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.639+0000 7fb9c9d30700 1 --2- 172.21.15.52:0/4011478152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9c4102620 0x7fb9c41988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:15.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.639+0000 7fb9c9d30700 1 --2- 172.21.15.52:0/4011478152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9c4103820 0x7fb9c4198e10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:15.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.639+0000 7fb9c9d30700 1 -- 172.21.15.52:0/4011478152 >> 172.21.15.52:0/4011478152 conn(0x7fb9c40fdc00 msgr2=0x7fb9c4106ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:15.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.639+0000 7fb9c9d30700 1 -- 172.21.15.52:0/4011478152 shutdown_connections 2024-04-02T13:53:15.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:15.639+0000 7fb9c9d30700 1 -- 172.21.15.52:0/4011478152 wait complete. 2024-04-02T13:53:16.316 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-02T13:53:16.334 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:53:16.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.437+0000 7fb8bc63a640 1 -- 172.21.15.52:0/3146916784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8b4100a40 msgr2=0x7fb8b4100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:16.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.437+0000 7fb8bc63a640 1 --2- 172.21.15.52:0/3146916784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8b4100a40 0x7fb8b4100e60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009900 tx=0x7fb8ac02bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.437+0000 7fb8bc63a640 1 -- 172.21.15.52:0/3146916784 shutdown_connections 2024-04-02T13:53:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.437+0000 7fb8bc63a640 1 --2- 172.21.15.52:0/3146916784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb8b41025f0 0x7fb8b4108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.437+0000 7fb8bc63a640 1 --2- 172.21.15.52:0/3146916784 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8b4101c30 0x7fb8b41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.437+0000 7fb8bc63a640 1 --2- 172.21.15.52:0/3146916784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8b4100a40 0x7fb8b4100e60 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.437+0000 7fb8bc63a640 1 -- 172.21.15.52:0/3146916784 >> 172.21.15.52:0/3146916784 conn(0x7fb8b40fbfb0 msgr2=0x7fb8b40fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.437+0000 7fb8bc63a640 1 -- 172.21.15.52:0/3146916784 shutdown_connections 2024-04-02T13:53:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.437+0000 7fb8bc63a640 1 -- 172.21.15.52:0/3146916784 wait complete. 2024-04-02T13:53:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.438+0000 7fb8bc63a640 1 Processor -- start 2024-04-02T13:53:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.438+0000 7fb8bc63a640 1 -- start start 2024-04-02T13:53:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.438+0000 7fb8bc63a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8b4100a40 0x7fb8b4104980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.438+0000 7fb8bc63a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb8b4101c30 0x7fb8b4103010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.438+0000 7fb8bc63a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8b41025f0 0x7fb8b4103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.438+0000 7fb8bc63a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b410c710 con 0x7fb8b41025f0 2024-04-02T13:53:16.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.438+0000 7fb8bc63a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b410c590 con 0x7fb8b4100a40 2024-04-02T13:53:16.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.438+0000 7fb8bc63a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b410c890 con 0x7fb8b4101c30 2024-04-02T13:53:16.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.438+0000 7fb8babd7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8b41025f0 0x7fb8b4103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:16.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.438+0000 7fb8babd7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8b41025f0 0x7fb8b4103550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40762/0 (socket says 172.21.15.52:40762) 2024-04-02T13:53:16.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.438+0000 7fb8babd7640 1 -- 172.21.15.52:0/1015451592 learned_addr learned my addr 172.21.15.52:0/1015451592 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:16.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.438+0000 7fb8ba3d6640 1 --2- 172.21.15.52:0/1015451592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8b4100a40 0x7fb8b4104980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:16.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.438+0000 7fb8b9bd5640 1 --2- 172.21.15.52:0/1015451592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb8b4101c30 0x7fb8b4103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:16.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.438+0000 7fb8babd7640 1 -- 172.21.15.52:0/1015451592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb8b4101c30 msgr2=0x7fb8b4103010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:16.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.438+0000 7fb8babd7640 1 --2- 172.21.15.52:0/1015451592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb8b4101c30 0x7fb8b4103010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:16.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.438+0000 7fb8babd7640 1 -- 172.21.15.52:0/1015451592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8b4100a40 msgr2=0x7fb8b4104980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:16.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.438+0000 7fb8babd7640 1 --2- 172.21.15.52:0/1015451592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8b4100a40 0x7fb8b4104980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:16.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.438+0000 7fb8babd7640 1 -- 172.21.15.52:0/1015451592 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b4103e10 con 0x7fb8b41025f0 2024-04-02T13:53:16.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.439+0000 7fb8babd7640 1 --2- 172.21.15.52:0/1015451592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8b41025f0 0x7fb8b4103550 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb8a400b970 tx=0x7fb8a400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:16.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.439+0000 7fb8b37fe640 1 -- 172.21.15.52:0/1015451592 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4019070 con 0x7fb8b41025f0 2024-04-02T13:53:16.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.439+0000 7fb8b37fe640 1 -- 172.21.15.52:0/1015451592 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8a400cb40 con 0x7fb8b41025f0 2024-04-02T13:53:16.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.439+0000 7fb8bc63a640 1 -- 172.21.15.52:0/1015451592 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b41a1ff0 con 0x7fb8b41025f0 2024-04-02T13:53:16.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.439+0000 7fb8b37fe640 1 -- 172.21.15.52:0/1015451592 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4007610 con 0x7fb8b41025f0 2024-04-02T13:53:16.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.439+0000 7fb8bc63a640 1 -- 172.21.15.52:0/1015451592 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b41a2510 con 0x7fb8b41025f0 2024-04-02T13:53:16.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.440+0000 7fb8bc63a640 1 -- 172.21.15.52:0/1015451592 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8b41a27a0 con 0x7fb8b41025f0 2024-04-02T13:53:16.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.441+0000 7fb8b37fe640 1 -- 172.21.15.52:0/1015451592 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb8a401c070 con 0x7fb8b41025f0 2024-04-02T13:53:16.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.441+0000 7fb8b37fe640 1 --2- 172.21.15.52:0/1015451592 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb89c0a3bb0 0x7fb89c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:16.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.441+0000 7fb8ba3d6640 1 --2- 172.21.15.52:0/1015451592 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb89c0a3bb0 0x7fb89c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:16.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.441+0000 7fb8b37fe640 1 -- 172.21.15.52:0/1015451592 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb8a40d01b0 con 0x7fb8b41025f0 2024-04-02T13:53:16.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.442+0000 7fb8ba3d6640 1 --2- 172.21.15.52:0/1015451592 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb89c0a3bb0 0x7fb89c0a6070 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac000c00 tx=0x7fb8ac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:16.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.445+0000 7fb8b37fe640 1 -- 172.21.15.52:0/1015451592 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb8a409c180 con 0x7fb8b41025f0 2024-04-02T13:53:16.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:16 smithi052 ceph-mon[32104]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:53:16.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:16 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/4011478152' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-02T13:53:16.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.635+0000 7fb8bc63a640 1 -- 172.21.15.52:0/1015451592 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8b41a2b30 con 0x7fb8b41025f0 2024-04-02T13:53:16.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.637+0000 7fb8b37fe640 1 -- 172.21.15.52:0/1015451592 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24549 (secure 0 0 0) 0x7fb8a40a0480 con 0x7fb8b41025f0 2024-04-02T13:53:16.641 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:53:16.641 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":92,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T13:51:56.472064+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:50:58.778690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:50:59.561527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:51:03.343487+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:53:16.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.639+0000 7fb8bc63a640 1 -- 172.21.15.52:0/1015451592 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb89c0a3bb0 msgr2=0x7fb89c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:16.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.639+0000 7fb8bc63a640 1 --2- 172.21.15.52:0/1015451592 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb89c0a3bb0 0x7fb89c0a6070 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac000c00 tx=0x7fb8ac005fd0 comp rx=0 tx=0).stop 2024-04-02T13:53:16.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.639+0000 7fb8bc63a640 1 -- 172.21.15.52:0/1015451592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8b41025f0 msgr2=0x7fb8b4103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:16.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.639+0000 7fb8bc63a640 1 --2- 172.21.15.52:0/1015451592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8b41025f0 0x7fb8b4103550 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb8a400b970 tx=0x7fb8a400be30 comp rx=0 tx=0).stop 2024-04-02T13:53:16.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.639+0000 7fb8bc63a640 1 -- 172.21.15.52:0/1015451592 shutdown_connections 2024-04-02T13:53:16.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.639+0000 7fb8bc63a640 1 --2- 172.21.15.52:0/1015451592 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb89c0a3bb0 0x7fb89c0a6070 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:16.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.639+0000 7fb8bc63a640 1 --2- 172.21.15.52:0/1015451592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8b41025f0 0x7fb8b4103550 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:16.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.639+0000 7fb8bc63a640 1 --2- 172.21.15.52:0/1015451592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb8b4101c30 0x7fb8b4103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:16.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.639+0000 7fb8bc63a640 1 --2- 172.21.15.52:0/1015451592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8b4100a40 0x7fb8b4104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:16.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.639+0000 7fb8bc63a640 1 -- 172.21.15.52:0/1015451592 >> 172.21.15.52:0/1015451592 conn(0x7fb8b40fbfb0 msgr2=0x7fb8b40fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:16.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.639+0000 7fb8bc63a640 1 -- 172.21.15.52:0/1015451592 shutdown_connections 2024-04-02T13:53:16.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.639+0000 7fb8bc63a640 1 -- 172.21.15.52:0/1015451592 wait complete. 2024-04-02T13:53:16.654 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:46:20.746053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:50:58.778690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:50:59.561527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:51:03.343487+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:53:16.654 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:53:16.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.757+0000 7f7394e1a640 1 -- 172.21.15.52:0/4130658870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73901077e0 msgr2=0x7f7390109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:16.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.757+0000 7f7394e1a640 1 --2- 172.21.15.52:0/4130658870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73901077e0 0x7f7390109c30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7388009ca0 tx=0x7f738802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:53:16.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.757+0000 7f7394e1a640 1 -- 172.21.15.52:0/4130658870 shutdown_connections 2024-04-02T13:53:16.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.757+0000 7f7394e1a640 1 --2- 172.21.15.52:0/4130658870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73901077e0 0x7f7390109c30 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:16.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.757+0000 7f7394e1a640 1 --2- 172.21.15.52:0/4130658870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7390104eb0 0x7f73901072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:16.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.757+0000 7f7394e1a640 1 --2- 172.21.15.52:0/4130658870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7390102580 0x7f7390104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:16.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.757+0000 7f7394e1a640 1 -- 172.21.15.52:0/4130658870 >> 172.21.15.52:0/4130658870 conn(0x7f73900fbf90 msgr2=0x7f73900fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:16.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.757+0000 7f7394e1a640 1 -- 172.21.15.52:0/4130658870 shutdown_connections 2024-04-02T13:53:16.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.757+0000 7f7394e1a640 1 -- 172.21.15.52:0/4130658870 wait complete. 2024-04-02T13:53:16.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.758+0000 7f7394e1a640 1 Processor -- start 2024-04-02T13:53:16.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.758+0000 7f7394e1a640 1 -- start start 2024-04-02T13:53:16.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.758+0000 7f7394e1a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7390102580 0x7f7390101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:16.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.758+0000 7f7394e1a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7390104eb0 0x7f7390100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:16.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.758+0000 7f7394e1a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73901077e0 0x7f7390100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:16.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.758+0000 7f7394e1a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f739010d780 con 0x7f73901077e0 2024-04-02T13:53:16.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.758+0000 7f7394e1a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f739010d600 con 0x7f7390104eb0 2024-04-02T13:53:16.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.758+0000 7f7394e1a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f739010d900 con 0x7f7390102580 2024-04-02T13:53:16.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.758+0000 7f738ed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73901077e0 0x7f7390100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:16.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.758+0000 7f738ed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73901077e0 0x7f7390100a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40782/0 (socket says 172.21.15.52:40782) 2024-04-02T13:53:16.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.758+0000 7f738ed9d640 1 -- 172.21.15.52:0/594548048 learned_addr learned my addr 172.21.15.52:0/594548048 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:16.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.758+0000 7f7385d9b640 1 --2- 172.21.15.52:0/594548048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7390104eb0 0x7f7390100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:16.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.758+0000 7f738e59c640 1 --2- 172.21.15.52:0/594548048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7390102580 0x7f7390101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:16.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.758+0000 7f738ed9d640 1 -- 172.21.15.52:0/594548048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7390102580 msgr2=0x7f7390101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:16.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.758+0000 7f738ed9d640 1 --2- 172.21.15.52:0/594548048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7390102580 0x7f7390101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:16.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.758+0000 7f738ed9d640 1 -- 172.21.15.52:0/594548048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7390104eb0 msgr2=0x7f7390100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:16.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.758+0000 7f738ed9d640 1 --2- 172.21.15.52:0/594548048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7390104eb0 0x7f7390100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:16.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.758+0000 7f738ed9d640 1 -- 172.21.15.52:0/594548048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7390100fb0 con 0x7f73901077e0 2024-04-02T13:53:16.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.758+0000 7f738e59c640 1 --2- 172.21.15.52:0/594548048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7390102580 0x7f7390101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:53:16.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.758+0000 7f7385d9b640 1 --2- 172.21.15.52:0/594548048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7390104eb0 0x7f7390100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:16.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.759+0000 7f738ed9d640 1 --2- 172.21.15.52:0/594548048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73901077e0 0x7f7390100a70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f738802c700 tx=0x7f7388032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:16.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.759+0000 7f7387fff640 1 -- 172.21.15.52:0/594548048 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7388039040 con 0x7f73901077e0 2024-04-02T13:53:16.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.759+0000 7f7394e1a640 1 -- 172.21.15.52:0/594548048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7390101240 con 0x7f73901077e0 2024-04-02T13:53:16.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.759+0000 7f7387fff640 1 -- 172.21.15.52:0/594548048 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7388044070 con 0x7f73901077e0 2024-04-02T13:53:16.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.759+0000 7f7387fff640 1 -- 172.21.15.52:0/594548048 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73880042a0 con 0x7f73901077e0 2024-04-02T13:53:16.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.759+0000 7f7394e1a640 1 -- 172.21.15.52:0/594548048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73901a6820 con 0x7f73901077e0 2024-04-02T13:53:16.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.760+0000 7f7394e1a640 1 -- 172.21.15.52:0/594548048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73900fda80 con 0x7f73901077e0 2024-04-02T13:53:16.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.761+0000 7f7387fff640 1 -- 172.21.15.52:0/594548048 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7388003970 con 0x7f73901077e0 2024-04-02T13:53:16.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.761+0000 7f7387fff640 1 --2- 172.21.15.52:0/594548048 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f737c0a3bb0 0x7f737c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:16.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.761+0000 7f738e59c640 1 --2- 172.21.15.52:0/594548048 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f737c0a3bb0 0x7f737c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:16.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.761+0000 7f7387fff640 1 -- 172.21.15.52:0/594548048 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f73880f2cf0 con 0x7f73901077e0 2024-04-02T13:53:16.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.762+0000 7f738e59c640 1 --2- 172.21.15.52:0/594548048 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f737c0a3bb0 0x7f737c0a6070 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f73800058c0 tx=0x7f7380005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:16.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.766+0000 7f7387fff640 1 -- 172.21.15.52:0/594548048 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f73880f7070 con 0x7f73901077e0 2024-04-02T13:53:16.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:16 smithi189 ceph-mon[36683]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:53:16.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:16 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/4011478152' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-02T13:53:16.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:16 smithi088 ceph-mon[36528]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:53:16.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:16 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/4011478152' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-02T13:53:16.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.949+0000 7f7394e1a640 1 -- 172.21.15.52:0/594548048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f739002d370 con 0x7f73901077e0 2024-04-02T13:53:16.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.949+0000 7f7387fff640 1 -- 172.21.15.52:0/594548048 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f73880333a0 con 0x7f73901077e0 2024-04-02T13:53:16.953 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-04-02T13:53:16.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.951+0000 7f7394e1a640 1 -- 172.21.15.52:0/594548048 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f737c0a3bb0 msgr2=0x7f737c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:16.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.951+0000 7f7394e1a640 1 --2- 172.21.15.52:0/594548048 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f737c0a3bb0 0x7f737c0a6070 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f73800058c0 tx=0x7f7380005d30 comp rx=0 tx=0).stop 2024-04-02T13:53:16.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.951+0000 7f7394e1a640 1 -- 172.21.15.52:0/594548048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73901077e0 msgr2=0x7f7390100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:16.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.951+0000 7f7394e1a640 1 --2- 172.21.15.52:0/594548048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73901077e0 0x7f7390100a70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f738802c700 tx=0x7f7388032f20 comp rx=0 tx=0).stop 2024-04-02T13:53:16.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.951+0000 7f7394e1a640 1 -- 172.21.15.52:0/594548048 shutdown_connections 2024-04-02T13:53:16.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.951+0000 7f7394e1a640 1 --2- 172.21.15.52:0/594548048 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f737c0a3bb0 0x7f737c0a6070 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:16.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.951+0000 7f7394e1a640 1 --2- 172.21.15.52:0/594548048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73901077e0 0x7f7390100a70 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:16.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.951+0000 7f7394e1a640 1 --2- 172.21.15.52:0/594548048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7390104eb0 0x7f7390100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:16.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.951+0000 7f7394e1a640 1 --2- 172.21.15.52:0/594548048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7390102580 0x7f7390101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:16.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.951+0000 7f7394e1a640 1 -- 172.21.15.52:0/594548048 >> 172.21.15.52:0/594548048 conn(0x7f73900fbf90 msgr2=0x7f7390102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:16.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.951+0000 7f7394e1a640 1 -- 172.21.15.52:0/594548048 shutdown_connections 2024-04-02T13:53:16.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:16.951+0000 7f7394e1a640 1 -- 172.21.15.52:0/594548048 wait complete. 2024-04-02T13:53:16.965 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:53:17.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.068+0000 7fb38d175640 1 -- 172.21.15.52:0/824492519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3881025b0 msgr2=0x7fb388108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:17.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.068+0000 7fb38d175640 1 --2- 172.21.15.52:0/824492519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3881025b0 0x7fb388108e40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb380009ca0 tx=0x7fb38002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.068+0000 7fb38d175640 1 -- 172.21.15.52:0/824492519 shutdown_connections 2024-04-02T13:53:17.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.068+0000 7fb38d175640 1 --2- 172.21.15.52:0/824492519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3881025b0 0x7fb388108e40 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.068+0000 7fb38d175640 1 --2- 172.21.15.52:0/824492519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb388101bf0 0x7fb388102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.068+0000 7fb38d175640 1 --2- 172.21.15.52:0/824492519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb388100a00 0x7fb388100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.068+0000 7fb38d175640 1 -- 172.21.15.52:0/824492519 >> 172.21.15.52:0/824492519 conn(0x7fb3880fbf90 msgr2=0x7fb3880fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:17.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.068+0000 7fb38d175640 1 -- 172.21.15.52:0/824492519 shutdown_connections 2024-04-02T13:53:17.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.068+0000 7fb38d175640 1 -- 172.21.15.52:0/824492519 wait complete. 2024-04-02T13:53:17.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.068+0000 7fb38d175640 1 Processor -- start 2024-04-02T13:53:17.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.069+0000 7fb38d175640 1 -- start start 2024-04-02T13:53:17.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.069+0000 7fb38d175640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb388100a00 0x7fb3880697f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:17.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.069+0000 7fb38d175640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb388101bf0 0x7fb38819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:17.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.069+0000 7fb38d175640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3881025b0 0x7fb38819f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:17.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.069+0000 7fb38d175640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb38810c680 con 0x7fb388100a00 2024-04-02T13:53:17.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.069+0000 7fb38d175640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb38810c500 con 0x7fb388101bf0 2024-04-02T13:53:17.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.069+0000 7fb38d175640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb38810c800 con 0x7fb3881025b0 2024-04-02T13:53:17.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.069+0000 7fb386d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb388100a00 0x7fb3880697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:17.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.069+0000 7fb386d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb388100a00 0x7fb3880697f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40812/0 (socket says 172.21.15.52:40812) 2024-04-02T13:53:17.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.069+0000 7fb386d9d640 1 -- 172.21.15.52:0/3529433114 learned_addr learned my addr 172.21.15.52:0/3529433114 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:17.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.069+0000 7fb38759e640 1 --2- 172.21.15.52:0/3529433114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3881025b0 0x7fb38819f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:17.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.069+0000 7fb38659c640 1 --2- 172.21.15.52:0/3529433114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb388101bf0 0x7fb38819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:17.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.069+0000 7fb386d9d640 1 -- 172.21.15.52:0/3529433114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3881025b0 msgr2=0x7fb38819f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:17.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.069+0000 7fb386d9d640 1 --2- 172.21.15.52:0/3529433114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3881025b0 0x7fb38819f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.069+0000 7fb386d9d640 1 -- 172.21.15.52:0/3529433114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb388101bf0 msgr2=0x7fb38819cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:17.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.069+0000 7fb386d9d640 1 --2- 172.21.15.52:0/3529433114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb388101bf0 0x7fb38819cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.069+0000 7fb386d9d640 1 -- 172.21.15.52:0/3529433114 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38819fa20 con 0x7fb388100a00 2024-04-02T13:53:17.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.069+0000 7fb386d9d640 1 --2- 172.21.15.52:0/3529433114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb388100a00 0x7fb3880697f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb37800d950 tx=0x7fb37800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:17.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.070+0000 7fb36ffff640 1 -- 172.21.15.52:0/3529433114 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378017020 con 0x7fb388100a00 2024-04-02T13:53:17.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.070+0000 7fb36ffff640 1 -- 172.21.15.52:0/3529433114 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb378009c30 con 0x7fb388100a00 2024-04-02T13:53:17.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.070+0000 7fb38d175640 1 -- 172.21.15.52:0/3529433114 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb38819fd10 con 0x7fb388100a00 2024-04-02T13:53:17.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.070+0000 7fb36ffff640 1 -- 172.21.15.52:0/3529433114 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37800f740 con 0x7fb388100a00 2024-04-02T13:53:17.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.070+0000 7fb38d175640 1 -- 172.21.15.52:0/3529433114 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3881a0200 con 0x7fb388100a00 2024-04-02T13:53:17.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.071+0000 7fb38d175640 1 -- 172.21.15.52:0/3529433114 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb388107d20 con 0x7fb388100a00 2024-04-02T13:53:17.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.072+0000 7fb36ffff640 1 -- 172.21.15.52:0/3529433114 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb37801a020 con 0x7fb388100a00 2024-04-02T13:53:17.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.072+0000 7fb36ffff640 1 --2- 172.21.15.52:0/3529433114 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb3700a3bb0 0x7fb3700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:17.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.072+0000 7fb38659c640 1 --2- 172.21.15.52:0/3529433114 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb3700a3bb0 0x7fb3700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:17.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.073+0000 7fb36ffff640 1 -- 172.21.15.52:0/3529433114 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb3780d0530 con 0x7fb388100a00 2024-04-02T13:53:17.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.073+0000 7fb38659c640 1 --2- 172.21.15.52:0/3529433114 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb3700a3bb0 0x7fb3700a6070 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb388069560 tx=0x7fb37c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:17.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.076+0000 7fb36ffff640 1 -- 172.21.15.52:0/3529433114 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb378021070 con 0x7fb388100a00 2024-04-02T13:53:17.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.263+0000 7fb38d175640 1 -- 172.21.15.52:0/3529433114 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb38802d500 con 0x7fb388100a00 2024-04-02T13:53:17.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.263+0000 7fb36ffff640 1 -- 172.21.15.52:0/3529433114 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7fb378009de0 con 0x7fb388100a00 2024-04-02T13:53:17.267 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-04-02T13:53:17.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.265+0000 7fb38d175640 1 -- 172.21.15.52:0/3529433114 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb3700a3bb0 msgr2=0x7fb3700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:17.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.265+0000 7fb38d175640 1 --2- 172.21.15.52:0/3529433114 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb3700a3bb0 0x7fb3700a6070 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb388069560 tx=0x7fb37c005f90 comp rx=0 tx=0).stop 2024-04-02T13:53:17.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.265+0000 7fb38d175640 1 -- 172.21.15.52:0/3529433114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb388100a00 msgr2=0x7fb3880697f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:17.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.265+0000 7fb38d175640 1 --2- 172.21.15.52:0/3529433114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb388100a00 0x7fb3880697f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb37800d950 tx=0x7fb37800de10 comp rx=0 tx=0).stop 2024-04-02T13:53:17.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.265+0000 7fb38d175640 1 -- 172.21.15.52:0/3529433114 shutdown_connections 2024-04-02T13:53:17.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.265+0000 7fb38d175640 1 --2- 172.21.15.52:0/3529433114 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fb3700a3bb0 0x7fb3700a6070 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.265+0000 7fb38d175640 1 --2- 172.21.15.52:0/3529433114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3881025b0 0x7fb38819f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.265+0000 7fb38d175640 1 --2- 172.21.15.52:0/3529433114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb388101bf0 0x7fb38819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.265+0000 7fb38d175640 1 --2- 172.21.15.52:0/3529433114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb388100a00 0x7fb3880697f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.265+0000 7fb38d175640 1 -- 172.21.15.52:0/3529433114 >> 172.21.15.52:0/3529433114 conn(0x7fb3880fbf90 msgr2=0x7fb388104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:17.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.266+0000 7fb38d175640 1 -- 172.21.15.52:0/3529433114 shutdown_connections 2024-04-02T13:53:17.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.266+0000 7fb38d175640 1 -- 172.21.15.52:0/3529433114 wait complete. 2024-04-02T13:53:17.280 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:53:17.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.382+0000 7feb73e06640 1 -- 172.21.15.52:0/220809219 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb6c100a00 msgr2=0x7feb6c100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:17.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.382+0000 7feb73e06640 1 --2- 172.21.15.52:0/220809219 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb6c100a00 0x7feb6c100e20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7feb64009900 tx=0x7feb6402bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:17.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.382+0000 7feb73e06640 1 -- 172.21.15.52:0/220809219 shutdown_connections 2024-04-02T13:53:17.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.383+0000 7feb73e06640 1 --2- 172.21.15.52:0/220809219 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb6c1025b0 0x7feb6c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.383+0000 7feb73e06640 1 --2- 172.21.15.52:0/220809219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb6c101bf0 0x7feb6c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.383+0000 7feb73e06640 1 --2- 172.21.15.52:0/220809219 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb6c100a00 0x7feb6c100e20 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.383+0000 7feb73e06640 1 -- 172.21.15.52:0/220809219 >> 172.21.15.52:0/220809219 conn(0x7feb6c0fbf90 msgr2=0x7feb6c0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:17.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.383+0000 7feb73e06640 1 -- 172.21.15.52:0/220809219 shutdown_connections 2024-04-02T13:53:17.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.383+0000 7feb73e06640 1 -- 172.21.15.52:0/220809219 wait complete. 2024-04-02T13:53:17.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.383+0000 7feb73e06640 1 Processor -- start 2024-04-02T13:53:17.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.383+0000 7feb73e06640 1 -- start start 2024-04-02T13:53:17.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.383+0000 7feb73e06640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb6c100a00 0x7feb6c196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:17.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.383+0000 7feb73e06640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb6c101bf0 0x7feb6c197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:17.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.383+0000 7feb73e06640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb6c1025b0 0x7feb6c19e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:17.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.383+0000 7feb73e06640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c10c650 con 0x7feb6c100a00 2024-04-02T13:53:17.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.383+0000 7feb73e06640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c10c4d0 con 0x7feb6c101bf0 2024-04-02T13:53:17.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.383+0000 7feb73e06640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c10c7d0 con 0x7feb6c1025b0 2024-04-02T13:53:17.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.383+0000 7feb71ba2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb6c100a00 0x7feb6c196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:17.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.383+0000 7feb71ba2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb6c100a00 0x7feb6c196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40824/0 (socket says 172.21.15.52:40824) 2024-04-02T13:53:17.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.383+0000 7feb71ba2640 1 -- 172.21.15.52:0/570545073 learned_addr learned my addr 172.21.15.52:0/570545073 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:17.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.384+0000 7feb713a1640 1 --2- 172.21.15.52:0/570545073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb6c101bf0 0x7feb6c197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:17.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.384+0000 7feb723a3640 1 --2- 172.21.15.52:0/570545073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb6c1025b0 0x7feb6c19e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:17.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.384+0000 7feb71ba2640 1 -- 172.21.15.52:0/570545073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb6c1025b0 msgr2=0x7feb6c19e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:17.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.384+0000 7feb71ba2640 1 --2- 172.21.15.52:0/570545073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb6c1025b0 0x7feb6c19e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.384+0000 7feb71ba2640 1 -- 172.21.15.52:0/570545073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb6c101bf0 msgr2=0x7feb6c197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:17.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.384+0000 7feb71ba2640 1 --2- 172.21.15.52:0/570545073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb6c101bf0 0x7feb6c197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.384+0000 7feb71ba2640 1 -- 172.21.15.52:0/570545073 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c19e920 con 0x7feb6c100a00 2024-04-02T13:53:17.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.384+0000 7feb723a3640 1 --2- 172.21.15.52:0/570545073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb6c1025b0 0x7feb6c19e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:53:17.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.384+0000 7feb713a1640 1 --2- 172.21.15.52:0/570545073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb6c101bf0 0x7feb6c197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:17.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.384+0000 7feb71ba2640 1 --2- 172.21.15.52:0/570545073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb6c100a00 0x7feb6c196d10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7feb64000c00 tx=0x7feb64004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:17.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.384+0000 7feb6affd640 1 -- 172.21.15.52:0/570545073 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb64041070 con 0x7feb6c100a00 2024-04-02T13:53:17.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.384+0000 7feb6affd640 1 -- 172.21.15.52:0/570545073 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb640054e0 con 0x7feb6c100a00 2024-04-02T13:53:17.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.384+0000 7feb73e06640 1 -- 172.21.15.52:0/570545073 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6c19ebb0 con 0x7feb6c100a00 2024-04-02T13:53:17.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.384+0000 7feb6affd640 1 -- 172.21.15.52:0/570545073 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb640346a0 con 0x7feb6c100a00 2024-04-02T13:53:17.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.384+0000 7feb73e06640 1 -- 172.21.15.52:0/570545073 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb6c19f0a0 con 0x7feb6c100a00 2024-04-02T13:53:17.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.385+0000 7feb73e06640 1 -- 172.21.15.52:0/570545073 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6c107d20 con 0x7feb6c100a00 2024-04-02T13:53:17.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.386+0000 7feb6affd640 1 -- 172.21.15.52:0/570545073 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7feb64044070 con 0x7feb6c100a00 2024-04-02T13:53:17.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.387+0000 7feb6affd640 1 --2- 172.21.15.52:0/570545073 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7feb540a3bb0 0x7feb540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:17.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.387+0000 7feb713a1640 1 --2- 172.21.15.52:0/570545073 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7feb540a3bb0 0x7feb540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:17.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.387+0000 7feb6affd640 1 -- 172.21.15.52:0/570545073 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7feb640f2860 con 0x7feb6c100a00 2024-04-02T13:53:17.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.388+0000 7feb713a1640 1 --2- 172.21.15.52:0/570545073 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7feb540a3bb0 0x7feb540a6070 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7feb6000abe0 tx=0x7feb6000a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:17.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.391+0000 7feb6affd640 1 -- 172.21.15.52:0/570545073 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7feb640bee80 con 0x7feb6c100a00 2024-04-02T13:53:17.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.579+0000 7feb73e06640 1 -- 172.21.15.52:0/570545073 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7feb6c02d600 con 0x7feb6c100a00 2024-04-02T13:53:17.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.579+0000 7feb6affd640 1 -- 172.21.15.52:0/570545073 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7feb64003e20 con 0x7feb6c100a00 2024-04-02T13:53:17.583 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-04-02T13:53:17.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.581+0000 7feb73e06640 1 -- 172.21.15.52:0/570545073 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7feb540a3bb0 msgr2=0x7feb540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:17.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.581+0000 7feb73e06640 1 --2- 172.21.15.52:0/570545073 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7feb540a3bb0 0x7feb540a6070 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7feb6000abe0 tx=0x7feb6000a5c0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.581+0000 7feb73e06640 1 -- 172.21.15.52:0/570545073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb6c100a00 msgr2=0x7feb6c196d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:17.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.581+0000 7feb73e06640 1 --2- 172.21.15.52:0/570545073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb6c100a00 0x7feb6c196d10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7feb64000c00 tx=0x7feb64004040 comp rx=0 tx=0).stop 2024-04-02T13:53:17.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.581+0000 7feb73e06640 1 -- 172.21.15.52:0/570545073 shutdown_connections 2024-04-02T13:53:17.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.581+0000 7feb73e06640 1 --2- 172.21.15.52:0/570545073 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7feb540a3bb0 0x7feb540a6070 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.581+0000 7feb73e06640 1 --2- 172.21.15.52:0/570545073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb6c1025b0 0x7feb6c19e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.581+0000 7feb73e06640 1 --2- 172.21.15.52:0/570545073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb6c101bf0 0x7feb6c197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.581+0000 7feb73e06640 1 --2- 172.21.15.52:0/570545073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb6c100a00 0x7feb6c196d10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.581+0000 7feb73e06640 1 -- 172.21.15.52:0/570545073 >> 172.21.15.52:0/570545073 conn(0x7feb6c0fbf90 msgr2=0x7feb6c104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:17.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.581+0000 7feb73e06640 1 -- 172.21.15.52:0/570545073 shutdown_connections 2024-04-02T13:53:17.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.581+0000 7feb73e06640 1 -- 172.21.15.52:0/570545073 wait complete. 2024-04-02T13:53:17.595 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:53:17.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.698+0000 7fa226dcd640 1 -- 172.21.15.52:0/3638636393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa220100a10 msgr2=0x7fa220100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:17.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.698+0000 7fa226dcd640 1 --2- 172.21.15.52:0/3638636393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa220100a10 0x7fa220100e30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa218009ca0 tx=0x7fa21802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.698+0000 7fa226dcd640 1 -- 172.21.15.52:0/3638636393 shutdown_connections 2024-04-02T13:53:17.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.698+0000 7fa226dcd640 1 --2- 172.21.15.52:0/3638636393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2201025c0 0x7fa220108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.698+0000 7fa226dcd640 1 --2- 172.21.15.52:0/3638636393 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa220101c00 0x7fa220102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.698+0000 7fa226dcd640 1 --2- 172.21.15.52:0/3638636393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa220100a10 0x7fa220100e30 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.698+0000 7fa226dcd640 1 -- 172.21.15.52:0/3638636393 >> 172.21.15.52:0/3638636393 conn(0x7fa2200fbfa0 msgr2=0x7fa2200fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:17.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.698+0000 7fa226dcd640 1 -- 172.21.15.52:0/3638636393 shutdown_connections 2024-04-02T13:53:17.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.698+0000 7fa226dcd640 1 -- 172.21.15.52:0/3638636393 wait complete. 2024-04-02T13:53:17.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.698+0000 7fa226dcd640 1 Processor -- start 2024-04-02T13:53:17.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.698+0000 7fa226dcd640 1 -- start start 2024-04-02T13:53:17.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.698+0000 7fa226dcd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa220100a10 0x7fa22010b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:17.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.698+0000 7fa226dcd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa220101c00 0x7fa2201099d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:17.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.698+0000 7fa226dcd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2201025c0 0x7fa220109f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:17.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.698+0000 7fa226dcd640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa22010d7b0 con 0x7fa2201025c0 2024-04-02T13:53:17.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.698+0000 7fa226dcd640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa22010d630 con 0x7fa220101c00 2024-04-02T13:53:17.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.698+0000 7fa226dcd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa22010d930 con 0x7fa220100a10 2024-04-02T13:53:17.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.699+0000 7fa22536a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2201025c0 0x7fa220109f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:17.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.699+0000 7fa22536a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2201025c0 0x7fa220109f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40840/0 (socket says 172.21.15.52:40840) 2024-04-02T13:53:17.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.699+0000 7fa22536a640 1 -- 172.21.15.52:0/3376118958 learned_addr learned my addr 172.21.15.52:0/3376118958 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:17.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.699+0000 7fa224b69640 1 --2- 172.21.15.52:0/3376118958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa220100a10 0x7fa22010b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:17.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.699+0000 7fa21ffff640 1 --2- 172.21.15.52:0/3376118958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa220101c00 0x7fa2201099d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:17.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.699+0000 7fa22536a640 1 -- 172.21.15.52:0/3376118958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa220100a10 msgr2=0x7fa22010b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:17.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.699+0000 7fa22536a640 1 --2- 172.21.15.52:0/3376118958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa220100a10 0x7fa22010b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.699+0000 7fa22536a640 1 -- 172.21.15.52:0/3376118958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa220101c00 msgr2=0x7fa2201099d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:17.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.699+0000 7fa22536a640 1 --2- 172.21.15.52:0/3376118958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa220101c00 0x7fa2201099d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.699+0000 7fa22536a640 1 -- 172.21.15.52:0/3376118958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22010a7a0 con 0x7fa2201025c0 2024-04-02T13:53:17.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.699+0000 7fa224b69640 1 --2- 172.21.15.52:0/3376118958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa220100a10 0x7fa22010b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:53:17.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.699+0000 7fa22536a640 1 --2- 172.21.15.52:0/3376118958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2201025c0 0x7fa220109f10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa21000d750 tx=0x7fa21000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:17.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.699+0000 7fa21dffb640 1 -- 172.21.15.52:0/3376118958 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa210017070 con 0x7fa2201025c0 2024-04-02T13:53:17.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.699+0000 7fa226dcd640 1 -- 172.21.15.52:0/3376118958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2201a1ef0 con 0x7fa2201025c0 2024-04-02T13:53:17.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.699+0000 7fa21dffb640 1 -- 172.21.15.52:0/3376118958 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa210005500 con 0x7fa2201025c0 2024-04-02T13:53:17.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.699+0000 7fa21dffb640 1 -- 172.21.15.52:0/3376118958 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa210004d90 con 0x7fa2201025c0 2024-04-02T13:53:17.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.699+0000 7fa226dcd640 1 -- 172.21.15.52:0/3376118958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2201a2350 con 0x7fa2201025c0 2024-04-02T13:53:17.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.700+0000 7fa226dcd640 1 -- 172.21.15.52:0/3376118958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa220107d80 con 0x7fa2201025c0 2024-04-02T13:53:17.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.701+0000 7fa21dffb640 1 -- 172.21.15.52:0/3376118958 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa210004f30 con 0x7fa2201025c0 2024-04-02T13:53:17.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.702+0000 7fa21dffb640 1 --2- 172.21.15.52:0/3376118958 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa2080a3bb0 0x7fa2080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:17.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.702+0000 7fa224b69640 1 --2- 172.21.15.52:0/3376118958 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa2080a3bb0 0x7fa2080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:17.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.702+0000 7fa21dffb640 1 -- 172.21.15.52:0/3376118958 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fa2100d0150 con 0x7fa2201025c0 2024-04-02T13:53:17.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.702+0000 7fa224b69640 1 --2- 172.21.15.52:0/3376118958 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa2080a3bb0 0x7fa2080a6070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa218006010 tx=0x7fa218005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:17.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.706+0000 7fa21dffb640 1 -- 172.21.15.52:0/3376118958 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa21009c7d0 con 0x7fa2201025c0 2024-04-02T13:53:17.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:17 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1015451592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:53:17.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:17 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/594548048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:53:17.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:17 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3529433114' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:53:17.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:17 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1015451592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:53:17.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:17 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/594548048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:53:17.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:17 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3529433114' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:53:17.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.891+0000 7fa226dcd640 1 -- 172.21.15.52:0/3376118958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa22002d370 con 0x7fa2201025c0 2024-04-02T13:53:17.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.892+0000 7fa21dffb640 1 -- 172.21.15.52:0/3376118958 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7fa21009c0d0 con 0x7fa2201025c0 2024-04-02T13:53:17.895 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-04-02T13:53:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.893+0000 7fa226dcd640 1 -- 172.21.15.52:0/3376118958 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa2080a3bb0 msgr2=0x7fa2080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.893+0000 7fa226dcd640 1 --2- 172.21.15.52:0/3376118958 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa2080a3bb0 0x7fa2080a6070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa218006010 tx=0x7fa218005c50 comp rx=0 tx=0).stop 2024-04-02T13:53:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.893+0000 7fa226dcd640 1 -- 172.21.15.52:0/3376118958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2201025c0 msgr2=0x7fa220109f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.893+0000 7fa226dcd640 1 --2- 172.21.15.52:0/3376118958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2201025c0 0x7fa220109f10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa21000d750 tx=0x7fa21000dc10 comp rx=0 tx=0).stop 2024-04-02T13:53:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.894+0000 7fa226dcd640 1 -- 172.21.15.52:0/3376118958 shutdown_connections 2024-04-02T13:53:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.894+0000 7fa226dcd640 1 --2- 172.21.15.52:0/3376118958 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa2080a3bb0 0x7fa2080a6070 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.894+0000 7fa226dcd640 1 --2- 172.21.15.52:0/3376118958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2201025c0 0x7fa220109f10 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.894+0000 7fa226dcd640 1 --2- 172.21.15.52:0/3376118958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa220101c00 0x7fa2201099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.894+0000 7fa226dcd640 1 --2- 172.21.15.52:0/3376118958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa220100a10 0x7fa22010b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:17.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.894+0000 7fa226dcd640 1 -- 172.21.15.52:0/3376118958 >> 172.21.15.52:0/3376118958 conn(0x7fa2200fbfa0 msgr2=0x7fa220104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:17.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.894+0000 7fa226dcd640 1 -- 172.21.15.52:0/3376118958 shutdown_connections 2024-04-02T13:53:17.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:17.894+0000 7fa226dcd640 1 -- 172.21.15.52:0/3376118958 wait complete. 2024-04-02T13:53:17.910 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:53:17.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:17 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1015451592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:53:17.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:17 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/594548048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:53:17.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:17 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3529433114' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:53:18.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.010+0000 7f2b8c8c5640 1 -- 172.21.15.52:0/2475559523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b88102dc0 msgr2=0x7f2b88103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:18.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.010+0000 7f2b8c8c5640 1 --2- 172.21.15.52:0/2475559523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b88102dc0 0x7f2b88103240 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009930 tx=0x7f2b7002bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:18.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.010+0000 7f2b8c8c5640 1 -- 172.21.15.52:0/2475559523 shutdown_connections 2024-04-02T13:53:18.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.010+0000 7f2b8c8c5640 1 --2- 172.21.15.52:0/2475559523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b88103780 0x7f2b8810a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.010+0000 7f2b8c8c5640 1 --2- 172.21.15.52:0/2475559523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b88102dc0 0x7f2b88103240 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.010+0000 7f2b8c8c5640 1 --2- 172.21.15.52:0/2475559523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b880ffdf0 0x7f2b88100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.010+0000 7f2b8c8c5640 1 -- 172.21.15.52:0/2475559523 >> 172.21.15.52:0/2475559523 conn(0x7f2b880fb720 msgr2=0x7f2b880fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:18.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.010+0000 7f2b8c8c5640 1 -- 172.21.15.52:0/2475559523 shutdown_connections 2024-04-02T13:53:18.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.010+0000 7f2b8c8c5640 1 -- 172.21.15.52:0/2475559523 wait complete. 2024-04-02T13:53:18.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.011+0000 7f2b8c8c5640 1 Processor -- start 2024-04-02T13:53:18.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.011+0000 7f2b8c8c5640 1 -- start start 2024-04-02T13:53:18.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.011+0000 7f2b8c8c5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b880ffdf0 0x7f2b88197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:18.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.011+0000 7f2b8c8c5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b88102dc0 0x7f2b88198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:18.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.011+0000 7f2b8c8c5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b88103780 0x7f2b8819f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:18.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.011+0000 7f2b8c8c5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8810d7b0 con 0x7f2b88103780 2024-04-02T13:53:18.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.011+0000 7f2b8c8c5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8810d630 con 0x7f2b880ffdf0 2024-04-02T13:53:18.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.011+0000 7f2b8c8c5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8810d930 con 0x7f2b88102dc0 2024-04-02T13:53:18.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.011+0000 7f2b86d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b88103780 0x7f2b8819f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:18.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.011+0000 7f2b86d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b88103780 0x7f2b8819f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40850/0 (socket says 172.21.15.52:40850) 2024-04-02T13:53:18.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.011+0000 7f2b86d9d640 1 -- 172.21.15.52:0/3174382691 learned_addr learned my addr 172.21.15.52:0/3174382691 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:18.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.011+0000 7f2b8659c640 1 --2- 172.21.15.52:0/3174382691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b880ffdf0 0x7f2b88197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:18.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.011+0000 7f2b86d9d640 1 -- 172.21.15.52:0/3174382691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b88102dc0 msgr2=0x7f2b88198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:18.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.011+0000 7f2b86d9d640 1 --2- 172.21.15.52:0/3174382691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b88102dc0 0x7f2b88198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.011+0000 7f2b86d9d640 1 -- 172.21.15.52:0/3174382691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b880ffdf0 msgr2=0x7f2b88197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:18.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.011+0000 7f2b86d9d640 1 --2- 172.21.15.52:0/3174382691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b880ffdf0 0x7f2b88197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.011+0000 7f2b86d9d640 1 -- 172.21.15.52:0/3174382691 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8819fa40 con 0x7f2b88103780 2024-04-02T13:53:18.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.011+0000 7f2b8659c640 1 --2- 172.21.15.52:0/3174382691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b880ffdf0 0x7f2b88197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:18.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.012+0000 7f2b86d9d640 1 --2- 172.21.15.52:0/3174382691 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b88103780 0x7f2b8819f3d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2b8000cce0 tx=0x7f2b8000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:18.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.012+0000 7f2b7f7fe640 1 -- 172.21.15.52:0/3174382691 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8000f450 con 0x7f2b88103780 2024-04-02T13:53:18.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.012+0000 7f2b7f7fe640 1 -- 172.21.15.52:0/3174382691 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b8000fad0 con 0x7f2b88103780 2024-04-02T13:53:18.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.012+0000 7f2b8c8c5640 1 -- 172.21.15.52:0/3174382691 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b8819fd30 con 0x7f2b88103780 2024-04-02T13:53:18.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.012+0000 7f2b7f7fe640 1 -- 172.21.15.52:0/3174382691 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b80018c10 con 0x7f2b88103780 2024-04-02T13:53:18.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.012+0000 7f2b8c8c5640 1 -- 172.21.15.52:0/3174382691 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b881a0230 con 0x7f2b88103780 2024-04-02T13:53:18.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.013+0000 7f2b8c8c5640 1 -- 172.21.15.52:0/3174382691 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b88108f10 con 0x7f2b88103780 2024-04-02T13:53:18.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.014+0000 7f2b7f7fe640 1 -- 172.21.15.52:0/3174382691 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2b8000f5f0 con 0x7f2b88103780 2024-04-02T13:53:18.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.014+0000 7f2b7f7fe640 1 --2- 172.21.15.52:0/3174382691 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2b740a3bb0 0x7f2b740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:18.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.015+0000 7f2b7f7fe640 1 -- 172.21.15.52:0/3174382691 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f2b800d0fa0 con 0x7f2b88103780 2024-04-02T13:53:18.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.015+0000 7f2b8659c640 1 --2- 172.21.15.52:0/3174382691 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2b740a3bb0 0x7f2b740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:18.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.015+0000 7f2b8659c640 1 --2- 172.21.15.52:0/3174382691 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2b740a3bb0 0x7f2b740a6070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2b88069fb0 tx=0x7f2b78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:18.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.019+0000 7f2b7f7fe640 1 -- 172.21.15.52:0/3174382691 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2b80014030 con 0x7f2b88103780 2024-04-02T13:53:18.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.245+0000 7f2b8c8c5640 1 -- 172.21.15.52:0/3174382691 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b8802d370 con 0x7f2b88103780 2024-04-02T13:53:18.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.246+0000 7f2b7f7fe640 1 -- 172.21.15.52:0/3174382691 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f2b8009d650 con 0x7f2b88103780 2024-04-02T13:53:18.249 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:53:18.249 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:51:29.345060+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24503,24475,24490,24467,24457]},"id":1}]} 2024-04-02T13:53:18.249 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 27 2024-04-02T13:53:18.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.248+0000 7f2b8c8c5640 1 -- 172.21.15.52:0/3174382691 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2b740a3bb0 msgr2=0x7f2b740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:18.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.248+0000 7f2b8c8c5640 1 --2- 172.21.15.52:0/3174382691 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2b740a3bb0 0x7f2b740a6070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2b88069fb0 tx=0x7f2b78005f90 comp rx=0 tx=0).stop 2024-04-02T13:53:18.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.248+0000 7f2b8c8c5640 1 -- 172.21.15.52:0/3174382691 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b88103780 msgr2=0x7f2b8819f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:18.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.248+0000 7f2b8c8c5640 1 --2- 172.21.15.52:0/3174382691 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b88103780 0x7f2b8819f3d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2b8000cce0 tx=0x7f2b8000a620 comp rx=0 tx=0).stop 2024-04-02T13:53:18.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.248+0000 7f2b8c8c5640 1 -- 172.21.15.52:0/3174382691 shutdown_connections 2024-04-02T13:53:18.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.248+0000 7f2b8c8c5640 1 --2- 172.21.15.52:0/3174382691 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2b740a3bb0 0x7f2b740a6070 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.248+0000 7f2b8c8c5640 1 --2- 172.21.15.52:0/3174382691 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b88103780 0x7f2b8819f3d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.248+0000 7f2b8c8c5640 1 --2- 172.21.15.52:0/3174382691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b88102dc0 0x7f2b88198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.248+0000 7f2b8c8c5640 1 --2- 172.21.15.52:0/3174382691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b880ffdf0 0x7f2b88197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.248+0000 7f2b8c8c5640 1 -- 172.21.15.52:0/3174382691 >> 172.21.15.52:0/3174382691 conn(0x7f2b880fb720 msgr2=0x7f2b881060e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:18.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.248+0000 7f2b8c8c5640 1 -- 172.21.15.52:0/3174382691 shutdown_connections 2024-04-02T13:53:18.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.248+0000 7f2b8c8c5640 1 -- 172.21.15.52:0/3174382691 wait complete. 2024-04-02T13:53:18.263 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:53:18.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.362+0000 7fdbc05e9640 1 -- 172.21.15.52:0/212444828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbb8101c10 msgr2=0x7fdbb8102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:18.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.362+0000 7fdbc05e9640 1 --2- 172.21.15.52:0/212444828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbb8101c10 0x7fdbb8102090 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009900 tx=0x7fdba802bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:18.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.362+0000 7fdbc05e9640 1 -- 172.21.15.52:0/212444828 shutdown_connections 2024-04-02T13:53:18.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.362+0000 7fdbc05e9640 1 --2- 172.21.15.52:0/212444828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbb81025d0 0x7fdbb8108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.362+0000 7fdbc05e9640 1 --2- 172.21.15.52:0/212444828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbb8101c10 0x7fdbb8102090 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.362+0000 7fdbc05e9640 1 --2- 172.21.15.52:0/212444828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbb8100a20 0x7fdbb8100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.362+0000 7fdbc05e9640 1 -- 172.21.15.52:0/212444828 >> 172.21.15.52:0/212444828 conn(0x7fdbb80fbf90 msgr2=0x7fdbb80fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:18.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.362+0000 7fdbc05e9640 1 -- 172.21.15.52:0/212444828 shutdown_connections 2024-04-02T13:53:18.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.363+0000 7fdbc05e9640 1 -- 172.21.15.52:0/212444828 wait complete. 2024-04-02T13:53:18.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.363+0000 7fdbc05e9640 1 Processor -- start 2024-04-02T13:53:18.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.363+0000 7fdbc05e9640 1 -- start start 2024-04-02T13:53:18.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.363+0000 7fdbc05e9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbb8100a20 0x7fdbb810b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:18.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.363+0000 7fdbc05e9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbb8101c10 0x7fdbb81099e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:18.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.363+0000 7fdbc05e9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbb81025d0 0x7fdbb8109f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:18.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.363+0000 7fdbc05e9640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb810d820 con 0x7fdbb8101c10 2024-04-02T13:53:18.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.363+0000 7fdbc05e9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb810d6a0 con 0x7fdbb8100a20 2024-04-02T13:53:18.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.363+0000 7fdbc05e9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb810d9a0 con 0x7fdbb81025d0 2024-04-02T13:53:18.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.363+0000 7fdbbdb84640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbb8101c10 0x7fdbb81099e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:18.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.363+0000 7fdbbdb84640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbb8101c10 0x7fdbb81099e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40868/0 (socket says 172.21.15.52:40868) 2024-04-02T13:53:18.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.363+0000 7fdbbdb84640 1 -- 172.21.15.52:0/377906626 learned_addr learned my addr 172.21.15.52:0/377906626 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:18.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.363+0000 7fdbbe385640 1 --2- 172.21.15.52:0/377906626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbb8100a20 0x7fdbb810b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:18.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.363+0000 7fdbbeb86640 1 --2- 172.21.15.52:0/377906626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbb81025d0 0x7fdbb8109f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:18.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.364+0000 7fdbbdb84640 1 -- 172.21.15.52:0/377906626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbb81025d0 msgr2=0x7fdbb8109f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:18.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.364+0000 7fdbbdb84640 1 --2- 172.21.15.52:0/377906626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbb81025d0 0x7fdbb8109f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.364+0000 7fdbbdb84640 1 -- 172.21.15.52:0/377906626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbb8100a20 msgr2=0x7fdbb810b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:18.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.364+0000 7fdbbdb84640 1 --2- 172.21.15.52:0/377906626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbb8100a20 0x7fdbb810b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.364+0000 7fdbbdb84640 1 -- 172.21.15.52:0/377906626 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb810a7b0 con 0x7fdbb8101c10 2024-04-02T13:53:18.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.364+0000 7fdbbe385640 1 --2- 172.21.15.52:0/377906626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbb8100a20 0x7fdbb810b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:18.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.364+0000 7fdbbeb86640 1 --2- 172.21.15.52:0/377906626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbb81025d0 0x7fdbb8109f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:53:18.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.364+0000 7fdbbdb84640 1 --2- 172.21.15.52:0/377906626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbb8101c10 0x7fdbb81099e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdba802c8e0 tx=0x7fdba8005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:18.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.364+0000 7fdbaf7fe640 1 -- 172.21.15.52:0/377906626 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8043020 con 0x7fdbb8101c10 2024-04-02T13:53:18.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.364+0000 7fdbaf7fe640 1 -- 172.21.15.52:0/377906626 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdba80054e0 con 0x7fdbb8101c10 2024-04-02T13:53:18.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.364+0000 7fdbc05e9640 1 -- 172.21.15.52:0/377906626 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb81a1f90 con 0x7fdbb8101c10 2024-04-02T13:53:18.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.364+0000 7fdbaf7fe640 1 -- 172.21.15.52:0/377906626 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8003d40 con 0x7fdbb8101c10 2024-04-02T13:53:18.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.364+0000 7fdbc05e9640 1 -- 172.21.15.52:0/377906626 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb81a2400 con 0x7fdbb8101c10 2024-04-02T13:53:18.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.365+0000 7fdbc05e9640 1 -- 172.21.15.52:0/377906626 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb8107ce0 con 0x7fdbb8101c10 2024-04-02T13:53:18.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.366+0000 7fdbaf7fe640 1 -- 172.21.15.52:0/377906626 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdba8046020 con 0x7fdbb8101c10 2024-04-02T13:53:18.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.367+0000 7fdbaf7fe640 1 --2- 172.21.15.52:0/377906626 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdba40a3c30 0x7fdba40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:18.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.367+0000 7fdbbe385640 1 --2- 172.21.15.52:0/377906626 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdba40a3c30 0x7fdba40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:18.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.367+0000 7fdbaf7fe640 1 -- 172.21.15.52:0/377906626 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdba80f2cf0 con 0x7fdbb8101c10 2024-04-02T13:53:18.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.367+0000 7fdbbe385640 1 --2- 172.21.15.52:0/377906626 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdba40a3c30 0x7fdba40a60f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8101a70 tx=0x7fdbb000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:18.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.370+0000 7fdbaf7fe640 1 -- 172.21.15.52:0/377906626 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fdba804d020 con 0x7fdbb8101c10 2024-04-02T13:53:18.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.559+0000 7fdbc05e9640 1 -- 172.21.15.52:0/377906626 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdbb802d5e0 con 0x7fdbb8101c10 2024-04-02T13:53:18.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.561+0000 7fdbaf7fe640 1 -- 172.21.15.52:0/377906626 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24549 (secure 0 0 0) 0x7fdba80bf430 con 0x7fdbb8101c10 2024-04-02T13:53:18.564 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:53:18.564 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":92,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T13:51:56.472064+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:50:58.778690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:50:59.561527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:51:03.343487+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:53:18.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.563+0000 7fdbc05e9640 1 -- 172.21.15.52:0/377906626 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdba40a3c30 msgr2=0x7fdba40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:18.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.563+0000 7fdbc05e9640 1 --2- 172.21.15.52:0/377906626 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdba40a3c30 0x7fdba40a60f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8101a70 tx=0x7fdbb000a560 comp rx=0 tx=0).stop 2024-04-02T13:53:18.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.563+0000 7fdbc05e9640 1 -- 172.21.15.52:0/377906626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbb8101c10 msgr2=0x7fdbb81099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:18.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.563+0000 7fdbc05e9640 1 --2- 172.21.15.52:0/377906626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbb8101c10 0x7fdbb81099e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdba802c8e0 tx=0x7fdba8005d50 comp rx=0 tx=0).stop 2024-04-02T13:53:18.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.563+0000 7fdbc05e9640 1 -- 172.21.15.52:0/377906626 shutdown_connections 2024-04-02T13:53:18.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.563+0000 7fdbc05e9640 1 --2- 172.21.15.52:0/377906626 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdba40a3c30 0x7fdba40a60f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.563+0000 7fdbc05e9640 1 --2- 172.21.15.52:0/377906626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdbb81025d0 0x7fdbb8109f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.563+0000 7fdbc05e9640 1 --2- 172.21.15.52:0/377906626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdbb8101c10 0x7fdbb81099e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.563+0000 7fdbc05e9640 1 --2- 172.21.15.52:0/377906626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbb8100a20 0x7fdbb810b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.563+0000 7fdbc05e9640 1 -- 172.21.15.52:0/377906626 >> 172.21.15.52:0/377906626 conn(0x7fdbb80fbf90 msgr2=0x7fdbb8104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:18.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.563+0000 7fdbc05e9640 1 -- 172.21.15.52:0/377906626 shutdown_connections 2024-04-02T13:53:18.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.563+0000 7fdbc05e9640 1 -- 172.21.15.52:0/377906626 wait complete. 2024-04-02T13:53:18.577 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:46:20.746053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:50:58.778690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:50:59.561527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:51:03.343487+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:53:18.578 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:53:18.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.680+0000 7f3e20ca5640 1 -- 172.21.15.52:0/3738370034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e1c1036e0 msgr2=0x7f3e1c109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:18.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.680+0000 7f3e20ca5640 1 --2- 172.21.15.52:0/3738370034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e1c1036e0 0x7f3e1c109f70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009ca0 tx=0x7f3e0c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.680+0000 7f3e20ca5640 1 -- 172.21.15.52:0/3738370034 shutdown_connections 2024-04-02T13:53:18.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.680+0000 7f3e20ca5640 1 --2- 172.21.15.52:0/3738370034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e1c1036e0 0x7f3e1c109f70 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.680+0000 7f3e20ca5640 1 --2- 172.21.15.52:0/3738370034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e1c102d20 0x7f3e1c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.680+0000 7f3e20ca5640 1 --2- 172.21.15.52:0/3738370034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e1c101b30 0x7f3e1c101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.680+0000 7f3e20ca5640 1 -- 172.21.15.52:0/3738370034 >> 172.21.15.52:0/3738370034 conn(0x7f3e1c0fd0a0 msgr2=0x7f3e1c0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:18.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.681+0000 7f3e20ca5640 1 -- 172.21.15.52:0/3738370034 shutdown_connections 2024-04-02T13:53:18.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.681+0000 7f3e20ca5640 1 -- 172.21.15.52:0/3738370034 wait complete. 2024-04-02T13:53:18.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.681+0000 7f3e20ca5640 1 Processor -- start 2024-04-02T13:53:18.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.681+0000 7f3e20ca5640 1 -- start start 2024-04-02T13:53:18.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.681+0000 7f3e20ca5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e1c101b30 0x7f3e1c199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:18.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.681+0000 7f3e20ca5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e1c102d20 0x7f3e1c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:18.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.681+0000 7f3e20ca5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e1c1036e0 0x7f3e1c1a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:18.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.681+0000 7f3e20ca5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c10d790 con 0x7f3e1c101b30 2024-04-02T13:53:18.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.681+0000 7f3e20ca5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c10d610 con 0x7f3e1c1036e0 2024-04-02T13:53:18.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.681+0000 7f3e20ca5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c10d910 con 0x7f3e1c102d20 2024-04-02T13:53:18.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.681+0000 7f3e1a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e1c101b30 0x7f3e1c199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:18.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.682+0000 7f3e1a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e1c101b30 0x7f3e1c199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40890/0 (socket says 172.21.15.52:40890) 2024-04-02T13:53:18.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.682+0000 7f3e1a59c640 1 -- 172.21.15.52:0/1241626752 learned_addr learned my addr 172.21.15.52:0/1241626752 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:18.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.682+0000 7f3e1ad9d640 1 --2- 172.21.15.52:0/1241626752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e1c1036e0 0x7f3e1c1a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:18.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.682+0000 7f3e19d9b640 1 --2- 172.21.15.52:0/1241626752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e1c102d20 0x7f3e1c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:18.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.682+0000 7f3e1a59c640 1 -- 172.21.15.52:0/1241626752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e1c102d20 msgr2=0x7f3e1c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:18.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.682+0000 7f3e1a59c640 1 --2- 172.21.15.52:0/1241626752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e1c102d20 0x7f3e1c19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.682+0000 7f3e1a59c640 1 -- 172.21.15.52:0/1241626752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e1c1036e0 msgr2=0x7f3e1c1a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:18.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.682+0000 7f3e1a59c640 1 --2- 172.21.15.52:0/1241626752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e1c1036e0 0x7f3e1c1a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.682+0000 7f3e1a59c640 1 -- 172.21.15.52:0/1241626752 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1c1a1b80 con 0x7f3e1c101b30 2024-04-02T13:53:18.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.682+0000 7f3e1a59c640 1 --2- 172.21.15.52:0/1241626752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e1c101b30 0x7f3e1c199f60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400c970 tx=0x7f3e1400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:18.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.682+0000 7f3e137fe640 1 -- 172.21.15.52:0/1241626752 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e14007c10 con 0x7f3e1c101b30 2024-04-02T13:53:18.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.682+0000 7f3e137fe640 1 -- 172.21.15.52:0/1241626752 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e14007db0 con 0x7f3e1c101b30 2024-04-02T13:53:18.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.682+0000 7f3e20ca5640 1 -- 172.21.15.52:0/1241626752 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e1c1a1e70 con 0x7f3e1c101b30 2024-04-02T13:53:18.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.682+0000 7f3e137fe640 1 -- 172.21.15.52:0/1241626752 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1400f940 con 0x7f3e1c101b30 2024-04-02T13:53:18.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.682+0000 7f3e20ca5640 1 -- 172.21.15.52:0/1241626752 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e1c1a22d0 con 0x7f3e1c101b30 2024-04-02T13:53:18.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.683+0000 7f3e20ca5640 1 -- 172.21.15.52:0/1241626752 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e1c1a2590 con 0x7f3e1c101b30 2024-04-02T13:53:18.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.684+0000 7f3e137fe640 1 -- 172.21.15.52:0/1241626752 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3e1400fae0 con 0x7f3e1c101b30 2024-04-02T13:53:18.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.685+0000 7f3e137fe640 1 --2- 172.21.15.52:0/1241626752 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3e080a3bb0 0x7f3e080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:18.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.685+0000 7f3e19d9b640 1 --2- 172.21.15.52:0/1241626752 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3e080a3bb0 0x7f3e080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:18.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.685+0000 7f3e137fe640 1 -- 172.21.15.52:0/1241626752 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f3e140d0e30 con 0x7f3e1c101b30 2024-04-02T13:53:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.685+0000 7f3e19d9b640 1 --2- 172.21.15.52:0/1241626752 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3e080a3bb0 0x7f3e080a6070 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3e04005fd0 tx=0x7f3e04005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:18.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.688+0000 7f3e137fe640 1 -- 172.21.15.52:0/1241626752 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3e1409d570 con 0x7f3e1c101b30 2024-04-02T13:53:18.837 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:18 smithi052 ceph-mon[32104]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-02T13:53:18.837 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:18 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/570545073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:53:18.837 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:18 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3376118958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:53:18.837 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:18 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3174382691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:53:18.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.873+0000 7f3e20ca5640 1 -- 172.21.15.52:0/1241626752 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3e1c1a2920 con 0x7f3e1c101b30 2024-04-02T13:53:18.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.874+0000 7f3e137fe640 1 -- 172.21.15.52:0/1241626752 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f3e1409ce70 con 0x7f3e1c101b30 2024-04-02T13:53:18.878 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-04-02T13:53:18.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.876+0000 7f3e20ca5640 1 -- 172.21.15.52:0/1241626752 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3e080a3bb0 msgr2=0x7f3e080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:18.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.876+0000 7f3e20ca5640 1 --2- 172.21.15.52:0/1241626752 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3e080a3bb0 0x7f3e080a6070 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3e04005fd0 tx=0x7f3e04005850 comp rx=0 tx=0).stop 2024-04-02T13:53:18.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.876+0000 7f3e20ca5640 1 -- 172.21.15.52:0/1241626752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e1c101b30 msgr2=0x7f3e1c199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:18.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.876+0000 7f3e20ca5640 1 --2- 172.21.15.52:0/1241626752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e1c101b30 0x7f3e1c199f60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400c970 tx=0x7f3e1400ce30 comp rx=0 tx=0).stop 2024-04-02T13:53:18.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.876+0000 7f3e20ca5640 1 -- 172.21.15.52:0/1241626752 shutdown_connections 2024-04-02T13:53:18.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.876+0000 7f3e20ca5640 1 --2- 172.21.15.52:0/1241626752 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3e080a3bb0 0x7f3e080a6070 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.876+0000 7f3e20ca5640 1 --2- 172.21.15.52:0/1241626752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e1c1036e0 0x7f3e1c1a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.876+0000 7f3e20ca5640 1 --2- 172.21.15.52:0/1241626752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e1c102d20 0x7f3e1c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.876+0000 7f3e20ca5640 1 --2- 172.21.15.52:0/1241626752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e1c101b30 0x7f3e1c199f60 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.876+0000 7f3e20ca5640 1 -- 172.21.15.52:0/1241626752 >> 172.21.15.52:0/1241626752 conn(0x7f3e1c0fd0a0 msgr2=0x7f3e1c073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:18.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.876+0000 7f3e20ca5640 1 -- 172.21.15.52:0/1241626752 shutdown_connections 2024-04-02T13:53:18.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.876+0000 7f3e20ca5640 1 -- 172.21.15.52:0/1241626752 wait complete. 2024-04-02T13:53:18.890 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:53:18.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:18 smithi088 ceph-mon[36528]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-02T13:53:18.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:18 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/570545073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:53:18.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:18 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3376118958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:53:18.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:18 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3174382691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:53:18.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.993+0000 7fa11bf52640 1 -- 172.21.15.52:0/4004370901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa114101bf0 msgr2=0x7fa114102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:18.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.993+0000 7fa11bf52640 1 --2- 172.21.15.52:0/4004370901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa114101bf0 0x7fa114102070 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa104009900 tx=0x7fa10402bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:18.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.993+0000 7fa11bf52640 1 -- 172.21.15.52:0/4004370901 shutdown_connections 2024-04-02T13:53:18.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.993+0000 7fa11bf52640 1 --2- 172.21.15.52:0/4004370901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1141025b0 0x7fa114108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.993+0000 7fa11bf52640 1 --2- 172.21.15.52:0/4004370901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa114101bf0 0x7fa114102070 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.993+0000 7fa11bf52640 1 --2- 172.21.15.52:0/4004370901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa114100a00 0x7fa114100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:18.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.993+0000 7fa11bf52640 1 -- 172.21.15.52:0/4004370901 >> 172.21.15.52:0/4004370901 conn(0x7fa1140fbf90 msgr2=0x7fa1140fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:18.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.993+0000 7fa11bf52640 1 -- 172.21.15.52:0/4004370901 shutdown_connections 2024-04-02T13:53:18.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.993+0000 7fa11bf52640 1 -- 172.21.15.52:0/4004370901 wait complete. 2024-04-02T13:53:18.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.993+0000 7fa11bf52640 1 Processor -- start 2024-04-02T13:53:18.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.993+0000 7fa11bf52640 1 -- start start 2024-04-02T13:53:18.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.993+0000 7fa11bf52640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa114100a00 0x7fa1140697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:18.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.993+0000 7fa11bf52640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa114101bf0 0x7fa11419cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:18.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.993+0000 7fa11bf52640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1141025b0 0x7fa11419f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:18.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.993+0000 7fa11bf52640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa11410c660 con 0x7fa114101bf0 2024-04-02T13:53:18.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.993+0000 7fa11bf52640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa11410c4e0 con 0x7fa1141025b0 2024-04-02T13:53:18.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.993+0000 7fa11bf52640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa11410c7e0 con 0x7fa114100a00 2024-04-02T13:53:18.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.994+0000 7fa1194ed640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa114101bf0 0x7fa11419cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:18.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.994+0000 7fa1194ed640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa114101bf0 0x7fa11419cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40900/0 (socket says 172.21.15.52:40900) 2024-04-02T13:53:18.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.994+0000 7fa1194ed640 1 -- 172.21.15.52:0/509621653 learned_addr learned my addr 172.21.15.52:0/509621653 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:18.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.994+0000 7fa119cee640 1 --2- 172.21.15.52:0/509621653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa114100a00 0x7fa1140697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:18.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.994+0000 7fa11a4ef640 1 --2- 172.21.15.52:0/509621653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1141025b0 0x7fa11419f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:19.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.994+0000 7fa1194ed640 1 -- 172.21.15.52:0/509621653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa114100a00 msgr2=0x7fa1140697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:19.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.994+0000 7fa1194ed640 1 --2- 172.21.15.52:0/509621653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa114100a00 0x7fa1140697d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.994+0000 7fa1194ed640 1 -- 172.21.15.52:0/509621653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1141025b0 msgr2=0x7fa11419f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:19.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.994+0000 7fa1194ed640 1 --2- 172.21.15.52:0/509621653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1141025b0 0x7fa11419f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.994+0000 7fa1194ed640 1 -- 172.21.15.52:0/509621653 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa11419fa50 con 0x7fa114101bf0 2024-04-02T13:53:19.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.994+0000 7fa11a4ef640 1 --2- 172.21.15.52:0/509621653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1141025b0 0x7fa11419f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:53:19.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.994+0000 7fa1194ed640 1 --2- 172.21.15.52:0/509621653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa114101bf0 0x7fa11419cb20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa104009230 tx=0x7fa104004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:19.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.994+0000 7fa10affd640 1 -- 172.21.15.52:0/509621653 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104041070 con 0x7fa114101bf0 2024-04-02T13:53:19.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.994+0000 7fa10affd640 1 -- 172.21.15.52:0/509621653 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1040054e0 con 0x7fa114101bf0 2024-04-02T13:53:19.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.994+0000 7fa11bf52640 1 -- 172.21.15.52:0/509621653 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa11419fc80 con 0x7fa114101bf0 2024-04-02T13:53:19.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.994+0000 7fa10affd640 1 -- 172.21.15.52:0/509621653 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1040346a0 con 0x7fa114101bf0 2024-04-02T13:53:19.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.994+0000 7fa11bf52640 1 -- 172.21.15.52:0/509621653 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1141a0170 con 0x7fa114101bf0 2024-04-02T13:53:19.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.995+0000 7fa11bf52640 1 -- 172.21.15.52:0/509621653 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa114107da0 con 0x7fa114101bf0 2024-04-02T13:53:19.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.997+0000 7fa10affd640 1 -- 172.21.15.52:0/509621653 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa104044070 con 0x7fa114101bf0 2024-04-02T13:53:19.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.997+0000 7fa10affd640 1 --2- 172.21.15.52:0/509621653 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa1000a3bb0 0x7fa1000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:19.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.997+0000 7fa119cee640 1 --2- 172.21.15.52:0/509621653 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa1000a3bb0 0x7fa1000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:19.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.997+0000 7fa10affd640 1 -- 172.21.15.52:0/509621653 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fa1040f2420 con 0x7fa114101bf0 2024-04-02T13:53:19.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:18.998+0000 7fa119cee640 1 --2- 172.21.15.52:0/509621653 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa1000a3bb0 0x7fa1000a6070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa114101a50 tx=0x7fa11000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:19.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.002+0000 7fa10affd640 1 -- 172.21.15.52:0/509621653 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa1040beaa0 con 0x7fa114101bf0 2024-04-02T13:53:19.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:18 smithi189 ceph-mon[36683]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-04-02T13:53:19.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:18 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/570545073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:53:19.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:18 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3376118958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:53:19.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:18 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3174382691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:53:19.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.186+0000 7fa11bf52640 1 -- 172.21.15.52:0/509621653 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa1140627a0 con 0x7fa114101bf0 2024-04-02T13:53:19.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.187+0000 7fa10affd640 1 -- 172.21.15.52:0/509621653 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7fa1040be3a0 con 0x7fa114101bf0 2024-04-02T13:53:19.190 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-04-02T13:53:19.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.188+0000 7fa11bf52640 1 -- 172.21.15.52:0/509621653 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa1000a3bb0 msgr2=0x7fa1000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:19.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.188+0000 7fa11bf52640 1 --2- 172.21.15.52:0/509621653 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa1000a3bb0 0x7fa1000a6070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa114101a50 tx=0x7fa11000a2d0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.188+0000 7fa11bf52640 1 -- 172.21.15.52:0/509621653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa114101bf0 msgr2=0x7fa11419cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:19.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.188+0000 7fa11bf52640 1 --2- 172.21.15.52:0/509621653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa114101bf0 0x7fa11419cb20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa104009230 tx=0x7fa104004040 comp rx=0 tx=0).stop 2024-04-02T13:53:19.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.188+0000 7fa11bf52640 1 -- 172.21.15.52:0/509621653 shutdown_connections 2024-04-02T13:53:19.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.188+0000 7fa11bf52640 1 --2- 172.21.15.52:0/509621653 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fa1000a3bb0 0x7fa1000a6070 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.188+0000 7fa11bf52640 1 --2- 172.21.15.52:0/509621653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1141025b0 0x7fa11419f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.189+0000 7fa11bf52640 1 --2- 172.21.15.52:0/509621653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa114101bf0 0x7fa11419cb20 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.189+0000 7fa11bf52640 1 --2- 172.21.15.52:0/509621653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa114100a00 0x7fa1140697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.189+0000 7fa11bf52640 1 -- 172.21.15.52:0/509621653 >> 172.21.15.52:0/509621653 conn(0x7fa1140fbf90 msgr2=0x7fa114104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:19.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.189+0000 7fa11bf52640 1 -- 172.21.15.52:0/509621653 shutdown_connections 2024-04-02T13:53:19.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.189+0000 7fa11bf52640 1 -- 172.21.15.52:0/509621653 wait complete. 2024-04-02T13:53:19.203 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:53:19.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.305+0000 7ffbc75db640 1 -- 172.21.15.52:0/814917614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbc01025a0 msgr2=0x7ffbc0104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:19.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.305+0000 7ffbc75db640 1 --2- 172.21.15.52:0/814917614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbc01025a0 0x7ffbc0104990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc009900 tx=0x7ffbbc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:19.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.306+0000 7ffbc75db640 1 -- 172.21.15.52:0/814917614 shutdown_connections 2024-04-02T13:53:19.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.306+0000 7ffbc75db640 1 --2- 172.21.15.52:0/814917614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbc0107800 0x7ffbc0109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.306+0000 7ffbc75db640 1 --2- 172.21.15.52:0/814917614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffbc0104ed0 0x7ffbc01072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.306+0000 7ffbc75db640 1 --2- 172.21.15.52:0/814917614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbc01025a0 0x7ffbc0104990 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.306+0000 7ffbc75db640 1 -- 172.21.15.52:0/814917614 >> 172.21.15.52:0/814917614 conn(0x7ffbc00fbfb0 msgr2=0x7ffbc00fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:19.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.306+0000 7ffbc75db640 1 -- 172.21.15.52:0/814917614 shutdown_connections 2024-04-02T13:53:19.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.306+0000 7ffbc75db640 1 -- 172.21.15.52:0/814917614 wait complete. 2024-04-02T13:53:19.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.306+0000 7ffbc75db640 1 Processor -- start 2024-04-02T13:53:19.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.306+0000 7ffbc75db640 1 -- start start 2024-04-02T13:53:19.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.306+0000 7ffbc75db640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffbc01025a0 0x7ffbc0197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:19.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.306+0000 7ffbc75db640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbc0104ed0 0x7ffbc01983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:19.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.306+0000 7ffbc75db640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbc0107800 0x7ffbc019f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:19.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.306+0000 7ffbc75db640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc010d7a0 con 0x7ffbc0107800 2024-04-02T13:53:19.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.306+0000 7ffbc75db640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc010d620 con 0x7ffbc01025a0 2024-04-02T13:53:19.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.306+0000 7ffbc75db640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc010d920 con 0x7ffbc0104ed0 2024-04-02T13:53:19.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.307+0000 7ffbc5b78640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbc0107800 0x7ffbc019f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:19.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.307+0000 7ffbc5b78640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbc0107800 0x7ffbc019f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40918/0 (socket says 172.21.15.52:40918) 2024-04-02T13:53:19.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.307+0000 7ffbc5b78640 1 -- 172.21.15.52:0/650613500 learned_addr learned my addr 172.21.15.52:0/650613500 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:19.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.307+0000 7ffbc4b76640 1 --2- 172.21.15.52:0/650613500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbc0104ed0 0x7ffbc01983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:19.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.307+0000 7ffbc5377640 1 --2- 172.21.15.52:0/650613500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffbc01025a0 0x7ffbc0197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:19.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.307+0000 7ffbc5b78640 1 -- 172.21.15.52:0/650613500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbc0104ed0 msgr2=0x7ffbc01983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:19.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.307+0000 7ffbc5b78640 1 --2- 172.21.15.52:0/650613500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbc0104ed0 0x7ffbc01983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.307+0000 7ffbc5b78640 1 -- 172.21.15.52:0/650613500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffbc01025a0 msgr2=0x7ffbc0197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:19.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.307+0000 7ffbc5b78640 1 --2- 172.21.15.52:0/650613500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffbc01025a0 0x7ffbc0197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.307+0000 7ffbc5b78640 1 -- 172.21.15.52:0/650613500 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc019fad0 con 0x7ffbc0107800 2024-04-02T13:53:19.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.307+0000 7ffbc5377640 1 --2- 172.21.15.52:0/650613500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffbc01025a0 0x7ffbc0197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:19.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.307+0000 7ffbc5b78640 1 --2- 172.21.15.52:0/650613500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbc0107800 0x7ffbc019f460 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8009fd0 tx=0x7ffbb800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:19.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.307+0000 7ffbb67fc640 1 -- 172.21.15.52:0/650613500 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8017070 con 0x7ffbc0107800 2024-04-02T13:53:19.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.307+0000 7ffbb67fc640 1 -- 172.21.15.52:0/650613500 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbb8003e50 con 0x7ffbc0107800 2024-04-02T13:53:19.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.307+0000 7ffbc75db640 1 -- 172.21.15.52:0/650613500 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc019fdc0 con 0x7ffbc0107800 2024-04-02T13:53:19.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.307+0000 7ffbb67fc640 1 -- 172.21.15.52:0/650613500 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb80106c0 con 0x7ffbc0107800 2024-04-02T13:53:19.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.307+0000 7ffbc75db640 1 -- 172.21.15.52:0/650613500 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc01a0280 con 0x7ffbc0107800 2024-04-02T13:53:19.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.308+0000 7ffbc75db640 1 -- 172.21.15.52:0/650613500 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbc00fdaa0 con 0x7ffbc0107800 2024-04-02T13:53:19.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.309+0000 7ffbb67fc640 1 -- 172.21.15.52:0/650613500 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ffbb8003970 con 0x7ffbc0107800 2024-04-02T13:53:19.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.310+0000 7ffbb67fc640 1 --2- 172.21.15.52:0/650613500 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ffbac0a3c30 0x7ffbac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:19.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.310+0000 7ffbc5377640 1 --2- 172.21.15.52:0/650613500 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ffbac0a3c30 0x7ffbac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:19.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.310+0000 7ffbb67fc640 1 -- 172.21.15.52:0/650613500 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ffbb80d0590 con 0x7ffbc0107800 2024-04-02T13:53:19.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.310+0000 7ffbc5377640 1 --2- 172.21.15.52:0/650613500 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ffbac0a3c30 0x7ffbac0a60f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc006010 tx=0x7ffbbc02c8a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:19.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.315+0000 7ffbb67fc640 1 -- 172.21.15.52:0/650613500 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ffbb809cbb0 con 0x7ffbc0107800 2024-04-02T13:53:19.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.504+0000 7ffbc75db640 1 -- 172.21.15.52:0/650613500 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffbc00627a0 con 0x7ffbc0107800 2024-04-02T13:53:19.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.505+0000 7ffbb67fc640 1 -- 172.21.15.52:0/650613500 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7ffbb809c4b0 con 0x7ffbc0107800 2024-04-02T13:53:19.508 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-04-02T13:53:19.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.507+0000 7ffbc75db640 1 -- 172.21.15.52:0/650613500 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ffbac0a3c30 msgr2=0x7ffbac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:19.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.507+0000 7ffbc75db640 1 --2- 172.21.15.52:0/650613500 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ffbac0a3c30 0x7ffbac0a60f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc006010 tx=0x7ffbbc02c8a0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.507+0000 7ffbc75db640 1 -- 172.21.15.52:0/650613500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbc0107800 msgr2=0x7ffbc019f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:19.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.507+0000 7ffbc75db640 1 --2- 172.21.15.52:0/650613500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbc0107800 0x7ffbc019f460 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8009fd0 tx=0x7ffbb800b260 comp rx=0 tx=0).stop 2024-04-02T13:53:19.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.507+0000 7ffbc75db640 1 -- 172.21.15.52:0/650613500 shutdown_connections 2024-04-02T13:53:19.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.507+0000 7ffbc75db640 1 --2- 172.21.15.52:0/650613500 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7ffbac0a3c30 0x7ffbac0a60f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.507+0000 7ffbc75db640 1 --2- 172.21.15.52:0/650613500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbc0107800 0x7ffbc019f460 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.507+0000 7ffbc75db640 1 --2- 172.21.15.52:0/650613500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbc0104ed0 0x7ffbc01983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.507+0000 7ffbc75db640 1 --2- 172.21.15.52:0/650613500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffbc01025a0 0x7ffbc0197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.507+0000 7ffbc75db640 1 -- 172.21.15.52:0/650613500 >> 172.21.15.52:0/650613500 conn(0x7ffbc00fbfb0 msgr2=0x7ffbc0102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:19.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.507+0000 7ffbc75db640 1 -- 172.21.15.52:0/650613500 shutdown_connections 2024-04-02T13:53:19.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.507+0000 7ffbc75db640 1 -- 172.21.15.52:0/650613500 wait complete. 2024-04-02T13:53:19.521 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:53:19.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.624+0000 7f46a1088640 1 -- 172.21.15.52:0/1945330781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f469c100a10 msgr2=0x7f469c100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:19.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.624+0000 7f46a1088640 1 --2- 172.21.15.52:0/1945330781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f469c100a10 0x7f469c100e30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f468c009900 tx=0x7f468c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:19.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.624+0000 7f46a1088640 1 -- 172.21.15.52:0/1945330781 shutdown_connections 2024-04-02T13:53:19.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.624+0000 7f46a1088640 1 --2- 172.21.15.52:0/1945330781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f469c1025c0 0x7f469c108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.624+0000 7f46a1088640 1 --2- 172.21.15.52:0/1945330781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f469c101c00 0x7f469c102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.624+0000 7f46a1088640 1 --2- 172.21.15.52:0/1945330781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f469c100a10 0x7f469c100e30 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.624+0000 7f46a1088640 1 -- 172.21.15.52:0/1945330781 >> 172.21.15.52:0/1945330781 conn(0x7f469c0fbfa0 msgr2=0x7f469c0fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:19.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.624+0000 7f46a1088640 1 -- 172.21.15.52:0/1945330781 shutdown_connections 2024-04-02T13:53:19.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.624+0000 7f46a1088640 1 -- 172.21.15.52:0/1945330781 wait complete. 2024-04-02T13:53:19.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.625+0000 7f46a1088640 1 Processor -- start 2024-04-02T13:53:19.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.625+0000 7f46a1088640 1 -- start start 2024-04-02T13:53:19.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.625+0000 7f46a1088640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f469c100a10 0x7f469c196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:19.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.625+0000 7f46a1088640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f469c101c00 0x7f469c197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:19.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.625+0000 7f46a1088640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f469c1025c0 0x7f469c19e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:19.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.625+0000 7f46a1088640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f469c10c660 con 0x7f469c1025c0 2024-04-02T13:53:19.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.625+0000 7f46a1088640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f469c10c4e0 con 0x7f469c101c00 2024-04-02T13:53:19.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.625+0000 7f46a1088640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f469c10c7e0 con 0x7f469c100a10 2024-04-02T13:53:19.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.625+0000 7f469b59e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f469c1025c0 0x7f469c19e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:19.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.625+0000 7f469b59e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f469c1025c0 0x7f469c19e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40942/0 (socket says 172.21.15.52:40942) 2024-04-02T13:53:19.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.625+0000 7f469b59e640 1 -- 172.21.15.52:0/567329474 learned_addr learned my addr 172.21.15.52:0/567329474 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:19.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.625+0000 7f469ad9d640 1 --2- 172.21.15.52:0/567329474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f469c100a10 0x7f469c196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:19.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.625+0000 7f4693fff640 1 --2- 172.21.15.52:0/567329474 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f469c101c00 0x7f469c197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:19.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.626+0000 7f469b59e640 1 -- 172.21.15.52:0/567329474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f469c100a10 msgr2=0x7f469c196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:19.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.626+0000 7f469b59e640 1 --2- 172.21.15.52:0/567329474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f469c100a10 0x7f469c196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.626+0000 7f469b59e640 1 -- 172.21.15.52:0/567329474 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f469c101c00 msgr2=0x7f469c197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:19.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.626+0000 7f469b59e640 1 --2- 172.21.15.52:0/567329474 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f469c101c00 0x7f469c197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.626+0000 7f469b59e640 1 -- 172.21.15.52:0/567329474 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469c19e9d0 con 0x7f469c1025c0 2024-04-02T13:53:19.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.626+0000 7f469b59e640 1 --2- 172.21.15.52:0/567329474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f469c1025c0 0x7f469c19e330 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f469400cad0 tx=0x7f469400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:19.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.626+0000 7f4698d99640 1 -- 172.21.15.52:0/567329474 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469400ae20 con 0x7f469c1025c0 2024-04-02T13:53:19.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.626+0000 7f46a1088640 1 -- 172.21.15.52:0/567329474 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f469c19ecc0 con 0x7f469c1025c0 2024-04-02T13:53:19.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.626+0000 7f4698d99640 1 -- 172.21.15.52:0/567329474 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4694005020 con 0x7f469c1025c0 2024-04-02T13:53:19.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.626+0000 7f4698d99640 1 -- 172.21.15.52:0/567329474 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469400f6f0 con 0x7f469c1025c0 2024-04-02T13:53:19.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.626+0000 7f46a1088640 1 -- 172.21.15.52:0/567329474 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f469c19f190 con 0x7f469c1025c0 2024-04-02T13:53:19.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.627+0000 7f46a1088640 1 -- 172.21.15.52:0/567329474 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f469c107e50 con 0x7f469c1025c0 2024-04-02T13:53:19.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.629+0000 7f4698d99640 1 -- 172.21.15.52:0/567329474 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f46940051d0 con 0x7f469c1025c0 2024-04-02T13:53:19.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.629+0000 7f4698d99640 1 --2- 172.21.15.52:0/567329474 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f46880a3bb0 0x7f46880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:19.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.630+0000 7f4698d99640 1 -- 172.21.15.52:0/567329474 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f46940d0bd0 con 0x7f469c1025c0 2024-04-02T13:53:19.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.630+0000 7f469ad9d640 1 --2- 172.21.15.52:0/567329474 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f46880a3bb0 0x7f46880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:19.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.630+0000 7f469ad9d640 1 --2- 172.21.15.52:0/567329474 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f46880a3bb0 0x7f46880a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f468c000c00 tx=0x7f468c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:19.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.633+0000 7f4698d99640 1 -- 172.21.15.52:0/567329474 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4694014030 con 0x7f469c1025c0 2024-04-02T13:53:19.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.819+0000 7f46a1088640 1 -- 172.21.15.52:0/567329474 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f469c02d5c0 con 0x7f469c1025c0 2024-04-02T13:53:19.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.820+0000 7f4698d99640 1 -- 172.21.15.52:0/567329474 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7f469409d250 con 0x7f469c1025c0 2024-04-02T13:53:19.823 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-04-02T13:53:19.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.822+0000 7f46a1088640 1 -- 172.21.15.52:0/567329474 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f46880a3bb0 msgr2=0x7f46880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:19.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.822+0000 7f46a1088640 1 --2- 172.21.15.52:0/567329474 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f46880a3bb0 0x7f46880a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f468c000c00 tx=0x7f468c005fd0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.822+0000 7f46a1088640 1 -- 172.21.15.52:0/567329474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f469c1025c0 msgr2=0x7f469c19e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:19.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.822+0000 7f46a1088640 1 --2- 172.21.15.52:0/567329474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f469c1025c0 0x7f469c19e330 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f469400cad0 tx=0x7f469400cf90 comp rx=0 tx=0).stop 2024-04-02T13:53:19.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.822+0000 7f46a1088640 1 -- 172.21.15.52:0/567329474 shutdown_connections 2024-04-02T13:53:19.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.822+0000 7f46a1088640 1 --2- 172.21.15.52:0/567329474 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f46880a3bb0 0x7f46880a6070 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.822+0000 7f46a1088640 1 --2- 172.21.15.52:0/567329474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f469c1025c0 0x7f469c19e330 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.822+0000 7f46a1088640 1 --2- 172.21.15.52:0/567329474 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f469c101c00 0x7f469c197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.822+0000 7f46a1088640 1 --2- 172.21.15.52:0/567329474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f469c100a10 0x7f469c196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.822+0000 7f46a1088640 1 -- 172.21.15.52:0/567329474 >> 172.21.15.52:0/567329474 conn(0x7f469c0fbfa0 msgr2=0x7f469c105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:19.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.822+0000 7f46a1088640 1 -- 172.21.15.52:0/567329474 shutdown_connections 2024-04-02T13:53:19.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.822+0000 7f46a1088640 1 -- 172.21.15.52:0/567329474 wait complete. 2024-04-02T13:53:19.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:19 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/377906626' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:53:19.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:19 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1241626752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:53:19.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:19 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/509621653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:53:19.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:19 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/650613500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:53:19.837 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:53:19.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:19 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/377906626' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:53:19.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:19 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1241626752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:53:19.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:19 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/509621653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:53:19.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:19 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/650613500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:53:19.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.936+0000 7f6ebb02c640 1 -- 172.21.15.52:0/1223570528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb4107920 msgr2=0x7f6eb4109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:19.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.936+0000 7f6ebb02c640 1 --2- 172.21.15.52:0/1223570528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb4107920 0x7f6eb4109d10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009900 tx=0x7f6e9c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:19.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.936+0000 7f6ebb02c640 1 -- 172.21.15.52:0/1223570528 shutdown_connections 2024-04-02T13:53:19.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.936+0000 7f6ebb02c640 1 --2- 172.21.15.52:0/1223570528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb4107920 0x7f6eb4109d10 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.936+0000 7f6ebb02c640 1 --2- 172.21.15.52:0/1223570528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6eb4104ff0 0x7f6eb41073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.936+0000 7f6ebb02c640 1 --2- 172.21.15.52:0/1223570528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6eb41026c0 0x7f6eb4104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.936+0000 7f6ebb02c640 1 -- 172.21.15.52:0/1223570528 >> 172.21.15.52:0/1223570528 conn(0x7f6eb40fc030 msgr2=0x7f6eb40fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:19.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.936+0000 7f6ebb02c640 1 -- 172.21.15.52:0/1223570528 shutdown_connections 2024-04-02T13:53:19.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.936+0000 7f6ebb02c640 1 -- 172.21.15.52:0/1223570528 wait complete. 2024-04-02T13:53:19.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.937+0000 7f6ebb02c640 1 Processor -- start 2024-04-02T13:53:19.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.937+0000 7f6ebb02c640 1 -- start start 2024-04-02T13:53:19.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.937+0000 7f6ebb02c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6eb41026c0 0x7f6eb4197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:19.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.937+0000 7f6ebb02c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6eb4104ff0 0x7f6eb4198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:19.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.937+0000 7f6ebb02c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb4107920 0x7f6eb419f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:19.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.937+0000 7f6ebb02c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb410d7b0 con 0x7f6eb4107920 2024-04-02T13:53:19.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.937+0000 7f6ebb02c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb410d630 con 0x7f6eb4104ff0 2024-04-02T13:53:19.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.937+0000 7f6ebb02c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb410d930 con 0x7f6eb41026c0 2024-04-02T13:53:19.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.937+0000 7f6eb95c9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb4107920 0x7f6eb419f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:19.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.937+0000 7f6eb95c9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb4107920 0x7f6eb419f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40958/0 (socket says 172.21.15.52:40958) 2024-04-02T13:53:19.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.937+0000 7f6eb95c9640 1 -- 172.21.15.52:0/3658683131 learned_addr learned my addr 172.21.15.52:0/3658683131 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:19.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.937+0000 7f6eb3fff640 1 --2- 172.21.15.52:0/3658683131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6eb4104ff0 0x7f6eb4198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:19.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.937+0000 7f6eb8dc8640 1 --2- 172.21.15.52:0/3658683131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6eb41026c0 0x7f6eb4197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:19.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.938+0000 7f6eb95c9640 1 -- 172.21.15.52:0/3658683131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6eb41026c0 msgr2=0x7f6eb4197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:19.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.938+0000 7f6eb95c9640 1 --2- 172.21.15.52:0/3658683131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6eb41026c0 0x7f6eb4197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.938+0000 7f6eb95c9640 1 -- 172.21.15.52:0/3658683131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6eb4104ff0 msgr2=0x7f6eb4198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:19.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.938+0000 7f6eb95c9640 1 --2- 172.21.15.52:0/3658683131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6eb4104ff0 0x7f6eb4198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:19.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.938+0000 7f6eb95c9640 1 -- 172.21.15.52:0/3658683131 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb419f9d0 con 0x7f6eb4107920 2024-04-02T13:53:19.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.938+0000 7f6eb95c9640 1 --2- 172.21.15.52:0/3658683131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb4107920 0x7f6eb419f360 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c02c7b0 tx=0x7f6e9c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:19.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.938+0000 7f6eb1ffb640 1 -- 172.21.15.52:0/3658683131 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c041070 con 0x7f6eb4107920 2024-04-02T13:53:19.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.938+0000 7f6eb1ffb640 1 -- 172.21.15.52:0/3658683131 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e9c0054e0 con 0x7f6eb4107920 2024-04-02T13:53:19.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.938+0000 7f6ebb02c640 1 -- 172.21.15.52:0/3658683131 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb419fc60 con 0x7f6eb4107920 2024-04-02T13:53:19.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.938+0000 7f6eb1ffb640 1 -- 172.21.15.52:0/3658683131 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c034440 con 0x7f6eb4107920 2024-04-02T13:53:19.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.938+0000 7f6ebb02c640 1 -- 172.21.15.52:0/3658683131 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb41a0150 con 0x7f6eb4107920 2024-04-02T13:53:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.939+0000 7f6ebb02c640 1 -- 172.21.15.52:0/3658683131 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eb40fdb20 con 0x7f6eb4107920 2024-04-02T13:53:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.940+0000 7f6eb1ffb640 1 -- 172.21.15.52:0/3658683131 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6e9c044070 con 0x7f6eb4107920 2024-04-02T13:53:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.941+0000 7f6eb1ffb640 1 --2- 172.21.15.52:0/3658683131 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f6ea00a3bb0 0x7f6ea00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.941+0000 7f6eb8dc8640 1 --2- 172.21.15.52:0/3658683131 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f6ea00a3bb0 0x7f6ea00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.941+0000 7f6eb1ffb640 1 -- 172.21.15.52:0/3658683131 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f6e9c0f2730 con 0x7f6eb4107920 2024-04-02T13:53:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.941+0000 7f6eb8dc8640 1 --2- 172.21.15.52:0/3658683131 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f6ea00a3bb0 0x7f6ea00a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6eac005fd0 tx=0x7f6eac009760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:19.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:19.944+0000 7f6eb1ffb640 1 -- 172.21.15.52:0/3658683131 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6e9c0be700 con 0x7f6eb4107920 2024-04-02T13:53:20.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:19 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/377906626' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:53:20.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:19 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1241626752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:53:20.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:19 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/509621653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:53:20.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:19 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/650613500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:53:20.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.169+0000 7f6ebb02c640 1 -- 172.21.15.52:0/3658683131 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6eb402d370 con 0x7f6eb4107920 2024-04-02T13:53:20.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.169+0000 7f6eb1ffb640 1 -- 172.21.15.52:0/3658683131 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f6e9c0c2a00 con 0x7f6eb4107920 2024-04-02T13:53:20.174 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:53:20.175 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:51:29.345060+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24503,24475,24490,24467,24457]},"id":1}]} 2024-04-02T13:53:20.175 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 27 2024-04-02T13:53:20.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.171+0000 7f6ebb02c640 1 -- 172.21.15.52:0/3658683131 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f6ea00a3bb0 msgr2=0x7f6ea00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.171+0000 7f6ebb02c640 1 --2- 172.21.15.52:0/3658683131 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f6ea00a3bb0 0x7f6ea00a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6eac005fd0 tx=0x7f6eac009760 comp rx=0 tx=0).stop 2024-04-02T13:53:20.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.171+0000 7f6ebb02c640 1 -- 172.21.15.52:0/3658683131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb4107920 msgr2=0x7f6eb419f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.171+0000 7f6ebb02c640 1 --2- 172.21.15.52:0/3658683131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb4107920 0x7f6eb419f360 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c02c7b0 tx=0x7f6e9c005dd0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.171+0000 7f6ebb02c640 1 -- 172.21.15.52:0/3658683131 shutdown_connections 2024-04-02T13:53:20.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.171+0000 7f6ebb02c640 1 --2- 172.21.15.52:0/3658683131 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f6ea00a3bb0 0x7f6ea00a6070 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.171+0000 7f6ebb02c640 1 --2- 172.21.15.52:0/3658683131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6eb4107920 0x7f6eb419f360 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.171+0000 7f6ebb02c640 1 --2- 172.21.15.52:0/3658683131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6eb4104ff0 0x7f6eb4198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.171+0000 7f6ebb02c640 1 --2- 172.21.15.52:0/3658683131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6eb41026c0 0x7f6eb4197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.171+0000 7f6ebb02c640 1 -- 172.21.15.52:0/3658683131 >> 172.21.15.52:0/3658683131 conn(0x7f6eb40fc030 msgr2=0x7f6eb4103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.171+0000 7f6ebb02c640 1 -- 172.21.15.52:0/3658683131 shutdown_connections 2024-04-02T13:53:20.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.172+0000 7f6ebb02c640 1 -- 172.21.15.52:0/3658683131 wait complete. 2024-04-02T13:53:20.186 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:53:20.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.285+0000 7fdb00a41640 1 -- 172.21.15.52:0/727559927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdafc104ed0 msgr2=0x7fdafc1072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.285+0000 7fdb00a41640 1 --2- 172.21.15.52:0/727559927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdafc104ed0 0x7fdafc1072c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009900 tx=0x7fdaf002bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:20.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.285+0000 7fdb00a41640 1 -- 172.21.15.52:0/727559927 shutdown_connections 2024-04-02T13:53:20.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.285+0000 7fdb00a41640 1 --2- 172.21.15.52:0/727559927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdafc107800 0x7fdafc109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.285+0000 7fdb00a41640 1 --2- 172.21.15.52:0/727559927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdafc104ed0 0x7fdafc1072c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.285+0000 7fdb00a41640 1 --2- 172.21.15.52:0/727559927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdafc1025a0 0x7fdafc104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.285+0000 7fdb00a41640 1 -- 172.21.15.52:0/727559927 >> 172.21.15.52:0/727559927 conn(0x7fdafc0fbf90 msgr2=0x7fdafc0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.285+0000 7fdb00a41640 1 -- 172.21.15.52:0/727559927 shutdown_connections 2024-04-02T13:53:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.286+0000 7fdb00a41640 1 -- 172.21.15.52:0/727559927 wait complete. 2024-04-02T13:53:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.286+0000 7fdb00a41640 1 Processor -- start 2024-04-02T13:53:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.286+0000 7fdb00a41640 1 -- start start 2024-04-02T13:53:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.286+0000 7fdb00a41640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdafc1025a0 0x7fdafc197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.286+0000 7fdb00a41640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdafc104ed0 0x7fdafc198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.286+0000 7fdb00a41640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdafc107800 0x7fdafc19f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.286+0000 7fdb00a41640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc10d790 con 0x7fdafc104ed0 2024-04-02T13:53:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.286+0000 7fdb00a41640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc10d610 con 0x7fdafc107800 2024-04-02T13:53:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.286+0000 7fdb00a41640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc10d910 con 0x7fdafc1025a0 2024-04-02T13:53:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.286+0000 7fdaf9d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdafc104ed0 0x7fdafc198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.287+0000 7fdaf9d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdafc104ed0 0x7fdafc198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40976/0 (socket says 172.21.15.52:40976) 2024-04-02T13:53:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.287+0000 7fdaf9d9b640 1 -- 172.21.15.52:0/1600672706 learned_addr learned my addr 172.21.15.52:0/1600672706 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.287+0000 7fdafa59c640 1 --2- 172.21.15.52:0/1600672706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdafc1025a0 0x7fdafc197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.287+0000 7fdafad9d640 1 --2- 172.21.15.52:0/1600672706 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdafc107800 0x7fdafc19f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.287+0000 7fdaf9d9b640 1 -- 172.21.15.52:0/1600672706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdafc1025a0 msgr2=0x7fdafc197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.287+0000 7fdaf9d9b640 1 --2- 172.21.15.52:0/1600672706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdafc1025a0 0x7fdafc197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.287+0000 7fdaf9d9b640 1 -- 172.21.15.52:0/1600672706 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdafc107800 msgr2=0x7fdafc19f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.287+0000 7fdaf9d9b640 1 --2- 172.21.15.52:0/1600672706 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdafc107800 0x7fdafc19f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.287+0000 7fdaf9d9b640 1 -- 172.21.15.52:0/1600672706 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdafc19fa00 con 0x7fdafc104ed0 2024-04-02T13:53:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.287+0000 7fdafad9d640 1 --2- 172.21.15.52:0/1600672706 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdafc107800 0x7fdafc19f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:53:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.287+0000 7fdafa59c640 1 --2- 172.21.15.52:0/1600672706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdafc1025a0 0x7fdafc197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.287+0000 7fdaf9d9b640 1 --2- 172.21.15.52:0/1600672706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdafc104ed0 0x7fdafc198320 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0005ab0 tx=0x7fdaf0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.287+0000 7fdaeb7fe640 1 -- 172.21.15.52:0/1600672706 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0041070 con 0x7fdafc104ed0 2024-04-02T13:53:20.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.287+0000 7fdaeb7fe640 1 -- 172.21.15.52:0/1600672706 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaf00054e0 con 0x7fdafc104ed0 2024-04-02T13:53:20.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.287+0000 7fdb00a41640 1 -- 172.21.15.52:0/1600672706 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdafc19fc90 con 0x7fdafc104ed0 2024-04-02T13:53:20.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.287+0000 7fdaeb7fe640 1 -- 172.21.15.52:0/1600672706 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0034440 con 0x7fdafc104ed0 2024-04-02T13:53:20.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.287+0000 7fdb00a41640 1 -- 172.21.15.52:0/1600672706 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdafc1a01d0 con 0x7fdafc104ed0 2024-04-02T13:53:20.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.288+0000 7fdb00a41640 1 -- 172.21.15.52:0/1600672706 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdafc0fda80 con 0x7fdafc104ed0 2024-04-02T13:53:20.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.289+0000 7fdaeb7fe640 1 -- 172.21.15.52:0/1600672706 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdaf0044070 con 0x7fdafc104ed0 2024-04-02T13:53:20.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.290+0000 7fdaeb7fe640 1 --2- 172.21.15.52:0/1600672706 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdae40a3c30 0x7fdae40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.290+0000 7fdafa59c640 1 --2- 172.21.15.52:0/1600672706 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdae40a3c30 0x7fdae40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.290+0000 7fdaeb7fe640 1 -- 172.21.15.52:0/1600672706 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdaf00f2390 con 0x7fdafc104ed0 2024-04-02T13:53:20.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.290+0000 7fdafa59c640 1 --2- 172.21.15.52:0/1600672706 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdae40a3c30 0x7fdae40a60f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400abe0 tx=0x7fdaf400a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.294+0000 7fdaeb7fe640 1 -- 172.21.15.52:0/1600672706 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fdaf00bea10 con 0x7fdafc104ed0 2024-04-02T13:53:20.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.486+0000 7fdb00a41640 1 -- 172.21.15.52:0/1600672706 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdafc0627a0 con 0x7fdafc104ed0 2024-04-02T13:53:20.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.487+0000 7fdaeb7fe640 1 -- 172.21.15.52:0/1600672706 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24549 (secure 0 0 0) 0x7fdaf00be310 con 0x7fdafc104ed0 2024-04-02T13:53:20.491 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:53:20.491 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":92,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T13:51:56.472064+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:50:58.778690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:50:59.561527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:51:03.343487+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:53:20.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.490+0000 7fdb00a41640 1 -- 172.21.15.52:0/1600672706 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdae40a3c30 msgr2=0x7fdae40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.490+0000 7fdb00a41640 1 --2- 172.21.15.52:0/1600672706 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdae40a3c30 0x7fdae40a60f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400abe0 tx=0x7fdaf400a5c0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.490+0000 7fdb00a41640 1 -- 172.21.15.52:0/1600672706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdafc104ed0 msgr2=0x7fdafc198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.490+0000 7fdb00a41640 1 --2- 172.21.15.52:0/1600672706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdafc104ed0 0x7fdafc198320 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0005ab0 tx=0x7fdaf0005dc0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.490+0000 7fdb00a41640 1 -- 172.21.15.52:0/1600672706 shutdown_connections 2024-04-02T13:53:20.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.490+0000 7fdb00a41640 1 --2- 172.21.15.52:0/1600672706 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fdae40a3c30 0x7fdae40a60f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.490+0000 7fdb00a41640 1 --2- 172.21.15.52:0/1600672706 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdafc107800 0x7fdafc19f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.490+0000 7fdb00a41640 1 --2- 172.21.15.52:0/1600672706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdafc104ed0 0x7fdafc198320 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.490+0000 7fdb00a41640 1 --2- 172.21.15.52:0/1600672706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdafc1025a0 0x7fdafc197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.490+0000 7fdb00a41640 1 -- 172.21.15.52:0/1600672706 >> 172.21.15.52:0/1600672706 conn(0x7fdafc0fbf90 msgr2=0x7fdafc102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.490+0000 7fdb00a41640 1 -- 172.21.15.52:0/1600672706 shutdown_connections 2024-04-02T13:53:20.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.490+0000 7fdb00a41640 1 -- 172.21.15.52:0/1600672706 wait complete. 2024-04-02T13:53:20.505 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-02T13:51:00.603246+0000', 'modified': '2024-04-02T13:51:29.345060+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24503, 'mds_1': 24490, 'mds_2': 24475, 'mds_3': 24467, 'mds_4': 24457}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14604': {'gid': 14604, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.52:6839/2213112521', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6838', 'nonce': 2213112521}, {'type': 'v1', 'addr': '172.21.15.52:6839', 'nonce': 2213112521}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14616': {'gid': 14616, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.52:6841/2689882080', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6840', 'nonce': 2689882080}, {'type': 'v1', 'addr': '172.21.15.52:6841', 'nonce': 2689882080}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24410': {'gid': 24410, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.88:6833/4248342942', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6832', 'nonce': 4248342942}, {'type': 'v1', 'addr': '172.21.15.88:6833', 'nonce': 4248342942}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6833/3157031038', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6832', 'nonce': 3157031038}, {'type': 'v1', 'addr': '172.21.15.189:6833', 'nonce': 3157031038}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24437': {'gid': 24437, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.88:6835/5019107', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6834', 'nonce': 5019107}, {'type': 'v1', 'addr': '172.21.15.88:6835', 'nonce': 5019107}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.189:6835/1647960400', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6834', 'nonce': 1647960400}, {'type': 'v1', 'addr': '172.21.15.189:6835', 'nonce': 1647960400}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.88:6837/1091039373', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6836', 'nonce': 1091039373}, {'type': 'v1', 'addr': '172.21.15.88:6837', 'nonce': 1091039373}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.189:6837/2295541152', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6836', 'nonce': 2295541152}, {'type': 'v1', 'addr': '172.21.15.189:6837', 'nonce': 2295541152}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.189:6839/729001189', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6838', 'nonce': 729001189}, {'type': 'v1', 'addr': '172.21.15.189:6839', 'nonce': 729001189}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24503': {'gid': 24503, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.88:6839/2513527493', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6838', 'nonce': 2513527493}, {'type': 'v1', 'addr': '172.21.15.88:6839', 'nonce': 2513527493}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24503, 'qdb_cluster': [24503, 24475, 24490, 24467, 24457]} 2024-04-02T13:53:20.505 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-02T13:53:20.505 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-02T13:53:20.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.607+0000 7f64c5767640 1 -- 172.21.15.52:0/4038456568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64c01077f0 msgr2=0x7f64c0109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.607+0000 7f64c5767640 1 --2- 172.21.15.52:0/4038456568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64c01077f0 0x7f64c0109c40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009ca0 tx=0x7f64b802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.607+0000 7f64c5767640 1 -- 172.21.15.52:0/4038456568 shutdown_connections 2024-04-02T13:53:20.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.607+0000 7f64c5767640 1 --2- 172.21.15.52:0/4038456568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64c01077f0 0x7f64c0109c40 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.607+0000 7f64c5767640 1 --2- 172.21.15.52:0/4038456568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64c0104ec0 0x7f64c01072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.607+0000 7f64c5767640 1 --2- 172.21.15.52:0/4038456568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64c0102590 0x7f64c0104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.607+0000 7f64c5767640 1 -- 172.21.15.52:0/4038456568 >> 172.21.15.52:0/4038456568 conn(0x7f64c00fbfa0 msgr2=0x7f64c00fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.607+0000 7f64c5767640 1 -- 172.21.15.52:0/4038456568 shutdown_connections 2024-04-02T13:53:20.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.607+0000 7f64c5767640 1 -- 172.21.15.52:0/4038456568 wait complete. 2024-04-02T13:53:20.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.608+0000 7f64c5767640 1 Processor -- start 2024-04-02T13:53:20.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.608+0000 7f64c5767640 1 -- start start 2024-04-02T13:53:20.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.608+0000 7f64c5767640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64c0102590 0x7f64c0197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.608+0000 7f64c5767640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64c0104ec0 0x7f64c0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.608+0000 7f64c5767640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64c01077f0 0x7f64c019f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.608+0000 7f64c5767640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f64c010d7b0 con 0x7f64c01077f0 2024-04-02T13:53:20.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.608+0000 7f64c5767640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f64c010d630 con 0x7f64c0104ec0 2024-04-02T13:53:20.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.608+0000 7f64c5767640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f64c010d930 con 0x7f64c0102590 2024-04-02T13:53:20.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.608+0000 7f64bf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64c01077f0 0x7f64c019f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.608+0000 7f64bf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64c01077f0 0x7f64c019f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40998/0 (socket says 172.21.15.52:40998) 2024-04-02T13:53:20.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.608+0000 7f64bf7fe640 1 -- 172.21.15.52:0/4037810890 learned_addr learned my addr 172.21.15.52:0/4037810890 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:20.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.608+0000 7f64b7fff640 1 --2- 172.21.15.52:0/4037810890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64c0104ec0 0x7f64c0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.608+0000 7f64beffd640 1 --2- 172.21.15.52:0/4037810890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64c0102590 0x7f64c0197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.608+0000 7f64bf7fe640 1 -- 172.21.15.52:0/4037810890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64c0102590 msgr2=0x7f64c0197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.608+0000 7f64bf7fe640 1 --2- 172.21.15.52:0/4037810890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64c0102590 0x7f64c0197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.608+0000 7f64bf7fe640 1 -- 172.21.15.52:0/4037810890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64c0104ec0 msgr2=0x7f64c0198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.608+0000 7f64bf7fe640 1 --2- 172.21.15.52:0/4037810890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64c0104ec0 0x7f64c0198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.608+0000 7f64bf7fe640 1 -- 172.21.15.52:0/4037810890 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c019fa10 con 0x7f64c01077f0 2024-04-02T13:53:20.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.608+0000 7f64beffd640 1 --2- 172.21.15.52:0/4037810890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64c0102590 0x7f64c0197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:53:20.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.608+0000 7f64b7fff640 1 --2- 172.21.15.52:0/4037810890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64c0104ec0 0x7f64c0198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:20.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.609+0000 7f64bf7fe640 1 --2- 172.21.15.52:0/4037810890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64c01077f0 0x7f64c019f3a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f64b8037040 tx=0x7f64b8032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.609+0000 7f64bcff9640 1 -- 172.21.15.52:0/4037810890 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b8041020 con 0x7f64c01077f0 2024-04-02T13:53:20.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.609+0000 7f64bcff9640 1 -- 172.21.15.52:0/4037810890 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64b8034d90 con 0x7f64c01077f0 2024-04-02T13:53:20.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.609+0000 7f64c5767640 1 -- 172.21.15.52:0/4037810890 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64c019fca0 con 0x7f64c01077f0 2024-04-02T13:53:20.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.609+0000 7f64bcff9640 1 -- 172.21.15.52:0/4037810890 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b8039040 con 0x7f64c01077f0 2024-04-02T13:53:20.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.609+0000 7f64c5767640 1 -- 172.21.15.52:0/4037810890 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64c01a0190 con 0x7f64c01077f0 2024-04-02T13:53:20.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.611+0000 7f64bcff9640 1 -- 172.21.15.52:0/4037810890 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f64b8044020 con 0x7f64c01077f0 2024-04-02T13:53:20.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.611+0000 7f64c842d740 1 Processor -- start 2024-04-02T13:53:20.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.611+0000 7f64c842d740 1 -- start start 2024-04-02T13:53:20.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.612+0000 7f64c842d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559022eceae0 0x559022ecef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.612+0000 7f64c842d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022ecf4d0 0x559022ed3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.612+0000 7f64c842d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559022ed4640 0x559022ed6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.612+0000 7f64c842d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559022e33060 con 0x559022ecf4d0 2024-04-02T13:53:20.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.612+0000 7f64c842d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559022ddb650 con 0x559022ed4640 2024-04-02T13:53:20.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.612+0000 7f64c842d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559022e2f010 con 0x559022eceae0 2024-04-02T13:53:20.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.612+0000 7f64bcff9640 1 --2- 172.21.15.52:0/4037810890 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f64ac0a3bb0 0x7f64ac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.612+0000 7f64beffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022ecf4d0 0x559022ed3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.612+0000 7f64beffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022ecf4d0 0x559022ed3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41012/0 (socket says 172.21.15.52:41012) 2024-04-02T13:53:20.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.612+0000 7f64beffd640 1 -- 172.21.15.52:0/1916126393 learned_addr learned my addr 172.21.15.52:0/1916126393 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:20.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.612+0000 7f64b7fff640 1 --2- 172.21.15.52:0/1916126393 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559022ed4640 0x559022ed6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.612+0000 7f64bf7fe640 1 --2- 172.21.15.52:0/4037810890 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f64ac0a3bb0 0x7f64ac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.612+0000 7f64bcff9640 1 -- 172.21.15.52:0/4037810890 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f64b80f2730 con 0x7f64c01077f0 2024-04-02T13:53:20.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.612+0000 7f64b7fff640 1 --2- 172.21.15.52:0/1916126393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559022eceae0 0x559022ecef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.612+0000 7f64beffd640 1 -- 172.21.15.52:0/1916126393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559022eceae0 msgr2=0x559022ecef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.612+0000 7f64beffd640 1 --2- 172.21.15.52:0/1916126393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559022eceae0 0x559022ecef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.612+0000 7f64beffd640 1 -- 172.21.15.52:0/1916126393 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559022ed4640 msgr2=0x559022ed6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.612+0000 7f64beffd640 1 --2- 172.21.15.52:0/1916126393 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559022ed4640 0x559022ed6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.612+0000 7f64beffd640 1 -- 172.21.15.52:0/1916126393 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559022b472d0 con 0x559022ecf4d0 2024-04-02T13:53:20.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.612+0000 7f64beffd640 1 --2- 172.21.15.52:0/1916126393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022ecf4d0 0x559022ed3fd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f64b000bb70 tx=0x7f64b000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2545dfec87c96965 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.613+0000 7f64bf7fe640 1 --2- 172.21.15.52:0/4037810890 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f64ac0a3bb0 0x7f64ac0a6070 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f64b80eff20 tx=0x7f64b802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.613+0000 7f64b57fa640 1 -- 172.21.15.52:0/1916126393 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0017620 con 0x559022ecf4d0 2024-04-02T13:53:20.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.613+0000 7f64b57fa640 1 -- 172.21.15.52:0/1916126393 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64b0017ca0 con 0x559022ecf4d0 2024-04-02T13:53:20.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.613+0000 7f64b57fa640 1 -- 172.21.15.52:0/1916126393 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b000fc40 con 0x559022ecf4d0 2024-04-02T13:53:20.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.613+0000 7f64c842d740 1 -- 172.21.15.52:0/1916126393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022ecf4d0 msgr2=0x559022ed3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.613+0000 7f64c842d740 1 --2- 172.21.15.52:0/1916126393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022ecf4d0 0x559022ed3fd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f64b000bb70 tx=0x7f64b000d710 comp rx=0 tx=0).stop 2024-04-02T13:53:20.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.613+0000 7f64c842d740 1 -- 172.21.15.52:0/1916126393 shutdown_connections 2024-04-02T13:53:20.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.613+0000 7f64c842d740 1 --2- 172.21.15.52:0/1916126393 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559022ed4640 0x559022ed6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.613+0000 7f64c842d740 1 --2- 172.21.15.52:0/1916126393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022ecf4d0 0x559022ed3fd0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.613+0000 7f64c842d740 1 --2- 172.21.15.52:0/1916126393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559022eceae0 0x559022ecef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.613+0000 7f64c842d740 1 -- 172.21.15.52:0/1916126393 >> 172.21.15.52:0/1916126393 conn(0x559022eca050 msgr2=0x559022eca450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.613+0000 7f64c842d740 1 -- 172.21.15.52:0/1916126393 shutdown_connections 2024-04-02T13:53:20.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.614+0000 7f64c842d740 1 -- 172.21.15.52:0/1916126393 wait complete. 2024-04-02T13:53:20.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.614+0000 7f64c842d740 1 -- start start 2024-04-02T13:53:20.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.614+0000 7f64c842d740 1 Processor -- start 2024-04-02T13:53:20.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.614+0000 7f64c842d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022eceae0 0x559022ed9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.614+0000 7f64c842d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559022ecf4d0 0x559022ed9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.614+0000 7f64c842d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559022ed4640 0x559022ee0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.614+0000 7f64c842d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559022ed75f0 con 0x559022eceae0 2024-04-02T13:53:20.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.614+0000 7f64c842d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559022deea10 con 0x559022ed4640 2024-04-02T13:53:20.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.614+0000 7f64c842d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559022ed7770 con 0x559022ecf4d0 2024-04-02T13:53:20.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.614+0000 7f64c842d740 10 client.0 _ll_register_callbacks cb 0x559022e2b0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:20.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.614+0000 7f64b7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022eceae0 0x559022ed9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.614+0000 7f64b7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022eceae0 0x559022ed9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41028/0 (socket says 172.21.15.52:41028) 2024-04-02T13:53:20.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.614+0000 7f64b7fff640 1 -- 172.21.15.52:0/2580811066 learned_addr learned my addr 172.21.15.52:0/2580811066 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:20.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.614+0000 7f64beffd640 1 --2- 172.21.15.52:0/2580811066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559022ecf4d0 0x559022ed9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.614+0000 7f64b7fff640 1 -- 172.21.15.52:0/2580811066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559022ecf4d0 msgr2=0x559022ed9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.614+0000 7f64b7fff640 1 --2- 172.21.15.52:0/2580811066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559022ecf4d0 0x559022ed9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.614+0000 7f64b7fff640 1 -- 172.21.15.52:0/2580811066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559022ed4640 msgr2=0x559022ee0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.614+0000 7f64b7fff640 1 --2- 172.21.15.52:0/2580811066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559022ed4640 0x559022ee0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.614+0000 7f64b7fff640 1 -- 172.21.15.52:0/2580811066 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559022c116b0 con 0x559022eceae0 2024-04-02T13:53:20.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.615+0000 7f64b7fff640 1 --2- 172.21.15.52:0/2580811066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022eceae0 0x559022ed9000 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f64a8013cd0 tx=0x7f64a800e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afbaaf5053afa037 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.615+0000 7f64a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:53:20.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.615+0000 7f64a67fc640 1 -- 172.21.15.52:0/2580811066 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a80130d0 con 0x559022eceae0 2024-04-02T13:53:20.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.615+0000 7f64a67fc640 1 -- 172.21.15.52:0/2580811066 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f64a8013270 con 0x559022eceae0 2024-04-02T13:53:20.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.615+0000 7f64c842d740 1 -- 172.21.15.52:0/2580811066 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64b0017ca0 con 0x559022eceae0 2024-04-02T13:53:20.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.615+0000 7f64a67fc640 1 -- 172.21.15.52:0/2580811066 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f64a8028050 con 0x559022eceae0 2024-04-02T13:53:20.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.615+0000 7f64c842d740 10 client.15420 fetch_fsmap learned FSMap version 27 2024-04-02T13:53:20.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.615+0000 7f64c842d740 1 -- 172.21.15.52:0/2580811066 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559022ee3990 con 0x559022eceae0 2024-04-02T13:53:20.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.615+0000 7f64a67fc640 1 -- 172.21.15.52:0/2580811066 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64b0017ca0 con 0x559022eceae0 2024-04-02T13:53:20.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.615+0000 7f64a67fc640 1 -- 172.21.15.52:0/2580811066 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f64a8003c50 con 0x559022eceae0 2024-04-02T13:53:20.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.615+0000 7f64c842d740 10 client.15420 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:53:20.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.615+0000 7f64c842d740 10 client.15420 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-02T13:53:20.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.615+0000 7f64c842d740 20 client.15420 populate_metadata read hostname 'smithi052' 2024-04-02T13:53:20.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.615+0000 7f64c842d740 1 --2- 172.21.15.52:0/2580811066 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559022ee60d0 0x559022ee7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.615+0000 7f64c842d740 4 client.15420 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:53:20.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.615+0000 7f64c842d740 1 -- 172.21.15.52:0/2580811066 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559022ddacf0 con 0x559022ee60d0 2024-04-02T13:53:20.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.616+0000 7f64beffd640 1 --2- 172.21.15.52:0/2580811066 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559022ee60d0 0x559022ee7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.616+0000 7f64beffd640 1 --2- 172.21.15.52:0/2580811066 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559022ee60d0 0x559022ee7540 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=66df963da684980b server_cookie=93e47b18ab0b81d3 in_seq=0 out_seq=0 2024-04-02T13:53:20.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.616+0000 7f64a67fc640 10 client.15420 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:53:20.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.617+0000 7f64a67fc640 1 -- 172.21.15.52:0/2580811066 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559022ddacf0 con 0x559022ee60d0 2024-04-02T13:53:20.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.617+0000 7f64a67fc640 10 client.15420 handle_command_reply: tid=0 2024-04-02T13:53:20.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.617+0000 7f64c842d740 1 client.15420 shutdown 2024-04-02T13:53:20.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.617+0000 7f64c842d740 1 -- 172.21.15.52:0/2580811066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022eceae0 msgr2=0x559022ed9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.617+0000 7f64c842d740 1 --2- 172.21.15.52:0/2580811066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022eceae0 0x559022ed9000 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f64a8013cd0 tx=0x7f64a800e830 comp rx=0 tx=0).stop 2024-04-02T13:53:20.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.617+0000 7f64c842d740 1 -- 172.21.15.52:0/2580811066 shutdown_connections 2024-04-02T13:53:20.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.617+0000 7f64c842d740 1 --2- 172.21.15.52:0/2580811066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559022ed4640 0x559022ee0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.617+0000 7f64c842d740 1 --2- 172.21.15.52:0/2580811066 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559022ee60d0 0x559022ee7540 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.617+0000 7f64c842d740 1 --2- 172.21.15.52:0/2580811066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559022ecf4d0 0x559022ed9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.617+0000 7f64c842d740 1 --2- 172.21.15.52:0/2580811066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022eceae0 0x559022ed9000 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.617+0000 7f64a67fc640 0 client.15420 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:53:20.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.617+0000 7f64c842d740 1 -- 172.21.15.52:0/2580811066 >> 172.21.15.52:0/2580811066 conn(0x559022eca050 msgr2=0x559022ec9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.617+0000 7f64c842d740 1 -- 172.21.15.52:0/2580811066 shutdown_connections 2024-04-02T13:53:20.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.617+0000 7f64c842d740 1 -- 172.21.15.52:0/2580811066 wait complete. 2024-04-02T13:53:20.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.617+0000 7f64c842d740 20 client.15420 trim_cache size 0 max 16384 2024-04-02T13:53:20.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.635+0000 7f64c842d740 1 Processor -- start 2024-04-02T13:53:20.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.635+0000 7f64c842d740 1 -- start start 2024-04-02T13:53:20.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.636+0000 7f64c842d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559022eceae0 0x559022ed6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.636+0000 7f64c842d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022ecf4d0 0x559022ed9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.636+0000 7f64c842d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559022ed4640 0x559022ec8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.636+0000 7f64c842d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559022ee5880 con 0x559022ecf4d0 2024-04-02T13:53:20.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.636+0000 7f64c842d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559022ee5f50 con 0x559022eceae0 2024-04-02T13:53:20.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.636+0000 7f64c842d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559022ee6620 con 0x559022ed4640 2024-04-02T13:53:20.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.636+0000 7f64beffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022ecf4d0 0x559022ed9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.636+0000 7f64beffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022ecf4d0 0x559022ed9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41044/0 (socket says 172.21.15.52:41044) 2024-04-02T13:53:20.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.636+0000 7f64beffd640 1 -- 172.21.15.52:0/1107683301 learned_addr learned my addr 172.21.15.52:0/1107683301 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:20.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.636+0000 7f64b7fff640 1 --2- 172.21.15.52:0/1107683301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559022ed4640 0x559022ec8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.636+0000 7f64beffd640 1 -- 172.21.15.52:0/1107683301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559022ed4640 msgr2=0x559022ec8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.636+0000 7f64beffd640 1 --2- 172.21.15.52:0/1107683301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559022ed4640 0x559022ec8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.636+0000 7f64beffd640 1 -- 172.21.15.52:0/1107683301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559022eceae0 msgr2=0x559022ed6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.636+0000 7f64beffd640 1 --2- 172.21.15.52:0/1107683301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559022eceae0 0x559022ed6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.636+0000 7f64beffd640 1 -- 172.21.15.52:0/1107683301 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559022ed7930 con 0x559022ecf4d0 2024-04-02T13:53:20.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.636+0000 7f64b7fff640 1 --2- 172.21.15.52:0/1107683301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559022ed4640 0x559022ec8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:53:20.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.636+0000 7f64beffd640 1 --2- 172.21.15.52:0/1107683301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022ecf4d0 0x559022ed9320 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f64b00058c0 tx=0x7f64b000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a64d2d89913c71 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.636+0000 7f64b4ff9640 1 -- 172.21.15.52:0/1107683301 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b002e070 con 0x559022ecf4d0 2024-04-02T13:53:20.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.636+0000 7f64b4ff9640 1 -- 172.21.15.52:0/1107683301 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64b0026070 con 0x559022ecf4d0 2024-04-02T13:53:20.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.636+0000 7f64b4ff9640 1 -- 172.21.15.52:0/1107683301 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0017680 con 0x559022ecf4d0 2024-04-02T13:53:20.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.637+0000 7f64c842d740 1 -- 172.21.15.52:0/1107683301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022ecf4d0 msgr2=0x559022ed9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.637+0000 7f64c842d740 1 --2- 172.21.15.52:0/1107683301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022ecf4d0 0x559022ed9320 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f64b00058c0 tx=0x7f64b000b4a0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.637+0000 7f64c842d740 1 -- 172.21.15.52:0/1107683301 shutdown_connections 2024-04-02T13:53:20.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.637+0000 7f64c842d740 1 --2- 172.21.15.52:0/1107683301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559022ed4640 0x559022ec8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.637+0000 7f64c842d740 1 --2- 172.21.15.52:0/1107683301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022ecf4d0 0x559022ed9320 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.637+0000 7f64c842d740 1 --2- 172.21.15.52:0/1107683301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559022eceae0 0x559022ed6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.637+0000 7f64c842d740 1 -- 172.21.15.52:0/1107683301 >> 172.21.15.52:0/1107683301 conn(0x559022eca050 msgr2=0x559022ed8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.637+0000 7f64c842d740 1 -- 172.21.15.52:0/1107683301 shutdown_connections 2024-04-02T13:53:20.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.637+0000 7f64c842d740 1 -- 172.21.15.52:0/1107683301 wait complete. 2024-04-02T13:53:20.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.637+0000 7f64c842d740 1 -- start start 2024-04-02T13:53:20.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.637+0000 7f64c842d740 1 Processor -- start 2024-04-02T13:53:20.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.637+0000 7f64c842d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559022eceae0 0x559022f482b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.638+0000 7f64c842d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022ecf4d0 0x559022f48870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.638+0000 7f64c842d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559022ed4640 0x559022f4f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.638+0000 7f64c842d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559022ee73c0 con 0x559022ecf4d0 2024-04-02T13:53:20.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.638+0000 7f64c842d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559022ee6cf0 con 0x559022ed4640 2024-04-02T13:53:20.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.638+0000 7f64c842d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559022ee7a90 con 0x559022eceae0 2024-04-02T13:53:20.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.638+0000 7f64c842d740 10 client.0 _ll_register_callbacks cb 0x7f64a800e3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:20.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.638+0000 7f64beffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022ecf4d0 0x559022f48870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.638+0000 7f64beffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022ecf4d0 0x559022f48870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41060/0 (socket says 172.21.15.52:41060) 2024-04-02T13:53:20.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.638+0000 7f64beffd640 1 -- 172.21.15.52:0/1640970351 learned_addr learned my addr 172.21.15.52:0/1640970351 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:20.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.638+0000 7f64b7fff640 1 --2- 172.21.15.52:0/1640970351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559022ed4640 0x559022f4f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.638+0000 7f64b7fff640 1 --2- 172.21.15.52:0/1640970351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559022eceae0 0x559022f482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.638+0000 7f64beffd640 1 -- 172.21.15.52:0/1640970351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559022eceae0 msgr2=0x559022f482b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.638+0000 7f64beffd640 1 --2- 172.21.15.52:0/1640970351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559022eceae0 0x559022f482b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.638+0000 7f64beffd640 1 -- 172.21.15.52:0/1640970351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559022ed4640 msgr2=0x559022f4f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.638+0000 7f64beffd640 1 --2- 172.21.15.52:0/1640970351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559022ed4640 0x559022f4f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.638+0000 7f64beffd640 1 -- 172.21.15.52:0/1640970351 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559022f4fbe0 con 0x559022ecf4d0 2024-04-02T13:53:20.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.638+0000 7f64beffd640 1 --2- 172.21.15.52:0/1640970351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022ecf4d0 0x559022f48870 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f64b0019040 tx=0x7f64b000dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe305c99b992a5f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.638+0000 7f64a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:53:20.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.638+0000 7f64a67fc640 1 -- 172.21.15.52:0/1640970351 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b002e040 con 0x559022ecf4d0 2024-04-02T13:53:20.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.638+0000 7f64a67fc640 1 -- 172.21.15.52:0/1640970351 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f64b000f710 con 0x559022ecf4d0 2024-04-02T13:53:20.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.638+0000 7f64c842d740 1 -- 172.21.15.52:0/1640970351 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64b0026070 con 0x559022ecf4d0 2024-04-02T13:53:20.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.638+0000 7f64a67fc640 1 -- 172.21.15.52:0/1640970351 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f64b000f930 con 0x559022ecf4d0 2024-04-02T13:53:20.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.638+0000 7f64c842d740 10 client.15426 fetch_fsmap learned FSMap version 27 2024-04-02T13:53:20.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.638+0000 7f64c842d740 1 -- 172.21.15.52:0/1640970351 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559022f50e50 con 0x559022ecf4d0 2024-04-02T13:53:20.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.639+0000 7f64a67fc640 1 -- 172.21.15.52:0/1640970351 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64b0026070 con 0x559022ecf4d0 2024-04-02T13:53:20.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.639+0000 7f64a67fc640 1 -- 172.21.15.52:0/1640970351 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f64b0017420 con 0x559022ecf4d0 2024-04-02T13:53:20.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.639+0000 7f64c842d740 10 client.15426 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:53:20.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.639+0000 7f64c842d740 10 client.15426 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-02T13:53:20.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.639+0000 7f64c842d740 20 client.15426 populate_metadata read hostname 'smithi052' 2024-04-02T13:53:20.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.639+0000 7f64c842d740 1 --2- 172.21.15.52:0/1640970351 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559022f55370 0x559022f54080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.639+0000 7f64c842d740 4 client.15426 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:53:20.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.639+0000 7f64c842d740 1 -- 172.21.15.52:0/1640970351 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "status"}) -- 0x559022e405a0 con 0x559022f55370 2024-04-02T13:53:20.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.639+0000 7f64b7fff640 1 --2- 172.21.15.52:0/1640970351 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559022f55370 0x559022f54080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.640+0000 7f64b7fff640 1 --2- 172.21.15.52:0/1640970351 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559022f55370 0x559022f54080 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ed3b124ccd7476bc server_cookie=bd2e5b17e8066c2e in_seq=0 out_seq=0 2024-04-02T13:53:20.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.640+0000 7f64a67fc640 10 client.15426 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:53:20.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.640+0000 7f64a67fc640 1 -- 172.21.15.52:0/1640970351 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559022e405a0 con 0x559022f55370 2024-04-02T13:53:20.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.640+0000 7f64a67fc640 10 client.15426 handle_command_reply: tid=0 2024-04-02T13:53:20.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.640+0000 7f64c842d740 1 client.15426 shutdown 2024-04-02T13:53:20.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.640+0000 7f64c842d740 1 -- 172.21.15.52:0/1640970351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022ecf4d0 msgr2=0x559022f48870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.640+0000 7f64c842d740 1 --2- 172.21.15.52:0/1640970351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022ecf4d0 0x559022f48870 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f64b0019040 tx=0x7f64b000dfd0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.640+0000 7f64c842d740 1 -- 172.21.15.52:0/1640970351 shutdown_connections 2024-04-02T13:53:20.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.640+0000 7f64c842d740 1 --2- 172.21.15.52:0/1640970351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559022ed4640 0x559022f4f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.640+0000 7f64c842d740 1 --2- 172.21.15.52:0/1640970351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559022ecf4d0 0x559022f48870 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.640+0000 7f64c842d740 1 --2- 172.21.15.52:0/1640970351 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559022f55370 0x559022f54080 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.640+0000 7f64c842d740 1 --2- 172.21.15.52:0/1640970351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559022eceae0 0x559022f482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.640+0000 7f64a67fc640 0 client.15426 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:53:20.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.640+0000 7f64c842d740 1 -- 172.21.15.52:0/1640970351 >> 172.21.15.52:0/1640970351 conn(0x559022eca050 msgr2=0x559022ec8900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.640+0000 7f64c842d740 1 -- 172.21.15.52:0/1640970351 shutdown_connections 2024-04-02T13:53:20.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.640+0000 7f64c842d740 1 -- 172.21.15.52:0/1640970351 wait complete. 2024-04-02T13:53:20.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.640+0000 7f64c842d740 20 client.15426 trim_cache size 0 max 16384 2024-04-02T13:53:20.651 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:53:20.651 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "bfa00e6e-f0f6-11ee-b647-cb9ed24678a4", 2024-04-02T13:53:20.651 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 4, 2024-04-02T13:53:20.651 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24457, 2024-04-02T13:53:20.651 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-04-02T13:53:20.651 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-04-02T13:53:20.651 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-04-02T13:53:20.651 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 112.299012163, 2024-04-02T13:53:20.651 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 27, 2024-04-02T13:53:20.652 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 92, 2024-04-02T13:53:20.652 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:53:20.652 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 162.43919999600001 2024-04-02T13:53:20.652 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:53:20.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.641+0000 7f64c5767640 1 -- 172.21.15.52:0/4037810890 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f64ac0a3bb0 msgr2=0x7f64ac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.641+0000 7f64c5767640 1 --2- 172.21.15.52:0/4037810890 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f64ac0a3bb0 0x7f64ac0a6070 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f64b80eff20 tx=0x7f64b802e040 comp rx=0 tx=0).stop 2024-04-02T13:53:20.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.641+0000 7f64c5767640 1 -- 172.21.15.52:0/4037810890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64c01077f0 msgr2=0x7f64c019f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.641+0000 7f64c5767640 1 --2- 172.21.15.52:0/4037810890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64c01077f0 0x7f64c019f3a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f64b8037040 tx=0x7f64b8032f20 comp rx=0 tx=0).stop 2024-04-02T13:53:20.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.641+0000 7f64c5767640 1 -- 172.21.15.52:0/4037810890 shutdown_connections 2024-04-02T13:53:20.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.641+0000 7f64c5767640 1 --2- 172.21.15.52:0/4037810890 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f64ac0a3bb0 0x7f64ac0a6070 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.641+0000 7f64c5767640 1 --2- 172.21.15.52:0/4037810890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64c01077f0 0x7f64c019f3a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.641+0000 7f64c5767640 1 --2- 172.21.15.52:0/4037810890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64c0104ec0 0x7f64c0198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.641+0000 7f64c5767640 1 --2- 172.21.15.52:0/4037810890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64c0102590 0x7f64c0197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.641+0000 7f64c5767640 1 -- 172.21.15.52:0/4037810890 >> 172.21.15.52:0/4037810890 conn(0x7f64c00fbfa0 msgr2=0x7f64c0102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.641+0000 7f64c5767640 1 -- 172.21.15.52:0/4037810890 shutdown_connections 2024-04-02T13:53:20.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.641+0000 7f64c5767640 1 -- 172.21.15.52:0/4037810890 wait complete. 2024-04-02T13:53:20.655 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-02T13:53:20.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.755+0000 7f3d57338640 1 -- 172.21.15.52:0/1215252019 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d50102580 msgr2=0x7f3d50104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.755+0000 7f3d57338640 1 --2- 172.21.15.52:0/1215252019 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d50102580 0x7f3d50104970 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3d48009900 tx=0x7f3d4802bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:20.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.755+0000 7f3d57338640 1 -- 172.21.15.52:0/1215252019 shutdown_connections 2024-04-02T13:53:20.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.755+0000 7f3d57338640 1 --2- 172.21.15.52:0/1215252019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d501077e0 0x7f3d50109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.755+0000 7f3d57338640 1 --2- 172.21.15.52:0/1215252019 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d50104eb0 0x7f3d501072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.755+0000 7f3d57338640 1 --2- 172.21.15.52:0/1215252019 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d50102580 0x7f3d50104970 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.755+0000 7f3d57338640 1 -- 172.21.15.52:0/1215252019 >> 172.21.15.52:0/1215252019 conn(0x7f3d500fbf90 msgr2=0x7f3d500fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.756+0000 7f3d57338640 1 -- 172.21.15.52:0/1215252019 shutdown_connections 2024-04-02T13:53:20.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.756+0000 7f3d57338640 1 -- 172.21.15.52:0/1215252019 wait complete. 2024-04-02T13:53:20.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.756+0000 7f3d57338640 1 Processor -- start 2024-04-02T13:53:20.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.756+0000 7f3d57338640 1 -- start start 2024-04-02T13:53:20.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.756+0000 7f3d57338640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d50102580 0x7f3d50197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.756+0000 7f3d57338640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d50104eb0 0x7f3d50198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.756+0000 7f3d57338640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d501077e0 0x7f3d5019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.756+0000 7f3d57338640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5010d7f0 con 0x7f3d50104eb0 2024-04-02T13:53:20.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.756+0000 7f3d57338640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5010d670 con 0x7f3d50102580 2024-04-02T13:53:20.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.756+0000 7f3d57338640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5010d970 con 0x7f3d501077e0 2024-04-02T13:53:20.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.756+0000 7f3d548d3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d50104eb0 0x7f3d50198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.756+0000 7f3d548d3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d50104eb0 0x7f3d50198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41088/0 (socket says 172.21.15.52:41088) 2024-04-02T13:53:20.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.756+0000 7f3d548d3640 1 -- 172.21.15.52:0/1211635498 learned_addr learned my addr 172.21.15.52:0/1211635498 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:20.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.757+0000 7f3d550d4640 1 --2- 172.21.15.52:0/1211635498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d50102580 0x7f3d50197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.757+0000 7f3d558d5640 1 --2- 172.21.15.52:0/1211635498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d501077e0 0x7f3d5019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.757+0000 7f3d548d3640 1 -- 172.21.15.52:0/1211635498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d501077e0 msgr2=0x7f3d5019f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.757+0000 7f3d548d3640 1 --2- 172.21.15.52:0/1211635498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d501077e0 0x7f3d5019f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.757+0000 7f3d548d3640 1 -- 172.21.15.52:0/1211635498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d50102580 msgr2=0x7f3d50197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.757+0000 7f3d548d3640 1 --2- 172.21.15.52:0/1211635498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d50102580 0x7f3d50197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.757+0000 7f3d548d3640 1 -- 172.21.15.52:0/1211635498 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5019f9f0 con 0x7f3d50104eb0 2024-04-02T13:53:20.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.757+0000 7f3d548d3640 1 --2- 172.21.15.52:0/1211635498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d50104eb0 0x7f3d50198310 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3d4400c970 tx=0x7f3d4400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.757+0000 7f3d4e7fc640 1 -- 172.21.15.52:0/1211635498 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d44014070 con 0x7f3d50104eb0 2024-04-02T13:53:20.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.757+0000 7f3d57338640 1 -- 172.21.15.52:0/1211635498 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d5019fce0 con 0x7f3d50104eb0 2024-04-02T13:53:20.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.757+0000 7f3d4e7fc640 1 -- 172.21.15.52:0/1211635498 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d44004440 con 0x7f3d50104eb0 2024-04-02T13:53:20.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.757+0000 7f3d4e7fc640 1 -- 172.21.15.52:0/1211635498 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4400f450 con 0x7f3d50104eb0 2024-04-02T13:53:20.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.757+0000 7f3d57338640 1 -- 172.21.15.52:0/1211635498 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d501a0220 con 0x7f3d50104eb0 2024-04-02T13:53:20.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.759+0000 7f3d4e7fc640 1 -- 172.21.15.52:0/1211635498 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3d44017070 con 0x7f3d50104eb0 2024-04-02T13:53:20.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.760+0000 7f3d59ff4740 1 Processor -- start 2024-04-02T13:53:20.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.760+0000 7f3d59ff4740 1 -- start start 2024-04-02T13:53:20.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.760+0000 7f3d59ff4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b91eb57ae0 0x55b91eb57f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.760+0000 7f3d59ff4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b91eb584d0 0x55b91eb5cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.760+0000 7f3d4e7fc640 1 --2- 172.21.15.52:0/1211635498 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3d380a3bb0 0x7f3d380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.760+0000 7f3d59ff4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb5d640 0x55b91eb5fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.760+0000 7f3d59ff4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b91eabc060 con 0x55b91eb5d640 2024-04-02T13:53:20.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.760+0000 7f3d59ff4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b91ea64650 con 0x55b91eb584d0 2024-04-02T13:53:20.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.760+0000 7f3d59ff4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b91eab8010 con 0x55b91eb57ae0 2024-04-02T13:53:20.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.760+0000 7f3d558d5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb5d640 0x55b91eb5fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.760+0000 7f3d548d3640 1 --2- 172.21.15.52:0/1211635498 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3d380a3bb0 0x7f3d380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.760+0000 7f3d558d5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb5d640 0x55b91eb5fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41094/0 (socket says 172.21.15.52:41094) 2024-04-02T13:53:20.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.760+0000 7f3d558d5640 1 -- 172.21.15.52:0/1228966336 learned_addr learned my addr 172.21.15.52:0/1228966336 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:20.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.760+0000 7f3d4e7fc640 1 -- 172.21.15.52:0/1211635498 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f3d440d0520 con 0x7f3d50104eb0 2024-04-02T13:53:20.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.760+0000 7f3d550d4640 1 --2- 172.21.15.52:0/1228966336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b91eb584d0 0x55b91eb5cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.761+0000 7f3d558d5640 1 --2- 172.21.15.52:0/1228966336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b91eb57ae0 0x55b91eb57f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.761+0000 7f3d558d5640 1 -- 172.21.15.52:0/1228966336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b91eb57ae0 msgr2=0x55b91eb57f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.761+0000 7f3d558d5640 1 --2- 172.21.15.52:0/1228966336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b91eb57ae0 0x55b91eb57f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.761+0000 7f3d558d5640 1 -- 172.21.15.52:0/1228966336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b91eb584d0 msgr2=0x55b91eb5cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.761+0000 7f3d558d5640 1 --2- 172.21.15.52:0/1228966336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b91eb584d0 0x55b91eb5cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.761+0000 7f3d558d5640 1 -- 172.21.15.52:0/1228966336 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b91e7d02d0 con 0x55b91eb5d640 2024-04-02T13:53:20.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.761+0000 7f3d548d3640 1 --2- 172.21.15.52:0/1211635498 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3d380a3bb0 0x7f3d380a6070 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3d44098e10 tx=0x7f3d44007f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.761+0000 7f3d558d5640 1 --2- 172.21.15.52:0/1228966336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb5d640 0x55b91eb5fad0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3d40005f40 tx=0x7f3d40009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c8cc1df481266e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.761+0000 7f3d377fe640 1 -- 172.21.15.52:0/1228966336 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d40014050 con 0x55b91eb5d640 2024-04-02T13:53:20.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.761+0000 7f3d377fe640 1 -- 172.21.15.52:0/1228966336 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d400141f0 con 0x55b91eb5d640 2024-04-02T13:53:20.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.761+0000 7f3d377fe640 1 -- 172.21.15.52:0/1228966336 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d40005280 con 0x55b91eb5d640 2024-04-02T13:53:20.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.762+0000 7f3d59ff4740 1 -- 172.21.15.52:0/1228966336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb5d640 msgr2=0x55b91eb5fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.762+0000 7f3d59ff4740 1 --2- 172.21.15.52:0/1228966336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb5d640 0x55b91eb5fad0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3d40005f40 tx=0x7f3d40009b80 comp rx=0 tx=0).stop 2024-04-02T13:53:20.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.762+0000 7f3d59ff4740 1 -- 172.21.15.52:0/1228966336 shutdown_connections 2024-04-02T13:53:20.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.762+0000 7f3d59ff4740 1 --2- 172.21.15.52:0/1228966336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb5d640 0x55b91eb5fad0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.762+0000 7f3d59ff4740 1 --2- 172.21.15.52:0/1228966336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b91eb584d0 0x55b91eb5cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.762+0000 7f3d59ff4740 1 --2- 172.21.15.52:0/1228966336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b91eb57ae0 0x55b91eb57f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.762+0000 7f3d59ff4740 1 -- 172.21.15.52:0/1228966336 >> 172.21.15.52:0/1228966336 conn(0x55b91eb53050 msgr2=0x55b91eb53450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.762+0000 7f3d59ff4740 1 -- 172.21.15.52:0/1228966336 shutdown_connections 2024-04-02T13:53:20.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.762+0000 7f3d59ff4740 1 -- 172.21.15.52:0/1228966336 wait complete. 2024-04-02T13:53:20.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.762+0000 7f3d59ff4740 1 -- start start 2024-04-02T13:53:20.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.762+0000 7f3d59ff4740 1 Processor -- start 2024-04-02T13:53:20.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.763+0000 7f3d59ff4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b91eb57ae0 0x55b91eb61f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.763+0000 7f3d59ff4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b91eb584d0 0x55b91eb624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.763+0000 7f3d59ff4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb5d640 0x55b91eb694f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.763+0000 7f3d59ff4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b91eb605f0 con 0x55b91eb5d640 2024-04-02T13:53:20.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.763+0000 7f3d59ff4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b91ea77a10 con 0x55b91eb57ae0 2024-04-02T13:53:20.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.763+0000 7f3d59ff4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b91eb60770 con 0x55b91eb584d0 2024-04-02T13:53:20.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.763+0000 7f3d59ff4740 10 client.0 _ll_register_callbacks cb 0x55b91eab40b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:20.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.763+0000 7f3d558d5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb5d640 0x55b91eb694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.763+0000 7f3d558d5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb5d640 0x55b91eb694f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41096/0 (socket says 172.21.15.52:41096) 2024-04-02T13:53:20.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.763+0000 7f3d558d5640 1 -- 172.21.15.52:0/1506792538 learned_addr learned my addr 172.21.15.52:0/1506792538 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:20.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.763+0000 7f3d558d5640 1 --2- 172.21.15.52:0/1506792538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b91eb57ae0 0x55b91eb61f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.763+0000 7f3d550d4640 1 --2- 172.21.15.52:0/1506792538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b91eb584d0 0x55b91eb624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.763+0000 7f3d558d5640 1 -- 172.21.15.52:0/1506792538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b91eb584d0 msgr2=0x55b91eb624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.763+0000 7f3d558d5640 1 --2- 172.21.15.52:0/1506792538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b91eb584d0 0x55b91eb624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.763+0000 7f3d558d5640 1 -- 172.21.15.52:0/1506792538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b91eb57ae0 msgr2=0x55b91eb61f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.763+0000 7f3d558d5640 1 --2- 172.21.15.52:0/1506792538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b91eb57ae0 0x55b91eb61f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.763+0000 7f3d558d5640 1 -- 172.21.15.52:0/1506792538 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b91e89a6b0 con 0x55b91eb5d640 2024-04-02T13:53:20.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.763+0000 7f3d558d5640 1 --2- 172.21.15.52:0/1506792538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb5d640 0x55b91eb694f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3d40005f10 tx=0x7f3d40005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2e4e293763ee2f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.763+0000 7f3d34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:53:20.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.763+0000 7f3d34ff9640 1 -- 172.21.15.52:0/1506792538 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d40004cd0 con 0x55b91eb5d640 2024-04-02T13:53:20.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.763+0000 7f3d34ff9640 1 -- 172.21.15.52:0/1506792538 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f3d40003970 con 0x55b91eb5d640 2024-04-02T13:53:20.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.763+0000 7f3d59ff4740 1 -- 172.21.15.52:0/1506792538 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d400141f0 con 0x55b91eb5d640 2024-04-02T13:53:20.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.763+0000 7f3d34ff9640 1 -- 172.21.15.52:0/1506792538 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d40033050 con 0x55b91eb5d640 2024-04-02T13:53:20.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.763+0000 7f3d59ff4740 10 client.15438 fetch_fsmap learned FSMap version 27 2024-04-02T13:53:20.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.763+0000 7f3d59ff4740 1 -- 172.21.15.52:0/1506792538 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b91eb6c860 con 0x55b91eb5d640 2024-04-02T13:53:20.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.764+0000 7f3d34ff9640 1 -- 172.21.15.52:0/1506792538 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d400141f0 con 0x55b91eb5d640 2024-04-02T13:53:20.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.764+0000 7f3d34ff9640 1 -- 172.21.15.52:0/1506792538 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3d4000f040 con 0x55b91eb5d640 2024-04-02T13:53:20.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.764+0000 7f3d59ff4740 10 client.15438 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:53:20.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.764+0000 7f3d59ff4740 10 client.15438 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-02T13:53:20.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.764+0000 7f3d59ff4740 20 client.15438 populate_metadata read hostname 'smithi052' 2024-04-02T13:53:20.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.764+0000 7f3d59ff4740 1 --2- 172.21.15.52:0/1506792538 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55b91eb6ead0 0x55b91eb6ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.764+0000 7f3d59ff4740 4 client.15438 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:53:20.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.764+0000 7f3d59ff4740 1 -- 172.21.15.52:0/1506792538 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b91ea63cf0 con 0x55b91eb6ead0 2024-04-02T13:53:20.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.764+0000 7f3d550d4640 1 --2- 172.21.15.52:0/1506792538 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55b91eb6ead0 0x55b91eb6ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.765+0000 7f3d550d4640 1 --2- 172.21.15.52:0/1506792538 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55b91eb6ead0 0x55b91eb6ff40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3b56d25e14bfeafc server_cookie=141a15fda8cb6a0c in_seq=0 out_seq=0 2024-04-02T13:53:20.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.765+0000 7f3d34ff9640 10 client.15438 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:53:20.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.765+0000 7f3d34ff9640 1 -- 172.21.15.52:0/1506792538 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b91ea63cf0 con 0x55b91eb6ead0 2024-04-02T13:53:20.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.765+0000 7f3d34ff9640 10 client.15438 handle_command_reply: tid=0 2024-04-02T13:53:20.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.765+0000 7f3d59ff4740 1 client.15438 shutdown 2024-04-02T13:53:20.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.765+0000 7f3d59ff4740 1 -- 172.21.15.52:0/1506792538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb5d640 msgr2=0x55b91eb694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.765+0000 7f3d59ff4740 1 --2- 172.21.15.52:0/1506792538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb5d640 0x55b91eb694f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3d40005f10 tx=0x7f3d40005bd0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.766+0000 7f3d59ff4740 1 -- 172.21.15.52:0/1506792538 shutdown_connections 2024-04-02T13:53:20.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.766+0000 7f3d59ff4740 1 --2- 172.21.15.52:0/1506792538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb5d640 0x55b91eb694f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.766+0000 7f3d59ff4740 1 --2- 172.21.15.52:0/1506792538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b91eb584d0 0x55b91eb624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.766+0000 7f3d59ff4740 1 --2- 172.21.15.52:0/1506792538 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55b91eb6ead0 0x55b91eb6ff40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.766+0000 7f3d59ff4740 1 --2- 172.21.15.52:0/1506792538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b91eb57ae0 0x55b91eb61f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.766+0000 7f3d34ff9640 0 client.15438 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:53:20.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.766+0000 7f3d59ff4740 1 -- 172.21.15.52:0/1506792538 >> 172.21.15.52:0/1506792538 conn(0x55b91eb53050 msgr2=0x55b91eb52190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.766+0000 7f3d59ff4740 1 -- 172.21.15.52:0/1506792538 shutdown_connections 2024-04-02T13:53:20.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.766+0000 7f3d59ff4740 1 -- 172.21.15.52:0/1506792538 wait complete. 2024-04-02T13:53:20.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.766+0000 7f3d59ff4740 20 client.15438 trim_cache size 0 max 16384 2024-04-02T13:53:20.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.784+0000 7f3d59ff4740 1 Processor -- start 2024-04-02T13:53:20.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.784+0000 7f3d59ff4740 1 -- start start 2024-04-02T13:53:20.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.784+0000 7f3d59ff4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b91eb57ae0 0x55b91eb53430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.784+0000 7f3d59ff4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b91eb584d0 0x55b91eb69410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.784+0000 7f3d59ff4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb5d640 0x55b91eb61b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.784+0000 7f3d59ff4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b91eb6e280 con 0x55b91eb5d640 2024-04-02T13:53:20.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.784+0000 7f3d59ff4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b91eb6e950 con 0x55b91eb584d0 2024-04-02T13:53:20.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.784+0000 7f3d59ff4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b91eb6f020 con 0x55b91eb57ae0 2024-04-02T13:53:20.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.785+0000 7f3d558d5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb5d640 0x55b91eb61b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.785+0000 7f3d558d5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb5d640 0x55b91eb61b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41106/0 (socket says 172.21.15.52:41106) 2024-04-02T13:53:20.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.785+0000 7f3d558d5640 1 -- 172.21.15.52:0/4058198384 learned_addr learned my addr 172.21.15.52:0/4058198384 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:20.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.785+0000 7f3d558d5640 1 -- 172.21.15.52:0/4058198384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b91eb57ae0 msgr2=0x55b91eb53430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.785+0000 7f3d558d5640 1 --2- 172.21.15.52:0/4058198384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b91eb57ae0 0x55b91eb53430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.785+0000 7f3d558d5640 1 -- 172.21.15.52:0/4058198384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b91eb584d0 msgr2=0x55b91eb69410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.785+0000 7f3d558d5640 1 --2- 172.21.15.52:0/4058198384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b91eb584d0 0x55b91eb69410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.785+0000 7f3d558d5640 1 -- 172.21.15.52:0/4058198384 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b91eb5cc20 con 0x55b91eb5d640 2024-04-02T13:53:20.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.785+0000 7f3d558d5640 1 --2- 172.21.15.52:0/4058198384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb5d640 0x55b91eb61b80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3d40009770 tx=0x7f3d400147f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847b9a9f0bdc6266 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.785+0000 7f3d36ffd640 1 -- 172.21.15.52:0/4058198384 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4003d070 con 0x55b91eb5d640 2024-04-02T13:53:20.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.785+0000 7f3d36ffd640 1 -- 172.21.15.52:0/4058198384 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d40024070 con 0x55b91eb5d640 2024-04-02T13:53:20.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.785+0000 7f3d36ffd640 1 -- 172.21.15.52:0/4058198384 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d40004dc0 con 0x55b91eb5d640 2024-04-02T13:53:20.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.786+0000 7f3d59ff4740 1 -- 172.21.15.52:0/4058198384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb5d640 msgr2=0x55b91eb61b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.786+0000 7f3d59ff4740 1 --2- 172.21.15.52:0/4058198384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb5d640 0x55b91eb61b80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3d40009770 tx=0x7f3d400147f0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.786+0000 7f3d59ff4740 1 -- 172.21.15.52:0/4058198384 shutdown_connections 2024-04-02T13:53:20.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.786+0000 7f3d59ff4740 1 --2- 172.21.15.52:0/4058198384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb5d640 0x55b91eb61b80 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.786+0000 7f3d59ff4740 1 --2- 172.21.15.52:0/4058198384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b91eb584d0 0x55b91eb69410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.786+0000 7f3d59ff4740 1 --2- 172.21.15.52:0/4058198384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b91eb57ae0 0x55b91eb53430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.786+0000 7f3d59ff4740 1 -- 172.21.15.52:0/4058198384 >> 172.21.15.52:0/4058198384 conn(0x55b91eb53050 msgr2=0x55b91eb4de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.786+0000 7f3d59ff4740 1 -- 172.21.15.52:0/4058198384 shutdown_connections 2024-04-02T13:53:20.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.786+0000 7f3d59ff4740 1 -- 172.21.15.52:0/4058198384 wait complete. 2024-04-02T13:53:20.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.786+0000 7f3d59ff4740 1 -- start start 2024-04-02T13:53:20.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.786+0000 7f3d59ff4740 1 Processor -- start 2024-04-02T13:53:20.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d59ff4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b91eb57ae0 0x55b91eb655b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d59ff4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb584d0 0x55b91eb65af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d59ff4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b91eb5d640 0x55b91eb68b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d59ff4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b91eb6fdc0 con 0x55b91eb584d0 2024-04-02T13:53:20.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d59ff4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b91eb6f6f0 con 0x55b91eb57ae0 2024-04-02T13:53:20.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d59ff4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b91eb70490 con 0x55b91eb5d640 2024-04-02T13:53:20.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d59ff4740 10 client.0 _ll_register_callbacks cb 0x7f3d4802c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:20.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d550d4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb584d0 0x55b91eb65af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d550d4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb584d0 0x55b91eb65af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41112/0 (socket says 172.21.15.52:41112) 2024-04-02T13:53:20.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d550d4640 1 -- 172.21.15.52:0/3068411829 learned_addr learned my addr 172.21.15.52:0/3068411829 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:20.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d558d5640 1 --2- 172.21.15.52:0/3068411829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b91eb57ae0 0x55b91eb655b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d558d5640 1 --2- 172.21.15.52:0/3068411829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b91eb5d640 0x55b91eb68b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d550d4640 1 -- 172.21.15.52:0/3068411829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b91eb5d640 msgr2=0x55b91eb68b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d550d4640 1 --2- 172.21.15.52:0/3068411829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b91eb5d640 0x55b91eb68b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d550d4640 1 -- 172.21.15.52:0/3068411829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b91eb57ae0 msgr2=0x55b91eb655b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d550d4640 1 --2- 172.21.15.52:0/3068411829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b91eb57ae0 0x55b91eb655b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d550d4640 1 -- 172.21.15.52:0/3068411829 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b91eb69100 con 0x55b91eb584d0 2024-04-02T13:53:20.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d558d5640 1 --2- 172.21.15.52:0/3068411829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b91eb5d640 0x55b91eb68b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:53:20.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d550d4640 1 --2- 172.21.15.52:0/3068411829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb584d0 0x55b91eb65af0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3d4802c8e0 tx=0x7f3d4802cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a40198ce39becee server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:53:20.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d34ff9640 1 -- 172.21.15.52:0/3068411829 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d48004250 con 0x55b91eb584d0 2024-04-02T13:53:20.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d34ff9640 1 -- 172.21.15.52:0/3068411829 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f3d480043f0 con 0x55b91eb584d0 2024-04-02T13:53:20.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d59ff4740 1 -- 172.21.15.52:0/3068411829 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d40024070 con 0x55b91eb584d0 2024-04-02T13:53:20.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d34ff9640 1 -- 172.21.15.52:0/3068411829 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d4805c050 con 0x55b91eb584d0 2024-04-02T13:53:20.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d59ff4740 10 client.15444 fetch_fsmap learned FSMap version 27 2024-04-02T13:53:20.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.787+0000 7f3d59ff4740 1 -- 172.21.15.52:0/3068411829 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b91ebd55e0 con 0x55b91eb584d0 2024-04-02T13:53:20.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.788+0000 7f3d34ff9640 1 -- 172.21.15.52:0/3068411829 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d40024070 con 0x55b91eb584d0 2024-04-02T13:53:20.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.788+0000 7f3d34ff9640 1 -- 172.21.15.52:0/3068411829 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3d48004eb0 con 0x55b91eb584d0 2024-04-02T13:53:20.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.788+0000 7f3d59ff4740 10 client.15444 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:53:20.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.788+0000 7f3d59ff4740 10 client.15444 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-02T13:53:20.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.788+0000 7f3d59ff4740 20 client.15444 populate_metadata read hostname 'smithi052' 2024-04-02T13:53:20.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.788+0000 7f3d59ff4740 1 --2- 172.21.15.52:0/3068411829 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55b91ebda1f0 0x55b91ebd8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.788+0000 7f3d59ff4740 4 client.15444 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:53:20.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.788+0000 7f3d59ff4740 1 -- 172.21.15.52:0/3068411829 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "status"}) -- 0x55b91eac95a0 con 0x55b91ebda1f0 2024-04-02T13:53:20.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.788+0000 7f3d558d5640 1 --2- 172.21.15.52:0/3068411829 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55b91ebda1f0 0x55b91ebd8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.788+0000 7f3d558d5640 1 --2- 172.21.15.52:0/3068411829 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55b91ebda1f0 0x55b91ebd8f00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cf5fbf1470781ea1 server_cookie=98a5b5386ba4b3f4 in_seq=0 out_seq=0 2024-04-02T13:53:20.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.788+0000 7f3d34ff9640 10 client.15444 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:53:20.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.789+0000 7f3d34ff9640 1 -- 172.21.15.52:0/3068411829 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55b91eac95a0 con 0x55b91ebda1f0 2024-04-02T13:53:20.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.789+0000 7f3d34ff9640 10 client.15444 handle_command_reply: tid=0 2024-04-02T13:53:20.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.789+0000 7f3d59ff4740 1 client.15444 shutdown 2024-04-02T13:53:20.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.789+0000 7f3d59ff4740 1 -- 172.21.15.52:0/3068411829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb584d0 msgr2=0x55b91eb65af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.789+0000 7f3d59ff4740 1 --2- 172.21.15.52:0/3068411829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb584d0 0x55b91eb65af0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3d4802c8e0 tx=0x7f3d4802cc60 comp rx=0 tx=0).stop 2024-04-02T13:53:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.789+0000 7f3d59ff4740 1 -- 172.21.15.52:0/3068411829 shutdown_connections 2024-04-02T13:53:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.789+0000 7f3d59ff4740 1 --2- 172.21.15.52:0/3068411829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b91eb5d640 0x55b91eb68b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.789+0000 7f3d59ff4740 1 --2- 172.21.15.52:0/3068411829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b91eb584d0 0x55b91eb65af0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.789+0000 7f3d59ff4740 1 --2- 172.21.15.52:0/3068411829 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55b91ebda1f0 0x55b91ebd8f00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.789+0000 7f3d59ff4740 1 --2- 172.21.15.52:0/3068411829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b91eb57ae0 0x55b91eb655b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.789+0000 7f3d34ff9640 0 client.15444 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:53:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.789+0000 7f3d59ff4740 1 -- 172.21.15.52:0/3068411829 >> 172.21.15.52:0/3068411829 conn(0x55b91eb53050 msgr2=0x55b91eb53430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.789+0000 7f3d59ff4740 1 -- 172.21.15.52:0/3068411829 shutdown_connections 2024-04-02T13:53:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.789+0000 7f3d59ff4740 1 -- 172.21.15.52:0/3068411829 wait complete. 2024-04-02T13:53:20.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.789+0000 7f3d59ff4740 20 client.15444 trim_cache size 0 max 16384 2024-04-02T13:53:20.803 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:53:20.803 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "bfa00e6e-f0f6-11ee-b647-cb9ed24678a4", 2024-04-02T13:53:20.803 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 3, 2024-04-02T13:53:20.803 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24467, 2024-04-02T13:53:20.803 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-04-02T13:53:20.803 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-04-02T13:53:20.803 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-04-02T13:53:20.803 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 113.455118494, 2024-04-02T13:53:20.803 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 27, 2024-04-02T13:53:20.803 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 92, 2024-04-02T13:53:20.803 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:53:20.804 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 156.950990733 2024-04-02T13:53:20.804 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:53:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.789+0000 7f3d57338640 1 -- 172.21.15.52:0/1211635498 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3d380a3bb0 msgr2=0x7f3d380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.789+0000 7f3d57338640 1 --2- 172.21.15.52:0/1211635498 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3d380a3bb0 0x7f3d380a6070 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3d44098e10 tx=0x7f3d44007f60 comp rx=0 tx=0).stop 2024-04-02T13:53:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.789+0000 7f3d57338640 1 -- 172.21.15.52:0/1211635498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d50104eb0 msgr2=0x7f3d50198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.789+0000 7f3d57338640 1 --2- 172.21.15.52:0/1211635498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d50104eb0 0x7f3d50198310 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3d4400c970 tx=0x7f3d4400ce30 comp rx=0 tx=0).stop 2024-04-02T13:53:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.790+0000 7f3d57338640 1 -- 172.21.15.52:0/1211635498 shutdown_connections 2024-04-02T13:53:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.790+0000 7f3d57338640 1 --2- 172.21.15.52:0/1211635498 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3d380a3bb0 0x7f3d380a6070 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.790+0000 7f3d57338640 1 --2- 172.21.15.52:0/1211635498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d501077e0 0x7f3d5019f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.790+0000 7f3d57338640 1 --2- 172.21.15.52:0/1211635498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d50104eb0 0x7f3d50198310 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.790+0000 7f3d57338640 1 --2- 172.21.15.52:0/1211635498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d50102580 0x7f3d50197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.790+0000 7f3d57338640 1 -- 172.21.15.52:0/1211635498 >> 172.21.15.52:0/1211635498 conn(0x7f3d500fbf90 msgr2=0x7f3d50102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.790+0000 7f3d57338640 1 -- 172.21.15.52:0/1211635498 shutdown_connections 2024-04-02T13:53:20.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.790+0000 7f3d57338640 1 -- 172.21.15.52:0/1211635498 wait complete. 2024-04-02T13:53:20.806 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:20 smithi052 ceph-mon[32104]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:53:20.806 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:20 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/567329474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:53:20.806 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:20 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3658683131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:53:20.806 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:20 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1600672706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:53:20.807 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-02T13:53:20.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.908+0000 7fd9d629a640 1 -- 172.21.15.52:0/243192530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9d0102d20 msgr2=0x7fd9d01031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.908+0000 7fd9d629a640 1 --2- 172.21.15.52:0/243192530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9d0102d20 0x7fd9d01031a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009900 tx=0x7fd9c402bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:20.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.908+0000 7fd9d629a640 1 -- 172.21.15.52:0/243192530 shutdown_connections 2024-04-02T13:53:20.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.908+0000 7fd9d629a640 1 --2- 172.21.15.52:0/243192530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9d01036e0 0x7fd9d0109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.908+0000 7fd9d629a640 1 --2- 172.21.15.52:0/243192530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9d0102d20 0x7fd9d01031a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.908+0000 7fd9d629a640 1 --2- 172.21.15.52:0/243192530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9d0101b30 0x7fd9d0101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.908+0000 7fd9d629a640 1 -- 172.21.15.52:0/243192530 >> 172.21.15.52:0/243192530 conn(0x7fd9d00fd0a0 msgr2=0x7fd9d00ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.908+0000 7fd9d629a640 1 -- 172.21.15.52:0/243192530 shutdown_connections 2024-04-02T13:53:20.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.908+0000 7fd9d629a640 1 -- 172.21.15.52:0/243192530 wait complete. 2024-04-02T13:53:20.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.908+0000 7fd9d629a640 1 Processor -- start 2024-04-02T13:53:20.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.908+0000 7fd9d629a640 1 -- start start 2024-04-02T13:53:20.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.909+0000 7fd9d629a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9d0101b30 0x7fd9d0197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.909+0000 7fd9d629a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9d0102d20 0x7fd9d0198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.909+0000 7fd9d629a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9d01036e0 0x7fd9d019f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.909+0000 7fd9d629a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d010d790 con 0x7fd9d0102d20 2024-04-02T13:53:20.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.909+0000 7fd9d629a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d010d610 con 0x7fd9d0101b30 2024-04-02T13:53:20.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.909+0000 7fd9d629a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d010d910 con 0x7fd9d01036e0 2024-04-02T13:53:20.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.909+0000 7fd9cf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9d0102d20 0x7fd9d0198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.909+0000 7fd9cf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9d0102d20 0x7fd9d0198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41130/0 (socket says 172.21.15.52:41130) 2024-04-02T13:53:20.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.909+0000 7fd9cf7fe640 1 -- 172.21.15.52:0/446781136 learned_addr learned my addr 172.21.15.52:0/446781136 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:20.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.909+0000 7fd9cffff640 1 --2- 172.21.15.52:0/446781136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9d0101b30 0x7fd9d0197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.909+0000 7fd9d4837640 1 --2- 172.21.15.52:0/446781136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9d01036e0 0x7fd9d019f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.909+0000 7fd9cf7fe640 1 -- 172.21.15.52:0/446781136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9d01036e0 msgr2=0x7fd9d019f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.909+0000 7fd9cf7fe640 1 --2- 172.21.15.52:0/446781136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9d01036e0 0x7fd9d019f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.909+0000 7fd9cf7fe640 1 -- 172.21.15.52:0/446781136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9d0101b30 msgr2=0x7fd9d0197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.909+0000 7fd9cf7fe640 1 --2- 172.21.15.52:0/446781136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9d0101b30 0x7fd9d0197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.909+0000 7fd9cf7fe640 1 -- 172.21.15.52:0/446781136 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d019fa20 con 0x7fd9d0102d20 2024-04-02T13:53:20.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.909+0000 7fd9d4837640 1 --2- 172.21.15.52:0/446781136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9d01036e0 0x7fd9d019f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:53:20.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.909+0000 7fd9cf7fe640 1 --2- 172.21.15.52:0/446781136 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9d0102d20 0x7fd9d0198340 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009230 tx=0x7fd9c4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.909+0000 7fd9cd7fa640 1 -- 172.21.15.52:0/446781136 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4041070 con 0x7fd9d0102d20 2024-04-02T13:53:20.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.910+0000 7fd9d629a640 1 -- 172.21.15.52:0/446781136 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9d019fcb0 con 0x7fd9d0102d20 2024-04-02T13:53:20.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.910+0000 7fd9cd7fa640 1 -- 172.21.15.52:0/446781136 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9c40054e0 con 0x7fd9d0102d20 2024-04-02T13:53:20.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.910+0000 7fd9cd7fa640 1 -- 172.21.15.52:0/446781136 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c40346a0 con 0x7fd9d0102d20 2024-04-02T13:53:20.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.910+0000 7fd9d629a640 1 -- 172.21.15.52:0/446781136 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9d01a01a0 con 0x7fd9d0102d20 2024-04-02T13:53:20.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:20 smithi088 ceph-mon[36528]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:53:20.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:20 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/567329474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:53:20.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:20 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3658683131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:53:20.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:20 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1600672706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:53:20.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.912+0000 7fd9cd7fa640 1 -- 172.21.15.52:0/446781136 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd9c4044070 con 0x7fd9d0102d20 2024-04-02T13:53:20.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.912+0000 7fd9d8f64740 1 Processor -- start 2024-04-02T13:53:20.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.912+0000 7fd9d8f64740 1 -- start start 2024-04-02T13:53:20.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.912+0000 7fd9cd7fa640 1 --2- 172.21.15.52:0/446781136 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fd9b80a3bb0 0x7fd9b80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.912+0000 7fd9d8f64740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55802ac59ae0 0x55802ac59f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.912+0000 7fd9d8f64740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55802ac5a4d0 0x55802ac5efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.912+0000 7fd9d8f64740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac5f640 0x55802ac61ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.912+0000 7fd9d8f64740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55802abbe060 con 0x55802ac5f640 2024-04-02T13:53:20.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.912+0000 7fd9d8f64740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55802ab66650 con 0x55802ac59ae0 2024-04-02T13:53:20.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.912+0000 7fd9d8f64740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55802abba010 con 0x55802ac5a4d0 2024-04-02T13:53:20.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.912+0000 7fd9cd7fa640 1 -- 172.21.15.52:0/446781136 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd9c40f2470 con 0x7fd9d0102d20 2024-04-02T13:53:20.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.912+0000 7fd9d4837640 1 --2- 172.21.15.52:0/446781136 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fd9b80a3bb0 0x7fd9b80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.913+0000 7fd9cffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55802ac59ae0 0x55802ac59f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.913+0000 7fd9cffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55802ac59ae0 0x55802ac59f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:57176/0 (socket says 172.21.15.52:57176) 2024-04-02T13:53:20.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.913+0000 7fd9cffff640 1 -- 172.21.15.52:0/2068218356 learned_addr learned my addr 172.21.15.52:0/2068218356 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:20.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.913+0000 7fd9d4837640 1 --2- 172.21.15.52:0/2068218356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac5f640 0x55802ac61ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.913+0000 7fd9cf7fe640 1 --2- 172.21.15.52:0/2068218356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55802ac5a4d0 0x55802ac5efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.913+0000 7fd9cffff640 1 -- 172.21.15.52:0/2068218356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55802ac5a4d0 msgr2=0x55802ac5efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.913+0000 7fd9cffff640 1 --2- 172.21.15.52:0/2068218356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55802ac5a4d0 0x55802ac5efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.913+0000 7fd9cffff640 1 -- 172.21.15.52:0/2068218356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac5f640 msgr2=0x55802ac61ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.913+0000 7fd9cffff640 1 --2- 172.21.15.52:0/2068218356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac5f640 0x55802ac61ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.913+0000 7fd9cffff640 1 -- 172.21.15.52:0/2068218356 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55802a8d22d0 con 0x55802ac59ae0 2024-04-02T13:53:20.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.913+0000 7fd9cf7fe640 1 --2- 172.21.15.52:0/2068218356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55802ac5a4d0 0x55802ac5efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:53:20.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.913+0000 7fd9d4837640 1 --2- 172.21.15.52:0/446781136 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fd9b80a3bb0 0x7fd9b80a6070 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8005fd0 tx=0x7fd9c8014ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.913+0000 7fd9cffff640 1 --2- 172.21.15.52:0/2068218356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55802ac59ae0 0x55802ac59f00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000abf0 tx=0x7fd9c000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94de5b1dd0e914eb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.913+0000 7fd9b67fc640 1 -- 172.21.15.52:0/2068218356 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0004220 con 0x55802ac59ae0 2024-04-02T13:53:20.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.913+0000 7fd9b67fc640 1 -- 172.21.15.52:0/2068218356 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9c00043c0 con 0x55802ac59ae0 2024-04-02T13:53:20.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.913+0000 7fd9b67fc640 1 -- 172.21.15.52:0/2068218356 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0018810 con 0x55802ac59ae0 2024-04-02T13:53:20.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.914+0000 7fd9d8f64740 1 -- 172.21.15.52:0/2068218356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55802ac59ae0 msgr2=0x55802ac59f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.914+0000 7fd9d8f64740 1 --2- 172.21.15.52:0/2068218356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55802ac59ae0 0x55802ac59f00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000abf0 tx=0x7fd9c000f790 comp rx=0 tx=0).stop 2024-04-02T13:53:20.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.914+0000 7fd9d8f64740 1 -- 172.21.15.52:0/2068218356 shutdown_connections 2024-04-02T13:53:20.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.914+0000 7fd9d8f64740 1 --2- 172.21.15.52:0/2068218356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac5f640 0x55802ac61ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.914+0000 7fd9d8f64740 1 --2- 172.21.15.52:0/2068218356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55802ac5a4d0 0x55802ac5efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.914+0000 7fd9d8f64740 1 --2- 172.21.15.52:0/2068218356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55802ac59ae0 0x55802ac59f00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.914+0000 7fd9d8f64740 1 -- 172.21.15.52:0/2068218356 >> 172.21.15.52:0/2068218356 conn(0x55802ac55050 msgr2=0x55802ac55450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.914+0000 7fd9d8f64740 1 -- 172.21.15.52:0/2068218356 shutdown_connections 2024-04-02T13:53:20.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.914+0000 7fd9d8f64740 1 -- 172.21.15.52:0/2068218356 wait complete. 2024-04-02T13:53:20.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.914+0000 7fd9d8f64740 1 -- start start 2024-04-02T13:53:20.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.915+0000 7fd9d8f64740 1 Processor -- start 2024-04-02T13:53:20.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.915+0000 7fd9d8f64740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55802ac59ae0 0x55802ac63ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.915+0000 7fd9d8f64740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55802ac5a4d0 0x55802ac64530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.915+0000 7fd9d8f64740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac5f640 0x55802ac6b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.915+0000 7fd9d8f64740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55802ac625f0 con 0x55802ac5f640 2024-04-02T13:53:20.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.915+0000 7fd9d8f64740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55802ab79a10 con 0x55802ac5a4d0 2024-04-02T13:53:20.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.915+0000 7fd9d8f64740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55802ac62770 con 0x55802ac59ae0 2024-04-02T13:53:20.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.915+0000 7fd9d8f64740 10 client.0 _ll_register_callbacks cb 0x55802abb60b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:20.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.915+0000 7fd9d4837640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac5f640 0x55802ac6b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.915+0000 7fd9d4837640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac5f640 0x55802ac6b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41146/0 (socket says 172.21.15.52:41146) 2024-04-02T13:53:20.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.915+0000 7fd9d4837640 1 -- 172.21.15.52:0/1549259140 learned_addr learned my addr 172.21.15.52:0/1549259140 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:20.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.915+0000 7fd9cf7fe640 1 --2- 172.21.15.52:0/1549259140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55802ac5a4d0 0x55802ac64530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.915+0000 7fd9cffff640 1 --2- 172.21.15.52:0/1549259140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55802ac59ae0 0x55802ac63ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.915+0000 7fd9d4837640 1 -- 172.21.15.52:0/1549259140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55802ac59ae0 msgr2=0x55802ac63ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.915+0000 7fd9d4837640 1 --2- 172.21.15.52:0/1549259140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55802ac59ae0 0x55802ac63ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.915+0000 7fd9d4837640 1 -- 172.21.15.52:0/1549259140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55802ac5a4d0 msgr2=0x55802ac64530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.915+0000 7fd9d4837640 1 --2- 172.21.15.52:0/1549259140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55802ac5a4d0 0x55802ac64530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.915+0000 7fd9d4837640 1 -- 172.21.15.52:0/1549259140 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55802a99c6b0 con 0x55802ac5f640 2024-04-02T13:53:20.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.915+0000 7fd9cffff640 1 --2- 172.21.15.52:0/1549259140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55802ac59ae0 0x55802ac63ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:20.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.915+0000 7fd9d4837640 1 --2- 172.21.15.52:0/1549259140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac5f640 0x55802ac6b570 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8011d50 tx=0x7fd9c8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99858c89b7b50ab server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.915+0000 7fd9af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:53:20.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.915+0000 7fd9af7fe640 1 -- 172.21.15.52:0/1549259140 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c801f630 con 0x55802ac5f640 2024-04-02T13:53:20.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.916+0000 7fd9d8f64740 1 -- 172.21.15.52:0/1549259140 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9c00043c0 con 0x55802ac5f640 2024-04-02T13:53:20.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.916+0000 7fd9af7fe640 1 -- 172.21.15.52:0/1549259140 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd9c801f7d0 con 0x55802ac5f640 2024-04-02T13:53:20.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.916+0000 7fd9af7fe640 1 -- 172.21.15.52:0/1549259140 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd9c8032050 con 0x55802ac5f640 2024-04-02T13:53:20.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.916+0000 7fd9d8f64740 10 client.15456 fetch_fsmap learned FSMap version 27 2024-04-02T13:53:20.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.916+0000 7fd9d8f64740 1 -- 172.21.15.52:0/1549259140 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55802ac6e8c0 con 0x55802ac5f640 2024-04-02T13:53:20.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.916+0000 7fd9af7fe640 1 -- 172.21.15.52:0/1549259140 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9c00043c0 con 0x55802ac5f640 2024-04-02T13:53:20.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.916+0000 7fd9af7fe640 1 -- 172.21.15.52:0/1549259140 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd9c80298f0 con 0x55802ac5f640 2024-04-02T13:53:20.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.916+0000 7fd9d8f64740 10 client.15456 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:53:20.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.916+0000 7fd9d8f64740 10 client.15456 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-02T13:53:20.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.916+0000 7fd9d8f64740 20 client.15456 populate_metadata read hostname 'smithi052' 2024-04-02T13:53:20.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.916+0000 7fd9d8f64740 1 --2- 172.21.15.52:0/1549259140 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55802ac70f70 0x55802ac723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.916+0000 7fd9d8f64740 4 client.15456 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:53:20.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.916+0000 7fd9d8f64740 1 -- 172.21.15.52:0/1549259140 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55802ab65cf0 con 0x55802ac70f70 2024-04-02T13:53:20.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.916+0000 7fd9cffff640 1 --2- 172.21.15.52:0/1549259140 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55802ac70f70 0x55802ac723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.917+0000 7fd9cffff640 1 --2- 172.21.15.52:0/1549259140 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55802ac70f70 0x55802ac723e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb24a4a758dc1c12 server_cookie=9fbf6e98a2f0b2a2 in_seq=0 out_seq=0 2024-04-02T13:53:20.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.917+0000 7fd9af7fe640 10 client.15456 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:53:20.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.918+0000 7fd9af7fe640 1 -- 172.21.15.52:0/1549259140 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55802ab65cf0 con 0x55802ac70f70 2024-04-02T13:53:20.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.918+0000 7fd9af7fe640 10 client.15456 handle_command_reply: tid=0 2024-04-02T13:53:20.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.918+0000 7fd9d8f64740 1 client.15456 shutdown 2024-04-02T13:53:20.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.918+0000 7fd9d8f64740 1 -- 172.21.15.52:0/1549259140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac5f640 msgr2=0x55802ac6b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.918+0000 7fd9d8f64740 1 --2- 172.21.15.52:0/1549259140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac5f640 0x55802ac6b570 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8011d50 tx=0x7fd9c8004ab0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.918+0000 7fd9d8f64740 1 -- 172.21.15.52:0/1549259140 shutdown_connections 2024-04-02T13:53:20.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.918+0000 7fd9d8f64740 1 --2- 172.21.15.52:0/1549259140 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55802ac70f70 0x55802ac723e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.918+0000 7fd9d8f64740 1 --2- 172.21.15.52:0/1549259140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac5f640 0x55802ac6b570 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.918+0000 7fd9d8f64740 1 --2- 172.21.15.52:0/1549259140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55802ac5a4d0 0x55802ac64530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.918+0000 7fd9d8f64740 1 --2- 172.21.15.52:0/1549259140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55802ac59ae0 0x55802ac63ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.918+0000 7fd9af7fe640 0 client.15456 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:53:20.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.918+0000 7fd9d8f64740 1 -- 172.21.15.52:0/1549259140 >> 172.21.15.52:0/1549259140 conn(0x55802ac55050 msgr2=0x55802ac54190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.918+0000 7fd9d8f64740 1 -- 172.21.15.52:0/1549259140 shutdown_connections 2024-04-02T13:53:20.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.918+0000 7fd9d8f64740 1 -- 172.21.15.52:0/1549259140 wait complete. 2024-04-02T13:53:20.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.918+0000 7fd9d8f64740 20 client.15456 trim_cache size 0 max 16384 2024-04-02T13:53:20.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.936+0000 7fd9d8f64740 1 Processor -- start 2024-04-02T13:53:20.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.936+0000 7fd9d8f64740 1 -- start start 2024-04-02T13:53:20.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.936+0000 7fd9d8f64740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55802ac59ae0 0x55802ac61a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.936+0000 7fd9d8f64740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac5a4d0 0x55802ac63c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.936+0000 7fd9d8f64740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55802ac5f640 0x55802ac64140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.936+0000 7fd9d8f64740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55802ac70720 con 0x55802ac5a4d0 2024-04-02T13:53:20.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.936+0000 7fd9d8f64740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55802ac70df0 con 0x55802ac5f640 2024-04-02T13:53:20.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.936+0000 7fd9d8f64740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55802ac714c0 con 0x55802ac59ae0 2024-04-02T13:53:20.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.936+0000 7fd9cf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac5a4d0 0x55802ac63c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.936+0000 7fd9cf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac5a4d0 0x55802ac63c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41154/0 (socket says 172.21.15.52:41154) 2024-04-02T13:53:20.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.936+0000 7fd9cf7fe640 1 -- 172.21.15.52:0/2286695529 learned_addr learned my addr 172.21.15.52:0/2286695529 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:20.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.936+0000 7fd9cffff640 1 --2- 172.21.15.52:0/2286695529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55802ac59ae0 0x55802ac61a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.936+0000 7fd9cf7fe640 1 -- 172.21.15.52:0/2286695529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55802ac59ae0 msgr2=0x55802ac61a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.936+0000 7fd9cf7fe640 1 --2- 172.21.15.52:0/2286695529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55802ac59ae0 0x55802ac61a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.936+0000 7fd9cf7fe640 1 -- 172.21.15.52:0/2286695529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55802ac5f640 msgr2=0x55802ac64140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.936+0000 7fd9cf7fe640 1 --2- 172.21.15.52:0/2286695529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55802ac5f640 0x55802ac64140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.936+0000 7fd9cf7fe640 1 -- 172.21.15.52:0/2286695529 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55802ac5ad60 con 0x55802ac5a4d0 2024-04-02T13:53:20.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.937+0000 7fd9cf7fe640 1 --2- 172.21.15.52:0/2286695529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac5a4d0 0x55802ac63c00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40badc0 tx=0x7fd9c4049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f120ebd70c36fa9f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.937+0000 7fd9b5ffb640 1 -- 172.21.15.52:0/2286695529 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c40d7070 con 0x55802ac5a4d0 2024-04-02T13:53:20.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.937+0000 7fd9b5ffb640 1 -- 172.21.15.52:0/2286695529 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9c40035f0 con 0x55802ac5a4d0 2024-04-02T13:53:20.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.937+0000 7fd9b5ffb640 1 -- 172.21.15.52:0/2286695529 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c404b040 con 0x55802ac5a4d0 2024-04-02T13:53:20.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.937+0000 7fd9d8f64740 1 -- 172.21.15.52:0/2286695529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac5a4d0 msgr2=0x55802ac63c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.937+0000 7fd9d8f64740 1 --2- 172.21.15.52:0/2286695529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac5a4d0 0x55802ac63c00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40badc0 tx=0x7fd9c4049000 comp rx=0 tx=0).stop 2024-04-02T13:53:20.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.937+0000 7fd9d8f64740 1 -- 172.21.15.52:0/2286695529 shutdown_connections 2024-04-02T13:53:20.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.937+0000 7fd9d8f64740 1 --2- 172.21.15.52:0/2286695529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55802ac5f640 0x55802ac64140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.937+0000 7fd9d8f64740 1 --2- 172.21.15.52:0/2286695529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac5a4d0 0x55802ac63c00 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.937+0000 7fd9d8f64740 1 --2- 172.21.15.52:0/2286695529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55802ac59ae0 0x55802ac61a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.937+0000 7fd9d8f64740 1 -- 172.21.15.52:0/2286695529 >> 172.21.15.52:0/2286695529 conn(0x55802ac55050 msgr2=0x55802ac5b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.937+0000 7fd9d8f64740 1 -- 172.21.15.52:0/2286695529 shutdown_connections 2024-04-02T13:53:20.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.937+0000 7fd9d8f64740 1 -- 172.21.15.52:0/2286695529 wait complete. 2024-04-02T13:53:20.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.938+0000 7fd9d8f64740 1 -- start start 2024-04-02T13:53:20.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.938+0000 7fd9d8f64740 1 Processor -- start 2024-04-02T13:53:20.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.938+0000 7fd9d8f64740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac59ae0 0x55802acd5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.938+0000 7fd9d8f64740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55802ac5a4d0 0x55802acd7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.938+0000 7fd9d8f64740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55802ac5f640 0x55802acd9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.938+0000 7fd9d8f64740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55802ac72260 con 0x55802ac59ae0 2024-04-02T13:53:20.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.938+0000 7fd9d8f64740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55802ac71b90 con 0x55802ac5f640 2024-04-02T13:53:20.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.938+0000 7fd9d8f64740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55802ac72930 con 0x55802ac5a4d0 2024-04-02T13:53:20.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.938+0000 7fd9d8f64740 10 client.0 _ll_register_callbacks cb 0x7fd9c80082b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:20.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.938+0000 7fd9cffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac59ae0 0x55802acd5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.938+0000 7fd9cffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac59ae0 0x55802acd5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41160/0 (socket says 172.21.15.52:41160) 2024-04-02T13:53:20.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.938+0000 7fd9cffff640 1 -- 172.21.15.52:0/1375094775 learned_addr learned my addr 172.21.15.52:0/1375094775 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:20.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.938+0000 7fd9cf7fe640 1 --2- 172.21.15.52:0/1375094775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55802ac5a4d0 0x55802acd7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.938+0000 7fd9d4837640 1 --2- 172.21.15.52:0/1375094775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55802ac5f640 0x55802acd9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.938+0000 7fd9cffff640 1 -- 172.21.15.52:0/1375094775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55802ac5a4d0 msgr2=0x55802acd7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.938+0000 7fd9cffff640 1 --2- 172.21.15.52:0/1375094775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55802ac5a4d0 0x55802acd7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.938+0000 7fd9cffff640 1 -- 172.21.15.52:0/1375094775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55802ac5f640 msgr2=0x55802acd9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.938+0000 7fd9cffff640 1 --2- 172.21.15.52:0/1375094775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55802ac5f640 0x55802acd9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.938+0000 7fd9cffff640 1 -- 172.21.15.52:0/1375094775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55802acda4c0 con 0x55802ac59ae0 2024-04-02T13:53:20.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.939+0000 7fd9cffff640 1 --2- 172.21.15.52:0/1375094775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac59ae0 0x55802acd5100 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000abf0 tx=0x7fd9c000a360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd9ba913f6be913 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.939+0000 7fd9af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:53:20.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.939+0000 7fd9af7fe640 1 -- 172.21.15.52:0/1375094775 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0032060 con 0x55802ac59ae0 2024-04-02T13:53:20.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.939+0000 7fd9af7fe640 1 -- 172.21.15.52:0/1375094775 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd9c0004330 con 0x55802ac59ae0 2024-04-02T13:53:20.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.939+0000 7fd9d8f64740 1 -- 172.21.15.52:0/1375094775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55802abcb5a0 con 0x55802ac59ae0 2024-04-02T13:53:20.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.939+0000 7fd9af7fe640 1 -- 172.21.15.52:0/1375094775 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd9c0039050 con 0x55802ac59ae0 2024-04-02T13:53:20.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.939+0000 7fd9d8f64740 10 client.15462 fetch_fsmap learned FSMap version 27 2024-04-02T13:53:20.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.939+0000 7fd9d8f64740 1 -- 172.21.15.52:0/1375094775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55802acdb830 con 0x55802ac59ae0 2024-04-02T13:53:20.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.939+0000 7fd9af7fe640 1 -- 172.21.15.52:0/1375094775 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55802abcb5a0 con 0x55802ac59ae0 2024-04-02T13:53:20.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.939+0000 7fd9af7fe640 1 -- 172.21.15.52:0/1375094775 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd9c00054e0 con 0x55802ac59ae0 2024-04-02T13:53:20.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.939+0000 7fd9d8f64740 10 client.15462 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:53:20.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.939+0000 7fd9d8f64740 10 client.15462 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-02T13:53:20.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.939+0000 7fd9d8f64740 20 client.15462 populate_metadata read hostname 'smithi052' 2024-04-02T13:53:20.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.939+0000 7fd9d8f64740 1 --2- 172.21.15.52:0/1375094775 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55802ace00c0 0x55802acdedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.939+0000 7fd9d8f64740 4 client.15462 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:53:20.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.939+0000 7fd9d8f64740 1 -- 172.21.15.52:0/1375094775 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "status"}) -- 0x55802abc58f0 con 0x55802ace00c0 2024-04-02T13:53:20.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.940+0000 7fd9cf7fe640 1 --2- 172.21.15.52:0/1375094775 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55802ace00c0 0x55802acdedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.940+0000 7fd9cf7fe640 1 --2- 172.21.15.52:0/1375094775 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55802ace00c0 0x55802acdedd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c4892685fe5f0ed2 server_cookie=422a3704ba5c7f44 in_seq=0 out_seq=0 2024-04-02T13:53:20.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.940+0000 7fd9af7fe640 10 client.15462 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:53:20.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.940+0000 7fd9af7fe640 1 -- 172.21.15.52:0/1375094775 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55802abc58f0 con 0x55802ace00c0 2024-04-02T13:53:20.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.940+0000 7fd9af7fe640 10 client.15462 handle_command_reply: tid=0 2024-04-02T13:53:20.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.940+0000 7fd9d8f64740 1 client.15462 shutdown 2024-04-02T13:53:20.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.940+0000 7fd9d8f64740 1 -- 172.21.15.52:0/1375094775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac59ae0 msgr2=0x55802acd5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.940+0000 7fd9d8f64740 1 --2- 172.21.15.52:0/1375094775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac59ae0 0x55802acd5100 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000abf0 tx=0x7fd9c000a360 comp rx=0 tx=0).stop 2024-04-02T13:53:20.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.941+0000 7fd9d8f64740 1 -- 172.21.15.52:0/1375094775 shutdown_connections 2024-04-02T13:53:20.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.941+0000 7fd9d8f64740 1 --2- 172.21.15.52:0/1375094775 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55802ace00c0 0x55802acdedd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.941+0000 7fd9d8f64740 1 --2- 172.21.15.52:0/1375094775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55802ac5f640 0x55802acd9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.941+0000 7fd9d8f64740 1 --2- 172.21.15.52:0/1375094775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55802ac5a4d0 0x55802acd7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.941+0000 7fd9af7fe640 0 client.15462 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:53:20.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.941+0000 7fd9d8f64740 1 --2- 172.21.15.52:0/1375094775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55802ac59ae0 0x55802acd5100 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.941+0000 7fd9d8f64740 1 -- 172.21.15.52:0/1375094775 >> 172.21.15.52:0/1375094775 conn(0x55802ac55050 msgr2=0x55802ac63c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.941+0000 7fd9d8f64740 1 -- 172.21.15.52:0/1375094775 shutdown_connections 2024-04-02T13:53:20.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.941+0000 7fd9d8f64740 1 -- 172.21.15.52:0/1375094775 wait complete. 2024-04-02T13:53:20.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.941+0000 7fd9d8f64740 20 client.15462 trim_cache size 0 max 16384 2024-04-02T13:53:20.951 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:53:20.951 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "bfa00e6e-f0f6-11ee-b647-cb9ed24678a4", 2024-04-02T13:53:20.951 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 2, 2024-04-02T13:53:20.951 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24475, 2024-04-02T13:53:20.951 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-04-02T13:53:20.951 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-04-02T13:53:20.951 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-04-02T13:53:20.951 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 114.613425389, 2024-04-02T13:53:20.951 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 27, 2024-04-02T13:53:20.951 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 92, 2024-04-02T13:53:20.952 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:53:20.952 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 159.95155232299999 2024-04-02T13:53:20.952 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:53:20.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.941+0000 7fd9d629a640 1 -- 172.21.15.52:0/446781136 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fd9b80a3bb0 msgr2=0x7fd9b80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.941+0000 7fd9d629a640 1 --2- 172.21.15.52:0/446781136 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fd9b80a3bb0 0x7fd9b80a6070 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8005fd0 tx=0x7fd9c8014ab0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.941+0000 7fd9d629a640 1 -- 172.21.15.52:0/446781136 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9d0102d20 msgr2=0x7fd9d0198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.941+0000 7fd9d629a640 1 --2- 172.21.15.52:0/446781136 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9d0102d20 0x7fd9d0198340 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009230 tx=0x7fd9c4004040 comp rx=0 tx=0).stop 2024-04-02T13:53:20.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.941+0000 7fd9d629a640 1 -- 172.21.15.52:0/446781136 shutdown_connections 2024-04-02T13:53:20.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.941+0000 7fd9d629a640 1 --2- 172.21.15.52:0/446781136 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7fd9b80a3bb0 0x7fd9b80a6070 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.941+0000 7fd9d629a640 1 --2- 172.21.15.52:0/446781136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9d01036e0 0x7fd9d019f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.941+0000 7fd9d629a640 1 --2- 172.21.15.52:0/446781136 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9d0102d20 0x7fd9d0198340 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.941+0000 7fd9d629a640 1 --2- 172.21.15.52:0/446781136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9d0101b30 0x7fd9d0197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.941+0000 7fd9d629a640 1 -- 172.21.15.52:0/446781136 >> 172.21.15.52:0/446781136 conn(0x7fd9d00fd0a0 msgr2=0x7fd9d0106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.941+0000 7fd9d629a640 1 -- 172.21.15.52:0/446781136 shutdown_connections 2024-04-02T13:53:20.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:20.941+0000 7fd9d629a640 1 -- 172.21.15.52:0/446781136 wait complete. 2024-04-02T13:53:20.955 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-02T13:53:21.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:20 smithi189 ceph-mon[36683]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-02T13:53:21.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:20 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/567329474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:53:21.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:20 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3658683131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:53:21.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:20 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1600672706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:53:21.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.056+0000 7f9c2b98d640 1 -- 172.21.15.52:0/2292319782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c24101d60 msgr2=0x7f9c241085f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.056+0000 7f9c2b98d640 1 --2- 172.21.15.52:0/2292319782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c24101d60 0x7f9c241085f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009ca0 tx=0x7f9c1c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.056+0000 7f9c2b98d640 1 -- 172.21.15.52:0/2292319782 shutdown_connections 2024-04-02T13:53:21.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.056+0000 7f9c2b98d640 1 --2- 172.21.15.52:0/2292319782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c24101d60 0x7f9c241085f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.056+0000 7f9c2b98d640 1 --2- 172.21.15.52:0/2292319782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c241013a0 0x7f9c24101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.056+0000 7f9c2b98d640 1 --2- 172.21.15.52:0/2292319782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c241001b0 0x7f9c241005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.056+0000 7f9c2b98d640 1 -- 172.21.15.52:0/2292319782 >> 172.21.15.52:0/2292319782 conn(0x7f9c240fb700 msgr2=0x7f9c240fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.056+0000 7f9c2b98d640 1 -- 172.21.15.52:0/2292319782 shutdown_connections 2024-04-02T13:53:21.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.056+0000 7f9c2b98d640 1 -- 172.21.15.52:0/2292319782 wait complete. 2024-04-02T13:53:21.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.056+0000 7f9c2b98d640 1 Processor -- start 2024-04-02T13:53:21.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.056+0000 7f9c2b98d640 1 -- start start 2024-04-02T13:53:21.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.056+0000 7f9c2b98d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c241001b0 0x7f9c24197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.056+0000 7f9c2b98d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c241013a0 0x7f9c24198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.056+0000 7f9c2b98d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c24101d60 0x7f9c2419f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.056+0000 7f9c2b98d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2410b5c0 con 0x7f9c24101d60 2024-04-02T13:53:21.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.056+0000 7f9c2b98d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2410b440 con 0x7f9c241013a0 2024-04-02T13:53:21.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.056+0000 7f9c2b98d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2410b740 con 0x7f9c241001b0 2024-04-02T13:53:21.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.057+0000 7f9c29f2a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c24101d60 0x7f9c2419f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.057+0000 7f9c29f2a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c24101d60 0x7f9c2419f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41184/0 (socket says 172.21.15.52:41184) 2024-04-02T13:53:21.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.057+0000 7f9c29f2a640 1 -- 172.21.15.52:0/2545503353 learned_addr learned my addr 172.21.15.52:0/2545503353 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:21.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.057+0000 7f9c29729640 1 --2- 172.21.15.52:0/2545503353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c241001b0 0x7f9c24197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.057+0000 7f9c28f28640 1 --2- 172.21.15.52:0/2545503353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c241013a0 0x7f9c24198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.057+0000 7f9c29f2a640 1 -- 172.21.15.52:0/2545503353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c241001b0 msgr2=0x7f9c24197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.057+0000 7f9c29f2a640 1 --2- 172.21.15.52:0/2545503353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c241001b0 0x7f9c24197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.057+0000 7f9c29f2a640 1 -- 172.21.15.52:0/2545503353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c241013a0 msgr2=0x7f9c24198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.057+0000 7f9c29f2a640 1 --2- 172.21.15.52:0/2545503353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c241013a0 0x7f9c24198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.057+0000 7f9c29f2a640 1 -- 172.21.15.52:0/2545503353 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c2419fa10 con 0x7f9c24101d60 2024-04-02T13:53:21.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.057+0000 7f9c29f2a640 1 --2- 172.21.15.52:0/2545503353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c24101d60 0x7f9c2419f3a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c037040 tx=0x7f9c1c032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.057+0000 7f9c1a7fc640 1 -- 172.21.15.52:0/2545503353 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c041020 con 0x7f9c24101d60 2024-04-02T13:53:21.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.057+0000 7f9c2b98d640 1 -- 172.21.15.52:0/2545503353 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c2419fca0 con 0x7f9c24101d60 2024-04-02T13:53:21.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.057+0000 7f9c1a7fc640 1 -- 172.21.15.52:0/2545503353 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c1c034d90 con 0x7f9c24101d60 2024-04-02T13:53:21.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.057+0000 7f9c1a7fc640 1 -- 172.21.15.52:0/2545503353 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c039040 con 0x7f9c24101d60 2024-04-02T13:53:21.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.057+0000 7f9c2b98d640 1 -- 172.21.15.52:0/2545503353 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c241a0190 con 0x7f9c24101d60 2024-04-02T13:53:21.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.059+0000 7f9c1a7fc640 1 -- 172.21.15.52:0/2545503353 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9c1c044020 con 0x7f9c24101d60 2024-04-02T13:53:21.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.060+0000 7f9c2e571740 1 Processor -- start 2024-04-02T13:53:21.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.060+0000 7f9c2e571740 1 -- start start 2024-04-02T13:53:21.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.060+0000 7f9c2e571740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a16fa93ef0 0x55a16fa94310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.060+0000 7f9c2e571740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a16fa948e0 0x55a16fa993e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.060+0000 7f9c2e571740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa99a50 0x55a16fa9bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.060+0000 7f9c2e571740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a16f9f4060 con 0x55a16fa99a50 2024-04-02T13:53:21.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.060+0000 7f9c2e571740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a16f99c650 con 0x55a16fa93ef0 2024-04-02T13:53:21.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.060+0000 7f9c2e571740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a16f9f0010 con 0x55a16fa948e0 2024-04-02T13:53:21.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.060+0000 7f9c1a7fc640 1 --2- 172.21.15.52:0/2545503353 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f9c100a3bb0 0x7f9c100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.060+0000 7f9c28f28640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa99a50 0x55a16fa9bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.060+0000 7f9c28f28640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa99a50 0x55a16fa9bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41190/0 (socket says 172.21.15.52:41190) 2024-04-02T13:53:21.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.060+0000 7f9c28f28640 1 -- 172.21.15.52:0/3351699570 learned_addr learned my addr 172.21.15.52:0/3351699570 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:21.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.060+0000 7f9c29f2a640 1 --2- 172.21.15.52:0/2545503353 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f9c100a3bb0 0x7f9c100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.060+0000 7f9c29729640 1 --2- 172.21.15.52:0/3351699570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a16fa948e0 0x55a16fa993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.060+0000 7f9c28f28640 1 --2- 172.21.15.52:0/3351699570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a16fa93ef0 0x55a16fa94310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.061+0000 7f9c1a7fc640 1 -- 172.21.15.52:0/2545503353 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f9c1c0f2710 con 0x7f9c24101d60 2024-04-02T13:53:21.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.061+0000 7f9c28f28640 1 -- 172.21.15.52:0/3351699570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a16fa948e0 msgr2=0x55a16fa993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.061+0000 7f9c28f28640 1 --2- 172.21.15.52:0/3351699570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a16fa948e0 0x55a16fa993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.061+0000 7f9c28f28640 1 -- 172.21.15.52:0/3351699570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a16fa93ef0 msgr2=0x55a16fa94310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.061+0000 7f9c28f28640 1 --2- 172.21.15.52:0/3351699570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a16fa93ef0 0x55a16fa94310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.061+0000 7f9c28f28640 1 -- 172.21.15.52:0/3351699570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a16f7082d0 con 0x55a16fa99a50 2024-04-02T13:53:21.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.061+0000 7f9c29729640 1 --2- 172.21.15.52:0/3351699570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a16fa948e0 0x55a16fa993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:21.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.061+0000 7f9c29f2a640 1 --2- 172.21.15.52:0/2545503353 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f9c100a3bb0 0x7f9c100a6070 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c0efe50 tx=0x7f9c1c08df60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.061+0000 7f9c28f28640 1 --2- 172.21.15.52:0/3351699570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa99a50 0x55a16fa9bee0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9c140124c0 tx=0x7f9c1400c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e216b58efaf19 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.061+0000 7f9c0f7fe640 1 -- 172.21.15.52:0/3351699570 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c14018030 con 0x55a16fa99a50 2024-04-02T13:53:21.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.061+0000 7f9c0f7fe640 1 -- 172.21.15.52:0/3351699570 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c14005500 con 0x55a16fa99a50 2024-04-02T13:53:21.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.061+0000 7f9c0f7fe640 1 -- 172.21.15.52:0/3351699570 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1400d7b0 con 0x55a16fa99a50 2024-04-02T13:53:21.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.062+0000 7f9c2e571740 1 -- 172.21.15.52:0/3351699570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa99a50 msgr2=0x55a16fa9bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.062+0000 7f9c2e571740 1 --2- 172.21.15.52:0/3351699570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa99a50 0x55a16fa9bee0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9c140124c0 tx=0x7f9c1400c5c0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.062+0000 7f9c2e571740 1 -- 172.21.15.52:0/3351699570 shutdown_connections 2024-04-02T13:53:21.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.062+0000 7f9c2e571740 1 --2- 172.21.15.52:0/3351699570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa99a50 0x55a16fa9bee0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.062+0000 7f9c2e571740 1 --2- 172.21.15.52:0/3351699570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a16fa948e0 0x55a16fa993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.062+0000 7f9c2e571740 1 --2- 172.21.15.52:0/3351699570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a16fa93ef0 0x55a16fa94310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.062+0000 7f9c2e571740 1 -- 172.21.15.52:0/3351699570 >> 172.21.15.52:0/3351699570 conn(0x55a16fa8b050 msgr2=0x55a16fa8b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.062+0000 7f9c2e571740 1 -- 172.21.15.52:0/3351699570 shutdown_connections 2024-04-02T13:53:21.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.062+0000 7f9c2e571740 1 -- 172.21.15.52:0/3351699570 wait complete. 2024-04-02T13:53:21.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.062+0000 7f9c2e571740 1 -- start start 2024-04-02T13:53:21.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.062+0000 7f9c2e571740 1 Processor -- start 2024-04-02T13:53:21.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.062+0000 7f9c2e571740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a16fa93ef0 0x55a16fa9e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.062+0000 7f9c2e571740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a16fa948e0 0x55a16fa9e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.063+0000 7f9c2e571740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa99a50 0x55a16faa5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.063+0000 7f9c2e571740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a16fa9ca00 con 0x55a16fa99a50 2024-04-02T13:53:21.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.063+0000 7f9c2e571740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a16f9afa10 con 0x55a16fa948e0 2024-04-02T13:53:21.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.063+0000 7f9c2e571740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a16fa9cb80 con 0x55a16fa93ef0 2024-04-02T13:53:21.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.063+0000 7f9c2e571740 10 client.0 _ll_register_callbacks cb 0x55a16f9ec0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:21.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.063+0000 7f9c28f28640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa99a50 0x55a16faa5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.063+0000 7f9c28f28640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa99a50 0x55a16faa5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41202/0 (socket says 172.21.15.52:41202) 2024-04-02T13:53:21.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.063+0000 7f9c28f28640 1 -- 172.21.15.52:0/2580345642 learned_addr learned my addr 172.21.15.52:0/2580345642 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:21.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.063+0000 7f9c28f28640 1 --2- 172.21.15.52:0/2580345642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a16fa93ef0 0x55a16fa9e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.063+0000 7f9c29729640 1 --2- 172.21.15.52:0/2580345642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a16fa948e0 0x55a16fa9e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.063+0000 7f9c28f28640 1 -- 172.21.15.52:0/2580345642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a16fa93ef0 msgr2=0x55a16fa9e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.063+0000 7f9c28f28640 1 --2- 172.21.15.52:0/2580345642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a16fa93ef0 0x55a16fa9e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.063+0000 7f9c28f28640 1 -- 172.21.15.52:0/2580345642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a16fa948e0 msgr2=0x55a16fa9e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.063+0000 7f9c28f28640 1 --2- 172.21.15.52:0/2580345642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a16fa948e0 0x55a16fa9e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.063+0000 7f9c28f28640 1 -- 172.21.15.52:0/2580345642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a16f7d26b0 con 0x55a16fa99a50 2024-04-02T13:53:21.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.063+0000 7f9c28f28640 1 --2- 172.21.15.52:0/2580345642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa99a50 0x55a16faa5a00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400c3f0 tx=0x7f9c14014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36547e6cf11f0d89 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.063+0000 7f9c0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:53:21.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.063+0000 7f9c0cff9640 1 -- 172.21.15.52:0/2580345642 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1400d760 con 0x55a16fa99a50 2024-04-02T13:53:21.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.063+0000 7f9c0cff9640 1 -- 172.21.15.52:0/2580345642 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f9c1400d900 con 0x55a16fa99a50 2024-04-02T13:53:21.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.063+0000 7f9c2e571740 1 -- 172.21.15.52:0/2580345642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c14005500 con 0x55a16fa99a50 2024-04-02T13:53:21.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.063+0000 7f9c0cff9640 1 -- 172.21.15.52:0/2580345642 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9c1400db00 con 0x55a16fa99a50 2024-04-02T13:53:21.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.063+0000 7f9c2e571740 10 client.15474 fetch_fsmap learned FSMap version 27 2024-04-02T13:53:21.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.063+0000 7f9c2e571740 1 -- 172.21.15.52:0/2580345642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a16faa8da0 con 0x55a16fa99a50 2024-04-02T13:53:21.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.064+0000 7f9c0cff9640 1 -- 172.21.15.52:0/2580345642 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c14005500 con 0x55a16fa99a50 2024-04-02T13:53:21.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.064+0000 7f9c0cff9640 1 -- 172.21.15.52:0/2580345642 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9c1400fc30 con 0x55a16fa99a50 2024-04-02T13:53:21.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.064+0000 7f9c2e571740 10 client.15474 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:53:21.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.064+0000 7f9c2e571740 10 client.15474 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-02T13:53:21.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.064+0000 7f9c2e571740 20 client.15474 populate_metadata read hostname 'smithi052' 2024-04-02T13:53:21.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.064+0000 7f9c2e571740 1 --2- 172.21.15.52:0/2580345642 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55a16faab2b0 0x55a16faac720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.064+0000 7f9c2e571740 4 client.15474 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:53:21.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.064+0000 7f9c2e571740 1 -- 172.21.15.52:0/2580345642 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a16f99bcf0 con 0x55a16faab2b0 2024-04-02T13:53:21.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.064+0000 7f9c29729640 1 --2- 172.21.15.52:0/2580345642 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55a16faab2b0 0x55a16faac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.065+0000 7f9c0cff9640 10 client.15474 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:53:21.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.065+0000 7f9c29729640 1 --2- 172.21.15.52:0/2580345642 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55a16faab2b0 0x55a16faac720 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c88d77305ce56214 server_cookie=702ac3a14e79ec7c in_seq=0 out_seq=0 2024-04-02T13:53:21.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.065+0000 7f9c0cff9640 1 -- 172.21.15.52:0/2580345642 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a16f99bcf0 con 0x55a16faab2b0 2024-04-02T13:53:21.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.065+0000 7f9c0cff9640 10 client.15474 handle_command_reply: tid=0 2024-04-02T13:53:21.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.065+0000 7f9c2e571740 1 client.15474 shutdown 2024-04-02T13:53:21.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.065+0000 7f9c2e571740 1 -- 172.21.15.52:0/2580345642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa99a50 msgr2=0x55a16faa5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.066+0000 7f9c2e571740 1 --2- 172.21.15.52:0/2580345642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa99a50 0x55a16faa5a00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400c3f0 tx=0x7f9c14014040 comp rx=0 tx=0).stop 2024-04-02T13:53:21.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.066+0000 7f9c2e571740 1 -- 172.21.15.52:0/2580345642 shutdown_connections 2024-04-02T13:53:21.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.066+0000 7f9c2e571740 1 --2- 172.21.15.52:0/2580345642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa99a50 0x55a16faa5a00 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.066+0000 7f9c2e571740 1 --2- 172.21.15.52:0/2580345642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a16fa948e0 0x55a16fa9e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.066+0000 7f9c2e571740 1 --2- 172.21.15.52:0/2580345642 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55a16faab2b0 0x55a16faac720 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.066+0000 7f9c2e571740 1 --2- 172.21.15.52:0/2580345642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a16fa93ef0 0x55a16fa9e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.066+0000 7f9c2e571740 1 -- 172.21.15.52:0/2580345642 >> 172.21.15.52:0/2580345642 conn(0x55a16fa8b050 msgr2=0x55a16fa8a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.066+0000 7f9c0cff9640 0 client.15474 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:53:21.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.066+0000 7f9c2e571740 1 -- 172.21.15.52:0/2580345642 shutdown_connections 2024-04-02T13:53:21.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.066+0000 7f9c2e571740 1 -- 172.21.15.52:0/2580345642 wait complete. 2024-04-02T13:53:21.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.066+0000 7f9c2e571740 20 client.15474 trim_cache size 0 max 16384 2024-04-02T13:53:21.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.084+0000 7f9c2e571740 1 Processor -- start 2024-04-02T13:53:21.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.084+0000 7f9c2e571740 1 -- start start 2024-04-02T13:53:21.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.084+0000 7f9c2e571740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a16fa93ef0 0x55a16faaa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.084+0000 7f9c2e571740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa948e0 0x55a16faa5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.084+0000 7f9c2e571740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a16fa99a50 0x55a16fa8b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.084+0000 7f9c2e571740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a16faaaa60 con 0x55a16fa948e0 2024-04-02T13:53:21.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.084+0000 7f9c2e571740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a16faab130 con 0x55a16fa99a50 2024-04-02T13:53:21.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.084+0000 7f9c2e571740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a16faab800 con 0x55a16fa93ef0 2024-04-02T13:53:21.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.084+0000 7f9c29729640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa948e0 0x55a16faa5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.084+0000 7f9c29729640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa948e0 0x55a16faa5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41218/0 (socket says 172.21.15.52:41218) 2024-04-02T13:53:21.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.084+0000 7f9c29729640 1 -- 172.21.15.52:0/1855589980 learned_addr learned my addr 172.21.15.52:0/1855589980 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:21.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.085+0000 7f9c28f28640 1 --2- 172.21.15.52:0/1855589980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a16fa99a50 0x55a16fa8b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.085+0000 7f9c29729640 1 -- 172.21.15.52:0/1855589980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a16fa93ef0 msgr2=0x55a16faaa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.085+0000 7f9c29729640 1 --2- 172.21.15.52:0/1855589980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a16fa93ef0 0x55a16faaa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.085+0000 7f9c29729640 1 -- 172.21.15.52:0/1855589980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a16fa99a50 msgr2=0x55a16fa8b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.085+0000 7f9c29729640 1 --2- 172.21.15.52:0/1855589980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a16fa99a50 0x55a16fa8b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.085+0000 7f9c29729640 1 -- 172.21.15.52:0/1855589980 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a16faaca50 con 0x55a16fa948e0 2024-04-02T13:53:21.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.085+0000 7f9c29729640 1 --2- 172.21.15.52:0/1855589980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa948e0 0x55a16faa5920 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9c24101200 tx=0x7f9c20005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292ff2e67128d7ad server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.085+0000 7f9c0effd640 1 -- 172.21.15.52:0/1855589980 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2001d360 con 0x55a16fa948e0 2024-04-02T13:53:21.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.085+0000 7f9c0effd640 1 -- 172.21.15.52:0/1855589980 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c2000ee40 con 0x55a16fa948e0 2024-04-02T13:53:21.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.085+0000 7f9c0effd640 1 -- 172.21.15.52:0/1855589980 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c20019410 con 0x55a16fa948e0 2024-04-02T13:53:21.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.086+0000 7f9c2e571740 1 -- 172.21.15.52:0/1855589980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa948e0 msgr2=0x55a16faa5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.086+0000 7f9c2e571740 1 --2- 172.21.15.52:0/1855589980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa948e0 0x55a16faa5920 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9c24101200 tx=0x7f9c20005f90 comp rx=0 tx=0).stop 2024-04-02T13:53:21.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.086+0000 7f9c2e571740 1 -- 172.21.15.52:0/1855589980 shutdown_connections 2024-04-02T13:53:21.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.086+0000 7f9c2e571740 1 --2- 172.21.15.52:0/1855589980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a16fa99a50 0x55a16fa8b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.086+0000 7f9c2e571740 1 --2- 172.21.15.52:0/1855589980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa948e0 0x55a16faa5920 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.086+0000 7f9c2e571740 1 --2- 172.21.15.52:0/1855589980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a16fa93ef0 0x55a16faaa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.086+0000 7f9c2e571740 1 -- 172.21.15.52:0/1855589980 >> 172.21.15.52:0/1855589980 conn(0x55a16fa8b050 msgr2=0x55a16fa9c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.086+0000 7f9c2e571740 1 -- 172.21.15.52:0/1855589980 shutdown_connections 2024-04-02T13:53:21.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.086+0000 7f9c2e571740 1 -- 172.21.15.52:0/1855589980 wait complete. 2024-04-02T13:53:21.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.086+0000 7f9c2e571740 1 -- start start 2024-04-02T13:53:21.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.086+0000 7f9c2e571740 1 Processor -- start 2024-04-02T13:53:21.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.086+0000 7f9c2e571740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a16fa93ef0 0x55a16faa3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.086+0000 7f9c2e571740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa948e0 0x55a16fb0d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.086+0000 7f9c2e571740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a16fa99a50 0x55a16fb0f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.086+0000 7f9c2e571740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a16faac5a0 con 0x55a16fa948e0 2024-04-02T13:53:21.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.086+0000 7f9c2e571740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a16faabed0 con 0x55a16fa93ef0 2024-04-02T13:53:21.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.086+0000 7f9c2e571740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a16faacc70 con 0x55a16fa99a50 2024-04-02T13:53:21.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.087+0000 7f9c2e571740 10 client.0 _ll_register_callbacks cb 0x7f9c1400cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:21.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.087+0000 7f9c29729640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa948e0 0x55a16fb0d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.087+0000 7f9c29729640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa948e0 0x55a16fb0d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41230/0 (socket says 172.21.15.52:41230) 2024-04-02T13:53:21.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.087+0000 7f9c29729640 1 -- 172.21.15.52:0/1752348005 learned_addr learned my addr 172.21.15.52:0/1752348005 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:21.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.087+0000 7f9c28f28640 1 --2- 172.21.15.52:0/1752348005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a16fa99a50 0x55a16fb0f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.087+0000 7f9c28f28640 1 --2- 172.21.15.52:0/1752348005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a16fa93ef0 0x55a16faa3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.087+0000 7f9c29729640 1 -- 172.21.15.52:0/1752348005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a16fa99a50 msgr2=0x55a16fb0f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.087+0000 7f9c29729640 1 --2- 172.21.15.52:0/1752348005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a16fa99a50 0x55a16fb0f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.087+0000 7f9c29729640 1 -- 172.21.15.52:0/1752348005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a16fa93ef0 msgr2=0x55a16faa3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.087+0000 7f9c29729640 1 --2- 172.21.15.52:0/1752348005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a16fa93ef0 0x55a16faa3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.087+0000 7f9c29729640 1 -- 172.21.15.52:0/1752348005 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a16fb0fcd0 con 0x55a16fa948e0 2024-04-02T13:53:21.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.087+0000 7f9c28f28640 1 --2- 172.21.15.52:0/1752348005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a16fa93ef0 0x55a16faa3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:53:21.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.087+0000 7f9c29729640 1 --2- 172.21.15.52:0/1752348005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa948e0 0x55a16fb0d240 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9c20010010 tx=0x7f9c2001af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec099a3524b648c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.087+0000 7f9c0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:53:21.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.087+0000 7f9c0cff9640 1 -- 172.21.15.52:0/1752348005 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2001a7f0 con 0x55a16fa948e0 2024-04-02T13:53:21.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.087+0000 7f9c0cff9640 1 -- 172.21.15.52:0/1752348005 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f9c20004050 con 0x55a16fa948e0 2024-04-02T13:53:21.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.087+0000 7f9c2e571740 1 -- 172.21.15.52:0/1752348005 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a16fa015a0 con 0x55a16fa948e0 2024-04-02T13:53:21.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.087+0000 7f9c0cff9640 1 -- 172.21.15.52:0/1752348005 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9c20035050 con 0x55a16fa948e0 2024-04-02T13:53:21.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.087+0000 7f9c2e571740 10 client.15480 fetch_fsmap learned FSMap version 27 2024-04-02T13:53:21.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.087+0000 7f9c2e571740 1 -- 172.21.15.52:0/1752348005 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a16fb10f60 con 0x55a16fa948e0 2024-04-02T13:53:21.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.088+0000 7f9c0cff9640 1 -- 172.21.15.52:0/1752348005 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a16fa015a0 con 0x55a16fa948e0 2024-04-02T13:53:21.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.088+0000 7f9c0cff9640 1 -- 172.21.15.52:0/1752348005 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9c2002e060 con 0x55a16fa948e0 2024-04-02T13:53:21.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.088+0000 7f9c2e571740 10 client.15480 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:53:21.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.088+0000 7f9c2e571740 10 client.15480 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-02T13:53:21.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.088+0000 7f9c2e571740 20 client.15480 populate_metadata read hostname 'smithi052' 2024-04-02T13:53:21.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.088+0000 7f9c2e571740 1 --2- 172.21.15.52:0/1752348005 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55a16fb15a00 0x55a16fb14710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.088+0000 7f9c2e571740 4 client.15480 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:53:21.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.088+0000 7f9c2e571740 1 -- 172.21.15.52:0/1752348005 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "status"}) -- 0x55a16f9fb8f0 con 0x55a16fb15a00 2024-04-02T13:53:21.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.088+0000 7f9c28f28640 1 --2- 172.21.15.52:0/1752348005 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55a16fb15a00 0x55a16fb14710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.088+0000 7f9c0cff9640 10 client.15480 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:53:21.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.088+0000 7f9c28f28640 1 --2- 172.21.15.52:0/1752348005 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55a16fb15a00 0x55a16fb14710 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3025d66947a8abaf server_cookie=beaf0d55326f6c00 in_seq=0 out_seq=0 2024-04-02T13:53:21.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.089+0000 7f9c0cff9640 1 -- 172.21.15.52:0/1752348005 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a16f9fb8f0 con 0x55a16fb15a00 2024-04-02T13:53:21.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.089+0000 7f9c0cff9640 10 client.15480 handle_command_reply: tid=0 2024-04-02T13:53:21.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.089+0000 7f9c2e571740 1 client.15480 shutdown 2024-04-02T13:53:21.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.089+0000 7f9c2e571740 1 -- 172.21.15.52:0/1752348005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa948e0 msgr2=0x55a16fb0d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.089+0000 7f9c2e571740 1 --2- 172.21.15.52:0/1752348005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa948e0 0x55a16fb0d240 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9c20010010 tx=0x7f9c2001af90 comp rx=0 tx=0).stop 2024-04-02T13:53:21.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.089+0000 7f9c2e571740 1 -- 172.21.15.52:0/1752348005 shutdown_connections 2024-04-02T13:53:21.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.089+0000 7f9c2e571740 1 --2- 172.21.15.52:0/1752348005 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55a16fb15a00 0x55a16fb14710 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.089+0000 7f9c2e571740 1 --2- 172.21.15.52:0/1752348005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a16fa99a50 0x55a16fb0f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.089+0000 7f9c2e571740 1 --2- 172.21.15.52:0/1752348005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a16fa948e0 0x55a16fb0d240 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.089+0000 7f9c0cff9640 0 client.15480 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:53:21.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.089+0000 7f9c2e571740 1 --2- 172.21.15.52:0/1752348005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a16fa93ef0 0x55a16faa3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.089+0000 7f9c2e571740 1 -- 172.21.15.52:0/1752348005 >> 172.21.15.52:0/1752348005 conn(0x55a16fa8b050 msgr2=0x55a16faaa260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.089+0000 7f9c2e571740 1 -- 172.21.15.52:0/1752348005 shutdown_connections 2024-04-02T13:53:21.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.089+0000 7f9c2e571740 1 -- 172.21.15.52:0/1752348005 wait complete. 2024-04-02T13:53:21.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.089+0000 7f9c2e571740 20 client.15480 trim_cache size 0 max 16384 2024-04-02T13:53:21.100 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:53:21.100 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "bfa00e6e-f0f6-11ee-b647-cb9ed24678a4", 2024-04-02T13:53:21.100 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 1, 2024-04-02T13:53:21.100 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24490, 2024-04-02T13:53:21.101 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-04-02T13:53:21.101 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-04-02T13:53:21.101 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-04-02T13:53:21.101 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 115.76756826800001, 2024-04-02T13:53:21.101 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 27, 2024-04-02T13:53:21.101 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 92, 2024-04-02T13:53:21.101 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:53:21.101 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 154.48207151899999 2024-04-02T13:53:21.101 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:53:21.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.090+0000 7f9c2b98d640 1 -- 172.21.15.52:0/2545503353 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f9c100a3bb0 msgr2=0x7f9c100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.090+0000 7f9c2b98d640 1 --2- 172.21.15.52:0/2545503353 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f9c100a3bb0 0x7f9c100a6070 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c0efe50 tx=0x7f9c1c08df60 comp rx=0 tx=0).stop 2024-04-02T13:53:21.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.090+0000 7f9c2b98d640 1 -- 172.21.15.52:0/2545503353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c24101d60 msgr2=0x7f9c2419f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.090+0000 7f9c2b98d640 1 --2- 172.21.15.52:0/2545503353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c24101d60 0x7f9c2419f3a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c037040 tx=0x7f9c1c032f20 comp rx=0 tx=0).stop 2024-04-02T13:53:21.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.090+0000 7f9c2b98d640 1 -- 172.21.15.52:0/2545503353 shutdown_connections 2024-04-02T13:53:21.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.090+0000 7f9c2b98d640 1 --2- 172.21.15.52:0/2545503353 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f9c100a3bb0 0x7f9c100a6070 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.090+0000 7f9c2b98d640 1 --2- 172.21.15.52:0/2545503353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c24101d60 0x7f9c2419f3a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.090+0000 7f9c2b98d640 1 --2- 172.21.15.52:0/2545503353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c241013a0 0x7f9c24198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.090+0000 7f9c2b98d640 1 --2- 172.21.15.52:0/2545503353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c241001b0 0x7f9c24197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.090+0000 7f9c2b98d640 1 -- 172.21.15.52:0/2545503353 >> 172.21.15.52:0/2545503353 conn(0x7f9c240fb700 msgr2=0x7f9c241068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.090+0000 7f9c2b98d640 1 -- 172.21.15.52:0/2545503353 shutdown_connections 2024-04-02T13:53:21.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.090+0000 7f9c2b98d640 1 -- 172.21.15.52:0/2545503353 wait complete. 2024-04-02T13:53:21.104 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-02T13:53:21.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.204+0000 7f3aa820d640 1 -- 172.21.15.52:0/3084248075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3aa01056b0 msgr2=0x7f3aa0105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.204+0000 7f3aa820d640 1 --2- 172.21.15.52:0/3084248075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3aa01056b0 0x7f3aa0105b30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009ca0 tx=0x7f3a9802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.204+0000 7f3aa820d640 1 -- 172.21.15.52:0/3084248075 shutdown_connections 2024-04-02T13:53:21.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.204+0000 7f3aa820d640 1 --2- 172.21.15.52:0/3084248075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3aa01056b0 0x7f3aa0105b30 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.204+0000 7f3aa820d640 1 --2- 172.21.15.52:0/3084248075 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3aa0102d00 0x7f3aa0105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.204+0000 7f3aa820d640 1 --2- 172.21.15.52:0/3084248075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3aa0100390 0x7f3aa01027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.204+0000 7f3aa820d640 1 -- 172.21.15.52:0/3084248075 >> 172.21.15.52:0/3084248075 conn(0x7f3aa00fbfb0 msgr2=0x7f3aa00fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.204+0000 7f3aa820d640 1 -- 172.21.15.52:0/3084248075 shutdown_connections 2024-04-02T13:53:21.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.204+0000 7f3aa820d640 1 -- 172.21.15.52:0/3084248075 wait complete. 2024-04-02T13:53:21.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.205+0000 7f3aa820d640 1 Processor -- start 2024-04-02T13:53:21.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.205+0000 7f3aa820d640 1 -- start start 2024-04-02T13:53:21.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.205+0000 7f3aa820d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3aa0100390 0x7f3aa019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.205+0000 7f3aa820d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3aa0102d00 0x7f3aa019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.205+0000 7f3aa820d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3aa01056b0 0x7f3aa01a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.205+0000 7f3aa820d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa0113e00 con 0x7f3aa01056b0 2024-04-02T13:53:21.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.205+0000 7f3aa820d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa0113c80 con 0x7f3aa0100390 2024-04-02T13:53:21.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.205+0000 7f3aa820d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa0113f80 con 0x7f3aa0102d00 2024-04-02T13:53:21.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.205+0000 7f3aa67aa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3aa01056b0 0x7f3aa01a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.205+0000 7f3aa67aa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3aa01056b0 0x7f3aa01a59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41254/0 (socket says 172.21.15.52:41254) 2024-04-02T13:53:21.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.205+0000 7f3aa67aa640 1 -- 172.21.15.52:0/323300990 learned_addr learned my addr 172.21.15.52:0/323300990 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:21.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.205+0000 7f3aa5fa9640 1 --2- 172.21.15.52:0/323300990 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3aa0100390 0x7f3aa019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.205+0000 7f3aa67aa640 1 -- 172.21.15.52:0/323300990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3aa0102d00 msgr2=0x7f3aa019e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.205+0000 7f3aa67aa640 1 --2- 172.21.15.52:0/323300990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3aa0102d00 0x7f3aa019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.205+0000 7f3aa67aa640 1 -- 172.21.15.52:0/323300990 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3aa0100390 msgr2=0x7f3aa019e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.205+0000 7f3aa67aa640 1 --2- 172.21.15.52:0/323300990 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3aa0100390 0x7f3aa019e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.205+0000 7f3aa67aa640 1 -- 172.21.15.52:0/323300990 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa01a6030 con 0x7f3aa01056b0 2024-04-02T13:53:21.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.206+0000 7f3aa67aa640 1 --2- 172.21.15.52:0/323300990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3aa01056b0 0x7f3aa01a59c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3a9802c700 tx=0x7f3a98032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.206+0000 7f3a96ffd640 1 -- 172.21.15.52:0/323300990 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98039040 con 0x7f3aa01056b0 2024-04-02T13:53:21.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.206+0000 7f3aa820d640 1 -- 172.21.15.52:0/323300990 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa01a62c0 con 0x7f3aa01056b0 2024-04-02T13:53:21.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.206+0000 7f3a96ffd640 1 -- 172.21.15.52:0/323300990 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a98044070 con 0x7f3aa01056b0 2024-04-02T13:53:21.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.206+0000 7f3a96ffd640 1 -- 172.21.15.52:0/323300990 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a980042a0 con 0x7f3aa01056b0 2024-04-02T13:53:21.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.206+0000 7f3aa820d640 1 -- 172.21.15.52:0/323300990 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa01a67b0 con 0x7f3aa01056b0 2024-04-02T13:53:21.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.208+0000 7f3a96ffd640 1 -- 172.21.15.52:0/323300990 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3a98003970 con 0x7f3aa01056b0 2024-04-02T13:53:21.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.208+0000 7f3aaadf7740 1 Processor -- start 2024-04-02T13:53:21.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.208+0000 7f3aaadf7740 1 -- start start 2024-04-02T13:53:21.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.209+0000 7f3aaadf7740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e6ae0 0x5589a12e6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.209+0000 7f3aaadf7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589a12e74d0 0x5589a12ebfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.209+0000 7f3aaadf7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589a12ec640 0x5589a12eead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.209+0000 7f3aaadf7740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5589a124b060 con 0x5589a12e6ae0 2024-04-02T13:53:21.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.209+0000 7f3aaadf7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5589a11f3650 con 0x5589a12ec640 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.209+0000 7f3aaadf7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5589a1247010 con 0x5589a12e74d0 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.209+0000 7f3aa57a8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e6ae0 0x5589a12e6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.209+0000 7f3aa57a8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e6ae0 0x5589a12e6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41266/0 (socket says 172.21.15.52:41266) 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.209+0000 7f3aa57a8640 1 -- 172.21.15.52:0/3600547262 learned_addr learned my addr 172.21.15.52:0/3600547262 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.209+0000 7f3a96ffd640 1 --2- 172.21.15.52:0/323300990 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3a8c0a3bb0 0x7f3a8c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.209+0000 7f3aa5fa9640 1 --2- 172.21.15.52:0/3600547262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589a12e74d0 0x5589a12ebfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.209+0000 7f3aa57a8640 1 -- 172.21.15.52:0/3600547262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589a12e74d0 msgr2=0x5589a12ebfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.209+0000 7f3aa57a8640 1 --2- 172.21.15.52:0/3600547262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589a12e74d0 0x5589a12ebfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.209+0000 7f3aa57a8640 1 -- 172.21.15.52:0/3600547262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589a12ec640 msgr2=0x5589a12eead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.209+0000 7f3aa57a8640 1 --2- 172.21.15.52:0/3600547262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589a12ec640 0x5589a12eead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.209+0000 7f3aa57a8640 1 -- 172.21.15.52:0/3600547262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589a0f5f2d0 con 0x5589a12e6ae0 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.209+0000 7f3aa67aa640 1 --2- 172.21.15.52:0/323300990 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3a8c0a3bb0 0x7f3a8c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.209+0000 7f3a96ffd640 1 -- 172.21.15.52:0/323300990 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f3a980f2da0 con 0x7f3aa01056b0 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.209+0000 7f3aa57a8640 1 --2- 172.21.15.52:0/3600547262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e6ae0 0x5589a12e6f00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3a900098f0 tx=0x7f3a90009db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400a7fe7627d760f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.209+0000 7f3a8bfff640 1 -- 172.21.15.52:0/3600547262 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9000f070 con 0x5589a12e6ae0 2024-04-02T13:53:21.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.209+0000 7f3a8bfff640 1 -- 172.21.15.52:0/3600547262 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a90017070 con 0x5589a12e6ae0 2024-04-02T13:53:21.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.210+0000 7f3a8bfff640 1 -- 172.21.15.52:0/3600547262 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90011350 con 0x5589a12e6ae0 2024-04-02T13:53:21.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.210+0000 7f3aaadf7740 1 -- 172.21.15.52:0/3600547262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e6ae0 msgr2=0x5589a12e6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.210+0000 7f3aa67aa640 1 --2- 172.21.15.52:0/323300990 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3a8c0a3bb0 0x7f3a8c0a6070 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3a980bb210 tx=0x7f3a980b4290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.210+0000 7f3aaadf7740 1 --2- 172.21.15.52:0/3600547262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e6ae0 0x5589a12e6f00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3a900098f0 tx=0x7f3a90009db0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.210+0000 7f3aaadf7740 1 -- 172.21.15.52:0/3600547262 shutdown_connections 2024-04-02T13:53:21.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.210+0000 7f3aaadf7740 1 --2- 172.21.15.52:0/3600547262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589a12ec640 0x5589a12eead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.210+0000 7f3aaadf7740 1 --2- 172.21.15.52:0/3600547262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589a12e74d0 0x5589a12ebfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.210+0000 7f3aaadf7740 1 --2- 172.21.15.52:0/3600547262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e6ae0 0x5589a12e6f00 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.210+0000 7f3aaadf7740 1 -- 172.21.15.52:0/3600547262 >> 172.21.15.52:0/3600547262 conn(0x5589a12e2050 msgr2=0x5589a12e2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.210+0000 7f3aaadf7740 1 -- 172.21.15.52:0/3600547262 shutdown_connections 2024-04-02T13:53:21.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.210+0000 7f3aaadf7740 1 -- 172.21.15.52:0/3600547262 wait complete. 2024-04-02T13:53:21.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.211+0000 7f3aaadf7740 1 -- start start 2024-04-02T13:53:21.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.211+0000 7f3aaadf7740 1 Processor -- start 2024-04-02T13:53:21.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.211+0000 7f3aaadf7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589a12e6ae0 0x5589a12f1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.211+0000 7f3aaadf7740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e74d0 0x5589a12f15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.211+0000 7f3aaadf7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589a12ec640 0x5589a12f85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.211+0000 7f3aaadf7740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5589a12ef5f0 con 0x5589a12e74d0 2024-04-02T13:53:21.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.211+0000 7f3aaadf7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5589a1206a10 con 0x5589a12ec640 2024-04-02T13:53:21.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.211+0000 7f3aaadf7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5589a12ef770 con 0x5589a12e6ae0 2024-04-02T13:53:21.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.211+0000 7f3aaadf7740 10 client.0 _ll_register_callbacks cb 0x5589a12430b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:21.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.211+0000 7f3aa5fa9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e74d0 0x5589a12f15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.211+0000 7f3aa5fa9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e74d0 0x5589a12f15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41276/0 (socket says 172.21.15.52:41276) 2024-04-02T13:53:21.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.211+0000 7f3aa5fa9640 1 -- 172.21.15.52:0/3013950793 learned_addr learned my addr 172.21.15.52:0/3013950793 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:21.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.211+0000 7f3aa57a8640 1 --2- 172.21.15.52:0/3013950793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589a12e6ae0 0x5589a12f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.211+0000 7f3aa57a8640 1 --2- 172.21.15.52:0/3013950793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589a12ec640 0x5589a12f85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.211+0000 7f3aa5fa9640 1 -- 172.21.15.52:0/3013950793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589a12e6ae0 msgr2=0x5589a12f1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.211+0000 7f3aa5fa9640 1 --2- 172.21.15.52:0/3013950793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589a12e6ae0 0x5589a12f1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.211+0000 7f3aa5fa9640 1 -- 172.21.15.52:0/3013950793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589a12ec640 msgr2=0x5589a12f85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.211+0000 7f3aa5fa9640 1 --2- 172.21.15.52:0/3013950793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589a12ec640 0x5589a12f85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.211+0000 7f3aa5fa9640 1 -- 172.21.15.52:0/3013950793 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589a10296b0 con 0x5589a12e74d0 2024-04-02T13:53:21.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.211+0000 7f3aa57a8640 1 --2- 172.21.15.52:0/3013950793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589a12e6ae0 0x5589a12f1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:21.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.212+0000 7f3aa5fa9640 1 --2- 172.21.15.52:0/3013950793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e74d0 0x5589a12f15a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c010420 tx=0x7f3a9c0108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73414853b8e467c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.212+0000 7f3a897fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:53:21.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.212+0000 7f3a897fa640 1 -- 172.21.15.52:0/3013950793 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c016070 con 0x5589a12e74d0 2024-04-02T13:53:21.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.212+0000 7f3a897fa640 1 -- 172.21.15.52:0/3013950793 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f3a9c005220 con 0x5589a12e74d0 2024-04-02T13:53:21.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.212+0000 7f3aaadf7740 1 -- 172.21.15.52:0/3013950793 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a90017070 con 0x5589a12e74d0 2024-04-02T13:53:21.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.212+0000 7f3a897fa640 1 -- 172.21.15.52:0/3013950793 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3a9c005420 con 0x5589a12e74d0 2024-04-02T13:53:21.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.212+0000 7f3aaadf7740 10 client.15492 fetch_fsmap learned FSMap version 27 2024-04-02T13:53:21.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.212+0000 7f3aaadf7740 1 -- 172.21.15.52:0/3013950793 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5589a12fba20 con 0x5589a12e74d0 2024-04-02T13:53:21.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.212+0000 7f3a897fa640 1 -- 172.21.15.52:0/3013950793 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a90017070 con 0x5589a12e74d0 2024-04-02T13:53:21.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.212+0000 7f3a897fa640 1 -- 172.21.15.52:0/3013950793 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3a9c00f040 con 0x5589a12e74d0 2024-04-02T13:53:21.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.212+0000 7f3aaadf7740 10 client.15492 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:53:21.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.212+0000 7f3aaadf7740 10 client.15492 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T13:53:21.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.212+0000 7f3aaadf7740 20 client.15492 populate_metadata read hostname 'smithi052' 2024-04-02T13:53:21.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.213+0000 7f3aaadf7740 1 --2- 172.21.15.52:0/3013950793 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5589a12fe110 0x5589a12ff580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.213+0000 7f3aaadf7740 4 client.15492 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:53:21.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.213+0000 7f3aaadf7740 1 -- 172.21.15.52:0/3013950793 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589a11f2cf0 con 0x5589a12fe110 2024-04-02T13:53:21.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.213+0000 7f3aa57a8640 1 --2- 172.21.15.52:0/3013950793 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5589a12fe110 0x5589a12ff580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.213+0000 7f3aa57a8640 1 --2- 172.21.15.52:0/3013950793 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5589a12fe110 0x5589a12ff580 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6c9bde6d2818695 server_cookie=b9a456a041d7f782 in_seq=0 out_seq=0 2024-04-02T13:53:21.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.213+0000 7f3a897fa640 10 client.15492 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:53:21.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.214+0000 7f3a897fa640 1 -- 172.21.15.52:0/3013950793 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5589a11f2cf0 con 0x5589a12fe110 2024-04-02T13:53:21.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.214+0000 7f3a897fa640 10 client.15492 handle_command_reply: tid=0 2024-04-02T13:53:21.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.214+0000 7f3aaadf7740 1 client.15492 shutdown 2024-04-02T13:53:21.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.214+0000 7f3aaadf7740 1 -- 172.21.15.52:0/3013950793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e74d0 msgr2=0x5589a12f15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.214+0000 7f3aaadf7740 1 --2- 172.21.15.52:0/3013950793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e74d0 0x5589a12f15a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c010420 tx=0x7f3a9c0108e0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.214+0000 7f3aaadf7740 1 -- 172.21.15.52:0/3013950793 shutdown_connections 2024-04-02T13:53:21.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.214+0000 7f3aaadf7740 1 --2- 172.21.15.52:0/3013950793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589a12ec640 0x5589a12f85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.214+0000 7f3aaadf7740 1 --2- 172.21.15.52:0/3013950793 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5589a12fe110 0x5589a12ff580 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.214+0000 7f3aaadf7740 1 --2- 172.21.15.52:0/3013950793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e74d0 0x5589a12f15a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.214+0000 7f3aaadf7740 1 --2- 172.21.15.52:0/3013950793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589a12e6ae0 0x5589a12f1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.214+0000 7f3a897fa640 0 client.15492 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:53:21.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.214+0000 7f3aaadf7740 1 -- 172.21.15.52:0/3013950793 >> 172.21.15.52:0/3013950793 conn(0x5589a12e2050 msgr2=0x5589a12e1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.214+0000 7f3aaadf7740 1 -- 172.21.15.52:0/3013950793 shutdown_connections 2024-04-02T13:53:21.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.214+0000 7f3aaadf7740 1 -- 172.21.15.52:0/3013950793 wait complete. 2024-04-02T13:53:21.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.214+0000 7f3aaadf7740 20 client.15492 trim_cache size 0 max 16384 2024-04-02T13:53:21.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.232+0000 7f3aaadf7740 1 Processor -- start 2024-04-02T13:53:21.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.232+0000 7f3aaadf7740 1 -- start start 2024-04-02T13:53:21.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.232+0000 7f3aaadf7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589a12e6ae0 0x5589a12ec000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.232+0000 7f3aaadf7740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e74d0 0x5589a12f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.233+0000 7f3aaadf7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589a12ec640 0x5589a12e0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.233+0000 7f3aaadf7740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5589a12fd8c0 con 0x5589a12e74d0 2024-04-02T13:53:21.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.233+0000 7f3aaadf7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5589a12fdf90 con 0x5589a12e6ae0 2024-04-02T13:53:21.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.233+0000 7f3aaadf7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5589a12fe660 con 0x5589a12ec640 2024-04-02T13:53:21.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.233+0000 7f3aa5fa9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e74d0 0x5589a12f85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.233+0000 7f3aa5fa9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e74d0 0x5589a12f85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41288/0 (socket says 172.21.15.52:41288) 2024-04-02T13:53:21.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.233+0000 7f3aa5fa9640 1 -- 172.21.15.52:0/1627631402 learned_addr learned my addr 172.21.15.52:0/1627631402 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:21.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.233+0000 7f3aa57a8640 1 --2- 172.21.15.52:0/1627631402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589a12e6ae0 0x5589a12ec000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.233+0000 7f3aa5fa9640 1 -- 172.21.15.52:0/1627631402 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589a12ec640 msgr2=0x5589a12e0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.233+0000 7f3aa5fa9640 1 --2- 172.21.15.52:0/1627631402 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589a12ec640 0x5589a12e0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.233+0000 7f3aa5fa9640 1 -- 172.21.15.52:0/1627631402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589a12e6ae0 msgr2=0x5589a12ec000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.233+0000 7f3aa5fa9640 1 --2- 172.21.15.52:0/1627631402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589a12e6ae0 0x5589a12ec000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.233+0000 7f3aa5fa9640 1 -- 172.21.15.52:0/1627631402 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589a12e78b0 con 0x5589a12e74d0 2024-04-02T13:53:21.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.233+0000 7f3aa57a8640 1 --2- 172.21.15.52:0/1627631402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589a12e6ae0 0x5589a12ec000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:53:21.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.233+0000 7f3aa5fa9640 1 --2- 172.21.15.52:0/1627631402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e74d0 0x5589a12f85a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009cb0 tx=0x7f3a9c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10de7e89bb114333 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.233+0000 7f3a8b7fe640 1 -- 172.21.15.52:0/1627631402 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c02b020 con 0x5589a12e74d0 2024-04-02T13:53:21.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.233+0000 7f3a8b7fe640 1 -- 172.21.15.52:0/1627631402 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a9c012960 con 0x5589a12e74d0 2024-04-02T13:53:21.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.233+0000 7f3a8b7fe640 1 -- 172.21.15.52:0/1627631402 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c016040 con 0x5589a12e74d0 2024-04-02T13:53:21.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.234+0000 7f3aaadf7740 1 -- 172.21.15.52:0/1627631402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e74d0 msgr2=0x5589a12f85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.234+0000 7f3aaadf7740 1 --2- 172.21.15.52:0/1627631402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e74d0 0x5589a12f85a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009cb0 tx=0x7f3a9c005850 comp rx=0 tx=0).stop 2024-04-02T13:53:21.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.234+0000 7f3aaadf7740 1 -- 172.21.15.52:0/1627631402 shutdown_connections 2024-04-02T13:53:21.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.234+0000 7f3aaadf7740 1 --2- 172.21.15.52:0/1627631402 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589a12ec640 0x5589a12e0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.234+0000 7f3aaadf7740 1 --2- 172.21.15.52:0/1627631402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e74d0 0x5589a12f85a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.234+0000 7f3aaadf7740 1 --2- 172.21.15.52:0/1627631402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589a12e6ae0 0x5589a12ec000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.234+0000 7f3aaadf7740 1 -- 172.21.15.52:0/1627631402 >> 172.21.15.52:0/1627631402 conn(0x5589a12e2050 msgr2=0x5589a12e8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.234+0000 7f3aaadf7740 1 -- 172.21.15.52:0/1627631402 shutdown_connections 2024-04-02T13:53:21.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.234+0000 7f3aaadf7740 1 -- 172.21.15.52:0/1627631402 wait complete. 2024-04-02T13:53:21.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.234+0000 7f3aaadf7740 1 -- start start 2024-04-02T13:53:21.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.234+0000 7f3aaadf7740 1 Processor -- start 2024-04-02T13:53:21.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.234+0000 7f3aaadf7740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e6ae0 0x5589a12f6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.234+0000 7f3aaadf7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589a12e74d0 0x5589a12f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.235+0000 7f3aaadf7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589a12ec640 0x5589a12f71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.235+0000 7f3aaadf7740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5589a12ff400 con 0x5589a12e6ae0 2024-04-02T13:53:21.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.235+0000 7f3aaadf7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5589a12fed30 con 0x5589a12e74d0 2024-04-02T13:53:21.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.235+0000 7f3aaadf7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5589a12ffad0 con 0x5589a12ec640 2024-04-02T13:53:21.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.235+0000 7f3aaadf7740 10 client.0 _ll_register_callbacks cb 0x7f3a9c005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:21.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.235+0000 7f3aa57a8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e6ae0 0x5589a12f6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.235+0000 7f3aa57a8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e6ae0 0x5589a12f6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41298/0 (socket says 172.21.15.52:41298) 2024-04-02T13:53:21.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.235+0000 7f3aa57a8640 1 -- 172.21.15.52:0/3460251378 learned_addr learned my addr 172.21.15.52:0/3460251378 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:21.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.235+0000 7f3aa57a8640 1 --2- 172.21.15.52:0/3460251378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589a12ec640 0x5589a12f71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.235+0000 7f3aa5fa9640 1 --2- 172.21.15.52:0/3460251378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589a12e74d0 0x5589a12f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.235+0000 7f3aa57a8640 1 -- 172.21.15.52:0/3460251378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589a12ec640 msgr2=0x5589a12f71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.235+0000 7f3aa57a8640 1 --2- 172.21.15.52:0/3460251378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589a12ec640 0x5589a12f71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.235+0000 7f3aa57a8640 1 -- 172.21.15.52:0/3460251378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589a12e74d0 msgr2=0x5589a12f6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.235+0000 7f3aa57a8640 1 --2- 172.21.15.52:0/3460251378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589a12e74d0 0x5589a12f6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.235+0000 7f3aa57a8640 1 -- 172.21.15.52:0/3460251378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589a12f79f0 con 0x5589a12e6ae0 2024-04-02T13:53:21.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.235+0000 7f3aa5fa9640 1 --2- 172.21.15.52:0/3460251378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589a12e74d0 0x5589a12f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:53:21.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.235+0000 7f3aa57a8640 1 --2- 172.21.15.52:0/3460251378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e6ae0 0x5589a12f6740 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3a90006bd0 tx=0x7f3a90009db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6fa7fa2f3f537 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.235+0000 7f3a897fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:53:21.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.235+0000 7f3a897fa640 1 -- 172.21.15.52:0/3460251378 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9002b020 con 0x5589a12e6ae0 2024-04-02T13:53:21.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.235+0000 7f3a897fa640 1 -- 172.21.15.52:0/3460251378 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f3a90025dc0 con 0x5589a12e6ae0 2024-04-02T13:53:21.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.235+0000 7f3aaadf7740 1 -- 172.21.15.52:0/3460251378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a9c012960 con 0x5589a12e6ae0 2024-04-02T13:53:21.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.235+0000 7f3a897fa640 1 -- 172.21.15.52:0/3460251378 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3a90036050 con 0x5589a12e6ae0 2024-04-02T13:53:21.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.235+0000 7f3aaadf7740 10 client.15498 fetch_fsmap learned FSMap version 27 2024-04-02T13:53:21.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.235+0000 7f3aaadf7740 1 -- 172.21.15.52:0/3460251378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5589a13641f0 con 0x5589a12e6ae0 2024-04-02T13:53:21.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.236+0000 7f3a897fa640 1 -- 172.21.15.52:0/3460251378 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a9c012960 con 0x5589a12e6ae0 2024-04-02T13:53:21.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.236+0000 7f3a897fa640 1 -- 172.21.15.52:0/3460251378 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3a90012570 con 0x5589a12e6ae0 2024-04-02T13:53:21.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.236+0000 7f3aaadf7740 10 client.15498 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:53:21.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.236+0000 7f3aaadf7740 10 client.15498 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T13:53:21.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.236+0000 7f3aaadf7740 20 client.15498 populate_metadata read hostname 'smithi052' 2024-04-02T13:53:21.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.236+0000 7f3aaadf7740 1 --2- 172.21.15.52:0/3460251378 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5589a1368a30 0x5589a1365540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.236+0000 7f3aaadf7740 4 client.15498 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:53:21.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.236+0000 7f3aaadf7740 1 -- 172.21.15.52:0/3460251378 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "status"}) -- 0x5589a12585a0 con 0x5589a1368a30 2024-04-02T13:53:21.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.236+0000 7f3aa5fa9640 1 --2- 172.21.15.52:0/3460251378 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5589a1368a30 0x5589a1365540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.236+0000 7f3a897fa640 10 client.15498 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:53:21.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.236+0000 7f3aa5fa9640 1 --2- 172.21.15.52:0/3460251378 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5589a1368a30 0x5589a1365540 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7e9fdbb067d669b server_cookie=72611c0cc16e8054 in_seq=0 out_seq=0 2024-04-02T13:53:21.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.237+0000 7f3a897fa640 1 -- 172.21.15.52:0/3460251378 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5589a12585a0 con 0x5589a1368a30 2024-04-02T13:53:21.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.237+0000 7f3a897fa640 10 client.15498 handle_command_reply: tid=0 2024-04-02T13:53:21.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.237+0000 7f3aaadf7740 1 client.15498 shutdown 2024-04-02T13:53:21.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.237+0000 7f3aaadf7740 1 -- 172.21.15.52:0/3460251378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e6ae0 msgr2=0x5589a12f6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.237+0000 7f3aaadf7740 1 --2- 172.21.15.52:0/3460251378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e6ae0 0x5589a12f6740 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3a90006bd0 tx=0x7f3a90009db0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.237+0000 7f3aaadf7740 1 -- 172.21.15.52:0/3460251378 shutdown_connections 2024-04-02T13:53:21.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.237+0000 7f3aaadf7740 1 --2- 172.21.15.52:0/3460251378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589a12ec640 0x5589a12f71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.237+0000 7f3aaadf7740 1 --2- 172.21.15.52:0/3460251378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589a12e74d0 0x5589a12f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.237+0000 7f3aaadf7740 1 --2- 172.21.15.52:0/3460251378 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5589a1368a30 0x5589a1365540 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.237+0000 7f3aaadf7740 1 --2- 172.21.15.52:0/3460251378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5589a12e6ae0 0x5589a12f6740 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.237+0000 7f3a897fa640 0 client.15498 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:53:21.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.237+0000 7f3aaadf7740 1 -- 172.21.15.52:0/3460251378 >> 172.21.15.52:0/3460251378 conn(0x5589a12e2050 msgr2=0x5589a12e0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.237+0000 7f3aaadf7740 1 -- 172.21.15.52:0/3460251378 shutdown_connections 2024-04-02T13:53:21.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.237+0000 7f3aaadf7740 1 -- 172.21.15.52:0/3460251378 wait complete. 2024-04-02T13:53:21.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.237+0000 7f3aaadf7740 20 client.15498 trim_cache size 0 max 16384 2024-04-02T13:53:21.253 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:53:21.253 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "bfa00e6e-f0f6-11ee-b647-cb9ed24678a4", 2024-04-02T13:53:21.253 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 0, 2024-04-02T13:53:21.253 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24503, 2024-04-02T13:53:21.253 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-04-02T13:53:21.253 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-04-02T13:53:21.254 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-04-02T13:53:21.254 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 139.913565804, 2024-04-02T13:53:21.254 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 27, 2024-04-02T13:53:21.254 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 92, 2024-04-02T13:53:21.254 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:53:21.254 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 146.01142693200001 2024-04-02T13:53:21.254 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:53:21.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.237+0000 7f3aa820d640 1 -- 172.21.15.52:0/323300990 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3a8c0a3bb0 msgr2=0x7f3a8c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.237+0000 7f3aa820d640 1 --2- 172.21.15.52:0/323300990 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3a8c0a3bb0 0x7f3a8c0a6070 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3a980bb210 tx=0x7f3a980b4290 comp rx=0 tx=0).stop 2024-04-02T13:53:21.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.237+0000 7f3aa820d640 1 -- 172.21.15.52:0/323300990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3aa01056b0 msgr2=0x7f3aa01a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.237+0000 7f3aa820d640 1 --2- 172.21.15.52:0/323300990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3aa01056b0 0x7f3aa01a59c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3a9802c700 tx=0x7f3a98032f20 comp rx=0 tx=0).stop 2024-04-02T13:53:21.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.237+0000 7f3aa820d640 1 -- 172.21.15.52:0/323300990 shutdown_connections 2024-04-02T13:53:21.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.237+0000 7f3aa820d640 1 --2- 172.21.15.52:0/323300990 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f3a8c0a3bb0 0x7f3a8c0a6070 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.238+0000 7f3aa820d640 1 --2- 172.21.15.52:0/323300990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3aa01056b0 0x7f3aa01a59c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.238+0000 7f3aa820d640 1 --2- 172.21.15.52:0/323300990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3aa0102d00 0x7f3aa019e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.238+0000 7f3aa820d640 1 --2- 172.21.15.52:0/323300990 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3aa0100390 0x7f3aa019e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.238+0000 7f3aa820d640 1 -- 172.21.15.52:0/323300990 >> 172.21.15.52:0/323300990 conn(0x7f3aa00fbfb0 msgr2=0x7f3aa0101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.238+0000 7f3aa820d640 1 -- 172.21.15.52:0/323300990 shutdown_connections 2024-04-02T13:53:21.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.238+0000 7f3aa820d640 1 -- 172.21.15.52:0/323300990 wait complete. 2024-04-02T13:53:21.256 INFO:teuthology.run_tasks:Running task sequential... 2024-04-02T13:53:21.266 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-02T13:53:21.267 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-02T13:53:21.267 INFO:teuthology.task.print:Enabling mgr modules 2024-04-02T13:53:21.267 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-02T13:53:21.268 INFO:teuthology.task.exec:Executing custom commands... 2024-04-02T13:53:21.268 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi052.front.sepia.ceph.com 2024-04-02T13:53:21.268 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-02T13:53:21.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.369+0000 7f47f6863640 1 -- 172.21.15.52:0/2033940614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47f0100a20 msgr2=0x7f47f0100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.369+0000 7f47f6863640 1 --2- 172.21.15.52:0/2033940614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47f0100a20 0x7f47f0100e40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009900 tx=0x7f47e002bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:21.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.369+0000 7f47f6863640 1 -- 172.21.15.52:0/2033940614 shutdown_connections 2024-04-02T13:53:21.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.369+0000 7f47f6863640 1 --2- 172.21.15.52:0/2033940614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47f01025d0 0x7f47f0108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.369+0000 7f47f6863640 1 --2- 172.21.15.52:0/2033940614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47f0101c10 0x7f47f0102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.369+0000 7f47f6863640 1 --2- 172.21.15.52:0/2033940614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47f0100a20 0x7f47f0100e40 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.369+0000 7f47f6863640 1 -- 172.21.15.52:0/2033940614 >> 172.21.15.52:0/2033940614 conn(0x7f47f00fbf90 msgr2=0x7f47f00fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.369+0000 7f47f6863640 1 -- 172.21.15.52:0/2033940614 shutdown_connections 2024-04-02T13:53:21.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.369+0000 7f47f6863640 1 -- 172.21.15.52:0/2033940614 wait complete. 2024-04-02T13:53:21.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.370+0000 7f47f6863640 1 Processor -- start 2024-04-02T13:53:21.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.370+0000 7f47f6863640 1 -- start start 2024-04-02T13:53:21.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.370+0000 7f47f6863640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47f0100a20 0x7f47f0199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.370+0000 7f47f6863640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47f0101c10 0x7f47f019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.370+0000 7f47f6863640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47f01025d0 0x7f47f01a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.370+0000 7f47f6863640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f47f010b5f0 con 0x7f47f0100a20 2024-04-02T13:53:21.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.370+0000 7f47f6863640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f47f010b470 con 0x7f47f01025d0 2024-04-02T13:53:21.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.370+0000 7f47f6863640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f47f010b770 con 0x7f47f0101c10 2024-04-02T13:53:21.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.370+0000 7f47effff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47f0100a20 0x7f47f0199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.370+0000 7f47effff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47f0100a20 0x7f47f0199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41316/0 (socket says 172.21.15.52:41316) 2024-04-02T13:53:21.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.370+0000 7f47effff640 1 -- 172.21.15.52:0/3506741754 learned_addr learned my addr 172.21.15.52:0/3506741754 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:21.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.370+0000 7f47e7fff640 1 --2- 172.21.15.52:0/3506741754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47f0101c10 0x7f47f019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.370+0000 7f47f4e00640 1 --2- 172.21.15.52:0/3506741754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47f01025d0 0x7f47f01a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.370+0000 7f47effff640 1 -- 172.21.15.52:0/3506741754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47f0101c10 msgr2=0x7f47f019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.370+0000 7f47effff640 1 --2- 172.21.15.52:0/3506741754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47f0101c10 0x7f47f019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.370+0000 7f47effff640 1 -- 172.21.15.52:0/3506741754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47f01025d0 msgr2=0x7f47f01a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.370+0000 7f47effff640 1 --2- 172.21.15.52:0/3506741754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47f01025d0 0x7f47f01a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.370+0000 7f47effff640 1 -- 172.21.15.52:0/3506741754 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f01a1bd0 con 0x7f47f0100a20 2024-04-02T13:53:21.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.370+0000 7f47effff640 1 --2- 172.21.15.52:0/3506741754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47f0100a20 0x7f47f0199fb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f47e0000c00 tx=0x7f47e002c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.371+0000 7f47edffb640 1 -- 172.21.15.52:0/3506741754 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0043020 con 0x7f47f0100a20 2024-04-02T13:53:21.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.371+0000 7f47edffb640 1 -- 172.21.15.52:0/3506741754 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e00054e0 con 0x7f47f0100a20 2024-04-02T13:53:21.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.371+0000 7f47f6863640 1 -- 172.21.15.52:0/3506741754 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f01a1e60 con 0x7f47f0100a20 2024-04-02T13:53:21.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.371+0000 7f47edffb640 1 -- 172.21.15.52:0/3506741754 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0003ce0 con 0x7f47f0100a20 2024-04-02T13:53:21.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.371+0000 7f47f6863640 1 -- 172.21.15.52:0/3506741754 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f01a22c0 con 0x7f47f0100a20 2024-04-02T13:53:21.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.372+0000 7f47f6863640 1 -- 172.21.15.52:0/3506741754 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f0193ac0 con 0x7f47f0100a20 2024-04-02T13:53:21.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.373+0000 7f47edffb640 1 -- 172.21.15.52:0/3506741754 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f47e0046020 con 0x7f47f0100a20 2024-04-02T13:53:21.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.374+0000 7f47edffb640 1 --2- 172.21.15.52:0/3506741754 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f47dc0a3c50 0x7f47dc0a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.374+0000 7f47e7fff640 1 --2- 172.21.15.52:0/3506741754 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f47dc0a3c50 0x7f47dc0a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.374+0000 7f47edffb640 1 -- 172.21.15.52:0/3506741754 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f47e00f28d0 con 0x7f47f0100a20 2024-04-02T13:53:21.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.374+0000 7f47e7fff640 1 --2- 172.21.15.52:0/3506741754 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f47dc0a3c50 0x7f47dc0a6110 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f47f019b4c0 tx=0x7f47d800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.378+0000 7f47edffb640 1 -- 172.21.15.52:0/3506741754 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f47e004d020 con 0x7f47f0100a20 2024-04-02T13:53:21.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:21.620+0000 7f47f6863640 1 -- 172.21.15.52:0/3506741754 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f47f0062840 con 0x7f47f0100a20 2024-04-02T13:53:22.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.550+0000 7f47edffb640 1 -- 172.21.15.52:0/3506741754 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f47e00bef50 con 0x7f47f0100a20 2024-04-02T13:53:22.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.555+0000 7f47f6863640 1 -- 172.21.15.52:0/3506741754 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f47dc0a3c50 msgr2=0x7f47dc0a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:22.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.555+0000 7f47f6863640 1 --2- 172.21.15.52:0/3506741754 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f47dc0a3c50 0x7f47dc0a6110 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f47f019b4c0 tx=0x7f47d800b390 comp rx=0 tx=0).stop 2024-04-02T13:53:22.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.555+0000 7f47f6863640 1 -- 172.21.15.52:0/3506741754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47f0100a20 msgr2=0x7f47f0199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:22.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.555+0000 7f47f6863640 1 --2- 172.21.15.52:0/3506741754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47f0100a20 0x7f47f0199fb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f47e0000c00 tx=0x7f47e002c6c0 comp rx=0 tx=0).stop 2024-04-02T13:53:22.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.555+0000 7f47f6863640 1 -- 172.21.15.52:0/3506741754 shutdown_connections 2024-04-02T13:53:22.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.555+0000 7f47f6863640 1 --2- 172.21.15.52:0/3506741754 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f47dc0a3c50 0x7f47dc0a6110 secure :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f47f019b4c0 tx=0x7f47d800b390 comp rx=0 tx=0).stop 2024-04-02T13:53:22.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.555+0000 7f47f6863640 1 --2- 172.21.15.52:0/3506741754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47f01025d0 0x7f47f01a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:22.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.555+0000 7f47f6863640 1 --2- 172.21.15.52:0/3506741754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47f0101c10 0x7f47f019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:22.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.555+0000 7f47f6863640 1 --2- 172.21.15.52:0/3506741754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47f0100a20 0x7f47f0199fb0 secure :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f47e0000c00 tx=0x7f47e002c6c0 comp rx=0 tx=0).stop 2024-04-02T13:53:22.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.555+0000 7f47f6863640 1 -- 172.21.15.52:0/3506741754 >> 172.21.15.52:0/3506741754 conn(0x7f47f00fbf90 msgr2=0x7f47f0190c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:22.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.556+0000 7f47f6863640 1 -- 172.21.15.52:0/3506741754 shutdown_connections 2024-04-02T13:53:22.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.557+0000 7f47f6863640 1 -- 172.21.15.52:0/3506741754 wait complete. 2024-04-02T13:53:22.577 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-02T13:53:22.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.686+0000 7f195cdcd640 1 -- 172.21.15.52:0/130817722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1958104ed0 msgr2=0x7f19581072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:22.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.686+0000 7f195cdcd640 1 --2- 172.21.15.52:0/130817722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1958104ed0 0x7f19581072c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1940009860 tx=0x7f194002bb70 comp rx=0 tx=0).stop 2024-04-02T13:53:22.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.686+0000 7f195cdcd640 1 -- 172.21.15.52:0/130817722 shutdown_connections 2024-04-02T13:53:22.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.686+0000 7f195cdcd640 1 --2- 172.21.15.52:0/130817722 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1958107800 0x7f1958109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:22.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.686+0000 7f195cdcd640 1 --2- 172.21.15.52:0/130817722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1958104ed0 0x7f19581072c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:22.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.686+0000 7f195cdcd640 1 --2- 172.21.15.52:0/130817722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19581025a0 0x7f1958104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:22.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.686+0000 7f195cdcd640 1 -- 172.21.15.52:0/130817722 >> 172.21.15.52:0/130817722 conn(0x7f19580fbf90 msgr2=0x7f19580fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:22.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.686+0000 7f195cdcd640 1 -- 172.21.15.52:0/130817722 shutdown_connections 2024-04-02T13:53:22.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.686+0000 7f195cdcd640 1 -- 172.21.15.52:0/130817722 wait complete. 2024-04-02T13:53:22.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.687+0000 7f195cdcd640 1 Processor -- start 2024-04-02T13:53:22.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.687+0000 7f195cdcd640 1 -- start start 2024-04-02T13:53:22.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.687+0000 7f195cdcd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19581025a0 0x7f1958101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:22.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.687+0000 7f195cdcd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1958104ed0 0x7f1958100550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:22.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.687+0000 7f195cdcd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1958107800 0x7f1958100a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:22.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.687+0000 7f195cdcd640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f195810d790 con 0x7f19581025a0 2024-04-02T13:53:22.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.687+0000 7f195cdcd640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f195810d610 con 0x7f1958107800 2024-04-02T13:53:22.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.687+0000 7f195cdcd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f195810d910 con 0x7f1958104ed0 2024-04-02T13:53:22.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.687+0000 7f195659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19581025a0 0x7f1958101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:22.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.687+0000 7f195659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19581025a0 0x7f1958101f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43466/0 (socket says 172.21.15.52:43466) 2024-04-02T13:53:22.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.687+0000 7f195659c640 1 -- 172.21.15.52:0/911772308 learned_addr learned my addr 172.21.15.52:0/911772308 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:22.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.687+0000 7f1956d9d640 1 --2- 172.21.15.52:0/911772308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1958107800 0x7f1958100a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:22.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.687+0000 7f194dd9b640 1 --2- 172.21.15.52:0/911772308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1958104ed0 0x7f1958100550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:22.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.688+0000 7f195659c640 1 -- 172.21.15.52:0/911772308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1958104ed0 msgr2=0x7f1958100550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:22.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.688+0000 7f195659c640 1 --2- 172.21.15.52:0/911772308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1958104ed0 0x7f1958100550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:22.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.688+0000 7f195659c640 1 -- 172.21.15.52:0/911772308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1958107800 msgr2=0x7f1958100a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:22.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.688+0000 7f195659c640 1 --2- 172.21.15.52:0/911772308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1958107800 0x7f1958100a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:22.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.688+0000 7f195659c640 1 -- 172.21.15.52:0/911772308 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1958101320 con 0x7f19581025a0 2024-04-02T13:53:22.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.688+0000 7f194dd9b640 1 --2- 172.21.15.52:0/911772308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1958104ed0 0x7f1958100550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:22.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.688+0000 7f195659c640 1 --2- 172.21.15.52:0/911772308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19581025a0 0x7f1958101f10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f194800e9c0 tx=0x7f194800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:22.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.688+0000 7f194ffff640 1 -- 172.21.15.52:0/911772308 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194800cd80 con 0x7f19581025a0 2024-04-02T13:53:22.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.688+0000 7f194ffff640 1 -- 172.21.15.52:0/911772308 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19480035f0 con 0x7f19581025a0 2024-04-02T13:53:22.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.688+0000 7f195cdcd640 1 -- 172.21.15.52:0/911772308 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19581a1ed0 con 0x7f19581025a0 2024-04-02T13:53:22.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.688+0000 7f194ffff640 1 -- 172.21.15.52:0/911772308 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948010670 con 0x7f19581025a0 2024-04-02T13:53:22.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.688+0000 7f195cdcd640 1 -- 172.21.15.52:0/911772308 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19581a22d0 con 0x7f19581025a0 2024-04-02T13:53:22.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.690+0000 7f195cdcd640 1 -- 172.21.15.52:0/911772308 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19580fda80 con 0x7f19581025a0 2024-04-02T13:53:22.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.690+0000 7f194ffff640 1 -- 172.21.15.52:0/911772308 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f19480037a0 con 0x7f19581025a0 2024-04-02T13:53:22.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.691+0000 7f194ffff640 1 --2- 172.21.15.52:0/911772308 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f19440a3ca0 0x7f19440a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:22.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.691+0000 7f194dd9b640 1 -- 172.21.15.52:0/911772308 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f19440a3ca0 msgr2=0x7f19440a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3836085837 2024-04-02T13:53:22.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.691+0000 7f194ffff640 1 -- 172.21.15.52:0/911772308 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f19480d0b50 con 0x7f19581025a0 2024-04-02T13:53:22.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.691+0000 7f194dd9b640 1 --2- 172.21.15.52:0/911772308 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f19440a3ca0 0x7f19440a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:53:22.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.696+0000 7f194ffff640 1 -- 172.21.15.52:0/911772308 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f194809d1d0 con 0x7f19581025a0 2024-04-02T13:53:22.839 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:22 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: ignoring --setuser ceph since I am not root 2024-04-02T13:53:22.839 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:22 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: ignoring --setgroup ceph since I am not root 2024-04-02T13:53:22.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:22 smithi052 ceph-mon[32104]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-02T13:53:22.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:22 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3506741754' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-02T13:53:22.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.892+0000 7f194dd9b640 1 -- 172.21.15.52:0/911772308 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f19440a3ca0 msgr2=0x7f19440a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3836085837 2024-04-02T13:53:22.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.892+0000 7f194dd9b640 1 --2- 172.21.15.52:0/911772308 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f19440a3ca0 0x7f19440a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T13:53:22.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.898+0000 7f195cdcd640 1 -- 172.21.15.52:0/911772308 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f1958062700 con 0x7f19581025a0 2024-04-02T13:53:22.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.902+0000 7f194ffff640 1 -- 172.21.15.52:0/911772308 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v32)=0 v32) ==== 138+0+0 (secure 0 0 0) 0x7f194809cad0 con 0x7f19581025a0 2024-04-02T13:53:22.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.904+0000 7f195cdcd640 1 -- 172.21.15.52:0/911772308 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f19440a3ca0 msgr2=0x7f19440a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:53:22.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.904+0000 7f195cdcd640 1 --2- 172.21.15.52:0/911772308 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f19440a3ca0 0x7f19440a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:22.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.904+0000 7f195cdcd640 1 -- 172.21.15.52:0/911772308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19581025a0 msgr2=0x7f1958101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:22.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.904+0000 7f195cdcd640 1 --2- 172.21.15.52:0/911772308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19581025a0 0x7f1958101f10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f194800e9c0 tx=0x7f194800edc0 comp rx=0 tx=0).stop 2024-04-02T13:53:22.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.904+0000 7f195cdcd640 1 -- 172.21.15.52:0/911772308 shutdown_connections 2024-04-02T13:53:22.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.904+0000 7f195cdcd640 1 --2- 172.21.15.52:0/911772308 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f19440a3ca0 0x7f19440a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:22.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.904+0000 7f195cdcd640 1 --2- 172.21.15.52:0/911772308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1958107800 0x7f1958100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:22.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.904+0000 7f195cdcd640 1 --2- 172.21.15.52:0/911772308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1958104ed0 0x7f1958100550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:22.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.904+0000 7f195cdcd640 1 --2- 172.21.15.52:0/911772308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19581025a0 0x7f1958101f10 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:22.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.904+0000 7f195cdcd640 1 -- 172.21.15.52:0/911772308 >> 172.21.15.52:0/911772308 conn(0x7f19580fbf90 msgr2=0x7f1958102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:22.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.904+0000 7f195cdcd640 1 -- 172.21.15.52:0/911772308 shutdown_connections 2024-04-02T13:53:22.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:22.904+0000 7f195cdcd640 1 -- 172.21.15.52:0/911772308 wait complete. 2024-04-02T13:53:22.919 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-02T13:53:22.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:22 smithi088 ceph-mon[36528]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-02T13:53:22.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:22 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3506741754' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-02T13:53:22.920 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:22 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: ignoring --setuser ceph since I am not root 2024-04-02T13:53:22.920 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:22 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: ignoring --setgroup ceph since I am not root 2024-04-02T13:53:22.920 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:22 smithi088 ceph-mgr[37924]: -- 172.21.15.88:0/1105051353 <== mon.2 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b5438565a0 con 0x55b54382a800 2024-04-02T13:53:23.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.022+0000 7f2fb8c67640 1 -- 172.21.15.52:0/2693742231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fb41025a0 msgr2=0x7f2fb4104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:23.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.022+0000 7f2fb8c67640 1 --2- 172.21.15.52:0/2693742231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fb41025a0 0x7f2fb4104990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009900 tx=0x7f2fa402bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:23.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.022+0000 7f2fb8c67640 1 -- 172.21.15.52:0/2693742231 shutdown_connections 2024-04-02T13:53:23.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.022+0000 7f2fb8c67640 1 --2- 172.21.15.52:0/2693742231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb4107800 0x7f2fb4109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.022+0000 7f2fb8c67640 1 --2- 172.21.15.52:0/2693742231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fb4104ed0 0x7f2fb41072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.022+0000 7f2fb8c67640 1 --2- 172.21.15.52:0/2693742231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fb41025a0 0x7f2fb4104990 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.022+0000 7f2fb8c67640 1 -- 172.21.15.52:0/2693742231 >> 172.21.15.52:0/2693742231 conn(0x7f2fb40fbf90 msgr2=0x7f2fb40fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:23.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.022+0000 7f2fb8c67640 1 -- 172.21.15.52:0/2693742231 shutdown_connections 2024-04-02T13:53:23.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.022+0000 7f2fb8c67640 1 -- 172.21.15.52:0/2693742231 wait complete. 2024-04-02T13:53:23.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.023+0000 7f2fb8c67640 1 Processor -- start 2024-04-02T13:53:23.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.023+0000 7f2fb8c67640 1 -- start start 2024-04-02T13:53:23.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.023+0000 7f2fb8c67640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fb41025a0 0x7f2fb4197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.023+0000 7f2fb8c67640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fb4104ed0 0x7f2fb4198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.023+0000 7f2fb8c67640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb4107800 0x7f2fb419f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.023+0000 7f2fb8c67640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb410d7d0 con 0x7f2fb4104ed0 2024-04-02T13:53:23.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.023+0000 7f2fb8c67640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb410d650 con 0x7f2fb4107800 2024-04-02T13:53:23.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.023+0000 7f2fb8c67640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb410d950 con 0x7f2fb41025a0 2024-04-02T13:53:23.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.023+0000 7f2fa9d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fb4104ed0 0x7f2fb4198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.023+0000 7f2fb259c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fb41025a0 0x7f2fb4197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.023+0000 7f2fb259c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fb41025a0 0x7f2fb4197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.52:47232/0 (socket says 172.21.15.52:47232) 2024-04-02T13:53:23.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.023+0000 7f2fb259c640 1 -- 172.21.15.52:0/821822697 learned_addr learned my addr 172.21.15.52:0/821822697 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:23.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.023+0000 7f2fb2d9d640 1 --2- 172.21.15.52:0/821822697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb4107800 0x7f2fb419f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.024+0000 7f2fb259c640 1 -- 172.21.15.52:0/821822697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb4107800 msgr2=0x7f2fb419f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:23.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.024+0000 7f2fb259c640 1 --2- 172.21.15.52:0/821822697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb4107800 0x7f2fb419f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.024+0000 7f2fb259c640 1 -- 172.21.15.52:0/821822697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fb4104ed0 msgr2=0x7f2fb4198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:23.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.024+0000 7f2fb259c640 1 --2- 172.21.15.52:0/821822697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fb4104ed0 0x7f2fb4198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.024+0000 7f2fb259c640 1 -- 172.21.15.52:0/821822697 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb419fa00 con 0x7f2fb41025a0 2024-04-02T13:53:23.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.024+0000 7f2fb259c640 1 --2- 172.21.15.52:0/821822697 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fb41025a0 0x7f2fb4197de0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009230 tx=0x7f2fa4005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:23.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.024+0000 7f2fabfff640 1 -- 172.21.15.52:0/821822697 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4041070 con 0x7f2fb41025a0 2024-04-02T13:53:23.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.024+0000 7f2fb8c67640 1 -- 172.21.15.52:0/821822697 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb419fc90 con 0x7f2fb41025a0 2024-04-02T13:53:23.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.024+0000 7f2fabfff640 1 -- 172.21.15.52:0/821822697 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fa40044d0 con 0x7f2fb41025a0 2024-04-02T13:53:23.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.024+0000 7f2fabfff640 1 -- 172.21.15.52:0/821822697 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4034440 con 0x7f2fb41025a0 2024-04-02T13:53:23.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.024+0000 7f2fb8c67640 1 -- 172.21.15.52:0/821822697 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb41a01d0 con 0x7f2fb41025a0 2024-04-02T13:53:23.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.025+0000 7f2fb8c67640 1 -- 172.21.15.52:0/821822697 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb40fda80 con 0x7f2fb41025a0 2024-04-02T13:53:23.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.026+0000 7f2fabfff640 1 -- 172.21.15.52:0/821822697 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f2fa4044070 con 0x7f2fb41025a0 2024-04-02T13:53:23.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.027+0000 7f2fabfff640 1 --2- 172.21.15.52:0/821822697 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2fa00a3ca0 0x7f2fa00a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.027+0000 7f2fa9d9b640 1 -- 172.21.15.52:0/821822697 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2fa00a3ca0 msgr2=0x7f2fa00a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3836085837 2024-04-02T13:53:23.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.027+0000 7f2fabfff640 1 -- 172.21.15.52:0/821822697 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f2fa40f2560 con 0x7f2fb41025a0 2024-04-02T13:53:23.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.027+0000 7f2fa9d9b640 1 --2- 172.21.15.52:0/821822697 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2fa00a3ca0 0x7f2fa00a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:53:23.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.030+0000 7f2fabfff640 1 -- 172.21.15.52:0/821822697 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2fa40be530 con 0x7f2fb41025a0 2024-04-02T13:53:23.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:22 smithi189 ceph-mon[36683]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-02T13:53:23.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:22 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3506741754' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-02T13:53:23.038 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:22 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: ignoring --setuser ceph since I am not root 2024-04-02T13:53:23.038 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:22 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: ignoring --setgroup ceph since I am not root 2024-04-02T13:53:23.038 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:22 smithi189 ceph-mgr[38543]: -- 172.21.15.189:0/3011073604 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5641ef3ea5a0 con 0x5641ef3be800 2024-04-02T13:53:23.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.220+0000 7f2fb8c67640 1 -- 172.21.15.52:0/821822697 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f2fb402d370 con 0x7f2fb41025a0 2024-04-02T13:53:23.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.226+0000 7f2fabfff640 1 -- 172.21.15.52:0/821822697 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v33)=0 v33) ==== 133+0+0 (secure 0 0 0) 0x7f2fa40c2830 con 0x7f2fb41025a0 2024-04-02T13:53:23.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.228+0000 7f2fa9d9b640 1 -- 172.21.15.52:0/821822697 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2fa00a3ca0 msgr2=0x7f2fa00a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3836085837 2024-04-02T13:53:23.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.228+0000 7f2fa9d9b640 1 --2- 172.21.15.52:0/821822697 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2fa00a3ca0 0x7f2fa00a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T13:53:23.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.228+0000 7f2fb8c67640 1 -- 172.21.15.52:0/821822697 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2fa00a3ca0 msgr2=0x7f2fa00a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:53:23.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.228+0000 7f2fb8c67640 1 --2- 172.21.15.52:0/821822697 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2fa00a3ca0 0x7f2fa00a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.228+0000 7f2fb8c67640 1 -- 172.21.15.52:0/821822697 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fb41025a0 msgr2=0x7f2fb4197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:23.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.228+0000 7f2fb8c67640 1 --2- 172.21.15.52:0/821822697 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fb41025a0 0x7f2fb4197de0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009230 tx=0x7f2fa4005d50 comp rx=0 tx=0).stop 2024-04-02T13:53:23.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.228+0000 7f2fb8c67640 1 -- 172.21.15.52:0/821822697 shutdown_connections 2024-04-02T13:53:23.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.228+0000 7f2fb8c67640 1 --2- 172.21.15.52:0/821822697 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f2fa00a3ca0 0x7f2fa00a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.228+0000 7f2fb8c67640 1 --2- 172.21.15.52:0/821822697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb4107800 0x7f2fb419f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.228+0000 7f2fb8c67640 1 --2- 172.21.15.52:0/821822697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fb4104ed0 0x7f2fb4198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.228+0000 7f2fb8c67640 1 --2- 172.21.15.52:0/821822697 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fb41025a0 0x7f2fb4197de0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.228+0000 7f2fb8c67640 1 -- 172.21.15.52:0/821822697 >> 172.21.15.52:0/821822697 conn(0x7f2fb40fbf90 msgr2=0x7f2fb4102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:23.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.229+0000 7f2fb8c67640 1 -- 172.21.15.52:0/821822697 shutdown_connections 2024-04-02T13:53:23.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:23.229+0000 7f2fb8c67640 1 -- 172.21.15.52:0/821822697 wait complete. 2024-04-02T13:53:23.243 INFO:teuthology.run_tasks:Running task kclient... 2024-04-02T13:53:23.259 INFO:tasks.kclient:Mounting kernel clients... 2024-04-02T13:53:23.259 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-02T13:53:23.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:53:23.260 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-04-02T13:53:23.275 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-02T13:53:23.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:53:23.276 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-02T13:53:23.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:53:23.343 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link delete ceph-brx 2024-04-02T13:53:23.419 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:23 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:53:23.111+0000 7f0a584fc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T13:53:23.419 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:23 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:53:23.204+0000 7f0a584fc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T13:53:23.441 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:23 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:23.167+0000 7ff8e131b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T13:53:23.442 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:23 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:23.267+0000 7ff8e131b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T13:53:23.443 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-02T13:53:23.443 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-02T13:53:23.443 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-02T13:53:23.443 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph fs ls 2024-04-02T13:53:23.538 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:23 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:53:23.115+0000 7fc09d7e4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T13:53:23.538 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:23 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:53:23.208+0000 7fc09d7e4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T13:53:23.697 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:53:23.825 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:23 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3506741754' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-02T13:53:23.825 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:23 smithi052 ceph-mon[32104]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-02T13:53:23.825 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:23 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/911772308' entity='client.admin' 2024-04-02T13:53:23.825 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:23 smithi052 ceph-mon[32104]: from='client.? ' entity='client.admin' 2024-04-02T13:53:23.847 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:23 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3506741754' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-02T13:53:23.848 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:23 smithi088 ceph-mon[36528]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-02T13:53:23.848 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:23 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/911772308' entity='client.admin' 2024-04-02T13:53:23.848 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:23 smithi088 ceph-mon[36528]: from='client.? ' entity='client.admin' 2024-04-02T13:53:23.854 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:23 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3506741754' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-02T13:53:23.854 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:23 smithi189 ceph-mon[36683]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-02T13:53:23.854 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:23 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/911772308' entity='client.admin' 2024-04-02T13:53:23.854 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:23 smithi189 ceph-mon[36683]: from='client.? ' entity='client.admin' 2024-04-02T13:53:24.089 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:23 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:23.971+0000 7ff8e131b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T13:53:24.090 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:24 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:24.070+0000 7ff8e131b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T13:53:24.169 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:23 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:53:23.844+0000 7f0a584fc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T13:53:24.170 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:23 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:53:23.936+0000 7f0a584fc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T13:53:24.170 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:24 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:53:24.078+0000 7f0a584fc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T13:53:24.185 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:23 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:53:23.851+0000 7fc09d7e4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T13:53:24.185 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:23 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:53:23.944+0000 7fc09d7e4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T13:53:24.185 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:24 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:53:24.087+0000 7fc09d7e4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T13:53:24.186 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:24 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:53:24.182+0000 7fc09d7e4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T13:53:24.363 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:24 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:24.219+0000 7ff8e131b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T13:53:24.363 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:24 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:24.320+0000 7ff8e131b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T13:53:24.521 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:24 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:53:24.172+0000 7f0a584fc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T13:53:24.521 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:24 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:53:24.255+0000 7f0a584fc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T13:53:24.521 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:24 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:53:24.518+0000 7f0a584fc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T13:53:24.537 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:24 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:53:24.267+0000 7fc09d7e4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T13:53:24.537 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:24 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:53:24.534+0000 7fc09d7e4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T13:53:24.687 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:24 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:24.408+0000 7ff8e131b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T13:53:24.788 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:24 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:53:24.644+0000 7fc09d7e4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T13:53:24.788 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:24 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:53:24.729+0000 7fc09d7e4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T13:53:24.808 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:24 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:53:24.627+0000 7f0a584fc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T13:53:24.808 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:24 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:53:24.712+0000 7f0a584fc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T13:53:24.808 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:24 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:53:24.805+0000 7f0a584fc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T13:53:24.992 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:24 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:24.684+0000 7ff8e131b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T13:53:24.992 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:24 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:24.799+0000 7ff8e131b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T13:53:24.992 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:24 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:24.889+0000 7ff8e131b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T13:53:25.169 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:24 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:53:24.980+0000 7f0a584fc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T13:53:25.249 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:24 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:24.988+0000 7ff8e131b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T13:53:25.249 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:25 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:25.174+0000 7ff8e131b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T13:53:25.288 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:24 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:53:24.824+0000 7fc09d7e4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T13:53:25.288 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:25 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:53:25.001+0000 7fc09d7e4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T13:53:25.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.669+0000 7f7786c8c700 1 -- 172.21.15.52:0/2873045937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f778006b4f0 msgr2=0x7f778010f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:25.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.669+0000 7f7786c8c700 1 --2- 172.21.15.52:0/2873045937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f778006b4f0 0x7f778010f470 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f777800b0a0 tx=0x7f777800b3b0 comp rx=0 tx=0).stop 2024-04-02T13:53:25.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.670+0000 7f7786c8c700 1 -- 172.21.15.52:0/2873045937 shutdown_connections 2024-04-02T13:53:25.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.670+0000 7f7786c8c700 1 --2- 172.21.15.52:0/2873045937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f778006b4f0 0x7f778010f470 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:25.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.670+0000 7f7786c8c700 1 --2- 172.21.15.52:0/2873045937 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7780108630 0x7f7780108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:25.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.670+0000 7f7786c8c700 1 --2- 172.21.15.52:0/2873045937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f778006ab30 0x7f778006afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:25.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.670+0000 7f7786c8c700 1 -- 172.21.15.52:0/2873045937 >> 172.21.15.52:0/2873045937 conn(0x7f7780077540 msgr2=0x7f7780077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:25.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.670+0000 7f7786c8c700 1 -- 172.21.15.52:0/2873045937 shutdown_connections 2024-04-02T13:53:25.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.670+0000 7f7786c8c700 1 -- 172.21.15.52:0/2873045937 wait complete. 2024-04-02T13:53:25.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.670+0000 7f7786c8c700 1 Processor -- start 2024-04-02T13:53:25.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.670+0000 7f7786c8c700 1 -- start start 2024-04-02T13:53:25.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.671+0000 7f7786c8c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f778006ab30 0x7f778019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:25.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.671+0000 7f7786c8c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f778006b4f0 0x7f778019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:25.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.671+0000 7f7786c8c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7780108630 0x7f7780198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:25.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.671+0000 7f7786c8c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f77801128f0 con 0x7f778006ab30 2024-04-02T13:53:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.671+0000 7f7786c8c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7780112770 con 0x7f778006b4f0 2024-04-02T13:53:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.671+0000 7f7786c8c700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7780112a70 con 0x7f7780108630 2024-04-02T13:53:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.671+0000 7f7784a28700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f778006ab30 0x7f778019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.671+0000 7f7784a28700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f778006ab30 0x7f778019e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43506/0 (socket says 172.21.15.52:43506) 2024-04-02T13:53:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.671+0000 7f7784a28700 1 -- 172.21.15.52:0/1805501762 learned_addr learned my addr 172.21.15.52:0/1805501762 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.671+0000 7f777ffff700 1 --2- 172.21.15.52:0/1805501762 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f778006b4f0 0x7f778019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.671+0000 7f7785229700 1 --2- 172.21.15.52:0/1805501762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7780108630 0x7f7780198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.671+0000 7f777ffff700 1 -- 172.21.15.52:0/1805501762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7780108630 msgr2=0x7f7780198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.671+0000 7f777ffff700 1 --2- 172.21.15.52:0/1805501762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7780108630 0x7f7780198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.671+0000 7f777ffff700 1 -- 172.21.15.52:0/1805501762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f778006ab30 msgr2=0x7f778019e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.671+0000 7f777ffff700 1 --2- 172.21.15.52:0/1805501762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f778006ab30 0x7f778019e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.671+0000 7f777ffff700 1 -- 172.21.15.52:0/1805501762 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77801991f0 con 0x7f778006b4f0 2024-04-02T13:53:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.671+0000 7f7785229700 1 --2- 172.21.15.52:0/1805501762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7780108630 0x7f7780198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.671+0000 7f7784a28700 1 --2- 172.21.15.52:0/1805501762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f778006ab30 0x7f778019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:25.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.671+0000 7f777ffff700 1 --2- 172.21.15.52:0/1805501762 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f778006b4f0 0x7f778019ee10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f776c00e9f0 tx=0x7f776c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:25.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.672+0000 7f777dffb700 1 -- 172.21.15.52:0/1805501762 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c00cbd0 con 0x7f778006b4f0 2024-04-02T13:53:25.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.672+0000 7f7786c8c700 1 -- 172.21.15.52:0/1805501762 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77801994e0 con 0x7f778006b4f0 2024-04-02T13:53:25.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.672+0000 7f777dffb700 1 -- 172.21.15.52:0/1805501762 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f776c004500 con 0x7f778006b4f0 2024-04-02T13:53:25.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.672+0000 7f777dffb700 1 -- 172.21.15.52:0/1805501762 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c005280 con 0x7f778006b4f0 2024-04-02T13:53:25.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.672+0000 7f7786c8c700 1 -- 172.21.15.52:0/1805501762 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7780101520 con 0x7f778006b4f0 2024-04-02T13:53:25.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.673+0000 7f7786c8c700 1 -- 172.21.15.52:0/1805501762 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77800692b0 con 0x7f778006b4f0 2024-04-02T13:53:25.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.674+0000 7f777dffb700 1 -- 172.21.15.52:0/1805501762 <== mon.2 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f776c005420 con 0x7f778006b4f0 2024-04-02T13:53:25.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.674+0000 7f777dffb700 1 --2- 172.21.15.52:0/1805501762 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f77700a3c90 0x7f77700a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:25.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.674+0000 7f7784a28700 1 -- 172.21.15.52:0/1805501762 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f77700a3c90 msgr2=0x7f77700a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3836085837 2024-04-02T13:53:25.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.675+0000 7f777dffb700 1 -- 172.21.15.52:0/1805501762 <== mon.2 v2:172.21.15.88:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f776c0d0750 con 0x7f778006b4f0 2024-04-02T13:53:25.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.675+0000 7f7784a28700 1 --2- 172.21.15.52:0/1805501762 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f77700a3c90 0x7f77700a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:53:25.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.679+0000 7f777dffb700 1 -- 172.21.15.52:0/1805501762 <== mon.2 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f776c09cdd0 con 0x7f778006b4f0 2024-04-02T13:53:25.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.876+0000 7f7784a28700 1 -- 172.21.15.52:0/1805501762 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f77700a3c90 msgr2=0x7f77700a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3836085837 2024-04-02T13:53:25.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.876+0000 7f7784a28700 1 --2- 172.21.15.52:0/1805501762 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f77700a3c90 0x7f77700a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T13:53:25.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.939+0000 7f7786c8c700 1 -- 172.21.15.52:0/1805501762 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f77801017e0 con 0x7f778006b4f0 2024-04-02T13:53:25.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.939+0000 7f777dffb700 1 -- 172.21.15.52:0/1805501762 <== mon.2 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+88 (secure 0 0 0) 0x7f776c09c6d0 con 0x7f778006b4f0 2024-04-02T13:53:25.943 INFO:teuthology.orchestra.run.smithi052.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-02T13:53:25.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.942+0000 7f7786c8c700 1 -- 172.21.15.52:0/1805501762 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f77700a3c90 msgr2=0x7f77700a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:53:25.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.942+0000 7f7786c8c700 1 --2- 172.21.15.52:0/1805501762 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f77700a3c90 0x7f77700a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:25.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.942+0000 7f7786c8c700 1 -- 172.21.15.52:0/1805501762 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f778006b4f0 msgr2=0x7f778019ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:25.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.942+0000 7f7786c8c700 1 --2- 172.21.15.52:0/1805501762 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f778006b4f0 0x7f778019ee10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f776c00e9f0 tx=0x7f776c00ed00 comp rx=0 tx=0).stop 2024-04-02T13:53:25.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.942+0000 7f7786c8c700 1 -- 172.21.15.52:0/1805501762 shutdown_connections 2024-04-02T13:53:25.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.942+0000 7f7786c8c700 1 --2- 172.21.15.52:0/1805501762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7780108630 0x7f7780198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:25.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.942+0000 7f7786c8c700 1 --2- 172.21.15.52:0/1805501762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f778006ab30 0x7f778019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:25.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.942+0000 7f7786c8c700 1 --2- 172.21.15.52:0/1805501762 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f77700a3c90 0x7f77700a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:25.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.942+0000 7f7786c8c700 1 --2- 172.21.15.52:0/1805501762 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f778006b4f0 0x7f778019ee10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:25.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.943+0000 7f7786c8c700 1 -- 172.21.15.52:0/1805501762 >> 172.21.15.52:0/1805501762 conn(0x7f7780077540 msgr2=0x7f77800fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:25.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.943+0000 7f7786c8c700 1 -- 172.21.15.52:0/1805501762 shutdown_connections 2024-04-02T13:53:25.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:25.943+0000 7f7786c8c700 1 -- 172.21.15.52:0/1805501762 wait complete. 2024-04-02T13:53:26.228 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:25 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1805501762' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T13:53:26.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:25 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1805501762' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T13:53:26.288 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:25 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:53:25.969+0000 7fc09d7e4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T13:53:26.288 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:26 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:53:26.100+0000 7fc09d7e4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T13:53:26.392 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:25 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1805501762' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T13:53:26.392 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:25 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:53:25.931+0000 7f0a584fc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T13:53:26.393 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:26 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:53:26.061+0000 7f0a584fc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T13:53:26.393 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:26 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:53:26.389+0000 7f0a584fc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T13:53:26.586 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:26 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:26.224+0000 7ff8e131b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T13:53:26.586 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:26 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:26.364+0000 7ff8e131b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T13:53:26.618 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-02T13:53:26.618 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-02T13:53:26.618 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-04-02T13:53:26.618 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-02T13:53:26.618 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-02T13:53:26.619 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-02T13:53:26.619 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-02T13:53:26.619 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-02T13:53:26.619 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-02T13:53:26.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:53:26.619 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-04-02T13:53:26.635 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-02T13:53:26.635 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-02T13:53:26.635 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-04-02T13:53:26.635 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-04-02T13:53:26.635 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-04-02T13:53:26.635 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-04-02T13:53:26.635 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T13:53:26.635 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-04-02T13:53:26.635 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-04-02T13:53:26.636 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T13:53:26.636 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-04-02T13:53:26.636 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-04-02T13:53:26.636 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-02T13:53:26.636 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-04-02T13:53:26.636 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-04-02T13:53:26.636 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-02T13:53:26.636 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 42072sec preferred_lft 42072sec 2024-04-02T13:53:26.636 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link noprefixroute 2024-04-02T13:53:26.636 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-04-02T13:53:26.636 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T13:53:26.636 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-04-02T13:53:26.636 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-04-02T13:53:26.637 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-02T13:53:26.637 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:53:26.637 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-04-02T13:53:26.637 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add name ceph-brx type bridge 2024-04-02T13:53:26.637 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr flush dev ceph-brx 2024-04-02T13:53:26.637 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set ceph-brx up 2024-04-02T13:53:26.637 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-02T13:53:26.637 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-04-02T13:53:26.669 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:26 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:53:26.479+0000 7f0a584fc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T13:53:26.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:53:26.727 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:26 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:53:26.437+0000 7fc09d7e4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T13:53:26.727 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:26 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:53:26.527+0000 7fc09d7e4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T13:53:26.727 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:26 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:53:26.723+0000 7fc09d7e4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:53:26.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:53:26.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:53:26.790 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-02T13:53:26.817 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-04-02T13:53:26.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:53:26.818 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-04-02T13:53:26.839 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:26 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:26.717+0000 7ff8e131b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T13:53:26.839 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:26 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:26.821+0000 7ff8e131b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T13:53:26.854 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-04-02T13:53:26.854 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-02T13:53:26.854 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-02T13:53:26.854 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-02T13:53:26.854 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-02T13:53:26.855 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:53:26.855 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-04-02T13:53:26.855 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-02T13:53:26.855 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-02T13:53:26.855 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-02T13:53:26.855 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-04-02T13:53:26.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:53:26.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:53:26.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:53:26.991 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-04-02T13:53:27.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:53:27.006 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list-id 2024-04-02T13:53:27.038 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:26 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:53:26.909+0000 7fc09d7e4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T13:53:27.062 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:53:27.062 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-04-02T13:53:27.062 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T13:53:27.062 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-02T13:53:27.062 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-04-02T13:53:27.145 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:26 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:53:26.674+0000 7f0a584fc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:53:27.145 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:26 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:53:26.859+0000 7f0a584fc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T13:53:27.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:53:27.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:53:27.187 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-02T13:53:27.187 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:53:27.187 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-04-02T13:53:27.187 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-02T13:53:27.187 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-02T13:53:27.188 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-02T13:53:27.188 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-02T13:53:27.188 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-02T13:53:27.188 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-04-02T13:53:27.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:53:27.240 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:27 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:27.033+0000 7ff8e131b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:53:27.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:53:27.316 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:53:27.316 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-04-02T13:53:27.317 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-04-02T13:53:27.317 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set dev brx.0 master ceph-brx 2024-04-02T13:53:27.317 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-04-02T13:53:27.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:53:27.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:53:27.401 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T13:53:27.401 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-02T13:53:27.402 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:27 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:53:27.142+0000 7f0a584fc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:53:27.416 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-02T13:53:27.416 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T13:53:27.416 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-02T13:53:27.457 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:27 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:53:27.194+0000 7fc09d7e4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:53:27.470 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-02T13:53:27.470 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-02T13:53:27.470 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/e3c6433a-5ae3-4174-8392-2dc149574911 2024-04-02T13:53:27.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:53:27.471 DEBUG:teuthology.orchestra.run.smithi052:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/e3c6433a-5ae3-4174-8392-2dc149574911 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-04-02T13:53:27.533 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:27 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:27.237+0000 7ff8e131b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T13:53:27.554 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-04-02T13:53:27.554 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync". 2024-04-02T13:53:27.554 INFO:teuthology.orchestra.run.smithi052.stdout:invalid new device string format 2024-04-02T13:53:27.669 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:27 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:53:27.399+0000 7f0a584fc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T13:53:27.669 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:27 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:53:27.636+0000 7f0a584fc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T13:53:27.671 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-04-02T13:53:27.671 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync". 2024-04-02T13:53:27.671 INFO:teuthology.orchestra.run.smithi052.stdout:invalid new device string format 2024-04-02T13:53:27.671 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: resolved to: "172.21.15.52:6789,172.21.15.88:6789,172.21.15.189:6789" 2024-04-02T13:53:27.671 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.52:6789,172.21.15.88:6789,172.21.15.189:6789:/volumes/qa/sv_1/e3c6433a-5ae3-4174-8392-2dc149574911 2024-04-02T13:53:27.671 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,key=0,fsid=bfa00e6e-f0f6-11ee-b647-cb9ed24678a4" will pass to kernel 2024-04-02T13:53:27.672 INFO:teuthology.orchestra.run.smithi052.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-02T13:53:27.672 INFO:teuthology.orchestra.run.smithi052.stdout: You just mounted a file system that supports labels which does not 2024-04-02T13:53:27.672 INFO:teuthology.orchestra.run.smithi052.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-02T13:53:27.672 INFO:teuthology.orchestra.run.smithi052.stdout: applications will generate AVC messages and not be allowed access to 2024-04-02T13:53:27.672 INFO:teuthology.orchestra.run.smithi052.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-02T13:53:27.674 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-02T13:53:27.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:53:27.674 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-02T13:53:27.744 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-02T13:53:27.788 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:27 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:53:27.454+0000 7fc09d7e4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T13:53:27.788 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:27 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:53:27.691+0000 7fc09d7e4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T13:53:27.824 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:27 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:27.550+0000 7ff8e131b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:53:27.824 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:27 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:27.821+0000 7ff8e131b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T13:53:27.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T13:53:27.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:53:27.842 INFO:teuthology.orchestra.run.smithi052.stdout:client25114 2024-04-02T13:53:27.842 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-04-02T13:53:27.842 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T13:53:27.898 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-02T13:53:27.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T13:53:27.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:53:27.994 INFO:teuthology.orchestra.run.smithi052.stdout:client25114 2024-04-02T13:53:27.995 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-02T13:53:28.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T13:53:28.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:53:28.052 INFO:teuthology.orchestra.run.smithi052.stdout:client25114 2024-04-02T13:53:28.052 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4.client25114/status') 2024-04-02T13:53:28.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:53:28.114 INFO:teuthology.orchestra.run.smithi052.stdout:instance: client.25114 (3)172.21.15.52:0/3159566124 2024-04-02T13:53:28.114 INFO:teuthology.orchestra.run.smithi052.stdout:blocklisted: false 2024-04-02T13:53:28.115 INFO:teuthology.orchestra.run.smithi052.stderr:0+1 records in 2024-04-02T13:53:28.115 INFO:teuthology.orchestra.run.smithi052.stderr:0+1 records out 2024-04-02T13:53:28.115 INFO:teuthology.orchestra.run.smithi052.stderr:71 bytes copied, 2.0391e-05 s, 3.5 MB/s 2024-04-02T13:53:28.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:53:28.122 INFO:teuthology.run_tasks:Running task exec... 2024-04-02T13:53:28.131 INFO:teuthology.task.exec:Executing custom commands... 2024-04-02T13:53:28.132 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi052.front.sepia.ceph.com 2024-04-02T13:53:28.132 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-02T13:53:28.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.234+0000 7f11bfe06640 1 -- 172.21.15.52:0/3815740126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11b80744e0 msgr2=0x7f11b8074940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:28.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.234+0000 7f11bfe06640 1 --2- 172.21.15.52:0/3815740126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11b80744e0 0x7f11b8074940 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009900 tx=0x7f11b402bcd0 comp rx=0 tx=0).stop 2024-04-02T13:53:28.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.234+0000 7f11bfe06640 1 -- 172.21.15.52:0/3815740126 shutdown_connections 2024-04-02T13:53:28.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.234+0000 7f11bfe06640 1 --2- 172.21.15.52:0/3815740126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11b8109a10 0x7f11b810bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:28.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.234+0000 7f11bfe06640 1 --2- 172.21.15.52:0/3815740126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11b80744e0 0x7f11b8074940 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:28.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.234+0000 7f11bfe06640 1 --2- 172.21.15.52:0/3815740126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11b8075ac0 0x7f11b8073f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:28.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.234+0000 7f11bfe06640 1 -- 172.21.15.52:0/3815740126 >> 172.21.15.52:0/3815740126 conn(0x7f11b80fd0d0 msgr2=0x7f11b80ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:28.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.234+0000 7f11bfe06640 1 -- 172.21.15.52:0/3815740126 shutdown_connections 2024-04-02T13:53:28.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.234+0000 7f11bfe06640 1 -- 172.21.15.52:0/3815740126 wait complete. 2024-04-02T13:53:28.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.235+0000 7f11bfe06640 1 Processor -- start 2024-04-02T13:53:28.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.235+0000 7f11bfe06640 1 -- start start 2024-04-02T13:53:28.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.235+0000 7f11bfe06640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11b80744e0 0x7f11b8199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:28.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.235+0000 7f11bfe06640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11b8075ac0 0x7f11b819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:28.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.235+0000 7f11bfe06640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11b8109a10 0x7f11b81a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:28.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.235+0000 7f11bfe06640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f11b810f9e0 con 0x7f11b80744e0 2024-04-02T13:53:28.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.235+0000 7f11bfe06640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f11b810f860 con 0x7f11b8109a10 2024-04-02T13:53:28.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.235+0000 7f11bfe06640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f11b810fb60 con 0x7f11b8075ac0 2024-04-02T13:53:28.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.235+0000 7f11bdba2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11b80744e0 0x7f11b8199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:28.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.235+0000 7f11bdba2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11b80744e0 0x7f11b8199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43514/0 (socket says 172.21.15.52:43514) 2024-04-02T13:53:28.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.235+0000 7f11bdba2640 1 -- 172.21.15.52:0/2162705315 learned_addr learned my addr 172.21.15.52:0/2162705315 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:28.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.235+0000 7f11be3a3640 1 --2- 172.21.15.52:0/2162705315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11b8109a10 0x7f11b81a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:28.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.235+0000 7f11bd3a1640 1 --2- 172.21.15.52:0/2162705315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11b8075ac0 0x7f11b819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:28.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.236+0000 7f11bdba2640 1 -- 172.21.15.52:0/2162705315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11b8075ac0 msgr2=0x7f11b819a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:28.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.236+0000 7f11bdba2640 1 --2- 172.21.15.52:0/2162705315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11b8075ac0 0x7f11b819a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:28.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.236+0000 7f11bdba2640 1 -- 172.21.15.52:0/2162705315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11b8109a10 msgr2=0x7f11b81a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:28.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.236+0000 7f11bdba2640 1 --2- 172.21.15.52:0/2162705315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11b8109a10 0x7f11b81a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:28.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.236+0000 7f11bdba2640 1 -- 172.21.15.52:0/2162705315 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b81a1bc0 con 0x7f11b80744e0 2024-04-02T13:53:28.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.236+0000 7f11be3a3640 1 --2- 172.21.15.52:0/2162705315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11b8109a10 0x7f11b81a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:28.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.236+0000 7f11bdba2640 1 --2- 172.21.15.52:0/2162705315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11b80744e0 0x7f11b8199ff0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00e9f0 tx=0x7f11ac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:28.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.236+0000 7f11aaffd640 1 -- 172.21.15.52:0/2162705315 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac00cd80 con 0x7f11b80744e0 2024-04-02T13:53:28.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.236+0000 7f11aaffd640 1 -- 172.21.15.52:0/2162705315 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11ac005500 con 0x7f11b80744e0 2024-04-02T13:53:28.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.236+0000 7f11bfe06640 1 -- 172.21.15.52:0/2162705315 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11b81a1eb0 con 0x7f11b80744e0 2024-04-02T13:53:28.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.236+0000 7f11aaffd640 1 -- 172.21.15.52:0/2162705315 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac010690 con 0x7f11b80744e0 2024-04-02T13:53:28.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.236+0000 7f11bfe06640 1 -- 172.21.15.52:0/2162705315 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11b81a22c0 con 0x7f11b80744e0 2024-04-02T13:53:28.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.237+0000 7f11bfe06640 1 -- 172.21.15.52:0/2162705315 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11b81087d0 con 0x7f11b80744e0 2024-04-02T13:53:28.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.238+0000 7f11aaffd640 1 -- 172.21.15.52:0/2162705315 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f11ac0046c0 con 0x7f11b80744e0 2024-04-02T13:53:28.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.239+0000 7f11aaffd640 1 --2- 172.21.15.52:0/2162705315 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f11a40a3ca0 0x7f11a40a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:28.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.239+0000 7f11bd3a1640 1 -- 172.21.15.52:0/2162705315 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f11a40a3ca0 msgr2=0x7f11a40a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3836085837 2024-04-02T13:53:28.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.239+0000 7f11aaffd640 1 -- 172.21.15.52:0/2162705315 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f11ac0d0e10 con 0x7f11b80744e0 2024-04-02T13:53:28.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.239+0000 7f11bd3a1640 1 --2- 172.21.15.52:0/2162705315 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f11a40a3ca0 0x7f11a40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:53:28.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.243+0000 7f11aaffd640 1 -- 172.21.15.52:0/2162705315 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f11ac053f20 con 0x7f11b80744e0 2024-04-02T13:53:28.339 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:28 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:28.079+0000 7ff8e131b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T13:53:28.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.440+0000 7f11bd3a1640 1 -- 172.21.15.52:0/2162705315 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f11a40a3ca0 msgr2=0x7f11a40a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3836085837 2024-04-02T13:53:28.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.440+0000 7f11bd3a1640 1 --2- 172.21.15.52:0/2162705315 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f11a40a3ca0 0x7f11a40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T13:53:28.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.473+0000 7f11bfe06640 1 -- 172.21.15.52:0/2162705315 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f11b802d370 con 0x7f11b80744e0 2024-04-02T13:53:28.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.525+0000 7f11aaffd640 1 -- 172.21.15.52:0/2162705315 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7f11ac09d430 con 0x7f11b80744e0 2024-04-02T13:53:28.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.527+0000 7f11bfe06640 1 -- 172.21.15.52:0/2162705315 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f11a40a3ca0 msgr2=0x7f11a40a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:53:28.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.527+0000 7f11bfe06640 1 --2- 172.21.15.52:0/2162705315 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f11a40a3ca0 0x7f11a40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:28.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.527+0000 7f11bfe06640 1 -- 172.21.15.52:0/2162705315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11b80744e0 msgr2=0x7f11b8199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:28.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.527+0000 7f11bfe06640 1 --2- 172.21.15.52:0/2162705315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11b80744e0 0x7f11b8199ff0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00e9f0 tx=0x7f11ac00edf0 comp rx=0 tx=0).stop 2024-04-02T13:53:28.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.527+0000 7f11bfe06640 1 -- 172.21.15.52:0/2162705315 shutdown_connections 2024-04-02T13:53:28.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.527+0000 7f11bfe06640 1 --2- 172.21.15.52:0/2162705315 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f11a40a3ca0 0x7f11a40a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:28.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.527+0000 7f11bfe06640 1 --2- 172.21.15.52:0/2162705315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11b8109a10 0x7f11b81a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:28.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.527+0000 7f11bfe06640 1 --2- 172.21.15.52:0/2162705315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11b8075ac0 0x7f11b819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:28.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.527+0000 7f11bfe06640 1 --2- 172.21.15.52:0/2162705315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11b80744e0 0x7f11b8199ff0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:28.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.527+0000 7f11bfe06640 1 -- 172.21.15.52:0/2162705315 >> 172.21.15.52:0/2162705315 conn(0x7f11b80fd0d0 msgr2=0x7f11b81059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:28.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.527+0000 7f11bfe06640 1 -- 172.21.15.52:0/2162705315 shutdown_connections 2024-04-02T13:53:28.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.527+0000 7f11bfe06640 1 -- 172.21.15.52:0/2162705315 wait complete. 2024-04-02T13:53:28.542 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-02T13:53:28.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.645+0000 7f08d1869640 1 -- 172.21.15.52:0/2490308150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08cc102600 msgr2=0x7f08cc108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:28.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.645+0000 7f08d1869640 1 --2- 172.21.15.52:0/2490308150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08cc102600 0x7f08cc108e90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f08c4009ca0 tx=0x7f08c402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:53:28.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.645+0000 7f08d1869640 1 -- 172.21.15.52:0/2490308150 shutdown_connections 2024-04-02T13:53:28.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.645+0000 7f08d1869640 1 --2- 172.21.15.52:0/2490308150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08cc102600 0x7f08cc108e90 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:28.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.645+0000 7f08d1869640 1 --2- 172.21.15.52:0/2490308150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08cc101c40 0x7f08cc1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.645+0000 7f08d1869640 1 --2- 172.21.15.52:0/2490308150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08cc100a50 0x7f08cc100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.645+0000 7f08d1869640 1 -- 172.21.15.52:0/2490308150 >> 172.21.15.52:0/2490308150 conn(0x7f08cc0fbfa0 msgr2=0x7f08cc0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.645+0000 7f08d1869640 1 -- 172.21.15.52:0/2490308150 shutdown_connections 2024-04-02T13:53:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.645+0000 7f08d1869640 1 -- 172.21.15.52:0/2490308150 wait complete. 2024-04-02T13:53:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.646+0000 7f08d1869640 1 Processor -- start 2024-04-02T13:53:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.646+0000 7f08d1869640 1 -- start start 2024-04-02T13:53:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.646+0000 7f08d1869640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08cc100a50 0x7f08cc197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.646+0000 7f08d1869640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08cc101c40 0x7f08cc198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.646+0000 7f08d1869640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08cc102600 0x7f08cc19f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.646+0000 7f08d1869640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc10d7f0 con 0x7f08cc101c40 2024-04-02T13:53:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.646+0000 7f08d1869640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc10d670 con 0x7f08cc100a50 2024-04-02T13:53:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.646+0000 7f08d1869640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc10d970 con 0x7f08cc102600 2024-04-02T13:53:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.646+0000 7f08ca7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08cc101c40 0x7f08cc198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:28.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.646+0000 7f08caffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08cc100a50 0x7f08cc197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:28.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.646+0000 7f08ca7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08cc101c40 0x7f08cc198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43534/0 (socket says 172.21.15.52:43534) 2024-04-02T13:53:28.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.646+0000 7f08ca7fc640 1 -- 172.21.15.52:0/1528122096 learned_addr learned my addr 172.21.15.52:0/1528122096 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:28.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.646+0000 7f08cb7fe640 1 --2- 172.21.15.52:0/1528122096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08cc102600 0x7f08cc19f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:28.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.646+0000 7f08caffd640 1 -- 172.21.15.52:0/1528122096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08cc102600 msgr2=0x7f08cc19f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:28.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.646+0000 7f08caffd640 1 --2- 172.21.15.52:0/1528122096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08cc102600 0x7f08cc19f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:28.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.646+0000 7f08caffd640 1 -- 172.21.15.52:0/1528122096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08cc101c40 msgr2=0x7f08cc198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:28.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.646+0000 7f08caffd640 1 --2- 172.21.15.52:0/1528122096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08cc101c40 0x7f08cc198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:28.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.646+0000 7f08caffd640 1 -- 172.21.15.52:0/1528122096 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08cc19f960 con 0x7f08cc100a50 2024-04-02T13:53:28.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.647+0000 7f08caffd640 1 --2- 172.21.15.52:0/1528122096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08cc100a50 0x7f08cc197d50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f08cc101aa0 tx=0x7f08bc00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:28.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.647+0000 7f08d0867640 1 -- 172.21.15.52:0/1528122096 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc017070 con 0x7f08cc100a50 2024-04-02T13:53:28.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.647+0000 7f08d1869640 1 -- 172.21.15.52:0/1528122096 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08cc19fc50 con 0x7f08cc100a50 2024-04-02T13:53:28.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.647+0000 7f08d0867640 1 -- 172.21.15.52:0/1528122096 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08bc009c30 con 0x7f08cc100a50 2024-04-02T13:53:28.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.647+0000 7f08d0867640 1 -- 172.21.15.52:0/1528122096 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc00f620 con 0x7f08cc100a50 2024-04-02T13:53:28.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.647+0000 7f08d1869640 1 -- 172.21.15.52:0/1528122096 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08cc1a01a0 con 0x7f08cc100a50 2024-04-02T13:53:28.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.648+0000 7f08d1869640 1 -- 172.21.15.52:0/1528122096 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08cc107d10 con 0x7f08cc100a50 2024-04-02T13:53:28.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.649+0000 7f08d0867640 1 -- 172.21.15.52:0/1528122096 <== mon.2 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f08bc01a070 con 0x7f08cc100a50 2024-04-02T13:53:28.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.649+0000 7f08d0867640 1 --2- 172.21.15.52:0/1528122096 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f08b40a3ca0 0x7f08b40a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:28.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.649+0000 7f08ca7fc640 1 -- 172.21.15.52:0/1528122096 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f08b40a3ca0 msgr2=0x7f08b40a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3836085837 2024-04-02T13:53:28.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.649+0000 7f08d0867640 1 -- 172.21.15.52:0/1528122096 <== mon.2 v2:172.21.15.88:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f08bc0d0410 con 0x7f08cc100a50 2024-04-02T13:53:28.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.650+0000 7f08ca7fc640 1 --2- 172.21.15.52:0/1528122096 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f08b40a3ca0 0x7f08b40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:53:28.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.653+0000 7f08d0867640 1 -- 172.21.15.52:0/1528122096 <== mon.2 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f08bc09c3e0 con 0x7f08cc100a50 2024-04-02T13:53:28.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:28 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2162705315' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-02T13:53:28.788 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:28 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:53:28.497+0000 7fc09d7e4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T13:53:28.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:28 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2162705315' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-02T13:53:28.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.851+0000 7f08ca7fc640 1 -- 172.21.15.52:0/1528122096 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f08b40a3ca0 msgr2=0x7f08b40a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3836085837 2024-04-02T13:53:28.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.851+0000 7f08ca7fc640 1 --2- 172.21.15.52:0/1528122096 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f08b40a3ca0 0x7f08b40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T13:53:28.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:28.868+0000 7f08d1869640 1 -- 172.21.15.52:0/1528122096 --> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f08cc061fe0 con 0x7f08b40a3ca0 2024-04-02T13:53:28.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:28 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2162705315' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-02T13:53:28.920 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:28 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:53:28.440+0000 7f0a584fc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T13:53:29.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:29.252+0000 7f08ca7fc640 1 -- 172.21.15.52:0/1528122096 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f08b40a3ca0 msgr2=0x7f08b40a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3836085837 2024-04-02T13:53:29.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:29.252+0000 7f08ca7fc640 1 --2- 172.21.15.52:0/1528122096 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f08b40a3ca0 0x7f08b40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-02T13:53:29.336 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:29 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:53:29.018+0000 7f0a584fc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T13:53:29.336 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:29 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:53:29.116+0000 7f0a584fc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T13:53:29.336 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 13:53:29 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y[37920]: 2024-04-02T13:53:29.333+0000 7f0a584fc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T13:53:29.339 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:28 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:28.952+0000 7ff8e131b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T13:53:29.392 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:29 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:53:29.072+0000 7fc09d7e4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T13:53:29.392 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:29 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:53:29.170+0000 7fc09d7e4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T13:53:29.392 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 13:53:29 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z[38539]: 2024-04-02T13:53:29.389+0000 7fc09d7e4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T13:53:29.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:29.544+0000 7f08d0867640 1 -- 172.21.15.52:0/1528122096 <== mon.2 v2:172.21.15.88:3300/0 7 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7f08bc0989c0 con 0x7f08cc100a50 2024-04-02T13:53:29.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:29 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2162705315' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-02T13:53:29.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:29 smithi088 ceph-mon[36528]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:53:29.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:29 smithi088 ceph-mon[36528]: Standby manager daemon y restarted 2024-04-02T13:53:29.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:29 smithi088 ceph-mon[36528]: Standby manager daemon y started 2024-04-02T13:53:29.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:29 smithi088 ceph-mon[36528]: from='mgr.? 172.21.15.88:0/722381598' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T13:53:29.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:29 smithi088 ceph-mon[36528]: from='mgr.? 172.21.15.88:0/722381598' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:53:29.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:29 smithi088 ceph-mon[36528]: from='mgr.? 172.21.15.88:0/722381598' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T13:53:29.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:29 smithi088 ceph-mon[36528]: from='mgr.? 172.21.15.88:0/722381598' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:53:29.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:29 smithi088 ceph-mon[36528]: Standby manager daemon z restarted 2024-04-02T13:53:29.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:29 smithi088 ceph-mon[36528]: Standby manager daemon z started 2024-04-02T13:53:29.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:29 smithi088 ceph-mon[36528]: from='mgr.? 172.21.15.189:0/2463865269' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T13:53:29.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:29 smithi088 ceph-mon[36528]: from='mgr.? 172.21.15.189:0/2463865269' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:53:29.671 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:29 smithi088 ceph-mon[36528]: from='mgr.? 172.21.15.189:0/2463865269' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T13:53:29.671 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:29 smithi088 ceph-mon[36528]: from='mgr.? 172.21.15.189:0/2463865269' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:53:29.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:29 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2162705315' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-02T13:53:29.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:29 smithi189 ceph-mon[36683]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:53:29.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:29 smithi189 ceph-mon[36683]: Standby manager daemon y restarted 2024-04-02T13:53:29.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:29 smithi189 ceph-mon[36683]: Standby manager daemon y started 2024-04-02T13:53:29.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:29 smithi189 ceph-mon[36683]: from='mgr.? 172.21.15.88:0/722381598' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T13:53:29.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:29 smithi189 ceph-mon[36683]: from='mgr.? 172.21.15.88:0/722381598' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:53:29.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:29 smithi189 ceph-mon[36683]: from='mgr.? 172.21.15.88:0/722381598' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T13:53:29.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:29 smithi189 ceph-mon[36683]: from='mgr.? 172.21.15.88:0/722381598' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:53:29.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:29 smithi189 ceph-mon[36683]: Standby manager daemon z restarted 2024-04-02T13:53:29.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:29 smithi189 ceph-mon[36683]: Standby manager daemon z started 2024-04-02T13:53:29.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:29 smithi189 ceph-mon[36683]: from='mgr.? 172.21.15.189:0/2463865269' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T13:53:29.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:29 smithi189 ceph-mon[36683]: from='mgr.? 172.21.15.189:0/2463865269' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:53:29.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:29 smithi189 ceph-mon[36683]: from='mgr.? 172.21.15.189:0/2463865269' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T13:53:29.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:29 smithi189 ceph-mon[36683]: from='mgr.? 172.21.15.189:0/2463865269' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:53:29.839 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:29 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:29.551+0000 7ff8e131b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T13:53:29.839 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:29 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:29.656+0000 7ff8e131b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T13:53:29.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:29 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2162705315' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-02T13:53:29.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:29 smithi052 ceph-mon[32104]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:53:29.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:29 smithi052 ceph-mon[32104]: Standby manager daemon y restarted 2024-04-02T13:53:29.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:29 smithi052 ceph-mon[32104]: Standby manager daemon y started 2024-04-02T13:53:29.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:29 smithi052 ceph-mon[32104]: from='mgr.? 172.21.15.88:0/722381598' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T13:53:29.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:29 smithi052 ceph-mon[32104]: from='mgr.? 172.21.15.88:0/722381598' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:53:29.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:29 smithi052 ceph-mon[32104]: from='mgr.? 172.21.15.88:0/722381598' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T13:53:29.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:29 smithi052 ceph-mon[32104]: from='mgr.? 172.21.15.88:0/722381598' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:53:29.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:29 smithi052 ceph-mon[32104]: Standby manager daemon z restarted 2024-04-02T13:53:29.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:29 smithi052 ceph-mon[32104]: Standby manager daemon z started 2024-04-02T13:53:29.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:29 smithi052 ceph-mon[32104]: from='mgr.? 172.21.15.189:0/2463865269' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T13:53:29.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:29 smithi052 ceph-mon[32104]: from='mgr.? 172.21.15.189:0/2463865269' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:53:29.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:29 smithi052 ceph-mon[32104]: from='mgr.? 172.21.15.189:0/2463865269' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T13:53:29.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:29 smithi052 ceph-mon[32104]: from='mgr.? 172.21.15.189:0/2463865269' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:53:29.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:29.902+0000 7f08d0867640 1 -- 172.21.15.52:0/1528122096 <== mon.2 v2:172.21.15.88:3300/0 8 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f08bc0cdbd0 con 0x7f08cc100a50 2024-04-02T13:53:29.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:29.903+0000 7f08d0867640 1 -- 172.21.15.52:0/1528122096 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f08b40a3ca0 msgr2=0x7f08b40a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:53:29.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:29.903+0000 7f08d0867640 1 --2- 172.21.15.52:0/1528122096 >> [v2:172.21.15.52:6800/3836085837,v1:172.21.15.52:6801/3836085837] conn(0x7f08b40a3ca0 0x7f08b40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:30.159 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:29 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:29.886+0000 7ff8e131b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T13:53:30.160 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:30.129+0000 7ff840d2a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:53:30.160 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:30.129+0000 7ff840d2a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:53:30.160 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:30.129+0000 7ff840d2a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:53:30.160 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:30.129+0000 7ff840d2a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:53:30.160 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:30.129+0000 7ff840d2a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:53:30.160 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:30.129+0000 7ff842d2e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:53:30.160 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:30.129+0000 7ff842d2e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:53:30.160 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:30.129+0000 7ff842d2e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:53:30.160 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:30.129+0000 7ff842d2e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:53:30.160 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:30.129+0000 7ff842d2e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:53:30.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-04-02T13:53:30.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: Active manager daemon x restarted 2024-04-02T13:53:30.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: Activating manager daemon x 2024-04-02T13:53:30.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: osdmap e93: 12 total, 12 up, 12 in 2024-04-02T13:53:30.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: mgrmap e21: x(active, starting, since 0.0106068s), standbys: y, z 2024-04-02T13:53:30.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:53:30.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:53:30.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:53:30.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T13:53:30.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T13:53:30.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T13:53:30.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T13:53:30.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T13:53:30.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T13:53:30.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T13:53:30.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T13:53:30.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T13:53:30.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T13:53:30.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T13:53:30.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T13:53:30.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T13:53:30.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T13:53:30.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T13:53:30.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:53:30.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:53:30.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:53:30.843 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:53:30.843 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:53:30.843 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:53:30.843 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:53:30.843 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:53:30.843 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:53:30.843 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:53:30.843 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:53:30.843 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:53:30.843 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T13:53:30.843 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T13:53:30.843 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T13:53:30.844 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: Manager daemon x is now available 2024-04-02T13:53:30.844 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:53:30.844 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T13:53:30.844 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T13:53:30.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:30.904+0000 7f08d0867640 1 -- 172.21.15.52:0/1528122096 <== mon.2 v2:172.21.15.88:3300/0 9 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f08bc0a9970 con 0x7f08cc100a50 2024-04-02T13:53:30.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:30.904+0000 7f08d0867640 1 --2- 172.21.15.52:0/1528122096 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f08b4032440 0x7f08b4032820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:30.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:30.904+0000 7f08d0867640 1 -- 172.21.15.52:0/1528122096 --> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f08bc097eb0 con 0x7f08b4032440 2024-04-02T13:53:30.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:30.906+0000 7f08ca7fc640 1 --2- 172.21.15.52:0/1528122096 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f08b4032440 0x7f08b4032820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:30.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:30.910+0000 7f08ca7fc640 1 --2- 172.21.15.52:0/1528122096 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f08b4032440 0x7f08b4032820 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f08c00098b0 tx=0x7f08c0005f90 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-04-02T13:53:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: Active manager daemon x restarted 2024-04-02T13:53:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: Activating manager daemon x 2024-04-02T13:53:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: osdmap e93: 12 total, 12 up, 12 in 2024-04-02T13:53:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: mgrmap e21: x(active, starting, since 0.0106068s), standbys: y, z 2024-04-02T13:53:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:53:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:53:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:53:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T13:53:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T13:53:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T13:53:30.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T13:53:30.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T13:53:30.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T13:53:30.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T13:53:30.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T13:53:30.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T13:53:30.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T13:53:30.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T13:53:30.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T13:53:30.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T13:53:30.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T13:53:30.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T13:53:30.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:53:30.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:53:30.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:53:30.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:53:30.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:53:30.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:53:30.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:53:30.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:53:30.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:53:30.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:53:30.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:53:30.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:53:30.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T13:53:30.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T13:53:30.923 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T13:53:30.923 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: Manager daemon x is now available 2024-04-02T13:53:30.923 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:53:30.923 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T13:53:30.923 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:30 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T13:53:30.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:30.933+0000 7f08d0867640 1 -- 172.21.15.52:0/1528122096 <== mgr.25160 v2:172.21.15.52:6800/1019901931 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f08bc097eb0 con 0x7f08b4032440 2024-04-02T13:53:30.936 INFO:teuthology.orchestra.run.smithi052.stdout:{} 2024-04-02T13:53:30.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:30.935+0000 7f08d1869640 1 -- 172.21.15.52:0/1528122096 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f08b4032440 msgr2=0x7f08b4032820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:30.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:30.935+0000 7f08d1869640 1 --2- 172.21.15.52:0/1528122096 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f08b4032440 0x7f08b4032820 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f08c00098b0 tx=0x7f08c0005f90 comp rx=0 tx=0).stop 2024-04-02T13:53:30.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:30.935+0000 7f08d1869640 1 -- 172.21.15.52:0/1528122096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08cc100a50 msgr2=0x7f08cc197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:30.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:30.935+0000 7f08d1869640 1 --2- 172.21.15.52:0/1528122096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08cc100a50 0x7f08cc197d50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f08cc101aa0 tx=0x7f08bc00dde0 comp rx=0 tx=0).stop 2024-04-02T13:53:30.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:30.935+0000 7f08cb7fe640 1 -- 172.21.15.52:0/1528122096 reap_dead start 2024-04-02T13:53:30.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:30.935+0000 7f08d1869640 1 -- 172.21.15.52:0/1528122096 shutdown_connections 2024-04-02T13:53:30.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:30.935+0000 7f08d1869640 1 -- 172.21.15.52:0/1528122096 >> 172.21.15.52:0/1528122096 conn(0x7f08cc0fbfa0 msgr2=0x7f08cc104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:30.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:30.935+0000 7f08d1869640 1 -- 172.21.15.52:0/1528122096 shutdown_connections 2024-04-02T13:53:30.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:30.935+0000 7f08d1869640 1 -- 172.21.15.52:0/1528122096 wait complete. 2024-04-02T13:53:30.950 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-02T13:53:30.961 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-02T13:53:30.972 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:53:31.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-04-02T13:53:31.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: Active manager daemon x restarted 2024-04-02T13:53:31.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: Activating manager daemon x 2024-04-02T13:53:31.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: osdmap e93: 12 total, 12 up, 12 in 2024-04-02T13:53:31.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: mgrmap e21: x(active, starting, since 0.0106068s), standbys: y, z 2024-04-02T13:53:31.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:53:31.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:53:31.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:53:31.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T13:53:31.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T13:53:31.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T13:53:31.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T13:53:31.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T13:53:31.040 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T13:53:31.040 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T13:53:31.040 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T13:53:31.040 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T13:53:31.040 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T13:53:31.040 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T13:53:31.040 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T13:53:31.040 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T13:53:31.040 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T13:53:31.040 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T13:53:31.040 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:53:31.040 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:53:31.040 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:53:31.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:53:31.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:53:31.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:53:31.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:53:31.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:53:31.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:53:31.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:53:31.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:53:31.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:53:31.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T13:53:31.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T13:53:31.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T13:53:31.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: Manager daemon x is now available 2024-04-02T13:53:31.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:53:31.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T13:53:31.042 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:30 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T13:53:31.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.073+0000 7fb92c52b640 1 -- 172.21.15.52:0/2708029005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9241025d0 msgr2=0x7fb924108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:31.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.073+0000 7fb92c52b640 1 --2- 172.21.15.52:0/2708029005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9241025d0 0x7fb924108e60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb914009ca0 tx=0x7fb91402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.074+0000 7fb92c52b640 1 -- 172.21.15.52:0/2708029005 shutdown_connections 2024-04-02T13:53:31.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.074+0000 7fb92c52b640 1 --2- 172.21.15.52:0/2708029005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9241025d0 0x7fb924108e60 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.074+0000 7fb92c52b640 1 --2- 172.21.15.52:0/2708029005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb924101c10 0x7fb924102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.074+0000 7fb92c52b640 1 --2- 172.21.15.52:0/2708029005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb924100a20 0x7fb924100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.074+0000 7fb92c52b640 1 -- 172.21.15.52:0/2708029005 >> 172.21.15.52:0/2708029005 conn(0x7fb9240fbf90 msgr2=0x7fb9240fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:31.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.074+0000 7fb92c52b640 1 -- 172.21.15.52:0/2708029005 shutdown_connections 2024-04-02T13:53:31.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.074+0000 7fb92c52b640 1 -- 172.21.15.52:0/2708029005 wait complete. 2024-04-02T13:53:31.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.074+0000 7fb92c52b640 1 Processor -- start 2024-04-02T13:53:31.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.074+0000 7fb92c52b640 1 -- start start 2024-04-02T13:53:31.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.074+0000 7fb92c52b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb924100a20 0x7fb924197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:31.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.074+0000 7fb92c52b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb924101c10 0x7fb924198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:31.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.074+0000 7fb92c52b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9241025d0 0x7fb92419f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:31.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.074+0000 7fb92c52b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb92410c6a0 con 0x7fb924101c10 2024-04-02T13:53:31.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.074+0000 7fb92c52b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb92410c520 con 0x7fb924100a20 2024-04-02T13:53:31.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.074+0000 7fb92c52b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb92410c820 con 0x7fb9241025d0 2024-04-02T13:53:31.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.074+0000 7fb929ac6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb924101c10 0x7fb924198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:31.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.074+0000 7fb929ac6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb924101c10 0x7fb924198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43712/0 (socket says 172.21.15.52:43712) 2024-04-02T13:53:31.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.074+0000 7fb929ac6640 1 -- 172.21.15.52:0/488710764 learned_addr learned my addr 172.21.15.52:0/488710764 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:31.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.074+0000 7fb92a2c7640 1 --2- 172.21.15.52:0/488710764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb924100a20 0x7fb924197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:31.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.075+0000 7fb92aac8640 1 --2- 172.21.15.52:0/488710764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9241025d0 0x7fb92419f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:31.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.075+0000 7fb929ac6640 1 -- 172.21.15.52:0/488710764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9241025d0 msgr2=0x7fb92419f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:31.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.075+0000 7fb929ac6640 1 --2- 172.21.15.52:0/488710764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9241025d0 0x7fb92419f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.075+0000 7fb929ac6640 1 -- 172.21.15.52:0/488710764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb924100a20 msgr2=0x7fb924197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:31.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.075+0000 7fb929ac6640 1 --2- 172.21.15.52:0/488710764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb924100a20 0x7fb924197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.075+0000 7fb929ac6640 1 -- 172.21.15.52:0/488710764 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92419fa30 con 0x7fb924101c10 2024-04-02T13:53:31.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.075+0000 7fb92aac8640 1 --2- 172.21.15.52:0/488710764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9241025d0 0x7fb92419f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:31.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.075+0000 7fb92a2c7640 1 --2- 172.21.15.52:0/488710764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb924100a20 0x7fb924197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:31.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.075+0000 7fb929ac6640 1 --2- 172.21.15.52:0/488710764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb924101c10 0x7fb924198350 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb91800d950 tx=0x7fb91800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:31.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.075+0000 7fb9237fe640 1 -- 172.21.15.52:0/488710764 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918017070 con 0x7fb924101c10 2024-04-02T13:53:31.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.075+0000 7fb92c52b640 1 -- 172.21.15.52:0/488710764 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92419fd20 con 0x7fb924101c10 2024-04-02T13:53:31.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.075+0000 7fb9237fe640 1 -- 172.21.15.52:0/488710764 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb918009c30 con 0x7fb924101c10 2024-04-02T13:53:31.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.075+0000 7fb9237fe640 1 -- 172.21.15.52:0/488710764 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91800f6b0 con 0x7fb924101c10 2024-04-02T13:53:31.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.075+0000 7fb92c52b640 1 -- 172.21.15.52:0/488710764 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9241a0210 con 0x7fb924101c10 2024-04-02T13:53:31.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.076+0000 7fb92c52b640 1 -- 172.21.15.52:0/488710764 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9241a0440 con 0x7fb924101c10 2024-04-02T13:53:31.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.077+0000 7fb9237fe640 1 -- 172.21.15.52:0/488710764 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fb9180109d0 con 0x7fb924101c10 2024-04-02T13:53:31.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.077+0000 7fb9237fe640 1 --2- 172.21.15.52:0/488710764 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb90c0a39b0 0x7fb90c0a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:31.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.078+0000 7fb9237fe640 1 -- 172.21.15.52:0/488710764 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb9180d00b0 con 0x7fb924101c10 2024-04-02T13:53:31.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.078+0000 7fb92a2c7640 1 --2- 172.21.15.52:0/488710764 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb90c0a39b0 0x7fb90c0a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:31.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.078+0000 7fb92a2c7640 1 --2- 172.21.15.52:0/488710764 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb90c0a39b0 0x7fb90c0a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb91c005fd0 tx=0x7fb91c00b390 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:31.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.081+0000 7fb9237fe640 1 -- 172.21.15.52:0/488710764 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb918009de0 con 0x7fb924101c10 2024-04-02T13:53:31.177 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:30.916+0000 7ff83dd24700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:53:31.178 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:30.916+0000 7ff83dd24700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:53:31.178 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:30.916+0000 7ff83dd24700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:53:31.178 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:30.916+0000 7ff83dd24700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:53:31.178 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:53:30 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:53:30.916+0000 7ff83dd24700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:53:31.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.269+0000 7fb92c52b640 1 -- 172.21.15.52:0/488710764 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9241a07d0 con 0x7fb924101c10 2024-04-02T13:53:31.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.271+0000 7fb9237fe640 1 -- 172.21.15.52:0/488710764 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24926 (secure 0 0 0) 0x7fb91809c920 con 0x7fb924101c10 2024-04-02T13:53:31.275 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:53:31.275 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":93,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T13:53:29.889192+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:50:58.778690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:50:59.561527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:51:03.343487+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/713755826":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/1937276046":"2024-04-03T13:53:29.889159+0000","172.21.15.52:6800/3836085837":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/2957262850":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/3836085837":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/2985814329":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:53:31.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.273+0000 7fb92c52b640 1 -- 172.21.15.52:0/488710764 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb90c0a39b0 msgr2=0x7fb90c0a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:31.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.273+0000 7fb92c52b640 1 --2- 172.21.15.52:0/488710764 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb90c0a39b0 0x7fb90c0a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb91c005fd0 tx=0x7fb91c00b390 comp rx=0 tx=0).stop 2024-04-02T13:53:31.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.273+0000 7fb92c52b640 1 -- 172.21.15.52:0/488710764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb924101c10 msgr2=0x7fb924198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:31.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.273+0000 7fb92c52b640 1 --2- 172.21.15.52:0/488710764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb924101c10 0x7fb924198350 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb91800d950 tx=0x7fb91800de10 comp rx=0 tx=0).stop 2024-04-02T13:53:31.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.273+0000 7fb92c52b640 1 -- 172.21.15.52:0/488710764 shutdown_connections 2024-04-02T13:53:31.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.273+0000 7fb92c52b640 1 --2- 172.21.15.52:0/488710764 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb90c0a39b0 0x7fb90c0a5e70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.273+0000 7fb92c52b640 1 --2- 172.21.15.52:0/488710764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9241025d0 0x7fb92419f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.273+0000 7fb92c52b640 1 --2- 172.21.15.52:0/488710764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb924101c10 0x7fb924198350 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.273+0000 7fb92c52b640 1 --2- 172.21.15.52:0/488710764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb924100a20 0x7fb924197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.273+0000 7fb92c52b640 1 -- 172.21.15.52:0/488710764 >> 172.21.15.52:0/488710764 conn(0x7fb9240fbf90 msgr2=0x7fb924068c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:31.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.273+0000 7fb92c52b640 1 -- 172.21.15.52:0/488710764 shutdown_connections 2024-04-02T13:53:31.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.273+0000 7fb92c52b640 1 -- 172.21.15.52:0/488710764 wait complete. 2024-04-02T13:53:31.289 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:46:20.746053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:50:58.778690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:50:59.561527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:51:03.343487+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:53:31.290 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:53:31.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.395+0000 7f299f0d3640 1 -- 172.21.15.52:0/3831810800 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2998101b30 msgr2=0x7f2998101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:31.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.395+0000 7f299f0d3640 1 --2- 172.21.15.52:0/3831810800 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2998101b30 0x7f2998101f50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2990009ca0 tx=0x7f299002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.395+0000 7f299f0d3640 1 -- 172.21.15.52:0/3831810800 shutdown_connections 2024-04-02T13:53:31.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.395+0000 7f299f0d3640 1 --2- 172.21.15.52:0/3831810800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29981036e0 0x7f2998109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.395+0000 7f299f0d3640 1 --2- 172.21.15.52:0/3831810800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2998102d20 0x7f29981031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.395+0000 7f299f0d3640 1 --2- 172.21.15.52:0/3831810800 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2998101b30 0x7f2998101f50 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.395+0000 7f299f0d3640 1 -- 172.21.15.52:0/3831810800 >> 172.21.15.52:0/3831810800 conn(0x7f29980fd0a0 msgr2=0x7f29980ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:31.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.395+0000 7f299f0d3640 1 -- 172.21.15.52:0/3831810800 shutdown_connections 2024-04-02T13:53:31.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.395+0000 7f299f0d3640 1 -- 172.21.15.52:0/3831810800 wait complete. 2024-04-02T13:53:31.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.396+0000 7f299f0d3640 1 Processor -- start 2024-04-02T13:53:31.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.396+0000 7f299f0d3640 1 -- start start 2024-04-02T13:53:31.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.396+0000 7f299f0d3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2998101b30 0x7f2998197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:31.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.396+0000 7f299f0d3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2998102d20 0x7f29981983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:31.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.396+0000 7f299f0d3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29981036e0 0x7f299819f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:31.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.396+0000 7f299f0d3640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f299810d7a0 con 0x7f29981036e0 2024-04-02T13:53:31.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.396+0000 7f299f0d3640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f299810d620 con 0x7f2998101b30 2024-04-02T13:53:31.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.396+0000 7f299f0d3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f299810d920 con 0x7f2998102d20 2024-04-02T13:53:31.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.396+0000 7f299d670640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29981036e0 0x7f299819f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:31.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.396+0000 7f299d670640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29981036e0 0x7f299819f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43722/0 (socket says 172.21.15.52:43722) 2024-04-02T13:53:31.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.396+0000 7f299d670640 1 -- 172.21.15.52:0/1793515580 learned_addr learned my addr 172.21.15.52:0/1793515580 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:31.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.396+0000 7f299ce6f640 1 --2- 172.21.15.52:0/1793515580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2998101b30 0x7f2998197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:31.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.396+0000 7f2997fff640 1 --2- 172.21.15.52:0/1793515580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2998102d20 0x7f29981983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:31.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.396+0000 7f299d670640 1 -- 172.21.15.52:0/1793515580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2998102d20 msgr2=0x7f29981983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:31.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.396+0000 7f299d670640 1 --2- 172.21.15.52:0/1793515580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2998102d20 0x7f29981983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.396+0000 7f299d670640 1 -- 172.21.15.52:0/1793515580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2998101b30 msgr2=0x7f2998197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:31.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.396+0000 7f299d670640 1 --2- 172.21.15.52:0/1793515580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2998101b30 0x7f2998197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.396+0000 7f299d670640 1 -- 172.21.15.52:0/1793515580 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299819fa90 con 0x7f29981036e0 2024-04-02T13:53:31.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.396+0000 7f299ce6f640 1 --2- 172.21.15.52:0/1793515580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2998101b30 0x7f2998197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:31.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.396+0000 7f2997fff640 1 --2- 172.21.15.52:0/1793515580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2998102d20 0x7f29981983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:31.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.396+0000 7f299d670640 1 --2- 172.21.15.52:0/1793515580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29981036e0 0x7f299819f420 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f298800d750 tx=0x7f298800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:31.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.396+0000 7f2995ffb640 1 -- 172.21.15.52:0/1793515580 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988017070 con 0x7f29981036e0 2024-04-02T13:53:31.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.397+0000 7f299f0d3640 1 -- 172.21.15.52:0/1793515580 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299819fd80 con 0x7f29981036e0 2024-04-02T13:53:31.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.397+0000 7f2995ffb640 1 -- 172.21.15.52:0/1793515580 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2988005500 con 0x7f29981036e0 2024-04-02T13:53:31.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.397+0000 7f2995ffb640 1 -- 172.21.15.52:0/1793515580 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988004d90 con 0x7f29981036e0 2024-04-02T13:53:31.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.397+0000 7f299f0d3640 1 -- 172.21.15.52:0/1793515580 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29981a0270 con 0x7f29981036e0 2024-04-02T13:53:31.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.398+0000 7f299f0d3640 1 -- 172.21.15.52:0/1793515580 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2998108ef0 con 0x7f29981036e0 2024-04-02T13:53:31.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.399+0000 7f2995ffb640 1 -- 172.21.15.52:0/1793515580 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f2988004f30 con 0x7f29981036e0 2024-04-02T13:53:31.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.399+0000 7f2995ffb640 1 --2- 172.21.15.52:0/1793515580 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f29840a3a30 0x7f29840a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:31.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.399+0000 7f299ce6f640 1 --2- 172.21.15.52:0/1793515580 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f29840a3a30 0x7f29840a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:31.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.399+0000 7f2995ffb640 1 -- 172.21.15.52:0/1793515580 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f29880cfe90 con 0x7f29981036e0 2024-04-02T13:53:31.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.400+0000 7f299ce6f640 1 --2- 172.21.15.52:0/1793515580 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f29840a3a30 0x7f29840a5ef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f299002c700 tx=0x7f29900325b0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:31.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.404+0000 7f2995ffb640 1 -- 172.21.15.52:0/1793515580 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f298809c700 con 0x7f29981036e0 2024-04-02T13:53:31.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.590+0000 7f299f0d3640 1 -- 172.21.15.52:0/1793515580 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f29980627a0 con 0x7f29981036e0 2024-04-02T13:53:31.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.590+0000 7f2995ffb640 1 -- 172.21.15.52:0/1793515580 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f298809c000 con 0x7f29981036e0 2024-04-02T13:53:31.594 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-04-02T13:53:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.592+0000 7f299f0d3640 1 -- 172.21.15.52:0/1793515580 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f29840a3a30 msgr2=0x7f29840a5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.592+0000 7f299f0d3640 1 --2- 172.21.15.52:0/1793515580 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f29840a3a30 0x7f29840a5ef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f299002c700 tx=0x7f29900325b0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.592+0000 7f299f0d3640 1 -- 172.21.15.52:0/1793515580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29981036e0 msgr2=0x7f299819f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.592+0000 7f299f0d3640 1 --2- 172.21.15.52:0/1793515580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29981036e0 0x7f299819f420 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f298800d750 tx=0x7f298800dc10 comp rx=0 tx=0).stop 2024-04-02T13:53:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.592+0000 7f299f0d3640 1 -- 172.21.15.52:0/1793515580 shutdown_connections 2024-04-02T13:53:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.592+0000 7f299f0d3640 1 --2- 172.21.15.52:0/1793515580 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f29840a3a30 0x7f29840a5ef0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.592+0000 7f299f0d3640 1 --2- 172.21.15.52:0/1793515580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29981036e0 0x7f299819f420 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.592+0000 7f299f0d3640 1 --2- 172.21.15.52:0/1793515580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2998102d20 0x7f29981983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.592+0000 7f299f0d3640 1 --2- 172.21.15.52:0/1793515580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2998101b30 0x7f2998197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.592+0000 7f299f0d3640 1 -- 172.21.15.52:0/1793515580 >> 172.21.15.52:0/1793515580 conn(0x7f29980fd0a0 msgr2=0x7f29981060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.592+0000 7f299f0d3640 1 -- 172.21.15.52:0/1793515580 shutdown_connections 2024-04-02T13:53:31.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.593+0000 7f299f0d3640 1 -- 172.21.15.52:0/1793515580 wait complete. 2024-04-02T13:53:31.606 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:53:31.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.715+0000 7f5b76029640 1 -- 172.21.15.52:0/330084830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b701025d0 msgr2=0x7f5b70108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:31.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.715+0000 7f5b76029640 1 --2- 172.21.15.52:0/330084830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b701025d0 0x7f5b70108e60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009ca0 tx=0x7f5b6802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.715+0000 7f5b76029640 1 -- 172.21.15.52:0/330084830 shutdown_connections 2024-04-02T13:53:31.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.715+0000 7f5b76029640 1 --2- 172.21.15.52:0/330084830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b701025d0 0x7f5b70108e60 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.715+0000 7f5b76029640 1 --2- 172.21.15.52:0/330084830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b70101c10 0x7f5b70102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.715+0000 7f5b76029640 1 --2- 172.21.15.52:0/330084830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b70100a20 0x7f5b70100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.715+0000 7f5b76029640 1 -- 172.21.15.52:0/330084830 >> 172.21.15.52:0/330084830 conn(0x7f5b700fbfb0 msgr2=0x7f5b700fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:31.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.715+0000 7f5b76029640 1 -- 172.21.15.52:0/330084830 shutdown_connections 2024-04-02T13:53:31.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.715+0000 7f5b76029640 1 -- 172.21.15.52:0/330084830 wait complete. 2024-04-02T13:53:31.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.716+0000 7f5b76029640 1 Processor -- start 2024-04-02T13:53:31.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.716+0000 7f5b76029640 1 -- start start 2024-04-02T13:53:31.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.716+0000 7f5b76029640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b70100a20 0x7f5b70197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:31.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.716+0000 7f5b76029640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b70101c10 0x7f5b70198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:31.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.716+0000 7f5b76029640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b701025d0 0x7f5b7019f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:31.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.716+0000 7f5b76029640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7010c680 con 0x7f5b70101c10 2024-04-02T13:53:31.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.716+0000 7f5b76029640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7010c500 con 0x7f5b70100a20 2024-04-02T13:53:31.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.716+0000 7f5b76029640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7010c800 con 0x7f5b701025d0 2024-04-02T13:53:31.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.716+0000 7f5b6effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b70101c10 0x7f5b70198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:31.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.716+0000 7f5b6effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b70101c10 0x7f5b70198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58242/0 (socket says 172.21.15.52:58242) 2024-04-02T13:53:31.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.716+0000 7f5b6effd640 1 -- 172.21.15.52:0/1172131276 learned_addr learned my addr 172.21.15.52:0/1172131276 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:31.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.716+0000 7f5b6f7fe640 1 --2- 172.21.15.52:0/1172131276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b70100a20 0x7f5b70197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:31.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.716+0000 7f5b6ffff640 1 --2- 172.21.15.52:0/1172131276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b701025d0 0x7f5b7019f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:31.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.716+0000 7f5b6effd640 1 -- 172.21.15.52:0/1172131276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b701025d0 msgr2=0x7f5b7019f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:31.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.716+0000 7f5b6effd640 1 --2- 172.21.15.52:0/1172131276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b701025d0 0x7f5b7019f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.716+0000 7f5b6effd640 1 -- 172.21.15.52:0/1172131276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b70100a20 msgr2=0x7f5b70197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:31.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.716+0000 7f5b6effd640 1 --2- 172.21.15.52:0/1172131276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b70100a20 0x7f5b70197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.717+0000 7f5b6effd640 1 -- 172.21.15.52:0/1172131276 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7019fa00 con 0x7f5b70101c10 2024-04-02T13:53:31.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.717+0000 7f5b6effd640 1 --2- 172.21.15.52:0/1172131276 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b70101c10 0x7f5b70198320 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400ea60 tx=0x7f5b6400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:31.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.717+0000 7f5b6cff9640 1 -- 172.21.15.52:0/1172131276 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6400cde0 con 0x7f5b70101c10 2024-04-02T13:53:31.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.717+0000 7f5b76029640 1 -- 172.21.15.52:0/1172131276 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b7019fcf0 con 0x7f5b70101c10 2024-04-02T13:53:31.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.717+0000 7f5b6cff9640 1 -- 172.21.15.52:0/1172131276 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b640035f0 con 0x7f5b70101c10 2024-04-02T13:53:31.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.717+0000 7f5b6cff9640 1 -- 172.21.15.52:0/1172131276 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b64010670 con 0x7f5b70101c10 2024-04-02T13:53:31.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.717+0000 7f5b76029640 1 -- 172.21.15.52:0/1172131276 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b701a0230 con 0x7f5b70101c10 2024-04-02T13:53:31.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.718+0000 7f5b76029640 1 -- 172.21.15.52:0/1172131276 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b701a0520 con 0x7f5b70101c10 2024-04-02T13:53:31.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.719+0000 7f5b6cff9640 1 -- 172.21.15.52:0/1172131276 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f5b640037a0 con 0x7f5b70101c10 2024-04-02T13:53:31.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.720+0000 7f5b6cff9640 1 --2- 172.21.15.52:0/1172131276 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5b580a39b0 0x7f5b580a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:31.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.720+0000 7f5b6f7fe640 1 --2- 172.21.15.52:0/1172131276 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5b580a39b0 0x7f5b580a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:31.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.720+0000 7f5b6cff9640 1 -- 172.21.15.52:0/1172131276 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f5b640d07e0 con 0x7f5b70101c10 2024-04-02T13:53:31.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.720+0000 7f5b6f7fe640 1 --2- 172.21.15.52:0/1172131276 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5b580a39b0 0x7f5b580a5e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5b70101a70 tx=0x7f5b60005f90 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:31.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.725+0000 7f5b6cff9640 1 -- 172.21.15.52:0/1172131276 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5b6409d0e0 con 0x7f5b70101c10 2024-04-02T13:53:31.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.915+0000 7f5b6cff9640 1 -- 172.21.15.52:0/1172131276 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f5b64099050 con 0x7f5b70101c10 2024-04-02T13:53:31.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.948+0000 7f5b76029640 1 -- 172.21.15.52:0/1172131276 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5b701a06f0 con 0x7f5b70101c10 2024-04-02T13:53:31.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.948+0000 7f5b6cff9640 1 -- 172.21.15.52:0/1172131276 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f5b64098150 con 0x7f5b70101c10 2024-04-02T13:53:31.952 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-04-02T13:53:31.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.950+0000 7f5b76029640 1 -- 172.21.15.52:0/1172131276 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5b580a39b0 msgr2=0x7f5b580a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:31.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.950+0000 7f5b76029640 1 --2- 172.21.15.52:0/1172131276 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5b580a39b0 0x7f5b580a5e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5b70101a70 tx=0x7f5b60005f90 comp rx=0 tx=0).stop 2024-04-02T13:53:31.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.950+0000 7f5b76029640 1 -- 172.21.15.52:0/1172131276 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b70101c10 msgr2=0x7f5b70198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:31.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.950+0000 7f5b76029640 1 --2- 172.21.15.52:0/1172131276 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b70101c10 0x7f5b70198320 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400ea60 tx=0x7f5b6400ef20 comp rx=0 tx=0).stop 2024-04-02T13:53:31.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.950+0000 7f5b76029640 1 -- 172.21.15.52:0/1172131276 shutdown_connections 2024-04-02T13:53:31.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.950+0000 7f5b76029640 1 --2- 172.21.15.52:0/1172131276 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5b580a39b0 0x7f5b580a5e70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.951+0000 7f5b76029640 1 --2- 172.21.15.52:0/1172131276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b701025d0 0x7f5b7019f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.951+0000 7f5b76029640 1 --2- 172.21.15.52:0/1172131276 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b70101c10 0x7f5b70198320 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.951+0000 7f5b76029640 1 --2- 172.21.15.52:0/1172131276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b70100a20 0x7f5b70197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:31.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.951+0000 7f5b76029640 1 -- 172.21.15.52:0/1172131276 >> 172.21.15.52:0/1172131276 conn(0x7f5b700fbfb0 msgr2=0x7f5b700fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:31.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.951+0000 7f5b76029640 1 -- 172.21.15.52:0/1172131276 shutdown_connections 2024-04-02T13:53:31.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:31.951+0000 7f5b76029640 1 -- 172.21.15.52:0/1172131276 wait complete. 2024-04-02T13:53:31.965 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:53:32.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.085+0000 7f1993293640 1 -- 172.21.15.52:0/2768585329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f198c103750 msgr2=0x7f198c109fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:32.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.085+0000 7f1993293640 1 --2- 172.21.15.52:0/2768585329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f198c103750 0x7f198c109fe0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1984009cd0 tx=0x7f198402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.085+0000 7f1993293640 1 -- 172.21.15.52:0/2768585329 shutdown_connections 2024-04-02T13:53:32.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.085+0000 7f1993293640 1 --2- 172.21.15.52:0/2768585329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f198c103750 0x7f198c109fe0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.085+0000 7f1993293640 1 --2- 172.21.15.52:0/2768585329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f198c102d90 0x7f198c103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.085+0000 7f1993293640 1 --2- 172.21.15.52:0/2768585329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f198c0ffdc0 0x7f198c1001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.085+0000 7f1993293640 1 -- 172.21.15.52:0/2768585329 >> 172.21.15.52:0/2768585329 conn(0x7f198c0fb6f0 msgr2=0x7f198c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:32.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.085+0000 7f1993293640 1 -- 172.21.15.52:0/2768585329 shutdown_connections 2024-04-02T13:53:32.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.085+0000 7f1993293640 1 -- 172.21.15.52:0/2768585329 wait complete. 2024-04-02T13:53:32.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.086+0000 7f1993293640 1 Processor -- start 2024-04-02T13:53:32.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.086+0000 7f1993293640 1 -- start start 2024-04-02T13:53:32.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.086+0000 7f1993293640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f198c0ffdc0 0x7f198c197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:32.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.086+0000 7f1993293640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f198c102d90 0x7f198c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:32.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.086+0000 7f1993293640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f198c103750 0x7f198c19f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:32.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.086+0000 7f1993293640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f198c10d7a0 con 0x7f198c103750 2024-04-02T13:53:32.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.086+0000 7f1993293640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f198c10d620 con 0x7f198c0ffdc0 2024-04-02T13:53:32.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.086+0000 7f1993293640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f198c10d920 con 0x7f198c102d90 2024-04-02T13:53:32.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.086+0000 7f1991830640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f198c103750 0x7f198c19f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:32.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.086+0000 7f199102f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f198c0ffdc0 0x7f198c197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:32.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.086+0000 7f1991830640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f198c103750 0x7f198c19f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58266/0 (socket says 172.21.15.52:58266) 2024-04-02T13:53:32.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.086+0000 7f1991830640 1 -- 172.21.15.52:0/3266463897 learned_addr learned my addr 172.21.15.52:0/3266463897 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:32.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.086+0000 7f199082e640 1 --2- 172.21.15.52:0/3266463897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f198c102d90 0x7f198c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:32.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.086+0000 7f1991830640 1 -- 172.21.15.52:0/3266463897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f198c102d90 msgr2=0x7f198c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:32.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.086+0000 7f1991830640 1 --2- 172.21.15.52:0/3266463897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f198c102d90 0x7f198c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.086+0000 7f1991830640 1 -- 172.21.15.52:0/3266463897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f198c0ffdc0 msgr2=0x7f198c197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:32.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.086+0000 7f1991830640 1 --2- 172.21.15.52:0/3266463897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f198c0ffdc0 0x7f198c197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.086+0000 7f1991830640 1 -- 172.21.15.52:0/3266463897 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198c19fa40 con 0x7f198c103750 2024-04-02T13:53:32.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.086+0000 7f199102f640 1 --2- 172.21.15.52:0/3266463897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f198c0ffdc0 0x7f198c197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:32.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.087+0000 7f1991830640 1 --2- 172.21.15.52:0/3266463897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f198c103750 0x7f198c19f3d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1984009ca0 tx=0x7f19840346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:32.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.087+0000 7f19827fc640 1 -- 172.21.15.52:0/3266463897 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1984041020 con 0x7f198c103750 2024-04-02T13:53:32.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.087+0000 7f1993293640 1 -- 172.21.15.52:0/3266463897 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f198c19fcd0 con 0x7f198c103750 2024-04-02T13:53:32.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.087+0000 7f19827fc640 1 -- 172.21.15.52:0/3266463897 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1984032890 con 0x7f198c103750 2024-04-02T13:53:32.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.087+0000 7f19827fc640 1 -- 172.21.15.52:0/3266463897 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1984039040 con 0x7f198c103750 2024-04-02T13:53:32.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.087+0000 7f1993293640 1 -- 172.21.15.52:0/3266463897 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f198c1a01c0 con 0x7f198c103750 2024-04-02T13:53:32.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.088+0000 7f1993293640 1 -- 172.21.15.52:0/3266463897 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f198c108f40 con 0x7f198c103750 2024-04-02T13:53:32.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.089+0000 7f19827fc640 1 -- 172.21.15.52:0/3266463897 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f1984044020 con 0x7f198c103750 2024-04-02T13:53:32.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.090+0000 7f19827fc640 1 --2- 172.21.15.52:0/3266463897 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f19780a3db0 0x7f19780a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:32.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.090+0000 7f199102f640 1 --2- 172.21.15.52:0/3266463897 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f19780a3db0 0x7f19780a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:32.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.090+0000 7f19827fc640 1 -- 172.21.15.52:0/3266463897 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f19840f2d30 con 0x7f198c103750 2024-04-02T13:53:32.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.090+0000 7f199102f640 1 --2- 172.21.15.52:0/3266463897 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f19780a3db0 0x7f19780a6270 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f198c068a60 tx=0x7f1988005f90 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:32.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.094+0000 7f19827fc640 1 -- 172.21.15.52:0/3266463897 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f198404b020 con 0x7f198c103750 2024-04-02T13:53:32.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:31 smithi088 ceph-mon[36528]: [02/Apr/2024:13:53:30] ENGINE Bus STARTING 2024-04-02T13:53:32.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:31 smithi088 ceph-mon[36528]: [02/Apr/2024:13:53:30] ENGINE Serving on http://172.21.15.52:8765 2024-04-02T13:53:32.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:31 smithi088 ceph-mon[36528]: mgrmap e22: x(active, since 1.01924s), standbys: y, z 2024-04-02T13:53:32.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:31 smithi088 ceph-mon[36528]: from='client.25190 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:32.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:31 smithi088 ceph-mon[36528]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:53:32.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:31 smithi088 ceph-mon[36528]: [02/Apr/2024:13:53:30] ENGINE Serving on https://172.21.15.52:7150 2024-04-02T13:53:32.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:31 smithi088 ceph-mon[36528]: [02/Apr/2024:13:53:30] ENGINE Bus STARTED 2024-04-02T13:53:32.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:31 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/488710764' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:53:32.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:31 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1793515580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:53:32.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:31 smithi189 ceph-mon[36683]: [02/Apr/2024:13:53:30] ENGINE Bus STARTING 2024-04-02T13:53:32.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:31 smithi189 ceph-mon[36683]: [02/Apr/2024:13:53:30] ENGINE Serving on http://172.21.15.52:8765 2024-04-02T13:53:32.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:31 smithi189 ceph-mon[36683]: mgrmap e22: x(active, since 1.01924s), standbys: y, z 2024-04-02T13:53:32.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:31 smithi189 ceph-mon[36683]: from='client.25190 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:32.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:31 smithi189 ceph-mon[36683]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:53:32.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:31 smithi189 ceph-mon[36683]: [02/Apr/2024:13:53:30] ENGINE Serving on https://172.21.15.52:7150 2024-04-02T13:53:32.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:31 smithi189 ceph-mon[36683]: [02/Apr/2024:13:53:30] ENGINE Bus STARTED 2024-04-02T13:53:32.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:31 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/488710764' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:53:32.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:31 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1793515580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:53:32.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.287+0000 7f1993293640 1 -- 172.21.15.52:0/3266463897 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f198c02d370 con 0x7f198c103750 2024-04-02T13:53:32.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.288+0000 7f19827fc640 1 -- 172.21.15.52:0/3266463897 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f19840bf030 con 0x7f198c103750 2024-04-02T13:53:32.291 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-04-02T13:53:32.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.290+0000 7f1993293640 1 -- 172.21.15.52:0/3266463897 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f19780a3db0 msgr2=0x7f19780a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:32.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.290+0000 7f1993293640 1 --2- 172.21.15.52:0/3266463897 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f19780a3db0 0x7f19780a6270 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f198c068a60 tx=0x7f1988005f90 comp rx=0 tx=0).stop 2024-04-02T13:53:32.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.290+0000 7f1993293640 1 -- 172.21.15.52:0/3266463897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f198c103750 msgr2=0x7f198c19f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:32.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.290+0000 7f1993293640 1 --2- 172.21.15.52:0/3266463897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f198c103750 0x7f198c19f3d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1984009ca0 tx=0x7f19840346d0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.290+0000 7f1993293640 1 -- 172.21.15.52:0/3266463897 shutdown_connections 2024-04-02T13:53:32.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.290+0000 7f1993293640 1 --2- 172.21.15.52:0/3266463897 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f19780a3db0 0x7f19780a6270 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.290+0000 7f1993293640 1 --2- 172.21.15.52:0/3266463897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f198c103750 0x7f198c19f3d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.290+0000 7f1993293640 1 --2- 172.21.15.52:0/3266463897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f198c102d90 0x7f198c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.290+0000 7f1993293640 1 --2- 172.21.15.52:0/3266463897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f198c0ffdc0 0x7f198c197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.290+0000 7f1993293640 1 -- 172.21.15.52:0/3266463897 >> 172.21.15.52:0/3266463897 conn(0x7f198c0fb6f0 msgr2=0x7f198c106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:32.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.290+0000 7f1993293640 1 -- 172.21.15.52:0/3266463897 shutdown_connections 2024-04-02T13:53:32.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.290+0000 7f1993293640 1 -- 172.21.15.52:0/3266463897 wait complete. 2024-04-02T13:53:32.303 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:31 smithi052 ceph-mon[32104]: [02/Apr/2024:13:53:30] ENGINE Bus STARTING 2024-04-02T13:53:32.303 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:31 smithi052 ceph-mon[32104]: [02/Apr/2024:13:53:30] ENGINE Serving on http://172.21.15.52:8765 2024-04-02T13:53:32.304 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:31 smithi052 ceph-mon[32104]: mgrmap e22: x(active, since 1.01924s), standbys: y, z 2024-04-02T13:53:32.304 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:31 smithi052 ceph-mon[32104]: from='client.25190 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:32.304 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:31 smithi052 ceph-mon[32104]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:53:32.304 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:31 smithi052 ceph-mon[32104]: [02/Apr/2024:13:53:30] ENGINE Serving on https://172.21.15.52:7150 2024-04-02T13:53:32.304 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:31 smithi052 ceph-mon[32104]: [02/Apr/2024:13:53:30] ENGINE Bus STARTED 2024-04-02T13:53:32.305 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:31 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/488710764' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:53:32.305 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:31 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1793515580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:53:32.306 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:53:32.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.410+0000 7fed40788640 1 -- 172.21.15.52:0/3846824185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed381077e0 msgr2=0x7fed38109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:32.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.410+0000 7fed40788640 1 --2- 172.21.15.52:0/3846824185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed381077e0 0x7fed38109c30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fed30009ca0 tx=0x7fed3002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.410+0000 7fed40788640 1 -- 172.21.15.52:0/3846824185 shutdown_connections 2024-04-02T13:53:32.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.410+0000 7fed40788640 1 --2- 172.21.15.52:0/3846824185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed381077e0 0x7fed38109c30 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.410+0000 7fed40788640 1 --2- 172.21.15.52:0/3846824185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed38104eb0 0x7fed381072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.410+0000 7fed40788640 1 --2- 172.21.15.52:0/3846824185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed38102580 0x7fed38104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.410+0000 7fed40788640 1 -- 172.21.15.52:0/3846824185 >> 172.21.15.52:0/3846824185 conn(0x7fed380fbf90 msgr2=0x7fed380fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:32.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.410+0000 7fed40788640 1 -- 172.21.15.52:0/3846824185 shutdown_connections 2024-04-02T13:53:32.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.410+0000 7fed40788640 1 -- 172.21.15.52:0/3846824185 wait complete. 2024-04-02T13:53:32.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.411+0000 7fed40788640 1 Processor -- start 2024-04-02T13:53:32.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.411+0000 7fed40788640 1 -- start start 2024-04-02T13:53:32.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.411+0000 7fed40788640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed38102580 0x7fed38197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:32.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.411+0000 7fed40788640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed38104eb0 0x7fed38198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:32.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.411+0000 7fed40788640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed381077e0 0x7fed3819f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:32.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.411+0000 7fed40788640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fed3810d7a0 con 0x7fed381077e0 2024-04-02T13:53:32.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.411+0000 7fed40788640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fed3810d620 con 0x7fed38102580 2024-04-02T13:53:32.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.411+0000 7fed40788640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fed3810d920 con 0x7fed38104eb0 2024-04-02T13:53:32.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.411+0000 7fed3ed25640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed381077e0 0x7fed3819f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:32.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.411+0000 7fed3e524640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed38102580 0x7fed38197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:32.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.411+0000 7fed3ed25640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed381077e0 0x7fed3819f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58292/0 (socket says 172.21.15.52:58292) 2024-04-02T13:53:32.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.411+0000 7fed3ed25640 1 -- 172.21.15.52:0/908197823 learned_addr learned my addr 172.21.15.52:0/908197823 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:32.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.411+0000 7fed3dd23640 1 --2- 172.21.15.52:0/908197823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed38104eb0 0x7fed38198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:32.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.411+0000 7fed3e524640 1 -- 172.21.15.52:0/908197823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed38104eb0 msgr2=0x7fed38198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:32.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.411+0000 7fed3e524640 1 --2- 172.21.15.52:0/908197823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed38104eb0 0x7fed38198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.411+0000 7fed3e524640 1 -- 172.21.15.52:0/908197823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed381077e0 msgr2=0x7fed3819f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:32.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.411+0000 7fed3e524640 1 --2- 172.21.15.52:0/908197823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed381077e0 0x7fed3819f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.411+0000 7fed3e524640 1 -- 172.21.15.52:0/908197823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed3819fa70 con 0x7fed38102580 2024-04-02T13:53:32.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.411+0000 7fed3ed25640 1 --2- 172.21.15.52:0/908197823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed381077e0 0x7fed3819f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:32.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.412+0000 7fed3e524640 1 --2- 172.21.15.52:0/908197823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed38102580 0x7fed38197e50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fed3400e9f0 tx=0x7fed3400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:32.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.412+0000 7fed2f7fe640 1 -- 172.21.15.52:0/908197823 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3400ccc0 con 0x7fed38102580 2024-04-02T13:53:32.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.412+0000 7fed40788640 1 -- 172.21.15.52:0/908197823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed3819fd60 con 0x7fed38102580 2024-04-02T13:53:32.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.412+0000 7fed2f7fe640 1 -- 172.21.15.52:0/908197823 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed34010910 con 0x7fed38102580 2024-04-02T13:53:32.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.412+0000 7fed2f7fe640 1 -- 172.21.15.52:0/908197823 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34018960 con 0x7fed38102580 2024-04-02T13:53:32.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.412+0000 7fed40788640 1 -- 172.21.15.52:0/908197823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed381a0220 con 0x7fed38102580 2024-04-02T13:53:32.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.413+0000 7fed40788640 1 -- 172.21.15.52:0/908197823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed380fda80 con 0x7fed38102580 2024-04-02T13:53:32.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.415+0000 7fed2f7fe640 1 -- 172.21.15.52:0/908197823 <== mon.2 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fed34010ac0 con 0x7fed38102580 2024-04-02T13:53:32.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.416+0000 7fed2f7fe640 1 --2- 172.21.15.52:0/908197823 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fed240a3db0 0x7fed240a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:32.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.416+0000 7fed3dd23640 1 --2- 172.21.15.52:0/908197823 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fed240a3db0 0x7fed240a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:32.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.416+0000 7fed2f7fe640 1 -- 172.21.15.52:0/908197823 <== mon.2 v2:172.21.15.88:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fed340d1580 con 0x7fed38102580 2024-04-02T13:53:32.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.416+0000 7fed3dd23640 1 --2- 172.21.15.52:0/908197823 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fed240a3db0 0x7fed240a6270 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fed38199360 tx=0x7fed28005f90 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:32.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.423+0000 7fed2f7fe640 1 -- 172.21.15.52:0/908197823 <== mon.2 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fed3409d0c0 con 0x7fed38102580 2024-04-02T13:53:32.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.653+0000 7fed40788640 1 -- 172.21.15.52:0/908197823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fed3802d590 con 0x7fed38102580 2024-04-02T13:53:32.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.654+0000 7fed2f7fe640 1 -- 172.21.15.52:0/908197823 <== mon.2 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7fed340a13c0 con 0x7fed38102580 2024-04-02T13:53:32.657 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-04-02T13:53:32.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.656+0000 7fed40788640 1 -- 172.21.15.52:0/908197823 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fed240a3db0 msgr2=0x7fed240a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:32.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.656+0000 7fed40788640 1 --2- 172.21.15.52:0/908197823 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fed240a3db0 0x7fed240a6270 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fed38199360 tx=0x7fed28005f90 comp rx=0 tx=0).stop 2024-04-02T13:53:32.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.656+0000 7fed40788640 1 -- 172.21.15.52:0/908197823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed38102580 msgr2=0x7fed38197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:32.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.656+0000 7fed40788640 1 --2- 172.21.15.52:0/908197823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed38102580 0x7fed38197e50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fed3400e9f0 tx=0x7fed3400eeb0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.656+0000 7fed40788640 1 -- 172.21.15.52:0/908197823 shutdown_connections 2024-04-02T13:53:32.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.656+0000 7fed40788640 1 --2- 172.21.15.52:0/908197823 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fed240a3db0 0x7fed240a6270 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.656+0000 7fed40788640 1 --2- 172.21.15.52:0/908197823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed381077e0 0x7fed3819f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.656+0000 7fed40788640 1 --2- 172.21.15.52:0/908197823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed38104eb0 0x7fed38198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.656+0000 7fed40788640 1 --2- 172.21.15.52:0/908197823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed38102580 0x7fed38197e50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.656+0000 7fed40788640 1 -- 172.21.15.52:0/908197823 >> 172.21.15.52:0/908197823 conn(0x7fed380fbf90 msgr2=0x7fed38102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:32.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.657+0000 7fed40788640 1 -- 172.21.15.52:0/908197823 shutdown_connections 2024-04-02T13:53:32.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.657+0000 7fed40788640 1 -- 172.21.15.52:0/908197823 wait complete. 2024-04-02T13:53:32.674 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:53:32.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.784+0000 7f5686196640 1 -- 172.21.15.52:0/359114200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5680100a20 msgr2=0x7f5680100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:32.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.784+0000 7f5686196640 1 --2- 172.21.15.52:0/359114200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5680100a20 0x7f5680100e40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5670009900 tx=0x7f567002bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:32.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.784+0000 7f5686196640 1 -- 172.21.15.52:0/359114200 shutdown_connections 2024-04-02T13:53:32.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.784+0000 7f5686196640 1 --2- 172.21.15.52:0/359114200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f56801025d0 0x7f5680108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.784+0000 7f5686196640 1 --2- 172.21.15.52:0/359114200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5680101c10 0x7f5680102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.784+0000 7f5686196640 1 --2- 172.21.15.52:0/359114200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5680100a20 0x7f5680100e40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.784+0000 7f5686196640 1 -- 172.21.15.52:0/359114200 >> 172.21.15.52:0/359114200 conn(0x7f56800fbf90 msgr2=0x7f56800fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:32.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.784+0000 7f5686196640 1 -- 172.21.15.52:0/359114200 shutdown_connections 2024-04-02T13:53:32.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.784+0000 7f5686196640 1 -- 172.21.15.52:0/359114200 wait complete. 2024-04-02T13:53:32.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.784+0000 7f5686196640 1 Processor -- start 2024-04-02T13:53:32.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.785+0000 7f5686196640 1 -- start start 2024-04-02T13:53:32.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.785+0000 7f5686196640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5680100a20 0x7f56801049b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:32.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.785+0000 7f5686196640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5680101c10 0x7f5680102ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:32.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.785+0000 7f5686196640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f56801025d0 0x7f5680103530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:32.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.785+0000 7f5686196640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f568010c6a0 con 0x7f5680100a20 2024-04-02T13:53:32.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.785+0000 7f5686196640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f568010c520 con 0x7f5680101c10 2024-04-02T13:53:32.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.785+0000 7f5686196640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f568010c820 con 0x7f56801025d0 2024-04-02T13:53:32.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.785+0000 7f567f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5680100a20 0x7f56801049b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:32.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.785+0000 7f567f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5680100a20 0x7f56801049b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58308/0 (socket says 172.21.15.52:58308) 2024-04-02T13:53:32.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.785+0000 7f567f7fe640 1 -- 172.21.15.52:0/3545880993 learned_addr learned my addr 172.21.15.52:0/3545880993 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:32.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.785+0000 7f567effd640 1 --2- 172.21.15.52:0/3545880993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5680101c10 0x7f5680102ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:32.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.785+0000 7f567ffff640 1 --2- 172.21.15.52:0/3545880993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f56801025d0 0x7f5680103530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:32.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.785+0000 7f567f7fe640 1 -- 172.21.15.52:0/3545880993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f56801025d0 msgr2=0x7f5680103530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:32.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.785+0000 7f567f7fe640 1 --2- 172.21.15.52:0/3545880993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f56801025d0 0x7f5680103530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.785+0000 7f567f7fe640 1 -- 172.21.15.52:0/3545880993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5680101c10 msgr2=0x7f5680102ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:32.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.785+0000 7f567f7fe640 1 --2- 172.21.15.52:0/3545880993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5680101c10 0x7f5680102ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.785+0000 7f567f7fe640 1 -- 172.21.15.52:0/3545880993 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5680103dc0 con 0x7f5680100a20 2024-04-02T13:53:32.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.786+0000 7f567f7fe640 1 --2- 172.21.15.52:0/3545880993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5680100a20 0x7f56801049b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5670000c00 tx=0x7f567002c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:32.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.786+0000 7f567cff9640 1 -- 172.21.15.52:0/3545880993 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670041070 con 0x7f5680100a20 2024-04-02T13:53:32.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.786+0000 7f5686196640 1 -- 172.21.15.52:0/3545880993 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56801a1f00 con 0x7f5680100a20 2024-04-02T13:53:32.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.786+0000 7f567cff9640 1 -- 172.21.15.52:0/3545880993 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5670004450 con 0x7f5680100a20 2024-04-02T13:53:32.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.786+0000 7f567cff9640 1 -- 172.21.15.52:0/3545880993 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670034440 con 0x7f5680100a20 2024-04-02T13:53:32.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.786+0000 7f5686196640 1 -- 172.21.15.52:0/3545880993 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56801a2310 con 0x7f5680100a20 2024-04-02T13:53:32.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.787+0000 7f5686196640 1 -- 172.21.15.52:0/3545880993 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56801a25a0 con 0x7f5680100a20 2024-04-02T13:53:32.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.788+0000 7f567cff9640 1 -- 172.21.15.52:0/3545880993 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f5670044070 con 0x7f5680100a20 2024-04-02T13:53:32.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.789+0000 7f567cff9640 1 --2- 172.21.15.52:0/3545880993 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f56680a3db0 0x7f56680a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:32.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.789+0000 7f567cff9640 1 -- 172.21.15.52:0/3545880993 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f56700f2620 con 0x7f5680100a20 2024-04-02T13:53:32.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.789+0000 7f567effd640 1 --2- 172.21.15.52:0/3545880993 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f56680a3db0 0x7f56680a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:32.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.789+0000 7f567effd640 1 --2- 172.21.15.52:0/3545880993 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f56680a3db0 0x7f56680a6270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5680104720 tx=0x7f5674005f90 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:32.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.793+0000 7f567cff9640 1 -- 172.21.15.52:0/3545880993 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f56700be360 con 0x7f5680100a20 2024-04-02T13:53:32.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.983+0000 7f5686196640 1 -- 172.21.15.52:0/3545880993 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56801a2a90 con 0x7f5680100a20 2024-04-02T13:53:32.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.984+0000 7f567cff9640 1 -- 172.21.15.52:0/3545880993 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24926 (secure 0 0 0) 0x7f56700c2660 con 0x7f5680100a20 2024-04-02T13:53:32.988 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:53:32.988 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":93,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T13:53:29.889192+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:50:58.778690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:50:59.561527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:51:03.343487+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/713755826":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/1937276046":"2024-04-03T13:53:29.889159+0000","172.21.15.52:6800/3836085837":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/2957262850":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/3836085837":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/2985814329":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:53:32.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.986+0000 7f5686196640 1 -- 172.21.15.52:0/3545880993 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f56680a3db0 msgr2=0x7f56680a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:32.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.986+0000 7f5686196640 1 --2- 172.21.15.52:0/3545880993 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f56680a3db0 0x7f56680a6270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5680104720 tx=0x7f5674005f90 comp rx=0 tx=0).stop 2024-04-02T13:53:32.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.986+0000 7f5686196640 1 -- 172.21.15.52:0/3545880993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5680100a20 msgr2=0x7f56801049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:32.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.986+0000 7f5686196640 1 --2- 172.21.15.52:0/3545880993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5680100a20 0x7f56801049b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5670000c00 tx=0x7f567002c640 comp rx=0 tx=0).stop 2024-04-02T13:53:32.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.987+0000 7f5686196640 1 -- 172.21.15.52:0/3545880993 shutdown_connections 2024-04-02T13:53:32.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.987+0000 7f5686196640 1 --2- 172.21.15.52:0/3545880993 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f56680a3db0 0x7f56680a6270 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.987+0000 7f5686196640 1 --2- 172.21.15.52:0/3545880993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f56801025d0 0x7f5680103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.987+0000 7f5686196640 1 --2- 172.21.15.52:0/3545880993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5680101c10 0x7f5680102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.987+0000 7f5686196640 1 --2- 172.21.15.52:0/3545880993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5680100a20 0x7f56801049b0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:32.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.987+0000 7f5686196640 1 -- 172.21.15.52:0/3545880993 >> 172.21.15.52:0/3545880993 conn(0x7f56800fbf90 msgr2=0x7f56800fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:32.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.987+0000 7f5686196640 1 -- 172.21.15.52:0/3545880993 shutdown_connections 2024-04-02T13:53:32.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:32.987+0000 7f5686196640 1 -- 172.21.15.52:0/3545880993 wait complete. 2024-04-02T13:53:33.000 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:32 smithi052 ceph-mon[32104]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:53:33.000 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:32 smithi052 ceph-mon[32104]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-02T13:53:33.001 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:32 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1172131276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:53:33.001 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:32 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:33.001 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:32 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:33.001 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:32 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:33.001 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:32 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:33.001 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:32 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:33.001 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:32 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:33.001 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:32 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3266463897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:53:33.001 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:32 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/908197823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:53:33.003 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:46:20.746053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:50:58.778690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:50:59.561527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:51:03.343487+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:53:33.003 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:53:33.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.107+0000 7f57be952640 1 -- 172.21.15.52:0/27324652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b8100a00 msgr2=0x7f57b8100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:33.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.107+0000 7f57be952640 1 --2- 172.21.15.52:0/27324652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b8100a00 0x7f57b8100e20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f57a0009900 tx=0x7f57a002bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:33.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.108+0000 7f57be952640 1 -- 172.21.15.52:0/27324652 shutdown_connections 2024-04-02T13:53:33.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.108+0000 7f57be952640 1 --2- 172.21.15.52:0/27324652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57b81025b0 0x7f57b8108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.108+0000 7f57be952640 1 --2- 172.21.15.52:0/27324652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57b8101bf0 0x7f57b8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.108+0000 7f57be952640 1 --2- 172.21.15.52:0/27324652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b8100a00 0x7f57b8100e20 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.108+0000 7f57be952640 1 -- 172.21.15.52:0/27324652 >> 172.21.15.52:0/27324652 conn(0x7f57b80fbf90 msgr2=0x7f57b80fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:33.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.108+0000 7f57be952640 1 -- 172.21.15.52:0/27324652 shutdown_connections 2024-04-02T13:53:33.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.108+0000 7f57be952640 1 -- 172.21.15.52:0/27324652 wait complete. 2024-04-02T13:53:33.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.108+0000 7f57be952640 1 Processor -- start 2024-04-02T13:53:33.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.108+0000 7f57be952640 1 -- start start 2024-04-02T13:53:33.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.108+0000 7f57be952640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57b8100a00 0x7f57b8104990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:33.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.108+0000 7f57be952640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57b8101bf0 0x7f57b8102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:33.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.108+0000 7f57be952640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b81025b0 0x7f57b8103510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:33.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.109+0000 7f57be952640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f57b810c670 con 0x7f57b81025b0 2024-04-02T13:53:33.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.109+0000 7f57be952640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f57b810c4f0 con 0x7f57b8101bf0 2024-04-02T13:53:33.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.109+0000 7f57be952640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f57b810c7f0 con 0x7f57b8100a00 2024-04-02T13:53:33.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.109+0000 7f57bceef640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b81025b0 0x7f57b8103510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:33.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.109+0000 7f57bceef640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b81025b0 0x7f57b8103510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58322/0 (socket says 172.21.15.52:58322) 2024-04-02T13:53:33.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.109+0000 7f57bceef640 1 -- 172.21.15.52:0/4224350558 learned_addr learned my addr 172.21.15.52:0/4224350558 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:33.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.109+0000 7f57b77fe640 1 --2- 172.21.15.52:0/4224350558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57b8101bf0 0x7f57b8102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:33.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.109+0000 7f57b7fff640 1 --2- 172.21.15.52:0/4224350558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57b8100a00 0x7f57b8104990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:33.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.109+0000 7f57b77fe640 1 -- 172.21.15.52:0/4224350558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57b8100a00 msgr2=0x7f57b8104990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:33.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.109+0000 7f57b77fe640 1 --2- 172.21.15.52:0/4224350558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57b8100a00 0x7f57b8104990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.109+0000 7f57b77fe640 1 -- 172.21.15.52:0/4224350558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b81025b0 msgr2=0x7f57b8103510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:33.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.109+0000 7f57b77fe640 1 --2- 172.21.15.52:0/4224350558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b81025b0 0x7f57b8103510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.109+0000 7f57b77fe640 1 -- 172.21.15.52:0/4224350558 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b8103da0 con 0x7f57b8101bf0 2024-04-02T13:53:33.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.109+0000 7f57b7fff640 1 --2- 172.21.15.52:0/4224350558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57b8100a00 0x7f57b8104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:33.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.109+0000 7f57bceef640 1 --2- 172.21.15.52:0/4224350558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b81025b0 0x7f57b8103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:33.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.109+0000 7f57b77fe640 1 --2- 172.21.15.52:0/4224350558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57b8101bf0 0x7f57b8102fd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f57a800e9f0 tx=0x7f57a800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:33.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.109+0000 7f57b57fa640 1 -- 172.21.15.52:0/4224350558 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a800ccc0 con 0x7f57b8101bf0 2024-04-02T13:53:33.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.109+0000 7f57b57fa640 1 -- 172.21.15.52:0/4224350558 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57a8005020 con 0x7f57b8101bf0 2024-04-02T13:53:33.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.109+0000 7f57be952640 1 -- 172.21.15.52:0/4224350558 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b81a1fb0 con 0x7f57b8101bf0 2024-04-02T13:53:33.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.109+0000 7f57b57fa640 1 -- 172.21.15.52:0/4224350558 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a80049e0 con 0x7f57b8101bf0 2024-04-02T13:53:33.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.109+0000 7f57be952640 1 -- 172.21.15.52:0/4224350558 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b81a23c0 con 0x7f57b8101bf0 2024-04-02T13:53:33.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.110+0000 7f57be952640 1 -- 172.21.15.52:0/4224350558 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57b81a26b0 con 0x7f57b8101bf0 2024-04-02T13:53:33.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.112+0000 7f57b57fa640 1 -- 172.21.15.52:0/4224350558 <== mon.2 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f57a8005560 con 0x7f57b8101bf0 2024-04-02T13:53:33.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.112+0000 7f57b57fa640 1 --2- 172.21.15.52:0/4224350558 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f57a40a3db0 0x7f57a40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:33.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.112+0000 7f57b7fff640 1 --2- 172.21.15.52:0/4224350558 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f57a40a3db0 0x7f57a40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:33.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.112+0000 7f57b57fa640 1 -- 172.21.15.52:0/4224350558 <== mon.2 v2:172.21.15.88:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f57a80d0c80 con 0x7f57b8101bf0 2024-04-02T13:53:33.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.113+0000 7f57b7fff640 1 --2- 172.21.15.52:0/4224350558 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f57a40a3db0 0x7f57a40a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f57a0000c00 tx=0x7f57a0005fd0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:33.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.116+0000 7f57b57fa640 1 -- 172.21.15.52:0/4224350558 <== mon.2 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f57a80051d0 con 0x7f57b8101bf0 2024-04-02T13:53:33.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:32 smithi088 ceph-mon[36528]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:53:33.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:32 smithi088 ceph-mon[36528]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-02T13:53:33.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:32 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1172131276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:53:33.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:32 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:33.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:32 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:33.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:32 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:33.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:32 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:33.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:32 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:33.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:32 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:33.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:32 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3266463897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:53:33.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:32 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/908197823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:53:33.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:32 smithi189 ceph-mon[36683]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:53:33.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:32 smithi189 ceph-mon[36683]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-02T13:53:33.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:32 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1172131276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:53:33.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:32 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:33.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:32 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:33.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:32 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:33.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:32 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:33.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:32 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:33.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:32 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:33.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:32 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3266463897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:53:33.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:32 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/908197823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:53:33.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.314+0000 7f57be952640 1 -- 172.21.15.52:0/4224350558 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f57b81a2880 con 0x7f57b8101bf0 2024-04-02T13:53:33.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.315+0000 7f57b57fa640 1 -- 172.21.15.52:0/4224350558 <== mon.2 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f57a809ce70 con 0x7f57b8101bf0 2024-04-02T13:53:33.319 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-04-02T13:53:33.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.317+0000 7f57be952640 1 -- 172.21.15.52:0/4224350558 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f57a40a3db0 msgr2=0x7f57a40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:33.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.317+0000 7f57be952640 1 --2- 172.21.15.52:0/4224350558 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f57a40a3db0 0x7f57a40a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f57a0000c00 tx=0x7f57a0005fd0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.317+0000 7f57be952640 1 -- 172.21.15.52:0/4224350558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57b8101bf0 msgr2=0x7f57b8102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:33.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.317+0000 7f57be952640 1 --2- 172.21.15.52:0/4224350558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57b8101bf0 0x7f57b8102fd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f57a800e9f0 tx=0x7f57a800eeb0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.317+0000 7f57be952640 1 -- 172.21.15.52:0/4224350558 shutdown_connections 2024-04-02T13:53:33.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.317+0000 7f57be952640 1 --2- 172.21.15.52:0/4224350558 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f57a40a3db0 0x7f57a40a6270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.317+0000 7f57be952640 1 --2- 172.21.15.52:0/4224350558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b81025b0 0x7f57b8103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.317+0000 7f57be952640 1 --2- 172.21.15.52:0/4224350558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57b8101bf0 0x7f57b8102fd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.317+0000 7f57be952640 1 --2- 172.21.15.52:0/4224350558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57b8100a00 0x7f57b8104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.317+0000 7f57be952640 1 -- 172.21.15.52:0/4224350558 >> 172.21.15.52:0/4224350558 conn(0x7f57b80fbf90 msgr2=0x7f57b80fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:33.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.317+0000 7f57be952640 1 -- 172.21.15.52:0/4224350558 shutdown_connections 2024-04-02T13:53:33.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.317+0000 7f57be952640 1 -- 172.21.15.52:0/4224350558 wait complete. 2024-04-02T13:53:33.334 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:53:33.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.439+0000 7f9207d74640 1 -- 172.21.15.52:0/698450422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92001025e0 msgr2=0x7f92001049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:33.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.439+0000 7f9207d74640 1 --2- 172.21.15.52:0/698450422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92001025e0 0x7f92001049d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f91f0009900 tx=0x7f91f002bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:33.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.439+0000 7f9207d74640 1 -- 172.21.15.52:0/698450422 shutdown_connections 2024-04-02T13:53:33.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.439+0000 7f9207d74640 1 --2- 172.21.15.52:0/698450422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9200107840 0x7f9200109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.439+0000 7f9207d74640 1 --2- 172.21.15.52:0/698450422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9200104f10 0x7f9200107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.439+0000 7f9207d74640 1 --2- 172.21.15.52:0/698450422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92001025e0 0x7f92001049d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.439+0000 7f9207d74640 1 -- 172.21.15.52:0/698450422 >> 172.21.15.52:0/698450422 conn(0x7f92000fbf90 msgr2=0x7f92000fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:33.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.439+0000 7f9207d74640 1 -- 172.21.15.52:0/698450422 shutdown_connections 2024-04-02T13:53:33.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.439+0000 7f9207d74640 1 -- 172.21.15.52:0/698450422 wait complete. 2024-04-02T13:53:33.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.440+0000 7f9207d74640 1 Processor -- start 2024-04-02T13:53:33.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.440+0000 7f9207d74640 1 -- start start 2024-04-02T13:53:33.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.440+0000 7f9207d74640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92001025e0 0x7f9200197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:33.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.440+0000 7f9207d74640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9200104f10 0x7f9200198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:33.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.440+0000 7f9207d74640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9200107840 0x7f920019f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:33.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.440+0000 7f9207d74640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f920010d7a0 con 0x7f9200104f10 2024-04-02T13:53:33.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.440+0000 7f9207d74640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f920010d620 con 0x7f9200107840 2024-04-02T13:53:33.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.440+0000 7f9207d74640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f920010d920 con 0x7f92001025e0 2024-04-02T13:53:33.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.441+0000 7f920530f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9200104f10 0x7f9200198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:33.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.441+0000 7f920530f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9200104f10 0x7f9200198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58338/0 (socket says 172.21.15.52:58338) 2024-04-02T13:53:33.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.441+0000 7f920530f640 1 -- 172.21.15.52:0/2999952206 learned_addr learned my addr 172.21.15.52:0/2999952206 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:33.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.441+0000 7f9205b10640 1 --2- 172.21.15.52:0/2999952206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92001025e0 0x7f9200197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:33.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.441+0000 7f9206311640 1 --2- 172.21.15.52:0/2999952206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9200107840 0x7f920019f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:33.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.441+0000 7f920530f640 1 -- 172.21.15.52:0/2999952206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92001025e0 msgr2=0x7f9200197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:33.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.441+0000 7f920530f640 1 --2- 172.21.15.52:0/2999952206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92001025e0 0x7f9200197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.441+0000 7f920530f640 1 -- 172.21.15.52:0/2999952206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9200107840 msgr2=0x7f920019f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:33.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.441+0000 7f920530f640 1 --2- 172.21.15.52:0/2999952206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9200107840 0x7f920019f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.441+0000 7f920530f640 1 -- 172.21.15.52:0/2999952206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920019fa40 con 0x7f9200104f10 2024-04-02T13:53:33.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.441+0000 7f920530f640 1 --2- 172.21.15.52:0/2999952206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9200104f10 0x7f9200198360 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00d740 tx=0x7f91fc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:33.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.441+0000 7f91f6ffd640 1 -- 172.21.15.52:0/2999952206 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc017070 con 0x7f9200104f10 2024-04-02T13:53:33.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.441+0000 7f91f6ffd640 1 -- 172.21.15.52:0/2999952206 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91fc00de50 con 0x7f9200104f10 2024-04-02T13:53:33.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.441+0000 7f9207d74640 1 -- 172.21.15.52:0/2999952206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920019fd30 con 0x7f9200104f10 2024-04-02T13:53:33.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.441+0000 7f91f6ffd640 1 -- 172.21.15.52:0/2999952206 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc004e00 con 0x7f9200104f10 2024-04-02T13:53:33.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.441+0000 7f9207d74640 1 -- 172.21.15.52:0/2999952206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92001a0220 con 0x7f9200104f10 2024-04-02T13:53:33.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.442+0000 7f9207d74640 1 -- 172.21.15.52:0/2999952206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92000fda80 con 0x7f9200104f10 2024-04-02T13:53:33.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.444+0000 7f91f6ffd640 1 -- 172.21.15.52:0/2999952206 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f91fc004fa0 con 0x7f9200104f10 2024-04-02T13:53:33.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.445+0000 7f91f6ffd640 1 --2- 172.21.15.52:0/2999952206 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f91ec0a3e30 0x7f91ec0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:33.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.445+0000 7f9205b10640 1 --2- 172.21.15.52:0/2999952206 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f91ec0a3e30 0x7f91ec0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:33.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.445+0000 7f91f6ffd640 1 -- 172.21.15.52:0/2999952206 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f91fc0d0560 con 0x7f9200104f10 2024-04-02T13:53:33.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.445+0000 7f9205b10640 1 --2- 172.21.15.52:0/2999952206 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f91ec0a3e30 0x7f91ec0a62f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f91f002c7b0 tx=0x7f91f0005fd0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:33.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.453+0000 7f91f6ffd640 1 -- 172.21.15.52:0/2999952206 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f91fc09c100 con 0x7f9200104f10 2024-04-02T13:53:33.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.645+0000 7f9207d74640 1 -- 172.21.15.52:0/2999952206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f920002d5c0 con 0x7f9200104f10 2024-04-02T13:53:33.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.646+0000 7f91f6ffd640 1 -- 172.21.15.52:0/2999952206 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f91fc0a0400 con 0x7f9200104f10 2024-04-02T13:53:33.649 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-04-02T13:53:33.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.648+0000 7f9207d74640 1 -- 172.21.15.52:0/2999952206 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f91ec0a3e30 msgr2=0x7f91ec0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:33.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.648+0000 7f9207d74640 1 --2- 172.21.15.52:0/2999952206 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f91ec0a3e30 0x7f91ec0a62f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f91f002c7b0 tx=0x7f91f0005fd0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.648+0000 7f9207d74640 1 -- 172.21.15.52:0/2999952206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9200104f10 msgr2=0x7f9200198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:33.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.648+0000 7f9207d74640 1 --2- 172.21.15.52:0/2999952206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9200104f10 0x7f9200198360 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00d740 tx=0x7f91fc00dc00 comp rx=0 tx=0).stop 2024-04-02T13:53:33.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.648+0000 7f9207d74640 1 -- 172.21.15.52:0/2999952206 shutdown_connections 2024-04-02T13:53:33.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.648+0000 7f9207d74640 1 --2- 172.21.15.52:0/2999952206 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f91ec0a3e30 0x7f91ec0a62f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.648+0000 7f9207d74640 1 --2- 172.21.15.52:0/2999952206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9200107840 0x7f920019f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.648+0000 7f9207d74640 1 --2- 172.21.15.52:0/2999952206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9200104f10 0x7f9200198360 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.648+0000 7f9207d74640 1 --2- 172.21.15.52:0/2999952206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92001025e0 0x7f9200197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.648+0000 7f9207d74640 1 -- 172.21.15.52:0/2999952206 >> 172.21.15.52:0/2999952206 conn(0x7f92000fbf90 msgr2=0x7f9200102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.648+0000 7f9207d74640 1 -- 172.21.15.52:0/2999952206 shutdown_connections 2024-04-02T13:53:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.648+0000 7f9207d74640 1 -- 172.21.15.52:0/2999952206 wait complete. 2024-04-02T13:53:33.663 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:53:33.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.768+0000 7fddf743a640 1 -- 172.21.15.52:0/2370051384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fddf0102600 msgr2=0x7fddf01049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:33.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.768+0000 7fddf743a640 1 --2- 172.21.15.52:0/2370051384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fddf0102600 0x7fddf01049f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fddec009900 tx=0x7fddec02bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:33.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.768+0000 7fddf743a640 1 -- 172.21.15.52:0/2370051384 shutdown_connections 2024-04-02T13:53:33.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.768+0000 7fddf743a640 1 --2- 172.21.15.52:0/2370051384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddf0107860 0x7fddf0109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.768+0000 7fddf743a640 1 --2- 172.21.15.52:0/2370051384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fddf0104f30 0x7fddf0107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.768+0000 7fddf743a640 1 --2- 172.21.15.52:0/2370051384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fddf0102600 0x7fddf01049f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.768+0000 7fddf743a640 1 -- 172.21.15.52:0/2370051384 >> 172.21.15.52:0/2370051384 conn(0x7fddf00fbf90 msgr2=0x7fddf00fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:33.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.768+0000 7fddf743a640 1 -- 172.21.15.52:0/2370051384 shutdown_connections 2024-04-02T13:53:33.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.768+0000 7fddf743a640 1 -- 172.21.15.52:0/2370051384 wait complete. 2024-04-02T13:53:33.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.769+0000 7fddf743a640 1 Processor -- start 2024-04-02T13:53:33.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.769+0000 7fddf743a640 1 -- start start 2024-04-02T13:53:33.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.769+0000 7fddf743a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fddf0102600 0x7fddf0197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:33.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.769+0000 7fddf743a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fddf0104f30 0x7fddf01983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:33.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.769+0000 7fddf743a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddf0107860 0x7fddf019f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:33.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.769+0000 7fddf743a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fddf010d770 con 0x7fddf0102600 2024-04-02T13:53:33.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.769+0000 7fddf743a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fddf010d5f0 con 0x7fddf0107860 2024-04-02T13:53:33.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.769+0000 7fddf743a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fddf010d8f0 con 0x7fddf0104f30 2024-04-02T13:53:33.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.769+0000 7fddf51d6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fddf0102600 0x7fddf0197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:33.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.769+0000 7fddf51d6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fddf0102600 0x7fddf0197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58344/0 (socket says 172.21.15.52:58344) 2024-04-02T13:53:33.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.769+0000 7fddf59d7640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddf0107860 0x7fddf019f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:33.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.769+0000 7fddf51d6640 1 -- 172.21.15.52:0/198583162 learned_addr learned my addr 172.21.15.52:0/198583162 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:33.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.769+0000 7fddf49d5640 1 --2- 172.21.15.52:0/198583162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fddf0104f30 0x7fddf01983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:33.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.769+0000 7fddf59d7640 1 -- 172.21.15.52:0/198583162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fddf0104f30 msgr2=0x7fddf01983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:33.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.769+0000 7fddf59d7640 1 --2- 172.21.15.52:0/198583162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fddf0104f30 0x7fddf01983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.769+0000 7fddf59d7640 1 -- 172.21.15.52:0/198583162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fddf0102600 msgr2=0x7fddf0197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:33.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.769+0000 7fddf59d7640 1 --2- 172.21.15.52:0/198583162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fddf0102600 0x7fddf0197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:33.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.769+0000 7fddf59d7640 1 -- 172.21.15.52:0/198583162 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf019fa90 con 0x7fddf0107860 2024-04-02T13:53:33.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.770+0000 7fddf59d7640 1 --2- 172.21.15.52:0/198583162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddf0107860 0x7fddf019f420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdde8009fd0 tx=0x7fdde800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:33.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.770+0000 7fdde67fc640 1 -- 172.21.15.52:0/198583162 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde800ce20 con 0x7fddf0107860 2024-04-02T13:53:33.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.770+0000 7fdde67fc640 1 -- 172.21.15.52:0/198583162 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdde8005020 con 0x7fddf0107860 2024-04-02T13:53:33.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.770+0000 7fddf743a640 1 -- 172.21.15.52:0/198583162 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf019fd80 con 0x7fddf0107860 2024-04-02T13:53:33.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.770+0000 7fdde67fc640 1 -- 172.21.15.52:0/198583162 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde8010720 con 0x7fddf0107860 2024-04-02T13:53:33.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.770+0000 7fddf743a640 1 -- 172.21.15.52:0/198583162 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf01a0220 con 0x7fddf0107860 2024-04-02T13:53:33.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.771+0000 7fddf743a640 1 -- 172.21.15.52:0/198583162 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddf00fda80 con 0x7fddf0107860 2024-04-02T13:53:33.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.772+0000 7fdde67fc640 1 -- 172.21.15.52:0/198583162 <== mon.2 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fdde8005540 con 0x7fddf0107860 2024-04-02T13:53:33.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.773+0000 7fdde67fc640 1 --2- 172.21.15.52:0/198583162 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdddc0a3db0 0x7fdddc0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:33.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.773+0000 7fddf51d6640 1 --2- 172.21.15.52:0/198583162 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdddc0a3db0 0x7fdddc0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:33.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.773+0000 7fdde67fc640 1 -- 172.21.15.52:0/198583162 <== mon.2 v2:172.21.15.88:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fdde80d19a0 con 0x7fddf0107860 2024-04-02T13:53:33.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.773+0000 7fddf51d6640 1 --2- 172.21.15.52:0/198583162 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdddc0a3db0 0x7fdddc0a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fddec000c00 tx=0x7fddec02c880 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:33.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.777+0000 7fdde67fc640 1 -- 172.21.15.52:0/198583162 <== mon.2 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fdde809db30 con 0x7fddf0107860 2024-04-02T13:53:33.913 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3545880993' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:53:33.913 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:33.913 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:33.913 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:33.913 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:33.913 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:33.913 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:33.913 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:33.914 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:33.914 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:33.914 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:33.914 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: Adjusting osd_memory_target on smithi088 to 174.1M 2024-04-02T13:53:33.914 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:33.914 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: Unable to set osd_memory_target on smithi088 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-04-02T13:53:33.914 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:33.914 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: Adjusting osd_memory_target on smithi189 to 174.1M 2024-04-02T13:53:33.914 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: Unable to set osd_memory_target on smithi189 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-04-02T13:53:33.914 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:33.914 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:33.914 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:33.914 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:33.914 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:33.915 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:33.915 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-04-02T13:53:33.915 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: Unable to set osd_memory_target on smithi052 to 185898496: error parsing value: Value '185898496' is below minimum 939524096 2024-04-02T13:53:33.915 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:53:33.915 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:53:33.915 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: Updating smithi052:/etc/ceph/ceph.conf 2024-04-02T13:53:33.915 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: Updating smithi088:/etc/ceph/ceph.conf 2024-04-02T13:53:33.915 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: Updating smithi189:/etc/ceph/ceph.conf 2024-04-02T13:53:33.915 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/4224350558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:53:33.915 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: Updating smithi052:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:53:33.915 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: Updating smithi088:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:53:33.915 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: Updating smithi189:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:53:33.915 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:33 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2999952206' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:53:33.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.995+0000 7fddf743a640 1 -- 172.21.15.52:0/198583162 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fddf00627a0 con 0x7fddf0107860 2024-04-02T13:53:33.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.996+0000 7fdde67fc640 1 -- 172.21.15.52:0/198583162 <== mon.2 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fdde809d430 con 0x7fddf0107860 2024-04-02T13:53:33.999 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-04-02T13:53:34.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.998+0000 7fddf743a640 1 -- 172.21.15.52:0/198583162 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdddc0a3db0 msgr2=0x7fdddc0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:34.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.998+0000 7fddf743a640 1 --2- 172.21.15.52:0/198583162 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdddc0a3db0 0x7fdddc0a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fddec000c00 tx=0x7fddec02c880 comp rx=0 tx=0).stop 2024-04-02T13:53:34.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.998+0000 7fddf743a640 1 -- 172.21.15.52:0/198583162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddf0107860 msgr2=0x7fddf019f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:34.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.998+0000 7fddf743a640 1 --2- 172.21.15.52:0/198583162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddf0107860 0x7fddf019f420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdde8009fd0 tx=0x7fdde800ef10 comp rx=0 tx=0).stop 2024-04-02T13:53:34.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.998+0000 7fddf743a640 1 -- 172.21.15.52:0/198583162 shutdown_connections 2024-04-02T13:53:34.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.998+0000 7fddf743a640 1 --2- 172.21.15.52:0/198583162 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdddc0a3db0 0x7fdddc0a6270 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.998+0000 7fddf743a640 1 --2- 172.21.15.52:0/198583162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddf0107860 0x7fddf019f420 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.998+0000 7fddf743a640 1 --2- 172.21.15.52:0/198583162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fddf0104f30 0x7fddf01983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.998+0000 7fddf743a640 1 --2- 172.21.15.52:0/198583162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fddf0102600 0x7fddf0197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.998+0000 7fddf743a640 1 -- 172.21.15.52:0/198583162 >> 172.21.15.52:0/198583162 conn(0x7fddf00fbf90 msgr2=0x7fddf01030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:34.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.998+0000 7fddf743a640 1 -- 172.21.15.52:0/198583162 shutdown_connections 2024-04-02T13:53:34.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:33.998+0000 7fddf743a640 1 -- 172.21.15.52:0/198583162 wait complete. 2024-04-02T13:53:34.013 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:53:34.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.118+0000 7f4e5eb9d640 1 -- 172.21.15.52:0/3152596165 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e581069a0 msgr2=0x7f4e58106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:34.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.118+0000 7f4e5eb9d640 1 --2- 172.21.15.52:0/3152596165 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e581069a0 0x7f4e58106d80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009930 tx=0x7f4e4c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:34.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.118+0000 7f4e5eb9d640 1 -- 172.21.15.52:0/3152596165 shutdown_connections 2024-04-02T13:53:34.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.118+0000 7f4e5eb9d640 1 --2- 172.21.15.52:0/3152596165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e58101350 0x7f4e5810d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.118+0000 7f4e5eb9d640 1 --2- 172.21.15.52:0/3152596165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e58100990 0x7f4e58100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.118+0000 7f4e5eb9d640 1 --2- 172.21.15.52:0/3152596165 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e581069a0 0x7f4e58106d80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.118+0000 7f4e5eb9d640 1 -- 172.21.15.52:0/3152596165 >> 172.21.15.52:0/3152596165 conn(0x7f4e580777b0 msgr2=0x7f4e580feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:34.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.119+0000 7f4e5eb9d640 1 -- 172.21.15.52:0/3152596165 shutdown_connections 2024-04-02T13:53:34.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.119+0000 7f4e5eb9d640 1 -- 172.21.15.52:0/3152596165 wait complete. 2024-04-02T13:53:34.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.119+0000 7f4e5eb9d640 1 Processor -- start 2024-04-02T13:53:34.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.119+0000 7f4e5eb9d640 1 -- start start 2024-04-02T13:53:34.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.119+0000 7f4e5eb9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e58100990 0x7f4e580fce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:34.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.119+0000 7f4e5eb9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e58101350 0x7f4e580fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:34.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.119+0000 7f4e5eb9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e581069a0 0x7f4e580fd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:34.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.119+0000 7f4e5eb9d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4e58110db0 con 0x7f4e58100990 2024-04-02T13:53:34.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.119+0000 7f4e5eb9d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4e58110c30 con 0x7f4e58101350 2024-04-02T13:53:34.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.119+0000 7f4e5eb9d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4e58110f30 con 0x7f4e581069a0 2024-04-02T13:53:34.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.119+0000 7f4e5c939640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e58100990 0x7f4e580fce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:34.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.119+0000 7f4e5c939640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e58100990 0x7f4e580fce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58368/0 (socket says 172.21.15.52:58368) 2024-04-02T13:53:34.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.120+0000 7f4e5c939640 1 -- 172.21.15.52:0/3209286520 learned_addr learned my addr 172.21.15.52:0/3209286520 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:34.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.120+0000 7f4e5d13a640 1 --2- 172.21.15.52:0/3209286520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e581069a0 0x7f4e580fd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:34.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.120+0000 7f4e57fff640 1 --2- 172.21.15.52:0/3209286520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e58101350 0x7f4e580fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:34.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.120+0000 7f4e5c939640 1 -- 172.21.15.52:0/3209286520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e581069a0 msgr2=0x7f4e580fd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:34.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.120+0000 7f4e5c939640 1 --2- 172.21.15.52:0/3209286520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e581069a0 0x7f4e580fd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.120+0000 7f4e5c939640 1 -- 172.21.15.52:0/3209286520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e58101350 msgr2=0x7f4e580fd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:34.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.120+0000 7f4e5c939640 1 --2- 172.21.15.52:0/3209286520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e58101350 0x7f4e580fd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.120+0000 7f4e5c939640 1 -- 172.21.15.52:0/3209286520 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e581a68e0 con 0x7f4e58100990 2024-04-02T13:53:34.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.120+0000 7f4e5c939640 1 --2- 172.21.15.52:0/3209286520 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e58100990 0x7f4e580fce60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009230 tx=0x7f4e4c02cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:34.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.120+0000 7f4e55ffb640 1 -- 172.21.15.52:0/3209286520 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c041070 con 0x7f4e58100990 2024-04-02T13:53:34.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.120+0000 7f4e55ffb640 1 -- 172.21.15.52:0/3209286520 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e4c004450 con 0x7f4e58100990 2024-04-02T13:53:34.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.120+0000 7f4e5eb9d640 1 -- 172.21.15.52:0/3209286520 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e581a6b70 con 0x7f4e58100990 2024-04-02T13:53:34.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.120+0000 7f4e55ffb640 1 -- 172.21.15.52:0/3209286520 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c034440 con 0x7f4e58100990 2024-04-02T13:53:34.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.120+0000 7f4e5eb9d640 1 -- 172.21.15.52:0/3209286520 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e581a7060 con 0x7f4e58100990 2024-04-02T13:53:34.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.122+0000 7f4e5eb9d640 1 -- 172.21.15.52:0/3209286520 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e58068060 con 0x7f4e58100990 2024-04-02T13:53:34.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.123+0000 7f4e55ffb640 1 -- 172.21.15.52:0/3209286520 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f4e4c044070 con 0x7f4e58100990 2024-04-02T13:53:34.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.123+0000 7f4e55ffb640 1 --2- 172.21.15.52:0/3209286520 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4e480a3db0 0x7f4e480a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:34.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.123+0000 7f4e57fff640 1 --2- 172.21.15.52:0/3209286520 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4e480a3db0 0x7f4e480a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:34.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.123+0000 7f4e55ffb640 1 -- 172.21.15.52:0/3209286520 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f4e4c0f2570 con 0x7f4e58100990 2024-04-02T13:53:34.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.124+0000 7f4e57fff640 1 --2- 172.21.15.52:0/3209286520 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4e480a3db0 0x7f4e480a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4e580fe8f0 tx=0x7f4e44005f90 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:34.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.128+0000 7f4e55ffb640 1 -- 172.21.15.52:0/3209286520 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4e4c0be220 con 0x7f4e58100990 2024-04-02T13:53:34.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3545880993' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:53:34.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:34.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:34.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:34.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:34.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:34.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:34.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:34.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:34.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:34.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:34.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: Adjusting osd_memory_target on smithi088 to 174.1M 2024-04-02T13:53:34.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:34.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: Unable to set osd_memory_target on smithi088 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-04-02T13:53:34.172 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:34.172 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: Adjusting osd_memory_target on smithi189 to 174.1M 2024-04-02T13:53:34.172 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: Unable to set osd_memory_target on smithi189 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-04-02T13:53:34.172 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:34.172 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:34.172 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:34.172 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:34.172 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:34.172 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:34.172 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-04-02T13:53:34.172 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: Unable to set osd_memory_target on smithi052 to 185898496: error parsing value: Value '185898496' is below minimum 939524096 2024-04-02T13:53:34.172 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:53:34.173 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:53:34.173 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: Updating smithi052:/etc/ceph/ceph.conf 2024-04-02T13:53:34.173 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: Updating smithi088:/etc/ceph/ceph.conf 2024-04-02T13:53:34.173 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: Updating smithi189:/etc/ceph/ceph.conf 2024-04-02T13:53:34.173 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/4224350558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:53:34.173 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: Updating smithi052:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:53:34.173 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: Updating smithi088:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:53:34.173 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: Updating smithi189:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:53:34.173 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:33 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2999952206' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:53:34.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3545880993' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:53:34.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:34.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:34.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:34.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:34.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:34.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:34.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:34.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:34.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:34.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:34.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: Adjusting osd_memory_target on smithi088 to 174.1M 2024-04-02T13:53:34.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:34.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: Unable to set osd_memory_target on smithi088 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-04-02T13:53:34.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:34.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: Adjusting osd_memory_target on smithi189 to 174.1M 2024-04-02T13:53:34.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: Unable to set osd_memory_target on smithi189 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-04-02T13:53:34.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:34.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:34.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:34.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:34.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:34.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:53:34.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-04-02T13:53:34.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: Unable to set osd_memory_target on smithi052 to 185898496: error parsing value: Value '185898496' is below minimum 939524096 2024-04-02T13:53:34.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:53:34.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:53:34.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: Updating smithi052:/etc/ceph/ceph.conf 2024-04-02T13:53:34.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: Updating smithi088:/etc/ceph/ceph.conf 2024-04-02T13:53:34.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: Updating smithi189:/etc/ceph/ceph.conf 2024-04-02T13:53:34.291 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/4224350558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:53:34.291 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: Updating smithi052:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:53:34.291 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: Updating smithi088:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:53:34.291 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: Updating smithi189:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:53:34.291 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:33 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2999952206' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:53:34.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.294+0000 7f4e55ffb640 1 -- 172.21.15.52:0/3209286520 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f4e4c0ba890 con 0x7f4e58100990 2024-04-02T13:53:34.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.327+0000 7f4e5eb9d640 1 -- 172.21.15.52:0/3209286520 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4e581a72d0 con 0x7f4e58100990 2024-04-02T13:53:34.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.327+0000 7f4e55ffb640 1 -- 172.21.15.52:0/3209286520 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f4e4c0ae570 con 0x7f4e58100990 2024-04-02T13:53:34.331 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-04-02T13:53:34.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.329+0000 7f4e5eb9d640 1 -- 172.21.15.52:0/3209286520 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4e480a3db0 msgr2=0x7f4e480a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:34.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.329+0000 7f4e5eb9d640 1 --2- 172.21.15.52:0/3209286520 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4e480a3db0 0x7f4e480a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4e580fe8f0 tx=0x7f4e44005f90 comp rx=0 tx=0).stop 2024-04-02T13:53:34.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.329+0000 7f4e5eb9d640 1 -- 172.21.15.52:0/3209286520 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e58100990 msgr2=0x7f4e580fce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:34.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.329+0000 7f4e5eb9d640 1 --2- 172.21.15.52:0/3209286520 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e58100990 0x7f4e580fce60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009230 tx=0x7f4e4c02cba0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.329+0000 7f4e5eb9d640 1 -- 172.21.15.52:0/3209286520 shutdown_connections 2024-04-02T13:53:34.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.329+0000 7f4e5eb9d640 1 --2- 172.21.15.52:0/3209286520 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4e480a3db0 0x7f4e480a6270 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.329+0000 7f4e5eb9d640 1 --2- 172.21.15.52:0/3209286520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e581069a0 0x7f4e580fd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.329+0000 7f4e5eb9d640 1 --2- 172.21.15.52:0/3209286520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e58101350 0x7f4e580fd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.329+0000 7f4e5eb9d640 1 --2- 172.21.15.52:0/3209286520 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e58100990 0x7f4e580fce60 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.329+0000 7f4e5eb9d640 1 -- 172.21.15.52:0/3209286520 >> 172.21.15.52:0/3209286520 conn(0x7f4e580777b0 msgr2=0x7f4e5806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:34.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.329+0000 7f4e5eb9d640 1 -- 172.21.15.52:0/3209286520 shutdown_connections 2024-04-02T13:53:34.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.329+0000 7f4e5eb9d640 1 -- 172.21.15.52:0/3209286520 wait complete. 2024-04-02T13:53:34.344 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-02T13:53:34.345 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:53:34.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.450+0000 7fca9d3ba640 1 -- 172.21.15.52:0/2412411190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca981013c0 msgr2=0x7fca9810d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:34.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.450+0000 7fca9d3ba640 1 --2- 172.21.15.52:0/2412411190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca981013c0 0x7fca9810d8d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fca90009cd0 tx=0x7fca9002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.450+0000 7fca9d3ba640 1 -- 172.21.15.52:0/2412411190 shutdown_connections 2024-04-02T13:53:34.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.450+0000 7fca9d3ba640 1 --2- 172.21.15.52:0/2412411190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca981013c0 0x7fca9810d8d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.450+0000 7fca9d3ba640 1 --2- 172.21.15.52:0/2412411190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca98100a00 0x7fca98100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.450+0000 7fca9d3ba640 1 --2- 172.21.15.52:0/2412411190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca98106a10 0x7fca98106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.450+0000 7fca9d3ba640 1 -- 172.21.15.52:0/2412411190 >> 172.21.15.52:0/2412411190 conn(0x7fca98077b00 msgr2=0x7fca98077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:34.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.450+0000 7fca9d3ba640 1 -- 172.21.15.52:0/2412411190 shutdown_connections 2024-04-02T13:53:34.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.450+0000 7fca9d3ba640 1 -- 172.21.15.52:0/2412411190 wait complete. 2024-04-02T13:53:34.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.451+0000 7fca9d3ba640 1 Processor -- start 2024-04-02T13:53:34.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.451+0000 7fca9d3ba640 1 -- start start 2024-04-02T13:53:34.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.451+0000 7fca9d3ba640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca98100a00 0x7fca9819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:34.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.451+0000 7fca9d3ba640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca981013c0 0x7fca9819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:34.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.451+0000 7fca9d3ba640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca98106a10 0x7fca9819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:34.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.451+0000 7fca9d3ba640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fca98110e20 con 0x7fca981013c0 2024-04-02T13:53:34.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.451+0000 7fca9d3ba640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fca98110ca0 con 0x7fca98106a10 2024-04-02T13:53:34.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.451+0000 7fca9d3ba640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fca98110fa0 con 0x7fca98100a00 2024-04-02T13:53:34.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.451+0000 7fca967fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca981013c0 0x7fca9819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:34.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.451+0000 7fca967fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca981013c0 0x7fca9819ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58386/0 (socket says 172.21.15.52:58386) 2024-04-02T13:53:34.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.451+0000 7fca967fc640 1 -- 172.21.15.52:0/2393661751 learned_addr learned my addr 172.21.15.52:0/2393661751 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:34.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.451+0000 7fca977fe640 1 --2- 172.21.15.52:0/2393661751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca98106a10 0x7fca9819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:34.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.451+0000 7fca96ffd640 1 --2- 172.21.15.52:0/2393661751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca98100a00 0x7fca9819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:34.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.451+0000 7fca967fc640 1 -- 172.21.15.52:0/2393661751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca98100a00 msgr2=0x7fca9819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:34.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.452+0000 7fca967fc640 1 --2- 172.21.15.52:0/2393661751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca98100a00 0x7fca9819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.452+0000 7fca967fc640 1 -- 172.21.15.52:0/2393661751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca98106a10 msgr2=0x7fca9819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:34.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.452+0000 7fca967fc640 1 --2- 172.21.15.52:0/2393661751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca98106a10 0x7fca9819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.452+0000 7fca967fc640 1 -- 172.21.15.52:0/2393661751 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9819f5e0 con 0x7fca981013c0 2024-04-02T13:53:34.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.452+0000 7fca967fc640 1 --2- 172.21.15.52:0/2393661751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca981013c0 0x7fca9819ab30 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00d770 tx=0x7fca8c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:34.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.452+0000 7fca7ffff640 1 -- 172.21.15.52:0/2393661751 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8c017070 con 0x7fca981013c0 2024-04-02T13:53:34.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.452+0000 7fca9d3ba640 1 -- 172.21.15.52:0/2393661751 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca9819f8d0 con 0x7fca981013c0 2024-04-02T13:53:34.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.452+0000 7fca7ffff640 1 -- 172.21.15.52:0/2393661751 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca8c005500 con 0x7fca981013c0 2024-04-02T13:53:34.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.452+0000 7fca7ffff640 1 -- 172.21.15.52:0/2393661751 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8c004e00 con 0x7fca981013c0 2024-04-02T13:53:34.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.452+0000 7fca9d3ba640 1 -- 172.21.15.52:0/2393661751 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca981a7160 con 0x7fca981013c0 2024-04-02T13:53:34.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.453+0000 7fca9d3ba640 1 -- 172.21.15.52:0/2393661751 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca980688f0 con 0x7fca981013c0 2024-04-02T13:53:34.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.454+0000 7fca7ffff640 1 -- 172.21.15.52:0/2393661751 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fca8c004fa0 con 0x7fca981013c0 2024-04-02T13:53:34.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.454+0000 7fca7ffff640 1 --2- 172.21.15.52:0/2393661751 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fca800a3e40 0x7fca800a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:34.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.455+0000 7fca96ffd640 1 --2- 172.21.15.52:0/2393661751 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fca800a3e40 0x7fca800a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:34.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.455+0000 7fca7ffff640 1 -- 172.21.15.52:0/2393661751 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fca8c0d07c0 con 0x7fca981013c0 2024-04-02T13:53:34.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.455+0000 7fca96ffd640 1 --2- 172.21.15.52:0/2393661751 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fca800a3e40 0x7fca800a6300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fca88009b50 tx=0x7fca88005f90 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:34.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.459+0000 7fca7ffff640 1 -- 172.21.15.52:0/2393661751 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fca8c09c280 con 0x7fca981013c0 2024-04-02T13:53:34.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.695+0000 7fca9d3ba640 1 -- 172.21.15.52:0/2393661751 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca981a7330 con 0x7fca981013c0 2024-04-02T13:53:34.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.696+0000 7fca7ffff640 1 -- 172.21.15.52:0/2393661751 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11923 (secure 0 0 0) 0x7fca8c0a0580 con 0x7fca981013c0 2024-04-02T13:53:34.699 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:53:34.699 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:53:28.475061+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24457,24467,24490,24503,24475]},"id":1}]} 2024-04-02T13:53:34.699 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 28 2024-04-02T13:53:34.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.698+0000 7fca9d3ba640 1 -- 172.21.15.52:0/2393661751 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fca800a3e40 msgr2=0x7fca800a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:34.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.698+0000 7fca9d3ba640 1 --2- 172.21.15.52:0/2393661751 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fca800a3e40 0x7fca800a6300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fca88009b50 tx=0x7fca88005f90 comp rx=0 tx=0).stop 2024-04-02T13:53:34.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.698+0000 7fca9d3ba640 1 -- 172.21.15.52:0/2393661751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca981013c0 msgr2=0x7fca9819ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:34.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.698+0000 7fca9d3ba640 1 --2- 172.21.15.52:0/2393661751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca981013c0 0x7fca9819ab30 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00d770 tx=0x7fca8c00dc30 comp rx=0 tx=0).stop 2024-04-02T13:53:34.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.698+0000 7fca9d3ba640 1 -- 172.21.15.52:0/2393661751 shutdown_connections 2024-04-02T13:53:34.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.698+0000 7fca9d3ba640 1 --2- 172.21.15.52:0/2393661751 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fca800a3e40 0x7fca800a6300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.698+0000 7fca9d3ba640 1 --2- 172.21.15.52:0/2393661751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca98106a10 0x7fca9819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.698+0000 7fca9d3ba640 1 --2- 172.21.15.52:0/2393661751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca981013c0 0x7fca9819ab30 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.698+0000 7fca9d3ba640 1 --2- 172.21.15.52:0/2393661751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca98100a00 0x7fca9819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.698+0000 7fca9d3ba640 1 -- 172.21.15.52:0/2393661751 >> 172.21.15.52:0/2393661751 conn(0x7fca98077b00 msgr2=0x7fca9810c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:34.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.698+0000 7fca9d3ba640 1 -- 172.21.15.52:0/2393661751 shutdown_connections 2024-04-02T13:53:34.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.698+0000 7fca9d3ba640 1 -- 172.21.15.52:0/2393661751 wait complete. 2024-04-02T13:53:34.712 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-02T13:53:34.712 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-02T13:53:34.712 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T13:53:34.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.812+0000 7faa86af2640 1 -- 172.21.15.52:0/3053814428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa80106a10 msgr2=0x7faa80106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:34.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.812+0000 7faa86af2640 1 --2- 172.21.15.52:0/3053814428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa80106a10 0x7faa80106df0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7faa78009930 tx=0x7faa7802bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:34.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.812+0000 7faa86af2640 1 -- 172.21.15.52:0/3053814428 shutdown_connections 2024-04-02T13:53:34.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.812+0000 7faa86af2640 1 --2- 172.21.15.52:0/3053814428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa801013c0 0x7faa8010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.812+0000 7faa86af2640 1 --2- 172.21.15.52:0/3053814428 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa80100a00 0x7faa80100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.812+0000 7faa86af2640 1 --2- 172.21.15.52:0/3053814428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa80106a10 0x7faa80106df0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.812+0000 7faa86af2640 1 -- 172.21.15.52:0/3053814428 >> 172.21.15.52:0/3053814428 conn(0x7faa80077b00 msgr2=0x7faa80077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:34.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.812+0000 7faa86af2640 1 -- 172.21.15.52:0/3053814428 shutdown_connections 2024-04-02T13:53:34.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.813+0000 7faa86af2640 1 -- 172.21.15.52:0/3053814428 wait complete. 2024-04-02T13:53:34.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.813+0000 7faa86af2640 1 Processor -- start 2024-04-02T13:53:34.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.813+0000 7faa86af2640 1 -- start start 2024-04-02T13:53:34.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.813+0000 7faa86af2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa80100a00 0x7faa80199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:34.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.813+0000 7faa86af2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa801013c0 0x7faa801997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:34.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.813+0000 7faa86af2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa80106a10 0x7faa801a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:34.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.813+0000 7faa86af2640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faa80110df0 con 0x7faa80100a00 2024-04-02T13:53:34.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.813+0000 7faa86af2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faa80110c70 con 0x7faa801013c0 2024-04-02T13:53:34.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.813+0000 7faa86af2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faa80110f70 con 0x7faa80106a10 2024-04-02T13:53:34.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.813+0000 7faa8488e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa80100a00 0x7faa80199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:34.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.813+0000 7faa8488e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa80100a00 0x7faa80199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58402/0 (socket says 172.21.15.52:58402) 2024-04-02T13:53:34.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.813+0000 7faa8488e640 1 -- 172.21.15.52:0/1129501524 learned_addr learned my addr 172.21.15.52:0/1129501524 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:34.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.813+0000 7faa8508f640 1 --2- 172.21.15.52:0/1129501524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa80106a10 0x7faa801a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:34.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.814+0000 7faa7ffff640 1 --2- 172.21.15.52:0/1129501524 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa801013c0 0x7faa801997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:34.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.814+0000 7faa8488e640 1 -- 172.21.15.52:0/1129501524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa80106a10 msgr2=0x7faa801a42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:34.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.814+0000 7faa8488e640 1 --2- 172.21.15.52:0/1129501524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa80106a10 0x7faa801a42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.814+0000 7faa8488e640 1 -- 172.21.15.52:0/1129501524 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa801013c0 msgr2=0x7faa801997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:34.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.814+0000 7faa8488e640 1 --2- 172.21.15.52:0/1129501524 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa801013c0 0x7faa801997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:34.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.814+0000 7faa8488e640 1 -- 172.21.15.52:0/1129501524 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa801a4830 con 0x7faa80100a00 2024-04-02T13:53:34.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.814+0000 7faa8488e640 1 --2- 172.21.15.52:0/1129501524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa80100a00 0x7faa80199270 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7faa78009230 tx=0x7faa7802cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:34.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.814+0000 7faa7dffb640 1 -- 172.21.15.52:0/1129501524 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78041070 con 0x7faa80100a00 2024-04-02T13:53:34.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.814+0000 7faa7dffb640 1 -- 172.21.15.52:0/1129501524 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa78004450 con 0x7faa80100a00 2024-04-02T13:53:34.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.814+0000 7faa86af2640 1 -- 172.21.15.52:0/1129501524 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa801a4a00 con 0x7faa80100a00 2024-04-02T13:53:34.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.814+0000 7faa7dffb640 1 -- 172.21.15.52:0/1129501524 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78034440 con 0x7faa80100a00 2024-04-02T13:53:34.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.814+0000 7faa86af2640 1 -- 172.21.15.52:0/1129501524 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa801a4e60 con 0x7faa80100a00 2024-04-02T13:53:34.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.815+0000 7faa86af2640 1 -- 172.21.15.52:0/1129501524 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa800688f0 con 0x7faa80100a00 2024-04-02T13:53:34.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.816+0000 7faa7dffb640 1 -- 172.21.15.52:0/1129501524 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7faa78044070 con 0x7faa80100a00 2024-04-02T13:53:34.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.817+0000 7faa7dffb640 1 --2- 172.21.15.52:0/1129501524 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7faa6c0a3ec0 0x7faa6c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:34.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.817+0000 7faa7ffff640 1 --2- 172.21.15.52:0/1129501524 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7faa6c0a3ec0 0x7faa6c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:34.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.817+0000 7faa7dffb640 1 -- 172.21.15.52:0/1129501524 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7faa780f2770 con 0x7faa80100a00 2024-04-02T13:53:34.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.817+0000 7faa7ffff640 1 --2- 172.21.15.52:0/1129501524 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7faa6c0a3ec0 0x7faa6c0a6380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faa80069760 tx=0x7faa70009500 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:34.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:34.821+0000 7faa7dffb640 1 -- 172.21.15.52:0/1129501524 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7faa780be930 con 0x7faa80100a00 2024-04-02T13:53:35.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.006+0000 7faa86af2640 1 -- 172.21.15.52:0/1129501524 --> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faa80061f40 con 0x7faa6c0a3ec0 2024-04-02T13:53:35.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.016+0000 7faa7dffb640 1 -- 172.21.15.52:0/1129501524 <== mgr.25160 v2:172.21.15.52:6800/1019901931 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339431 (secure 0 0 0) 0x7faa80061f40 con 0x7faa6c0a3ec0 2024-04-02T13:53:35.024 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:53:35.026 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-04-02T13:53:35.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.019+0000 7faa86af2640 1 -- 172.21.15.52:0/1129501524 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7faa6c0a3ec0 msgr2=0x7faa6c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:35.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.019+0000 7faa86af2640 1 --2- 172.21.15.52:0/1129501524 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7faa6c0a3ec0 0x7faa6c0a6380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faa80069760 tx=0x7faa70009500 comp rx=0 tx=0).stop 2024-04-02T13:53:35.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.019+0000 7faa86af2640 1 -- 172.21.15.52:0/1129501524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa80100a00 msgr2=0x7faa80199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:35.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.019+0000 7faa86af2640 1 --2- 172.21.15.52:0/1129501524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa80100a00 0x7faa80199270 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7faa78009230 tx=0x7faa7802cba0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.019+0000 7faa86af2640 1 -- 172.21.15.52:0/1129501524 shutdown_connections 2024-04-02T13:53:35.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.019+0000 7faa86af2640 1 --2- 172.21.15.52:0/1129501524 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7faa6c0a3ec0 0x7faa6c0a6380 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.019+0000 7faa86af2640 1 --2- 172.21.15.52:0/1129501524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa80106a10 0x7faa801a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.019+0000 7faa86af2640 1 --2- 172.21.15.52:0/1129501524 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa801013c0 0x7faa801997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.019+0000 7faa86af2640 1 --2- 172.21.15.52:0/1129501524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa80100a00 0x7faa80199270 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.019+0000 7faa86af2640 1 -- 172.21.15.52:0/1129501524 >> 172.21.15.52:0/1129501524 conn(0x7faa80077b00 msgr2=0x7faa801032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:35.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.020+0000 7faa86af2640 1 -- 172.21.15.52:0/1129501524 shutdown_connections 2024-04-02T13:53:35.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.020+0000 7faa86af2640 1 -- 172.21.15.52:0/1129501524 wait complete. 2024-04-02T13:53:35.033 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:34 smithi052 ceph-mon[32104]: Updating smithi052:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:53:35.033 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:34 smithi052 ceph-mon[32104]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:53:35.033 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:34 smithi052 ceph-mon[32104]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:53:35.033 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:34 smithi052 ceph-mon[32104]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:53:35.033 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:34 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/198583162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:53:35.034 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:34 smithi052 ceph-mon[32104]: Updating smithi052:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:53:35.034 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:34 smithi052 ceph-mon[32104]: Updating smithi189:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:53:35.034 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:34 smithi052 ceph-mon[32104]: Updating smithi088:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:53:35.034 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:34 smithi052 ceph-mon[32104]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-02T13:53:35.034 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:34 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:35.034 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:34 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:35.034 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:34 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3209286520' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:53:35.034 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:34 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:35.034 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:34 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:35.034 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:34 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:35.034 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:34 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:35.035 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:34 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:35.035 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:34 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:53:35.035 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:34 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:53:35.035 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:34 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:53:35.035 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:34 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:35.035 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:34 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2393661751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:53:35.036 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-02T13:53:33.911035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192590,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1491,"num_read_kb":915,"num_write":553,"num_write_kb":4408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":939,"ondisk_log_size":939,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":414500,"kb_used_data":15388,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124527324,"statfs":{"total":1151940427776,"available":1151515979776,"internally_reserved":0,"allocated":15757312,"data_stored":8995522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":408532282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.991083"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.175053+0000","last_change":"2024-04-02T13:51:01.253563+0000","last_active":"2024-04-02T13:53:30.175053+0000","last_peered":"2024-04-02T13:53:30.175053+0000","last_clean":"2024-04-02T13:53:30.175053+0000","last_became_active":"2024-04-02T13:51:01.253185+0000","last_became_peered":"2024-04-02T13:51:01.253185+0000","last_unstale":"2024-04-02T13:53:30.175053+0000","last_undegraded":"2024-04-02T13:53:30.175053+0000","last_fullsized":"2024-04-02T13:53:30.175053+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:51:05.301880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.175203+0000","last_change":"2024-04-02T13:51:01.256406+0000","last_active":"2024-04-02T13:53:30.175203+0000","last_peered":"2024-04-02T13:53:30.175203+0000","last_clean":"2024-04-02T13:53:30.175203+0000","last_became_active":"2024-04-02T13:51:01.256068+0000","last_became_peered":"2024-04-02T13:51:01.256068+0000","last_unstale":"2024-04-02T13:53:30.175203+0000","last_undegraded":"2024-04-02T13:53:30.175203+0000","last_fullsized":"2024-04-02T13:53:30.175203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:56:51.184932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174787+0000","last_change":"2024-04-02T13:51:01.265925+0000","last_active":"2024-04-02T13:53:30.174787+0000","last_peered":"2024-04-02T13:53:30.174787+0000","last_clean":"2024-04-02T13:53:30.174787+0000","last_became_active":"2024-04-02T13:51:01.265587+0000","last_became_peered":"2024-04-02T13:51:01.265587+0000","last_unstale":"2024-04-02T13:53:30.174787+0000","last_undegraded":"2024-04-02T13:53:30.174787+0000","last_fullsized":"2024-04-02T13:53:30.174787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:40:59.426886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.994837+0000","last_change":"2024-04-02T13:51:56.482512+0000","last_active":"2024-04-02T13:53:29.994837+0000","last_peered":"2024-04-02T13:53:29.994837+0000","last_clean":"2024-04-02T13:53:29.994837+0000","last_became_active":"2024-04-02T13:51:56.481439+0000","last_became_peered":"2024-04-02T13:51:56.481439+0000","last_unstale":"2024-04-02T13:53:29.994837+0000","last_undegraded":"2024-04-02T13:53:29.994837+0000","last_fullsized":"2024-04-02T13:53:29.994837+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:23:16.969641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997088+0000","last_change":"2024-04-02T13:51:01.250934+0000","last_active":"2024-04-02T13:53:29.997088+0000","last_peered":"2024-04-02T13:53:29.997088+0000","last_clean":"2024-04-02T13:53:29.997088+0000","last_became_active":"2024-04-02T13:51:01.250546+0000","last_became_peered":"2024-04-02T13:51:01.250546+0000","last_unstale":"2024-04-02T13:53:29.997088+0000","last_undegraded":"2024-04-02T13:53:29.997088+0000","last_fullsized":"2024-04-02T13:53:29.997088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:34:17.839207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.003203+0000","last_change":"2024-04-02T13:51:01.256160+0000","last_active":"2024-04-02T13:53:30.003203+0000","last_peered":"2024-04-02T13:53:30.003203+0000","last_clean":"2024-04-02T13:53:30.003203+0000","last_became_active":"2024-04-02T13:51:01.255776+0000","last_became_peered":"2024-04-02T13:51:01.255776+0000","last_unstale":"2024-04-02T13:53:30.003203+0000","last_undegraded":"2024-04-02T13:53:30.003203+0000","last_fullsized":"2024-04-02T13:53:30.003203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:53:16.724662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135704+0000","last_change":"2024-04-02T13:51:01.263049+0000","last_active":"2024-04-02T13:53:30.135704+0000","last_peered":"2024-04-02T13:53:30.135704+0000","last_clean":"2024-04-02T13:53:30.135704+0000","last_became_active":"2024-04-02T13:51:01.262660+0000","last_became_peered":"2024-04-02T13:51:01.262660+0000","last_unstale":"2024-04-02T13:53:30.135704+0000","last_undegraded":"2024-04-02T13:53:30.135704+0000","last_fullsized":"2024-04-02T13:53:30.135704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:15:32.069188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996829+0000","last_change":"2024-04-02T13:51:01.263064+0000","last_active":"2024-04-02T13:53:29.996829+0000","last_peered":"2024-04-02T13:53:29.996829+0000","last_clean":"2024-04-02T13:53:29.996829+0000","last_became_active":"2024-04-02T13:51:01.262706+0000","last_became_peered":"2024-04-02T13:51:01.262706+0000","last_unstale":"2024-04-02T13:53:29.996829+0000","last_undegraded":"2024-04-02T13:53:29.996829+0000","last_fullsized":"2024-04-02T13:53:29.996829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:46.680723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136166+0000","last_change":"2024-04-02T13:51:01.257347+0000","last_active":"2024-04-02T13:53:30.136166+0000","last_peered":"2024-04-02T13:53:30.136166+0000","last_clean":"2024-04-02T13:53:30.136166+0000","last_became_active":"2024-04-02T13:51:01.257001+0000","last_became_peered":"2024-04-02T13:51:01.257001+0000","last_unstale":"2024-04-02T13:53:30.136166+0000","last_undegraded":"2024-04-02T13:53:30.136166+0000","last_fullsized":"2024-04-02T13:53:30.136166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:31:22.628436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.003671+0000","last_change":"2024-04-02T13:51:01.260533+0000","last_active":"2024-04-02T13:53:30.003671+0000","last_peered":"2024-04-02T13:53:30.003671+0000","last_clean":"2024-04-02T13:53:30.003671+0000","last_became_active":"2024-04-02T13:51:01.260203+0000","last_became_peered":"2024-04-02T13:51:01.260203+0000","last_unstale":"2024-04-02T13:53:30.003671+0000","last_undegraded":"2024-04-02T13:53:30.003671+0000","last_fullsized":"2024-04-02T13:53:30.003671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:00:06.008243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136480+0000","last_change":"2024-04-02T13:51:01.267733+0000","last_active":"2024-04-02T13:53:30.136480+0000","last_peered":"2024-04-02T13:53:30.136480+0000","last_clean":"2024-04-02T13:53:30.136480+0000","last_became_active":"2024-04-02T13:51:01.266642+0000","last_became_peered":"2024-04-02T13:51:01.266642+0000","last_unstale":"2024-04-02T13:53:30.136480+0000","last_undegraded":"2024-04-02T13:53:30.136480+0000","last_fullsized":"2024-04-02T13:53:30.136480+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:39:25.696112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898481+0000","last_change":"2024-04-02T13:51:01.255933+0000","last_active":"2024-04-02T13:53:29.898481+0000","last_peered":"2024-04-02T13:53:29.898481+0000","last_clean":"2024-04-02T13:53:29.898481+0000","last_became_active":"2024-04-02T13:51:01.255558+0000","last_became_peered":"2024-04-02T13:51:01.255558+0000","last_unstale":"2024-04-02T13:53:29.898481+0000","last_undegraded":"2024-04-02T13:53:29.898481+0000","last_fullsized":"2024-04-02T13:53:29.898481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:13:11.537415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894118+0000","last_change":"2024-04-02T13:51:01.256242+0000","last_active":"2024-04-02T13:53:29.894118+0000","last_peered":"2024-04-02T13:53:29.894118+0000","last_clean":"2024-04-02T13:53:29.894118+0000","last_became_active":"2024-04-02T13:51:01.255830+0000","last_became_peered":"2024-04-02T13:51:01.255830+0000","last_unstale":"2024-04-02T13:53:29.894118+0000","last_undegraded":"2024-04-02T13:53:29.894118+0000","last_fullsized":"2024-04-02T13:53:29.894118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:39.130894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174674+0000","last_change":"2024-04-02T13:51:01.259738+0000","last_active":"2024-04-02T13:53:30.174674+0000","last_peered":"2024-04-02T13:53:30.174674+0000","last_clean":"2024-04-02T13:53:30.174674+0000","last_became_active":"2024-04-02T13:51:01.258630+0000","last_became_peered":"2024-04-02T13:51:01.258630+0000","last_unstale":"2024-04-02T13:53:30.174674+0000","last_undegraded":"2024-04-02T13:53:30.174674+0000","last_fullsized":"2024-04-02T13:53:30.174674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:12:05.401171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.175018+0000","last_change":"2024-04-02T13:51:01.257684+0000","last_active":"2024-04-02T13:53:30.175018+0000","last_peered":"2024-04-02T13:53:30.175018+0000","last_clean":"2024-04-02T13:53:30.175018+0000","last_became_active":"2024-04-02T13:51:01.257267+0000","last_became_peered":"2024-04-02T13:51:01.257267+0000","last_unstale":"2024-04-02T13:53:30.175018+0000","last_undegraded":"2024-04-02T13:53:30.175018+0000","last_fullsized":"2024-04-02T13:53:30.175018+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:43:45.728233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135320+0000","last_change":"2024-04-02T13:51:01.261720+0000","last_active":"2024-04-02T13:53:30.135320+0000","last_peered":"2024-04-02T13:53:30.135320+0000","last_clean":"2024-04-02T13:53:30.135320+0000","last_became_active":"2024-04-02T13:51:01.261407+0000","last_became_peered":"2024-04-02T13:51:01.261407+0000","last_unstale":"2024-04-02T13:53:30.135320+0000","last_undegraded":"2024-04-02T13:53:30.135320+0000","last_fullsized":"2024-04-02T13:53:30.135320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:54:58.807606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174875+0000","last_change":"2024-04-02T13:51:01.254488+0000","last_active":"2024-04-02T13:53:30.174875+0000","last_peered":"2024-04-02T13:53:30.174875+0000","last_clean":"2024-04-02T13:53:30.174875+0000","last_became_active":"2024-04-02T13:51:01.254139+0000","last_became_peered":"2024-04-02T13:51:01.254139+0000","last_unstale":"2024-04-02T13:53:30.174875+0000","last_undegraded":"2024-04-02T13:53:30.174875+0000","last_fullsized":"2024-04-02T13:53:30.174875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:02:41.075727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.995012+0000","last_change":"2024-04-02T13:51:01.256107+0000","last_active":"2024-04-02T13:53:29.995012+0000","last_peered":"2024-04-02T13:53:29.995012+0000","last_clean":"2024-04-02T13:53:29.995012+0000","last_became_active":"2024-04-02T13:51:01.255754+0000","last_became_peered":"2024-04-02T13:51:01.255754+0000","last_unstale":"2024-04-02T13:53:29.995012+0000","last_undegraded":"2024-04-02T13:53:29.995012+0000","last_fullsized":"2024-04-02T13:53:29.995012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:36:59.793519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135517+0000","last_change":"2024-04-02T13:51:01.247550+0000","last_active":"2024-04-02T13:53:30.135517+0000","last_peered":"2024-04-02T13:53:30.135517+0000","last_clean":"2024-04-02T13:53:30.135517+0000","last_became_active":"2024-04-02T13:51:01.247236+0000","last_became_peered":"2024-04-02T13:51:01.247236+0000","last_unstale":"2024-04-02T13:53:30.135517+0000","last_undegraded":"2024-04-02T13:53:30.135517+0000","last_fullsized":"2024-04-02T13:53:30.135517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:16:02.940700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135216+0000","last_change":"2024-04-02T13:51:56.663779+0000","last_active":"2024-04-02T13:53:30.135216+0000","last_peered":"2024-04-02T13:53:30.135216+0000","last_clean":"2024-04-02T13:53:30.135216+0000","last_became_active":"2024-04-02T13:51:56.663327+0000","last_became_peered":"2024-04-02T13:51:56.663327+0000","last_unstale":"2024-04-02T13:53:30.135216+0000","last_undegraded":"2024-04-02T13:53:30.135216+0000","last_fullsized":"2024-04-02T13:53:30.135216+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:35:34.186278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"93'14","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.137322+0000","last_change":"2024-04-02T13:51:00.243244+0000","last_active":"2024-04-02T13:53:30.137322+0000","last_peered":"2024-04-02T13:53:30.137322+0000","last_clean":"2024-04-02T13:53:30.137322+0000","last_became_active":"2024-04-02T13:51:00.242892+0000","last_became_peered":"2024-04-02T13:51:00.242892+0000","last_unstale":"2024-04-02T13:53:30.137322+0000","last_undegraded":"2024-04-02T13:53:30.137322+0000","last_fullsized":"2024-04-02T13:53:30.137322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:49:23.338215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":123,"num_read_kb":123,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997790+0000","last_change":"2024-04-02T13:51:01.264640+0000","last_active":"2024-04-02T13:53:29.997790+0000","last_peered":"2024-04-02T13:53:29.997790+0000","last_clean":"2024-04-02T13:53:29.997790+0000","last_became_active":"2024-04-02T13:51:01.264161+0000","last_became_peered":"2024-04-02T13:51:01.264161+0000","last_unstale":"2024-04-02T13:53:29.997790+0000","last_undegraded":"2024-04-02T13:53:29.997790+0000","last_fullsized":"2024-04-02T13:53:29.997790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:30:47.245647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898007+0000","last_change":"2024-04-02T13:51:05.362888+0000","last_active":"2024-04-02T13:53:29.898007+0000","last_peered":"2024-04-02T13:53:29.898007+0000","last_clean":"2024-04-02T13:53:29.898007+0000","last_became_active":"2024-04-02T13:51:05.362478+0000","last_became_peered":"2024-04-02T13:51:05.362478+0000","last_unstale":"2024-04-02T13:53:29.898007+0000","last_undegraded":"2024-04-02T13:53:29.898007+0000","last_fullsized":"2024-04-02T13:53:29.898007+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:05:54.304458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997831+0000","last_change":"2024-04-02T13:51:00.247800+0000","last_active":"2024-04-02T13:53:29.997831+0000","last_peered":"2024-04-02T13:53:29.997831+0000","last_clean":"2024-04-02T13:53:29.997831+0000","last_became_active":"2024-04-02T13:51:00.247485+0000","last_became_peered":"2024-04-02T13:51:00.247485+0000","last_unstale":"2024-04-02T13:53:29.997831+0000","last_undegraded":"2024-04-02T13:53:29.997831+0000","last_fullsized":"2024-04-02T13:53:29.997831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:17:24.767558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.175194+0000","last_change":"2024-04-02T13:51:01.261416+0000","last_active":"2024-04-02T13:53:30.175194+0000","last_peered":"2024-04-02T13:53:30.175194+0000","last_clean":"2024-04-02T13:53:30.175194+0000","last_became_active":"2024-04-02T13:51:01.261099+0000","last_became_peered":"2024-04-02T13:51:01.261099+0000","last_unstale":"2024-04-02T13:53:30.175194+0000","last_undegraded":"2024-04-02T13:53:30.175194+0000","last_fullsized":"2024-04-02T13:53:30.175194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:10:04.999402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.897988+0000","last_change":"2024-04-02T13:51:05.361747+0000","last_active":"2024-04-02T13:53:29.897988+0000","last_peered":"2024-04-02T13:53:29.897988+0000","last_clean":"2024-04-02T13:53:29.897988+0000","last_became_active":"2024-04-02T13:51:05.361098+0000","last_became_peered":"2024-04-02T13:51:05.361098+0000","last_unstale":"2024-04-02T13:53:29.897988+0000","last_undegraded":"2024-04-02T13:53:29.897988+0000","last_fullsized":"2024-04-02T13:53:29.897988+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:45:35.048870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997422+0000","last_change":"2024-04-02T13:51:00.244048+0000","last_active":"2024-04-02T13:53:29.997422+0000","last_peered":"2024-04-02T13:53:29.997422+0000","last_clean":"2024-04-02T13:53:29.997422+0000","last_became_active":"2024-04-02T13:51:00.243510+0000","last_became_peered":"2024-04-02T13:51:00.243510+0000","last_unstale":"2024-04-02T13:53:29.997422+0000","last_undegraded":"2024-04-02T13:53:29.997422+0000","last_fullsized":"2024-04-02T13:53:29.997422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:53:48.364137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136246+0000","last_change":"2024-04-02T13:51:01.264711+0000","last_active":"2024-04-02T13:53:30.136246+0000","last_peered":"2024-04-02T13:53:30.136246+0000","last_clean":"2024-04-02T13:53:30.136246+0000","last_became_active":"2024-04-02T13:51:01.264289+0000","last_became_peered":"2024-04-02T13:51:01.264289+0000","last_unstale":"2024-04-02T13:53:30.136246+0000","last_undegraded":"2024-04-02T13:53:30.136246+0000","last_fullsized":"2024-04-02T13:53:30.136246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:34:41.590071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.897930+0000","last_change":"2024-04-02T13:51:05.363121+0000","last_active":"2024-04-02T13:53:29.897930+0000","last_peered":"2024-04-02T13:53:29.897930+0000","last_clean":"2024-04-02T13:53:29.897930+0000","last_became_active":"2024-04-02T13:51:05.362642+0000","last_became_peered":"2024-04-02T13:51:05.362642+0000","last_unstale":"2024-04-02T13:53:29.897930+0000","last_undegraded":"2024-04-02T13:53:29.897930+0000","last_fullsized":"2024-04-02T13:53:29.897930+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:55:40.848731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997155+0000","last_change":"2024-04-02T13:51:00.236519+0000","last_active":"2024-04-02T13:53:29.997155+0000","last_peered":"2024-04-02T13:53:29.997155+0000","last_clean":"2024-04-02T13:53:29.997155+0000","last_became_active":"2024-04-02T13:51:00.235985+0000","last_became_peered":"2024-04-02T13:51:00.235985+0000","last_unstale":"2024-04-02T13:53:29.997155+0000","last_undegraded":"2024-04-02T13:53:29.997155+0000","last_fullsized":"2024-04-02T13:53:29.997155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:08.321765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895438+0000","last_change":"2024-04-02T13:51:01.260333+0000","last_active":"2024-04-02T13:53:29.895438+0000","last_peered":"2024-04-02T13:53:29.895438+0000","last_clean":"2024-04-02T13:53:29.895438+0000","last_became_active":"2024-04-02T13:51:01.259995+0000","last_became_peered":"2024-04-02T13:51:01.259995+0000","last_unstale":"2024-04-02T13:53:29.895438+0000","last_undegraded":"2024-04-02T13:53:29.895438+0000","last_fullsized":"2024-04-02T13:53:29.895438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:03:58.088653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135314+0000","last_change":"2024-04-02T13:51:05.359281+0000","last_active":"2024-04-02T13:53:30.135314+0000","last_peered":"2024-04-02T13:53:30.135314+0000","last_clean":"2024-04-02T13:53:30.135314+0000","last_became_active":"2024-04-02T13:51:05.358831+0000","last_became_peered":"2024-04-02T13:51:05.358831+0000","last_unstale":"2024-04-02T13:53:30.135314+0000","last_undegraded":"2024-04-02T13:53:30.135314+0000","last_fullsized":"2024-04-02T13:53:30.135314+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:48.461298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'7","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.508533+0000","last_change":"2024-04-02T13:51:00.241707+0000","last_active":"2024-04-02T13:53:30.508533+0000","last_peered":"2024-04-02T13:53:30.508533+0000","last_clean":"2024-04-02T13:53:30.508533+0000","last_became_active":"2024-04-02T13:51:00.241225+0000","last_became_peered":"2024-04-02T13:51:00.241225+0000","last_unstale":"2024-04-02T13:53:30.508533+0000","last_undegraded":"2024-04-02T13:53:30.508533+0000","last_fullsized":"2024-04-02T13:53:30.508533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:59:44.537503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1404,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997009+0000","last_change":"2024-04-02T13:51:01.252942+0000","last_active":"2024-04-02T13:53:29.997009+0000","last_peered":"2024-04-02T13:53:29.997009+0000","last_clean":"2024-04-02T13:53:29.997009+0000","last_became_active":"2024-04-02T13:51:01.252631+0000","last_became_peered":"2024-04-02T13:51:01.252631+0000","last_unstale":"2024-04-02T13:53:29.997009+0000","last_undegraded":"2024-04-02T13:53:29.997009+0000","last_fullsized":"2024-04-02T13:53:29.997009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:43:24.725688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135372+0000","last_change":"2024-04-02T13:51:05.359208+0000","last_active":"2024-04-02T13:53:30.135372+0000","last_peered":"2024-04-02T13:53:30.135372+0000","last_clean":"2024-04-02T13:53:30.135372+0000","last_became_active":"2024-04-02T13:51:05.358738+0000","last_became_peered":"2024-04-02T13:51:05.358738+0000","last_unstale":"2024-04-02T13:53:30.135372+0000","last_undegraded":"2024-04-02T13:53:30.135372+0000","last_fullsized":"2024-04-02T13:53:30.135372+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:47:46.034562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.003266+0000","last_change":"2024-04-02T13:51:00.236413+0000","last_active":"2024-04-02T13:53:30.003266+0000","last_peered":"2024-04-02T13:53:30.003266+0000","last_clean":"2024-04-02T13:53:30.003266+0000","last_became_active":"2024-04-02T13:51:00.236046+0000","last_became_peered":"2024-04-02T13:51:00.236046+0000","last_unstale":"2024-04-02T13:53:30.003266+0000","last_undegraded":"2024-04-02T13:53:30.003266+0000","last_fullsized":"2024-04-02T13:53:30.003266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:25:15.000726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895344+0000","last_change":"2024-04-02T13:51:01.265471+0000","last_active":"2024-04-02T13:53:29.895344+0000","last_peered":"2024-04-02T13:53:29.895344+0000","last_clean":"2024-04-02T13:53:29.895344+0000","last_became_active":"2024-04-02T13:51:01.265114+0000","last_became_peered":"2024-04-02T13:51:01.265114+0000","last_unstale":"2024-04-02T13:53:29.895344+0000","last_undegraded":"2024-04-02T13:53:29.895344+0000","last_fullsized":"2024-04-02T13:53:29.895344+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:01:01.675686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174824+0000","last_change":"2024-04-02T13:51:56.664063+0000","last_active":"2024-04-02T13:53:30.174824+0000","last_peered":"2024-04-02T13:53:30.174824+0000","last_clean":"2024-04-02T13:53:30.174824+0000","last_became_active":"2024-04-02T13:51:56.663590+0000","last_became_peered":"2024-04-02T13:51:56.663590+0000","last_unstale":"2024-04-02T13:53:30.174824+0000","last_undegraded":"2024-04-02T13:53:30.174824+0000","last_fullsized":"2024-04-02T13:53:30.174824+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:43:51.735851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.893833+0000","last_change":"2024-04-02T13:51:00.237204+0000","last_active":"2024-04-02T13:53:29.893833+0000","last_peered":"2024-04-02T13:53:29.893833+0000","last_clean":"2024-04-02T13:53:29.893833+0000","last_became_active":"2024-04-02T13:51:00.236707+0000","last_became_peered":"2024-04-02T13:51:00.236707+0000","last_unstale":"2024-04-02T13:53:29.893833+0000","last_undegraded":"2024-04-02T13:53:29.893833+0000","last_fullsized":"2024-04-02T13:53:29.893833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:00:17.771428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997017+0000","last_change":"2024-04-02T13:51:01.250152+0000","last_active":"2024-04-02T13:53:29.997017+0000","last_peered":"2024-04-02T13:53:29.997017+0000","last_clean":"2024-04-02T13:53:29.997017+0000","last_became_active":"2024-04-02T13:51:01.249817+0000","last_became_peered":"2024-04-02T13:51:01.249817+0000","last_unstale":"2024-04-02T13:53:29.997017+0000","last_undegraded":"2024-04-02T13:53:29.997017+0000","last_fullsized":"2024-04-02T13:53:29.997017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:13:31.719380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.175093+0000","last_change":"2024-04-02T13:51:05.366551+0000","last_active":"2024-04-02T13:53:30.175093+0000","last_peered":"2024-04-02T13:53:30.175093+0000","last_clean":"2024-04-02T13:53:30.175093+0000","last_became_active":"2024-04-02T13:51:05.366175+0000","last_became_peered":"2024-04-02T13:51:05.366175+0000","last_unstale":"2024-04-02T13:53:30.175093+0000","last_undegraded":"2024-04-02T13:53:30.175093+0000","last_fullsized":"2024-04-02T13:53:30.175093+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:47:49.822918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895052+0000","last_change":"2024-04-02T13:51:00.239521+0000","last_active":"2024-04-02T13:53:29.895052+0000","last_peered":"2024-04-02T13:53:29.895052+0000","last_clean":"2024-04-02T13:53:29.895052+0000","last_became_active":"2024-04-02T13:51:00.238608+0000","last_became_peered":"2024-04-02T13:51:00.238608+0000","last_unstale":"2024-04-02T13:53:29.895052+0000","last_undegraded":"2024-04-02T13:53:29.895052+0000","last_fullsized":"2024-04-02T13:53:29.895052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:00.983006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.994418+0000","last_change":"2024-04-02T13:51:01.255021+0000","last_active":"2024-04-02T13:53:29.994418+0000","last_peered":"2024-04-02T13:53:29.994418+0000","last_clean":"2024-04-02T13:53:29.994418+0000","last_became_active":"2024-04-02T13:51:01.254630+0000","last_became_peered":"2024-04-02T13:51:01.254630+0000","last_unstale":"2024-04-02T13:53:29.994418+0000","last_undegraded":"2024-04-02T13:53:29.994418+0000","last_fullsized":"2024-04-02T13:53:29.994418+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:36:08.773431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996891+0000","last_change":"2024-04-02T13:51:05.362004+0000","last_active":"2024-04-02T13:53:29.996891+0000","last_peered":"2024-04-02T13:53:29.996891+0000","last_clean":"2024-04-02T13:53:29.996891+0000","last_became_active":"2024-04-02T13:51:05.361538+0000","last_became_peered":"2024-04-02T13:51:05.361538+0000","last_unstale":"2024-04-02T13:53:29.996891+0000","last_undegraded":"2024-04-02T13:53:29.996891+0000","last_fullsized":"2024-04-02T13:53:29.996891+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:27:21.728695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'6","reported_seq":280,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.507365+0000","last_change":"2024-04-02T13:51:00.234814+0000","last_active":"2024-04-02T13:53:30.507365+0000","last_peered":"2024-04-02T13:53:30.507365+0000","last_clean":"2024-04-02T13:53:30.507365+0000","last_became_active":"2024-04-02T13:51:00.234481+0000","last_became_peered":"2024-04-02T13:51:00.234481+0000","last_unstale":"2024-04-02T13:53:30.507365+0000","last_undegraded":"2024-04-02T13:53:30.507365+0000","last_fullsized":"2024-04-02T13:53:30.507365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:45:34.901226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":244,"num_read_kb":244,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135656+0000","last_change":"2024-04-02T13:51:01.264093+0000","last_active":"2024-04-02T13:53:30.135656+0000","last_peered":"2024-04-02T13:53:30.135656+0000","last_clean":"2024-04-02T13:53:30.135656+0000","last_became_active":"2024-04-02T13:51:01.263754+0000","last_became_peered":"2024-04-02T13:51:01.263754+0000","last_unstale":"2024-04-02T13:53:30.135656+0000","last_undegraded":"2024-04-02T13:53:30.135656+0000","last_fullsized":"2024-04-02T13:53:30.135656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:49:18.630195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.994363+0000","last_change":"2024-04-02T13:51:05.370930+0000","last_active":"2024-04-02T13:53:29.994363+0000","last_peered":"2024-04-02T13:53:29.994363+0000","last_clean":"2024-04-02T13:53:29.994363+0000","last_became_active":"2024-04-02T13:51:05.370443+0000","last_became_peered":"2024-04-02T13:51:05.370443+0000","last_unstale":"2024-04-02T13:53:29.994363+0000","last_undegraded":"2024-04-02T13:53:29.994363+0000","last_fullsized":"2024-04-02T13:53:29.994363+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:24:27.055494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136337+0000","last_change":"2024-04-02T13:51:00.241206+0000","last_active":"2024-04-02T13:53:30.136337+0000","last_peered":"2024-04-02T13:53:30.136337+0000","last_clean":"2024-04-02T13:53:30.136337+0000","last_became_active":"2024-04-02T13:51:00.240832+0000","last_became_peered":"2024-04-02T13:51:00.240832+0000","last_unstale":"2024-04-02T13:53:30.136337+0000","last_undegraded":"2024-04-02T13:53:30.136337+0000","last_fullsized":"2024-04-02T13:53:30.136337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:44:11.626435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.175384+0000","last_change":"2024-04-02T13:51:01.259315+0000","last_active":"2024-04-02T13:53:30.175384+0000","last_peered":"2024-04-02T13:53:30.175384+0000","last_clean":"2024-04-02T13:53:30.175384+0000","last_became_active":"2024-04-02T13:51:01.258951+0000","last_became_peered":"2024-04-02T13:51:01.258951+0000","last_unstale":"2024-04-02T13:53:30.175384+0000","last_undegraded":"2024-04-02T13:53:30.175384+0000","last_fullsized":"2024-04-02T13:53:30.175384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:59:43.714044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898359+0000","last_change":"2024-04-02T13:51:56.665826+0000","last_active":"2024-04-02T13:53:29.898359+0000","last_peered":"2024-04-02T13:53:29.898359+0000","last_clean":"2024-04-02T13:53:29.898359+0000","last_became_active":"2024-04-02T13:51:56.665213+0000","last_became_peered":"2024-04-02T13:51:56.665213+0000","last_unstale":"2024-04-02T13:53:29.898359+0000","last_undegraded":"2024-04-02T13:53:29.898359+0000","last_fullsized":"2024-04-02T13:53:29.898359+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:45:46.625063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895401+0000","last_change":"2024-04-02T13:51:00.230938+0000","last_active":"2024-04-02T13:53:29.895401+0000","last_peered":"2024-04-02T13:53:29.895401+0000","last_clean":"2024-04-02T13:53:29.895401+0000","last_became_active":"2024-04-02T13:51:00.230585+0000","last_became_peered":"2024-04-02T13:51:00.230585+0000","last_unstale":"2024-04-02T13:53:29.895401+0000","last_undegraded":"2024-04-02T13:53:29.895401+0000","last_fullsized":"2024-04-02T13:53:29.895401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:51:37.463876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136591+0000","last_change":"2024-04-02T13:51:01.264216+0000","last_active":"2024-04-02T13:53:30.136591+0000","last_peered":"2024-04-02T13:53:30.136591+0000","last_clean":"2024-04-02T13:53:30.136591+0000","last_became_active":"2024-04-02T13:51:01.263824+0000","last_became_peered":"2024-04-02T13:51:01.263824+0000","last_unstale":"2024-04-02T13:53:30.136591+0000","last_undegraded":"2024-04-02T13:53:30.136591+0000","last_fullsized":"2024-04-02T13:53:30.136591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:16:26.391163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.994569+0000","last_change":"2024-04-02T13:51:05.366002+0000","last_active":"2024-04-02T13:53:29.994569+0000","last_peered":"2024-04-02T13:53:29.994569+0000","last_clean":"2024-04-02T13:53:29.994569+0000","last_became_active":"2024-04-02T13:51:05.365547+0000","last_became_peered":"2024-04-02T13:51:05.365547+0000","last_unstale":"2024-04-02T13:53:29.994569+0000","last_undegraded":"2024-04-02T13:53:29.994569+0000","last_fullsized":"2024-04-02T13:53:29.994569+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:31:38.990617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'43","reported_seq":327,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.507596+0000","last_change":"2024-04-02T13:51:00.233925+0000","last_active":"2024-04-02T13:53:30.507596+0000","last_peered":"2024-04-02T13:53:30.507596+0000","last_clean":"2024-04-02T13:53:30.507596+0000","last_became_active":"2024-04-02T13:51:00.233574+0000","last_became_peered":"2024-04-02T13:51:00.233574+0000","last_unstale":"2024-04-02T13:53:30.507596+0000","last_undegraded":"2024-04-02T13:53:30.507596+0000","last_fullsized":"2024-04-02T13:53:30.507596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:33:32.758874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69045,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":254,"num_read_kb":196,"num_write":45,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997226+0000","last_change":"2024-04-02T13:51:01.258825+0000","last_active":"2024-04-02T13:53:29.997226+0000","last_peered":"2024-04-02T13:53:29.997226+0000","last_clean":"2024-04-02T13:53:29.997226+0000","last_became_active":"2024-04-02T13:51:01.258500+0000","last_became_peered":"2024-04-02T13:51:01.258500+0000","last_unstale":"2024-04-02T13:53:29.997226+0000","last_undegraded":"2024-04-02T13:53:29.997226+0000","last_fullsized":"2024-04-02T13:53:29.997226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:33:15.409765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895709+0000","last_change":"2024-04-02T13:51:05.365010+0000","last_active":"2024-04-02T13:53:29.895709+0000","last_peered":"2024-04-02T13:53:29.895709+0000","last_clean":"2024-04-02T13:53:29.895709+0000","last_became_active":"2024-04-02T13:51:05.364499+0000","last_became_peered":"2024-04-02T13:51:05.364499+0000","last_unstale":"2024-04-02T13:53:29.895709+0000","last_undegraded":"2024-04-02T13:53:29.895709+0000","last_fullsized":"2024-04-02T13:53:29.895709+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:07:03.500017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174673+0000","last_change":"2024-04-02T13:51:56.496876+0000","last_active":"2024-04-02T13:53:30.174673+0000","last_peered":"2024-04-02T13:53:30.174673+0000","last_clean":"2024-04-02T13:53:30.174673+0000","last_became_active":"2024-04-02T13:51:56.484672+0000","last_became_peered":"2024-04-02T13:51:56.484672+0000","last_unstale":"2024-04-02T13:53:30.174673+0000","last_undegraded":"2024-04-02T13:53:30.174673+0000","last_fullsized":"2024-04-02T13:53:30.174673+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:31:40.027658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.893864+0000","last_change":"2024-04-02T13:51:01.256334+0000","last_active":"2024-04-02T13:53:29.893864+0000","last_peered":"2024-04-02T13:53:29.893864+0000","last_clean":"2024-04-02T13:53:29.893864+0000","last_became_active":"2024-04-02T13:51:01.255964+0000","last_became_peered":"2024-04-02T13:51:01.255964+0000","last_unstale":"2024-04-02T13:53:29.893864+0000","last_undegraded":"2024-04-02T13:53:29.893864+0000","last_fullsized":"2024-04-02T13:53:29.893864+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:43:04.842812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174877+0000","last_change":"2024-04-02T13:51:05.362696+0000","last_active":"2024-04-02T13:53:30.174877+0000","last_peered":"2024-04-02T13:53:30.174877+0000","last_clean":"2024-04-02T13:53:30.174877+0000","last_became_active":"2024-04-02T13:51:05.362225+0000","last_became_peered":"2024-04-02T13:51:05.362225+0000","last_unstale":"2024-04-02T13:53:30.174877+0000","last_undegraded":"2024-04-02T13:53:30.174877+0000","last_fullsized":"2024-04-02T13:53:30.174877+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:23:56.308286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895454+0000","last_change":"2024-04-02T13:51:00.239673+0000","last_active":"2024-04-02T13:53:29.895454+0000","last_peered":"2024-04-02T13:53:29.895454+0000","last_clean":"2024-04-02T13:53:29.895454+0000","last_became_active":"2024-04-02T13:51:00.239233+0000","last_became_peered":"2024-04-02T13:51:00.239233+0000","last_unstale":"2024-04-02T13:53:29.895454+0000","last_undegraded":"2024-04-02T13:53:29.895454+0000","last_fullsized":"2024-04-02T13:53:29.895454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:05:48.119953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135940+0000","last_change":"2024-04-02T13:51:01.257474+0000","last_active":"2024-04-02T13:53:30.135940+0000","last_peered":"2024-04-02T13:53:30.135940+0000","last_clean":"2024-04-02T13:53:30.135940+0000","last_became_active":"2024-04-02T13:51:01.254977+0000","last_became_peered":"2024-04-02T13:51:01.254977+0000","last_unstale":"2024-04-02T13:53:30.135940+0000","last_undegraded":"2024-04-02T13:53:30.135940+0000","last_fullsized":"2024-04-02T13:53:30.135940+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:13:07.446520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.994825+0000","last_change":"2024-04-02T13:51:05.371531+0000","last_active":"2024-04-02T13:53:29.994825+0000","last_peered":"2024-04-02T13:53:29.994825+0000","last_clean":"2024-04-02T13:53:29.994825+0000","last_became_active":"2024-04-02T13:51:05.370551+0000","last_became_peered":"2024-04-02T13:51:05.370551+0000","last_unstale":"2024-04-02T13:53:29.994825+0000","last_undegraded":"2024-04-02T13:53:29.994825+0000","last_fullsized":"2024-04-02T13:53:29.994825+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:43:48.796009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'5","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898007+0000","last_change":"2024-04-02T13:51:00.237715+0000","last_active":"2024-04-02T13:53:29.898007+0000","last_peered":"2024-04-02T13:53:29.898007+0000","last_clean":"2024-04-02T13:53:29.898007+0000","last_became_active":"2024-04-02T13:51:00.237349+0000","last_became_peered":"2024-04-02T13:51:00.237349+0000","last_unstale":"2024-04-02T13:53:29.898007+0000","last_undegraded":"2024-04-02T13:53:29.898007+0000","last_fullsized":"2024-04-02T13:53:29.898007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:33:20.651562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135285+0000","last_change":"2024-04-02T13:51:01.259447+0000","last_active":"2024-04-02T13:53:30.135285+0000","last_peered":"2024-04-02T13:53:30.135285+0000","last_clean":"2024-04-02T13:53:30.135285+0000","last_became_active":"2024-04-02T13:51:01.259101+0000","last_became_peered":"2024-04-02T13:51:01.259101+0000","last_unstale":"2024-04-02T13:53:30.135285+0000","last_undegraded":"2024-04-02T13:53:30.135285+0000","last_fullsized":"2024-04-02T13:53:30.135285+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:51:44.311440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894370+0000","last_change":"2024-04-02T13:51:05.372446+0000","last_active":"2024-04-02T13:53:29.894370+0000","last_peered":"2024-04-02T13:53:29.894370+0000","last_clean":"2024-04-02T13:53:29.894370+0000","last_became_active":"2024-04-02T13:51:05.372007+0000","last_became_peered":"2024-04-02T13:51:05.372007+0000","last_unstale":"2024-04-02T13:53:29.894370+0000","last_undegraded":"2024-04-02T13:53:29.894370+0000","last_fullsized":"2024-04-02T13:53:29.894370+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:54:23.463179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996804+0000","last_change":"2024-04-02T13:51:00.241492+0000","last_active":"2024-04-02T13:53:29.996804+0000","last_peered":"2024-04-02T13:53:29.996804+0000","last_clean":"2024-04-02T13:53:29.996804+0000","last_became_active":"2024-04-02T13:51:00.241167+0000","last_became_peered":"2024-04-02T13:51:00.241167+0000","last_unstale":"2024-04-02T13:53:29.996804+0000","last_undegraded":"2024-04-02T13:53:29.996804+0000","last_fullsized":"2024-04-02T13:53:29.996804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:44:44.070187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895050+0000","last_change":"2024-04-02T13:51:01.263200+0000","last_active":"2024-04-02T13:53:29.895050+0000","last_peered":"2024-04-02T13:53:29.895050+0000","last_clean":"2024-04-02T13:53:29.895050+0000","last_became_active":"2024-04-02T13:51:01.262812+0000","last_became_peered":"2024-04-02T13:51:01.262812+0000","last_unstale":"2024-04-02T13:53:29.895050+0000","last_undegraded":"2024-04-02T13:53:29.895050+0000","last_fullsized":"2024-04-02T13:53:29.895050+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:33:35.898047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895041+0000","last_change":"2024-04-02T13:51:05.366729+0000","last_active":"2024-04-02T13:53:29.895041+0000","last_peered":"2024-04-02T13:53:29.895041+0000","last_clean":"2024-04-02T13:53:29.895041+0000","last_became_active":"2024-04-02T13:51:05.366257+0000","last_became_peered":"2024-04-02T13:51:05.366257+0000","last_unstale":"2024-04-02T13:53:29.895041+0000","last_undegraded":"2024-04-02T13:53:29.895041+0000","last_fullsized":"2024-04-02T13:53:29.895041+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:46:03.400300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997357+0000","last_change":"2024-04-02T13:51:00.236004+0000","last_active":"2024-04-02T13:53:29.997357+0000","last_peered":"2024-04-02T13:53:29.997357+0000","last_clean":"2024-04-02T13:53:29.997357+0000","last_became_active":"2024-04-02T13:51:00.235160+0000","last_became_peered":"2024-04-02T13:51:00.235160+0000","last_unstale":"2024-04-02T13:53:29.997357+0000","last_undegraded":"2024-04-02T13:53:29.997357+0000","last_fullsized":"2024-04-02T13:53:29.997357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:16:35.617346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174808+0000","last_change":"2024-04-02T13:51:01.259565+0000","last_active":"2024-04-02T13:53:30.174808+0000","last_peered":"2024-04-02T13:53:30.174808+0000","last_clean":"2024-04-02T13:53:30.174808+0000","last_became_active":"2024-04-02T13:51:01.259225+0000","last_became_peered":"2024-04-02T13:51:01.259225+0000","last_unstale":"2024-04-02T13:53:30.174808+0000","last_undegraded":"2024-04-02T13:53:30.174808+0000","last_fullsized":"2024-04-02T13:53:30.174808+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:22:18.878029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996871+0000","last_change":"2024-04-02T13:51:05.518750+0000","last_active":"2024-04-02T13:53:29.996871+0000","last_peered":"2024-04-02T13:53:29.996871+0000","last_clean":"2024-04-02T13:53:29.996871+0000","last_became_active":"2024-04-02T13:51:05.518247+0000","last_became_peered":"2024-04-02T13:51:05.518247+0000","last_unstale":"2024-04-02T13:53:29.996871+0000","last_undegraded":"2024-04-02T13:53:29.996871+0000","last_fullsized":"2024-04-02T13:53:29.996871+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:08:10.232509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.994718+0000","last_change":"2024-04-02T13:51:00.241476+0000","last_active":"2024-04-02T13:53:29.994718+0000","last_peered":"2024-04-02T13:53:29.994718+0000","last_clean":"2024-04-02T13:53:29.994718+0000","last_became_active":"2024-04-02T13:51:00.241085+0000","last_became_peered":"2024-04-02T13:51:00.241085+0000","last_unstale":"2024-04-02T13:53:29.994718+0000","last_undegraded":"2024-04-02T13:53:29.994718+0000","last_fullsized":"2024-04-02T13:53:29.994718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:29:01.503901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.003509+0000","last_change":"2024-04-02T13:51:01.259791+0000","last_active":"2024-04-02T13:53:30.003509+0000","last_peered":"2024-04-02T13:53:30.003509+0000","last_clean":"2024-04-02T13:53:30.003509+0000","last_became_active":"2024-04-02T13:51:01.259440+0000","last_became_peered":"2024-04-02T13:51:01.259440+0000","last_unstale":"2024-04-02T13:53:30.003509+0000","last_undegraded":"2024-04-02T13:53:30.003509+0000","last_fullsized":"2024-04-02T13:53:30.003509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:43:52.422263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894028+0000","last_change":"2024-04-02T13:51:05.366243+0000","last_active":"2024-04-02T13:53:29.894028+0000","last_peered":"2024-04-02T13:53:29.894028+0000","last_clean":"2024-04-02T13:53:29.894028+0000","last_became_active":"2024-04-02T13:51:05.365801+0000","last_became_peered":"2024-04-02T13:51:05.365801+0000","last_unstale":"2024-04-02T13:53:29.894028+0000","last_undegraded":"2024-04-02T13:53:29.894028+0000","last_fullsized":"2024-04-02T13:53:29.894028+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:33:56.853434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"93'6","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.507013+0000","last_change":"2024-04-02T13:51:00.248695+0000","last_active":"2024-04-02T13:53:30.507013+0000","last_peered":"2024-04-02T13:53:30.507013+0000","last_clean":"2024-04-02T13:53:30.507013+0000","last_became_active":"2024-04-02T13:51:00.248313+0000","last_became_peered":"2024-04-02T13:51:00.248313+0000","last_unstale":"2024-04-02T13:53:30.507013+0000","last_undegraded":"2024-04-02T13:53:30.507013+0000","last_fullsized":"2024-04-02T13:53:30.507013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:43:21.410823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.003516+0000","last_change":"2024-04-02T13:51:01.260993+0000","last_active":"2024-04-02T13:53:30.003516+0000","last_peered":"2024-04-02T13:53:30.003516+0000","last_clean":"2024-04-02T13:53:30.003516+0000","last_became_active":"2024-04-02T13:51:01.260510+0000","last_became_peered":"2024-04-02T13:51:01.260510+0000","last_unstale":"2024-04-02T13:53:30.003516+0000","last_undegraded":"2024-04-02T13:53:30.003516+0000","last_fullsized":"2024-04-02T13:53:30.003516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:17:04.941822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894181+0000","last_change":"2024-04-02T13:51:05.518455+0000","last_active":"2024-04-02T13:53:29.894181+0000","last_peered":"2024-04-02T13:53:29.894181+0000","last_clean":"2024-04-02T13:53:29.894181+0000","last_became_active":"2024-04-02T13:51:05.517962+0000","last_became_peered":"2024-04-02T13:51:05.517962+0000","last_unstale":"2024-04-02T13:53:29.894181+0000","last_undegraded":"2024-04-02T13:53:29.894181+0000","last_fullsized":"2024-04-02T13:53:29.894181+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:19:47.734233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895152+0000","last_change":"2024-04-02T13:51:01.259220+0000","last_active":"2024-04-02T13:53:29.895152+0000","last_peered":"2024-04-02T13:53:29.895152+0000","last_clean":"2024-04-02T13:53:29.895152+0000","last_became_active":"2024-04-02T13:51:01.258890+0000","last_became_peered":"2024-04-02T13:51:01.258890+0000","last_unstale":"2024-04-02T13:53:29.895152+0000","last_undegraded":"2024-04-02T13:53:29.895152+0000","last_fullsized":"2024-04-02T13:53:29.895152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:56:15.444736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.893734+0000","last_change":"2024-04-02T13:51:00.246727+0000","last_active":"2024-04-02T13:53:29.893734+0000","last_peered":"2024-04-02T13:53:29.893734+0000","last_clean":"2024-04-02T13:53:29.893734+0000","last_became_active":"2024-04-02T13:51:00.246364+0000","last_became_peered":"2024-04-02T13:51:00.246364+0000","last_unstale":"2024-04-02T13:53:29.893734+0000","last_undegraded":"2024-04-02T13:53:29.893734+0000","last_fullsized":"2024-04-02T13:53:29.893734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:35:13.249972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":123,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136219+0000","last_change":"2024-04-02T13:51:05.358506+0000","last_active":"2024-04-02T13:53:30.136219+0000","last_peered":"2024-04-02T13:53:30.136219+0000","last_clean":"2024-04-02T13:53:30.136219+0000","last_became_active":"2024-04-02T13:51:05.358157+0000","last_became_peered":"2024-04-02T13:51:05.358157+0000","last_unstale":"2024-04-02T13:53:30.136219+0000","last_undegraded":"2024-04-02T13:53:30.136219+0000","last_fullsized":"2024-04-02T13:53:30.136219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:24:15.585124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.994632+0000","last_change":"2024-04-02T13:51:56.665307+0000","last_active":"2024-04-02T13:53:29.994632+0000","last_peered":"2024-04-02T13:53:29.994632+0000","last_clean":"2024-04-02T13:53:29.994632+0000","last_became_active":"2024-04-02T13:51:56.665003+0000","last_became_peered":"2024-04-02T13:51:56.665003+0000","last_unstale":"2024-04-02T13:53:29.994632+0000","last_undegraded":"2024-04-02T13:53:29.994632+0000","last_fullsized":"2024-04-02T13:53:29.994632+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:02:46.421524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898110+0000","last_change":"2024-04-02T13:51:00.238962+0000","last_active":"2024-04-02T13:53:29.898110+0000","last_peered":"2024-04-02T13:53:29.898110+0000","last_clean":"2024-04-02T13:53:29.898110+0000","last_became_active":"2024-04-02T13:51:00.238475+0000","last_became_peered":"2024-04-02T13:51:00.238475+0000","last_unstale":"2024-04-02T13:53:29.898110+0000","last_undegraded":"2024-04-02T13:53:29.898110+0000","last_fullsized":"2024-04-02T13:53:29.898110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:41:07.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":364,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135140+0000","last_change":"2024-04-02T13:49:47.847355+0000","last_active":"2024-04-02T13:53:30.135140+0000","last_peered":"2024-04-02T13:53:30.135140+0000","last_clean":"2024-04-02T13:53:30.135140+0000","last_became_active":"2024-04-02T13:49:47.846768+0000","last_became_peered":"2024-04-02T13:49:47.846768+0000","last_unstale":"2024-04-02T13:53:30.135140+0000","last_undegraded":"2024-04-02T13:53:30.135140+0000","last_fullsized":"2024-04-02T13:53:30.135140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:46:21.098780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:46:21.098780+0000","last_clean_scrub_stamp":"2024-04-02T13:46:21.098780+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:03:52.323921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895591+0000","last_change":"2024-04-02T13:51:05.518859+0000","last_active":"2024-04-02T13:53:29.895591+0000","last_peered":"2024-04-02T13:53:29.895591+0000","last_clean":"2024-04-02T13:53:29.895591+0000","last_became_active":"2024-04-02T13:51:05.518364+0000","last_became_peered":"2024-04-02T13:51:05.518364+0000","last_unstale":"2024-04-02T13:53:29.895591+0000","last_undegraded":"2024-04-02T13:53:29.895591+0000","last_fullsized":"2024-04-02T13:53:29.895591+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:10:45.877853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.893663+0000","last_change":"2024-04-02T13:51:01.254795+0000","last_active":"2024-04-02T13:53:29.893663+0000","last_peered":"2024-04-02T13:53:29.893663+0000","last_clean":"2024-04-02T13:53:29.893663+0000","last_became_active":"2024-04-02T13:51:01.254452+0000","last_became_peered":"2024-04-02T13:51:01.254452+0000","last_unstale":"2024-04-02T13:53:29.893663+0000","last_undegraded":"2024-04-02T13:53:29.893663+0000","last_fullsized":"2024-04-02T13:53:29.893663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:13:53.747308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135234+0000","last_change":"2024-04-02T13:51:00.241323+0000","last_active":"2024-04-02T13:53:30.135234+0000","last_peered":"2024-04-02T13:53:30.135234+0000","last_clean":"2024-04-02T13:53:30.135234+0000","last_became_active":"2024-04-02T13:51:00.240989+0000","last_became_peered":"2024-04-02T13:51:00.240989+0000","last_unstale":"2024-04-02T13:53:30.135234+0000","last_undegraded":"2024-04-02T13:53:30.135234+0000","last_fullsized":"2024-04-02T13:53:30.135234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:47:52.601517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174717+0000","last_change":"2024-04-02T13:51:56.665905+0000","last_active":"2024-04-02T13:53:30.174717+0000","last_peered":"2024-04-02T13:53:30.174717+0000","last_clean":"2024-04-02T13:53:30.174717+0000","last_became_active":"2024-04-02T13:51:56.665315+0000","last_became_peered":"2024-04-02T13:51:56.665315+0000","last_unstale":"2024-04-02T13:53:30.174717+0000","last_undegraded":"2024-04-02T13:53:30.174717+0000","last_fullsized":"2024-04-02T13:53:30.174717+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:44:34.548102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997346+0000","last_change":"2024-04-02T13:51:00.241416+0000","last_active":"2024-04-02T13:53:29.997346+0000","last_peered":"2024-04-02T13:53:29.997346+0000","last_clean":"2024-04-02T13:53:29.997346+0000","last_became_active":"2024-04-02T13:51:00.241071+0000","last_became_peered":"2024-04-02T13:51:00.241071+0000","last_unstale":"2024-04-02T13:53:29.997346+0000","last_undegraded":"2024-04-02T13:53:29.997346+0000","last_fullsized":"2024-04-02T13:53:29.997346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:24:16.243737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894214+0000","last_change":"2024-04-02T13:51:01.252156+0000","last_active":"2024-04-02T13:53:29.894214+0000","last_peered":"2024-04-02T13:53:29.894214+0000","last_clean":"2024-04-02T13:53:29.894214+0000","last_became_active":"2024-04-02T13:51:01.251733+0000","last_became_peered":"2024-04-02T13:51:01.251733+0000","last_unstale":"2024-04-02T13:53:29.894214+0000","last_undegraded":"2024-04-02T13:53:29.894214+0000","last_fullsized":"2024-04-02T13:53:29.894214+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:54:24.082643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996784+0000","last_change":"2024-04-02T13:51:05.518367+0000","last_active":"2024-04-02T13:53:29.996784+0000","last_peered":"2024-04-02T13:53:29.996784+0000","last_clean":"2024-04-02T13:53:29.996784+0000","last_became_active":"2024-04-02T13:51:05.517783+0000","last_became_peered":"2024-04-02T13:51:05.517783+0000","last_unstale":"2024-04-02T13:53:29.996784+0000","last_undegraded":"2024-04-02T13:53:29.996784+0000","last_fullsized":"2024-04-02T13:53:29.996784+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:10:53.880909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'48","reported_seq":101,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.107358+0000","last_change":"2024-04-02T13:51:00.248998+0000","last_active":"2024-04-02T13:53:30.107358+0000","last_peered":"2024-04-02T13:53:30.107358+0000","last_clean":"2024-04-02T13:53:30.107358+0000","last_became_active":"2024-04-02T13:51:00.248613+0000","last_became_peered":"2024-04-02T13:51:00.248613+0000","last_unstale":"2024-04-02T13:53:30.107358+0000","last_undegraded":"2024-04-02T13:53:30.107358+0000","last_fullsized":"2024-04-02T13:53:30.107358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:53.948645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997762+0000","last_change":"2024-04-02T13:51:01.257244+0000","last_active":"2024-04-02T13:53:29.997762+0000","last_peered":"2024-04-02T13:53:29.997762+0000","last_clean":"2024-04-02T13:53:29.997762+0000","last_became_active":"2024-04-02T13:51:01.256739+0000","last_became_peered":"2024-04-02T13:51:01.256739+0000","last_unstale":"2024-04-02T13:53:29.997762+0000","last_undegraded":"2024-04-02T13:53:29.997762+0000","last_fullsized":"2024-04-02T13:53:29.997762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:38:18.786553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.134886+0000","last_change":"2024-04-02T13:51:05.356177+0000","last_active":"2024-04-02T13:53:30.134886+0000","last_peered":"2024-04-02T13:53:30.134886+0000","last_clean":"2024-04-02T13:53:30.134886+0000","last_became_active":"2024-04-02T13:51:05.354704+0000","last_became_peered":"2024-04-02T13:51:05.354704+0000","last_unstale":"2024-04-02T13:53:30.134886+0000","last_undegraded":"2024-04-02T13:53:30.134886+0000","last_fullsized":"2024-04-02T13:53:30.134886+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:27:55.761490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898390+0000","last_change":"2024-04-02T13:51:00.245945+0000","last_active":"2024-04-02T13:53:29.898390+0000","last_peered":"2024-04-02T13:53:29.898390+0000","last_clean":"2024-04-02T13:53:29.898390+0000","last_became_active":"2024-04-02T13:51:00.245528+0000","last_became_peered":"2024-04-02T13:51:00.245528+0000","last_unstale":"2024-04-02T13:53:29.898390+0000","last_undegraded":"2024-04-02T13:53:29.898390+0000","last_fullsized":"2024-04-02T13:53:29.898390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:37:49.263614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.175330+0000","last_change":"2024-04-02T13:51:01.260268+0000","last_active":"2024-04-02T13:53:30.175330+0000","last_peered":"2024-04-02T13:53:30.175330+0000","last_clean":"2024-04-02T13:53:30.175330+0000","last_became_active":"2024-04-02T13:51:01.258182+0000","last_became_peered":"2024-04-02T13:51:01.258182+0000","last_unstale":"2024-04-02T13:53:30.175330+0000","last_undegraded":"2024-04-02T13:53:30.175330+0000","last_fullsized":"2024-04-02T13:53:30.175330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:31:31.423988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898255+0000","last_change":"2024-04-02T13:51:05.518597+0000","last_active":"2024-04-02T13:53:29.898255+0000","last_peered":"2024-04-02T13:53:29.898255+0000","last_clean":"2024-04-02T13:53:29.898255+0000","last_became_active":"2024-04-02T13:51:05.518134+0000","last_became_peered":"2024-04-02T13:51:05.518134+0000","last_unstale":"2024-04-02T13:53:29.898255+0000","last_undegraded":"2024-04-02T13:53:29.898255+0000","last_fullsized":"2024-04-02T13:53:29.898255+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:00:38.632939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135401+0000","last_change":"2024-04-02T13:51:00.241680+0000","last_active":"2024-04-02T13:53:30.135401+0000","last_peered":"2024-04-02T13:53:30.135401+0000","last_clean":"2024-04-02T13:53:30.135401+0000","last_became_active":"2024-04-02T13:51:00.241301+0000","last_became_peered":"2024-04-02T13:51:00.241301+0000","last_unstale":"2024-04-02T13:53:30.135401+0000","last_undegraded":"2024-04-02T13:53:30.135401+0000","last_fullsized":"2024-04-02T13:53:30.135401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:12:10.606973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996895+0000","last_change":"2024-04-02T13:51:01.254795+0000","last_active":"2024-04-02T13:53:29.996895+0000","last_peered":"2024-04-02T13:53:29.996895+0000","last_clean":"2024-04-02T13:53:29.996895+0000","last_became_active":"2024-04-02T13:51:01.254451+0000","last_became_peered":"2024-04-02T13:51:01.254451+0000","last_unstale":"2024-04-02T13:53:29.996895+0000","last_undegraded":"2024-04-02T13:53:29.996895+0000","last_fullsized":"2024-04-02T13:53:29.996895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:58:04.202996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.995353+0000","last_change":"2024-04-02T13:51:05.363592+0000","last_active":"2024-04-02T13:53:29.995353+0000","last_peered":"2024-04-02T13:53:29.995353+0000","last_clean":"2024-04-02T13:53:29.995353+0000","last_became_active":"2024-04-02T13:51:05.363211+0000","last_became_peered":"2024-04-02T13:51:05.363211+0000","last_unstale":"2024-04-02T13:53:29.995353+0000","last_undegraded":"2024-04-02T13:53:29.995353+0000","last_fullsized":"2024-04-02T13:53:29.995353+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:31:56.561505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996907+0000","last_change":"2024-04-02T13:51:00.243681+0000","last_active":"2024-04-02T13:53:29.996907+0000","last_peered":"2024-04-02T13:53:29.996907+0000","last_clean":"2024-04-02T13:53:29.996907+0000","last_became_active":"2024-04-02T13:51:00.241615+0000","last_became_peered":"2024-04-02T13:51:00.241615+0000","last_unstale":"2024-04-02T13:53:29.996907+0000","last_undegraded":"2024-04-02T13:53:29.996907+0000","last_fullsized":"2024-04-02T13:53:29.996907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:54:30.112735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174537+0000","last_change":"2024-04-02T13:51:01.252429+0000","last_active":"2024-04-02T13:53:30.174537+0000","last_peered":"2024-04-02T13:53:30.174537+0000","last_clean":"2024-04-02T13:53:30.174537+0000","last_became_active":"2024-04-02T13:51:01.252017+0000","last_became_peered":"2024-04-02T13:51:01.252017+0000","last_unstale":"2024-04-02T13:53:30.174537+0000","last_undegraded":"2024-04-02T13:53:30.174537+0000","last_fullsized":"2024-04-02T13:53:30.174537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:32:02.152578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898141+0000","last_change":"2024-04-02T13:51:05.361297+0000","last_active":"2024-04-02T13:53:29.898141+0000","last_peered":"2024-04-02T13:53:29.898141+0000","last_clean":"2024-04-02T13:53:29.898141+0000","last_became_active":"2024-04-02T13:51:05.360916+0000","last_became_peered":"2024-04-02T13:51:05.360916+0000","last_unstale":"2024-04-02T13:53:29.898141+0000","last_undegraded":"2024-04-02T13:53:29.898141+0000","last_fullsized":"2024-04-02T13:53:29.898141+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:52:48.608587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"93'4","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174766+0000","last_change":"2024-04-02T13:51:56.488312+0000","last_active":"2024-04-02T13:53:30.174766+0000","last_peered":"2024-04-02T13:53:30.174766+0000","last_clean":"2024-04-02T13:53:30.174766+0000","last_became_active":"2024-04-02T13:51:56.483987+0000","last_became_peered":"2024-04-02T13:51:56.483987+0000","last_unstale":"2024-04-02T13:53:30.174766+0000","last_undegraded":"2024-04-02T13:53:30.174766+0000","last_fullsized":"2024-04-02T13:53:30.174766+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:46:27.396997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997510+0000","last_change":"2024-04-02T13:51:01.255557+0000","last_active":"2024-04-02T13:53:29.997510+0000","last_peered":"2024-04-02T13:53:29.997510+0000","last_clean":"2024-04-02T13:53:29.997510+0000","last_became_active":"2024-04-02T13:51:01.255011+0000","last_became_peered":"2024-04-02T13:51:01.255011+0000","last_unstale":"2024-04-02T13:53:29.997510+0000","last_undegraded":"2024-04-02T13:53:29.997510+0000","last_fullsized":"2024-04-02T13:53:29.997510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:01:52.105965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898183+0000","last_change":"2024-04-02T13:51:05.518764+0000","last_active":"2024-04-02T13:53:29.898183+0000","last_peered":"2024-04-02T13:53:29.898183+0000","last_clean":"2024-04-02T13:53:29.898183+0000","last_became_active":"2024-04-02T13:51:05.518088+0000","last_became_peered":"2024-04-02T13:51:05.518088+0000","last_unstale":"2024-04-02T13:53:29.898183+0000","last_undegraded":"2024-04-02T13:53:29.898183+0000","last_fullsized":"2024-04-02T13:53:29.898183+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:03:02.901384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996946+0000","last_change":"2024-04-02T13:51:00.244147+0000","last_active":"2024-04-02T13:53:29.996946+0000","last_peered":"2024-04-02T13:53:29.996946+0000","last_clean":"2024-04-02T13:53:29.996946+0000","last_became_active":"2024-04-02T13:51:00.243763+0000","last_became_peered":"2024-04-02T13:51:00.243763+0000","last_unstale":"2024-04-02T13:53:29.996946+0000","last_undegraded":"2024-04-02T13:53:29.996946+0000","last_fullsized":"2024-04-02T13:53:29.996946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:34:27.738746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135780+0000","last_change":"2024-04-02T13:51:01.268121+0000","last_active":"2024-04-02T13:53:30.135780+0000","last_peered":"2024-04-02T13:53:30.135780+0000","last_clean":"2024-04-02T13:53:30.135780+0000","last_became_active":"2024-04-02T13:51:01.266363+0000","last_became_peered":"2024-04-02T13:51:01.266363+0000","last_unstale":"2024-04-02T13:53:30.135780+0000","last_undegraded":"2024-04-02T13:53:30.135780+0000","last_fullsized":"2024-04-02T13:53:30.135780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:30:50.254916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135846+0000","last_change":"2024-04-02T13:51:05.361285+0000","last_active":"2024-04-02T13:53:30.135846+0000","last_peered":"2024-04-02T13:53:30.135846+0000","last_clean":"2024-04-02T13:53:30.135846+0000","last_became_active":"2024-04-02T13:51:05.360122+0000","last_became_peered":"2024-04-02T13:51:05.360122+0000","last_unstale":"2024-04-02T13:53:30.135846+0000","last_undegraded":"2024-04-02T13:53:30.135846+0000","last_fullsized":"2024-04-02T13:53:30.135846+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:30:59.935538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'7","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135534+0000","last_change":"2024-04-02T13:51:00.247235+0000","last_active":"2024-04-02T13:53:30.135534+0000","last_peered":"2024-04-02T13:53:30.135534+0000","last_clean":"2024-04-02T13:53:30.135534+0000","last_became_active":"2024-04-02T13:51:00.246856+0000","last_became_peered":"2024-04-02T13:51:00.246856+0000","last_unstale":"2024-04-02T13:53:30.135534+0000","last_undegraded":"2024-04-02T13:53:30.135534+0000","last_fullsized":"2024-04-02T13:53:30.135534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:17:51.875952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136471+0000","last_change":"2024-04-02T13:51:01.263866+0000","last_active":"2024-04-02T13:53:30.136471+0000","last_peered":"2024-04-02T13:53:30.136471+0000","last_clean":"2024-04-02T13:53:30.136471+0000","last_became_active":"2024-04-02T13:51:01.263564+0000","last_became_peered":"2024-04-02T13:51:01.263564+0000","last_unstale":"2024-04-02T13:53:30.136471+0000","last_undegraded":"2024-04-02T13:53:30.136471+0000","last_fullsized":"2024-04-02T13:53:30.136471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:59:33.663277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174890+0000","last_change":"2024-04-02T13:51:05.363079+0000","last_active":"2024-04-02T13:53:30.174890+0000","last_peered":"2024-04-02T13:53:30.174890+0000","last_clean":"2024-04-02T13:53:30.174890+0000","last_became_active":"2024-04-02T13:51:05.362631+0000","last_became_peered":"2024-04-02T13:51:05.362631+0000","last_unstale":"2024-04-02T13:53:30.174890+0000","last_undegraded":"2024-04-02T13:53:30.174890+0000","last_fullsized":"2024-04-02T13:53:30.174890+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:23.656269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"93'41","reported_seq":79,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.102154+0000","last_change":"2024-04-02T13:51:00.242820+0000","last_active":"2024-04-02T13:53:30.102154+0000","last_peered":"2024-04-02T13:53:30.102154+0000","last_clean":"2024-04-02T13:53:30.102154+0000","last_became_active":"2024-04-02T13:51:00.242262+0000","last_became_peered":"2024-04-02T13:51:00.242262+0000","last_unstale":"2024-04-02T13:53:30.102154+0000","last_undegraded":"2024-04-02T13:53:30.102154+0000","last_fullsized":"2024-04-02T13:53:30.102154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:12:22.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136066+0000","last_change":"2024-04-02T13:51:01.266548+0000","last_active":"2024-04-02T13:53:30.136066+0000","last_peered":"2024-04-02T13:53:30.136066+0000","last_clean":"2024-04-02T13:53:30.136066+0000","last_became_active":"2024-04-02T13:51:01.259003+0000","last_became_peered":"2024-04-02T13:51:01.259003+0000","last_unstale":"2024-04-02T13:53:30.136066+0000","last_undegraded":"2024-04-02T13:53:30.136066+0000","last_fullsized":"2024-04-02T13:53:30.136066+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:38:53.692231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894871+0000","last_change":"2024-04-02T13:51:05.359994+0000","last_active":"2024-04-02T13:53:29.894871+0000","last_peered":"2024-04-02T13:53:29.894871+0000","last_clean":"2024-04-02T13:53:29.894871+0000","last_became_active":"2024-04-02T13:51:05.359438+0000","last_became_peered":"2024-04-02T13:51:05.359438+0000","last_unstale":"2024-04-02T13:53:29.894871+0000","last_undegraded":"2024-04-02T13:53:29.894871+0000","last_fullsized":"2024-04-02T13:53:29.894871+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:09:43.929446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894377+0000","last_change":"2024-04-02T13:51:00.237889+0000","last_active":"2024-04-02T13:53:29.894377+0000","last_peered":"2024-04-02T13:53:29.894377+0000","last_clean":"2024-04-02T13:53:29.894377+0000","last_became_active":"2024-04-02T13:51:00.237428+0000","last_became_peered":"2024-04-02T13:51:00.237428+0000","last_unstale":"2024-04-02T13:53:29.894377+0000","last_undegraded":"2024-04-02T13:53:29.894377+0000","last_fullsized":"2024-04-02T13:53:29.894377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:21:52.483656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136447+0000","last_change":"2024-04-02T13:51:01.262093+0000","last_active":"2024-04-02T13:53:30.136447+0000","last_peered":"2024-04-02T13:53:30.136447+0000","last_clean":"2024-04-02T13:53:30.136447+0000","last_became_active":"2024-04-02T13:51:01.258357+0000","last_became_peered":"2024-04-02T13:51:01.258357+0000","last_unstale":"2024-04-02T13:53:30.136447+0000","last_undegraded":"2024-04-02T13:53:30.136447+0000","last_fullsized":"2024-04-02T13:53:30.136447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:00:06.317307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174608+0000","last_change":"2024-04-02T13:51:56.483560+0000","last_active":"2024-04-02T13:53:30.174608+0000","last_peered":"2024-04-02T13:53:30.174608+0000","last_clean":"2024-04-02T13:53:30.174608+0000","last_became_active":"2024-04-02T13:51:56.483090+0000","last_became_peered":"2024-04-02T13:51:56.483090+0000","last_unstale":"2024-04-02T13:53:30.174608+0000","last_undegraded":"2024-04-02T13:53:30.174608+0000","last_fullsized":"2024-04-02T13:53:30.174608+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:09:35.640492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135057+0000","last_change":"2024-04-02T13:51:56.663817+0000","last_active":"2024-04-02T13:53:30.135057+0000","last_peered":"2024-04-02T13:53:30.135057+0000","last_clean":"2024-04-02T13:53:30.135057+0000","last_became_active":"2024-04-02T13:51:56.663473+0000","last_became_peered":"2024-04-02T13:51:56.663473+0000","last_unstale":"2024-04-02T13:53:30.135057+0000","last_undegraded":"2024-04-02T13:53:30.135057+0000","last_fullsized":"2024-04-02T13:53:30.135057+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:26:15.566030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135112+0000","last_change":"2024-04-02T13:51:01.255563+0000","last_active":"2024-04-02T13:53:30.135112+0000","last_peered":"2024-04-02T13:53:30.135112+0000","last_clean":"2024-04-02T13:53:30.135112+0000","last_became_active":"2024-04-02T13:51:01.255228+0000","last_became_peered":"2024-04-02T13:51:01.255228+0000","last_unstale":"2024-04-02T13:53:30.135112+0000","last_undegraded":"2024-04-02T13:53:30.135112+0000","last_fullsized":"2024-04-02T13:53:30.135112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:48:28.200776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.175260+0000","last_change":"2024-04-02T13:51:01.258153+0000","last_active":"2024-04-02T13:53:30.175260+0000","last_peered":"2024-04-02T13:53:30.175260+0000","last_clean":"2024-04-02T13:53:30.175260+0000","last_became_active":"2024-04-02T13:51:01.257656+0000","last_became_peered":"2024-04-02T13:51:01.257656+0000","last_unstale":"2024-04-02T13:53:30.175260+0000","last_undegraded":"2024-04-02T13:53:30.175260+0000","last_fullsized":"2024-04-02T13:53:30.175260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:37.887427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894267+0000","last_change":"2024-04-02T13:51:01.254675+0000","last_active":"2024-04-02T13:53:29.894267+0000","last_peered":"2024-04-02T13:53:29.894267+0000","last_clean":"2024-04-02T13:53:29.894267+0000","last_became_active":"2024-04-02T13:51:01.254336+0000","last_became_peered":"2024-04-02T13:51:01.254336+0000","last_unstale":"2024-04-02T13:53:29.894267+0000","last_undegraded":"2024-04-02T13:53:29.894267+0000","last_fullsized":"2024-04-02T13:53:29.894267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:36.171961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174947+0000","last_change":"2024-04-02T13:51:01.259457+0000","last_active":"2024-04-02T13:53:30.174947+0000","last_peered":"2024-04-02T13:53:30.174947+0000","last_clean":"2024-04-02T13:53:30.174947+0000","last_became_active":"2024-04-02T13:51:01.259099+0000","last_became_peered":"2024-04-02T13:51:01.259099+0000","last_unstale":"2024-04-02T13:53:30.174947+0000","last_undegraded":"2024-04-02T13:53:30.174947+0000","last_fullsized":"2024-04-02T13:53:30.174947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:13:06.594132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135619+0000","last_change":"2024-04-02T13:51:01.248177+0000","last_active":"2024-04-02T13:53:30.135619+0000","last_peered":"2024-04-02T13:53:30.135619+0000","last_clean":"2024-04-02T13:53:30.135619+0000","last_became_active":"2024-04-02T13:51:01.247843+0000","last_became_peered":"2024-04-02T13:51:01.247843+0000","last_unstale":"2024-04-02T13:53:30.135619+0000","last_undegraded":"2024-04-02T13:53:30.135619+0000","last_fullsized":"2024-04-02T13:53:30.135619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:38:05.805382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997866+0000","last_change":"2024-04-02T13:51:01.254822+0000","last_active":"2024-04-02T13:53:29.997866+0000","last_peered":"2024-04-02T13:53:29.997866+0000","last_clean":"2024-04-02T13:53:29.997866+0000","last_became_active":"2024-04-02T13:51:01.254451+0000","last_became_peered":"2024-04-02T13:51:01.254451+0000","last_unstale":"2024-04-02T13:53:29.997866+0000","last_undegraded":"2024-04-02T13:53:29.997866+0000","last_fullsized":"2024-04-02T13:53:29.997866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:15:35.790371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895040+0000","last_change":"2024-04-02T13:51:01.265467+0000","last_active":"2024-04-02T13:53:29.895040+0000","last_peered":"2024-04-02T13:53:29.895040+0000","last_clean":"2024-04-02T13:53:29.895040+0000","last_became_active":"2024-04-02T13:51:01.265072+0000","last_became_peered":"2024-04-02T13:51:01.265072+0000","last_unstale":"2024-04-02T13:53:29.895040+0000","last_undegraded":"2024-04-02T13:53:29.895040+0000","last_fullsized":"2024-04-02T13:53:29.895040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:12:07.702222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996951+0000","last_change":"2024-04-02T13:51:01.252844+0000","last_active":"2024-04-02T13:53:29.996951+0000","last_peered":"2024-04-02T13:53:29.996951+0000","last_clean":"2024-04-02T13:53:29.996951+0000","last_became_active":"2024-04-02T13:51:01.252515+0000","last_became_peered":"2024-04-02T13:51:01.252515+0000","last_unstale":"2024-04-02T13:53:29.996951+0000","last_undegraded":"2024-04-02T13:53:29.996951+0000","last_fullsized":"2024-04-02T13:53:29.996951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:20:16.216418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136142+0000","last_change":"2024-04-02T13:51:01.260141+0000","last_active":"2024-04-02T13:53:30.136142+0000","last_peered":"2024-04-02T13:53:30.136142+0000","last_clean":"2024-04-02T13:53:30.136142+0000","last_became_active":"2024-04-02T13:51:01.259530+0000","last_became_peered":"2024-04-02T13:51:01.259530+0000","last_unstale":"2024-04-02T13:53:30.136142+0000","last_undegraded":"2024-04-02T13:53:30.136142+0000","last_fullsized":"2024-04-02T13:53:30.136142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:00:03.579414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894065+0000","last_change":"2024-04-02T13:51:01.259981+0000","last_active":"2024-04-02T13:53:29.894065+0000","last_peered":"2024-04-02T13:53:29.894065+0000","last_clean":"2024-04-02T13:53:29.894065+0000","last_became_active":"2024-04-02T13:51:01.259647+0000","last_became_peered":"2024-04-02T13:51:01.259647+0000","last_unstale":"2024-04-02T13:53:29.894065+0000","last_undegraded":"2024-04-02T13:53:29.894065+0000","last_fullsized":"2024-04-02T13:53:29.894065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:10:22.753767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174911+0000","last_change":"2024-04-02T13:51:01.255342+0000","last_active":"2024-04-02T13:53:30.174911+0000","last_peered":"2024-04-02T13:53:30.174911+0000","last_clean":"2024-04-02T13:53:30.174911+0000","last_became_active":"2024-04-02T13:51:01.254679+0000","last_became_peered":"2024-04-02T13:51:01.254679+0000","last_unstale":"2024-04-02T13:53:30.174911+0000","last_undegraded":"2024-04-02T13:53:30.174911+0000","last_fullsized":"2024-04-02T13:53:30.174911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:25:12.198270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601988,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1355,"num_read_kb":796,"num_write":370,"num_write_kb":1468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1805964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":300,"ondisk_log_size":300,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351019,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1069056,"data_stored":517607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612656,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28804,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716348,"statfs":{"total":95995035648,"available":95965540352,"internally_reserved":0,"allocated":1826816,"data_stored":1232765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776181,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28468,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716684,"statfs":{"total":95995035648,"available":95965884416,"internally_reserved":0,"allocated":1482752,"data_stored":914249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037819,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36780,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708372,"statfs":{"total":95995035648,"available":95957372928,"internally_reserved":0,"allocated":1474560,"data_stored":910531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332160,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36980,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708172,"statfs":{"total":95995035648,"available":95957168128,"internally_reserved":0,"allocated":1679360,"data_stored":1114594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495684,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40536,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704616,"statfs":{"total":95995035648,"available":95953526784,"internally_reserved":0,"allocated":1060864,"data_stored":512832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1134592,"data_stored":582907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1085440,"data_stored":514429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1126400,"data_stored":567237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":502341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920847,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1069056,"data_stored":516515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757322,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28672,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716480,"statfs":{"total":95995035648,"available":95965675520,"internally_reserved":0,"allocated":1691648,"data_stored":1109515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":72263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":202381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T13:53:35.042 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T13:53:35.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:34 smithi088 ceph-mon[36528]: Updating smithi052:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:53:35.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:34 smithi088 ceph-mon[36528]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:53:35.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:34 smithi088 ceph-mon[36528]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:53:35.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:34 smithi088 ceph-mon[36528]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:53:35.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:34 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/198583162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:53:35.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:34 smithi088 ceph-mon[36528]: Updating smithi052:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:53:35.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:34 smithi088 ceph-mon[36528]: Updating smithi189:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:53:35.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:34 smithi088 ceph-mon[36528]: Updating smithi088:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:53:35.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:34 smithi088 ceph-mon[36528]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-02T13:53:35.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:34 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:35.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:34 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:35.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:34 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3209286520' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:53:35.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:34 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:35.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:34 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:35.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:34 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:35.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:34 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:35.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:34 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:35.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:34 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:53:35.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:34 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:53:35.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:34 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:53:35.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:34 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:35.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:34 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2393661751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:53:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.176+0000 7fb9ee42a640 1 -- 172.21.15.52:0/470054494 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9e8106a40 msgr2=0x7fb9e8106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.176+0000 7fb9ee42a640 1 --2- 172.21.15.52:0/470054494 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9e8106a40 0x7fb9e8106e20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8009930 tx=0x7fb9d802bc20 comp rx=0 tx=0).stop 2024-04-02T13:53:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.176+0000 7fb9ee42a640 1 -- 172.21.15.52:0/470054494 shutdown_connections 2024-04-02T13:53:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.176+0000 7fb9ee42a640 1 --2- 172.21.15.52:0/470054494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9e8101440 0x7fb9e810d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.176+0000 7fb9ee42a640 1 --2- 172.21.15.52:0/470054494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9e8100a80 0x7fb9e8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.176+0000 7fb9ee42a640 1 --2- 172.21.15.52:0/470054494 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9e8106a40 0x7fb9e8106e20 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.176+0000 7fb9ee42a640 1 -- 172.21.15.52:0/470054494 >> 172.21.15.52:0/470054494 conn(0x7fb9e8077b60 msgr2=0x7fb9e8077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.176+0000 7fb9ee42a640 1 -- 172.21.15.52:0/470054494 shutdown_connections 2024-04-02T13:53:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.176+0000 7fb9ee42a640 1 -- 172.21.15.52:0/470054494 wait complete. 2024-04-02T13:53:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.177+0000 7fb9ee42a640 1 Processor -- start 2024-04-02T13:53:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.177+0000 7fb9ee42a640 1 -- start start 2024-04-02T13:53:35.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.177+0000 7fb9ee42a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9e8100a80 0x7fb9e81972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:35.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.177+0000 7fb9ee42a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9e8101440 0x7fb9e8197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.177+0000 7fb9ee42a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9e8106a40 0x7fb9e819bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.177+0000 7fb9ee42a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e8110e20 con 0x7fb9e8101440 2024-04-02T13:53:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.177+0000 7fb9ee42a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e8110ca0 con 0x7fb9e8106a40 2024-04-02T13:53:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.177+0000 7fb9ee42a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e8110fa0 con 0x7fb9e8100a80 2024-04-02T13:53:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.177+0000 7fb9e77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9e8101440 0x7fb9e8197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.177+0000 7fb9e77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9e8101440 0x7fb9e8197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58426/0 (socket says 172.21.15.52:58426) 2024-04-02T13:53:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.177+0000 7fb9e77fe640 1 -- 172.21.15.52:0/3361293223 learned_addr learned my addr 172.21.15.52:0/3361293223 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.177+0000 7fb9ec9c7640 1 --2- 172.21.15.52:0/3361293223 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9e8106a40 0x7fb9e819bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.177+0000 7fb9e7fff640 1 --2- 172.21.15.52:0/3361293223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9e8100a80 0x7fb9e81972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.178+0000 7fb9e77fe640 1 -- 172.21.15.52:0/3361293223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9e8100a80 msgr2=0x7fb9e81972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.178+0000 7fb9e77fe640 1 --2- 172.21.15.52:0/3361293223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9e8100a80 0x7fb9e81972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.178+0000 7fb9e77fe640 1 -- 172.21.15.52:0/3361293223 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9e8106a40 msgr2=0x7fb9e819bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:35.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.178+0000 7fb9e77fe640 1 --2- 172.21.15.52:0/3361293223 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9e8106a40 0x7fb9e819bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.178+0000 7fb9e77fe640 1 -- 172.21.15.52:0/3361293223 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e819c2b0 con 0x7fb9e8101440 2024-04-02T13:53:35.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.178+0000 7fb9e77fe640 1 --2- 172.21.15.52:0/3361293223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9e8101440 0x7fb9e8197800 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc00c920 tx=0x7fb9dc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:35.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.178+0000 7fb9e57fa640 1 -- 172.21.15.52:0/3361293223 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc007c70 con 0x7fb9e8101440 2024-04-02T13:53:35.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.178+0000 7fb9e57fa640 1 -- 172.21.15.52:0/3361293223 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9dc007e10 con 0x7fb9e8101440 2024-04-02T13:53:35.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.178+0000 7fb9e57fa640 1 -- 172.21.15.52:0/3361293223 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc018890 con 0x7fb9e8101440 2024-04-02T13:53:35.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.178+0000 7fb9ee42a640 1 -- 172.21.15.52:0/3361293223 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9e819c5a0 con 0x7fb9e8101440 2024-04-02T13:53:35.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.178+0000 7fb9ee42a640 1 -- 172.21.15.52:0/3361293223 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9e81a3e30 con 0x7fb9e8101440 2024-04-02T13:53:35.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.179+0000 7fb9ee42a640 1 -- 172.21.15.52:0/3361293223 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9e8068d40 con 0x7fb9e8101440 2024-04-02T13:53:35.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.180+0000 7fb9e57fa640 1 -- 172.21.15.52:0/3361293223 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fb9dc018a70 con 0x7fb9e8101440 2024-04-02T13:53:35.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.180+0000 7fb9e57fa640 1 --2- 172.21.15.52:0/3361293223 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb9d00a3ec0 0x7fb9d00a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:35.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.181+0000 7fb9e7fff640 1 --2- 172.21.15.52:0/3361293223 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb9d00a3ec0 0x7fb9d00a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:35.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.181+0000 7fb9e57fa640 1 -- 172.21.15.52:0/3361293223 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb9dc0d0ea0 con 0x7fb9e8101440 2024-04-02T13:53:35.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.181+0000 7fb9e7fff640 1 --2- 172.21.15.52:0/3361293223 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb9d00a3ec0 0x7fb9d00a6380 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8006010 tx=0x7fb9d8000bc0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:35.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.185+0000 7fb9e57fa640 1 -- 172.21.15.52:0/3361293223 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb9dc014030 con 0x7fb9e8101440 2024-04-02T13:53:35.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:34 smithi189 ceph-mon[36683]: Updating smithi052:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:53:35.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:34 smithi189 ceph-mon[36683]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:53:35.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:34 smithi189 ceph-mon[36683]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:53:35.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:34 smithi189 ceph-mon[36683]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:53:35.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:34 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/198583162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:53:35.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:34 smithi189 ceph-mon[36683]: Updating smithi052:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:53:35.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:34 smithi189 ceph-mon[36683]: Updating smithi189:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:53:35.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:34 smithi189 ceph-mon[36683]: Updating smithi088:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:53:35.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:34 smithi189 ceph-mon[36683]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-02T13:53:35.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:34 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:35.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:34 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:35.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:34 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3209286520' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:53:35.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:34 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:35.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:34 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:35.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:34 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:35.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:34 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:35.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:34 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:35.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:34 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:53:35.290 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:34 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:53:35.291 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:34 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:53:35.291 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:34 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:53:35.291 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:34 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2393661751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:53:35.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.369+0000 7fb9ee42a640 1 -- 172.21.15.52:0/3361293223 --> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb9e8061f40 con 0x7fb9d00a3ec0 2024-04-02T13:53:35.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.378+0000 7fb9e57fa640 1 -- 172.21.15.52:0/3361293223 <== mgr.25160 v2:172.21.15.52:6800/1019901931 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339431 (secure 0 0 0) 0x7fb9e8061f40 con 0x7fb9d00a3ec0 2024-04-02T13:53:35.385 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:53:35.387 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-04-02T13:53:35.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.382+0000 7fb9ee42a640 1 -- 172.21.15.52:0/3361293223 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb9d00a3ec0 msgr2=0x7fb9d00a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:35.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.382+0000 7fb9ee42a640 1 --2- 172.21.15.52:0/3361293223 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb9d00a3ec0 0x7fb9d00a6380 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8006010 tx=0x7fb9d8000bc0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.382+0000 7fb9ee42a640 1 -- 172.21.15.52:0/3361293223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9e8101440 msgr2=0x7fb9e8197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:35.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.382+0000 7fb9ee42a640 1 --2- 172.21.15.52:0/3361293223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9e8101440 0x7fb9e8197800 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc00c920 tx=0x7fb9dc00cde0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.382+0000 7fb9ee42a640 1 -- 172.21.15.52:0/3361293223 shutdown_connections 2024-04-02T13:53:35.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.382+0000 7fb9ee42a640 1 --2- 172.21.15.52:0/3361293223 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb9d00a3ec0 0x7fb9d00a6380 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.382+0000 7fb9ee42a640 1 --2- 172.21.15.52:0/3361293223 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9e8106a40 0x7fb9e819bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.382+0000 7fb9ee42a640 1 --2- 172.21.15.52:0/3361293223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9e8101440 0x7fb9e8197800 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.382+0000 7fb9ee42a640 1 --2- 172.21.15.52:0/3361293223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9e8100a80 0x7fb9e81972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.382+0000 7fb9ee42a640 1 -- 172.21.15.52:0/3361293223 >> 172.21.15.52:0/3361293223 conn(0x7fb9e8077b60 msgr2=0x7fb9e8103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:35.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.382+0000 7fb9ee42a640 1 -- 172.21.15.52:0/3361293223 shutdown_connections 2024-04-02T13:53:35.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.382+0000 7fb9ee42a640 1 -- 172.21.15.52:0/3361293223 wait complete. 2024-04-02T13:53:35.396 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-02T13:53:33.911035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192590,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1491,"num_read_kb":915,"num_write":553,"num_write_kb":4408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":939,"ondisk_log_size":939,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":414500,"kb_used_data":15388,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124527324,"statfs":{"total":1151940427776,"available":1151515979776,"internally_reserved":0,"allocated":15757312,"data_stored":8995522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":408532282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.991083"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.175053+0000","last_change":"2024-04-02T13:51:01.253563+0000","last_active":"2024-04-02T13:53:30.175053+0000","last_peered":"2024-04-02T13:53:30.175053+0000","last_clean":"2024-04-02T13:53:30.175053+0000","last_became_active":"2024-04-02T13:51:01.253185+0000","last_became_peered":"2024-04-02T13:51:01.253185+0000","last_unstale":"2024-04-02T13:53:30.175053+0000","last_undegraded":"2024-04-02T13:53:30.175053+0000","last_fullsized":"2024-04-02T13:53:30.175053+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:51:05.301880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.175203+0000","last_change":"2024-04-02T13:51:01.256406+0000","last_active":"2024-04-02T13:53:30.175203+0000","last_peered":"2024-04-02T13:53:30.175203+0000","last_clean":"2024-04-02T13:53:30.175203+0000","last_became_active":"2024-04-02T13:51:01.256068+0000","last_became_peered":"2024-04-02T13:51:01.256068+0000","last_unstale":"2024-04-02T13:53:30.175203+0000","last_undegraded":"2024-04-02T13:53:30.175203+0000","last_fullsized":"2024-04-02T13:53:30.175203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:56:51.184932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174787+0000","last_change":"2024-04-02T13:51:01.265925+0000","last_active":"2024-04-02T13:53:30.174787+0000","last_peered":"2024-04-02T13:53:30.174787+0000","last_clean":"2024-04-02T13:53:30.174787+0000","last_became_active":"2024-04-02T13:51:01.265587+0000","last_became_peered":"2024-04-02T13:51:01.265587+0000","last_unstale":"2024-04-02T13:53:30.174787+0000","last_undegraded":"2024-04-02T13:53:30.174787+0000","last_fullsized":"2024-04-02T13:53:30.174787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:40:59.426886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.994837+0000","last_change":"2024-04-02T13:51:56.482512+0000","last_active":"2024-04-02T13:53:29.994837+0000","last_peered":"2024-04-02T13:53:29.994837+0000","last_clean":"2024-04-02T13:53:29.994837+0000","last_became_active":"2024-04-02T13:51:56.481439+0000","last_became_peered":"2024-04-02T13:51:56.481439+0000","last_unstale":"2024-04-02T13:53:29.994837+0000","last_undegraded":"2024-04-02T13:53:29.994837+0000","last_fullsized":"2024-04-02T13:53:29.994837+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:23:16.969641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997088+0000","last_change":"2024-04-02T13:51:01.250934+0000","last_active":"2024-04-02T13:53:29.997088+0000","last_peered":"2024-04-02T13:53:29.997088+0000","last_clean":"2024-04-02T13:53:29.997088+0000","last_became_active":"2024-04-02T13:51:01.250546+0000","last_became_peered":"2024-04-02T13:51:01.250546+0000","last_unstale":"2024-04-02T13:53:29.997088+0000","last_undegraded":"2024-04-02T13:53:29.997088+0000","last_fullsized":"2024-04-02T13:53:29.997088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:34:17.839207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.003203+0000","last_change":"2024-04-02T13:51:01.256160+0000","last_active":"2024-04-02T13:53:30.003203+0000","last_peered":"2024-04-02T13:53:30.003203+0000","last_clean":"2024-04-02T13:53:30.003203+0000","last_became_active":"2024-04-02T13:51:01.255776+0000","last_became_peered":"2024-04-02T13:51:01.255776+0000","last_unstale":"2024-04-02T13:53:30.003203+0000","last_undegraded":"2024-04-02T13:53:30.003203+0000","last_fullsized":"2024-04-02T13:53:30.003203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:53:16.724662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135704+0000","last_change":"2024-04-02T13:51:01.263049+0000","last_active":"2024-04-02T13:53:30.135704+0000","last_peered":"2024-04-02T13:53:30.135704+0000","last_clean":"2024-04-02T13:53:30.135704+0000","last_became_active":"2024-04-02T13:51:01.262660+0000","last_became_peered":"2024-04-02T13:51:01.262660+0000","last_unstale":"2024-04-02T13:53:30.135704+0000","last_undegraded":"2024-04-02T13:53:30.135704+0000","last_fullsized":"2024-04-02T13:53:30.135704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:15:32.069188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996829+0000","last_change":"2024-04-02T13:51:01.263064+0000","last_active":"2024-04-02T13:53:29.996829+0000","last_peered":"2024-04-02T13:53:29.996829+0000","last_clean":"2024-04-02T13:53:29.996829+0000","last_became_active":"2024-04-02T13:51:01.262706+0000","last_became_peered":"2024-04-02T13:51:01.262706+0000","last_unstale":"2024-04-02T13:53:29.996829+0000","last_undegraded":"2024-04-02T13:53:29.996829+0000","last_fullsized":"2024-04-02T13:53:29.996829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:46.680723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136166+0000","last_change":"2024-04-02T13:51:01.257347+0000","last_active":"2024-04-02T13:53:30.136166+0000","last_peered":"2024-04-02T13:53:30.136166+0000","last_clean":"2024-04-02T13:53:30.136166+0000","last_became_active":"2024-04-02T13:51:01.257001+0000","last_became_peered":"2024-04-02T13:51:01.257001+0000","last_unstale":"2024-04-02T13:53:30.136166+0000","last_undegraded":"2024-04-02T13:53:30.136166+0000","last_fullsized":"2024-04-02T13:53:30.136166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:31:22.628436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.003671+0000","last_change":"2024-04-02T13:51:01.260533+0000","last_active":"2024-04-02T13:53:30.003671+0000","last_peered":"2024-04-02T13:53:30.003671+0000","last_clean":"2024-04-02T13:53:30.003671+0000","last_became_active":"2024-04-02T13:51:01.260203+0000","last_became_peered":"2024-04-02T13:51:01.260203+0000","last_unstale":"2024-04-02T13:53:30.003671+0000","last_undegraded":"2024-04-02T13:53:30.003671+0000","last_fullsized":"2024-04-02T13:53:30.003671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:00:06.008243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136480+0000","last_change":"2024-04-02T13:51:01.267733+0000","last_active":"2024-04-02T13:53:30.136480+0000","last_peered":"2024-04-02T13:53:30.136480+0000","last_clean":"2024-04-02T13:53:30.136480+0000","last_became_active":"2024-04-02T13:51:01.266642+0000","last_became_peered":"2024-04-02T13:51:01.266642+0000","last_unstale":"2024-04-02T13:53:30.136480+0000","last_undegraded":"2024-04-02T13:53:30.136480+0000","last_fullsized":"2024-04-02T13:53:30.136480+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:39:25.696112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898481+0000","last_change":"2024-04-02T13:51:01.255933+0000","last_active":"2024-04-02T13:53:29.898481+0000","last_peered":"2024-04-02T13:53:29.898481+0000","last_clean":"2024-04-02T13:53:29.898481+0000","last_became_active":"2024-04-02T13:51:01.255558+0000","last_became_peered":"2024-04-02T13:51:01.255558+0000","last_unstale":"2024-04-02T13:53:29.898481+0000","last_undegraded":"2024-04-02T13:53:29.898481+0000","last_fullsized":"2024-04-02T13:53:29.898481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:13:11.537415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894118+0000","last_change":"2024-04-02T13:51:01.256242+0000","last_active":"2024-04-02T13:53:29.894118+0000","last_peered":"2024-04-02T13:53:29.894118+0000","last_clean":"2024-04-02T13:53:29.894118+0000","last_became_active":"2024-04-02T13:51:01.255830+0000","last_became_peered":"2024-04-02T13:51:01.255830+0000","last_unstale":"2024-04-02T13:53:29.894118+0000","last_undegraded":"2024-04-02T13:53:29.894118+0000","last_fullsized":"2024-04-02T13:53:29.894118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:39.130894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174674+0000","last_change":"2024-04-02T13:51:01.259738+0000","last_active":"2024-04-02T13:53:30.174674+0000","last_peered":"2024-04-02T13:53:30.174674+0000","last_clean":"2024-04-02T13:53:30.174674+0000","last_became_active":"2024-04-02T13:51:01.258630+0000","last_became_peered":"2024-04-02T13:51:01.258630+0000","last_unstale":"2024-04-02T13:53:30.174674+0000","last_undegraded":"2024-04-02T13:53:30.174674+0000","last_fullsized":"2024-04-02T13:53:30.174674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:12:05.401171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.175018+0000","last_change":"2024-04-02T13:51:01.257684+0000","last_active":"2024-04-02T13:53:30.175018+0000","last_peered":"2024-04-02T13:53:30.175018+0000","last_clean":"2024-04-02T13:53:30.175018+0000","last_became_active":"2024-04-02T13:51:01.257267+0000","last_became_peered":"2024-04-02T13:51:01.257267+0000","last_unstale":"2024-04-02T13:53:30.175018+0000","last_undegraded":"2024-04-02T13:53:30.175018+0000","last_fullsized":"2024-04-02T13:53:30.175018+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:43:45.728233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135320+0000","last_change":"2024-04-02T13:51:01.261720+0000","last_active":"2024-04-02T13:53:30.135320+0000","last_peered":"2024-04-02T13:53:30.135320+0000","last_clean":"2024-04-02T13:53:30.135320+0000","last_became_active":"2024-04-02T13:51:01.261407+0000","last_became_peered":"2024-04-02T13:51:01.261407+0000","last_unstale":"2024-04-02T13:53:30.135320+0000","last_undegraded":"2024-04-02T13:53:30.135320+0000","last_fullsized":"2024-04-02T13:53:30.135320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:54:58.807606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174875+0000","last_change":"2024-04-02T13:51:01.254488+0000","last_active":"2024-04-02T13:53:30.174875+0000","last_peered":"2024-04-02T13:53:30.174875+0000","last_clean":"2024-04-02T13:53:30.174875+0000","last_became_active":"2024-04-02T13:51:01.254139+0000","last_became_peered":"2024-04-02T13:51:01.254139+0000","last_unstale":"2024-04-02T13:53:30.174875+0000","last_undegraded":"2024-04-02T13:53:30.174875+0000","last_fullsized":"2024-04-02T13:53:30.174875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:02:41.075727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.995012+0000","last_change":"2024-04-02T13:51:01.256107+0000","last_active":"2024-04-02T13:53:29.995012+0000","last_peered":"2024-04-02T13:53:29.995012+0000","last_clean":"2024-04-02T13:53:29.995012+0000","last_became_active":"2024-04-02T13:51:01.255754+0000","last_became_peered":"2024-04-02T13:51:01.255754+0000","last_unstale":"2024-04-02T13:53:29.995012+0000","last_undegraded":"2024-04-02T13:53:29.995012+0000","last_fullsized":"2024-04-02T13:53:29.995012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:36:59.793519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135517+0000","last_change":"2024-04-02T13:51:01.247550+0000","last_active":"2024-04-02T13:53:30.135517+0000","last_peered":"2024-04-02T13:53:30.135517+0000","last_clean":"2024-04-02T13:53:30.135517+0000","last_became_active":"2024-04-02T13:51:01.247236+0000","last_became_peered":"2024-04-02T13:51:01.247236+0000","last_unstale":"2024-04-02T13:53:30.135517+0000","last_undegraded":"2024-04-02T13:53:30.135517+0000","last_fullsized":"2024-04-02T13:53:30.135517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:16:02.940700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135216+0000","last_change":"2024-04-02T13:51:56.663779+0000","last_active":"2024-04-02T13:53:30.135216+0000","last_peered":"2024-04-02T13:53:30.135216+0000","last_clean":"2024-04-02T13:53:30.135216+0000","last_became_active":"2024-04-02T13:51:56.663327+0000","last_became_peered":"2024-04-02T13:51:56.663327+0000","last_unstale":"2024-04-02T13:53:30.135216+0000","last_undegraded":"2024-04-02T13:53:30.135216+0000","last_fullsized":"2024-04-02T13:53:30.135216+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:35:34.186278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"93'14","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.137322+0000","last_change":"2024-04-02T13:51:00.243244+0000","last_active":"2024-04-02T13:53:30.137322+0000","last_peered":"2024-04-02T13:53:30.137322+0000","last_clean":"2024-04-02T13:53:30.137322+0000","last_became_active":"2024-04-02T13:51:00.242892+0000","last_became_peered":"2024-04-02T13:51:00.242892+0000","last_unstale":"2024-04-02T13:53:30.137322+0000","last_undegraded":"2024-04-02T13:53:30.137322+0000","last_fullsized":"2024-04-02T13:53:30.137322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:49:23.338215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":123,"num_read_kb":123,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997790+0000","last_change":"2024-04-02T13:51:01.264640+0000","last_active":"2024-04-02T13:53:29.997790+0000","last_peered":"2024-04-02T13:53:29.997790+0000","last_clean":"2024-04-02T13:53:29.997790+0000","last_became_active":"2024-04-02T13:51:01.264161+0000","last_became_peered":"2024-04-02T13:51:01.264161+0000","last_unstale":"2024-04-02T13:53:29.997790+0000","last_undegraded":"2024-04-02T13:53:29.997790+0000","last_fullsized":"2024-04-02T13:53:29.997790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:30:47.245647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898007+0000","last_change":"2024-04-02T13:51:05.362888+0000","last_active":"2024-04-02T13:53:29.898007+0000","last_peered":"2024-04-02T13:53:29.898007+0000","last_clean":"2024-04-02T13:53:29.898007+0000","last_became_active":"2024-04-02T13:51:05.362478+0000","last_became_peered":"2024-04-02T13:51:05.362478+0000","last_unstale":"2024-04-02T13:53:29.898007+0000","last_undegraded":"2024-04-02T13:53:29.898007+0000","last_fullsized":"2024-04-02T13:53:29.898007+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:05:54.304458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997831+0000","last_change":"2024-04-02T13:51:00.247800+0000","last_active":"2024-04-02T13:53:29.997831+0000","last_peered":"2024-04-02T13:53:29.997831+0000","last_clean":"2024-04-02T13:53:29.997831+0000","last_became_active":"2024-04-02T13:51:00.247485+0000","last_became_peered":"2024-04-02T13:51:00.247485+0000","last_unstale":"2024-04-02T13:53:29.997831+0000","last_undegraded":"2024-04-02T13:53:29.997831+0000","last_fullsized":"2024-04-02T13:53:29.997831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:17:24.767558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.175194+0000","last_change":"2024-04-02T13:51:01.261416+0000","last_active":"2024-04-02T13:53:30.175194+0000","last_peered":"2024-04-02T13:53:30.175194+0000","last_clean":"2024-04-02T13:53:30.175194+0000","last_became_active":"2024-04-02T13:51:01.261099+0000","last_became_peered":"2024-04-02T13:51:01.261099+0000","last_unstale":"2024-04-02T13:53:30.175194+0000","last_undegraded":"2024-04-02T13:53:30.175194+0000","last_fullsized":"2024-04-02T13:53:30.175194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:10:04.999402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.897988+0000","last_change":"2024-04-02T13:51:05.361747+0000","last_active":"2024-04-02T13:53:29.897988+0000","last_peered":"2024-04-02T13:53:29.897988+0000","last_clean":"2024-04-02T13:53:29.897988+0000","last_became_active":"2024-04-02T13:51:05.361098+0000","last_became_peered":"2024-04-02T13:51:05.361098+0000","last_unstale":"2024-04-02T13:53:29.897988+0000","last_undegraded":"2024-04-02T13:53:29.897988+0000","last_fullsized":"2024-04-02T13:53:29.897988+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:45:35.048870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997422+0000","last_change":"2024-04-02T13:51:00.244048+0000","last_active":"2024-04-02T13:53:29.997422+0000","last_peered":"2024-04-02T13:53:29.997422+0000","last_clean":"2024-04-02T13:53:29.997422+0000","last_became_active":"2024-04-02T13:51:00.243510+0000","last_became_peered":"2024-04-02T13:51:00.243510+0000","last_unstale":"2024-04-02T13:53:29.997422+0000","last_undegraded":"2024-04-02T13:53:29.997422+0000","last_fullsized":"2024-04-02T13:53:29.997422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:53:48.364137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136246+0000","last_change":"2024-04-02T13:51:01.264711+0000","last_active":"2024-04-02T13:53:30.136246+0000","last_peered":"2024-04-02T13:53:30.136246+0000","last_clean":"2024-04-02T13:53:30.136246+0000","last_became_active":"2024-04-02T13:51:01.264289+0000","last_became_peered":"2024-04-02T13:51:01.264289+0000","last_unstale":"2024-04-02T13:53:30.136246+0000","last_undegraded":"2024-04-02T13:53:30.136246+0000","last_fullsized":"2024-04-02T13:53:30.136246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:34:41.590071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.897930+0000","last_change":"2024-04-02T13:51:05.363121+0000","last_active":"2024-04-02T13:53:29.897930+0000","last_peered":"2024-04-02T13:53:29.897930+0000","last_clean":"2024-04-02T13:53:29.897930+0000","last_became_active":"2024-04-02T13:51:05.362642+0000","last_became_peered":"2024-04-02T13:51:05.362642+0000","last_unstale":"2024-04-02T13:53:29.897930+0000","last_undegraded":"2024-04-02T13:53:29.897930+0000","last_fullsized":"2024-04-02T13:53:29.897930+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:55:40.848731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997155+0000","last_change":"2024-04-02T13:51:00.236519+0000","last_active":"2024-04-02T13:53:29.997155+0000","last_peered":"2024-04-02T13:53:29.997155+0000","last_clean":"2024-04-02T13:53:29.997155+0000","last_became_active":"2024-04-02T13:51:00.235985+0000","last_became_peered":"2024-04-02T13:51:00.235985+0000","last_unstale":"2024-04-02T13:53:29.997155+0000","last_undegraded":"2024-04-02T13:53:29.997155+0000","last_fullsized":"2024-04-02T13:53:29.997155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:08.321765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895438+0000","last_change":"2024-04-02T13:51:01.260333+0000","last_active":"2024-04-02T13:53:29.895438+0000","last_peered":"2024-04-02T13:53:29.895438+0000","last_clean":"2024-04-02T13:53:29.895438+0000","last_became_active":"2024-04-02T13:51:01.259995+0000","last_became_peered":"2024-04-02T13:51:01.259995+0000","last_unstale":"2024-04-02T13:53:29.895438+0000","last_undegraded":"2024-04-02T13:53:29.895438+0000","last_fullsized":"2024-04-02T13:53:29.895438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:03:58.088653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135314+0000","last_change":"2024-04-02T13:51:05.359281+0000","last_active":"2024-04-02T13:53:30.135314+0000","last_peered":"2024-04-02T13:53:30.135314+0000","last_clean":"2024-04-02T13:53:30.135314+0000","last_became_active":"2024-04-02T13:51:05.358831+0000","last_became_peered":"2024-04-02T13:51:05.358831+0000","last_unstale":"2024-04-02T13:53:30.135314+0000","last_undegraded":"2024-04-02T13:53:30.135314+0000","last_fullsized":"2024-04-02T13:53:30.135314+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:48.461298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'7","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.508533+0000","last_change":"2024-04-02T13:51:00.241707+0000","last_active":"2024-04-02T13:53:30.508533+0000","last_peered":"2024-04-02T13:53:30.508533+0000","last_clean":"2024-04-02T13:53:30.508533+0000","last_became_active":"2024-04-02T13:51:00.241225+0000","last_became_peered":"2024-04-02T13:51:00.241225+0000","last_unstale":"2024-04-02T13:53:30.508533+0000","last_undegraded":"2024-04-02T13:53:30.508533+0000","last_fullsized":"2024-04-02T13:53:30.508533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:59:44.537503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1404,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997009+0000","last_change":"2024-04-02T13:51:01.252942+0000","last_active":"2024-04-02T13:53:29.997009+0000","last_peered":"2024-04-02T13:53:29.997009+0000","last_clean":"2024-04-02T13:53:29.997009+0000","last_became_active":"2024-04-02T13:51:01.252631+0000","last_became_peered":"2024-04-02T13:51:01.252631+0000","last_unstale":"2024-04-02T13:53:29.997009+0000","last_undegraded":"2024-04-02T13:53:29.997009+0000","last_fullsized":"2024-04-02T13:53:29.997009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:43:24.725688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135372+0000","last_change":"2024-04-02T13:51:05.359208+0000","last_active":"2024-04-02T13:53:30.135372+0000","last_peered":"2024-04-02T13:53:30.135372+0000","last_clean":"2024-04-02T13:53:30.135372+0000","last_became_active":"2024-04-02T13:51:05.358738+0000","last_became_peered":"2024-04-02T13:51:05.358738+0000","last_unstale":"2024-04-02T13:53:30.135372+0000","last_undegraded":"2024-04-02T13:53:30.135372+0000","last_fullsized":"2024-04-02T13:53:30.135372+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:47:46.034562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.003266+0000","last_change":"2024-04-02T13:51:00.236413+0000","last_active":"2024-04-02T13:53:30.003266+0000","last_peered":"2024-04-02T13:53:30.003266+0000","last_clean":"2024-04-02T13:53:30.003266+0000","last_became_active":"2024-04-02T13:51:00.236046+0000","last_became_peered":"2024-04-02T13:51:00.236046+0000","last_unstale":"2024-04-02T13:53:30.003266+0000","last_undegraded":"2024-04-02T13:53:30.003266+0000","last_fullsized":"2024-04-02T13:53:30.003266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:25:15.000726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895344+0000","last_change":"2024-04-02T13:51:01.265471+0000","last_active":"2024-04-02T13:53:29.895344+0000","last_peered":"2024-04-02T13:53:29.895344+0000","last_clean":"2024-04-02T13:53:29.895344+0000","last_became_active":"2024-04-02T13:51:01.265114+0000","last_became_peered":"2024-04-02T13:51:01.265114+0000","last_unstale":"2024-04-02T13:53:29.895344+0000","last_undegraded":"2024-04-02T13:53:29.895344+0000","last_fullsized":"2024-04-02T13:53:29.895344+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:01:01.675686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174824+0000","last_change":"2024-04-02T13:51:56.664063+0000","last_active":"2024-04-02T13:53:30.174824+0000","last_peered":"2024-04-02T13:53:30.174824+0000","last_clean":"2024-04-02T13:53:30.174824+0000","last_became_active":"2024-04-02T13:51:56.663590+0000","last_became_peered":"2024-04-02T13:51:56.663590+0000","last_unstale":"2024-04-02T13:53:30.174824+0000","last_undegraded":"2024-04-02T13:53:30.174824+0000","last_fullsized":"2024-04-02T13:53:30.174824+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:43:51.735851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.893833+0000","last_change":"2024-04-02T13:51:00.237204+0000","last_active":"2024-04-02T13:53:29.893833+0000","last_peered":"2024-04-02T13:53:29.893833+0000","last_clean":"2024-04-02T13:53:29.893833+0000","last_became_active":"2024-04-02T13:51:00.236707+0000","last_became_peered":"2024-04-02T13:51:00.236707+0000","last_unstale":"2024-04-02T13:53:29.893833+0000","last_undegraded":"2024-04-02T13:53:29.893833+0000","last_fullsized":"2024-04-02T13:53:29.893833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:00:17.771428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997017+0000","last_change":"2024-04-02T13:51:01.250152+0000","last_active":"2024-04-02T13:53:29.997017+0000","last_peered":"2024-04-02T13:53:29.997017+0000","last_clean":"2024-04-02T13:53:29.997017+0000","last_became_active":"2024-04-02T13:51:01.249817+0000","last_became_peered":"2024-04-02T13:51:01.249817+0000","last_unstale":"2024-04-02T13:53:29.997017+0000","last_undegraded":"2024-04-02T13:53:29.997017+0000","last_fullsized":"2024-04-02T13:53:29.997017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:13:31.719380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.175093+0000","last_change":"2024-04-02T13:51:05.366551+0000","last_active":"2024-04-02T13:53:30.175093+0000","last_peered":"2024-04-02T13:53:30.175093+0000","last_clean":"2024-04-02T13:53:30.175093+0000","last_became_active":"2024-04-02T13:51:05.366175+0000","last_became_peered":"2024-04-02T13:51:05.366175+0000","last_unstale":"2024-04-02T13:53:30.175093+0000","last_undegraded":"2024-04-02T13:53:30.175093+0000","last_fullsized":"2024-04-02T13:53:30.175093+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:47:49.822918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895052+0000","last_change":"2024-04-02T13:51:00.239521+0000","last_active":"2024-04-02T13:53:29.895052+0000","last_peered":"2024-04-02T13:53:29.895052+0000","last_clean":"2024-04-02T13:53:29.895052+0000","last_became_active":"2024-04-02T13:51:00.238608+0000","last_became_peered":"2024-04-02T13:51:00.238608+0000","last_unstale":"2024-04-02T13:53:29.895052+0000","last_undegraded":"2024-04-02T13:53:29.895052+0000","last_fullsized":"2024-04-02T13:53:29.895052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:00.983006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.994418+0000","last_change":"2024-04-02T13:51:01.255021+0000","last_active":"2024-04-02T13:53:29.994418+0000","last_peered":"2024-04-02T13:53:29.994418+0000","last_clean":"2024-04-02T13:53:29.994418+0000","last_became_active":"2024-04-02T13:51:01.254630+0000","last_became_peered":"2024-04-02T13:51:01.254630+0000","last_unstale":"2024-04-02T13:53:29.994418+0000","last_undegraded":"2024-04-02T13:53:29.994418+0000","last_fullsized":"2024-04-02T13:53:29.994418+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:36:08.773431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996891+0000","last_change":"2024-04-02T13:51:05.362004+0000","last_active":"2024-04-02T13:53:29.996891+0000","last_peered":"2024-04-02T13:53:29.996891+0000","last_clean":"2024-04-02T13:53:29.996891+0000","last_became_active":"2024-04-02T13:51:05.361538+0000","last_became_peered":"2024-04-02T13:51:05.361538+0000","last_unstale":"2024-04-02T13:53:29.996891+0000","last_undegraded":"2024-04-02T13:53:29.996891+0000","last_fullsized":"2024-04-02T13:53:29.996891+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:27:21.728695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'6","reported_seq":280,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.507365+0000","last_change":"2024-04-02T13:51:00.234814+0000","last_active":"2024-04-02T13:53:30.507365+0000","last_peered":"2024-04-02T13:53:30.507365+0000","last_clean":"2024-04-02T13:53:30.507365+0000","last_became_active":"2024-04-02T13:51:00.234481+0000","last_became_peered":"2024-04-02T13:51:00.234481+0000","last_unstale":"2024-04-02T13:53:30.507365+0000","last_undegraded":"2024-04-02T13:53:30.507365+0000","last_fullsized":"2024-04-02T13:53:30.507365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:45:34.901226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":244,"num_read_kb":244,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135656+0000","last_change":"2024-04-02T13:51:01.264093+0000","last_active":"2024-04-02T13:53:30.135656+0000","last_peered":"2024-04-02T13:53:30.135656+0000","last_clean":"2024-04-02T13:53:30.135656+0000","last_became_active":"2024-04-02T13:51:01.263754+0000","last_became_peered":"2024-04-02T13:51:01.263754+0000","last_unstale":"2024-04-02T13:53:30.135656+0000","last_undegraded":"2024-04-02T13:53:30.135656+0000","last_fullsized":"2024-04-02T13:53:30.135656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:49:18.630195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.994363+0000","last_change":"2024-04-02T13:51:05.370930+0000","last_active":"2024-04-02T13:53:29.994363+0000","last_peered":"2024-04-02T13:53:29.994363+0000","last_clean":"2024-04-02T13:53:29.994363+0000","last_became_active":"2024-04-02T13:51:05.370443+0000","last_became_peered":"2024-04-02T13:51:05.370443+0000","last_unstale":"2024-04-02T13:53:29.994363+0000","last_undegraded":"2024-04-02T13:53:29.994363+0000","last_fullsized":"2024-04-02T13:53:29.994363+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:24:27.055494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136337+0000","last_change":"2024-04-02T13:51:00.241206+0000","last_active":"2024-04-02T13:53:30.136337+0000","last_peered":"2024-04-02T13:53:30.136337+0000","last_clean":"2024-04-02T13:53:30.136337+0000","last_became_active":"2024-04-02T13:51:00.240832+0000","last_became_peered":"2024-04-02T13:51:00.240832+0000","last_unstale":"2024-04-02T13:53:30.136337+0000","last_undegraded":"2024-04-02T13:53:30.136337+0000","last_fullsized":"2024-04-02T13:53:30.136337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:44:11.626435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.175384+0000","last_change":"2024-04-02T13:51:01.259315+0000","last_active":"2024-04-02T13:53:30.175384+0000","last_peered":"2024-04-02T13:53:30.175384+0000","last_clean":"2024-04-02T13:53:30.175384+0000","last_became_active":"2024-04-02T13:51:01.258951+0000","last_became_peered":"2024-04-02T13:51:01.258951+0000","last_unstale":"2024-04-02T13:53:30.175384+0000","last_undegraded":"2024-04-02T13:53:30.175384+0000","last_fullsized":"2024-04-02T13:53:30.175384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:59:43.714044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898359+0000","last_change":"2024-04-02T13:51:56.665826+0000","last_active":"2024-04-02T13:53:29.898359+0000","last_peered":"2024-04-02T13:53:29.898359+0000","last_clean":"2024-04-02T13:53:29.898359+0000","last_became_active":"2024-04-02T13:51:56.665213+0000","last_became_peered":"2024-04-02T13:51:56.665213+0000","last_unstale":"2024-04-02T13:53:29.898359+0000","last_undegraded":"2024-04-02T13:53:29.898359+0000","last_fullsized":"2024-04-02T13:53:29.898359+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:45:46.625063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895401+0000","last_change":"2024-04-02T13:51:00.230938+0000","last_active":"2024-04-02T13:53:29.895401+0000","last_peered":"2024-04-02T13:53:29.895401+0000","last_clean":"2024-04-02T13:53:29.895401+0000","last_became_active":"2024-04-02T13:51:00.230585+0000","last_became_peered":"2024-04-02T13:51:00.230585+0000","last_unstale":"2024-04-02T13:53:29.895401+0000","last_undegraded":"2024-04-02T13:53:29.895401+0000","last_fullsized":"2024-04-02T13:53:29.895401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:51:37.463876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136591+0000","last_change":"2024-04-02T13:51:01.264216+0000","last_active":"2024-04-02T13:53:30.136591+0000","last_peered":"2024-04-02T13:53:30.136591+0000","last_clean":"2024-04-02T13:53:30.136591+0000","last_became_active":"2024-04-02T13:51:01.263824+0000","last_became_peered":"2024-04-02T13:51:01.263824+0000","last_unstale":"2024-04-02T13:53:30.136591+0000","last_undegraded":"2024-04-02T13:53:30.136591+0000","last_fullsized":"2024-04-02T13:53:30.136591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:16:26.391163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.994569+0000","last_change":"2024-04-02T13:51:05.366002+0000","last_active":"2024-04-02T13:53:29.994569+0000","last_peered":"2024-04-02T13:53:29.994569+0000","last_clean":"2024-04-02T13:53:29.994569+0000","last_became_active":"2024-04-02T13:51:05.365547+0000","last_became_peered":"2024-04-02T13:51:05.365547+0000","last_unstale":"2024-04-02T13:53:29.994569+0000","last_undegraded":"2024-04-02T13:53:29.994569+0000","last_fullsized":"2024-04-02T13:53:29.994569+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:31:38.990617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'43","reported_seq":327,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.507596+0000","last_change":"2024-04-02T13:51:00.233925+0000","last_active":"2024-04-02T13:53:30.507596+0000","last_peered":"2024-04-02T13:53:30.507596+0000","last_clean":"2024-04-02T13:53:30.507596+0000","last_became_active":"2024-04-02T13:51:00.233574+0000","last_became_peered":"2024-04-02T13:51:00.233574+0000","last_unstale":"2024-04-02T13:53:30.507596+0000","last_undegraded":"2024-04-02T13:53:30.507596+0000","last_fullsized":"2024-04-02T13:53:30.507596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:33:32.758874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69045,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":254,"num_read_kb":196,"num_write":45,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997226+0000","last_change":"2024-04-02T13:51:01.258825+0000","last_active":"2024-04-02T13:53:29.997226+0000","last_peered":"2024-04-02T13:53:29.997226+0000","last_clean":"2024-04-02T13:53:29.997226+0000","last_became_active":"2024-04-02T13:51:01.258500+0000","last_became_peered":"2024-04-02T13:51:01.258500+0000","last_unstale":"2024-04-02T13:53:29.997226+0000","last_undegraded":"2024-04-02T13:53:29.997226+0000","last_fullsized":"2024-04-02T13:53:29.997226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:33:15.409765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895709+0000","last_change":"2024-04-02T13:51:05.365010+0000","last_active":"2024-04-02T13:53:29.895709+0000","last_peered":"2024-04-02T13:53:29.895709+0000","last_clean":"2024-04-02T13:53:29.895709+0000","last_became_active":"2024-04-02T13:51:05.364499+0000","last_became_peered":"2024-04-02T13:51:05.364499+0000","last_unstale":"2024-04-02T13:53:29.895709+0000","last_undegraded":"2024-04-02T13:53:29.895709+0000","last_fullsized":"2024-04-02T13:53:29.895709+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:07:03.500017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174673+0000","last_change":"2024-04-02T13:51:56.496876+0000","last_active":"2024-04-02T13:53:30.174673+0000","last_peered":"2024-04-02T13:53:30.174673+0000","last_clean":"2024-04-02T13:53:30.174673+0000","last_became_active":"2024-04-02T13:51:56.484672+0000","last_became_peered":"2024-04-02T13:51:56.484672+0000","last_unstale":"2024-04-02T13:53:30.174673+0000","last_undegraded":"2024-04-02T13:53:30.174673+0000","last_fullsized":"2024-04-02T13:53:30.174673+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:31:40.027658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.893864+0000","last_change":"2024-04-02T13:51:01.256334+0000","last_active":"2024-04-02T13:53:29.893864+0000","last_peered":"2024-04-02T13:53:29.893864+0000","last_clean":"2024-04-02T13:53:29.893864+0000","last_became_active":"2024-04-02T13:51:01.255964+0000","last_became_peered":"2024-04-02T13:51:01.255964+0000","last_unstale":"2024-04-02T13:53:29.893864+0000","last_undegraded":"2024-04-02T13:53:29.893864+0000","last_fullsized":"2024-04-02T13:53:29.893864+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:43:04.842812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174877+0000","last_change":"2024-04-02T13:51:05.362696+0000","last_active":"2024-04-02T13:53:30.174877+0000","last_peered":"2024-04-02T13:53:30.174877+0000","last_clean":"2024-04-02T13:53:30.174877+0000","last_became_active":"2024-04-02T13:51:05.362225+0000","last_became_peered":"2024-04-02T13:51:05.362225+0000","last_unstale":"2024-04-02T13:53:30.174877+0000","last_undegraded":"2024-04-02T13:53:30.174877+0000","last_fullsized":"2024-04-02T13:53:30.174877+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:23:56.308286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895454+0000","last_change":"2024-04-02T13:51:00.239673+0000","last_active":"2024-04-02T13:53:29.895454+0000","last_peered":"2024-04-02T13:53:29.895454+0000","last_clean":"2024-04-02T13:53:29.895454+0000","last_became_active":"2024-04-02T13:51:00.239233+0000","last_became_peered":"2024-04-02T13:51:00.239233+0000","last_unstale":"2024-04-02T13:53:29.895454+0000","last_undegraded":"2024-04-02T13:53:29.895454+0000","last_fullsized":"2024-04-02T13:53:29.895454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:05:48.119953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135940+0000","last_change":"2024-04-02T13:51:01.257474+0000","last_active":"2024-04-02T13:53:30.135940+0000","last_peered":"2024-04-02T13:53:30.135940+0000","last_clean":"2024-04-02T13:53:30.135940+0000","last_became_active":"2024-04-02T13:51:01.254977+0000","last_became_peered":"2024-04-02T13:51:01.254977+0000","last_unstale":"2024-04-02T13:53:30.135940+0000","last_undegraded":"2024-04-02T13:53:30.135940+0000","last_fullsized":"2024-04-02T13:53:30.135940+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:13:07.446520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.994825+0000","last_change":"2024-04-02T13:51:05.371531+0000","last_active":"2024-04-02T13:53:29.994825+0000","last_peered":"2024-04-02T13:53:29.994825+0000","last_clean":"2024-04-02T13:53:29.994825+0000","last_became_active":"2024-04-02T13:51:05.370551+0000","last_became_peered":"2024-04-02T13:51:05.370551+0000","last_unstale":"2024-04-02T13:53:29.994825+0000","last_undegraded":"2024-04-02T13:53:29.994825+0000","last_fullsized":"2024-04-02T13:53:29.994825+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:43:48.796009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'5","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898007+0000","last_change":"2024-04-02T13:51:00.237715+0000","last_active":"2024-04-02T13:53:29.898007+0000","last_peered":"2024-04-02T13:53:29.898007+0000","last_clean":"2024-04-02T13:53:29.898007+0000","last_became_active":"2024-04-02T13:51:00.237349+0000","last_became_peered":"2024-04-02T13:51:00.237349+0000","last_unstale":"2024-04-02T13:53:29.898007+0000","last_undegraded":"2024-04-02T13:53:29.898007+0000","last_fullsized":"2024-04-02T13:53:29.898007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:33:20.651562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135285+0000","last_change":"2024-04-02T13:51:01.259447+0000","last_active":"2024-04-02T13:53:30.135285+0000","last_peered":"2024-04-02T13:53:30.135285+0000","last_clean":"2024-04-02T13:53:30.135285+0000","last_became_active":"2024-04-02T13:51:01.259101+0000","last_became_peered":"2024-04-02T13:51:01.259101+0000","last_unstale":"2024-04-02T13:53:30.135285+0000","last_undegraded":"2024-04-02T13:53:30.135285+0000","last_fullsized":"2024-04-02T13:53:30.135285+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:51:44.311440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894370+0000","last_change":"2024-04-02T13:51:05.372446+0000","last_active":"2024-04-02T13:53:29.894370+0000","last_peered":"2024-04-02T13:53:29.894370+0000","last_clean":"2024-04-02T13:53:29.894370+0000","last_became_active":"2024-04-02T13:51:05.372007+0000","last_became_peered":"2024-04-02T13:51:05.372007+0000","last_unstale":"2024-04-02T13:53:29.894370+0000","last_undegraded":"2024-04-02T13:53:29.894370+0000","last_fullsized":"2024-04-02T13:53:29.894370+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:54:23.463179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996804+0000","last_change":"2024-04-02T13:51:00.241492+0000","last_active":"2024-04-02T13:53:29.996804+0000","last_peered":"2024-04-02T13:53:29.996804+0000","last_clean":"2024-04-02T13:53:29.996804+0000","last_became_active":"2024-04-02T13:51:00.241167+0000","last_became_peered":"2024-04-02T13:51:00.241167+0000","last_unstale":"2024-04-02T13:53:29.996804+0000","last_undegraded":"2024-04-02T13:53:29.996804+0000","last_fullsized":"2024-04-02T13:53:29.996804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:44:44.070187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895050+0000","last_change":"2024-04-02T13:51:01.263200+0000","last_active":"2024-04-02T13:53:29.895050+0000","last_peered":"2024-04-02T13:53:29.895050+0000","last_clean":"2024-04-02T13:53:29.895050+0000","last_became_active":"2024-04-02T13:51:01.262812+0000","last_became_peered":"2024-04-02T13:51:01.262812+0000","last_unstale":"2024-04-02T13:53:29.895050+0000","last_undegraded":"2024-04-02T13:53:29.895050+0000","last_fullsized":"2024-04-02T13:53:29.895050+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:33:35.898047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895041+0000","last_change":"2024-04-02T13:51:05.366729+0000","last_active":"2024-04-02T13:53:29.895041+0000","last_peered":"2024-04-02T13:53:29.895041+0000","last_clean":"2024-04-02T13:53:29.895041+0000","last_became_active":"2024-04-02T13:51:05.366257+0000","last_became_peered":"2024-04-02T13:51:05.366257+0000","last_unstale":"2024-04-02T13:53:29.895041+0000","last_undegraded":"2024-04-02T13:53:29.895041+0000","last_fullsized":"2024-04-02T13:53:29.895041+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:46:03.400300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997357+0000","last_change":"2024-04-02T13:51:00.236004+0000","last_active":"2024-04-02T13:53:29.997357+0000","last_peered":"2024-04-02T13:53:29.997357+0000","last_clean":"2024-04-02T13:53:29.997357+0000","last_became_active":"2024-04-02T13:51:00.235160+0000","last_became_peered":"2024-04-02T13:51:00.235160+0000","last_unstale":"2024-04-02T13:53:29.997357+0000","last_undegraded":"2024-04-02T13:53:29.997357+0000","last_fullsized":"2024-04-02T13:53:29.997357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:16:35.617346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174808+0000","last_change":"2024-04-02T13:51:01.259565+0000","last_active":"2024-04-02T13:53:30.174808+0000","last_peered":"2024-04-02T13:53:30.174808+0000","last_clean":"2024-04-02T13:53:30.174808+0000","last_became_active":"2024-04-02T13:51:01.259225+0000","last_became_peered":"2024-04-02T13:51:01.259225+0000","last_unstale":"2024-04-02T13:53:30.174808+0000","last_undegraded":"2024-04-02T13:53:30.174808+0000","last_fullsized":"2024-04-02T13:53:30.174808+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:22:18.878029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996871+0000","last_change":"2024-04-02T13:51:05.518750+0000","last_active":"2024-04-02T13:53:29.996871+0000","last_peered":"2024-04-02T13:53:29.996871+0000","last_clean":"2024-04-02T13:53:29.996871+0000","last_became_active":"2024-04-02T13:51:05.518247+0000","last_became_peered":"2024-04-02T13:51:05.518247+0000","last_unstale":"2024-04-02T13:53:29.996871+0000","last_undegraded":"2024-04-02T13:53:29.996871+0000","last_fullsized":"2024-04-02T13:53:29.996871+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:08:10.232509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.994718+0000","last_change":"2024-04-02T13:51:00.241476+0000","last_active":"2024-04-02T13:53:29.994718+0000","last_peered":"2024-04-02T13:53:29.994718+0000","last_clean":"2024-04-02T13:53:29.994718+0000","last_became_active":"2024-04-02T13:51:00.241085+0000","last_became_peered":"2024-04-02T13:51:00.241085+0000","last_unstale":"2024-04-02T13:53:29.994718+0000","last_undegraded":"2024-04-02T13:53:29.994718+0000","last_fullsized":"2024-04-02T13:53:29.994718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:29:01.503901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.003509+0000","last_change":"2024-04-02T13:51:01.259791+0000","last_active":"2024-04-02T13:53:30.003509+0000","last_peered":"2024-04-02T13:53:30.003509+0000","last_clean":"2024-04-02T13:53:30.003509+0000","last_became_active":"2024-04-02T13:51:01.259440+0000","last_became_peered":"2024-04-02T13:51:01.259440+0000","last_unstale":"2024-04-02T13:53:30.003509+0000","last_undegraded":"2024-04-02T13:53:30.003509+0000","last_fullsized":"2024-04-02T13:53:30.003509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:43:52.422263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894028+0000","last_change":"2024-04-02T13:51:05.366243+0000","last_active":"2024-04-02T13:53:29.894028+0000","last_peered":"2024-04-02T13:53:29.894028+0000","last_clean":"2024-04-02T13:53:29.894028+0000","last_became_active":"2024-04-02T13:51:05.365801+0000","last_became_peered":"2024-04-02T13:51:05.365801+0000","last_unstale":"2024-04-02T13:53:29.894028+0000","last_undegraded":"2024-04-02T13:53:29.894028+0000","last_fullsized":"2024-04-02T13:53:29.894028+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:33:56.853434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"93'6","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.507013+0000","last_change":"2024-04-02T13:51:00.248695+0000","last_active":"2024-04-02T13:53:30.507013+0000","last_peered":"2024-04-02T13:53:30.507013+0000","last_clean":"2024-04-02T13:53:30.507013+0000","last_became_active":"2024-04-02T13:51:00.248313+0000","last_became_peered":"2024-04-02T13:51:00.248313+0000","last_unstale":"2024-04-02T13:53:30.507013+0000","last_undegraded":"2024-04-02T13:53:30.507013+0000","last_fullsized":"2024-04-02T13:53:30.507013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:43:21.410823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.003516+0000","last_change":"2024-04-02T13:51:01.260993+0000","last_active":"2024-04-02T13:53:30.003516+0000","last_peered":"2024-04-02T13:53:30.003516+0000","last_clean":"2024-04-02T13:53:30.003516+0000","last_became_active":"2024-04-02T13:51:01.260510+0000","last_became_peered":"2024-04-02T13:51:01.260510+0000","last_unstale":"2024-04-02T13:53:30.003516+0000","last_undegraded":"2024-04-02T13:53:30.003516+0000","last_fullsized":"2024-04-02T13:53:30.003516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:17:04.941822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894181+0000","last_change":"2024-04-02T13:51:05.518455+0000","last_active":"2024-04-02T13:53:29.894181+0000","last_peered":"2024-04-02T13:53:29.894181+0000","last_clean":"2024-04-02T13:53:29.894181+0000","last_became_active":"2024-04-02T13:51:05.517962+0000","last_became_peered":"2024-04-02T13:51:05.517962+0000","last_unstale":"2024-04-02T13:53:29.894181+0000","last_undegraded":"2024-04-02T13:53:29.894181+0000","last_fullsized":"2024-04-02T13:53:29.894181+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:19:47.734233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895152+0000","last_change":"2024-04-02T13:51:01.259220+0000","last_active":"2024-04-02T13:53:29.895152+0000","last_peered":"2024-04-02T13:53:29.895152+0000","last_clean":"2024-04-02T13:53:29.895152+0000","last_became_active":"2024-04-02T13:51:01.258890+0000","last_became_peered":"2024-04-02T13:51:01.258890+0000","last_unstale":"2024-04-02T13:53:29.895152+0000","last_undegraded":"2024-04-02T13:53:29.895152+0000","last_fullsized":"2024-04-02T13:53:29.895152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:56:15.444736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.893734+0000","last_change":"2024-04-02T13:51:00.246727+0000","last_active":"2024-04-02T13:53:29.893734+0000","last_peered":"2024-04-02T13:53:29.893734+0000","last_clean":"2024-04-02T13:53:29.893734+0000","last_became_active":"2024-04-02T13:51:00.246364+0000","last_became_peered":"2024-04-02T13:51:00.246364+0000","last_unstale":"2024-04-02T13:53:29.893734+0000","last_undegraded":"2024-04-02T13:53:29.893734+0000","last_fullsized":"2024-04-02T13:53:29.893734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:35:13.249972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":123,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136219+0000","last_change":"2024-04-02T13:51:05.358506+0000","last_active":"2024-04-02T13:53:30.136219+0000","last_peered":"2024-04-02T13:53:30.136219+0000","last_clean":"2024-04-02T13:53:30.136219+0000","last_became_active":"2024-04-02T13:51:05.358157+0000","last_became_peered":"2024-04-02T13:51:05.358157+0000","last_unstale":"2024-04-02T13:53:30.136219+0000","last_undegraded":"2024-04-02T13:53:30.136219+0000","last_fullsized":"2024-04-02T13:53:30.136219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:24:15.585124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.994632+0000","last_change":"2024-04-02T13:51:56.665307+0000","last_active":"2024-04-02T13:53:29.994632+0000","last_peered":"2024-04-02T13:53:29.994632+0000","last_clean":"2024-04-02T13:53:29.994632+0000","last_became_active":"2024-04-02T13:51:56.665003+0000","last_became_peered":"2024-04-02T13:51:56.665003+0000","last_unstale":"2024-04-02T13:53:29.994632+0000","last_undegraded":"2024-04-02T13:53:29.994632+0000","last_fullsized":"2024-04-02T13:53:29.994632+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:02:46.421524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898110+0000","last_change":"2024-04-02T13:51:00.238962+0000","last_active":"2024-04-02T13:53:29.898110+0000","last_peered":"2024-04-02T13:53:29.898110+0000","last_clean":"2024-04-02T13:53:29.898110+0000","last_became_active":"2024-04-02T13:51:00.238475+0000","last_became_peered":"2024-04-02T13:51:00.238475+0000","last_unstale":"2024-04-02T13:53:29.898110+0000","last_undegraded":"2024-04-02T13:53:29.898110+0000","last_fullsized":"2024-04-02T13:53:29.898110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:41:07.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":364,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135140+0000","last_change":"2024-04-02T13:49:47.847355+0000","last_active":"2024-04-02T13:53:30.135140+0000","last_peered":"2024-04-02T13:53:30.135140+0000","last_clean":"2024-04-02T13:53:30.135140+0000","last_became_active":"2024-04-02T13:49:47.846768+0000","last_became_peered":"2024-04-02T13:49:47.846768+0000","last_unstale":"2024-04-02T13:53:30.135140+0000","last_undegraded":"2024-04-02T13:53:30.135140+0000","last_fullsized":"2024-04-02T13:53:30.135140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:46:21.098780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:46:21.098780+0000","last_clean_scrub_stamp":"2024-04-02T13:46:21.098780+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:03:52.323921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895591+0000","last_change":"2024-04-02T13:51:05.518859+0000","last_active":"2024-04-02T13:53:29.895591+0000","last_peered":"2024-04-02T13:53:29.895591+0000","last_clean":"2024-04-02T13:53:29.895591+0000","last_became_active":"2024-04-02T13:51:05.518364+0000","last_became_peered":"2024-04-02T13:51:05.518364+0000","last_unstale":"2024-04-02T13:53:29.895591+0000","last_undegraded":"2024-04-02T13:53:29.895591+0000","last_fullsized":"2024-04-02T13:53:29.895591+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:10:45.877853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.893663+0000","last_change":"2024-04-02T13:51:01.254795+0000","last_active":"2024-04-02T13:53:29.893663+0000","last_peered":"2024-04-02T13:53:29.893663+0000","last_clean":"2024-04-02T13:53:29.893663+0000","last_became_active":"2024-04-02T13:51:01.254452+0000","last_became_peered":"2024-04-02T13:51:01.254452+0000","last_unstale":"2024-04-02T13:53:29.893663+0000","last_undegraded":"2024-04-02T13:53:29.893663+0000","last_fullsized":"2024-04-02T13:53:29.893663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:13:53.747308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135234+0000","last_change":"2024-04-02T13:51:00.241323+0000","last_active":"2024-04-02T13:53:30.135234+0000","last_peered":"2024-04-02T13:53:30.135234+0000","last_clean":"2024-04-02T13:53:30.135234+0000","last_became_active":"2024-04-02T13:51:00.240989+0000","last_became_peered":"2024-04-02T13:51:00.240989+0000","last_unstale":"2024-04-02T13:53:30.135234+0000","last_undegraded":"2024-04-02T13:53:30.135234+0000","last_fullsized":"2024-04-02T13:53:30.135234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:47:52.601517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174717+0000","last_change":"2024-04-02T13:51:56.665905+0000","last_active":"2024-04-02T13:53:30.174717+0000","last_peered":"2024-04-02T13:53:30.174717+0000","last_clean":"2024-04-02T13:53:30.174717+0000","last_became_active":"2024-04-02T13:51:56.665315+0000","last_became_peered":"2024-04-02T13:51:56.665315+0000","last_unstale":"2024-04-02T13:53:30.174717+0000","last_undegraded":"2024-04-02T13:53:30.174717+0000","last_fullsized":"2024-04-02T13:53:30.174717+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:44:34.548102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997346+0000","last_change":"2024-04-02T13:51:00.241416+0000","last_active":"2024-04-02T13:53:29.997346+0000","last_peered":"2024-04-02T13:53:29.997346+0000","last_clean":"2024-04-02T13:53:29.997346+0000","last_became_active":"2024-04-02T13:51:00.241071+0000","last_became_peered":"2024-04-02T13:51:00.241071+0000","last_unstale":"2024-04-02T13:53:29.997346+0000","last_undegraded":"2024-04-02T13:53:29.997346+0000","last_fullsized":"2024-04-02T13:53:29.997346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:24:16.243737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894214+0000","last_change":"2024-04-02T13:51:01.252156+0000","last_active":"2024-04-02T13:53:29.894214+0000","last_peered":"2024-04-02T13:53:29.894214+0000","last_clean":"2024-04-02T13:53:29.894214+0000","last_became_active":"2024-04-02T13:51:01.251733+0000","last_became_peered":"2024-04-02T13:51:01.251733+0000","last_unstale":"2024-04-02T13:53:29.894214+0000","last_undegraded":"2024-04-02T13:53:29.894214+0000","last_fullsized":"2024-04-02T13:53:29.894214+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:54:24.082643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996784+0000","last_change":"2024-04-02T13:51:05.518367+0000","last_active":"2024-04-02T13:53:29.996784+0000","last_peered":"2024-04-02T13:53:29.996784+0000","last_clean":"2024-04-02T13:53:29.996784+0000","last_became_active":"2024-04-02T13:51:05.517783+0000","last_became_peered":"2024-04-02T13:51:05.517783+0000","last_unstale":"2024-04-02T13:53:29.996784+0000","last_undegraded":"2024-04-02T13:53:29.996784+0000","last_fullsized":"2024-04-02T13:53:29.996784+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:10:53.880909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'48","reported_seq":101,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.107358+0000","last_change":"2024-04-02T13:51:00.248998+0000","last_active":"2024-04-02T13:53:30.107358+0000","last_peered":"2024-04-02T13:53:30.107358+0000","last_clean":"2024-04-02T13:53:30.107358+0000","last_became_active":"2024-04-02T13:51:00.248613+0000","last_became_peered":"2024-04-02T13:51:00.248613+0000","last_unstale":"2024-04-02T13:53:30.107358+0000","last_undegraded":"2024-04-02T13:53:30.107358+0000","last_fullsized":"2024-04-02T13:53:30.107358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:53.948645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997762+0000","last_change":"2024-04-02T13:51:01.257244+0000","last_active":"2024-04-02T13:53:29.997762+0000","last_peered":"2024-04-02T13:53:29.997762+0000","last_clean":"2024-04-02T13:53:29.997762+0000","last_became_active":"2024-04-02T13:51:01.256739+0000","last_became_peered":"2024-04-02T13:51:01.256739+0000","last_unstale":"2024-04-02T13:53:29.997762+0000","last_undegraded":"2024-04-02T13:53:29.997762+0000","last_fullsized":"2024-04-02T13:53:29.997762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:38:18.786553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.134886+0000","last_change":"2024-04-02T13:51:05.356177+0000","last_active":"2024-04-02T13:53:30.134886+0000","last_peered":"2024-04-02T13:53:30.134886+0000","last_clean":"2024-04-02T13:53:30.134886+0000","last_became_active":"2024-04-02T13:51:05.354704+0000","last_became_peered":"2024-04-02T13:51:05.354704+0000","last_unstale":"2024-04-02T13:53:30.134886+0000","last_undegraded":"2024-04-02T13:53:30.134886+0000","last_fullsized":"2024-04-02T13:53:30.134886+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:27:55.761490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898390+0000","last_change":"2024-04-02T13:51:00.245945+0000","last_active":"2024-04-02T13:53:29.898390+0000","last_peered":"2024-04-02T13:53:29.898390+0000","last_clean":"2024-04-02T13:53:29.898390+0000","last_became_active":"2024-04-02T13:51:00.245528+0000","last_became_peered":"2024-04-02T13:51:00.245528+0000","last_unstale":"2024-04-02T13:53:29.898390+0000","last_undegraded":"2024-04-02T13:53:29.898390+0000","last_fullsized":"2024-04-02T13:53:29.898390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:37:49.263614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.175330+0000","last_change":"2024-04-02T13:51:01.260268+0000","last_active":"2024-04-02T13:53:30.175330+0000","last_peered":"2024-04-02T13:53:30.175330+0000","last_clean":"2024-04-02T13:53:30.175330+0000","last_became_active":"2024-04-02T13:51:01.258182+0000","last_became_peered":"2024-04-02T13:51:01.258182+0000","last_unstale":"2024-04-02T13:53:30.175330+0000","last_undegraded":"2024-04-02T13:53:30.175330+0000","last_fullsized":"2024-04-02T13:53:30.175330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:31:31.423988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898255+0000","last_change":"2024-04-02T13:51:05.518597+0000","last_active":"2024-04-02T13:53:29.898255+0000","last_peered":"2024-04-02T13:53:29.898255+0000","last_clean":"2024-04-02T13:53:29.898255+0000","last_became_active":"2024-04-02T13:51:05.518134+0000","last_became_peered":"2024-04-02T13:51:05.518134+0000","last_unstale":"2024-04-02T13:53:29.898255+0000","last_undegraded":"2024-04-02T13:53:29.898255+0000","last_fullsized":"2024-04-02T13:53:29.898255+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:00:38.632939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135401+0000","last_change":"2024-04-02T13:51:00.241680+0000","last_active":"2024-04-02T13:53:30.135401+0000","last_peered":"2024-04-02T13:53:30.135401+0000","last_clean":"2024-04-02T13:53:30.135401+0000","last_became_active":"2024-04-02T13:51:00.241301+0000","last_became_peered":"2024-04-02T13:51:00.241301+0000","last_unstale":"2024-04-02T13:53:30.135401+0000","last_undegraded":"2024-04-02T13:53:30.135401+0000","last_fullsized":"2024-04-02T13:53:30.135401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:12:10.606973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996895+0000","last_change":"2024-04-02T13:51:01.254795+0000","last_active":"2024-04-02T13:53:29.996895+0000","last_peered":"2024-04-02T13:53:29.996895+0000","last_clean":"2024-04-02T13:53:29.996895+0000","last_became_active":"2024-04-02T13:51:01.254451+0000","last_became_peered":"2024-04-02T13:51:01.254451+0000","last_unstale":"2024-04-02T13:53:29.996895+0000","last_undegraded":"2024-04-02T13:53:29.996895+0000","last_fullsized":"2024-04-02T13:53:29.996895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:58:04.202996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.995353+0000","last_change":"2024-04-02T13:51:05.363592+0000","last_active":"2024-04-02T13:53:29.995353+0000","last_peered":"2024-04-02T13:53:29.995353+0000","last_clean":"2024-04-02T13:53:29.995353+0000","last_became_active":"2024-04-02T13:51:05.363211+0000","last_became_peered":"2024-04-02T13:51:05.363211+0000","last_unstale":"2024-04-02T13:53:29.995353+0000","last_undegraded":"2024-04-02T13:53:29.995353+0000","last_fullsized":"2024-04-02T13:53:29.995353+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:31:56.561505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996907+0000","last_change":"2024-04-02T13:51:00.243681+0000","last_active":"2024-04-02T13:53:29.996907+0000","last_peered":"2024-04-02T13:53:29.996907+0000","last_clean":"2024-04-02T13:53:29.996907+0000","last_became_active":"2024-04-02T13:51:00.241615+0000","last_became_peered":"2024-04-02T13:51:00.241615+0000","last_unstale":"2024-04-02T13:53:29.996907+0000","last_undegraded":"2024-04-02T13:53:29.996907+0000","last_fullsized":"2024-04-02T13:53:29.996907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:54:30.112735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174537+0000","last_change":"2024-04-02T13:51:01.252429+0000","last_active":"2024-04-02T13:53:30.174537+0000","last_peered":"2024-04-02T13:53:30.174537+0000","last_clean":"2024-04-02T13:53:30.174537+0000","last_became_active":"2024-04-02T13:51:01.252017+0000","last_became_peered":"2024-04-02T13:51:01.252017+0000","last_unstale":"2024-04-02T13:53:30.174537+0000","last_undegraded":"2024-04-02T13:53:30.174537+0000","last_fullsized":"2024-04-02T13:53:30.174537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:32:02.152578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898141+0000","last_change":"2024-04-02T13:51:05.361297+0000","last_active":"2024-04-02T13:53:29.898141+0000","last_peered":"2024-04-02T13:53:29.898141+0000","last_clean":"2024-04-02T13:53:29.898141+0000","last_became_active":"2024-04-02T13:51:05.360916+0000","last_became_peered":"2024-04-02T13:51:05.360916+0000","last_unstale":"2024-04-02T13:53:29.898141+0000","last_undegraded":"2024-04-02T13:53:29.898141+0000","last_fullsized":"2024-04-02T13:53:29.898141+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:52:48.608587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"93'4","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174766+0000","last_change":"2024-04-02T13:51:56.488312+0000","last_active":"2024-04-02T13:53:30.174766+0000","last_peered":"2024-04-02T13:53:30.174766+0000","last_clean":"2024-04-02T13:53:30.174766+0000","last_became_active":"2024-04-02T13:51:56.483987+0000","last_became_peered":"2024-04-02T13:51:56.483987+0000","last_unstale":"2024-04-02T13:53:30.174766+0000","last_undegraded":"2024-04-02T13:53:30.174766+0000","last_fullsized":"2024-04-02T13:53:30.174766+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:46:27.396997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997510+0000","last_change":"2024-04-02T13:51:01.255557+0000","last_active":"2024-04-02T13:53:29.997510+0000","last_peered":"2024-04-02T13:53:29.997510+0000","last_clean":"2024-04-02T13:53:29.997510+0000","last_became_active":"2024-04-02T13:51:01.255011+0000","last_became_peered":"2024-04-02T13:51:01.255011+0000","last_unstale":"2024-04-02T13:53:29.997510+0000","last_undegraded":"2024-04-02T13:53:29.997510+0000","last_fullsized":"2024-04-02T13:53:29.997510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:01:52.105965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898183+0000","last_change":"2024-04-02T13:51:05.518764+0000","last_active":"2024-04-02T13:53:29.898183+0000","last_peered":"2024-04-02T13:53:29.898183+0000","last_clean":"2024-04-02T13:53:29.898183+0000","last_became_active":"2024-04-02T13:51:05.518088+0000","last_became_peered":"2024-04-02T13:51:05.518088+0000","last_unstale":"2024-04-02T13:53:29.898183+0000","last_undegraded":"2024-04-02T13:53:29.898183+0000","last_fullsized":"2024-04-02T13:53:29.898183+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:03:02.901384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996946+0000","last_change":"2024-04-02T13:51:00.244147+0000","last_active":"2024-04-02T13:53:29.996946+0000","last_peered":"2024-04-02T13:53:29.996946+0000","last_clean":"2024-04-02T13:53:29.996946+0000","last_became_active":"2024-04-02T13:51:00.243763+0000","last_became_peered":"2024-04-02T13:51:00.243763+0000","last_unstale":"2024-04-02T13:53:29.996946+0000","last_undegraded":"2024-04-02T13:53:29.996946+0000","last_fullsized":"2024-04-02T13:53:29.996946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:34:27.738746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135780+0000","last_change":"2024-04-02T13:51:01.268121+0000","last_active":"2024-04-02T13:53:30.135780+0000","last_peered":"2024-04-02T13:53:30.135780+0000","last_clean":"2024-04-02T13:53:30.135780+0000","last_became_active":"2024-04-02T13:51:01.266363+0000","last_became_peered":"2024-04-02T13:51:01.266363+0000","last_unstale":"2024-04-02T13:53:30.135780+0000","last_undegraded":"2024-04-02T13:53:30.135780+0000","last_fullsized":"2024-04-02T13:53:30.135780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:30:50.254916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135846+0000","last_change":"2024-04-02T13:51:05.361285+0000","last_active":"2024-04-02T13:53:30.135846+0000","last_peered":"2024-04-02T13:53:30.135846+0000","last_clean":"2024-04-02T13:53:30.135846+0000","last_became_active":"2024-04-02T13:51:05.360122+0000","last_became_peered":"2024-04-02T13:51:05.360122+0000","last_unstale":"2024-04-02T13:53:30.135846+0000","last_undegraded":"2024-04-02T13:53:30.135846+0000","last_fullsized":"2024-04-02T13:53:30.135846+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:30:59.935538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'7","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135534+0000","last_change":"2024-04-02T13:51:00.247235+0000","last_active":"2024-04-02T13:53:30.135534+0000","last_peered":"2024-04-02T13:53:30.135534+0000","last_clean":"2024-04-02T13:53:30.135534+0000","last_became_active":"2024-04-02T13:51:00.246856+0000","last_became_peered":"2024-04-02T13:51:00.246856+0000","last_unstale":"2024-04-02T13:53:30.135534+0000","last_undegraded":"2024-04-02T13:53:30.135534+0000","last_fullsized":"2024-04-02T13:53:30.135534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:17:51.875952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136471+0000","last_change":"2024-04-02T13:51:01.263866+0000","last_active":"2024-04-02T13:53:30.136471+0000","last_peered":"2024-04-02T13:53:30.136471+0000","last_clean":"2024-04-02T13:53:30.136471+0000","last_became_active":"2024-04-02T13:51:01.263564+0000","last_became_peered":"2024-04-02T13:51:01.263564+0000","last_unstale":"2024-04-02T13:53:30.136471+0000","last_undegraded":"2024-04-02T13:53:30.136471+0000","last_fullsized":"2024-04-02T13:53:30.136471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:59:33.663277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174890+0000","last_change":"2024-04-02T13:51:05.363079+0000","last_active":"2024-04-02T13:53:30.174890+0000","last_peered":"2024-04-02T13:53:30.174890+0000","last_clean":"2024-04-02T13:53:30.174890+0000","last_became_active":"2024-04-02T13:51:05.362631+0000","last_became_peered":"2024-04-02T13:51:05.362631+0000","last_unstale":"2024-04-02T13:53:30.174890+0000","last_undegraded":"2024-04-02T13:53:30.174890+0000","last_fullsized":"2024-04-02T13:53:30.174890+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:23.656269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"93'41","reported_seq":79,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.102154+0000","last_change":"2024-04-02T13:51:00.242820+0000","last_active":"2024-04-02T13:53:30.102154+0000","last_peered":"2024-04-02T13:53:30.102154+0000","last_clean":"2024-04-02T13:53:30.102154+0000","last_became_active":"2024-04-02T13:51:00.242262+0000","last_became_peered":"2024-04-02T13:51:00.242262+0000","last_unstale":"2024-04-02T13:53:30.102154+0000","last_undegraded":"2024-04-02T13:53:30.102154+0000","last_fullsized":"2024-04-02T13:53:30.102154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:12:22.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136066+0000","last_change":"2024-04-02T13:51:01.266548+0000","last_active":"2024-04-02T13:53:30.136066+0000","last_peered":"2024-04-02T13:53:30.136066+0000","last_clean":"2024-04-02T13:53:30.136066+0000","last_became_active":"2024-04-02T13:51:01.259003+0000","last_became_peered":"2024-04-02T13:51:01.259003+0000","last_unstale":"2024-04-02T13:53:30.136066+0000","last_undegraded":"2024-04-02T13:53:30.136066+0000","last_fullsized":"2024-04-02T13:53:30.136066+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:38:53.692231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894871+0000","last_change":"2024-04-02T13:51:05.359994+0000","last_active":"2024-04-02T13:53:29.894871+0000","last_peered":"2024-04-02T13:53:29.894871+0000","last_clean":"2024-04-02T13:53:29.894871+0000","last_became_active":"2024-04-02T13:51:05.359438+0000","last_became_peered":"2024-04-02T13:51:05.359438+0000","last_unstale":"2024-04-02T13:53:29.894871+0000","last_undegraded":"2024-04-02T13:53:29.894871+0000","last_fullsized":"2024-04-02T13:53:29.894871+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:09:43.929446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894377+0000","last_change":"2024-04-02T13:51:00.237889+0000","last_active":"2024-04-02T13:53:29.894377+0000","last_peered":"2024-04-02T13:53:29.894377+0000","last_clean":"2024-04-02T13:53:29.894377+0000","last_became_active":"2024-04-02T13:51:00.237428+0000","last_became_peered":"2024-04-02T13:51:00.237428+0000","last_unstale":"2024-04-02T13:53:29.894377+0000","last_undegraded":"2024-04-02T13:53:29.894377+0000","last_fullsized":"2024-04-02T13:53:29.894377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:21:52.483656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136447+0000","last_change":"2024-04-02T13:51:01.262093+0000","last_active":"2024-04-02T13:53:30.136447+0000","last_peered":"2024-04-02T13:53:30.136447+0000","last_clean":"2024-04-02T13:53:30.136447+0000","last_became_active":"2024-04-02T13:51:01.258357+0000","last_became_peered":"2024-04-02T13:51:01.258357+0000","last_unstale":"2024-04-02T13:53:30.136447+0000","last_undegraded":"2024-04-02T13:53:30.136447+0000","last_fullsized":"2024-04-02T13:53:30.136447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:00:06.317307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174608+0000","last_change":"2024-04-02T13:51:56.483560+0000","last_active":"2024-04-02T13:53:30.174608+0000","last_peered":"2024-04-02T13:53:30.174608+0000","last_clean":"2024-04-02T13:53:30.174608+0000","last_became_active":"2024-04-02T13:51:56.483090+0000","last_became_peered":"2024-04-02T13:51:56.483090+0000","last_unstale":"2024-04-02T13:53:30.174608+0000","last_undegraded":"2024-04-02T13:53:30.174608+0000","last_fullsized":"2024-04-02T13:53:30.174608+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:09:35.640492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135057+0000","last_change":"2024-04-02T13:51:56.663817+0000","last_active":"2024-04-02T13:53:30.135057+0000","last_peered":"2024-04-02T13:53:30.135057+0000","last_clean":"2024-04-02T13:53:30.135057+0000","last_became_active":"2024-04-02T13:51:56.663473+0000","last_became_peered":"2024-04-02T13:51:56.663473+0000","last_unstale":"2024-04-02T13:53:30.135057+0000","last_undegraded":"2024-04-02T13:53:30.135057+0000","last_fullsized":"2024-04-02T13:53:30.135057+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:26:15.566030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135112+0000","last_change":"2024-04-02T13:51:01.255563+0000","last_active":"2024-04-02T13:53:30.135112+0000","last_peered":"2024-04-02T13:53:30.135112+0000","last_clean":"2024-04-02T13:53:30.135112+0000","last_became_active":"2024-04-02T13:51:01.255228+0000","last_became_peered":"2024-04-02T13:51:01.255228+0000","last_unstale":"2024-04-02T13:53:30.135112+0000","last_undegraded":"2024-04-02T13:53:30.135112+0000","last_fullsized":"2024-04-02T13:53:30.135112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:48:28.200776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.175260+0000","last_change":"2024-04-02T13:51:01.258153+0000","last_active":"2024-04-02T13:53:30.175260+0000","last_peered":"2024-04-02T13:53:30.175260+0000","last_clean":"2024-04-02T13:53:30.175260+0000","last_became_active":"2024-04-02T13:51:01.257656+0000","last_became_peered":"2024-04-02T13:51:01.257656+0000","last_unstale":"2024-04-02T13:53:30.175260+0000","last_undegraded":"2024-04-02T13:53:30.175260+0000","last_fullsized":"2024-04-02T13:53:30.175260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:37.887427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894267+0000","last_change":"2024-04-02T13:51:01.254675+0000","last_active":"2024-04-02T13:53:29.894267+0000","last_peered":"2024-04-02T13:53:29.894267+0000","last_clean":"2024-04-02T13:53:29.894267+0000","last_became_active":"2024-04-02T13:51:01.254336+0000","last_became_peered":"2024-04-02T13:51:01.254336+0000","last_unstale":"2024-04-02T13:53:29.894267+0000","last_undegraded":"2024-04-02T13:53:29.894267+0000","last_fullsized":"2024-04-02T13:53:29.894267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:36.171961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174947+0000","last_change":"2024-04-02T13:51:01.259457+0000","last_active":"2024-04-02T13:53:30.174947+0000","last_peered":"2024-04-02T13:53:30.174947+0000","last_clean":"2024-04-02T13:53:30.174947+0000","last_became_active":"2024-04-02T13:51:01.259099+0000","last_became_peered":"2024-04-02T13:51:01.259099+0000","last_unstale":"2024-04-02T13:53:30.174947+0000","last_undegraded":"2024-04-02T13:53:30.174947+0000","last_fullsized":"2024-04-02T13:53:30.174947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:13:06.594132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135619+0000","last_change":"2024-04-02T13:51:01.248177+0000","last_active":"2024-04-02T13:53:30.135619+0000","last_peered":"2024-04-02T13:53:30.135619+0000","last_clean":"2024-04-02T13:53:30.135619+0000","last_became_active":"2024-04-02T13:51:01.247843+0000","last_became_peered":"2024-04-02T13:51:01.247843+0000","last_unstale":"2024-04-02T13:53:30.135619+0000","last_undegraded":"2024-04-02T13:53:30.135619+0000","last_fullsized":"2024-04-02T13:53:30.135619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:38:05.805382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997866+0000","last_change":"2024-04-02T13:51:01.254822+0000","last_active":"2024-04-02T13:53:29.997866+0000","last_peered":"2024-04-02T13:53:29.997866+0000","last_clean":"2024-04-02T13:53:29.997866+0000","last_became_active":"2024-04-02T13:51:01.254451+0000","last_became_peered":"2024-04-02T13:51:01.254451+0000","last_unstale":"2024-04-02T13:53:29.997866+0000","last_undegraded":"2024-04-02T13:53:29.997866+0000","last_fullsized":"2024-04-02T13:53:29.997866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:15:35.790371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895040+0000","last_change":"2024-04-02T13:51:01.265467+0000","last_active":"2024-04-02T13:53:29.895040+0000","last_peered":"2024-04-02T13:53:29.895040+0000","last_clean":"2024-04-02T13:53:29.895040+0000","last_became_active":"2024-04-02T13:51:01.265072+0000","last_became_peered":"2024-04-02T13:51:01.265072+0000","last_unstale":"2024-04-02T13:53:29.895040+0000","last_undegraded":"2024-04-02T13:53:29.895040+0000","last_fullsized":"2024-04-02T13:53:29.895040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:12:07.702222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996951+0000","last_change":"2024-04-02T13:51:01.252844+0000","last_active":"2024-04-02T13:53:29.996951+0000","last_peered":"2024-04-02T13:53:29.996951+0000","last_clean":"2024-04-02T13:53:29.996951+0000","last_became_active":"2024-04-02T13:51:01.252515+0000","last_became_peered":"2024-04-02T13:51:01.252515+0000","last_unstale":"2024-04-02T13:53:29.996951+0000","last_undegraded":"2024-04-02T13:53:29.996951+0000","last_fullsized":"2024-04-02T13:53:29.996951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:20:16.216418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136142+0000","last_change":"2024-04-02T13:51:01.260141+0000","last_active":"2024-04-02T13:53:30.136142+0000","last_peered":"2024-04-02T13:53:30.136142+0000","last_clean":"2024-04-02T13:53:30.136142+0000","last_became_active":"2024-04-02T13:51:01.259530+0000","last_became_peered":"2024-04-02T13:51:01.259530+0000","last_unstale":"2024-04-02T13:53:30.136142+0000","last_undegraded":"2024-04-02T13:53:30.136142+0000","last_fullsized":"2024-04-02T13:53:30.136142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:00:03.579414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894065+0000","last_change":"2024-04-02T13:51:01.259981+0000","last_active":"2024-04-02T13:53:29.894065+0000","last_peered":"2024-04-02T13:53:29.894065+0000","last_clean":"2024-04-02T13:53:29.894065+0000","last_became_active":"2024-04-02T13:51:01.259647+0000","last_became_peered":"2024-04-02T13:51:01.259647+0000","last_unstale":"2024-04-02T13:53:29.894065+0000","last_undegraded":"2024-04-02T13:53:29.894065+0000","last_fullsized":"2024-04-02T13:53:29.894065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:10:22.753767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174911+0000","last_change":"2024-04-02T13:51:01.255342+0000","last_active":"2024-04-02T13:53:30.174911+0000","last_peered":"2024-04-02T13:53:30.174911+0000","last_clean":"2024-04-02T13:53:30.174911+0000","last_became_active":"2024-04-02T13:51:01.254679+0000","last_became_peered":"2024-04-02T13:51:01.254679+0000","last_unstale":"2024-04-02T13:53:30.174911+0000","last_undegraded":"2024-04-02T13:53:30.174911+0000","last_fullsized":"2024-04-02T13:53:30.174911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:25:12.198270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601988,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1355,"num_read_kb":796,"num_write":370,"num_write_kb":1468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1805964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":300,"ondisk_log_size":300,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351019,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1069056,"data_stored":517607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612656,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28804,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716348,"statfs":{"total":95995035648,"available":95965540352,"internally_reserved":0,"allocated":1826816,"data_stored":1232765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776181,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28468,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716684,"statfs":{"total":95995035648,"available":95965884416,"internally_reserved":0,"allocated":1482752,"data_stored":914249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037819,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36780,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708372,"statfs":{"total":95995035648,"available":95957372928,"internally_reserved":0,"allocated":1474560,"data_stored":910531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332160,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36980,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708172,"statfs":{"total":95995035648,"available":95957168128,"internally_reserved":0,"allocated":1679360,"data_stored":1114594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495684,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40536,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704616,"statfs":{"total":95995035648,"available":95953526784,"internally_reserved":0,"allocated":1060864,"data_stored":512832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1134592,"data_stored":582907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1085440,"data_stored":514429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1126400,"data_stored":567237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":502341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920847,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1069056,"data_stored":516515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757322,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28672,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716480,"statfs":{"total":95995035648,"available":95965675520,"internally_reserved":0,"allocated":1691648,"data_stored":1109515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":72263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":202381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T13:53:35.401 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-02T13:53:35.401 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T13:53:35.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.536+0000 7f2d765a8640 1 -- 172.21.15.52:0/3115524356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d7006aaf0 msgr2=0x7f2d7010e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:35.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.536+0000 7f2d765a8640 1 --2- 172.21.15.52:0/3115524356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d7006aaf0 0x7f2d7010e950 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009ca0 tx=0x7f2d6802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.536+0000 7f2d765a8640 1 -- 172.21.15.52:0/3115524356 shutdown_connections 2024-04-02T13:53:35.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.536+0000 7f2d765a8640 1 --2- 172.21.15.52:0/3115524356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d7006aaf0 0x7f2d7010e950 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.536+0000 7f2d765a8640 1 --2- 172.21.15.52:0/3115524356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d7006a150 0x7f2d7006a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.536+0000 7f2d765a8640 1 --2- 172.21.15.52:0/3115524356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d70107b00 0x7f2d70107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.536+0000 7f2d765a8640 1 -- 172.21.15.52:0/3115524356 >> 172.21.15.52:0/3115524356 conn(0x7f2d70076800 msgr2=0x7f2d70076c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:35.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.536+0000 7f2d765a8640 1 -- 172.21.15.52:0/3115524356 shutdown_connections 2024-04-02T13:53:35.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.536+0000 7f2d765a8640 1 -- 172.21.15.52:0/3115524356 wait complete. 2024-04-02T13:53:35.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.537+0000 7f2d765a8640 1 Processor -- start 2024-04-02T13:53:35.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.537+0000 7f2d765a8640 1 -- start start 2024-04-02T13:53:35.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.537+0000 7f2d765a8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d7006a150 0x7f2d70198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:35.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.537+0000 7f2d765a8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d7006aaf0 0x7f2d701988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:35.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.537+0000 7f2d765a8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d70107b00 0x7f2d7019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:35.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.537+0000 7f2d765a8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2d70111ee0 con 0x7f2d7006a150 2024-04-02T13:53:35.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.537+0000 7f2d765a8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2d70111d60 con 0x7f2d70107b00 2024-04-02T13:53:35.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.537+0000 7f2d765a8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2d70112060 con 0x7f2d7006aaf0 2024-04-02T13:53:35.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.537+0000 7f2d6ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d7006a150 0x7f2d70198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:35.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.537+0000 7f2d6ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d7006a150 0x7f2d70198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58442/0 (socket says 172.21.15.52:58442) 2024-04-02T13:53:35.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.537+0000 7f2d6ffff640 1 -- 172.21.15.52:0/1060363430 learned_addr learned my addr 172.21.15.52:0/1060363430 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:35.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.537+0000 7f2d74b45640 1 --2- 172.21.15.52:0/1060363430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d70107b00 0x7f2d7019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:35.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.537+0000 7f2d6f7fe640 1 --2- 172.21.15.52:0/1060363430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d7006aaf0 0x7f2d701988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:35.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.537+0000 7f2d6ffff640 1 -- 172.21.15.52:0/1060363430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d7006aaf0 msgr2=0x7f2d701988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:35.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.537+0000 7f2d6ffff640 1 --2- 172.21.15.52:0/1060363430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d7006aaf0 0x7f2d701988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.537+0000 7f2d6ffff640 1 -- 172.21.15.52:0/1060363430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d70107b00 msgr2=0x7f2d7019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:35.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.537+0000 7f2d6ffff640 1 --2- 172.21.15.52:0/1060363430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d70107b00 0x7f2d7019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.537+0000 7f2d6ffff640 1 -- 172.21.15.52:0/1060363430 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7019d370 con 0x7f2d7006a150 2024-04-02T13:53:35.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.537+0000 7f2d6ffff640 1 --2- 172.21.15.52:0/1060363430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d7006a150 0x7f2d70198380 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000d950 tx=0x7f2d6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:35.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.538+0000 7f2d6d7fa640 1 -- 172.21.15.52:0/1060363430 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60017030 con 0x7f2d7006a150 2024-04-02T13:53:35.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.538+0000 7f2d765a8640 1 -- 172.21.15.52:0/1060363430 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7019d660 con 0x7f2d7006a150 2024-04-02T13:53:35.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.538+0000 7f2d6d7fa640 1 -- 172.21.15.52:0/1060363430 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d60004020 con 0x7f2d7006a150 2024-04-02T13:53:35.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.538+0000 7f2d6d7fa640 1 -- 172.21.15.52:0/1060363430 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d600047a0 con 0x7f2d7006a150 2024-04-02T13:53:35.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.538+0000 7f2d765a8640 1 -- 172.21.15.52:0/1060363430 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d701a4ef0 con 0x7f2d7006a150 2024-04-02T13:53:35.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.539+0000 7f2d765a8640 1 -- 172.21.15.52:0/1060363430 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d701031c0 con 0x7f2d7006a150 2024-04-02T13:53:35.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.541+0000 7f2d6d7fa640 1 -- 172.21.15.52:0/1060363430 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f2d600041d0 con 0x7f2d7006a150 2024-04-02T13:53:35.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.541+0000 7f2d6d7fa640 1 --2- 172.21.15.52:0/1060363430 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2d580a3e40 0x7f2d580a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:35.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.541+0000 7f2d6f7fe640 1 --2- 172.21.15.52:0/1060363430 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2d580a3e40 0x7f2d580a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:35.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.541+0000 7f2d6d7fa640 1 -- 172.21.15.52:0/1060363430 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f2d600d0dd0 con 0x7f2d7006a150 2024-04-02T13:53:35.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.542+0000 7f2d6f7fe640 1 --2- 172.21.15.52:0/1060363430 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2d580a3e40 0x7f2d580a6300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2d64005fd0 tx=0x7f2d64005850 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:35.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.545+0000 7f2d6d7fa640 1 -- 172.21.15.52:0/1060363430 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2d6009cee0 con 0x7f2d7006a150 2024-04-02T13:53:35.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.728+0000 7f2d765a8640 1 -- 172.21.15.52:0/1060363430 --> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2d70061f40 con 0x7f2d580a3e40 2024-04-02T13:53:35.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.737+0000 7f2d6d7fa640 1 -- 172.21.15.52:0/1060363430 <== mgr.25160 v2:172.21.15.52:6800/1019901931 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339431 (secure 0 0 0) 0x7f2d70061f40 con 0x7f2d580a3e40 2024-04-02T13:53:35.748 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:53:35.750 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-04-02T13:53:35.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.741+0000 7f2d765a8640 1 -- 172.21.15.52:0/1060363430 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2d580a3e40 msgr2=0x7f2d580a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:35.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.741+0000 7f2d765a8640 1 --2- 172.21.15.52:0/1060363430 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2d580a3e40 0x7f2d580a6300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2d64005fd0 tx=0x7f2d64005850 comp rx=0 tx=0).stop 2024-04-02T13:53:35.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.741+0000 7f2d765a8640 1 -- 172.21.15.52:0/1060363430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d7006a150 msgr2=0x7f2d70198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:35.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.741+0000 7f2d765a8640 1 --2- 172.21.15.52:0/1060363430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d7006a150 0x7f2d70198380 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000d950 tx=0x7f2d6000de10 comp rx=0 tx=0).stop 2024-04-02T13:53:35.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.741+0000 7f2d765a8640 1 -- 172.21.15.52:0/1060363430 shutdown_connections 2024-04-02T13:53:35.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.741+0000 7f2d765a8640 1 --2- 172.21.15.52:0/1060363430 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2d580a3e40 0x7f2d580a6300 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.741+0000 7f2d765a8640 1 --2- 172.21.15.52:0/1060363430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d70107b00 0x7f2d7019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.741+0000 7f2d765a8640 1 --2- 172.21.15.52:0/1060363430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d7006aaf0 0x7f2d701988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.741+0000 7f2d765a8640 1 --2- 172.21.15.52:0/1060363430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d7006a150 0x7f2d70198380 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.741+0000 7f2d765a8640 1 -- 172.21.15.52:0/1060363430 >> 172.21.15.52:0/1060363430 conn(0x7f2d70076800 msgr2=0x7f2d700fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:35.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.741+0000 7f2d765a8640 1 -- 172.21.15.52:0/1060363430 shutdown_connections 2024-04-02T13:53:35.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.741+0000 7f2d765a8640 1 -- 172.21.15.52:0/1060363430 wait complete. 2024-04-02T13:53:35.755 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-02T13:53:33.911035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192590,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1491,"num_read_kb":915,"num_write":553,"num_write_kb":4408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":939,"ondisk_log_size":939,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":414500,"kb_used_data":15388,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124527324,"statfs":{"total":1151940427776,"available":1151515979776,"internally_reserved":0,"allocated":15757312,"data_stored":8995522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":408532282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.991083"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.175053+0000","last_change":"2024-04-02T13:51:01.253563+0000","last_active":"2024-04-02T13:53:30.175053+0000","last_peered":"2024-04-02T13:53:30.175053+0000","last_clean":"2024-04-02T13:53:30.175053+0000","last_became_active":"2024-04-02T13:51:01.253185+0000","last_became_peered":"2024-04-02T13:51:01.253185+0000","last_unstale":"2024-04-02T13:53:30.175053+0000","last_undegraded":"2024-04-02T13:53:30.175053+0000","last_fullsized":"2024-04-02T13:53:30.175053+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:51:05.301880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.175203+0000","last_change":"2024-04-02T13:51:01.256406+0000","last_active":"2024-04-02T13:53:30.175203+0000","last_peered":"2024-04-02T13:53:30.175203+0000","last_clean":"2024-04-02T13:53:30.175203+0000","last_became_active":"2024-04-02T13:51:01.256068+0000","last_became_peered":"2024-04-02T13:51:01.256068+0000","last_unstale":"2024-04-02T13:53:30.175203+0000","last_undegraded":"2024-04-02T13:53:30.175203+0000","last_fullsized":"2024-04-02T13:53:30.175203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:56:51.184932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174787+0000","last_change":"2024-04-02T13:51:01.265925+0000","last_active":"2024-04-02T13:53:30.174787+0000","last_peered":"2024-04-02T13:53:30.174787+0000","last_clean":"2024-04-02T13:53:30.174787+0000","last_became_active":"2024-04-02T13:51:01.265587+0000","last_became_peered":"2024-04-02T13:51:01.265587+0000","last_unstale":"2024-04-02T13:53:30.174787+0000","last_undegraded":"2024-04-02T13:53:30.174787+0000","last_fullsized":"2024-04-02T13:53:30.174787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:40:59.426886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.994837+0000","last_change":"2024-04-02T13:51:56.482512+0000","last_active":"2024-04-02T13:53:29.994837+0000","last_peered":"2024-04-02T13:53:29.994837+0000","last_clean":"2024-04-02T13:53:29.994837+0000","last_became_active":"2024-04-02T13:51:56.481439+0000","last_became_peered":"2024-04-02T13:51:56.481439+0000","last_unstale":"2024-04-02T13:53:29.994837+0000","last_undegraded":"2024-04-02T13:53:29.994837+0000","last_fullsized":"2024-04-02T13:53:29.994837+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:23:16.969641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997088+0000","last_change":"2024-04-02T13:51:01.250934+0000","last_active":"2024-04-02T13:53:29.997088+0000","last_peered":"2024-04-02T13:53:29.997088+0000","last_clean":"2024-04-02T13:53:29.997088+0000","last_became_active":"2024-04-02T13:51:01.250546+0000","last_became_peered":"2024-04-02T13:51:01.250546+0000","last_unstale":"2024-04-02T13:53:29.997088+0000","last_undegraded":"2024-04-02T13:53:29.997088+0000","last_fullsized":"2024-04-02T13:53:29.997088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:34:17.839207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.003203+0000","last_change":"2024-04-02T13:51:01.256160+0000","last_active":"2024-04-02T13:53:30.003203+0000","last_peered":"2024-04-02T13:53:30.003203+0000","last_clean":"2024-04-02T13:53:30.003203+0000","last_became_active":"2024-04-02T13:51:01.255776+0000","last_became_peered":"2024-04-02T13:51:01.255776+0000","last_unstale":"2024-04-02T13:53:30.003203+0000","last_undegraded":"2024-04-02T13:53:30.003203+0000","last_fullsized":"2024-04-02T13:53:30.003203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:53:16.724662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135704+0000","last_change":"2024-04-02T13:51:01.263049+0000","last_active":"2024-04-02T13:53:30.135704+0000","last_peered":"2024-04-02T13:53:30.135704+0000","last_clean":"2024-04-02T13:53:30.135704+0000","last_became_active":"2024-04-02T13:51:01.262660+0000","last_became_peered":"2024-04-02T13:51:01.262660+0000","last_unstale":"2024-04-02T13:53:30.135704+0000","last_undegraded":"2024-04-02T13:53:30.135704+0000","last_fullsized":"2024-04-02T13:53:30.135704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:15:32.069188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996829+0000","last_change":"2024-04-02T13:51:01.263064+0000","last_active":"2024-04-02T13:53:29.996829+0000","last_peered":"2024-04-02T13:53:29.996829+0000","last_clean":"2024-04-02T13:53:29.996829+0000","last_became_active":"2024-04-02T13:51:01.262706+0000","last_became_peered":"2024-04-02T13:51:01.262706+0000","last_unstale":"2024-04-02T13:53:29.996829+0000","last_undegraded":"2024-04-02T13:53:29.996829+0000","last_fullsized":"2024-04-02T13:53:29.996829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:35:46.680723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136166+0000","last_change":"2024-04-02T13:51:01.257347+0000","last_active":"2024-04-02T13:53:30.136166+0000","last_peered":"2024-04-02T13:53:30.136166+0000","last_clean":"2024-04-02T13:53:30.136166+0000","last_became_active":"2024-04-02T13:51:01.257001+0000","last_became_peered":"2024-04-02T13:51:01.257001+0000","last_unstale":"2024-04-02T13:53:30.136166+0000","last_undegraded":"2024-04-02T13:53:30.136166+0000","last_fullsized":"2024-04-02T13:53:30.136166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:31:22.628436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.003671+0000","last_change":"2024-04-02T13:51:01.260533+0000","last_active":"2024-04-02T13:53:30.003671+0000","last_peered":"2024-04-02T13:53:30.003671+0000","last_clean":"2024-04-02T13:53:30.003671+0000","last_became_active":"2024-04-02T13:51:01.260203+0000","last_became_peered":"2024-04-02T13:51:01.260203+0000","last_unstale":"2024-04-02T13:53:30.003671+0000","last_undegraded":"2024-04-02T13:53:30.003671+0000","last_fullsized":"2024-04-02T13:53:30.003671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:00:06.008243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136480+0000","last_change":"2024-04-02T13:51:01.267733+0000","last_active":"2024-04-02T13:53:30.136480+0000","last_peered":"2024-04-02T13:53:30.136480+0000","last_clean":"2024-04-02T13:53:30.136480+0000","last_became_active":"2024-04-02T13:51:01.266642+0000","last_became_peered":"2024-04-02T13:51:01.266642+0000","last_unstale":"2024-04-02T13:53:30.136480+0000","last_undegraded":"2024-04-02T13:53:30.136480+0000","last_fullsized":"2024-04-02T13:53:30.136480+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:39:25.696112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898481+0000","last_change":"2024-04-02T13:51:01.255933+0000","last_active":"2024-04-02T13:53:29.898481+0000","last_peered":"2024-04-02T13:53:29.898481+0000","last_clean":"2024-04-02T13:53:29.898481+0000","last_became_active":"2024-04-02T13:51:01.255558+0000","last_became_peered":"2024-04-02T13:51:01.255558+0000","last_unstale":"2024-04-02T13:53:29.898481+0000","last_undegraded":"2024-04-02T13:53:29.898481+0000","last_fullsized":"2024-04-02T13:53:29.898481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:13:11.537415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894118+0000","last_change":"2024-04-02T13:51:01.256242+0000","last_active":"2024-04-02T13:53:29.894118+0000","last_peered":"2024-04-02T13:53:29.894118+0000","last_clean":"2024-04-02T13:53:29.894118+0000","last_became_active":"2024-04-02T13:51:01.255830+0000","last_became_peered":"2024-04-02T13:51:01.255830+0000","last_unstale":"2024-04-02T13:53:29.894118+0000","last_undegraded":"2024-04-02T13:53:29.894118+0000","last_fullsized":"2024-04-02T13:53:29.894118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:39.130894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174674+0000","last_change":"2024-04-02T13:51:01.259738+0000","last_active":"2024-04-02T13:53:30.174674+0000","last_peered":"2024-04-02T13:53:30.174674+0000","last_clean":"2024-04-02T13:53:30.174674+0000","last_became_active":"2024-04-02T13:51:01.258630+0000","last_became_peered":"2024-04-02T13:51:01.258630+0000","last_unstale":"2024-04-02T13:53:30.174674+0000","last_undegraded":"2024-04-02T13:53:30.174674+0000","last_fullsized":"2024-04-02T13:53:30.174674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:12:05.401171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.175018+0000","last_change":"2024-04-02T13:51:01.257684+0000","last_active":"2024-04-02T13:53:30.175018+0000","last_peered":"2024-04-02T13:53:30.175018+0000","last_clean":"2024-04-02T13:53:30.175018+0000","last_became_active":"2024-04-02T13:51:01.257267+0000","last_became_peered":"2024-04-02T13:51:01.257267+0000","last_unstale":"2024-04-02T13:53:30.175018+0000","last_undegraded":"2024-04-02T13:53:30.175018+0000","last_fullsized":"2024-04-02T13:53:30.175018+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:43:45.728233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135320+0000","last_change":"2024-04-02T13:51:01.261720+0000","last_active":"2024-04-02T13:53:30.135320+0000","last_peered":"2024-04-02T13:53:30.135320+0000","last_clean":"2024-04-02T13:53:30.135320+0000","last_became_active":"2024-04-02T13:51:01.261407+0000","last_became_peered":"2024-04-02T13:51:01.261407+0000","last_unstale":"2024-04-02T13:53:30.135320+0000","last_undegraded":"2024-04-02T13:53:30.135320+0000","last_fullsized":"2024-04-02T13:53:30.135320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:54:58.807606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174875+0000","last_change":"2024-04-02T13:51:01.254488+0000","last_active":"2024-04-02T13:53:30.174875+0000","last_peered":"2024-04-02T13:53:30.174875+0000","last_clean":"2024-04-02T13:53:30.174875+0000","last_became_active":"2024-04-02T13:51:01.254139+0000","last_became_peered":"2024-04-02T13:51:01.254139+0000","last_unstale":"2024-04-02T13:53:30.174875+0000","last_undegraded":"2024-04-02T13:53:30.174875+0000","last_fullsized":"2024-04-02T13:53:30.174875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:02:41.075727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.995012+0000","last_change":"2024-04-02T13:51:01.256107+0000","last_active":"2024-04-02T13:53:29.995012+0000","last_peered":"2024-04-02T13:53:29.995012+0000","last_clean":"2024-04-02T13:53:29.995012+0000","last_became_active":"2024-04-02T13:51:01.255754+0000","last_became_peered":"2024-04-02T13:51:01.255754+0000","last_unstale":"2024-04-02T13:53:29.995012+0000","last_undegraded":"2024-04-02T13:53:29.995012+0000","last_fullsized":"2024-04-02T13:53:29.995012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:36:59.793519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135517+0000","last_change":"2024-04-02T13:51:01.247550+0000","last_active":"2024-04-02T13:53:30.135517+0000","last_peered":"2024-04-02T13:53:30.135517+0000","last_clean":"2024-04-02T13:53:30.135517+0000","last_became_active":"2024-04-02T13:51:01.247236+0000","last_became_peered":"2024-04-02T13:51:01.247236+0000","last_unstale":"2024-04-02T13:53:30.135517+0000","last_undegraded":"2024-04-02T13:53:30.135517+0000","last_fullsized":"2024-04-02T13:53:30.135517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:16:02.940700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135216+0000","last_change":"2024-04-02T13:51:56.663779+0000","last_active":"2024-04-02T13:53:30.135216+0000","last_peered":"2024-04-02T13:53:30.135216+0000","last_clean":"2024-04-02T13:53:30.135216+0000","last_became_active":"2024-04-02T13:51:56.663327+0000","last_became_peered":"2024-04-02T13:51:56.663327+0000","last_unstale":"2024-04-02T13:53:30.135216+0000","last_undegraded":"2024-04-02T13:53:30.135216+0000","last_fullsized":"2024-04-02T13:53:30.135216+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:35:34.186278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"93'14","reported_seq":167,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.137322+0000","last_change":"2024-04-02T13:51:00.243244+0000","last_active":"2024-04-02T13:53:30.137322+0000","last_peered":"2024-04-02T13:53:30.137322+0000","last_clean":"2024-04-02T13:53:30.137322+0000","last_became_active":"2024-04-02T13:51:00.242892+0000","last_became_peered":"2024-04-02T13:51:00.242892+0000","last_unstale":"2024-04-02T13:53:30.137322+0000","last_undegraded":"2024-04-02T13:53:30.137322+0000","last_fullsized":"2024-04-02T13:53:30.137322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:49:23.338215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":123,"num_read_kb":123,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997790+0000","last_change":"2024-04-02T13:51:01.264640+0000","last_active":"2024-04-02T13:53:29.997790+0000","last_peered":"2024-04-02T13:53:29.997790+0000","last_clean":"2024-04-02T13:53:29.997790+0000","last_became_active":"2024-04-02T13:51:01.264161+0000","last_became_peered":"2024-04-02T13:51:01.264161+0000","last_unstale":"2024-04-02T13:53:29.997790+0000","last_undegraded":"2024-04-02T13:53:29.997790+0000","last_fullsized":"2024-04-02T13:53:29.997790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:30:47.245647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898007+0000","last_change":"2024-04-02T13:51:05.362888+0000","last_active":"2024-04-02T13:53:29.898007+0000","last_peered":"2024-04-02T13:53:29.898007+0000","last_clean":"2024-04-02T13:53:29.898007+0000","last_became_active":"2024-04-02T13:51:05.362478+0000","last_became_peered":"2024-04-02T13:51:05.362478+0000","last_unstale":"2024-04-02T13:53:29.898007+0000","last_undegraded":"2024-04-02T13:53:29.898007+0000","last_fullsized":"2024-04-02T13:53:29.898007+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:05:54.304458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997831+0000","last_change":"2024-04-02T13:51:00.247800+0000","last_active":"2024-04-02T13:53:29.997831+0000","last_peered":"2024-04-02T13:53:29.997831+0000","last_clean":"2024-04-02T13:53:29.997831+0000","last_became_active":"2024-04-02T13:51:00.247485+0000","last_became_peered":"2024-04-02T13:51:00.247485+0000","last_unstale":"2024-04-02T13:53:29.997831+0000","last_undegraded":"2024-04-02T13:53:29.997831+0000","last_fullsized":"2024-04-02T13:53:29.997831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:17:24.767558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.175194+0000","last_change":"2024-04-02T13:51:01.261416+0000","last_active":"2024-04-02T13:53:30.175194+0000","last_peered":"2024-04-02T13:53:30.175194+0000","last_clean":"2024-04-02T13:53:30.175194+0000","last_became_active":"2024-04-02T13:51:01.261099+0000","last_became_peered":"2024-04-02T13:51:01.261099+0000","last_unstale":"2024-04-02T13:53:30.175194+0000","last_undegraded":"2024-04-02T13:53:30.175194+0000","last_fullsized":"2024-04-02T13:53:30.175194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:10:04.999402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.897988+0000","last_change":"2024-04-02T13:51:05.361747+0000","last_active":"2024-04-02T13:53:29.897988+0000","last_peered":"2024-04-02T13:53:29.897988+0000","last_clean":"2024-04-02T13:53:29.897988+0000","last_became_active":"2024-04-02T13:51:05.361098+0000","last_became_peered":"2024-04-02T13:51:05.361098+0000","last_unstale":"2024-04-02T13:53:29.897988+0000","last_undegraded":"2024-04-02T13:53:29.897988+0000","last_fullsized":"2024-04-02T13:53:29.897988+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:45:35.048870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997422+0000","last_change":"2024-04-02T13:51:00.244048+0000","last_active":"2024-04-02T13:53:29.997422+0000","last_peered":"2024-04-02T13:53:29.997422+0000","last_clean":"2024-04-02T13:53:29.997422+0000","last_became_active":"2024-04-02T13:51:00.243510+0000","last_became_peered":"2024-04-02T13:51:00.243510+0000","last_unstale":"2024-04-02T13:53:29.997422+0000","last_undegraded":"2024-04-02T13:53:29.997422+0000","last_fullsized":"2024-04-02T13:53:29.997422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:53:48.364137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136246+0000","last_change":"2024-04-02T13:51:01.264711+0000","last_active":"2024-04-02T13:53:30.136246+0000","last_peered":"2024-04-02T13:53:30.136246+0000","last_clean":"2024-04-02T13:53:30.136246+0000","last_became_active":"2024-04-02T13:51:01.264289+0000","last_became_peered":"2024-04-02T13:51:01.264289+0000","last_unstale":"2024-04-02T13:53:30.136246+0000","last_undegraded":"2024-04-02T13:53:30.136246+0000","last_fullsized":"2024-04-02T13:53:30.136246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:34:41.590071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.897930+0000","last_change":"2024-04-02T13:51:05.363121+0000","last_active":"2024-04-02T13:53:29.897930+0000","last_peered":"2024-04-02T13:53:29.897930+0000","last_clean":"2024-04-02T13:53:29.897930+0000","last_became_active":"2024-04-02T13:51:05.362642+0000","last_became_peered":"2024-04-02T13:51:05.362642+0000","last_unstale":"2024-04-02T13:53:29.897930+0000","last_undegraded":"2024-04-02T13:53:29.897930+0000","last_fullsized":"2024-04-02T13:53:29.897930+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:55:40.848731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997155+0000","last_change":"2024-04-02T13:51:00.236519+0000","last_active":"2024-04-02T13:53:29.997155+0000","last_peered":"2024-04-02T13:53:29.997155+0000","last_clean":"2024-04-02T13:53:29.997155+0000","last_became_active":"2024-04-02T13:51:00.235985+0000","last_became_peered":"2024-04-02T13:51:00.235985+0000","last_unstale":"2024-04-02T13:53:29.997155+0000","last_undegraded":"2024-04-02T13:53:29.997155+0000","last_fullsized":"2024-04-02T13:53:29.997155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:08.321765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895438+0000","last_change":"2024-04-02T13:51:01.260333+0000","last_active":"2024-04-02T13:53:29.895438+0000","last_peered":"2024-04-02T13:53:29.895438+0000","last_clean":"2024-04-02T13:53:29.895438+0000","last_became_active":"2024-04-02T13:51:01.259995+0000","last_became_peered":"2024-04-02T13:51:01.259995+0000","last_unstale":"2024-04-02T13:53:29.895438+0000","last_undegraded":"2024-04-02T13:53:29.895438+0000","last_fullsized":"2024-04-02T13:53:29.895438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:03:58.088653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135314+0000","last_change":"2024-04-02T13:51:05.359281+0000","last_active":"2024-04-02T13:53:30.135314+0000","last_peered":"2024-04-02T13:53:30.135314+0000","last_clean":"2024-04-02T13:53:30.135314+0000","last_became_active":"2024-04-02T13:51:05.358831+0000","last_became_peered":"2024-04-02T13:51:05.358831+0000","last_unstale":"2024-04-02T13:53:30.135314+0000","last_undegraded":"2024-04-02T13:53:30.135314+0000","last_fullsized":"2024-04-02T13:53:30.135314+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:48.461298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'7","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.508533+0000","last_change":"2024-04-02T13:51:00.241707+0000","last_active":"2024-04-02T13:53:30.508533+0000","last_peered":"2024-04-02T13:53:30.508533+0000","last_clean":"2024-04-02T13:53:30.508533+0000","last_became_active":"2024-04-02T13:51:00.241225+0000","last_became_peered":"2024-04-02T13:51:00.241225+0000","last_unstale":"2024-04-02T13:53:30.508533+0000","last_undegraded":"2024-04-02T13:53:30.508533+0000","last_fullsized":"2024-04-02T13:53:30.508533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:59:44.537503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1404,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":125,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997009+0000","last_change":"2024-04-02T13:51:01.252942+0000","last_active":"2024-04-02T13:53:29.997009+0000","last_peered":"2024-04-02T13:53:29.997009+0000","last_clean":"2024-04-02T13:53:29.997009+0000","last_became_active":"2024-04-02T13:51:01.252631+0000","last_became_peered":"2024-04-02T13:51:01.252631+0000","last_unstale":"2024-04-02T13:53:29.997009+0000","last_undegraded":"2024-04-02T13:53:29.997009+0000","last_fullsized":"2024-04-02T13:53:29.997009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:43:24.725688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135372+0000","last_change":"2024-04-02T13:51:05.359208+0000","last_active":"2024-04-02T13:53:30.135372+0000","last_peered":"2024-04-02T13:53:30.135372+0000","last_clean":"2024-04-02T13:53:30.135372+0000","last_became_active":"2024-04-02T13:51:05.358738+0000","last_became_peered":"2024-04-02T13:51:05.358738+0000","last_unstale":"2024-04-02T13:53:30.135372+0000","last_undegraded":"2024-04-02T13:53:30.135372+0000","last_fullsized":"2024-04-02T13:53:30.135372+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:47:46.034562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.003266+0000","last_change":"2024-04-02T13:51:00.236413+0000","last_active":"2024-04-02T13:53:30.003266+0000","last_peered":"2024-04-02T13:53:30.003266+0000","last_clean":"2024-04-02T13:53:30.003266+0000","last_became_active":"2024-04-02T13:51:00.236046+0000","last_became_peered":"2024-04-02T13:51:00.236046+0000","last_unstale":"2024-04-02T13:53:30.003266+0000","last_undegraded":"2024-04-02T13:53:30.003266+0000","last_fullsized":"2024-04-02T13:53:30.003266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:25:15.000726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895344+0000","last_change":"2024-04-02T13:51:01.265471+0000","last_active":"2024-04-02T13:53:29.895344+0000","last_peered":"2024-04-02T13:53:29.895344+0000","last_clean":"2024-04-02T13:53:29.895344+0000","last_became_active":"2024-04-02T13:51:01.265114+0000","last_became_peered":"2024-04-02T13:51:01.265114+0000","last_unstale":"2024-04-02T13:53:29.895344+0000","last_undegraded":"2024-04-02T13:53:29.895344+0000","last_fullsized":"2024-04-02T13:53:29.895344+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:01:01.675686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174824+0000","last_change":"2024-04-02T13:51:56.664063+0000","last_active":"2024-04-02T13:53:30.174824+0000","last_peered":"2024-04-02T13:53:30.174824+0000","last_clean":"2024-04-02T13:53:30.174824+0000","last_became_active":"2024-04-02T13:51:56.663590+0000","last_became_peered":"2024-04-02T13:51:56.663590+0000","last_unstale":"2024-04-02T13:53:30.174824+0000","last_undegraded":"2024-04-02T13:53:30.174824+0000","last_fullsized":"2024-04-02T13:53:30.174824+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:43:51.735851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.893833+0000","last_change":"2024-04-02T13:51:00.237204+0000","last_active":"2024-04-02T13:53:29.893833+0000","last_peered":"2024-04-02T13:53:29.893833+0000","last_clean":"2024-04-02T13:53:29.893833+0000","last_became_active":"2024-04-02T13:51:00.236707+0000","last_became_peered":"2024-04-02T13:51:00.236707+0000","last_unstale":"2024-04-02T13:53:29.893833+0000","last_undegraded":"2024-04-02T13:53:29.893833+0000","last_fullsized":"2024-04-02T13:53:29.893833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:00:17.771428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997017+0000","last_change":"2024-04-02T13:51:01.250152+0000","last_active":"2024-04-02T13:53:29.997017+0000","last_peered":"2024-04-02T13:53:29.997017+0000","last_clean":"2024-04-02T13:53:29.997017+0000","last_became_active":"2024-04-02T13:51:01.249817+0000","last_became_peered":"2024-04-02T13:51:01.249817+0000","last_unstale":"2024-04-02T13:53:29.997017+0000","last_undegraded":"2024-04-02T13:53:29.997017+0000","last_fullsized":"2024-04-02T13:53:29.997017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:13:31.719380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.175093+0000","last_change":"2024-04-02T13:51:05.366551+0000","last_active":"2024-04-02T13:53:30.175093+0000","last_peered":"2024-04-02T13:53:30.175093+0000","last_clean":"2024-04-02T13:53:30.175093+0000","last_became_active":"2024-04-02T13:51:05.366175+0000","last_became_peered":"2024-04-02T13:51:05.366175+0000","last_unstale":"2024-04-02T13:53:30.175093+0000","last_undegraded":"2024-04-02T13:53:30.175093+0000","last_fullsized":"2024-04-02T13:53:30.175093+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:47:49.822918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895052+0000","last_change":"2024-04-02T13:51:00.239521+0000","last_active":"2024-04-02T13:53:29.895052+0000","last_peered":"2024-04-02T13:53:29.895052+0000","last_clean":"2024-04-02T13:53:29.895052+0000","last_became_active":"2024-04-02T13:51:00.238608+0000","last_became_peered":"2024-04-02T13:51:00.238608+0000","last_unstale":"2024-04-02T13:53:29.895052+0000","last_undegraded":"2024-04-02T13:53:29.895052+0000","last_fullsized":"2024-04-02T13:53:29.895052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:31:00.983006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.994418+0000","last_change":"2024-04-02T13:51:01.255021+0000","last_active":"2024-04-02T13:53:29.994418+0000","last_peered":"2024-04-02T13:53:29.994418+0000","last_clean":"2024-04-02T13:53:29.994418+0000","last_became_active":"2024-04-02T13:51:01.254630+0000","last_became_peered":"2024-04-02T13:51:01.254630+0000","last_unstale":"2024-04-02T13:53:29.994418+0000","last_undegraded":"2024-04-02T13:53:29.994418+0000","last_fullsized":"2024-04-02T13:53:29.994418+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:36:08.773431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996891+0000","last_change":"2024-04-02T13:51:05.362004+0000","last_active":"2024-04-02T13:53:29.996891+0000","last_peered":"2024-04-02T13:53:29.996891+0000","last_clean":"2024-04-02T13:53:29.996891+0000","last_became_active":"2024-04-02T13:51:05.361538+0000","last_became_peered":"2024-04-02T13:51:05.361538+0000","last_unstale":"2024-04-02T13:53:29.996891+0000","last_undegraded":"2024-04-02T13:53:29.996891+0000","last_fullsized":"2024-04-02T13:53:29.996891+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:27:21.728695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'6","reported_seq":280,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.507365+0000","last_change":"2024-04-02T13:51:00.234814+0000","last_active":"2024-04-02T13:53:30.507365+0000","last_peered":"2024-04-02T13:53:30.507365+0000","last_clean":"2024-04-02T13:53:30.507365+0000","last_became_active":"2024-04-02T13:51:00.234481+0000","last_became_peered":"2024-04-02T13:51:00.234481+0000","last_unstale":"2024-04-02T13:53:30.507365+0000","last_undegraded":"2024-04-02T13:53:30.507365+0000","last_fullsized":"2024-04-02T13:53:30.507365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:45:34.901226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":244,"num_read_kb":244,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135656+0000","last_change":"2024-04-02T13:51:01.264093+0000","last_active":"2024-04-02T13:53:30.135656+0000","last_peered":"2024-04-02T13:53:30.135656+0000","last_clean":"2024-04-02T13:53:30.135656+0000","last_became_active":"2024-04-02T13:51:01.263754+0000","last_became_peered":"2024-04-02T13:51:01.263754+0000","last_unstale":"2024-04-02T13:53:30.135656+0000","last_undegraded":"2024-04-02T13:53:30.135656+0000","last_fullsized":"2024-04-02T13:53:30.135656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:49:18.630195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.994363+0000","last_change":"2024-04-02T13:51:05.370930+0000","last_active":"2024-04-02T13:53:29.994363+0000","last_peered":"2024-04-02T13:53:29.994363+0000","last_clean":"2024-04-02T13:53:29.994363+0000","last_became_active":"2024-04-02T13:51:05.370443+0000","last_became_peered":"2024-04-02T13:51:05.370443+0000","last_unstale":"2024-04-02T13:53:29.994363+0000","last_undegraded":"2024-04-02T13:53:29.994363+0000","last_fullsized":"2024-04-02T13:53:29.994363+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:24:27.055494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136337+0000","last_change":"2024-04-02T13:51:00.241206+0000","last_active":"2024-04-02T13:53:30.136337+0000","last_peered":"2024-04-02T13:53:30.136337+0000","last_clean":"2024-04-02T13:53:30.136337+0000","last_became_active":"2024-04-02T13:51:00.240832+0000","last_became_peered":"2024-04-02T13:51:00.240832+0000","last_unstale":"2024-04-02T13:53:30.136337+0000","last_undegraded":"2024-04-02T13:53:30.136337+0000","last_fullsized":"2024-04-02T13:53:30.136337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:44:11.626435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.175384+0000","last_change":"2024-04-02T13:51:01.259315+0000","last_active":"2024-04-02T13:53:30.175384+0000","last_peered":"2024-04-02T13:53:30.175384+0000","last_clean":"2024-04-02T13:53:30.175384+0000","last_became_active":"2024-04-02T13:51:01.258951+0000","last_became_peered":"2024-04-02T13:51:01.258951+0000","last_unstale":"2024-04-02T13:53:30.175384+0000","last_undegraded":"2024-04-02T13:53:30.175384+0000","last_fullsized":"2024-04-02T13:53:30.175384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:59:43.714044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898359+0000","last_change":"2024-04-02T13:51:56.665826+0000","last_active":"2024-04-02T13:53:29.898359+0000","last_peered":"2024-04-02T13:53:29.898359+0000","last_clean":"2024-04-02T13:53:29.898359+0000","last_became_active":"2024-04-02T13:51:56.665213+0000","last_became_peered":"2024-04-02T13:51:56.665213+0000","last_unstale":"2024-04-02T13:53:29.898359+0000","last_undegraded":"2024-04-02T13:53:29.898359+0000","last_fullsized":"2024-04-02T13:53:29.898359+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:45:46.625063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895401+0000","last_change":"2024-04-02T13:51:00.230938+0000","last_active":"2024-04-02T13:53:29.895401+0000","last_peered":"2024-04-02T13:53:29.895401+0000","last_clean":"2024-04-02T13:53:29.895401+0000","last_became_active":"2024-04-02T13:51:00.230585+0000","last_became_peered":"2024-04-02T13:51:00.230585+0000","last_unstale":"2024-04-02T13:53:29.895401+0000","last_undegraded":"2024-04-02T13:53:29.895401+0000","last_fullsized":"2024-04-02T13:53:29.895401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:51:37.463876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136591+0000","last_change":"2024-04-02T13:51:01.264216+0000","last_active":"2024-04-02T13:53:30.136591+0000","last_peered":"2024-04-02T13:53:30.136591+0000","last_clean":"2024-04-02T13:53:30.136591+0000","last_became_active":"2024-04-02T13:51:01.263824+0000","last_became_peered":"2024-04-02T13:51:01.263824+0000","last_unstale":"2024-04-02T13:53:30.136591+0000","last_undegraded":"2024-04-02T13:53:30.136591+0000","last_fullsized":"2024-04-02T13:53:30.136591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:16:26.391163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.994569+0000","last_change":"2024-04-02T13:51:05.366002+0000","last_active":"2024-04-02T13:53:29.994569+0000","last_peered":"2024-04-02T13:53:29.994569+0000","last_clean":"2024-04-02T13:53:29.994569+0000","last_became_active":"2024-04-02T13:51:05.365547+0000","last_became_peered":"2024-04-02T13:51:05.365547+0000","last_unstale":"2024-04-02T13:53:29.994569+0000","last_undegraded":"2024-04-02T13:53:29.994569+0000","last_fullsized":"2024-04-02T13:53:29.994569+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:31:38.990617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'43","reported_seq":327,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.507596+0000","last_change":"2024-04-02T13:51:00.233925+0000","last_active":"2024-04-02T13:53:30.507596+0000","last_peered":"2024-04-02T13:53:30.507596+0000","last_clean":"2024-04-02T13:53:30.507596+0000","last_became_active":"2024-04-02T13:51:00.233574+0000","last_became_peered":"2024-04-02T13:51:00.233574+0000","last_unstale":"2024-04-02T13:53:30.507596+0000","last_undegraded":"2024-04-02T13:53:30.507596+0000","last_fullsized":"2024-04-02T13:53:30.507596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:33:32.758874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69045,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":254,"num_read_kb":196,"num_write":45,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997226+0000","last_change":"2024-04-02T13:51:01.258825+0000","last_active":"2024-04-02T13:53:29.997226+0000","last_peered":"2024-04-02T13:53:29.997226+0000","last_clean":"2024-04-02T13:53:29.997226+0000","last_became_active":"2024-04-02T13:51:01.258500+0000","last_became_peered":"2024-04-02T13:51:01.258500+0000","last_unstale":"2024-04-02T13:53:29.997226+0000","last_undegraded":"2024-04-02T13:53:29.997226+0000","last_fullsized":"2024-04-02T13:53:29.997226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:33:15.409765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895709+0000","last_change":"2024-04-02T13:51:05.365010+0000","last_active":"2024-04-02T13:53:29.895709+0000","last_peered":"2024-04-02T13:53:29.895709+0000","last_clean":"2024-04-02T13:53:29.895709+0000","last_became_active":"2024-04-02T13:51:05.364499+0000","last_became_peered":"2024-04-02T13:51:05.364499+0000","last_unstale":"2024-04-02T13:53:29.895709+0000","last_undegraded":"2024-04-02T13:53:29.895709+0000","last_fullsized":"2024-04-02T13:53:29.895709+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:07:03.500017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174673+0000","last_change":"2024-04-02T13:51:56.496876+0000","last_active":"2024-04-02T13:53:30.174673+0000","last_peered":"2024-04-02T13:53:30.174673+0000","last_clean":"2024-04-02T13:53:30.174673+0000","last_became_active":"2024-04-02T13:51:56.484672+0000","last_became_peered":"2024-04-02T13:51:56.484672+0000","last_unstale":"2024-04-02T13:53:30.174673+0000","last_undegraded":"2024-04-02T13:53:30.174673+0000","last_fullsized":"2024-04-02T13:53:30.174673+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:31:40.027658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.893864+0000","last_change":"2024-04-02T13:51:01.256334+0000","last_active":"2024-04-02T13:53:29.893864+0000","last_peered":"2024-04-02T13:53:29.893864+0000","last_clean":"2024-04-02T13:53:29.893864+0000","last_became_active":"2024-04-02T13:51:01.255964+0000","last_became_peered":"2024-04-02T13:51:01.255964+0000","last_unstale":"2024-04-02T13:53:29.893864+0000","last_undegraded":"2024-04-02T13:53:29.893864+0000","last_fullsized":"2024-04-02T13:53:29.893864+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:43:04.842812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174877+0000","last_change":"2024-04-02T13:51:05.362696+0000","last_active":"2024-04-02T13:53:30.174877+0000","last_peered":"2024-04-02T13:53:30.174877+0000","last_clean":"2024-04-02T13:53:30.174877+0000","last_became_active":"2024-04-02T13:51:05.362225+0000","last_became_peered":"2024-04-02T13:51:05.362225+0000","last_unstale":"2024-04-02T13:53:30.174877+0000","last_undegraded":"2024-04-02T13:53:30.174877+0000","last_fullsized":"2024-04-02T13:53:30.174877+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:23:56.308286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895454+0000","last_change":"2024-04-02T13:51:00.239673+0000","last_active":"2024-04-02T13:53:29.895454+0000","last_peered":"2024-04-02T13:53:29.895454+0000","last_clean":"2024-04-02T13:53:29.895454+0000","last_became_active":"2024-04-02T13:51:00.239233+0000","last_became_peered":"2024-04-02T13:51:00.239233+0000","last_unstale":"2024-04-02T13:53:29.895454+0000","last_undegraded":"2024-04-02T13:53:29.895454+0000","last_fullsized":"2024-04-02T13:53:29.895454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:05:48.119953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135940+0000","last_change":"2024-04-02T13:51:01.257474+0000","last_active":"2024-04-02T13:53:30.135940+0000","last_peered":"2024-04-02T13:53:30.135940+0000","last_clean":"2024-04-02T13:53:30.135940+0000","last_became_active":"2024-04-02T13:51:01.254977+0000","last_became_peered":"2024-04-02T13:51:01.254977+0000","last_unstale":"2024-04-02T13:53:30.135940+0000","last_undegraded":"2024-04-02T13:53:30.135940+0000","last_fullsized":"2024-04-02T13:53:30.135940+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:13:07.446520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.994825+0000","last_change":"2024-04-02T13:51:05.371531+0000","last_active":"2024-04-02T13:53:29.994825+0000","last_peered":"2024-04-02T13:53:29.994825+0000","last_clean":"2024-04-02T13:53:29.994825+0000","last_became_active":"2024-04-02T13:51:05.370551+0000","last_became_peered":"2024-04-02T13:51:05.370551+0000","last_unstale":"2024-04-02T13:53:29.994825+0000","last_undegraded":"2024-04-02T13:53:29.994825+0000","last_fullsized":"2024-04-02T13:53:29.994825+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:43:48.796009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'5","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898007+0000","last_change":"2024-04-02T13:51:00.237715+0000","last_active":"2024-04-02T13:53:29.898007+0000","last_peered":"2024-04-02T13:53:29.898007+0000","last_clean":"2024-04-02T13:53:29.898007+0000","last_became_active":"2024-04-02T13:51:00.237349+0000","last_became_peered":"2024-04-02T13:51:00.237349+0000","last_unstale":"2024-04-02T13:53:29.898007+0000","last_undegraded":"2024-04-02T13:53:29.898007+0000","last_fullsized":"2024-04-02T13:53:29.898007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:33:20.651562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135285+0000","last_change":"2024-04-02T13:51:01.259447+0000","last_active":"2024-04-02T13:53:30.135285+0000","last_peered":"2024-04-02T13:53:30.135285+0000","last_clean":"2024-04-02T13:53:30.135285+0000","last_became_active":"2024-04-02T13:51:01.259101+0000","last_became_peered":"2024-04-02T13:51:01.259101+0000","last_unstale":"2024-04-02T13:53:30.135285+0000","last_undegraded":"2024-04-02T13:53:30.135285+0000","last_fullsized":"2024-04-02T13:53:30.135285+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:51:44.311440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894370+0000","last_change":"2024-04-02T13:51:05.372446+0000","last_active":"2024-04-02T13:53:29.894370+0000","last_peered":"2024-04-02T13:53:29.894370+0000","last_clean":"2024-04-02T13:53:29.894370+0000","last_became_active":"2024-04-02T13:51:05.372007+0000","last_became_peered":"2024-04-02T13:51:05.372007+0000","last_unstale":"2024-04-02T13:53:29.894370+0000","last_undegraded":"2024-04-02T13:53:29.894370+0000","last_fullsized":"2024-04-02T13:53:29.894370+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:54:23.463179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996804+0000","last_change":"2024-04-02T13:51:00.241492+0000","last_active":"2024-04-02T13:53:29.996804+0000","last_peered":"2024-04-02T13:53:29.996804+0000","last_clean":"2024-04-02T13:53:29.996804+0000","last_became_active":"2024-04-02T13:51:00.241167+0000","last_became_peered":"2024-04-02T13:51:00.241167+0000","last_unstale":"2024-04-02T13:53:29.996804+0000","last_undegraded":"2024-04-02T13:53:29.996804+0000","last_fullsized":"2024-04-02T13:53:29.996804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:44:44.070187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895050+0000","last_change":"2024-04-02T13:51:01.263200+0000","last_active":"2024-04-02T13:53:29.895050+0000","last_peered":"2024-04-02T13:53:29.895050+0000","last_clean":"2024-04-02T13:53:29.895050+0000","last_became_active":"2024-04-02T13:51:01.262812+0000","last_became_peered":"2024-04-02T13:51:01.262812+0000","last_unstale":"2024-04-02T13:53:29.895050+0000","last_undegraded":"2024-04-02T13:53:29.895050+0000","last_fullsized":"2024-04-02T13:53:29.895050+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:33:35.898047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895041+0000","last_change":"2024-04-02T13:51:05.366729+0000","last_active":"2024-04-02T13:53:29.895041+0000","last_peered":"2024-04-02T13:53:29.895041+0000","last_clean":"2024-04-02T13:53:29.895041+0000","last_became_active":"2024-04-02T13:51:05.366257+0000","last_became_peered":"2024-04-02T13:51:05.366257+0000","last_unstale":"2024-04-02T13:53:29.895041+0000","last_undegraded":"2024-04-02T13:53:29.895041+0000","last_fullsized":"2024-04-02T13:53:29.895041+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:46:03.400300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997357+0000","last_change":"2024-04-02T13:51:00.236004+0000","last_active":"2024-04-02T13:53:29.997357+0000","last_peered":"2024-04-02T13:53:29.997357+0000","last_clean":"2024-04-02T13:53:29.997357+0000","last_became_active":"2024-04-02T13:51:00.235160+0000","last_became_peered":"2024-04-02T13:51:00.235160+0000","last_unstale":"2024-04-02T13:53:29.997357+0000","last_undegraded":"2024-04-02T13:53:29.997357+0000","last_fullsized":"2024-04-02T13:53:29.997357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:16:35.617346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174808+0000","last_change":"2024-04-02T13:51:01.259565+0000","last_active":"2024-04-02T13:53:30.174808+0000","last_peered":"2024-04-02T13:53:30.174808+0000","last_clean":"2024-04-02T13:53:30.174808+0000","last_became_active":"2024-04-02T13:51:01.259225+0000","last_became_peered":"2024-04-02T13:51:01.259225+0000","last_unstale":"2024-04-02T13:53:30.174808+0000","last_undegraded":"2024-04-02T13:53:30.174808+0000","last_fullsized":"2024-04-02T13:53:30.174808+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:22:18.878029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996871+0000","last_change":"2024-04-02T13:51:05.518750+0000","last_active":"2024-04-02T13:53:29.996871+0000","last_peered":"2024-04-02T13:53:29.996871+0000","last_clean":"2024-04-02T13:53:29.996871+0000","last_became_active":"2024-04-02T13:51:05.518247+0000","last_became_peered":"2024-04-02T13:51:05.518247+0000","last_unstale":"2024-04-02T13:53:29.996871+0000","last_undegraded":"2024-04-02T13:53:29.996871+0000","last_fullsized":"2024-04-02T13:53:29.996871+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:08:10.232509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.994718+0000","last_change":"2024-04-02T13:51:00.241476+0000","last_active":"2024-04-02T13:53:29.994718+0000","last_peered":"2024-04-02T13:53:29.994718+0000","last_clean":"2024-04-02T13:53:29.994718+0000","last_became_active":"2024-04-02T13:51:00.241085+0000","last_became_peered":"2024-04-02T13:51:00.241085+0000","last_unstale":"2024-04-02T13:53:29.994718+0000","last_undegraded":"2024-04-02T13:53:29.994718+0000","last_fullsized":"2024-04-02T13:53:29.994718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:29:01.503901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.003509+0000","last_change":"2024-04-02T13:51:01.259791+0000","last_active":"2024-04-02T13:53:30.003509+0000","last_peered":"2024-04-02T13:53:30.003509+0000","last_clean":"2024-04-02T13:53:30.003509+0000","last_became_active":"2024-04-02T13:51:01.259440+0000","last_became_peered":"2024-04-02T13:51:01.259440+0000","last_unstale":"2024-04-02T13:53:30.003509+0000","last_undegraded":"2024-04-02T13:53:30.003509+0000","last_fullsized":"2024-04-02T13:53:30.003509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:43:52.422263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894028+0000","last_change":"2024-04-02T13:51:05.366243+0000","last_active":"2024-04-02T13:53:29.894028+0000","last_peered":"2024-04-02T13:53:29.894028+0000","last_clean":"2024-04-02T13:53:29.894028+0000","last_became_active":"2024-04-02T13:51:05.365801+0000","last_became_peered":"2024-04-02T13:51:05.365801+0000","last_unstale":"2024-04-02T13:53:29.894028+0000","last_undegraded":"2024-04-02T13:53:29.894028+0000","last_fullsized":"2024-04-02T13:53:29.894028+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:33:56.853434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"93'6","reported_seq":160,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.507013+0000","last_change":"2024-04-02T13:51:00.248695+0000","last_active":"2024-04-02T13:53:30.507013+0000","last_peered":"2024-04-02T13:53:30.507013+0000","last_clean":"2024-04-02T13:53:30.507013+0000","last_became_active":"2024-04-02T13:51:00.248313+0000","last_became_peered":"2024-04-02T13:51:00.248313+0000","last_unstale":"2024-04-02T13:53:30.507013+0000","last_undegraded":"2024-04-02T13:53:30.507013+0000","last_fullsized":"2024-04-02T13:53:30.507013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:43:21.410823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.003516+0000","last_change":"2024-04-02T13:51:01.260993+0000","last_active":"2024-04-02T13:53:30.003516+0000","last_peered":"2024-04-02T13:53:30.003516+0000","last_clean":"2024-04-02T13:53:30.003516+0000","last_became_active":"2024-04-02T13:51:01.260510+0000","last_became_peered":"2024-04-02T13:51:01.260510+0000","last_unstale":"2024-04-02T13:53:30.003516+0000","last_undegraded":"2024-04-02T13:53:30.003516+0000","last_fullsized":"2024-04-02T13:53:30.003516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:17:04.941822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894181+0000","last_change":"2024-04-02T13:51:05.518455+0000","last_active":"2024-04-02T13:53:29.894181+0000","last_peered":"2024-04-02T13:53:29.894181+0000","last_clean":"2024-04-02T13:53:29.894181+0000","last_became_active":"2024-04-02T13:51:05.517962+0000","last_became_peered":"2024-04-02T13:51:05.517962+0000","last_unstale":"2024-04-02T13:53:29.894181+0000","last_undegraded":"2024-04-02T13:53:29.894181+0000","last_fullsized":"2024-04-02T13:53:29.894181+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:19:47.734233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895152+0000","last_change":"2024-04-02T13:51:01.259220+0000","last_active":"2024-04-02T13:53:29.895152+0000","last_peered":"2024-04-02T13:53:29.895152+0000","last_clean":"2024-04-02T13:53:29.895152+0000","last_became_active":"2024-04-02T13:51:01.258890+0000","last_became_peered":"2024-04-02T13:51:01.258890+0000","last_unstale":"2024-04-02T13:53:29.895152+0000","last_undegraded":"2024-04-02T13:53:29.895152+0000","last_fullsized":"2024-04-02T13:53:29.895152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:56:15.444736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":158,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.893734+0000","last_change":"2024-04-02T13:51:00.246727+0000","last_active":"2024-04-02T13:53:29.893734+0000","last_peered":"2024-04-02T13:53:29.893734+0000","last_clean":"2024-04-02T13:53:29.893734+0000","last_became_active":"2024-04-02T13:51:00.246364+0000","last_became_peered":"2024-04-02T13:51:00.246364+0000","last_unstale":"2024-04-02T13:53:29.893734+0000","last_undegraded":"2024-04-02T13:53:29.893734+0000","last_fullsized":"2024-04-02T13:53:29.893734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:35:13.249972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":123,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136219+0000","last_change":"2024-04-02T13:51:05.358506+0000","last_active":"2024-04-02T13:53:30.136219+0000","last_peered":"2024-04-02T13:53:30.136219+0000","last_clean":"2024-04-02T13:53:30.136219+0000","last_became_active":"2024-04-02T13:51:05.358157+0000","last_became_peered":"2024-04-02T13:51:05.358157+0000","last_unstale":"2024-04-02T13:53:30.136219+0000","last_undegraded":"2024-04-02T13:53:30.136219+0000","last_fullsized":"2024-04-02T13:53:30.136219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:24:15.585124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.994632+0000","last_change":"2024-04-02T13:51:56.665307+0000","last_active":"2024-04-02T13:53:29.994632+0000","last_peered":"2024-04-02T13:53:29.994632+0000","last_clean":"2024-04-02T13:53:29.994632+0000","last_became_active":"2024-04-02T13:51:56.665003+0000","last_became_peered":"2024-04-02T13:51:56.665003+0000","last_unstale":"2024-04-02T13:53:29.994632+0000","last_undegraded":"2024-04-02T13:53:29.994632+0000","last_fullsized":"2024-04-02T13:53:29.994632+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:02:46.421524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898110+0000","last_change":"2024-04-02T13:51:00.238962+0000","last_active":"2024-04-02T13:53:29.898110+0000","last_peered":"2024-04-02T13:53:29.898110+0000","last_clean":"2024-04-02T13:53:29.898110+0000","last_became_active":"2024-04-02T13:51:00.238475+0000","last_became_peered":"2024-04-02T13:51:00.238475+0000","last_unstale":"2024-04-02T13:53:29.898110+0000","last_undegraded":"2024-04-02T13:53:29.898110+0000","last_fullsized":"2024-04-02T13:53:29.898110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:41:07.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":364,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135140+0000","last_change":"2024-04-02T13:49:47.847355+0000","last_active":"2024-04-02T13:53:30.135140+0000","last_peered":"2024-04-02T13:53:30.135140+0000","last_clean":"2024-04-02T13:53:30.135140+0000","last_became_active":"2024-04-02T13:49:47.846768+0000","last_became_peered":"2024-04-02T13:49:47.846768+0000","last_unstale":"2024-04-02T13:53:30.135140+0000","last_undegraded":"2024-04-02T13:53:30.135140+0000","last_fullsized":"2024-04-02T13:53:30.135140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:46:21.098780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:46:21.098780+0000","last_clean_scrub_stamp":"2024-04-02T13:46:21.098780+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:03:52.323921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895591+0000","last_change":"2024-04-02T13:51:05.518859+0000","last_active":"2024-04-02T13:53:29.895591+0000","last_peered":"2024-04-02T13:53:29.895591+0000","last_clean":"2024-04-02T13:53:29.895591+0000","last_became_active":"2024-04-02T13:51:05.518364+0000","last_became_peered":"2024-04-02T13:51:05.518364+0000","last_unstale":"2024-04-02T13:53:29.895591+0000","last_undegraded":"2024-04-02T13:53:29.895591+0000","last_fullsized":"2024-04-02T13:53:29.895591+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:10:45.877853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.893663+0000","last_change":"2024-04-02T13:51:01.254795+0000","last_active":"2024-04-02T13:53:29.893663+0000","last_peered":"2024-04-02T13:53:29.893663+0000","last_clean":"2024-04-02T13:53:29.893663+0000","last_became_active":"2024-04-02T13:51:01.254452+0000","last_became_peered":"2024-04-02T13:51:01.254452+0000","last_unstale":"2024-04-02T13:53:29.893663+0000","last_undegraded":"2024-04-02T13:53:29.893663+0000","last_fullsized":"2024-04-02T13:53:29.893663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:13:53.747308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135234+0000","last_change":"2024-04-02T13:51:00.241323+0000","last_active":"2024-04-02T13:53:30.135234+0000","last_peered":"2024-04-02T13:53:30.135234+0000","last_clean":"2024-04-02T13:53:30.135234+0000","last_became_active":"2024-04-02T13:51:00.240989+0000","last_became_peered":"2024-04-02T13:51:00.240989+0000","last_unstale":"2024-04-02T13:53:30.135234+0000","last_undegraded":"2024-04-02T13:53:30.135234+0000","last_fullsized":"2024-04-02T13:53:30.135234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:47:52.601517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174717+0000","last_change":"2024-04-02T13:51:56.665905+0000","last_active":"2024-04-02T13:53:30.174717+0000","last_peered":"2024-04-02T13:53:30.174717+0000","last_clean":"2024-04-02T13:53:30.174717+0000","last_became_active":"2024-04-02T13:51:56.665315+0000","last_became_peered":"2024-04-02T13:51:56.665315+0000","last_unstale":"2024-04-02T13:53:30.174717+0000","last_undegraded":"2024-04-02T13:53:30.174717+0000","last_fullsized":"2024-04-02T13:53:30.174717+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:44:34.548102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997346+0000","last_change":"2024-04-02T13:51:00.241416+0000","last_active":"2024-04-02T13:53:29.997346+0000","last_peered":"2024-04-02T13:53:29.997346+0000","last_clean":"2024-04-02T13:53:29.997346+0000","last_became_active":"2024-04-02T13:51:00.241071+0000","last_became_peered":"2024-04-02T13:51:00.241071+0000","last_unstale":"2024-04-02T13:53:29.997346+0000","last_undegraded":"2024-04-02T13:53:29.997346+0000","last_fullsized":"2024-04-02T13:53:29.997346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:24:16.243737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894214+0000","last_change":"2024-04-02T13:51:01.252156+0000","last_active":"2024-04-02T13:53:29.894214+0000","last_peered":"2024-04-02T13:53:29.894214+0000","last_clean":"2024-04-02T13:53:29.894214+0000","last_became_active":"2024-04-02T13:51:01.251733+0000","last_became_peered":"2024-04-02T13:51:01.251733+0000","last_unstale":"2024-04-02T13:53:29.894214+0000","last_undegraded":"2024-04-02T13:53:29.894214+0000","last_fullsized":"2024-04-02T13:53:29.894214+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:54:24.082643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996784+0000","last_change":"2024-04-02T13:51:05.518367+0000","last_active":"2024-04-02T13:53:29.996784+0000","last_peered":"2024-04-02T13:53:29.996784+0000","last_clean":"2024-04-02T13:53:29.996784+0000","last_became_active":"2024-04-02T13:51:05.517783+0000","last_became_peered":"2024-04-02T13:51:05.517783+0000","last_unstale":"2024-04-02T13:53:29.996784+0000","last_undegraded":"2024-04-02T13:53:29.996784+0000","last_fullsized":"2024-04-02T13:53:29.996784+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:10:53.880909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"93'48","reported_seq":101,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.107358+0000","last_change":"2024-04-02T13:51:00.248998+0000","last_active":"2024-04-02T13:53:30.107358+0000","last_peered":"2024-04-02T13:53:30.107358+0000","last_clean":"2024-04-02T13:53:30.107358+0000","last_became_active":"2024-04-02T13:51:00.248613+0000","last_became_peered":"2024-04-02T13:51:00.248613+0000","last_unstale":"2024-04-02T13:53:30.107358+0000","last_undegraded":"2024-04-02T13:53:30.107358+0000","last_fullsized":"2024-04-02T13:53:30.107358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:18:53.948645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997762+0000","last_change":"2024-04-02T13:51:01.257244+0000","last_active":"2024-04-02T13:53:29.997762+0000","last_peered":"2024-04-02T13:53:29.997762+0000","last_clean":"2024-04-02T13:53:29.997762+0000","last_became_active":"2024-04-02T13:51:01.256739+0000","last_became_peered":"2024-04-02T13:51:01.256739+0000","last_unstale":"2024-04-02T13:53:29.997762+0000","last_undegraded":"2024-04-02T13:53:29.997762+0000","last_fullsized":"2024-04-02T13:53:29.997762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:38:18.786553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.134886+0000","last_change":"2024-04-02T13:51:05.356177+0000","last_active":"2024-04-02T13:53:30.134886+0000","last_peered":"2024-04-02T13:53:30.134886+0000","last_clean":"2024-04-02T13:53:30.134886+0000","last_became_active":"2024-04-02T13:51:05.354704+0000","last_became_peered":"2024-04-02T13:51:05.354704+0000","last_unstale":"2024-04-02T13:53:30.134886+0000","last_undegraded":"2024-04-02T13:53:30.134886+0000","last_fullsized":"2024-04-02T13:53:30.134886+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:27:55.761490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898390+0000","last_change":"2024-04-02T13:51:00.245945+0000","last_active":"2024-04-02T13:53:29.898390+0000","last_peered":"2024-04-02T13:53:29.898390+0000","last_clean":"2024-04-02T13:53:29.898390+0000","last_became_active":"2024-04-02T13:51:00.245528+0000","last_became_peered":"2024-04-02T13:51:00.245528+0000","last_unstale":"2024-04-02T13:53:29.898390+0000","last_undegraded":"2024-04-02T13:53:29.898390+0000","last_fullsized":"2024-04-02T13:53:29.898390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:37:49.263614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.175330+0000","last_change":"2024-04-02T13:51:01.260268+0000","last_active":"2024-04-02T13:53:30.175330+0000","last_peered":"2024-04-02T13:53:30.175330+0000","last_clean":"2024-04-02T13:53:30.175330+0000","last_became_active":"2024-04-02T13:51:01.258182+0000","last_became_peered":"2024-04-02T13:51:01.258182+0000","last_unstale":"2024-04-02T13:53:30.175330+0000","last_undegraded":"2024-04-02T13:53:30.175330+0000","last_fullsized":"2024-04-02T13:53:30.175330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:31:31.423988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898255+0000","last_change":"2024-04-02T13:51:05.518597+0000","last_active":"2024-04-02T13:53:29.898255+0000","last_peered":"2024-04-02T13:53:29.898255+0000","last_clean":"2024-04-02T13:53:29.898255+0000","last_became_active":"2024-04-02T13:51:05.518134+0000","last_became_peered":"2024-04-02T13:51:05.518134+0000","last_unstale":"2024-04-02T13:53:29.898255+0000","last_undegraded":"2024-04-02T13:53:29.898255+0000","last_fullsized":"2024-04-02T13:53:29.898255+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:00:38.632939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135401+0000","last_change":"2024-04-02T13:51:00.241680+0000","last_active":"2024-04-02T13:53:30.135401+0000","last_peered":"2024-04-02T13:53:30.135401+0000","last_clean":"2024-04-02T13:53:30.135401+0000","last_became_active":"2024-04-02T13:51:00.241301+0000","last_became_peered":"2024-04-02T13:51:00.241301+0000","last_unstale":"2024-04-02T13:53:30.135401+0000","last_undegraded":"2024-04-02T13:53:30.135401+0000","last_fullsized":"2024-04-02T13:53:30.135401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:12:10.606973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996895+0000","last_change":"2024-04-02T13:51:01.254795+0000","last_active":"2024-04-02T13:53:29.996895+0000","last_peered":"2024-04-02T13:53:29.996895+0000","last_clean":"2024-04-02T13:53:29.996895+0000","last_became_active":"2024-04-02T13:51:01.254451+0000","last_became_peered":"2024-04-02T13:51:01.254451+0000","last_unstale":"2024-04-02T13:53:29.996895+0000","last_undegraded":"2024-04-02T13:53:29.996895+0000","last_fullsized":"2024-04-02T13:53:29.996895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:58:04.202996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.995353+0000","last_change":"2024-04-02T13:51:05.363592+0000","last_active":"2024-04-02T13:53:29.995353+0000","last_peered":"2024-04-02T13:53:29.995353+0000","last_clean":"2024-04-02T13:53:29.995353+0000","last_became_active":"2024-04-02T13:51:05.363211+0000","last_became_peered":"2024-04-02T13:51:05.363211+0000","last_unstale":"2024-04-02T13:53:29.995353+0000","last_undegraded":"2024-04-02T13:53:29.995353+0000","last_fullsized":"2024-04-02T13:53:29.995353+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:31:56.561505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996907+0000","last_change":"2024-04-02T13:51:00.243681+0000","last_active":"2024-04-02T13:53:29.996907+0000","last_peered":"2024-04-02T13:53:29.996907+0000","last_clean":"2024-04-02T13:53:29.996907+0000","last_became_active":"2024-04-02T13:51:00.241615+0000","last_became_peered":"2024-04-02T13:51:00.241615+0000","last_unstale":"2024-04-02T13:53:29.996907+0000","last_undegraded":"2024-04-02T13:53:29.996907+0000","last_fullsized":"2024-04-02T13:53:29.996907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:54:30.112735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174537+0000","last_change":"2024-04-02T13:51:01.252429+0000","last_active":"2024-04-02T13:53:30.174537+0000","last_peered":"2024-04-02T13:53:30.174537+0000","last_clean":"2024-04-02T13:53:30.174537+0000","last_became_active":"2024-04-02T13:51:01.252017+0000","last_became_peered":"2024-04-02T13:51:01.252017+0000","last_unstale":"2024-04-02T13:53:30.174537+0000","last_undegraded":"2024-04-02T13:53:30.174537+0000","last_fullsized":"2024-04-02T13:53:30.174537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:32:02.152578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898141+0000","last_change":"2024-04-02T13:51:05.361297+0000","last_active":"2024-04-02T13:53:29.898141+0000","last_peered":"2024-04-02T13:53:29.898141+0000","last_clean":"2024-04-02T13:53:29.898141+0000","last_became_active":"2024-04-02T13:51:05.360916+0000","last_became_peered":"2024-04-02T13:51:05.360916+0000","last_unstale":"2024-04-02T13:53:29.898141+0000","last_undegraded":"2024-04-02T13:53:29.898141+0000","last_fullsized":"2024-04-02T13:53:29.898141+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:52:48.608587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"93'4","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174766+0000","last_change":"2024-04-02T13:51:56.488312+0000","last_active":"2024-04-02T13:53:30.174766+0000","last_peered":"2024-04-02T13:53:30.174766+0000","last_clean":"2024-04-02T13:53:30.174766+0000","last_became_active":"2024-04-02T13:51:56.483987+0000","last_became_peered":"2024-04-02T13:51:56.483987+0000","last_unstale":"2024-04-02T13:53:30.174766+0000","last_undegraded":"2024-04-02T13:53:30.174766+0000","last_fullsized":"2024-04-02T13:53:30.174766+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:46:27.396997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997510+0000","last_change":"2024-04-02T13:51:01.255557+0000","last_active":"2024-04-02T13:53:29.997510+0000","last_peered":"2024-04-02T13:53:29.997510+0000","last_clean":"2024-04-02T13:53:29.997510+0000","last_became_active":"2024-04-02T13:51:01.255011+0000","last_became_peered":"2024-04-02T13:51:01.255011+0000","last_unstale":"2024-04-02T13:53:29.997510+0000","last_undegraded":"2024-04-02T13:53:29.997510+0000","last_fullsized":"2024-04-02T13:53:29.997510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:01:52.105965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.898183+0000","last_change":"2024-04-02T13:51:05.518764+0000","last_active":"2024-04-02T13:53:29.898183+0000","last_peered":"2024-04-02T13:53:29.898183+0000","last_clean":"2024-04-02T13:53:29.898183+0000","last_became_active":"2024-04-02T13:51:05.518088+0000","last_became_peered":"2024-04-02T13:51:05.518088+0000","last_unstale":"2024-04-02T13:53:29.898183+0000","last_undegraded":"2024-04-02T13:53:29.898183+0000","last_fullsized":"2024-04-02T13:53:29.898183+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:03:02.901384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996946+0000","last_change":"2024-04-02T13:51:00.244147+0000","last_active":"2024-04-02T13:53:29.996946+0000","last_peered":"2024-04-02T13:53:29.996946+0000","last_clean":"2024-04-02T13:53:29.996946+0000","last_became_active":"2024-04-02T13:51:00.243763+0000","last_became_peered":"2024-04-02T13:51:00.243763+0000","last_unstale":"2024-04-02T13:53:29.996946+0000","last_undegraded":"2024-04-02T13:53:29.996946+0000","last_fullsized":"2024-04-02T13:53:29.996946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:34:27.738746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135780+0000","last_change":"2024-04-02T13:51:01.268121+0000","last_active":"2024-04-02T13:53:30.135780+0000","last_peered":"2024-04-02T13:53:30.135780+0000","last_clean":"2024-04-02T13:53:30.135780+0000","last_became_active":"2024-04-02T13:51:01.266363+0000","last_became_peered":"2024-04-02T13:51:01.266363+0000","last_unstale":"2024-04-02T13:53:30.135780+0000","last_undegraded":"2024-04-02T13:53:30.135780+0000","last_fullsized":"2024-04-02T13:53:30.135780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:30:50.254916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135846+0000","last_change":"2024-04-02T13:51:05.361285+0000","last_active":"2024-04-02T13:53:30.135846+0000","last_peered":"2024-04-02T13:53:30.135846+0000","last_clean":"2024-04-02T13:53:30.135846+0000","last_became_active":"2024-04-02T13:51:05.360122+0000","last_became_peered":"2024-04-02T13:51:05.360122+0000","last_unstale":"2024-04-02T13:53:30.135846+0000","last_undegraded":"2024-04-02T13:53:30.135846+0000","last_fullsized":"2024-04-02T13:53:30.135846+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:30:59.935538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'7","reported_seq":162,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135534+0000","last_change":"2024-04-02T13:51:00.247235+0000","last_active":"2024-04-02T13:53:30.135534+0000","last_peered":"2024-04-02T13:53:30.135534+0000","last_clean":"2024-04-02T13:53:30.135534+0000","last_became_active":"2024-04-02T13:51:00.246856+0000","last_became_peered":"2024-04-02T13:51:00.246856+0000","last_unstale":"2024-04-02T13:53:30.135534+0000","last_undegraded":"2024-04-02T13:53:30.135534+0000","last_fullsized":"2024-04-02T13:53:30.135534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:17:51.875952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136471+0000","last_change":"2024-04-02T13:51:01.263866+0000","last_active":"2024-04-02T13:53:30.136471+0000","last_peered":"2024-04-02T13:53:30.136471+0000","last_clean":"2024-04-02T13:53:30.136471+0000","last_became_active":"2024-04-02T13:51:01.263564+0000","last_became_peered":"2024-04-02T13:51:01.263564+0000","last_unstale":"2024-04-02T13:53:30.136471+0000","last_undegraded":"2024-04-02T13:53:30.136471+0000","last_fullsized":"2024-04-02T13:53:30.136471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:59:33.663277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174890+0000","last_change":"2024-04-02T13:51:05.363079+0000","last_active":"2024-04-02T13:53:30.174890+0000","last_peered":"2024-04-02T13:53:30.174890+0000","last_clean":"2024-04-02T13:53:30.174890+0000","last_became_active":"2024-04-02T13:51:05.362631+0000","last_became_peered":"2024-04-02T13:51:05.362631+0000","last_unstale":"2024-04-02T13:53:30.174890+0000","last_undegraded":"2024-04-02T13:53:30.174890+0000","last_fullsized":"2024-04-02T13:53:30.174890+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:39:23.656269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"93'41","reported_seq":79,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.102154+0000","last_change":"2024-04-02T13:51:00.242820+0000","last_active":"2024-04-02T13:53:30.102154+0000","last_peered":"2024-04-02T13:53:30.102154+0000","last_clean":"2024-04-02T13:53:30.102154+0000","last_became_active":"2024-04-02T13:51:00.242262+0000","last_became_peered":"2024-04-02T13:51:00.242262+0000","last_unstale":"2024-04-02T13:53:30.102154+0000","last_undegraded":"2024-04-02T13:53:30.102154+0000","last_fullsized":"2024-04-02T13:53:30.102154+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:12:22.169019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136066+0000","last_change":"2024-04-02T13:51:01.266548+0000","last_active":"2024-04-02T13:53:30.136066+0000","last_peered":"2024-04-02T13:53:30.136066+0000","last_clean":"2024-04-02T13:53:30.136066+0000","last_became_active":"2024-04-02T13:51:01.259003+0000","last_became_peered":"2024-04-02T13:51:01.259003+0000","last_unstale":"2024-04-02T13:53:30.136066+0000","last_undegraded":"2024-04-02T13:53:30.136066+0000","last_fullsized":"2024-04-02T13:53:30.136066+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:38:53.692231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894871+0000","last_change":"2024-04-02T13:51:05.359994+0000","last_active":"2024-04-02T13:53:29.894871+0000","last_peered":"2024-04-02T13:53:29.894871+0000","last_clean":"2024-04-02T13:53:29.894871+0000","last_became_active":"2024-04-02T13:51:05.359438+0000","last_became_peered":"2024-04-02T13:51:05.359438+0000","last_unstale":"2024-04-02T13:53:29.894871+0000","last_undegraded":"2024-04-02T13:53:29.894871+0000","last_fullsized":"2024-04-02T13:53:29.894871+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:04.335415+0000","last_clean_scrub_stamp":"2024-04-02T13:51:04.335415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:09:43.929446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894377+0000","last_change":"2024-04-02T13:51:00.237889+0000","last_active":"2024-04-02T13:53:29.894377+0000","last_peered":"2024-04-02T13:53:29.894377+0000","last_clean":"2024-04-02T13:53:29.894377+0000","last_became_active":"2024-04-02T13:51:00.237428+0000","last_became_peered":"2024-04-02T13:51:00.237428+0000","last_unstale":"2024-04-02T13:53:29.894377+0000","last_undegraded":"2024-04-02T13:53:29.894377+0000","last_fullsized":"2024-04-02T13:53:29.894377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:50:59.214152+0000","last_clean_scrub_stamp":"2024-04-02T13:50:59.214152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:21:52.483656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136447+0000","last_change":"2024-04-02T13:51:01.262093+0000","last_active":"2024-04-02T13:53:30.136447+0000","last_peered":"2024-04-02T13:53:30.136447+0000","last_clean":"2024-04-02T13:53:30.136447+0000","last_became_active":"2024-04-02T13:51:01.258357+0000","last_became_peered":"2024-04-02T13:51:01.258357+0000","last_unstale":"2024-04-02T13:53:30.136447+0000","last_undegraded":"2024-04-02T13:53:30.136447+0000","last_fullsized":"2024-04-02T13:53:30.136447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:00:06.317307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174608+0000","last_change":"2024-04-02T13:51:56.483560+0000","last_active":"2024-04-02T13:53:30.174608+0000","last_peered":"2024-04-02T13:53:30.174608+0000","last_clean":"2024-04-02T13:53:30.174608+0000","last_became_active":"2024-04-02T13:51:56.483090+0000","last_became_peered":"2024-04-02T13:51:56.483090+0000","last_unstale":"2024-04-02T13:53:30.174608+0000","last_undegraded":"2024-04-02T13:53:30.174608+0000","last_fullsized":"2024-04-02T13:53:30.174608+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:09:35.640492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135057+0000","last_change":"2024-04-02T13:51:56.663817+0000","last_active":"2024-04-02T13:53:30.135057+0000","last_peered":"2024-04-02T13:53:30.135057+0000","last_clean":"2024-04-02T13:53:30.135057+0000","last_became_active":"2024-04-02T13:51:56.663473+0000","last_became_peered":"2024-04-02T13:51:56.663473+0000","last_unstale":"2024-04-02T13:53:30.135057+0000","last_undegraded":"2024-04-02T13:53:30.135057+0000","last_fullsized":"2024-04-02T13:53:30.135057+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:26:15.566030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135112+0000","last_change":"2024-04-02T13:51:01.255563+0000","last_active":"2024-04-02T13:53:30.135112+0000","last_peered":"2024-04-02T13:53:30.135112+0000","last_clean":"2024-04-02T13:53:30.135112+0000","last_became_active":"2024-04-02T13:51:01.255228+0000","last_became_peered":"2024-04-02T13:51:01.255228+0000","last_unstale":"2024-04-02T13:53:30.135112+0000","last_undegraded":"2024-04-02T13:53:30.135112+0000","last_fullsized":"2024-04-02T13:53:30.135112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:48:28.200776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.175260+0000","last_change":"2024-04-02T13:51:01.258153+0000","last_active":"2024-04-02T13:53:30.175260+0000","last_peered":"2024-04-02T13:53:30.175260+0000","last_clean":"2024-04-02T13:53:30.175260+0000","last_became_active":"2024-04-02T13:51:01.257656+0000","last_became_peered":"2024-04-02T13:51:01.257656+0000","last_unstale":"2024-04-02T13:53:30.175260+0000","last_undegraded":"2024-04-02T13:53:30.175260+0000","last_fullsized":"2024-04-02T13:53:30.175260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:37.887427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894267+0000","last_change":"2024-04-02T13:51:01.254675+0000","last_active":"2024-04-02T13:53:29.894267+0000","last_peered":"2024-04-02T13:53:29.894267+0000","last_clean":"2024-04-02T13:53:29.894267+0000","last_became_active":"2024-04-02T13:51:01.254336+0000","last_became_peered":"2024-04-02T13:51:01.254336+0000","last_unstale":"2024-04-02T13:53:29.894267+0000","last_undegraded":"2024-04-02T13:53:29.894267+0000","last_fullsized":"2024-04-02T13:53:29.894267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:28:36.171961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174947+0000","last_change":"2024-04-02T13:51:01.259457+0000","last_active":"2024-04-02T13:53:30.174947+0000","last_peered":"2024-04-02T13:53:30.174947+0000","last_clean":"2024-04-02T13:53:30.174947+0000","last_became_active":"2024-04-02T13:51:01.259099+0000","last_became_peered":"2024-04-02T13:51:01.259099+0000","last_unstale":"2024-04-02T13:53:30.174947+0000","last_undegraded":"2024-04-02T13:53:30.174947+0000","last_fullsized":"2024-04-02T13:53:30.174947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:13:06.594132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.135619+0000","last_change":"2024-04-02T13:51:01.248177+0000","last_active":"2024-04-02T13:53:30.135619+0000","last_peered":"2024-04-02T13:53:30.135619+0000","last_clean":"2024-04-02T13:53:30.135619+0000","last_became_active":"2024-04-02T13:51:01.247843+0000","last_became_peered":"2024-04-02T13:51:01.247843+0000","last_unstale":"2024-04-02T13:53:30.135619+0000","last_undegraded":"2024-04-02T13:53:30.135619+0000","last_fullsized":"2024-04-02T13:53:30.135619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:38:05.805382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.997866+0000","last_change":"2024-04-02T13:51:01.254822+0000","last_active":"2024-04-02T13:53:29.997866+0000","last_peered":"2024-04-02T13:53:29.997866+0000","last_clean":"2024-04-02T13:53:29.997866+0000","last_became_active":"2024-04-02T13:51:01.254451+0000","last_became_peered":"2024-04-02T13:51:01.254451+0000","last_unstale":"2024-04-02T13:53:29.997866+0000","last_undegraded":"2024-04-02T13:53:29.997866+0000","last_fullsized":"2024-04-02T13:53:29.997866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:15:35.790371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.895040+0000","last_change":"2024-04-02T13:51:01.265467+0000","last_active":"2024-04-02T13:53:29.895040+0000","last_peered":"2024-04-02T13:53:29.895040+0000","last_clean":"2024-04-02T13:53:29.895040+0000","last_became_active":"2024-04-02T13:51:01.265072+0000","last_became_peered":"2024-04-02T13:51:01.265072+0000","last_unstale":"2024-04-02T13:53:29.895040+0000","last_undegraded":"2024-04-02T13:53:29.895040+0000","last_fullsized":"2024-04-02T13:53:29.895040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:12:07.702222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.996951+0000","last_change":"2024-04-02T13:51:01.252844+0000","last_active":"2024-04-02T13:53:29.996951+0000","last_peered":"2024-04-02T13:53:29.996951+0000","last_clean":"2024-04-02T13:53:29.996951+0000","last_became_active":"2024-04-02T13:51:01.252515+0000","last_became_peered":"2024-04-02T13:51:01.252515+0000","last_unstale":"2024-04-02T13:53:29.996951+0000","last_undegraded":"2024-04-02T13:53:29.996951+0000","last_fullsized":"2024-04-02T13:53:29.996951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:20:16.216418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.136142+0000","last_change":"2024-04-02T13:51:01.260141+0000","last_active":"2024-04-02T13:53:30.136142+0000","last_peered":"2024-04-02T13:53:30.136142+0000","last_clean":"2024-04-02T13:53:30.136142+0000","last_became_active":"2024-04-02T13:51:01.259530+0000","last_became_peered":"2024-04-02T13:51:01.259530+0000","last_unstale":"2024-04-02T13:53:30.136142+0000","last_undegraded":"2024-04-02T13:53:30.136142+0000","last_fullsized":"2024-04-02T13:53:30.136142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:00:03.579414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:29.894065+0000","last_change":"2024-04-02T13:51:01.259981+0000","last_active":"2024-04-02T13:53:29.894065+0000","last_peered":"2024-04-02T13:53:29.894065+0000","last_clean":"2024-04-02T13:53:29.894065+0000","last_became_active":"2024-04-02T13:51:01.259647+0000","last_became_peered":"2024-04-02T13:51:01.259647+0000","last_unstale":"2024-04-02T13:53:29.894065+0000","last_undegraded":"2024-04-02T13:53:29.894065+0000","last_fullsized":"2024-04-02T13:53:29.894065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:10:22.753767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-02T13:53:30.174911+0000","last_change":"2024-04-02T13:51:01.255342+0000","last_active":"2024-04-02T13:53:30.174911+0000","last_peered":"2024-04-02T13:53:30.174911+0000","last_clean":"2024-04-02T13:53:30.174911+0000","last_became_active":"2024-04-02T13:51:01.254679+0000","last_became_peered":"2024-04-02T13:51:01.254679+0000","last_unstale":"2024-04-02T13:53:30.174911+0000","last_undegraded":"2024-04-02T13:53:30.174911+0000","last_fullsized":"2024-04-02T13:53:30.174911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:51:00.219153+0000","last_clean_scrub_stamp":"2024-04-02T13:51:00.219153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:25:12.198270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601988,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1355,"num_read_kb":796,"num_write":370,"num_write_kb":1468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1805964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":300,"ondisk_log_size":300,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351019,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1069056,"data_stored":517607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612656,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28804,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716348,"statfs":{"total":95995035648,"available":95965540352,"internally_reserved":0,"allocated":1826816,"data_stored":1232765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776181,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28468,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716684,"statfs":{"total":95995035648,"available":95965884416,"internally_reserved":0,"allocated":1482752,"data_stored":914249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037819,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36780,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708372,"statfs":{"total":95995035648,"available":95957372928,"internally_reserved":0,"allocated":1474560,"data_stored":910531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332160,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36980,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708172,"statfs":{"total":95995035648,"available":95957168128,"internally_reserved":0,"allocated":1679360,"data_stored":1114594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495684,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40536,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704616,"statfs":{"total":95995035648,"available":95953526784,"internally_reserved":0,"allocated":1060864,"data_stored":512832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1134592,"data_stored":582907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1085440,"data_stored":514429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1126400,"data_stored":567237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":502341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920847,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1069056,"data_stored":516515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757322,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28672,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716480,"statfs":{"total":95995035648,"available":95965675520,"internally_reserved":0,"allocated":1691648,"data_stored":1109515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":72263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":202381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T13:53:35.760 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:53:35.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.896+0000 7fe2d294d640 1 -- 172.21.15.52:0/2453568316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2cc100a00 msgr2=0x7fe2cc100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:35.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.896+0000 7fe2d294d640 1 --2- 172.21.15.52:0/2453568316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2cc100a00 0x7fe2cc100e80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0009930 tx=0x7fe2c002bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:35.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.896+0000 7fe2d294d640 1 -- 172.21.15.52:0/2453568316 shutdown_connections 2024-04-02T13:53:35.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.896+0000 7fe2d294d640 1 --2- 172.21.15.52:0/2453568316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2cc1013c0 0x7fe2cc10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.896+0000 7fe2d294d640 1 --2- 172.21.15.52:0/2453568316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2cc100a00 0x7fe2cc100e80 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.896+0000 7fe2d294d640 1 --2- 172.21.15.52:0/2453568316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2cc106a10 0x7fe2cc106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.896+0000 7fe2d294d640 1 -- 172.21.15.52:0/2453568316 >> 172.21.15.52:0/2453568316 conn(0x7fe2cc077b00 msgr2=0x7fe2cc077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:35.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.896+0000 7fe2d294d640 1 -- 172.21.15.52:0/2453568316 shutdown_connections 2024-04-02T13:53:35.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.896+0000 7fe2d294d640 1 -- 172.21.15.52:0/2453568316 wait complete. 2024-04-02T13:53:35.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.897+0000 7fe2d294d640 1 Processor -- start 2024-04-02T13:53:35.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.897+0000 7fe2d294d640 1 -- start start 2024-04-02T13:53:35.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.897+0000 7fe2d294d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2cc100a00 0x7fe2cc199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.897+0000 7fe2d294d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2cc1013c0 0x7fe2cc199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.897+0000 7fe2d294d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2cc106a10 0x7fe2cc1a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.897+0000 7fe2d294d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc110df0 con 0x7fe2cc106a10 2024-04-02T13:53:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.897+0000 7fe2d294d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc110c70 con 0x7fe2cc1013c0 2024-04-02T13:53:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.897+0000 7fe2d294d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc110f70 con 0x7fe2cc100a00 2024-04-02T13:53:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.897+0000 7fe2d0eea640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2cc106a10 0x7fe2cc1a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.897+0000 7fe2d0eea640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2cc106a10 0x7fe2cc1a4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58472/0 (socket says 172.21.15.52:58472) 2024-04-02T13:53:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.898+0000 7fe2d0eea640 1 -- 172.21.15.52:0/2554809458 learned_addr learned my addr 172.21.15.52:0/2554809458 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.898+0000 7fe2cbfff640 1 --2- 172.21.15.52:0/2554809458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2cc100a00 0x7fe2cc199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.898+0000 7fe2cb7fe640 1 --2- 172.21.15.52:0/2554809458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2cc1013c0 0x7fe2cc199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.898+0000 7fe2d0eea640 1 -- 172.21.15.52:0/2554809458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2cc100a00 msgr2=0x7fe2cc199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.898+0000 7fe2d0eea640 1 --2- 172.21.15.52:0/2554809458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2cc100a00 0x7fe2cc199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.898+0000 7fe2d0eea640 1 -- 172.21.15.52:0/2554809458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2cc1013c0 msgr2=0x7fe2cc199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:35.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.898+0000 7fe2d0eea640 1 --2- 172.21.15.52:0/2554809458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2cc1013c0 0x7fe2cc199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:35.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.898+0000 7fe2d0eea640 1 -- 172.21.15.52:0/2554809458 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2cc1a47d0 con 0x7fe2cc106a10 2024-04-02T13:53:35.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.898+0000 7fe2d0eea640 1 --2- 172.21.15.52:0/2554809458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2cc106a10 0x7fe2cc1a4290 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009fd0 tx=0x7fe2c400def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:35.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.898+0000 7fe2c97fa640 1 -- 172.21.15.52:0/2554809458 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4017070 con 0x7fe2cc106a10 2024-04-02T13:53:35.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.898+0000 7fe2c97fa640 1 -- 172.21.15.52:0/2554809458 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2c4003e50 con 0x7fe2cc106a10 2024-04-02T13:53:35.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.898+0000 7fe2d294d640 1 -- 172.21.15.52:0/2554809458 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2cc1a49a0 con 0x7fe2cc106a10 2024-04-02T13:53:35.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.898+0000 7fe2c97fa640 1 -- 172.21.15.52:0/2554809458 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4010460 con 0x7fe2cc106a10 2024-04-02T13:53:35.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.898+0000 7fe2d294d640 1 -- 172.21.15.52:0/2554809458 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2cc1a4e10 con 0x7fe2cc106a10 2024-04-02T13:53:35.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.899+0000 7fe2d294d640 1 -- 172.21.15.52:0/2554809458 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2cc0688f0 con 0x7fe2cc106a10 2024-04-02T13:53:35.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.900+0000 7fe2c97fa640 1 -- 172.21.15.52:0/2554809458 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fe2c4003970 con 0x7fe2cc106a10 2024-04-02T13:53:35.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.901+0000 7fe2c97fa640 1 --2- 172.21.15.52:0/2554809458 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe2b40a3e40 0x7fe2b40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:35.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.901+0000 7fe2cbfff640 1 --2- 172.21.15.52:0/2554809458 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe2b40a3e40 0x7fe2b40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:35.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.901+0000 7fe2c97fa640 1 -- 172.21.15.52:0/2554809458 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe2c40d0a70 con 0x7fe2cc106a10 2024-04-02T13:53:35.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.901+0000 7fe2cbfff640 1 --2- 172.21.15.52:0/2554809458 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe2b40a3e40 0x7fe2b40a6300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc00ab60 tx=0x7fe2bc00a2d0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:35.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:35.906+0000 7fe2c97fa640 1 -- 172.21.15.52:0/2554809458 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe2c409cb20 con 0x7fe2cc106a10 2024-04-02T13:53:36.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:35 smithi052 ceph-mon[32104]: from='client.15672 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:53:36.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:35 smithi052 ceph-mon[32104]: from='client.15678 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:53:36.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.097+0000 7fe2d294d640 1 -- 172.21.15.52:0/2554809458 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2cc108f90 con 0x7fe2cc106a10 2024-04-02T13:53:36.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.098+0000 7fe2c97fa640 1 -- 172.21.15.52:0/2554809458 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24926 (secure 0 0 0) 0x7fe2c409c420 con 0x7fe2cc106a10 2024-04-02T13:53:36.102 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:53:36.102 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":93,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T13:53:29.889192+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:50:58.778690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:50:59.561527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:51:03.343487+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/713755826":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/1937276046":"2024-04-03T13:53:29.889159+0000","172.21.15.52:6800/3836085837":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/2957262850":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/3836085837":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/2985814329":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:53:36.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.100+0000 7fe2d294d640 1 -- 172.21.15.52:0/2554809458 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe2b40a3e40 msgr2=0x7fe2b40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:36.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.100+0000 7fe2d294d640 1 --2- 172.21.15.52:0/2554809458 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe2b40a3e40 0x7fe2b40a6300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc00ab60 tx=0x7fe2bc00a2d0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.100+0000 7fe2d294d640 1 -- 172.21.15.52:0/2554809458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2cc106a10 msgr2=0x7fe2cc1a4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:36.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.100+0000 7fe2d294d640 1 --2- 172.21.15.52:0/2554809458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2cc106a10 0x7fe2cc1a4290 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009fd0 tx=0x7fe2c400def0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.100+0000 7fe2d294d640 1 -- 172.21.15.52:0/2554809458 shutdown_connections 2024-04-02T13:53:36.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.100+0000 7fe2d294d640 1 --2- 172.21.15.52:0/2554809458 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe2b40a3e40 0x7fe2b40a6300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.100+0000 7fe2d294d640 1 --2- 172.21.15.52:0/2554809458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2cc106a10 0x7fe2cc1a4290 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.100+0000 7fe2d294d640 1 --2- 172.21.15.52:0/2554809458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2cc1013c0 0x7fe2cc199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.100+0000 7fe2d294d640 1 --2- 172.21.15.52:0/2554809458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2cc100a00 0x7fe2cc199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.100+0000 7fe2d294d640 1 -- 172.21.15.52:0/2554809458 >> 172.21.15.52:0/2554809458 conn(0x7fe2cc077b00 msgr2=0x7fe2cc103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:36.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.100+0000 7fe2d294d640 1 -- 172.21.15.52:0/2554809458 shutdown_connections 2024-04-02T13:53:36.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.101+0000 7fe2d294d640 1 -- 172.21.15.52:0/2554809458 wait complete. 2024-04-02T13:53:36.115 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:46:20.746053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:50:58.778690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:50:59.561527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:51:03.343487+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:53:36.115 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:53:36.189 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:35 smithi189 ceph-mon[36683]: from='client.15672 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:53:36.189 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:35 smithi189 ceph-mon[36683]: from='client.15678 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:53:36.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.218+0000 7f31c0c81640 1 -- 172.21.15.52:0/2682953934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31bc101740 msgr2=0x7f31bc101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:36.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.218+0000 7f31c0c81640 1 --2- 172.21.15.52:0/2682953934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31bc101740 0x7f31bc101b20 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f31b4009900 tx=0x7f31b402bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:36.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.218+0000 7f31c0c81640 1 -- 172.21.15.52:0/2682953934 shutdown_connections 2024-04-02T13:53:36.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.218+0000 7f31c0c81640 1 --2- 172.21.15.52:0/2682953934 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31bc105dc0 0x7f31bc10d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.218+0000 7f31c0c81640 1 --2- 172.21.15.52:0/2682953934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31bc1020f0 0x7f31bc105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.218+0000 7f31c0c81640 1 --2- 172.21.15.52:0/2682953934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31bc101740 0x7f31bc101b20 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.218+0000 7f31c0c81640 1 -- 172.21.15.52:0/2682953934 >> 172.21.15.52:0/2682953934 conn(0x7f31bc075d40 msgr2=0x7f31bc076150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:36.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.218+0000 7f31c0c81640 1 -- 172.21.15.52:0/2682953934 shutdown_connections 2024-04-02T13:53:36.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.218+0000 7f31c0c81640 1 -- 172.21.15.52:0/2682953934 wait complete. 2024-04-02T13:53:36.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.219+0000 7f31c0c81640 1 Processor -- start 2024-04-02T13:53:36.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.219+0000 7f31c0c81640 1 -- start start 2024-04-02T13:53:36.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.219+0000 7f31c0c81640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31bc101740 0x7f31bc19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:36.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.219+0000 7f31c0c81640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31bc105dc0 0x7f31bc19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:36.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.219+0000 7f31c0c81640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31bc19ee20 0x7f31bc19f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:36.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.219+0000 7f31c0c81640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc110de0 con 0x7f31bc101740 2024-04-02T13:53:36.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.219+0000 7f31c0c81640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc110c60 con 0x7f31bc19ee20 2024-04-02T13:53:36.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.219+0000 7f31c0c81640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc110f60 con 0x7f31bc105dc0 2024-04-02T13:53:36.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.219+0000 7f31ba59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31bc101740 0x7f31bc19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:36.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.219+0000 7f31ba59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31bc101740 0x7f31bc19a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58488/0 (socket says 172.21.15.52:58488) 2024-04-02T13:53:36.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.219+0000 7f31ba59c640 1 -- 172.21.15.52:0/431365005 learned_addr learned my addr 172.21.15.52:0/431365005 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:36.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.219+0000 7f31bad9d640 1 --2- 172.21.15.52:0/431365005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31bc19ee20 0x7f31bc19f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:36.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.219+0000 7f31b9d9b640 1 --2- 172.21.15.52:0/431365005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31bc105dc0 0x7f31bc19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:36.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.219+0000 7f31ba59c640 1 -- 172.21.15.52:0/431365005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31bc105dc0 msgr2=0x7f31bc19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:36.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.219+0000 7f31ba59c640 1 --2- 172.21.15.52:0/431365005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31bc105dc0 0x7f31bc19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.219+0000 7f31ba59c640 1 -- 172.21.15.52:0/431365005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31bc19ee20 msgr2=0x7f31bc19f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:36.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.219+0000 7f31ba59c640 1 --2- 172.21.15.52:0/431365005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31bc19ee20 0x7f31bc19f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.219+0000 7f31ba59c640 1 -- 172.21.15.52:0/431365005 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31bc19f910 con 0x7f31bc101740 2024-04-02T13:53:36.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.219+0000 7f31b9d9b640 1 --2- 172.21.15.52:0/431365005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31bc105dc0 0x7f31bc19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:36.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.219+0000 7f31ba59c640 1 --2- 172.21.15.52:0/431365005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31bc101740 0x7f31bc19a560 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f31b4006010 tx=0x7f31b402cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:36.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.220+0000 7f31ab7fe640 1 -- 172.21.15.52:0/431365005 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b4041070 con 0x7f31bc101740 2024-04-02T13:53:36.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.220+0000 7f31ab7fe640 1 -- 172.21.15.52:0/431365005 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31b4004450 con 0x7f31bc101740 2024-04-02T13:53:36.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.220+0000 7f31c0c81640 1 -- 172.21.15.52:0/431365005 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31bc19fba0 con 0x7f31bc101740 2024-04-02T13:53:36.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.220+0000 7f31ab7fe640 1 -- 172.21.15.52:0/431365005 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b4034440 con 0x7f31bc101740 2024-04-02T13:53:36.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.220+0000 7f31c0c81640 1 -- 172.21.15.52:0/431365005 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31bc1a7360 con 0x7f31bc101740 2024-04-02T13:53:36.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.221+0000 7f31c0c81640 1 -- 172.21.15.52:0/431365005 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31bc069d00 con 0x7f31bc101740 2024-04-02T13:53:36.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.222+0000 7f31ab7fe640 1 -- 172.21.15.52:0/431365005 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f31b4044070 con 0x7f31bc101740 2024-04-02T13:53:36.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.222+0000 7f31ab7fe640 1 --2- 172.21.15.52:0/431365005 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f31a40a3e40 0x7f31a40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:36.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.223+0000 7f31b9d9b640 1 --2- 172.21.15.52:0/431365005 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f31a40a3e40 0x7f31a40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:36.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.223+0000 7f31ab7fe640 1 -- 172.21.15.52:0/431365005 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f31b40f28f0 con 0x7f31bc101740 2024-04-02T13:53:36.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.223+0000 7f31b9d9b640 1 --2- 172.21.15.52:0/431365005 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f31a40a3e40 0x7f31a40a6300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f31b00058c0 tx=0x7f31b0005850 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:36.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.227+0000 7f31ab7fe640 1 -- 172.21.15.52:0/431365005 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f31b40beb10 con 0x7f31bc101740 2024-04-02T13:53:36.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.411+0000 7f31c0c81640 1 -- 172.21.15.52:0/431365005 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f31bc1020f0 con 0x7f31bc101740 2024-04-02T13:53:36.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.411+0000 7f31ab7fe640 1 -- 172.21.15.52:0/431365005 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f31b40be410 con 0x7f31bc101740 2024-04-02T13:53:36.415 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-04-02T13:53:36.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.413+0000 7f31c0c81640 1 -- 172.21.15.52:0/431365005 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f31a40a3e40 msgr2=0x7f31a40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:36.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.413+0000 7f31c0c81640 1 --2- 172.21.15.52:0/431365005 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f31a40a3e40 0x7f31a40a6300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f31b00058c0 tx=0x7f31b0005850 comp rx=0 tx=0).stop 2024-04-02T13:53:36.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.413+0000 7f31c0c81640 1 -- 172.21.15.52:0/431365005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31bc101740 msgr2=0x7f31bc19a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:36.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.413+0000 7f31c0c81640 1 --2- 172.21.15.52:0/431365005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31bc101740 0x7f31bc19a560 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f31b4006010 tx=0x7f31b402cc70 comp rx=0 tx=0).stop 2024-04-02T13:53:36.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.413+0000 7f31c0c81640 1 -- 172.21.15.52:0/431365005 shutdown_connections 2024-04-02T13:53:36.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.413+0000 7f31c0c81640 1 --2- 172.21.15.52:0/431365005 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f31a40a3e40 0x7f31a40a6300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.413+0000 7f31c0c81640 1 --2- 172.21.15.52:0/431365005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31bc19ee20 0x7f31bc19f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.413+0000 7f31c0c81640 1 --2- 172.21.15.52:0/431365005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31bc105dc0 0x7f31bc19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.413+0000 7f31c0c81640 1 --2- 172.21.15.52:0/431365005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31bc101740 0x7f31bc19a560 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.413+0000 7f31c0c81640 1 -- 172.21.15.52:0/431365005 >> 172.21.15.52:0/431365005 conn(0x7f31bc075d40 msgr2=0x7f31bc10c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:36.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.413+0000 7f31c0c81640 1 -- 172.21.15.52:0/431365005 shutdown_connections 2024-04-02T13:53:36.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.413+0000 7f31c0c81640 1 -- 172.21.15.52:0/431365005 wait complete. 2024-04-02T13:53:36.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:35 smithi088 ceph-mon[36528]: from='client.15672 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:53:36.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:35 smithi088 ceph-mon[36528]: from='client.15678 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:53:36.427 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:53:36.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.533+0000 7efe11b9d640 1 -- 172.21.15.52:0/3909417727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe0c068d00 msgr2=0x7efe0c069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:36.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.533+0000 7efe11b9d640 1 --2- 172.21.15.52:0/3909417727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe0c068d00 0x7efe0c069180 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efe00009930 tx=0x7efe0002bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:36.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.533+0000 7efe11b9d640 1 -- 172.21.15.52:0/3909417727 shutdown_connections 2024-04-02T13:53:36.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.533+0000 7efe11b9d640 1 --2- 172.21.15.52:0/3909417727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe0c0696c0 0x7efe0c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.533+0000 7efe11b9d640 1 --2- 172.21.15.52:0/3909417727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe0c068d00 0x7efe0c069180 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.533+0000 7efe11b9d640 1 --2- 172.21.15.52:0/3909417727 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe0c107b50 0x7efe0c107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.533+0000 7efe11b9d640 1 -- 172.21.15.52:0/3909417727 >> 172.21.15.52:0/3909417727 conn(0x7efe0c077b00 msgr2=0x7efe0c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:36.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.533+0000 7efe11b9d640 1 -- 172.21.15.52:0/3909417727 shutdown_connections 2024-04-02T13:53:36.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.533+0000 7efe11b9d640 1 -- 172.21.15.52:0/3909417727 wait complete. 2024-04-02T13:53:36.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.533+0000 7efe11b9d640 1 Processor -- start 2024-04-02T13:53:36.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.534+0000 7efe11b9d640 1 -- start start 2024-04-02T13:53:36.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.534+0000 7efe11b9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe0c068d00 0x7efe0c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:36.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.534+0000 7efe11b9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe0c0696c0 0x7efe0c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:36.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.534+0000 7efe11b9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe0c107b50 0x7efe0c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:36.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.534+0000 7efe11b9d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c111ea0 con 0x7efe0c0696c0 2024-04-02T13:53:36.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.534+0000 7efe11b9d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c111d20 con 0x7efe0c068d00 2024-04-02T13:53:36.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.534+0000 7efe11b9d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c112020 con 0x7efe0c107b50 2024-04-02T13:53:36.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.534+0000 7efe0affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe0c0696c0 0x7efe0c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:36.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.534+0000 7efe0affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe0c0696c0 0x7efe0c198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58504/0 (socket says 172.21.15.52:58504) 2024-04-02T13:53:36.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.534+0000 7efe0affd640 1 -- 172.21.15.52:0/4113659964 learned_addr learned my addr 172.21.15.52:0/4113659964 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:36.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.534+0000 7efe0bfff640 1 --2- 172.21.15.52:0/4113659964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe0c107b50 0x7efe0c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:36.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.534+0000 7efe0b7fe640 1 --2- 172.21.15.52:0/4113659964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe0c068d00 0x7efe0c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:36.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.534+0000 7efe0affd640 1 -- 172.21.15.52:0/4113659964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe0c107b50 msgr2=0x7efe0c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:36.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.534+0000 7efe0affd640 1 --2- 172.21.15.52:0/4113659964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe0c107b50 0x7efe0c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.534+0000 7efe0affd640 1 -- 172.21.15.52:0/4113659964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe0c068d00 msgr2=0x7efe0c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:36.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.534+0000 7efe0affd640 1 --2- 172.21.15.52:0/4113659964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe0c068d00 0x7efe0c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.534+0000 7efe0affd640 1 -- 172.21.15.52:0/4113659964 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0c19d3b0 con 0x7efe0c0696c0 2024-04-02T13:53:36.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.535+0000 7efe0affd640 1 --2- 172.21.15.52:0/4113659964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe0c0696c0 0x7efe0c198900 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efe00009230 tx=0x7efe000043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:36.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.535+0000 7efe08ff9640 1 -- 172.21.15.52:0/4113659964 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe00041070 con 0x7efe0c0696c0 2024-04-02T13:53:36.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.535+0000 7efe11b9d640 1 -- 172.21.15.52:0/4113659964 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe0c19d640 con 0x7efe0c0696c0 2024-04-02T13:53:36.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.535+0000 7efe08ff9640 1 -- 172.21.15.52:0/4113659964 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe0002ccc0 con 0x7efe0c0696c0 2024-04-02T13:53:36.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.535+0000 7efe08ff9640 1 -- 172.21.15.52:0/4113659964 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe00003e20 con 0x7efe0c0696c0 2024-04-02T13:53:36.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.535+0000 7efe11b9d640 1 -- 172.21.15.52:0/4113659964 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe0c1008f0 con 0x7efe0c0696c0 2024-04-02T13:53:36.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.536+0000 7efe11b9d640 1 -- 172.21.15.52:0/4113659964 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe0c199710 con 0x7efe0c0696c0 2024-04-02T13:53:36.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.538+0000 7efe08ff9640 1 -- 172.21.15.52:0/4113659964 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7efe00044070 con 0x7efe0c0696c0 2024-04-02T13:53:36.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.538+0000 7efe08ff9640 1 --2- 172.21.15.52:0/4113659964 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efdf40a3e40 0x7efdf40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:36.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.538+0000 7efe0b7fe640 1 --2- 172.21.15.52:0/4113659964 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efdf40a3e40 0x7efdf40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:36.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.538+0000 7efe08ff9640 1 -- 172.21.15.52:0/4113659964 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7efe000f2b20 con 0x7efe0c0696c0 2024-04-02T13:53:36.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.539+0000 7efe0b7fe640 1 --2- 172.21.15.52:0/4113659964 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efdf40a3e40 0x7efdf40a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00ff70 tx=0x7efdfc00f7a0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:36.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.542+0000 7efe08ff9640 1 -- 172.21.15.52:0/4113659964 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7efe000be6f0 con 0x7efe0c0696c0 2024-04-02T13:53:36.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.726+0000 7efe11b9d640 1 -- 172.21.15.52:0/4113659964 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efe0c1031d0 con 0x7efe0c0696c0 2024-04-02T13:53:36.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.726+0000 7efe08ff9640 1 -- 172.21.15.52:0/4113659964 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7efe000c29f0 con 0x7efe0c0696c0 2024-04-02T13:53:36.730 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-04-02T13:53:36.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.728+0000 7efe11b9d640 1 -- 172.21.15.52:0/4113659964 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efdf40a3e40 msgr2=0x7efdf40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:36.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.728+0000 7efe11b9d640 1 --2- 172.21.15.52:0/4113659964 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efdf40a3e40 0x7efdf40a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00ff70 tx=0x7efdfc00f7a0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.728+0000 7efe11b9d640 1 -- 172.21.15.52:0/4113659964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe0c0696c0 msgr2=0x7efe0c198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:36.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.728+0000 7efe11b9d640 1 --2- 172.21.15.52:0/4113659964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe0c0696c0 0x7efe0c198900 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efe00009230 tx=0x7efe000043b0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.728+0000 7efe11b9d640 1 -- 172.21.15.52:0/4113659964 shutdown_connections 2024-04-02T13:53:36.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.728+0000 7efe11b9d640 1 --2- 172.21.15.52:0/4113659964 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efdf40a3e40 0x7efdf40a6300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.728+0000 7efe11b9d640 1 --2- 172.21.15.52:0/4113659964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe0c107b50 0x7efe0c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.728+0000 7efe11b9d640 1 --2- 172.21.15.52:0/4113659964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe0c0696c0 0x7efe0c198900 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.728+0000 7efe11b9d640 1 --2- 172.21.15.52:0/4113659964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe0c068d00 0x7efe0c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.728+0000 7efe11b9d640 1 -- 172.21.15.52:0/4113659964 >> 172.21.15.52:0/4113659964 conn(0x7efe0c077b00 msgr2=0x7efe0c0fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:36.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.728+0000 7efe11b9d640 1 -- 172.21.15.52:0/4113659964 shutdown_connections 2024-04-02T13:53:36.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.728+0000 7efe11b9d640 1 -- 172.21.15.52:0/4113659964 wait complete. 2024-04-02T13:53:36.743 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:53:36.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.845+0000 7fcd0e206640 1 -- 172.21.15.52:0/705178773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd081020f0 msgr2=0x7fcd08105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:36.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.845+0000 7fcd0e206640 1 --2- 172.21.15.52:0/705178773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd081020f0 0x7fcd08105690 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fccfc009900 tx=0x7fccfc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:36.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.845+0000 7fcd0e206640 1 -- 172.21.15.52:0/705178773 shutdown_connections 2024-04-02T13:53:36.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.845+0000 7fcd0e206640 1 --2- 172.21.15.52:0/705178773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd08105dc0 0x7fcd0810d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.845+0000 7fcd0e206640 1 --2- 172.21.15.52:0/705178773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd081020f0 0x7fcd08105690 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.845+0000 7fcd0e206640 1 --2- 172.21.15.52:0/705178773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd08101740 0x7fcd08101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.845+0000 7fcd0e206640 1 -- 172.21.15.52:0/705178773 >> 172.21.15.52:0/705178773 conn(0x7fcd08075d40 msgr2=0x7fcd08076150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:36.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.845+0000 7fcd0e206640 1 -- 172.21.15.52:0/705178773 shutdown_connections 2024-04-02T13:53:36.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.845+0000 7fcd0e206640 1 -- 172.21.15.52:0/705178773 wait complete. 2024-04-02T13:53:36.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.846+0000 7fcd0e206640 1 Processor -- start 2024-04-02T13:53:36.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.846+0000 7fcd0e206640 1 -- start start 2024-04-02T13:53:36.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.846+0000 7fcd0e206640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd08101740 0x7fcd08199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:36.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.846+0000 7fcd0e206640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd08105dc0 0x7fcd081997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:36.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.846+0000 7fcd0e206640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd08199ce0 0x7fcd081a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:36.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.846+0000 7fcd0e206640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcd08110de0 con 0x7fcd08105dc0 2024-04-02T13:53:36.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.846+0000 7fcd0e206640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcd08110c60 con 0x7fcd08101740 2024-04-02T13:53:36.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.846+0000 7fcd0e206640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcd08110f60 con 0x7fcd08199ce0 2024-04-02T13:53:36.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.846+0000 7fcd06ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd08105dc0 0x7fcd081997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:36.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.846+0000 7fcd06ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd08105dc0 0x7fcd081997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58518/0 (socket says 172.21.15.52:58518) 2024-04-02T13:53:36.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.846+0000 7fcd06ffd640 1 -- 172.21.15.52:0/3966271530 learned_addr learned my addr 172.21.15.52:0/3966271530 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:36.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.846+0000 7fcd07fff640 1 --2- 172.21.15.52:0/3966271530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd08199ce0 0x7fcd081a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:36.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.846+0000 7fcd06ffd640 1 -- 172.21.15.52:0/3966271530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd08199ce0 msgr2=0x7fcd081a42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:36.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.846+0000 7fcd06ffd640 1 --2- 172.21.15.52:0/3966271530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd08199ce0 0x7fcd081a42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.846+0000 7fcd077fe640 1 --2- 172.21.15.52:0/3966271530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd08101740 0x7fcd08199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:36.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.847+0000 7fcd06ffd640 1 -- 172.21.15.52:0/3966271530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd08101740 msgr2=0x7fcd08199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:36.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.847+0000 7fcd06ffd640 1 --2- 172.21.15.52:0/3966271530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd08101740 0x7fcd08199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:36.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.847+0000 7fcd06ffd640 1 -- 172.21.15.52:0/3966271530 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd081a4a60 con 0x7fcd08105dc0 2024-04-02T13:53:36.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.847+0000 7fcd07fff640 1 --2- 172.21.15.52:0/3966271530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd08199ce0 0x7fcd081a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:36.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.847+0000 7fcd06ffd640 1 --2- 172.21.15.52:0/3966271530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd08105dc0 0x7fcd081997a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fccfc036040 tx=0x7fccfc02ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:36.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.847+0000 7fcd04ff9640 1 -- 172.21.15.52:0/3966271530 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc041070 con 0x7fcd08105dc0 2024-04-02T13:53:36.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.847+0000 7fcd0e206640 1 -- 172.21.15.52:0/3966271530 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd081a4cf0 con 0x7fcd08105dc0 2024-04-02T13:53:36.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.847+0000 7fcd04ff9640 1 -- 172.21.15.52:0/3966271530 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccfc0054e0 con 0x7fcd08105dc0 2024-04-02T13:53:36.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.847+0000 7fcd04ff9640 1 -- 172.21.15.52:0/3966271530 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc034440 con 0x7fcd08105dc0 2024-04-02T13:53:36.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.847+0000 7fcd0e206640 1 -- 172.21.15.52:0/3966271530 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd081a5180 con 0x7fcd08105dc0 2024-04-02T13:53:36.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.848+0000 7fcd0e206640 1 -- 172.21.15.52:0/3966271530 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd08068d40 con 0x7fcd08105dc0 2024-04-02T13:53:36.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.849+0000 7fcd04ff9640 1 -- 172.21.15.52:0/3966271530 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fccfc044070 con 0x7fcd08105dc0 2024-04-02T13:53:36.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.849+0000 7fcd04ff9640 1 --2- 172.21.15.52:0/3966271530 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fccf00a3e40 0x7fccf00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:36.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.850+0000 7fcd04ff9640 1 -- 172.21.15.52:0/3966271530 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fccfc0f29b0 con 0x7fcd08105dc0 2024-04-02T13:53:36.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.850+0000 7fcd077fe640 1 --2- 172.21.15.52:0/3966271530 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fccf00a3e40 0x7fccf00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:36.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.850+0000 7fcd077fe640 1 --2- 172.21.15.52:0/3966271530 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fccf00a3e40 0x7fccf00a6300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fccf8005fd0 tx=0x7fccf8009470 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:36.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:36.854+0000 7fcd04ff9640 1 -- 172.21.15.52:0/3966271530 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fccfc0bec60 con 0x7fcd08105dc0 2024-04-02T13:53:37.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.041+0000 7fcd0e206640 1 -- 172.21.15.52:0/3966271530 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcd081020f0 con 0x7fcd08105dc0 2024-04-02T13:53:37.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.042+0000 7fcd04ff9640 1 -- 172.21.15.52:0/3966271530 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fccfc0be560 con 0x7fcd08105dc0 2024-04-02T13:53:37.045 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-04-02T13:53:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.044+0000 7fcd0e206640 1 -- 172.21.15.52:0/3966271530 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fccf00a3e40 msgr2=0x7fccf00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.044+0000 7fcd0e206640 1 --2- 172.21.15.52:0/3966271530 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fccf00a3e40 0x7fccf00a6300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fccf8005fd0 tx=0x7fccf8009470 comp rx=0 tx=0).stop 2024-04-02T13:53:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.044+0000 7fcd0e206640 1 -- 172.21.15.52:0/3966271530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd08105dc0 msgr2=0x7fcd081997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.044+0000 7fcd0e206640 1 --2- 172.21.15.52:0/3966271530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd08105dc0 0x7fcd081997a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fccfc036040 tx=0x7fccfc02ce20 comp rx=0 tx=0).stop 2024-04-02T13:53:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.044+0000 7fcd0e206640 1 -- 172.21.15.52:0/3966271530 shutdown_connections 2024-04-02T13:53:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.044+0000 7fcd0e206640 1 --2- 172.21.15.52:0/3966271530 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fccf00a3e40 0x7fccf00a6300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.044+0000 7fcd0e206640 1 --2- 172.21.15.52:0/3966271530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd08199ce0 0x7fcd081a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.044+0000 7fcd0e206640 1 --2- 172.21.15.52:0/3966271530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd08105dc0 0x7fcd081997a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.044+0000 7fcd0e206640 1 --2- 172.21.15.52:0/3966271530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd08101740 0x7fcd08199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.044+0000 7fcd0e206640 1 -- 172.21.15.52:0/3966271530 >> 172.21.15.52:0/3966271530 conn(0x7fcd08075d40 msgr2=0x7fcd080fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.044+0000 7fcd0e206640 1 -- 172.21.15.52:0/3966271530 shutdown_connections 2024-04-02T13:53:37.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.044+0000 7fcd0e206640 1 -- 172.21.15.52:0/3966271530 wait complete. 2024-04-02T13:53:37.057 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:36 smithi052 ceph-mon[32104]: from='client.15684 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:53:37.057 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:36 smithi052 ceph-mon[32104]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 410 B/s wr, 3 op/s 2024-04-02T13:53:37.057 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:36 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2554809458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:53:37.057 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:36 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/431365005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:53:37.058 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:36 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/4113659964' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:53:37.059 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:53:37.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.162+0000 7f3d02fe4640 1 -- 172.21.15.52:0/1200243009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cfc100c90 msgr2=0x7f3cfc1067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:37.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.162+0000 7f3d02fe4640 1 --2- 172.21.15.52:0/1200243009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cfc100c90 0x7f3cfc1067c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009900 tx=0x7f3ce802bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:37.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.162+0000 7f3d02fe4640 1 -- 172.21.15.52:0/1200243009 shutdown_connections 2024-04-02T13:53:37.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.162+0000 7f3d02fe4640 1 --2- 172.21.15.52:0/1200243009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cfc106ef0 0x7f3cfc10ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.162+0000 7f3d02fe4640 1 --2- 172.21.15.52:0/1200243009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cfc100c90 0x7f3cfc1067c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.162+0000 7f3d02fe4640 1 --2- 172.21.15.52:0/1200243009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cfc1002e0 0x7f3cfc1006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.162+0000 7f3d02fe4640 1 -- 172.21.15.52:0/1200243009 >> 172.21.15.52:0/1200243009 conn(0x7f3cfc077030 msgr2=0x7f3cfc0fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:37.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.162+0000 7f3d02fe4640 1 -- 172.21.15.52:0/1200243009 shutdown_connections 2024-04-02T13:53:37.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.162+0000 7f3d02fe4640 1 -- 172.21.15.52:0/1200243009 wait complete. 2024-04-02T13:53:37.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.163+0000 7f3d02fe4640 1 Processor -- start 2024-04-02T13:53:37.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.163+0000 7f3d02fe4640 1 -- start start 2024-04-02T13:53:37.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.163+0000 7f3d02fe4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cfc1002e0 0x7f3cfc1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:37.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.163+0000 7f3d02fe4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cfc106ef0 0x7f3cfc198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:37.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.163+0000 7f3d02fe4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cfc19cc90 0x7f3cfc19d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:37.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.163+0000 7f3d02fe4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3cfc111f50 con 0x7f3cfc19cc90 2024-04-02T13:53:37.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.163+0000 7f3d02fe4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3cfc111dd0 con 0x7f3cfc1002e0 2024-04-02T13:53:37.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.163+0000 7f3d02fe4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3cfc1120d0 con 0x7f3cfc106ef0 2024-04-02T13:53:37.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.163+0000 7f3d01581640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cfc19cc90 0x7f3cfc19d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:37.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.163+0000 7f3d01581640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cfc19cc90 0x7f3cfc19d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58526/0 (socket says 172.21.15.52:58526) 2024-04-02T13:53:37.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.163+0000 7f3d01581640 1 -- 172.21.15.52:0/2086769883 learned_addr learned my addr 172.21.15.52:0/2086769883 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:37.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.163+0000 7f3d00d80640 1 --2- 172.21.15.52:0/2086769883 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cfc1002e0 0x7f3cfc1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:37.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.163+0000 7f3cfbfff640 1 --2- 172.21.15.52:0/2086769883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cfc106ef0 0x7f3cfc198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:37.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.163+0000 7f3d01581640 1 -- 172.21.15.52:0/2086769883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cfc106ef0 msgr2=0x7f3cfc198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:37.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.163+0000 7f3d01581640 1 --2- 172.21.15.52:0/2086769883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cfc106ef0 0x7f3cfc198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.163+0000 7f3d01581640 1 -- 172.21.15.52:0/2086769883 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cfc1002e0 msgr2=0x7f3cfc1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:37.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.163+0000 7f3d01581640 1 --2- 172.21.15.52:0/2086769883 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cfc1002e0 0x7f3cfc1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.163+0000 7f3d01581640 1 -- 172.21.15.52:0/2086769883 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cfc19d780 con 0x7f3cfc19cc90 2024-04-02T13:53:37.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.164+0000 7f3d01581640 1 --2- 172.21.15.52:0/2086769883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cfc19cc90 0x7f3cfc19d140 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3cf400eb10 tx=0x7f3cf400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:37.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.164+0000 7f3cf9ffb640 1 -- 172.21.15.52:0/2086769883 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf400ce20 con 0x7f3cfc19cc90 2024-04-02T13:53:37.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.164+0000 7f3cf9ffb640 1 -- 172.21.15.52:0/2086769883 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cf4005020 con 0x7f3cfc19cc90 2024-04-02T13:53:37.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.164+0000 7f3d02fe4640 1 -- 172.21.15.52:0/2086769883 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cfc19da70 con 0x7f3cfc19cc90 2024-04-02T13:53:37.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.164+0000 7f3cf9ffb640 1 -- 172.21.15.52:0/2086769883 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4010690 con 0x7f3cfc19cc90 2024-04-02T13:53:37.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.164+0000 7f3d02fe4640 1 -- 172.21.15.52:0/2086769883 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cfc1a5230 con 0x7f3cfc19cc90 2024-04-02T13:53:37.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.165+0000 7f3d02fe4640 1 -- 172.21.15.52:0/2086769883 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cfc068d40 con 0x7f3cfc19cc90 2024-04-02T13:53:37.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.166+0000 7f3cf9ffb640 1 -- 172.21.15.52:0/2086769883 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f3cf4005540 con 0x7f3cfc19cc90 2024-04-02T13:53:37.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.166+0000 7f3cf9ffb640 1 --2- 172.21.15.52:0/2086769883 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3cec0a3e40 0x7f3cec0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:37.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.167+0000 7f3cf9ffb640 1 -- 172.21.15.52:0/2086769883 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f3cf40d1490 con 0x7f3cfc19cc90 2024-04-02T13:53:37.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.167+0000 7f3d00d80640 1 --2- 172.21.15.52:0/2086769883 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3cec0a3e40 0x7f3cec0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:37.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.167+0000 7f3d00d80640 1 --2- 172.21.15.52:0/2086769883 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3cec0a3e40 0x7f3cec0a6300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000abe0 tx=0x7f3cf000a2d0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:37.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.171+0000 7f3cf9ffb640 1 -- 172.21.15.52:0/2086769883 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3cf409d540 con 0x7f3cfc19cc90 2024-04-02T13:53:37.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:36 smithi189 ceph-mon[36683]: from='client.15684 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:53:37.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:36 smithi189 ceph-mon[36683]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 410 B/s wr, 3 op/s 2024-04-02T13:53:37.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:36 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2554809458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:53:37.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:36 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/431365005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:53:37.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:36 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/4113659964' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:53:37.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.356+0000 7f3d02fe4640 1 -- 172.21.15.52:0/2086769883 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f3cfc100c90 con 0x7f3cfc19cc90 2024-04-02T13:53:37.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.357+0000 7f3cf9ffb640 1 -- 172.21.15.52:0/2086769883 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f3cf409ce40 con 0x7f3cfc19cc90 2024-04-02T13:53:37.360 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-04-02T13:53:37.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.359+0000 7f3d02fe4640 1 -- 172.21.15.52:0/2086769883 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3cec0a3e40 msgr2=0x7f3cec0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:37.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.359+0000 7f3d02fe4640 1 --2- 172.21.15.52:0/2086769883 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3cec0a3e40 0x7f3cec0a6300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000abe0 tx=0x7f3cf000a2d0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.359+0000 7f3d02fe4640 1 -- 172.21.15.52:0/2086769883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cfc19cc90 msgr2=0x7f3cfc19d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:37.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.359+0000 7f3d02fe4640 1 --2- 172.21.15.52:0/2086769883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cfc19cc90 0x7f3cfc19d140 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3cf400eb10 tx=0x7f3cf400ef10 comp rx=0 tx=0).stop 2024-04-02T13:53:37.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.359+0000 7f3d02fe4640 1 -- 172.21.15.52:0/2086769883 shutdown_connections 2024-04-02T13:53:37.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.359+0000 7f3d02fe4640 1 --2- 172.21.15.52:0/2086769883 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3cec0a3e40 0x7f3cec0a6300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.359+0000 7f3d02fe4640 1 --2- 172.21.15.52:0/2086769883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3cfc19cc90 0x7f3cfc19d140 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.359+0000 7f3d02fe4640 1 --2- 172.21.15.52:0/2086769883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cfc106ef0 0x7f3cfc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.359+0000 7f3d02fe4640 1 --2- 172.21.15.52:0/2086769883 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cfc1002e0 0x7f3cfc1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.359+0000 7f3d02fe4640 1 -- 172.21.15.52:0/2086769883 >> 172.21.15.52:0/2086769883 conn(0x7f3cfc077030 msgr2=0x7f3cfc0fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:37.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.359+0000 7f3d02fe4640 1 -- 172.21.15.52:0/2086769883 shutdown_connections 2024-04-02T13:53:37.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.359+0000 7f3d02fe4640 1 -- 172.21.15.52:0/2086769883 wait complete. 2024-04-02T13:53:37.374 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:53:37.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:36 smithi088 ceph-mon[36528]: from='client.15684 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:53:37.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:36 smithi088 ceph-mon[36528]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 410 B/s wr, 3 op/s 2024-04-02T13:53:37.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:36 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2554809458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:53:37.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:36 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/431365005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:53:37.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:36 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/4113659964' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:53:37.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.474+0000 7fd5dbb9d640 1 -- 172.21.15.52:0/2516685296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d4102d30 msgr2=0x7fd5d4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:37.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.474+0000 7fd5dbb9d640 1 --2- 172.21.15.52:0/2516685296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d4102d30 0x7fd5d4103110 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009900 tx=0x7fd5d002bcd0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.474+0000 7fd5dbb9d640 1 -- 172.21.15.52:0/2516685296 shutdown_connections 2024-04-02T13:53:37.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.474+0000 7fd5dbb9d640 1 --2- 172.21.15.52:0/2516685296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5d410c100 0x7fd5d410e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.474+0000 7fd5dbb9d640 1 --2- 172.21.15.52:0/2516685296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5d4103650 0x7fd5d410bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.474+0000 7fd5dbb9d640 1 --2- 172.21.15.52:0/2516685296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d4102d30 0x7fd5d4103110 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.474+0000 7fd5dbb9d640 1 -- 172.21.15.52:0/2516685296 >> 172.21.15.52:0/2516685296 conn(0x7fd5d4076d80 msgr2=0x7fd5d4077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:37.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.474+0000 7fd5dbb9d640 1 -- 172.21.15.52:0/2516685296 shutdown_connections 2024-04-02T13:53:37.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.474+0000 7fd5dbb9d640 1 -- 172.21.15.52:0/2516685296 wait complete. 2024-04-02T13:53:37.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.475+0000 7fd5dbb9d640 1 Processor -- start 2024-04-02T13:53:37.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.475+0000 7fd5dbb9d640 1 -- start start 2024-04-02T13:53:37.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.475+0000 7fd5dbb9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d4102d30 0x7fd5d41982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:37.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.475+0000 7fd5dbb9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5d4103650 0x7fd5d41987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:37.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.475+0000 7fd5dbb9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5d410c100 0x7fd5d419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:37.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.475+0000 7fd5dbb9d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d4111e90 con 0x7fd5d4102d30 2024-04-02T13:53:37.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.475+0000 7fd5dbb9d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d4111d10 con 0x7fd5d4103650 2024-04-02T13:53:37.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.475+0000 7fd5dbb9d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d4112010 con 0x7fd5d410c100 2024-04-02T13:53:37.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.475+0000 7fd5d9939640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d4102d30 0x7fd5d41982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:37.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.475+0000 7fd5d9939640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d4102d30 0x7fd5d41982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58538/0 (socket says 172.21.15.52:58538) 2024-04-02T13:53:37.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.475+0000 7fd5d9939640 1 -- 172.21.15.52:0/3561269454 learned_addr learned my addr 172.21.15.52:0/3561269454 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:37.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.475+0000 7fd5da13a640 1 --2- 172.21.15.52:0/3561269454 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5d410c100 0x7fd5d419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:37.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.475+0000 7fd5d9138640 1 --2- 172.21.15.52:0/3561269454 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5d4103650 0x7fd5d41987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:37.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.475+0000 7fd5d9939640 1 -- 172.21.15.52:0/3561269454 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5d410c100 msgr2=0x7fd5d419cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:37.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.475+0000 7fd5d9939640 1 --2- 172.21.15.52:0/3561269454 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5d410c100 0x7fd5d419cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.475+0000 7fd5d9939640 1 -- 172.21.15.52:0/3561269454 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5d4103650 msgr2=0x7fd5d41987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:37.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.475+0000 7fd5d9939640 1 --2- 172.21.15.52:0/3561269454 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5d4103650 0x7fd5d41987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.475+0000 7fd5d9939640 1 -- 172.21.15.52:0/3561269454 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d419d2a0 con 0x7fd5d4102d30 2024-04-02T13:53:37.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.475+0000 7fd5da13a640 1 --2- 172.21.15.52:0/3561269454 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5d410c100 0x7fd5d419cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:37.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.475+0000 7fd5d9138640 1 --2- 172.21.15.52:0/3561269454 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5d4103650 0x7fd5d41987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:53:37.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.475+0000 7fd5d9939640 1 --2- 172.21.15.52:0/3561269454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d4102d30 0x7fd5d41982b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd5d002efd0 tx=0x7fd5d002ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:37.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.476+0000 7fd5caffd640 1 -- 172.21.15.52:0/3561269454 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0041070 con 0x7fd5d4102d30 2024-04-02T13:53:37.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.476+0000 7fd5caffd640 1 -- 172.21.15.52:0/3561269454 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5d00054e0 con 0x7fd5d4102d30 2024-04-02T13:53:37.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.476+0000 7fd5dbb9d640 1 -- 172.21.15.52:0/3561269454 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d419d530 con 0x7fd5d4102d30 2024-04-02T13:53:37.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.476+0000 7fd5caffd640 1 -- 172.21.15.52:0/3561269454 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0034440 con 0x7fd5d4102d30 2024-04-02T13:53:37.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.476+0000 7fd5dbb9d640 1 -- 172.21.15.52:0/3561269454 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d41a4e20 con 0x7fd5d4102d30 2024-04-02T13:53:37.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.477+0000 7fd5dbb9d640 1 -- 172.21.15.52:0/3561269454 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5d406a950 con 0x7fd5d4102d30 2024-04-02T13:53:37.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.478+0000 7fd5caffd640 1 -- 172.21.15.52:0/3561269454 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fd5d0044070 con 0x7fd5d4102d30 2024-04-02T13:53:37.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.479+0000 7fd5caffd640 1 --2- 172.21.15.52:0/3561269454 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd5c00a3ec0 0x7fd5c00a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:37.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.479+0000 7fd5d9138640 1 --2- 172.21.15.52:0/3561269454 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd5c00a3ec0 0x7fd5c00a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:37.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.479+0000 7fd5caffd640 1 -- 172.21.15.52:0/3561269454 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd5d00f2a20 con 0x7fd5d4102d30 2024-04-02T13:53:37.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.479+0000 7fd5d9138640 1 --2- 172.21.15.52:0/3561269454 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd5c00a3ec0 0x7fd5c00a6380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd5d41000e0 tx=0x7fd5c4005d90 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:37.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.483+0000 7fd5caffd640 1 -- 172.21.15.52:0/3561269454 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd5d00bebe0 con 0x7fd5d4102d30 2024-04-02T13:53:37.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.710+0000 7fd5dbb9d640 1 -- 172.21.15.52:0/3561269454 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5d4068d40 con 0x7fd5d4102d30 2024-04-02T13:53:37.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.711+0000 7fd5caffd640 1 -- 172.21.15.52:0/3561269454 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11923 (secure 0 0 0) 0x7fd5d00be4e0 con 0x7fd5d4102d30 2024-04-02T13:53:37.714 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:53:37.715 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:53:28.475061+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24457,24467,24490,24503,24475]},"id":1}]} 2024-04-02T13:53:37.715 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 28 2024-04-02T13:53:37.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.713+0000 7fd5dbb9d640 1 -- 172.21.15.52:0/3561269454 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd5c00a3ec0 msgr2=0x7fd5c00a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:37.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.713+0000 7fd5dbb9d640 1 --2- 172.21.15.52:0/3561269454 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd5c00a3ec0 0x7fd5c00a6380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd5d41000e0 tx=0x7fd5c4005d90 comp rx=0 tx=0).stop 2024-04-02T13:53:37.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.713+0000 7fd5dbb9d640 1 -- 172.21.15.52:0/3561269454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d4102d30 msgr2=0x7fd5d41982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:37.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.713+0000 7fd5dbb9d640 1 --2- 172.21.15.52:0/3561269454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d4102d30 0x7fd5d41982b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd5d002efd0 tx=0x7fd5d002ce50 comp rx=0 tx=0).stop 2024-04-02T13:53:37.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.713+0000 7fd5dbb9d640 1 -- 172.21.15.52:0/3561269454 shutdown_connections 2024-04-02T13:53:37.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.713+0000 7fd5dbb9d640 1 --2- 172.21.15.52:0/3561269454 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd5c00a3ec0 0x7fd5c00a6380 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.713+0000 7fd5dbb9d640 1 --2- 172.21.15.52:0/3561269454 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5d410c100 0x7fd5d419cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.713+0000 7fd5dbb9d640 1 --2- 172.21.15.52:0/3561269454 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5d4103650 0x7fd5d41987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.713+0000 7fd5dbb9d640 1 --2- 172.21.15.52:0/3561269454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d4102d30 0x7fd5d41982b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.713+0000 7fd5dbb9d640 1 -- 172.21.15.52:0/3561269454 >> 172.21.15.52:0/3561269454 conn(0x7fd5d4076d80 msgr2=0x7fd5d40fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:37.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.713+0000 7fd5dbb9d640 1 -- 172.21.15.52:0/3561269454 shutdown_connections 2024-04-02T13:53:37.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.713+0000 7fd5dbb9d640 1 -- 172.21.15.52:0/3561269454 wait complete. 2024-04-02T13:53:37.728 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:53:37.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.827+0000 7f3abb5e2640 1 -- 172.21.15.52:0/3738972491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ab4105460 msgr2=0x7f3ab410d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:37.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.827+0000 7f3abb5e2640 1 --2- 172.21.15.52:0/3738972491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ab4105460 0x7f3ab410d030 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009ca0 tx=0x7f3aac02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.827+0000 7f3abb5e2640 1 -- 172.21.15.52:0/3738972491 shutdown_connections 2024-04-02T13:53:37.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.827+0000 7f3abb5e2640 1 --2- 172.21.15.52:0/3738972491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ab4105460 0x7f3ab410d030 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.827+0000 7f3abb5e2640 1 --2- 172.21.15.52:0/3738972491 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ab4100d90 0x7f3ab4104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.827+0000 7f3abb5e2640 1 --2- 172.21.15.52:0/3738972491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ab41003e0 0x7f3ab41007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.827+0000 7f3abb5e2640 1 -- 172.21.15.52:0/3738972491 >> 172.21.15.52:0/3738972491 conn(0x7f3ab4076f90 msgr2=0x7f3ab40fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:37.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.827+0000 7f3abb5e2640 1 -- 172.21.15.52:0/3738972491 shutdown_connections 2024-04-02T13:53:37.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.827+0000 7f3abb5e2640 1 -- 172.21.15.52:0/3738972491 wait complete. 2024-04-02T13:53:37.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.828+0000 7f3abb5e2640 1 Processor -- start 2024-04-02T13:53:37.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.828+0000 7f3abb5e2640 1 -- start start 2024-04-02T13:53:37.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.828+0000 7f3abb5e2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ab41003e0 0x7f3ab4198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:37.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.828+0000 7f3abb5e2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ab4105460 0x7f3ab4198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:37.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.828+0000 7f3abb5e2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ab419cc10 0x7f3ab419d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:37.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.828+0000 7f3abb5e2640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab410fc90 con 0x7f3ab41003e0 2024-04-02T13:53:37.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.828+0000 7f3abb5e2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab410fb10 con 0x7f3ab419cc10 2024-04-02T13:53:37.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.828+0000 7f3abb5e2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab410fe10 con 0x7f3ab4105460 2024-04-02T13:53:37.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.828+0000 7f3ab937e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ab41003e0 0x7f3ab4198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:37.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.828+0000 7f3ab937e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ab41003e0 0x7f3ab4198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58570/0 (socket says 172.21.15.52:58570) 2024-04-02T13:53:37.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.828+0000 7f3ab937e640 1 -- 172.21.15.52:0/3020094607 learned_addr learned my addr 172.21.15.52:0/3020094607 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:37.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.828+0000 7f3ab8b7d640 1 --2- 172.21.15.52:0/3020094607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ab4105460 0x7f3ab4198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:37.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.829+0000 7f3ab937e640 1 -- 172.21.15.52:0/3020094607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ab4105460 msgr2=0x7f3ab4198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:37.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.829+0000 7f3ab937e640 1 --2- 172.21.15.52:0/3020094607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ab4105460 0x7f3ab4198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.829+0000 7f3ab937e640 1 -- 172.21.15.52:0/3020094607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ab419cc10 msgr2=0x7f3ab419d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:37.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.829+0000 7f3ab937e640 1 --2- 172.21.15.52:0/3020094607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ab419cc10 0x7f3ab419d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:37.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.829+0000 7f3ab937e640 1 -- 172.21.15.52:0/3020094607 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab419d700 con 0x7f3ab41003e0 2024-04-02T13:53:37.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.829+0000 7f3ab8b7d640 1 --2- 172.21.15.52:0/3020094607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ab4105460 0x7f3ab4198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:37.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.829+0000 7f3ab937e640 1 --2- 172.21.15.52:0/3020094607 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ab41003e0 0x7f3ab4198350 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000dbc0 tx=0x7f3ab000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:37.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.829+0000 7f3aaa7fc640 1 -- 172.21.15.52:0/3020094607 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0005220 con 0x7f3ab41003e0 2024-04-02T13:53:37.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.829+0000 7f3abb5e2640 1 -- 172.21.15.52:0/3020094607 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab419d9f0 con 0x7f3ab41003e0 2024-04-02T13:53:37.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.829+0000 7f3aaa7fc640 1 -- 172.21.15.52:0/3020094607 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ab000ae50 con 0x7f3ab41003e0 2024-04-02T13:53:37.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.829+0000 7f3aaa7fc640 1 -- 172.21.15.52:0/3020094607 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab000e870 con 0x7f3ab41003e0 2024-04-02T13:53:37.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.829+0000 7f3abb5e2640 1 -- 172.21.15.52:0/3020094607 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab41a5190 con 0x7f3ab41003e0 2024-04-02T13:53:37.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.830+0000 7f3abb5e2640 1 -- 172.21.15.52:0/3020094607 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ab4068d40 con 0x7f3ab41003e0 2024-04-02T13:53:37.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.831+0000 7f3aaa7fc640 1 -- 172.21.15.52:0/3020094607 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f3ab000ea10 con 0x7f3ab41003e0 2024-04-02T13:53:37.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.832+0000 7f3aaa7fc640 1 --2- 172.21.15.52:0/3020094607 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3aa00a3e40 0x7f3aa00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:37.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.832+0000 7f3ab8b7d640 1 --2- 172.21.15.52:0/3020094607 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3aa00a3e40 0x7f3aa00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:37.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.832+0000 7f3aaa7fc640 1 -- 172.21.15.52:0/3020094607 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f3ab00d12e0 con 0x7f3ab41003e0 2024-04-02T13:53:37.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.832+0000 7f3ab8b7d640 1 --2- 172.21.15.52:0/3020094607 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3aa00a3e40 0x7f3aa00a6300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4005e10 tx=0x7f3aa4005da0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:37.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:37.835+0000 7f3aaa7fc640 1 -- 172.21.15.52:0/3020094607 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3ab009d500 con 0x7f3ab41003e0 2024-04-02T13:53:38.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.023+0000 7f3abb5e2640 1 -- 172.21.15.52:0/3020094607 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ab4100d90 con 0x7f3ab41003e0 2024-04-02T13:53:38.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.025+0000 7f3aaa7fc640 1 -- 172.21.15.52:0/3020094607 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24926 (secure 0 0 0) 0x7f3ab009ce00 con 0x7f3ab41003e0 2024-04-02T13:53:38.028 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:53:38.028 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":93,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T13:53:29.889192+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:50:58.778690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:50:59.561527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:51:03.343487+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/713755826":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/1937276046":"2024-04-03T13:53:29.889159+0000","172.21.15.52:6800/3836085837":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/2957262850":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/3836085837":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/2985814329":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:53:38.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.027+0000 7f3abb5e2640 1 -- 172.21.15.52:0/3020094607 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3aa00a3e40 msgr2=0x7f3aa00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.027+0000 7f3abb5e2640 1 --2- 172.21.15.52:0/3020094607 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3aa00a3e40 0x7f3aa00a6300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4005e10 tx=0x7f3aa4005da0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.027+0000 7f3abb5e2640 1 -- 172.21.15.52:0/3020094607 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ab41003e0 msgr2=0x7f3ab4198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.027+0000 7f3abb5e2640 1 --2- 172.21.15.52:0/3020094607 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ab41003e0 0x7f3ab4198350 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000dbc0 tx=0x7f3ab000a620 comp rx=0 tx=0).stop 2024-04-02T13:53:38.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.027+0000 7f3abb5e2640 1 -- 172.21.15.52:0/3020094607 shutdown_connections 2024-04-02T13:53:38.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.027+0000 7f3abb5e2640 1 --2- 172.21.15.52:0/3020094607 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3aa00a3e40 0x7f3aa00a6300 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.027+0000 7f3abb5e2640 1 --2- 172.21.15.52:0/3020094607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ab419cc10 0x7f3ab419d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.027+0000 7f3abb5e2640 1 --2- 172.21.15.52:0/3020094607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ab4105460 0x7f3ab4198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.027+0000 7f3abb5e2640 1 --2- 172.21.15.52:0/3020094607 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ab41003e0 0x7f3ab4198350 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.027+0000 7f3abb5e2640 1 -- 172.21.15.52:0/3020094607 >> 172.21.15.52:0/3020094607 conn(0x7f3ab4076f90 msgr2=0x7f3ab41038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:38.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.027+0000 7f3abb5e2640 1 -- 172.21.15.52:0/3020094607 shutdown_connections 2024-04-02T13:53:38.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.027+0000 7f3abb5e2640 1 -- 172.21.15.52:0/3020094607 wait complete. 2024-04-02T13:53:38.040 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:37 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3966271530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:53:38.040 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:37 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2086769883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:53:38.040 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:37 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3561269454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:53:38.042 DEBUG:tasks.fwd_scrub:Yielding 2024-04-02T13:53:38.042 INFO:teuthology.run_tasks:Running task exec... 2024-04-02T13:53:38.054 INFO:teuthology.task.exec:Executing custom commands... 2024-04-02T13:53:38.055 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi052.front.sepia.ceph.com 2024-04-02T13:53:38.055 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-02T13:53:38.055 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-02T13:53:38.056 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:53:38.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:53:38.056 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:53:38.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.159+0000 7fe8f907b640 1 -- 172.21.15.52:0/1301390430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8f4101610 msgr2=0x7fe8f410d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.159+0000 7fe8f907b640 1 --2- 172.21.15.52:0/1301390430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8f4101610 0x7fe8f410d830 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009ca0 tx=0x7fe8e402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.159+0000 7fe8f907b640 1 -- 172.21.15.52:0/1301390430 shutdown_connections 2024-04-02T13:53:38.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.159+0000 7fe8f907b640 1 --2- 172.21.15.52:0/1301390430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8f4101610 0x7fe8f410d830 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.159+0000 7fe8f907b640 1 --2- 172.21.15.52:0/1301390430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8f4100c50 0x7fe8f41010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.159+0000 7fe8f907b640 1 --2- 172.21.15.52:0/1301390430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8f41059a0 0x7fe8f4105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.159+0000 7fe8f907b640 1 -- 172.21.15.52:0/1301390430 >> 172.21.15.52:0/1301390430 conn(0x7fe8f4077850 msgr2=0x7fe8f4077c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:38.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.159+0000 7fe8f907b640 1 -- 172.21.15.52:0/1301390430 shutdown_connections 2024-04-02T13:53:38.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.159+0000 7fe8f907b640 1 -- 172.21.15.52:0/1301390430 wait complete. 2024-04-02T13:53:38.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.159+0000 7fc9385e2640 1 -- 172.21.15.52:0/1485740624 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc930101330 msgr2=0x7fc93010d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.159+0000 7fc9385e2640 1 --2- 172.21.15.52:0/1485740624 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc930101330 0x7fc93010d840 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc928009cd0 tx=0x7fc92802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.159+0000 7fc9385e2640 1 -- 172.21.15.52:0/1485740624 shutdown_connections 2024-04-02T13:53:38.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.159+0000 7fc9385e2640 1 --2- 172.21.15.52:0/1485740624 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc930101330 0x7fc93010d840 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.159+0000 7fc9385e2640 1 --2- 172.21.15.52:0/1485740624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc930100970 0x7fc930100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.159+0000 7fc9385e2640 1 --2- 172.21.15.52:0/1485740624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc930106980 0x7fc930106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.159+0000 7fc9385e2640 1 -- 172.21.15.52:0/1485740624 >> 172.21.15.52:0/1485740624 conn(0x7fc930077b00 msgr2=0x7fc930077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:38.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.159+0000 7fc9385e2640 1 -- 172.21.15.52:0/1485740624 shutdown_connections 2024-04-02T13:53:38.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.159+0000 7fc9385e2640 1 -- 172.21.15.52:0/1485740624 wait complete. 2024-04-02T13:53:38.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.160+0000 7fe8f907b640 1 Processor -- start 2024-04-02T13:53:38.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.160+0000 7fe8f907b640 1 -- start start 2024-04-02T13:53:38.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.160+0000 7fc9385e2640 1 Processor -- start 2024-04-02T13:53:38.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.160+0000 7fc9385e2640 1 -- start start 2024-04-02T13:53:38.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.160+0000 7fe8f907b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8f4100c50 0x7fe8f4199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.160+0000 7fe8f907b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8f4101610 0x7fe8f4199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.160+0000 7fe8f907b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8f41059a0 0x7fe8f41a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.160+0000 7fe8f907b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f4110ec0 con 0x7fe8f41059a0 2024-04-02T13:53:38.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.160+0000 7fe8f907b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f4110d40 con 0x7fe8f4100c50 2024-04-02T13:53:38.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.160+0000 7fe8f907b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f4111040 con 0x7fe8f4101610 2024-04-02T13:53:38.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.160+0000 7fe8f359e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8f41059a0 0x7fe8f41a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.160+0000 7fc9385e2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc930100970 0x7fc930197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.160+0000 7fe8f359e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8f41059a0 0x7fe8f41a42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58604/0 (socket says 172.21.15.52:58604) 2024-04-02T13:53:38.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.160+0000 7fe8f359e640 1 -- 172.21.15.52:0/3019528482 learned_addr learned my addr 172.21.15.52:0/3019528482 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:38.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.160+0000 7fc9385e2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc930101330 0x7fc930197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.160+0000 7fc9385e2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc930106980 0x7fc93019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.160+0000 7fc9385e2640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc930110d90 con 0x7fc930101330 2024-04-02T13:53:38.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.160+0000 7fc9385e2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc930110c10 con 0x7fc930106980 2024-04-02T13:53:38.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.160+0000 7fc9385e2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc930110f10 con 0x7fc930100970 2024-04-02T13:53:38.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.160+0000 7fe8f259c640 1 --2- 172.21.15.52:0/3019528482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8f4101610 0x7fe8f4199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.160+0000 7fc93637e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc930100970 0x7fc930197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.160+0000 7fc935b7d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc930101330 0x7fc930197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.160+0000 7fc93637e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc930100970 0x7fc930197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.52:34254/0 (socket says 172.21.15.52:34254) 2024-04-02T13:53:38.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.160+0000 7fc93637e640 1 -- 172.21.15.52:0/1302518545 learned_addr learned my addr 172.21.15.52:0/1302518545 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:38.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fc936b7f640 1 --2- 172.21.15.52:0/1302518545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc930106980 0x7fc93019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fe8f2d9d640 1 --2- 172.21.15.52:0/3019528482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8f4100c50 0x7fe8f4199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fe8f359e640 1 -- 172.21.15.52:0/3019528482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8f4101610 msgr2=0x7fe8f4199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fe8f359e640 1 --2- 172.21.15.52:0/3019528482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8f4101610 0x7fe8f4199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fe8f359e640 1 -- 172.21.15.52:0/3019528482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8f4100c50 msgr2=0x7fe8f4199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fe8f359e640 1 --2- 172.21.15.52:0/3019528482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8f4100c50 0x7fe8f4199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fe8f359e640 1 -- 172.21.15.52:0/3019528482 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f41a4800 con 0x7fe8f41059a0 2024-04-02T13:53:38.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fe8f2d9d640 1 --2- 172.21.15.52:0/3019528482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8f4100c50 0x7fe8f4199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:38.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fc93637e640 1 -- 172.21.15.52:0/1302518545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc930106980 msgr2=0x7fc93019bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fc93637e640 1 --2- 172.21.15.52:0/1302518545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc930106980 0x7fc93019bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fc93637e640 1 -- 172.21.15.52:0/1302518545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc930101330 msgr2=0x7fc930197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fc93637e640 1 --2- 172.21.15.52:0/1302518545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc930101330 0x7fc930197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fc93637e640 1 -- 172.21.15.52:0/1302518545 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93019c240 con 0x7fc930100970 2024-04-02T13:53:38.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fc935b7d640 1 --2- 172.21.15.52:0/1302518545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc930101330 0x7fc930197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:38.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fc936b7f640 1 --2- 172.21.15.52:0/1302518545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc930106980 0x7fc93019bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:38.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fe8f359e640 1 --2- 172.21.15.52:0/3019528482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8f41059a0 0x7fe8f41a42c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4005e30 tx=0x7fe8e4032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:38.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fc93637e640 1 --2- 172.21.15.52:0/1302518545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc930100970 0x7fc930197250 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00eb00 tx=0x7fc92c009f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:38.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fe8dbfff640 1 -- 172.21.15.52:0/3019528482 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4042020 con 0x7fe8f41059a0 2024-04-02T13:53:38.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fe8f907b640 1 -- 172.21.15.52:0/3019528482 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f41a49d0 con 0x7fe8f41059a0 2024-04-02T13:53:38.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fe8dbfff640 1 -- 172.21.15.52:0/3019528482 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e4034640 con 0x7fe8f41059a0 2024-04-02T13:53:38.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fe8dbfff640 1 -- 172.21.15.52:0/3019528482 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4039040 con 0x7fe8f41059a0 2024-04-02T13:53:38.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fe8f907b640 1 -- 172.21.15.52:0/3019528482 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f41a4e40 con 0x7fe8f41059a0 2024-04-02T13:53:38.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fc9277fe640 1 -- 172.21.15.52:0/1302518545 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c01c070 con 0x7fc930100970 2024-04-02T13:53:38.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fc9385e2640 1 -- 172.21.15.52:0/1302518545 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc93019c530 con 0x7fc930100970 2024-04-02T13:53:38.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fc9277fe640 1 -- 172.21.15.52:0/1302518545 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc92c00bc30 con 0x7fc930100970 2024-04-02T13:53:38.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fc9385e2640 1 -- 172.21.15.52:0/1302518545 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc930068c60 con 0x7fc930100970 2024-04-02T13:53:38.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.161+0000 7fc9277fe640 1 -- 172.21.15.52:0/1302518545 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c0166a0 con 0x7fc930100970 2024-04-02T13:53:38.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.163+0000 7fe8f907b640 1 -- 172.21.15.52:0/3019528482 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8f404f860 con 0x7fe8f41059a0 2024-04-02T13:53:38.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.163+0000 7fe8dbfff640 1 -- 172.21.15.52:0/3019528482 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fe8e4045020 con 0x7fe8f41059a0 2024-04-02T13:53:38.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.164+0000 7fc9277fe640 1 -- 172.21.15.52:0/1302518545 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fc92c01f070 con 0x7fc930100970 2024-04-02T13:53:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.164+0000 7fe8dbfff640 1 --2- 172.21.15.52:0/3019528482 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe8dc0a3ee0 0x7fe8dc0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.164+0000 7fe8dbfff640 1 -- 172.21.15.52:0/3019528482 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe8e40f2a60 con 0x7fe8f41059a0 2024-04-02T13:53:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.164+0000 7fe8f2d9d640 1 --2- 172.21.15.52:0/3019528482 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe8dc0a3ee0 0x7fe8dc0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.164+0000 7fc9277fe640 1 --2- 172.21.15.52:0/1302518545 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc91c0a3ec0 0x7fc91c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.164+0000 7fc93b1c6740 1 Processor -- start 2024-04-02T13:53:38.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.164+0000 7fc935b7d640 1 --2- 172.21.15.52:0/1302518545 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc91c0a3ec0 0x7fc91c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.164+0000 7fc9277fe640 1 -- 172.21.15.52:0/1302518545 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc92c0d22d0 con 0x7fc930100970 2024-04-02T13:53:38.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.165+0000 7fc93b1c6740 1 -- start start 2024-04-02T13:53:38.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.165+0000 7fe8f2d9d640 1 --2- 172.21.15.52:0/3019528482 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe8dc0a3ee0 0x7fe8dc0a63a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec005fd0 tx=0x7fe8ec005850 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:38.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.165+0000 7fc93b1c6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a516d0 0x55c930a51ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.165+0000 7fc93b1c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c930a51ff0 0x55c930a5a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.165+0000 7fc93b1c6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c930a5abd0 0x55c930a5cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.165+0000 7fc93b1c6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c9309b1770 con 0x55c930a516d0 2024-04-02T13:53:38.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.165+0000 7fc93b1c6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c930959960 con 0x55c930a5abd0 2024-04-02T13:53:38.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.165+0000 7fc93b1c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c9309ad8c0 con 0x55c930a51ff0 2024-04-02T13:53:38.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.165+0000 7fc935b7d640 1 --2- 172.21.15.52:0/1302518545 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc91c0a3ec0 0x7fc91c0a6380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc930105030 tx=0x7fc920005f90 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:38.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.165+0000 7fc93637e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a516d0 0x55c930a51ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.165+0000 7fc93637e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a516d0 0x55c930a51ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58618/0 (socket says 172.21.15.52:58618) 2024-04-02T13:53:38.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.165+0000 7fc93637e640 1 -- 172.21.15.52:0/302261475 learned_addr learned my addr 172.21.15.52:0/302261475 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:38.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.165+0000 7fc935b7d640 1 --2- 172.21.15.52:0/302261475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c930a51ff0 0x55c930a5a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.165+0000 7fc936b7f640 1 --2- 172.21.15.52:0/302261475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c930a5abd0 0x55c930a5cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.165+0000 7fc93637e640 1 -- 172.21.15.52:0/302261475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c930a51ff0 msgr2=0x55c930a5a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:38.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.165+0000 7fc93637e640 1 --2- 172.21.15.52:0/302261475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c930a51ff0 0x55c930a5a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.165+0000 7fc93637e640 1 -- 172.21.15.52:0/302261475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c930a5abd0 msgr2=0x55c930a5cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:38.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.165+0000 7fc93637e640 1 --2- 172.21.15.52:0/302261475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c930a5abd0 0x55c930a5cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.165+0000 7fc93637e640 1 -- 172.21.15.52:0/302261475 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c93078d7e0 con 0x55c930a516d0 2024-04-02T13:53:38.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.165+0000 7fc936b7f640 1 --2- 172.21.15.52:0/302261475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c930a5abd0 0x55c930a5cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:53:38.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.166+0000 7fc93637e640 1 --2- 172.21.15.52:0/302261475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a516d0 0x55c930a51ab0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc92c09a760 tx=0x7fc92c09d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a92f4bfc1bb178a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:38.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.169+0000 7fc924bfe640 1 -- 172.21.15.52:0/302261475 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c0a2090 con 0x55c930a516d0 2024-04-02T13:53:38.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.169+0000 7fc924bfe640 1 -- 172.21.15.52:0/302261475 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc92c0a6950 con 0x55c930a516d0 2024-04-02T13:53:38.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.169+0000 7fc924bfe640 1 -- 172.21.15.52:0/302261475 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c0cfaa0 con 0x55c930a516d0 2024-04-02T13:53:38.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.169+0000 7fe8dbfff640 1 -- 172.21.15.52:0/3019528482 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe8e404c020 con 0x7fe8f41059a0 2024-04-02T13:53:38.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.170+0000 7fc93b1c6740 1 -- 172.21.15.52:0/302261475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a516d0 msgr2=0x55c930a51ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.170+0000 7fc93b1c6740 1 --2- 172.21.15.52:0/302261475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a516d0 0x55c930a51ab0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc92c09a760 tx=0x7fc92c09d7b0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.170+0000 7fc93b1c6740 1 -- 172.21.15.52:0/302261475 shutdown_connections 2024-04-02T13:53:38.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.170+0000 7fc93b1c6740 1 --2- 172.21.15.52:0/302261475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c930a5abd0 0x55c930a5cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.170+0000 7fc93b1c6740 1 --2- 172.21.15.52:0/302261475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c930a51ff0 0x55c930a5a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.170+0000 7fc93b1c6740 1 --2- 172.21.15.52:0/302261475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a516d0 0x55c930a51ab0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.170+0000 7fc93b1c6740 1 -- 172.21.15.52:0/302261475 >> 172.21.15.52:0/302261475 conn(0x55c930a4be00 msgr2=0x55c930a4c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:38.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.170+0000 7fc93b1c6740 1 -- 172.21.15.52:0/302261475 shutdown_connections 2024-04-02T13:53:38.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.170+0000 7fc93b1c6740 1 -- 172.21.15.52:0/302261475 wait complete. 2024-04-02T13:53:38.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.170+0000 7fc93b1c6740 1 -- start start 2024-04-02T13:53:38.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.171+0000 7fc93b1c6740 1 Processor -- start 2024-04-02T13:53:38.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.171+0000 7fc93b1c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c930a516d0 0x55c930a57cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.171+0000 7fc93b1c6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a51ff0 0x55c930a58200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.171+0000 7fc93b1c6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c930a5abd0 0x55c930a58740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.171+0000 7fc93b1c6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c930a5d9c0 con 0x55c930a51ff0 2024-04-02T13:53:38.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.171+0000 7fc93b1c6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c93096cd20 con 0x55c930a5abd0 2024-04-02T13:53:38.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.171+0000 7fc93b1c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c930a5db40 con 0x55c930a516d0 2024-04-02T13:53:38.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.171+0000 7fc935b7d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a51ff0 0x55c930a58200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.171+0000 7fc935b7d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a51ff0 0x55c930a58200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58630/0 (socket says 172.21.15.52:58630) 2024-04-02T13:53:38.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.171+0000 7fc935b7d640 1 -- 172.21.15.52:0/964039046 learned_addr learned my addr 172.21.15.52:0/964039046 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:38.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.171+0000 7fc93b1c6740 10 client.0 _ll_register_callbacks cb 0x55c930939a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:38.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.171+0000 7fc93637e640 1 --2- 172.21.15.52:0/964039046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c930a516d0 0x55c930a57cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.171+0000 7fc936b7f640 1 --2- 172.21.15.52:0/964039046 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c930a5abd0 0x55c930a58740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.171+0000 7fc935b7d640 1 -- 172.21.15.52:0/964039046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c930a516d0 msgr2=0x55c930a57cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:38.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.171+0000 7fc935b7d640 1 --2- 172.21.15.52:0/964039046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c930a516d0 0x55c930a57cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.171+0000 7fc935b7d640 1 -- 172.21.15.52:0/964039046 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c930a5abd0 msgr2=0x55c930a58740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:38.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.171+0000 7fc935b7d640 1 --2- 172.21.15.52:0/964039046 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c930a5abd0 0x55c930a58740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.171+0000 7fc935b7d640 1 -- 172.21.15.52:0/964039046 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9307edf70 con 0x55c930a51ff0 2024-04-02T13:53:38.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.171+0000 7fc935b7d640 1 --2- 172.21.15.52:0/964039046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a51ff0 0x55c930a58200 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc9200035f0 tx=0x7fc920004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=243e8290f6e29964 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:38.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.171+0000 7fc919ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:53:38.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.172+0000 7fc919ffb640 1 -- 172.21.15.52:0/964039046 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92000fa90 con 0x55c930a51ff0 2024-04-02T13:53:38.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.172+0000 7fc919ffb640 1 -- 172.21.15.52:0/964039046 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc92000fc30 con 0x55c930a51ff0 2024-04-02T13:53:38.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.172+0000 7fc93b1c6740 1 -- 172.21.15.52:0/964039046 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc92c0a6950 con 0x55c930a51ff0 2024-04-02T13:53:38.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.172+0000 7fc919ffb640 1 -- 172.21.15.52:0/964039046 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc920030050 con 0x55c930a51ff0 2024-04-02T13:53:38.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.172+0000 7fc93b1c6740 10 client.15744 fetch_fsmap learned FSMap version 28 2024-04-02T13:53:38.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.172+0000 7fc93b1c6740 1 -- 172.21.15.52:0/964039046 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c930685170 con 0x55c930a51ff0 2024-04-02T13:53:38.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.172+0000 7fc919ffb640 1 -- 172.21.15.52:0/964039046 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc92c0a6950 con 0x55c930a51ff0 2024-04-02T13:53:38.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.172+0000 7fc919ffb640 1 -- 172.21.15.52:0/964039046 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc92001a040 con 0x55c930a51ff0 2024-04-02T13:53:38.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.172+0000 7fc93b1c6740 10 client.15744 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:53:38.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.172+0000 7fc93b1c6740 10 client.15744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:53:38.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.172+0000 7fc93b1c6740 20 client.15744 populate_metadata read hostname 'smithi052' 2024-04-02T13:53:38.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.172+0000 7fc93b1c6740 1 --2- 172.21.15.52:0/964039046 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c930a6afe0 0x55c930a6d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.172+0000 7fc93b1c6740 4 client.15744 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:53:38.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.172+0000 7fc93b1c6740 1 -- 172.21.15.52:0/964039046 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9309becb0 con 0x55c930a6afe0 2024-04-02T13:53:38.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.172+0000 7fc93637e640 1 --2- 172.21.15.52:0/964039046 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c930a6afe0 0x55c930a6d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.173+0000 7fc93637e640 1 --2- 172.21.15.52:0/964039046 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c930a6afe0 0x55c930a6d3f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9626d79c4791ce4d server_cookie=787224f539ea135 in_seq=0 out_seq=0 2024-04-02T13:53:38.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.173+0000 7fc919ffb640 10 client.15744 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:53:38.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.173+0000 7fc919ffb640 1 -- 172.21.15.52:0/964039046 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c9309becb0 con 0x55c930a6afe0 2024-04-02T13:53:38.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.173+0000 7fc919ffb640 10 client.15744 handle_command_reply: tid=0 2024-04-02T13:53:38.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.174+0000 7fc93b1c6740 1 client.15744 shutdown 2024-04-02T13:53:38.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.174+0000 7fc93b1c6740 1 -- 172.21.15.52:0/964039046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a51ff0 msgr2=0x55c930a58200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.174+0000 7fc93b1c6740 1 --2- 172.21.15.52:0/964039046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a51ff0 0x55c930a58200 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc9200035f0 tx=0x7fc920004730 comp rx=0 tx=0).stop 2024-04-02T13:53:38.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.174+0000 7fc93b1c6740 1 -- 172.21.15.52:0/964039046 shutdown_connections 2024-04-02T13:53:38.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.174+0000 7fc93b1c6740 1 --2- 172.21.15.52:0/964039046 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c930a5abd0 0x55c930a58740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.174+0000 7fc93b1c6740 1 --2- 172.21.15.52:0/964039046 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c930a6afe0 0x55c930a6d3f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.174+0000 7fc93b1c6740 1 --2- 172.21.15.52:0/964039046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a51ff0 0x55c930a58200 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.174+0000 7fc919ffb640 0 client.15744 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:53:38.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.174+0000 7fc93b1c6740 1 --2- 172.21.15.52:0/964039046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c930a516d0 0x55c930a57cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.174+0000 7fc93b1c6740 1 -- 172.21.15.52:0/964039046 >> 172.21.15.52:0/964039046 conn(0x55c930a4be00 msgr2=0x55c930a4d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:38.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.174+0000 7fc93b1c6740 1 -- 172.21.15.52:0/964039046 shutdown_connections 2024-04-02T13:53:38.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.174+0000 7fc93b1c6740 1 -- 172.21.15.52:0/964039046 wait complete. 2024-04-02T13:53:38.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.174+0000 7fc93b1c6740 20 client.15744 trim_cache size 0 max 16384 2024-04-02T13:53:38.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.191+0000 7fc93b1c6740 1 Processor -- start 2024-04-02T13:53:38.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.191+0000 7fc93b1c6740 1 -- start start 2024-04-02T13:53:38.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.192+0000 7fc93b1c6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a516d0 0x55c930a66880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.192+0000 7fc93b1c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c930a51ff0 0x55c930a4d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.192+0000 7fc93b1c6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c930a5abd0 0x55c930a46930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.192+0000 7fc93b1c6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc9301acc50 con 0x55c930a516d0 2024-04-02T13:53:38.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.192+0000 7fc93b1c6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc9301ae790 con 0x55c930a5abd0 2024-04-02T13:53:38.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.192+0000 7fc93b1c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc9301ad320 con 0x55c930a51ff0 2024-04-02T13:53:38.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.192+0000 7fc93637e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a516d0 0x55c930a66880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.192+0000 7fc93637e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a516d0 0x55c930a66880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58634/0 (socket says 172.21.15.52:58634) 2024-04-02T13:53:38.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.192+0000 7fc93637e640 1 -- 172.21.15.52:0/870523811 learned_addr learned my addr 172.21.15.52:0/870523811 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:38.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.192+0000 7fc935b7d640 1 --2- 172.21.15.52:0/870523811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c930a51ff0 0x55c930a4d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.192+0000 7fc93637e640 1 -- 172.21.15.52:0/870523811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c930a51ff0 msgr2=0x55c930a4d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:38.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.192+0000 7fc936b7f640 1 --2- 172.21.15.52:0/870523811 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c930a5abd0 0x55c930a46930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.192+0000 7fc93637e640 1 --2- 172.21.15.52:0/870523811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c930a51ff0 0x55c930a4d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.192+0000 7fc93637e640 1 -- 172.21.15.52:0/870523811 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c930a5abd0 msgr2=0x55c930a46930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:38.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.192+0000 7fc93637e640 1 --2- 172.21.15.52:0/870523811 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c930a5abd0 0x55c930a46930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.192+0000 7fc93637e640 1 -- 172.21.15.52:0/870523811 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c930a51e20 con 0x55c930a516d0 2024-04-02T13:53:38.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.192+0000 7fc93637e640 1 --2- 172.21.15.52:0/870523811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a516d0 0x55c930a66880 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc92c09da90 tx=0x7fc92c0b4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e5de66e14154d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:38.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.192+0000 7fc91bfff640 1 -- 172.21.15.52:0/870523811 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c09dd90 con 0x55c930a516d0 2024-04-02T13:53:38.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.192+0000 7fc91bfff640 1 -- 172.21.15.52:0/870523811 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc92c09a400 con 0x55c930a516d0 2024-04-02T13:53:38.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.192+0000 7fc91bfff640 1 -- 172.21.15.52:0/870523811 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c09d1e0 con 0x55c930a516d0 2024-04-02T13:53:38.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.193+0000 7fc93b1c6740 1 -- 172.21.15.52:0/870523811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a516d0 msgr2=0x55c930a66880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.193+0000 7fc93b1c6740 1 --2- 172.21.15.52:0/870523811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a516d0 0x55c930a66880 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc92c09da90 tx=0x7fc92c0b4040 comp rx=0 tx=0).stop 2024-04-02T13:53:38.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.193+0000 7fc93b1c6740 1 -- 172.21.15.52:0/870523811 shutdown_connections 2024-04-02T13:53:38.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.193+0000 7fc93b1c6740 1 --2- 172.21.15.52:0/870523811 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c930a5abd0 0x55c930a46930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.193+0000 7fc93b1c6740 1 --2- 172.21.15.52:0/870523811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c930a51ff0 0x55c930a4d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.193+0000 7fc93b1c6740 1 --2- 172.21.15.52:0/870523811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a516d0 0x55c930a66880 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.193+0000 7fc93b1c6740 1 -- 172.21.15.52:0/870523811 >> 172.21.15.52:0/870523811 conn(0x55c930a4be00 msgr2=0x55c930a46220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:38.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.193+0000 7fc93b1c6740 1 -- 172.21.15.52:0/870523811 shutdown_connections 2024-04-02T13:53:38.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.193+0000 7fc93b1c6740 1 -- 172.21.15.52:0/870523811 wait complete. 2024-04-02T13:53:38.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.193+0000 7fc93b1c6740 1 -- start start 2024-04-02T13:53:38.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.194+0000 7fc93b1c6740 1 Processor -- start 2024-04-02T13:53:38.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.194+0000 7fc93b1c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c930a516d0 0x55c930a5fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.194+0000 7fc93b1c6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c930a51ff0 0x55c930a602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.194+0000 7fc93b1c6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a5abd0 0x55c930a607e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.194+0000 7fc93b1c6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc9301ae0c0 con 0x55c930a5abd0 2024-04-02T13:53:38.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.194+0000 7fc93b1c6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc9301ad9f0 con 0x55c930a51ff0 2024-04-02T13:53:38.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.194+0000 7fc93b1c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc9301aac30 con 0x55c930a516d0 2024-04-02T13:53:38.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.194+0000 7fc93b1c6740 10 client.0 _ll_register_callbacks cb 0x7fc920009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:38.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.194+0000 7fc936b7f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a5abd0 0x55c930a607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.194+0000 7fc936b7f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a5abd0 0x55c930a607e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58640/0 (socket says 172.21.15.52:58640) 2024-04-02T13:53:38.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.194+0000 7fc936b7f640 1 -- 172.21.15.52:0/2628835981 learned_addr learned my addr 172.21.15.52:0/2628835981 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:38.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.194+0000 7fc93637e640 1 --2- 172.21.15.52:0/2628835981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c930a516d0 0x55c930a5fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.194+0000 7fc935b7d640 1 --2- 172.21.15.52:0/2628835981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c930a51ff0 0x55c930a602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.194+0000 7fc936b7f640 1 -- 172.21.15.52:0/2628835981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c930a516d0 msgr2=0x55c930a5fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:38.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.194+0000 7fc936b7f640 1 --2- 172.21.15.52:0/2628835981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c930a516d0 0x55c930a5fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.194+0000 7fc936b7f640 1 -- 172.21.15.52:0/2628835981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c930a51ff0 msgr2=0x55c930a602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:38.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.194+0000 7fc936b7f640 1 --2- 172.21.15.52:0/2628835981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c930a51ff0 0x55c930a602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.194+0000 7fc936b7f640 1 -- 172.21.15.52:0/2628835981 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c930ad1570 con 0x55c930a5abd0 2024-04-02T13:53:38.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.194+0000 7fc936b7f640 1 --2- 172.21.15.52:0/2628835981 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a5abd0 0x55c930a607e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc928009ca0 tx=0x7fc92802cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e132b9bc0dd38a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:38.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.194+0000 7fc919ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:53:38.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.195+0000 7fc919ffb640 1 -- 172.21.15.52:0/2628835981 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc928045070 con 0x55c930a5abd0 2024-04-02T13:53:38.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.195+0000 7fc93b1c6740 1 -- 172.21.15.52:0/2628835981 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc92c09a400 con 0x55c930a5abd0 2024-04-02T13:53:38.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.195+0000 7fc919ffb640 1 -- 172.21.15.52:0/2628835981 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc928032590 con 0x55c930a5abd0 2024-04-02T13:53:38.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.195+0000 7fc919ffb640 1 -- 172.21.15.52:0/2628835981 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc928052050 con 0x55c930a5abd0 2024-04-02T13:53:38.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.195+0000 7fc93b1c6740 10 client.15750 fetch_fsmap learned FSMap version 28 2024-04-02T13:53:38.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.195+0000 7fc93b1c6740 1 -- 172.21.15.52:0/2628835981 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c930ad28b0 con 0x55c930a5abd0 2024-04-02T13:53:38.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.195+0000 7fc919ffb640 1 -- 172.21.15.52:0/2628835981 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc92c09a400 con 0x55c930a5abd0 2024-04-02T13:53:38.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.195+0000 7fc919ffb640 1 -- 172.21.15.52:0/2628835981 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc928034970 con 0x55c930a5abd0 2024-04-02T13:53:38.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.195+0000 7fc93b1c6740 10 client.15750 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:53:38.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.195+0000 7fc93b1c6740 10 client.15750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:53:38.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.195+0000 7fc93b1c6740 20 client.15750 populate_metadata read hostname 'smithi052' 2024-04-02T13:53:38.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.195+0000 7fc93b1c6740 1 --2- 172.21.15.52:0/2628835981 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c930ad3020 0x55c930ad5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.195+0000 7fc93b1c6740 4 client.15750 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:53:38.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.195+0000 7fc93b1c6740 1 -- 172.21.15.52:0/2628835981 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c9309b9000 con 0x55c930ad3020 2024-04-02T13:53:38.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.196+0000 7fc93637e640 1 --2- 172.21.15.52:0/2628835981 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c930ad3020 0x55c930ad5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.196+0000 7fc93637e640 1 --2- 172.21.15.52:0/2628835981 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c930ad3020 0x55c930ad5410 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57fbda694f927fd8 server_cookie=891e26781068dee1 in_seq=0 out_seq=0 2024-04-02T13:53:38.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.196+0000 7fc919ffb640 10 client.15750 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:53:38.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.197+0000 7fc919ffb640 1 -- 172.21.15.52:0/2628835981 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c9309b9000 con 0x55c930ad3020 2024-04-02T13:53:38.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.197+0000 7fc919ffb640 10 client.15750 handle_command_reply: tid=0 2024-04-02T13:53:38.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.197+0000 7fc93b1c6740 1 client.15750 shutdown 2024-04-02T13:53:38.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.198+0000 7fc93b1c6740 1 -- 172.21.15.52:0/2628835981 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a5abd0 msgr2=0x55c930a607e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.198+0000 7fc93b1c6740 1 --2- 172.21.15.52:0/2628835981 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a5abd0 0x55c930a607e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc928009ca0 tx=0x7fc92802cca0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.198+0000 7fc93b1c6740 1 -- 172.21.15.52:0/2628835981 shutdown_connections 2024-04-02T13:53:38.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.198+0000 7fc93b1c6740 1 --2- 172.21.15.52:0/2628835981 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c930ad3020 0x55c930ad5410 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.198+0000 7fc93b1c6740 1 --2- 172.21.15.52:0/2628835981 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c930a5abd0 0x55c930a607e0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.198+0000 7fc93b1c6740 1 --2- 172.21.15.52:0/2628835981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c930a51ff0 0x55c930a602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.198+0000 7fc919ffb640 0 client.15750 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:53:38.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.198+0000 7fc93b1c6740 1 --2- 172.21.15.52:0/2628835981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c930a516d0 0x55c930a5fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.198+0000 7fc93b1c6740 1 -- 172.21.15.52:0/2628835981 >> 172.21.15.52:0/2628835981 conn(0x55c930a4be00 msgr2=0x55c930a4d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:38.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.198+0000 7fc93b1c6740 1 -- 172.21.15.52:0/2628835981 shutdown_connections 2024-04-02T13:53:38.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.198+0000 7fc93b1c6740 1 -- 172.21.15.52:0/2628835981 wait complete. 2024-04-02T13:53:38.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.198+0000 7fc93b1c6740 20 client.15750 trim_cache size 0 max 16384 2024-04-02T13:53:38.209 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:53:38.209 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-04-02T13:53:38.209 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "fcb7243d-2196-46d8-bc6a-ee3dcb73c06a", 2024-04-02T13:53:38.209 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-04-02T13:53:38.209 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:53:38.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.198+0000 7fc9385e2640 1 -- 172.21.15.52:0/1302518545 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc91c0a3ec0 msgr2=0x7fc91c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.198+0000 7fc9385e2640 1 --2- 172.21.15.52:0/1302518545 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc91c0a3ec0 0x7fc91c0a6380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc930105030 tx=0x7fc920005f90 comp rx=0 tx=0).stop 2024-04-02T13:53:38.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.198+0000 7fc9385e2640 1 -- 172.21.15.52:0/1302518545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc930100970 msgr2=0x7fc930197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.198+0000 7fc9385e2640 1 --2- 172.21.15.52:0/1302518545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc930100970 0x7fc930197250 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00eb00 tx=0x7fc92c009f20 comp rx=0 tx=0).stop 2024-04-02T13:53:38.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.198+0000 7fc9385e2640 1 -- 172.21.15.52:0/1302518545 shutdown_connections 2024-04-02T13:53:38.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.198+0000 7fc9385e2640 1 --2- 172.21.15.52:0/1302518545 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc91c0a3ec0 0x7fc91c0a6380 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.198+0000 7fc9385e2640 1 --2- 172.21.15.52:0/1302518545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc930106980 0x7fc93019bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.198+0000 7fc9385e2640 1 --2- 172.21.15.52:0/1302518545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc930101330 0x7fc930197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.198+0000 7fc9385e2640 1 --2- 172.21.15.52:0/1302518545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc930100970 0x7fc930197250 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.198+0000 7fc9385e2640 1 -- 172.21.15.52:0/1302518545 >> 172.21.15.52:0/1302518545 conn(0x7fc930077b00 msgr2=0x7fc930103170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:38.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.199+0000 7fc9385e2640 1 -- 172.21.15.52:0/1302518545 shutdown_connections 2024-04-02T13:53:38.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.199+0000 7fc9385e2640 1 -- 172.21.15.52:0/1302518545 wait complete. 2024-04-02T13:53:38.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:53:38.212 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:53:38.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:37 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3966271530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:53:38.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:37 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2086769883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:53:38.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:37 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3561269454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:53:38.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.314+0000 7f9767f90640 1 -- 172.21.15.52:0/2558622473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9760102460 msgr2=0x7f976010e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.314+0000 7f9767f90640 1 --2- 172.21.15.52:0/2558622473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9760102460 0x7f976010e970 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9758009cd0 tx=0x7f975802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.314+0000 7f9767f90640 1 -- 172.21.15.52:0/2558622473 shutdown_connections 2024-04-02T13:53:38.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.314+0000 7f9767f90640 1 --2- 172.21.15.52:0/2558622473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9760102460 0x7f976010e970 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.314+0000 7f9767f90640 1 --2- 172.21.15.52:0/2558622473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9760101aa0 0x7f9760101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.314+0000 7f9767f90640 1 --2- 172.21.15.52:0/2558622473 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9760107ab0 0x7f9760107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.314+0000 7f9767f90640 1 -- 172.21.15.52:0/2558622473 >> 172.21.15.52:0/2558622473 conn(0x7f97600fd590 msgr2=0x7f97600ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:38.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.314+0000 7f9767f90640 1 -- 172.21.15.52:0/2558622473 shutdown_connections 2024-04-02T13:53:38.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.314+0000 7f9767f90640 1 -- 172.21.15.52:0/2558622473 wait complete. 2024-04-02T13:53:38.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.315+0000 7f9767f90640 1 Processor -- start 2024-04-02T13:53:38.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.315+0000 7f9767f90640 1 -- start start 2024-04-02T13:53:38.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.315+0000 7f9767f90640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9760101aa0 0x7f9760073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.315+0000 7f9767f90640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9760102460 0x7f9760074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.315+0000 7f9767f90640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9760107ab0 0x7f9760074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.315+0000 7f9767f90640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9760111ec0 con 0x7f9760107ab0 2024-04-02T13:53:38.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.315+0000 7f9767f90640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9760111d40 con 0x7f9760101aa0 2024-04-02T13:53:38.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.315+0000 7f9767f90640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9760112040 con 0x7f9760102460 2024-04-02T13:53:38.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.315+0000 7f976652d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9760107ab0 0x7f9760074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.315+0000 7f976652d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9760107ab0 0x7f9760074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58666/0 (socket says 172.21.15.52:58666) 2024-04-02T13:53:38.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.315+0000 7f976652d640 1 -- 172.21.15.52:0/3811695773 learned_addr learned my addr 172.21.15.52:0/3811695773 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:38.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.315+0000 7f976552b640 1 --2- 172.21.15.52:0/3811695773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9760102460 0x7f9760074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.316+0000 7f9765d2c640 1 --2- 172.21.15.52:0/3811695773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9760101aa0 0x7f9760073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.316+0000 7f976652d640 1 -- 172.21.15.52:0/3811695773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9760102460 msgr2=0x7f9760074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.316+0000 7f976652d640 1 --2- 172.21.15.52:0/3811695773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9760102460 0x7f9760074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.316+0000 7f976652d640 1 -- 172.21.15.52:0/3811695773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9760101aa0 msgr2=0x7f9760073e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.316+0000 7f976652d640 1 --2- 172.21.15.52:0/3811695773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9760101aa0 0x7f9760073e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.316+0000 7f976652d640 1 -- 172.21.15.52:0/3811695773 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97601a68a0 con 0x7f9760107ab0 2024-04-02T13:53:38.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.316+0000 7f976652d640 1 --2- 172.21.15.52:0/3811695773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9760107ab0 0x7f9760074880 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9758004af0 tx=0x7f9758032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:38.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.316+0000 7f9756ffd640 1 -- 172.21.15.52:0/3811695773 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758041020 con 0x7f9760107ab0 2024-04-02T13:53:38.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.316+0000 7f9756ffd640 1 -- 172.21.15.52:0/3811695773 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9758034cd0 con 0x7f9760107ab0 2024-04-02T13:53:38.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.316+0000 7f9767f90640 1 -- 172.21.15.52:0/3811695773 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97601a6b30 con 0x7f9760107ab0 2024-04-02T13:53:38.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.316+0000 7f9756ffd640 1 -- 172.21.15.52:0/3811695773 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758039040 con 0x7f9760107ab0 2024-04-02T13:53:38.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.316+0000 7f9767f90640 1 -- 172.21.15.52:0/3811695773 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97601a7070 con 0x7f9760107ab0 2024-04-02T13:53:38.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.319+0000 7f976ab78740 1 Processor -- start 2024-04-02T13:53:38.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.319+0000 7f9756ffd640 1 -- 172.21.15.52:0/3811695773 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f9758044020 con 0x7f9760107ab0 2024-04-02T13:53:38.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.320+0000 7f976ab78740 1 -- start start 2024-04-02T13:53:38.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.320+0000 7f976ab78740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b700108d0 0x561b70010cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.320+0000 7f976ab78740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b700111f0 0x561b70019890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.320+0000 7f976ab78740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b70019dd0 0x561b7001c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.320+0000 7f976ab78740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561b6ff74830 con 0x561b700111f0 2024-04-02T13:53:38.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.320+0000 7f976ab78740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x561b6ff1c820 con 0x561b700108d0 2024-04-02T13:53:38.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.320+0000 7f976ab78740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561b6ff707e0 con 0x561b70019dd0 2024-04-02T13:53:38.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.320+0000 7f9765d2c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b700111f0 0x561b70019890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.320+0000 7f9765d2c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b700111f0 0x561b70019890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58674/0 (socket says 172.21.15.52:58674) 2024-04-02T13:53:38.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.320+0000 7f9765d2c640 1 -- 172.21.15.52:0/3148529323 learned_addr learned my addr 172.21.15.52:0/3148529323 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:38.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.320+0000 7f9756ffd640 1 --2- 172.21.15.52:0/3811695773 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f974c0a3ec0 0x7f974c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.320+0000 7f976552b640 1 --2- 172.21.15.52:0/3148529323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b700108d0 0x561b70010cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.320+0000 7f976552b640 1 --2- 172.21.15.52:0/3148529323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b70019dd0 0x561b7001c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.320+0000 7f976652d640 1 --2- 172.21.15.52:0/3811695773 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f974c0a3ec0 0x7f974c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.320+0000 7f9765d2c640 1 -- 172.21.15.52:0/3148529323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b70019dd0 msgr2=0x561b7001c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:38.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.320+0000 7f9765d2c640 1 --2- 172.21.15.52:0/3148529323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b70019dd0 0x561b7001c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.320+0000 7f9765d2c640 1 -- 172.21.15.52:0/3148529323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b700108d0 msgr2=0x561b70010cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:38.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.320+0000 7f9765d2c640 1 --2- 172.21.15.52:0/3148529323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b700108d0 0x561b70010cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.320+0000 7f9756ffd640 1 -- 172.21.15.52:0/3811695773 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f97580f2930 con 0x7f9760107ab0 2024-04-02T13:53:38.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.320+0000 7f9765d2c640 1 -- 172.21.15.52:0/3148529323 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b6fd527d0 con 0x561b700111f0 2024-04-02T13:53:38.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.321+0000 7f9765d2c640 1 --2- 172.21.15.52:0/3148529323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b700111f0 0x561b70019890 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f975c009c70 tx=0x7f975c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85994fb21259415 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:38.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.321+0000 7f976652d640 1 --2- 172.21.15.52:0/3811695773 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f974c0a3ec0 0x7f974c0a6380 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f97580efd30 tx=0x7f975802e040 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:38.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.321+0000 7f974bfff640 1 -- 172.21.15.52:0/3148529323 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c01e070 con 0x561b700111f0 2024-04-02T13:53:38.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.321+0000 7f974bfff640 1 -- 172.21.15.52:0/3148529323 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f975c009e20 con 0x561b700111f0 2024-04-02T13:53:38.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.321+0000 7f974bfff640 1 -- 172.21.15.52:0/3148529323 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c019670 con 0x561b700111f0 2024-04-02T13:53:38.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.321+0000 7f976ab78740 1 -- 172.21.15.52:0/3148529323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b700111f0 msgr2=0x561b70019890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.321+0000 7f976ab78740 1 --2- 172.21.15.52:0/3148529323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b700111f0 0x561b70019890 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f975c009c70 tx=0x7f975c011810 comp rx=0 tx=0).stop 2024-04-02T13:53:38.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.322+0000 7f976ab78740 1 -- 172.21.15.52:0/3148529323 shutdown_connections 2024-04-02T13:53:38.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.322+0000 7f976ab78740 1 --2- 172.21.15.52:0/3148529323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b70019dd0 0x561b7001c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.322+0000 7f976ab78740 1 --2- 172.21.15.52:0/3148529323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b700111f0 0x561b70019890 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.322+0000 7f976ab78740 1 --2- 172.21.15.52:0/3148529323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b700108d0 0x561b70010cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.322+0000 7f976ab78740 1 -- 172.21.15.52:0/3148529323 >> 172.21.15.52:0/3148529323 conn(0x561b7000b000 msgr2=0x561b7000b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:38.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.322+0000 7f976ab78740 1 -- 172.21.15.52:0/3148529323 shutdown_connections 2024-04-02T13:53:38.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.322+0000 7f976ab78740 1 -- 172.21.15.52:0/3148529323 wait complete. 2024-04-02T13:53:38.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.322+0000 7f976ab78740 1 -- start start 2024-04-02T13:53:38.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.322+0000 7f976ab78740 1 Processor -- start 2024-04-02T13:53:38.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.322+0000 7f976ab78740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b700108d0 0x561b70019660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.322+0000 7f976ab78740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b700111f0 0x561b700169f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.322+0000 7f976ab78740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b70019dd0 0x561b70016f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.322+0000 7f976ab78740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561b7001cbc0 con 0x561b700111f0 2024-04-02T13:53:38.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.322+0000 7f976ab78740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x561b6ff2fd30 con 0x561b700108d0 2024-04-02T13:53:38.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.322+0000 7f976ab78740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561b7001cd40 con 0x561b70019dd0 2024-04-02T13:53:38.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.323+0000 7f976ab78740 10 client.0 _ll_register_callbacks cb 0x561b6ff2f720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:38.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.323+0000 7f9765d2c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b700111f0 0x561b700169f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.323+0000 7f9765d2c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b700111f0 0x561b700169f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58684/0 (socket says 172.21.15.52:58684) 2024-04-02T13:53:38.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.323+0000 7f9765d2c640 1 -- 172.21.15.52:0/2698716047 learned_addr learned my addr 172.21.15.52:0/2698716047 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:38.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.323+0000 7f976552b640 1 --2- 172.21.15.52:0/2698716047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b70019dd0 0x561b70016f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.323+0000 7f976552b640 1 --2- 172.21.15.52:0/2698716047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b700108d0 0x561b70019660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.323+0000 7f9765d2c640 1 -- 172.21.15.52:0/2698716047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b70019dd0 msgr2=0x561b70016f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:38.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.323+0000 7f9765d2c640 1 --2- 172.21.15.52:0/2698716047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b70019dd0 0x561b70016f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.323+0000 7f9765d2c640 1 -- 172.21.15.52:0/2698716047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b700108d0 msgr2=0x561b70019660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:38.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.323+0000 7f9765d2c640 1 --2- 172.21.15.52:0/2698716047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b700108d0 0x561b70019660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.323+0000 7f9765d2c640 1 -- 172.21.15.52:0/2698716047 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b6fc883f0 con 0x561b700111f0 2024-04-02T13:53:38.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.323+0000 7f9765d2c640 1 --2- 172.21.15.52:0/2698716047 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b700111f0 0x561b700169f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f975c0097e0 tx=0x7f975c005290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0b58ef53cd026c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:38.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.323+0000 7f97497fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:53:38.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.323+0000 7f97497fa640 1 -- 172.21.15.52:0/2698716047 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c027020 con 0x561b700111f0 2024-04-02T13:53:38.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.323+0000 7f97497fa640 1 -- 172.21.15.52:0/2698716047 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f975c019430 con 0x561b700111f0 2024-04-02T13:53:38.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.323+0000 7f976ab78740 1 -- 172.21.15.52:0/2698716047 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f975c009e20 con 0x561b700111f0 2024-04-02T13:53:38.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.324+0000 7f97497fa640 1 -- 172.21.15.52:0/2698716047 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f975c01e040 con 0x561b700111f0 2024-04-02T13:53:38.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.324+0000 7f976ab78740 10 client.15762 fetch_fsmap learned FSMap version 28 2024-04-02T13:53:38.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.324+0000 7f976ab78740 1 -- 172.21.15.52:0/2698716047 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b6fc480f0 con 0x561b700111f0 2024-04-02T13:53:38.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.324+0000 7f97497fa640 1 -- 172.21.15.52:0/2698716047 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f975c009e20 con 0x561b700111f0 2024-04-02T13:53:38.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.324+0000 7f97497fa640 1 -- 172.21.15.52:0/2698716047 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f975c023410 con 0x561b700111f0 2024-04-02T13:53:38.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.324+0000 7f976ab78740 10 client.15762 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:53:38.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.324+0000 7f976ab78740 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:53:38.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.324+0000 7f976ab78740 20 client.15762 populate_metadata read hostname 'smithi052' 2024-04-02T13:53:38.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.324+0000 7f976ab78740 1 --2- 172.21.15.52:0/2698716047 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x561b70029d00 0x561b7002c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.324+0000 7f976ab78740 4 client.15762 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:53:38.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.324+0000 7f976ab78740 1 -- 172.21.15.52:0/2698716047 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b6ff81d70 con 0x561b70029d00 2024-04-02T13:53:38.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.324+0000 7f976552b640 1 --2- 172.21.15.52:0/2698716047 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x561b70029d00 0x561b7002c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.325+0000 7f976552b640 1 --2- 172.21.15.52:0/2698716047 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x561b70029d00 0x561b7002c0f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1ebfcfc30aed889 server_cookie=b64d77cb56b48b83 in_seq=0 out_seq=0 2024-04-02T13:53:38.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.325+0000 7f97497fa640 10 client.15762 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:53:38.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.325+0000 7f97497fa640 1 -- 172.21.15.52:0/2698716047 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561b6ff81d70 con 0x561b70029d00 2024-04-02T13:53:38.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.325+0000 7f97497fa640 10 client.15762 handle_command_reply: tid=0 2024-04-02T13:53:38.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.325+0000 7f976ab78740 1 client.15762 shutdown 2024-04-02T13:53:38.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.325+0000 7f976ab78740 1 -- 172.21.15.52:0/2698716047 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b700111f0 msgr2=0x561b700169f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.325+0000 7f976ab78740 1 --2- 172.21.15.52:0/2698716047 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b700111f0 0x561b700169f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f975c0097e0 tx=0x7f975c005290 comp rx=0 tx=0).stop 2024-04-02T13:53:38.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.326+0000 7f976ab78740 1 -- 172.21.15.52:0/2698716047 shutdown_connections 2024-04-02T13:53:38.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.326+0000 7f976ab78740 1 --2- 172.21.15.52:0/2698716047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b70019dd0 0x561b70016f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.326+0000 7f976ab78740 1 --2- 172.21.15.52:0/2698716047 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x561b70029d00 0x561b7002c0f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.326+0000 7f976ab78740 1 --2- 172.21.15.52:0/2698716047 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b700111f0 0x561b700169f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.326+0000 7f976ab78740 1 --2- 172.21.15.52:0/2698716047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b700108d0 0x561b70019660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.326+0000 7f97497fa640 0 client.15762 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:53:38.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.326+0000 7f976ab78740 1 -- 172.21.15.52:0/2698716047 >> 172.21.15.52:0/2698716047 conn(0x561b7000b000 msgr2=0x561b7000cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:38.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.326+0000 7f976ab78740 1 -- 172.21.15.52:0/2698716047 shutdown_connections 2024-04-02T13:53:38.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.326+0000 7f976ab78740 1 -- 172.21.15.52:0/2698716047 wait complete. 2024-04-02T13:53:38.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.326+0000 7f976ab78740 20 client.15762 trim_cache size 0 max 16384 2024-04-02T13:53:38.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.343+0000 7f976ab78740 1 Processor -- start 2024-04-02T13:53:38.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.343+0000 7f976ab78740 1 -- start start 2024-04-02T13:53:38.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.343+0000 7f976ab78740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b700108d0 0x561b70025ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.343+0000 7f976ab78740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b700111f0 0x561b7000a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.343+0000 7f976ab78740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b70019dd0 0x561b700317b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.343+0000 7f976ab78740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f97601b10e0 con 0x561b700108d0 2024-04-02T13:53:38.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.343+0000 7f976ab78740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f97601b2c20 con 0x561b700111f0 2024-04-02T13:53:38.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.343+0000 7f976ab78740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f97601b17b0 con 0x561b70019dd0 2024-04-02T13:53:38.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.344+0000 7f976552b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b700108d0 0x561b70025ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.344+0000 7f976552b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b700108d0 0x561b70025ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58692/0 (socket says 172.21.15.52:58692) 2024-04-02T13:53:38.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.344+0000 7f976552b640 1 -- 172.21.15.52:0/2494299806 learned_addr learned my addr 172.21.15.52:0/2494299806 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:38.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.344+0000 7f976552b640 1 --2- 172.21.15.52:0/2494299806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b70019dd0 0x561b700317b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.344+0000 7f976552b640 1 -- 172.21.15.52:0/2494299806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b70019dd0 msgr2=0x561b700317b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:38.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.344+0000 7f976552b640 1 --2- 172.21.15.52:0/2494299806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b70019dd0 0x561b700317b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.344+0000 7f976552b640 1 -- 172.21.15.52:0/2494299806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b700111f0 msgr2=0x561b7000a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:38.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.344+0000 7f9765d2c640 1 --2- 172.21.15.52:0/2494299806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b700111f0 0x561b7000a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.344+0000 7f976552b640 1 --2- 172.21.15.52:0/2494299806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b700111f0 0x561b7000a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.344+0000 7f976552b640 1 -- 172.21.15.52:0/2494299806 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b7000ae30 con 0x561b700108d0 2024-04-02T13:53:38.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.344+0000 7f9765d2c640 1 --2- 172.21.15.52:0/2494299806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b700111f0 0x561b7000a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:53:38.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.344+0000 7f976552b640 1 --2- 172.21.15.52:0/2494299806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b700108d0 0x561b70025ab0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9750005d00 tx=0x7f97500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac4ce81d2914939 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:38.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.344+0000 7f974b7fe640 1 -- 172.21.15.52:0/2494299806 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975002e070 con 0x561b700108d0 2024-04-02T13:53:38.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.344+0000 7f974b7fe640 1 -- 172.21.15.52:0/2494299806 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9750012d30 con 0x561b700108d0 2024-04-02T13:53:38.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.344+0000 7f974b7fe640 1 -- 172.21.15.52:0/2494299806 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750026e60 con 0x561b700108d0 2024-04-02T13:53:38.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.345+0000 7f976ab78740 1 -- 172.21.15.52:0/2494299806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b700108d0 msgr2=0x561b70025ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.345+0000 7f976ab78740 1 --2- 172.21.15.52:0/2494299806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b700108d0 0x561b70025ab0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9750005d00 tx=0x7f97500046c0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.345+0000 7f976ab78740 1 -- 172.21.15.52:0/2494299806 shutdown_connections 2024-04-02T13:53:38.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.345+0000 7f976ab78740 1 --2- 172.21.15.52:0/2494299806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b70019dd0 0x561b700317b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.345+0000 7f976ab78740 1 --2- 172.21.15.52:0/2494299806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b700111f0 0x561b7000a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.345+0000 7f976ab78740 1 --2- 172.21.15.52:0/2494299806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b700108d0 0x561b70025ab0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.345+0000 7f976ab78740 1 -- 172.21.15.52:0/2494299806 >> 172.21.15.52:0/2494299806 conn(0x561b7000b000 msgr2=0x561b700099b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:38.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.345+0000 7f976ab78740 1 -- 172.21.15.52:0/2494299806 shutdown_connections 2024-04-02T13:53:38.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.345+0000 7f976ab78740 1 -- 172.21.15.52:0/2494299806 wait complete. 2024-04-02T13:53:38.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.345+0000 7f976ab78740 1 -- start start 2024-04-02T13:53:38.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.345+0000 7f976ab78740 1 Processor -- start 2024-04-02T13:53:38.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.346+0000 7f976ab78740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b700108d0 0x561b700224d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.346+0000 7f976ab78740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b700111f0 0x561b7001d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.346+0000 7f976ab78740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b70019dd0 0x561b7001df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.346+0000 7f976ab78740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f97601b2550 con 0x561b70019dd0 2024-04-02T13:53:38.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.346+0000 7f976ab78740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f97601b1e80 con 0x561b700108d0 2024-04-02T13:53:38.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.346+0000 7f976ab78740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f97601af0c0 con 0x561b700111f0 2024-04-02T13:53:38.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.346+0000 7f976ab78740 10 client.0 _ll_register_callbacks cb 0x7f975000be40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:38.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.346+0000 7f976552b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b70019dd0 0x561b7001df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.346+0000 7f976552b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b70019dd0 0x561b7001df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58696/0 (socket says 172.21.15.52:58696) 2024-04-02T13:53:38.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.346+0000 7f976552b640 1 -- 172.21.15.52:0/2211923073 learned_addr learned my addr 172.21.15.52:0/2211923073 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:38.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.346+0000 7f9765d2c640 1 --2- 172.21.15.52:0/2211923073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b700111f0 0x561b7001d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.346+0000 7f976552b640 1 -- 172.21.15.52:0/2211923073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b700111f0 msgr2=0x561b7001d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:38.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.346+0000 7f976552b640 1 --2- 172.21.15.52:0/2211923073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b700111f0 0x561b7001d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.346+0000 7f976552b640 1 -- 172.21.15.52:0/2211923073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b700108d0 msgr2=0x561b700224d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:38.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.346+0000 7f976552b640 1 --2- 172.21.15.52:0/2211923073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b700108d0 0x561b700224d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.346+0000 7f976552b640 1 -- 172.21.15.52:0/2211923073 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b7008f020 con 0x561b70019dd0 2024-04-02T13:53:38.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.346+0000 7f976552b640 1 --2- 172.21.15.52:0/2211923073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b70019dd0 0x561b7001df80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f975000b3d0 tx=0x7f9750012f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b115fee171cd1cd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:38.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.346+0000 7f97497fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:53:38.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.346+0000 7f97497fa640 1 -- 172.21.15.52:0/2211923073 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975002e070 con 0x561b70019dd0 2024-04-02T13:53:38.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.346+0000 7f97497fa640 1 -- 172.21.15.52:0/2211923073 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f9750012fe0 con 0x561b70019dd0 2024-04-02T13:53:38.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.346+0000 7f976ab78740 1 -- 172.21.15.52:0/2211923073 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9750012d30 con 0x561b70019dd0 2024-04-02T13:53:38.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.346+0000 7f97497fa640 1 -- 172.21.15.52:0/2211923073 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9750026e60 con 0x561b70019dd0 2024-04-02T13:53:38.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.346+0000 7f976ab78740 10 client.15768 fetch_fsmap learned FSMap version 28 2024-04-02T13:53:38.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.346+0000 7f976ab78740 1 -- 172.21.15.52:0/2211923073 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b700900e0 con 0x561b70019dd0 2024-04-02T13:53:38.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.347+0000 7f97497fa640 1 -- 172.21.15.52:0/2211923073 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9750012d30 con 0x561b70019dd0 2024-04-02T13:53:38.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.347+0000 7f97497fa640 1 -- 172.21.15.52:0/2211923073 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9750026040 con 0x561b70019dd0 2024-04-02T13:53:38.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.347+0000 7f976ab78740 10 client.15768 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:53:38.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.347+0000 7f976ab78740 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:53:38.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.347+0000 7f976ab78740 20 client.15768 populate_metadata read hostname 'smithi052' 2024-04-02T13:53:38.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.347+0000 7f976ab78740 1 --2- 172.21.15.52:0/2211923073 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x561b70090750 0x561b70092b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.347+0000 7f976ab78740 4 client.15768 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:53:38.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.347+0000 7f976ab78740 1 -- 172.21.15.52:0/2211923073 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561b6ff7c0c0 con 0x561b70090750 2024-04-02T13:53:38.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.347+0000 7f9765d2c640 1 --2- 172.21.15.52:0/2211923073 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x561b70090750 0x561b70092b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.347+0000 7f97497fa640 10 client.15768 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:53:38.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.347+0000 7f9765d2c640 1 --2- 172.21.15.52:0/2211923073 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x561b70090750 0x561b70092b70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=783598fbacc8f065 server_cookie=933e2ba9e052ffb8 in_seq=0 out_seq=0 2024-04-02T13:53:38.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.348+0000 7f97497fa640 1 -- 172.21.15.52:0/2211923073 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561b6ff7c0c0 con 0x561b70090750 2024-04-02T13:53:38.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.348+0000 7f97497fa640 10 client.15768 handle_command_reply: tid=0 2024-04-02T13:53:38.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.348+0000 7f976ab78740 1 client.15768 shutdown 2024-04-02T13:53:38.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.348+0000 7f976ab78740 1 -- 172.21.15.52:0/2211923073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b70019dd0 msgr2=0x561b7001df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.348+0000 7f976ab78740 1 --2- 172.21.15.52:0/2211923073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b70019dd0 0x561b7001df80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f975000b3d0 tx=0x7f9750012f70 comp rx=0 tx=0).stop 2024-04-02T13:53:38.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.348+0000 7f976ab78740 1 -- 172.21.15.52:0/2211923073 shutdown_connections 2024-04-02T13:53:38.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.348+0000 7f976ab78740 1 --2- 172.21.15.52:0/2211923073 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x561b70090750 0x561b70092b70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.348+0000 7f976ab78740 1 --2- 172.21.15.52:0/2211923073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b70019dd0 0x561b7001df80 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.348+0000 7f976ab78740 1 --2- 172.21.15.52:0/2211923073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b700111f0 0x561b7001d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.348+0000 7f97497fa640 0 client.15768 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:53:38.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.348+0000 7f976ab78740 1 --2- 172.21.15.52:0/2211923073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b700108d0 0x561b700224d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.348+0000 7f976ab78740 1 -- 172.21.15.52:0/2211923073 >> 172.21.15.52:0/2211923073 conn(0x561b7000b000 msgr2=0x561b7000a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:38.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.348+0000 7f976ab78740 1 -- 172.21.15.52:0/2211923073 shutdown_connections 2024-04-02T13:53:38.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.348+0000 7f976ab78740 1 -- 172.21.15.52:0/2211923073 wait complete. 2024-04-02T13:53:38.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.348+0000 7f976ab78740 20 client.15768 trim_cache size 0 max 16384 2024-04-02T13:53:38.358 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:53:38.358 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:53:38.358 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-04-02T13:53:38.358 INFO:teuthology.orchestra.run.smithi052.stdout: "fcb7243d-2196-46d8-bc6a-ee3dcb73c06a": { 2024-04-02T13:53:38.358 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-04-02T13:53:38.358 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "fcb7243d-2196-46d8-bc6a-ee3dcb73c06a", 2024-04-02T13:53:38.358 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-04-02T13:53:38.358 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:53:38.358 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:53:38.358 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:53:38.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.349+0000 7f9767f90640 1 -- 172.21.15.52:0/3811695773 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f974c0a3ec0 msgr2=0x7f974c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.349+0000 7f9767f90640 1 --2- 172.21.15.52:0/3811695773 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f974c0a3ec0 0x7f974c0a6380 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f97580efd30 tx=0x7f975802e040 comp rx=0 tx=0).stop 2024-04-02T13:53:38.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.349+0000 7f9767f90640 1 -- 172.21.15.52:0/3811695773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9760107ab0 msgr2=0x7f9760074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.349+0000 7f9767f90640 1 --2- 172.21.15.52:0/3811695773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9760107ab0 0x7f9760074880 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9758004af0 tx=0x7f9758032ec0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.349+0000 7f9767f90640 1 -- 172.21.15.52:0/3811695773 shutdown_connections 2024-04-02T13:53:38.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.349+0000 7f9767f90640 1 --2- 172.21.15.52:0/3811695773 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f974c0a3ec0 0x7f974c0a6380 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.349+0000 7f9767f90640 1 --2- 172.21.15.52:0/3811695773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9760107ab0 0x7f9760074880 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.349+0000 7f9767f90640 1 --2- 172.21.15.52:0/3811695773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9760102460 0x7f9760074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.349+0000 7f9767f90640 1 --2- 172.21.15.52:0/3811695773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9760101aa0 0x7f9760073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.349+0000 7f9767f90640 1 -- 172.21.15.52:0/3811695773 >> 172.21.15.52:0/3811695773 conn(0x7f97600fd590 msgr2=0x7f97600fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:38.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.349+0000 7f9767f90640 1 -- 172.21.15.52:0/3811695773 shutdown_connections 2024-04-02T13:53:38.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.349+0000 7f9767f90640 1 -- 172.21.15.52:0/3811695773 wait complete. 2024-04-02T13:53:38.362 INFO:tasks.cephfs.filesystem:scrub status for tag:fcb7243d-2196-46d8-bc6a-ee3dcb73c06a - {'path': '/', 'tag': 'fcb7243d-2196-46d8-bc6a-ee3dcb73c06a', 'options': 'recursive,force'} 2024-04-02T13:53:38.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.387+0000 7fe8f907b640 1 -- 172.21.15.52:0/3019528482 --> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fe8f4061f40 con 0x7fe8dc0a3ee0 2024-04-02T13:53:38.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:37 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3966271530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:53:38.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:37 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2086769883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:53:38.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:37 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3561269454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:53:38.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.451+0000 7fe8dbfff640 1 -- 172.21.15.52:0/3019528482 <== mgr.25160 v2:172.21.15.52:6800/1019901931 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe8f4061f40 con 0x7fe8dc0a3ee0 2024-04-02T13:53:38.455 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule set for path / 2024-04-02T13:53:38.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.453+0000 7fe8f907b640 1 -- 172.21.15.52:0/3019528482 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe8dc0a3ee0 msgr2=0x7fe8dc0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.453+0000 7fe8f907b640 1 --2- 172.21.15.52:0/3019528482 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe8dc0a3ee0 0x7fe8dc0a63a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec005fd0 tx=0x7fe8ec005850 comp rx=0 tx=0).stop 2024-04-02T13:53:38.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.453+0000 7fe8f907b640 1 -- 172.21.15.52:0/3019528482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8f41059a0 msgr2=0x7fe8f41a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.453+0000 7fe8f907b640 1 --2- 172.21.15.52:0/3019528482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8f41059a0 0x7fe8f41a42c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4005e30 tx=0x7fe8e4032f40 comp rx=0 tx=0).stop 2024-04-02T13:53:38.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.453+0000 7fe8f907b640 1 -- 172.21.15.52:0/3019528482 shutdown_connections 2024-04-02T13:53:38.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.453+0000 7fe8f907b640 1 --2- 172.21.15.52:0/3019528482 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe8dc0a3ee0 0x7fe8dc0a63a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.453+0000 7fe8f907b640 1 --2- 172.21.15.52:0/3019528482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8f41059a0 0x7fe8f41a42c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.453+0000 7fe8f907b640 1 --2- 172.21.15.52:0/3019528482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8f4101610 0x7fe8f4199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.453+0000 7fe8f907b640 1 --2- 172.21.15.52:0/3019528482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8f4100c50 0x7fe8f4199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.453+0000 7fe8f907b640 1 -- 172.21.15.52:0/3019528482 >> 172.21.15.52:0/3019528482 conn(0x7fe8f4077850 msgr2=0x7fe8f40fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:38.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.453+0000 7fe8f907b640 1 -- 172.21.15.52:0/3019528482 shutdown_connections 2024-04-02T13:53:38.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.453+0000 7fe8f907b640 1 -- 172.21.15.52:0/3019528482 wait complete. 2024-04-02T13:53:38.467 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-02T13:53:38.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.611+0000 7f4cb119f640 1 -- 172.21.15.52:0/1454873243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cac10bf20 msgr2=0x7f4cac10e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.611+0000 7f4cb119f640 1 --2- 172.21.15.52:0/1454873243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cac10bf20 0x7f4cac10e370 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4009ca0 tx=0x7f4ca402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.611+0000 7f4cb119f640 1 -- 172.21.15.52:0/1454873243 shutdown_connections 2024-04-02T13:53:38.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.611+0000 7f4cb119f640 1 --2- 172.21.15.52:0/1454873243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cac10bf20 0x7f4cac10e370 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.611+0000 7f4cb119f640 1 --2- 172.21.15.52:0/1454873243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cac1033d0 0x7f4cac10b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.611+0000 7f4cb119f640 1 --2- 172.21.15.52:0/1454873243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cac102ab0 0x7f4cac102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.611+0000 7f4cb119f640 1 -- 172.21.15.52:0/1454873243 >> 172.21.15.52:0/1454873243 conn(0x7f4cac0fc2f0 msgr2=0x7f4cac0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:38.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.611+0000 7f4cb119f640 1 -- 172.21.15.52:0/1454873243 shutdown_connections 2024-04-02T13:53:38.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.611+0000 7f4cb119f640 1 -- 172.21.15.52:0/1454873243 wait complete. 2024-04-02T13:53:38.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.611+0000 7f4cb119f640 1 Processor -- start 2024-04-02T13:53:38.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.611+0000 7f4cb119f640 1 -- start start 2024-04-02T13:53:38.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.612+0000 7f4cb119f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cac102ab0 0x7f4cac198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.612+0000 7f4cb119f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cac1033d0 0x7f4cac1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.612+0000 7f4cb119f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cac10bf20 0x7f4cac19ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.612+0000 7f4cb119f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac111cd0 con 0x7f4cac10bf20 2024-04-02T13:53:38.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.612+0000 7f4cb119f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac111b50 con 0x7f4cac1033d0 2024-04-02T13:53:38.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.612+0000 7f4cb119f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac111e50 con 0x7f4cac102ab0 2024-04-02T13:53:38.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.612+0000 7f4cb099e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cac10bf20 0x7f4cac19ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.612+0000 7f4cb099e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cac10bf20 0x7f4cac19ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58706/0 (socket says 172.21.15.52:58706) 2024-04-02T13:53:38.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.612+0000 7f4cb099e640 1 -- 172.21.15.52:0/2616256418 learned_addr learned my addr 172.21.15.52:0/2616256418 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:38.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.612+0000 7f4cabfff640 1 --2- 172.21.15.52:0/2616256418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cac102ab0 0x7f4cac198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.612+0000 7f4cab7fe640 1 --2- 172.21.15.52:0/2616256418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cac1033d0 0x7f4cac1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.612+0000 7f4cb099e640 1 -- 172.21.15.52:0/2616256418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cac102ab0 msgr2=0x7f4cac198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.612+0000 7f4cb099e640 1 --2- 172.21.15.52:0/2616256418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cac102ab0 0x7f4cac198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.612+0000 7f4cb099e640 1 -- 172.21.15.52:0/2616256418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cac1033d0 msgr2=0x7f4cac1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.612+0000 7f4cb099e640 1 --2- 172.21.15.52:0/2616256418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cac1033d0 0x7f4cac1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.612+0000 7f4cb099e640 1 -- 172.21.15.52:0/2616256418 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cac19d130 con 0x7f4cac10bf20 2024-04-02T13:53:38.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.612+0000 7f4cb099e640 1 --2- 172.21.15.52:0/2616256418 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cac10bf20 0x7f4cac19ca00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4ca402c700 tx=0x7f4ca4032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:38.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.613+0000 7f4ca97fa640 1 -- 172.21.15.52:0/2616256418 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4039040 con 0x7f4cac10bf20 2024-04-02T13:53:38.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.613+0000 7f4cb119f640 1 -- 172.21.15.52:0/2616256418 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cac19d3c0 con 0x7f4cac10bf20 2024-04-02T13:53:38.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.613+0000 7f4ca97fa640 1 -- 172.21.15.52:0/2616256418 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ca4044070 con 0x7f4cac10bf20 2024-04-02T13:53:38.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.613+0000 7f4ca97fa640 1 -- 172.21.15.52:0/2616256418 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca40042a0 con 0x7f4cac10bf20 2024-04-02T13:53:38.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.613+0000 7f4cb119f640 1 -- 172.21.15.52:0/2616256418 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cac1a4cb0 con 0x7f4cac10bf20 2024-04-02T13:53:38.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.614+0000 7f4cb119f640 1 -- 172.21.15.52:0/2616256418 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cac0500b0 con 0x7f4cac10bf20 2024-04-02T13:53:38.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.615+0000 7f4ca97fa640 1 -- 172.21.15.52:0/2616256418 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f4ca4004440 con 0x7f4cac10bf20 2024-04-02T13:53:38.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.615+0000 7f4ca97fa640 1 --2- 172.21.15.52:0/2616256418 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4c940a3e30 0x7f4c940a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:38.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.615+0000 7f4cabfff640 1 --2- 172.21.15.52:0/2616256418 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4c940a3e30 0x7f4c940a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:38.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.615+0000 7f4ca97fa640 1 -- 172.21.15.52:0/2616256418 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f4ca40f30a0 con 0x7f4cac10bf20 2024-04-02T13:53:38.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.616+0000 7f4cabfff640 1 --2- 172.21.15.52:0/2616256418 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4c940a3e30 0x7f4c940a62f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4ca00058c0 tx=0x7f4ca000a2d0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:38.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.620+0000 7f4ca97fa640 1 -- 172.21.15.52:0/2616256418 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4ca40bf2c0 con 0x7f4cac10bf20 2024-04-02T13:53:38.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.830+0000 7f4cb119f640 1 -- 172.21.15.52:0/2616256418 --> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f4cac061f40 con 0x7f4c940a3e30 2024-04-02T13:53:38.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.890+0000 7f4ca97fa640 1 -- 172.21.15.52:0/2616256418 <== mgr.25160 v2:172.21.15.52:6800/1019901931 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f4cac061f40 con 0x7f4c940a3e30 2024-04-02T13:53:38.894 INFO:teuthology.orchestra.run.smithi052.stdout:Retention added to path / 2024-04-02T13:53:38.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.893+0000 7f4cb119f640 1 -- 172.21.15.52:0/2616256418 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4c940a3e30 msgr2=0x7f4c940a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.893+0000 7f4cb119f640 1 --2- 172.21.15.52:0/2616256418 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4c940a3e30 0x7f4c940a62f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4ca00058c0 tx=0x7f4ca000a2d0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.893+0000 7f4cb119f640 1 -- 172.21.15.52:0/2616256418 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cac10bf20 msgr2=0x7f4cac19ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:38.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.893+0000 7f4cb119f640 1 --2- 172.21.15.52:0/2616256418 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cac10bf20 0x7f4cac19ca00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4ca402c700 tx=0x7f4ca4032f20 comp rx=0 tx=0).stop 2024-04-02T13:53:38.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.893+0000 7f4cb119f640 1 -- 172.21.15.52:0/2616256418 shutdown_connections 2024-04-02T13:53:38.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.893+0000 7f4cb119f640 1 --2- 172.21.15.52:0/2616256418 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4c940a3e30 0x7f4c940a62f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.893+0000 7f4cb119f640 1 --2- 172.21.15.52:0/2616256418 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cac10bf20 0x7f4cac19ca00 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.893+0000 7f4cb119f640 1 --2- 172.21.15.52:0/2616256418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cac1033d0 0x7f4cac1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.893+0000 7f4cb119f640 1 --2- 172.21.15.52:0/2616256418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cac102ab0 0x7f4cac198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:38.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.893+0000 7f4cb119f640 1 -- 172.21.15.52:0/2616256418 >> 172.21.15.52:0/2616256418 conn(0x7f4cac0fc2f0 msgr2=0x7f4cac0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:38.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.893+0000 7f4cb119f640 1 -- 172.21.15.52:0/2616256418 shutdown_connections 2024-04-02T13:53:38.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:38.893+0000 7f4cb119f640 1 -- 172.21.15.52:0/2616256418 wait complete. 2024-04-02T13:53:38.907 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-02T13:53:39.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.008+0000 7f60f6847640 1 -- 172.21.15.52:0/222337117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60f0107ac0 msgr2=0x7f60f0107ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:39.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.008+0000 7f60f6847640 1 --2- 172.21.15.52:0/222337117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60f0107ac0 0x7f60f0107ea0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f60d8009930 tx=0x7f60d802bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:39.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.008+0000 7f60f6847640 1 -- 172.21.15.52:0/222337117 shutdown_connections 2024-04-02T13:53:39.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.008+0000 7f60f6847640 1 --2- 172.21.15.52:0/222337117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60f0102470 0x7f60f010e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:39.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.008+0000 7f60f6847640 1 --2- 172.21.15.52:0/222337117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60f0101ab0 0x7f60f0101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:39.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.008+0000 7f60f6847640 1 --2- 172.21.15.52:0/222337117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60f0107ac0 0x7f60f0107ea0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:39.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.008+0000 7f60f6847640 1 -- 172.21.15.52:0/222337117 >> 172.21.15.52:0/222337117 conn(0x7f60f00fd5c0 msgr2=0x7f60f00ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:39.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.009+0000 7f60f6847640 1 -- 172.21.15.52:0/222337117 shutdown_connections 2024-04-02T13:53:39.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.009+0000 7f60f6847640 1 -- 172.21.15.52:0/222337117 wait complete. 2024-04-02T13:53:39.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.009+0000 7f60f6847640 1 Processor -- start 2024-04-02T13:53:39.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.009+0000 7f60f6847640 1 -- start start 2024-04-02T13:53:39.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.009+0000 7f60f6847640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60f0101ab0 0x7f60f0198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:39.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.009+0000 7f60f6847640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60f0102470 0x7f60f01988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:39.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.009+0000 7f60f6847640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60f0107ac0 0x7f60f019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.009+0000 7f60f6847640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f60f0111ef0 con 0x7f60f0101ab0 2024-04-02T13:53:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.009+0000 7f60f6847640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f60f0111d70 con 0x7f60f0107ac0 2024-04-02T13:53:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.009+0000 7f60f6847640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f60f0112070 con 0x7f60f0102470 2024-04-02T13:53:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.010+0000 7f60effff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60f0101ab0 0x7f60f0198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.010+0000 7f60effff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60f0101ab0 0x7f60f0198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58716/0 (socket says 172.21.15.52:58716) 2024-04-02T13:53:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.010+0000 7f60effff640 1 -- 172.21.15.52:0/844106829 learned_addr learned my addr 172.21.15.52:0/844106829 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.010+0000 7f60ef7fe640 1 --2- 172.21.15.52:0/844106829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60f0102470 0x7f60f01988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.010+0000 7f60effff640 1 -- 172.21.15.52:0/844106829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60f0102470 msgr2=0x7f60f01988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.010+0000 7f60f4de4640 1 --2- 172.21.15.52:0/844106829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60f0107ac0 0x7f60f019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.010+0000 7f60effff640 1 --2- 172.21.15.52:0/844106829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60f0102470 0x7f60f01988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.010+0000 7f60effff640 1 -- 172.21.15.52:0/844106829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60f0107ac0 msgr2=0x7f60f019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.010+0000 7f60effff640 1 --2- 172.21.15.52:0/844106829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60f0107ac0 0x7f60f019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.010+0000 7f60effff640 1 -- 172.21.15.52:0/844106829 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f019d380 con 0x7f60f0101ab0 2024-04-02T13:53:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.010+0000 7f60f4de4640 1 --2- 172.21.15.52:0/844106829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60f0107ac0 0x7f60f019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:53:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.010+0000 7f60effff640 1 --2- 172.21.15.52:0/844106829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60f0101ab0 0x7f60f0198390 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f60d8009230 tx=0x7f60d802cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.010+0000 7f60ed7fa640 1 -- 172.21.15.52:0/844106829 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d8041070 con 0x7f60f0101ab0 2024-04-02T13:53:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.010+0000 7f60ed7fa640 1 -- 172.21.15.52:0/844106829 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60d8004450 con 0x7f60f0101ab0 2024-04-02T13:53:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.010+0000 7f60f6847640 1 -- 172.21.15.52:0/844106829 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f019d610 con 0x7f60f0101ab0 2024-04-02T13:53:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.010+0000 7f60ed7fa640 1 -- 172.21.15.52:0/844106829 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d8034440 con 0x7f60f0101ab0 2024-04-02T13:53:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.010+0000 7f60f6847640 1 -- 172.21.15.52:0/844106829 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f01a4f00 con 0x7f60f0101ab0 2024-04-02T13:53:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.011+0000 7f60f6847640 1 -- 172.21.15.52:0/844106829 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60f004f860 con 0x7f60f0101ab0 2024-04-02T13:53:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.012+0000 7f60ed7fa640 1 -- 172.21.15.52:0/844106829 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f60d8044070 con 0x7f60f0101ab0 2024-04-02T13:53:39.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.013+0000 7f60ed7fa640 1 --2- 172.21.15.52:0/844106829 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f60dc0a3ee0 0x7f60dc0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:39.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.013+0000 7f60ef7fe640 1 --2- 172.21.15.52:0/844106829 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f60dc0a3ee0 0x7f60dc0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:39.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.013+0000 7f60ed7fa640 1 -- 172.21.15.52:0/844106829 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f60d80f2800 con 0x7f60f0101ab0 2024-04-02T13:53:39.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.013+0000 7f60ef7fe640 1 --2- 172.21.15.52:0/844106829 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f60dc0a3ee0 0x7f60dc0a63a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f60f00ffb40 tx=0x7f60e000b390 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:39.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.018+0000 7f60ed7fa640 1 -- 172.21.15.52:0/844106829 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f60d80be9c0 con 0x7f60f0101ab0 2024-04-02T13:53:39.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.225+0000 7f60f6847640 1 -- 172.21.15.52:0/844106829 --> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f60f00fe1f0 con 0x7f60dc0a3ee0 2024-04-02T13:53:39.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.232+0000 7f60ed7fa640 1 -- 172.21.15.52:0/844106829 <== mgr.25160 v2:172.21.15.52:6800/1019901931 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f60f00fe1f0 con 0x7f60dc0a3ee0 2024-04-02T13:53:39.236 INFO:teuthology.orchestra.run.smithi052.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-02T00:00:00", "created": "2024-04-02T13:53:38", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-02T13:53:39.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.234+0000 7f60f6847640 1 -- 172.21.15.52:0/844106829 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f60dc0a3ee0 msgr2=0x7f60dc0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:39.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.234+0000 7f60f6847640 1 --2- 172.21.15.52:0/844106829 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f60dc0a3ee0 0x7f60dc0a63a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f60f00ffb40 tx=0x7f60e000b390 comp rx=0 tx=0).stop 2024-04-02T13:53:39.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.234+0000 7f60f6847640 1 -- 172.21.15.52:0/844106829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60f0101ab0 msgr2=0x7f60f0198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:39.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.234+0000 7f60f6847640 1 --2- 172.21.15.52:0/844106829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60f0101ab0 0x7f60f0198390 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f60d8009230 tx=0x7f60d802cc70 comp rx=0 tx=0).stop 2024-04-02T13:53:39.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.234+0000 7f60f6847640 1 -- 172.21.15.52:0/844106829 shutdown_connections 2024-04-02T13:53:39.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.234+0000 7f60f6847640 1 --2- 172.21.15.52:0/844106829 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f60dc0a3ee0 0x7f60dc0a63a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:39.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.234+0000 7f60f6847640 1 --2- 172.21.15.52:0/844106829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60f0107ac0 0x7f60f019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:39.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.234+0000 7f60f6847640 1 --2- 172.21.15.52:0/844106829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60f0102470 0x7f60f01988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:39.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.234+0000 7f60f6847640 1 --2- 172.21.15.52:0/844106829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60f0101ab0 0x7f60f0198390 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:39.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.234+0000 7f60f6847640 1 -- 172.21.15.52:0/844106829 >> 172.21.15.52:0/844106829 conn(0x7f60f00fd5c0 msgr2=0x7f60f00fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:39.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.234+0000 7f60f6847640 1 -- 172.21.15.52:0/844106829 shutdown_connections 2024-04-02T13:53:39.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.234+0000 7f60f6847640 1 -- 172.21.15.52:0/844106829 wait complete. 2024-04-02T13:53:39.247 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:38 smithi052 ceph-mon[32104]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.4 KiB/s wr, 26 op/s 2024-04-02T13:53:39.248 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:38 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3020094607' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:53:39.248 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:38 smithi052 ceph-mon[32104]: scrub queued for path: / 2024-04-02T13:53:39.248 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:38 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:39.248 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:38 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:53:39.248 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:38 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:39.248 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:38 smithi052 ceph-mon[32104]: from='client.15735 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:39.249 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-02T13:53:39.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:38 smithi189 ceph-mon[36683]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.4 KiB/s wr, 26 op/s 2024-04-02T13:53:39.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:38 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3020094607' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:53:39.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:38 smithi189 ceph-mon[36683]: scrub queued for path: / 2024-04-02T13:53:39.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:38 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:39.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:38 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:53:39.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:38 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:39.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:38 smithi189 ceph-mon[36683]: from='client.15735 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:39.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.351+0000 7f9da635f640 1 -- 172.21.15.52:0/1113090432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9da0101680 msgr2=0x7f9da010d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:39.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.351+0000 7f9da635f640 1 --2- 172.21.15.52:0/1113090432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9da0101680 0x7f9da010d7a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009ca0 tx=0x7f9d9802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:53:39.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.351+0000 7f9da635f640 1 -- 172.21.15.52:0/1113090432 shutdown_connections 2024-04-02T13:53:39.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.351+0000 7f9da635f640 1 --2- 172.21.15.52:0/1113090432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9da0101680 0x7f9da010d7a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:39.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.351+0000 7f9da635f640 1 --2- 172.21.15.52:0/1113090432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9da0100ce0 0x7f9da0101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:39.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.351+0000 7f9da635f640 1 --2- 172.21.15.52:0/1113090432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9da0106a40 0x7f9da0106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:39.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.351+0000 7f9da635f640 1 -- 172.21.15.52:0/1113090432 >> 172.21.15.52:0/1113090432 conn(0x7f9da0077720 msgr2=0x7f9da0077b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:39.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.351+0000 7f9da635f640 1 -- 172.21.15.52:0/1113090432 shutdown_connections 2024-04-02T13:53:39.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.351+0000 7f9da635f640 1 -- 172.21.15.52:0/1113090432 wait complete. 2024-04-02T13:53:39.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.351+0000 7f9da635f640 1 Processor -- start 2024-04-02T13:53:39.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.351+0000 7f9da635f640 1 -- start start 2024-04-02T13:53:39.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.352+0000 7f9da635f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9da0100ce0 0x7f9da0198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:39.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.352+0000 7f9da635f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9da0101680 0x7f9da0198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:39.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.352+0000 7f9da635f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9da0106a40 0x7f9da019cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:39.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.352+0000 7f9da635f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9da0110db0 con 0x7f9da0100ce0 2024-04-02T13:53:39.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.352+0000 7f9da635f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9da0110c30 con 0x7f9da0101680 2024-04-02T13:53:39.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.352+0000 7f9da635f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9da0110f30 con 0x7f9da0106a40 2024-04-02T13:53:39.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.352+0000 7f9d9ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9da0100ce0 0x7f9da0198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:39.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.352+0000 7f9d9ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9da0100ce0 0x7f9da0198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58726/0 (socket says 172.21.15.52:58726) 2024-04-02T13:53:39.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.352+0000 7f9d9ffff640 1 -- 172.21.15.52:0/4278556396 learned_addr learned my addr 172.21.15.52:0/4278556396 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:53:39.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.352+0000 7f9d9f7fe640 1 --2- 172.21.15.52:0/4278556396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9da0101680 0x7f9da0198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:39.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.352+0000 7f9da48fc640 1 --2- 172.21.15.52:0/4278556396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9da0106a40 0x7f9da019cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:39.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.352+0000 7f9d9ffff640 1 -- 172.21.15.52:0/4278556396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9da0106a40 msgr2=0x7f9da019cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:39.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.352+0000 7f9d9ffff640 1 --2- 172.21.15.52:0/4278556396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9da0106a40 0x7f9da019cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:39.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.352+0000 7f9d9ffff640 1 -- 172.21.15.52:0/4278556396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9da0101680 msgr2=0x7f9da0198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:39.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.352+0000 7f9d9ffff640 1 --2- 172.21.15.52:0/4278556396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9da0101680 0x7f9da0198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:39.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.352+0000 7f9d9ffff640 1 -- 172.21.15.52:0/4278556396 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da019d230 con 0x7f9da0100ce0 2024-04-02T13:53:39.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.352+0000 7f9d9ffff640 1 --2- 172.21.15.52:0/4278556396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9da0100ce0 0x7f9da0198240 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000d950 tx=0x7f9d9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:39.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.353+0000 7f9d9d7fa640 1 -- 172.21.15.52:0/4278556396 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90017070 con 0x7f9da0100ce0 2024-04-02T13:53:39.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.353+0000 7f9d9d7fa640 1 -- 172.21.15.52:0/4278556396 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d90009c30 con 0x7f9da0100ce0 2024-04-02T13:53:39.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.353+0000 7f9da635f640 1 -- 172.21.15.52:0/4278556396 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da019d520 con 0x7f9da0100ce0 2024-04-02T13:53:39.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.353+0000 7f9d9d7fa640 1 -- 172.21.15.52:0/4278556396 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9000f6b0 con 0x7f9da0100ce0 2024-04-02T13:53:39.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.353+0000 7f9da635f640 1 -- 172.21.15.52:0/4278556396 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da01a4db0 con 0x7f9da0100ce0 2024-04-02T13:53:39.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.354+0000 7f9da635f640 1 -- 172.21.15.52:0/4278556396 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9da01020e0 con 0x7f9da0100ce0 2024-04-02T13:53:39.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.355+0000 7f9d9d7fa640 1 -- 172.21.15.52:0/4278556396 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f9d900109d0 con 0x7f9da0100ce0 2024-04-02T13:53:39.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.356+0000 7f9d9d7fa640 1 --2- 172.21.15.52:0/4278556396 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9d880a3f60 0x7f9d880a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:39.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.356+0000 7f9d9f7fe640 1 --2- 172.21.15.52:0/4278556396 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9d880a3f60 0x7f9d880a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:39.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.356+0000 7f9d9d7fa640 1 -- 172.21.15.52:0/4278556396 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f9d900d0a40 con 0x7f9da0100ce0 2024-04-02T13:53:39.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.356+0000 7f9d9f7fe640 1 --2- 172.21.15.52:0/4278556396 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9d880a3f60 0x7f9d880a6420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9d94005fd0 tx=0x7f9d94005e20 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:39.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.360+0000 7f9d9d7fa640 1 -- 172.21.15.52:0/4278556396 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9d90009de0 con 0x7f9da0100ce0 2024-04-02T13:53:39.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:38 smithi088 ceph-mon[36528]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.4 KiB/s wr, 26 op/s 2024-04-02T13:53:39.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:38 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3020094607' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:53:39.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:38 smithi088 ceph-mon[36528]: scrub queued for path: / 2024-04-02T13:53:39.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:38 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:39.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:38 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:53:39.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:38 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:39.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:38 smithi088 ceph-mon[36528]: from='client.15735 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:39.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.570+0000 7f9da635f640 1 -- 172.21.15.52:0/4278556396 --> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f9da0104560 con 0x7f9d880a3f60 2024-04-02T13:53:39.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.577+0000 7f9d9d7fa640 1 -- 172.21.15.52:0/4278556396 <== mgr.25160 v2:172.21.15.52:6800/1019901931 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f9da0104560 con 0x7f9d880a3f60 2024-04-02T13:53:39.580 INFO:teuthology.orchestra.run.smithi052.stdout:/ 1m 6m3h 2024-04-02T13:53:39.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.579+0000 7f9da635f640 1 -- 172.21.15.52:0/4278556396 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9d880a3f60 msgr2=0x7f9d880a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:39.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.579+0000 7f9da635f640 1 --2- 172.21.15.52:0/4278556396 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9d880a3f60 0x7f9d880a6420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9d94005fd0 tx=0x7f9d94005e20 comp rx=0 tx=0).stop 2024-04-02T13:53:39.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.579+0000 7f9da635f640 1 -- 172.21.15.52:0/4278556396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9da0100ce0 msgr2=0x7f9da0198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:39.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.579+0000 7f9da635f640 1 --2- 172.21.15.52:0/4278556396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9da0100ce0 0x7f9da0198240 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000d950 tx=0x7f9d9000de10 comp rx=0 tx=0).stop 2024-04-02T13:53:39.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.579+0000 7f9da635f640 1 -- 172.21.15.52:0/4278556396 shutdown_connections 2024-04-02T13:53:39.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.579+0000 7f9da635f640 1 --2- 172.21.15.52:0/4278556396 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9d880a3f60 0x7f9d880a6420 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:39.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.579+0000 7f9da635f640 1 --2- 172.21.15.52:0/4278556396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9da0106a40 0x7f9da019cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:39.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.579+0000 7f9da635f640 1 --2- 172.21.15.52:0/4278556396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9da0101680 0x7f9da0198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:39.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.579+0000 7f9da635f640 1 --2- 172.21.15.52:0/4278556396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9da0100ce0 0x7f9da0198240 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:39.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.579+0000 7f9da635f640 1 -- 172.21.15.52:0/4278556396 >> 172.21.15.52:0/4278556396 conn(0x7f9da0077720 msgr2=0x7f9da00ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:39.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.579+0000 7f9da635f640 1 -- 172.21.15.52:0/4278556396 shutdown_connections 2024-04-02T13:53:39.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:53:39.579+0000 7f9da635f640 1 -- 172.21.15.52:0/4278556396 wait complete. 2024-04-02T13:53:39.593 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-02T13:53:39.621 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-02T13:53:39.637 INFO:teuthology.run_tasks:Running task workunit... 2024-04-02T13:53:39.659 INFO:tasks.workunit:Pulling workunits from ref 191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:53:39.660 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-02T13:53:39.660 INFO:tasks.workunit:timeout=3h 2024-04-02T13:53:39.660 INFO:tasks.workunit:cleanup=True 2024-04-02T13:53:39.660 DEBUG:teuthology.orchestra.run.smithi052:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-02T13:53:39.676 INFO:teuthology.orchestra.run.smithi052.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-02T13:53:39.676 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-02T13:53:39.676 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-04-02T13:53:39.676 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:53:39.676 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-02T13:53:39.676 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-04-02 13:51:29.448091974 +0000 2024-04-02T13:53:39.676 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-04-02 13:51:29.448091974 +0000 2024-04-02T13:53:39.677 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-04-02 13:53:27.737768995 +0000 2024-04-02T13:53:39.677 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-04-02 13:51:29.448091974 +0000 2024-04-02T13:53:39.677 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-02T13:53:39.677 DEBUG:teuthology.orchestra.run.smithi052:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-02T13:53:39.753 DEBUG:teuthology.orchestra.run.smithi052:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:53:39.808 INFO:tasks.workunit.client.0.smithi052.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-02T13:53:40.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:39 smithi189 ceph-mon[36683]: from='client.15774 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:40.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:39 smithi189 ceph-mon[36683]: from='client.15780 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:40.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:39 smithi189 ceph-mon[36683]: from='client.15786 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:40.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:39 smithi052 ceph-mon[32104]: from='client.15774 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:40.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:39 smithi052 ceph-mon[32104]: from='client.15780 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:40.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:39 smithi052 ceph-mon[32104]: from='client.15786 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:40.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:39 smithi088 ceph-mon[36528]: from='client.15774 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:40.446 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:39 smithi088 ceph-mon[36528]: from='client.15780 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:40.446 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:39 smithi088 ceph-mon[36528]: from='client.15786 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:53:41.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:40 smithi189 ceph-mon[36683]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.1 KiB/s wr, 20 op/s 2024-04-02T13:53:41.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:40 smithi052 ceph-mon[32104]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.1 KiB/s wr, 20 op/s 2024-04-02T13:53:41.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:40 smithi088 ceph-mon[36528]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.1 KiB/s wr, 20 op/s 2024-04-02T13:53:43.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:42 smithi189 ceph-mon[36683]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 57 op/s 2024-04-02T13:53:43.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:42 smithi052 ceph-mon[32104]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 57 op/s 2024-04-02T13:53:43.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:42 smithi088 ceph-mon[36528]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 57 op/s 2024-04-02T13:53:45.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:44 smithi189 ceph-mon[36683]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 52 op/s 2024-04-02T13:53:45.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:44 smithi052 ceph-mon[32104]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 52 op/s 2024-04-02T13:53:45.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:44 smithi088 ceph-mon[36528]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 52 op/s 2024-04-02T13:53:47.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:46 smithi189 ceph-mon[36683]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 78 KiB/s wr, 55 op/s 2024-04-02T13:53:47.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:46 smithi052 ceph-mon[32104]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 78 KiB/s wr, 55 op/s 2024-04-02T13:53:47.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:46 smithi088 ceph-mon[36528]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 78 KiB/s wr, 55 op/s 2024-04-02T13:53:49.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:48 smithi189 ceph-mon[36683]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 78 KiB/s wr, 54 op/s 2024-04-02T13:53:49.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:48 smithi052 ceph-mon[32104]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 78 KiB/s wr, 54 op/s 2024-04-02T13:53:49.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:48 smithi088 ceph-mon[36528]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 78 KiB/s wr, 54 op/s 2024-04-02T13:53:51.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:50 smithi189 ceph-mon[36683]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-04-02T13:53:51.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:50 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T13:53:51.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:50 smithi052 ceph-mon[32104]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-04-02T13:53:51.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:50 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T13:53:51.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:50 smithi088 ceph-mon[36528]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-04-02T13:53:51.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:50 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T13:53:53.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:52 smithi189 ceph-mon[36683]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-04-02T13:53:53.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:52 smithi052 ceph-mon[32104]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-04-02T13:53:53.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:52 smithi088 ceph-mon[36528]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-04-02T13:53:55.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:54 smithi189 ceph-mon[36683]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:53:55.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:54 smithi052 ceph-mon[32104]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:53:55.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:54 smithi088 ceph-mon[36528]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:53:57.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:56 smithi189 ceph-mon[36683]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:53:57.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:56 smithi052 ceph-mon[32104]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:53:57.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:56 smithi088 ceph-mon[36528]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:53:59.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:53:58 smithi189 ceph-mon[36683]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T13:53:59.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:53:58 smithi052 ceph-mon[32104]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T13:53:59.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:53:58 smithi088 ceph-mon[36528]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T13:54:01.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:00 smithi189 ceph-mon[36683]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:54:01.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:00 smithi052 ceph-mon[32104]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:54:01.340 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:54:00 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:54:00.901+0000 7ff818f9b700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:54:01.340 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:54:00 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:54:00.901+0000 7ff818f9b700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:54:01.340 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:54:00 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:54:00.901+0000 7ff818f9b700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:54:01.340 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:54:00 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:54:00.901+0000 7ff818f9b700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:54:01.340 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 13:54:00 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T13:54:00.901+0000 7ff818f9b700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:54:01.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:00 smithi088 ceph-mon[36528]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:54:03.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:02 smithi189 ceph-mon[36683]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-02T13:54:03.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:02 smithi189 ceph-mon[36683]: mgrmap e25: x(active, since 32s), standbys: y, z 2024-04-02T13:54:03.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:02 smithi052 ceph-mon[32104]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-02T13:54:03.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:02 smithi052 ceph-mon[32104]: mgrmap e25: x(active, since 32s), standbys: y, z 2024-04-02T13:54:03.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:02 smithi088 ceph-mon[36528]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-02T13:54:03.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:02 smithi088 ceph-mon[36528]: mgrmap e25: x(active, since 32s), standbys: y, z 2024-04-02T13:54:05.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:04 smithi189 ceph-mon[36683]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-02T13:54:05.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:04 smithi052 ceph-mon[32104]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-02T13:54:05.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:04 smithi088 ceph-mon[36528]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-02T13:54:07.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:06 smithi189 ceph-mon[36683]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-04-02T13:54:07.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:06 smithi052 ceph-mon[32104]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-04-02T13:54:07.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:06 smithi088 ceph-mon[36528]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-04-02T13:54:08.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:54:08.364 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:54:08.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.481+0000 7fda512b1640 1 -- 172.21.15.52:0/3976556153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda4c101150 msgr2=0x7fda4c1015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:08.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.481+0000 7fda512b1640 1 --2- 172.21.15.52:0/3976556153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda4c101150 0x7fda4c1015b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fda34009900 tx=0x7fda3402bc80 comp rx=0 tx=0).stop 2024-04-02T13:54:08.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.481+0000 7fda512b1640 1 -- 172.21.15.52:0/3976556153 shutdown_connections 2024-04-02T13:54:08.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.481+0000 7fda512b1640 1 --2- 172.21.15.52:0/3976556153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda4c101af0 0x7fda4c10d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.481+0000 7fda512b1640 1 --2- 172.21.15.52:0/3976556153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda4c101150 0x7fda4c1015b0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.481+0000 7fda512b1640 1 --2- 172.21.15.52:0/3976556153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda4c1059b0 0x7fda4c105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.481+0000 7fda512b1640 1 -- 172.21.15.52:0/3976556153 >> 172.21.15.52:0/3976556153 conn(0x7fda4c076fc0 msgr2=0x7fda4c0fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:08.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.481+0000 7fda512b1640 1 -- 172.21.15.52:0/3976556153 shutdown_connections 2024-04-02T13:54:08.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.481+0000 7fda512b1640 1 -- 172.21.15.52:0/3976556153 wait complete. 2024-04-02T13:54:08.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.482+0000 7fda512b1640 1 Processor -- start 2024-04-02T13:54:08.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.482+0000 7fda512b1640 1 -- start start 2024-04-02T13:54:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.482+0000 7fda512b1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda4c101150 0x7fda4c0fc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.482+0000 7fda512b1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda4c101af0 0x7fda4c0fc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.482+0000 7fda512b1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda4c1059b0 0x7fda4c0fce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.482+0000 7fda512b1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c110eb0 con 0x7fda4c101af0 2024-04-02T13:54:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.482+0000 7fda512b1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c110d30 con 0x7fda4c1059b0 2024-04-02T13:54:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.482+0000 7fda512b1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c111030 con 0x7fda4c101150 2024-04-02T13:54:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.482+0000 7fda43fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda4c101af0 0x7fda4c0fc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.482+0000 7fda43fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda4c101af0 0x7fda4c0fc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33084/0 (socket says 172.21.15.52:33084) 2024-04-02T13:54:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.482+0000 7fda43fff640 1 -- 172.21.15.52:0/4181990075 learned_addr learned my addr 172.21.15.52:0/4181990075 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.482+0000 7fda4b7fe640 1 --2- 172.21.15.52:0/4181990075 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda4c1059b0 0x7fda4c0fce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.482+0000 7fda4affd640 1 --2- 172.21.15.52:0/4181990075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda4c101150 0x7fda4c0fc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.482+0000 7fda43fff640 1 -- 172.21.15.52:0/4181990075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda4c101150 msgr2=0x7fda4c0fc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.482+0000 7fda43fff640 1 --2- 172.21.15.52:0/4181990075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda4c101150 0x7fda4c0fc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.482+0000 7fda43fff640 1 -- 172.21.15.52:0/4181990075 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda4c1059b0 msgr2=0x7fda4c0fce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:08.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.482+0000 7fda43fff640 1 --2- 172.21.15.52:0/4181990075 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda4c1059b0 0x7fda4c0fce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.482+0000 7fda43fff640 1 -- 172.21.15.52:0/4181990075 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4c1a6920 con 0x7fda4c101af0 2024-04-02T13:54:08.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.483+0000 7fda43fff640 1 --2- 172.21.15.52:0/4181990075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda4c101af0 0x7fda4c0fc910 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fda34006010 tx=0x7fda340043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:08.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.483+0000 7fda48ff9640 1 -- 172.21.15.52:0/4181990075 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34041070 con 0x7fda4c101af0 2024-04-02T13:54:08.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.483+0000 7fda48ff9640 1 -- 172.21.15.52:0/4181990075 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda3402ccb0 con 0x7fda4c101af0 2024-04-02T13:54:08.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.483+0000 7fda48ff9640 1 -- 172.21.15.52:0/4181990075 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34003e20 con 0x7fda4c101af0 2024-04-02T13:54:08.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.483+0000 7fda512b1640 1 -- 172.21.15.52:0/4181990075 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda4c1a6bb0 con 0x7fda4c101af0 2024-04-02T13:54:08.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.483+0000 7fda512b1640 1 -- 172.21.15.52:0/4181990075 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda4c1a70b0 con 0x7fda4c101af0 2024-04-02T13:54:08.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.485+0000 7fda48ff9640 1 -- 172.21.15.52:0/4181990075 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fda34044070 con 0x7fda4c101af0 2024-04-02T13:54:08.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.486+0000 7fda53f71740 1 Processor -- start 2024-04-02T13:54:08.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.486+0000 7fda53f71740 1 -- start start 2024-04-02T13:54:08.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.486+0000 7fda53f71740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f486378d0 0x564f48637cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.486+0000 7fda53f71740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f486381f0 0x564f48640890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.486+0000 7fda53f71740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f48640dd0 0x564f486431c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.486+0000 7fda53f71740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564f4859b830 con 0x564f48640dd0 2024-04-02T13:54:08.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.486+0000 7fda53f71740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564f48543820 con 0x564f486378d0 2024-04-02T13:54:08.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.486+0000 7fda53f71740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564f485977e0 con 0x564f486381f0 2024-04-02T13:54:08.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.486+0000 7fda48ff9640 1 --2- 172.21.15.52:0/4181990075 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fda380a3ed0 0x7fda380a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.486+0000 7fda4b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f486378d0 0x564f48637cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:08.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.486+0000 7fda43fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f48640dd0 0x564f486431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:08.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.486+0000 7fda4b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f486378d0 0x564f48637cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:36590/0 (socket says 172.21.15.52:36590) 2024-04-02T13:54:08.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.486+0000 7fda4b7fe640 1 -- 172.21.15.52:0/3011577365 learned_addr learned my addr 172.21.15.52:0/3011577365 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:08.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.486+0000 7fda4b7fe640 1 --2- 172.21.15.52:0/4181990075 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fda380a3ed0 0x7fda380a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:08.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.486+0000 7fda48ff9640 1 -- 172.21.15.52:0/4181990075 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fda340f2950 con 0x7fda4c101af0 2024-04-02T13:54:08.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.486+0000 7fda4affd640 1 --2- 172.21.15.52:0/3011577365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f486381f0 0x564f48640890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:08.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.487+0000 7fda4b7fe640 1 -- 172.21.15.52:0/3011577365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f486381f0 msgr2=0x564f48640890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:08.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.487+0000 7fda4b7fe640 1 --2- 172.21.15.52:0/3011577365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f486381f0 0x564f48640890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.487+0000 7fda4b7fe640 1 -- 172.21.15.52:0/3011577365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f48640dd0 msgr2=0x564f486431c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:08.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.487+0000 7fda4b7fe640 1 --2- 172.21.15.52:0/3011577365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f48640dd0 0x564f486431c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.487+0000 7fda4b7fe640 1 -- 172.21.15.52:0/3011577365 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f483797d0 con 0x564f486378d0 2024-04-02T13:54:08.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.487+0000 7fda43fff640 1 --2- 172.21.15.52:0/3011577365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f48640dd0 0x564f486431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:54:08.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.487+0000 7fda4b7fe640 1 --2- 172.21.15.52:0/4181990075 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fda380a3ed0 0x7fda380a6390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fda44005c40 tx=0x7fda4400b000 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:08.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.487+0000 7fda4b7fe640 1 --2- 172.21.15.52:0/3011577365 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f486378d0 0x564f48637cb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fda44012cf0 tx=0x7fda44010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1714e60e5dc4c86 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:08.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.487+0000 7fda417fa640 1 -- 172.21.15.52:0/3011577365 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda44023030 con 0x564f486378d0 2024-04-02T13:54:08.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.487+0000 7fda417fa640 1 -- 172.21.15.52:0/3011577365 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda44009c80 con 0x564f486378d0 2024-04-02T13:54:08.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.487+0000 7fda417fa640 1 -- 172.21.15.52:0/3011577365 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4400db80 con 0x564f486378d0 2024-04-02T13:54:08.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.488+0000 7fda53f71740 1 -- 172.21.15.52:0/3011577365 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f486378d0 msgr2=0x564f48637cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:08.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.488+0000 7fda53f71740 1 --2- 172.21.15.52:0/3011577365 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f486378d0 0x564f48637cb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fda44012cf0 tx=0x7fda44010740 comp rx=0 tx=0).stop 2024-04-02T13:54:08.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.488+0000 7fda53f71740 1 -- 172.21.15.52:0/3011577365 shutdown_connections 2024-04-02T13:54:08.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.488+0000 7fda53f71740 1 --2- 172.21.15.52:0/3011577365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f48640dd0 0x564f486431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.488+0000 7fda53f71740 1 --2- 172.21.15.52:0/3011577365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f486381f0 0x564f48640890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.488+0000 7fda53f71740 1 --2- 172.21.15.52:0/3011577365 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f486378d0 0x564f48637cb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.488+0000 7fda53f71740 1 -- 172.21.15.52:0/3011577365 >> 172.21.15.52:0/3011577365 conn(0x564f48632000 msgr2=0x564f486323e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:08.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.488+0000 7fda53f71740 1 -- 172.21.15.52:0/3011577365 shutdown_connections 2024-04-02T13:54:08.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.488+0000 7fda53f71740 1 -- 172.21.15.52:0/3011577365 wait complete. 2024-04-02T13:54:08.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.488+0000 7fda53f71740 1 -- start start 2024-04-02T13:54:08.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.488+0000 7fda53f71740 1 Processor -- start 2024-04-02T13:54:08.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.489+0000 7fda53f71740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f486378d0 0x564f48640750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.489+0000 7fda53f71740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f486381f0 0x564f4863dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.489+0000 7fda53f71740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f48640dd0 0x564f4863e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.489+0000 7fda53f71740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564f48643b60 con 0x564f48640dd0 2024-04-02T13:54:08.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.489+0000 7fda53f71740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564f48556d30 con 0x564f486378d0 2024-04-02T13:54:08.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.489+0000 7fda53f71740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564f48643ce0 con 0x564f486381f0 2024-04-02T13:54:08.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.489+0000 7fda53f71740 10 client.0 _ll_register_callbacks cb 0x564f48556720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:08.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.489+0000 7fda4b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f48640dd0 0x564f4863e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:08.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.489+0000 7fda4affd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f486378d0 0x564f48640750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:08.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.489+0000 7fda4b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f48640dd0 0x564f4863e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33104/0 (socket says 172.21.15.52:33104) 2024-04-02T13:54:08.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.489+0000 7fda4affd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f486378d0 0x564f48640750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:36606/0 (socket says 172.21.15.52:36606) 2024-04-02T13:54:08.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.489+0000 7fda4b7fe640 1 -- 172.21.15.52:0/3618259860 learned_addr learned my addr 172.21.15.52:0/3618259860 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:08.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.489+0000 7fda43fff640 1 --2- 172.21.15.52:0/3618259860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f486381f0 0x564f4863dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:08.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.489+0000 7fda4b7fe640 1 -- 172.21.15.52:0/3618259860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f486381f0 msgr2=0x564f4863dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:08.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.489+0000 7fda4b7fe640 1 --2- 172.21.15.52:0/3618259860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f486381f0 0x564f4863dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.489+0000 7fda4b7fe640 1 -- 172.21.15.52:0/3618259860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f486378d0 msgr2=0x564f48640750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:08.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.489+0000 7fda4b7fe640 1 --2- 172.21.15.52:0/3618259860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f486378d0 0x564f48640750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.489+0000 7fda4b7fe640 1 -- 172.21.15.52:0/3618259860 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f482af3f0 con 0x564f48640dd0 2024-04-02T13:54:08.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.489+0000 7fda4affd640 1 --2- 172.21.15.52:0/3618259860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f486378d0 0x564f48640750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:54:08.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.489+0000 7fda4b7fe640 1 --2- 172.21.15.52:0/3618259860 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f48640dd0 0x564f4863e060 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fda44006010 tx=0x7fda4400dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dece2b55e7ed680 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:08.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.489+0000 7fda327fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:08.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.489+0000 7fda327fc640 1 -- 172.21.15.52:0/3618259860 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4402c070 con 0x564f48640dd0 2024-04-02T13:54:08.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.489+0000 7fda327fc640 1 -- 172.21.15.52:0/3618259860 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fda44004bf0 con 0x564f48640dd0 2024-04-02T13:54:08.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.489+0000 7fda53f71740 1 -- 172.21.15.52:0/3618259860 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564f485a8d70 con 0x564f48640dd0 2024-04-02T13:54:08.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.490+0000 7fda327fc640 1 -- 172.21.15.52:0/3618259860 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fda44038050 con 0x564f48640dd0 2024-04-02T13:54:08.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.490+0000 7fda53f71740 10 client.15804 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:08.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.490+0000 7fda53f71740 1 -- 172.21.15.52:0/3618259860 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564f4826f0f0 con 0x564f48640dd0 2024-04-02T13:54:08.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.490+0000 7fda327fc640 1 -- 172.21.15.52:0/3618259860 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564f485a8d70 con 0x564f48640dd0 2024-04-02T13:54:08.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.490+0000 7fda327fc640 1 -- 172.21.15.52:0/3618259860 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fda4400c040 con 0x564f48640dd0 2024-04-02T13:54:08.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.490+0000 7fda53f71740 10 client.15804 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:08.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.490+0000 7fda53f71740 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:54:08.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.490+0000 7fda53f71740 20 client.15804 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:08.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.490+0000 7fda53f71740 1 --2- 172.21.15.52:0/3618259860 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564f48650d30 0x564f48653120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.490+0000 7fda53f71740 4 client.15804 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:54:08.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.490+0000 7fda53f71740 1 -- 172.21.15.52:0/3618259860 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f485a30c0 con 0x564f48650d30 2024-04-02T13:54:08.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.490+0000 7fda4affd640 1 --2- 172.21.15.52:0/3618259860 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564f48650d30 0x564f48653120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:08.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.491+0000 7fda4affd640 1 --2- 172.21.15.52:0/3618259860 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564f48650d30 0x564f48653120 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98df5d747d660987 server_cookie=42d9cc63c68d4a6e in_seq=0 out_seq=0 2024-04-02T13:54:08.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.491+0000 7fda327fc640 10 client.15804 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:08.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.491+0000 7fda327fc640 1 -- 172.21.15.52:0/3618259860 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f485a30c0 con 0x564f48650d30 2024-04-02T13:54:08.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.491+0000 7fda327fc640 10 client.15804 handle_command_reply: tid=0 2024-04-02T13:54:08.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.491+0000 7fda53f71740 1 client.15804 shutdown 2024-04-02T13:54:08.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.492+0000 7fda53f71740 1 -- 172.21.15.52:0/3618259860 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f48640dd0 msgr2=0x564f4863e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:08.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.492+0000 7fda53f71740 1 --2- 172.21.15.52:0/3618259860 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f48640dd0 0x564f4863e060 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fda44006010 tx=0x7fda4400dd70 comp rx=0 tx=0).stop 2024-04-02T13:54:08.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.492+0000 7fda53f71740 1 -- 172.21.15.52:0/3618259860 shutdown_connections 2024-04-02T13:54:08.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.492+0000 7fda53f71740 1 --2- 172.21.15.52:0/3618259860 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564f48650d30 0x564f48653120 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.492+0000 7fda53f71740 1 --2- 172.21.15.52:0/3618259860 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f48640dd0 0x564f4863e060 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.492+0000 7fda327fc640 0 client.15804 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:08.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.492+0000 7fda53f71740 1 --2- 172.21.15.52:0/3618259860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f486381f0 0x564f4863dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.492+0000 7fda53f71740 1 --2- 172.21.15.52:0/3618259860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f486378d0 0x564f48640750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.492+0000 7fda53f71740 1 -- 172.21.15.52:0/3618259860 >> 172.21.15.52:0/3618259860 conn(0x564f48632000 msgr2=0x564f48633d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:08.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.492+0000 7fda53f71740 1 -- 172.21.15.52:0/3618259860 shutdown_connections 2024-04-02T13:54:08.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.492+0000 7fda53f71740 1 -- 172.21.15.52:0/3618259860 wait complete. 2024-04-02T13:54:08.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.492+0000 7fda53f71740 20 client.15804 trim_cache size 0 max 16384 2024-04-02T13:54:08.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.509+0000 7fda53f71740 1 Processor -- start 2024-04-02T13:54:08.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.509+0000 7fda53f71740 1 -- start start 2024-04-02T13:54:08.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.510+0000 7fda53f71740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f486378d0 0x564f4864ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.510+0000 7fda53f71740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f486381f0 0x564f4863c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.510+0000 7fda53f71740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f48640dd0 0x564f48640210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.510+0000 7fda53f71740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c1b0fb0 con 0x564f48640dd0 2024-04-02T13:54:08.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.510+0000 7fda53f71740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c1b2af0 con 0x564f486378d0 2024-04-02T13:54:08.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.510+0000 7fda53f71740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c1b1680 con 0x564f486381f0 2024-04-02T13:54:08.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.510+0000 7fda4b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f48640dd0 0x564f48640210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:08.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.510+0000 7fda4b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f48640dd0 0x564f48640210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33110/0 (socket says 172.21.15.52:33110) 2024-04-02T13:54:08.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.510+0000 7fda4b7fe640 1 -- 172.21.15.52:0/4223189782 learned_addr learned my addr 172.21.15.52:0/4223189782 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:08.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.510+0000 7fda43fff640 1 --2- 172.21.15.52:0/4223189782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f486381f0 0x564f4863c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:08.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.510+0000 7fda4affd640 1 --2- 172.21.15.52:0/4223189782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f486378d0 0x564f4864ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:08.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.510+0000 7fda4b7fe640 1 -- 172.21.15.52:0/4223189782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f486381f0 msgr2=0x564f4863c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:08.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.510+0000 7fda4b7fe640 1 --2- 172.21.15.52:0/4223189782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f486381f0 0x564f4863c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.510+0000 7fda4b7fe640 1 -- 172.21.15.52:0/4223189782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f486378d0 msgr2=0x564f4864ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:08.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.510+0000 7fda4b7fe640 1 --2- 172.21.15.52:0/4223189782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f486378d0 0x564f4864ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.510+0000 7fda4b7fe640 1 -- 172.21.15.52:0/4223189782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f486431c0 con 0x564f48640dd0 2024-04-02T13:54:08.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.510+0000 7fda4b7fe640 1 --2- 172.21.15.52:0/4223189782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f48640dd0 0x564f48640210 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fda44012300 tx=0x7fda44010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890cf1253d25fbd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:08.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.510+0000 7fda40ff9640 1 -- 172.21.15.52:0/4223189782 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4402c070 con 0x564f48640dd0 2024-04-02T13:54:08.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.511+0000 7fda40ff9640 1 -- 172.21.15.52:0/4223189782 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda44010c40 con 0x564f48640dd0 2024-04-02T13:54:08.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.511+0000 7fda40ff9640 1 -- 172.21.15.52:0/4223189782 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda44023040 con 0x564f48640dd0 2024-04-02T13:54:08.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.511+0000 7fda53f71740 1 -- 172.21.15.52:0/4223189782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f48640dd0 msgr2=0x564f48640210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:08.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.511+0000 7fda53f71740 1 --2- 172.21.15.52:0/4223189782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f48640dd0 0x564f48640210 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fda44012300 tx=0x7fda44010740 comp rx=0 tx=0).stop 2024-04-02T13:54:08.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.511+0000 7fda53f71740 1 -- 172.21.15.52:0/4223189782 shutdown_connections 2024-04-02T13:54:08.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.511+0000 7fda53f71740 1 --2- 172.21.15.52:0/4223189782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f48640dd0 0x564f48640210 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.511+0000 7fda53f71740 1 --2- 172.21.15.52:0/4223189782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f486381f0 0x564f4863c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.511+0000 7fda53f71740 1 --2- 172.21.15.52:0/4223189782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f486378d0 0x564f4864ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.511+0000 7fda53f71740 1 -- 172.21.15.52:0/4223189782 >> 172.21.15.52:0/4223189782 conn(0x564f48632000 msgr2=0x564f4863c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:08.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.511+0000 7fda53f71740 1 -- 172.21.15.52:0/4223189782 shutdown_connections 2024-04-02T13:54:08.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.511+0000 7fda53f71740 1 -- 172.21.15.52:0/4223189782 wait complete. 2024-04-02T13:54:08.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.512+0000 7fda53f71740 1 -- start start 2024-04-02T13:54:08.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.512+0000 7fda53f71740 1 Processor -- start 2024-04-02T13:54:08.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.512+0000 7fda53f71740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f486378d0 0x564f48648f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.512+0000 7fda53f71740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f486381f0 0x564f48649470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.512+0000 7fda53f71740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f48640dd0 0x564f48639e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.512+0000 7fda53f71740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c1b2420 con 0x564f486378d0 2024-04-02T13:54:08.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.512+0000 7fda53f71740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c1b1d50 con 0x564f486381f0 2024-04-02T13:54:08.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.512+0000 7fda53f71740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c1aef90 con 0x564f48640dd0 2024-04-02T13:54:08.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.512+0000 7fda53f71740 10 client.0 _ll_register_callbacks cb 0x7fda44012ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:08.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.512+0000 7fda4affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f486378d0 0x564f48648f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:08.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.512+0000 7fda4affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f486378d0 0x564f48648f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33114/0 (socket says 172.21.15.52:33114) 2024-04-02T13:54:08.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.512+0000 7fda4affd640 1 -- 172.21.15.52:0/1695352628 learned_addr learned my addr 172.21.15.52:0/1695352628 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:08.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.512+0000 7fda43fff640 1 --2- 172.21.15.52:0/1695352628 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f486381f0 0x564f48649470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:08.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.512+0000 7fda4b7fe640 1 --2- 172.21.15.52:0/1695352628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f48640dd0 0x564f48639e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:08.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.512+0000 7fda4affd640 1 -- 172.21.15.52:0/1695352628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f48640dd0 msgr2=0x564f48639e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:08.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.512+0000 7fda4affd640 1 --2- 172.21.15.52:0/1695352628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f48640dd0 0x564f48639e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.512+0000 7fda4affd640 1 -- 172.21.15.52:0/1695352628 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f486381f0 msgr2=0x564f48649470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:08.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.512+0000 7fda4affd640 1 --2- 172.21.15.52:0/1695352628 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f486381f0 0x564f48649470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.512+0000 7fda4affd640 1 -- 172.21.15.52:0/1695352628 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f4863a600 con 0x564f486378d0 2024-04-02T13:54:08.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.512+0000 7fda4b7fe640 1 --2- 172.21.15.52:0/1695352628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f48640dd0 0x564f48639e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:54:08.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.512+0000 7fda4affd640 1 --2- 172.21.15.52:0/1695352628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f486378d0 0x564f48648f30 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00b4a0 tx=0x7fda3c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c079366fa9fb5c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:08.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.512+0000 7fda327fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:08.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.513+0000 7fda327fc640 1 -- 172.21.15.52:0/1695352628 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c004300 con 0x564f486378d0 2024-04-02T13:54:08.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.513+0000 7fda327fc640 1 -- 172.21.15.52:0/1695352628 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fda3c01d3b0 con 0x564f486378d0 2024-04-02T13:54:08.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.513+0000 7fda53f71740 1 -- 172.21.15.52:0/1695352628 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda44010c40 con 0x564f486378d0 2024-04-02T13:54:08.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.513+0000 7fda327fc640 1 -- 172.21.15.52:0/1695352628 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fda3c02c050 con 0x564f486378d0 2024-04-02T13:54:08.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.513+0000 7fda53f71740 10 client.15810 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:08.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.513+0000 7fda53f71740 1 -- 172.21.15.52:0/1695352628 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564f486b6ac0 con 0x564f486378d0 2024-04-02T13:54:08.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.513+0000 7fda327fc640 1 -- 172.21.15.52:0/1695352628 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda44010c40 con 0x564f486378d0 2024-04-02T13:54:08.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.513+0000 7fda327fc640 1 -- 172.21.15.52:0/1695352628 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fda3c0253b0 con 0x564f486378d0 2024-04-02T13:54:08.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.513+0000 7fda53f71740 10 client.15810 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:08.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.513+0000 7fda53f71740 10 client.15810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:54:08.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.513+0000 7fda53f71740 20 client.15810 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:08.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.513+0000 7fda53f71740 1 --2- 172.21.15.52:0/1695352628 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564f486b7240 0x564f486b9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.513+0000 7fda53f71740 4 client.15810 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:54:08.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.513+0000 7fda53f71740 1 -- 172.21.15.52:0/1695352628 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564f48520560 con 0x564f486b7240 2024-04-02T13:54:08.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.514+0000 7fda43fff640 1 --2- 172.21.15.52:0/1695352628 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564f486b7240 0x564f486b9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:08.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.514+0000 7fda327fc640 10 client.15810 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:08.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.514+0000 7fda43fff640 1 --2- 172.21.15.52:0/1695352628 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564f486b7240 0x564f486b9660 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e26f95c40de8510f server_cookie=1f043afd2169273f in_seq=0 out_seq=0 2024-04-02T13:54:08.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.514+0000 7fda327fc640 1 -- 172.21.15.52:0/1695352628 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564f48520560 con 0x564f486b7240 2024-04-02T13:54:08.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.514+0000 7fda327fc640 10 client.15810 handle_command_reply: tid=0 2024-04-02T13:54:08.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.514+0000 7fda53f71740 1 client.15810 shutdown 2024-04-02T13:54:08.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.514+0000 7fda53f71740 1 -- 172.21.15.52:0/1695352628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f486378d0 msgr2=0x564f48648f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:08.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.514+0000 7fda53f71740 1 --2- 172.21.15.52:0/1695352628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f486378d0 0x564f48648f30 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00b4a0 tx=0x7fda3c0118d0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.514+0000 7fda53f71740 1 -- 172.21.15.52:0/1695352628 shutdown_connections 2024-04-02T13:54:08.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.514+0000 7fda53f71740 1 --2- 172.21.15.52:0/1695352628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f48640dd0 0x564f48639e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.514+0000 7fda53f71740 1 --2- 172.21.15.52:0/1695352628 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f486381f0 0x564f48649470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.514+0000 7fda53f71740 1 --2- 172.21.15.52:0/1695352628 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564f486b7240 0x564f486b9660 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.514+0000 7fda53f71740 1 --2- 172.21.15.52:0/1695352628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564f486378d0 0x564f48648f30 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.514+0000 7fda327fc640 0 client.15810 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:08.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.514+0000 7fda53f71740 1 -- 172.21.15.52:0/1695352628 >> 172.21.15.52:0/1695352628 conn(0x564f48632000 msgr2=0x564f48633cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:08.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.515+0000 7fda53f71740 1 -- 172.21.15.52:0/1695352628 shutdown_connections 2024-04-02T13:54:08.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.515+0000 7fda53f71740 1 -- 172.21.15.52:0/1695352628 wait complete. 2024-04-02T13:54:08.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.515+0000 7fda53f71740 20 client.15810 trim_cache size 0 max 16384 2024-04-02T13:54:08.524 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:54:08.524 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-04-02T13:54:08.524 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-04-02T13:54:08.524 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:54:08.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.515+0000 7fda512b1640 1 -- 172.21.15.52:0/4181990075 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fda380a3ed0 msgr2=0x7fda380a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:08.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.515+0000 7fda512b1640 1 --2- 172.21.15.52:0/4181990075 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fda380a3ed0 0x7fda380a6390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fda44005c40 tx=0x7fda4400b000 comp rx=0 tx=0).stop 2024-04-02T13:54:08.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.515+0000 7fda512b1640 1 -- 172.21.15.52:0/4181990075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda4c101af0 msgr2=0x7fda4c0fc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:08.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.515+0000 7fda512b1640 1 --2- 172.21.15.52:0/4181990075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda4c101af0 0x7fda4c0fc910 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fda34006010 tx=0x7fda340043b0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.515+0000 7fda512b1640 1 -- 172.21.15.52:0/4181990075 shutdown_connections 2024-04-02T13:54:08.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.515+0000 7fda512b1640 1 --2- 172.21.15.52:0/4181990075 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fda380a3ed0 0x7fda380a6390 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.515+0000 7fda512b1640 1 --2- 172.21.15.52:0/4181990075 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda4c1059b0 0x7fda4c0fce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.515+0000 7fda512b1640 1 --2- 172.21.15.52:0/4181990075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda4c101af0 0x7fda4c0fc910 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.515+0000 7fda512b1640 1 --2- 172.21.15.52:0/4181990075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda4c101150 0x7fda4c0fc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.515+0000 7fda512b1640 1 -- 172.21.15.52:0/4181990075 >> 172.21.15.52:0/4181990075 conn(0x7fda4c076fc0 msgr2=0x7fda4c1002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:08.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.515+0000 7fda512b1640 1 -- 172.21.15.52:0/4181990075 shutdown_connections 2024-04-02T13:54:08.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.515+0000 7fda512b1640 1 -- 172.21.15.52:0/4181990075 wait complete. 2024-04-02T13:54:08.529 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:54:08.529 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:54:08.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.629+0000 7f133b893640 1 -- 172.21.15.52:0/3809746751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13341013c0 msgr2=0x7f133410d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:08.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.629+0000 7f133b893640 1 --2- 172.21.15.52:0/3809746751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13341013c0 0x7f133410d8d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f132c009cd0 tx=0x7f132c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.629+0000 7f133b893640 1 -- 172.21.15.52:0/3809746751 shutdown_connections 2024-04-02T13:54:08.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.629+0000 7f133b893640 1 --2- 172.21.15.52:0/3809746751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13341013c0 0x7f133410d8d0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.629+0000 7f133b893640 1 --2- 172.21.15.52:0/3809746751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1334100a00 0x7f1334100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.629+0000 7f133b893640 1 --2- 172.21.15.52:0/3809746751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1334106a10 0x7f1334106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.629+0000 7f133b893640 1 -- 172.21.15.52:0/3809746751 >> 172.21.15.52:0/3809746751 conn(0x7f1334077b00 msgr2=0x7f1334077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:08.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.630+0000 7f133b893640 1 -- 172.21.15.52:0/3809746751 shutdown_connections 2024-04-02T13:54:08.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.630+0000 7f133b893640 1 -- 172.21.15.52:0/3809746751 wait complete. 2024-04-02T13:54:08.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.630+0000 7f133b893640 1 Processor -- start 2024-04-02T13:54:08.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.630+0000 7f133b893640 1 -- start start 2024-04-02T13:54:08.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.630+0000 7f133b893640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1334100a00 0x7f13341983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.630+0000 7f133b893640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13341013c0 0x7f13341988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.630+0000 7f133b893640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1334106a10 0x7f133419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.630+0000 7f133b893640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1334110e20 con 0x7f1334100a00 2024-04-02T13:54:08.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.630+0000 7f133b893640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1334110ca0 con 0x7f13341013c0 2024-04-02T13:54:08.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.630+0000 7f133b893640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1334110fa0 con 0x7f1334106a10 2024-04-02T13:54:08.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.631+0000 7f133962f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1334100a00 0x7f13341983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:08.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.631+0000 7f133962f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1334100a00 0x7f13341983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33132/0 (socket says 172.21.15.52:33132) 2024-04-02T13:54:08.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.631+0000 7f133962f640 1 -- 172.21.15.52:0/389018545 learned_addr learned my addr 172.21.15.52:0/389018545 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:08.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.631+0000 7f1338e2e640 1 --2- 172.21.15.52:0/389018545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13341013c0 0x7f13341988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:08.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.631+0000 7f1339e30640 1 --2- 172.21.15.52:0/389018545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1334106a10 0x7f133419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:08.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.631+0000 7f133962f640 1 -- 172.21.15.52:0/389018545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1334106a10 msgr2=0x7f133419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:08.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.631+0000 7f133962f640 1 --2- 172.21.15.52:0/389018545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1334106a10 0x7f133419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.631+0000 7f133962f640 1 -- 172.21.15.52:0/389018545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13341013c0 msgr2=0x7f13341988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:08.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.631+0000 7f133962f640 1 --2- 172.21.15.52:0/389018545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13341013c0 0x7f13341988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.631+0000 7f133962f640 1 -- 172.21.15.52:0/389018545 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133419d390 con 0x7f1334100a00 2024-04-02T13:54:08.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.631+0000 7f133962f640 1 --2- 172.21.15.52:0/389018545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1334100a00 0x7f13341983a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f133000d770 tx=0x7f133000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:08.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.631+0000 7f132a7fc640 1 -- 172.21.15.52:0/389018545 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330017070 con 0x7f1334100a00 2024-04-02T13:54:08.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.631+0000 7f133b893640 1 -- 172.21.15.52:0/389018545 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f133419d680 con 0x7f1334100a00 2024-04-02T13:54:08.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.631+0000 7f132a7fc640 1 -- 172.21.15.52:0/389018545 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1330005500 con 0x7f1334100a00 2024-04-02T13:54:08.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.631+0000 7f132a7fc640 1 -- 172.21.15.52:0/389018545 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330004e00 con 0x7f1334100a00 2024-04-02T13:54:08.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.631+0000 7f133b893640 1 -- 172.21.15.52:0/389018545 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13341a4f10 con 0x7f1334100a00 2024-04-02T13:54:08.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.632+0000 7f133b893640 1 -- 172.21.15.52:0/389018545 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13340688f0 con 0x7f1334100a00 2024-04-02T13:54:08.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.634+0000 7f132a7fc640 1 -- 172.21.15.52:0/389018545 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f1330004fa0 con 0x7f1334100a00 2024-04-02T13:54:08.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.635+0000 7f132a7fc640 1 --2- 172.21.15.52:0/389018545 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f13200a3ed0 0x7f13200a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.635+0000 7f1338e2e640 1 --2- 172.21.15.52:0/389018545 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f13200a3ed0 0x7f13200a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:08.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.635+0000 7f132a7fc640 1 -- 172.21.15.52:0/389018545 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f13300d0b50 con 0x7f1334100a00 2024-04-02T13:54:08.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.635+0000 7f1338e2e640 1 --2- 172.21.15.52:0/389018545 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f13200a3ed0 0x7f13200a6390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1334069d40 tx=0x7f132400b390 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:08.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.638+0000 7f132a7fc640 1 -- 172.21.15.52:0/389018545 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f133009c620 con 0x7f1334100a00 2024-04-02T13:54:08.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.864+0000 7f133b893640 1 -- 172.21.15.52:0/389018545 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13341a50e0 con 0x7f1334100a00 2024-04-02T13:54:08.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.865+0000 7f132a7fc640 1 -- 172.21.15.52:0/389018545 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11923 (secure 0 0 0) 0x7f13300a0920 con 0x7f1334100a00 2024-04-02T13:54:08.868 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:54:08.868 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:53:28.475061+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24457,24467,24490,24503,24475]},"id":1}]} 2024-04-02T13:54:08.868 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 28 2024-04-02T13:54:08.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.867+0000 7f133b893640 1 -- 172.21.15.52:0/389018545 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f13200a3ed0 msgr2=0x7f13200a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:08.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.867+0000 7f133b893640 1 --2- 172.21.15.52:0/389018545 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f13200a3ed0 0x7f13200a6390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1334069d40 tx=0x7f132400b390 comp rx=0 tx=0).stop 2024-04-02T13:54:08.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.867+0000 7f133b893640 1 -- 172.21.15.52:0/389018545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1334100a00 msgr2=0x7f13341983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:08.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.867+0000 7f133b893640 1 --2- 172.21.15.52:0/389018545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1334100a00 0x7f13341983a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f133000d770 tx=0x7f133000dc30 comp rx=0 tx=0).stop 2024-04-02T13:54:08.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.867+0000 7f133b893640 1 -- 172.21.15.52:0/389018545 shutdown_connections 2024-04-02T13:54:08.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.867+0000 7f133b893640 1 --2- 172.21.15.52:0/389018545 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f13200a3ed0 0x7f13200a6390 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.867+0000 7f133b893640 1 --2- 172.21.15.52:0/389018545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1334106a10 0x7f133419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.867+0000 7f133b893640 1 --2- 172.21.15.52:0/389018545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13341013c0 0x7f13341988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.867+0000 7f133b893640 1 --2- 172.21.15.52:0/389018545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1334100a00 0x7f13341983a0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.867+0000 7f133b893640 1 -- 172.21.15.52:0/389018545 >> 172.21.15.52:0/389018545 conn(0x7f1334077b00 msgr2=0x7f13340ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:08.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.867+0000 7f133b893640 1 -- 172.21.15.52:0/389018545 shutdown_connections 2024-04-02T13:54:08.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.867+0000 7f133b893640 1 -- 172.21.15.52:0/389018545 wait complete. 2024-04-02T13:54:08.881 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:54:08.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.985+0000 7f006391c640 1 -- 172.21.15.52:0/1297678837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f005c100970 msgr2=0x7f005c100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:08.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.985+0000 7f006391c640 1 --2- 172.21.15.52:0/1297678837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f005c100970 0x7f005c100df0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f004c009930 tx=0x7f004c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:54:08.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.985+0000 7f006391c640 1 -- 172.21.15.52:0/1297678837 shutdown_connections 2024-04-02T13:54:08.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.985+0000 7f006391c640 1 --2- 172.21.15.52:0/1297678837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f005c101330 0x7f005c10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.985+0000 7f006391c640 1 --2- 172.21.15.52:0/1297678837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f005c100970 0x7f005c100df0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.985+0000 7f006391c640 1 --2- 172.21.15.52:0/1297678837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f005c106980 0x7f005c106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.985+0000 7f006391c640 1 -- 172.21.15.52:0/1297678837 >> 172.21.15.52:0/1297678837 conn(0x7f005c077b00 msgr2=0x7f005c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:08.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.985+0000 7f006391c640 1 -- 172.21.15.52:0/1297678837 shutdown_connections 2024-04-02T13:54:08.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.985+0000 7f006391c640 1 -- 172.21.15.52:0/1297678837 wait complete. 2024-04-02T13:54:08.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.985+0000 7f006391c640 1 Processor -- start 2024-04-02T13:54:08.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.985+0000 7f006391c640 1 -- start start 2024-04-02T13:54:08.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.986+0000 7f006391c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f005c100970 0x7f005c1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.986+0000 7f006391c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f005c101330 0x7f005c197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.986+0000 7f006391c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f005c106980 0x7f005c19bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.986+0000 7f006391c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f005c110d60 con 0x7f005c106980 2024-04-02T13:54:08.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.986+0000 7f006391c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f005c110be0 con 0x7f005c100970 2024-04-02T13:54:08.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.986+0000 7f006391c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f005c110ee0 con 0x7f005c101330 2024-04-02T13:54:08.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.986+0000 7f0061eb9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f005c106980 0x7f005c19bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:08.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.986+0000 7f0061eb9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f005c106980 0x7f005c19bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33154/0 (socket says 172.21.15.52:33154) 2024-04-02T13:54:08.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.986+0000 7f0061eb9640 1 -- 172.21.15.52:0/2632493608 learned_addr learned my addr 172.21.15.52:0/2632493608 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:08.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.986+0000 7f0060eb7640 1 --2- 172.21.15.52:0/2632493608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f005c101330 0x7f005c197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:08.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.986+0000 7f00616b8640 1 --2- 172.21.15.52:0/2632493608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f005c100970 0x7f005c1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:08.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.986+0000 7f0061eb9640 1 -- 172.21.15.52:0/2632493608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f005c101330 msgr2=0x7f005c197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:08.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.986+0000 7f0061eb9640 1 --2- 172.21.15.52:0/2632493608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f005c101330 0x7f005c197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.986+0000 7f0061eb9640 1 -- 172.21.15.52:0/2632493608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f005c100970 msgr2=0x7f005c1972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:08.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.986+0000 7f0061eb9640 1 --2- 172.21.15.52:0/2632493608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f005c100970 0x7f005c1972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:08.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.986+0000 7f0061eb9640 1 -- 172.21.15.52:0/2632493608 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005c19c2b0 con 0x7f005c106980 2024-04-02T13:54:08.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.986+0000 7f00616b8640 1 --2- 172.21.15.52:0/2632493608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f005c100970 0x7f005c1972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:54:08.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.986+0000 7f0061eb9640 1 --2- 172.21.15.52:0/2632493608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f005c106980 0x7f005c19bb80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f005400eb10 tx=0x7f005400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:08.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.986+0000 7f00527fc640 1 -- 172.21.15.52:0/2632493608 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054004b20 con 0x7f005c106980 2024-04-02T13:54:08.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.986+0000 7f00527fc640 1 -- 172.21.15.52:0/2632493608 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0054004cc0 con 0x7f005c106980 2024-04-02T13:54:08.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.986+0000 7f00527fc640 1 -- 172.21.15.52:0/2632493608 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054010760 con 0x7f005c106980 2024-04-02T13:54:08.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.986+0000 7f006391c640 1 -- 172.21.15.52:0/2632493608 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005c19c5a0 con 0x7f005c106980 2024-04-02T13:54:08.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.986+0000 7f006391c640 1 -- 172.21.15.52:0/2632493608 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f005c1a3e30 con 0x7f005c106980 2024-04-02T13:54:08.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.989+0000 7f00527fc640 1 -- 172.21.15.52:0/2632493608 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f0054005020 con 0x7f005c106980 2024-04-02T13:54:08.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.989+0000 7f00665e2740 1 Processor -- start 2024-04-02T13:54:08.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.989+0000 7f00665e2740 1 -- start start 2024-04-02T13:54:08.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.990+0000 7f00527fc640 1 --2- 172.21.15.52:0/2632493608 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f00480a3ed0 0x7f00480a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.990+0000 7f00665e2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6a992460 0x559e6a992840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:08.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.990+0000 7f00665e2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6a992d80 0x559e6a99d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.990+0000 7f00665e2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a99deb0 0x559e6a9a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.990+0000 7f00665e2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559e6a8f67b0 con 0x559e6a99deb0 2024-04-02T13:54:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.990+0000 7f00665e2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559e6a895570 con 0x559e6a992460 2024-04-02T13:54:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.990+0000 7f00665e2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559e6a8f2760 con 0x559e6a992d80 2024-04-02T13:54:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.990+0000 7f0061eb9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a99deb0 0x559e6a9a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.990+0000 7f0060eb7640 1 --2- 172.21.15.52:0/2632493608 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f00480a3ed0 0x7f00480a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.990+0000 7f0061eb9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a99deb0 0x559e6a9a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33166/0 (socket says 172.21.15.52:33166) 2024-04-02T13:54:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.990+0000 7f0061eb9640 1 -- 172.21.15.52:0/2912791842 learned_addr learned my addr 172.21.15.52:0/2912791842 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.990+0000 7f00527fc640 1 -- 172.21.15.52:0/2632493608 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f00540d1450 con 0x7f005c106980 2024-04-02T13:54:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.990+0000 7f00616b8640 1 --2- 172.21.15.52:0/2912791842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6a992460 0x559e6a992840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.990+0000 7f0061eb9640 1 -- 172.21.15.52:0/2912791842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6a992d80 msgr2=0x559e6a99d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.990+0000 7f0061eb9640 1 --2- 172.21.15.52:0/2912791842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6a992d80 0x559e6a99d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.990+0000 7f0061eb9640 1 -- 172.21.15.52:0/2912791842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6a992460 msgr2=0x559e6a992840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.990+0000 7f0061eb9640 1 --2- 172.21.15.52:0/2912791842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6a992460 0x559e6a992840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.990+0000 7f0061eb9640 1 -- 172.21.15.52:0/2912791842 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e6a60a3f0 con 0x559e6a99deb0 2024-04-02T13:54:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.990+0000 7f00616b8640 1 --2- 172.21.15.52:0/2912791842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6a992460 0x559e6a992840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:54:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.990+0000 7f0060eb7640 1 --2- 172.21.15.52:0/2632493608 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f00480a3ed0 0x7f00480a6390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f004c005ab0 tx=0x7f004c02c5d0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.990+0000 7f0061eb9640 1 --2- 172.21.15.52:0/2912791842 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a99deb0 0x559e6a9a02a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f00540ce760 tx=0x7f00540058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a01cacbaf74464 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.991+0000 7f00477fe640 1 -- 172.21.15.52:0/2912791842 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005409c9b0 con 0x559e6a99deb0 2024-04-02T13:54:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.991+0000 7f00477fe640 1 -- 172.21.15.52:0/2912791842 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f005409d570 con 0x559e6a99deb0 2024-04-02T13:54:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.991+0000 7f00477fe640 1 -- 172.21.15.52:0/2912791842 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005409c7d0 con 0x559e6a99deb0 2024-04-02T13:54:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.991+0000 7f00665e2740 1 -- 172.21.15.52:0/2912791842 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a99deb0 msgr2=0x559e6a9a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.991+0000 7f00665e2740 1 --2- 172.21.15.52:0/2912791842 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a99deb0 0x559e6a9a02a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f00540ce760 tx=0x7f00540058d0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.991+0000 7f00665e2740 1 -- 172.21.15.52:0/2912791842 shutdown_connections 2024-04-02T13:54:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.991+0000 7f00665e2740 1 --2- 172.21.15.52:0/2912791842 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a99deb0 0x559e6a9a02a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.991+0000 7f00665e2740 1 --2- 172.21.15.52:0/2912791842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6a992d80 0x559e6a99d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.991+0000 7f00665e2740 1 --2- 172.21.15.52:0/2912791842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6a992460 0x559e6a992840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.991+0000 7f00665e2740 1 -- 172.21.15.52:0/2912791842 >> 172.21.15.52:0/2912791842 conn(0x559e6a98ced0 msgr2=0x559e6a98d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.991+0000 7f00665e2740 1 -- 172.21.15.52:0/2912791842 shutdown_connections 2024-04-02T13:54:09.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.991+0000 7f00665e2740 1 -- 172.21.15.52:0/2912791842 wait complete. 2024-04-02T13:54:09.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.992+0000 7f00665e2740 1 -- start start 2024-04-02T13:54:09.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.992+0000 7f00665e2740 1 Processor -- start 2024-04-02T13:54:09.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.992+0000 7f00665e2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6a992460 0x559e6a99d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.992+0000 7f00665e2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a992d80 0x559e6a99ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.992+0000 7f00665e2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6a99deb0 0x559e6a99b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.992+0000 7f00665e2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559e6a9a0ca0 con 0x559e6a992d80 2024-04-02T13:54:09.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.992+0000 7f00665e2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559e6a8b1e80 con 0x559e6a992460 2024-04-02T13:54:09.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.992+0000 7f00665e2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559e6a9a0e20 con 0x559e6a99deb0 2024-04-02T13:54:09.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.992+0000 7f0060eb7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a992d80 0x559e6a99ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.992+0000 7f00665e2740 10 client.0 _ll_register_callbacks cb 0x559e6a8edaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:09.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.992+0000 7f0060eb7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a992d80 0x559e6a99ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33168/0 (socket says 172.21.15.52:33168) 2024-04-02T13:54:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.992+0000 7f0060eb7640 1 -- 172.21.15.52:0/3758662623 learned_addr learned my addr 172.21.15.52:0/3758662623 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.993+0000 7f00616b8640 1 --2- 172.21.15.52:0/3758662623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6a992460 0x559e6a99d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.993+0000 7f0060eb7640 1 -- 172.21.15.52:0/3758662623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6a99deb0 msgr2=0x559e6a99b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.993+0000 7f0061eb9640 1 --2- 172.21.15.52:0/3758662623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6a99deb0 0x559e6a99b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.993+0000 7f0060eb7640 1 --2- 172.21.15.52:0/3758662623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6a99deb0 0x559e6a99b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.993+0000 7f0060eb7640 1 -- 172.21.15.52:0/3758662623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6a992460 msgr2=0x559e6a99d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.993+0000 7f0060eb7640 1 --2- 172.21.15.52:0/3758662623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6a992460 0x559e6a99d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.993+0000 7f0060eb7640 1 -- 172.21.15.52:0/3758662623 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e6a6d47d0 con 0x559e6a992d80 2024-04-02T13:54:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.993+0000 7f00616b8640 1 --2- 172.21.15.52:0/3758662623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6a992460 0x559e6a99d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:54:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.993+0000 7f0061eb9640 1 --2- 172.21.15.52:0/3758662623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6a99deb0 0x559e6a99b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:54:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.993+0000 7f0060eb7640 1 --2- 172.21.15.52:0/3758662623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a992d80 0x559e6a99ac10 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f004c009930 tx=0x7f004c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fdcb68ea73a61dc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.993+0000 7f0044ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.993+0000 7f0044ff9640 1 -- 172.21.15.52:0/3758662623 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c004c50 con 0x559e6a992d80 2024-04-02T13:54:09.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.993+0000 7f0044ff9640 1 -- 172.21.15.52:0/3758662623 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f004c004f50 con 0x559e6a992d80 2024-04-02T13:54:09.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.993+0000 7f00665e2740 1 -- 172.21.15.52:0/3758662623 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f005409d570 con 0x559e6a992d80 2024-04-02T13:54:09.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.993+0000 7f0044ff9640 1 -- 172.21.15.52:0/3758662623 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f004c05a050 con 0x559e6a992d80 2024-04-02T13:54:09.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.993+0000 7f00665e2740 10 client.15828 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:09.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.993+0000 7f00665e2740 1 -- 172.21.15.52:0/3758662623 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559e6a9acce0 con 0x559e6a992d80 2024-04-02T13:54:09.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.993+0000 7f0044ff9640 1 -- 172.21.15.52:0/3758662623 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f005409d570 con 0x559e6a992d80 2024-04-02T13:54:09.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.994+0000 7f0044ff9640 1 -- 172.21.15.52:0/3758662623 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f004c04c470 con 0x559e6a992d80 2024-04-02T13:54:09.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.994+0000 7f00665e2740 10 client.15828 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:09.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.994+0000 7f00665e2740 10 client.15828 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T13:54:09.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.994+0000 7f00665e2740 20 client.15828 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:09.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.994+0000 7f00665e2740 1 --2- 172.21.15.52:0/3758662623 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559e6a9ae240 0x559e6a9b0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.994+0000 7f00665e2740 4 client.15828 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:54:09.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.994+0000 7f00665e2740 1 -- 172.21.15.52:0/3758662623 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e6a903cf0 con 0x559e6a9ae240 2024-04-02T13:54:09.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.994+0000 7f00616b8640 1 --2- 172.21.15.52:0/3758662623 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559e6a9ae240 0x559e6a9b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.994+0000 7f0044ff9640 10 client.15828 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:54:09.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.994+0000 7f00616b8640 1 --2- 172.21.15.52:0/3758662623 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559e6a9ae240 0x559e6a9b0680 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e45cad0bef5ae5d9 server_cookie=7dac0aafdd34f007 in_seq=0 out_seq=0 2024-04-02T13:54:09.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.995+0000 7f0044ff9640 1 -- 172.21.15.52:0/3758662623 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e6a903cf0 con 0x559e6a9ae240 2024-04-02T13:54:09.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.995+0000 7f0044ff9640 10 client.15828 handle_command_reply: tid=0 2024-04-02T13:54:09.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.995+0000 7f00665e2740 1 client.15828 shutdown 2024-04-02T13:54:09.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.995+0000 7f00665e2740 1 -- 172.21.15.52:0/3758662623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a992d80 msgr2=0x559e6a99ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.995+0000 7f00665e2740 1 --2- 172.21.15.52:0/3758662623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a992d80 0x559e6a99ac10 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f004c009930 tx=0x7f004c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:54:09.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.995+0000 7f00665e2740 1 -- 172.21.15.52:0/3758662623 shutdown_connections 2024-04-02T13:54:09.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.995+0000 7f00665e2740 1 --2- 172.21.15.52:0/3758662623 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559e6a9ae240 0x559e6a9b0680 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.995+0000 7f00665e2740 1 --2- 172.21.15.52:0/3758662623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6a99deb0 0x559e6a99b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.995+0000 7f00665e2740 1 --2- 172.21.15.52:0/3758662623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a992d80 0x559e6a99ac10 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.995+0000 7f00665e2740 1 --2- 172.21.15.52:0/3758662623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6a992460 0x559e6a99d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.995+0000 7f00665e2740 1 -- 172.21.15.52:0/3758662623 >> 172.21.15.52:0/3758662623 conn(0x559e6a98ced0 msgr2=0x559e6a98ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.995+0000 7f0044ff9640 0 client.15828 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:54:09.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.995+0000 7f00665e2740 1 -- 172.21.15.52:0/3758662623 shutdown_connections 2024-04-02T13:54:09.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.995+0000 7f00665e2740 1 -- 172.21.15.52:0/3758662623 wait complete. 2024-04-02T13:54:09.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:08.996+0000 7f00665e2740 20 client.15828 trim_cache size 0 max 16384 2024-04-02T13:54:09.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.009+0000 7f00665e2740 1 Processor -- start 2024-04-02T13:54:09.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.009+0000 7f00665e2740 1 -- start start 2024-04-02T13:54:09.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.009+0000 7f00665e2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6a992460 0x559e6a9881f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.009+0000 7f00665e2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a99deb0 0x559e6a98e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.009+0000 7f00665e2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6a9965c0 0x559e6a99d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.009+0000 7f00665e2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f005c1acc50 con 0x559e6a99deb0 2024-04-02T13:54:09.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.009+0000 7f00665e2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f005c1ae790 con 0x559e6a9965c0 2024-04-02T13:54:09.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.009+0000 7f00665e2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f005c1ad320 con 0x559e6a992460 2024-04-02T13:54:09.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.009+0000 7f0060eb7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a99deb0 0x559e6a98e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.009+0000 7f0060eb7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a99deb0 0x559e6a98e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33182/0 (socket says 172.21.15.52:33182) 2024-04-02T13:54:09.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.009+0000 7f0060eb7640 1 -- 172.21.15.52:0/3775168449 learned_addr learned my addr 172.21.15.52:0/3775168449 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:09.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.009+0000 7f0060eb7640 1 -- 172.21.15.52:0/3775168449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6a992460 msgr2=0x559e6a9881f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.009+0000 7f0060eb7640 1 --2- 172.21.15.52:0/3775168449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6a992460 0x559e6a9881f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.009+0000 7f0060eb7640 1 -- 172.21.15.52:0/3775168449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6a9965c0 msgr2=0x559e6a99d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.009+0000 7f0060eb7640 1 --2- 172.21.15.52:0/3775168449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6a9965c0 0x559e6a99d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.009+0000 7f0060eb7640 1 -- 172.21.15.52:0/3775168449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e6a992bb0 con 0x559e6a99deb0 2024-04-02T13:54:09.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.009+0000 7f0060eb7640 1 --2- 172.21.15.52:0/3775168449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a99deb0 0x559e6a98e9a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f004c036040 tx=0x7f004c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88573af60c654481 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.009+0000 7f0046ffd640 1 -- 172.21.15.52:0/3775168449 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c04cd00 con 0x559e6a99deb0 2024-04-02T13:54:09.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.010+0000 7f0046ffd640 1 -- 172.21.15.52:0/3775168449 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f004c051070 con 0x559e6a99deb0 2024-04-02T13:54:09.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.010+0000 7f0046ffd640 1 -- 172.21.15.52:0/3775168449 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c004cb0 con 0x559e6a99deb0 2024-04-02T13:54:09.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.010+0000 7f00665e2740 1 -- 172.21.15.52:0/3775168449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a99deb0 msgr2=0x559e6a98e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.010+0000 7f00665e2740 1 --2- 172.21.15.52:0/3775168449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a99deb0 0x559e6a98e9a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f004c036040 tx=0x7f004c005fd0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.010+0000 7f00665e2740 1 -- 172.21.15.52:0/3775168449 shutdown_connections 2024-04-02T13:54:09.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.010+0000 7f00665e2740 1 --2- 172.21.15.52:0/3775168449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6a9965c0 0x559e6a99d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.010+0000 7f00665e2740 1 --2- 172.21.15.52:0/3775168449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a99deb0 0x559e6a98e9a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.010+0000 7f00665e2740 1 --2- 172.21.15.52:0/3775168449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6a992460 0x559e6a9881f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.010+0000 7f00665e2740 1 -- 172.21.15.52:0/3775168449 >> 172.21.15.52:0/3775168449 conn(0x559e6a98ced0 msgr2=0x559e6a98e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.010+0000 7f00665e2740 1 -- 172.21.15.52:0/3775168449 shutdown_connections 2024-04-02T13:54:09.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.010+0000 7f00665e2740 1 -- 172.21.15.52:0/3775168449 wait complete. 2024-04-02T13:54:09.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.010+0000 7f00665e2740 1 -- start start 2024-04-02T13:54:09.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.011+0000 7f00665e2740 1 Processor -- start 2024-04-02T13:54:09.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.011+0000 7f00665e2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6a992460 0x559e6a9a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.011+0000 7f00665e2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a9965c0 0x559e6a9a6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.011+0000 7f00665e2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6a99deb0 0x559e6a99f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.011+0000 7f00665e2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f005c1ae0c0 con 0x559e6a9965c0 2024-04-02T13:54:09.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.011+0000 7f00665e2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f005c1ad9f0 con 0x559e6a99deb0 2024-04-02T13:54:09.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.011+0000 7f00665e2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f005c1aac30 con 0x559e6a992460 2024-04-02T13:54:09.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.011+0000 7f00665e2740 10 client.0 _ll_register_callbacks cb 0x7f004c02cc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:09.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.011+0000 7f0060eb7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a9965c0 0x559e6a9a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.011+0000 7f0060eb7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a9965c0 0x559e6a9a6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33192/0 (socket says 172.21.15.52:33192) 2024-04-02T13:54:09.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.011+0000 7f0060eb7640 1 -- 172.21.15.52:0/2008993115 learned_addr learned my addr 172.21.15.52:0/2008993115 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:09.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.011+0000 7f0061eb9640 1 --2- 172.21.15.52:0/2008993115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6a99deb0 0x559e6a99f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.011+0000 7f0060eb7640 1 -- 172.21.15.52:0/2008993115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6a992460 msgr2=0x559e6a9a60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.011+0000 7f00616b8640 1 --2- 172.21.15.52:0/2008993115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6a992460 0x559e6a9a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.011+0000 7f0060eb7640 1 --2- 172.21.15.52:0/2008993115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6a992460 0x559e6a9a60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.011+0000 7f0060eb7640 1 -- 172.21.15.52:0/2008993115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6a99deb0 msgr2=0x559e6a99f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.011+0000 7f0060eb7640 1 --2- 172.21.15.52:0/2008993115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6a99deb0 0x559e6a99f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.011+0000 7f0060eb7640 1 -- 172.21.15.52:0/2008993115 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e6a99f6c0 con 0x559e6a9965c0 2024-04-02T13:54:09.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.011+0000 7f0061eb9640 1 --2- 172.21.15.52:0/2008993115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6a99deb0 0x559e6a99f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:54:09.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.012+0000 7f0060eb7640 1 --2- 172.21.15.52:0/2008993115 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a9965c0 0x559e6a9a6620 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f004c02c820 tx=0x7f004c04c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d611ebec8ceba3ef server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.012+0000 7f0044ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:09.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.012+0000 7f0044ff9640 1 -- 172.21.15.52:0/2008993115 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c06c060 con 0x559e6a9965c0 2024-04-02T13:54:09.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.012+0000 7f00665e2740 1 -- 172.21.15.52:0/2008993115 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f004c051070 con 0x559e6a9965c0 2024-04-02T13:54:09.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.012+0000 7f0044ff9640 1 -- 172.21.15.52:0/2008993115 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f004c004c50 con 0x559e6a9965c0 2024-04-02T13:54:09.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.012+0000 7f0044ff9640 1 -- 172.21.15.52:0/2008993115 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f004c073050 con 0x559e6a9965c0 2024-04-02T13:54:09.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.012+0000 7f00665e2740 10 client.15834 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:09.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.012+0000 7f00665e2740 1 -- 172.21.15.52:0/2008993115 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559e6aa14fa0 con 0x559e6a9965c0 2024-04-02T13:54:09.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.012+0000 7f0044ff9640 1 -- 172.21.15.52:0/2008993115 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f004c051070 con 0x559e6a9965c0 2024-04-02T13:54:09.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.012+0000 7f0044ff9640 1 -- 172.21.15.52:0/2008993115 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f004c003b80 con 0x559e6a9965c0 2024-04-02T13:54:09.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.012+0000 7f00665e2740 10 client.15834 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:09.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.012+0000 7f00665e2740 10 client.15834 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T13:54:09.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.012+0000 7f00665e2740 20 client.15834 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:09.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.012+0000 7f00665e2740 1 --2- 172.21.15.52:0/2008993115 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559e6aa15610 0x559e6aa17a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.012+0000 7f00665e2740 4 client.15834 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:54:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.012+0000 7f00665e2740 1 -- 172.21.15.52:0/2008993115 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e6a8fe040 con 0x559e6aa15610 2024-04-02T13:54:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.013+0000 7f00616b8640 1 --2- 172.21.15.52:0/2008993115 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559e6aa15610 0x559e6aa17a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.013+0000 7f0044ff9640 10 client.15834 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:54:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.013+0000 7f00616b8640 1 --2- 172.21.15.52:0/2008993115 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559e6aa15610 0x559e6aa17a30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8ae6287e6d42ca16 server_cookie=d293dcfa08d536c1 in_seq=0 out_seq=0 2024-04-02T13:54:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.013+0000 7f0044ff9640 1 -- 172.21.15.52:0/2008993115 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e6a8fe040 con 0x559e6aa15610 2024-04-02T13:54:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.013+0000 7f0044ff9640 10 client.15834 handle_command_reply: tid=0 2024-04-02T13:54:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.013+0000 7f00665e2740 1 client.15834 shutdown 2024-04-02T13:54:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.013+0000 7f00665e2740 1 -- 172.21.15.52:0/2008993115 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a9965c0 msgr2=0x559e6a9a6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.013+0000 7f00665e2740 1 --2- 172.21.15.52:0/2008993115 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a9965c0 0x559e6a9a6620 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f004c02c820 tx=0x7f004c04c660 comp rx=0 tx=0).stop 2024-04-02T13:54:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.013+0000 7f00665e2740 1 -- 172.21.15.52:0/2008993115 shutdown_connections 2024-04-02T13:54:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.013+0000 7f00665e2740 1 --2- 172.21.15.52:0/2008993115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6a99deb0 0x559e6a99f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.013+0000 7f00665e2740 1 --2- 172.21.15.52:0/2008993115 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6a9965c0 0x559e6a9a6620 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.013+0000 7f00665e2740 1 --2- 172.21.15.52:0/2008993115 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559e6aa15610 0x559e6aa17a30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.013+0000 7f00665e2740 1 --2- 172.21.15.52:0/2008993115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6a992460 0x559e6a9a60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.013+0000 7f0044ff9640 0 client.15834 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:54:09.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.013+0000 7f00665e2740 1 -- 172.21.15.52:0/2008993115 >> 172.21.15.52:0/2008993115 conn(0x559e6a98ced0 msgr2=0x559e6a98eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.014+0000 7f00665e2740 1 -- 172.21.15.52:0/2008993115 shutdown_connections 2024-04-02T13:54:09.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.014+0000 7f00665e2740 1 -- 172.21.15.52:0/2008993115 wait complete. 2024-04-02T13:54:09.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.014+0000 7f00665e2740 20 client.15834 trim_cache size 0 max 16384 2024-04-02T13:54:09.023 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:54:09.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.014+0000 7f006391c640 1 -- 172.21.15.52:0/2632493608 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f00480a3ed0 msgr2=0x7f00480a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.014+0000 7f006391c640 1 --2- 172.21.15.52:0/2632493608 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f00480a3ed0 0x7f00480a6390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f004c005ab0 tx=0x7f004c02c5d0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.014+0000 7f006391c640 1 -- 172.21.15.52:0/2632493608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f005c106980 msgr2=0x7f005c19bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.014+0000 7f006391c640 1 --2- 172.21.15.52:0/2632493608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f005c106980 0x7f005c19bb80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f005400eb10 tx=0x7f005400c2d0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.014+0000 7f006391c640 1 -- 172.21.15.52:0/2632493608 shutdown_connections 2024-04-02T13:54:09.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.014+0000 7f006391c640 1 --2- 172.21.15.52:0/2632493608 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f00480a3ed0 0x7f00480a6390 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.014+0000 7f006391c640 1 --2- 172.21.15.52:0/2632493608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f005c106980 0x7f005c19bb80 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.014+0000 7f006391c640 1 --2- 172.21.15.52:0/2632493608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f005c101330 0x7f005c197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.014+0000 7f006391c640 1 --2- 172.21.15.52:0/2632493608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f005c100970 0x7f005c1972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.014+0000 7f006391c640 1 -- 172.21.15.52:0/2632493608 >> 172.21.15.52:0/2632493608 conn(0x7f005c077b00 msgr2=0x7f005c1031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.014+0000 7f006391c640 1 -- 172.21.15.52:0/2632493608 shutdown_connections 2024-04-02T13:54:09.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.014+0000 7f006391c640 1 -- 172.21.15.52:0/2632493608 wait complete. 2024-04-02T13:54:09.028 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:54:09.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.129+0000 7f835c59d640 1 -- 172.21.15.52:0/291321935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8354105dc0 msgr2=0x7f835410d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.129+0000 7f835c59d640 1 --2- 172.21.15.52:0/291321935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8354105dc0 0x7f835410d900 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8344009ca0 tx=0x7f834402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.129+0000 7f835c59d640 1 -- 172.21.15.52:0/291321935 shutdown_connections 2024-04-02T13:54:09.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.129+0000 7f835c59d640 1 --2- 172.21.15.52:0/291321935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8354105dc0 0x7f835410d900 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.130+0000 7f835c59d640 1 --2- 172.21.15.52:0/291321935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83541020f0 0x7f8354105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.130+0000 7f835c59d640 1 --2- 172.21.15.52:0/291321935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8354101740 0x7f8354101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.130+0000 7f835c59d640 1 -- 172.21.15.52:0/291321935 >> 172.21.15.52:0/291321935 conn(0x7f8354075d40 msgr2=0x7f8354076150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.130+0000 7f835c59d640 1 -- 172.21.15.52:0/291321935 shutdown_connections 2024-04-02T13:54:09.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.130+0000 7f835c59d640 1 -- 172.21.15.52:0/291321935 wait complete. 2024-04-02T13:54:09.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.130+0000 7f835c59d640 1 Processor -- start 2024-04-02T13:54:09.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.130+0000 7f835c59d640 1 -- start start 2024-04-02T13:54:09.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.130+0000 7f835c59d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8354101740 0x7f8354199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.130+0000 7f835c59d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8354105dc0 0x7f8354199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.130+0000 7f835c59d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8354199c40 0x7f83541a41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.130+0000 7f835c59d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8354110de0 con 0x7f8354105dc0 2024-04-02T13:54:09.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.130+0000 7f835c59d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8354110c60 con 0x7f8354199c40 2024-04-02T13:54:09.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.130+0000 7f835c59d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8354110f60 con 0x7f8354101740 2024-04-02T13:54:09.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.130+0000 7f8359b38640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8354105dc0 0x7f8354199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.130+0000 7f8359b38640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8354105dc0 0x7f8354199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33208/0 (socket says 172.21.15.52:33208) 2024-04-02T13:54:09.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.130+0000 7f8359b38640 1 -- 172.21.15.52:0/2248297017 learned_addr learned my addr 172.21.15.52:0/2248297017 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:09.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.131+0000 7f835a339640 1 --2- 172.21.15.52:0/2248297017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8354101740 0x7f8354199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.131+0000 7f835ab3a640 1 --2- 172.21.15.52:0/2248297017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8354199c40 0x7f83541a41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.131+0000 7f8359b38640 1 -- 172.21.15.52:0/2248297017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8354101740 msgr2=0x7f8354199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.131+0000 7f8359b38640 1 --2- 172.21.15.52:0/2248297017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8354101740 0x7f8354199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.131+0000 7f8359b38640 1 -- 172.21.15.52:0/2248297017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8354199c40 msgr2=0x7f83541a41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.131+0000 7f8359b38640 1 --2- 172.21.15.52:0/2248297017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8354199c40 0x7f83541a41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.131+0000 7f8359b38640 1 -- 172.21.15.52:0/2248297017 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83541a49c0 con 0x7f8354105dc0 2024-04-02T13:54:09.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.131+0000 7f835ab3a640 1 --2- 172.21.15.52:0/2248297017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8354199c40 0x7f83541a41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:54:09.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.131+0000 7f8359b38640 1 --2- 172.21.15.52:0/2248297017 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8354105dc0 0x7f8354199660 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f834800e9f0 tx=0x7f834800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.131+0000 7f83537fe640 1 -- 172.21.15.52:0/2248297017 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834800ccc0 con 0x7f8354105dc0 2024-04-02T13:54:09.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.131+0000 7f835c59d640 1 -- 172.21.15.52:0/2248297017 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83541a4cb0 con 0x7f8354105dc0 2024-04-02T13:54:09.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.131+0000 7f83537fe640 1 -- 172.21.15.52:0/2248297017 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8348010910 con 0x7f8354105dc0 2024-04-02T13:54:09.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.131+0000 7f83537fe640 1 -- 172.21.15.52:0/2248297017 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348018920 con 0x7f8354105dc0 2024-04-02T13:54:09.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.131+0000 7f835c59d640 1 -- 172.21.15.52:0/2248297017 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83541a51a0 con 0x7f8354105dc0 2024-04-02T13:54:09.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.133+0000 7f83537fe640 1 -- 172.21.15.52:0/2248297017 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f8348010ac0 con 0x7f8354105dc0 2024-04-02T13:54:09.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.134+0000 7f835f187740 1 Processor -- start 2024-04-02T13:54:09.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.134+0000 7f835f187740 1 -- start start 2024-04-02T13:54:09.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.134+0000 7f835f187740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603c2db07a0 0x5603c2db0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.134+0000 7f835f187740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603c2db10c0 0x5603c2db9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.134+0000 7f835f187740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 0x5603c2dbc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.134+0000 7f83537fe640 1 --2- 172.21.15.52:0/2248297017 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f833c0a3ed0 0x7f833c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.134+0000 7f835f187740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5603c2d147b0 con 0x5603c2db9ca0 2024-04-02T13:54:09.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.134+0000 7f835f187740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5603c2cb3570 con 0x5603c2db10c0 2024-04-02T13:54:09.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.134+0000 7f835f187740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5603c2d10760 con 0x5603c2db07a0 2024-04-02T13:54:09.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.134+0000 7f835ab3a640 1 --2- 172.21.15.52:0/2248297017 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f833c0a3ed0 0x7f833c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.134+0000 7f8359b38640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 0x5603c2dbc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.134+0000 7f835a339640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603c2db10c0 0x5603c2db9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.134+0000 7f8359b38640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 0x5603c2dbc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33222/0 (socket says 172.21.15.52:33222) 2024-04-02T13:54:09.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.134+0000 7f835ab3a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603c2db07a0 0x5603c2db0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.134+0000 7f835a339640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603c2db10c0 0x5603c2db9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:36706/0 (socket says 172.21.15.52:36706) 2024-04-02T13:54:09.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.134+0000 7f8359b38640 1 -- 172.21.15.52:0/1625947760 learned_addr learned my addr 172.21.15.52:0/1625947760 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:09.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.134+0000 7f83537fe640 1 -- 172.21.15.52:0/2248297017 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f83480d11a0 con 0x7f8354105dc0 2024-04-02T13:54:09.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.135+0000 7f8359b38640 1 -- 172.21.15.52:0/1625947760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603c2db07a0 msgr2=0x5603c2db0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.135+0000 7f8359b38640 1 --2- 172.21.15.52:0/1625947760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603c2db07a0 0x5603c2db0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.135+0000 7f8359b38640 1 -- 172.21.15.52:0/1625947760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603c2db10c0 msgr2=0x5603c2db9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.135+0000 7f8359b38640 1 --2- 172.21.15.52:0/1625947760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603c2db10c0 0x5603c2db9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.135+0000 7f8359b38640 1 -- 172.21.15.52:0/1625947760 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603c2a283f0 con 0x5603c2db9ca0 2024-04-02T13:54:09.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.135+0000 7f835ab3a640 1 --2- 172.21.15.52:0/2248297017 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f833c0a3ed0 0x7f833c0a6390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8344004af0 tx=0x7f83440324f0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.135+0000 7f8359b38640 1 --2- 172.21.15.52:0/1625947760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 0x5603c2dbc090 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f834809ba80 tx=0x7f83480671c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ff78b27e9f0713 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.135+0000 7f8350bfe640 1 -- 172.21.15.52:0/1625947760 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348026040 con 0x5603c2db9ca0 2024-04-02T13:54:09.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.135+0000 7f8350bfe640 1 -- 172.21.15.52:0/1625947760 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f834809d2c0 con 0x5603c2db9ca0 2024-04-02T13:54:09.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.135+0000 7f8350bfe640 1 -- 172.21.15.52:0/1625947760 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83480676d0 con 0x5603c2db9ca0 2024-04-02T13:54:09.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.136+0000 7f835f187740 1 -- 172.21.15.52:0/1625947760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 msgr2=0x5603c2dbc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.136+0000 7f835f187740 1 --2- 172.21.15.52:0/1625947760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 0x5603c2dbc090 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f834809ba80 tx=0x7f83480671c0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.136+0000 7f835f187740 1 -- 172.21.15.52:0/1625947760 shutdown_connections 2024-04-02T13:54:09.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.136+0000 7f835f187740 1 --2- 172.21.15.52:0/1625947760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 0x5603c2dbc090 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.136+0000 7f835f187740 1 --2- 172.21.15.52:0/1625947760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603c2db10c0 0x5603c2db9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.136+0000 7f835f187740 1 --2- 172.21.15.52:0/1625947760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603c2db07a0 0x5603c2db0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.136+0000 7f835f187740 1 -- 172.21.15.52:0/1625947760 >> 172.21.15.52:0/1625947760 conn(0x5603c2daaed0 msgr2=0x5603c2dab2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.136+0000 7f835f187740 1 -- 172.21.15.52:0/1625947760 shutdown_connections 2024-04-02T13:54:09.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.136+0000 7f835f187740 1 -- 172.21.15.52:0/1625947760 wait complete. 2024-04-02T13:54:09.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.136+0000 7f835f187740 1 -- start start 2024-04-02T13:54:09.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.136+0000 7f835f187740 1 Processor -- start 2024-04-02T13:54:09.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.137+0000 7f835f187740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603c2db07a0 0x5603c2db9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.137+0000 7f835f187740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603c2db10c0 0x5603c2db6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.137+0000 7f835f187740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 0x5603c2db71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.137+0000 7f835f187740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5603c2dbca30 con 0x5603c2db9ca0 2024-04-02T13:54:09.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.137+0000 7f835f187740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5603c2ccfe80 con 0x5603c2db10c0 2024-04-02T13:54:09.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.137+0000 7f835f187740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5603c2dbcbb0 con 0x5603c2db07a0 2024-04-02T13:54:09.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.137+0000 7f835f187740 10 client.0 _ll_register_callbacks cb 0x5603c2d0baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:09.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.137+0000 7f835ab3a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 0x5603c2db71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.137+0000 7f835ab3a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 0x5603c2db71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33224/0 (socket says 172.21.15.52:33224) 2024-04-02T13:54:09.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.137+0000 7f835ab3a640 1 -- 172.21.15.52:0/3866064549 learned_addr learned my addr 172.21.15.52:0/3866064549 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:09.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.137+0000 7f835a339640 1 --2- 172.21.15.52:0/3866064549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603c2db07a0 0x5603c2db9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.137+0000 7f8359b38640 1 --2- 172.21.15.52:0/3866064549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603c2db10c0 0x5603c2db6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.137+0000 7f835ab3a640 1 -- 172.21.15.52:0/3866064549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603c2db07a0 msgr2=0x5603c2db9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.137+0000 7f835ab3a640 1 --2- 172.21.15.52:0/3866064549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603c2db07a0 0x5603c2db9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.137+0000 7f835ab3a640 1 -- 172.21.15.52:0/3866064549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603c2db10c0 msgr2=0x5603c2db6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.137+0000 7f835ab3a640 1 --2- 172.21.15.52:0/3866064549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603c2db10c0 0x5603c2db6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.137+0000 7f835ab3a640 1 -- 172.21.15.52:0/3866064549 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603c2af27d0 con 0x5603c2db9ca0 2024-04-02T13:54:09.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.137+0000 7f835ab3a640 1 --2- 172.21.15.52:0/3866064549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 0x5603c2db71f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8344004d70 tx=0x7f8344045070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53454c3888341a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.137+0000 7f8339ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:09.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.137+0000 7f8339ffb640 1 -- 172.21.15.52:0/3866064549 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344049040 con 0x5603c2db9ca0 2024-04-02T13:54:09.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.137+0000 7f8339ffb640 1 -- 172.21.15.52:0/3866064549 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f8344004240 con 0x5603c2db9ca0 2024-04-02T13:54:09.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.137+0000 7f835f187740 1 -- 172.21.15.52:0/3866064549 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f834809d2c0 con 0x5603c2db9ca0 2024-04-02T13:54:09.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.137+0000 7f8339ffb640 1 -- 172.21.15.52:0/3866064549 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f834405b050 con 0x5603c2db9ca0 2024-04-02T13:54:09.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.138+0000 7f835f187740 10 client.15846 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:09.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.138+0000 7f835f187740 1 -- 172.21.15.52:0/3866064549 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5603c2dc8b10 con 0x5603c2db9ca0 2024-04-02T13:54:09.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.138+0000 7f8339ffb640 1 -- 172.21.15.52:0/3866064549 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f834809d2c0 con 0x5603c2db9ca0 2024-04-02T13:54:09.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.138+0000 7f8339ffb640 1 -- 172.21.15.52:0/3866064549 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f834404c070 con 0x5603c2db9ca0 2024-04-02T13:54:09.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.138+0000 7f835f187740 10 client.15846 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:09.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.138+0000 7f835f187740 10 client.15846 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T13:54:09.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.138+0000 7f835f187740 20 client.15846 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:09.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.138+0000 7f835f187740 1 --2- 172.21.15.52:0/3866064549 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5603c2dca020 0x5603c2dcc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.138+0000 7f835f187740 4 client.15846 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:54:09.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.138+0000 7f835f187740 1 -- 172.21.15.52:0/3866064549 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603c2d21cf0 con 0x5603c2dca020 2024-04-02T13:54:09.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.138+0000 7f835a339640 1 --2- 172.21.15.52:0/3866064549 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5603c2dca020 0x5603c2dcc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.139+0000 7f8339ffb640 10 client.15846 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:54:09.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.139+0000 7f835a339640 1 --2- 172.21.15.52:0/3866064549 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5603c2dca020 0x5603c2dcc450 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9a201facfd8118a5 server_cookie=6d007d3442378515 in_seq=0 out_seq=0 2024-04-02T13:54:09.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.139+0000 7f8339ffb640 1 -- 172.21.15.52:0/3866064549 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5603c2d21cf0 con 0x5603c2dca020 2024-04-02T13:54:09.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.139+0000 7f8339ffb640 10 client.15846 handle_command_reply: tid=0 2024-04-02T13:54:09.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.139+0000 7f835f187740 1 client.15846 shutdown 2024-04-02T13:54:09.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.139+0000 7f835f187740 1 -- 172.21.15.52:0/3866064549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 msgr2=0x5603c2db71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.139+0000 7f835f187740 1 --2- 172.21.15.52:0/3866064549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 0x5603c2db71f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8344004d70 tx=0x7f8344045070 comp rx=0 tx=0).stop 2024-04-02T13:54:09.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.139+0000 7f835f187740 1 -- 172.21.15.52:0/3866064549 shutdown_connections 2024-04-02T13:54:09.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.139+0000 7f835f187740 1 --2- 172.21.15.52:0/3866064549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 0x5603c2db71f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.139+0000 7f835f187740 1 --2- 172.21.15.52:0/3866064549 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5603c2dca020 0x5603c2dcc450 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.139+0000 7f835f187740 1 --2- 172.21.15.52:0/3866064549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603c2db10c0 0x5603c2db6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.139+0000 7f8339ffb640 0 client.15846 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:54:09.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.139+0000 7f835f187740 1 --2- 172.21.15.52:0/3866064549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603c2db07a0 0x5603c2db9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.139+0000 7f835f187740 1 -- 172.21.15.52:0/3866064549 >> 172.21.15.52:0/3866064549 conn(0x5603c2daaed0 msgr2=0x5603c2dacc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.139+0000 7f835f187740 1 -- 172.21.15.52:0/3866064549 shutdown_connections 2024-04-02T13:54:09.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.140+0000 7f835f187740 1 -- 172.21.15.52:0/3866064549 wait complete. 2024-04-02T13:54:09.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.140+0000 7f835f187740 20 client.15846 trim_cache size 0 max 16384 2024-04-02T13:54:09.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.153+0000 7f835f187740 1 Processor -- start 2024-04-02T13:54:09.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.153+0000 7f835f187740 1 -- start start 2024-04-02T13:54:09.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.153+0000 7f835f187740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603c2db07a0 0x5603c2db15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.153+0000 7f835f187740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603c2db10c0 0x5603c2dc58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.153+0000 7f835f187740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 0x5603c2db5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.153+0000 7f835f187740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f83541aed70 con 0x5603c2db9ca0 2024-04-02T13:54:09.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.153+0000 7f835f187740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f83541b08b0 con 0x5603c2db10c0 2024-04-02T13:54:09.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.153+0000 7f835f187740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f83541af440 con 0x5603c2db07a0 2024-04-02T13:54:09.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.153+0000 7f835ab3a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 0x5603c2db5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.153+0000 7f835ab3a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 0x5603c2db5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33230/0 (socket says 172.21.15.52:33230) 2024-04-02T13:54:09.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.153+0000 7f835ab3a640 1 -- 172.21.15.52:0/4047370099 learned_addr learned my addr 172.21.15.52:0/4047370099 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:09.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.153+0000 7f8359b38640 1 --2- 172.21.15.52:0/4047370099 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603c2db10c0 0x5603c2dc58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.153+0000 7f835ab3a640 1 -- 172.21.15.52:0/4047370099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603c2db07a0 msgr2=0x5603c2db15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.153+0000 7f835ab3a640 1 --2- 172.21.15.52:0/4047370099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603c2db07a0 0x5603c2db15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.153+0000 7f835ab3a640 1 -- 172.21.15.52:0/4047370099 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603c2db10c0 msgr2=0x5603c2dc58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.153+0000 7f835ab3a640 1 --2- 172.21.15.52:0/4047370099 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603c2db10c0 0x5603c2dc58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.153+0000 7f835ab3a640 1 -- 172.21.15.52:0/4047370099 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603c2dc6560 con 0x5603c2db9ca0 2024-04-02T13:54:09.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.154+0000 7f835ab3a640 1 --2- 172.21.15.52:0/4047370099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 0x5603c2db5a90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8344003fd0 tx=0x7f83440095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ed6a363ffcfa89 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.154+0000 7f833bfff640 1 -- 172.21.15.52:0/4047370099 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344064070 con 0x5603c2db9ca0 2024-04-02T13:54:09.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.154+0000 7f833bfff640 1 -- 172.21.15.52:0/4047370099 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8344052070 con 0x5603c2db9ca0 2024-04-02T13:54:09.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.154+0000 7f833bfff640 1 -- 172.21.15.52:0/4047370099 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344039070 con 0x5603c2db9ca0 2024-04-02T13:54:09.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.154+0000 7f835f187740 1 -- 172.21.15.52:0/4047370099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 msgr2=0x5603c2db5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.154+0000 7f835f187740 1 --2- 172.21.15.52:0/4047370099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 0x5603c2db5a90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8344003fd0 tx=0x7f83440095c0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.154+0000 7f835f187740 1 -- 172.21.15.52:0/4047370099 shutdown_connections 2024-04-02T13:54:09.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.154+0000 7f835f187740 1 --2- 172.21.15.52:0/4047370099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 0x5603c2db5a90 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.154+0000 7f835f187740 1 --2- 172.21.15.52:0/4047370099 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603c2db10c0 0x5603c2dc58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.154+0000 7f835f187740 1 --2- 172.21.15.52:0/4047370099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603c2db07a0 0x5603c2db15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.154+0000 7f835f187740 1 -- 172.21.15.52:0/4047370099 >> 172.21.15.52:0/4047370099 conn(0x5603c2daaed0 msgr2=0x5603c2db5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.154+0000 7f835f187740 1 -- 172.21.15.52:0/4047370099 shutdown_connections 2024-04-02T13:54:09.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.154+0000 7f835f187740 1 -- 172.21.15.52:0/4047370099 wait complete. 2024-04-02T13:54:09.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.154+0000 7f835f187740 1 -- start start 2024-04-02T13:54:09.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.155+0000 7f835f187740 1 Processor -- start 2024-04-02T13:54:09.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.155+0000 7f835f187740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603c2db07a0 0x5603c2dc1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.155+0000 7f835f187740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603c2db10c0 0x5603c2dc2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.155+0000 7f835f187740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 0x5603c2db3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.155+0000 7f835f187740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f83541b01e0 con 0x5603c2db9ca0 2024-04-02T13:54:09.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.155+0000 7f835f187740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f83541afb10 con 0x5603c2db10c0 2024-04-02T13:54:09.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.155+0000 7f835f187740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f83541acd50 con 0x5603c2db07a0 2024-04-02T13:54:09.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.155+0000 7f835f187740 10 client.0 _ll_register_callbacks cb 0x7f8348005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:09.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.155+0000 7f835ab3a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 0x5603c2db3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.155+0000 7f835ab3a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 0x5603c2db3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33244/0 (socket says 172.21.15.52:33244) 2024-04-02T13:54:09.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.155+0000 7f835ab3a640 1 -- 172.21.15.52:0/3649599460 learned_addr learned my addr 172.21.15.52:0/3649599460 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:09.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.155+0000 7f8359b38640 1 --2- 172.21.15.52:0/3649599460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603c2db10c0 0x5603c2dc2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.155+0000 7f835a339640 1 --2- 172.21.15.52:0/3649599460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603c2db07a0 0x5603c2dc1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.155+0000 7f835ab3a640 1 -- 172.21.15.52:0/3649599460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603c2db07a0 msgr2=0x5603c2dc1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.155+0000 7f835ab3a640 1 --2- 172.21.15.52:0/3649599460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603c2db07a0 0x5603c2dc1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.155+0000 7f835ab3a640 1 -- 172.21.15.52:0/3649599460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603c2db10c0 msgr2=0x5603c2dc2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.155+0000 7f835ab3a640 1 --2- 172.21.15.52:0/3649599460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603c2db10c0 0x5603c2dc2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.155+0000 7f835ab3a640 1 -- 172.21.15.52:0/3649599460 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603c2db3ac0 con 0x5603c2db9ca0 2024-04-02T13:54:09.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.155+0000 7f835ab3a640 1 --2- 172.21.15.52:0/3649599460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 0x5603c2db3290 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8344005540 tx=0x7f8344005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b03c801fe9c281 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.155+0000 7f8339ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:09.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.156+0000 7f8339ffb640 1 -- 172.21.15.52:0/3649599460 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344064070 con 0x5603c2db9ca0 2024-04-02T13:54:09.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.156+0000 7f8339ffb640 1 -- 172.21.15.52:0/3649599460 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f8344004040 con 0x5603c2db9ca0 2024-04-02T13:54:09.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.156+0000 7f835f187740 1 -- 172.21.15.52:0/3649599460 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8344052070 con 0x5603c2db9ca0 2024-04-02T13:54:09.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.156+0000 7f8339ffb640 1 -- 172.21.15.52:0/3649599460 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8344071050 con 0x5603c2db9ca0 2024-04-02T13:54:09.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.156+0000 7f835f187740 10 client.15852 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:09.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.156+0000 7f835f187740 1 -- 172.21.15.52:0/3649599460 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5603c2e30130 con 0x5603c2db9ca0 2024-04-02T13:54:09.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.156+0000 7f8339ffb640 1 -- 172.21.15.52:0/3649599460 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8344052070 con 0x5603c2db9ca0 2024-04-02T13:54:09.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.156+0000 7f8339ffb640 1 -- 172.21.15.52:0/3649599460 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f834404c070 con 0x5603c2db9ca0 2024-04-02T13:54:09.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.156+0000 7f835f187740 10 client.15852 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:09.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.156+0000 7f835f187740 10 client.15852 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T13:54:09.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.156+0000 7f835f187740 20 client.15852 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:09.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.156+0000 7f835f187740 1 --2- 172.21.15.52:0/3649599460 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5603c2e30730 0x5603c2e32b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.156+0000 7f835f187740 4 client.15852 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:54:09.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.156+0000 7f835f187740 1 -- 172.21.15.52:0/3649599460 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5603c2d1c040 con 0x5603c2e30730 2024-04-02T13:54:09.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.156+0000 7f835a339640 1 --2- 172.21.15.52:0/3649599460 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5603c2e30730 0x5603c2e32b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.157+0000 7f8339ffb640 10 client.15852 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:54:09.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.157+0000 7f835a339640 1 --2- 172.21.15.52:0/3649599460 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5603c2e30730 0x5603c2e32b50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e56156a21d7bae40 server_cookie=bddd08fafd7fd97f in_seq=0 out_seq=0 2024-04-02T13:54:09.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.157+0000 7f8339ffb640 1 -- 172.21.15.52:0/3649599460 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5603c2d1c040 con 0x5603c2e30730 2024-04-02T13:54:09.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.157+0000 7f8339ffb640 10 client.15852 handle_command_reply: tid=0 2024-04-02T13:54:09.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.157+0000 7f835f187740 1 client.15852 shutdown 2024-04-02T13:54:09.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.157+0000 7f835f187740 1 -- 172.21.15.52:0/3649599460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 msgr2=0x5603c2db3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.157+0000 7f835f187740 1 --2- 172.21.15.52:0/3649599460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 0x5603c2db3290 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8344005540 tx=0x7f8344005840 comp rx=0 tx=0).stop 2024-04-02T13:54:09.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.157+0000 7f835f187740 1 -- 172.21.15.52:0/3649599460 shutdown_connections 2024-04-02T13:54:09.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.157+0000 7f835f187740 1 --2- 172.21.15.52:0/3649599460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603c2db9ca0 0x5603c2db3290 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.157+0000 7f835f187740 1 --2- 172.21.15.52:0/3649599460 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5603c2e30730 0x5603c2e32b50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.157+0000 7f835f187740 1 --2- 172.21.15.52:0/3649599460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603c2db10c0 0x5603c2dc2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.157+0000 7f835f187740 1 --2- 172.21.15.52:0/3649599460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603c2db07a0 0x5603c2dc1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.157+0000 7f8339ffb640 0 client.15852 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:54:09.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.157+0000 7f835f187740 1 -- 172.21.15.52:0/3649599460 >> 172.21.15.52:0/3649599460 conn(0x5603c2daaed0 msgr2=0x5603c2db5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.157+0000 7f835f187740 1 -- 172.21.15.52:0/3649599460 shutdown_connections 2024-04-02T13:54:09.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.157+0000 7f835f187740 1 -- 172.21.15.52:0/3649599460 wait complete. 2024-04-02T13:54:09.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.157+0000 7f835f187740 20 client.15852 trim_cache size 0 max 16384 2024-04-02T13:54:09.167 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:54:09.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.158+0000 7f835c59d640 1 -- 172.21.15.52:0/2248297017 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f833c0a3ed0 msgr2=0x7f833c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.158+0000 7f835c59d640 1 --2- 172.21.15.52:0/2248297017 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f833c0a3ed0 0x7f833c0a6390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8344004af0 tx=0x7f83440324f0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.158+0000 7f835c59d640 1 -- 172.21.15.52:0/2248297017 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8354105dc0 msgr2=0x7f8354199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.158+0000 7f835c59d640 1 --2- 172.21.15.52:0/2248297017 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8354105dc0 0x7f8354199660 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f834800e9f0 tx=0x7f834800eeb0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.158+0000 7f835c59d640 1 -- 172.21.15.52:0/2248297017 shutdown_connections 2024-04-02T13:54:09.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.158+0000 7f835c59d640 1 --2- 172.21.15.52:0/2248297017 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f833c0a3ed0 0x7f833c0a6390 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.158+0000 7f835c59d640 1 --2- 172.21.15.52:0/2248297017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8354199c40 0x7f83541a41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.158+0000 7f835c59d640 1 --2- 172.21.15.52:0/2248297017 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8354105dc0 0x7f8354199660 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.158+0000 7f835c59d640 1 --2- 172.21.15.52:0/2248297017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8354101740 0x7f8354199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.158+0000 7f835c59d640 1 -- 172.21.15.52:0/2248297017 >> 172.21.15.52:0/2248297017 conn(0x7f8354075d40 msgr2=0x7f83540fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.158+0000 7f835c59d640 1 -- 172.21.15.52:0/2248297017 shutdown_connections 2024-04-02T13:54:09.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.158+0000 7f835c59d640 1 -- 172.21.15.52:0/2248297017 wait complete. 2024-04-02T13:54:09.170 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:08 smithi052 ceph-mon[32104]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-02T13:54:09.170 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:08 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/389018545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:54:09.172 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:54:09.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.273+0000 7f79ccf50640 1 -- 172.21.15.52:0/2863166345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79c8101af0 msgr2=0x7f79c810d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.273+0000 7f79ccf50640 1 --2- 172.21.15.52:0/2863166345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79c8101af0 0x7f79c810d820 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009ca0 tx=0x7f79b802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.273+0000 7f79ccf50640 1 -- 172.21.15.52:0/2863166345 shutdown_connections 2024-04-02T13:54:09.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.273+0000 7f79ccf50640 1 --2- 172.21.15.52:0/2863166345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79c8101af0 0x7f79c810d820 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.273+0000 7f79ccf50640 1 --2- 172.21.15.52:0/2863166345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79c8101150 0x7f79c81015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.273+0000 7f79ccf50640 1 --2- 172.21.15.52:0/2863166345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79c81059b0 0x7f79c8105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.273+0000 7f79ccf50640 1 -- 172.21.15.52:0/2863166345 >> 172.21.15.52:0/2863166345 conn(0x7f79c8076fc0 msgr2=0x7f79c80fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.273+0000 7f79ccf50640 1 -- 172.21.15.52:0/2863166345 shutdown_connections 2024-04-02T13:54:09.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.273+0000 7f79ccf50640 1 -- 172.21.15.52:0/2863166345 wait complete. 2024-04-02T13:54:09.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.274+0000 7f79ccf50640 1 Processor -- start 2024-04-02T13:54:09.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.274+0000 7f79ccf50640 1 -- start start 2024-04-02T13:54:09.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.274+0000 7f79ccf50640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79c8101150 0x7f79c8198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.274+0000 7f79ccf50640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79c8101af0 0x7f79c8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.274+0000 7f79ccf50640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79c81059b0 0x7f79c819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.274+0000 7f79ccf50640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f79c8111fa0 con 0x7f79c8101af0 2024-04-02T13:54:09.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.274+0000 7f79ccf50640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f79c8111e20 con 0x7f79c81059b0 2024-04-02T13:54:09.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.274+0000 7f79ccf50640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f79c8112120 con 0x7f79c8101150 2024-04-02T13:54:09.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.274+0000 7f79c5d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79c8101af0 0x7f79c8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.274+0000 7f79c5d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79c8101af0 0x7f79c8198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33254/0 (socket says 172.21.15.52:33254) 2024-04-02T13:54:09.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.274+0000 7f79c5d9b640 1 -- 172.21.15.52:0/71352618 learned_addr learned my addr 172.21.15.52:0/71352618 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:09.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.274+0000 7f79c6d9d640 1 --2- 172.21.15.52:0/71352618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79c81059b0 0x7f79c819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.275+0000 7f79c659c640 1 --2- 172.21.15.52:0/71352618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79c8101150 0x7f79c8198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.275+0000 7f79c5d9b640 1 -- 172.21.15.52:0/71352618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79c8101150 msgr2=0x7f79c8198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.275+0000 7f79c5d9b640 1 --2- 172.21.15.52:0/71352618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79c8101150 0x7f79c8198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.275+0000 7f79c5d9b640 1 -- 172.21.15.52:0/71352618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79c81059b0 msgr2=0x7f79c819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.275+0000 7f79c5d9b640 1 --2- 172.21.15.52:0/71352618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79c81059b0 0x7f79c819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.275+0000 7f79c5d9b640 1 -- 172.21.15.52:0/71352618 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c819d3f0 con 0x7f79c8101af0 2024-04-02T13:54:09.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.275+0000 7f79c5d9b640 1 --2- 172.21.15.52:0/71352618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79c8101af0 0x7f79c8198940 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00e9f0 tx=0x7f79bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.275+0000 7f79b77fe640 1 -- 172.21.15.52:0/71352618 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc00cd80 con 0x7f79c8101af0 2024-04-02T13:54:09.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.275+0000 7f79ccf50640 1 -- 172.21.15.52:0/71352618 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c819d6e0 con 0x7f79c8101af0 2024-04-02T13:54:09.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.275+0000 7f79b77fe640 1 -- 172.21.15.52:0/71352618 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79bc010910 con 0x7f79c8101af0 2024-04-02T13:54:09.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.275+0000 7f79b77fe640 1 -- 172.21.15.52:0/71352618 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc0188f0 con 0x7f79c8101af0 2024-04-02T13:54:09.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.275+0000 7f79ccf50640 1 -- 172.21.15.52:0/71352618 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c81a4f70 con 0x7f79c8101af0 2024-04-02T13:54:09.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:08 smithi189 ceph-mon[36683]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-02T13:54:09.289 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:08 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/389018545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:54:09.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.278+0000 7f79b77fe640 1 -- 172.21.15.52:0/71352618 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f79bc010ac0 con 0x7f79c8101af0 2024-04-02T13:54:09.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.278+0000 7f79cfc0f740 1 Processor -- start 2024-04-02T13:54:09.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.278+0000 7f79cfc0f740 1 -- start start 2024-04-02T13:54:09.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.278+0000 7f79cfc0f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557b21de97a0 0x557b21de9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.278+0000 7f79cfc0f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b21dea0c0 0x557b21df2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.278+0000 7f79cfc0f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21df2ca0 0x557b21df5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.278+0000 7f79cfc0f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557b21d4d7b0 con 0x557b21df2ca0 2024-04-02T13:54:09.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.278+0000 7f79cfc0f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557b21cec570 con 0x557b21de97a0 2024-04-02T13:54:09.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.278+0000 7f79cfc0f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557b21d49760 con 0x557b21dea0c0 2024-04-02T13:54:09.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.278+0000 7f79b77fe640 1 --2- 172.21.15.52:0/71352618 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f79b00a3ed0 0x7f79b00a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.279+0000 7f79c5d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21df2ca0 0x557b21df5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.279+0000 7f79c5d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21df2ca0 0x557b21df5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33266/0 (socket says 172.21.15.52:33266) 2024-04-02T13:54:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.279+0000 7f79c5d9b640 1 -- 172.21.15.52:0/293916374 learned_addr learned my addr 172.21.15.52:0/293916374 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.279+0000 7f79c659c640 1 --2- 172.21.15.52:0/293916374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b21dea0c0 0x557b21df2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.279+0000 7f79c6d9d640 1 --2- 172.21.15.52:0/293916374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557b21de97a0 0x557b21de9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.279+0000 7f79c6d9d640 1 --2- 172.21.15.52:0/71352618 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f79b00a3ed0 0x7f79b00a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.279+0000 7f79b77fe640 1 -- 172.21.15.52:0/71352618 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f79bc0d1260 con 0x7f79c8101af0 2024-04-02T13:54:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.279+0000 7f79c5d9b640 1 -- 172.21.15.52:0/293916374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b21dea0c0 msgr2=0x557b21df2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.279+0000 7f79c5d9b640 1 --2- 172.21.15.52:0/293916374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b21dea0c0 0x557b21df2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.279+0000 7f79c5d9b640 1 -- 172.21.15.52:0/293916374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557b21de97a0 msgr2=0x557b21de9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.279+0000 7f79c5d9b640 1 --2- 172.21.15.52:0/293916374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557b21de97a0 0x557b21de9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.279+0000 7f79c5d9b640 1 -- 172.21.15.52:0/293916374 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b21a613f0 con 0x557b21df2ca0 2024-04-02T13:54:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.279+0000 7f79c659c640 1 --2- 172.21.15.52:0/293916374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b21dea0c0 0x557b21df2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:54:09.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.279+0000 7f79c6d9d640 1 --2- 172.21.15.52:0/71352618 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f79b00a3ed0 0x7f79b00a6390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f79b8004af0 tx=0x7f79b8005c50 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.279+0000 7f79c5d9b640 1 --2- 172.21.15.52:0/293916374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21df2ca0 0x557b21df5090 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f79bc099650 tx=0x7f79bc009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acca602401b4f4d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.279+0000 7f79b4ff9640 1 -- 172.21.15.52:0/293916374 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc09cc80 con 0x557b21df2ca0 2024-04-02T13:54:09.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.279+0000 7f79b4ff9640 1 -- 172.21.15.52:0/293916374 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79bc0a1030 con 0x557b21df2ca0 2024-04-02T13:54:09.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.279+0000 7f79b4ff9640 1 -- 172.21.15.52:0/293916374 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc0bd0d0 con 0x557b21df2ca0 2024-04-02T13:54:09.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.280+0000 7f79cfc0f740 1 -- 172.21.15.52:0/293916374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21df2ca0 msgr2=0x557b21df5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.280+0000 7f79cfc0f740 1 --2- 172.21.15.52:0/293916374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21df2ca0 0x557b21df5090 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f79bc099650 tx=0x7f79bc009770 comp rx=0 tx=0).stop 2024-04-02T13:54:09.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.280+0000 7f79cfc0f740 1 -- 172.21.15.52:0/293916374 shutdown_connections 2024-04-02T13:54:09.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.280+0000 7f79cfc0f740 1 --2- 172.21.15.52:0/293916374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21df2ca0 0x557b21df5090 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.280+0000 7f79cfc0f740 1 --2- 172.21.15.52:0/293916374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b21dea0c0 0x557b21df2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.280+0000 7f79cfc0f740 1 --2- 172.21.15.52:0/293916374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557b21de97a0 0x557b21de9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.280+0000 7f79cfc0f740 1 -- 172.21.15.52:0/293916374 >> 172.21.15.52:0/293916374 conn(0x557b21de3ed0 msgr2=0x557b21de42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.280+0000 7f79cfc0f740 1 -- 172.21.15.52:0/293916374 shutdown_connections 2024-04-02T13:54:09.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.280+0000 7f79cfc0f740 1 -- 172.21.15.52:0/293916374 wait complete. 2024-04-02T13:54:09.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.280+0000 7f79cfc0f740 1 -- start start 2024-04-02T13:54:09.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.281+0000 7f79cfc0f740 1 Processor -- start 2024-04-02T13:54:09.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.281+0000 7f79cfc0f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b21de97a0 0x557b21df26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.281+0000 7f79cfc0f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557b21dea0c0 0x557b21defa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.281+0000 7f79cfc0f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21df2ca0 0x557b21df0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.281+0000 7f79cfc0f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557b21df59d0 con 0x557b21df2ca0 2024-04-02T13:54:09.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.281+0000 7f79cfc0f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557b21d08e80 con 0x557b21dea0c0 2024-04-02T13:54:09.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.281+0000 7f79cfc0f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557b21df5b50 con 0x557b21de97a0 2024-04-02T13:54:09.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.281+0000 7f79cfc0f740 10 client.0 _ll_register_callbacks cb 0x557b21d44aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:09.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.281+0000 7f79c6d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21df2ca0 0x557b21df0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.281+0000 7f79c6d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21df2ca0 0x557b21df0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33278/0 (socket says 172.21.15.52:33278) 2024-04-02T13:54:09.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.281+0000 7f79c6d9d640 1 -- 172.21.15.52:0/4072039974 learned_addr learned my addr 172.21.15.52:0/4072039974 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:09.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.281+0000 7f79c5d9b640 1 --2- 172.21.15.52:0/4072039974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557b21dea0c0 0x557b21defa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.281+0000 7f79c659c640 1 --2- 172.21.15.52:0/4072039974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b21de97a0 0x557b21df26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.281+0000 7f79c6d9d640 1 -- 172.21.15.52:0/4072039974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b21de97a0 msgr2=0x557b21df26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.281+0000 7f79c6d9d640 1 --2- 172.21.15.52:0/4072039974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b21de97a0 0x557b21df26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.281+0000 7f79c6d9d640 1 -- 172.21.15.52:0/4072039974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557b21dea0c0 msgr2=0x557b21defa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.281+0000 7f79c6d9d640 1 --2- 172.21.15.52:0/4072039974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557b21dea0c0 0x557b21defa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.281+0000 7f79c6d9d640 1 -- 172.21.15.52:0/4072039974 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b21b2b7d0 con 0x557b21df2ca0 2024-04-02T13:54:09.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.281+0000 7f79c6d9d640 1 --2- 172.21.15.52:0/4072039974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21df2ca0 0x557b21df0040 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f79b8036000 tx=0x7f79b80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7b07cdce94d2a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.281+0000 7f79adffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:09.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.281+0000 7f79adffb640 1 -- 172.21.15.52:0/4072039974 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8039040 con 0x557b21df2ca0 2024-04-02T13:54:09.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.282+0000 7f79adffb640 1 -- 172.21.15.52:0/4072039974 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f79b8003d80 con 0x557b21df2ca0 2024-04-02T13:54:09.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.282+0000 7f79cfc0f740 1 -- 172.21.15.52:0/4072039974 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79bc0a1030 con 0x557b21df2ca0 2024-04-02T13:54:09.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.282+0000 7f79adffb640 1 -- 172.21.15.52:0/4072039974 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f79b805b050 con 0x557b21df2ca0 2024-04-02T13:54:09.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.282+0000 7f79cfc0f740 10 client.15864 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:09.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.282+0000 7f79cfc0f740 1 -- 172.21.15.52:0/4072039974 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557b21e01a40 con 0x557b21df2ca0 2024-04-02T13:54:09.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.282+0000 7f79adffb640 1 -- 172.21.15.52:0/4072039974 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79bc0a1030 con 0x557b21df2ca0 2024-04-02T13:54:09.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.282+0000 7f79adffb640 1 -- 172.21.15.52:0/4072039974 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f79b80428f0 con 0x557b21df2ca0 2024-04-02T13:54:09.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.282+0000 7f79cfc0f740 10 client.15864 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:09.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.282+0000 7f79cfc0f740 10 client.15864 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T13:54:09.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.282+0000 7f79cfc0f740 20 client.15864 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:09.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.282+0000 7f79cfc0f740 1 --2- 172.21.15.52:0/4072039974 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x557b21e02f50 0x557b21e053c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.282+0000 7f79cfc0f740 4 client.15864 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:54:09.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.282+0000 7f79cfc0f740 1 -- 172.21.15.52:0/4072039974 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b21d5acf0 con 0x557b21e02f50 2024-04-02T13:54:09.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.283+0000 7f79c659c640 1 --2- 172.21.15.52:0/4072039974 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x557b21e02f50 0x557b21e053c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.283+0000 7f79adffb640 10 client.15864 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:54:09.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.283+0000 7f79c659c640 1 --2- 172.21.15.52:0/4072039974 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x557b21e02f50 0x557b21e053c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b98aaf5e19d6d854 server_cookie=92d5717faac96f83 in_seq=0 out_seq=0 2024-04-02T13:54:09.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.284+0000 7f79adffb640 1 -- 172.21.15.52:0/4072039974 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557b21d5acf0 con 0x557b21e02f50 2024-04-02T13:54:09.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.284+0000 7f79adffb640 10 client.15864 handle_command_reply: tid=0 2024-04-02T13:54:09.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.284+0000 7f79cfc0f740 1 client.15864 shutdown 2024-04-02T13:54:09.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.284+0000 7f79cfc0f740 1 -- 172.21.15.52:0/4072039974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21df2ca0 msgr2=0x557b21df0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.284+0000 7f79cfc0f740 1 --2- 172.21.15.52:0/4072039974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21df2ca0 0x557b21df0040 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f79b8036000 tx=0x7f79b80095c0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.284+0000 7f79cfc0f740 1 -- 172.21.15.52:0/4072039974 shutdown_connections 2024-04-02T13:54:09.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.284+0000 7f79cfc0f740 1 --2- 172.21.15.52:0/4072039974 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x557b21e02f50 0x557b21e053c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.284+0000 7f79cfc0f740 1 --2- 172.21.15.52:0/4072039974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21df2ca0 0x557b21df0040 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.284+0000 7f79cfc0f740 1 --2- 172.21.15.52:0/4072039974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557b21dea0c0 0x557b21defa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.284+0000 7f79adffb640 0 client.15864 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:54:09.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.284+0000 7f79cfc0f740 1 --2- 172.21.15.52:0/4072039974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b21de97a0 0x557b21df26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.284+0000 7f79cfc0f740 1 -- 172.21.15.52:0/4072039974 >> 172.21.15.52:0/4072039974 conn(0x557b21de3ed0 msgr2=0x557b21de5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.284+0000 7f79cfc0f740 1 -- 172.21.15.52:0/4072039974 shutdown_connections 2024-04-02T13:54:09.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.284+0000 7f79cfc0f740 1 -- 172.21.15.52:0/4072039974 wait complete. 2024-04-02T13:54:09.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.284+0000 7f79cfc0f740 20 client.15864 trim_cache size 0 max 16384 2024-04-02T13:54:09.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.297+0000 7f79cfc0f740 1 Processor -- start 2024-04-02T13:54:09.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.298+0000 7f79cfc0f740 1 -- start start 2024-04-02T13:54:09.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.298+0000 7f79cfc0f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557b21de97a0 0x557b21df2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.298+0000 7f79cfc0f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21dea0c0 0x557b21df26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.298+0000 7f79cfc0f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b21df2ca0 0x557b21e0ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.298+0000 7f79cfc0f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f79c81aee10 con 0x557b21dea0c0 2024-04-02T13:54:09.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.298+0000 7f79cfc0f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f79c81b0950 con 0x557b21de97a0 2024-04-02T13:54:09.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.298+0000 7f79cfc0f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f79c81af4e0 con 0x557b21df2ca0 2024-04-02T13:54:09.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.298+0000 7f79c5d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21dea0c0 0x557b21df26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.298+0000 7f79c5d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21dea0c0 0x557b21df26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33288/0 (socket says 172.21.15.52:33288) 2024-04-02T13:54:09.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.298+0000 7f79c5d9b640 1 -- 172.21.15.52:0/73259432 learned_addr learned my addr 172.21.15.52:0/73259432 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:09.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.298+0000 7f79c6d9d640 1 --2- 172.21.15.52:0/73259432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b21df2ca0 0x557b21e0ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.298+0000 7f79c659c640 1 --2- 172.21.15.52:0/73259432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557b21de97a0 0x557b21df2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.298+0000 7f79c5d9b640 1 -- 172.21.15.52:0/73259432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b21df2ca0 msgr2=0x557b21e0ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.298+0000 7f79c5d9b640 1 --2- 172.21.15.52:0/73259432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b21df2ca0 0x557b21e0ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.298+0000 7f79c5d9b640 1 -- 172.21.15.52:0/73259432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557b21de97a0 msgr2=0x557b21df2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.298+0000 7f79c5d9b640 1 --2- 172.21.15.52:0/73259432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557b21de97a0 0x557b21df2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.298+0000 7f79c5d9b640 1 -- 172.21.15.52:0/73259432 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b21de8890 con 0x557b21dea0c0 2024-04-02T13:54:09.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.298+0000 7f79c659c640 1 --2- 172.21.15.52:0/73259432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557b21de97a0 0x557b21df2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:54:09.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.298+0000 7f79c6d9d640 1 --2- 172.21.15.52:0/73259432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b21df2ca0 0x557b21e0ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:54:09.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.298+0000 7f79c5d9b640 1 --2- 172.21.15.52:0/73259432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21dea0c0 0x557b21df26a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0990e0 tx=0x7f79bc0a76b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ca19525563d77 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.298+0000 7f79affff640 1 -- 172.21.15.52:0/73259432 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc0a1030 con 0x557b21dea0c0 2024-04-02T13:54:09.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.298+0000 7f79affff640 1 -- 172.21.15.52:0/73259432 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79bc09cc80 con 0x557b21dea0c0 2024-04-02T13:54:09.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.298+0000 7f79affff640 1 -- 172.21.15.52:0/73259432 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc0bd070 con 0x557b21dea0c0 2024-04-02T13:54:09.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.299+0000 7f79cfc0f740 1 -- 172.21.15.52:0/73259432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21dea0c0 msgr2=0x557b21df26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.299+0000 7f79cfc0f740 1 --2- 172.21.15.52:0/73259432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21dea0c0 0x557b21df26a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0990e0 tx=0x7f79bc0a76b0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.299+0000 7f79cfc0f740 1 -- 172.21.15.52:0/73259432 shutdown_connections 2024-04-02T13:54:09.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.299+0000 7f79cfc0f740 1 --2- 172.21.15.52:0/73259432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b21df2ca0 0x557b21e0ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.299+0000 7f79cfc0f740 1 --2- 172.21.15.52:0/73259432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21dea0c0 0x557b21df26a0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.299+0000 7f79cfc0f740 1 --2- 172.21.15.52:0/73259432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557b21de97a0 0x557b21df2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.299+0000 7f79cfc0f740 1 -- 172.21.15.52:0/73259432 >> 172.21.15.52:0/73259432 conn(0x557b21de3ed0 msgr2=0x557b21de26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.299+0000 7f79cfc0f740 1 -- 172.21.15.52:0/73259432 shutdown_connections 2024-04-02T13:54:09.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.299+0000 7f79cfc0f740 1 -- 172.21.15.52:0/73259432 wait complete. 2024-04-02T13:54:09.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.299+0000 7f79cfc0f740 1 -- start start 2024-04-02T13:54:09.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.299+0000 7f79cfc0f740 1 Processor -- start 2024-04-02T13:54:09.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.300+0000 7f79cfc0f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557b21de97a0 0x557b21df3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.300+0000 7f79cfc0f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21dea0c0 0x557b21df4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.300+0000 7f79cfc0f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b21df2ca0 0x557b21df93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.300+0000 7f79cfc0f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f79c81b0280 con 0x557b21dea0c0 2024-04-02T13:54:09.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.300+0000 7f79cfc0f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f79c81afbb0 con 0x557b21de97a0 2024-04-02T13:54:09.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.300+0000 7f79cfc0f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f79c81acdf0 con 0x557b21df2ca0 2024-04-02T13:54:09.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.300+0000 7f79cfc0f740 10 client.0 _ll_register_callbacks cb 0x7f79c0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:09.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.300+0000 7f79c5d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21dea0c0 0x557b21df4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.300+0000 7f79c5d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21dea0c0 0x557b21df4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33304/0 (socket says 172.21.15.52:33304) 2024-04-02T13:54:09.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.300+0000 7f79c5d9b640 1 -- 172.21.15.52:0/3582555561 learned_addr learned my addr 172.21.15.52:0/3582555561 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:09.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.300+0000 7f79c659c640 1 --2- 172.21.15.52:0/3582555561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557b21de97a0 0x557b21df3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.300+0000 7f79c5d9b640 1 -- 172.21.15.52:0/3582555561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b21df2ca0 msgr2=0x557b21df93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.300+0000 7f79c5d9b640 1 --2- 172.21.15.52:0/3582555561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b21df2ca0 0x557b21df93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.300+0000 7f79c5d9b640 1 -- 172.21.15.52:0/3582555561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557b21de97a0 msgr2=0x557b21df3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.300+0000 7f79c5d9b640 1 --2- 172.21.15.52:0/3582555561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557b21de97a0 0x557b21df3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.300+0000 7f79c5d9b640 1 -- 172.21.15.52:0/3582555561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b21df9920 con 0x557b21dea0c0 2024-04-02T13:54:09.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.300+0000 7f79c5d9b640 1 --2- 172.21.15.52:0/3582555561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21dea0c0 0x557b21df4400 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f79bc09af30 tx=0x7f79bc009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9fb6a2e0a082b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.300+0000 7f79adffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:09.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.300+0000 7f79adffb640 1 -- 172.21.15.52:0/3582555561 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc0a1030 con 0x557b21dea0c0 2024-04-02T13:54:09.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.300+0000 7f79adffb640 1 -- 172.21.15.52:0/3582555561 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f79bc0a70b0 con 0x557b21dea0c0 2024-04-02T13:54:09.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.300+0000 7f79cfc0f740 1 -- 172.21.15.52:0/3582555561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79bc09cc80 con 0x557b21dea0c0 2024-04-02T13:54:09.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.300+0000 7f79adffb640 1 -- 172.21.15.52:0/3582555561 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f79bc0a58f0 con 0x557b21dea0c0 2024-04-02T13:54:09.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.300+0000 7f79cfc0f740 10 client.15870 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:09.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.300+0000 7f79cfc0f740 1 -- 172.21.15.52:0/3582555561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557b21e69c10 con 0x557b21dea0c0 2024-04-02T13:54:09.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.300+0000 7f79adffb640 1 -- 172.21.15.52:0/3582555561 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79bc09cc80 con 0x557b21dea0c0 2024-04-02T13:54:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.301+0000 7f79adffb640 1 -- 172.21.15.52:0/3582555561 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f79bc09c0d0 con 0x557b21dea0c0 2024-04-02T13:54:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.301+0000 7f79cfc0f740 10 client.15870 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.301+0000 7f79cfc0f740 10 client.15870 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T13:54:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.301+0000 7f79cfc0f740 20 client.15870 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.301+0000 7f79cfc0f740 1 --2- 172.21.15.52:0/3582555561 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x557b21e69e90 0x557b21e6c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.301+0000 7f79cfc0f740 4 client.15870 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:54:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.301+0000 7f79cfc0f740 1 -- 172.21.15.52:0/3582555561 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b21d55040 con 0x557b21e69e90 2024-04-02T13:54:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.301+0000 7f79c659c640 1 --2- 172.21.15.52:0/3582555561 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x557b21e69e90 0x557b21e6c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.301+0000 7f79c659c640 1 --2- 172.21.15.52:0/3582555561 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x557b21e69e90 0x557b21e6c280 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a80b7c6474178c8 server_cookie=38d5b60a49eebc22 in_seq=0 out_seq=0 2024-04-02T13:54:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.301+0000 7f79adffb640 10 client.15870 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:54:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79adffb640 1 -- 172.21.15.52:0/3582555561 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b21d55040 con 0x557b21e69e90 2024-04-02T13:54:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79adffb640 10 client.15870 handle_command_reply: tid=0 2024-04-02T13:54:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79cfc0f740 1 client.15870 shutdown 2024-04-02T13:54:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79cfc0f740 1 -- 172.21.15.52:0/3582555561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21dea0c0 msgr2=0x557b21df4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79cfc0f740 1 --2- 172.21.15.52:0/3582555561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21dea0c0 0x557b21df4400 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f79bc09af30 tx=0x7f79bc009770 comp rx=0 tx=0).stop 2024-04-02T13:54:09.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79cfc0f740 1 -- 172.21.15.52:0/3582555561 shutdown_connections 2024-04-02T13:54:09.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79cfc0f740 1 --2- 172.21.15.52:0/3582555561 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x557b21e69e90 0x557b21e6c280 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79cfc0f740 1 --2- 172.21.15.52:0/3582555561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b21df2ca0 0x557b21df93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79adffb640 0 client.15870 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:54:09.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79cfc0f740 1 --2- 172.21.15.52:0/3582555561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b21dea0c0 0x557b21df4400 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79cfc0f740 1 --2- 172.21.15.52:0/3582555561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557b21de97a0 0x557b21df3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79cfc0f740 1 -- 172.21.15.52:0/3582555561 >> 172.21.15.52:0/3582555561 conn(0x557b21de3ed0 msgr2=0x557b21de2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79cfc0f740 1 -- 172.21.15.52:0/3582555561 shutdown_connections 2024-04-02T13:54:09.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79cfc0f740 1 -- 172.21.15.52:0/3582555561 wait complete. 2024-04-02T13:54:09.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79cfc0f740 20 client.15870 trim_cache size 0 max 16384 2024-04-02T13:54:09.313 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:54:09.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79ccf50640 1 -- 172.21.15.52:0/71352618 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f79b00a3ed0 msgr2=0x7f79b00a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79ccf50640 1 --2- 172.21.15.52:0/71352618 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f79b00a3ed0 0x7f79b00a6390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f79b8004af0 tx=0x7f79b8005c50 comp rx=0 tx=0).stop 2024-04-02T13:54:09.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79ccf50640 1 -- 172.21.15.52:0/71352618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79c8101af0 msgr2=0x7f79c8198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79ccf50640 1 --2- 172.21.15.52:0/71352618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79c8101af0 0x7f79c8198940 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00e9f0 tx=0x7f79bc00eeb0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79ccf50640 1 -- 172.21.15.52:0/71352618 shutdown_connections 2024-04-02T13:54:09.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79ccf50640 1 --2- 172.21.15.52:0/71352618 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f79b00a3ed0 0x7f79b00a6390 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79ccf50640 1 --2- 172.21.15.52:0/71352618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79c81059b0 0x7f79c819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79ccf50640 1 --2- 172.21.15.52:0/71352618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79c8101af0 0x7f79c8198940 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79ccf50640 1 --2- 172.21.15.52:0/71352618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79c8101150 0x7f79c8198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79ccf50640 1 -- 172.21.15.52:0/71352618 >> 172.21.15.52:0/71352618 conn(0x7f79c8076fc0 msgr2=0x7f79c8103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.302+0000 7f79ccf50640 1 -- 172.21.15.52:0/71352618 shutdown_connections 2024-04-02T13:54:09.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.303+0000 7f79ccf50640 1 -- 172.21.15.52:0/71352618 wait complete. 2024-04-02T13:54:09.316 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:54:09.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:08 smithi088 ceph-mon[36528]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-02T13:54:09.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:08 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/389018545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:54:09.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.419+0000 7f9094115640 1 -- 172.21.15.52:0/1148617065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f908c106980 msgr2=0x7f908c106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.419+0000 7f9094115640 1 --2- 172.21.15.52:0/1148617065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f908c106980 0x7f908c106d60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9088009930 tx=0x7f908802bc30 comp rx=0 tx=0).stop 2024-04-02T13:54:09.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.419+0000 7f9094115640 1 -- 172.21.15.52:0/1148617065 shutdown_connections 2024-04-02T13:54:09.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.419+0000 7f9094115640 1 --2- 172.21.15.52:0/1148617065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f908c101330 0x7f908c10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.419+0000 7f9094115640 1 --2- 172.21.15.52:0/1148617065 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f908c100970 0x7f908c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.419+0000 7f9094115640 1 --2- 172.21.15.52:0/1148617065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f908c106980 0x7f908c106d60 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.419+0000 7f9094115640 1 -- 172.21.15.52:0/1148617065 >> 172.21.15.52:0/1148617065 conn(0x7f908c077b00 msgr2=0x7f908c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.419+0000 7f9094115640 1 -- 172.21.15.52:0/1148617065 shutdown_connections 2024-04-02T13:54:09.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.419+0000 7f9094115640 1 -- 172.21.15.52:0/1148617065 wait complete. 2024-04-02T13:54:09.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.419+0000 7f9094115640 1 Processor -- start 2024-04-02T13:54:09.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.419+0000 7f9094115640 1 -- start start 2024-04-02T13:54:09.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.420+0000 7f9094115640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f908c100970 0x7f908c1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.420+0000 7f9094115640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f908c101330 0x7f908c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.420+0000 7f9094115640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f908c106980 0x7f908c1a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.420+0000 7f9094115640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f908c110d90 con 0x7f908c106980 2024-04-02T13:54:09.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.420+0000 7f9094115640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f908c110c10 con 0x7f908c101330 2024-04-02T13:54:09.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.420+0000 7f9094115640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f908c110f10 con 0x7f908c100970 2024-04-02T13:54:09.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.420+0000 7f90926b2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f908c106980 0x7f908c1a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.420+0000 7f90926b2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f908c106980 0x7f908c1a4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33322/0 (socket says 172.21.15.52:33322) 2024-04-02T13:54:09.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.420+0000 7f90926b2640 1 -- 172.21.15.52:0/2028097508 learned_addr learned my addr 172.21.15.52:0/2028097508 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:09.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.420+0000 7f9091eb1640 1 --2- 172.21.15.52:0/2028097508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f908c100970 0x7f908c1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.420+0000 7f90916b0640 1 --2- 172.21.15.52:0/2028097508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f908c101330 0x7f908c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.420+0000 7f90926b2640 1 -- 172.21.15.52:0/2028097508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f908c100970 msgr2=0x7f908c1991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.420+0000 7f90926b2640 1 --2- 172.21.15.52:0/2028097508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f908c100970 0x7f908c1991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.420+0000 7f90926b2640 1 -- 172.21.15.52:0/2028097508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f908c101330 msgr2=0x7f908c199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.420+0000 7f90926b2640 1 --2- 172.21.15.52:0/2028097508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f908c101330 0x7f908c199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.420+0000 7f90926b2640 1 -- 172.21.15.52:0/2028097508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908c1a47b0 con 0x7f908c106980 2024-04-02T13:54:09.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.420+0000 7f90926b2640 1 --2- 172.21.15.52:0/2028097508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f908c106980 0x7f908c1a4270 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f908400eb10 tx=0x7f908400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.420+0000 7f9082ffd640 1 -- 172.21.15.52:0/2028097508 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9084004b20 con 0x7f908c106980 2024-04-02T13:54:09.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.420+0000 7f9082ffd640 1 -- 172.21.15.52:0/2028097508 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9084004cc0 con 0x7f908c106980 2024-04-02T13:54:09.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.420+0000 7f9094115640 1 -- 172.21.15.52:0/2028097508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f908c1a4980 con 0x7f908c106980 2024-04-02T13:54:09.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.420+0000 7f9082ffd640 1 -- 172.21.15.52:0/2028097508 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9084010760 con 0x7f908c106980 2024-04-02T13:54:09.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.420+0000 7f9094115640 1 -- 172.21.15.52:0/2028097508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f908c1a4e80 con 0x7f908c106980 2024-04-02T13:54:09.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.423+0000 7f9082ffd640 1 -- 172.21.15.52:0/2028097508 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f9084005020 con 0x7f908c106980 2024-04-02T13:54:09.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.423+0000 7f9096ddb740 1 Processor -- start 2024-04-02T13:54:09.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.423+0000 7f9096ddb740 1 -- start start 2024-04-02T13:54:09.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.423+0000 7f9096ddb740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55620bb177a0 0x55620bb17b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.423+0000 7f9096ddb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55620bb180c0 0x55620bb20760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.423+0000 7f9096ddb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb20ca0 0x55620bb23090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.423+0000 7f9096ddb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55620ba7b7b0 con 0x55620bb20ca0 2024-04-02T13:54:09.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.423+0000 7f9096ddb740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55620ba1a570 con 0x55620bb177a0 2024-04-02T13:54:09.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.423+0000 7f9096ddb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55620ba77760 con 0x55620bb180c0 2024-04-02T13:54:09.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.423+0000 7f9082ffd640 1 --2- 172.21.15.52:0/2028097508 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f90780a3ed0 0x7f90780a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.423+0000 7f90916b0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb20ca0 0x55620bb23090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.424+0000 7f90926b2640 1 --2- 172.21.15.52:0/2028097508 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f90780a3ed0 0x7f90780a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.424+0000 7f90916b0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb20ca0 0x55620bb23090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33326/0 (socket says 172.21.15.52:33326) 2024-04-02T13:54:09.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.424+0000 7f90916b0640 1 -- 172.21.15.52:0/1948251000 learned_addr learned my addr 172.21.15.52:0/1948251000 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:09.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.424+0000 7f90916b0640 1 --2- 172.21.15.52:0/1948251000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55620bb177a0 0x55620bb17b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.424+0000 7f9091eb1640 1 --2- 172.21.15.52:0/1948251000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55620bb180c0 0x55620bb20760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.424+0000 7f9082ffd640 1 -- 172.21.15.52:0/2028097508 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f90840d1320 con 0x7f908c106980 2024-04-02T13:54:09.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.424+0000 7f90916b0640 1 -- 172.21.15.52:0/1948251000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55620bb180c0 msgr2=0x55620bb20760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.424+0000 7f90916b0640 1 --2- 172.21.15.52:0/1948251000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55620bb180c0 0x55620bb20760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.424+0000 7f90916b0640 1 -- 172.21.15.52:0/1948251000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55620bb177a0 msgr2=0x55620bb17b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.424+0000 7f90916b0640 1 --2- 172.21.15.52:0/1948251000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55620bb177a0 0x55620bb17b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.424+0000 7f90916b0640 1 -- 172.21.15.52:0/1948251000 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55620b78f3f0 con 0x55620bb20ca0 2024-04-02T13:54:09.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.424+0000 7f9091eb1640 1 --2- 172.21.15.52:0/1948251000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55620bb180c0 0x55620bb20760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:54:09.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.424+0000 7f90916b0640 1 --2- 172.21.15.52:0/1948251000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb20ca0 0x55620bb23090 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f907c0127a0 tx=0x7f907c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964813fed313aa68 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.424+0000 7f90926b2640 1 --2- 172.21.15.52:0/2028097508 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f90780a3ed0 0x7f90780a6390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f90840ce550 tx=0x7f908406f830 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.424+0000 7f9077fff640 1 -- 172.21.15.52:0/1948251000 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c012d20 con 0x55620bb20ca0 2024-04-02T13:54:09.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.424+0000 7f9077fff640 1 -- 172.21.15.52:0/1948251000 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f907c005020 con 0x55620bb20ca0 2024-04-02T13:54:09.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.424+0000 7f9077fff640 1 -- 172.21.15.52:0/1948251000 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c004ae0 con 0x55620bb20ca0 2024-04-02T13:54:09.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.425+0000 7f9096ddb740 1 -- 172.21.15.52:0/1948251000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb20ca0 msgr2=0x55620bb23090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.425+0000 7f9096ddb740 1 --2- 172.21.15.52:0/1948251000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb20ca0 0x55620bb23090 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f907c0127a0 tx=0x7f907c00b7c0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.425+0000 7f9096ddb740 1 -- 172.21.15.52:0/1948251000 shutdown_connections 2024-04-02T13:54:09.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.425+0000 7f9096ddb740 1 --2- 172.21.15.52:0/1948251000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb20ca0 0x55620bb23090 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.425+0000 7f9096ddb740 1 --2- 172.21.15.52:0/1948251000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55620bb180c0 0x55620bb20760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.425+0000 7f9096ddb740 1 --2- 172.21.15.52:0/1948251000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55620bb177a0 0x55620bb17b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.425+0000 7f9096ddb740 1 -- 172.21.15.52:0/1948251000 >> 172.21.15.52:0/1948251000 conn(0x55620bb11ed0 msgr2=0x55620bb122b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.425+0000 7f9096ddb740 1 -- 172.21.15.52:0/1948251000 shutdown_connections 2024-04-02T13:54:09.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.425+0000 7f9096ddb740 1 -- 172.21.15.52:0/1948251000 wait complete. 2024-04-02T13:54:09.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.425+0000 7f9096ddb740 1 -- start start 2024-04-02T13:54:09.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.425+0000 7f9096ddb740 1 Processor -- start 2024-04-02T13:54:09.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.426+0000 7f9096ddb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb177a0 0x55620bb20750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.426+0000 7f9096ddb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55620bb180c0 0x55620bb1dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.426+0000 7f9096ddb740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55620bb20ca0 0x55620bb1e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.426+0000 7f9096ddb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55620bb23a90 con 0x55620bb177a0 2024-04-02T13:54:09.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.426+0000 7f9096ddb740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55620ba36e80 con 0x55620bb20ca0 2024-04-02T13:54:09.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.426+0000 7f9096ddb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55620bb23c10 con 0x55620bb180c0 2024-04-02T13:54:09.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.426+0000 7f9096ddb740 10 client.0 _ll_register_callbacks cb 0x55620ba72aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:09.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.426+0000 7f90916b0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb177a0 0x55620bb20750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.426+0000 7f90916b0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb177a0 0x55620bb20750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33342/0 (socket says 172.21.15.52:33342) 2024-04-02T13:54:09.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.426+0000 7f90916b0640 1 -- 172.21.15.52:0/2754273253 learned_addr learned my addr 172.21.15.52:0/2754273253 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:09.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.426+0000 7f90916b0640 1 --2- 172.21.15.52:0/2754273253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55620bb20ca0 0x55620bb1e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.426+0000 7f9091eb1640 1 --2- 172.21.15.52:0/2754273253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55620bb180c0 0x55620bb1dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.426+0000 7f90916b0640 1 -- 172.21.15.52:0/2754273253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55620bb180c0 msgr2=0x55620bb1dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.426+0000 7f90916b0640 1 --2- 172.21.15.52:0/2754273253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55620bb180c0 0x55620bb1dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.426+0000 7f90916b0640 1 -- 172.21.15.52:0/2754273253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55620bb20ca0 msgr2=0x55620bb1e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.426+0000 7f90916b0640 1 --2- 172.21.15.52:0/2754273253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55620bb20ca0 0x55620bb1e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.426+0000 7f90916b0640 1 -- 172.21.15.52:0/2754273253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55620b8597d0 con 0x55620bb177a0 2024-04-02T13:54:09.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.426+0000 7f9091eb1640 1 --2- 172.21.15.52:0/2754273253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55620bb180c0 0x55620bb1dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:54:09.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.426+0000 7f90916b0640 1 --2- 172.21.15.52:0/2754273253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb177a0 0x55620bb20750 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f907c001eb0 tx=0x7f907c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a850938de2cf0ae3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.426+0000 7f90757fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:09.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.427+0000 7f90757fa640 1 -- 172.21.15.52:0/2754273253 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c004880 con 0x55620bb177a0 2024-04-02T13:54:09.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.427+0000 7f9096ddb740 1 -- 172.21.15.52:0/2754273253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f907c005020 con 0x55620bb177a0 2024-04-02T13:54:09.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.427+0000 7f90757fa640 1 -- 172.21.15.52:0/2754273253 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f907c004a20 con 0x55620bb177a0 2024-04-02T13:54:09.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.427+0000 7f90757fa640 1 -- 172.21.15.52:0/2754273253 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f907c035050 con 0x55620bb177a0 2024-04-02T13:54:09.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.427+0000 7f9096ddb740 10 client.15882 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:09.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.427+0000 7f9096ddb740 1 -- 172.21.15.52:0/2754273253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55620bb2fb40 con 0x55620bb177a0 2024-04-02T13:54:09.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.427+0000 7f90757fa640 1 -- 172.21.15.52:0/2754273253 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f907c005020 con 0x55620bb177a0 2024-04-02T13:54:09.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.427+0000 7f90757fa640 1 -- 172.21.15.52:0/2754273253 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f907c01f420 con 0x55620bb177a0 2024-04-02T13:54:09.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.427+0000 7f9096ddb740 10 client.15882 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:09.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.427+0000 7f9096ddb740 10 client.15882 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:54:09.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.427+0000 7f9096ddb740 20 client.15882 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:09.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.427+0000 7f9096ddb740 1 --2- 172.21.15.52:0/2754273253 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55620bb30f50 0x55620bb33340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.427+0000 7f9096ddb740 4 client.15882 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:54:09.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.427+0000 7f9096ddb740 1 -- 172.21.15.52:0/2754273253 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55620ba88cf0 con 0x55620bb30f50 2024-04-02T13:54:09.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.428+0000 7f9091eb1640 1 --2- 172.21.15.52:0/2754273253 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55620bb30f50 0x55620bb33340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.428+0000 7f90757fa640 10 client.15882 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:54:09.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.428+0000 7f9091eb1640 1 --2- 172.21.15.52:0/2754273253 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55620bb30f50 0x55620bb33340 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=641565da614fadeb server_cookie=cdfa25f20bbd31b9 in_seq=0 out_seq=0 2024-04-02T13:54:09.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.429+0000 7f90757fa640 1 -- 172.21.15.52:0/2754273253 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55620ba88cf0 con 0x55620bb30f50 2024-04-02T13:54:09.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.429+0000 7f90757fa640 10 client.15882 handle_command_reply: tid=0 2024-04-02T13:54:09.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.429+0000 7f9096ddb740 1 client.15882 shutdown 2024-04-02T13:54:09.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.429+0000 7f9096ddb740 1 -- 172.21.15.52:0/2754273253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb177a0 msgr2=0x55620bb20750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.429+0000 7f9096ddb740 1 --2- 172.21.15.52:0/2754273253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb177a0 0x55620bb20750 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f907c001eb0 tx=0x7f907c005c50 comp rx=0 tx=0).stop 2024-04-02T13:54:09.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.429+0000 7f9096ddb740 1 -- 172.21.15.52:0/2754273253 shutdown_connections 2024-04-02T13:54:09.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.429+0000 7f9096ddb740 1 --2- 172.21.15.52:0/2754273253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55620bb20ca0 0x55620bb1e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.429+0000 7f9096ddb740 1 --2- 172.21.15.52:0/2754273253 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55620bb30f50 0x55620bb33340 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.429+0000 7f9096ddb740 1 --2- 172.21.15.52:0/2754273253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55620bb180c0 0x55620bb1dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.429+0000 7f9096ddb740 1 --2- 172.21.15.52:0/2754273253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb177a0 0x55620bb20750 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.429+0000 7f90757fa640 0 client.15882 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:54:09.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.429+0000 7f9096ddb740 1 -- 172.21.15.52:0/2754273253 >> 172.21.15.52:0/2754273253 conn(0x55620bb11ed0 msgr2=0x55620bb13c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.429+0000 7f9096ddb740 1 -- 172.21.15.52:0/2754273253 shutdown_connections 2024-04-02T13:54:09.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.429+0000 7f9096ddb740 1 -- 172.21.15.52:0/2754273253 wait complete. 2024-04-02T13:54:09.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.429+0000 7f9096ddb740 20 client.15882 trim_cache size 0 max 16384 2024-04-02T13:54:09.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.442+0000 7f9096ddb740 1 Processor -- start 2024-04-02T13:54:09.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.442+0000 7f9096ddb740 1 -- start start 2024-04-02T13:54:09.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.443+0000 7f9096ddb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb177a0 0x55620bb2c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.443+0000 7f9096ddb740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55620bb180c0 0x55620bb20210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.443+0000 7f9096ddb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55620bb20ca0 0x55620bb20750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.443+0000 7f9096ddb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f908c1aee90 con 0x55620bb177a0 2024-04-02T13:54:09.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.443+0000 7f9096ddb740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f908c1b09d0 con 0x55620bb180c0 2024-04-02T13:54:09.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.443+0000 7f9096ddb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f908c1af560 con 0x55620bb20ca0 2024-04-02T13:54:09.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.443+0000 7f90916b0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb177a0 0x55620bb2c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.443+0000 7f90916b0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb177a0 0x55620bb2c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33352/0 (socket says 172.21.15.52:33352) 2024-04-02T13:54:09.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.443+0000 7f90916b0640 1 -- 172.21.15.52:0/2555879305 learned_addr learned my addr 172.21.15.52:0/2555879305 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:09.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.443+0000 7f90916b0640 1 -- 172.21.15.52:0/2555879305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55620bb20ca0 msgr2=0x55620bb20750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.443+0000 7f90916b0640 1 --2- 172.21.15.52:0/2555879305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55620bb20ca0 0x55620bb20750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.443+0000 7f90916b0640 1 -- 172.21.15.52:0/2555879305 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55620bb180c0 msgr2=0x55620bb20210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.443+0000 7f90916b0640 1 --2- 172.21.15.52:0/2555879305 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55620bb180c0 0x55620bb20210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.443+0000 7f90916b0640 1 -- 172.21.15.52:0/2555879305 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55620bb1e590 con 0x55620bb177a0 2024-04-02T13:54:09.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.443+0000 7f90916b0640 1 --2- 172.21.15.52:0/2555879305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb177a0 0x55620bb2c950 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f907c00b6b0 tx=0x7f907c01f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee441c7dba822fe server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.443+0000 7f90777fe640 1 -- 172.21.15.52:0/2555879305 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c03e070 con 0x55620bb177a0 2024-04-02T13:54:09.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.443+0000 7f90777fe640 1 -- 172.21.15.52:0/2555879305 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f907c032070 con 0x55620bb177a0 2024-04-02T13:54:09.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.443+0000 7f90777fe640 1 -- 172.21.15.52:0/2555879305 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c01f5e0 con 0x55620bb177a0 2024-04-02T13:54:09.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.444+0000 7f9096ddb740 1 -- 172.21.15.52:0/2555879305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb177a0 msgr2=0x55620bb2c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.444+0000 7f9096ddb740 1 --2- 172.21.15.52:0/2555879305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb177a0 0x55620bb2c950 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f907c00b6b0 tx=0x7f907c01f9f0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.444+0000 7f9096ddb740 1 -- 172.21.15.52:0/2555879305 shutdown_connections 2024-04-02T13:54:09.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.444+0000 7f9096ddb740 1 --2- 172.21.15.52:0/2555879305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55620bb20ca0 0x55620bb20750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.444+0000 7f9096ddb740 1 --2- 172.21.15.52:0/2555879305 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55620bb180c0 0x55620bb20210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.444+0000 7f9096ddb740 1 --2- 172.21.15.52:0/2555879305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb177a0 0x55620bb2c950 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.444+0000 7f9096ddb740 1 -- 172.21.15.52:0/2555879305 >> 172.21.15.52:0/2555879305 conn(0x55620bb11ed0 msgr2=0x55620bb122b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.444+0000 7f9096ddb740 1 -- 172.21.15.52:0/2555879305 shutdown_connections 2024-04-02T13:54:09.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.444+0000 7f9096ddb740 1 -- 172.21.15.52:0/2555879305 wait complete. 2024-04-02T13:54:09.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.444+0000 7f9096ddb740 1 -- start start 2024-04-02T13:54:09.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.444+0000 7f9096ddb740 1 Processor -- start 2024-04-02T13:54:09.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.445+0000 7f9096ddb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55620bb177a0 0x55620bb1b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.445+0000 7f9096ddb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb180c0 0x55620bb19d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.445+0000 7f9096ddb740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55620bb20ca0 0x55620bb1a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.445+0000 7f9096ddb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f908c1b0300 con 0x55620bb180c0 2024-04-02T13:54:09.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.445+0000 7f9096ddb740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f908c1afc30 con 0x55620bb20ca0 2024-04-02T13:54:09.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.445+0000 7f9096ddb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f908c1ace70 con 0x55620bb177a0 2024-04-02T13:54:09.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.445+0000 7f9096ddb740 10 client.0 _ll_register_callbacks cb 0x7f907c00be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:09.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.445+0000 7f9091eb1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb180c0 0x55620bb19d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.445+0000 7f9091eb1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb180c0 0x55620bb19d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33368/0 (socket says 172.21.15.52:33368) 2024-04-02T13:54:09.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.445+0000 7f9091eb1640 1 -- 172.21.15.52:0/1529018742 learned_addr learned my addr 172.21.15.52:0/1529018742 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:09.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.445+0000 7f90916b0640 1 --2- 172.21.15.52:0/1529018742 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55620bb177a0 0x55620bb1b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.445+0000 7f9091eb1640 1 -- 172.21.15.52:0/1529018742 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55620bb177a0 msgr2=0x55620bb1b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.445+0000 7f9091eb1640 1 --2- 172.21.15.52:0/1529018742 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55620bb177a0 0x55620bb1b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.445+0000 7f9091eb1640 1 -- 172.21.15.52:0/1529018742 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55620bb20ca0 msgr2=0x55620bb1a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.445+0000 7f9091eb1640 1 --2- 172.21.15.52:0/1529018742 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55620bb20ca0 0x55620bb1a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.445+0000 7f9091eb1640 1 -- 172.21.15.52:0/1529018742 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55620bb1ab80 con 0x55620bb180c0 2024-04-02T13:54:09.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.445+0000 7f90916b0640 1 --2- 172.21.15.52:0/1529018742 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55620bb177a0 0x55620bb1b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:54:09.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.445+0000 7f9091eb1640 1 --2- 172.21.15.52:0/1529018742 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb180c0 0x55620bb19d50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f908802cfd0 tx=0x7f9088046e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808a6399f8faa643 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.445+0000 7f90757fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:09.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.445+0000 7f90757fa640 1 -- 172.21.15.52:0/1529018742 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908804c060 con 0x55620bb180c0 2024-04-02T13:54:09.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.445+0000 7f90757fa640 1 -- 172.21.15.52:0/1529018742 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f908802cb70 con 0x55620bb180c0 2024-04-02T13:54:09.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.445+0000 7f9096ddb740 1 -- 172.21.15.52:0/1529018742 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f907c032070 con 0x55620bb180c0 2024-04-02T13:54:09.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.445+0000 7f90757fa640 1 -- 172.21.15.52:0/1529018742 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9088053050 con 0x55620bb180c0 2024-04-02T13:54:09.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.445+0000 7f9096ddb740 10 client.15888 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:09.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.446+0000 7f9096ddb740 1 -- 172.21.15.52:0/1529018742 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55620bb28fc0 con 0x55620bb180c0 2024-04-02T13:54:09.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.446+0000 7f90757fa640 1 -- 172.21.15.52:0/1529018742 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f907c032070 con 0x55620bb180c0 2024-04-02T13:54:09.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.446+0000 7f90757fa640 1 -- 172.21.15.52:0/1529018742 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9088004a60 con 0x55620bb180c0 2024-04-02T13:54:09.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.446+0000 7f9096ddb740 10 client.15888 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:09.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.446+0000 7f9096ddb740 10 client.15888 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:54:09.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.446+0000 7f9096ddb740 20 client.15888 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:09.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.446+0000 7f9096ddb740 1 --2- 172.21.15.52:0/1529018742 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55620bb97340 0x55620bb99730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.446+0000 7f9096ddb740 4 client.15888 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:54:09.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.446+0000 7f9096ddb740 1 -- 172.21.15.52:0/1529018742 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55620ba83040 con 0x55620bb97340 2024-04-02T13:54:09.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.446+0000 7f90916b0640 1 --2- 172.21.15.52:0/1529018742 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55620bb97340 0x55620bb99730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.447+0000 7f90757fa640 10 client.15888 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:54:09.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.447+0000 7f90916b0640 1 --2- 172.21.15.52:0/1529018742 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55620bb97340 0x55620bb99730 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6a1a5777a661241f server_cookie=89c0509cbad9bb94 in_seq=0 out_seq=0 2024-04-02T13:54:09.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.447+0000 7f90757fa640 1 -- 172.21.15.52:0/1529018742 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55620ba83040 con 0x55620bb97340 2024-04-02T13:54:09.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.447+0000 7f90757fa640 10 client.15888 handle_command_reply: tid=0 2024-04-02T13:54:09.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.447+0000 7f9096ddb740 1 client.15888 shutdown 2024-04-02T13:54:09.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.447+0000 7f9096ddb740 1 -- 172.21.15.52:0/1529018742 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb180c0 msgr2=0x55620bb19d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.447+0000 7f9096ddb740 1 --2- 172.21.15.52:0/1529018742 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb180c0 0x55620bb19d50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f908802cfd0 tx=0x7f9088046e80 comp rx=0 tx=0).stop 2024-04-02T13:54:09.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.447+0000 7f9096ddb740 1 -- 172.21.15.52:0/1529018742 shutdown_connections 2024-04-02T13:54:09.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.447+0000 7f9096ddb740 1 --2- 172.21.15.52:0/1529018742 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55620bb20ca0 0x55620bb1a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.447+0000 7f9096ddb740 1 --2- 172.21.15.52:0/1529018742 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55620bb180c0 0x55620bb19d50 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.447+0000 7f9096ddb740 1 --2- 172.21.15.52:0/1529018742 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55620bb97340 0x55620bb99730 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.447+0000 7f9096ddb740 1 --2- 172.21.15.52:0/1529018742 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55620bb177a0 0x55620bb1b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.447+0000 7f9096ddb740 1 -- 172.21.15.52:0/1529018742 >> 172.21.15.52:0/1529018742 conn(0x55620bb11ed0 msgr2=0x55620bb106a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.447+0000 7f90757fa640 0 client.15888 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:54:09.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.447+0000 7f9096ddb740 1 -- 172.21.15.52:0/1529018742 shutdown_connections 2024-04-02T13:54:09.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.447+0000 7f9096ddb740 1 -- 172.21.15.52:0/1529018742 wait complete. 2024-04-02T13:54:09.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.447+0000 7f9096ddb740 20 client.15888 trim_cache size 0 max 16384 2024-04-02T13:54:09.457 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:54:09.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.448+0000 7f9094115640 1 -- 172.21.15.52:0/2028097508 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f90780a3ed0 msgr2=0x7f90780a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.448+0000 7f9094115640 1 --2- 172.21.15.52:0/2028097508 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f90780a3ed0 0x7f90780a6390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f90840ce550 tx=0x7f908406f830 comp rx=0 tx=0).stop 2024-04-02T13:54:09.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.448+0000 7f9094115640 1 -- 172.21.15.52:0/2028097508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f908c106980 msgr2=0x7f908c1a4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.448+0000 7f9094115640 1 --2- 172.21.15.52:0/2028097508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f908c106980 0x7f908c1a4270 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f908400eb10 tx=0x7f908400c2d0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.448+0000 7f9094115640 1 -- 172.21.15.52:0/2028097508 shutdown_connections 2024-04-02T13:54:09.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.448+0000 7f9094115640 1 --2- 172.21.15.52:0/2028097508 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f90780a3ed0 0x7f90780a6390 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.448+0000 7f9094115640 1 --2- 172.21.15.52:0/2028097508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f908c106980 0x7f908c1a4270 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.448+0000 7f9094115640 1 --2- 172.21.15.52:0/2028097508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f908c101330 0x7f908c199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.448+0000 7f9094115640 1 --2- 172.21.15.52:0/2028097508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f908c100970 0x7f908c1991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.448+0000 7f9094115640 1 -- 172.21.15.52:0/2028097508 >> 172.21.15.52:0/2028097508 conn(0x7f908c077b00 msgr2=0x7f908c1031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.448+0000 7f9094115640 1 -- 172.21.15.52:0/2028097508 shutdown_connections 2024-04-02T13:54:09.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.448+0000 7f9094115640 1 -- 172.21.15.52:0/2028097508 wait complete. 2024-04-02T13:54:09.462 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:54:09.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.563+0000 7fb876b1c640 1 -- 172.21.15.52:0/2293829648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb870100970 msgr2=0x7fb870100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.563+0000 7fb876b1c640 1 --2- 172.21.15.52:0/2293829648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb870100970 0x7fb870100df0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009930 tx=0x7fb85c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:54:09.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.564+0000 7fb876b1c640 1 -- 172.21.15.52:0/2293829648 shutdown_connections 2024-04-02T13:54:09.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.564+0000 7fb876b1c640 1 --2- 172.21.15.52:0/2293829648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb870101330 0x7fb87010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.564+0000 7fb876b1c640 1 --2- 172.21.15.52:0/2293829648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb870100970 0x7fb870100df0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.564+0000 7fb876b1c640 1 --2- 172.21.15.52:0/2293829648 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb870106980 0x7fb870106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.564+0000 7fb876b1c640 1 -- 172.21.15.52:0/2293829648 >> 172.21.15.52:0/2293829648 conn(0x7fb870077b00 msgr2=0x7fb870077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.564+0000 7fb876b1c640 1 -- 172.21.15.52:0/2293829648 shutdown_connections 2024-04-02T13:54:09.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.564+0000 7fb876b1c640 1 -- 172.21.15.52:0/2293829648 wait complete. 2024-04-02T13:54:09.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.564+0000 7fb876b1c640 1 Processor -- start 2024-04-02T13:54:09.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.564+0000 7fb876b1c640 1 -- start start 2024-04-02T13:54:09.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.564+0000 7fb876b1c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb870100970 0x7fb8701983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.564+0000 7fb876b1c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb870101330 0x7fb8701988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.564+0000 7fb876b1c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb870106980 0x7fb87019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.564+0000 7fb876b1c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb870110d90 con 0x7fb870106980 2024-04-02T13:54:09.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.564+0000 7fb876b1c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb870110c10 con 0x7fb870101330 2024-04-02T13:54:09.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.564+0000 7fb876b1c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb870110f10 con 0x7fb870100970 2024-04-02T13:54:09.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.565+0000 7fb8750b9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb870106980 0x7fb87019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.565+0000 7fb8750b9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb870106980 0x7fb87019cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33376/0 (socket says 172.21.15.52:33376) 2024-04-02T13:54:09.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.565+0000 7fb8750b9640 1 -- 172.21.15.52:0/1463523934 learned_addr learned my addr 172.21.15.52:0/1463523934 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:09.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.565+0000 7fb8748b8640 1 --2- 172.21.15.52:0/1463523934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb870100970 0x7fb8701983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.565+0000 7fb86ffff640 1 --2- 172.21.15.52:0/1463523934 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb870101330 0x7fb8701988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.565+0000 7fb8750b9640 1 -- 172.21.15.52:0/1463523934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb870100970 msgr2=0x7fb8701983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.565+0000 7fb8750b9640 1 --2- 172.21.15.52:0/1463523934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb870100970 0x7fb8701983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.565+0000 7fb8750b9640 1 -- 172.21.15.52:0/1463523934 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb870101330 msgr2=0x7fb8701988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.565+0000 7fb8750b9640 1 --2- 172.21.15.52:0/1463523934 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb870101330 0x7fb8701988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.565+0000 7fb8750b9640 1 -- 172.21.15.52:0/1463523934 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87019d3a0 con 0x7fb870106980 2024-04-02T13:54:09.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.565+0000 7fb8750b9640 1 --2- 172.21.15.52:0/1463523934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb870106980 0x7fb87019cc70 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb86800eb10 tx=0x7fb86800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.565+0000 7fb86dffb640 1 -- 172.21.15.52:0/1463523934 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb868004b20 con 0x7fb870106980 2024-04-02T13:54:09.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.565+0000 7fb876b1c640 1 -- 172.21.15.52:0/1463523934 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb87019d690 con 0x7fb870106980 2024-04-02T13:54:09.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.565+0000 7fb86dffb640 1 -- 172.21.15.52:0/1463523934 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb868004cc0 con 0x7fb870106980 2024-04-02T13:54:09.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.565+0000 7fb86dffb640 1 -- 172.21.15.52:0/1463523934 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb868010760 con 0x7fb870106980 2024-04-02T13:54:09.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.565+0000 7fb876b1c640 1 -- 172.21.15.52:0/1463523934 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8701a4f20 con 0x7fb870106980 2024-04-02T13:54:09.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.567+0000 7fb86dffb640 1 -- 172.21.15.52:0/1463523934 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fb868005020 con 0x7fb870106980 2024-04-02T13:54:09.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.568+0000 7fb8797e8740 1 Processor -- start 2024-04-02T13:54:09.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.568+0000 7fb8797e8740 1 -- start start 2024-04-02T13:54:09.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.568+0000 7fb8797e8740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e107a0 0x55b371e10b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.568+0000 7fb8797e8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b371e110c0 0x55b371e19760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.568+0000 7fb8797e8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b371e19ca0 0x55b371e1c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.568+0000 7fb8797e8740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b371d747b0 con 0x55b371e107a0 2024-04-02T13:54:09.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.568+0000 7fb8797e8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b371d13570 con 0x55b371e110c0 2024-04-02T13:54:09.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.568+0000 7fb8797e8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b371d70760 con 0x55b371e19ca0 2024-04-02T13:54:09.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.568+0000 7fb86dffb640 1 --2- 172.21.15.52:0/1463523934 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb8600a3ed0 0x7fb8600a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.568+0000 7fb86ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e107a0 0x55b371e10b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.568+0000 7fb86ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e107a0 0x55b371e10b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33384/0 (socket says 172.21.15.52:33384) 2024-04-02T13:54:09.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.568+0000 7fb86ffff640 1 -- 172.21.15.52:0/1068326793 learned_addr learned my addr 172.21.15.52:0/1068326793 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:09.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.568+0000 7fb8750b9640 1 --2- 172.21.15.52:0/1463523934 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb8600a3ed0 0x7fb8600a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.568+0000 7fb8748b8640 1 --2- 172.21.15.52:0/1068326793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b371e110c0 0x55b371e19760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.568+0000 7fb86ffff640 1 --2- 172.21.15.52:0/1068326793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b371e19ca0 0x55b371e1c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.568+0000 7fb86dffb640 1 -- 172.21.15.52:0/1463523934 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb8680d14d0 con 0x7fb870106980 2024-04-02T13:54:09.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.568+0000 7fb86ffff640 1 -- 172.21.15.52:0/1068326793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b371e19ca0 msgr2=0x55b371e1c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.568+0000 7fb86ffff640 1 --2- 172.21.15.52:0/1068326793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b371e19ca0 0x55b371e1c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.568+0000 7fb86ffff640 1 -- 172.21.15.52:0/1068326793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b371e110c0 msgr2=0x55b371e19760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.568+0000 7fb86ffff640 1 --2- 172.21.15.52:0/1068326793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b371e110c0 0x55b371e19760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.568+0000 7fb86ffff640 1 -- 172.21.15.52:0/1068326793 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b371a883f0 con 0x55b371e107a0 2024-04-02T13:54:09.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.568+0000 7fb8748b8640 1 --2- 172.21.15.52:0/1068326793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b371e110c0 0x55b371e19760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:54:09.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.569+0000 7fb86ffff640 1 --2- 172.21.15.52:0/1068326793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e107a0 0x55b371e10b80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb85c02cfd0 tx=0x7fb85c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a64ab8864ebb1c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.569+0000 7fb8750b9640 1 --2- 172.21.15.52:0/1463523934 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb8600a3ed0 0x7fb8600a6390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb8680ce700 tx=0x7fb86806b8a0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.569+0000 7fb85affd640 1 -- 172.21.15.52:0/1068326793 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c049070 con 0x55b371e107a0 2024-04-02T13:54:09.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.569+0000 7fb85affd640 1 -- 172.21.15.52:0/1068326793 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb85c02cb00 con 0x55b371e107a0 2024-04-02T13:54:09.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.569+0000 7fb85affd640 1 -- 172.21.15.52:0/1068326793 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c004f50 con 0x55b371e107a0 2024-04-02T13:54:09.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.569+0000 7fb8797e8740 1 -- 172.21.15.52:0/1068326793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e107a0 msgr2=0x55b371e10b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.570+0000 7fb8797e8740 1 --2- 172.21.15.52:0/1068326793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e107a0 0x55b371e10b80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb85c02cfd0 tx=0x7fb85c005b50 comp rx=0 tx=0).stop 2024-04-02T13:54:09.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.570+0000 7fb8797e8740 1 -- 172.21.15.52:0/1068326793 shutdown_connections 2024-04-02T13:54:09.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.570+0000 7fb8797e8740 1 --2- 172.21.15.52:0/1068326793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b371e19ca0 0x55b371e1c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.570+0000 7fb8797e8740 1 --2- 172.21.15.52:0/1068326793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b371e110c0 0x55b371e19760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.570+0000 7fb8797e8740 1 --2- 172.21.15.52:0/1068326793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e107a0 0x55b371e10b80 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.570+0000 7fb8797e8740 1 -- 172.21.15.52:0/1068326793 >> 172.21.15.52:0/1068326793 conn(0x55b371e0aed0 msgr2=0x55b371e0b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.570+0000 7fb8797e8740 1 -- 172.21.15.52:0/1068326793 shutdown_connections 2024-04-02T13:54:09.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.570+0000 7fb8797e8740 1 -- 172.21.15.52:0/1068326793 wait complete. 2024-04-02T13:54:09.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.570+0000 7fb8797e8740 1 -- start start 2024-04-02T13:54:09.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.570+0000 7fb8797e8740 1 Processor -- start 2024-04-02T13:54:09.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.570+0000 7fb8797e8740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e107a0 0x55b371e16b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.570+0000 7fb8797e8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b371e110c0 0x55b371e17060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.570+0000 7fb8797e8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b371e19ca0 0x55b371e175a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.570+0000 7fb8797e8740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b371e1ca30 con 0x55b371e107a0 2024-04-02T13:54:09.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.570+0000 7fb8797e8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b371d2fe80 con 0x55b371e19ca0 2024-04-02T13:54:09.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.570+0000 7fb8797e8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b371e1cbb0 con 0x55b371e110c0 2024-04-02T13:54:09.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.570+0000 7fb8797e8740 10 client.0 _ll_register_callbacks cb 0x55b371d6baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:09.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.570+0000 7fb86ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e107a0 0x55b371e16b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.571+0000 7fb86ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e107a0 0x55b371e16b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33388/0 (socket says 172.21.15.52:33388) 2024-04-02T13:54:09.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.571+0000 7fb86ffff640 1 -- 172.21.15.52:0/3986427496 learned_addr learned my addr 172.21.15.52:0/3986427496 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:09.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.571+0000 7fb8748b8640 1 --2- 172.21.15.52:0/3986427496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b371e110c0 0x55b371e17060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.571+0000 7fb86ffff640 1 -- 172.21.15.52:0/3986427496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b371e110c0 msgr2=0x55b371e17060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.571+0000 7fb86ffff640 1 --2- 172.21.15.52:0/3986427496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b371e110c0 0x55b371e17060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.571+0000 7fb86ffff640 1 -- 172.21.15.52:0/3986427496 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b371e19ca0 msgr2=0x55b371e175a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.571+0000 7fb86ffff640 1 --2- 172.21.15.52:0/3986427496 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b371e19ca0 0x55b371e175a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.571+0000 7fb86ffff640 1 -- 172.21.15.52:0/3986427496 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b371b527d0 con 0x55b371e107a0 2024-04-02T13:54:09.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.571+0000 7fb86ffff640 1 --2- 172.21.15.52:0/3986427496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e107a0 0x55b371e16b20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb85c004b30 tx=0x7fb85c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b347c5798cd04f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.571+0000 7fb84ffff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:09.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.571+0000 7fb84ffff640 1 -- 172.21.15.52:0/3986427496 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c049030 con 0x55b371e107a0 2024-04-02T13:54:09.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.571+0000 7fb84ffff640 1 -- 172.21.15.52:0/3986427496 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb85c004240 con 0x55b371e107a0 2024-04-02T13:54:09.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.571+0000 7fb8797e8740 1 -- 172.21.15.52:0/3986427496 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb85c02cb00 con 0x55b371e107a0 2024-04-02T13:54:09.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.571+0000 7fb84ffff640 1 -- 172.21.15.52:0/3986427496 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb85c056050 con 0x55b371e107a0 2024-04-02T13:54:09.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.571+0000 7fb8797e8740 10 client.15900 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:09.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.571+0000 7fb8797e8740 1 -- 172.21.15.52:0/3986427496 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b371e28b20 con 0x55b371e107a0 2024-04-02T13:54:09.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.571+0000 7fb84ffff640 1 -- 172.21.15.52:0/3986427496 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb85c02cb00 con 0x55b371e107a0 2024-04-02T13:54:09.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.572+0000 7fb84ffff640 1 -- 172.21.15.52:0/3986427496 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb85c003950 con 0x55b371e107a0 2024-04-02T13:54:09.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.572+0000 7fb8797e8740 10 client.15900 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:09.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.572+0000 7fb8797e8740 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:54:09.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.572+0000 7fb8797e8740 20 client.15900 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:09.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.572+0000 7fb8797e8740 1 --2- 172.21.15.52:0/3986427496 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55b371e29e70 0x55b371e2c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.572+0000 7fb8797e8740 4 client.15900 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:54:09.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.572+0000 7fb8797e8740 1 -- 172.21.15.52:0/3986427496 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b371d81cf0 con 0x55b371e29e70 2024-04-02T13:54:09.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.572+0000 7fb8748b8640 1 --2- 172.21.15.52:0/3986427496 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55b371e29e70 0x55b371e2c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.572+0000 7fb8748b8640 1 --2- 172.21.15.52:0/3986427496 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55b371e29e70 0x55b371e2c260 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec157a5d3bd6ba2e server_cookie=a53a57761da160c8 in_seq=0 out_seq=0 2024-04-02T13:54:09.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.572+0000 7fb84ffff640 10 client.15900 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:09.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.573+0000 7fb84ffff640 1 -- 172.21.15.52:0/3986427496 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b371d81cf0 con 0x55b371e29e70 2024-04-02T13:54:09.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.573+0000 7fb84ffff640 10 client.15900 handle_command_reply: tid=0 2024-04-02T13:54:09.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.573+0000 7fb8797e8740 1 client.15900 shutdown 2024-04-02T13:54:09.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.573+0000 7fb8797e8740 1 -- 172.21.15.52:0/3986427496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e107a0 msgr2=0x55b371e16b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.573+0000 7fb8797e8740 1 --2- 172.21.15.52:0/3986427496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e107a0 0x55b371e16b20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb85c004b30 tx=0x7fb85c0048f0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.573+0000 7fb8797e8740 1 -- 172.21.15.52:0/3986427496 shutdown_connections 2024-04-02T13:54:09.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.573+0000 7fb8797e8740 1 --2- 172.21.15.52:0/3986427496 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b371e19ca0 0x55b371e175a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.573+0000 7fb8797e8740 1 --2- 172.21.15.52:0/3986427496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b371e110c0 0x55b371e17060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.573+0000 7fb8797e8740 1 --2- 172.21.15.52:0/3986427496 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55b371e29e70 0x55b371e2c260 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.573+0000 7fb8797e8740 1 --2- 172.21.15.52:0/3986427496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e107a0 0x55b371e16b20 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.573+0000 7fb84ffff640 0 client.15900 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:09.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.573+0000 7fb8797e8740 1 -- 172.21.15.52:0/3986427496 >> 172.21.15.52:0/3986427496 conn(0x55b371e0aed0 msgr2=0x55b371e0cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.573+0000 7fb8797e8740 1 -- 172.21.15.52:0/3986427496 shutdown_connections 2024-04-02T13:54:09.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.574+0000 7fb8797e8740 1 -- 172.21.15.52:0/3986427496 wait complete. 2024-04-02T13:54:09.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.574+0000 7fb8797e8740 20 client.15900 trim_cache size 0 max 16384 2024-04-02T13:54:09.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.587+0000 7fb8797e8740 1 Processor -- start 2024-04-02T13:54:09.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.587+0000 7fb8797e8740 1 -- start start 2024-04-02T13:54:09.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.587+0000 7fb8797e8740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e107a0 0x55b371e19250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.587+0000 7fb8797e8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b371e110c0 0x55b371e159d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.587+0000 7fb8797e8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b371e19ca0 0x55b371e2f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.587+0000 7fb8797e8740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb8701aeda0 con 0x55b371e107a0 2024-04-02T13:54:09.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.587+0000 7fb8797e8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb8701b08e0 con 0x55b371e19ca0 2024-04-02T13:54:09.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.587+0000 7fb8797e8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb8701af470 con 0x55b371e110c0 2024-04-02T13:54:09.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.587+0000 7fb86ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e107a0 0x55b371e19250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.587+0000 7fb86ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e107a0 0x55b371e19250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33400/0 (socket says 172.21.15.52:33400) 2024-04-02T13:54:09.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.587+0000 7fb86ffff640 1 -- 172.21.15.52:0/2647809291 learned_addr learned my addr 172.21.15.52:0/2647809291 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:09.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.588+0000 7fb86ffff640 1 -- 172.21.15.52:0/2647809291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b371e110c0 msgr2=0x55b371e159d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.588+0000 7fb86ffff640 1 --2- 172.21.15.52:0/2647809291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b371e110c0 0x55b371e159d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.588+0000 7fb86ffff640 1 -- 172.21.15.52:0/2647809291 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b371e19ca0 msgr2=0x55b371e2f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.588+0000 7fb86ffff640 1 --2- 172.21.15.52:0/2647809291 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b371e19ca0 0x55b371e2f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.588+0000 7fb86ffff640 1 -- 172.21.15.52:0/2647809291 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b371e1c970 con 0x55b371e107a0 2024-04-02T13:54:09.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.588+0000 7fb86ffff640 1 --2- 172.21.15.52:0/2647809291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e107a0 0x55b371e19250 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb85c004a00 tx=0x7fb85c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6829882160b314 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.588+0000 7fb85a7fc640 1 -- 172.21.15.52:0/2647809291 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c049030 con 0x55b371e107a0 2024-04-02T13:54:09.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.588+0000 7fb85a7fc640 1 -- 172.21.15.52:0/2647809291 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb85c045070 con 0x55b371e107a0 2024-04-02T13:54:09.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.588+0000 7fb85a7fc640 1 -- 172.21.15.52:0/2647809291 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c004240 con 0x55b371e107a0 2024-04-02T13:54:09.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.589+0000 7fb8797e8740 1 -- 172.21.15.52:0/2647809291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e107a0 msgr2=0x55b371e19250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.589+0000 7fb8797e8740 1 --2- 172.21.15.52:0/2647809291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e107a0 0x55b371e19250 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb85c004a00 tx=0x7fb85c005fd0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.589+0000 7fb8797e8740 1 -- 172.21.15.52:0/2647809291 shutdown_connections 2024-04-02T13:54:09.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.589+0000 7fb8797e8740 1 --2- 172.21.15.52:0/2647809291 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b371e19ca0 0x55b371e2f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.589+0000 7fb8797e8740 1 --2- 172.21.15.52:0/2647809291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b371e110c0 0x55b371e159d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.589+0000 7fb8797e8740 1 --2- 172.21.15.52:0/2647809291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e107a0 0x55b371e19250 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.589+0000 7fb8797e8740 1 -- 172.21.15.52:0/2647809291 >> 172.21.15.52:0/2647809291 conn(0x55b371e0aed0 msgr2=0x55b371e152c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.589+0000 7fb8797e8740 1 -- 172.21.15.52:0/2647809291 shutdown_connections 2024-04-02T13:54:09.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.589+0000 7fb8797e8740 1 -- 172.21.15.52:0/2647809291 wait complete. 2024-04-02T13:54:09.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.589+0000 7fb8797e8740 1 -- start start 2024-04-02T13:54:09.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.589+0000 7fb8797e8740 1 Processor -- start 2024-04-02T13:54:09.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.589+0000 7fb8797e8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b371e107a0 0x55b371e2f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.589+0000 7fb8797e8740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e110c0 0x55b371e2caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.589+0000 7fb8797e8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b371e19ca0 0x55b371e2d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.589+0000 7fb8797e8740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb8701b0210 con 0x55b371e110c0 2024-04-02T13:54:09.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.589+0000 7fb8797e8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb8701afb40 con 0x55b371e19ca0 2024-04-02T13:54:09.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.589+0000 7fb8797e8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb8701acd80 con 0x55b371e107a0 2024-04-02T13:54:09.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.589+0000 7fb8797e8740 10 client.0 _ll_register_callbacks cb 0x7fb85c047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:09.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.589+0000 7fb8748b8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e110c0 0x55b371e2caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.589+0000 7fb8748b8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e110c0 0x55b371e2caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33408/0 (socket says 172.21.15.52:33408) 2024-04-02T13:54:09.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.589+0000 7fb8748b8640 1 -- 172.21.15.52:0/745976417 learned_addr learned my addr 172.21.15.52:0/745976417 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:09.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.590+0000 7fb86ffff640 1 --2- 172.21.15.52:0/745976417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b371e107a0 0x55b371e2f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.590+0000 7fb86ffff640 1 --2- 172.21.15.52:0/745976417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b371e19ca0 0x55b371e2d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.590+0000 7fb8748b8640 1 -- 172.21.15.52:0/745976417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b371e107a0 msgr2=0x55b371e2f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.590+0000 7fb8748b8640 1 --2- 172.21.15.52:0/745976417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b371e107a0 0x55b371e2f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.590+0000 7fb8748b8640 1 -- 172.21.15.52:0/745976417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b371e19ca0 msgr2=0x55b371e2d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:09.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.590+0000 7fb8748b8640 1 --2- 172.21.15.52:0/745976417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b371e19ca0 0x55b371e2d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.590+0000 7fb8748b8640 1 -- 172.21.15.52:0/745976417 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b371e2d870 con 0x55b371e110c0 2024-04-02T13:54:09.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.590+0000 7fb86ffff640 1 --2- 172.21.15.52:0/745976417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b371e107a0 0x55b371e2f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:54:09.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.590+0000 7fb8748b8640 1 --2- 172.21.15.52:0/745976417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e110c0 0x55b371e2caa0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x55b371e1cbd0 tx=0x7fb864011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=affe434bfbf219af server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:09.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.590+0000 7fb84ffff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:09.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.590+0000 7fb84ffff640 1 -- 172.21.15.52:0/745976417 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864004240 con 0x55b371e110c0 2024-04-02T13:54:09.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.590+0000 7fb84ffff640 1 -- 172.21.15.52:0/745976417 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb8640043e0 con 0x55b371e110c0 2024-04-02T13:54:09.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.590+0000 7fb8797e8740 1 -- 172.21.15.52:0/745976417 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb85c045070 con 0x55b371e110c0 2024-04-02T13:54:09.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.590+0000 7fb84ffff640 1 -- 172.21.15.52:0/745976417 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb86402e050 con 0x55b371e110c0 2024-04-02T13:54:09.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.590+0000 7fb8797e8740 10 client.15906 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:09.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.590+0000 7fb8797e8740 1 -- 172.21.15.52:0/745976417 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b371e901b0 con 0x55b371e110c0 2024-04-02T13:54:09.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.590+0000 7fb84ffff640 1 -- 172.21.15.52:0/745976417 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb85c045070 con 0x55b371e110c0 2024-04-02T13:54:09.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.591+0000 7fb84ffff640 1 -- 172.21.15.52:0/745976417 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb864003950 con 0x55b371e110c0 2024-04-02T13:54:09.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.591+0000 7fb8797e8740 10 client.15906 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:09.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.591+0000 7fb8797e8740 10 client.15906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:54:09.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.591+0000 7fb8797e8740 20 client.15906 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:09.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.591+0000 7fb8797e8740 1 --2- 172.21.15.52:0/745976417 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55b371e90960 0x55b371e92d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:09.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.591+0000 7fb8797e8740 4 client.15906 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:54:09.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.591+0000 7fb8797e8740 1 -- 172.21.15.52:0/745976417 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b371e11950 con 0x55b371e90960 2024-04-02T13:54:09.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.591+0000 7fb86ffff640 1 --2- 172.21.15.52:0/745976417 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55b371e90960 0x55b371e92d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:09.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.591+0000 7fb84ffff640 10 client.15906 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:09.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.591+0000 7fb86ffff640 1 --2- 172.21.15.52:0/745976417 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55b371e90960 0x55b371e92d80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30b54058d452d9fd server_cookie=4648f88ba8488984 in_seq=0 out_seq=0 2024-04-02T13:54:09.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.592+0000 7fb84ffff640 1 -- 172.21.15.52:0/745976417 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b371e11950 con 0x55b371e90960 2024-04-02T13:54:09.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.592+0000 7fb84ffff640 10 client.15906 handle_command_reply: tid=0 2024-04-02T13:54:09.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.592+0000 7fb8797e8740 1 client.15906 shutdown 2024-04-02T13:54:09.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.592+0000 7fb8797e8740 1 -- 172.21.15.52:0/745976417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e110c0 msgr2=0x55b371e2caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.592+0000 7fb8797e8740 1 --2- 172.21.15.52:0/745976417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e110c0 0x55b371e2caa0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x55b371e1cbd0 tx=0x7fb864011810 comp rx=0 tx=0).stop 2024-04-02T13:54:09.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.592+0000 7fb8797e8740 1 -- 172.21.15.52:0/745976417 shutdown_connections 2024-04-02T13:54:09.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.592+0000 7fb8797e8740 1 --2- 172.21.15.52:0/745976417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b371e19ca0 0x55b371e2d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.592+0000 7fb8797e8740 1 --2- 172.21.15.52:0/745976417 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55b371e90960 0x55b371e92d80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.592+0000 7fb8797e8740 1 --2- 172.21.15.52:0/745976417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b371e110c0 0x55b371e2caa0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.592+0000 7fb8797e8740 1 --2- 172.21.15.52:0/745976417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b371e107a0 0x55b371e2f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.592+0000 7fb84ffff640 0 client.15906 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:09.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.592+0000 7fb8797e8740 1 -- 172.21.15.52:0/745976417 >> 172.21.15.52:0/745976417 conn(0x55b371e0aed0 msgr2=0x55b371e0ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.592+0000 7fb8797e8740 1 -- 172.21.15.52:0/745976417 shutdown_connections 2024-04-02T13:54:09.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.592+0000 7fb8797e8740 1 -- 172.21.15.52:0/745976417 wait complete. 2024-04-02T13:54:09.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.592+0000 7fb8797e8740 20 client.15906 trim_cache size 0 max 16384 2024-04-02T13:54:09.604 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:54:09.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.592+0000 7fb876b1c640 1 -- 172.21.15.52:0/1463523934 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb8600a3ed0 msgr2=0x7fb8600a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.592+0000 7fb876b1c640 1 --2- 172.21.15.52:0/1463523934 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb8600a3ed0 0x7fb8600a6390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb8680ce700 tx=0x7fb86806b8a0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.592+0000 7fb876b1c640 1 -- 172.21.15.52:0/1463523934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb870106980 msgr2=0x7fb87019cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:09.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.592+0000 7fb876b1c640 1 --2- 172.21.15.52:0/1463523934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb870106980 0x7fb87019cc70 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb86800eb10 tx=0x7fb86800c2d0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.593+0000 7fb876b1c640 1 -- 172.21.15.52:0/1463523934 shutdown_connections 2024-04-02T13:54:09.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.593+0000 7fb876b1c640 1 --2- 172.21.15.52:0/1463523934 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb8600a3ed0 0x7fb8600a6390 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.593+0000 7fb876b1c640 1 --2- 172.21.15.52:0/1463523934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb870106980 0x7fb87019cc70 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.593+0000 7fb876b1c640 1 --2- 172.21.15.52:0/1463523934 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb870101330 0x7fb8701988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.593+0000 7fb876b1c640 1 --2- 172.21.15.52:0/1463523934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb870100970 0x7fb8701983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:09.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.593+0000 7fb876b1c640 1 -- 172.21.15.52:0/1463523934 >> 172.21.15.52:0/1463523934 conn(0x7fb870077b00 msgr2=0x7fb870068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:09.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.593+0000 7fb876b1c640 1 -- 172.21.15.52:0/1463523934 shutdown_connections 2024-04-02T13:54:09.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:09.593+0000 7fb876b1c640 1 -- 172.21.15.52:0/1463523934 wait complete. 2024-04-02T13:54:10.608 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:54:10.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:54:10.608 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:54:10.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.724+0000 7ff8932b1640 1 -- 172.21.15.52:0/2397016324 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff88c069240 msgr2=0x7ff88c0696a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:10.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.724+0000 7ff8932b1640 1 --2- 172.21.15.52:0/2397016324 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff88c069240 0x7ff88c0696a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff87c009930 tx=0x7ff87c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:54:10.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.724+0000 7ff8932b1640 1 -- 172.21.15.52:0/2397016324 shutdown_connections 2024-04-02T13:54:10.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.724+0000 7ff8932b1640 1 --2- 172.21.15.52:0/2397016324 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff88c1045c0 0x7ff88c104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.724+0000 7ff8932b1640 1 --2- 172.21.15.52:0/2397016324 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff88c069240 0x7ff88c0696a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.724+0000 7ff8932b1640 1 --2- 172.21.15.52:0/2397016324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff88c106a50 0x7ff88c068d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.724+0000 7ff8932b1640 1 -- 172.21.15.52:0/2397016324 >> 172.21.15.52:0/2397016324 conn(0x7ff88c077b00 msgr2=0x7ff88c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:10.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.724+0000 7ff8932b1640 1 -- 172.21.15.52:0/2397016324 shutdown_connections 2024-04-02T13:54:10.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.724+0000 7ff8932b1640 1 -- 172.21.15.52:0/2397016324 wait complete. 2024-04-02T13:54:10.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.725+0000 7ff8932b1640 1 Processor -- start 2024-04-02T13:54:10.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.725+0000 7ff8932b1640 1 -- start start 2024-04-02T13:54:10.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.725+0000 7ff8932b1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff88c069240 0x7ff88c1a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.725+0000 7ff8932b1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff88c1045c0 0x7ff88c1a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.725+0000 7ff8932b1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff88c106a50 0x7ff88c19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.725+0000 7ff8932b1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c118430 con 0x7ff88c106a50 2024-04-02T13:54:10.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.725+0000 7ff8932b1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c1182b0 con 0x7ff88c1045c0 2024-04-02T13:54:10.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.725+0000 7ff8932b1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c1185b0 con 0x7ff88c069240 2024-04-02T13:54:10.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.725+0000 7ff89184e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff88c106a50 0x7ff88c19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.725+0000 7ff89184e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff88c106a50 0x7ff88c19e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33426/0 (socket says 172.21.15.52:33426) 2024-04-02T13:54:10.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.725+0000 7ff89184e640 1 -- 172.21.15.52:0/4247388896 learned_addr learned my addr 172.21.15.52:0/4247388896 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:10.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.726+0000 7ff89084c640 1 --2- 172.21.15.52:0/4247388896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff88c1045c0 0x7ff88c1a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.726+0000 7ff89104d640 1 --2- 172.21.15.52:0/4247388896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff88c069240 0x7ff88c1a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.726+0000 7ff89184e640 1 -- 172.21.15.52:0/4247388896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff88c069240 msgr2=0x7ff88c1a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:10.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.726+0000 7ff89184e640 1 --2- 172.21.15.52:0/4247388896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff88c069240 0x7ff88c1a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.726+0000 7ff89184e640 1 -- 172.21.15.52:0/4247388896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff88c1045c0 msgr2=0x7ff88c1a4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:10.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.726+0000 7ff89184e640 1 --2- 172.21.15.52:0/4247388896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff88c1045c0 0x7ff88c1a4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.726+0000 7ff89184e640 1 -- 172.21.15.52:0/4247388896 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88c19ef60 con 0x7ff88c106a50 2024-04-02T13:54:10.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.726+0000 7ff89104d640 1 --2- 172.21.15.52:0/4247388896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff88c069240 0x7ff88c1a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:54:10.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.726+0000 7ff89184e640 1 --2- 172.21.15.52:0/4247388896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff88c106a50 0x7ff88c19e790 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff88400eaa0 tx=0x7ff88400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:10.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.726+0000 7ff8827fc640 1 -- 172.21.15.52:0/4247388896 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88400cdb0 con 0x7ff88c106a50 2024-04-02T13:54:10.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.726+0000 7ff8827fc640 1 -- 172.21.15.52:0/4247388896 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff884005020 con 0x7ff88c106a50 2024-04-02T13:54:10.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.726+0000 7ff8932b1640 1 -- 172.21.15.52:0/4247388896 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff88c19f250 con 0x7ff88c106a50 2024-04-02T13:54:10.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.726+0000 7ff8827fc640 1 -- 172.21.15.52:0/4247388896 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8840116b0 con 0x7ff88c106a50 2024-04-02T13:54:10.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.726+0000 7ff8932b1640 1 -- 172.21.15.52:0/4247388896 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff88c1ab470 con 0x7ff88c106a50 2024-04-02T13:54:10.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.728+0000 7ff8827fc640 1 -- 172.21.15.52:0/4247388896 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7ff884005560 con 0x7ff88c106a50 2024-04-02T13:54:10.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.729+0000 7ff895f73740 1 Processor -- start 2024-04-02T13:54:10.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.729+0000 7ff895f73740 1 -- start start 2024-04-02T13:54:10.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.729+0000 7ff895f73740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb20bd2c0 0x560cb20bd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.729+0000 7ff895f73740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20bdbe0 0x560cb20c6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.729+0000 7ff895f73740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560cb20c67c0 0x560cb20c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.729+0000 7ff895f73740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560cb2021770 con 0x560cb20bdbe0 2024-04-02T13:54:10.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.729+0000 7ff895f73740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560cb1fc9960 con 0x560cb20c67c0 2024-04-02T13:54:10.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.729+0000 7ff895f73740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560cb201d8c0 con 0x560cb20bd2c0 2024-04-02T13:54:10.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.729+0000 7ff8827fc640 1 --2- 172.21.15.52:0/4247388896 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff8780a3ed0 0x7ff8780a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.729+0000 7ff89104d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20bdbe0 0x560cb20c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.729+0000 7ff89104d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20bdbe0 0x560cb20c6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33434/0 (socket says 172.21.15.52:33434) 2024-04-02T13:54:10.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.729+0000 7ff89104d640 1 -- 172.21.15.52:0/1833163212 learned_addr learned my addr 172.21.15.52:0/1833163212 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:10.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.729+0000 7ff89084c640 1 --2- 172.21.15.52:0/1833163212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560cb20c67c0 0x560cb20c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.729+0000 7ff89084c640 1 --2- 172.21.15.52:0/1833163212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb20bd2c0 0x560cb20bd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.729+0000 7ff89184e640 1 --2- 172.21.15.52:0/4247388896 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff8780a3ed0 0x7ff8780a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.729+0000 7ff8827fc640 1 -- 172.21.15.52:0/4247388896 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7ff8840d1070 con 0x7ff88c106a50 2024-04-02T13:54:10.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.730+0000 7ff89104d640 1 -- 172.21.15.52:0/1833163212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb20bd2c0 msgr2=0x560cb20bd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:10.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.730+0000 7ff89104d640 1 --2- 172.21.15.52:0/1833163212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb20bd2c0 0x560cb20bd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.730+0000 7ff89104d640 1 -- 172.21.15.52:0/1833163212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560cb20c67c0 msgr2=0x560cb20c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:10.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.730+0000 7ff89104d640 1 --2- 172.21.15.52:0/1833163212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560cb20c67c0 0x560cb20c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.730+0000 7ff89104d640 1 -- 172.21.15.52:0/1833163212 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cb1dfd7e0 con 0x560cb20bdbe0 2024-04-02T13:54:10.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.730+0000 7ff89084c640 1 --2- 172.21.15.52:0/1833163212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560cb20c67c0 0x560cb20c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:54:10.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.730+0000 7ff89104d640 1 --2- 172.21.15.52:0/1833163212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20bdbe0 0x560cb20c6280 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff88800bb70 tx=0x7ff88800bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69cc71a63379a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:10.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.730+0000 7ff89184e640 1 --2- 172.21.15.52:0/4247388896 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff8780a3ed0 0x7ff8780a6390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff8840ce2a0 tx=0x7ff884053040 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:10.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.730+0000 7ff8777fe640 1 -- 172.21.15.52:0/1833163212 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff888017420 con 0x560cb20bdbe0 2024-04-02T13:54:10.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.730+0000 7ff8777fe640 1 -- 172.21.15.52:0/1833163212 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff888017aa0 con 0x560cb20bdbe0 2024-04-02T13:54:10.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.730+0000 7ff8777fe640 1 -- 172.21.15.52:0/1833163212 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88800fa60 con 0x560cb20bdbe0 2024-04-02T13:54:10.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.731+0000 7ff895f73740 1 -- 172.21.15.52:0/1833163212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20bdbe0 msgr2=0x560cb20c6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:10.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.731+0000 7ff895f73740 1 --2- 172.21.15.52:0/1833163212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20bdbe0 0x560cb20c6280 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff88800bb70 tx=0x7ff88800bf70 comp rx=0 tx=0).stop 2024-04-02T13:54:10.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.731+0000 7ff895f73740 1 -- 172.21.15.52:0/1833163212 shutdown_connections 2024-04-02T13:54:10.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.731+0000 7ff895f73740 1 --2- 172.21.15.52:0/1833163212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560cb20c67c0 0x560cb20c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.731+0000 7ff895f73740 1 --2- 172.21.15.52:0/1833163212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20bdbe0 0x560cb20c6280 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.731+0000 7ff895f73740 1 --2- 172.21.15.52:0/1833163212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb20bd2c0 0x560cb20bd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.731+0000 7ff895f73740 1 -- 172.21.15.52:0/1833163212 >> 172.21.15.52:0/1833163212 conn(0x560cb20b79f0 msgr2=0x560cb20b7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:10.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.731+0000 7ff895f73740 1 -- 172.21.15.52:0/1833163212 shutdown_connections 2024-04-02T13:54:10.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.731+0000 7ff895f73740 1 -- 172.21.15.52:0/1833163212 wait complete. 2024-04-02T13:54:10.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.731+0000 7ff895f73740 1 -- start start 2024-04-02T13:54:10.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.731+0000 7ff895f73740 1 Processor -- start 2024-04-02T13:54:10.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.732+0000 7ff895f73740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20bd2c0 0x560cb20c3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.732+0000 7ff895f73740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb20bdbe0 0x560cb20c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.732+0000 7ff895f73740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560cb20c67c0 0x560cb20c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.732+0000 7ff895f73740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560cb20c95b0 con 0x560cb20bd2c0 2024-04-02T13:54:10.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.732+0000 7ff895f73740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560cb1fdcd20 con 0x560cb20c67c0 2024-04-02T13:54:10.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.732+0000 7ff895f73740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560cb20c9730 con 0x560cb20bdbe0 2024-04-02T13:54:10.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.732+0000 7ff895f73740 10 client.0 _ll_register_callbacks cb 0x560cb1fa9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:10.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.732+0000 7ff89084c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20bd2c0 0x560cb20c3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.732+0000 7ff89084c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20bd2c0 0x560cb20c3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33442/0 (socket says 172.21.15.52:33442) 2024-04-02T13:54:10.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.732+0000 7ff89084c640 1 -- 172.21.15.52:0/274198146 learned_addr learned my addr 172.21.15.52:0/274198146 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:10.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.732+0000 7ff89104d640 1 --2- 172.21.15.52:0/274198146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb20bdbe0 0x560cb20c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.732+0000 7ff89084c640 1 --2- 172.21.15.52:0/274198146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560cb20c67c0 0x560cb20c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.732+0000 7ff89084c640 1 -- 172.21.15.52:0/274198146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb20bdbe0 msgr2=0x560cb20c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:10.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.732+0000 7ff89084c640 1 --2- 172.21.15.52:0/274198146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb20bdbe0 0x560cb20c3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.732+0000 7ff89084c640 1 -- 172.21.15.52:0/274198146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560cb20c67c0 msgr2=0x560cb20c42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:10.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.732+0000 7ff89084c640 1 --2- 172.21.15.52:0/274198146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560cb20c67c0 0x560cb20c42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.732+0000 7ff89084c640 1 -- 172.21.15.52:0/274198146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cb1e5df70 con 0x560cb20bd2c0 2024-04-02T13:54:10.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.733+0000 7ff89084c640 1 --2- 172.21.15.52:0/274198146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20bd2c0 0x560cb20c3830 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff87c004ee0 tx=0x7ff87c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7eb2be6c17553be server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:10.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.733+0000 7ff874ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:10.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.733+0000 7ff874ff9640 1 -- 172.21.15.52:0/274198146 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c004bd0 con 0x560cb20bd2c0 2024-04-02T13:54:10.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.733+0000 7ff895f73740 1 -- 172.21.15.52:0/274198146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff888017aa0 con 0x560cb20bd2c0 2024-04-02T13:54:10.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.733+0000 7ff874ff9640 1 -- 172.21.15.52:0/274198146 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7ff87c004f50 con 0x560cb20bd2c0 2024-04-02T13:54:10.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.733+0000 7ff874ff9640 1 -- 172.21.15.52:0/274198146 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff87c054050 con 0x560cb20bd2c0 2024-04-02T13:54:10.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.733+0000 7ff895f73740 10 client.15918 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:10.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.733+0000 7ff895f73740 1 -- 172.21.15.52:0/274198146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560cb1cf5170 con 0x560cb20bd2c0 2024-04-02T13:54:10.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.733+0000 7ff874ff9640 1 -- 172.21.15.52:0/274198146 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff888017aa0 con 0x560cb20bd2c0 2024-04-02T13:54:10.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.733+0000 7ff874ff9640 1 -- 172.21.15.52:0/274198146 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff87c003c30 con 0x560cb20bd2c0 2024-04-02T13:54:10.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.733+0000 7ff895f73740 10 client.15918 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:10.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.733+0000 7ff895f73740 10 client.15918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:54:10.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.733+0000 7ff895f73740 20 client.15918 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:10.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.733+0000 7ff895f73740 1 --2- 172.21.15.52:0/274198146 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x560cb20d6ae0 0x560cb20d8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.733+0000 7ff895f73740 4 client.15918 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:54:10.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.733+0000 7ff895f73740 1 -- 172.21.15.52:0/274198146 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cb202ecb0 con 0x560cb20d6ae0 2024-04-02T13:54:10.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.734+0000 7ff89104d640 1 --2- 172.21.15.52:0/274198146 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x560cb20d6ae0 0x560cb20d8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.734+0000 7ff874ff9640 10 client.15918 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:10.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.734+0000 7ff89104d640 1 --2- 172.21.15.52:0/274198146 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x560cb20d6ae0 0x560cb20d8ed0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfb4808547f448ec server_cookie=8d1bd794095f6911 in_seq=0 out_seq=0 2024-04-02T13:54:10.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.735+0000 7ff874ff9640 1 -- 172.21.15.52:0/274198146 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560cb202ecb0 con 0x560cb20d6ae0 2024-04-02T13:54:10.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.735+0000 7ff874ff9640 10 client.15918 handle_command_reply: tid=0 2024-04-02T13:54:10.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.735+0000 7ff895f73740 1 client.15918 shutdown 2024-04-02T13:54:10.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.735+0000 7ff895f73740 1 -- 172.21.15.52:0/274198146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20bd2c0 msgr2=0x560cb20c3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:10.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.735+0000 7ff895f73740 1 --2- 172.21.15.52:0/274198146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20bd2c0 0x560cb20c3830 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff87c004ee0 tx=0x7ff87c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:54:10.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.735+0000 7ff895f73740 1 -- 172.21.15.52:0/274198146 shutdown_connections 2024-04-02T13:54:10.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.735+0000 7ff895f73740 1 --2- 172.21.15.52:0/274198146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560cb20c67c0 0x560cb20c42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.735+0000 7ff895f73740 1 --2- 172.21.15.52:0/274198146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb20bdbe0 0x560cb20c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.735+0000 7ff895f73740 1 --2- 172.21.15.52:0/274198146 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x560cb20d6ae0 0x560cb20d8ed0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.735+0000 7ff895f73740 1 --2- 172.21.15.52:0/274198146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20bd2c0 0x560cb20c3830 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.735+0000 7ff874ff9640 0 client.15918 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:10.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.735+0000 7ff895f73740 1 -- 172.21.15.52:0/274198146 >> 172.21.15.52:0/274198146 conn(0x560cb20b79f0 msgr2=0x560cb20b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:10.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.735+0000 7ff895f73740 1 -- 172.21.15.52:0/274198146 shutdown_connections 2024-04-02T13:54:10.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.735+0000 7ff895f73740 1 -- 172.21.15.52:0/274198146 wait complete. 2024-04-02T13:54:10.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.735+0000 7ff895f73740 20 client.15918 trim_cache size 0 max 16384 2024-04-02T13:54:10.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.753+0000 7ff895f73740 1 Processor -- start 2024-04-02T13:54:10.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.753+0000 7ff895f73740 1 -- start start 2024-04-02T13:54:10.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.753+0000 7ff895f73740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20bd2c0 0x560cb20c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.753+0000 7ff895f73740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb20bdbe0 0x560cb20d2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.753+0000 7ff895f73740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560cb20c67c0 0x560cb20de7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.753+0000 7ff895f73740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c1b54c0 con 0x560cb20bd2c0 2024-04-02T13:54:10.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.753+0000 7ff895f73740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c1b7000 con 0x560cb20c67c0 2024-04-02T13:54:10.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.753+0000 7ff895f73740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c1b5b90 con 0x560cb20bdbe0 2024-04-02T13:54:10.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.753+0000 7ff89084c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20bd2c0 0x560cb20c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.753+0000 7ff89084c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20bd2c0 0x560cb20c5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33452/0 (socket says 172.21.15.52:33452) 2024-04-02T13:54:10.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.753+0000 7ff89084c640 1 -- 172.21.15.52:0/4074973796 learned_addr learned my addr 172.21.15.52:0/4074973796 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:10.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.753+0000 7ff89104d640 1 --2- 172.21.15.52:0/4074973796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb20bdbe0 0x560cb20d2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.754+0000 7ff89084c640 1 -- 172.21.15.52:0/4074973796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb20bdbe0 msgr2=0x560cb20d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:10.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.754+0000 7ff89084c640 1 --2- 172.21.15.52:0/4074973796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb20bdbe0 0x560cb20d2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.754+0000 7ff89084c640 1 -- 172.21.15.52:0/4074973796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560cb20c67c0 msgr2=0x560cb20de7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:10.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.754+0000 7ff89084c640 1 --2- 172.21.15.52:0/4074973796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560cb20c67c0 0x560cb20de7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.754+0000 7ff89084c640 1 -- 172.21.15.52:0/4074973796 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cb20c4790 con 0x560cb20bd2c0 2024-04-02T13:54:10.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.754+0000 7ff89104d640 1 --2- 172.21.15.52:0/4074973796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb20bdbe0 0x560cb20d2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:54:10.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.754+0000 7ff89084c640 1 --2- 172.21.15.52:0/4074973796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20bd2c0 0x560cb20c5f60 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff87c001eb0 tx=0x7ff87c02c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f04a0c355b9270 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:10.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.754+0000 7ff876ffd640 1 -- 172.21.15.52:0/4074973796 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c0049d0 con 0x560cb20bd2c0 2024-04-02T13:54:10.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.754+0000 7ff876ffd640 1 -- 172.21.15.52:0/4074973796 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff87c004b70 con 0x560cb20bd2c0 2024-04-02T13:54:10.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.754+0000 7ff876ffd640 1 -- 172.21.15.52:0/4074973796 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c0052d0 con 0x560cb20bd2c0 2024-04-02T13:54:10.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.754+0000 7ff895f73740 1 -- 172.21.15.52:0/4074973796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20bd2c0 msgr2=0x560cb20c5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:10.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.754+0000 7ff895f73740 1 --2- 172.21.15.52:0/4074973796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20bd2c0 0x560cb20c5f60 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff87c001eb0 tx=0x7ff87c02c970 comp rx=0 tx=0).stop 2024-04-02T13:54:10.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.755+0000 7ff895f73740 1 -- 172.21.15.52:0/4074973796 shutdown_connections 2024-04-02T13:54:10.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.755+0000 7ff895f73740 1 --2- 172.21.15.52:0/4074973796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560cb20c67c0 0x560cb20de7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.755+0000 7ff895f73740 1 --2- 172.21.15.52:0/4074973796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb20bdbe0 0x560cb20d2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.755+0000 7ff895f73740 1 --2- 172.21.15.52:0/4074973796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20bd2c0 0x560cb20c5f60 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.755+0000 7ff895f73740 1 -- 172.21.15.52:0/4074973796 >> 172.21.15.52:0/4074973796 conn(0x560cb20b79f0 msgr2=0x560cb20c4080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:10.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.755+0000 7ff895f73740 1 -- 172.21.15.52:0/4074973796 shutdown_connections 2024-04-02T13:54:10.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.755+0000 7ff895f73740 1 -- 172.21.15.52:0/4074973796 wait complete. 2024-04-02T13:54:10.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.755+0000 7ff895f73740 1 -- start start 2024-04-02T13:54:10.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.755+0000 7ff895f73740 1 Processor -- start 2024-04-02T13:54:10.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.755+0000 7ff895f73740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb20bd2c0 0x560cb20cae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.755+0000 7ff895f73740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560cb20bdbe0 0x560cb20cb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.755+0000 7ff895f73740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20c67c0 0x560cb20ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.755+0000 7ff895f73740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c1b6930 con 0x560cb20c67c0 2024-04-02T13:54:10.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.755+0000 7ff895f73740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c1b6260 con 0x560cb20bdbe0 2024-04-02T13:54:10.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.755+0000 7ff895f73740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c1b34a0 con 0x560cb20bd2c0 2024-04-02T13:54:10.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.755+0000 7ff895f73740 10 client.0 _ll_register_callbacks cb 0x7ff87c005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:10.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.756+0000 7ff89084c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20c67c0 0x560cb20ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.756+0000 7ff89084c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20c67c0 0x560cb20ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33466/0 (socket says 172.21.15.52:33466) 2024-04-02T13:54:10.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.756+0000 7ff89084c640 1 -- 172.21.15.52:0/4212205034 learned_addr learned my addr 172.21.15.52:0/4212205034 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:10.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.756+0000 7ff89104d640 1 --2- 172.21.15.52:0/4212205034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560cb20bdbe0 0x560cb20cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.756+0000 7ff89084c640 1 -- 172.21.15.52:0/4212205034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb20bd2c0 msgr2=0x560cb20cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:10.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.756+0000 7ff89084c640 1 --2- 172.21.15.52:0/4212205034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb20bd2c0 0x560cb20cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.756+0000 7ff89084c640 1 -- 172.21.15.52:0/4212205034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560cb20bdbe0 msgr2=0x560cb20cb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:10.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.756+0000 7ff89084c640 1 --2- 172.21.15.52:0/4212205034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560cb20bdbe0 0x560cb20cb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.756+0000 7ff89084c640 1 -- 172.21.15.52:0/4212205034 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cb20ce980 con 0x560cb20c67c0 2024-04-02T13:54:10.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.756+0000 7ff89104d640 1 --2- 172.21.15.52:0/4212205034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560cb20bdbe0 0x560cb20cb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:54:10.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.756+0000 7ff89084c640 1 --2- 172.21.15.52:0/4212205034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20c67c0 0x560cb20ce340 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff87c02c610 tx=0x7ff87c034660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd7b9960455d4cf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:10.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.756+0000 7ff874ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:10.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.756+0000 7ff874ff9640 1 -- 172.21.15.52:0/4212205034 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c0049d0 con 0x560cb20c67c0 2024-04-02T13:54:10.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.756+0000 7ff874ff9640 1 -- 172.21.15.52:0/4212205034 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7ff87c004f50 con 0x560cb20c67c0 2024-04-02T13:54:10.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.756+0000 7ff895f73740 1 -- 172.21.15.52:0/4212205034 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff87c004b70 con 0x560cb20c67c0 2024-04-02T13:54:10.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.756+0000 7ff874ff9640 1 -- 172.21.15.52:0/4212205034 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff87c005170 con 0x560cb20c67c0 2024-04-02T13:54:10.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.756+0000 7ff895f73740 10 client.15924 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:10.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.756+0000 7ff895f73740 1 -- 172.21.15.52:0/4212205034 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560cb213da10 con 0x560cb20c67c0 2024-04-02T13:54:10.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.756+0000 7ff874ff9640 1 -- 172.21.15.52:0/4212205034 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff87c004b70 con 0x560cb20c67c0 2024-04-02T13:54:10.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.757+0000 7ff874ff9640 1 -- 172.21.15.52:0/4212205034 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff87c003ab0 con 0x560cb20c67c0 2024-04-02T13:54:10.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.757+0000 7ff895f73740 10 client.15924 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:10.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.757+0000 7ff895f73740 10 client.15924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:54:10.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.757+0000 7ff895f73740 20 client.15924 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:10.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.757+0000 7ff895f73740 1 --2- 172.21.15.52:0/4212205034 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x560cb213dfc0 0x560cb21403b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.757+0000 7ff895f73740 4 client.15924 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:54:10.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.757+0000 7ff895f73740 1 -- 172.21.15.52:0/4212205034 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560cb2029000 con 0x560cb213dfc0 2024-04-02T13:54:10.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.757+0000 7ff89104d640 1 --2- 172.21.15.52:0/4212205034 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x560cb213dfc0 0x560cb21403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.757+0000 7ff874ff9640 10 client.15924 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:10.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.757+0000 7ff89104d640 1 --2- 172.21.15.52:0/4212205034 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x560cb213dfc0 0x560cb21403b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9824de46d86bcf5 server_cookie=e95f6771c786bcd3 in_seq=0 out_seq=0 2024-04-02T13:54:10.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.759+0000 7ff874ff9640 1 -- 172.21.15.52:0/4212205034 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560cb2029000 con 0x560cb213dfc0 2024-04-02T13:54:10.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.759+0000 7ff874ff9640 10 client.15924 handle_command_reply: tid=0 2024-04-02T13:54:10.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.759+0000 7ff895f73740 1 client.15924 shutdown 2024-04-02T13:54:10.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.759+0000 7ff895f73740 1 -- 172.21.15.52:0/4212205034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20c67c0 msgr2=0x560cb20ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:10.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.759+0000 7ff895f73740 1 --2- 172.21.15.52:0/4212205034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20c67c0 0x560cb20ce340 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff87c02c610 tx=0x7ff87c034660 comp rx=0 tx=0).stop 2024-04-02T13:54:10.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.759+0000 7ff895f73740 1 -- 172.21.15.52:0/4212205034 shutdown_connections 2024-04-02T13:54:10.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.759+0000 7ff895f73740 1 --2- 172.21.15.52:0/4212205034 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x560cb213dfc0 0x560cb21403b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.759+0000 7ff895f73740 1 --2- 172.21.15.52:0/4212205034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cb20c67c0 0x560cb20ce340 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.759+0000 7ff895f73740 1 --2- 172.21.15.52:0/4212205034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560cb20bdbe0 0x560cb20cb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.759+0000 7ff874ff9640 0 client.15924 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:10.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.759+0000 7ff895f73740 1 --2- 172.21.15.52:0/4212205034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb20bd2c0 0x560cb20cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.759+0000 7ff895f73740 1 -- 172.21.15.52:0/4212205034 >> 172.21.15.52:0/4212205034 conn(0x560cb20b79f0 msgr2=0x560cb20b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:10.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.759+0000 7ff895f73740 1 -- 172.21.15.52:0/4212205034 shutdown_connections 2024-04-02T13:54:10.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.759+0000 7ff895f73740 1 -- 172.21.15.52:0/4212205034 wait complete. 2024-04-02T13:54:10.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.759+0000 7ff895f73740 20 client.15924 trim_cache size 0 max 16384 2024-04-02T13:54:10.768 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:54:10.768 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-04-02T13:54:10.768 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "1773ec65-cb8b-48bd-b20c-a1e8fd5aaaf8", 2024-04-02T13:54:10.768 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-04-02T13:54:10.768 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:54:10.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.760+0000 7ff8932b1640 1 -- 172.21.15.52:0/4247388896 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff8780a3ed0 msgr2=0x7ff8780a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:10.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.760+0000 7ff8932b1640 1 --2- 172.21.15.52:0/4247388896 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff8780a3ed0 0x7ff8780a6390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff8840ce2a0 tx=0x7ff884053040 comp rx=0 tx=0).stop 2024-04-02T13:54:10.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.760+0000 7ff8932b1640 1 -- 172.21.15.52:0/4247388896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff88c106a50 msgr2=0x7ff88c19e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:10.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.760+0000 7ff8932b1640 1 --2- 172.21.15.52:0/4247388896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff88c106a50 0x7ff88c19e790 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff88400eaa0 tx=0x7ff88400ef60 comp rx=0 tx=0).stop 2024-04-02T13:54:10.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.760+0000 7ff8932b1640 1 -- 172.21.15.52:0/4247388896 shutdown_connections 2024-04-02T13:54:10.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.760+0000 7ff8932b1640 1 --2- 172.21.15.52:0/4247388896 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff8780a3ed0 0x7ff8780a6390 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.760+0000 7ff8932b1640 1 --2- 172.21.15.52:0/4247388896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff88c106a50 0x7ff88c19e790 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.760+0000 7ff8932b1640 1 --2- 172.21.15.52:0/4247388896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff88c1045c0 0x7ff88c1a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.760+0000 7ff8932b1640 1 --2- 172.21.15.52:0/4247388896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff88c069240 0x7ff88c1a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.760+0000 7ff8932b1640 1 -- 172.21.15.52:0/4247388896 >> 172.21.15.52:0/4247388896 conn(0x7ff88c077b00 msgr2=0x7ff88c101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:10.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.760+0000 7ff8932b1640 1 -- 172.21.15.52:0/4247388896 shutdown_connections 2024-04-02T13:54:10.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.760+0000 7ff8932b1640 1 -- 172.21.15.52:0/4247388896 wait complete. 2024-04-02T13:54:10.774 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:54:10.774 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:54:10.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.878+0000 7facb1915640 1 -- 172.21.15.52:0/4272853806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facac1069c0 msgr2=0x7facac106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:10.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.878+0000 7facb1915640 1 --2- 172.21.15.52:0/4272853806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facac1069c0 0x7facac106da0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009930 tx=0x7fac9c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:54:10.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.878+0000 7facb1915640 1 -- 172.21.15.52:0/4272853806 shutdown_connections 2024-04-02T13:54:10.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.878+0000 7facb1915640 1 --2- 172.21.15.52:0/4272853806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facac101370 0x7facac10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.878+0000 7facb1915640 1 --2- 172.21.15.52:0/4272853806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facac1009b0 0x7facac100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.878+0000 7facb1915640 1 --2- 172.21.15.52:0/4272853806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facac1069c0 0x7facac106da0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.878+0000 7facb1915640 1 -- 172.21.15.52:0/4272853806 >> 172.21.15.52:0/4272853806 conn(0x7facac076d80 msgr2=0x7facac077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:10.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.878+0000 7facb1915640 1 -- 172.21.15.52:0/4272853806 shutdown_connections 2024-04-02T13:54:10.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.878+0000 7facb1915640 1 -- 172.21.15.52:0/4272853806 wait complete. 2024-04-02T13:54:10.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.879+0000 7facb1915640 1 Processor -- start 2024-04-02T13:54:10.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.879+0000 7facb1915640 1 -- start start 2024-04-02T13:54:10.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.879+0000 7facb1915640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facac1009b0 0x7facac198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.879+0000 7facb1915640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facac101370 0x7facac1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.879+0000 7facb1915640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facac1069c0 0x7facac19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.879+0000 7facb1915640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7facac111e90 con 0x7facac101370 2024-04-02T13:54:10.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.879+0000 7facb1915640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7facac111d10 con 0x7facac1009b0 2024-04-02T13:54:10.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.879+0000 7facb1915640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7facac112010 con 0x7facac1069c0 2024-04-02T13:54:10.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.879+0000 7faca3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facac101370 0x7facac1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.879+0000 7faca3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facac101370 0x7facac1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33492/0 (socket says 172.21.15.52:33492) 2024-04-02T13:54:10.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.879+0000 7faca3fff640 1 -- 172.21.15.52:0/1567794970 learned_addr learned my addr 172.21.15.52:0/1567794970 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:10.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.879+0000 7facaaffd640 1 --2- 172.21.15.52:0/1567794970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facac1009b0 0x7facac198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.879+0000 7facab7fe640 1 --2- 172.21.15.52:0/1567794970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facac1069c0 0x7facac19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.879+0000 7faca3fff640 1 -- 172.21.15.52:0/1567794970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facac1069c0 msgr2=0x7facac19cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:10.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.879+0000 7faca3fff640 1 --2- 172.21.15.52:0/1567794970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facac1069c0 0x7facac19cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.879+0000 7faca3fff640 1 -- 172.21.15.52:0/1567794970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facac1009b0 msgr2=0x7facac198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:10.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.879+0000 7faca3fff640 1 --2- 172.21.15.52:0/1567794970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facac1009b0 0x7facac198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.879+0000 7faca3fff640 1 -- 172.21.15.52:0/1567794970 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facac19d360 con 0x7facac101370 2024-04-02T13:54:10.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.879+0000 7facaaffd640 1 --2- 172.21.15.52:0/1567794970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facac1009b0 0x7facac198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:54:10.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.879+0000 7facab7fe640 1 --2- 172.21.15.52:0/1567794970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facac1069c0 0x7facac19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:54:10.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.879+0000 7faca3fff640 1 --2- 172.21.15.52:0/1567794970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facac101370 0x7facac1988b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fac9400c890 tx=0x7fac9400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:10.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.880+0000 7facb0913640 1 -- 172.21.15.52:0/1567794970 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac94004220 con 0x7facac101370 2024-04-02T13:54:10.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.880+0000 7facb0913640 1 -- 172.21.15.52:0/1567794970 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac940043c0 con 0x7facac101370 2024-04-02T13:54:10.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.880+0000 7facb1915640 1 -- 172.21.15.52:0/1567794970 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facac19d650 con 0x7facac101370 2024-04-02T13:54:10.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.880+0000 7facb0913640 1 -- 172.21.15.52:0/1567794970 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac940049e0 con 0x7facac101370 2024-04-02T13:54:10.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.880+0000 7facb1915640 1 -- 172.21.15.52:0/1567794970 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facac1a4ee0 con 0x7facac101370 2024-04-02T13:54:10.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.882+0000 7facb0913640 1 -- 172.21.15.52:0/1567794970 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fac940079b0 con 0x7facac101370 2024-04-02T13:54:10.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.883+0000 7facb45db740 1 Processor -- start 2024-04-02T13:54:10.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.883+0000 7facb45db740 1 -- start start 2024-04-02T13:54:10.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.883+0000 7facb45db740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557999d00600 0x557999d009e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.883+0000 7facb45db740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d00f20 0x557999d0dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.883+0000 7facb45db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557999d0e1e0 0x557999d105d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.883+0000 7facb45db740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557999c64830 con 0x557999d00f20 2024-04-02T13:54:10.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.883+0000 7facb45db740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557999c0c820 con 0x557999d00600 2024-04-02T13:54:10.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.883+0000 7facb45db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557999c607e0 con 0x557999d0e1e0 2024-04-02T13:54:10.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.883+0000 7facaaffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d00f20 0x557999d0dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.883+0000 7facb0913640 1 --2- 172.21.15.52:0/1567794970 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fac980a3ed0 0x7fac980a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.883+0000 7facaaffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d00f20 0x557999d0dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33496/0 (socket says 172.21.15.52:33496) 2024-04-02T13:54:10.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.883+0000 7facaaffd640 1 -- 172.21.15.52:0/3576991710 learned_addr learned my addr 172.21.15.52:0/3576991710 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:10.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.883+0000 7facab7fe640 1 --2- 172.21.15.52:0/1567794970 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fac980a3ed0 0x7fac980a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.883+0000 7facab7fe640 1 --2- 172.21.15.52:0/3576991710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557999d00600 0x557999d009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.883+0000 7facb0913640 1 -- 172.21.15.52:0/1567794970 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fac940d0fe0 con 0x7facac101370 2024-04-02T13:54:10.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.883+0000 7faca3fff640 1 --2- 172.21.15.52:0/3576991710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557999d0e1e0 0x557999d105d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.883+0000 7facaaffd640 1 -- 172.21.15.52:0/3576991710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557999d0e1e0 msgr2=0x557999d105d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:10.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.883+0000 7facaaffd640 1 --2- 172.21.15.52:0/3576991710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557999d0e1e0 0x557999d105d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.883+0000 7facaaffd640 1 -- 172.21.15.52:0/3576991710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557999d00600 msgr2=0x557999d009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:10.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.883+0000 7facaaffd640 1 --2- 172.21.15.52:0/3576991710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557999d00600 0x557999d009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.883+0000 7facaaffd640 1 -- 172.21.15.52:0/3576991710 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557999a427d0 con 0x557999d00f20 2024-04-02T13:54:10.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.883+0000 7facab7fe640 1 --2- 172.21.15.52:0/3576991710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557999d00600 0x557999d009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:54:10.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.883+0000 7faca3fff640 1 --2- 172.21.15.52:0/3576991710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557999d0e1e0 0x557999d105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:54:10.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.884+0000 7facaaffd640 1 --2- 172.21.15.52:0/3576991710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d00f20 0x557999d0dca0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fac9c036040 tx=0x7fac9c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=289faa68f25baa52 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:10.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.884+0000 7facab7fe640 1 --2- 172.21.15.52:0/1567794970 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fac980a3ed0 0x7fac980a6390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faca400af30 tx=0x7faca400a360 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:10.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.884+0000 7faca1ffb640 1 -- 172.21.15.52:0/3576991710 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c02cbc0 con 0x557999d00f20 2024-04-02T13:54:10.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.884+0000 7faca1ffb640 1 -- 172.21.15.52:0/3576991710 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac9c005040 con 0x557999d00f20 2024-04-02T13:54:10.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.884+0000 7faca1ffb640 1 -- 172.21.15.52:0/3576991710 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c004a40 con 0x557999d00f20 2024-04-02T13:54:10.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.884+0000 7facb45db740 1 -- 172.21.15.52:0/3576991710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d00f20 msgr2=0x557999d0dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:10.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.884+0000 7facb45db740 1 --2- 172.21.15.52:0/3576991710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d00f20 0x557999d0dca0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fac9c036040 tx=0x7fac9c02c650 comp rx=0 tx=0).stop 2024-04-02T13:54:10.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.884+0000 7facb45db740 1 -- 172.21.15.52:0/3576991710 shutdown_connections 2024-04-02T13:54:10.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.884+0000 7facb45db740 1 --2- 172.21.15.52:0/3576991710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557999d0e1e0 0x557999d105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.884+0000 7facb45db740 1 --2- 172.21.15.52:0/3576991710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d00f20 0x557999d0dca0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.884+0000 7facb45db740 1 --2- 172.21.15.52:0/3576991710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557999d00600 0x557999d009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.884+0000 7facb45db740 1 -- 172.21.15.52:0/3576991710 >> 172.21.15.52:0/3576991710 conn(0x557999cfb000 msgr2=0x557999cfb3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:10.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.884+0000 7facb45db740 1 -- 172.21.15.52:0/3576991710 shutdown_connections 2024-04-02T13:54:10.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.884+0000 7facb45db740 1 -- 172.21.15.52:0/3576991710 wait complete. 2024-04-02T13:54:10.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.885+0000 7facb45db740 1 -- start start 2024-04-02T13:54:10.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.885+0000 7facb45db740 1 Processor -- start 2024-04-02T13:54:10.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.885+0000 7facb45db740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557999d00600 0x557999d0d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.885+0000 7facb45db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557999d00f20 0x557999d0ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.885+0000 7facb45db740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d0e1e0 0x557999d0b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.885+0000 7facb45db740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557999d10f40 con 0x557999d0e1e0 2024-04-02T13:54:10.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.885+0000 7facb45db740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557999c1fd30 con 0x557999d00600 2024-04-02T13:54:10.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.885+0000 7facb45db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557999d110c0 con 0x557999d00f20 2024-04-02T13:54:10.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.885+0000 7facb45db740 10 client.0 _ll_register_callbacks cb 0x557999c1f720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:10.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.885+0000 7facab7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d0e1e0 0x557999d0b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.885+0000 7facab7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d0e1e0 0x557999d0b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33512/0 (socket says 172.21.15.52:33512) 2024-04-02T13:54:10.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.885+0000 7facab7fe640 1 -- 172.21.15.52:0/3699022433 learned_addr learned my addr 172.21.15.52:0/3699022433 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:10.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.885+0000 7faca3fff640 1 --2- 172.21.15.52:0/3699022433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557999d00f20 0x557999d0ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.886+0000 7facaaffd640 1 --2- 172.21.15.52:0/3699022433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557999d00600 0x557999d0d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.886+0000 7facab7fe640 1 -- 172.21.15.52:0/3699022433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557999d00f20 msgr2=0x557999d0ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:10.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.886+0000 7facab7fe640 1 --2- 172.21.15.52:0/3699022433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557999d00f20 0x557999d0ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.886+0000 7facab7fe640 1 -- 172.21.15.52:0/3699022433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557999d00600 msgr2=0x557999d0d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:10.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.886+0000 7facab7fe640 1 --2- 172.21.15.52:0/3699022433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557999d00600 0x557999d0d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.886+0000 7facab7fe640 1 -- 172.21.15.52:0/3699022433 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579999783f0 con 0x557999d0e1e0 2024-04-02T13:54:10.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.886+0000 7facaaffd640 1 --2- 172.21.15.52:0/3699022433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557999d00600 0x557999d0d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:54:10.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.886+0000 7facab7fe640 1 --2- 172.21.15.52:0/3699022433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d0e1e0 0x557999d0b260 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7faca400ad60 tx=0x7faca4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73bb932870b293d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:10.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.886+0000 7fac8affd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:10.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.886+0000 7fac8affd640 1 -- 172.21.15.52:0/3699022433 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca4021070 con 0x557999d0e1e0 2024-04-02T13:54:10.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.886+0000 7fac8affd640 1 -- 172.21.15.52:0/3699022433 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7faca401d450 con 0x557999d0e1e0 2024-04-02T13:54:10.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.886+0000 7facb45db740 1 -- 172.21.15.52:0/3699022433 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac9c005040 con 0x557999d0e1e0 2024-04-02T13:54:10.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.886+0000 7fac8affd640 1 -- 172.21.15.52:0/3699022433 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faca402d050 con 0x557999d0e1e0 2024-04-02T13:54:10.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.886+0000 7facb45db740 10 client.15936 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:10.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.886+0000 7facb45db740 1 -- 172.21.15.52:0/3699022433 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5579999380f0 con 0x557999d0e1e0 2024-04-02T13:54:10.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.886+0000 7fac8affd640 1 -- 172.21.15.52:0/3699022433 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac9c005040 con 0x557999d0e1e0 2024-04-02T13:54:10.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.886+0000 7fac8affd640 1 -- 172.21.15.52:0/3699022433 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faca401a040 con 0x557999d0e1e0 2024-04-02T13:54:10.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.887+0000 7facb45db740 10 client.15936 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:10.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.887+0000 7facb45db740 10 client.15936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:54:10.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.887+0000 7facb45db740 20 client.15936 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:10.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.887+0000 7facb45db740 1 --2- 172.21.15.52:0/3699022433 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557999d1df30 0x557999d203b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.887+0000 7facb45db740 4 client.15936 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:54:10.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.887+0000 7facb45db740 1 -- 172.21.15.52:0/3699022433 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557999c71d70 con 0x557999d1df30 2024-04-02T13:54:10.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.887+0000 7facaaffd640 1 --2- 172.21.15.52:0/3699022433 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557999d1df30 0x557999d203b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.887+0000 7fac8affd640 10 client.15936 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:10.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.887+0000 7facaaffd640 1 --2- 172.21.15.52:0/3699022433 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557999d1df30 0x557999d203b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=264eb6c9da4a9d56 server_cookie=b1ca16fdce3162fb in_seq=0 out_seq=0 2024-04-02T13:54:10.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.888+0000 7fac8affd640 1 -- 172.21.15.52:0/3699022433 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557999c71d70 con 0x557999d1df30 2024-04-02T13:54:10.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.888+0000 7fac8affd640 10 client.15936 handle_command_reply: tid=0 2024-04-02T13:54:10.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.888+0000 7facb45db740 1 client.15936 shutdown 2024-04-02T13:54:10.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.888+0000 7facb45db740 1 -- 172.21.15.52:0/3699022433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d0e1e0 msgr2=0x557999d0b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:10.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.888+0000 7facb45db740 1 --2- 172.21.15.52:0/3699022433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d0e1e0 0x557999d0b260 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7faca400ad60 tx=0x7faca4004ab0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.888+0000 7facb45db740 1 -- 172.21.15.52:0/3699022433 shutdown_connections 2024-04-02T13:54:10.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.888+0000 7facb45db740 1 --2- 172.21.15.52:0/3699022433 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557999d1df30 0x557999d203b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.888+0000 7facb45db740 1 --2- 172.21.15.52:0/3699022433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d0e1e0 0x557999d0b260 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.888+0000 7facb45db740 1 --2- 172.21.15.52:0/3699022433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557999d00f20 0x557999d0ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.888+0000 7fac8affd640 0 client.15936 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:10.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.888+0000 7facb45db740 1 --2- 172.21.15.52:0/3699022433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557999d00600 0x557999d0d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.888+0000 7facb45db740 1 -- 172.21.15.52:0/3699022433 >> 172.21.15.52:0/3699022433 conn(0x557999cfb000 msgr2=0x557999cfcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:10.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.888+0000 7facb45db740 1 -- 172.21.15.52:0/3699022433 shutdown_connections 2024-04-02T13:54:10.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.888+0000 7facb45db740 1 -- 172.21.15.52:0/3699022433 wait complete. 2024-04-02T13:54:10.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.888+0000 7facb45db740 20 client.15936 trim_cache size 0 max 16384 2024-04-02T13:54:10.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.906+0000 7facb45db740 1 Processor -- start 2024-04-02T13:54:10.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.906+0000 7facb45db740 1 -- start start 2024-04-02T13:54:10.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.906+0000 7facb45db740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d00600 0x557999d19e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.906+0000 7facb45db740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557999d00f20 0x557999d06350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.906+0000 7facb45db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557999d0e1e0 0x557999d25c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.906+0000 7facb45db740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7facac1acce0 con 0x557999d00600 2024-04-02T13:54:10.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.906+0000 7facb45db740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7facac1ae820 con 0x557999d00f20 2024-04-02T13:54:10.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.906+0000 7facb45db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7facac1ad3b0 con 0x557999d0e1e0 2024-04-02T13:54:10.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.906+0000 7facaaffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d00600 0x557999d19e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.906+0000 7facaaffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d00600 0x557999d19e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33522/0 (socket says 172.21.15.52:33522) 2024-04-02T13:54:10.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.906+0000 7facaaffd640 1 -- 172.21.15.52:0/4256231349 learned_addr learned my addr 172.21.15.52:0/4256231349 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:10.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.906+0000 7faca3fff640 1 --2- 172.21.15.52:0/4256231349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557999d00f20 0x557999d06350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.906+0000 7facaaffd640 1 -- 172.21.15.52:0/4256231349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557999d0e1e0 msgr2=0x557999d25c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:10.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.906+0000 7facaaffd640 1 --2- 172.21.15.52:0/4256231349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557999d0e1e0 0x557999d25c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.906+0000 7facaaffd640 1 -- 172.21.15.52:0/4256231349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557999d00f20 msgr2=0x557999d06350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:10.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.906+0000 7facaaffd640 1 --2- 172.21.15.52:0/4256231349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557999d00f20 0x557999d06350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.906+0000 7facaaffd640 1 -- 172.21.15.52:0/4256231349 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557999cfae30 con 0x557999d00600 2024-04-02T13:54:10.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.906+0000 7facaaffd640 1 --2- 172.21.15.52:0/4256231349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d00600 0x557999d19e40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fac9c036040 tx=0x7fac9c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4984576bfabf1d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:10.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.906+0000 7faca17fa640 1 -- 172.21.15.52:0/4256231349 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c04c070 con 0x557999d00600 2024-04-02T13:54:10.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.906+0000 7faca17fa640 1 -- 172.21.15.52:0/4256231349 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac9c04d2b0 con 0x557999d00600 2024-04-02T13:54:10.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.906+0000 7faca17fa640 1 -- 172.21.15.52:0/4256231349 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c003850 con 0x557999d00600 2024-04-02T13:54:10.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.907+0000 7facb45db740 1 -- 172.21.15.52:0/4256231349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d00600 msgr2=0x557999d19e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:10.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.907+0000 7facb45db740 1 --2- 172.21.15.52:0/4256231349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d00600 0x557999d19e40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fac9c036040 tx=0x7fac9c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:54:10.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.907+0000 7facb45db740 1 -- 172.21.15.52:0/4256231349 shutdown_connections 2024-04-02T13:54:10.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.907+0000 7facb45db740 1 --2- 172.21.15.52:0/4256231349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557999d0e1e0 0x557999d25c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.907+0000 7facb45db740 1 --2- 172.21.15.52:0/4256231349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557999d00f20 0x557999d06350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.907+0000 7facb45db740 1 --2- 172.21.15.52:0/4256231349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d00600 0x557999d19e40 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.907+0000 7facb45db740 1 -- 172.21.15.52:0/4256231349 >> 172.21.15.52:0/4256231349 conn(0x557999cfb000 msgr2=0x557999d05c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:10.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.907+0000 7facb45db740 1 -- 172.21.15.52:0/4256231349 shutdown_connections 2024-04-02T13:54:10.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.907+0000 7facb45db740 1 -- 172.21.15.52:0/4256231349 wait complete. 2024-04-02T13:54:10.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.907+0000 7facb45db740 1 -- start start 2024-04-02T13:54:10.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.908+0000 7facb45db740 1 Processor -- start 2024-04-02T13:54:10.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.908+0000 7facb45db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557999d00600 0x557999d07930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.908+0000 7facb45db740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d00f20 0x557999d07e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.908+0000 7facb45db740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557999d0e1e0 0x557999d083b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.908+0000 7facb45db740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7facac1ae150 con 0x557999d00f20 2024-04-02T13:54:10.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.908+0000 7facb45db740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7facac1ada80 con 0x557999d0e1e0 2024-04-02T13:54:10.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.908+0000 7facb45db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7facac1aacc0 con 0x557999d00600 2024-04-02T13:54:10.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.908+0000 7facb45db740 10 client.0 _ll_register_callbacks cb 0x7faca40117b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:10.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.908+0000 7faca3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d00f20 0x557999d07e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.908+0000 7faca3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d00f20 0x557999d07e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33530/0 (socket says 172.21.15.52:33530) 2024-04-02T13:54:10.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.908+0000 7faca3fff640 1 -- 172.21.15.52:0/1568698262 learned_addr learned my addr 172.21.15.52:0/1568698262 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:10.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.908+0000 7facab7fe640 1 --2- 172.21.15.52:0/1568698262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557999d0e1e0 0x557999d083b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.908+0000 7facaaffd640 1 --2- 172.21.15.52:0/1568698262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557999d00600 0x557999d07930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.908+0000 7faca3fff640 1 -- 172.21.15.52:0/1568698262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557999d00600 msgr2=0x557999d07930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:10.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.908+0000 7faca3fff640 1 --2- 172.21.15.52:0/1568698262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557999d00600 0x557999d07930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.908+0000 7faca3fff640 1 -- 172.21.15.52:0/1568698262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557999d0e1e0 msgr2=0x557999d083b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:10.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.908+0000 7faca3fff640 1 --2- 172.21.15.52:0/1568698262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557999d0e1e0 0x557999d083b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.908+0000 7faca3fff640 1 -- 172.21.15.52:0/1568698262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557999d834f0 con 0x557999d00f20 2024-04-02T13:54:10.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.908+0000 7facaaffd640 1 --2- 172.21.15.52:0/1568698262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557999d00600 0x557999d07930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:54:10.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.908+0000 7faca3fff640 1 --2- 172.21.15.52:0/1568698262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d00f20 0x557999d07e70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fac94007590 tx=0x7fac94005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13756b74b625ea3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:10.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.908+0000 7fac8affd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:10.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.908+0000 7fac8affd640 1 -- 172.21.15.52:0/1568698262 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac940a9020 con 0x557999d00f20 2024-04-02T13:54:10.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.908+0000 7fac8affd640 1 -- 172.21.15.52:0/1568698262 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fac9409cd80 con 0x557999d00f20 2024-04-02T13:54:10.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.908+0000 7facb45db740 1 -- 172.21.15.52:0/1568698262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac9c04d2b0 con 0x557999d00f20 2024-04-02T13:54:10.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.909+0000 7fac8affd640 1 -- 172.21.15.52:0/1568698262 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fac94085e60 con 0x557999d00f20 2024-04-02T13:54:10.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.909+0000 7facb45db740 10 client.15942 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:10.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.909+0000 7facb45db740 1 -- 172.21.15.52:0/1568698262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557999d84810 con 0x557999d00f20 2024-04-02T13:54:10.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.909+0000 7fac8affd640 1 -- 172.21.15.52:0/1568698262 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac9c04d2b0 con 0x557999d00f20 2024-04-02T13:54:10.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.909+0000 7fac8affd640 1 -- 172.21.15.52:0/1568698262 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fac940ceb40 con 0x557999d00f20 2024-04-02T13:54:10.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.909+0000 7facb45db740 10 client.15942 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:10.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.909+0000 7facb45db740 10 client.15942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:54:10.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.909+0000 7facb45db740 20 client.15942 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:10.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.909+0000 7facb45db740 1 --2- 172.21.15.52:0/1568698262 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557999d85030 0x557999d87450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:10.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.909+0000 7facb45db740 4 client.15942 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:54:10.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.909+0000 7facb45db740 1 -- 172.21.15.52:0/1568698262 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557999c6c0c0 con 0x557999d85030 2024-04-02T13:54:10.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.909+0000 7facaaffd640 1 --2- 172.21.15.52:0/1568698262 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557999d85030 0x557999d87450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:10.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.910+0000 7facaaffd640 1 --2- 172.21.15.52:0/1568698262 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557999d85030 0x557999d87450 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd662edcc9881679 server_cookie=b48788f10a43af08 in_seq=0 out_seq=0 2024-04-02T13:54:10.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.910+0000 7fac8affd640 10 client.15942 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:10.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.910+0000 7fac8affd640 1 -- 172.21.15.52:0/1568698262 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557999c6c0c0 con 0x557999d85030 2024-04-02T13:54:10.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.910+0000 7fac8affd640 10 client.15942 handle_command_reply: tid=0 2024-04-02T13:54:10.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.910+0000 7facb45db740 1 client.15942 shutdown 2024-04-02T13:54:10.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.910+0000 7facb45db740 1 -- 172.21.15.52:0/1568698262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d00f20 msgr2=0x557999d07e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:10.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.910+0000 7facb45db740 1 --2- 172.21.15.52:0/1568698262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d00f20 0x557999d07e70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fac94007590 tx=0x7fac94005300 comp rx=0 tx=0).stop 2024-04-02T13:54:10.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.910+0000 7facb45db740 1 -- 172.21.15.52:0/1568698262 shutdown_connections 2024-04-02T13:54:10.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.910+0000 7facb45db740 1 --2- 172.21.15.52:0/1568698262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557999d0e1e0 0x557999d083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.910+0000 7facb45db740 1 --2- 172.21.15.52:0/1568698262 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557999d85030 0x557999d87450 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.910+0000 7facb45db740 1 --2- 172.21.15.52:0/1568698262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557999d00f20 0x557999d07e70 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.910+0000 7facb45db740 1 --2- 172.21.15.52:0/1568698262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557999d00600 0x557999d07930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.910+0000 7facb45db740 1 -- 172.21.15.52:0/1568698262 >> 172.21.15.52:0/1568698262 conn(0x557999cfb000 msgr2=0x557999cfa240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:10.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.910+0000 7fac8affd640 0 client.15942 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:10.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.910+0000 7facb45db740 1 -- 172.21.15.52:0/1568698262 shutdown_connections 2024-04-02T13:54:10.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.910+0000 7facb45db740 1 -- 172.21.15.52:0/1568698262 wait complete. 2024-04-02T13:54:10.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.910+0000 7facb45db740 20 client.15942 trim_cache size 0 max 16384 2024-04-02T13:54:10.921 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:54:10.921 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:54:10.921 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-04-02T13:54:10.921 INFO:teuthology.orchestra.run.smithi052.stdout: "1773ec65-cb8b-48bd-b20c-a1e8fd5aaaf8": { 2024-04-02T13:54:10.921 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-04-02T13:54:10.921 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "1773ec65-cb8b-48bd-b20c-a1e8fd5aaaf8", 2024-04-02T13:54:10.921 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-04-02T13:54:10.921 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:54:10.921 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:54:10.921 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:54:10.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.911+0000 7facb1915640 1 -- 172.21.15.52:0/1567794970 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fac980a3ed0 msgr2=0x7fac980a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:10.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.911+0000 7facb1915640 1 --2- 172.21.15.52:0/1567794970 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fac980a3ed0 0x7fac980a6390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faca400af30 tx=0x7faca400a360 comp rx=0 tx=0).stop 2024-04-02T13:54:10.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.911+0000 7facb1915640 1 -- 172.21.15.52:0/1567794970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facac101370 msgr2=0x7facac1988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:10.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.911+0000 7facb1915640 1 --2- 172.21.15.52:0/1567794970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facac101370 0x7facac1988b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fac9400c890 tx=0x7fac9400cd50 comp rx=0 tx=0).stop 2024-04-02T13:54:10.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.911+0000 7facb1915640 1 -- 172.21.15.52:0/1567794970 shutdown_connections 2024-04-02T13:54:10.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.911+0000 7facb1915640 1 --2- 172.21.15.52:0/1567794970 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fac980a3ed0 0x7fac980a6390 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.911+0000 7facb1915640 1 --2- 172.21.15.52:0/1567794970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facac1069c0 0x7facac19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.911+0000 7facb1915640 1 --2- 172.21.15.52:0/1567794970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facac101370 0x7facac1988b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.911+0000 7facb1915640 1 --2- 172.21.15.52:0/1567794970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facac1009b0 0x7facac198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:10.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.911+0000 7facb1915640 1 -- 172.21.15.52:0/1567794970 >> 172.21.15.52:0/1567794970 conn(0x7facac076d80 msgr2=0x7facac0fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:10.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.911+0000 7facb1915640 1 -- 172.21.15.52:0/1567794970 shutdown_connections 2024-04-02T13:54:10.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:10.911+0000 7facb1915640 1 -- 172.21.15.52:0/1567794970 wait complete. 2024-04-02T13:54:10.926 INFO:tasks.cephfs.filesystem:scrub status for tag:1773ec65-cb8b-48bd-b20c-a1e8fd5aaaf8 - {'path': '/', 'tag': '1773ec65-cb8b-48bd-b20c-a1e8fd5aaaf8', 'options': 'recursive,force'} 2024-04-02T13:54:11.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:10 smithi189 ceph-mon[36683]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-02T13:54:11.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:10 smithi052 ceph-mon[32104]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-02T13:54:11.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:10 smithi088 ceph-mon[36528]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-02T13:54:12.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:11 smithi189 ceph-mon[36683]: scrub queued for path: / 2024-04-02T13:54:12.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:11 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:12.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:11 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:54:12.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:11 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:12.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:11 smithi052 ceph-mon[32104]: scrub queued for path: / 2024-04-02T13:54:12.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:11 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:12.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:11 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:54:12.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:11 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:12.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:11 smithi088 ceph-mon[36528]: scrub queued for path: / 2024-04-02T13:54:12.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:11 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:12.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:11 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:54:12.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:11 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:13.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:12 smithi189 ceph-mon[36683]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-02T13:54:13.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:12 smithi052 ceph-mon[32104]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-02T13:54:13.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:12 smithi088 ceph-mon[36528]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-02T13:54:15.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:14 smithi189 ceph-mon[36683]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-02T13:54:15.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:14 smithi052 ceph-mon[32104]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-02T13:54:15.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:14 smithi088 ceph-mon[36528]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-02T13:54:16.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:16 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T13:54:16.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:15 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T13:54:16.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:15 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T13:54:17.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:17 smithi189 ceph-mon[36683]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-04-02T13:54:17.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:17 smithi052 ceph-mon[32104]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-04-02T13:54:17.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:17 smithi088 ceph-mon[36528]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-04-02T13:54:19.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:19 smithi189 ceph-mon[36683]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-02T13:54:19.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:19 smithi052 ceph-mon[32104]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-02T13:54:19.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:19 smithi088 ceph-mon[36528]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-02T13:54:21.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:21 smithi189 ceph-mon[36683]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-02T13:54:21.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:21 smithi052 ceph-mon[32104]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-02T13:54:21.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:21 smithi088 ceph-mon[36528]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-02T13:54:23.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:23 smithi189 ceph-mon[36683]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-04-02T13:54:23.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:23 smithi052 ceph-mon[32104]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-04-02T13:54:23.422 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:23 smithi088 ceph-mon[36528]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 938 B/s wr, 13 op/s 2024-04-02T13:54:25.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:25 smithi189 ceph-mon[36683]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-02T13:54:25.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:25 smithi052 ceph-mon[32104]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-02T13:54:25.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:25 smithi088 ceph-mon[36528]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 8 op/s 2024-04-02T13:54:27.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:27 smithi189 ceph-mon[36683]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-04-02T13:54:27.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:27 smithi052 ceph-mon[32104]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-04-02T13:54:27.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:27 smithi088 ceph-mon[36528]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-04-02T13:54:29.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:29 smithi189 ceph-mon[36683]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:54:29.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:29 smithi052 ceph-mon[32104]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:54:29.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:29 smithi088 ceph-mon[36528]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:54:31.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:31 smithi189 ceph-mon[36683]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:54:31.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:31 smithi052 ceph-mon[32104]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:54:31.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:31 smithi088 ceph-mon[36528]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:54:33.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:33 smithi189 ceph-mon[36683]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:54:33.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:33 smithi052 ceph-mon[32104]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:54:33.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:33 smithi088 ceph-mon[36528]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:54:35.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:35 smithi189 ceph-mon[36683]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:54:35.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:35 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:54:35.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:35 smithi052 ceph-mon[32104]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:54:35.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:35 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:54:35.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:35 smithi088 ceph-mon[36528]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:54:35.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:35 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:54:36.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:36 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:54:36.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:36 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:54:36.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:36 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:54:36.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:36 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:54:36.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:36 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:54:36.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:36 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:54:36.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:36 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:54:36.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:36 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:54:36.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:36 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:54:36.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:36 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:54:36.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:36 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:54:36.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:36 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:54:36.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:36 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:54:36.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:36 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:54:36.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:36 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:54:36.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:36 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:54:36.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:36 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:54:36.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:36 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:54:36.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:36 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:54:36.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:36 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:54:36.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:36 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:54:36.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:36 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:54:36.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:36 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:54:36.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:36 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:54:36.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:36 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:54:36.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:36 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:54:36.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:36 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:54:37.297 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:37 smithi052 ceph-mon[32104]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:54:37.571 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:37 smithi088 ceph-mon[36528]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:54:37.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:37 smithi189 ceph-mon[36683]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:54:39.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:39 smithi052 ceph-mon[32104]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:54:39.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:39 smithi088 ceph-mon[36528]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:54:39.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:39 smithi189 ceph-mon[36683]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:54:40.927 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:54:40.928 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:54:41.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.035+0000 7fb6cbc0d640 1 -- 172.21.15.52:0/1389624538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6c4107b80 msgr2=0x7fb6c4107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.035+0000 7fb6cbc0d640 1 --2- 172.21.15.52:0/1389624538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6c4107b80 0x7fb6c4107f60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0009900 tx=0x7fb6c002bcd0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.035+0000 7fb6cbc0d640 1 -- 172.21.15.52:0/1389624538 shutdown_connections 2024-04-02T13:54:41.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.035+0000 7fb6cbc0d640 1 --2- 172.21.15.52:0/1389624538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c406a860 0x7fb6c410e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.035+0000 7fb6cbc0d640 1 --2- 172.21.15.52:0/1389624538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6c4069ec0 0x7fb6c406a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.035+0000 7fb6cbc0d640 1 --2- 172.21.15.52:0/1389624538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6c4107b80 0x7fb6c4107f60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.035+0000 7fb6cbc0d640 1 -- 172.21.15.52:0/1389624538 >> 172.21.15.52:0/1389624538 conn(0x7fb6c4077a90 msgr2=0x7fb6c4077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:41.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.035+0000 7fb6cbc0d640 1 -- 172.21.15.52:0/1389624538 shutdown_connections 2024-04-02T13:54:41.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.035+0000 7fb6cbc0d640 1 -- 172.21.15.52:0/1389624538 wait complete. 2024-04-02T13:54:41.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.035+0000 7fb6cbc0d640 1 Processor -- start 2024-04-02T13:54:41.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.035+0000 7fb6cbc0d640 1 -- start start 2024-04-02T13:54:41.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.036+0000 7fb6cbc0d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6c4069ec0 0x7fb6c419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.036+0000 7fb6cbc0d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c406a860 0x7fb6c419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.036+0000 7fb6cbc0d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6c4107b80 0x7fb6c419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.036+0000 7fb6cbc0d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c4111eb0 con 0x7fb6c4069ec0 2024-04-02T13:54:41.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.036+0000 7fb6cbc0d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c4111d30 con 0x7fb6c4107b80 2024-04-02T13:54:41.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.036+0000 7fb6cbc0d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c4112030 con 0x7fb6c406a860 2024-04-02T13:54:41.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.036+0000 7fb6c99a9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6c4069ec0 0x7fb6c419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.036+0000 7fb6c99a9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6c4069ec0 0x7fb6c419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42826/0 (socket says 172.21.15.52:42826) 2024-04-02T13:54:41.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.036+0000 7fb6c99a9640 1 -- 172.21.15.52:0/2693811116 learned_addr learned my addr 172.21.15.52:0/2693811116 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:41.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.036+0000 7fb6ca1aa640 1 --2- 172.21.15.52:0/2693811116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6c4107b80 0x7fb6c419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.036+0000 7fb6c91a8640 1 --2- 172.21.15.52:0/2693811116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c406a860 0x7fb6c419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.036+0000 7fb6c99a9640 1 -- 172.21.15.52:0/2693811116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c406a860 msgr2=0x7fb6c419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.036+0000 7fb6c99a9640 1 --2- 172.21.15.52:0/2693811116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c406a860 0x7fb6c419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.036+0000 7fb6c99a9640 1 -- 172.21.15.52:0/2693811116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6c4107b80 msgr2=0x7fb6c419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.036+0000 7fb6c99a9640 1 --2- 172.21.15.52:0/2693811116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6c4107b80 0x7fb6c419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.036+0000 7fb6c99a9640 1 -- 172.21.15.52:0/2693811116 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c419f5a0 con 0x7fb6c4069ec0 2024-04-02T13:54:41.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.036+0000 7fb6ca1aa640 1 --2- 172.21.15.52:0/2693811116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6c4107b80 0x7fb6c419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:54:41.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.036+0000 7fb6c99a9640 1 --2- 172.21.15.52:0/2693811116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6c4069ec0 0x7fb6c419a5b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0005940 tx=0x7fb6c002cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:41.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.037+0000 7fb6baffd640 1 -- 172.21.15.52:0/2693811116 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0041070 con 0x7fb6c4069ec0 2024-04-02T13:54:41.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.037+0000 7fb6cbc0d640 1 -- 172.21.15.52:0/2693811116 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c419f830 con 0x7fb6c4069ec0 2024-04-02T13:54:41.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.037+0000 7fb6cbc0d640 1 -- 172.21.15.52:0/2693811116 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c41a7120 con 0x7fb6c4069ec0 2024-04-02T13:54:41.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.037+0000 7fb6baffd640 1 -- 172.21.15.52:0/2693811116 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6c0004450 con 0x7fb6c4069ec0 2024-04-02T13:54:41.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.037+0000 7fb6baffd640 1 -- 172.21.15.52:0/2693811116 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0034440 con 0x7fb6c4069ec0 2024-04-02T13:54:41.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.039+0000 7fb6baffd640 1 -- 172.21.15.52:0/2693811116 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fb6c0044070 con 0x7fb6c4069ec0 2024-04-02T13:54:41.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.040+0000 7fb6ce7f4740 1 Processor -- start 2024-04-02T13:54:41.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.040+0000 7fb6ce7f4740 1 -- start start 2024-04-02T13:54:41.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.040+0000 7fb6baffd640 1 --2- 172.21.15.52:0/2693811116 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb6b00a3f50 0x7fb6b00a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.040+0000 7fb6ce7f4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104df8d0 0x5636104dfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.040+0000 7fb6ce7f4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636104e01f0 0x5636104e8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.040+0000 7fb6ce7f4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636104e8dd0 0x5636104eb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.040+0000 7fb6ce7f4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563610443830 con 0x5636104df8d0 2024-04-02T13:54:41.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.040+0000 7fb6ce7f4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5636103eb820 con 0x5636104e8dd0 2024-04-02T13:54:41.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.040+0000 7fb6ce7f4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56361043f7e0 con 0x5636104e01f0 2024-04-02T13:54:41.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.041+0000 7fb6c99a9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104df8d0 0x5636104dfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.041+0000 7fb6c99a9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104df8d0 0x5636104dfcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42838/0 (socket says 172.21.15.52:42838) 2024-04-02T13:54:41.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.041+0000 7fb6c99a9640 1 -- 172.21.15.52:0/1991564029 learned_addr learned my addr 172.21.15.52:0/1991564029 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:41.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.041+0000 7fb6ca1aa640 1 --2- 172.21.15.52:0/2693811116 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb6b00a3f50 0x7fb6b00a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.041+0000 7fb6c91a8640 1 --2- 172.21.15.52:0/1991564029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636104e01f0 0x5636104e8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.041+0000 7fb6baffd640 1 -- 172.21.15.52:0/2693811116 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb6c00f2980 con 0x7fb6c4069ec0 2024-04-02T13:54:41.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.041+0000 7fb6c99a9640 1 -- 172.21.15.52:0/1991564029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636104e01f0 msgr2=0x5636104e8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:41.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.041+0000 7fb6c99a9640 1 --2- 172.21.15.52:0/1991564029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636104e01f0 0x5636104e8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.041+0000 7fb6c99a9640 1 -- 172.21.15.52:0/1991564029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636104e8dd0 msgr2=0x5636104eb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:41.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.041+0000 7fb6c99a9640 1 --2- 172.21.15.52:0/1991564029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636104e8dd0 0x5636104eb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.041+0000 7fb6c99a9640 1 -- 172.21.15.52:0/1991564029 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636102217d0 con 0x5636104df8d0 2024-04-02T13:54:41.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.041+0000 7fb6c99a9640 1 --2- 172.21.15.52:0/1991564029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104df8d0 0x5636104dfcb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb6c004c000 tx=0x7fb6c0090fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a598d9549567c6fa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:41.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.041+0000 7fb6affff640 1 -- 172.21.15.52:0/1991564029 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c004b070 con 0x5636104df8d0 2024-04-02T13:54:41.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.042+0000 7fb6affff640 1 -- 172.21.15.52:0/1991564029 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6c00bebb0 con 0x5636104df8d0 2024-04-02T13:54:41.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.042+0000 7fb6affff640 1 -- 172.21.15.52:0/1991564029 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c00035f0 con 0x5636104df8d0 2024-04-02T13:54:41.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.042+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/1991564029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104df8d0 msgr2=0x5636104dfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.042+0000 7fb6ca1aa640 1 --2- 172.21.15.52:0/2693811116 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb6b00a3f50 0x7fb6b00a6410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x5636104db190 tx=0x7fb6bc009900 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:41.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.042+0000 7fb6ce7f4740 1 --2- 172.21.15.52:0/1991564029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104df8d0 0x5636104dfcb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb6c004c000 tx=0x7fb6c0090fa0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.042+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/1991564029 shutdown_connections 2024-04-02T13:54:41.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.042+0000 7fb6ce7f4740 1 --2- 172.21.15.52:0/1991564029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636104e8dd0 0x5636104eb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.042+0000 7fb6ce7f4740 1 --2- 172.21.15.52:0/1991564029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636104e01f0 0x5636104e8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.042+0000 7fb6ce7f4740 1 --2- 172.21.15.52:0/1991564029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104df8d0 0x5636104dfcb0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.043+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/1991564029 >> 172.21.15.52:0/1991564029 conn(0x5636104da000 msgr2=0x5636104da3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:41.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.043+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/1991564029 shutdown_connections 2024-04-02T13:54:41.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.043+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/1991564029 wait complete. 2024-04-02T13:54:41.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.043+0000 7fb6ce7f4740 1 -- start start 2024-04-02T13:54:41.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.043+0000 7fb6ce7f4740 1 Processor -- start 2024-04-02T13:54:41.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.043+0000 7fb6ce7f4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104df8d0 0x5636104e8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.044+0000 7fb6ce7f4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636104e01f0 0x5636104e5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.044+0000 7fb6c99a9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104df8d0 0x5636104e8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.044+0000 7fb6c99a9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104df8d0 0x5636104e8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42844/0 (socket says 172.21.15.52:42844) 2024-04-02T13:54:41.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.044+0000 7fb6ce7f4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636104e8dd0 0x5636104e6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.044+0000 7fb6ce7f4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5636104eb980 con 0x5636104df8d0 2024-04-02T13:54:41.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.044+0000 7fb6ce7f4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5636103fed30 con 0x5636104e01f0 2024-04-02T13:54:41.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.044+0000 7fb6ce7f4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5636104ebb00 con 0x5636104e8dd0 2024-04-02T13:54:41.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.044+0000 7fb6ce7f4740 10 client.0 _ll_register_callbacks cb 0x5636103fe720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:41.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.044+0000 7fb6c99a9640 1 -- 172.21.15.52:0/2777497064 learned_addr learned my addr 172.21.15.52:0/2777497064 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:41.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.044+0000 7fb6ca1aa640 1 --2- 172.21.15.52:0/2777497064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636104e8dd0 0x5636104e6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.044+0000 7fb6c91a8640 1 --2- 172.21.15.52:0/2777497064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636104e01f0 0x5636104e5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.044+0000 7fb6c99a9640 1 -- 172.21.15.52:0/2777497064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636104e8dd0 msgr2=0x5636104e6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:41.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.044+0000 7fb6c99a9640 1 --2- 172.21.15.52:0/2777497064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636104e8dd0 0x5636104e6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.044+0000 7fb6c99a9640 1 -- 172.21.15.52:0/2777497064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636104e01f0 msgr2=0x5636104e5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:41.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.044+0000 7fb6c99a9640 1 --2- 172.21.15.52:0/2777497064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636104e01f0 0x5636104e5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.044+0000 7fb6c99a9640 1 -- 172.21.15.52:0/2777497064 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636101573f0 con 0x5636104df8d0 2024-04-02T13:54:41.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.044+0000 7fb6c99a9640 1 --2- 172.21.15.52:0/2777497064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104df8d0 0x5636104e8750 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00ba910 tx=0x7fb6c00ba0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d477e16ac4c9b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:41.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.044+0000 7fb6ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:41.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.045+0000 7fb6ad7fa640 1 -- 172.21.15.52:0/2777497064 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c00d7060 con 0x5636104df8d0 2024-04-02T13:54:41.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.045+0000 7fb6ad7fa640 1 -- 172.21.15.52:0/2777497064 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb6c00bcf40 con 0x5636104df8d0 2024-04-02T13:54:41.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.045+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/2777497064 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6c00bebb0 con 0x5636104df8d0 2024-04-02T13:54:41.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.045+0000 7fb6ad7fa640 1 -- 172.21.15.52:0/2777497064 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb6c004b040 con 0x5636104df8d0 2024-04-02T13:54:41.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.045+0000 7fb6ce7f4740 10 client.15954 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:41.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.045+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/2777497064 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5636101170f0 con 0x5636104df8d0 2024-04-02T13:54:41.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.045+0000 7fb6ad7fa640 1 -- 172.21.15.52:0/2777497064 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6c00bebb0 con 0x5636104df8d0 2024-04-02T13:54:41.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.045+0000 7fb6ad7fa640 1 -- 172.21.15.52:0/2777497064 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb6c00be710 con 0x5636104df8d0 2024-04-02T13:54:41.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.045+0000 7fb6ce7f4740 10 client.15954 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:41.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.045+0000 7fb6ce7f4740 10 client.15954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:54:41.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.045+0000 7fb6ce7f4740 20 client.15954 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:41.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.045+0000 7fb6ce7f4740 1 --2- 172.21.15.52:0/2777497064 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5636104f8a00 0x5636104fadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.045+0000 7fb6ce7f4740 4 client.15954 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:54:41.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.045+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/2777497064 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563610450d70 con 0x5636104f8a00 2024-04-02T13:54:41.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.046+0000 7fb6c91a8640 1 --2- 172.21.15.52:0/2777497064 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5636104f8a00 0x5636104fadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.046+0000 7fb6ad7fa640 10 client.15954 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:41.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.046+0000 7fb6c91a8640 1 --2- 172.21.15.52:0/2777497064 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5636104f8a00 0x5636104fadf0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc92504661631d17 server_cookie=667367f0ab8dce5a in_seq=0 out_seq=0 2024-04-02T13:54:41.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.047+0000 7fb6ad7fa640 1 -- 172.21.15.52:0/2777497064 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563610450d70 con 0x5636104f8a00 2024-04-02T13:54:41.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.047+0000 7fb6ad7fa640 10 client.15954 handle_command_reply: tid=0 2024-04-02T13:54:41.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.047+0000 7fb6ce7f4740 1 client.15954 shutdown 2024-04-02T13:54:41.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.047+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/2777497064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104df8d0 msgr2=0x5636104e8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.047+0000 7fb6ce7f4740 1 --2- 172.21.15.52:0/2777497064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104df8d0 0x5636104e8750 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00ba910 tx=0x7fb6c00ba0a0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.047+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/2777497064 shutdown_connections 2024-04-02T13:54:41.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.047+0000 7fb6ce7f4740 1 --2- 172.21.15.52:0/2777497064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636104e8dd0 0x5636104e6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.047+0000 7fb6ce7f4740 1 --2- 172.21.15.52:0/2777497064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636104e01f0 0x5636104e5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.047+0000 7fb6ce7f4740 1 --2- 172.21.15.52:0/2777497064 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5636104f8a00 0x5636104fadf0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.047+0000 7fb6ce7f4740 1 --2- 172.21.15.52:0/2777497064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104df8d0 0x5636104e8750 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.047+0000 7fb6ad7fa640 0 client.15954 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:41.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.047+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/2777497064 >> 172.21.15.52:0/2777497064 conn(0x5636104da000 msgr2=0x5636104dbd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:41.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.047+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/2777497064 shutdown_connections 2024-04-02T13:54:41.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.047+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/2777497064 wait complete. 2024-04-02T13:54:41.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.047+0000 7fb6ce7f4740 20 client.15954 trim_cache size 0 max 16384 2024-04-02T13:54:41.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.065+0000 7fb6ce7f4740 1 Processor -- start 2024-04-02T13:54:41.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.065+0000 7fb6ce7f4740 1 -- start start 2024-04-02T13:54:41.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.065+0000 7fb6ce7f4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636104df8d0 0x5636104d9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.065+0000 7fb6ce7f4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104e01f0 0x5636104dbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.065+0000 7fb6ce7f4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636104e8dd0 0x5636104e8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.065+0000 7fb6ce7f4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c41b1030 con 0x5636104e01f0 2024-04-02T13:54:41.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.065+0000 7fb6ce7f4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c41b2b70 con 0x5636104df8d0 2024-04-02T13:54:41.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.065+0000 7fb6ce7f4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c41b1700 con 0x5636104e8dd0 2024-04-02T13:54:41.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.065+0000 7fb6c91a8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104e01f0 0x5636104dbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.065+0000 7fb6c91a8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104e01f0 0x5636104dbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42850/0 (socket says 172.21.15.52:42850) 2024-04-02T13:54:41.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.065+0000 7fb6c91a8640 1 -- 172.21.15.52:0/4020551008 learned_addr learned my addr 172.21.15.52:0/4020551008 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:41.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.065+0000 7fb6c99a9640 1 --2- 172.21.15.52:0/4020551008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636104df8d0 0x5636104d9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.066+0000 7fb6ca1aa640 1 --2- 172.21.15.52:0/4020551008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636104e8dd0 0x5636104e8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.066+0000 7fb6c91a8640 1 -- 172.21.15.52:0/4020551008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636104e8dd0 msgr2=0x5636104e8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:41.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.066+0000 7fb6c91a8640 1 --2- 172.21.15.52:0/4020551008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636104e8dd0 0x5636104e8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.066+0000 7fb6c91a8640 1 -- 172.21.15.52:0/4020551008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636104df8d0 msgr2=0x5636104d9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:41.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.066+0000 7fb6c91a8640 1 --2- 172.21.15.52:0/4020551008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636104df8d0 0x5636104d9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.066+0000 7fb6c91a8640 1 -- 172.21.15.52:0/4020551008 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636104fb3d0 con 0x5636104e01f0 2024-04-02T13:54:41.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.066+0000 7fb6c99a9640 1 --2- 172.21.15.52:0/4020551008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636104df8d0 0x5636104d9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:54:41.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.066+0000 7fb6c91a8640 1 --2- 172.21.15.52:0/4020551008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104e01f0 0x5636104dbac0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x5636104e5340 tx=0x7fb6b400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c4fb7a03a388fa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:41.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.066+0000 7fb6af7fe640 1 -- 172.21.15.52:0/4020551008 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4003b20 con 0x5636104e01f0 2024-04-02T13:54:41.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.066+0000 7fb6af7fe640 1 -- 172.21.15.52:0/4020551008 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6b4023020 con 0x5636104e01f0 2024-04-02T13:54:41.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.066+0000 7fb6af7fe640 1 -- 172.21.15.52:0/4020551008 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4011680 con 0x5636104e01f0 2024-04-02T13:54:41.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.067+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/4020551008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104e01f0 msgr2=0x5636104dbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.067+0000 7fb6ce7f4740 1 --2- 172.21.15.52:0/4020551008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104e01f0 0x5636104dbac0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x5636104e5340 tx=0x7fb6b400b390 comp rx=0 tx=0).stop 2024-04-02T13:54:41.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.067+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/4020551008 shutdown_connections 2024-04-02T13:54:41.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.067+0000 7fb6ce7f4740 1 --2- 172.21.15.52:0/4020551008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636104e8dd0 0x5636104e8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.067+0000 7fb6ce7f4740 1 --2- 172.21.15.52:0/4020551008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104e01f0 0x5636104dbac0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.067+0000 7fb6ce7f4740 1 --2- 172.21.15.52:0/4020551008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636104df8d0 0x5636104d9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.067+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/4020551008 >> 172.21.15.52:0/4020551008 conn(0x5636104da000 msgr2=0x5636104db3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:41.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.067+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/4020551008 shutdown_connections 2024-04-02T13:54:41.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.067+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/4020551008 wait complete. 2024-04-02T13:54:41.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.067+0000 7fb6ce7f4740 1 -- start start 2024-04-02T13:54:41.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.067+0000 7fb6ce7f4740 1 Processor -- start 2024-04-02T13:54:41.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.068+0000 7fb6ce7f4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104df8d0 0x5636104f0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.068+0000 7fb6ce7f4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636104e8dd0 0x5636104e22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.068+0000 7fb6ce7f4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636104e2830 0x5636104e2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.068+0000 7fb6ce7f4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c41b24a0 con 0x5636104df8d0 2024-04-02T13:54:41.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.068+0000 7fb6ce7f4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c41b1dd0 con 0x5636104e2830 2024-04-02T13:54:41.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.068+0000 7fb6ce7f4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c41af010 con 0x5636104e8dd0 2024-04-02T13:54:41.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.068+0000 7fb6c99a9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104df8d0 0x5636104f0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.068+0000 7fb6ce7f4740 10 client.0 _ll_register_callbacks cb 0x7fb6c00ba7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:41.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.068+0000 7fb6c99a9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104df8d0 0x5636104f0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42866/0 (socket says 172.21.15.52:42866) 2024-04-02T13:54:41.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.068+0000 7fb6c99a9640 1 -- 172.21.15.52:0/3295506189 learned_addr learned my addr 172.21.15.52:0/3295506189 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:41.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.068+0000 7fb6c91a8640 1 --2- 172.21.15.52:0/3295506189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636104e8dd0 0x5636104e22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.068+0000 7fb6ca1aa640 1 --2- 172.21.15.52:0/3295506189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636104e2830 0x5636104e2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.068+0000 7fb6c99a9640 1 -- 172.21.15.52:0/3295506189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636104e8dd0 msgr2=0x5636104e22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:41.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.068+0000 7fb6c99a9640 1 --2- 172.21.15.52:0/3295506189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636104e8dd0 0x5636104e22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.068+0000 7fb6c99a9640 1 -- 172.21.15.52:0/3295506189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636104e2830 msgr2=0x5636104e2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:41.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.068+0000 7fb6c99a9640 1 --2- 172.21.15.52:0/3295506189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636104e2830 0x5636104e2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.068+0000 7fb6c99a9640 1 -- 172.21.15.52:0/3295506189 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56361055dee0 con 0x5636104df8d0 2024-04-02T13:54:41.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.069+0000 7fb6c99a9640 1 --2- 172.21.15.52:0/3295506189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104df8d0 0x5636104f0ec0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00efd50 tx=0x7fb6c00bd0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc92728d72f75a9d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:41.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.069+0000 7fb6ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:41.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.069+0000 7fb6ad7fa640 1 -- 172.21.15.52:0/3295506189 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c00d7070 con 0x5636104df8d0 2024-04-02T13:54:41.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.069+0000 7fb6ad7fa640 1 -- 172.21.15.52:0/3295506189 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb6c00bd650 con 0x5636104df8d0 2024-04-02T13:54:41.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.069+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/3295506189 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6b4023020 con 0x5636104df8d0 2024-04-02T13:54:41.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.069+0000 7fb6ad7fa640 1 -- 172.21.15.52:0/3295506189 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb6c00bde10 con 0x5636104df8d0 2024-04-02T13:54:41.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.069+0000 7fb6ce7f4740 10 client.15960 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:41.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.069+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/3295506189 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56361055dee0 con 0x5636104df8d0 2024-04-02T13:54:41.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.069+0000 7fb6ad7fa640 1 -- 172.21.15.52:0/3295506189 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6b4023020 con 0x5636104df8d0 2024-04-02T13:54:41.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.069+0000 7fb6ad7fa640 1 -- 172.21.15.52:0/3295506189 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb6c00bd900 con 0x5636104df8d0 2024-04-02T13:54:41.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.069+0000 7fb6ce7f4740 10 client.15960 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:41.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.069+0000 7fb6ce7f4740 10 client.15960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:54:41.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.069+0000 7fb6ce7f4740 20 client.15960 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:41.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.069+0000 7fb6ce7f4740 1 --2- 172.21.15.52:0/3295506189 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x56361055f4c0 0x5636105618b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.069+0000 7fb6ce7f4740 4 client.15960 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:54:41.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.069+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/3295506189 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56361044b0c0 con 0x56361055f4c0 2024-04-02T13:54:41.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.070+0000 7fb6c91a8640 1 --2- 172.21.15.52:0/3295506189 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x56361055f4c0 0x5636105618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.070+0000 7fb6ad7fa640 10 client.15960 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:41.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.070+0000 7fb6c91a8640 1 --2- 172.21.15.52:0/3295506189 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x56361055f4c0 0x5636105618b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7670f6720afe845 server_cookie=f3995ab6e3c3a9e8 in_seq=0 out_seq=0 2024-04-02T13:54:41.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.070+0000 7fb6ad7fa640 1 -- 172.21.15.52:0/3295506189 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56361044b0c0 con 0x56361055f4c0 2024-04-02T13:54:41.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.070+0000 7fb6ad7fa640 10 client.15960 handle_command_reply: tid=0 2024-04-02T13:54:41.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.070+0000 7fb6ce7f4740 1 client.15960 shutdown 2024-04-02T13:54:41.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.071+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/3295506189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104df8d0 msgr2=0x5636104f0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.071+0000 7fb6ce7f4740 1 --2- 172.21.15.52:0/3295506189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104df8d0 0x5636104f0ec0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00efd50 tx=0x7fb6c00bd0d0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.071+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/3295506189 shutdown_connections 2024-04-02T13:54:41.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.071+0000 7fb6ce7f4740 1 --2- 172.21.15.52:0/3295506189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636104e2830 0x5636104e2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.071+0000 7fb6ce7f4740 1 --2- 172.21.15.52:0/3295506189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636104e8dd0 0x5636104e22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.071+0000 7fb6ce7f4740 1 --2- 172.21.15.52:0/3295506189 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x56361055f4c0 0x5636105618b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.071+0000 7fb6ce7f4740 1 --2- 172.21.15.52:0/3295506189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636104df8d0 0x5636104f0ec0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.071+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/3295506189 >> 172.21.15.52:0/3295506189 conn(0x5636104da000 msgr2=0x5636104dbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:41.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.071+0000 7fb6ad7fa640 0 client.15960 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:41.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.071+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/3295506189 shutdown_connections 2024-04-02T13:54:41.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.071+0000 7fb6ce7f4740 1 -- 172.21.15.52:0/3295506189 wait complete. 2024-04-02T13:54:41.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.071+0000 7fb6ce7f4740 20 client.15960 trim_cache size 0 max 16384 2024-04-02T13:54:41.083 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:54:41.083 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-04-02T13:54:41.083 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-04-02T13:54:41.083 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:54:41.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.071+0000 7fb6cbc0d640 1 -- 172.21.15.52:0/2693811116 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb6b00a3f50 msgr2=0x7fb6b00a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.071+0000 7fb6cbc0d640 1 --2- 172.21.15.52:0/2693811116 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb6b00a3f50 0x7fb6b00a6410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x5636104db190 tx=0x7fb6bc009900 comp rx=0 tx=0).stop 2024-04-02T13:54:41.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.071+0000 7fb6cbc0d640 1 -- 172.21.15.52:0/2693811116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6c4069ec0 msgr2=0x7fb6c419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.071+0000 7fb6cbc0d640 1 --2- 172.21.15.52:0/2693811116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6c4069ec0 0x7fb6c419a5b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0005940 tx=0x7fb6c002cd10 comp rx=0 tx=0).stop 2024-04-02T13:54:41.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.072+0000 7fb6cbc0d640 1 -- 172.21.15.52:0/2693811116 shutdown_connections 2024-04-02T13:54:41.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.072+0000 7fb6cbc0d640 1 --2- 172.21.15.52:0/2693811116 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb6b00a3f50 0x7fb6b00a6410 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.072+0000 7fb6cbc0d640 1 --2- 172.21.15.52:0/2693811116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6c4107b80 0x7fb6c419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.072+0000 7fb6cbc0d640 1 --2- 172.21.15.52:0/2693811116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c406a860 0x7fb6c419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.072+0000 7fb6cbc0d640 1 --2- 172.21.15.52:0/2693811116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6c4069ec0 0x7fb6c419a5b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.072+0000 7fb6cbc0d640 1 -- 172.21.15.52:0/2693811116 >> 172.21.15.52:0/2693811116 conn(0x7fb6c4077a90 msgr2=0x7fb6c410cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:41.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.072+0000 7fb6cbc0d640 1 -- 172.21.15.52:0/2693811116 shutdown_connections 2024-04-02T13:54:41.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.072+0000 7fb6cbc0d640 1 -- 172.21.15.52:0/2693811116 wait complete. 2024-04-02T13:54:41.086 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:54:41.086 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:54:41.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.215+0000 7f1eaf7fe640 1 -- 172.21.15.52:0/4007700633 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4032c00 con 0x7f1eb0106a10 2024-04-02T13:54:41.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.216+0000 7f1eb7241640 1 -- 172.21.15.52:0/4007700633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1eb0106a10 msgr2=0x7f1eb0106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.216+0000 7f1eb7241640 1 --2- 172.21.15.52:0/4007700633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1eb0106a10 0x7f1eb0106df0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400bac0 tx=0x7f1ea402d990 comp rx=0 tx=0).stop 2024-04-02T13:54:41.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.216+0000 7f1eb7241640 1 -- 172.21.15.52:0/4007700633 shutdown_connections 2024-04-02T13:54:41.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.216+0000 7f1eb7241640 1 --2- 172.21.15.52:0/4007700633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1eb01013c0 0x7f1eb010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.216+0000 7f1eb7241640 1 --2- 172.21.15.52:0/4007700633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1eb0100a00 0x7f1eb0100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.216+0000 7f1eb7241640 1 --2- 172.21.15.52:0/4007700633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1eb0106a10 0x7f1eb0106df0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.216+0000 7f1eb7241640 1 -- 172.21.15.52:0/4007700633 >> 172.21.15.52:0/4007700633 conn(0x7f1eb0077b00 msgr2=0x7f1eb0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:41.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.216+0000 7f1eb7241640 1 -- 172.21.15.52:0/4007700633 shutdown_connections 2024-04-02T13:54:41.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.216+0000 7f1eb7241640 1 -- 172.21.15.52:0/4007700633 wait complete. 2024-04-02T13:54:41.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.217+0000 7f1eb7241640 1 Processor -- start 2024-04-02T13:54:41.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.217+0000 7f1eb7241640 1 -- start start 2024-04-02T13:54:41.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.217+0000 7f1eb7241640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1eb0100a00 0x7f1eb0196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.217+0000 7f1eb7241640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1eb01013c0 0x7f1eb0196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.217+0000 7f1eb7241640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1eb0106a10 0x7f1eb019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.217+0000 7f1eb7241640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb010fd10 con 0x7f1eb0106a10 2024-04-02T13:54:41.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.217+0000 7f1eb57de640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1eb0106a10 0x7f1eb019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.217+0000 7f1eb4fdd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1eb0100a00 0x7f1eb0196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.217+0000 7f1eb4fdd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1eb0100a00 0x7f1eb0196210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:49754/0 (socket says 172.21.15.52:49754) 2024-04-02T13:54:41.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.217+0000 7f1eb4fdd640 1 -- 172.21.15.52:0/3849123170 learned_addr learned my addr 172.21.15.52:0/3849123170 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:41.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.217+0000 7f1eb7241640 1 -- 172.21.15.52:0/3849123170 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb010fb90 con 0x7f1eb0100a00 2024-04-02T13:54:41.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.217+0000 7f1eb7241640 1 -- 172.21.15.52:0/3849123170 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb010fe90 con 0x7f1eb01013c0 2024-04-02T13:54:41.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.217+0000 7f1eaffff640 1 --2- 172.21.15.52:0/3849123170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1eb01013c0 0x7f1eb0196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.218+0000 7f1eb4fdd640 1 -- 172.21.15.52:0/3849123170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1eb01013c0 msgr2=0x7f1eb0196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.218+0000 7f1eb4fdd640 1 --2- 172.21.15.52:0/3849123170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1eb01013c0 0x7f1eb0196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.218+0000 7f1eb4fdd640 1 -- 172.21.15.52:0/3849123170 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1eb0106a10 msgr2=0x7f1eb019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.218+0000 7f1eb4fdd640 1 --2- 172.21.15.52:0/3849123170 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1eb0106a10 0x7f1eb019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.218+0000 7f1eb4fdd640 1 -- 172.21.15.52:0/3849123170 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb019b1b0 con 0x7f1eb0100a00 2024-04-02T13:54:41.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.218+0000 7f1eaffff640 1 --2- 172.21.15.52:0/3849123170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1eb01013c0 0x7f1eb0196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:54:41.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.218+0000 7f1eb4fdd640 1 --2- 172.21.15.52:0/3849123170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1eb0100a00 0x7f1eb0196210 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4006010 tx=0x7f1ea403a9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:41.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.218+0000 7f1eadffb640 1 -- 172.21.15.52:0/3849123170 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4047070 con 0x7f1eb0100a00 2024-04-02T13:54:41.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.218+0000 7f1eb7241640 1 -- 172.21.15.52:0/3849123170 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb019b440 con 0x7f1eb0100a00 2024-04-02T13:54:41.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.218+0000 7f1eadffb640 1 -- 172.21.15.52:0/3849123170 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ea40428d0 con 0x7f1eb0100a00 2024-04-02T13:54:41.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.218+0000 7f1eadffb640 1 -- 172.21.15.52:0/3849123170 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4036070 con 0x7f1eb0100a00 2024-04-02T13:54:41.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.218+0000 7f1eb7241640 1 -- 172.21.15.52:0/3849123170 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb01a2d30 con 0x7f1eb0100a00 2024-04-02T13:54:41.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.220+0000 7f1eb7241640 1 -- 172.21.15.52:0/3849123170 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb00688f0 con 0x7f1eb0100a00 2024-04-02T13:54:41.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.222+0000 7f1eadffb640 1 -- 172.21.15.52:0/3849123170 <== mon.2 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f1ea404a070 con 0x7f1eb0100a00 2024-04-02T13:54:41.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.223+0000 7f1eadffb640 1 --2- 172.21.15.52:0/3849123170 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1ea00a82e0 0x7f1ea00aa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.223+0000 7f1eaffff640 1 --2- 172.21.15.52:0/3849123170 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1ea00a82e0 0x7f1ea00aa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.223+0000 7f1eadffb640 1 -- 172.21.15.52:0/3849123170 <== mon.2 v2:172.21.15.88:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f1ea40f2af0 con 0x7f1eb0100a00 2024-04-02T13:54:41.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.224+0000 7f1eaffff640 1 --2- 172.21.15.52:0/3849123170 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1ea00a82e0 0x7f1ea00aa7a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0105100 tx=0x7f1e9c005f90 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:41.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.225+0000 7f1eadffb640 1 -- 172.21.15.52:0/3849123170 <== mon.2 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1ea403f070 con 0x7f1eb0100a00 2024-04-02T13:54:41.302 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:41 smithi052 ceph-mon[32104]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:54:41.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.567+0000 7f1eb7241640 1 -- 172.21.15.52:0/3849123170 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1eb0108f70 con 0x7f1eb0100a00 2024-04-02T13:54:41.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.568+0000 7f1eadffb640 1 -- 172.21.15.52:0/3849123170 <== mon.2 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11923 (secure 0 0 0) 0x7f1ea40be6e0 con 0x7f1eb0100a00 2024-04-02T13:54:41.572 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:54:41.572 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:53:28.475061+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24457,24467,24490,24503,24475]},"id":1}]} 2024-04-02T13:54:41.572 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 28 2024-04-02T13:54:41.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.570+0000 7f1eb7241640 1 -- 172.21.15.52:0/3849123170 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1ea00a82e0 msgr2=0x7f1ea00aa7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.570+0000 7f1eb7241640 1 --2- 172.21.15.52:0/3849123170 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1ea00a82e0 0x7f1ea00aa7a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0105100 tx=0x7f1e9c005f90 comp rx=0 tx=0).stop 2024-04-02T13:54:41.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.570+0000 7f1eb7241640 1 -- 172.21.15.52:0/3849123170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1eb0100a00 msgr2=0x7f1eb0196210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.570+0000 7f1eb7241640 1 --2- 172.21.15.52:0/3849123170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1eb0100a00 0x7f1eb0196210 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4006010 tx=0x7f1ea403a9a0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.570+0000 7f1eb7241640 1 -- 172.21.15.52:0/3849123170 shutdown_connections 2024-04-02T13:54:41.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.570+0000 7f1eb7241640 1 --2- 172.21.15.52:0/3849123170 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1ea00a82e0 0x7f1ea00aa7a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.570+0000 7f1eb7241640 1 --2- 172.21.15.52:0/3849123170 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1eb0106a10 0x7f1eb019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.570+0000 7f1eb7241640 1 --2- 172.21.15.52:0/3849123170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1eb01013c0 0x7f1eb0196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.570+0000 7f1eb7241640 1 --2- 172.21.15.52:0/3849123170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1eb0100a00 0x7f1eb0196210 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.570+0000 7f1eb7241640 1 -- 172.21.15.52:0/3849123170 >> 172.21.15.52:0/3849123170 conn(0x7f1eb0077b00 msgr2=0x7f1eb01032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:41.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.571+0000 7f1eb7241640 1 -- 172.21.15.52:0/3849123170 shutdown_connections 2024-04-02T13:54:41.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.571+0000 7f1eb7241640 1 -- 172.21.15.52:0/3849123170 wait complete. 2024-04-02T13:54:41.585 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:54:41.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:41 smithi088 ceph-mon[36528]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:54:41.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.691+0000 7f527e0e3640 1 -- 172.21.15.52:0/1961490029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f527810e4a0 msgr2=0x7f5278110890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.691+0000 7f527e0e3640 1 --2- 172.21.15.52:0/1961490029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f527810e4a0 0x7f5278110890 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5270009ca0 tx=0x7f527002c100 comp rx=0 tx=0).stop 2024-04-02T13:54:41.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.691+0000 7f527e0e3640 1 -- 172.21.15.52:0/1961490029 shutdown_connections 2024-04-02T13:54:41.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.691+0000 7f527e0e3640 1 --2- 172.21.15.52:0/1961490029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f527810e4a0 0x7f5278110890 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.691+0000 7f527e0e3640 1 --2- 172.21.15.52:0/1961490029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5278069600 0x7f527810df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.691+0000 7f527e0e3640 1 --2- 172.21.15.52:0/1961490029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5278068ce0 0x7f52780690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.691+0000 7f527e0e3640 1 -- 172.21.15.52:0/1961490029 >> 172.21.15.52:0/1961490029 conn(0x7f5278077b00 msgr2=0x7f5278077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:41.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.692+0000 7f527e0e3640 1 -- 172.21.15.52:0/1961490029 shutdown_connections 2024-04-02T13:54:41.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.692+0000 7f527e0e3640 1 -- 172.21.15.52:0/1961490029 wait complete. 2024-04-02T13:54:41.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.692+0000 7f527e0e3640 1 Processor -- start 2024-04-02T13:54:41.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.692+0000 7f527e0e3640 1 -- start start 2024-04-02T13:54:41.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.693+0000 7f527e0e3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5278068ce0 0x7f527819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.693+0000 7f527e0e3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5278069600 0x7f527819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.693+0000 7f527e0e3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f527810e4a0 0x7f527819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.693+0000 7f527e0e3640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5278114060 con 0x7f5278068ce0 2024-04-02T13:54:41.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.693+0000 7f527e0e3640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5278113ee0 con 0x7f527810e4a0 2024-04-02T13:54:41.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.693+0000 7f527e0e3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f52781141e0 con 0x7f5278069600 2024-04-02T13:54:41.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.693+0000 7f52777fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5278068ce0 0x7f527819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.693+0000 7f5276ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5278069600 0x7f527819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.693+0000 7f5276ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5278069600 0x7f527819aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.52:43806/0 (socket says 172.21.15.52:43806) 2024-04-02T13:54:41.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.693+0000 7f5276ffd640 1 -- 172.21.15.52:0/3227892056 learned_addr learned my addr 172.21.15.52:0/3227892056 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:41.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.693+0000 7f5277fff640 1 --2- 172.21.15.52:0/3227892056 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f527810e4a0 0x7f527819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.693+0000 7f52777fe640 1 -- 172.21.15.52:0/3227892056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5278069600 msgr2=0x7f527819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.693+0000 7f52777fe640 1 --2- 172.21.15.52:0/3227892056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5278069600 0x7f527819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.693+0000 7f52777fe640 1 -- 172.21.15.52:0/3227892056 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f527810e4a0 msgr2=0x7f527819ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.693+0000 7f52777fe640 1 --2- 172.21.15.52:0/3227892056 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f527810e4a0 0x7f527819ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.693+0000 7f52777fe640 1 -- 172.21.15.52:0/3227892056 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f527819f530 con 0x7f5278068ce0 2024-04-02T13:54:41.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.693+0000 7f5276ffd640 1 --2- 172.21.15.52:0/3227892056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5278069600 0x7f527819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:54:41.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.693+0000 7f5277fff640 1 --2- 172.21.15.52:0/3227892056 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f527810e4a0 0x7f527819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:54:41.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.694+0000 7f52777fe640 1 --2- 172.21.15.52:0/3227892056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5278068ce0 0x7f527819a540 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f526800d6d0 tx=0x7f526800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:41.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.694+0000 7f5274ff9640 1 -- 172.21.15.52:0/3227892056 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268017070 con 0x7f5278068ce0 2024-04-02T13:54:41.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.694+0000 7f5274ff9640 1 -- 172.21.15.52:0/3227892056 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5268009820 con 0x7f5278068ce0 2024-04-02T13:54:41.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.694+0000 7f5274ff9640 1 -- 172.21.15.52:0/3227892056 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268004d00 con 0x7f5278068ce0 2024-04-02T13:54:41.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.694+0000 7f527e0e3640 1 -- 172.21.15.52:0/3227892056 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f527819f820 con 0x7f5278068ce0 2024-04-02T13:54:41.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.694+0000 7f527e0e3640 1 -- 172.21.15.52:0/3227892056 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52781a70b0 con 0x7f5278068ce0 2024-04-02T13:54:41.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.697+0000 7f5280d9c740 1 Processor -- start 2024-04-02T13:54:41.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.697+0000 7f5280d9c740 1 -- start start 2024-04-02T13:54:41.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.697+0000 7f5280d9c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115ce7a0 0x5573115ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.697+0000 7f5280d9c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573115cf0c0 0x5573115d7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.697+0000 7f5280d9c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573115d7ca0 0x5573115da090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.697+0000 7f5280d9c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5573115327b0 con 0x5573115ce7a0 2024-04-02T13:54:41.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.697+0000 7f5280d9c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5573114d1570 con 0x5573115d7ca0 2024-04-02T13:54:41.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.697+0000 7f5280d9c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55731152e760 con 0x5573115cf0c0 2024-04-02T13:54:41.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.697+0000 7f5277fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115ce7a0 0x5573115ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.697+0000 7f5277fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115ce7a0 0x5573115ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34754/0 (socket says 172.21.15.52:34754) 2024-04-02T13:54:41.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.697+0000 7f5277fff640 1 -- 172.21.15.52:0/3506536248 learned_addr learned my addr 172.21.15.52:0/3506536248 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:41.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.697+0000 7f5276ffd640 1 --2- 172.21.15.52:0/3506536248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573115d7ca0 0x5573115da090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.697+0000 7f5276ffd640 1 -- 172.21.15.52:0/3506536248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573115cf0c0 msgr2=0x5573115d7760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:54:41.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.698+0000 7f5276ffd640 1 --2- 172.21.15.52:0/3506536248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573115cf0c0 0x5573115d7760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.698+0000 7f5276ffd640 1 -- 172.21.15.52:0/3506536248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115ce7a0 msgr2=0x5573115ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:41.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.698+0000 7f5276ffd640 1 --2- 172.21.15.52:0/3506536248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115ce7a0 0x5573115ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.698+0000 7f5276ffd640 1 -- 172.21.15.52:0/3506536248 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573112463f0 con 0x5573115d7ca0 2024-04-02T13:54:41.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.698+0000 7f5277fff640 1 --2- 172.21.15.52:0/3506536248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115ce7a0 0x5573115ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:54:41.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.698+0000 7f5276ffd640 1 --2- 172.21.15.52:0/3506536248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573115d7ca0 0x5573115da090 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f526c005fd0 tx=0x7f526c010930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48f9424aa4fa9a7c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:41.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.698+0000 7f525dffb640 1 -- 172.21.15.52:0/3506536248 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c011760 con 0x5573115d7ca0 2024-04-02T13:54:41.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.698+0000 7f525dffb640 1 -- 172.21.15.52:0/3506536248 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f526c011de0 con 0x5573115d7ca0 2024-04-02T13:54:41.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.698+0000 7f525dffb640 1 -- 172.21.15.52:0/3506536248 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c018380 con 0x5573115d7ca0 2024-04-02T13:54:41.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.699+0000 7f5274ff9640 1 -- 172.21.15.52:0/3227892056 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f52680099d0 con 0x7f5278068ce0 2024-04-02T13:54:41.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.699+0000 7f5280d9c740 1 -- 172.21.15.52:0/3506536248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573115d7ca0 msgr2=0x5573115da090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.699+0000 7f5280d9c740 1 --2- 172.21.15.52:0/3506536248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573115d7ca0 0x5573115da090 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f526c005fd0 tx=0x7f526c010930 comp rx=0 tx=0).stop 2024-04-02T13:54:41.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.699+0000 7f5280d9c740 1 -- 172.21.15.52:0/3506536248 shutdown_connections 2024-04-02T13:54:41.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.699+0000 7f5280d9c740 1 --2- 172.21.15.52:0/3506536248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573115d7ca0 0x5573115da090 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.699+0000 7f5280d9c740 1 --2- 172.21.15.52:0/3506536248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573115cf0c0 0x5573115d7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.699+0000 7f5280d9c740 1 --2- 172.21.15.52:0/3506536248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115ce7a0 0x5573115ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.699+0000 7f5280d9c740 1 -- 172.21.15.52:0/3506536248 >> 172.21.15.52:0/3506536248 conn(0x5573115c8ed0 msgr2=0x5573115c92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:41.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.699+0000 7f5280d9c740 1 -- 172.21.15.52:0/3506536248 shutdown_connections 2024-04-02T13:54:41.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.699+0000 7f5280d9c740 1 -- 172.21.15.52:0/3506536248 wait complete. 2024-04-02T13:54:41.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.699+0000 7f5280d9c740 1 -- start start 2024-04-02T13:54:41.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.699+0000 7f5274ff9640 1 --2- 172.21.15.52:0/3227892056 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f52600a3e60 0x7f52600a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.700+0000 7f5277fff640 1 --2- 172.21.15.52:0/3227892056 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f52600a3e60 0x7f52600a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.700+0000 7f5280d9c740 1 Processor -- start 2024-04-02T13:54:41.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.700+0000 7f5274ff9640 1 -- 172.21.15.52:0/3227892056 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f52680d0590 con 0x7f5278068ce0 2024-04-02T13:54:41.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.700+0000 7f5280d9c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573115ce7a0 0x5573115d7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.700+0000 7f5280d9c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573115cf0c0 0x5573115d49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.700+0000 7f5280d9c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115d7ca0 0x5573115d4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.700+0000 7f5280d9c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5573115daa90 con 0x5573115d7ca0 2024-04-02T13:54:41.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.700+0000 7f5280d9c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5573114ede80 con 0x5573115ce7a0 2024-04-02T13:54:41.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.700+0000 7f5280d9c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5573115dac10 con 0x5573115cf0c0 2024-04-02T13:54:41.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.700+0000 7f5280d9c740 10 client.0 _ll_register_callbacks cb 0x557311529aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:41.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.700+0000 7f52777fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573115ce7a0 0x5573115d7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.700+0000 7f52777fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573115ce7a0 0x5573115d7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:56146/0 (socket says 172.21.15.52:56146) 2024-04-02T13:54:41.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.700+0000 7f52777fe640 1 -- 172.21.15.52:0/315560694 learned_addr learned my addr 172.21.15.52:0/315560694 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:41.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.700+0000 7f5277fff640 1 --2- 172.21.15.52:0/3227892056 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f52600a3e60 0x7f52600a6320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5270006010 tx=0x7f527002e040 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:41.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.701+0000 7f5277fff640 1 --2- 172.21.15.52:0/315560694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115d7ca0 0x5573115d4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.701+0000 7f5276ffd640 1 --2- 172.21.15.52:0/315560694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573115cf0c0 0x5573115d49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.701+0000 7f52777fe640 1 -- 172.21.15.52:0/315560694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573115cf0c0 msgr2=0x5573115d49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:41.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.701+0000 7f52777fe640 1 --2- 172.21.15.52:0/315560694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573115cf0c0 0x5573115d49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.701+0000 7f52777fe640 1 -- 172.21.15.52:0/315560694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115d7ca0 msgr2=0x5573115d4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:41.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.701+0000 7f52777fe640 1 --2- 172.21.15.52:0/315560694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115d7ca0 0x5573115d4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.701+0000 7f52777fe640 1 -- 172.21.15.52:0/315560694 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573113107d0 con 0x5573115ce7a0 2024-04-02T13:54:41.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.701+0000 7f5277fff640 1 --2- 172.21.15.52:0/315560694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115d7ca0 0x5573115d4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:54:41.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.701+0000 7f5256ffd640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-02T13:54:41.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.701+0000 7f52777fe640 1 --2- 172.21.15.52:0/315560694 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573115ce7a0 0x5573115d7660 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f526809bae0 tx=0x7f526809b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14b8585e52823db9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:41.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.701+0000 7f5256ffd640 1 -- 172.21.15.52:0/315560694 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52680b5060 con 0x5573115ce7a0 2024-04-02T13:54:41.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.701+0000 7f5280d9c740 1 -- 172.21.15.52:0/315560694 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f526c011de0 con 0x5573115ce7a0 2024-04-02T13:54:41.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.701+0000 7f5256ffd640 1 -- 172.21.15.52:0/315560694 <== mon.2 v2:172.21.15.88:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f5268098450 con 0x5573115ce7a0 2024-04-02T13:54:41.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.701+0000 7f5256ffd640 1 -- 172.21.15.52:0/315560694 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f526809f070 con 0x5573115ce7a0 2024-04-02T13:54:41.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.701+0000 7f5280d9c740 10 client.25526 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:41.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.701+0000 7f5280d9c740 1 -- 172.21.15.52:0/315560694 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5573115e6ad0 con 0x5573115ce7a0 2024-04-02T13:54:41.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.702+0000 7f5256ffd640 1 -- 172.21.15.52:0/315560694 <== mon.2 v2:172.21.15.88:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f526c011de0 con 0x5573115ce7a0 2024-04-02T13:54:41.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.702+0000 7f5256ffd640 1 -- 172.21.15.52:0/315560694 <== mon.2 v2:172.21.15.88:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5268023060 con 0x5573115ce7a0 2024-04-02T13:54:41.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.702+0000 7f5280d9c740 10 client.25526 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:41.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.702+0000 7f5280d9c740 10 client.25526 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T13:54:41.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.702+0000 7f5280d9c740 20 client.25526 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:41.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.702+0000 7f5280d9c740 1 --2- 172.21.15.52:0/315560694 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5573115e7ee0 0x5573115ea2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.702+0000 7f5280d9c740 4 client.25526 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:54:41.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.702+0000 7f5280d9c740 1 -- 172.21.15.52:0/315560694 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55731153fcf0 con 0x5573115e7ee0 2024-04-02T13:54:41.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.703+0000 7f5276ffd640 1 --2- 172.21.15.52:0/315560694 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5573115e7ee0 0x5573115ea2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.703+0000 7f5276ffd640 1 --2- 172.21.15.52:0/315560694 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5573115e7ee0 0x5573115ea2d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dfa7c172fe7f3a25 server_cookie=1137b9a87e1d3785 in_seq=0 out_seq=0 2024-04-02T13:54:41.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.703+0000 7f5256ffd640 10 client.25526 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:54:41.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.704+0000 7f5256ffd640 1 -- 172.21.15.52:0/315560694 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55731153fcf0 con 0x5573115e7ee0 2024-04-02T13:54:41.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.704+0000 7f5256ffd640 10 client.25526 handle_command_reply: tid=0 2024-04-02T13:54:41.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.704+0000 7f5280d9c740 1 client.25526 shutdown 2024-04-02T13:54:41.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.704+0000 7f5280d9c740 1 -- 172.21.15.52:0/315560694 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573115ce7a0 msgr2=0x5573115d7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.704+0000 7f5280d9c740 1 --2- 172.21.15.52:0/315560694 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573115ce7a0 0x5573115d7660 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f526809bae0 tx=0x7f526809b800 comp rx=0 tx=0).stop 2024-04-02T13:54:41.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.704+0000 7f5280d9c740 1 -- 172.21.15.52:0/315560694 shutdown_connections 2024-04-02T13:54:41.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.704+0000 7f5280d9c740 1 --2- 172.21.15.52:0/315560694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115d7ca0 0x5573115d4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.704+0000 7f5280d9c740 1 --2- 172.21.15.52:0/315560694 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5573115e7ee0 0x5573115ea2d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.704+0000 7f5280d9c740 1 --2- 172.21.15.52:0/315560694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573115cf0c0 0x5573115d49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.704+0000 7f5256ffd640 0 client.25526 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:54:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.704+0000 7f5280d9c740 1 --2- 172.21.15.52:0/315560694 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573115ce7a0 0x5573115d7660 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.704+0000 7f5280d9c740 1 -- 172.21.15.52:0/315560694 >> 172.21.15.52:0/315560694 conn(0x5573115c8ed0 msgr2=0x5573115cac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.704+0000 7f5280d9c740 1 -- 172.21.15.52:0/315560694 shutdown_connections 2024-04-02T13:54:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.704+0000 7f5280d9c740 1 -- 172.21.15.52:0/315560694 wait complete. 2024-04-02T13:54:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.704+0000 7f5280d9c740 20 client.25526 trim_cache size 0 max 16384 2024-04-02T13:54:41.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.731+0000 7f5280d9c740 1 Processor -- start 2024-04-02T13:54:41.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.731+0000 7f5280d9c740 1 -- start start 2024-04-02T13:54:41.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.732+0000 7f5280d9c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573115ce7a0 0x5573115e3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.732+0000 7f5280d9c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115cf0c0 0x5573115d7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.732+0000 7f5280d9c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573115d7ca0 0x5573115d7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.732+0000 7f5276ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115cf0c0 0x5573115d7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.732+0000 7f5276ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115cf0c0 0x5573115d7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34774/0 (socket says 172.21.15.52:34774) 2024-04-02T13:54:41.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.732+0000 7f5276ffd640 1 -- 172.21.15.52:0/608776790 learned_addr learned my addr 172.21.15.52:0/608776790 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:41.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.732+0000 7f5280d9c740 1 -- 172.21.15.52:0/608776790 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f52781b10c0 con 0x5573115cf0c0 2024-04-02T13:54:41.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.732+0000 7f5280d9c740 1 -- 172.21.15.52:0/608776790 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f52781b2c00 con 0x5573115ce7a0 2024-04-02T13:54:41.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.732+0000 7f5280d9c740 1 -- 172.21.15.52:0/608776790 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f52781b1790 con 0x5573115d7ca0 2024-04-02T13:54:41.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.732+0000 7f52777fe640 1 --2- 172.21.15.52:0/608776790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573115ce7a0 0x5573115e3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.732+0000 7f52777fe640 1 -- 172.21.15.52:0/608776790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573115d7ca0 msgr2=0x5573115d7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:41.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.732+0000 7f52777fe640 1 --2- 172.21.15.52:0/608776790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573115d7ca0 0x5573115d7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.732+0000 7f52777fe640 1 -- 172.21.15.52:0/608776790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115cf0c0 msgr2=0x5573115d7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:41.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.732+0000 7f52777fe640 1 --2- 172.21.15.52:0/608776790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115cf0c0 0x5573115d7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.732+0000 7f52777fe640 1 -- 172.21.15.52:0/608776790 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573115d52e0 con 0x5573115ce7a0 2024-04-02T13:54:41.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.732+0000 7f5276ffd640 1 --2- 172.21.15.52:0/608776790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115cf0c0 0x5573115d7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:54:41.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.733+0000 7f52777fe640 1 --2- 172.21.15.52:0/608776790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573115ce7a0 0x5573115e3950 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5268098240 tx=0x7f526809b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf0ef4078135c58c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:41.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.733+0000 7f525d7fa640 1 -- 172.21.15.52:0/608776790 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52680b9070 con 0x5573115ce7a0 2024-04-02T13:54:41.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.733+0000 7f525d7fa640 1 -- 172.21.15.52:0/608776790 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52680b5060 con 0x5573115ce7a0 2024-04-02T13:54:41.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.733+0000 7f525d7fa640 1 -- 172.21.15.52:0/608776790 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268020040 con 0x5573115ce7a0 2024-04-02T13:54:41.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.734+0000 7f5280d9c740 1 -- 172.21.15.52:0/608776790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573115ce7a0 msgr2=0x5573115e3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.734+0000 7f5280d9c740 1 --2- 172.21.15.52:0/608776790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573115ce7a0 0x5573115e3950 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5268098240 tx=0x7f526809b870 comp rx=0 tx=0).stop 2024-04-02T13:54:41.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.734+0000 7f5280d9c740 1 -- 172.21.15.52:0/608776790 shutdown_connections 2024-04-02T13:54:41.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.734+0000 7f5280d9c740 1 --2- 172.21.15.52:0/608776790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573115d7ca0 0x5573115d7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.734+0000 7f5280d9c740 1 --2- 172.21.15.52:0/608776790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115cf0c0 0x5573115d7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.734+0000 7f5280d9c740 1 --2- 172.21.15.52:0/608776790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573115ce7a0 0x5573115e3950 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.734+0000 7f5280d9c740 1 -- 172.21.15.52:0/608776790 >> 172.21.15.52:0/608776790 conn(0x5573115c8ed0 msgr2=0x5573115c92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:41.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.734+0000 7f5280d9c740 1 -- 172.21.15.52:0/608776790 shutdown_connections 2024-04-02T13:54:41.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.734+0000 7f5280d9c740 1 -- 172.21.15.52:0/608776790 wait complete. 2024-04-02T13:54:41.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.734+0000 7f5280d9c740 1 -- start start 2024-04-02T13:54:41.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.735+0000 7f5280d9c740 1 Processor -- start 2024-04-02T13:54:41.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.735+0000 7f5280d9c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115ce7a0 0x5573115e0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.735+0000 7f5280d9c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573115cf0c0 0x5573115d8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.735+0000 7f5280d9c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573115d7ca0 0x5573115d8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.735+0000 7f5280d9c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f52781b2530 con 0x5573115ce7a0 2024-04-02T13:54:41.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.735+0000 7f52777fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115ce7a0 0x5573115e0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.735+0000 7f5280d9c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f52781b1e60 con 0x5573115d7ca0 2024-04-02T13:54:41.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.735+0000 7f5280d9c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f52781af0a0 con 0x5573115cf0c0 2024-04-02T13:54:41.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.735+0000 7f52777fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115ce7a0 0x5573115e0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34782/0 (socket says 172.21.15.52:34782) 2024-04-02T13:54:41.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.735+0000 7f52777fe640 1 -- 172.21.15.52:0/3939036494 learned_addr learned my addr 172.21.15.52:0/3939036494 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:41.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.735+0000 7f5280d9c740 10 client.0 _ll_register_callbacks cb 0x7f526809b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:41.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.735+0000 7f5276ffd640 1 --2- 172.21.15.52:0/3939036494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573115cf0c0 0x5573115d8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.735+0000 7f5277fff640 1 --2- 172.21.15.52:0/3939036494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573115d7ca0 0x5573115d8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.735+0000 7f52777fe640 1 -- 172.21.15.52:0/3939036494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573115cf0c0 msgr2=0x5573115d8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:41.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.735+0000 7f52777fe640 1 --2- 172.21.15.52:0/3939036494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573115cf0c0 0x5573115d8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.735+0000 7f52777fe640 1 -- 172.21.15.52:0/3939036494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573115d7ca0 msgr2=0x5573115d8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:41.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.735+0000 7f52777fe640 1 --2- 172.21.15.52:0/3939036494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573115d7ca0 0x5573115d8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.735+0000 7f52777fe640 1 -- 172.21.15.52:0/3939036494 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573115ceef0 con 0x5573115ce7a0 2024-04-02T13:54:41.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.735+0000 7f5277fff640 1 --2- 172.21.15.52:0/3939036494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573115d7ca0 0x5573115d8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:54:41.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.736+0000 7f52777fe640 1 --2- 172.21.15.52:0/3939036494 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115ce7a0 0x5573115e0160 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f52680987b0 tx=0x7f526809b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4629a1c895d3ac06 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:41.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.736+0000 7f5256ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:41.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.736+0000 7f5256ffd640 1 -- 172.21.15.52:0/3939036494 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268020050 con 0x5573115ce7a0 2024-04-02T13:54:41.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.736+0000 7f5256ffd640 1 -- 172.21.15.52:0/3939036494 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f5268097d30 con 0x5573115ce7a0 2024-04-02T13:54:41.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.736+0000 7f5280d9c740 1 -- 172.21.15.52:0/3939036494 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52680b5060 con 0x5573115ce7a0 2024-04-02T13:54:41.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.736+0000 7f5256ffd640 1 -- 172.21.15.52:0/3939036494 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f52680b9070 con 0x5573115ce7a0 2024-04-02T13:54:41.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.736+0000 7f5280d9c740 10 client.15984 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:41.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.736+0000 7f5280d9c740 1 -- 172.21.15.52:0/3939036494 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5573115dfc10 con 0x5573115ce7a0 2024-04-02T13:54:41.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.736+0000 7f5256ffd640 1 -- 172.21.15.52:0/3939036494 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f526807f070 con 0x5573115ce7a0 2024-04-02T13:54:41.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.736+0000 7f5256ffd640 1 -- 172.21.15.52:0/3939036494 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5268082020 con 0x5573115ce7a0 2024-04-02T13:54:41.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.736+0000 7f5280d9c740 10 client.15984 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:41.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.736+0000 7f5280d9c740 10 client.15984 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T13:54:41.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.736+0000 7f5280d9c740 20 client.15984 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:41.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.736+0000 7f5280d9c740 1 --2- 172.21.15.52:0/3939036494 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5573115e7ee0 0x557311650600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.736+0000 7f5280d9c740 4 client.15984 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:54:41.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.736+0000 7f5280d9c740 1 -- 172.21.15.52:0/3939036494 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f52680b5060 con 0x5573115e7ee0 2024-04-02T13:54:41.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.737+0000 7f5276ffd640 1 --2- 172.21.15.52:0/3939036494 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5573115e7ee0 0x557311650600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.737+0000 7f5256ffd640 10 client.15984 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:54:41.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.737+0000 7f5276ffd640 1 --2- 172.21.15.52:0/3939036494 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5573115e7ee0 0x557311650600 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=af9802318b12b81d server_cookie=adcb6c65d54846f2 in_seq=0 out_seq=0 2024-04-02T13:54:41.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.737+0000 7f5256ffd640 1 -- 172.21.15.52:0/3939036494 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f526c029e50 con 0x5573115e7ee0 2024-04-02T13:54:41.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.737+0000 7f5256ffd640 10 client.15984 handle_command_reply: tid=0 2024-04-02T13:54:41.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.737+0000 7f5280d9c740 1 client.15984 shutdown 2024-04-02T13:54:41.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.737+0000 7f5280d9c740 1 -- 172.21.15.52:0/3939036494 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115ce7a0 msgr2=0x5573115e0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.737+0000 7f5280d9c740 1 --2- 172.21.15.52:0/3939036494 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115ce7a0 0x5573115e0160 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f52680987b0 tx=0x7f526809b800 comp rx=0 tx=0).stop 2024-04-02T13:54:41.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.737+0000 7f5280d9c740 1 -- 172.21.15.52:0/3939036494 shutdown_connections 2024-04-02T13:54:41.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.738+0000 7f5280d9c740 1 --2- 172.21.15.52:0/3939036494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573115d7ca0 0x5573115d8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.738+0000 7f5280d9c740 1 --2- 172.21.15.52:0/3939036494 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5573115e7ee0 0x557311650600 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.738+0000 7f5256ffd640 0 client.15984 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:54:41.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.738+0000 7f5280d9c740 1 --2- 172.21.15.52:0/3939036494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573115cf0c0 0x5573115d8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.738+0000 7f5280d9c740 1 --2- 172.21.15.52:0/3939036494 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573115ce7a0 0x5573115e0160 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.738+0000 7f5280d9c740 1 -- 172.21.15.52:0/3939036494 >> 172.21.15.52:0/3939036494 conn(0x5573115c8ed0 msgr2=0x5573115d3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:41.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.738+0000 7f5280d9c740 1 -- 172.21.15.52:0/3939036494 shutdown_connections 2024-04-02T13:54:41.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.738+0000 7f5280d9c740 1 -- 172.21.15.52:0/3939036494 wait complete. 2024-04-02T13:54:41.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.738+0000 7f5280d9c740 20 client.15984 trim_cache size 0 max 16384 2024-04-02T13:54:41.747 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:54:41.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.738+0000 7f527e0e3640 1 -- 172.21.15.52:0/3227892056 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f52600a3e60 msgr2=0x7f52600a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.738+0000 7f527e0e3640 1 --2- 172.21.15.52:0/3227892056 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f52600a3e60 0x7f52600a6320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5270006010 tx=0x7f527002e040 comp rx=0 tx=0).stop 2024-04-02T13:54:41.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.738+0000 7f527e0e3640 1 -- 172.21.15.52:0/3227892056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5278068ce0 msgr2=0x7f527819a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.738+0000 7f527e0e3640 1 --2- 172.21.15.52:0/3227892056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5278068ce0 0x7f527819a540 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f526800d6d0 tx=0x7f526800db90 comp rx=0 tx=0).stop 2024-04-02T13:54:41.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.738+0000 7f527e0e3640 1 -- 172.21.15.52:0/3227892056 shutdown_connections 2024-04-02T13:54:41.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.738+0000 7f527e0e3640 1 --2- 172.21.15.52:0/3227892056 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f52600a3e60 0x7f52600a6320 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.738+0000 7f527e0e3640 1 --2- 172.21.15.52:0/3227892056 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f527810e4a0 0x7f527819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.738+0000 7f527e0e3640 1 --2- 172.21.15.52:0/3227892056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5278069600 0x7f527819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.738+0000 7f527e0e3640 1 --2- 172.21.15.52:0/3227892056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5278068ce0 0x7f527819a540 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.739+0000 7f527e0e3640 1 -- 172.21.15.52:0/3227892056 >> 172.21.15.52:0/3227892056 conn(0x7f5278077b00 msgr2=0x7f527810f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:41.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.739+0000 7f527e0e3640 1 -- 172.21.15.52:0/3227892056 shutdown_connections 2024-04-02T13:54:41.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.739+0000 7f527e0e3640 1 -- 172.21.15.52:0/3227892056 wait complete. 2024-04-02T13:54:41.753 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:54:41.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:41 smithi189 ceph-mon[36683]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:54:41.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.858+0000 7fe8638f4640 1 -- 172.21.15.52:0/2376679902 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe85c101330 msgr2=0x7fe85c10d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.858+0000 7fe8638f4640 1 --2- 172.21.15.52:0/2376679902 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe85c101330 0x7fe85c10d840 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe854009be0 tx=0x7fe85402bef0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.858+0000 7fe8638f4640 1 -- 172.21.15.52:0/2376679902 shutdown_connections 2024-04-02T13:54:41.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.858+0000 7fe8638f4640 1 --2- 172.21.15.52:0/2376679902 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe85c101330 0x7fe85c10d840 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.858+0000 7fe8638f4640 1 --2- 172.21.15.52:0/2376679902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe85c100970 0x7fe85c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.858+0000 7fe8638f4640 1 --2- 172.21.15.52:0/2376679902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe85c106980 0x7fe85c106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.858+0000 7fe8638f4640 1 -- 172.21.15.52:0/2376679902 >> 172.21.15.52:0/2376679902 conn(0x7fe85c077b00 msgr2=0x7fe85c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:41.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.858+0000 7fe8638f4640 1 -- 172.21.15.52:0/2376679902 shutdown_connections 2024-04-02T13:54:41.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.859+0000 7fe8638f4640 1 -- 172.21.15.52:0/2376679902 wait complete. 2024-04-02T13:54:41.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.859+0000 7fe8638f4640 1 Processor -- start 2024-04-02T13:54:41.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.859+0000 7fe8638f4640 1 -- start start 2024-04-02T13:54:41.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.859+0000 7fe8638f4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe85c100970 0x7fe85c199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.859+0000 7fe8638f4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe85c101330 0x7fe85c1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.859+0000 7fe8638f4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe85c106980 0x7fe85c1a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.859+0000 7fe8638f4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c110d60 con 0x7fe85c106980 2024-04-02T13:54:41.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.859+0000 7fe8638f4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c110be0 con 0x7fe85c101330 2024-04-02T13:54:41.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.859+0000 7fe8638f4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c110ee0 con 0x7fe85c100970 2024-04-02T13:54:41.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.860+0000 7fe861e91640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe85c106980 0x7fe85c1a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.860+0000 7fe861e91640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe85c106980 0x7fe85c1a41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34802/0 (socket says 172.21.15.52:34802) 2024-04-02T13:54:41.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.860+0000 7fe861e91640 1 -- 172.21.15.52:0/2594929544 learned_addr learned my addr 172.21.15.52:0/2594929544 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:41.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.860+0000 7fe860e8f640 1 --2- 172.21.15.52:0/2594929544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe85c101330 0x7fe85c1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.860+0000 7fe861690640 1 --2- 172.21.15.52:0/2594929544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe85c100970 0x7fe85c199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.860+0000 7fe861e91640 1 -- 172.21.15.52:0/2594929544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe85c100970 msgr2=0x7fe85c199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.860+0000 7fe861e91640 1 --2- 172.21.15.52:0/2594929544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe85c100970 0x7fe85c199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.860+0000 7fe861e91640 1 -- 172.21.15.52:0/2594929544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe85c101330 msgr2=0x7fe85c1996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.860+0000 7fe861e91640 1 --2- 172.21.15.52:0/2594929544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe85c101330 0x7fe85c1996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.860+0000 7fe861e91640 1 -- 172.21.15.52:0/2594929544 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe85c1a4720 con 0x7fe85c106980 2024-04-02T13:54:41.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.860+0000 7fe860e8f640 1 --2- 172.21.15.52:0/2594929544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe85c101330 0x7fe85c1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:54:41.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.860+0000 7fe861690640 1 --2- 172.21.15.52:0/2594929544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe85c100970 0x7fe85c199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:54:41.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.860+0000 7fe861e91640 1 --2- 172.21.15.52:0/2594929544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe85c106980 0x7fe85c1a41e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe8540096f0 tx=0x7fe8540336b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:41.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.860+0000 7fe8527fc640 1 -- 172.21.15.52:0/2594929544 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe854043020 con 0x7fe85c106980 2024-04-02T13:54:41.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.860+0000 7fe8638f4640 1 -- 172.21.15.52:0/2594929544 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe85c1a48f0 con 0x7fe85c106980 2024-04-02T13:54:41.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.861+0000 7fe8638f4640 1 -- 172.21.15.52:0/2594929544 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe85c1a4e00 con 0x7fe85c106980 2024-04-02T13:54:41.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.861+0000 7fe8527fc640 1 -- 172.21.15.52:0/2594929544 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe854003e50 con 0x7fe85c106980 2024-04-02T13:54:41.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.861+0000 7fe8527fc640 1 -- 172.21.15.52:0/2594929544 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe854034690 con 0x7fe85c106980 2024-04-02T13:54:41.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.863+0000 7fe8527fc640 1 -- 172.21.15.52:0/2594929544 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fe854046020 con 0x7fe85c106980 2024-04-02T13:54:41.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.863+0000 7fe8527fc640 1 --2- 172.21.15.52:0/2594929544 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe8480a3f50 0x7fe8480a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.864+0000 7fe8527fc640 1 -- 172.21.15.52:0/2594929544 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe8540f32d0 con 0x7fe85c106980 2024-04-02T13:54:41.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.864+0000 7fe861690640 1 --2- 172.21.15.52:0/2594929544 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe8480a3f50 0x7fe8480a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.864+0000 7fe8665b0740 1 Processor -- start 2024-04-02T13:54:41.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.864+0000 7fe8665b0740 1 -- start start 2024-04-02T13:54:41.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.864+0000 7fe861690640 1 --2- 172.21.15.52:0/2594929544 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe8480a3f50 0x7fe8480a6410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe858005e00 tx=0x7fe858005c70 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:41.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.865+0000 7fe8665b0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ee97d7490 0x562ee97d7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.865+0000 7fe8665b0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97d7db0 0x562ee97e4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.865+0000 7fe8665b0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562ee97e50b0 0x562ee97e74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.865+0000 7fe8665b0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562ee973b7b0 con 0x562ee97d7db0 2024-04-02T13:54:41.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.865+0000 7fe8665b0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x562ee96da570 con 0x562ee97e50b0 2024-04-02T13:54:41.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.865+0000 7fe8665b0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562ee9737760 con 0x562ee97d7490 2024-04-02T13:54:41.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.865+0000 7fe860e8f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97d7db0 0x562ee97e4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.865+0000 7fe860e8f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97d7db0 0x562ee97e4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34806/0 (socket says 172.21.15.52:34806) 2024-04-02T13:54:41.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.865+0000 7fe860e8f640 1 -- 172.21.15.52:0/43261893 learned_addr learned my addr 172.21.15.52:0/43261893 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:41.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.865+0000 7fe861690640 1 --2- 172.21.15.52:0/43261893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ee97d7490 0x562ee97d7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.865+0000 7fe861e91640 1 --2- 172.21.15.52:0/43261893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562ee97e50b0 0x562ee97e74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.865+0000 7fe860e8f640 1 -- 172.21.15.52:0/43261893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ee97d7490 msgr2=0x562ee97d7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:41.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.865+0000 7fe860e8f640 1 --2- 172.21.15.52:0/43261893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ee97d7490 0x562ee97d7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.865+0000 7fe860e8f640 1 -- 172.21.15.52:0/43261893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562ee97e50b0 msgr2=0x562ee97e74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:41.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.865+0000 7fe860e8f640 1 --2- 172.21.15.52:0/43261893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562ee97e50b0 0x562ee97e74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.865+0000 7fe860e8f640 1 -- 172.21.15.52:0/43261893 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ee944f3f0 con 0x562ee97d7db0 2024-04-02T13:54:41.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.865+0000 7fe861690640 1 --2- 172.21.15.52:0/43261893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ee97d7490 0x562ee97d7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:54:41.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.865+0000 7fe860e8f640 1 --2- 172.21.15.52:0/43261893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97d7db0 0x562ee97e4b70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe84c0114f0 tx=0x7fe84c0119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed1ae43d4c9defb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:41.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.866+0000 7fe8477fe640 1 -- 172.21.15.52:0/43261893 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c005020 con 0x562ee97d7db0 2024-04-02T13:54:41.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.866+0000 7fe8477fe640 1 -- 172.21.15.52:0/43261893 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe84c0047a0 con 0x562ee97d7db0 2024-04-02T13:54:41.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.866+0000 7fe8477fe640 1 -- 172.21.15.52:0/43261893 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c00f9e0 con 0x562ee97d7db0 2024-04-02T13:54:41.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.867+0000 7fe8665b0740 1 -- 172.21.15.52:0/43261893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97d7db0 msgr2=0x562ee97e4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.867+0000 7fe8665b0740 1 --2- 172.21.15.52:0/43261893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97d7db0 0x562ee97e4b70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe84c0114f0 tx=0x7fe84c0119b0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.867+0000 7fe8665b0740 1 -- 172.21.15.52:0/43261893 shutdown_connections 2024-04-02T13:54:41.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.867+0000 7fe8665b0740 1 --2- 172.21.15.52:0/43261893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562ee97e50b0 0x562ee97e74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.867+0000 7fe8665b0740 1 --2- 172.21.15.52:0/43261893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97d7db0 0x562ee97e4b70 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.867+0000 7fe8665b0740 1 --2- 172.21.15.52:0/43261893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ee97d7490 0x562ee97d7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.867+0000 7fe8665b0740 1 -- 172.21.15.52:0/43261893 >> 172.21.15.52:0/43261893 conn(0x562ee97d1ed0 msgr2=0x562ee97d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:41.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.867+0000 7fe8665b0740 1 -- 172.21.15.52:0/43261893 shutdown_connections 2024-04-02T13:54:41.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.867+0000 7fe8665b0740 1 -- 172.21.15.52:0/43261893 wait complete. 2024-04-02T13:54:41.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.867+0000 7fe8665b0740 1 -- start start 2024-04-02T13:54:41.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.867+0000 7fe8665b0740 1 Processor -- start 2024-04-02T13:54:41.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.868+0000 7fe8665b0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ee97d7490 0x562ee97e1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.868+0000 7fe8665b0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562ee97e50b0 0x562ee97e2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.868+0000 7fe8665b0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97e2940 0x562ee97f13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.868+0000 7fe861e91640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97e2940 0x562ee97f13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.868+0000 7fe861e91640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97e2940 0x562ee97f13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34816/0 (socket says 172.21.15.52:34816) 2024-04-02T13:54:41.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.868+0000 7fe860e8f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562ee97e50b0 0x562ee97e2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.868+0000 7fe861690640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ee97d7490 0x562ee97e1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.868+0000 7fe861e91640 1 -- 172.21.15.52:0/44146986 learned_addr learned my addr 172.21.15.52:0/44146986 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:41.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.868+0000 7fe8665b0740 1 -- 172.21.15.52:0/44146986 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562ee97e7ea0 con 0x562ee97e2940 2024-04-02T13:54:41.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.868+0000 7fe8665b0740 1 -- 172.21.15.52:0/44146986 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x562ee96f6e80 con 0x562ee97e50b0 2024-04-02T13:54:41.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.868+0000 7fe8665b0740 1 -- 172.21.15.52:0/44146986 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562ee97e8020 con 0x562ee97d7490 2024-04-02T13:54:41.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.868+0000 7fe8665b0740 10 client.0 _ll_register_callbacks cb 0x562ee9732aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:41.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.869+0000 7fe861e91640 1 -- 172.21.15.52:0/44146986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ee97d7490 msgr2=0x562ee97e1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:41.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.869+0000 7fe861e91640 1 --2- 172.21.15.52:0/44146986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ee97d7490 0x562ee97e1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.869+0000 7fe861e91640 1 -- 172.21.15.52:0/44146986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562ee97e50b0 msgr2=0x562ee97e2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:41.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.869+0000 7fe861e91640 1 --2- 172.21.15.52:0/44146986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562ee97e50b0 0x562ee97e2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.869+0000 7fe861e91640 1 -- 172.21.15.52:0/44146986 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ee95197d0 con 0x562ee97e2940 2024-04-02T13:54:41.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.869+0000 7fe861e91640 1 --2- 172.21.15.52:0/44146986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97e2940 0x562ee97f13e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe854004af0 tx=0x7fe8540bf500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec897e36bb5a314 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:41.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.869+0000 7fe844ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:41.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.869+0000 7fe844ff9640 1 -- 172.21.15.52:0/44146986 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8540c7a70 con 0x562ee97e2940 2024-04-02T13:54:41.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.869+0000 7fe8665b0740 1 -- 172.21.15.52:0/44146986 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe84c0047a0 con 0x562ee97e2940 2024-04-02T13:54:41.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.869+0000 7fe844ff9640 1 -- 172.21.15.52:0/44146986 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe8540c75e0 con 0x562ee97e2940 2024-04-02T13:54:41.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.870+0000 7fe844ff9640 1 -- 172.21.15.52:0/44146986 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe8540c77e0 con 0x562ee97e2940 2024-04-02T13:54:41.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.870+0000 7fe8665b0740 10 client.15996 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:41.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.870+0000 7fe8665b0740 1 -- 172.21.15.52:0/44146986 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562ee97f4460 con 0x562ee97e2940 2024-04-02T13:54:41.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.870+0000 7fe844ff9640 1 -- 172.21.15.52:0/44146986 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe84c0047a0 con 0x562ee97e2940 2024-04-02T13:54:41.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.870+0000 7fe844ff9640 1 -- 172.21.15.52:0/44146986 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe854003970 con 0x562ee97e2940 2024-04-02T13:54:41.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.870+0000 7fe8665b0740 10 client.15996 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:41.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.870+0000 7fe8665b0740 10 client.15996 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T13:54:41.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.870+0000 7fe8665b0740 20 client.15996 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:41.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.870+0000 7fe8665b0740 1 --2- 172.21.15.52:0/44146986 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x562ee97f5800 0x562ee97f7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.870+0000 7fe8665b0740 4 client.15996 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:54:41.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.870+0000 7fe8665b0740 1 -- 172.21.15.52:0/44146986 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ee9748cf0 con 0x562ee97f5800 2024-04-02T13:54:41.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.871+0000 7fe861690640 1 --2- 172.21.15.52:0/44146986 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x562ee97f5800 0x562ee97f7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.871+0000 7fe861690640 1 --2- 172.21.15.52:0/44146986 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x562ee97f5800 0x562ee97f7c10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9521647932139c23 server_cookie=90272d48b7818fdd in_seq=0 out_seq=0 2024-04-02T13:54:41.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.871+0000 7fe844ff9640 10 client.15996 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:54:41.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.872+0000 7fe844ff9640 1 -- 172.21.15.52:0/44146986 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562ee9748cf0 con 0x562ee97f5800 2024-04-02T13:54:41.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.872+0000 7fe844ff9640 10 client.15996 handle_command_reply: tid=0 2024-04-02T13:54:41.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.872+0000 7fe8665b0740 1 client.15996 shutdown 2024-04-02T13:54:41.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.872+0000 7fe8665b0740 1 -- 172.21.15.52:0/44146986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97e2940 msgr2=0x562ee97f13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.872+0000 7fe8665b0740 1 --2- 172.21.15.52:0/44146986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97e2940 0x562ee97f13e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe854004af0 tx=0x7fe8540bf500 comp rx=0 tx=0).stop 2024-04-02T13:54:41.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.872+0000 7fe8665b0740 1 -- 172.21.15.52:0/44146986 shutdown_connections 2024-04-02T13:54:41.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.872+0000 7fe8665b0740 1 --2- 172.21.15.52:0/44146986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97e2940 0x562ee97f13e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.872+0000 7fe8665b0740 1 --2- 172.21.15.52:0/44146986 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x562ee97f5800 0x562ee97f7c10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.872+0000 7fe844ff9640 0 client.15996 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:54:41.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.872+0000 7fe8665b0740 1 --2- 172.21.15.52:0/44146986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562ee97e50b0 0x562ee97e2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.872+0000 7fe8665b0740 1 --2- 172.21.15.52:0/44146986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ee97d7490 0x562ee97e1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.872+0000 7fe8665b0740 1 -- 172.21.15.52:0/44146986 >> 172.21.15.52:0/44146986 conn(0x562ee97d1ed0 msgr2=0x562ee97d3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:41.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.872+0000 7fe8665b0740 1 -- 172.21.15.52:0/44146986 shutdown_connections 2024-04-02T13:54:41.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.872+0000 7fe8665b0740 1 -- 172.21.15.52:0/44146986 wait complete. 2024-04-02T13:54:41.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.873+0000 7fe8665b0740 20 client.15996 trim_cache size 0 max 16384 2024-04-02T13:54:41.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.886+0000 7fe8665b0740 1 Processor -- start 2024-04-02T13:54:41.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.886+0000 7fe8665b0740 1 -- start start 2024-04-02T13:54:41.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.886+0000 7fe8665b0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ee97d7490 0x562ee97fd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.886+0000 7fe8665b0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562ee97e2940 0x562ee97fda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.886+0000 7fe8665b0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97e50b0 0x562ee97f0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.886+0000 7fe8665b0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe84000fd60 con 0x562ee97e50b0 2024-04-02T13:54:41.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.886+0000 7fe8665b0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe8400118a0 con 0x562ee97e2940 2024-04-02T13:54:41.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.886+0000 7fe8665b0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe840010430 con 0x562ee97d7490 2024-04-02T13:54:41.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.887+0000 7fe861e91640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97e50b0 0x562ee97f0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.887+0000 7fe860e8f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562ee97e2940 0x562ee97fda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.887+0000 7fe860e8f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562ee97e2940 0x562ee97fda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:56214/0 (socket says 172.21.15.52:56214) 2024-04-02T13:54:41.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.887+0000 7fe860e8f640 1 -- 172.21.15.52:0/3918866252 learned_addr learned my addr 172.21.15.52:0/3918866252 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:41.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.887+0000 7fe861e91640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97e50b0 0x562ee97f0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34822/0 (socket says 172.21.15.52:34822) 2024-04-02T13:54:41.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.887+0000 7fe861690640 1 --2- 172.21.15.52:0/3918866252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ee97d7490 0x562ee97fd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.887+0000 7fe861e91640 1 -- 172.21.15.52:0/3918866252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ee97d7490 msgr2=0x562ee97fd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:41.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.887+0000 7fe861e91640 1 --2- 172.21.15.52:0/3918866252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ee97d7490 0x562ee97fd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.887+0000 7fe861e91640 1 -- 172.21.15.52:0/3918866252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562ee97e2940 msgr2=0x562ee97fda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:41.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.887+0000 7fe861e91640 1 --2- 172.21.15.52:0/3918866252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562ee97e2940 0x562ee97fda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.887+0000 7fe861e91640 1 -- 172.21.15.52:0/3918866252 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ee97e2e80 con 0x562ee97e50b0 2024-04-02T13:54:41.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.887+0000 7fe860e8f640 1 --2- 172.21.15.52:0/3918866252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562ee97e2940 0x562ee97fda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:54:41.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.887+0000 7fe861e91640 1 --2- 172.21.15.52:0/3918866252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97e50b0 0x562ee97f0e30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe8540befc0 tx=0x7fe85402bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a37e0298bf5f98 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:41.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.887+0000 7fe846ffd640 1 -- 172.21.15.52:0/3918866252 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8540c7a70 con 0x562ee97e50b0 2024-04-02T13:54:41.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.888+0000 7fe846ffd640 1 -- 172.21.15.52:0/3918866252 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8540c75e0 con 0x562ee97e50b0 2024-04-02T13:54:41.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.888+0000 7fe846ffd640 1 -- 172.21.15.52:0/3918866252 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8540bebd0 con 0x562ee97e50b0 2024-04-02T13:54:41.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.888+0000 7fe8665b0740 1 -- 172.21.15.52:0/3918866252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97e50b0 msgr2=0x562ee97f0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.888+0000 7fe8665b0740 1 --2- 172.21.15.52:0/3918866252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97e50b0 0x562ee97f0e30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe8540befc0 tx=0x7fe85402bef0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.889+0000 7fe8665b0740 1 -- 172.21.15.52:0/3918866252 shutdown_connections 2024-04-02T13:54:41.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.889+0000 7fe8665b0740 1 --2- 172.21.15.52:0/3918866252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97e50b0 0x562ee97f0e30 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.889+0000 7fe8665b0740 1 --2- 172.21.15.52:0/3918866252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562ee97e2940 0x562ee97fda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.889+0000 7fe8665b0740 1 --2- 172.21.15.52:0/3918866252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ee97d7490 0x562ee97fd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.889+0000 7fe8665b0740 1 -- 172.21.15.52:0/3918866252 >> 172.21.15.52:0/3918866252 conn(0x562ee97d1ed0 msgr2=0x562ee97e0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:41.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.889+0000 7fe8665b0740 1 -- 172.21.15.52:0/3918866252 shutdown_connections 2024-04-02T13:54:41.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.889+0000 7fe8665b0740 1 -- 172.21.15.52:0/3918866252 wait complete. 2024-04-02T13:54:41.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.889+0000 7fe8665b0740 1 -- start start 2024-04-02T13:54:41.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.889+0000 7fe8665b0740 1 Processor -- start 2024-04-02T13:54:41.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.890+0000 7fe8665b0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ee97d7490 0x562ee97ed820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.890+0000 7fe8665b0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562ee97e2940 0x562ee97df300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.890+0000 7fe8665b0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97e50b0 0x562ee97df880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.890+0000 7fe8665b0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe8400111d0 con 0x562ee97e50b0 2024-04-02T13:54:41.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.890+0000 7fe8665b0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe840010b00 con 0x562ee97e2940 2024-04-02T13:54:41.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.890+0000 7fe8665b0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe84000dd40 con 0x562ee97d7490 2024-04-02T13:54:41.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.890+0000 7fe8665b0740 10 client.0 _ll_register_callbacks cb 0x7fe85402cb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:41.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.890+0000 7fe861690640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ee97d7490 0x562ee97ed820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.890+0000 7fe861690640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ee97d7490 0x562ee97ed820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.52:43852/0 (socket says 172.21.15.52:43852) 2024-04-02T13:54:41.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.890+0000 7fe861e91640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97e50b0 0x562ee97df880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.890+0000 7fe861690640 1 -- 172.21.15.52:0/1963741538 learned_addr learned my addr 172.21.15.52:0/1963741538 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:41.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.890+0000 7fe860e8f640 1 --2- 172.21.15.52:0/1963741538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562ee97e2940 0x562ee97df300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.890+0000 7fe861690640 1 -- 172.21.15.52:0/1963741538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562ee97e2940 msgr2=0x562ee97df300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:41.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.890+0000 7fe861690640 1 --2- 172.21.15.52:0/1963741538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562ee97e2940 0x562ee97df300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.890+0000 7fe861690640 1 -- 172.21.15.52:0/1963741538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97e50b0 msgr2=0x562ee97df880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:41.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.890+0000 7fe861690640 1 --2- 172.21.15.52:0/1963741538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97e50b0 0x562ee97df880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.890+0000 7fe861690640 1 -- 172.21.15.52:0/1963741538 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ee97e5e20 con 0x562ee97d7490 2024-04-02T13:54:41.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.891+0000 7fe861690640 1 --2- 172.21.15.52:0/1963741538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ee97d7490 0x562ee97ed820 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe858014040 tx=0x7fe858004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=593e3e1725c55b6f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:41.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.891+0000 7fe844ff9640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-02T13:54:41.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.891+0000 7fe844ff9640 1 -- 172.21.15.52:0/1963741538 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85802c070 con 0x562ee97d7490 2024-04-02T13:54:41.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.891+0000 7fe8665b0740 1 -- 172.21.15.52:0/1963741538 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8540c75e0 con 0x562ee97d7490 2024-04-02T13:54:41.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.891+0000 7fe844ff9640 1 -- 172.21.15.52:0/1963741538 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe858028d50 con 0x562ee97d7490 2024-04-02T13:54:41.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.891+0000 7fe844ff9640 1 -- 172.21.15.52:0/1963741538 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe858038050 con 0x562ee97d7490 2024-04-02T13:54:41.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.891+0000 7fe8665b0740 10 client.25495 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:41.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.891+0000 7fe8665b0740 1 -- 172.21.15.52:0/1963741538 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562ee985c060 con 0x562ee97d7490 2024-04-02T13:54:41.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.891+0000 7fe844ff9640 1 -- 172.21.15.52:0/1963741538 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8540c75e0 con 0x562ee97d7490 2024-04-02T13:54:41.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.891+0000 7fe844ff9640 1 -- 172.21.15.52:0/1963741538 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe858020a10 con 0x562ee97d7490 2024-04-02T13:54:41.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.892+0000 7fe8665b0740 10 client.25495 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:41.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.892+0000 7fe8665b0740 10 client.25495 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T13:54:41.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.892+0000 7fe8665b0740 20 client.25495 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:41.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.892+0000 7fe8665b0740 1 --2- 172.21.15.52:0/1963741538 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x562ee985c790 0x562ee985ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:41.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.892+0000 7fe8665b0740 4 client.25495 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:54:41.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.892+0000 7fe8665b0740 1 -- 172.21.15.52:0/1963741538 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ee9743040 con 0x562ee985c790 2024-04-02T13:54:41.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.892+0000 7fe860e8f640 1 --2- 172.21.15.52:0/1963741538 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x562ee985c790 0x562ee985ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:41.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.892+0000 7fe860e8f640 1 --2- 172.21.15.52:0/1963741538 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x562ee985c790 0x562ee985ebb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=392c1b7e98d26c02 server_cookie=11e4b211333a7feb in_seq=0 out_seq=0 2024-04-02T13:54:41.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.892+0000 7fe844ff9640 10 client.25495 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:54:41.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.893+0000 7fe844ff9640 1 -- 172.21.15.52:0/1963741538 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ee9743040 con 0x562ee985c790 2024-04-02T13:54:41.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.893+0000 7fe844ff9640 10 client.25495 handle_command_reply: tid=0 2024-04-02T13:54:41.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.893+0000 7fe8665b0740 1 client.25495 shutdown 2024-04-02T13:54:41.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.893+0000 7fe8665b0740 1 -- 172.21.15.52:0/1963741538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ee97d7490 msgr2=0x562ee97ed820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.893+0000 7fe8665b0740 1 --2- 172.21.15.52:0/1963741538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ee97d7490 0x562ee97ed820 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe858014040 tx=0x7fe858004730 comp rx=0 tx=0).stop 2024-04-02T13:54:41.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.893+0000 7fe8665b0740 1 -- 172.21.15.52:0/1963741538 shutdown_connections 2024-04-02T13:54:41.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.893+0000 7fe8665b0740 1 --2- 172.21.15.52:0/1963741538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ee97e50b0 0x562ee97df880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.893+0000 7fe8665b0740 1 --2- 172.21.15.52:0/1963741538 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x562ee985c790 0x562ee985ebb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.893+0000 7fe844ff9640 0 client.25495 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:54:41.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.893+0000 7fe8665b0740 1 --2- 172.21.15.52:0/1963741538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562ee97e2940 0x562ee97df300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.893+0000 7fe8665b0740 1 --2- 172.21.15.52:0/1963741538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ee97d7490 0x562ee97ed820 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.893+0000 7fe8665b0740 1 -- 172.21.15.52:0/1963741538 >> 172.21.15.52:0/1963741538 conn(0x562ee97d1ed0 msgr2=0x562ee97e0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:41.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.893+0000 7fe8665b0740 1 -- 172.21.15.52:0/1963741538 shutdown_connections 2024-04-02T13:54:41.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.893+0000 7fe8665b0740 1 -- 172.21.15.52:0/1963741538 wait complete. 2024-04-02T13:54:41.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.893+0000 7fe8665b0740 20 client.25495 trim_cache size 0 max 16384 2024-04-02T13:54:41.904 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:54:41.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.894+0000 7fe8638f4640 1 -- 172.21.15.52:0/2594929544 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe8480a3f50 msgr2=0x7fe8480a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.894+0000 7fe8638f4640 1 --2- 172.21.15.52:0/2594929544 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe8480a3f50 0x7fe8480a6410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe858005e00 tx=0x7fe858005c70 comp rx=0 tx=0).stop 2024-04-02T13:54:41.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.894+0000 7fe8638f4640 1 -- 172.21.15.52:0/2594929544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe85c106980 msgr2=0x7fe85c1a41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:41.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.894+0000 7fe8638f4640 1 --2- 172.21.15.52:0/2594929544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe85c106980 0x7fe85c1a41e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe8540096f0 tx=0x7fe8540336b0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.894+0000 7fe8638f4640 1 -- 172.21.15.52:0/2594929544 shutdown_connections 2024-04-02T13:54:41.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.894+0000 7fe8638f4640 1 --2- 172.21.15.52:0/2594929544 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe8480a3f50 0x7fe8480a6410 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.894+0000 7fe8638f4640 1 --2- 172.21.15.52:0/2594929544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe85c106980 0x7fe85c1a41e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.894+0000 7fe8638f4640 1 --2- 172.21.15.52:0/2594929544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe85c101330 0x7fe85c1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.894+0000 7fe8638f4640 1 --2- 172.21.15.52:0/2594929544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe85c100970 0x7fe85c199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:41.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.894+0000 7fe8638f4640 1 -- 172.21.15.52:0/2594929544 >> 172.21.15.52:0/2594929544 conn(0x7fe85c077b00 msgr2=0x7fe85c103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:41.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.894+0000 7fe8638f4640 1 -- 172.21.15.52:0/2594929544 shutdown_connections 2024-04-02T13:54:41.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:41.894+0000 7fe8638f4640 1 -- 172.21.15.52:0/2594929544 wait complete. 2024-04-02T13:54:41.909 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:54:42.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.013+0000 7f4ecf35f640 1 -- 172.21.15.52:0/656257277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ec8068d00 msgr2=0x7f4ec8069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.013+0000 7f4ecf35f640 1 --2- 172.21.15.52:0/656257277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ec8068d00 0x7f4ec8069180 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8009930 tx=0x7f4eb802bc30 comp rx=0 tx=0).stop 2024-04-02T13:54:42.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.013+0000 7f4ecf35f640 1 -- 172.21.15.52:0/656257277 shutdown_connections 2024-04-02T13:54:42.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.013+0000 7f4ecf35f640 1 --2- 172.21.15.52:0/656257277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ec80696c0 0x7f4ec810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.013+0000 7f4ecf35f640 1 --2- 172.21.15.52:0/656257277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ec8068d00 0x7f4ec8069180 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.013+0000 7f4ecf35f640 1 --2- 172.21.15.52:0/656257277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ec8107b50 0x7f4ec8107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.013+0000 7f4ecf35f640 1 -- 172.21.15.52:0/656257277 >> 172.21.15.52:0/656257277 conn(0x7f4ec8077b00 msgr2=0x7f4ec8077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:42.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.013+0000 7f4ecf35f640 1 -- 172.21.15.52:0/656257277 shutdown_connections 2024-04-02T13:54:42.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.013+0000 7f4ecf35f640 1 -- 172.21.15.52:0/656257277 wait complete. 2024-04-02T13:54:42.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.014+0000 7f4ecf35f640 1 Processor -- start 2024-04-02T13:54:42.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.014+0000 7f4ecf35f640 1 -- start start 2024-04-02T13:54:42.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.014+0000 7f4ecf35f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ec8068d00 0x7f4ec81009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.014+0000 7f4ecf35f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ec80696c0 0x7f4ec8100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.014+0000 7f4ecf35f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ec8107b50 0x7f4ec8101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.014+0000 7f4ecf35f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec8111ea0 con 0x7f4ec8068d00 2024-04-02T13:54:42.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.014+0000 7f4ecf35f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec8111d20 con 0x7f4ec8107b50 2024-04-02T13:54:42.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.014+0000 7f4ecf35f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec8112020 con 0x7f4ec80696c0 2024-04-02T13:54:42.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.014+0000 7f4ecd0fb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ec8068d00 0x7f4ec81009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.014+0000 7f4ecd0fb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ec8068d00 0x7f4ec81009c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34842/0 (socket says 172.21.15.52:34842) 2024-04-02T13:54:42.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.014+0000 7f4ecd0fb640 1 -- 172.21.15.52:0/1448192005 learned_addr learned my addr 172.21.15.52:0/1448192005 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:42.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.014+0000 7f4ecd8fc640 1 --2- 172.21.15.52:0/1448192005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ec8107b50 0x7f4ec8101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.014+0000 7f4ecc8fa640 1 --2- 172.21.15.52:0/1448192005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ec80696c0 0x7f4ec8100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.015+0000 7f4ecd0fb640 1 -- 172.21.15.52:0/1448192005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ec80696c0 msgr2=0x7f4ec8100f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.015+0000 7f4ecd0fb640 1 --2- 172.21.15.52:0/1448192005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ec80696c0 0x7f4ec8100f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.015+0000 7f4ecd0fb640 1 -- 172.21.15.52:0/1448192005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ec8107b50 msgr2=0x7f4ec8101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.015+0000 7f4ecd0fb640 1 --2- 172.21.15.52:0/1448192005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ec8107b50 0x7f4ec8101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.015+0000 7f4ecd0fb640 1 -- 172.21.15.52:0/1448192005 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec81a6860 con 0x7f4ec8068d00 2024-04-02T13:54:42.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.015+0000 7f4ecd0fb640 1 --2- 172.21.15.52:0/1448192005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ec8068d00 0x7f4ec81009c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400e9f0 tx=0x7f4ec400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:42.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.015+0000 7f4ebe7fc640 1 -- 172.21.15.52:0/1448192005 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec400cd80 con 0x7f4ec8068d00 2024-04-02T13:54:42.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.015+0000 7f4ebe7fc640 1 -- 172.21.15.52:0/1448192005 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ec4010910 con 0x7f4ec8068d00 2024-04-02T13:54:42.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.015+0000 7f4ecf35f640 1 -- 172.21.15.52:0/1448192005 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec81a6b50 con 0x7f4ec8068d00 2024-04-02T13:54:42.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.015+0000 7f4ebe7fc640 1 -- 172.21.15.52:0/1448192005 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec40188f0 con 0x7f4ec8068d00 2024-04-02T13:54:42.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.015+0000 7f4ecf35f640 1 -- 172.21.15.52:0/1448192005 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec81a7070 con 0x7f4ec8068d00 2024-04-02T13:54:42.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.018+0000 7f4ebe7fc640 1 -- 172.21.15.52:0/1448192005 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f4ec4010ac0 con 0x7f4ec8068d00 2024-04-02T13:54:42.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.019+0000 7f4ed201f740 1 Processor -- start 2024-04-02T13:54:42.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.019+0000 7f4ed201f740 1 -- start start 2024-04-02T13:54:42.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.019+0000 7f4ed201f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564bfacce7a0 0x564bfacceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.019+0000 7f4ed201f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfaccf0c0 0x564bfacd7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.019+0000 7f4ebe7fc640 1 --2- 172.21.15.52:0/1448192005 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4eb40a3ed0 0x7f4eb40a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.019+0000 7f4ed201f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564bfacd7ca0 0x564bfacda090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.019+0000 7f4ed201f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564bfac327b0 con 0x564bfaccf0c0 2024-04-02T13:54:42.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.019+0000 7f4ed201f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564bfabd1570 con 0x564bfacce7a0 2024-04-02T13:54:42.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.019+0000 7f4ed201f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564bfac2e760 con 0x564bfacd7ca0 2024-04-02T13:54:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.019+0000 7f4ecd0fb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfaccf0c0 0x564bfacd7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.019+0000 7f4ecd0fb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfaccf0c0 0x564bfacd7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34852/0 (socket says 172.21.15.52:34852) 2024-04-02T13:54:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.019+0000 7f4ecd0fb640 1 -- 172.21.15.52:0/2648032250 learned_addr learned my addr 172.21.15.52:0/2648032250 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.019+0000 7f4ecd8fc640 1 --2- 172.21.15.52:0/2648032250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564bfacce7a0 0x564bfacceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.019+0000 7f4ecd8fc640 1 --2- 172.21.15.52:0/2648032250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564bfacd7ca0 0x564bfacda090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.019+0000 7f4ecc8fa640 1 --2- 172.21.15.52:0/1448192005 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4eb40a3ed0 0x7f4eb40a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.020+0000 7f4ebe7fc640 1 -- 172.21.15.52:0/1448192005 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f4ec40d1410 con 0x7f4ec8068d00 2024-04-02T13:54:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.020+0000 7f4ecd0fb640 1 -- 172.21.15.52:0/2648032250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564bfacd7ca0 msgr2=0x564bfacda090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.020+0000 7f4ecd0fb640 1 --2- 172.21.15.52:0/2648032250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564bfacd7ca0 0x564bfacda090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.020+0000 7f4ecd0fb640 1 -- 172.21.15.52:0/2648032250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564bfacce7a0 msgr2=0x564bfacceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.020+0000 7f4ecd0fb640 1 --2- 172.21.15.52:0/2648032250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564bfacce7a0 0x564bfacceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.020+0000 7f4ecd0fb640 1 -- 172.21.15.52:0/2648032250 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bfa9463f0 con 0x564bfaccf0c0 2024-04-02T13:54:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.020+0000 7f4ecd0fb640 1 --2- 172.21.15.52:0/2648032250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfaccf0c0 0x564bfacd7760 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4ec409cb30 tx=0x7f4ec40b6790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1fc2ab07cbf6c47 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.020+0000 7f4eb37fe640 1 -- 172.21.15.52:0/2648032250 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec40a0040 con 0x564bfaccf0c0 2024-04-02T13:54:42.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.020+0000 7f4ecc8fa640 1 --2- 172.21.15.52:0/1448192005 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4eb40a3ed0 0x7f4eb40a6390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8006010 tx=0x7f4eb802c650 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:42.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.020+0000 7f4eb37fe640 1 -- 172.21.15.52:0/2648032250 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ec409d530 con 0x564bfaccf0c0 2024-04-02T13:54:42.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.020+0000 7f4eb37fe640 1 -- 172.21.15.52:0/2648032250 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec40bc250 con 0x564bfaccf0c0 2024-04-02T13:54:42.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.021+0000 7f4ed201f740 1 -- 172.21.15.52:0/2648032250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfaccf0c0 msgr2=0x564bfacd7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.021+0000 7f4ed201f740 1 --2- 172.21.15.52:0/2648032250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfaccf0c0 0x564bfacd7760 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4ec409cb30 tx=0x7f4ec40b6790 comp rx=0 tx=0).stop 2024-04-02T13:54:42.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.021+0000 7f4ed201f740 1 -- 172.21.15.52:0/2648032250 shutdown_connections 2024-04-02T13:54:42.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.021+0000 7f4ed201f740 1 --2- 172.21.15.52:0/2648032250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564bfacd7ca0 0x564bfacda090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.021+0000 7f4ed201f740 1 --2- 172.21.15.52:0/2648032250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfaccf0c0 0x564bfacd7760 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.021+0000 7f4ed201f740 1 --2- 172.21.15.52:0/2648032250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564bfacce7a0 0x564bfacceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.021+0000 7f4ed201f740 1 -- 172.21.15.52:0/2648032250 >> 172.21.15.52:0/2648032250 conn(0x564bfacc8ed0 msgr2=0x564bfacc92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:42.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.021+0000 7f4ed201f740 1 -- 172.21.15.52:0/2648032250 shutdown_connections 2024-04-02T13:54:42.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.021+0000 7f4ed201f740 1 -- 172.21.15.52:0/2648032250 wait complete. 2024-04-02T13:54:42.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.022+0000 7f4ed201f740 1 -- start start 2024-04-02T13:54:42.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.022+0000 7f4ed201f740 1 Processor -- start 2024-04-02T13:54:42.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.022+0000 7f4ed201f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564bfacce7a0 0x564bfacd4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.022+0000 7f4ed201f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfaccf0c0 0x564bfacd5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.022+0000 7f4ed201f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564bfacd7ca0 0x564bfacd55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.022+0000 7f4ed201f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564bfacdaa90 con 0x564bfaccf0c0 2024-04-02T13:54:42.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.022+0000 7f4ed201f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564bfabede80 con 0x564bfacce7a0 2024-04-02T13:54:42.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.022+0000 7f4ed201f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564bfacdac10 con 0x564bfacd7ca0 2024-04-02T13:54:42.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.022+0000 7f4ecc8fa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfaccf0c0 0x564bfacd5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.022+0000 7f4ecc8fa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfaccf0c0 0x564bfacd5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34868/0 (socket says 172.21.15.52:34868) 2024-04-02T13:54:42.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.022+0000 7f4ecc8fa640 1 -- 172.21.15.52:0/1858892485 learned_addr learned my addr 172.21.15.52:0/1858892485 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:42.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.022+0000 7f4ecd0fb640 1 --2- 172.21.15.52:0/1858892485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564bfacce7a0 0x564bfacd4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.022+0000 7f4ed201f740 10 client.0 _ll_register_callbacks cb 0x564bfac29aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:42.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.023+0000 7f4ecd8fc640 1 --2- 172.21.15.52:0/1858892485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564bfacd7ca0 0x564bfacd55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.023+0000 7f4ecc8fa640 1 -- 172.21.15.52:0/1858892485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564bfacd7ca0 msgr2=0x564bfacd55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:42.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.023+0000 7f4ecc8fa640 1 --2- 172.21.15.52:0/1858892485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564bfacd7ca0 0x564bfacd55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.023+0000 7f4ecc8fa640 1 -- 172.21.15.52:0/1858892485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564bfacce7a0 msgr2=0x564bfacd4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:42.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.023+0000 7f4ecc8fa640 1 --2- 172.21.15.52:0/1858892485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564bfacce7a0 0x564bfacd4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.023+0000 7f4ecc8fa640 1 -- 172.21.15.52:0/1858892485 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bfaa107d0 con 0x564bfaccf0c0 2024-04-02T13:54:42.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.023+0000 7f4ecd0fb640 1 --2- 172.21.15.52:0/1858892485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564bfacce7a0 0x564bfacd4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:54:42.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.023+0000 7f4ecc8fa640 1 --2- 172.21.15.52:0/1858892485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfaccf0c0 0x564bfacd5090 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8000c00 tx=0x7f4eb8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb1cdc87bb6dc6f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:42.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.023+0000 7f4eb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:42.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.023+0000 7f4eb0ff9640 1 -- 172.21.15.52:0/1858892485 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb804e020 con 0x564bfaccf0c0 2024-04-02T13:54:42.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.023+0000 7f4eb0ff9640 1 -- 172.21.15.52:0/1858892485 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f4eb8005510 con 0x564bfaccf0c0 2024-04-02T13:54:42.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.023+0000 7f4ed201f740 1 -- 172.21.15.52:0/1858892485 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ec409d530 con 0x564bfaccf0c0 2024-04-02T13:54:42.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.023+0000 7f4eb0ff9640 1 -- 172.21.15.52:0/1858892485 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4eb80046c0 con 0x564bfaccf0c0 2024-04-02T13:54:42.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.023+0000 7f4ed201f740 10 client.16014 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:42.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.023+0000 7f4ed201f740 1 -- 172.21.15.52:0/1858892485 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564bface6ba0 con 0x564bfaccf0c0 2024-04-02T13:54:42.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.024+0000 7f4eb0ff9640 1 -- 172.21.15.52:0/1858892485 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ec409d530 con 0x564bfaccf0c0 2024-04-02T13:54:42.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.024+0000 7f4eb0ff9640 1 -- 172.21.15.52:0/1858892485 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4eb8003bb0 con 0x564bfaccf0c0 2024-04-02T13:54:42.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.024+0000 7f4ed201f740 10 client.16014 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:42.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.024+0000 7f4ed201f740 10 client.16014 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T13:54:42.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.024+0000 7f4ed201f740 20 client.16014 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:42.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.024+0000 7f4ed201f740 1 --2- 172.21.15.52:0/1858892485 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x564bface8340 0x564bfacea7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.024+0000 7f4ed201f740 4 client.16014 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:54:42.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.024+0000 7f4ed201f740 1 -- 172.21.15.52:0/1858892485 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bfac3fcf0 con 0x564bface8340 2024-04-02T13:54:42.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.024+0000 7f4ecd0fb640 1 --2- 172.21.15.52:0/1858892485 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x564bface8340 0x564bfacea7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.025+0000 7f4ecd0fb640 1 --2- 172.21.15.52:0/1858892485 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x564bface8340 0x564bfacea7e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ae92c36010695a02 server_cookie=60a1608f0310c1b3 in_seq=0 out_seq=0 2024-04-02T13:54:42.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.025+0000 7f4eb0ff9640 10 client.16014 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:54:42.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.026+0000 7f4eb0ff9640 1 -- 172.21.15.52:0/1858892485 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564bfac3fcf0 con 0x564bface8340 2024-04-02T13:54:42.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.026+0000 7f4eb0ff9640 10 client.16014 handle_command_reply: tid=0 2024-04-02T13:54:42.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.026+0000 7f4ed201f740 1 client.16014 shutdown 2024-04-02T13:54:42.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.026+0000 7f4ed201f740 1 -- 172.21.15.52:0/1858892485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfaccf0c0 msgr2=0x564bfacd5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.026+0000 7f4ed201f740 1 --2- 172.21.15.52:0/1858892485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfaccf0c0 0x564bfacd5090 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8000c00 tx=0x7f4eb8009260 comp rx=0 tx=0).stop 2024-04-02T13:54:42.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.026+0000 7f4ed201f740 1 -- 172.21.15.52:0/1858892485 shutdown_connections 2024-04-02T13:54:42.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.026+0000 7f4ed201f740 1 --2- 172.21.15.52:0/1858892485 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x564bface8340 0x564bfacea7e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.026+0000 7f4ed201f740 1 --2- 172.21.15.52:0/1858892485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564bfacd7ca0 0x564bfacd55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.026+0000 7f4ed201f740 1 --2- 172.21.15.52:0/1858892485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfaccf0c0 0x564bfacd5090 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.026+0000 7f4ed201f740 1 --2- 172.21.15.52:0/1858892485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564bfacce7a0 0x564bfacd4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.026+0000 7f4ed201f740 1 -- 172.21.15.52:0/1858892485 >> 172.21.15.52:0/1858892485 conn(0x564bfacc8ed0 msgr2=0x564bfaccac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:42.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.026+0000 7f4eb0ff9640 0 client.16014 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:54:42.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.026+0000 7f4ed201f740 1 -- 172.21.15.52:0/1858892485 shutdown_connections 2024-04-02T13:54:42.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.026+0000 7f4ed201f740 1 -- 172.21.15.52:0/1858892485 wait complete. 2024-04-02T13:54:42.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.026+0000 7f4ed201f740 20 client.16014 trim_cache size 0 max 16384 2024-04-02T13:54:42.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.040+0000 7f4ed201f740 1 Processor -- start 2024-04-02T13:54:42.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.040+0000 7f4ed201f740 1 -- start start 2024-04-02T13:54:42.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.040+0000 7f4ed201f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564bfacce7a0 0x564bfacd7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.040+0000 7f4ed201f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfacd7ca0 0x564bface3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.040+0000 7f4ed201f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564bfacd3cd0 0x564bfacf00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.040+0000 7f4ed201f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81b1180 con 0x564bfacd7ca0 2024-04-02T13:54:42.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.040+0000 7f4ed201f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81b2cc0 con 0x564bfacd3cd0 2024-04-02T13:54:42.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.040+0000 7f4ed201f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81b1850 con 0x564bfacce7a0 2024-04-02T13:54:42.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.040+0000 7f4ecc8fa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfacd7ca0 0x564bface3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.040+0000 7f4ecc8fa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfacd7ca0 0x564bface3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34870/0 (socket says 172.21.15.52:34870) 2024-04-02T13:54:42.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.040+0000 7f4ecc8fa640 1 -- 172.21.15.52:0/1643069198 learned_addr learned my addr 172.21.15.52:0/1643069198 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:42.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.040+0000 7f4ecd0fb640 1 --2- 172.21.15.52:0/1643069198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564bfacce7a0 0x564bfacd7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.040+0000 7f4ecd8fc640 1 --2- 172.21.15.52:0/1643069198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564bfacd3cd0 0x564bfacf00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.041+0000 7f4ecc8fa640 1 -- 172.21.15.52:0/1643069198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564bfacce7a0 msgr2=0x564bfacd7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:42.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.041+0000 7f4ecc8fa640 1 --2- 172.21.15.52:0/1643069198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564bfacce7a0 0x564bfacd7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.041+0000 7f4ecc8fa640 1 -- 172.21.15.52:0/1643069198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564bfacd3cd0 msgr2=0x564bfacf00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:42.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.041+0000 7f4ecc8fa640 1 --2- 172.21.15.52:0/1643069198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564bfacd3cd0 0x564bfacf00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.041+0000 7f4ecc8fa640 1 -- 172.21.15.52:0/1643069198 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bfacc47a0 con 0x564bfacd7ca0 2024-04-02T13:54:42.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.041+0000 7f4ecc8fa640 1 --2- 172.21.15.52:0/1643069198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfacd7ca0 0x564bface3950 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8034650 tx=0x7f4eb802c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f341e23f87d5b083 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:42.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.041+0000 7f4eb2ffd640 1 -- 172.21.15.52:0/1643069198 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8061020 con 0x564bfacd7ca0 2024-04-02T13:54:42.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.041+0000 7f4eb2ffd640 1 -- 172.21.15.52:0/1643069198 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eb8004d40 con 0x564bfacd7ca0 2024-04-02T13:54:42.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.041+0000 7f4eb2ffd640 1 -- 172.21.15.52:0/1643069198 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb804e040 con 0x564bfacd7ca0 2024-04-02T13:54:42.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.042+0000 7f4ed201f740 1 -- 172.21.15.52:0/1643069198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfacd7ca0 msgr2=0x564bface3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.042+0000 7f4ed201f740 1 --2- 172.21.15.52:0/1643069198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfacd7ca0 0x564bface3950 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8034650 tx=0x7f4eb802c350 comp rx=0 tx=0).stop 2024-04-02T13:54:42.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.042+0000 7f4ed201f740 1 -- 172.21.15.52:0/1643069198 shutdown_connections 2024-04-02T13:54:42.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.042+0000 7f4ed201f740 1 --2- 172.21.15.52:0/1643069198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564bfacd3cd0 0x564bfacf00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.042+0000 7f4ed201f740 1 --2- 172.21.15.52:0/1643069198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfacd7ca0 0x564bface3950 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.042+0000 7f4ed201f740 1 --2- 172.21.15.52:0/1643069198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564bfacce7a0 0x564bfacd7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.042+0000 7f4ed201f740 1 -- 172.21.15.52:0/1643069198 >> 172.21.15.52:0/1643069198 conn(0x564bfacc8ed0 msgr2=0x564bfacc92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:42.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.042+0000 7f4ed201f740 1 -- 172.21.15.52:0/1643069198 shutdown_connections 2024-04-02T13:54:42.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.042+0000 7f4ed201f740 1 -- 172.21.15.52:0/1643069198 wait complete. 2024-04-02T13:54:42.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.042+0000 7f4ed201f740 1 -- start start 2024-04-02T13:54:42.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.042+0000 7f4ed201f740 1 Processor -- start 2024-04-02T13:54:42.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.043+0000 7f4ed201f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfacce7a0 0x564bfacdfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.043+0000 7f4ed201f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564bfacd3cd0 0x564bface03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.043+0000 7f4ecd0fb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfacce7a0 0x564bfacdfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.043+0000 7f4ecd0fb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfacce7a0 0x564bfacdfe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34884/0 (socket says 172.21.15.52:34884) 2024-04-02T13:54:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.043+0000 7f4ed201f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564bfacd7ca0 0x564bfacd0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.043+0000 7f4ecd0fb640 1 -- 172.21.15.52:0/3123058830 learned_addr learned my addr 172.21.15.52:0/3123058830 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.043+0000 7f4ecc8fa640 1 --2- 172.21.15.52:0/3123058830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564bfacd3cd0 0x564bface03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.043+0000 7f4ed201f740 1 -- 172.21.15.52:0/3123058830 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81b25f0 con 0x564bfacce7a0 2024-04-02T13:54:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.043+0000 7f4ed201f740 1 -- 172.21.15.52:0/3123058830 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81b1f20 con 0x564bfacd3cd0 2024-04-02T13:54:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.043+0000 7f4ed201f740 1 -- 172.21.15.52:0/3123058830 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81af160 con 0x564bfacd7ca0 2024-04-02T13:54:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.043+0000 7f4ecd8fc640 1 --2- 172.21.15.52:0/3123058830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564bfacd7ca0 0x564bfacd0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.043+0000 7f4ed201f740 10 client.0 _ll_register_callbacks cb 0x7f4ec4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.043+0000 7f4ecd0fb640 1 -- 172.21.15.52:0/3123058830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564bfacd7ca0 msgr2=0x564bfacd0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.043+0000 7f4ecd0fb640 1 --2- 172.21.15.52:0/3123058830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564bfacd7ca0 0x564bfacd0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.043+0000 7f4ecd0fb640 1 -- 172.21.15.52:0/3123058830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564bfacd3cd0 msgr2=0x564bface03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.043+0000 7f4ecd0fb640 1 --2- 172.21.15.52:0/3123058830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564bfacd3cd0 0x564bface03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.043+0000 7f4ecd0fb640 1 -- 172.21.15.52:0/3123058830 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bfacd1410 con 0x564bfacce7a0 2024-04-02T13:54:42.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.044+0000 7f4ecd0fb640 1 --2- 172.21.15.52:0/3123058830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfacce7a0 0x564bfacdfe60 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4099290 tx=0x7f4ec400c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb26380547d4ccc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:42.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.044+0000 7f4eb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:42.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.044+0000 7f4eb0ff9640 1 -- 172.21.15.52:0/3123058830 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec40b62d0 con 0x564bfacce7a0 2024-04-02T13:54:42.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.044+0000 7f4eb0ff9640 1 -- 172.21.15.52:0/3123058830 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f4ec40b8d60 con 0x564bfacce7a0 2024-04-02T13:54:42.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.044+0000 7f4ed201f740 1 -- 172.21.15.52:0/3123058830 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eb8004d40 con 0x564bfacce7a0 2024-04-02T13:54:42.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.044+0000 7f4eb0ff9640 1 -- 172.21.15.52:0/3123058830 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4ec40d11b0 con 0x564bfacce7a0 2024-04-02T13:54:42.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.044+0000 7f4ed201f740 10 client.16020 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:42.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.044+0000 7f4ed201f740 1 -- 172.21.15.52:0/3123058830 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564bfad4e1c0 con 0x564bfacce7a0 2024-04-02T13:54:42.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.044+0000 7f4eb0ff9640 1 -- 172.21.15.52:0/3123058830 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eb8004d40 con 0x564bfacce7a0 2024-04-02T13:54:42.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.045+0000 7f4eb0ff9640 1 -- 172.21.15.52:0/3123058830 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4ec409c280 con 0x564bfacce7a0 2024-04-02T13:54:42.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.045+0000 7f4ed201f740 10 client.16020 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:42.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.045+0000 7f4ed201f740 10 client.16020 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T13:54:42.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.045+0000 7f4ed201f740 20 client.16020 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:42.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.045+0000 7f4ed201f740 1 --2- 172.21.15.52:0/3123058830 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x564bfad4e450 0x564bfad50840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.045+0000 7f4ed201f740 4 client.16020 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:54:42.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.045+0000 7f4ed201f740 1 -- 172.21.15.52:0/3123058830 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564bfac3a040 con 0x564bfad4e450 2024-04-02T13:54:42.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.045+0000 7f4ecc8fa640 1 --2- 172.21.15.52:0/3123058830 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x564bfad4e450 0x564bfad50840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.045+0000 7f4eb0ff9640 10 client.16020 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:54:42.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.045+0000 7f4ecc8fa640 1 --2- 172.21.15.52:0/3123058830 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x564bfad4e450 0x564bfad50840 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3508f844eba3cca5 server_cookie=b6b2cef24f98b231 in_seq=0 out_seq=0 2024-04-02T13:54:42.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.046+0000 7f4eb0ff9640 1 -- 172.21.15.52:0/3123058830 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564bfac3a040 con 0x564bfad4e450 2024-04-02T13:54:42.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.046+0000 7f4eb0ff9640 10 client.16020 handle_command_reply: tid=0 2024-04-02T13:54:42.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.046+0000 7f4ed201f740 1 client.16020 shutdown 2024-04-02T13:54:42.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.046+0000 7f4ed201f740 1 -- 172.21.15.52:0/3123058830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfacce7a0 msgr2=0x564bfacdfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.046+0000 7f4ed201f740 1 --2- 172.21.15.52:0/3123058830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfacce7a0 0x564bfacdfe60 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4099290 tx=0x7f4ec400c7d0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.046+0000 7f4ed201f740 1 -- 172.21.15.52:0/3123058830 shutdown_connections 2024-04-02T13:54:42.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.046+0000 7f4ed201f740 1 --2- 172.21.15.52:0/3123058830 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x564bfad4e450 0x564bfad50840 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.046+0000 7f4ed201f740 1 --2- 172.21.15.52:0/3123058830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564bfacd7ca0 0x564bfacd0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.046+0000 7f4ed201f740 1 --2- 172.21.15.52:0/3123058830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564bfacd3cd0 0x564bface03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.046+0000 7f4ed201f740 1 --2- 172.21.15.52:0/3123058830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bfacce7a0 0x564bfacdfe60 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.046+0000 7f4eb0ff9640 0 client.16020 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:54:42.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.046+0000 7f4ed201f740 1 -- 172.21.15.52:0/3123058830 >> 172.21.15.52:0/3123058830 conn(0x564bfacc8ed0 msgr2=0x564bfacd4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:42.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.046+0000 7f4ed201f740 1 -- 172.21.15.52:0/3123058830 shutdown_connections 2024-04-02T13:54:42.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.046+0000 7f4ed201f740 1 -- 172.21.15.52:0/3123058830 wait complete. 2024-04-02T13:54:42.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.046+0000 7f4ed201f740 20 client.16020 trim_cache size 0 max 16384 2024-04-02T13:54:42.055 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:54:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.047+0000 7f4ecf35f640 1 -- 172.21.15.52:0/1448192005 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4eb40a3ed0 msgr2=0x7f4eb40a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.047+0000 7f4ecf35f640 1 --2- 172.21.15.52:0/1448192005 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4eb40a3ed0 0x7f4eb40a6390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8006010 tx=0x7f4eb802c650 comp rx=0 tx=0).stop 2024-04-02T13:54:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.047+0000 7f4ecf35f640 1 -- 172.21.15.52:0/1448192005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ec8068d00 msgr2=0x7f4ec81009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.047+0000 7f4ecf35f640 1 --2- 172.21.15.52:0/1448192005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ec8068d00 0x7f4ec81009c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400e9f0 tx=0x7f4ec400eeb0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.047+0000 7f4ecf35f640 1 -- 172.21.15.52:0/1448192005 shutdown_connections 2024-04-02T13:54:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.047+0000 7f4ecf35f640 1 --2- 172.21.15.52:0/1448192005 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4eb40a3ed0 0x7f4eb40a6390 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.047+0000 7f4ecf35f640 1 --2- 172.21.15.52:0/1448192005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ec8107b50 0x7f4ec8101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.047+0000 7f4ecf35f640 1 --2- 172.21.15.52:0/1448192005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ec80696c0 0x7f4ec8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.047+0000 7f4ecf35f640 1 --2- 172.21.15.52:0/1448192005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ec8068d00 0x7f4ec81009c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.047+0000 7f4ecf35f640 1 -- 172.21.15.52:0/1448192005 >> 172.21.15.52:0/1448192005 conn(0x7f4ec8077b00 msgr2=0x7f4ec80fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.047+0000 7f4ecf35f640 1 -- 172.21.15.52:0/1448192005 shutdown_connections 2024-04-02T13:54:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.047+0000 7f4ecf35f640 1 -- 172.21.15.52:0/1448192005 wait complete. 2024-04-02T13:54:42.062 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:54:42.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.166+0000 7fd79c17b640 1 -- 172.21.15.52:0/2633536589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd794101330 msgr2=0x7fd79410d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.166+0000 7fd79c17b640 1 --2- 172.21.15.52:0/2633536589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd794101330 0x7fd79410d840 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd78c009cd0 tx=0x7fd78c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.167+0000 7fd79c17b640 1 -- 172.21.15.52:0/2633536589 shutdown_connections 2024-04-02T13:54:42.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.167+0000 7fd79c17b640 1 --2- 172.21.15.52:0/2633536589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd794101330 0x7fd79410d840 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.167+0000 7fd79c17b640 1 --2- 172.21.15.52:0/2633536589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd794100970 0x7fd794100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.167+0000 7fd79c17b640 1 --2- 172.21.15.52:0/2633536589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd794106980 0x7fd794106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.167+0000 7fd79c17b640 1 -- 172.21.15.52:0/2633536589 >> 172.21.15.52:0/2633536589 conn(0x7fd794077b00 msgr2=0x7fd794077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:42.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.167+0000 7fd79c17b640 1 -- 172.21.15.52:0/2633536589 shutdown_connections 2024-04-02T13:54:42.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.167+0000 7fd79c17b640 1 -- 172.21.15.52:0/2633536589 wait complete. 2024-04-02T13:54:42.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.167+0000 7fd79c17b640 1 Processor -- start 2024-04-02T13:54:42.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.167+0000 7fd79c17b640 1 -- start start 2024-04-02T13:54:42.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.167+0000 7fd79c17b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd794100970 0x7fd794102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.167+0000 7fd79c17b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd794101330 0x7fd794103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.167+0000 7fd79c17b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd794106980 0x7fd794103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.167+0000 7fd79c17b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd794110d90 con 0x7fd794100970 2024-04-02T13:54:42.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.168+0000 7fd79c17b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd794110c10 con 0x7fd794101330 2024-04-02T13:54:42.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.168+0000 7fd79c17b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd794110f10 con 0x7fd794106980 2024-04-02T13:54:42.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.168+0000 7fd799f17640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd794100970 0x7fd794102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.168+0000 7fd799f17640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd794100970 0x7fd794102f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34894/0 (socket says 172.21.15.52:34894) 2024-04-02T13:54:42.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.168+0000 7fd799f17640 1 -- 172.21.15.52:0/3242000167 learned_addr learned my addr 172.21.15.52:0/3242000167 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:42.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.168+0000 7fd79a718640 1 --2- 172.21.15.52:0/3242000167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd794106980 0x7fd794103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.168+0000 7fd799716640 1 --2- 172.21.15.52:0/3242000167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd794101330 0x7fd794103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.168+0000 7fd799f17640 1 -- 172.21.15.52:0/3242000167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd794106980 msgr2=0x7fd794103990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.168+0000 7fd799f17640 1 --2- 172.21.15.52:0/3242000167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd794106980 0x7fd794103990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.168+0000 7fd799f17640 1 -- 172.21.15.52:0/3242000167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd794101330 msgr2=0x7fd794103450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.168+0000 7fd799f17640 1 --2- 172.21.15.52:0/3242000167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd794101330 0x7fd794103450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.168+0000 7fd799f17640 1 -- 172.21.15.52:0/3242000167 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7941a68c0 con 0x7fd794100970 2024-04-02T13:54:42.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.168+0000 7fd799f17640 1 --2- 172.21.15.52:0/3242000167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd794100970 0x7fd794102f10 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd79000e9f0 tx=0x7fd79000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:42.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.168+0000 7fd78affd640 1 -- 172.21.15.52:0/3242000167 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79000cd80 con 0x7fd794100970 2024-04-02T13:54:42.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.168+0000 7fd78affd640 1 -- 172.21.15.52:0/3242000167 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd790010910 con 0x7fd794100970 2024-04-02T13:54:42.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.168+0000 7fd79c17b640 1 -- 172.21.15.52:0/3242000167 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7941a6bb0 con 0x7fd794100970 2024-04-02T13:54:42.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.168+0000 7fd78affd640 1 -- 172.21.15.52:0/3242000167 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790018990 con 0x7fd794100970 2024-04-02T13:54:42.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.169+0000 7fd79c17b640 1 -- 172.21.15.52:0/3242000167 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7941a7100 con 0x7fd794100970 2024-04-02T13:54:42.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.171+0000 7fd78affd640 1 -- 172.21.15.52:0/3242000167 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fd790010ac0 con 0x7fd794100970 2024-04-02T13:54:42.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.171+0000 7fd79ed67740 1 Processor -- start 2024-04-02T13:54:42.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.171+0000 7fd78affd640 1 --2- 172.21.15.52:0/3242000167 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd7800a3ed0 0x7fd7800a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.171+0000 7fd78affd640 1 -- 172.21.15.52:0/3242000167 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd7900d10a0 con 0x7fd794100970 2024-04-02T13:54:42.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.171+0000 7fd79ed67740 1 -- start start 2024-04-02T13:54:42.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.172+0000 7fd79a718640 1 --2- 172.21.15.52:0/3242000167 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd7800a3ed0 0x7fd7800a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.172+0000 7fd79ed67740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55660cc695b0 0x55660cc69990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.172+0000 7fd79ed67740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc69f60 0x55660cc7af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.172+0000 7fd79ed67740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55660cc7b4b0 0x55660cc7d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.172+0000 7fd79ed67740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55660cbcd7b0 con 0x55660cc69f60 2024-04-02T13:54:42.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.172+0000 7fd79ed67740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55660cb6c570 con 0x55660cc695b0 2024-04-02T13:54:42.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.172+0000 7fd79ed67740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55660cbc9760 con 0x55660cc7b4b0 2024-04-02T13:54:42.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.172+0000 7fd799716640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc69f60 0x55660cc7af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.172+0000 7fd799f17640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55660cc695b0 0x55660cc69990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.172+0000 7fd799f17640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55660cc695b0 0x55660cc69990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:56298/0 (socket says 172.21.15.52:56298) 2024-04-02T13:54:42.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.172+0000 7fd799f17640 1 -- 172.21.15.52:0/3612990220 learned_addr learned my addr 172.21.15.52:0/3612990220 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:42.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.172+0000 7fd79a718640 1 --2- 172.21.15.52:0/3612990220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55660cc7b4b0 0x55660cc7d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.172+0000 7fd79a718640 1 --2- 172.21.15.52:0/3242000167 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd7800a3ed0 0x7fd7800a6390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd78c005cc0 tx=0x7fd78c005c50 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.172+0000 7fd799f17640 1 -- 172.21.15.52:0/3612990220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55660cc7b4b0 msgr2=0x55660cc7d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.172+0000 7fd799f17640 1 --2- 172.21.15.52:0/3612990220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55660cc7b4b0 0x55660cc7d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.172+0000 7fd799f17640 1 -- 172.21.15.52:0/3612990220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc69f60 msgr2=0x55660cc7af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.172+0000 7fd799f17640 1 --2- 172.21.15.52:0/3612990220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc69f60 0x55660cc7af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.172+0000 7fd799f17640 1 -- 172.21.15.52:0/3612990220 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55660c8e13f0 con 0x55660cc695b0 2024-04-02T13:54:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.173+0000 7fd799f17640 1 --2- 172.21.15.52:0/3612990220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55660cc695b0 0x55660cc69990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd7900ce360 tx=0x7fd79009cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4912f5bbd372f20 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.173+0000 7fd77ffff640 1 -- 172.21.15.52:0/3612990220 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79009ae70 con 0x55660cc695b0 2024-04-02T13:54:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.173+0000 7fd77ffff640 1 -- 172.21.15.52:0/3612990220 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7900991e0 con 0x55660cc695b0 2024-04-02T13:54:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.173+0000 7fd77ffff640 1 -- 172.21.15.52:0/3612990220 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79009c220 con 0x55660cc695b0 2024-04-02T13:54:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.174+0000 7fd79ed67740 1 -- 172.21.15.52:0/3612990220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55660cc695b0 msgr2=0x55660cc69990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.174+0000 7fd79ed67740 1 --2- 172.21.15.52:0/3612990220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55660cc695b0 0x55660cc69990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd7900ce360 tx=0x7fd79009cc30 comp rx=0 tx=0).stop 2024-04-02T13:54:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.174+0000 7fd79ed67740 1 -- 172.21.15.52:0/3612990220 shutdown_connections 2024-04-02T13:54:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.174+0000 7fd79ed67740 1 --2- 172.21.15.52:0/3612990220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55660cc7b4b0 0x55660cc7d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.174+0000 7fd79ed67740 1 --2- 172.21.15.52:0/3612990220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc69f60 0x55660cc7af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.174+0000 7fd79ed67740 1 --2- 172.21.15.52:0/3612990220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55660cc695b0 0x55660cc69990 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.174+0000 7fd79ed67740 1 -- 172.21.15.52:0/3612990220 >> 172.21.15.52:0/3612990220 conn(0x55660cc63ed0 msgr2=0x55660cc642b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.174+0000 7fd79ed67740 1 -- 172.21.15.52:0/3612990220 shutdown_connections 2024-04-02T13:54:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.174+0000 7fd79ed67740 1 -- 172.21.15.52:0/3612990220 wait complete. 2024-04-02T13:54:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.174+0000 7fd79ed67740 1 -- start start 2024-04-02T13:54:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.174+0000 7fd79ed67740 1 Processor -- start 2024-04-02T13:54:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.174+0000 7fd79ed67740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55660cc695b0 0x55660cc7af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.175+0000 7fd79ed67740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55660cc69f60 0x55660cc782b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.175+0000 7fd79ed67740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc7b4b0 0x55660cc789f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.175+0000 7fd79ed67740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55660cc7e210 con 0x55660cc7b4b0 2024-04-02T13:54:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.175+0000 7fd79ed67740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55660cb88e80 con 0x55660cc695b0 2024-04-02T13:54:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.175+0000 7fd79ed67740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55660cc7e390 con 0x55660cc69f60 2024-04-02T13:54:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.175+0000 7fd79ed67740 10 client.0 _ll_register_callbacks cb 0x55660cbc4aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.175+0000 7fd799716640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55660cc69f60 0x55660cc782b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.175+0000 7fd79a718640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc7b4b0 0x55660cc789f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.175+0000 7fd79a718640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc7b4b0 0x55660cc789f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34902/0 (socket says 172.21.15.52:34902) 2024-04-02T13:54:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.175+0000 7fd79a718640 1 -- 172.21.15.52:0/2107469000 learned_addr learned my addr 172.21.15.52:0/2107469000 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.175+0000 7fd799f17640 1 --2- 172.21.15.52:0/2107469000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55660cc695b0 0x55660cc7af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.175+0000 7fd79a718640 1 -- 172.21.15.52:0/2107469000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55660cc69f60 msgr2=0x55660cc782b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.175+0000 7fd79a718640 1 --2- 172.21.15.52:0/2107469000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55660cc69f60 0x55660cc782b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.175+0000 7fd79a718640 1 -- 172.21.15.52:0/2107469000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55660cc695b0 msgr2=0x55660cc7af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.175+0000 7fd79a718640 1 --2- 172.21.15.52:0/2107469000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55660cc695b0 0x55660cc7af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.175+0000 7fd79a718640 1 -- 172.21.15.52:0/2107469000 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55660c9ab7d0 con 0x55660cc7b4b0 2024-04-02T13:54:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.175+0000 7fd799f17640 1 --2- 172.21.15.52:0/2107469000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55660cc695b0 0x55660cc7af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:54:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.175+0000 7fd79a718640 1 --2- 172.21.15.52:0/2107469000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc7b4b0 0x55660cc789f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd78c02c700 tx=0x7fd78c02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9012231a6d37147c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:42.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.175+0000 7fd77d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:42.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.176+0000 7fd77d7fa640 1 -- 172.21.15.52:0/2107469000 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c04f020 con 0x55660cc7b4b0 2024-04-02T13:54:42.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.176+0000 7fd79ed67740 1 -- 172.21.15.52:0/2107469000 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7900991e0 con 0x55660cc7b4b0 2024-04-02T13:54:42.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.176+0000 7fd77d7fa640 1 -- 172.21.15.52:0/2107469000 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd78c003d80 con 0x55660cc7b4b0 2024-04-02T13:54:42.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.176+0000 7fd77d7fa640 1 -- 172.21.15.52:0/2107469000 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd78c05a050 con 0x55660cc7b4b0 2024-04-02T13:54:42.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.176+0000 7fd79ed67740 10 client.16032 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:42.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.176+0000 7fd79ed67740 1 -- 172.21.15.52:0/2107469000 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55660cc8a330 con 0x55660cc7b4b0 2024-04-02T13:54:42.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.176+0000 7fd77d7fa640 1 -- 172.21.15.52:0/2107469000 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7900991e0 con 0x55660cc7b4b0 2024-04-02T13:54:42.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.177+0000 7fd77d7fa640 1 -- 172.21.15.52:0/2107469000 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd78c03f7b0 con 0x55660cc7b4b0 2024-04-02T13:54:42.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.177+0000 7fd79ed67740 10 client.16032 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:42.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.177+0000 7fd79ed67740 10 client.16032 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:54:42.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.177+0000 7fd79ed67740 20 client.16032 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:42.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.177+0000 7fd79ed67740 1 --2- 172.21.15.52:0/2107469000 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55660cc8bc00 0x55660cc8e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.177+0000 7fd79ed67740 4 client.16032 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:54:42.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.177+0000 7fd79ed67740 1 -- 172.21.15.52:0/2107469000 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55660cbdacf0 con 0x55660cc8bc00 2024-04-02T13:54:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.177+0000 7fd799f17640 1 --2- 172.21.15.52:0/2107469000 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55660cc8bc00 0x55660cc8e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.178+0000 7fd799f17640 1 --2- 172.21.15.52:0/2107469000 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55660cc8bc00 0x55660cc8e060 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1a96c24654fe98ce server_cookie=a54efd2c34656979 in_seq=0 out_seq=0 2024-04-02T13:54:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.178+0000 7fd77d7fa640 10 client.16032 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:54:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.178+0000 7fd77d7fa640 1 -- 172.21.15.52:0/2107469000 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55660cbdacf0 con 0x55660cc8bc00 2024-04-02T13:54:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.178+0000 7fd77d7fa640 10 client.16032 handle_command_reply: tid=0 2024-04-02T13:54:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.178+0000 7fd79ed67740 1 client.16032 shutdown 2024-04-02T13:54:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.178+0000 7fd79ed67740 1 -- 172.21.15.52:0/2107469000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc7b4b0 msgr2=0x55660cc789f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.178+0000 7fd79ed67740 1 --2- 172.21.15.52:0/2107469000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc7b4b0 0x55660cc789f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd78c02c700 tx=0x7fd78c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.179+0000 7fd79ed67740 1 -- 172.21.15.52:0/2107469000 shutdown_connections 2024-04-02T13:54:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.179+0000 7fd79ed67740 1 --2- 172.21.15.52:0/2107469000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc7b4b0 0x55660cc789f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.179+0000 7fd79ed67740 1 --2- 172.21.15.52:0/2107469000 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55660cc8bc00 0x55660cc8e060 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.179+0000 7fd79ed67740 1 --2- 172.21.15.52:0/2107469000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55660cc69f60 0x55660cc782b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.179+0000 7fd79ed67740 1 --2- 172.21.15.52:0/2107469000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55660cc695b0 0x55660cc7af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.179+0000 7fd79ed67740 1 -- 172.21.15.52:0/2107469000 >> 172.21.15.52:0/2107469000 conn(0x55660cc63ed0 msgr2=0x55660cc65c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:42.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.179+0000 7fd77d7fa640 0 client.16032 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:54:42.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.179+0000 7fd79ed67740 1 -- 172.21.15.52:0/2107469000 shutdown_connections 2024-04-02T13:54:42.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.179+0000 7fd79ed67740 1 -- 172.21.15.52:0/2107469000 wait complete. 2024-04-02T13:54:42.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.179+0000 7fd79ed67740 20 client.16032 trim_cache size 0 max 16384 2024-04-02T13:54:42.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.192+0000 7fd79ed67740 1 Processor -- start 2024-04-02T13:54:42.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.193+0000 7fd79ed67740 1 -- start start 2024-04-02T13:54:42.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.193+0000 7fd79ed67740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55660cc695b0 0x55660cc87110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.193+0000 7fd79ed67740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc69f60 0x55660cc7a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.193+0000 7fd79ed67740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55660cc7b4b0 0x55660cc7af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.193+0000 7fd79ed67740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd7941accb0 con 0x55660cc69f60 2024-04-02T13:54:42.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.193+0000 7fd79ed67740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd7941ae7f0 con 0x55660cc695b0 2024-04-02T13:54:42.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.193+0000 7fd79ed67740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd7941ad380 con 0x55660cc7b4b0 2024-04-02T13:54:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.193+0000 7fd799716640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc69f60 0x55660cc7a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.193+0000 7fd799716640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc69f60 0x55660cc7a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34904/0 (socket says 172.21.15.52:34904) 2024-04-02T13:54:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.193+0000 7fd799716640 1 -- 172.21.15.52:0/2683997738 learned_addr learned my addr 172.21.15.52:0/2683997738 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.193+0000 7fd799f17640 1 --2- 172.21.15.52:0/2683997738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55660cc695b0 0x55660cc87110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.193+0000 7fd79a718640 1 --2- 172.21.15.52:0/2683997738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55660cc7b4b0 0x55660cc7af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.193+0000 7fd799716640 1 -- 172.21.15.52:0/2683997738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55660cc7b4b0 msgr2=0x55660cc7af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.193+0000 7fd799716640 1 --2- 172.21.15.52:0/2683997738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55660cc7b4b0 0x55660cc7af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.193+0000 7fd799716640 1 -- 172.21.15.52:0/2683997738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55660cc695b0 msgr2=0x55660cc87110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.193+0000 7fd799716640 1 --2- 172.21.15.52:0/2683997738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55660cc695b0 0x55660cc87110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.193+0000 7fd799716640 1 -- 172.21.15.52:0/2683997738 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55660cc69d00 con 0x55660cc69f60 2024-04-02T13:54:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.194+0000 7fd799716640 1 --2- 172.21.15.52:0/2683997738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc69f60 0x55660cc7a9e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd784004650 tx=0x7fd78400bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02f181dc6f7b8b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.194+0000 7fd77f7fe640 1 -- 172.21.15.52:0/2683997738 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd784003d50 con 0x55660cc69f60 2024-04-02T13:54:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.194+0000 7fd77f7fe640 1 -- 172.21.15.52:0/2683997738 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd784020070 con 0x55660cc69f60 2024-04-02T13:54:42.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.194+0000 7fd77f7fe640 1 -- 172.21.15.52:0/2683997738 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78401b760 con 0x55660cc69f60 2024-04-02T13:54:42.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.194+0000 7fd79ed67740 1 -- 172.21.15.52:0/2683997738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc69f60 msgr2=0x55660cc7a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.194+0000 7fd79ed67740 1 --2- 172.21.15.52:0/2683997738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc69f60 0x55660cc7a9e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd784004650 tx=0x7fd78400bd00 comp rx=0 tx=0).stop 2024-04-02T13:54:42.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.195+0000 7fd79ed67740 1 -- 172.21.15.52:0/2683997738 shutdown_connections 2024-04-02T13:54:42.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.195+0000 7fd79ed67740 1 --2- 172.21.15.52:0/2683997738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55660cc7b4b0 0x55660cc7af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.195+0000 7fd79ed67740 1 --2- 172.21.15.52:0/2683997738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc69f60 0x55660cc7a9e0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.195+0000 7fd79ed67740 1 --2- 172.21.15.52:0/2683997738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55660cc695b0 0x55660cc87110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.195+0000 7fd79ed67740 1 -- 172.21.15.52:0/2683997738 >> 172.21.15.52:0/2683997738 conn(0x55660cc63ed0 msgr2=0x55660cc642b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:42.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.195+0000 7fd79ed67740 1 -- 172.21.15.52:0/2683997738 shutdown_connections 2024-04-02T13:54:42.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.195+0000 7fd79ed67740 1 -- 172.21.15.52:0/2683997738 wait complete. 2024-04-02T13:54:42.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.195+0000 7fd79ed67740 1 -- start start 2024-04-02T13:54:42.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.195+0000 7fd79ed67740 1 Processor -- start 2024-04-02T13:54:42.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.196+0000 7fd79ed67740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55660cc695b0 0x55660cc83ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.196+0000 7fd79ed67740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc69f60 0x55660cc7cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.196+0000 7fd79ed67740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55660cc7b4b0 0x55660cc7d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.196+0000 7fd79ed67740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd7941ae120 con 0x55660cc69f60 2024-04-02T13:54:42.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.196+0000 7fd799716640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc69f60 0x55660cc7cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.196+0000 7fd79ed67740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd7941ada50 con 0x55660cc7b4b0 2024-04-02T13:54:42.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.196+0000 7fd79ed67740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd7941aac90 con 0x55660cc695b0 2024-04-02T13:54:42.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.196+0000 7fd799716640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc69f60 0x55660cc7cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34916/0 (socket says 172.21.15.52:34916) 2024-04-02T13:54:42.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.196+0000 7fd799716640 1 -- 172.21.15.52:0/29540510 learned_addr learned my addr 172.21.15.52:0/29540510 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:42.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.196+0000 7fd79ed67740 10 client.0 _ll_register_callbacks cb 0x7fd790005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:42.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.196+0000 7fd799f17640 1 --2- 172.21.15.52:0/29540510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55660cc695b0 0x55660cc83ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.196+0000 7fd79a718640 1 --2- 172.21.15.52:0/29540510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55660cc7b4b0 0x55660cc7d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.196+0000 7fd799716640 1 -- 172.21.15.52:0/29540510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55660cc695b0 msgr2=0x55660cc83ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:42.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.196+0000 7fd799716640 1 --2- 172.21.15.52:0/29540510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55660cc695b0 0x55660cc83ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.196+0000 7fd799716640 1 -- 172.21.15.52:0/29540510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55660cc7b4b0 msgr2=0x55660cc7d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:42.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.196+0000 7fd799716640 1 --2- 172.21.15.52:0/29540510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55660cc7b4b0 0x55660cc7d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.196+0000 7fd799716640 1 -- 172.21.15.52:0/29540510 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55660cc756e0 con 0x55660cc69f60 2024-04-02T13:54:42.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.196+0000 7fd79a718640 1 --2- 172.21.15.52:0/29540510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55660cc7b4b0 0x55660cc7d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:54:42.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.197+0000 7fd799716640 1 --2- 172.21.15.52:0/29540510 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc69f60 0x55660cc7cba0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd784005c20 tx=0x7fd784004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d36566f3ff27789 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:42.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.197+0000 7fd77d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:42.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.197+0000 7fd77d7fa640 1 -- 172.21.15.52:0/29540510 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd784005460 con 0x55660cc69f60 2024-04-02T13:54:42.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.197+0000 7fd77d7fa640 1 -- 172.21.15.52:0/29540510 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd784011440 con 0x55660cc69f60 2024-04-02T13:54:42.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.197+0000 7fd79ed67740 1 -- 172.21.15.52:0/29540510 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd784020070 con 0x55660cc69f60 2024-04-02T13:54:42.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.197+0000 7fd77d7fa640 1 -- 172.21.15.52:0/29540510 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd784004360 con 0x55660cc69f60 2024-04-02T13:54:42.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.197+0000 7fd79ed67740 10 client.16038 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:42.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.197+0000 7fd79ed67740 1 -- 172.21.15.52:0/29540510 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55660cc7bcf0 con 0x55660cc69f60 2024-04-02T13:54:42.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.197+0000 7fd77d7fa640 1 -- 172.21.15.52:0/29540510 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd784020070 con 0x55660cc69f60 2024-04-02T13:54:42.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.198+0000 7fd77d7fa640 1 -- 172.21.15.52:0/29540510 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd784004b30 con 0x55660cc69f60 2024-04-02T13:54:42.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.198+0000 7fd79ed67740 10 client.16038 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:42.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.198+0000 7fd79ed67740 10 client.16038 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:54:42.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.198+0000 7fd79ed67740 20 client.16038 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:42.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.198+0000 7fd79ed67740 1 --2- 172.21.15.52:0/29540510 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55660cc7c3a0 0x55660ccf41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.198+0000 7fd79ed67740 4 client.16038 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:54:42.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.198+0000 7fd79ed67740 1 -- 172.21.15.52:0/29540510 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55660cbd5040 con 0x55660cc7c3a0 2024-04-02T13:54:42.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.198+0000 7fd799f17640 1 --2- 172.21.15.52:0/29540510 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55660cc7c3a0 0x55660ccf41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.198+0000 7fd77d7fa640 10 client.16038 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:54:42.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.198+0000 7fd799f17640 1 --2- 172.21.15.52:0/29540510 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55660cc7c3a0 0x55660ccf41c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b894b01d711ccb8d server_cookie=2c4951bbd2058512 in_seq=0 out_seq=0 2024-04-02T13:54:42.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.198+0000 7fd77d7fa640 1 -- 172.21.15.52:0/29540510 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55660cbd5040 con 0x55660cc7c3a0 2024-04-02T13:54:42.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.198+0000 7fd77d7fa640 10 client.16038 handle_command_reply: tid=0 2024-04-02T13:54:42.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.198+0000 7fd79ed67740 1 client.16038 shutdown 2024-04-02T13:54:42.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.199+0000 7fd79ed67740 1 -- 172.21.15.52:0/29540510 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc69f60 msgr2=0x55660cc7cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.199+0000 7fd79ed67740 1 --2- 172.21.15.52:0/29540510 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc69f60 0x55660cc7cba0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd784005c20 tx=0x7fd784004180 comp rx=0 tx=0).stop 2024-04-02T13:54:42.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.199+0000 7fd79ed67740 1 -- 172.21.15.52:0/29540510 shutdown_connections 2024-04-02T13:54:42.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.199+0000 7fd79ed67740 1 --2- 172.21.15.52:0/29540510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55660cc7b4b0 0x55660cc7d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.199+0000 7fd79ed67740 1 --2- 172.21.15.52:0/29540510 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55660cc69f60 0x55660cc7cba0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.199+0000 7fd79ed67740 1 --2- 172.21.15.52:0/29540510 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55660cc7c3a0 0x55660ccf41c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.199+0000 7fd79ed67740 1 --2- 172.21.15.52:0/29540510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55660cc695b0 0x55660cc83ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.199+0000 7fd77d7fa640 0 client.16038 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:54:42.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.199+0000 7fd79ed67740 1 -- 172.21.15.52:0/29540510 >> 172.21.15.52:0/29540510 conn(0x55660cc63ed0 msgr2=0x55660cc73550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:42.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.199+0000 7fd79ed67740 1 -- 172.21.15.52:0/29540510 shutdown_connections 2024-04-02T13:54:42.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.199+0000 7fd79ed67740 1 -- 172.21.15.52:0/29540510 wait complete. 2024-04-02T13:54:42.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.199+0000 7fd79ed67740 20 client.16038 trim_cache size 0 max 16384 2024-04-02T13:54:42.209 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:54:42.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.199+0000 7fd79c17b640 1 -- 172.21.15.52:0/3242000167 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd7800a3ed0 msgr2=0x7fd7800a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.199+0000 7fd79c17b640 1 --2- 172.21.15.52:0/3242000167 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd7800a3ed0 0x7fd7800a6390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd78c005cc0 tx=0x7fd78c005c50 comp rx=0 tx=0).stop 2024-04-02T13:54:42.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.199+0000 7fd79c17b640 1 -- 172.21.15.52:0/3242000167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd794100970 msgr2=0x7fd794102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.199+0000 7fd79c17b640 1 --2- 172.21.15.52:0/3242000167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd794100970 0x7fd794102f10 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd79000e9f0 tx=0x7fd79000eeb0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.200+0000 7fd79c17b640 1 -- 172.21.15.52:0/3242000167 shutdown_connections 2024-04-02T13:54:42.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.200+0000 7fd79c17b640 1 --2- 172.21.15.52:0/3242000167 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd7800a3ed0 0x7fd7800a6390 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.200+0000 7fd79c17b640 1 --2- 172.21.15.52:0/3242000167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd794106980 0x7fd794103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.200+0000 7fd79c17b640 1 --2- 172.21.15.52:0/3242000167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd794101330 0x7fd794103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.200+0000 7fd79c17b640 1 --2- 172.21.15.52:0/3242000167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd794100970 0x7fd794102f10 secure :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd79000e9f0 tx=0x7fd79000eeb0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.200+0000 7fd79c17b640 1 -- 172.21.15.52:0/3242000167 >> 172.21.15.52:0/3242000167 conn(0x7fd794077b00 msgr2=0x7fd794068fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:42.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.200+0000 7fd79c17b640 1 -- 172.21.15.52:0/3242000167 shutdown_connections 2024-04-02T13:54:42.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.200+0000 7fd79c17b640 1 -- 172.21.15.52:0/3242000167 wait complete. 2024-04-02T13:54:42.214 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:54:42.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.321+0000 7f28add50640 1 -- 172.21.15.52:0/343411870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28a8068d00 msgr2=0x7f28a8069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.321+0000 7f28add50640 1 --2- 172.21.15.52:0/343411870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28a8068d00 0x7f28a8069180 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f289c009890 tx=0x7f289c02bb70 comp rx=0 tx=0).stop 2024-04-02T13:54:42.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.321+0000 7f28add50640 1 -- 172.21.15.52:0/343411870 shutdown_connections 2024-04-02T13:54:42.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.321+0000 7f28add50640 1 --2- 172.21.15.52:0/343411870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f28a80696c0 0x7f28a810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.321+0000 7f28add50640 1 --2- 172.21.15.52:0/343411870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28a8068d00 0x7f28a8069180 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.321+0000 7f28add50640 1 --2- 172.21.15.52:0/343411870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28a8107b50 0x7f28a8107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.321+0000 7f28add50640 1 -- 172.21.15.52:0/343411870 >> 172.21.15.52:0/343411870 conn(0x7f28a8077b00 msgr2=0x7f28a8077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:42.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.321+0000 7f28add50640 1 -- 172.21.15.52:0/343411870 shutdown_connections 2024-04-02T13:54:42.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.321+0000 7f28add50640 1 -- 172.21.15.52:0/343411870 wait complete. 2024-04-02T13:54:42.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.322+0000 7f28add50640 1 Processor -- start 2024-04-02T13:54:42.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.322+0000 7f28add50640 1 -- start start 2024-04-02T13:54:42.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.322+0000 7f28add50640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28a8068d00 0x7f28a81983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.322+0000 7f28add50640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f28a80696c0 0x7f28a8198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.322+0000 7f28add50640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28a8107b50 0x7f28a819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.322+0000 7f28add50640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f28a8111ea0 con 0x7f28a8068d00 2024-04-02T13:54:42.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.322+0000 7f28add50640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f28a8111d20 con 0x7f28a8107b50 2024-04-02T13:54:42.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.322+0000 7f28add50640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f28a8112020 con 0x7f28a80696c0 2024-04-02T13:54:42.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.322+0000 7f28a77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28a8068d00 0x7f28a81983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.322+0000 7f28a77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28a8068d00 0x7f28a81983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34924/0 (socket says 172.21.15.52:34924) 2024-04-02T13:54:42.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.322+0000 7f28a77fe640 1 -- 172.21.15.52:0/1930420843 learned_addr learned my addr 172.21.15.52:0/1930420843 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:42.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.322+0000 7f28a6ffd640 1 --2- 172.21.15.52:0/1930420843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f28a80696c0 0x7f28a8198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.322+0000 7f28a7fff640 1 --2- 172.21.15.52:0/1930420843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28a8107b50 0x7f28a819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.323+0000 7f28a77fe640 1 -- 172.21.15.52:0/1930420843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f28a80696c0 msgr2=0x7f28a8198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.323+0000 7f28a77fe640 1 --2- 172.21.15.52:0/1930420843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f28a80696c0 0x7f28a8198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.323+0000 7f28a77fe640 1 -- 172.21.15.52:0/1930420843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28a8107b50 msgr2=0x7f28a819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.323+0000 7f28a77fe640 1 --2- 172.21.15.52:0/1930420843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28a8107b50 0x7f28a819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.323+0000 7f28a77fe640 1 -- 172.21.15.52:0/1930420843 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a819d3b0 con 0x7f28a8068d00 2024-04-02T13:54:42.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.323+0000 7f28a77fe640 1 --2- 172.21.15.52:0/1930420843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28a8068d00 0x7f28a81983c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f289800e9c0 tx=0x7f289800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:42.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.323+0000 7f28a4ff9640 1 -- 172.21.15.52:0/1930420843 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289800ccc0 con 0x7f28a8068d00 2024-04-02T13:54:42.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.323+0000 7f28add50640 1 -- 172.21.15.52:0/1930420843 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28a819d6a0 con 0x7f28a8068d00 2024-04-02T13:54:42.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.323+0000 7f28a4ff9640 1 -- 172.21.15.52:0/1930420843 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2898005500 con 0x7f28a8068d00 2024-04-02T13:54:42.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.323+0000 7f28a4ff9640 1 -- 172.21.15.52:0/1930420843 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2898010430 con 0x7f28a8068d00 2024-04-02T13:54:42.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.323+0000 7f28add50640 1 -- 172.21.15.52:0/1930420843 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28a81008f0 con 0x7f28a8068d00 2024-04-02T13:54:42.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.326+0000 7f28b0a12740 1 Processor -- start 2024-04-02T13:54:42.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.326+0000 7f28b0a12740 1 -- start start 2024-04-02T13:54:42.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.326+0000 7f28b0a12740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597b21ed7a0 0x5597b21edb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.326+0000 7f28b0a12740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5597b21ee0c0 0x5597b21f6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.326+0000 7f28b0a12740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21f6ca0 0x5597b21f9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.326+0000 7f28b0a12740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5597b21517b0 con 0x5597b21f6ca0 2024-04-02T13:54:42.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.326+0000 7f28b0a12740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5597b20f0570 con 0x5597b21ee0c0 2024-04-02T13:54:42.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.326+0000 7f28b0a12740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5597b214d760 con 0x5597b21ed7a0 2024-04-02T13:54:42.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.326+0000 7f28a6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21f6ca0 0x5597b21f9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.326+0000 7f28a6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21f6ca0 0x5597b21f9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34936/0 (socket says 172.21.15.52:34936) 2024-04-02T13:54:42.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.326+0000 7f28a6ffd640 1 -- 172.21.15.52:0/3337407872 learned_addr learned my addr 172.21.15.52:0/3337407872 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:42.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.326+0000 7f28a7fff640 1 --2- 172.21.15.52:0/3337407872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597b21ed7a0 0x5597b21edb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.327+0000 7f28a6ffd640 1 -- 172.21.15.52:0/3337407872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597b21ed7a0 msgr2=0x5597b21edb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:42.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.327+0000 7f28a6ffd640 1 --2- 172.21.15.52:0/3337407872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597b21ed7a0 0x5597b21edb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.327+0000 7f28a6ffd640 1 -- 172.21.15.52:0/3337407872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5597b21ee0c0 msgr2=0x5597b21f6760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:54:42.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.327+0000 7f28a6ffd640 1 --2- 172.21.15.52:0/3337407872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5597b21ee0c0 0x5597b21f6760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.327+0000 7f28a6ffd640 1 -- 172.21.15.52:0/3337407872 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597b1e653f0 con 0x5597b21f6ca0 2024-04-02T13:54:42.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.327+0000 7f28a6ffd640 1 --2- 172.21.15.52:0/3337407872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21f6ca0 0x5597b21f9090 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f289c02c0b0 tx=0x7f289c005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5c63738d997c54 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:42.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.327+0000 7f28a4ff9640 1 -- 172.21.15.52:0/1930420843 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f28980046c0 con 0x7f28a8068d00 2024-04-02T13:54:42.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.327+0000 7f288dffb640 1 -- 172.21.15.52:0/3337407872 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c043020 con 0x5597b21f6ca0 2024-04-02T13:54:42.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.327+0000 7f288dffb640 1 -- 172.21.15.52:0/3337407872 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f289c034ce0 con 0x5597b21f6ca0 2024-04-02T13:54:42.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.327+0000 7f288dffb640 1 -- 172.21.15.52:0/3337407872 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c039040 con 0x5597b21f6ca0 2024-04-02T13:54:42.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.328+0000 7f28a4ff9640 1 --2- 172.21.15.52:0/1930420843 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f28900a3ed0 0x7f28900a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.328+0000 7f28a4ff9640 1 -- 172.21.15.52:0/1930420843 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f28980d10a0 con 0x7f28a8068d00 2024-04-02T13:54:42.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.328+0000 7f28b0a12740 1 -- 172.21.15.52:0/3337407872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21f6ca0 msgr2=0x5597b21f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.328+0000 7f28a7fff640 1 --2- 172.21.15.52:0/1930420843 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f28900a3ed0 0x7f28900a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.328+0000 7f28b0a12740 1 --2- 172.21.15.52:0/3337407872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21f6ca0 0x5597b21f9090 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f289c02c0b0 tx=0x7f289c005b20 comp rx=0 tx=0).stop 2024-04-02T13:54:42.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.329+0000 7f28a7fff640 1 --2- 172.21.15.52:0/1930420843 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f28900a3ed0 0x7f28900a6390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f28a0005fd0 tx=0x7f28a0009500 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:42.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.329+0000 7f28b0a12740 1 -- 172.21.15.52:0/3337407872 shutdown_connections 2024-04-02T13:54:42.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.329+0000 7f28b0a12740 1 --2- 172.21.15.52:0/3337407872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21f6ca0 0x5597b21f9090 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.329+0000 7f28b0a12740 1 --2- 172.21.15.52:0/3337407872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5597b21ee0c0 0x5597b21f6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.329+0000 7f28b0a12740 1 --2- 172.21.15.52:0/3337407872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597b21ed7a0 0x5597b21edb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.329+0000 7f28b0a12740 1 -- 172.21.15.52:0/3337407872 >> 172.21.15.52:0/3337407872 conn(0x5597b21e7ed0 msgr2=0x5597b21e82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:42.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.329+0000 7f28b0a12740 1 -- 172.21.15.52:0/3337407872 shutdown_connections 2024-04-02T13:54:42.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.329+0000 7f28b0a12740 1 -- 172.21.15.52:0/3337407872 wait complete. 2024-04-02T13:54:42.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.329+0000 7f28b0a12740 1 -- start start 2024-04-02T13:54:42.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.330+0000 7f28b0a12740 1 Processor -- start 2024-04-02T13:54:42.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.330+0000 7f28b0a12740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597b21ed7a0 0x5597b21f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.330+0000 7f28b0a12740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21ee0c0 0x5597b21f4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.330+0000 7f28b0a12740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5597b21f6ca0 0x5597b21f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.330+0000 7f28b0a12740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5597b21f9a30 con 0x5597b21ee0c0 2024-04-02T13:54:42.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.330+0000 7f28b0a12740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5597b210ce80 con 0x5597b21f6ca0 2024-04-02T13:54:42.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.330+0000 7f28b0a12740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5597b21f9bb0 con 0x5597b21ed7a0 2024-04-02T13:54:42.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.330+0000 7f28b0a12740 10 client.0 _ll_register_callbacks cb 0x5597b2148aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:42.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.330+0000 7f28a6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21ee0c0 0x5597b21f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.330+0000 7f28a6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21ee0c0 0x5597b21f4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34952/0 (socket says 172.21.15.52:34952) 2024-04-02T13:54:42.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.330+0000 7f28a6ffd640 1 -- 172.21.15.52:0/2902402202 learned_addr learned my addr 172.21.15.52:0/2902402202 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:42.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.330+0000 7f28a77fe640 1 --2- 172.21.15.52:0/2902402202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597b21ed7a0 0x5597b21f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.330+0000 7f28a6ffd640 1 -- 172.21.15.52:0/2902402202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597b21ed7a0 msgr2=0x5597b21f3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:42.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.330+0000 7f28a7fff640 1 --2- 172.21.15.52:0/2902402202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5597b21f6ca0 0x5597b21f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.330+0000 7f28a6ffd640 1 --2- 172.21.15.52:0/2902402202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597b21ed7a0 0x5597b21f3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.330+0000 7f28a6ffd640 1 -- 172.21.15.52:0/2902402202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5597b21f6ca0 msgr2=0x5597b21f45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:42.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.330+0000 7f28a6ffd640 1 --2- 172.21.15.52:0/2902402202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5597b21f6ca0 0x5597b21f45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.330+0000 7f28a6ffd640 1 -- 172.21.15.52:0/2902402202 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597b1f2f7d0 con 0x5597b21ee0c0 2024-04-02T13:54:42.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.330+0000 7f28a77fe640 1 --2- 172.21.15.52:0/2902402202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597b21ed7a0 0x5597b21f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:54:42.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.330+0000 7f28a7fff640 1 --2- 172.21.15.52:0/2902402202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5597b21f6ca0 0x5597b21f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:54:42.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.331+0000 7f28a6ffd640 1 --2- 172.21.15.52:0/2902402202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21ee0c0 0x5597b21f4060 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f289c037040 tx=0x7f289c0044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3fbdfd1ca666ee server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:42.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.331+0000 7f2886ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:42.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.331+0000 7f2886ffd640 1 -- 172.21.15.52:0/2902402202 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c039050 con 0x5597b21ee0c0 2024-04-02T13:54:42.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.331+0000 7f28b0a12740 1 -- 172.21.15.52:0/2902402202 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f289c034ce0 con 0x5597b21ee0c0 2024-04-02T13:54:42.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.331+0000 7f2886ffd640 1 -- 172.21.15.52:0/2902402202 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f289c02c720 con 0x5597b21ee0c0 2024-04-02T13:54:42.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.331+0000 7f2886ffd640 1 -- 172.21.15.52:0/2902402202 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f289c05b050 con 0x5597b21ee0c0 2024-04-02T13:54:42.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.331+0000 7f28b0a12740 10 client.16050 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:42.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.331+0000 7f28b0a12740 1 -- 172.21.15.52:0/2902402202 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5597b2205b20 con 0x5597b21ee0c0 2024-04-02T13:54:42.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.331+0000 7f2886ffd640 1 -- 172.21.15.52:0/2902402202 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f289c034ce0 con 0x5597b21ee0c0 2024-04-02T13:54:42.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.332+0000 7f2886ffd640 1 -- 172.21.15.52:0/2902402202 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f289c032bb0 con 0x5597b21ee0c0 2024-04-02T13:54:42.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.332+0000 7f28b0a12740 10 client.16050 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:42.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.332+0000 7f28b0a12740 10 client.16050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:54:42.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.332+0000 7f28b0a12740 20 client.16050 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:42.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.332+0000 7f28b0a12740 1 --2- 172.21.15.52:0/2902402202 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5597b2206e70 0x5597b2209260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.332+0000 7f28b0a12740 4 client.16050 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:54:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.332+0000 7f28b0a12740 1 -- 172.21.15.52:0/2902402202 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597b215ecf0 con 0x5597b2206e70 2024-04-02T13:54:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.332+0000 7f28a77fe640 1 --2- 172.21.15.52:0/2902402202 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5597b2206e70 0x5597b2209260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.332+0000 7f2886ffd640 10 client.16050 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.332+0000 7f28a77fe640 1 --2- 172.21.15.52:0/2902402202 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5597b2206e70 0x5597b2209260 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7caf8bd7df8af96 server_cookie=4ac4a207bbc1fff5 in_seq=0 out_seq=0 2024-04-02T13:54:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.333+0000 7f2886ffd640 1 -- 172.21.15.52:0/2902402202 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5597b215ecf0 con 0x5597b2206e70 2024-04-02T13:54:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.333+0000 7f2886ffd640 10 client.16050 handle_command_reply: tid=0 2024-04-02T13:54:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.333+0000 7f28b0a12740 1 client.16050 shutdown 2024-04-02T13:54:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.333+0000 7f28b0a12740 1 -- 172.21.15.52:0/2902402202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21ee0c0 msgr2=0x5597b21f4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.333+0000 7f28b0a12740 1 --2- 172.21.15.52:0/2902402202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21ee0c0 0x5597b21f4060 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f289c037040 tx=0x7f289c0044a0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.333+0000 7f28b0a12740 1 -- 172.21.15.52:0/2902402202 shutdown_connections 2024-04-02T13:54:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.333+0000 7f28b0a12740 1 --2- 172.21.15.52:0/2902402202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5597b21f6ca0 0x5597b21f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.333+0000 7f28b0a12740 1 --2- 172.21.15.52:0/2902402202 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5597b2206e70 0x5597b2209260 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.333+0000 7f2886ffd640 0 client.16050 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.333+0000 7f28b0a12740 1 --2- 172.21.15.52:0/2902402202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21ee0c0 0x5597b21f4060 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.333+0000 7f28b0a12740 1 --2- 172.21.15.52:0/2902402202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597b21ed7a0 0x5597b21f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.333+0000 7f28b0a12740 1 -- 172.21.15.52:0/2902402202 >> 172.21.15.52:0/2902402202 conn(0x5597b21e7ed0 msgr2=0x5597b21e9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:42.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.334+0000 7f28b0a12740 1 -- 172.21.15.52:0/2902402202 shutdown_connections 2024-04-02T13:54:42.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.334+0000 7f28b0a12740 1 -- 172.21.15.52:0/2902402202 wait complete. 2024-04-02T13:54:42.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.334+0000 7f28b0a12740 20 client.16050 trim_cache size 0 max 16384 2024-04-02T13:54:42.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.347+0000 7f28b0a12740 1 Processor -- start 2024-04-02T13:54:42.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.347+0000 7f28b0a12740 1 -- start start 2024-04-02T13:54:42.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.347+0000 7f28b0a12740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597b21ed7a0 0x5597b21f6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.347+0000 7f28b0a12740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21ee0c0 0x5597b21f2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.348+0000 7f28b0a12740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5597b21f6ca0 0x5597b21f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.348+0000 7f28b0a12740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f28a81b10b0 con 0x5597b21ee0c0 2024-04-02T13:54:42.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.348+0000 7f28b0a12740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f28a81b2bf0 con 0x5597b21f6ca0 2024-04-02T13:54:42.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.348+0000 7f28b0a12740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f28a81b1780 con 0x5597b21ed7a0 2024-04-02T13:54:42.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.348+0000 7f28a6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21ee0c0 0x5597b21f2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.348+0000 7f28a6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21ee0c0 0x5597b21f2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34956/0 (socket says 172.21.15.52:34956) 2024-04-02T13:54:42.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.348+0000 7f28a6ffd640 1 -- 172.21.15.52:0/1010548400 learned_addr learned my addr 172.21.15.52:0/1010548400 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:42.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.348+0000 7f28a7fff640 1 --2- 172.21.15.52:0/1010548400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5597b21f6ca0 0x5597b21f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.348+0000 7f28a77fe640 1 --2- 172.21.15.52:0/1010548400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597b21ed7a0 0x5597b21f6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.348+0000 7f28a6ffd640 1 -- 172.21.15.52:0/1010548400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597b21ed7a0 msgr2=0x5597b21f6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:42.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.348+0000 7f28a6ffd640 1 --2- 172.21.15.52:0/1010548400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597b21ed7a0 0x5597b21f6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.348+0000 7f28a6ffd640 1 -- 172.21.15.52:0/1010548400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5597b21f6ca0 msgr2=0x5597b21f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:42.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.348+0000 7f28a6ffd640 1 --2- 172.21.15.52:0/1010548400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5597b21f6ca0 0x5597b21f43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.348+0000 7f28a6ffd640 1 -- 172.21.15.52:0/1010548400 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597b21f2070 con 0x5597b21ee0c0 2024-04-02T13:54:42.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.348+0000 7f28a6ffd640 1 --2- 172.21.15.52:0/1010548400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21ee0c0 0x5597b21f2c80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f289c0036e0 tx=0x7f289c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f6ec61d659a1a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:42.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.349+0000 7f288d7fa640 1 -- 172.21.15.52:0/1010548400 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c043070 con 0x5597b21ee0c0 2024-04-02T13:54:42.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.349+0000 7f288d7fa640 1 -- 172.21.15.52:0/1010548400 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f289c009450 con 0x5597b21ee0c0 2024-04-02T13:54:42.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.349+0000 7f288d7fa640 1 -- 172.21.15.52:0/1010548400 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c039040 con 0x5597b21ee0c0 2024-04-02T13:54:42.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.349+0000 7f28b0a12740 1 -- 172.21.15.52:0/1010548400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21ee0c0 msgr2=0x5597b21f2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.349+0000 7f28b0a12740 1 --2- 172.21.15.52:0/1010548400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21ee0c0 0x5597b21f2c80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f289c0036e0 tx=0x7f289c0056d0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.349+0000 7f28b0a12740 1 -- 172.21.15.52:0/1010548400 shutdown_connections 2024-04-02T13:54:42.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.349+0000 7f28b0a12740 1 --2- 172.21.15.52:0/1010548400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5597b21f6ca0 0x5597b21f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.349+0000 7f28b0a12740 1 --2- 172.21.15.52:0/1010548400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21ee0c0 0x5597b21f2c80 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.349+0000 7f28b0a12740 1 --2- 172.21.15.52:0/1010548400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597b21ed7a0 0x5597b21f6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.349+0000 7f28b0a12740 1 -- 172.21.15.52:0/1010548400 >> 172.21.15.52:0/1010548400 conn(0x5597b21e7ed0 msgr2=0x5597b2201e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:42.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.349+0000 7f28b0a12740 1 -- 172.21.15.52:0/1010548400 shutdown_connections 2024-04-02T13:54:42.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.349+0000 7f28b0a12740 1 -- 172.21.15.52:0/1010548400 wait complete. 2024-04-02T13:54:42.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.350+0000 7f28b0a12740 1 -- start start 2024-04-02T13:54:42.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.350+0000 7f28b0a12740 1 Processor -- start 2024-04-02T13:54:42.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.350+0000 7f28b0a12740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5597b21ed7a0 0x5597b22074c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.350+0000 7f28b0a12740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21ee0c0 0x5597b2207a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.350+0000 7f28b0a12740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597b21f6ca0 0x5597b2265000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.350+0000 7f28b0a12740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f28a81b2520 con 0x5597b21ee0c0 2024-04-02T13:54:42.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.350+0000 7f28b0a12740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f28a81b1e50 con 0x5597b21ed7a0 2024-04-02T13:54:42.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.350+0000 7f28b0a12740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f28a81af090 con 0x5597b21f6ca0 2024-04-02T13:54:42.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.351+0000 7f28b0a12740 10 client.0 _ll_register_callbacks cb 0x7f28a0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:42.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.351+0000 7f28a6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21ee0c0 0x5597b2207a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.351+0000 7f28a7fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597b21f6ca0 0x5597b2265000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.351+0000 7f28a7fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597b21f6ca0 0x5597b2265000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.52:44014/0 (socket says 172.21.15.52:44014) 2024-04-02T13:54:42.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.351+0000 7f28a7fff640 1 -- 172.21.15.52:0/3227730555 learned_addr learned my addr 172.21.15.52:0/3227730555 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:42.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.351+0000 7f28a77fe640 1 --2- 172.21.15.52:0/3227730555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5597b21ed7a0 0x5597b22074c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.351+0000 7f28a6ffd640 1 -- 172.21.15.52:0/3227730555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597b21f6ca0 msgr2=0x5597b2265000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:42.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.351+0000 7f28a6ffd640 1 --2- 172.21.15.52:0/3227730555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597b21f6ca0 0x5597b2265000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.351+0000 7f28a6ffd640 1 -- 172.21.15.52:0/3227730555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5597b21ed7a0 msgr2=0x5597b22074c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:42.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.351+0000 7f28a6ffd640 1 --2- 172.21.15.52:0/3227730555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5597b21ed7a0 0x5597b22074c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.351+0000 7f28a6ffd640 1 -- 172.21.15.52:0/3227730555 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597b22655c0 con 0x5597b21ee0c0 2024-04-02T13:54:42.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.351+0000 7f28a7fff640 1 --2- 172.21.15.52:0/3227730555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597b21f6ca0 0x5597b2265000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:54:42.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.351+0000 7f28a6ffd640 1 --2- 172.21.15.52:0/3227730555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21ee0c0 0x5597b2207a00 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x5597b21d98a0 tx=0x7f289c0044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04d04883f989f86 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:42.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.351+0000 7f2886ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:42.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.351+0000 7f2886ffd640 1 -- 172.21.15.52:0/3227730555 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c043070 con 0x5597b21ee0c0 2024-04-02T13:54:42.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.351+0000 7f2886ffd640 1 -- 172.21.15.52:0/3227730555 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f289c02c720 con 0x5597b21ee0c0 2024-04-02T13:54:42.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.351+0000 7f28b0a12740 1 -- 172.21.15.52:0/3227730555 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f289c009450 con 0x5597b21ee0c0 2024-04-02T13:54:42.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.352+0000 7f2886ffd640 1 -- 172.21.15.52:0/3227730555 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f289c06f050 con 0x5597b21ee0c0 2024-04-02T13:54:42.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.352+0000 7f28b0a12740 10 client.16056 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:42.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.352+0000 7f28b0a12740 1 -- 172.21.15.52:0/3227730555 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5597b226c970 con 0x5597b21ee0c0 2024-04-02T13:54:42.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.352+0000 7f2886ffd640 1 -- 172.21.15.52:0/3227730555 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f289c009450 con 0x5597b21ee0c0 2024-04-02T13:54:42.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.352+0000 7f2886ffd640 1 -- 172.21.15.52:0/3227730555 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f289c0326d0 con 0x5597b21ee0c0 2024-04-02T13:54:42.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.352+0000 7f28b0a12740 10 client.16056 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:42.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.352+0000 7f28b0a12740 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:54:42.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.352+0000 7f28b0a12740 20 client.16056 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:42.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.352+0000 7f28b0a12740 1 --2- 172.21.15.52:0/3227730555 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5597b226cfa0 0x5597b226f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:42.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.352+0000 7f28b0a12740 4 client.16056 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:54:42.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.352+0000 7f28b0a12740 1 -- 172.21.15.52:0/3227730555 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5597b2159040 con 0x5597b226cfa0 2024-04-02T13:54:42.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.352+0000 7f28a77fe640 1 --2- 172.21.15.52:0/3227730555 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5597b226cfa0 0x5597b226f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:42.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.353+0000 7f2886ffd640 10 client.16056 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:42.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.353+0000 7f28a77fe640 1 --2- 172.21.15.52:0/3227730555 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5597b226cfa0 0x5597b226f3c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9370df89613f4b0 server_cookie=1ad4ffaac066c59b in_seq=0 out_seq=0 2024-04-02T13:54:42.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.353+0000 7f2886ffd640 1 -- 172.21.15.52:0/3227730555 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5597b2159040 con 0x5597b226cfa0 2024-04-02T13:54:42.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.353+0000 7f2886ffd640 10 client.16056 handle_command_reply: tid=0 2024-04-02T13:54:42.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.353+0000 7f28b0a12740 1 client.16056 shutdown 2024-04-02T13:54:42.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.353+0000 7f28b0a12740 1 -- 172.21.15.52:0/3227730555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21ee0c0 msgr2=0x5597b2207a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.353+0000 7f28b0a12740 1 --2- 172.21.15.52:0/3227730555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21ee0c0 0x5597b2207a00 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x5597b21d98a0 tx=0x7f289c0044a0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.353+0000 7f28b0a12740 1 -- 172.21.15.52:0/3227730555 shutdown_connections 2024-04-02T13:54:42.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.353+0000 7f28b0a12740 1 --2- 172.21.15.52:0/3227730555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5597b21f6ca0 0x5597b2265000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.353+0000 7f28b0a12740 1 --2- 172.21.15.52:0/3227730555 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5597b226cfa0 0x5597b226f3c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.353+0000 7f28b0a12740 1 --2- 172.21.15.52:0/3227730555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597b21ee0c0 0x5597b2207a00 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.353+0000 7f2886ffd640 0 client.16056 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:42.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.353+0000 7f28b0a12740 1 --2- 172.21.15.52:0/3227730555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5597b21ed7a0 0x5597b22074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.353+0000 7f28b0a12740 1 -- 172.21.15.52:0/3227730555 >> 172.21.15.52:0/3227730555 conn(0x5597b21e7ed0 msgr2=0x5597b21e66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:42.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.354+0000 7f28b0a12740 1 -- 172.21.15.52:0/3227730555 shutdown_connections 2024-04-02T13:54:42.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.354+0000 7f28b0a12740 1 -- 172.21.15.52:0/3227730555 wait complete. 2024-04-02T13:54:42.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.354+0000 7f28b0a12740 20 client.16056 trim_cache size 0 max 16384 2024-04-02T13:54:42.364 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:54:42.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.354+0000 7f28add50640 1 -- 172.21.15.52:0/1930420843 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f28900a3ed0 msgr2=0x7f28900a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.354+0000 7f28add50640 1 --2- 172.21.15.52:0/1930420843 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f28900a3ed0 0x7f28900a6390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f28a0005fd0 tx=0x7f28a0009500 comp rx=0 tx=0).stop 2024-04-02T13:54:42.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.354+0000 7f28add50640 1 -- 172.21.15.52:0/1930420843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28a8068d00 msgr2=0x7f28a81983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:42.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.354+0000 7f28add50640 1 --2- 172.21.15.52:0/1930420843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28a8068d00 0x7f28a81983c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f289800e9c0 tx=0x7f289800ee80 comp rx=0 tx=0).stop 2024-04-02T13:54:42.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.354+0000 7f28add50640 1 -- 172.21.15.52:0/1930420843 shutdown_connections 2024-04-02T13:54:42.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.354+0000 7f28add50640 1 --2- 172.21.15.52:0/1930420843 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f28900a3ed0 0x7f28900a6390 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.354+0000 7f28add50640 1 --2- 172.21.15.52:0/1930420843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28a8107b50 0x7f28a819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.354+0000 7f28add50640 1 --2- 172.21.15.52:0/1930420843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f28a80696c0 0x7f28a8198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.354+0000 7f28add50640 1 --2- 172.21.15.52:0/1930420843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28a8068d00 0x7f28a81983c0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:42.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.354+0000 7f28add50640 1 -- 172.21.15.52:0/1930420843 >> 172.21.15.52:0/1930420843 conn(0x7f28a8077b00 msgr2=0x7f28a80fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:42.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.354+0000 7f28add50640 1 -- 172.21.15.52:0/1930420843 shutdown_connections 2024-04-02T13:54:42.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:42.354+0000 7f28add50640 1 -- 172.21.15.52:0/1930420843 wait complete. 2024-04-02T13:54:42.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:42 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3849123170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:54:42.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:42 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3849123170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:54:42.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:42 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3849123170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:54:43.369 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:54:43.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:54:43.370 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:54:43.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.475+0000 7faf6891b640 1 -- 172.21.15.52:0/751609684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf6410c140 msgr2=0x7faf6410e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:43.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.475+0000 7faf6891b640 1 --2- 172.21.15.52:0/751609684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf6410c140 0x7faf6410e5b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009ca0 tx=0x7faf5c02c180 comp rx=0 tx=0).stop 2024-04-02T13:54:43.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.475+0000 7faf6891b640 1 -- 172.21.15.52:0/751609684 shutdown_connections 2024-04-02T13:54:43.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.475+0000 7faf6891b640 1 --2- 172.21.15.52:0/751609684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf6410c140 0x7faf6410e5b0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.475+0000 7faf6891b640 1 --2- 172.21.15.52:0/751609684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf64103690 0x7faf6410bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.475+0000 7faf6891b640 1 --2- 172.21.15.52:0/751609684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf64102d70 0x7faf64103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.475+0000 7faf6891b640 1 -- 172.21.15.52:0/751609684 >> 172.21.15.52:0/751609684 conn(0x7faf64077b60 msgr2=0x7faf64077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:43.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.475+0000 7faf6891b640 1 -- 172.21.15.52:0/751609684 shutdown_connections 2024-04-02T13:54:43.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.475+0000 7faf6891b640 1 -- 172.21.15.52:0/751609684 wait complete. 2024-04-02T13:54:43.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.476+0000 7faf6891b640 1 Processor -- start 2024-04-02T13:54:43.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.476+0000 7faf6891b640 1 -- start start 2024-04-02T13:54:43.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.476+0000 7faf6891b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf64102d70 0x7faf641009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.476+0000 7faf6891b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf64103690 0x7faf64100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.476+0000 7faf6891b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf6410c140 0x7faf64101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.476+0000 7faf6891b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faf64111f10 con 0x7faf6410c140 2024-04-02T13:54:43.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.476+0000 7faf6891b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faf64111d90 con 0x7faf64103690 2024-04-02T13:54:43.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.476+0000 7faf6891b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faf64112090 con 0x7faf64102d70 2024-04-02T13:54:43.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.477+0000 7faf62d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf6410c140 0x7faf64101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.477+0000 7faf62d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf6410c140 0x7faf64101440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34970/0 (socket says 172.21.15.52:34970) 2024-04-02T13:54:43.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.477+0000 7faf62d9d640 1 -- 172.21.15.52:0/488644507 learned_addr learned my addr 172.21.15.52:0/488644507 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:43.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.477+0000 7faf6259c640 1 --2- 172.21.15.52:0/488644507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf64102d70 0x7faf641009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.477+0000 7faf61d9b640 1 --2- 172.21.15.52:0/488644507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf64103690 0x7faf64100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.477+0000 7faf62d9d640 1 -- 172.21.15.52:0/488644507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf64102d70 msgr2=0x7faf641009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:43.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.477+0000 7faf62d9d640 1 --2- 172.21.15.52:0/488644507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf64102d70 0x7faf641009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.477+0000 7faf62d9d640 1 -- 172.21.15.52:0/488644507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf64103690 msgr2=0x7faf64100f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:43.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.477+0000 7faf62d9d640 1 --2- 172.21.15.52:0/488644507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf64103690 0x7faf64100f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.477+0000 7faf62d9d640 1 -- 172.21.15.52:0/488644507 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf641a6870 con 0x7faf6410c140 2024-04-02T13:54:43.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.477+0000 7faf62d9d640 1 --2- 172.21.15.52:0/488644507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf6410c140 0x7faf64101440 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7faf5c02c790 tx=0x7faf5c035e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:43.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.477+0000 7faf537fe640 1 -- 172.21.15.52:0/488644507 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c039040 con 0x7faf6410c140 2024-04-02T13:54:43.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.477+0000 7faf537fe640 1 -- 172.21.15.52:0/488644507 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf5c044070 con 0x7faf6410c140 2024-04-02T13:54:43.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.477+0000 7faf6891b640 1 -- 172.21.15.52:0/488644507 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf641a6b00 con 0x7faf6410c140 2024-04-02T13:54:43.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.477+0000 7faf537fe640 1 -- 172.21.15.52:0/488644507 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c032940 con 0x7faf6410c140 2024-04-02T13:54:43.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.478+0000 7faf6891b640 1 -- 172.21.15.52:0/488644507 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf641a7040 con 0x7faf6410c140 2024-04-02T13:54:43.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.480+0000 7faf537fe640 1 -- 172.21.15.52:0/488644507 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7faf5c032ae0 con 0x7faf6410c140 2024-04-02T13:54:43.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.480+0000 7faf6b5d7740 1 Processor -- start 2024-04-02T13:54:43.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.481+0000 7faf6b5d7740 1 -- start start 2024-04-02T13:54:43.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.481+0000 7faf6b5d7740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307aa2c0 0x561b307aa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.481+0000 7faf6b5d7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b307aabe0 0x561b307b3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.481+0000 7faf6b5d7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b307b37c0 0x561b307b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.481+0000 7faf6b5d7740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561b3070e770 con 0x561b307aa2c0 2024-04-02T13:54:43.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.481+0000 7faf6b5d7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x561b306b6960 con 0x561b307b37c0 2024-04-02T13:54:43.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.481+0000 7faf6b5d7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561b3070a8c0 con 0x561b307aabe0 2024-04-02T13:54:43.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.481+0000 7faf537fe640 1 --2- 172.21.15.52:0/488644507 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7faf4c0a3f50 0x7faf4c0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.481+0000 7faf61d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307aa2c0 0x561b307aa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.481+0000 7faf61d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307aa2c0 0x561b307aa6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34986/0 (socket says 172.21.15.52:34986) 2024-04-02T13:54:43.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.481+0000 7faf61d9b640 1 -- 172.21.15.52:0/2830442561 learned_addr learned my addr 172.21.15.52:0/2830442561 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:43.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.481+0000 7faf537fe640 1 -- 172.21.15.52:0/488644507 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7faf5c0f30f0 con 0x7faf6410c140 2024-04-02T13:54:43.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.481+0000 7faf62d9d640 1 --2- 172.21.15.52:0/488644507 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7faf4c0a3f50 0x7faf4c0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.481+0000 7faf6259c640 1 --2- 172.21.15.52:0/2830442561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b307aabe0 0x561b307b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.482+0000 7faf61d9b640 1 -- 172.21.15.52:0/2830442561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b307aabe0 msgr2=0x561b307b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:43.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.482+0000 7faf61d9b640 1 --2- 172.21.15.52:0/2830442561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b307aabe0 0x561b307b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.482+0000 7faf61d9b640 1 -- 172.21.15.52:0/2830442561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b307b37c0 msgr2=0x561b307b5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:43.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.482+0000 7faf61d9b640 1 --2- 172.21.15.52:0/2830442561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b307b37c0 0x561b307b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.482+0000 7faf61d9b640 1 -- 172.21.15.52:0/2830442561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b304ea7e0 con 0x561b307aa2c0 2024-04-02T13:54:43.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.482+0000 7faf62d9d640 1 --2- 172.21.15.52:0/488644507 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7faf4c0a3f50 0x7faf4c0a6410 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0f0430 tx=0x7faf5c0095c0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:43.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.482+0000 7faf61d9b640 1 --2- 172.21.15.52:0/2830442561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307aa2c0 0x561b307aa6a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7faf58009c70 tx=0x7faf58013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7495ec65eb504751 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:43.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.482+0000 7faf50ff9640 1 -- 172.21.15.52:0/2830442561 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf58010620 con 0x561b307aa2c0 2024-04-02T13:54:43.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.482+0000 7faf50ff9640 1 -- 172.21.15.52:0/2830442561 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf58009e20 con 0x561b307aa2c0 2024-04-02T13:54:43.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.482+0000 7faf50ff9640 1 -- 172.21.15.52:0/2830442561 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5801bcc0 con 0x561b307aa2c0 2024-04-02T13:54:43.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.483+0000 7faf6b5d7740 1 -- 172.21.15.52:0/2830442561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307aa2c0 msgr2=0x561b307aa6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:43.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.483+0000 7faf6b5d7740 1 --2- 172.21.15.52:0/2830442561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307aa2c0 0x561b307aa6a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7faf58009c70 tx=0x7faf58013830 comp rx=0 tx=0).stop 2024-04-02T13:54:43.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.483+0000 7faf6b5d7740 1 -- 172.21.15.52:0/2830442561 shutdown_connections 2024-04-02T13:54:43.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.483+0000 7faf6b5d7740 1 --2- 172.21.15.52:0/2830442561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b307b37c0 0x561b307b5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.483+0000 7faf6b5d7740 1 --2- 172.21.15.52:0/2830442561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b307aabe0 0x561b307b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.483+0000 7faf6b5d7740 1 --2- 172.21.15.52:0/2830442561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307aa2c0 0x561b307aa6a0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.483+0000 7faf6b5d7740 1 -- 172.21.15.52:0/2830442561 >> 172.21.15.52:0/2830442561 conn(0x561b307a49f0 msgr2=0x561b307a4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:43.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.483+0000 7faf6b5d7740 1 -- 172.21.15.52:0/2830442561 shutdown_connections 2024-04-02T13:54:43.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.483+0000 7faf6b5d7740 1 -- 172.21.15.52:0/2830442561 wait complete. 2024-04-02T13:54:43.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.483+0000 7faf6b5d7740 1 -- start start 2024-04-02T13:54:43.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.484+0000 7faf6b5d7740 1 Processor -- start 2024-04-02T13:54:43.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.484+0000 7faf6b5d7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b307aa2c0 0x561b307b0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.484+0000 7faf6b5d7740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307aabe0 0x561b307b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.484+0000 7faf6b5d7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b307b37c0 0x561b307b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.484+0000 7faf6b5d7740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561b307b6550 con 0x561b307aabe0 2024-04-02T13:54:43.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.484+0000 7faf6b5d7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x561b306c9d20 con 0x561b307aa2c0 2024-04-02T13:54:43.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.484+0000 7faf6b5d7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561b307b66d0 con 0x561b307b37c0 2024-04-02T13:54:43.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.484+0000 7faf6b5d7740 10 client.0 _ll_register_callbacks cb 0x561b30696a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:43.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.484+0000 7faf6259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307aabe0 0x561b307b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.484+0000 7faf6259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307aabe0 0x561b307b0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34996/0 (socket says 172.21.15.52:34996) 2024-04-02T13:54:43.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.484+0000 7faf6259c640 1 -- 172.21.15.52:0/3891800419 learned_addr learned my addr 172.21.15.52:0/3891800419 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:43.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.484+0000 7faf61d9b640 1 --2- 172.21.15.52:0/3891800419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b307aa2c0 0x561b307b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.484+0000 7faf61d9b640 1 --2- 172.21.15.52:0/3891800419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b307b37c0 0x561b307b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.484+0000 7faf6259c640 1 -- 172.21.15.52:0/3891800419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b307b37c0 msgr2=0x561b307b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:43.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.485+0000 7faf6259c640 1 --2- 172.21.15.52:0/3891800419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b307b37c0 0x561b307b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.485+0000 7faf6259c640 1 -- 172.21.15.52:0/3891800419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b307aa2c0 msgr2=0x561b307b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:43.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.485+0000 7faf6259c640 1 --2- 172.21.15.52:0/3891800419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b307aa2c0 0x561b307b0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.485+0000 7faf6259c640 1 -- 172.21.15.52:0/3891800419 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b3054af70 con 0x561b307aabe0 2024-04-02T13:54:43.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.485+0000 7faf6259c640 1 --2- 172.21.15.52:0/3891800419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307aabe0 0x561b307b0c80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7faf540126b0 tx=0x7faf54012b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33f1ee07d0e8309 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:43.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.485+0000 7faf49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:43.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.485+0000 7faf49ffb640 1 -- 172.21.15.52:0/3891800419 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54004020 con 0x561b307aabe0 2024-04-02T13:54:43.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.485+0000 7faf49ffb640 1 -- 172.21.15.52:0/3891800419 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7faf540041c0 con 0x561b307aabe0 2024-04-02T13:54:43.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.485+0000 7faf6b5d7740 1 -- 172.21.15.52:0/3891800419 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf58009e20 con 0x561b307aabe0 2024-04-02T13:54:43.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.485+0000 7faf49ffb640 1 -- 172.21.15.52:0/3891800419 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faf54025050 con 0x561b307aabe0 2024-04-02T13:54:43.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.485+0000 7faf6b5d7740 10 client.16068 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:43.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.485+0000 7faf6b5d7740 1 -- 172.21.15.52:0/3891800419 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b303e2170 con 0x561b307aabe0 2024-04-02T13:54:43.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.485+0000 7faf49ffb640 1 -- 172.21.15.52:0/3891800419 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf58009e20 con 0x561b307aabe0 2024-04-02T13:54:43.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.486+0000 7faf49ffb640 1 -- 172.21.15.52:0/3891800419 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faf54003a00 con 0x561b307aabe0 2024-04-02T13:54:43.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.486+0000 7faf6b5d7740 10 client.16068 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:43.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.486+0000 7faf6b5d7740 10 client.16068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:54:43.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.486+0000 7faf6b5d7740 20 client.16068 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:43.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.486+0000 7faf6b5d7740 1 --2- 172.21.15.52:0/3891800419 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x561b307c3b10 0x561b307c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.486+0000 7faf6b5d7740 4 client.16068 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:54:43.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.486+0000 7faf6b5d7740 1 -- 172.21.15.52:0/3891800419 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b3071bcb0 con 0x561b307c3b10 2024-04-02T13:54:43.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.486+0000 7faf61d9b640 1 --2- 172.21.15.52:0/3891800419 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x561b307c3b10 0x561b307c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.486+0000 7faf61d9b640 1 --2- 172.21.15.52:0/3891800419 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x561b307c3b10 0x561b307c5f00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=926bb8a2f2a129da server_cookie=6f0c81687fd79bdc in_seq=0 out_seq=0 2024-04-02T13:54:43.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.486+0000 7faf49ffb640 10 client.16068 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:43.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.487+0000 7faf49ffb640 1 -- 172.21.15.52:0/3891800419 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561b3071bcb0 con 0x561b307c3b10 2024-04-02T13:54:43.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.487+0000 7faf49ffb640 10 client.16068 handle_command_reply: tid=0 2024-04-02T13:54:43.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.487+0000 7faf6b5d7740 1 client.16068 shutdown 2024-04-02T13:54:43.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.487+0000 7faf6b5d7740 1 -- 172.21.15.52:0/3891800419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307aabe0 msgr2=0x561b307b0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:43.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.487+0000 7faf6b5d7740 1 --2- 172.21.15.52:0/3891800419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307aabe0 0x561b307b0c80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7faf540126b0 tx=0x7faf54012b70 comp rx=0 tx=0).stop 2024-04-02T13:54:43.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.487+0000 7faf6b5d7740 1 -- 172.21.15.52:0/3891800419 shutdown_connections 2024-04-02T13:54:43.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.487+0000 7faf6b5d7740 1 --2- 172.21.15.52:0/3891800419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b307b37c0 0x561b307b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.487+0000 7faf6b5d7740 1 --2- 172.21.15.52:0/3891800419 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x561b307c3b10 0x561b307c5f00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.487+0000 7faf6b5d7740 1 --2- 172.21.15.52:0/3891800419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307aabe0 0x561b307b0c80 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.487+0000 7faf6b5d7740 1 --2- 172.21.15.52:0/3891800419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b307aa2c0 0x561b307b0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.487+0000 7faf6b5d7740 1 -- 172.21.15.52:0/3891800419 >> 172.21.15.52:0/3891800419 conn(0x561b307a49f0 msgr2=0x561b307a6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:43.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.487+0000 7faf49ffb640 0 client.16068 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:43.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.487+0000 7faf6b5d7740 1 -- 172.21.15.52:0/3891800419 shutdown_connections 2024-04-02T13:54:43.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.487+0000 7faf6b5d7740 1 -- 172.21.15.52:0/3891800419 wait complete. 2024-04-02T13:54:43.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.487+0000 7faf6b5d7740 20 client.16068 trim_cache size 0 max 16384 2024-04-02T13:54:43.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.505+0000 7faf6b5d7740 1 Processor -- start 2024-04-02T13:54:43.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.505+0000 7faf6b5d7740 1 -- start start 2024-04-02T13:54:43.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.506+0000 7faf6b5d7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b307aa2c0 0x561b307b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.506+0000 7faf6b5d7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b307aabe0 0x561b307bf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.506+0000 7faf6b5d7740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307b37c0 0x561b307cb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.506+0000 7faf6b5d7740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faf641b0f90 con 0x561b307b37c0 2024-04-02T13:54:43.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.506+0000 7faf6b5d7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faf641b2ad0 con 0x561b307aabe0 2024-04-02T13:54:43.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.506+0000 7faf6b5d7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faf641b1660 con 0x561b307aa2c0 2024-04-02T13:54:43.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.506+0000 7faf61d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307b37c0 0x561b307cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.506+0000 7faf61d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307b37c0 0x561b307cb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35006/0 (socket says 172.21.15.52:35006) 2024-04-02T13:54:43.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.506+0000 7faf61d9b640 1 -- 172.21.15.52:0/2718449845 learned_addr learned my addr 172.21.15.52:0/2718449845 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:43.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.506+0000 7faf61d9b640 1 --2- 172.21.15.52:0/2718449845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b307aa2c0 0x561b307b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.506+0000 7faf6259c640 1 --2- 172.21.15.52:0/2718449845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b307aabe0 0x561b307bf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.506+0000 7faf61d9b640 1 -- 172.21.15.52:0/2718449845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b307aa2c0 msgr2=0x561b307b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:43.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.506+0000 7faf61d9b640 1 --2- 172.21.15.52:0/2718449845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b307aa2c0 0x561b307b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.506+0000 7faf61d9b640 1 -- 172.21.15.52:0/2718449845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b307aabe0 msgr2=0x561b307bf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:43.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.506+0000 7faf61d9b640 1 --2- 172.21.15.52:0/2718449845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b307aabe0 0x561b307bf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.506+0000 7faf61d9b640 1 -- 172.21.15.52:0/2718449845 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b307a4820 con 0x561b307b37c0 2024-04-02T13:54:43.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.506+0000 7faf6259c640 1 --2- 172.21.15.52:0/2718449845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b307aabe0 0x561b307bf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:54:43.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.507+0000 7faf61d9b640 1 --2- 172.21.15.52:0/2718449845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307b37c0 0x561b307cb810 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7faf580095f0 tx=0x7faf58015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8e9496a343a953 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:43.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.507+0000 7faf4bfff640 1 -- 172.21.15.52:0/2718449845 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf580050b0 con 0x561b307b37c0 2024-04-02T13:54:43.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.507+0000 7faf4bfff640 1 -- 172.21.15.52:0/2718449845 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf58005250 con 0x561b307b37c0 2024-04-02T13:54:43.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.507+0000 7faf4bfff640 1 -- 172.21.15.52:0/2718449845 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5802f580 con 0x561b307b37c0 2024-04-02T13:54:43.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.508+0000 7faf6b5d7740 1 -- 172.21.15.52:0/2718449845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307b37c0 msgr2=0x561b307cb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:43.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.508+0000 7faf6b5d7740 1 --2- 172.21.15.52:0/2718449845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307b37c0 0x561b307cb810 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7faf580095f0 tx=0x7faf58015040 comp rx=0 tx=0).stop 2024-04-02T13:54:43.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.508+0000 7faf6b5d7740 1 -- 172.21.15.52:0/2718449845 shutdown_connections 2024-04-02T13:54:43.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.508+0000 7faf6b5d7740 1 --2- 172.21.15.52:0/2718449845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307b37c0 0x561b307cb810 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.508+0000 7faf6b5d7740 1 --2- 172.21.15.52:0/2718449845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b307aabe0 0x561b307bf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.508+0000 7faf6b5d7740 1 --2- 172.21.15.52:0/2718449845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b307aa2c0 0x561b307b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.508+0000 7faf6b5d7740 1 -- 172.21.15.52:0/2718449845 >> 172.21.15.52:0/2718449845 conn(0x561b307a49f0 msgr2=0x561b307a3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:43.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.508+0000 7faf6b5d7740 1 -- 172.21.15.52:0/2718449845 shutdown_connections 2024-04-02T13:54:43.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.508+0000 7faf6b5d7740 1 -- 172.21.15.52:0/2718449845 wait complete. 2024-04-02T13:54:43.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.508+0000 7faf6b5d7740 1 -- start start 2024-04-02T13:54:43.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.508+0000 7faf6b5d7740 1 Processor -- start 2024-04-02T13:54:43.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.508+0000 7faf6b5d7740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307aa2c0 0x561b307bb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.509+0000 7faf6b5d7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b307aabe0 0x561b307bbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.509+0000 7faf6b5d7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b307b37c0 0x561b307b8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.509+0000 7faf6b5d7740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faf641b2400 con 0x561b307aa2c0 2024-04-02T13:54:43.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.509+0000 7faf6b5d7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faf641b1d30 con 0x561b307aabe0 2024-04-02T13:54:43.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.509+0000 7faf6b5d7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faf641aef70 con 0x561b307b37c0 2024-04-02T13:54:43.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.509+0000 7faf6b5d7740 10 client.0 _ll_register_callbacks cb 0x7faf54009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:43.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.509+0000 7faf61d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307aa2c0 0x561b307bb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.509+0000 7faf61d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307aa2c0 0x561b307bb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35010/0 (socket says 172.21.15.52:35010) 2024-04-02T13:54:43.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.509+0000 7faf61d9b640 1 -- 172.21.15.52:0/2445690783 learned_addr learned my addr 172.21.15.52:0/2445690783 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:43.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.509+0000 7faf6259c640 1 --2- 172.21.15.52:0/2445690783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b307aabe0 0x561b307bbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.509+0000 7faf61d9b640 1 -- 172.21.15.52:0/2445690783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b307b37c0 msgr2=0x561b307b8980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:54:43.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.509+0000 7faf61d9b640 1 --2- 172.21.15.52:0/2445690783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b307b37c0 0x561b307b8980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.509+0000 7faf61d9b640 1 -- 172.21.15.52:0/2445690783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b307aabe0 msgr2=0x561b307bbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:43.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.509+0000 7faf61d9b640 1 --2- 172.21.15.52:0/2445690783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b307aabe0 0x561b307bbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.509+0000 7faf61d9b640 1 -- 172.21.15.52:0/2445690783 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b307b8fc0 con 0x561b307aa2c0 2024-04-02T13:54:43.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.509+0000 7faf61d9b640 1 --2- 172.21.15.52:0/2445690783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307aa2c0 0x561b307bb9b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7faf5801b550 tx=0x7faf5801b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489bcec537d9e505 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:43.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.509+0000 7faf49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:54:43.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.510+0000 7faf49ffb640 1 -- 172.21.15.52:0/2445690783 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf58009920 con 0x561b307aa2c0 2024-04-02T13:54:43.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.510+0000 7faf49ffb640 1 -- 172.21.15.52:0/2445690783 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7faf5801b950 con 0x561b307aa2c0 2024-04-02T13:54:43.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.510+0000 7faf6b5d7740 1 -- 172.21.15.52:0/2445690783 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf58005250 con 0x561b307aa2c0 2024-04-02T13:54:43.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.510+0000 7faf49ffb640 1 -- 172.21.15.52:0/2445690783 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faf58005070 con 0x561b307aa2c0 2024-04-02T13:54:43.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.510+0000 7faf6b5d7740 10 client.16074 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:43.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.510+0000 7faf6b5d7740 1 -- 172.21.15.52:0/2445690783 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b3082a980 con 0x561b307aa2c0 2024-04-02T13:54:43.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.510+0000 7faf49ffb640 1 -- 172.21.15.52:0/2445690783 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf58005250 con 0x561b307aa2c0 2024-04-02T13:54:43.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.510+0000 7faf49ffb640 1 -- 172.21.15.52:0/2445690783 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faf58027040 con 0x561b307aa2c0 2024-04-02T13:54:43.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.511+0000 7faf6b5d7740 10 client.16074 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:43.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.511+0000 7faf6b5d7740 10 client.16074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:54:43.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.511+0000 7faf6b5d7740 20 client.16074 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:43.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.511+0000 7faf6b5d7740 1 --2- 172.21.15.52:0/2445690783 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x561b3082ae60 0x561b3082d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.511+0000 7faf6b5d7740 4 client.16074 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:54:43.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.511+0000 7faf6b5d7740 1 -- 172.21.15.52:0/2445690783 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561b30716000 con 0x561b3082ae60 2024-04-02T13:54:43.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.511+0000 7faf6259c640 1 --2- 172.21.15.52:0/2445690783 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x561b3082ae60 0x561b3082d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.512+0000 7faf49ffb640 10 client.16074 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:43.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.512+0000 7faf6259c640 1 --2- 172.21.15.52:0/2445690783 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x561b3082ae60 0x561b3082d250 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70efb9cc516116b4 server_cookie=7ff352967f9aae7e in_seq=0 out_seq=0 2024-04-02T13:54:43.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.513+0000 7faf49ffb640 1 -- 172.21.15.52:0/2445690783 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561b30716000 con 0x561b3082ae60 2024-04-02T13:54:43.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.513+0000 7faf49ffb640 10 client.16074 handle_command_reply: tid=0 2024-04-02T13:54:43.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.513+0000 7faf6b5d7740 1 client.16074 shutdown 2024-04-02T13:54:43.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.513+0000 7faf6b5d7740 1 -- 172.21.15.52:0/2445690783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307aa2c0 msgr2=0x561b307bb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:43.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.513+0000 7faf6b5d7740 1 --2- 172.21.15.52:0/2445690783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307aa2c0 0x561b307bb9b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7faf5801b550 tx=0x7faf5801b8a0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.513+0000 7faf6b5d7740 1 -- 172.21.15.52:0/2445690783 shutdown_connections 2024-04-02T13:54:43.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.513+0000 7faf6b5d7740 1 --2- 172.21.15.52:0/2445690783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b307b37c0 0x561b307b8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.513+0000 7faf6b5d7740 1 --2- 172.21.15.52:0/2445690783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561b307aabe0 0x561b307bbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.513+0000 7faf6b5d7740 1 --2- 172.21.15.52:0/2445690783 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x561b3082ae60 0x561b3082d250 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.513+0000 7faf49ffb640 0 client.16074 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:43.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.513+0000 7faf6b5d7740 1 --2- 172.21.15.52:0/2445690783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b307aa2c0 0x561b307bb9b0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.513+0000 7faf6b5d7740 1 -- 172.21.15.52:0/2445690783 >> 172.21.15.52:0/2445690783 conn(0x561b307a49f0 msgr2=0x561b307b0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:43.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.513+0000 7faf6b5d7740 1 -- 172.21.15.52:0/2445690783 shutdown_connections 2024-04-02T13:54:43.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.513+0000 7faf6b5d7740 1 -- 172.21.15.52:0/2445690783 wait complete. 2024-04-02T13:54:43.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.513+0000 7faf6b5d7740 20 client.16074 trim_cache size 0 max 16384 2024-04-02T13:54:43.521 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:54:43.521 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-04-02T13:54:43.521 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "aedc2fd7-95b2-4cfa-b105-da6182892f71", 2024-04-02T13:54:43.521 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-04-02T13:54:43.521 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:54:43.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.514+0000 7faf6891b640 1 -- 172.21.15.52:0/488644507 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7faf4c0a3f50 msgr2=0x7faf4c0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:43.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.514+0000 7faf6891b640 1 --2- 172.21.15.52:0/488644507 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7faf4c0a3f50 0x7faf4c0a6410 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0f0430 tx=0x7faf5c0095c0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.514+0000 7faf6891b640 1 -- 172.21.15.52:0/488644507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf6410c140 msgr2=0x7faf64101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:43.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.514+0000 7faf6891b640 1 --2- 172.21.15.52:0/488644507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf6410c140 0x7faf64101440 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7faf5c02c790 tx=0x7faf5c035e00 comp rx=0 tx=0).stop 2024-04-02T13:54:43.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.514+0000 7faf6891b640 1 -- 172.21.15.52:0/488644507 shutdown_connections 2024-04-02T13:54:43.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.514+0000 7faf6891b640 1 --2- 172.21.15.52:0/488644507 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7faf4c0a3f50 0x7faf4c0a6410 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.514+0000 7faf6891b640 1 --2- 172.21.15.52:0/488644507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf6410c140 0x7faf64101440 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.514+0000 7faf6891b640 1 --2- 172.21.15.52:0/488644507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf64103690 0x7faf64100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.514+0000 7faf6891b640 1 --2- 172.21.15.52:0/488644507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf64102d70 0x7faf641009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.514+0000 7faf6891b640 1 -- 172.21.15.52:0/488644507 >> 172.21.15.52:0/488644507 conn(0x7faf64077b60 msgr2=0x7faf640fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:43.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.514+0000 7faf6891b640 1 -- 172.21.15.52:0/488644507 shutdown_connections 2024-04-02T13:54:43.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.514+0000 7faf6891b640 1 -- 172.21.15.52:0/488644507 wait complete. 2024-04-02T13:54:43.529 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:54:43.529 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:54:43.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:43 smithi052 ceph-mon[32104]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:54:43.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.632+0000 7f45b2293640 1 -- 172.21.15.52:0/1996221163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45ac068d00 msgr2=0x7f45ac069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:43.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.632+0000 7f45b2293640 1 --2- 172.21.15.52:0/1996221163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45ac068d00 0x7f45ac069180 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009930 tx=0x7f45a002bc30 comp rx=0 tx=0).stop 2024-04-02T13:54:43.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.632+0000 7f45b2293640 1 -- 172.21.15.52:0/1996221163 shutdown_connections 2024-04-02T13:54:43.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.632+0000 7f45b2293640 1 --2- 172.21.15.52:0/1996221163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45ac0696c0 0x7f45ac10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.632+0000 7f45b2293640 1 --2- 172.21.15.52:0/1996221163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45ac068d00 0x7f45ac069180 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.632+0000 7f45b2293640 1 --2- 172.21.15.52:0/1996221163 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45ac107b50 0x7f45ac107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.632+0000 7f45b2293640 1 -- 172.21.15.52:0/1996221163 >> 172.21.15.52:0/1996221163 conn(0x7f45ac077b00 msgr2=0x7f45ac077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:43.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.632+0000 7f45b2293640 1 -- 172.21.15.52:0/1996221163 shutdown_connections 2024-04-02T13:54:43.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.632+0000 7f45b2293640 1 -- 172.21.15.52:0/1996221163 wait complete. 2024-04-02T13:54:43.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.633+0000 7f45b2293640 1 Processor -- start 2024-04-02T13:54:43.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.633+0000 7f45b2293640 1 -- start start 2024-04-02T13:54:43.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.634+0000 7f45b2293640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45ac068d00 0x7f45ac198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.634+0000 7f45b2293640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45ac0696c0 0x7f45ac1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.634+0000 7f45b2293640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45ac107b50 0x7f45ac19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.634+0000 7f45b2293640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac111ea0 con 0x7f45ac107b50 2024-04-02T13:54:43.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.634+0000 7f45b2293640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac111d20 con 0x7f45ac0696c0 2024-04-02T13:54:43.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.634+0000 7f45b2293640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac112020 con 0x7f45ac068d00 2024-04-02T13:54:43.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.634+0000 7f45b0830640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45ac107b50 0x7f45ac19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.634+0000 7f45b0830640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45ac107b50 0x7f45ac19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35030/0 (socket says 172.21.15.52:35030) 2024-04-02T13:54:43.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.634+0000 7f45ab7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45ac0696c0 0x7f45ac1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.634+0000 7f45b0830640 1 -- 172.21.15.52:0/3665924484 learned_addr learned my addr 172.21.15.52:0/3665924484 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:43.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.634+0000 7f45abfff640 1 --2- 172.21.15.52:0/3665924484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45ac068d00 0x7f45ac198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.634+0000 7f45b0830640 1 -- 172.21.15.52:0/3665924484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45ac068d00 msgr2=0x7f45ac198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:43.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.634+0000 7f45b0830640 1 --2- 172.21.15.52:0/3665924484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45ac068d00 0x7f45ac198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.634+0000 7f45b0830640 1 -- 172.21.15.52:0/3665924484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45ac0696c0 msgr2=0x7f45ac1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:43.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.634+0000 7f45b0830640 1 --2- 172.21.15.52:0/3665924484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45ac0696c0 0x7f45ac1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.634+0000 7f45b0830640 1 -- 172.21.15.52:0/3665924484 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ac19d350 con 0x7f45ac107b50 2024-04-02T13:54:43.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.634+0000 7f45ab7fe640 1 --2- 172.21.15.52:0/3665924484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45ac0696c0 0x7f45ac1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:54:43.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.635+0000 7f45b0830640 1 --2- 172.21.15.52:0/3665924484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45ac107b50 0x7f45ac19cc20 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f45a400eb10 tx=0x7f45a400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:43.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.635+0000 7f45a97fa640 1 -- 172.21.15.52:0/3665924484 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a4004b20 con 0x7f45ac107b50 2024-04-02T13:54:43.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.635+0000 7f45a97fa640 1 -- 172.21.15.52:0/3665924484 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45a4004cc0 con 0x7f45ac107b50 2024-04-02T13:54:43.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.635+0000 7f45b2293640 1 -- 172.21.15.52:0/3665924484 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45ac19d640 con 0x7f45ac107b50 2024-04-02T13:54:43.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.636+0000 7f45a97fa640 1 -- 172.21.15.52:0/3665924484 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a4010760 con 0x7f45ac107b50 2024-04-02T13:54:43.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.636+0000 7f45b2293640 1 -- 172.21.15.52:0/3665924484 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45ac1a4ed0 con 0x7f45ac107b50 2024-04-02T13:54:43.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.638+0000 7f45a97fa640 1 -- 172.21.15.52:0/3665924484 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f45a4005000 con 0x7f45ac107b50 2024-04-02T13:54:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.639+0000 7f45b4b22740 1 Processor -- start 2024-04-02T13:54:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.639+0000 7f45b4b22740 1 -- start start 2024-04-02T13:54:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.640+0000 7f45b4b22740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d168d0 0x557c93d16cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.640+0000 7f45b4b22740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93d171f0 0x557c93d1f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.640+0000 7f45b4b22740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d1fdd0 0x557c93d221c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.640+0000 7f45b4b22740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557c93c7a830 con 0x557c93d1fdd0 2024-04-02T13:54:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.640+0000 7f45b4b22740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557c93c22820 con 0x557c93d168d0 2024-04-02T13:54:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.640+0000 7f45b4b22740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557c93c767e0 con 0x557c93d171f0 2024-04-02T13:54:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.640+0000 7f45a97fa640 1 --2- 172.21.15.52:0/3665924484 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f45940a3e30 0x7f45940a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.640+0000 7f45b0830640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d1fdd0 0x557c93d221c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.640+0000 7f45b0830640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d1fdd0 0x557c93d221c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35040/0 (socket says 172.21.15.52:35040) 2024-04-02T13:54:43.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.640+0000 7f45b0830640 1 -- 172.21.15.52:0/3191479887 learned_addr learned my addr 172.21.15.52:0/3191479887 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:43.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.640+0000 7f45ab7fe640 1 --2- 172.21.15.52:0/3191479887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d168d0 0x557c93d16cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.640+0000 7f45ab7fe640 1 --2- 172.21.15.52:0/3191479887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93d171f0 0x557c93d1f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.640+0000 7f45ab7fe640 1 -- 172.21.15.52:0/3191479887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93d171f0 msgr2=0x557c93d1f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:43.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.640+0000 7f45ab7fe640 1 --2- 172.21.15.52:0/3191479887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93d171f0 0x557c93d1f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.640+0000 7f45ab7fe640 1 -- 172.21.15.52:0/3191479887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d1fdd0 msgr2=0x557c93d221c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:43.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.640+0000 7f45ab7fe640 1 --2- 172.21.15.52:0/3191479887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d1fdd0 0x557c93d221c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.640+0000 7f45ab7fe640 1 -- 172.21.15.52:0/3191479887 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c93a587d0 con 0x557c93d168d0 2024-04-02T13:54:43.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.640+0000 7f45b0830640 1 --2- 172.21.15.52:0/3191479887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d1fdd0 0x557c93d221c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:54:43.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.640+0000 7f45abfff640 1 --2- 172.21.15.52:0/3665924484 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f45940a3e30 0x7f45940a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.641+0000 7f45ab7fe640 1 --2- 172.21.15.52:0/3191479887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d168d0 0x557c93d16cb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009230 tx=0x7f45a0005bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d521d2c56aa568b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:43.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.641+0000 7f45927fc640 1 -- 172.21.15.52:0/3191479887 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0004030 con 0x557c93d168d0 2024-04-02T13:54:43.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.641+0000 7f45927fc640 1 -- 172.21.15.52:0/3191479887 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45a0047070 con 0x557c93d168d0 2024-04-02T13:54:43.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.641+0000 7f45927fc640 1 -- 172.21.15.52:0/3191479887 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a00052b0 con 0x557c93d168d0 2024-04-02T13:54:43.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.641+0000 7f45a97fa640 1 -- 172.21.15.52:0/3665924484 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f45a40d1290 con 0x7f45ac107b50 2024-04-02T13:54:43.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.642+0000 7f45b4b22740 1 -- 172.21.15.52:0/3191479887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d168d0 msgr2=0x557c93d16cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:43.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.642+0000 7f45abfff640 1 --2- 172.21.15.52:0/3665924484 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f45940a3e30 0x7f45940a62f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f459c00bbe0 tx=0x7f459c00b390 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:43.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.642+0000 7f45b4b22740 1 --2- 172.21.15.52:0/3191479887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d168d0 0x557c93d16cb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009230 tx=0x7f45a0005bb0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.642+0000 7f45b4b22740 1 -- 172.21.15.52:0/3191479887 shutdown_connections 2024-04-02T13:54:43.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.642+0000 7f45b4b22740 1 --2- 172.21.15.52:0/3191479887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d1fdd0 0x557c93d221c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.642+0000 7f45b4b22740 1 --2- 172.21.15.52:0/3191479887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93d171f0 0x557c93d1f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.642+0000 7f45b4b22740 1 --2- 172.21.15.52:0/3191479887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d168d0 0x557c93d16cb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.642+0000 7f45b4b22740 1 -- 172.21.15.52:0/3191479887 >> 172.21.15.52:0/3191479887 conn(0x557c93d11000 msgr2=0x557c93d113e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:43.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.642+0000 7f45b4b22740 1 -- 172.21.15.52:0/3191479887 shutdown_connections 2024-04-02T13:54:43.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.642+0000 7f45b4b22740 1 -- 172.21.15.52:0/3191479887 wait complete. 2024-04-02T13:54:43.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.642+0000 7f45b4b22740 1 -- start start 2024-04-02T13:54:43.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.643+0000 7f45b4b22740 1 Processor -- start 2024-04-02T13:54:43.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.643+0000 7f45b4b22740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d168d0 0x557c93d1f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.643+0000 7f45b4b22740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d171f0 0x557c93d1ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.643+0000 7f45b4b22740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93d1fdd0 0x557c93d1cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.643+0000 7f45b4b22740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557c93d26f70 con 0x557c93d171f0 2024-04-02T13:54:43.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.643+0000 7f45b4b22740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557c93c35d30 con 0x557c93d168d0 2024-04-02T13:54:43.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.643+0000 7f45b4b22740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557c93d270f0 con 0x557c93d1fdd0 2024-04-02T13:54:43.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.643+0000 7f45ab7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d171f0 0x557c93d1ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.643+0000 7f45abfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d168d0 0x557c93d1f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.643+0000 7f45ab7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d171f0 0x557c93d1ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35056/0 (socket says 172.21.15.52:35056) 2024-04-02T13:54:43.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.643+0000 7f45abfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d168d0 0x557c93d1f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:56446/0 (socket says 172.21.15.52:56446) 2024-04-02T13:54:43.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.643+0000 7f45ab7fe640 1 -- 172.21.15.52:0/2113005137 learned_addr learned my addr 172.21.15.52:0/2113005137 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:43.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.643+0000 7f45b4b22740 10 client.0 _ll_register_callbacks cb 0x557c93c35720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:43.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.643+0000 7f45b0830640 1 --2- 172.21.15.52:0/2113005137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93d1fdd0 0x557c93d1cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.644+0000 7f45abfff640 1 -- 172.21.15.52:0/2113005137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93d1fdd0 msgr2=0x557c93d1cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:43.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.644+0000 7f45abfff640 1 --2- 172.21.15.52:0/2113005137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93d1fdd0 0x557c93d1cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.644+0000 7f45abfff640 1 -- 172.21.15.52:0/2113005137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d171f0 msgr2=0x557c93d1ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.644+0000 7f45abfff640 1 --2- 172.21.15.52:0/2113005137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d171f0 0x557c93d1ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.644+0000 7f45abfff640 1 -- 172.21.15.52:0/2113005137 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c9398e3f0 con 0x557c93d168d0 2024-04-02T13:54:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.644+0000 7f45ab7fe640 1 --2- 172.21.15.52:0/2113005137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d171f0 0x557c93d1ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:54:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.644+0000 7f45abfff640 1 --2- 172.21.15.52:0/2113005137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d168d0 0x557c93d1f6d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f459c010860 tx=0x7f459c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2fc433f735805b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.644+0000 7f458b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-02T13:54:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.644+0000 7f458b7fe640 1 -- 172.21.15.52:0/2113005137 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c005220 con 0x557c93d168d0 2024-04-02T13:54:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.644+0000 7f45b4b22740 1 -- 172.21.15.52:0/2113005137 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45a0047070 con 0x557c93d168d0 2024-04-02T13:54:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.644+0000 7f458b7fe640 1 -- 172.21.15.52:0/2113005137 <== mon.2 v2:172.21.15.88:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f459c0053c0 con 0x557c93d168d0 2024-04-02T13:54:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.644+0000 7f458b7fe640 1 -- 172.21.15.52:0/2113005137 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f459c02e050 con 0x557c93d168d0 2024-04-02T13:54:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.644+0000 7f45b4b22740 10 client.25619 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.644+0000 7f45b4b22740 1 -- 172.21.15.52:0/2113005137 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557c9394e0f0 con 0x557c93d168d0 2024-04-02T13:54:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.645+0000 7f458b7fe640 1 -- 172.21.15.52:0/2113005137 <== mon.2 v2:172.21.15.88:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45a0047070 con 0x557c93d168d0 2024-04-02T13:54:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.645+0000 7f458b7fe640 1 -- 172.21.15.52:0/2113005137 <== mon.2 v2:172.21.15.88:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f459c01ebb0 con 0x557c93d168d0 2024-04-02T13:54:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.645+0000 7f45b4b22740 10 client.25619 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:43.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.645+0000 7f45b4b22740 10 client.25619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:54:43.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.645+0000 7f45b4b22740 20 client.25619 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:43.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.645+0000 7f45b4b22740 1 --2- 172.21.15.52:0/2113005137 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557c93d341a0 0x557c93d36660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.645+0000 7f45b4b22740 4 client.25619 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:54:43.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.645+0000 7f45b4b22740 1 -- 172.21.15.52:0/2113005137 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c93c87d70 con 0x557c93d341a0 2024-04-02T13:54:43.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.646+0000 7f45ab7fe640 1 --2- 172.21.15.52:0/2113005137 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557c93d341a0 0x557c93d36660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.646+0000 7f45ab7fe640 1 --2- 172.21.15.52:0/2113005137 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557c93d341a0 0x557c93d36660 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4e81d5ad221782f server_cookie=3ccce84aee1a4858 in_seq=0 out_seq=0 2024-04-02T13:54:43.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.646+0000 7f458b7fe640 10 client.25619 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:43.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.647+0000 7f458b7fe640 1 -- 172.21.15.52:0/2113005137 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557c93c87d70 con 0x557c93d341a0 2024-04-02T13:54:43.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.647+0000 7f458b7fe640 10 client.25619 handle_command_reply: tid=0 2024-04-02T13:54:43.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.647+0000 7f45b4b22740 1 client.25619 shutdown 2024-04-02T13:54:43.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.647+0000 7f45b4b22740 1 -- 172.21.15.52:0/2113005137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d168d0 msgr2=0x557c93d1f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:43.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.647+0000 7f45b4b22740 1 --2- 172.21.15.52:0/2113005137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d168d0 0x557c93d1f6d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f459c010860 tx=0x7f459c004730 comp rx=0 tx=0).stop 2024-04-02T13:54:43.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.647+0000 7f45b4b22740 1 -- 172.21.15.52:0/2113005137 shutdown_connections 2024-04-02T13:54:43.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.647+0000 7f45b4b22740 1 --2- 172.21.15.52:0/2113005137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93d1fdd0 0x557c93d1cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.647+0000 7f45b4b22740 1 --2- 172.21.15.52:0/2113005137 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557c93d341a0 0x557c93d36660 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.647+0000 7f45b4b22740 1 --2- 172.21.15.52:0/2113005137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d171f0 0x557c93d1ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.647+0000 7f45b4b22740 1 --2- 172.21.15.52:0/2113005137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d168d0 0x557c93d1f6d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.647+0000 7f458b7fe640 0 client.25619 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:43.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.647+0000 7f45b4b22740 1 -- 172.21.15.52:0/2113005137 >> 172.21.15.52:0/2113005137 conn(0x557c93d11000 msgr2=0x557c93d12d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:43.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.647+0000 7f45b4b22740 1 -- 172.21.15.52:0/2113005137 shutdown_connections 2024-04-02T13:54:43.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.647+0000 7f45b4b22740 1 -- 172.21.15.52:0/2113005137 wait complete. 2024-04-02T13:54:43.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.648+0000 7f45b4b22740 20 client.25619 trim_cache size 0 max 16384 2024-04-02T13:54:43.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:43 smithi088 ceph-mon[36528]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:54:43.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.683+0000 7f45b4b22740 1 Processor -- start 2024-04-02T13:54:43.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.683+0000 7f45b4b22740 1 -- start start 2024-04-02T13:54:43.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.683+0000 7f45b4b22740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d168d0 0x557c93d100c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.683+0000 7f45b4b22740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93d171f0 0x557c93d1f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.683+0000 7f45b4b22740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d1fdd0 0x557c93d1f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.683+0000 7f45b4b22740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac1ac940 con 0x557c93d168d0 2024-04-02T13:54:43.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.683+0000 7f45b4b22740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac1ae480 con 0x557c93d1fdd0 2024-04-02T13:54:43.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.683+0000 7f45b4b22740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac1ad010 con 0x557c93d171f0 2024-04-02T13:54:43.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.683+0000 7f45abfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d168d0 0x557c93d100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.683+0000 7f45abfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d168d0 0x557c93d100c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35064/0 (socket says 172.21.15.52:35064) 2024-04-02T13:54:43.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.683+0000 7f45abfff640 1 -- 172.21.15.52:0/3932976034 learned_addr learned my addr 172.21.15.52:0/3932976034 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:43.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.683+0000 7f45b0830640 1 --2- 172.21.15.52:0/3932976034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d1fdd0 0x557c93d1f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.683+0000 7f45ab7fe640 1 --2- 172.21.15.52:0/3932976034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93d171f0 0x557c93d1f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.684+0000 7f45abfff640 1 -- 172.21.15.52:0/3932976034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93d171f0 msgr2=0x557c93d1f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:43.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.684+0000 7f45abfff640 1 --2- 172.21.15.52:0/3932976034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93d171f0 0x557c93d1f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.684+0000 7f45abfff640 1 -- 172.21.15.52:0/3932976034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d1fdd0 msgr2=0x557c93d1f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:43.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.684+0000 7f45abfff640 1 --2- 172.21.15.52:0/3932976034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d1fdd0 0x557c93d1f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.684+0000 7f45abfff640 1 -- 172.21.15.52:0/3932976034 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c93d10e30 con 0x557c93d168d0 2024-04-02T13:54:43.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.684+0000 7f45b0830640 1 --2- 172.21.15.52:0/3932976034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d1fdd0 0x557c93d1f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:54:43.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.684+0000 7f45abfff640 1 --2- 172.21.15.52:0/3932976034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d168d0 0x557c93d100c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f459c010a70 tx=0x7f459c00b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a614e9f4b93441 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:43.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.684+0000 7f4591ffb640 1 -- 172.21.15.52:0/3932976034 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c026a10 con 0x557c93d168d0 2024-04-02T13:54:43.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.684+0000 7f4591ffb640 1 -- 172.21.15.52:0/3932976034 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f459c02b050 con 0x557c93d168d0 2024-04-02T13:54:43.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.684+0000 7f4591ffb640 1 -- 172.21.15.52:0/3932976034 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c004a90 con 0x557c93d168d0 2024-04-02T13:54:43.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.685+0000 7f45b4b22740 1 -- 172.21.15.52:0/3932976034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d168d0 msgr2=0x557c93d100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:43.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.685+0000 7f45b4b22740 1 --2- 172.21.15.52:0/3932976034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d168d0 0x557c93d100c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f459c010a70 tx=0x7f459c00b9a0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.685+0000 7f45b4b22740 1 -- 172.21.15.52:0/3932976034 shutdown_connections 2024-04-02T13:54:43.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.685+0000 7f45b4b22740 1 --2- 172.21.15.52:0/3932976034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d1fdd0 0x557c93d1f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.685+0000 7f45b4b22740 1 --2- 172.21.15.52:0/3932976034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93d171f0 0x557c93d1f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.685+0000 7f45b4b22740 1 --2- 172.21.15.52:0/3932976034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d168d0 0x557c93d100c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.685+0000 7f45b4b22740 1 -- 172.21.15.52:0/3932976034 >> 172.21.15.52:0/3932976034 conn(0x557c93d11000 msgr2=0x557c93d0f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:43.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.685+0000 7f45b4b22740 1 -- 172.21.15.52:0/3932976034 shutdown_connections 2024-04-02T13:54:43.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.685+0000 7f45b4b22740 1 -- 172.21.15.52:0/3932976034 wait complete. 2024-04-02T13:54:43.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.685+0000 7f45b4b22740 1 -- start start 2024-04-02T13:54:43.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.686+0000 7f45b4b22740 1 Processor -- start 2024-04-02T13:54:43.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.686+0000 7f45b4b22740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d168d0 0x557c93d28390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.686+0000 7f45b4b22740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93d171f0 0x557c93d288d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.686+0000 7f45b4b22740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d1fdd0 0x557c93d997a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.686+0000 7f45b4b22740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac1addb0 con 0x557c93d1fdd0 2024-04-02T13:54:43.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.686+0000 7f45b4b22740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac1ad6e0 con 0x557c93d168d0 2024-04-02T13:54:43.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.686+0000 7f45b4b22740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac1aa920 con 0x557c93d171f0 2024-04-02T13:54:43.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.686+0000 7f45abfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d168d0 0x557c93d28390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.686+0000 7f45b0830640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d1fdd0 0x557c93d997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.686+0000 7f45abfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d168d0 0x557c93d28390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:56464/0 (socket says 172.21.15.52:56464) 2024-04-02T13:54:43.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.686+0000 7f45abfff640 1 -- 172.21.15.52:0/1531325636 learned_addr learned my addr 172.21.15.52:0/1531325636 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:54:43.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.686+0000 7f45b4b22740 10 client.0 _ll_register_callbacks cb 0x7f459c010ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:43.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.686+0000 7f45ab7fe640 1 --2- 172.21.15.52:0/1531325636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93d171f0 0x557c93d288d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.687+0000 7f45abfff640 1 -- 172.21.15.52:0/1531325636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93d171f0 msgr2=0x557c93d288d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:43.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.687+0000 7f45abfff640 1 --2- 172.21.15.52:0/1531325636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93d171f0 0x557c93d288d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.687+0000 7f45abfff640 1 -- 172.21.15.52:0/1531325636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d1fdd0 msgr2=0x557c93d997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:43.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.687+0000 7f45abfff640 1 --2- 172.21.15.52:0/1531325636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d1fdd0 0x557c93d997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.687+0000 7f45abfff640 1 -- 172.21.15.52:0/1531325636 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c93d99ce0 con 0x557c93d168d0 2024-04-02T13:54:43.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.687+0000 7f45b0830640 1 --2- 172.21.15.52:0/1531325636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d1fdd0 0x557c93d997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:54:43.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.687+0000 7f45abfff640 1 --2- 172.21.15.52:0/1531325636 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d168d0 0x557c93d28390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f459c0264f0 tx=0x7f459c00b480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fbd9887b1de1eb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:43.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.687+0000 7f458b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-02T13:54:43.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.687+0000 7f458b7fe640 1 -- 172.21.15.52:0/1531325636 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c048060 con 0x557c93d168d0 2024-04-02T13:54:43.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.687+0000 7f458b7fe640 1 -- 172.21.15.52:0/1531325636 <== mon.2 v2:172.21.15.88:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f459c01e6f0 con 0x557c93d168d0 2024-04-02T13:54:43.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.687+0000 7f45b4b22740 1 -- 172.21.15.52:0/1531325636 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f459c02b050 con 0x557c93d168d0 2024-04-02T13:54:43.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.687+0000 7f458b7fe640 1 -- 172.21.15.52:0/1531325636 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f459c04f050 con 0x557c93d168d0 2024-04-02T13:54:43.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.687+0000 7f45b4b22740 10 client.25625 fetch_fsmap learned FSMap version 28 2024-04-02T13:54:43.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.687+0000 7f45b4b22740 1 -- 172.21.15.52:0/1531325636 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557c93d9aed0 con 0x557c93d168d0 2024-04-02T13:54:43.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.688+0000 7f458b7fe640 1 -- 172.21.15.52:0/1531325636 <== mon.2 v2:172.21.15.88:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f459c02b050 con 0x557c93d168d0 2024-04-02T13:54:43.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.688+0000 7f458b7fe640 1 -- 172.21.15.52:0/1531325636 <== mon.2 v2:172.21.15.88:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f459c03b470 con 0x557c93d168d0 2024-04-02T13:54:43.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.688+0000 7f45b4b22740 10 client.25625 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:54:43.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.688+0000 7f45b4b22740 10 client.25625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:54:43.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.688+0000 7f45b4b22740 20 client.25625 populate_metadata read hostname 'smithi052' 2024-04-02T13:54:43.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.688+0000 7f45b4b22740 1 --2- 172.21.15.52:0/1531325636 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557c93d9b4e0 0x557c93d9d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:43.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.688+0000 7f45b4b22740 4 client.25625 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:54:43.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.688+0000 7f45b4b22740 1 -- 172.21.15.52:0/1531325636 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557c93c820c0 con 0x557c93d9b4e0 2024-04-02T13:54:43.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.688+0000 7f45ab7fe640 1 --2- 172.21.15.52:0/1531325636 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557c93d9b4e0 0x557c93d9d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:43.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.688+0000 7f458b7fe640 10 client.25625 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:43.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.688+0000 7f45ab7fe640 1 --2- 172.21.15.52:0/1531325636 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557c93d9b4e0 0x557c93d9d8f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d38e0318d2cd145 server_cookie=b208e5ae841d73c0 in_seq=0 out_seq=0 2024-04-02T13:54:43.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.689+0000 7f458b7fe640 1 -- 172.21.15.52:0/1531325636 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557c93c820c0 con 0x557c93d9b4e0 2024-04-02T13:54:43.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.689+0000 7f458b7fe640 10 client.25625 handle_command_reply: tid=0 2024-04-02T13:54:43.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.689+0000 7f45b4b22740 1 client.25625 shutdown 2024-04-02T13:54:43.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.689+0000 7f45b4b22740 1 -- 172.21.15.52:0/1531325636 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d168d0 msgr2=0x557c93d28390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:43.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.689+0000 7f45b4b22740 1 --2- 172.21.15.52:0/1531325636 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d168d0 0x557c93d28390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f459c0264f0 tx=0x7f459c00b480 comp rx=0 tx=0).stop 2024-04-02T13:54:43.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.689+0000 7f45b4b22740 1 -- 172.21.15.52:0/1531325636 shutdown_connections 2024-04-02T13:54:43.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.689+0000 7f45b4b22740 1 --2- 172.21.15.52:0/1531325636 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557c93d9b4e0 0x557c93d9d8f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.689+0000 7f45b4b22740 1 --2- 172.21.15.52:0/1531325636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c93d1fdd0 0x557c93d997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.689+0000 7f45b4b22740 1 --2- 172.21.15.52:0/1531325636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93d171f0 0x557c93d288d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.689+0000 7f45b4b22740 1 --2- 172.21.15.52:0/1531325636 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557c93d168d0 0x557c93d28390 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.689+0000 7f45b4b22740 1 -- 172.21.15.52:0/1531325636 >> 172.21.15.52:0/1531325636 conn(0x557c93d11000 msgr2=0x557c93d10240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:43.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.689+0000 7f458b7fe640 0 client.25625 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:54:43.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.689+0000 7f45b4b22740 1 -- 172.21.15.52:0/1531325636 shutdown_connections 2024-04-02T13:54:43.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.689+0000 7f45b4b22740 1 -- 172.21.15.52:0/1531325636 wait complete. 2024-04-02T13:54:43.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.689+0000 7f45b4b22740 20 client.25625 trim_cache size 0 max 16384 2024-04-02T13:54:43.700 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:54:43.700 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:54:43.700 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-04-02T13:54:43.700 INFO:teuthology.orchestra.run.smithi052.stdout: "aedc2fd7-95b2-4cfa-b105-da6182892f71": { 2024-04-02T13:54:43.700 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-04-02T13:54:43.700 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "aedc2fd7-95b2-4cfa-b105-da6182892f71", 2024-04-02T13:54:43.700 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-04-02T13:54:43.700 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:54:43.700 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:54:43.700 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:54:43.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.690+0000 7f45b2293640 1 -- 172.21.15.52:0/3665924484 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f45940a3e30 msgr2=0x7f45940a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:43.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.690+0000 7f45b2293640 1 --2- 172.21.15.52:0/3665924484 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f45940a3e30 0x7f45940a62f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f459c00bbe0 tx=0x7f459c00b390 comp rx=0 tx=0).stop 2024-04-02T13:54:43.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.690+0000 7f45b2293640 1 -- 172.21.15.52:0/3665924484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45ac107b50 msgr2=0x7f45ac19cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:43.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.690+0000 7f45b2293640 1 --2- 172.21.15.52:0/3665924484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45ac107b50 0x7f45ac19cc20 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f45a400eb10 tx=0x7f45a400c2d0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.690+0000 7f45b2293640 1 -- 172.21.15.52:0/3665924484 shutdown_connections 2024-04-02T13:54:43.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.690+0000 7f45b2293640 1 --2- 172.21.15.52:0/3665924484 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f45940a3e30 0x7f45940a62f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.690+0000 7f45b2293640 1 --2- 172.21.15.52:0/3665924484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45ac107b50 0x7f45ac19cc20 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.690+0000 7f45b2293640 1 --2- 172.21.15.52:0/3665924484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45ac0696c0 0x7f45ac1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.690+0000 7f45b2293640 1 --2- 172.21.15.52:0/3665924484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45ac068d00 0x7f45ac198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:43.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.690+0000 7f45b2293640 1 -- 172.21.15.52:0/3665924484 >> 172.21.15.52:0/3665924484 conn(0x7f45ac077b00 msgr2=0x7f45ac0fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:43.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.690+0000 7f45b2293640 1 -- 172.21.15.52:0/3665924484 shutdown_connections 2024-04-02T13:54:43.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:54:43.690+0000 7f45b2293640 1 -- 172.21.15.52:0/3665924484 wait complete. 2024-04-02T13:54:43.708 INFO:tasks.cephfs.filesystem:scrub status for tag:aedc2fd7-95b2-4cfa-b105-da6182892f71 - {'path': '/', 'tag': 'aedc2fd7-95b2-4cfa-b105-da6182892f71', 'options': 'recursive,force'} 2024-04-02T13:54:43.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:43 smithi189 ceph-mon[36683]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:54:44.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:44 smithi052 ceph-mon[32104]: scrub queued for path: / 2024-04-02T13:54:44.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:44 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:44.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:44 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:54:44.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:44 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:44.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:44 smithi088 ceph-mon[36528]: scrub queued for path: / 2024-04-02T13:54:44.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:44 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:44.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:44 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:54:44.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:44 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:44.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:44 smithi189 ceph-mon[36683]: scrub queued for path: / 2024-04-02T13:54:44.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:44 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:44.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:44 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:54:44.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:44 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:45.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:45 smithi052 ceph-mon[32104]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:54:45.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:45 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T13:54:45.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:45 smithi088 ceph-mon[36528]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:54:45.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:45 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T13:54:45.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:45 smithi189 ceph-mon[36683]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:54:45.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:45 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T13:54:47.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:47 smithi052 ceph-mon[32104]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:54:47.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:47 smithi088 ceph-mon[36528]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:54:47.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:47 smithi189 ceph-mon[36683]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:54:49.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:49 smithi052 ceph-mon[32104]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:54:49.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:49 smithi088 ceph-mon[36528]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:54:49.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:49 smithi189 ceph-mon[36683]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:54:51.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:51 smithi052 ceph-mon[32104]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:54:51.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:51 smithi088 ceph-mon[36528]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:54:51.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:51 smithi189 ceph-mon[36683]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:54:53.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:53 smithi052 ceph-mon[32104]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:54:53.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:53 smithi088 ceph-mon[36528]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:54:53.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:53 smithi189 ceph-mon[36683]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:54:55.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:55 smithi052 ceph-mon[32104]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:54:55.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:55 smithi088 ceph-mon[36528]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:54:55.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:55 smithi189 ceph-mon[36683]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:54:57.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:57 smithi052 ceph-mon[32104]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:54:57.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:57 smithi088 ceph-mon[36528]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:54:57.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:57 smithi189 ceph-mon[36683]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:54:59.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:54:59 smithi052 ceph-mon[32104]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:54:59.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:54:59 smithi088 ceph-mon[36528]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:54:59.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:54:59 smithi189 ceph-mon[36683]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:55:01.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:01 smithi088 ceph-mon[36528]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:55:01.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:01 smithi189 ceph-mon[36683]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:55:01.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:01 smithi052 ceph-mon[32104]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:55:03.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:03 smithi088 ceph-mon[36528]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-02T13:55:03.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:03 smithi088 ceph-mon[36528]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-02T13:55:03.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:03 smithi189 ceph-mon[36683]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-02T13:55:03.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:03 smithi189 ceph-mon[36683]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-02T13:55:03.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:03 smithi052 ceph-mon[32104]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-02T13:55:03.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:03 smithi052 ceph-mon[32104]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-02T13:55:04.946 INFO:tasks.workunit.client.0.smithi052.stderr:Updating files: 95% (11688/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-02T13:55:05.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:05 smithi088 ceph-mon[36528]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-02T13:55:05.779 INFO:tasks.workunit.client.0.smithi052.stderr:Note: switching to '191d963ac9de7322da8aff20b567b3fa458e2ef6'. 2024-04-02T13:55:05.779 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-04-02T13:55:05.780 INFO:tasks.workunit.client.0.smithi052.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-02T13:55:05.780 INFO:tasks.workunit.client.0.smithi052.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-02T13:55:05.780 INFO:tasks.workunit.client.0.smithi052.stderr:state without impacting any branches by switching back to a branch. 2024-04-02T13:55:05.780 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-04-02T13:55:05.780 INFO:tasks.workunit.client.0.smithi052.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-02T13:55:05.780 INFO:tasks.workunit.client.0.smithi052.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-02T13:55:05.780 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-04-02T13:55:05.780 INFO:tasks.workunit.client.0.smithi052.stderr: git switch -c 2024-04-02T13:55:05.780 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-04-02T13:55:05.780 INFO:tasks.workunit.client.0.smithi052.stderr:Or undo this operation with: 2024-04-02T13:55:05.780 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-04-02T13:55:05.780 INFO:tasks.workunit.client.0.smithi052.stderr: git switch - 2024-04-02T13:55:05.780 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-04-02T13:55:05.780 INFO:tasks.workunit.client.0.smithi052.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-02T13:55:05.781 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-04-02T13:55:05.781 INFO:tasks.workunit.client.0.smithi052.stderr:HEAD is now at 191d963ac9d Revert "Merge pull request #55436 from tchaikov/mgr-python-3.12" 2024-04-02T13:55:05.785 DEBUG:teuthology.orchestra.run.smithi052:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-02T13:55:05.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:05 smithi189 ceph-mon[36683]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-02T13:55:05.818 INFO:tasks.workunit.client.0.smithi052.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-02T13:55:05.820 INFO:tasks.workunit.client.0.smithi052.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-02T13:55:05.820 INFO:tasks.workunit.client.0.smithi052.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-02T13:55:05.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:05 smithi052 ceph-mon[32104]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-02T13:55:06.528 INFO:tasks.workunit.client.0.smithi052.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-02T13:55:06.598 INFO:tasks.workunit.client.0.smithi052.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-02T13:55:06.636 INFO:tasks.workunit.client.0.smithi052.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-02T13:55:06.637 INFO:tasks.workunit.client.0.smithi052.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-02T13:55:06.638 INFO:tasks.workunit.client.0.smithi052.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-02T13:55:06.676 INFO:tasks.workunit.client.0.smithi052.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-02T13:55:06.679 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-04-02T13:55:06.680 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-02T13:55:06.735 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-04-02T13:55:06.736 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-04-02T13:55:06.736 DEBUG:teuthology.orchestra.run.smithi052:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=191d963ac9de7322da8aff20b567b3fa458e2ef6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-04-02T13:55:06.795 INFO:tasks.workunit.client.0.smithi052.stderr:+ set -e 2024-04-02T13:55:06.795 INFO:tasks.workunit.client.0.smithi052.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-04-02T13:55:06.797 INFO:tasks.workunit.client.0.smithi052.stderr:Cloning into 'xfstests-dev'... 2024-04-02T13:55:07.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:07 smithi088 ceph-mon[36528]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-02T13:55:07.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:07 smithi189 ceph-mon[36683]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-02T13:55:07.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:07 smithi052 ceph-mon[32104]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-02T13:55:09.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:09 smithi088 ceph-mon[36528]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-04-02T13:55:09.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:09 smithi189 ceph-mon[36683]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-04-02T13:55:09.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:09 smithi052 ceph-mon[32104]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-04-02T13:55:11.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:11 smithi088 ceph-mon[36528]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-04-02T13:55:11.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:11 smithi189 ceph-mon[36683]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-04-02T13:55:11.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:11 smithi052 ceph-mon[32104]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-04-02T13:55:13.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:13 smithi088 ceph-mon[36528]: pgmap v54: 129 pgs: 129 active+clean; 15 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 1.2 MiB/s wr, 34 op/s 2024-04-02T13:55:13.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:55:13.709 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:55:13.737 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:13 smithi052 ceph-mon[32104]: pgmap v54: 129 pgs: 129 active+clean; 15 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 1.2 MiB/s wr, 34 op/s 2024-04-02T13:55:13.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:13 smithi189 ceph-mon[36683]: pgmap v54: 129 pgs: 129 active+clean; 15 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 1.2 MiB/s wr, 34 op/s 2024-04-02T13:55:13.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.823+0000 7fdd4bbc7640 1 -- 172.21.15.52:0/3770717011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd4410e2b0 msgr2=0x7fdd44110700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:13.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.823+0000 7fdd4bbc7640 1 --2- 172.21.15.52:0/3770717011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd4410e2b0 0x7fdd44110700 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fdd34009ca0 tx=0x7fdd3402c180 comp rx=0 tx=0).stop 2024-04-02T13:55:13.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.823+0000 7fdd4bbc7640 1 -- 172.21.15.52:0/3770717011 shutdown_connections 2024-04-02T13:55:13.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.823+0000 7fdd4bbc7640 1 --2- 172.21.15.52:0/3770717011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd4410e2b0 0x7fdd44110700 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.823+0000 7fdd4bbc7640 1 --2- 172.21.15.52:0/3770717011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd44074850 0x7fdd4410dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.823+0000 7fdd4bbc7640 1 --2- 172.21.15.52:0/3770717011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd44073f30 0x7fdd44074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.823+0000 7fdd4bbc7640 1 -- 172.21.15.52:0/3770717011 >> 172.21.15.52:0/3770717011 conn(0x7fdd440fd560 msgr2=0x7fdd440ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:13.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.823+0000 7fdd4bbc7640 1 -- 172.21.15.52:0/3770717011 shutdown_connections 2024-04-02T13:55:13.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.823+0000 7fdd4bbc7640 1 -- 172.21.15.52:0/3770717011 wait complete. 2024-04-02T13:55:13.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.824+0000 7fdd4bbc7640 1 Processor -- start 2024-04-02T13:55:13.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.824+0000 7fdd4bbc7640 1 -- start start 2024-04-02T13:55:13.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.824+0000 7fdd4bbc7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd44073f30 0x7fdd4419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:13.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.824+0000 7fdd4bbc7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd44074850 0x7fdd4419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:13.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.824+0000 7fdd4bbc7640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd4410e2b0 0x7fdd4419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:13.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.824+0000 7fdd4bbc7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdd44114050 con 0x7fdd44073f30 2024-04-02T13:55:13.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.824+0000 7fdd4bbc7640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdd44113ed0 con 0x7fdd4410e2b0 2024-04-02T13:55:13.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.824+0000 7fdd4bbc7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdd441141d0 con 0x7fdd44074850 2024-04-02T13:55:13.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.824+0000 7fdd49963640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd44073f30 0x7fdd4419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:13.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.824+0000 7fdd49963640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd44073f30 0x7fdd4419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47196/0 (socket says 172.21.15.52:47196) 2024-04-02T13:55:13.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.824+0000 7fdd49963640 1 -- 172.21.15.52:0/3728314405 learned_addr learned my addr 172.21.15.52:0/3728314405 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:13.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.824+0000 7fdd49162640 1 --2- 172.21.15.52:0/3728314405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd44074850 0x7fdd4419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:13.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.824+0000 7fdd4a164640 1 --2- 172.21.15.52:0/3728314405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd4410e2b0 0x7fdd4419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.824+0000 7fdd49963640 1 -- 172.21.15.52:0/3728314405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd44074850 msgr2=0x7fdd4419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.824+0000 7fdd49963640 1 --2- 172.21.15.52:0/3728314405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd44074850 0x7fdd4419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.824+0000 7fdd49963640 1 -- 172.21.15.52:0/3728314405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd4410e2b0 msgr2=0x7fdd4419ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.824+0000 7fdd49963640 1 --2- 172.21.15.52:0/3728314405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd4410e2b0 0x7fdd4419ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.824+0000 7fdd49963640 1 -- 172.21.15.52:0/3728314405 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd4419f550 con 0x7fdd44073f30 2024-04-02T13:55:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.825+0000 7fdd49963640 1 --2- 172.21.15.52:0/3728314405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd44073f30 0x7fdd4419a5b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c00d950 tx=0x7fdd3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.825+0000 7fdd42ffd640 1 -- 172.21.15.52:0/3728314405 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c017070 con 0x7fdd44073f30 2024-04-02T13:55:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.825+0000 7fdd42ffd640 1 -- 172.21.15.52:0/3728314405 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd3c009c30 con 0x7fdd44073f30 2024-04-02T13:55:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.825+0000 7fdd4bbc7640 1 -- 172.21.15.52:0/3728314405 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd4419f840 con 0x7fdd44073f30 2024-04-02T13:55:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.825+0000 7fdd42ffd640 1 -- 172.21.15.52:0/3728314405 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c00f6b0 con 0x7fdd44073f30 2024-04-02T13:55:13.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.825+0000 7fdd4bbc7640 1 -- 172.21.15.52:0/3728314405 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd441a70d0 con 0x7fdd44073f30 2024-04-02T13:55:13.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.827+0000 7fdd42ffd640 1 -- 172.21.15.52:0/3728314405 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdd3c0109d0 con 0x7fdd44073f30 2024-04-02T13:55:13.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.827+0000 7fdd4e7a8740 1 Processor -- start 2024-04-02T13:55:13.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.828+0000 7fdd4e7a8740 1 -- start start 2024-04-02T13:55:13.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.828+0000 7fdd42ffd640 1 --2- 172.21.15.52:0/3728314405 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdd2c0a3d20 0x7fdd2c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:13.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.828+0000 7fdd4e7a8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbe1c478d0 0x55cbe1c47cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:13.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.828+0000 7fdd4e7a8740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c481f0 0x55cbe1c50890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:13.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.828+0000 7fdd4e7a8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbe1c50dd0 0x55cbe1c531c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:13.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.828+0000 7fdd4e7a8740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cbe1bab830 con 0x55cbe1c481f0 2024-04-02T13:55:13.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.828+0000 7fdd4e7a8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cbe1b53820 con 0x55cbe1c478d0 2024-04-02T13:55:13.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.828+0000 7fdd4e7a8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cbe1ba77e0 con 0x55cbe1c50dd0 2024-04-02T13:55:13.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.828+0000 7fdd4a164640 1 --2- 172.21.15.52:0/3728314405 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdd2c0a3d20 0x7fdd2c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:13.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.828+0000 7fdd42ffd640 1 -- 172.21.15.52:0/3728314405 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fdd3c0d07d0 con 0x7fdd44073f30 2024-04-02T13:55:13.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.828+0000 7fdd49162640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c481f0 0x55cbe1c50890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:13.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.828+0000 7fdd49963640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbe1c478d0 0x55cbe1c47cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:13.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.828+0000 7fdd49162640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c481f0 0x55cbe1c50890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47200/0 (socket says 172.21.15.52:47200) 2024-04-02T13:55:13.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.828+0000 7fdd49162640 1 -- 172.21.15.52:0/2136575350 learned_addr learned my addr 172.21.15.52:0/2136575350 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:13.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.828+0000 7fdd4a164640 1 --2- 172.21.15.52:0/2136575350 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbe1c50dd0 0x55cbe1c531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:13.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.829+0000 7fdd49162640 1 -- 172.21.15.52:0/2136575350 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbe1c50dd0 msgr2=0x55cbe1c531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:13.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.829+0000 7fdd49162640 1 --2- 172.21.15.52:0/2136575350 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbe1c50dd0 0x55cbe1c531c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.829+0000 7fdd49162640 1 -- 172.21.15.52:0/2136575350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbe1c478d0 msgr2=0x55cbe1c47cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:13.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.829+0000 7fdd49162640 1 --2- 172.21.15.52:0/2136575350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbe1c478d0 0x55cbe1c47cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.829+0000 7fdd49162640 1 -- 172.21.15.52:0/2136575350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbe19897d0 con 0x55cbe1c481f0 2024-04-02T13:55:13.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.829+0000 7fdd49963640 1 --2- 172.21.15.52:0/2136575350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbe1c478d0 0x55cbe1c47cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:13.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.829+0000 7fdd4a164640 1 --2- 172.21.15.52:0/3728314405 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdd2c0a3d20 0x7fdd2c0a61e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdd34005cc0 tx=0x7fdd340325d0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:13.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.829+0000 7fdd49162640 1 --2- 172.21.15.52:0/2136575350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c481f0 0x55cbe1c50890 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fdd38009c20 tx=0x7fdd38011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4ff1948dbc9928 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:13.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.829+0000 7fdd2bfff640 1 -- 172.21.15.52:0/2136575350 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd38004230 con 0x55cbe1c481f0 2024-04-02T13:55:13.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.829+0000 7fdd2bfff640 1 -- 172.21.15.52:0/2136575350 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd380043d0 con 0x55cbe1c481f0 2024-04-02T13:55:13.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.829+0000 7fdd2bfff640 1 -- 172.21.15.52:0/2136575350 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd38004d70 con 0x55cbe1c481f0 2024-04-02T13:55:13.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.830+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/2136575350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c481f0 msgr2=0x55cbe1c50890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:13.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.830+0000 7fdd4e7a8740 1 --2- 172.21.15.52:0/2136575350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c481f0 0x55cbe1c50890 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fdd38009c20 tx=0x7fdd38011810 comp rx=0 tx=0).stop 2024-04-02T13:55:13.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.830+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/2136575350 shutdown_connections 2024-04-02T13:55:13.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.830+0000 7fdd4e7a8740 1 --2- 172.21.15.52:0/2136575350 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbe1c50dd0 0x55cbe1c531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.830+0000 7fdd4e7a8740 1 --2- 172.21.15.52:0/2136575350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c481f0 0x55cbe1c50890 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.830+0000 7fdd4e7a8740 1 --2- 172.21.15.52:0/2136575350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbe1c478d0 0x55cbe1c47cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.830+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/2136575350 >> 172.21.15.52:0/2136575350 conn(0x55cbe1c42000 msgr2=0x55cbe1c423e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:13.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.830+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/2136575350 shutdown_connections 2024-04-02T13:55:13.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.830+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/2136575350 wait complete. 2024-04-02T13:55:13.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.830+0000 7fdd4e7a8740 1 -- start start 2024-04-02T13:55:13.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.830+0000 7fdd4e7a8740 1 Processor -- start 2024-04-02T13:55:13.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.831+0000 7fdd4e7a8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbe1c478d0 0x55cbe1c50680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:13.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.831+0000 7fdd4e7a8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbe1c481f0 0x55cbe1c4da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:13.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.831+0000 7fdd4e7a8740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c50dd0 0x55cbe1c4df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:13.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.831+0000 7fdd4e7a8740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cbe1c53bc0 con 0x55cbe1c50dd0 2024-04-02T13:55:13.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.831+0000 7fdd4e7a8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cbe1b66d30 con 0x55cbe1c478d0 2024-04-02T13:55:13.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.831+0000 7fdd4e7a8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cbe1c53d40 con 0x55cbe1c481f0 2024-04-02T13:55:13.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.831+0000 7fdd4e7a8740 10 client.0 _ll_register_callbacks cb 0x55cbe1b66720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:13.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.831+0000 7fdd4a164640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c50dd0 0x55cbe1c4df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:13.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.831+0000 7fdd4a164640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c50dd0 0x55cbe1c4df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47208/0 (socket says 172.21.15.52:47208) 2024-04-02T13:55:13.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.831+0000 7fdd4a164640 1 -- 172.21.15.52:0/2522568872 learned_addr learned my addr 172.21.15.52:0/2522568872 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:13.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.831+0000 7fdd49963640 1 --2- 172.21.15.52:0/2522568872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbe1c478d0 0x55cbe1c50680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:13.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.831+0000 7fdd49162640 1 --2- 172.21.15.52:0/2522568872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbe1c481f0 0x55cbe1c4da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:13.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.831+0000 7fdd4a164640 1 -- 172.21.15.52:0/2522568872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbe1c481f0 msgr2=0x55cbe1c4da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:13.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.831+0000 7fdd4a164640 1 --2- 172.21.15.52:0/2522568872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbe1c481f0 0x55cbe1c4da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.831+0000 7fdd4a164640 1 -- 172.21.15.52:0/2522568872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbe1c478d0 msgr2=0x55cbe1c50680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:13.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.831+0000 7fdd4a164640 1 --2- 172.21.15.52:0/2522568872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbe1c478d0 0x55cbe1c50680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.831+0000 7fdd4a164640 1 -- 172.21.15.52:0/2522568872 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbe18bf3f0 con 0x55cbe1c50dd0 2024-04-02T13:55:13.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.831+0000 7fdd49162640 1 --2- 172.21.15.52:0/2522568872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbe1c481f0 0x55cbe1c4da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:13.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.831+0000 7fdd49963640 1 --2- 172.21.15.52:0/2522568872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbe1c478d0 0x55cbe1c50680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:13.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.831+0000 7fdd4a164640 1 --2- 172.21.15.52:0/2522568872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c50dd0 0x55cbe1c4df50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fdd34036000 tx=0x7fdd3402c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22dd5f95dbe7c8c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:13.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.831+0000 7fdd297fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:13.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.832+0000 7fdd297fa640 1 -- 172.21.15.52:0/2522568872 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd34039040 con 0x55cbe1c50dd0 2024-04-02T13:55:13.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.832+0000 7fdd297fa640 1 -- 172.21.15.52:0/2522568872 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fdd3404e660 con 0x55cbe1c50dd0 2024-04-02T13:55:13.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.832+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/2522568872 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd380043d0 con 0x55cbe1c50dd0 2024-04-02T13:55:13.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.832+0000 7fdd297fa640 1 -- 172.21.15.52:0/2522568872 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdd3405c050 con 0x55cbe1c50dd0 2024-04-02T13:55:13.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.832+0000 7fdd4e7a8740 10 client.16104 fetch_fsmap learned FSMap version 28 2024-04-02T13:55:13.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.832+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/2522568872 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cbe187f0f0 con 0x55cbe1c50dd0 2024-04-02T13:55:13.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.832+0000 7fdd297fa640 1 -- 172.21.15.52:0/2522568872 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd380043d0 con 0x55cbe1c50dd0 2024-04-02T13:55:13.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.832+0000 7fdd297fa640 1 -- 172.21.15.52:0/2522568872 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdd340347e0 con 0x55cbe1c50dd0 2024-04-02T13:55:13.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.832+0000 7fdd4e7a8740 10 client.16104 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:55:13.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.832+0000 7fdd4e7a8740 10 client.16104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:13.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.832+0000 7fdd4e7a8740 20 client.16104 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:13.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.832+0000 7fdd4e7a8740 1 --2- 172.21.15.52:0/2522568872 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cbe1c60db0 0x55cbe1c63200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:13.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.832+0000 7fdd4e7a8740 4 client.16104 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:13.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.832+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/2522568872 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbe1bb8d70 con 0x55cbe1c60db0 2024-04-02T13:55:13.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.833+0000 7fdd49963640 1 --2- 172.21.15.52:0/2522568872 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cbe1c60db0 0x55cbe1c63200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:13.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.833+0000 7fdd297fa640 10 client.16104 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:13.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.833+0000 7fdd49963640 1 --2- 172.21.15.52:0/2522568872 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cbe1c60db0 0x55cbe1c63200 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f109c924aec59b6e server_cookie=8cfe47b2e033aee9 in_seq=0 out_seq=0 2024-04-02T13:55:13.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.834+0000 7fdd297fa640 1 -- 172.21.15.52:0/2522568872 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cbe1bb8d70 con 0x55cbe1c60db0 2024-04-02T13:55:13.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.834+0000 7fdd297fa640 10 client.16104 handle_command_reply: tid=0 2024-04-02T13:55:13.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.834+0000 7fdd4e7a8740 1 client.16104 shutdown 2024-04-02T13:55:13.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.834+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/2522568872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c50dd0 msgr2=0x55cbe1c4df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:13.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.834+0000 7fdd4e7a8740 1 --2- 172.21.15.52:0/2522568872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c50dd0 0x55cbe1c4df50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fdd34036000 tx=0x7fdd3402c180 comp rx=0 tx=0).stop 2024-04-02T13:55:13.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.834+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/2522568872 shutdown_connections 2024-04-02T13:55:13.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.834+0000 7fdd4e7a8740 1 --2- 172.21.15.52:0/2522568872 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cbe1c60db0 0x55cbe1c63200 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.834+0000 7fdd4e7a8740 1 --2- 172.21.15.52:0/2522568872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c50dd0 0x55cbe1c4df50 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.834+0000 7fdd4e7a8740 1 --2- 172.21.15.52:0/2522568872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbe1c481f0 0x55cbe1c4da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.834+0000 7fdd297fa640 0 client.16104 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:13.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.834+0000 7fdd4e7a8740 1 --2- 172.21.15.52:0/2522568872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbe1c478d0 0x55cbe1c50680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.834+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/2522568872 >> 172.21.15.52:0/2522568872 conn(0x55cbe1c42000 msgr2=0x55cbe1c43d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:13.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.834+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/2522568872 shutdown_connections 2024-04-02T13:55:13.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.834+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/2522568872 wait complete. 2024-04-02T13:55:13.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.834+0000 7fdd4e7a8740 20 client.16104 trim_cache size 0 max 16384 2024-04-02T13:55:13.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.852+0000 7fdd4e7a8740 1 Processor -- start 2024-04-02T13:55:13.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.852+0000 7fdd4e7a8740 1 -- start start 2024-04-02T13:55:13.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.852+0000 7fdd4e7a8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbe1c478d0 0x55cbe1c41240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:13.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.852+0000 7fdd4e7a8740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c481f0 0x55cbe1c48e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:13.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.852+0000 7fdd4e7a8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbe1c50dd0 0x55cbe1c50140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:13.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.852+0000 7fdd4e7a8740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdd441b1080 con 0x55cbe1c481f0 2024-04-02T13:55:13.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.852+0000 7fdd4e7a8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdd441b2bc0 con 0x55cbe1c478d0 2024-04-02T13:55:13.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.852+0000 7fdd4e7a8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdd441b1750 con 0x55cbe1c50dd0 2024-04-02T13:55:13.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.852+0000 7fdd49162640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c481f0 0x55cbe1c48e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:13.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.852+0000 7fdd49162640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c481f0 0x55cbe1c48e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47218/0 (socket says 172.21.15.52:47218) 2024-04-02T13:55:13.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.852+0000 7fdd49162640 1 -- 172.21.15.52:0/2093931824 learned_addr learned my addr 172.21.15.52:0/2093931824 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:13.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.852+0000 7fdd49162640 1 -- 172.21.15.52:0/2093931824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbe1c50dd0 msgr2=0x55cbe1c50140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:13.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.852+0000 7fdd49162640 1 --2- 172.21.15.52:0/2093931824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbe1c50dd0 0x55cbe1c50140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.852+0000 7fdd49963640 1 --2- 172.21.15.52:0/2093931824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbe1c478d0 0x55cbe1c41240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:13.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.852+0000 7fdd49162640 1 -- 172.21.15.52:0/2093931824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbe1c478d0 msgr2=0x55cbe1c41240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:13.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.852+0000 7fdd49162640 1 --2- 172.21.15.52:0/2093931824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbe1c478d0 0x55cbe1c41240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.852+0000 7fdd49162640 1 -- 172.21.15.52:0/2093931824 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbe1c3dc50 con 0x55cbe1c481f0 2024-04-02T13:55:13.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.852+0000 7fdd49963640 1 --2- 172.21.15.52:0/2093931824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbe1c478d0 0x55cbe1c41240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:13.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.852+0000 7fdd49162640 1 --2- 172.21.15.52:0/2093931824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c481f0 0x55cbe1c48e20 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x55cbe1c473a0 tx=0x7fdd38009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b96b500b85b9a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:13.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.853+0000 7fdd2b7fe640 1 -- 172.21.15.52:0/2093931824 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3801e040 con 0x55cbe1c481f0 2024-04-02T13:55:13.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.853+0000 7fdd2b7fe640 1 -- 172.21.15.52:0/2093931824 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd380035f0 con 0x55cbe1c481f0 2024-04-02T13:55:13.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.853+0000 7fdd2b7fe640 1 -- 172.21.15.52:0/2093931824 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd38022690 con 0x55cbe1c481f0 2024-04-02T13:55:13.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.853+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/2093931824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c481f0 msgr2=0x55cbe1c48e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:13.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.853+0000 7fdd4e7a8740 1 --2- 172.21.15.52:0/2093931824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c481f0 0x55cbe1c48e20 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x55cbe1c473a0 tx=0x7fdd38009f10 comp rx=0 tx=0).stop 2024-04-02T13:55:13.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.853+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/2093931824 shutdown_connections 2024-04-02T13:55:13.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.853+0000 7fdd4e7a8740 1 --2- 172.21.15.52:0/2093931824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbe1c50dd0 0x55cbe1c50140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.853+0000 7fdd4e7a8740 1 --2- 172.21.15.52:0/2093931824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c481f0 0x55cbe1c48e20 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.853+0000 7fdd4e7a8740 1 --2- 172.21.15.52:0/2093931824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbe1c478d0 0x55cbe1c41240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.853+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/2093931824 >> 172.21.15.52:0/2093931824 conn(0x55cbe1c42000 msgr2=0x55cbe1c48710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:13.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.853+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/2093931824 shutdown_connections 2024-04-02T13:55:13.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.853+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/2093931824 wait complete. 2024-04-02T13:55:13.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.854+0000 7fdd4e7a8740 1 -- start start 2024-04-02T13:55:13.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.854+0000 7fdd4e7a8740 1 Processor -- start 2024-04-02T13:55:13.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.854+0000 7fdd4e7a8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbe1c478d0 0x55cbe1c59420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:13.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.854+0000 7fdd4e7a8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbe1c481f0 0x55cbe1c4a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:13.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.854+0000 7fdd4e7a8740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c50dd0 0x55cbe1c4ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:13.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.854+0000 7fdd4e7a8740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdd441b24f0 con 0x55cbe1c50dd0 2024-04-02T13:55:13.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.854+0000 7fdd4e7a8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdd441b1e20 con 0x55cbe1c481f0 2024-04-02T13:55:13.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.854+0000 7fdd4e7a8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdd441af060 con 0x55cbe1c478d0 2024-04-02T13:55:13.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.854+0000 7fdd4e7a8740 10 client.0 _ll_register_callbacks cb 0x7fdd38005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:13.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.854+0000 7fdd4a164640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c50dd0 0x55cbe1c4ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:13.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.854+0000 7fdd4a164640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c50dd0 0x55cbe1c4ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47234/0 (socket says 172.21.15.52:47234) 2024-04-02T13:55:13.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.854+0000 7fdd4a164640 1 -- 172.21.15.52:0/560788818 learned_addr learned my addr 172.21.15.52:0/560788818 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:13.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.854+0000 7fdd49162640 1 --2- 172.21.15.52:0/560788818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbe1c481f0 0x55cbe1c4a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:13.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.854+0000 7fdd4a164640 1 -- 172.21.15.52:0/560788818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbe1c478d0 msgr2=0x55cbe1c59420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:13.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.854+0000 7fdd4a164640 1 --2- 172.21.15.52:0/560788818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbe1c478d0 0x55cbe1c59420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.854+0000 7fdd4a164640 1 -- 172.21.15.52:0/560788818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbe1c481f0 msgr2=0x55cbe1c4a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:13.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.854+0000 7fdd4a164640 1 --2- 172.21.15.52:0/560788818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbe1c481f0 0x55cbe1c4a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.855+0000 7fdd4a164640 1 -- 172.21.15.52:0/560788818 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbe1c4b330 con 0x55cbe1c50dd0 2024-04-02T13:55:13.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.855+0000 7fdd4a164640 1 --2- 172.21.15.52:0/560788818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c50dd0 0x55cbe1c4ab00 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fdd3402c790 tx=0x7fdd34009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8221734abc863a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:13.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.855+0000 7fdd297fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:13.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.855+0000 7fdd297fa640 1 -- 172.21.15.52:0/560788818 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd34064070 con 0x55cbe1c50dd0 2024-04-02T13:55:13.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.855+0000 7fdd297fa640 1 -- 172.21.15.52:0/560788818 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fdd34034850 con 0x55cbe1c50dd0 2024-04-02T13:55:13.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.855+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/560788818 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd380035f0 con 0x55cbe1c50dd0 2024-04-02T13:55:13.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.855+0000 7fdd297fa640 1 -- 172.21.15.52:0/560788818 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdd34032aa0 con 0x55cbe1c50dd0 2024-04-02T13:55:13.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.855+0000 7fdd4e7a8740 10 client.16110 fetch_fsmap learned FSMap version 28 2024-04-02T13:55:13.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.855+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/560788818 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cbe1cc73c0 con 0x55cbe1c50dd0 2024-04-02T13:55:13.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.855+0000 7fdd297fa640 1 -- 172.21.15.52:0/560788818 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd380035f0 con 0x55cbe1c50dd0 2024-04-02T13:55:13.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.855+0000 7fdd297fa640 1 -- 172.21.15.52:0/560788818 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdd3404e460 con 0x55cbe1c50dd0 2024-04-02T13:55:13.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.855+0000 7fdd4e7a8740 10 client.16110 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:55:13.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.855+0000 7fdd4e7a8740 10 client.16110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:13.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.855+0000 7fdd4e7a8740 20 client.16110 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:13.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.855+0000 7fdd4e7a8740 1 --2- 172.21.15.52:0/560788818 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cbe1cc7a80 0x55cbe1cc9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:13.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.855+0000 7fdd4e7a8740 4 client.16110 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:55:13.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.855+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/560788818 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cbe1bb30c0 con 0x55cbe1cc7a80 2024-04-02T13:55:13.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.856+0000 7fdd49963640 1 --2- 172.21.15.52:0/560788818 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cbe1cc7a80 0x55cbe1cc9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:13.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.856+0000 7fdd49963640 1 --2- 172.21.15.52:0/560788818 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cbe1cc7a80 0x55cbe1cc9e70 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=279af4f53e24d9be server_cookie=39b7ef69ceaf3125 in_seq=0 out_seq=0 2024-04-02T13:55:13.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.856+0000 7fdd297fa640 10 client.16110 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:13.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.856+0000 7fdd297fa640 1 -- 172.21.15.52:0/560788818 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cbe1bb30c0 con 0x55cbe1cc7a80 2024-04-02T13:55:13.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.856+0000 7fdd297fa640 10 client.16110 handle_command_reply: tid=0 2024-04-02T13:55:13.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.856+0000 7fdd4e7a8740 1 client.16110 shutdown 2024-04-02T13:55:13.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.856+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/560788818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c50dd0 msgr2=0x55cbe1c4ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:13.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.856+0000 7fdd4e7a8740 1 --2- 172.21.15.52:0/560788818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c50dd0 0x55cbe1c4ab00 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fdd3402c790 tx=0x7fdd34009360 comp rx=0 tx=0).stop 2024-04-02T13:55:13.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.857+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/560788818 shutdown_connections 2024-04-02T13:55:13.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.857+0000 7fdd4e7a8740 1 --2- 172.21.15.52:0/560788818 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cbe1cc7a80 0x55cbe1cc9e70 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.857+0000 7fdd4e7a8740 1 --2- 172.21.15.52:0/560788818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbe1c50dd0 0x55cbe1c4ab00 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.857+0000 7fdd4e7a8740 1 --2- 172.21.15.52:0/560788818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbe1c481f0 0x55cbe1c4a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.857+0000 7fdd297fa640 0 client.16110 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:13.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.857+0000 7fdd4e7a8740 1 --2- 172.21.15.52:0/560788818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbe1c478d0 0x55cbe1c59420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.857+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/560788818 >> 172.21.15.52:0/560788818 conn(0x55cbe1c42000 msgr2=0x55cbe1c48fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:13.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.857+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/560788818 shutdown_connections 2024-04-02T13:55:13.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.857+0000 7fdd4e7a8740 1 -- 172.21.15.52:0/560788818 wait complete. 2024-04-02T13:55:13.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.857+0000 7fdd4e7a8740 20 client.16110 trim_cache size 0 max 16384 2024-04-02T13:55:13.866 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:55:13.867 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-04-02T13:55:13.867 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-04-02T13:55:13.867 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:55:13.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.857+0000 7fdd4bbc7640 1 -- 172.21.15.52:0/3728314405 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdd2c0a3d20 msgr2=0x7fdd2c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:13.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.857+0000 7fdd4bbc7640 1 --2- 172.21.15.52:0/3728314405 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdd2c0a3d20 0x7fdd2c0a61e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdd34005cc0 tx=0x7fdd340325d0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.857+0000 7fdd4bbc7640 1 -- 172.21.15.52:0/3728314405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd44073f30 msgr2=0x7fdd4419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:13.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.857+0000 7fdd4bbc7640 1 --2- 172.21.15.52:0/3728314405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd44073f30 0x7fdd4419a5b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c00d950 tx=0x7fdd3c00de10 comp rx=0 tx=0).stop 2024-04-02T13:55:13.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.857+0000 7fdd4bbc7640 1 -- 172.21.15.52:0/3728314405 shutdown_connections 2024-04-02T13:55:13.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.857+0000 7fdd4bbc7640 1 --2- 172.21.15.52:0/3728314405 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdd2c0a3d20 0x7fdd2c0a61e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.857+0000 7fdd4bbc7640 1 --2- 172.21.15.52:0/3728314405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd4410e2b0 0x7fdd4419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.857+0000 7fdd4bbc7640 1 --2- 172.21.15.52:0/3728314405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd44074850 0x7fdd4419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.857+0000 7fdd4bbc7640 1 --2- 172.21.15.52:0/3728314405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd44073f30 0x7fdd4419a5b0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:13.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.857+0000 7fdd4bbc7640 1 -- 172.21.15.52:0/3728314405 >> 172.21.15.52:0/3728314405 conn(0x7fdd440fd560 msgr2=0x7fdd440fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:13.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.857+0000 7fdd4bbc7640 1 -- 172.21.15.52:0/3728314405 shutdown_connections 2024-04-02T13:55:13.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:13.857+0000 7fdd4bbc7640 1 -- 172.21.15.52:0/3728314405 wait complete. 2024-04-02T13:55:13.872 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:55:13.872 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:55:14.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.013+0000 7f7db0ba1640 1 -- 172.21.15.52:0/2397205052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dac100c70 msgr2=0x7f7dac10e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.013+0000 7f7db0ba1640 1 --2- 172.21.15.52:0/2397205052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dac100c70 0x7f7dac10e8c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7da4009ca0 tx=0x7f7da402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.013+0000 7f7db0ba1640 1 -- 172.21.15.52:0/2397205052 shutdown_connections 2024-04-02T13:55:14.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.013+0000 7f7db0ba1640 1 --2- 172.21.15.52:0/2397205052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dac100c70 0x7f7dac10e8c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.013+0000 7f7db0ba1640 1 --2- 172.21.15.52:0/2397205052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7dac1002b0 0x7f7dac100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.013+0000 7f7db0ba1640 1 --2- 172.21.15.52:0/2397205052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dac1038b0 0x7f7dac103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.013+0000 7f7db0ba1640 1 -- 172.21.15.52:0/2397205052 >> 172.21.15.52:0/2397205052 conn(0x7f7dac075a50 msgr2=0x7f7dac075e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.013+0000 7f7db0ba1640 1 -- 172.21.15.52:0/2397205052 shutdown_connections 2024-04-02T13:55:14.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.013+0000 7f7db0ba1640 1 -- 172.21.15.52:0/2397205052 wait complete. 2024-04-02T13:55:14.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.014+0000 7f7db0ba1640 1 Processor -- start 2024-04-02T13:55:14.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.014+0000 7f7db0ba1640 1 -- start start 2024-04-02T13:55:14.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.014+0000 7f7db0ba1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dac1002b0 0x7f7dac1981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.014+0000 7f7db0ba1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dac100c70 0x7f7dac198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.014+0000 7f7db0ba1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7dac1038b0 0x7f7dac19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.014+0000 7f7db0ba1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac111de0 con 0x7f7dac1002b0 2024-04-02T13:55:14.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.014+0000 7f7db0ba1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac111c60 con 0x7f7dac100c70 2024-04-02T13:55:14.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.014+0000 7f7db0ba1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac111f60 con 0x7f7dac1038b0 2024-04-02T13:55:14.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.014+0000 7f7dab7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dac1002b0 0x7f7dac1981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.014+0000 7f7dab7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dac1002b0 0x7f7dac1981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47248/0 (socket says 172.21.15.52:47248) 2024-04-02T13:55:14.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.014+0000 7f7dab7fe640 1 -- 172.21.15.52:0/2370771187 learned_addr learned my addr 172.21.15.52:0/2370771187 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.014+0000 7f7daaffd640 1 --2- 172.21.15.52:0/2370771187 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dac100c70 0x7f7dac198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.014+0000 7f7dabfff640 1 --2- 172.21.15.52:0/2370771187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7dac1038b0 0x7f7dac19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.014+0000 7f7dab7fe640 1 -- 172.21.15.52:0/2370771187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7dac1038b0 msgr2=0x7f7dac19caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.014+0000 7f7dab7fe640 1 --2- 172.21.15.52:0/2370771187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7dac1038b0 0x7f7dac19caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.014+0000 7f7dab7fe640 1 -- 172.21.15.52:0/2370771187 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dac100c70 msgr2=0x7f7dac198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.014+0000 7f7dab7fe640 1 --2- 172.21.15.52:0/2370771187 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dac100c70 0x7f7dac198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.014+0000 7f7dab7fe640 1 -- 172.21.15.52:0/2370771187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dac19d1d0 con 0x7f7dac1002b0 2024-04-02T13:55:14.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.015+0000 7f7dab7fe640 1 --2- 172.21.15.52:0/2370771187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dac1002b0 0x7f7dac1981e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7da000e9f0 tx=0x7f7da000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.015+0000 7f7da8ff9640 1 -- 172.21.15.52:0/2370771187 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da000cd80 con 0x7f7dac1002b0 2024-04-02T13:55:14.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.015+0000 7f7da8ff9640 1 -- 172.21.15.52:0/2370771187 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7da0010910 con 0x7f7dac1002b0 2024-04-02T13:55:14.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.015+0000 7f7db0ba1640 1 -- 172.21.15.52:0/2370771187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dac19d4c0 con 0x7f7dac1002b0 2024-04-02T13:55:14.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.015+0000 7f7da8ff9640 1 -- 172.21.15.52:0/2370771187 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da0018990 con 0x7f7dac1002b0 2024-04-02T13:55:14.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.015+0000 7f7db0ba1640 1 -- 172.21.15.52:0/2370771187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dac1a4d50 con 0x7f7dac1002b0 2024-04-02T13:55:14.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.016+0000 7f7db0ba1640 1 -- 172.21.15.52:0/2370771187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dac068c60 con 0x7f7dac1002b0 2024-04-02T13:55:14.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.017+0000 7f7da8ff9640 1 -- 172.21.15.52:0/2370771187 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7da0010ac0 con 0x7f7dac1002b0 2024-04-02T13:55:14.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.017+0000 7f7da8ff9640 1 --2- 172.21.15.52:0/2370771187 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f7d940a3da0 0x7f7d940a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.017+0000 7f7daaffd640 1 --2- 172.21.15.52:0/2370771187 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f7d940a3da0 0x7f7d940a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.017+0000 7f7da8ff9640 1 -- 172.21.15.52:0/2370771187 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f7da00d0e70 con 0x7f7dac1002b0 2024-04-02T13:55:14.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.018+0000 7f7daaffd640 1 --2- 172.21.15.52:0/2370771187 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f7d940a3da0 0x7f7d940a6260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7dac1082b0 tx=0x7f7d9c00b390 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.022+0000 7f7da8ff9640 1 -- 172.21.15.52:0/2370771187 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7da009d1d0 con 0x7f7dac1002b0 2024-04-02T13:55:14.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.249+0000 7f7db0ba1640 1 -- 172.21.15.52:0/2370771187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7dac109f10 con 0x7f7dac1002b0 2024-04-02T13:55:14.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.249+0000 7f7da8ff9640 1 -- 172.21.15.52:0/2370771187 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11923 (secure 0 0 0) 0x7f7da009cad0 con 0x7f7dac1002b0 2024-04-02T13:55:14.253 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:55:14.253 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:53:28.475061+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24457,24467,24490,24503,24475]},"id":1}]} 2024-04-02T13:55:14.253 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 28 2024-04-02T13:55:14.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.252+0000 7f7db0ba1640 1 -- 172.21.15.52:0/2370771187 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f7d940a3da0 msgr2=0x7f7d940a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.252+0000 7f7db0ba1640 1 --2- 172.21.15.52:0/2370771187 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f7d940a3da0 0x7f7d940a6260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7dac1082b0 tx=0x7f7d9c00b390 comp rx=0 tx=0).stop 2024-04-02T13:55:14.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.252+0000 7f7db0ba1640 1 -- 172.21.15.52:0/2370771187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dac1002b0 msgr2=0x7f7dac1981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.252+0000 7f7db0ba1640 1 --2- 172.21.15.52:0/2370771187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dac1002b0 0x7f7dac1981e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7da000e9f0 tx=0x7f7da000eeb0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.252+0000 7f7db0ba1640 1 -- 172.21.15.52:0/2370771187 shutdown_connections 2024-04-02T13:55:14.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.252+0000 7f7db0ba1640 1 --2- 172.21.15.52:0/2370771187 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f7d940a3da0 0x7f7d940a6260 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.252+0000 7f7db0ba1640 1 --2- 172.21.15.52:0/2370771187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7dac1038b0 0x7f7dac19caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.252+0000 7f7db0ba1640 1 --2- 172.21.15.52:0/2370771187 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dac100c70 0x7f7dac198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.252+0000 7f7db0ba1640 1 --2- 172.21.15.52:0/2370771187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dac1002b0 0x7f7dac1981e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.252+0000 7f7db0ba1640 1 -- 172.21.15.52:0/2370771187 >> 172.21.15.52:0/2370771187 conn(0x7f7dac075a50 msgr2=0x7f7dac0feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.252+0000 7f7db0ba1640 1 -- 172.21.15.52:0/2370771187 shutdown_connections 2024-04-02T13:55:14.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.252+0000 7f7db0ba1640 1 -- 172.21.15.52:0/2370771187 wait complete. 2024-04-02T13:55:14.266 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:55:14.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.368+0000 7fc270b00640 1 -- 172.21.15.52:0/2408649943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc26c101a90 msgr2=0x7fc26c101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.368+0000 7fc270b00640 1 --2- 172.21.15.52:0/2408649943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc26c101a90 0x7fc26c101f10 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc254009930 tx=0x7fc25402bc30 comp rx=0 tx=0).stop 2024-04-02T13:55:14.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.368+0000 7fc270b00640 1 -- 172.21.15.52:0/2408649943 shutdown_connections 2024-04-02T13:55:14.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.368+0000 7fc270b00640 1 --2- 172.21.15.52:0/2408649943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc26c102450 0x7fc26c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.368+0000 7fc270b00640 1 --2- 172.21.15.52:0/2408649943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc26c101a90 0x7fc26c101f10 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.368+0000 7fc270b00640 1 --2- 172.21.15.52:0/2408649943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc26c107aa0 0x7fc26c107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.368+0000 7fc270b00640 1 -- 172.21.15.52:0/2408649943 >> 172.21.15.52:0/2408649943 conn(0x7fc26c078f00 msgr2=0x7fc26c079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.368+0000 7fc270b00640 1 -- 172.21.15.52:0/2408649943 shutdown_connections 2024-04-02T13:55:14.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.368+0000 7fc270b00640 1 -- 172.21.15.52:0/2408649943 wait complete. 2024-04-02T13:55:14.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.369+0000 7fc270b00640 1 Processor -- start 2024-04-02T13:55:14.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.369+0000 7fc270b00640 1 -- start start 2024-04-02T13:55:14.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.369+0000 7fc270b00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc26c101a90 0x7fc26c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.369+0000 7fc270b00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc26c102450 0x7fc26c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.369+0000 7fc270b00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc26c107aa0 0x7fc26c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.369+0000 7fc270b00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c111eb0 con 0x7fc26c102450 2024-04-02T13:55:14.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.369+0000 7fc270b00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c111d30 con 0x7fc26c107aa0 2024-04-02T13:55:14.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.369+0000 7fc270b00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c112030 con 0x7fc26c101a90 2024-04-02T13:55:14.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.369+0000 7fc261d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc26c102450 0x7fc26c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.369+0000 7fc261d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc26c102450 0x7fc26c1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47262/0 (socket says 172.21.15.52:47262) 2024-04-02T13:55:14.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.369+0000 7fc261d9b640 1 -- 172.21.15.52:0/2032219968 learned_addr learned my addr 172.21.15.52:0/2032219968 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.369+0000 7fc26ad9d640 1 --2- 172.21.15.52:0/2032219968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc26c107aa0 0x7fc26c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.369+0000 7fc261d9b640 1 -- 172.21.15.52:0/2032219968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc26c101a90 msgr2=0x7fc26c198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.369+0000 7fc261d9b640 1 --2- 172.21.15.52:0/2032219968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc26c101a90 0x7fc26c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.370+0000 7fc261d9b640 1 -- 172.21.15.52:0/2032219968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc26c107aa0 msgr2=0x7fc26c19cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.370+0000 7fc261d9b640 1 --2- 172.21.15.52:0/2032219968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc26c107aa0 0x7fc26c19cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.370+0000 7fc261d9b640 1 -- 172.21.15.52:0/2032219968 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc26c19d360 con 0x7fc26c102450 2024-04-02T13:55:14.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.370+0000 7fc26ad9d640 1 --2- 172.21.15.52:0/2032219968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc26c107aa0 0x7fc26c19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:14.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.370+0000 7fc261d9b640 1 --2- 172.21.15.52:0/2032219968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc26c102450 0x7fc26c1988b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc254009230 tx=0x7fc25402cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.370+0000 7fc263fff640 1 -- 172.21.15.52:0/2032219968 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254041070 con 0x7fc26c102450 2024-04-02T13:55:14.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.370+0000 7fc270b00640 1 -- 172.21.15.52:0/2032219968 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc26c19d5f0 con 0x7fc26c102450 2024-04-02T13:55:14.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.370+0000 7fc263fff640 1 -- 172.21.15.52:0/2032219968 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc254004450 con 0x7fc26c102450 2024-04-02T13:55:14.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.370+0000 7fc263fff640 1 -- 172.21.15.52:0/2032219968 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254034440 con 0x7fc26c102450 2024-04-02T13:55:14.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.370+0000 7fc270b00640 1 -- 172.21.15.52:0/2032219968 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc26c1a4ee0 con 0x7fc26c102450 2024-04-02T13:55:14.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.372+0000 7fc263fff640 1 -- 172.21.15.52:0/2032219968 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc254044070 con 0x7fc26c102450 2024-04-02T13:55:14.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.372+0000 7fc2737bf740 1 Processor -- start 2024-04-02T13:55:14.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.372+0000 7fc2737bf740 1 -- start start 2024-04-02T13:55:14.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.373+0000 7fc263fff640 1 --2- 172.21.15.52:0/2032219968 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc2580a3d20 0x7fc2580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.373+0000 7fc2737bf740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56125c105450 0x56125c105830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.373+0000 7fc2737bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56125c105d70 0x56125c112b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.373+0000 7fc2737bf740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c1130b0 0x56125c1154a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.373+0000 7fc2737bf740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56125c0697b0 con 0x56125c1130b0 2024-04-02T13:55:14.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.373+0000 7fc2737bf740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56125c008570 con 0x56125c105450 2024-04-02T13:55:14.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.373+0000 7fc2737bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56125c065760 con 0x56125c105d70 2024-04-02T13:55:14.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.373+0000 7fc263fff640 1 -- 172.21.15.52:0/2032219968 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc2540f2530 con 0x7fc26c102450 2024-04-02T13:55:14.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.373+0000 7fc26ad9d640 1 --2- 172.21.15.52:0/2032219968 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc2580a3d20 0x7fc2580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.373+0000 7fc26ad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c1130b0 0x56125c1154a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.373+0000 7fc261d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56125c105d70 0x56125c112b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.373+0000 7fc261d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56125c105d70 0x56125c112b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.52:59918/0 (socket says 172.21.15.52:59918) 2024-04-02T13:55:14.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.373+0000 7fc26ad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c1130b0 0x56125c1154a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47272/0 (socket says 172.21.15.52:47272) 2024-04-02T13:55:14.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.373+0000 7fc261d9b640 1 -- 172.21.15.52:0/648911158 learned_addr learned my addr 172.21.15.52:0/648911158 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.373+0000 7fc26a59c640 1 --2- 172.21.15.52:0/648911158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56125c105450 0x56125c105830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.373+0000 7fc261d9b640 1 -- 172.21.15.52:0/648911158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56125c105450 msgr2=0x56125c105830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.373+0000 7fc261d9b640 1 --2- 172.21.15.52:0/648911158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56125c105450 0x56125c105830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.373+0000 7fc261d9b640 1 -- 172.21.15.52:0/648911158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c1130b0 msgr2=0x56125c1154a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.373+0000 7fc261d9b640 1 --2- 172.21.15.52:0/648911158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c1130b0 0x56125c1154a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.373+0000 7fc261d9b640 1 -- 172.21.15.52:0/648911158 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56125bd7d3f0 con 0x56125c105d70 2024-04-02T13:55:14.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.373+0000 7fc26ad9d640 1 --2- 172.21.15.52:0/2032219968 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc2580a3d20 0x7fc2580a61e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc264005fd0 tx=0x7fc26400ad30 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.373+0000 7fc261d9b640 1 --2- 172.21.15.52:0/648911158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56125c105d70 0x56125c112b70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc2540bac70 tx=0x7fc2540be070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7f06ba9929994f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.374+0000 7fc260d99640 1 -- 172.21.15.52:0/648911158 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25404b070 con 0x56125c105d70 2024-04-02T13:55:14.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.374+0000 7fc260d99640 1 -- 172.21.15.52:0/648911158 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc254004c80 con 0x56125c105d70 2024-04-02T13:55:14.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.374+0000 7fc260d99640 1 -- 172.21.15.52:0/648911158 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2540c6f10 con 0x56125c105d70 2024-04-02T13:55:14.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.374+0000 7fc2737bf740 1 -- 172.21.15.52:0/648911158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56125c105d70 msgr2=0x56125c112b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.374+0000 7fc2737bf740 1 --2- 172.21.15.52:0/648911158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56125c105d70 0x56125c112b70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc2540bac70 tx=0x7fc2540be070 comp rx=0 tx=0).stop 2024-04-02T13:55:14.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.374+0000 7fc2737bf740 1 -- 172.21.15.52:0/648911158 shutdown_connections 2024-04-02T13:55:14.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.374+0000 7fc2737bf740 1 --2- 172.21.15.52:0/648911158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c1130b0 0x56125c1154a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.374+0000 7fc2737bf740 1 --2- 172.21.15.52:0/648911158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56125c105d70 0x56125c112b70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.374+0000 7fc2737bf740 1 --2- 172.21.15.52:0/648911158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56125c105450 0x56125c105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.374+0000 7fc2737bf740 1 -- 172.21.15.52:0/648911158 >> 172.21.15.52:0/648911158 conn(0x56125c0ffed0 msgr2=0x56125c1002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.375+0000 7fc2737bf740 1 -- 172.21.15.52:0/648911158 shutdown_connections 2024-04-02T13:55:14.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.375+0000 7fc2737bf740 1 -- 172.21.15.52:0/648911158 wait complete. 2024-04-02T13:55:14.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.375+0000 7fc2737bf740 1 -- start start 2024-04-02T13:55:14.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.375+0000 7fc2737bf740 1 Processor -- start 2024-04-02T13:55:14.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.375+0000 7fc2737bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56125c105450 0x56125c112a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.375+0000 7fc2737bf740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c105d70 0x56125c10fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.375+0000 7fc2737bf740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56125c1130b0 0x56125c110350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.375+0000 7fc2737bf740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56125c115cb0 con 0x56125c105d70 2024-04-02T13:55:14.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.375+0000 7fc2737bf740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56125c024e80 con 0x56125c1130b0 2024-04-02T13:55:14.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.375+0000 7fc2737bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56125c115e30 con 0x56125c105450 2024-04-02T13:55:14.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.375+0000 7fc2737bf740 10 client.0 _ll_register_callbacks cb 0x56125c060aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:14.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.375+0000 7fc261d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c105d70 0x56125c10fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.375+0000 7fc261d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c105d70 0x56125c10fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47284/0 (socket says 172.21.15.52:47284) 2024-04-02T13:55:14.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.375+0000 7fc261d9b640 1 -- 172.21.15.52:0/3948181983 learned_addr learned my addr 172.21.15.52:0/3948181983 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.375+0000 7fc26a59c640 1 --2- 172.21.15.52:0/3948181983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56125c105450 0x56125c112a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.375+0000 7fc26ad9d640 1 --2- 172.21.15.52:0/3948181983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56125c1130b0 0x56125c110350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.375+0000 7fc261d9b640 1 -- 172.21.15.52:0/3948181983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56125c105450 msgr2=0x56125c112a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.375+0000 7fc261d9b640 1 --2- 172.21.15.52:0/3948181983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56125c105450 0x56125c112a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.375+0000 7fc261d9b640 1 -- 172.21.15.52:0/3948181983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56125c1130b0 msgr2=0x56125c110350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.375+0000 7fc261d9b640 1 --2- 172.21.15.52:0/3948181983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56125c1130b0 0x56125c110350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.376+0000 7fc261d9b640 1 -- 172.21.15.52:0/3948181983 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56125be477d0 con 0x56125c105d70 2024-04-02T13:55:14.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.376+0000 7fc26ad9d640 1 --2- 172.21.15.52:0/3948181983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56125c1130b0 0x56125c110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:14.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.376+0000 7fc261d9b640 1 --2- 172.21.15.52:0/3948181983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c105d70 0x56125c10fdd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc2540efb00 tx=0x7fc2540bdd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45c7464324c7cdf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.376+0000 7fc251ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:14.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.376+0000 7fc251ffb640 1 -- 172.21.15.52:0/3948181983 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25404b050 con 0x56125c105d70 2024-04-02T13:55:14.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.376+0000 7fc251ffb640 1 -- 172.21.15.52:0/3948181983 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc2540f2190 con 0x56125c105d70 2024-04-02T13:55:14.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.376+0000 7fc2737bf740 1 -- 172.21.15.52:0/3948181983 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc254004c80 con 0x56125c105d70 2024-04-02T13:55:14.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.376+0000 7fc251ffb640 1 -- 172.21.15.52:0/3948181983 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc2540be9a0 con 0x56125c105d70 2024-04-02T13:55:14.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.376+0000 7fc2737bf740 10 client.16128 fetch_fsmap learned FSMap version 28 2024-04-02T13:55:14.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.376+0000 7fc2737bf740 1 -- 172.21.15.52:0/3948181983 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56125c115cb0 con 0x56125c105d70 2024-04-02T13:55:14.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.376+0000 7fc251ffb640 1 -- 172.21.15.52:0/3948181983 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc254004c80 con 0x56125c105d70 2024-04-02T13:55:14.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.376+0000 7fc251ffb640 1 -- 172.21.15.52:0/3948181983 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc2540c6a80 con 0x56125c105d70 2024-04-02T13:55:14.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.376+0000 7fc2737bf740 10 client.16128 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:55:14.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.376+0000 7fc2737bf740 10 client.16128 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T13:55:14.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.376+0000 7fc2737bf740 20 client.16128 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:14.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.376+0000 7fc2737bf740 1 --2- 172.21.15.52:0/3948181983 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x56125c123010 0x56125c125430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.376+0000 7fc2737bf740 4 client.16128 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:14.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.376+0000 7fc2737bf740 1 -- 172.21.15.52:0/3948181983 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56125c076cf0 con 0x56125c123010 2024-04-02T13:55:14.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.377+0000 7fc26a59c640 1 --2- 172.21.15.52:0/3948181983 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x56125c123010 0x56125c125430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.377+0000 7fc251ffb640 10 client.16128 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:55:14.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.377+0000 7fc26a59c640 1 --2- 172.21.15.52:0/3948181983 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x56125c123010 0x56125c125430 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2422c4583c8a08f server_cookie=15e12af08c136a4 in_seq=0 out_seq=0 2024-04-02T13:55:14.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.378+0000 7fc251ffb640 1 -- 172.21.15.52:0/3948181983 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56125c076cf0 con 0x56125c123010 2024-04-02T13:55:14.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.378+0000 7fc251ffb640 10 client.16128 handle_command_reply: tid=0 2024-04-02T13:55:14.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.378+0000 7fc2737bf740 1 client.16128 shutdown 2024-04-02T13:55:14.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.378+0000 7fc2737bf740 1 -- 172.21.15.52:0/3948181983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c105d70 msgr2=0x56125c10fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.378+0000 7fc2737bf740 1 --2- 172.21.15.52:0/3948181983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c105d70 0x56125c10fdd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc2540efb00 tx=0x7fc2540bdd30 comp rx=0 tx=0).stop 2024-04-02T13:55:14.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.378+0000 7fc2737bf740 1 -- 172.21.15.52:0/3948181983 shutdown_connections 2024-04-02T13:55:14.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.378+0000 7fc2737bf740 1 --2- 172.21.15.52:0/3948181983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56125c1130b0 0x56125c110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.378+0000 7fc2737bf740 1 --2- 172.21.15.52:0/3948181983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c105d70 0x56125c10fdd0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.378+0000 7fc2737bf740 1 --2- 172.21.15.52:0/3948181983 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x56125c123010 0x56125c125430 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.378+0000 7fc2737bf740 1 --2- 172.21.15.52:0/3948181983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56125c105450 0x56125c112a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.378+0000 7fc251ffb640 0 client.16128 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:55:14.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.378+0000 7fc2737bf740 1 -- 172.21.15.52:0/3948181983 >> 172.21.15.52:0/3948181983 conn(0x56125c0ffed0 msgr2=0x56125c101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.378+0000 7fc2737bf740 1 -- 172.21.15.52:0/3948181983 shutdown_connections 2024-04-02T13:55:14.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.378+0000 7fc2737bf740 1 -- 172.21.15.52:0/3948181983 wait complete. 2024-04-02T13:55:14.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.378+0000 7fc2737bf740 20 client.16128 trim_cache size 0 max 16384 2024-04-02T13:55:14.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.392+0000 7fc2737bf740 1 Processor -- start 2024-04-02T13:55:14.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.392+0000 7fc2737bf740 1 -- start start 2024-04-02T13:55:14.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.392+0000 7fc2737bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56125c105450 0x56125c10fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.392+0000 7fc2737bf740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56125c105d70 0x56125c11eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.392+0000 7fc2737bf740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c1130b0 0x56125c112500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.392+0000 7fc2737bf740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c1acae0 con 0x56125c1130b0 2024-04-02T13:55:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.392+0000 7fc2737bf740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c1ae620 con 0x56125c105d70 2024-04-02T13:55:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.392+0000 7fc2737bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c1ad1b0 con 0x56125c105450 2024-04-02T13:55:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.392+0000 7fc26ad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c1130b0 0x56125c112500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.392+0000 7fc26ad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c1130b0 0x56125c112500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47296/0 (socket says 172.21.15.52:47296) 2024-04-02T13:55:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.392+0000 7fc26ad9d640 1 -- 172.21.15.52:0/2276553752 learned_addr learned my addr 172.21.15.52:0/2276553752 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.392+0000 7fc261d9b640 1 --2- 172.21.15.52:0/2276553752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56125c105d70 0x56125c11eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.392+0000 7fc26a59c640 1 --2- 172.21.15.52:0/2276553752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56125c105450 0x56125c10fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.393+0000 7fc26ad9d640 1 -- 172.21.15.52:0/2276553752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56125c105450 msgr2=0x56125c10fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.393+0000 7fc26ad9d640 1 --2- 172.21.15.52:0/2276553752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56125c105450 0x56125c10fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.393+0000 7fc26ad9d640 1 -- 172.21.15.52:0/2276553752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56125c105d70 msgr2=0x56125c11eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.393+0000 7fc26ad9d640 1 --2- 172.21.15.52:0/2276553752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56125c105d70 0x56125c11eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.393+0000 7fc26ad9d640 1 -- 172.21.15.52:0/2276553752 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56125c1257a0 con 0x56125c1130b0 2024-04-02T13:55:14.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.393+0000 7fc261d9b640 1 --2- 172.21.15.52:0/2276553752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56125c105d70 0x56125c11eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:14.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.393+0000 7fc26ad9d640 1 --2- 172.21.15.52:0/2276553752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c1130b0 0x56125c112500 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc264012560 tx=0x7fc26401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0a83e7e88e918 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.395+0000 7fc253fff640 1 -- 172.21.15.52:0/2276553752 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26402d070 con 0x56125c1130b0 2024-04-02T13:55:14.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.395+0000 7fc253fff640 1 -- 172.21.15.52:0/2276553752 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc264020d90 con 0x56125c1130b0 2024-04-02T13:55:14.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.395+0000 7fc253fff640 1 -- 172.21.15.52:0/2276553752 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26401e670 con 0x56125c1130b0 2024-04-02T13:55:14.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.395+0000 7fc2737bf740 1 -- 172.21.15.52:0/2276553752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c1130b0 msgr2=0x56125c112500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.395+0000 7fc2737bf740 1 --2- 172.21.15.52:0/2276553752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c1130b0 0x56125c112500 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc264012560 tx=0x7fc26401c000 comp rx=0 tx=0).stop 2024-04-02T13:55:14.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.396+0000 7fc2737bf740 1 -- 172.21.15.52:0/2276553752 shutdown_connections 2024-04-02T13:55:14.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.396+0000 7fc2737bf740 1 --2- 172.21.15.52:0/2276553752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c1130b0 0x56125c112500 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.396+0000 7fc2737bf740 1 --2- 172.21.15.52:0/2276553752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56125c105d70 0x56125c11eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.396+0000 7fc2737bf740 1 --2- 172.21.15.52:0/2276553752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56125c105450 0x56125c10fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.396+0000 7fc2737bf740 1 -- 172.21.15.52:0/2276553752 >> 172.21.15.52:0/2276553752 conn(0x56125c0ffed0 msgr2=0x56125c10b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.396+0000 7fc2737bf740 1 -- 172.21.15.52:0/2276553752 shutdown_connections 2024-04-02T13:55:14.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.396+0000 7fc2737bf740 1 -- 172.21.15.52:0/2276553752 wait complete. 2024-04-02T13:55:14.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.396+0000 7fc2737bf740 1 -- start start 2024-04-02T13:55:14.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.396+0000 7fc2737bf740 1 Processor -- start 2024-04-02T13:55:14.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.396+0000 7fc2737bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56125c105450 0x56125c115660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.396+0000 7fc2737bf740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c105d70 0x56125c113a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.396+0000 7fc2737bf740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56125c1130b0 0x56125c114110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.396+0000 7fc2737bf740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c1adf50 con 0x56125c105d70 2024-04-02T13:55:14.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.396+0000 7fc2737bf740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c1ad880 con 0x56125c1130b0 2024-04-02T13:55:14.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.396+0000 7fc2737bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c1aaac0 con 0x56125c105450 2024-04-02T13:55:14.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.396+0000 7fc2737bf740 10 client.0 _ll_register_callbacks cb 0x7fc25c009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:14.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.396+0000 7fc261d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c105d70 0x56125c113a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.396+0000 7fc261d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c105d70 0x56125c113a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47300/0 (socket says 172.21.15.52:47300) 2024-04-02T13:55:14.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.396+0000 7fc261d9b640 1 -- 172.21.15.52:0/4166530466 learned_addr learned my addr 172.21.15.52:0/4166530466 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.396+0000 7fc26ad9d640 1 --2- 172.21.15.52:0/4166530466 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56125c1130b0 0x56125c114110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.397+0000 7fc26a59c640 1 --2- 172.21.15.52:0/4166530466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56125c105450 0x56125c115660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.397+0000 7fc261d9b640 1 -- 172.21.15.52:0/4166530466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56125c105450 msgr2=0x56125c115660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.397+0000 7fc261d9b640 1 --2- 172.21.15.52:0/4166530466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56125c105450 0x56125c115660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.397+0000 7fc261d9b640 1 -- 172.21.15.52:0/4166530466 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56125c1130b0 msgr2=0x56125c114110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.397+0000 7fc261d9b640 1 --2- 172.21.15.52:0/4166530466 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56125c1130b0 0x56125c114110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.397+0000 7fc261d9b640 1 -- 172.21.15.52:0/4166530466 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56125c11a080 con 0x56125c105d70 2024-04-02T13:55:14.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.397+0000 7fc26ad9d640 1 --2- 172.21.15.52:0/4166530466 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56125c1130b0 0x56125c114110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:14.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.397+0000 7fc261d9b640 1 --2- 172.21.15.52:0/4166530466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c105d70 0x56125c113a90 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc2540eff40 tx=0x7fc2540bc7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef7ea403089ce19 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.397+0000 7fc251ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:14.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.397+0000 7fc251ffb640 1 -- 172.21.15.52:0/4166530466 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25404b040 con 0x56125c105d70 2024-04-02T13:55:14.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.397+0000 7fc251ffb640 1 -- 172.21.15.52:0/4166530466 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc2540a74a0 con 0x56125c105d70 2024-04-02T13:55:14.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.397+0000 7fc2737bf740 1 -- 172.21.15.52:0/4166530466 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc264020d90 con 0x56125c105d70 2024-04-02T13:55:14.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.397+0000 7fc251ffb640 1 -- 172.21.15.52:0/4166530466 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc254004c80 con 0x56125c105d70 2024-04-02T13:55:14.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.397+0000 7fc2737bf740 10 client.16134 fetch_fsmap learned FSMap version 28 2024-04-02T13:55:14.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.397+0000 7fc2737bf740 1 -- 172.21.15.52:0/4166530466 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56125c11a080 con 0x56125c105d70 2024-04-02T13:55:14.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.397+0000 7fc251ffb640 1 -- 172.21.15.52:0/4166530466 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc264020d90 con 0x56125c105d70 2024-04-02T13:55:14.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.397+0000 7fc251ffb640 1 -- 172.21.15.52:0/4166530466 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc2540bd6f0 con 0x56125c105d70 2024-04-02T13:55:14.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.397+0000 7fc2737bf740 10 client.16134 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:55:14.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.397+0000 7fc2737bf740 10 client.16134 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T13:55:14.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.397+0000 7fc2737bf740 20 client.16134 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:14.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.397+0000 7fc2737bf740 1 --2- 172.21.15.52:0/4166530466 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x56125c11b5b0 0x56125c18b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.397+0000 7fc2737bf740 4 client.16134 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:55:14.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.397+0000 7fc2737bf740 1 -- 172.21.15.52:0/4166530466 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56125c071040 con 0x56125c11b5b0 2024-04-02T13:55:14.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.398+0000 7fc26a59c640 1 --2- 172.21.15.52:0/4166530466 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x56125c11b5b0 0x56125c18b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.398+0000 7fc26a59c640 1 --2- 172.21.15.52:0/4166530466 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x56125c11b5b0 0x56125c18b580 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f88cb54fd979728 server_cookie=c9c8c51810e7a068 in_seq=0 out_seq=0 2024-04-02T13:55:14.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.398+0000 7fc251ffb640 10 client.16134 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:55:14.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.398+0000 7fc251ffb640 1 -- 172.21.15.52:0/4166530466 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56125c071040 con 0x56125c11b5b0 2024-04-02T13:55:14.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.398+0000 7fc251ffb640 10 client.16134 handle_command_reply: tid=0 2024-04-02T13:55:14.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.398+0000 7fc2737bf740 1 client.16134 shutdown 2024-04-02T13:55:14.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.398+0000 7fc2737bf740 1 -- 172.21.15.52:0/4166530466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c105d70 msgr2=0x56125c113a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.398+0000 7fc2737bf740 1 --2- 172.21.15.52:0/4166530466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c105d70 0x56125c113a90 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc2540eff40 tx=0x7fc2540bc7b0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.398+0000 7fc2737bf740 1 -- 172.21.15.52:0/4166530466 shutdown_connections 2024-04-02T13:55:14.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.398+0000 7fc2737bf740 1 --2- 172.21.15.52:0/4166530466 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56125c1130b0 0x56125c114110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.398+0000 7fc2737bf740 1 --2- 172.21.15.52:0/4166530466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56125c105d70 0x56125c113a90 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.398+0000 7fc2737bf740 1 --2- 172.21.15.52:0/4166530466 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x56125c11b5b0 0x56125c18b580 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.398+0000 7fc2737bf740 1 --2- 172.21.15.52:0/4166530466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56125c105450 0x56125c115660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.398+0000 7fc251ffb640 0 client.16134 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:55:14.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.398+0000 7fc2737bf740 1 -- 172.21.15.52:0/4166530466 >> 172.21.15.52:0/4166530466 conn(0x56125c0ffed0 msgr2=0x56125c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.399+0000 7fc2737bf740 1 -- 172.21.15.52:0/4166530466 shutdown_connections 2024-04-02T13:55:14.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.399+0000 7fc2737bf740 1 -- 172.21.15.52:0/4166530466 wait complete. 2024-04-02T13:55:14.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.399+0000 7fc2737bf740 20 client.16134 trim_cache size 0 max 16384 2024-04-02T13:55:14.408 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:55:14.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.399+0000 7fc270b00640 1 -- 172.21.15.52:0/2032219968 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc2580a3d20 msgr2=0x7fc2580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.399+0000 7fc270b00640 1 --2- 172.21.15.52:0/2032219968 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc2580a3d20 0x7fc2580a61e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc264005fd0 tx=0x7fc26400ad30 comp rx=0 tx=0).stop 2024-04-02T13:55:14.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.399+0000 7fc270b00640 1 -- 172.21.15.52:0/2032219968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc26c102450 msgr2=0x7fc26c1988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.399+0000 7fc270b00640 1 --2- 172.21.15.52:0/2032219968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc26c102450 0x7fc26c1988b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fc254009230 tx=0x7fc25402cba0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.399+0000 7fc270b00640 1 -- 172.21.15.52:0/2032219968 shutdown_connections 2024-04-02T13:55:14.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.399+0000 7fc270b00640 1 --2- 172.21.15.52:0/2032219968 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc2580a3d20 0x7fc2580a61e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.399+0000 7fc270b00640 1 --2- 172.21.15.52:0/2032219968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc26c107aa0 0x7fc26c19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.399+0000 7fc270b00640 1 --2- 172.21.15.52:0/2032219968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc26c102450 0x7fc26c1988b0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.399+0000 7fc270b00640 1 --2- 172.21.15.52:0/2032219968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc26c101a90 0x7fc26c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.399+0000 7fc270b00640 1 -- 172.21.15.52:0/2032219968 >> 172.21.15.52:0/2032219968 conn(0x7fc26c078f00 msgr2=0x7fc26c104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.399+0000 7fc270b00640 1 -- 172.21.15.52:0/2032219968 shutdown_connections 2024-04-02T13:55:14.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.399+0000 7fc270b00640 1 -- 172.21.15.52:0/2032219968 wait complete. 2024-04-02T13:55:14.413 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:55:14.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.515+0000 7f5ae42eb640 1 -- 172.21.15.52:0/1840892467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5adc100970 msgr2=0x7f5adc100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.515+0000 7f5ae42eb640 1 --2- 172.21.15.52:0/1840892467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5adc100970 0x7f5adc100df0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5acc009930 tx=0x7f5acc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:55:14.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.515+0000 7f5ae42eb640 1 -- 172.21.15.52:0/1840892467 shutdown_connections 2024-04-02T13:55:14.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.515+0000 7f5ae42eb640 1 --2- 172.21.15.52:0/1840892467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5adc101330 0x7f5adc10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.515+0000 7f5ae42eb640 1 --2- 172.21.15.52:0/1840892467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5adc100970 0x7f5adc100df0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.515+0000 7f5ae42eb640 1 --2- 172.21.15.52:0/1840892467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5adc106980 0x7f5adc106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.515+0000 7f5ae42eb640 1 -- 172.21.15.52:0/1840892467 >> 172.21.15.52:0/1840892467 conn(0x7f5adc077b00 msgr2=0x7f5adc077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.515+0000 7f5ae42eb640 1 -- 172.21.15.52:0/1840892467 shutdown_connections 2024-04-02T13:55:14.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.515+0000 7f5ae42eb640 1 -- 172.21.15.52:0/1840892467 wait complete. 2024-04-02T13:55:14.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.515+0000 7f5ae42eb640 1 Processor -- start 2024-04-02T13:55:14.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.515+0000 7f5ae42eb640 1 -- start start 2024-04-02T13:55:14.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.516+0000 7f5ae42eb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5adc100970 0x7f5adc102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.516+0000 7f5ae42eb640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5adc101330 0x7f5adc1033e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.516+0000 7f5ae42eb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5adc106980 0x7f5adc103920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.516+0000 7f5ae42eb640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc110d60 con 0x7f5adc106980 2024-04-02T13:55:14.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.516+0000 7f5ae42eb640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc110be0 con 0x7f5adc101330 2024-04-02T13:55:14.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.516+0000 7f5ae42eb640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc110ee0 con 0x7f5adc100970 2024-04-02T13:55:14.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.516+0000 7f5ae2888640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5adc106980 0x7f5adc103920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.516+0000 7f5ae2888640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5adc106980 0x7f5adc103920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47326/0 (socket says 172.21.15.52:47326) 2024-04-02T13:55:14.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.516+0000 7f5ae2888640 1 -- 172.21.15.52:0/306518787 learned_addr learned my addr 172.21.15.52:0/306518787 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.516+0000 7f5ae1886640 1 --2- 172.21.15.52:0/306518787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5adc101330 0x7f5adc1033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.516+0000 7f5ae2087640 1 --2- 172.21.15.52:0/306518787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5adc100970 0x7f5adc102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.516+0000 7f5ae2888640 1 -- 172.21.15.52:0/306518787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5adc100970 msgr2=0x7f5adc102ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.516+0000 7f5ae2888640 1 --2- 172.21.15.52:0/306518787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5adc100970 0x7f5adc102ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.516+0000 7f5ae2888640 1 -- 172.21.15.52:0/306518787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5adc101330 msgr2=0x7f5adc1033e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.516+0000 7f5ae2888640 1 --2- 172.21.15.52:0/306518787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5adc101330 0x7f5adc1033e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.516+0000 7f5ae2888640 1 -- 172.21.15.52:0/306518787 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5adc1a6850 con 0x7f5adc106980 2024-04-02T13:55:14.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.516+0000 7f5ae2888640 1 --2- 172.21.15.52:0/306518787 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5adc106980 0x7f5adc103920 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400eaa0 tx=0x7f5ad400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.516+0000 7f5ad37fe640 1 -- 172.21.15.52:0/306518787 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad400cdb0 con 0x7f5adc106980 2024-04-02T13:55:14.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.516+0000 7f5ad37fe640 1 -- 172.21.15.52:0/306518787 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ad4005020 con 0x7f5adc106980 2024-04-02T13:55:14.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.516+0000 7f5ae42eb640 1 -- 172.21.15.52:0/306518787 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5adc1a6b40 con 0x7f5adc106980 2024-04-02T13:55:14.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.516+0000 7f5ad37fe640 1 -- 172.21.15.52:0/306518787 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad40116b0 con 0x7f5adc106980 2024-04-02T13:55:14.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.516+0000 7f5ae42eb640 1 -- 172.21.15.52:0/306518787 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5adc1a7060 con 0x7f5adc106980 2024-04-02T13:55:14.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.518+0000 7f5ad37fe640 1 -- 172.21.15.52:0/306518787 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5ad4005560 con 0x7f5adc106980 2024-04-02T13:55:14.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.519+0000 7f5ae6faf740 1 Processor -- start 2024-04-02T13:55:14.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.519+0000 7f5ae6faf740 1 -- start start 2024-04-02T13:55:14.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.519+0000 7f5ad37fe640 1 --2- 172.21.15.52:0/306518787 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5ac80a3da0 0x7f5ac80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.519+0000 7f5ae6faf740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf49f7a0 0x55cfdf49fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.519+0000 7f5ae6faf740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cfdf4a00c0 0x55cfdf4a8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.519+0000 7f5ae6faf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cfdf4a8ca0 0x55cfdf4ab090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.519+0000 7f5ad37fe640 1 -- 172.21.15.52:0/306518787 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f5ad40d0d20 con 0x7f5adc106980 2024-04-02T13:55:14.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.519+0000 7f5ae6faf740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cfdf4037b0 con 0x55cfdf49f7a0 2024-04-02T13:55:14.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.519+0000 7f5ae6faf740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cfdf3a2570 con 0x55cfdf4a00c0 2024-04-02T13:55:14.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.519+0000 7f5ae6faf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cfdf3ff760 con 0x55cfdf4a8ca0 2024-04-02T13:55:14.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.520+0000 7f5ae1886640 1 --2- 172.21.15.52:0/306518787 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5ac80a3da0 0x7f5ac80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.520+0000 7f5ae2087640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf49f7a0 0x55cfdf49fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.520+0000 7f5ae2087640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf49f7a0 0x55cfdf49fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47338/0 (socket says 172.21.15.52:47338) 2024-04-02T13:55:14.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.520+0000 7f5ae2087640 1 -- 172.21.15.52:0/3365739181 learned_addr learned my addr 172.21.15.52:0/3365739181 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.520+0000 7f5ae2888640 1 --2- 172.21.15.52:0/3365739181 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cfdf4a8ca0 0x55cfdf4ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.520+0000 7f5ae2087640 1 -- 172.21.15.52:0/3365739181 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cfdf4a8ca0 msgr2=0x55cfdf4ab090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.520+0000 7f5ae2087640 1 --2- 172.21.15.52:0/3365739181 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cfdf4a8ca0 0x55cfdf4ab090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.520+0000 7f5ae2087640 1 -- 172.21.15.52:0/3365739181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cfdf4a00c0 msgr2=0x55cfdf4a8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.520+0000 7f5ae2087640 1 --2- 172.21.15.52:0/3365739181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cfdf4a00c0 0x55cfdf4a8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.520+0000 7f5ae2087640 1 -- 172.21.15.52:0/3365739181 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfdf1173f0 con 0x55cfdf49f7a0 2024-04-02T13:55:14.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.520+0000 7f5ae2888640 1 --2- 172.21.15.52:0/3365739181 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cfdf4a8ca0 0x55cfdf4ab090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:14.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.520+0000 7f5ae2087640 1 --2- 172.21.15.52:0/3365739181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf49f7a0 0x55cfdf49fb80 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800cbe0 tx=0x7f5ad800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c7edb9a67252b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.520+0000 7f5ae1886640 1 --2- 172.21.15.52:0/306518787 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5ac80a3da0 0x7f5ac80a6260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5acc005ab0 tx=0x7f5acc009560 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.520+0000 7f5ad0bfe640 1 -- 172.21.15.52:0/3365739181 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8005220 con 0x55cfdf49f7a0 2024-04-02T13:55:14.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.520+0000 7f5ad0bfe640 1 -- 172.21.15.52:0/3365739181 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ad8004c80 con 0x55cfdf49f7a0 2024-04-02T13:55:14.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.520+0000 7f5ad0bfe640 1 -- 172.21.15.52:0/3365739181 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad80107f0 con 0x55cfdf49f7a0 2024-04-02T13:55:14.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.521+0000 7f5ae6faf740 1 -- 172.21.15.52:0/3365739181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf49f7a0 msgr2=0x55cfdf49fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.521+0000 7f5ae6faf740 1 --2- 172.21.15.52:0/3365739181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf49f7a0 0x55cfdf49fb80 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800cbe0 tx=0x7f5ad800e720 comp rx=0 tx=0).stop 2024-04-02T13:55:14.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.521+0000 7f5ae6faf740 1 -- 172.21.15.52:0/3365739181 shutdown_connections 2024-04-02T13:55:14.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.521+0000 7f5ae6faf740 1 --2- 172.21.15.52:0/3365739181 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cfdf4a8ca0 0x55cfdf4ab090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.521+0000 7f5ae6faf740 1 --2- 172.21.15.52:0/3365739181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cfdf4a00c0 0x55cfdf4a8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.521+0000 7f5ae6faf740 1 --2- 172.21.15.52:0/3365739181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf49f7a0 0x55cfdf49fb80 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.521+0000 7f5ae6faf740 1 -- 172.21.15.52:0/3365739181 >> 172.21.15.52:0/3365739181 conn(0x55cfdf499ed0 msgr2=0x55cfdf49a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.521+0000 7f5ae6faf740 1 -- 172.21.15.52:0/3365739181 shutdown_connections 2024-04-02T13:55:14.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.521+0000 7f5ae6faf740 1 -- 172.21.15.52:0/3365739181 wait complete. 2024-04-02T13:55:14.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.521+0000 7f5ae6faf740 1 -- start start 2024-04-02T13:55:14.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.521+0000 7f5ae6faf740 1 Processor -- start 2024-04-02T13:55:14.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.522+0000 7f5ae6faf740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cfdf49f7a0 0x55cfdf4a8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.522+0000 7f5ae6faf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cfdf4a00c0 0x55cfdf4a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.522+0000 7f5ae6faf740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf4a8ca0 0x55cfdf4a6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.522+0000 7f5ae6faf740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cfdf4aba90 con 0x55cfdf4a8ca0 2024-04-02T13:55:14.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.522+0000 7f5ae6faf740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cfdf3bee80 con 0x55cfdf49f7a0 2024-04-02T13:55:14.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.522+0000 7f5ae6faf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cfdf4abc10 con 0x55cfdf4a00c0 2024-04-02T13:55:14.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.522+0000 7f5ae6faf740 10 client.0 _ll_register_callbacks cb 0x55cfdf3faaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:14.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.522+0000 7f5ae2888640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf4a8ca0 0x55cfdf4a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.522+0000 7f5ae2888640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf4a8ca0 0x55cfdf4a6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47352/0 (socket says 172.21.15.52:47352) 2024-04-02T13:55:14.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.522+0000 7f5ae2888640 1 -- 172.21.15.52:0/417185432 learned_addr learned my addr 172.21.15.52:0/417185432 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.522+0000 7f5ae2087640 1 --2- 172.21.15.52:0/417185432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cfdf49f7a0 0x55cfdf4a8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.522+0000 7f5ae1886640 1 --2- 172.21.15.52:0/417185432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cfdf4a00c0 0x55cfdf4a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.522+0000 7f5ae2888640 1 -- 172.21.15.52:0/417185432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cfdf4a00c0 msgr2=0x55cfdf4a5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.522+0000 7f5ae2888640 1 --2- 172.21.15.52:0/417185432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cfdf4a00c0 0x55cfdf4a5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.522+0000 7f5ae2888640 1 -- 172.21.15.52:0/417185432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cfdf49f7a0 msgr2=0x55cfdf4a8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.522+0000 7f5ae2888640 1 --2- 172.21.15.52:0/417185432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cfdf49f7a0 0x55cfdf4a8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.522+0000 7f5ae2888640 1 -- 172.21.15.52:0/417185432 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfdf1e17d0 con 0x55cfdf4a8ca0 2024-04-02T13:55:14.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.522+0000 7f5ae2888640 1 --2- 172.21.15.52:0/417185432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf4a8ca0 0x55cfdf4a6220 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4098de0 tx=0x7f5ad409cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f446089472c0f09 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.522+0000 7f5ac5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:14.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.522+0000 7f5ac5ffb640 1 -- 172.21.15.52:0/417185432 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad401f040 con 0x55cfdf4a8ca0 2024-04-02T13:55:14.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.523+0000 7f5ac5ffb640 1 -- 172.21.15.52:0/417185432 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f5ad40870b0 con 0x55cfdf4a8ca0 2024-04-02T13:55:14.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.523+0000 7f5ae6faf740 1 -- 172.21.15.52:0/417185432 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ad8004c80 con 0x55cfdf4a8ca0 2024-04-02T13:55:14.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.523+0000 7f5ac5ffb640 1 -- 172.21.15.52:0/417185432 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5ad40872b0 con 0x55cfdf4a8ca0 2024-04-02T13:55:14.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.523+0000 7f5ae6faf740 10 client.16146 fetch_fsmap learned FSMap version 28 2024-04-02T13:55:14.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.523+0000 7f5ae6faf740 1 -- 172.21.15.52:0/417185432 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cfdf4b7bd0 con 0x55cfdf4a8ca0 2024-04-02T13:55:14.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.523+0000 7f5ac5ffb640 1 -- 172.21.15.52:0/417185432 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ad8004c80 con 0x55cfdf4a8ca0 2024-04-02T13:55:14.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.523+0000 7f5ac5ffb640 1 -- 172.21.15.52:0/417185432 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5ad400f040 con 0x55cfdf4a8ca0 2024-04-02T13:55:14.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.523+0000 7f5ae6faf740 10 client.16146 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:55:14.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.523+0000 7f5ae6faf740 10 client.16146 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T13:55:14.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.523+0000 7f5ae6faf740 20 client.16146 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:14.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.523+0000 7f5ae6faf740 1 --2- 172.21.15.52:0/417185432 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55cfdf4b9090 0x55cfdf4bb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.523+0000 7f5ae6faf740 4 client.16146 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:14.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.523+0000 7f5ae6faf740 1 -- 172.21.15.52:0/417185432 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfdf410cf0 con 0x55cfdf4b9090 2024-04-02T13:55:14.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.523+0000 7f5ae2087640 1 --2- 172.21.15.52:0/417185432 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55cfdf4b9090 0x55cfdf4bb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.524+0000 7f5ac5ffb640 10 client.16146 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:55:14.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.524+0000 7f5ae2087640 1 --2- 172.21.15.52:0/417185432 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55cfdf4b9090 0x55cfdf4bb4e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dca25bb55bdfd5c5 server_cookie=ba86657d30cb5b4c in_seq=0 out_seq=0 2024-04-02T13:55:14.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.524+0000 7f5ac5ffb640 1 -- 172.21.15.52:0/417185432 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cfdf410cf0 con 0x55cfdf4b9090 2024-04-02T13:55:14.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.524+0000 7f5ac5ffb640 10 client.16146 handle_command_reply: tid=0 2024-04-02T13:55:14.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.525+0000 7f5ae6faf740 1 client.16146 shutdown 2024-04-02T13:55:14.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.525+0000 7f5ae6faf740 1 -- 172.21.15.52:0/417185432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf4a8ca0 msgr2=0x55cfdf4a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.525+0000 7f5ae6faf740 1 --2- 172.21.15.52:0/417185432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf4a8ca0 0x55cfdf4a6220 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4098de0 tx=0x7f5ad409cac0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.525+0000 7f5ae6faf740 1 -- 172.21.15.52:0/417185432 shutdown_connections 2024-04-02T13:55:14.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.525+0000 7f5ae6faf740 1 --2- 172.21.15.52:0/417185432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf4a8ca0 0x55cfdf4a6220 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.525+0000 7f5ae6faf740 1 --2- 172.21.15.52:0/417185432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cfdf4a00c0 0x55cfdf4a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.525+0000 7f5ae6faf740 1 --2- 172.21.15.52:0/417185432 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55cfdf4b9090 0x55cfdf4bb4e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.525+0000 7f5ae6faf740 1 --2- 172.21.15.52:0/417185432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cfdf49f7a0 0x55cfdf4a8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.525+0000 7f5ac5ffb640 0 client.16146 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:55:14.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.525+0000 7f5ae6faf740 1 -- 172.21.15.52:0/417185432 >> 172.21.15.52:0/417185432 conn(0x55cfdf499ed0 msgr2=0x55cfdf49bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.525+0000 7f5ae6faf740 1 -- 172.21.15.52:0/417185432 shutdown_connections 2024-04-02T13:55:14.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.525+0000 7f5ae6faf740 1 -- 172.21.15.52:0/417185432 wait complete. 2024-04-02T13:55:14.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.525+0000 7f5ae6faf740 20 client.16146 trim_cache size 0 max 16384 2024-04-02T13:55:14.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.538+0000 7f5ae6faf740 1 Processor -- start 2024-04-02T13:55:14.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.538+0000 7f5ae6faf740 1 -- start start 2024-04-02T13:55:14.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.538+0000 7f5ae6faf740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cfdf49f7a0 0x55cfdf4b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.538+0000 7f5ae6faf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cfdf4a00c0 0x55cfdf4b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.538+0000 7f5ae6faf740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf4a8ca0 0x55cfdf4a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.538+0000 7f5ae6faf740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc1b0ff0 con 0x55cfdf4a8ca0 2024-04-02T13:55:14.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.538+0000 7f5ae6faf740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc1b2b30 con 0x55cfdf49f7a0 2024-04-02T13:55:14.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.538+0000 7f5ae6faf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc1b16c0 con 0x55cfdf4a00c0 2024-04-02T13:55:14.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.539+0000 7f5ae2888640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf4a8ca0 0x55cfdf4a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.539+0000 7f5ae2888640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf4a8ca0 0x55cfdf4a4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47368/0 (socket says 172.21.15.52:47368) 2024-04-02T13:55:14.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.539+0000 7f5ae2888640 1 -- 172.21.15.52:0/2792452968 learned_addr learned my addr 172.21.15.52:0/2792452968 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.539+0000 7f5ae2087640 1 --2- 172.21.15.52:0/2792452968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cfdf49f7a0 0x55cfdf4b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.539+0000 7f5ae1886640 1 --2- 172.21.15.52:0/2792452968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cfdf4a00c0 0x55cfdf4b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.539+0000 7f5ae2888640 1 -- 172.21.15.52:0/2792452968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cfdf4a00c0 msgr2=0x55cfdf4b52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.539+0000 7f5ae2888640 1 --2- 172.21.15.52:0/2792452968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cfdf4a00c0 0x55cfdf4b52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.539+0000 7f5ae2888640 1 -- 172.21.15.52:0/2792452968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cfdf49f7a0 msgr2=0x55cfdf4b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.539+0000 7f5ae2888640 1 --2- 172.21.15.52:0/2792452968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cfdf49f7a0 0x55cfdf4b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.539+0000 7f5ae2888640 1 -- 172.21.15.52:0/2792452968 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfdf4abae0 con 0x55cfdf4a8ca0 2024-04-02T13:55:14.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.539+0000 7f5ae2888640 1 --2- 172.21.15.52:0/2792452968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf4a8ca0 0x55cfdf4a4ac0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5ad409ac30 tx=0x7f5ad40053e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c86b3424ce05d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.539+0000 7f5ac7fff640 1 -- 172.21.15.52:0/2792452968 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad409ba80 con 0x55cfdf4a8ca0 2024-04-02T13:55:14.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.539+0000 7f5ac7fff640 1 -- 172.21.15.52:0/2792452968 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ad4098ff0 con 0x55cfdf4a8ca0 2024-04-02T13:55:14.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.539+0000 7f5ac7fff640 1 -- 172.21.15.52:0/2792452968 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad409b890 con 0x55cfdf4a8ca0 2024-04-02T13:55:14.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.540+0000 7f5ae6faf740 1 -- 172.21.15.52:0/2792452968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf4a8ca0 msgr2=0x55cfdf4a4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.540+0000 7f5ae6faf740 1 --2- 172.21.15.52:0/2792452968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf4a8ca0 0x55cfdf4a4ac0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5ad409ac30 tx=0x7f5ad40053e0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.540+0000 7f5ae6faf740 1 -- 172.21.15.52:0/2792452968 shutdown_connections 2024-04-02T13:55:14.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.540+0000 7f5ae6faf740 1 --2- 172.21.15.52:0/2792452968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf4a8ca0 0x55cfdf4a4ac0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.540+0000 7f5ae6faf740 1 --2- 172.21.15.52:0/2792452968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cfdf4a00c0 0x55cfdf4b52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.540+0000 7f5ae6faf740 1 --2- 172.21.15.52:0/2792452968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cfdf49f7a0 0x55cfdf4b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.540+0000 7f5ae6faf740 1 -- 172.21.15.52:0/2792452968 >> 172.21.15.52:0/2792452968 conn(0x55cfdf499ed0 msgr2=0x55cfdf4a43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.540+0000 7f5ae6faf740 1 -- 172.21.15.52:0/2792452968 shutdown_connections 2024-04-02T13:55:14.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.540+0000 7f5ae6faf740 1 -- 172.21.15.52:0/2792452968 wait complete. 2024-04-02T13:55:14.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.540+0000 7f5ae6faf740 1 -- start start 2024-04-02T13:55:14.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.540+0000 7f5ae6faf740 1 Processor -- start 2024-04-02T13:55:14.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.540+0000 7f5ae6faf740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cfdf49f7a0 0x55cfdf4a9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.540+0000 7f5ae6faf740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf4a00c0 0x55cfdf4a99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.540+0000 7f5ae6faf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cfdf4a8ca0 0x55cfdf4a9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.540+0000 7f5ae6faf740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc1b2460 con 0x55cfdf4a00c0 2024-04-02T13:55:14.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.540+0000 7f5ae6faf740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc1b1d90 con 0x55cfdf49f7a0 2024-04-02T13:55:14.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.540+0000 7f5ae6faf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc1aefd0 con 0x55cfdf4a8ca0 2024-04-02T13:55:14.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.540+0000 7f5ae6faf740 10 client.0 _ll_register_callbacks cb 0x7f5ad409c750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:14.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.540+0000 7f5ae1886640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf4a00c0 0x55cfdf4a99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.540+0000 7f5ae1886640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf4a00c0 0x55cfdf4a99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47376/0 (socket says 172.21.15.52:47376) 2024-04-02T13:55:14.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.540+0000 7f5ae1886640 1 -- 172.21.15.52:0/142775195 learned_addr learned my addr 172.21.15.52:0/142775195 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.541+0000 7f5ae2087640 1 --2- 172.21.15.52:0/142775195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cfdf49f7a0 0x55cfdf4a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.541+0000 7f5ae2888640 1 --2- 172.21.15.52:0/142775195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cfdf4a8ca0 0x55cfdf4a9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.541+0000 7f5ae1886640 1 -- 172.21.15.52:0/142775195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cfdf4a8ca0 msgr2=0x55cfdf4a9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.541+0000 7f5ae1886640 1 --2- 172.21.15.52:0/142775195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cfdf4a8ca0 0x55cfdf4a9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.541+0000 7f5ae1886640 1 -- 172.21.15.52:0/142775195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cfdf49f7a0 msgr2=0x55cfdf4a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.541+0000 7f5ae1886640 1 --2- 172.21.15.52:0/142775195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cfdf49f7a0 0x55cfdf4a9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.541+0000 7f5ae1886640 1 -- 172.21.15.52:0/142775195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfdf4b0100 con 0x55cfdf4a00c0 2024-04-02T13:55:14.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.541+0000 7f5ae2087640 1 --2- 172.21.15.52:0/142775195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cfdf49f7a0 0x55cfdf4a9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:14.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.541+0000 7f5ae1886640 1 --2- 172.21.15.52:0/142775195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf4a00c0 0x55cfdf4a99d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5acc009930 tx=0x7f5acc02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e9285be70e2aa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.541+0000 7f5ac5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:14.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.541+0000 7f5ac5ffb640 1 -- 172.21.15.52:0/142775195 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc053070 con 0x55cfdf4a00c0 2024-04-02T13:55:14.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.541+0000 7f5ac5ffb640 1 -- 172.21.15.52:0/142775195 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f5acc005150 con 0x55cfdf4a00c0 2024-04-02T13:55:14.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.541+0000 7f5ae6faf740 1 -- 172.21.15.52:0/142775195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ad4098ff0 con 0x55cfdf4a00c0 2024-04-02T13:55:14.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.541+0000 7f5ac5ffb640 1 -- 172.21.15.52:0/142775195 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5acc05f050 con 0x55cfdf4a00c0 2024-04-02T13:55:14.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.541+0000 7f5ae6faf740 10 client.16152 fetch_fsmap learned FSMap version 28 2024-04-02T13:55:14.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.541+0000 7f5ae6faf740 1 -- 172.21.15.52:0/142775195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cfdf4b1400 con 0x55cfdf4a00c0 2024-04-02T13:55:14.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.541+0000 7f5ac5ffb640 1 -- 172.21.15.52:0/142775195 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ad4098ff0 con 0x55cfdf4a00c0 2024-04-02T13:55:14.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.541+0000 7f5ac5ffb640 1 -- 172.21.15.52:0/142775195 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5acc003c70 con 0x55cfdf4a00c0 2024-04-02T13:55:14.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.541+0000 7f5ae6faf740 10 client.16152 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:55:14.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.541+0000 7f5ae6faf740 10 client.16152 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T13:55:14.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.541+0000 7f5ae6faf740 20 client.16152 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:14.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.542+0000 7f5ae6faf740 1 --2- 172.21.15.52:0/142775195 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55cfdf51f880 0x55cfdf521ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.542+0000 7f5ae6faf740 4 client.16152 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:55:14.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.542+0000 7f5ae6faf740 1 -- 172.21.15.52:0/142775195 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cfdf40b040 con 0x55cfdf51f880 2024-04-02T13:55:14.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.542+0000 7f5ae2087640 1 --2- 172.21.15.52:0/142775195 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55cfdf51f880 0x55cfdf521ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.542+0000 7f5ae2087640 1 --2- 172.21.15.52:0/142775195 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55cfdf51f880 0x55cfdf521ca0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c448edec8ed825b1 server_cookie=54a200eadd4de257 in_seq=0 out_seq=0 2024-04-02T13:55:14.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.542+0000 7f5ac5ffb640 10 client.16152 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:55:14.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.542+0000 7f5ac5ffb640 1 -- 172.21.15.52:0/142775195 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cfdf40b040 con 0x55cfdf51f880 2024-04-02T13:55:14.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.542+0000 7f5ac5ffb640 10 client.16152 handle_command_reply: tid=0 2024-04-02T13:55:14.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.542+0000 7f5ae6faf740 1 client.16152 shutdown 2024-04-02T13:55:14.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.542+0000 7f5ae6faf740 1 -- 172.21.15.52:0/142775195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf4a00c0 msgr2=0x55cfdf4a99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.542+0000 7f5ae6faf740 1 --2- 172.21.15.52:0/142775195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf4a00c0 0x55cfdf4a99d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5acc009930 tx=0x7f5acc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:55:14.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.542+0000 7f5ae6faf740 1 -- 172.21.15.52:0/142775195 shutdown_connections 2024-04-02T13:55:14.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.542+0000 7f5ae6faf740 1 --2- 172.21.15.52:0/142775195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cfdf4a8ca0 0x55cfdf4a9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.542+0000 7f5ae6faf740 1 --2- 172.21.15.52:0/142775195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfdf4a00c0 0x55cfdf4a99d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.542+0000 7f5ae6faf740 1 --2- 172.21.15.52:0/142775195 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55cfdf51f880 0x55cfdf521ca0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.542+0000 7f5ae6faf740 1 --2- 172.21.15.52:0/142775195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cfdf49f7a0 0x55cfdf4a9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.542+0000 7f5ac5ffb640 0 client.16152 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:55:14.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.542+0000 7f5ae6faf740 1 -- 172.21.15.52:0/142775195 >> 172.21.15.52:0/142775195 conn(0x55cfdf499ed0 msgr2=0x55cfdf498f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.543+0000 7f5ae6faf740 1 -- 172.21.15.52:0/142775195 shutdown_connections 2024-04-02T13:55:14.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.543+0000 7f5ae6faf740 1 -- 172.21.15.52:0/142775195 wait complete. 2024-04-02T13:55:14.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.543+0000 7f5ae6faf740 20 client.16152 trim_cache size 0 max 16384 2024-04-02T13:55:14.553 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:55:14.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.543+0000 7f5ae42eb640 1 -- 172.21.15.52:0/306518787 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5ac80a3da0 msgr2=0x7f5ac80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.543+0000 7f5ae42eb640 1 --2- 172.21.15.52:0/306518787 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5ac80a3da0 0x7f5ac80a6260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5acc005ab0 tx=0x7f5acc009560 comp rx=0 tx=0).stop 2024-04-02T13:55:14.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.543+0000 7f5ae42eb640 1 -- 172.21.15.52:0/306518787 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5adc106980 msgr2=0x7f5adc103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.543+0000 7f5ae42eb640 1 --2- 172.21.15.52:0/306518787 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5adc106980 0x7f5adc103920 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400eaa0 tx=0x7f5ad400ef60 comp rx=0 tx=0).stop 2024-04-02T13:55:14.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.543+0000 7f5ae42eb640 1 -- 172.21.15.52:0/306518787 shutdown_connections 2024-04-02T13:55:14.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.543+0000 7f5ae42eb640 1 --2- 172.21.15.52:0/306518787 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5ac80a3da0 0x7f5ac80a6260 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.543+0000 7f5ae42eb640 1 --2- 172.21.15.52:0/306518787 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5adc106980 0x7f5adc103920 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.543+0000 7f5ae42eb640 1 --2- 172.21.15.52:0/306518787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5adc101330 0x7f5adc1033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.543+0000 7f5ae42eb640 1 --2- 172.21.15.52:0/306518787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5adc100970 0x7f5adc102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.543+0000 7f5ae42eb640 1 -- 172.21.15.52:0/306518787 >> 172.21.15.52:0/306518787 conn(0x7f5adc077b00 msgr2=0x7f5adc068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.543+0000 7f5ae42eb640 1 -- 172.21.15.52:0/306518787 shutdown_connections 2024-04-02T13:55:14.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.543+0000 7f5ae42eb640 1 -- 172.21.15.52:0/306518787 wait complete. 2024-04-02T13:55:14.556 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:14 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2370771187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:55:14.557 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:55:14.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.659+0000 7f4bc1c63640 1 -- 172.21.15.52:0/2381477422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bbc107a80 msgr2=0x7f4bbc107e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.659+0000 7f4bc1c63640 1 --2- 172.21.15.52:0/2381477422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bbc107a80 0x7f4bbc107e60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009930 tx=0x7f4bac02bc30 comp rx=0 tx=0).stop 2024-04-02T13:55:14.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.659+0000 7f4bc1c63640 1 -- 172.21.15.52:0/2381477422 shutdown_connections 2024-04-02T13:55:14.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.659+0000 7f4bc1c63640 1 --2- 172.21.15.52:0/2381477422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bbc0696a0 0x7f4bbc10e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.659+0000 7f4bc1c63640 1 --2- 172.21.15.52:0/2381477422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bbc068ce0 0x7f4bbc069160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.659+0000 7f4bc1c63640 1 --2- 172.21.15.52:0/2381477422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bbc107a80 0x7f4bbc107e60 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.659+0000 7f4bc1c63640 1 -- 172.21.15.52:0/2381477422 >> 172.21.15.52:0/2381477422 conn(0x7f4bbc077b00 msgr2=0x7f4bbc077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.659+0000 7f4bc1c63640 1 -- 172.21.15.52:0/2381477422 shutdown_connections 2024-04-02T13:55:14.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.659+0000 7f4bc1c63640 1 -- 172.21.15.52:0/2381477422 wait complete. 2024-04-02T13:55:14.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.659+0000 7f4bc1c63640 1 Processor -- start 2024-04-02T13:55:14.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.660+0000 7f4bc1c63640 1 -- start start 2024-04-02T13:55:14.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.660+0000 7f4bc1c63640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bbc068ce0 0x7f4bbc1009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.660+0000 7f4bc1c63640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bbc0696a0 0x7f4bbc100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.660+0000 7f4bc1c63640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bbc107a80 0x7f4bbc101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.660+0000 7f4bc1c63640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc111e60 con 0x7f4bbc0696a0 2024-04-02T13:55:14.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.660+0000 7f4bc1c63640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc111ce0 con 0x7f4bbc068ce0 2024-04-02T13:55:14.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.660+0000 7f4bc1c63640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc111fe0 con 0x7f4bbc107a80 2024-04-02T13:55:14.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.660+0000 7f4bbaffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bbc0696a0 0x7f4bbc100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.660+0000 7f4bbaffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bbc0696a0 0x7f4bbc100f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47388/0 (socket says 172.21.15.52:47388) 2024-04-02T13:55:14.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.660+0000 7f4bbaffd640 1 -- 172.21.15.52:0/3138053133 learned_addr learned my addr 172.21.15.52:0/3138053133 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.660+0000 7f4bbb7fe640 1 --2- 172.21.15.52:0/3138053133 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bbc068ce0 0x7f4bbc1009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.660+0000 7f4bbbfff640 1 --2- 172.21.15.52:0/3138053133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bbc107a80 0x7f4bbc101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.660+0000 7f4bbaffd640 1 -- 172.21.15.52:0/3138053133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bbc107a80 msgr2=0x7f4bbc101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.660+0000 7f4bbaffd640 1 --2- 172.21.15.52:0/3138053133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bbc107a80 0x7f4bbc101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.660+0000 7f4bbaffd640 1 -- 172.21.15.52:0/3138053133 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bbc068ce0 msgr2=0x7f4bbc1009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.660+0000 7f4bbaffd640 1 --2- 172.21.15.52:0/3138053133 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bbc068ce0 0x7f4bbc1009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.660+0000 7f4bbaffd640 1 -- 172.21.15.52:0/3138053133 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bbc1a6880 con 0x7f4bbc0696a0 2024-04-02T13:55:14.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.660+0000 7f4bbaffd640 1 --2- 172.21.15.52:0/3138053133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bbc0696a0 0x7f4bbc100f00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000d950 tx=0x7f4bb000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.661+0000 7f4bb8ff9640 1 -- 172.21.15.52:0/3138053133 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0017070 con 0x7f4bbc0696a0 2024-04-02T13:55:14.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.661+0000 7f4bc1c63640 1 -- 172.21.15.52:0/3138053133 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bbc1a6b70 con 0x7f4bbc0696a0 2024-04-02T13:55:14.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.661+0000 7f4bb8ff9640 1 -- 172.21.15.52:0/3138053133 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bb0009c30 con 0x7f4bbc0696a0 2024-04-02T13:55:14.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.661+0000 7f4bb8ff9640 1 -- 172.21.15.52:0/3138053133 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb000f6b0 con 0x7f4bbc0696a0 2024-04-02T13:55:14.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.661+0000 7f4bc1c63640 1 -- 172.21.15.52:0/3138053133 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bbc1a7110 con 0x7f4bbc0696a0 2024-04-02T13:55:14.673 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:14 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2370771187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:55:14.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.663+0000 7f4bb8ff9640 1 -- 172.21.15.52:0/3138053133 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4bb00109d0 con 0x7f4bbc0696a0 2024-04-02T13:55:14.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.663+0000 7f4bc4850740 1 Processor -- start 2024-04-02T13:55:14.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.663+0000 7f4bc4850740 1 -- start start 2024-04-02T13:55:14.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.663+0000 7f4bc4850740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d64bc7a0 0x5564d64bcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.663+0000 7f4bc4850740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64bd0c0 0x5564d64c5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.663+0000 7f4bb8ff9640 1 --2- 172.21.15.52:0/3138053133 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4ba40a3d20 0x7f4ba40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.663+0000 7f4bc4850740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564d64c5ca0 0x5564d64c8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.663+0000 7f4bc4850740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5564d64207b0 con 0x5564d64bd0c0 2024-04-02T13:55:14.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.663+0000 7f4bc4850740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5564d63bf570 con 0x5564d64c5ca0 2024-04-02T13:55:14.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.663+0000 7f4bc4850740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5564d641c760 con 0x5564d64bc7a0 2024-04-02T13:55:14.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.664+0000 7f4bbaffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64bd0c0 0x5564d64c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.664+0000 7f4bbb7fe640 1 --2- 172.21.15.52:0/3138053133 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4ba40a3d20 0x7f4ba40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.664+0000 7f4bbaffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64bd0c0 0x5564d64c5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47404/0 (socket says 172.21.15.52:47404) 2024-04-02T13:55:14.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.664+0000 7f4bbaffd640 1 -- 172.21.15.52:0/3060318422 learned_addr learned my addr 172.21.15.52:0/3060318422 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.664+0000 7f4bbbfff640 1 --2- 172.21.15.52:0/3060318422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564d64c5ca0 0x5564d64c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.664+0000 7f4bb8ff9640 1 -- 172.21.15.52:0/3138053133 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f4bb00d07b0 con 0x7f4bbc0696a0 2024-04-02T13:55:14.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.664+0000 7f4bbbfff640 1 --2- 172.21.15.52:0/3060318422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d64bc7a0 0x5564d64bcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.664+0000 7f4bbaffd640 1 -- 172.21.15.52:0/3060318422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d64bc7a0 msgr2=0x5564d64bcb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.664+0000 7f4bbaffd640 1 --2- 172.21.15.52:0/3060318422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d64bc7a0 0x5564d64bcb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.664+0000 7f4bbaffd640 1 -- 172.21.15.52:0/3060318422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564d64c5ca0 msgr2=0x5564d64c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.664+0000 7f4bbaffd640 1 --2- 172.21.15.52:0/3060318422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564d64c5ca0 0x5564d64c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.664+0000 7f4bbaffd640 1 -- 172.21.15.52:0/3060318422 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564d61343f0 con 0x5564d64bd0c0 2024-04-02T13:55:14.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.664+0000 7f4bbb7fe640 1 --2- 172.21.15.52:0/3138053133 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4ba40a3d20 0x7f4ba40a61e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4bac005ab0 tx=0x7f4bac005fd0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.664+0000 7f4bbaffd640 1 --2- 172.21.15.52:0/3060318422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64bd0c0 0x5564d64c5760 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0098d80 tx=0x7f4bb0010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ecd8ec417a4c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.664+0000 7f4ba1ffb640 1 -- 172.21.15.52:0/3060318422 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0020070 con 0x5564d64bd0c0 2024-04-02T13:55:14.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.664+0000 7f4ba1ffb640 1 -- 172.21.15.52:0/3060318422 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bb009cab0 con 0x5564d64bd0c0 2024-04-02T13:55:14.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.664+0000 7f4ba1ffb640 1 -- 172.21.15.52:0/3060318422 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb00a5020 con 0x5564d64bd0c0 2024-04-02T13:55:14.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.665+0000 7f4bc4850740 1 -- 172.21.15.52:0/3060318422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64bd0c0 msgr2=0x5564d64c5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.665+0000 7f4bc4850740 1 --2- 172.21.15.52:0/3060318422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64bd0c0 0x5564d64c5760 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0098d80 tx=0x7f4bb0010d70 comp rx=0 tx=0).stop 2024-04-02T13:55:14.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.665+0000 7f4bc4850740 1 -- 172.21.15.52:0/3060318422 shutdown_connections 2024-04-02T13:55:14.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.665+0000 7f4bc4850740 1 --2- 172.21.15.52:0/3060318422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564d64c5ca0 0x5564d64c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.665+0000 7f4bc4850740 1 --2- 172.21.15.52:0/3060318422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64bd0c0 0x5564d64c5760 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.665+0000 7f4bc4850740 1 --2- 172.21.15.52:0/3060318422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d64bc7a0 0x5564d64bcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.665+0000 7f4bc4850740 1 -- 172.21.15.52:0/3060318422 >> 172.21.15.52:0/3060318422 conn(0x5564d64b6ed0 msgr2=0x5564d64b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.665+0000 7f4bc4850740 1 -- 172.21.15.52:0/3060318422 shutdown_connections 2024-04-02T13:55:14.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.665+0000 7f4bc4850740 1 -- 172.21.15.52:0/3060318422 wait complete. 2024-04-02T13:55:14.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.665+0000 7f4bc4850740 1 -- start start 2024-04-02T13:55:14.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.666+0000 7f4bc4850740 1 Processor -- start 2024-04-02T13:55:14.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.666+0000 7f4bc4850740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564d64bc7a0 0x5564d64c56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.666+0000 7f4bc4850740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64bd0c0 0x5564d64c2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.666+0000 7f4bc4850740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d64c5ca0 0x5564d64c3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.666+0000 7f4bc4850740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5564d64c89d0 con 0x5564d64bd0c0 2024-04-02T13:55:14.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.666+0000 7f4bc4850740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5564d63dbe80 con 0x5564d64bc7a0 2024-04-02T13:55:14.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.666+0000 7f4bc4850740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5564d64c8b50 con 0x5564d64c5ca0 2024-04-02T13:55:14.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.666+0000 7f4bc4850740 10 client.0 _ll_register_callbacks cb 0x5564d6417aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:14.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.666+0000 7f4bbaffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64bd0c0 0x5564d64c2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.666+0000 7f4bbaffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64bd0c0 0x5564d64c2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47416/0 (socket says 172.21.15.52:47416) 2024-04-02T13:55:14.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.666+0000 7f4bbaffd640 1 -- 172.21.15.52:0/3690778013 learned_addr learned my addr 172.21.15.52:0/3690778013 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.666+0000 7f4bbb7fe640 1 --2- 172.21.15.52:0/3690778013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564d64bc7a0 0x5564d64c56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.666+0000 7f4bbaffd640 1 -- 172.21.15.52:0/3690778013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d64c5ca0 msgr2=0x5564d64c3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.666+0000 7f4bbaffd640 1 --2- 172.21.15.52:0/3690778013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d64c5ca0 0x5564d64c3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.666+0000 7f4bbaffd640 1 -- 172.21.15.52:0/3690778013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564d64bc7a0 msgr2=0x5564d64c56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.666+0000 7f4bbaffd640 1 --2- 172.21.15.52:0/3690778013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564d64bc7a0 0x5564d64c56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.666+0000 7f4bbaffd640 1 -- 172.21.15.52:0/3690778013 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564d61fe7d0 con 0x5564d64bd0c0 2024-04-02T13:55:14.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.666+0000 7f4bbb7fe640 1 --2- 172.21.15.52:0/3690778013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564d64bc7a0 0x5564d64c56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:14.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.666+0000 7f4bbaffd640 1 --2- 172.21.15.52:0/3690778013 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64bd0c0 0x5564d64c2a50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00058c0 tx=0x7f4bb0065120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1dc0d7304872ac server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.666+0000 7f4b9affd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:14.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.666+0000 7f4b9affd640 1 -- 172.21.15.52:0/3690778013 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0020070 con 0x5564d64bd0c0 2024-04-02T13:55:14.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.667+0000 7f4b9affd640 1 -- 172.21.15.52:0/3690778013 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f4bb00cdde0 con 0x5564d64bd0c0 2024-04-02T13:55:14.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.667+0000 7f4bc4850740 1 -- 172.21.15.52:0/3690778013 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bb009cab0 con 0x5564d64bd0c0 2024-04-02T13:55:14.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.667+0000 7f4b9affd640 1 -- 172.21.15.52:0/3690778013 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4bb00cdfe0 con 0x5564d64bd0c0 2024-04-02T13:55:14.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.667+0000 7f4bc4850740 10 client.16164 fetch_fsmap learned FSMap version 28 2024-04-02T13:55:14.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.667+0000 7f4bc4850740 1 -- 172.21.15.52:0/3690778013 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5564d64d4a60 con 0x5564d64bd0c0 2024-04-02T13:55:14.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.667+0000 7f4b9affd640 1 -- 172.21.15.52:0/3690778013 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bb009cab0 con 0x5564d64bd0c0 2024-04-02T13:55:14.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.667+0000 7f4b9affd640 1 -- 172.21.15.52:0/3690778013 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4bb009b800 con 0x5564d64bd0c0 2024-04-02T13:55:14.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.667+0000 7f4bc4850740 10 client.16164 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:55:14.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.667+0000 7f4bc4850740 10 client.16164 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T13:55:14.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.667+0000 7f4bc4850740 20 client.16164 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:14.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.667+0000 7f4bc4850740 1 --2- 172.21.15.52:0/3690778013 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5564d64d5db0 0x5564d64d81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.667+0000 7f4bc4850740 4 client.16164 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:14.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.667+0000 7f4bc4850740 1 -- 172.21.15.52:0/3690778013 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564d642dcf0 con 0x5564d64d5db0 2024-04-02T13:55:14.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.667+0000 7f4bbb7fe640 1 --2- 172.21.15.52:0/3690778013 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5564d64d5db0 0x5564d64d81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.668+0000 7f4b9affd640 10 client.16164 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:55:14.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.668+0000 7f4bbb7fe640 1 --2- 172.21.15.52:0/3690778013 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5564d64d5db0 0x5564d64d81a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f8b2ec4172ad6c8 server_cookie=2ce0d28504b79ac7 in_seq=0 out_seq=0 2024-04-02T13:55:14.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.669+0000 7f4b9affd640 1 -- 172.21.15.52:0/3690778013 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5564d642dcf0 con 0x5564d64d5db0 2024-04-02T13:55:14.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.669+0000 7f4b9affd640 10 client.16164 handle_command_reply: tid=0 2024-04-02T13:55:14.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.669+0000 7f4bc4850740 1 client.16164 shutdown 2024-04-02T13:55:14.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.669+0000 7f4bc4850740 1 -- 172.21.15.52:0/3690778013 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64bd0c0 msgr2=0x5564d64c2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.669+0000 7f4bc4850740 1 --2- 172.21.15.52:0/3690778013 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64bd0c0 0x5564d64c2a50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00058c0 tx=0x7f4bb0065120 comp rx=0 tx=0).stop 2024-04-02T13:55:14.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.669+0000 7f4bc4850740 1 -- 172.21.15.52:0/3690778013 shutdown_connections 2024-04-02T13:55:14.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.669+0000 7f4bc4850740 1 --2- 172.21.15.52:0/3690778013 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5564d64d5db0 0x5564d64d81a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.669+0000 7f4bc4850740 1 --2- 172.21.15.52:0/3690778013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d64c5ca0 0x5564d64c3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.669+0000 7f4bc4850740 1 --2- 172.21.15.52:0/3690778013 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64bd0c0 0x5564d64c2a50 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.669+0000 7f4bc4850740 1 --2- 172.21.15.52:0/3690778013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564d64bc7a0 0x5564d64c56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.669+0000 7f4b9affd640 0 client.16164 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:55:14.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.669+0000 7f4bc4850740 1 -- 172.21.15.52:0/3690778013 >> 172.21.15.52:0/3690778013 conn(0x5564d64b6ed0 msgr2=0x5564d64b8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.669+0000 7f4bc4850740 1 -- 172.21.15.52:0/3690778013 shutdown_connections 2024-04-02T13:55:14.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.669+0000 7f4bc4850740 1 -- 172.21.15.52:0/3690778013 wait complete. 2024-04-02T13:55:14.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.669+0000 7f4bc4850740 20 client.16164 trim_cache size 0 max 16384 2024-04-02T13:55:14.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.682+0000 7f4bc4850740 1 Processor -- start 2024-04-02T13:55:14.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.682+0000 7f4bc4850740 1 -- start start 2024-04-02T13:55:14.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.682+0000 7f4bc4850740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d64bc7a0 0x5564d64d18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.682+0000 7f4bc4850740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564d64bd0c0 0x5564d64b2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.682+0000 7f4bc4850740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64c5ca0 0x5564d64b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.682+0000 7f4bc4850740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1b0fa0 con 0x5564d64c5ca0 2024-04-02T13:55:14.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.682+0000 7f4bc4850740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1b2ae0 con 0x5564d64bd0c0 2024-04-02T13:55:14.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.682+0000 7f4bc4850740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1b1670 con 0x5564d64bc7a0 2024-04-02T13:55:14.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.682+0000 7f4bbbfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64c5ca0 0x5564d64b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.682+0000 7f4bbbfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64c5ca0 0x5564d64b72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47428/0 (socket says 172.21.15.52:47428) 2024-04-02T13:55:14.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.682+0000 7f4bbbfff640 1 -- 172.21.15.52:0/851248714 learned_addr learned my addr 172.21.15.52:0/851248714 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.683+0000 7f4bbbfff640 1 -- 172.21.15.52:0/851248714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d64bc7a0 msgr2=0x5564d64d18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.683+0000 7f4bbbfff640 1 --2- 172.21.15.52:0/851248714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d64bc7a0 0x5564d64d18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.683+0000 7f4bbaffd640 1 --2- 172.21.15.52:0/851248714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564d64bd0c0 0x5564d64b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.683+0000 7f4bbbfff640 1 -- 172.21.15.52:0/851248714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564d64bd0c0 msgr2=0x5564d64b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.683+0000 7f4bbbfff640 1 --2- 172.21.15.52:0/851248714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564d64bd0c0 0x5564d64b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.683+0000 7f4bbbfff640 1 -- 172.21.15.52:0/851248714 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564d64be630 con 0x5564d64c5ca0 2024-04-02T13:55:14.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.683+0000 7f4bbaffd640 1 --2- 172.21.15.52:0/851248714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564d64bd0c0 0x5564d64b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:14.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.683+0000 7f4bbbfff640 1 --2- 172.21.15.52:0/851248714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64c5ca0 0x5564d64b72b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4007fc0 tx=0x7f4bb4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ed40de28c70566 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.683+0000 7f4ba17fa640 1 -- 172.21.15.52:0/851248714 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4004b20 con 0x5564d64c5ca0 2024-04-02T13:55:14.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.683+0000 7f4ba17fa640 1 -- 172.21.15.52:0/851248714 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bb4004cc0 con 0x5564d64c5ca0 2024-04-02T13:55:14.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.683+0000 7f4ba17fa640 1 -- 172.21.15.52:0/851248714 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4005680 con 0x5564d64c5ca0 2024-04-02T13:55:14.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.683+0000 7f4bc4850740 1 -- 172.21.15.52:0/851248714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64c5ca0 msgr2=0x5564d64b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.683+0000 7f4bc4850740 1 --2- 172.21.15.52:0/851248714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64c5ca0 0x5564d64b72b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4007fc0 tx=0x7f4bb4011810 comp rx=0 tx=0).stop 2024-04-02T13:55:14.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.684+0000 7f4bc4850740 1 -- 172.21.15.52:0/851248714 shutdown_connections 2024-04-02T13:55:14.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.684+0000 7f4bc4850740 1 --2- 172.21.15.52:0/851248714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64c5ca0 0x5564d64b72b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.684+0000 7f4bc4850740 1 --2- 172.21.15.52:0/851248714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564d64bd0c0 0x5564d64b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.684+0000 7f4bc4850740 1 --2- 172.21.15.52:0/851248714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d64bc7a0 0x5564d64d18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.684+0000 7f4bc4850740 1 -- 172.21.15.52:0/851248714 >> 172.21.15.52:0/851248714 conn(0x5564d64b6ed0 msgr2=0x5564d64c22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.684+0000 7f4bc4850740 1 -- 172.21.15.52:0/851248714 shutdown_connections 2024-04-02T13:55:14.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.684+0000 7f4bc4850740 1 -- 172.21.15.52:0/851248714 wait complete. 2024-04-02T13:55:14.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.684+0000 7f4bc4850740 1 -- start start 2024-04-02T13:55:14.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.684+0000 7f4bc4850740 1 Processor -- start 2024-04-02T13:55:14.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.684+0000 7f4bc4850740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64bc7a0 0x5564d64ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.684+0000 7f4bc4850740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564d64bd0c0 0x5564d64bfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.684+0000 7f4bc4850740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d64c5ca0 0x5564d64c02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.684+0000 7f4bc4850740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1b2410 con 0x5564d64bc7a0 2024-04-02T13:55:14.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.684+0000 7f4bc4850740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1b1d40 con 0x5564d64bd0c0 2024-04-02T13:55:14.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.684+0000 7f4bc4850740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1aef80 con 0x5564d64c5ca0 2024-04-02T13:55:14.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.684+0000 7f4bc4850740 10 client.0 _ll_register_callbacks cb 0x7f4bb4007760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:14.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.684+0000 7f4bbb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64bc7a0 0x5564d64ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.684+0000 7f4bbb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64bc7a0 0x5564d64ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47430/0 (socket says 172.21.15.52:47430) 2024-04-02T13:55:14.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.684+0000 7f4bbb7fe640 1 -- 172.21.15.52:0/4267497768 learned_addr learned my addr 172.21.15.52:0/4267497768 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.685+0000 7f4bbbfff640 1 --2- 172.21.15.52:0/4267497768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d64c5ca0 0x5564d64c02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.685+0000 7f4bbaffd640 1 --2- 172.21.15.52:0/4267497768 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564d64bd0c0 0x5564d64bfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.685+0000 7f4bbb7fe640 1 -- 172.21.15.52:0/4267497768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d64c5ca0 msgr2=0x5564d64c02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.685+0000 7f4bbb7fe640 1 --2- 172.21.15.52:0/4267497768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d64c5ca0 0x5564d64c02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.685+0000 7f4bbb7fe640 1 -- 172.21.15.52:0/4267497768 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564d64bd0c0 msgr2=0x5564d64bfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.685+0000 7f4bbb7fe640 1 --2- 172.21.15.52:0/4267497768 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564d64bd0c0 0x5564d64bfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.685+0000 7f4bbb7fe640 1 -- 172.21.15.52:0/4267497768 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564d64c09b0 con 0x5564d64bc7a0 2024-04-02T13:55:14.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.685+0000 7f4bbaffd640 1 --2- 172.21.15.52:0/4267497768 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564d64bd0c0 0x5564d64bfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:14.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.685+0000 7f4bbb7fe640 1 --2- 172.21.15.52:0/4267497768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64bc7a0 0x5564d64ce0d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009900 tx=0x7f4bac009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1b92a692054dbd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.685+0000 7f4b9affd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:14.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.685+0000 7f4b9affd640 1 -- 172.21.15.52:0/4267497768 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac057060 con 0x5564d64bc7a0 2024-04-02T13:55:14.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.685+0000 7f4b9affd640 1 -- 172.21.15.52:0/4267497768 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f4bac004780 con 0x5564d64bc7a0 2024-04-02T13:55:14.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.685+0000 7f4bc4850740 1 -- 172.21.15.52:0/4267497768 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bb4004cc0 con 0x5564d64bc7a0 2024-04-02T13:55:14.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.685+0000 7f4b9affd640 1 -- 172.21.15.52:0/4267497768 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4bac05e050 con 0x5564d64bc7a0 2024-04-02T13:55:14.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.685+0000 7f4bc4850740 10 client.16170 fetch_fsmap learned FSMap version 28 2024-04-02T13:55:14.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.685+0000 7f4bc4850740 1 -- 172.21.15.52:0/4267497768 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5564d653c100 con 0x5564d64bc7a0 2024-04-02T13:55:14.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.685+0000 7f4b9affd640 1 -- 172.21.15.52:0/4267497768 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bb4004cc0 con 0x5564d64bc7a0 2024-04-02T13:55:14.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.686+0000 7f4b9affd640 1 -- 172.21.15.52:0/4267497768 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4bac0035f0 con 0x5564d64bc7a0 2024-04-02T13:55:14.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.686+0000 7f4bc4850740 10 client.16170 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:55:14.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.686+0000 7f4bc4850740 10 client.16170 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T13:55:14.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.686+0000 7f4bc4850740 20 client.16170 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:14.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.686+0000 7f4bc4850740 1 --2- 172.21.15.52:0/4267497768 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5564d653c750 0x5564d653eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.686+0000 7f4bc4850740 4 client.16170 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:55:14.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.686+0000 7f4bc4850740 1 -- 172.21.15.52:0/4267497768 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5564d6428040 con 0x5564d653c750 2024-04-02T13:55:14.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.686+0000 7f4bbaffd640 1 --2- 172.21.15.52:0/4267497768 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5564d653c750 0x5564d653eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.686+0000 7f4b9affd640 10 client.16170 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:55:14.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.686+0000 7f4bbaffd640 1 --2- 172.21.15.52:0/4267497768 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5564d653c750 0x5564d653eb70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=44c5c67a95bd5c0d server_cookie=77b77f7e814a591c in_seq=0 out_seq=0 2024-04-02T13:55:14.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.687+0000 7f4b9affd640 1 -- 172.21.15.52:0/4267497768 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5564d6428040 con 0x5564d653c750 2024-04-02T13:55:14.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.687+0000 7f4b9affd640 10 client.16170 handle_command_reply: tid=0 2024-04-02T13:55:14.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.687+0000 7f4bc4850740 1 client.16170 shutdown 2024-04-02T13:55:14.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.687+0000 7f4bc4850740 1 -- 172.21.15.52:0/4267497768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64bc7a0 msgr2=0x5564d64ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.687+0000 7f4bc4850740 1 --2- 172.21.15.52:0/4267497768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64bc7a0 0x5564d64ce0d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009900 tx=0x7f4bac009260 comp rx=0 tx=0).stop 2024-04-02T13:55:14.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.687+0000 7f4bc4850740 1 -- 172.21.15.52:0/4267497768 shutdown_connections 2024-04-02T13:55:14.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.687+0000 7f4bc4850740 1 --2- 172.21.15.52:0/4267497768 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5564d653c750 0x5564d653eb70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.687+0000 7f4bc4850740 1 --2- 172.21.15.52:0/4267497768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564d64c5ca0 0x5564d64c02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.687+0000 7f4bc4850740 1 --2- 172.21.15.52:0/4267497768 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564d64bd0c0 0x5564d64bfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.687+0000 7f4b9affd640 0 client.16170 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:55:14.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.687+0000 7f4bc4850740 1 --2- 172.21.15.52:0/4267497768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564d64bc7a0 0x5564d64ce0d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.687+0000 7f4bc4850740 1 -- 172.21.15.52:0/4267497768 >> 172.21.15.52:0/4267497768 conn(0x5564d64b6ed0 msgr2=0x5564d64b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.687+0000 7f4bc4850740 1 -- 172.21.15.52:0/4267497768 shutdown_connections 2024-04-02T13:55:14.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.687+0000 7f4bc4850740 1 -- 172.21.15.52:0/4267497768 wait complete. 2024-04-02T13:55:14.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.687+0000 7f4bc4850740 20 client.16170 trim_cache size 0 max 16384 2024-04-02T13:55:14.698 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:55:14.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.687+0000 7f4bc1c63640 1 -- 172.21.15.52:0/3138053133 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4ba40a3d20 msgr2=0x7f4ba40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.687+0000 7f4bc1c63640 1 --2- 172.21.15.52:0/3138053133 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4ba40a3d20 0x7f4ba40a61e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4bac005ab0 tx=0x7f4bac005fd0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.688+0000 7f4bc1c63640 1 -- 172.21.15.52:0/3138053133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bbc0696a0 msgr2=0x7f4bbc100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.688+0000 7f4bc1c63640 1 --2- 172.21.15.52:0/3138053133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bbc0696a0 0x7f4bbc100f00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000d950 tx=0x7f4bb000de10 comp rx=0 tx=0).stop 2024-04-02T13:55:14.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.688+0000 7f4bc1c63640 1 -- 172.21.15.52:0/3138053133 shutdown_connections 2024-04-02T13:55:14.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.688+0000 7f4bc1c63640 1 --2- 172.21.15.52:0/3138053133 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4ba40a3d20 0x7f4ba40a61e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.688+0000 7f4bc1c63640 1 --2- 172.21.15.52:0/3138053133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bbc107a80 0x7f4bbc101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.688+0000 7f4bc1c63640 1 --2- 172.21.15.52:0/3138053133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bbc0696a0 0x7f4bbc100f00 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.688+0000 7f4bc1c63640 1 --2- 172.21.15.52:0/3138053133 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bbc068ce0 0x7f4bbc1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.688+0000 7f4bc1c63640 1 -- 172.21.15.52:0/3138053133 >> 172.21.15.52:0/3138053133 conn(0x7f4bbc077b00 msgr2=0x7f4bbc0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.688+0000 7f4bc1c63640 1 -- 172.21.15.52:0/3138053133 shutdown_connections 2024-04-02T13:55:14.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.688+0000 7f4bc1c63640 1 -- 172.21.15.52:0/3138053133 wait complete. 2024-04-02T13:55:14.702 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:55:14.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:14 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2370771187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:55:14.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.804+0000 7fe263596640 1 -- 172.21.15.52:0/1830202456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe25c106980 msgr2=0x7fe25c106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.804+0000 7fe263596640 1 --2- 172.21.15.52:0/1830202456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe25c106980 0x7fe25c106d60 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe254009930 tx=0x7fe25402bc30 comp rx=0 tx=0).stop 2024-04-02T13:55:14.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.804+0000 7fe263596640 1 -- 172.21.15.52:0/1830202456 shutdown_connections 2024-04-02T13:55:14.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.804+0000 7fe263596640 1 --2- 172.21.15.52:0/1830202456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe25c101330 0x7fe25c10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.804+0000 7fe263596640 1 --2- 172.21.15.52:0/1830202456 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe25c100970 0x7fe25c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.804+0000 7fe263596640 1 --2- 172.21.15.52:0/1830202456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe25c106980 0x7fe25c106d60 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.804+0000 7fe263596640 1 -- 172.21.15.52:0/1830202456 >> 172.21.15.52:0/1830202456 conn(0x7fe25c077b00 msgr2=0x7fe25c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.804+0000 7fe263596640 1 -- 172.21.15.52:0/1830202456 shutdown_connections 2024-04-02T13:55:14.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.804+0000 7fe263596640 1 -- 172.21.15.52:0/1830202456 wait complete. 2024-04-02T13:55:14.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.804+0000 7fe263596640 1 Processor -- start 2024-04-02T13:55:14.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.804+0000 7fe263596640 1 -- start start 2024-04-02T13:55:14.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.805+0000 7fe263596640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe25c100970 0x7fe25c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.805+0000 7fe263596640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe25c101330 0x7fe25c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.805+0000 7fe263596640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe25c106980 0x7fe25c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.805+0000 7fe263596640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c111e90 con 0x7fe25c101330 2024-04-02T13:55:14.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.805+0000 7fe263596640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c111d10 con 0x7fe25c106980 2024-04-02T13:55:14.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.805+0000 7fe263596640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c112010 con 0x7fe25c100970 2024-04-02T13:55:14.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.805+0000 7fe260b31640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe25c101330 0x7fe25c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.805+0000 7fe260b31640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe25c101330 0x7fe25c1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47452/0 (socket says 172.21.15.52:47452) 2024-04-02T13:55:14.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.805+0000 7fe260b31640 1 -- 172.21.15.52:0/1867337141 learned_addr learned my addr 172.21.15.52:0/1867337141 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.805+0000 7fe261b33640 1 --2- 172.21.15.52:0/1867337141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe25c106980 0x7fe25c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.805+0000 7fe261332640 1 --2- 172.21.15.52:0/1867337141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe25c100970 0x7fe25c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.805+0000 7fe260b31640 1 -- 172.21.15.52:0/1867337141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe25c100970 msgr2=0x7fe25c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.805+0000 7fe260b31640 1 --2- 172.21.15.52:0/1867337141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe25c100970 0x7fe25c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.805+0000 7fe260b31640 1 -- 172.21.15.52:0/1867337141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe25c106980 msgr2=0x7fe25c19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.805+0000 7fe260b31640 1 --2- 172.21.15.52:0/1867337141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe25c106980 0x7fe25c19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.805+0000 7fe260b31640 1 -- 172.21.15.52:0/1867337141 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe25c19d3a0 con 0x7fe25c101330 2024-04-02T13:55:14.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.805+0000 7fe260b31640 1 --2- 172.21.15.52:0/1867337141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe25c101330 0x7fe25c1988f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe25000e9f0 tx=0x7fe25000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.805+0000 7fe25a7fc640 1 -- 172.21.15.52:0/1867337141 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25000cd80 con 0x7fe25c101330 2024-04-02T13:55:14.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.805+0000 7fe25a7fc640 1 -- 172.21.15.52:0/1867337141 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe250010910 con 0x7fe25c101330 2024-04-02T13:55:14.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.805+0000 7fe25a7fc640 1 -- 172.21.15.52:0/1867337141 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250018990 con 0x7fe25c101330 2024-04-02T13:55:14.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.805+0000 7fe263596640 1 -- 172.21.15.52:0/1867337141 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe25c19d690 con 0x7fe25c101330 2024-04-02T13:55:14.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.806+0000 7fe263596640 1 -- 172.21.15.52:0/1867337141 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe25c1a4f20 con 0x7fe25c101330 2024-04-02T13:55:14.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.808+0000 7fe25a7fc640 1 -- 172.21.15.52:0/1867337141 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe250010ac0 con 0x7fe25c101330 2024-04-02T13:55:14.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.808+0000 7fe26617f740 1 Processor -- start 2024-04-02T13:55:14.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.808+0000 7fe26617f740 1 -- start start 2024-04-02T13:55:14.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.808+0000 7fe26617f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb0f7a0 0x55fd8fb0fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.808+0000 7fe26617f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8fb100c0 0x55fd8fb18760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.808+0000 7fe26617f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fd8fb18ca0 0x55fd8fb1b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.808+0000 7fe26617f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fd8fa737b0 con 0x55fd8fb0f7a0 2024-04-02T13:55:14.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.808+0000 7fe26617f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55fd8fa12570 con 0x55fd8fb18ca0 2024-04-02T13:55:14.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.808+0000 7fe26617f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fd8fa6f760 con 0x55fd8fb100c0 2024-04-02T13:55:14.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.808+0000 7fe25a7fc640 1 --2- 172.21.15.52:0/1867337141 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe2440a3da0 0x7fe2440a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.809+0000 7fe261b33640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb0f7a0 0x55fd8fb0fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.809+0000 7fe261b33640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb0f7a0 0x55fd8fb0fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47458/0 (socket says 172.21.15.52:47458) 2024-04-02T13:55:14.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.809+0000 7fe261b33640 1 -- 172.21.15.52:0/1161763260 learned_addr learned my addr 172.21.15.52:0/1161763260 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.809+0000 7fe261b33640 1 --2- 172.21.15.52:0/1867337141 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe2440a3da0 0x7fe2440a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.809+0000 7fe25a7fc640 1 -- 172.21.15.52:0/1867337141 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe2500d0dc0 con 0x7fe25c101330 2024-04-02T13:55:14.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.809+0000 7fe261332640 1 --2- 172.21.15.52:0/1161763260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8fb100c0 0x55fd8fb18760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.809+0000 7fe260b31640 1 --2- 172.21.15.52:0/1161763260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fd8fb18ca0 0x55fd8fb1b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.809+0000 7fe261b33640 1 -- 172.21.15.52:0/1161763260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8fb100c0 msgr2=0x55fd8fb18760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.809+0000 7fe261b33640 1 --2- 172.21.15.52:0/1161763260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8fb100c0 0x55fd8fb18760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.809+0000 7fe261b33640 1 -- 172.21.15.52:0/1161763260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fd8fb18ca0 msgr2=0x55fd8fb1b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.809+0000 7fe261b33640 1 --2- 172.21.15.52:0/1161763260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fd8fb18ca0 0x55fd8fb1b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.809+0000 7fe261b33640 1 -- 172.21.15.52:0/1161763260 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd8f7873f0 con 0x55fd8fb0f7a0 2024-04-02T13:55:14.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.809+0000 7fe261332640 1 --2- 172.21.15.52:0/1161763260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8fb100c0 0x55fd8fb18760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:14.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.809+0000 7fe261b33640 1 --2- 172.21.15.52:0/1161763260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb0f7a0 0x55fd8fb0fb80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe24c011da0 tx=0x7fe24c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414883b67a5d7e2b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.809+0000 7fe261b33640 1 --2- 172.21.15.52:0/1867337141 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe2440a3da0 0x7fe2440a6260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00fc90 tx=0x7fe24c01e000 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.809+0000 7fe2437fe640 1 -- 172.21.15.52:0/1161763260 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c022070 con 0x55fd8fb0f7a0 2024-04-02T13:55:14.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.809+0000 7fe2437fe640 1 -- 172.21.15.52:0/1161763260 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe24c009dc0 con 0x55fd8fb0f7a0 2024-04-02T13:55:14.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.809+0000 7fe2437fe640 1 -- 172.21.15.52:0/1161763260 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c0042b0 con 0x55fd8fb0f7a0 2024-04-02T13:55:14.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.810+0000 7fe26617f740 1 -- 172.21.15.52:0/1161763260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb0f7a0 msgr2=0x55fd8fb0fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.810+0000 7fe26617f740 1 --2- 172.21.15.52:0/1161763260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb0f7a0 0x55fd8fb0fb80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe24c011da0 tx=0x7fe24c00f790 comp rx=0 tx=0).stop 2024-04-02T13:55:14.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.810+0000 7fe26617f740 1 -- 172.21.15.52:0/1161763260 shutdown_connections 2024-04-02T13:55:14.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.810+0000 7fe26617f740 1 --2- 172.21.15.52:0/1161763260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fd8fb18ca0 0x55fd8fb1b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.810+0000 7fe26617f740 1 --2- 172.21.15.52:0/1161763260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8fb100c0 0x55fd8fb18760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.810+0000 7fe26617f740 1 --2- 172.21.15.52:0/1161763260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb0f7a0 0x55fd8fb0fb80 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.810+0000 7fe26617f740 1 -- 172.21.15.52:0/1161763260 >> 172.21.15.52:0/1161763260 conn(0x55fd8fb09ed0 msgr2=0x55fd8fb0a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.810+0000 7fe26617f740 1 -- 172.21.15.52:0/1161763260 shutdown_connections 2024-04-02T13:55:14.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.810+0000 7fe26617f740 1 -- 172.21.15.52:0/1161763260 wait complete. 2024-04-02T13:55:14.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.810+0000 7fe26617f740 1 -- start start 2024-04-02T13:55:14.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.811+0000 7fe26617f740 1 Processor -- start 2024-04-02T13:55:14.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.811+0000 7fe26617f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8fb0f7a0 0x55fd8fb15b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.811+0000 7fe26617f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fd8fb100c0 0x55fd8fb160a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.811+0000 7fe26617f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb18ca0 0x55fd8fb165e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.811+0000 7fe26617f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fd8fb1ba30 con 0x55fd8fb18ca0 2024-04-02T13:55:14.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.811+0000 7fe26617f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55fd8fa2ee80 con 0x55fd8fb100c0 2024-04-02T13:55:14.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.811+0000 7fe26617f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fd8fb1bbb0 con 0x55fd8fb0f7a0 2024-04-02T13:55:14.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.811+0000 7fe26617f740 10 client.0 _ll_register_callbacks cb 0x55fd8fa6aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:14.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.811+0000 7fe261b33640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb18ca0 0x55fd8fb165e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.811+0000 7fe261b33640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb18ca0 0x55fd8fb165e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47470/0 (socket says 172.21.15.52:47470) 2024-04-02T13:55:14.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.811+0000 7fe261b33640 1 -- 172.21.15.52:0/161047573 learned_addr learned my addr 172.21.15.52:0/161047573 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.811+0000 7fe260b31640 1 --2- 172.21.15.52:0/161047573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fd8fb100c0 0x55fd8fb160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.811+0000 7fe261332640 1 --2- 172.21.15.52:0/161047573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8fb0f7a0 0x55fd8fb15b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.811+0000 7fe261b33640 1 -- 172.21.15.52:0/161047573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8fb0f7a0 msgr2=0x55fd8fb15b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.811+0000 7fe261b33640 1 --2- 172.21.15.52:0/161047573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8fb0f7a0 0x55fd8fb15b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.811+0000 7fe261b33640 1 -- 172.21.15.52:0/161047573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fd8fb100c0 msgr2=0x55fd8fb160a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.811+0000 7fe261b33640 1 --2- 172.21.15.52:0/161047573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fd8fb100c0 0x55fd8fb160a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.811+0000 7fe261b33640 1 -- 172.21.15.52:0/161047573 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd8f8517d0 con 0x55fd8fb18ca0 2024-04-02T13:55:14.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.812+0000 7fe261b33640 1 --2- 172.21.15.52:0/161047573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb18ca0 0x55fd8fb165e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe24c003d00 tx=0x7fe24c003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc38fe0b92b3c5d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.812+0000 7fe240ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:14.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.812+0000 7fe240ff9640 1 -- 172.21.15.52:0/161047573 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c022040 con 0x55fd8fb18ca0 2024-04-02T13:55:14.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.812+0000 7fe240ff9640 1 -- 172.21.15.52:0/161047573 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe24c011910 con 0x55fd8fb18ca0 2024-04-02T13:55:14.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.812+0000 7fe26617f740 1 -- 172.21.15.52:0/161047573 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe24c009dc0 con 0x55fd8fb18ca0 2024-04-02T13:55:14.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.812+0000 7fe240ff9640 1 -- 172.21.15.52:0/161047573 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe24c03c050 con 0x55fd8fb18ca0 2024-04-02T13:55:14.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.812+0000 7fe26617f740 10 client.16182 fetch_fsmap learned FSMap version 28 2024-04-02T13:55:14.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.812+0000 7fe26617f740 1 -- 172.21.15.52:0/161047573 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fd8fb27bb0 con 0x55fd8fb18ca0 2024-04-02T13:55:14.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.812+0000 7fe240ff9640 1 -- 172.21.15.52:0/161047573 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe24c009dc0 con 0x55fd8fb18ca0 2024-04-02T13:55:14.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.812+0000 7fe240ff9640 1 -- 172.21.15.52:0/161047573 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe24c02e7e0 con 0x55fd8fb18ca0 2024-04-02T13:55:14.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.812+0000 7fe26617f740 10 client.16182 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:55:14.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.812+0000 7fe26617f740 10 client.16182 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:55:14.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.812+0000 7fe26617f740 20 client.16182 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:14.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.812+0000 7fe26617f740 1 --2- 172.21.15.52:0/161047573 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55fd8fb29010 0x55fd8fb2b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.812+0000 7fe26617f740 4 client.16182 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:14.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.812+0000 7fe26617f740 1 -- 172.21.15.52:0/161047573 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd8fa80cf0 con 0x55fd8fb29010 2024-04-02T13:55:14.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.813+0000 7fe261332640 1 --2- 172.21.15.52:0/161047573 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55fd8fb29010 0x55fd8fb2b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.813+0000 7fe261332640 1 --2- 172.21.15.52:0/161047573 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55fd8fb29010 0x55fd8fb2b400 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=623efa50204cf825 server_cookie=976ec3f0f44b5d8a in_seq=0 out_seq=0 2024-04-02T13:55:14.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.813+0000 7fe240ff9640 10 client.16182 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:55:14.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.814+0000 7fe240ff9640 1 -- 172.21.15.52:0/161047573 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fd8fa80cf0 con 0x55fd8fb29010 2024-04-02T13:55:14.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.814+0000 7fe240ff9640 10 client.16182 handle_command_reply: tid=0 2024-04-02T13:55:14.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.814+0000 7fe26617f740 1 client.16182 shutdown 2024-04-02T13:55:14.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.814+0000 7fe26617f740 1 -- 172.21.15.52:0/161047573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb18ca0 msgr2=0x55fd8fb165e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.814+0000 7fe26617f740 1 --2- 172.21.15.52:0/161047573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb18ca0 0x55fd8fb165e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe24c003d00 tx=0x7fe24c003e60 comp rx=0 tx=0).stop 2024-04-02T13:55:14.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.814+0000 7fe26617f740 1 -- 172.21.15.52:0/161047573 shutdown_connections 2024-04-02T13:55:14.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.814+0000 7fe26617f740 1 --2- 172.21.15.52:0/161047573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb18ca0 0x55fd8fb165e0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.814+0000 7fe26617f740 1 --2- 172.21.15.52:0/161047573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fd8fb100c0 0x55fd8fb160a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.814+0000 7fe26617f740 1 --2- 172.21.15.52:0/161047573 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55fd8fb29010 0x55fd8fb2b400 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.814+0000 7fe26617f740 1 --2- 172.21.15.52:0/161047573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8fb0f7a0 0x55fd8fb15b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.814+0000 7fe240ff9640 0 client.16182 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:55:14.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.814+0000 7fe26617f740 1 -- 172.21.15.52:0/161047573 >> 172.21.15.52:0/161047573 conn(0x55fd8fb09ed0 msgr2=0x55fd8fb0bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.814+0000 7fe26617f740 1 -- 172.21.15.52:0/161047573 shutdown_connections 2024-04-02T13:55:14.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.814+0000 7fe26617f740 1 -- 172.21.15.52:0/161047573 wait complete. 2024-04-02T13:55:14.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.814+0000 7fe26617f740 20 client.16182 trim_cache size 0 max 16384 2024-04-02T13:55:14.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.827+0000 7fe26617f740 1 Processor -- start 2024-04-02T13:55:14.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.827+0000 7fe26617f740 1 -- start start 2024-04-02T13:55:14.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.827+0000 7fe26617f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8fb0f7a0 0x55fd8fb18290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.827+0000 7fe26617f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb100c0 0x55fd8fb248f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.827+0000 7fe26617f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fd8fb18ca0 0x55fd8fb14cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.827+0000 7fe26617f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1aeea0 con 0x55fd8fb100c0 2024-04-02T13:55:14.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.827+0000 7fe26617f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1b09e0 con 0x55fd8fb18ca0 2024-04-02T13:55:14.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.827+0000 7fe26617f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1af570 con 0x55fd8fb0f7a0 2024-04-02T13:55:14.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.828+0000 7fe260b31640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb100c0 0x55fd8fb248f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.828+0000 7fe260b31640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb100c0 0x55fd8fb248f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47480/0 (socket says 172.21.15.52:47480) 2024-04-02T13:55:14.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.828+0000 7fe260b31640 1 -- 172.21.15.52:0/3665579837 learned_addr learned my addr 172.21.15.52:0/3665579837 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.828+0000 7fe261b33640 1 --2- 172.21.15.52:0/3665579837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fd8fb18ca0 0x55fd8fb14cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.828+0000 7fe260b31640 1 -- 172.21.15.52:0/3665579837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8fb0f7a0 msgr2=0x55fd8fb18290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.828+0000 7fe260b31640 1 --2- 172.21.15.52:0/3665579837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8fb0f7a0 0x55fd8fb18290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.828+0000 7fe260b31640 1 -- 172.21.15.52:0/3665579837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fd8fb18ca0 msgr2=0x55fd8fb14cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.828+0000 7fe260b31640 1 --2- 172.21.15.52:0/3665579837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fd8fb18ca0 0x55fd8fb14cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.828+0000 7fe260b31640 1 -- 172.21.15.52:0/3665579837 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd8fb2b9c0 con 0x55fd8fb100c0 2024-04-02T13:55:14.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.828+0000 7fe261b33640 1 --2- 172.21.15.52:0/3665579837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fd8fb18ca0 0x55fd8fb14cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:14.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.828+0000 7fe260b31640 1 --2- 172.21.15.52:0/3665579837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb100c0 0x55fd8fb248f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe250009380 tx=0x7fe250098df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241aaff317c0dd93 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.828+0000 7fe242ffd640 1 -- 172.21.15.52:0/3665579837 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2500a5690 con 0x55fd8fb100c0 2024-04-02T13:55:14.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.828+0000 7fe242ffd640 1 -- 172.21.15.52:0/3665579837 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2500ce450 con 0x55fd8fb100c0 2024-04-02T13:55:14.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.828+0000 7fe242ffd640 1 -- 172.21.15.52:0/3665579837 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250018d90 con 0x55fd8fb100c0 2024-04-02T13:55:14.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.829+0000 7fe26617f740 1 -- 172.21.15.52:0/3665579837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb100c0 msgr2=0x55fd8fb248f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.829+0000 7fe26617f740 1 --2- 172.21.15.52:0/3665579837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb100c0 0x55fd8fb248f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe250009380 tx=0x7fe250098df0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.829+0000 7fe26617f740 1 -- 172.21.15.52:0/3665579837 shutdown_connections 2024-04-02T13:55:14.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.829+0000 7fe26617f740 1 --2- 172.21.15.52:0/3665579837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fd8fb18ca0 0x55fd8fb14cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.829+0000 7fe26617f740 1 --2- 172.21.15.52:0/3665579837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb100c0 0x55fd8fb248f0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.829+0000 7fe26617f740 1 --2- 172.21.15.52:0/3665579837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8fb0f7a0 0x55fd8fb18290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.829+0000 7fe26617f740 1 -- 172.21.15.52:0/3665579837 >> 172.21.15.52:0/3665579837 conn(0x55fd8fb09ed0 msgr2=0x55fd8fb086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.829+0000 7fe26617f740 1 -- 172.21.15.52:0/3665579837 shutdown_connections 2024-04-02T13:55:14.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.829+0000 7fe26617f740 1 -- 172.21.15.52:0/3665579837 wait complete. 2024-04-02T13:55:14.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.829+0000 7fe26617f740 1 -- start start 2024-04-02T13:55:14.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.829+0000 7fe26617f740 1 Processor -- start 2024-04-02T13:55:14.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.829+0000 7fe26617f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fd8fb0f7a0 0x55fd8fb1c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.830+0000 7fe26617f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb100c0 0x55fd8fb1c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.830+0000 7fe26617f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8fb18ca0 0x55fd8fb874b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.830+0000 7fe26617f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1b0310 con 0x55fd8fb100c0 2024-04-02T13:55:14.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.830+0000 7fe26617f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1afc40 con 0x55fd8fb0f7a0 2024-04-02T13:55:14.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.830+0000 7fe26617f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1ace80 con 0x55fd8fb18ca0 2024-04-02T13:55:14.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.830+0000 7fe26617f740 10 client.0 _ll_register_callbacks cb 0x7fe250009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:14.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.830+0000 7fe260b31640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb100c0 0x55fd8fb1c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.830+0000 7fe260b31640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb100c0 0x55fd8fb1c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47484/0 (socket says 172.21.15.52:47484) 2024-04-02T13:55:14.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.830+0000 7fe260b31640 1 -- 172.21.15.52:0/3175623924 learned_addr learned my addr 172.21.15.52:0/3175623924 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.830+0000 7fe261332640 1 --2- 172.21.15.52:0/3175623924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fd8fb0f7a0 0x55fd8fb1c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.830+0000 7fe261b33640 1 --2- 172.21.15.52:0/3175623924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8fb18ca0 0x55fd8fb874b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.830+0000 7fe260b31640 1 -- 172.21.15.52:0/3175623924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8fb18ca0 msgr2=0x55fd8fb874b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.830+0000 7fe260b31640 1 --2- 172.21.15.52:0/3175623924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8fb18ca0 0x55fd8fb874b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.830+0000 7fe260b31640 1 -- 172.21.15.52:0/3175623924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fd8fb0f7a0 msgr2=0x55fd8fb1c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.830+0000 7fe260b31640 1 --2- 172.21.15.52:0/3175623924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fd8fb0f7a0 0x55fd8fb1c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.830+0000 7fe260b31640 1 -- 172.21.15.52:0/3175623924 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd8fb87a70 con 0x55fd8fb100c0 2024-04-02T13:55:14.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.830+0000 7fe261b33640 1 --2- 172.21.15.52:0/3175623924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8fb18ca0 0x55fd8fb874b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:14.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.830+0000 7fe260b31640 1 --2- 172.21.15.52:0/3175623924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb100c0 0x55fd8fb1c610 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe2500ce7d0 tx=0x7fe2500bd1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f870b0d8fb15fab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.830+0000 7fe240ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:14.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.830+0000 7fe240ff9640 1 -- 172.21.15.52:0/3175623924 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250018d30 con 0x55fd8fb100c0 2024-04-02T13:55:14.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.830+0000 7fe240ff9640 1 -- 172.21.15.52:0/3175623924 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe25009bf20 con 0x55fd8fb100c0 2024-04-02T13:55:14.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.830+0000 7fe26617f740 1 -- 172.21.15.52:0/3175623924 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2500ce450 con 0x55fd8fb100c0 2024-04-02T13:55:14.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.830+0000 7fe240ff9640 1 -- 172.21.15.52:0/3175623924 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe25009c120 con 0x55fd8fb100c0 2024-04-02T13:55:14.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.830+0000 7fe26617f740 10 client.16188 fetch_fsmap learned FSMap version 28 2024-04-02T13:55:14.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.830+0000 7fe26617f740 1 -- 172.21.15.52:0/3175623924 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fd8fb8ed90 con 0x55fd8fb100c0 2024-04-02T13:55:14.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.831+0000 7fe240ff9640 1 -- 172.21.15.52:0/3175623924 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2500ce450 con 0x55fd8fb100c0 2024-04-02T13:55:14.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.831+0000 7fe240ff9640 1 -- 172.21.15.52:0/3175623924 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe2500ba070 con 0x55fd8fb100c0 2024-04-02T13:55:14.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.831+0000 7fe26617f740 10 client.16188 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:55:14.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.831+0000 7fe26617f740 10 client.16188 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:55:14.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.831+0000 7fe26617f740 20 client.16188 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:14.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.831+0000 7fe26617f740 1 --2- 172.21.15.52:0/3175623924 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55fd8fb8f2f0 0x55fd8fb91710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.831+0000 7fe26617f740 4 client.16188 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:55:14.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.831+0000 7fe26617f740 1 -- 172.21.15.52:0/3175623924 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd8fa7b040 con 0x55fd8fb8f2f0 2024-04-02T13:55:14.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.831+0000 7fe261332640 1 --2- 172.21.15.52:0/3175623924 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55fd8fb8f2f0 0x55fd8fb91710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.831+0000 7fe240ff9640 10 client.16188 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:55:14.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.831+0000 7fe261332640 1 --2- 172.21.15.52:0/3175623924 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55fd8fb8f2f0 0x55fd8fb91710 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d6c7e0db5ddabec0 server_cookie=7ec0400f045bfb98 in_seq=0 out_seq=0 2024-04-02T13:55:14.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.832+0000 7fe240ff9640 1 -- 172.21.15.52:0/3175623924 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd8fa7b040 con 0x55fd8fb8f2f0 2024-04-02T13:55:14.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.832+0000 7fe240ff9640 10 client.16188 handle_command_reply: tid=0 2024-04-02T13:55:14.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.832+0000 7fe26617f740 1 client.16188 shutdown 2024-04-02T13:55:14.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.832+0000 7fe26617f740 1 -- 172.21.15.52:0/3175623924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb100c0 msgr2=0x55fd8fb1c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.832+0000 7fe26617f740 1 --2- 172.21.15.52:0/3175623924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb100c0 0x55fd8fb1c610 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe2500ce7d0 tx=0x7fe2500bd1a0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.832+0000 7fe26617f740 1 -- 172.21.15.52:0/3175623924 shutdown_connections 2024-04-02T13:55:14.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.832+0000 7fe26617f740 1 --2- 172.21.15.52:0/3175623924 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55fd8fb8f2f0 0x55fd8fb91710 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.832+0000 7fe26617f740 1 --2- 172.21.15.52:0/3175623924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8fb18ca0 0x55fd8fb874b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.832+0000 7fe26617f740 1 --2- 172.21.15.52:0/3175623924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd8fb100c0 0x55fd8fb1c610 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.832+0000 7fe26617f740 1 --2- 172.21.15.52:0/3175623924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fd8fb0f7a0 0x55fd8fb1c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.832+0000 7fe240ff9640 0 client.16188 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:55:14.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.832+0000 7fe26617f740 1 -- 172.21.15.52:0/3175623924 >> 172.21.15.52:0/3175623924 conn(0x55fd8fb09ed0 msgr2=0x55fd8fb086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.832+0000 7fe26617f740 1 -- 172.21.15.52:0/3175623924 shutdown_connections 2024-04-02T13:55:14.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.832+0000 7fe26617f740 1 -- 172.21.15.52:0/3175623924 wait complete. 2024-04-02T13:55:14.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.832+0000 7fe26617f740 20 client.16188 trim_cache size 0 max 16384 2024-04-02T13:55:14.843 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:55:14.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.833+0000 7fe263596640 1 -- 172.21.15.52:0/1867337141 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe2440a3da0 msgr2=0x7fe2440a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.833+0000 7fe263596640 1 --2- 172.21.15.52:0/1867337141 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe2440a3da0 0x7fe2440a6260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00fc90 tx=0x7fe24c01e000 comp rx=0 tx=0).stop 2024-04-02T13:55:14.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.833+0000 7fe263596640 1 -- 172.21.15.52:0/1867337141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe25c101330 msgr2=0x7fe25c1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.833+0000 7fe263596640 1 --2- 172.21.15.52:0/1867337141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe25c101330 0x7fe25c1988f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe25000e9f0 tx=0x7fe25000eeb0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.833+0000 7fe263596640 1 -- 172.21.15.52:0/1867337141 shutdown_connections 2024-04-02T13:55:14.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.833+0000 7fe263596640 1 --2- 172.21.15.52:0/1867337141 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe2440a3da0 0x7fe2440a6260 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.833+0000 7fe263596640 1 --2- 172.21.15.52:0/1867337141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe25c106980 0x7fe25c19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.833+0000 7fe263596640 1 --2- 172.21.15.52:0/1867337141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe25c101330 0x7fe25c1988f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.833+0000 7fe263596640 1 --2- 172.21.15.52:0/1867337141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe25c100970 0x7fe25c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.833+0000 7fe263596640 1 -- 172.21.15.52:0/1867337141 >> 172.21.15.52:0/1867337141 conn(0x7fe25c077b00 msgr2=0x7fe25c103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.833+0000 7fe263596640 1 -- 172.21.15.52:0/1867337141 shutdown_connections 2024-04-02T13:55:14.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.833+0000 7fe263596640 1 -- 172.21.15.52:0/1867337141 wait complete. 2024-04-02T13:55:14.847 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:55:14.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.949+0000 7f603e1bc640 1 -- 172.21.15.52:0/256463614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6038107a80 msgr2=0x7f6038107e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.949+0000 7f603e1bc640 1 --2- 172.21.15.52:0/256463614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6038107a80 0x7f6038107e60 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6028009930 tx=0x7f602802bc30 comp rx=0 tx=0).stop 2024-04-02T13:55:14.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.949+0000 7f603e1bc640 1 -- 172.21.15.52:0/256463614 shutdown_connections 2024-04-02T13:55:14.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.949+0000 7f603e1bc640 1 --2- 172.21.15.52:0/256463614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f60380696a0 0x7f603810e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.949+0000 7f603e1bc640 1 --2- 172.21.15.52:0/256463614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6038068ce0 0x7f6038069160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.949+0000 7f603e1bc640 1 --2- 172.21.15.52:0/256463614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6038107a80 0x7f6038107e60 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.949+0000 7f603e1bc640 1 -- 172.21.15.52:0/256463614 >> 172.21.15.52:0/256463614 conn(0x7f6038077b00 msgr2=0x7f6038077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.949+0000 7f603e1bc640 1 -- 172.21.15.52:0/256463614 shutdown_connections 2024-04-02T13:55:14.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.950+0000 7f603e1bc640 1 -- 172.21.15.52:0/256463614 wait complete. 2024-04-02T13:55:14.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.950+0000 7f603e1bc640 1 Processor -- start 2024-04-02T13:55:14.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.950+0000 7f603e1bc640 1 -- start start 2024-04-02T13:55:14.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.950+0000 7f603e1bc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6038068ce0 0x7f60381983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.950+0000 7f603e1bc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60380696a0 0x7f6038198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.950+0000 7f603e1bc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6038107a80 0x7f603819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.950+0000 7f603e1bc640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6038111e90 con 0x7f60380696a0 2024-04-02T13:55:14.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.950+0000 7f603e1bc640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6038111d10 con 0x7f6038068ce0 2024-04-02T13:55:14.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.950+0000 7f603e1bc640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6038112010 con 0x7f6038107a80 2024-04-02T13:55:14.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.950+0000 7f6036ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60380696a0 0x7f6038198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.951+0000 7f6036ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60380696a0 0x7f6038198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47514/0 (socket says 172.21.15.52:47514) 2024-04-02T13:55:14.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.951+0000 7f6036ffd640 1 -- 172.21.15.52:0/3915214781 learned_addr learned my addr 172.21.15.52:0/3915214781 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.951+0000 7f60377fe640 1 --2- 172.21.15.52:0/3915214781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6038068ce0 0x7f60381983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.951+0000 7f6037fff640 1 --2- 172.21.15.52:0/3915214781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6038107a80 0x7f603819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.951+0000 7f6036ffd640 1 -- 172.21.15.52:0/3915214781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6038107a80 msgr2=0x7f603819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.951+0000 7f6036ffd640 1 --2- 172.21.15.52:0/3915214781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6038107a80 0x7f603819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.951+0000 7f6036ffd640 1 -- 172.21.15.52:0/3915214781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6038068ce0 msgr2=0x7f60381983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.951+0000 7f6036ffd640 1 --2- 172.21.15.52:0/3915214781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6038068ce0 0x7f60381983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.951+0000 7f6036ffd640 1 -- 172.21.15.52:0/3915214781 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603819d3c0 con 0x7f60380696a0 2024-04-02T13:55:14.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.951+0000 7f6036ffd640 1 --2- 172.21.15.52:0/3915214781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60380696a0 0x7f6038198910 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f602c00e9f0 tx=0x7f602c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.951+0000 7f6034ff9640 1 -- 172.21.15.52:0/3915214781 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c00cd80 con 0x7f60380696a0 2024-04-02T13:55:14.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.951+0000 7f6034ff9640 1 -- 172.21.15.52:0/3915214781 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f602c010910 con 0x7f60380696a0 2024-04-02T13:55:14.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.951+0000 7f603e1bc640 1 -- 172.21.15.52:0/3915214781 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f603819d6b0 con 0x7f60380696a0 2024-04-02T13:55:14.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.951+0000 7f6034ff9640 1 -- 172.21.15.52:0/3915214781 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c0188f0 con 0x7f60380696a0 2024-04-02T13:55:14.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.951+0000 7f603e1bc640 1 -- 172.21.15.52:0/3915214781 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60381a4f40 con 0x7f60380696a0 2024-04-02T13:55:14.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.953+0000 7f6034ff9640 1 -- 172.21.15.52:0/3915214781 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f602c010ac0 con 0x7f60380696a0 2024-04-02T13:55:14.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.954+0000 7f6040d9b740 1 Processor -- start 2024-04-02T13:55:14.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.954+0000 7f6040d9b740 1 -- start start 2024-04-02T13:55:14.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.954+0000 7f6034ff9640 1 --2- 172.21.15.52:0/3915214781 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f60200a3d20 0x7f60200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.954+0000 7f6034ff9640 1 -- 172.21.15.52:0/3915214781 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f602c0d0f80 con 0x7f60380696a0 2024-04-02T13:55:14.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.954+0000 7f6040d9b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9126d85b0 0x55c9126d8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.954+0000 7f6037fff640 1 --2- 172.21.15.52:0/3915214781 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f60200a3d20 0x7f60200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.954+0000 7f6040d9b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126d8f60 0x55c9126e5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.954+0000 7f6040d9b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9126e60b0 0x55c9126e84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.954+0000 7f6040d9b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c91263c7b0 con 0x55c9126d8f60 2024-04-02T13:55:14.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.954+0000 7f6040d9b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c9125db570 con 0x55c9126d85b0 2024-04-02T13:55:14.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.954+0000 7f6040d9b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c912638760 con 0x55c9126e60b0 2024-04-02T13:55:14.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.954+0000 7f6036ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126d8f60 0x55c9126e5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.954+0000 7f60377fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9126d85b0 0x55c9126d8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.954+0000 7f6036ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126d8f60 0x55c9126e5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47518/0 (socket says 172.21.15.52:47518) 2024-04-02T13:55:14.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.954+0000 7f6036ffd640 1 -- 172.21.15.52:0/564517232 learned_addr learned my addr 172.21.15.52:0/564517232 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.954+0000 7f6036ffd640 1 -- 172.21.15.52:0/564517232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9126e60b0 msgr2=0x55c9126e84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.954+0000 7f6036ffd640 1 --2- 172.21.15.52:0/564517232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9126e60b0 0x55c9126e84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.954+0000 7f6036ffd640 1 -- 172.21.15.52:0/564517232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9126d85b0 msgr2=0x55c9126d8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.954+0000 7f6036ffd640 1 --2- 172.21.15.52:0/564517232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9126d85b0 0x55c9126d8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.954+0000 7f6036ffd640 1 -- 172.21.15.52:0/564517232 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9123503f0 con 0x55c9126d8f60 2024-04-02T13:55:14.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.954+0000 7f60377fe640 1 --2- 172.21.15.52:0/564517232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9126d85b0 0x55c9126d8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:14.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.954+0000 7f6037fff640 1 --2- 172.21.15.52:0/3915214781 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f60200a3d20 0x7f60200a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f60300092f0 tx=0x7f6030010740 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.955+0000 7f6036ffd640 1 --2- 172.21.15.52:0/564517232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126d8f60 0x55c9126e5b70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f602c0995b0 tx=0x7f602c09d420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcbd8640ca86b2e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.955+0000 7f601dffb640 1 -- 172.21.15.52:0/564517232 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c0ce610 con 0x55c9126d8f60 2024-04-02T13:55:14.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.955+0000 7f601dffb640 1 -- 172.21.15.52:0/564517232 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f602c099250 con 0x55c9126d8f60 2024-04-02T13:55:14.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.955+0000 7f601dffb640 1 -- 172.21.15.52:0/564517232 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c09c320 con 0x55c9126d8f60 2024-04-02T13:55:14.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.955+0000 7f6040d9b740 1 -- 172.21.15.52:0/564517232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126d8f60 msgr2=0x55c9126e5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.955+0000 7f6040d9b740 1 --2- 172.21.15.52:0/564517232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126d8f60 0x55c9126e5b70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f602c0995b0 tx=0x7f602c09d420 comp rx=0 tx=0).stop 2024-04-02T13:55:14.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.955+0000 7f6040d9b740 1 -- 172.21.15.52:0/564517232 shutdown_connections 2024-04-02T13:55:14.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.955+0000 7f6040d9b740 1 --2- 172.21.15.52:0/564517232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9126e60b0 0x55c9126e84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.955+0000 7f6040d9b740 1 --2- 172.21.15.52:0/564517232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126d8f60 0x55c9126e5b70 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.955+0000 7f6040d9b740 1 --2- 172.21.15.52:0/564517232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9126d85b0 0x55c9126d8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.955+0000 7f6040d9b740 1 -- 172.21.15.52:0/564517232 >> 172.21.15.52:0/564517232 conn(0x55c9126d2ed0 msgr2=0x55c9126d32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.955+0000 7f6040d9b740 1 -- 172.21.15.52:0/564517232 shutdown_connections 2024-04-02T13:55:14.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.955+0000 7f6040d9b740 1 -- 172.21.15.52:0/564517232 wait complete. 2024-04-02T13:55:14.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.956+0000 7f6040d9b740 1 -- start start 2024-04-02T13:55:14.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.956+0000 7f6040d9b740 1 Processor -- start 2024-04-02T13:55:14.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.956+0000 7f6040d9b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9126d85b0 0x55c9126e2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.956+0000 7f6040d9b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9126d8f60 0x55c9126e34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.956+0000 7f6040d9b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126e60b0 0x55c9126e39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.956+0000 7f6040d9b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c9126e8e10 con 0x55c9126e60b0 2024-04-02T13:55:14.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.956+0000 7f6040d9b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c9125f7e80 con 0x55c9126d8f60 2024-04-02T13:55:14.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.956+0000 7f6040d9b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c9126e8f90 con 0x55c9126d85b0 2024-04-02T13:55:14.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.956+0000 7f6040d9b740 10 client.0 _ll_register_callbacks cb 0x55c912633aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:14.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.956+0000 7f6037fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126e60b0 0x55c9126e39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.956+0000 7f6037fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126e60b0 0x55c9126e39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47522/0 (socket says 172.21.15.52:47522) 2024-04-02T13:55:14.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.956+0000 7f6037fff640 1 -- 172.21.15.52:0/3609428387 learned_addr learned my addr 172.21.15.52:0/3609428387 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.956+0000 7f6036ffd640 1 --2- 172.21.15.52:0/3609428387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9126d8f60 0x55c9126e34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.957+0000 7f60377fe640 1 --2- 172.21.15.52:0/3609428387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9126d85b0 0x55c9126e2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.957+0000 7f6037fff640 1 -- 172.21.15.52:0/3609428387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9126d85b0 msgr2=0x55c9126e2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.957+0000 7f6037fff640 1 --2- 172.21.15.52:0/3609428387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9126d85b0 0x55c9126e2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.957+0000 7f6037fff640 1 -- 172.21.15.52:0/3609428387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9126d8f60 msgr2=0x55c9126e34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.957+0000 7f6037fff640 1 --2- 172.21.15.52:0/3609428387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9126d8f60 0x55c9126e34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.957+0000 7f6037fff640 1 -- 172.21.15.52:0/3609428387 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c91241a7d0 con 0x55c9126e60b0 2024-04-02T13:55:14.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.957+0000 7f60377fe640 1 --2- 172.21.15.52:0/3609428387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9126d85b0 0x55c9126e2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:14.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.957+0000 7f6037fff640 1 --2- 172.21.15.52:0/3609428387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126e60b0 0x55c9126e39f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f6030013760 tx=0x7f6030004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104df86a1ca85f04 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.957+0000 7f6016ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:14.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.957+0000 7f6016ffd640 1 -- 172.21.15.52:0/3609428387 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030024070 con 0x55c9126e60b0 2024-04-02T13:55:14.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.957+0000 7f6016ffd640 1 -- 172.21.15.52:0/3609428387 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f6030011b20 con 0x55c9126e60b0 2024-04-02T13:55:14.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.957+0000 7f6040d9b740 1 -- 172.21.15.52:0/3609428387 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f602c099250 con 0x55c9126e60b0 2024-04-02T13:55:14.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.957+0000 7f6016ffd640 1 -- 172.21.15.52:0/3609428387 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6030030050 con 0x55c9126e60b0 2024-04-02T13:55:14.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.957+0000 7f6040d9b740 10 client.16200 fetch_fsmap learned FSMap version 28 2024-04-02T13:55:14.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.957+0000 7f6040d9b740 1 -- 172.21.15.52:0/3609428387 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c9126f4fe0 con 0x55c9126e60b0 2024-04-02T13:55:14.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.957+0000 7f6016ffd640 1 -- 172.21.15.52:0/3609428387 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f602c099250 con 0x55c9126e60b0 2024-04-02T13:55:14.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.957+0000 7f6016ffd640 1 -- 172.21.15.52:0/3609428387 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f603000f040 con 0x55c9126e60b0 2024-04-02T13:55:14.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.957+0000 7f6040d9b740 10 client.16200 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:55:14.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.957+0000 7f6040d9b740 10 client.16200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:14.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.957+0000 7f6040d9b740 20 client.16200 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:14.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.958+0000 7f6040d9b740 1 --2- 172.21.15.52:0/3609428387 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c9126f6440 0x55c9126f8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.958+0000 7f6040d9b740 4 client.16200 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:14.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.958+0000 7f6040d9b740 1 -- 172.21.15.52:0/3609428387 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c912649cf0 con 0x55c9126f6440 2024-04-02T13:55:14.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.958+0000 7f60377fe640 1 --2- 172.21.15.52:0/3609428387 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c9126f6440 0x55c9126f8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.958+0000 7f60377fe640 1 --2- 172.21.15.52:0/3609428387 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c9126f6440 0x55c9126f8890 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=298dd85998b7044b server_cookie=b2df9c27a18e1e96 in_seq=0 out_seq=0 2024-04-02T13:55:14.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.958+0000 7f6016ffd640 10 client.16200 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:14.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.959+0000 7f6016ffd640 1 -- 172.21.15.52:0/3609428387 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c912649cf0 con 0x55c9126f6440 2024-04-02T13:55:14.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.959+0000 7f6016ffd640 10 client.16200 handle_command_reply: tid=0 2024-04-02T13:55:14.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.959+0000 7f6040d9b740 1 client.16200 shutdown 2024-04-02T13:55:14.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.959+0000 7f6040d9b740 1 -- 172.21.15.52:0/3609428387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126e60b0 msgr2=0x55c9126e39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.959+0000 7f6040d9b740 1 --2- 172.21.15.52:0/3609428387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126e60b0 0x55c9126e39f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f6030013760 tx=0x7f6030004ab0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.959+0000 7f6040d9b740 1 -- 172.21.15.52:0/3609428387 shutdown_connections 2024-04-02T13:55:14.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.959+0000 7f6040d9b740 1 --2- 172.21.15.52:0/3609428387 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c9126f6440 0x55c9126f8890 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.959+0000 7f6040d9b740 1 --2- 172.21.15.52:0/3609428387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126e60b0 0x55c9126e39f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.959+0000 7f6040d9b740 1 --2- 172.21.15.52:0/3609428387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9126d8f60 0x55c9126e34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.959+0000 7f6040d9b740 1 --2- 172.21.15.52:0/3609428387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9126d85b0 0x55c9126e2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.959+0000 7f6040d9b740 1 -- 172.21.15.52:0/3609428387 >> 172.21.15.52:0/3609428387 conn(0x55c9126d2ed0 msgr2=0x55c9126d4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.959+0000 7f6016ffd640 0 client.16200 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:14.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.959+0000 7f6040d9b740 1 -- 172.21.15.52:0/3609428387 shutdown_connections 2024-04-02T13:55:14.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.959+0000 7f6040d9b740 1 -- 172.21.15.52:0/3609428387 wait complete. 2024-04-02T13:55:14.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.959+0000 7f6040d9b740 20 client.16200 trim_cache size 0 max 16384 2024-04-02T13:55:14.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.973+0000 7f6040d9b740 1 Processor -- start 2024-04-02T13:55:14.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.973+0000 7f6040d9b740 1 -- start start 2024-04-02T13:55:14.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.973+0000 7f6040d9b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9126d85b0 0x55c9126f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.973+0000 7f6040d9b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126d8f60 0x55c9126d4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.973+0000 7f6040d9b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9126e60b0 0x55c9126e2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.973+0000 7f6040d9b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f60381acd00 con 0x55c9126d8f60 2024-04-02T13:55:14.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.973+0000 7f6040d9b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f60381ae840 con 0x55c9126d85b0 2024-04-02T13:55:14.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.973+0000 7f6040d9b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f60381ad3d0 con 0x55c9126e60b0 2024-04-02T13:55:14.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.973+0000 7f6036ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126d8f60 0x55c9126d4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.973+0000 7f6036ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126d8f60 0x55c9126d4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47524/0 (socket says 172.21.15.52:47524) 2024-04-02T13:55:14.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.973+0000 7f6036ffd640 1 -- 172.21.15.52:0/2693355758 learned_addr learned my addr 172.21.15.52:0/2693355758 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.973+0000 7f6036ffd640 1 -- 172.21.15.52:0/2693355758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9126e60b0 msgr2=0x55c9126e2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.973+0000 7f6036ffd640 1 --2- 172.21.15.52:0/2693355758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9126e60b0 0x55c9126e2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.973+0000 7f6036ffd640 1 -- 172.21.15.52:0/2693355758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9126d85b0 msgr2=0x55c9126f1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.973+0000 7f60377fe640 1 --2- 172.21.15.52:0/2693355758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9126d85b0 0x55c9126f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.973+0000 7f6036ffd640 1 --2- 172.21.15.52:0/2693355758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9126d85b0 0x55c9126f1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.973+0000 7f6036ffd640 1 -- 172.21.15.52:0/2693355758 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9126d8d00 con 0x55c9126d8f60 2024-04-02T13:55:14.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.973+0000 7f6036ffd640 1 --2- 172.21.15.52:0/2693355758 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126d8f60 0x55c9126d4df0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f602c09d490 tx=0x7f602c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5708d268ac74e86 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.974+0000 7f601d7fa640 1 -- 172.21.15.52:0/2693355758 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c09afa0 con 0x55c9126d8f60 2024-04-02T13:55:14.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.974+0000 7f601d7fa640 1 -- 172.21.15.52:0/2693355758 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f602c0bd0c0 con 0x55c9126d8f60 2024-04-02T13:55:14.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.974+0000 7f601d7fa640 1 -- 172.21.15.52:0/2693355758 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c0ceae0 con 0x55c9126d8f60 2024-04-02T13:55:14.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.974+0000 7f6040d9b740 1 -- 172.21.15.52:0/2693355758 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126d8f60 msgr2=0x55c9126d4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.974+0000 7f6040d9b740 1 --2- 172.21.15.52:0/2693355758 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126d8f60 0x55c9126d4df0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f602c09d490 tx=0x7f602c009310 comp rx=0 tx=0).stop 2024-04-02T13:55:14.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.974+0000 7f6040d9b740 1 -- 172.21.15.52:0/2693355758 shutdown_connections 2024-04-02T13:55:14.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.974+0000 7f6040d9b740 1 --2- 172.21.15.52:0/2693355758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9126e60b0 0x55c9126e2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.974+0000 7f6040d9b740 1 --2- 172.21.15.52:0/2693355758 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126d8f60 0x55c9126d4df0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.974+0000 7f6040d9b740 1 --2- 172.21.15.52:0/2693355758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9126d85b0 0x55c9126f1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.974+0000 7f6040d9b740 1 -- 172.21.15.52:0/2693355758 >> 172.21.15.52:0/2693355758 conn(0x55c9126d2ed0 msgr2=0x55c9126d1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.974+0000 7f6040d9b740 1 -- 172.21.15.52:0/2693355758 shutdown_connections 2024-04-02T13:55:14.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.974+0000 7f6040d9b740 1 -- 172.21.15.52:0/2693355758 wait complete. 2024-04-02T13:55:14.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.974+0000 7f6040d9b740 1 -- start start 2024-04-02T13:55:14.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.975+0000 7f6040d9b740 1 Processor -- start 2024-04-02T13:55:14.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.975+0000 7f6040d9b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126d85b0 0x55c9126e9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.975+0000 7f6040d9b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9126d8f60 0x55c9126ea070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.975+0000 7f6040d9b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9126e60b0 0x55c9126ea5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.975+0000 7f6040d9b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f60381ae170 con 0x55c9126d85b0 2024-04-02T13:55:14.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.975+0000 7f6040d9b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f60381adaa0 con 0x55c9126e60b0 2024-04-02T13:55:14.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.975+0000 7f6040d9b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f60381aace0 con 0x55c9126d8f60 2024-04-02T13:55:14.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.975+0000 7f6040d9b740 10 client.0 _ll_register_callbacks cb 0x7f6030009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:14.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.975+0000 7f60377fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126d85b0 0x55c9126e9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.975+0000 7f60377fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126d85b0 0x55c9126e9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47540/0 (socket says 172.21.15.52:47540) 2024-04-02T13:55:14.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.975+0000 7f60377fe640 1 -- 172.21.15.52:0/3363626035 learned_addr learned my addr 172.21.15.52:0/3363626035 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:14.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.975+0000 7f6037fff640 1 --2- 172.21.15.52:0/3363626035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9126e60b0 0x55c9126ea5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.975+0000 7f6036ffd640 1 --2- 172.21.15.52:0/3363626035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9126d8f60 0x55c9126ea070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.975+0000 7f60377fe640 1 -- 172.21.15.52:0/3363626035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9126d8f60 msgr2=0x55c9126ea070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.975+0000 7f60377fe640 1 --2- 172.21.15.52:0/3363626035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9126d8f60 0x55c9126ea070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.975+0000 7f60377fe640 1 -- 172.21.15.52:0/3363626035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9126e60b0 msgr2=0x55c9126ea5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:14.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.975+0000 7f60377fe640 1 --2- 172.21.15.52:0/3363626035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9126e60b0 0x55c9126ea5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.975+0000 7f60377fe640 1 -- 172.21.15.52:0/3363626035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c91275b9d0 con 0x55c9126d85b0 2024-04-02T13:55:14.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.975+0000 7f6036ffd640 1 --2- 172.21.15.52:0/3363626035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9126d8f60 0x55c9126ea070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:14.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.975+0000 7f60377fe640 1 --2- 172.21.15.52:0/3363626035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126d85b0 0x55c9126e9b30 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f602802cfd0 tx=0x7f6028046e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24658b427ba0404 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:14.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.975+0000 7f6016ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:14.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.976+0000 7f6016ffd640 1 -- 172.21.15.52:0/3363626035 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602804c060 con 0x55c9126d85b0 2024-04-02T13:55:14.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.976+0000 7f6016ffd640 1 -- 172.21.15.52:0/3363626035 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f6028004310 con 0x55c9126d85b0 2024-04-02T13:55:14.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.976+0000 7f6040d9b740 1 -- 172.21.15.52:0/3363626035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f602c0bd0c0 con 0x55c9126d85b0 2024-04-02T13:55:14.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.976+0000 7f6016ffd640 1 -- 172.21.15.52:0/3363626035 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6028053050 con 0x55c9126d85b0 2024-04-02T13:55:14.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.976+0000 7f6040d9b740 10 client.16206 fetch_fsmap learned FSMap version 28 2024-04-02T13:55:14.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.976+0000 7f6040d9b740 1 -- 172.21.15.52:0/3363626035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c91275cc10 con 0x55c9126d85b0 2024-04-02T13:55:14.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.976+0000 7f6016ffd640 1 -- 172.21.15.52:0/3363626035 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f602c0bd0c0 con 0x55c9126d85b0 2024-04-02T13:55:14.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.976+0000 7f6016ffd640 1 -- 172.21.15.52:0/3363626035 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f60280046c0 con 0x55c9126d85b0 2024-04-02T13:55:14.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.976+0000 7f6040d9b740 10 client.16206 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:55:14.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.976+0000 7f6040d9b740 10 client.16206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:14.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.976+0000 7f6040d9b740 20 client.16206 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:14.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.976+0000 7f6040d9b740 1 --2- 172.21.15.52:0/3363626035 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c91275d2e0 0x55c91275f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:14.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.976+0000 7f6040d9b740 4 client.16206 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:55:14.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.976+0000 7f6040d9b740 1 -- 172.21.15.52:0/3363626035 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c912644040 con 0x55c91275d2e0 2024-04-02T13:55:14.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.976+0000 7f6036ffd640 1 --2- 172.21.15.52:0/3363626035 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c91275d2e0 0x55c91275f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:14.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.977+0000 7f6016ffd640 10 client.16206 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:14.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.977+0000 7f6036ffd640 1 --2- 172.21.15.52:0/3363626035 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c91275d2e0 0x55c91275f6d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf29d680cadf6947 server_cookie=d6b97d377cb50549 in_seq=0 out_seq=0 2024-04-02T13:55:14.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.977+0000 7f6016ffd640 1 -- 172.21.15.52:0/3363626035 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c912644040 con 0x55c91275d2e0 2024-04-02T13:55:14.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.977+0000 7f6016ffd640 10 client.16206 handle_command_reply: tid=0 2024-04-02T13:55:14.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.977+0000 7f6040d9b740 1 client.16206 shutdown 2024-04-02T13:55:14.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.977+0000 7f6040d9b740 1 -- 172.21.15.52:0/3363626035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126d85b0 msgr2=0x55c9126e9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.977+0000 7f6040d9b740 1 --2- 172.21.15.52:0/3363626035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126d85b0 0x55c9126e9b30 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f602802cfd0 tx=0x7f6028046e80 comp rx=0 tx=0).stop 2024-04-02T13:55:14.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.977+0000 7f6040d9b740 1 -- 172.21.15.52:0/3363626035 shutdown_connections 2024-04-02T13:55:14.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.977+0000 7f6040d9b740 1 --2- 172.21.15.52:0/3363626035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9126e60b0 0x55c9126ea5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.977+0000 7f6040d9b740 1 --2- 172.21.15.52:0/3363626035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9126d8f60 0x55c9126ea070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.977+0000 7f6040d9b740 1 --2- 172.21.15.52:0/3363626035 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c91275d2e0 0x55c91275f6d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.977+0000 7f6040d9b740 1 --2- 172.21.15.52:0/3363626035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9126d85b0 0x55c9126e9b30 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.977+0000 7f6040d9b740 1 -- 172.21.15.52:0/3363626035 >> 172.21.15.52:0/3363626035 conn(0x55c9126d2ed0 msgr2=0x55c9126e2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.977+0000 7f6016ffd640 0 client.16206 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:14.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.977+0000 7f6040d9b740 1 -- 172.21.15.52:0/3363626035 shutdown_connections 2024-04-02T13:55:14.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.977+0000 7f6040d9b740 1 -- 172.21.15.52:0/3363626035 wait complete. 2024-04-02T13:55:14.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.977+0000 7f6040d9b740 20 client.16206 trim_cache size 0 max 16384 2024-04-02T13:55:14.987 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:55:14.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.978+0000 7f603e1bc640 1 -- 172.21.15.52:0/3915214781 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f60200a3d20 msgr2=0x7f60200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.978+0000 7f603e1bc640 1 --2- 172.21.15.52:0/3915214781 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f60200a3d20 0x7f60200a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f60300092f0 tx=0x7f6030010740 comp rx=0 tx=0).stop 2024-04-02T13:55:14.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.978+0000 7f603e1bc640 1 -- 172.21.15.52:0/3915214781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60380696a0 msgr2=0x7f6038198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:14.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.978+0000 7f603e1bc640 1 --2- 172.21.15.52:0/3915214781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60380696a0 0x7f6038198910 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f602c00e9f0 tx=0x7f602c00eeb0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.978+0000 7f603e1bc640 1 -- 172.21.15.52:0/3915214781 shutdown_connections 2024-04-02T13:55:14.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.978+0000 7f603e1bc640 1 --2- 172.21.15.52:0/3915214781 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f60200a3d20 0x7f60200a61e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.978+0000 7f603e1bc640 1 --2- 172.21.15.52:0/3915214781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6038107a80 0x7f603819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.978+0000 7f603e1bc640 1 --2- 172.21.15.52:0/3915214781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60380696a0 0x7f6038198910 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.978+0000 7f603e1bc640 1 --2- 172.21.15.52:0/3915214781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6038068ce0 0x7f60381983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:14.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.978+0000 7f603e1bc640 1 -- 172.21.15.52:0/3915214781 >> 172.21.15.52:0/3915214781 conn(0x7f6038077b00 msgr2=0x7f60380fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:14.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.978+0000 7f603e1bc640 1 -- 172.21.15.52:0/3915214781 shutdown_connections 2024-04-02T13:55:14.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:14.978+0000 7f603e1bc640 1 -- 172.21.15.52:0/3915214781 wait complete. 2024-04-02T13:55:15.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:15 smithi088 ceph-mon[36528]: pgmap v55: 129 pgs: 129 active+clean; 15 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.2 MiB/s wr, 28 op/s 2024-04-02T13:55:15.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:15 smithi189 ceph-mon[36683]: pgmap v55: 129 pgs: 129 active+clean; 15 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.2 MiB/s wr, 28 op/s 2024-04-02T13:55:15.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:15 smithi052 ceph-mon[32104]: pgmap v55: 129 pgs: 129 active+clean; 15 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.2 MiB/s wr, 28 op/s 2024-04-02T13:55:15.993 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:55:15.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:55:15.993 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:55:16.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.100+0000 7fd77d59d640 1 -- 172.21.15.52:0/1442314436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd7780696c0 msgr2=0x7fd77810e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:16.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.100+0000 7fd77d59d640 1 --2- 172.21.15.52:0/1442314436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd7780696c0 0x7fd77810e960 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd768009cd0 tx=0x7fd76802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.100+0000 7fd77d59d640 1 -- 172.21.15.52:0/1442314436 shutdown_connections 2024-04-02T13:55:16.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.100+0000 7fd77d59d640 1 --2- 172.21.15.52:0/1442314436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd7780696c0 0x7fd77810e960 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.100+0000 7fd77d59d640 1 --2- 172.21.15.52:0/1442314436 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd778068d00 0x7fd778069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.100+0000 7fd77d59d640 1 --2- 172.21.15.52:0/1442314436 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd778107b50 0x7fd778107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.100+0000 7fd77d59d640 1 -- 172.21.15.52:0/1442314436 >> 172.21.15.52:0/1442314436 conn(0x7fd778077b00 msgr2=0x7fd778077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:16.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.100+0000 7fd77d59d640 1 -- 172.21.15.52:0/1442314436 shutdown_connections 2024-04-02T13:55:16.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.100+0000 7fd77d59d640 1 -- 172.21.15.52:0/1442314436 wait complete. 2024-04-02T13:55:16.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.101+0000 7fd77d59d640 1 Processor -- start 2024-04-02T13:55:16.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.101+0000 7fd77d59d640 1 -- start start 2024-04-02T13:55:16.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.101+0000 7fd77d59d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd778068d00 0x7fd7781009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.101+0000 7fd77d59d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd7780696c0 0x7fd778100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.101+0000 7fd77d59d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd778107b50 0x7fd778101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.101+0000 7fd77d59d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd778111ea0 con 0x7fd7780696c0 2024-04-02T13:55:16.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.101+0000 7fd77d59d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd778111d20 con 0x7fd778107b50 2024-04-02T13:55:16.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.101+0000 7fd77d59d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd778112020 con 0x7fd778068d00 2024-04-02T13:55:16.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.101+0000 7fd7767fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd7780696c0 0x7fd778100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.101+0000 7fd7767fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd7780696c0 0x7fd778100f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47564/0 (socket says 172.21.15.52:47564) 2024-04-02T13:55:16.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.101+0000 7fd7767fc640 1 -- 172.21.15.52:0/3455552419 learned_addr learned my addr 172.21.15.52:0/3455552419 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:16.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.102+0000 7fd7777fe640 1 --2- 172.21.15.52:0/3455552419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd778107b50 0x7fd778101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.102+0000 7fd776ffd640 1 --2- 172.21.15.52:0/3455552419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd778068d00 0x7fd7781009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.102+0000 7fd7767fc640 1 -- 172.21.15.52:0/3455552419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd778068d00 msgr2=0x7fd7781009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:16.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.102+0000 7fd7767fc640 1 --2- 172.21.15.52:0/3455552419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd778068d00 0x7fd7781009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.102+0000 7fd7767fc640 1 -- 172.21.15.52:0/3455552419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd778107b50 msgr2=0x7fd778101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:16.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.102+0000 7fd7767fc640 1 --2- 172.21.15.52:0/3455552419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd778107b50 0x7fd778101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.102+0000 7fd7767fc640 1 -- 172.21.15.52:0/3455552419 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7781a68a0 con 0x7fd7780696c0 2024-04-02T13:55:16.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.102+0000 7fd776ffd640 1 --2- 172.21.15.52:0/3455552419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd778068d00 0x7fd7781009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:16.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.102+0000 7fd7767fc640 1 --2- 172.21.15.52:0/3455552419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd7780696c0 0x7fd778100f00 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd76000d950 tx=0x7fd76000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:16.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.102+0000 7fd76ffff640 1 -- 172.21.15.52:0/3455552419 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd760017030 con 0x7fd7780696c0 2024-04-02T13:55:16.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.102+0000 7fd77d59d640 1 -- 172.21.15.52:0/3455552419 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7781a6b90 con 0x7fd7780696c0 2024-04-02T13:55:16.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.102+0000 7fd76ffff640 1 -- 172.21.15.52:0/3455552419 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd760004020 con 0x7fd7780696c0 2024-04-02T13:55:16.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.102+0000 7fd76ffff640 1 -- 172.21.15.52:0/3455552419 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7600047a0 con 0x7fd7780696c0 2024-04-02T13:55:16.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.102+0000 7fd77d59d640 1 -- 172.21.15.52:0/3455552419 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7781a7130 con 0x7fd7780696c0 2024-04-02T13:55:16.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.104+0000 7fd76ffff640 1 -- 172.21.15.52:0/3455552419 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd7600041d0 con 0x7fd7780696c0 2024-04-02T13:55:16.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.105+0000 7fd780188740 1 Processor -- start 2024-04-02T13:55:16.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.105+0000 7fd780188740 1 -- start start 2024-04-02T13:55:16.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.105+0000 7fd76ffff640 1 --2- 172.21.15.52:0/3455552419 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd7640a3d20 0x7fd7640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.105+0000 7fd780188740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8dda2c0 0x5564a8dda6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.105+0000 7fd780188740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564a8ddabe0 0x5564a8de3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.105+0000 7fd780188740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564a8de37c0 0x5564a8de5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.105+0000 7fd780188740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5564a8d3e770 con 0x5564a8dda2c0 2024-04-02T13:55:16.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.105+0000 7fd780188740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5564a8ce6960 con 0x5564a8ddabe0 2024-04-02T13:55:16.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.105+0000 7fd780188740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5564a8d3a8c0 con 0x5564a8de37c0 2024-04-02T13:55:16.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.105+0000 7fd776ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8dda2c0 0x5564a8dda6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.105+0000 7fd776ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8dda2c0 0x5564a8dda6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47576/0 (socket says 172.21.15.52:47576) 2024-04-02T13:55:16.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.105+0000 7fd776ffd640 1 -- 172.21.15.52:0/3090389692 learned_addr learned my addr 172.21.15.52:0/3090389692 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:16.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.105+0000 7fd7767fc640 1 --2- 172.21.15.52:0/3090389692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564a8ddabe0 0x5564a8de3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.105+0000 7fd7777fe640 1 --2- 172.21.15.52:0/3455552419 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd7640a3d20 0x7fd7640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.105+0000 7fd76ffff640 1 -- 172.21.15.52:0/3455552419 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd7600d0520 con 0x7fd7780696c0 2024-04-02T13:55:16.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.106+0000 7fd776ffd640 1 -- 172.21.15.52:0/3090389692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564a8de37c0 msgr2=0x5564a8de5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:16.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.106+0000 7fd776ffd640 1 --2- 172.21.15.52:0/3090389692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564a8de37c0 0x5564a8de5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.106+0000 7fd776ffd640 1 -- 172.21.15.52:0/3090389692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564a8ddabe0 msgr2=0x5564a8de3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:16.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.106+0000 7fd776ffd640 1 --2- 172.21.15.52:0/3090389692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564a8ddabe0 0x5564a8de3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.106+0000 7fd776ffd640 1 -- 172.21.15.52:0/3090389692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564a8b1a7e0 con 0x5564a8dda2c0 2024-04-02T13:55:16.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.106+0000 7fd7767fc640 1 --2- 172.21.15.52:0/3090389692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564a8ddabe0 0x5564a8de3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:16.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.106+0000 7fd776ffd640 1 --2- 172.21.15.52:0/3090389692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8dda2c0 0x5564a8dda6a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd77000aa10 tx=0x7fd77000ae10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed82e0c220dc03d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:16.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.106+0000 7fd7777fe640 1 --2- 172.21.15.52:0/3455552419 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd7640a3d20 0x7fd7640a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd768005cc0 tx=0x7fd768005f10 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:16.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.106+0000 7fd76d7fa640 1 -- 172.21.15.52:0/3090389692 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd770015070 con 0x5564a8dda2c0 2024-04-02T13:55:16.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.106+0000 7fd76d7fa640 1 -- 172.21.15.52:0/3090389692 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd77000cc00 con 0x5564a8dda2c0 2024-04-02T13:55:16.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.106+0000 7fd76d7fa640 1 -- 172.21.15.52:0/3090389692 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd770010590 con 0x5564a8dda2c0 2024-04-02T13:55:16.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.107+0000 7fd780188740 1 -- 172.21.15.52:0/3090389692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8dda2c0 msgr2=0x5564a8dda6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:16.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.107+0000 7fd780188740 1 --2- 172.21.15.52:0/3090389692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8dda2c0 0x5564a8dda6a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd77000aa10 tx=0x7fd77000ae10 comp rx=0 tx=0).stop 2024-04-02T13:55:16.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.107+0000 7fd780188740 1 -- 172.21.15.52:0/3090389692 shutdown_connections 2024-04-02T13:55:16.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.107+0000 7fd780188740 1 --2- 172.21.15.52:0/3090389692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564a8de37c0 0x5564a8de5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.107+0000 7fd780188740 1 --2- 172.21.15.52:0/3090389692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564a8ddabe0 0x5564a8de3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.107+0000 7fd780188740 1 --2- 172.21.15.52:0/3090389692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8dda2c0 0x5564a8dda6a0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.107+0000 7fd780188740 1 -- 172.21.15.52:0/3090389692 >> 172.21.15.52:0/3090389692 conn(0x5564a8dd49f0 msgr2=0x5564a8dd4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:16.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.107+0000 7fd780188740 1 -- 172.21.15.52:0/3090389692 shutdown_connections 2024-04-02T13:55:16.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.107+0000 7fd780188740 1 -- 172.21.15.52:0/3090389692 wait complete. 2024-04-02T13:55:16.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.107+0000 7fd780188740 1 -- start start 2024-04-02T13:55:16.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.107+0000 7fd780188740 1 Processor -- start 2024-04-02T13:55:16.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.108+0000 7fd780188740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564a8dda2c0 0x5564a8de0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.108+0000 7fd780188740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564a8ddabe0 0x5564a8de0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.108+0000 7fd780188740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8de37c0 0x5564a8de12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.108+0000 7fd780188740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5564a8de65b0 con 0x5564a8de37c0 2024-04-02T13:55:16.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.108+0000 7fd780188740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5564a8cf9d20 con 0x5564a8dda2c0 2024-04-02T13:55:16.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.108+0000 7fd780188740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5564a8de6730 con 0x5564a8ddabe0 2024-04-02T13:55:16.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.108+0000 7fd780188740 10 client.0 _ll_register_callbacks cb 0x5564a8cc6a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:16.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.108+0000 7fd7777fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8de37c0 0x5564a8de12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.108+0000 7fd7777fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8de37c0 0x5564a8de12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47592/0 (socket says 172.21.15.52:47592) 2024-04-02T13:55:16.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.108+0000 7fd7777fe640 1 -- 172.21.15.52:0/1569974379 learned_addr learned my addr 172.21.15.52:0/1569974379 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:16.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.108+0000 7fd776ffd640 1 --2- 172.21.15.52:0/1569974379 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564a8dda2c0 0x5564a8de0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.108+0000 7fd7767fc640 1 --2- 172.21.15.52:0/1569974379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564a8ddabe0 0x5564a8de0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.108+0000 7fd776ffd640 1 -- 172.21.15.52:0/1569974379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564a8ddabe0 msgr2=0x5564a8de0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:16.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.108+0000 7fd776ffd640 1 --2- 172.21.15.52:0/1569974379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564a8ddabe0 0x5564a8de0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.108+0000 7fd776ffd640 1 -- 172.21.15.52:0/1569974379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8de37c0 msgr2=0x5564a8de12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:16.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.108+0000 7fd776ffd640 1 --2- 172.21.15.52:0/1569974379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8de37c0 0x5564a8de12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.108+0000 7fd776ffd640 1 -- 172.21.15.52:0/1569974379 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564a8b7af70 con 0x5564a8dda2c0 2024-04-02T13:55:16.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.108+0000 7fd776ffd640 1 --2- 172.21.15.52:0/1569974379 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564a8dda2c0 0x5564a8de0830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd77000c520 tx=0x7fd770012fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba7477237ea46735 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:16.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.108+0000 7fd75e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-02T13:55:16.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.108+0000 7fd75e7fc640 1 -- 172.21.15.52:0/1569974379 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd770015050 con 0x5564a8dda2c0 2024-04-02T13:55:16.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.108+0000 7fd75e7fc640 1 -- 172.21.15.52:0/1569974379 <== mon.2 v2:172.21.15.88:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd77001a660 con 0x5564a8dda2c0 2024-04-02T13:55:16.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.108+0000 7fd780188740 1 -- 172.21.15.52:0/1569974379 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd77000cc00 con 0x5564a8dda2c0 2024-04-02T13:55:16.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.109+0000 7fd75e7fc640 1 -- 172.21.15.52:0/1569974379 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd77000a5e0 con 0x5564a8dda2c0 2024-04-02T13:55:16.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.109+0000 7fd780188740 10 client.25724 fetch_fsmap learned FSMap version 29 2024-04-02T13:55:16.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.109+0000 7fd780188740 1 -- 172.21.15.52:0/1569974379 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5564a8a12170 con 0x5564a8dda2c0 2024-04-02T13:55:16.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.109+0000 7fd75e7fc640 1 -- 172.21.15.52:0/1569974379 <== mon.2 v2:172.21.15.88:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd77000cc00 con 0x5564a8dda2c0 2024-04-02T13:55:16.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.109+0000 7fd75e7fc640 1 -- 172.21.15.52:0/1569974379 <== mon.2 v2:172.21.15.88:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fd770024960 con 0x5564a8dda2c0 2024-04-02T13:55:16.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.109+0000 7fd780188740 10 client.25724 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:55:16.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.109+0000 7fd780188740 10 client.25724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:16.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.109+0000 7fd780188740 20 client.25724 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:16.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.109+0000 7fd780188740 1 --2- 172.21.15.52:0/1569974379 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5564a8df3a20 0x5564a8df5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.109+0000 7fd780188740 4 client.25724 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:16.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.109+0000 7fd780188740 1 -- 172.21.15.52:0/1569974379 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564a8d4bcb0 con 0x5564a8df3a20 2024-04-02T13:55:16.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.109+0000 7fd7767fc640 1 --2- 172.21.15.52:0/1569974379 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5564a8df3a20 0x5564a8df5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.109+0000 7fd7767fc640 1 --2- 172.21.15.52:0/1569974379 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5564a8df3a20 0x5564a8df5e10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9156633137beeb6 server_cookie=5fb249986533b857 in_seq=0 out_seq=0 2024-04-02T13:55:16.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.109+0000 7fd75e7fc640 10 client.25724 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:16.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.111+0000 7fd75e7fc640 1 -- 172.21.15.52:0/1569974379 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5564a8d4bcb0 con 0x5564a8df3a20 2024-04-02T13:55:16.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.111+0000 7fd75e7fc640 10 client.25724 handle_command_reply: tid=0 2024-04-02T13:55:16.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.111+0000 7fd780188740 1 client.25724 shutdown 2024-04-02T13:55:16.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.111+0000 7fd780188740 1 -- 172.21.15.52:0/1569974379 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564a8dda2c0 msgr2=0x5564a8de0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:16.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.111+0000 7fd780188740 1 --2- 172.21.15.52:0/1569974379 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564a8dda2c0 0x5564a8de0830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd77000c520 tx=0x7fd770012fb0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.111+0000 7fd780188740 1 -- 172.21.15.52:0/1569974379 shutdown_connections 2024-04-02T13:55:16.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.111+0000 7fd780188740 1 --2- 172.21.15.52:0/1569974379 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5564a8df3a20 0x5564a8df5e10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.111+0000 7fd780188740 1 --2- 172.21.15.52:0/1569974379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8de37c0 0x5564a8de12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.111+0000 7fd780188740 1 --2- 172.21.15.52:0/1569974379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564a8ddabe0 0x5564a8de0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.111+0000 7fd780188740 1 --2- 172.21.15.52:0/1569974379 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564a8dda2c0 0x5564a8de0830 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.111+0000 7fd75e7fc640 0 client.25724 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:16.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.111+0000 7fd780188740 1 -- 172.21.15.52:0/1569974379 >> 172.21.15.52:0/1569974379 conn(0x5564a8dd49f0 msgr2=0x5564a8dd6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:16.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.111+0000 7fd780188740 1 -- 172.21.15.52:0/1569974379 shutdown_connections 2024-04-02T13:55:16.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.111+0000 7fd780188740 1 -- 172.21.15.52:0/1569974379 wait complete. 2024-04-02T13:55:16.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.111+0000 7fd780188740 20 client.25724 trim_cache size 0 max 16384 2024-04-02T13:55:16.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.130+0000 7fd780188740 1 Processor -- start 2024-04-02T13:55:16.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.130+0000 7fd780188740 1 -- start start 2024-04-02T13:55:16.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.130+0000 7fd780188740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564a8dda2c0 0x5564a8de2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.130+0000 7fd780188740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564a8ddabe0 0x5564a8def470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.130+0000 7fd780188740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8de37c0 0x5564a8dfb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.130+0000 7fd780188740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd7781b1110 con 0x5564a8de37c0 2024-04-02T13:55:16.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.130+0000 7fd780188740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd7781b2c80 con 0x5564a8ddabe0 2024-04-02T13:55:16.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.130+0000 7fd780188740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd7781b17e0 con 0x5564a8dda2c0 2024-04-02T13:55:16.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.131+0000 7fd7777fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8de37c0 0x5564a8dfb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.131+0000 7fd7777fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8de37c0 0x5564a8dfb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47594/0 (socket says 172.21.15.52:47594) 2024-04-02T13:55:16.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.131+0000 7fd7777fe640 1 -- 172.21.15.52:0/2561958641 learned_addr learned my addr 172.21.15.52:0/2561958641 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:16.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.131+0000 7fd7767fc640 1 --2- 172.21.15.52:0/2561958641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564a8ddabe0 0x5564a8def470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.131+0000 7fd776ffd640 1 --2- 172.21.15.52:0/2561958641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564a8dda2c0 0x5564a8de2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.131+0000 7fd7777fe640 1 -- 172.21.15.52:0/2561958641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564a8dda2c0 msgr2=0x5564a8de2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:16.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.131+0000 7fd7777fe640 1 --2- 172.21.15.52:0/2561958641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564a8dda2c0 0x5564a8de2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.131+0000 7fd7777fe640 1 -- 172.21.15.52:0/2561958641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564a8ddabe0 msgr2=0x5564a8def470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:16.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.131+0000 7fd7777fe640 1 --2- 172.21.15.52:0/2561958641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564a8ddabe0 0x5564a8def470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.131+0000 7fd7777fe640 1 -- 172.21.15.52:0/2561958641 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564a8dd9480 con 0x5564a8de37c0 2024-04-02T13:55:16.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.131+0000 7fd776ffd640 1 --2- 172.21.15.52:0/2561958641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564a8dda2c0 0x5564a8de2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:16.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.131+0000 7fd7777fe640 1 --2- 172.21.15.52:0/2561958641 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8de37c0 0x5564a8dfb530 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd76802c700 tx=0x7fd76803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=714c6383d7030c9e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:16.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.131+0000 7fd76cff9640 1 -- 172.21.15.52:0/2561958641 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd768048040 con 0x5564a8de37c0 2024-04-02T13:55:16.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.131+0000 7fd76cff9640 1 -- 172.21.15.52:0/2561958641 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd768053070 con 0x5564a8de37c0 2024-04-02T13:55:16.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.131+0000 7fd76cff9640 1 -- 172.21.15.52:0/2561958641 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd768039040 con 0x5564a8de37c0 2024-04-02T13:55:16.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.132+0000 7fd780188740 1 -- 172.21.15.52:0/2561958641 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8de37c0 msgr2=0x5564a8dfb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:16.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.132+0000 7fd780188740 1 --2- 172.21.15.52:0/2561958641 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8de37c0 0x5564a8dfb530 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd76802c700 tx=0x7fd76803f000 comp rx=0 tx=0).stop 2024-04-02T13:55:16.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.132+0000 7fd780188740 1 -- 172.21.15.52:0/2561958641 shutdown_connections 2024-04-02T13:55:16.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.132+0000 7fd780188740 1 --2- 172.21.15.52:0/2561958641 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8de37c0 0x5564a8dfb530 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.132+0000 7fd780188740 1 --2- 172.21.15.52:0/2561958641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564a8ddabe0 0x5564a8def470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.132+0000 7fd780188740 1 --2- 172.21.15.52:0/2561958641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564a8dda2c0 0x5564a8de2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.132+0000 7fd780188740 1 -- 172.21.15.52:0/2561958641 >> 172.21.15.52:0/2561958641 conn(0x5564a8dd49f0 msgr2=0x5564a8dd3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:16.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.132+0000 7fd780188740 1 -- 172.21.15.52:0/2561958641 shutdown_connections 2024-04-02T13:55:16.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.132+0000 7fd780188740 1 -- 172.21.15.52:0/2561958641 wait complete. 2024-04-02T13:55:16.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.132+0000 7fd780188740 1 -- start start 2024-04-02T13:55:16.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.132+0000 7fd780188740 1 Processor -- start 2024-04-02T13:55:16.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.133+0000 7fd780188740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564a8dda2c0 0x5564a8de7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.133+0000 7fd780188740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8ddabe0 0x5564a8de7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.133+0000 7fd780188740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564a8de37c0 0x5564a8e58da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.133+0000 7fd780188740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd7781b2580 con 0x5564a8ddabe0 2024-04-02T13:55:16.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.133+0000 7fd780188740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd7781b1eb0 con 0x5564a8dda2c0 2024-04-02T13:55:16.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.133+0000 7fd780188740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd7781af0f0 con 0x5564a8de37c0 2024-04-02T13:55:16.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.133+0000 7fd780188740 10 client.0 _ll_register_callbacks cb 0x7fd77000a860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:16.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.133+0000 7fd7767fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8ddabe0 0x5564a8de7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.133+0000 7fd776ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564a8dda2c0 0x5564a8de7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.133+0000 7fd776ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564a8dda2c0 0x5564a8de7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:38550/0 (socket says 172.21.15.52:38550) 2024-04-02T13:55:16.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.133+0000 7fd776ffd640 1 -- 172.21.15.52:0/2252870628 learned_addr learned my addr 172.21.15.52:0/2252870628 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:16.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.133+0000 7fd7777fe640 1 --2- 172.21.15.52:0/2252870628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564a8de37c0 0x5564a8e58da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.133+0000 7fd7767fc640 1 -- 172.21.15.52:0/2252870628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564a8de37c0 msgr2=0x5564a8e58da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:16.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.133+0000 7fd7767fc640 1 --2- 172.21.15.52:0/2252870628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564a8de37c0 0x5564a8e58da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.133+0000 7fd7767fc640 1 -- 172.21.15.52:0/2252870628 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564a8dda2c0 msgr2=0x5564a8de7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:16.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.133+0000 7fd7767fc640 1 --2- 172.21.15.52:0/2252870628 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564a8dda2c0 0x5564a8de7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.133+0000 7fd7767fc640 1 -- 172.21.15.52:0/2252870628 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564a8e592e0 con 0x5564a8ddabe0 2024-04-02T13:55:16.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.133+0000 7fd776ffd640 1 --2- 172.21.15.52:0/2252870628 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564a8dda2c0 0x5564a8de7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:16.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.133+0000 7fd7767fc640 1 --2- 172.21.15.52:0/2252870628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8ddabe0 0x5564a8de7f90 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd7600cd930 tx=0x7fd760021c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246b569e745acdf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:16.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.133+0000 7fd75e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:16.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.133+0000 7fd75e7fc640 1 -- 172.21.15.52:0/2252870628 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76009c180 con 0x5564a8ddabe0 2024-04-02T13:55:16.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.133+0000 7fd75e7fc640 1 -- 172.21.15.52:0/2252870628 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd7600cdbb0 con 0x5564a8ddabe0 2024-04-02T13:55:16.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.133+0000 7fd780188740 1 -- 172.21.15.52:0/2252870628 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd768053070 con 0x5564a8ddabe0 2024-04-02T13:55:16.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.134+0000 7fd75e7fc640 1 -- 172.21.15.52:0/2252870628 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd7600a0530 con 0x5564a8ddabe0 2024-04-02T13:55:16.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.134+0000 7fd780188740 10 client.16224 fetch_fsmap learned FSMap version 29 2024-04-02T13:55:16.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.134+0000 7fd780188740 1 -- 172.21.15.52:0/2252870628 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5564a8e5a5f0 con 0x5564a8ddabe0 2024-04-02T13:55:16.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.134+0000 7fd75e7fc640 1 -- 172.21.15.52:0/2252870628 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd768053070 con 0x5564a8ddabe0 2024-04-02T13:55:16.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.134+0000 7fd75e7fc640 1 -- 172.21.15.52:0/2252870628 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fd760021ca0 con 0x5564a8ddabe0 2024-04-02T13:55:16.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.134+0000 7fd780188740 10 client.16224 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:55:16.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.134+0000 7fd780188740 10 client.16224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:16.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.134+0000 7fd780188740 20 client.16224 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:16.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.134+0000 7fd780188740 1 --2- 172.21.15.52:0/2252870628 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5564a8e5ad50 0x5564a8e5d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.134+0000 7fd780188740 4 client.16224 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:55:16.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.134+0000 7fd780188740 1 -- 172.21.15.52:0/2252870628 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5564a8d46000 con 0x5564a8e5ad50 2024-04-02T13:55:16.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.134+0000 7fd776ffd640 1 --2- 172.21.15.52:0/2252870628 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5564a8e5ad50 0x5564a8e5d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.134+0000 7fd776ffd640 1 --2- 172.21.15.52:0/2252870628 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5564a8e5ad50 0x5564a8e5d170 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e35300f72fa40ce server_cookie=1eb534809a3f9171 in_seq=0 out_seq=0 2024-04-02T13:55:16.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.134+0000 7fd75e7fc640 10 client.16224 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:16.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.136+0000 7fd75e7fc640 1 -- 172.21.15.52:0/2252870628 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5564a8d46000 con 0x5564a8e5ad50 2024-04-02T13:55:16.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.136+0000 7fd75e7fc640 10 client.16224 handle_command_reply: tid=0 2024-04-02T13:55:16.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.136+0000 7fd780188740 1 client.16224 shutdown 2024-04-02T13:55:16.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.136+0000 7fd780188740 1 -- 172.21.15.52:0/2252870628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8ddabe0 msgr2=0x5564a8de7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:16.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.136+0000 7fd780188740 1 --2- 172.21.15.52:0/2252870628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8ddabe0 0x5564a8de7f90 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd7600cd930 tx=0x7fd760021c30 comp rx=0 tx=0).stop 2024-04-02T13:55:16.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.136+0000 7fd780188740 1 -- 172.21.15.52:0/2252870628 shutdown_connections 2024-04-02T13:55:16.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.136+0000 7fd780188740 1 --2- 172.21.15.52:0/2252870628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564a8de37c0 0x5564a8e58da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.136+0000 7fd780188740 1 --2- 172.21.15.52:0/2252870628 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5564a8e5ad50 0x5564a8e5d170 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.136+0000 7fd780188740 1 --2- 172.21.15.52:0/2252870628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a8ddabe0 0x5564a8de7f90 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.136+0000 7fd780188740 1 --2- 172.21.15.52:0/2252870628 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5564a8dda2c0 0x5564a8de7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.136+0000 7fd75e7fc640 0 client.16224 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:16.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.136+0000 7fd780188740 1 -- 172.21.15.52:0/2252870628 >> 172.21.15.52:0/2252870628 conn(0x5564a8dd49f0 msgr2=0x5564a8de0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:16.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.136+0000 7fd780188740 1 -- 172.21.15.52:0/2252870628 shutdown_connections 2024-04-02T13:55:16.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.136+0000 7fd780188740 1 -- 172.21.15.52:0/2252870628 wait complete. 2024-04-02T13:55:16.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.136+0000 7fd780188740 20 client.16224 trim_cache size 0 max 16384 2024-04-02T13:55:16.146 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:55:16.146 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-04-02T13:55:16.146 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "043737e3-e755-4d51-b4ac-7e900b33d231", 2024-04-02T13:55:16.146 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-04-02T13:55:16.146 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:55:16.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.137+0000 7fd77d59d640 1 -- 172.21.15.52:0/3455552419 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd7640a3d20 msgr2=0x7fd7640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:16.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.137+0000 7fd77d59d640 1 --2- 172.21.15.52:0/3455552419 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd7640a3d20 0x7fd7640a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd768005cc0 tx=0x7fd768005f10 comp rx=0 tx=0).stop 2024-04-02T13:55:16.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.137+0000 7fd77d59d640 1 -- 172.21.15.52:0/3455552419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd7780696c0 msgr2=0x7fd778100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:16.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.137+0000 7fd77d59d640 1 --2- 172.21.15.52:0/3455552419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd7780696c0 0x7fd778100f00 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd76000d950 tx=0x7fd76000de10 comp rx=0 tx=0).stop 2024-04-02T13:55:16.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.137+0000 7fd77d59d640 1 -- 172.21.15.52:0/3455552419 shutdown_connections 2024-04-02T13:55:16.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.137+0000 7fd77d59d640 1 --2- 172.21.15.52:0/3455552419 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd7640a3d20 0x7fd7640a61e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.137+0000 7fd77d59d640 1 --2- 172.21.15.52:0/3455552419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd778107b50 0x7fd778101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.137+0000 7fd77d59d640 1 --2- 172.21.15.52:0/3455552419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd7780696c0 0x7fd778100f00 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.137+0000 7fd77d59d640 1 --2- 172.21.15.52:0/3455552419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd778068d00 0x7fd7781009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.137+0000 7fd77d59d640 1 -- 172.21.15.52:0/3455552419 >> 172.21.15.52:0/3455552419 conn(0x7fd778077b00 msgr2=0x7fd7780fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:16.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.137+0000 7fd77d59d640 1 -- 172.21.15.52:0/3455552419 shutdown_connections 2024-04-02T13:55:16.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.137+0000 7fd77d59d640 1 -- 172.21.15.52:0/3455552419 wait complete. 2024-04-02T13:55:16.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:55:16.151 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:55:16.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.298+0000 7f2a252d6640 1 -- 172.21.15.52:0/647623404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a20106980 msgr2=0x7f2a20106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:16.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.298+0000 7f2a252d6640 1 --2- 172.21.15.52:0/647623404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a20106980 0x7f2a20106d60 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009930 tx=0x7f2a1002bc30 comp rx=0 tx=0).stop 2024-04-02T13:55:16.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.298+0000 7f2a252d6640 1 -- 172.21.15.52:0/647623404 shutdown_connections 2024-04-02T13:55:16.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.298+0000 7f2a252d6640 1 --2- 172.21.15.52:0/647623404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a20101330 0x7f2a2010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.298+0000 7f2a252d6640 1 --2- 172.21.15.52:0/647623404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a20100970 0x7f2a20100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.298+0000 7f2a252d6640 1 --2- 172.21.15.52:0/647623404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a20106980 0x7f2a20106d60 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.298+0000 7f2a252d6640 1 -- 172.21.15.52:0/647623404 >> 172.21.15.52:0/647623404 conn(0x7f2a20077b00 msgr2=0x7f2a20077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:16.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.298+0000 7f2a252d6640 1 -- 172.21.15.52:0/647623404 shutdown_connections 2024-04-02T13:55:16.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.298+0000 7f2a252d6640 1 -- 172.21.15.52:0/647623404 wait complete. 2024-04-02T13:55:16.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.298+0000 7f2a252d6640 1 Processor -- start 2024-04-02T13:55:16.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.298+0000 7f2a252d6640 1 -- start start 2024-04-02T13:55:16.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.299+0000 7f2a252d6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a20100970 0x7f2a20198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.299+0000 7f2a252d6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a20101330 0x7f2a201989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.299+0000 7f2a252d6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a20106980 0x7f2a2019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.299+0000 7f2a252d6640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2a20110d90 con 0x7f2a20101330 2024-04-02T13:55:16.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.299+0000 7f2a252d6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2a20110c10 con 0x7f2a20106980 2024-04-02T13:55:16.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.299+0000 7f2a252d6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2a20110f10 con 0x7f2a20100970 2024-04-02T13:55:16.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.299+0000 7f2a17fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a20101330 0x7f2a201989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.299+0000 7f2a17fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a20101330 0x7f2a201989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47620/0 (socket says 172.21.15.52:47620) 2024-04-02T13:55:16.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.299+0000 7f2a17fff640 1 -- 172.21.15.52:0/562618519 learned_addr learned my addr 172.21.15.52:0/562618519 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:16.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.299+0000 7f2a1effd640 1 --2- 172.21.15.52:0/562618519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a20100970 0x7f2a20198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.299+0000 7f2a1f7fe640 1 --2- 172.21.15.52:0/562618519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a20106980 0x7f2a2019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.299+0000 7f2a17fff640 1 -- 172.21.15.52:0/562618519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a20100970 msgr2=0x7f2a20198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:16.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.299+0000 7f2a17fff640 1 --2- 172.21.15.52:0/562618519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a20100970 0x7f2a20198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.299+0000 7f2a17fff640 1 -- 172.21.15.52:0/562618519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a20106980 msgr2=0x7f2a2019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:16.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.299+0000 7f2a17fff640 1 --2- 172.21.15.52:0/562618519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a20106980 0x7f2a2019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.299+0000 7f2a17fff640 1 -- 172.21.15.52:0/562618519 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a2019d380 con 0x7f2a20101330 2024-04-02T13:55:16.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.299+0000 7f2a17fff640 1 --2- 172.21.15.52:0/562618519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a20101330 0x7f2a201989c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800e9f0 tx=0x7f2a0800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:16.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.299+0000 7f2a1cff9640 1 -- 172.21.15.52:0/562618519 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0800cd80 con 0x7f2a20101330 2024-04-02T13:55:16.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.299+0000 7f2a1cff9640 1 -- 172.21.15.52:0/562618519 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a08005500 con 0x7f2a20101330 2024-04-02T13:55:16.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.300+0000 7f2a1cff9640 1 -- 172.21.15.52:0/562618519 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08010690 con 0x7f2a20101330 2024-04-02T13:55:16.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.300+0000 7f2a252d6640 1 -- 172.21.15.52:0/562618519 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a2019d670 con 0x7f2a20101330 2024-04-02T13:55:16.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.300+0000 7f2a252d6640 1 -- 172.21.15.52:0/562618519 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a201a4f00 con 0x7f2a20101330 2024-04-02T13:55:16.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.302+0000 7f2a1cff9640 1 -- 172.21.15.52:0/562618519 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2a080046c0 con 0x7f2a20101330 2024-04-02T13:55:16.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.303+0000 7f2a27f97740 1 Processor -- start 2024-04-02T13:55:16.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.303+0000 7f2a27f97740 1 -- start start 2024-04-02T13:55:16.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.303+0000 7f2a27f97740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c851938d0 0x564c85193cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.303+0000 7f2a27f97740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c851941f0 0x564c8519c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.303+0000 7f2a27f97740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564c8519cdd0 0x564c8519f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.303+0000 7f2a27f97740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564c850f7830 con 0x564c851938d0 2024-04-02T13:55:16.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.303+0000 7f2a27f97740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564c8509f820 con 0x564c8519cdd0 2024-04-02T13:55:16.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.303+0000 7f2a27f97740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564c850f37e0 con 0x564c851941f0 2024-04-02T13:55:16.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.303+0000 7f2a1cff9640 1 --2- 172.21.15.52:0/562618519 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2a0c0a3d20 0x7f2a0c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.303+0000 7f2a17fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564c8519cdd0 0x564c8519f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.303+0000 7f2a1effd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c851941f0 0x564c8519c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.303+0000 7f2a1f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c851938d0 0x564c85193cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.303+0000 7f2a1effd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c851941f0 0x564c8519c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.52:60234/0 (socket says 172.21.15.52:60234) 2024-04-02T13:55:16.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.303+0000 7f2a1f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c851938d0 0x564c85193cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47626/0 (socket says 172.21.15.52:47626) 2024-04-02T13:55:16.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.303+0000 7f2a1effd640 1 -- 172.21.15.52:0/917526585 learned_addr learned my addr 172.21.15.52:0/917526585 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:16.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.303+0000 7f2a1f7fe640 1 --2- 172.21.15.52:0/562618519 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2a0c0a3d20 0x7f2a0c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.303+0000 7f2a1cff9640 1 -- 172.21.15.52:0/562618519 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f2a080d0f90 con 0x7f2a20101330 2024-04-02T13:55:16.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.303+0000 7f2a1effd640 1 -- 172.21.15.52:0/917526585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564c8519cdd0 msgr2=0x564c8519f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:16.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.303+0000 7f2a1effd640 1 --2- 172.21.15.52:0/917526585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564c8519cdd0 0x564c8519f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.303+0000 7f2a1effd640 1 -- 172.21.15.52:0/917526585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c851938d0 msgr2=0x564c85193cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:16.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.303+0000 7f2a1effd640 1 --2- 172.21.15.52:0/917526585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c851938d0 0x564c85193cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.304+0000 7f2a1effd640 1 -- 172.21.15.52:0/917526585 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c84ed57d0 con 0x564c851941f0 2024-04-02T13:55:16.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.304+0000 7f2a1f7fe640 1 --2- 172.21.15.52:0/917526585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c851938d0 0x564c85193cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:16.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.304+0000 7f2a1effd640 1 --2- 172.21.15.52:0/917526585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c851941f0 0x564c8519c890 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2a1002cfd0 tx=0x7f2a100095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79dfa78a2d6412cb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:16.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.304+0000 7f2a1f7fe640 1 --2- 172.21.15.52:0/562618519 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2a0c0a3d20 0x7f2a0c0a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2a18005fd0 tx=0x7f2a18009500 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:16.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.304+0000 7f2a157fa640 1 -- 172.21.15.52:0/917526585 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a10042070 con 0x564c851941f0 2024-04-02T13:55:16.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.304+0000 7f2a157fa640 1 -- 172.21.15.52:0/917526585 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a10004030 con 0x564c851941f0 2024-04-02T13:55:16.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.304+0000 7f2a157fa640 1 -- 172.21.15.52:0/917526585 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a10004bf0 con 0x564c851941f0 2024-04-02T13:55:16.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.305+0000 7f2a27f97740 1 -- 172.21.15.52:0/917526585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c851941f0 msgr2=0x564c8519c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:16.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.305+0000 7f2a27f97740 1 --2- 172.21.15.52:0/917526585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c851941f0 0x564c8519c890 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2a1002cfd0 tx=0x7f2a100095f0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.305+0000 7f2a27f97740 1 -- 172.21.15.52:0/917526585 shutdown_connections 2024-04-02T13:55:16.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.305+0000 7f2a27f97740 1 --2- 172.21.15.52:0/917526585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564c8519cdd0 0x564c8519f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.305+0000 7f2a27f97740 1 --2- 172.21.15.52:0/917526585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c851941f0 0x564c8519c890 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.305+0000 7f2a27f97740 1 --2- 172.21.15.52:0/917526585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c851938d0 0x564c85193cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.305+0000 7f2a27f97740 1 -- 172.21.15.52:0/917526585 >> 172.21.15.52:0/917526585 conn(0x564c8518e000 msgr2=0x564c8518e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:16.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.305+0000 7f2a27f97740 1 -- 172.21.15.52:0/917526585 shutdown_connections 2024-04-02T13:55:16.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.305+0000 7f2a27f97740 1 -- 172.21.15.52:0/917526585 wait complete. 2024-04-02T13:55:16.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.305+0000 7f2a27f97740 1 -- start start 2024-04-02T13:55:16.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.305+0000 7f2a27f97740 1 Processor -- start 2024-04-02T13:55:16.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a27f97740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564c851938d0 0x564c8519c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a27f97740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c851941f0 0x564c85199a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a27f97740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c8519cdd0 0x564c85199f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a27f97740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564c8519fb60 con 0x564c851941f0 2024-04-02T13:55:16.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a27f97740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564c850b2d30 con 0x564c851938d0 2024-04-02T13:55:16.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a27f97740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564c8519fce0 con 0x564c8519cdd0 2024-04-02T13:55:16.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a27f97740 10 client.0 _ll_register_callbacks cb 0x564c850b2720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:16.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a17fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c851941f0 0x564c85199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a17fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c851941f0 0x564c85199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47638/0 (socket says 172.21.15.52:47638) 2024-04-02T13:55:16.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a17fff640 1 -- 172.21.15.52:0/3749653425 learned_addr learned my addr 172.21.15.52:0/3749653425 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:16.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a1effd640 1 --2- 172.21.15.52:0/3749653425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564c851938d0 0x564c8519c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a1f7fe640 1 --2- 172.21.15.52:0/3749653425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c8519cdd0 0x564c85199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a17fff640 1 -- 172.21.15.52:0/3749653425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c8519cdd0 msgr2=0x564c85199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:16.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a17fff640 1 --2- 172.21.15.52:0/3749653425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c8519cdd0 0x564c85199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a17fff640 1 -- 172.21.15.52:0/3749653425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564c851938d0 msgr2=0x564c8519c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:16.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a17fff640 1 --2- 172.21.15.52:0/3749653425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564c851938d0 0x564c8519c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a17fff640 1 -- 172.21.15.52:0/3749653425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c84e0b3f0 con 0x564c851941f0 2024-04-02T13:55:16.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a1f7fe640 1 --2- 172.21.15.52:0/3749653425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c8519cdd0 0x564c85199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:16.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a17fff640 1 --2- 172.21.15.52:0/3749653425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c851941f0 0x564c85199a00 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2a0809cb20 tx=0x7f2a0806f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eadec6ebe4340746 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:16.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a067fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:16.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a067fc640 1 -- 172.21.15.52:0/3749653425 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a080b5070 con 0x564c851941f0 2024-04-02T13:55:16.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a067fc640 1 -- 172.21.15.52:0/3749653425 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f2a080a5370 con 0x564c851941f0 2024-04-02T13:55:16.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a27f97740 1 -- 172.21.15.52:0/3749653425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a10004030 con 0x564c851941f0 2024-04-02T13:55:16.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a067fc640 1 -- 172.21.15.52:0/3749653425 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2a080ce7c0 con 0x564c851941f0 2024-04-02T13:55:16.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a27f97740 10 client.16236 fetch_fsmap learned FSMap version 29 2024-04-02T13:55:16.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.306+0000 7f2a27f97740 1 -- 172.21.15.52:0/3749653425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564c84dcb0f0 con 0x564c851941f0 2024-04-02T13:55:16.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.307+0000 7f2a067fc640 1 -- 172.21.15.52:0/3749653425 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a10004030 con 0x564c851941f0 2024-04-02T13:55:16.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.307+0000 7f2a067fc640 1 -- 172.21.15.52:0/3749653425 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f2a0809c170 con 0x564c851941f0 2024-04-02T13:55:16.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.307+0000 7f2a27f97740 10 client.16236 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:55:16.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.307+0000 7f2a27f97740 10 client.16236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:16.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.307+0000 7f2a27f97740 20 client.16236 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:16.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.307+0000 7f2a27f97740 1 --2- 172.21.15.52:0/3749653425 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564c851acdc0 0x564c851af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.307+0000 7f2a27f97740 4 client.16236 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:16.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.307+0000 7f2a27f97740 1 -- 172.21.15.52:0/3749653425 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c85104d70 con 0x564c851acdc0 2024-04-02T13:55:16.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.307+0000 7f2a1effd640 1 --2- 172.21.15.52:0/3749653425 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564c851acdc0 0x564c851af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.307+0000 7f2a067fc640 10 client.16236 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:16.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.307+0000 7f2a1effd640 1 --2- 172.21.15.52:0/3749653425 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564c851acdc0 0x564c851af1d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f666b997cab08bb server_cookie=63b2ed896147731c in_seq=0 out_seq=0 2024-04-02T13:55:16.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.309+0000 7f2a067fc640 1 -- 172.21.15.52:0/3749653425 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564c85104d70 con 0x564c851acdc0 2024-04-02T13:55:16.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.309+0000 7f2a067fc640 10 client.16236 handle_command_reply: tid=0 2024-04-02T13:55:16.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.309+0000 7f2a27f97740 1 client.16236 shutdown 2024-04-02T13:55:16.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.309+0000 7f2a27f97740 1 -- 172.21.15.52:0/3749653425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c851941f0 msgr2=0x564c85199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:16.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.309+0000 7f2a27f97740 1 --2- 172.21.15.52:0/3749653425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c851941f0 0x564c85199a00 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2a0809cb20 tx=0x7f2a0806f740 comp rx=0 tx=0).stop 2024-04-02T13:55:16.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.309+0000 7f2a27f97740 1 -- 172.21.15.52:0/3749653425 shutdown_connections 2024-04-02T13:55:16.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.309+0000 7f2a27f97740 1 --2- 172.21.15.52:0/3749653425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c8519cdd0 0x564c85199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.309+0000 7f2a27f97740 1 --2- 172.21.15.52:0/3749653425 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564c851acdc0 0x564c851af1d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.309+0000 7f2a27f97740 1 --2- 172.21.15.52:0/3749653425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c851941f0 0x564c85199a00 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.309+0000 7f2a27f97740 1 --2- 172.21.15.52:0/3749653425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564c851938d0 0x564c8519c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.309+0000 7f2a067fc640 0 client.16236 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:16.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.309+0000 7f2a27f97740 1 -- 172.21.15.52:0/3749653425 >> 172.21.15.52:0/3749653425 conn(0x564c8518e000 msgr2=0x564c8518fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:16.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.309+0000 7f2a27f97740 1 -- 172.21.15.52:0/3749653425 shutdown_connections 2024-04-02T13:55:16.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.309+0000 7f2a27f97740 1 -- 172.21.15.52:0/3749653425 wait complete. 2024-04-02T13:55:16.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.309+0000 7f2a27f97740 20 client.16236 trim_cache size 0 max 16384 2024-04-02T13:55:16.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.327+0000 7f2a27f97740 1 Processor -- start 2024-04-02T13:55:16.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.327+0000 7f2a27f97740 1 -- start start 2024-04-02T13:55:16.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.327+0000 7f2a27f97740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564c851938d0 0x564c851a8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.327+0000 7f2a27f97740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c851941f0 0x564c85198940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.327+0000 7f2a27f97740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c8519cdd0 0x564c8519c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.327+0000 7f2a27f97740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2a201aed20 con 0x564c851941f0 2024-04-02T13:55:16.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.327+0000 7f2a27f97740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2a201b0890 con 0x564c851938d0 2024-04-02T13:55:16.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.327+0000 7f2a27f97740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2a201af3f0 con 0x564c8519cdd0 2024-04-02T13:55:16.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.327+0000 7f2a17fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c851941f0 0x564c85198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.327+0000 7f2a1effd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564c851938d0 0x564c851a8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.327+0000 7f2a17fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c851941f0 0x564c85198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47644/0 (socket says 172.21.15.52:47644) 2024-04-02T13:55:16.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.327+0000 7f2a17fff640 1 -- 172.21.15.52:0/584098985 learned_addr learned my addr 172.21.15.52:0/584098985 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:16.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.327+0000 7f2a1f7fe640 1 --2- 172.21.15.52:0/584098985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c8519cdd0 0x564c8519c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.327+0000 7f2a17fff640 1 -- 172.21.15.52:0/584098985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c8519cdd0 msgr2=0x564c8519c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:16.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.327+0000 7f2a17fff640 1 --2- 172.21.15.52:0/584098985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c8519cdd0 0x564c8519c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.328+0000 7f2a17fff640 1 -- 172.21.15.52:0/584098985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564c851938d0 msgr2=0x564c851a8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:16.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.328+0000 7f2a17fff640 1 --2- 172.21.15.52:0/584098985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564c851938d0 0x564c851a8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.328+0000 7f2a17fff640 1 -- 172.21.15.52:0/584098985 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c8518de30 con 0x564c851941f0 2024-04-02T13:55:16.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.328+0000 7f2a1effd640 1 --2- 172.21.15.52:0/584098985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564c851938d0 0x564c851a8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:16.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.328+0000 7f2a17fff640 1 --2- 172.21.15.52:0/584098985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c851941f0 0x564c85198940 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2a0809ae40 tx=0x7f2a080bd2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548153ab9b344f5a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:16.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.328+0000 7f2a14ff9640 1 -- 172.21.15.52:0/584098985 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a080ce7c0 con 0x564c851941f0 2024-04-02T13:55:16.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.328+0000 7f2a14ff9640 1 -- 172.21.15.52:0/584098985 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a080a0f40 con 0x564c851941f0 2024-04-02T13:55:16.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.328+0000 7f2a14ff9640 1 -- 172.21.15.52:0/584098985 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0809c240 con 0x564c851941f0 2024-04-02T13:55:16.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.329+0000 7f2a27f97740 1 -- 172.21.15.52:0/584098985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c851941f0 msgr2=0x564c85198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:16.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.329+0000 7f2a27f97740 1 --2- 172.21.15.52:0/584098985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c851941f0 0x564c85198940 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2a0809ae40 tx=0x7f2a080bd2c0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.329+0000 7f2a27f97740 1 -- 172.21.15.52:0/584098985 shutdown_connections 2024-04-02T13:55:16.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.329+0000 7f2a27f97740 1 --2- 172.21.15.52:0/584098985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c8519cdd0 0x564c8519c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.329+0000 7f2a27f97740 1 --2- 172.21.15.52:0/584098985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c851941f0 0x564c85198940 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.329+0000 7f2a27f97740 1 --2- 172.21.15.52:0/584098985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564c851938d0 0x564c851a8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.329+0000 7f2a27f97740 1 -- 172.21.15.52:0/584098985 >> 172.21.15.52:0/584098985 conn(0x564c8518e000 msgr2=0x564c851950c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:16.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.329+0000 7f2a27f97740 1 -- 172.21.15.52:0/584098985 shutdown_connections 2024-04-02T13:55:16.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.329+0000 7f2a27f97740 1 -- 172.21.15.52:0/584098985 wait complete. 2024-04-02T13:55:16.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.329+0000 7f2a27f97740 1 -- start start 2024-04-02T13:55:16.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.329+0000 7f2a27f97740 1 Processor -- start 2024-04-02T13:55:16.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.329+0000 7f2a27f97740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c851938d0 0x564c851a5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.329+0000 7f2a27f97740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564c851941f0 0x564c85196270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.329+0000 7f2a27f97740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c8519cdd0 0x564c851967b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.329+0000 7f2a27f97740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2a201b0190 con 0x564c8519cdd0 2024-04-02T13:55:16.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.329+0000 7f2a27f97740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2a201afac0 con 0x564c851941f0 2024-04-02T13:55:16.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.329+0000 7f2a27f97740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2a201acd00 con 0x564c851938d0 2024-04-02T13:55:16.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.329+0000 7f2a27f97740 10 client.0 _ll_register_callbacks cb 0x7f2a0809ca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:16.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.329+0000 7f2a1f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c8519cdd0 0x564c851967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.329+0000 7f2a1f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c8519cdd0 0x564c851967b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47660/0 (socket says 172.21.15.52:47660) 2024-04-02T13:55:16.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.329+0000 7f2a17fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564c851941f0 0x564c85196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.329+0000 7f2a1f7fe640 1 -- 172.21.15.52:0/2574110619 learned_addr learned my addr 172.21.15.52:0/2574110619 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:16.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.329+0000 7f2a1effd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c851938d0 0x564c851a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.330+0000 7f2a17fff640 1 -- 172.21.15.52:0/2574110619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c851938d0 msgr2=0x564c851a5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:16.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.330+0000 7f2a17fff640 1 --2- 172.21.15.52:0/2574110619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c851938d0 0x564c851a5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.330+0000 7f2a17fff640 1 -- 172.21.15.52:0/2574110619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c8519cdd0 msgr2=0x564c851967b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:16.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.330+0000 7f2a17fff640 1 --2- 172.21.15.52:0/2574110619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c8519cdd0 0x564c851967b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.330+0000 7f2a17fff640 1 -- 172.21.15.52:0/2574110619 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c85196cf0 con 0x564c851941f0 2024-04-02T13:55:16.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.330+0000 7f2a17fff640 1 --2- 172.21.15.52:0/2574110619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564c851941f0 0x564c85196270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x564c8519ac40 tx=0x7f2a080a5700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8dcef54de5867f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:16.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.330+0000 7f2a067fc640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-02T13:55:16.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.330+0000 7f2a067fc640 1 -- 172.21.15.52:0/2574110619 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a080ce7c0 con 0x564c851941f0 2024-04-02T13:55:16.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.330+0000 7f2a27f97740 1 -- 172.21.15.52:0/2574110619 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a080a0f40 con 0x564c851941f0 2024-04-02T13:55:16.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.330+0000 7f2a067fc640 1 -- 172.21.15.52:0/2574110619 <== mon.2 v2:172.21.15.88:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f2a0809bfe0 con 0x564c851941f0 2024-04-02T13:55:16.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.330+0000 7f2a067fc640 1 -- 172.21.15.52:0/2574110619 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2a080a5800 con 0x564c851941f0 2024-04-02T13:55:16.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.330+0000 7f2a27f97740 10 client.25748 fetch_fsmap learned FSMap version 29 2024-04-02T13:55:16.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.330+0000 7f2a27f97740 1 -- 172.21.15.52:0/2574110619 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564c85212f80 con 0x564c851941f0 2024-04-02T13:55:16.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.330+0000 7f2a067fc640 1 -- 172.21.15.52:0/2574110619 <== mon.2 v2:172.21.15.88:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a080a0f40 con 0x564c851941f0 2024-04-02T13:55:16.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.331+0000 7f2a067fc640 1 -- 172.21.15.52:0/2574110619 <== mon.2 v2:172.21.15.88:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f2a080a20d0 con 0x564c851941f0 2024-04-02T13:55:16.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.331+0000 7f2a27f97740 10 client.25748 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:55:16.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.331+0000 7f2a27f97740 10 client.25748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:16.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.331+0000 7f2a27f97740 20 client.25748 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:16.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.331+0000 7f2a27f97740 1 --2- 172.21.15.52:0/2574110619 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564c85213450 0x564c85215840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:16.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.331+0000 7f2a27f97740 4 client.25748 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:55:16.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.331+0000 7f2a27f97740 1 -- 172.21.15.52:0/2574110619 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564c850ff0c0 con 0x564c85213450 2024-04-02T13:55:16.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.331+0000 7f2a1effd640 1 --2- 172.21.15.52:0/2574110619 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564c85213450 0x564c85215840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:16.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.331+0000 7f2a067fc640 10 client.25748 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:16.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.331+0000 7f2a1effd640 1 --2- 172.21.15.52:0/2574110619 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564c85213450 0x564c85215840 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9fa13a2ba59e8ab server_cookie=cc26060ff81b5285 in_seq=0 out_seq=0 2024-04-02T13:55:16.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.333+0000 7f2a067fc640 1 -- 172.21.15.52:0/2574110619 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x564c850ff0c0 con 0x564c85213450 2024-04-02T13:55:16.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.333+0000 7f2a067fc640 10 client.25748 handle_command_reply: tid=0 2024-04-02T13:55:16.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.333+0000 7f2a27f97740 1 client.25748 shutdown 2024-04-02T13:55:16.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.333+0000 7f2a27f97740 1 -- 172.21.15.52:0/2574110619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564c851941f0 msgr2=0x564c85196270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:16.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.333+0000 7f2a27f97740 1 --2- 172.21.15.52:0/2574110619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564c851941f0 0x564c85196270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x564c8519ac40 tx=0x7f2a080a5700 comp rx=0 tx=0).stop 2024-04-02T13:55:16.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.333+0000 7f2a27f97740 1 -- 172.21.15.52:0/2574110619 shutdown_connections 2024-04-02T13:55:16.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.333+0000 7f2a27f97740 1 --2- 172.21.15.52:0/2574110619 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564c85213450 0x564c85215840 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.333+0000 7f2a27f97740 1 --2- 172.21.15.52:0/2574110619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564c8519cdd0 0x564c851967b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.333+0000 7f2a27f97740 1 --2- 172.21.15.52:0/2574110619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564c851941f0 0x564c85196270 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.333+0000 7f2a27f97740 1 --2- 172.21.15.52:0/2574110619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c851938d0 0x564c851a5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.333+0000 7f2a067fc640 0 client.25748 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:16.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.333+0000 7f2a27f97740 1 -- 172.21.15.52:0/2574110619 >> 172.21.15.52:0/2574110619 conn(0x564c8518e000 msgr2=0x564c8518d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:16.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.333+0000 7f2a27f97740 1 -- 172.21.15.52:0/2574110619 shutdown_connections 2024-04-02T13:55:16.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.333+0000 7f2a27f97740 1 -- 172.21.15.52:0/2574110619 wait complete. 2024-04-02T13:55:16.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.333+0000 7f2a27f97740 20 client.25748 trim_cache size 0 max 16384 2024-04-02T13:55:16.342 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:55:16.342 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "scrub active (331 inodes in the stack)", 2024-04-02T13:55:16.342 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-04-02T13:55:16.342 INFO:teuthology.orchestra.run.smithi052.stdout: "043737e3-e755-4d51-b4ac-7e900b33d231": { 2024-04-02T13:55:16.342 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-04-02T13:55:16.342 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "043737e3-e755-4d51-b4ac-7e900b33d231", 2024-04-02T13:55:16.342 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-04-02T13:55:16.342 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:55:16.342 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:55:16.342 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:55:16.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.333+0000 7f2a252d6640 1 -- 172.21.15.52:0/562618519 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2a0c0a3d20 msgr2=0x7f2a0c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:16.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.333+0000 7f2a252d6640 1 --2- 172.21.15.52:0/562618519 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2a0c0a3d20 0x7f2a0c0a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2a18005fd0 tx=0x7f2a18009500 comp rx=0 tx=0).stop 2024-04-02T13:55:16.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.333+0000 7f2a252d6640 1 -- 172.21.15.52:0/562618519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a20101330 msgr2=0x7f2a201989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:16.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.333+0000 7f2a252d6640 1 --2- 172.21.15.52:0/562618519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a20101330 0x7f2a201989c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800e9f0 tx=0x7f2a0800edf0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.334+0000 7f2a252d6640 1 -- 172.21.15.52:0/562618519 shutdown_connections 2024-04-02T13:55:16.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.334+0000 7f2a252d6640 1 --2- 172.21.15.52:0/562618519 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2a0c0a3d20 0x7f2a0c0a61e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.334+0000 7f2a252d6640 1 --2- 172.21.15.52:0/562618519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a20106980 0x7f2a2019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.334+0000 7f2a252d6640 1 --2- 172.21.15.52:0/562618519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a20101330 0x7f2a201989c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.334+0000 7f2a252d6640 1 --2- 172.21.15.52:0/562618519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a20100970 0x7f2a20198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:16.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.334+0000 7f2a252d6640 1 -- 172.21.15.52:0/562618519 >> 172.21.15.52:0/562618519 conn(0x7f2a20077b00 msgr2=0x7f2a20069040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:16.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.334+0000 7f2a252d6640 1 -- 172.21.15.52:0/562618519 shutdown_connections 2024-04-02T13:55:16.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:16.334+0000 7f2a252d6640 1 -- 172.21.15.52:0/562618519 wait complete. 2024-04-02T13:55:16.348 INFO:tasks.cephfs.filesystem:scrub status for tag:043737e3-e755-4d51-b4ac-7e900b33d231 - {'path': '/', 'tag': '043737e3-e755-4d51-b4ac-7e900b33d231', 'options': 'recursive,force'} 2024-04-02T13:55:16.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:16 smithi088 ceph-mon[36528]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:55:16.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:16 smithi088 ceph-mon[36528]: scrub queued for path: / 2024-04-02T13:55:16.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:16 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:16.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:16 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:55:16.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:16 smithi189 ceph-mon[36683]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:55:16.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:16 smithi189 ceph-mon[36683]: scrub queued for path: / 2024-04-02T13:55:16.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:16 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:16.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:16 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:55:16.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:16 smithi052 ceph-mon[32104]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:55:16.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:16 smithi052 ceph-mon[32104]: scrub queued for path: / 2024-04-02T13:55:16.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:16 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:16.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:16 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:55:17.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:17 smithi088 ceph-mon[36528]: pgmap v56: 129 pgs: 129 active+clean; 16 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.2 MiB/s wr, 37 op/s 2024-04-02T13:55:17.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:17 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:55:17.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:17 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:17.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:17 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:17.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:17 smithi189 ceph-mon[36683]: pgmap v56: 129 pgs: 129 active+clean; 16 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.2 MiB/s wr, 37 op/s 2024-04-02T13:55:17.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:17 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:55:17.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:17 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:17.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:17 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:17.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:17 smithi052 ceph-mon[32104]: pgmap v56: 129 pgs: 129 active+clean; 16 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.2 MiB/s wr, 37 op/s 2024-04-02T13:55:17.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:17 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:55:17.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:17 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:17.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:17 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:19.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:19 smithi088 ceph-mon[36528]: pgmap v57: 129 pgs: 129 active+clean; 16 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.2 MiB/s wr, 18 op/s 2024-04-02T13:55:19.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:19 smithi189 ceph-mon[36683]: pgmap v57: 129 pgs: 129 active+clean; 16 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.2 MiB/s wr, 18 op/s 2024-04-02T13:55:19.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:19 smithi052 ceph-mon[32104]: pgmap v57: 129 pgs: 129 active+clean; 16 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.2 MiB/s wr, 18 op/s 2024-04-02T13:55:21.669 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:21 smithi088 ceph-mon[36528]: pgmap v58: 129 pgs: 129 active+clean; 16 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.2 MiB/s wr, 18 op/s 2024-04-02T13:55:21.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:21 smithi189 ceph-mon[36683]: pgmap v58: 129 pgs: 129 active+clean; 16 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.2 MiB/s wr, 18 op/s 2024-04-02T13:55:21.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:21 smithi052 ceph-mon[32104]: pgmap v58: 129 pgs: 129 active+clean; 16 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.2 MiB/s wr, 18 op/s 2024-04-02T13:55:23.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:23 smithi088 ceph-mon[36528]: pgmap v59: 129 pgs: 129 active+clean; 17 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.3 MiB/s wr, 25 op/s 2024-04-02T13:55:23.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:23 smithi189 ceph-mon[36683]: pgmap v59: 129 pgs: 129 active+clean; 17 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.3 MiB/s wr, 25 op/s 2024-04-02T13:55:23.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:23 smithi052 ceph-mon[32104]: pgmap v59: 129 pgs: 129 active+clean; 17 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.3 MiB/s wr, 25 op/s 2024-04-02T13:55:25.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:25 smithi189 ceph-mon[36683]: pgmap v60: 129 pgs: 129 active+clean; 17 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 199 KiB/s wr, 15 op/s 2024-04-02T13:55:25.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:25 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T13:55:25.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:25 smithi052 ceph-mon[32104]: pgmap v60: 129 pgs: 129 active+clean; 17 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 199 KiB/s wr, 15 op/s 2024-04-02T13:55:25.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:25 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T13:55:25.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:25 smithi088 ceph-mon[36528]: pgmap v60: 129 pgs: 129 active+clean; 17 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 199 KiB/s wr, 15 op/s 2024-04-02T13:55:25.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:25 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T13:55:26.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:26 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T13:55:26.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:26 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T13:55:26.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:26 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T13:55:27.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:27 smithi189 ceph-mon[36683]: pgmap v61: 129 pgs: 129 active+clean; 20 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 563 KiB/s wr, 37 op/s 2024-04-02T13:55:27.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:27 smithi052 ceph-mon[32104]: pgmap v61: 129 pgs: 129 active+clean; 20 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 563 KiB/s wr, 37 op/s 2024-04-02T13:55:27.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:27 smithi088 ceph-mon[36528]: pgmap v61: 129 pgs: 129 active+clean; 20 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 563 KiB/s wr, 37 op/s 2024-04-02T13:55:29.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:29 smithi189 ceph-mon[36683]: pgmap v62: 129 pgs: 129 active+clean; 20 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 496 KiB/s wr, 28 op/s 2024-04-02T13:55:29.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:29 smithi052 ceph-mon[32104]: pgmap v62: 129 pgs: 129 active+clean; 20 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 496 KiB/s wr, 28 op/s 2024-04-02T13:55:29.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:29 smithi088 ceph-mon[36528]: pgmap v62: 129 pgs: 129 active+clean; 20 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 496 KiB/s wr, 28 op/s 2024-04-02T13:55:31.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:31 smithi189 ceph-mon[36683]: pgmap v63: 129 pgs: 129 active+clean; 20 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 496 KiB/s wr, 28 op/s 2024-04-02T13:55:31.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:31 smithi052 ceph-mon[32104]: pgmap v63: 129 pgs: 129 active+clean; 20 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 496 KiB/s wr, 28 op/s 2024-04-02T13:55:31.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:31 smithi088 ceph-mon[36528]: pgmap v63: 129 pgs: 129 active+clean; 20 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 496 KiB/s wr, 28 op/s 2024-04-02T13:55:32.292 INFO:tasks.workunit.client.0.smithi052.stderr:Updating files: 7% (321/4281) Updating files: 8% (343/4281) Updating files: 9% (386/4281) Updating files: 10% (429/4281) Updating files: 11% (471/4281) Updating files: 11% (489/4281) Updating files: 12% (514/4281) Updating files: 13% (557/4281) Updating files: 14% (600/4281) Updating files: 15% (643/4281) Updating files: 16% (685/4281) Updating files: 17% (728/4281) Updating files: 18% (771/4281) Updating files: 19% (814/4281) Updating files: 20% (857/4281) Updating files: 21% (900/4281) Updating files: 22% (942/4281) Updating files: 23% (985/4281) Updating files: 23% (1023/4281) Updating files: 24% (1028/4281) Updating files: 25% (1071/4281) Updating files: 26% (1114/4281) Updating files: 27% (1156/4281) Updating files: 28% (1199/4281) Updating files: 29% (1242/4281) Updating files: 30% (1285/4281) Updating files: 31% (1328/4281) Updating files: 32% (1370/4281) Updating files: 33% (1413/4281) Updating files: 34% (1456/4281) Updating files: 35% (1499/4281) Updating files: 36% (1542/4281) Updating files: 37% (1584/4281) Updating files: 38% (1627/4281) Updating files: 39% (1670/4281) Updating files: 39% (1707/4281) Updating files: 40% (1713/4281) Updating files: 41% (1756/4281) Updating files: 42% (1799/4281) Updating files: 43% (1841/4281) Updating files: 44% (1884/4281) Updating files: 45% (1927/4281) Updating files: 46% (1970/4281) Updating files: 47% (2013/4281) Updating files: 48% (2055/4281) Updating files: 49% (2098/4281) Updating files: 50% (2141/4281) Updating files: 51% (2184/4281) Updating files: 52% (2227/4281) Updating files: 53% (2269/4281) Updating files: 53% (2291/4281) Updating files: 54% (2312/4281) Updating files: 55% (2355/4281) Updating files: 56% (2398/4281) Updating files: 57% (2441/4281) Updating files: 58% (2483/4281) Updating files: 59% (2526/4281) Updating files: 60% (2569/4281) Updating files: 61% (2612/4281) Updating files: 62% (2655/4281) Updating files: 63% (2698/4281) Updating files: 64% (2740/4281) Updating files: 64% (2775/4281) Updating files: 65% (2783/4281) Updating files: 66% (2826/4281) Updating files: 67% (2869/4281) Updating files: 68% (2912/4281) Updating files: 69% (2954/4281) Updating files: 70% (2997/4281) Updating files: 70% (3013/4281) Updating files: 71% (3040/4281) Updating files: 72% (3083/4281) Updating files: 73% (3126/4281) Updating files: 74% (3168/4281) Updating files: 75% (3211/4281) Updating files: 76% (3254/4281) Updating files: 77% (3297/4281) Updating files: 78% (3340/4281) Updating files: 79% (3382/4281) Updating files: 80% (3425/4281) Updating files: 81% (3468/4281) Updating files: 82% (3511/4281) Updating files: 83% (3554/4281) Updating files: 84% (3597/4281) Updating files: 84% (3601/4281) Updating files: 85% (3639/4281) Updating files: 86% (3682/4281) Updating files: 87% (3725/4281) Updating files: 88% (3768/4281) Updating files: 89% (3811/4281) Updating files: 90% (3853/4281) Updating files: 91% (3896/4281) Updating files: 92% (3939/4281) Updating files: 93% (3982/4281) Updating files: 94% (4025/4281) Updating files: 95% (4067/4281) Updating files: 96% (4110/4281) Updating files: 96% (4123/4281) Updating files: 97% (4153/4281) Updating files: 98% (4196/4281) Updating files: 99% (4239/4281) Updating files: 100% (4281/4281) Updating files: 100% (4281/4281), done. 2024-04-02T13:55:32.298 INFO:tasks.workunit.client.0.smithi052.stderr:+ cd xfstests-dev 2024-04-02T13:55:32.298 INFO:tasks.workunit.client.0.smithi052.stderr:+ git checkout 12973fc04fd10d4af086901e10ffa8e48866b735 2024-04-02T13:55:33.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:33 smithi189 ceph-mon[36683]: pgmap v64: 129 pgs: 129 active+clean; 21 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 550 KiB/s wr, 35 op/s 2024-04-02T13:55:33.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:33 smithi052 ceph-mon[32104]: pgmap v64: 129 pgs: 129 active+clean; 21 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 550 KiB/s wr, 35 op/s 2024-04-02T13:55:33.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:33 smithi088 ceph-mon[36528]: pgmap v64: 129 pgs: 129 active+clean; 21 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 550 KiB/s wr, 35 op/s 2024-04-02T13:55:35.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:35 smithi189 ceph-mon[36683]: pgmap v65: 129 pgs: 129 active+clean; 21 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 418 KiB/s wr, 28 op/s 2024-04-02T13:55:35.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:35 smithi052 ceph-mon[32104]: pgmap v65: 129 pgs: 129 active+clean; 21 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 418 KiB/s wr, 28 op/s 2024-04-02T13:55:35.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:35 smithi088 ceph-mon[36528]: pgmap v65: 129 pgs: 129 active+clean; 21 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 418 KiB/s wr, 28 op/s 2024-04-02T13:55:36.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:36 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:55:36.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:36 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:55:36.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:36 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:55:37.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:37 smithi189 ceph-mon[36683]: pgmap v66: 129 pgs: 129 active+clean; 42 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 416 op/s 2024-04-02T13:55:37.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:37 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:55:37.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:37 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:55:37.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:37 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:55:37.821 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:37 smithi088 ceph-mon[36528]: pgmap v66: 129 pgs: 129 active+clean; 42 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 416 op/s 2024-04-02T13:55:37.821 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:37 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:55:37.821 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:37 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:55:37.821 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:37 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:55:37.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:37 smithi052 ceph-mon[32104]: pgmap v66: 129 pgs: 129 active+clean; 42 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 416 op/s 2024-04-02T13:55:37.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:37 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:55:37.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:37 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:55:37.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:37 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:55:39.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:39 smithi189 ceph-mon[36683]: pgmap v67: 129 pgs: 129 active+clean; 42 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 394 op/s 2024-04-02T13:55:39.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:39 smithi052 ceph-mon[32104]: pgmap v67: 129 pgs: 129 active+clean; 42 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 394 op/s 2024-04-02T13:55:39.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:39 smithi088 ceph-mon[36528]: pgmap v67: 129 pgs: 129 active+clean; 42 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 394 op/s 2024-04-02T13:55:41.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:41 smithi189 ceph-mon[36683]: pgmap v68: 129 pgs: 129 active+clean; 42 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 394 op/s 2024-04-02T13:55:41.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:41 smithi052 ceph-mon[32104]: pgmap v68: 129 pgs: 129 active+clean; 42 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 394 op/s 2024-04-02T13:55:41.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:41 smithi088 ceph-mon[36528]: pgmap v68: 129 pgs: 129 active+clean; 42 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 394 op/s 2024-04-02T13:55:43.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:43 smithi189 ceph-mon[36683]: pgmap v69: 129 pgs: 129 active+clean; 48 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 497 op/s 2024-04-02T13:55:43.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:43 smithi052 ceph-mon[32104]: pgmap v69: 129 pgs: 129 active+clean; 48 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 497 op/s 2024-04-02T13:55:43.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:43 smithi088 ceph-mon[36528]: pgmap v69: 129 pgs: 129 active+clean; 48 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 497 op/s 2024-04-02T13:55:45.513 INFO:tasks.workunit.client.0.smithi052.stderr:Note: switching to '12973fc04fd10d4af086901e10ffa8e48866b735'. 2024-04-02T13:55:45.513 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-04-02T13:55:45.513 INFO:tasks.workunit.client.0.smithi052.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-02T13:55:45.513 INFO:tasks.workunit.client.0.smithi052.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-02T13:55:45.513 INFO:tasks.workunit.client.0.smithi052.stderr:state without impacting any branches by switching back to a branch. 2024-04-02T13:55:45.513 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-04-02T13:55:45.513 INFO:tasks.workunit.client.0.smithi052.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-02T13:55:45.513 INFO:tasks.workunit.client.0.smithi052.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-02T13:55:45.513 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-04-02T13:55:45.513 INFO:tasks.workunit.client.0.smithi052.stderr: git switch -c 2024-04-02T13:55:45.513 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-04-02T13:55:45.514 INFO:tasks.workunit.client.0.smithi052.stderr:Or undo this operation with: 2024-04-02T13:55:45.514 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-04-02T13:55:45.514 INFO:tasks.workunit.client.0.smithi052.stderr: git switch - 2024-04-02T13:55:45.514 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-04-02T13:55:45.514 INFO:tasks.workunit.client.0.smithi052.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-02T13:55:45.514 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-04-02T13:55:45.514 INFO:tasks.workunit.client.0.smithi052.stderr:HEAD is now at 12973fc0 btrfs/022: Correct seq number in golden output 2024-04-02T13:55:45.515 INFO:tasks.workunit.client.0.smithi052.stderr:+ make -j4 2024-04-02T13:55:45.523 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize -cfi 2024-04-02T13:55:45.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:45 smithi189 ceph-mon[36683]: pgmap v70: 129 pgs: 129 active+clean; 48 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 490 op/s 2024-04-02T13:55:45.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:45 smithi052 ceph-mon[32104]: pgmap v70: 129 pgs: 129 active+clean; 48 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 490 op/s 2024-04-02T13:55:45.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:45 smithi088 ceph-mon[36528]: pgmap v70: 129 pgs: 129 active+clean; 48 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 490 op/s 2024-04-02T13:55:45.927 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-02T13:55:45.928 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize: copying file './config.guess' 2024-04-02T13:55:45.933 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize: copying file './config.sub' 2024-04-02T13:55:45.939 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize: copying file './install-sh' 2024-04-02T13:55:45.944 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize: copying file './ltmain.sh' 2024-04-02T13:55:45.968 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-02T13:55:45.968 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-02T13:55:45.974 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-02T13:55:45.981 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-02T13:55:45.988 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-02T13:55:45.994 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-02T13:55:46.000 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-02T13:55:46.001 INFO:tasks.workunit.client.0.smithi052.stdout:cp include/install-sh . 2024-04-02T13:55:46.026 INFO:tasks.workunit.client.0.smithi052.stdout:aclocal -I m4 2024-04-02T13:55:46.348 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:55:46.349 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:55:46.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.454+0000 7fae5f1d1640 1 -- 172.21.15.52:0/727666362 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae58100970 msgr2=0x7fae58100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:46.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.454+0000 7fae5f1d1640 1 --2- 172.21.15.52:0/727666362 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae58100970 0x7fae58100df0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fae44009930 tx=0x7fae4402bc30 comp rx=0 tx=0).stop 2024-04-02T13:55:46.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.454+0000 7fae5f1d1640 1 -- 172.21.15.52:0/727666362 shutdown_connections 2024-04-02T13:55:46.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.454+0000 7fae5f1d1640 1 --2- 172.21.15.52:0/727666362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae58101330 0x7fae5810d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.454+0000 7fae5f1d1640 1 --2- 172.21.15.52:0/727666362 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae58100970 0x7fae58100df0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.454+0000 7fae5f1d1640 1 --2- 172.21.15.52:0/727666362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae58106980 0x7fae58106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.454+0000 7fae5f1d1640 1 -- 172.21.15.52:0/727666362 >> 172.21.15.52:0/727666362 conn(0x7fae58077b00 msgr2=0x7fae58077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:46.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.454+0000 7fae5f1d1640 1 -- 172.21.15.52:0/727666362 shutdown_connections 2024-04-02T13:55:46.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.454+0000 7fae5f1d1640 1 -- 172.21.15.52:0/727666362 wait complete. 2024-04-02T13:55:46.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.455+0000 7fae5f1d1640 1 Processor -- start 2024-04-02T13:55:46.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.455+0000 7fae5f1d1640 1 -- start start 2024-04-02T13:55:46.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.455+0000 7fae5f1d1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae58100970 0x7fae581984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.455+0000 7fae5f1d1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae58101330 0x7fae581989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.455+0000 7fae5f1d1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae58106980 0x7fae5819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.455+0000 7fae5f1d1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fae58110d90 con 0x7fae58100970 2024-04-02T13:55:46.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.455+0000 7fae5f1d1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fae58110c10 con 0x7fae58106980 2024-04-02T13:55:46.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.455+0000 7fae5f1d1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae58110f10 con 0x7fae58101330 2024-04-02T13:55:46.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.455+0000 7fae5cf6d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae58100970 0x7fae581984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.455+0000 7fae5cf6d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae58100970 0x7fae581984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33760/0 (socket says 172.21.15.52:33760) 2024-04-02T13:55:46.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.455+0000 7fae5cf6d640 1 -- 172.21.15.52:0/2381666279 learned_addr learned my addr 172.21.15.52:0/2381666279 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:46.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.455+0000 7fae5d76e640 1 --2- 172.21.15.52:0/2381666279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae58106980 0x7fae5819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.455+0000 7fae57fff640 1 --2- 172.21.15.52:0/2381666279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae58101330 0x7fae581989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.455+0000 7fae5cf6d640 1 -- 172.21.15.52:0/2381666279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae58101330 msgr2=0x7fae581989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:46.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.455+0000 7fae5cf6d640 1 --2- 172.21.15.52:0/2381666279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae58101330 0x7fae581989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.455+0000 7fae5cf6d640 1 -- 172.21.15.52:0/2381666279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae58106980 msgr2=0x7fae5819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:46.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.455+0000 7fae5cf6d640 1 --2- 172.21.15.52:0/2381666279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae58106980 0x7fae5819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.455+0000 7fae5cf6d640 1 -- 172.21.15.52:0/2381666279 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae5819d440 con 0x7fae58100970 2024-04-02T13:55:46.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.456+0000 7fae5cf6d640 1 --2- 172.21.15.52:0/2381666279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae58100970 0x7fae581984a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00d950 tx=0x7fae4c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:46.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.456+0000 7fae55ffb640 1 -- 172.21.15.52:0/2381666279 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c017070 con 0x7fae58100970 2024-04-02T13:55:46.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.456+0000 7fae55ffb640 1 -- 172.21.15.52:0/2381666279 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae4c009c30 con 0x7fae58100970 2024-04-02T13:55:46.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.456+0000 7fae5f1d1640 1 -- 172.21.15.52:0/2381666279 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae5819d730 con 0x7fae58100970 2024-04-02T13:55:46.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.456+0000 7fae55ffb640 1 -- 172.21.15.52:0/2381666279 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c00f6b0 con 0x7fae58100970 2024-04-02T13:55:46.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.456+0000 7fae5f1d1640 1 -- 172.21.15.52:0/2381666279 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae581a4fc0 con 0x7fae58100970 2024-04-02T13:55:46.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.459+0000 7fae55ffb640 1 -- 172.21.15.52:0/2381666279 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fae4c0109d0 con 0x7fae58100970 2024-04-02T13:55:46.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.459+0000 7fae61db0740 1 Processor -- start 2024-04-02T13:55:46.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.459+0000 7fae61db0740 1 -- start start 2024-04-02T13:55:46.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.459+0000 7fae61db0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f264bff8d0 0x55f264bffcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.459+0000 7fae61db0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f264c001f0 0x55f264c08890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.459+0000 7fae61db0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264c08dd0 0x55f264c0b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.459+0000 7fae61db0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f264b63830 con 0x55f264c08dd0 2024-04-02T13:55:46.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.459+0000 7fae61db0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f264b0b820 con 0x55f264c001f0 2024-04-02T13:55:46.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.459+0000 7fae61db0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f264b5f7e0 con 0x55f264bff8d0 2024-04-02T13:55:46.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.459+0000 7fae57fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264c08dd0 0x55f264c0b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.459+0000 7fae57fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264c08dd0 0x55f264c0b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33770/0 (socket says 172.21.15.52:33770) 2024-04-02T13:55:46.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.459+0000 7fae57fff640 1 -- 172.21.15.52:0/249214513 learned_addr learned my addr 172.21.15.52:0/249214513 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:46.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.459+0000 7fae55ffb640 1 --2- 172.21.15.52:0/2381666279 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fae480a3da0 0x7fae480a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.459+0000 7fae5cf6d640 1 --2- 172.21.15.52:0/249214513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f264c001f0 0x55f264c08890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.460+0000 7fae5d76e640 1 --2- 172.21.15.52:0/249214513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f264bff8d0 0x55f264bffcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.460+0000 7fae5d76e640 1 --2- 172.21.15.52:0/2381666279 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fae480a3da0 0x7fae480a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.460+0000 7fae57fff640 1 -- 172.21.15.52:0/249214513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f264bff8d0 msgr2=0x55f264bffcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:46.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.460+0000 7fae57fff640 1 --2- 172.21.15.52:0/249214513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f264bff8d0 0x55f264bffcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.460+0000 7fae57fff640 1 -- 172.21.15.52:0/249214513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f264c001f0 msgr2=0x55f264c08890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:46.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.460+0000 7fae57fff640 1 --2- 172.21.15.52:0/249214513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f264c001f0 0x55f264c08890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.460+0000 7fae57fff640 1 -- 172.21.15.52:0/249214513 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2649417d0 con 0x55f264c08dd0 2024-04-02T13:55:46.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.460+0000 7fae55ffb640 1 -- 172.21.15.52:0/2381666279 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fae4c0d05c0 con 0x7fae58100970 2024-04-02T13:55:46.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.460+0000 7fae5d76e640 1 --2- 172.21.15.52:0/249214513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f264bff8d0 0x55f264bffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:46.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.460+0000 7fae57fff640 1 --2- 172.21.15.52:0/249214513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264c08dd0 0x55f264c0b1c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fae4402cfd0 tx=0x7fae44005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0f958c0a093523 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:46.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.460+0000 7fae5d76e640 1 --2- 172.21.15.52:0/2381666279 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fae480a3da0 0x7fae480a6260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fae50009fd0 tx=0x7fae500096a0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:46.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.460+0000 7fae42ffd640 1 -- 172.21.15.52:0/249214513 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae44042070 con 0x55f264c08dd0 2024-04-02T13:55:46.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.460+0000 7fae42ffd640 1 -- 172.21.15.52:0/249214513 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae4402ca80 con 0x55f264c08dd0 2024-04-02T13:55:46.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.460+0000 7fae42ffd640 1 -- 172.21.15.52:0/249214513 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae44004f50 con 0x55f264c08dd0 2024-04-02T13:55:46.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.461+0000 7fae61db0740 1 -- 172.21.15.52:0/249214513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264c08dd0 msgr2=0x55f264c0b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:46.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.461+0000 7fae61db0740 1 --2- 172.21.15.52:0/249214513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264c08dd0 0x55f264c0b1c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fae4402cfd0 tx=0x7fae44005c00 comp rx=0 tx=0).stop 2024-04-02T13:55:46.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.461+0000 7fae61db0740 1 -- 172.21.15.52:0/249214513 shutdown_connections 2024-04-02T13:55:46.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.461+0000 7fae61db0740 1 --2- 172.21.15.52:0/249214513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264c08dd0 0x55f264c0b1c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.461+0000 7fae61db0740 1 --2- 172.21.15.52:0/249214513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f264c001f0 0x55f264c08890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.461+0000 7fae61db0740 1 --2- 172.21.15.52:0/249214513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f264bff8d0 0x55f264bffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.461+0000 7fae61db0740 1 -- 172.21.15.52:0/249214513 >> 172.21.15.52:0/249214513 conn(0x55f264bfa000 msgr2=0x55f264bfa3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:46.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.461+0000 7fae61db0740 1 -- 172.21.15.52:0/249214513 shutdown_connections 2024-04-02T13:55:46.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.461+0000 7fae61db0740 1 -- 172.21.15.52:0/249214513 wait complete. 2024-04-02T13:55:46.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.461+0000 7fae61db0740 1 -- start start 2024-04-02T13:55:46.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.461+0000 7fae61db0740 1 Processor -- start 2024-04-02T13:55:46.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.462+0000 7fae61db0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264bff8d0 0x55f264c08620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.462+0000 7fae61db0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f264c001f0 0x55f264c059b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.462+0000 7fae61db0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f264c08dd0 0x55f264c05ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.462+0000 7fae61db0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f264c0bb60 con 0x55f264bff8d0 2024-04-02T13:55:46.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.462+0000 7fae61db0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f264b1ed30 con 0x55f264c001f0 2024-04-02T13:55:46.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.462+0000 7fae61db0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f264c0bce0 con 0x55f264c08dd0 2024-04-02T13:55:46.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.462+0000 7fae61db0740 10 client.0 _ll_register_callbacks cb 0x55f264b1e720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:46.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.462+0000 7fae5cf6d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264bff8d0 0x55f264c08620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.462+0000 7fae5cf6d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264bff8d0 0x55f264c08620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33784/0 (socket says 172.21.15.52:33784) 2024-04-02T13:55:46.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.462+0000 7fae5cf6d640 1 -- 172.21.15.52:0/4036909887 learned_addr learned my addr 172.21.15.52:0/4036909887 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:46.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.462+0000 7fae5d76e640 1 --2- 172.21.15.52:0/4036909887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f264c08dd0 0x55f264c05ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.462+0000 7fae57fff640 1 --2- 172.21.15.52:0/4036909887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f264c001f0 0x55f264c059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.462+0000 7fae5cf6d640 1 -- 172.21.15.52:0/4036909887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f264c08dd0 msgr2=0x55f264c05ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:46.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.462+0000 7fae5cf6d640 1 --2- 172.21.15.52:0/4036909887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f264c08dd0 0x55f264c05ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.462+0000 7fae5cf6d640 1 -- 172.21.15.52:0/4036909887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f264c001f0 msgr2=0x55f264c059b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:46.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.462+0000 7fae5cf6d640 1 --2- 172.21.15.52:0/4036909887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f264c001f0 0x55f264c059b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.462+0000 7fae5cf6d640 1 -- 172.21.15.52:0/4036909887 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2648773f0 con 0x55f264bff8d0 2024-04-02T13:55:46.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.462+0000 7fae57fff640 1 --2- 172.21.15.52:0/4036909887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f264c001f0 0x55f264c059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:46.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.462+0000 7fae5cf6d640 1 --2- 172.21.15.52:0/4036909887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264bff8d0 0x55f264c08620 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fae4c098730 tx=0x7fae4c09c190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3ee28cf81c5471 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:46.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.462+0000 7fae37fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:46.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.463+0000 7fae37fff640 1 -- 172.21.15.52:0/4036909887 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c020070 con 0x55f264bff8d0 2024-04-02T13:55:46.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.463+0000 7fae37fff640 1 -- 172.21.15.52:0/4036909887 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fae4c0d0210 con 0x55f264bff8d0 2024-04-02T13:55:46.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.463+0000 7fae61db0740 1 -- 172.21.15.52:0/4036909887 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae4402ca80 con 0x55f264bff8d0 2024-04-02T13:55:46.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.463+0000 7fae37fff640 1 -- 172.21.15.52:0/4036909887 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fae4c0d0410 con 0x55f264bff8d0 2024-04-02T13:55:46.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.463+0000 7fae61db0740 10 client.16254 fetch_fsmap learned FSMap version 29 2024-04-02T13:55:46.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.463+0000 7fae61db0740 1 -- 172.21.15.52:0/4036909887 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f2648370f0 con 0x55f264bff8d0 2024-04-02T13:55:46.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.463+0000 7fae37fff640 1 -- 172.21.15.52:0/4036909887 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae4402ca80 con 0x55f264bff8d0 2024-04-02T13:55:46.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.463+0000 7fae37fff640 1 -- 172.21.15.52:0/4036909887 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fae4c0a00b0 con 0x55f264bff8d0 2024-04-02T13:55:46.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.463+0000 7fae61db0740 10 client.16254 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:55:46.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.463+0000 7fae61db0740 10 client.16254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:46.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.463+0000 7fae61db0740 20 client.16254 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:46.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.463+0000 7fae61db0740 1 --2- 172.21.15.52:0/4036909887 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55f264c1d1b0 0x55f264c1f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.463+0000 7fae61db0740 4 client.16254 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:46.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.463+0000 7fae61db0740 1 -- 172.21.15.52:0/4036909887 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f264b70d70 con 0x55f264c1d1b0 2024-04-02T13:55:46.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.463+0000 7fae57fff640 1 --2- 172.21.15.52:0/4036909887 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55f264c1d1b0 0x55f264c1f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.464+0000 7fae57fff640 1 --2- 172.21.15.52:0/4036909887 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55f264c1d1b0 0x55f264c1f600 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49970d6e118a7de1 server_cookie=6b95ba64b7ab0526 in_seq=0 out_seq=0 2024-04-02T13:55:46.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.464+0000 7fae37fff640 10 client.16254 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:46.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.464+0000 7fae37fff640 1 -- 172.21.15.52:0/4036909887 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f264b70d70 con 0x55f264c1d1b0 2024-04-02T13:55:46.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.464+0000 7fae37fff640 10 client.16254 handle_command_reply: tid=0 2024-04-02T13:55:46.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.464+0000 7fae61db0740 1 client.16254 shutdown 2024-04-02T13:55:46.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.465+0000 7fae61db0740 1 -- 172.21.15.52:0/4036909887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264bff8d0 msgr2=0x55f264c08620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:46.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.465+0000 7fae61db0740 1 --2- 172.21.15.52:0/4036909887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264bff8d0 0x55f264c08620 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fae4c098730 tx=0x7fae4c09c190 comp rx=0 tx=0).stop 2024-04-02T13:55:46.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.465+0000 7fae61db0740 1 -- 172.21.15.52:0/4036909887 shutdown_connections 2024-04-02T13:55:46.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.465+0000 7fae61db0740 1 --2- 172.21.15.52:0/4036909887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f264c08dd0 0x55f264c05ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.465+0000 7fae61db0740 1 --2- 172.21.15.52:0/4036909887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f264c001f0 0x55f264c059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.465+0000 7fae61db0740 1 --2- 172.21.15.52:0/4036909887 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55f264c1d1b0 0x55f264c1f600 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.465+0000 7fae61db0740 1 --2- 172.21.15.52:0/4036909887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264bff8d0 0x55f264c08620 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.465+0000 7fae37fff640 0 client.16254 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:46.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.465+0000 7fae61db0740 1 -- 172.21.15.52:0/4036909887 >> 172.21.15.52:0/4036909887 conn(0x55f264bfa000 msgr2=0x55f264bfbd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:46.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.465+0000 7fae61db0740 1 -- 172.21.15.52:0/4036909887 shutdown_connections 2024-04-02T13:55:46.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.465+0000 7fae61db0740 1 -- 172.21.15.52:0/4036909887 wait complete. 2024-04-02T13:55:46.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.465+0000 7fae61db0740 20 client.16254 trim_cache size 0 max 16384 2024-04-02T13:55:46.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.482+0000 7fae61db0740 1 Processor -- start 2024-04-02T13:55:46.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.482+0000 7fae61db0740 1 -- start start 2024-04-02T13:55:46.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.482+0000 7fae61db0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f264bff8d0 0x55f264bf9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.482+0000 7fae61db0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f264c001f0 0x55f264c00e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.482+0000 7fae61db0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264c08dd0 0x55f264c080e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.482+0000 7fae61db0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fae581acb70 con 0x55f264c08dd0 2024-04-02T13:55:46.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.482+0000 7fae61db0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fae581ae6e0 con 0x55f264bff8d0 2024-04-02T13:55:46.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.482+0000 7fae61db0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae581ad240 con 0x55f264c001f0 2024-04-02T13:55:46.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.482+0000 7fae5d76e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264c08dd0 0x55f264c080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.483+0000 7fae5d76e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264c08dd0 0x55f264c080e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33794/0 (socket says 172.21.15.52:33794) 2024-04-02T13:55:46.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.483+0000 7fae5d76e640 1 -- 172.21.15.52:0/3327983168 learned_addr learned my addr 172.21.15.52:0/3327983168 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:46.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.483+0000 7fae5cf6d640 1 --2- 172.21.15.52:0/3327983168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f264bff8d0 0x55f264bf9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.483+0000 7fae57fff640 1 --2- 172.21.15.52:0/3327983168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f264c001f0 0x55f264c00e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.483+0000 7fae5d76e640 1 -- 172.21.15.52:0/3327983168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f264c001f0 msgr2=0x55f264c00e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:46.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.483+0000 7fae5d76e640 1 --2- 172.21.15.52:0/3327983168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f264c001f0 0x55f264c00e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.483+0000 7fae5d76e640 1 -- 172.21.15.52:0/3327983168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f264bff8d0 msgr2=0x55f264bf9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:46.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.483+0000 7fae5d76e640 1 --2- 172.21.15.52:0/3327983168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f264bff8d0 0x55f264bf9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.483+0000 7fae5d76e640 1 -- 172.21.15.52:0/3327983168 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f264bf9e30 con 0x55f264c08dd0 2024-04-02T13:55:46.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.483+0000 7fae5d76e640 1 --2- 172.21.15.52:0/3327983168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264c08dd0 0x55f264c080e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fae50003ce0 tx=0x7fae50009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507bec462dbb7763 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:46.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.483+0000 7fae427fc640 1 -- 172.21.15.52:0/3327983168 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae50018070 con 0x55f264c08dd0 2024-04-02T13:55:46.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.483+0000 7fae427fc640 1 -- 172.21.15.52:0/3327983168 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae50026070 con 0x55f264c08dd0 2024-04-02T13:55:46.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.483+0000 7fae427fc640 1 -- 172.21.15.52:0/3327983168 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae50010b60 con 0x55f264c08dd0 2024-04-02T13:55:46.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.484+0000 7fae61db0740 1 -- 172.21.15.52:0/3327983168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264c08dd0 msgr2=0x55f264c080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:46.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.484+0000 7fae61db0740 1 --2- 172.21.15.52:0/3327983168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264c08dd0 0x55f264c080e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fae50003ce0 tx=0x7fae50009920 comp rx=0 tx=0).stop 2024-04-02T13:55:46.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.484+0000 7fae61db0740 1 -- 172.21.15.52:0/3327983168 shutdown_connections 2024-04-02T13:55:46.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.484+0000 7fae61db0740 1 --2- 172.21.15.52:0/3327983168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264c08dd0 0x55f264c080e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.484+0000 7fae61db0740 1 --2- 172.21.15.52:0/3327983168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f264c001f0 0x55f264c00e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.484+0000 7fae61db0740 1 --2- 172.21.15.52:0/3327983168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f264bff8d0 0x55f264bf9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.484+0000 7fae61db0740 1 -- 172.21.15.52:0/3327983168 >> 172.21.15.52:0/3327983168 conn(0x55f264bfa000 msgr2=0x55f264c00710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:46.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.484+0000 7fae61db0740 1 -- 172.21.15.52:0/3327983168 shutdown_connections 2024-04-02T13:55:46.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.484+0000 7fae61db0740 1 -- 172.21.15.52:0/3327983168 wait complete. 2024-04-02T13:55:46.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.484+0000 7fae61db0740 1 -- start start 2024-04-02T13:55:46.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.484+0000 7fae61db0740 1 Processor -- start 2024-04-02T13:55:46.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.485+0000 7fae61db0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264bff8d0 0x55f264c155b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.485+0000 7fae61db0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f264c001f0 0x55f264c02260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.485+0000 7fae61db0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f264c08dd0 0x55f264c027a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.485+0000 7fae61db0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fae581adfe0 con 0x55f264bff8d0 2024-04-02T13:55:46.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.485+0000 7fae61db0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fae581ad910 con 0x55f264c001f0 2024-04-02T13:55:46.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.485+0000 7fae61db0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae581aab50 con 0x55f264c08dd0 2024-04-02T13:55:46.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.485+0000 7fae61db0740 10 client.0 _ll_register_callbacks cb 0x7fae4c09c0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:46.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.485+0000 7fae5cf6d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264bff8d0 0x55f264c155b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.485+0000 7fae5cf6d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264bff8d0 0x55f264c155b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33796/0 (socket says 172.21.15.52:33796) 2024-04-02T13:55:46.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.485+0000 7fae5cf6d640 1 -- 172.21.15.52:0/1298430568 learned_addr learned my addr 172.21.15.52:0/1298430568 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:46.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.485+0000 7fae57fff640 1 --2- 172.21.15.52:0/1298430568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f264c001f0 0x55f264c02260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.485+0000 7fae5d76e640 1 --2- 172.21.15.52:0/1298430568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f264c08dd0 0x55f264c027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.485+0000 7fae5cf6d640 1 -- 172.21.15.52:0/1298430568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f264c08dd0 msgr2=0x55f264c027a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:46.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.485+0000 7fae5cf6d640 1 --2- 172.21.15.52:0/1298430568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f264c08dd0 0x55f264c027a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.485+0000 7fae5cf6d640 1 -- 172.21.15.52:0/1298430568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f264c001f0 msgr2=0x55f264c02260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:46.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.485+0000 7fae5cf6d640 1 --2- 172.21.15.52:0/1298430568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f264c001f0 0x55f264c02260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.485+0000 7fae5cf6d640 1 -- 172.21.15.52:0/1298430568 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f264c02d90 con 0x55f264bff8d0 2024-04-02T13:55:46.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.485+0000 7fae5d76e640 1 --2- 172.21.15.52:0/1298430568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f264c08dd0 0x55f264c027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:46.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.485+0000 7fae5cf6d640 1 --2- 172.21.15.52:0/1298430568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264bff8d0 0x55f264c155b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fae4c09bbc0 tx=0x7fae4c010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee5ad1635f8566a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:46.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.485+0000 7fae37fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:46.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.486+0000 7fae37fff640 1 -- 172.21.15.52:0/1298430568 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c09bc30 con 0x55f264bff8d0 2024-04-02T13:55:46.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.486+0000 7fae37fff640 1 -- 172.21.15.52:0/1298430568 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fae4c098220 con 0x55f264bff8d0 2024-04-02T13:55:46.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.486+0000 7fae61db0740 1 -- 172.21.15.52:0/1298430568 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae50026070 con 0x55f264bff8d0 2024-04-02T13:55:46.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.486+0000 7fae37fff640 1 -- 172.21.15.52:0/1298430568 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fae4c083050 con 0x55f264bff8d0 2024-04-02T13:55:46.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.486+0000 7fae61db0740 10 client.16260 fetch_fsmap learned FSMap version 29 2024-04-02T13:55:46.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.486+0000 7fae61db0740 1 -- 172.21.15.52:0/1298430568 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f264c83550 con 0x55f264bff8d0 2024-04-02T13:55:46.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.486+0000 7fae37fff640 1 -- 172.21.15.52:0/1298430568 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae4c07d070 con 0x55f264bff8d0 2024-04-02T13:55:46.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.486+0000 7fae37fff640 1 -- 172.21.15.52:0/1298430568 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fae4c09b190 con 0x55f264bff8d0 2024-04-02T13:55:46.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.486+0000 7fae61db0740 10 client.16260 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:55:46.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.486+0000 7fae61db0740 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:46.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.486+0000 7fae61db0740 20 client.16260 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:46.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.486+0000 7fae61db0740 1 --2- 172.21.15.52:0/1298430568 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55f264c83cd0 0x55f264c860f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.486+0000 7fae61db0740 4 client.16260 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:55:46.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.486+0000 7fae61db0740 1 -- 172.21.15.52:0/1298430568 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f264b6b0c0 con 0x55f264c83cd0 2024-04-02T13:55:46.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.486+0000 7fae57fff640 1 --2- 172.21.15.52:0/1298430568 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55f264c83cd0 0x55f264c860f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.487+0000 7fae37fff640 10 client.16260 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:46.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.487+0000 7fae57fff640 1 --2- 172.21.15.52:0/1298430568 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55f264c83cd0 0x55f264c860f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4030db4b0e580d1f server_cookie=3c13e2738aa150cf in_seq=0 out_seq=0 2024-04-02T13:55:46.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.487+0000 7fae37fff640 1 -- 172.21.15.52:0/1298430568 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f264b6b0c0 con 0x55f264c83cd0 2024-04-02T13:55:46.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.487+0000 7fae37fff640 10 client.16260 handle_command_reply: tid=0 2024-04-02T13:55:46.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.487+0000 7fae61db0740 1 client.16260 shutdown 2024-04-02T13:55:46.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.487+0000 7fae61db0740 1 -- 172.21.15.52:0/1298430568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264bff8d0 msgr2=0x55f264c155b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:46.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.487+0000 7fae61db0740 1 --2- 172.21.15.52:0/1298430568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264bff8d0 0x55f264c155b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fae4c09bbc0 tx=0x7fae4c010d70 comp rx=0 tx=0).stop 2024-04-02T13:55:46.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.487+0000 7fae61db0740 1 -- 172.21.15.52:0/1298430568 shutdown_connections 2024-04-02T13:55:46.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.487+0000 7fae61db0740 1 --2- 172.21.15.52:0/1298430568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f264c08dd0 0x55f264c027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.487+0000 7fae61db0740 1 --2- 172.21.15.52:0/1298430568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f264c001f0 0x55f264c02260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.487+0000 7fae61db0740 1 --2- 172.21.15.52:0/1298430568 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55f264c83cd0 0x55f264c860f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.487+0000 7fae61db0740 1 --2- 172.21.15.52:0/1298430568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f264bff8d0 0x55f264c155b0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.487+0000 7fae37fff640 0 client.16260 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:46.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.487+0000 7fae61db0740 1 -- 172.21.15.52:0/1298430568 >> 172.21.15.52:0/1298430568 conn(0x55f264bfa000 msgr2=0x55f264c00fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:46.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.487+0000 7fae61db0740 1 -- 172.21.15.52:0/1298430568 shutdown_connections 2024-04-02T13:55:46.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.487+0000 7fae61db0740 1 -- 172.21.15.52:0/1298430568 wait complete. 2024-04-02T13:55:46.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.487+0000 7fae61db0740 20 client.16260 trim_cache size 0 max 16384 2024-04-02T13:55:46.497 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:55:46.498 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-04-02T13:55:46.498 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-04-02T13:55:46.498 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:55:46.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.488+0000 7fae5f1d1640 1 -- 172.21.15.52:0/2381666279 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fae480a3da0 msgr2=0x7fae480a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:46.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.488+0000 7fae5f1d1640 1 --2- 172.21.15.52:0/2381666279 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fae480a3da0 0x7fae480a6260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fae50009fd0 tx=0x7fae500096a0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.488+0000 7fae5f1d1640 1 -- 172.21.15.52:0/2381666279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae58100970 msgr2=0x7fae581984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:46.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.488+0000 7fae5f1d1640 1 --2- 172.21.15.52:0/2381666279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae58100970 0x7fae581984a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00d950 tx=0x7fae4c00de10 comp rx=0 tx=0).stop 2024-04-02T13:55:46.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.488+0000 7fae5f1d1640 1 -- 172.21.15.52:0/2381666279 shutdown_connections 2024-04-02T13:55:46.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.488+0000 7fae5f1d1640 1 --2- 172.21.15.52:0/2381666279 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fae480a3da0 0x7fae480a6260 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.488+0000 7fae5f1d1640 1 --2- 172.21.15.52:0/2381666279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae58106980 0x7fae5819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.488+0000 7fae5f1d1640 1 --2- 172.21.15.52:0/2381666279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae58101330 0x7fae581989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.488+0000 7fae5f1d1640 1 --2- 172.21.15.52:0/2381666279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae58100970 0x7fae581984a0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.488+0000 7fae5f1d1640 1 -- 172.21.15.52:0/2381666279 >> 172.21.15.52:0/2381666279 conn(0x7fae58077b00 msgr2=0x7fae580690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:46.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.488+0000 7fae5f1d1640 1 -- 172.21.15.52:0/2381666279 shutdown_connections 2024-04-02T13:55:46.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.488+0000 7fae5f1d1640 1 -- 172.21.15.52:0/2381666279 wait complete. 2024-04-02T13:55:46.505 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:55:46.505 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:55:46.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.629+0000 7f9690fba640 1 -- 172.21.15.52:0/252940538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f968c1015a0 msgr2=0x7f968c101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:46.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.629+0000 7f9690fba640 1 --2- 172.21.15.52:0/252940538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f968c1015a0 0x7f968c101a20 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9674009930 tx=0x7f967402bc30 comp rx=0 tx=0).stop 2024-04-02T13:55:46.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.629+0000 7f9690fba640 1 -- 172.21.15.52:0/252940538 shutdown_connections 2024-04-02T13:55:46.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.629+0000 7f9690fba640 1 --2- 172.21.15.52:0/252940538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f968c101f60 0x7f968c10d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.629+0000 7f9690fba640 1 --2- 172.21.15.52:0/252940538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f968c1015a0 0x7f968c101a20 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.629+0000 7f9690fba640 1 --2- 172.21.15.52:0/252940538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f968c105a30 0x7f968c105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.629+0000 7f9690fba640 1 -- 172.21.15.52:0/252940538 >> 172.21.15.52:0/252940538 conn(0x7f968c075d10 msgr2=0x7f968c076120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:46.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.629+0000 7f9690fba640 1 -- 172.21.15.52:0/252940538 shutdown_connections 2024-04-02T13:55:46.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.629+0000 7f9690fba640 1 -- 172.21.15.52:0/252940538 wait complete. 2024-04-02T13:55:46.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.630+0000 7f9690fba640 1 Processor -- start 2024-04-02T13:55:46.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.630+0000 7f9690fba640 1 -- start start 2024-04-02T13:55:46.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.630+0000 7f9690fba640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f968c1015a0 0x7f968c102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.630+0000 7f9690fba640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f968c101f60 0x7f968c102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.630+0000 7f9690fba640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f968c105a30 0x7f968c1031a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.630+0000 7f9690fba640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f968c110e60 con 0x7f968c1015a0 2024-04-02T13:55:46.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.630+0000 7f9690fba640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f968c110ce0 con 0x7f968c101f60 2024-04-02T13:55:46.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.630+0000 7f9690fba640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f968c110fe0 con 0x7f968c105a30 2024-04-02T13:55:46.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.631+0000 7f968a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f968c1015a0 0x7f968c102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.631+0000 7f968a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f968c1015a0 0x7f968c102720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33810/0 (socket says 172.21.15.52:33810) 2024-04-02T13:55:46.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.631+0000 7f968a59c640 1 -- 172.21.15.52:0/1276192279 learned_addr learned my addr 172.21.15.52:0/1276192279 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:46.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.631+0000 7f9689d9b640 1 --2- 172.21.15.52:0/1276192279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f968c101f60 0x7f968c102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.631+0000 7f968ad9d640 1 --2- 172.21.15.52:0/1276192279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f968c105a30 0x7f968c1031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.631+0000 7f968a59c640 1 -- 172.21.15.52:0/1276192279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f968c105a30 msgr2=0x7f968c1031a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:46.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.631+0000 7f968a59c640 1 --2- 172.21.15.52:0/1276192279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f968c105a30 0x7f968c1031a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.631+0000 7f968a59c640 1 -- 172.21.15.52:0/1276192279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f968c101f60 msgr2=0x7f968c102c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:46.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.631+0000 7f968a59c640 1 --2- 172.21.15.52:0/1276192279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f968c101f60 0x7f968c102c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.631+0000 7f968a59c640 1 -- 172.21.15.52:0/1276192279 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968c1a68f0 con 0x7f968c1015a0 2024-04-02T13:55:46.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.631+0000 7f968ad9d640 1 --2- 172.21.15.52:0/1276192279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f968c105a30 0x7f968c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:46.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.631+0000 7f968a59c640 1 --2- 172.21.15.52:0/1276192279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f968c1015a0 0x7f968c102720 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f968400ebc0 tx=0x7f968400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:46.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.631+0000 7f96837fe640 1 -- 172.21.15.52:0/1276192279 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684003b70 con 0x7f968c1015a0 2024-04-02T13:55:46.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.631+0000 7f96837fe640 1 -- 172.21.15.52:0/1276192279 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9684003d10 con 0x7f968c1015a0 2024-04-02T13:55:46.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.631+0000 7f9690fba640 1 -- 172.21.15.52:0/1276192279 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f968c1a6be0 con 0x7f968c1015a0 2024-04-02T13:55:46.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.631+0000 7f96837fe640 1 -- 172.21.15.52:0/1276192279 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684019980 con 0x7f968c1015a0 2024-04-02T13:55:46.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.631+0000 7f9690fba640 1 -- 172.21.15.52:0/1276192279 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f968c1a7100 con 0x7f968c1015a0 2024-04-02T13:55:46.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.633+0000 7f9690fba640 1 -- 172.21.15.52:0/1276192279 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f968c068d40 con 0x7f968c1015a0 2024-04-02T13:55:46.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.634+0000 7f96837fe640 1 -- 172.21.15.52:0/1276192279 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9684019b20 con 0x7f968c1015a0 2024-04-02T13:55:46.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.634+0000 7f96837fe640 1 --2- 172.21.15.52:0/1276192279 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f96780a3d20 0x7f96780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.634+0000 7f96837fe640 1 -- 172.21.15.52:0/1276192279 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f96840d1370 con 0x7f968c1015a0 2024-04-02T13:55:46.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.634+0000 7f9689d9b640 1 --2- 172.21.15.52:0/1276192279 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f96780a3d20 0x7f96780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.635+0000 7f9689d9b640 1 --2- 172.21.15.52:0/1276192279 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f96780a3d20 0x7f96780a61e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9674005ab0 tx=0x7f9674000bc0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:46.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.639+0000 7f96837fe640 1 -- 172.21.15.52:0/1276192279 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f968409d6d0 con 0x7f968c1015a0 2024-04-02T13:55:46.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.865+0000 7f9690fba640 1 -- 172.21.15.52:0/1276192279 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f968c02d370 con 0x7f968c1015a0 2024-04-02T13:55:46.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.866+0000 7f96837fe640 1 -- 172.21.15.52:0/1276192279 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11924 (secure 0 0 0) 0x7f968409cfd0 con 0x7f968c1015a0 2024-04-02T13:55:46.870 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:55:46.870 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:55:15.239889+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24475,24503,24490,24457,24467]},"id":1}]} 2024-04-02T13:55:46.870 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 29 2024-04-02T13:55:46.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.868+0000 7f9690fba640 1 -- 172.21.15.52:0/1276192279 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f96780a3d20 msgr2=0x7f96780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:46.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.868+0000 7f9690fba640 1 --2- 172.21.15.52:0/1276192279 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f96780a3d20 0x7f96780a61e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9674005ab0 tx=0x7f9674000bc0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.868+0000 7f9690fba640 1 -- 172.21.15.52:0/1276192279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f968c1015a0 msgr2=0x7f968c102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:46.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.868+0000 7f9690fba640 1 --2- 172.21.15.52:0/1276192279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f968c1015a0 0x7f968c102720 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f968400ebc0 tx=0x7f968400c6a0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.868+0000 7f9690fba640 1 -- 172.21.15.52:0/1276192279 shutdown_connections 2024-04-02T13:55:46.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.868+0000 7f9690fba640 1 --2- 172.21.15.52:0/1276192279 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f96780a3d20 0x7f96780a61e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.868+0000 7f9690fba640 1 --2- 172.21.15.52:0/1276192279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f968c105a30 0x7f968c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.868+0000 7f9690fba640 1 --2- 172.21.15.52:0/1276192279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f968c101f60 0x7f968c102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.868+0000 7f9690fba640 1 --2- 172.21.15.52:0/1276192279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f968c1015a0 0x7f968c102720 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.868+0000 7f9690fba640 1 -- 172.21.15.52:0/1276192279 >> 172.21.15.52:0/1276192279 conn(0x7f968c075d10 msgr2=0x7f968c0ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:46.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.869+0000 7f9690fba640 1 -- 172.21.15.52:0/1276192279 shutdown_connections 2024-04-02T13:55:46.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.869+0000 7f9690fba640 1 -- 172.21.15.52:0/1276192279 wait complete. 2024-04-02T13:55:46.883 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:55:46.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.985+0000 7f25f1c93640 1 -- 172.21.15.52:0/254826703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25ec106980 msgr2=0x7f25ec106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:46.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.985+0000 7f25f1c93640 1 --2- 172.21.15.52:0/254826703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25ec106980 0x7f25ec106d60 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f25dc009cd0 tx=0x7f25dc02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.985+0000 7f25f1c93640 1 -- 172.21.15.52:0/254826703 shutdown_connections 2024-04-02T13:55:46.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.985+0000 7f25f1c93640 1 --2- 172.21.15.52:0/254826703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25ec101330 0x7f25ec10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.985+0000 7f25f1c93640 1 --2- 172.21.15.52:0/254826703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25ec100970 0x7f25ec100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.985+0000 7f25f1c93640 1 --2- 172.21.15.52:0/254826703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25ec106980 0x7f25ec106d60 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.985+0000 7f25f1c93640 1 -- 172.21.15.52:0/254826703 >> 172.21.15.52:0/254826703 conn(0x7f25ec077b00 msgr2=0x7f25ec077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:46.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.985+0000 7f25f1c93640 1 -- 172.21.15.52:0/254826703 shutdown_connections 2024-04-02T13:55:46.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.985+0000 7f25f1c93640 1 -- 172.21.15.52:0/254826703 wait complete. 2024-04-02T13:55:46.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.985+0000 7f25f1c93640 1 Processor -- start 2024-04-02T13:55:46.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.985+0000 7f25f1c93640 1 -- start start 2024-04-02T13:55:46.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.986+0000 7f25f1c93640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25ec100970 0x7f25ec1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.986+0000 7f25f1c93640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25ec101330 0x7f25ec199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.986+0000 7f25f1c93640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25ec106980 0x7f25ec1a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.986+0000 7f25f1c93640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec110d90 con 0x7f25ec106980 2024-04-02T13:55:46.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.986+0000 7f25f1c93640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec110c10 con 0x7f25ec100970 2024-04-02T13:55:46.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.986+0000 7f25f1c93640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec110f10 con 0x7f25ec101330 2024-04-02T13:55:46.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.986+0000 7f25ebfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25ec106980 0x7f25ec1a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.986+0000 7f25ebfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25ec106980 0x7f25ec1a4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33818/0 (socket says 172.21.15.52:33818) 2024-04-02T13:55:46.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.986+0000 7f25ebfff640 1 -- 172.21.15.52:0/480531993 learned_addr learned my addr 172.21.15.52:0/480531993 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:46.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.986+0000 7f25eb7fe640 1 --2- 172.21.15.52:0/480531993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25ec100970 0x7f25ec1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.986+0000 7f25eaffd640 1 --2- 172.21.15.52:0/480531993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25ec101330 0x7f25ec199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.986+0000 7f25ebfff640 1 -- 172.21.15.52:0/480531993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25ec101330 msgr2=0x7f25ec199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:46.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.986+0000 7f25ebfff640 1 --2- 172.21.15.52:0/480531993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25ec101330 0x7f25ec199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.986+0000 7f25ebfff640 1 -- 172.21.15.52:0/480531993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25ec100970 msgr2=0x7f25ec1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:46.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.986+0000 7f25ebfff640 1 --2- 172.21.15.52:0/480531993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25ec100970 0x7f25ec1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:46.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.986+0000 7f25ebfff640 1 -- 172.21.15.52:0/480531993 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ec1a4790 con 0x7f25ec106980 2024-04-02T13:55:46.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.986+0000 7f25ebfff640 1 --2- 172.21.15.52:0/480531993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25ec106980 0x7f25ec1a4250 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f25e400d750 tx=0x7f25e400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:46.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.986+0000 7f25e8ff9640 1 -- 172.21.15.52:0/480531993 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e4017070 con 0x7f25ec106980 2024-04-02T13:55:46.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.987+0000 7f25e8ff9640 1 -- 172.21.15.52:0/480531993 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25e4005500 con 0x7f25ec106980 2024-04-02T13:55:46.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.987+0000 7f25f1c93640 1 -- 172.21.15.52:0/480531993 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25ec1a4960 con 0x7f25ec106980 2024-04-02T13:55:46.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.987+0000 7f25e8ff9640 1 -- 172.21.15.52:0/480531993 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e4004d90 con 0x7f25ec106980 2024-04-02T13:55:46.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.987+0000 7f25f1c93640 1 -- 172.21.15.52:0/480531993 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25ec1a4f00 con 0x7f25ec106980 2024-04-02T13:55:46.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.989+0000 7f25e8ff9640 1 -- 172.21.15.52:0/480531993 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f25e4004f30 con 0x7f25ec106980 2024-04-02T13:55:46.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.989+0000 7f25f487f740 1 Processor -- start 2024-04-02T13:55:46.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.989+0000 7f25f487f740 1 -- start start 2024-04-02T13:55:46.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.989+0000 7f25f487f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d009c8d7a0 0x55d009c8db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.989+0000 7f25f487f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c8e0c0 0x55d009c96760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.989+0000 7f25f487f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d009c96ca0 0x55d009c99090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.989+0000 7f25f487f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d009bf17b0 con 0x55d009c8e0c0 2024-04-02T13:55:46.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.989+0000 7f25f487f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d009b90570 con 0x55d009c96ca0 2024-04-02T13:55:46.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.989+0000 7f25f487f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d009bed760 con 0x55d009c8d7a0 2024-04-02T13:55:46.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.989+0000 7f25eb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c8e0c0 0x55d009c96760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.989+0000 7f25eb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c8e0c0 0x55d009c96760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33830/0 (socket says 172.21.15.52:33830) 2024-04-02T13:55:46.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.989+0000 7f25eb7fe640 1 -- 172.21.15.52:0/3386895412 learned_addr learned my addr 172.21.15.52:0/3386895412 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:46.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.989+0000 7f25eaffd640 1 --2- 172.21.15.52:0/3386895412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d009c96ca0 0x55d009c99090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:46.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.989+0000 7f25e8ff9640 1 --2- 172.21.15.52:0/480531993 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f25d80a3da0 0x7f25d80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:46.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.989+0000 7f25eaffd640 1 --2- 172.21.15.52:0/3386895412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d009c8d7a0 0x55d009c8db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.990+0000 7f25eb7fe640 1 -- 172.21.15.52:0/3386895412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d009c8d7a0 msgr2=0x55d009c8db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.990+0000 7f25eb7fe640 1 --2- 172.21.15.52:0/3386895412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d009c8d7a0 0x55d009c8db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.990+0000 7f25eb7fe640 1 -- 172.21.15.52:0/3386895412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d009c96ca0 msgr2=0x55d009c99090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.990+0000 7f25eb7fe640 1 --2- 172.21.15.52:0/3386895412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d009c96ca0 0x55d009c99090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.990+0000 7f25ebfff640 1 --2- 172.21.15.52:0/480531993 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f25d80a3da0 0x7f25d80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.990+0000 7f25eb7fe640 1 -- 172.21.15.52:0/3386895412 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0099053f0 con 0x55d009c8e0c0 2024-04-02T13:55:47.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.990+0000 7f25eaffd640 1 --2- 172.21.15.52:0/3386895412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d009c8d7a0 0x55d009c8db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:47.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.990+0000 7f25e8ff9640 1 -- 172.21.15.52:0/480531993 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f25e40d0350 con 0x7f25ec106980 2024-04-02T13:55:47.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.990+0000 7f25eb7fe640 1 --2- 172.21.15.52:0/3386895412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c8e0c0 0x55d009c96760 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f25dc037040 tx=0x7f25dc0099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea987c56d7b838ec server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.990+0000 7f25e1ffb640 1 -- 172.21.15.52:0/3386895412 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc043070 con 0x55d009c8e0c0 2024-04-02T13:55:47.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.990+0000 7f25ebfff640 1 --2- 172.21.15.52:0/480531993 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f25d80a3da0 0x7f25d80a6260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f25e40cd810 tx=0x7f25e4009bd0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.990+0000 7f25e1ffb640 1 -- 172.21.15.52:0/3386895412 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25dc032c90 con 0x55d009c8e0c0 2024-04-02T13:55:47.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.990+0000 7f25e1ffb640 1 -- 172.21.15.52:0/3386895412 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc039040 con 0x55d009c8e0c0 2024-04-02T13:55:47.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.991+0000 7f25f487f740 1 -- 172.21.15.52:0/3386895412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c8e0c0 msgr2=0x55d009c96760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.991+0000 7f25f487f740 1 --2- 172.21.15.52:0/3386895412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c8e0c0 0x55d009c96760 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f25dc037040 tx=0x7f25dc0099b0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.991+0000 7f25f487f740 1 -- 172.21.15.52:0/3386895412 shutdown_connections 2024-04-02T13:55:47.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.991+0000 7f25f487f740 1 --2- 172.21.15.52:0/3386895412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d009c96ca0 0x55d009c99090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.991+0000 7f25f487f740 1 --2- 172.21.15.52:0/3386895412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c8e0c0 0x55d009c96760 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.991+0000 7f25f487f740 1 --2- 172.21.15.52:0/3386895412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d009c8d7a0 0x55d009c8db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.991+0000 7f25f487f740 1 -- 172.21.15.52:0/3386895412 >> 172.21.15.52:0/3386895412 conn(0x55d009c87ed0 msgr2=0x55d009c882b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.991+0000 7f25f487f740 1 -- 172.21.15.52:0/3386895412 shutdown_connections 2024-04-02T13:55:47.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.991+0000 7f25f487f740 1 -- 172.21.15.52:0/3386895412 wait complete. 2024-04-02T13:55:47.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.991+0000 7f25f487f740 1 -- start start 2024-04-02T13:55:47.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.991+0000 7f25f487f740 1 Processor -- start 2024-04-02T13:55:47.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.992+0000 7f25f487f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d009c8d7a0 0x55d009c93b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.992+0000 7f25f487f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d009c8e0c0 0x55d009c94070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.992+0000 7f25f487f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c96ca0 0x55d009c945b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.992+0000 7f25f487f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d009c99a30 con 0x55d009c96ca0 2024-04-02T13:55:47.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.992+0000 7f25f487f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d009bace80 con 0x55d009c8e0c0 2024-04-02T13:55:47.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.992+0000 7f25f487f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d009c99bb0 con 0x55d009c8d7a0 2024-04-02T13:55:47.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.992+0000 7f25f487f740 10 client.0 _ll_register_callbacks cb 0x55d009be8aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:47.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.992+0000 7f25eaffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c96ca0 0x55d009c945b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.992+0000 7f25eaffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c96ca0 0x55d009c945b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33836/0 (socket says 172.21.15.52:33836) 2024-04-02T13:55:47.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.992+0000 7f25eaffd640 1 -- 172.21.15.52:0/3679672994 learned_addr learned my addr 172.21.15.52:0/3679672994 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:47.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.992+0000 7f25eaffd640 1 --2- 172.21.15.52:0/3679672994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d009c8d7a0 0x55d009c93b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.992+0000 7f25eb7fe640 1 --2- 172.21.15.52:0/3679672994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d009c8e0c0 0x55d009c94070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.992+0000 7f25eaffd640 1 -- 172.21.15.52:0/3679672994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d009c8d7a0 msgr2=0x55d009c93b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.992+0000 7f25eaffd640 1 --2- 172.21.15.52:0/3679672994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d009c8d7a0 0x55d009c93b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.992+0000 7f25eaffd640 1 -- 172.21.15.52:0/3679672994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d009c8e0c0 msgr2=0x55d009c94070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.992+0000 7f25eaffd640 1 --2- 172.21.15.52:0/3679672994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d009c8e0c0 0x55d009c94070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.992+0000 7f25eaffd640 1 -- 172.21.15.52:0/3679672994 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0099cf7d0 con 0x55d009c96ca0 2024-04-02T13:55:47.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.992+0000 7f25eaffd640 1 --2- 172.21.15.52:0/3679672994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c96ca0 0x55d009c945b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f25d4011ed0 tx=0x7f25d4013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737f836ea31f4828 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.992+0000 7f25caffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:47.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.992+0000 7f25caffd640 1 -- 172.21.15.52:0/3679672994 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d401a040 con 0x55d009c96ca0 2024-04-02T13:55:47.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.992+0000 7f25caffd640 1 -- 172.21.15.52:0/3679672994 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f25d4005220 con 0x55d009c96ca0 2024-04-02T13:55:47.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.992+0000 7f25f487f740 1 -- 172.21.15.52:0/3679672994 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25dc032c90 con 0x55d009c96ca0 2024-04-02T13:55:47.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.993+0000 7f25caffd640 1 -- 172.21.15.52:0/3679672994 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f25d402b050 con 0x55d009c96ca0 2024-04-02T13:55:47.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.993+0000 7f25f487f740 10 client.16278 fetch_fsmap learned FSMap version 29 2024-04-02T13:55:47.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.993+0000 7f25f487f740 1 -- 172.21.15.52:0/3679672994 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d009ca5b60 con 0x55d009c96ca0 2024-04-02T13:55:47.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.993+0000 7f25caffd640 1 -- 172.21.15.52:0/3679672994 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25dc032c90 con 0x55d009c96ca0 2024-04-02T13:55:47.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.993+0000 7f25caffd640 1 -- 172.21.15.52:0/3679672994 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f25d4020420 con 0x55d009c96ca0 2024-04-02T13:55:47.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.993+0000 7f25f487f740 10 client.16278 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:55:47.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.993+0000 7f25f487f740 10 client.16278 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T13:55:47.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.993+0000 7f25f487f740 20 client.16278 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:47.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.993+0000 7f25f487f740 1 --2- 172.21.15.52:0/3679672994 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55d009ca7020 0x55d009ca9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.993+0000 7f25f487f740 4 client.16278 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:47.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.993+0000 7f25f487f740 1 -- 172.21.15.52:0/3679672994 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d009bfecf0 con 0x55d009ca7020 2024-04-02T13:55:47.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.994+0000 7f25eb7fe640 1 --2- 172.21.15.52:0/3679672994 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55d009ca7020 0x55d009ca9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.994+0000 7f25caffd640 10 client.16278 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:55:47.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.994+0000 7f25eb7fe640 1 --2- 172.21.15.52:0/3679672994 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55d009ca7020 0x55d009ca9450 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d0ab5eee8320e2f9 server_cookie=3e2d34041ab019e1 in_seq=0 out_seq=0 2024-04-02T13:55:47.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.995+0000 7f25caffd640 1 -- 172.21.15.52:0/3679672994 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d009bfecf0 con 0x55d009ca7020 2024-04-02T13:55:47.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.995+0000 7f25caffd640 10 client.16278 handle_command_reply: tid=0 2024-04-02T13:55:47.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.995+0000 7f25f487f740 1 client.16278 shutdown 2024-04-02T13:55:47.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.995+0000 7f25f487f740 1 -- 172.21.15.52:0/3679672994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c96ca0 msgr2=0x55d009c945b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.995+0000 7f25f487f740 1 --2- 172.21.15.52:0/3679672994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c96ca0 0x55d009c945b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f25d4011ed0 tx=0x7f25d4013040 comp rx=0 tx=0).stop 2024-04-02T13:55:47.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.995+0000 7f25f487f740 1 -- 172.21.15.52:0/3679672994 shutdown_connections 2024-04-02T13:55:47.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.995+0000 7f25f487f740 1 --2- 172.21.15.52:0/3679672994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c96ca0 0x55d009c945b0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.995+0000 7f25f487f740 1 --2- 172.21.15.52:0/3679672994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d009c8e0c0 0x55d009c94070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.995+0000 7f25f487f740 1 --2- 172.21.15.52:0/3679672994 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55d009ca7020 0x55d009ca9450 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.995+0000 7f25f487f740 1 --2- 172.21.15.52:0/3679672994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d009c8d7a0 0x55d009c93b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.995+0000 7f25caffd640 0 client.16278 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:55:47.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.995+0000 7f25f487f740 1 -- 172.21.15.52:0/3679672994 >> 172.21.15.52:0/3679672994 conn(0x55d009c87ed0 msgr2=0x55d009c89c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.995+0000 7f25f487f740 1 -- 172.21.15.52:0/3679672994 shutdown_connections 2024-04-02T13:55:47.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.995+0000 7f25f487f740 1 -- 172.21.15.52:0/3679672994 wait complete. 2024-04-02T13:55:47.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:46.995+0000 7f25f487f740 20 client.16278 trim_cache size 0 max 16384 2024-04-02T13:55:47.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.008+0000 7f25f487f740 1 Processor -- start 2024-04-02T13:55:47.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.008+0000 7f25f487f740 1 -- start start 2024-04-02T13:55:47.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.009+0000 7f25f487f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d009c8d7a0 0x55d009c96260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.009+0000 7f25f487f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c8e0c0 0x55d009ca2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.009+0000 7f25f487f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d009c96ca0 0x55d009c92da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.009+0000 7f25f487f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1aeef0 con 0x55d009c8e0c0 2024-04-02T13:55:47.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.009+0000 7f25f487f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1b0a60 con 0x55d009c8d7a0 2024-04-02T13:55:47.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.009+0000 7f25f487f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1af5c0 con 0x55d009c96ca0 2024-04-02T13:55:47.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.009+0000 7f25eb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c8e0c0 0x55d009ca2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.009+0000 7f25eb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c8e0c0 0x55d009ca2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33848/0 (socket says 172.21.15.52:33848) 2024-04-02T13:55:47.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.009+0000 7f25eb7fe640 1 -- 172.21.15.52:0/2536846654 learned_addr learned my addr 172.21.15.52:0/2536846654 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:47.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.009+0000 7f25eaffd640 1 --2- 172.21.15.52:0/2536846654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d009c96ca0 0x55d009c92da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.009+0000 7f25eaffd640 1 --2- 172.21.15.52:0/2536846654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d009c8d7a0 0x55d009c96260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.009+0000 7f25eb7fe640 1 -- 172.21.15.52:0/2536846654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d009c96ca0 msgr2=0x55d009c92da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.009+0000 7f25eb7fe640 1 --2- 172.21.15.52:0/2536846654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d009c96ca0 0x55d009c92da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.009+0000 7f25eb7fe640 1 -- 172.21.15.52:0/2536846654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d009c8d7a0 msgr2=0x55d009c96260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.009+0000 7f25eb7fe640 1 --2- 172.21.15.52:0/2536846654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d009c8d7a0 0x55d009c96260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.009+0000 7f25eb7fe640 1 -- 172.21.15.52:0/2536846654 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d009c8def0 con 0x55d009c8e0c0 2024-04-02T13:55:47.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.009+0000 7f25eaffd640 1 --2- 172.21.15.52:0/2536846654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d009c8d7a0 0x55d009c96260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:47.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.009+0000 7f25eb7fe640 1 --2- 172.21.15.52:0/2536846654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c8e0c0 0x55d009ca2920 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f25dc005cc0 tx=0x7f25dc04a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7890121692e881ea server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.010+0000 7f25e17fa640 1 -- 172.21.15.52:0/2536846654 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc039040 con 0x55d009c8e0c0 2024-04-02T13:55:47.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.010+0000 7f25e17fa640 1 -- 172.21.15.52:0/2536846654 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25dc032c90 con 0x55d009c8e0c0 2024-04-02T13:55:47.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.010+0000 7f25e17fa640 1 -- 172.21.15.52:0/2536846654 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc043040 con 0x55d009c8e0c0 2024-04-02T13:55:47.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.010+0000 7f25f487f740 1 -- 172.21.15.52:0/2536846654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c8e0c0 msgr2=0x55d009ca2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.010+0000 7f25f487f740 1 --2- 172.21.15.52:0/2536846654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c8e0c0 0x55d009ca2920 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f25dc005cc0 tx=0x7f25dc04a040 comp rx=0 tx=0).stop 2024-04-02T13:55:47.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.010+0000 7f25f487f740 1 -- 172.21.15.52:0/2536846654 shutdown_connections 2024-04-02T13:55:47.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.010+0000 7f25f487f740 1 --2- 172.21.15.52:0/2536846654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d009c96ca0 0x55d009c92da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.010+0000 7f25f487f740 1 --2- 172.21.15.52:0/2536846654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c8e0c0 0x55d009ca2920 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.010+0000 7f25f487f740 1 --2- 172.21.15.52:0/2536846654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d009c8d7a0 0x55d009c96260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.010+0000 7f25f487f740 1 -- 172.21.15.52:0/2536846654 >> 172.21.15.52:0/2536846654 conn(0x55d009c87ed0 msgr2=0x55d009c94380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.010+0000 7f25f487f740 1 -- 172.21.15.52:0/2536846654 shutdown_connections 2024-04-02T13:55:47.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.010+0000 7f25f487f740 1 -- 172.21.15.52:0/2536846654 wait complete. 2024-04-02T13:55:47.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.011+0000 7f25f487f740 1 -- start start 2024-04-02T13:55:47.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.011+0000 7f25f487f740 1 Processor -- start 2024-04-02T13:55:47.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.011+0000 7f25f487f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d009c8d7a0 0x55d009c9f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.011+0000 7f25f487f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d009c8e0c0 0x55d009c9a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.011+0000 7f25f487f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c96ca0 0x55d009c9adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.011+0000 7f25f487f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1b0360 con 0x55d009c96ca0 2024-04-02T13:55:47.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.011+0000 7f25f487f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1afc90 con 0x55d009c8d7a0 2024-04-02T13:55:47.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.011+0000 7f25f487f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1aced0 con 0x55d009c8e0c0 2024-04-02T13:55:47.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.011+0000 7f25f487f740 10 client.0 _ll_register_callbacks cb 0x7f25d400acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:47.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.011+0000 7f25eaffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c96ca0 0x55d009c9adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.011+0000 7f25eaffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c96ca0 0x55d009c9adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33858/0 (socket says 172.21.15.52:33858) 2024-04-02T13:55:47.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.011+0000 7f25eaffd640 1 -- 172.21.15.52:0/3959640602 learned_addr learned my addr 172.21.15.52:0/3959640602 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:47.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.011+0000 7f25eaffd640 1 --2- 172.21.15.52:0/3959640602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d009c8d7a0 0x55d009c9f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.011+0000 7f25eb7fe640 1 --2- 172.21.15.52:0/3959640602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d009c8e0c0 0x55d009c9a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.011+0000 7f25eaffd640 1 -- 172.21.15.52:0/3959640602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d009c8e0c0 msgr2=0x55d009c9a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.011+0000 7f25eaffd640 1 --2- 172.21.15.52:0/3959640602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d009c8e0c0 0x55d009c9a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.011+0000 7f25eaffd640 1 -- 172.21.15.52:0/3959640602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d009c8d7a0 msgr2=0x55d009c9f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.011+0000 7f25eaffd640 1 --2- 172.21.15.52:0/3959640602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d009c8d7a0 0x55d009c9f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.011+0000 7f25eaffd640 1 -- 172.21.15.52:0/3959640602 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d009d0c590 con 0x55d009c96ca0 2024-04-02T13:55:47.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.012+0000 7f25eaffd640 1 --2- 172.21.15.52:0/3959640602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c96ca0 0x55d009c9adf0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f25d400ffc0 tx=0x7f25d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7adeadc3774c11f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.012+0000 7f25caffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:47.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.012+0000 7f25caffd640 1 -- 172.21.15.52:0/3959640602 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d4011250 con 0x55d009c96ca0 2024-04-02T13:55:47.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.012+0000 7f25caffd640 1 -- 172.21.15.52:0/3959640602 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f25d40047a0 con 0x55d009c96ca0 2024-04-02T13:55:47.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.012+0000 7f25f487f740 1 -- 172.21.15.52:0/3959640602 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25dc032c90 con 0x55d009c96ca0 2024-04-02T13:55:47.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.012+0000 7f25caffd640 1 -- 172.21.15.52:0/3959640602 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f25d40049a0 con 0x55d009c96ca0 2024-04-02T13:55:47.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.012+0000 7f25f487f740 10 client.16284 fetch_fsmap learned FSMap version 29 2024-04-02T13:55:47.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.012+0000 7f25f487f740 1 -- 172.21.15.52:0/3959640602 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d009d0d730 con 0x55d009c96ca0 2024-04-02T13:55:47.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.012+0000 7f25caffd640 1 -- 172.21.15.52:0/3959640602 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25dc032c90 con 0x55d009c96ca0 2024-04-02T13:55:47.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.012+0000 7f25caffd640 1 -- 172.21.15.52:0/3959640602 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f25d4005070 con 0x55d009c96ca0 2024-04-02T13:55:47.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.012+0000 7f25f487f740 10 client.16284 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:55:47.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.012+0000 7f25f487f740 10 client.16284 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T13:55:47.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.012+0000 7f25f487f740 20 client.16284 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:47.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.012+0000 7f25f487f740 1 --2- 172.21.15.52:0/3959640602 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55d009d0dd90 0x55d009d101b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.012+0000 7f25f487f740 4 client.16284 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:55:47.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.012+0000 7f25f487f740 1 -- 172.21.15.52:0/3959640602 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d009bf9040 con 0x55d009d0dd90 2024-04-02T13:55:47.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.013+0000 7f25eb7fe640 1 --2- 172.21.15.52:0/3959640602 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55d009d0dd90 0x55d009d101b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.013+0000 7f25eb7fe640 1 --2- 172.21.15.52:0/3959640602 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55d009d0dd90 0x55d009d101b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a940318745e920d2 server_cookie=822e6a4061cbc793 in_seq=0 out_seq=0 2024-04-02T13:55:47.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.013+0000 7f25caffd640 10 client.16284 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:55:47.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.013+0000 7f25caffd640 1 -- 172.21.15.52:0/3959640602 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d009bf9040 con 0x55d009d0dd90 2024-04-02T13:55:47.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.013+0000 7f25caffd640 10 client.16284 handle_command_reply: tid=0 2024-04-02T13:55:47.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.013+0000 7f25f487f740 1 client.16284 shutdown 2024-04-02T13:55:47.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.013+0000 7f25f487f740 1 -- 172.21.15.52:0/3959640602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c96ca0 msgr2=0x55d009c9adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.013+0000 7f25f487f740 1 --2- 172.21.15.52:0/3959640602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c96ca0 0x55d009c9adf0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f25d400ffc0 tx=0x7f25d4004730 comp rx=0 tx=0).stop 2024-04-02T13:55:47.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.013+0000 7f25f487f740 1 -- 172.21.15.52:0/3959640602 shutdown_connections 2024-04-02T13:55:47.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.013+0000 7f25f487f740 1 --2- 172.21.15.52:0/3959640602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d009c96ca0 0x55d009c9adf0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.013+0000 7f25f487f740 1 --2- 172.21.15.52:0/3959640602 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55d009d0dd90 0x55d009d101b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.013+0000 7f25f487f740 1 --2- 172.21.15.52:0/3959640602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d009c8e0c0 0x55d009c9a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.013+0000 7f25f487f740 1 --2- 172.21.15.52:0/3959640602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d009c8d7a0 0x55d009c9f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.013+0000 7f25caffd640 0 client.16284 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:55:47.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.013+0000 7f25f487f740 1 -- 172.21.15.52:0/3959640602 >> 172.21.15.52:0/3959640602 conn(0x55d009c87ed0 msgr2=0x55d009c866a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.013+0000 7f25f487f740 1 -- 172.21.15.52:0/3959640602 shutdown_connections 2024-04-02T13:55:47.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.013+0000 7f25f487f740 1 -- 172.21.15.52:0/3959640602 wait complete. 2024-04-02T13:55:47.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.013+0000 7f25f487f740 20 client.16284 trim_cache size 0 max 16384 2024-04-02T13:55:47.023 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:55:47.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.014+0000 7f25f1c93640 1 -- 172.21.15.52:0/480531993 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f25d80a3da0 msgr2=0x7f25d80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.014+0000 7f25f1c93640 1 --2- 172.21.15.52:0/480531993 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f25d80a3da0 0x7f25d80a6260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f25e40cd810 tx=0x7f25e4009bd0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.014+0000 7f25f1c93640 1 -- 172.21.15.52:0/480531993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25ec106980 msgr2=0x7f25ec1a4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.014+0000 7f25f1c93640 1 --2- 172.21.15.52:0/480531993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25ec106980 0x7f25ec1a4250 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f25e400d750 tx=0x7f25e400dc10 comp rx=0 tx=0).stop 2024-04-02T13:55:47.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.014+0000 7f25f1c93640 1 -- 172.21.15.52:0/480531993 shutdown_connections 2024-04-02T13:55:47.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.014+0000 7f25f1c93640 1 --2- 172.21.15.52:0/480531993 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f25d80a3da0 0x7f25d80a6260 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.014+0000 7f25f1c93640 1 --2- 172.21.15.52:0/480531993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25ec106980 0x7f25ec1a4250 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.014+0000 7f25f1c93640 1 --2- 172.21.15.52:0/480531993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25ec101330 0x7f25ec199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.014+0000 7f25f1c93640 1 --2- 172.21.15.52:0/480531993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25ec100970 0x7f25ec1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.014+0000 7f25f1c93640 1 -- 172.21.15.52:0/480531993 >> 172.21.15.52:0/480531993 conn(0x7f25ec077b00 msgr2=0x7f25ec103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.014+0000 7f25f1c93640 1 -- 172.21.15.52:0/480531993 shutdown_connections 2024-04-02T13:55:47.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.014+0000 7f25f1c93640 1 -- 172.21.15.52:0/480531993 wait complete. 2024-04-02T13:55:47.028 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:55:47.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.129+0000 7f971890e640 1 -- 172.21.15.52:0/2558254320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9714100970 msgr2=0x7f9714100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.129+0000 7f971890e640 1 --2- 172.21.15.52:0/2558254320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9714100970 0x7f9714100df0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009930 tx=0x7f96fc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:55:47.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.129+0000 7f971890e640 1 -- 172.21.15.52:0/2558254320 shutdown_connections 2024-04-02T13:55:47.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.129+0000 7f971890e640 1 --2- 172.21.15.52:0/2558254320 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9714101330 0x7f971410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.129+0000 7f971890e640 1 --2- 172.21.15.52:0/2558254320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9714100970 0x7f9714100df0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.129+0000 7f971890e640 1 --2- 172.21.15.52:0/2558254320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9714106980 0x7f9714106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.129+0000 7f971890e640 1 -- 172.21.15.52:0/2558254320 >> 172.21.15.52:0/2558254320 conn(0x7f9714077b00 msgr2=0x7f9714077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.129+0000 7f971890e640 1 -- 172.21.15.52:0/2558254320 shutdown_connections 2024-04-02T13:55:47.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.129+0000 7f971890e640 1 -- 172.21.15.52:0/2558254320 wait complete. 2024-04-02T13:55:47.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.130+0000 7f971890e640 1 Processor -- start 2024-04-02T13:55:47.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.130+0000 7f971890e640 1 -- start start 2024-04-02T13:55:47.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.130+0000 7f971890e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9714100970 0x7f971410e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.130+0000 7f971890e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9714101330 0x7f971410e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.130+0000 7f971890e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9714106980 0x7f971410ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.130+0000 7f971890e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9714111ea0 con 0x7f9714101330 2024-04-02T13:55:47.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.130+0000 7f971890e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9714111d20 con 0x7f9714100970 2024-04-02T13:55:47.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.130+0000 7f971890e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9714112020 con 0x7f9714106980 2024-04-02T13:55:47.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.130+0000 7f970bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9714101330 0x7f971410e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.130+0000 7f970bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9714101330 0x7f971410e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33880/0 (socket says 172.21.15.52:33880) 2024-04-02T13:55:47.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.130+0000 7f970bfff640 1 -- 172.21.15.52:0/3121624485 learned_addr learned my addr 172.21.15.52:0/3121624485 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:47.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.130+0000 7f971259c640 1 --2- 172.21.15.52:0/3121624485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9714100970 0x7f971410e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.131+0000 7f970bfff640 1 -- 172.21.15.52:0/3121624485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9714106980 msgr2=0x7f971410ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.131+0000 7f9712d9d640 1 --2- 172.21.15.52:0/3121624485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9714106980 0x7f971410ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.131+0000 7f970bfff640 1 --2- 172.21.15.52:0/3121624485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9714106980 0x7f971410ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.131+0000 7f970bfff640 1 -- 172.21.15.52:0/3121624485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9714100970 msgr2=0x7f971410e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.131+0000 7f970bfff640 1 --2- 172.21.15.52:0/3121624485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9714100970 0x7f971410e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.131+0000 7f970bfff640 1 -- 172.21.15.52:0/3121624485 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97141a68e0 con 0x7f9714101330 2024-04-02T13:55:47.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.131+0000 7f970bfff640 1 --2- 172.21.15.52:0/3121624485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9714101330 0x7f971410e690 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009230 tx=0x7f96fc02cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.131+0000 7f970b7fe640 1 -- 172.21.15.52:0/3121624485 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc041070 con 0x7f9714101330 2024-04-02T13:55:47.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.131+0000 7f970b7fe640 1 -- 172.21.15.52:0/3121624485 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96fc004450 con 0x7f9714101330 2024-04-02T13:55:47.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.131+0000 7f971890e640 1 -- 172.21.15.52:0/3121624485 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97141a6b70 con 0x7f9714101330 2024-04-02T13:55:47.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.131+0000 7f970b7fe640 1 -- 172.21.15.52:0/3121624485 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc034440 con 0x7f9714101330 2024-04-02T13:55:47.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.131+0000 7f971890e640 1 -- 172.21.15.52:0/3121624485 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97141a70b0 con 0x7f9714101330 2024-04-02T13:55:47.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.133+0000 7f970b7fe640 1 -- 172.21.15.52:0/3121624485 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f96fc044070 con 0x7f9714101330 2024-04-02T13:55:47.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.133+0000 7f971b5cd740 1 Processor -- start 2024-04-02T13:55:47.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.134+0000 7f971b5cd740 1 -- start start 2024-04-02T13:55:47.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.134+0000 7f971b5cd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a484787a0 0x559a48478b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.134+0000 7f971b5cd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a484790c0 0x559a48481760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.134+0000 7f971b5cd740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a48481ca0 0x559a48484090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.134+0000 7f971b5cd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559a483dc7b0 con 0x559a484787a0 2024-04-02T13:55:47.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.134+0000 7f971b5cd740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559a4837b570 con 0x559a48481ca0 2024-04-02T13:55:47.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.134+0000 7f971b5cd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559a483d8760 con 0x559a484790c0 2024-04-02T13:55:47.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.134+0000 7f970b7fe640 1 --2- 172.21.15.52:0/3121624485 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f97000a3da0 0x7f97000a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.134+0000 7f9712d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a484787a0 0x559a48478b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.134+0000 7f9712d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a484787a0 0x559a48478b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33886/0 (socket says 172.21.15.52:33886) 2024-04-02T13:55:47.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.134+0000 7f9712d9d640 1 -- 172.21.15.52:0/3660065684 learned_addr learned my addr 172.21.15.52:0/3660065684 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:47.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.134+0000 7f9712d9d640 1 --2- 172.21.15.52:0/3121624485 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f97000a3da0 0x7f97000a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.134+0000 7f970bfff640 1 --2- 172.21.15.52:0/3660065684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a48481ca0 0x559a48484090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.134+0000 7f971259c640 1 --2- 172.21.15.52:0/3660065684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a484790c0 0x559a48481760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.134+0000 7f970b7fe640 1 -- 172.21.15.52:0/3121624485 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f96fc0f23e0 con 0x7f9714101330 2024-04-02T13:55:47.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.134+0000 7f9712d9d640 1 -- 172.21.15.52:0/3660065684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a484790c0 msgr2=0x559a48481760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.134+0000 7f9712d9d640 1 --2- 172.21.15.52:0/3660065684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a484790c0 0x559a48481760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.134+0000 7f9712d9d640 1 -- 172.21.15.52:0/3660065684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a48481ca0 msgr2=0x559a48484090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.134+0000 7f9712d9d640 1 --2- 172.21.15.52:0/3660065684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a48481ca0 0x559a48484090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.134+0000 7f9712d9d640 1 -- 172.21.15.52:0/3660065684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a480f03f0 con 0x559a484787a0 2024-04-02T13:55:47.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.135+0000 7f9712d9d640 1 --2- 172.21.15.52:0/3660065684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a484787a0 0x559a48478b80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f970c009270 tx=0x7f970c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698b2602b6adf8ba server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.135+0000 7f9712d9d640 1 --2- 172.21.15.52:0/3121624485 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f97000a3da0 0x7f97000a6260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f970c011cb0 tx=0x7f970c01c000 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.135+0000 7f9708ff9640 1 -- 172.21.15.52:0/3660065684 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c00cce0 con 0x559a484787a0 2024-04-02T13:55:47.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.135+0000 7f9708ff9640 1 -- 172.21.15.52:0/3660065684 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f970c003e50 con 0x559a484787a0 2024-04-02T13:55:47.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.135+0000 7f9708ff9640 1 -- 172.21.15.52:0/3660065684 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c004230 con 0x559a484787a0 2024-04-02T13:55:47.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.135+0000 7f971b5cd740 1 -- 172.21.15.52:0/3660065684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a484787a0 msgr2=0x559a48478b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.135+0000 7f971b5cd740 1 --2- 172.21.15.52:0/3660065684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a484787a0 0x559a48478b80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f970c009270 tx=0x7f970c0117b0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.136+0000 7f971b5cd740 1 -- 172.21.15.52:0/3660065684 shutdown_connections 2024-04-02T13:55:47.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.136+0000 7f971b5cd740 1 --2- 172.21.15.52:0/3660065684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a48481ca0 0x559a48484090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.136+0000 7f971b5cd740 1 --2- 172.21.15.52:0/3660065684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a484790c0 0x559a48481760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.136+0000 7f971b5cd740 1 --2- 172.21.15.52:0/3660065684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a484787a0 0x559a48478b80 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.136+0000 7f971b5cd740 1 -- 172.21.15.52:0/3660065684 >> 172.21.15.52:0/3660065684 conn(0x559a48472ed0 msgr2=0x559a484732b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.136+0000 7f971b5cd740 1 -- 172.21.15.52:0/3660065684 shutdown_connections 2024-04-02T13:55:47.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.136+0000 7f971b5cd740 1 -- 172.21.15.52:0/3660065684 wait complete. 2024-04-02T13:55:47.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.136+0000 7f971b5cd740 1 -- start start 2024-04-02T13:55:47.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.136+0000 7f971b5cd740 1 Processor -- start 2024-04-02T13:55:47.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.136+0000 7f971b5cd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a484787a0 0x559a4847eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.136+0000 7f971b5cd740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a484790c0 0x559a4847f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.136+0000 7f971b5cd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a48481ca0 0x559a4847f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.136+0000 7f971b5cd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559a48484a30 con 0x559a48481ca0 2024-04-02T13:55:47.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.136+0000 7f971b5cd740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559a48397e80 con 0x559a484790c0 2024-04-02T13:55:47.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.136+0000 7f971b5cd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559a48484bb0 con 0x559a484787a0 2024-04-02T13:55:47.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.136+0000 7f971b5cd740 10 client.0 _ll_register_callbacks cb 0x559a483d3aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:47.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.136+0000 7f9712d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a48481ca0 0x559a4847f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.136+0000 7f9712d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a48481ca0 0x559a4847f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33896/0 (socket says 172.21.15.52:33896) 2024-04-02T13:55:47.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.136+0000 7f9712d9d640 1 -- 172.21.15.52:0/171207797 learned_addr learned my addr 172.21.15.52:0/171207797 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:47.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.136+0000 7f971259c640 1 --2- 172.21.15.52:0/171207797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a484787a0 0x559a4847eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.137+0000 7f970bfff640 1 --2- 172.21.15.52:0/171207797 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a484790c0 0x559a4847f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.137+0000 7f9712d9d640 1 -- 172.21.15.52:0/171207797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a484787a0 msgr2=0x559a4847eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.137+0000 7f9712d9d640 1 --2- 172.21.15.52:0/171207797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a484787a0 0x559a4847eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.137+0000 7f9712d9d640 1 -- 172.21.15.52:0/171207797 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a484790c0 msgr2=0x559a4847f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.137+0000 7f9712d9d640 1 --2- 172.21.15.52:0/171207797 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a484790c0 0x559a4847f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.137+0000 7f9712d9d640 1 -- 172.21.15.52:0/171207797 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a481ba7d0 con 0x559a48481ca0 2024-04-02T13:55:47.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.137+0000 7f9712d9d640 1 --2- 172.21.15.52:0/171207797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a48481ca0 0x559a4847f5e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f970c009540 tx=0x7f970c0042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba500d1f51ddb5e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.137+0000 7f96f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:47.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.137+0000 7f96f9ffb640 1 -- 172.21.15.52:0/171207797 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c033060 con 0x559a48481ca0 2024-04-02T13:55:47.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.137+0000 7f96f9ffb640 1 -- 172.21.15.52:0/171207797 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f970c003b80 con 0x559a48481ca0 2024-04-02T13:55:47.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.137+0000 7f971b5cd740 1 -- 172.21.15.52:0/171207797 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f970c003e50 con 0x559a48481ca0 2024-04-02T13:55:47.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.137+0000 7f96f9ffb640 1 -- 172.21.15.52:0/171207797 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f970c03a050 con 0x559a48481ca0 2024-04-02T13:55:47.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.137+0000 7f971b5cd740 10 client.16296 fetch_fsmap learned FSMap version 29 2024-04-02T13:55:47.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.137+0000 7f971b5cd740 1 -- 172.21.15.52:0/171207797 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559a48490b90 con 0x559a48481ca0 2024-04-02T13:55:47.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.137+0000 7f96f9ffb640 1 -- 172.21.15.52:0/171207797 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f970c003e50 con 0x559a48481ca0 2024-04-02T13:55:47.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.138+0000 7f96f9ffb640 1 -- 172.21.15.52:0/171207797 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f970c01d070 con 0x559a48481ca0 2024-04-02T13:55:47.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.138+0000 7f971b5cd740 10 client.16296 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:55:47.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.138+0000 7f971b5cd740 10 client.16296 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T13:55:47.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.138+0000 7f971b5cd740 20 client.16296 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:47.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.138+0000 7f971b5cd740 1 --2- 172.21.15.52:0/171207797 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x559a48492050 0x559a48494440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.138+0000 7f971b5cd740 4 client.16296 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:47.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.138+0000 7f971b5cd740 1 -- 172.21.15.52:0/171207797 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a483e9cf0 con 0x559a48492050 2024-04-02T13:55:47.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.138+0000 7f971259c640 1 --2- 172.21.15.52:0/171207797 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x559a48492050 0x559a48494440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.138+0000 7f971259c640 1 --2- 172.21.15.52:0/171207797 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x559a48492050 0x559a48494440 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=41223796d6575e69 server_cookie=79ba033a5f6da384 in_seq=0 out_seq=0 2024-04-02T13:55:47.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.138+0000 7f96f9ffb640 10 client.16296 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:55:47.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.139+0000 7f96f9ffb640 1 -- 172.21.15.52:0/171207797 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a483e9cf0 con 0x559a48492050 2024-04-02T13:55:47.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.139+0000 7f96f9ffb640 10 client.16296 handle_command_reply: tid=0 2024-04-02T13:55:47.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.139+0000 7f971b5cd740 1 client.16296 shutdown 2024-04-02T13:55:47.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.139+0000 7f971b5cd740 1 -- 172.21.15.52:0/171207797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a48481ca0 msgr2=0x559a4847f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.139+0000 7f971b5cd740 1 --2- 172.21.15.52:0/171207797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a48481ca0 0x559a4847f5e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f970c009540 tx=0x7f970c0042b0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.139+0000 7f971b5cd740 1 -- 172.21.15.52:0/171207797 shutdown_connections 2024-04-02T13:55:47.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.139+0000 7f971b5cd740 1 --2- 172.21.15.52:0/171207797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a48481ca0 0x559a4847f5e0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.139+0000 7f971b5cd740 1 --2- 172.21.15.52:0/171207797 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x559a48492050 0x559a48494440 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.139+0000 7f971b5cd740 1 --2- 172.21.15.52:0/171207797 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a484790c0 0x559a4847f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.139+0000 7f971b5cd740 1 --2- 172.21.15.52:0/171207797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a484787a0 0x559a4847eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.139+0000 7f971b5cd740 1 -- 172.21.15.52:0/171207797 >> 172.21.15.52:0/171207797 conn(0x559a48472ed0 msgr2=0x559a48474c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.139+0000 7f96f9ffb640 0 client.16296 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:55:47.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.139+0000 7f971b5cd740 1 -- 172.21.15.52:0/171207797 shutdown_connections 2024-04-02T13:55:47.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.139+0000 7f971b5cd740 1 -- 172.21.15.52:0/171207797 wait complete. 2024-04-02T13:55:47.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.139+0000 7f971b5cd740 20 client.16296 trim_cache size 0 max 16384 2024-04-02T13:55:47.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.153+0000 7f971b5cd740 1 Processor -- start 2024-04-02T13:55:47.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.153+0000 7f971b5cd740 1 -- start start 2024-04-02T13:55:47.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.153+0000 7f971b5cd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a484787a0 0x559a4848d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.153+0000 7f971b5cd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a484790c0 0x559a4847dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.153+0000 7f971b5cd740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a48481ca0 0x559a4847f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.153+0000 7f971b5cd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f97141b1030 con 0x559a484790c0 2024-04-02T13:55:47.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.153+0000 7f971b5cd740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f97141b2ba0 con 0x559a48481ca0 2024-04-02T13:55:47.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.153+0000 7f971b5cd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f97141b1700 con 0x559a484787a0 2024-04-02T13:55:47.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.153+0000 7f970bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a484790c0 0x559a4847dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.153+0000 7f970bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a484790c0 0x559a4847dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33910/0 (socket says 172.21.15.52:33910) 2024-04-02T13:55:47.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.153+0000 7f970bfff640 1 -- 172.21.15.52:0/1699985982 learned_addr learned my addr 172.21.15.52:0/1699985982 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:47.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.153+0000 7f9712d9d640 1 --2- 172.21.15.52:0/1699985982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a48481ca0 0x559a4847f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.153+0000 7f971259c640 1 --2- 172.21.15.52:0/1699985982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a484787a0 0x559a4848d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.153+0000 7f970bfff640 1 -- 172.21.15.52:0/1699985982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a484787a0 msgr2=0x559a4848d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.153+0000 7f970bfff640 1 --2- 172.21.15.52:0/1699985982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a484787a0 0x559a4848d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.153+0000 7f970bfff640 1 -- 172.21.15.52:0/1699985982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a48481ca0 msgr2=0x559a4847f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.153+0000 7f970bfff640 1 --2- 172.21.15.52:0/1699985982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a48481ca0 0x559a4847f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.153+0000 7f970bfff640 1 -- 172.21.15.52:0/1699985982 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a48478170 con 0x559a484790c0 2024-04-02T13:55:47.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.153+0000 7f9712d9d640 1 --2- 172.21.15.52:0/1699985982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a48481ca0 0x559a4847f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:47.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.153+0000 7f970bfff640 1 --2- 172.21.15.52:0/1699985982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a484790c0 0x559a4847dcc0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0ba5b0 tx=0x7f96fc0f2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fa9253d1505817 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.154+0000 7f96fbfff640 1 -- 172.21.15.52:0/1699985982 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc04b040 con 0x559a484790c0 2024-04-02T13:55:47.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.154+0000 7f96fbfff640 1 -- 172.21.15.52:0/1699985982 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96fc0be850 con 0x559a484790c0 2024-04-02T13:55:47.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.154+0000 7f96fbfff640 1 -- 172.21.15.52:0/1699985982 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc0f20b0 con 0x559a484790c0 2024-04-02T13:55:47.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.154+0000 7f971b5cd740 1 -- 172.21.15.52:0/1699985982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a484790c0 msgr2=0x559a4847dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.154+0000 7f971b5cd740 1 --2- 172.21.15.52:0/1699985982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a484790c0 0x559a4847dcc0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0ba5b0 tx=0x7f96fc0f2040 comp rx=0 tx=0).stop 2024-04-02T13:55:47.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.154+0000 7f971b5cd740 1 -- 172.21.15.52:0/1699985982 shutdown_connections 2024-04-02T13:55:47.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.154+0000 7f971b5cd740 1 --2- 172.21.15.52:0/1699985982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a48481ca0 0x559a4847f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.154+0000 7f971b5cd740 1 --2- 172.21.15.52:0/1699985982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a484790c0 0x559a4847dcc0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.154+0000 7f971b5cd740 1 --2- 172.21.15.52:0/1699985982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a484787a0 0x559a4848d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.154+0000 7f971b5cd740 1 -- 172.21.15.52:0/1699985982 >> 172.21.15.52:0/1699985982 conn(0x559a48472ed0 msgr2=0x559a484742c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.154+0000 7f971b5cd740 1 -- 172.21.15.52:0/1699985982 shutdown_connections 2024-04-02T13:55:47.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.154+0000 7f971b5cd740 1 -- 172.21.15.52:0/1699985982 wait complete. 2024-04-02T13:55:47.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.155+0000 7f971b5cd740 1 -- start start 2024-04-02T13:55:47.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.155+0000 7f971b5cd740 1 Processor -- start 2024-04-02T13:55:47.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.155+0000 7f971b5cd740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a484787a0 0x559a48489bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.155+0000 7f971b5cd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a484790c0 0x559a4848a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.155+0000 7f971b5cd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a48481ca0 0x559a48484ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.155+0000 7f971b5cd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f97141b24a0 con 0x559a484790c0 2024-04-02T13:55:47.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.155+0000 7f971b5cd740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f97141b1dd0 con 0x559a484787a0 2024-04-02T13:55:47.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.155+0000 7f971b5cd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f97141af010 con 0x559a48481ca0 2024-04-02T13:55:47.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.155+0000 7f971b5cd740 10 client.0 _ll_register_callbacks cb 0x7f970c014f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:47.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.155+0000 7f970bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a484790c0 0x559a4848a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.155+0000 7f970bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a484790c0 0x559a4848a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33924/0 (socket says 172.21.15.52:33924) 2024-04-02T13:55:47.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.155+0000 7f970bfff640 1 -- 172.21.15.52:0/166806759 learned_addr learned my addr 172.21.15.52:0/166806759 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:47.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.155+0000 7f971259c640 1 --2- 172.21.15.52:0/166806759 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a484787a0 0x559a48489bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.155+0000 7f970bfff640 1 -- 172.21.15.52:0/166806759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a48481ca0 msgr2=0x559a48484ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.155+0000 7f970bfff640 1 --2- 172.21.15.52:0/166806759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a48481ca0 0x559a48484ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.155+0000 7f970bfff640 1 -- 172.21.15.52:0/166806759 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a484787a0 msgr2=0x559a48489bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.155+0000 7f970bfff640 1 --2- 172.21.15.52:0/166806759 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a484787a0 0x559a48489bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.155+0000 7f970bfff640 1 -- 172.21.15.52:0/166806759 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a484857f0 con 0x559a484790c0 2024-04-02T13:55:47.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.156+0000 7f970bfff640 1 --2- 172.21.15.52:0/166806759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a484790c0 0x559a4848a130 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0ef9b0 tx=0x7f96fc0bdc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5ada46eff73553 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.156+0000 7f96f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:47.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.156+0000 7f96f9ffb640 1 -- 172.21.15.52:0/166806759 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc04b040 con 0x559a484790c0 2024-04-02T13:55:47.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.156+0000 7f96f9ffb640 1 -- 172.21.15.52:0/166806759 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f96fc0ba7c0 con 0x559a484790c0 2024-04-02T13:55:47.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.156+0000 7f971b5cd740 1 -- 172.21.15.52:0/166806759 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96fc0be850 con 0x559a484790c0 2024-04-02T13:55:47.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.156+0000 7f96f9ffb640 1 -- 172.21.15.52:0/166806759 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f96fc0be150 con 0x559a484790c0 2024-04-02T13:55:47.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.156+0000 7f971b5cd740 10 client.16302 fetch_fsmap learned FSMap version 29 2024-04-02T13:55:47.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.156+0000 7f971b5cd740 1 -- 172.21.15.52:0/166806759 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559a484f7cc0 con 0x559a484790c0 2024-04-02T13:55:47.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.156+0000 7f96f9ffb640 1 -- 172.21.15.52:0/166806759 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96fc0be850 con 0x559a484790c0 2024-04-02T13:55:47.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.156+0000 7f96f9ffb640 1 -- 172.21.15.52:0/166806759 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f96fc0c20d0 con 0x559a484790c0 2024-04-02T13:55:47.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.156+0000 7f971b5cd740 10 client.16302 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:55:47.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.156+0000 7f971b5cd740 10 client.16302 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T13:55:47.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.156+0000 7f971b5cd740 20 client.16302 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:47.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.156+0000 7f971b5cd740 1 --2- 172.21.15.52:0/166806759 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x559a484f8050 0x559a484fa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.156+0000 7f971b5cd740 4 client.16302 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:55:47.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.156+0000 7f971b5cd740 1 -- 172.21.15.52:0/166806759 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a483e4040 con 0x559a484f8050 2024-04-02T13:55:47.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.156+0000 7f971259c640 1 --2- 172.21.15.52:0/166806759 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x559a484f8050 0x559a484fa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.157+0000 7f96f9ffb640 10 client.16302 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:55:47.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.157+0000 7f971259c640 1 --2- 172.21.15.52:0/166806759 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x559a484f8050 0x559a484fa440 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4fe9453977dbc7b0 server_cookie=756ee970b3860e5e in_seq=0 out_seq=0 2024-04-02T13:55:47.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.157+0000 7f96f9ffb640 1 -- 172.21.15.52:0/166806759 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a483e4040 con 0x559a484f8050 2024-04-02T13:55:47.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.157+0000 7f96f9ffb640 10 client.16302 handle_command_reply: tid=0 2024-04-02T13:55:47.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.157+0000 7f971b5cd740 1 client.16302 shutdown 2024-04-02T13:55:47.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.157+0000 7f971b5cd740 1 -- 172.21.15.52:0/166806759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a484790c0 msgr2=0x559a4848a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.157+0000 7f971b5cd740 1 --2- 172.21.15.52:0/166806759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a484790c0 0x559a4848a130 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0ef9b0 tx=0x7f96fc0bdc90 comp rx=0 tx=0).stop 2024-04-02T13:55:47.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.157+0000 7f971b5cd740 1 -- 172.21.15.52:0/166806759 shutdown_connections 2024-04-02T13:55:47.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.157+0000 7f971b5cd740 1 --2- 172.21.15.52:0/166806759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a48481ca0 0x559a48484ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.157+0000 7f971b5cd740 1 --2- 172.21.15.52:0/166806759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a484790c0 0x559a4848a130 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.157+0000 7f971b5cd740 1 --2- 172.21.15.52:0/166806759 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x559a484f8050 0x559a484fa440 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.157+0000 7f971b5cd740 1 --2- 172.21.15.52:0/166806759 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a484787a0 0x559a48489bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.157+0000 7f96f9ffb640 0 client.16302 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:55:47.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.157+0000 7f971b5cd740 1 -- 172.21.15.52:0/166806759 >> 172.21.15.52:0/166806759 conn(0x559a48472ed0 msgr2=0x559a484716a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.157+0000 7f971b5cd740 1 -- 172.21.15.52:0/166806759 shutdown_connections 2024-04-02T13:55:47.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.157+0000 7f971b5cd740 1 -- 172.21.15.52:0/166806759 wait complete. 2024-04-02T13:55:47.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.157+0000 7f971b5cd740 20 client.16302 trim_cache size 0 max 16384 2024-04-02T13:55:47.168 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:55:47.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.158+0000 7f971890e640 1 -- 172.21.15.52:0/3121624485 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f97000a3da0 msgr2=0x7f97000a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.158+0000 7f971890e640 1 --2- 172.21.15.52:0/3121624485 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f97000a3da0 0x7f97000a6260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f970c011cb0 tx=0x7f970c01c000 comp rx=0 tx=0).stop 2024-04-02T13:55:47.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.158+0000 7f971890e640 1 -- 172.21.15.52:0/3121624485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9714101330 msgr2=0x7f971410e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.158+0000 7f971890e640 1 --2- 172.21.15.52:0/3121624485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9714101330 0x7f971410e690 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009230 tx=0x7f96fc02cba0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.158+0000 7f971890e640 1 -- 172.21.15.52:0/3121624485 shutdown_connections 2024-04-02T13:55:47.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.158+0000 7f971890e640 1 --2- 172.21.15.52:0/3121624485 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f97000a3da0 0x7f97000a6260 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.158+0000 7f971890e640 1 --2- 172.21.15.52:0/3121624485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9714106980 0x7f971410ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.158+0000 7f971890e640 1 --2- 172.21.15.52:0/3121624485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9714101330 0x7f971410e690 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.158+0000 7f971890e640 1 --2- 172.21.15.52:0/3121624485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9714100970 0x7f971410e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.158+0000 7f971890e640 1 -- 172.21.15.52:0/3121624485 >> 172.21.15.52:0/3121624485 conn(0x7f9714077b00 msgr2=0x7f9714103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.158+0000 7f971890e640 1 -- 172.21.15.52:0/3121624485 shutdown_connections 2024-04-02T13:55:47.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.158+0000 7f971890e640 1 -- 172.21.15.52:0/3121624485 wait complete. 2024-04-02T13:55:47.172 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:55:47.180 INFO:tasks.workunit.client.0.smithi052.stdout:autoheader 2024-04-02T13:55:47.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.274+0000 7fd221277640 1 -- 172.21.15.52:0/1652746418 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd21c102450 msgr2=0x7fd21c10e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.274+0000 7fd221277640 1 --2- 172.21.15.52:0/1652746418 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd21c102450 0x7fd21c10e960 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd214009cd0 tx=0x7fd21402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.274+0000 7fd221277640 1 -- 172.21.15.52:0/1652746418 shutdown_connections 2024-04-02T13:55:47.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.274+0000 7fd221277640 1 --2- 172.21.15.52:0/1652746418 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd21c102450 0x7fd21c10e960 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.274+0000 7fd221277640 1 --2- 172.21.15.52:0/1652746418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd21c101a90 0x7fd21c101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.274+0000 7fd221277640 1 --2- 172.21.15.52:0/1652746418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd21c107aa0 0x7fd21c107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.274+0000 7fd221277640 1 -- 172.21.15.52:0/1652746418 >> 172.21.15.52:0/1652746418 conn(0x7fd21c078f00 msgr2=0x7fd21c079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.274+0000 7fd221277640 1 -- 172.21.15.52:0/1652746418 shutdown_connections 2024-04-02T13:55:47.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.274+0000 7fd221277640 1 -- 172.21.15.52:0/1652746418 wait complete. 2024-04-02T13:55:47.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.275+0000 7fd221277640 1 Processor -- start 2024-04-02T13:55:47.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.275+0000 7fd221277640 1 -- start start 2024-04-02T13:55:47.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.275+0000 7fd221277640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd21c101a90 0x7fd21c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.275+0000 7fd221277640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd21c102450 0x7fd21c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.275+0000 7fd221277640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd21c107aa0 0x7fd21c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.275+0000 7fd221277640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c111eb0 con 0x7fd21c102450 2024-04-02T13:55:47.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.275+0000 7fd221277640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c111d30 con 0x7fd21c107aa0 2024-04-02T13:55:47.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.275+0000 7fd221277640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c112030 con 0x7fd21c101a90 2024-04-02T13:55:47.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.275+0000 7fd21a7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd21c102450 0x7fd21c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.275+0000 7fd21a7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd21c102450 0x7fd21c1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33942/0 (socket says 172.21.15.52:33942) 2024-04-02T13:55:47.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.275+0000 7fd21a7fc640 1 -- 172.21.15.52:0/838248364 learned_addr learned my addr 172.21.15.52:0/838248364 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:47.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.275+0000 7fd21affd640 1 --2- 172.21.15.52:0/838248364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd21c101a90 0x7fd21c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.275+0000 7fd21b7fe640 1 --2- 172.21.15.52:0/838248364 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd21c107aa0 0x7fd21c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.276+0000 7fd21a7fc640 1 -- 172.21.15.52:0/838248364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd21c101a90 msgr2=0x7fd21c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.276+0000 7fd21a7fc640 1 --2- 172.21.15.52:0/838248364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd21c101a90 0x7fd21c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.276+0000 7fd21a7fc640 1 -- 172.21.15.52:0/838248364 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd21c107aa0 msgr2=0x7fd21c19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.276+0000 7fd21a7fc640 1 --2- 172.21.15.52:0/838248364 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd21c107aa0 0x7fd21c19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.276+0000 7fd21a7fc640 1 -- 172.21.15.52:0/838248364 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21c19d3a0 con 0x7fd21c102450 2024-04-02T13:55:47.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.276+0000 7fd21a7fc640 1 --2- 172.21.15.52:0/838248364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd21c102450 0x7fd21c1988f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd21000e9f0 tx=0x7fd21000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.276+0000 7fd203fff640 1 -- 172.21.15.52:0/838248364 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21000cd80 con 0x7fd21c102450 2024-04-02T13:55:47.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.276+0000 7fd203fff640 1 -- 172.21.15.52:0/838248364 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd210010910 con 0x7fd21c102450 2024-04-02T13:55:47.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.276+0000 7fd221277640 1 -- 172.21.15.52:0/838248364 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd21c19d690 con 0x7fd21c102450 2024-04-02T13:55:47.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.276+0000 7fd203fff640 1 -- 172.21.15.52:0/838248364 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd210018990 con 0x7fd21c102450 2024-04-02T13:55:47.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.276+0000 7fd221277640 1 -- 172.21.15.52:0/838248364 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd21c1a4f20 con 0x7fd21c102450 2024-04-02T13:55:47.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.279+0000 7fd203fff640 1 -- 172.21.15.52:0/838248364 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd210010ac0 con 0x7fd21c102450 2024-04-02T13:55:47.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.279+0000 7fd223e5c740 1 Processor -- start 2024-04-02T13:55:47.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.279+0000 7fd223e5c740 1 -- start start 2024-04-02T13:55:47.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.279+0000 7fd223e5c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0dd2574d0 0x55c0dd2578b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.279+0000 7fd223e5c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd257df0 0x55c0dd264b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.279+0000 7fd223e5c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c0dd2650b0 0x55c0dd2674a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.279+0000 7fd223e5c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c0dd1bb7b0 con 0x55c0dd257df0 2024-04-02T13:55:47.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.279+0000 7fd223e5c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c0dd15a570 con 0x55c0dd2650b0 2024-04-02T13:55:47.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.279+0000 7fd223e5c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c0dd1b7760 con 0x55c0dd2574d0 2024-04-02T13:55:47.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.279+0000 7fd203fff640 1 --2- 172.21.15.52:0/838248364 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd2040a3d20 0x7fd2040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.279+0000 7fd21affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd257df0 0x55c0dd264b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.279+0000 7fd21affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd257df0 0x55c0dd264b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33946/0 (socket says 172.21.15.52:33946) 2024-04-02T13:55:47.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.279+0000 7fd21affd640 1 -- 172.21.15.52:0/3547590236 learned_addr learned my addr 172.21.15.52:0/3547590236 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:47.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.279+0000 7fd21a7fc640 1 --2- 172.21.15.52:0/3547590236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c0dd2650b0 0x55c0dd2674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.280+0000 7fd21b7fe640 1 --2- 172.21.15.52:0/838248364 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd2040a3d20 0x7fd2040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.280+0000 7fd21b7fe640 1 --2- 172.21.15.52:0/3547590236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0dd2574d0 0x55c0dd2578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.280+0000 7fd203fff640 1 -- 172.21.15.52:0/838248364 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd2100d0fc0 con 0x7fd21c102450 2024-04-02T13:55:47.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.280+0000 7fd21affd640 1 -- 172.21.15.52:0/3547590236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0dd2574d0 msgr2=0x55c0dd2578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.280+0000 7fd21affd640 1 --2- 172.21.15.52:0/3547590236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0dd2574d0 0x55c0dd2578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.280+0000 7fd21affd640 1 -- 172.21.15.52:0/3547590236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c0dd2650b0 msgr2=0x55c0dd2674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.280+0000 7fd21affd640 1 --2- 172.21.15.52:0/3547590236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c0dd2650b0 0x55c0dd2674a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.280+0000 7fd21affd640 1 -- 172.21.15.52:0/3547590236 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0dcecf3f0 con 0x55c0dd257df0 2024-04-02T13:55:47.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.280+0000 7fd21b7fe640 1 --2- 172.21.15.52:0/3547590236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0dd2574d0 0x55c0dd2578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:47.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.280+0000 7fd21a7fc640 1 --2- 172.21.15.52:0/3547590236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c0dd2650b0 0x55c0dd2674a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:47.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.280+0000 7fd21affd640 1 --2- 172.21.15.52:0/3547590236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd257df0 0x55c0dd264b70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd20c009c10 tx=0x7fd20c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf65d5df10083b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.280+0000 7fd2017fa640 1 -- 172.21.15.52:0/3547590236 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c012450 con 0x55c0dd257df0 2024-04-02T13:55:47.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.280+0000 7fd21b7fe640 1 --2- 172.21.15.52:0/838248364 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd2040a3d20 0x7fd2040a61e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd214006010 tx=0x7fd21402e040 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.280+0000 7fd2017fa640 1 -- 172.21.15.52:0/3547590236 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd20c012ad0 con 0x55c0dd257df0 2024-04-02T13:55:47.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.280+0000 7fd2017fa640 1 -- 172.21.15.52:0/3547590236 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c00fb60 con 0x55c0dd257df0 2024-04-02T13:55:47.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.281+0000 7fd223e5c740 1 -- 172.21.15.52:0/3547590236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd257df0 msgr2=0x55c0dd264b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.281+0000 7fd223e5c740 1 --2- 172.21.15.52:0/3547590236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd257df0 0x55c0dd264b70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd20c009c10 tx=0x7fd20c0117b0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.281+0000 7fd223e5c740 1 -- 172.21.15.52:0/3547590236 shutdown_connections 2024-04-02T13:55:47.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.281+0000 7fd223e5c740 1 --2- 172.21.15.52:0/3547590236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c0dd2650b0 0x55c0dd2674a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.281+0000 7fd223e5c740 1 --2- 172.21.15.52:0/3547590236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd257df0 0x55c0dd264b70 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.281+0000 7fd223e5c740 1 --2- 172.21.15.52:0/3547590236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0dd2574d0 0x55c0dd2578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.281+0000 7fd223e5c740 1 -- 172.21.15.52:0/3547590236 >> 172.21.15.52:0/3547590236 conn(0x55c0dd251ed0 msgr2=0x55c0dd2522b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.281+0000 7fd223e5c740 1 -- 172.21.15.52:0/3547590236 shutdown_connections 2024-04-02T13:55:47.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.281+0000 7fd223e5c740 1 -- 172.21.15.52:0/3547590236 wait complete. 2024-04-02T13:55:47.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.281+0000 7fd223e5c740 1 -- start start 2024-04-02T13:55:47.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.281+0000 7fd223e5c740 1 Processor -- start 2024-04-02T13:55:47.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.281+0000 7fd223e5c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd2574d0 0x55c0dd264b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.282+0000 7fd223e5c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c0dd257df0 0x55c0dd261ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.282+0000 7fd223e5c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0dd2650b0 0x55c0dd262600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.282+0000 7fd223e5c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c0dd267e70 con 0x55c0dd2574d0 2024-04-02T13:55:47.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.282+0000 7fd223e5c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c0dd176e80 con 0x55c0dd257df0 2024-04-02T13:55:47.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.282+0000 7fd223e5c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c0dd267ff0 con 0x55c0dd2650b0 2024-04-02T13:55:47.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.282+0000 7fd223e5c740 10 client.0 _ll_register_callbacks cb 0x55c0dd1b2aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:47.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.282+0000 7fd21affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd2574d0 0x55c0dd264b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.282+0000 7fd21affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd2574d0 0x55c0dd264b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33952/0 (socket says 172.21.15.52:33952) 2024-04-02T13:55:47.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.282+0000 7fd21affd640 1 -- 172.21.15.52:0/571902131 learned_addr learned my addr 172.21.15.52:0/571902131 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:47.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.282+0000 7fd21a7fc640 1 --2- 172.21.15.52:0/571902131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c0dd257df0 0x55c0dd261ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.282+0000 7fd21b7fe640 1 --2- 172.21.15.52:0/571902131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0dd2650b0 0x55c0dd262600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.282+0000 7fd21affd640 1 -- 172.21.15.52:0/571902131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0dd2650b0 msgr2=0x55c0dd262600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.282+0000 7fd21affd640 1 --2- 172.21.15.52:0/571902131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0dd2650b0 0x55c0dd262600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.282+0000 7fd21affd640 1 -- 172.21.15.52:0/571902131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c0dd257df0 msgr2=0x55c0dd261ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.282+0000 7fd21affd640 1 --2- 172.21.15.52:0/571902131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c0dd257df0 0x55c0dd261ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.282+0000 7fd21affd640 1 -- 172.21.15.52:0/571902131 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0dcf997d0 con 0x55c0dd2574d0 2024-04-02T13:55:47.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.282+0000 7fd21a7fc640 1 --2- 172.21.15.52:0/571902131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c0dd257df0 0x55c0dd261ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:47.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.282+0000 7fd21affd640 1 --2- 172.21.15.52:0/571902131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd2574d0 0x55c0dd264b30 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd20c009be0 tx=0x7fd20c012700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a54767d3545b30 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.282+0000 7fd1feff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:47.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.282+0000 7fd1feff5640 1 -- 172.21.15.52:0/571902131 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c029020 con 0x55c0dd2574d0 2024-04-02T13:55:47.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.282+0000 7fd1feff5640 1 -- 172.21.15.52:0/571902131 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd20c00fab0 con 0x55c0dd2574d0 2024-04-02T13:55:47.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.282+0000 7fd223e5c740 1 -- 172.21.15.52:0/571902131 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd20c012ad0 con 0x55c0dd2574d0 2024-04-02T13:55:47.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.283+0000 7fd1feff5640 1 -- 172.21.15.52:0/571902131 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd20c034050 con 0x55c0dd2574d0 2024-04-02T13:55:47.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.283+0000 7fd223e5c740 10 client.16314 fetch_fsmap learned FSMap version 29 2024-04-02T13:55:47.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.283+0000 7fd223e5c740 1 -- 172.21.15.52:0/571902131 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c0dd273f40 con 0x55c0dd2574d0 2024-04-02T13:55:47.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.283+0000 7fd1feff5640 1 -- 172.21.15.52:0/571902131 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd20c012ad0 con 0x55c0dd2574d0 2024-04-02T13:55:47.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.283+0000 7fd1feff5640 1 -- 172.21.15.52:0/571902131 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fd20c0046c0 con 0x55c0dd2574d0 2024-04-02T13:55:47.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.283+0000 7fd223e5c740 10 client.16314 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:55:47.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.283+0000 7fd223e5c740 10 client.16314 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T13:55:47.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.283+0000 7fd223e5c740 20 client.16314 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:47.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.283+0000 7fd223e5c740 1 --2- 172.21.15.52:0/571902131 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55c0dd275400 0x55c0dd2777f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.283+0000 7fd223e5c740 4 client.16314 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:47.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.283+0000 7fd223e5c740 1 -- 172.21.15.52:0/571902131 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0dd1c8cf0 con 0x55c0dd275400 2024-04-02T13:55:47.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.283+0000 7fd21a7fc640 1 --2- 172.21.15.52:0/571902131 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55c0dd275400 0x55c0dd2777f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.284+0000 7fd21a7fc640 1 --2- 172.21.15.52:0/571902131 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55c0dd275400 0x55c0dd2777f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=259bf36faca7169b server_cookie=6c2b5f0b762e1306 in_seq=0 out_seq=0 2024-04-02T13:55:47.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.284+0000 7fd1feff5640 10 client.16314 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:55:47.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.284+0000 7fd1feff5640 1 -- 172.21.15.52:0/571902131 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c0dd1c8cf0 con 0x55c0dd275400 2024-04-02T13:55:47.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.284+0000 7fd1feff5640 10 client.16314 handle_command_reply: tid=0 2024-04-02T13:55:47.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.284+0000 7fd223e5c740 1 client.16314 shutdown 2024-04-02T13:55:47.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.285+0000 7fd223e5c740 1 -- 172.21.15.52:0/571902131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd2574d0 msgr2=0x55c0dd264b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.285+0000 7fd223e5c740 1 --2- 172.21.15.52:0/571902131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd2574d0 0x55c0dd264b30 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd20c009be0 tx=0x7fd20c012700 comp rx=0 tx=0).stop 2024-04-02T13:55:47.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.285+0000 7fd223e5c740 1 -- 172.21.15.52:0/571902131 shutdown_connections 2024-04-02T13:55:47.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.285+0000 7fd223e5c740 1 --2- 172.21.15.52:0/571902131 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55c0dd275400 0x55c0dd2777f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.285+0000 7fd223e5c740 1 --2- 172.21.15.52:0/571902131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0dd2650b0 0x55c0dd262600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.285+0000 7fd223e5c740 1 --2- 172.21.15.52:0/571902131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c0dd257df0 0x55c0dd261ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.285+0000 7fd223e5c740 1 --2- 172.21.15.52:0/571902131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd2574d0 0x55c0dd264b30 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.285+0000 7fd1feff5640 0 client.16314 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:55:47.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.285+0000 7fd223e5c740 1 -- 172.21.15.52:0/571902131 >> 172.21.15.52:0/571902131 conn(0x55c0dd251ed0 msgr2=0x55c0dd253c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.285+0000 7fd223e5c740 1 -- 172.21.15.52:0/571902131 shutdown_connections 2024-04-02T13:55:47.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.285+0000 7fd223e5c740 1 -- 172.21.15.52:0/571902131 wait complete. 2024-04-02T13:55:47.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.285+0000 7fd223e5c740 20 client.16314 trim_cache size 0 max 16384 2024-04-02T13:55:47.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.298+0000 7fd223e5c740 1 Processor -- start 2024-04-02T13:55:47.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.298+0000 7fd223e5c740 1 -- start start 2024-04-02T13:55:47.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.298+0000 7fd223e5c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd2574d0 0x55c0dd270e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.298+0000 7fd223e5c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0dd257df0 0x55c0dd2645f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.298+0000 7fd223e5c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c0dd2650b0 0x55c0dd264b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.298+0000 7fd223e5c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c1acc70 con 0x55c0dd2574d0 2024-04-02T13:55:47.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.298+0000 7fd223e5c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c1ae7e0 con 0x55c0dd2650b0 2024-04-02T13:55:47.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.298+0000 7fd223e5c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c1ad340 con 0x55c0dd257df0 2024-04-02T13:55:47.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.299+0000 7fd21affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd2574d0 0x55c0dd270e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.299+0000 7fd21affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd2574d0 0x55c0dd270e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33962/0 (socket says 172.21.15.52:33962) 2024-04-02T13:55:47.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.299+0000 7fd21affd640 1 -- 172.21.15.52:0/3487945664 learned_addr learned my addr 172.21.15.52:0/3487945664 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:47.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.299+0000 7fd21b7fe640 1 --2- 172.21.15.52:0/3487945664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c0dd2650b0 0x55c0dd264b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.299+0000 7fd21a7fc640 1 --2- 172.21.15.52:0/3487945664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0dd257df0 0x55c0dd2645f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.299+0000 7fd21affd640 1 -- 172.21.15.52:0/3487945664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0dd257df0 msgr2=0x55c0dd2645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.299+0000 7fd21affd640 1 --2- 172.21.15.52:0/3487945664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0dd257df0 0x55c0dd2645f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.299+0000 7fd21affd640 1 -- 172.21.15.52:0/3487945664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c0dd2650b0 msgr2=0x55c0dd264b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.299+0000 7fd21affd640 1 --2- 172.21.15.52:0/3487945664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c0dd2650b0 0x55c0dd264b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.299+0000 7fd21affd640 1 -- 172.21.15.52:0/3487945664 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0dd262970 con 0x55c0dd2574d0 2024-04-02T13:55:47.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.299+0000 7fd21b7fe640 1 --2- 172.21.15.52:0/3487945664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c0dd2650b0 0x55c0dd264b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:47.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.299+0000 7fd21affd640 1 --2- 172.21.15.52:0/3487945664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd2574d0 0x55c0dd270e00 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00a010 tx=0x7fd20c011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b713c2840863623 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.299+0000 7fd200ff9640 1 -- 172.21.15.52:0/3487945664 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c029040 con 0x55c0dd2574d0 2024-04-02T13:55:47.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.299+0000 7fd200ff9640 1 -- 172.21.15.52:0/3487945664 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd20c012770 con 0x55c0dd2574d0 2024-04-02T13:55:47.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.299+0000 7fd200ff9640 1 -- 172.21.15.52:0/3487945664 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c00f4c0 con 0x55c0dd2574d0 2024-04-02T13:55:47.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.300+0000 7fd223e5c740 1 -- 172.21.15.52:0/3487945664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd2574d0 msgr2=0x55c0dd270e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.300+0000 7fd223e5c740 1 --2- 172.21.15.52:0/3487945664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd2574d0 0x55c0dd270e00 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00a010 tx=0x7fd20c011de0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.300+0000 7fd223e5c740 1 -- 172.21.15.52:0/3487945664 shutdown_connections 2024-04-02T13:55:47.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.300+0000 7fd223e5c740 1 --2- 172.21.15.52:0/3487945664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c0dd2650b0 0x55c0dd264b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.300+0000 7fd223e5c740 1 --2- 172.21.15.52:0/3487945664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0dd257df0 0x55c0dd2645f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.300+0000 7fd223e5c740 1 --2- 172.21.15.52:0/3487945664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd2574d0 0x55c0dd270e00 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.300+0000 7fd223e5c740 1 -- 172.21.15.52:0/3487945664 >> 172.21.15.52:0/3487945664 conn(0x55c0dd251ed0 msgr2=0x55c0dd2522b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.300+0000 7fd223e5c740 1 -- 172.21.15.52:0/3487945664 shutdown_connections 2024-04-02T13:55:47.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.300+0000 7fd223e5c740 1 -- 172.21.15.52:0/3487945664 wait complete. 2024-04-02T13:55:47.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.300+0000 7fd223e5c740 1 -- start start 2024-04-02T13:55:47.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.300+0000 7fd223e5c740 1 Processor -- start 2024-04-02T13:55:47.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.301+0000 7fd223e5c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd2574d0 0x55c0dd25fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.301+0000 7fd223e5c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c0dd257df0 0x55c0dd25e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.301+0000 7fd223e5c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0dd2650b0 0x55c0dd25e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.301+0000 7fd223e5c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c1ae0e0 con 0x55c0dd2574d0 2024-04-02T13:55:47.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.301+0000 7fd223e5c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c1ada10 con 0x55c0dd257df0 2024-04-02T13:55:47.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.301+0000 7fd223e5c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c1aac50 con 0x55c0dd2650b0 2024-04-02T13:55:47.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.301+0000 7fd223e5c740 10 client.0 _ll_register_callbacks cb 0x7fd20c009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:47.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.301+0000 7fd21affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd2574d0 0x55c0dd25fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.301+0000 7fd21affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd2574d0 0x55c0dd25fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33976/0 (socket says 172.21.15.52:33976) 2024-04-02T13:55:47.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.301+0000 7fd21affd640 1 -- 172.21.15.52:0/2101684580 learned_addr learned my addr 172.21.15.52:0/2101684580 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:47.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.301+0000 7fd21a7fc640 1 --2- 172.21.15.52:0/2101684580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c0dd257df0 0x55c0dd25e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.301+0000 7fd21b7fe640 1 --2- 172.21.15.52:0/2101684580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0dd2650b0 0x55c0dd25e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.301+0000 7fd21affd640 1 -- 172.21.15.52:0/2101684580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0dd2650b0 msgr2=0x55c0dd25e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.301+0000 7fd21affd640 1 --2- 172.21.15.52:0/2101684580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0dd2650b0 0x55c0dd25e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.301+0000 7fd21affd640 1 -- 172.21.15.52:0/2101684580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c0dd257df0 msgr2=0x55c0dd25e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.301+0000 7fd21affd640 1 --2- 172.21.15.52:0/2101684580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c0dd257df0 0x55c0dd25e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.301+0000 7fd21affd640 1 -- 172.21.15.52:0/2101684580 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0dd26c310 con 0x55c0dd2574d0 2024-04-02T13:55:47.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.301+0000 7fd21b7fe640 1 --2- 172.21.15.52:0/2101684580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0dd2650b0 0x55c0dd25e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:47.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.301+0000 7fd21affd640 1 --2- 172.21.15.52:0/2101684580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd2574d0 0x55c0dd25fb60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd20c004b60 tx=0x7fd20c009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74149c641a25dfc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.301+0000 7fd1feff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:47.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.301+0000 7fd1feff5640 1 -- 172.21.15.52:0/2101684580 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c029040 con 0x55c0dd2574d0 2024-04-02T13:55:47.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.302+0000 7fd1feff5640 1 -- 172.21.15.52:0/2101684580 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd20c00fab0 con 0x55c0dd2574d0 2024-04-02T13:55:47.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.302+0000 7fd223e5c740 1 -- 172.21.15.52:0/2101684580 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd20c012770 con 0x55c0dd2574d0 2024-04-02T13:55:47.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.302+0000 7fd1feff5640 1 -- 172.21.15.52:0/2101684580 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd20c04f050 con 0x55c0dd2574d0 2024-04-02T13:55:47.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.302+0000 7fd223e5c740 10 client.16320 fetch_fsmap learned FSMap version 29 2024-04-02T13:55:47.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.302+0000 7fd223e5c740 1 -- 172.21.15.52:0/2101684580 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c0dd26d580 con 0x55c0dd2574d0 2024-04-02T13:55:47.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.302+0000 7fd1feff5640 1 -- 172.21.15.52:0/2101684580 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd20c012770 con 0x55c0dd2574d0 2024-04-02T13:55:47.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.302+0000 7fd1feff5640 1 -- 172.21.15.52:0/2101684580 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fd20c004040 con 0x55c0dd2574d0 2024-04-02T13:55:47.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.302+0000 7fd223e5c740 10 client.16320 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:55:47.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.302+0000 7fd223e5c740 10 client.16320 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T13:55:47.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.302+0000 7fd223e5c740 20 client.16320 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:47.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.302+0000 7fd223e5c740 1 --2- 172.21.15.52:0/2101684580 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55c0dd2db780 0x55c0dd2ddb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.302+0000 7fd223e5c740 4 client.16320 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:55:47.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.302+0000 7fd223e5c740 1 -- 172.21.15.52:0/2101684580 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0dd1c3040 con 0x55c0dd2db780 2024-04-02T13:55:47.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.302+0000 7fd21a7fc640 1 --2- 172.21.15.52:0/2101684580 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55c0dd2db780 0x55c0dd2ddb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.303+0000 7fd1feff5640 10 client.16320 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:55:47.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.303+0000 7fd21a7fc640 1 --2- 172.21.15.52:0/2101684580 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55c0dd2db780 0x55c0dd2ddb70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=afbaad9b52cf82e5 server_cookie=195eba3bceaee505 in_seq=0 out_seq=0 2024-04-02T13:55:47.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.303+0000 7fd1feff5640 1 -- 172.21.15.52:0/2101684580 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0dd1c3040 con 0x55c0dd2db780 2024-04-02T13:55:47.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.303+0000 7fd1feff5640 10 client.16320 handle_command_reply: tid=0 2024-04-02T13:55:47.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.303+0000 7fd223e5c740 1 client.16320 shutdown 2024-04-02T13:55:47.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.303+0000 7fd223e5c740 1 -- 172.21.15.52:0/2101684580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd2574d0 msgr2=0x55c0dd25fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.303+0000 7fd223e5c740 1 --2- 172.21.15.52:0/2101684580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd2574d0 0x55c0dd25fb60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd20c004b60 tx=0x7fd20c009530 comp rx=0 tx=0).stop 2024-04-02T13:55:47.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.303+0000 7fd223e5c740 1 -- 172.21.15.52:0/2101684580 shutdown_connections 2024-04-02T13:55:47.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.303+0000 7fd223e5c740 1 --2- 172.21.15.52:0/2101684580 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55c0dd2db780 0x55c0dd2ddb70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.303+0000 7fd223e5c740 1 --2- 172.21.15.52:0/2101684580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0dd2650b0 0x55c0dd25e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.303+0000 7fd223e5c740 1 --2- 172.21.15.52:0/2101684580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c0dd257df0 0x55c0dd25e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.303+0000 7fd223e5c740 1 --2- 172.21.15.52:0/2101684580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0dd2574d0 0x55c0dd25fb60 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.303+0000 7fd223e5c740 1 -- 172.21.15.52:0/2101684580 >> 172.21.15.52:0/2101684580 conn(0x55c0dd251ed0 msgr2=0x55c0dd260660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.303+0000 7fd1feff5640 0 client.16320 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:55:47.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.303+0000 7fd223e5c740 1 -- 172.21.15.52:0/2101684580 shutdown_connections 2024-04-02T13:55:47.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.303+0000 7fd223e5c740 1 -- 172.21.15.52:0/2101684580 wait complete. 2024-04-02T13:55:47.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.303+0000 7fd223e5c740 20 client.16320 trim_cache size 0 max 16384 2024-04-02T13:55:47.313 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:55:47.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.304+0000 7fd221277640 1 -- 172.21.15.52:0/838248364 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd2040a3d20 msgr2=0x7fd2040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.304+0000 7fd221277640 1 --2- 172.21.15.52:0/838248364 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd2040a3d20 0x7fd2040a61e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd214006010 tx=0x7fd21402e040 comp rx=0 tx=0).stop 2024-04-02T13:55:47.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.304+0000 7fd221277640 1 -- 172.21.15.52:0/838248364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd21c102450 msgr2=0x7fd21c1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.304+0000 7fd221277640 1 --2- 172.21.15.52:0/838248364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd21c102450 0x7fd21c1988f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd21000e9f0 tx=0x7fd21000eeb0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.304+0000 7fd221277640 1 -- 172.21.15.52:0/838248364 shutdown_connections 2024-04-02T13:55:47.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.304+0000 7fd221277640 1 --2- 172.21.15.52:0/838248364 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd2040a3d20 0x7fd2040a61e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.304+0000 7fd221277640 1 --2- 172.21.15.52:0/838248364 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd21c107aa0 0x7fd21c19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.304+0000 7fd221277640 1 --2- 172.21.15.52:0/838248364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd21c102450 0x7fd21c1988f0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.304+0000 7fd221277640 1 --2- 172.21.15.52:0/838248364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd21c101a90 0x7fd21c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.304+0000 7fd221277640 1 -- 172.21.15.52:0/838248364 >> 172.21.15.52:0/838248364 conn(0x7fd21c078f00 msgr2=0x7fd21c1043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.304+0000 7fd221277640 1 -- 172.21.15.52:0/838248364 shutdown_connections 2024-04-02T13:55:47.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.304+0000 7fd221277640 1 -- 172.21.15.52:0/838248364 wait complete. 2024-04-02T13:55:47.318 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:55:47.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.461+0000 7fea62160640 1 -- 172.21.15.52:0/3119246870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea5c107a80 msgr2=0x7fea5c107e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.461+0000 7fea62160640 1 --2- 172.21.15.52:0/3119246870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea5c107a80 0x7fea5c107e60 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009930 tx=0x7fea4c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:55:47.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.461+0000 7fea62160640 1 -- 172.21.15.52:0/3119246870 shutdown_connections 2024-04-02T13:55:47.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.461+0000 7fea62160640 1 --2- 172.21.15.52:0/3119246870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea5c0696a0 0x7fea5c10e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.461+0000 7fea62160640 1 --2- 172.21.15.52:0/3119246870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea5c068ce0 0x7fea5c069160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.461+0000 7fea62160640 1 --2- 172.21.15.52:0/3119246870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea5c107a80 0x7fea5c107e60 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.461+0000 7fea62160640 1 -- 172.21.15.52:0/3119246870 >> 172.21.15.52:0/3119246870 conn(0x7fea5c077b00 msgr2=0x7fea5c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.461+0000 7fea62160640 1 -- 172.21.15.52:0/3119246870 shutdown_connections 2024-04-02T13:55:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.461+0000 7fea62160640 1 -- 172.21.15.52:0/3119246870 wait complete. 2024-04-02T13:55:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.461+0000 7fea62160640 1 Processor -- start 2024-04-02T13:55:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.461+0000 7fea62160640 1 -- start start 2024-04-02T13:55:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.462+0000 7fea62160640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea5c068ce0 0x7fea5c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.462+0000 7fea62160640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea5c0696a0 0x7fea5c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.462+0000 7fea62160640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea5c107a80 0x7fea5c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.462+0000 7fea62160640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c111e90 con 0x7fea5c107a80 2024-04-02T13:55:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.462+0000 7fea62160640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c111d10 con 0x7fea5c0696a0 2024-04-02T13:55:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.462+0000 7fea62160640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c112010 con 0x7fea5c068ce0 2024-04-02T13:55:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.462+0000 7fea5bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea5c107a80 0x7fea5c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.462+0000 7fea5bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea5c107a80 0x7fea5c19cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34004/0 (socket says 172.21.15.52:34004) 2024-04-02T13:55:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.462+0000 7fea5bfff640 1 -- 172.21.15.52:0/1933895748 learned_addr learned my addr 172.21.15.52:0/1933895748 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.462+0000 7fea5b7fe640 1 --2- 172.21.15.52:0/1933895748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea5c068ce0 0x7fea5c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.462+0000 7fea5affd640 1 --2- 172.21.15.52:0/1933895748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea5c0696a0 0x7fea5c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.462+0000 7fea5bfff640 1 -- 172.21.15.52:0/1933895748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea5c068ce0 msgr2=0x7fea5c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.462+0000 7fea5bfff640 1 --2- 172.21.15.52:0/1933895748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea5c068ce0 0x7fea5c198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.462+0000 7fea5bfff640 1 -- 172.21.15.52:0/1933895748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea5c0696a0 msgr2=0x7fea5c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.462+0000 7fea5bfff640 1 --2- 172.21.15.52:0/1933895748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea5c0696a0 0x7fea5c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.462+0000 7fea5bfff640 1 -- 172.21.15.52:0/1933895748 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5c19d380 con 0x7fea5c107a80 2024-04-02T13:55:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.462+0000 7fea5affd640 1 --2- 172.21.15.52:0/1933895748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea5c0696a0 0x7fea5c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.462+0000 7fea5bfff640 1 --2- 172.21.15.52:0/1933895748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea5c107a80 0x7fea5c19cc50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fea5400eb10 tx=0x7fea5400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.462+0000 7fea58ff9640 1 -- 172.21.15.52:0/1933895748 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54004b20 con 0x7fea5c107a80 2024-04-02T13:55:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.462+0000 7fea58ff9640 1 -- 172.21.15.52:0/1933895748 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea54004cc0 con 0x7fea5c107a80 2024-04-02T13:55:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.462+0000 7fea62160640 1 -- 172.21.15.52:0/1933895748 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5c19d670 con 0x7fea5c107a80 2024-04-02T13:55:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.462+0000 7fea58ff9640 1 -- 172.21.15.52:0/1933895748 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54010760 con 0x7fea5c107a80 2024-04-02T13:55:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.462+0000 7fea62160640 1 -- 172.21.15.52:0/1933895748 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea5c1a4f00 con 0x7fea5c107a80 2024-04-02T13:55:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.465+0000 7fea58ff9640 1 -- 172.21.15.52:0/1933895748 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fea54005020 con 0x7fea5c107a80 2024-04-02T13:55:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.465+0000 7fea64e27740 1 Processor -- start 2024-04-02T13:55:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.465+0000 7fea64e27740 1 -- start start 2024-04-02T13:55:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.465+0000 7fea64e27740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557868d047a0 0x557868d04b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.465+0000 7fea64e27740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d050c0 0x557868d0d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.465+0000 7fea64e27740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557868d0dca0 0x557868d10090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.465+0000 7fea64e27740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557868c687b0 con 0x557868d050c0 2024-04-02T13:55:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.465+0000 7fea64e27740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557868c07570 con 0x557868d047a0 2024-04-02T13:55:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.465+0000 7fea64e27740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557868c64760 con 0x557868d0dca0 2024-04-02T13:55:47.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.465+0000 7fea5b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d050c0 0x557868d0d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.465+0000 7fea5b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d050c0 0x557868d0d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34008/0 (socket says 172.21.15.52:34008) 2024-04-02T13:55:47.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.465+0000 7fea5b7fe640 1 -- 172.21.15.52:0/1822400237 learned_addr learned my addr 172.21.15.52:0/1822400237 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:47.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.465+0000 7fea58ff9640 1 --2- 172.21.15.52:0/1933895748 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fea440a3d20 0x7fea440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.466+0000 7fea5affd640 1 --2- 172.21.15.52:0/1822400237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557868d047a0 0x557868d04b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.466+0000 7fea5affd640 1 --2- 172.21.15.52:0/1822400237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557868d0dca0 0x557868d10090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.466+0000 7fea5b7fe640 1 -- 172.21.15.52:0/1822400237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557868d0dca0 msgr2=0x557868d10090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.466+0000 7fea5bfff640 1 --2- 172.21.15.52:0/1933895748 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fea440a3d20 0x7fea440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.466+0000 7fea5b7fe640 1 --2- 172.21.15.52:0/1822400237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557868d0dca0 0x557868d10090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.466+0000 7fea5b7fe640 1 -- 172.21.15.52:0/1822400237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557868d047a0 msgr2=0x557868d04b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.466+0000 7fea5b7fe640 1 --2- 172.21.15.52:0/1822400237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557868d047a0 0x557868d04b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.466+0000 7fea5b7fe640 1 -- 172.21.15.52:0/1822400237 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55786897c3f0 con 0x557868d050c0 2024-04-02T13:55:47.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.466+0000 7fea5affd640 1 --2- 172.21.15.52:0/1822400237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557868d047a0 0x557868d04b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:47.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.466+0000 7fea58ff9640 1 -- 172.21.15.52:0/1933895748 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fea540d1220 con 0x7fea5c107a80 2024-04-02T13:55:47.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.466+0000 7fea5b7fe640 1 --2- 172.21.15.52:0/1822400237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d050c0 0x557868d0d760 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fea4c02cfd0 tx=0x7fea4c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0627252bcfddb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.466+0000 7fea41ffb640 1 -- 172.21.15.52:0/1822400237 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c042070 con 0x557868d050c0 2024-04-02T13:55:47.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.466+0000 7fea5bfff640 1 --2- 172.21.15.52:0/1933895748 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fea440a3d20 0x7fea440a61e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fea540992e0 tx=0x7fea5406f970 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.466+0000 7fea41ffb640 1 -- 172.21.15.52:0/1822400237 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea4c004030 con 0x557868d050c0 2024-04-02T13:55:47.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.466+0000 7fea41ffb640 1 -- 172.21.15.52:0/1822400237 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c004f50 con 0x557868d050c0 2024-04-02T13:55:47.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.467+0000 7fea64e27740 1 -- 172.21.15.52:0/1822400237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d050c0 msgr2=0x557868d0d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.467+0000 7fea64e27740 1 --2- 172.21.15.52:0/1822400237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d050c0 0x557868d0d760 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fea4c02cfd0 tx=0x7fea4c005d50 comp rx=0 tx=0).stop 2024-04-02T13:55:47.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.467+0000 7fea64e27740 1 -- 172.21.15.52:0/1822400237 shutdown_connections 2024-04-02T13:55:47.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.467+0000 7fea64e27740 1 --2- 172.21.15.52:0/1822400237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557868d0dca0 0x557868d10090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.467+0000 7fea64e27740 1 --2- 172.21.15.52:0/1822400237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d050c0 0x557868d0d760 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.467+0000 7fea64e27740 1 --2- 172.21.15.52:0/1822400237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557868d047a0 0x557868d04b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.467+0000 7fea64e27740 1 -- 172.21.15.52:0/1822400237 >> 172.21.15.52:0/1822400237 conn(0x557868cfeed0 msgr2=0x557868cff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.467+0000 7fea64e27740 1 -- 172.21.15.52:0/1822400237 shutdown_connections 2024-04-02T13:55:47.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.467+0000 7fea64e27740 1 -- 172.21.15.52:0/1822400237 wait complete. 2024-04-02T13:55:47.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.467+0000 7fea64e27740 1 -- start start 2024-04-02T13:55:47.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.467+0000 7fea64e27740 1 Processor -- start 2024-04-02T13:55:47.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.468+0000 7fea64e27740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d047a0 0x557868d0d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.468+0000 7fea64e27740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557868d050c0 0x557868d0aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.468+0000 7fea64e27740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557868d0dca0 0x557868d0b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.468+0000 7fea64e27740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557868d10a30 con 0x557868d047a0 2024-04-02T13:55:47.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.468+0000 7fea64e27740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557868c23e80 con 0x557868d050c0 2024-04-02T13:55:47.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.468+0000 7fea64e27740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557868d10bb0 con 0x557868d0dca0 2024-04-02T13:55:47.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.468+0000 7fea64e27740 10 client.0 _ll_register_callbacks cb 0x557868c5faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:47.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.468+0000 7fea5affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d047a0 0x557868d0d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.468+0000 7fea5affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d047a0 0x557868d0d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34016/0 (socket says 172.21.15.52:34016) 2024-04-02T13:55:47.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.468+0000 7fea5affd640 1 -- 172.21.15.52:0/1434981660 learned_addr learned my addr 172.21.15.52:0/1434981660 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:47.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.468+0000 7fea5b7fe640 1 --2- 172.21.15.52:0/1434981660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557868d050c0 0x557868d0aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.468+0000 7fea5affd640 1 --2- 172.21.15.52:0/1434981660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557868d0dca0 0x557868d0b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.468+0000 7fea5affd640 1 -- 172.21.15.52:0/1434981660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557868d0dca0 msgr2=0x557868d0b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.468+0000 7fea5affd640 1 --2- 172.21.15.52:0/1434981660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557868d0dca0 0x557868d0b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.468+0000 7fea5affd640 1 -- 172.21.15.52:0/1434981660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557868d050c0 msgr2=0x557868d0aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.468+0000 7fea5affd640 1 --2- 172.21.15.52:0/1434981660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557868d050c0 0x557868d0aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.468+0000 7fea5affd640 1 -- 172.21.15.52:0/1434981660 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557868a467d0 con 0x557868d047a0 2024-04-02T13:55:47.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.468+0000 7fea5b7fe640 1 --2- 172.21.15.52:0/1434981660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557868d050c0 0x557868d0aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:47.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.468+0000 7fea5affd640 1 --2- 172.21.15.52:0/1434981660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d047a0 0x557868d0d6b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fea5000bf00 tx=0x7fea50015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917ae665f16f2c41 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.468+0000 7fea3affd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:47.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.468+0000 7fea3affd640 1 -- 172.21.15.52:0/1434981660 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50005020 con 0x557868d047a0 2024-04-02T13:55:47.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.468+0000 7fea3affd640 1 -- 172.21.15.52:0/1434981660 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fea500051c0 con 0x557868d047a0 2024-04-02T13:55:47.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.468+0000 7fea64e27740 1 -- 172.21.15.52:0/1434981660 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea4c004030 con 0x557868d047a0 2024-04-02T13:55:47.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.469+0000 7fea3affd640 1 -- 172.21.15.52:0/1434981660 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fea50027050 con 0x557868d047a0 2024-04-02T13:55:47.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.469+0000 7fea64e27740 10 client.16332 fetch_fsmap learned FSMap version 29 2024-04-02T13:55:47.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.469+0000 7fea64e27740 1 -- 172.21.15.52:0/1434981660 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557868d1cb20 con 0x557868d047a0 2024-04-02T13:55:47.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.469+0000 7fea3affd640 1 -- 172.21.15.52:0/1434981660 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea4c004030 con 0x557868d047a0 2024-04-02T13:55:47.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.469+0000 7fea3affd640 1 -- 172.21.15.52:0/1434981660 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fea5000f040 con 0x557868d047a0 2024-04-02T13:55:47.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.469+0000 7fea64e27740 10 client.16332 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:55:47.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.469+0000 7fea64e27740 10 client.16332 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:55:47.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.469+0000 7fea64e27740 20 client.16332 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:47.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.469+0000 7fea64e27740 1 --2- 172.21.15.52:0/1434981660 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x557868d1df30 0x557868d20360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.469+0000 7fea64e27740 4 client.16332 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:47.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.469+0000 7fea64e27740 1 -- 172.21.15.52:0/1434981660 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557868c75cf0 con 0x557868d1df30 2024-04-02T13:55:47.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.469+0000 7fea5b7fe640 1 --2- 172.21.15.52:0/1434981660 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x557868d1df30 0x557868d20360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.470+0000 7fea5b7fe640 1 --2- 172.21.15.52:0/1434981660 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x557868d1df30 0x557868d20360 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=158ac82c3e8e841 server_cookie=c6199f665ffca12c in_seq=0 out_seq=0 2024-04-02T13:55:47.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.470+0000 7fea3affd640 10 client.16332 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:55:47.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.471+0000 7fea3affd640 1 -- 172.21.15.52:0/1434981660 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557868c75cf0 con 0x557868d1df30 2024-04-02T13:55:47.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.471+0000 7fea3affd640 10 client.16332 handle_command_reply: tid=0 2024-04-02T13:55:47.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.471+0000 7fea64e27740 1 client.16332 shutdown 2024-04-02T13:55:47.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.471+0000 7fea64e27740 1 -- 172.21.15.52:0/1434981660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d047a0 msgr2=0x557868d0d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.471+0000 7fea64e27740 1 --2- 172.21.15.52:0/1434981660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d047a0 0x557868d0d6b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fea5000bf00 tx=0x7fea50015040 comp rx=0 tx=0).stop 2024-04-02T13:55:47.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.471+0000 7fea64e27740 1 -- 172.21.15.52:0/1434981660 shutdown_connections 2024-04-02T13:55:47.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.471+0000 7fea64e27740 1 --2- 172.21.15.52:0/1434981660 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x557868d1df30 0x557868d20360 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.471+0000 7fea64e27740 1 --2- 172.21.15.52:0/1434981660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557868d0dca0 0x557868d0b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.471+0000 7fea64e27740 1 --2- 172.21.15.52:0/1434981660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557868d050c0 0x557868d0aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.471+0000 7fea3affd640 0 client.16332 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:55:47.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.471+0000 7fea64e27740 1 --2- 172.21.15.52:0/1434981660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d047a0 0x557868d0d6b0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.471+0000 7fea64e27740 1 -- 172.21.15.52:0/1434981660 >> 172.21.15.52:0/1434981660 conn(0x557868cfeed0 msgr2=0x557868d00c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.471+0000 7fea64e27740 1 -- 172.21.15.52:0/1434981660 shutdown_connections 2024-04-02T13:55:47.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.471+0000 7fea64e27740 1 -- 172.21.15.52:0/1434981660 wait complete. 2024-04-02T13:55:47.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.471+0000 7fea64e27740 20 client.16332 trim_cache size 0 max 16384 2024-04-02T13:55:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.491+0000 7fea64e27740 1 Processor -- start 2024-04-02T13:55:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.491+0000 7fea64e27740 1 -- start start 2024-04-02T13:55:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.491+0000 7fea64e27740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557868d047a0 0x557868d19950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.491+0000 7fea64e27740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d050c0 0x557868d0d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.492+0000 7fea64e27740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557868d0dca0 0x557868d0d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.492+0000 7fea64e27740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c1aee20 con 0x557868d050c0 2024-04-02T13:55:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.492+0000 7fea64e27740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c1b0990 con 0x557868d047a0 2024-04-02T13:55:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.492+0000 7fea64e27740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c1af4f0 con 0x557868d0dca0 2024-04-02T13:55:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.492+0000 7fea5b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d050c0 0x557868d0d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.492+0000 7fea5b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d050c0 0x557868d0d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34026/0 (socket says 172.21.15.52:34026) 2024-04-02T13:55:47.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.492+0000 7fea5b7fe640 1 -- 172.21.15.52:0/4278239271 learned_addr learned my addr 172.21.15.52:0/4278239271 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:47.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.492+0000 7fea5affd640 1 --2- 172.21.15.52:0/4278239271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557868d047a0 0x557868d19950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.492+0000 7fea5affd640 1 --2- 172.21.15.52:0/4278239271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557868d0dca0 0x557868d0d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.492+0000 7fea5b7fe640 1 -- 172.21.15.52:0/4278239271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557868d0dca0 msgr2=0x557868d0d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.492+0000 7fea5b7fe640 1 --2- 172.21.15.52:0/4278239271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557868d0dca0 0x557868d0d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.492+0000 7fea5b7fe640 1 -- 172.21.15.52:0/4278239271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557868d047a0 msgr2=0x557868d19950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.492+0000 7fea5b7fe640 1 --2- 172.21.15.52:0/4278239271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557868d047a0 0x557868d19950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.492+0000 7fea5b7fe640 1 -- 172.21.15.52:0/4278239271 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557868d04ef0 con 0x557868d050c0 2024-04-02T13:55:47.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.492+0000 7fea5affd640 1 --2- 172.21.15.52:0/4278239271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557868d047a0 0x557868d19950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:47.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.492+0000 7fea5b7fe640 1 --2- 172.21.15.52:0/4278239271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d050c0 0x557868d0d170 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x557868d04270 tx=0x7fea4c048040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8bcc844813ad0f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.492+0000 7fea417fa640 1 -- 172.21.15.52:0/4278239271 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c042070 con 0x557868d050c0 2024-04-02T13:55:47.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.492+0000 7fea417fa640 1 -- 172.21.15.52:0/4278239271 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea4c04f420 con 0x557868d050c0 2024-04-02T13:55:47.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.492+0000 7fea417fa640 1 -- 172.21.15.52:0/4278239271 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c0051b0 con 0x557868d050c0 2024-04-02T13:55:47.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.493+0000 7fea64e27740 1 -- 172.21.15.52:0/4278239271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d050c0 msgr2=0x557868d0d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.493+0000 7fea64e27740 1 --2- 172.21.15.52:0/4278239271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d050c0 0x557868d0d170 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x557868d04270 tx=0x7fea4c048040 comp rx=0 tx=0).stop 2024-04-02T13:55:47.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.493+0000 7fea64e27740 1 -- 172.21.15.52:0/4278239271 shutdown_connections 2024-04-02T13:55:47.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.493+0000 7fea64e27740 1 --2- 172.21.15.52:0/4278239271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557868d0dca0 0x557868d0d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.493+0000 7fea64e27740 1 --2- 172.21.15.52:0/4278239271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d050c0 0x557868d0d170 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.493+0000 7fea64e27740 1 --2- 172.21.15.52:0/4278239271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557868d047a0 0x557868d19950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.493+0000 7fea64e27740 1 -- 172.21.15.52:0/4278239271 >> 172.21.15.52:0/4278239271 conn(0x557868cfeed0 msgr2=0x557868d05d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.493+0000 7fea64e27740 1 -- 172.21.15.52:0/4278239271 shutdown_connections 2024-04-02T13:55:47.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.493+0000 7fea64e27740 1 -- 172.21.15.52:0/4278239271 wait complete. 2024-04-02T13:55:47.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.493+0000 7fea64e27740 1 -- start start 2024-04-02T13:55:47.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.493+0000 7fea64e27740 1 Processor -- start 2024-04-02T13:55:47.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.494+0000 7fea64e27740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d047a0 0x557868d16340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.494+0000 7fea64e27740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557868d050c0 0x557868d07d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.494+0000 7fea64e27740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557868d0dca0 0x557868d082a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.494+0000 7fea64e27740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c1b0290 con 0x557868d047a0 2024-04-02T13:55:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.494+0000 7fea64e27740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c1afbc0 con 0x557868d050c0 2024-04-02T13:55:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.494+0000 7fea64e27740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c1ace00 con 0x557868d0dca0 2024-04-02T13:55:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.494+0000 7fea64e27740 10 client.0 _ll_register_callbacks cb 0x7fea50013a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.494+0000 7fea5affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d047a0 0x557868d16340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.494+0000 7fea5affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d047a0 0x557868d16340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34042/0 (socket says 172.21.15.52:34042) 2024-04-02T13:55:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.494+0000 7fea5affd640 1 -- 172.21.15.52:0/3177718605 learned_addr learned my addr 172.21.15.52:0/3177718605 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.494+0000 7fea5b7fe640 1 --2- 172.21.15.52:0/3177718605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557868d050c0 0x557868d07d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.494+0000 7fea5affd640 1 --2- 172.21.15.52:0/3177718605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557868d0dca0 0x557868d082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.494+0000 7fea5affd640 1 -- 172.21.15.52:0/3177718605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557868d0dca0 msgr2=0x557868d082a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.494+0000 7fea5affd640 1 --2- 172.21.15.52:0/3177718605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557868d0dca0 0x557868d082a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.494+0000 7fea5affd640 1 -- 172.21.15.52:0/3177718605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557868d050c0 msgr2=0x557868d07d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.494+0000 7fea5affd640 1 --2- 172.21.15.52:0/3177718605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557868d050c0 0x557868d07d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.494+0000 7fea5affd640 1 -- 172.21.15.52:0/3177718605 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557868d08ad0 con 0x557868d047a0 2024-04-02T13:55:47.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.494+0000 7fea5affd640 1 --2- 172.21.15.52:0/3177718605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d047a0 0x557868d16340 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fea50013b20 tx=0x7fea50013800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e08d8637d49aeb03 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.494+0000 7fea3affd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.494+0000 7fea3affd640 1 -- 172.21.15.52:0/3177718605 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50032070 con 0x557868d047a0 2024-04-02T13:55:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.495+0000 7fea64e27740 1 -- 172.21.15.52:0/3177718605 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea4c04f420 con 0x557868d047a0 2024-04-02T13:55:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.495+0000 7fea3affd640 1 -- 172.21.15.52:0/3177718605 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fea5001fa10 con 0x557868d047a0 2024-04-02T13:55:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.495+0000 7fea3affd640 1 -- 172.21.15.52:0/3177718605 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fea5003d050 con 0x557868d047a0 2024-04-02T13:55:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.495+0000 7fea64e27740 10 client.16338 fetch_fsmap learned FSMap version 29 2024-04-02T13:55:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.495+0000 7fea64e27740 1 -- 172.21.15.52:0/3177718605 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557868d84560 con 0x557868d047a0 2024-04-02T13:55:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.495+0000 7fea3affd640 1 -- 172.21.15.52:0/3177718605 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea4c04f420 con 0x557868d047a0 2024-04-02T13:55:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.495+0000 7fea3affd640 1 -- 172.21.15.52:0/3177718605 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fea500038f0 con 0x557868d047a0 2024-04-02T13:55:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.495+0000 7fea64e27740 10 client.16338 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:55:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.495+0000 7fea64e27740 10 client.16338 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:55:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.495+0000 7fea64e27740 20 client.16338 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.495+0000 7fea64e27740 1 --2- 172.21.15.52:0/3177718605 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x557868d84c30 0x557868d87050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.495+0000 7fea64e27740 4 client.16338 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:55:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.495+0000 7fea64e27740 1 -- 172.21.15.52:0/3177718605 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557868c70040 con 0x557868d84c30 2024-04-02T13:55:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.495+0000 7fea5b7fe640 1 --2- 172.21.15.52:0/3177718605 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x557868d84c30 0x557868d87050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.496+0000 7fea5b7fe640 1 --2- 172.21.15.52:0/3177718605 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x557868d84c30 0x557868d87050 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=910702f7828de259 server_cookie=8e008163ce5df01c in_seq=0 out_seq=0 2024-04-02T13:55:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.496+0000 7fea3affd640 10 client.16338 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:55:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.496+0000 7fea3affd640 1 -- 172.21.15.52:0/3177718605 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557868c70040 con 0x557868d84c30 2024-04-02T13:55:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.496+0000 7fea3affd640 10 client.16338 handle_command_reply: tid=0 2024-04-02T13:55:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.496+0000 7fea64e27740 1 client.16338 shutdown 2024-04-02T13:55:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.496+0000 7fea64e27740 1 -- 172.21.15.52:0/3177718605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d047a0 msgr2=0x557868d16340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.496+0000 7fea64e27740 1 --2- 172.21.15.52:0/3177718605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d047a0 0x557868d16340 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fea50013b20 tx=0x7fea50013800 comp rx=0 tx=0).stop 2024-04-02T13:55:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.496+0000 7fea64e27740 1 -- 172.21.15.52:0/3177718605 shutdown_connections 2024-04-02T13:55:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.496+0000 7fea64e27740 1 --2- 172.21.15.52:0/3177718605 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x557868d84c30 0x557868d87050 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.496+0000 7fea64e27740 1 --2- 172.21.15.52:0/3177718605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557868d0dca0 0x557868d082a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.496+0000 7fea64e27740 1 --2- 172.21.15.52:0/3177718605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557868d050c0 0x557868d07d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.496+0000 7fea64e27740 1 --2- 172.21.15.52:0/3177718605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557868d047a0 0x557868d16340 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.496+0000 7fea3affd640 0 client.16338 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:55:47.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.496+0000 7fea64e27740 1 -- 172.21.15.52:0/3177718605 >> 172.21.15.52:0/3177718605 conn(0x557868cfeed0 msgr2=0x557868cfd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.496+0000 7fea64e27740 1 -- 172.21.15.52:0/3177718605 shutdown_connections 2024-04-02T13:55:47.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.496+0000 7fea64e27740 1 -- 172.21.15.52:0/3177718605 wait complete. 2024-04-02T13:55:47.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.496+0000 7fea64e27740 20 client.16338 trim_cache size 0 max 16384 2024-04-02T13:55:47.506 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:55:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.497+0000 7fea62160640 1 -- 172.21.15.52:0/1933895748 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fea440a3d20 msgr2=0x7fea440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.497+0000 7fea62160640 1 --2- 172.21.15.52:0/1933895748 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fea440a3d20 0x7fea440a61e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fea540992e0 tx=0x7fea5406f970 comp rx=0 tx=0).stop 2024-04-02T13:55:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.497+0000 7fea62160640 1 -- 172.21.15.52:0/1933895748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea5c107a80 msgr2=0x7fea5c19cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.497+0000 7fea62160640 1 --2- 172.21.15.52:0/1933895748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea5c107a80 0x7fea5c19cc50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fea5400eb10 tx=0x7fea5400c2d0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.497+0000 7fea62160640 1 -- 172.21.15.52:0/1933895748 shutdown_connections 2024-04-02T13:55:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.497+0000 7fea62160640 1 --2- 172.21.15.52:0/1933895748 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fea440a3d20 0x7fea440a61e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.497+0000 7fea62160640 1 --2- 172.21.15.52:0/1933895748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea5c107a80 0x7fea5c19cc50 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.497+0000 7fea62160640 1 --2- 172.21.15.52:0/1933895748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea5c0696a0 0x7fea5c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.497+0000 7fea62160640 1 --2- 172.21.15.52:0/1933895748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea5c068ce0 0x7fea5c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.497+0000 7fea62160640 1 -- 172.21.15.52:0/1933895748 >> 172.21.15.52:0/1933895748 conn(0x7fea5c077b00 msgr2=0x7fea5c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.497+0000 7fea62160640 1 -- 172.21.15.52:0/1933895748 shutdown_connections 2024-04-02T13:55:47.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.497+0000 7fea62160640 1 -- 172.21.15.52:0/1933895748 wait complete. 2024-04-02T13:55:47.509 INFO:tasks.workunit.client.0.smithi052.stdout:autoconf 2024-04-02T13:55:47.511 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:55:47.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.615+0000 7fb14a4d5640 1 -- 172.21.15.52:0/1422029442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb144068ce0 msgr2=0x7fb144069160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.615+0000 7fb14a4d5640 1 --2- 172.21.15.52:0/1422029442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb144068ce0 0x7fb144069160 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb12c009930 tx=0x7fb12c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:55:47.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.615+0000 7fb14a4d5640 1 -- 172.21.15.52:0/1422029442 shutdown_connections 2024-04-02T13:55:47.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.615+0000 7fb14a4d5640 1 --2- 172.21.15.52:0/1422029442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1440696a0 0x7fb14410e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.615+0000 7fb14a4d5640 1 --2- 172.21.15.52:0/1422029442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb144068ce0 0x7fb144069160 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.615+0000 7fb14a4d5640 1 --2- 172.21.15.52:0/1422029442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb144107a80 0x7fb144107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.615+0000 7fb14a4d5640 1 -- 172.21.15.52:0/1422029442 >> 172.21.15.52:0/1422029442 conn(0x7fb144077b00 msgr2=0x7fb144077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.615+0000 7fb14a4d5640 1 -- 172.21.15.52:0/1422029442 shutdown_connections 2024-04-02T13:55:47.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.615+0000 7fb14a4d5640 1 -- 172.21.15.52:0/1422029442 wait complete. 2024-04-02T13:55:47.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.616+0000 7fb14a4d5640 1 Processor -- start 2024-04-02T13:55:47.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.616+0000 7fb14a4d5640 1 -- start start 2024-04-02T13:55:47.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.616+0000 7fb14a4d5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb144068ce0 0x7fb1441983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.616+0000 7fb14a4d5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1440696a0 0x7fb144198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.616+0000 7fb14a4d5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb144107a80 0x7fb14419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.616+0000 7fb14a4d5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb144111e90 con 0x7fb144068ce0 2024-04-02T13:55:47.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.616+0000 7fb14a4d5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb144111d10 con 0x7fb144107a80 2024-04-02T13:55:47.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.616+0000 7fb14a4d5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb144112010 con 0x7fb1440696a0 2024-04-02T13:55:47.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.616+0000 7fb143fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb144068ce0 0x7fb1441983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.617+0000 7fb143fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb144068ce0 0x7fb1441983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34064/0 (socket says 172.21.15.52:34064) 2024-04-02T13:55:47.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.617+0000 7fb143fff640 1 -- 172.21.15.52:0/2393926306 learned_addr learned my addr 172.21.15.52:0/2393926306 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:47.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.617+0000 7fb148a72640 1 --2- 172.21.15.52:0/2393926306 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb144107a80 0x7fb14419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.617+0000 7fb1437fe640 1 --2- 172.21.15.52:0/2393926306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1440696a0 0x7fb144198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.617+0000 7fb143fff640 1 -- 172.21.15.52:0/2393926306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1440696a0 msgr2=0x7fb144198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.617+0000 7fb143fff640 1 --2- 172.21.15.52:0/2393926306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1440696a0 0x7fb144198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.617+0000 7fb143fff640 1 -- 172.21.15.52:0/2393926306 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb144107a80 msgr2=0x7fb14419cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.617+0000 7fb143fff640 1 --2- 172.21.15.52:0/2393926306 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb144107a80 0x7fb14419cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.617+0000 7fb143fff640 1 -- 172.21.15.52:0/2393926306 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14419d3c0 con 0x7fb144068ce0 2024-04-02T13:55:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.617+0000 7fb143fff640 1 --2- 172.21.15.52:0/2393926306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb144068ce0 0x7fb1441983d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb13400d950 tx=0x7fb13400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.617+0000 7fb1417fa640 1 -- 172.21.15.52:0/2393926306 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb134017070 con 0x7fb144068ce0 2024-04-02T13:55:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.617+0000 7fb14a4d5640 1 -- 172.21.15.52:0/2393926306 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb14419d6b0 con 0x7fb144068ce0 2024-04-02T13:55:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.617+0000 7fb1417fa640 1 -- 172.21.15.52:0/2393926306 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb134009c30 con 0x7fb144068ce0 2024-04-02T13:55:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.617+0000 7fb1417fa640 1 -- 172.21.15.52:0/2393926306 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13400f6b0 con 0x7fb144068ce0 2024-04-02T13:55:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.617+0000 7fb14a4d5640 1 -- 172.21.15.52:0/2393926306 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1441a4f40 con 0x7fb144068ce0 2024-04-02T13:55:47.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.620+0000 7fb14d18a740 1 Processor -- start 2024-04-02T13:55:47.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.620+0000 7fb14d18a740 1 -- start start 2024-04-02T13:55:47.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.620+0000 7fb14d18a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e04d7a0 0x557a9e04db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.620+0000 7fb14d18a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9e04e0c0 0x557a9e056760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.620+0000 7fb14d18a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a9e056ca0 0x557a9e059090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.620+0000 7fb14d18a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557a9dfb17b0 con 0x557a9e04d7a0 2024-04-02T13:55:47.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.620+0000 7fb14d18a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557a9df50570 con 0x557a9e056ca0 2024-04-02T13:55:47.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.620+0000 7fb14d18a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557a9dfad760 con 0x557a9e04e0c0 2024-04-02T13:55:47.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.620+0000 7fb148a72640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e04d7a0 0x557a9e04db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.620+0000 7fb148a72640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e04d7a0 0x557a9e04db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34078/0 (socket says 172.21.15.52:34078) 2024-04-02T13:55:47.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.620+0000 7fb148a72640 1 -- 172.21.15.52:0/530859526 learned_addr learned my addr 172.21.15.52:0/530859526 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:47.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.620+0000 7fb1417fa640 1 -- 172.21.15.52:0/2393926306 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb1340109d0 con 0x7fb144068ce0 2024-04-02T13:55:47.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.620+0000 7fb1437fe640 1 --2- 172.21.15.52:0/530859526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a9e056ca0 0x557a9e059090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.620+0000 7fb148a72640 1 -- 172.21.15.52:0/530859526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9e04e0c0 msgr2=0x557a9e056760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:55:47.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.620+0000 7fb148a72640 1 --2- 172.21.15.52:0/530859526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9e04e0c0 0x557a9e056760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.620+0000 7fb148a72640 1 -- 172.21.15.52:0/530859526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a9e056ca0 msgr2=0x557a9e059090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.620+0000 7fb148a72640 1 --2- 172.21.15.52:0/530859526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a9e056ca0 0x557a9e059090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.621+0000 7fb148a72640 1 -- 172.21.15.52:0/530859526 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a9dcc53f0 con 0x557a9e04d7a0 2024-04-02T13:55:47.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.621+0000 7fb148a72640 1 --2- 172.21.15.52:0/530859526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e04d7a0 0x557a9e04db80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb13c010370 tx=0x7fb13c010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a2373ac52a3e8b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.621+0000 7fb13a7fc640 1 -- 172.21.15.52:0/530859526 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c016070 con 0x557a9e04d7a0 2024-04-02T13:55:47.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.621+0000 7fb13a7fc640 1 -- 172.21.15.52:0/530859526 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb13c004d20 con 0x557a9e04d7a0 2024-04-02T13:55:47.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.621+0000 7fb13a7fc640 1 -- 172.21.15.52:0/530859526 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c005420 con 0x557a9e04d7a0 2024-04-02T13:55:47.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.621+0000 7fb1417fa640 1 --2- 172.21.15.52:0/2393926306 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb1300a3d20 0x7fb1300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.621+0000 7fb143fff640 1 --2- 172.21.15.52:0/2393926306 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb1300a3d20 0x7fb1300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.621+0000 7fb1417fa640 1 -- 172.21.15.52:0/2393926306 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb1340d06f0 con 0x7fb144068ce0 2024-04-02T13:55:47.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.622+0000 7fb143fff640 1 --2- 172.21.15.52:0/2393926306 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb1300a3d20 0x7fb1300a61e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x557a9e04d270 tx=0x7fb134000bc0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.622+0000 7fb14d18a740 1 -- 172.21.15.52:0/530859526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e04d7a0 msgr2=0x557a9e04db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.622+0000 7fb14d18a740 1 --2- 172.21.15.52:0/530859526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e04d7a0 0x557a9e04db80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb13c010370 tx=0x7fb13c010830 comp rx=0 tx=0).stop 2024-04-02T13:55:47.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.622+0000 7fb14d18a740 1 -- 172.21.15.52:0/530859526 shutdown_connections 2024-04-02T13:55:47.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.622+0000 7fb14d18a740 1 --2- 172.21.15.52:0/530859526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a9e056ca0 0x557a9e059090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.622+0000 7fb14d18a740 1 --2- 172.21.15.52:0/530859526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9e04e0c0 0x557a9e056760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.622+0000 7fb14d18a740 1 --2- 172.21.15.52:0/530859526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e04d7a0 0x557a9e04db80 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.622+0000 7fb14d18a740 1 -- 172.21.15.52:0/530859526 >> 172.21.15.52:0/530859526 conn(0x557a9e047ed0 msgr2=0x557a9e0482b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.622+0000 7fb14d18a740 1 -- 172.21.15.52:0/530859526 shutdown_connections 2024-04-02T13:55:47.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.622+0000 7fb14d18a740 1 -- 172.21.15.52:0/530859526 wait complete. 2024-04-02T13:55:47.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.622+0000 7fb14d18a740 1 -- start start 2024-04-02T13:55:47.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.622+0000 7fb14d18a740 1 Processor -- start 2024-04-02T13:55:47.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.623+0000 7fb14d18a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e04d7a0 0x557a9e053b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.623+0000 7fb14d18a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9e04e0c0 0x557a9e054050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.623+0000 7fb14d18a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a9e056ca0 0x557a9e054590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.623+0000 7fb14d18a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557a9e059a90 con 0x557a9e04d7a0 2024-04-02T13:55:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.623+0000 7fb14d18a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557a9df6ce80 con 0x557a9e056ca0 2024-04-02T13:55:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.623+0000 7fb14d18a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557a9e059c10 con 0x557a9e04e0c0 2024-04-02T13:55:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.623+0000 7fb14d18a740 10 client.0 _ll_register_callbacks cb 0x557a9dfa8aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.623+0000 7fb148a72640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e04d7a0 0x557a9e053b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.623+0000 7fb148a72640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e04d7a0 0x557a9e053b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34080/0 (socket says 172.21.15.52:34080) 2024-04-02T13:55:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.623+0000 7fb148a72640 1 -- 172.21.15.52:0/774518819 learned_addr learned my addr 172.21.15.52:0/774518819 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.623+0000 7fb1437fe640 1 --2- 172.21.15.52:0/774518819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9e04e0c0 0x557a9e054050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.623+0000 7fb148a72640 1 -- 172.21.15.52:0/774518819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9e04e0c0 msgr2=0x557a9e054050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.623+0000 7fb148a72640 1 --2- 172.21.15.52:0/774518819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9e04e0c0 0x557a9e054050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.623+0000 7fb148a72640 1 -- 172.21.15.52:0/774518819 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a9e056ca0 msgr2=0x557a9e054590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.623+0000 7fb148a72640 1 --2- 172.21.15.52:0/774518819 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a9e056ca0 0x557a9e054590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.623+0000 7fb148a72640 1 -- 172.21.15.52:0/774518819 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a9dd8f7d0 con 0x557a9e04d7a0 2024-04-02T13:55:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.623+0000 7fb148a72640 1 --2- 172.21.15.52:0/774518819 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e04d7a0 0x557a9e053b10 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb13c009880 tx=0x7fb13c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d2cf062ee18a03 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.623+0000 7fb1237fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.623+0000 7fb1237fe640 1 -- 172.21.15.52:0/774518819 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c016070 con 0x557a9e04d7a0 2024-04-02T13:55:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.623+0000 7fb1237fe640 1 -- 172.21.15.52:0/774518819 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb13c004240 con 0x557a9e04d7a0 2024-04-02T13:55:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.623+0000 7fb14d18a740 1 -- 172.21.15.52:0/774518819 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb13c004d20 con 0x557a9e04d7a0 2024-04-02T13:55:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.624+0000 7fb1237fe640 1 -- 172.21.15.52:0/774518819 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb13c004b80 con 0x557a9e04d7a0 2024-04-02T13:55:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.624+0000 7fb14d18a740 10 client.16350 fetch_fsmap learned FSMap version 29 2024-04-02T13:55:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.624+0000 7fb14d18a740 1 -- 172.21.15.52:0/774518819 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557a9e065bb0 con 0x557a9e04d7a0 2024-04-02T13:55:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.624+0000 7fb1237fe640 1 -- 172.21.15.52:0/774518819 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb13c004d20 con 0x557a9e04d7a0 2024-04-02T13:55:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.624+0000 7fb1237fe640 1 -- 172.21.15.52:0/774518819 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fb13c00f040 con 0x557a9e04d7a0 2024-04-02T13:55:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.624+0000 7fb14d18a740 10 client.16350 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:55:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.624+0000 7fb14d18a740 10 client.16350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.624+0000 7fb14d18a740 20 client.16350 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.624+0000 7fb14d18a740 1 --2- 172.21.15.52:0/774518819 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557a9e066f60 0x557a9e069350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.624+0000 7fb14d18a740 4 client.16350 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.624+0000 7fb14d18a740 1 -- 172.21.15.52:0/774518819 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a9dfbecf0 con 0x557a9e066f60 2024-04-02T13:55:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.624+0000 7fb1437fe640 1 --2- 172.21.15.52:0/774518819 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557a9e066f60 0x557a9e069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.625+0000 7fb1437fe640 1 --2- 172.21.15.52:0/774518819 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557a9e066f60 0x557a9e069350 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3251624730d1746f server_cookie=c7adda202ee2ca8f in_seq=0 out_seq=0 2024-04-02T13:55:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.625+0000 7fb1237fe640 10 client.16350 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.625+0000 7fb1237fe640 1 -- 172.21.15.52:0/774518819 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557a9dfbecf0 con 0x557a9e066f60 2024-04-02T13:55:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.625+0000 7fb1237fe640 10 client.16350 handle_command_reply: tid=0 2024-04-02T13:55:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.625+0000 7fb14d18a740 1 client.16350 shutdown 2024-04-02T13:55:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.626+0000 7fb14d18a740 1 -- 172.21.15.52:0/774518819 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e04d7a0 msgr2=0x557a9e053b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.626+0000 7fb14d18a740 1 --2- 172.21.15.52:0/774518819 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e04d7a0 0x557a9e053b10 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb13c009880 tx=0x7fb13c003ba0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.626+0000 7fb14d18a740 1 -- 172.21.15.52:0/774518819 shutdown_connections 2024-04-02T13:55:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.626+0000 7fb14d18a740 1 --2- 172.21.15.52:0/774518819 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a9e056ca0 0x557a9e054590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.626+0000 7fb14d18a740 1 --2- 172.21.15.52:0/774518819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9e04e0c0 0x557a9e054050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.626+0000 7fb14d18a740 1 --2- 172.21.15.52:0/774518819 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557a9e066f60 0x557a9e069350 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.626+0000 7fb1237fe640 0 client.16350 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.626+0000 7fb14d18a740 1 --2- 172.21.15.52:0/774518819 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e04d7a0 0x557a9e053b10 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.626+0000 7fb14d18a740 1 -- 172.21.15.52:0/774518819 >> 172.21.15.52:0/774518819 conn(0x557a9e047ed0 msgr2=0x557a9e049c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.626+0000 7fb14d18a740 1 -- 172.21.15.52:0/774518819 shutdown_connections 2024-04-02T13:55:47.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.626+0000 7fb14d18a740 1 -- 172.21.15.52:0/774518819 wait complete. 2024-04-02T13:55:47.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.626+0000 7fb14d18a740 20 client.16350 trim_cache size 0 max 16384 2024-04-02T13:55:47.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.650+0000 7fb14d18a740 1 Processor -- start 2024-04-02T13:55:47.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.650+0000 7fb14d18a740 1 -- start start 2024-04-02T13:55:47.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.650+0000 7fb14d18a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9e04d7a0 0x557a9e056240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.650+0000 7fb14d18a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a9e04e0c0 0x557a9e062980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.650+0000 7fb14d18a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e056ca0 0x557a9e06eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.650+0000 7fb14d18a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb1441aef00 con 0x557a9e056ca0 2024-04-02T13:55:47.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.650+0000 7fb14d18a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb1441b0a70 con 0x557a9e04e0c0 2024-04-02T13:55:47.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.650+0000 7fb14d18a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb1441af5d0 con 0x557a9e04d7a0 2024-04-02T13:55:47.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.650+0000 7fb148a72640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e056ca0 0x557a9e06eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.650+0000 7fb148a72640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e056ca0 0x557a9e06eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34084/0 (socket says 172.21.15.52:34084) 2024-04-02T13:55:47.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.650+0000 7fb148a72640 1 -- 172.21.15.52:0/45110696 learned_addr learned my addr 172.21.15.52:0/45110696 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:47.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.650+0000 7fb1437fe640 1 --2- 172.21.15.52:0/45110696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a9e04e0c0 0x557a9e062980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.650+0000 7fb148a72640 1 --2- 172.21.15.52:0/45110696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9e04d7a0 0x557a9e056240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.650+0000 7fb148a72640 1 -- 172.21.15.52:0/45110696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9e04d7a0 msgr2=0x557a9e056240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.650+0000 7fb148a72640 1 --2- 172.21.15.52:0/45110696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9e04d7a0 0x557a9e056240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.650+0000 7fb148a72640 1 -- 172.21.15.52:0/45110696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a9e04e0c0 msgr2=0x557a9e062980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.650+0000 7fb148a72640 1 --2- 172.21.15.52:0/45110696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a9e04e0c0 0x557a9e062980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.650+0000 7fb148a72640 1 -- 172.21.15.52:0/45110696 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a9e0698e0 con 0x557a9e056ca0 2024-04-02T13:55:47.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.650+0000 7fb148a72640 1 --2- 172.21.15.52:0/45110696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e056ca0 0x557a9e06eb10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb13c000f80 tx=0x7fb13c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bfcb17d85e03dca server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.651+0000 7fb139ffb640 1 -- 172.21.15.52:0/45110696 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c016070 con 0x557a9e056ca0 2024-04-02T13:55:47.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.651+0000 7fb139ffb640 1 -- 172.21.15.52:0/45110696 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb13c028070 con 0x557a9e056ca0 2024-04-02T13:55:47.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.651+0000 7fb139ffb640 1 -- 172.21.15.52:0/45110696 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c012800 con 0x557a9e056ca0 2024-04-02T13:55:47.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.651+0000 7fb14d18a740 1 -- 172.21.15.52:0/45110696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e056ca0 msgr2=0x557a9e06eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.651+0000 7fb14d18a740 1 --2- 172.21.15.52:0/45110696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e056ca0 0x557a9e06eb10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb13c000f80 tx=0x7fb13c003f90 comp rx=0 tx=0).stop 2024-04-02T13:55:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.651+0000 7fb14d18a740 1 -- 172.21.15.52:0/45110696 shutdown_connections 2024-04-02T13:55:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.651+0000 7fb14d18a740 1 --2- 172.21.15.52:0/45110696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e056ca0 0x557a9e06eb10 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.651+0000 7fb14d18a740 1 --2- 172.21.15.52:0/45110696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a9e04e0c0 0x557a9e062980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.651+0000 7fb14d18a740 1 --2- 172.21.15.52:0/45110696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9e04d7a0 0x557a9e056240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.651+0000 7fb14d18a740 1 -- 172.21.15.52:0/45110696 >> 172.21.15.52:0/45110696 conn(0x557a9e047ed0 msgr2=0x557a9e054360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.651+0000 7fb14d18a740 1 -- 172.21.15.52:0/45110696 shutdown_connections 2024-04-02T13:55:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.651+0000 7fb14d18a740 1 -- 172.21.15.52:0/45110696 wait complete. 2024-04-02T13:55:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.652+0000 7fb14d18a740 1 -- start start 2024-04-02T13:55:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.652+0000 7fb14d18a740 1 Processor -- start 2024-04-02T13:55:47.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.652+0000 7fb14d18a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a9e04d7a0 0x557a9e05ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.652+0000 7fb14d18a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e04e0c0 0x557a9e05f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.652+0000 7fb14d18a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9e056ca0 0x557a9e050f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.652+0000 7fb14d18a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb1441b0370 con 0x557a9e04e0c0 2024-04-02T13:55:47.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.652+0000 7fb14d18a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb1441afca0 con 0x557a9e04d7a0 2024-04-02T13:55:47.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.652+0000 7fb14d18a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb1441acee0 con 0x557a9e056ca0 2024-04-02T13:55:47.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.652+0000 7fb14d18a740 10 client.0 _ll_register_callbacks cb 0x557a9e056bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:47.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.652+0000 7fb1437fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e04e0c0 0x557a9e05f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.652+0000 7fb1437fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e04e0c0 0x557a9e05f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34086/0 (socket says 172.21.15.52:34086) 2024-04-02T13:55:47.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.652+0000 7fb1437fe640 1 -- 172.21.15.52:0/2544927683 learned_addr learned my addr 172.21.15.52:0/2544927683 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:47.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.652+0000 7fb148a72640 1 --2- 172.21.15.52:0/2544927683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9e056ca0 0x557a9e050f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.652+0000 7fb148a72640 1 --2- 172.21.15.52:0/2544927683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a9e04d7a0 0x557a9e05ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.653+0000 7fb1437fe640 1 -- 172.21.15.52:0/2544927683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9e056ca0 msgr2=0x557a9e050f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.653+0000 7fb1437fe640 1 --2- 172.21.15.52:0/2544927683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9e056ca0 0x557a9e050f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.653+0000 7fb1437fe640 1 -- 172.21.15.52:0/2544927683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a9e04d7a0 msgr2=0x557a9e05ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:47.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.653+0000 7fb1437fe640 1 --2- 172.21.15.52:0/2544927683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a9e04d7a0 0x557a9e05ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.653+0000 7fb1437fe640 1 -- 172.21.15.52:0/2544927683 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a9e051740 con 0x557a9e04e0c0 2024-04-02T13:55:47.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.653+0000 7fb148a72640 1 --2- 172.21.15.52:0/2544927683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a9e04d7a0 0x557a9e05ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:47.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.653+0000 7fb1437fe640 1 --2- 172.21.15.52:0/2544927683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e04e0c0 0x557a9e05f3d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb12c036040 tx=0x7fb12c02c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d114f23a28059d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:47.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.653+0000 7fb1237fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:47.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.653+0000 7fb1237fe640 1 -- 172.21.15.52:0/2544927683 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c04c060 con 0x557a9e04e0c0 2024-04-02T13:55:47.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.653+0000 7fb1237fe640 1 -- 172.21.15.52:0/2544927683 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb12c047dd0 con 0x557a9e04e0c0 2024-04-02T13:55:47.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.653+0000 7fb14d18a740 1 -- 172.21.15.52:0/2544927683 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb13c028070 con 0x557a9e04e0c0 2024-04-02T13:55:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.653+0000 7fb1237fe640 1 -- 172.21.15.52:0/2544927683 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb12c053050 con 0x557a9e04e0c0 2024-04-02T13:55:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.653+0000 7fb14d18a740 10 client.16356 fetch_fsmap learned FSMap version 29 2024-04-02T13:55:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.653+0000 7fb14d18a740 1 -- 172.21.15.52:0/2544927683 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557a9e058160 con 0x557a9e04e0c0 2024-04-02T13:55:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.653+0000 7fb1237fe640 1 -- 172.21.15.52:0/2544927683 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb13c028070 con 0x557a9e04e0c0 2024-04-02T13:55:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.653+0000 7fb1237fe640 1 -- 172.21.15.52:0/2544927683 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fb12c0472d0 con 0x557a9e04e0c0 2024-04-02T13:55:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.653+0000 7fb14d18a740 10 client.16356 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:55:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.653+0000 7fb14d18a740 10 client.16356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.653+0000 7fb14d18a740 20 client.16356 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.654+0000 7fb14d18a740 1 --2- 172.21.15.52:0/2544927683 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557a9e0cd7f0 0x557a9e0cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.654+0000 7fb14d18a740 4 client.16356 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:55:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.654+0000 7fb14d18a740 1 -- 172.21.15.52:0/2544927683 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a9dfb9040 con 0x557a9e0cd7f0 2024-04-02T13:55:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.654+0000 7fb148a72640 1 --2- 172.21.15.52:0/2544927683 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557a9e0cd7f0 0x557a9e0cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.654+0000 7fb148a72640 1 --2- 172.21.15.52:0/2544927683 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557a9e0cd7f0 0x557a9e0cfc10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd36bb38d3a3ac77 server_cookie=b842b12663f0b2d3 in_seq=0 out_seq=0 2024-04-02T13:55:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.654+0000 7fb1237fe640 10 client.16356 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.654+0000 7fb1237fe640 1 -- 172.21.15.52:0/2544927683 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a9dfb9040 con 0x557a9e0cd7f0 2024-04-02T13:55:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.654+0000 7fb1237fe640 10 client.16356 handle_command_reply: tid=0 2024-04-02T13:55:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.654+0000 7fb14d18a740 1 client.16356 shutdown 2024-04-02T13:55:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.655+0000 7fb14d18a740 1 -- 172.21.15.52:0/2544927683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e04e0c0 msgr2=0x557a9e05f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.655+0000 7fb14d18a740 1 --2- 172.21.15.52:0/2544927683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e04e0c0 0x557a9e05f3d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb12c036040 tx=0x7fb12c02c9e0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.655+0000 7fb14d18a740 1 -- 172.21.15.52:0/2544927683 shutdown_connections 2024-04-02T13:55:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.655+0000 7fb14d18a740 1 --2- 172.21.15.52:0/2544927683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9e056ca0 0x557a9e050f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.655+0000 7fb14d18a740 1 --2- 172.21.15.52:0/2544927683 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557a9e0cd7f0 0x557a9e0cfc10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.655+0000 7fb14d18a740 1 --2- 172.21.15.52:0/2544927683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a9e04e0c0 0x557a9e05f3d0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.655+0000 7fb1237fe640 0 client.16356 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.655+0000 7fb14d18a740 1 --2- 172.21.15.52:0/2544927683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a9e04d7a0 0x557a9e05ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.655+0000 7fb14d18a740 1 -- 172.21.15.52:0/2544927683 >> 172.21.15.52:0/2544927683 conn(0x557a9e047ed0 msgr2=0x557a9e053ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.655+0000 7fb14d18a740 1 -- 172.21.15.52:0/2544927683 shutdown_connections 2024-04-02T13:55:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.655+0000 7fb14d18a740 1 -- 172.21.15.52:0/2544927683 wait complete. 2024-04-02T13:55:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.655+0000 7fb14d18a740 20 client.16356 trim_cache size 0 max 16384 2024-04-02T13:55:47.665 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:55:47.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.655+0000 7fb14a4d5640 1 -- 172.21.15.52:0/2393926306 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb1300a3d20 msgr2=0x7fb1300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.655+0000 7fb14a4d5640 1 --2- 172.21.15.52:0/2393926306 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb1300a3d20 0x7fb1300a61e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x557a9e04d270 tx=0x7fb134000bc0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.655+0000 7fb14a4d5640 1 -- 172.21.15.52:0/2393926306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb144068ce0 msgr2=0x7fb1441983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.655+0000 7fb14a4d5640 1 --2- 172.21.15.52:0/2393926306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb144068ce0 0x7fb1441983d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb13400d950 tx=0x7fb13400de10 comp rx=0 tx=0).stop 2024-04-02T13:55:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.655+0000 7fb14a4d5640 1 -- 172.21.15.52:0/2393926306 shutdown_connections 2024-04-02T13:55:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.655+0000 7fb14a4d5640 1 --2- 172.21.15.52:0/2393926306 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb1300a3d20 0x7fb1300a61e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.655+0000 7fb14a4d5640 1 --2- 172.21.15.52:0/2393926306 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb144107a80 0x7fb14419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.655+0000 7fb14a4d5640 1 --2- 172.21.15.52:0/2393926306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1440696a0 0x7fb144198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.655+0000 7fb14a4d5640 1 --2- 172.21.15.52:0/2393926306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb144068ce0 0x7fb1441983d0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.655+0000 7fb14a4d5640 1 -- 172.21.15.52:0/2393926306 >> 172.21.15.52:0/2393926306 conn(0x7fb144077b00 msgr2=0x7fb1440fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.655+0000 7fb14a4d5640 1 -- 172.21.15.52:0/2393926306 shutdown_connections 2024-04-02T13:55:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:47.656+0000 7fb14a4d5640 1 -- 172.21.15.52:0/2393926306 wait complete. 2024-04-02T13:55:47.710 INFO:tasks.workunit.client.0.smithi052.stdout:./configure \ 2024-04-02T13:55:47.710 INFO:tasks.workunit.client.0.smithi052.stdout: --libexecdir=/usr/lib \ 2024-04-02T13:55:47.710 INFO:tasks.workunit.client.0.smithi052.stdout: --exec_prefix=/var/lib 2024-04-02T13:55:47.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:47 smithi189 ceph-mon[36683]: pgmap v71: 129 pgs: 129 active+clean; 52 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 630 op/s 2024-04-02T13:55:47.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:47 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1276192279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:55:47.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:47 smithi052 ceph-mon[32104]: pgmap v71: 129 pgs: 129 active+clean; 52 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 630 op/s 2024-04-02T13:55:47.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:47 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1276192279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:55:47.841 INFO:tasks.workunit.client.0.smithi052.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-02T13:55:47.841 INFO:tasks.workunit.client.0.smithi052.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-02T13:55:47.842 INFO:tasks.workunit.client.0.smithi052.stdout:checking how to print strings... printf 2024-04-02T13:55:47.843 INFO:tasks.workunit.client.0.smithi052.stdout:checking for gcc... gcc 2024-04-02T13:55:47.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:47 smithi088 ceph-mon[36528]: pgmap v71: 129 pgs: 129 active+clean; 52 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 630 op/s 2024-04-02T13:55:47.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:47 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1276192279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:55:47.925 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether the C compiler works... yes 2024-04-02T13:55:47.925 INFO:tasks.workunit.client.0.smithi052.stdout:checking for C compiler default output file name... a.out 2024-04-02T13:55:47.962 INFO:tasks.workunit.client.0.smithi052.stdout:checking for suffix of executables... 2024-04-02T13:55:47.999 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether we are cross compiling... no 2024-04-02T13:55:48.036 INFO:tasks.workunit.client.0.smithi052.stdout:checking for suffix of object files... o 2024-04-02T13:55:48.060 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether we are using the GNU C compiler... yes 2024-04-02T13:55:48.084 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether gcc accepts -g... yes 2024-04-02T13:55:48.172 INFO:tasks.workunit.client.0.smithi052.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-02T13:55:48.176 INFO:tasks.workunit.client.0.smithi052.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-04-02T13:55:48.180 INFO:tasks.workunit.client.0.smithi052.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-02T13:55:48.181 INFO:tasks.workunit.client.0.smithi052.stdout:checking for egrep... /usr/bin/grep -E 2024-04-02T13:55:48.182 INFO:tasks.workunit.client.0.smithi052.stdout:checking for fgrep... /usr/bin/grep -F 2024-04-02T13:55:48.185 INFO:tasks.workunit.client.0.smithi052.stdout:checking for ld used by gcc... /usr/bin/ld 2024-04-02T13:55:48.187 INFO:tasks.workunit.client.0.smithi052.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-04-02T13:55:48.203 INFO:tasks.workunit.client.0.smithi052.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-04-02T13:55:48.670 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:55:48.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:55:48.671 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:55:48.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.776+0000 7fda1981b640 1 -- 172.21.15.52:0/3665199307 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda14100a20 msgr2=0x7fda14100ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:48.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.776+0000 7fda1981b640 1 --2- 172.21.15.52:0/3665199307 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda14100a20 0x7fda14100ea0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc009930 tx=0x7fd9fc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:55:48.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.777+0000 7fda1981b640 1 -- 172.21.15.52:0/3665199307 shutdown_connections 2024-04-02T13:55:48.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.777+0000 7fda1981b640 1 --2- 172.21.15.52:0/3665199307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda141013e0 0x7fda1410d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.777+0000 7fda1981b640 1 --2- 172.21.15.52:0/3665199307 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda14100a20 0x7fda14100ea0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.777+0000 7fda1981b640 1 --2- 172.21.15.52:0/3665199307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda14106a30 0x7fda14106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.777+0000 7fda1981b640 1 -- 172.21.15.52:0/3665199307 >> 172.21.15.52:0/3665199307 conn(0x7fda14077b60 msgr2=0x7fda14077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:48.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.777+0000 7fda1981b640 1 -- 172.21.15.52:0/3665199307 shutdown_connections 2024-04-02T13:55:48.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.777+0000 7fda1981b640 1 -- 172.21.15.52:0/3665199307 wait complete. 2024-04-02T13:55:48.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.777+0000 7fda1981b640 1 Processor -- start 2024-04-02T13:55:48.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.777+0000 7fda1981b640 1 -- start start 2024-04-02T13:55:48.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.777+0000 7fda1981b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda14100a20 0x7fda14198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.777+0000 7fda1981b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda141013e0 0x7fda141989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.778+0000 7fda1981b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda14106a30 0x7fda1419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.778+0000 7fda1981b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fda14110e40 con 0x7fda141013e0 2024-04-02T13:55:48.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.778+0000 7fda1981b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fda14110cc0 con 0x7fda14100a20 2024-04-02T13:55:48.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.778+0000 7fda1981b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fda14110fc0 con 0x7fda14106a30 2024-04-02T13:55:48.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.778+0000 7fda127fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda141013e0 0x7fda141989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.778+0000 7fda127fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda141013e0 0x7fda141989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34096/0 (socket says 172.21.15.52:34096) 2024-04-02T13:55:48.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.778+0000 7fda127fc640 1 -- 172.21.15.52:0/4167552101 learned_addr learned my addr 172.21.15.52:0/4167552101 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:48.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.778+0000 7fda12ffd640 1 --2- 172.21.15.52:0/4167552101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda14100a20 0x7fda14198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.778+0000 7fda137fe640 1 --2- 172.21.15.52:0/4167552101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda14106a30 0x7fda1419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.778+0000 7fda127fc640 1 -- 172.21.15.52:0/4167552101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda14106a30 msgr2=0x7fda1419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:48.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.778+0000 7fda127fc640 1 --2- 172.21.15.52:0/4167552101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda14106a30 0x7fda1419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.778+0000 7fda127fc640 1 -- 172.21.15.52:0/4167552101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda14100a20 msgr2=0x7fda14198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:48.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.778+0000 7fda127fc640 1 --2- 172.21.15.52:0/4167552101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda14100a20 0x7fda14198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.778+0000 7fda127fc640 1 -- 172.21.15.52:0/4167552101 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1419d480 con 0x7fda141013e0 2024-04-02T13:55:48.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.778+0000 7fda137fe640 1 --2- 172.21.15.52:0/4167552101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda14106a30 0x7fda1419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:48.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.778+0000 7fda127fc640 1 --2- 172.21.15.52:0/4167552101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda141013e0 0x7fda141989d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc009230 tx=0x7fd9fc02cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:48.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.778+0000 7fda18819640 1 -- 172.21.15.52:0/4167552101 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc041070 con 0x7fda141013e0 2024-04-02T13:55:48.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.778+0000 7fda1981b640 1 -- 172.21.15.52:0/4167552101 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda1419d710 con 0x7fda141013e0 2024-04-02T13:55:48.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.778+0000 7fda18819640 1 -- 172.21.15.52:0/4167552101 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9fc004450 con 0x7fda141013e0 2024-04-02T13:55:48.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.778+0000 7fda18819640 1 -- 172.21.15.52:0/4167552101 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc034440 con 0x7fda141013e0 2024-04-02T13:55:48.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.778+0000 7fda1981b640 1 -- 172.21.15.52:0/4167552101 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda141a5000 con 0x7fda141013e0 2024-04-02T13:55:48.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.781+0000 7fda18819640 1 -- 172.21.15.52:0/4167552101 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd9fc044070 con 0x7fda141013e0 2024-04-02T13:55:48.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.781+0000 7fda1c400740 1 Processor -- start 2024-04-02T13:55:48.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.781+0000 7fda1c400740 1 -- start start 2024-04-02T13:55:48.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.781+0000 7fda1c400740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a92c0 0x555c774a96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.781+0000 7fda1c400740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c774a9be0 0x555c774b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.781+0000 7fda1c400740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555c774b27c0 0x555c774b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.781+0000 7fda1c400740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555c7740d770 con 0x555c774a92c0 2024-04-02T13:55:48.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.781+0000 7fda1c400740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x555c773b5960 con 0x555c774b27c0 2024-04-02T13:55:48.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.781+0000 7fda1c400740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555c774098c0 con 0x555c774a9be0 2024-04-02T13:55:48.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.781+0000 7fda137fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a92c0 0x555c774a96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.781+0000 7fda137fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a92c0 0x555c774a96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34100/0 (socket says 172.21.15.52:34100) 2024-04-02T13:55:48.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.781+0000 7fda137fe640 1 -- 172.21.15.52:0/787962699 learned_addr learned my addr 172.21.15.52:0/787962699 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:48.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.781+0000 7fda18819640 1 --2- 172.21.15.52:0/4167552101 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fda000a3d20 0x7fda000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.781+0000 7fda137fe640 1 --2- 172.21.15.52:0/4167552101 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fda000a3d20 0x7fda000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.781+0000 7fda127fc640 1 --2- 172.21.15.52:0/787962699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555c774b27c0 0x555c774b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.781+0000 7fda12ffd640 1 --2- 172.21.15.52:0/787962699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c774a9be0 0x555c774b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.782+0000 7fda18819640 1 -- 172.21.15.52:0/4167552101 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd9fc0f2480 con 0x7fda141013e0 2024-04-02T13:55:48.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.782+0000 7fda137fe640 1 -- 172.21.15.52:0/787962699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c774a9be0 msgr2=0x555c774b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:48.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.782+0000 7fda137fe640 1 --2- 172.21.15.52:0/787962699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c774a9be0 0x555c774b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.782+0000 7fda137fe640 1 -- 172.21.15.52:0/787962699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555c774b27c0 msgr2=0x555c774b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:48.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.782+0000 7fda137fe640 1 --2- 172.21.15.52:0/787962699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555c774b27c0 0x555c774b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.782+0000 7fda137fe640 1 -- 172.21.15.52:0/787962699 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c771e97e0 con 0x555c774a92c0 2024-04-02T13:55:48.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.782+0000 7fda12ffd640 1 --2- 172.21.15.52:0/787962699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c774a9be0 0x555c774b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:48.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.782+0000 7fda137fe640 1 --2- 172.21.15.52:0/787962699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a92c0 0x555c774a96a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fda0c014890 tx=0x7fda0c014d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5de0dd4b346c650 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:48.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.782+0000 7fda09ffb640 1 -- 172.21.15.52:0/787962699 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c021070 con 0x555c774a92c0 2024-04-02T13:55:48.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.782+0000 7fda09ffb640 1 -- 172.21.15.52:0/787962699 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda0c01cb40 con 0x555c774a92c0 2024-04-02T13:55:48.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.782+0000 7fda09ffb640 1 -- 172.21.15.52:0/787962699 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c01b390 con 0x555c774a92c0 2024-04-02T13:55:48.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.783+0000 7fda1c400740 1 -- 172.21.15.52:0/787962699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a92c0 msgr2=0x555c774a96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:48.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.783+0000 7fda1c400740 1 --2- 172.21.15.52:0/787962699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a92c0 0x555c774a96a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fda0c014890 tx=0x7fda0c014d50 comp rx=0 tx=0).stop 2024-04-02T13:55:48.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.783+0000 7fda137fe640 1 --2- 172.21.15.52:0/4167552101 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fda000a3d20 0x7fda000a61e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fda0c01e000 tx=0x7fda0c01b000 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:48.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.783+0000 7fda1c400740 1 -- 172.21.15.52:0/787962699 shutdown_connections 2024-04-02T13:55:48.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.783+0000 7fda1c400740 1 --2- 172.21.15.52:0/787962699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555c774b27c0 0x555c774b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.783+0000 7fda1c400740 1 --2- 172.21.15.52:0/787962699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c774a9be0 0x555c774b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.783+0000 7fda1c400740 1 --2- 172.21.15.52:0/787962699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a92c0 0x555c774a96a0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.783+0000 7fda1c400740 1 -- 172.21.15.52:0/787962699 >> 172.21.15.52:0/787962699 conn(0x555c774a39f0 msgr2=0x555c774a3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:48.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.783+0000 7fda1c400740 1 -- 172.21.15.52:0/787962699 shutdown_connections 2024-04-02T13:55:48.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.783+0000 7fda1c400740 1 -- 172.21.15.52:0/787962699 wait complete. 2024-04-02T13:55:48.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.783+0000 7fda1c400740 1 -- start start 2024-04-02T13:55:48.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.783+0000 7fda1c400740 1 Processor -- start 2024-04-02T13:55:48.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.784+0000 7fda1c400740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c774a92c0 0x555c774af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.784+0000 7fda1c400740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a9be0 0x555c774afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.784+0000 7fda1c400740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555c774b27c0 0x555c774b0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.784+0000 7fda1c400740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555c774b55b0 con 0x555c774a9be0 2024-04-02T13:55:48.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.784+0000 7fda1c400740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x555c773c8d20 con 0x555c774b27c0 2024-04-02T13:55:48.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.784+0000 7fda1c400740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555c774b5730 con 0x555c774a92c0 2024-04-02T13:55:48.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.784+0000 7fda127fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a9be0 0x555c774afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.784+0000 7fda1c400740 10 client.0 _ll_register_callbacks cb 0x555c77395a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.784+0000 7fda127fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a9be0 0x555c774afcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34102/0 (socket says 172.21.15.52:34102) 2024-04-02T13:55:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.784+0000 7fda127fc640 1 -- 172.21.15.52:0/2647189786 learned_addr learned my addr 172.21.15.52:0/2647189786 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.784+0000 7fda137fe640 1 --2- 172.21.15.52:0/2647189786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555c774b27c0 0x555c774b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.784+0000 7fda12ffd640 1 --2- 172.21.15.52:0/2647189786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c774a92c0 0x555c774af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.784+0000 7fda127fc640 1 -- 172.21.15.52:0/2647189786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c774a92c0 msgr2=0x555c774af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.784+0000 7fda127fc640 1 --2- 172.21.15.52:0/2647189786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c774a92c0 0x555c774af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.784+0000 7fda127fc640 1 -- 172.21.15.52:0/2647189786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555c774b27c0 msgr2=0x555c774b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.784+0000 7fda127fc640 1 --2- 172.21.15.52:0/2647189786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555c774b27c0 0x555c774b0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.784+0000 7fda127fc640 1 -- 172.21.15.52:0/2647189786 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c77249f70 con 0x555c774a9be0 2024-04-02T13:55:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.784+0000 7fda137fe640 1 --2- 172.21.15.52:0/2647189786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555c774b27c0 0x555c774b0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.784+0000 7fda127fc640 1 --2- 172.21.15.52:0/2647189786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a9be0 0x555c774afcd0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc0efe90 tx=0x7fd9fc02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7950121f0a6cdea4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.784+0000 7fd9f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:48.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.784+0000 7fd9f2ffd640 1 -- 172.21.15.52:0/2647189786 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc04b050 con 0x555c774a9be0 2024-04-02T13:55:48.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.784+0000 7fd9f2ffd640 1 -- 172.21.15.52:0/2647189786 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd9fc0035f0 con 0x555c774a9be0 2024-04-02T13:55:48.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.784+0000 7fda1c400740 1 -- 172.21.15.52:0/2647189786 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda0c01cb40 con 0x555c774a9be0 2024-04-02T13:55:48.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.784+0000 7fd9f2ffd640 1 -- 172.21.15.52:0/2647189786 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd9fc0bdb50 con 0x555c774a9be0 2024-04-02T13:55:48.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.784+0000 7fda1c400740 10 client.16368 fetch_fsmap learned FSMap version 29 2024-04-02T13:55:48.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.785+0000 7fda1c400740 1 -- 172.21.15.52:0/2647189786 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555c770e1170 con 0x555c774a9be0 2024-04-02T13:55:48.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.785+0000 7fd9f2ffd640 1 -- 172.21.15.52:0/2647189786 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda0c01cb40 con 0x555c774a9be0 2024-04-02T13:55:48.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.785+0000 7fd9f2ffd640 1 -- 172.21.15.52:0/2647189786 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fd9fc0c2160 con 0x555c774a9be0 2024-04-02T13:55:48.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.785+0000 7fda1c400740 10 client.16368 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:55:48.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.785+0000 7fda1c400740 10 client.16368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:48.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.785+0000 7fda1c400740 20 client.16368 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:48.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.785+0000 7fda1c400740 1 --2- 172.21.15.52:0/2647189786 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x555c774c2aa0 0x555c774c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.785+0000 7fda1c400740 4 client.16368 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:48.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.785+0000 7fda1c400740 1 -- 172.21.15.52:0/2647189786 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c7741acb0 con 0x555c774c2aa0 2024-04-02T13:55:48.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.785+0000 7fda12ffd640 1 --2- 172.21.15.52:0/2647189786 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x555c774c2aa0 0x555c774c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.785+0000 7fd9f2ffd640 10 client.16368 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:48.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.785+0000 7fda12ffd640 1 --2- 172.21.15.52:0/2647189786 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x555c774c2aa0 0x555c774c4ee0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d93ddb2ff30e5b33 server_cookie=e02ca80ba6ea2de8 in_seq=0 out_seq=0 2024-04-02T13:55:48.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.786+0000 7fd9f2ffd640 1 -- 172.21.15.52:0/2647189786 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555c7741acb0 con 0x555c774c2aa0 2024-04-02T13:55:48.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.786+0000 7fd9f2ffd640 10 client.16368 handle_command_reply: tid=0 2024-04-02T13:55:48.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.786+0000 7fda1c400740 1 client.16368 shutdown 2024-04-02T13:55:48.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.786+0000 7fda1c400740 1 -- 172.21.15.52:0/2647189786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a9be0 msgr2=0x555c774afcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:48.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.786+0000 7fda1c400740 1 --2- 172.21.15.52:0/2647189786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a9be0 0x555c774afcd0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc0efe90 tx=0x7fd9fc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:55:48.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.786+0000 7fda1c400740 1 -- 172.21.15.52:0/2647189786 shutdown_connections 2024-04-02T13:55:48.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.786+0000 7fda1c400740 1 --2- 172.21.15.52:0/2647189786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555c774b27c0 0x555c774b0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.786+0000 7fda1c400740 1 --2- 172.21.15.52:0/2647189786 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x555c774c2aa0 0x555c774c4ee0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.786+0000 7fda1c400740 1 --2- 172.21.15.52:0/2647189786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a9be0 0x555c774afcd0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.786+0000 7fd9f2ffd640 0 client.16368 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:48.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.786+0000 7fda1c400740 1 --2- 172.21.15.52:0/2647189786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c774a92c0 0x555c774af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.786+0000 7fda1c400740 1 -- 172.21.15.52:0/2647189786 >> 172.21.15.52:0/2647189786 conn(0x555c774a39f0 msgr2=0x555c774a5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:48.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.786+0000 7fda1c400740 1 -- 172.21.15.52:0/2647189786 shutdown_connections 2024-04-02T13:55:48.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.786+0000 7fda1c400740 1 -- 172.21.15.52:0/2647189786 wait complete. 2024-04-02T13:55:48.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.786+0000 7fda1c400740 20 client.16368 trim_cache size 0 max 16384 2024-04-02T13:55:48.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.804+0000 7fda1c400740 1 Processor -- start 2024-04-02T13:55:48.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.804+0000 7fda1c400740 1 -- start start 2024-04-02T13:55:48.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.804+0000 7fda1c400740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c774a92c0 0x555c774b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.804+0000 7fda1c400740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a9be0 0x555c774be4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.804+0000 7fda1c400740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555c774b27c0 0x555c774ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.804+0000 7fda1c400740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fda141aed10 con 0x555c774a9be0 2024-04-02T13:55:48.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.804+0000 7fda1c400740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fda141b0880 con 0x555c774b27c0 2024-04-02T13:55:48.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.804+0000 7fda1c400740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fda141af3e0 con 0x555c774a92c0 2024-04-02T13:55:48.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.804+0000 7fda127fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a9be0 0x555c774be4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.804+0000 7fda127fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a9be0 0x555c774be4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34108/0 (socket says 172.21.15.52:34108) 2024-04-02T13:55:48.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.804+0000 7fda127fc640 1 -- 172.21.15.52:0/3400447059 learned_addr learned my addr 172.21.15.52:0/3400447059 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:48.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.805+0000 7fda137fe640 1 --2- 172.21.15.52:0/3400447059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555c774b27c0 0x555c774ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.805+0000 7fda127fc640 1 -- 172.21.15.52:0/3400447059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c774a92c0 msgr2=0x555c774b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:48.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.805+0000 7fda127fc640 1 --2- 172.21.15.52:0/3400447059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c774a92c0 0x555c774b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.805+0000 7fda127fc640 1 -- 172.21.15.52:0/3400447059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555c774b27c0 msgr2=0x555c774ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:48.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.805+0000 7fda127fc640 1 --2- 172.21.15.52:0/3400447059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555c774b27c0 0x555c774ca7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.805+0000 7fda127fc640 1 -- 172.21.15.52:0/3400447059 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c774ae740 con 0x555c774a9be0 2024-04-02T13:55:48.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.805+0000 7fda137fe640 1 --2- 172.21.15.52:0/3400447059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555c774b27c0 0x555c774ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:48.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.805+0000 7fda127fc640 1 --2- 172.21.15.52:0/3400447059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a9be0 0x555c774be4d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc009430 tx=0x7fd9fc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f967899d935d31fd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:48.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.805+0000 7fda097fa640 1 -- 172.21.15.52:0/3400447059 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc04b070 con 0x555c774a9be0 2024-04-02T13:55:48.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.805+0000 7fda097fa640 1 -- 172.21.15.52:0/3400447059 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9fc004c80 con 0x555c774a9be0 2024-04-02T13:55:48.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.805+0000 7fda097fa640 1 -- 172.21.15.52:0/3400447059 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc0f2120 con 0x555c774a9be0 2024-04-02T13:55:48.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.805+0000 7fda1c400740 1 -- 172.21.15.52:0/3400447059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a9be0 msgr2=0x555c774be4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:48.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.805+0000 7fda1c400740 1 --2- 172.21.15.52:0/3400447059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a9be0 0x555c774be4d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc009430 tx=0x7fd9fc000bc0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.806+0000 7fda1c400740 1 -- 172.21.15.52:0/3400447059 shutdown_connections 2024-04-02T13:55:48.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.806+0000 7fda1c400740 1 --2- 172.21.15.52:0/3400447059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555c774b27c0 0x555c774ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.806+0000 7fda1c400740 1 --2- 172.21.15.52:0/3400447059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a9be0 0x555c774be4d0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.806+0000 7fda1c400740 1 --2- 172.21.15.52:0/3400447059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c774a92c0 0x555c774b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.806+0000 7fda1c400740 1 -- 172.21.15.52:0/3400447059 >> 172.21.15.52:0/3400447059 conn(0x555c774a39f0 msgr2=0x555c774a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:48.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.806+0000 7fda1c400740 1 -- 172.21.15.52:0/3400447059 shutdown_connections 2024-04-02T13:55:48.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.806+0000 7fda1c400740 1 -- 172.21.15.52:0/3400447059 wait complete. 2024-04-02T13:55:48.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.806+0000 7fda1c400740 1 -- start start 2024-04-02T13:55:48.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.806+0000 7fda1c400740 1 Processor -- start 2024-04-02T13:55:48.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.806+0000 7fda1c400740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a92c0 0x555c774ba680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.806+0000 7fda1c400740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c774a9be0 0x555c774babc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.806+0000 7fda1c400740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555c774b27c0 0x555c774b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.806+0000 7fda1c400740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fda141b0180 con 0x555c774a92c0 2024-04-02T13:55:48.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.806+0000 7fda1c400740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fda141afab0 con 0x555c774b27c0 2024-04-02T13:55:48.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.806+0000 7fda1c400740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fda141accf0 con 0x555c774a9be0 2024-04-02T13:55:48.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.806+0000 7fda1c400740 10 client.0 _ll_register_callbacks cb 0x7fd9fc003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:48.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.806+0000 7fda12ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a92c0 0x555c774ba680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.806+0000 7fda12ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a92c0 0x555c774ba680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34118/0 (socket says 172.21.15.52:34118) 2024-04-02T13:55:48.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.806+0000 7fda12ffd640 1 -- 172.21.15.52:0/1144958844 learned_addr learned my addr 172.21.15.52:0/1144958844 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:48.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.806+0000 7fda137fe640 1 --2- 172.21.15.52:0/1144958844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555c774b27c0 0x555c774b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.807+0000 7fda127fc640 1 --2- 172.21.15.52:0/1144958844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c774a9be0 0x555c774babc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.807+0000 7fda12ffd640 1 -- 172.21.15.52:0/1144958844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c774a9be0 msgr2=0x555c774babc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:48.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.807+0000 7fda12ffd640 1 --2- 172.21.15.52:0/1144958844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c774a9be0 0x555c774babc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.807+0000 7fda12ffd640 1 -- 172.21.15.52:0/1144958844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555c774b27c0 msgr2=0x555c774b6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:48.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.807+0000 7fda12ffd640 1 --2- 172.21.15.52:0/1144958844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555c774b27c0 0x555c774b6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.807+0000 7fda12ffd640 1 -- 172.21.15.52:0/1144958844 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c774b7210 con 0x555c774a92c0 2024-04-02T13:55:48.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.807+0000 7fda12ffd640 1 --2- 172.21.15.52:0/1144958844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a92c0 0x555c774ba680 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fda040125a0 tx=0x7fda04013b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd8d92c8dd2966 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:48.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.807+0000 7fd9f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:48.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.807+0000 7fd9f2ffd640 1 -- 172.21.15.52:0/1144958844 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda04004d20 con 0x555c774a92c0 2024-04-02T13:55:48.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.807+0000 7fd9f2ffd640 1 -- 172.21.15.52:0/1144958844 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fda0401d470 con 0x555c774a92c0 2024-04-02T13:55:48.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.807+0000 7fda1c400740 1 -- 172.21.15.52:0/1144958844 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9fc004c80 con 0x555c774a92c0 2024-04-02T13:55:48.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.807+0000 7fd9f2ffd640 1 -- 172.21.15.52:0/1144958844 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fda0402c050 con 0x555c774a92c0 2024-04-02T13:55:48.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.807+0000 7fda1c400740 10 client.16374 fetch_fsmap learned FSMap version 29 2024-04-02T13:55:48.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.807+0000 7fda1c400740 1 -- 172.21.15.52:0/1144958844 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555c77529a60 con 0x555c774a92c0 2024-04-02T13:55:48.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.807+0000 7fd9f2ffd640 1 -- 172.21.15.52:0/1144958844 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9fc004c80 con 0x555c774a92c0 2024-04-02T13:55:48.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.807+0000 7fd9f2ffd640 1 -- 172.21.15.52:0/1144958844 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fda04005050 con 0x555c774a92c0 2024-04-02T13:55:48.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.808+0000 7fda1c400740 10 client.16374 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:55:48.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.808+0000 7fda1c400740 10 client.16374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:48.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.808+0000 7fda1c400740 20 client.16374 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:48.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.808+0000 7fda1c400740 1 --2- 172.21.15.52:0/1144958844 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x555c7752a210 0x555c7752c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.808+0000 7fda1c400740 4 client.16374 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:55:48.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.808+0000 7fda1c400740 1 -- 172.21.15.52:0/1144958844 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555c77415000 con 0x555c7752a210 2024-04-02T13:55:48.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.808+0000 7fda127fc640 1 --2- 172.21.15.52:0/1144958844 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x555c7752a210 0x555c7752c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.808+0000 7fda127fc640 1 --2- 172.21.15.52:0/1144958844 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x555c7752a210 0x555c7752c630 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b935b932074ee99f server_cookie=74500f0416d522b5 in_seq=0 out_seq=0 2024-04-02T13:55:48.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.808+0000 7fd9f2ffd640 10 client.16374 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:48.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.809+0000 7fd9f2ffd640 1 -- 172.21.15.52:0/1144958844 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555c77415000 con 0x555c7752a210 2024-04-02T13:55:48.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.809+0000 7fd9f2ffd640 10 client.16374 handle_command_reply: tid=0 2024-04-02T13:55:48.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.809+0000 7fda1c400740 1 client.16374 shutdown 2024-04-02T13:55:48.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.809+0000 7fda1c400740 1 -- 172.21.15.52:0/1144958844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a92c0 msgr2=0x555c774ba680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:48.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.809+0000 7fda1c400740 1 --2- 172.21.15.52:0/1144958844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a92c0 0x555c774ba680 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fda040125a0 tx=0x7fda04013b10 comp rx=0 tx=0).stop 2024-04-02T13:55:48.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.809+0000 7fda1c400740 1 -- 172.21.15.52:0/1144958844 shutdown_connections 2024-04-02T13:55:48.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.809+0000 7fda1c400740 1 --2- 172.21.15.52:0/1144958844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555c774b27c0 0x555c774b6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.809+0000 7fda1c400740 1 --2- 172.21.15.52:0/1144958844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c774a9be0 0x555c774babc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.809+0000 7fda1c400740 1 --2- 172.21.15.52:0/1144958844 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x555c7752a210 0x555c7752c630 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.809+0000 7fda1c400740 1 --2- 172.21.15.52:0/1144958844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c774a92c0 0x555c774ba680 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.809+0000 7fd9f2ffd640 0 client.16374 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:48.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.809+0000 7fda1c400740 1 -- 172.21.15.52:0/1144958844 >> 172.21.15.52:0/1144958844 conn(0x555c774a39f0 msgr2=0x555c774af750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:48.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.809+0000 7fda1c400740 1 -- 172.21.15.52:0/1144958844 shutdown_connections 2024-04-02T13:55:48.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.810+0000 7fda1c400740 1 -- 172.21.15.52:0/1144958844 wait complete. 2024-04-02T13:55:48.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.810+0000 7fda1c400740 20 client.16374 trim_cache size 0 max 16384 2024-04-02T13:55:48.821 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:55:48.821 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-04-02T13:55:48.821 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "1ab104e5-8378-4f0e-96c6-af3968f2fce3", 2024-04-02T13:55:48.821 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-04-02T13:55:48.821 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:55:48.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.810+0000 7fda1981b640 1 -- 172.21.15.52:0/4167552101 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fda000a3d20 msgr2=0x7fda000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:48.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.810+0000 7fda1981b640 1 --2- 172.21.15.52:0/4167552101 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fda000a3d20 0x7fda000a61e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fda0c01e000 tx=0x7fda0c01b000 comp rx=0 tx=0).stop 2024-04-02T13:55:48.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.810+0000 7fda1981b640 1 -- 172.21.15.52:0/4167552101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda141013e0 msgr2=0x7fda141989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:48.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.810+0000 7fda1981b640 1 --2- 172.21.15.52:0/4167552101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda141013e0 0x7fda141989d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc009230 tx=0x7fd9fc02cba0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.810+0000 7fda1981b640 1 -- 172.21.15.52:0/4167552101 shutdown_connections 2024-04-02T13:55:48.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.810+0000 7fda1981b640 1 --2- 172.21.15.52:0/4167552101 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fda000a3d20 0x7fda000a61e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.810+0000 7fda1981b640 1 --2- 172.21.15.52:0/4167552101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda14106a30 0x7fda1419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.810+0000 7fda1981b640 1 --2- 172.21.15.52:0/4167552101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda141013e0 0x7fda141989d0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.810+0000 7fda1981b640 1 --2- 172.21.15.52:0/4167552101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda14100a20 0x7fda14198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.810+0000 7fda1981b640 1 -- 172.21.15.52:0/4167552101 >> 172.21.15.52:0/4167552101 conn(0x7fda14077b60 msgr2=0x7fda1406a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:48.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.810+0000 7fda1981b640 1 -- 172.21.15.52:0/4167552101 shutdown_connections 2024-04-02T13:55:48.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.810+0000 7fda1981b640 1 -- 172.21.15.52:0/4167552101 wait complete. 2024-04-02T13:55:48.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:55:48.824 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:55:48.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.933+0000 7fe2c28c5640 1 -- 172.21.15.52:0/2221809269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2bc100970 msgr2=0x7fe2bc100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:48.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.933+0000 7fe2c28c5640 1 --2- 172.21.15.52:0/2221809269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2bc100970 0x7fe2bc100df0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009930 tx=0x7fe2b002bc30 comp rx=0 tx=0).stop 2024-04-02T13:55:48.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.933+0000 7fe2c28c5640 1 -- 172.21.15.52:0/2221809269 shutdown_connections 2024-04-02T13:55:48.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.933+0000 7fe2c28c5640 1 --2- 172.21.15.52:0/2221809269 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2bc101330 0x7fe2bc10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.933+0000 7fe2c28c5640 1 --2- 172.21.15.52:0/2221809269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2bc100970 0x7fe2bc100df0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.933+0000 7fe2c28c5640 1 --2- 172.21.15.52:0/2221809269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2bc106980 0x7fe2bc106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.933+0000 7fe2c28c5640 1 -- 172.21.15.52:0/2221809269 >> 172.21.15.52:0/2221809269 conn(0x7fe2bc077b00 msgr2=0x7fe2bc077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:48.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.933+0000 7fe2c28c5640 1 -- 172.21.15.52:0/2221809269 shutdown_connections 2024-04-02T13:55:48.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.933+0000 7fe2c28c5640 1 -- 172.21.15.52:0/2221809269 wait complete. 2024-04-02T13:55:48.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.934+0000 7fe2c28c5640 1 Processor -- start 2024-04-02T13:55:48.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.934+0000 7fe2c28c5640 1 -- start start 2024-04-02T13:55:48.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.934+0000 7fe2c28c5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2bc100970 0x7fe2bc1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.934+0000 7fe2c28c5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2bc101330 0x7fe2bc198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.934+0000 7fe2c28c5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2bc106980 0x7fe2bc19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.934+0000 7fe2c28c5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc110d60 con 0x7fe2bc101330 2024-04-02T13:55:48.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.934+0000 7fe2c28c5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc110be0 con 0x7fe2bc100970 2024-04-02T13:55:48.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.934+0000 7fe2c28c5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc110ee0 con 0x7fe2bc106980 2024-04-02T13:55:48.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.934+0000 7fe2bb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2bc101330 0x7fe2bc198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.935+0000 7fe2bb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2bc101330 0x7fe2bc198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34130/0 (socket says 172.21.15.52:34130) 2024-04-02T13:55:48.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.935+0000 7fe2bb7fe640 1 -- 172.21.15.52:0/3958112646 learned_addr learned my addr 172.21.15.52:0/3958112646 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:48.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.935+0000 7fe2bbfff640 1 --2- 172.21.15.52:0/3958112646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2bc100970 0x7fe2bc1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.935+0000 7fe2c0e62640 1 --2- 172.21.15.52:0/3958112646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2bc106980 0x7fe2bc19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.935+0000 7fe2bb7fe640 1 -- 172.21.15.52:0/3958112646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2bc106980 msgr2=0x7fe2bc19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:48.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.935+0000 7fe2bb7fe640 1 --2- 172.21.15.52:0/3958112646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2bc106980 0x7fe2bc19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.935+0000 7fe2bb7fe640 1 -- 172.21.15.52:0/3958112646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2bc100970 msgr2=0x7fe2bc1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:48.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.935+0000 7fe2bb7fe640 1 --2- 172.21.15.52:0/3958112646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2bc100970 0x7fe2bc1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.935+0000 7fe2bb7fe640 1 -- 172.21.15.52:0/3958112646 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2bc19d3c0 con 0x7fe2bc101330 2024-04-02T13:55:48.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.935+0000 7fe2bb7fe640 1 --2- 172.21.15.52:0/3958112646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2bc101330 0x7fe2bc198910 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009230 tx=0x7fe2b002cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:48.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.935+0000 7fe2b97fa640 1 -- 172.21.15.52:0/3958112646 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0041070 con 0x7fe2bc101330 2024-04-02T13:55:48.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.935+0000 7fe2c28c5640 1 -- 172.21.15.52:0/3958112646 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2bc19d650 con 0x7fe2bc101330 2024-04-02T13:55:48.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.935+0000 7fe2b97fa640 1 -- 172.21.15.52:0/3958112646 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2b0004450 con 0x7fe2bc101330 2024-04-02T13:55:48.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.936+0000 7fe2b97fa640 1 -- 172.21.15.52:0/3958112646 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0003e50 con 0x7fe2bc101330 2024-04-02T13:55:48.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.936+0000 7fe2c28c5640 1 -- 172.21.15.52:0/3958112646 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2bc1a4f40 con 0x7fe2bc101330 2024-04-02T13:55:48.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.938+0000 7fe2b97fa640 1 -- 172.21.15.52:0/3958112646 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe2b0044070 con 0x7fe2bc101330 2024-04-02T13:55:48.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.938+0000 7fe2b97fa640 1 --2- 172.21.15.52:0/3958112646 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe2a40a3ec0 0x7fe2a40a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.938+0000 7fe2b97fa640 1 -- 172.21.15.52:0/3958112646 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe2b00f1c30 con 0x7fe2bc101330 2024-04-02T13:55:48.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.938+0000 7fe2bbfff640 1 --2- 172.21.15.52:0/3958112646 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe2a40a3ec0 0x7fe2a40a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.939+0000 7fe2c5586740 1 Processor -- start 2024-04-02T13:55:48.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.939+0000 7fe2c5586740 1 -- start start 2024-04-02T13:55:48.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.939+0000 7fe2bbfff640 1 --2- 172.21.15.52:0/3958112646 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe2a40a3ec0 0x7fe2a40a6380 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009b50 tx=0x7fe2ac005f90 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:48.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.939+0000 7fe2c5586740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aad0bae2b0 0x55aad0bae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.939+0000 7fe2c5586740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aad0baec60 0x55aad0bbbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.939+0000 7fe2c5586740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0bbc1e0 0x55aad0bbe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.939+0000 7fe2c5586740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55aad0b12830 con 0x55aad0bbc1e0 2024-04-02T13:55:48.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.939+0000 7fe2c5586740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55aad0aba820 con 0x55aad0bae2b0 2024-04-02T13:55:48.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.939+0000 7fe2c5586740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55aad0b0e7e0 con 0x55aad0baec60 2024-04-02T13:55:48.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.939+0000 7fe2c0e62640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0bbc1e0 0x55aad0bbe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.939+0000 7fe2bbfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aad0bae2b0 0x55aad0bae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.939+0000 7fe2bbfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aad0bae2b0 0x55aad0bae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:33476/0 (socket says 172.21.15.52:33476) 2024-04-02T13:55:48.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.939+0000 7fe2bbfff640 1 -- 172.21.15.52:0/4027880865 learned_addr learned my addr 172.21.15.52:0/4027880865 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:48.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.939+0000 7fe2bb7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aad0baec60 0x55aad0bbbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.939+0000 7fe2c0e62640 1 -- 172.21.15.52:0/4027880865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aad0baec60 msgr2=0x55aad0bbbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:48.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.940+0000 7fe2c0e62640 1 --2- 172.21.15.52:0/4027880865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aad0baec60 0x55aad0bbbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.940+0000 7fe2c0e62640 1 -- 172.21.15.52:0/4027880865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aad0bae2b0 msgr2=0x55aad0bae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:48.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.940+0000 7fe2c0e62640 1 --2- 172.21.15.52:0/4027880865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aad0bae2b0 0x55aad0bae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.940+0000 7fe2c0e62640 1 -- 172.21.15.52:0/4027880865 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aad08f07d0 con 0x55aad0bbc1e0 2024-04-02T13:55:48.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.940+0000 7fe2bbfff640 1 --2- 172.21.15.52:0/4027880865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aad0bae2b0 0x55aad0bae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:48.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.940+0000 7fe2c0e62640 1 --2- 172.21.15.52:0/4027880865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0bbc1e0 0x55aad0bbe5d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009f00 tx=0x7fe2b4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66b617d8df12544 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:48.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.940+0000 7fe2a27fc640 1 -- 172.21.15.52:0/4027880865 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4004d20 con 0x55aad0bbc1e0 2024-04-02T13:55:48.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.940+0000 7fe2a27fc640 1 -- 172.21.15.52:0/4027880865 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2b4004030 con 0x55aad0bbc1e0 2024-04-02T13:55:48.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.940+0000 7fe2a27fc640 1 -- 172.21.15.52:0/4027880865 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4005280 con 0x55aad0bbc1e0 2024-04-02T13:55:48.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.941+0000 7fe2c5586740 1 -- 172.21.15.52:0/4027880865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0bbc1e0 msgr2=0x55aad0bbe5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:48.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.941+0000 7fe2c5586740 1 --2- 172.21.15.52:0/4027880865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0bbc1e0 0x55aad0bbe5d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009f00 tx=0x7fe2b4011810 comp rx=0 tx=0).stop 2024-04-02T13:55:48.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.941+0000 7fe2c5586740 1 -- 172.21.15.52:0/4027880865 shutdown_connections 2024-04-02T13:55:48.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.941+0000 7fe2c5586740 1 --2- 172.21.15.52:0/4027880865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0bbc1e0 0x55aad0bbe5d0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.941+0000 7fe2c5586740 1 --2- 172.21.15.52:0/4027880865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aad0baec60 0x55aad0bbbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.941+0000 7fe2c5586740 1 --2- 172.21.15.52:0/4027880865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aad0bae2b0 0x55aad0bae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.941+0000 7fe2c5586740 1 -- 172.21.15.52:0/4027880865 >> 172.21.15.52:0/4027880865 conn(0x55aad0ba9000 msgr2=0x55aad0ba93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:48.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.941+0000 7fe2c5586740 1 -- 172.21.15.52:0/4027880865 shutdown_connections 2024-04-02T13:55:48.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.941+0000 7fe2c5586740 1 -- 172.21.15.52:0/4027880865 wait complete. 2024-04-02T13:55:48.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.941+0000 7fe2c5586740 1 -- start start 2024-04-02T13:55:48.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.941+0000 7fe2c5586740 1 Processor -- start 2024-04-02T13:55:48.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.942+0000 7fe2c5586740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aad0bae2b0 0x55aad0bbba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.942+0000 7fe2c5586740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aad0baec60 0x55aad0bb8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.942+0000 7fe2c5586740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0bbc1e0 0x55aad0bb92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.942+0000 7fe2c5586740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55aad0bbefa0 con 0x55aad0bbc1e0 2024-04-02T13:55:48.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.942+0000 7fe2c5586740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55aad0acdd30 con 0x55aad0baec60 2024-04-02T13:55:48.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.942+0000 7fe2c5586740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55aad0bbf120 con 0x55aad0bae2b0 2024-04-02T13:55:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.942+0000 7fe2c0e62640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0bbc1e0 0x55aad0bb92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.942+0000 7fe2c0e62640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0bbc1e0 0x55aad0bb92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34136/0 (socket says 172.21.15.52:34136) 2024-04-02T13:55:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.942+0000 7fe2c0e62640 1 -- 172.21.15.52:0/3209882837 learned_addr learned my addr 172.21.15.52:0/3209882837 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.942+0000 7fe2c5586740 10 client.0 _ll_register_callbacks cb 0x55aad0acd720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.942+0000 7fe2bbfff640 1 --2- 172.21.15.52:0/3209882837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aad0bae2b0 0x55aad0bbba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.942+0000 7fe2bb7fe640 1 --2- 172.21.15.52:0/3209882837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aad0baec60 0x55aad0bb8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.942+0000 7fe2c0e62640 1 -- 172.21.15.52:0/3209882837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aad0bae2b0 msgr2=0x55aad0bbba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.942+0000 7fe2c0e62640 1 --2- 172.21.15.52:0/3209882837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aad0bae2b0 0x55aad0bbba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.942+0000 7fe2c0e62640 1 -- 172.21.15.52:0/3209882837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aad0baec60 msgr2=0x55aad0bb8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.942+0000 7fe2c0e62640 1 --2- 172.21.15.52:0/3209882837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aad0baec60 0x55aad0bb8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.942+0000 7fe2c0e62640 1 -- 172.21.15.52:0/3209882837 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aad08263f0 con 0x55aad0bbc1e0 2024-04-02T13:55:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.942+0000 7fe2c0e62640 1 --2- 172.21.15.52:0/3209882837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0bbc1e0 0x55aad0bb92f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009270 tx=0x7fe2b4004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01d765bfc191307 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.942+0000 7fe29b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.943+0000 7fe29b7fe640 1 -- 172.21.15.52:0/3209882837 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4027020 con 0x55aad0bbc1e0 2024-04-02T13:55:48.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.943+0000 7fe29b7fe640 1 -- 172.21.15.52:0/3209882837 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe2b40041e0 con 0x55aad0bbc1e0 2024-04-02T13:55:48.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.943+0000 7fe2c5586740 1 -- 172.21.15.52:0/3209882837 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2b4004030 con 0x55aad0bbc1e0 2024-04-02T13:55:48.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.943+0000 7fe29b7fe640 1 -- 172.21.15.52:0/3209882837 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe2b4032050 con 0x55aad0bbc1e0 2024-04-02T13:55:48.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.943+0000 7fe2c5586740 10 client.16386 fetch_fsmap learned FSMap version 29 2024-04-02T13:55:48.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.943+0000 7fe2c5586740 1 -- 172.21.15.52:0/3209882837 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aad07e60f0 con 0x55aad0bbc1e0 2024-04-02T13:55:48.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.943+0000 7fe29b7fe640 1 -- 172.21.15.52:0/3209882837 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2b4004030 con 0x55aad0bbc1e0 2024-04-02T13:55:48.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.943+0000 7fe29b7fe640 1 -- 172.21.15.52:0/3209882837 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fe2b4018070 con 0x55aad0bbc1e0 2024-04-02T13:55:48.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.943+0000 7fe2c5586740 10 client.16386 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:55:48.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.943+0000 7fe2c5586740 10 client.16386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:48.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.943+0000 7fe2c5586740 20 client.16386 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:48.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.943+0000 7fe2c5586740 1 --2- 172.21.15.52:0/3209882837 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55aad0bcc0a0 0x55aad0bce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.943+0000 7fe2c5586740 4 client.16386 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:48.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.943+0000 7fe2c5586740 1 -- 172.21.15.52:0/3209882837 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aad0b1fd70 con 0x55aad0bcc0a0 2024-04-02T13:55:48.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.944+0000 7fe2bbfff640 1 --2- 172.21.15.52:0/3209882837 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55aad0bcc0a0 0x55aad0bce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.944+0000 7fe2bbfff640 1 --2- 172.21.15.52:0/3209882837 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55aad0bcc0a0 0x55aad0bce490 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ecd95ee1b28d0e1 server_cookie=e77d8d472999eb1c in_seq=0 out_seq=0 2024-04-02T13:55:48.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.944+0000 7fe29b7fe640 10 client.16386 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:48.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.945+0000 7fe29b7fe640 1 -- 172.21.15.52:0/3209882837 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aad0b1fd70 con 0x55aad0bcc0a0 2024-04-02T13:55:48.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.945+0000 7fe29b7fe640 10 client.16386 handle_command_reply: tid=0 2024-04-02T13:55:48.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.945+0000 7fe2c5586740 1 client.16386 shutdown 2024-04-02T13:55:48.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.945+0000 7fe2c5586740 1 -- 172.21.15.52:0/3209882837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0bbc1e0 msgr2=0x55aad0bb92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:48.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.945+0000 7fe2c5586740 1 --2- 172.21.15.52:0/3209882837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0bbc1e0 0x55aad0bb92f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009270 tx=0x7fe2b4004f50 comp rx=0 tx=0).stop 2024-04-02T13:55:48.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.945+0000 7fe2c5586740 1 -- 172.21.15.52:0/3209882837 shutdown_connections 2024-04-02T13:55:48.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.945+0000 7fe2c5586740 1 --2- 172.21.15.52:0/3209882837 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55aad0bcc0a0 0x55aad0bce490 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.945+0000 7fe2c5586740 1 --2- 172.21.15.52:0/3209882837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0bbc1e0 0x55aad0bb92f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.945+0000 7fe2c5586740 1 --2- 172.21.15.52:0/3209882837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aad0baec60 0x55aad0bb8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.945+0000 7fe29b7fe640 0 client.16386 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:48.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.945+0000 7fe2c5586740 1 --2- 172.21.15.52:0/3209882837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aad0bae2b0 0x55aad0bbba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.945+0000 7fe2c5586740 1 -- 172.21.15.52:0/3209882837 >> 172.21.15.52:0/3209882837 conn(0x55aad0ba9000 msgr2=0x55aad0baad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:48.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.945+0000 7fe2c5586740 1 -- 172.21.15.52:0/3209882837 shutdown_connections 2024-04-02T13:55:48.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.945+0000 7fe2c5586740 1 -- 172.21.15.52:0/3209882837 wait complete. 2024-04-02T13:55:48.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.945+0000 7fe2c5586740 20 client.16386 trim_cache size 0 max 16384 2024-04-02T13:55:48.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.965+0000 7fe2c5586740 1 Processor -- start 2024-04-02T13:55:48.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.965+0000 7fe2c5586740 1 -- start start 2024-04-02T13:55:48.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.965+0000 7fe2c5586740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aad0bae2b0 0x55aad0bb44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.965+0000 7fe2c5586740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0baec60 0x55aad0ba80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.965+0000 7fe2c5586740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aad0bbc1e0 0x55aad0bbb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.965+0000 7fe2c5586740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1acc70 con 0x55aad0baec60 2024-04-02T13:55:48.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.965+0000 7fe2c5586740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1ae7e0 con 0x55aad0bbc1e0 2024-04-02T13:55:48.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.965+0000 7fe2c5586740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1ad340 con 0x55aad0bae2b0 2024-04-02T13:55:48.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.965+0000 7fe2bbfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aad0bae2b0 0x55aad0bb44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.965+0000 7fe2bb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0baec60 0x55aad0ba80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.965+0000 7fe2bbfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aad0bae2b0 0x55aad0bb44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.52:44274/0 (socket says 172.21.15.52:44274) 2024-04-02T13:55:48.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.965+0000 7fe2bbfff640 1 -- 172.21.15.52:0/365087626 learned_addr learned my addr 172.21.15.52:0/365087626 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:48.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.965+0000 7fe2c0e62640 1 --2- 172.21.15.52:0/365087626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aad0bbc1e0 0x55aad0bbb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.965+0000 7fe2bbfff640 1 -- 172.21.15.52:0/365087626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aad0bbc1e0 msgr2=0x55aad0bbb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:48.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.965+0000 7fe2bbfff640 1 --2- 172.21.15.52:0/365087626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aad0bbc1e0 0x55aad0bbb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.965+0000 7fe2bbfff640 1 -- 172.21.15.52:0/365087626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0baec60 msgr2=0x55aad0ba80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:48.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.965+0000 7fe2bbfff640 1 --2- 172.21.15.52:0/365087626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0baec60 0x55aad0ba80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.965+0000 7fe2bbfff640 1 -- 172.21.15.52:0/365087626 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aad0ba8e30 con 0x55aad0bae2b0 2024-04-02T13:55:48.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.965+0000 7fe2bb7fe640 1 --2- 172.21.15.52:0/365087626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0baec60 0x55aad0ba80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:48.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.966+0000 7fe2bbfff640 1 --2- 172.21.15.52:0/365087626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aad0bae2b0 0x55aad0bb44d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009a10 tx=0x7fe2ac014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0beb2e84cafc542 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:48.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.966+0000 7fe2a1ffb640 1 -- 172.21.15.52:0/365087626 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac02a6e0 con 0x55aad0bae2b0 2024-04-02T13:55:48.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.966+0000 7fe2a1ffb640 1 -- 172.21.15.52:0/365087626 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2ac02f070 con 0x55aad0bae2b0 2024-04-02T13:55:48.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.966+0000 7fe2a1ffb640 1 -- 172.21.15.52:0/365087626 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac00e610 con 0x55aad0bae2b0 2024-04-02T13:55:48.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.967+0000 7fe2c5586740 1 -- 172.21.15.52:0/365087626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aad0bae2b0 msgr2=0x55aad0bb44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:48.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.967+0000 7fe2c5586740 1 --2- 172.21.15.52:0/365087626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aad0bae2b0 0x55aad0bb44d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009a10 tx=0x7fe2ac014000 comp rx=0 tx=0).stop 2024-04-02T13:55:48.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.967+0000 7fe2c5586740 1 -- 172.21.15.52:0/365087626 shutdown_connections 2024-04-02T13:55:48.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.967+0000 7fe2c5586740 1 --2- 172.21.15.52:0/365087626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aad0bbc1e0 0x55aad0bbb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.967+0000 7fe2c5586740 1 --2- 172.21.15.52:0/365087626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0baec60 0x55aad0ba80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.967+0000 7fe2c5586740 1 --2- 172.21.15.52:0/365087626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aad0bae2b0 0x55aad0bb44d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.967+0000 7fe2c5586740 1 -- 172.21.15.52:0/365087626 >> 172.21.15.52:0/365087626 conn(0x55aad0ba9000 msgr2=0x55aad0ba79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:48.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.967+0000 7fe2c5586740 1 -- 172.21.15.52:0/365087626 shutdown_connections 2024-04-02T13:55:48.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.967+0000 7fe2c5586740 1 -- 172.21.15.52:0/365087626 wait complete. 2024-04-02T13:55:48.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.967+0000 7fe2c5586740 1 -- start start 2024-04-02T13:55:48.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.967+0000 7fe2c5586740 1 Processor -- start 2024-04-02T13:55:48.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.968+0000 7fe2c5586740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0bae2b0 0x55aad0bc4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.968+0000 7fe2c5586740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aad0baec60 0x55aad0bb5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.968+0000 7fe2c5586740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aad0bbc1e0 0x55aad0bb5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.968+0000 7fe2c5586740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1ae0e0 con 0x55aad0bae2b0 2024-04-02T13:55:48.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.968+0000 7fe2c5586740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1ada10 con 0x55aad0baec60 2024-04-02T13:55:48.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.968+0000 7fe2c5586740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1aac50 con 0x55aad0bbc1e0 2024-04-02T13:55:48.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.968+0000 7fe2c5586740 10 client.0 _ll_register_callbacks cb 0x7fe2b4009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:48.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.968+0000 7fe2bbfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0bae2b0 0x55aad0bc4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.968+0000 7fe2bb7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aad0baec60 0x55aad0bb5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.968+0000 7fe2bbfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0bae2b0 0x55aad0bc4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34166/0 (socket says 172.21.15.52:34166) 2024-04-02T13:55:48.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.968+0000 7fe2bbfff640 1 -- 172.21.15.52:0/1491803816 learned_addr learned my addr 172.21.15.52:0/1491803816 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:55:48.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.968+0000 7fe2c0e62640 1 --2- 172.21.15.52:0/1491803816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aad0bbc1e0 0x55aad0bb5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.968+0000 7fe2bbfff640 1 -- 172.21.15.52:0/1491803816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aad0bbc1e0 msgr2=0x55aad0bb5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:48.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.968+0000 7fe2bbfff640 1 --2- 172.21.15.52:0/1491803816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aad0bbc1e0 0x55aad0bb5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.968+0000 7fe2bbfff640 1 -- 172.21.15.52:0/1491803816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aad0baec60 msgr2=0x55aad0bb5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:48.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.968+0000 7fe2bbfff640 1 --2- 172.21.15.52:0/1491803816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aad0baec60 0x55aad0bb5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.968+0000 7fe2bbfff640 1 -- 172.21.15.52:0/1491803816 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aad0bad9c0 con 0x55aad0bae2b0 2024-04-02T13:55:48.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.968+0000 7fe2bb7fe640 1 --2- 172.21.15.52:0/1491803816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aad0baec60 0x55aad0bb5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:48.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.968+0000 7fe2bbfff640 1 --2- 172.21.15.52:0/1491803816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0bae2b0 0x55aad0bc4600 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac004440 tx=0x7fe2ac004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ba4ae7742e715 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:48.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.968+0000 7fe29b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:55:48.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.969+0000 7fe29b7fe640 1 -- 172.21.15.52:0/1491803816 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac037070 con 0x55aad0bae2b0 2024-04-02T13:55:48.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.969+0000 7fe29b7fe640 1 -- 172.21.15.52:0/1491803816 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe2ac01fe00 con 0x55aad0bae2b0 2024-04-02T13:55:48.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.969+0000 7fe2c5586740 1 -- 172.21.15.52:0/1491803816 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2ac02f070 con 0x55aad0bae2b0 2024-04-02T13:55:48.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.969+0000 7fe29b7fe640 1 -- 172.21.15.52:0/1491803816 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe2ac045050 con 0x55aad0bae2b0 2024-04-02T13:55:48.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.969+0000 7fe2c5586740 10 client.16392 fetch_fsmap learned FSMap version 29 2024-04-02T13:55:48.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.969+0000 7fe2c5586740 1 -- 172.21.15.52:0/1491803816 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aad0c31f30 con 0x55aad0bae2b0 2024-04-02T13:55:48.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.969+0000 7fe29b7fe640 1 -- 172.21.15.52:0/1491803816 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2ac02f070 con 0x55aad0bae2b0 2024-04-02T13:55:48.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.969+0000 7fe29b7fe640 1 -- 172.21.15.52:0/1491803816 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fe2ac0046c0 con 0x55aad0bae2b0 2024-04-02T13:55:48.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.969+0000 7fe2c5586740 10 client.16392 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:55:48.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.969+0000 7fe2c5586740 10 client.16392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:48.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.969+0000 7fe2c5586740 20 client.16392 populate_metadata read hostname 'smithi052' 2024-04-02T13:55:48.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.969+0000 7fe2c5586740 1 --2- 172.21.15.52:0/1491803816 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55aad0c32680 0x55aad0c34aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:48.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.969+0000 7fe2c5586740 4 client.16392 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:55:48.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.969+0000 7fe2c5586740 1 -- 172.21.15.52:0/1491803816 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aad0bb9680 con 0x55aad0c32680 2024-04-02T13:55:48.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.969+0000 7fe2bb7fe640 1 --2- 172.21.15.52:0/1491803816 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55aad0c32680 0x55aad0c34aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:48.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.970+0000 7fe2bb7fe640 1 --2- 172.21.15.52:0/1491803816 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55aad0c32680 0x55aad0c34aa0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82b488efaa447558 server_cookie=b4f603e6b31f1c96 in_seq=0 out_seq=0 2024-04-02T13:55:48.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.970+0000 7fe29b7fe640 10 client.16392 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:48.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.970+0000 7fe29b7fe640 1 -- 172.21.15.52:0/1491803816 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55aad0bb9680 con 0x55aad0c32680 2024-04-02T13:55:48.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.970+0000 7fe29b7fe640 10 client.16392 handle_command_reply: tid=0 2024-04-02T13:55:48.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.970+0000 7fe2c5586740 1 client.16392 shutdown 2024-04-02T13:55:48.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.970+0000 7fe2c5586740 1 -- 172.21.15.52:0/1491803816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0bae2b0 msgr2=0x55aad0bc4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:48.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.970+0000 7fe2c5586740 1 --2- 172.21.15.52:0/1491803816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0bae2b0 0x55aad0bc4600 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac004440 tx=0x7fe2ac004470 comp rx=0 tx=0).stop 2024-04-02T13:55:48.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.971+0000 7fe2c5586740 1 -- 172.21.15.52:0/1491803816 shutdown_connections 2024-04-02T13:55:48.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.971+0000 7fe2c5586740 1 --2- 172.21.15.52:0/1491803816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aad0bbc1e0 0x55aad0bb5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.971+0000 7fe2c5586740 1 --2- 172.21.15.52:0/1491803816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aad0baec60 0x55aad0bb5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.971+0000 7fe2c5586740 1 --2- 172.21.15.52:0/1491803816 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55aad0c32680 0x55aad0c34aa0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.971+0000 7fe2c5586740 1 --2- 172.21.15.52:0/1491803816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aad0bae2b0 0x55aad0bc4600 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.971+0000 7fe29b7fe640 0 client.16392 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:55:48.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.971+0000 7fe2c5586740 1 -- 172.21.15.52:0/1491803816 >> 172.21.15.52:0/1491803816 conn(0x55aad0ba9000 msgr2=0x55aad0ba8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:48.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.971+0000 7fe2c5586740 1 -- 172.21.15.52:0/1491803816 shutdown_connections 2024-04-02T13:55:48.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.971+0000 7fe2c5586740 1 -- 172.21.15.52:0/1491803816 wait complete. 2024-04-02T13:55:48.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.971+0000 7fe2c5586740 20 client.16392 trim_cache size 0 max 16384 2024-04-02T13:55:48.979 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:55:48.979 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "scrub active (2274 inodes in the stack)", 2024-04-02T13:55:48.979 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-04-02T13:55:48.980 INFO:teuthology.orchestra.run.smithi052.stdout: "1ab104e5-8378-4f0e-96c6-af3968f2fce3": { 2024-04-02T13:55:48.980 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-04-02T13:55:48.980 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "1ab104e5-8378-4f0e-96c6-af3968f2fce3", 2024-04-02T13:55:48.980 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-04-02T13:55:48.980 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:55:48.980 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:55:48.980 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:55:48.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.971+0000 7fe2c28c5640 1 -- 172.21.15.52:0/3958112646 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe2a40a3ec0 msgr2=0x7fe2a40a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:48.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.971+0000 7fe2c28c5640 1 --2- 172.21.15.52:0/3958112646 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe2a40a3ec0 0x7fe2a40a6380 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009b50 tx=0x7fe2ac005f90 comp rx=0 tx=0).stop 2024-04-02T13:55:48.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.971+0000 7fe2c28c5640 1 -- 172.21.15.52:0/3958112646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2bc101330 msgr2=0x7fe2bc198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:48.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.971+0000 7fe2c28c5640 1 --2- 172.21.15.52:0/3958112646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2bc101330 0x7fe2bc198910 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009230 tx=0x7fe2b002cb30 comp rx=0 tx=0).stop 2024-04-02T13:55:48.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.972+0000 7fe2c28c5640 1 -- 172.21.15.52:0/3958112646 shutdown_connections 2024-04-02T13:55:48.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.972+0000 7fe2c28c5640 1 --2- 172.21.15.52:0/3958112646 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe2a40a3ec0 0x7fe2a40a6380 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.972+0000 7fe2c28c5640 1 --2- 172.21.15.52:0/3958112646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2bc106980 0x7fe2bc19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.972+0000 7fe2c28c5640 1 --2- 172.21.15.52:0/3958112646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2bc101330 0x7fe2bc198910 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.972+0000 7fe2c28c5640 1 --2- 172.21.15.52:0/3958112646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2bc100970 0x7fe2bc1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:48.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.972+0000 7fe2c28c5640 1 -- 172.21.15.52:0/3958112646 >> 172.21.15.52:0/3958112646 conn(0x7fe2bc077b00 msgr2=0x7fe2bc068fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:48.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.972+0000 7fe2c28c5640 1 -- 172.21.15.52:0/3958112646 shutdown_connections 2024-04-02T13:55:48.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:55:48.972+0000 7fe2c28c5640 1 -- 172.21.15.52:0/3958112646 wait complete. 2024-04-02T13:55:48.987 INFO:tasks.cephfs.filesystem:scrub status for tag:1ab104e5-8378-4f0e-96c6-af3968f2fce3 - {'path': '/', 'tag': '1ab104e5-8378-4f0e-96c6-af3968f2fce3', 'options': 'recursive,force'} 2024-04-02T13:55:49.172 INFO:tasks.workunit.client.0.smithi052.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-04-02T13:55:49.172 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether ln -s works... yes 2024-04-02T13:55:49.177 INFO:tasks.workunit.client.0.smithi052.stdout:checking the maximum length of command line arguments... 1572864 2024-04-02T13:55:49.180 INFO:tasks.workunit.client.0.smithi052.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-02T13:55:49.180 INFO:tasks.workunit.client.0.smithi052.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-02T13:55:49.180 INFO:tasks.workunit.client.0.smithi052.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-04-02T13:55:49.180 INFO:tasks.workunit.client.0.smithi052.stdout:checking for objdump... objdump 2024-04-02T13:55:49.181 INFO:tasks.workunit.client.0.smithi052.stdout:checking how to recognize dependent libraries... pass_all 2024-04-02T13:55:49.181 INFO:tasks.workunit.client.0.smithi052.stdout:checking for dlltool... no 2024-04-02T13:55:49.181 INFO:tasks.workunit.client.0.smithi052.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-02T13:55:49.181 INFO:tasks.workunit.client.0.smithi052.stdout:checking for ar... ar 2024-04-02T13:55:49.276 INFO:tasks.workunit.client.0.smithi052.stdout:checking for archiver @FILE support... @ 2024-04-02T13:55:49.276 INFO:tasks.workunit.client.0.smithi052.stdout:checking for strip... strip 2024-04-02T13:55:49.277 INFO:tasks.workunit.client.0.smithi052.stdout:checking for ranlib... ranlib 2024-04-02T13:55:49.277 INFO:tasks.workunit.client.0.smithi052.stdout:checking for gawk... gawk 2024-04-02T13:55:49.388 INFO:tasks.workunit.client.0.smithi052.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-04-02T13:55:49.389 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sysroot... no 2024-04-02T13:55:49.405 INFO:tasks.workunit.client.0.smithi052.stdout:checking for a working dd... /usr/bin/dd 2024-04-02T13:55:49.421 INFO:tasks.workunit.client.0.smithi052.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-04-02T13:55:49.456 INFO:tasks.workunit.client.0.smithi052.stdout:checking for mt... no 2024-04-02T13:55:49.471 INFO:tasks.workunit.client.0.smithi052.stdout:checking if : is a manifest tool... no 2024-04-02T13:55:49.577 INFO:tasks.workunit.client.0.smithi052.stdout:checking how to run the C preprocessor... gcc -E 2024-04-02T13:55:49.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:49 smithi189 ceph-mon[36683]: pgmap v72: 129 pgs: 129 active+clean; 52 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 243 op/s 2024-04-02T13:55:49.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:49 smithi189 ceph-mon[36683]: scrub queued for path: / 2024-04-02T13:55:49.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:49 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:49.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:49 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:55:49.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:49 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:55:49.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:49 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:49.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:49 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:49.801 INFO:tasks.workunit.client.0.smithi052.stdout:checking for ANSI C header files... yes 2024-04-02T13:55:49.837 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/types.h... yes 2024-04-02T13:55:49.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:49 smithi052 ceph-mon[32104]: pgmap v72: 129 pgs: 129 active+clean; 52 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 243 op/s 2024-04-02T13:55:49.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:49 smithi052 ceph-mon[32104]: scrub queued for path: / 2024-04-02T13:55:49.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:49 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:49.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:49 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:55:49.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:49 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:55:49.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:49 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:49.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:49 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:49.874 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/stat.h... yes 2024-04-02T13:55:49.913 INFO:tasks.workunit.client.0.smithi052.stdout:checking for stdlib.h... yes 2024-04-02T13:55:49.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:49 smithi088 ceph-mon[36528]: pgmap v72: 129 pgs: 129 active+clean; 52 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 243 op/s 2024-04-02T13:55:49.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:49 smithi088 ceph-mon[36528]: scrub queued for path: / 2024-04-02T13:55:49.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:49 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:49.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:49 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:55:49.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:49 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:55:49.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:49 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:49.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:49 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:49.952 INFO:tasks.workunit.client.0.smithi052.stdout:checking for string.h... yes 2024-04-02T13:55:49.998 INFO:tasks.workunit.client.0.smithi052.stdout:checking for memory.h... yes 2024-04-02T13:55:50.037 INFO:tasks.workunit.client.0.smithi052.stdout:checking for strings.h... yes 2024-04-02T13:55:50.077 INFO:tasks.workunit.client.0.smithi052.stdout:checking for inttypes.h... yes 2024-04-02T13:55:50.117 INFO:tasks.workunit.client.0.smithi052.stdout:checking for stdint.h... yes 2024-04-02T13:55:50.158 INFO:tasks.workunit.client.0.smithi052.stdout:checking for unistd.h... yes 2024-04-02T13:55:50.208 INFO:tasks.workunit.client.0.smithi052.stdout:checking for dlfcn.h... yes 2024-04-02T13:55:50.213 INFO:tasks.workunit.client.0.smithi052.stdout:checking for objdir... .libs 2024-04-02T13:55:50.352 INFO:tasks.workunit.client.0.smithi052.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-02T13:55:50.352 INFO:tasks.workunit.client.0.smithi052.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-02T13:55:50.399 INFO:tasks.workunit.client.0.smithi052.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-02T13:55:50.447 INFO:tasks.workunit.client.0.smithi052.stdout:checking if gcc static flag -static works... no 2024-04-02T13:55:50.516 INFO:tasks.workunit.client.0.smithi052.stdout:checking if gcc supports -c -o file.o... yes 2024-04-02T13:55:50.516 INFO:tasks.workunit.client.0.smithi052.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-02T13:55:50.522 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-02T13:55:50.580 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether -lc should be explicitly linked in... no 2024-04-02T13:55:50.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:50 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:55:50.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:50 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:50.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:50 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:55:50.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:50 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:50.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:50 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:55:50.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:50 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:51.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:51 smithi189 ceph-mon[36683]: pgmap v73: 129 pgs: 129 active+clean; 52 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 243 op/s 2024-04-02T13:55:51.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:51 smithi052 ceph-mon[32104]: pgmap v73: 129 pgs: 129 active+clean; 52 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 243 op/s 2024-04-02T13:55:51.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:51 smithi088 ceph-mon[36528]: pgmap v73: 129 pgs: 129 active+clean; 52 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 243 op/s 2024-04-02T13:55:53.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:53 smithi189 ceph-mon[36683]: pgmap v74: 129 pgs: 129 active+clean; 53 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2024-04-02T13:55:53.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:53 smithi052 ceph-mon[32104]: pgmap v74: 129 pgs: 129 active+clean; 53 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2024-04-02T13:55:53.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:53 smithi088 ceph-mon[36528]: pgmap v74: 129 pgs: 129 active+clean; 53 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2024-04-02T13:55:54.854 INFO:tasks.workunit.client.0.smithi052.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-02T13:55:54.854 INFO:tasks.workunit.client.0.smithi052.stdout:checking how to hardcode library paths into programs... immediate 2024-04-02T13:55:54.855 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether stripping libraries is possible... yes 2024-04-02T13:55:54.855 INFO:tasks.workunit.client.0.smithi052.stdout:checking if libtool supports shared libraries... yes 2024-04-02T13:55:54.855 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether to build shared libraries... yes 2024-04-02T13:55:54.855 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether to build static libraries... yes 2024-04-02T13:55:54.863 INFO:tasks.workunit.client.0.smithi052.stdout:checking for gcc... (cached) gcc 2024-04-02T13:55:54.903 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-02T13:55:54.903 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-02T13:55:54.903 INFO:tasks.workunit.client.0.smithi052.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-02T13:55:54.903 INFO:tasks.workunit.client.0.smithi052.stdout:checking for gmake... /usr/bin/gmake 2024-04-02T13:55:54.904 INFO:tasks.workunit.client.0.smithi052.stderr:./configure: line 11968: test: $(SHELL): binary operator expected 2024-04-02T13:55:54.904 INFO:tasks.workunit.client.0.smithi052.stdout:checking for tar... /bin/tar 2024-04-02T13:55:54.904 INFO:tasks.workunit.client.0.smithi052.stdout:checking for gzip... /bin/gzip 2024-04-02T13:55:54.934 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether gcc -MM is supported... yes 2024-04-02T13:55:54.934 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sort... /bin/sort 2024-04-02T13:55:54.934 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether ln -s works... yes 2024-04-02T13:55:54.934 INFO:tasks.workunit.client.0.smithi052.stdout:checking for rpm... /bin/rpm 2024-04-02T13:55:54.938 INFO:tasks.workunit.client.0.smithi052.stdout:checking for rpmbuild... no 2024-04-02T13:55:54.938 INFO:tasks.workunit.client.0.smithi052.stdout:checking for ANSI C header files... (cached) yes 2024-04-02T13:55:55.213 INFO:tasks.workunit.client.0.smithi052.stdout:checking assert.h usability... yes 2024-04-02T13:55:55.230 INFO:tasks.workunit.client.0.smithi052.stdout:checking assert.h presence... yes 2024-04-02T13:55:55.230 INFO:tasks.workunit.client.0.smithi052.stdout:checking for assert.h... yes 2024-04-02T13:55:55.285 INFO:tasks.workunit.client.0.smithi052.stdout:checking bstring.h usability... no 2024-04-02T13:55:55.315 INFO:tasks.workunit.client.0.smithi052.stdout:checking bstring.h presence... no 2024-04-02T13:55:55.315 INFO:tasks.workunit.client.0.smithi052.stdout:checking for bstring.h... no 2024-04-02T13:55:55.354 INFO:tasks.workunit.client.0.smithi052.stdout:checking libgen.h usability... yes 2024-04-02T13:55:55.371 INFO:tasks.workunit.client.0.smithi052.stdout:checking libgen.h presence... yes 2024-04-02T13:55:55.371 INFO:tasks.workunit.client.0.smithi052.stdout:checking for libgen.h... yes 2024-04-02T13:55:55.429 INFO:tasks.workunit.client.0.smithi052.stdout:checking dirent.h usability... yes 2024-04-02T13:55:55.446 INFO:tasks.workunit.client.0.smithi052.stdout:checking dirent.h presence... yes 2024-04-02T13:55:55.447 INFO:tasks.workunit.client.0.smithi052.stdout:checking for dirent.h... yes 2024-04-02T13:55:55.493 INFO:tasks.workunit.client.0.smithi052.stdout:checking errno.h usability... yes 2024-04-02T13:55:55.510 INFO:tasks.workunit.client.0.smithi052.stdout:checking errno.h presence... yes 2024-04-02T13:55:55.510 INFO:tasks.workunit.client.0.smithi052.stdout:checking for errno.h... yes 2024-04-02T13:55:55.575 INFO:tasks.workunit.client.0.smithi052.stdout:checking malloc.h usability... yes 2024-04-02T13:55:55.600 INFO:tasks.workunit.client.0.smithi052.stdout:checking malloc.h presence... yes 2024-04-02T13:55:55.600 INFO:tasks.workunit.client.0.smithi052.stdout:checking for malloc.h... yes 2024-04-02T13:55:55.668 INFO:tasks.workunit.client.0.smithi052.stdout:checking uuid.h usability... no 2024-04-02T13:55:55.697 INFO:tasks.workunit.client.0.smithi052.stdout:checking uuid.h presence... no 2024-04-02T13:55:55.697 INFO:tasks.workunit.client.0.smithi052.stdout:checking for uuid.h... no 2024-04-02T13:55:55.742 INFO:tasks.workunit.client.0.smithi052.stdout:checking uuid/uuid.h usability... yes 2024-04-02T13:55:55.761 INFO:tasks.workunit.client.0.smithi052.stdout:checking uuid/uuid.h presence... yes 2024-04-02T13:55:55.761 INFO:tasks.workunit.client.0.smithi052.stdout:checking for uuid/uuid.h... yes 2024-04-02T13:55:55.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:55 smithi189 ceph-mon[36683]: pgmap v75: 129 pgs: 129 active+clean; 53 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 783 KiB/s wr, 157 op/s 2024-04-02T13:55:55.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:55 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T13:55:55.813 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/uuid.h usability... no 2024-04-02T13:55:55.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:55 smithi052 ceph-mon[32104]: pgmap v75: 129 pgs: 129 active+clean; 53 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 783 KiB/s wr, 157 op/s 2024-04-02T13:55:55.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:55 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T13:55:55.842 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/uuid.h presence... no 2024-04-02T13:55:55.842 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/uuid.h... no 2024-04-02T13:55:55.880 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/file.h usability... yes 2024-04-02T13:55:55.897 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/file.h presence... yes 2024-04-02T13:55:55.897 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/file.h... yes 2024-04-02T13:55:55.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:55 smithi088 ceph-mon[36528]: pgmap v75: 129 pgs: 129 active+clean; 53 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 783 KiB/s wr, 157 op/s 2024-04-02T13:55:55.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:55 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T13:55:55.955 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/fcntl.h usability... yes 2024-04-02T13:55:55.972 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/fcntl.h presence... yes 2024-04-02T13:55:55.972 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/fcntl.h... yes 2024-04-02T13:55:56.023 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/syssgi.h usability... no 2024-04-02T13:55:56.052 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/syssgi.h presence... no 2024-04-02T13:55:56.052 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/syssgi.h... no 2024-04-02T13:55:56.139 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/param.h usability... yes 2024-04-02T13:55:56.159 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/param.h presence... yes 2024-04-02T13:55:56.159 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/param.h... yes 2024-04-02T13:55:56.173 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/stat.h... (cached) yes 2024-04-02T13:55:56.226 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/statvfs.h usability... yes 2024-04-02T13:55:56.243 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/statvfs.h presence... yes 2024-04-02T13:55:56.243 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/statvfs.h... yes 2024-04-02T13:55:56.289 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/time.h usability... yes 2024-04-02T13:55:56.306 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/time.h presence... yes 2024-04-02T13:55:56.307 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/time.h... yes 2024-04-02T13:55:56.352 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/ioctl.h usability... yes 2024-04-02T13:55:56.369 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/ioctl.h presence... yes 2024-04-02T13:55:56.369 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/ioctl.h... yes 2024-04-02T13:55:56.421 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/wait.h usability... yes 2024-04-02T13:55:56.439 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/wait.h presence... yes 2024-04-02T13:55:56.440 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/wait.h... yes 2024-04-02T13:55:56.453 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/types.h... (cached) yes 2024-04-02T13:55:56.457 INFO:tasks.workunit.client.0.smithi052.stdout:checking for strings.h... (cached) yes 2024-04-02T13:55:56.494 INFO:tasks.workunit.client.0.smithi052.stdout:checking err.h usability... yes 2024-04-02T13:55:56.511 INFO:tasks.workunit.client.0.smithi052.stdout:checking err.h presence... yes 2024-04-02T13:55:56.511 INFO:tasks.workunit.client.0.smithi052.stdout:checking for err.h... yes 2024-04-02T13:55:56.556 INFO:tasks.workunit.client.0.smithi052.stdout:checking linux/falloc.h usability... yes 2024-04-02T13:55:56.572 INFO:tasks.workunit.client.0.smithi052.stdout:checking linux/falloc.h presence... yes 2024-04-02T13:55:56.572 INFO:tasks.workunit.client.0.smithi052.stdout:checking for linux/falloc.h... yes 2024-04-02T13:55:56.622 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-04-02T13:55:56.651 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-04-02T13:55:56.651 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-02T13:55:56.695 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/fs/xfs_itable.h usability... no 2024-04-02T13:55:56.726 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/fs/xfs_itable.h presence... no 2024-04-02T13:55:56.726 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-02T13:55:56.769 INFO:tasks.workunit.client.0.smithi052.stdout:checking xfs/platform_defs.h usability... no 2024-04-02T13:55:56.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:56 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T13:55:56.799 INFO:tasks.workunit.client.0.smithi052.stdout:checking xfs/platform_defs.h presence... no 2024-04-02T13:55:56.799 INFO:tasks.workunit.client.0.smithi052.stdout:checking for xfs/platform_defs.h... no 2024-04-02T13:55:56.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:56 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T13:55:56.843 INFO:tasks.workunit.client.0.smithi052.stdout:checking btrfs/ioctl.h usability... no 2024-04-02T13:55:56.876 INFO:tasks.workunit.client.0.smithi052.stdout:checking btrfs/ioctl.h presence... no 2024-04-02T13:55:56.876 INFO:tasks.workunit.client.0.smithi052.stdout:checking for btrfs/ioctl.h... no 2024-04-02T13:55:56.919 INFO:tasks.workunit.client.0.smithi052.stdout:checking cifs/ioctl.h usability... no 2024-04-02T13:55:56.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:56 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T13:55:56.947 INFO:tasks.workunit.client.0.smithi052.stdout:checking cifs/ioctl.h presence... no 2024-04-02T13:55:56.947 INFO:tasks.workunit.client.0.smithi052.stdout:checking for cifs/ioctl.h... no 2024-04-02T13:55:56.986 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/mman.h usability... yes 2024-04-02T13:55:57.003 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/mman.h presence... yes 2024-04-02T13:55:57.003 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/mman.h... yes 2024-04-02T13:55:57.042 INFO:tasks.workunit.client.0.smithi052.stdout:checking for xfs/xfs_log_format.h... no 2024-04-02T13:55:57.097 INFO:tasks.workunit.client.0.smithi052.stdout:checking for uuid_compare... no 2024-04-02T13:55:57.139 INFO:tasks.workunit.client.0.smithi052.stdout:checking for uuid_compare in -luuid... yes 2024-04-02T13:55:57.203 INFO:tasks.workunit.client.0.smithi052.stdout:checking for xfs/xfs.h... yes 2024-04-02T13:55:57.239 INFO:tasks.workunit.client.0.smithi052.stdout:checking for xfs/libxfs.h... no 2024-04-02T13:55:57.271 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-02T13:55:57.309 INFO:tasks.workunit.client.0.smithi052.stdout:checking for xfs/xqm.h... yes 2024-04-02T13:55:57.361 INFO:tasks.workunit.client.0.smithi052.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-02T13:55:57.397 INFO:tasks.workunit.client.0.smithi052.stdout:checking xfs/handle.h usability... yes 2024-04-02T13:55:57.413 INFO:tasks.workunit.client.0.smithi052.stdout:checking xfs/handle.h presence... yes 2024-04-02T13:55:57.413 INFO:tasks.workunit.client.0.smithi052.stdout:checking for xfs/handle.h... yes 2024-04-02T13:55:57.453 INFO:tasks.workunit.client.0.smithi052.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-02T13:55:57.523 INFO:tasks.workunit.client.0.smithi052.stdout:checking attr/attributes.h usability... yes 2024-04-02T13:55:57.539 INFO:tasks.workunit.client.0.smithi052.stdout:checking attr/attributes.h presence... yes 2024-04-02T13:55:57.539 INFO:tasks.workunit.client.0.smithi052.stdout:checking for attr/attributes.h... yes 2024-04-02T13:55:57.600 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/attributes.h usability... no 2024-04-02T13:55:57.634 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/attributes.h presence... no 2024-04-02T13:55:57.634 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/attributes.h... no 2024-04-02T13:55:57.677 INFO:tasks.workunit.client.0.smithi052.stdout:checking for attr_list in -lattr... yes 2024-04-02T13:55:57.720 INFO:tasks.workunit.client.0.smithi052.stdout:checking for attr_set in -lattr... yes 2024-04-02T13:55:57.756 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/acl.h usability... yes 2024-04-02T13:55:57.777 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/acl.h presence... yes 2024-04-02T13:55:57.777 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/acl.h... yes 2024-04-02T13:55:57.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:57 smithi189 ceph-mon[36683]: pgmap v76: 129 pgs: 129 active+clean; 66 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 334 op/s 2024-04-02T13:55:57.814 INFO:tasks.workunit.client.0.smithi052.stdout:checking acl/libacl.h usability... yes 2024-04-02T13:55:57.837 INFO:tasks.workunit.client.0.smithi052.stdout:checking acl/libacl.h presence... yes 2024-04-02T13:55:57.837 INFO:tasks.workunit.client.0.smithi052.stdout:checking for acl/libacl.h... yes 2024-04-02T13:55:57.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:57 smithi052 ceph-mon[32104]: pgmap v76: 129 pgs: 129 active+clean; 66 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 334 op/s 2024-04-02T13:55:57.881 INFO:tasks.workunit.client.0.smithi052.stdout:checking for acl_init in -lacl... yes 2024-04-02T13:55:57.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:57 smithi088 ceph-mon[36528]: pgmap v76: 129 pgs: 129 active+clean; 66 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 334 op/s 2024-04-02T13:55:57.947 INFO:tasks.workunit.client.0.smithi052.stdout:checking gdbm-ndbm.h usability... no 2024-04-02T13:55:57.976 INFO:tasks.workunit.client.0.smithi052.stdout:checking gdbm-ndbm.h presence... no 2024-04-02T13:55:57.976 INFO:tasks.workunit.client.0.smithi052.stdout:checking for gdbm-ndbm.h... no 2024-04-02T13:55:58.014 INFO:tasks.workunit.client.0.smithi052.stdout:checking gdbm/ndbm.h usability... no 2024-04-02T13:55:58.042 INFO:tasks.workunit.client.0.smithi052.stdout:checking gdbm/ndbm.h presence... no 2024-04-02T13:55:58.042 INFO:tasks.workunit.client.0.smithi052.stdout:checking for gdbm/ndbm.h... no 2024-04-02T13:55:58.079 INFO:tasks.workunit.client.0.smithi052.stdout:checking ndbm.h usability... no 2024-04-02T13:55:58.108 INFO:tasks.workunit.client.0.smithi052.stdout:checking ndbm.h presence... no 2024-04-02T13:55:58.108 INFO:tasks.workunit.client.0.smithi052.stdout:checking for ndbm.h... no 2024-04-02T13:55:58.139 INFO:tasks.workunit.client.0.smithi052.stdout:checking libaio.h usability... yes 2024-04-02T13:55:58.157 INFO:tasks.workunit.client.0.smithi052.stdout:checking libaio.h presence... yes 2024-04-02T13:55:58.157 INFO:tasks.workunit.client.0.smithi052.stdout:checking for libaio.h... yes 2024-04-02T13:55:58.208 INFO:tasks.workunit.client.0.smithi052.stdout:checking for dm_make_handle in -ldm... no 2024-04-02T13:55:58.208 INFO:tasks.workunit.client.0.smithi052.stdout: 2024-04-02T13:55:58.208 INFO:tasks.workunit.client.0.smithi052.stdout:WARNING: could not find a valid DMAPI base library. 2024-04-02T13:55:58.208 INFO:tasks.workunit.client.0.smithi052.stdout:If you want DMAPI support please install the data migration 2024-04-02T13:55:58.208 INFO:tasks.workunit.client.0.smithi052.stdout:API (dmapi) library package. Alternatively, run "make install" 2024-04-02T13:55:58.208 INFO:tasks.workunit.client.0.smithi052.stdout:from the dmapi source. 2024-04-02T13:55:58.208 INFO:tasks.workunit.client.0.smithi052.stdout: 2024-04-02T13:55:58.238 INFO:tasks.workunit.client.0.smithi052.stdout:checking linux/fiemap.h usability... yes 2024-04-02T13:55:58.254 INFO:tasks.workunit.client.0.smithi052.stdout:checking linux/fiemap.h presence... yes 2024-04-02T13:55:58.254 INFO:tasks.workunit.client.0.smithi052.stdout:checking for linux/fiemap.h... yes 2024-04-02T13:55:58.291 INFO:tasks.workunit.client.0.smithi052.stdout:checking for fallocate... yes 2024-04-02T13:55:58.338 INFO:tasks.workunit.client.0.smithi052.stdout:checking for open_by_handle_at... yes 2024-04-02T13:55:58.383 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/prctl.h usability... yes 2024-04-02T13:55:58.400 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/prctl.h presence... yes 2024-04-02T13:55:58.400 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/prctl.h... yes 2024-04-02T13:55:58.432 INFO:tasks.workunit.client.0.smithi052.stdout:checking linux/fs.h usability... yes 2024-04-02T13:55:58.449 INFO:tasks.workunit.client.0.smithi052.stdout:checking linux/fs.h presence... yes 2024-04-02T13:55:58.450 INFO:tasks.workunit.client.0.smithi052.stdout:checking for linux/fs.h... yes 2024-04-02T13:55:58.512 INFO:tasks.workunit.client.0.smithi052.stdout:checking btrfsutil.h usability... no 2024-04-02T13:55:58.541 INFO:tasks.workunit.client.0.smithi052.stdout:checking btrfsutil.h presence... no 2024-04-02T13:55:58.541 INFO:tasks.workunit.client.0.smithi052.stdout:checking for btrfsutil.h... no 2024-04-02T13:55:58.597 INFO:tasks.workunit.client.0.smithi052.stdout:checking for copy_file_range... yes 2024-04-02T13:55:58.643 INFO:tasks.workunit.client.0.smithi052.stdout:checking for renameat2... yes 2024-04-02T13:55:58.662 INFO:tasks.workunit.client.0.smithi052.stdout:configure: creating ./config.status 2024-04-02T13:55:58.993 INFO:tasks.workunit.client.0.smithi052.stdout:config.status: creating include/builddefs 2024-04-02T13:55:59.006 INFO:tasks.workunit.client.0.smithi052.stdout:config.status: creating include/config.h 2024-04-02T13:55:59.016 INFO:tasks.workunit.client.0.smithi052.stdout:config.status: executing libtool commands 2024-04-02T13:55:59.098 INFO:tasks.workunit.client.0.smithi052.stdout:Building include 2024-04-02T13:55:59.098 INFO:tasks.workunit.client.0.smithi052.stdout:Building lib 2024-04-02T13:55:59.098 INFO:tasks.workunit.client.0.smithi052.stdout:Building tests 2024-04-02T13:55:59.103 INFO:tasks.workunit.client.0.smithi052.stdout: [LTDEP] 2024-04-02T13:55:59.273 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] dataascii.lo 2024-04-02T13:55:59.273 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] databin.lo 2024-04-02T13:55:59.273 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] datapid.lo 2024-04-02T13:55:59.273 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] file_lock.lo 2024-04-02T13:55:59.385 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] forker.lo 2024-04-02T13:55:59.398 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] pattern.lo 2024-04-02T13:55:59.434 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] open_flags.lo 2024-04-02T13:55:59.440 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] random_range.lo 2024-04-02T13:55:59.527 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] string_to_tokens.lo 2024-04-02T13:55:59.541 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] str_to_bytes.lo 2024-04-02T13:55:59.598 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] tlibio.lo 2024-04-02T13:55:59.642 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] write_log.lo 2024-04-02T13:55:59.684 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] random.lo 2024-04-02T13:55:59.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:55:59 smithi189 ceph-mon[36683]: pgmap v77: 129 pgs: 129 active+clean; 66 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.3 MiB/s wr, 194 op/s 2024-04-02T13:55:59.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:55:59 smithi052 ceph-mon[32104]: pgmap v77: 129 pgs: 129 active+clean; 66 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.3 MiB/s wr, 194 op/s 2024-04-02T13:55:59.885 INFO:tasks.workunit.client.0.smithi052.stdout: [LD] 2024-04-02T13:55:59.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:55:59 smithi088 ceph-mon[36528]: pgmap v77: 129 pgs: 129 active+clean; 66 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.3 MiB/s wr, 194 op/s 2024-04-02T13:56:00.007 INFO:tasks.workunit.client.0.smithi052.stdout:Building ltp 2024-04-02T13:56:00.007 INFO:tasks.workunit.client.0.smithi052.stdout:Building src 2024-04-02T13:56:00.007 INFO:tasks.workunit.client.0.smithi052.stdout:Building m4 2024-04-02T13:56:00.007 INFO:tasks.workunit.client.0.smithi052.stdout:Building common 2024-04-02T13:56:00.013 INFO:tasks.workunit.client.0.smithi052.stdout: [DEP] 2024-04-02T13:56:00.030 INFO:tasks.workunit.client.0.smithi052.stdout: [DEP] 2024-04-02T13:56:00.237 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] doio 2024-04-02T13:56:00.237 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] fsstress 2024-04-02T13:56:00.237 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] fsx 2024-04-02T13:56:00.386 INFO:tasks.workunit.client.0.smithi052.stderr:fsstress.c: In function 'attr_list_path': 2024-04-02T13:56:00.386 INFO:tasks.workunit.client.0.smithi052.stderr:fsstress.c:859:9: warning: 'attr_list' is deprecated: Use listxattr or llistxattr instead [-Wdeprecated-declarations] 2024-04-02T13:56:00.386 INFO:tasks.workunit.client.0.smithi052.stderr: 859 | rval = attr_list(name->path, buffer, buffersize, flags, cursor); 2024-04-02T13:56:00.386 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~ 2024-04-02T13:56:00.386 INFO:tasks.workunit.client.0.smithi052.stderr:In file included from ../src/global.h:25, 2024-04-02T13:56:00.386 INFO:tasks.workunit.client.0.smithi052.stderr: from fsstress.c:12: 2024-04-02T13:56:00.386 INFO:tasks.workunit.client.0.smithi052.stderr:/usr/include/attr/attributes.h:165:12: note: declared here 2024-04-02T13:56:00.386 INFO:tasks.workunit.client.0.smithi052.stderr: 165 | extern int attr_list(const char *__path, char *__buffer, const int __buffersize, 2024-04-02T13:56:00.386 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~~~~~ 2024-04-02T13:56:00.387 INFO:tasks.workunit.client.0.smithi052.stderr:fsstress.c: In function 'attr_remove_path': 2024-04-02T13:56:00.387 INFO:tasks.workunit.client.0.smithi052.stderr:fsstress.c:878:9: warning: 'attr_remove' is deprecated: Use removexattr or lremovexattr instead [-Wdeprecated-declarations] 2024-04-02T13:56:00.387 INFO:tasks.workunit.client.0.smithi052.stderr: 878 | rval = attr_remove(name->path, attrname, flags); 2024-04-02T13:56:00.387 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~ 2024-04-02T13:56:00.387 INFO:tasks.workunit.client.0.smithi052.stderr:In file included from ../src/global.h:25, 2024-04-02T13:56:00.387 INFO:tasks.workunit.client.0.smithi052.stderr: from fsstress.c:12: 2024-04-02T13:56:00.387 INFO:tasks.workunit.client.0.smithi052.stderr:/usr/include/attr/attributes.h:152:12: note: declared here 2024-04-02T13:56:00.387 INFO:tasks.workunit.client.0.smithi052.stderr: 152 | extern int attr_remove (const char *__path, const char *__attrname, 2024-04-02T13:56:00.387 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~~~~~~~ 2024-04-02T13:56:00.387 INFO:tasks.workunit.client.0.smithi052.stderr:fsstress.c: In function 'attr_set_path': 2024-04-02T13:56:00.387 INFO:tasks.workunit.client.0.smithi052.stderr:fsstress.c:898:9: warning: 'attr_set' is deprecated: Use setxattr or lsetxattr instead [-Wdeprecated-declarations] 2024-04-02T13:56:00.387 INFO:tasks.workunit.client.0.smithi052.stderr: 898 | rval = attr_set(name->path, attrname, attrvalue, valuelength, flags); 2024-04-02T13:56:00.387 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~ 2024-04-02T13:56:00.387 INFO:tasks.workunit.client.0.smithi052.stderr:In file included from ../src/global.h:25, 2024-04-02T13:56:00.387 INFO:tasks.workunit.client.0.smithi052.stderr: from fsstress.c:12: 2024-04-02T13:56:00.388 INFO:tasks.workunit.client.0.smithi052.stderr:/usr/include/attr/attributes.h:139:12: note: declared here 2024-04-02T13:56:00.388 INFO:tasks.workunit.client.0.smithi052.stderr: 139 | extern int attr_set (const char *__path, const char *__attrname, 2024-04-02T13:56:00.388 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~~~~ 2024-04-02T13:56:00.396 INFO:tasks.workunit.client.0.smithi052.stderr:fsstress.c: In function 'allocsp_f': 2024-04-02T13:56:00.396 INFO:tasks.workunit.client.0.smithi052.stderr:fsstress.c:2047:32: error: 'XFS_IOC_ALLOCSP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-04-02T13:56:00.396 INFO:tasks.workunit.client.0.smithi052.stderr: 2047 | e = xfsctl(f.path, fd, XFS_IOC_ALLOCSP64, &fl) < 0 ? errno : 0; 2024-04-02T13:56:00.397 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-02T13:56:00.397 INFO:tasks.workunit.client.0.smithi052.stderr: | XFS_IOC_RESVSP64 2024-04-02T13:56:00.397 INFO:tasks.workunit.client.0.smithi052.stderr:fsstress.c:2047:32: note: each undeclared identifier is reported only once for each function it appears in 2024-04-02T13:56:00.415 INFO:tasks.workunit.client.0.smithi052.stderr:fsstress.c: In function 'freesp_f': 2024-04-02T13:56:00.415 INFO:tasks.workunit.client.0.smithi052.stderr:fsstress.c:3640:32: error: 'XFS_IOC_FREESP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-04-02T13:56:00.415 INFO:tasks.workunit.client.0.smithi052.stderr: 3640 | e = xfsctl(f.path, fd, XFS_IOC_FREESP64, &fl) < 0 ? errno : 0; 2024-04-02T13:56:00.415 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-02T13:56:00.415 INFO:tasks.workunit.client.0.smithi052.stderr: | XFS_IOC_RESVSP64 2024-04-02T13:56:00.447 INFO:tasks.workunit.client.0.smithi052.stderr:gmake[3]: *** [Makefile:47: fsstress] Error 1 2024-04-02T13:56:00.448 INFO:tasks.workunit.client.0.smithi052.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-04-02T13:56:00.820 INFO:tasks.workunit.client.0.smithi052.stderr:doio.c: In function 'parse_cmdline': 2024-04-02T13:56:00.820 INFO:tasks.workunit.client.0.smithi052.stderr:doio.c:3113:33: warning: '%s' directive argument is null [-Wformat-overflow=] 2024-04-02T13:56:00.820 INFO:tasks.workunit.client.0.smithi052.stderr: 3113 | fprintf(stderr, 2024-04-02T13:56:00.820 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~~~~~~~~~~~ 2024-04-02T13:56:00.821 INFO:tasks.workunit.client.0.smithi052.stderr: 3114 | "%s%s: Illegal -C arg (%s). Must be one of: ", 2024-04-02T13:56:00.821 INFO:tasks.workunit.client.0.smithi052.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-02T13:56:00.821 INFO:tasks.workunit.client.0.smithi052.stderr: 3115 | Prog, TagName, tok); 2024-04-02T13:56:00.821 INFO:tasks.workunit.client.0.smithi052.stderr: | ~~~~~~~~~~~~~~~~~~~ 2024-04-02T13:56:00.849 INFO:tasks.workunit.client.0.smithi052.stderr:fsx.c: In function 'main': 2024-04-02T13:56:00.849 INFO:tasks.workunit.client.0.smithi052.stderr:fsx.c:2662:33: warning: 'strncpy' specified bound 4096 equals destination size [-Wstringop-truncation] 2024-04-02T13:56:00.849 INFO:tasks.workunit.client.0.smithi052.stderr: 2662 | strncpy(opsfile, optarg, sizeof(opsfile)); 2024-04-02T13:56:00.849 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-02T13:56:00.849 INFO:tasks.workunit.client.0.smithi052.stderr:fsx.c:2632:25: warning: 'strncpy' specified bound 1024 equals destination size [-Wstringop-truncation] 2024-04-02T13:56:00.849 INFO:tasks.workunit.client.0.smithi052.stderr: 2632 | strncpy(dname, optarg, sizeof(dname)); 2024-04-02T13:56:00.849 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-02T13:56:00.951 INFO:tasks.workunit.client.0.smithi052.stderr:gmake[2]: *** [include/buildrules:30: ltp] Error 2 2024-04-02T13:56:00.951 INFO:tasks.workunit.client.0.smithi052.stderr:gmake[2]: *** Waiting for unfinished jobs.... 2024-04-02T13:56:01.545 INFO:tasks.workunit.client.0.smithi052.stdout: [DEP] 2024-04-02T13:56:01.545 INFO:tasks.workunit.client.0.smithi052.stdout: [DEP] 2024-04-02T13:56:01.546 INFO:tasks.workunit.client.0.smithi052.stdout: [DEP] 2024-04-02T13:56:01.763 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] dirstress 2024-04-02T13:56:01.765 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] fill 2024-04-02T13:56:01.765 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] fill2 2024-04-02T13:56:01.766 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] getpagesize 2024-04-02T13:56:01.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:01 smithi189 ceph-mon[36683]: pgmap v78: 129 pgs: 129 active+clean; 66 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.3 MiB/s wr, 194 op/s 2024-04-02T13:56:01.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:01 smithi052 ceph-mon[32104]: pgmap v78: 129 pgs: 129 active+clean; 66 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.3 MiB/s wr, 194 op/s 2024-04-02T13:56:01.892 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] holes 2024-04-02T13:56:01.914 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] lstat64 2024-04-02T13:56:01.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:01 smithi088 ceph-mon[36528]: pgmap v78: 129 pgs: 129 active+clean; 66 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.3 MiB/s wr, 194 op/s 2024-04-02T13:56:01.991 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] nametest 2024-04-02T13:56:01.992 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] permname 2024-04-02T13:56:02.092 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] randholes 2024-04-02T13:56:02.100 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] runas 2024-04-02T13:56:02.219 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] truncfile 2024-04-02T13:56:02.244 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] usemem 2024-04-02T13:56:02.285 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] mmapcat 2024-04-02T13:56:02.365 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] append_reader 2024-04-02T13:56:02.397 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] append_writer 2024-04-02T13:56:02.398 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] dirperf 2024-04-02T13:56:02.437 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] metaperf 2024-04-02T13:56:02.520 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] devzero 2024-04-02T13:56:02.560 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] feature 2024-04-02T13:56:02.665 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] alloc 2024-04-02T13:56:02.709 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] fault 2024-04-02T13:56:02.766 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] fstest 2024-04-02T13:56:02.783 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] t_access_root 2024-04-02T13:56:02.801 INFO:tasks.workunit.client.0.smithi052.stderr:alloc.c: In function 'main': 2024-04-02T13:56:02.801 INFO:tasks.workunit.client.0.smithi052.stderr:alloc.c:154:1: error: 'bozo' undeclared (first use in this function) 2024-04-02T13:56:02.802 INFO:tasks.workunit.client.0.smithi052.stderr: 154 | bozo! 2024-04-02T13:56:02.802 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~ 2024-04-02T13:56:02.802 INFO:tasks.workunit.client.0.smithi052.stderr:alloc.c:154:1: note: each undeclared identifier is reported only once for each function it appears in 2024-04-02T13:56:02.802 INFO:tasks.workunit.client.0.smithi052.stderr:alloc.c:154:5: error: expected ';' before '!' token 2024-04-02T13:56:02.802 INFO:tasks.workunit.client.0.smithi052.stderr: 154 | bozo! 2024-04-02T13:56:02.802 INFO:tasks.workunit.client.0.smithi052.stderr: | ^ 2024-04-02T13:56:02.802 INFO:tasks.workunit.client.0.smithi052.stderr: | ; 2024-04-02T13:56:02.802 INFO:tasks.workunit.client.0.smithi052.stderr:alloc.c:184:25: error: 'rflag' undeclared (first use in this function); did you mean 'nflag'? 2024-04-02T13:56:02.802 INFO:tasks.workunit.client.0.smithi052.stderr: 184 | rflag = 1; 2024-04-02T13:56:02.802 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~ 2024-04-02T13:56:02.802 INFO:tasks.workunit.client.0.smithi052.stderr: | nflag 2024-04-02T13:56:02.803 INFO:tasks.workunit.client.0.smithi052.stderr:alloc.c:320:5: error: expected ';' before '!' token 2024-04-02T13:56:02.803 INFO:tasks.workunit.client.0.smithi052.stderr: 320 | bozo! 2024-04-02T13:56:02.803 INFO:tasks.workunit.client.0.smithi052.stderr: | ^ 2024-04-02T13:56:02.803 INFO:tasks.workunit.client.0.smithi052.stderr: | ; 2024-04-02T13:56:02.803 INFO:tasks.workunit.client.0.smithi052.stderr:alloc.c:128:25: warning: variable 'f' set but not used [-Wunused-but-set-variable] 2024-04-02T13:56:02.803 INFO:tasks.workunit.client.0.smithi052.stderr: 128 | struct flock64 f; 2024-04-02T13:56:02.803 INFO:tasks.workunit.client.0.smithi052.stderr: | ^ 2024-04-02T13:56:02.806 INFO:tasks.workunit.client.0.smithi052.stderr:gmake[3]: *** [Makefile:83: alloc] Error 1 2024-04-02T13:56:02.806 INFO:tasks.workunit.client.0.smithi052.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-04-02T13:56:02.907 INFO:tasks.workunit.client.0.smithi052.stderr:fstest.c: In function 'run_child': 2024-04-02T13:56:02.907 INFO:tasks.workunit.client.0.smithi052.stderr:fstest.c:95:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-04-02T13:56:02.907 INFO:tasks.workunit.client.0.smithi052.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-02T13:56:02.907 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~ 2024-04-02T13:56:02.907 INFO:tasks.workunit.client.0.smithi052.stderr:fstest.c:95:24: note: directive argument in the range [0, 2147483647] 2024-04-02T13:56:02.907 INFO:tasks.workunit.client.0.smithi052.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-02T13:56:02.907 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~~~~~~~ 2024-04-02T13:56:02.907 INFO:tasks.workunit.client.0.smithi052.stderr:fstest.c:95:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-04-02T13:56:02.908 INFO:tasks.workunit.client.0.smithi052.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-02T13:56:02.908 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-02T13:56:02.908 INFO:tasks.workunit.client.0.smithi052.stderr:fstest.c:166:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-04-02T13:56:02.908 INFO:tasks.workunit.client.0.smithi052.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-02T13:56:02.908 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~ 2024-04-02T13:56:02.908 INFO:tasks.workunit.client.0.smithi052.stderr:fstest.c:166:24: note: directive argument in the range [0, 2147483647] 2024-04-02T13:56:02.908 INFO:tasks.workunit.client.0.smithi052.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-02T13:56:02.908 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~~~~~~~ 2024-04-02T13:56:02.908 INFO:tasks.workunit.client.0.smithi052.stderr:fstest.c:166:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-04-02T13:56:02.908 INFO:tasks.workunit.client.0.smithi052.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-04-02T13:56:02.908 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-02T13:56:02.979 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-02T13:56:02.979 INFO:tasks.workunit.client.0.smithi052.stderr:gmake[2]: *** [include/buildrules:30: src] Error 2 2024-04-02T13:56:02.979 INFO:tasks.workunit.client.0.smithi052.stderr:make[1]: *** [Makefile:53: default] Error 2 2024-04-02T13:56:02.979 INFO:tasks.workunit.client.0.smithi052.stderr:make: *** [Makefile:51: default] Error 2 2024-04-02T13:56:02.979 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-04-02T13:56:02.980 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-02T13:56:03.256 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi052 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=191d963ac9de7322da8aff20b567b3fa458e2ef6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-04-02T13:56:03.496 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=526d8653bc7b422e89150358f99eb5b5 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi052 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=191d963ac9de7322da8aff20b567b3fa458e2ef6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-04-02T13:56:03.499 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-02T13:56:03.510 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-02T13:56:03.510 INFO:teuthology.task.exec:Executing custom commands... 2024-04-02T13:56:03.510 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi052.front.sepia.ceph.com 2024-04-02T13:56:03.510 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-02T13:56:03.539 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:03 smithi052 ceph-mon[32104]: pgmap v79: 129 pgs: 129 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.5 MiB/s wr, 222 op/s 2024-04-02T13:56:03.543 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-02T13:56:03.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:03 smithi189 ceph-mon[36683]: pgmap v79: 129 pgs: 129 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.5 MiB/s wr, 222 op/s 2024-04-02T13:56:03.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:03 smithi088 ceph-mon[36528]: pgmap v79: 129 pgs: 129 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.5 MiB/s wr, 222 op/s 2024-04-02T13:56:05.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:05 smithi189 ceph-mon[36683]: pgmap v80: 129 pgs: 129 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.4 MiB/s wr, 205 op/s 2024-04-02T13:56:05.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:05 smithi052 ceph-mon[32104]: pgmap v80: 129 pgs: 129 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.4 MiB/s wr, 205 op/s 2024-04-02T13:56:05.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:05 smithi088 ceph-mon[36528]: pgmap v80: 129 pgs: 129 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.4 MiB/s wr, 205 op/s 2024-04-02T13:56:07.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:07 smithi189 ceph-mon[36683]: pgmap v81: 129 pgs: 129 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 1.4 MiB/s wr, 211 op/s 2024-04-02T13:56:07.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:07 smithi052 ceph-mon[32104]: pgmap v81: 129 pgs: 129 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 1.4 MiB/s wr, 211 op/s 2024-04-02T13:56:07.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:07 smithi088 ceph-mon[36528]: pgmap v81: 129 pgs: 129 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 1.4 MiB/s wr, 211 op/s 2024-04-02T13:56:09.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:09 smithi189 ceph-mon[36683]: pgmap v82: 129 pgs: 129 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 221 KiB/s wr, 33 op/s 2024-04-02T13:56:09.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:09 smithi052 ceph-mon[32104]: pgmap v82: 129 pgs: 129 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 221 KiB/s wr, 33 op/s 2024-04-02T13:56:09.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:09 smithi088 ceph-mon[36528]: pgmap v82: 129 pgs: 129 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 221 KiB/s wr, 33 op/s 2024-04-02T13:56:11.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:11 smithi189 ceph-mon[36683]: pgmap v83: 129 pgs: 129 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 221 KiB/s wr, 33 op/s 2024-04-02T13:56:11.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:11 smithi052 ceph-mon[32104]: pgmap v83: 129 pgs: 129 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 221 KiB/s wr, 33 op/s 2024-04-02T13:56:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:11 smithi088 ceph-mon[36528]: pgmap v83: 129 pgs: 129 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 221 KiB/s wr, 33 op/s 2024-04-02T13:56:13.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:13 smithi189 ceph-mon[36683]: pgmap v84: 129 pgs: 129 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 223 KiB/s wr, 38 op/s 2024-04-02T13:56:13.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:13 smithi052 ceph-mon[32104]: pgmap v84: 129 pgs: 129 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 223 KiB/s wr, 38 op/s 2024-04-02T13:56:13.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:13 smithi088 ceph-mon[36528]: pgmap v84: 129 pgs: 129 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 223 KiB/s wr, 38 op/s 2024-04-02T13:56:15.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:15 smithi189 ceph-mon[36683]: pgmap v85: 129 pgs: 129 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-04-02T13:56:15.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:15 smithi052 ceph-mon[32104]: pgmap v85: 129 pgs: 129 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-04-02T13:56:15.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:15 smithi088 ceph-mon[36528]: pgmap v85: 129 pgs: 129 active+clean; 68 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-04-02T13:56:17.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:17 smithi189 ceph-mon[36683]: pgmap v86: 129 pgs: 129 active+clean; 71 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 386 KiB/s wr, 65 op/s 2024-04-02T13:56:17.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:17 smithi052 ceph-mon[32104]: pgmap v86: 129 pgs: 129 active+clean; 71 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 386 KiB/s wr, 65 op/s 2024-04-02T13:56:17.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:17 smithi088 ceph-mon[36528]: pgmap v86: 129 pgs: 129 active+clean; 71 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 386 KiB/s wr, 65 op/s 2024-04-02T13:56:18.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:56:18.988 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:56:19.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.103+0000 7fbc3acba640 1 -- 172.21.15.52:0/1450000890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc34106a30 msgr2=0x7fbc34106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.103+0000 7fbc3acba640 1 --2- 172.21.15.52:0/1450000890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc34106a30 0x7fbc34106e10 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fbc28009930 tx=0x7fbc2802bc30 comp rx=0 tx=0).stop 2024-04-02T13:56:19.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.103+0000 7fbc3acba640 1 -- 172.21.15.52:0/1450000890 shutdown_connections 2024-04-02T13:56:19.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.103+0000 7fbc3acba640 1 --2- 172.21.15.52:0/1450000890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc341013e0 0x7fbc3410d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.103+0000 7fbc3acba640 1 --2- 172.21.15.52:0/1450000890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc34100a20 0x7fbc34100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.103+0000 7fbc3acba640 1 --2- 172.21.15.52:0/1450000890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc34106a30 0x7fbc34106e10 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.103+0000 7fbc3acba640 1 -- 172.21.15.52:0/1450000890 >> 172.21.15.52:0/1450000890 conn(0x7fbc34077b60 msgr2=0x7fbc34077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.103+0000 7fbc3acba640 1 -- 172.21.15.52:0/1450000890 shutdown_connections 2024-04-02T13:56:19.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.103+0000 7fbc3acba640 1 -- 172.21.15.52:0/1450000890 wait complete. 2024-04-02T13:56:19.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.103+0000 7fbc3acba640 1 Processor -- start 2024-04-02T13:56:19.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.103+0000 7fbc3acba640 1 -- start start 2024-04-02T13:56:19.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.104+0000 7fbc3acba640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc34100a20 0x7fbc340fd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.104+0000 7fbc3acba640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc341013e0 0x7fbc340fdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.104+0000 7fbc3acba640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc34106a30 0x7fbc340fe420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.104+0000 7fbc3acba640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbc34110e10 con 0x7fbc341013e0 2024-04-02T13:56:19.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.104+0000 7fbc3acba640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbc34110c90 con 0x7fbc34106a30 2024-04-02T13:56:19.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.104+0000 7fbc3acba640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbc34110f90 con 0x7fbc34100a20 2024-04-02T13:56:19.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.104+0000 7fbc33fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc341013e0 0x7fbc340fdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.104+0000 7fbc33fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc341013e0 0x7fbc340fdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37870/0 (socket says 172.21.15.52:37870) 2024-04-02T13:56:19.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.104+0000 7fbc33fff640 1 -- 172.21.15.52:0/3891637563 learned_addr learned my addr 172.21.15.52:0/3891637563 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:19.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.104+0000 7fbc38a56640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc34100a20 0x7fbc340fd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.104+0000 7fbc39257640 1 --2- 172.21.15.52:0/3891637563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc34106a30 0x7fbc340fe420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.104+0000 7fbc33fff640 1 -- 172.21.15.52:0/3891637563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc34100a20 msgr2=0x7fbc340fd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.104+0000 7fbc33fff640 1 --2- 172.21.15.52:0/3891637563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc34100a20 0x7fbc340fd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.104+0000 7fbc33fff640 1 -- 172.21.15.52:0/3891637563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc34106a30 msgr2=0x7fbc340fe420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.104+0000 7fbc33fff640 1 --2- 172.21.15.52:0/3891637563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc34106a30 0x7fbc340fe420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.104+0000 7fbc33fff640 1 -- 172.21.15.52:0/3891637563 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc341a6900 con 0x7fbc341013e0 2024-04-02T13:56:19.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.104+0000 7fbc33fff640 1 --2- 172.21.15.52:0/3891637563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc341013e0 0x7fbc340fdee0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000d950 tx=0x7fbc2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.104+0000 7fbc31ffb640 1 -- 172.21.15.52:0/3891637563 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc20017070 con 0x7fbc341013e0 2024-04-02T13:56:19.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.105+0000 7fbc3acba640 1 -- 172.21.15.52:0/3891637563 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc341a6bf0 con 0x7fbc341013e0 2024-04-02T13:56:19.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.105+0000 7fbc31ffb640 1 -- 172.21.15.52:0/3891637563 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc20009c30 con 0x7fbc341013e0 2024-04-02T13:56:19.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.105+0000 7fbc31ffb640 1 -- 172.21.15.52:0/3891637563 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2000f6b0 con 0x7fbc341013e0 2024-04-02T13:56:19.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.105+0000 7fbc3acba640 1 -- 172.21.15.52:0/3891637563 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc341a7190 con 0x7fbc341013e0 2024-04-02T13:56:19.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.107+0000 7fbc31ffb640 1 -- 172.21.15.52:0/3891637563 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbc200109d0 con 0x7fbc341013e0 2024-04-02T13:56:19.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.108+0000 7fbc3d97d740 1 Processor -- start 2024-04-02T13:56:19.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.108+0000 7fbc3d97d740 1 -- start start 2024-04-02T13:56:19.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.108+0000 7fbc31ffb640 1 --2- 172.21.15.52:0/3891637563 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fbc240a3d20 0x7fbc240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.108+0000 7fbc31ffb640 1 -- 172.21.15.52:0/3891637563 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fbc200d0700 con 0x7fbc341013e0 2024-04-02T13:56:19.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.108+0000 7fbc39257640 1 --2- 172.21.15.52:0/3891637563 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fbc240a3d20 0x7fbc240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.108+0000 7fbc3d97d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578d00b28d0 0x5578d00b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.108+0000 7fbc3d97d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5578d00b31f0 0x5578d00bb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.108+0000 7fbc3d97d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00bbdd0 0x5578d00be1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.108+0000 7fbc3d97d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5578d0016830 con 0x5578d00bbdd0 2024-04-02T13:56:19.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.108+0000 7fbc3d97d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5578cffbe820 con 0x5578d00b31f0 2024-04-02T13:56:19.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.108+0000 7fbc3d97d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5578d00127e0 con 0x5578d00b28d0 2024-04-02T13:56:19.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.108+0000 7fbc39257640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00bbdd0 0x5578d00be1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.108+0000 7fbc39257640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00bbdd0 0x5578d00be1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37874/0 (socket says 172.21.15.52:37874) 2024-04-02T13:56:19.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.108+0000 7fbc39257640 1 -- 172.21.15.52:0/737874971 learned_addr learned my addr 172.21.15.52:0/737874971 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:19.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.108+0000 7fbc33fff640 1 --2- 172.21.15.52:0/737874971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5578d00b31f0 0x5578d00bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.108+0000 7fbc38a56640 1 --2- 172.21.15.52:0/737874971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578d00b28d0 0x5578d00b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.108+0000 7fbc39257640 1 --2- 172.21.15.52:0/3891637563 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fbc240a3d20 0x7fbc240a61e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c012780 tx=0x7fbc2c012b80 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.108+0000 7fbc39257640 1 -- 172.21.15.52:0/737874971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578d00b28d0 msgr2=0x5578d00b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.108+0000 7fbc39257640 1 --2- 172.21.15.52:0/737874971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578d00b28d0 0x5578d00b2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.108+0000 7fbc39257640 1 -- 172.21.15.52:0/737874971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5578d00b31f0 msgr2=0x5578d00bb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.108+0000 7fbc39257640 1 --2- 172.21.15.52:0/737874971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5578d00b31f0 0x5578d00bb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.108+0000 7fbc39257640 1 -- 172.21.15.52:0/737874971 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578cfdf47d0 con 0x5578d00bbdd0 2024-04-02T13:56:19.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.108+0000 7fbc33fff640 1 --2- 172.21.15.52:0/737874971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5578d00b31f0 0x5578d00bb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:19.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.109+0000 7fbc39257640 1 --2- 172.21.15.52:0/737874971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00bbdd0 0x5578d00be1c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009d90 tx=0x7fbc2c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4842bf670d1195 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.109+0000 7fbc1effd640 1 -- 172.21.15.52:0/737874971 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c003b70 con 0x5578d00bbdd0 2024-04-02T13:56:19.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.109+0000 7fbc1effd640 1 -- 172.21.15.52:0/737874971 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2c023070 con 0x5578d00bbdd0 2024-04-02T13:56:19.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.109+0000 7fbc1effd640 1 -- 172.21.15.52:0/737874971 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c004690 con 0x5578d00bbdd0 2024-04-02T13:56:19.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.109+0000 7fbc3d97d740 1 -- 172.21.15.52:0/737874971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00bbdd0 msgr2=0x5578d00be1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.109+0000 7fbc3d97d740 1 --2- 172.21.15.52:0/737874971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00bbdd0 0x5578d00be1c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009d90 tx=0x7fbc2c01e000 comp rx=0 tx=0).stop 2024-04-02T13:56:19.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.109+0000 7fbc3d97d740 1 -- 172.21.15.52:0/737874971 shutdown_connections 2024-04-02T13:56:19.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.109+0000 7fbc3d97d740 1 --2- 172.21.15.52:0/737874971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00bbdd0 0x5578d00be1c0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.109+0000 7fbc3d97d740 1 --2- 172.21.15.52:0/737874971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5578d00b31f0 0x5578d00bb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.109+0000 7fbc3d97d740 1 --2- 172.21.15.52:0/737874971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578d00b28d0 0x5578d00b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.109+0000 7fbc3d97d740 1 -- 172.21.15.52:0/737874971 >> 172.21.15.52:0/737874971 conn(0x5578d00ad000 msgr2=0x5578d00ad3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.109+0000 7fbc3d97d740 1 -- 172.21.15.52:0/737874971 shutdown_connections 2024-04-02T13:56:19.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.109+0000 7fbc3d97d740 1 -- 172.21.15.52:0/737874971 wait complete. 2024-04-02T13:56:19.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.110+0000 7fbc3d97d740 1 -- start start 2024-04-02T13:56:19.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.110+0000 7fbc3d97d740 1 Processor -- start 2024-04-02T13:56:19.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.110+0000 7fbc3d97d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5578d00b28d0 0x5578d00bb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.110+0000 7fbc3d97d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00b31f0 0x5578d00b89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.110+0000 7fbc3d97d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578d00bbdd0 0x5578d00b8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.110+0000 7fbc3d97d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5578d00beb60 con 0x5578d00b31f0 2024-04-02T13:56:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.110+0000 7fbc3d97d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5578cffd1d30 con 0x5578d00b28d0 2024-04-02T13:56:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.110+0000 7fbc3d97d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5578d00bece0 con 0x5578d00bbdd0 2024-04-02T13:56:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.110+0000 7fbc3d97d740 10 client.0 _ll_register_callbacks cb 0x5578cffd1720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.110+0000 7fbc33fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00b31f0 0x5578d00b89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.111+0000 7fbc33fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00b31f0 0x5578d00b89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37890/0 (socket says 172.21.15.52:37890) 2024-04-02T13:56:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.111+0000 7fbc33fff640 1 -- 172.21.15.52:0/3010037431 learned_addr learned my addr 172.21.15.52:0/3010037431 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.111+0000 7fbc39257640 1 --2- 172.21.15.52:0/3010037431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578d00bbdd0 0x5578d00b8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.111+0000 7fbc38a56640 1 --2- 172.21.15.52:0/3010037431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5578d00b28d0 0x5578d00bb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.111+0000 7fbc33fff640 1 -- 172.21.15.52:0/3010037431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578d00bbdd0 msgr2=0x5578d00b8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.111+0000 7fbc33fff640 1 --2- 172.21.15.52:0/3010037431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578d00bbdd0 0x5578d00b8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.111+0000 7fbc33fff640 1 -- 172.21.15.52:0/3010037431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5578d00b28d0 msgr2=0x5578d00bb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.111+0000 7fbc33fff640 1 --2- 172.21.15.52:0/3010037431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5578d00b28d0 0x5578d00bb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.111+0000 7fbc33fff640 1 -- 172.21.15.52:0/3010037431 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578cfd2a3f0 con 0x5578d00b31f0 2024-04-02T13:56:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.111+0000 7fbc33fff640 1 --2- 172.21.15.52:0/3010037431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00b31f0 0x5578d00b89f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fbc20000c00 tx=0x7fbc2009c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd556126eb67d9ac server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.111+0000 7fbc13fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.111+0000 7fbc13fff640 1 -- 172.21.15.52:0/3010037431 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc20020070 con 0x5578d00b31f0 2024-04-02T13:56:19.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.111+0000 7fbc13fff640 1 -- 172.21.15.52:0/3010037431 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fbc200b55f0 con 0x5578d00b31f0 2024-04-02T13:56:19.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.111+0000 7fbc3d97d740 1 -- 172.21.15.52:0/3010037431 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc2c023070 con 0x5578d00b31f0 2024-04-02T13:56:19.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.111+0000 7fbc13fff640 1 -- 172.21.15.52:0/3010037431 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbc200cdeb0 con 0x5578d00b31f0 2024-04-02T13:56:19.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.111+0000 7fbc3d97d740 10 client.16404 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:19.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.111+0000 7fbc3d97d740 1 -- 172.21.15.52:0/3010037431 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5578cfcea0f0 con 0x5578d00b31f0 2024-04-02T13:56:19.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.112+0000 7fbc13fff640 1 -- 172.21.15.52:0/3010037431 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2c023070 con 0x5578d00b31f0 2024-04-02T13:56:19.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.112+0000 7fbc13fff640 1 -- 172.21.15.52:0/3010037431 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fbc2009b920 con 0x5578d00b31f0 2024-04-02T13:56:19.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.112+0000 7fbc3d97d740 10 client.16404 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:19.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.112+0000 7fbc3d97d740 10 client.16404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:19.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.112+0000 7fbc3d97d740 20 client.16404 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:19.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.112+0000 7fbc3d97d740 1 --2- 172.21.15.52:0/3010037431 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5578d00cbd30 0x5578d00ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.112+0000 7fbc3d97d740 4 client.16404 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:19.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.112+0000 7fbc3d97d740 1 -- 172.21.15.52:0/3010037431 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578d0023d70 con 0x5578d00cbd30 2024-04-02T13:56:19.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.112+0000 7fbc38a56640 1 --2- 172.21.15.52:0/3010037431 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5578d00cbd30 0x5578d00ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.113+0000 7fbc13fff640 10 client.16404 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:19.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.113+0000 7fbc38a56640 1 --2- 172.21.15.52:0/3010037431 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5578d00cbd30 0x5578d00ce1b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6a0414bebc02033 server_cookie=e4e4653409d299f4 in_seq=0 out_seq=0 2024-04-02T13:56:19.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.113+0000 7fbc13fff640 1 -- 172.21.15.52:0/3010037431 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5578d0023d70 con 0x5578d00cbd30 2024-04-02T13:56:19.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.113+0000 7fbc13fff640 10 client.16404 handle_command_reply: tid=0 2024-04-02T13:56:19.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.113+0000 7fbc3d97d740 1 client.16404 shutdown 2024-04-02T13:56:19.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.113+0000 7fbc3d97d740 1 -- 172.21.15.52:0/3010037431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00b31f0 msgr2=0x5578d00b89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.113+0000 7fbc3d97d740 1 --2- 172.21.15.52:0/3010037431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00b31f0 0x5578d00b89f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fbc20000c00 tx=0x7fbc2009c4f0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.114+0000 7fbc3d97d740 1 -- 172.21.15.52:0/3010037431 shutdown_connections 2024-04-02T13:56:19.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.114+0000 7fbc3d97d740 1 --2- 172.21.15.52:0/3010037431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578d00bbdd0 0x5578d00b8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.114+0000 7fbc3d97d740 1 --2- 172.21.15.52:0/3010037431 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5578d00cbd30 0x5578d00ce1b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.114+0000 7fbc3d97d740 1 --2- 172.21.15.52:0/3010037431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00b31f0 0x5578d00b89f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.114+0000 7fbc3d97d740 1 --2- 172.21.15.52:0/3010037431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5578d00b28d0 0x5578d00bb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.114+0000 7fbc13fff640 0 client.16404 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:19.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.114+0000 7fbc3d97d740 1 -- 172.21.15.52:0/3010037431 >> 172.21.15.52:0/3010037431 conn(0x5578d00ad000 msgr2=0x5578d00aed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.114+0000 7fbc3d97d740 1 -- 172.21.15.52:0/3010037431 shutdown_connections 2024-04-02T13:56:19.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.114+0000 7fbc3d97d740 1 -- 172.21.15.52:0/3010037431 wait complete. 2024-04-02T13:56:19.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.114+0000 7fbc3d97d740 20 client.16404 trim_cache size 0 max 16384 2024-04-02T13:56:19.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.131+0000 7fbc3d97d740 1 Processor -- start 2024-04-02T13:56:19.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.131+0000 7fbc3d97d740 1 -- start start 2024-04-02T13:56:19.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.132+0000 7fbc3d97d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578d00b28d0 0x5578d00aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.132+0000 7fbc3d97d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00b31f0 0x5578d00ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.132+0000 7fbc3d97d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5578d00bbdd0 0x5578d00bb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.132+0000 7fbc3d97d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbc341b0f10 con 0x5578d00b31f0 2024-04-02T13:56:19.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.132+0000 7fbc3d97d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbc341b2a80 con 0x5578d00bbdd0 2024-04-02T13:56:19.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.132+0000 7fbc3d97d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbc341b15e0 con 0x5578d00b28d0 2024-04-02T13:56:19.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.132+0000 7fbc33fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00b31f0 0x5578d00ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.132+0000 7fbc33fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00b31f0 0x5578d00ac0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37902/0 (socket says 172.21.15.52:37902) 2024-04-02T13:56:19.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.132+0000 7fbc33fff640 1 -- 172.21.15.52:0/2773240174 learned_addr learned my addr 172.21.15.52:0/2773240174 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:19.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.132+0000 7fbc33fff640 1 -- 172.21.15.52:0/2773240174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578d00b28d0 msgr2=0x5578d00aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.132+0000 7fbc33fff640 1 --2- 172.21.15.52:0/2773240174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578d00b28d0 0x5578d00aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.132+0000 7fbc33fff640 1 -- 172.21.15.52:0/2773240174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5578d00bbdd0 msgr2=0x5578d00bb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.132+0000 7fbc33fff640 1 --2- 172.21.15.52:0/2773240174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5578d00bbdd0 0x5578d00bb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.132+0000 7fbc33fff640 1 -- 172.21.15.52:0/2773240174 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578d00ace30 con 0x5578d00b31f0 2024-04-02T13:56:19.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.132+0000 7fbc33fff640 1 --2- 172.21.15.52:0/2773240174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00b31f0 0x5578d00ac0c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000dfe0 tx=0x7fbc20098780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e9587509b8bd14 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.132+0000 7fbc1e7fc640 1 -- 172.21.15.52:0/2773240174 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc200a4f70 con 0x5578d00b31f0 2024-04-02T13:56:19.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.132+0000 7fbc1e7fc640 1 -- 172.21.15.52:0/2773240174 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc200d0320 con 0x5578d00b31f0 2024-04-02T13:56:19.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.132+0000 7fbc1e7fc640 1 -- 172.21.15.52:0/2773240174 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2009b4d0 con 0x5578d00b31f0 2024-04-02T13:56:19.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.133+0000 7fbc3d97d740 1 -- 172.21.15.52:0/2773240174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00b31f0 msgr2=0x5578d00ac0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.133+0000 7fbc3d97d740 1 --2- 172.21.15.52:0/2773240174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00b31f0 0x5578d00ac0c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000dfe0 tx=0x7fbc20098780 comp rx=0 tx=0).stop 2024-04-02T13:56:19.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.133+0000 7fbc3d97d740 1 -- 172.21.15.52:0/2773240174 shutdown_connections 2024-04-02T13:56:19.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.133+0000 7fbc3d97d740 1 --2- 172.21.15.52:0/2773240174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5578d00bbdd0 0x5578d00bb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.133+0000 7fbc3d97d740 1 --2- 172.21.15.52:0/2773240174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00b31f0 0x5578d00ac0c0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.133+0000 7fbc3d97d740 1 --2- 172.21.15.52:0/2773240174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578d00b28d0 0x5578d00aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.133+0000 7fbc3d97d740 1 -- 172.21.15.52:0/2773240174 >> 172.21.15.52:0/2773240174 conn(0x5578d00ad000 msgr2=0x5578d00ab9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.133+0000 7fbc3d97d740 1 -- 172.21.15.52:0/2773240174 shutdown_connections 2024-04-02T13:56:19.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.133+0000 7fbc3d97d740 1 -- 172.21.15.52:0/2773240174 wait complete. 2024-04-02T13:56:19.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.133+0000 7fbc3d97d740 1 -- start start 2024-04-02T13:56:19.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.133+0000 7fbc3d97d740 1 Processor -- start 2024-04-02T13:56:19.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.134+0000 7fbc3d97d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00b28d0 0x5578d00bfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.134+0000 7fbc3d97d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5578d00b31f0 0x5578d00c00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.134+0000 7fbc3d97d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578d00bbdd0 0x5578d00c0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.134+0000 7fbc3d97d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbc341b2380 con 0x5578d00b28d0 2024-04-02T13:56:19.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.134+0000 7fbc3d97d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbc341b1cb0 con 0x5578d00b31f0 2024-04-02T13:56:19.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.134+0000 7fbc3d97d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbc341aeef0 con 0x5578d00bbdd0 2024-04-02T13:56:19.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.134+0000 7fbc3d97d740 10 client.0 _ll_register_callbacks cb 0x7fbc2009c3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:19.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.134+0000 7fbc38a56640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00b28d0 0x5578d00bfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.134+0000 7fbc38a56640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00b28d0 0x5578d00bfba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37912/0 (socket says 172.21.15.52:37912) 2024-04-02T13:56:19.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.134+0000 7fbc38a56640 1 -- 172.21.15.52:0/3529232157 learned_addr learned my addr 172.21.15.52:0/3529232157 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:19.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.134+0000 7fbc39257640 1 --2- 172.21.15.52:0/3529232157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578d00bbdd0 0x5578d00c0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.134+0000 7fbc38a56640 1 -- 172.21.15.52:0/3529232157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578d00bbdd0 msgr2=0x5578d00c0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.134+0000 7fbc38a56640 1 --2- 172.21.15.52:0/3529232157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578d00bbdd0 0x5578d00c0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.134+0000 7fbc33fff640 1 --2- 172.21.15.52:0/3529232157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5578d00b31f0 0x5578d00c00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.134+0000 7fbc38a56640 1 -- 172.21.15.52:0/3529232157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5578d00b31f0 msgr2=0x5578d00c00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.134+0000 7fbc38a56640 1 --2- 172.21.15.52:0/3529232157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5578d00b31f0 0x5578d00c00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.134+0000 7fbc38a56640 1 -- 172.21.15.52:0/3529232157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578d01314f0 con 0x5578d00b28d0 2024-04-02T13:56:19.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.134+0000 7fbc39257640 1 --2- 172.21.15.52:0/3529232157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578d00bbdd0 0x5578d00c0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:56:19.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.134+0000 7fbc38a56640 1 --2- 172.21.15.52:0/3529232157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00b28d0 0x5578d00bfba0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbc28009900 tx=0x7fbc2802ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b435b28f11e79060 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.134+0000 7fbc13fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:19.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.134+0000 7fbc13fff640 1 -- 172.21.15.52:0/3529232157 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2804d070 con 0x5578d00b28d0 2024-04-02T13:56:19.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.135+0000 7fbc13fff640 1 -- 172.21.15.52:0/3529232157 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fbc28004240 con 0x5578d00b28d0 2024-04-02T13:56:19.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.135+0000 7fbc3d97d740 1 -- 172.21.15.52:0/3529232157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc200d0320 con 0x5578d00b28d0 2024-04-02T13:56:19.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.135+0000 7fbc13fff640 1 -- 172.21.15.52:0/3529232157 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbc28059050 con 0x5578d00b28d0 2024-04-02T13:56:19.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.135+0000 7fbc3d97d740 10 client.16410 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:19.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.135+0000 7fbc3d97d740 1 -- 172.21.15.52:0/3529232157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5578d0132840 con 0x5578d00b28d0 2024-04-02T13:56:19.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.135+0000 7fbc13fff640 1 -- 172.21.15.52:0/3529232157 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc200d0320 con 0x5578d00b28d0 2024-04-02T13:56:19.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.135+0000 7fbc13fff640 1 -- 172.21.15.52:0/3529232157 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fbc28004a70 con 0x5578d00b28d0 2024-04-02T13:56:19.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.135+0000 7fbc3d97d740 10 client.16410 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:19.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.135+0000 7fbc3d97d740 10 client.16410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:19.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.135+0000 7fbc3d97d740 20 client.16410 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:19.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.135+0000 7fbc3d97d740 1 --2- 172.21.15.52:0/3529232157 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5578d0132fd0 0x5578d01353f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.135+0000 7fbc3d97d740 4 client.16410 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:56:19.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.135+0000 7fbc3d97d740 1 -- 172.21.15.52:0/3529232157 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5578d001e0c0 con 0x5578d0132fd0 2024-04-02T13:56:19.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.135+0000 7fbc33fff640 1 --2- 172.21.15.52:0/3529232157 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5578d0132fd0 0x5578d01353f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.136+0000 7fbc13fff640 10 client.16410 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:19.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.136+0000 7fbc33fff640 1 --2- 172.21.15.52:0/3529232157 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5578d0132fd0 0x5578d01353f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c379896e32785ccb server_cookie=52aaaf46bd095aeb in_seq=0 out_seq=0 2024-04-02T13:56:19.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.136+0000 7fbc13fff640 1 -- 172.21.15.52:0/3529232157 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5578d001e0c0 con 0x5578d0132fd0 2024-04-02T13:56:19.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.136+0000 7fbc13fff640 10 client.16410 handle_command_reply: tid=0 2024-04-02T13:56:19.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.136+0000 7fbc3d97d740 1 client.16410 shutdown 2024-04-02T13:56:19.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.136+0000 7fbc3d97d740 1 -- 172.21.15.52:0/3529232157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00b28d0 msgr2=0x5578d00bfba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.136+0000 7fbc3d97d740 1 --2- 172.21.15.52:0/3529232157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00b28d0 0x5578d00bfba0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbc28009900 tx=0x7fbc2802ca60 comp rx=0 tx=0).stop 2024-04-02T13:56:19.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.136+0000 7fbc3d97d740 1 -- 172.21.15.52:0/3529232157 shutdown_connections 2024-04-02T13:56:19.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.136+0000 7fbc3d97d740 1 --2- 172.21.15.52:0/3529232157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578d00bbdd0 0x5578d00c0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.136+0000 7fbc3d97d740 1 --2- 172.21.15.52:0/3529232157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5578d00b31f0 0x5578d00c00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.136+0000 7fbc3d97d740 1 --2- 172.21.15.52:0/3529232157 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5578d0132fd0 0x5578d01353f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.136+0000 7fbc3d97d740 1 --2- 172.21.15.52:0/3529232157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d00b28d0 0x5578d00bfba0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.136+0000 7fbc3d97d740 1 -- 172.21.15.52:0/3529232157 >> 172.21.15.52:0/3529232157 conn(0x5578d00ad000 msgr2=0x5578d00ac240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.136+0000 7fbc13fff640 0 client.16410 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:19.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.136+0000 7fbc3d97d740 1 -- 172.21.15.52:0/3529232157 shutdown_connections 2024-04-02T13:56:19.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.136+0000 7fbc3d97d740 1 -- 172.21.15.52:0/3529232157 wait complete. 2024-04-02T13:56:19.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.136+0000 7fbc3d97d740 20 client.16410 trim_cache size 0 max 16384 2024-04-02T13:56:19.152 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:56:19.153 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-04-02T13:56:19.153 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-04-02T13:56:19.153 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:56:19.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.137+0000 7fbc3acba640 1 -- 172.21.15.52:0/3891637563 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fbc240a3d20 msgr2=0x7fbc240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.137+0000 7fbc3acba640 1 --2- 172.21.15.52:0/3891637563 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fbc240a3d20 0x7fbc240a61e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c012780 tx=0x7fbc2c012b80 comp rx=0 tx=0).stop 2024-04-02T13:56:19.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.137+0000 7fbc3acba640 1 -- 172.21.15.52:0/3891637563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc341013e0 msgr2=0x7fbc340fdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.137+0000 7fbc3acba640 1 --2- 172.21.15.52:0/3891637563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc341013e0 0x7fbc340fdee0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000d950 tx=0x7fbc2000de10 comp rx=0 tx=0).stop 2024-04-02T13:56:19.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.137+0000 7fbc3acba640 1 -- 172.21.15.52:0/3891637563 shutdown_connections 2024-04-02T13:56:19.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.137+0000 7fbc3acba640 1 --2- 172.21.15.52:0/3891637563 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fbc240a3d20 0x7fbc240a61e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.137+0000 7fbc3acba640 1 --2- 172.21.15.52:0/3891637563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc34106a30 0x7fbc340fe420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.137+0000 7fbc3acba640 1 --2- 172.21.15.52:0/3891637563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc341013e0 0x7fbc340fdee0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.137+0000 7fbc3acba640 1 --2- 172.21.15.52:0/3891637563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc34100a20 0x7fbc340fd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.137+0000 7fbc3acba640 1 -- 172.21.15.52:0/3891637563 >> 172.21.15.52:0/3891637563 conn(0x7fbc34077b60 msgr2=0x7fbc340ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.137+0000 7fbc3acba640 1 -- 172.21.15.52:0/3891637563 shutdown_connections 2024-04-02T13:56:19.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.137+0000 7fbc3acba640 1 -- 172.21.15.52:0/3891637563 wait complete. 2024-04-02T13:56:19.156 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:56:19.156 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:56:19.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.256+0000 7f6f5078d640 1 -- 172.21.15.52:0/2236493854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f4810e350 msgr2=0x7f6f48110740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.256+0000 7f6f5078d640 1 --2- 172.21.15.52:0/2236493854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f4810e350 0x7f6f48110740 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f6f40009ca0 tx=0x7f6f4002c180 comp rx=0 tx=0).stop 2024-04-02T13:56:19.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.256+0000 7f6f5078d640 1 -- 172.21.15.52:0/2236493854 shutdown_connections 2024-04-02T13:56:19.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.256+0000 7f6f5078d640 1 --2- 172.21.15.52:0/2236493854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f4810e350 0x7f6f48110740 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.256+0000 7f6f5078d640 1 --2- 172.21.15.52:0/2236493854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f480747d0 0x7f6f4810de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.256+0000 7f6f5078d640 1 --2- 172.21.15.52:0/2236493854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f48073eb0 0x7f6f48074290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.256+0000 7f6f5078d640 1 -- 172.21.15.52:0/2236493854 >> 172.21.15.52:0/2236493854 conn(0x7f6f480fd5a0 msgr2=0x7f6f480ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.256+0000 7f6f5078d640 1 -- 172.21.15.52:0/2236493854 shutdown_connections 2024-04-02T13:56:19.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.256+0000 7f6f5078d640 1 -- 172.21.15.52:0/2236493854 wait complete. 2024-04-02T13:56:19.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.257+0000 7f6f5078d640 1 Processor -- start 2024-04-02T13:56:19.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.257+0000 7f6f5078d640 1 -- start start 2024-04-02T13:56:19.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.257+0000 7f6f5078d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f48073eb0 0x7f6f4819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.257+0000 7f6f5078d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f480747d0 0x7f6f4819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.257+0000 7f6f5078d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f4810e350 0x7f6f4819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.257+0000 7f6f5078d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6f48114070 con 0x7f6f4810e350 2024-04-02T13:56:19.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.257+0000 7f6f5078d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6f48113ef0 con 0x7f6f48073eb0 2024-04-02T13:56:19.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.257+0000 7f6f5078d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6f481141f0 con 0x7f6f480747d0 2024-04-02T13:56:19.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.257+0000 7f6f4ed2a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f4810e350 0x7f6f4819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.257+0000 7f6f4ed2a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f4810e350 0x7f6f4819eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37924/0 (socket says 172.21.15.52:37924) 2024-04-02T13:56:19.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.257+0000 7f6f4ed2a640 1 -- 172.21.15.52:0/1872746701 learned_addr learned my addr 172.21.15.52:0/1872746701 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:19.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.257+0000 7f6f4e529640 1 --2- 172.21.15.52:0/1872746701 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f48073eb0 0x7f6f4819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.257+0000 7f6f4dd28640 1 --2- 172.21.15.52:0/1872746701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f480747d0 0x7f6f4819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.258+0000 7f6f4ed2a640 1 -- 172.21.15.52:0/1872746701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f480747d0 msgr2=0x7f6f4819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.258+0000 7f6f4ed2a640 1 --2- 172.21.15.52:0/1872746701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f480747d0 0x7f6f4819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.258+0000 7f6f4ed2a640 1 -- 172.21.15.52:0/1872746701 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f48073eb0 msgr2=0x7f6f4819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.258+0000 7f6f4ed2a640 1 --2- 172.21.15.52:0/1872746701 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f48073eb0 0x7f6f4819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.258+0000 7f6f4ed2a640 1 -- 172.21.15.52:0/1872746701 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4819f610 con 0x7f6f4810e350 2024-04-02T13:56:19.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.258+0000 7f6f4ed2a640 1 --2- 172.21.15.52:0/1872746701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f4810e350 0x7f6f4819eee0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f6f4002c790 tx=0x7f6f40035ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.258+0000 7f6f3f7fe640 1 -- 172.21.15.52:0/1872746701 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f40039040 con 0x7f6f4810e350 2024-04-02T13:56:19.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.258+0000 7f6f3f7fe640 1 -- 172.21.15.52:0/1872746701 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f40044070 con 0x7f6f4810e350 2024-04-02T13:56:19.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.258+0000 7f6f5078d640 1 -- 172.21.15.52:0/1872746701 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4819f8a0 con 0x7f6f4810e350 2024-04-02T13:56:19.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.258+0000 7f6f3f7fe640 1 -- 172.21.15.52:0/1872746701 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f400329f0 con 0x7f6f4810e350 2024-04-02T13:56:19.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.258+0000 7f6f5078d640 1 -- 172.21.15.52:0/1872746701 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f481a7190 con 0x7f6f4810e350 2024-04-02T13:56:19.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.259+0000 7f6f5078d640 1 -- 172.21.15.52:0/1872746701 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f4819b970 con 0x7f6f4810e350 2024-04-02T13:56:19.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.260+0000 7f6f3f7fe640 1 -- 172.21.15.52:0/1872746701 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6f40032b90 con 0x7f6f4810e350 2024-04-02T13:56:19.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.261+0000 7f6f3f7fe640 1 --2- 172.21.15.52:0/1872746701 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6f340a3d20 0x7f6f340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.261+0000 7f6f4e529640 1 --2- 172.21.15.52:0/1872746701 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6f340a3d20 0x7f6f340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.261+0000 7f6f3f7fe640 1 -- 172.21.15.52:0/1872746701 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f6f400f3720 con 0x7f6f4810e350 2024-04-02T13:56:19.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.261+0000 7f6f4e529640 1 --2- 172.21.15.52:0/1872746701 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6f340a3d20 0x7f6f340a61e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6f44005fd0 tx=0x7f6f4400b590 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.264+0000 7f6f3f7fe640 1 -- 172.21.15.52:0/1872746701 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6f400bfb30 con 0x7f6f4810e350 2024-04-02T13:56:19.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.489+0000 7f6f5078d640 1 -- 172.21.15.52:0/1872746701 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f481094d0 con 0x7f6f4810e350 2024-04-02T13:56:19.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.490+0000 7f6f3f7fe640 1 -- 172.21.15.52:0/1872746701 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11924 (secure 0 0 0) 0x7f6f400bf430 con 0x7f6f4810e350 2024-04-02T13:56:19.493 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:56:19.494 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:55:15.239889+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24475,24503,24490,24457,24467]},"id":1}]} 2024-04-02T13:56:19.494 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 29 2024-04-02T13:56:19.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.492+0000 7f6f5078d640 1 -- 172.21.15.52:0/1872746701 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6f340a3d20 msgr2=0x7f6f340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.492+0000 7f6f5078d640 1 --2- 172.21.15.52:0/1872746701 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6f340a3d20 0x7f6f340a61e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6f44005fd0 tx=0x7f6f4400b590 comp rx=0 tx=0).stop 2024-04-02T13:56:19.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.492+0000 7f6f5078d640 1 -- 172.21.15.52:0/1872746701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f4810e350 msgr2=0x7f6f4819eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.492+0000 7f6f5078d640 1 --2- 172.21.15.52:0/1872746701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f4810e350 0x7f6f4819eee0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f6f4002c790 tx=0x7f6f40035ed0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.492+0000 7f6f5078d640 1 -- 172.21.15.52:0/1872746701 shutdown_connections 2024-04-02T13:56:19.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.492+0000 7f6f5078d640 1 --2- 172.21.15.52:0/1872746701 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6f340a3d20 0x7f6f340a61e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.492+0000 7f6f5078d640 1 --2- 172.21.15.52:0/1872746701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f4810e350 0x7f6f4819eee0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.492+0000 7f6f5078d640 1 --2- 172.21.15.52:0/1872746701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f480747d0 0x7f6f4819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.492+0000 7f6f5078d640 1 --2- 172.21.15.52:0/1872746701 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f48073eb0 0x7f6f4819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.492+0000 7f6f5078d640 1 -- 172.21.15.52:0/1872746701 >> 172.21.15.52:0/1872746701 conn(0x7f6f480fd5a0 msgr2=0x7f6f480fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.492+0000 7f6f5078d640 1 -- 172.21.15.52:0/1872746701 shutdown_connections 2024-04-02T13:56:19.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.492+0000 7f6f5078d640 1 -- 172.21.15.52:0/1872746701 wait complete. 2024-04-02T13:56:19.507 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:56:19.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.609+0000 7fb4a3da8640 1 -- 172.21.15.52:0/2690303758 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb49c107b70 msgr2=0x7fb49c107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.609+0000 7fb4a3da8640 1 --2- 172.21.15.52:0/2690303758 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb49c107b70 0x7fb49c107f50 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb498009930 tx=0x7fb49802bc30 comp rx=0 tx=0).stop 2024-04-02T13:56:19.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.609+0000 7fb4a3da8640 1 -- 172.21.15.52:0/2690303758 shutdown_connections 2024-04-02T13:56:19.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.609+0000 7fb4a3da8640 1 --2- 172.21.15.52:0/2690303758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb49c06ab70 0x7fb49c10e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.609+0000 7fb4a3da8640 1 --2- 172.21.15.52:0/2690303758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb49c06a1b0 0x7fb49c06a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.609+0000 7fb4a3da8640 1 --2- 172.21.15.52:0/2690303758 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb49c107b70 0x7fb49c107f50 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.609+0000 7fb4a3da8640 1 -- 172.21.15.52:0/2690303758 >> 172.21.15.52:0/2690303758 conn(0x7fb49c077ad0 msgr2=0x7fb49c077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.609+0000 7fb4a3da8640 1 -- 172.21.15.52:0/2690303758 shutdown_connections 2024-04-02T13:56:19.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.609+0000 7fb4a3da8640 1 -- 172.21.15.52:0/2690303758 wait complete. 2024-04-02T13:56:19.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.610+0000 7fb4a3da8640 1 Processor -- start 2024-04-02T13:56:19.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.610+0000 7fb4a3da8640 1 -- start start 2024-04-02T13:56:19.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.610+0000 7fb4a3da8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb49c06a1b0 0x7fb49c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.610+0000 7fb4a3da8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb49c06ab70 0x7fb49c198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.610+0000 7fb4a3da8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb49c107b70 0x7fb49c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.610+0000 7fb4a3da8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c111ec0 con 0x7fb49c107b70 2024-04-02T13:56:19.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.610+0000 7fb4a3da8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c111d40 con 0x7fb49c06ab70 2024-04-02T13:56:19.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.610+0000 7fb4a3da8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c112040 con 0x7fb49c06a1b0 2024-04-02T13:56:19.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.610+0000 7fb4a2345640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb49c107b70 0x7fb49c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.610+0000 7fb4a2345640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb49c107b70 0x7fb49c19cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37942/0 (socket says 172.21.15.52:37942) 2024-04-02T13:56:19.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.610+0000 7fb4a2345640 1 -- 172.21.15.52:0/1053331556 learned_addr learned my addr 172.21.15.52:0/1053331556 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:19.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.610+0000 7fb4a1343640 1 --2- 172.21.15.52:0/1053331556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb49c06ab70 0x7fb49c198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.610+0000 7fb4a1b44640 1 --2- 172.21.15.52:0/1053331556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb49c06a1b0 0x7fb49c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.610+0000 7fb4a2345640 1 -- 172.21.15.52:0/1053331556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb49c06a1b0 msgr2=0x7fb49c198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.610+0000 7fb4a2345640 1 --2- 172.21.15.52:0/1053331556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb49c06a1b0 0x7fb49c198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.610+0000 7fb4a2345640 1 -- 172.21.15.52:0/1053331556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb49c06ab70 msgr2=0x7fb49c198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.610+0000 7fb4a2345640 1 --2- 172.21.15.52:0/1053331556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb49c06ab70 0x7fb49c198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.610+0000 7fb4a2345640 1 -- 172.21.15.52:0/1053331556 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb49c19d2f0 con 0x7fb49c107b70 2024-04-02T13:56:19.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.610+0000 7fb4a1b44640 1 --2- 172.21.15.52:0/1053331556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb49c06a1b0 0x7fb49c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:19.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.611+0000 7fb4a2345640 1 --2- 172.21.15.52:0/1053331556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb49c107b70 0x7fb49c19cbc0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb49400eaa0 tx=0x7fb49400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.611+0000 7fb492ffd640 1 -- 172.21.15.52:0/1053331556 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49400cdb0 con 0x7fb49c107b70 2024-04-02T13:56:19.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.611+0000 7fb492ffd640 1 -- 172.21.15.52:0/1053331556 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb494005020 con 0x7fb49c107b70 2024-04-02T13:56:19.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.611+0000 7fb492ffd640 1 -- 172.21.15.52:0/1053331556 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4940116b0 con 0x7fb49c107b70 2024-04-02T13:56:19.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.611+0000 7fb4a3da8640 1 -- 172.21.15.52:0/1053331556 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb49c19d5e0 con 0x7fb49c107b70 2024-04-02T13:56:19.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.611+0000 7fb4a3da8640 1 -- 172.21.15.52:0/1053331556 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb49c1a4e70 con 0x7fb49c107b70 2024-04-02T13:56:19.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.613+0000 7fb492ffd640 1 -- 172.21.15.52:0/1053331556 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb494005560 con 0x7fb49c107b70 2024-04-02T13:56:19.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.614+0000 7fb4a698d740 1 Processor -- start 2024-04-02T13:56:19.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.614+0000 7fb4a698d740 1 -- start start 2024-04-02T13:56:19.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.614+0000 7fb4a698d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5626f3b7b4d0 0x5626f3b7b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.614+0000 7fb4a698d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626f3b7bdf0 0x5626f3b88b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.614+0000 7fb4a698d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b890b0 0x5626f3b8b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.614+0000 7fb4a698d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5626f3adf7b0 con 0x5626f3b890b0 2024-04-02T13:56:19.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.614+0000 7fb4a698d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5626f3a7e570 con 0x5626f3b7b4d0 2024-04-02T13:56:19.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.614+0000 7fb492ffd640 1 --2- 172.21.15.52:0/1053331556 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb4880a3d20 0x7fb4880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.614+0000 7fb4a698d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5626f3adb760 con 0x5626f3b7bdf0 2024-04-02T13:56:19.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.614+0000 7fb4a2345640 1 --2- 172.21.15.52:0/1053331556 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb4880a3d20 0x7fb4880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.614+0000 7fb4a1343640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b890b0 0x5626f3b8b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.614+0000 7fb4a1343640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b890b0 0x5626f3b8b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37946/0 (socket says 172.21.15.52:37946) 2024-04-02T13:56:19.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.614+0000 7fb4a1343640 1 -- 172.21.15.52:0/1660617030 learned_addr learned my addr 172.21.15.52:0/1660617030 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:19.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.614+0000 7fb492ffd640 1 -- 172.21.15.52:0/1053331556 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb4940d0d70 con 0x7fb49c107b70 2024-04-02T13:56:19.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.614+0000 7fb4a1b44640 1 --2- 172.21.15.52:0/1660617030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626f3b7bdf0 0x5626f3b88b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.614+0000 7fb4a1343640 1 --2- 172.21.15.52:0/1660617030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5626f3b7b4d0 0x5626f3b7b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.615+0000 7fb4a1343640 1 -- 172.21.15.52:0/1660617030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626f3b7bdf0 msgr2=0x5626f3b88b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.615+0000 7fb4a1343640 1 --2- 172.21.15.52:0/1660617030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626f3b7bdf0 0x5626f3b88b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.615+0000 7fb4a1343640 1 -- 172.21.15.52:0/1660617030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5626f3b7b4d0 msgr2=0x5626f3b7b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.615+0000 7fb4a1343640 1 --2- 172.21.15.52:0/1660617030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5626f3b7b4d0 0x5626f3b7b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.615+0000 7fb4a1343640 1 -- 172.21.15.52:0/1660617030 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626f37f33f0 con 0x5626f3b890b0 2024-04-02T13:56:19.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.615+0000 7fb4a2345640 1 --2- 172.21.15.52:0/1053331556 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb4880a3d20 0x7fb4880a61e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb49400efd0 tx=0x7fb494087040 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.615+0000 7fb4a1343640 1 --2- 172.21.15.52:0/1660617030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b890b0 0x5626f3b8b4a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb48c0127c0 tx=0x7fb48c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1787460b85f27b9c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.615+0000 7fb487fff640 1 -- 172.21.15.52:0/1660617030 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c012e00 con 0x5626f3b890b0 2024-04-02T13:56:19.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.615+0000 7fb487fff640 1 -- 172.21.15.52:0/1660617030 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb48c010c50 con 0x5626f3b890b0 2024-04-02T13:56:19.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.615+0000 7fb487fff640 1 -- 172.21.15.52:0/1660617030 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c01eb30 con 0x5626f3b890b0 2024-04-02T13:56:19.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.616+0000 7fb4a698d740 1 -- 172.21.15.52:0/1660617030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b890b0 msgr2=0x5626f3b8b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.616+0000 7fb4a698d740 1 --2- 172.21.15.52:0/1660617030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b890b0 0x5626f3b8b4a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb48c0127c0 tx=0x7fb48c00b7c0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.616+0000 7fb4a698d740 1 -- 172.21.15.52:0/1660617030 shutdown_connections 2024-04-02T13:56:19.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.616+0000 7fb4a698d740 1 --2- 172.21.15.52:0/1660617030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b890b0 0x5626f3b8b4a0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.616+0000 7fb4a698d740 1 --2- 172.21.15.52:0/1660617030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626f3b7bdf0 0x5626f3b88b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.616+0000 7fb4a698d740 1 --2- 172.21.15.52:0/1660617030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5626f3b7b4d0 0x5626f3b7b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.616+0000 7fb4a698d740 1 -- 172.21.15.52:0/1660617030 >> 172.21.15.52:0/1660617030 conn(0x5626f3b75ed0 msgr2=0x5626f3b762b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.616+0000 7fb4a698d740 1 -- 172.21.15.52:0/1660617030 shutdown_connections 2024-04-02T13:56:19.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.616+0000 7fb4a698d740 1 -- 172.21.15.52:0/1660617030 wait complete. 2024-04-02T13:56:19.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.616+0000 7fb4a698d740 1 -- start start 2024-04-02T13:56:19.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.616+0000 7fb4a698d740 1 Processor -- start 2024-04-02T13:56:19.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.617+0000 7fb4a698d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626f3b7b4d0 0x5626f3b88b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.617+0000 7fb4a698d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b7bdf0 0x5626f3b85ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.617+0000 7fb4a698d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5626f3b890b0 0x5626f3b86630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.617+0000 7fb4a698d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5626f3b8be70 con 0x5626f3b7bdf0 2024-04-02T13:56:19.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.617+0000 7fb4a698d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5626f3a9ae80 con 0x5626f3b890b0 2024-04-02T13:56:19.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.617+0000 7fb4a698d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5626f3b8bff0 con 0x5626f3b7b4d0 2024-04-02T13:56:19.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.617+0000 7fb4a1b44640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b7bdf0 0x5626f3b85ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.617+0000 7fb4a698d740 10 client.0 _ll_register_callbacks cb 0x5626f3ad6aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:19.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.617+0000 7fb4a1b44640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b7bdf0 0x5626f3b85ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37960/0 (socket says 172.21.15.52:37960) 2024-04-02T13:56:19.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.617+0000 7fb4a1b44640 1 -- 172.21.15.52:0/3139786940 learned_addr learned my addr 172.21.15.52:0/3139786940 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:19.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.617+0000 7fb4a1343640 1 --2- 172.21.15.52:0/3139786940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626f3b7b4d0 0x5626f3b88b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.617+0000 7fb4a1b44640 1 -- 172.21.15.52:0/3139786940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626f3b7b4d0 msgr2=0x5626f3b88b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.617+0000 7fb4a1343640 1 --2- 172.21.15.52:0/3139786940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5626f3b890b0 0x5626f3b86630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.617+0000 7fb4a1b44640 1 --2- 172.21.15.52:0/3139786940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626f3b7b4d0 0x5626f3b88b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.617+0000 7fb4a1b44640 1 -- 172.21.15.52:0/3139786940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5626f3b890b0 msgr2=0x5626f3b86630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.617+0000 7fb4a1b44640 1 --2- 172.21.15.52:0/3139786940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5626f3b890b0 0x5626f3b86630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.617+0000 7fb4a1b44640 1 -- 172.21.15.52:0/3139786940 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626f38bd7d0 con 0x5626f3b7bdf0 2024-04-02T13:56:19.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.617+0000 7fb4a1343640 1 --2- 172.21.15.52:0/3139786940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5626f3b890b0 0x5626f3b86630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:56:19.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.617+0000 7fb4a1b44640 1 --2- 172.21.15.52:0/3139786940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b7bdf0 0x5626f3b85ef0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb498036040 tx=0x7fb498005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48083054031a44c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.617+0000 7fb4857fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:19.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.617+0000 7fb4857fa640 1 -- 172.21.15.52:0/3139786940 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49802cc70 con 0x5626f3b7bdf0 2024-04-02T13:56:19.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.617+0000 7fb4857fa640 1 -- 172.21.15.52:0/3139786940 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb498005030 con 0x5626f3b7bdf0 2024-04-02T13:56:19.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.617+0000 7fb4a698d740 1 -- 172.21.15.52:0/3139786940 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb48c010c50 con 0x5626f3b7bdf0 2024-04-02T13:56:19.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.618+0000 7fb4857fa640 1 -- 172.21.15.52:0/3139786940 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb498050050 con 0x5626f3b7bdf0 2024-04-02T13:56:19.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.618+0000 7fb4a698d740 10 client.16428 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:19.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.618+0000 7fb4a698d740 1 -- 172.21.15.52:0/3139786940 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5626f3b97f60 con 0x5626f3b7bdf0 2024-04-02T13:56:19.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.618+0000 7fb4857fa640 1 -- 172.21.15.52:0/3139786940 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb48c010c50 con 0x5626f3b7bdf0 2024-04-02T13:56:19.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.618+0000 7fb4857fa640 1 -- 172.21.15.52:0/3139786940 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fb4980035f0 con 0x5626f3b7bdf0 2024-04-02T13:56:19.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.618+0000 7fb4a698d740 10 client.16428 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:19.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.618+0000 7fb4a698d740 10 client.16428 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T13:56:19.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.618+0000 7fb4a698d740 20 client.16428 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:19.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.618+0000 7fb4a698d740 1 --2- 172.21.15.52:0/3139786940 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5626f3b99470 0x5626f3b9b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.618+0000 7fb4a698d740 4 client.16428 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:19.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.618+0000 7fb4a698d740 1 -- 172.21.15.52:0/3139786940 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626f3aeccf0 con 0x5626f3b99470 2024-04-02T13:56:19.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.618+0000 7fb4a1343640 1 --2- 172.21.15.52:0/3139786940 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5626f3b99470 0x5626f3b9b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.619+0000 7fb4a1343640 1 --2- 172.21.15.52:0/3139786940 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5626f3b99470 0x5626f3b9b8b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b915a7db952aa2fd server_cookie=f15b8b0a9ca05a3e in_seq=0 out_seq=0 2024-04-02T13:56:19.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.619+0000 7fb4857fa640 10 client.16428 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:56:19.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.619+0000 7fb4857fa640 1 -- 172.21.15.52:0/3139786940 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5626f3aeccf0 con 0x5626f3b99470 2024-04-02T13:56:19.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.620+0000 7fb4857fa640 10 client.16428 handle_command_reply: tid=0 2024-04-02T13:56:19.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.620+0000 7fb4a698d740 1 client.16428 shutdown 2024-04-02T13:56:19.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.620+0000 7fb4a698d740 1 -- 172.21.15.52:0/3139786940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b7bdf0 msgr2=0x5626f3b85ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.620+0000 7fb4a698d740 1 --2- 172.21.15.52:0/3139786940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b7bdf0 0x5626f3b85ef0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb498036040 tx=0x7fb498005dd0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.620+0000 7fb4a698d740 1 -- 172.21.15.52:0/3139786940 shutdown_connections 2024-04-02T13:56:19.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.620+0000 7fb4a698d740 1 --2- 172.21.15.52:0/3139786940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5626f3b890b0 0x5626f3b86630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.620+0000 7fb4a698d740 1 --2- 172.21.15.52:0/3139786940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b7bdf0 0x5626f3b85ef0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.620+0000 7fb4a698d740 1 --2- 172.21.15.52:0/3139786940 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5626f3b99470 0x5626f3b9b8b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.620+0000 7fb4a698d740 1 --2- 172.21.15.52:0/3139786940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626f3b7b4d0 0x5626f3b88b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.620+0000 7fb4857fa640 0 client.16428 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:56:19.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.620+0000 7fb4a698d740 1 -- 172.21.15.52:0/3139786940 >> 172.21.15.52:0/3139786940 conn(0x5626f3b75ed0 msgr2=0x5626f3b77c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.620+0000 7fb4a698d740 1 -- 172.21.15.52:0/3139786940 shutdown_connections 2024-04-02T13:56:19.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.620+0000 7fb4a698d740 1 -- 172.21.15.52:0/3139786940 wait complete. 2024-04-02T13:56:19.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.620+0000 7fb4a698d740 20 client.16428 trim_cache size 0 max 16384 2024-04-02T13:56:19.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.633+0000 7fb4a698d740 1 Processor -- start 2024-04-02T13:56:19.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.633+0000 7fb4a698d740 1 -- start start 2024-04-02T13:56:19.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.634+0000 7fb4a698d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b7b4d0 0x5626f3b9b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.634+0000 7fb4a698d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5626f3b7bdf0 0x5626f3b94d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.634+0000 7fb4a698d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626f3b890b0 0x5626f3b84ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.634+0000 7fb4a698d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c1acbc0 con 0x5626f3b7b4d0 2024-04-02T13:56:19.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.634+0000 7fb4a698d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c1ae730 con 0x5626f3b7bdf0 2024-04-02T13:56:19.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.634+0000 7fb4a698d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c1ad290 con 0x5626f3b890b0 2024-04-02T13:56:19.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.634+0000 7fb4a1343640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b7b4d0 0x5626f3b9b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.634+0000 7fb4a1343640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b7b4d0 0x5626f3b9b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37970/0 (socket says 172.21.15.52:37970) 2024-04-02T13:56:19.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.634+0000 7fb4a1343640 1 -- 172.21.15.52:0/810921786 learned_addr learned my addr 172.21.15.52:0/810921786 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:19.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.634+0000 7fb4a1343640 1 --2- 172.21.15.52:0/810921786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626f3b890b0 0x5626f3b84ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.634+0000 7fb4a1b44640 1 --2- 172.21.15.52:0/810921786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5626f3b7bdf0 0x5626f3b94d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.634+0000 7fb4a1343640 1 -- 172.21.15.52:0/810921786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626f3b890b0 msgr2=0x5626f3b84ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.634+0000 7fb4a1343640 1 --2- 172.21.15.52:0/810921786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626f3b890b0 0x5626f3b84ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.634+0000 7fb4a1343640 1 -- 172.21.15.52:0/810921786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5626f3b7bdf0 msgr2=0x5626f3b94d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.634+0000 7fb4a1343640 1 --2- 172.21.15.52:0/810921786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5626f3b7bdf0 0x5626f3b94d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.634+0000 7fb4a1343640 1 -- 172.21.15.52:0/810921786 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626f3b84440 con 0x5626f3b7b4d0 2024-04-02T13:56:19.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.634+0000 7fb4a1343640 1 --2- 172.21.15.52:0/810921786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b7b4d0 0x5626f3b9b860 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb48c0127c0 tx=0x7fb48c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a31faea65fd6ec server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.634+0000 7fb4877fe640 1 -- 172.21.15.52:0/810921786 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c034070 con 0x5626f3b7b4d0 2024-04-02T13:56:19.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.634+0000 7fb4877fe640 1 -- 172.21.15.52:0/810921786 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb48c005520 con 0x5626f3b7b4d0 2024-04-02T13:56:19.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.634+0000 7fb4877fe640 1 -- 172.21.15.52:0/810921786 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c010df0 con 0x5626f3b7b4d0 2024-04-02T13:56:19.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.635+0000 7fb4a698d740 1 -- 172.21.15.52:0/810921786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b7b4d0 msgr2=0x5626f3b9b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.635+0000 7fb4a698d740 1 --2- 172.21.15.52:0/810921786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b7b4d0 0x5626f3b9b860 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb48c0127c0 tx=0x7fb48c00b7c0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.635+0000 7fb4a698d740 1 -- 172.21.15.52:0/810921786 shutdown_connections 2024-04-02T13:56:19.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.635+0000 7fb4a698d740 1 --2- 172.21.15.52:0/810921786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626f3b890b0 0x5626f3b84ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.635+0000 7fb4a698d740 1 --2- 172.21.15.52:0/810921786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5626f3b7bdf0 0x5626f3b94d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.635+0000 7fb4a698d740 1 --2- 172.21.15.52:0/810921786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b7b4d0 0x5626f3b9b860 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.635+0000 7fb4a698d740 1 -- 172.21.15.52:0/810921786 >> 172.21.15.52:0/810921786 conn(0x5626f3b75ed0 msgr2=0x5626f3b847c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.635+0000 7fb4a698d740 1 -- 172.21.15.52:0/810921786 shutdown_connections 2024-04-02T13:56:19.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.635+0000 7fb4a698d740 1 -- 172.21.15.52:0/810921786 wait complete. 2024-04-02T13:56:19.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.635+0000 7fb4a698d740 1 -- start start 2024-04-02T13:56:19.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.635+0000 7fb4a698d740 1 Processor -- start 2024-04-02T13:56:19.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.636+0000 7fb4a698d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626f3b7b4d0 0x5626f3b82970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.636+0000 7fb4a698d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b7bdf0 0x5626f3b82eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.636+0000 7fb4a698d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5626f3b890b0 0x5626f3b90280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.636+0000 7fb4a698d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c1ae030 con 0x5626f3b7bdf0 2024-04-02T13:56:19.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.636+0000 7fb4a698d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c1ad960 con 0x5626f3b890b0 2024-04-02T13:56:19.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.636+0000 7fb4a698d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c1aabc0 con 0x5626f3b7b4d0 2024-04-02T13:56:19.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.636+0000 7fb4a698d740 10 client.0 _ll_register_callbacks cb 0x7fb498004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:19.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.636+0000 7fb4a1b44640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b7bdf0 0x5626f3b82eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.636+0000 7fb4a1b44640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b7bdf0 0x5626f3b82eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37976/0 (socket says 172.21.15.52:37976) 2024-04-02T13:56:19.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.636+0000 7fb4a1b44640 1 -- 172.21.15.52:0/1877525930 learned_addr learned my addr 172.21.15.52:0/1877525930 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:19.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.636+0000 7fb4a1343640 1 --2- 172.21.15.52:0/1877525930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626f3b7b4d0 0x5626f3b82970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.636+0000 7fb4a1b44640 1 -- 172.21.15.52:0/1877525930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626f3b7b4d0 msgr2=0x5626f3b82970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.636+0000 7fb4a1343640 1 --2- 172.21.15.52:0/1877525930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5626f3b890b0 0x5626f3b90280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.636+0000 7fb4a1b44640 1 --2- 172.21.15.52:0/1877525930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626f3b7b4d0 0x5626f3b82970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.636+0000 7fb4a1b44640 1 -- 172.21.15.52:0/1877525930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5626f3b890b0 msgr2=0x5626f3b90280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.636+0000 7fb4a1b44640 1 --2- 172.21.15.52:0/1877525930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5626f3b890b0 0x5626f3b90280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.636+0000 7fb4a1b44640 1 -- 172.21.15.52:0/1877525930 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626f3b907c0 con 0x5626f3b7bdf0 2024-04-02T13:56:19.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.636+0000 7fb4a1b44640 1 --2- 172.21.15.52:0/1877525930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b7bdf0 0x5626f3b82eb0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb498000c00 tx=0x7fb498003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa3cbb436980ad9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.636+0000 7fb4857fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:19.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.637+0000 7fb4857fa640 1 -- 172.21.15.52:0/1877525930 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498009520 con 0x5626f3b7bdf0 2024-04-02T13:56:19.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.637+0000 7fb4a698d740 1 -- 172.21.15.52:0/1877525930 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb48c005520 con 0x5626f3b7bdf0 2024-04-02T13:56:19.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.637+0000 7fb4857fa640 1 -- 172.21.15.52:0/1877525930 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb49802ca70 con 0x5626f3b7bdf0 2024-04-02T13:56:19.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.637+0000 7fb4857fa640 1 -- 172.21.15.52:0/1877525930 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb49802cc90 con 0x5626f3b7bdf0 2024-04-02T13:56:19.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.637+0000 7fb4a698d740 10 client.16434 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:19.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.637+0000 7fb4a698d740 1 -- 172.21.15.52:0/1877525930 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5626f3b91b50 con 0x5626f3b7bdf0 2024-04-02T13:56:19.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.637+0000 7fb4857fa640 1 -- 172.21.15.52:0/1877525930 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb48c005520 con 0x5626f3b7bdf0 2024-04-02T13:56:19.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.637+0000 7fb4857fa640 1 -- 172.21.15.52:0/1877525930 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fb498004b50 con 0x5626f3b7bdf0 2024-04-02T13:56:19.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.637+0000 7fb4a698d740 10 client.16434 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:19.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.637+0000 7fb4a698d740 10 client.16434 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T13:56:19.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.637+0000 7fb4a698d740 20 client.16434 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:19.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.637+0000 7fb4a698d740 1 --2- 172.21.15.52:0/1877525930 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5626f3bffef0 0x5626f3c02310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.637+0000 7fb4a698d740 4 client.16434 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:56:19.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.637+0000 7fb4a698d740 1 -- 172.21.15.52:0/1877525930 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5626f3ae7040 con 0x5626f3bffef0 2024-04-02T13:56:19.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.637+0000 7fb4a1343640 1 --2- 172.21.15.52:0/1877525930 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5626f3bffef0 0x5626f3c02310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.638+0000 7fb4857fa640 10 client.16434 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:56:19.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.638+0000 7fb4a1343640 1 --2- 172.21.15.52:0/1877525930 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5626f3bffef0 0x5626f3c02310 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=826a7b412fdc351b server_cookie=76d69fa6adbe5d6b in_seq=0 out_seq=0 2024-04-02T13:56:19.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.638+0000 7fb4857fa640 1 -- 172.21.15.52:0/1877525930 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5626f3ae7040 con 0x5626f3bffef0 2024-04-02T13:56:19.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.638+0000 7fb4857fa640 10 client.16434 handle_command_reply: tid=0 2024-04-02T13:56:19.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.638+0000 7fb4a698d740 1 client.16434 shutdown 2024-04-02T13:56:19.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.638+0000 7fb4a698d740 1 -- 172.21.15.52:0/1877525930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b7bdf0 msgr2=0x5626f3b82eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.638+0000 7fb4a698d740 1 --2- 172.21.15.52:0/1877525930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b7bdf0 0x5626f3b82eb0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb498000c00 tx=0x7fb498003920 comp rx=0 tx=0).stop 2024-04-02T13:56:19.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.638+0000 7fb4a698d740 1 -- 172.21.15.52:0/1877525930 shutdown_connections 2024-04-02T13:56:19.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.638+0000 7fb4a698d740 1 --2- 172.21.15.52:0/1877525930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5626f3b890b0 0x5626f3b90280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.638+0000 7fb4a698d740 1 --2- 172.21.15.52:0/1877525930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626f3b7bdf0 0x5626f3b82eb0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.638+0000 7fb4a698d740 1 --2- 172.21.15.52:0/1877525930 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5626f3bffef0 0x5626f3c02310 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.638+0000 7fb4a698d740 1 --2- 172.21.15.52:0/1877525930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626f3b7b4d0 0x5626f3b82970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.638+0000 7fb4a698d740 1 -- 172.21.15.52:0/1877525930 >> 172.21.15.52:0/1877525930 conn(0x5626f3b75ed0 msgr2=0x5626f3b746a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.638+0000 7fb4857fa640 0 client.16434 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:56:19.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.638+0000 7fb4a698d740 1 -- 172.21.15.52:0/1877525930 shutdown_connections 2024-04-02T13:56:19.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.638+0000 7fb4a698d740 1 -- 172.21.15.52:0/1877525930 wait complete. 2024-04-02T13:56:19.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.638+0000 7fb4a698d740 20 client.16434 trim_cache size 0 max 16384 2024-04-02T13:56:19.654 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:56:19.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.639+0000 7fb4a3da8640 1 -- 172.21.15.52:0/1053331556 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb4880a3d20 msgr2=0x7fb4880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.639+0000 7fb4a3da8640 1 --2- 172.21.15.52:0/1053331556 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb4880a3d20 0x7fb4880a61e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb49400efd0 tx=0x7fb494087040 comp rx=0 tx=0).stop 2024-04-02T13:56:19.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.639+0000 7fb4a3da8640 1 -- 172.21.15.52:0/1053331556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb49c107b70 msgr2=0x7fb49c19cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.639+0000 7fb4a3da8640 1 --2- 172.21.15.52:0/1053331556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb49c107b70 0x7fb49c19cbc0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb49400eaa0 tx=0x7fb49400ef60 comp rx=0 tx=0).stop 2024-04-02T13:56:19.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.639+0000 7fb4a3da8640 1 -- 172.21.15.52:0/1053331556 shutdown_connections 2024-04-02T13:56:19.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.639+0000 7fb4a3da8640 1 --2- 172.21.15.52:0/1053331556 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb4880a3d20 0x7fb4880a61e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.639+0000 7fb4a3da8640 1 --2- 172.21.15.52:0/1053331556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb49c107b70 0x7fb49c19cbc0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.639+0000 7fb4a3da8640 1 --2- 172.21.15.52:0/1053331556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb49c06ab70 0x7fb49c198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.639+0000 7fb4a3da8640 1 --2- 172.21.15.52:0/1053331556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb49c06a1b0 0x7fb49c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.639+0000 7fb4a3da8640 1 -- 172.21.15.52:0/1053331556 >> 172.21.15.52:0/1053331556 conn(0x7fb49c077ad0 msgr2=0x7fb49c0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.639+0000 7fb4a3da8640 1 -- 172.21.15.52:0/1053331556 shutdown_connections 2024-04-02T13:56:19.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.639+0000 7fb4a3da8640 1 -- 172.21.15.52:0/1053331556 wait complete. 2024-04-02T13:56:19.657 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:56:19.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.759+0000 7ff92b4b1640 1 -- 172.21.15.52:0/3444876818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff924101a90 msgr2=0x7ff924101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.759+0000 7ff92b4b1640 1 --2- 172.21.15.52:0/3444876818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff924101a90 0x7ff924101f10 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ff914009930 tx=0x7ff91402bc30 comp rx=0 tx=0).stop 2024-04-02T13:56:19.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.759+0000 7ff92b4b1640 1 -- 172.21.15.52:0/3444876818 shutdown_connections 2024-04-02T13:56:19.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.759+0000 7ff92b4b1640 1 --2- 172.21.15.52:0/3444876818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff924102450 0x7ff92410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.759+0000 7ff92b4b1640 1 --2- 172.21.15.52:0/3444876818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff924101a90 0x7ff924101f10 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.759+0000 7ff92b4b1640 1 --2- 172.21.15.52:0/3444876818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff924107aa0 0x7ff924107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.759+0000 7ff92b4b1640 1 -- 172.21.15.52:0/3444876818 >> 172.21.15.52:0/3444876818 conn(0x7ff924078f00 msgr2=0x7ff924079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.759+0000 7ff92b4b1640 1 -- 172.21.15.52:0/3444876818 shutdown_connections 2024-04-02T13:56:19.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.759+0000 7ff92b4b1640 1 -- 172.21.15.52:0/3444876818 wait complete. 2024-04-02T13:56:19.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.760+0000 7ff92b4b1640 1 Processor -- start 2024-04-02T13:56:19.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.760+0000 7ff92b4b1640 1 -- start start 2024-04-02T13:56:19.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.760+0000 7ff92b4b1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff924101a90 0x7ff924198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.760+0000 7ff92b4b1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff924102450 0x7ff924198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.760+0000 7ff92b4b1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff924107aa0 0x7ff92419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.760+0000 7ff92b4b1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff924111eb0 con 0x7ff924101a90 2024-04-02T13:56:19.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.760+0000 7ff92b4b1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff924111d30 con 0x7ff924107aa0 2024-04-02T13:56:19.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.760+0000 7ff92b4b1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff924112030 con 0x7ff924102450 2024-04-02T13:56:19.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.760+0000 7ff92924d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff924101a90 0x7ff924198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.760+0000 7ff92924d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff924101a90 0x7ff924198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37992/0 (socket says 172.21.15.52:37992) 2024-04-02T13:56:19.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.760+0000 7ff92924d640 1 -- 172.21.15.52:0/2023462735 learned_addr learned my addr 172.21.15.52:0/2023462735 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:19.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.760+0000 7ff928a4c640 1 --2- 172.21.15.52:0/2023462735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff924102450 0x7ff924198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.761+0000 7ff929a4e640 1 --2- 172.21.15.52:0/2023462735 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff924107aa0 0x7ff92419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.761+0000 7ff92924d640 1 -- 172.21.15.52:0/2023462735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff924102450 msgr2=0x7ff924198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.761+0000 7ff92924d640 1 --2- 172.21.15.52:0/2023462735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff924102450 0x7ff924198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.761+0000 7ff92924d640 1 -- 172.21.15.52:0/2023462735 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff924107aa0 msgr2=0x7ff92419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.761+0000 7ff92924d640 1 --2- 172.21.15.52:0/2023462735 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff924107aa0 0x7ff92419cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.761+0000 7ff92924d640 1 -- 172.21.15.52:0/2023462735 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92419d320 con 0x7ff924101a90 2024-04-02T13:56:19.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.761+0000 7ff929a4e640 1 --2- 172.21.15.52:0/2023462735 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff924107aa0 0x7ff92419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:56:19.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.761+0000 7ff928a4c640 1 --2- 172.21.15.52:0/2023462735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff924102450 0x7ff924198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:19.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.761+0000 7ff92924d640 1 --2- 172.21.15.52:0/2023462735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff924101a90 0x7ff924198330 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ff92000d770 tx=0x7ff92000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.761+0000 7ff91a7fc640 1 -- 172.21.15.52:0/2023462735 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920017070 con 0x7ff924101a90 2024-04-02T13:56:19.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.761+0000 7ff91a7fc640 1 -- 172.21.15.52:0/2023462735 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff920005500 con 0x7ff924101a90 2024-04-02T13:56:19.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.761+0000 7ff92b4b1640 1 -- 172.21.15.52:0/2023462735 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff92419d610 con 0x7ff924101a90 2024-04-02T13:56:19.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.761+0000 7ff91a7fc640 1 -- 172.21.15.52:0/2023462735 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920004e00 con 0x7ff924101a90 2024-04-02T13:56:19.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.761+0000 7ff92b4b1640 1 -- 172.21.15.52:0/2023462735 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9241a4ea0 con 0x7ff924101a90 2024-04-02T13:56:19.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.764+0000 7ff91a7fc640 1 -- 172.21.15.52:0/2023462735 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff920004fa0 con 0x7ff924101a90 2024-04-02T13:56:19.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.764+0000 7ff92e175740 1 Processor -- start 2024-04-02T13:56:19.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.764+0000 7ff92e175740 1 -- start start 2024-04-02T13:56:19.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.764+0000 7ff92e175740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81cee07a0 0x55f81cee0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.764+0000 7ff92e175740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f81cee10c0 0x55f81cee9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.764+0000 7ff92e175740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee9ca0 0x55f81ceec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.764+0000 7ff92e175740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f81ce447b0 con 0x55f81cee9ca0 2024-04-02T13:56:19.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.764+0000 7ff92e175740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f81cde3570 con 0x55f81cee10c0 2024-04-02T13:56:19.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.764+0000 7ff92e175740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f81ce40760 con 0x55f81cee07a0 2024-04-02T13:56:19.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.764+0000 7ff928a4c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee9ca0 0x55f81ceec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.764+0000 7ff928a4c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee9ca0 0x55f81ceec090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37998/0 (socket says 172.21.15.52:37998) 2024-04-02T13:56:19.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.764+0000 7ff928a4c640 1 -- 172.21.15.52:0/1615884663 learned_addr learned my addr 172.21.15.52:0/1615884663 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:19.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.764+0000 7ff929a4e640 1 --2- 172.21.15.52:0/1615884663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81cee07a0 0x55f81cee0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.764+0000 7ff91a7fc640 1 --2- 172.21.15.52:0/2023462735 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff9100a3d20 0x7ff9100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.764+0000 7ff92924d640 1 --2- 172.21.15.52:0/1615884663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f81cee10c0 0x55f81cee9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.764+0000 7ff928a4c640 1 -- 172.21.15.52:0/1615884663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81cee07a0 msgr2=0x55f81cee0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.764+0000 7ff928a4c640 1 --2- 172.21.15.52:0/1615884663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81cee07a0 0x55f81cee0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.764+0000 7ff928a4c640 1 -- 172.21.15.52:0/1615884663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f81cee10c0 msgr2=0x55f81cee9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.764+0000 7ff928a4c640 1 --2- 172.21.15.52:0/1615884663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f81cee10c0 0x55f81cee9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.764+0000 7ff928a4c640 1 -- 172.21.15.52:0/1615884663 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f81cb583f0 con 0x55f81cee9ca0 2024-04-02T13:56:19.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.764+0000 7ff929a4e640 1 --2- 172.21.15.52:0/1615884663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81cee07a0 0x55f81cee0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:19.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.764+0000 7ff92924d640 1 --2- 172.21.15.52:0/1615884663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f81cee10c0 0x55f81cee9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:19.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.764+0000 7ff929a4e640 1 --2- 172.21.15.52:0/2023462735 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff9100a3d20 0x7ff9100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.764+0000 7ff91a7fc640 1 -- 172.21.15.52:0/2023462735 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7ff9200d05a0 con 0x7ff924101a90 2024-04-02T13:56:19.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.765+0000 7ff928a4c640 1 --2- 172.21.15.52:0/1615884663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee9ca0 0x55f81ceec090 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ff914009930 tx=0x7ff91402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1718589023039b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.765+0000 7ff90f7fe640 1 -- 172.21.15.52:0/1615884663 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91402cbc0 con 0x55f81cee9ca0 2024-04-02T13:56:19.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.765+0000 7ff90f7fe640 1 -- 172.21.15.52:0/1615884663 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff914005040 con 0x55f81cee9ca0 2024-04-02T13:56:19.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.765+0000 7ff90f7fe640 1 -- 172.21.15.52:0/1615884663 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914004a40 con 0x55f81cee9ca0 2024-04-02T13:56:19.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.765+0000 7ff92e175740 1 -- 172.21.15.52:0/1615884663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee9ca0 msgr2=0x55f81ceec090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.765+0000 7ff929a4e640 1 --2- 172.21.15.52:0/2023462735 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff9100a3d20 0x7ff9100a61e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff91c005fd0 tx=0x7ff91c004a40 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.765+0000 7ff92e175740 1 --2- 172.21.15.52:0/1615884663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee9ca0 0x55f81ceec090 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ff914009930 tx=0x7ff91402c650 comp rx=0 tx=0).stop 2024-04-02T13:56:19.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.766+0000 7ff92e175740 1 -- 172.21.15.52:0/1615884663 shutdown_connections 2024-04-02T13:56:19.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.766+0000 7ff92e175740 1 --2- 172.21.15.52:0/1615884663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee9ca0 0x55f81ceec090 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.766+0000 7ff92e175740 1 --2- 172.21.15.52:0/1615884663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f81cee10c0 0x55f81cee9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.766+0000 7ff92e175740 1 --2- 172.21.15.52:0/1615884663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81cee07a0 0x55f81cee0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.766+0000 7ff92e175740 1 -- 172.21.15.52:0/1615884663 >> 172.21.15.52:0/1615884663 conn(0x55f81cedaed0 msgr2=0x55f81cedb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.766+0000 7ff92e175740 1 -- 172.21.15.52:0/1615884663 shutdown_connections 2024-04-02T13:56:19.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.766+0000 7ff92e175740 1 -- 172.21.15.52:0/1615884663 wait complete. 2024-04-02T13:56:19.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.766+0000 7ff92e175740 1 -- start start 2024-04-02T13:56:19.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.766+0000 7ff92e175740 1 Processor -- start 2024-04-02T13:56:19.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.766+0000 7ff92e175740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee07a0 0x55f81cee96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.766+0000 7ff92e175740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81cee10c0 0x55f81cee6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.766+0000 7ff92e175740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f81cee9ca0 0x55f81cee7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.766+0000 7ff92e175740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f81ceeca30 con 0x55f81cee07a0 2024-04-02T13:56:19.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.766+0000 7ff92e175740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f81cdffe80 con 0x55f81cee9ca0 2024-04-02T13:56:19.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.766+0000 7ff92e175740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f81ceecbb0 con 0x55f81cee10c0 2024-04-02T13:56:19.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.766+0000 7ff92e175740 10 client.0 _ll_register_callbacks cb 0x55f81ce3baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:19.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.766+0000 7ff92924d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee07a0 0x55f81cee96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.766+0000 7ff92924d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee07a0 0x55f81cee96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38004/0 (socket says 172.21.15.52:38004) 2024-04-02T13:56:19.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.766+0000 7ff92924d640 1 -- 172.21.15.52:0/3994479503 learned_addr learned my addr 172.21.15.52:0/3994479503 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:19.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.767+0000 7ff928a4c640 1 --2- 172.21.15.52:0/3994479503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81cee10c0 0x55f81cee6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.767+0000 7ff929a4e640 1 --2- 172.21.15.52:0/3994479503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f81cee9ca0 0x55f81cee7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.767+0000 7ff92924d640 1 -- 172.21.15.52:0/3994479503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81cee10c0 msgr2=0x55f81cee6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.767+0000 7ff92924d640 1 --2- 172.21.15.52:0/3994479503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81cee10c0 0x55f81cee6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.767+0000 7ff92924d640 1 -- 172.21.15.52:0/3994479503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f81cee9ca0 msgr2=0x55f81cee7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.767+0000 7ff92924d640 1 --2- 172.21.15.52:0/3994479503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f81cee9ca0 0x55f81cee7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.767+0000 7ff92924d640 1 -- 172.21.15.52:0/3994479503 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f81cc227d0 con 0x55f81cee07a0 2024-04-02T13:56:19.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.767+0000 7ff929a4e640 1 --2- 172.21.15.52:0/3994479503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f81cee9ca0 0x55f81cee7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:56:19.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.767+0000 7ff92924d640 1 --2- 172.21.15.52:0/3994479503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee07a0 0x55f81cee96b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff920098bd0 tx=0x7ff9200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed71462399a63674 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.767+0000 7ff90cff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:19.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.767+0000 7ff90cff9640 1 -- 172.21.15.52:0/3994479503 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920020070 con 0x55f81cee07a0 2024-04-02T13:56:19.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.767+0000 7ff90cff9640 1 -- 172.21.15.52:0/3994479503 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7ff9200cdc30 con 0x55f81cee07a0 2024-04-02T13:56:19.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.767+0000 7ff92e175740 1 -- 172.21.15.52:0/3994479503 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff914005040 con 0x55f81cee07a0 2024-04-02T13:56:19.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.767+0000 7ff90cff9640 1 -- 172.21.15.52:0/3994479503 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff92009c200 con 0x55f81cee07a0 2024-04-02T13:56:19.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.767+0000 7ff92e175740 10 client.16446 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:19.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.767+0000 7ff92e175740 1 -- 172.21.15.52:0/3994479503 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f81cef8a90 con 0x55f81cee07a0 2024-04-02T13:56:19.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.767+0000 7ff90cff9640 1 -- 172.21.15.52:0/3994479503 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff914005040 con 0x55f81cee07a0 2024-04-02T13:56:19.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.768+0000 7ff90cff9640 1 -- 172.21.15.52:0/3994479503 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7ff9200a0130 con 0x55f81cee07a0 2024-04-02T13:56:19.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.768+0000 7ff92e175740 10 client.16446 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:19.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.768+0000 7ff92e175740 10 client.16446 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T13:56:19.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.768+0000 7ff92e175740 20 client.16446 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:19.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.768+0000 7ff92e175740 1 --2- 172.21.15.52:0/3994479503 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55f81cef9f50 0x55f81cefc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.768+0000 7ff92e175740 4 client.16446 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:19.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.768+0000 7ff92e175740 1 -- 172.21.15.52:0/3994479503 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f81ce51cf0 con 0x55f81cef9f50 2024-04-02T13:56:19.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.768+0000 7ff928a4c640 1 --2- 172.21.15.52:0/3994479503 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55f81cef9f50 0x55f81cefc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.768+0000 7ff90cff9640 10 client.16446 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:56:19.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.768+0000 7ff928a4c640 1 --2- 172.21.15.52:0/3994479503 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55f81cef9f50 0x55f81cefc340 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=acc13cea520a59ae server_cookie=7b7642d0b4620a4d in_seq=0 out_seq=0 2024-04-02T13:56:19.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.769+0000 7ff90cff9640 1 -- 172.21.15.52:0/3994479503 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f81ce51cf0 con 0x55f81cef9f50 2024-04-02T13:56:19.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.769+0000 7ff90cff9640 10 client.16446 handle_command_reply: tid=0 2024-04-02T13:56:19.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.769+0000 7ff92e175740 1 client.16446 shutdown 2024-04-02T13:56:19.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.769+0000 7ff92e175740 1 -- 172.21.15.52:0/3994479503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee07a0 msgr2=0x55f81cee96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.769+0000 7ff92e175740 1 --2- 172.21.15.52:0/3994479503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee07a0 0x55f81cee96b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff920098bd0 tx=0x7ff9200046c0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.769+0000 7ff92e175740 1 -- 172.21.15.52:0/3994479503 shutdown_connections 2024-04-02T13:56:19.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.769+0000 7ff92e175740 1 --2- 172.21.15.52:0/3994479503 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55f81cef9f50 0x55f81cefc340 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.769+0000 7ff92e175740 1 --2- 172.21.15.52:0/3994479503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f81cee9ca0 0x55f81cee7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.769+0000 7ff92e175740 1 --2- 172.21.15.52:0/3994479503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81cee10c0 0x55f81cee6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.769+0000 7ff90cff9640 0 client.16446 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:56:19.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.769+0000 7ff92e175740 1 --2- 172.21.15.52:0/3994479503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee07a0 0x55f81cee96b0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.769+0000 7ff92e175740 1 -- 172.21.15.52:0/3994479503 >> 172.21.15.52:0/3994479503 conn(0x55f81cedaed0 msgr2=0x55f81cedcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.770+0000 7ff92e175740 1 -- 172.21.15.52:0/3994479503 shutdown_connections 2024-04-02T13:56:19.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.770+0000 7ff92e175740 1 -- 172.21.15.52:0/3994479503 wait complete. 2024-04-02T13:56:19.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.770+0000 7ff92e175740 20 client.16446 trim_cache size 0 max 16384 2024-04-02T13:56:19.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:19 smithi189 ceph-mon[36683]: pgmap v87: 129 pgs: 129 active+clean; 71 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 372 KiB/s wr, 58 op/s 2024-04-02T13:56:19.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:19 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1872746701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:56:19.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.783+0000 7ff92e175740 1 Processor -- start 2024-04-02T13:56:19.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.783+0000 7ff92e175740 1 -- start start 2024-04-02T13:56:19.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.783+0000 7ff92e175740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f81cee07a0 0x55f81cef5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.783+0000 7ff92e175740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee10c0 0x55f81cee6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.783+0000 7ff92e175740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81cee9ca0 0x55f81cee9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.783+0000 7ff92e175740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241aed40 con 0x55f81cee10c0 2024-04-02T13:56:19.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.783+0000 7ff92e175740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241b08b0 con 0x55f81cee07a0 2024-04-02T13:56:19.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.783+0000 7ff92e175740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241af410 con 0x55f81cee9ca0 2024-04-02T13:56:19.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.783+0000 7ff928a4c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee10c0 0x55f81cee6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.783+0000 7ff928a4c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee10c0 0x55f81cee6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38016/0 (socket says 172.21.15.52:38016) 2024-04-02T13:56:19.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.783+0000 7ff928a4c640 1 -- 172.21.15.52:0/1762930795 learned_addr learned my addr 172.21.15.52:0/1762930795 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:19.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.784+0000 7ff929a4e640 1 --2- 172.21.15.52:0/1762930795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81cee9ca0 0x55f81cee9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.784+0000 7ff92924d640 1 --2- 172.21.15.52:0/1762930795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f81cee07a0 0x55f81cef5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.784+0000 7ff928a4c640 1 -- 172.21.15.52:0/1762930795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81cee9ca0 msgr2=0x55f81cee9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.784+0000 7ff928a4c640 1 --2- 172.21.15.52:0/1762930795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81cee9ca0 0x55f81cee9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.784+0000 7ff928a4c640 1 -- 172.21.15.52:0/1762930795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f81cee07a0 msgr2=0x55f81cef5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.784+0000 7ff928a4c640 1 --2- 172.21.15.52:0/1762930795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f81cee07a0 0x55f81cef5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.784+0000 7ff928a4c640 1 -- 172.21.15.52:0/1762930795 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f81cee73c0 con 0x55f81cee10c0 2024-04-02T13:56:19.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.784+0000 7ff929a4e640 1 --2- 172.21.15.52:0/1762930795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81cee9ca0 0x55f81cee9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:56:19.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.784+0000 7ff928a4c640 1 --2- 172.21.15.52:0/1762930795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee10c0 0x55f81cee6a00 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff914004620 tx=0x7ff91402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e4b47150761e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.784+0000 7ff90effd640 1 -- 172.21.15.52:0/1762930795 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91402cae0 con 0x55f81cee10c0 2024-04-02T13:56:19.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.784+0000 7ff90effd640 1 -- 172.21.15.52:0/1762930795 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff914005030 con 0x55f81cee10c0 2024-04-02T13:56:19.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.784+0000 7ff90effd640 1 -- 172.21.15.52:0/1762930795 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914004920 con 0x55f81cee10c0 2024-04-02T13:56:19.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.784+0000 7ff92e175740 1 -- 172.21.15.52:0/1762930795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee10c0 msgr2=0x55f81cee6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.784+0000 7ff92e175740 1 --2- 172.21.15.52:0/1762930795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee10c0 0x55f81cee6a00 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff914004620 tx=0x7ff91402bc30 comp rx=0 tx=0).stop 2024-04-02T13:56:19.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.784+0000 7ff92e175740 1 -- 172.21.15.52:0/1762930795 shutdown_connections 2024-04-02T13:56:19.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.784+0000 7ff92e175740 1 --2- 172.21.15.52:0/1762930795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81cee9ca0 0x55f81cee9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.784+0000 7ff92e175740 1 --2- 172.21.15.52:0/1762930795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee10c0 0x55f81cee6a00 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.784+0000 7ff92e175740 1 --2- 172.21.15.52:0/1762930795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f81cee07a0 0x55f81cef5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.785+0000 7ff92e175740 1 -- 172.21.15.52:0/1762930795 >> 172.21.15.52:0/1762930795 conn(0x55f81cedaed0 msgr2=0x55f81cee5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.785+0000 7ff92e175740 1 -- 172.21.15.52:0/1762930795 shutdown_connections 2024-04-02T13:56:19.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.785+0000 7ff92e175740 1 -- 172.21.15.52:0/1762930795 wait complete. 2024-04-02T13:56:19.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.785+0000 7ff92e175740 1 -- start start 2024-04-02T13:56:19.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.785+0000 7ff92e175740 1 Processor -- start 2024-04-02T13:56:19.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.785+0000 7ff92e175740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee07a0 0x55f81cef1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.785+0000 7ff92e175740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81cee10c0 0x55f81cef2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.785+0000 7ff92e175740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f81cee9ca0 0x55f81cee3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.785+0000 7ff92e175740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241b01b0 con 0x55f81cee07a0 2024-04-02T13:56:19.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.785+0000 7ff92e175740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241afae0 con 0x55f81cee9ca0 2024-04-02T13:56:19.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.785+0000 7ff92e175740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241acd20 con 0x55f81cee10c0 2024-04-02T13:56:19.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.785+0000 7ff92e175740 10 client.0 _ll_register_callbacks cb 0x7ff92000de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:19.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.785+0000 7ff92924d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee07a0 0x55f81cef1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.785+0000 7ff92924d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee07a0 0x55f81cef1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38032/0 (socket says 172.21.15.52:38032) 2024-04-02T13:56:19.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.785+0000 7ff92924d640 1 -- 172.21.15.52:0/3026836691 learned_addr learned my addr 172.21.15.52:0/3026836691 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:19.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.786+0000 7ff928a4c640 1 --2- 172.21.15.52:0/3026836691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81cee10c0 0x55f81cef2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.786+0000 7ff929a4e640 1 --2- 172.21.15.52:0/3026836691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f81cee9ca0 0x55f81cee3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.786+0000 7ff92924d640 1 -- 172.21.15.52:0/3026836691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81cee10c0 msgr2=0x55f81cef2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.786+0000 7ff92924d640 1 --2- 172.21.15.52:0/3026836691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81cee10c0 0x55f81cef2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.786+0000 7ff92924d640 1 -- 172.21.15.52:0/3026836691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f81cee9ca0 msgr2=0x55f81cee3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.786+0000 7ff92924d640 1 --2- 172.21.15.52:0/3026836691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f81cee9ca0 0x55f81cee3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.786+0000 7ff92924d640 1 -- 172.21.15.52:0/3026836691 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f81cee45c0 con 0x55f81cee07a0 2024-04-02T13:56:19.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.786+0000 7ff92924d640 1 --2- 172.21.15.52:0/3026836691 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee07a0 0x55f81cef1e30 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff92009c3c0 tx=0x7ff920009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b313883ec283c7a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.786+0000 7ff90cff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:19.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.786+0000 7ff90cff9640 1 -- 172.21.15.52:0/3026836691 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920020070 con 0x55f81cee07a0 2024-04-02T13:56:19.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.786+0000 7ff90cff9640 1 -- 172.21.15.52:0/3026836691 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7ff92009c530 con 0x55f81cee07a0 2024-04-02T13:56:19.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.786+0000 7ff92e175740 1 -- 172.21.15.52:0/3026836691 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff914005030 con 0x55f81cee07a0 2024-04-02T13:56:19.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.786+0000 7ff90cff9640 1 -- 172.21.15.52:0/3026836691 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff9200a4e70 con 0x55f81cee07a0 2024-04-02T13:56:19.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.786+0000 7ff92e175740 10 client.16452 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:19.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.786+0000 7ff92e175740 1 -- 172.21.15.52:0/3026836691 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f81ceeaf80 con 0x55f81cee07a0 2024-04-02T13:56:19.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.786+0000 7ff90cff9640 1 -- 172.21.15.52:0/3026836691 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff914005030 con 0x55f81cee07a0 2024-04-02T13:56:19.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.786+0000 7ff90cff9640 1 -- 172.21.15.52:0/3026836691 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7ff92009b650 con 0x55f81cee07a0 2024-04-02T13:56:19.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.786+0000 7ff92e175740 10 client.16452 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:19.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.786+0000 7ff92e175740 10 client.16452 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T13:56:19.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.786+0000 7ff92e175740 20 client.16452 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:19.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.786+0000 7ff92e175740 1 --2- 172.21.15.52:0/3026836691 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55f81cf60500 0x55f81cf628f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.786+0000 7ff92e175740 4 client.16452 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:56:19.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.787+0000 7ff92e175740 1 -- 172.21.15.52:0/3026836691 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f81ce4c040 con 0x55f81cf60500 2024-04-02T13:56:19.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.787+0000 7ff928a4c640 1 --2- 172.21.15.52:0/3026836691 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55f81cf60500 0x55f81cf628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.787+0000 7ff928a4c640 1 --2- 172.21.15.52:0/3026836691 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55f81cf60500 0x55f81cf628f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=64ec7c322fecf7d server_cookie=ce9f1b2c24e8c326 in_seq=0 out_seq=0 2024-04-02T13:56:19.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.787+0000 7ff90cff9640 10 client.16452 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:56:19.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.787+0000 7ff90cff9640 1 -- 172.21.15.52:0/3026836691 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f81ce4c040 con 0x55f81cf60500 2024-04-02T13:56:19.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.787+0000 7ff90cff9640 10 client.16452 handle_command_reply: tid=0 2024-04-02T13:56:19.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.787+0000 7ff92e175740 1 client.16452 shutdown 2024-04-02T13:56:19.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.787+0000 7ff92e175740 1 -- 172.21.15.52:0/3026836691 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee07a0 msgr2=0x55f81cef1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.787+0000 7ff92e175740 1 --2- 172.21.15.52:0/3026836691 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee07a0 0x55f81cef1e30 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff92009c3c0 tx=0x7ff920009ef0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.787+0000 7ff92e175740 1 -- 172.21.15.52:0/3026836691 shutdown_connections 2024-04-02T13:56:19.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.787+0000 7ff92e175740 1 --2- 172.21.15.52:0/3026836691 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55f81cf60500 0x55f81cf628f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.787+0000 7ff92e175740 1 --2- 172.21.15.52:0/3026836691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f81cee9ca0 0x55f81cee3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.787+0000 7ff92e175740 1 --2- 172.21.15.52:0/3026836691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f81cee10c0 0x55f81cef2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.787+0000 7ff92e175740 1 --2- 172.21.15.52:0/3026836691 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f81cee07a0 0x55f81cef1e30 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.787+0000 7ff90cff9640 0 client.16452 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:56:19.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.788+0000 7ff92e175740 1 -- 172.21.15.52:0/3026836691 >> 172.21.15.52:0/3026836691 conn(0x55f81cedaed0 msgr2=0x55f81cedc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.788+0000 7ff92e175740 1 -- 172.21.15.52:0/3026836691 shutdown_connections 2024-04-02T13:56:19.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.788+0000 7ff92e175740 1 -- 172.21.15.52:0/3026836691 wait complete. 2024-04-02T13:56:19.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.788+0000 7ff92e175740 20 client.16452 trim_cache size 0 max 16384 2024-04-02T13:56:19.805 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:56:19.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.788+0000 7ff92b4b1640 1 -- 172.21.15.52:0/2023462735 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff9100a3d20 msgr2=0x7ff9100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.788+0000 7ff92b4b1640 1 --2- 172.21.15.52:0/2023462735 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff9100a3d20 0x7ff9100a61e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff91c005fd0 tx=0x7ff91c004a40 comp rx=0 tx=0).stop 2024-04-02T13:56:19.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.788+0000 7ff92b4b1640 1 -- 172.21.15.52:0/2023462735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff924101a90 msgr2=0x7ff924198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.788+0000 7ff92b4b1640 1 --2- 172.21.15.52:0/2023462735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff924101a90 0x7ff924198330 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ff92000d770 tx=0x7ff92000dc30 comp rx=0 tx=0).stop 2024-04-02T13:56:19.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.788+0000 7ff92b4b1640 1 -- 172.21.15.52:0/2023462735 shutdown_connections 2024-04-02T13:56:19.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.788+0000 7ff92b4b1640 1 --2- 172.21.15.52:0/2023462735 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff9100a3d20 0x7ff9100a61e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.788+0000 7ff92b4b1640 1 --2- 172.21.15.52:0/2023462735 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff924107aa0 0x7ff92419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.788+0000 7ff92b4b1640 1 --2- 172.21.15.52:0/2023462735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff924102450 0x7ff924198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.788+0000 7ff92b4b1640 1 --2- 172.21.15.52:0/2023462735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff924101a90 0x7ff924198330 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.788+0000 7ff92b4b1640 1 -- 172.21.15.52:0/2023462735 >> 172.21.15.52:0/2023462735 conn(0x7ff924078f00 msgr2=0x7ff924104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.788+0000 7ff92b4b1640 1 -- 172.21.15.52:0/2023462735 shutdown_connections 2024-04-02T13:56:19.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.788+0000 7ff92b4b1640 1 -- 172.21.15.52:0/2023462735 wait complete. 2024-04-02T13:56:19.808 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:19 smithi052 ceph-mon[32104]: pgmap v87: 129 pgs: 129 active+clean; 71 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 372 KiB/s wr, 58 op/s 2024-04-02T13:56:19.808 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:19 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1872746701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:56:19.809 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:56:19.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.910+0000 7fe61776d640 1 -- 172.21.15.52:0/3349779218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe610103660 msgr2=0x7fe61010bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.910+0000 7fe61776d640 1 --2- 172.21.15.52:0/3349779218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe610103660 0x7fe61010bb40 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe600009900 tx=0x7fe60002bcd0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.910+0000 7fe61776d640 1 -- 172.21.15.52:0/3349779218 shutdown_connections 2024-04-02T13:56:19.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.910+0000 7fe61776d640 1 --2- 172.21.15.52:0/3349779218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe61010c110 0x7fe61010e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.910+0000 7fe61776d640 1 --2- 172.21.15.52:0/3349779218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe610103660 0x7fe61010bb40 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.910+0000 7fe61776d640 1 --2- 172.21.15.52:0/3349779218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe610102d40 0x7fe610103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.910+0000 7fe61776d640 1 -- 172.21.15.52:0/3349779218 >> 172.21.15.52:0/3349779218 conn(0x7fe610076d80 msgr2=0x7fe610077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.910+0000 7fe61776d640 1 -- 172.21.15.52:0/3349779218 shutdown_connections 2024-04-02T13:56:19.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.910+0000 7fe61776d640 1 -- 172.21.15.52:0/3349779218 wait complete. 2024-04-02T13:56:19.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.911+0000 7fe61776d640 1 Processor -- start 2024-04-02T13:56:19.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.911+0000 7fe61776d640 1 -- start start 2024-04-02T13:56:19.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.911+0000 7fe61776d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe610102d40 0x7fe6101983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.911+0000 7fe61776d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe610103660 0x7fe6101988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.911+0000 7fe61776d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe61010c110 0x7fe61019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.911+0000 7fe61776d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe610111ea0 con 0x7fe610102d40 2024-04-02T13:56:19.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.911+0000 7fe61776d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe610111d20 con 0x7fe610103660 2024-04-02T13:56:19.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.911+0000 7fe61776d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe610112020 con 0x7fe61010c110 2024-04-02T13:56:19.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.911+0000 7fe615509640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe610102d40 0x7fe6101983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.911+0000 7fe615509640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe610102d40 0x7fe6101983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38058/0 (socket says 172.21.15.52:38058) 2024-04-02T13:56:19.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.911+0000 7fe615509640 1 -- 172.21.15.52:0/1401058410 learned_addr learned my addr 172.21.15.52:0/1401058410 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:19.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.912+0000 7fe615d0a640 1 --2- 172.21.15.52:0/1401058410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe61010c110 0x7fe61019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.912+0000 7fe614d08640 1 --2- 172.21.15.52:0/1401058410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe610103660 0x7fe6101988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.912+0000 7fe615509640 1 -- 172.21.15.52:0/1401058410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe61010c110 msgr2=0x7fe61019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.912+0000 7fe615509640 1 --2- 172.21.15.52:0/1401058410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe61010c110 0x7fe61019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.912+0000 7fe615509640 1 -- 172.21.15.52:0/1401058410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe610103660 msgr2=0x7fe6101988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.912+0000 7fe615509640 1 --2- 172.21.15.52:0/1401058410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe610103660 0x7fe6101988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.912+0000 7fe615509640 1 -- 172.21.15.52:0/1401058410 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61019d390 con 0x7fe610102d40 2024-04-02T13:56:19.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.912+0000 7fe614d08640 1 --2- 172.21.15.52:0/1401058410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe610103660 0x7fe6101988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:19.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.912+0000 7fe615509640 1 --2- 172.21.15.52:0/1401058410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe610102d40 0x7fe6101983a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00e9f0 tx=0x7fe60c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.912+0000 7fe6067fc640 1 -- 172.21.15.52:0/1401058410 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c00cd80 con 0x7fe610102d40 2024-04-02T13:56:19.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.912+0000 7fe6067fc640 1 -- 172.21.15.52:0/1401058410 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe60c010910 con 0x7fe610102d40 2024-04-02T13:56:19.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.912+0000 7fe61776d640 1 -- 172.21.15.52:0/1401058410 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61019d680 con 0x7fe610102d40 2024-04-02T13:56:19.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.912+0000 7fe6067fc640 1 -- 172.21.15.52:0/1401058410 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c018990 con 0x7fe610102d40 2024-04-02T13:56:19.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.912+0000 7fe61776d640 1 -- 172.21.15.52:0/1401058410 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6101a4f10 con 0x7fe610102d40 2024-04-02T13:56:19.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:19 smithi088 ceph-mon[36528]: pgmap v87: 129 pgs: 129 active+clean; 71 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 372 KiB/s wr, 58 op/s 2024-04-02T13:56:19.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:19 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1872746701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:56:19.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.914+0000 7fe6067fc640 1 -- 172.21.15.52:0/1401058410 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe60c010ac0 con 0x7fe610102d40 2024-04-02T13:56:19.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.915+0000 7fe61a42a740 1 Processor -- start 2024-04-02T13:56:19.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.915+0000 7fe61a42a740 1 -- start start 2024-04-02T13:56:19.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.915+0000 7fe6067fc640 1 --2- 172.21.15.52:0/1401058410 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe5fc0a3d20 0x7fe5fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.915+0000 7fe61a42a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559217a27a0 0x5559217a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.915+0000 7fe61a42a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5559217a30c0 0x5559217ab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.915+0000 7fe615d0a640 1 --2- 172.21.15.52:0/1401058410 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe5fc0a3d20 0x7fe5fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.915+0000 7fe6067fc640 1 -- 172.21.15.52:0/1401058410 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe60c0d0fc0 con 0x7fe610102d40 2024-04-02T13:56:19.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.915+0000 7fe61a42a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217abca0 0x5559217ae090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.915+0000 7fe61a42a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5559217067b0 con 0x5559217abca0 2024-04-02T13:56:19.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.915+0000 7fe61a42a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5559216a5570 con 0x5559217a30c0 2024-04-02T13:56:19.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.915+0000 7fe61a42a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555921702760 con 0x5559217a27a0 2024-04-02T13:56:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.915+0000 7fe614d08640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5559217a30c0 0x5559217ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.915+0000 7fe615d0a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217abca0 0x5559217ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.915+0000 7fe614d08640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5559217a30c0 0x5559217ab760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:46884/0 (socket says 172.21.15.52:46884) 2024-04-02T13:56:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.915+0000 7fe614d08640 1 -- 172.21.15.52:0/903838861 learned_addr learned my addr 172.21.15.52:0/903838861 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.915+0000 7fe615509640 1 --2- 172.21.15.52:0/903838861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559217a27a0 0x5559217a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.916+0000 7fe614d08640 1 -- 172.21.15.52:0/903838861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559217a27a0 msgr2=0x5559217a2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.916+0000 7fe614d08640 1 --2- 172.21.15.52:0/903838861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559217a27a0 0x5559217a2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.916+0000 7fe615d0a640 1 --2- 172.21.15.52:0/1401058410 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe5fc0a3d20 0x7fe5fc0a61e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe6080092f0 tx=0x7fe608014a50 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.916+0000 7fe614d08640 1 -- 172.21.15.52:0/903838861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217abca0 msgr2=0x5559217ae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.916+0000 7fe614d08640 1 --2- 172.21.15.52:0/903838861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217abca0 0x5559217ae090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.916+0000 7fe614d08640 1 -- 172.21.15.52:0/903838861 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55592141a3f0 con 0x5559217a30c0 2024-04-02T13:56:19.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.916+0000 7fe615d0a640 1 --2- 172.21.15.52:0/903838861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217abca0 0x5559217ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:19.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.916+0000 7fe614d08640 1 --2- 172.21.15.52:0/903838861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5559217a30c0 0x5559217ab760 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe60002c210 tx=0x7fe600005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=515dd590896a0754 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.916+0000 7fe5fb7fe640 1 -- 172.21.15.52:0/903838861 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600042070 con 0x5559217a30c0 2024-04-02T13:56:19.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.916+0000 7fe5fb7fe640 1 -- 172.21.15.52:0/903838861 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe60002cb20 con 0x5559217a30c0 2024-04-02T13:56:19.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.916+0000 7fe5fb7fe640 1 -- 172.21.15.52:0/903838861 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600005620 con 0x5559217a30c0 2024-04-02T13:56:19.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.917+0000 7fe61a42a740 1 -- 172.21.15.52:0/903838861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5559217a30c0 msgr2=0x5559217ab760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.917+0000 7fe61a42a740 1 --2- 172.21.15.52:0/903838861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5559217a30c0 0x5559217ab760 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe60002c210 tx=0x7fe600005dd0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.917+0000 7fe61a42a740 1 -- 172.21.15.52:0/903838861 shutdown_connections 2024-04-02T13:56:19.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.917+0000 7fe61a42a740 1 --2- 172.21.15.52:0/903838861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217abca0 0x5559217ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.917+0000 7fe61a42a740 1 --2- 172.21.15.52:0/903838861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5559217a30c0 0x5559217ab760 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.917+0000 7fe61a42a740 1 --2- 172.21.15.52:0/903838861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559217a27a0 0x5559217a2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.917+0000 7fe61a42a740 1 -- 172.21.15.52:0/903838861 >> 172.21.15.52:0/903838861 conn(0x55592179ced0 msgr2=0x55592179d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.917+0000 7fe61a42a740 1 -- 172.21.15.52:0/903838861 shutdown_connections 2024-04-02T13:56:19.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.917+0000 7fe61a42a740 1 -- 172.21.15.52:0/903838861 wait complete. 2024-04-02T13:56:19.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.917+0000 7fe61a42a740 1 -- start start 2024-04-02T13:56:19.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.917+0000 7fe61a42a740 1 Processor -- start 2024-04-02T13:56:19.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.918+0000 7fe61a42a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5559217a27a0 0x5559217ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.918+0000 7fe61a42a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217a30c0 0x5559217a8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.918+0000 7fe61a42a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559217abca0 0x5559217a91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.918+0000 7fe61a42a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5559217b2e40 con 0x5559217a30c0 2024-04-02T13:56:19.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.918+0000 7fe61a42a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5559216c1e80 con 0x5559217a27a0 2024-04-02T13:56:19.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.918+0000 7fe61a42a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5559217b2fc0 con 0x5559217abca0 2024-04-02T13:56:19.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.918+0000 7fe61a42a740 10 client.0 _ll_register_callbacks cb 0x5559216fdaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:19.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.918+0000 7fe614d08640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217a30c0 0x5559217a8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.918+0000 7fe614d08640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217a30c0 0x5559217a8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38076/0 (socket says 172.21.15.52:38076) 2024-04-02T13:56:19.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.918+0000 7fe614d08640 1 -- 172.21.15.52:0/48729694 learned_addr learned my addr 172.21.15.52:0/48729694 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:19.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.918+0000 7fe615509640 1 --2- 172.21.15.52:0/48729694 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5559217a27a0 0x5559217ab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.918+0000 7fe615d0a640 1 --2- 172.21.15.52:0/48729694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559217abca0 0x5559217a91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.918+0000 7fe614d08640 1 -- 172.21.15.52:0/48729694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559217abca0 msgr2=0x5559217a91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.918+0000 7fe614d08640 1 --2- 172.21.15.52:0/48729694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559217abca0 0x5559217a91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.918+0000 7fe614d08640 1 -- 172.21.15.52:0/48729694 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5559217a27a0 msgr2=0x5559217ab6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.918+0000 7fe614d08640 1 --2- 172.21.15.52:0/48729694 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5559217a27a0 0x5559217ab6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.918+0000 7fe614d08640 1 -- 172.21.15.52:0/48729694 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559214e47d0 con 0x5559217a30c0 2024-04-02T13:56:19.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.918+0000 7fe5f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:19.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.918+0000 7fe614d08640 1 --2- 172.21.15.52:0/48729694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217a30c0 0x5559217a8a80 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe600005300 tx=0x7fe600005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926d7da05af1d3a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.918+0000 7fe5f8ff9640 1 -- 172.21.15.52:0/48729694 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600042050 con 0x5559217a30c0 2024-04-02T13:56:19.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.918+0000 7fe5f8ff9640 1 -- 172.21.15.52:0/48729694 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe600047860 con 0x5559217a30c0 2024-04-02T13:56:19.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.918+0000 7fe61a42a740 1 -- 172.21.15.52:0/48729694 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe60002cb20 con 0x5559217a30c0 2024-04-02T13:56:19.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.919+0000 7fe5f8ff9640 1 -- 172.21.15.52:0/48729694 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe600047a60 con 0x5559217a30c0 2024-04-02T13:56:19.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.919+0000 7fe61a42a740 10 client.16464 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:19.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.919+0000 7fe61a42a740 1 -- 172.21.15.52:0/48729694 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5559217beec0 con 0x5559217a30c0 2024-04-02T13:56:19.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.919+0000 7fe5f8ff9640 1 -- 172.21.15.52:0/48729694 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe60002cb20 con 0x5559217a30c0 2024-04-02T13:56:19.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.919+0000 7fe5f8ff9640 1 -- 172.21.15.52:0/48729694 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fe600034440 con 0x5559217a30c0 2024-04-02T13:56:19.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.919+0000 7fe61a42a740 10 client.16464 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:19.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.919+0000 7fe61a42a740 10 client.16464 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T13:56:19.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.919+0000 7fe61a42a740 20 client.16464 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:19.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.919+0000 7fe61a42a740 1 --2- 172.21.15.52:0/48729694 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5559217c0210 0x5559217c2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.919+0000 7fe61a42a740 4 client.16464 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.919+0000 7fe61a42a740 1 -- 172.21.15.52:0/48729694 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555921713cf0 con 0x5559217c0210 2024-04-02T13:56:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.919+0000 7fe615509640 1 --2- 172.21.15.52:0/48729694 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5559217c0210 0x5559217c2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.920+0000 7fe5f8ff9640 10 client.16464 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:56:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.920+0000 7fe615509640 1 --2- 172.21.15.52:0/48729694 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5559217c0210 0x5559217c2600 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7d93693cbb786bb0 server_cookie=4ad776a3e3ff8ac6 in_seq=0 out_seq=0 2024-04-02T13:56:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.920+0000 7fe5f8ff9640 1 -- 172.21.15.52:0/48729694 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555921713cf0 con 0x5559217c0210 2024-04-02T13:56:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.920+0000 7fe5f8ff9640 10 client.16464 handle_command_reply: tid=0 2024-04-02T13:56:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.920+0000 7fe61a42a740 1 client.16464 shutdown 2024-04-02T13:56:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.920+0000 7fe61a42a740 1 -- 172.21.15.52:0/48729694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217a30c0 msgr2=0x5559217a8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.920+0000 7fe61a42a740 1 --2- 172.21.15.52:0/48729694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217a30c0 0x5559217a8a80 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe600005300 tx=0x7fe600005780 comp rx=0 tx=0).stop 2024-04-02T13:56:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.920+0000 7fe61a42a740 1 -- 172.21.15.52:0/48729694 shutdown_connections 2024-04-02T13:56:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.920+0000 7fe61a42a740 1 --2- 172.21.15.52:0/48729694 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5559217c0210 0x5559217c2600 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.920+0000 7fe61a42a740 1 --2- 172.21.15.52:0/48729694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559217abca0 0x5559217a91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.921+0000 7fe61a42a740 1 --2- 172.21.15.52:0/48729694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217a30c0 0x5559217a8a80 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.921+0000 7fe5f8ff9640 0 client.16464 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:56:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.921+0000 7fe61a42a740 1 --2- 172.21.15.52:0/48729694 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5559217a27a0 0x5559217ab6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.921+0000 7fe61a42a740 1 -- 172.21.15.52:0/48729694 >> 172.21.15.52:0/48729694 conn(0x55592179ced0 msgr2=0x55592179ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.921+0000 7fe61a42a740 1 -- 172.21.15.52:0/48729694 shutdown_connections 2024-04-02T13:56:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.921+0000 7fe61a42a740 1 -- 172.21.15.52:0/48729694 wait complete. 2024-04-02T13:56:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.921+0000 7fe61a42a740 20 client.16464 trim_cache size 0 max 16384 2024-04-02T13:56:19.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.934+0000 7fe61a42a740 1 Processor -- start 2024-04-02T13:56:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.934+0000 7fe61a42a740 1 -- start start 2024-04-02T13:56:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.934+0000 7fe61a42a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559217a27a0 0x5559217ab1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.934+0000 7fe61a42a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217a30c0 0x5559217ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.934+0000 7fe61a42a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5559217abca0 0x5559217a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.934+0000 7fe61a42a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe6101acb60 con 0x5559217a30c0 2024-04-02T13:56:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.934+0000 7fe61a42a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe6101ae6d0 con 0x5559217abca0 2024-04-02T13:56:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.934+0000 7fe61a42a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe6101ad230 con 0x5559217a27a0 2024-04-02T13:56:19.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.934+0000 7fe614d08640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217a30c0 0x5559217ab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.934+0000 7fe614d08640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217a30c0 0x5559217ab6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38084/0 (socket says 172.21.15.52:38084) 2024-04-02T13:56:19.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.934+0000 7fe614d08640 1 -- 172.21.15.52:0/2914798388 learned_addr learned my addr 172.21.15.52:0/2914798388 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:19.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.935+0000 7fe615d0a640 1 --2- 172.21.15.52:0/2914798388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5559217abca0 0x5559217a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.935+0000 7fe614d08640 1 -- 172.21.15.52:0/2914798388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559217a27a0 msgr2=0x5559217ab1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.935+0000 7fe614d08640 1 --2- 172.21.15.52:0/2914798388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559217a27a0 0x5559217ab1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.935+0000 7fe614d08640 1 -- 172.21.15.52:0/2914798388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5559217abca0 msgr2=0x5559217a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.935+0000 7fe614d08640 1 --2- 172.21.15.52:0/2914798388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5559217abca0 0x5559217a5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.935+0000 7fe614d08640 1 -- 172.21.15.52:0/2914798388 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559217a9530 con 0x5559217a30c0 2024-04-02T13:56:19.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.935+0000 7fe615d0a640 1 --2- 172.21.15.52:0/2914798388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5559217abca0 0x5559217a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:56:19.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.935+0000 7fe614d08640 1 --2- 172.21.15.52:0/2914798388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217a30c0 0x5559217ab6f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe600034830 tx=0x7fe600005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246f9b04249683d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.935+0000 7fe5faffd640 1 -- 172.21.15.52:0/2914798388 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600042070 con 0x5559217a30c0 2024-04-02T13:56:19.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.935+0000 7fe5faffd640 1 -- 172.21.15.52:0/2914798388 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe600034aa0 con 0x5559217a30c0 2024-04-02T13:56:19.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.935+0000 7fe5faffd640 1 -- 172.21.15.52:0/2914798388 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60003d900 con 0x5559217a30c0 2024-04-02T13:56:19.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.935+0000 7fe61a42a740 1 -- 172.21.15.52:0/2914798388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217a30c0 msgr2=0x5559217ab6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.935+0000 7fe61a42a740 1 --2- 172.21.15.52:0/2914798388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217a30c0 0x5559217ab6f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe600034830 tx=0x7fe600005dd0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.935+0000 7fe61a42a740 1 -- 172.21.15.52:0/2914798388 shutdown_connections 2024-04-02T13:56:19.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.935+0000 7fe61a42a740 1 --2- 172.21.15.52:0/2914798388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5559217abca0 0x5559217a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.935+0000 7fe61a42a740 1 --2- 172.21.15.52:0/2914798388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217a30c0 0x5559217ab6f0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.935+0000 7fe61a42a740 1 --2- 172.21.15.52:0/2914798388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559217a27a0 0x5559217ab1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.935+0000 7fe61a42a740 1 -- 172.21.15.52:0/2914798388 >> 172.21.15.52:0/2914798388 conn(0x55592179ced0 msgr2=0x5559217a3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.936+0000 7fe61a42a740 1 -- 172.21.15.52:0/2914798388 shutdown_connections 2024-04-02T13:56:19.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.936+0000 7fe61a42a740 1 -- 172.21.15.52:0/2914798388 wait complete. 2024-04-02T13:56:19.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.936+0000 7fe61a42a740 1 -- start start 2024-04-02T13:56:19.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.936+0000 7fe61a42a740 1 Processor -- start 2024-04-02T13:56:19.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.936+0000 7fe61a42a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559217a27a0 0x5559217c1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.936+0000 7fe61a42a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217a30c0 0x5559217c1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.936+0000 7fe61a42a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5559217abca0 0x5559218253e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.936+0000 7fe61a42a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe6101adfd0 con 0x5559217a30c0 2024-04-02T13:56:19.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.936+0000 7fe61a42a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe6101ad900 con 0x5559217abca0 2024-04-02T13:56:19.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.936+0000 7fe61a42a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe6101aab40 con 0x5559217a27a0 2024-04-02T13:56:19.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.936+0000 7fe61a42a740 10 client.0 _ll_register_callbacks cb 0x7fe608009bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:19.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.936+0000 7fe614d08640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217a30c0 0x5559217c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.936+0000 7fe614d08640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217a30c0 0x5559217c1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38088/0 (socket says 172.21.15.52:38088) 2024-04-02T13:56:19.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.936+0000 7fe614d08640 1 -- 172.21.15.52:0/1129718643 learned_addr learned my addr 172.21.15.52:0/1129718643 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:19.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.937+0000 7fe615d0a640 1 --2- 172.21.15.52:0/1129718643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5559217abca0 0x5559218253e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.937+0000 7fe614d08640 1 -- 172.21.15.52:0/1129718643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559217a27a0 msgr2=0x5559217c1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.937+0000 7fe615509640 1 --2- 172.21.15.52:0/1129718643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559217a27a0 0x5559217c1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.937+0000 7fe614d08640 1 --2- 172.21.15.52:0/1129718643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559217a27a0 0x5559217c1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.937+0000 7fe614d08640 1 -- 172.21.15.52:0/1129718643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5559217abca0 msgr2=0x5559218253e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:19.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.937+0000 7fe614d08640 1 --2- 172.21.15.52:0/1129718643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5559217abca0 0x5559218253e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.937+0000 7fe614d08640 1 -- 172.21.15.52:0/1129718643 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555921825920 con 0x5559217a30c0 2024-04-02T13:56:19.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.937+0000 7fe615509640 1 --2- 172.21.15.52:0/1129718643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559217a27a0 0x5559217c1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:56:19.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.937+0000 7fe615d0a640 1 --2- 172.21.15.52:0/1129718643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5559217abca0 0x5559218253e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:19.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.937+0000 7fe614d08640 1 --2- 172.21.15.52:0/1129718643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217a30c0 0x5559217c1950 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe60003f040 tx=0x7fe60002cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74bbb4dbcbfd7c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:19.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.937+0000 7fe5f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:19.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.937+0000 7fe5f8ff9640 1 -- 172.21.15.52:0/1129718643 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600042050 con 0x5559217a30c0 2024-04-02T13:56:19.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.937+0000 7fe5f8ff9640 1 -- 172.21.15.52:0/1129718643 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe60005d430 con 0x5559217a30c0 2024-04-02T13:56:19.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.937+0000 7fe61a42a740 1 -- 172.21.15.52:0/1129718643 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe600034aa0 con 0x5559217a30c0 2024-04-02T13:56:19.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.937+0000 7fe5f8ff9640 1 -- 172.21.15.52:0/1129718643 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe60006e050 con 0x5559217a30c0 2024-04-02T13:56:19.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.937+0000 7fe61a42a740 10 client.16470 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:19.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.937+0000 7fe61a42a740 1 -- 172.21.15.52:0/1129718643 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555921826a10 con 0x5559217a30c0 2024-04-02T13:56:19.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.937+0000 7fe5f8ff9640 1 -- 172.21.15.52:0/1129718643 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe600034aa0 con 0x5559217a30c0 2024-04-02T13:56:19.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.938+0000 7fe5f8ff9640 1 -- 172.21.15.52:0/1129718643 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fe60003d440 con 0x5559217a30c0 2024-04-02T13:56:19.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.938+0000 7fe61a42a740 10 client.16470 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:19.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.938+0000 7fe61a42a740 10 client.16470 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T13:56:19.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.938+0000 7fe61a42a740 20 client.16470 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:19.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.938+0000 7fe61a42a740 1 --2- 172.21.15.52:0/1129718643 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x555921826f30 0x555921829350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:19.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.938+0000 7fe61a42a740 4 client.16470 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:56:19.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.938+0000 7fe61a42a740 1 -- 172.21.15.52:0/1129718643 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55592170e040 con 0x555921826f30 2024-04-02T13:56:19.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.938+0000 7fe615509640 1 --2- 172.21.15.52:0/1129718643 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x555921826f30 0x555921829350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:19.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.938+0000 7fe5f8ff9640 10 client.16470 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:56:19.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.938+0000 7fe615509640 1 --2- 172.21.15.52:0/1129718643 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x555921826f30 0x555921829350 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a7f22211f36fc8aa server_cookie=bd9218c90eed1664 in_seq=0 out_seq=0 2024-04-02T13:56:19.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.938+0000 7fe5f8ff9640 1 -- 172.21.15.52:0/1129718643 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55592170e040 con 0x555921826f30 2024-04-02T13:56:19.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.938+0000 7fe5f8ff9640 10 client.16470 handle_command_reply: tid=0 2024-04-02T13:56:19.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.938+0000 7fe61a42a740 1 client.16470 shutdown 2024-04-02T13:56:19.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.938+0000 7fe61a42a740 1 -- 172.21.15.52:0/1129718643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217a30c0 msgr2=0x5559217c1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.938+0000 7fe61a42a740 1 --2- 172.21.15.52:0/1129718643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217a30c0 0x5559217c1950 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe60003f040 tx=0x7fe60002cbd0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.939+0000 7fe61a42a740 1 -- 172.21.15.52:0/1129718643 shutdown_connections 2024-04-02T13:56:19.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.939+0000 7fe61a42a740 1 --2- 172.21.15.52:0/1129718643 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x555921826f30 0x555921829350 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.939+0000 7fe61a42a740 1 --2- 172.21.15.52:0/1129718643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5559217abca0 0x5559218253e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.939+0000 7fe61a42a740 1 --2- 172.21.15.52:0/1129718643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5559217a30c0 0x5559217c1950 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.939+0000 7fe61a42a740 1 --2- 172.21.15.52:0/1129718643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559217a27a0 0x5559217c1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.939+0000 7fe5f8ff9640 0 client.16470 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:56:19.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.939+0000 7fe61a42a740 1 -- 172.21.15.52:0/1129718643 >> 172.21.15.52:0/1129718643 conn(0x55592179ced0 msgr2=0x5559217bb240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.939+0000 7fe61a42a740 1 -- 172.21.15.52:0/1129718643 shutdown_connections 2024-04-02T13:56:19.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.939+0000 7fe61a42a740 1 -- 172.21.15.52:0/1129718643 wait complete. 2024-04-02T13:56:19.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.939+0000 7fe61a42a740 20 client.16470 trim_cache size 0 max 16384 2024-04-02T13:56:19.955 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:56:19.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.939+0000 7fe61776d640 1 -- 172.21.15.52:0/1401058410 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe5fc0a3d20 msgr2=0x7fe5fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.939+0000 7fe61776d640 1 --2- 172.21.15.52:0/1401058410 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe5fc0a3d20 0x7fe5fc0a61e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe6080092f0 tx=0x7fe608014a50 comp rx=0 tx=0).stop 2024-04-02T13:56:19.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.939+0000 7fe61776d640 1 -- 172.21.15.52:0/1401058410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe610102d40 msgr2=0x7fe6101983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:19.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.939+0000 7fe61776d640 1 --2- 172.21.15.52:0/1401058410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe610102d40 0x7fe6101983a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00e9f0 tx=0x7fe60c00eeb0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.939+0000 7fe61776d640 1 -- 172.21.15.52:0/1401058410 shutdown_connections 2024-04-02T13:56:19.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.939+0000 7fe61776d640 1 --2- 172.21.15.52:0/1401058410 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe5fc0a3d20 0x7fe5fc0a61e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.939+0000 7fe61776d640 1 --2- 172.21.15.52:0/1401058410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe61010c110 0x7fe61019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.939+0000 7fe61776d640 1 --2- 172.21.15.52:0/1401058410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe610103660 0x7fe6101988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.939+0000 7fe61776d640 1 --2- 172.21.15.52:0/1401058410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe610102d40 0x7fe6101983a0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:19.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.939+0000 7fe61776d640 1 -- 172.21.15.52:0/1401058410 >> 172.21.15.52:0/1401058410 conn(0x7fe610076d80 msgr2=0x7fe6100fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:19.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.939+0000 7fe61776d640 1 -- 172.21.15.52:0/1401058410 shutdown_connections 2024-04-02T13:56:19.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:19.939+0000 7fe61776d640 1 -- 172.21.15.52:0/1401058410 wait complete. 2024-04-02T13:56:19.959 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:56:20.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.059+0000 7f6a48b7b640 1 -- 172.21.15.52:0/791582310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a441058a0 msgr2=0x7f6a44105c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:20.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.059+0000 7f6a48b7b640 1 --2- 172.21.15.52:0/791582310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a441058a0 0x7f6a44105c80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009900 tx=0x7f6a3402bcd0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.060+0000 7f6a48b7b640 1 -- 172.21.15.52:0/791582310 shutdown_connections 2024-04-02T13:56:20.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.060+0000 7f6a48b7b640 1 --2- 172.21.15.52:0/791582310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a44101360 0x7f6a4410d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.060+0000 7f6a48b7b640 1 --2- 172.21.15.52:0/791582310 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a441009c0 0x7f6a44100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.060+0000 7f6a48b7b640 1 --2- 172.21.15.52:0/791582310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a441058a0 0x7f6a44105c80 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.060+0000 7f6a48b7b640 1 -- 172.21.15.52:0/791582310 >> 172.21.15.52:0/791582310 conn(0x7f6a44077a90 msgr2=0x7f6a44077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:20.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.060+0000 7f6a48b7b640 1 -- 172.21.15.52:0/791582310 shutdown_connections 2024-04-02T13:56:20.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.060+0000 7f6a48b7b640 1 -- 172.21.15.52:0/791582310 wait complete. 2024-04-02T13:56:20.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.060+0000 7f6a48b7b640 1 Processor -- start 2024-04-02T13:56:20.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.060+0000 7f6a48b7b640 1 -- start start 2024-04-02T13:56:20.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.061+0000 7f6a48b7b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a441009c0 0x7f6a44198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.061+0000 7f6a48b7b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a44101360 0x7f6a441989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.061+0000 7f6a48b7b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a441058a0 0x7f6a4419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.061+0000 7f6a48b7b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6a44111ef0 con 0x7f6a441058a0 2024-04-02T13:56:20.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.061+0000 7f6a48b7b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6a44111d70 con 0x7f6a44101360 2024-04-02T13:56:20.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.061+0000 7f6a48b7b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6a44112070 con 0x7f6a441009c0 2024-04-02T13:56:20.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.061+0000 7f6a42d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a441058a0 0x7f6a4419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.061+0000 7f6a42d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a441058a0 0x7f6a4419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38106/0 (socket says 172.21.15.52:38106) 2024-04-02T13:56:20.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.061+0000 7f6a42d9d640 1 -- 172.21.15.52:0/1793976605 learned_addr learned my addr 172.21.15.52:0/1793976605 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:20.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.061+0000 7f6a4259c640 1 --2- 172.21.15.52:0/1793976605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a441009c0 0x7f6a44198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.061+0000 7f6a41d9b640 1 --2- 172.21.15.52:0/1793976605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a44101360 0x7f6a441989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.061+0000 7f6a42d9d640 1 -- 172.21.15.52:0/1793976605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a441009c0 msgr2=0x7f6a44198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:20.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.061+0000 7f6a42d9d640 1 --2- 172.21.15.52:0/1793976605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a441009c0 0x7f6a44198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.061+0000 7f6a42d9d640 1 -- 172.21.15.52:0/1793976605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a44101360 msgr2=0x7f6a441989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:20.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.061+0000 7f6a42d9d640 1 --2- 172.21.15.52:0/1793976605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a44101360 0x7f6a441989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.061+0000 7f6a42d9d640 1 -- 172.21.15.52:0/1793976605 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4419d480 con 0x7f6a441058a0 2024-04-02T13:56:20.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.061+0000 7f6a4259c640 1 --2- 172.21.15.52:0/1793976605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a441009c0 0x7f6a44198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:56:20.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.061+0000 7f6a41d9b640 1 --2- 172.21.15.52:0/1793976605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a44101360 0x7f6a441989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:56:20.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.061+0000 7f6a42d9d640 1 --2- 172.21.15.52:0/1793976605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a441058a0 0x7f6a4419cd50 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00eaa0 tx=0x7f6a3c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:20.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.061+0000 7f6a337fe640 1 -- 172.21.15.52:0/1793976605 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c00cdb0 con 0x7f6a441058a0 2024-04-02T13:56:20.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.061+0000 7f6a48b7b640 1 -- 172.21.15.52:0/1793976605 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a4419d770 con 0x7f6a441058a0 2024-04-02T13:56:20.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.061+0000 7f6a337fe640 1 -- 172.21.15.52:0/1793976605 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a3c005020 con 0x7f6a441058a0 2024-04-02T13:56:20.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.061+0000 7f6a337fe640 1 -- 172.21.15.52:0/1793976605 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c0116d0 con 0x7f6a441058a0 2024-04-02T13:56:20.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.061+0000 7f6a48b7b640 1 -- 172.21.15.52:0/1793976605 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a441a5000 con 0x7f6a441058a0 2024-04-02T13:56:20.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.063+0000 7f6a337fe640 1 -- 172.21.15.52:0/1793976605 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6a3c0054d0 con 0x7f6a441058a0 2024-04-02T13:56:20.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.064+0000 7f6a4b766740 1 Processor -- start 2024-04-02T13:56:20.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.064+0000 7f6a4b766740 1 -- start start 2024-04-02T13:56:20.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.064+0000 7f6a4b766740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7a031490 0x55cd7a031870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.064+0000 7f6a4b766740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7a031db0 0x55cd7a03eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.064+0000 7f6a4b766740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a03f0b0 0x55cd7a0414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.064+0000 7f6a4b766740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cd79f957b0 con 0x55cd7a03f0b0 2024-04-02T13:56:20.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.064+0000 7f6a4b766740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cd79f34570 con 0x55cd7a031db0 2024-04-02T13:56:20.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.064+0000 7f6a4b766740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cd79f91760 con 0x55cd7a031490 2024-04-02T13:56:20.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.064+0000 7f6a337fe640 1 --2- 172.21.15.52:0/1793976605 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6a2c0a3d20 0x7f6a2c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.064+0000 7f6a41d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a03f0b0 0x55cd7a0414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.064+0000 7f6a41d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a03f0b0 0x55cd7a0414a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38112/0 (socket says 172.21.15.52:38112) 2024-04-02T13:56:20.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.064+0000 7f6a41d9b640 1 -- 172.21.15.52:0/975621172 learned_addr learned my addr 172.21.15.52:0/975621172 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:20.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.064+0000 7f6a42d9d640 1 --2- 172.21.15.52:0/1793976605 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6a2c0a3d20 0x7f6a2c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.064+0000 7f6a4259c640 1 --2- 172.21.15.52:0/975621172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7a031db0 0x55cd7a03eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.065+0000 7f6a41d9b640 1 --2- 172.21.15.52:0/975621172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7a031490 0x55cd7a031870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.065+0000 7f6a337fe640 1 -- 172.21.15.52:0/1793976605 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f6a3c0d0e20 con 0x7f6a441058a0 2024-04-02T13:56:20.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.065+0000 7f6a41d9b640 1 -- 172.21.15.52:0/975621172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7a031490 msgr2=0x55cd7a031870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:20.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.065+0000 7f6a41d9b640 1 --2- 172.21.15.52:0/975621172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7a031490 0x55cd7a031870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.065+0000 7f6a41d9b640 1 -- 172.21.15.52:0/975621172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7a031db0 msgr2=0x55cd7a03eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:20.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.065+0000 7f6a41d9b640 1 --2- 172.21.15.52:0/975621172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7a031db0 0x55cd7a03eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.065+0000 7f6a41d9b640 1 -- 172.21.15.52:0/975621172 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd79ca93f0 con 0x55cd7a03f0b0 2024-04-02T13:56:20.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.065+0000 7f6a41d9b640 1 --2- 172.21.15.52:0/975621172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a03f0b0 0x55cd7a0414a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800af40 tx=0x7f6a3800a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bbd39d4394b8791 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:20.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.065+0000 7f6a42d9d640 1 --2- 172.21.15.52:0/1793976605 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6a2c0a3d20 0x7f6a2c0a61e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c0ce2e0 tx=0x7f6a3c087040 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:20.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.065+0000 7f6a30ff9640 1 -- 172.21.15.52:0/975621172 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3801b070 con 0x55cd7a03f0b0 2024-04-02T13:56:20.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.065+0000 7f6a30ff9640 1 -- 172.21.15.52:0/975621172 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a380168d0 con 0x55cd7a03f0b0 2024-04-02T13:56:20.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.065+0000 7f6a30ff9640 1 -- 172.21.15.52:0/975621172 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3800f040 con 0x55cd7a03f0b0 2024-04-02T13:56:20.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.066+0000 7f6a4b766740 1 -- 172.21.15.52:0/975621172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a03f0b0 msgr2=0x55cd7a0414a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:20.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.066+0000 7f6a4b766740 1 --2- 172.21.15.52:0/975621172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a03f0b0 0x55cd7a0414a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800af40 tx=0x7f6a3800a410 comp rx=0 tx=0).stop 2024-04-02T13:56:20.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.066+0000 7f6a4b766740 1 -- 172.21.15.52:0/975621172 shutdown_connections 2024-04-02T13:56:20.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.066+0000 7f6a4b766740 1 --2- 172.21.15.52:0/975621172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a03f0b0 0x55cd7a0414a0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.066+0000 7f6a4b766740 1 --2- 172.21.15.52:0/975621172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7a031db0 0x55cd7a03eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.066+0000 7f6a4b766740 1 --2- 172.21.15.52:0/975621172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7a031490 0x55cd7a031870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.066+0000 7f6a4b766740 1 -- 172.21.15.52:0/975621172 >> 172.21.15.52:0/975621172 conn(0x55cd7a02bed0 msgr2=0x55cd7a02c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:20.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.066+0000 7f6a4b766740 1 -- 172.21.15.52:0/975621172 shutdown_connections 2024-04-02T13:56:20.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.066+0000 7f6a4b766740 1 -- 172.21.15.52:0/975621172 wait complete. 2024-04-02T13:56:20.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.066+0000 7f6a4b766740 1 -- start start 2024-04-02T13:56:20.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.066+0000 7f6a4b766740 1 Processor -- start 2024-04-02T13:56:20.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.066+0000 7f6a4b766740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7a031490 0x55cd7a03ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.066+0000 7f6a4b766740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7a031db0 0x55cd7a03be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.067+0000 7f6a4b766740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a03f0b0 0x55cd7a03c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.067+0000 7f6a4b766740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cd7a041ea0 con 0x55cd7a03f0b0 2024-04-02T13:56:20.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.067+0000 7f6a4b766740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cd79f50e80 con 0x55cd7a031db0 2024-04-02T13:56:20.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.067+0000 7f6a4b766740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cd7a042020 con 0x55cd7a031490 2024-04-02T13:56:20.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.067+0000 7f6a4b766740 10 client.0 _ll_register_callbacks cb 0x55cd79f8caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:20.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.067+0000 7f6a41d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a03f0b0 0x55cd7a03c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.067+0000 7f6a41d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a03f0b0 0x55cd7a03c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38120/0 (socket says 172.21.15.52:38120) 2024-04-02T13:56:20.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.067+0000 7f6a41d9b640 1 -- 172.21.15.52:0/2662000141 learned_addr learned my addr 172.21.15.52:0/2662000141 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:20.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.067+0000 7f6a41d9b640 1 --2- 172.21.15.52:0/2662000141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7a031490 0x55cd7a03ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.067+0000 7f6a4259c640 1 --2- 172.21.15.52:0/2662000141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7a031db0 0x55cd7a03be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.067+0000 7f6a41d9b640 1 -- 172.21.15.52:0/2662000141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7a031490 msgr2=0x55cd7a03ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:20.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.067+0000 7f6a41d9b640 1 --2- 172.21.15.52:0/2662000141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7a031490 0x55cd7a03ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.067+0000 7f6a41d9b640 1 -- 172.21.15.52:0/2662000141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7a031db0 msgr2=0x55cd7a03be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:20.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.067+0000 7f6a41d9b640 1 --2- 172.21.15.52:0/2662000141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7a031db0 0x55cd7a03be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.067+0000 7f6a41d9b640 1 -- 172.21.15.52:0/2662000141 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd79d737d0 con 0x55cd7a03f0b0 2024-04-02T13:56:20.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.067+0000 7f6a41d9b640 1 --2- 172.21.15.52:0/2662000141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a03f0b0 0x55cd7a03c470 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f6a38006dc0 tx=0x7f6a380154d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db9750c5e839808 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:20.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.067+0000 7f6a29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:20.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.067+0000 7f6a29ffb640 1 -- 172.21.15.52:0/2662000141 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3801b070 con 0x55cd7a03f0b0 2024-04-02T13:56:20.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.067+0000 7f6a29ffb640 1 -- 172.21.15.52:0/2662000141 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f6a38015670 con 0x55cd7a03f0b0 2024-04-02T13:56:20.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.067+0000 7f6a4b766740 1 -- 172.21.15.52:0/2662000141 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a380168d0 con 0x55cd7a03f0b0 2024-04-02T13:56:20.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.067+0000 7f6a29ffb640 1 -- 172.21.15.52:0/2662000141 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6a38015890 con 0x55cd7a03f0b0 2024-04-02T13:56:20.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.067+0000 7f6a4b766740 10 client.16482 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:20.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.067+0000 7f6a4b766740 1 -- 172.21.15.52:0/2662000141 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cd7a04df20 con 0x55cd7a03f0b0 2024-04-02T13:56:20.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.068+0000 7f6a29ffb640 1 -- 172.21.15.52:0/2662000141 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a380168d0 con 0x55cd7a03f0b0 2024-04-02T13:56:20.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.068+0000 7f6a29ffb640 1 -- 172.21.15.52:0/2662000141 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f6a38015070 con 0x55cd7a03f0b0 2024-04-02T13:56:20.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.068+0000 7f6a4b766740 10 client.16482 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:20.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.068+0000 7f6a4b766740 10 client.16482 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:56:20.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.068+0000 7f6a4b766740 20 client.16482 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:20.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.068+0000 7f6a4b766740 1 --2- 172.21.15.52:0/2662000141 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55cd7a04f2d0 0x55cd7a0516c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.068+0000 7f6a4b766740 4 client.16482 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:20.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.068+0000 7f6a4b766740 1 -- 172.21.15.52:0/2662000141 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd79fa2cf0 con 0x55cd7a04f2d0 2024-04-02T13:56:20.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.068+0000 7f6a4259c640 1 --2- 172.21.15.52:0/2662000141 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55cd7a04f2d0 0x55cd7a0516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.069+0000 7f6a29ffb640 10 client.16482 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:56:20.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.069+0000 7f6a4259c640 1 --2- 172.21.15.52:0/2662000141 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55cd7a04f2d0 0x55cd7a0516c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4eeeed57148577b6 server_cookie=31a7e042c249d29a in_seq=0 out_seq=0 2024-04-02T13:56:20.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.069+0000 7f6a29ffb640 1 -- 172.21.15.52:0/2662000141 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cd79fa2cf0 con 0x55cd7a04f2d0 2024-04-02T13:56:20.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.069+0000 7f6a29ffb640 10 client.16482 handle_command_reply: tid=0 2024-04-02T13:56:20.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.069+0000 7f6a4b766740 1 client.16482 shutdown 2024-04-02T13:56:20.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.069+0000 7f6a4b766740 1 -- 172.21.15.52:0/2662000141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a03f0b0 msgr2=0x55cd7a03c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:20.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.069+0000 7f6a4b766740 1 --2- 172.21.15.52:0/2662000141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a03f0b0 0x55cd7a03c470 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f6a38006dc0 tx=0x7f6a380154d0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.070+0000 7f6a4b766740 1 -- 172.21.15.52:0/2662000141 shutdown_connections 2024-04-02T13:56:20.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.070+0000 7f6a4b766740 1 --2- 172.21.15.52:0/2662000141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a03f0b0 0x55cd7a03c470 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.070+0000 7f6a4b766740 1 --2- 172.21.15.52:0/2662000141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7a031db0 0x55cd7a03be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.070+0000 7f6a4b766740 1 --2- 172.21.15.52:0/2662000141 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55cd7a04f2d0 0x55cd7a0516c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.070+0000 7f6a4b766740 1 --2- 172.21.15.52:0/2662000141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7a031490 0x55cd7a03ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.070+0000 7f6a29ffb640 0 client.16482 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:56:20.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.070+0000 7f6a4b766740 1 -- 172.21.15.52:0/2662000141 >> 172.21.15.52:0/2662000141 conn(0x55cd7a02bed0 msgr2=0x55cd7a02dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:20.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.070+0000 7f6a4b766740 1 -- 172.21.15.52:0/2662000141 shutdown_connections 2024-04-02T13:56:20.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.070+0000 7f6a4b766740 1 -- 172.21.15.52:0/2662000141 wait complete. 2024-04-02T13:56:20.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.070+0000 7f6a4b766740 20 client.16482 trim_cache size 0 max 16384 2024-04-02T13:56:20.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.083+0000 7f6a4b766740 1 Processor -- start 2024-04-02T13:56:20.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.083+0000 7f6a4b766740 1 -- start start 2024-04-02T13:56:20.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.083+0000 7f6a4b766740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7a031490 0x55cd7a04add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.083+0000 7f6a4b766740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7a031db0 0x55cd7a03ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.083+0000 7f6a4b766740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a03f0b0 0x55cd7a03e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.083+0000 7f6a4b766740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6a441acd30 con 0x55cd7a03f0b0 2024-04-02T13:56:20.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.083+0000 7f6a4b766740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6a441ae8a0 con 0x55cd7a031db0 2024-04-02T13:56:20.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.083+0000 7f6a4b766740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6a441ad400 con 0x55cd7a031490 2024-04-02T13:56:20.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.084+0000 7f6a41d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a03f0b0 0x55cd7a03e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.084+0000 7f6a41d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a03f0b0 0x55cd7a03e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38134/0 (socket says 172.21.15.52:38134) 2024-04-02T13:56:20.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.084+0000 7f6a41d9b640 1 -- 172.21.15.52:0/1958226642 learned_addr learned my addr 172.21.15.52:0/1958226642 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:20.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.084+0000 7f6a41d9b640 1 --2- 172.21.15.52:0/1958226642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7a031490 0x55cd7a04add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.084+0000 7f6a4259c640 1 --2- 172.21.15.52:0/1958226642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7a031db0 0x55cd7a03ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.084+0000 7f6a41d9b640 1 -- 172.21.15.52:0/1958226642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7a031490 msgr2=0x55cd7a04add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:20.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.084+0000 7f6a41d9b640 1 --2- 172.21.15.52:0/1958226642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7a031490 0x55cd7a04add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.084+0000 7f6a41d9b640 1 -- 172.21.15.52:0/1958226642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7a031db0 msgr2=0x55cd7a03ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:20.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.084+0000 7f6a41d9b640 1 --2- 172.21.15.52:0/1958226642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7a031db0 0x55cd7a03ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.084+0000 7f6a41d9b640 1 -- 172.21.15.52:0/1958226642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd7a041d80 con 0x55cd7a03f0b0 2024-04-02T13:56:20.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.084+0000 7f6a41d9b640 1 --2- 172.21.15.52:0/1958226642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a03f0b0 0x55cd7a03e590 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6a38015360 tx=0x7f6a3800ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa5e95e1a633afe server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:20.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.084+0000 7f6a2bfff640 1 -- 172.21.15.52:0/1958226642 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3801b070 con 0x55cd7a03f0b0 2024-04-02T13:56:20.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.084+0000 7f6a2bfff640 1 -- 172.21.15.52:0/1958226642 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a38006ba0 con 0x55cd7a03f0b0 2024-04-02T13:56:20.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.084+0000 7f6a2bfff640 1 -- 172.21.15.52:0/1958226642 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38016700 con 0x55cd7a03f0b0 2024-04-02T13:56:20.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.085+0000 7f6a4b766740 1 -- 172.21.15.52:0/1958226642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a03f0b0 msgr2=0x55cd7a03e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:20.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.085+0000 7f6a4b766740 1 --2- 172.21.15.52:0/1958226642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a03f0b0 0x55cd7a03e590 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6a38015360 tx=0x7f6a3800ac00 comp rx=0 tx=0).stop 2024-04-02T13:56:20.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.085+0000 7f6a4b766740 1 -- 172.21.15.52:0/1958226642 shutdown_connections 2024-04-02T13:56:20.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.085+0000 7f6a4b766740 1 --2- 172.21.15.52:0/1958226642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a03f0b0 0x55cd7a03e590 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.085+0000 7f6a4b766740 1 --2- 172.21.15.52:0/1958226642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7a031db0 0x55cd7a03ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.085+0000 7f6a4b766740 1 --2- 172.21.15.52:0/1958226642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7a031490 0x55cd7a04add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.085+0000 7f6a4b766740 1 -- 172.21.15.52:0/1958226642 >> 172.21.15.52:0/1958226642 conn(0x55cd7a02bed0 msgr2=0x55cd7a03a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:20.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.085+0000 7f6a4b766740 1 -- 172.21.15.52:0/1958226642 shutdown_connections 2024-04-02T13:56:20.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.085+0000 7f6a4b766740 1 -- 172.21.15.52:0/1958226642 wait complete. 2024-04-02T13:56:20.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.085+0000 7f6a4b766740 1 -- start start 2024-04-02T13:56:20.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.085+0000 7f6a4b766740 1 Processor -- start 2024-04-02T13:56:20.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.085+0000 7f6a4b766740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7a031490 0x55cd7a03f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.085+0000 7f6a4b766740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a031db0 0x55cd7a03fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.085+0000 7f6a4b766740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7a03f0b0 0x55cd7a040050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.085+0000 7f6a4b766740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6a441ae1a0 con 0x55cd7a031db0 2024-04-02T13:56:20.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.085+0000 7f6a4b766740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6a441adad0 con 0x55cd7a03f0b0 2024-04-02T13:56:20.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.085+0000 7f6a4b766740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6a441aad10 con 0x55cd7a031490 2024-04-02T13:56:20.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.085+0000 7f6a4b766740 10 client.0 _ll_register_callbacks cb 0x7f6a38019070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:20.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.085+0000 7f6a4259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a031db0 0x55cd7a03fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.086+0000 7f6a4259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a031db0 0x55cd7a03fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38142/0 (socket says 172.21.15.52:38142) 2024-04-02T13:56:20.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.086+0000 7f6a4259c640 1 -- 172.21.15.52:0/1009469220 learned_addr learned my addr 172.21.15.52:0/1009469220 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:20.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.086+0000 7f6a41d9b640 1 --2- 172.21.15.52:0/1009469220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7a031490 0x55cd7a03f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.086+0000 7f6a41d9b640 1 --2- 172.21.15.52:0/1009469220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7a03f0b0 0x55cd7a040050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.086+0000 7f6a4259c640 1 -- 172.21.15.52:0/1009469220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7a031490 msgr2=0x55cd7a03f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:20.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.086+0000 7f6a4259c640 1 --2- 172.21.15.52:0/1009469220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7a031490 0x55cd7a03f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.086+0000 7f6a4259c640 1 -- 172.21.15.52:0/1009469220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7a03f0b0 msgr2=0x55cd7a040050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:20.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.086+0000 7f6a4259c640 1 --2- 172.21.15.52:0/1009469220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7a03f0b0 0x55cd7a040050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.086+0000 7f6a4259c640 1 -- 172.21.15.52:0/1009469220 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd7a046580 con 0x55cd7a031db0 2024-04-02T13:56:20.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.086+0000 7f6a4259c640 1 --2- 172.21.15.52:0/1009469220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a031db0 0x55cd7a03fb10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6a34005ab0 tx=0x7f6a3402cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a39a645f4c005577 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:20.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.086+0000 7f6a29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:20.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.086+0000 7f6a29ffb640 1 -- 172.21.15.52:0/1009469220 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3404e020 con 0x55cd7a031db0 2024-04-02T13:56:20.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.086+0000 7f6a4b766740 1 -- 172.21.15.52:0/1009469220 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a38006ba0 con 0x55cd7a031db0 2024-04-02T13:56:20.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.086+0000 7f6a29ffb640 1 -- 172.21.15.52:0/1009469220 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f6a34004240 con 0x55cd7a031db0 2024-04-02T13:56:20.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.086+0000 7f6a29ffb640 1 -- 172.21.15.52:0/1009469220 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6a34056050 con 0x55cd7a031db0 2024-04-02T13:56:20.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.086+0000 7f6a4b766740 10 client.16488 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:20.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.086+0000 7f6a4b766740 1 -- 172.21.15.52:0/1009469220 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cd7a0478a0 con 0x55cd7a031db0 2024-04-02T13:56:20.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.087+0000 7f6a29ffb640 1 -- 172.21.15.52:0/1009469220 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a38006ba0 con 0x55cd7a031db0 2024-04-02T13:56:20.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.087+0000 7f6a29ffb640 1 -- 172.21.15.52:0/1009469220 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f6a34004970 con 0x55cd7a031db0 2024-04-02T13:56:20.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.087+0000 7f6a4b766740 10 client.16488 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:20.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.087+0000 7f6a4b766740 10 client.16488 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:56:20.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.087+0000 7f6a4b766740 20 client.16488 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:20.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.087+0000 7f6a4b766740 1 --2- 172.21.15.52:0/1009469220 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55cd7a0b5860 0x55cd7a0b7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.087+0000 7f6a4b766740 4 client.16488 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:56:20.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.087+0000 7f6a4b766740 1 -- 172.21.15.52:0/1009469220 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd79f9d040 con 0x55cd7a0b5860 2024-04-02T13:56:20.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.087+0000 7f6a41d9b640 1 --2- 172.21.15.52:0/1009469220 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55cd7a0b5860 0x55cd7a0b7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.087+0000 7f6a41d9b640 1 --2- 172.21.15.52:0/1009469220 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55cd7a0b5860 0x55cd7a0b7c80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a83bc6f538199b93 server_cookie=181bf2e6c560c628 in_seq=0 out_seq=0 2024-04-02T13:56:20.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.087+0000 7f6a29ffb640 10 client.16488 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:56:20.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.088+0000 7f6a29ffb640 1 -- 172.21.15.52:0/1009469220 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd79f9d040 con 0x55cd7a0b5860 2024-04-02T13:56:20.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.088+0000 7f6a29ffb640 10 client.16488 handle_command_reply: tid=0 2024-04-02T13:56:20.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.088+0000 7f6a4b766740 1 client.16488 shutdown 2024-04-02T13:56:20.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.088+0000 7f6a4b766740 1 -- 172.21.15.52:0/1009469220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a031db0 msgr2=0x55cd7a03fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:20.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.088+0000 7f6a4b766740 1 --2- 172.21.15.52:0/1009469220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a031db0 0x55cd7a03fb10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6a34005ab0 tx=0x7f6a3402cb70 comp rx=0 tx=0).stop 2024-04-02T13:56:20.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.088+0000 7f6a4b766740 1 -- 172.21.15.52:0/1009469220 shutdown_connections 2024-04-02T13:56:20.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.088+0000 7f6a4b766740 1 --2- 172.21.15.52:0/1009469220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7a03f0b0 0x55cd7a040050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.088+0000 7f6a4b766740 1 --2- 172.21.15.52:0/1009469220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7a031db0 0x55cd7a03fb10 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.088+0000 7f6a4b766740 1 --2- 172.21.15.52:0/1009469220 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55cd7a0b5860 0x55cd7a0b7c80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.088+0000 7f6a4b766740 1 --2- 172.21.15.52:0/1009469220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7a031490 0x55cd7a03f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.088+0000 7f6a29ffb640 0 client.16488 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:56:20.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.088+0000 7f6a4b766740 1 -- 172.21.15.52:0/1009469220 >> 172.21.15.52:0/1009469220 conn(0x55cd7a02bed0 msgr2=0x55cd7a02af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:20.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.088+0000 7f6a4b766740 1 -- 172.21.15.52:0/1009469220 shutdown_connections 2024-04-02T13:56:20.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.088+0000 7f6a4b766740 1 -- 172.21.15.52:0/1009469220 wait complete. 2024-04-02T13:56:20.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.088+0000 7f6a4b766740 20 client.16488 trim_cache size 0 max 16384 2024-04-02T13:56:20.105 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:56:20.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.088+0000 7f6a48b7b640 1 -- 172.21.15.52:0/1793976605 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6a2c0a3d20 msgr2=0x7f6a2c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:20.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.088+0000 7f6a48b7b640 1 --2- 172.21.15.52:0/1793976605 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6a2c0a3d20 0x7f6a2c0a61e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c0ce2e0 tx=0x7f6a3c087040 comp rx=0 tx=0).stop 2024-04-02T13:56:20.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.088+0000 7f6a48b7b640 1 -- 172.21.15.52:0/1793976605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a441058a0 msgr2=0x7f6a4419cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:20.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.088+0000 7f6a48b7b640 1 --2- 172.21.15.52:0/1793976605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a441058a0 0x7f6a4419cd50 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00eaa0 tx=0x7f6a3c00ef60 comp rx=0 tx=0).stop 2024-04-02T13:56:20.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.089+0000 7f6a48b7b640 1 -- 172.21.15.52:0/1793976605 shutdown_connections 2024-04-02T13:56:20.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.089+0000 7f6a48b7b640 1 --2- 172.21.15.52:0/1793976605 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6a2c0a3d20 0x7f6a2c0a61e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.089+0000 7f6a48b7b640 1 --2- 172.21.15.52:0/1793976605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a441058a0 0x7f6a4419cd50 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.089+0000 7f6a48b7b640 1 --2- 172.21.15.52:0/1793976605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a44101360 0x7f6a441989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.089+0000 7f6a48b7b640 1 --2- 172.21.15.52:0/1793976605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a441009c0 0x7f6a44198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.089+0000 7f6a48b7b640 1 -- 172.21.15.52:0/1793976605 >> 172.21.15.52:0/1793976605 conn(0x7f6a44077a90 msgr2=0x7f6a440feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:20.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.089+0000 7f6a48b7b640 1 -- 172.21.15.52:0/1793976605 shutdown_connections 2024-04-02T13:56:20.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.089+0000 7f6a48b7b640 1 -- 172.21.15.52:0/1793976605 wait complete. 2024-04-02T13:56:20.108 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:56:20.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.210+0000 7f57bd115640 1 -- 172.21.15.52:0/691141262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b810e470 msgr2=0x7f57b8110860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:20.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.210+0000 7f57bd115640 1 --2- 172.21.15.52:0/691141262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b810e470 0x7f57b8110860 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009ca0 tx=0x7f57b002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.210+0000 7f57bd115640 1 -- 172.21.15.52:0/691141262 shutdown_connections 2024-04-02T13:56:20.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.210+0000 7f57bd115640 1 --2- 172.21.15.52:0/691141262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b810e470 0x7f57b8110860 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.210+0000 7f57bd115640 1 --2- 172.21.15.52:0/691141262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57b81011e0 0x7f57b810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.210+0000 7f57bd115640 1 --2- 172.21.15.52:0/691141262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57b81008c0 0x7f57b8100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.210+0000 7f57bd115640 1 -- 172.21.15.52:0/691141262 >> 172.21.15.52:0/691141262 conn(0x7f57b8076d80 msgr2=0x7f57b8077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:20.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.210+0000 7f57bd115640 1 -- 172.21.15.52:0/691141262 shutdown_connections 2024-04-02T13:56:20.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.210+0000 7f57bd115640 1 -- 172.21.15.52:0/691141262 wait complete. 2024-04-02T13:56:20.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.211+0000 7f57bd115640 1 Processor -- start 2024-04-02T13:56:20.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.211+0000 7f57bd115640 1 -- start start 2024-04-02T13:56:20.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.211+0000 7f57bd115640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b81008c0 0x7f57b819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.211+0000 7f57bd115640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57b81011e0 0x7f57b819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.211+0000 7f57bd115640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57b810e470 0x7f57b819ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.211+0000 7f57bd115640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f57b8114090 con 0x7f57b81008c0 2024-04-02T13:56:20.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.211+0000 7f57bd115640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f57b8113f10 con 0x7f57b810e470 2024-04-02T13:56:20.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.211+0000 7f57bd115640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f57b8114210 con 0x7f57b81011e0 2024-04-02T13:56:20.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.211+0000 7f57b6d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b81008c0 0x7f57b819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.211+0000 7f57b6d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b81008c0 0x7f57b819a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38152/0 (socket says 172.21.15.52:38152) 2024-04-02T13:56:20.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.211+0000 7f57b6d9d640 1 -- 172.21.15.52:0/3054065233 learned_addr learned my addr 172.21.15.52:0/3054065233 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:20.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.212+0000 7f57affff640 1 --2- 172.21.15.52:0/3054065233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57b81011e0 0x7f57b819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.212+0000 7f57b759e640 1 --2- 172.21.15.52:0/3054065233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57b810e470 0x7f57b819ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.212+0000 7f57b6d9d640 1 -- 172.21.15.52:0/3054065233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57b81011e0 msgr2=0x7f57b819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:20.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.212+0000 7f57b6d9d640 1 --2- 172.21.15.52:0/3054065233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57b81011e0 0x7f57b819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.212+0000 7f57b6d9d640 1 -- 172.21.15.52:0/3054065233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57b810e470 msgr2=0x7f57b819ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:20.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.212+0000 7f57b6d9d640 1 --2- 172.21.15.52:0/3054065233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57b810e470 0x7f57b819ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.212+0000 7f57b6d9d640 1 -- 172.21.15.52:0/3054065233 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b819f560 con 0x7f57b81008c0 2024-04-02T13:56:20.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.212+0000 7f57b759e640 1 --2- 172.21.15.52:0/3054065233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57b810e470 0x7f57b819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:20.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.212+0000 7f57b6d9d640 1 --2- 172.21.15.52:0/3054065233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b81008c0 0x7f57b819a570 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f57a800d770 tx=0x7f57a800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:20.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.212+0000 7f57b4d99640 1 -- 172.21.15.52:0/3054065233 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8017070 con 0x7f57b81008c0 2024-04-02T13:56:20.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.212+0000 7f57bd115640 1 -- 172.21.15.52:0/3054065233 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b819f850 con 0x7f57b81008c0 2024-04-02T13:56:20.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.212+0000 7f57b4d99640 1 -- 172.21.15.52:0/3054065233 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57a8005500 con 0x7f57b81008c0 2024-04-02T13:56:20.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.212+0000 7f57b4d99640 1 -- 172.21.15.52:0/3054065233 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8004e00 con 0x7f57b81008c0 2024-04-02T13:56:20.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.212+0000 7f57bd115640 1 -- 172.21.15.52:0/3054065233 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b81a70e0 con 0x7f57b81008c0 2024-04-02T13:56:20.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.215+0000 7f57b4d99640 1 -- 172.21.15.52:0/3054065233 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f57a8004fa0 con 0x7f57b81008c0 2024-04-02T13:56:20.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.215+0000 7f57bfddc740 1 Processor -- start 2024-04-02T13:56:20.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.215+0000 7f57bfddc740 1 -- start start 2024-04-02T13:56:20.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.215+0000 7f57bfddc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e967a0 0x564b04e96b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.215+0000 7f57bfddc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b04e970c0 0x564b04e9f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.215+0000 7f57bfddc740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b04e9fca0 0x564b04ea2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.215+0000 7f57bfddc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564b04dfa7b0 con 0x564b04e967a0 2024-04-02T13:56:20.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.215+0000 7f57bfddc740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564b04d99570 con 0x564b04e9fca0 2024-04-02T13:56:20.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.215+0000 7f57bfddc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564b04df6760 con 0x564b04e970c0 2024-04-02T13:56:20.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.216+0000 7f57b759e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e967a0 0x564b04e96b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.216+0000 7f57b759e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e967a0 0x564b04e96b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38162/0 (socket says 172.21.15.52:38162) 2024-04-02T13:56:20.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.216+0000 7f57b759e640 1 -- 172.21.15.52:0/2435774949 learned_addr learned my addr 172.21.15.52:0/2435774949 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:20.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.216+0000 7f57b4d99640 1 --2- 172.21.15.52:0/3054065233 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f57a40a3d20 0x7f57a40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.216+0000 7f57b759e640 1 --2- 172.21.15.52:0/3054065233 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f57a40a3d20 0x7f57a40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.216+0000 7f57b4d99640 1 -- 172.21.15.52:0/3054065233 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f57a80d06a0 con 0x7f57b81008c0 2024-04-02T13:56:20.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.216+0000 7f57b6d9d640 1 --2- 172.21.15.52:0/2435774949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b04e970c0 0x564b04e9f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.216+0000 7f57affff640 1 --2- 172.21.15.52:0/2435774949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b04e9fca0 0x564b04ea2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.216+0000 7f57b759e640 1 -- 172.21.15.52:0/2435774949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b04e970c0 msgr2=0x564b04e9f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:20.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.216+0000 7f57b759e640 1 --2- 172.21.15.52:0/2435774949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b04e970c0 0x564b04e9f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.216+0000 7f57b759e640 1 -- 172.21.15.52:0/2435774949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b04e9fca0 msgr2=0x564b04ea2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:20.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.216+0000 7f57b759e640 1 --2- 172.21.15.52:0/2435774949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b04e9fca0 0x564b04ea2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.216+0000 7f57b759e640 1 -- 172.21.15.52:0/2435774949 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b04b0e3f0 con 0x564b04e967a0 2024-04-02T13:56:20.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.216+0000 7f57b6d9d640 1 --2- 172.21.15.52:0/2435774949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b04e970c0 0x564b04e9f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:56:20.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.216+0000 7f57affff640 1 --2- 172.21.15.52:0/2435774949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b04e9fca0 0x564b04ea2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:56:20.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.216+0000 7f57b759e640 1 --2- 172.21.15.52:0/2435774949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e967a0 0x564b04e96b80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009ca0 tx=0x7f57b002ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=408864e673102546 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:20.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.216+0000 7f57ad7fa640 1 -- 172.21.15.52:0/2435774949 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0039040 con 0x564b04e967a0 2024-04-02T13:56:20.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.216+0000 7f57ad7fa640 1 -- 172.21.15.52:0/2435774949 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57b0003b70 con 0x564b04e967a0 2024-04-02T13:56:20.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.216+0000 7f57ad7fa640 1 -- 172.21.15.52:0/2435774949 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0034440 con 0x564b04e967a0 2024-04-02T13:56:20.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.217+0000 7f57bfddc740 1 -- 172.21.15.52:0/2435774949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e967a0 msgr2=0x564b04e96b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:20.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.217+0000 7f57b759e640 1 --2- 172.21.15.52:0/3054065233 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f57a40a3d20 0x7f57a40a61e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f57b004c000 tx=0x7f57b0048000 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:20.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.217+0000 7f57bfddc740 1 --2- 172.21.15.52:0/2435774949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e967a0 0x564b04e96b80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009ca0 tx=0x7f57b002ceb0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.217+0000 7f57bfddc740 1 -- 172.21.15.52:0/2435774949 shutdown_connections 2024-04-02T13:56:20.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.217+0000 7f57bfddc740 1 --2- 172.21.15.52:0/2435774949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b04e9fca0 0x564b04ea2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.217+0000 7f57bfddc740 1 --2- 172.21.15.52:0/2435774949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b04e970c0 0x564b04e9f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.217+0000 7f57bfddc740 1 --2- 172.21.15.52:0/2435774949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e967a0 0x564b04e96b80 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.217+0000 7f57bfddc740 1 -- 172.21.15.52:0/2435774949 >> 172.21.15.52:0/2435774949 conn(0x564b04e90ed0 msgr2=0x564b04e912b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:20.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.217+0000 7f57bfddc740 1 -- 172.21.15.52:0/2435774949 shutdown_connections 2024-04-02T13:56:20.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.217+0000 7f57bfddc740 1 -- 172.21.15.52:0/2435774949 wait complete. 2024-04-02T13:56:20.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.217+0000 7f57bfddc740 1 -- start start 2024-04-02T13:56:20.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.217+0000 7f57bfddc740 1 Processor -- start 2024-04-02T13:56:20.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.218+0000 7f57bfddc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e967a0 0x564b04e9f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.218+0000 7f57bfddc740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b04e970c0 0x564b04e9ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.218+0000 7f57bfddc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b04e9fca0 0x564b04e9d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.218+0000 7f57bfddc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564b04ea2a90 con 0x564b04e967a0 2024-04-02T13:56:20.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.218+0000 7f57bfddc740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564b04db5e80 con 0x564b04e970c0 2024-04-02T13:56:20.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.218+0000 7f57bfddc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564b04ea2c10 con 0x564b04e9fca0 2024-04-02T13:56:20.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.218+0000 7f57bfddc740 10 client.0 _ll_register_callbacks cb 0x564b04df1aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:20.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.218+0000 7f57b6d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e967a0 0x564b04e9f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.218+0000 7f57b6d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e967a0 0x564b04e9f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38178/0 (socket says 172.21.15.52:38178) 2024-04-02T13:56:20.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.218+0000 7f57b6d9d640 1 -- 172.21.15.52:0/3961030098 learned_addr learned my addr 172.21.15.52:0/3961030098 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:20.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.218+0000 7f57affff640 1 --2- 172.21.15.52:0/3961030098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b04e970c0 0x564b04e9ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.218+0000 7f57b759e640 1 --2- 172.21.15.52:0/3961030098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b04e9fca0 0x564b04e9d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.218+0000 7f57b6d9d640 1 -- 172.21.15.52:0/3961030098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b04e9fca0 msgr2=0x564b04e9d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:20.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.218+0000 7f57b6d9d640 1 --2- 172.21.15.52:0/3961030098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b04e9fca0 0x564b04e9d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.218+0000 7f57b6d9d640 1 -- 172.21.15.52:0/3961030098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b04e970c0 msgr2=0x564b04e9ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:20.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.218+0000 7f57b6d9d640 1 --2- 172.21.15.52:0/3961030098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b04e970c0 0x564b04e9ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.218+0000 7f57b6d9d640 1 -- 172.21.15.52:0/3961030098 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b04bd87d0 con 0x564b04e967a0 2024-04-02T13:56:20.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.218+0000 7f57b6d9d640 1 --2- 172.21.15.52:0/3961030098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e967a0 0x564b04e9f640 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f57a8009610 tx=0x7f57a8004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ac437d3257345d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:20.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.218+0000 7f579e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:20.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.218+0000 7f579e7fc640 1 -- 172.21.15.52:0/3961030098 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8020040 con 0x564b04e967a0 2024-04-02T13:56:20.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.218+0000 7f579e7fc640 1 -- 172.21.15.52:0/3961030098 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f57a8098910 con 0x564b04e967a0 2024-04-02T13:56:20.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.218+0000 7f57bfddc740 1 -- 172.21.15.52:0/3961030098 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57b0003b70 con 0x564b04e967a0 2024-04-02T13:56:20.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.219+0000 7f579e7fc640 1 -- 172.21.15.52:0/3961030098 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f57a80ba050 con 0x564b04e967a0 2024-04-02T13:56:20.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.219+0000 7f57bfddc740 10 client.16500 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:20.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.219+0000 7f57bfddc740 1 -- 172.21.15.52:0/3961030098 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564b04eaea90 con 0x564b04e967a0 2024-04-02T13:56:20.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.219+0000 7f579e7fc640 1 -- 172.21.15.52:0/3961030098 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57b0003b70 con 0x564b04e967a0 2024-04-02T13:56:20.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.219+0000 7f579e7fc640 1 -- 172.21.15.52:0/3961030098 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f57a80cdcd0 con 0x564b04e967a0 2024-04-02T13:56:20.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.219+0000 7f57bfddc740 10 client.16500 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:20.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.219+0000 7f57bfddc740 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:20.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.219+0000 7f57bfddc740 20 client.16500 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:20.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.219+0000 7f57bfddc740 1 --2- 172.21.15.52:0/3961030098 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564b04eaffa0 0x564b04eb2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.219+0000 7f57bfddc740 4 client.16500 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:20.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.219+0000 7f57bfddc740 1 -- 172.21.15.52:0/3961030098 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b04e07cf0 con 0x564b04eaffa0 2024-04-02T13:56:20.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.219+0000 7f57affff640 1 --2- 172.21.15.52:0/3961030098 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564b04eaffa0 0x564b04eb2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.220+0000 7f579e7fc640 10 client.16500 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:20.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.220+0000 7f57affff640 1 --2- 172.21.15.52:0/3961030098 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564b04eaffa0 0x564b04eb2410 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af40d63956601a14 server_cookie=ac1943fb2d19d546 in_seq=0 out_seq=0 2024-04-02T13:56:20.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.220+0000 7f579e7fc640 1 -- 172.21.15.52:0/3961030098 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564b04e07cf0 con 0x564b04eaffa0 2024-04-02T13:56:20.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.220+0000 7f579e7fc640 10 client.16500 handle_command_reply: tid=0 2024-04-02T13:56:20.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.220+0000 7f57bfddc740 1 client.16500 shutdown 2024-04-02T13:56:20.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.220+0000 7f57bfddc740 1 -- 172.21.15.52:0/3961030098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e967a0 msgr2=0x564b04e9f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:20.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.220+0000 7f57bfddc740 1 --2- 172.21.15.52:0/3961030098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e967a0 0x564b04e9f640 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f57a8009610 tx=0x7f57a8004780 comp rx=0 tx=0).stop 2024-04-02T13:56:20.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.220+0000 7f57bfddc740 1 -- 172.21.15.52:0/3961030098 shutdown_connections 2024-04-02T13:56:20.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.220+0000 7f57bfddc740 1 --2- 172.21.15.52:0/3961030098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b04e9fca0 0x564b04e9d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.220+0000 7f57bfddc740 1 --2- 172.21.15.52:0/3961030098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b04e970c0 0x564b04e9ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.220+0000 7f57bfddc740 1 --2- 172.21.15.52:0/3961030098 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564b04eaffa0 0x564b04eb2410 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.220+0000 7f57bfddc740 1 --2- 172.21.15.52:0/3961030098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e967a0 0x564b04e9f640 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.220+0000 7f579e7fc640 0 client.16500 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:20.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.220+0000 7f57bfddc740 1 -- 172.21.15.52:0/3961030098 >> 172.21.15.52:0/3961030098 conn(0x564b04e90ed0 msgr2=0x564b04e92c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:20.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.221+0000 7f57bfddc740 1 -- 172.21.15.52:0/3961030098 shutdown_connections 2024-04-02T13:56:20.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.221+0000 7f57bfddc740 1 -- 172.21.15.52:0/3961030098 wait complete. 2024-04-02T13:56:20.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.221+0000 7f57bfddc740 20 client.16500 trim_cache size 0 max 16384 2024-04-02T13:56:20.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.234+0000 7f57bfddc740 1 Processor -- start 2024-04-02T13:56:20.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.234+0000 7f57bfddc740 1 -- start start 2024-04-02T13:56:20.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.234+0000 7f57bfddc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e967a0 0x564b04eab980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.234+0000 7f57bfddc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b04e970c0 0x564b04e9f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.234+0000 7f57bfddc740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b04e9fca0 0x564b04eb7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.234+0000 7f57bfddc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f57b81b0fc0 con 0x564b04e967a0 2024-04-02T13:56:20.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.234+0000 7f57bfddc740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f57b81b2b30 con 0x564b04e9fca0 2024-04-02T13:56:20.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.234+0000 7f57bfddc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f57b81b1690 con 0x564b04e970c0 2024-04-02T13:56:20.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.234+0000 7f57b6d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e967a0 0x564b04eab980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.234+0000 7f57b6d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e967a0 0x564b04eab980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38192/0 (socket says 172.21.15.52:38192) 2024-04-02T13:56:20.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.234+0000 7f57b6d9d640 1 -- 172.21.15.52:0/2911506837 learned_addr learned my addr 172.21.15.52:0/2911506837 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:20.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.234+0000 7f57b6d9d640 1 -- 172.21.15.52:0/2911506837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b04e970c0 msgr2=0x564b04e9f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:20.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.234+0000 7f57b6d9d640 1 --2- 172.21.15.52:0/2911506837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b04e970c0 0x564b04e9f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.234+0000 7f57b759e640 1 --2- 172.21.15.52:0/2911506837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b04e9fca0 0x564b04eb7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.234+0000 7f57b6d9d640 1 -- 172.21.15.52:0/2911506837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b04e9fca0 msgr2=0x564b04eb7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:20.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.234+0000 7f57b6d9d640 1 --2- 172.21.15.52:0/2911506837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b04e9fca0 0x564b04eb7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.235+0000 7f57b6d9d640 1 -- 172.21.15.52:0/2911506837 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b04e96ef0 con 0x564b04e967a0 2024-04-02T13:56:20.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.235+0000 7f57b6d9d640 1 --2- 172.21.15.52:0/2911506837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e967a0 0x564b04eab980 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f57a8098c70 tx=0x7f57a809a7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563d5b9f53a079aa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:20.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.235+0000 7f57acff9640 1 -- 172.21.15.52:0/2911506837 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8020070 con 0x564b04e967a0 2024-04-02T13:56:20.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.235+0000 7f57acff9640 1 -- 172.21.15.52:0/2911506837 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57a80a0650 con 0x564b04e967a0 2024-04-02T13:56:20.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.235+0000 7f57acff9640 1 -- 172.21.15.52:0/2911506837 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a80cdcd0 con 0x564b04e967a0 2024-04-02T13:56:20.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.235+0000 7f57bfddc740 1 -- 172.21.15.52:0/2911506837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e967a0 msgr2=0x564b04eab980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:20.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.235+0000 7f57bfddc740 1 --2- 172.21.15.52:0/2911506837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e967a0 0x564b04eab980 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f57a8098c70 tx=0x7f57a809a7b0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.235+0000 7f57bfddc740 1 -- 172.21.15.52:0/2911506837 shutdown_connections 2024-04-02T13:56:20.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.235+0000 7f57bfddc740 1 --2- 172.21.15.52:0/2911506837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b04e9fca0 0x564b04eb7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.235+0000 7f57bfddc740 1 --2- 172.21.15.52:0/2911506837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b04e970c0 0x564b04e9f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.235+0000 7f57bfddc740 1 --2- 172.21.15.52:0/2911506837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e967a0 0x564b04eab980 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.235+0000 7f57bfddc740 1 -- 172.21.15.52:0/2911506837 >> 172.21.15.52:0/2911506837 conn(0x564b04e90ed0 msgr2=0x564b04e8f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:20.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.236+0000 7f57bfddc740 1 -- 172.21.15.52:0/2911506837 shutdown_connections 2024-04-02T13:56:20.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.236+0000 7f57bfddc740 1 -- 172.21.15.52:0/2911506837 wait complete. 2024-04-02T13:56:20.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.236+0000 7f57bfddc740 1 -- start start 2024-04-02T13:56:20.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.236+0000 7f57bfddc740 1 Processor -- start 2024-04-02T13:56:20.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.236+0000 7f57bfddc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b04e967a0 0x564b04ea8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.236+0000 7f57bfddc740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b04e970c0 0x564b04e99f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.236+0000 7f57bfddc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e9fca0 0x564b04e9a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.236+0000 7f57bfddc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f57b81b2430 con 0x564b04e9fca0 2024-04-02T13:56:20.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.236+0000 7f57bfddc740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f57b81b1d60 con 0x564b04e970c0 2024-04-02T13:56:20.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.236+0000 7f57bfddc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f57b81aefa0 con 0x564b04e967a0 2024-04-02T13:56:20.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.236+0000 7f57bfddc740 10 client.0 _ll_register_callbacks cb 0x7f57a800df20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:20.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.236+0000 7f57b759e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e9fca0 0x564b04e9a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.236+0000 7f57b759e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e9fca0 0x564b04e9a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38194/0 (socket says 172.21.15.52:38194) 2024-04-02T13:56:20.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.236+0000 7f57b759e640 1 -- 172.21.15.52:0/1191299245 learned_addr learned my addr 172.21.15.52:0/1191299245 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:20.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.236+0000 7f57b6d9d640 1 --2- 172.21.15.52:0/1191299245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b04e967a0 0x564b04ea8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.236+0000 7f57affff640 1 --2- 172.21.15.52:0/1191299245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b04e970c0 0x564b04e99f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.236+0000 7f57b759e640 1 -- 172.21.15.52:0/1191299245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b04e967a0 msgr2=0x564b04ea8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:20.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.236+0000 7f57b759e640 1 --2- 172.21.15.52:0/1191299245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b04e967a0 0x564b04ea8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.236+0000 7f57b759e640 1 -- 172.21.15.52:0/1191299245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b04e970c0 msgr2=0x564b04e99f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:20.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.236+0000 7f57b759e640 1 --2- 172.21.15.52:0/1191299245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b04e970c0 0x564b04e99f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.236+0000 7f57b759e640 1 -- 172.21.15.52:0/1191299245 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b04ea0bf0 con 0x564b04e9fca0 2024-04-02T13:56:20.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.237+0000 7f57b759e640 1 --2- 172.21.15.52:0/1191299245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e9fca0 0x564b04e9a560 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f57b0032d40 tx=0x7f57b00095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0db0019d074b9a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:20.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.237+0000 7f579e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:20.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.237+0000 7f579e7fc640 1 -- 172.21.15.52:0/1191299245 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0034440 con 0x564b04e9fca0 2024-04-02T13:56:20.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.237+0000 7f579e7fc640 1 -- 172.21.15.52:0/1191299245 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f57b0003b70 con 0x564b04e9fca0 2024-04-02T13:56:20.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.237+0000 7f57bfddc740 1 -- 172.21.15.52:0/1191299245 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57a80a0650 con 0x564b04e9fca0 2024-04-02T13:56:20.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.237+0000 7f579e7fc640 1 -- 172.21.15.52:0/1191299245 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f57b0066050 con 0x564b04e9fca0 2024-04-02T13:56:20.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.237+0000 7f57bfddc740 10 client.16506 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:20.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.237+0000 7f57bfddc740 1 -- 172.21.15.52:0/1191299245 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564b04f168f0 con 0x564b04e9fca0 2024-04-02T13:56:20.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.237+0000 7f579e7fc640 1 -- 172.21.15.52:0/1191299245 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57a80a0650 con 0x564b04e9fca0 2024-04-02T13:56:20.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.237+0000 7f579e7fc640 1 -- 172.21.15.52:0/1191299245 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f57b0004270 con 0x564b04e9fca0 2024-04-02T13:56:20.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.237+0000 7f57bfddc740 10 client.16506 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:20.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.237+0000 7f57bfddc740 10 client.16506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:20.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.237+0000 7f57bfddc740 20 client.16506 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:20.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.237+0000 7f57bfddc740 1 --2- 172.21.15.52:0/1191299245 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564b04f16c20 0x564b04f19040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:20.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.237+0000 7f57bfddc740 4 client.16506 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:56:20.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.237+0000 7f57bfddc740 1 -- 172.21.15.52:0/1191299245 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564b04e02040 con 0x564b04f16c20 2024-04-02T13:56:20.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.238+0000 7f57b6d9d640 1 --2- 172.21.15.52:0/1191299245 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564b04f16c20 0x564b04f19040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:20.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.238+0000 7f579e7fc640 10 client.16506 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:20.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.238+0000 7f57b6d9d640 1 --2- 172.21.15.52:0/1191299245 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564b04f16c20 0x564b04f19040 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a2277ebd156a6b server_cookie=986cdea7456a7ad0 in_seq=0 out_seq=0 2024-04-02T13:56:20.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.241+0000 7f579e7fc640 1 -- 172.21.15.52:0/1191299245 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564b04e02040 con 0x564b04f16c20 2024-04-02T13:56:20.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.241+0000 7f579e7fc640 10 client.16506 handle_command_reply: tid=0 2024-04-02T13:56:20.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.241+0000 7f57bfddc740 1 client.16506 shutdown 2024-04-02T13:56:20.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.241+0000 7f57bfddc740 1 -- 172.21.15.52:0/1191299245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e9fca0 msgr2=0x564b04e9a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:20.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.241+0000 7f57bfddc740 1 --2- 172.21.15.52:0/1191299245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e9fca0 0x564b04e9a560 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f57b0032d40 tx=0x7f57b00095a0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.241+0000 7f57bfddc740 1 -- 172.21.15.52:0/1191299245 shutdown_connections 2024-04-02T13:56:20.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.241+0000 7f57bfddc740 1 --2- 172.21.15.52:0/1191299245 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564b04f16c20 0x564b04f19040 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.241+0000 7f57bfddc740 1 --2- 172.21.15.52:0/1191299245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b04e9fca0 0x564b04e9a560 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.241+0000 7f57bfddc740 1 --2- 172.21.15.52:0/1191299245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b04e970c0 0x564b04e99f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.241+0000 7f579e7fc640 0 client.16506 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:20.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.241+0000 7f57bfddc740 1 --2- 172.21.15.52:0/1191299245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b04e967a0 0x564b04ea8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.241+0000 7f57bfddc740 1 -- 172.21.15.52:0/1191299245 >> 172.21.15.52:0/1191299245 conn(0x564b04e90ed0 msgr2=0x564b04e8ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:20.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.241+0000 7f57bfddc740 1 -- 172.21.15.52:0/1191299245 shutdown_connections 2024-04-02T13:56:20.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.241+0000 7f57bfddc740 1 -- 172.21.15.52:0/1191299245 wait complete. 2024-04-02T13:56:20.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.241+0000 7f57bfddc740 20 client.16506 trim_cache size 0 max 16384 2024-04-02T13:56:20.256 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:56:20.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.242+0000 7f57bd115640 1 -- 172.21.15.52:0/3054065233 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f57a40a3d20 msgr2=0x7f57a40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:20.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.242+0000 7f57bd115640 1 --2- 172.21.15.52:0/3054065233 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f57a40a3d20 0x7f57a40a61e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f57b004c000 tx=0x7f57b0048000 comp rx=0 tx=0).stop 2024-04-02T13:56:20.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.242+0000 7f57bd115640 1 -- 172.21.15.52:0/3054065233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b81008c0 msgr2=0x7f57b819a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:20.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.242+0000 7f57bd115640 1 --2- 172.21.15.52:0/3054065233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b81008c0 0x7f57b819a570 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f57a800d770 tx=0x7f57a800dc30 comp rx=0 tx=0).stop 2024-04-02T13:56:20.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.242+0000 7f57bd115640 1 -- 172.21.15.52:0/3054065233 shutdown_connections 2024-04-02T13:56:20.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.242+0000 7f57bd115640 1 --2- 172.21.15.52:0/3054065233 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f57a40a3d20 0x7f57a40a61e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.242+0000 7f57bd115640 1 --2- 172.21.15.52:0/3054065233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57b810e470 0x7f57b819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.242+0000 7f57bd115640 1 --2- 172.21.15.52:0/3054065233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57b81011e0 0x7f57b819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.242+0000 7f57bd115640 1 --2- 172.21.15.52:0/3054065233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b81008c0 0x7f57b819a570 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:20.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.242+0000 7f57bd115640 1 -- 172.21.15.52:0/3054065233 >> 172.21.15.52:0/3054065233 conn(0x7f57b8076d80 msgr2=0x7f57b810f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:20.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.242+0000 7f57bd115640 1 -- 172.21.15.52:0/3054065233 shutdown_connections 2024-04-02T13:56:20.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:20.242+0000 7f57bd115640 1 -- 172.21.15.52:0/3054065233 wait complete. 2024-04-02T13:56:21.261 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:56:21.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:56:21.261 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:56:21.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.380+0000 7ff92a43a640 1 -- 172.21.15.52:0/1945462241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff924068ce0 msgr2=0x7ff9240690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:21.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.380+0000 7ff92a43a640 1 --2- 172.21.15.52:0/1945462241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff924068ce0 0x7ff9240690c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff914009900 tx=0x7ff91402bc30 comp rx=0 tx=0).stop 2024-04-02T13:56:21.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.380+0000 7ff92a43a640 1 -- 172.21.15.52:0/1945462241 shutdown_connections 2024-04-02T13:56:21.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.380+0000 7ff92a43a640 1 --2- 172.21.15.52:0/1945462241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff92410e4a0 0x7ff924110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.380+0000 7ff92a43a640 1 --2- 172.21.15.52:0/1945462241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff924069600 0x7ff92410df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.380+0000 7ff92a43a640 1 --2- 172.21.15.52:0/1945462241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff924068ce0 0x7ff9240690c0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.380+0000 7ff92a43a640 1 -- 172.21.15.52:0/1945462241 >> 172.21.15.52:0/1945462241 conn(0x7ff924077b00 msgr2=0x7ff924077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:21.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.380+0000 7ff92a43a640 1 -- 172.21.15.52:0/1945462241 shutdown_connections 2024-04-02T13:56:21.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.380+0000 7ff92a43a640 1 -- 172.21.15.52:0/1945462241 wait complete. 2024-04-02T13:56:21.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.380+0000 7ff92a43a640 1 Processor -- start 2024-04-02T13:56:21.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.380+0000 7ff92a43a640 1 -- start start 2024-04-02T13:56:21.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.381+0000 7ff92a43a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff924068ce0 0x7ff92419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.381+0000 7ff92a43a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff924069600 0x7ff92419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.381+0000 7ff92a43a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff92410e4a0 0x7ff92419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.381+0000 7ff92a43a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff924114060 con 0x7ff92410e4a0 2024-04-02T13:56:21.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.381+0000 7ff92a43a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff924113ee0 con 0x7ff924068ce0 2024-04-02T13:56:21.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.381+0000 7ff92a43a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241141e0 con 0x7ff924069600 2024-04-02T13:56:21.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.381+0000 7ff9289d7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff92410e4a0 0x7ff92419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.381+0000 7ff9289d7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff92410e4a0 0x7ff92419ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38222/0 (socket says 172.21.15.52:38222) 2024-04-02T13:56:21.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.381+0000 7ff9289d7640 1 -- 172.21.15.52:0/1826653614 learned_addr learned my addr 172.21.15.52:0/1826653614 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:21.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.381+0000 7ff923fff640 1 --2- 172.21.15.52:0/1826653614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff924068ce0 0x7ff92419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.381+0000 7ff9237fe640 1 --2- 172.21.15.52:0/1826653614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff924069600 0x7ff92419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.381+0000 7ff9289d7640 1 -- 172.21.15.52:0/1826653614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff924069600 msgr2=0x7ff92419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:21.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.381+0000 7ff9289d7640 1 --2- 172.21.15.52:0/1826653614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff924069600 0x7ff92419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.381+0000 7ff9289d7640 1 -- 172.21.15.52:0/1826653614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff924068ce0 msgr2=0x7ff92419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:21.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.381+0000 7ff9289d7640 1 --2- 172.21.15.52:0/1826653614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff924068ce0 0x7ff92419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.381+0000 7ff9289d7640 1 -- 172.21.15.52:0/1826653614 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92419f490 con 0x7ff92410e4a0 2024-04-02T13:56:21.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.381+0000 7ff923fff640 1 --2- 172.21.15.52:0/1826653614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff924068ce0 0x7ff92419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:21.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.381+0000 7ff9289d7640 1 --2- 172.21.15.52:0/1826653614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff92410e4a0 0x7ff92419ed60 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff91c00ccb0 tx=0x7ff91c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:21.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.381+0000 7ff9217fa640 1 -- 172.21.15.52:0/1826653614 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91c00f450 con 0x7ff92410e4a0 2024-04-02T13:56:21.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.382+0000 7ff9217fa640 1 -- 172.21.15.52:0/1826653614 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff91c00fad0 con 0x7ff92410e4a0 2024-04-02T13:56:21.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.382+0000 7ff92a43a640 1 -- 172.21.15.52:0/1826653614 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff92419f780 con 0x7ff92410e4a0 2024-04-02T13:56:21.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.382+0000 7ff9217fa640 1 -- 172.21.15.52:0/1826653614 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91c018bf0 con 0x7ff92410e4a0 2024-04-02T13:56:21.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.382+0000 7ff92a43a640 1 -- 172.21.15.52:0/1826653614 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9241a7010 con 0x7ff92410e4a0 2024-04-02T13:56:21.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.384+0000 7ff9217fa640 1 -- 172.21.15.52:0/1826653614 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff91c00f5f0 con 0x7ff92410e4a0 2024-04-02T13:56:21.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.384+0000 7ff92d01a740 1 Processor -- start 2024-04-02T13:56:21.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.384+0000 7ff92d01a740 1 -- start start 2024-04-02T13:56:21.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.384+0000 7ff92d01a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5585676602c0 0x5585676606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.384+0000 7ff92d01a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558567660be0 0x558567669280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.384+0000 7ff92d01a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585676697c0 0x55856766bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.384+0000 7ff92d01a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5585675c4770 con 0x5585676697c0 2024-04-02T13:56:21.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.384+0000 7ff92d01a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55856756c960 con 0x5585676602c0 2024-04-02T13:56:21.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.384+0000 7ff92d01a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5585675c08c0 con 0x558567660be0 2024-04-02T13:56:21.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.385+0000 7ff9217fa640 1 --2- 172.21.15.52:0/1826653614 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff9100a3d20 0x7ff9100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.385+0000 7ff9237fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585676697c0 0x55856766bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.385+0000 7ff9237fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585676697c0 0x55856766bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38232/0 (socket says 172.21.15.52:38232) 2024-04-02T13:56:21.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.385+0000 7ff9237fe640 1 -- 172.21.15.52:0/3239833646 learned_addr learned my addr 172.21.15.52:0/3239833646 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:21.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.385+0000 7ff9237fe640 1 --2- 172.21.15.52:0/3239833646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5585676602c0 0x5585676606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.385+0000 7ff9289d7640 1 --2- 172.21.15.52:0/1826653614 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff9100a3d20 0x7ff9100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.385+0000 7ff923fff640 1 --2- 172.21.15.52:0/3239833646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558567660be0 0x558567669280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.385+0000 7ff9217fa640 1 -- 172.21.15.52:0/1826653614 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7ff91c0d1260 con 0x7ff92410e4a0 2024-04-02T13:56:21.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.385+0000 7ff9237fe640 1 -- 172.21.15.52:0/3239833646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558567660be0 msgr2=0x558567669280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:21.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.385+0000 7ff9237fe640 1 --2- 172.21.15.52:0/3239833646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558567660be0 0x558567669280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.385+0000 7ff9237fe640 1 -- 172.21.15.52:0/3239833646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5585676602c0 msgr2=0x5585676606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:21.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.385+0000 7ff9237fe640 1 --2- 172.21.15.52:0/3239833646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5585676602c0 0x5585676606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.385+0000 7ff9237fe640 1 -- 172.21.15.52:0/3239833646 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585673a07e0 con 0x5585676697c0 2024-04-02T13:56:21.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.385+0000 7ff9237fe640 1 --2- 172.21.15.52:0/3239833646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585676697c0 0x55856766bbb0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff90c013930 tx=0x7ff90c00b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61928d863301498 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:21.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.385+0000 7ff9289d7640 1 --2- 172.21.15.52:0/1826653614 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff9100a3d20 0x7ff9100a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff91c09cbc0 tx=0x7ff91c06e900 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:21.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.385+0000 7ff91a7fc640 1 -- 172.21.15.52:0/3239833646 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c00e950 con 0x5585676697c0 2024-04-02T13:56:21.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.385+0000 7ff91a7fc640 1 -- 172.21.15.52:0/3239833646 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff90c011440 con 0x5585676697c0 2024-04-02T13:56:21.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.385+0000 7ff91a7fc640 1 -- 172.21.15.52:0/3239833646 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c010630 con 0x5585676697c0 2024-04-02T13:56:21.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.386+0000 7ff92d01a740 1 -- 172.21.15.52:0/3239833646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585676697c0 msgr2=0x55856766bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:21.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.386+0000 7ff92d01a740 1 --2- 172.21.15.52:0/3239833646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585676697c0 0x55856766bbb0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff90c013930 tx=0x7ff90c00b460 comp rx=0 tx=0).stop 2024-04-02T13:56:21.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.386+0000 7ff92d01a740 1 -- 172.21.15.52:0/3239833646 shutdown_connections 2024-04-02T13:56:21.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.386+0000 7ff92d01a740 1 --2- 172.21.15.52:0/3239833646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585676697c0 0x55856766bbb0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.386+0000 7ff92d01a740 1 --2- 172.21.15.52:0/3239833646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558567660be0 0x558567669280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.386+0000 7ff92d01a740 1 --2- 172.21.15.52:0/3239833646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5585676602c0 0x5585676606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.386+0000 7ff92d01a740 1 -- 172.21.15.52:0/3239833646 >> 172.21.15.52:0/3239833646 conn(0x55856765a9f0 msgr2=0x55856765add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:21.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.386+0000 7ff92d01a740 1 -- 172.21.15.52:0/3239833646 shutdown_connections 2024-04-02T13:56:21.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.386+0000 7ff92d01a740 1 -- 172.21.15.52:0/3239833646 wait complete. 2024-04-02T13:56:21.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.386+0000 7ff92d01a740 1 -- start start 2024-04-02T13:56:21.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.387+0000 7ff92d01a740 1 Processor -- start 2024-04-02T13:56:21.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.387+0000 7ff92d01a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5585676602c0 0x5585676667c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.387+0000 7ff92d01a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558567660be0 0x558567666d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.387+0000 7ff92d01a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585676697c0 0x558567667240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.387+0000 7ff92d01a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55856766c5b0 con 0x5585676697c0 2024-04-02T13:56:21.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.387+0000 7ff92d01a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55856757fd20 con 0x5585676602c0 2024-04-02T13:56:21.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.387+0000 7ff92d01a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55856766c730 con 0x558567660be0 2024-04-02T13:56:21.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.387+0000 7ff92d01a740 10 client.0 _ll_register_callbacks cb 0x55856754ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:21.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.387+0000 7ff9237fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585676697c0 0x558567667240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.387+0000 7ff9237fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585676697c0 0x558567667240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38240/0 (socket says 172.21.15.52:38240) 2024-04-02T13:56:21.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.387+0000 7ff9237fe640 1 -- 172.21.15.52:0/1406391280 learned_addr learned my addr 172.21.15.52:0/1406391280 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:21.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.387+0000 7ff9237fe640 1 --2- 172.21.15.52:0/1406391280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5585676602c0 0x5585676667c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.387+0000 7ff923fff640 1 --2- 172.21.15.52:0/1406391280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558567660be0 0x558567666d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.387+0000 7ff9237fe640 1 -- 172.21.15.52:0/1406391280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558567660be0 msgr2=0x558567666d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:21.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.387+0000 7ff9237fe640 1 --2- 172.21.15.52:0/1406391280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558567660be0 0x558567666d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.388+0000 7ff9237fe640 1 -- 172.21.15.52:0/1406391280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5585676602c0 msgr2=0x5585676667c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:21.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.388+0000 7ff9237fe640 1 --2- 172.21.15.52:0/1406391280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5585676602c0 0x5585676667c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.388+0000 7ff9237fe640 1 -- 172.21.15.52:0/1406391280 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558567400f70 con 0x5585676697c0 2024-04-02T13:56:21.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.388+0000 7ff923fff640 1 --2- 172.21.15.52:0/1406391280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558567660be0 0x558567666d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:21.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.388+0000 7ff9237fe640 1 --2- 172.21.15.52:0/1406391280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585676697c0 0x558567667240 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff90c005fd0 tx=0x7ff90c011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ffb07c9138d73bd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:21.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.388+0000 7ff9037fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:21.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.388+0000 7ff9037fe640 1 -- 172.21.15.52:0/1406391280 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c0115f0 con 0x5585676697c0 2024-04-02T13:56:21.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.388+0000 7ff9037fe640 1 -- 172.21.15.52:0/1406391280 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7ff90c011790 con 0x5585676697c0 2024-04-02T13:56:21.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.388+0000 7ff92d01a740 1 -- 172.21.15.52:0/1406391280 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff90c011440 con 0x5585676697c0 2024-04-02T13:56:21.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.388+0000 7ff9037fe640 1 -- 172.21.15.52:0/1406391280 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff90c034050 con 0x5585676697c0 2024-04-02T13:56:21.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.388+0000 7ff92d01a740 10 client.16518 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:21.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.388+0000 7ff92d01a740 1 -- 172.21.15.52:0/1406391280 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558567298170 con 0x5585676697c0 2024-04-02T13:56:21.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.388+0000 7ff9037fe640 1 -- 172.21.15.52:0/1406391280 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff90c011440 con 0x5585676697c0 2024-04-02T13:56:21.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.388+0000 7ff9037fe640 1 -- 172.21.15.52:0/1406391280 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7ff90c00e840 con 0x5585676697c0 2024-04-02T13:56:21.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.388+0000 7ff92d01a740 10 client.16518 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:21.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.388+0000 7ff92d01a740 10 client.16518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:21.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.388+0000 7ff92d01a740 20 client.16518 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:21.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.388+0000 7ff92d01a740 1 --2- 172.21.15.52:0/1406391280 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5585676799f0 0x55856767bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.389+0000 7ff92d01a740 4 client.16518 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:21.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.389+0000 7ff92d01a740 1 -- 172.21.15.52:0/1406391280 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585675d1cb0 con 0x5585676799f0 2024-04-02T13:56:21.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.389+0000 7ff923fff640 1 --2- 172.21.15.52:0/1406391280 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5585676799f0 0x55856767bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.390+0000 7ff923fff640 1 --2- 172.21.15.52:0/1406391280 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5585676799f0 0x55856767bde0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=971eaba760b6bb79 server_cookie=594d036c3904e510 in_seq=0 out_seq=0 2024-04-02T13:56:21.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.390+0000 7ff9037fe640 10 client.16518 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:21.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.390+0000 7ff9037fe640 1 -- 172.21.15.52:0/1406391280 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5585675d1cb0 con 0x5585676799f0 2024-04-02T13:56:21.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.390+0000 7ff9037fe640 10 client.16518 handle_command_reply: tid=0 2024-04-02T13:56:21.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.390+0000 7ff92d01a740 1 client.16518 shutdown 2024-04-02T13:56:21.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.391+0000 7ff92d01a740 1 -- 172.21.15.52:0/1406391280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585676697c0 msgr2=0x558567667240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:21.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.391+0000 7ff92d01a740 1 --2- 172.21.15.52:0/1406391280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585676697c0 0x558567667240 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff90c005fd0 tx=0x7ff90c011e30 comp rx=0 tx=0).stop 2024-04-02T13:56:21.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.391+0000 7ff92d01a740 1 -- 172.21.15.52:0/1406391280 shutdown_connections 2024-04-02T13:56:21.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.391+0000 7ff92d01a740 1 --2- 172.21.15.52:0/1406391280 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5585676799f0 0x55856767bde0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.391+0000 7ff92d01a740 1 --2- 172.21.15.52:0/1406391280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585676697c0 0x558567667240 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.391+0000 7ff92d01a740 1 --2- 172.21.15.52:0/1406391280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558567660be0 0x558567666d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.391+0000 7ff92d01a740 1 --2- 172.21.15.52:0/1406391280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5585676602c0 0x5585676667c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.391+0000 7ff9037fe640 0 client.16518 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:21.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.391+0000 7ff92d01a740 1 -- 172.21.15.52:0/1406391280 >> 172.21.15.52:0/1406391280 conn(0x55856765a9f0 msgr2=0x55856765c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:21.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.391+0000 7ff92d01a740 1 -- 172.21.15.52:0/1406391280 shutdown_connections 2024-04-02T13:56:21.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.391+0000 7ff92d01a740 1 -- 172.21.15.52:0/1406391280 wait complete. 2024-04-02T13:56:21.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.391+0000 7ff92d01a740 20 client.16518 trim_cache size 0 max 16384 2024-04-02T13:56:21.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.409+0000 7ff92d01a740 1 Processor -- start 2024-04-02T13:56:21.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.409+0000 7ff92d01a740 1 -- start start 2024-04-02T13:56:21.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.409+0000 7ff92d01a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5585676602c0 0x55856765c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.409+0000 7ff92d01a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558567660be0 0x558567675470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.409+0000 7ff92d01a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585676697c0 0x558567665f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.409+0000 7ff92d01a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241b0fb0 con 0x558567660be0 2024-04-02T13:56:21.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.409+0000 7ff92d01a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241b2b20 con 0x5585676602c0 2024-04-02T13:56:21.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.409+0000 7ff92d01a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241b1680 con 0x5585676697c0 2024-04-02T13:56:21.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.409+0000 7ff923fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558567660be0 0x558567675470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.409+0000 7ff923fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558567660be0 0x558567675470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38252/0 (socket says 172.21.15.52:38252) 2024-04-02T13:56:21.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.409+0000 7ff923fff640 1 -- 172.21.15.52:0/3079096422 learned_addr learned my addr 172.21.15.52:0/3079096422 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:21.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.409+0000 7ff9237fe640 1 --2- 172.21.15.52:0/3079096422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5585676602c0 0x55856765c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.409+0000 7ff9237fe640 1 --2- 172.21.15.52:0/3079096422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585676697c0 0x558567665f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.409+0000 7ff923fff640 1 -- 172.21.15.52:0/3079096422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585676697c0 msgr2=0x558567665f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:21.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.409+0000 7ff923fff640 1 --2- 172.21.15.52:0/3079096422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585676697c0 0x558567665f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.409+0000 7ff923fff640 1 -- 172.21.15.52:0/3079096422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5585676602c0 msgr2=0x55856765c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:21.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.409+0000 7ff923fff640 1 --2- 172.21.15.52:0/3079096422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5585676602c0 0x55856765c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.409+0000 7ff923fff640 1 -- 172.21.15.52:0/3079096422 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558567660a10 con 0x558567660be0 2024-04-02T13:56:21.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.409+0000 7ff923fff640 1 --2- 172.21.15.52:0/3079096422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558567660be0 0x558567675470 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff914009900 tx=0x7ff91402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed055aa5c0f2a999 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:21.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.410+0000 7ff919ffb640 1 -- 172.21.15.52:0/3079096422 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914048bc0 con 0x558567660be0 2024-04-02T13:56:21.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.410+0000 7ff919ffb640 1 -- 172.21.15.52:0/3079096422 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff914048d60 con 0x558567660be0 2024-04-02T13:56:21.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.410+0000 7ff919ffb640 1 -- 172.21.15.52:0/3079096422 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9140046c0 con 0x558567660be0 2024-04-02T13:56:21.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.410+0000 7ff92d01a740 1 -- 172.21.15.52:0/3079096422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558567660be0 msgr2=0x558567675470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:21.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.410+0000 7ff92d01a740 1 --2- 172.21.15.52:0/3079096422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558567660be0 0x558567675470 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff914009900 tx=0x7ff91402e040 comp rx=0 tx=0).stop 2024-04-02T13:56:21.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.410+0000 7ff92d01a740 1 -- 172.21.15.52:0/3079096422 shutdown_connections 2024-04-02T13:56:21.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.410+0000 7ff92d01a740 1 --2- 172.21.15.52:0/3079096422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585676697c0 0x558567665f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.410+0000 7ff92d01a740 1 --2- 172.21.15.52:0/3079096422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558567660be0 0x558567675470 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.410+0000 7ff92d01a740 1 --2- 172.21.15.52:0/3079096422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5585676602c0 0x55856765c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.410+0000 7ff92d01a740 1 -- 172.21.15.52:0/3079096422 >> 172.21.15.52:0/3079096422 conn(0x55856765a9f0 msgr2=0x558567667010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:21.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.410+0000 7ff92d01a740 1 -- 172.21.15.52:0/3079096422 shutdown_connections 2024-04-02T13:56:21.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.410+0000 7ff92d01a740 1 -- 172.21.15.52:0/3079096422 wait complete. 2024-04-02T13:56:21.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.411+0000 7ff92d01a740 1 -- start start 2024-04-02T13:56:21.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.411+0000 7ff92d01a740 1 Processor -- start 2024-04-02T13:56:21.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.411+0000 7ff92d01a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5585676602c0 0x558567671af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.411+0000 7ff92d01a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558567660be0 0x55856766cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.411+0000 7ff92d01a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585676697c0 0x55856766d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.411+0000 7ff92d01a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241b2420 con 0x558567660be0 2024-04-02T13:56:21.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.411+0000 7ff92d01a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241b1d50 con 0x5585676602c0 2024-04-02T13:56:21.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.411+0000 7ff92d01a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241aef90 con 0x5585676697c0 2024-04-02T13:56:21.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.411+0000 7ff92d01a740 10 client.0 _ll_register_callbacks cb 0x7ff90c00bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:21.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.411+0000 7ff923fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558567660be0 0x55856766cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.411+0000 7ff923fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558567660be0 0x55856766cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38262/0 (socket says 172.21.15.52:38262) 2024-04-02T13:56:21.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.411+0000 7ff923fff640 1 -- 172.21.15.52:0/2744551460 learned_addr learned my addr 172.21.15.52:0/2744551460 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:21.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.411+0000 7ff9237fe640 1 --2- 172.21.15.52:0/2744551460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5585676602c0 0x558567671af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.411+0000 7ff9237fe640 1 --2- 172.21.15.52:0/2744551460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585676697c0 0x55856766d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.411+0000 7ff923fff640 1 -- 172.21.15.52:0/2744551460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585676697c0 msgr2=0x55856766d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:21.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.411+0000 7ff923fff640 1 --2- 172.21.15.52:0/2744551460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585676697c0 0x55856766d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.411+0000 7ff923fff640 1 -- 172.21.15.52:0/2744551460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5585676602c0 msgr2=0x558567671af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:21.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.411+0000 7ff923fff640 1 --2- 172.21.15.52:0/2744551460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5585676602c0 0x558567671af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.411+0000 7ff923fff640 1 -- 172.21.15.52:0/2744551460 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55856766da40 con 0x558567660be0 2024-04-02T13:56:21.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.411+0000 7ff9237fe640 1 --2- 172.21.15.52:0/2744551460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585676697c0 0x55856766d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:21.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.412+0000 7ff923fff640 1 --2- 172.21.15.52:0/2744551460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558567660be0 0x55856766cf40 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff91402c170 tx=0x7ff914000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95eafaaa943520c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:21.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.412+0000 7ff9037fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:21.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.412+0000 7ff9037fe640 1 -- 172.21.15.52:0/2744551460 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914056060 con 0x558567660be0 2024-04-02T13:56:21.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.412+0000 7ff9037fe640 1 -- 172.21.15.52:0/2744551460 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7ff9140489b0 con 0x558567660be0 2024-04-02T13:56:21.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.412+0000 7ff92d01a740 1 -- 172.21.15.52:0/2744551460 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff914048d60 con 0x558567660be0 2024-04-02T13:56:21.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.412+0000 7ff9037fe640 1 -- 172.21.15.52:0/2744551460 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff91405d050 con 0x558567660be0 2024-04-02T13:56:21.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.412+0000 7ff92d01a740 10 client.16524 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:21.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.412+0000 7ff92d01a740 1 -- 172.21.15.52:0/2744551460 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5585676dfc00 con 0x558567660be0 2024-04-02T13:56:21.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.412+0000 7ff9037fe640 1 -- 172.21.15.52:0/2744551460 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff914048d60 con 0x558567660be0 2024-04-02T13:56:21.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.412+0000 7ff9037fe640 1 -- 172.21.15.52:0/2744551460 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7ff914004f10 con 0x558567660be0 2024-04-02T13:56:21.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.412+0000 7ff92d01a740 10 client.16524 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:21.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.412+0000 7ff92d01a740 10 client.16524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:21.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.412+0000 7ff92d01a740 20 client.16524 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:21.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.412+0000 7ff92d01a740 1 --2- 172.21.15.52:0/2744551460 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5585676e0380 0x5585676e27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.412+0000 7ff92d01a740 4 client.16524 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:56:21.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.412+0000 7ff92d01a740 1 -- 172.21.15.52:0/2744551460 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5585675cc000 con 0x5585676e0380 2024-04-02T13:56:21.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.413+0000 7ff9237fe640 1 --2- 172.21.15.52:0/2744551460 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5585676e0380 0x5585676e27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.413+0000 7ff9037fe640 10 client.16524 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:21.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.413+0000 7ff9237fe640 1 --2- 172.21.15.52:0/2744551460 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5585676e0380 0x5585676e27a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea0c3b46e9e239bc server_cookie=580d8dece38cae6d in_seq=0 out_seq=0 2024-04-02T13:56:21.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.414+0000 7ff9037fe640 1 -- 172.21.15.52:0/2744551460 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5585675cc000 con 0x5585676e0380 2024-04-02T13:56:21.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.414+0000 7ff9037fe640 10 client.16524 handle_command_reply: tid=0 2024-04-02T13:56:21.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.414+0000 7ff92d01a740 1 client.16524 shutdown 2024-04-02T13:56:21.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.414+0000 7ff92d01a740 1 -- 172.21.15.52:0/2744551460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558567660be0 msgr2=0x55856766cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:21.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.414+0000 7ff92d01a740 1 --2- 172.21.15.52:0/2744551460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558567660be0 0x55856766cf40 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff91402c170 tx=0x7ff914000bc0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.414+0000 7ff92d01a740 1 -- 172.21.15.52:0/2744551460 shutdown_connections 2024-04-02T13:56:21.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.414+0000 7ff92d01a740 1 --2- 172.21.15.52:0/2744551460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585676697c0 0x55856766d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.414+0000 7ff92d01a740 1 --2- 172.21.15.52:0/2744551460 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5585676e0380 0x5585676e27a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.414+0000 7ff92d01a740 1 --2- 172.21.15.52:0/2744551460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558567660be0 0x55856766cf40 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.414+0000 7ff92d01a740 1 --2- 172.21.15.52:0/2744551460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5585676602c0 0x558567671af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.414+0000 7ff9037fe640 0 client.16524 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:21.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.414+0000 7ff92d01a740 1 -- 172.21.15.52:0/2744551460 >> 172.21.15.52:0/2744551460 conn(0x55856765a9f0 msgr2=0x558567666780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:21.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.414+0000 7ff92d01a740 1 -- 172.21.15.52:0/2744551460 shutdown_connections 2024-04-02T13:56:21.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.414+0000 7ff92d01a740 1 -- 172.21.15.52:0/2744551460 wait complete. 2024-04-02T13:56:21.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.414+0000 7ff92d01a740 20 client.16524 trim_cache size 0 max 16384 2024-04-02T13:56:21.444 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:56:21.445 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-04-02T13:56:21.445 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "97d404a0-2d0c-4a9c-a196-1d04ff8ce889", 2024-04-02T13:56:21.445 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-04-02T13:56:21.445 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:56:21.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.415+0000 7ff92a43a640 1 -- 172.21.15.52:0/1826653614 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff9100a3d20 msgr2=0x7ff9100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:21.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.415+0000 7ff92a43a640 1 --2- 172.21.15.52:0/1826653614 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff9100a3d20 0x7ff9100a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff91c09cbc0 tx=0x7ff91c06e900 comp rx=0 tx=0).stop 2024-04-02T13:56:21.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.415+0000 7ff92a43a640 1 -- 172.21.15.52:0/1826653614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff92410e4a0 msgr2=0x7ff92419ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:21.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.415+0000 7ff92a43a640 1 --2- 172.21.15.52:0/1826653614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff92410e4a0 0x7ff92419ed60 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff91c00ccb0 tx=0x7ff91c00a620 comp rx=0 tx=0).stop 2024-04-02T13:56:21.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.415+0000 7ff92a43a640 1 -- 172.21.15.52:0/1826653614 shutdown_connections 2024-04-02T13:56:21.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.415+0000 7ff92a43a640 1 --2- 172.21.15.52:0/1826653614 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff9100a3d20 0x7ff9100a61e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.415+0000 7ff92a43a640 1 --2- 172.21.15.52:0/1826653614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff92410e4a0 0x7ff92419ed60 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.415+0000 7ff92a43a640 1 --2- 172.21.15.52:0/1826653614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff924069600 0x7ff92419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.415+0000 7ff92a43a640 1 --2- 172.21.15.52:0/1826653614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff924068ce0 0x7ff92419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.415+0000 7ff92a43a640 1 -- 172.21.15.52:0/1826653614 >> 172.21.15.52:0/1826653614 conn(0x7ff924077b00 msgr2=0x7ff92410f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:21.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.415+0000 7ff92a43a640 1 -- 172.21.15.52:0/1826653614 shutdown_connections 2024-04-02T13:56:21.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.415+0000 7ff92a43a640 1 -- 172.21.15.52:0/1826653614 wait complete. 2024-04-02T13:56:21.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:56:21.449 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:56:21.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.559+0000 7f1feae31640 1 -- 172.21.15.52:0/3478184438 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1fe4106980 msgr2=0x7f1fe4106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:21.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.559+0000 7f1feae31640 1 --2- 172.21.15.52:0/3478184438 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1fe4106980 0x7f1fe4106d60 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc009930 tx=0x7f1fcc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:56:21.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.559+0000 7f1feae31640 1 -- 172.21.15.52:0/3478184438 shutdown_connections 2024-04-02T13:56:21.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.559+0000 7f1feae31640 1 --2- 172.21.15.52:0/3478184438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fe4101330 0x7f1fe410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.559+0000 7f1feae31640 1 --2- 172.21.15.52:0/3478184438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe4100970 0x7f1fe4100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.559+0000 7f1feae31640 1 --2- 172.21.15.52:0/3478184438 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1fe4106980 0x7f1fe4106d60 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.559+0000 7f1feae31640 1 -- 172.21.15.52:0/3478184438 >> 172.21.15.52:0/3478184438 conn(0x7f1fe4077b00 msgr2=0x7f1fe4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:21.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.559+0000 7f1feae31640 1 -- 172.21.15.52:0/3478184438 shutdown_connections 2024-04-02T13:56:21.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.559+0000 7f1feae31640 1 -- 172.21.15.52:0/3478184438 wait complete. 2024-04-02T13:56:21.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.560+0000 7f1feae31640 1 Processor -- start 2024-04-02T13:56:21.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.560+0000 7f1feae31640 1 -- start start 2024-04-02T13:56:21.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.560+0000 7f1feae31640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe4100970 0x7f1fe4102f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.560+0000 7f1feae31640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fe4101330 0x7f1fe4103490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.560+0000 7f1feae31640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1fe4106980 0x7f1fe41039d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.560+0000 7f1feae31640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe4110d90 con 0x7f1fe4106980 2024-04-02T13:56:21.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.560+0000 7f1feae31640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe4110c10 con 0x7f1fe4100970 2024-04-02T13:56:21.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.560+0000 7f1feae31640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe4110f10 con 0x7f1fe4101330 2024-04-02T13:56:21.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.560+0000 7f1fe93ce640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1fe4106980 0x7f1fe41039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.560+0000 7f1fe8bcd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe4100970 0x7f1fe4102f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.560+0000 7f1fe93ce640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1fe4106980 0x7f1fe41039d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35298/0 (socket says 172.21.15.52:35298) 2024-04-02T13:56:21.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.560+0000 7f1fe93ce640 1 -- 172.21.15.52:0/3532517548 learned_addr learned my addr 172.21.15.52:0/3532517548 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:21.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.560+0000 7f1fe3fff640 1 --2- 172.21.15.52:0/3532517548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fe4101330 0x7f1fe4103490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.560+0000 7f1fe93ce640 1 -- 172.21.15.52:0/3532517548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fe4101330 msgr2=0x7f1fe4103490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:21.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.560+0000 7f1fe93ce640 1 --2- 172.21.15.52:0/3532517548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fe4101330 0x7f1fe4103490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.560+0000 7f1fe93ce640 1 -- 172.21.15.52:0/3532517548 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe4100970 msgr2=0x7f1fe4102f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:21.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.560+0000 7f1fe93ce640 1 --2- 172.21.15.52:0/3532517548 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe4100970 0x7f1fe4102f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.560+0000 7f1fe93ce640 1 -- 172.21.15.52:0/3532517548 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe41a68e0 con 0x7f1fe4106980 2024-04-02T13:56:21.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.561+0000 7f1fe8bcd640 1 --2- 172.21.15.52:0/3532517548 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe4100970 0x7f1fe4102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:21.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.561+0000 7f1fe93ce640 1 --2- 172.21.15.52:0/3532517548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1fe4106980 0x7f1fe41039d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00eb10 tx=0x7f1fdc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:21.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.561+0000 7f1fe1ffb640 1 -- 172.21.15.52:0/3532517548 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc004b20 con 0x7f1fe4106980 2024-04-02T13:56:21.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.561+0000 7f1fe1ffb640 1 -- 172.21.15.52:0/3532517548 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fdc004cc0 con 0x7f1fe4106980 2024-04-02T13:56:21.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.561+0000 7f1feae31640 1 -- 172.21.15.52:0/3532517548 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe41a6bd0 con 0x7f1fe4106980 2024-04-02T13:56:21.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.561+0000 7f1fe1ffb640 1 -- 172.21.15.52:0/3532517548 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc010760 con 0x7f1fe4106980 2024-04-02T13:56:21.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.561+0000 7f1feae31640 1 -- 172.21.15.52:0/3532517548 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe41a7120 con 0x7f1fe4106980 2024-04-02T13:56:21.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.563+0000 7f1fe1ffb640 1 -- 172.21.15.52:0/3532517548 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1fdc005020 con 0x7f1fe4106980 2024-04-02T13:56:21.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.564+0000 7f1feda18740 1 Processor -- start 2024-04-02T13:56:21.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.564+0000 7f1feda18740 1 -- start start 2024-04-02T13:56:21.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.564+0000 7f1fe1ffb640 1 --2- 172.21.15.52:0/3532517548 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1fd00a3d20 0x7f1fd00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.564+0000 7f1feda18740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ea51f68d0 0x563ea51f6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.564+0000 7f1feda18740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 0x563ea51ff890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.564+0000 7f1feda18740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563ea51ffdd0 0x563ea52021c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.564+0000 7f1feda18740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563ea515a830 con 0x563ea51f71f0 2024-04-02T13:56:21.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.564+0000 7f1feda18740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563ea5102820 con 0x563ea51ffdd0 2024-04-02T13:56:21.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.564+0000 7f1feda18740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563ea51567e0 con 0x563ea51f68d0 2024-04-02T13:56:21.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.565+0000 7f1fe3fff640 1 --2- 172.21.15.52:0/3532517548 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1fd00a3d20 0x7f1fd00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.565+0000 7f1fe1ffb640 1 -- 172.21.15.52:0/3532517548 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f1fdc0d1170 con 0x7f1fe4106980 2024-04-02T13:56:21.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.565+0000 7f1fe8bcd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ea51f68d0 0x563ea51f6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.565+0000 7f1fe8bcd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ea51f68d0 0x563ea51f6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.52:39926/0 (socket says 172.21.15.52:39926) 2024-04-02T13:56:21.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.565+0000 7f1fe8bcd640 1 -- 172.21.15.52:0/2441038788 learned_addr learned my addr 172.21.15.52:0/2441038788 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:21.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.565+0000 7f1fe3fff640 1 --2- 172.21.15.52:0/2441038788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 0x563ea51ff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.565+0000 7f1fe93ce640 1 --2- 172.21.15.52:0/2441038788 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563ea51ffdd0 0x563ea52021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.565+0000 7f1fe8bcd640 1 -- 172.21.15.52:0/2441038788 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563ea51ffdd0 msgr2=0x563ea52021c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:21.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.565+0000 7f1fe8bcd640 1 --2- 172.21.15.52:0/2441038788 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563ea51ffdd0 0x563ea52021c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.565+0000 7f1fe8bcd640 1 -- 172.21.15.52:0/2441038788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 msgr2=0x563ea51ff890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:21.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.565+0000 7f1fe8bcd640 1 --2- 172.21.15.52:0/2441038788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 0x563ea51ff890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.565+0000 7f1fe8bcd640 1 -- 172.21.15.52:0/2441038788 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ea4f387d0 con 0x563ea51f68d0 2024-04-02T13:56:21.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.565+0000 7f1fe3fff640 1 --2- 172.21.15.52:0/3532517548 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1fd00a3d20 0x7f1fd00a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1fe41049e0 tx=0x7f1fd4009b60 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:21.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.565+0000 7f1fe8bcd640 1 --2- 172.21.15.52:0/2441038788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ea51f68d0 0x563ea51f6cb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc006010 tx=0x7f1fcc009630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdda0832933b99bd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:21.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.565+0000 7f1fdaffd640 1 -- 172.21.15.52:0/2441038788 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc042070 con 0x563ea51f68d0 2024-04-02T13:56:21.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.565+0000 7f1fdaffd640 1 -- 172.21.15.52:0/2441038788 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fcc004a70 con 0x563ea51f68d0 2024-04-02T13:56:21.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.566+0000 7f1fdaffd640 1 -- 172.21.15.52:0/2441038788 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc0044e0 con 0x563ea51f68d0 2024-04-02T13:56:21.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.566+0000 7f1feda18740 1 -- 172.21.15.52:0/2441038788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ea51f68d0 msgr2=0x563ea51f6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:21.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.566+0000 7f1feda18740 1 --2- 172.21.15.52:0/2441038788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ea51f68d0 0x563ea51f6cb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc006010 tx=0x7f1fcc009630 comp rx=0 tx=0).stop 2024-04-02T13:56:21.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.566+0000 7f1feda18740 1 -- 172.21.15.52:0/2441038788 shutdown_connections 2024-04-02T13:56:21.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.566+0000 7f1feda18740 1 --2- 172.21.15.52:0/2441038788 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563ea51ffdd0 0x563ea52021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.566+0000 7f1feda18740 1 --2- 172.21.15.52:0/2441038788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 0x563ea51ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.566+0000 7f1feda18740 1 --2- 172.21.15.52:0/2441038788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ea51f68d0 0x563ea51f6cb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.566+0000 7f1feda18740 1 -- 172.21.15.52:0/2441038788 >> 172.21.15.52:0/2441038788 conn(0x563ea51f1000 msgr2=0x563ea51f13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:21.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.566+0000 7f1feda18740 1 -- 172.21.15.52:0/2441038788 shutdown_connections 2024-04-02T13:56:21.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.566+0000 7f1feda18740 1 -- 172.21.15.52:0/2441038788 wait complete. 2024-04-02T13:56:21.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.567+0000 7f1feda18740 1 -- start start 2024-04-02T13:56:21.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.567+0000 7f1feda18740 1 Processor -- start 2024-04-02T13:56:21.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.567+0000 7f1feda18740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ea51f68d0 0x563ea51ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.567+0000 7f1feda18740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 0x563ea51fc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.567+0000 7f1feda18740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563ea51ffdd0 0x563ea51fceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.567+0000 7f1feda18740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563ea5202bb0 con 0x563ea51f71f0 2024-04-02T13:56:21.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.567+0000 7f1feda18740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563ea5115d30 con 0x563ea51ffdd0 2024-04-02T13:56:21.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.567+0000 7f1feda18740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563ea5202d30 con 0x563ea51f68d0 2024-04-02T13:56:21.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.567+0000 7f1feda18740 10 client.0 _ll_register_callbacks cb 0x563ea5115720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:21.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.567+0000 7f1fe3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 0x563ea51fc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.567+0000 7f1fe3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 0x563ea51fc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35318/0 (socket says 172.21.15.52:35318) 2024-04-02T13:56:21.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.567+0000 7f1fe3fff640 1 -- 172.21.15.52:0/2928483676 learned_addr learned my addr 172.21.15.52:0/2928483676 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:21.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.567+0000 7f1fe8bcd640 1 --2- 172.21.15.52:0/2928483676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ea51f68d0 0x563ea51ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.567+0000 7f1fe93ce640 1 --2- 172.21.15.52:0/2928483676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563ea51ffdd0 0x563ea51fceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.568+0000 7f1fe8bcd640 1 -- 172.21.15.52:0/2928483676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563ea51ffdd0 msgr2=0x563ea51fceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:21.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.568+0000 7f1fe8bcd640 1 --2- 172.21.15.52:0/2928483676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563ea51ffdd0 0x563ea51fceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.568+0000 7f1fe8bcd640 1 -- 172.21.15.52:0/2928483676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 msgr2=0x563ea51fc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:21.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.568+0000 7f1fe8bcd640 1 --2- 172.21.15.52:0/2928483676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 0x563ea51fc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.568+0000 7f1fe8bcd640 1 -- 172.21.15.52:0/2928483676 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ea4e6e3f0 con 0x563ea51f68d0 2024-04-02T13:56:21.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.568+0000 7f1fe3fff640 1 --2- 172.21.15.52:0/2928483676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 0x563ea51fc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:21.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.568+0000 7f1fe93ce640 1 --2- 172.21.15.52:0/2928483676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563ea51ffdd0 0x563ea51fceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:21.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.568+0000 7f1fe8bcd640 1 --2- 172.21.15.52:0/2928483676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ea51f68d0 0x563ea51ff5e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc0096c0 tx=0x7f1fcc02c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adb5663489bf280c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:21.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.568+0000 7f1fc3fff640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-02T13:56:21.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.568+0000 7f1fc3fff640 1 -- 172.21.15.52:0/2928483676 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc04e060 con 0x563ea51f68d0 2024-04-02T13:56:21.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.568+0000 7f1fc3fff640 1 -- 172.21.15.52:0/2928483676 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f1fcc004f30 con 0x563ea51f68d0 2024-04-02T13:56:21.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.568+0000 7f1feda18740 1 -- 172.21.15.52:0/2928483676 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fcc004a70 con 0x563ea51f68d0 2024-04-02T13:56:21.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.569+0000 7f1fc3fff640 1 -- 172.21.15.52:0/2928483676 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1fcc055050 con 0x563ea51f68d0 2024-04-02T13:56:21.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.569+0000 7f1feda18740 10 client.25909 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:21.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.569+0000 7f1feda18740 1 -- 172.21.15.52:0/2928483676 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563ea4e2e0f0 con 0x563ea51f68d0 2024-04-02T13:56:21.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.569+0000 7f1fc3fff640 1 -- 172.21.15.52:0/2928483676 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fcc004a70 con 0x563ea51f68d0 2024-04-02T13:56:21.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.569+0000 7f1fc3fff640 1 -- 172.21.15.52:0/2928483676 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f1fcc003b60 con 0x563ea51f68d0 2024-04-02T13:56:21.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.569+0000 7f1feda18740 10 client.25909 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:21.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.569+0000 7f1feda18740 10 client.25909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:21.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.569+0000 7f1feda18740 20 client.25909 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:21.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.569+0000 7f1feda18740 1 --2- 172.21.15.52:0/2928483676 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x563ea520fb60 0x563ea5211f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.569+0000 7f1feda18740 4 client.25909 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:21.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.569+0000 7f1feda18740 1 -- 172.21.15.52:0/2928483676 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ea5167d70 con 0x563ea520fb60 2024-04-02T13:56:21.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.569+0000 7f1fe3fff640 1 --2- 172.21.15.52:0/2928483676 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x563ea520fb60 0x563ea5211f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.570+0000 7f1fc3fff640 10 client.25909 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:21.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.570+0000 7f1fe3fff640 1 --2- 172.21.15.52:0/2928483676 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x563ea520fb60 0x563ea5211f50 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba09ff61649582b4 server_cookie=573b68509d8eb8d8 in_seq=0 out_seq=0 2024-04-02T13:56:21.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.571+0000 7f1fc3fff640 1 -- 172.21.15.52:0/2928483676 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563ea5167d70 con 0x563ea520fb60 2024-04-02T13:56:21.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.571+0000 7f1fc3fff640 10 client.25909 handle_command_reply: tid=0 2024-04-02T13:56:21.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.571+0000 7f1feda18740 1 client.25909 shutdown 2024-04-02T13:56:21.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.571+0000 7f1feda18740 1 -- 172.21.15.52:0/2928483676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ea51f68d0 msgr2=0x563ea51ff5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:21.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.571+0000 7f1feda18740 1 --2- 172.21.15.52:0/2928483676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ea51f68d0 0x563ea51ff5e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc0096c0 tx=0x7f1fcc02c330 comp rx=0 tx=0).stop 2024-04-02T13:56:21.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.571+0000 7f1feda18740 1 -- 172.21.15.52:0/2928483676 shutdown_connections 2024-04-02T13:56:21.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.571+0000 7f1feda18740 1 --2- 172.21.15.52:0/2928483676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563ea51ffdd0 0x563ea51fceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.571+0000 7f1feda18740 1 --2- 172.21.15.52:0/2928483676 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x563ea520fb60 0x563ea5211f50 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.571+0000 7f1feda18740 1 --2- 172.21.15.52:0/2928483676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 0x563ea51fc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.571+0000 7f1feda18740 1 --2- 172.21.15.52:0/2928483676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ea51f68d0 0x563ea51ff5e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.571+0000 7f1feda18740 1 -- 172.21.15.52:0/2928483676 >> 172.21.15.52:0/2928483676 conn(0x563ea51f1000 msgr2=0x563ea51f2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:21.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.571+0000 7f1fc3fff640 0 client.25909 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:21.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.571+0000 7f1feda18740 1 -- 172.21.15.52:0/2928483676 shutdown_connections 2024-04-02T13:56:21.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.571+0000 7f1feda18740 1 -- 172.21.15.52:0/2928483676 wait complete. 2024-04-02T13:56:21.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.571+0000 7f1feda18740 20 client.25909 trim_cache size 0 max 16384 2024-04-02T13:56:21.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:21 smithi052 ceph-mon[32104]: pgmap v88: 129 pgs: 129 active+clean; 71 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 372 KiB/s wr, 58 op/s 2024-04-02T13:56:21.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.591+0000 7f1feda18740 1 Processor -- start 2024-04-02T13:56:21.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.591+0000 7f1feda18740 1 -- start start 2024-04-02T13:56:21.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.591+0000 7f1feda18740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563ea51f68d0 0x563ea520ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.591+0000 7f1feda18740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 0x563ea51f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.591+0000 7f1feda18740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ea51ffdd0 0x563ea51f00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.591+0000 7f1feda18740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe41b1050 con 0x563ea51f71f0 2024-04-02T13:56:21.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.591+0000 7f1feda18740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe41b2bc0 con 0x563ea51f68d0 2024-04-02T13:56:21.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.591+0000 7f1feda18740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe41b1720 con 0x563ea51ffdd0 2024-04-02T13:56:21.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.591+0000 7f1fe3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 0x563ea51f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.591+0000 7f1fe3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 0x563ea51f7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35324/0 (socket says 172.21.15.52:35324) 2024-04-02T13:56:21.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.591+0000 7f1fe3fff640 1 -- 172.21.15.52:0/3225590707 learned_addr learned my addr 172.21.15.52:0/3225590707 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:21.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.591+0000 7f1fe8bcd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563ea51f68d0 0x563ea520ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.591+0000 7f1fe93ce640 1 --2- 172.21.15.52:0/3225590707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ea51ffdd0 0x563ea51f00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.591+0000 7f1fe3fff640 1 -- 172.21.15.52:0/3225590707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ea51ffdd0 msgr2=0x563ea51f00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:21.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.591+0000 7f1fe3fff640 1 --2- 172.21.15.52:0/3225590707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ea51ffdd0 0x563ea51f00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.591+0000 7f1fe3fff640 1 -- 172.21.15.52:0/3225590707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563ea51f68d0 msgr2=0x563ea520ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:21.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.591+0000 7f1fe3fff640 1 --2- 172.21.15.52:0/3225590707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563ea51f68d0 0x563ea520ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.592+0000 7f1fe3fff640 1 -- 172.21.15.52:0/3225590707 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ea51f0e30 con 0x563ea51f71f0 2024-04-02T13:56:21.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.592+0000 7f1fe93ce640 1 --2- 172.21.15.52:0/3225590707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ea51ffdd0 0x563ea51f00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:21.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.592+0000 7f1fe3fff640 1 --2- 172.21.15.52:0/3225590707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 0x563ea51f7fa0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x563ea51fc1d0 tx=0x7f1fd4021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a691c996c073acd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:21.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.592+0000 7f1fda7fc640 1 -- 172.21.15.52:0/3225590707 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd40097a0 con 0x563ea51f71f0 2024-04-02T13:56:21.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.592+0000 7f1fda7fc640 1 -- 172.21.15.52:0/3225590707 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fd402c930 con 0x563ea51f71f0 2024-04-02T13:56:21.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.592+0000 7f1fda7fc640 1 -- 172.21.15.52:0/3225590707 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4005610 con 0x563ea51f71f0 2024-04-02T13:56:21.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.593+0000 7f1feda18740 1 -- 172.21.15.52:0/3225590707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 msgr2=0x563ea51f7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:21.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.593+0000 7f1feda18740 1 --2- 172.21.15.52:0/3225590707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 0x563ea51f7fa0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x563ea51fc1d0 tx=0x7f1fd4021000 comp rx=0 tx=0).stop 2024-04-02T13:56:21.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.593+0000 7f1feda18740 1 -- 172.21.15.52:0/3225590707 shutdown_connections 2024-04-02T13:56:21.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.593+0000 7f1feda18740 1 --2- 172.21.15.52:0/3225590707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ea51ffdd0 0x563ea51f00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.593+0000 7f1feda18740 1 --2- 172.21.15.52:0/3225590707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 0x563ea51f7fa0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.593+0000 7f1feda18740 1 --2- 172.21.15.52:0/3225590707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563ea51f68d0 0x563ea520ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.593+0000 7f1feda18740 1 -- 172.21.15.52:0/3225590707 >> 172.21.15.52:0/3225590707 conn(0x563ea51f1000 msgr2=0x563ea51ef9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:21.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.593+0000 7f1feda18740 1 -- 172.21.15.52:0/3225590707 shutdown_connections 2024-04-02T13:56:21.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.593+0000 7f1feda18740 1 -- 172.21.15.52:0/3225590707 wait complete. 2024-04-02T13:56:21.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.593+0000 7f1feda18740 1 -- start start 2024-04-02T13:56:21.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.593+0000 7f1feda18740 1 Processor -- start 2024-04-02T13:56:21.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.594+0000 7f1feda18740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563ea51f68d0 0x563ea52080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.594+0000 7f1feda18740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 0x563ea5203540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.594+0000 7f1feda18740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ea51ffdd0 0x563ea5203a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.594+0000 7f1feda18740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe41b24c0 con 0x563ea51f71f0 2024-04-02T13:56:21.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.594+0000 7f1feda18740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe41b1df0 con 0x563ea51f68d0 2024-04-02T13:56:21.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.594+0000 7f1feda18740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe41af030 con 0x563ea51ffdd0 2024-04-02T13:56:21.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.594+0000 7f1feda18740 10 client.0 _ll_register_callbacks cb 0x7f1fcc02cbf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:21.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.594+0000 7f1fe3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 0x563ea5203540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.594+0000 7f1fe8bcd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563ea51f68d0 0x563ea52080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.594+0000 7f1fe93ce640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ea51ffdd0 0x563ea5203a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.594+0000 7f1fe3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 0x563ea5203540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35340/0 (socket says 172.21.15.52:35340) 2024-04-02T13:56:21.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.594+0000 7f1fe8bcd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563ea51f68d0 0x563ea52080a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:36656/0 (socket says 172.21.15.52:36656) 2024-04-02T13:56:21.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.594+0000 7f1fe3fff640 1 -- 172.21.15.52:0/2406000899 learned_addr learned my addr 172.21.15.52:0/2406000899 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:21.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.594+0000 7f1fe3fff640 1 -- 172.21.15.52:0/2406000899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ea51ffdd0 msgr2=0x563ea5203a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:21.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.594+0000 7f1fe3fff640 1 --2- 172.21.15.52:0/2406000899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ea51ffdd0 0x563ea5203a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.594+0000 7f1fe3fff640 1 -- 172.21.15.52:0/2406000899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563ea51f68d0 msgr2=0x563ea52080a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:21.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.594+0000 7f1fe3fff640 1 --2- 172.21.15.52:0/2406000899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563ea51f68d0 0x563ea52080a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.594+0000 7f1fe3fff640 1 -- 172.21.15.52:0/2406000899 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ea5203fc0 con 0x563ea51f71f0 2024-04-02T13:56:21.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.594+0000 7f1fe93ce640 1 --2- 172.21.15.52:0/2406000899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ea51ffdd0 0x563ea5203a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:21.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.594+0000 7f1fe8bcd640 1 --2- 172.21.15.52:0/2406000899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563ea51f68d0 0x563ea52080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:21.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.594+0000 7f1fe3fff640 1 --2- 172.21.15.52:0/2406000899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 0x563ea5203540 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4004700 tx=0x7f1fd4005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58fd415f51d4760 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:21.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.594+0000 7f1fc3fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:21.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.595+0000 7f1fc3fff640 1 -- 172.21.15.52:0/2406000899 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd40097a0 con 0x563ea51f71f0 2024-04-02T13:56:21.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.595+0000 7f1fc3fff640 1 -- 172.21.15.52:0/2406000899 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f1fd401a050 con 0x563ea51f71f0 2024-04-02T13:56:21.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.595+0000 7f1feda18740 1 -- 172.21.15.52:0/2406000899 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fd402c930 con 0x563ea51f71f0 2024-04-02T13:56:21.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.595+0000 7f1fc3fff640 1 -- 172.21.15.52:0/2406000899 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1fd4044050 con 0x563ea51f71f0 2024-04-02T13:56:21.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.595+0000 7f1feda18740 10 client.16542 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:21.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.595+0000 7f1feda18740 1 -- 172.21.15.52:0/2406000899 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563ea5275b20 con 0x563ea51f71f0 2024-04-02T13:56:21.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.595+0000 7f1fc3fff640 1 -- 172.21.15.52:0/2406000899 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fd402c930 con 0x563ea51f71f0 2024-04-02T13:56:21.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.595+0000 7f1fc3fff640 1 -- 172.21.15.52:0/2406000899 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f1fd4003bc0 con 0x563ea51f71f0 2024-04-02T13:56:21.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.595+0000 7f1feda18740 10 client.16542 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:21.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.595+0000 7f1feda18740 10 client.16542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:21.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.595+0000 7f1feda18740 20 client.16542 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:21.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.595+0000 7f1feda18740 1 --2- 172.21.15.52:0/2406000899 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x563ea5276150 0x563ea5278570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:21.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.595+0000 7f1feda18740 4 client.16542 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:56:21.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.595+0000 7f1feda18740 1 -- 172.21.15.52:0/2406000899 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563ea51620c0 con 0x563ea5276150 2024-04-02T13:56:21.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.596+0000 7f1fe8bcd640 1 --2- 172.21.15.52:0/2406000899 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x563ea5276150 0x563ea5278570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:21.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.596+0000 7f1fc3fff640 10 client.16542 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:21.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.596+0000 7f1fe8bcd640 1 --2- 172.21.15.52:0/2406000899 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x563ea5276150 0x563ea5278570 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcdf87767af19aae server_cookie=ab4db2aa46e666b in_seq=0 out_seq=0 2024-04-02T13:56:21.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.596+0000 7f1fc3fff640 1 -- 172.21.15.52:0/2406000899 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x563ea51620c0 con 0x563ea5276150 2024-04-02T13:56:21.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.596+0000 7f1fc3fff640 10 client.16542 handle_command_reply: tid=0 2024-04-02T13:56:21.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.596+0000 7f1feda18740 1 client.16542 shutdown 2024-04-02T13:56:21.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.597+0000 7f1feda18740 1 -- 172.21.15.52:0/2406000899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 msgr2=0x563ea5203540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:21.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.597+0000 7f1feda18740 1 --2- 172.21.15.52:0/2406000899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 0x563ea5203540 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4004700 tx=0x7f1fd4005fb0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.597+0000 7f1feda18740 1 -- 172.21.15.52:0/2406000899 shutdown_connections 2024-04-02T13:56:21.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.597+0000 7f1feda18740 1 --2- 172.21.15.52:0/2406000899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ea51ffdd0 0x563ea5203a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.597+0000 7f1feda18740 1 --2- 172.21.15.52:0/2406000899 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x563ea5276150 0x563ea5278570 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.597+0000 7f1feda18740 1 --2- 172.21.15.52:0/2406000899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea51f71f0 0x563ea5203540 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.597+0000 7f1feda18740 1 --2- 172.21.15.52:0/2406000899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563ea51f68d0 0x563ea52080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.597+0000 7f1feda18740 1 -- 172.21.15.52:0/2406000899 >> 172.21.15.52:0/2406000899 conn(0x563ea51f1000 msgr2=0x563ea51f0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:21.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.597+0000 7f1fc3fff640 0 client.16542 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:21.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.597+0000 7f1feda18740 1 -- 172.21.15.52:0/2406000899 shutdown_connections 2024-04-02T13:56:21.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.597+0000 7f1feda18740 1 -- 172.21.15.52:0/2406000899 wait complete. 2024-04-02T13:56:21.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.597+0000 7f1feda18740 20 client.16542 trim_cache size 0 max 16384 2024-04-02T13:56:21.611 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:56:21.611 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "scrub active (2256 inodes in the stack)", 2024-04-02T13:56:21.611 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-04-02T13:56:21.612 INFO:teuthology.orchestra.run.smithi052.stdout: "97d404a0-2d0c-4a9c-a196-1d04ff8ce889": { 2024-04-02T13:56:21.612 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-04-02T13:56:21.612 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "97d404a0-2d0c-4a9c-a196-1d04ff8ce889", 2024-04-02T13:56:21.612 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-04-02T13:56:21.612 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:56:21.612 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:56:21.612 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:56:21.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.597+0000 7f1feae31640 1 -- 172.21.15.52:0/3532517548 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1fd00a3d20 msgr2=0x7f1fd00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:21.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.597+0000 7f1feae31640 1 --2- 172.21.15.52:0/3532517548 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1fd00a3d20 0x7f1fd00a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1fe41049e0 tx=0x7f1fd4009b60 comp rx=0 tx=0).stop 2024-04-02T13:56:21.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.598+0000 7f1feae31640 1 -- 172.21.15.52:0/3532517548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1fe4106980 msgr2=0x7f1fe41039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:21.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.598+0000 7f1feae31640 1 --2- 172.21.15.52:0/3532517548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1fe4106980 0x7f1fe41039d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00eb10 tx=0x7f1fdc00c2d0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.598+0000 7f1feae31640 1 -- 172.21.15.52:0/3532517548 shutdown_connections 2024-04-02T13:56:21.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.598+0000 7f1feae31640 1 --2- 172.21.15.52:0/3532517548 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1fd00a3d20 0x7f1fd00a61e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.598+0000 7f1feae31640 1 --2- 172.21.15.52:0/3532517548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1fe4106980 0x7f1fe41039d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.598+0000 7f1feae31640 1 --2- 172.21.15.52:0/3532517548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fe4101330 0x7f1fe4103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.598+0000 7f1feae31640 1 --2- 172.21.15.52:0/3532517548 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe4100970 0x7f1fe4102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:21.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.598+0000 7f1feae31640 1 -- 172.21.15.52:0/3532517548 >> 172.21.15.52:0/3532517548 conn(0x7f1fe4077b00 msgr2=0x7f1fe4069000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:21.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.598+0000 7f1feae31640 1 -- 172.21.15.52:0/3532517548 shutdown_connections 2024-04-02T13:56:21.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:21.598+0000 7f1feae31640 1 -- 172.21.15.52:0/3532517548 wait complete. 2024-04-02T13:56:21.615 INFO:tasks.cephfs.filesystem:scrub status for tag:97d404a0-2d0c-4a9c-a196-1d04ff8ce889 - {'path': '/', 'tag': '97d404a0-2d0c-4a9c-a196-1d04ff8ce889', 'options': 'recursive,force'} 2024-04-02T13:56:21.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:21 smithi189 ceph-mon[36683]: pgmap v88: 129 pgs: 129 active+clean; 71 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 372 KiB/s wr, 58 op/s 2024-04-02T13:56:21.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:21 smithi088 ceph-mon[36528]: pgmap v88: 129 pgs: 129 active+clean; 71 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 372 KiB/s wr, 58 op/s 2024-04-02T13:56:22.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:22 smithi189 ceph-mon[36683]: scrub queued for path: / 2024-04-02T13:56:22.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:22 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:22.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:22 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:56:22.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:22 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:56:22.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:22 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:22.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:22 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:56:22.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:22 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:22.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:22 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:22.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:22 smithi052 ceph-mon[32104]: scrub queued for path: / 2024-04-02T13:56:22.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:22 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:22.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:22 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:56:22.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:22 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:56:22.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:22 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:22.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:22 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:56:22.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:22 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:22.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:22 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:22.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:22 smithi088 ceph-mon[36528]: scrub queued for path: / 2024-04-02T13:56:22.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:22 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:22.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:22 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:56:22.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:22 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:56:22.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:22 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:22.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:22 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:56:22.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:22 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:22.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:22 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:23.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:23 smithi189 ceph-mon[36683]: pgmap v89: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 387 KiB/s wr, 63 op/s 2024-04-02T13:56:23.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:23 smithi052 ceph-mon[32104]: pgmap v89: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 387 KiB/s wr, 63 op/s 2024-04-02T13:56:23.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:23 smithi088 ceph-mon[36528]: pgmap v89: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 387 KiB/s wr, 63 op/s 2024-04-02T13:56:25.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:25 smithi189 ceph-mon[36683]: pgmap v90: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 384 KiB/s wr, 59 op/s 2024-04-02T13:56:25.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:25 smithi052 ceph-mon[32104]: pgmap v90: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 384 KiB/s wr, 59 op/s 2024-04-02T13:56:25.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:25 smithi088 ceph-mon[36528]: pgmap v90: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 384 KiB/s wr, 59 op/s 2024-04-02T13:56:27.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:27 smithi189 ceph-mon[36683]: pgmap v91: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 386 KiB/s wr, 64 op/s 2024-04-02T13:56:27.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:27 smithi052 ceph-mon[32104]: pgmap v91: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 386 KiB/s wr, 64 op/s 2024-04-02T13:56:27.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:27 smithi088 ceph-mon[36528]: pgmap v91: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 386 KiB/s wr, 64 op/s 2024-04-02T13:56:29.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:29 smithi189 ceph-mon[36683]: pgmap v92: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-04-02T13:56:29.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:29 smithi052 ceph-mon[32104]: pgmap v92: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-04-02T13:56:29.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:29 smithi088 ceph-mon[36528]: pgmap v92: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-04-02T13:56:30.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:30 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T13:56:30.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:30 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T13:56:30.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:30 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T13:56:31.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:31 smithi189 ceph-mon[36683]: pgmap v93: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-04-02T13:56:31.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:31 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T13:56:31.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:31 smithi052 ceph-mon[32104]: pgmap v93: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-04-02T13:56:31.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:31 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T13:56:31.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:31 smithi088 ceph-mon[36528]: pgmap v93: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-04-02T13:56:31.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:31 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T13:56:33.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:33 smithi189 ceph-mon[36683]: pgmap v94: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-04-02T13:56:33.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:33 smithi052 ceph-mon[32104]: pgmap v94: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-04-02T13:56:33.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:33 smithi088 ceph-mon[36528]: pgmap v94: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-04-02T13:56:35.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:35 smithi189 ceph-mon[36683]: pgmap v95: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-02T13:56:35.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:35 smithi052 ceph-mon[32104]: pgmap v95: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-02T13:56:35.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:35 smithi088 ceph-mon[36528]: pgmap v95: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-02T13:56:37.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:37 smithi052 ceph-mon[32104]: pgmap v96: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.3 KiB/s wr, 14 op/s 2024-04-02T13:56:37.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:37 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:56:37.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:37 smithi088 ceph-mon[36528]: pgmap v96: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.3 KiB/s wr, 14 op/s 2024-04-02T13:56:37.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:37 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:56:38.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:37 smithi189 ceph-mon[36683]: pgmap v96: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.3 KiB/s wr, 14 op/s 2024-04-02T13:56:38.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:37 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:56:38.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:38 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:56:38.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:38 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:56:38.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:38 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:56:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:38 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:56:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:38 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:56:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:38 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:56:39.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:38 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:56:39.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:38 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:56:39.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:38 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:56:39.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:39 smithi052 ceph-mon[32104]: pgmap v97: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-04-02T13:56:39.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:39 smithi088 ceph-mon[36528]: pgmap v97: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-04-02T13:56:40.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:39 smithi189 ceph-mon[36683]: pgmap v97: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-04-02T13:56:41.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:41 smithi052 ceph-mon[32104]: pgmap v98: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-04-02T13:56:41.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:41 smithi088 ceph-mon[36528]: pgmap v98: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-04-02T13:56:42.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:41 smithi189 ceph-mon[36683]: pgmap v98: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-04-02T13:56:43.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:43 smithi052 ceph-mon[32104]: pgmap v99: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-04-02T13:56:43.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:43 smithi088 ceph-mon[36528]: pgmap v99: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-04-02T13:56:44.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:43 smithi189 ceph-mon[36683]: pgmap v99: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-04-02T13:56:45.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:45 smithi052 ceph-mon[32104]: pgmap v100: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-04-02T13:56:45.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:45 smithi088 ceph-mon[36528]: pgmap v100: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-04-02T13:56:46.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:45 smithi189 ceph-mon[36683]: pgmap v100: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-04-02T13:56:47.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:47 smithi052 ceph-mon[32104]: pgmap v101: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-04-02T13:56:47.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:47 smithi088 ceph-mon[36528]: pgmap v101: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-04-02T13:56:48.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:47 smithi189 ceph-mon[36683]: pgmap v101: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-04-02T13:56:49.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:49 smithi052 ceph-mon[32104]: pgmap v102: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-02T13:56:49.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:49 smithi088 ceph-mon[36528]: pgmap v102: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-02T13:56:50.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:49 smithi189 ceph-mon[36683]: pgmap v102: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-02T13:56:51.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:56:51.617 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:56:51.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.731+0000 7f530b532640 1 -- 172.21.15.52:0/3390755035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5304101450 msgr2=0x7f5304105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:51.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.731+0000 7f530b532640 1 --2- 172.21.15.52:0/3390755035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5304101450 0x7f5304105910 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f52f8009930 tx=0x7f52f802bcd0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.731+0000 7f530b532640 1 -- 172.21.15.52:0/3390755035 shutdown_connections 2024-04-02T13:56:51.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.731+0000 7f530b532640 1 --2- 172.21.15.52:0/3390755035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5304105f30 0x7f53041063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.731+0000 7f530b532640 1 --2- 172.21.15.52:0/3390755035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5304101450 0x7f5304105910 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.731+0000 7f530b532640 1 --2- 172.21.15.52:0/3390755035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5304100b30 0x7f5304100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.731+0000 7f530b532640 1 -- 172.21.15.52:0/3390755035 >> 172.21.15.52:0/3390755035 conn(0x7f5304076d80 msgr2=0x7f5304077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:51.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.731+0000 7f530b532640 1 -- 172.21.15.52:0/3390755035 shutdown_connections 2024-04-02T13:56:51.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.731+0000 7f530b532640 1 -- 172.21.15.52:0/3390755035 wait complete. 2024-04-02T13:56:51.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.732+0000 7f530b532640 1 Processor -- start 2024-04-02T13:56:51.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.732+0000 7f530b532640 1 -- start start 2024-04-02T13:56:51.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.732+0000 7f530b532640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5304100b30 0x7f53041a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:51.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.732+0000 7f530b532640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5304101450 0x7f53041a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:51.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.732+0000 7f530b532640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5304105f30 0x7f530419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:51.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.732+0000 7f530b532640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5304118480 con 0x7f5304101450 2024-04-02T13:56:51.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.732+0000 7f530b532640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5304118300 con 0x7f5304105f30 2024-04-02T13:56:51.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.732+0000 7f530b532640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5304118600 con 0x7f5304100b30 2024-04-02T13:56:51.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.732+0000 7f5308acd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5304101450 0x7f53041a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:51.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.732+0000 7f5308acd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5304101450 0x7f53041a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51736/0 (socket says 172.21.15.52:51736) 2024-04-02T13:56:51.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.732+0000 7f5308acd640 1 -- 172.21.15.52:0/2993479737 learned_addr learned my addr 172.21.15.52:0/2993479737 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:51.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.732+0000 7f5309acf640 1 --2- 172.21.15.52:0/2993479737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5304105f30 0x7f530419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:51.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.732+0000 7f53092ce640 1 --2- 172.21.15.52:0/2993479737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5304100b30 0x7f53041a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:51.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.732+0000 7f5308acd640 1 -- 172.21.15.52:0/2993479737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5304100b30 msgr2=0x7f53041a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:51.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.732+0000 7f5308acd640 1 --2- 172.21.15.52:0/2993479737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5304100b30 0x7f53041a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.732+0000 7f5308acd640 1 -- 172.21.15.52:0/2993479737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5304105f30 msgr2=0x7f530419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:51.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.732+0000 7f5308acd640 1 --2- 172.21.15.52:0/2993479737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5304105f30 0x7f530419e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.732+0000 7f5308acd640 1 -- 172.21.15.52:0/2993479737 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f530419eff0 con 0x7f5304101450 2024-04-02T13:56:51.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.732+0000 7f5308acd640 1 --2- 172.21.15.52:0/2993479737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5304101450 0x7f53041a4bf0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f52f8009230 tx=0x7f52f802cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:51.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.733+0000 7f53027fc640 1 -- 172.21.15.52:0/2993479737 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f8041070 con 0x7f5304101450 2024-04-02T13:56:51.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.733+0000 7f530b532640 1 -- 172.21.15.52:0/2993479737 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f530419f280 con 0x7f5304101450 2024-04-02T13:56:51.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.733+0000 7f53027fc640 1 -- 172.21.15.52:0/2993479737 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52f80054e0 con 0x7f5304101450 2024-04-02T13:56:51.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.733+0000 7f53027fc640 1 -- 172.21.15.52:0/2993479737 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f8034440 con 0x7f5304101450 2024-04-02T13:56:51.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.733+0000 7f530b532640 1 -- 172.21.15.52:0/2993479737 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53041ab500 con 0x7f5304101450 2024-04-02T13:56:51.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.735+0000 7f53027fc640 1 -- 172.21.15.52:0/2993479737 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f52f8044070 con 0x7f5304101450 2024-04-02T13:56:51.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.735+0000 7f530e1f6740 1 Processor -- start 2024-04-02T13:56:51.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.735+0000 7f530e1f6740 1 -- start start 2024-04-02T13:56:51.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.736+0000 7f530e1f6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b496798d0 0x556b49679cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:51.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.736+0000 7f530e1f6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b4967a1f0 0x556b49682890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:51.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.736+0000 7f530e1f6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 0x556b496851c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:51.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.736+0000 7f530e1f6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556b495dd830 con 0x556b49682dd0 2024-04-02T13:56:51.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.736+0000 7f530e1f6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556b49585820 con 0x556b4967a1f0 2024-04-02T13:56:51.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.736+0000 7f530e1f6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556b495d97e0 con 0x556b496798d0 2024-04-02T13:56:51.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.736+0000 7f53027fc640 1 --2- 172.21.15.52:0/2993479737 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f52ec0a3d20 0x7f52ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:51.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.736+0000 7f5308acd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 0x556b496851c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:51.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.736+0000 7f5308acd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 0x556b496851c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51750/0 (socket says 172.21.15.52:51750) 2024-04-02T13:56:51.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.736+0000 7f5308acd640 1 -- 172.21.15.52:0/2355560989 learned_addr learned my addr 172.21.15.52:0/2355560989 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:51.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.736+0000 7f5309acf640 1 --2- 172.21.15.52:0/2993479737 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f52ec0a3d20 0x7f52ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:51.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.736+0000 7f5309acf640 1 --2- 172.21.15.52:0/2355560989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b496798d0 0x556b49679cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:51.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.736+0000 7f53027fc640 1 -- 172.21.15.52:0/2993479737 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f52f80f2680 con 0x7f5304101450 2024-04-02T13:56:51.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.736+0000 7f53092ce640 1 --2- 172.21.15.52:0/2355560989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b4967a1f0 0x556b49682890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:51.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.736+0000 7f5308acd640 1 -- 172.21.15.52:0/2355560989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b496798d0 msgr2=0x556b49679cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:51.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.736+0000 7f5308acd640 1 --2- 172.21.15.52:0/2355560989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b496798d0 0x556b49679cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.736+0000 7f5308acd640 1 -- 172.21.15.52:0/2355560989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b4967a1f0 msgr2=0x556b49682890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:51.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.736+0000 7f5308acd640 1 --2- 172.21.15.52:0/2355560989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b4967a1f0 0x556b49682890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.736+0000 7f5308acd640 1 -- 172.21.15.52:0/2355560989 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b493bb7d0 con 0x556b49682dd0 2024-04-02T13:56:51.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.736+0000 7f5309acf640 1 --2- 172.21.15.52:0/2993479737 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f52ec0a3d20 0x7f52ec0a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f52fc005d80 tx=0x7f52fc005cf0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:51.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.736+0000 7f5308acd640 1 --2- 172.21.15.52:0/2355560989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 0x556b496851c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f52f804c000 tx=0x7f52f8009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcfcf301d17bb7bf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:51.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.737+0000 7f52eb7fe640 1 -- 172.21.15.52:0/2355560989 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f804b070 con 0x556b49682dd0 2024-04-02T13:56:51.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.737+0000 7f52eb7fe640 1 -- 172.21.15.52:0/2355560989 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52f8005690 con 0x556b49682dd0 2024-04-02T13:56:51.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.737+0000 7f52eb7fe640 1 -- 172.21.15.52:0/2355560989 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f80be6e0 con 0x556b49682dd0 2024-04-02T13:56:51.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.737+0000 7f530e1f6740 1 -- 172.21.15.52:0/2355560989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 msgr2=0x556b496851c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:51.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.737+0000 7f530e1f6740 1 --2- 172.21.15.52:0/2355560989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 0x556b496851c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f52f804c000 tx=0x7f52f8009680 comp rx=0 tx=0).stop 2024-04-02T13:56:51.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.737+0000 7f530e1f6740 1 -- 172.21.15.52:0/2355560989 shutdown_connections 2024-04-02T13:56:51.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.737+0000 7f530e1f6740 1 --2- 172.21.15.52:0/2355560989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 0x556b496851c0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.737+0000 7f530e1f6740 1 --2- 172.21.15.52:0/2355560989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b4967a1f0 0x556b49682890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.737+0000 7f530e1f6740 1 --2- 172.21.15.52:0/2355560989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b496798d0 0x556b49679cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.737+0000 7f530e1f6740 1 -- 172.21.15.52:0/2355560989 >> 172.21.15.52:0/2355560989 conn(0x556b49674000 msgr2=0x556b496743e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:51.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.737+0000 7f530e1f6740 1 -- 172.21.15.52:0/2355560989 shutdown_connections 2024-04-02T13:56:51.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.737+0000 7f530e1f6740 1 -- 172.21.15.52:0/2355560989 wait complete. 2024-04-02T13:56:51.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.738+0000 7f530e1f6740 1 -- start start 2024-04-02T13:56:51.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.738+0000 7f530e1f6740 1 Processor -- start 2024-04-02T13:56:51.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.738+0000 7f530e1f6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b496798d0 0x556b496825e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:51.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.738+0000 7f530e1f6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b4967a1f0 0x556b4967f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:51.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.738+0000 7f530e1f6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 0x556b4967feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:51.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.738+0000 7f530e1f6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556b49685980 con 0x556b49682dd0 2024-04-02T13:56:51.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.738+0000 7f530e1f6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556b49598d30 con 0x556b496798d0 2024-04-02T13:56:51.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.738+0000 7f530e1f6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556b49685b00 con 0x556b4967a1f0 2024-04-02T13:56:51.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.738+0000 7f530e1f6740 10 client.0 _ll_register_callbacks cb 0x556b49598720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:51.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.738+0000 7f5309acf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 0x556b4967feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:51.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.738+0000 7f5309acf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 0x556b4967feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51764/0 (socket says 172.21.15.52:51764) 2024-04-02T13:56:51.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.738+0000 7f5309acf640 1 -- 172.21.15.52:0/3767027481 learned_addr learned my addr 172.21.15.52:0/3767027481 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:51.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.739+0000 7f5308acd640 1 --2- 172.21.15.52:0/3767027481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b4967a1f0 0x556b4967f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:51.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.739+0000 7f53092ce640 1 --2- 172.21.15.52:0/3767027481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b496798d0 0x556b496825e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:51.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.739+0000 7f5309acf640 1 -- 172.21.15.52:0/3767027481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b4967a1f0 msgr2=0x556b4967f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:51.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.739+0000 7f5309acf640 1 --2- 172.21.15.52:0/3767027481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b4967a1f0 0x556b4967f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.739+0000 7f5309acf640 1 -- 172.21.15.52:0/3767027481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b496798d0 msgr2=0x556b496825e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:51.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.739+0000 7f5309acf640 1 --2- 172.21.15.52:0/3767027481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b496798d0 0x556b496825e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.739+0000 7f5309acf640 1 -- 172.21.15.52:0/3767027481 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b492f13f0 con 0x556b49682dd0 2024-04-02T13:56:51.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.739+0000 7f5309acf640 1 --2- 172.21.15.52:0/3767027481 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 0x556b4967feb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f52fc00b4e0 tx=0x7f52fc01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce8b40a4f67c69e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:51.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.739+0000 7f52e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:51.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.739+0000 7f52e8ff9640 1 -- 172.21.15.52:0/3767027481 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc00fe60 con 0x556b49682dd0 2024-04-02T13:56:51.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.739+0000 7f530e1f6740 1 -- 172.21.15.52:0/3767027481 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556b495ead70 con 0x556b49682dd0 2024-04-02T13:56:51.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.739+0000 7f52e8ff9640 1 -- 172.21.15.52:0/3767027481 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f52fc005020 con 0x556b49682dd0 2024-04-02T13:56:51.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.739+0000 7f52e8ff9640 1 -- 172.21.15.52:0/3767027481 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f52fc033050 con 0x556b49682dd0 2024-04-02T13:56:51.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.739+0000 7f530e1f6740 10 client.16554 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:51.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.739+0000 7f530e1f6740 1 -- 172.21.15.52:0/3767027481 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b492b10f0 con 0x556b49682dd0 2024-04-02T13:56:51.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.739+0000 7f52e8ff9640 1 -- 172.21.15.52:0/3767027481 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556b495ead70 con 0x556b49682dd0 2024-04-02T13:56:51.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.740+0000 7f52e8ff9640 1 -- 172.21.15.52:0/3767027481 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f52fc01f070 con 0x556b49682dd0 2024-04-02T13:56:51.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.740+0000 7f530e1f6740 10 client.16554 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:51.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.740+0000 7f530e1f6740 10 client.16554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:51.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.740+0000 7f530e1f6740 20 client.16554 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:51.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.740+0000 7f530e1f6740 1 --2- 172.21.15.52:0/3767027481 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556b49692a10 0x556b49694eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:51.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.740+0000 7f530e1f6740 4 client.16554 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:51.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.740+0000 7f530e1f6740 1 -- 172.21.15.52:0/3767027481 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b495e50c0 con 0x556b49692a10 2024-04-02T13:56:51.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.740+0000 7f53092ce640 1 --2- 172.21.15.52:0/3767027481 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556b49692a10 0x556b49694eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:51.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.740+0000 7f52e8ff9640 10 client.16554 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:51.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.740+0000 7f53092ce640 1 --2- 172.21.15.52:0/3767027481 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556b49692a10 0x556b49694eb0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78bd32e8b7bfb672 server_cookie=4a91bcde21c7855 in_seq=0 out_seq=0 2024-04-02T13:56:51.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.741+0000 7f52e8ff9640 1 -- 172.21.15.52:0/3767027481 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556b495e50c0 con 0x556b49692a10 2024-04-02T13:56:51.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.741+0000 7f52e8ff9640 10 client.16554 handle_command_reply: tid=0 2024-04-02T13:56:51.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.741+0000 7f530e1f6740 1 client.16554 shutdown 2024-04-02T13:56:51.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.741+0000 7f530e1f6740 1 -- 172.21.15.52:0/3767027481 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 msgr2=0x556b4967feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:51.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.741+0000 7f530e1f6740 1 --2- 172.21.15.52:0/3767027481 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 0x556b4967feb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f52fc00b4e0 tx=0x7f52fc01c000 comp rx=0 tx=0).stop 2024-04-02T13:56:51.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.741+0000 7f530e1f6740 1 -- 172.21.15.52:0/3767027481 shutdown_connections 2024-04-02T13:56:51.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.741+0000 7f530e1f6740 1 --2- 172.21.15.52:0/3767027481 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556b49692a10 0x556b49694eb0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.741+0000 7f530e1f6740 1 --2- 172.21.15.52:0/3767027481 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 0x556b4967feb0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.741+0000 7f52e8ff9640 0 client.16554 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:51.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.741+0000 7f530e1f6740 1 --2- 172.21.15.52:0/3767027481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b4967a1f0 0x556b4967f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.741+0000 7f530e1f6740 1 --2- 172.21.15.52:0/3767027481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b496798d0 0x556b496825e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.741+0000 7f530e1f6740 1 -- 172.21.15.52:0/3767027481 >> 172.21.15.52:0/3767027481 conn(0x556b49674000 msgr2=0x556b49675d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:51.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.742+0000 7f530e1f6740 1 -- 172.21.15.52:0/3767027481 shutdown_connections 2024-04-02T13:56:51.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.742+0000 7f530e1f6740 1 -- 172.21.15.52:0/3767027481 wait complete. 2024-04-02T13:56:51.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.742+0000 7f530e1f6740 20 client.16554 trim_cache size 0 max 16384 2024-04-02T13:56:51.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.759+0000 7f530e1f6740 1 Processor -- start 2024-04-02T13:56:51.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.759+0000 7f530e1f6740 1 -- start start 2024-04-02T13:56:51.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.760+0000 7f530e1f6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b496798d0 0x556b4968e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:51.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.760+0000 7f530e1f6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b4967a1f0 0x556b4967a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:51.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.760+0000 7f530e1f6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 0x556b496820a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:51.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.760+0000 7f530e1f6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f53041b54a0 con 0x556b49682dd0 2024-04-02T13:56:51.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.760+0000 7f530e1f6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f53041b7010 con 0x556b4967a1f0 2024-04-02T13:56:51.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.760+0000 7f530e1f6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f53041b5b70 con 0x556b496798d0 2024-04-02T13:56:51.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.760+0000 7f5309acf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 0x556b496820a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:51.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.760+0000 7f5309acf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 0x556b496820a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51778/0 (socket says 172.21.15.52:51778) 2024-04-02T13:56:51.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.760+0000 7f5309acf640 1 -- 172.21.15.52:0/2072144513 learned_addr learned my addr 172.21.15.52:0/2072144513 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:51.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.760+0000 7f53092ce640 1 --2- 172.21.15.52:0/2072144513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b496798d0 0x556b4968e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:51.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.760+0000 7f5309acf640 1 -- 172.21.15.52:0/2072144513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b496798d0 msgr2=0x556b4968e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:51.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.760+0000 7f5308acd640 1 --2- 172.21.15.52:0/2072144513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b4967a1f0 0x556b4967a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:51.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.760+0000 7f5309acf640 1 --2- 172.21.15.52:0/2072144513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b496798d0 0x556b4968e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.760+0000 7f5309acf640 1 -- 172.21.15.52:0/2072144513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b4967a1f0 msgr2=0x556b4967a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:51.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.760+0000 7f5309acf640 1 --2- 172.21.15.52:0/2072144513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b4967a1f0 0x556b4967a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.760+0000 7f5309acf640 1 -- 172.21.15.52:0/2072144513 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b49673e30 con 0x556b49682dd0 2024-04-02T13:56:51.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.760+0000 7f5308acd640 1 --2- 172.21.15.52:0/2072144513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b4967a1f0 0x556b4967a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:56:51.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.760+0000 7f5309acf640 1 --2- 172.21.15.52:0/2072144513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 0x556b496820a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f52fc000c00 tx=0x7f52fc00b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4f0a0f350144e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:51.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.760+0000 7f52eaffd640 1 -- 172.21.15.52:0/2072144513 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc03c070 con 0x556b49682dd0 2024-04-02T13:56:51.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.760+0000 7f52eaffd640 1 -- 172.21.15.52:0/2072144513 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52fc003ce0 con 0x556b49682dd0 2024-04-02T13:56:51.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.760+0000 7f52eaffd640 1 -- 172.21.15.52:0/2072144513 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc02a8f0 con 0x556b49682dd0 2024-04-02T13:56:51.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.761+0000 7f530e1f6740 1 -- 172.21.15.52:0/2072144513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 msgr2=0x556b496820a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:51.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.761+0000 7f530e1f6740 1 --2- 172.21.15.52:0/2072144513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 0x556b496820a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f52fc000c00 tx=0x7f52fc00b2e0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.761+0000 7f530e1f6740 1 -- 172.21.15.52:0/2072144513 shutdown_connections 2024-04-02T13:56:51.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.761+0000 7f530e1f6740 1 --2- 172.21.15.52:0/2072144513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 0x556b496820a0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.761+0000 7f530e1f6740 1 --2- 172.21.15.52:0/2072144513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b4967a1f0 0x556b4967a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.761+0000 7f530e1f6740 1 --2- 172.21.15.52:0/2072144513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b496798d0 0x556b4968e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.761+0000 7f530e1f6740 1 -- 172.21.15.52:0/2072144513 >> 172.21.15.52:0/2072144513 conn(0x556b49674000 msgr2=0x556b49673240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:51.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.761+0000 7f530e1f6740 1 -- 172.21.15.52:0/2072144513 shutdown_connections 2024-04-02T13:56:51.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.761+0000 7f530e1f6740 1 -- 172.21.15.52:0/2072144513 wait complete. 2024-04-02T13:56:51.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.761+0000 7f530e1f6740 1 -- start start 2024-04-02T13:56:51.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.761+0000 7f530e1f6740 1 Processor -- start 2024-04-02T13:56:51.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.762+0000 7f530e1f6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b496798d0 0x556b4968b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:51.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.762+0000 7f530e1f6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b4967a1f0 0x556b4967c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:51.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.762+0000 7f530e1f6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 0x556b4967c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:51.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.762+0000 7f530e1f6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f53041b6910 con 0x556b49682dd0 2024-04-02T13:56:51.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.762+0000 7f530e1f6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f53041b6240 con 0x556b4967a1f0 2024-04-02T13:56:51.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.762+0000 7f530e1f6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f53041b3480 con 0x556b496798d0 2024-04-02T13:56:51.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.762+0000 7f530e1f6740 10 client.0 _ll_register_callbacks cb 0x556b49598720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:51.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.762+0000 7f5309acf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 0x556b4967c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:51.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.762+0000 7f5309acf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 0x556b4967c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51782/0 (socket says 172.21.15.52:51782) 2024-04-02T13:56:51.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.762+0000 7f5309acf640 1 -- 172.21.15.52:0/1721338517 learned_addr learned my addr 172.21.15.52:0/1721338517 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:51.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.762+0000 7f5308acd640 1 --2- 172.21.15.52:0/1721338517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b4967a1f0 0x556b4967c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:51.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.762+0000 7f5309acf640 1 -- 172.21.15.52:0/1721338517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b496798d0 msgr2=0x556b4968b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:51.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.762+0000 7f53092ce640 1 --2- 172.21.15.52:0/1721338517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b496798d0 0x556b4968b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:51.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.762+0000 7f5309acf640 1 --2- 172.21.15.52:0/1721338517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b496798d0 0x556b4968b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.762+0000 7f5309acf640 1 -- 172.21.15.52:0/1721338517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b4967a1f0 msgr2=0x556b4967c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:51.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.762+0000 7f5309acf640 1 --2- 172.21.15.52:0/1721338517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b4967a1f0 0x556b4967c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.762+0000 7f5309acf640 1 -- 172.21.15.52:0/1721338517 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b496851c0 con 0x556b49682dd0 2024-04-02T13:56:51.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.762+0000 7f5308acd640 1 --2- 172.21.15.52:0/1721338517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b4967a1f0 0x556b4967c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:51.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.762+0000 7f53092ce640 1 --2- 172.21.15.52:0/1721338517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b496798d0 0x556b4968b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:56:51.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.762+0000 7f5309acf640 1 --2- 172.21.15.52:0/1721338517 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 0x556b4967c8c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f52fc03a040 tx=0x7f52fc00b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6fd14f6e71f738 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:51.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.762+0000 7f52e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:51.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.763+0000 7f52e8ff9640 1 -- 172.21.15.52:0/1721338517 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc03c070 con 0x556b49682dd0 2024-04-02T13:56:51.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.763+0000 7f52e8ff9640 1 -- 172.21.15.52:0/1721338517 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f52fc02a980 con 0x556b49682dd0 2024-04-02T13:56:51.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.763+0000 7f530e1f6740 1 -- 172.21.15.52:0/1721338517 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52fc003ce0 con 0x556b49682dd0 2024-04-02T13:56:51.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.763+0000 7f52e8ff9640 1 -- 172.21.15.52:0/1721338517 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f52fc049050 con 0x556b49682dd0 2024-04-02T13:56:51.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.763+0000 7f530e1f6740 10 client.16560 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:51.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.763+0000 7f530e1f6740 1 -- 172.21.15.52:0/1721338517 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b496f8e00 con 0x556b49682dd0 2024-04-02T13:56:51.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.763+0000 7f52e8ff9640 1 -- 172.21.15.52:0/1721338517 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52fc003ce0 con 0x556b49682dd0 2024-04-02T13:56:51.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.763+0000 7f52e8ff9640 1 -- 172.21.15.52:0/1721338517 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f52fc036030 con 0x556b49682dd0 2024-04-02T13:56:51.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.763+0000 7f530e1f6740 10 client.16560 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:51.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.763+0000 7f530e1f6740 10 client.16560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:51.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.763+0000 7f530e1f6740 20 client.16560 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:51.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.763+0000 7f530e1f6740 1 --2- 172.21.15.52:0/1721338517 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556b496f9460 0x556b496fb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:51.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.763+0000 7f530e1f6740 4 client.16560 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:56:51.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.763+0000 7f530e1f6740 1 -- 172.21.15.52:0/1721338517 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556b49562560 con 0x556b496f9460 2024-04-02T13:56:51.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.763+0000 7f53092ce640 1 --2- 172.21.15.52:0/1721338517 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556b496f9460 0x556b496fb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:51.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.764+0000 7f52e8ff9640 10 client.16560 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:51.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.764+0000 7f53092ce640 1 --2- 172.21.15.52:0/1721338517 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556b496f9460 0x556b496fb880 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=882a0b55dff6a013 server_cookie=75ce7e85b870e5d2 in_seq=0 out_seq=0 2024-04-02T13:56:51.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.764+0000 7f52e8ff9640 1 -- 172.21.15.52:0/1721338517 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556b49562560 con 0x556b496f9460 2024-04-02T13:56:51.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.764+0000 7f52e8ff9640 10 client.16560 handle_command_reply: tid=0 2024-04-02T13:56:51.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.764+0000 7f530e1f6740 1 client.16560 shutdown 2024-04-02T13:56:51.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.764+0000 7f530e1f6740 1 -- 172.21.15.52:0/1721338517 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 msgr2=0x556b4967c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:51.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.764+0000 7f530e1f6740 1 --2- 172.21.15.52:0/1721338517 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 0x556b4967c8c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f52fc03a040 tx=0x7f52fc00b690 comp rx=0 tx=0).stop 2024-04-02T13:56:51.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.764+0000 7f530e1f6740 1 -- 172.21.15.52:0/1721338517 shutdown_connections 2024-04-02T13:56:51.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.764+0000 7f530e1f6740 1 --2- 172.21.15.52:0/1721338517 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556b496f9460 0x556b496fb880 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.764+0000 7f530e1f6740 1 --2- 172.21.15.52:0/1721338517 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b49682dd0 0x556b4967c8c0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.764+0000 7f530e1f6740 1 --2- 172.21.15.52:0/1721338517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b4967a1f0 0x556b4967c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.764+0000 7f52e8ff9640 0 client.16560 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:51.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.764+0000 7f530e1f6740 1 --2- 172.21.15.52:0/1721338517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b496798d0 0x556b4968b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.764+0000 7f530e1f6740 1 -- 172.21.15.52:0/1721338517 >> 172.21.15.52:0/1721338517 conn(0x556b49674000 msgr2=0x556b496729b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:51.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.764+0000 7f530e1f6740 1 -- 172.21.15.52:0/1721338517 shutdown_connections 2024-04-02T13:56:51.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.764+0000 7f530e1f6740 1 -- 172.21.15.52:0/1721338517 wait complete. 2024-04-02T13:56:51.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.764+0000 7f530e1f6740 20 client.16560 trim_cache size 0 max 16384 2024-04-02T13:56:51.779 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:56:51.779 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-04-02T13:56:51.779 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-04-02T13:56:51.780 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:56:51.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.765+0000 7f530b532640 1 -- 172.21.15.52:0/2993479737 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f52ec0a3d20 msgr2=0x7f52ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:51.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.765+0000 7f530b532640 1 --2- 172.21.15.52:0/2993479737 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f52ec0a3d20 0x7f52ec0a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f52fc005d80 tx=0x7f52fc005cf0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.765+0000 7f530b532640 1 -- 172.21.15.52:0/2993479737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5304101450 msgr2=0x7f53041a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:51.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.765+0000 7f530b532640 1 --2- 172.21.15.52:0/2993479737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5304101450 0x7f53041a4bf0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f52f8009230 tx=0x7f52f802cec0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.765+0000 7f530b532640 1 -- 172.21.15.52:0/2993479737 shutdown_connections 2024-04-02T13:56:51.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.765+0000 7f530b532640 1 --2- 172.21.15.52:0/2993479737 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f52ec0a3d20 0x7f52ec0a61e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.765+0000 7f530b532640 1 --2- 172.21.15.52:0/2993479737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5304105f30 0x7f530419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.765+0000 7f530b532640 1 --2- 172.21.15.52:0/2993479737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5304101450 0x7f53041a4bf0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.765+0000 7f530b532640 1 --2- 172.21.15.52:0/2993479737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5304100b30 0x7f53041a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.765+0000 7f530b532640 1 -- 172.21.15.52:0/2993479737 >> 172.21.15.52:0/2993479737 conn(0x7f5304076d80 msgr2=0x7f53041038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:51.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.765+0000 7f530b532640 1 -- 172.21.15.52:0/2993479737 shutdown_connections 2024-04-02T13:56:51.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.765+0000 7f530b532640 1 -- 172.21.15.52:0/2993479737 wait complete. 2024-04-02T13:56:51.783 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:56:51.783 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:56:51.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:51 smithi052 ceph-mon[32104]: pgmap v103: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-02T13:56:51.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.883+0000 7f68c3dcd640 1 -- 172.21.15.52:0/2853429024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68bc1069b0 msgr2=0x7f68bc106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:51.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.883+0000 7f68c3dcd640 1 --2- 172.21.15.52:0/2853429024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68bc1069b0 0x7f68bc106d90 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009930 tx=0x7f68b802bc30 comp rx=0 tx=0).stop 2024-04-02T13:56:51.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.883+0000 7f68c3dcd640 1 -- 172.21.15.52:0/2853429024 shutdown_connections 2024-04-02T13:56:51.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.883+0000 7f68c3dcd640 1 --2- 172.21.15.52:0/2853429024 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68bc101360 0x7f68bc10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.883+0000 7f68c3dcd640 1 --2- 172.21.15.52:0/2853429024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68bc1009a0 0x7f68bc100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.883+0000 7f68c3dcd640 1 --2- 172.21.15.52:0/2853429024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68bc1069b0 0x7f68bc106d90 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.883+0000 7f68c3dcd640 1 -- 172.21.15.52:0/2853429024 >> 172.21.15.52:0/2853429024 conn(0x7f68bc076d80 msgr2=0x7f68bc077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:51.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.883+0000 7f68c3dcd640 1 -- 172.21.15.52:0/2853429024 shutdown_connections 2024-04-02T13:56:51.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.883+0000 7f68c3dcd640 1 -- 172.21.15.52:0/2853429024 wait complete. 2024-04-02T13:56:51.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.884+0000 7f68c3dcd640 1 Processor -- start 2024-04-02T13:56:51.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.884+0000 7f68c3dcd640 1 -- start start 2024-04-02T13:56:51.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.884+0000 7f68c3dcd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68bc1009a0 0x7f68bc198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:51.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.884+0000 7f68c3dcd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68bc101360 0x7f68bc198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:51.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.884+0000 7f68c3dcd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68bc1069b0 0x7f68bc19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:51.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.884+0000 7f68c3dcd640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc111ec0 con 0x7f68bc1009a0 2024-04-02T13:56:51.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.884+0000 7f68c3dcd640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc111d40 con 0x7f68bc1069b0 2024-04-02T13:56:51.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.884+0000 7f68c3dcd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc112040 con 0x7f68bc101360 2024-04-02T13:56:51.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.884+0000 7f68c1b69640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68bc1009a0 0x7f68bc198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:51.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.884+0000 7f68c1b69640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68bc1009a0 0x7f68bc198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51802/0 (socket says 172.21.15.52:51802) 2024-04-02T13:56:51.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.884+0000 7f68c1b69640 1 -- 172.21.15.52:0/4236803421 learned_addr learned my addr 172.21.15.52:0/4236803421 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:51.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.884+0000 7f68c236a640 1 --2- 172.21.15.52:0/4236803421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68bc1069b0 0x7f68bc19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:51.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.884+0000 7f68c1b69640 1 -- 172.21.15.52:0/4236803421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68bc101360 msgr2=0x7f68bc198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:51.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.884+0000 7f68c1b69640 1 --2- 172.21.15.52:0/4236803421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68bc101360 0x7f68bc198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.884+0000 7f68c1b69640 1 -- 172.21.15.52:0/4236803421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68bc1069b0 msgr2=0x7f68bc19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:51.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.884+0000 7f68c1b69640 1 --2- 172.21.15.52:0/4236803421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68bc1069b0 0x7f68bc19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:51.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.884+0000 7f68c1b69640 1 -- 172.21.15.52:0/4236803421 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68bc19d3c0 con 0x7f68bc1009a0 2024-04-02T13:56:51.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.885+0000 7f68c1b69640 1 --2- 172.21.15.52:0/4236803421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68bc1009a0 0x7f68bc198420 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009230 tx=0x7f68b802cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:51.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.885+0000 7f68b2ffd640 1 -- 172.21.15.52:0/4236803421 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8041070 con 0x7f68bc1009a0 2024-04-02T13:56:51.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.885+0000 7f68b2ffd640 1 -- 172.21.15.52:0/4236803421 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b8004450 con 0x7f68bc1009a0 2024-04-02T13:56:51.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.885+0000 7f68b2ffd640 1 -- 172.21.15.52:0/4236803421 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8034440 con 0x7f68bc1009a0 2024-04-02T13:56:51.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.885+0000 7f68c3dcd640 1 -- 172.21.15.52:0/4236803421 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68bc19d650 con 0x7f68bc1009a0 2024-04-02T13:56:51.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.885+0000 7f68c3dcd640 1 -- 172.21.15.52:0/4236803421 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68bc1a4f40 con 0x7f68bc1009a0 2024-04-02T13:56:51.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.886+0000 7f68c3dcd640 1 -- 172.21.15.52:0/4236803421 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68bc1020e0 con 0x7f68bc1009a0 2024-04-02T13:56:51.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.887+0000 7f68b2ffd640 1 -- 172.21.15.52:0/4236803421 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f68b8044070 con 0x7f68bc1009a0 2024-04-02T13:56:51.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.887+0000 7f68b2ffd640 1 --2- 172.21.15.52:0/4236803421 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f68a80a3d20 0x7f68a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:51.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.887+0000 7f68c1368640 1 --2- 172.21.15.52:0/4236803421 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f68a80a3d20 0x7f68a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:51.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.887+0000 7f68b2ffd640 1 -- 172.21.15.52:0/4236803421 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f68b80f28f0 con 0x7f68bc1009a0 2024-04-02T13:56:51.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.888+0000 7f68c1368640 1 --2- 172.21.15.52:0/4236803421 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f68a80a3d20 0x7f68a80a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f68ac005fd0 tx=0x7f68ac005850 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:51.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:51.891+0000 7f68b2ffd640 1 -- 172.21.15.52:0/4236803421 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f68b80bed00 con 0x7f68bc1009a0 2024-04-02T13:56:51.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:51 smithi088 ceph-mon[36528]: pgmap v103: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-02T13:56:52.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:51 smithi189 ceph-mon[36683]: pgmap v103: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-02T13:56:52.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.119+0000 7f68c3dcd640 1 -- 172.21.15.52:0/4236803421 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68bc068d40 con 0x7f68bc1009a0 2024-04-02T13:56:52.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.120+0000 7f68b2ffd640 1 -- 172.21.15.52:0/4236803421 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11924 (secure 0 0 0) 0x7f68bc068d40 con 0x7f68bc1009a0 2024-04-02T13:56:52.123 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:56:52.123 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:55:15.239889+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24475,24503,24490,24457,24467]},"id":1}]} 2024-04-02T13:56:52.124 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 29 2024-04-02T13:56:52.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.121+0000 7f68c3dcd640 1 -- 172.21.15.52:0/4236803421 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f68a80a3d20 msgr2=0x7f68a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.121+0000 7f68c3dcd640 1 --2- 172.21.15.52:0/4236803421 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f68a80a3d20 0x7f68a80a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f68ac005fd0 tx=0x7f68ac005850 comp rx=0 tx=0).stop 2024-04-02T13:56:52.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.121+0000 7f68c3dcd640 1 -- 172.21.15.52:0/4236803421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68bc1009a0 msgr2=0x7f68bc198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.121+0000 7f68c3dcd640 1 --2- 172.21.15.52:0/4236803421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68bc1009a0 0x7f68bc198420 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009230 tx=0x7f68b802cba0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.122+0000 7f68c3dcd640 1 -- 172.21.15.52:0/4236803421 shutdown_connections 2024-04-02T13:56:52.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.122+0000 7f68c3dcd640 1 --2- 172.21.15.52:0/4236803421 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f68a80a3d20 0x7f68a80a61e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.122+0000 7f68c3dcd640 1 --2- 172.21.15.52:0/4236803421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68bc1069b0 0x7f68bc19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.122+0000 7f68c3dcd640 1 --2- 172.21.15.52:0/4236803421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68bc101360 0x7f68bc198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.122+0000 7f68c3dcd640 1 --2- 172.21.15.52:0/4236803421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68bc1009a0 0x7f68bc198420 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.122+0000 7f68c3dcd640 1 -- 172.21.15.52:0/4236803421 >> 172.21.15.52:0/4236803421 conn(0x7f68bc076d80 msgr2=0x7f68bc0fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.122+0000 7f68c3dcd640 1 -- 172.21.15.52:0/4236803421 shutdown_connections 2024-04-02T13:56:52.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.122+0000 7f68c3dcd640 1 -- 172.21.15.52:0/4236803421 wait complete. 2024-04-02T13:56:52.136 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:56:52.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.237+0000 7f6596f3d640 1 -- 172.21.15.52:0/2489945516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6590101370 msgr2=0x7f659010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.237+0000 7f6596f3d640 1 --2- 172.21.15.52:0/2489945516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6590101370 0x7f659010d880 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6588009cd0 tx=0x7f658802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.237+0000 7f6596f3d640 1 -- 172.21.15.52:0/2489945516 shutdown_connections 2024-04-02T13:56:52.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.237+0000 7f6596f3d640 1 --2- 172.21.15.52:0/2489945516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6590101370 0x7f659010d880 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.237+0000 7f6596f3d640 1 --2- 172.21.15.52:0/2489945516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65901009b0 0x7f6590100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.237+0000 7f6596f3d640 1 --2- 172.21.15.52:0/2489945516 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65901069c0 0x7f6590106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.237+0000 7f6596f3d640 1 -- 172.21.15.52:0/2489945516 >> 172.21.15.52:0/2489945516 conn(0x7f6590076d80 msgr2=0x7f6590077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.237+0000 7f6596f3d640 1 -- 172.21.15.52:0/2489945516 shutdown_connections 2024-04-02T13:56:52.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.237+0000 7f6596f3d640 1 -- 172.21.15.52:0/2489945516 wait complete. 2024-04-02T13:56:52.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.238+0000 7f6596f3d640 1 Processor -- start 2024-04-02T13:56:52.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.238+0000 7f6596f3d640 1 -- start start 2024-04-02T13:56:52.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.238+0000 7f6596f3d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65901009b0 0x7f6590199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.238+0000 7f6596f3d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6590101370 0x7f65901996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.238+0000 7f6596f3d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65901069c0 0x7f65901a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.238+0000 7f6596f3d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6590110da0 con 0x7f65901069c0 2024-04-02T13:56:52.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.238+0000 7f6596f3d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6590110c20 con 0x7f65901009b0 2024-04-02T13:56:52.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.238+0000 7f6596f3d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6590110f20 con 0x7f6590101370 2024-04-02T13:56:52.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.238+0000 7f65954da640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65901069c0 0x7f65901a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.238+0000 7f65954da640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65901069c0 0x7f65901a4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51818/0 (socket says 172.21.15.52:51818) 2024-04-02T13:56:52.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.238+0000 7f65954da640 1 -- 172.21.15.52:0/1285145057 learned_addr learned my addr 172.21.15.52:0/1285145057 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:52.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.238+0000 7f658ffff640 1 --2- 172.21.15.52:0/1285145057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6590101370 0x7f65901996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.238+0000 7f6594cd9640 1 --2- 172.21.15.52:0/1285145057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65901009b0 0x7f6590199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.238+0000 7f65954da640 1 -- 172.21.15.52:0/1285145057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6590101370 msgr2=0x7f65901996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.238+0000 7f65954da640 1 --2- 172.21.15.52:0/1285145057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6590101370 0x7f65901996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.238+0000 7f65954da640 1 -- 172.21.15.52:0/1285145057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65901009b0 msgr2=0x7f6590199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.238+0000 7f65954da640 1 --2- 172.21.15.52:0/1285145057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65901009b0 0x7f6590199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.238+0000 7f65954da640 1 -- 172.21.15.52:0/1285145057 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65901a4740 con 0x7f65901069c0 2024-04-02T13:56:52.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.238+0000 7f658ffff640 1 --2- 172.21.15.52:0/1285145057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6590101370 0x7f65901996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:52.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.239+0000 7f65954da640 1 --2- 172.21.15.52:0/1285145057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65901069c0 0x7f65901a4200 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f65880348f0 tx=0x7f6588032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.239+0000 7f658dffb640 1 -- 172.21.15.52:0/1285145057 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588041020 con 0x7f65901069c0 2024-04-02T13:56:52.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.239+0000 7f658dffb640 1 -- 172.21.15.52:0/1285145057 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6588034cd0 con 0x7f65901069c0 2024-04-02T13:56:52.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.239+0000 7f6596f3d640 1 -- 172.21.15.52:0/1285145057 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65901a4910 con 0x7f65901069c0 2024-04-02T13:56:52.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.239+0000 7f658dffb640 1 -- 172.21.15.52:0/1285145057 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588039040 con 0x7f65901069c0 2024-04-02T13:56:52.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.239+0000 7f6596f3d640 1 -- 172.21.15.52:0/1285145057 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65901a4d80 con 0x7f65901069c0 2024-04-02T13:56:52.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.241+0000 7f658dffb640 1 -- 172.21.15.52:0/1285145057 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6588044020 con 0x7f65901069c0 2024-04-02T13:56:52.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.241+0000 7f6599bfe740 1 Processor -- start 2024-04-02T13:56:52.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.241+0000 7f6599bfe740 1 -- start start 2024-04-02T13:56:52.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.242+0000 7f6599bfe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555969a927a0 0x555969a92b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.242+0000 7f6599bfe740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a930c0 0x555969a9b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.242+0000 7f6599bfe740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555969a9bca0 0x555969a9e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.242+0000 7f6599bfe740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5559699f67b0 con 0x555969a930c0 2024-04-02T13:56:52.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.242+0000 7f6599bfe740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x555969995570 con 0x555969a9bca0 2024-04-02T13:56:52.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.242+0000 7f6599bfe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5559699f2760 con 0x555969a927a0 2024-04-02T13:56:52.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.242+0000 7f658dffb640 1 --2- 172.21.15.52:0/1285145057 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f65800a3d20 0x7f65800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.242+0000 7f6594cd9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a930c0 0x555969a9b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.242+0000 7f6594cd9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a930c0 0x555969a9b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51824/0 (socket says 172.21.15.52:51824) 2024-04-02T13:56:52.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.242+0000 7f6594cd9640 1 -- 172.21.15.52:0/1847971941 learned_addr learned my addr 172.21.15.52:0/1847971941 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:52.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.242+0000 7f65954da640 1 --2- 172.21.15.52:0/1285145057 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f65800a3d20 0x7f65800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.242+0000 7f658dffb640 1 -- 172.21.15.52:0/1285145057 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f65880f2a40 con 0x7f65901069c0 2024-04-02T13:56:52.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.242+0000 7f658ffff640 1 --2- 172.21.15.52:0/1847971941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555969a927a0 0x555969a92b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.242+0000 7f658ffff640 1 --2- 172.21.15.52:0/1847971941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555969a9bca0 0x555969a9e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.242+0000 7f6594cd9640 1 -- 172.21.15.52:0/1847971941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555969a927a0 msgr2=0x555969a92b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.242+0000 7f6594cd9640 1 --2- 172.21.15.52:0/1847971941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555969a927a0 0x555969a92b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.242+0000 7f6594cd9640 1 -- 172.21.15.52:0/1847971941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555969a9bca0 msgr2=0x555969a9e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.242+0000 7f6594cd9640 1 --2- 172.21.15.52:0/1847971941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555969a9bca0 0x555969a9e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.242+0000 7f6594cd9640 1 -- 172.21.15.52:0/1847971941 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55596970a3f0 con 0x555969a930c0 2024-04-02T13:56:52.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.242+0000 7f658ffff640 1 --2- 172.21.15.52:0/1847971941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555969a9bca0 0x555969a9e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:52.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.242+0000 7f65954da640 1 --2- 172.21.15.52:0/1285145057 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f65800a3d20 0x7f65800a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f65880be450 tx=0x7f6588047f90 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.242+0000 7f6594cd9640 1 --2- 172.21.15.52:0/1847971941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a930c0 0x555969a9b760 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f658400bc60 tx=0x7f658400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aba62bc7a68533d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.243+0000 7f657affd640 1 -- 172.21.15.52:0/1847971941 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6584015070 con 0x555969a930c0 2024-04-02T13:56:52.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.243+0000 7f657affd640 1 -- 172.21.15.52:0/1847971941 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6584011920 con 0x555969a930c0 2024-04-02T13:56:52.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.243+0000 7f657affd640 1 -- 172.21.15.52:0/1847971941 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65840108f0 con 0x555969a930c0 2024-04-02T13:56:52.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.243+0000 7f6599bfe740 1 -- 172.21.15.52:0/1847971941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a930c0 msgr2=0x555969a9b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.243+0000 7f6599bfe740 1 --2- 172.21.15.52:0/1847971941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a930c0 0x555969a9b760 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f658400bc60 tx=0x7f658400f730 comp rx=0 tx=0).stop 2024-04-02T13:56:52.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.243+0000 7f6599bfe740 1 -- 172.21.15.52:0/1847971941 shutdown_connections 2024-04-02T13:56:52.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.243+0000 7f6599bfe740 1 --2- 172.21.15.52:0/1847971941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555969a9bca0 0x555969a9e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.243+0000 7f6599bfe740 1 --2- 172.21.15.52:0/1847971941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a930c0 0x555969a9b760 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.243+0000 7f6599bfe740 1 --2- 172.21.15.52:0/1847971941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555969a927a0 0x555969a92b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.243+0000 7f6599bfe740 1 -- 172.21.15.52:0/1847971941 >> 172.21.15.52:0/1847971941 conn(0x555969a8ced0 msgr2=0x555969a8d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.243+0000 7f6599bfe740 1 -- 172.21.15.52:0/1847971941 shutdown_connections 2024-04-02T13:56:52.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.243+0000 7f6599bfe740 1 -- 172.21.15.52:0/1847971941 wait complete. 2024-04-02T13:56:52.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.244+0000 7f6599bfe740 1 -- start start 2024-04-02T13:56:52.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.244+0000 7f6599bfe740 1 Processor -- start 2024-04-02T13:56:52.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.244+0000 7f6599bfe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555969a927a0 0x555969a9b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.244+0000 7f6599bfe740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555969a930c0 0x555969a98af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.244+0000 7f6599bfe740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a9bca0 0x555969a99230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.244+0000 7f6599bfe740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555969a9ea90 con 0x555969a9bca0 2024-04-02T13:56:52.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.244+0000 7f6599bfe740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5559699b1e80 con 0x555969a930c0 2024-04-02T13:56:52.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.244+0000 7f6599bfe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555969a9ec10 con 0x555969a927a0 2024-04-02T13:56:52.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.244+0000 7f6599bfe740 10 client.0 _ll_register_callbacks cb 0x5559699edaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:52.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.244+0000 7f658ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a9bca0 0x555969a99230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.245+0000 7f658ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a9bca0 0x555969a99230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51838/0 (socket says 172.21.15.52:51838) 2024-04-02T13:56:52.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.245+0000 7f658ffff640 1 -- 172.21.15.52:0/2857511991 learned_addr learned my addr 172.21.15.52:0/2857511991 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:52.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.245+0000 7f6594cd9640 1 --2- 172.21.15.52:0/2857511991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555969a930c0 0x555969a98af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.245+0000 7f658ffff640 1 --2- 172.21.15.52:0/2857511991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555969a927a0 0x555969a9b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.245+0000 7f658ffff640 1 -- 172.21.15.52:0/2857511991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555969a927a0 msgr2=0x555969a9b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.245+0000 7f658ffff640 1 --2- 172.21.15.52:0/2857511991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555969a927a0 0x555969a9b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.245+0000 7f658ffff640 1 -- 172.21.15.52:0/2857511991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555969a930c0 msgr2=0x555969a98af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.245+0000 7f658ffff640 1 --2- 172.21.15.52:0/2857511991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555969a930c0 0x555969a98af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.245+0000 7f658ffff640 1 -- 172.21.15.52:0/2857511991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559697d47d0 con 0x555969a9bca0 2024-04-02T13:56:52.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.245+0000 7f658ffff640 1 --2- 172.21.15.52:0/2857511991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a9bca0 0x555969a99230 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f657c012f10 tx=0x7f657c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334f18784e637443 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.245+0000 7f656ffff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:52.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.245+0000 7f656ffff640 1 -- 172.21.15.52:0/2857511991 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c004030 con 0x555969a9bca0 2024-04-02T13:56:52.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.245+0000 7f656ffff640 1 -- 172.21.15.52:0/2857511991 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f657c0041d0 con 0x555969a9bca0 2024-04-02T13:56:52.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.245+0000 7f6599bfe740 1 -- 172.21.15.52:0/2857511991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6584011920 con 0x555969a9bca0 2024-04-02T13:56:52.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.245+0000 7f656ffff640 1 -- 172.21.15.52:0/2857511991 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f657c028050 con 0x555969a9bca0 2024-04-02T13:56:52.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.245+0000 7f6599bfe740 10 client.16578 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:52.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.245+0000 7f6599bfe740 1 -- 172.21.15.52:0/2857511991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555969aaaba0 con 0x555969a9bca0 2024-04-02T13:56:52.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.246+0000 7f656ffff640 1 -- 172.21.15.52:0/2857511991 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6584011920 con 0x555969a9bca0 2024-04-02T13:56:52.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.246+0000 7f656ffff640 1 -- 172.21.15.52:0/2857511991 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f657c0094c0 con 0x555969a9bca0 2024-04-02T13:56:52.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.246+0000 7f6599bfe740 10 client.16578 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:52.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.246+0000 7f6599bfe740 10 client.16578 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T13:56:52.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.246+0000 7f6599bfe740 20 client.16578 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:52.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.246+0000 7f6599bfe740 1 --2- 172.21.15.52:0/2857511991 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x555969aabfb0 0x555969aae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.246+0000 7f6599bfe740 4 client.16578 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:52.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.246+0000 7f6599bfe740 1 -- 172.21.15.52:0/2857511991 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555969a03cf0 con 0x555969aabfb0 2024-04-02T13:56:52.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.246+0000 7f6594cd9640 1 --2- 172.21.15.52:0/2857511991 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x555969aabfb0 0x555969aae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.246+0000 7f6594cd9640 1 --2- 172.21.15.52:0/2857511991 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x555969aabfb0 0x555969aae3c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=508723e1ee6e5e87 server_cookie=bb2937b9b17116f0 in_seq=0 out_seq=0 2024-04-02T13:56:52.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.246+0000 7f656ffff640 10 client.16578 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:56:52.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.247+0000 7f656ffff640 1 -- 172.21.15.52:0/2857511991 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555969a03cf0 con 0x555969aabfb0 2024-04-02T13:56:52.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.247+0000 7f656ffff640 10 client.16578 handle_command_reply: tid=0 2024-04-02T13:56:52.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.247+0000 7f6599bfe740 1 client.16578 shutdown 2024-04-02T13:56:52.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.247+0000 7f6599bfe740 1 -- 172.21.15.52:0/2857511991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a9bca0 msgr2=0x555969a99230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.247+0000 7f6599bfe740 1 --2- 172.21.15.52:0/2857511991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a9bca0 0x555969a99230 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f657c012f10 tx=0x7f657c0046c0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.247+0000 7f6599bfe740 1 -- 172.21.15.52:0/2857511991 shutdown_connections 2024-04-02T13:56:52.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.247+0000 7f6599bfe740 1 --2- 172.21.15.52:0/2857511991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a9bca0 0x555969a99230 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.247+0000 7f6599bfe740 1 --2- 172.21.15.52:0/2857511991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555969a930c0 0x555969a98af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.247+0000 7f6599bfe740 1 --2- 172.21.15.52:0/2857511991 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x555969aabfb0 0x555969aae3c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.247+0000 7f6599bfe740 1 --2- 172.21.15.52:0/2857511991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555969a927a0 0x555969a9b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.247+0000 7f656ffff640 0 client.16578 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:56:52.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.247+0000 7f6599bfe740 1 -- 172.21.15.52:0/2857511991 >> 172.21.15.52:0/2857511991 conn(0x555969a8ced0 msgr2=0x555969a8ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.248+0000 7f6599bfe740 1 -- 172.21.15.52:0/2857511991 shutdown_connections 2024-04-02T13:56:52.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.248+0000 7f6599bfe740 1 -- 172.21.15.52:0/2857511991 wait complete. 2024-04-02T13:56:52.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.248+0000 7f6599bfe740 20 client.16578 trim_cache size 0 max 16384 2024-04-02T13:56:52.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.262+0000 7f6599bfe740 1 Processor -- start 2024-04-02T13:56:52.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.262+0000 7f6599bfe740 1 -- start start 2024-04-02T13:56:52.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.262+0000 7f6599bfe740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a927a0 0x555969aa7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.262+0000 7f6599bfe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555969a930c0 0x555969a97ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.262+0000 7f6599bfe740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555969a9bca0 0x555969a9b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.262+0000 7f6599bfe740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f65901aee90 con 0x555969a927a0 2024-04-02T13:56:52.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.262+0000 7f6599bfe740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f65901b0a00 con 0x555969a9bca0 2024-04-02T13:56:52.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.262+0000 7f6599bfe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f65901af560 con 0x555969a930c0 2024-04-02T13:56:52.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.262+0000 7f658ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a927a0 0x555969aa7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.262+0000 7f658ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a927a0 0x555969aa7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51846/0 (socket says 172.21.15.52:51846) 2024-04-02T13:56:52.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.262+0000 7f658ffff640 1 -- 172.21.15.52:0/792228027 learned_addr learned my addr 172.21.15.52:0/792228027 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:52.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.262+0000 7f6594cd9640 1 --2- 172.21.15.52:0/792228027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555969a930c0 0x555969a97ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.262+0000 7f658ffff640 1 -- 172.21.15.52:0/792228027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555969a930c0 msgr2=0x555969a97ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.262+0000 7f658ffff640 1 --2- 172.21.15.52:0/792228027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555969a930c0 0x555969a97ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.262+0000 7f658ffff640 1 -- 172.21.15.52:0/792228027 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555969a9bca0 msgr2=0x555969a9b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.262+0000 7f658ffff640 1 --2- 172.21.15.52:0/792228027 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555969a9bca0 0x555969a9b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.262+0000 7f658ffff640 1 -- 172.21.15.52:0/792228027 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555969a8dc60 con 0x555969a927a0 2024-04-02T13:56:52.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.262+0000 7f6594cd9640 1 --2- 172.21.15.52:0/792228027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555969a930c0 0x555969a97ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:56:52.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.262+0000 7f658ffff640 1 --2- 172.21.15.52:0/792228027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a927a0 0x555969aa7950 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f657c00d010 tx=0x7f657c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b7a2a79d546b5a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.263+0000 7f657a7fc640 1 -- 172.21.15.52:0/792228027 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c0096c0 con 0x555969a927a0 2024-04-02T13:56:52.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.263+0000 7f657a7fc640 1 -- 172.21.15.52:0/792228027 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f657c031070 con 0x555969a927a0 2024-04-02T13:56:52.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.263+0000 7f657a7fc640 1 -- 172.21.15.52:0/792228027 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c0052c0 con 0x555969a927a0 2024-04-02T13:56:52.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.263+0000 7f6599bfe740 1 -- 172.21.15.52:0/792228027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a927a0 msgr2=0x555969aa7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.263+0000 7f6599bfe740 1 --2- 172.21.15.52:0/792228027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a927a0 0x555969aa7950 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f657c00d010 tx=0x7f657c00cc80 comp rx=0 tx=0).stop 2024-04-02T13:56:52.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.263+0000 7f6599bfe740 1 -- 172.21.15.52:0/792228027 shutdown_connections 2024-04-02T13:56:52.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.263+0000 7f6599bfe740 1 --2- 172.21.15.52:0/792228027 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555969a9bca0 0x555969a9b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.263+0000 7f6599bfe740 1 --2- 172.21.15.52:0/792228027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555969a930c0 0x555969a97ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.263+0000 7f6599bfe740 1 --2- 172.21.15.52:0/792228027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a927a0 0x555969aa7950 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.263+0000 7f6599bfe740 1 -- 172.21.15.52:0/792228027 >> 172.21.15.52:0/792228027 conn(0x555969a8ced0 msgr2=0x555969a973c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.263+0000 7f6599bfe740 1 -- 172.21.15.52:0/792228027 shutdown_connections 2024-04-02T13:56:52.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.263+0000 7f6599bfe740 1 -- 172.21.15.52:0/792228027 wait complete. 2024-04-02T13:56:52.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.264+0000 7f6599bfe740 1 -- start start 2024-04-02T13:56:52.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.264+0000 7f6599bfe740 1 Processor -- start 2024-04-02T13:56:52.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.264+0000 7f6599bfe740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555969a927a0 0x555969aa3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.264+0000 7f6599bfe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555969a930c0 0x555969a9f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.264+0000 7f6599bfe740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a9bca0 0x555969a9f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.264+0000 7f6599bfe740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f65901b0300 con 0x555969a9bca0 2024-04-02T13:56:52.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.264+0000 7f6599bfe740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f65901afc30 con 0x555969a927a0 2024-04-02T13:56:52.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.264+0000 7f6599bfe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f65901ace70 con 0x555969a930c0 2024-04-02T13:56:52.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.264+0000 7f6599bfe740 10 client.0 _ll_register_callbacks cb 0x7f657c012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:52.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.264+0000 7f658ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a9bca0 0x555969a9f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.264+0000 7f658ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a9bca0 0x555969a9f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51862/0 (socket says 172.21.15.52:51862) 2024-04-02T13:56:52.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.264+0000 7f658ffff640 1 -- 172.21.15.52:0/3898206545 learned_addr learned my addr 172.21.15.52:0/3898206545 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:52.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.264+0000 7f6594cd9640 1 --2- 172.21.15.52:0/3898206545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555969a930c0 0x555969a9f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.264+0000 7f658ffff640 1 --2- 172.21.15.52:0/3898206545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555969a927a0 0x555969aa3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.264+0000 7f658ffff640 1 -- 172.21.15.52:0/3898206545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555969a930c0 msgr2=0x555969a9f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.265+0000 7f658ffff640 1 --2- 172.21.15.52:0/3898206545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555969a930c0 0x555969a9f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.265+0000 7f658ffff640 1 -- 172.21.15.52:0/3898206545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555969a927a0 msgr2=0x555969aa3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.265+0000 7f658ffff640 1 --2- 172.21.15.52:0/3898206545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555969a927a0 0x555969aa3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.265+0000 7f658ffff640 1 -- 172.21.15.52:0/3898206545 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555969a9ff70 con 0x555969a9bca0 2024-04-02T13:56:52.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.265+0000 7f658ffff640 1 --2- 172.21.15.52:0/3898206545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a9bca0 0x555969a9f980 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f657c009a30 tx=0x7f657c009cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b802fc08b7fba31 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.265+0000 7f656ffff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:52.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.265+0000 7f656ffff640 1 -- 172.21.15.52:0/3898206545 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c0096c0 con 0x555969a9bca0 2024-04-02T13:56:52.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.265+0000 7f656ffff640 1 -- 172.21.15.52:0/3898206545 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f657c004030 con 0x555969a9bca0 2024-04-02T13:56:52.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.265+0000 7f6599bfe740 1 -- 172.21.15.52:0/3898206545 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f657c031070 con 0x555969a9bca0 2024-04-02T13:56:52.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.265+0000 7f656ffff640 1 -- 172.21.15.52:0/3898206545 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f657c009ab0 con 0x555969a9bca0 2024-04-02T13:56:52.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.265+0000 7f6599bfe740 10 client.16584 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:52.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.265+0000 7f6599bfe740 1 -- 172.21.15.52:0/3898206545 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555969b12250 con 0x555969a9bca0 2024-04-02T13:56:52.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.265+0000 7f656ffff640 1 -- 172.21.15.52:0/3898206545 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f657c031070 con 0x555969a9bca0 2024-04-02T13:56:52.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.265+0000 7f656ffff640 1 -- 172.21.15.52:0/3898206545 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f657c004850 con 0x555969a9bca0 2024-04-02T13:56:52.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.265+0000 7f6599bfe740 10 client.16584 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:52.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.265+0000 7f6599bfe740 10 client.16584 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T13:56:52.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.265+0000 7f6599bfe740 20 client.16584 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:52.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.265+0000 7f6599bfe740 1 --2- 172.21.15.52:0/3898206545 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x555969b12850 0x555969b14c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.265+0000 7f6599bfe740 4 client.16584 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:56:52.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.265+0000 7f6599bfe740 1 -- 172.21.15.52:0/3898206545 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5559699fe040 con 0x555969b12850 2024-04-02T13:56:52.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.266+0000 7f6594cd9640 1 --2- 172.21.15.52:0/3898206545 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x555969b12850 0x555969b14c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.266+0000 7f656ffff640 10 client.16584 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:56:52.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.266+0000 7f6594cd9640 1 --2- 172.21.15.52:0/3898206545 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x555969b12850 0x555969b14c40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1a8a2ecc61e4897d server_cookie=bda1638266955aed in_seq=0 out_seq=0 2024-04-02T13:56:52.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.266+0000 7f656ffff640 1 -- 172.21.15.52:0/3898206545 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5559699fe040 con 0x555969b12850 2024-04-02T13:56:52.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.266+0000 7f656ffff640 10 client.16584 handle_command_reply: tid=0 2024-04-02T13:56:52.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.266+0000 7f6599bfe740 1 client.16584 shutdown 2024-04-02T13:56:52.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.266+0000 7f6599bfe740 1 -- 172.21.15.52:0/3898206545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a9bca0 msgr2=0x555969a9f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.266+0000 7f6599bfe740 1 --2- 172.21.15.52:0/3898206545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a9bca0 0x555969a9f980 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f657c009a30 tx=0x7f657c009cc0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.266+0000 7f6599bfe740 1 -- 172.21.15.52:0/3898206545 shutdown_connections 2024-04-02T13:56:52.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.266+0000 7f6599bfe740 1 --2- 172.21.15.52:0/3898206545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555969a9bca0 0x555969a9f980 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.266+0000 7f6599bfe740 1 --2- 172.21.15.52:0/3898206545 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x555969b12850 0x555969b14c40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.266+0000 7f6599bfe740 1 --2- 172.21.15.52:0/3898206545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555969a930c0 0x555969a9f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.266+0000 7f6599bfe740 1 --2- 172.21.15.52:0/3898206545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555969a927a0 0x555969aa3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.266+0000 7f656ffff640 0 client.16584 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:56:52.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.266+0000 7f6599bfe740 1 -- 172.21.15.52:0/3898206545 >> 172.21.15.52:0/3898206545 conn(0x555969a8ced0 msgr2=0x555969a8b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.267+0000 7f6599bfe740 1 -- 172.21.15.52:0/3898206545 shutdown_connections 2024-04-02T13:56:52.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.267+0000 7f6599bfe740 1 -- 172.21.15.52:0/3898206545 wait complete. 2024-04-02T13:56:52.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.267+0000 7f6599bfe740 20 client.16584 trim_cache size 0 max 16384 2024-04-02T13:56:52.281 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:56:52.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.267+0000 7f6596f3d640 1 -- 172.21.15.52:0/1285145057 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f65800a3d20 msgr2=0x7f65800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.267+0000 7f6596f3d640 1 --2- 172.21.15.52:0/1285145057 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f65800a3d20 0x7f65800a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f65880be450 tx=0x7f6588047f90 comp rx=0 tx=0).stop 2024-04-02T13:56:52.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.267+0000 7f6596f3d640 1 -- 172.21.15.52:0/1285145057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65901069c0 msgr2=0x7f65901a4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.267+0000 7f6596f3d640 1 --2- 172.21.15.52:0/1285145057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65901069c0 0x7f65901a4200 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f65880348f0 tx=0x7f6588032ec0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.267+0000 7f6596f3d640 1 -- 172.21.15.52:0/1285145057 shutdown_connections 2024-04-02T13:56:52.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.267+0000 7f6596f3d640 1 --2- 172.21.15.52:0/1285145057 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f65800a3d20 0x7f65800a61e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.267+0000 7f6596f3d640 1 --2- 172.21.15.52:0/1285145057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65901069c0 0x7f65901a4200 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.267+0000 7f6596f3d640 1 --2- 172.21.15.52:0/1285145057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6590101370 0x7f65901996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.267+0000 7f6596f3d640 1 --2- 172.21.15.52:0/1285145057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65901009b0 0x7f6590199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.267+0000 7f6596f3d640 1 -- 172.21.15.52:0/1285145057 >> 172.21.15.52:0/1285145057 conn(0x7f6590076d80 msgr2=0x7f65900fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.267+0000 7f6596f3d640 1 -- 172.21.15.52:0/1285145057 shutdown_connections 2024-04-02T13:56:52.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.267+0000 7f6596f3d640 1 -- 172.21.15.52:0/1285145057 wait complete. 2024-04-02T13:56:52.284 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:56:52.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.386+0000 7fcecd59d640 1 -- 172.21.15.52:0/2629575945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcec8068d00 msgr2=0x7fcec8069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.386+0000 7fcecd59d640 1 --2- 172.21.15.52:0/2629575945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcec8068d00 0x7fcec8069180 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009930 tx=0x7fcebc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:56:52.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.386+0000 7fcecd59d640 1 -- 172.21.15.52:0/2629575945 shutdown_connections 2024-04-02T13:56:52.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.386+0000 7fcecd59d640 1 --2- 172.21.15.52:0/2629575945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcec80696c0 0x7fcec810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.386+0000 7fcecd59d640 1 --2- 172.21.15.52:0/2629575945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcec8068d00 0x7fcec8069180 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.386+0000 7fcecd59d640 1 --2- 172.21.15.52:0/2629575945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcec8107b50 0x7fcec8107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.386+0000 7fcecd59d640 1 -- 172.21.15.52:0/2629575945 >> 172.21.15.52:0/2629575945 conn(0x7fcec8077b00 msgr2=0x7fcec8077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.386+0000 7fcecd59d640 1 -- 172.21.15.52:0/2629575945 shutdown_connections 2024-04-02T13:56:52.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.386+0000 7fcecd59d640 1 -- 172.21.15.52:0/2629575945 wait complete. 2024-04-02T13:56:52.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.387+0000 7fcecd59d640 1 Processor -- start 2024-04-02T13:56:52.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.387+0000 7fcecd59d640 1 -- start start 2024-04-02T13:56:52.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.387+0000 7fcecd59d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcec8068d00 0x7fcec81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.387+0000 7fcecd59d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcec80696c0 0x7fcec8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.387+0000 7fcecd59d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcec8107b50 0x7fcec819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.387+0000 7fcecd59d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcec8111e70 con 0x7fcec8107b50 2024-04-02T13:56:52.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.387+0000 7fcecd59d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcec8111cf0 con 0x7fcec80696c0 2024-04-02T13:56:52.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.387+0000 7fcecd59d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcec8111ff0 con 0x7fcec8068d00 2024-04-02T13:56:52.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.387+0000 7fcec77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcec8107b50 0x7fcec819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.387+0000 7fcec77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcec8107b50 0x7fcec819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51874/0 (socket says 172.21.15.52:51874) 2024-04-02T13:56:52.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.387+0000 7fcec77fe640 1 -- 172.21.15.52:0/1389042522 learned_addr learned my addr 172.21.15.52:0/1389042522 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:52.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.387+0000 7fcec6ffd640 1 --2- 172.21.15.52:0/1389042522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcec8068d00 0x7fcec81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.387+0000 7fcec67fc640 1 --2- 172.21.15.52:0/1389042522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcec80696c0 0x7fcec8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.387+0000 7fcec77fe640 1 -- 172.21.15.52:0/1389042522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcec8068d00 msgr2=0x7fcec81983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.387+0000 7fcec77fe640 1 --2- 172.21.15.52:0/1389042522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcec8068d00 0x7fcec81983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.387+0000 7fcec77fe640 1 -- 172.21.15.52:0/1389042522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcec80696c0 msgr2=0x7fcec8198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.387+0000 7fcec77fe640 1 --2- 172.21.15.52:0/1389042522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcec80696c0 0x7fcec8198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.387+0000 7fcec77fe640 1 -- 172.21.15.52:0/1389042522 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec819d370 con 0x7fcec8107b50 2024-04-02T13:56:52.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.387+0000 7fcec67fc640 1 --2- 172.21.15.52:0/1389042522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcec80696c0 0x7fcec8198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:56:52.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.388+0000 7fcec77fe640 1 --2- 172.21.15.52:0/1389042522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcec8107b50 0x7fcec819cc40 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fcec000eb10 tx=0x7fcec000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.388+0000 7fceaffff640 1 -- 172.21.15.52:0/1389042522 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec0004b20 con 0x7fcec8107b50 2024-04-02T13:56:52.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.388+0000 7fceaffff640 1 -- 172.21.15.52:0/1389042522 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcec0004cc0 con 0x7fcec8107b50 2024-04-02T13:56:52.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.388+0000 7fcecd59d640 1 -- 172.21.15.52:0/1389042522 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec819d660 con 0x7fcec8107b50 2024-04-02T13:56:52.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.388+0000 7fceaffff640 1 -- 172.21.15.52:0/1389042522 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec0010760 con 0x7fcec8107b50 2024-04-02T13:56:52.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.388+0000 7fcecd59d640 1 -- 172.21.15.52:0/1389042522 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec81a4ef0 con 0x7fcec8107b50 2024-04-02T13:56:52.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.390+0000 7fceaffff640 1 -- 172.21.15.52:0/1389042522 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcec0005020 con 0x7fcec8107b50 2024-04-02T13:56:52.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.390+0000 7fced0186740 1 Processor -- start 2024-04-02T13:56:52.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.390+0000 7fced0186740 1 -- start start 2024-04-02T13:56:52.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.390+0000 7fced0186740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede227a0 0x5586ede22b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.390+0000 7fced0186740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586ede230c0 0x5586ede2b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.390+0000 7fced0186740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586ede2bca0 0x5586ede2e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.390+0000 7fced0186740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5586edd867b0 con 0x5586ede227a0 2024-04-02T13:56:52.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.390+0000 7fced0186740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5586edd25570 con 0x5586ede230c0 2024-04-02T13:56:52.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.390+0000 7fced0186740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5586edd82760 con 0x5586ede2bca0 2024-04-02T13:56:52.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.390+0000 7fceaffff640 1 --2- 172.21.15.52:0/1389042522 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fceb00a3d20 0x7fceb00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.390+0000 7fcec67fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede227a0 0x5586ede22b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.390+0000 7fcec67fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede227a0 0x5586ede22b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51886/0 (socket says 172.21.15.52:51886) 2024-04-02T13:56:52.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.390+0000 7fcec67fc640 1 -- 172.21.15.52:0/3045046914 learned_addr learned my addr 172.21.15.52:0/3045046914 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:52.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.391+0000 7fcec6ffd640 1 --2- 172.21.15.52:0/3045046914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586ede230c0 0x5586ede2b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.391+0000 7fcec77fe640 1 --2- 172.21.15.52:0/1389042522 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fceb00a3d20 0x7fceb00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.391+0000 7fcec67fc640 1 --2- 172.21.15.52:0/3045046914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586ede2bca0 0x5586ede2e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.391+0000 7fceaffff640 1 -- 172.21.15.52:0/1389042522 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fcec00d11d0 con 0x7fcec8107b50 2024-04-02T13:56:52.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.391+0000 7fcec67fc640 1 -- 172.21.15.52:0/3045046914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586ede2bca0 msgr2=0x5586ede2e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.391+0000 7fcec67fc640 1 --2- 172.21.15.52:0/3045046914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586ede2bca0 0x5586ede2e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.391+0000 7fcec67fc640 1 -- 172.21.15.52:0/3045046914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586ede230c0 msgr2=0x5586ede2b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.391+0000 7fcec67fc640 1 --2- 172.21.15.52:0/3045046914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586ede230c0 0x5586ede2b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.391+0000 7fcec67fc640 1 -- 172.21.15.52:0/3045046914 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586eda9a3f0 con 0x5586ede227a0 2024-04-02T13:56:52.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.391+0000 7fcec6ffd640 1 --2- 172.21.15.52:0/3045046914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586ede230c0 0x5586ede2b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:52.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.391+0000 7fcec67fc640 1 --2- 172.21.15.52:0/3045046914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede227a0 0x5586ede22b80 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fcebc02efd0 tx=0x7fcebc005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc526c029d58989 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.391+0000 7fcec77fe640 1 --2- 172.21.15.52:0/1389042522 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fceb00a3d20 0x7fceb00a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcec0099230 tx=0x7fcec006b780 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.391+0000 7fcead7fa640 1 -- 172.21.15.52:0/3045046914 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc047070 con 0x5586ede227a0 2024-04-02T13:56:52.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.391+0000 7fcead7fa640 1 -- 172.21.15.52:0/3045046914 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcebc004400 con 0x5586ede227a0 2024-04-02T13:56:52.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.391+0000 7fcead7fa640 1 -- 172.21.15.52:0/3045046914 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc004920 con 0x5586ede227a0 2024-04-02T13:56:52.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.392+0000 7fced0186740 1 -- 172.21.15.52:0/3045046914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede227a0 msgr2=0x5586ede22b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.392+0000 7fced0186740 1 --2- 172.21.15.52:0/3045046914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede227a0 0x5586ede22b80 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fcebc02efd0 tx=0x7fcebc005b50 comp rx=0 tx=0).stop 2024-04-02T13:56:52.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.392+0000 7fced0186740 1 -- 172.21.15.52:0/3045046914 shutdown_connections 2024-04-02T13:56:52.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.392+0000 7fced0186740 1 --2- 172.21.15.52:0/3045046914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586ede2bca0 0x5586ede2e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.392+0000 7fced0186740 1 --2- 172.21.15.52:0/3045046914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586ede230c0 0x5586ede2b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.392+0000 7fced0186740 1 --2- 172.21.15.52:0/3045046914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede227a0 0x5586ede22b80 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.392+0000 7fced0186740 1 -- 172.21.15.52:0/3045046914 >> 172.21.15.52:0/3045046914 conn(0x5586ede1ced0 msgr2=0x5586ede1d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.392+0000 7fced0186740 1 -- 172.21.15.52:0/3045046914 shutdown_connections 2024-04-02T13:56:52.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.392+0000 7fced0186740 1 -- 172.21.15.52:0/3045046914 wait complete. 2024-04-02T13:56:52.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.392+0000 7fced0186740 1 -- start start 2024-04-02T13:56:52.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.392+0000 7fced0186740 1 Processor -- start 2024-04-02T13:56:52.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.392+0000 7fced0186740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede227a0 0x5586ede28b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.392+0000 7fced0186740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586ede230c0 0x5586ede29040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.392+0000 7fced0186740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586ede2bca0 0x5586ede29580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.392+0000 7fced0186740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5586ede2ea30 con 0x5586ede227a0 2024-04-02T13:56:52.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.392+0000 7fced0186740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5586edd41e80 con 0x5586ede2bca0 2024-04-02T13:56:52.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.392+0000 7fced0186740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5586ede2ebb0 con 0x5586ede230c0 2024-04-02T13:56:52.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.393+0000 7fced0186740 10 client.0 _ll_register_callbacks cb 0x5586edd7daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:52.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.393+0000 7fcec67fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede227a0 0x5586ede28b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.393+0000 7fcec67fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede227a0 0x5586ede28b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51894/0 (socket says 172.21.15.52:51894) 2024-04-02T13:56:52.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.393+0000 7fcec67fc640 1 -- 172.21.15.52:0/3236513536 learned_addr learned my addr 172.21.15.52:0/3236513536 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:52.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.393+0000 7fcec6ffd640 1 --2- 172.21.15.52:0/3236513536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586ede230c0 0x5586ede29040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.393+0000 7fcec67fc640 1 --2- 172.21.15.52:0/3236513536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586ede2bca0 0x5586ede29580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.393+0000 7fcec67fc640 1 -- 172.21.15.52:0/3236513536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586ede230c0 msgr2=0x5586ede29040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.393+0000 7fcec67fc640 1 --2- 172.21.15.52:0/3236513536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586ede230c0 0x5586ede29040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.393+0000 7fcec67fc640 1 -- 172.21.15.52:0/3236513536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586ede2bca0 msgr2=0x5586ede29580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.393+0000 7fcec67fc640 1 --2- 172.21.15.52:0/3236513536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586ede2bca0 0x5586ede29580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.393+0000 7fcec67fc640 1 -- 172.21.15.52:0/3236513536 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586edb647d0 con 0x5586ede227a0 2024-04-02T13:56:52.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.393+0000 7fcec67fc640 1 --2- 172.21.15.52:0/3236513536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede227a0 0x5586ede28b00 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fcebc004630 tx=0x7fcebc005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ccf82bc03e36424 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.393+0000 7fceaaff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:52.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.393+0000 7fceaaff5640 1 -- 172.21.15.52:0/3236513536 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc055060 con 0x5586ede227a0 2024-04-02T13:56:52.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.393+0000 7fceaaff5640 1 -- 172.21.15.52:0/3236513536 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fcebc003ab0 con 0x5586ede227a0 2024-04-02T13:56:52.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.393+0000 7fced0186740 1 -- 172.21.15.52:0/3236513536 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcebc004400 con 0x5586ede227a0 2024-04-02T13:56:52.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.393+0000 7fceaaff5640 1 -- 172.21.15.52:0/3236513536 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcebc05c050 con 0x5586ede227a0 2024-04-02T13:56:52.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.393+0000 7fced0186740 10 client.16596 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:52.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.393+0000 7fced0186740 1 -- 172.21.15.52:0/3236513536 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5586ede3ab00 con 0x5586ede227a0 2024-04-02T13:56:52.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.394+0000 7fceaaff5640 1 -- 172.21.15.52:0/3236513536 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcebc004400 con 0x5586ede227a0 2024-04-02T13:56:52.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.394+0000 7fceaaff5640 1 -- 172.21.15.52:0/3236513536 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fcebc034bc0 con 0x5586ede227a0 2024-04-02T13:56:52.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.394+0000 7fced0186740 10 client.16596 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:52.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.394+0000 7fced0186740 10 client.16596 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T13:56:52.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.394+0000 7fced0186740 20 client.16596 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:52.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.394+0000 7fced0186740 1 --2- 172.21.15.52:0/3236513536 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5586ede3be50 0x5586ede3e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.394+0000 7fced0186740 4 client.16596 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:52.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.394+0000 7fced0186740 1 -- 172.21.15.52:0/3236513536 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586edd93cf0 con 0x5586ede3be50 2024-04-02T13:56:52.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.394+0000 7fcec6ffd640 1 --2- 172.21.15.52:0/3236513536 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5586ede3be50 0x5586ede3e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.395+0000 7fceaaff5640 10 client.16596 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:56:52.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.395+0000 7fcec6ffd640 1 --2- 172.21.15.52:0/3236513536 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5586ede3be50 0x5586ede3e240 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=85e8ad82c60e441f server_cookie=4fb19d73eceb37e0 in_seq=0 out_seq=0 2024-04-02T13:56:52.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.395+0000 7fceaaff5640 1 -- 172.21.15.52:0/3236513536 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5586edd93cf0 con 0x5586ede3be50 2024-04-02T13:56:52.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.395+0000 7fceaaff5640 10 client.16596 handle_command_reply: tid=0 2024-04-02T13:56:52.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.395+0000 7fced0186740 1 client.16596 shutdown 2024-04-02T13:56:52.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.396+0000 7fced0186740 1 -- 172.21.15.52:0/3236513536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede227a0 msgr2=0x5586ede28b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.396+0000 7fced0186740 1 --2- 172.21.15.52:0/3236513536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede227a0 0x5586ede28b00 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fcebc004630 tx=0x7fcebc005520 comp rx=0 tx=0).stop 2024-04-02T13:56:52.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.396+0000 7fced0186740 1 -- 172.21.15.52:0/3236513536 shutdown_connections 2024-04-02T13:56:52.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.396+0000 7fced0186740 1 --2- 172.21.15.52:0/3236513536 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5586ede3be50 0x5586ede3e240 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.396+0000 7fced0186740 1 --2- 172.21.15.52:0/3236513536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586ede2bca0 0x5586ede29580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.396+0000 7fced0186740 1 --2- 172.21.15.52:0/3236513536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586ede230c0 0x5586ede29040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.396+0000 7fceaaff5640 0 client.16596 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:56:52.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.396+0000 7fced0186740 1 --2- 172.21.15.52:0/3236513536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede227a0 0x5586ede28b00 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.396+0000 7fced0186740 1 -- 172.21.15.52:0/3236513536 >> 172.21.15.52:0/3236513536 conn(0x5586ede1ced0 msgr2=0x5586ede1ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.396+0000 7fced0186740 1 -- 172.21.15.52:0/3236513536 shutdown_connections 2024-04-02T13:56:52.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.396+0000 7fced0186740 1 -- 172.21.15.52:0/3236513536 wait complete. 2024-04-02T13:56:52.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.396+0000 7fced0186740 20 client.16596 trim_cache size 0 max 16384 2024-04-02T13:56:52.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.409+0000 7fced0186740 1 Processor -- start 2024-04-02T13:56:52.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.409+0000 7fced0186740 1 -- start start 2024-04-02T13:56:52.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.409+0000 7fced0186740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586ede227a0 0x5586ede2b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.409+0000 7fced0186740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586ede230c0 0x5586ede27c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.409+0000 7fced0186740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede2bca0 0x5586ede37510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.409+0000 7fced0186740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcec81aedf0 con 0x5586ede2bca0 2024-04-02T13:56:52.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.409+0000 7fced0186740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcec81b0960 con 0x5586ede227a0 2024-04-02T13:56:52.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.409+0000 7fced0186740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcec81af4c0 con 0x5586ede230c0 2024-04-02T13:56:52.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.409+0000 7fcec67fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede2bca0 0x5586ede37510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.409+0000 7fcec67fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede2bca0 0x5586ede37510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51900/0 (socket says 172.21.15.52:51900) 2024-04-02T13:56:52.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.409+0000 7fcec67fc640 1 -- 172.21.15.52:0/3634488587 learned_addr learned my addr 172.21.15.52:0/3634488587 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:52.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.409+0000 7fcec67fc640 1 --2- 172.21.15.52:0/3634488587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586ede227a0 0x5586ede2b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.410+0000 7fcec6ffd640 1 --2- 172.21.15.52:0/3634488587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586ede230c0 0x5586ede27c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.410+0000 7fcec67fc640 1 -- 172.21.15.52:0/3634488587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586ede230c0 msgr2=0x5586ede27c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.410+0000 7fcec67fc640 1 --2- 172.21.15.52:0/3634488587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586ede230c0 0x5586ede27c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.410+0000 7fcec67fc640 1 -- 172.21.15.52:0/3634488587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586ede227a0 msgr2=0x5586ede2b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.410+0000 7fcec67fc640 1 --2- 172.21.15.52:0/3634488587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586ede227a0 0x5586ede2b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.410+0000 7fcec67fc640 1 -- 172.21.15.52:0/3634488587 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586ede22ef0 con 0x5586ede2bca0 2024-04-02T13:56:52.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.410+0000 7fcec67fc640 1 --2- 172.21.15.52:0/3634488587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede2bca0 0x5586ede37510 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0424b0 tx=0x7fcebc005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b5d9c26302e04c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.410+0000 7fceacff9640 1 -- 172.21.15.52:0/3634488587 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc055050 con 0x5586ede2bca0 2024-04-02T13:56:52.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.410+0000 7fceacff9640 1 -- 172.21.15.52:0/3634488587 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcebc042d90 con 0x5586ede2bca0 2024-04-02T13:56:52.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.410+0000 7fceacff9640 1 -- 172.21.15.52:0/3634488587 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc047070 con 0x5586ede2bca0 2024-04-02T13:56:52.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.410+0000 7fced0186740 1 -- 172.21.15.52:0/3634488587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede2bca0 msgr2=0x5586ede37510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.410+0000 7fced0186740 1 --2- 172.21.15.52:0/3634488587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede2bca0 0x5586ede37510 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0424b0 tx=0x7fcebc005a20 comp rx=0 tx=0).stop 2024-04-02T13:56:52.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.410+0000 7fced0186740 1 -- 172.21.15.52:0/3634488587 shutdown_connections 2024-04-02T13:56:52.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.410+0000 7fced0186740 1 --2- 172.21.15.52:0/3634488587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede2bca0 0x5586ede37510 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.410+0000 7fced0186740 1 --2- 172.21.15.52:0/3634488587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586ede230c0 0x5586ede27c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.411+0000 7fced0186740 1 --2- 172.21.15.52:0/3634488587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586ede227a0 0x5586ede2b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.411+0000 7fced0186740 1 -- 172.21.15.52:0/3634488587 >> 172.21.15.52:0/3634488587 conn(0x5586ede1ced0 msgr2=0x5586ede36e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.411+0000 7fced0186740 1 -- 172.21.15.52:0/3634488587 shutdown_connections 2024-04-02T13:56:52.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.411+0000 7fced0186740 1 -- 172.21.15.52:0/3634488587 wait complete. 2024-04-02T13:56:52.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.411+0000 7fced0186740 1 -- start start 2024-04-02T13:56:52.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.411+0000 7fced0186740 1 Processor -- start 2024-04-02T13:56:52.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.411+0000 7fced0186740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede227a0 0x5586ede41710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.411+0000 7fced0186740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586ede230c0 0x5586ede3eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.411+0000 7fced0186740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586ede2bca0 0x5586ede3f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.411+0000 7fced0186740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcec81b0260 con 0x5586ede227a0 2024-04-02T13:56:52.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.411+0000 7fced0186740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcec81afb90 con 0x5586ede2bca0 2024-04-02T13:56:52.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.411+0000 7fced0186740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcec81acdd0 con 0x5586ede230c0 2024-04-02T13:56:52.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.411+0000 7fced0186740 10 client.0 _ll_register_callbacks cb 0x7fcebc045070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:52.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.411+0000 7fcec67fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede227a0 0x5586ede41710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.411+0000 7fcec67fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede227a0 0x5586ede41710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51910/0 (socket says 172.21.15.52:51910) 2024-04-02T13:56:52.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.411+0000 7fcec67fc640 1 -- 172.21.15.52:0/1084626983 learned_addr learned my addr 172.21.15.52:0/1084626983 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:52.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.411+0000 7fcec67fc640 1 -- 172.21.15.52:0/1084626983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586ede230c0 msgr2=0x5586ede3eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.411+0000 7fcec6ffd640 1 --2- 172.21.15.52:0/1084626983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586ede230c0 0x5586ede3eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.411+0000 7fcec67fc640 1 --2- 172.21.15.52:0/1084626983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586ede230c0 0x5586ede3eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.411+0000 7fcec67fc640 1 -- 172.21.15.52:0/1084626983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586ede2bca0 msgr2=0x5586ede3f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.411+0000 7fcec67fc640 1 --2- 172.21.15.52:0/1084626983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586ede2bca0 0x5586ede3f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.412+0000 7fcec67fc640 1 -- 172.21.15.52:0/1084626983 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586ede3f880 con 0x5586ede227a0 2024-04-02T13:56:52.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.412+0000 7fcec6ffd640 1 --2- 172.21.15.52:0/1084626983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586ede230c0 0x5586ede3eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:56:52.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.412+0000 7fcec67fc640 1 --2- 172.21.15.52:0/1084626983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede227a0 0x5586ede41710 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fcebc005c40 tx=0x7fcebc02c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e5fbbfd04f1019 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.412+0000 7fceaaff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:52.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.412+0000 7fceaaff5640 1 -- 172.21.15.52:0/1084626983 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc064030 con 0x5586ede227a0 2024-04-02T13:56:52.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.412+0000 7fceaaff5640 1 -- 172.21.15.52:0/1084626983 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fcebc0427e0 con 0x5586ede227a0 2024-04-02T13:56:52.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.412+0000 7fced0186740 1 -- 172.21.15.52:0/1084626983 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcebc042d90 con 0x5586ede227a0 2024-04-02T13:56:52.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.412+0000 7fceaaff5640 1 -- 172.21.15.52:0/1084626983 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcebc06e050 con 0x5586ede227a0 2024-04-02T13:56:52.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.412+0000 7fced0186740 10 client.16602 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:52.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.412+0000 7fced0186740 1 -- 172.21.15.52:0/1084626983 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5586edea2100 con 0x5586ede227a0 2024-04-02T13:56:52.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.412+0000 7fceaaff5640 1 -- 172.21.15.52:0/1084626983 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcebc042d90 con 0x5586ede227a0 2024-04-02T13:56:52.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.412+0000 7fceaaff5640 1 -- 172.21.15.52:0/1084626983 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fcebc004010 con 0x5586ede227a0 2024-04-02T13:56:52.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.412+0000 7fced0186740 10 client.16602 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:52.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.412+0000 7fced0186740 10 client.16602 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T13:56:52.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.412+0000 7fced0186740 20 client.16602 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:52.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.412+0000 7fced0186740 1 --2- 172.21.15.52:0/1084626983 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5586edea2840 0x5586edea4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.412+0000 7fced0186740 4 client.16602 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:56:52.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.412+0000 7fced0186740 1 -- 172.21.15.52:0/1084626983 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586edd8e040 con 0x5586edea2840 2024-04-02T13:56:52.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.413+0000 7fcec6ffd640 1 --2- 172.21.15.52:0/1084626983 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5586edea2840 0x5586edea4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.413+0000 7fcec6ffd640 1 --2- 172.21.15.52:0/1084626983 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5586edea2840 0x5586edea4c60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6c9c836b7c80bde server_cookie=6eeb52a87b53d83a in_seq=0 out_seq=0 2024-04-02T13:56:52.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.413+0000 7fceaaff5640 10 client.16602 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:56:52.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.413+0000 7fceaaff5640 1 -- 172.21.15.52:0/1084626983 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586edd8e040 con 0x5586edea2840 2024-04-02T13:56:52.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.413+0000 7fceaaff5640 10 client.16602 handle_command_reply: tid=0 2024-04-02T13:56:52.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.413+0000 7fced0186740 1 client.16602 shutdown 2024-04-02T13:56:52.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.413+0000 7fced0186740 1 -- 172.21.15.52:0/1084626983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede227a0 msgr2=0x5586ede41710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.413+0000 7fced0186740 1 --2- 172.21.15.52:0/1084626983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede227a0 0x5586ede41710 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fcebc005c40 tx=0x7fcebc02c370 comp rx=0 tx=0).stop 2024-04-02T13:56:52.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.413+0000 7fced0186740 1 -- 172.21.15.52:0/1084626983 shutdown_connections 2024-04-02T13:56:52.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.413+0000 7fced0186740 1 --2- 172.21.15.52:0/1084626983 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5586edea2840 0x5586edea4c60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.413+0000 7fced0186740 1 --2- 172.21.15.52:0/1084626983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586ede2bca0 0x5586ede3f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.413+0000 7fced0186740 1 --2- 172.21.15.52:0/1084626983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586ede230c0 0x5586ede3eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.413+0000 7fced0186740 1 --2- 172.21.15.52:0/1084626983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586ede227a0 0x5586ede41710 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.413+0000 7fceaaff5640 0 client.16602 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:56:52.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.413+0000 7fced0186740 1 -- 172.21.15.52:0/1084626983 >> 172.21.15.52:0/1084626983 conn(0x5586ede1ced0 msgr2=0x5586ede37690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.413+0000 7fced0186740 1 -- 172.21.15.52:0/1084626983 shutdown_connections 2024-04-02T13:56:52.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.413+0000 7fced0186740 1 -- 172.21.15.52:0/1084626983 wait complete. 2024-04-02T13:56:52.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.414+0000 7fced0186740 20 client.16602 trim_cache size 0 max 16384 2024-04-02T13:56:52.429 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:56:52.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.414+0000 7fcecd59d640 1 -- 172.21.15.52:0/1389042522 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fceb00a3d20 msgr2=0x7fceb00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.414+0000 7fcecd59d640 1 --2- 172.21.15.52:0/1389042522 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fceb00a3d20 0x7fceb00a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcec0099230 tx=0x7fcec006b780 comp rx=0 tx=0).stop 2024-04-02T13:56:52.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.414+0000 7fcecd59d640 1 -- 172.21.15.52:0/1389042522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcec8107b50 msgr2=0x7fcec819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.414+0000 7fcecd59d640 1 --2- 172.21.15.52:0/1389042522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcec8107b50 0x7fcec819cc40 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fcec000eb10 tx=0x7fcec000c2d0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.414+0000 7fcecd59d640 1 -- 172.21.15.52:0/1389042522 shutdown_connections 2024-04-02T13:56:52.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.414+0000 7fcecd59d640 1 --2- 172.21.15.52:0/1389042522 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fceb00a3d20 0x7fceb00a61e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.414+0000 7fcecd59d640 1 --2- 172.21.15.52:0/1389042522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcec8107b50 0x7fcec819cc40 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.414+0000 7fcecd59d640 1 --2- 172.21.15.52:0/1389042522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcec80696c0 0x7fcec8198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.414+0000 7fcecd59d640 1 --2- 172.21.15.52:0/1389042522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcec8068d00 0x7fcec81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.414+0000 7fcecd59d640 1 -- 172.21.15.52:0/1389042522 >> 172.21.15.52:0/1389042522 conn(0x7fcec8077b00 msgr2=0x7fcec80fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.414+0000 7fcecd59d640 1 -- 172.21.15.52:0/1389042522 shutdown_connections 2024-04-02T13:56:52.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.414+0000 7fcecd59d640 1 -- 172.21.15.52:0/1389042522 wait complete. 2024-04-02T13:56:52.432 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:56:52.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.534+0000 7fc2fc4f2640 1 -- 172.21.15.52:0/1999696066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2f4102d70 msgr2=0x7fc2f4103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.534+0000 7fc2fc4f2640 1 --2- 172.21.15.52:0/1999696066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2f4102d70 0x7fc2f4103150 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009900 tx=0x7fc2ec02bcd0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.534+0000 7fc2fc4f2640 1 -- 172.21.15.52:0/1999696066 shutdown_connections 2024-04-02T13:56:52.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.534+0000 7fc2fc4f2640 1 --2- 172.21.15.52:0/1999696066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2f410c140 0x7fc2f410e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.534+0000 7fc2fc4f2640 1 --2- 172.21.15.52:0/1999696066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2f4103690 0x7fc2f410bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.534+0000 7fc2fc4f2640 1 --2- 172.21.15.52:0/1999696066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2f4102d70 0x7fc2f4103150 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.534+0000 7fc2fc4f2640 1 -- 172.21.15.52:0/1999696066 >> 172.21.15.52:0/1999696066 conn(0x7fc2f4077b60 msgr2=0x7fc2f4077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.534+0000 7fc2fc4f2640 1 -- 172.21.15.52:0/1999696066 shutdown_connections 2024-04-02T13:56:52.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.534+0000 7fc2fc4f2640 1 -- 172.21.15.52:0/1999696066 wait complete. 2024-04-02T13:56:52.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.535+0000 7fc2fc4f2640 1 Processor -- start 2024-04-02T13:56:52.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.535+0000 7fc2fc4f2640 1 -- start start 2024-04-02T13:56:52.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.535+0000 7fc2fc4f2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2f4102d70 0x7fc2f4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.535+0000 7fc2fc4f2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2f4103690 0x7fc2f41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.535+0000 7fc2fc4f2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2f410c140 0x7fc2f419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.535+0000 7fc2fc4f2640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f4111f10 con 0x7fc2f4102d70 2024-04-02T13:56:52.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.535+0000 7fc2fc4f2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f4111d90 con 0x7fc2f410c140 2024-04-02T13:56:52.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.535+0000 7fc2fc4f2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f4112090 con 0x7fc2f4103690 2024-04-02T13:56:52.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.535+0000 7fc2fa28e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2f4102d70 0x7fc2f4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.535+0000 7fc2fa28e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2f4102d70 0x7fc2f4198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51940/0 (socket says 172.21.15.52:51940) 2024-04-02T13:56:52.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.535+0000 7fc2fa28e640 1 -- 172.21.15.52:0/3668843736 learned_addr learned my addr 172.21.15.52:0/3668843736 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:52.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.535+0000 7fc2f9a8d640 1 --2- 172.21.15.52:0/3668843736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2f4103690 0x7fc2f41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.536+0000 7fc2fa28e640 1 -- 172.21.15.52:0/3668843736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2f4103690 msgr2=0x7fc2f41988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.536+0000 7fc2fa28e640 1 --2- 172.21.15.52:0/3668843736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2f4103690 0x7fc2f41988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.536+0000 7fc2fa28e640 1 -- 172.21.15.52:0/3668843736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2f410c140 msgr2=0x7fc2f419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.536+0000 7fc2fa28e640 1 --2- 172.21.15.52:0/3668843736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2f410c140 0x7fc2f419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.536+0000 7fc2fa28e640 1 -- 172.21.15.52:0/3668843736 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f419d360 con 0x7fc2f4102d70 2024-04-02T13:56:52.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.536+0000 7fc2fa28e640 1 --2- 172.21.15.52:0/3668843736 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2f4102d70 0x7fc2f4198370 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec005940 tx=0x7fc2ec02cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.536+0000 7fc2eb7fe640 1 -- 172.21.15.52:0/3668843736 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec041070 con 0x7fc2f4102d70 2024-04-02T13:56:52.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.536+0000 7fc2fc4f2640 1 -- 172.21.15.52:0/3668843736 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f419d5f0 con 0x7fc2f4102d70 2024-04-02T13:56:52.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.536+0000 7fc2eb7fe640 1 -- 172.21.15.52:0/3668843736 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2ec004450 con 0x7fc2f4102d70 2024-04-02T13:56:52.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.536+0000 7fc2eb7fe640 1 -- 172.21.15.52:0/3668843736 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec034440 con 0x7fc2f4102d70 2024-04-02T13:56:52.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.536+0000 7fc2fc4f2640 1 -- 172.21.15.52:0/3668843736 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f41a4ee0 con 0x7fc2f4102d70 2024-04-02T13:56:52.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.538+0000 7fc2eb7fe640 1 -- 172.21.15.52:0/3668843736 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc2ec044070 con 0x7fc2f4102d70 2024-04-02T13:56:52.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.539+0000 7fc2ff1b4740 1 Processor -- start 2024-04-02T13:56:52.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.539+0000 7fc2ff1b4740 1 -- start start 2024-04-02T13:56:52.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.539+0000 7fc2ff1b4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c142ee7a0 0x561c142eeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.539+0000 7fc2ff1b4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c142ef0c0 0x561c142f7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.539+0000 7fc2eb7fe640 1 --2- 172.21.15.52:0/3668843736 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc2e00a3d20 0x7fc2e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.539+0000 7fc2ff1b4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142f7ca0 0x561c142fa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.539+0000 7fc2ff1b4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561c142527b0 con 0x561c142f7ca0 2024-04-02T13:56:52.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.539+0000 7fc2ff1b4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x561c141f1570 con 0x561c142ee7a0 2024-04-02T13:56:52.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.539+0000 7fc2ff1b4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561c1424e760 con 0x561c142ef0c0 2024-04-02T13:56:52.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.539+0000 7fc2f9a8d640 1 --2- 172.21.15.52:0/3668843736 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc2e00a3d20 0x7fc2e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.539+0000 7fc2faa8f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142f7ca0 0x561c142fa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.539+0000 7fc2faa8f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142f7ca0 0x561c142fa090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51952/0 (socket says 172.21.15.52:51952) 2024-04-02T13:56:52.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.539+0000 7fc2faa8f640 1 -- 172.21.15.52:0/1604874123 learned_addr learned my addr 172.21.15.52:0/1604874123 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:52.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.539+0000 7fc2faa8f640 1 --2- 172.21.15.52:0/1604874123 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c142ee7a0 0x561c142eeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.539+0000 7fc2eb7fe640 1 -- 172.21.15.52:0/3668843736 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc2ec0f2780 con 0x7fc2f4102d70 2024-04-02T13:56:52.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.539+0000 7fc2fa28e640 1 --2- 172.21.15.52:0/1604874123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c142ef0c0 0x561c142f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.540+0000 7fc2faa8f640 1 -- 172.21.15.52:0/1604874123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c142ef0c0 msgr2=0x561c142f7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.540+0000 7fc2faa8f640 1 --2- 172.21.15.52:0/1604874123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c142ef0c0 0x561c142f7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.540+0000 7fc2faa8f640 1 -- 172.21.15.52:0/1604874123 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c142ee7a0 msgr2=0x561c142eeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.540+0000 7fc2faa8f640 1 --2- 172.21.15.52:0/1604874123 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c142ee7a0 0x561c142eeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.540+0000 7fc2faa8f640 1 -- 172.21.15.52:0/1604874123 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c13f663f0 con 0x561c142f7ca0 2024-04-02T13:56:52.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.540+0000 7fc2f9a8d640 1 --2- 172.21.15.52:0/3668843736 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc2e00a3d20 0x7fc2e00a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4005fd0 tx=0x7fc2e4005850 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.540+0000 7fc2faa8f640 1 --2- 172.21.15.52:0/1604874123 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142f7ca0 0x561c142fa090 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0013a20 tx=0x7fc2f0009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39245b841f57ffb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.540+0000 7fc2e8bfe640 1 -- 172.21.15.52:0/1604874123 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0004d20 con 0x561c142f7ca0 2024-04-02T13:56:52.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.540+0000 7fc2e8bfe640 1 -- 172.21.15.52:0/1604874123 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f0003e50 con 0x561c142f7ca0 2024-04-02T13:56:52.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.540+0000 7fc2e8bfe640 1 -- 172.21.15.52:0/1604874123 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0004280 con 0x561c142f7ca0 2024-04-02T13:56:52.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.541+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/1604874123 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142f7ca0 msgr2=0x561c142fa090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.541+0000 7fc2ff1b4740 1 --2- 172.21.15.52:0/1604874123 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142f7ca0 0x561c142fa090 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0013a20 tx=0x7fc2f0009590 comp rx=0 tx=0).stop 2024-04-02T13:56:52.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.541+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/1604874123 shutdown_connections 2024-04-02T13:56:52.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.541+0000 7fc2ff1b4740 1 --2- 172.21.15.52:0/1604874123 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142f7ca0 0x561c142fa090 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.541+0000 7fc2ff1b4740 1 --2- 172.21.15.52:0/1604874123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c142ef0c0 0x561c142f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.541+0000 7fc2ff1b4740 1 --2- 172.21.15.52:0/1604874123 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c142ee7a0 0x561c142eeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.541+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/1604874123 >> 172.21.15.52:0/1604874123 conn(0x561c142e8ed0 msgr2=0x561c142e92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.541+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/1604874123 shutdown_connections 2024-04-02T13:56:52.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.541+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/1604874123 wait complete. 2024-04-02T13:56:52.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.541+0000 7fc2ff1b4740 1 -- start start 2024-04-02T13:56:52.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.541+0000 7fc2ff1b4740 1 Processor -- start 2024-04-02T13:56:52.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.541+0000 7fc2ff1b4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142ee7a0 0x561c142f7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.542+0000 7fc2ff1b4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c142ef0c0 0x561c142f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.542+0000 7fc2ff1b4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c142f7ca0 0x561c142f5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.542+0000 7fc2ff1b4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561c142faa30 con 0x561c142ee7a0 2024-04-02T13:56:52.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.542+0000 7fc2ff1b4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x561c1420de80 con 0x561c142ef0c0 2024-04-02T13:56:52.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.542+0000 7fc2ff1b4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561c142fabb0 con 0x561c142f7ca0 2024-04-02T13:56:52.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.542+0000 7fc2ff1b4740 10 client.0 _ll_register_callbacks cb 0x561c14249aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:52.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.542+0000 7fc2fa28e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142ee7a0 0x561c142f7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.542+0000 7fc2fa28e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142ee7a0 0x561c142f7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51964/0 (socket says 172.21.15.52:51964) 2024-04-02T13:56:52.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.542+0000 7fc2fa28e640 1 -- 172.21.15.52:0/1079430801 learned_addr learned my addr 172.21.15.52:0/1079430801 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:52.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.542+0000 7fc2faa8f640 1 --2- 172.21.15.52:0/1079430801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c142f7ca0 0x561c142f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.542+0000 7fc2f9a8d640 1 --2- 172.21.15.52:0/1079430801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c142ef0c0 0x561c142f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.542+0000 7fc2fa28e640 1 -- 172.21.15.52:0/1079430801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c142f7ca0 msgr2=0x561c142f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.542+0000 7fc2fa28e640 1 --2- 172.21.15.52:0/1079430801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c142f7ca0 0x561c142f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.542+0000 7fc2fa28e640 1 -- 172.21.15.52:0/1079430801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c142ef0c0 msgr2=0x561c142f4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.542+0000 7fc2fa28e640 1 --2- 172.21.15.52:0/1079430801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c142ef0c0 0x561c142f4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.542+0000 7fc2fa28e640 1 -- 172.21.15.52:0/1079430801 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c140307d0 con 0x561c142ee7a0 2024-04-02T13:56:52.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.542+0000 7fc2fa28e640 1 --2- 172.21.15.52:0/1079430801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142ee7a0 0x561c142f7750 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec04c000 tx=0x7fc2ec0be320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f7673efb596116 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.542+0000 7fc2ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:52.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.542+0000 7fc2ddffb640 1 -- 172.21.15.52:0/1079430801 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec04b040 con 0x561c142ee7a0 2024-04-02T13:56:52.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.542+0000 7fc2ddffb640 1 -- 172.21.15.52:0/1079430801 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc2ec0be6f0 con 0x561c142ee7a0 2024-04-02T13:56:52.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.542+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/1079430801 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f0003e50 con 0x561c142ee7a0 2024-04-02T13:56:52.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.542+0000 7fc2ddffb640 1 -- 172.21.15.52:0/1079430801 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc2ec0effb0 con 0x561c142ee7a0 2024-04-02T13:56:52.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.542+0000 7fc2ff1b4740 10 client.16614 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:52.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.542+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/1079430801 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561c14306ae0 con 0x561c142ee7a0 2024-04-02T13:56:52.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.543+0000 7fc2ddffb640 1 -- 172.21.15.52:0/1079430801 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f0003e50 con 0x561c142ee7a0 2024-04-02T13:56:52.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.543+0000 7fc2ddffb640 1 -- 172.21.15.52:0/1079430801 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fc2ec0c6c70 con 0x561c142ee7a0 2024-04-02T13:56:52.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.543+0000 7fc2ff1b4740 10 client.16614 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:52.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.543+0000 7fc2ff1b4740 10 client.16614 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T13:56:52.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.543+0000 7fc2ff1b4740 20 client.16614 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:52.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.543+0000 7fc2ff1b4740 1 --2- 172.21.15.52:0/1079430801 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x561c14307f90 0x561c1430a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.543+0000 7fc2ff1b4740 4 client.16614 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:52.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.543+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/1079430801 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c1425fcf0 con 0x561c14307f90 2024-04-02T13:56:52.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.543+0000 7fc2f9a8d640 1 --2- 172.21.15.52:0/1079430801 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x561c14307f90 0x561c1430a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.544+0000 7fc2ddffb640 10 client.16614 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:56:52.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.544+0000 7fc2f9a8d640 1 --2- 172.21.15.52:0/1079430801 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x561c14307f90 0x561c1430a400 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d084cc7c3b0f9a8 server_cookie=ae403db8be2a00c2 in_seq=0 out_seq=0 2024-04-02T13:56:52.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.544+0000 7fc2ddffb640 1 -- 172.21.15.52:0/1079430801 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561c1425fcf0 con 0x561c14307f90 2024-04-02T13:56:52.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.544+0000 7fc2ddffb640 10 client.16614 handle_command_reply: tid=0 2024-04-02T13:56:52.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.544+0000 7fc2ff1b4740 1 client.16614 shutdown 2024-04-02T13:56:52.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.544+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/1079430801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142ee7a0 msgr2=0x561c142f7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.544+0000 7fc2ff1b4740 1 --2- 172.21.15.52:0/1079430801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142ee7a0 0x561c142f7750 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec04c000 tx=0x7fc2ec0be320 comp rx=0 tx=0).stop 2024-04-02T13:56:52.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.544+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/1079430801 shutdown_connections 2024-04-02T13:56:52.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.544+0000 7fc2ff1b4740 1 --2- 172.21.15.52:0/1079430801 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x561c14307f90 0x561c1430a400 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.544+0000 7fc2ff1b4740 1 --2- 172.21.15.52:0/1079430801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c142f7ca0 0x561c142f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.545+0000 7fc2ff1b4740 1 --2- 172.21.15.52:0/1079430801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c142ef0c0 0x561c142f4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.545+0000 7fc2ff1b4740 1 --2- 172.21.15.52:0/1079430801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142ee7a0 0x561c142f7750 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.545+0000 7fc2ddffb640 0 client.16614 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:56:52.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.545+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/1079430801 >> 172.21.15.52:0/1079430801 conn(0x561c142e8ed0 msgr2=0x561c142eac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.545+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/1079430801 shutdown_connections 2024-04-02T13:56:52.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.545+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/1079430801 wait complete. 2024-04-02T13:56:52.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.545+0000 7fc2ff1b4740 20 client.16614 trim_cache size 0 max 16384 2024-04-02T13:56:52.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.558+0000 7fc2ff1b4740 1 Processor -- start 2024-04-02T13:56:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.558+0000 7fc2ff1b4740 1 -- start start 2024-04-02T13:56:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.558+0000 7fc2ff1b4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c142ee7a0 0x561c143038f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.558+0000 7fc2ff1b4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142ef0c0 0x561c14304230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.558+0000 7fc2ff1b4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c142f7ca0 0x561c142f7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.558+0000 7fc2ff1b4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f41aedb0 con 0x561c142ef0c0 2024-04-02T13:56:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.558+0000 7fc2ff1b4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f41b0920 con 0x561c142ee7a0 2024-04-02T13:56:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.558+0000 7fc2ff1b4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f41af480 con 0x561c142f7ca0 2024-04-02T13:56:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.558+0000 7fc2f9a8d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142ef0c0 0x561c14304230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.558+0000 7fc2f9a8d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142ef0c0 0x561c14304230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51970/0 (socket says 172.21.15.52:51970) 2024-04-02T13:56:52.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.559+0000 7fc2f9a8d640 1 -- 172.21.15.52:0/789487295 learned_addr learned my addr 172.21.15.52:0/789487295 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:52.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.559+0000 7fc2fa28e640 1 --2- 172.21.15.52:0/789487295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c142ee7a0 0x561c143038f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.559+0000 7fc2faa8f640 1 --2- 172.21.15.52:0/789487295 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c142f7ca0 0x561c142f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.559+0000 7fc2f9a8d640 1 -- 172.21.15.52:0/789487295 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c142f7ca0 msgr2=0x561c142f7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.559+0000 7fc2f9a8d640 1 --2- 172.21.15.52:0/789487295 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c142f7ca0 0x561c142f7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.559+0000 7fc2f9a8d640 1 -- 172.21.15.52:0/789487295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c142ee7a0 msgr2=0x561c143038f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.559+0000 7fc2f9a8d640 1 --2- 172.21.15.52:0/789487295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c142ee7a0 0x561c143038f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.559+0000 7fc2f9a8d640 1 -- 172.21.15.52:0/789487295 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c142f5590 con 0x561c142ef0c0 2024-04-02T13:56:52.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.559+0000 7fc2f9a8d640 1 --2- 172.21.15.52:0/789487295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142ef0c0 0x561c14304230 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc2e40058c0 tx=0x7fc2e4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446df3c98b89e189 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.559+0000 7fc2dffff640 1 -- 172.21.15.52:0/789487295 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4020a10 con 0x561c142ef0c0 2024-04-02T13:56:52.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.559+0000 7fc2dffff640 1 -- 172.21.15.52:0/789487295 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2e4026730 con 0x561c142ef0c0 2024-04-02T13:56:52.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.559+0000 7fc2dffff640 1 -- 172.21.15.52:0/789487295 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4011a00 con 0x561c142ef0c0 2024-04-02T13:56:52.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.560+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/789487295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142ef0c0 msgr2=0x561c14304230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.560+0000 7fc2ff1b4740 1 --2- 172.21.15.52:0/789487295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142ef0c0 0x561c14304230 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc2e40058c0 tx=0x7fc2e4015000 comp rx=0 tx=0).stop 2024-04-02T13:56:52.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.560+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/789487295 shutdown_connections 2024-04-02T13:56:52.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.560+0000 7fc2ff1b4740 1 --2- 172.21.15.52:0/789487295 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c142f7ca0 0x561c142f7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.560+0000 7fc2ff1b4740 1 --2- 172.21.15.52:0/789487295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142ef0c0 0x561c14304230 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.560+0000 7fc2ff1b4740 1 --2- 172.21.15.52:0/789487295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c142ee7a0 0x561c143038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.560+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/789487295 >> 172.21.15.52:0/789487295 conn(0x561c142e8ed0 msgr2=0x561c142f33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.560+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/789487295 shutdown_connections 2024-04-02T13:56:52.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.560+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/789487295 wait complete. 2024-04-02T13:56:52.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.560+0000 7fc2ff1b4740 1 -- start start 2024-04-02T13:56:52.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.560+0000 7fc2ff1b4740 1 Processor -- start 2024-04-02T13:56:52.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.561+0000 7fc2ff1b4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c142ee7a0 0x561c142f1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.561+0000 7fc2ff1b4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142ef0c0 0x561c142f1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.561+0000 7fc2ff1b4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c142f7ca0 0x561c142fee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.561+0000 7fc2ff1b4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f41b0220 con 0x561c142ef0c0 2024-04-02T13:56:52.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.561+0000 7fc2ff1b4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f41afb50 con 0x561c142f7ca0 2024-04-02T13:56:52.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.561+0000 7fc2ff1b4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f41acd90 con 0x561c142ee7a0 2024-04-02T13:56:52.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.561+0000 7fc2ff1b4740 10 client.0 _ll_register_callbacks cb 0x7fc2ec009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:52.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.561+0000 7fc2f9a8d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142ef0c0 0x561c142f1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.561+0000 7fc2f9a8d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142ef0c0 0x561c142f1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51980/0 (socket says 172.21.15.52:51980) 2024-04-02T13:56:52.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.561+0000 7fc2f9a8d640 1 -- 172.21.15.52:0/577064658 learned_addr learned my addr 172.21.15.52:0/577064658 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:52.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.561+0000 7fc2fa28e640 1 --2- 172.21.15.52:0/577064658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c142ee7a0 0x561c142f1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.561+0000 7fc2faa8f640 1 --2- 172.21.15.52:0/577064658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c142f7ca0 0x561c142fee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.561+0000 7fc2f9a8d640 1 -- 172.21.15.52:0/577064658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c142ee7a0 msgr2=0x561c142f1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.561+0000 7fc2f9a8d640 1 --2- 172.21.15.52:0/577064658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c142ee7a0 0x561c142f1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.561+0000 7fc2f9a8d640 1 -- 172.21.15.52:0/577064658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c142f7ca0 msgr2=0x561c142fee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.561+0000 7fc2f9a8d640 1 --2- 172.21.15.52:0/577064658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c142f7ca0 0x561c142fee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.561+0000 7fc2f9a8d640 1 -- 172.21.15.52:0/577064658 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c142ff440 con 0x561c142ef0c0 2024-04-02T13:56:52.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.561+0000 7fc2f9a8d640 1 --2- 172.21.15.52:0/577064658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142ef0c0 0x561c142f1a60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400c010 tx=0x7fc2e400fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2327dd428fbfa415 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.561+0000 7fc2ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:52.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.563+0000 7fc2ddffb640 1 -- 172.21.15.52:0/577064658 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4020e20 con 0x561c142ef0c0 2024-04-02T13:56:52.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.563+0000 7fc2ddffb640 1 -- 172.21.15.52:0/577064658 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc2e4026050 con 0x561c142ef0c0 2024-04-02T13:56:52.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.563+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/577064658 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2e4026730 con 0x561c142ef0c0 2024-04-02T13:56:52.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.563+0000 7fc2ddffb640 1 -- 172.21.15.52:0/577064658 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc2e4040050 con 0x561c142ef0c0 2024-04-02T13:56:52.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.563+0000 7fc2ff1b4740 10 client.16620 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:52.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.563+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/577064658 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561c1436e580 con 0x561c142ef0c0 2024-04-02T13:56:52.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.563+0000 7fc2ddffb640 1 -- 172.21.15.52:0/577064658 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2e4026730 con 0x561c142ef0c0 2024-04-02T13:56:52.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.563+0000 7fc2ddffb640 1 -- 172.21.15.52:0/577064658 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fc2e4004700 con 0x561c142ef0c0 2024-04-02T13:56:52.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.564+0000 7fc2ff1b4740 10 client.16620 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:52.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.564+0000 7fc2ff1b4740 10 client.16620 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T13:56:52.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.564+0000 7fc2ff1b4740 20 client.16620 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:52.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.564+0000 7fc2ff1b4740 1 --2- 172.21.15.52:0/577064658 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x561c1436eb40 0x561c14370f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.564+0000 7fc2ff1b4740 4 client.16620 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:56:52.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.564+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/577064658 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c1425a040 con 0x561c1436eb40 2024-04-02T13:56:52.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.564+0000 7fc2fa28e640 1 --2- 172.21.15.52:0/577064658 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x561c1436eb40 0x561c14370f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.564+0000 7fc2ddffb640 10 client.16620 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:56:52.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.564+0000 7fc2fa28e640 1 --2- 172.21.15.52:0/577064658 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x561c1436eb40 0x561c14370f60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5afa799ef8a4e8e1 server_cookie=7358b4393bdf43af in_seq=0 out_seq=0 2024-04-02T13:56:52.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.564+0000 7fc2ddffb640 1 -- 172.21.15.52:0/577064658 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c1425a040 con 0x561c1436eb40 2024-04-02T13:56:52.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.564+0000 7fc2ddffb640 10 client.16620 handle_command_reply: tid=0 2024-04-02T13:56:52.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.564+0000 7fc2ff1b4740 1 client.16620 shutdown 2024-04-02T13:56:52.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.564+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/577064658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142ef0c0 msgr2=0x561c142f1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.564+0000 7fc2ff1b4740 1 --2- 172.21.15.52:0/577064658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142ef0c0 0x561c142f1a60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400c010 tx=0x7fc2e400fc90 comp rx=0 tx=0).stop 2024-04-02T13:56:52.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.564+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/577064658 shutdown_connections 2024-04-02T13:56:52.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.564+0000 7fc2ff1b4740 1 --2- 172.21.15.52:0/577064658 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x561c1436eb40 0x561c14370f60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.564+0000 7fc2ff1b4740 1 --2- 172.21.15.52:0/577064658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c142f7ca0 0x561c142fee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.565+0000 7fc2ddffb640 0 client.16620 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:56:52.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.565+0000 7fc2ff1b4740 1 --2- 172.21.15.52:0/577064658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c142ef0c0 0x561c142f1a60 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.565+0000 7fc2ff1b4740 1 --2- 172.21.15.52:0/577064658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c142ee7a0 0x561c142f1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.565+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/577064658 >> 172.21.15.52:0/577064658 conn(0x561c142e8ed0 msgr2=0x561c142e76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.565+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/577064658 shutdown_connections 2024-04-02T13:56:52.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.565+0000 7fc2ff1b4740 1 -- 172.21.15.52:0/577064658 wait complete. 2024-04-02T13:56:52.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.565+0000 7fc2ff1b4740 20 client.16620 trim_cache size 0 max 16384 2024-04-02T13:56:52.577 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:56:52.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.565+0000 7fc2fc4f2640 1 -- 172.21.15.52:0/3668843736 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc2e00a3d20 msgr2=0x7fc2e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.565+0000 7fc2fc4f2640 1 --2- 172.21.15.52:0/3668843736 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc2e00a3d20 0x7fc2e00a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4005fd0 tx=0x7fc2e4005850 comp rx=0 tx=0).stop 2024-04-02T13:56:52.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.565+0000 7fc2fc4f2640 1 -- 172.21.15.52:0/3668843736 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2f4102d70 msgr2=0x7fc2f4198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.565+0000 7fc2fc4f2640 1 --2- 172.21.15.52:0/3668843736 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2f4102d70 0x7fc2f4198370 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec005940 tx=0x7fc2ec02cd10 comp rx=0 tx=0).stop 2024-04-02T13:56:52.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.565+0000 7fc2fc4f2640 1 -- 172.21.15.52:0/3668843736 shutdown_connections 2024-04-02T13:56:52.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.565+0000 7fc2fc4f2640 1 --2- 172.21.15.52:0/3668843736 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc2e00a3d20 0x7fc2e00a61e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.565+0000 7fc2fc4f2640 1 --2- 172.21.15.52:0/3668843736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2f410c140 0x7fc2f419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.565+0000 7fc2fc4f2640 1 --2- 172.21.15.52:0/3668843736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2f4103690 0x7fc2f41988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.565+0000 7fc2fc4f2640 1 --2- 172.21.15.52:0/3668843736 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2f4102d70 0x7fc2f4198370 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.565+0000 7fc2fc4f2640 1 -- 172.21.15.52:0/3668843736 >> 172.21.15.52:0/3668843736 conn(0x7fc2f4077b60 msgr2=0x7fc2f40fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.565+0000 7fc2fc4f2640 1 -- 172.21.15.52:0/3668843736 shutdown_connections 2024-04-02T13:56:52.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.565+0000 7fc2fc4f2640 1 -- 172.21.15.52:0/3668843736 wait complete. 2024-04-02T13:56:52.581 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:56:52.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.682+0000 7fdf4aa0d640 1 -- 172.21.15.52:0/1610788311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf44068d00 msgr2=0x7fdf44069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.682+0000 7fdf4aa0d640 1 --2- 172.21.15.52:0/1610788311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf44068d00 0x7fdf44069180 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c009930 tx=0x7fdf2c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:56:52.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.682+0000 7fdf4aa0d640 1 -- 172.21.15.52:0/1610788311 shutdown_connections 2024-04-02T13:56:52.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.682+0000 7fdf4aa0d640 1 --2- 172.21.15.52:0/1610788311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf440696c0 0x7fdf4410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.682+0000 7fdf4aa0d640 1 --2- 172.21.15.52:0/1610788311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf44068d00 0x7fdf44069180 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.682+0000 7fdf4aa0d640 1 --2- 172.21.15.52:0/1610788311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf44107b50 0x7fdf44107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.682+0000 7fdf4aa0d640 1 -- 172.21.15.52:0/1610788311 >> 172.21.15.52:0/1610788311 conn(0x7fdf44077b00 msgr2=0x7fdf44077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.683+0000 7fdf4aa0d640 1 -- 172.21.15.52:0/1610788311 shutdown_connections 2024-04-02T13:56:52.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.683+0000 7fdf4aa0d640 1 -- 172.21.15.52:0/1610788311 wait complete. 2024-04-02T13:56:52.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.683+0000 7fdf4aa0d640 1 Processor -- start 2024-04-02T13:56:52.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.683+0000 7fdf4aa0d640 1 -- start start 2024-04-02T13:56:52.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.683+0000 7fdf4aa0d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf44068d00 0x7fdf441983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.683+0000 7fdf4aa0d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf440696c0 0x7fdf44198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.683+0000 7fdf4aa0d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf44107b50 0x7fdf4419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.683+0000 7fdf4aa0d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdf44111e70 con 0x7fdf440696c0 2024-04-02T13:56:52.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.683+0000 7fdf4aa0d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdf44111cf0 con 0x7fdf44107b50 2024-04-02T13:56:52.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.683+0000 7fdf4aa0d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdf44111ff0 con 0x7fdf44068d00 2024-04-02T13:56:52.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.684+0000 7fdf437fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf440696c0 0x7fdf44198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.684+0000 7fdf437fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf440696c0 0x7fdf44198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51990/0 (socket says 172.21.15.52:51990) 2024-04-02T13:56:52.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.684+0000 7fdf437fe640 1 -- 172.21.15.52:0/831223388 learned_addr learned my addr 172.21.15.52:0/831223388 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:52.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.684+0000 7fdf48faa640 1 --2- 172.21.15.52:0/831223388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf44107b50 0x7fdf4419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.684+0000 7fdf43fff640 1 --2- 172.21.15.52:0/831223388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf44068d00 0x7fdf441983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.684+0000 7fdf437fe640 1 -- 172.21.15.52:0/831223388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf44068d00 msgr2=0x7fdf441983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.684+0000 7fdf437fe640 1 --2- 172.21.15.52:0/831223388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf44068d00 0x7fdf441983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.684+0000 7fdf437fe640 1 -- 172.21.15.52:0/831223388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf44107b50 msgr2=0x7fdf4419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.684+0000 7fdf437fe640 1 --2- 172.21.15.52:0/831223388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf44107b50 0x7fdf4419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.684+0000 7fdf437fe640 1 -- 172.21.15.52:0/831223388 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4419d370 con 0x7fdf440696c0 2024-04-02T13:56:52.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.684+0000 7fdf43fff640 1 --2- 172.21.15.52:0/831223388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf44068d00 0x7fdf441983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:56:52.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.684+0000 7fdf437fe640 1 --2- 172.21.15.52:0/831223388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf440696c0 0x7fdf44198910 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c009230 tx=0x7fdf2c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.684+0000 7fdf417fa640 1 -- 172.21.15.52:0/831223388 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c041070 con 0x7fdf440696c0 2024-04-02T13:56:52.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.684+0000 7fdf4aa0d640 1 -- 172.21.15.52:0/831223388 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf4419d600 con 0x7fdf440696c0 2024-04-02T13:56:52.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.684+0000 7fdf417fa640 1 -- 172.21.15.52:0/831223388 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf2c02ccc0 con 0x7fdf440696c0 2024-04-02T13:56:52.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.684+0000 7fdf417fa640 1 -- 172.21.15.52:0/831223388 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c003e20 con 0x7fdf440696c0 2024-04-02T13:56:52.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.684+0000 7fdf4aa0d640 1 -- 172.21.15.52:0/831223388 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf441a4ef0 con 0x7fdf440696c0 2024-04-02T13:56:52.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.686+0000 7fdf417fa640 1 -- 172.21.15.52:0/831223388 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdf2c044070 con 0x7fdf440696c0 2024-04-02T13:56:52.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.687+0000 7fdf4d5fc740 1 Processor -- start 2024-04-02T13:56:52.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.687+0000 7fdf4d5fc740 1 -- start start 2024-04-02T13:56:52.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.687+0000 7fdf4d5fc740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643f95247a0 0x5643f9524b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.687+0000 7fdf4d5fc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f95250c0 0x5643f952d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.687+0000 7fdf4d5fc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643f952dca0 0x5643f9530090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.687+0000 7fdf4d5fc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5643f94887b0 con 0x5643f95250c0 2024-04-02T13:56:52.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.687+0000 7fdf4d5fc740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5643f9427570 con 0x5643f95247a0 2024-04-02T13:56:52.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.687+0000 7fdf4d5fc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5643f9484760 con 0x5643f952dca0 2024-04-02T13:56:52.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.687+0000 7fdf43fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f95250c0 0x5643f952d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.687+0000 7fdf417fa640 1 --2- 172.21.15.52:0/831223388 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdf300a3d20 0x7fdf300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.687+0000 7fdf43fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f95250c0 0x5643f952d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51992/0 (socket says 172.21.15.52:51992) 2024-04-02T13:56:52.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.687+0000 7fdf43fff640 1 -- 172.21.15.52:0/1739723254 learned_addr learned my addr 172.21.15.52:0/1739723254 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:52.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.687+0000 7fdf48faa640 1 --2- 172.21.15.52:0/1739723254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643f95247a0 0x5643f9524b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.687+0000 7fdf48faa640 1 --2- 172.21.15.52:0/831223388 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdf300a3d20 0x7fdf300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.687+0000 7fdf417fa640 1 -- 172.21.15.52:0/831223388 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fdf2c0f2500 con 0x7fdf440696c0 2024-04-02T13:56:52.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.687+0000 7fdf437fe640 1 --2- 172.21.15.52:0/1739723254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643f952dca0 0x5643f9530090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.687+0000 7fdf43fff640 1 -- 172.21.15.52:0/1739723254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643f952dca0 msgr2=0x5643f9530090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.687+0000 7fdf43fff640 1 --2- 172.21.15.52:0/1739723254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643f952dca0 0x5643f9530090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.687+0000 7fdf43fff640 1 -- 172.21.15.52:0/1739723254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643f95247a0 msgr2=0x5643f9524b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.687+0000 7fdf43fff640 1 --2- 172.21.15.52:0/1739723254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643f95247a0 0x5643f9524b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.687+0000 7fdf43fff640 1 -- 172.21.15.52:0/1739723254 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643f919c3f0 con 0x5643f95250c0 2024-04-02T13:56:52.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.687+0000 7fdf48faa640 1 --2- 172.21.15.52:0/1739723254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643f95247a0 0x5643f9524b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:52.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.688+0000 7fdf43fff640 1 --2- 172.21.15.52:0/1739723254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f95250c0 0x5643f952d760 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009c50 tx=0x7fdf3400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5aa3fa9e1cdd0cc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.688+0000 7fdf48faa640 1 --2- 172.21.15.52:0/831223388 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdf300a3d20 0x7fdf300a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00a780 tx=0x7fdf3c00a4d0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.688+0000 7fdf3a7fc640 1 -- 172.21.15.52:0/1739723254 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf34016070 con 0x5643f95250c0 2024-04-02T13:56:52.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.688+0000 7fdf3a7fc640 1 -- 172.21.15.52:0/1739723254 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf34005500 con 0x5643f95250c0 2024-04-02T13:56:52.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.688+0000 7fdf3a7fc640 1 -- 172.21.15.52:0/1739723254 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf34011670 con 0x5643f95250c0 2024-04-02T13:56:52.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.688+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/1739723254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f95250c0 msgr2=0x5643f952d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.688+0000 7fdf4d5fc740 1 --2- 172.21.15.52:0/1739723254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f95250c0 0x5643f952d760 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009c50 tx=0x7fdf3400e6c0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.688+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/1739723254 shutdown_connections 2024-04-02T13:56:52.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.688+0000 7fdf4d5fc740 1 --2- 172.21.15.52:0/1739723254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643f952dca0 0x5643f9530090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.688+0000 7fdf4d5fc740 1 --2- 172.21.15.52:0/1739723254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f95250c0 0x5643f952d760 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.688+0000 7fdf4d5fc740 1 --2- 172.21.15.52:0/1739723254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643f95247a0 0x5643f9524b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.688+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/1739723254 >> 172.21.15.52:0/1739723254 conn(0x5643f951eed0 msgr2=0x5643f951f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.688+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/1739723254 shutdown_connections 2024-04-02T13:56:52.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.688+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/1739723254 wait complete. 2024-04-02T13:56:52.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.689+0000 7fdf4d5fc740 1 -- start start 2024-04-02T13:56:52.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.689+0000 7fdf4d5fc740 1 Processor -- start 2024-04-02T13:56:52.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.689+0000 7fdf4d5fc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643f95247a0 0x5643f952ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.689+0000 7fdf4d5fc740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643f95250c0 0x5643f952b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.689+0000 7fdf4d5fc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f952dca0 0x5643f952b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.689+0000 7fdf4d5fc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5643f9530a90 con 0x5643f952dca0 2024-04-02T13:56:52.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.689+0000 7fdf4d5fc740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5643f9443e80 con 0x5643f95250c0 2024-04-02T13:56:52.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.689+0000 7fdf4d5fc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5643f9530c10 con 0x5643f95247a0 2024-04-02T13:56:52.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.689+0000 7fdf4d5fc740 10 client.0 _ll_register_callbacks cb 0x5643f947faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:52.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.689+0000 7fdf48faa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f952dca0 0x5643f952b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.689+0000 7fdf48faa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f952dca0 0x5643f952b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52008/0 (socket says 172.21.15.52:52008) 2024-04-02T13:56:52.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.689+0000 7fdf48faa640 1 -- 172.21.15.52:0/1292883217 learned_addr learned my addr 172.21.15.52:0/1292883217 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:52.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.689+0000 7fdf437fe640 1 --2- 172.21.15.52:0/1292883217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643f95250c0 0x5643f952b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.690+0000 7fdf43fff640 1 --2- 172.21.15.52:0/1292883217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643f95247a0 0x5643f952ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.690+0000 7fdf48faa640 1 -- 172.21.15.52:0/1292883217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643f95247a0 msgr2=0x5643f952ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.690+0000 7fdf48faa640 1 --2- 172.21.15.52:0/1292883217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643f95247a0 0x5643f952ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.690+0000 7fdf48faa640 1 -- 172.21.15.52:0/1292883217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643f95250c0 msgr2=0x5643f952b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.690+0000 7fdf48faa640 1 --2- 172.21.15.52:0/1292883217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643f95250c0 0x5643f952b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.690+0000 7fdf48faa640 1 -- 172.21.15.52:0/1292883217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643f92667d0 con 0x5643f952dca0 2024-04-02T13:56:52.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.690+0000 7fdf48faa640 1 --2- 172.21.15.52:0/1292883217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f952dca0 0x5643f952b5a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00ad90 tx=0x7fdf3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c540637977874871 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.690+0000 7fdf237fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:52.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.690+0000 7fdf237fe640 1 -- 172.21.15.52:0/1292883217 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c026070 con 0x5643f952dca0 2024-04-02T13:56:52.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.690+0000 7fdf237fe640 1 -- 172.21.15.52:0/1292883217 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fdf3c004b20 con 0x5643f952dca0 2024-04-02T13:56:52.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.690+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/1292883217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf34005500 con 0x5643f952dca0 2024-04-02T13:56:52.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.690+0000 7fdf237fe640 1 -- 172.21.15.52:0/1292883217 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdf3c004d20 con 0x5643f952dca0 2024-04-02T13:56:52.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.690+0000 7fdf4d5fc740 10 client.16632 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:52.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.690+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/1292883217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5643f953caf0 con 0x5643f952dca0 2024-04-02T13:56:52.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.690+0000 7fdf237fe640 1 -- 172.21.15.52:0/1292883217 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf34005500 con 0x5643f952dca0 2024-04-02T13:56:52.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.691+0000 7fdf237fe640 1 -- 172.21.15.52:0/1292883217 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fdf3c004660 con 0x5643f952dca0 2024-04-02T13:56:52.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.691+0000 7fdf4d5fc740 10 client.16632 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:52.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.691+0000 7fdf4d5fc740 10 client.16632 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:56:52.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.691+0000 7fdf4d5fc740 20 client.16632 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:52.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.691+0000 7fdf4d5fc740 1 --2- 172.21.15.52:0/1292883217 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5643f953e000 0x5643f9540430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.691+0000 7fdf4d5fc740 4 client.16632 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:52.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.691+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/1292883217 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643f9495cf0 con 0x5643f953e000 2024-04-02T13:56:52.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.691+0000 7fdf43fff640 1 --2- 172.21.15.52:0/1292883217 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5643f953e000 0x5643f9540430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.691+0000 7fdf237fe640 10 client.16632 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:56:52.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.691+0000 7fdf43fff640 1 --2- 172.21.15.52:0/1292883217 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5643f953e000 0x5643f9540430 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7a4d90910b6a118 server_cookie=7c987ee98d07f40b in_seq=0 out_seq=0 2024-04-02T13:56:52.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.692+0000 7fdf237fe640 1 -- 172.21.15.52:0/1292883217 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5643f9495cf0 con 0x5643f953e000 2024-04-02T13:56:52.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.692+0000 7fdf237fe640 10 client.16632 handle_command_reply: tid=0 2024-04-02T13:56:52.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.692+0000 7fdf4d5fc740 1 client.16632 shutdown 2024-04-02T13:56:52.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.693+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/1292883217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f952dca0 msgr2=0x5643f952b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.693+0000 7fdf4d5fc740 1 --2- 172.21.15.52:0/1292883217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f952dca0 0x5643f952b5a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00ad90 tx=0x7fdf3c004a40 comp rx=0 tx=0).stop 2024-04-02T13:56:52.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.693+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/1292883217 shutdown_connections 2024-04-02T13:56:52.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.693+0000 7fdf4d5fc740 1 --2- 172.21.15.52:0/1292883217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f952dca0 0x5643f952b5a0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.693+0000 7fdf4d5fc740 1 --2- 172.21.15.52:0/1292883217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643f95250c0 0x5643f952b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.693+0000 7fdf4d5fc740 1 --2- 172.21.15.52:0/1292883217 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5643f953e000 0x5643f9540430 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.693+0000 7fdf4d5fc740 1 --2- 172.21.15.52:0/1292883217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643f95247a0 0x5643f952ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.693+0000 7fdf237fe640 0 client.16632 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:56:52.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.693+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/1292883217 >> 172.21.15.52:0/1292883217 conn(0x5643f951eed0 msgr2=0x5643f9520c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.693+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/1292883217 shutdown_connections 2024-04-02T13:56:52.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.693+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/1292883217 wait complete. 2024-04-02T13:56:52.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.693+0000 7fdf4d5fc740 20 client.16632 trim_cache size 0 max 16384 2024-04-02T13:56:52.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.706+0000 7fdf4d5fc740 1 Processor -- start 2024-04-02T13:56:52.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.706+0000 7fdf4d5fc740 1 -- start start 2024-04-02T13:56:52.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.706+0000 7fdf4d5fc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643f95247a0 0x5643f952d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.706+0000 7fdf4d5fc740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643f95250c0 0x5643f9539950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.706+0000 7fdf4d5fc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f952dca0 0x5643f9529c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.706+0000 7fdf4d5fc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdf441aee80 con 0x5643f952dca0 2024-04-02T13:56:52.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.706+0000 7fdf4d5fc740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdf441b09f0 con 0x5643f95250c0 2024-04-02T13:56:52.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.706+0000 7fdf4d5fc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdf441af550 con 0x5643f95247a0 2024-04-02T13:56:52.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.706+0000 7fdf48faa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f952dca0 0x5643f9529c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.706+0000 7fdf48faa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f952dca0 0x5643f9529c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52020/0 (socket says 172.21.15.52:52020) 2024-04-02T13:56:52.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.706+0000 7fdf48faa640 1 -- 172.21.15.52:0/3167581545 learned_addr learned my addr 172.21.15.52:0/3167581545 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:52.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.706+0000 7fdf43fff640 1 --2- 172.21.15.52:0/3167581545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643f95247a0 0x5643f952d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.706+0000 7fdf437fe640 1 --2- 172.21.15.52:0/3167581545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643f95250c0 0x5643f9539950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.706+0000 7fdf48faa640 1 -- 172.21.15.52:0/3167581545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643f95247a0 msgr2=0x5643f952d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.706+0000 7fdf48faa640 1 --2- 172.21.15.52:0/3167581545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643f95247a0 0x5643f952d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.706+0000 7fdf48faa640 1 -- 172.21.15.52:0/3167581545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643f95250c0 msgr2=0x5643f9539950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.707+0000 7fdf48faa640 1 --2- 172.21.15.52:0/3167581545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643f95250c0 0x5643f9539950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.707+0000 7fdf48faa640 1 -- 172.21.15.52:0/3167581545 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643f9524ef0 con 0x5643f952dca0 2024-04-02T13:56:52.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.707+0000 7fdf48faa640 1 --2- 172.21.15.52:0/3167581545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f952dca0 0x5643f9529c80 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c004c40 tx=0x7fdf3c004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d484a6ed533c2d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.707+0000 7fdf39ffb640 1 -- 172.21.15.52:0/3167581545 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c026070 con 0x5643f952dca0 2024-04-02T13:56:52.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.707+0000 7fdf39ffb640 1 -- 172.21.15.52:0/3167581545 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf3c003e50 con 0x5643f952dca0 2024-04-02T13:56:52.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.707+0000 7fdf39ffb640 1 -- 172.21.15.52:0/3167581545 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c014030 con 0x5643f952dca0 2024-04-02T13:56:52.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.707+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/3167581545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f952dca0 msgr2=0x5643f9529c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.707+0000 7fdf4d5fc740 1 --2- 172.21.15.52:0/3167581545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f952dca0 0x5643f9529c80 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c004c40 tx=0x7fdf3c004c70 comp rx=0 tx=0).stop 2024-04-02T13:56:52.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.707+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/3167581545 shutdown_connections 2024-04-02T13:56:52.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.707+0000 7fdf4d5fc740 1 --2- 172.21.15.52:0/3167581545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f952dca0 0x5643f9529c80 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.707+0000 7fdf4d5fc740 1 --2- 172.21.15.52:0/3167581545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643f95250c0 0x5643f9539950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.707+0000 7fdf4d5fc740 1 --2- 172.21.15.52:0/3167581545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643f95247a0 0x5643f952d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.707+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/3167581545 >> 172.21.15.52:0/3167581545 conn(0x5643f951eed0 msgr2=0x5643f951d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.707+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/3167581545 shutdown_connections 2024-04-02T13:56:52.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.707+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/3167581545 wait complete. 2024-04-02T13:56:52.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.708+0000 7fdf4d5fc740 1 -- start start 2024-04-02T13:56:52.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.708+0000 7fdf4d5fc740 1 Processor -- start 2024-04-02T13:56:52.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.708+0000 7fdf4d5fc740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643f95247a0 0x5643f9536180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.708+0000 7fdf4d5fc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f95250c0 0x5643f95314a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.708+0000 7fdf4d5fc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643f952dca0 0x5643f95319e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.708+0000 7fdf4d5fc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdf441b02f0 con 0x5643f95250c0 2024-04-02T13:56:52.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.708+0000 7fdf4d5fc740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdf441afc20 con 0x5643f95247a0 2024-04-02T13:56:52.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.708+0000 7fdf4d5fc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdf441ace60 con 0x5643f952dca0 2024-04-02T13:56:52.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.708+0000 7fdf4d5fc740 10 client.0 _ll_register_callbacks cb 0x7fdf3c014830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:52.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.708+0000 7fdf437fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f95250c0 0x5643f95314a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.708+0000 7fdf437fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f95250c0 0x5643f95314a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52032/0 (socket says 172.21.15.52:52032) 2024-04-02T13:56:52.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.708+0000 7fdf437fe640 1 -- 172.21.15.52:0/1191179778 learned_addr learned my addr 172.21.15.52:0/1191179778 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:52.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.708+0000 7fdf43fff640 1 --2- 172.21.15.52:0/1191179778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643f95247a0 0x5643f9536180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.708+0000 7fdf48faa640 1 --2- 172.21.15.52:0/1191179778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643f952dca0 0x5643f95319e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.709+0000 7fdf437fe640 1 -- 172.21.15.52:0/1191179778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643f952dca0 msgr2=0x5643f95319e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.709+0000 7fdf437fe640 1 --2- 172.21.15.52:0/1191179778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643f952dca0 0x5643f95319e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.709+0000 7fdf437fe640 1 -- 172.21.15.52:0/1191179778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643f95247a0 msgr2=0x5643f9536180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.709+0000 7fdf437fe640 1 --2- 172.21.15.52:0/1191179778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643f95247a0 0x5643f9536180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.709+0000 7fdf437fe640 1 -- 172.21.15.52:0/1191179778 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643f9532120 con 0x5643f95250c0 2024-04-02T13:56:52.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.709+0000 7fdf48faa640 1 --2- 172.21.15.52:0/1191179778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643f952dca0 0x5643f95319e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:52.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.709+0000 7fdf437fe640 1 --2- 172.21.15.52:0/1191179778 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f95250c0 0x5643f95314a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c009430 tx=0x7fdf2c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918cd97c2603b2af server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.709+0000 7fdf237fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:52.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.709+0000 7fdf237fe640 1 -- 172.21.15.52:0/1191179778 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c04b040 con 0x5643f95250c0 2024-04-02T13:56:52.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.709+0000 7fdf237fe640 1 -- 172.21.15.52:0/1191179778 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fdf2c0efb90 con 0x5643f95250c0 2024-04-02T13:56:52.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.709+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/1191179778 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf3c003e50 con 0x5643f95250c0 2024-04-02T13:56:52.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.709+0000 7fdf237fe640 1 -- 172.21.15.52:0/1191179778 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdf2c0c2620 con 0x5643f95250c0 2024-04-02T13:56:52.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.709+0000 7fdf4d5fc740 10 client.16638 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:52.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.709+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/1191179778 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5643f95a4500 con 0x5643f95250c0 2024-04-02T13:56:52.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.709+0000 7fdf237fe640 1 -- 172.21.15.52:0/1191179778 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf3c003e50 con 0x5643f95250c0 2024-04-02T13:56:52.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.709+0000 7fdf237fe640 1 -- 172.21.15.52:0/1191179778 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fdf2c0f2140 con 0x5643f95250c0 2024-04-02T13:56:52.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.709+0000 7fdf4d5fc740 10 client.16638 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:52.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.709+0000 7fdf4d5fc740 10 client.16638 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:56:52.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.709+0000 7fdf4d5fc740 20 client.16638 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:52.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.709+0000 7fdf4d5fc740 1 --2- 172.21.15.52:0/1191179778 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5643f95a4c30 0x5643f95a7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.709+0000 7fdf4d5fc740 4 client.16638 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:56:52.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.709+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/1191179778 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5643f9490040 con 0x5643f95a4c30 2024-04-02T13:56:52.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.710+0000 7fdf43fff640 1 --2- 172.21.15.52:0/1191179778 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5643f95a4c30 0x5643f95a7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.710+0000 7fdf237fe640 10 client.16638 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:56:52.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.710+0000 7fdf43fff640 1 --2- 172.21.15.52:0/1191179778 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5643f95a4c30 0x5643f95a7050 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b811ead46076d95 server_cookie=2b634f81dc653162 in_seq=0 out_seq=0 2024-04-02T13:56:52.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.710+0000 7fdf237fe640 1 -- 172.21.15.52:0/1191179778 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5643f9490040 con 0x5643f95a4c30 2024-04-02T13:56:52.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.710+0000 7fdf237fe640 10 client.16638 handle_command_reply: tid=0 2024-04-02T13:56:52.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.710+0000 7fdf4d5fc740 1 client.16638 shutdown 2024-04-02T13:56:52.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.710+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/1191179778 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f95250c0 msgr2=0x5643f95314a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.710+0000 7fdf4d5fc740 1 --2- 172.21.15.52:0/1191179778 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f95250c0 0x5643f95314a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c009430 tx=0x7fdf2c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:56:52.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.710+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/1191179778 shutdown_connections 2024-04-02T13:56:52.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.710+0000 7fdf4d5fc740 1 --2- 172.21.15.52:0/1191179778 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5643f95a4c30 0x5643f95a7050 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.710+0000 7fdf4d5fc740 1 --2- 172.21.15.52:0/1191179778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643f952dca0 0x5643f95319e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.710+0000 7fdf4d5fc740 1 --2- 172.21.15.52:0/1191179778 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f95250c0 0x5643f95314a0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.710+0000 7fdf237fe640 0 client.16638 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:56:52.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.710+0000 7fdf4d5fc740 1 --2- 172.21.15.52:0/1191179778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643f95247a0 0x5643f9536180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.711+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/1191179778 >> 172.21.15.52:0/1191179778 conn(0x5643f951eed0 msgr2=0x5643f951d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.711+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/1191179778 shutdown_connections 2024-04-02T13:56:52.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.711+0000 7fdf4d5fc740 1 -- 172.21.15.52:0/1191179778 wait complete. 2024-04-02T13:56:52.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.711+0000 7fdf4d5fc740 20 client.16638 trim_cache size 0 max 16384 2024-04-02T13:56:52.727 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:56:52.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.711+0000 7fdf4aa0d640 1 -- 172.21.15.52:0/831223388 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdf300a3d20 msgr2=0x7fdf300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.711+0000 7fdf4aa0d640 1 --2- 172.21.15.52:0/831223388 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdf300a3d20 0x7fdf300a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00a780 tx=0x7fdf3c00a4d0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.711+0000 7fdf4aa0d640 1 -- 172.21.15.52:0/831223388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf440696c0 msgr2=0x7fdf44198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.711+0000 7fdf4aa0d640 1 --2- 172.21.15.52:0/831223388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf440696c0 0x7fdf44198910 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c009230 tx=0x7fdf2c0043b0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.711+0000 7fdf4aa0d640 1 -- 172.21.15.52:0/831223388 shutdown_connections 2024-04-02T13:56:52.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.711+0000 7fdf4aa0d640 1 --2- 172.21.15.52:0/831223388 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdf300a3d20 0x7fdf300a61e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.711+0000 7fdf4aa0d640 1 --2- 172.21.15.52:0/831223388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf44107b50 0x7fdf4419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.711+0000 7fdf4aa0d640 1 --2- 172.21.15.52:0/831223388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf440696c0 0x7fdf44198910 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.711+0000 7fdf4aa0d640 1 --2- 172.21.15.52:0/831223388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf44068d00 0x7fdf441983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.711+0000 7fdf4aa0d640 1 -- 172.21.15.52:0/831223388 >> 172.21.15.52:0/831223388 conn(0x7fdf44077b00 msgr2=0x7fdf440fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.711+0000 7fdf4aa0d640 1 -- 172.21.15.52:0/831223388 shutdown_connections 2024-04-02T13:56:52.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.711+0000 7fdf4aa0d640 1 -- 172.21.15.52:0/831223388 wait complete. 2024-04-02T13:56:52.729 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:52 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/4236803421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:56:52.730 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:56:52.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.832+0000 7f5563a14640 1 -- 172.21.15.52:0/822335535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f555c107aa0 msgr2=0x7f555c107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.832+0000 7f5563a14640 1 --2- 172.21.15.52:0/822335535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f555c107aa0 0x7f555c107e80 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5554009930 tx=0x7f555402bc30 comp rx=0 tx=0).stop 2024-04-02T13:56:52.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.832+0000 7f5563a14640 1 -- 172.21.15.52:0/822335535 shutdown_connections 2024-04-02T13:56:52.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.832+0000 7f5563a14640 1 --2- 172.21.15.52:0/822335535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f555c102450 0x7f555c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.832+0000 7f5563a14640 1 --2- 172.21.15.52:0/822335535 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f555c101a90 0x7f555c101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.832+0000 7f5563a14640 1 --2- 172.21.15.52:0/822335535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f555c107aa0 0x7f555c107e80 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.832+0000 7f5563a14640 1 -- 172.21.15.52:0/822335535 >> 172.21.15.52:0/822335535 conn(0x7f555c078f00 msgr2=0x7f555c079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.832+0000 7f5563a14640 1 -- 172.21.15.52:0/822335535 shutdown_connections 2024-04-02T13:56:52.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.832+0000 7f5563a14640 1 -- 172.21.15.52:0/822335535 wait complete. 2024-04-02T13:56:52.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.833+0000 7f5563a14640 1 Processor -- start 2024-04-02T13:56:52.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.833+0000 7f5563a14640 1 -- start start 2024-04-02T13:56:52.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.833+0000 7f5563a14640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f555c101a90 0x7f555c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.833+0000 7f5563a14640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f555c102450 0x7f555c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.833+0000 7f5563a14640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f555c107aa0 0x7f555c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.833+0000 7f5563a14640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f555c111e80 con 0x7f555c107aa0 2024-04-02T13:56:52.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.833+0000 7f5563a14640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f555c111d00 con 0x7f555c101a90 2024-04-02T13:56:52.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.833+0000 7f5563a14640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f555c112000 con 0x7f555c102450 2024-04-02T13:56:52.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.833+0000 7f5561fb1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f555c107aa0 0x7f555c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.833+0000 7f5561fb1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f555c107aa0 0x7f555c19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52050/0 (socket says 172.21.15.52:52050) 2024-04-02T13:56:52.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.833+0000 7f5561fb1640 1 -- 172.21.15.52:0/2834778929 learned_addr learned my addr 172.21.15.52:0/2834778929 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:52.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.833+0000 7f55617b0640 1 --2- 172.21.15.52:0/2834778929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f555c101a90 0x7f555c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.833+0000 7f5560faf640 1 --2- 172.21.15.52:0/2834778929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f555c102450 0x7f555c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.834+0000 7f5561fb1640 1 -- 172.21.15.52:0/2834778929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f555c102450 msgr2=0x7f555c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.834+0000 7f5561fb1640 1 --2- 172.21.15.52:0/2834778929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f555c102450 0x7f555c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.834+0000 7f5561fb1640 1 -- 172.21.15.52:0/2834778929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f555c101a90 msgr2=0x7f555c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.834+0000 7f5561fb1640 1 --2- 172.21.15.52:0/2834778929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f555c101a90 0x7f555c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.834+0000 7f5561fb1640 1 -- 172.21.15.52:0/2834778929 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555c19d360 con 0x7f555c107aa0 2024-04-02T13:56:52.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.834+0000 7f5560faf640 1 --2- 172.21.15.52:0/2834778929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f555c102450 0x7f555c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:52.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.834+0000 7f5561fb1640 1 --2- 172.21.15.52:0/2834778929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f555c107aa0 0x7f555c19cc30 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f555800cce0 tx=0x7f555800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.834+0000 7f55527fc640 1 -- 172.21.15.52:0/2834778929 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555800f450 con 0x7f555c107aa0 2024-04-02T13:56:52.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.834+0000 7f55527fc640 1 -- 172.21.15.52:0/2834778929 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f555800fad0 con 0x7f555c107aa0 2024-04-02T13:56:52.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.834+0000 7f5563a14640 1 -- 172.21.15.52:0/2834778929 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f555c19d650 con 0x7f555c107aa0 2024-04-02T13:56:52.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.834+0000 7f55527fc640 1 -- 172.21.15.52:0/2834778929 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558018c10 con 0x7f555c107aa0 2024-04-02T13:56:52.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.834+0000 7f5563a14640 1 -- 172.21.15.52:0/2834778929 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f555c1a4ee0 con 0x7f555c107aa0 2024-04-02T13:56:52.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.837+0000 7f55665fe740 1 Processor -- start 2024-04-02T13:56:52.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.837+0000 7f55527fc640 1 -- 172.21.15.52:0/2834778929 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f555800f5f0 con 0x7f555c107aa0 2024-04-02T13:56:52.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.837+0000 7f55665fe740 1 -- start start 2024-04-02T13:56:52.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.837+0000 7f55665fe740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134cebb0 0x5569134cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.837+0000 7f55665fe740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5569134cf4d0 0x5569134d7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.837+0000 7f55665fe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569134d80b0 0x5569134da4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.837+0000 7f55665fe740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55691342e7b0 con 0x5569134cebb0 2024-04-02T13:56:52.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.837+0000 7f55665fe740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5569133cd570 con 0x5569134cf4d0 2024-04-02T13:56:52.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.837+0000 7f55665fe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55691342a760 con 0x5569134d80b0 2024-04-02T13:56:52.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.838+0000 7f5560faf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134cebb0 0x5569134cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.838+0000 7f5560faf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134cebb0 0x5569134cef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52066/0 (socket says 172.21.15.52:52066) 2024-04-02T13:56:52.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.838+0000 7f5560faf640 1 -- 172.21.15.52:0/3500617679 learned_addr learned my addr 172.21.15.52:0/3500617679 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:52.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.838+0000 7f55617b0640 1 --2- 172.21.15.52:0/3500617679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5569134cf4d0 0x5569134d7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.838+0000 7f55527fc640 1 --2- 172.21.15.52:0/2834778929 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f55480a3da0 0x7f55480a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.838+0000 7f5560faf640 1 -- 172.21.15.52:0/3500617679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569134d80b0 msgr2=0x5569134da4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.838+0000 7f5560faf640 1 --2- 172.21.15.52:0/3500617679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569134d80b0 0x5569134da4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.838+0000 7f5560faf640 1 -- 172.21.15.52:0/3500617679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5569134cf4d0 msgr2=0x5569134d7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.838+0000 7f5560faf640 1 --2- 172.21.15.52:0/3500617679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5569134cf4d0 0x5569134d7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.838+0000 7f5560faf640 1 -- 172.21.15.52:0/3500617679 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569131423f0 con 0x5569134cebb0 2024-04-02T13:56:52.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.838+0000 7f5561fb1640 1 --2- 172.21.15.52:0/2834778929 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f55480a3da0 0x7f55480a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.838+0000 7f55527fc640 1 -- 172.21.15.52:0/2834778929 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f55580d1120 con 0x7f555c107aa0 2024-04-02T13:56:52.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.838+0000 7f5560faf640 1 --2- 172.21.15.52:0/3500617679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134cebb0 0x5569134cef90 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f554c00bd60 tx=0x7f554c011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c93d59069207ca server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.838+0000 7f55477fe640 1 -- 172.21.15.52:0/3500617679 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c016070 con 0x5569134cebb0 2024-04-02T13:56:52.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.838+0000 7f55477fe640 1 -- 172.21.15.52:0/3500617679 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f554c00be50 con 0x5569134cebb0 2024-04-02T13:56:52.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.838+0000 7f55477fe640 1 -- 172.21.15.52:0/3500617679 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c01a640 con 0x5569134cebb0 2024-04-02T13:56:52.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.839+0000 7f55665fe740 1 -- 172.21.15.52:0/3500617679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134cebb0 msgr2=0x5569134cef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.839+0000 7f5561fb1640 1 --2- 172.21.15.52:0/2834778929 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f55480a3da0 0x7f55480a6260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f555809ca20 tx=0x7f555800af30 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.839+0000 7f55665fe740 1 --2- 172.21.15.52:0/3500617679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134cebb0 0x5569134cef90 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f554c00bd60 tx=0x7f554c011750 comp rx=0 tx=0).stop 2024-04-02T13:56:52.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.839+0000 7f55665fe740 1 -- 172.21.15.52:0/3500617679 shutdown_connections 2024-04-02T13:56:52.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.839+0000 7f55665fe740 1 --2- 172.21.15.52:0/3500617679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569134d80b0 0x5569134da4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.839+0000 7f55665fe740 1 --2- 172.21.15.52:0/3500617679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5569134cf4d0 0x5569134d7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.839+0000 7f55665fe740 1 --2- 172.21.15.52:0/3500617679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134cebb0 0x5569134cef90 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.839+0000 7f55665fe740 1 -- 172.21.15.52:0/3500617679 >> 172.21.15.52:0/3500617679 conn(0x5569134c92e0 msgr2=0x5569134c96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.839+0000 7f55665fe740 1 -- 172.21.15.52:0/3500617679 shutdown_connections 2024-04-02T13:56:52.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.839+0000 7f55665fe740 1 -- 172.21.15.52:0/3500617679 wait complete. 2024-04-02T13:56:52.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.839+0000 7f55665fe740 1 -- start start 2024-04-02T13:56:52.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.839+0000 7f55665fe740 1 Processor -- start 2024-04-02T13:56:52.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.840+0000 7f55665fe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569134cebb0 0x5569134d7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.840+0000 7f55665fe740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134cf4d0 0x5569134d4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.840+0000 7f55665fe740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5569134d80b0 0x5569134d55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.840+0000 7f55665fe740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5569134daea0 con 0x5569134cf4d0 2024-04-02T13:56:52.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.840+0000 7f55665fe740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5569133e9e80 con 0x5569134d80b0 2024-04-02T13:56:52.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.840+0000 7f55665fe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5569134db020 con 0x5569134cebb0 2024-04-02T13:56:52.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.840+0000 7f55665fe740 10 client.0 _ll_register_callbacks cb 0x556913425aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:52.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.840+0000 7f55617b0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134cf4d0 0x5569134d4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.840+0000 7f55617b0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134cf4d0 0x5569134d4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52070/0 (socket says 172.21.15.52:52070) 2024-04-02T13:56:52.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.840+0000 7f55617b0640 1 -- 172.21.15.52:0/333204720 learned_addr learned my addr 172.21.15.52:0/333204720 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:52.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.840+0000 7f5560faf640 1 --2- 172.21.15.52:0/333204720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5569134d80b0 0x5569134d55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.840+0000 7f5560faf640 1 --2- 172.21.15.52:0/333204720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569134cebb0 0x5569134d7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.840+0000 7f55617b0640 1 -- 172.21.15.52:0/333204720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569134cebb0 msgr2=0x5569134d7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.840+0000 7f55617b0640 1 --2- 172.21.15.52:0/333204720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569134cebb0 0x5569134d7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.840+0000 7f55617b0640 1 -- 172.21.15.52:0/333204720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5569134d80b0 msgr2=0x5569134d55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.840+0000 7f55617b0640 1 --2- 172.21.15.52:0/333204720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5569134d80b0 0x5569134d55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.840+0000 7f55617b0640 1 -- 172.21.15.52:0/333204720 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55691320c7d0 con 0x5569134cf4d0 2024-04-02T13:56:52.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.841+0000 7f55617b0640 1 --2- 172.21.15.52:0/333204720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134cf4d0 0x5569134d4ed0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5554036040 tx=0x7f555402cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571e35c5a4413d32 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.841+0000 7f5544ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:52.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.841+0000 7f5544ff9640 1 -- 172.21.15.52:0/333204720 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5554004340 con 0x5569134cf4d0 2024-04-02T13:56:52.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.841+0000 7f5544ff9640 1 -- 172.21.15.52:0/333204720 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f55540046c0 con 0x5569134cf4d0 2024-04-02T13:56:52.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.841+0000 7f55665fe740 1 -- 172.21.15.52:0/333204720 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f554c00be50 con 0x5569134cf4d0 2024-04-02T13:56:52.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.841+0000 7f5544ff9640 1 -- 172.21.15.52:0/333204720 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f55540044e0 con 0x5569134cf4d0 2024-04-02T13:56:52.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.841+0000 7f55665fe740 10 client.16650 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:52.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.841+0000 7f55665fe740 1 -- 172.21.15.52:0/333204720 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5569134e6f90 con 0x5569134cf4d0 2024-04-02T13:56:52.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.841+0000 7f5544ff9640 1 -- 172.21.15.52:0/333204720 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f554c00be50 con 0x5569134cf4d0 2024-04-02T13:56:52.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.841+0000 7f5544ff9640 1 -- 172.21.15.52:0/333204720 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f55540035f0 con 0x5569134cf4d0 2024-04-02T13:56:52.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.841+0000 7f55665fe740 10 client.16650 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:52.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.841+0000 7f55665fe740 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:52.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.841+0000 7f55665fe740 20 client.16650 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:52.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.841+0000 7f55665fe740 1 --2- 172.21.15.52:0/333204720 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5569134e8570 0x5569134ea990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.841+0000 7f55665fe740 4 client.16650 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:52.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.841+0000 7f55665fe740 1 -- 172.21.15.52:0/333204720 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55691343bcf0 con 0x5569134e8570 2024-04-02T13:56:52.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.842+0000 7f5560faf640 1 --2- 172.21.15.52:0/333204720 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5569134e8570 0x5569134ea990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.842+0000 7f5560faf640 1 --2- 172.21.15.52:0/333204720 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5569134e8570 0x5569134ea990 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0a290b79f0e6d81 server_cookie=55a462e6e0222a38 in_seq=0 out_seq=0 2024-04-02T13:56:52.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.842+0000 7f5544ff9640 10 client.16650 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:52.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.843+0000 7f5544ff9640 1 -- 172.21.15.52:0/333204720 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55691343bcf0 con 0x5569134e8570 2024-04-02T13:56:52.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.843+0000 7f5544ff9640 10 client.16650 handle_command_reply: tid=0 2024-04-02T13:56:52.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.843+0000 7f55665fe740 1 client.16650 shutdown 2024-04-02T13:56:52.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.843+0000 7f55665fe740 1 -- 172.21.15.52:0/333204720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134cf4d0 msgr2=0x5569134d4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.843+0000 7f55665fe740 1 --2- 172.21.15.52:0/333204720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134cf4d0 0x5569134d4ed0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5554036040 tx=0x7f555402cdd0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.843+0000 7f55665fe740 1 -- 172.21.15.52:0/333204720 shutdown_connections 2024-04-02T13:56:52.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.843+0000 7f55665fe740 1 --2- 172.21.15.52:0/333204720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5569134d80b0 0x5569134d55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.843+0000 7f55665fe740 1 --2- 172.21.15.52:0/333204720 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5569134e8570 0x5569134ea990 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.843+0000 7f55665fe740 1 --2- 172.21.15.52:0/333204720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134cf4d0 0x5569134d4ed0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.843+0000 7f55665fe740 1 --2- 172.21.15.52:0/333204720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569134cebb0 0x5569134d7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.843+0000 7f5544ff9640 0 client.16650 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:52.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.843+0000 7f55665fe740 1 -- 172.21.15.52:0/333204720 >> 172.21.15.52:0/333204720 conn(0x5569134c92e0 msgr2=0x5569134cb030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.843+0000 7f55665fe740 1 -- 172.21.15.52:0/333204720 shutdown_connections 2024-04-02T13:56:52.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.843+0000 7f55665fe740 1 -- 172.21.15.52:0/333204720 wait complete. 2024-04-02T13:56:52.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.843+0000 7f55665fe740 20 client.16650 trim_cache size 0 max 16384 2024-04-02T13:56:52.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.856+0000 7f55665fe740 1 Processor -- start 2024-04-02T13:56:52.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.856+0000 7f55665fe740 1 -- start start 2024-04-02T13:56:52.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.857+0000 7f55665fe740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134cebb0 0x5569134ea960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.857+0000 7f55665fe740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5569134cf4d0 0x5569134e3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.857+0000 7f55665fe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569134d80b0 0x5569134d7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.857+0000 7f55665fe740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f555c1acb10 con 0x5569134cebb0 2024-04-02T13:56:52.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.857+0000 7f55665fe740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f555c1ae680 con 0x5569134cf4d0 2024-04-02T13:56:52.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.857+0000 7f55665fe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f555c1ad1e0 con 0x5569134d80b0 2024-04-02T13:56:52.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.857+0000 7f5560faf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134cebb0 0x5569134ea960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.857+0000 7f5560faf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134cebb0 0x5569134ea960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52072/0 (socket says 172.21.15.52:52072) 2024-04-02T13:56:52.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.857+0000 7f5560faf640 1 -- 172.21.15.52:0/25894222 learned_addr learned my addr 172.21.15.52:0/25894222 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:52.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.857+0000 7f5560faf640 1 --2- 172.21.15.52:0/25894222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569134d80b0 0x5569134d7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.857+0000 7f5560faf640 1 -- 172.21.15.52:0/25894222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569134d80b0 msgr2=0x5569134d7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.857+0000 7f5560faf640 1 --2- 172.21.15.52:0/25894222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569134d80b0 0x5569134d7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.857+0000 7f5560faf640 1 -- 172.21.15.52:0/25894222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5569134cf4d0 msgr2=0x5569134e3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.857+0000 7f5560faf640 1 --2- 172.21.15.52:0/25894222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5569134cf4d0 0x5569134e3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.857+0000 7f5560faf640 1 -- 172.21.15.52:0/25894222 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569134cf300 con 0x5569134cebb0 2024-04-02T13:56:52.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.857+0000 7f5560faf640 1 --2- 172.21.15.52:0/25894222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134cebb0 0x5569134ea960 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f554c003e00 tx=0x7f554c003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d1e74e8c0cd1ef server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.857+0000 7f5546ffd640 1 -- 172.21.15.52:0/25894222 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c016040 con 0x5569134cebb0 2024-04-02T13:56:52.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.857+0000 7f5546ffd640 1 -- 172.21.15.52:0/25894222 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f554c003990 con 0x5569134cebb0 2024-04-02T13:56:52.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.857+0000 7f5546ffd640 1 -- 172.21.15.52:0/25894222 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c023040 con 0x5569134cebb0 2024-04-02T13:56:52.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.858+0000 7f55665fe740 1 -- 172.21.15.52:0/25894222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134cebb0 msgr2=0x5569134ea960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.858+0000 7f55665fe740 1 --2- 172.21.15.52:0/25894222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134cebb0 0x5569134ea960 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f554c003e00 tx=0x7f554c003e30 comp rx=0 tx=0).stop 2024-04-02T13:56:52.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.858+0000 7f55665fe740 1 -- 172.21.15.52:0/25894222 shutdown_connections 2024-04-02T13:56:52.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.858+0000 7f55665fe740 1 --2- 172.21.15.52:0/25894222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569134d80b0 0x5569134d7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.858+0000 7f55665fe740 1 --2- 172.21.15.52:0/25894222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5569134cf4d0 0x5569134e3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.858+0000 7f55665fe740 1 --2- 172.21.15.52:0/25894222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134cebb0 0x5569134ea960 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.858+0000 7f55665fe740 1 -- 172.21.15.52:0/25894222 >> 172.21.15.52:0/25894222 conn(0x5569134c92e0 msgr2=0x5569134d37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.858+0000 7f55665fe740 1 -- 172.21.15.52:0/25894222 shutdown_connections 2024-04-02T13:56:52.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.858+0000 7f55665fe740 1 -- 172.21.15.52:0/25894222 wait complete. 2024-04-02T13:56:52.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.858+0000 7f55665fe740 1 -- start start 2024-04-02T13:56:52.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.858+0000 7f55665fe740 1 Processor -- start 2024-04-02T13:56:52.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.859+0000 7f55665fe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569134cebb0 0x5569134d88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.859+0000 7f55665fe740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5569134cf4d0 0x5569134d8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.859+0000 7f55665fe740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134d80b0 0x5569134df2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.859+0000 7f55665fe740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f555c1adf80 con 0x5569134d80b0 2024-04-02T13:56:52.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.859+0000 7f55665fe740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f555c1ad8b0 con 0x5569134cf4d0 2024-04-02T13:56:52.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.859+0000 7f55665fe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f555c1aaaf0 con 0x5569134cebb0 2024-04-02T13:56:52.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.859+0000 7f55665fe740 10 client.0 _ll_register_callbacks cb 0x7f5554009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:52.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.859+0000 7f5560faf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134d80b0 0x5569134df2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.859+0000 7f5560faf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134d80b0 0x5569134df2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52086/0 (socket says 172.21.15.52:52086) 2024-04-02T13:56:52.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.859+0000 7f5560faf640 1 -- 172.21.15.52:0/861670298 learned_addr learned my addr 172.21.15.52:0/861670298 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:52.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.859+0000 7f55617b0640 1 --2- 172.21.15.52:0/861670298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5569134cf4d0 0x5569134d8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.859+0000 7f5560faf640 1 --2- 172.21.15.52:0/861670298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569134cebb0 0x5569134d88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.859+0000 7f5560faf640 1 -- 172.21.15.52:0/861670298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569134cebb0 msgr2=0x5569134d88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.859+0000 7f5560faf640 1 --2- 172.21.15.52:0/861670298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569134cebb0 0x5569134d88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.859+0000 7f5560faf640 1 -- 172.21.15.52:0/861670298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5569134cf4d0 msgr2=0x5569134d8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:52.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.859+0000 7f5560faf640 1 --2- 172.21.15.52:0/861670298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5569134cf4d0 0x5569134d8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.859+0000 7f5560faf640 1 -- 172.21.15.52:0/861670298 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569134d96d0 con 0x5569134d80b0 2024-04-02T13:56:52.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.859+0000 7f5560faf640 1 --2- 172.21.15.52:0/861670298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134d80b0 0x5569134df2d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f554c00bfb0 tx=0x7f554c00b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23fbdda7c93b7120 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:52.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.859+0000 7f5544ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:52.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.859+0000 7f5544ff9640 1 -- 172.21.15.52:0/861670298 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c016040 con 0x5569134d80b0 2024-04-02T13:56:52.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.860+0000 7f5544ff9640 1 -- 172.21.15.52:0/861670298 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f554c02b510 con 0x5569134d80b0 2024-04-02T13:56:52.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.860+0000 7f55665fe740 1 -- 172.21.15.52:0/861670298 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f554c003990 con 0x5569134d80b0 2024-04-02T13:56:52.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.860+0000 7f5544ff9640 1 -- 172.21.15.52:0/861670298 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f554c041050 con 0x5569134d80b0 2024-04-02T13:56:52.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.860+0000 7f55665fe740 10 client.16656 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:52.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.860+0000 7f55665fe740 1 -- 172.21.15.52:0/861670298 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5569134e08c0 con 0x5569134d80b0 2024-04-02T13:56:52.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.860+0000 7f5544ff9640 1 -- 172.21.15.52:0/861670298 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f554c003990 con 0x5569134d80b0 2024-04-02T13:56:52.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.860+0000 7f5544ff9640 1 -- 172.21.15.52:0/861670298 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f554c034070 con 0x5569134d80b0 2024-04-02T13:56:52.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.860+0000 7f55665fe740 10 client.16656 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:52.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.860+0000 7f55665fe740 10 client.16656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:52.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.860+0000 7f55665fe740 20 client.16656 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:52.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.860+0000 7f55665fe740 1 --2- 172.21.15.52:0/861670298 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55691354eb10 0x556913550f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:52.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.860+0000 7f55665fe740 4 client.16656 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:56:52.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.860+0000 7f55665fe740 1 -- 172.21.15.52:0/861670298 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556913436040 con 0x55691354eb10 2024-04-02T13:56:52.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.860+0000 7f55617b0640 1 --2- 172.21.15.52:0/861670298 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55691354eb10 0x556913550f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:52.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.861+0000 7f55617b0640 1 --2- 172.21.15.52:0/861670298 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55691354eb10 0x556913550f00 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff299a4e0c009e10 server_cookie=e0119254c2f941e6 in_seq=0 out_seq=0 2024-04-02T13:56:52.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.861+0000 7f5544ff9640 10 client.16656 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:52.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.861+0000 7f5544ff9640 1 -- 172.21.15.52:0/861670298 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556913436040 con 0x55691354eb10 2024-04-02T13:56:52.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.861+0000 7f5544ff9640 10 client.16656 handle_command_reply: tid=0 2024-04-02T13:56:52.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.861+0000 7f55665fe740 1 client.16656 shutdown 2024-04-02T13:56:52.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.861+0000 7f55665fe740 1 -- 172.21.15.52:0/861670298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134d80b0 msgr2=0x5569134df2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.861+0000 7f55665fe740 1 --2- 172.21.15.52:0/861670298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134d80b0 0x5569134df2d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f554c00bfb0 tx=0x7f554c00b610 comp rx=0 tx=0).stop 2024-04-02T13:56:52.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.861+0000 7f55665fe740 1 -- 172.21.15.52:0/861670298 shutdown_connections 2024-04-02T13:56:52.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.861+0000 7f55665fe740 1 --2- 172.21.15.52:0/861670298 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55691354eb10 0x556913550f00 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.861+0000 7f55665fe740 1 --2- 172.21.15.52:0/861670298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569134d80b0 0x5569134df2d0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.861+0000 7f55665fe740 1 --2- 172.21.15.52:0/861670298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5569134cf4d0 0x5569134d8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.861+0000 7f5544ff9640 0 client.16656 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:52.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.861+0000 7f55665fe740 1 --2- 172.21.15.52:0/861670298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569134cebb0 0x5569134d88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.861+0000 7f55665fe740 1 -- 172.21.15.52:0/861670298 >> 172.21.15.52:0/861670298 conn(0x5569134c92e0 msgr2=0x5569134d4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.861+0000 7f55665fe740 1 -- 172.21.15.52:0/861670298 shutdown_connections 2024-04-02T13:56:52.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.861+0000 7f55665fe740 1 -- 172.21.15.52:0/861670298 wait complete. 2024-04-02T13:56:52.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.861+0000 7f55665fe740 20 client.16656 trim_cache size 0 max 16384 2024-04-02T13:56:52.875 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:56:52.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.862+0000 7f5563a14640 1 -- 172.21.15.52:0/2834778929 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f55480a3da0 msgr2=0x7f55480a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.862+0000 7f5563a14640 1 --2- 172.21.15.52:0/2834778929 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f55480a3da0 0x7f55480a6260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f555809ca20 tx=0x7f555800af30 comp rx=0 tx=0).stop 2024-04-02T13:56:52.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.862+0000 7f5563a14640 1 -- 172.21.15.52:0/2834778929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f555c107aa0 msgr2=0x7f555c19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:52.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.862+0000 7f5563a14640 1 --2- 172.21.15.52:0/2834778929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f555c107aa0 0x7f555c19cc30 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f555800cce0 tx=0x7f555800a620 comp rx=0 tx=0).stop 2024-04-02T13:56:52.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.862+0000 7f5563a14640 1 -- 172.21.15.52:0/2834778929 shutdown_connections 2024-04-02T13:56:52.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.862+0000 7f5563a14640 1 --2- 172.21.15.52:0/2834778929 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f55480a3da0 0x7f55480a6260 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.862+0000 7f5563a14640 1 --2- 172.21.15.52:0/2834778929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f555c107aa0 0x7f555c19cc30 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.862+0000 7f5563a14640 1 --2- 172.21.15.52:0/2834778929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f555c102450 0x7f555c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.862+0000 7f5563a14640 1 --2- 172.21.15.52:0/2834778929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f555c101a90 0x7f555c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:52.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.862+0000 7f5563a14640 1 -- 172.21.15.52:0/2834778929 >> 172.21.15.52:0/2834778929 conn(0x7f555c078f00 msgr2=0x7f555c104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:52.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.862+0000 7f5563a14640 1 -- 172.21.15.52:0/2834778929 shutdown_connections 2024-04-02T13:56:52.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:52.862+0000 7f5563a14640 1 -- 172.21.15.52:0/2834778929 wait complete. 2024-04-02T13:56:52.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:52 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/4236803421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:56:53.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:52 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/4236803421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:56:53.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:53 smithi052 ceph-mon[32104]: pgmap v104: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-02T13:56:53.879 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:56:53.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:56:53.880 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:56:53.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:53 smithi088 ceph-mon[36528]: pgmap v104: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-02T13:56:54.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.996+0000 7f971ba0d640 1 -- 172.21.15.52:0/3890272399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9714068ce0 msgr2=0x7f9714069160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:54.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.996+0000 7f971ba0d640 1 --2- 172.21.15.52:0/3890272399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9714068ce0 0x7f9714069160 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f9704009930 tx=0x7f970402bc30 comp rx=0 tx=0).stop 2024-04-02T13:56:54.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.996+0000 7f971ba0d640 1 -- 172.21.15.52:0/3890272399 shutdown_connections 2024-04-02T13:56:54.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.996+0000 7f971ba0d640 1 --2- 172.21.15.52:0/3890272399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97140696a0 0x7f971410e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.996+0000 7f971ba0d640 1 --2- 172.21.15.52:0/3890272399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9714068ce0 0x7f9714069160 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.996+0000 7f971ba0d640 1 --2- 172.21.15.52:0/3890272399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9714107a80 0x7f9714107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.996+0000 7f971ba0d640 1 -- 172.21.15.52:0/3890272399 >> 172.21.15.52:0/3890272399 conn(0x7f9714077b00 msgr2=0x7f9714077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:54.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.996+0000 7f971ba0d640 1 -- 172.21.15.52:0/3890272399 shutdown_connections 2024-04-02T13:56:54.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.996+0000 7f971ba0d640 1 -- 172.21.15.52:0/3890272399 wait complete. 2024-04-02T13:56:54.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.996+0000 7f971ba0d640 1 Processor -- start 2024-04-02T13:56:54.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.996+0000 7f971ba0d640 1 -- start start 2024-04-02T13:56:54.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.997+0000 7f971ba0d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9714068ce0 0x7f97141983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.997+0000 7f971ba0d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f97140696a0 0x7f9714198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.997+0000 7f971ba0d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9714107a80 0x7f971419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.997+0000 7f971ba0d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9714111e90 con 0x7f9714107a80 2024-04-02T13:56:54.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.997+0000 7f971ba0d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9714111d10 con 0x7f9714068ce0 2024-04-02T13:56:54.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.997+0000 7f971ba0d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9714112010 con 0x7f97140696a0 2024-04-02T13:56:54.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.997+0000 7f9719faa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9714107a80 0x7f971419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.997+0000 7f9719faa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9714107a80 0x7f971419ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52102/0 (socket says 172.21.15.52:52102) 2024-04-02T13:56:54.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.997+0000 7f9719faa640 1 -- 172.21.15.52:0/1146550562 learned_addr learned my addr 172.21.15.52:0/1146550562 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:54.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.997+0000 7f97197a9640 1 --2- 172.21.15.52:0/1146550562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9714068ce0 0x7f97141983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.997+0000 7f9718fa8640 1 --2- 172.21.15.52:0/1146550562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f97140696a0 0x7f9714198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.997+0000 7f9719faa640 1 -- 172.21.15.52:0/1146550562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f97140696a0 msgr2=0x7f9714198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:54.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.997+0000 7f9719faa640 1 --2- 172.21.15.52:0/1146550562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f97140696a0 0x7f9714198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.997+0000 7f9719faa640 1 -- 172.21.15.52:0/1146550562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9714068ce0 msgr2=0x7f97141983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:54.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.997+0000 7f9719faa640 1 --2- 172.21.15.52:0/1146550562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9714068ce0 0x7f97141983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.997+0000 7f9719faa640 1 -- 172.21.15.52:0/1146550562 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971419d3e0 con 0x7f9714107a80 2024-04-02T13:56:54.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.997+0000 7f97197a9640 1 --2- 172.21.15.52:0/1146550562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9714068ce0 0x7f97141983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:54.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.997+0000 7f9719faa640 1 --2- 172.21.15.52:0/1146550562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9714107a80 0x7f971419ccb0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f970c009fd0 tx=0x7f970c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:54.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.997+0000 7f970a7fc640 1 -- 172.21.15.52:0/1146550562 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c017070 con 0x7f9714107a80 2024-04-02T13:56:54.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.998+0000 7f970a7fc640 1 -- 172.21.15.52:0/1146550562 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f970c003e50 con 0x7f9714107a80 2024-04-02T13:56:54.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.998+0000 7f971ba0d640 1 -- 172.21.15.52:0/1146550562 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f971419d6d0 con 0x7f9714107a80 2024-04-02T13:56:54.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.998+0000 7f970a7fc640 1 -- 172.21.15.52:0/1146550562 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c0106c0 con 0x7f9714107a80 2024-04-02T13:56:54.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:53.998+0000 7f971ba0d640 1 -- 172.21.15.52:0/1146550562 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97141a4f60 con 0x7f9714107a80 2024-04-02T13:56:54.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.000+0000 7f970a7fc640 1 -- 172.21.15.52:0/1146550562 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f970c003970 con 0x7f9714107a80 2024-04-02T13:56:54.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.000+0000 7f971e5f8740 1 Processor -- start 2024-04-02T13:56:54.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.000+0000 7f971e5f8740 1 -- start start 2024-04-02T13:56:54.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.000+0000 7f971e5f8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc273df2c0 0x55cc273df6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.000+0000 7f970a7fc640 1 --2- 172.21.15.52:0/1146550562 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f97000a3d20 0x7f97000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.000+0000 7f971e5f8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc273dfbe0 0x55cc273e8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.000+0000 7f971e5f8740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273e87c0 0x55cc273eabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.000+0000 7f971e5f8740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cc27343770 con 0x55cc273e87c0 2024-04-02T13:56:54.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.000+0000 7f971e5f8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cc272eb960 con 0x55cc273dfbe0 2024-04-02T13:56:54.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.000+0000 7f971e5f8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cc2733f8c0 con 0x55cc273df2c0 2024-04-02T13:56:54.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.001+0000 7f97197a9640 1 --2- 172.21.15.52:0/1146550562 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f97000a3d20 0x7f97000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.001+0000 7f9719faa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273e87c0 0x55cc273eabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.001+0000 7f9719faa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273e87c0 0x55cc273eabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52108/0 (socket says 172.21.15.52:52108) 2024-04-02T13:56:54.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.001+0000 7f970a7fc640 1 -- 172.21.15.52:0/1146550562 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f970c0d0970 con 0x7f9714107a80 2024-04-02T13:56:54.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.001+0000 7f9719faa640 1 -- 172.21.15.52:0/2433778601 learned_addr learned my addr 172.21.15.52:0/2433778601 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:54.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.001+0000 7f9718fa8640 1 --2- 172.21.15.52:0/2433778601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc273df2c0 0x55cc273df6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.001+0000 7f9718fa8640 1 --2- 172.21.15.52:0/2433778601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc273dfbe0 0x55cc273e8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.001+0000 7f9719faa640 1 -- 172.21.15.52:0/2433778601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc273df2c0 msgr2=0x55cc273df6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:54.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.001+0000 7f9719faa640 1 --2- 172.21.15.52:0/2433778601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc273df2c0 0x55cc273df6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.001+0000 7f9719faa640 1 -- 172.21.15.52:0/2433778601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc273dfbe0 msgr2=0x55cc273e8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:54.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.001+0000 7f9719faa640 1 --2- 172.21.15.52:0/2433778601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc273dfbe0 0x55cc273e8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.001+0000 7f9719faa640 1 -- 172.21.15.52:0/2433778601 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc2711f7e0 con 0x55cc273e87c0 2024-04-02T13:56:54.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.001+0000 7f9718fa8640 1 --2- 172.21.15.52:0/2433778601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc273dfbe0 0x55cc273e8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:54.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.001+0000 7f97197a9640 1 --2- 172.21.15.52:0/1146550562 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f97000a3d20 0x7f97000a61e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9710005fd0 tx=0x7f971000b5d0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:54.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.001+0000 7f9719faa640 1 --2- 172.21.15.52:0/2433778601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273e87c0 0x55cc273eabb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f970c005c40 tx=0x7f970c004020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e6a831b4be46d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:54.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.001+0000 7f96ff7fe640 1 -- 172.21.15.52:0/2433778601 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c020070 con 0x55cc273e87c0 2024-04-02T13:56:54.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.001+0000 7f96ff7fe640 1 -- 172.21.15.52:0/2433778601 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f970c0be0b0 con 0x55cc273e87c0 2024-04-02T13:56:54.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.001+0000 7f96ff7fe640 1 -- 172.21.15.52:0/2433778601 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c003c60 con 0x55cc273e87c0 2024-04-02T13:56:54.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.002+0000 7f971e5f8740 1 -- 172.21.15.52:0/2433778601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273e87c0 msgr2=0x55cc273eabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:54.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.002+0000 7f971e5f8740 1 --2- 172.21.15.52:0/2433778601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273e87c0 0x55cc273eabb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f970c005c40 tx=0x7f970c004020 comp rx=0 tx=0).stop 2024-04-02T13:56:54.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.002+0000 7f971e5f8740 1 -- 172.21.15.52:0/2433778601 shutdown_connections 2024-04-02T13:56:54.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.002+0000 7f971e5f8740 1 --2- 172.21.15.52:0/2433778601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273e87c0 0x55cc273eabb0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.002+0000 7f971e5f8740 1 --2- 172.21.15.52:0/2433778601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc273dfbe0 0x55cc273e8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.002+0000 7f971e5f8740 1 --2- 172.21.15.52:0/2433778601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc273df2c0 0x55cc273df6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.002+0000 7f971e5f8740 1 -- 172.21.15.52:0/2433778601 >> 172.21.15.52:0/2433778601 conn(0x55cc273d99f0 msgr2=0x55cc273d9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:54.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.002+0000 7f971e5f8740 1 -- 172.21.15.52:0/2433778601 shutdown_connections 2024-04-02T13:56:54.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.002+0000 7f971e5f8740 1 -- 172.21.15.52:0/2433778601 wait complete. 2024-04-02T13:56:54.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.002+0000 7f971e5f8740 1 -- start start 2024-04-02T13:56:54.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.003+0000 7f971e5f8740 1 Processor -- start 2024-04-02T13:56:54.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.003+0000 7f971e5f8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc273df2c0 0x55cc273e5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.003+0000 7f971e5f8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc273dfbe0 0x55cc273e5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.003+0000 7f971e5f8740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273e87c0 0x55cc273e6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.003+0000 7f971e5f8740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cc273eb4f0 con 0x55cc273e87c0 2024-04-02T13:56:54.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.003+0000 7f971e5f8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cc272fed20 con 0x55cc273df2c0 2024-04-02T13:56:54.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.003+0000 7f971e5f8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cc273eb670 con 0x55cc273dfbe0 2024-04-02T13:56:54.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.003+0000 7f971e5f8740 10 client.0 _ll_register_callbacks cb 0x55cc272cba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:54.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.003+0000 7f9719faa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273e87c0 0x55cc273e6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.003+0000 7f9719faa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273e87c0 0x55cc273e6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52114/0 (socket says 172.21.15.52:52114) 2024-04-02T13:56:54.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.003+0000 7f9719faa640 1 -- 172.21.15.52:0/2966765217 learned_addr learned my addr 172.21.15.52:0/2966765217 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:54.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.003+0000 7f9718fa8640 1 --2- 172.21.15.52:0/2966765217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc273dfbe0 0x55cc273e5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.003+0000 7f97197a9640 1 --2- 172.21.15.52:0/2966765217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc273df2c0 0x55cc273e5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.003+0000 7f9719faa640 1 -- 172.21.15.52:0/2966765217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc273dfbe0 msgr2=0x55cc273e5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:54.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.003+0000 7f9719faa640 1 --2- 172.21.15.52:0/2966765217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc273dfbe0 0x55cc273e5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.003+0000 7f9719faa640 1 -- 172.21.15.52:0/2966765217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc273df2c0 msgr2=0x55cc273e5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:54.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.003+0000 7f9719faa640 1 --2- 172.21.15.52:0/2966765217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc273df2c0 0x55cc273e5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.003+0000 7f9719faa640 1 -- 172.21.15.52:0/2966765217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc2717ff70 con 0x55cc273e87c0 2024-04-02T13:56:54.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.003+0000 7f9718fa8640 1 --2- 172.21.15.52:0/2966765217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc273dfbe0 0x55cc273e5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:54.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.004+0000 7f9719faa640 1 --2- 172.21.15.52:0/2966765217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273e87c0 0x55cc273e6290 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f970c00b760 tx=0x7f970c09aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba5a4aae125ca80 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:54.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.004+0000 7f96fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:54.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.004+0000 7f96fcff9640 1 -- 172.21.15.52:0/2966765217 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c020070 con 0x55cc273e87c0 2024-04-02T13:56:54.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.004+0000 7f96fcff9640 1 -- 172.21.15.52:0/2966765217 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f970c0ce000 con 0x55cc273e87c0 2024-04-02T13:56:54.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.004+0000 7f971e5f8740 1 -- 172.21.15.52:0/2966765217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cc27350cb0 con 0x55cc273e87c0 2024-04-02T13:56:54.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.004+0000 7f96fcff9640 1 -- 172.21.15.52:0/2966765217 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f970c0ba050 con 0x55cc273e87c0 2024-04-02T13:56:54.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.004+0000 7f971e5f8740 10 client.16668 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:54.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.004+0000 7f971e5f8740 1 -- 172.21.15.52:0/2966765217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cc27017170 con 0x55cc273e87c0 2024-04-02T13:56:54.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.004+0000 7f96fcff9640 1 -- 172.21.15.52:0/2966765217 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55cc27350cb0 con 0x55cc273e87c0 2024-04-02T13:56:54.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.004+0000 7f96fcff9640 1 -- 172.21.15.52:0/2966765217 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f970c098c40 con 0x55cc273e87c0 2024-04-02T13:56:54.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.004+0000 7f971e5f8740 10 client.16668 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:54.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.004+0000 7f971e5f8740 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:54.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.004+0000 7f971e5f8740 20 client.16668 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:54.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.004+0000 7f971e5f8740 1 --2- 172.21.15.52:0/2966765217 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cc273f8930 0x55cc273fad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.004+0000 7f971e5f8740 4 client.16668 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:54.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.004+0000 7f971e5f8740 1 -- 172.21.15.52:0/2966765217 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc2734b000 con 0x55cc273f8930 2024-04-02T13:56:54.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.005+0000 7f97197a9640 1 --2- 172.21.15.52:0/2966765217 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cc273f8930 0x55cc273fad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.005+0000 7f96fcff9640 10 client.16668 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:54.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.005+0000 7f97197a9640 1 --2- 172.21.15.52:0/2966765217 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cc273f8930 0x55cc273fad20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8cb82648e8fe94b server_cookie=4de3bc120d14f6f in_seq=0 out_seq=0 2024-04-02T13:56:54.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.006+0000 7f96fcff9640 1 -- 172.21.15.52:0/2966765217 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cc2734b000 con 0x55cc273f8930 2024-04-02T13:56:54.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.006+0000 7f96fcff9640 10 client.16668 handle_command_reply: tid=0 2024-04-02T13:56:54.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.006+0000 7f971e5f8740 1 client.16668 shutdown 2024-04-02T13:56:54.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.006+0000 7f971e5f8740 1 -- 172.21.15.52:0/2966765217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273e87c0 msgr2=0x55cc273e6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:54.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.006+0000 7f971e5f8740 1 --2- 172.21.15.52:0/2966765217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273e87c0 0x55cc273e6290 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f970c00b760 tx=0x7f970c09aae0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.006+0000 7f971e5f8740 1 -- 172.21.15.52:0/2966765217 shutdown_connections 2024-04-02T13:56:54.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.006+0000 7f971e5f8740 1 --2- 172.21.15.52:0/2966765217 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cc273f8930 0x55cc273fad20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.006+0000 7f971e5f8740 1 --2- 172.21.15.52:0/2966765217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273e87c0 0x55cc273e6290 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.006+0000 7f971e5f8740 1 --2- 172.21.15.52:0/2966765217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc273dfbe0 0x55cc273e5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.006+0000 7f96fcff9640 0 client.16668 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:54.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.006+0000 7f971e5f8740 1 --2- 172.21.15.52:0/2966765217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc273df2c0 0x55cc273e5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.006+0000 7f971e5f8740 1 -- 172.21.15.52:0/2966765217 >> 172.21.15.52:0/2966765217 conn(0x55cc273d99f0 msgr2=0x55cc273db740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:54.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.006+0000 7f971e5f8740 1 -- 172.21.15.52:0/2966765217 shutdown_connections 2024-04-02T13:56:54.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.006+0000 7f971e5f8740 1 -- 172.21.15.52:0/2966765217 wait complete. 2024-04-02T13:56:54.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.006+0000 7f971e5f8740 20 client.16668 trim_cache size 0 max 16384 2024-04-02T13:56:54.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.024+0000 7f971e5f8740 1 Processor -- start 2024-04-02T13:56:54.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.024+0000 7f971e5f8740 1 -- start start 2024-04-02T13:56:54.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.024+0000 7f971e5f8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc273df2c0 0x55cc273e7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.024+0000 7f971e5f8740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273dfbe0 0x55cc273e60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.024+0000 7f971e5f8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc273e87c0 0x55cc273fdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.024+0000 7f971e5f8740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f97141aef20 con 0x55cc273dfbe0 2024-04-02T13:56:54.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.024+0000 7f971e5f8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f97141b0a90 con 0x55cc273df2c0 2024-04-02T13:56:54.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.024+0000 7f971e5f8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f97141af5f0 con 0x55cc273e87c0 2024-04-02T13:56:54.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.024+0000 7f9718fa8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273dfbe0 0x55cc273e60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.024+0000 7f9718fa8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273dfbe0 0x55cc273e60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52122/0 (socket says 172.21.15.52:52122) 2024-04-02T13:56:54.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.024+0000 7f9718fa8640 1 -- 172.21.15.52:0/824488479 learned_addr learned my addr 172.21.15.52:0/824488479 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:54.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.024+0000 7f97197a9640 1 --2- 172.21.15.52:0/824488479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc273df2c0 0x55cc273e7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.024+0000 7f9718fa8640 1 -- 172.21.15.52:0/824488479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc273e87c0 msgr2=0x55cc273fdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:54.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.024+0000 7f9718fa8640 1 --2- 172.21.15.52:0/824488479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc273e87c0 0x55cc273fdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.024+0000 7f9718fa8640 1 -- 172.21.15.52:0/824488479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc273df2c0 msgr2=0x55cc273e7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:54.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.024+0000 7f9718fa8640 1 --2- 172.21.15.52:0/824488479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc273df2c0 0x55cc273e7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.024+0000 7f9718fa8640 1 -- 172.21.15.52:0/824488479 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc273d9820 con 0x55cc273dfbe0 2024-04-02T13:56:54.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.025+0000 7f9718fa8640 1 --2- 172.21.15.52:0/824488479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273dfbe0 0x55cc273e60b0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9704036040 tx=0x7f9704009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7662f3765b31c56f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:54.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.025+0000 7f96feffd640 1 -- 172.21.15.52:0/824488479 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970402cb00 con 0x55cc273dfbe0 2024-04-02T13:56:54.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.025+0000 7f96feffd640 1 -- 172.21.15.52:0/824488479 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f970402cca0 con 0x55cc273dfbe0 2024-04-02T13:56:54.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.025+0000 7f96feffd640 1 -- 172.21.15.52:0/824488479 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704004920 con 0x55cc273dfbe0 2024-04-02T13:56:54.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.025+0000 7f971e5f8740 1 -- 172.21.15.52:0/824488479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273dfbe0 msgr2=0x55cc273e60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:54.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.025+0000 7f971e5f8740 1 --2- 172.21.15.52:0/824488479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273dfbe0 0x55cc273e60b0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9704036040 tx=0x7f9704009260 comp rx=0 tx=0).stop 2024-04-02T13:56:54.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.025+0000 7f971e5f8740 1 -- 172.21.15.52:0/824488479 shutdown_connections 2024-04-02T13:56:54.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.025+0000 7f971e5f8740 1 --2- 172.21.15.52:0/824488479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc273e87c0 0x55cc273fdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.025+0000 7f971e5f8740 1 --2- 172.21.15.52:0/824488479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273dfbe0 0x55cc273e60b0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.025+0000 7f971e5f8740 1 --2- 172.21.15.52:0/824488479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc273df2c0 0x55cc273e7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.026+0000 7f971e5f8740 1 -- 172.21.15.52:0/824488479 >> 172.21.15.52:0/824488479 conn(0x55cc273d99f0 msgr2=0x55cc273db6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:54.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.026+0000 7f971e5f8740 1 -- 172.21.15.52:0/824488479 shutdown_connections 2024-04-02T13:56:54.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.026+0000 7f971e5f8740 1 -- 172.21.15.52:0/824488479 wait complete. 2024-04-02T13:56:54.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.026+0000 7f971e5f8740 1 -- start start 2024-04-02T13:56:54.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.026+0000 7f971e5f8740 1 Processor -- start 2024-04-02T13:56:54.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.026+0000 7f971e5f8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc273df2c0 0x55cc273fd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.026+0000 7f971e5f8740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273dfbe0 0x55cc273fdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.026+0000 7f971e5f8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc273e87c0 0x55cc273fbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.026+0000 7f971e5f8740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f97141b0390 con 0x55cc273dfbe0 2024-04-02T13:56:54.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.026+0000 7f971e5f8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f97141afcc0 con 0x55cc273df2c0 2024-04-02T13:56:54.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.026+0000 7f971e5f8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f97141acf00 con 0x55cc273e87c0 2024-04-02T13:56:54.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.026+0000 7f971e5f8740 10 client.0 _ll_register_callbacks cb 0x7f970c00ec00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:54.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.026+0000 7f9718fa8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273dfbe0 0x55cc273fdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.026+0000 7f9718fa8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273dfbe0 0x55cc273fdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52124/0 (socket says 172.21.15.52:52124) 2024-04-02T13:56:54.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.026+0000 7f9718fa8640 1 -- 172.21.15.52:0/568728127 learned_addr learned my addr 172.21.15.52:0/568728127 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:54.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.026+0000 7f97197a9640 1 --2- 172.21.15.52:0/568728127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc273df2c0 0x55cc273fd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.027+0000 7f9719faa640 1 --2- 172.21.15.52:0/568728127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc273e87c0 0x55cc273fbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.027+0000 7f9718fa8640 1 -- 172.21.15.52:0/568728127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc273e87c0 msgr2=0x55cc273fbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:54.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.027+0000 7f9718fa8640 1 --2- 172.21.15.52:0/568728127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc273e87c0 0x55cc273fbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.027+0000 7f9718fa8640 1 -- 172.21.15.52:0/568728127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc273df2c0 msgr2=0x55cc273fd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:54.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.027+0000 7f9718fa8640 1 --2- 172.21.15.52:0/568728127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc273df2c0 0x55cc273fd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.027+0000 7f9718fa8640 1 -- 172.21.15.52:0/568728127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc273fc620 con 0x55cc273dfbe0 2024-04-02T13:56:54.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.027+0000 7f9719faa640 1 --2- 172.21.15.52:0/568728127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc273e87c0 0x55cc273fbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:56:54.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.027+0000 7f9718fa8640 1 --2- 172.21.15.52:0/568728127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273dfbe0 0x55cc273fdef0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f970402c170 tx=0x7f9704005380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b653cc5195db3e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:54.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.027+0000 7f96fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:54.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.027+0000 7f96fcff9640 1 -- 172.21.15.52:0/568728127 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704051070 con 0x55cc273dfbe0 2024-04-02T13:56:54.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.027+0000 7f96fcff9640 1 -- 172.21.15.52:0/568728127 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f9704004450 con 0x55cc273dfbe0 2024-04-02T13:56:54.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.027+0000 7f971e5f8740 1 -- 172.21.15.52:0/568728127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f970402cca0 con 0x55cc273dfbe0 2024-04-02T13:56:54.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.027+0000 7f96fcff9640 1 -- 172.21.15.52:0/568728127 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f970405d050 con 0x55cc273dfbe0 2024-04-02T13:56:54.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.027+0000 7f971e5f8740 10 client.16674 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:54.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.027+0000 7f971e5f8740 1 -- 172.21.15.52:0/568728127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cc2745f630 con 0x55cc273dfbe0 2024-04-02T13:56:54.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.027+0000 7f96fcff9640 1 -- 172.21.15.52:0/568728127 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f970402cca0 con 0x55cc273dfbe0 2024-04-02T13:56:54.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.028+0000 7f96fcff9640 1 -- 172.21.15.52:0/568728127 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f97040038c0 con 0x55cc273dfbe0 2024-04-02T13:56:54.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.028+0000 7f971e5f8740 10 client.16674 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:54.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.028+0000 7f971e5f8740 10 client.16674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:54.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.028+0000 7f971e5f8740 20 client.16674 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:54.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.028+0000 7f971e5f8740 1 --2- 172.21.15.52:0/568728127 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cc2745fd50 0x55cc27462170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.028+0000 7f971e5f8740 4 client.16674 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:56:54.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.028+0000 7f971e5f8740 1 -- 172.21.15.52:0/568728127 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cc272c8be0 con 0x55cc2745fd50 2024-04-02T13:56:54.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.028+0000 7f97197a9640 1 --2- 172.21.15.52:0/568728127 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cc2745fd50 0x55cc27462170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.028+0000 7f96fcff9640 10 client.16674 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:54.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.028+0000 7f97197a9640 1 --2- 172.21.15.52:0/568728127 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cc2745fd50 0x55cc27462170 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=416ef650321f6cd7 server_cookie=6e331e8c036a31c0 in_seq=0 out_seq=0 2024-04-02T13:56:54.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.029+0000 7f96fcff9640 1 -- 172.21.15.52:0/568728127 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cc272c8be0 con 0x55cc2745fd50 2024-04-02T13:56:54.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.029+0000 7f96fcff9640 10 client.16674 handle_command_reply: tid=0 2024-04-02T13:56:54.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.029+0000 7f971e5f8740 1 client.16674 shutdown 2024-04-02T13:56:54.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.030+0000 7f971e5f8740 1 -- 172.21.15.52:0/568728127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273dfbe0 msgr2=0x55cc273fdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:54.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.030+0000 7f971e5f8740 1 --2- 172.21.15.52:0/568728127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273dfbe0 0x55cc273fdef0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f970402c170 tx=0x7f9704005380 comp rx=0 tx=0).stop 2024-04-02T13:56:54.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.030+0000 7f971e5f8740 1 -- 172.21.15.52:0/568728127 shutdown_connections 2024-04-02T13:56:54.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.030+0000 7f971e5f8740 1 --2- 172.21.15.52:0/568728127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc273e87c0 0x55cc273fbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.030+0000 7f971e5f8740 1 --2- 172.21.15.52:0/568728127 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cc2745fd50 0x55cc27462170 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.030+0000 7f971e5f8740 1 --2- 172.21.15.52:0/568728127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cc273dfbe0 0x55cc273fdef0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.030+0000 7f971e5f8740 1 --2- 172.21.15.52:0/568728127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc273df2c0 0x55cc273fd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.030+0000 7f971e5f8740 1 -- 172.21.15.52:0/568728127 >> 172.21.15.52:0/568728127 conn(0x55cc273d99f0 msgr2=0x55cc273d8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:54.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.030+0000 7f96fcff9640 0 client.16674 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:54.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.030+0000 7f971e5f8740 1 -- 172.21.15.52:0/568728127 shutdown_connections 2024-04-02T13:56:54.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.030+0000 7f971e5f8740 1 -- 172.21.15.52:0/568728127 wait complete. 2024-04-02T13:56:54.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.030+0000 7f971e5f8740 20 client.16674 trim_cache size 0 max 16384 2024-04-02T13:56:54.043 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:56:54.043 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-04-02T13:56:54.044 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "8994c12c-13b6-4046-8543-05f1662e4793", 2024-04-02T13:56:54.044 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-04-02T13:56:54.044 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:56:54.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.030+0000 7f971ba0d640 1 -- 172.21.15.52:0/1146550562 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f97000a3d20 msgr2=0x7f97000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:54.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.030+0000 7f971ba0d640 1 --2- 172.21.15.52:0/1146550562 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f97000a3d20 0x7f97000a61e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9710005fd0 tx=0x7f971000b5d0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.030+0000 7f971ba0d640 1 -- 172.21.15.52:0/1146550562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9714107a80 msgr2=0x7f971419ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:54.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.030+0000 7f971ba0d640 1 --2- 172.21.15.52:0/1146550562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9714107a80 0x7f971419ccb0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f970c009fd0 tx=0x7f970c00b260 comp rx=0 tx=0).stop 2024-04-02T13:56:54.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.030+0000 7f971ba0d640 1 -- 172.21.15.52:0/1146550562 shutdown_connections 2024-04-02T13:56:54.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.030+0000 7f971ba0d640 1 --2- 172.21.15.52:0/1146550562 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f97000a3d20 0x7f97000a61e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.031+0000 7f971ba0d640 1 --2- 172.21.15.52:0/1146550562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9714107a80 0x7f971419ccb0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.031+0000 7f971ba0d640 1 --2- 172.21.15.52:0/1146550562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f97140696a0 0x7f9714198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.031+0000 7f971ba0d640 1 --2- 172.21.15.52:0/1146550562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9714068ce0 0x7f97141983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.031+0000 7f971ba0d640 1 -- 172.21.15.52:0/1146550562 >> 172.21.15.52:0/1146550562 conn(0x7f9714077b00 msgr2=0x7f97140fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:54.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.031+0000 7f971ba0d640 1 -- 172.21.15.52:0/1146550562 shutdown_connections 2024-04-02T13:56:54.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.031+0000 7f971ba0d640 1 -- 172.21.15.52:0/1146550562 wait complete. 2024-04-02T13:56:54.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:56:54.047 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:56:54.048 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:53 smithi189 ceph-mon[36683]: pgmap v104: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-02T13:56:54.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.152+0000 7f7375ee4640 1 -- 172.21.15.52:0/461343232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7370106980 msgr2=0x7f7370106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:54.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.152+0000 7f7375ee4640 1 --2- 172.21.15.52:0/461343232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7370106980 0x7f7370106d60 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7360009930 tx=0x7f736002bc30 comp rx=0 tx=0).stop 2024-04-02T13:56:54.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.152+0000 7f7375ee4640 1 -- 172.21.15.52:0/461343232 shutdown_connections 2024-04-02T13:56:54.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.152+0000 7f7375ee4640 1 --2- 172.21.15.52:0/461343232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7370101330 0x7f737010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.152+0000 7f7375ee4640 1 --2- 172.21.15.52:0/461343232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7370100970 0x7f7370100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.152+0000 7f7375ee4640 1 --2- 172.21.15.52:0/461343232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7370106980 0x7f7370106d60 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.152+0000 7f7375ee4640 1 -- 172.21.15.52:0/461343232 >> 172.21.15.52:0/461343232 conn(0x7f7370077b00 msgr2=0x7f7370077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:54.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.152+0000 7f7375ee4640 1 -- 172.21.15.52:0/461343232 shutdown_connections 2024-04-02T13:56:54.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.152+0000 7f7375ee4640 1 -- 172.21.15.52:0/461343232 wait complete. 2024-04-02T13:56:54.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.153+0000 7f7375ee4640 1 Processor -- start 2024-04-02T13:56:54.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.153+0000 7f7375ee4640 1 -- start start 2024-04-02T13:56:54.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.153+0000 7f7375ee4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7370100970 0x7f737019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.153+0000 7f7375ee4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7370101330 0x7f737019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.153+0000 7f7375ee4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7370106980 0x7f737019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.153+0000 7f7375ee4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7370110d90 con 0x7f7370100970 2024-04-02T13:56:54.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.153+0000 7f7375ee4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7370110c10 con 0x7f7370101330 2024-04-02T13:56:54.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.153+0000 7f7375ee4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7370110f10 con 0x7f7370106980 2024-04-02T13:56:54.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.153+0000 7f736f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7370100970 0x7f737019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.153+0000 7f736f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7370100970 0x7f737019a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52146/0 (socket says 172.21.15.52:52146) 2024-04-02T13:56:54.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.153+0000 7f736f7fe640 1 -- 172.21.15.52:0/1888461599 learned_addr learned my addr 172.21.15.52:0/1888461599 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:54.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.153+0000 7f7366ffd640 1 --2- 172.21.15.52:0/1888461599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7370101330 0x7f737019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.153+0000 7f736ffff640 1 --2- 172.21.15.52:0/1888461599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7370106980 0x7f737019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.154+0000 7f736f7fe640 1 -- 172.21.15.52:0/1888461599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7370106980 msgr2=0x7f737019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:54.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.154+0000 7f736f7fe640 1 --2- 172.21.15.52:0/1888461599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7370106980 0x7f737019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.154+0000 7f736f7fe640 1 -- 172.21.15.52:0/1888461599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7370101330 msgr2=0x7f737019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:54.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.154+0000 7f736f7fe640 1 --2- 172.21.15.52:0/1888461599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7370101330 0x7f737019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.154+0000 7f736f7fe640 1 -- 172.21.15.52:0/1888461599 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737019f5f0 con 0x7f7370100970 2024-04-02T13:56:54.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.154+0000 7f736f7fe640 1 --2- 172.21.15.52:0/1888461599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7370100970 0x7f737019a650 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f7360009230 tx=0x7f736002cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:54.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.154+0000 7f736d7fa640 1 -- 172.21.15.52:0/1888461599 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360041070 con 0x7f7370100970 2024-04-02T13:56:54.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.154+0000 7f736d7fa640 1 -- 172.21.15.52:0/1888461599 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7360004450 con 0x7f7370100970 2024-04-02T13:56:54.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.154+0000 7f7375ee4640 1 -- 172.21.15.52:0/1888461599 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f737019f880 con 0x7f7370100970 2024-04-02T13:56:54.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.154+0000 7f736d7fa640 1 -- 172.21.15.52:0/1888461599 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360003de0 con 0x7f7370100970 2024-04-02T13:56:54.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.154+0000 7f7375ee4640 1 -- 172.21.15.52:0/1888461599 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73701a7170 con 0x7f7370100970 2024-04-02T13:56:54.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.157+0000 7f736d7fa640 1 -- 172.21.15.52:0/1888461599 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7360044070 con 0x7f7370100970 2024-04-02T13:56:54.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.157+0000 7f7378ba5740 1 Processor -- start 2024-04-02T13:56:54.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.157+0000 7f7378ba5740 1 -- start start 2024-04-02T13:56:54.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.157+0000 7f7378ba5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e8cdb18d0 0x559e8cdb1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.157+0000 7f7378ba5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e8cdb21f0 0x559e8cdba890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.157+0000 7f736d7fa640 1 --2- 172.21.15.52:0/1888461599 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f735c0a3d20 0x7f735c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.157+0000 7f7378ba5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdbadd0 0x559e8cdbd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.157+0000 7f7378ba5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559e8cd15830 con 0x559e8cdbadd0 2024-04-02T13:56:54.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.157+0000 7f7378ba5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559e8ccbd820 con 0x559e8cdb18d0 2024-04-02T13:56:54.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.157+0000 7f7378ba5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559e8cd117e0 con 0x559e8cdb21f0 2024-04-02T13:56:54.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.157+0000 7f736ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdbadd0 0x559e8cdbd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.157+0000 7f736ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdbadd0 0x559e8cdbd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52148/0 (socket says 172.21.15.52:52148) 2024-04-02T13:56:54.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.157+0000 7f736ffff640 1 -- 172.21.15.52:0/2031177983 learned_addr learned my addr 172.21.15.52:0/2031177983 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:54.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.157+0000 7f7366ffd640 1 --2- 172.21.15.52:0/1888461599 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f735c0a3d20 0x7f735c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.157+0000 7f736d7fa640 1 -- 172.21.15.52:0/1888461599 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f73600f2360 con 0x7f7370100970 2024-04-02T13:56:54.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.157+0000 7f736f7fe640 1 --2- 172.21.15.52:0/2031177983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e8cdb21f0 0x559e8cdba890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.157+0000 7f736ffff640 1 --2- 172.21.15.52:0/2031177983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e8cdb18d0 0x559e8cdb1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.158+0000 7f736ffff640 1 -- 172.21.15.52:0/2031177983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e8cdb21f0 msgr2=0x559e8cdba890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:54.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.158+0000 7f736ffff640 1 --2- 172.21.15.52:0/2031177983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e8cdb21f0 0x559e8cdba890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.158+0000 7f736ffff640 1 -- 172.21.15.52:0/2031177983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e8cdb18d0 msgr2=0x559e8cdb1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:54.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.158+0000 7f736ffff640 1 --2- 172.21.15.52:0/2031177983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e8cdb18d0 0x559e8cdb1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.158+0000 7f736ffff640 1 -- 172.21.15.52:0/2031177983 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e8caf37d0 con 0x559e8cdbadd0 2024-04-02T13:56:54.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.158+0000 7f736f7fe640 1 --2- 172.21.15.52:0/2031177983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e8cdb21f0 0x559e8cdba890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:54.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.158+0000 7f7366ffd640 1 --2- 172.21.15.52:0/1888461599 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f735c0a3d20 0x7f735c0a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f737010d740 tx=0x7f7358005f90 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:54.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.158+0000 7f736ffff640 1 --2- 172.21.15.52:0/2031177983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdbadd0 0x559e8cdbd1c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f7368011ad0 tx=0x7f7368004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45a1a07dd60b690 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:54.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.158+0000 7f7365ffb640 1 -- 172.21.15.52:0/2031177983 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736800f7c0 con 0x559e8cdbadd0 2024-04-02T13:56:54.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.158+0000 7f7365ffb640 1 -- 172.21.15.52:0/2031177983 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f736800fe40 con 0x559e8cdbadd0 2024-04-02T13:56:54.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.158+0000 7f7365ffb640 1 -- 172.21.15.52:0/2031177983 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736801fe00 con 0x559e8cdbadd0 2024-04-02T13:56:54.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.159+0000 7f7378ba5740 1 -- 172.21.15.52:0/2031177983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdbadd0 msgr2=0x559e8cdbd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:54.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.159+0000 7f7378ba5740 1 --2- 172.21.15.52:0/2031177983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdbadd0 0x559e8cdbd1c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f7368011ad0 tx=0x7f7368004ab0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.159+0000 7f7378ba5740 1 -- 172.21.15.52:0/2031177983 shutdown_connections 2024-04-02T13:56:54.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.159+0000 7f7378ba5740 1 --2- 172.21.15.52:0/2031177983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdbadd0 0x559e8cdbd1c0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.159+0000 7f7378ba5740 1 --2- 172.21.15.52:0/2031177983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e8cdb21f0 0x559e8cdba890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.159+0000 7f7378ba5740 1 --2- 172.21.15.52:0/2031177983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e8cdb18d0 0x559e8cdb1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.159+0000 7f7378ba5740 1 -- 172.21.15.52:0/2031177983 >> 172.21.15.52:0/2031177983 conn(0x559e8cdac000 msgr2=0x559e8cdac3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:54.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.159+0000 7f7378ba5740 1 -- 172.21.15.52:0/2031177983 shutdown_connections 2024-04-02T13:56:54.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.159+0000 7f7378ba5740 1 -- 172.21.15.52:0/2031177983 wait complete. 2024-04-02T13:56:54.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.159+0000 7f7378ba5740 1 -- start start 2024-04-02T13:56:54.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.159+0000 7f7378ba5740 1 Processor -- start 2024-04-02T13:56:54.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.160+0000 7f7378ba5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e8cdb18d0 0x559e8cdba720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.160+0000 7f7378ba5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e8cdb21f0 0x559e8cdb7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.160+0000 7f7378ba5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdbadd0 0x559e8cdb8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.160+0000 7f7378ba5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559e8cdbdbc0 con 0x559e8cdbadd0 2024-04-02T13:56:54.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.160+0000 7f7378ba5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559e8ccd0d30 con 0x559e8cdb21f0 2024-04-02T13:56:54.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.160+0000 7f7378ba5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559e8cdbdd40 con 0x559e8cdb18d0 2024-04-02T13:56:54.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.160+0000 7f7378ba5740 10 client.0 _ll_register_callbacks cb 0x559e8ccd0720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:54.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.160+0000 7f736ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdbadd0 0x559e8cdb8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.160+0000 7f7366ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e8cdb21f0 0x559e8cdb7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.160+0000 7f736ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdbadd0 0x559e8cdb8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52164/0 (socket says 172.21.15.52:52164) 2024-04-02T13:56:54.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.160+0000 7f736ffff640 1 -- 172.21.15.52:0/3798817151 learned_addr learned my addr 172.21.15.52:0/3798817151 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:54.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.160+0000 7f736f7fe640 1 --2- 172.21.15.52:0/3798817151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e8cdb18d0 0x559e8cdba720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.160+0000 7f736ffff640 1 -- 172.21.15.52:0/3798817151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e8cdb18d0 msgr2=0x559e8cdba720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:54.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.160+0000 7f736ffff640 1 --2- 172.21.15.52:0/3798817151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e8cdb18d0 0x559e8cdba720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.160+0000 7f736ffff640 1 -- 172.21.15.52:0/3798817151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e8cdb21f0 msgr2=0x559e8cdb7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:54.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.160+0000 7f736ffff640 1 --2- 172.21.15.52:0/3798817151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e8cdb21f0 0x559e8cdb7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.160+0000 7f736ffff640 1 -- 172.21.15.52:0/3798817151 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e8ca293f0 con 0x559e8cdbadd0 2024-04-02T13:56:54.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.160+0000 7f736ffff640 1 --2- 172.21.15.52:0/3798817151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdbadd0 0x559e8cdb8010 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f7368010010 tx=0x7f73680021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45dc384a627771a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:54.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.160+0000 7f734effd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:54.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.161+0000 7f734effd640 1 -- 172.21.15.52:0/3798817151 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368011030 con 0x559e8cdbadd0 2024-04-02T13:56:54.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.161+0000 7f734effd640 1 -- 172.21.15.52:0/3798817151 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f73680111d0 con 0x559e8cdbadd0 2024-04-02T13:56:54.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.161+0000 7f7378ba5740 1 -- 172.21.15.52:0/3798817151 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f736800fe40 con 0x559e8cdbadd0 2024-04-02T13:56:54.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.161+0000 7f734effd640 1 -- 172.21.15.52:0/3798817151 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f736800f790 con 0x559e8cdbadd0 2024-04-02T13:56:54.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.161+0000 7f7378ba5740 10 client.16686 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:54.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.161+0000 7f7378ba5740 1 -- 172.21.15.52:0/3798817151 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559e8c9e90f0 con 0x559e8cdbadd0 2024-04-02T13:56:54.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.161+0000 7f734effd640 1 -- 172.21.15.52:0/3798817151 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f736800fe40 con 0x559e8cdbadd0 2024-04-02T13:56:54.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.161+0000 7f734effd640 1 -- 172.21.15.52:0/3798817151 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f7368026070 con 0x559e8cdbadd0 2024-04-02T13:56:54.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.161+0000 7f7378ba5740 10 client.16686 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:54.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.161+0000 7f7378ba5740 10 client.16686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:54.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.161+0000 7f7378ba5740 20 client.16686 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:54.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.161+0000 7f7378ba5740 1 --2- 172.21.15.52:0/3798817151 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x559e8cdcacb0 0x559e8cdcd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.161+0000 7f7378ba5740 4 client.16686 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:54.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.161+0000 7f7378ba5740 1 -- 172.21.15.52:0/3798817151 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e8cd22d70 con 0x559e8cdcacb0 2024-04-02T13:56:54.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.162+0000 7f736f7fe640 1 --2- 172.21.15.52:0/3798817151 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x559e8cdcacb0 0x559e8cdcd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.162+0000 7f736f7fe640 1 --2- 172.21.15.52:0/3798817151 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x559e8cdcacb0 0x559e8cdcd0a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a601790b3d6055a6 server_cookie=5cfe9d982c3ffd92 in_seq=0 out_seq=0 2024-04-02T13:56:54.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.162+0000 7f734effd640 10 client.16686 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:54.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.163+0000 7f734effd640 1 -- 172.21.15.52:0/3798817151 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e8cd22d70 con 0x559e8cdcacb0 2024-04-02T13:56:54.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.163+0000 7f734effd640 10 client.16686 handle_command_reply: tid=0 2024-04-02T13:56:54.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.163+0000 7f7378ba5740 1 client.16686 shutdown 2024-04-02T13:56:54.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.163+0000 7f7378ba5740 1 -- 172.21.15.52:0/3798817151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdbadd0 msgr2=0x559e8cdb8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:54.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.163+0000 7f7378ba5740 1 --2- 172.21.15.52:0/3798817151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdbadd0 0x559e8cdb8010 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f7368010010 tx=0x7f73680021d0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.163+0000 7f7378ba5740 1 -- 172.21.15.52:0/3798817151 shutdown_connections 2024-04-02T13:56:54.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.163+0000 7f7378ba5740 1 --2- 172.21.15.52:0/3798817151 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x559e8cdcacb0 0x559e8cdcd0a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.163+0000 7f7378ba5740 1 --2- 172.21.15.52:0/3798817151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdbadd0 0x559e8cdb8010 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.163+0000 7f7378ba5740 1 --2- 172.21.15.52:0/3798817151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e8cdb21f0 0x559e8cdb7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.163+0000 7f7378ba5740 1 --2- 172.21.15.52:0/3798817151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e8cdb18d0 0x559e8cdba720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.163+0000 7f7378ba5740 1 -- 172.21.15.52:0/3798817151 >> 172.21.15.52:0/3798817151 conn(0x559e8cdac000 msgr2=0x559e8cdadd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:54.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.163+0000 7f734effd640 0 client.16686 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:54.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.163+0000 7f7378ba5740 1 -- 172.21.15.52:0/3798817151 shutdown_connections 2024-04-02T13:56:54.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.163+0000 7f7378ba5740 1 -- 172.21.15.52:0/3798817151 wait complete. 2024-04-02T13:56:54.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.163+0000 7f7378ba5740 20 client.16686 trim_cache size 0 max 16384 2024-04-02T13:56:54.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.185+0000 7f7378ba5740 1 Processor -- start 2024-04-02T13:56:54.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.185+0000 7f7378ba5740 1 -- start start 2024-04-02T13:56:54.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.185+0000 7f7378ba5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e8cdac000 0x559e8cdc6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.185+0000 7f7378ba5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e8cdb18d0 0x559e8cdba1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.185+0000 7f7378ba5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdb21f0 0x559e8cdba720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.185+0000 7f7378ba5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f73701b1080 con 0x559e8cdb21f0 2024-04-02T13:56:54.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.185+0000 7f7378ba5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f73701b2bf0 con 0x559e8cdb18d0 2024-04-02T13:56:54.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.185+0000 7f7378ba5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f73701b1750 con 0x559e8cdac000 2024-04-02T13:56:54.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.185+0000 7f7366ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e8cdb18d0 0x559e8cdba1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.185+0000 7f736ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdb21f0 0x559e8cdba720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.185+0000 7f7366ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e8cdb18d0 0x559e8cdba1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:59298/0 (socket says 172.21.15.52:59298) 2024-04-02T13:56:54.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.185+0000 7f7366ffd640 1 -- 172.21.15.52:0/3564990338 learned_addr learned my addr 172.21.15.52:0/3564990338 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:54.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.185+0000 7f736f7fe640 1 --2- 172.21.15.52:0/3564990338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e8cdac000 0x559e8cdc6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.185+0000 7f736ffff640 1 -- 172.21.15.52:0/3564990338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e8cdac000 msgr2=0x559e8cdc6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:54.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.185+0000 7f736ffff640 1 --2- 172.21.15.52:0/3564990338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e8cdac000 0x559e8cdc6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.185+0000 7f736ffff640 1 -- 172.21.15.52:0/3564990338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e8cdb18d0 msgr2=0x559e8cdba1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:54.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.185+0000 7f736ffff640 1 --2- 172.21.15.52:0/3564990338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e8cdb18d0 0x559e8cdba1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.185+0000 7f736ffff640 1 -- 172.21.15.52:0/3564990338 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e8cdb2020 con 0x559e8cdb21f0 2024-04-02T13:56:54.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.186+0000 7f736ffff640 1 --2- 172.21.15.52:0/3564990338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdb21f0 0x559e8cdba720 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f736801fab0 tx=0x7f7368009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d109d6b475b90fb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:54.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.186+0000 7f73657fa640 1 -- 172.21.15.52:0/3564990338 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736801f850 con 0x559e8cdb21f0 2024-04-02T13:56:54.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.186+0000 7f73657fa640 1 -- 172.21.15.52:0/3564990338 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7368005bd0 con 0x559e8cdb21f0 2024-04-02T13:56:54.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.186+0000 7f73657fa640 1 -- 172.21.15.52:0/3564990338 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736800f930 con 0x559e8cdb21f0 2024-04-02T13:56:54.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.187+0000 7f7378ba5740 1 -- 172.21.15.52:0/3564990338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdb21f0 msgr2=0x559e8cdba720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:54.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.187+0000 7f7378ba5740 1 --2- 172.21.15.52:0/3564990338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdb21f0 0x559e8cdba720 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f736801fab0 tx=0x7f7368009bf0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.187+0000 7f7378ba5740 1 -- 172.21.15.52:0/3564990338 shutdown_connections 2024-04-02T13:56:54.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.187+0000 7f7378ba5740 1 --2- 172.21.15.52:0/3564990338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdb21f0 0x559e8cdba720 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.187+0000 7f7378ba5740 1 --2- 172.21.15.52:0/3564990338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e8cdb18d0 0x559e8cdba1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.187+0000 7f7378ba5740 1 --2- 172.21.15.52:0/3564990338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e8cdac000 0x559e8cdc6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.187+0000 7f7378ba5740 1 -- 172.21.15.52:0/3564990338 >> 172.21.15.52:0/3564990338 conn(0x559e8cdcd410 msgr2=0x559e8cdb30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:54.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.187+0000 7f7378ba5740 1 -- 172.21.15.52:0/3564990338 shutdown_connections 2024-04-02T13:56:54.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.187+0000 7f7378ba5740 1 -- 172.21.15.52:0/3564990338 wait complete. 2024-04-02T13:56:54.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.187+0000 7f7378ba5740 1 -- start start 2024-04-02T13:56:54.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.187+0000 7f7378ba5740 1 Processor -- start 2024-04-02T13:56:54.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.188+0000 7f7378ba5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdac000 0x559e8cdc3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.188+0000 7f7378ba5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e8cdb18d0 0x559e8cdbe570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.188+0000 7f7378ba5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e8cdb21f0 0x559e8cdbeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.188+0000 7f7378ba5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f73701b24f0 con 0x559e8cdac000 2024-04-02T13:56:54.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.188+0000 7f7378ba5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f73701b1e20 con 0x559e8cdb18d0 2024-04-02T13:56:54.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.188+0000 7f7378ba5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f73701af060 con 0x559e8cdb21f0 2024-04-02T13:56:54.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.188+0000 7f7378ba5740 10 client.0 _ll_register_callbacks cb 0x7f736801ff10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:54.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.188+0000 7f736f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdac000 0x559e8cdc3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.188+0000 7f7366ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e8cdb18d0 0x559e8cdbe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.188+0000 7f736f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdac000 0x559e8cdc3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52182/0 (socket says 172.21.15.52:52182) 2024-04-02T13:56:54.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.188+0000 7f736f7fe640 1 -- 172.21.15.52:0/4078944492 learned_addr learned my addr 172.21.15.52:0/4078944492 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:56:54.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.188+0000 7f736ffff640 1 --2- 172.21.15.52:0/4078944492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e8cdb21f0 0x559e8cdbeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.188+0000 7f736f7fe640 1 -- 172.21.15.52:0/4078944492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e8cdb21f0 msgr2=0x559e8cdbeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:54.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.188+0000 7f736f7fe640 1 --2- 172.21.15.52:0/4078944492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e8cdb21f0 0x559e8cdbeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.188+0000 7f736f7fe640 1 -- 172.21.15.52:0/4078944492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e8cdb18d0 msgr2=0x559e8cdbe570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:54.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.188+0000 7f736f7fe640 1 --2- 172.21.15.52:0/4078944492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e8cdb18d0 0x559e8cdbe570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.188+0000 7f736f7fe640 1 -- 172.21.15.52:0/4078944492 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e8cdbeff0 con 0x559e8cdac000 2024-04-02T13:56:54.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.188+0000 7f736f7fe640 1 --2- 172.21.15.52:0/4078944492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdac000 0x559e8cdc3060 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f73600be060 tx=0x7f7360009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9dd15a3dc031a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:54.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.188+0000 7f734effd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:56:54.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.188+0000 7f734effd640 1 -- 172.21.15.52:0/4078944492 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736004b040 con 0x559e8cdac000 2024-04-02T13:56:54.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.189+0000 7f734effd640 1 -- 172.21.15.52:0/4078944492 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f73600035f0 con 0x559e8cdac000 2024-04-02T13:56:54.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.189+0000 7f7378ba5740 1 -- 172.21.15.52:0/4078944492 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7368005bd0 con 0x559e8cdac000 2024-04-02T13:56:54.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.189+0000 7f734effd640 1 -- 172.21.15.52:0/4078944492 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f73600be7d0 con 0x559e8cdac000 2024-04-02T13:56:54.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.189+0000 7f7378ba5740 10 client.16692 fetch_fsmap learned FSMap version 29 2024-04-02T13:56:54.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.189+0000 7f7378ba5740 1 -- 172.21.15.52:0/4078944492 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559e8ce30e60 con 0x559e8cdac000 2024-04-02T13:56:54.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.189+0000 7f734effd640 1 -- 172.21.15.52:0/4078944492 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7368005bd0 con 0x559e8cdac000 2024-04-02T13:56:54.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.189+0000 7f734effd640 1 -- 172.21.15.52:0/4078944492 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f73600ef9f0 con 0x559e8cdac000 2024-04-02T13:56:54.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.189+0000 7f7378ba5740 10 client.16692 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:56:54.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.189+0000 7f7378ba5740 10 client.16692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:54.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.189+0000 7f7378ba5740 20 client.16692 populate_metadata read hostname 'smithi052' 2024-04-02T13:56:54.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.189+0000 7f7378ba5740 1 --2- 172.21.15.52:0/4078944492 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x559e8cdbadd0 0x559e8ce33600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:54.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.189+0000 7f7378ba5740 4 client.16692 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:56:54.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.189+0000 7f7378ba5740 1 -- 172.21.15.52:0/4078944492 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559e8cd1d0c0 con 0x559e8cdbadd0 2024-04-02T13:56:54.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.189+0000 7f7366ffd640 1 --2- 172.21.15.52:0/4078944492 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x559e8cdbadd0 0x559e8ce33600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:54.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.190+0000 7f7366ffd640 1 --2- 172.21.15.52:0/4078944492 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x559e8cdbadd0 0x559e8ce33600 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c525a1113e4f9ca0 server_cookie=c56d4dbd8063cc47 in_seq=0 out_seq=0 2024-04-02T13:56:54.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.190+0000 7f734effd640 10 client.16692 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:54.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.190+0000 7f734effd640 1 -- 172.21.15.52:0/4078944492 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x559e8cd1d0c0 con 0x559e8cdbadd0 2024-04-02T13:56:54.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.190+0000 7f734effd640 10 client.16692 handle_command_reply: tid=0 2024-04-02T13:56:54.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.190+0000 7f7378ba5740 1 client.16692 shutdown 2024-04-02T13:56:54.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.190+0000 7f7378ba5740 1 -- 172.21.15.52:0/4078944492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdac000 msgr2=0x559e8cdc3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:54.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.190+0000 7f7378ba5740 1 --2- 172.21.15.52:0/4078944492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdac000 0x559e8cdc3060 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f73600be060 tx=0x7f7360009260 comp rx=0 tx=0).stop 2024-04-02T13:56:54.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.190+0000 7f7378ba5740 1 -- 172.21.15.52:0/4078944492 shutdown_connections 2024-04-02T13:56:54.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.190+0000 7f7378ba5740 1 --2- 172.21.15.52:0/4078944492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e8cdb21f0 0x559e8cdbeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.190+0000 7f7378ba5740 1 --2- 172.21.15.52:0/4078944492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e8cdb18d0 0x559e8cdbe570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.190+0000 7f7378ba5740 1 --2- 172.21.15.52:0/4078944492 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x559e8cdbadd0 0x559e8ce33600 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.190+0000 7f7378ba5740 1 --2- 172.21.15.52:0/4078944492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e8cdac000 0x559e8cdc3060 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.190+0000 7f734effd640 0 client.16692 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:56:54.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.190+0000 7f7378ba5740 1 -- 172.21.15.52:0/4078944492 >> 172.21.15.52:0/4078944492 conn(0x559e8cdcd410 msgr2=0x559e8cdad430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:54.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.191+0000 7f7378ba5740 1 -- 172.21.15.52:0/4078944492 shutdown_connections 2024-04-02T13:56:54.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.191+0000 7f7378ba5740 1 -- 172.21.15.52:0/4078944492 wait complete. 2024-04-02T13:56:54.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.191+0000 7f7378ba5740 20 client.16692 trim_cache size 0 max 16384 2024-04-02T13:56:54.204 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:56:54.204 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "scrub active (2711 inodes in the stack)", 2024-04-02T13:56:54.204 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-04-02T13:56:54.205 INFO:teuthology.orchestra.run.smithi052.stdout: "8994c12c-13b6-4046-8543-05f1662e4793": { 2024-04-02T13:56:54.205 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-04-02T13:56:54.205 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "8994c12c-13b6-4046-8543-05f1662e4793", 2024-04-02T13:56:54.205 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-04-02T13:56:54.205 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:56:54.206 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:56:54.206 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:56:54.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.191+0000 7f7375ee4640 1 -- 172.21.15.52:0/1888461599 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f735c0a3d20 msgr2=0x7f735c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:54.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.191+0000 7f7375ee4640 1 --2- 172.21.15.52:0/1888461599 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f735c0a3d20 0x7f735c0a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f737010d740 tx=0x7f7358005f90 comp rx=0 tx=0).stop 2024-04-02T13:56:54.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.191+0000 7f7375ee4640 1 -- 172.21.15.52:0/1888461599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7370100970 msgr2=0x7f737019a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:54.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.191+0000 7f7375ee4640 1 --2- 172.21.15.52:0/1888461599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7370100970 0x7f737019a650 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f7360009230 tx=0x7f736002cb30 comp rx=0 tx=0).stop 2024-04-02T13:56:54.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.191+0000 7f7375ee4640 1 -- 172.21.15.52:0/1888461599 shutdown_connections 2024-04-02T13:56:54.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.191+0000 7f7375ee4640 1 --2- 172.21.15.52:0/1888461599 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f735c0a3d20 0x7f735c0a61e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.191+0000 7f7375ee4640 1 --2- 172.21.15.52:0/1888461599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7370106980 0x7f737019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.191+0000 7f7375ee4640 1 --2- 172.21.15.52:0/1888461599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7370101330 0x7f737019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.191+0000 7f7375ee4640 1 --2- 172.21.15.52:0/1888461599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7370100970 0x7f737019a650 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:54.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.191+0000 7f7375ee4640 1 -- 172.21.15.52:0/1888461599 >> 172.21.15.52:0/1888461599 conn(0x7f7370077b00 msgr2=0x7f737010c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:54.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.191+0000 7f7375ee4640 1 -- 172.21.15.52:0/1888461599 shutdown_connections 2024-04-02T13:56:54.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:56:54.191+0000 7f7375ee4640 1 -- 172.21.15.52:0/1888461599 wait complete. 2024-04-02T13:56:54.209 INFO:tasks.cephfs.filesystem:scrub status for tag:8994c12c-13b6-4046-8543-05f1662e4793 - {'path': '/', 'tag': '8994c12c-13b6-4046-8543-05f1662e4793', 'options': 'recursive,force'} 2024-04-02T13:56:54.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:54 smithi052 ceph-mon[32104]: scrub queued for path: / 2024-04-02T13:56:54.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:54 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:54.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:54 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:56:54.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:54 smithi088 ceph-mon[36528]: scrub queued for path: / 2024-04-02T13:56:54.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:54 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:54.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:54 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:56:55.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:54 smithi189 ceph-mon[36683]: scrub queued for path: / 2024-04-02T13:56:55.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:54 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:55.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:54 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:56:55.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:55 smithi052 ceph-mon[32104]: pgmap v105: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-02T13:56:55.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:55 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:56:55.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:55 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:55.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:55 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:56:55.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:55 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:55.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:55 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:55 smithi088 ceph-mon[36528]: pgmap v105: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-02T13:56:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:55 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:56:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:55 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:55 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:56:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:55 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:55 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:56.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:55 smithi189 ceph-mon[36683]: pgmap v105: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-02T13:56:56.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:55 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:56:56.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:55 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:56.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:55 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:56:56.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:55 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:56.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:55 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:57.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:57 smithi052 ceph-mon[32104]: pgmap v106: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.1 KiB/s wr, 13 op/s 2024-04-02T13:56:57.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:57 smithi088 ceph-mon[36528]: pgmap v106: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.1 KiB/s wr, 13 op/s 2024-04-02T13:56:58.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:57 smithi189 ceph-mon[36683]: pgmap v106: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.1 KiB/s wr, 13 op/s 2024-04-02T13:56:59.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:56:59 smithi052 ceph-mon[32104]: pgmap v107: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-04-02T13:56:59.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:56:59 smithi088 ceph-mon[36528]: pgmap v107: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-04-02T13:57:00.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:56:59 smithi189 ceph-mon[36683]: pgmap v107: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-04-02T13:57:00.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:00 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T13:57:00.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:00 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T13:57:01.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:00 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T13:57:01.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:01 smithi052 ceph-mon[32104]: pgmap v108: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-04-02T13:57:01.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:01 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T13:57:01.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:01 smithi088 ceph-mon[36528]: pgmap v108: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-04-02T13:57:01.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:01 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T13:57:02.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:01 smithi189 ceph-mon[36683]: pgmap v108: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-04-02T13:57:02.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:01 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T13:57:03.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:03 smithi052 ceph-mon[32104]: pgmap v109: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-02T13:57:03.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:03 smithi088 ceph-mon[36528]: pgmap v109: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-02T13:57:04.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:03 smithi189 ceph-mon[36683]: pgmap v109: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-02T13:57:05.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:05 smithi088 ceph-mon[36528]: pgmap v110: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-02T13:57:06.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:05 smithi189 ceph-mon[36683]: pgmap v110: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-02T13:57:06.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:05 smithi052 ceph-mon[32104]: pgmap v110: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-02T13:57:07.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:07 smithi088 ceph-mon[36528]: pgmap v111: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-02T13:57:08.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:07 smithi189 ceph-mon[36683]: pgmap v111: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-02T13:57:08.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:07 smithi052 ceph-mon[32104]: pgmap v111: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-02T13:57:09.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:09 smithi088 ceph-mon[36528]: pgmap v112: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:57:10.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:09 smithi189 ceph-mon[36683]: pgmap v112: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:57:10.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:09 smithi052 ceph-mon[32104]: pgmap v112: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:57:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:11 smithi088 ceph-mon[36528]: pgmap v113: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:57:12.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:11 smithi189 ceph-mon[36683]: pgmap v113: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:57:12.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:11 smithi052 ceph-mon[32104]: pgmap v113: 129 pgs: 129 active+clean; 72 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:57:13.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:13 smithi088 ceph-mon[36528]: pgmap v114: 129 pgs: 129 active+clean; 73 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 109 KiB/s wr, 27 op/s 2024-04-02T13:57:14.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:13 smithi189 ceph-mon[36683]: pgmap v114: 129 pgs: 129 active+clean; 73 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 109 KiB/s wr, 27 op/s 2024-04-02T13:57:14.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:13 smithi052 ceph-mon[32104]: pgmap v114: 129 pgs: 129 active+clean; 73 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 109 KiB/s wr, 27 op/s 2024-04-02T13:57:15.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:15 smithi088 ceph-mon[36528]: pgmap v115: 129 pgs: 129 active+clean; 73 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 8 op/s 2024-04-02T13:57:16.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:15 smithi189 ceph-mon[36683]: pgmap v115: 129 pgs: 129 active+clean; 73 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 8 op/s 2024-04-02T13:57:16.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:15 smithi052 ceph-mon[32104]: pgmap v115: 129 pgs: 129 active+clean; 73 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 8 op/s 2024-04-02T13:57:17.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:17 smithi088 ceph-mon[36528]: pgmap v116: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 90 KiB/s wr, 13 op/s 2024-04-02T13:57:18.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:17 smithi189 ceph-mon[36683]: pgmap v116: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 90 KiB/s wr, 13 op/s 2024-04-02T13:57:18.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:17 smithi052 ceph-mon[32104]: pgmap v116: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 90 KiB/s wr, 13 op/s 2024-04-02T13:57:19.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:19 smithi088 ceph-mon[36528]: pgmap v117: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 9 op/s 2024-04-02T13:57:20.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:19 smithi189 ceph-mon[36683]: pgmap v117: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 9 op/s 2024-04-02T13:57:20.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:19 smithi052 ceph-mon[32104]: pgmap v117: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 9 op/s 2024-04-02T13:57:21.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:21 smithi088 ceph-mon[36528]: pgmap v118: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 9 op/s 2024-04-02T13:57:22.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:21 smithi189 ceph-mon[36683]: pgmap v118: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 9 op/s 2024-04-02T13:57:22.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:21 smithi052 ceph-mon[32104]: pgmap v118: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 90 KiB/s wr, 9 op/s 2024-04-02T13:57:23.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:23 smithi088 ceph-mon[36528]: pgmap v119: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 90 KiB/s wr, 13 op/s 2024-04-02T13:57:24.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:23 smithi189 ceph-mon[36683]: pgmap v119: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 90 KiB/s wr, 13 op/s 2024-04-02T13:57:24.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:23 smithi052 ceph-mon[32104]: pgmap v119: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 90 KiB/s wr, 13 op/s 2024-04-02T13:57:24.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:57:24.210 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:57:24.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.326+0000 7fdee85ba640 1 -- 172.21.15.52:0/3214870194 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdee0102450 msgr2=0x7fdee010e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.326+0000 7fdee85ba640 1 --2- 172.21.15.52:0/3214870194 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdee0102450 0x7fdee010e960 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fded8009930 tx=0x7fded802bc30 comp rx=0 tx=0).stop 2024-04-02T13:57:24.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.326+0000 7fdee85ba640 1 -- 172.21.15.52:0/3214870194 shutdown_connections 2024-04-02T13:57:24.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.326+0000 7fdee85ba640 1 --2- 172.21.15.52:0/3214870194 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdee0102450 0x7fdee010e960 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.326+0000 7fdee85ba640 1 --2- 172.21.15.52:0/3214870194 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdee0101a90 0x7fdee0101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.326+0000 7fdee85ba640 1 --2- 172.21.15.52:0/3214870194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdee0107aa0 0x7fdee0107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.326+0000 7fdee85ba640 1 -- 172.21.15.52:0/3214870194 >> 172.21.15.52:0/3214870194 conn(0x7fdee0078f00 msgr2=0x7fdee0079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:24.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.326+0000 7fdee85ba640 1 -- 172.21.15.52:0/3214870194 shutdown_connections 2024-04-02T13:57:24.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.326+0000 7fdee85ba640 1 -- 172.21.15.52:0/3214870194 wait complete. 2024-04-02T13:57:24.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.326+0000 7fdee85ba640 1 Processor -- start 2024-04-02T13:57:24.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.326+0000 7fdee85ba640 1 -- start start 2024-04-02T13:57:24.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.327+0000 7fdee85ba640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdee0101a90 0x7fdee0198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.327+0000 7fdee85ba640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdee0102450 0x7fdee01988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.327+0000 7fdee85ba640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdee0107aa0 0x7fdee019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.327+0000 7fdee85ba640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdee0111eb0 con 0x7fdee0101a90 2024-04-02T13:57:24.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.327+0000 7fdee85ba640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdee0111d30 con 0x7fdee0107aa0 2024-04-02T13:57:24.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.327+0000 7fdee85ba640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdee0112030 con 0x7fdee0102450 2024-04-02T13:57:24.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.327+0000 7fdee6356640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdee0101a90 0x7fdee0198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.327+0000 7fdee6356640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdee0101a90 0x7fdee0198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47916/0 (socket says 172.21.15.52:47916) 2024-04-02T13:57:24.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.327+0000 7fdee6356640 1 -- 172.21.15.52:0/3619186650 learned_addr learned my addr 172.21.15.52:0/3619186650 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:24.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.327+0000 7fdee5b55640 1 --2- 172.21.15.52:0/3619186650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdee0102450 0x7fdee01988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.327+0000 7fdee6356640 1 -- 172.21.15.52:0/3619186650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdee0102450 msgr2=0x7fdee01988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.327+0000 7fdee6b57640 1 --2- 172.21.15.52:0/3619186650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdee0107aa0 0x7fdee019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.327+0000 7fdee6356640 1 --2- 172.21.15.52:0/3619186650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdee0102450 0x7fdee01988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.327+0000 7fdee6356640 1 -- 172.21.15.52:0/3619186650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdee0107aa0 msgr2=0x7fdee019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.327+0000 7fdee6356640 1 --2- 172.21.15.52:0/3619186650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdee0107aa0 0x7fdee019cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.327+0000 7fdee6356640 1 -- 172.21.15.52:0/3619186650 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee019d360 con 0x7fdee0101a90 2024-04-02T13:57:24.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.327+0000 7fdee5b55640 1 --2- 172.21.15.52:0/3619186650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdee0102450 0x7fdee01988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:24.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.327+0000 7fdee6356640 1 --2- 172.21.15.52:0/3619186650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdee0101a90 0x7fdee0198370 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00ea80 tx=0x7fdedc00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:24.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.327+0000 7fded77fe640 1 -- 172.21.15.52:0/3619186650 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc00ce40 con 0x7fdee0101a90 2024-04-02T13:57:24.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.327+0000 7fded77fe640 1 -- 172.21.15.52:0/3619186650 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdedc005020 con 0x7fdee0101a90 2024-04-02T13:57:24.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.327+0000 7fdee85ba640 1 -- 172.21.15.52:0/3619186650 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee019d650 con 0x7fdee0101a90 2024-04-02T13:57:24.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.327+0000 7fded77fe640 1 -- 172.21.15.52:0/3619186650 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc011760 con 0x7fdee0101a90 2024-04-02T13:57:24.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.327+0000 7fdee85ba640 1 -- 172.21.15.52:0/3619186650 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee01a4ee0 con 0x7fdee0101a90 2024-04-02T13:57:24.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.330+0000 7fded77fe640 1 -- 172.21.15.52:0/3619186650 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdedc0051d0 con 0x7fdee0101a90 2024-04-02T13:57:24.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.330+0000 7fdeeb19e740 1 Processor -- start 2024-04-02T13:57:24.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.330+0000 7fdeeb19e740 1 -- start start 2024-04-02T13:57:24.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.330+0000 7fded77fe640 1 --2- 172.21.15.52:0/3619186650 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdecc0a3d20 0x7fdecc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.331+0000 7fdeeb19e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e7c1d628d0 0x55e7c1d62cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.331+0000 7fdeeb19e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7c1d631f0 0x55e7c1d6b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.331+0000 7fdeeb19e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d6bdd0 0x55e7c1d6e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.331+0000 7fdeeb19e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e7c1cc6830 con 0x55e7c1d6bdd0 2024-04-02T13:57:24.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.331+0000 7fdeeb19e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e7c1c6e820 con 0x55e7c1d628d0 2024-04-02T13:57:24.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.331+0000 7fdeeb19e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e7c1cc27e0 con 0x55e7c1d631f0 2024-04-02T13:57:24.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.331+0000 7fded77fe640 1 -- 172.21.15.52:0/3619186650 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fdedc0d11b0 con 0x7fdee0101a90 2024-04-02T13:57:24.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.331+0000 7fdee6b57640 1 --2- 172.21.15.52:0/3619186650 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdecc0a3d20 0x7fdecc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.331+0000 7fdee6b57640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d6bdd0 0x55e7c1d6e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.331+0000 7fdee6b57640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d6bdd0 0x55e7c1d6e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47922/0 (socket says 172.21.15.52:47922) 2024-04-02T13:57:24.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.331+0000 7fdee6b57640 1 -- 172.21.15.52:0/3340245245 learned_addr learned my addr 172.21.15.52:0/3340245245 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:24.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.331+0000 7fdee5b55640 1 --2- 172.21.15.52:0/3340245245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7c1d631f0 0x55e7c1d6b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.331+0000 7fdee6356640 1 --2- 172.21.15.52:0/3340245245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e7c1d628d0 0x55e7c1d62cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.331+0000 7fdee6b57640 1 -- 172.21.15.52:0/3340245245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7c1d631f0 msgr2=0x55e7c1d6b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:24.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.331+0000 7fdee6b57640 1 --2- 172.21.15.52:0/3340245245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7c1d631f0 0x55e7c1d6b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.331+0000 7fdee6b57640 1 -- 172.21.15.52:0/3340245245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e7c1d628d0 msgr2=0x55e7c1d62cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:24.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.331+0000 7fdee6b57640 1 --2- 172.21.15.52:0/3340245245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e7c1d628d0 0x55e7c1d62cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.331+0000 7fdee6b57640 1 -- 172.21.15.52:0/3340245245 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7c1aa47d0 con 0x55e7c1d6bdd0 2024-04-02T13:57:24.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.331+0000 7fdee6b57640 1 --2- 172.21.15.52:0/3619186650 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdecc0a3d20 0x7fdecc0a61e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fded8005ec0 tx=0x7fded8005dd0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:24.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.331+0000 7fdee6b57640 1 --2- 172.21.15.52:0/3340245245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d6bdd0 0x55e7c1d6e1c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fded802cfd0 tx=0x7fded802ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79e57801a8e9489 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:24.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.332+0000 7fded4bfe640 1 -- 172.21.15.52:0/3340245245 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8005050 con 0x55e7c1d6bdd0 2024-04-02T13:57:24.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.332+0000 7fded4bfe640 1 -- 172.21.15.52:0/3340245245 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fded8050070 con 0x55e7c1d6bdd0 2024-04-02T13:57:24.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.332+0000 7fded4bfe640 1 -- 172.21.15.52:0/3340245245 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8004e60 con 0x55e7c1d6bdd0 2024-04-02T13:57:24.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.332+0000 7fdeeb19e740 1 -- 172.21.15.52:0/3340245245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d6bdd0 msgr2=0x55e7c1d6e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.332+0000 7fdeeb19e740 1 --2- 172.21.15.52:0/3340245245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d6bdd0 0x55e7c1d6e1c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fded802cfd0 tx=0x7fded802ce30 comp rx=0 tx=0).stop 2024-04-02T13:57:24.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.332+0000 7fdeeb19e740 1 -- 172.21.15.52:0/3340245245 shutdown_connections 2024-04-02T13:57:24.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.332+0000 7fdeeb19e740 1 --2- 172.21.15.52:0/3340245245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d6bdd0 0x55e7c1d6e1c0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.332+0000 7fdeeb19e740 1 --2- 172.21.15.52:0/3340245245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7c1d631f0 0x55e7c1d6b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.332+0000 7fdeeb19e740 1 --2- 172.21.15.52:0/3340245245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e7c1d628d0 0x55e7c1d62cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.332+0000 7fdeeb19e740 1 -- 172.21.15.52:0/3340245245 >> 172.21.15.52:0/3340245245 conn(0x55e7c1d5d000 msgr2=0x55e7c1d5d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:24.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.333+0000 7fdeeb19e740 1 -- 172.21.15.52:0/3340245245 shutdown_connections 2024-04-02T13:57:24.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.333+0000 7fdeeb19e740 1 -- 172.21.15.52:0/3340245245 wait complete. 2024-04-02T13:57:24.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.333+0000 7fdeeb19e740 1 -- start start 2024-04-02T13:57:24.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.333+0000 7fdeeb19e740 1 Processor -- start 2024-04-02T13:57:24.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.333+0000 7fdeeb19e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e7c1d628d0 0x55e7c1d6b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.333+0000 7fdeeb19e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d631f0 0x55e7c1d68a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.333+0000 7fdeeb19e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7c1d6bdd0 0x55e7c1d68f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.333+0000 7fdeeb19e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e7c1d6eb60 con 0x55e7c1d631f0 2024-04-02T13:57:24.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.333+0000 7fdeeb19e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e7c1c81d30 con 0x55e7c1d628d0 2024-04-02T13:57:24.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.333+0000 7fdeeb19e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e7c1d6ece0 con 0x55e7c1d6bdd0 2024-04-02T13:57:24.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.333+0000 7fdeeb19e740 10 client.0 _ll_register_callbacks cb 0x55e7c1c81720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:24.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.333+0000 7fdee5b55640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d631f0 0x55e7c1d68a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.333+0000 7fdee5b55640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d631f0 0x55e7c1d68a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47934/0 (socket says 172.21.15.52:47934) 2024-04-02T13:57:24.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.333+0000 7fdee5b55640 1 -- 172.21.15.52:0/3711740121 learned_addr learned my addr 172.21.15.52:0/3711740121 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:24.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.333+0000 7fdee6356640 1 --2- 172.21.15.52:0/3711740121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e7c1d628d0 0x55e7c1d6b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.334+0000 7fdee5b55640 1 -- 172.21.15.52:0/3711740121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7c1d6bdd0 msgr2=0x55e7c1d68f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:24.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.334+0000 7fdee5b55640 1 --2- 172.21.15.52:0/3711740121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7c1d6bdd0 0x55e7c1d68f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.334+0000 7fdee5b55640 1 -- 172.21.15.52:0/3711740121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e7c1d628d0 msgr2=0x55e7c1d6b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:24.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.334+0000 7fdee5b55640 1 --2- 172.21.15.52:0/3711740121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e7c1d628d0 0x55e7c1d6b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.334+0000 7fdee5b55640 1 -- 172.21.15.52:0/3711740121 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7c19da3f0 con 0x55e7c1d631f0 2024-04-02T13:57:24.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.334+0000 7fdee5b55640 1 --2- 172.21.15.52:0/3711740121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d631f0 0x55e7c1d68a00 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fded0011a60 tx=0x7fded0011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6828cb2cff0edd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:24.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.334+0000 7fdec9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:24.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.334+0000 7fdec9ffb640 1 -- 172.21.15.52:0/3711740121 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded000fd30 con 0x55e7c1d631f0 2024-04-02T13:57:24.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.334+0000 7fdec9ffb640 1 -- 172.21.15.52:0/3711740121 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fded0005020 con 0x55e7c1d631f0 2024-04-02T13:57:24.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.334+0000 7fdeeb19e740 1 -- 172.21.15.52:0/3711740121 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fded8050070 con 0x55e7c1d631f0 2024-04-02T13:57:24.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.334+0000 7fdec9ffb640 1 -- 172.21.15.52:0/3711740121 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fded0005220 con 0x55e7c1d631f0 2024-04-02T13:57:24.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.334+0000 7fdeeb19e740 10 client.16704 fetch_fsmap learned FSMap version 29 2024-04-02T13:57:24.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.334+0000 7fdeeb19e740 1 -- 172.21.15.52:0/3711740121 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e7c199a0f0 con 0x55e7c1d631f0 2024-04-02T13:57:24.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.334+0000 7fdec9ffb640 1 -- 172.21.15.52:0/3711740121 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fded8050070 con 0x55e7c1d631f0 2024-04-02T13:57:24.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.334+0000 7fdec9ffb640 1 -- 172.21.15.52:0/3711740121 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fded001b420 con 0x55e7c1d631f0 2024-04-02T13:57:24.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.334+0000 7fdeeb19e740 10 client.16704 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:57:24.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.334+0000 7fdeeb19e740 10 client.16704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:57:24.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.335+0000 7fdeeb19e740 20 client.16704 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:24.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.335+0000 7fdeeb19e740 1 --2- 172.21.15.52:0/3711740121 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55e7c1d7bf90 0x55e7c1d7e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.335+0000 7fdeeb19e740 4 client.16704 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:57:24.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.335+0000 7fdeeb19e740 1 -- 172.21.15.52:0/3711740121 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7c1cd3d70 con 0x55e7c1d7bf90 2024-04-02T13:57:24.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.335+0000 7fdee6356640 1 --2- 172.21.15.52:0/3711740121 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55e7c1d7bf90 0x55e7c1d7e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.335+0000 7fdee6356640 1 --2- 172.21.15.52:0/3711740121 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55e7c1d7bf90 0x55e7c1d7e450 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f476570d38ce9df server_cookie=a51c5f0b3ed8d0bb in_seq=0 out_seq=0 2024-04-02T13:57:24.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.335+0000 7fdec9ffb640 10 client.16704 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:24.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.336+0000 7fdec9ffb640 1 -- 172.21.15.52:0/3711740121 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e7c1cd3d70 con 0x55e7c1d7bf90 2024-04-02T13:57:24.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.336+0000 7fdec9ffb640 10 client.16704 handle_command_reply: tid=0 2024-04-02T13:57:24.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.336+0000 7fdeeb19e740 1 client.16704 shutdown 2024-04-02T13:57:24.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.336+0000 7fdeeb19e740 1 -- 172.21.15.52:0/3711740121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d631f0 msgr2=0x55e7c1d68a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.336+0000 7fdeeb19e740 1 --2- 172.21.15.52:0/3711740121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d631f0 0x55e7c1d68a00 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fded0011a60 tx=0x7fded0011e60 comp rx=0 tx=0).stop 2024-04-02T13:57:24.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.336+0000 7fdeeb19e740 1 -- 172.21.15.52:0/3711740121 shutdown_connections 2024-04-02T13:57:24.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.336+0000 7fdeeb19e740 1 --2- 172.21.15.52:0/3711740121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7c1d6bdd0 0x55e7c1d68f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.336+0000 7fdeeb19e740 1 --2- 172.21.15.52:0/3711740121 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55e7c1d7bf90 0x55e7c1d7e450 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.336+0000 7fdeeb19e740 1 --2- 172.21.15.52:0/3711740121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d631f0 0x55e7c1d68a00 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.336+0000 7fdeeb19e740 1 --2- 172.21.15.52:0/3711740121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e7c1d628d0 0x55e7c1d6b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.336+0000 7fdeeb19e740 1 -- 172.21.15.52:0/3711740121 >> 172.21.15.52:0/3711740121 conn(0x55e7c1d5d000 msgr2=0x55e7c1d5ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:24.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.336+0000 7fdec9ffb640 0 client.16704 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:24.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.336+0000 7fdeeb19e740 1 -- 172.21.15.52:0/3711740121 shutdown_connections 2024-04-02T13:57:24.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.336+0000 7fdeeb19e740 1 -- 172.21.15.52:0/3711740121 wait complete. 2024-04-02T13:57:24.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.336+0000 7fdeeb19e740 20 client.16704 trim_cache size 0 max 16384 2024-04-02T13:57:24.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.354+0000 7fdeeb19e740 1 Processor -- start 2024-04-02T13:57:24.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.354+0000 7fdeeb19e740 1 -- start start 2024-04-02T13:57:24.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.354+0000 7fdeeb19e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7c1d628d0 0x55e7c1d77b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.354+0000 7fdeeb19e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e7c1d631f0 0x55e7c1d5c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.354+0000 7fdeeb19e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d6bdd0 0x55e7c1d6b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.354+0000 7fdeeb19e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdee01aee50 con 0x55e7c1d6bdd0 2024-04-02T13:57:24.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.354+0000 7fdeeb19e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdee01b09c0 con 0x55e7c1d631f0 2024-04-02T13:57:24.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.354+0000 7fdeeb19e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdee01af520 con 0x55e7c1d628d0 2024-04-02T13:57:24.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.354+0000 7fdee6b57640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d6bdd0 0x55e7c1d6b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.354+0000 7fdee6b57640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d6bdd0 0x55e7c1d6b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47940/0 (socket says 172.21.15.52:47940) 2024-04-02T13:57:24.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.354+0000 7fdee6b57640 1 -- 172.21.15.52:0/1644157835 learned_addr learned my addr 172.21.15.52:0/1644157835 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:24.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.354+0000 7fdee5b55640 1 --2- 172.21.15.52:0/1644157835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e7c1d631f0 0x55e7c1d5c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.354+0000 7fdee6356640 1 --2- 172.21.15.52:0/1644157835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7c1d628d0 0x55e7c1d77b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.355+0000 7fdee6b57640 1 -- 172.21.15.52:0/1644157835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7c1d628d0 msgr2=0x55e7c1d77b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:24.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.355+0000 7fdee6b57640 1 --2- 172.21.15.52:0/1644157835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7c1d628d0 0x55e7c1d77b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.355+0000 7fdee6b57640 1 -- 172.21.15.52:0/1644157835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e7c1d631f0 msgr2=0x55e7c1d5c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:24.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.355+0000 7fdee6b57640 1 --2- 172.21.15.52:0/1644157835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e7c1d631f0 0x55e7c1d5c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.355+0000 7fdee6b57640 1 -- 172.21.15.52:0/1644157835 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7c1d5ce30 con 0x55e7c1d6bdd0 2024-04-02T13:57:24.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.355+0000 7fdee5b55640 1 --2- 172.21.15.52:0/1644157835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e7c1d631f0 0x55e7c1d5c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:24.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.355+0000 7fdee6b57640 1 --2- 172.21.15.52:0/1644157835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d6bdd0 0x55e7c1d6b130 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fded802c8d0 tx=0x7fded802cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de96b60976d13f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:24.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.355+0000 7fdecbfff640 1 -- 172.21.15.52:0/1644157835 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8059070 con 0x55e7c1d6bdd0 2024-04-02T13:57:24.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.355+0000 7fdecbfff640 1 -- 172.21.15.52:0/1644157835 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fded8050070 con 0x55e7c1d6bdd0 2024-04-02T13:57:24.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.355+0000 7fdecbfff640 1 -- 172.21.15.52:0/1644157835 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8003850 con 0x55e7c1d6bdd0 2024-04-02T13:57:24.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.355+0000 7fdeeb19e740 1 -- 172.21.15.52:0/1644157835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d6bdd0 msgr2=0x55e7c1d6b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.355+0000 7fdeeb19e740 1 --2- 172.21.15.52:0/1644157835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d6bdd0 0x55e7c1d6b130 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fded802c8d0 tx=0x7fded802cd30 comp rx=0 tx=0).stop 2024-04-02T13:57:24.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.355+0000 7fdeeb19e740 1 -- 172.21.15.52:0/1644157835 shutdown_connections 2024-04-02T13:57:24.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.355+0000 7fdeeb19e740 1 --2- 172.21.15.52:0/1644157835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d6bdd0 0x55e7c1d6b130 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.355+0000 7fdeeb19e740 1 --2- 172.21.15.52:0/1644157835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e7c1d631f0 0x55e7c1d5c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.355+0000 7fdeeb19e740 1 --2- 172.21.15.52:0/1644157835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7c1d628d0 0x55e7c1d77b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.355+0000 7fdeeb19e740 1 -- 172.21.15.52:0/1644157835 >> 172.21.15.52:0/1644157835 conn(0x55e7c1d5d000 msgr2=0x55e7c1d63ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:24.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.356+0000 7fdeeb19e740 1 -- 172.21.15.52:0/1644157835 shutdown_connections 2024-04-02T13:57:24.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.356+0000 7fdeeb19e740 1 -- 172.21.15.52:0/1644157835 wait complete. 2024-04-02T13:57:24.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.356+0000 7fdeeb19e740 1 -- start start 2024-04-02T13:57:24.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.356+0000 7fdeeb19e740 1 Processor -- start 2024-04-02T13:57:24.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.356+0000 7fdeeb19e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d628d0 0x55e7c1d743a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.356+0000 7fdeeb19e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e7c1d631f0 0x55e7c1d6f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.356+0000 7fdeeb19e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7c1d6bdd0 0x55e7c1d6fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.356+0000 7fdeeb19e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdee01b02c0 con 0x55e7c1d628d0 2024-04-02T13:57:24.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.356+0000 7fdeeb19e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdee01afbf0 con 0x55e7c1d631f0 2024-04-02T13:57:24.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.356+0000 7fdeeb19e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdee01ace30 con 0x55e7c1d6bdd0 2024-04-02T13:57:24.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.356+0000 7fdeeb19e740 10 client.0 _ll_register_callbacks cb 0x7fded0011670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:24.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.356+0000 7fdee6356640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d628d0 0x55e7c1d743a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.356+0000 7fdee6356640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d628d0 0x55e7c1d743a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47956/0 (socket says 172.21.15.52:47956) 2024-04-02T13:57:24.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.356+0000 7fdee6356640 1 -- 172.21.15.52:0/211763486 learned_addr learned my addr 172.21.15.52:0/211763486 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:24.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.356+0000 7fdee6b57640 1 --2- 172.21.15.52:0/211763486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7c1d6bdd0 0x55e7c1d6fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.356+0000 7fdee5b55640 1 --2- 172.21.15.52:0/211763486 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e7c1d631f0 0x55e7c1d6f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.356+0000 7fdee6356640 1 -- 172.21.15.52:0/211763486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7c1d6bdd0 msgr2=0x55e7c1d6fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:24.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.357+0000 7fdee6356640 1 --2- 172.21.15.52:0/211763486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7c1d6bdd0 0x55e7c1d6fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.357+0000 7fdee6356640 1 -- 172.21.15.52:0/211763486 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e7c1d631f0 msgr2=0x55e7c1d6f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.357+0000 7fdee6356640 1 --2- 172.21.15.52:0/211763486 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e7c1d631f0 0x55e7c1d6f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.357+0000 7fdee6356640 1 -- 172.21.15.52:0/211763486 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7c1d70460 con 0x55e7c1d628d0 2024-04-02T13:57:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.357+0000 7fdee5b55640 1 --2- 172.21.15.52:0/211763486 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e7c1d631f0 0x55e7c1d6f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.357+0000 7fdee6356640 1 --2- 172.21.15.52:0/211763486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d628d0 0x55e7c1d743a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fdedc09cb10 tx=0x7fdedc007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c156dbead5a7e817 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.357+0000 7fdec9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.357+0000 7fdec9ffb640 1 -- 172.21.15.52:0/211763486 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc01f040 con 0x55e7c1d628d0 2024-04-02T13:57:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.357+0000 7fdec9ffb640 1 -- 172.21.15.52:0/211763486 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fdedc09aeb0 con 0x55e7c1d628d0 2024-04-02T13:57:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.357+0000 7fdeeb19e740 1 -- 172.21.15.52:0/211763486 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fded8050070 con 0x55e7c1d628d0 2024-04-02T13:57:24.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.357+0000 7fdec9ffb640 1 -- 172.21.15.52:0/211763486 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdedc0a5a80 con 0x55e7c1d628d0 2024-04-02T13:57:24.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.357+0000 7fdeeb19e740 10 client.16710 fetch_fsmap learned FSMap version 29 2024-04-02T13:57:24.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.357+0000 7fdeeb19e740 1 -- 172.21.15.52:0/211763486 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e7c1de69e0 con 0x55e7c1d628d0 2024-04-02T13:57:24.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.357+0000 7fdec9ffb640 1 -- 172.21.15.52:0/211763486 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fded8050070 con 0x55e7c1d628d0 2024-04-02T13:57:24.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.357+0000 7fdec9ffb640 1 -- 172.21.15.52:0/211763486 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fdedc00f040 con 0x55e7c1d628d0 2024-04-02T13:57:24.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.357+0000 7fdeeb19e740 10 client.16710 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:57:24.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.357+0000 7fdeeb19e740 10 client.16710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:57:24.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.357+0000 7fdeeb19e740 20 client.16710 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:24.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.357+0000 7fdeeb19e740 1 --2- 172.21.15.52:0/211763486 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55e7c1de70e0 0x55e7c1de9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.357+0000 7fdeeb19e740 4 client.16710 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:57:24.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.357+0000 7fdeeb19e740 1 -- 172.21.15.52:0/211763486 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e7c1cce0c0 con 0x55e7c1de70e0 2024-04-02T13:57:24.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.358+0000 7fdee5b55640 1 --2- 172.21.15.52:0/211763486 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55e7c1de70e0 0x55e7c1de9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.358+0000 7fdee5b55640 1 --2- 172.21.15.52:0/211763486 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55e7c1de70e0 0x55e7c1de9500 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e09b3730da1f7e0b server_cookie=5565dc864f4bf093 in_seq=0 out_seq=0 2024-04-02T13:57:24.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.358+0000 7fdec9ffb640 10 client.16710 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:24.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.358+0000 7fdec9ffb640 1 -- 172.21.15.52:0/211763486 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e7c1cce0c0 con 0x55e7c1de70e0 2024-04-02T13:57:24.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.358+0000 7fdec9ffb640 10 client.16710 handle_command_reply: tid=0 2024-04-02T13:57:24.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.358+0000 7fdeeb19e740 1 client.16710 shutdown 2024-04-02T13:57:24.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.358+0000 7fdeeb19e740 1 -- 172.21.15.52:0/211763486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d628d0 msgr2=0x55e7c1d743a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.358+0000 7fdeeb19e740 1 --2- 172.21.15.52:0/211763486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d628d0 0x55e7c1d743a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fdedc09cb10 tx=0x7fdedc007eb0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.358+0000 7fdeeb19e740 1 -- 172.21.15.52:0/211763486 shutdown_connections 2024-04-02T13:57:24.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.358+0000 7fdeeb19e740 1 --2- 172.21.15.52:0/211763486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7c1d6bdd0 0x55e7c1d6fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.358+0000 7fdeeb19e740 1 --2- 172.21.15.52:0/211763486 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e7c1d631f0 0x55e7c1d6f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.358+0000 7fdeeb19e740 1 --2- 172.21.15.52:0/211763486 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55e7c1de70e0 0x55e7c1de9500 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.358+0000 7fdeeb19e740 1 --2- 172.21.15.52:0/211763486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c1d628d0 0x55e7c1d743a0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.358+0000 7fdec9ffb640 0 client.16710 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:24.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.358+0000 7fdeeb19e740 1 -- 172.21.15.52:0/211763486 >> 172.21.15.52:0/211763486 conn(0x55e7c1d5d000 msgr2=0x55e7c1d5ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:24.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.358+0000 7fdeeb19e740 1 -- 172.21.15.52:0/211763486 shutdown_connections 2024-04-02T13:57:24.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.358+0000 7fdeeb19e740 1 -- 172.21.15.52:0/211763486 wait complete. 2024-04-02T13:57:24.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.359+0000 7fdeeb19e740 20 client.16710 trim_cache size 0 max 16384 2024-04-02T13:57:24.373 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:57:24.373 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-04-02T13:57:24.374 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-04-02T13:57:24.374 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:57:24.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.359+0000 7fdee85ba640 1 -- 172.21.15.52:0/3619186650 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdecc0a3d20 msgr2=0x7fdecc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.359+0000 7fdee85ba640 1 --2- 172.21.15.52:0/3619186650 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdecc0a3d20 0x7fdecc0a61e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fded8005ec0 tx=0x7fded8005dd0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.359+0000 7fdee85ba640 1 -- 172.21.15.52:0/3619186650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdee0101a90 msgr2=0x7fdee0198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.359+0000 7fdee85ba640 1 --2- 172.21.15.52:0/3619186650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdee0101a90 0x7fdee0198370 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00ea80 tx=0x7fdedc00ef40 comp rx=0 tx=0).stop 2024-04-02T13:57:24.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.359+0000 7fdee85ba640 1 -- 172.21.15.52:0/3619186650 shutdown_connections 2024-04-02T13:57:24.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.359+0000 7fdee85ba640 1 --2- 172.21.15.52:0/3619186650 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdecc0a3d20 0x7fdecc0a61e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.359+0000 7fdee85ba640 1 --2- 172.21.15.52:0/3619186650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdee0107aa0 0x7fdee019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.359+0000 7fdee85ba640 1 --2- 172.21.15.52:0/3619186650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdee0102450 0x7fdee01988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.359+0000 7fdee85ba640 1 --2- 172.21.15.52:0/3619186650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdee0101a90 0x7fdee0198370 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.359+0000 7fdee85ba640 1 -- 172.21.15.52:0/3619186650 >> 172.21.15.52:0/3619186650 conn(0x7fdee0078f00 msgr2=0x7fdee0104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:24.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.359+0000 7fdee85ba640 1 -- 172.21.15.52:0/3619186650 shutdown_connections 2024-04-02T13:57:24.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.359+0000 7fdee85ba640 1 -- 172.21.15.52:0/3619186650 wait complete. 2024-04-02T13:57:24.377 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:57:24.377 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:57:24.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.477+0000 7fd6d3da3640 1 -- 172.21.15.52:0/2331793154 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6cc103650 msgr2=0x7fd6cc10bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.477+0000 7fd6d3da3640 1 --2- 172.21.15.52:0/2331793154 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6cc103650 0x7fd6cc10bb30 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc009900 tx=0x7fd6bc02bcd0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.477+0000 7fd6d3da3640 1 -- 172.21.15.52:0/2331793154 shutdown_connections 2024-04-02T13:57:24.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.477+0000 7fd6d3da3640 1 --2- 172.21.15.52:0/2331793154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6cc10c100 0x7fd6cc10e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.477+0000 7fd6d3da3640 1 --2- 172.21.15.52:0/2331793154 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6cc103650 0x7fd6cc10bb30 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.477+0000 7fd6d3da3640 1 --2- 172.21.15.52:0/2331793154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6cc102d30 0x7fd6cc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.477+0000 7fd6d3da3640 1 -- 172.21.15.52:0/2331793154 >> 172.21.15.52:0/2331793154 conn(0x7fd6cc076d80 msgr2=0x7fd6cc077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:24.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.477+0000 7fd6d3da3640 1 -- 172.21.15.52:0/2331793154 shutdown_connections 2024-04-02T13:57:24.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.477+0000 7fd6d3da3640 1 -- 172.21.15.52:0/2331793154 wait complete. 2024-04-02T13:57:24.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.478+0000 7fd6d3da3640 1 Processor -- start 2024-04-02T13:57:24.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.478+0000 7fd6d3da3640 1 -- start start 2024-04-02T13:57:24.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.478+0000 7fd6d3da3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6cc102d30 0x7fd6cc198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.478+0000 7fd6d3da3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6cc103650 0x7fd6cc198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.478+0000 7fd6d3da3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6cc10c100 0x7fd6cc19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.478+0000 7fd6d3da3640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc111e90 con 0x7fd6cc103650 2024-04-02T13:57:24.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.478+0000 7fd6d3da3640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc111d10 con 0x7fd6cc102d30 2024-04-02T13:57:24.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.478+0000 7fd6d3da3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc112010 con 0x7fd6cc10c100 2024-04-02T13:57:24.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.478+0000 7fd6d133e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6cc103650 0x7fd6cc198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.478+0000 7fd6d133e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6cc103650 0x7fd6cc198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47972/0 (socket says 172.21.15.52:47972) 2024-04-02T13:57:24.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.478+0000 7fd6d133e640 1 -- 172.21.15.52:0/1807898247 learned_addr learned my addr 172.21.15.52:0/1807898247 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:24.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.478+0000 7fd6d2340640 1 --2- 172.21.15.52:0/1807898247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6cc10c100 0x7fd6cc19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.478+0000 7fd6d1b3f640 1 --2- 172.21.15.52:0/1807898247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6cc102d30 0x7fd6cc198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.478+0000 7fd6d133e640 1 -- 172.21.15.52:0/1807898247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6cc10c100 msgr2=0x7fd6cc19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.478+0000 7fd6d133e640 1 --2- 172.21.15.52:0/1807898247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6cc10c100 0x7fd6cc19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.478+0000 7fd6d133e640 1 -- 172.21.15.52:0/1807898247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6cc102d30 msgr2=0x7fd6cc198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.478+0000 7fd6d133e640 1 --2- 172.21.15.52:0/1807898247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6cc102d30 0x7fd6cc198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.478+0000 7fd6d133e640 1 -- 172.21.15.52:0/1807898247 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6cc19d340 con 0x7fd6cc103650 2024-04-02T13:57:24.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.478+0000 7fd6d1b3f640 1 --2- 172.21.15.52:0/1807898247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6cc102d30 0x7fd6cc198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:24.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.478+0000 7fd6d133e640 1 --2- 172.21.15.52:0/1807898247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6cc103650 0x7fd6cc198890 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc02efd0 tx=0x7fd6bc02ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:24.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.478+0000 7fd6c2ffd640 1 -- 172.21.15.52:0/1807898247 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc041070 con 0x7fd6cc103650 2024-04-02T13:57:24.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.479+0000 7fd6d3da3640 1 -- 172.21.15.52:0/1807898247 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6cc19d5d0 con 0x7fd6cc103650 2024-04-02T13:57:24.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.479+0000 7fd6c2ffd640 1 -- 172.21.15.52:0/1807898247 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6bc0054e0 con 0x7fd6cc103650 2024-04-02T13:57:24.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.479+0000 7fd6c2ffd640 1 -- 172.21.15.52:0/1807898247 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc034440 con 0x7fd6cc103650 2024-04-02T13:57:24.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.479+0000 7fd6d3da3640 1 -- 172.21.15.52:0/1807898247 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6cc1a4ec0 con 0x7fd6cc103650 2024-04-02T13:57:24.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.480+0000 7fd6d3da3640 1 -- 172.21.15.52:0/1807898247 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6cc06a950 con 0x7fd6cc103650 2024-04-02T13:57:24.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.481+0000 7fd6c2ffd640 1 -- 172.21.15.52:0/1807898247 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd6bc044070 con 0x7fd6cc103650 2024-04-02T13:57:24.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.481+0000 7fd6c2ffd640 1 --2- 172.21.15.52:0/1807898247 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd6b80a3d20 0x7fd6b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.481+0000 7fd6d1b3f640 1 --2- 172.21.15.52:0/1807898247 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd6b80a3d20 0x7fd6b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.481+0000 7fd6c2ffd640 1 -- 172.21.15.52:0/1807898247 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd6bc0f2790 con 0x7fd6cc103650 2024-04-02T13:57:24.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.482+0000 7fd6d1b3f640 1 --2- 172.21.15.52:0/1807898247 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd6b80a3d20 0x7fd6b80a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800baa0 tx=0x7fd6c8005d90 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:24.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.486+0000 7fd6c2ffd640 1 -- 172.21.15.52:0/1807898247 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd6bc0bec00 con 0x7fd6cc103650 2024-04-02T13:57:24.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.716+0000 7fd6d3da3640 1 -- 172.21.15.52:0/1807898247 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6cc1a5090 con 0x7fd6cc103650 2024-04-02T13:57:24.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.717+0000 7fd6c2ffd640 1 -- 172.21.15.52:0/1807898247 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11924 (secure 0 0 0) 0x7fd6bc0be500 con 0x7fd6cc103650 2024-04-02T13:57:24.722 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:57:24.722 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:55:15.239889+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24475,24503,24490,24457,24467]},"id":1}]} 2024-04-02T13:57:24.723 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 29 2024-04-02T13:57:24.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.719+0000 7fd6d3da3640 1 -- 172.21.15.52:0/1807898247 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd6b80a3d20 msgr2=0x7fd6b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.719+0000 7fd6d3da3640 1 --2- 172.21.15.52:0/1807898247 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd6b80a3d20 0x7fd6b80a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800baa0 tx=0x7fd6c8005d90 comp rx=0 tx=0).stop 2024-04-02T13:57:24.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.719+0000 7fd6d3da3640 1 -- 172.21.15.52:0/1807898247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6cc103650 msgr2=0x7fd6cc198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.719+0000 7fd6d3da3640 1 --2- 172.21.15.52:0/1807898247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6cc103650 0x7fd6cc198890 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc02efd0 tx=0x7fd6bc02ce50 comp rx=0 tx=0).stop 2024-04-02T13:57:24.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.719+0000 7fd6d3da3640 1 -- 172.21.15.52:0/1807898247 shutdown_connections 2024-04-02T13:57:24.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.719+0000 7fd6d3da3640 1 --2- 172.21.15.52:0/1807898247 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd6b80a3d20 0x7fd6b80a61e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.719+0000 7fd6d3da3640 1 --2- 172.21.15.52:0/1807898247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6cc10c100 0x7fd6cc19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.719+0000 7fd6d3da3640 1 --2- 172.21.15.52:0/1807898247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6cc103650 0x7fd6cc198890 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.719+0000 7fd6d3da3640 1 --2- 172.21.15.52:0/1807898247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6cc102d30 0x7fd6cc198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.719+0000 7fd6d3da3640 1 -- 172.21.15.52:0/1807898247 >> 172.21.15.52:0/1807898247 conn(0x7fd6cc076d80 msgr2=0x7fd6cc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:24.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.719+0000 7fd6d3da3640 1 -- 172.21.15.52:0/1807898247 shutdown_connections 2024-04-02T13:57:24.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.719+0000 7fd6d3da3640 1 -- 172.21.15.52:0/1807898247 wait complete. 2024-04-02T13:57:24.734 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:57:24.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.836+0000 7fb7bc575640 1 -- 172.21.15.52:0/253423437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7b4102450 msgr2=0x7fb7b410e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.836+0000 7fb7bc575640 1 --2- 172.21.15.52:0/253423437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7b4102450 0x7fb7b410e960 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009cd0 tx=0x7fb7ac02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.836+0000 7fb7bc575640 1 -- 172.21.15.52:0/253423437 shutdown_connections 2024-04-02T13:57:24.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.836+0000 7fb7bc575640 1 --2- 172.21.15.52:0/253423437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7b4102450 0x7fb7b410e960 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.836+0000 7fb7bc575640 1 --2- 172.21.15.52:0/253423437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7b4101a90 0x7fb7b4101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.836+0000 7fb7bc575640 1 --2- 172.21.15.52:0/253423437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7b4107aa0 0x7fb7b4107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.836+0000 7fb7bc575640 1 -- 172.21.15.52:0/253423437 >> 172.21.15.52:0/253423437 conn(0x7fb7b4078f00 msgr2=0x7fb7b4079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:24.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.836+0000 7fb7bc575640 1 -- 172.21.15.52:0/253423437 shutdown_connections 2024-04-02T13:57:24.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.836+0000 7fb7bc575640 1 -- 172.21.15.52:0/253423437 wait complete. 2024-04-02T13:57:24.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.837+0000 7fb7bc575640 1 Processor -- start 2024-04-02T13:57:24.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.837+0000 7fb7bc575640 1 -- start start 2024-04-02T13:57:24.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.837+0000 7fb7bc575640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7b4101a90 0x7fb7b41982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.837+0000 7fb7bc575640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7b4102450 0x7fb7b41987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.837+0000 7fb7bc575640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7b4107aa0 0x7fb7b419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.837+0000 7fb7bc575640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b4111eb0 con 0x7fb7b4102450 2024-04-02T13:57:24.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.837+0000 7fb7bc575640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b4111d30 con 0x7fb7b4107aa0 2024-04-02T13:57:24.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.837+0000 7fb7bc575640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b4112030 con 0x7fb7b4101a90 2024-04-02T13:57:24.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.837+0000 7fb7b9b10640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7b4102450 0x7fb7b41987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.837+0000 7fb7b9b10640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7b4102450 0x7fb7b41987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47996/0 (socket says 172.21.15.52:47996) 2024-04-02T13:57:24.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.837+0000 7fb7b9b10640 1 -- 172.21.15.52:0/403046604 learned_addr learned my addr 172.21.15.52:0/403046604 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:24.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.837+0000 7fb7ba311640 1 --2- 172.21.15.52:0/403046604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7b4101a90 0x7fb7b41982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.837+0000 7fb7bab12640 1 --2- 172.21.15.52:0/403046604 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7b4107aa0 0x7fb7b419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.837+0000 7fb7b9b10640 1 -- 172.21.15.52:0/403046604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7b4101a90 msgr2=0x7fb7b41982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.837+0000 7fb7b9b10640 1 --2- 172.21.15.52:0/403046604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7b4101a90 0x7fb7b41982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.837+0000 7fb7b9b10640 1 -- 172.21.15.52:0/403046604 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7b4107aa0 msgr2=0x7fb7b419cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.837+0000 7fb7b9b10640 1 --2- 172.21.15.52:0/403046604 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7b4107aa0 0x7fb7b419cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.837+0000 7fb7b9b10640 1 -- 172.21.15.52:0/403046604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b419d2a0 con 0x7fb7b4102450 2024-04-02T13:57:24.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.837+0000 7fb7bab12640 1 --2- 172.21.15.52:0/403046604 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7b4107aa0 0x7fb7b419cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:24.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.838+0000 7fb7b9b10640 1 --2- 172.21.15.52:0/403046604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7b4102450 0x7fb7b41987f0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400d770 tx=0x7fb7a400db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:24.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.838+0000 7fb7ab7fe640 1 -- 172.21.15.52:0/403046604 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4017070 con 0x7fb7b4102450 2024-04-02T13:57:24.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.838+0000 7fb7ab7fe640 1 -- 172.21.15.52:0/403046604 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a4009820 con 0x7fb7b4102450 2024-04-02T13:57:24.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.838+0000 7fb7bc575640 1 -- 172.21.15.52:0/403046604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b419d590 con 0x7fb7b4102450 2024-04-02T13:57:24.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.838+0000 7fb7ab7fe640 1 -- 172.21.15.52:0/403046604 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4004cd0 con 0x7fb7b4102450 2024-04-02T13:57:24.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.838+0000 7fb7bc575640 1 -- 172.21.15.52:0/403046604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b41a4e20 con 0x7fb7b4102450 2024-04-02T13:57:24.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.840+0000 7fb7ab7fe640 1 -- 172.21.15.52:0/403046604 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb7a40099d0 con 0x7fb7b4102450 2024-04-02T13:57:24.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.840+0000 7fb7bf238740 1 Processor -- start 2024-04-02T13:57:24.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.841+0000 7fb7bf238740 1 -- start start 2024-04-02T13:57:24.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.841+0000 7fb7ab7fe640 1 --2- 172.21.15.52:0/403046604 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb7a00a3d20 0x7fb7a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.841+0000 7fb7bf238740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654da3927a0 0x5654da392b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.841+0000 7fb7bf238740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654da3930c0 0x5654da39b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.841+0000 7fb7bf238740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da39bca0 0x5654da39e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.841+0000 7fb7bf238740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5654da2f67b0 con 0x5654da39bca0 2024-04-02T13:57:24.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.841+0000 7fb7bf238740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5654da295570 con 0x5654da3930c0 2024-04-02T13:57:24.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.841+0000 7fb7bf238740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5654da2f2760 con 0x5654da3927a0 2024-04-02T13:57:24.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.841+0000 7fb7bab12640 1 --2- 172.21.15.52:0/403046604 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb7a00a3d20 0x7fb7a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.841+0000 7fb7bab12640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da39bca0 0x5654da39e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.841+0000 7fb7ab7fe640 1 -- 172.21.15.52:0/403046604 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb7a40d05d0 con 0x7fb7b4102450 2024-04-02T13:57:24.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.841+0000 7fb7bab12640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da39bca0 0x5654da39e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48010/0 (socket says 172.21.15.52:48010) 2024-04-02T13:57:24.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.841+0000 7fb7bab12640 1 -- 172.21.15.52:0/2282756021 learned_addr learned my addr 172.21.15.52:0/2282756021 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:24.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.841+0000 7fb7b9b10640 1 --2- 172.21.15.52:0/2282756021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654da3930c0 0x5654da39b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.841+0000 7fb7ba311640 1 --2- 172.21.15.52:0/2282756021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654da3927a0 0x5654da392b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.841+0000 7fb7bab12640 1 -- 172.21.15.52:0/2282756021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654da3927a0 msgr2=0x5654da392b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:24.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.841+0000 7fb7bab12640 1 --2- 172.21.15.52:0/2282756021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654da3927a0 0x5654da392b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.841+0000 7fb7bab12640 1 -- 172.21.15.52:0/2282756021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654da3930c0 msgr2=0x5654da39b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:24.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.841+0000 7fb7bab12640 1 --2- 172.21.15.52:0/2282756021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654da3930c0 0x5654da39b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.841+0000 7fb7bab12640 1 -- 172.21.15.52:0/2282756021 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654da00a3f0 con 0x5654da39bca0 2024-04-02T13:57:24.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.841+0000 7fb7ba311640 1 --2- 172.21.15.52:0/2282756021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654da3927a0 0x5654da392b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:57:24.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.842+0000 7fb7bab12640 1 --2- 172.21.15.52:0/403046604 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb7a00a3d20 0x7fb7a00a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac02c700 tx=0x7fb7ac009910 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:24.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.842+0000 7fb7bab12640 1 --2- 172.21.15.52:0/2282756021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da39bca0 0x5654da39e090 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009cd0 tx=0x7fb7ac046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a6eb9324cdfde7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:24.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.842+0000 7fb7a8bfe640 1 -- 172.21.15.52:0/2282756021 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac0329b0 con 0x5654da39bca0 2024-04-02T13:57:24.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.842+0000 7fb7a8bfe640 1 -- 172.21.15.52:0/2282756021 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7ac051070 con 0x5654da39bca0 2024-04-02T13:57:24.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.842+0000 7fb7a8bfe640 1 -- 172.21.15.52:0/2282756021 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac004710 con 0x5654da39bca0 2024-04-02T13:57:24.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.842+0000 7fb7bf238740 1 -- 172.21.15.52:0/2282756021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da39bca0 msgr2=0x5654da39e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.842+0000 7fb7bf238740 1 --2- 172.21.15.52:0/2282756021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da39bca0 0x5654da39e090 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009cd0 tx=0x7fb7ac046000 comp rx=0 tx=0).stop 2024-04-02T13:57:24.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.843+0000 7fb7bf238740 1 -- 172.21.15.52:0/2282756021 shutdown_connections 2024-04-02T13:57:24.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.843+0000 7fb7bf238740 1 --2- 172.21.15.52:0/2282756021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da39bca0 0x5654da39e090 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.843+0000 7fb7bf238740 1 --2- 172.21.15.52:0/2282756021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654da3930c0 0x5654da39b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.843+0000 7fb7bf238740 1 --2- 172.21.15.52:0/2282756021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654da3927a0 0x5654da392b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.843+0000 7fb7bf238740 1 -- 172.21.15.52:0/2282756021 >> 172.21.15.52:0/2282756021 conn(0x5654da38ced0 msgr2=0x5654da38d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:24.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.843+0000 7fb7bf238740 1 -- 172.21.15.52:0/2282756021 shutdown_connections 2024-04-02T13:57:24.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.843+0000 7fb7bf238740 1 -- 172.21.15.52:0/2282756021 wait complete. 2024-04-02T13:57:24.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.843+0000 7fb7bf238740 1 -- start start 2024-04-02T13:57:24.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.843+0000 7fb7bf238740 1 Processor -- start 2024-04-02T13:57:24.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.843+0000 7fb7bf238740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da3927a0 0x5654da39b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.843+0000 7fb7bf238740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654da3930c0 0x5654da398ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.843+0000 7fb7bf238740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654da39bca0 0x5654da3991c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.843+0000 7fb7bf238740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5654da39ea30 con 0x5654da3927a0 2024-04-02T13:57:24.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.843+0000 7fb7bf238740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5654da2b1e80 con 0x5654da39bca0 2024-04-02T13:57:24.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.843+0000 7fb7bf238740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5654da39ebb0 con 0x5654da3930c0 2024-04-02T13:57:24.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.843+0000 7fb7ba311640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da3927a0 0x5654da39b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.843+0000 7fb7ba311640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da3927a0 0x5654da39b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48018/0 (socket says 172.21.15.52:48018) 2024-04-02T13:57:24.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.843+0000 7fb7ba311640 1 -- 172.21.15.52:0/395789439 learned_addr learned my addr 172.21.15.52:0/395789439 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:24.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.843+0000 7fb7bf238740 10 client.0 _ll_register_callbacks cb 0x5654da2edaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:24.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.844+0000 7fb7b9b10640 1 --2- 172.21.15.52:0/395789439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654da3930c0 0x5654da398ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.844+0000 7fb7ba311640 1 -- 172.21.15.52:0/395789439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654da3930c0 msgr2=0x5654da398ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:24.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.844+0000 7fb7ba311640 1 --2- 172.21.15.52:0/395789439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654da3930c0 0x5654da398ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.844+0000 7fb7bab12640 1 --2- 172.21.15.52:0/395789439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654da39bca0 0x5654da3991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.844+0000 7fb7ba311640 1 -- 172.21.15.52:0/395789439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654da39bca0 msgr2=0x5654da3991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:24.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.844+0000 7fb7ba311640 1 --2- 172.21.15.52:0/395789439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654da39bca0 0x5654da3991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.844+0000 7fb7ba311640 1 -- 172.21.15.52:0/395789439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654da0d47d0 con 0x5654da3927a0 2024-04-02T13:57:24.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.844+0000 7fb7bab12640 1 --2- 172.21.15.52:0/395789439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654da39bca0 0x5654da3991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:24.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.844+0000 7fb7ba311640 1 --2- 172.21.15.52:0/395789439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da3927a0 0x5654da39b740 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0010a20 tx=0x7fb7b0010ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d31e6875966560b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:24.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.844+0000 7fb79dffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:24.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.844+0000 7fb79dffb640 1 -- 172.21.15.52:0/395789439 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b001e060 con 0x5654da3927a0 2024-04-02T13:57:24.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.844+0000 7fb79dffb640 1 -- 172.21.15.52:0/395789439 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb7b000fc50 con 0x5654da3927a0 2024-04-02T13:57:24.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.844+0000 7fb7bf238740 1 -- 172.21.15.52:0/395789439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7ac051070 con 0x5654da3927a0 2024-04-02T13:57:24.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.844+0000 7fb79dffb640 1 -- 172.21.15.52:0/395789439 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb7b0025050 con 0x5654da3927a0 2024-04-02T13:57:24.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.844+0000 7fb7bf238740 10 client.16728 fetch_fsmap learned FSMap version 29 2024-04-02T13:57:24.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.844+0000 7fb7bf238740 1 -- 172.21.15.52:0/395789439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5654da3aabd0 con 0x5654da3927a0 2024-04-02T13:57:24.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.844+0000 7fb79dffb640 1 -- 172.21.15.52:0/395789439 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7ac051070 con 0x5654da3927a0 2024-04-02T13:57:24.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.844+0000 7fb79dffb640 1 -- 172.21.15.52:0/395789439 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fb7b000e3d0 con 0x5654da3927a0 2024-04-02T13:57:24.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.845+0000 7fb7bf238740 10 client.16728 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:57:24.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.845+0000 7fb7bf238740 10 client.16728 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T13:57:24.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.845+0000 7fb7bf238740 20 client.16728 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:24.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.845+0000 7fb7bf238740 1 --2- 172.21.15.52:0/395789439 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5654da3ac090 0x5654da3ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.845+0000 7fb7bf238740 4 client.16728 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:57:24.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.845+0000 7fb7bf238740 1 -- 172.21.15.52:0/395789439 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654da303cf0 con 0x5654da3ac090 2024-04-02T13:57:24.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.845+0000 7fb7b9b10640 1 --2- 172.21.15.52:0/395789439 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5654da3ac090 0x5654da3ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.845+0000 7fb7b9b10640 1 --2- 172.21.15.52:0/395789439 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5654da3ac090 0x5654da3ae4c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bcf83faf285c0642 server_cookie=111a627672036cf3 in_seq=0 out_seq=0 2024-04-02T13:57:24.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.845+0000 7fb79dffb640 10 client.16728 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:57:24.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.846+0000 7fb79dffb640 1 -- 172.21.15.52:0/395789439 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5654da303cf0 con 0x5654da3ac090 2024-04-02T13:57:24.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.846+0000 7fb79dffb640 10 client.16728 handle_command_reply: tid=0 2024-04-02T13:57:24.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.846+0000 7fb7bf238740 1 client.16728 shutdown 2024-04-02T13:57:24.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.846+0000 7fb7bf238740 1 -- 172.21.15.52:0/395789439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da3927a0 msgr2=0x5654da39b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.846+0000 7fb7bf238740 1 --2- 172.21.15.52:0/395789439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da3927a0 0x5654da39b740 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0010a20 tx=0x7fb7b0010ee0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.846+0000 7fb7bf238740 1 -- 172.21.15.52:0/395789439 shutdown_connections 2024-04-02T13:57:24.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.846+0000 7fb7bf238740 1 --2- 172.21.15.52:0/395789439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654da39bca0 0x5654da3991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.846+0000 7fb7bf238740 1 --2- 172.21.15.52:0/395789439 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5654da3ac090 0x5654da3ae4c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.846+0000 7fb7bf238740 1 --2- 172.21.15.52:0/395789439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654da3930c0 0x5654da398ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.846+0000 7fb7bf238740 1 --2- 172.21.15.52:0/395789439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da3927a0 0x5654da39b740 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.846+0000 7fb79dffb640 0 client.16728 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:57:24.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.846+0000 7fb7bf238740 1 -- 172.21.15.52:0/395789439 >> 172.21.15.52:0/395789439 conn(0x5654da38ced0 msgr2=0x5654da38ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:24.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.846+0000 7fb7bf238740 1 -- 172.21.15.52:0/395789439 shutdown_connections 2024-04-02T13:57:24.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.846+0000 7fb7bf238740 1 -- 172.21.15.52:0/395789439 wait complete. 2024-04-02T13:57:24.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.846+0000 7fb7bf238740 20 client.16728 trim_cache size 0 max 16384 2024-04-02T13:57:24.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.860+0000 7fb7bf238740 1 Processor -- start 2024-04-02T13:57:24.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.860+0000 7fb7bf238740 1 -- start start 2024-04-02T13:57:24.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.860+0000 7fb7bf238740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654da3927a0 0x5654da398a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.860+0000 7fb7bf238740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654da3930c0 0x5654da3a7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.860+0000 7fb7bf238740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da39bca0 0x5654da39b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.860+0000 7fb7bf238740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b41aed30 con 0x5654da39bca0 2024-04-02T13:57:24.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.860+0000 7fb7bf238740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b41b08a0 con 0x5654da3927a0 2024-04-02T13:57:24.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.860+0000 7fb7bf238740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b41af400 con 0x5654da3930c0 2024-04-02T13:57:24.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.860+0000 7fb7bab12640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da39bca0 0x5654da39b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.860+0000 7fb7bab12640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da39bca0 0x5654da39b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48030/0 (socket says 172.21.15.52:48030) 2024-04-02T13:57:24.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.860+0000 7fb7bab12640 1 -- 172.21.15.52:0/1619454492 learned_addr learned my addr 172.21.15.52:0/1619454492 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:24.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.860+0000 7fb7b9b10640 1 --2- 172.21.15.52:0/1619454492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654da3930c0 0x5654da3a7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.860+0000 7fb7bab12640 1 -- 172.21.15.52:0/1619454492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654da3930c0 msgr2=0x5654da3a7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:24.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.860+0000 7fb7ba311640 1 --2- 172.21.15.52:0/1619454492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654da3927a0 0x5654da398a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.860+0000 7fb7bab12640 1 --2- 172.21.15.52:0/1619454492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654da3930c0 0x5654da3a7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.860+0000 7fb7bab12640 1 -- 172.21.15.52:0/1619454492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654da3927a0 msgr2=0x5654da398a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:24.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.860+0000 7fb7bab12640 1 --2- 172.21.15.52:0/1619454492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654da3927a0 0x5654da398a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.860+0000 7fb7bab12640 1 -- 172.21.15.52:0/1619454492 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654da392ef0 con 0x5654da39bca0 2024-04-02T13:57:24.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.860+0000 7fb7b9b10640 1 --2- 172.21.15.52:0/1619454492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654da3930c0 0x5654da3a7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:24.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.860+0000 7fb7ba311640 1 --2- 172.21.15.52:0/1619454492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654da3927a0 0x5654da398a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:24.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.860+0000 7fb7bab12640 1 --2- 172.21.15.52:0/1619454492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da39bca0 0x5654da39b200 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac02cfd0 tx=0x7fb7ac005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e81a09a66e1f28 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:24.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.861+0000 7fb79ffff640 1 -- 172.21.15.52:0/1619454492 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac059070 con 0x5654da39bca0 2024-04-02T13:57:24.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.861+0000 7fb79ffff640 1 -- 172.21.15.52:0/1619454492 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7ac051050 con 0x5654da39bca0 2024-04-02T13:57:24.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.861+0000 7fb79ffff640 1 -- 172.21.15.52:0/1619454492 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac004600 con 0x5654da39bca0 2024-04-02T13:57:24.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.861+0000 7fb7bf238740 1 -- 172.21.15.52:0/1619454492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da39bca0 msgr2=0x5654da39b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.861+0000 7fb7bf238740 1 --2- 172.21.15.52:0/1619454492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da39bca0 0x5654da39b200 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac02cfd0 tx=0x7fb7ac005f70 comp rx=0 tx=0).stop 2024-04-02T13:57:24.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.861+0000 7fb7bf238740 1 -- 172.21.15.52:0/1619454492 shutdown_connections 2024-04-02T13:57:24.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.861+0000 7fb7bf238740 1 --2- 172.21.15.52:0/1619454492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da39bca0 0x5654da39b200 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.861+0000 7fb7bf238740 1 --2- 172.21.15.52:0/1619454492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654da3930c0 0x5654da3a7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.861+0000 7fb7bf238740 1 --2- 172.21.15.52:0/1619454492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654da3927a0 0x5654da398a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.861+0000 7fb7bf238740 1 -- 172.21.15.52:0/1619454492 >> 172.21.15.52:0/1619454492 conn(0x5654da38ced0 msgr2=0x5654da3973a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:24.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.861+0000 7fb7bf238740 1 -- 172.21.15.52:0/1619454492 shutdown_connections 2024-04-02T13:57:24.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.861+0000 7fb7bf238740 1 -- 172.21.15.52:0/1619454492 wait complete. 2024-04-02T13:57:24.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.861+0000 7fb7bf238740 1 -- start start 2024-04-02T13:57:24.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.862+0000 7fb7bf238740 1 Processor -- start 2024-04-02T13:57:24.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.862+0000 7fb7bf238740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654da3927a0 0x5654da39c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.862+0000 7fb7bf238740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654da3930c0 0x5654da39cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.862+0000 7fb7bf238740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da39bca0 0x5654da3a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.862+0000 7fb7bf238740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b41b01a0 con 0x5654da39bca0 2024-04-02T13:57:24.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.862+0000 7fb7bf238740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b41afad0 con 0x5654da3927a0 2024-04-02T13:57:24.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.862+0000 7fb7bf238740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b41acd10 con 0x5654da3930c0 2024-04-02T13:57:24.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.862+0000 7fb7bf238740 10 client.0 _ll_register_callbacks cb 0x7fb7b000a7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:24.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.862+0000 7fb7bab12640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da39bca0 0x5654da3a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.862+0000 7fb7bab12640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da39bca0 0x5654da3a2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48032/0 (socket says 172.21.15.52:48032) 2024-04-02T13:57:24.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.862+0000 7fb7bab12640 1 -- 172.21.15.52:0/1447880569 learned_addr learned my addr 172.21.15.52:0/1447880569 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:24.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.862+0000 7fb7ba311640 1 --2- 172.21.15.52:0/1447880569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654da3927a0 0x5654da39c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.862+0000 7fb7b9b10640 1 --2- 172.21.15.52:0/1447880569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654da3930c0 0x5654da39cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.862+0000 7fb7bab12640 1 -- 172.21.15.52:0/1447880569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654da3930c0 msgr2=0x5654da39cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:24.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.862+0000 7fb7bab12640 1 --2- 172.21.15.52:0/1447880569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654da3930c0 0x5654da39cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.862+0000 7fb7bab12640 1 -- 172.21.15.52:0/1447880569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654da3927a0 msgr2=0x5654da39c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:24.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.862+0000 7fb7bab12640 1 --2- 172.21.15.52:0/1447880569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654da3927a0 0x5654da39c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.862+0000 7fb7bab12640 1 -- 172.21.15.52:0/1447880569 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654da3a33c0 con 0x5654da39bca0 2024-04-02T13:57:24.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.862+0000 7fb7b9b10640 1 --2- 172.21.15.52:0/1447880569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654da3930c0 0x5654da39cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:24.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.862+0000 7fb7bab12640 1 --2- 172.21.15.52:0/1447880569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da39bca0 0x5654da3a2e80 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac02c630 tx=0x7fb7ac032a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd572266d1ce7cc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:24.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.862+0000 7fb79dffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:24.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.862+0000 7fb79dffb640 1 -- 172.21.15.52:0/1447880569 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac062070 con 0x5654da39bca0 2024-04-02T13:57:24.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.863+0000 7fb79dffb640 1 -- 172.21.15.52:0/1447880569 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb7ac003d80 con 0x5654da39bca0 2024-04-02T13:57:24.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.863+0000 7fb7bf238740 1 -- 172.21.15.52:0/1447880569 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7ac051050 con 0x5654da39bca0 2024-04-02T13:57:24.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.863+0000 7fb79dffb640 1 -- 172.21.15.52:0/1447880569 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb7ac059040 con 0x5654da39bca0 2024-04-02T13:57:24.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.863+0000 7fb7bf238740 10 client.16734 fetch_fsmap learned FSMap version 29 2024-04-02T13:57:24.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.863+0000 7fb7bf238740 1 -- 172.21.15.52:0/1447880569 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5654da3a4630 con 0x5654da39bca0 2024-04-02T13:57:24.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.863+0000 7fb79dffb640 1 -- 172.21.15.52:0/1447880569 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7ac051050 con 0x5654da39bca0 2024-04-02T13:57:24.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.863+0000 7fb79dffb640 1 -- 172.21.15.52:0/1447880569 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fb7ac04b070 con 0x5654da39bca0 2024-04-02T13:57:24.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.863+0000 7fb7bf238740 10 client.16734 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:57:24.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.863+0000 7fb7bf238740 10 client.16734 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T13:57:24.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.863+0000 7fb7bf238740 20 client.16734 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:24.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.863+0000 7fb7bf238740 1 --2- 172.21.15.52:0/1447880569 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5654da416a60 0x5654da418e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.863+0000 7fb7bf238740 4 client.16734 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:57:24.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.863+0000 7fb7bf238740 1 -- 172.21.15.52:0/1447880569 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5654da2fe040 con 0x5654da416a60 2024-04-02T13:57:24.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.863+0000 7fb7ba311640 1 --2- 172.21.15.52:0/1447880569 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5654da416a60 0x5654da418e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.864+0000 7fb79dffb640 10 client.16734 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:57:24.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.864+0000 7fb7ba311640 1 --2- 172.21.15.52:0/1447880569 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5654da416a60 0x5654da418e50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=64cece18f7400de0 server_cookie=4982bf8727cec406 in_seq=0 out_seq=0 2024-04-02T13:57:24.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.864+0000 7fb79dffb640 1 -- 172.21.15.52:0/1447880569 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5654da2fe040 con 0x5654da416a60 2024-04-02T13:57:24.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.864+0000 7fb79dffb640 10 client.16734 handle_command_reply: tid=0 2024-04-02T13:57:24.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.864+0000 7fb7bf238740 1 client.16734 shutdown 2024-04-02T13:57:24.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.864+0000 7fb7bf238740 1 -- 172.21.15.52:0/1447880569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da39bca0 msgr2=0x5654da3a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.864+0000 7fb7bf238740 1 --2- 172.21.15.52:0/1447880569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da39bca0 0x5654da3a2e80 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac02c630 tx=0x7fb7ac032a60 comp rx=0 tx=0).stop 2024-04-02T13:57:24.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.864+0000 7fb7bf238740 1 -- 172.21.15.52:0/1447880569 shutdown_connections 2024-04-02T13:57:24.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.864+0000 7fb7bf238740 1 --2- 172.21.15.52:0/1447880569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654da39bca0 0x5654da3a2e80 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.864+0000 7fb7bf238740 1 --2- 172.21.15.52:0/1447880569 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5654da416a60 0x5654da418e50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.864+0000 7fb7bf238740 1 --2- 172.21.15.52:0/1447880569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654da3930c0 0x5654da39cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.864+0000 7fb7bf238740 1 --2- 172.21.15.52:0/1447880569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654da3927a0 0x5654da39c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.864+0000 7fb79dffb640 0 client.16734 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:57:24.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.864+0000 7fb7bf238740 1 -- 172.21.15.52:0/1447880569 >> 172.21.15.52:0/1447880569 conn(0x5654da38ced0 msgr2=0x5654da38b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:24.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.864+0000 7fb7bf238740 1 -- 172.21.15.52:0/1447880569 shutdown_connections 2024-04-02T13:57:24.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.864+0000 7fb7bf238740 1 -- 172.21.15.52:0/1447880569 wait complete. 2024-04-02T13:57:24.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.864+0000 7fb7bf238740 20 client.16734 trim_cache size 0 max 16384 2024-04-02T13:57:24.879 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:57:24.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.865+0000 7fb7bc575640 1 -- 172.21.15.52:0/403046604 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb7a00a3d20 msgr2=0x7fb7a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.865+0000 7fb7bc575640 1 --2- 172.21.15.52:0/403046604 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb7a00a3d20 0x7fb7a00a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac02c700 tx=0x7fb7ac009910 comp rx=0 tx=0).stop 2024-04-02T13:57:24.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.865+0000 7fb7bc575640 1 -- 172.21.15.52:0/403046604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7b4102450 msgr2=0x7fb7b41987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.865+0000 7fb7bc575640 1 --2- 172.21.15.52:0/403046604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7b4102450 0x7fb7b41987f0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400d770 tx=0x7fb7a400db70 comp rx=0 tx=0).stop 2024-04-02T13:57:24.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.865+0000 7fb7bc575640 1 -- 172.21.15.52:0/403046604 shutdown_connections 2024-04-02T13:57:24.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.865+0000 7fb7bc575640 1 --2- 172.21.15.52:0/403046604 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb7a00a3d20 0x7fb7a00a61e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.865+0000 7fb7bc575640 1 --2- 172.21.15.52:0/403046604 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7b4107aa0 0x7fb7b419cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.865+0000 7fb7bc575640 1 --2- 172.21.15.52:0/403046604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7b4102450 0x7fb7b41987f0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.865+0000 7fb7bc575640 1 --2- 172.21.15.52:0/403046604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7b4101a90 0x7fb7b41982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.865+0000 7fb7bc575640 1 -- 172.21.15.52:0/403046604 >> 172.21.15.52:0/403046604 conn(0x7fb7b4078f00 msgr2=0x7fb7b4104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:24.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.865+0000 7fb7bc575640 1 -- 172.21.15.52:0/403046604 shutdown_connections 2024-04-02T13:57:24.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.865+0000 7fb7bc575640 1 -- 172.21.15.52:0/403046604 wait complete. 2024-04-02T13:57:24.882 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:57:24.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.984+0000 7f5874e0f640 1 -- 172.21.15.52:0/847606719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58701069c0 msgr2=0x7f5870106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.984+0000 7f5874e0f640 1 --2- 172.21.15.52:0/847606719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58701069c0 0x7f5870106da0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5868009930 tx=0x7f586802bc30 comp rx=0 tx=0).stop 2024-04-02T13:57:24.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.984+0000 7f5874e0f640 1 -- 172.21.15.52:0/847606719 shutdown_connections 2024-04-02T13:57:24.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.984+0000 7f5874e0f640 1 --2- 172.21.15.52:0/847606719 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5870101370 0x7f587010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.984+0000 7f5874e0f640 1 --2- 172.21.15.52:0/847606719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58701009b0 0x7f5870100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.984+0000 7f5874e0f640 1 --2- 172.21.15.52:0/847606719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58701069c0 0x7f5870106da0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.984+0000 7f5874e0f640 1 -- 172.21.15.52:0/847606719 >> 172.21.15.52:0/847606719 conn(0x7f5870076d80 msgr2=0x7f5870077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:24.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.984+0000 7f5874e0f640 1 -- 172.21.15.52:0/847606719 shutdown_connections 2024-04-02T13:57:24.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.984+0000 7f5874e0f640 1 -- 172.21.15.52:0/847606719 wait complete. 2024-04-02T13:57:24.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.985+0000 7f5874e0f640 1 Processor -- start 2024-04-02T13:57:24.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.985+0000 7f5874e0f640 1 -- start start 2024-04-02T13:57:24.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.985+0000 7f5874e0f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58701009b0 0x7f5870199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.985+0000 7f5874e0f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5870101370 0x7f5870199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.985+0000 7f5874e0f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58701069c0 0x7f58701a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:24.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.985+0000 7f5874e0f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5870110da0 con 0x7f58701009b0 2024-04-02T13:57:24.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.985+0000 7f5874e0f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5870110c20 con 0x7f5870101370 2024-04-02T13:57:24.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.985+0000 7f5874e0f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5870110f20 con 0x7f58701069c0 2024-04-02T13:57:24.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.986+0000 7f586e59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58701009b0 0x7f5870199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.986+0000 7f586e59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58701009b0 0x7f5870199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48044/0 (socket says 172.21.15.52:48044) 2024-04-02T13:57:24.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.986+0000 7f586e59c640 1 -- 172.21.15.52:0/915247828 learned_addr learned my addr 172.21.15.52:0/915247828 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:24.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.986+0000 7f586ed9d640 1 --2- 172.21.15.52:0/915247828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58701069c0 0x7f58701a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.986+0000 7f586dd9b640 1 --2- 172.21.15.52:0/915247828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5870101370 0x7f5870199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:24.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.986+0000 7f586e59c640 1 -- 172.21.15.52:0/915247828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58701069c0 msgr2=0x7f58701a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.986+0000 7f586e59c640 1 --2- 172.21.15.52:0/915247828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58701069c0 0x7f58701a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.986+0000 7f586e59c640 1 -- 172.21.15.52:0/915247828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5870101370 msgr2=0x7f5870199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:24.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.986+0000 7f586e59c640 1 --2- 172.21.15.52:0/915247828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5870101370 0x7f5870199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:24.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.986+0000 7f586e59c640 1 -- 172.21.15.52:0/915247828 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58701a47c0 con 0x7f58701009b0 2024-04-02T13:57:24.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.986+0000 7f586dd9b640 1 --2- 172.21.15.52:0/915247828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5870101370 0x7f5870199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:24.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.986+0000 7f586e59c640 1 --2- 172.21.15.52:0/915247828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58701009b0 0x7f5870199200 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5868009230 tx=0x7f586802cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:24.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.986+0000 7f58677fe640 1 -- 172.21.15.52:0/915247828 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868041070 con 0x7f58701009b0 2024-04-02T13:57:24.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.986+0000 7f58677fe640 1 -- 172.21.15.52:0/915247828 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5868004450 con 0x7f58701009b0 2024-04-02T13:57:24.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.986+0000 7f5874e0f640 1 -- 172.21.15.52:0/915247828 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58701a4990 con 0x7f58701009b0 2024-04-02T13:57:24.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.986+0000 7f58677fe640 1 -- 172.21.15.52:0/915247828 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868034440 con 0x7f58701009b0 2024-04-02T13:57:24.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.986+0000 7f5874e0f640 1 -- 172.21.15.52:0/915247828 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58701a4df0 con 0x7f58701009b0 2024-04-02T13:57:25.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.989+0000 7f58677fe640 1 -- 172.21.15.52:0/915247828 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5868044070 con 0x7f58701009b0 2024-04-02T13:57:25.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.989+0000 7f58779f1740 1 Processor -- start 2024-04-02T13:57:25.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.989+0000 7f58779f1740 1 -- start start 2024-04-02T13:57:25.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.989+0000 7f58779f1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dce7a0 0x563f69dceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.989+0000 7f58779f1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563f69dcf0c0 0x563f69dd7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.989+0000 7f58779f1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f69dd7ca0 0x563f69dda090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.989+0000 7f58779f1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563f69d327b0 con 0x563f69dce7a0 2024-04-02T13:57:25.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.989+0000 7f58779f1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563f69cd1570 con 0x563f69dcf0c0 2024-04-02T13:57:25.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.989+0000 7f58779f1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563f69d2e760 con 0x563f69dd7ca0 2024-04-02T13:57:25.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.989+0000 7f58677fe640 1 --2- 172.21.15.52:0/915247828 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f585c0a3d20 0x7f585c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.989+0000 7f586ed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dce7a0 0x563f69dceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.989+0000 7f586ed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dce7a0 0x563f69dceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48060/0 (socket says 172.21.15.52:48060) 2024-04-02T13:57:25.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.989+0000 7f586ed9d640 1 -- 172.21.15.52:0/3776844475 learned_addr learned my addr 172.21.15.52:0/3776844475 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:25.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.989+0000 7f586e59c640 1 --2- 172.21.15.52:0/3776844475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563f69dcf0c0 0x563f69dd7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.989+0000 7f586ed9d640 1 --2- 172.21.15.52:0/915247828 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f585c0a3d20 0x7f585c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.990+0000 7f58677fe640 1 -- 172.21.15.52:0/915247828 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f58680f2580 con 0x7f58701009b0 2024-04-02T13:57:25.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.990+0000 7f586dd9b640 1 --2- 172.21.15.52:0/3776844475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f69dd7ca0 0x563f69dda090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.990+0000 7f586ed9d640 1 -- 172.21.15.52:0/3776844475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f69dd7ca0 msgr2=0x563f69dda090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.990+0000 7f586ed9d640 1 --2- 172.21.15.52:0/3776844475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f69dd7ca0 0x563f69dda090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.990+0000 7f586ed9d640 1 -- 172.21.15.52:0/3776844475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563f69dcf0c0 msgr2=0x563f69dd7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.990+0000 7f586ed9d640 1 --2- 172.21.15.52:0/3776844475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563f69dcf0c0 0x563f69dd7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.990+0000 7f586ed9d640 1 -- 172.21.15.52:0/3776844475 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f69a463f0 con 0x563f69dce7a0 2024-04-02T13:57:25.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.990+0000 7f586ed9d640 1 --2- 172.21.15.52:0/3776844475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dce7a0 0x563f69dceb80 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5860012d00 tx=0x7f5860010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9997ccb913cf07b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:25.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.990+0000 7f586ed9d640 1 --2- 172.21.15.52:0/915247828 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f585c0a3d20 0x7f585c0a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5860010c40 tx=0x7f586000c000 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:25.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.990+0000 7f5864ff9640 1 -- 172.21.15.52:0/3776844475 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860024070 con 0x563f69dce7a0 2024-04-02T13:57:25.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.990+0000 7f5864ff9640 1 -- 172.21.15.52:0/3776844475 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5860009d20 con 0x563f69dce7a0 2024-04-02T13:57:25.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.990+0000 7f5864ff9640 1 -- 172.21.15.52:0/3776844475 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58600053c0 con 0x563f69dce7a0 2024-04-02T13:57:25.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.991+0000 7f58779f1740 1 -- 172.21.15.52:0/3776844475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dce7a0 msgr2=0x563f69dceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.991+0000 7f58779f1740 1 --2- 172.21.15.52:0/3776844475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dce7a0 0x563f69dceb80 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5860012d00 tx=0x7f5860010740 comp rx=0 tx=0).stop 2024-04-02T13:57:25.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.991+0000 7f58779f1740 1 -- 172.21.15.52:0/3776844475 shutdown_connections 2024-04-02T13:57:25.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.991+0000 7f58779f1740 1 --2- 172.21.15.52:0/3776844475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f69dd7ca0 0x563f69dda090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.991+0000 7f58779f1740 1 --2- 172.21.15.52:0/3776844475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563f69dcf0c0 0x563f69dd7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.991+0000 7f58779f1740 1 --2- 172.21.15.52:0/3776844475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dce7a0 0x563f69dceb80 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.991+0000 7f58779f1740 1 -- 172.21.15.52:0/3776844475 >> 172.21.15.52:0/3776844475 conn(0x563f69dc8ed0 msgr2=0x563f69dc92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:25.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.991+0000 7f58779f1740 1 -- 172.21.15.52:0/3776844475 shutdown_connections 2024-04-02T13:57:25.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.991+0000 7f58779f1740 1 -- 172.21.15.52:0/3776844475 wait complete. 2024-04-02T13:57:25.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.991+0000 7f58779f1740 1 -- start start 2024-04-02T13:57:25.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.991+0000 7f58779f1740 1 Processor -- start 2024-04-02T13:57:25.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.992+0000 7f58779f1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dce7a0 0x563f69dd7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.992+0000 7f58779f1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f69dcf0c0 0x563f69dd4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.992+0000 7f58779f1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563f69dd7ca0 0x563f69dd5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.992+0000 7f58779f1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563f69ddaa30 con 0x563f69dce7a0 2024-04-02T13:57:25.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.992+0000 7f58779f1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563f69cede80 con 0x563f69dd7ca0 2024-04-02T13:57:25.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.992+0000 7f58779f1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563f69ddabb0 con 0x563f69dcf0c0 2024-04-02T13:57:25.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.992+0000 7f58779f1740 10 client.0 _ll_register_callbacks cb 0x563f69d29aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:25.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.992+0000 7f586e59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dce7a0 0x563f69dd7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.992+0000 7f586e59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dce7a0 0x563f69dd7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48062/0 (socket says 172.21.15.52:48062) 2024-04-02T13:57:25.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.992+0000 7f586e59c640 1 -- 172.21.15.52:0/217427507 learned_addr learned my addr 172.21.15.52:0/217427507 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:25.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.992+0000 7f586dd9b640 1 --2- 172.21.15.52:0/217427507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f69dcf0c0 0x563f69dd4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.992+0000 7f586ed9d640 1 --2- 172.21.15.52:0/217427507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563f69dd7ca0 0x563f69dd5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.992+0000 7f586e59c640 1 -- 172.21.15.52:0/217427507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f69dcf0c0 msgr2=0x563f69dd4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.992+0000 7f586e59c640 1 --2- 172.21.15.52:0/217427507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f69dcf0c0 0x563f69dd4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.992+0000 7f586e59c640 1 -- 172.21.15.52:0/217427507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563f69dd7ca0 msgr2=0x563f69dd5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.992+0000 7f586e59c640 1 --2- 172.21.15.52:0/217427507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563f69dd7ca0 0x563f69dd5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.992+0000 7f586e59c640 1 -- 172.21.15.52:0/217427507 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f69b107d0 con 0x563f69dce7a0 2024-04-02T13:57:25.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.992+0000 7f586e59c640 1 --2- 172.21.15.52:0/217427507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dce7a0 0x563f69dd7750 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5868006010 tx=0x7f5868009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f12d5bafed2d75b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:25.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.992+0000 7f5855ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:25.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.992+0000 7f5855ffb640 1 -- 172.21.15.52:0/217427507 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58680d7060 con 0x563f69dce7a0 2024-04-02T13:57:25.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.992+0000 7f5855ffb640 1 -- 172.21.15.52:0/217427507 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f58680efbb0 con 0x563f69dce7a0 2024-04-02T13:57:25.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.992+0000 7f58779f1740 1 -- 172.21.15.52:0/217427507 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563f69d3fcf0 con 0x563f69dce7a0 2024-04-02T13:57:25.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.992+0000 7f5855ffb640 1 -- 172.21.15.52:0/217427507 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f586804b040 con 0x563f69dce7a0 2024-04-02T13:57:25.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.992+0000 7f58779f1740 10 client.16746 fetch_fsmap learned FSMap version 29 2024-04-02T13:57:25.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.992+0000 7f58779f1740 1 -- 172.21.15.52:0/217427507 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563f69de6ae0 con 0x563f69dce7a0 2024-04-02T13:57:25.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.993+0000 7f5855ffb640 1 -- 172.21.15.52:0/217427507 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563f69d3fcf0 con 0x563f69dce7a0 2024-04-02T13:57:25.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.993+0000 7f5855ffb640 1 -- 172.21.15.52:0/217427507 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f58680be4f0 con 0x563f69dce7a0 2024-04-02T13:57:25.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.993+0000 7f58779f1740 10 client.16746 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:57:25.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.993+0000 7f58779f1740 10 client.16746 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T13:57:25.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.993+0000 7f58779f1740 20 client.16746 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:25.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.993+0000 7f58779f1740 1 --2- 172.21.15.52:0/217427507 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x563f69de7f90 0x563f69dea3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.993+0000 7f58779f1740 4 client.16746 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:57:25.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.993+0000 7f58779f1740 1 -- 172.21.15.52:0/217427507 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f69d3a040 con 0x563f69de7f90 2024-04-02T13:57:25.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.993+0000 7f586dd9b640 1 --2- 172.21.15.52:0/217427507 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x563f69de7f90 0x563f69dea3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.994+0000 7f5855ffb640 10 client.16746 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:57:25.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.993+0000 7f586dd9b640 1 --2- 172.21.15.52:0/217427507 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x563f69de7f90 0x563f69dea3e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a51c22d11ff5b9f7 server_cookie=6d109f290841b4f8 in_seq=0 out_seq=0 2024-04-02T13:57:25.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.994+0000 7f5855ffb640 1 -- 172.21.15.52:0/217427507 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f69d3a040 con 0x563f69de7f90 2024-04-02T13:57:25.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.994+0000 7f5855ffb640 10 client.16746 handle_command_reply: tid=0 2024-04-02T13:57:25.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.994+0000 7f58779f1740 1 client.16746 shutdown 2024-04-02T13:57:25.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.994+0000 7f58779f1740 1 -- 172.21.15.52:0/217427507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dce7a0 msgr2=0x563f69dd7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.994+0000 7f58779f1740 1 --2- 172.21.15.52:0/217427507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dce7a0 0x563f69dd7750 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5868006010 tx=0x7f5868009260 comp rx=0 tx=0).stop 2024-04-02T13:57:25.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.994+0000 7f58779f1740 1 -- 172.21.15.52:0/217427507 shutdown_connections 2024-04-02T13:57:25.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.994+0000 7f58779f1740 1 --2- 172.21.15.52:0/217427507 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x563f69de7f90 0x563f69dea3e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.994+0000 7f58779f1740 1 --2- 172.21.15.52:0/217427507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563f69dd7ca0 0x563f69dd5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.994+0000 7f58779f1740 1 --2- 172.21.15.52:0/217427507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f69dcf0c0 0x563f69dd4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.994+0000 7f5855ffb640 0 client.16746 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:57:25.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.994+0000 7f58779f1740 1 --2- 172.21.15.52:0/217427507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dce7a0 0x563f69dd7750 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.994+0000 7f58779f1740 1 -- 172.21.15.52:0/217427507 >> 172.21.15.52:0/217427507 conn(0x563f69dc8ed0 msgr2=0x563f69dcac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:25.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.994+0000 7f58779f1740 1 -- 172.21.15.52:0/217427507 shutdown_connections 2024-04-02T13:57:25.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.994+0000 7f58779f1740 1 -- 172.21.15.52:0/217427507 wait complete. 2024-04-02T13:57:25.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:24.995+0000 7f58779f1740 20 client.16746 trim_cache size 0 max 16384 2024-04-02T13:57:25.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.008+0000 7f58779f1740 1 Processor -- start 2024-04-02T13:57:25.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.008+0000 7f58779f1740 1 -- start start 2024-04-02T13:57:25.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.008+0000 7f58779f1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563f69dce7a0 0x563f69de38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.008+0000 7f58779f1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dcf0c0 0x563f69dd7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.008+0000 7f58779f1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f69dd7ca0 0x563f69dd7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.008+0000 7f58779f1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f58701aedc0 con 0x563f69dcf0c0 2024-04-02T13:57:25.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.008+0000 7f58779f1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f58701b0930 con 0x563f69dce7a0 2024-04-02T13:57:25.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.008+0000 7f58779f1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f58701af490 con 0x563f69dd7ca0 2024-04-02T13:57:25.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.008+0000 7f586dd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dcf0c0 0x563f69dd7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.008+0000 7f586dd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dcf0c0 0x563f69dd7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48076/0 (socket says 172.21.15.52:48076) 2024-04-02T13:57:25.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.008+0000 7f586dd9b640 1 -- 172.21.15.52:0/3704822741 learned_addr learned my addr 172.21.15.52:0/3704822741 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:25.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.008+0000 7f586e59c640 1 --2- 172.21.15.52:0/3704822741 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563f69dce7a0 0x563f69de38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.008+0000 7f586ed9d640 1 --2- 172.21.15.52:0/3704822741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f69dd7ca0 0x563f69dd7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.008+0000 7f586dd9b640 1 -- 172.21.15.52:0/3704822741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f69dd7ca0 msgr2=0x563f69dd7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.008+0000 7f586dd9b640 1 --2- 172.21.15.52:0/3704822741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f69dd7ca0 0x563f69dd7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.008+0000 7f586dd9b640 1 -- 172.21.15.52:0/3704822741 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563f69dce7a0 msgr2=0x563f69de38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.008+0000 7f586dd9b640 1 --2- 172.21.15.52:0/3704822741 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563f69dce7a0 0x563f69de38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.008+0000 7f586dd9b640 1 -- 172.21.15.52:0/3704822741 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f69dceef0 con 0x563f69dcf0c0 2024-04-02T13:57:25.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.008+0000 7f586ed9d640 1 --2- 172.21.15.52:0/3704822741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f69dd7ca0 0x563f69dd7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:25.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.009+0000 7f586dd9b640 1 --2- 172.21.15.52:0/3704822741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dcf0c0 0x563f69dd7210 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f5858005fd0 tx=0x7f5858004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c9978cead3af34 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:25.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.009+0000 7f5857fff640 1 -- 172.21.15.52:0/3704822741 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858022020 con 0x563f69dcf0c0 2024-04-02T13:57:25.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.009+0000 7f5857fff640 1 -- 172.21.15.52:0/3704822741 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f585801e900 con 0x563f69dcf0c0 2024-04-02T13:57:25.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.009+0000 7f5857fff640 1 -- 172.21.15.52:0/3704822741 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858012de0 con 0x563f69dcf0c0 2024-04-02T13:57:25.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.009+0000 7f58779f1740 1 -- 172.21.15.52:0/3704822741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dcf0c0 msgr2=0x563f69dd7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.009+0000 7f58779f1740 1 --2- 172.21.15.52:0/3704822741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dcf0c0 0x563f69dd7210 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f5858005fd0 tx=0x7f5858004410 comp rx=0 tx=0).stop 2024-04-02T13:57:25.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.009+0000 7f58779f1740 1 -- 172.21.15.52:0/3704822741 shutdown_connections 2024-04-02T13:57:25.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.009+0000 7f58779f1740 1 --2- 172.21.15.52:0/3704822741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f69dd7ca0 0x563f69dd7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.009+0000 7f58779f1740 1 --2- 172.21.15.52:0/3704822741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dcf0c0 0x563f69dd7210 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.009+0000 7f58779f1740 1 --2- 172.21.15.52:0/3704822741 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563f69dce7a0 0x563f69de38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.009+0000 7f58779f1740 1 -- 172.21.15.52:0/3704822741 >> 172.21.15.52:0/3704822741 conn(0x563f69dc8ed0 msgr2=0x563f69dd33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:25.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.009+0000 7f58779f1740 1 -- 172.21.15.52:0/3704822741 shutdown_connections 2024-04-02T13:57:25.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.009+0000 7f58779f1740 1 -- 172.21.15.52:0/3704822741 wait complete. 2024-04-02T13:57:25.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.010+0000 7f58779f1740 1 -- start start 2024-04-02T13:57:25.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.010+0000 7f58779f1740 1 Processor -- start 2024-04-02T13:57:25.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.010+0000 7f58779f1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563f69dce7a0 0x563f69de0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.010+0000 7f58779f1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dcf0c0 0x563f69dd12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.010+0000 7f58779f1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f69dd7ca0 0x563f69dd1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.010+0000 7f58779f1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f58701b0230 con 0x563f69dcf0c0 2024-04-02T13:57:25.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.010+0000 7f58779f1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f58701afb60 con 0x563f69dce7a0 2024-04-02T13:57:25.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.010+0000 7f58779f1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f58701acda0 con 0x563f69dd7ca0 2024-04-02T13:57:25.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.010+0000 7f58779f1740 10 client.0 _ll_register_callbacks cb 0x7f5868009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:25.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.010+0000 7f586dd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dcf0c0 0x563f69dd12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.010+0000 7f586dd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dcf0c0 0x563f69dd12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48080/0 (socket says 172.21.15.52:48080) 2024-04-02T13:57:25.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.010+0000 7f586dd9b640 1 -- 172.21.15.52:0/3863550688 learned_addr learned my addr 172.21.15.52:0/3863550688 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:25.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.010+0000 7f586e59c640 1 --2- 172.21.15.52:0/3863550688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563f69dce7a0 0x563f69de0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.010+0000 7f586dd9b640 1 -- 172.21.15.52:0/3863550688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f69dd7ca0 msgr2=0x563f69dd1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.010+0000 7f586ed9d640 1 --2- 172.21.15.52:0/3863550688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f69dd7ca0 0x563f69dd1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.010+0000 7f586dd9b640 1 --2- 172.21.15.52:0/3863550688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f69dd7ca0 0x563f69dd1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.010+0000 7f586dd9b640 1 -- 172.21.15.52:0/3863550688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563f69dce7a0 msgr2=0x563f69de0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.010+0000 7f586dd9b640 1 --2- 172.21.15.52:0/3863550688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563f69dce7a0 0x563f69de0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.010+0000 7f586dd9b640 1 -- 172.21.15.52:0/3863550688 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f69dd1e70 con 0x563f69dcf0c0 2024-04-02T13:57:25.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.010+0000 7f586e59c640 1 --2- 172.21.15.52:0/3863550688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563f69dce7a0 0x563f69de0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:25.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.011+0000 7f586dd9b640 1 --2- 172.21.15.52:0/3863550688 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dcf0c0 0x563f69dd12f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x563f69dcee80 tx=0x7f5858003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03443788da5e272 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:25.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.011+0000 7f5855ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:25.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.011+0000 7f5855ffb640 1 -- 172.21.15.52:0/3863550688 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858022030 con 0x563f69dcf0c0 2024-04-02T13:57:25.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.011+0000 7f5855ffb640 1 -- 172.21.15.52:0/3863550688 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f5858005040 con 0x563f69dcf0c0 2024-04-02T13:57:25.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.011+0000 7f58779f1740 1 -- 172.21.15.52:0/3863550688 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f585801e900 con 0x563f69dcf0c0 2024-04-02T13:57:25.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.011+0000 7f5855ffb640 1 -- 172.21.15.52:0/3863550688 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f585803a050 con 0x563f69dcf0c0 2024-04-02T13:57:25.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.011+0000 7f58779f1740 10 client.16752 fetch_fsmap learned FSMap version 29 2024-04-02T13:57:25.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.011+0000 7f58779f1740 1 -- 172.21.15.52:0/3863550688 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563f69ddfbc0 con 0x563f69dcf0c0 2024-04-02T13:57:25.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.011+0000 7f5855ffb640 1 -- 172.21.15.52:0/3863550688 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f585801e900 con 0x563f69dcf0c0 2024-04-02T13:57:25.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.011+0000 7f5855ffb640 1 -- 172.21.15.52:0/3863550688 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f58580124e0 con 0x563f69dcf0c0 2024-04-02T13:57:25.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.011+0000 7f58779f1740 10 client.16752 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:57:25.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.011+0000 7f58779f1740 10 client.16752 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T13:57:25.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.011+0000 7f58779f1740 20 client.16752 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:25.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.011+0000 7f58779f1740 1 --2- 172.21.15.52:0/3863550688 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x563f69e4e9e0 0x563f69e50e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.011+0000 7f58779f1740 4 client.16752 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:57:25.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.011+0000 7f58779f1740 1 -- 172.21.15.52:0/3863550688 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563f69cb76b0 con 0x563f69e4e9e0 2024-04-02T13:57:25.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.011+0000 7f586e59c640 1 --2- 172.21.15.52:0/3863550688 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x563f69e4e9e0 0x563f69e50e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.012+0000 7f5855ffb640 10 client.16752 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:57:25.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.012+0000 7f586e59c640 1 --2- 172.21.15.52:0/3863550688 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x563f69e4e9e0 0x563f69e50e00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=95dc1ae6f3666109 server_cookie=8899e4ae3c86d17 in_seq=0 out_seq=0 2024-04-02T13:57:25.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.012+0000 7f5855ffb640 1 -- 172.21.15.52:0/3863550688 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563f69cb76b0 con 0x563f69e4e9e0 2024-04-02T13:57:25.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.012+0000 7f5855ffb640 10 client.16752 handle_command_reply: tid=0 2024-04-02T13:57:25.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.012+0000 7f58779f1740 1 client.16752 shutdown 2024-04-02T13:57:25.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.012+0000 7f58779f1740 1 -- 172.21.15.52:0/3863550688 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dcf0c0 msgr2=0x563f69dd12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.012+0000 7f58779f1740 1 --2- 172.21.15.52:0/3863550688 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dcf0c0 0x563f69dd12f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x563f69dcee80 tx=0x7f5858003df0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.012+0000 7f58779f1740 1 -- 172.21.15.52:0/3863550688 shutdown_connections 2024-04-02T13:57:25.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.012+0000 7f58779f1740 1 --2- 172.21.15.52:0/3863550688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f69dd7ca0 0x563f69dd1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.012+0000 7f58779f1740 1 --2- 172.21.15.52:0/3863550688 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f69dcf0c0 0x563f69dd12f0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.012+0000 7f58779f1740 1 --2- 172.21.15.52:0/3863550688 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x563f69e4e9e0 0x563f69e50e00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.012+0000 7f58779f1740 1 --2- 172.21.15.52:0/3863550688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563f69dce7a0 0x563f69de0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.012+0000 7f5855ffb640 0 client.16752 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:57:25.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.012+0000 7f58779f1740 1 -- 172.21.15.52:0/3863550688 >> 172.21.15.52:0/3863550688 conn(0x563f69dc8ed0 msgr2=0x563f69dca1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:25.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.012+0000 7f58779f1740 1 -- 172.21.15.52:0/3863550688 shutdown_connections 2024-04-02T13:57:25.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.012+0000 7f58779f1740 1 -- 172.21.15.52:0/3863550688 wait complete. 2024-04-02T13:57:25.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.012+0000 7f58779f1740 20 client.16752 trim_cache size 0 max 16384 2024-04-02T13:57:25.028 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:57:25.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.013+0000 7f5874e0f640 1 -- 172.21.15.52:0/915247828 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f585c0a3d20 msgr2=0x7f585c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.013+0000 7f5874e0f640 1 --2- 172.21.15.52:0/915247828 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f585c0a3d20 0x7f585c0a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5860010c40 tx=0x7f586000c000 comp rx=0 tx=0).stop 2024-04-02T13:57:25.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.013+0000 7f5874e0f640 1 -- 172.21.15.52:0/915247828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58701009b0 msgr2=0x7f5870199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.013+0000 7f5874e0f640 1 --2- 172.21.15.52:0/915247828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58701009b0 0x7f5870199200 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5868009230 tx=0x7f586802cc70 comp rx=0 tx=0).stop 2024-04-02T13:57:25.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.013+0000 7f5874e0f640 1 -- 172.21.15.52:0/915247828 shutdown_connections 2024-04-02T13:57:25.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.013+0000 7f5874e0f640 1 --2- 172.21.15.52:0/915247828 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f585c0a3d20 0x7f585c0a61e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.013+0000 7f5874e0f640 1 --2- 172.21.15.52:0/915247828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58701069c0 0x7f58701a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.013+0000 7f5874e0f640 1 --2- 172.21.15.52:0/915247828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5870101370 0x7f5870199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.013+0000 7f5874e0f640 1 --2- 172.21.15.52:0/915247828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58701009b0 0x7f5870199200 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.013+0000 7f5874e0f640 1 -- 172.21.15.52:0/915247828 >> 172.21.15.52:0/915247828 conn(0x7f5870076d80 msgr2=0x7f58700fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:25.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.013+0000 7f5874e0f640 1 -- 172.21.15.52:0/915247828 shutdown_connections 2024-04-02T13:57:25.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.013+0000 7f5874e0f640 1 -- 172.21.15.52:0/915247828 wait complete. 2024-04-02T13:57:25.031 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:57:25.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.133+0000 7f31667a3640 1 -- 172.21.15.52:0/587645397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3160101370 msgr2=0x7f316010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.133+0000 7f31667a3640 1 --2- 172.21.15.52:0/587645397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3160101370 0x7f316010d880 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f3158009cd0 tx=0x7f315802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.133+0000 7f31667a3640 1 -- 172.21.15.52:0/587645397 shutdown_connections 2024-04-02T13:57:25.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.133+0000 7f31667a3640 1 --2- 172.21.15.52:0/587645397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3160101370 0x7f316010d880 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.133+0000 7f31667a3640 1 --2- 172.21.15.52:0/587645397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31601009b0 0x7f3160100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.133+0000 7f31667a3640 1 --2- 172.21.15.52:0/587645397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31601069c0 0x7f3160106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.133+0000 7f31667a3640 1 -- 172.21.15.52:0/587645397 >> 172.21.15.52:0/587645397 conn(0x7f3160076d80 msgr2=0x7f3160077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:25.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.133+0000 7f31667a3640 1 -- 172.21.15.52:0/587645397 shutdown_connections 2024-04-02T13:57:25.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.133+0000 7f31667a3640 1 -- 172.21.15.52:0/587645397 wait complete. 2024-04-02T13:57:25.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.133+0000 7f31667a3640 1 Processor -- start 2024-04-02T13:57:25.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.133+0000 7f31667a3640 1 -- start start 2024-04-02T13:57:25.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.134+0000 7f31667a3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31601009b0 0x7f3160198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.134+0000 7f31667a3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3160101370 0x7f3160198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.134+0000 7f31667a3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31601069c0 0x7f316019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.134+0000 7f31667a3640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3160110dd0 con 0x7f31601069c0 2024-04-02T13:57:25.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.134+0000 7f31667a3640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3160110c50 con 0x7f3160101370 2024-04-02T13:57:25.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.134+0000 7f31667a3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3160110f50 con 0x7f31601009b0 2024-04-02T13:57:25.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.134+0000 7f3164d40640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31601069c0 0x7f316019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.134+0000 7f3164d40640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31601069c0 0x7f316019cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48092/0 (socket says 172.21.15.52:48092) 2024-04-02T13:57:25.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.134+0000 7f3164d40640 1 -- 172.21.15.52:0/3824685331 learned_addr learned my addr 172.21.15.52:0/3824685331 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:25.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.134+0000 7f315ffff640 1 --2- 172.21.15.52:0/3824685331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31601009b0 0x7f3160198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.134+0000 7f315f7fe640 1 --2- 172.21.15.52:0/3824685331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3160101370 0x7f3160198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.134+0000 7f3164d40640 1 -- 172.21.15.52:0/3824685331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31601009b0 msgr2=0x7f3160198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.134+0000 7f3164d40640 1 --2- 172.21.15.52:0/3824685331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31601009b0 0x7f3160198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.134+0000 7f3164d40640 1 -- 172.21.15.52:0/3824685331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3160101370 msgr2=0x7f3160198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.134+0000 7f3164d40640 1 --2- 172.21.15.52:0/3824685331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3160101370 0x7f3160198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.134+0000 7f3164d40640 1 -- 172.21.15.52:0/3824685331 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f316019d320 con 0x7f31601069c0 2024-04-02T13:57:25.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.134+0000 7f3164d40640 1 --2- 172.21.15.52:0/3824685331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31601069c0 0x7f316019cbf0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f31580348f0 tx=0x7f3158032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:25.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.135+0000 7f315d7fa640 1 -- 172.21.15.52:0/3824685331 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158041020 con 0x7f31601069c0 2024-04-02T13:57:25.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.135+0000 7f31667a3640 1 -- 172.21.15.52:0/3824685331 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f316019d5b0 con 0x7f31601069c0 2024-04-02T13:57:25.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.135+0000 7f315d7fa640 1 -- 172.21.15.52:0/3824685331 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3158034cd0 con 0x7f31601069c0 2024-04-02T13:57:25.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.135+0000 7f315d7fa640 1 -- 172.21.15.52:0/3824685331 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158039040 con 0x7f31601069c0 2024-04-02T13:57:25.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.135+0000 7f31667a3640 1 -- 172.21.15.52:0/3824685331 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31601a4ea0 con 0x7f31601069c0 2024-04-02T13:57:25.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.137+0000 7f315d7fa640 1 -- 172.21.15.52:0/3824685331 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3158044020 con 0x7f31601069c0 2024-04-02T13:57:25.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.137+0000 7f3169381740 1 Processor -- start 2024-04-02T13:57:25.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.137+0000 7f3169381740 1 -- start start 2024-04-02T13:57:25.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.137+0000 7f315d7fa640 1 --2- 172.21.15.52:0/3824685331 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f31480a3d20 0x7f31480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.137+0000 7f3169381740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9069477a0 0x55c906947b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.137+0000 7f3169381740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9069480c0 0x55c906950760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.138+0000 7f3169381740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c906950ca0 0x55c906953090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.138+0000 7f3169381740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c9068ab7b0 con 0x55c9069480c0 2024-04-02T13:57:25.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.138+0000 7f315d7fa640 1 -- 172.21.15.52:0/3824685331 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f31580f2820 con 0x7f31601069c0 2024-04-02T13:57:25.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.138+0000 7f3169381740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c90684a570 con 0x55c906950ca0 2024-04-02T13:57:25.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.138+0000 7f3169381740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c9068a7760 con 0x55c9069477a0 2024-04-02T13:57:25.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.138+0000 7f315f7fe640 1 --2- 172.21.15.52:0/3824685331 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f31480a3d20 0x7f31480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.138+0000 7f315f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9069480c0 0x55c906950760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.138+0000 7f315f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9069480c0 0x55c906950760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48094/0 (socket says 172.21.15.52:48094) 2024-04-02T13:57:25.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.138+0000 7f315f7fe640 1 -- 172.21.15.52:0/2073818591 learned_addr learned my addr 172.21.15.52:0/2073818591 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:25.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.138+0000 7f315ffff640 1 --2- 172.21.15.52:0/2073818591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9069477a0 0x55c906947b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.138+0000 7f3164d40640 1 --2- 172.21.15.52:0/2073818591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c906950ca0 0x55c906953090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.138+0000 7f315ffff640 1 -- 172.21.15.52:0/2073818591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c906950ca0 msgr2=0x55c906953090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.138+0000 7f315ffff640 1 --2- 172.21.15.52:0/2073818591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c906950ca0 0x55c906953090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.138+0000 7f315ffff640 1 -- 172.21.15.52:0/2073818591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9069480c0 msgr2=0x55c906950760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.138+0000 7f315ffff640 1 --2- 172.21.15.52:0/2073818591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9069480c0 0x55c906950760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.138+0000 7f315ffff640 1 -- 172.21.15.52:0/2073818591 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9065bf3f0 con 0x55c9069477a0 2024-04-02T13:57:25.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.138+0000 7f315f7fe640 1 --2- 172.21.15.52:0/2073818591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9069480c0 0x55c906950760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:57:25.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.138+0000 7f315f7fe640 1 --2- 172.21.15.52:0/3824685331 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f31480a3d20 0x7f31480a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3154012960 tx=0x7f315400bb70 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:25.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.138+0000 7f315ffff640 1 --2- 172.21.15.52:0/2073818591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9069477a0 0x55c906947b80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f315000bb70 tx=0x7f31500107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3032b169df713b61 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:25.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.138+0000 7f31467fc640 1 -- 172.21.15.52:0/2073818591 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3150004030 con 0x55c9069477a0 2024-04-02T13:57:25.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.138+0000 7f31467fc640 1 -- 172.21.15.52:0/2073818591 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31500041d0 con 0x55c9069477a0 2024-04-02T13:57:25.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.138+0000 7f31467fc640 1 -- 172.21.15.52:0/2073818591 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31500116a0 con 0x55c9069477a0 2024-04-02T13:57:25.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.139+0000 7f3169381740 1 -- 172.21.15.52:0/2073818591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9069477a0 msgr2=0x55c906947b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.139+0000 7f3169381740 1 --2- 172.21.15.52:0/2073818591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9069477a0 0x55c906947b80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f315000bb70 tx=0x7f31500107a0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.139+0000 7f3169381740 1 -- 172.21.15.52:0/2073818591 shutdown_connections 2024-04-02T13:57:25.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.139+0000 7f3169381740 1 --2- 172.21.15.52:0/2073818591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c906950ca0 0x55c906953090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.139+0000 7f3169381740 1 --2- 172.21.15.52:0/2073818591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9069480c0 0x55c906950760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.139+0000 7f3169381740 1 --2- 172.21.15.52:0/2073818591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9069477a0 0x55c906947b80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.139+0000 7f3169381740 1 -- 172.21.15.52:0/2073818591 >> 172.21.15.52:0/2073818591 conn(0x55c906941ed0 msgr2=0x55c9069422b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:25.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.139+0000 7f3169381740 1 -- 172.21.15.52:0/2073818591 shutdown_connections 2024-04-02T13:57:25.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.139+0000 7f3169381740 1 -- 172.21.15.52:0/2073818591 wait complete. 2024-04-02T13:57:25.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.139+0000 7f3169381740 1 -- start start 2024-04-02T13:57:25.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.140+0000 7f3169381740 1 Processor -- start 2024-04-02T13:57:25.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.140+0000 7f3169381740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9069477a0 0x55c906950750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.140+0000 7f3169381740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9069480c0 0x55c90694dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.140+0000 7f3169381740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c906950ca0 0x55c90694e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.140+0000 7f3169381740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c906953a90 con 0x55c906950ca0 2024-04-02T13:57:25.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.140+0000 7f3169381740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c906866e80 con 0x55c9069480c0 2024-04-02T13:57:25.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.140+0000 7f3169381740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c906953c10 con 0x55c9069477a0 2024-04-02T13:57:25.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.140+0000 7f3169381740 10 client.0 _ll_register_callbacks cb 0x55c9068a2aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:25.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.140+0000 7f3164d40640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c906950ca0 0x55c90694e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.140+0000 7f3164d40640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c906950ca0 0x55c90694e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48110/0 (socket says 172.21.15.52:48110) 2024-04-02T13:57:25.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.140+0000 7f3164d40640 1 -- 172.21.15.52:0/2347141439 learned_addr learned my addr 172.21.15.52:0/2347141439 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:25.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.140+0000 7f315ffff640 1 --2- 172.21.15.52:0/2347141439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9069477a0 0x55c906950750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.140+0000 7f315f7fe640 1 --2- 172.21.15.52:0/2347141439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9069480c0 0x55c90694dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.140+0000 7f3164d40640 1 -- 172.21.15.52:0/2347141439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9069477a0 msgr2=0x55c906950750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.140+0000 7f3164d40640 1 --2- 172.21.15.52:0/2347141439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9069477a0 0x55c906950750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.140+0000 7f3164d40640 1 -- 172.21.15.52:0/2347141439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9069480c0 msgr2=0x55c90694dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.140+0000 7f3164d40640 1 --2- 172.21.15.52:0/2347141439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9069480c0 0x55c90694dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.140+0000 7f3164d40640 1 -- 172.21.15.52:0/2347141439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9066897d0 con 0x55c906950ca0 2024-04-02T13:57:25.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.141+0000 7f3164d40640 1 --2- 172.21.15.52:0/2347141439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c906950ca0 0x55c90694e220 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f31580baf60 tx=0x7f31580be360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebf6835b5aae486 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:25.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.141+0000 7f313f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:25.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.141+0000 7f313f7fe640 1 -- 172.21.15.52:0/2347141439 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31580bec90 con 0x55c906950ca0 2024-04-02T13:57:25.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.141+0000 7f313f7fe640 1 -- 172.21.15.52:0/2347141439 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f31580c6d70 con 0x55c906950ca0 2024-04-02T13:57:25.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.141+0000 7f3169381740 1 -- 172.21.15.52:0/2347141439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31500041d0 con 0x55c906950ca0 2024-04-02T13:57:25.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.141+0000 7f313f7fe640 1 -- 172.21.15.52:0/2347141439 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f31580be590 con 0x55c906950ca0 2024-04-02T13:57:25.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.141+0000 7f3169381740 10 client.16764 fetch_fsmap learned FSMap version 29 2024-04-02T13:57:25.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.141+0000 7f3169381740 1 -- 172.21.15.52:0/2347141439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c90695fb40 con 0x55c906950ca0 2024-04-02T13:57:25.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.141+0000 7f313f7fe640 1 -- 172.21.15.52:0/2347141439 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31500041d0 con 0x55c906950ca0 2024-04-02T13:57:25.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.141+0000 7f313f7fe640 1 -- 172.21.15.52:0/2347141439 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f315803c030 con 0x55c906950ca0 2024-04-02T13:57:25.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.141+0000 7f3169381740 10 client.16764 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:57:25.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.141+0000 7f3169381740 10 client.16764 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T13:57:25.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.141+0000 7f3169381740 20 client.16764 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:25.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.141+0000 7f3169381740 1 --2- 172.21.15.52:0/2347141439 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55c906961030 0x55c906963460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.141+0000 7f3169381740 4 client.16764 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:57:25.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.141+0000 7f3169381740 1 -- 172.21.15.52:0/2347141439 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9068b8cf0 con 0x55c906961030 2024-04-02T13:57:25.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.142+0000 7f315ffff640 1 --2- 172.21.15.52:0/2347141439 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55c906961030 0x55c906963460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.142+0000 7f315ffff640 1 --2- 172.21.15.52:0/2347141439 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55c906961030 0x55c906963460 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f9bf6521a2177425 server_cookie=cd5522976b52e967 in_seq=0 out_seq=0 2024-04-02T13:57:25.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.142+0000 7f313f7fe640 10 client.16764 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:57:25.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.143+0000 7f313f7fe640 1 -- 172.21.15.52:0/2347141439 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c9068b8cf0 con 0x55c906961030 2024-04-02T13:57:25.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.143+0000 7f313f7fe640 10 client.16764 handle_command_reply: tid=0 2024-04-02T13:57:25.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.143+0000 7f3169381740 1 client.16764 shutdown 2024-04-02T13:57:25.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.143+0000 7f3169381740 1 -- 172.21.15.52:0/2347141439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c906950ca0 msgr2=0x55c90694e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.143+0000 7f3169381740 1 --2- 172.21.15.52:0/2347141439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c906950ca0 0x55c90694e220 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f31580baf60 tx=0x7f31580be360 comp rx=0 tx=0).stop 2024-04-02T13:57:25.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.143+0000 7f3169381740 1 -- 172.21.15.52:0/2347141439 shutdown_connections 2024-04-02T13:57:25.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.143+0000 7f3169381740 1 --2- 172.21.15.52:0/2347141439 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55c906961030 0x55c906963460 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.143+0000 7f3169381740 1 --2- 172.21.15.52:0/2347141439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c906950ca0 0x55c90694e220 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.143+0000 7f3169381740 1 --2- 172.21.15.52:0/2347141439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9069480c0 0x55c90694dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.143+0000 7f313f7fe640 0 client.16764 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:57:25.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.143+0000 7f3169381740 1 --2- 172.21.15.52:0/2347141439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9069477a0 0x55c906950750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.143+0000 7f3169381740 1 -- 172.21.15.52:0/2347141439 >> 172.21.15.52:0/2347141439 conn(0x55c906941ed0 msgr2=0x55c906943c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:25.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.143+0000 7f3169381740 1 -- 172.21.15.52:0/2347141439 shutdown_connections 2024-04-02T13:57:25.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.143+0000 7f3169381740 1 -- 172.21.15.52:0/2347141439 wait complete. 2024-04-02T13:57:25.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.143+0000 7f3169381740 20 client.16764 trim_cache size 0 max 16384 2024-04-02T13:57:25.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.156+0000 7f3169381740 1 Processor -- start 2024-04-02T13:57:25.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.156+0000 7f3169381740 1 -- start start 2024-04-02T13:57:25.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.156+0000 7f3169381740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9069477a0 0x55c90695c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.156+0000 7f3169381740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9069480c0 0x55c906968d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.156+0000 7f3169381740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c906950ca0 0x55c906969280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.156+0000 7f3169381740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f31601aed60 con 0x55c9069477a0 2024-04-02T13:57:25.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.156+0000 7f3169381740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f31601b08d0 con 0x55c906950ca0 2024-04-02T13:57:25.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.156+0000 7f3169381740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f31601af430 con 0x55c9069480c0 2024-04-02T13:57:25.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.156+0000 7f315ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9069477a0 0x55c90695c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.157+0000 7f315ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9069477a0 0x55c90695c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48120/0 (socket says 172.21.15.52:48120) 2024-04-02T13:57:25.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.157+0000 7f315ffff640 1 -- 172.21.15.52:0/2069469172 learned_addr learned my addr 172.21.15.52:0/2069469172 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:25.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.157+0000 7f315ffff640 1 -- 172.21.15.52:0/2069469172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9069480c0 msgr2=0x55c906968d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.157+0000 7f315ffff640 1 --2- 172.21.15.52:0/2069469172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9069480c0 0x55c906968d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.157+0000 7f315ffff640 1 -- 172.21.15.52:0/2069469172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c906950ca0 msgr2=0x55c906969280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.157+0000 7f315ffff640 1 --2- 172.21.15.52:0/2069469172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c906950ca0 0x55c906969280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.157+0000 7f315ffff640 1 -- 172.21.15.52:0/2069469172 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c906953970 con 0x55c9069477a0 2024-04-02T13:57:25.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.157+0000 7f315ffff640 1 --2- 172.21.15.52:0/2069469172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9069477a0 0x55c90695c950 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f315000bb70 tx=0x7f3150020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146e7d5a721872d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:25.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.157+0000 7f3145ffb640 1 -- 172.21.15.52:0/2069469172 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315002e070 con 0x55c9069477a0 2024-04-02T13:57:25.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.157+0000 7f3145ffb640 1 -- 172.21.15.52:0/2069469172 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f315000be60 con 0x55c9069477a0 2024-04-02T13:57:25.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.157+0000 7f3145ffb640 1 -- 172.21.15.52:0/2069469172 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31500052a0 con 0x55c9069477a0 2024-04-02T13:57:25.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.158+0000 7f3169381740 1 -- 172.21.15.52:0/2069469172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9069477a0 msgr2=0x55c90695c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.158+0000 7f3169381740 1 --2- 172.21.15.52:0/2069469172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9069477a0 0x55c90695c950 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f315000bb70 tx=0x7f3150020040 comp rx=0 tx=0).stop 2024-04-02T13:57:25.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.158+0000 7f3169381740 1 -- 172.21.15.52:0/2069469172 shutdown_connections 2024-04-02T13:57:25.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.158+0000 7f3169381740 1 --2- 172.21.15.52:0/2069469172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c906950ca0 0x55c906969280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.158+0000 7f3169381740 1 --2- 172.21.15.52:0/2069469172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9069480c0 0x55c906968d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.158+0000 7f3169381740 1 --2- 172.21.15.52:0/2069469172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9069477a0 0x55c90695c950 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.158+0000 7f3169381740 1 -- 172.21.15.52:0/2069469172 >> 172.21.15.52:0/2069469172 conn(0x55c906941ed0 msgr2=0x55c90694c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.158+0000 7f3169381740 1 -- 172.21.15.52:0/2069469172 shutdown_connections 2024-04-02T13:57:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.158+0000 7f3169381740 1 -- 172.21.15.52:0/2069469172 wait complete. 2024-04-02T13:57:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.158+0000 7f3169381740 1 -- start start 2024-04-02T13:57:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.158+0000 7f3169381740 1 Processor -- start 2024-04-02T13:57:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.158+0000 7f3169381740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9069477a0 0x55c9069593d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.158+0000 7f3169381740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9069480c0 0x55c90694add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.158+0000 7f3169381740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c906950ca0 0x55c90694b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.158+0000 7f3169381740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f31601b01d0 con 0x55c9069477a0 2024-04-02T13:57:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.158+0000 7f3169381740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f31601afb00 con 0x55c9069480c0 2024-04-02T13:57:25.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.158+0000 7f3169381740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f31601acd40 con 0x55c906950ca0 2024-04-02T13:57:25.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.158+0000 7f3169381740 10 client.0 _ll_register_callbacks cb 0x7f31580ba110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:25.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.159+0000 7f315ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9069477a0 0x55c9069593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.159+0000 7f315ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9069477a0 0x55c9069593d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48124/0 (socket says 172.21.15.52:48124) 2024-04-02T13:57:25.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.159+0000 7f315ffff640 1 -- 172.21.15.52:0/317116343 learned_addr learned my addr 172.21.15.52:0/317116343 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:25.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.159+0000 7f315f7fe640 1 --2- 172.21.15.52:0/317116343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9069480c0 0x55c90694add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.159+0000 7f3164d40640 1 --2- 172.21.15.52:0/317116343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c906950ca0 0x55c90694b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.159+0000 7f315ffff640 1 -- 172.21.15.52:0/317116343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c906950ca0 msgr2=0x55c90694b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.159+0000 7f315ffff640 1 --2- 172.21.15.52:0/317116343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c906950ca0 0x55c90694b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.159+0000 7f315ffff640 1 -- 172.21.15.52:0/317116343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9069480c0 msgr2=0x55c90694add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.159+0000 7f315ffff640 1 --2- 172.21.15.52:0/317116343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9069480c0 0x55c90694add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.159+0000 7f315ffff640 1 -- 172.21.15.52:0/317116343 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c906951920 con 0x55c9069477a0 2024-04-02T13:57:25.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.159+0000 7f3164d40640 1 --2- 172.21.15.52:0/317116343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c906950ca0 0x55c90694b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:25.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.159+0000 7f315ffff640 1 --2- 172.21.15.52:0/317116343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9069477a0 0x55c9069593d0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f315000ba90 tx=0x7f3150010e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781cef6506dfa5a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:25.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.159+0000 7f313f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:25.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.159+0000 7f313f7fe640 1 -- 172.21.15.52:0/317116343 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315002e040 con 0x55c9069477a0 2024-04-02T13:57:25.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.159+0000 7f313f7fe640 1 -- 172.21.15.52:0/317116343 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f31500037f0 con 0x55c9069477a0 2024-04-02T13:57:25.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.159+0000 7f3169381740 1 -- 172.21.15.52:0/317116343 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f315000be60 con 0x55c9069477a0 2024-04-02T13:57:25.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.159+0000 7f313f7fe640 1 -- 172.21.15.52:0/317116343 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3150003d90 con 0x55c9069477a0 2024-04-02T13:57:25.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.159+0000 7f3169381740 10 client.16770 fetch_fsmap learned FSMap version 29 2024-04-02T13:57:25.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.159+0000 7f3169381740 1 -- 172.21.15.52:0/317116343 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c9069c7790 con 0x55c9069477a0 2024-04-02T13:57:25.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.160+0000 7f313f7fe640 1 -- 172.21.15.52:0/317116343 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f315000be60 con 0x55c9069477a0 2024-04-02T13:57:25.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.160+0000 7f313f7fe640 1 -- 172.21.15.52:0/317116343 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f3150031070 con 0x55c9069477a0 2024-04-02T13:57:25.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.160+0000 7f3169381740 10 client.16770 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:57:25.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.160+0000 7f3169381740 10 client.16770 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T13:57:25.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.160+0000 7f3169381740 20 client.16770 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:25.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.160+0000 7f3169381740 1 --2- 172.21.15.52:0/317116343 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55c9069c7c50 0x55c9069ca040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.160+0000 7f3169381740 4 client.16770 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:57:25.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.160+0000 7f3169381740 1 -- 172.21.15.52:0/317116343 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c9068b3040 con 0x55c9069c7c50 2024-04-02T13:57:25.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.160+0000 7f315f7fe640 1 --2- 172.21.15.52:0/317116343 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55c9069c7c50 0x55c9069ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.161+0000 7f313f7fe640 10 client.16770 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:57:25.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.161+0000 7f315f7fe640 1 --2- 172.21.15.52:0/317116343 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55c9069c7c50 0x55c9069ca040 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2069776c56996dc3 server_cookie=c9717b96cd0f1c1f in_seq=0 out_seq=0 2024-04-02T13:57:25.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.161+0000 7f313f7fe640 1 -- 172.21.15.52:0/317116343 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c9068b3040 con 0x55c9069c7c50 2024-04-02T13:57:25.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.161+0000 7f313f7fe640 10 client.16770 handle_command_reply: tid=0 2024-04-02T13:57:25.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.161+0000 7f3169381740 1 client.16770 shutdown 2024-04-02T13:57:25.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.161+0000 7f3169381740 1 -- 172.21.15.52:0/317116343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9069477a0 msgr2=0x55c9069593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.161+0000 7f3169381740 1 --2- 172.21.15.52:0/317116343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9069477a0 0x55c9069593d0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f315000ba90 tx=0x7f3150010e40 comp rx=0 tx=0).stop 2024-04-02T13:57:25.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.161+0000 7f3169381740 1 -- 172.21.15.52:0/317116343 shutdown_connections 2024-04-02T13:57:25.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.161+0000 7f3169381740 1 --2- 172.21.15.52:0/317116343 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55c9069c7c50 0x55c9069ca040 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.161+0000 7f3169381740 1 --2- 172.21.15.52:0/317116343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c906950ca0 0x55c90694b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.161+0000 7f3169381740 1 --2- 172.21.15.52:0/317116343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c9069480c0 0x55c90694add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.161+0000 7f313f7fe640 0 client.16770 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:57:25.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.161+0000 7f3169381740 1 --2- 172.21.15.52:0/317116343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9069477a0 0x55c9069593d0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.161+0000 7f3169381740 1 -- 172.21.15.52:0/317116343 >> 172.21.15.52:0/317116343 conn(0x55c906941ed0 msgr2=0x55c9069406a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:25.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.161+0000 7f3169381740 1 -- 172.21.15.52:0/317116343 shutdown_connections 2024-04-02T13:57:25.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.161+0000 7f3169381740 1 -- 172.21.15.52:0/317116343 wait complete. 2024-04-02T13:57:25.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.161+0000 7f3169381740 20 client.16770 trim_cache size 0 max 16384 2024-04-02T13:57:25.176 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:57:25.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.162+0000 7f31667a3640 1 -- 172.21.15.52:0/3824685331 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f31480a3d20 msgr2=0x7f31480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.162+0000 7f31667a3640 1 --2- 172.21.15.52:0/3824685331 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f31480a3d20 0x7f31480a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3154012960 tx=0x7f315400bb70 comp rx=0 tx=0).stop 2024-04-02T13:57:25.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.162+0000 7f31667a3640 1 -- 172.21.15.52:0/3824685331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31601069c0 msgr2=0x7f316019cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.162+0000 7f31667a3640 1 --2- 172.21.15.52:0/3824685331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31601069c0 0x7f316019cbf0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f31580348f0 tx=0x7f3158032ec0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.162+0000 7f31667a3640 1 -- 172.21.15.52:0/3824685331 shutdown_connections 2024-04-02T13:57:25.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.162+0000 7f31667a3640 1 --2- 172.21.15.52:0/3824685331 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f31480a3d20 0x7f31480a61e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.162+0000 7f31667a3640 1 --2- 172.21.15.52:0/3824685331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f31601069c0 0x7f316019cbf0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.162+0000 7f31667a3640 1 --2- 172.21.15.52:0/3824685331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3160101370 0x7f3160198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.162+0000 7f31667a3640 1 --2- 172.21.15.52:0/3824685331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31601009b0 0x7f3160198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.162+0000 7f31667a3640 1 -- 172.21.15.52:0/3824685331 >> 172.21.15.52:0/3824685331 conn(0x7f3160076d80 msgr2=0x7f3160104760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:25.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.162+0000 7f31667a3640 1 -- 172.21.15.52:0/3824685331 shutdown_connections 2024-04-02T13:57:25.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.162+0000 7f31667a3640 1 -- 172.21.15.52:0/3824685331 wait complete. 2024-04-02T13:57:25.180 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:57:25.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.283+0000 7f89e77ba640 1 -- 172.21.15.52:0/568554478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89e0102450 msgr2=0x7f89e010e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.283+0000 7f89e77ba640 1 --2- 172.21.15.52:0/568554478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89e0102450 0x7f89e010e960 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009cd0 tx=0x7f89d002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.283+0000 7f89e77ba640 1 -- 172.21.15.52:0/568554478 shutdown_connections 2024-04-02T13:57:25.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.283+0000 7f89e77ba640 1 --2- 172.21.15.52:0/568554478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89e0102450 0x7f89e010e960 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.283+0000 7f89e77ba640 1 --2- 172.21.15.52:0/568554478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89e0101a90 0x7f89e0101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.283+0000 7f89e77ba640 1 --2- 172.21.15.52:0/568554478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89e0107aa0 0x7f89e0107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.283+0000 7f89e77ba640 1 -- 172.21.15.52:0/568554478 >> 172.21.15.52:0/568554478 conn(0x7f89e0078f00 msgr2=0x7f89e0079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:25.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.283+0000 7f89e77ba640 1 -- 172.21.15.52:0/568554478 shutdown_connections 2024-04-02T13:57:25.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.284+0000 7f89e77ba640 1 -- 172.21.15.52:0/568554478 wait complete. 2024-04-02T13:57:25.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.284+0000 7f89e77ba640 1 Processor -- start 2024-04-02T13:57:25.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.284+0000 7f89e77ba640 1 -- start start 2024-04-02T13:57:25.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.284+0000 7f89e77ba640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89e0101a90 0x7f89e01983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.284+0000 7f89e77ba640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89e0102450 0x7f89e01988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.284+0000 7f89e77ba640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89e0107aa0 0x7f89e019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.284+0000 7f89e77ba640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f89e0111eb0 con 0x7f89e0102450 2024-04-02T13:57:25.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.284+0000 7f89e77ba640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f89e0111d30 con 0x7f89e0107aa0 2024-04-02T13:57:25.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.284+0000 7f89e77ba640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f89e0112030 con 0x7f89e0101a90 2024-04-02T13:57:25.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.284+0000 7f89e4d55640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89e0102450 0x7f89e01988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.284+0000 7f89e4d55640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89e0102450 0x7f89e01988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48152/0 (socket says 172.21.15.52:48152) 2024-04-02T13:57:25.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.284+0000 7f89e4d55640 1 -- 172.21.15.52:0/258660685 learned_addr learned my addr 172.21.15.52:0/258660685 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:25.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.284+0000 7f89e5556640 1 --2- 172.21.15.52:0/258660685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89e0101a90 0x7f89e01983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.285+0000 7f89e5d57640 1 --2- 172.21.15.52:0/258660685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89e0107aa0 0x7f89e019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.285+0000 7f89e4d55640 1 -- 172.21.15.52:0/258660685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89e0101a90 msgr2=0x7f89e01983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.285+0000 7f89e4d55640 1 --2- 172.21.15.52:0/258660685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89e0101a90 0x7f89e01983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.285+0000 7f89e4d55640 1 -- 172.21.15.52:0/258660685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89e0107aa0 msgr2=0x7f89e019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.285+0000 7f89e4d55640 1 --2- 172.21.15.52:0/258660685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89e0107aa0 0x7f89e019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.285+0000 7f89e4d55640 1 -- 172.21.15.52:0/258660685 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e019d350 con 0x7f89e0102450 2024-04-02T13:57:25.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.285+0000 7f89e4d55640 1 --2- 172.21.15.52:0/258660685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89e0102450 0x7f89e01988f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f89d400d950 tx=0x7f89d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:25.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.285+0000 7f89de7fc640 1 -- 172.21.15.52:0/258660685 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d4017070 con 0x7f89e0102450 2024-04-02T13:57:25.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.285+0000 7f89e77ba640 1 -- 172.21.15.52:0/258660685 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89e019d640 con 0x7f89e0102450 2024-04-02T13:57:25.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.285+0000 7f89de7fc640 1 -- 172.21.15.52:0/258660685 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89d4009c30 con 0x7f89e0102450 2024-04-02T13:57:25.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.285+0000 7f89de7fc640 1 -- 172.21.15.52:0/258660685 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d400f6b0 con 0x7f89e0102450 2024-04-02T13:57:25.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.285+0000 7f89e77ba640 1 -- 172.21.15.52:0/258660685 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89e01a4ed0 con 0x7f89e0102450 2024-04-02T13:57:25.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.288+0000 7f89ea3a7740 1 Processor -- start 2024-04-02T13:57:25.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.288+0000 7f89ea3a7740 1 -- start start 2024-04-02T13:57:25.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.288+0000 7f89de7fc640 1 -- 172.21.15.52:0/258660685 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f89d40109d0 con 0x7f89e0102450 2024-04-02T13:57:25.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.288+0000 7f89ea3a7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fa2fff450 0x556fa2fff830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.288+0000 7f89ea3a7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556fa2fffd70 0x556fa300cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.288+0000 7f89ea3a7740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa300d0b0 0x556fa300f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.288+0000 7f89ea3a7740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556fa2f637b0 con 0x556fa300d0b0 2024-04-02T13:57:25.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.288+0000 7f89ea3a7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556fa2f02570 con 0x556fa2fffd70 2024-04-02T13:57:25.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.288+0000 7f89ea3a7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556fa2f5f760 con 0x556fa2fff450 2024-04-02T13:57:25.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.288+0000 7f89e4d55640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa300d0b0 0x556fa300f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.289+0000 7f89e4d55640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa300d0b0 0x556fa300f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48158/0 (socket says 172.21.15.52:48158) 2024-04-02T13:57:25.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.289+0000 7f89e4d55640 1 -- 172.21.15.52:0/3827879903 learned_addr learned my addr 172.21.15.52:0/3827879903 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:25.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.289+0000 7f89e5d57640 1 --2- 172.21.15.52:0/3827879903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fa2fff450 0x556fa2fff830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.289+0000 7f89de7fc640 1 --2- 172.21.15.52:0/258660685 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f89c80a3d20 0x7f89c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.289+0000 7f89e5556640 1 --2- 172.21.15.52:0/3827879903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556fa2fffd70 0x556fa300cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.289+0000 7f89e5d57640 1 --2- 172.21.15.52:0/258660685 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f89c80a3d20 0x7f89c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.289+0000 7f89de7fc640 1 -- 172.21.15.52:0/258660685 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f89d40d0770 con 0x7f89e0102450 2024-04-02T13:57:25.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.289+0000 7f89e4d55640 1 -- 172.21.15.52:0/3827879903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fa2fff450 msgr2=0x556fa2fff830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.289+0000 7f89e4d55640 1 --2- 172.21.15.52:0/3827879903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fa2fff450 0x556fa2fff830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.289+0000 7f89e4d55640 1 -- 172.21.15.52:0/3827879903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556fa2fffd70 msgr2=0x556fa300cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.289+0000 7f89e4d55640 1 --2- 172.21.15.52:0/3827879903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556fa2fffd70 0x556fa300cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.289+0000 7f89e4d55640 1 -- 172.21.15.52:0/3827879903 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fa2c773f0 con 0x556fa300d0b0 2024-04-02T13:57:25.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.289+0000 7f89e4d55640 1 --2- 172.21.15.52:0/3827879903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa300d0b0 0x556fa300f4a0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f89d409c0d0 tx=0x7f89d4010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe23bf9b4165cc45 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:25.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.289+0000 7f89c77fe640 1 -- 172.21.15.52:0/3827879903 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d4020070 con 0x556fa300d0b0 2024-04-02T13:57:25.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.289+0000 7f89e5d57640 1 --2- 172.21.15.52:0/258660685 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f89c80a3d20 0x7f89c80a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f89d0006010 tx=0x7f89d002c0f0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:25.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.289+0000 7f89c77fe640 1 -- 172.21.15.52:0/3827879903 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89d4009de0 con 0x556fa300d0b0 2024-04-02T13:57:25.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.289+0000 7f89c77fe640 1 -- 172.21.15.52:0/3827879903 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d40cde00 con 0x556fa300d0b0 2024-04-02T13:57:25.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.290+0000 7f89ea3a7740 1 -- 172.21.15.52:0/3827879903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa300d0b0 msgr2=0x556fa300f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.290+0000 7f89ea3a7740 1 --2- 172.21.15.52:0/3827879903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa300d0b0 0x556fa300f4a0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f89d409c0d0 tx=0x7f89d4010d70 comp rx=0 tx=0).stop 2024-04-02T13:57:25.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.290+0000 7f89ea3a7740 1 -- 172.21.15.52:0/3827879903 shutdown_connections 2024-04-02T13:57:25.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.290+0000 7f89ea3a7740 1 --2- 172.21.15.52:0/3827879903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa300d0b0 0x556fa300f4a0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.290+0000 7f89ea3a7740 1 --2- 172.21.15.52:0/3827879903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556fa2fffd70 0x556fa300cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.290+0000 7f89ea3a7740 1 --2- 172.21.15.52:0/3827879903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fa2fff450 0x556fa2fff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.290+0000 7f89ea3a7740 1 -- 172.21.15.52:0/3827879903 >> 172.21.15.52:0/3827879903 conn(0x556fa2ff9ed0 msgr2=0x556fa2ffa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:25.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.290+0000 7f89ea3a7740 1 -- 172.21.15.52:0/3827879903 shutdown_connections 2024-04-02T13:57:25.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.290+0000 7f89ea3a7740 1 -- 172.21.15.52:0/3827879903 wait complete. 2024-04-02T13:57:25.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.290+0000 7f89ea3a7740 1 -- start start 2024-04-02T13:57:25.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.290+0000 7f89ea3a7740 1 Processor -- start 2024-04-02T13:57:25.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.291+0000 7f89ea3a7740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa2fff450 0x556fa300cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.291+0000 7f89ea3a7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556fa2fffd70 0x556fa3009e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.291+0000 7f89ea3a7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fa300d0b0 0x556fa300a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.291+0000 7f89ea3a7740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556fa300fe90 con 0x556fa2fff450 2024-04-02T13:57:25.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.291+0000 7f89ea3a7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556fa2f1ee80 con 0x556fa2fffd70 2024-04-02T13:57:25.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.291+0000 7f89ea3a7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556fa3010010 con 0x556fa300d0b0 2024-04-02T13:57:25.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.291+0000 7f89e5556640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa2fff450 0x556fa300cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.291+0000 7f89ea3a7740 10 client.0 _ll_register_callbacks cb 0x556fa2f5aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:25.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.291+0000 7f89e5556640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa2fff450 0x556fa300cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48172/0 (socket says 172.21.15.52:48172) 2024-04-02T13:57:25.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.291+0000 7f89e5556640 1 -- 172.21.15.52:0/827564958 learned_addr learned my addr 172.21.15.52:0/827564958 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:25.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.291+0000 7f89e4d55640 1 --2- 172.21.15.52:0/827564958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556fa2fffd70 0x556fa3009e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.291+0000 7f89e5d57640 1 --2- 172.21.15.52:0/827564958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fa300d0b0 0x556fa300a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.291+0000 7f89e5556640 1 -- 172.21.15.52:0/827564958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fa300d0b0 msgr2=0x556fa300a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.291+0000 7f89e5556640 1 --2- 172.21.15.52:0/827564958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fa300d0b0 0x556fa300a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.291+0000 7f89e5556640 1 -- 172.21.15.52:0/827564958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556fa2fffd70 msgr2=0x556fa3009e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.291+0000 7f89e5556640 1 --2- 172.21.15.52:0/827564958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556fa2fffd70 0x556fa3009e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.291+0000 7f89e5556640 1 -- 172.21.15.52:0/827564958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fa2d417d0 con 0x556fa2fff450 2024-04-02T13:57:25.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.292+0000 7f89e5556640 1 --2- 172.21.15.52:0/827564958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa2fff450 0x556fa300cad0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f89d8010820 tx=0x7f89d8010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6385416874ff1009 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:25.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.292+0000 7f89c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:25.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.292+0000 7f89c4ff9640 1 -- 172.21.15.52:0/827564958 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d801e060 con 0x556fa2fff450 2024-04-02T13:57:25.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.292+0000 7f89c4ff9640 1 -- 172.21.15.52:0/827564958 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f89d800fb90 con 0x556fa2fff450 2024-04-02T13:57:25.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.292+0000 7f89ea3a7740 1 -- 172.21.15.52:0/827564958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89d4009de0 con 0x556fa2fff450 2024-04-02T13:57:25.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.292+0000 7f89c4ff9640 1 -- 172.21.15.52:0/827564958 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f89d8025050 con 0x556fa2fff450 2024-04-02T13:57:25.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.292+0000 7f89ea3a7740 10 client.16782 fetch_fsmap learned FSMap version 29 2024-04-02T13:57:25.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.292+0000 7f89ea3a7740 1 -- 172.21.15.52:0/827564958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556fa301bff0 con 0x556fa2fff450 2024-04-02T13:57:25.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.292+0000 7f89c4ff9640 1 -- 172.21.15.52:0/827564958 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89d4009de0 con 0x556fa2fff450 2024-04-02T13:57:25.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.292+0000 7f89c4ff9640 1 -- 172.21.15.52:0/827564958 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f89d8018370 con 0x556fa2fff450 2024-04-02T13:57:25.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.292+0000 7f89ea3a7740 10 client.16782 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:57:25.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.292+0000 7f89ea3a7740 10 client.16782 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:57:25.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.292+0000 7f89ea3a7740 20 client.16782 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:25.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.292+0000 7f89ea3a7740 1 --2- 172.21.15.52:0/827564958 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x556fa301d4b0 0x556fa301f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.292+0000 7f89ea3a7740 4 client.16782 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:57:25.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.292+0000 7f89ea3a7740 1 -- 172.21.15.52:0/827564958 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fa2f70cf0 con 0x556fa301d4b0 2024-04-02T13:57:25.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.293+0000 7f89e4d55640 1 --2- 172.21.15.52:0/827564958 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x556fa301d4b0 0x556fa301f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.293+0000 7f89e4d55640 1 --2- 172.21.15.52:0/827564958 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x556fa301d4b0 0x556fa301f900 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=155a1a9847a81123 server_cookie=1808283f9ef47b28 in_seq=0 out_seq=0 2024-04-02T13:57:25.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.293+0000 7f89c4ff9640 10 client.16782 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:57:25.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.294+0000 7f89c4ff9640 1 -- 172.21.15.52:0/827564958 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556fa2f70cf0 con 0x556fa301d4b0 2024-04-02T13:57:25.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.294+0000 7f89c4ff9640 10 client.16782 handle_command_reply: tid=0 2024-04-02T13:57:25.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.294+0000 7f89ea3a7740 1 client.16782 shutdown 2024-04-02T13:57:25.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.294+0000 7f89ea3a7740 1 -- 172.21.15.52:0/827564958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa2fff450 msgr2=0x556fa300cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.294+0000 7f89ea3a7740 1 --2- 172.21.15.52:0/827564958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa2fff450 0x556fa300cad0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f89d8010820 tx=0x7f89d8010ce0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.294+0000 7f89ea3a7740 1 -- 172.21.15.52:0/827564958 shutdown_connections 2024-04-02T13:57:25.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.294+0000 7f89ea3a7740 1 --2- 172.21.15.52:0/827564958 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x556fa301d4b0 0x556fa301f900 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.294+0000 7f89ea3a7740 1 --2- 172.21.15.52:0/827564958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fa300d0b0 0x556fa300a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.294+0000 7f89ea3a7740 1 --2- 172.21.15.52:0/827564958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556fa2fffd70 0x556fa3009e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.294+0000 7f89ea3a7740 1 --2- 172.21.15.52:0/827564958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa2fff450 0x556fa300cad0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.294+0000 7f89c4ff9640 0 client.16782 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:57:25.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.294+0000 7f89ea3a7740 1 -- 172.21.15.52:0/827564958 >> 172.21.15.52:0/827564958 conn(0x556fa2ff9ed0 msgr2=0x556fa2ffbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:25.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.294+0000 7f89ea3a7740 1 -- 172.21.15.52:0/827564958 shutdown_connections 2024-04-02T13:57:25.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.294+0000 7f89ea3a7740 1 -- 172.21.15.52:0/827564958 wait complete. 2024-04-02T13:57:25.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.294+0000 7f89ea3a7740 20 client.16782 trim_cache size 0 max 16384 2024-04-02T13:57:25.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.308+0000 7f89ea3a7740 1 Processor -- start 2024-04-02T13:57:25.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.308+0000 7f89ea3a7740 1 -- start start 2024-04-02T13:57:25.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.308+0000 7f89ea3a7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556fa2fff450 0x556fa3018e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.308+0000 7f89ea3a7740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa2fffd70 0x556fa3008e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.308+0000 7f89ea3a7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fa300d0b0 0x556fa300c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.308+0000 7f89ea3a7740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f89e01acd40 con 0x556fa2fffd70 2024-04-02T13:57:25.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.308+0000 7f89ea3a7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f89e01ae8b0 con 0x556fa2fff450 2024-04-02T13:57:25.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.308+0000 7f89ea3a7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f89e01ad410 con 0x556fa300d0b0 2024-04-02T13:57:25.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.308+0000 7f89e4d55640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa2fffd70 0x556fa3008e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.308+0000 7f89e4d55640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa2fffd70 0x556fa3008e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48180/0 (socket says 172.21.15.52:48180) 2024-04-02T13:57:25.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.308+0000 7f89e4d55640 1 -- 172.21.15.52:0/2514393952 learned_addr learned my addr 172.21.15.52:0/2514393952 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:25.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.308+0000 7f89e5556640 1 --2- 172.21.15.52:0/2514393952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556fa2fff450 0x556fa3018e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.308+0000 7f89e5d57640 1 --2- 172.21.15.52:0/2514393952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fa300d0b0 0x556fa300c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.308+0000 7f89e4d55640 1 -- 172.21.15.52:0/2514393952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fa300d0b0 msgr2=0x556fa300c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.308+0000 7f89e4d55640 1 --2- 172.21.15.52:0/2514393952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fa300d0b0 0x556fa300c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.308+0000 7f89e4d55640 1 -- 172.21.15.52:0/2514393952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556fa2fff450 msgr2=0x556fa3018e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.308+0000 7f89e4d55640 1 --2- 172.21.15.52:0/2514393952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556fa2fff450 0x556fa3018e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.308+0000 7f89e4d55640 1 -- 172.21.15.52:0/2514393952 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fa300a7e0 con 0x556fa2fffd70 2024-04-02T13:57:25.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.308+0000 7f89e5556640 1 --2- 172.21.15.52:0/2514393952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556fa2fff450 0x556fa3018e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:25.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.308+0000 7f89e4d55640 1 --2- 172.21.15.52:0/2514393952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa2fffd70 0x556fa3008e40 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f89d4000c00 tx=0x7f89d400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94dec1110533b1c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:25.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.308+0000 7f89c6ffd640 1 -- 172.21.15.52:0/2514393952 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d4020040 con 0x556fa2fffd70 2024-04-02T13:57:25.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.308+0000 7f89c6ffd640 1 -- 172.21.15.52:0/2514393952 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89d40a0120 con 0x556fa2fffd70 2024-04-02T13:57:25.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.308+0000 7f89c6ffd640 1 -- 172.21.15.52:0/2514393952 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d40d03c0 con 0x556fa2fffd70 2024-04-02T13:57:25.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.309+0000 7f89ea3a7740 1 -- 172.21.15.52:0/2514393952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa2fffd70 msgr2=0x556fa3008e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.309+0000 7f89ea3a7740 1 --2- 172.21.15.52:0/2514393952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa2fffd70 0x556fa3008e40 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f89d4000c00 tx=0x7f89d400bf60 comp rx=0 tx=0).stop 2024-04-02T13:57:25.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.309+0000 7f89ea3a7740 1 -- 172.21.15.52:0/2514393952 shutdown_connections 2024-04-02T13:57:25.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.309+0000 7f89ea3a7740 1 --2- 172.21.15.52:0/2514393952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fa300d0b0 0x556fa300c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.309+0000 7f89ea3a7740 1 --2- 172.21.15.52:0/2514393952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa2fffd70 0x556fa3008e40 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.309+0000 7f89ea3a7740 1 --2- 172.21.15.52:0/2514393952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556fa2fff450 0x556fa3018e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.309+0000 7f89ea3a7740 1 -- 172.21.15.52:0/2514393952 >> 172.21.15.52:0/2514393952 conn(0x556fa2ff9ed0 msgr2=0x556fa3008730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:25.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.309+0000 7f89ea3a7740 1 -- 172.21.15.52:0/2514393952 shutdown_connections 2024-04-02T13:57:25.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.309+0000 7f89ea3a7740 1 -- 172.21.15.52:0/2514393952 wait complete. 2024-04-02T13:57:25.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.309+0000 7f89ea3a7740 1 -- start start 2024-04-02T13:57:25.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.309+0000 7f89ea3a7740 1 Processor -- start 2024-04-02T13:57:25.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.310+0000 7f89ea3a7740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa2fff450 0x556fa300d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.310+0000 7f89ea3a7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556fa2fffd70 0x556fa300dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.310+0000 7f89ea3a7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fa300d0b0 0x556fa300e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.310+0000 7f89ea3a7740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f89e01ae1b0 con 0x556fa2fff450 2024-04-02T13:57:25.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.310+0000 7f89ea3a7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f89e01adae0 con 0x556fa2fffd70 2024-04-02T13:57:25.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.310+0000 7f89ea3a7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f89e01aad20 con 0x556fa300d0b0 2024-04-02T13:57:25.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.310+0000 7f89ea3a7740 10 client.0 _ll_register_callbacks cb 0x7f89d800a720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:25.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.310+0000 7f89e5556640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa2fff450 0x556fa300d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.310+0000 7f89e5556640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa2fff450 0x556fa300d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48184/0 (socket says 172.21.15.52:48184) 2024-04-02T13:57:25.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.310+0000 7f89e5556640 1 -- 172.21.15.52:0/3538469797 learned_addr learned my addr 172.21.15.52:0/3538469797 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:25.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.310+0000 7f89e5d57640 1 --2- 172.21.15.52:0/3538469797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fa300d0b0 0x556fa300e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.310+0000 7f89e4d55640 1 --2- 172.21.15.52:0/3538469797 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556fa2fffd70 0x556fa300dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.310+0000 7f89e5556640 1 -- 172.21.15.52:0/3538469797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fa300d0b0 msgr2=0x556fa300e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.310+0000 7f89e5556640 1 --2- 172.21.15.52:0/3538469797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fa300d0b0 0x556fa300e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.310+0000 7f89e5556640 1 -- 172.21.15.52:0/3538469797 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556fa2fffd70 msgr2=0x556fa300dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.310+0000 7f89e5556640 1 --2- 172.21.15.52:0/3538469797 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556fa2fffd70 0x556fa300dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.310+0000 7f89e5556640 1 -- 172.21.15.52:0/3538469797 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fa3014580 con 0x556fa2fff450 2024-04-02T13:57:25.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.310+0000 7f89e5d57640 1 --2- 172.21.15.52:0/3538469797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fa300d0b0 0x556fa300e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:57:25.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.310+0000 7f89e5556640 1 --2- 172.21.15.52:0/3538469797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa2fff450 0x556fa300d8a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f89d80187c0 tx=0x7f89d8005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1cb43ab0896a44 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:25.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.310+0000 7f89c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:25.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.311+0000 7f89c4ff9640 1 -- 172.21.15.52:0/3538469797 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d802d070 con 0x556fa2fff450 2024-04-02T13:57:25.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.311+0000 7f89c4ff9640 1 -- 172.21.15.52:0/3538469797 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f89d800fb90 con 0x556fa2fff450 2024-04-02T13:57:25.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.311+0000 7f89ea3a7740 1 -- 172.21.15.52:0/3538469797 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89d40a0120 con 0x556fa2fff450 2024-04-02T13:57:25.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.311+0000 7f89c4ff9640 1 -- 172.21.15.52:0/3538469797 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f89d801e020 con 0x556fa2fff450 2024-04-02T13:57:25.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.311+0000 7f89ea3a7740 10 client.16788 fetch_fsmap learned FSMap version 29 2024-04-02T13:57:25.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.311+0000 7f89ea3a7740 1 -- 172.21.15.52:0/3538469797 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556fa3015880 con 0x556fa2fff450 2024-04-02T13:57:25.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.311+0000 7f89c4ff9640 1 -- 172.21.15.52:0/3538469797 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89d40a0120 con 0x556fa2fff450 2024-04-02T13:57:25.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.311+0000 7f89c4ff9640 1 -- 172.21.15.52:0/3538469797 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f89d8032850 con 0x556fa2fff450 2024-04-02T13:57:25.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.311+0000 7f89ea3a7740 10 client.16788 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:57:25.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.311+0000 7f89ea3a7740 10 client.16788 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:57:25.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.311+0000 7f89ea3a7740 20 client.16788 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:25.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.311+0000 7f89ea3a7740 1 --2- 172.21.15.52:0/3538469797 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x556fa3083c20 0x556fa3086040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.311+0000 7f89ea3a7740 4 client.16788 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:57:25.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.311+0000 7f89ea3a7740 1 -- 172.21.15.52:0/3538469797 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556fa2f6b040 con 0x556fa3083c20 2024-04-02T13:57:25.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.312+0000 7f89e4d55640 1 --2- 172.21.15.52:0/3538469797 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x556fa3083c20 0x556fa3086040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.312+0000 7f89c4ff9640 10 client.16788 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:57:25.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.312+0000 7f89e4d55640 1 --2- 172.21.15.52:0/3538469797 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x556fa3083c20 0x556fa3086040 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1459dfdd5f89650e server_cookie=b1ddb887df87b2f3 in_seq=0 out_seq=0 2024-04-02T13:57:25.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.312+0000 7f89c4ff9640 1 -- 172.21.15.52:0/3538469797 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556fa2f6b040 con 0x556fa3083c20 2024-04-02T13:57:25.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.312+0000 7f89c4ff9640 10 client.16788 handle_command_reply: tid=0 2024-04-02T13:57:25.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.312+0000 7f89ea3a7740 1 client.16788 shutdown 2024-04-02T13:57:25.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.312+0000 7f89ea3a7740 1 -- 172.21.15.52:0/3538469797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa2fff450 msgr2=0x556fa300d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.312+0000 7f89ea3a7740 1 --2- 172.21.15.52:0/3538469797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa2fff450 0x556fa300d8a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f89d80187c0 tx=0x7f89d8005670 comp rx=0 tx=0).stop 2024-04-02T13:57:25.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.312+0000 7f89ea3a7740 1 -- 172.21.15.52:0/3538469797 shutdown_connections 2024-04-02T13:57:25.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.312+0000 7f89ea3a7740 1 --2- 172.21.15.52:0/3538469797 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x556fa3083c20 0x556fa3086040 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.312+0000 7f89ea3a7740 1 --2- 172.21.15.52:0/3538469797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fa300d0b0 0x556fa300e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.312+0000 7f89ea3a7740 1 --2- 172.21.15.52:0/3538469797 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556fa2fffd70 0x556fa300dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.312+0000 7f89ea3a7740 1 --2- 172.21.15.52:0/3538469797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556fa2fff450 0x556fa300d8a0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.312+0000 7f89ea3a7740 1 -- 172.21.15.52:0/3538469797 >> 172.21.15.52:0/3538469797 conn(0x556fa2ff9ed0 msgr2=0x556fa3008fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:25.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.312+0000 7f89c4ff9640 0 client.16788 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:57:25.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.312+0000 7f89ea3a7740 1 -- 172.21.15.52:0/3538469797 shutdown_connections 2024-04-02T13:57:25.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.312+0000 7f89ea3a7740 1 -- 172.21.15.52:0/3538469797 wait complete. 2024-04-02T13:57:25.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.313+0000 7f89ea3a7740 20 client.16788 trim_cache size 0 max 16384 2024-04-02T13:57:25.331 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:57:25.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.313+0000 7f89e77ba640 1 -- 172.21.15.52:0/258660685 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f89c80a3d20 msgr2=0x7f89c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.313+0000 7f89e77ba640 1 --2- 172.21.15.52:0/258660685 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f89c80a3d20 0x7f89c80a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f89d0006010 tx=0x7f89d002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.313+0000 7f89e77ba640 1 -- 172.21.15.52:0/258660685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89e0102450 msgr2=0x7f89e01988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.313+0000 7f89e77ba640 1 --2- 172.21.15.52:0/258660685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89e0102450 0x7f89e01988f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f89d400d950 tx=0x7f89d400de10 comp rx=0 tx=0).stop 2024-04-02T13:57:25.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.313+0000 7f89e77ba640 1 -- 172.21.15.52:0/258660685 shutdown_connections 2024-04-02T13:57:25.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.313+0000 7f89e77ba640 1 --2- 172.21.15.52:0/258660685 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f89c80a3d20 0x7f89c80a61e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.313+0000 7f89e77ba640 1 --2- 172.21.15.52:0/258660685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89e0107aa0 0x7f89e019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.313+0000 7f89e77ba640 1 --2- 172.21.15.52:0/258660685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89e0102450 0x7f89e01988f0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.313+0000 7f89e77ba640 1 --2- 172.21.15.52:0/258660685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89e0101a90 0x7f89e01983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.313+0000 7f89e77ba640 1 -- 172.21.15.52:0/258660685 >> 172.21.15.52:0/258660685 conn(0x7f89e0078f00 msgr2=0x7f89e0104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:25.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.313+0000 7f89e77ba640 1 -- 172.21.15.52:0/258660685 shutdown_connections 2024-04-02T13:57:25.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.313+0000 7f89e77ba640 1 -- 172.21.15.52:0/258660685 wait complete. 2024-04-02T13:57:25.334 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:57:25.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.436+0000 7f7fd37a8640 1 -- 172.21.15.52:0/1622708533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fcc106980 msgr2=0x7f7fcc106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.436+0000 7f7fd37a8640 1 --2- 172.21.15.52:0/1622708533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fcc106980 0x7f7fcc106d60 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009930 tx=0x7f7fc402bc30 comp rx=0 tx=0).stop 2024-04-02T13:57:25.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.436+0000 7f7fd37a8640 1 -- 172.21.15.52:0/1622708533 shutdown_connections 2024-04-02T13:57:25.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.436+0000 7f7fd37a8640 1 --2- 172.21.15.52:0/1622708533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fcc101330 0x7f7fcc10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.436+0000 7f7fd37a8640 1 --2- 172.21.15.52:0/1622708533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fcc100970 0x7f7fcc100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.436+0000 7f7fd37a8640 1 --2- 172.21.15.52:0/1622708533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fcc106980 0x7f7fcc106d60 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.436+0000 7f7fd37a8640 1 -- 172.21.15.52:0/1622708533 >> 172.21.15.52:0/1622708533 conn(0x7f7fcc077b00 msgr2=0x7f7fcc077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:25.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.436+0000 7f7fd37a8640 1 -- 172.21.15.52:0/1622708533 shutdown_connections 2024-04-02T13:57:25.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.436+0000 7f7fd37a8640 1 -- 172.21.15.52:0/1622708533 wait complete. 2024-04-02T13:57:25.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.436+0000 7f7fd37a8640 1 Processor -- start 2024-04-02T13:57:25.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.436+0000 7f7fd37a8640 1 -- start start 2024-04-02T13:57:25.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.436+0000 7f7fd37a8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fcc100970 0x7f7fcc198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.436+0000 7f7fd37a8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fcc101330 0x7f7fcc1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.437+0000 7f7fd37a8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fcc106980 0x7f7fcc19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.437+0000 7f7fd37a8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc110d90 con 0x7f7fcc106980 2024-04-02T13:57:25.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.437+0000 7f7fd37a8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc110c10 con 0x7f7fcc100970 2024-04-02T13:57:25.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.437+0000 7f7fd37a8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc110f10 con 0x7f7fcc101330 2024-04-02T13:57:25.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.437+0000 7f7fd1d45640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fcc106980 0x7f7fcc19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.437+0000 7f7fd1d45640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fcc106980 0x7f7fcc19cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48190/0 (socket says 172.21.15.52:48190) 2024-04-02T13:57:25.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.437+0000 7f7fd1d45640 1 -- 172.21.15.52:0/1324691294 learned_addr learned my addr 172.21.15.52:0/1324691294 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:25.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.437+0000 7f7fd0d43640 1 --2- 172.21.15.52:0/1324691294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fcc101330 0x7f7fcc1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.437+0000 7f7fd1544640 1 --2- 172.21.15.52:0/1324691294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fcc100970 0x7f7fcc198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.437+0000 7f7fd1d45640 1 -- 172.21.15.52:0/1324691294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fcc101330 msgr2=0x7f7fcc1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.437+0000 7f7fd1d45640 1 --2- 172.21.15.52:0/1324691294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fcc101330 0x7f7fcc1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.437+0000 7f7fd1d45640 1 -- 172.21.15.52:0/1324691294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fcc100970 msgr2=0x7f7fcc198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.437+0000 7f7fd1d45640 1 --2- 172.21.15.52:0/1324691294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fcc100970 0x7f7fcc198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.437+0000 7f7fd1d45640 1 -- 172.21.15.52:0/1324691294 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fcc19d380 con 0x7f7fcc106980 2024-04-02T13:57:25.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.437+0000 7f7fd1d45640 1 --2- 172.21.15.52:0/1324691294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fcc106980 0x7f7fcc19cc50 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800eaa0 tx=0x7f7fc800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:25.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.437+0000 7f7fc27fc640 1 -- 172.21.15.52:0/1324691294 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc800cdb0 con 0x7f7fcc106980 2024-04-02T13:57:25.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.437+0000 7f7fd37a8640 1 -- 172.21.15.52:0/1324691294 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fcc19d670 con 0x7f7fcc106980 2024-04-02T13:57:25.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.437+0000 7f7fc27fc640 1 -- 172.21.15.52:0/1324691294 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc8005020 con 0x7f7fcc106980 2024-04-02T13:57:25.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.437+0000 7f7fc27fc640 1 -- 172.21.15.52:0/1324691294 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc80116b0 con 0x7f7fcc106980 2024-04-02T13:57:25.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.437+0000 7f7fd37a8640 1 -- 172.21.15.52:0/1324691294 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fcc1a4f00 con 0x7f7fcc106980 2024-04-02T13:57:25.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.439+0000 7f7fc27fc640 1 -- 172.21.15.52:0/1324691294 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7fc8005560 con 0x7f7fcc106980 2024-04-02T13:57:25.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.440+0000 7f7fd6394740 1 Processor -- start 2024-04-02T13:57:25.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.440+0000 7f7fd6394740 1 -- start start 2024-04-02T13:57:25.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.440+0000 7f7fc27fc640 1 --2- 172.21.15.52:0/1324691294 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f7fb80a3d20 0x7f7fb80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.440+0000 7f7fd6394740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558f99767a0 0x5558f9976b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.440+0000 7f7fd6394740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558f99770c0 0x5558f997f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.440+0000 7f7fd6394740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f997fca0 0x5558f9982090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.440+0000 7f7fd6394740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5558f98da7b0 con 0x5558f997fca0 2024-04-02T13:57:25.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.440+0000 7f7fd6394740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5558f9879570 con 0x5558f99770c0 2024-04-02T13:57:25.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.440+0000 7f7fd6394740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5558f98d6760 con 0x5558f99767a0 2024-04-02T13:57:25.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.440+0000 7f7fd1d45640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f997fca0 0x5558f9982090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.440+0000 7f7fd1d45640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f997fca0 0x5558f9982090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48202/0 (socket says 172.21.15.52:48202) 2024-04-02T13:57:25.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.440+0000 7f7fd0d43640 1 --2- 172.21.15.52:0/1324691294 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f7fb80a3d20 0x7f7fb80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.440+0000 7f7fd1d45640 1 -- 172.21.15.52:0/3277637530 learned_addr learned my addr 172.21.15.52:0/3277637530 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:25.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.441+0000 7f7fc27fc640 1 -- 172.21.15.52:0/1324691294 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f7fc80d0d40 con 0x7f7fcc106980 2024-04-02T13:57:25.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.441+0000 7f7fd1544640 1 --2- 172.21.15.52:0/3277637530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558f99767a0 0x5558f9976b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.441+0000 7f7fd1d45640 1 -- 172.21.15.52:0/3277637530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558f99767a0 msgr2=0x5558f9976b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.441+0000 7f7fd1d45640 1 --2- 172.21.15.52:0/3277637530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558f99767a0 0x5558f9976b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.441+0000 7f7fd1d45640 1 -- 172.21.15.52:0/3277637530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558f99770c0 msgr2=0x5558f997f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.441+0000 7f7fd1d45640 1 --2- 172.21.15.52:0/3277637530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558f99770c0 0x5558f997f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.441+0000 7f7fd1d45640 1 -- 172.21.15.52:0/3277637530 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558f95ee3f0 con 0x5558f997fca0 2024-04-02T13:57:25.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.441+0000 7f7fd1544640 1 --2- 172.21.15.52:0/3277637530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558f99767a0 0x5558f9976b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:25.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.441+0000 7f7fd1d45640 1 --2- 172.21.15.52:0/3277637530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f997fca0 0x5558f9982090 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8098680 tx=0x7f7fc80054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369f4a49cd1012b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:25.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.441+0000 7f7fd0d43640 1 --2- 172.21.15.52:0/1324691294 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f7fb80a3d20 0x7f7fb80a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009710 tx=0x7f7fbc009840 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:25.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.441+0000 7f7fb77fe640 1 -- 172.21.15.52:0/3277637530 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc801f070 con 0x5558f997fca0 2024-04-02T13:57:25.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.441+0000 7f7fb77fe640 1 -- 172.21.15.52:0/3277637530 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc809d0a0 con 0x5558f997fca0 2024-04-02T13:57:25.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.441+0000 7f7fb77fe640 1 -- 172.21.15.52:0/3277637530 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc80b10b0 con 0x5558f997fca0 2024-04-02T13:57:25.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.442+0000 7f7fd6394740 1 -- 172.21.15.52:0/3277637530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f997fca0 msgr2=0x5558f9982090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.442+0000 7f7fd6394740 1 --2- 172.21.15.52:0/3277637530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f997fca0 0x5558f9982090 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8098680 tx=0x7f7fc80054d0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.442+0000 7f7fd6394740 1 -- 172.21.15.52:0/3277637530 shutdown_connections 2024-04-02T13:57:25.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.442+0000 7f7fd6394740 1 --2- 172.21.15.52:0/3277637530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f997fca0 0x5558f9982090 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.442+0000 7f7fd6394740 1 --2- 172.21.15.52:0/3277637530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558f99770c0 0x5558f997f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.442+0000 7f7fd6394740 1 --2- 172.21.15.52:0/3277637530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558f99767a0 0x5558f9976b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.442+0000 7f7fd6394740 1 -- 172.21.15.52:0/3277637530 >> 172.21.15.52:0/3277637530 conn(0x5558f9970ed0 msgr2=0x5558f99712b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:25.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.442+0000 7f7fd6394740 1 -- 172.21.15.52:0/3277637530 shutdown_connections 2024-04-02T13:57:25.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.442+0000 7f7fd6394740 1 -- 172.21.15.52:0/3277637530 wait complete. 2024-04-02T13:57:25.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.442+0000 7f7fd6394740 1 -- start start 2024-04-02T13:57:25.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.442+0000 7f7fd6394740 1 Processor -- start 2024-04-02T13:57:25.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.443+0000 7f7fd6394740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f99767a0 0x5558f997f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.443+0000 7f7fd6394740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558f99770c0 0x5558f997cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.443+0000 7f7fd6394740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558f997fca0 0x5558f997d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.443+0000 7f7fd6394740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5558f99829d0 con 0x5558f99767a0 2024-04-02T13:57:25.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.443+0000 7f7fd6394740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5558f9895e80 con 0x5558f997fca0 2024-04-02T13:57:25.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.443+0000 7f7fd6394740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5558f9982b50 con 0x5558f99770c0 2024-04-02T13:57:25.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.443+0000 7f7fd6394740 10 client.0 _ll_register_callbacks cb 0x5558f98d1aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:25.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.443+0000 7f7fd1544640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f99767a0 0x5558f997f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.443+0000 7f7fd1544640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f99767a0 0x5558f997f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48204/0 (socket says 172.21.15.52:48204) 2024-04-02T13:57:25.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.443+0000 7f7fd1544640 1 -- 172.21.15.52:0/3971428070 learned_addr learned my addr 172.21.15.52:0/3971428070 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:25.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.443+0000 7f7fd0d43640 1 --2- 172.21.15.52:0/3971428070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558f99770c0 0x5558f997cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.443+0000 7f7fd1d45640 1 --2- 172.21.15.52:0/3971428070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558f997fca0 0x5558f997d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.443+0000 7f7fd1544640 1 -- 172.21.15.52:0/3971428070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558f99770c0 msgr2=0x5558f997cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.443+0000 7f7fd1544640 1 --2- 172.21.15.52:0/3971428070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558f99770c0 0x5558f997cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.443+0000 7f7fd1544640 1 -- 172.21.15.52:0/3971428070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558f997fca0 msgr2=0x5558f997d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.443+0000 7f7fd1544640 1 --2- 172.21.15.52:0/3971428070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558f997fca0 0x5558f997d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.443+0000 7f7fd1544640 1 -- 172.21.15.52:0/3971428070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558f96b87d0 con 0x5558f99767a0 2024-04-02T13:57:25.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.443+0000 7f7fd1544640 1 --2- 172.21.15.52:0/3971428070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f99767a0 0x5558f997f720 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009900 tx=0x7f7fc40095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b95eaa3d069345c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:25.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.443+0000 7f7fb4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:25.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.443+0000 7f7fb4ff9640 1 -- 172.21.15.52:0/3971428070 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4005030 con 0x5558f99767a0 2024-04-02T13:57:25.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.443+0000 7f7fb4ff9640 1 -- 172.21.15.52:0/3971428070 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f7fc40051d0 con 0x5558f99767a0 2024-04-02T13:57:25.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.443+0000 7f7fd6394740 1 -- 172.21.15.52:0/3971428070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fc809d0a0 con 0x5558f99767a0 2024-04-02T13:57:25.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.444+0000 7f7fb4ff9640 1 -- 172.21.15.52:0/3971428070 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7fc404d050 con 0x5558f99767a0 2024-04-02T13:57:25.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.444+0000 7f7fd6394740 10 client.16800 fetch_fsmap learned FSMap version 29 2024-04-02T13:57:25.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.444+0000 7f7fd6394740 1 -- 172.21.15.52:0/3971428070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5558f998ea60 con 0x5558f99767a0 2024-04-02T13:57:25.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.444+0000 7f7fb4ff9640 1 -- 172.21.15.52:0/3971428070 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc809d0a0 con 0x5558f99767a0 2024-04-02T13:57:25.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.444+0000 7f7fb4ff9640 1 -- 172.21.15.52:0/3971428070 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f7fc40035f0 con 0x5558f99767a0 2024-04-02T13:57:25.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.444+0000 7f7fd6394740 10 client.16800 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:57:25.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.444+0000 7f7fd6394740 10 client.16800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:57:25.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.444+0000 7f7fd6394740 20 client.16800 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:25.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.444+0000 7f7fd6394740 1 --2- 172.21.15.52:0/3971428070 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5558f998ffc0 0x5558f9992400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.444+0000 7f7fd6394740 4 client.16800 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:57:25.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.444+0000 7f7fd6394740 1 -- 172.21.15.52:0/3971428070 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558f98e7cf0 con 0x5558f998ffc0 2024-04-02T13:57:25.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.444+0000 7f7fd0d43640 1 --2- 172.21.15.52:0/3971428070 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5558f998ffc0 0x5558f9992400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.445+0000 7f7fb4ff9640 10 client.16800 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:25.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.445+0000 7f7fd0d43640 1 --2- 172.21.15.52:0/3971428070 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5558f998ffc0 0x5558f9992400 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a3ae582d87b5e6 server_cookie=a99e1608ecfbb4c7 in_seq=0 out_seq=0 2024-04-02T13:57:25.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.445+0000 7f7fb4ff9640 1 -- 172.21.15.52:0/3971428070 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5558f98e7cf0 con 0x5558f998ffc0 2024-04-02T13:57:25.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.445+0000 7f7fb4ff9640 10 client.16800 handle_command_reply: tid=0 2024-04-02T13:57:25.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.445+0000 7f7fd6394740 1 client.16800 shutdown 2024-04-02T13:57:25.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.445+0000 7f7fd6394740 1 -- 172.21.15.52:0/3971428070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f99767a0 msgr2=0x5558f997f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.445+0000 7f7fd6394740 1 --2- 172.21.15.52:0/3971428070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f99767a0 0x5558f997f720 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009900 tx=0x7f7fc40095e0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.445+0000 7f7fd6394740 1 -- 172.21.15.52:0/3971428070 shutdown_connections 2024-04-02T13:57:25.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.445+0000 7f7fd6394740 1 --2- 172.21.15.52:0/3971428070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558f997fca0 0x5558f997d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.445+0000 7f7fd6394740 1 --2- 172.21.15.52:0/3971428070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558f99770c0 0x5558f997cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.445+0000 7f7fd6394740 1 --2- 172.21.15.52:0/3971428070 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5558f998ffc0 0x5558f9992400 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.445+0000 7f7fd6394740 1 --2- 172.21.15.52:0/3971428070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f99767a0 0x5558f997f720 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.445+0000 7f7fb4ff9640 0 client.16800 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:25.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.445+0000 7f7fd6394740 1 -- 172.21.15.52:0/3971428070 >> 172.21.15.52:0/3971428070 conn(0x5558f9970ed0 msgr2=0x5558f9972c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:25.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.446+0000 7f7fd6394740 1 -- 172.21.15.52:0/3971428070 shutdown_connections 2024-04-02T13:57:25.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.446+0000 7f7fd6394740 1 -- 172.21.15.52:0/3971428070 wait complete. 2024-04-02T13:57:25.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.446+0000 7f7fd6394740 20 client.16800 trim_cache size 0 max 16384 2024-04-02T13:57:25.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.459+0000 7f7fd6394740 1 Processor -- start 2024-04-02T13:57:25.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.459+0000 7f7fd6394740 1 -- start start 2024-04-02T13:57:25.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.459+0000 7f7fd6394740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558f99767a0 0x5558f998b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.460+0000 7f7fd6394740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f99770c0 0x5558f997bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.460+0000 7f7fd6394740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558f997fca0 0x5558f9997cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.460+0000 7f7fd6394740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc1aee90 con 0x5558f99770c0 2024-04-02T13:57:25.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.460+0000 7f7fd6394740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc1b0a00 con 0x5558f997fca0 2024-04-02T13:57:25.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.460+0000 7f7fd6394740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc1af560 con 0x5558f99767a0 2024-04-02T13:57:25.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.460+0000 7f7fd0d43640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f99770c0 0x5558f997bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.460+0000 7f7fd0d43640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f99770c0 0x5558f997bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48218/0 (socket says 172.21.15.52:48218) 2024-04-02T13:57:25.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.460+0000 7f7fd0d43640 1 -- 172.21.15.52:0/1923153929 learned_addr learned my addr 172.21.15.52:0/1923153929 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:25.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.460+0000 7f7fd1544640 1 --2- 172.21.15.52:0/1923153929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558f99767a0 0x5558f998b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.460+0000 7f7fd0d43640 1 -- 172.21.15.52:0/1923153929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558f99767a0 msgr2=0x5558f998b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.460+0000 7f7fd0d43640 1 --2- 172.21.15.52:0/1923153929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558f99767a0 0x5558f998b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.460+0000 7f7fd0d43640 1 -- 172.21.15.52:0/1923153929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558f997fca0 msgr2=0x5558f9997cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.460+0000 7f7fd0d43640 1 --2- 172.21.15.52:0/1923153929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558f997fca0 0x5558f9997cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.460+0000 7f7fd0d43640 1 -- 172.21.15.52:0/1923153929 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558f997d560 con 0x5558f99770c0 2024-04-02T13:57:25.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.460+0000 7f7fd0d43640 1 --2- 172.21.15.52:0/1923153929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f99770c0 0x5558f997bb00 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc012040 tx=0x7f7fbc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17debb4be03a621b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:25.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.460+0000 7f7fb6ffd640 1 -- 172.21.15.52:0/1923153929 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc009d40 con 0x5558f99770c0 2024-04-02T13:57:25.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.460+0000 7f7fb6ffd640 1 -- 172.21.15.52:0/1923153929 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fbc02c070 con 0x5558f99770c0 2024-04-02T13:57:25.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.460+0000 7f7fb6ffd640 1 -- 172.21.15.52:0/1923153929 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc0273a0 con 0x5558f99770c0 2024-04-02T13:57:25.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.461+0000 7f7fd6394740 1 -- 172.21.15.52:0/1923153929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f99770c0 msgr2=0x5558f997bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.461+0000 7f7fd6394740 1 --2- 172.21.15.52:0/1923153929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f99770c0 0x5558f997bb00 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc012040 tx=0x7f7fbc016000 comp rx=0 tx=0).stop 2024-04-02T13:57:25.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.461+0000 7f7fd6394740 1 -- 172.21.15.52:0/1923153929 shutdown_connections 2024-04-02T13:57:25.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.461+0000 7f7fd6394740 1 --2- 172.21.15.52:0/1923153929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558f997fca0 0x5558f9997cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.461+0000 7f7fd6394740 1 --2- 172.21.15.52:0/1923153929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f99770c0 0x5558f997bb00 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.461+0000 7f7fd6394740 1 --2- 172.21.15.52:0/1923153929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558f99767a0 0x5558f998b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.461+0000 7f7fd6394740 1 -- 172.21.15.52:0/1923153929 >> 172.21.15.52:0/1923153929 conn(0x5558f9970ed0 msgr2=0x5558f99712b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:25.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.461+0000 7f7fd6394740 1 -- 172.21.15.52:0/1923153929 shutdown_connections 2024-04-02T13:57:25.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.461+0000 7f7fd6394740 1 -- 172.21.15.52:0/1923153929 wait complete. 2024-04-02T13:57:25.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.461+0000 7f7fd6394740 1 -- start start 2024-04-02T13:57:25.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.461+0000 7f7fd6394740 1 Processor -- start 2024-04-02T13:57:25.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.462+0000 7f7fd6394740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558f99767a0 0x5558f9988200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.462+0000 7f7fd6394740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558f99770c0 0x5558f99794c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.462+0000 7f7fd6394740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f997fca0 0x5558f9979a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.462+0000 7f7fd6394740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc1b0300 con 0x5558f997fca0 2024-04-02T13:57:25.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.462+0000 7f7fd6394740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc1afc30 con 0x5558f99767a0 2024-04-02T13:57:25.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.462+0000 7f7fd6394740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc1ace70 con 0x5558f99770c0 2024-04-02T13:57:25.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.462+0000 7f7fd6394740 10 client.0 _ll_register_callbacks cb 0x7f7fbc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:25.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.462+0000 7f7fd1d45640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f997fca0 0x5558f9979a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.462+0000 7f7fd1d45640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f997fca0 0x5558f9979a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48220/0 (socket says 172.21.15.52:48220) 2024-04-02T13:57:25.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.462+0000 7f7fd1d45640 1 -- 172.21.15.52:0/2855766425 learned_addr learned my addr 172.21.15.52:0/2855766425 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:25.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.462+0000 7f7fd0d43640 1 --2- 172.21.15.52:0/2855766425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558f99770c0 0x5558f99794c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.462+0000 7f7fd1d45640 1 -- 172.21.15.52:0/2855766425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558f99770c0 msgr2=0x5558f99794c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.462+0000 7f7fd1d45640 1 --2- 172.21.15.52:0/2855766425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558f99770c0 0x5558f99794c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.462+0000 7f7fd1d45640 1 -- 172.21.15.52:0/2855766425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558f99767a0 msgr2=0x5558f9988200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:25.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.462+0000 7f7fd1544640 1 --2- 172.21.15.52:0/2855766425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558f99767a0 0x5558f9988200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.462+0000 7f7fd1d45640 1 --2- 172.21.15.52:0/2855766425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558f99767a0 0x5558f9988200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.462+0000 7f7fd1d45640 1 -- 172.21.15.52:0/2855766425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558f997a270 con 0x5558f997fca0 2024-04-02T13:57:25.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.462+0000 7f7fd1d45640 1 --2- 172.21.15.52:0/2855766425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f997fca0 0x5558f9979a40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f7fc80ce200 tx=0x7f7fc80d0f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a0031a8bd926ac server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:25.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.462+0000 7f7fb4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:25.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.462+0000 7f7fb4ff9640 1 -- 172.21.15.52:0/2855766425 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc801f040 con 0x5558f997fca0 2024-04-02T13:57:25.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.462+0000 7f7fb4ff9640 1 -- 172.21.15.52:0/2855766425 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f7fc80051d0 con 0x5558f997fca0 2024-04-02T13:57:25.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.462+0000 7f7fd6394740 1 -- 172.21.15.52:0/2855766425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fbc02c070 con 0x5558f997fca0 2024-04-02T13:57:25.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.462+0000 7f7fb4ff9640 1 -- 172.21.15.52:0/2855766425 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7fc80afa60 con 0x5558f997fca0 2024-04-02T13:57:25.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.462+0000 7f7fd6394740 10 client.16806 fetch_fsmap learned FSMap version 29 2024-04-02T13:57:25.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.462+0000 7f7fd6394740 1 -- 172.21.15.52:0/2855766425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5558f99f65e0 con 0x5558f997fca0 2024-04-02T13:57:25.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.463+0000 7f7fb4ff9640 1 -- 172.21.15.52:0/2855766425 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fbc02c070 con 0x5558f997fca0 2024-04-02T13:57:25.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.463+0000 7f7fb4ff9640 1 -- 172.21.15.52:0/2855766425 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f7fc800f040 con 0x5558f997fca0 2024-04-02T13:57:25.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.463+0000 7f7fd6394740 10 client.16806 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:57:25.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.463+0000 7f7fd6394740 10 client.16806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:57:25.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.463+0000 7f7fd6394740 20 client.16806 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:25.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.463+0000 7f7fd6394740 1 --2- 172.21.15.52:0/2855766425 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5558f99f69e0 0x5558f99f8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:25.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.463+0000 7f7fd6394740 4 client.16806 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:57:25.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.463+0000 7f7fd6394740 1 -- 172.21.15.52:0/2855766425 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5558f98e2040 con 0x5558f99f69e0 2024-04-02T13:57:25.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.463+0000 7f7fd1544640 1 --2- 172.21.15.52:0/2855766425 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5558f99f69e0 0x5558f99f8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:25.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.463+0000 7f7fd1544640 1 --2- 172.21.15.52:0/2855766425 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5558f99f69e0 0x5558f99f8dd0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79b0404cdddbc253 server_cookie=6146629c22d0011e in_seq=0 out_seq=0 2024-04-02T13:57:25.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.463+0000 7f7fb4ff9640 10 client.16806 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:25.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.464+0000 7f7fb4ff9640 1 -- 172.21.15.52:0/2855766425 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5558f98e2040 con 0x5558f99f69e0 2024-04-02T13:57:25.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.464+0000 7f7fb4ff9640 10 client.16806 handle_command_reply: tid=0 2024-04-02T13:57:25.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.464+0000 7f7fd6394740 1 client.16806 shutdown 2024-04-02T13:57:25.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.464+0000 7f7fd6394740 1 -- 172.21.15.52:0/2855766425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f997fca0 msgr2=0x5558f9979a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.464+0000 7f7fd6394740 1 --2- 172.21.15.52:0/2855766425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f997fca0 0x5558f9979a40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f7fc80ce200 tx=0x7f7fc80d0f40 comp rx=0 tx=0).stop 2024-04-02T13:57:25.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.464+0000 7f7fd6394740 1 -- 172.21.15.52:0/2855766425 shutdown_connections 2024-04-02T13:57:25.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.464+0000 7f7fd6394740 1 --2- 172.21.15.52:0/2855766425 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5558f99f69e0 0x5558f99f8dd0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.464+0000 7f7fd6394740 1 --2- 172.21.15.52:0/2855766425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5558f997fca0 0x5558f9979a40 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.464+0000 7f7fd6394740 1 --2- 172.21.15.52:0/2855766425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558f99770c0 0x5558f99794c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.464+0000 7f7fb4ff9640 0 client.16806 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:25.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.464+0000 7f7fd6394740 1 --2- 172.21.15.52:0/2855766425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558f99767a0 0x5558f9988200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.464+0000 7f7fd6394740 1 -- 172.21.15.52:0/2855766425 >> 172.21.15.52:0/2855766425 conn(0x5558f9970ed0 msgr2=0x5558f996ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:25.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.464+0000 7f7fd6394740 1 -- 172.21.15.52:0/2855766425 shutdown_connections 2024-04-02T13:57:25.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.464+0000 7f7fd6394740 1 -- 172.21.15.52:0/2855766425 wait complete. 2024-04-02T13:57:25.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.464+0000 7f7fd6394740 20 client.16806 trim_cache size 0 max 16384 2024-04-02T13:57:25.484 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:57:25.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.464+0000 7f7fd37a8640 1 -- 172.21.15.52:0/1324691294 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f7fb80a3d20 msgr2=0x7f7fb80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.464+0000 7f7fd37a8640 1 --2- 172.21.15.52:0/1324691294 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f7fb80a3d20 0x7f7fb80a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009710 tx=0x7f7fbc009840 comp rx=0 tx=0).stop 2024-04-02T13:57:25.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.464+0000 7f7fd37a8640 1 -- 172.21.15.52:0/1324691294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fcc106980 msgr2=0x7f7fcc19cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:25.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.464+0000 7f7fd37a8640 1 --2- 172.21.15.52:0/1324691294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fcc106980 0x7f7fcc19cc50 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800eaa0 tx=0x7f7fc800ef60 comp rx=0 tx=0).stop 2024-04-02T13:57:25.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.464+0000 7f7fd37a8640 1 -- 172.21.15.52:0/1324691294 shutdown_connections 2024-04-02T13:57:25.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.464+0000 7f7fd37a8640 1 --2- 172.21.15.52:0/1324691294 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f7fb80a3d20 0x7f7fb80a61e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.464+0000 7f7fd37a8640 1 --2- 172.21.15.52:0/1324691294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fcc106980 0x7f7fcc19cc50 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.464+0000 7f7fd37a8640 1 --2- 172.21.15.52:0/1324691294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fcc101330 0x7f7fcc1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.464+0000 7f7fd37a8640 1 --2- 172.21.15.52:0/1324691294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fcc100970 0x7f7fcc198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:25.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.464+0000 7f7fd37a8640 1 -- 172.21.15.52:0/1324691294 >> 172.21.15.52:0/1324691294 conn(0x7f7fcc077b00 msgr2=0x7f7fcc0ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:25.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.465+0000 7f7fd37a8640 1 -- 172.21.15.52:0/1324691294 shutdown_connections 2024-04-02T13:57:25.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:25.465+0000 7f7fd37a8640 1 -- 172.21.15.52:0/1324691294 wait complete. 2024-04-02T13:57:25.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:25 smithi088 ceph-mon[36528]: pgmap v120: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-04-02T13:57:25.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:25 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1807898247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:57:26.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:25 smithi189 ceph-mon[36683]: pgmap v120: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-04-02T13:57:26.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:25 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1807898247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:57:26.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:25 smithi052 ceph-mon[32104]: pgmap v120: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-04-02T13:57:26.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:25 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1807898247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:57:26.488 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:57:26.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:57:26.489 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:57:26.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.605+0000 7f5ff872b640 1 -- 172.21.15.52:0/588618964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ff0068d00 msgr2=0x7f5ff0069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:26.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.605+0000 7f5ff872b640 1 --2- 172.21.15.52:0/588618964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ff0068d00 0x7f5ff0069180 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009930 tx=0x7f5fe402bc30 comp rx=0 tx=0).stop 2024-04-02T13:57:26.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.606+0000 7f5ff872b640 1 -- 172.21.15.52:0/588618964 shutdown_connections 2024-04-02T13:57:26.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.606+0000 7f5ff872b640 1 --2- 172.21.15.52:0/588618964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ff00696c0 0x7f5ff010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.606+0000 7f5ff872b640 1 --2- 172.21.15.52:0/588618964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ff0068d00 0x7f5ff0069180 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.606+0000 7f5ff872b640 1 --2- 172.21.15.52:0/588618964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ff0107b50 0x7f5ff0107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.606+0000 7f5ff872b640 1 -- 172.21.15.52:0/588618964 >> 172.21.15.52:0/588618964 conn(0x7f5ff0077b00 msgr2=0x7f5ff0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:26.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.606+0000 7f5ff872b640 1 -- 172.21.15.52:0/588618964 shutdown_connections 2024-04-02T13:57:26.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.606+0000 7f5ff872b640 1 -- 172.21.15.52:0/588618964 wait complete. 2024-04-02T13:57:26.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.606+0000 7f5ff872b640 1 Processor -- start 2024-04-02T13:57:26.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.606+0000 7f5ff872b640 1 -- start start 2024-04-02T13:57:26.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.606+0000 7f5ff872b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ff0068d00 0x7f5ff01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.606+0000 7f5ff872b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ff00696c0 0x7f5ff0198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.606+0000 7f5ff872b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ff0107b50 0x7f5ff019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.606+0000 7f5ff872b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff0111ea0 con 0x7f5ff0068d00 2024-04-02T13:57:26.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.606+0000 7f5ff872b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff0111d20 con 0x7f5ff00696c0 2024-04-02T13:57:26.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.606+0000 7f5ff872b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff0112020 con 0x7f5ff0107b50 2024-04-02T13:57:26.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.607+0000 7f5ff64c7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ff0068d00 0x7f5ff01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.607+0000 7f5ff64c7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ff0068d00 0x7f5ff01983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48240/0 (socket says 172.21.15.52:48240) 2024-04-02T13:57:26.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.607+0000 7f5ff64c7640 1 -- 172.21.15.52:0/927962088 learned_addr learned my addr 172.21.15.52:0/927962088 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:26.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.607+0000 7f5ff6cc8640 1 --2- 172.21.15.52:0/927962088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ff0107b50 0x7f5ff019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.607+0000 7f5ff5cc6640 1 --2- 172.21.15.52:0/927962088 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ff00696c0 0x7f5ff0198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.607+0000 7f5ff64c7640 1 -- 172.21.15.52:0/927962088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ff0107b50 msgr2=0x7f5ff019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:26.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.607+0000 7f5ff64c7640 1 --2- 172.21.15.52:0/927962088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ff0107b50 0x7f5ff019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.607+0000 7f5ff64c7640 1 -- 172.21.15.52:0/927962088 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ff00696c0 msgr2=0x7f5ff0198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:26.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.607+0000 7f5ff64c7640 1 --2- 172.21.15.52:0/927962088 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ff00696c0 0x7f5ff0198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.607+0000 7f5ff64c7640 1 -- 172.21.15.52:0/927962088 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff019d3b0 con 0x7f5ff0068d00 2024-04-02T13:57:26.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.607+0000 7f5ff5cc6640 1 --2- 172.21.15.52:0/927962088 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ff00696c0 0x7f5ff0198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:26.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.607+0000 7f5ff64c7640 1 --2- 172.21.15.52:0/927962088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ff0068d00 0x7f5ff01983c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000cce0 tx=0x7f5fe000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:26.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.607+0000 7f5fef7fe640 1 -- 172.21.15.52:0/927962088 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0004b20 con 0x7f5ff0068d00 2024-04-02T13:57:26.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.607+0000 7f5fef7fe640 1 -- 172.21.15.52:0/927962088 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fe0004cc0 con 0x7f5ff0068d00 2024-04-02T13:57:26.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.607+0000 7f5ff872b640 1 -- 172.21.15.52:0/927962088 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff019d6a0 con 0x7f5ff0068d00 2024-04-02T13:57:26.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.607+0000 7f5fef7fe640 1 -- 172.21.15.52:0/927962088 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe000f7d0 con 0x7f5ff0068d00 2024-04-02T13:57:26.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.607+0000 7f5ff872b640 1 -- 172.21.15.52:0/927962088 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff01a4f30 con 0x7f5ff0068d00 2024-04-02T13:57:26.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.609+0000 7f5fef7fe640 1 -- 172.21.15.52:0/927962088 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5fe000f970 con 0x7f5ff0068d00 2024-04-02T13:57:26.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.610+0000 7f5ffb3eb740 1 Processor -- start 2024-04-02T13:57:26.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.610+0000 7f5ffb3eb740 1 -- start start 2024-04-02T13:57:26.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.610+0000 7f5fef7fe640 1 --2- 172.21.15.52:0/927962088 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5fd80a3da0 0x7f5fd80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.610+0000 7f5ffb3eb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8cdcff0 0x55f8a8cdd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.610+0000 7f5ffb3eb740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8a8cdd910 0x55f8a8cea690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.610+0000 7f5ffb3eb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8a8ceabd0 0x55f8a8cecfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.610+0000 7f5ffb3eb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f8a8c41770 con 0x55f8a8cdcff0 2024-04-02T13:57:26.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.610+0000 7f5ffb3eb740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f8a8be9960 con 0x55f8a8cdd910 2024-04-02T13:57:26.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.610+0000 7f5ffb3eb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f8a8c3d8c0 con 0x55f8a8ceabd0 2024-04-02T13:57:26.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.611+0000 7f5fef7fe640 1 -- 172.21.15.52:0/927962088 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f5fe00d0bd0 con 0x7f5ff0068d00 2024-04-02T13:57:26.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.611+0000 7f5ff6cc8640 1 --2- 172.21.15.52:0/927962088 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5fd80a3da0 0x7f5fd80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.611+0000 7f5ff64c7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8cdcff0 0x55f8a8cdd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.611+0000 7f5ff64c7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8cdcff0 0x55f8a8cdd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48256/0 (socket says 172.21.15.52:48256) 2024-04-02T13:57:26.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.611+0000 7f5ff64c7640 1 -- 172.21.15.52:0/2163346016 learned_addr learned my addr 172.21.15.52:0/2163346016 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:26.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.611+0000 7f5ff6cc8640 1 --2- 172.21.15.52:0/2163346016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8a8ceabd0 0x55f8a8cecfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.611+0000 7f5ff5cc6640 1 --2- 172.21.15.52:0/2163346016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8a8cdd910 0x55f8a8cea690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.611+0000 7f5ff64c7640 1 -- 172.21.15.52:0/2163346016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8a8ceabd0 msgr2=0x55f8a8cecfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:26.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.611+0000 7f5ff64c7640 1 --2- 172.21.15.52:0/2163346016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8a8ceabd0 0x55f8a8cecfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.611+0000 7f5ff64c7640 1 -- 172.21.15.52:0/2163346016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8a8cdd910 msgr2=0x55f8a8cea690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:26.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.611+0000 7f5ff64c7640 1 --2- 172.21.15.52:0/2163346016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8a8cdd910 0x55f8a8cea690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.611+0000 7f5ff64c7640 1 -- 172.21.15.52:0/2163346016 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8a8a1d7e0 con 0x55f8a8cdcff0 2024-04-02T13:57:26.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.611+0000 7f5ff64c7640 1 --2- 172.21.15.52:0/2163346016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8cdcff0 0x55f8a8cdd3d0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0098c90 tx=0x7f5fe009c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c0f7499c547ef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:26.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.611+0000 7f5ff6cc8640 1 --2- 172.21.15.52:0/927962088 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5fd80a3da0 0x7f5fd80a6260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009dc0 tx=0x7f5fe80156d0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:26.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.611+0000 7f5fecff9640 1 -- 172.21.15.52:0/2163346016 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe001d040 con 0x55f8a8cdcff0 2024-04-02T13:57:26.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.611+0000 7f5fecff9640 1 -- 172.21.15.52:0/2163346016 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fe009cf30 con 0x55f8a8cdcff0 2024-04-02T13:57:26.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.611+0000 7f5fecff9640 1 -- 172.21.15.52:0/2163346016 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe00ce260 con 0x55f8a8cdcff0 2024-04-02T13:57:26.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.612+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/2163346016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8cdcff0 msgr2=0x55f8a8cdd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:26.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.612+0000 7f5ffb3eb740 1 --2- 172.21.15.52:0/2163346016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8cdcff0 0x55f8a8cdd3d0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0098c90 tx=0x7f5fe009c430 comp rx=0 tx=0).stop 2024-04-02T13:57:26.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.612+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/2163346016 shutdown_connections 2024-04-02T13:57:26.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.612+0000 7f5ffb3eb740 1 --2- 172.21.15.52:0/2163346016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8a8ceabd0 0x55f8a8cecfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.612+0000 7f5ffb3eb740 1 --2- 172.21.15.52:0/2163346016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8a8cdd910 0x55f8a8cea690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.612+0000 7f5ffb3eb740 1 --2- 172.21.15.52:0/2163346016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8cdcff0 0x55f8a8cdd3d0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.612+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/2163346016 >> 172.21.15.52:0/2163346016 conn(0x55f8a8cd79f0 msgr2=0x55f8a8cd7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:26.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.612+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/2163346016 shutdown_connections 2024-04-02T13:57:26.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.612+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/2163346016 wait complete. 2024-04-02T13:57:26.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.612+0000 7f5ffb3eb740 1 -- start start 2024-04-02T13:57:26.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.613+0000 7f5ffb3eb740 1 Processor -- start 2024-04-02T13:57:26.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.613+0000 7f5ffb3eb740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8a8cdcff0 0x55f8a8ce7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.613+0000 7f5ffb3eb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8cdd910 0x55f8a8ce81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.613+0000 7f5ffb3eb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8a8ceabd0 0x55f8a8ce8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.613+0000 7f5ffb3eb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f8a8ced990 con 0x55f8a8cdd910 2024-04-02T13:57:26.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.613+0000 7f5ffb3eb740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f8a8bfcd20 con 0x55f8a8cdcff0 2024-04-02T13:57:26.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.613+0000 7f5ffb3eb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f8a8cedb10 con 0x55f8a8ceabd0 2024-04-02T13:57:26.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.613+0000 7f5ffb3eb740 10 client.0 _ll_register_callbacks cb 0x55f8a8bc9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:26.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.613+0000 7f5ff5cc6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8cdd910 0x55f8a8ce81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.613+0000 7f5ff5cc6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8cdd910 0x55f8a8ce81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48272/0 (socket says 172.21.15.52:48272) 2024-04-02T13:57:26.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.613+0000 7f5ff5cc6640 1 -- 172.21.15.52:0/1896159964 learned_addr learned my addr 172.21.15.52:0/1896159964 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:26.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.613+0000 7f5ff64c7640 1 --2- 172.21.15.52:0/1896159964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8a8cdcff0 0x55f8a8ce7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.613+0000 7f5ff5cc6640 1 -- 172.21.15.52:0/1896159964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8a8ceabd0 msgr2=0x55f8a8ce8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:26.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.613+0000 7f5ff5cc6640 1 --2- 172.21.15.52:0/1896159964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8a8ceabd0 0x55f8a8ce8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.613+0000 7f5ff5cc6640 1 -- 172.21.15.52:0/1896159964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8a8cdcff0 msgr2=0x55f8a8ce7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:26.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.613+0000 7f5ff5cc6640 1 --2- 172.21.15.52:0/1896159964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8a8cdcff0 0x55f8a8ce7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.613+0000 7f5ff5cc6640 1 -- 172.21.15.52:0/1896159964 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8a8a7df70 con 0x55f8a8cdd910 2024-04-02T13:57:26.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.613+0000 7f5ff5cc6640 1 --2- 172.21.15.52:0/1896159964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8cdd910 0x55f8a8ce81f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009900 tx=0x7f5fe4005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38f3fb9b5075f24 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:26.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.613+0000 7f5fd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:26.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.613+0000 7f5fd5ffb640 1 -- 172.21.15.52:0/1896159964 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4043030 con 0x55f8a8cdd910 2024-04-02T13:57:26.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.614+0000 7f5fd5ffb640 1 -- 172.21.15.52:0/1896159964 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f5fe402cc40 con 0x55f8a8cdd910 2024-04-02T13:57:26.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.614+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/1896159964 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fe009cf30 con 0x55f8a8cdd910 2024-04-02T13:57:26.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.614+0000 7f5fd5ffb640 1 -- 172.21.15.52:0/1896159964 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5fe4050050 con 0x55f8a8cdd910 2024-04-02T13:57:26.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.614+0000 7f5ffb3eb740 10 client.16818 fetch_fsmap learned FSMap version 29 2024-04-02T13:57:26.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.614+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/1896159964 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f8a8915170 con 0x55f8a8cdd910 2024-04-02T13:57:26.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.614+0000 7f5fd5ffb640 1 -- 172.21.15.52:0/1896159964 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fe009cf30 con 0x55f8a8cdd910 2024-04-02T13:57:26.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.614+0000 7f5fd5ffb640 1 -- 172.21.15.52:0/1896159964 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f5fe40035f0 con 0x55f8a8cdd910 2024-04-02T13:57:26.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.614+0000 7f5ffb3eb740 10 client.16818 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:57:26.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.614+0000 7f5ffb3eb740 10 client.16818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:57:26.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.614+0000 7f5ffb3eb740 20 client.16818 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:26.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.614+0000 7f5ffb3eb740 1 --2- 172.21.15.52:0/1896159964 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55f8a8cfb130 0x55f8a8cfd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.614+0000 7f5ffb3eb740 4 client.16818 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:57:26.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.614+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/1896159964 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8a8c4ecb0 con 0x55f8a8cfb130 2024-04-02T13:57:26.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.614+0000 7f5ff64c7640 1 --2- 172.21.15.52:0/1896159964 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55f8a8cfb130 0x55f8a8cfd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.615+0000 7f5ff64c7640 1 --2- 172.21.15.52:0/1896159964 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55f8a8cfb130 0x55f8a8cfd570 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e36ed88676db25ce server_cookie=f7b9f646c4cb55c4 in_seq=0 out_seq=0 2024-04-02T13:57:26.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.615+0000 7f5fd5ffb640 10 client.16818 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:26.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.616+0000 7f5fd5ffb640 1 -- 172.21.15.52:0/1896159964 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f8a8c4ecb0 con 0x55f8a8cfb130 2024-04-02T13:57:26.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.616+0000 7f5fd5ffb640 10 client.16818 handle_command_reply: tid=0 2024-04-02T13:57:26.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.616+0000 7f5ffb3eb740 1 client.16818 shutdown 2024-04-02T13:57:26.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.616+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/1896159964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8cdd910 msgr2=0x55f8a8ce81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:26.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.616+0000 7f5ffb3eb740 1 --2- 172.21.15.52:0/1896159964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8cdd910 0x55f8a8ce81f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009900 tx=0x7f5fe4005e30 comp rx=0 tx=0).stop 2024-04-02T13:57:26.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.616+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/1896159964 shutdown_connections 2024-04-02T13:57:26.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.616+0000 7f5ffb3eb740 1 --2- 172.21.15.52:0/1896159964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8a8ceabd0 0x55f8a8ce8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.616+0000 7f5ffb3eb740 1 --2- 172.21.15.52:0/1896159964 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55f8a8cfb130 0x55f8a8cfd570 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.616+0000 7f5ffb3eb740 1 --2- 172.21.15.52:0/1896159964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8cdd910 0x55f8a8ce81f0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.616+0000 7f5fd5ffb640 0 client.16818 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:26.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.616+0000 7f5ffb3eb740 1 --2- 172.21.15.52:0/1896159964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8a8cdcff0 0x55f8a8ce7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.616+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/1896159964 >> 172.21.15.52:0/1896159964 conn(0x55f8a8cd79f0 msgr2=0x55f8a8cd9740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:26.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.616+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/1896159964 shutdown_connections 2024-04-02T13:57:26.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.616+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/1896159964 wait complete. 2024-04-02T13:57:26.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.616+0000 7f5ffb3eb740 20 client.16818 trim_cache size 0 max 16384 2024-04-02T13:57:26.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.634+0000 7f5ffb3eb740 1 Processor -- start 2024-04-02T13:57:26.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.634+0000 7f5ffb3eb740 1 -- start start 2024-04-02T13:57:26.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.634+0000 7f5ffb3eb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8a8cdcff0 0x55f8a8cf6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.634+0000 7f5ffb3eb740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8a8cdd910 0x55f8a8ce8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.634+0000 7f5ffb3eb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8ceabd0 0x55f8a8d02e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.634+0000 7f5ffb3eb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff01accc0 con 0x55f8a8ceabd0 2024-04-02T13:57:26.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.634+0000 7f5ffb3eb740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff01ae830 con 0x55f8a8cdd910 2024-04-02T13:57:26.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.634+0000 7f5ffb3eb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff01ad390 con 0x55f8a8cdcff0 2024-04-02T13:57:26.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.634+0000 7f5ff6cc8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8ceabd0 0x55f8a8d02e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.635+0000 7f5ff6cc8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8ceabd0 0x55f8a8d02e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48280/0 (socket says 172.21.15.52:48280) 2024-04-02T13:57:26.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.635+0000 7f5ff6cc8640 1 -- 172.21.15.52:0/2619681079 learned_addr learned my addr 172.21.15.52:0/2619681079 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:26.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.635+0000 7f5ff64c7640 1 --2- 172.21.15.52:0/2619681079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8a8cdcff0 0x55f8a8cf6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.635+0000 7f5ff5cc6640 1 --2- 172.21.15.52:0/2619681079 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8a8cdd910 0x55f8a8ce8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.635+0000 7f5ff6cc8640 1 -- 172.21.15.52:0/2619681079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8a8cdcff0 msgr2=0x55f8a8cf6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:26.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.635+0000 7f5ff6cc8640 1 --2- 172.21.15.52:0/2619681079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8a8cdcff0 0x55f8a8cf6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.635+0000 7f5ff6cc8640 1 -- 172.21.15.52:0/2619681079 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8a8cdd910 msgr2=0x55f8a8ce8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:26.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.635+0000 7f5ff6cc8640 1 --2- 172.21.15.52:0/2619681079 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8a8cdd910 0x55f8a8ce8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.635+0000 7f5ff6cc8640 1 -- 172.21.15.52:0/2619681079 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8a8cdc3b0 con 0x55f8a8ceabd0 2024-04-02T13:57:26.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.635+0000 7f5ff5cc6640 1 --2- 172.21.15.52:0/2619681079 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8a8cdd910 0x55f8a8ce8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:26.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.635+0000 7f5ff64c7640 1 --2- 172.21.15.52:0/2619681079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8a8cdcff0 0x55f8a8cf6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:26.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.635+0000 7f5ff6cc8640 1 --2- 172.21.15.52:0/2619681079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8ceabd0 0x55f8a8d02e40 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009e20 tx=0x7f5fe8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27603e63a1f7c80 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:26.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.635+0000 7f5fd7fff640 1 -- 172.21.15.52:0/2619681079 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8018e10 con 0x55f8a8ceabd0 2024-04-02T13:57:26.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.635+0000 7f5fd7fff640 1 -- 172.21.15.52:0/2619681079 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fe8029070 con 0x55f8a8ceabd0 2024-04-02T13:57:26.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.635+0000 7f5fd7fff640 1 -- 172.21.15.52:0/2619681079 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8024b10 con 0x55f8a8ceabd0 2024-04-02T13:57:26.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.636+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/2619681079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8ceabd0 msgr2=0x55f8a8d02e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:26.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.636+0000 7f5ffb3eb740 1 --2- 172.21.15.52:0/2619681079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8ceabd0 0x55f8a8d02e40 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009e20 tx=0x7f5fe8018000 comp rx=0 tx=0).stop 2024-04-02T13:57:26.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.636+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/2619681079 shutdown_connections 2024-04-02T13:57:26.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.636+0000 7f5ffb3eb740 1 --2- 172.21.15.52:0/2619681079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8ceabd0 0x55f8a8d02e40 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.636+0000 7f5ffb3eb740 1 --2- 172.21.15.52:0/2619681079 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8a8cdd910 0x55f8a8ce8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.636+0000 7f5ffb3eb740 1 --2- 172.21.15.52:0/2619681079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8a8cdcff0 0x55f8a8cf6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.636+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/2619681079 >> 172.21.15.52:0/2619681079 conn(0x55f8a8cd79f0 msgr2=0x55f8a8cd6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:26.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.636+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/2619681079 shutdown_connections 2024-04-02T13:57:26.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.636+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/2619681079 wait complete. 2024-04-02T13:57:26.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.636+0000 7f5ffb3eb740 1 -- start start 2024-04-02T13:57:26.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.636+0000 7f5ffb3eb740 1 Processor -- start 2024-04-02T13:57:26.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.637+0000 7f5ffb3eb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8a8cdcff0 0x55f8a8ceee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.637+0000 7f5ffb3eb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8cdd910 0x55f8a8cef3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.637+0000 7f5ffb3eb740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8a8ceabd0 0x55f8a8d606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.637+0000 7f5ffb3eb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff01ae130 con 0x55f8a8cdd910 2024-04-02T13:57:26.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.637+0000 7f5ffb3eb740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff01ada60 con 0x55f8a8ceabd0 2024-04-02T13:57:26.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.637+0000 7f5ffb3eb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff01aaca0 con 0x55f8a8cdcff0 2024-04-02T13:57:26.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.637+0000 7f5ffb3eb740 10 client.0 _ll_register_callbacks cb 0x7f5fe4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:26.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.637+0000 7f5ff5cc6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8cdd910 0x55f8a8cef3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.637+0000 7f5ff64c7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8a8cdcff0 0x55f8a8ceee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.637+0000 7f5ff5cc6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8cdd910 0x55f8a8cef3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48290/0 (socket says 172.21.15.52:48290) 2024-04-02T13:57:26.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.637+0000 7f5ff5cc6640 1 -- 172.21.15.52:0/980949139 learned_addr learned my addr 172.21.15.52:0/980949139 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:26.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.637+0000 7f5ff6cc8640 1 --2- 172.21.15.52:0/980949139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8a8ceabd0 0x55f8a8d606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.637+0000 7f5ff5cc6640 1 -- 172.21.15.52:0/980949139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8a8cdcff0 msgr2=0x55f8a8ceee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:26.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.637+0000 7f5ff5cc6640 1 --2- 172.21.15.52:0/980949139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8a8cdcff0 0x55f8a8ceee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.637+0000 7f5ff5cc6640 1 -- 172.21.15.52:0/980949139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8a8ceabd0 msgr2=0x55f8a8d606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:26.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.637+0000 7f5ff5cc6640 1 --2- 172.21.15.52:0/980949139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8a8ceabd0 0x55f8a8d606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.637+0000 7f5ff5cc6640 1 -- 172.21.15.52:0/980949139 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8a8d60bf0 con 0x55f8a8cdd910 2024-04-02T13:57:26.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.637+0000 7f5ff6cc8640 1 --2- 172.21.15.52:0/980949139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8a8ceabd0 0x55f8a8d606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:26.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.637+0000 7f5ff64c7640 1 --2- 172.21.15.52:0/980949139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8a8cdcff0 0x55f8a8ceee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:57:26.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.637+0000 7f5ff5cc6640 1 --2- 172.21.15.52:0/980949139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8cdd910 0x55f8a8cef3b0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4003bb0 tx=0x7f5fe402c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43a779ea4200916 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:26.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.637+0000 7f5fd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:26.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.637+0000 7f5fd5ffb640 1 -- 172.21.15.52:0/980949139 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4043040 con 0x55f8a8cdd910 2024-04-02T13:57:26.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.637+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/980949139 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fe8029070 con 0x55f8a8cdd910 2024-04-02T13:57:26.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.637+0000 7f5fd5ffb640 1 -- 172.21.15.52:0/980949139 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f5fe4003c30 con 0x55f8a8cdd910 2024-04-02T13:57:26.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.638+0000 7f5fd5ffb640 1 -- 172.21.15.52:0/980949139 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5fe4003e50 con 0x55f8a8cdd910 2024-04-02T13:57:26.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.638+0000 7f5ffb3eb740 10 client.16824 fetch_fsmap learned FSMap version 29 2024-04-02T13:57:26.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.638+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/980949139 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f8a8d61e10 con 0x55f8a8cdd910 2024-04-02T13:57:26.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.638+0000 7f5fd5ffb640 1 -- 172.21.15.52:0/980949139 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fe8029070 con 0x55f8a8cdd910 2024-04-02T13:57:26.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.638+0000 7f5fd5ffb640 1 -- 172.21.15.52:0/980949139 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f5fe4004a30 con 0x55f8a8cdd910 2024-04-02T13:57:26.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.638+0000 7f5ffb3eb740 10 client.16824 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:57:26.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.638+0000 7f5ffb3eb740 10 client.16824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:57:26.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.638+0000 7f5ffb3eb740 20 client.16824 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:26.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.638+0000 7f5ffb3eb740 1 --2- 172.21.15.52:0/980949139 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55f8a8cfb130 0x55f8a8d645c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.638+0000 7f5ffb3eb740 4 client.16824 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:57:26.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.638+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/980949139 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f8a8c49000 con 0x55f8a8cfb130 2024-04-02T13:57:26.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.638+0000 7f5ff64c7640 1 --2- 172.21.15.52:0/980949139 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55f8a8cfb130 0x55f8a8d645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.639+0000 7f5fd5ffb640 10 client.16824 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:26.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.639+0000 7f5ff64c7640 1 --2- 172.21.15.52:0/980949139 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55f8a8cfb130 0x55f8a8d645c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6814d2aa3dd39625 server_cookie=a9fb9ff1e597cd81 in_seq=0 out_seq=0 2024-04-02T13:57:26.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.640+0000 7f5fd5ffb640 1 -- 172.21.15.52:0/980949139 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f8a8c49000 con 0x55f8a8cfb130 2024-04-02T13:57:26.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.640+0000 7f5fd5ffb640 10 client.16824 handle_command_reply: tid=0 2024-04-02T13:57:26.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.640+0000 7f5ffb3eb740 1 client.16824 shutdown 2024-04-02T13:57:26.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.640+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/980949139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8cdd910 msgr2=0x55f8a8cef3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:26.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.640+0000 7f5ffb3eb740 1 --2- 172.21.15.52:0/980949139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8cdd910 0x55f8a8cef3b0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4003bb0 tx=0x7f5fe402c3b0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.640+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/980949139 shutdown_connections 2024-04-02T13:57:26.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.640+0000 7f5ffb3eb740 1 --2- 172.21.15.52:0/980949139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8a8ceabd0 0x55f8a8d606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.640+0000 7f5ffb3eb740 1 --2- 172.21.15.52:0/980949139 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55f8a8cfb130 0x55f8a8d645c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.640+0000 7f5ffb3eb740 1 --2- 172.21.15.52:0/980949139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f8a8cdd910 0x55f8a8cef3b0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.640+0000 7f5fd5ffb640 0 client.16824 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:26.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.640+0000 7f5ffb3eb740 1 --2- 172.21.15.52:0/980949139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8a8cdcff0 0x55f8a8ceee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.640+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/980949139 >> 172.21.15.52:0/980949139 conn(0x55f8a8cd79f0 msgr2=0x55f8a8cd8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:26.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.640+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/980949139 shutdown_connections 2024-04-02T13:57:26.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.640+0000 7f5ffb3eb740 1 -- 172.21.15.52:0/980949139 wait complete. 2024-04-02T13:57:26.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.640+0000 7f5ffb3eb740 20 client.16824 trim_cache size 0 max 16384 2024-04-02T13:57:26.660 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:57:26.660 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-04-02T13:57:26.660 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "54871015-5d0d-4b59-9325-4e8a0e7f0ac1", 2024-04-02T13:57:26.660 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-04-02T13:57:26.661 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:57:26.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.641+0000 7f5ff872b640 1 -- 172.21.15.52:0/927962088 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5fd80a3da0 msgr2=0x7f5fd80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:26.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.641+0000 7f5ff872b640 1 --2- 172.21.15.52:0/927962088 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5fd80a3da0 0x7f5fd80a6260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009dc0 tx=0x7f5fe80156d0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.641+0000 7f5ff872b640 1 -- 172.21.15.52:0/927962088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ff0068d00 msgr2=0x7f5ff01983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:26.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.641+0000 7f5ff872b640 1 --2- 172.21.15.52:0/927962088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ff0068d00 0x7f5ff01983c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000cce0 tx=0x7f5fe000a620 comp rx=0 tx=0).stop 2024-04-02T13:57:26.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.641+0000 7f5ff872b640 1 -- 172.21.15.52:0/927962088 shutdown_connections 2024-04-02T13:57:26.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.641+0000 7f5ff872b640 1 --2- 172.21.15.52:0/927962088 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5fd80a3da0 0x7f5fd80a6260 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.641+0000 7f5ff872b640 1 --2- 172.21.15.52:0/927962088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ff0107b50 0x7f5ff019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.641+0000 7f5ff872b640 1 --2- 172.21.15.52:0/927962088 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ff00696c0 0x7f5ff0198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.641+0000 7f5ff872b640 1 --2- 172.21.15.52:0/927962088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ff0068d00 0x7f5ff01983c0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.641+0000 7f5ff872b640 1 -- 172.21.15.52:0/927962088 >> 172.21.15.52:0/927962088 conn(0x7f5ff0077b00 msgr2=0x7f5ff00fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:26.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.641+0000 7f5ff872b640 1 -- 172.21.15.52:0/927962088 shutdown_connections 2024-04-02T13:57:26.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.641+0000 7f5ff872b640 1 -- 172.21.15.52:0/927962088 wait complete. 2024-04-02T13:57:26.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:57:26.666 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:57:26.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.772+0000 7f3e23870640 1 -- 172.21.15.52:0/1253234270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e1c0696a0 msgr2=0x7f3e1c10e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:26.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.772+0000 7f3e23870640 1 --2- 172.21.15.52:0/1253234270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e1c0696a0 0x7f3e1c10e940 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f3e14009930 tx=0x7f3e1402bc30 comp rx=0 tx=0).stop 2024-04-02T13:57:26.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.772+0000 7f3e23870640 1 -- 172.21.15.52:0/1253234270 shutdown_connections 2024-04-02T13:57:26.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.772+0000 7f3e23870640 1 --2- 172.21.15.52:0/1253234270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e1c0696a0 0x7f3e1c10e940 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.772+0000 7f3e23870640 1 --2- 172.21.15.52:0/1253234270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e1c068ce0 0x7f3e1c069160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.772+0000 7f3e23870640 1 --2- 172.21.15.52:0/1253234270 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e1c107a80 0x7f3e1c107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.772+0000 7f3e23870640 1 -- 172.21.15.52:0/1253234270 >> 172.21.15.52:0/1253234270 conn(0x7f3e1c077b00 msgr2=0x7f3e1c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:26.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.772+0000 7f3e23870640 1 -- 172.21.15.52:0/1253234270 shutdown_connections 2024-04-02T13:57:26.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.772+0000 7f3e23870640 1 -- 172.21.15.52:0/1253234270 wait complete. 2024-04-02T13:57:26.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.773+0000 7f3e23870640 1 Processor -- start 2024-04-02T13:57:26.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.773+0000 7f3e23870640 1 -- start start 2024-04-02T13:57:26.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.773+0000 7f3e23870640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e1c068ce0 0x7f3e1c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.773+0000 7f3e23870640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e1c0696a0 0x7f3e1c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.773+0000 7f3e23870640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e1c107a80 0x7f3e1c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.773+0000 7f3e23870640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c111e90 con 0x7f3e1c068ce0 2024-04-02T13:57:26.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.773+0000 7f3e23870640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c111d10 con 0x7f3e1c107a80 2024-04-02T13:57:26.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.773+0000 7f3e23870640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c112010 con 0x7f3e1c0696a0 2024-04-02T13:57:26.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.773+0000 7f3e2160c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e1c068ce0 0x7f3e1c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.774+0000 7f3e2160c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e1c068ce0 0x7f3e1c1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48318/0 (socket says 172.21.15.52:48318) 2024-04-02T13:57:26.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.774+0000 7f3e2160c640 1 -- 172.21.15.52:0/558034097 learned_addr learned my addr 172.21.15.52:0/558034097 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:26.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.774+0000 7f3e21e0d640 1 --2- 172.21.15.52:0/558034097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e1c107a80 0x7f3e1c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.774+0000 7f3e20e0b640 1 --2- 172.21.15.52:0/558034097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e1c0696a0 0x7f3e1c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.774+0000 7f3e2160c640 1 -- 172.21.15.52:0/558034097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e1c0696a0 msgr2=0x7f3e1c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:26.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.774+0000 7f3e2160c640 1 --2- 172.21.15.52:0/558034097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e1c0696a0 0x7f3e1c198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.774+0000 7f3e2160c640 1 -- 172.21.15.52:0/558034097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e1c107a80 msgr2=0x7f3e1c19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:26.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.774+0000 7f3e2160c640 1 --2- 172.21.15.52:0/558034097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e1c107a80 0x7f3e1c19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.774+0000 7f3e2160c640 1 -- 172.21.15.52:0/558034097 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1c19d390 con 0x7f3e1c068ce0 2024-04-02T13:57:26.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.774+0000 7f3e2160c640 1 --2- 172.21.15.52:0/558034097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e1c068ce0 0x7f3e1c1983f0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800ebc0 tx=0x7f3e1800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:26.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.774+0000 7f3e127fc640 1 -- 172.21.15.52:0/558034097 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e18010630 con 0x7f3e1c068ce0 2024-04-02T13:57:26.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.774+0000 7f3e127fc640 1 -- 172.21.15.52:0/558034097 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e18010cb0 con 0x7f3e1c068ce0 2024-04-02T13:57:26.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.774+0000 7f3e23870640 1 -- 172.21.15.52:0/558034097 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e1c19d680 con 0x7f3e1c068ce0 2024-04-02T13:57:26.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.774+0000 7f3e127fc640 1 -- 172.21.15.52:0/558034097 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e18018bd0 con 0x7f3e1c068ce0 2024-04-02T13:57:26.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.774+0000 7f3e23870640 1 -- 172.21.15.52:0/558034097 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e1c1a4f10 con 0x7f3e1c068ce0 2024-04-02T13:57:26.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.777+0000 7f3e127fc640 1 -- 172.21.15.52:0/558034097 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3e180107d0 con 0x7f3e1c068ce0 2024-04-02T13:57:26.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.777+0000 7f3e26457740 1 Processor -- start 2024-04-02T13:57:26.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.777+0000 7f3e26457740 1 -- start start 2024-04-02T13:57:26.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.778+0000 7f3e26457740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2bade28d0 0x55c2bade2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.778+0000 7f3e26457740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2bade31f0 0x55c2badeb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.778+0000 7f3e26457740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2badebdd0 0x55c2badee1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.778+0000 7f3e26457740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c2bad46830 con 0x55c2badebdd0 2024-04-02T13:57:26.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.778+0000 7f3e26457740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c2bacee820 con 0x55c2bade31f0 2024-04-02T13:57:26.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.778+0000 7f3e26457740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c2bad427e0 con 0x55c2bade28d0 2024-04-02T13:57:26.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.778+0000 7f3e21e0d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2bade28d0 0x55c2bade2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.778+0000 7f3e21e0d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2bade28d0 0x55c2bade2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.52:38688/0 (socket says 172.21.15.52:38688) 2024-04-02T13:57:26.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.778+0000 7f3e21e0d640 1 -- 172.21.15.52:0/2639488101 learned_addr learned my addr 172.21.15.52:0/2639488101 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:26.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.778+0000 7f3e2160c640 1 --2- 172.21.15.52:0/2639488101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2bade31f0 0x55c2badeb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.778+0000 7f3e20e0b640 1 --2- 172.21.15.52:0/2639488101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2badebdd0 0x55c2badee1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.778+0000 7f3e127fc640 1 --2- 172.21.15.52:0/558034097 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3e080a3d20 0x7f3e080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.779+0000 7f3e20e0b640 1 -- 172.21.15.52:0/2639488101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2bade28d0 msgr2=0x55c2bade2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:26.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.779+0000 7f3e20e0b640 1 --2- 172.21.15.52:0/2639488101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2bade28d0 0x55c2bade2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.779+0000 7f3e20e0b640 1 -- 172.21.15.52:0/2639488101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2bade31f0 msgr2=0x55c2badeb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:26.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.779+0000 7f3e20e0b640 1 --2- 172.21.15.52:0/2639488101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2bade31f0 0x55c2badeb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.779+0000 7f3e20e0b640 1 -- 172.21.15.52:0/2639488101 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2bab247d0 con 0x55c2badebdd0 2024-04-02T13:57:26.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.779+0000 7f3e21e0d640 1 --2- 172.21.15.52:0/2639488101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2bade28d0 0x55c2bade2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:57:26.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.779+0000 7f3e21e0d640 1 --2- 172.21.15.52:0/558034097 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3e080a3d20 0x7f3e080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.779+0000 7f3e127fc640 1 -- 172.21.15.52:0/558034097 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f3e180d10c0 con 0x7f3e1c068ce0 2024-04-02T13:57:26.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.779+0000 7f3e20e0b640 1 --2- 172.21.15.52:0/2639488101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2badebdd0 0x55c2badee1c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009bd0 tx=0x7f3e0c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4be48ff7e925b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:26.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.779+0000 7f3e077fe640 1 -- 172.21.15.52:0/2639488101 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c004230 con 0x55c2badebdd0 2024-04-02T13:57:26.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.779+0000 7f3e077fe640 1 -- 172.21.15.52:0/2639488101 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e0c009e40 con 0x55c2badebdd0 2024-04-02T13:57:26.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.779+0000 7f3e077fe640 1 -- 172.21.15.52:0/2639488101 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c004db0 con 0x55c2badebdd0 2024-04-02T13:57:26.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.779+0000 7f3e21e0d640 1 --2- 172.21.15.52:0/558034097 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3e080a3d20 0x7f3e080a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3e14006010 tx=0x7f3e1402e040 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:26.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.780+0000 7f3e26457740 1 -- 172.21.15.52:0/2639488101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2badebdd0 msgr2=0x55c2badee1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:26.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.780+0000 7f3e26457740 1 --2- 172.21.15.52:0/2639488101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2badebdd0 0x55c2badee1c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009bd0 tx=0x7f3e0c011810 comp rx=0 tx=0).stop 2024-04-02T13:57:26.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.780+0000 7f3e26457740 1 -- 172.21.15.52:0/2639488101 shutdown_connections 2024-04-02T13:57:26.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.780+0000 7f3e26457740 1 --2- 172.21.15.52:0/2639488101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2badebdd0 0x55c2badee1c0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.780+0000 7f3e26457740 1 --2- 172.21.15.52:0/2639488101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2bade31f0 0x55c2badeb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.780+0000 7f3e26457740 1 --2- 172.21.15.52:0/2639488101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2bade28d0 0x55c2bade2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.780+0000 7f3e26457740 1 -- 172.21.15.52:0/2639488101 >> 172.21.15.52:0/2639488101 conn(0x55c2baddd000 msgr2=0x55c2baddd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:26.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.780+0000 7f3e26457740 1 -- 172.21.15.52:0/2639488101 shutdown_connections 2024-04-02T13:57:26.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.780+0000 7f3e26457740 1 -- 172.21.15.52:0/2639488101 wait complete. 2024-04-02T13:57:26.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.780+0000 7f3e26457740 1 -- start start 2024-04-02T13:57:26.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.781+0000 7f3e26457740 1 Processor -- start 2024-04-02T13:57:26.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.781+0000 7f3e26457740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2bade28d0 0x55c2badeb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.781+0000 7f3e26457740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2bade31f0 0x55c2bade89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.781+0000 7f3e26457740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2badebdd0 0x55c2bade8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.781+0000 7f3e26457740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c2badeebc0 con 0x55c2badebdd0 2024-04-02T13:57:26.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.781+0000 7f3e26457740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c2bad01d30 con 0x55c2bade28d0 2024-04-02T13:57:26.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.781+0000 7f3e26457740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c2badeed40 con 0x55c2bade31f0 2024-04-02T13:57:26.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.781+0000 7f3e26457740 10 client.0 _ll_register_callbacks cb 0x55c2bad01720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:26.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.781+0000 7f3e2160c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2bade28d0 0x55c2badeb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.781+0000 7f3e2160c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2bade28d0 0x55c2badeb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:56144/0 (socket says 172.21.15.52:56144) 2024-04-02T13:57:26.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.781+0000 7f3e2160c640 1 -- 172.21.15.52:0/489168640 learned_addr learned my addr 172.21.15.52:0/489168640 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:26.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.781+0000 7f3e21e0d640 1 --2- 172.21.15.52:0/489168640 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2badebdd0 0x55c2bade8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.782+0000 7f3e2160c640 1 -- 172.21.15.52:0/489168640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2bade31f0 msgr2=0x55c2bade89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:26.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.782+0000 7f3e2160c640 1 --2- 172.21.15.52:0/489168640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2bade31f0 0x55c2bade89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.782+0000 7f3e2160c640 1 -- 172.21.15.52:0/489168640 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2badebdd0 msgr2=0x55c2bade8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:26.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.782+0000 7f3e2160c640 1 --2- 172.21.15.52:0/489168640 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2badebdd0 0x55c2bade8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.782+0000 7f3e2160c640 1 -- 172.21.15.52:0/489168640 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2baa5a3f0 con 0x55c2bade28d0 2024-04-02T13:57:26.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.782+0000 7f3e2160c640 1 --2- 172.21.15.52:0/489168640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2bade28d0 0x55c2badeb620 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3e180997a0 tx=0x7f3e18009200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa0db9c04e0905c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:26.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.782+0000 7f3e04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-02T13:57:26.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.782+0000 7f3e04ff9640 1 -- 172.21.15.52:0/489168640 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e18010e60 con 0x55c2bade28d0 2024-04-02T13:57:26.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.782+0000 7f3e26457740 1 -- 172.21.15.52:0/489168640 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c2bad53d70 con 0x55c2bade28d0 2024-04-02T13:57:26.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.782+0000 7f3e04ff9640 1 -- 172.21.15.52:0/489168640 <== mon.2 v2:172.21.15.88:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f3e180bd0b0 con 0x55c2bade28d0 2024-04-02T13:57:26.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.782+0000 7f3e04ff9640 1 -- 172.21.15.52:0/489168640 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3e1809c730 con 0x55c2bade28d0 2024-04-02T13:57:26.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.782+0000 7f3e26457740 10 client.26210 fetch_fsmap learned FSMap version 29 2024-04-02T13:57:26.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.782+0000 7f3e26457740 1 -- 172.21.15.52:0/489168640 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c2baa1a0f0 con 0x55c2bade28d0 2024-04-02T13:57:26.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.782+0000 7f3e04ff9640 1 -- 172.21.15.52:0/489168640 <== mon.2 v2:172.21.15.88:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c2bad53d70 con 0x55c2bade28d0 2024-04-02T13:57:26.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.783+0000 7f3e04ff9640 1 -- 172.21.15.52:0/489168640 <== mon.2 v2:172.21.15.88:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f3e180b60f0 con 0x55c2bade28d0 2024-04-02T13:57:26.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.783+0000 7f3e26457740 10 client.26210 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:57:26.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.783+0000 7f3e26457740 10 client.26210 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:57:26.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.783+0000 7f3e26457740 20 client.26210 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:26.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.783+0000 7f3e26457740 1 --2- 172.21.15.52:0/489168640 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c2bae00120 0x55c2bae025b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.783+0000 7f3e26457740 4 client.26210 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:57:26.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.783+0000 7f3e26457740 1 -- 172.21.15.52:0/489168640 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2bad4e0c0 con 0x55c2bae00120 2024-04-02T13:57:26.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.783+0000 7f3e20e0b640 1 --2- 172.21.15.52:0/489168640 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c2bae00120 0x55c2bae025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.783+0000 7f3e20e0b640 1 --2- 172.21.15.52:0/489168640 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c2bae00120 0x55c2bae025b0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe4cd0cad11012aa server_cookie=41c5c7eab87a1359 in_seq=0 out_seq=0 2024-04-02T13:57:26.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.783+0000 7f3e04ff9640 10 client.26210 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:26.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.784+0000 7f3e04ff9640 1 -- 172.21.15.52:0/489168640 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c2bad4e0c0 con 0x55c2bae00120 2024-04-02T13:57:26.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.784+0000 7f3e04ff9640 10 client.26210 handle_command_reply: tid=0 2024-04-02T13:57:26.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.784+0000 7f3e26457740 1 client.26210 shutdown 2024-04-02T13:57:26.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.784+0000 7f3e26457740 1 -- 172.21.15.52:0/489168640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2bade28d0 msgr2=0x55c2badeb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:26.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.784+0000 7f3e26457740 1 --2- 172.21.15.52:0/489168640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2bade28d0 0x55c2badeb620 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3e180997a0 tx=0x7f3e18009200 comp rx=0 tx=0).stop 2024-04-02T13:57:26.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.784+0000 7f3e26457740 1 -- 172.21.15.52:0/489168640 shutdown_connections 2024-04-02T13:57:26.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.784+0000 7f3e26457740 1 --2- 172.21.15.52:0/489168640 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c2bae00120 0x55c2bae025b0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.784+0000 7f3e26457740 1 --2- 172.21.15.52:0/489168640 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2badebdd0 0x55c2bade8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.784+0000 7f3e26457740 1 --2- 172.21.15.52:0/489168640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2bade31f0 0x55c2bade89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.784+0000 7f3e26457740 1 --2- 172.21.15.52:0/489168640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2bade28d0 0x55c2badeb620 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.784+0000 7f3e04ff9640 0 client.26210 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:26.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.784+0000 7f3e26457740 1 -- 172.21.15.52:0/489168640 >> 172.21.15.52:0/489168640 conn(0x55c2baddd000 msgr2=0x55c2badded50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:26.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.785+0000 7f3e26457740 1 -- 172.21.15.52:0/489168640 shutdown_connections 2024-04-02T13:57:26.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.785+0000 7f3e26457740 1 -- 172.21.15.52:0/489168640 wait complete. 2024-04-02T13:57:26.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.785+0000 7f3e26457740 20 client.26210 trim_cache size 0 max 16384 2024-04-02T13:57:26.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.805+0000 7f3e26457740 1 Processor -- start 2024-04-02T13:57:26.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.805+0000 7f3e26457740 1 -- start start 2024-04-02T13:57:26.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.805+0000 7f3e26457740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2bade28d0 0x55c2bade3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.805+0000 7f3e26457740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2bade31f0 0x55c2badeb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.805+0000 7f3e26457740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2badebdd0 0x55c2badeb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.805+0000 7f3e26457740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c1acc30 con 0x55c2badebdd0 2024-04-02T13:57:26.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.805+0000 7f3e26457740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c1ae7a0 con 0x55c2bade31f0 2024-04-02T13:57:26.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.805+0000 7f3e26457740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c1ad300 con 0x55c2bade28d0 2024-04-02T13:57:26.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.805+0000 7f3e21e0d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2badebdd0 0x55c2badeb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.805+0000 7f3e21e0d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2badebdd0 0x55c2badeb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48346/0 (socket says 172.21.15.52:48346) 2024-04-02T13:57:26.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.805+0000 7f3e21e0d640 1 -- 172.21.15.52:0/3223000135 learned_addr learned my addr 172.21.15.52:0/3223000135 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:26.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.805+0000 7f3e20e0b640 1 --2- 172.21.15.52:0/3223000135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2bade31f0 0x55c2badeb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.805+0000 7f3e2160c640 1 --2- 172.21.15.52:0/3223000135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2bade28d0 0x55c2bade3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.806+0000 7f3e21e0d640 1 -- 172.21.15.52:0/3223000135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2bade28d0 msgr2=0x55c2bade3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:26.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.806+0000 7f3e21e0d640 1 --2- 172.21.15.52:0/3223000135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2bade28d0 0x55c2bade3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.806+0000 7f3e21e0d640 1 -- 172.21.15.52:0/3223000135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2bade31f0 msgr2=0x55c2badeb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:26.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.806+0000 7f3e21e0d640 1 --2- 172.21.15.52:0/3223000135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2bade31f0 0x55c2badeb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.806+0000 7f3e21e0d640 1 -- 172.21.15.52:0/3223000135 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2baddce30 con 0x55c2badebdd0 2024-04-02T13:57:26.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.806+0000 7f3e21e0d640 1 --2- 172.21.15.52:0/3223000135 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2badebdd0 0x55c2badeb620 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3e1402e010 tx=0x7f3e14041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27199b71682ce4f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:26.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.806+0000 7f3e06ffd640 1 -- 172.21.15.52:0/3223000135 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e14004980 con 0x55c2badebdd0 2024-04-02T13:57:26.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.806+0000 7f3e06ffd640 1 -- 172.21.15.52:0/3223000135 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e14004b20 con 0x55c2badebdd0 2024-04-02T13:57:26.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.806+0000 7f3e06ffd640 1 -- 172.21.15.52:0/3223000135 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e14003a50 con 0x55c2badebdd0 2024-04-02T13:57:26.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.807+0000 7f3e26457740 1 -- 172.21.15.52:0/3223000135 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2badebdd0 msgr2=0x55c2badeb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:26.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.807+0000 7f3e26457740 1 --2- 172.21.15.52:0/3223000135 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2badebdd0 0x55c2badeb620 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3e1402e010 tx=0x7f3e14041000 comp rx=0 tx=0).stop 2024-04-02T13:57:26.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.807+0000 7f3e26457740 1 -- 172.21.15.52:0/3223000135 shutdown_connections 2024-04-02T13:57:26.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.807+0000 7f3e26457740 1 --2- 172.21.15.52:0/3223000135 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2badebdd0 0x55c2badeb620 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.807+0000 7f3e26457740 1 --2- 172.21.15.52:0/3223000135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2bade31f0 0x55c2badeb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.807+0000 7f3e26457740 1 --2- 172.21.15.52:0/3223000135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2bade28d0 0x55c2bade3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.807+0000 7f3e26457740 1 -- 172.21.15.52:0/3223000135 >> 172.21.15.52:0/3223000135 conn(0x55c2baddd000 msgr2=0x55c2bade3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:26.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.807+0000 7f3e26457740 1 -- 172.21.15.52:0/3223000135 shutdown_connections 2024-04-02T13:57:26.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.807+0000 7f3e26457740 1 -- 172.21.15.52:0/3223000135 wait complete. 2024-04-02T13:57:26.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.807+0000 7f3e26457740 1 -- start start 2024-04-02T13:57:26.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.807+0000 7f3e26457740 1 Processor -- start 2024-04-02T13:57:26.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.808+0000 7f3e26457740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2bade28d0 0x55c2bade59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.808+0000 7f3e26457740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2bade31f0 0x55c2bade5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.808+0000 7f3e26457740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2badebdd0 0x55c2bae656a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.808+0000 7f3e26457740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c1ae0a0 con 0x55c2bade31f0 2024-04-02T13:57:26.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.808+0000 7f3e26457740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c1ad9d0 con 0x55c2bade28d0 2024-04-02T13:57:26.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.808+0000 7f3e26457740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c1aac10 con 0x55c2badebdd0 2024-04-02T13:57:26.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.808+0000 7f3e26457740 10 client.0 _ll_register_callbacks cb 0x7f3e180d48e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:26.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.808+0000 7f3e20e0b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2bade31f0 0x55c2bade5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.808+0000 7f3e20e0b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2bade31f0 0x55c2bade5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48354/0 (socket says 172.21.15.52:48354) 2024-04-02T13:57:26.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.808+0000 7f3e20e0b640 1 -- 172.21.15.52:0/2377759946 learned_addr learned my addr 172.21.15.52:0/2377759946 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:26.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.808+0000 7f3e2160c640 1 --2- 172.21.15.52:0/2377759946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2bade28d0 0x55c2bade59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.808+0000 7f3e21e0d640 1 --2- 172.21.15.52:0/2377759946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2badebdd0 0x55c2bae656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.808+0000 7f3e20e0b640 1 -- 172.21.15.52:0/2377759946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2badebdd0 msgr2=0x55c2bae656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:26.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.808+0000 7f3e20e0b640 1 --2- 172.21.15.52:0/2377759946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2badebdd0 0x55c2bae656a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.808+0000 7f3e20e0b640 1 -- 172.21.15.52:0/2377759946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2bade28d0 msgr2=0x55c2bade59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:26.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.808+0000 7f3e20e0b640 1 --2- 172.21.15.52:0/2377759946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2bade28d0 0x55c2bade59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.808+0000 7f3e20e0b640 1 -- 172.21.15.52:0/2377759946 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2bade67c0 con 0x55c2bade31f0 2024-04-02T13:57:26.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.808+0000 7f3e20e0b640 1 --2- 172.21.15.52:0/2377759946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2bade31f0 0x55c2bade5ef0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x55c2badde190 tx=0x7f3e0c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16192e17c5ddf342 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:26.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.808+0000 7f3e04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:26.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.809+0000 7f3e04ff9640 1 -- 172.21.15.52:0/2377759946 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c01e040 con 0x55c2bade31f0 2024-04-02T13:57:26.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.809+0000 7f3e04ff9640 1 -- 172.21.15.52:0/2377759946 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f3e0c004200 con 0x55c2bade31f0 2024-04-02T13:57:26.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.809+0000 7f3e26457740 1 -- 172.21.15.52:0/2377759946 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e14004b20 con 0x55c2bade31f0 2024-04-02T13:57:26.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.809+0000 7f3e04ff9640 1 -- 172.21.15.52:0/2377759946 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3e0c037050 con 0x55c2bade31f0 2024-04-02T13:57:26.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.809+0000 7f3e26457740 10 client.16842 fetch_fsmap learned FSMap version 29 2024-04-02T13:57:26.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.809+0000 7f3e26457740 1 -- 172.21.15.52:0/2377759946 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c2bae66cf0 con 0x55c2bade31f0 2024-04-02T13:57:26.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.809+0000 7f3e04ff9640 1 -- 172.21.15.52:0/2377759946 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e14004b20 con 0x55c2bade31f0 2024-04-02T13:57:26.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.809+0000 7f3e04ff9640 1 -- 172.21.15.52:0/2377759946 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f3e0c018070 con 0x55c2bade31f0 2024-04-02T13:57:26.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.809+0000 7f3e26457740 10 client.16842 fetch_fsmap finished waiting for FSMap version 29 2024-04-02T13:57:26.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.809+0000 7f3e26457740 10 client.16842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:57:26.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.809+0000 7f3e26457740 20 client.16842 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:26.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.809+0000 7f3e26457740 1 --2- 172.21.15.52:0/2377759946 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c2bae67310 0x55c2bae69730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:26.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.810+0000 7f3e26457740 4 client.16842 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:57:26.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.810+0000 7f3e26457740 1 -- 172.21.15.52:0/2377759946 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c2baccb560 con 0x55c2bae67310 2024-04-02T13:57:26.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.810+0000 7f3e2160c640 1 --2- 172.21.15.52:0/2377759946 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c2bae67310 0x55c2bae69730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:26.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.810+0000 7f3e2160c640 1 --2- 172.21.15.52:0/2377759946 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c2bae67310 0x55c2bae69730 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b5d431ea250aedf server_cookie=eebb94f7ff77beb4 in_seq=0 out_seq=0 2024-04-02T13:57:26.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.810+0000 7f3e04ff9640 10 client.16842 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:26.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.810+0000 7f3e04ff9640 1 -- 172.21.15.52:0/2377759946 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55c2baccb560 con 0x55c2bae67310 2024-04-02T13:57:26.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.810+0000 7f3e04ff9640 10 client.16842 handle_command_reply: tid=0 2024-04-02T13:57:26.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.810+0000 7f3e26457740 1 client.16842 shutdown 2024-04-02T13:57:26.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.811+0000 7f3e26457740 1 -- 172.21.15.52:0/2377759946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2bade31f0 msgr2=0x55c2bade5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:26.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.811+0000 7f3e26457740 1 --2- 172.21.15.52:0/2377759946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2bade31f0 0x55c2bade5ef0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x55c2badde190 tx=0x7f3e0c0096a0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.811+0000 7f3e26457740 1 -- 172.21.15.52:0/2377759946 shutdown_connections 2024-04-02T13:57:26.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.811+0000 7f3e26457740 1 --2- 172.21.15.52:0/2377759946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2badebdd0 0x55c2bae656a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.811+0000 7f3e26457740 1 --2- 172.21.15.52:0/2377759946 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c2bae67310 0x55c2bae69730 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.811+0000 7f3e26457740 1 --2- 172.21.15.52:0/2377759946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2bade31f0 0x55c2bade5ef0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.811+0000 7f3e26457740 1 --2- 172.21.15.52:0/2377759946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2bade28d0 0x55c2bade59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.811+0000 7f3e04ff9640 0 client.16842 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:26.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.811+0000 7f3e26457740 1 -- 172.21.15.52:0/2377759946 >> 172.21.15.52:0/2377759946 conn(0x55c2baddd000 msgr2=0x55c2bade3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:26.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.811+0000 7f3e26457740 1 -- 172.21.15.52:0/2377759946 shutdown_connections 2024-04-02T13:57:26.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.811+0000 7f3e26457740 1 -- 172.21.15.52:0/2377759946 wait complete. 2024-04-02T13:57:26.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.811+0000 7f3e26457740 20 client.16842 trim_cache size 0 max 16384 2024-04-02T13:57:26.824 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:57:26.824 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "scrub active (2467 inodes in the stack)", 2024-04-02T13:57:26.824 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-04-02T13:57:26.824 INFO:teuthology.orchestra.run.smithi052.stdout: "54871015-5d0d-4b59-9325-4e8a0e7f0ac1": { 2024-04-02T13:57:26.824 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-04-02T13:57:26.825 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "54871015-5d0d-4b59-9325-4e8a0e7f0ac1", 2024-04-02T13:57:26.825 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-04-02T13:57:26.825 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:57:26.825 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:57:26.825 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:57:26.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.812+0000 7f3e23870640 1 -- 172.21.15.52:0/558034097 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3e080a3d20 msgr2=0x7f3e080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:26.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.812+0000 7f3e23870640 1 --2- 172.21.15.52:0/558034097 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3e080a3d20 0x7f3e080a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3e14006010 tx=0x7f3e1402e040 comp rx=0 tx=0).stop 2024-04-02T13:57:26.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.812+0000 7f3e23870640 1 -- 172.21.15.52:0/558034097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e1c068ce0 msgr2=0x7f3e1c1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:26.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.812+0000 7f3e23870640 1 --2- 172.21.15.52:0/558034097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e1c068ce0 0x7f3e1c1983f0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800ebc0 tx=0x7f3e1800c6a0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.812+0000 7f3e23870640 1 -- 172.21.15.52:0/558034097 shutdown_connections 2024-04-02T13:57:26.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.812+0000 7f3e23870640 1 --2- 172.21.15.52:0/558034097 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3e080a3d20 0x7f3e080a61e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.812+0000 7f3e23870640 1 --2- 172.21.15.52:0/558034097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e1c107a80 0x7f3e1c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.812+0000 7f3e23870640 1 --2- 172.21.15.52:0/558034097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e1c0696a0 0x7f3e1c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.812+0000 7f3e23870640 1 --2- 172.21.15.52:0/558034097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e1c068ce0 0x7f3e1c1983f0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:26.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.812+0000 7f3e23870640 1 -- 172.21.15.52:0/558034097 >> 172.21.15.52:0/558034097 conn(0x7f3e1c077b00 msgr2=0x7f3e1c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:26.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.812+0000 7f3e23870640 1 -- 172.21.15.52:0/558034097 shutdown_connections 2024-04-02T13:57:26.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:26.812+0000 7f3e23870640 1 -- 172.21.15.52:0/558034097 wait complete. 2024-04-02T13:57:26.828 INFO:tasks.cephfs.filesystem:scrub status for tag:54871015-5d0d-4b59-9325-4e8a0e7f0ac1 - {'path': '/', 'tag': '54871015-5d0d-4b59-9325-4e8a0e7f0ac1', 'options': 'recursive,force'} 2024-04-02T13:57:27.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:27 smithi088 ceph-mon[36528]: pgmap v121: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-04-02T13:57:27.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:27 smithi088 ceph-mon[36528]: scrub queued for path: / 2024-04-02T13:57:27.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:27 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:57:27.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:27 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:57:27.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:27 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:57:27.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:27 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:57:27.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:27 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:57:28.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:27 smithi189 ceph-mon[36683]: pgmap v121: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-04-02T13:57:28.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:27 smithi189 ceph-mon[36683]: scrub queued for path: / 2024-04-02T13:57:28.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:27 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:57:28.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:27 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:57:28.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:27 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:57:28.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:27 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:57:28.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:27 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:57:28.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:27 smithi052 ceph-mon[32104]: pgmap v121: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-04-02T13:57:28.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:27 smithi052 ceph-mon[32104]: scrub queued for path: / 2024-04-02T13:57:28.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:27 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:57:28.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:27 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:57:28.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:27 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:57:28.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:27 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:57:28.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:27 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:57:28.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:28 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:57:28.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:28 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:57:29.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:28 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:57:29.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:28 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:57:29.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:28 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:57:29.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:28 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:57:29.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:29 smithi088 ceph-mon[36528]: pgmap v122: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-04-02T13:57:30.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:29 smithi189 ceph-mon[36683]: pgmap v122: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-04-02T13:57:30.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:29 smithi052 ceph-mon[32104]: pgmap v122: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-04-02T13:57:31.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:31 smithi088 ceph-mon[36528]: pgmap v123: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-04-02T13:57:32.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:31 smithi189 ceph-mon[36683]: pgmap v123: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-04-02T13:57:32.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:31 smithi052 ceph-mon[32104]: pgmap v123: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-04-02T13:57:33.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:33 smithi088 ceph-mon[36528]: pgmap v124: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-04-02T13:57:34.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:33 smithi189 ceph-mon[36683]: pgmap v124: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-04-02T13:57:34.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:33 smithi052 ceph-mon[32104]: pgmap v124: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-04-02T13:57:35.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:35 smithi088 ceph-mon[36528]: pgmap v125: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-04-02T13:57:35.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:35 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T13:57:36.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:35 smithi189 ceph-mon[36683]: pgmap v125: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-04-02T13:57:36.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:35 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T13:57:36.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:35 smithi052 ceph-mon[32104]: pgmap v125: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-04-02T13:57:36.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:35 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T13:57:36.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:36 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T13:57:37.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:36 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T13:57:37.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:36 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T13:57:37.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:37 smithi088 ceph-mon[36528]: pgmap v126: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-04-02T13:57:37.939 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:37 smithi189 ceph-mon[36683]: pgmap v126: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-04-02T13:57:38.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:37 smithi052 ceph-mon[32104]: pgmap v126: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-04-02T13:57:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:38 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:57:39.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:38 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:57:39.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:38 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:57:39.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:39 smithi088 ceph-mon[36528]: pgmap v127: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-02T13:57:39.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:39 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:57:39.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:39 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:57:39.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:39 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:57:40.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:39 smithi189 ceph-mon[36683]: pgmap v127: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-02T13:57:40.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:39 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:57:40.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:39 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:57:40.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:39 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:57:40.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:39 smithi052 ceph-mon[32104]: pgmap v127: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-02T13:57:40.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:39 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:57:40.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:39 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:57:40.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:39 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:57:41.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:41 smithi088 ceph-mon[36528]: pgmap v128: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-02T13:57:42.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:41 smithi189 ceph-mon[36683]: pgmap v128: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-02T13:57:42.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:41 smithi052 ceph-mon[32104]: pgmap v128: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-02T13:57:43.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:43 smithi088 ceph-mon[36528]: pgmap v129: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-04-02T13:57:44.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:43 smithi189 ceph-mon[36683]: pgmap v129: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-04-02T13:57:44.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:43 smithi052 ceph-mon[32104]: pgmap v129: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-04-02T13:57:45.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:45 smithi088 ceph-mon[36528]: pgmap v130: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-02T13:57:46.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:45 smithi189 ceph-mon[36683]: pgmap v130: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-02T13:57:46.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:45 smithi052 ceph-mon[32104]: pgmap v130: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-02T13:57:47.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:46 smithi189 ceph-mon[36683]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:57:47.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:46 smithi189 ceph-mon[36683]: pgmap v131: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-04-02T13:57:47.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:46 smithi052 ceph-mon[32104]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:57:47.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:46 smithi052 ceph-mon[32104]: pgmap v131: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-04-02T13:57:47.169 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:46 smithi088 ceph-mon[36528]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:57:47.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:46 smithi088 ceph-mon[36528]: pgmap v131: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-04-02T13:57:49.288 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:49 smithi189 ceph-mon[36683]: pgmap v132: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T13:57:49.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:49 smithi052 ceph-mon[32104]: pgmap v132: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T13:57:49.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:49 smithi088 ceph-mon[36528]: pgmap v132: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T13:57:51.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:51 smithi052 ceph-mon[32104]: pgmap v133: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T13:57:51.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:51 smithi088 ceph-mon[36528]: pgmap v133: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T13:57:51.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:51 smithi189 ceph-mon[36683]: pgmap v133: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T13:57:53.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:53 smithi052 ceph-mon[32104]: pgmap v134: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:57:53.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:53 smithi088 ceph-mon[36528]: pgmap v134: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:57:53.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:53 smithi189 ceph-mon[36683]: pgmap v134: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:57:55.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:55 smithi052 ceph-mon[32104]: pgmap v135: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:57:55.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:55 smithi088 ceph-mon[36528]: pgmap v135: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:57:55.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:55 smithi189 ceph-mon[36683]: pgmap v135: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:57:56.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:57:56.830 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:57:56.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.943+0000 7f78c8c2c640 1 -- 172.21.15.52:0/3845862315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78c4100a20 msgr2=0x7f78c4100ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:56.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.943+0000 7f78c8c2c640 1 --2- 172.21.15.52:0/3845862315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78c4100a20 0x7f78c4100ea0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009930 tx=0x7f78b802bc30 comp rx=0 tx=0).stop 2024-04-02T13:57:56.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.944+0000 7f78c8c2c640 1 -- 172.21.15.52:0/3845862315 shutdown_connections 2024-04-02T13:57:56.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.944+0000 7f78c8c2c640 1 --2- 172.21.15.52:0/3845862315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78c41013e0 0x7f78c410d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.944+0000 7f78c8c2c640 1 --2- 172.21.15.52:0/3845862315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78c4100a20 0x7f78c4100ea0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.944+0000 7f78c8c2c640 1 --2- 172.21.15.52:0/3845862315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78c4106a30 0x7f78c4106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.944+0000 7f78c8c2c640 1 -- 172.21.15.52:0/3845862315 >> 172.21.15.52:0/3845862315 conn(0x7f78c4077b60 msgr2=0x7f78c4077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:56.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.944+0000 7f78c8c2c640 1 -- 172.21.15.52:0/3845862315 shutdown_connections 2024-04-02T13:57:56.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.944+0000 7f78c8c2c640 1 -- 172.21.15.52:0/3845862315 wait complete. 2024-04-02T13:57:56.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.944+0000 7f78c8c2c640 1 Processor -- start 2024-04-02T13:57:56.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.944+0000 7f78c8c2c640 1 -- start start 2024-04-02T13:57:56.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.945+0000 7f78c8c2c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78c4100a20 0x7f78c41984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:56.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.945+0000 7f78c8c2c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78c41013e0 0x7f78c4198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:56.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.945+0000 7f78c8c2c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78c4106a30 0x7f78c419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:56.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.945+0000 7f78c8c2c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f78c4111f50 con 0x7f78c41013e0 2024-04-02T13:57:56.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.945+0000 7f78c8c2c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f78c4111dd0 con 0x7f78c4100a20 2024-04-02T13:57:56.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.945+0000 7f78c8c2c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f78c41120d0 con 0x7f78c4106a30 2024-04-02T13:57:56.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.945+0000 7f78c1d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78c41013e0 0x7f78c4198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:56.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.945+0000 7f78c1d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78c41013e0 0x7f78c4198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44120/0 (socket says 172.21.15.52:44120) 2024-04-02T13:57:56.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.945+0000 7f78c1d9b640 1 -- 172.21.15.52:0/2907852021 learned_addr learned my addr 172.21.15.52:0/2907852021 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:56.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.945+0000 7f78c259c640 1 --2- 172.21.15.52:0/2907852021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78c4100a20 0x7f78c41984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:56.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.945+0000 7f78c2d9d640 1 --2- 172.21.15.52:0/2907852021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78c4106a30 0x7f78c419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:56.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.945+0000 7f78c1d9b640 1 -- 172.21.15.52:0/2907852021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78c4106a30 msgr2=0x7f78c419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:56.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.945+0000 7f78c1d9b640 1 --2- 172.21.15.52:0/2907852021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78c4106a30 0x7f78c419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.945+0000 7f78c1d9b640 1 -- 172.21.15.52:0/2907852021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78c4100a20 msgr2=0x7f78c41984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:56.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.945+0000 7f78c1d9b640 1 --2- 172.21.15.52:0/2907852021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78c4100a20 0x7f78c41984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.945+0000 7f78c1d9b640 1 -- 172.21.15.52:0/2907852021 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c419d470 con 0x7f78c41013e0 2024-04-02T13:57:56.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.945+0000 7f78c1d9b640 1 --2- 172.21.15.52:0/2907852021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78c41013e0 0x7f78c4198a10 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009230 tx=0x7f78b802cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:56.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.945+0000 7f78b37fe640 1 -- 172.21.15.52:0/2907852021 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8041070 con 0x7f78c41013e0 2024-04-02T13:57:56.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.946+0000 7f78b37fe640 1 -- 172.21.15.52:0/2907852021 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78b8004450 con 0x7f78c41013e0 2024-04-02T13:57:56.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.946+0000 7f78c8c2c640 1 -- 172.21.15.52:0/2907852021 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c419d700 con 0x7f78c41013e0 2024-04-02T13:57:56.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.946+0000 7f78b37fe640 1 -- 172.21.15.52:0/2907852021 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8034440 con 0x7f78c41013e0 2024-04-02T13:57:56.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.946+0000 7f78c8c2c640 1 -- 172.21.15.52:0/2907852021 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c41a4ff0 con 0x7f78c41013e0 2024-04-02T13:57:56.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.948+0000 7f78b37fe640 1 -- 172.21.15.52:0/2907852021 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f78b8044070 con 0x7f78c41013e0 2024-04-02T13:57:56.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.948+0000 7f78cb80e740 1 Processor -- start 2024-04-02T13:57:56.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.949+0000 7f78cb80e740 1 -- start start 2024-04-02T13:57:56.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.949+0000 7f78b37fe640 1 --2- 172.21.15.52:0/2907852021 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f78ac0a3d20 0x7f78ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:56.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.949+0000 7f78cb80e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c76df1d8d0 0x55c76df1dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:56.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.949+0000 7f78cb80e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df1e1f0 0x55c76df26890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:56.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.949+0000 7f78c2d9d640 1 --2- 172.21.15.52:0/2907852021 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f78ac0a3d20 0x7f78ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:56.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.949+0000 7f78cb80e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c76df26dd0 0x55c76df291c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:56.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.949+0000 7f78cb80e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c76de81830 con 0x55c76df1e1f0 2024-04-02T13:57:56.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.949+0000 7f78cb80e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c76de29820 con 0x55c76df1d8d0 2024-04-02T13:57:56.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.949+0000 7f78cb80e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c76de7d7e0 con 0x55c76df26dd0 2024-04-02T13:57:56.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.949+0000 7f78b37fe640 1 -- 172.21.15.52:0/2907852021 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f78b80f2440 con 0x7f78c41013e0 2024-04-02T13:57:56.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.949+0000 7f78c1d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df1e1f0 0x55c76df26890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:56.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.949+0000 7f78c1d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df1e1f0 0x55c76df26890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44122/0 (socket says 172.21.15.52:44122) 2024-04-02T13:57:56.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.949+0000 7f78c1d9b640 1 -- 172.21.15.52:0/3510570492 learned_addr learned my addr 172.21.15.52:0/3510570492 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:56.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.949+0000 7f78c259c640 1 --2- 172.21.15.52:0/3510570492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c76df1d8d0 0x55c76df1dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:56.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.949+0000 7f78c1d9b640 1 -- 172.21.15.52:0/3510570492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c76df26dd0 msgr2=0x55c76df291c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:56.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.949+0000 7f78c1d9b640 1 --2- 172.21.15.52:0/3510570492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c76df26dd0 0x55c76df291c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.949+0000 7f78c1d9b640 1 -- 172.21.15.52:0/3510570492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c76df1d8d0 msgr2=0x55c76df1dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:56.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.949+0000 7f78c1d9b640 1 --2- 172.21.15.52:0/3510570492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c76df1d8d0 0x55c76df1dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.949+0000 7f78c1d9b640 1 -- 172.21.15.52:0/3510570492 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c76dc5f7d0 con 0x55c76df1e1f0 2024-04-02T13:57:56.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.950+0000 7f78c2d9d640 1 --2- 172.21.15.52:0/2907852021 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f78ac0a3d20 0x7f78ac0a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f78bc00a780 tx=0x7f78bc00a950 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:56.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.950+0000 7f78c1d9b640 1 --2- 172.21.15.52:0/3510570492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df1e1f0 0x55c76df26890 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f78b80ba6c0 tx=0x7f78b8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cfca0d3c7536914 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:56.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.950+0000 7f78b0ff9640 1 -- 172.21.15.52:0/3510570492 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b80bdbc0 con 0x55c76df1e1f0 2024-04-02T13:57:56.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.950+0000 7f78b0ff9640 1 -- 172.21.15.52:0/3510570492 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78b8037eb0 con 0x55c76df1e1f0 2024-04-02T13:57:56.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.950+0000 7f78b0ff9640 1 -- 172.21.15.52:0/3510570492 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b800aea0 con 0x55c76df1e1f0 2024-04-02T13:57:56.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.950+0000 7f78cb80e740 1 -- 172.21.15.52:0/3510570492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df1e1f0 msgr2=0x55c76df26890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:56.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.950+0000 7f78cb80e740 1 --2- 172.21.15.52:0/3510570492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df1e1f0 0x55c76df26890 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f78b80ba6c0 tx=0x7f78b8000bc0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.950+0000 7f78cb80e740 1 -- 172.21.15.52:0/3510570492 shutdown_connections 2024-04-02T13:57:56.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.951+0000 7f78cb80e740 1 --2- 172.21.15.52:0/3510570492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c76df26dd0 0x55c76df291c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.951+0000 7f78cb80e740 1 --2- 172.21.15.52:0/3510570492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df1e1f0 0x55c76df26890 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.951+0000 7f78cb80e740 1 --2- 172.21.15.52:0/3510570492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c76df1d8d0 0x55c76df1dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.951+0000 7f78cb80e740 1 -- 172.21.15.52:0/3510570492 >> 172.21.15.52:0/3510570492 conn(0x55c76df18000 msgr2=0x55c76df183e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:56.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.951+0000 7f78cb80e740 1 -- 172.21.15.52:0/3510570492 shutdown_connections 2024-04-02T13:57:56.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.951+0000 7f78cb80e740 1 -- 172.21.15.52:0/3510570492 wait complete. 2024-04-02T13:57:56.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.951+0000 7f78cb80e740 1 -- start start 2024-04-02T13:57:56.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.951+0000 7f78cb80e740 1 Processor -- start 2024-04-02T13:57:56.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.951+0000 7f78cb80e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df1d8d0 0x55c76df26590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:56.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.951+0000 7f78cb80e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c76df1e1f0 0x55c76df23920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:56.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.951+0000 7f78cb80e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c76df26dd0 0x55c76df23e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:56.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.951+0000 7f78cb80e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c76df29980 con 0x55c76df1d8d0 2024-04-02T13:57:56.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.951+0000 7f78cb80e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c76de3cd30 con 0x55c76df26dd0 2024-04-02T13:57:56.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.951+0000 7f78cb80e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c76df29b00 con 0x55c76df1e1f0 2024-04-02T13:57:56.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.951+0000 7f78cb80e740 10 client.0 _ll_register_callbacks cb 0x55c76de3c720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.951+0000 7f78c259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df1d8d0 0x55c76df26590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.951+0000 7f78c259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df1d8d0 0x55c76df26590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44126/0 (socket says 172.21.15.52:44126) 2024-04-02T13:57:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.951+0000 7f78c259c640 1 -- 172.21.15.52:0/3545911563 learned_addr learned my addr 172.21.15.52:0/3545911563 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.952+0000 7f78c1d9b640 1 --2- 172.21.15.52:0/3545911563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c76df1e1f0 0x55c76df23920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.952+0000 7f78c2d9d640 1 --2- 172.21.15.52:0/3545911563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c76df26dd0 0x55c76df23e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.952+0000 7f78c259c640 1 -- 172.21.15.52:0/3545911563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c76df1e1f0 msgr2=0x55c76df23920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.952+0000 7f78c259c640 1 --2- 172.21.15.52:0/3545911563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c76df1e1f0 0x55c76df23920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.952+0000 7f78c259c640 1 -- 172.21.15.52:0/3545911563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c76df26dd0 msgr2=0x55c76df23e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.952+0000 7f78c259c640 1 --2- 172.21.15.52:0/3545911563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c76df26dd0 0x55c76df23e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.952+0000 7f78c259c640 1 -- 172.21.15.52:0/3545911563 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c76db953f0 con 0x55c76df1d8d0 2024-04-02T13:57:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.952+0000 7f78c1d9b640 1 --2- 172.21.15.52:0/3545911563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c76df1e1f0 0x55c76df23920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.952+0000 7f78c2d9d640 1 --2- 172.21.15.52:0/3545911563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c76df26dd0 0x55c76df23e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.952+0000 7f78c259c640 1 --2- 172.21.15.52:0/3545911563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df1d8d0 0x55c76df26590 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f78b400b940 tx=0x7f78b400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b710f76c9bff77 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.952+0000 7f78a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.952+0000 7f78a9ffb640 1 -- 172.21.15.52:0/3545911563 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4003ea0 con 0x55c76df1d8d0 2024-04-02T13:57:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.952+0000 7f78a9ffb640 1 -- 172.21.15.52:0/3545911563 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f78b4011440 con 0x55c76df1d8d0 2024-04-02T13:57:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.952+0000 7f78cb80e740 1 -- 172.21.15.52:0/3545911563 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c76de8ed70 con 0x55c76df1d8d0 2024-04-02T13:57:56.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.952+0000 7f78a9ffb640 1 -- 172.21.15.52:0/3545911563 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f78b4011640 con 0x55c76df1d8d0 2024-04-02T13:57:56.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.952+0000 7f78cb80e740 10 client.16854 fetch_fsmap learned FSMap version 30 2024-04-02T13:57:56.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.952+0000 7f78cb80e740 1 -- 172.21.15.52:0/3545911563 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c76db550f0 con 0x55c76df1d8d0 2024-04-02T13:57:56.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.952+0000 7f78a9ffb640 1 -- 172.21.15.52:0/3545911563 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c76de8ed70 con 0x55c76df1d8d0 2024-04-02T13:57:56.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.953+0000 7f78a9ffb640 1 -- 172.21.15.52:0/3545911563 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f78b401b9d0 con 0x55c76df1d8d0 2024-04-02T13:57:56.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.953+0000 7f78cb80e740 10 client.16854 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:57:56.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.953+0000 7f78cb80e740 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:57:56.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.953+0000 7f78cb80e740 20 client.16854 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:56.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.953+0000 7f78cb80e740 1 --2- 172.21.15.52:0/3545911563 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c76df369e0 0x55c76df38e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:56.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.953+0000 7f78cb80e740 4 client.16854 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:57:56.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.953+0000 7f78cb80e740 1 -- 172.21.15.52:0/3545911563 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c76de890c0 con 0x55c76df369e0 2024-04-02T13:57:56.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.953+0000 7f78c1d9b640 1 --2- 172.21.15.52:0/3545911563 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c76df369e0 0x55c76df38e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:56.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.953+0000 7f78a9ffb640 10 client.16854 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:56.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.953+0000 7f78c1d9b640 1 --2- 172.21.15.52:0/3545911563 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c76df369e0 0x55c76df38e50 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6818a0aab15763ce server_cookie=2ab55f2fa8238156 in_seq=0 out_seq=0 2024-04-02T13:57:56.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.954+0000 7f78a9ffb640 1 -- 172.21.15.52:0/3545911563 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c76de890c0 con 0x55c76df369e0 2024-04-02T13:57:56.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.954+0000 7f78a9ffb640 10 client.16854 handle_command_reply: tid=0 2024-04-02T13:57:56.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.954+0000 7f78cb80e740 1 client.16854 shutdown 2024-04-02T13:57:56.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.954+0000 7f78cb80e740 1 -- 172.21.15.52:0/3545911563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df1d8d0 msgr2=0x55c76df26590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:56.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.954+0000 7f78cb80e740 1 --2- 172.21.15.52:0/3545911563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df1d8d0 0x55c76df26590 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f78b400b940 tx=0x7f78b400be00 comp rx=0 tx=0).stop 2024-04-02T13:57:56.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.954+0000 7f78cb80e740 1 -- 172.21.15.52:0/3545911563 shutdown_connections 2024-04-02T13:57:56.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.954+0000 7f78cb80e740 1 --2- 172.21.15.52:0/3545911563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c76df26dd0 0x55c76df23e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.954+0000 7f78cb80e740 1 --2- 172.21.15.52:0/3545911563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c76df1e1f0 0x55c76df23920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.954+0000 7f78cb80e740 1 --2- 172.21.15.52:0/3545911563 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c76df369e0 0x55c76df38e50 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.954+0000 7f78cb80e740 1 --2- 172.21.15.52:0/3545911563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df1d8d0 0x55c76df26590 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.954+0000 7f78a9ffb640 0 client.16854 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.954+0000 7f78cb80e740 1 -- 172.21.15.52:0/3545911563 >> 172.21.15.52:0/3545911563 conn(0x55c76df18000 msgr2=0x55c76df19d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.954+0000 7f78cb80e740 1 -- 172.21.15.52:0/3545911563 shutdown_connections 2024-04-02T13:57:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.954+0000 7f78cb80e740 1 -- 172.21.15.52:0/3545911563 wait complete. 2024-04-02T13:57:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.954+0000 7f78cb80e740 20 client.16854 trim_cache size 0 max 16384 2024-04-02T13:57:56.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.972+0000 7f78cb80e740 1 Processor -- start 2024-04-02T13:57:56.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.972+0000 7f78cb80e740 1 -- start start 2024-04-02T13:57:56.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.972+0000 7f78cb80e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df1d8d0 0x55c76df32670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:56.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.972+0000 7f78cb80e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c76df1e1f0 0x55c76df170c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:56.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.972+0000 7f78cb80e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c76df26dd0 0x55c76df26050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:56.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.972+0000 7f78cb80e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f78c41aee70 con 0x55c76df1d8d0 2024-04-02T13:57:56.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.972+0000 7f78cb80e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f78c41b09b0 con 0x55c76df26dd0 2024-04-02T13:57:56.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.972+0000 7f78cb80e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f78c41af540 con 0x55c76df1e1f0 2024-04-02T13:57:56.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.972+0000 7f78c259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df1d8d0 0x55c76df32670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:56.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.973+0000 7f78c259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df1d8d0 0x55c76df32670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44132/0 (socket says 172.21.15.52:44132) 2024-04-02T13:57:56.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.973+0000 7f78c259c640 1 -- 172.21.15.52:0/3702369874 learned_addr learned my addr 172.21.15.52:0/3702369874 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:56.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.973+0000 7f78c2d9d640 1 --2- 172.21.15.52:0/3702369874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c76df26dd0 0x55c76df26050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:56.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.973+0000 7f78c259c640 1 -- 172.21.15.52:0/3702369874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c76df1e1f0 msgr2=0x55c76df170c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:56.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.973+0000 7f78c259c640 1 --2- 172.21.15.52:0/3702369874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c76df1e1f0 0x55c76df170c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.973+0000 7f78c259c640 1 -- 172.21.15.52:0/3702369874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c76df26dd0 msgr2=0x55c76df26050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:56.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.973+0000 7f78c259c640 1 --2- 172.21.15.52:0/3702369874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c76df26dd0 0x55c76df26050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.973+0000 7f78c259c640 1 -- 172.21.15.52:0/3702369874 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c76df17e30 con 0x55c76df1d8d0 2024-04-02T13:57:56.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.973+0000 7f78c2d9d640 1 --2- 172.21.15.52:0/3702369874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c76df26dd0 0x55c76df26050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:56.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.973+0000 7f78c259c640 1 --2- 172.21.15.52:0/3702369874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df1d8d0 0x55c76df32670 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f78b401be40 tx=0x7f78b4004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65477143559db67f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:56.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.973+0000 7f78abfff640 1 -- 172.21.15.52:0/3702369874 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4005040 con 0x55c76df1d8d0 2024-04-02T13:57:56.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.973+0000 7f78abfff640 1 -- 172.21.15.52:0/3702369874 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78b402e070 con 0x55c76df1d8d0 2024-04-02T13:57:56.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.973+0000 7f78abfff640 1 -- 172.21.15.52:0/3702369874 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b40048e0 con 0x55c76df1d8d0 2024-04-02T13:57:56.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.974+0000 7f78cb80e740 1 -- 172.21.15.52:0/3702369874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df1d8d0 msgr2=0x55c76df32670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:56.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.974+0000 7f78cb80e740 1 --2- 172.21.15.52:0/3702369874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df1d8d0 0x55c76df32670 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f78b401be40 tx=0x7f78b4004170 comp rx=0 tx=0).stop 2024-04-02T13:57:56.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.974+0000 7f78cb80e740 1 -- 172.21.15.52:0/3702369874 shutdown_connections 2024-04-02T13:57:56.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.974+0000 7f78cb80e740 1 --2- 172.21.15.52:0/3702369874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c76df26dd0 0x55c76df26050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.974+0000 7f78cb80e740 1 --2- 172.21.15.52:0/3702369874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c76df1e1f0 0x55c76df170c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.974+0000 7f78cb80e740 1 --2- 172.21.15.52:0/3702369874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df1d8d0 0x55c76df32670 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.974+0000 7f78cb80e740 1 -- 172.21.15.52:0/3702369874 >> 172.21.15.52:0/3702369874 conn(0x55c76df18000 msgr2=0x55c76df169b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:56.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.974+0000 7f78cb80e740 1 -- 172.21.15.52:0/3702369874 shutdown_connections 2024-04-02T13:57:56.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.974+0000 7f78cb80e740 1 -- 172.21.15.52:0/3702369874 wait complete. 2024-04-02T13:57:56.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.974+0000 7f78cb80e740 1 -- start start 2024-04-02T13:57:56.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.974+0000 7f78cb80e740 1 Processor -- start 2024-04-02T13:57:56.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.974+0000 7f78cb80e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c76df1d8d0 0x55c76df2ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:56.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.974+0000 7f78cb80e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c76df1e1f0 0x55c76df2a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:56.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.974+0000 7f78cb80e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df26dd0 0x55c76df2a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:56.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.974+0000 7f78cb80e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f78c41b02e0 con 0x55c76df26dd0 2024-04-02T13:57:56.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.974+0000 7f78cb80e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f78c41afc10 con 0x55c76df1e1f0 2024-04-02T13:57:56.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.974+0000 7f78cb80e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f78c41ace50 con 0x55c76df1d8d0 2024-04-02T13:57:56.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.974+0000 7f78cb80e740 10 client.0 _ll_register_callbacks cb 0x7f78b4004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:56.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.974+0000 7f78c2d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df26dd0 0x55c76df2a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:56.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.974+0000 7f78c2d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df26dd0 0x55c76df2a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44144/0 (socket says 172.21.15.52:44144) 2024-04-02T13:57:56.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.974+0000 7f78c2d9d640 1 -- 172.21.15.52:0/2614123321 learned_addr learned my addr 172.21.15.52:0/2614123321 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:56.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.975+0000 7f78c1d9b640 1 --2- 172.21.15.52:0/2614123321 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c76df1e1f0 0x55c76df2a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:56.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.975+0000 7f78c259c640 1 --2- 172.21.15.52:0/2614123321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c76df1d8d0 0x55c76df2ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:56.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.975+0000 7f78c2d9d640 1 -- 172.21.15.52:0/2614123321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c76df1d8d0 msgr2=0x55c76df2ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:56.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.975+0000 7f78c2d9d640 1 --2- 172.21.15.52:0/2614123321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c76df1d8d0 0x55c76df2ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.975+0000 7f78c2d9d640 1 -- 172.21.15.52:0/2614123321 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c76df1e1f0 msgr2=0x55c76df2a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:56.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.975+0000 7f78c2d9d640 1 --2- 172.21.15.52:0/2614123321 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c76df1e1f0 0x55c76df2a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.975+0000 7f78c2d9d640 1 -- 172.21.15.52:0/2614123321 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c76df1d070 con 0x55c76df26dd0 2024-04-02T13:57:56.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.975+0000 7f78c1d9b640 1 --2- 172.21.15.52:0/2614123321 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c76df1e1f0 0x55c76df2a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:56.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.975+0000 7f78c2d9d640 1 --2- 172.21.15.52:0/2614123321 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df26dd0 0x55c76df2a640 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f78bc012e80 tx=0x7f78bc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6273436470cf5422 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:56.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.975+0000 7f78a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:56.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.975+0000 7f78a9ffb640 1 -- 172.21.15.52:0/2614123321 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc018070 con 0x55c76df26dd0 2024-04-02T13:57:56.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.975+0000 7f78a9ffb640 1 -- 172.21.15.52:0/2614123321 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f78bc004d20 con 0x55c76df26dd0 2024-04-02T13:57:56.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.975+0000 7f78cb80e740 1 -- 172.21.15.52:0/2614123321 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78b402e070 con 0x55c76df26dd0 2024-04-02T13:57:56.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.975+0000 7f78a9ffb640 1 -- 172.21.15.52:0/2614123321 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f78bc00e680 con 0x55c76df26dd0 2024-04-02T13:57:56.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.975+0000 7f78cb80e740 10 client.16860 fetch_fsmap learned FSMap version 30 2024-04-02T13:57:56.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.975+0000 7f78cb80e740 1 -- 172.21.15.52:0/2614123321 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c76dfa1050 con 0x55c76df26dd0 2024-04-02T13:57:56.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.975+0000 7f78a9ffb640 1 -- 172.21.15.52:0/2614123321 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78b402e070 con 0x55c76df26dd0 2024-04-02T13:57:56.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.976+0000 7f78a9ffb640 1 -- 172.21.15.52:0/2614123321 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f78bc026070 con 0x55c76df26dd0 2024-04-02T13:57:56.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.976+0000 7f78cb80e740 10 client.16860 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:57:56.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.976+0000 7f78cb80e740 10 client.16860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:57:56.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.976+0000 7f78cb80e740 20 client.16860 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:56.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.976+0000 7f78cb80e740 1 --2- 172.21.15.52:0/2614123321 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c76dfa1820 0x55c76dfa3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:56.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.976+0000 7f78cb80e740 4 client.16860 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:57:56.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.976+0000 7f78cb80e740 1 -- 172.21.15.52:0/2614123321 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c76de06560 con 0x55c76dfa1820 2024-04-02T13:57:56.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.976+0000 7f78c259c640 1 --2- 172.21.15.52:0/2614123321 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c76dfa1820 0x55c76dfa3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:56.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.976+0000 7f78a9ffb640 10 client.16860 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:56.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.976+0000 7f78c259c640 1 --2- 172.21.15.52:0/2614123321 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c76dfa1820 0x55c76dfa3c40 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aedf1fc0b9c16fb5 server_cookie=eba3167411a59d5f in_seq=0 out_seq=0 2024-04-02T13:57:56.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.977+0000 7f78a9ffb640 1 -- 172.21.15.52:0/2614123321 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c76de06560 con 0x55c76dfa1820 2024-04-02T13:57:56.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.977+0000 7f78a9ffb640 10 client.16860 handle_command_reply: tid=0 2024-04-02T13:57:56.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.977+0000 7f78cb80e740 1 client.16860 shutdown 2024-04-02T13:57:56.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.977+0000 7f78cb80e740 1 -- 172.21.15.52:0/2614123321 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df26dd0 msgr2=0x55c76df2a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:56.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.977+0000 7f78cb80e740 1 --2- 172.21.15.52:0/2614123321 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df26dd0 0x55c76df2a640 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f78bc012e80 tx=0x7f78bc004ab0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.977+0000 7f78cb80e740 1 -- 172.21.15.52:0/2614123321 shutdown_connections 2024-04-02T13:57:56.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.977+0000 7f78cb80e740 1 --2- 172.21.15.52:0/2614123321 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c76dfa1820 0x55c76dfa3c40 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.977+0000 7f78cb80e740 1 --2- 172.21.15.52:0/2614123321 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c76df26dd0 0x55c76df2a640 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.977+0000 7f78a9ffb640 0 client.16860 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:56.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.977+0000 7f78cb80e740 1 --2- 172.21.15.52:0/2614123321 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c76df1e1f0 0x55c76df2a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.977+0000 7f78cb80e740 1 --2- 172.21.15.52:0/2614123321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c76df1d8d0 0x55c76df2ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.977+0000 7f78cb80e740 1 -- 172.21.15.52:0/2614123321 >> 172.21.15.52:0/2614123321 conn(0x55c76df18000 msgr2=0x55c76df17240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:56.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.977+0000 7f78cb80e740 1 -- 172.21.15.52:0/2614123321 shutdown_connections 2024-04-02T13:57:56.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.977+0000 7f78cb80e740 1 -- 172.21.15.52:0/2614123321 wait complete. 2024-04-02T13:57:56.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.977+0000 7f78cb80e740 20 client.16860 trim_cache size 0 max 16384 2024-04-02T13:57:56.993 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:57:56.994 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-04-02T13:57:56.994 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-04-02T13:57:56.994 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:57:56.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.977+0000 7f78c8c2c640 1 -- 172.21.15.52:0/2907852021 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f78ac0a3d20 msgr2=0x7f78ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:56.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.977+0000 7f78c8c2c640 1 --2- 172.21.15.52:0/2907852021 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f78ac0a3d20 0x7f78ac0a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f78bc00a780 tx=0x7f78bc00a950 comp rx=0 tx=0).stop 2024-04-02T13:57:56.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.977+0000 7f78c8c2c640 1 -- 172.21.15.52:0/2907852021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78c41013e0 msgr2=0x7f78c4198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:56.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.977+0000 7f78c8c2c640 1 --2- 172.21.15.52:0/2907852021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78c41013e0 0x7f78c4198a10 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009230 tx=0x7f78b802cc70 comp rx=0 tx=0).stop 2024-04-02T13:57:56.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.977+0000 7f78c8c2c640 1 -- 172.21.15.52:0/2907852021 shutdown_connections 2024-04-02T13:57:56.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.977+0000 7f78c8c2c640 1 --2- 172.21.15.52:0/2907852021 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f78ac0a3d20 0x7f78ac0a61e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.977+0000 7f78c8c2c640 1 --2- 172.21.15.52:0/2907852021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78c4106a30 0x7f78c419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.977+0000 7f78c8c2c640 1 --2- 172.21.15.52:0/2907852021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78c41013e0 0x7f78c4198a10 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.977+0000 7f78c8c2c640 1 --2- 172.21.15.52:0/2907852021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78c4100a20 0x7f78c41984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:56.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.977+0000 7f78c8c2c640 1 -- 172.21.15.52:0/2907852021 >> 172.21.15.52:0/2907852021 conn(0x7f78c4077b60 msgr2=0x7f78c4103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:56.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.978+0000 7f78c8c2c640 1 -- 172.21.15.52:0/2907852021 shutdown_connections 2024-04-02T13:57:56.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:56.978+0000 7f78c8c2c640 1 -- 172.21.15.52:0/2907852021 wait complete. 2024-04-02T13:57:56.997 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:57:56.997 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:57:57.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.098+0000 7fb4a607b640 1 -- 172.21.15.52:0/3373123020 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4a0106ec0 msgr2=0x7fb4a010ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.098+0000 7fb4a607b640 1 --2- 172.21.15.52:0/3373123020 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4a0106ec0 0x7fb4a010ead0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fb498009ca0 tx=0x7fb49802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.098+0000 7fb4a607b640 1 -- 172.21.15.52:0/3373123020 shutdown_connections 2024-04-02T13:57:57.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.098+0000 7fb4a607b640 1 --2- 172.21.15.52:0/3373123020 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4a0106ec0 0x7fb4a010ead0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.098+0000 7fb4a607b640 1 --2- 172.21.15.52:0/3373123020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4a01010b0 0x7fb4a01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.098+0000 7fb4a607b640 1 --2- 172.21.15.52:0/3373123020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4a0100700 0x7fb4a0100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.098+0000 7fb4a607b640 1 -- 172.21.15.52:0/3373123020 >> 172.21.15.52:0/3373123020 conn(0x7fb4a0075d10 msgr2=0x7fb4a0076120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.098+0000 7fb4a607b640 1 -- 172.21.15.52:0/3373123020 shutdown_connections 2024-04-02T13:57:57.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.098+0000 7fb4a607b640 1 -- 172.21.15.52:0/3373123020 wait complete. 2024-04-02T13:57:57.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.098+0000 7fb4a607b640 1 Processor -- start 2024-04-02T13:57:57.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.098+0000 7fb4a607b640 1 -- start start 2024-04-02T13:57:57.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.098+0000 7fb4a607b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4a0100700 0x7fb4a01983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.098+0000 7fb4a607b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4a0106ec0 0x7fb4a0198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.099+0000 7fb4a607b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4a019ccb0 0x7fb4a019d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.099+0000 7fb4a607b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0111fb0 con 0x7fb4a0106ec0 2024-04-02T13:57:57.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.099+0000 7fb4a607b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0111e30 con 0x7fb4a019ccb0 2024-04-02T13:57:57.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.099+0000 7fb4a607b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0112130 con 0x7fb4a0100700 2024-04-02T13:57:57.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.099+0000 7fb49effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4a0106ec0 0x7fb4a0198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.099+0000 7fb49effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4a0106ec0 0x7fb4a0198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44150/0 (socket says 172.21.15.52:44150) 2024-04-02T13:57:57.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.099+0000 7fb49effd640 1 -- 172.21.15.52:0/3919768972 learned_addr learned my addr 172.21.15.52:0/3919768972 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:57.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.099+0000 7fb49f7fe640 1 --2- 172.21.15.52:0/3919768972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4a0100700 0x7fb4a01983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.099+0000 7fb49ffff640 1 --2- 172.21.15.52:0/3919768972 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4a019ccb0 0x7fb4a019d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.099+0000 7fb49effd640 1 -- 172.21.15.52:0/3919768972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4a0100700 msgr2=0x7fb4a01983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.099+0000 7fb49effd640 1 --2- 172.21.15.52:0/3919768972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4a0100700 0x7fb4a01983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.099+0000 7fb49effd640 1 -- 172.21.15.52:0/3919768972 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4a019ccb0 msgr2=0x7fb4a019d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.099+0000 7fb49effd640 1 --2- 172.21.15.52:0/3919768972 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4a019ccb0 0x7fb4a019d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.099+0000 7fb49effd640 1 -- 172.21.15.52:0/3919768972 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a019d7a0 con 0x7fb4a0106ec0 2024-04-02T13:57:57.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.099+0000 7fb49effd640 1 --2- 172.21.15.52:0/3919768972 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4a0106ec0 0x7fb4a0198930 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb49400c970 tx=0x7fb49400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.099+0000 7fb49cff9640 1 -- 172.21.15.52:0/3919768972 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb494007bb0 con 0x7fb4a0106ec0 2024-04-02T13:57:57.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.099+0000 7fb4a607b640 1 -- 172.21.15.52:0/3919768972 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a019da90 con 0x7fb4a0106ec0 2024-04-02T13:57:57.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.099+0000 7fb49cff9640 1 -- 172.21.15.52:0/3919768972 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb494007d50 con 0x7fb4a0106ec0 2024-04-02T13:57:57.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.099+0000 7fb49cff9640 1 -- 172.21.15.52:0/3919768972 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49400f450 con 0x7fb4a0106ec0 2024-04-02T13:57:57.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.099+0000 7fb4a607b640 1 -- 172.21.15.52:0/3919768972 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a01a5220 con 0x7fb4a0106ec0 2024-04-02T13:57:57.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.100+0000 7fb4a607b640 1 -- 172.21.15.52:0/3919768972 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a0069c30 con 0x7fb4a0106ec0 2024-04-02T13:57:57.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.102+0000 7fb49cff9640 1 -- 172.21.15.52:0/3919768972 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb494004ee0 con 0x7fb4a0106ec0 2024-04-02T13:57:57.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.103+0000 7fb49cff9640 1 --2- 172.21.15.52:0/3919768972 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb4880a3d20 0x7fb4880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.103+0000 7fb49f7fe640 1 --2- 172.21.15.52:0/3919768972 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb4880a3d20 0x7fb4880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.103+0000 7fb49cff9640 1 -- 172.21.15.52:0/3919768972 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb4940d0ba0 con 0x7fb4a0106ec0 2024-04-02T13:57:57.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.103+0000 7fb49f7fe640 1 --2- 172.21.15.52:0/3919768972 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb4880a3d20 0x7fb4880a61e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb49000abe0 tx=0x7fb49000a2d0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.106+0000 7fb49cff9640 1 -- 172.21.15.52:0/3919768972 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb4940d0e20 con 0x7fb4a0106ec0 2024-04-02T13:57:57.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.336+0000 7fb4a607b640 1 -- 172.21.15.52:0/3919768972 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4a0101290 con 0x7fb4a0106ec0 2024-04-02T13:57:57.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:57 smithi052 ceph-mon[32104]: pgmap v136: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:57:57.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.337+0000 7fb49cff9640 1 -- 172.21.15.52:0/3919768972 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11923 (secure 0 0 0) 0x7fb494014030 con 0x7fb4a0106ec0 2024-04-02T13:57:57.341 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:57:57.341 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:57:45.240902+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24467,24457,24490,24475,24503]},"id":1}]} 2024-04-02T13:57:57.341 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-04-02T13:57:57.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.339+0000 7fb4a607b640 1 -- 172.21.15.52:0/3919768972 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb4880a3d20 msgr2=0x7fb4880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.339+0000 7fb4a607b640 1 --2- 172.21.15.52:0/3919768972 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb4880a3d20 0x7fb4880a61e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb49000abe0 tx=0x7fb49000a2d0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.339+0000 7fb4a607b640 1 -- 172.21.15.52:0/3919768972 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4a0106ec0 msgr2=0x7fb4a0198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.339+0000 7fb4a607b640 1 --2- 172.21.15.52:0/3919768972 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4a0106ec0 0x7fb4a0198930 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb49400c970 tx=0x7fb49400cd70 comp rx=0 tx=0).stop 2024-04-02T13:57:57.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.339+0000 7fb4a607b640 1 -- 172.21.15.52:0/3919768972 shutdown_connections 2024-04-02T13:57:57.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.339+0000 7fb4a607b640 1 --2- 172.21.15.52:0/3919768972 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb4880a3d20 0x7fb4880a61e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.339+0000 7fb4a607b640 1 --2- 172.21.15.52:0/3919768972 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4a019ccb0 0x7fb4a019d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.339+0000 7fb4a607b640 1 --2- 172.21.15.52:0/3919768972 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4a0106ec0 0x7fb4a0198930 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.339+0000 7fb4a607b640 1 --2- 172.21.15.52:0/3919768972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4a0100700 0x7fb4a01983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.339+0000 7fb4a607b640 1 -- 172.21.15.52:0/3919768972 >> 172.21.15.52:0/3919768972 conn(0x7fb4a0075d10 msgr2=0x7fb4a00fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.339+0000 7fb4a607b640 1 -- 172.21.15.52:0/3919768972 shutdown_connections 2024-04-02T13:57:57.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.339+0000 7fb4a607b640 1 -- 172.21.15.52:0/3919768972 wait complete. 2024-04-02T13:57:57.353 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:57:57.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:57 smithi088 ceph-mon[36528]: pgmap v136: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:57:57.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.455+0000 7fb28b03a640 1 -- 172.21.15.52:0/2798633173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb284101370 msgr2=0x7fb28410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.455+0000 7fb28b03a640 1 --2- 172.21.15.52:0/2798633173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb284101370 0x7fb28410d880 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009cd0 tx=0x7fb27c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.455+0000 7fb28b03a640 1 -- 172.21.15.52:0/2798633173 shutdown_connections 2024-04-02T13:57:57.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.455+0000 7fb28b03a640 1 --2- 172.21.15.52:0/2798633173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb284101370 0x7fb28410d880 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.455+0000 7fb28b03a640 1 --2- 172.21.15.52:0/2798633173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2841009b0 0x7fb284100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.455+0000 7fb28b03a640 1 --2- 172.21.15.52:0/2798633173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2841069c0 0x7fb284106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.455+0000 7fb28b03a640 1 -- 172.21.15.52:0/2798633173 >> 172.21.15.52:0/2798633173 conn(0x7fb284076d80 msgr2=0x7fb284077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.455+0000 7fb28b03a640 1 -- 172.21.15.52:0/2798633173 shutdown_connections 2024-04-02T13:57:57.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.455+0000 7fb28b03a640 1 -- 172.21.15.52:0/2798633173 wait complete. 2024-04-02T13:57:57.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.456+0000 7fb28b03a640 1 Processor -- start 2024-04-02T13:57:57.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.456+0000 7fb28b03a640 1 -- start start 2024-04-02T13:57:57.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.456+0000 7fb28b03a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2841009b0 0x7fb284198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.456+0000 7fb28b03a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb284101370 0x7fb284198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.456+0000 7fb28b03a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2841069c0 0x7fb28419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.456+0000 7fb28b03a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb284110dd0 con 0x7fb2841069c0 2024-04-02T13:57:57.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.456+0000 7fb28b03a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb284110c50 con 0x7fb284101370 2024-04-02T13:57:57.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.456+0000 7fb28b03a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb284110f50 con 0x7fb2841009b0 2024-04-02T13:57:57.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.456+0000 7fb2895d7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2841069c0 0x7fb28419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.456+0000 7fb2895d7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2841069c0 0x7fb28419cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44180/0 (socket says 172.21.15.52:44180) 2024-04-02T13:57:57.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.456+0000 7fb2895d7640 1 -- 172.21.15.52:0/3891022735 learned_addr learned my addr 172.21.15.52:0/3891022735 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:57.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.456+0000 7fb283fff640 1 --2- 172.21.15.52:0/3891022735 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb284101370 0x7fb284198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.456+0000 7fb2895d7640 1 -- 172.21.15.52:0/3891022735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2841009b0 msgr2=0x7fb284198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.456+0000 7fb288dd6640 1 --2- 172.21.15.52:0/3891022735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2841009b0 0x7fb284198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.456+0000 7fb2895d7640 1 --2- 172.21.15.52:0/3891022735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2841009b0 0x7fb284198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.456+0000 7fb2895d7640 1 -- 172.21.15.52:0/3891022735 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb284101370 msgr2=0x7fb284198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.456+0000 7fb2895d7640 1 --2- 172.21.15.52:0/3891022735 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb284101370 0x7fb284198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.456+0000 7fb2895d7640 1 -- 172.21.15.52:0/3891022735 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb28419d300 con 0x7fb2841069c0 2024-04-02T13:57:57.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.456+0000 7fb2895d7640 1 --2- 172.21.15.52:0/3891022735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2841069c0 0x7fb28419cbd0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb27c0348f0 tx=0x7fb27c032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.457+0000 7fb281ffb640 1 -- 172.21.15.52:0/3891022735 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c041020 con 0x7fb2841069c0 2024-04-02T13:57:57.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.457+0000 7fb281ffb640 1 -- 172.21.15.52:0/3891022735 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb27c034cd0 con 0x7fb2841069c0 2024-04-02T13:57:57.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.457+0000 7fb28b03a640 1 -- 172.21.15.52:0/3891022735 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb28419d590 con 0x7fb2841069c0 2024-04-02T13:57:57.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.457+0000 7fb281ffb640 1 -- 172.21.15.52:0/3891022735 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c039040 con 0x7fb2841069c0 2024-04-02T13:57:57.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.457+0000 7fb28b03a640 1 -- 172.21.15.52:0/3891022735 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2841a4e80 con 0x7fb2841069c0 2024-04-02T13:57:57.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.459+0000 7fb281ffb640 1 -- 172.21.15.52:0/3891022735 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb27c044020 con 0x7fb2841069c0 2024-04-02T13:57:57.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.459+0000 7fb28dc19740 1 Processor -- start 2024-04-02T13:57:57.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.459+0000 7fb28dc19740 1 -- start start 2024-04-02T13:57:57.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.459+0000 7fb28dc19740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15bc7a0 0x5557b15bcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.459+0000 7fb28dc19740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557b15bd0c0 0x5557b15c5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.459+0000 7fb28dc19740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b15c5ca0 0x5557b15c8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.459+0000 7fb28dc19740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5557b15207b0 con 0x5557b15bc7a0 2024-04-02T13:57:57.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.459+0000 7fb28dc19740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5557b14bf570 con 0x5557b15bd0c0 2024-04-02T13:57:57.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.459+0000 7fb28dc19740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5557b151c760 con 0x5557b15c5ca0 2024-04-02T13:57:57.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.460+0000 7fb283fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15bc7a0 0x5557b15bcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.460+0000 7fb281ffb640 1 --2- 172.21.15.52:0/3891022735 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb2740a3da0 0x7fb2740a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.460+0000 7fb283fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15bc7a0 0x5557b15bcb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44182/0 (socket says 172.21.15.52:44182) 2024-04-02T13:57:57.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.460+0000 7fb283fff640 1 -- 172.21.15.52:0/4111044029 learned_addr learned my addr 172.21.15.52:0/4111044029 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:57.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.460+0000 7fb283fff640 1 --2- 172.21.15.52:0/4111044029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b15c5ca0 0x5557b15c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.460+0000 7fb2895d7640 1 --2- 172.21.15.52:0/3891022735 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb2740a3da0 0x7fb2740a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.460+0000 7fb288dd6640 1 --2- 172.21.15.52:0/4111044029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557b15bd0c0 0x5557b15c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.460+0000 7fb281ffb640 1 -- 172.21.15.52:0/3891022735 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb27c0f2910 con 0x7fb2841069c0 2024-04-02T13:57:57.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.460+0000 7fb283fff640 1 -- 172.21.15.52:0/4111044029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b15c5ca0 msgr2=0x5557b15c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.460+0000 7fb283fff640 1 --2- 172.21.15.52:0/4111044029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b15c5ca0 0x5557b15c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.460+0000 7fb283fff640 1 -- 172.21.15.52:0/4111044029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557b15bd0c0 msgr2=0x5557b15c5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.460+0000 7fb283fff640 1 --2- 172.21.15.52:0/4111044029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557b15bd0c0 0x5557b15c5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.460+0000 7fb283fff640 1 -- 172.21.15.52:0/4111044029 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557b12343f0 con 0x5557b15bc7a0 2024-04-02T13:57:57.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.460+0000 7fb283fff640 1 --2- 172.21.15.52:0/4111044029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15bc7a0 0x5557b15bcb80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb27000cfa0 tx=0x7fb27000c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816ed3a2a057a4c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.460+0000 7fb2895d7640 1 --2- 172.21.15.52:0/3891022735 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb2740a3da0 0x7fb2740a6260 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb27c004af0 tx=0x7fb27c0a9040 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.460+0000 7fb26effd640 1 -- 172.21.15.52:0/4111044029 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270018030 con 0x5557b15bc7a0 2024-04-02T13:57:57.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.460+0000 7fb26effd640 1 -- 172.21.15.52:0/4111044029 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb270010ae0 con 0x5557b15bc7a0 2024-04-02T13:57:57.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.460+0000 7fb26effd640 1 -- 172.21.15.52:0/4111044029 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27000fc00 con 0x5557b15bc7a0 2024-04-02T13:57:57.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.461+0000 7fb28dc19740 1 -- 172.21.15.52:0/4111044029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15bc7a0 msgr2=0x5557b15bcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.461+0000 7fb28dc19740 1 --2- 172.21.15.52:0/4111044029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15bc7a0 0x5557b15bcb80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb27000cfa0 tx=0x7fb27000c400 comp rx=0 tx=0).stop 2024-04-02T13:57:57.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.461+0000 7fb28dc19740 1 -- 172.21.15.52:0/4111044029 shutdown_connections 2024-04-02T13:57:57.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.461+0000 7fb28dc19740 1 --2- 172.21.15.52:0/4111044029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b15c5ca0 0x5557b15c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.461+0000 7fb28dc19740 1 --2- 172.21.15.52:0/4111044029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557b15bd0c0 0x5557b15c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.461+0000 7fb28dc19740 1 --2- 172.21.15.52:0/4111044029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15bc7a0 0x5557b15bcb80 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.461+0000 7fb28dc19740 1 -- 172.21.15.52:0/4111044029 >> 172.21.15.52:0/4111044029 conn(0x5557b15b6ed0 msgr2=0x5557b15b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.461+0000 7fb28dc19740 1 -- 172.21.15.52:0/4111044029 shutdown_connections 2024-04-02T13:57:57.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.461+0000 7fb28dc19740 1 -- 172.21.15.52:0/4111044029 wait complete. 2024-04-02T13:57:57.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.461+0000 7fb28dc19740 1 -- start start 2024-04-02T13:57:57.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.461+0000 7fb28dc19740 1 Processor -- start 2024-04-02T13:57:57.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.462+0000 7fb28dc19740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b15bc7a0 0x5557b15c2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.462+0000 7fb28dc19740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557b15bd0c0 0x5557b15c3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.462+0000 7fb28dc19740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15c5ca0 0x5557b15c3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.462+0000 7fb28dc19740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5557b15c8a90 con 0x5557b15c5ca0 2024-04-02T13:57:57.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.462+0000 7fb28dc19740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5557b14dbe80 con 0x5557b15bd0c0 2024-04-02T13:57:57.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.462+0000 7fb28dc19740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5557b15c8c10 con 0x5557b15bc7a0 2024-04-02T13:57:57.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.462+0000 7fb28dc19740 10 client.0 _ll_register_callbacks cb 0x5557b1517aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:57.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.462+0000 7fb283fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15c5ca0 0x5557b15c3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.462+0000 7fb283fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15c5ca0 0x5557b15c3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44196/0 (socket says 172.21.15.52:44196) 2024-04-02T13:57:57.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.462+0000 7fb283fff640 1 -- 172.21.15.52:0/1776771 learned_addr learned my addr 172.21.15.52:0/1776771 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:57.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.462+0000 7fb288dd6640 1 --2- 172.21.15.52:0/1776771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557b15bd0c0 0x5557b15c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.462+0000 7fb283fff640 1 --2- 172.21.15.52:0/1776771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b15bc7a0 0x5557b15c2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.462+0000 7fb283fff640 1 -- 172.21.15.52:0/1776771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b15bc7a0 msgr2=0x5557b15c2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.462+0000 7fb283fff640 1 --2- 172.21.15.52:0/1776771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b15bc7a0 0x5557b15c2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.462+0000 7fb283fff640 1 -- 172.21.15.52:0/1776771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557b15bd0c0 msgr2=0x5557b15c3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.462+0000 7fb283fff640 1 --2- 172.21.15.52:0/1776771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557b15bd0c0 0x5557b15c3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.462+0000 7fb283fff640 1 -- 172.21.15.52:0/1776771 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557b12fe7d0 con 0x5557b15c5ca0 2024-04-02T13:57:57.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.462+0000 7fb288dd6640 1 --2- 172.21.15.52:0/1776771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557b15bd0c0 0x5557b15c3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:57:57.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.463+0000 7fb283fff640 1 --2- 172.21.15.52:0/1776771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15c5ca0 0x5557b15c3580 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb27000cf20 tx=0x7fb27000f7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790be266124b67e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.463+0000 7fb263fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:57.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.463+0000 7fb263fff640 1 -- 172.21.15.52:0/1776771 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270018040 con 0x5557b15c5ca0 2024-04-02T13:57:57.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.463+0000 7fb263fff640 1 -- 172.21.15.52:0/1776771 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb27000fbe0 con 0x5557b15c5ca0 2024-04-02T13:57:57.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.463+0000 7fb28dc19740 1 -- 172.21.15.52:0/1776771 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb270010ae0 con 0x5557b15c5ca0 2024-04-02T13:57:57.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.463+0000 7fb263fff640 1 -- 172.21.15.52:0/1776771 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb270032050 con 0x5557b15c5ca0 2024-04-02T13:57:57.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.463+0000 7fb28dc19740 10 client.16878 fetch_fsmap learned FSMap version 30 2024-04-02T13:57:57.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.463+0000 7fb28dc19740 1 -- 172.21.15.52:0/1776771 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5557b15d4b90 con 0x5557b15c5ca0 2024-04-02T13:57:57.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.463+0000 7fb263fff640 1 -- 172.21.15.52:0/1776771 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb270010ae0 con 0x5557b15c5ca0 2024-04-02T13:57:57.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.463+0000 7fb263fff640 1 -- 172.21.15.52:0/1776771 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb270003710 con 0x5557b15c5ca0 2024-04-02T13:57:57.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.463+0000 7fb28dc19740 10 client.16878 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:57:57.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.463+0000 7fb28dc19740 10 client.16878 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T13:57:57.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.463+0000 7fb28dc19740 20 client.16878 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:57.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.463+0000 7fb28dc19740 1 --2- 172.21.15.52:0/1776771 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5557b15d5ee0 0x5557b15d82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.463+0000 7fb28dc19740 4 client.16878 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:57:57.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.463+0000 7fb28dc19740 1 -- 172.21.15.52:0/1776771 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557b152dcf0 con 0x5557b15d5ee0 2024-04-02T13:57:57.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.464+0000 7fb288dd6640 1 --2- 172.21.15.52:0/1776771 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5557b15d5ee0 0x5557b15d82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.464+0000 7fb263fff640 10 client.16878 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:57:57.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.464+0000 7fb288dd6640 1 --2- 172.21.15.52:0/1776771 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5557b15d5ee0 0x5557b15d82d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e3cf8fb5908d26c7 server_cookie=2afef2f8e4085661 in_seq=0 out_seq=0 2024-04-02T13:57:57.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.465+0000 7fb263fff640 1 -- 172.21.15.52:0/1776771 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5557b152dcf0 con 0x5557b15d5ee0 2024-04-02T13:57:57.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.465+0000 7fb263fff640 10 client.16878 handle_command_reply: tid=0 2024-04-02T13:57:57.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.465+0000 7fb28dc19740 1 client.16878 shutdown 2024-04-02T13:57:57.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.465+0000 7fb28dc19740 1 -- 172.21.15.52:0/1776771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15c5ca0 msgr2=0x5557b15c3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.465+0000 7fb28dc19740 1 --2- 172.21.15.52:0/1776771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15c5ca0 0x5557b15c3580 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb27000cf20 tx=0x7fb27000f7a0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.465+0000 7fb28dc19740 1 -- 172.21.15.52:0/1776771 shutdown_connections 2024-04-02T13:57:57.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.465+0000 7fb28dc19740 1 --2- 172.21.15.52:0/1776771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15c5ca0 0x5557b15c3580 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.465+0000 7fb28dc19740 1 --2- 172.21.15.52:0/1776771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557b15bd0c0 0x5557b15c3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.465+0000 7fb28dc19740 1 --2- 172.21.15.52:0/1776771 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5557b15d5ee0 0x5557b15d82d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.465+0000 7fb28dc19740 1 --2- 172.21.15.52:0/1776771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b15bc7a0 0x5557b15c2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.465+0000 7fb28dc19740 1 -- 172.21.15.52:0/1776771 >> 172.21.15.52:0/1776771 conn(0x5557b15b6ed0 msgr2=0x5557b15b8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.465+0000 7fb263fff640 0 client.16878 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:57:57.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.465+0000 7fb28dc19740 1 -- 172.21.15.52:0/1776771 shutdown_connections 2024-04-02T13:57:57.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.465+0000 7fb28dc19740 1 -- 172.21.15.52:0/1776771 wait complete. 2024-04-02T13:57:57.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.465+0000 7fb28dc19740 20 client.16878 trim_cache size 0 max 16384 2024-04-02T13:57:57.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.479+0000 7fb28dc19740 1 Processor -- start 2024-04-02T13:57:57.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.479+0000 7fb28dc19740 1 -- start start 2024-04-02T13:57:57.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.479+0000 7fb28dc19740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15bc7a0 0x5557b15c5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.479+0000 7fb28dc19740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b15bd0c0 0x5557b15d1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.479+0000 7fb28dc19740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557b15c5ca0 0x5557b15c1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.479+0000 7fb28dc19740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb2841aed10 con 0x5557b15bc7a0 2024-04-02T13:57:57.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.479+0000 7fb28dc19740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb2841b0850 con 0x5557b15c5ca0 2024-04-02T13:57:57.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.479+0000 7fb28dc19740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb2841af3e0 con 0x5557b15bd0c0 2024-04-02T13:57:57.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.479+0000 7fb283fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15bc7a0 0x5557b15c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.479+0000 7fb283fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15bc7a0 0x5557b15c5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44200/0 (socket says 172.21.15.52:44200) 2024-04-02T13:57:57.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.479+0000 7fb283fff640 1 -- 172.21.15.52:0/97597305 learned_addr learned my addr 172.21.15.52:0/97597305 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:57.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.479+0000 7fb283fff640 1 -- 172.21.15.52:0/97597305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b15bd0c0 msgr2=0x5557b15d1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.479+0000 7fb283fff640 1 --2- 172.21.15.52:0/97597305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b15bd0c0 0x5557b15d1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.479+0000 7fb283fff640 1 -- 172.21.15.52:0/97597305 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557b15c5ca0 msgr2=0x5557b15c1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.479+0000 7fb283fff640 1 --2- 172.21.15.52:0/97597305 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557b15c5ca0 0x5557b15c1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.479+0000 7fb283fff640 1 -- 172.21.15.52:0/97597305 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557b15bcef0 con 0x5557b15bc7a0 2024-04-02T13:57:57.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.479+0000 7fb283fff640 1 --2- 172.21.15.52:0/97597305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15bc7a0 0x5557b15c5230 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fb270030040 tx=0x7fb27000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4709f39ca08ad034 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.480+0000 7fb26e7fc640 1 -- 172.21.15.52:0/97597305 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270021070 con 0x5557b15bc7a0 2024-04-02T13:57:57.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.480+0000 7fb26e7fc640 1 -- 172.21.15.52:0/97597305 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb270010600 con 0x5557b15bc7a0 2024-04-02T13:57:57.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.480+0000 7fb26e7fc640 1 -- 172.21.15.52:0/97597305 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270018040 con 0x5557b15bc7a0 2024-04-02T13:57:57.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.480+0000 7fb28dc19740 1 -- 172.21.15.52:0/97597305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15bc7a0 msgr2=0x5557b15c5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.480+0000 7fb28dc19740 1 --2- 172.21.15.52:0/97597305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15bc7a0 0x5557b15c5230 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fb270030040 tx=0x7fb27000f690 comp rx=0 tx=0).stop 2024-04-02T13:57:57.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.480+0000 7fb28dc19740 1 -- 172.21.15.52:0/97597305 shutdown_connections 2024-04-02T13:57:57.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.480+0000 7fb28dc19740 1 --2- 172.21.15.52:0/97597305 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557b15c5ca0 0x5557b15c1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.480+0000 7fb28dc19740 1 --2- 172.21.15.52:0/97597305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b15bd0c0 0x5557b15d1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.480+0000 7fb28dc19740 1 --2- 172.21.15.52:0/97597305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15bc7a0 0x5557b15c5230 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.480+0000 7fb28dc19740 1 -- 172.21.15.52:0/97597305 >> 172.21.15.52:0/97597305 conn(0x5557b15b6ed0 msgr2=0x5557b15c13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.480+0000 7fb28dc19740 1 -- 172.21.15.52:0/97597305 shutdown_connections 2024-04-02T13:57:57.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.480+0000 7fb28dc19740 1 -- 172.21.15.52:0/97597305 wait complete. 2024-04-02T13:57:57.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.480+0000 7fb28dc19740 1 -- start start 2024-04-02T13:57:57.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.481+0000 7fb28dc19740 1 Processor -- start 2024-04-02T13:57:57.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.481+0000 7fb28dc19740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557b15bc7a0 0x5557b15c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.481+0000 7fb28dc19740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b15bd0c0 0x5557b15c9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.481+0000 7fb28dc19740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15c5ca0 0x5557b15c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.481+0000 7fb28dc19740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb2841b0180 con 0x5557b15c5ca0 2024-04-02T13:57:57.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.481+0000 7fb28dc19740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb2841afab0 con 0x5557b15bc7a0 2024-04-02T13:57:57.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.481+0000 7fb28dc19740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb2841accf0 con 0x5557b15bd0c0 2024-04-02T13:57:57.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.481+0000 7fb28dc19740 10 client.0 _ll_register_callbacks cb 0x7fb2700123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:57.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.481+0000 7fb283fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15c5ca0 0x5557b15c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.481+0000 7fb283fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15c5ca0 0x5557b15c98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44214/0 (socket says 172.21.15.52:44214) 2024-04-02T13:57:57.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.481+0000 7fb283fff640 1 -- 172.21.15.52:0/573426978 learned_addr learned my addr 172.21.15.52:0/573426978 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:57.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.481+0000 7fb283fff640 1 --2- 172.21.15.52:0/573426978 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557b15bc7a0 0x5557b15c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.481+0000 7fb288dd6640 1 --2- 172.21.15.52:0/573426978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b15bd0c0 0x5557b15c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.481+0000 7fb283fff640 1 -- 172.21.15.52:0/573426978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b15bd0c0 msgr2=0x5557b15c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.481+0000 7fb283fff640 1 --2- 172.21.15.52:0/573426978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b15bd0c0 0x5557b15c9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.481+0000 7fb283fff640 1 -- 172.21.15.52:0/573426978 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557b15bc7a0 msgr2=0x5557b15c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.481+0000 7fb283fff640 1 --2- 172.21.15.52:0/573426978 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557b15bc7a0 0x5557b15c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.481+0000 7fb283fff640 1 -- 172.21.15.52:0/573426978 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557b163a3b0 con 0x5557b15c5ca0 2024-04-02T13:57:57.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.482+0000 7fb283fff640 1 --2- 172.21.15.52:0/573426978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15c5ca0 0x5557b15c98d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fb27000c710 tx=0x7fb27001d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17296c3ee49aa75 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.482+0000 7fb263fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:57.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.482+0000 7fb263fff640 1 -- 172.21.15.52:0/573426978 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270021040 con 0x5557b15c5ca0 2024-04-02T13:57:57.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.482+0000 7fb263fff640 1 -- 172.21.15.52:0/573426978 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb2700109b0 con 0x5557b15c5ca0 2024-04-02T13:57:57.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.482+0000 7fb28dc19740 1 -- 172.21.15.52:0/573426978 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb270010600 con 0x5557b15c5ca0 2024-04-02T13:57:57.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.482+0000 7fb263fff640 1 -- 172.21.15.52:0/573426978 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb27000fcb0 con 0x5557b15c5ca0 2024-04-02T13:57:57.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.482+0000 7fb28dc19740 10 client.16884 fetch_fsmap learned FSMap version 30 2024-04-02T13:57:57.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.482+0000 7fb28dc19740 1 -- 172.21.15.52:0/573426978 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5557b163b650 con 0x5557b15c5ca0 2024-04-02T13:57:57.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.482+0000 7fb263fff640 1 -- 172.21.15.52:0/573426978 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb270010600 con 0x5557b15c5ca0 2024-04-02T13:57:57.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.482+0000 7fb263fff640 1 -- 172.21.15.52:0/573426978 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb27001d550 con 0x5557b15c5ca0 2024-04-02T13:57:57.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.482+0000 7fb28dc19740 10 client.16884 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:57:57.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.482+0000 7fb28dc19740 10 client.16884 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T13:57:57.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.482+0000 7fb28dc19740 20 client.16884 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:57.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.482+0000 7fb28dc19740 1 --2- 172.21.15.52:0/573426978 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5557b163bc80 0x5557b163e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.482+0000 7fb28dc19740 4 client.16884 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:57:57.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.482+0000 7fb28dc19740 1 -- 172.21.15.52:0/573426978 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557b1528040 con 0x5557b163bc80 2024-04-02T13:57:57.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.482+0000 7fb288dd6640 1 --2- 172.21.15.52:0/573426978 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5557b163bc80 0x5557b163e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.483+0000 7fb288dd6640 1 --2- 172.21.15.52:0/573426978 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5557b163bc80 0x5557b163e0a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da520baa17b30f38 server_cookie=15ea3d4533109eb9 in_seq=0 out_seq=0 2024-04-02T13:57:57.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.483+0000 7fb263fff640 10 client.16884 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:57:57.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.483+0000 7fb263fff640 1 -- 172.21.15.52:0/573426978 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557b1528040 con 0x5557b163bc80 2024-04-02T13:57:57.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.483+0000 7fb263fff640 10 client.16884 handle_command_reply: tid=0 2024-04-02T13:57:57.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.483+0000 7fb28dc19740 1 client.16884 shutdown 2024-04-02T13:57:57.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.483+0000 7fb28dc19740 1 -- 172.21.15.52:0/573426978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15c5ca0 msgr2=0x5557b15c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.483+0000 7fb28dc19740 1 --2- 172.21.15.52:0/573426978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15c5ca0 0x5557b15c98d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fb27000c710 tx=0x7fb27001d460 comp rx=0 tx=0).stop 2024-04-02T13:57:57.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.483+0000 7fb28dc19740 1 -- 172.21.15.52:0/573426978 shutdown_connections 2024-04-02T13:57:57.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.483+0000 7fb28dc19740 1 --2- 172.21.15.52:0/573426978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557b15c5ca0 0x5557b15c98d0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.483+0000 7fb28dc19740 1 --2- 172.21.15.52:0/573426978 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x5557b163bc80 0x5557b163e0a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.483+0000 7fb28dc19740 1 --2- 172.21.15.52:0/573426978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b15bd0c0 0x5557b15c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.483+0000 7fb263fff640 0 client.16884 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:57:57.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.483+0000 7fb28dc19740 1 --2- 172.21.15.52:0/573426978 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557b15bc7a0 0x5557b15c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.483+0000 7fb28dc19740 1 -- 172.21.15.52:0/573426978 >> 172.21.15.52:0/573426978 conn(0x5557b15b6ed0 msgr2=0x5557b15c2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.483+0000 7fb28dc19740 1 -- 172.21.15.52:0/573426978 shutdown_connections 2024-04-02T13:57:57.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.483+0000 7fb28dc19740 1 -- 172.21.15.52:0/573426978 wait complete. 2024-04-02T13:57:57.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.483+0000 7fb28dc19740 20 client.16884 trim_cache size 0 max 16384 2024-04-02T13:57:57.505 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:57:57.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.484+0000 7fb28b03a640 1 -- 172.21.15.52:0/3891022735 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb2740a3da0 msgr2=0x7fb2740a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.484+0000 7fb28b03a640 1 --2- 172.21.15.52:0/3891022735 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb2740a3da0 0x7fb2740a6260 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb27c004af0 tx=0x7fb27c0a9040 comp rx=0 tx=0).stop 2024-04-02T13:57:57.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.484+0000 7fb28b03a640 1 -- 172.21.15.52:0/3891022735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2841069c0 msgr2=0x7fb28419cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.484+0000 7fb28b03a640 1 --2- 172.21.15.52:0/3891022735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2841069c0 0x7fb28419cbd0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb27c0348f0 tx=0x7fb27c032ec0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.484+0000 7fb28b03a640 1 -- 172.21.15.52:0/3891022735 shutdown_connections 2024-04-02T13:57:57.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.484+0000 7fb28b03a640 1 --2- 172.21.15.52:0/3891022735 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb2740a3da0 0x7fb2740a6260 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.484+0000 7fb28b03a640 1 --2- 172.21.15.52:0/3891022735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2841069c0 0x7fb28419cbd0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.484+0000 7fb28b03a640 1 --2- 172.21.15.52:0/3891022735 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb284101370 0x7fb284198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.484+0000 7fb28b03a640 1 --2- 172.21.15.52:0/3891022735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2841009b0 0x7fb284198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.484+0000 7fb28b03a640 1 -- 172.21.15.52:0/3891022735 >> 172.21.15.52:0/3891022735 conn(0x7fb284076d80 msgr2=0x7fb28406a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.484+0000 7fb28b03a640 1 -- 172.21.15.52:0/3891022735 shutdown_connections 2024-04-02T13:57:57.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.484+0000 7fb28b03a640 1 -- 172.21.15.52:0/3891022735 wait complete. 2024-04-02T13:57:57.508 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:57:57.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:57 smithi189 ceph-mon[36683]: pgmap v136: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:57:57.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.610+0000 7fb42fd74640 1 -- 172.21.15.52:0/2613937783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4281069a0 msgr2=0x7fb428106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.610+0000 7fb42fd74640 1 --2- 172.21.15.52:0/2613937783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4281069a0 0x7fb428106d80 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fb424009930 tx=0x7fb42402bc30 comp rx=0 tx=0).stop 2024-04-02T13:57:57.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.610+0000 7fb42fd74640 1 -- 172.21.15.52:0/2613937783 shutdown_connections 2024-04-02T13:57:57.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.610+0000 7fb42fd74640 1 --2- 172.21.15.52:0/2613937783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb428101350 0x7fb42810d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.610+0000 7fb42fd74640 1 --2- 172.21.15.52:0/2613937783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb428100990 0x7fb428100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.610+0000 7fb42fd74640 1 --2- 172.21.15.52:0/2613937783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4281069a0 0x7fb428106d80 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.610+0000 7fb42fd74640 1 -- 172.21.15.52:0/2613937783 >> 172.21.15.52:0/2613937783 conn(0x7fb428077ad0 msgr2=0x7fb428077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.610+0000 7fb42fd74640 1 -- 172.21.15.52:0/2613937783 shutdown_connections 2024-04-02T13:57:57.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.610+0000 7fb42fd74640 1 -- 172.21.15.52:0/2613937783 wait complete. 2024-04-02T13:57:57.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.611+0000 7fb42fd74640 1 Processor -- start 2024-04-02T13:57:57.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.611+0000 7fb42fd74640 1 -- start start 2024-04-02T13:57:57.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.611+0000 7fb42fd74640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb428100990 0x7fb4281983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.611+0000 7fb42fd74640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb428101350 0x7fb4281988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.611+0000 7fb42fd74640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4281069a0 0x7fb42819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.611+0000 7fb42fd74640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb428111eb0 con 0x7fb4281069a0 2024-04-02T13:57:57.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.611+0000 7fb42fd74640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb428111d30 con 0x7fb428100990 2024-04-02T13:57:57.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.611+0000 7fb42fd74640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb428112030 con 0x7fb428101350 2024-04-02T13:57:57.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.611+0000 7fb42e311640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4281069a0 0x7fb42819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.611+0000 7fb42e311640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4281069a0 0x7fb42819cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44240/0 (socket says 172.21.15.52:44240) 2024-04-02T13:57:57.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.611+0000 7fb42e311640 1 -- 172.21.15.52:0/3179279546 learned_addr learned my addr 172.21.15.52:0/3179279546 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:57.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.611+0000 7fb42db10640 1 --2- 172.21.15.52:0/3179279546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb428100990 0x7fb4281983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.611+0000 7fb42d30f640 1 --2- 172.21.15.52:0/3179279546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb428101350 0x7fb4281988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.611+0000 7fb42e311640 1 -- 172.21.15.52:0/3179279546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb428101350 msgr2=0x7fb4281988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.611+0000 7fb42e311640 1 --2- 172.21.15.52:0/3179279546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb428101350 0x7fb4281988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.611+0000 7fb42e311640 1 -- 172.21.15.52:0/3179279546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb428100990 msgr2=0x7fb4281983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.611+0000 7fb42e311640 1 --2- 172.21.15.52:0/3179279546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb428100990 0x7fb4281983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.611+0000 7fb42e311640 1 -- 172.21.15.52:0/3179279546 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42819d3a0 con 0x7fb4281069a0 2024-04-02T13:57:57.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.611+0000 7fb42d30f640 1 --2- 172.21.15.52:0/3179279546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb428101350 0x7fb4281988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:57.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.611+0000 7fb42e311640 1 --2- 172.21.15.52:0/3179279546 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4281069a0 0x7fb42819cc70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fb42000eaa0 tx=0x7fb42000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.612+0000 7fb41effd640 1 -- 172.21.15.52:0/3179279546 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42000cdb0 con 0x7fb4281069a0 2024-04-02T13:57:57.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.612+0000 7fb41effd640 1 -- 172.21.15.52:0/3179279546 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb420005020 con 0x7fb4281069a0 2024-04-02T13:57:57.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.612+0000 7fb42fd74640 1 -- 172.21.15.52:0/3179279546 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb42819d690 con 0x7fb4281069a0 2024-04-02T13:57:57.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.612+0000 7fb41effd640 1 -- 172.21.15.52:0/3179279546 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4200116b0 con 0x7fb4281069a0 2024-04-02T13:57:57.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.612+0000 7fb42fd74640 1 -- 172.21.15.52:0/3179279546 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4281a4f20 con 0x7fb4281069a0 2024-04-02T13:57:57.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.614+0000 7fb41effd640 1 -- 172.21.15.52:0/3179279546 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb420005560 con 0x7fb4281069a0 2024-04-02T13:57:57.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.614+0000 7fb432a35740 1 Processor -- start 2024-04-02T13:57:57.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.614+0000 7fb432a35740 1 -- start start 2024-04-02T13:57:57.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.615+0000 7fb432a35740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a13896f7a0 0x55a13896fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.615+0000 7fb432a35740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1389700c0 0x55a138978760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.615+0000 7fb432a35740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a138978ca0 0x55a13897b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.615+0000 7fb432a35740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a1388d37b0 con 0x55a13896f7a0 2024-04-02T13:57:57.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.615+0000 7fb432a35740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a138872570 con 0x55a138978ca0 2024-04-02T13:57:57.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.615+0000 7fb432a35740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a1388cf760 con 0x55a1389700c0 2024-04-02T13:57:57.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.615+0000 7fb41effd640 1 --2- 172.21.15.52:0/3179279546 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb4140a3da0 0x7fb4140a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.615+0000 7fb42d30f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a13896f7a0 0x55a13896fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.615+0000 7fb42d30f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a13896f7a0 0x55a13896fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44250/0 (socket says 172.21.15.52:44250) 2024-04-02T13:57:57.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.615+0000 7fb42d30f640 1 -- 172.21.15.52:0/1647223376 learned_addr learned my addr 172.21.15.52:0/1647223376 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:57.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.615+0000 7fb42e311640 1 --2- 172.21.15.52:0/3179279546 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb4140a3da0 0x7fb4140a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.615+0000 7fb41effd640 1 -- 172.21.15.52:0/3179279546 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb4200d0d10 con 0x7fb4281069a0 2024-04-02T13:57:57.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.615+0000 7fb42db10640 1 --2- 172.21.15.52:0/1647223376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1389700c0 0x55a138978760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.615+0000 7fb42d30f640 1 --2- 172.21.15.52:0/1647223376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a138978ca0 0x55a13897b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.615+0000 7fb42d30f640 1 -- 172.21.15.52:0/1647223376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1389700c0 msgr2=0x55a138978760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.615+0000 7fb42d30f640 1 --2- 172.21.15.52:0/1647223376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1389700c0 0x55a138978760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.615+0000 7fb42d30f640 1 -- 172.21.15.52:0/1647223376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a138978ca0 msgr2=0x55a13897b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.615+0000 7fb42d30f640 1 --2- 172.21.15.52:0/1647223376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a138978ca0 0x55a13897b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.615+0000 7fb42d30f640 1 -- 172.21.15.52:0/1647223376 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1385e73f0 con 0x55a13896f7a0 2024-04-02T13:57:57.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.615+0000 7fb42e311640 1 --2- 172.21.15.52:0/3179279546 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb4140a3da0 0x7fb4140a6260 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb4200ce1d0 tx=0x7fb420053040 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.615+0000 7fb42d30f640 1 --2- 172.21.15.52:0/1647223376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a13896f7a0 0x55a13896fb80 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb418005fd0 tx=0x7fb418009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8aeb7006a2985fc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.615+0000 7fb413fff640 1 -- 172.21.15.52:0/1647223376 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4180150f0 con 0x55a13896f7a0 2024-04-02T13:57:57.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.616+0000 7fb413fff640 1 -- 172.21.15.52:0/1647223376 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb418005500 con 0x55a13896f7a0 2024-04-02T13:57:57.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.616+0000 7fb413fff640 1 -- 172.21.15.52:0/1647223376 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4180104d0 con 0x55a13896f7a0 2024-04-02T13:57:57.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.616+0000 7fb432a35740 1 -- 172.21.15.52:0/1647223376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a13896f7a0 msgr2=0x55a13896fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.616+0000 7fb432a35740 1 --2- 172.21.15.52:0/1647223376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a13896f7a0 0x55a13896fb80 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb418005fd0 tx=0x7fb418009670 comp rx=0 tx=0).stop 2024-04-02T13:57:57.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.616+0000 7fb432a35740 1 -- 172.21.15.52:0/1647223376 shutdown_connections 2024-04-02T13:57:57.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.616+0000 7fb432a35740 1 --2- 172.21.15.52:0/1647223376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a138978ca0 0x55a13897b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.616+0000 7fb432a35740 1 --2- 172.21.15.52:0/1647223376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1389700c0 0x55a138978760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.616+0000 7fb432a35740 1 --2- 172.21.15.52:0/1647223376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a13896f7a0 0x55a13896fb80 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.616+0000 7fb432a35740 1 -- 172.21.15.52:0/1647223376 >> 172.21.15.52:0/1647223376 conn(0x55a138969ed0 msgr2=0x55a13896a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.616+0000 7fb432a35740 1 -- 172.21.15.52:0/1647223376 shutdown_connections 2024-04-02T13:57:57.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.616+0000 7fb432a35740 1 -- 172.21.15.52:0/1647223376 wait complete. 2024-04-02T13:57:57.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.617+0000 7fb432a35740 1 -- start start 2024-04-02T13:57:57.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.617+0000 7fb432a35740 1 Processor -- start 2024-04-02T13:57:57.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.617+0000 7fb432a35740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a13896f7a0 0x55a138978760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.617+0000 7fb432a35740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a1389700c0 0x55a138975af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.617+0000 7fb432a35740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a138978ca0 0x55a138976230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.617+0000 7fb432a35740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a13897ba30 con 0x55a1389700c0 2024-04-02T13:57:57.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.617+0000 7fb432a35740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a13888ee80 con 0x55a138978ca0 2024-04-02T13:57:57.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.617+0000 7fb432a35740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a13897bbb0 con 0x55a13896f7a0 2024-04-02T13:57:57.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.617+0000 7fb432a35740 10 client.0 _ll_register_callbacks cb 0x55a1388caaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:57.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.617+0000 7fb42db10640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a1389700c0 0x55a138975af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.617+0000 7fb42db10640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a1389700c0 0x55a138975af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44256/0 (socket says 172.21.15.52:44256) 2024-04-02T13:57:57.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.617+0000 7fb42db10640 1 -- 172.21.15.52:0/415789657 learned_addr learned my addr 172.21.15.52:0/415789657 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:57.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.617+0000 7fb42d30f640 1 --2- 172.21.15.52:0/415789657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a138978ca0 0x55a138976230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.617+0000 7fb42d30f640 1 --2- 172.21.15.52:0/415789657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a13896f7a0 0x55a138978760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.617+0000 7fb42db10640 1 -- 172.21.15.52:0/415789657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a13896f7a0 msgr2=0x55a138978760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.618+0000 7fb42db10640 1 --2- 172.21.15.52:0/415789657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a13896f7a0 0x55a138978760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.618+0000 7fb42db10640 1 -- 172.21.15.52:0/415789657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a138978ca0 msgr2=0x55a138976230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.618+0000 7fb42db10640 1 --2- 172.21.15.52:0/415789657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a138978ca0 0x55a138976230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.618+0000 7fb42db10640 1 -- 172.21.15.52:0/415789657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1386b17d0 con 0x55a1389700c0 2024-04-02T13:57:57.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.618+0000 7fb42d30f640 1 --2- 172.21.15.52:0/415789657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a13896f7a0 0x55a138978760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:57.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.618+0000 7fb42db10640 1 --2- 172.21.15.52:0/415789657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a1389700c0 0x55a138975af0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fb424005ab0 tx=0x7fb424005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638919c7edd43882 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.618+0000 7fb4117fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:57.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.618+0000 7fb4117fa640 1 -- 172.21.15.52:0/415789657 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424005230 con 0x55a1389700c0 2024-04-02T13:57:57.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.618+0000 7fb4117fa640 1 -- 172.21.15.52:0/415789657 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb4240053d0 con 0x55a1389700c0 2024-04-02T13:57:57.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.618+0000 7fb432a35740 1 -- 172.21.15.52:0/415789657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb418005500 con 0x55a1389700c0 2024-04-02T13:57:57.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.618+0000 7fb4117fa640 1 -- 172.21.15.52:0/415789657 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb424004490 con 0x55a1389700c0 2024-04-02T13:57:57.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.618+0000 7fb432a35740 10 client.16896 fetch_fsmap learned FSMap version 30 2024-04-02T13:57:57.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.618+0000 7fb432a35740 1 -- 172.21.15.52:0/415789657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a138987b10 con 0x55a1389700c0 2024-04-02T13:57:57.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.618+0000 7fb4117fa640 1 -- 172.21.15.52:0/415789657 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb418005500 con 0x55a1389700c0 2024-04-02T13:57:57.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.618+0000 7fb4117fa640 1 -- 172.21.15.52:0/415789657 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb4240039c0 con 0x55a1389700c0 2024-04-02T13:57:57.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.618+0000 7fb432a35740 10 client.16896 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:57:57.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.618+0000 7fb432a35740 10 client.16896 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T13:57:57.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.618+0000 7fb432a35740 20 client.16896 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:57.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.618+0000 7fb432a35740 1 --2- 172.21.15.52:0/415789657 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55a138989070 0x55a13898b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.618+0000 7fb432a35740 4 client.16896 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:57:57.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.618+0000 7fb432a35740 1 -- 172.21.15.52:0/415789657 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1388e0cf0 con 0x55a138989070 2024-04-02T13:57:57.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.619+0000 7fb42d30f640 1 --2- 172.21.15.52:0/415789657 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55a138989070 0x55a13898b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.619+0000 7fb4117fa640 10 client.16896 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:57:57.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.619+0000 7fb42d30f640 1 --2- 172.21.15.52:0/415789657 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55a138989070 0x55a13898b4b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fba257afcfd03d20 server_cookie=316ddefe82df6711 in_seq=0 out_seq=0 2024-04-02T13:57:57.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.620+0000 7fb4117fa640 1 -- 172.21.15.52:0/415789657 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a1388e0cf0 con 0x55a138989070 2024-04-02T13:57:57.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.620+0000 7fb4117fa640 10 client.16896 handle_command_reply: tid=0 2024-04-02T13:57:57.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.620+0000 7fb432a35740 1 client.16896 shutdown 2024-04-02T13:57:57.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.620+0000 7fb432a35740 1 -- 172.21.15.52:0/415789657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a1389700c0 msgr2=0x55a138975af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.620+0000 7fb432a35740 1 --2- 172.21.15.52:0/415789657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a1389700c0 0x55a138975af0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fb424005ab0 tx=0x7fb424005e20 comp rx=0 tx=0).stop 2024-04-02T13:57:57.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.620+0000 7fb432a35740 1 -- 172.21.15.52:0/415789657 shutdown_connections 2024-04-02T13:57:57.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.620+0000 7fb432a35740 1 --2- 172.21.15.52:0/415789657 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55a138989070 0x55a13898b4b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.620+0000 7fb432a35740 1 --2- 172.21.15.52:0/415789657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a138978ca0 0x55a138976230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.620+0000 7fb432a35740 1 --2- 172.21.15.52:0/415789657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a1389700c0 0x55a138975af0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.620+0000 7fb4117fa640 0 client.16896 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:57:57.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.620+0000 7fb432a35740 1 --2- 172.21.15.52:0/415789657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a13896f7a0 0x55a138978760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.620+0000 7fb432a35740 1 -- 172.21.15.52:0/415789657 >> 172.21.15.52:0/415789657 conn(0x55a138969ed0 msgr2=0x55a13896bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.620+0000 7fb432a35740 1 -- 172.21.15.52:0/415789657 shutdown_connections 2024-04-02T13:57:57.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.620+0000 7fb432a35740 1 -- 172.21.15.52:0/415789657 wait complete. 2024-04-02T13:57:57.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.620+0000 7fb432a35740 20 client.16896 trim_cache size 0 max 16384 2024-04-02T13:57:57.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.634+0000 7fb432a35740 1 Processor -- start 2024-04-02T13:57:57.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.634+0000 7fb432a35740 1 -- start start 2024-04-02T13:57:57.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.634+0000 7fb432a35740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a13896f7a0 0x55a13898b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.634+0000 7fb432a35740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1389700c0 0x55a138984920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.634+0000 7fb432a35740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a138978ca0 0x55a138978220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.634+0000 7fb432a35740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb4281aee40 con 0x55a138978ca0 2024-04-02T13:57:57.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.634+0000 7fb432a35740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb4281b0980 con 0x55a13896f7a0 2024-04-02T13:57:57.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.634+0000 7fb432a35740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb4281af510 con 0x55a1389700c0 2024-04-02T13:57:57.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.634+0000 7fb42d30f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a138978ca0 0x55a138978220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.634+0000 7fb42d30f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a138978ca0 0x55a138978220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44262/0 (socket says 172.21.15.52:44262) 2024-04-02T13:57:57.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.634+0000 7fb42d30f640 1 -- 172.21.15.52:0/1617666153 learned_addr learned my addr 172.21.15.52:0/1617666153 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:57.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.634+0000 7fb42d30f640 1 --2- 172.21.15.52:0/1617666153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a13896f7a0 0x55a13898b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.634+0000 7fb42db10640 1 --2- 172.21.15.52:0/1617666153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1389700c0 0x55a138984920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.634+0000 7fb42d30f640 1 -- 172.21.15.52:0/1617666153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1389700c0 msgr2=0x55a138984920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.634+0000 7fb42d30f640 1 --2- 172.21.15.52:0/1617666153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1389700c0 0x55a138984920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.634+0000 7fb42d30f640 1 -- 172.21.15.52:0/1617666153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a13896f7a0 msgr2=0x55a13898b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.634+0000 7fb42d30f640 1 --2- 172.21.15.52:0/1617666153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a13896f7a0 0x55a13898b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.634+0000 7fb42d30f640 1 -- 172.21.15.52:0/1617666153 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a13896fef0 con 0x55a138978ca0 2024-04-02T13:57:57.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.634+0000 7fb42d30f640 1 --2- 172.21.15.52:0/1617666153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a138978ca0 0x55a138978220 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fb418005fd0 tx=0x7fb418017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9fb9f81655f2214 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.635+0000 7fb4137fe640 1 -- 172.21.15.52:0/1617666153 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb418004680 con 0x55a138978ca0 2024-04-02T13:57:57.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.635+0000 7fb4137fe640 1 -- 172.21.15.52:0/1617666153 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb418004820 con 0x55a138978ca0 2024-04-02T13:57:57.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.635+0000 7fb4137fe640 1 -- 172.21.15.52:0/1617666153 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41802a230 con 0x55a138978ca0 2024-04-02T13:57:57.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.635+0000 7fb432a35740 1 -- 172.21.15.52:0/1617666153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a138978ca0 msgr2=0x55a138978220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.635+0000 7fb432a35740 1 --2- 172.21.15.52:0/1617666153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a138978ca0 0x55a138978220 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fb418005fd0 tx=0x7fb418017040 comp rx=0 tx=0).stop 2024-04-02T13:57:57.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.635+0000 7fb432a35740 1 -- 172.21.15.52:0/1617666153 shutdown_connections 2024-04-02T13:57:57.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.635+0000 7fb432a35740 1 --2- 172.21.15.52:0/1617666153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a138978ca0 0x55a138978220 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.635+0000 7fb432a35740 1 --2- 172.21.15.52:0/1617666153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1389700c0 0x55a138984920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.635+0000 7fb432a35740 1 --2- 172.21.15.52:0/1617666153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a13896f7a0 0x55a13898b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.635+0000 7fb432a35740 1 -- 172.21.15.52:0/1617666153 >> 172.21.15.52:0/1617666153 conn(0x55a138969ed0 msgr2=0x55a13896a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.635+0000 7fb432a35740 1 -- 172.21.15.52:0/1617666153 shutdown_connections 2024-04-02T13:57:57.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.635+0000 7fb432a35740 1 -- 172.21.15.52:0/1617666153 wait complete. 2024-04-02T13:57:57.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.636+0000 7fb432a35740 1 -- start start 2024-04-02T13:57:57.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.636+0000 7fb432a35740 1 Processor -- start 2024-04-02T13:57:57.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.636+0000 7fb432a35740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a13896f7a0 0x55a1389791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.636+0000 7fb432a35740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1389700c0 0x55a138979700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.636+0000 7fb432a35740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a138978ca0 0x55a138979c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.636+0000 7fb432a35740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb4281b02b0 con 0x55a13896f7a0 2024-04-02T13:57:57.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.636+0000 7fb432a35740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb4281afbe0 con 0x55a138978ca0 2024-04-02T13:57:57.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.636+0000 7fb432a35740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb4281ace20 con 0x55a1389700c0 2024-04-02T13:57:57.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.636+0000 7fb432a35740 10 client.0 _ll_register_callbacks cb 0x7fb424009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:57.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.636+0000 7fb42d30f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a13896f7a0 0x55a1389791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.636+0000 7fb42d30f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a13896f7a0 0x55a1389791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44276/0 (socket says 172.21.15.52:44276) 2024-04-02T13:57:57.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.636+0000 7fb42d30f640 1 -- 172.21.15.52:0/1747103396 learned_addr learned my addr 172.21.15.52:0/1747103396 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:57.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.636+0000 7fb42d30f640 1 --2- 172.21.15.52:0/1747103396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a138978ca0 0x55a138979c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.636+0000 7fb42db10640 1 --2- 172.21.15.52:0/1747103396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1389700c0 0x55a138979700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.636+0000 7fb42d30f640 1 -- 172.21.15.52:0/1747103396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1389700c0 msgr2=0x55a138979700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.636+0000 7fb42d30f640 1 --2- 172.21.15.52:0/1747103396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1389700c0 0x55a138979700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.636+0000 7fb42d30f640 1 -- 172.21.15.52:0/1747103396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a138978ca0 msgr2=0x55a138979c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.636+0000 7fb42d30f640 1 --2- 172.21.15.52:0/1747103396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a138978ca0 0x55a138979c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.636+0000 7fb42d30f640 1 -- 172.21.15.52:0/1747103396 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1389800a0 con 0x55a13896f7a0 2024-04-02T13:57:57.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.636+0000 7fb42db10640 1 --2- 172.21.15.52:0/1747103396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1389700c0 0x55a138979700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:57.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.637+0000 7fb42d30f640 1 --2- 172.21.15.52:0/1747103396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a13896f7a0 0x55a1389791c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb418009640 tx=0x7fb4180153d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d32a1507943851 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.637+0000 7fb4117fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:57.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.637+0000 7fb4117fa640 1 -- 172.21.15.52:0/1747103396 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb418004020 con 0x55a13896f7a0 2024-04-02T13:57:57.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.637+0000 7fb4117fa640 1 -- 172.21.15.52:0/1747103396 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fb4180041c0 con 0x55a13896f7a0 2024-04-02T13:57:57.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.637+0000 7fb432a35740 1 -- 172.21.15.52:0/1747103396 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb418004820 con 0x55a13896f7a0 2024-04-02T13:57:57.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.637+0000 7fb4117fa640 1 -- 172.21.15.52:0/1747103396 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb4180043c0 con 0x55a13896f7a0 2024-04-02T13:57:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.637+0000 7fb432a35740 10 client.16902 fetch_fsmap learned FSMap version 30 2024-04-02T13:57:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.637+0000 7fb432a35740 1 -- 172.21.15.52:0/1747103396 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a138981340 con 0x55a13896f7a0 2024-04-02T13:57:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.637+0000 7fb4117fa640 1 -- 172.21.15.52:0/1747103396 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb418004820 con 0x55a13896f7a0 2024-04-02T13:57:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.637+0000 7fb4117fa640 1 -- 172.21.15.52:0/1747103396 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb418033030 con 0x55a13896f7a0 2024-04-02T13:57:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.637+0000 7fb432a35740 10 client.16902 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:57:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.637+0000 7fb432a35740 10 client.16902 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T13:57:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.637+0000 7fb432a35740 20 client.16902 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.637+0000 7fb432a35740 1 --2- 172.21.15.52:0/1747103396 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55a1389ef640 0x55a1389f1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.637+0000 7fb432a35740 4 client.16902 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:57:57.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.637+0000 7fb432a35740 1 -- 172.21.15.52:0/1747103396 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1388db040 con 0x55a1389ef640 2024-04-02T13:57:57.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.638+0000 7fb42db10640 1 --2- 172.21.15.52:0/1747103396 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55a1389ef640 0x55a1389f1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.638+0000 7fb4117fa640 10 client.16902 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:57:57.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.638+0000 7fb42db10640 1 --2- 172.21.15.52:0/1747103396 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55a1389ef640 0x55a1389f1a30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=533e8cd1122793f server_cookie=55f385a4f5c96f75 in_seq=0 out_seq=0 2024-04-02T13:57:57.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.638+0000 7fb4117fa640 1 -- 172.21.15.52:0/1747103396 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1388db040 con 0x55a1389ef640 2024-04-02T13:57:57.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.638+0000 7fb4117fa640 10 client.16902 handle_command_reply: tid=0 2024-04-02T13:57:57.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.638+0000 7fb432a35740 1 client.16902 shutdown 2024-04-02T13:57:57.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.638+0000 7fb432a35740 1 -- 172.21.15.52:0/1747103396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a13896f7a0 msgr2=0x55a1389791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.638+0000 7fb432a35740 1 --2- 172.21.15.52:0/1747103396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a13896f7a0 0x55a1389791c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb418009640 tx=0x7fb4180153d0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.638+0000 7fb432a35740 1 -- 172.21.15.52:0/1747103396 shutdown_connections 2024-04-02T13:57:57.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.638+0000 7fb432a35740 1 --2- 172.21.15.52:0/1747103396 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55a1389ef640 0x55a1389f1a30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.639+0000 7fb432a35740 1 --2- 172.21.15.52:0/1747103396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a138978ca0 0x55a138979c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.639+0000 7fb4117fa640 0 client.16902 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:57:57.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.639+0000 7fb432a35740 1 --2- 172.21.15.52:0/1747103396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1389700c0 0x55a138979700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.639+0000 7fb432a35740 1 --2- 172.21.15.52:0/1747103396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a13896f7a0 0x55a1389791c0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.639+0000 7fb432a35740 1 -- 172.21.15.52:0/1747103396 >> 172.21.15.52:0/1747103396 conn(0x55a138969ed0 msgr2=0x55a138974380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.639+0000 7fb432a35740 1 -- 172.21.15.52:0/1747103396 shutdown_connections 2024-04-02T13:57:57.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.639+0000 7fb432a35740 1 -- 172.21.15.52:0/1747103396 wait complete. 2024-04-02T13:57:57.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.639+0000 7fb432a35740 20 client.16902 trim_cache size 0 max 16384 2024-04-02T13:57:57.654 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:57:57.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.639+0000 7fb42fd74640 1 -- 172.21.15.52:0/3179279546 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb4140a3da0 msgr2=0x7fb4140a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.639+0000 7fb42fd74640 1 --2- 172.21.15.52:0/3179279546 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb4140a3da0 0x7fb4140a6260 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb4200ce1d0 tx=0x7fb420053040 comp rx=0 tx=0).stop 2024-04-02T13:57:57.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.639+0000 7fb42fd74640 1 -- 172.21.15.52:0/3179279546 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4281069a0 msgr2=0x7fb42819cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.639+0000 7fb42fd74640 1 --2- 172.21.15.52:0/3179279546 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4281069a0 0x7fb42819cc70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fb42000eaa0 tx=0x7fb42000ef60 comp rx=0 tx=0).stop 2024-04-02T13:57:57.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.639+0000 7fb42fd74640 1 -- 172.21.15.52:0/3179279546 shutdown_connections 2024-04-02T13:57:57.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.639+0000 7fb42fd74640 1 --2- 172.21.15.52:0/3179279546 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb4140a3da0 0x7fb4140a6260 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.639+0000 7fb42fd74640 1 --2- 172.21.15.52:0/3179279546 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4281069a0 0x7fb42819cc70 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.639+0000 7fb42fd74640 1 --2- 172.21.15.52:0/3179279546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb428101350 0x7fb4281988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.639+0000 7fb42fd74640 1 --2- 172.21.15.52:0/3179279546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb428100990 0x7fb4281983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.639+0000 7fb42fd74640 1 -- 172.21.15.52:0/3179279546 >> 172.21.15.52:0/3179279546 conn(0x7fb428077ad0 msgr2=0x7fb4281030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.639+0000 7fb42fd74640 1 -- 172.21.15.52:0/3179279546 shutdown_connections 2024-04-02T13:57:57.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.639+0000 7fb42fd74640 1 -- 172.21.15.52:0/3179279546 wait complete. 2024-04-02T13:57:57.657 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:57:57.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.759+0000 7f8cb29c8640 1 -- 172.21.15.52:0/209032396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cac102450 msgr2=0x7f8cac10e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.759+0000 7f8cb29c8640 1 --2- 172.21.15.52:0/209032396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cac102450 0x7f8cac10e960 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4009cd0 tx=0x7f8ca402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.759+0000 7f8cb29c8640 1 -- 172.21.15.52:0/209032396 shutdown_connections 2024-04-02T13:57:57.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.759+0000 7f8cb29c8640 1 --2- 172.21.15.52:0/209032396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cac102450 0x7f8cac10e960 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.759+0000 7f8cb29c8640 1 --2- 172.21.15.52:0/209032396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cac101a90 0x7f8cac101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.759+0000 7f8cb29c8640 1 --2- 172.21.15.52:0/209032396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cac107aa0 0x7f8cac107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.759+0000 7f8cb29c8640 1 -- 172.21.15.52:0/209032396 >> 172.21.15.52:0/209032396 conn(0x7f8cac078f00 msgr2=0x7f8cac079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.759+0000 7f8cb29c8640 1 -- 172.21.15.52:0/209032396 shutdown_connections 2024-04-02T13:57:57.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.760+0000 7f8cb29c8640 1 -- 172.21.15.52:0/209032396 wait complete. 2024-04-02T13:57:57.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.760+0000 7f8cb29c8640 1 Processor -- start 2024-04-02T13:57:57.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.760+0000 7f8cb29c8640 1 -- start start 2024-04-02T13:57:57.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.760+0000 7f8cb29c8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cac101a90 0x7f8cac198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.760+0000 7f8cb29c8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cac102450 0x7f8cac1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.760+0000 7f8cb29c8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cac107aa0 0x7f8cac19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.760+0000 7f8cb29c8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac111eb0 con 0x7f8cac107aa0 2024-04-02T13:57:57.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.760+0000 7f8cb29c8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac111d30 con 0x7f8cac102450 2024-04-02T13:57:57.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.760+0000 7f8cb29c8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac112030 con 0x7f8cac101a90 2024-04-02T13:57:57.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.761+0000 7f8cb0f65640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cac107aa0 0x7f8cac19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.761+0000 7f8cb0f65640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cac107aa0 0x7f8cac19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44302/0 (socket says 172.21.15.52:44302) 2024-04-02T13:57:57.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.761+0000 7f8cb0f65640 1 -- 172.21.15.52:0/752274840 learned_addr learned my addr 172.21.15.52:0/752274840 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:57.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.761+0000 7f8cabfff640 1 --2- 172.21.15.52:0/752274840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cac101a90 0x7f8cac198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.761+0000 7f8cab7fe640 1 --2- 172.21.15.52:0/752274840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cac102450 0x7f8cac1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.761+0000 7f8cb0f65640 1 -- 172.21.15.52:0/752274840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cac101a90 msgr2=0x7f8cac198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.761+0000 7f8cb0f65640 1 --2- 172.21.15.52:0/752274840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cac101a90 0x7f8cac198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.761+0000 7f8cb0f65640 1 -- 172.21.15.52:0/752274840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cac102450 msgr2=0x7f8cac1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.761+0000 7f8cb0f65640 1 --2- 172.21.15.52:0/752274840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cac102450 0x7f8cac1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.761+0000 7f8cb0f65640 1 -- 172.21.15.52:0/752274840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cac19d360 con 0x7f8cac107aa0 2024-04-02T13:57:57.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.761+0000 7f8cb0f65640 1 --2- 172.21.15.52:0/752274840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cac107aa0 0x7f8cac19cc30 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4004af0 tx=0x7f8ca4032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.761+0000 7f8ca97fa640 1 -- 172.21.15.52:0/752274840 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4041020 con 0x7f8cac107aa0 2024-04-02T13:57:57.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.761+0000 7f8ca97fa640 1 -- 172.21.15.52:0/752274840 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ca4034cd0 con 0x7f8cac107aa0 2024-04-02T13:57:57.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.761+0000 7f8cb29c8640 1 -- 172.21.15.52:0/752274840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cac19d5f0 con 0x7f8cac107aa0 2024-04-02T13:57:57.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.761+0000 7f8ca97fa640 1 -- 172.21.15.52:0/752274840 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4039040 con 0x7f8cac107aa0 2024-04-02T13:57:57.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.761+0000 7f8cb29c8640 1 -- 172.21.15.52:0/752274840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cac1a4ee0 con 0x7f8cac107aa0 2024-04-02T13:57:57.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.763+0000 7f8ca97fa640 1 -- 172.21.15.52:0/752274840 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8ca4044020 con 0x7f8cac107aa0 2024-04-02T13:57:57.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.764+0000 7f8cb55ac740 1 Processor -- start 2024-04-02T13:57:57.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.764+0000 7f8cb55ac740 1 -- start start 2024-04-02T13:57:57.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.764+0000 7f8ca97fa640 1 --2- 172.21.15.52:0/752274840 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f8c940a3d20 0x7f8c940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.764+0000 7f8cb55ac740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a29a2e7a0 0x564a29a2eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.764+0000 7f8cb55ac740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a29a2f0c0 0x564a29a37760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.764+0000 7f8cb55ac740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a37ca0 0x564a29a3a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.764+0000 7f8cb55ac740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564a299927b0 con 0x564a29a37ca0 2024-04-02T13:57:57.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.764+0000 7f8cb55ac740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564a29931570 con 0x564a29a2f0c0 2024-04-02T13:57:57.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.764+0000 7f8cb55ac740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564a2998e760 con 0x564a29a2e7a0 2024-04-02T13:57:57.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.764+0000 7f8cab7fe640 1 --2- 172.21.15.52:0/752274840 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f8c940a3d20 0x7f8c940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.764+0000 7f8ca97fa640 1 -- 172.21.15.52:0/752274840 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f8ca40f27e0 con 0x7f8cac107aa0 2024-04-02T13:57:57.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.764+0000 7f8cb0f65640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a37ca0 0x564a29a3a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.764+0000 7f8cb0f65640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a37ca0 0x564a29a3a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44306/0 (socket says 172.21.15.52:44306) 2024-04-02T13:57:57.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.764+0000 7f8cb0f65640 1 -- 172.21.15.52:0/2915661667 learned_addr learned my addr 172.21.15.52:0/2915661667 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:57.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.764+0000 7f8cab7fe640 1 --2- 172.21.15.52:0/2915661667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a29a2f0c0 0x564a29a37760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.764+0000 7f8cabfff640 1 --2- 172.21.15.52:0/2915661667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a29a2e7a0 0x564a29a2eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.764+0000 7f8cb0f65640 1 -- 172.21.15.52:0/2915661667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a29a2e7a0 msgr2=0x564a29a2eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.764+0000 7f8cb0f65640 1 --2- 172.21.15.52:0/2915661667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a29a2e7a0 0x564a29a2eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.764+0000 7f8cb0f65640 1 -- 172.21.15.52:0/2915661667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a29a2f0c0 msgr2=0x564a29a37760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.764+0000 7f8cb0f65640 1 --2- 172.21.15.52:0/2915661667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a29a2f0c0 0x564a29a37760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.764+0000 7f8cb0f65640 1 -- 172.21.15.52:0/2915661667 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a296a63f0 con 0x564a29a37ca0 2024-04-02T13:57:57.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.765+0000 7f8cab7fe640 1 --2- 172.21.15.52:0/752274840 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f8c940a3d20 0x7f8c940a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0058c0 tx=0x7f8c9c00bf90 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.765+0000 7f8cb0f65640 1 --2- 172.21.15.52:0/2915661667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a37ca0 0x564a29a3a090 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f8ca40c6c90 tx=0x7f8ca4003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91bc881b9a64d3d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.765+0000 7f8c927fc640 1 -- 172.21.15.52:0/2915661667 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca40bec50 con 0x564a29a37ca0 2024-04-02T13:57:57.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.765+0000 7f8c927fc640 1 -- 172.21.15.52:0/2915661667 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ca40348f0 con 0x564a29a37ca0 2024-04-02T13:57:57.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.765+0000 7f8c927fc640 1 -- 172.21.15.52:0/2915661667 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca40babc0 con 0x564a29a37ca0 2024-04-02T13:57:57.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.765+0000 7f8cb55ac740 1 -- 172.21.15.52:0/2915661667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a37ca0 msgr2=0x564a29a3a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.765+0000 7f8cb55ac740 1 --2- 172.21.15.52:0/2915661667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a37ca0 0x564a29a3a090 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f8ca40c6c90 tx=0x7f8ca4003f90 comp rx=0 tx=0).stop 2024-04-02T13:57:57.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.765+0000 7f8cb55ac740 1 -- 172.21.15.52:0/2915661667 shutdown_connections 2024-04-02T13:57:57.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.765+0000 7f8cb55ac740 1 --2- 172.21.15.52:0/2915661667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a37ca0 0x564a29a3a090 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.765+0000 7f8cb55ac740 1 --2- 172.21.15.52:0/2915661667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a29a2f0c0 0x564a29a37760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.765+0000 7f8cb55ac740 1 --2- 172.21.15.52:0/2915661667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a29a2e7a0 0x564a29a2eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.765+0000 7f8cb55ac740 1 -- 172.21.15.52:0/2915661667 >> 172.21.15.52:0/2915661667 conn(0x564a29a28ed0 msgr2=0x564a29a292b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.765+0000 7f8cb55ac740 1 -- 172.21.15.52:0/2915661667 shutdown_connections 2024-04-02T13:57:57.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.765+0000 7f8cb55ac740 1 -- 172.21.15.52:0/2915661667 wait complete. 2024-04-02T13:57:57.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.766+0000 7f8cb55ac740 1 -- start start 2024-04-02T13:57:57.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.766+0000 7f8cb55ac740 1 Processor -- start 2024-04-02T13:57:57.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.766+0000 7f8cb55ac740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a29a2e7a0 0x564a29a37600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.766+0000 7f8cb55ac740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a29a2f0c0 0x564a29a34990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.766+0000 7f8cb55ac740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a37ca0 0x564a29a34ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.766+0000 7f8cb55ac740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564a29a3a850 con 0x564a29a37ca0 2024-04-02T13:57:57.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.766+0000 7f8cb55ac740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564a2994de80 con 0x564a29a2e7a0 2024-04-02T13:57:57.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.766+0000 7f8cb55ac740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564a29a3a9d0 con 0x564a29a2f0c0 2024-04-02T13:57:57.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.766+0000 7f8cb55ac740 10 client.0 _ll_register_callbacks cb 0x564a29989aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:57.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.766+0000 7f8cb0f65640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a37ca0 0x564a29a34ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.766+0000 7f8cb0f65640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a37ca0 0x564a29a34ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44316/0 (socket says 172.21.15.52:44316) 2024-04-02T13:57:57.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.766+0000 7f8cb0f65640 1 -- 172.21.15.52:0/2638135778 learned_addr learned my addr 172.21.15.52:0/2638135778 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:57.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.767+0000 7f8cab7fe640 1 --2- 172.21.15.52:0/2638135778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a29a2f0c0 0x564a29a34990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.767+0000 7f8cabfff640 1 --2- 172.21.15.52:0/2638135778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a29a2e7a0 0x564a29a37600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.767+0000 7f8cb0f65640 1 -- 172.21.15.52:0/2638135778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a29a2f0c0 msgr2=0x564a29a34990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.767+0000 7f8cb0f65640 1 --2- 172.21.15.52:0/2638135778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a29a2f0c0 0x564a29a34990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.767+0000 7f8cb0f65640 1 -- 172.21.15.52:0/2638135778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a29a2e7a0 msgr2=0x564a29a37600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.767+0000 7f8cb0f65640 1 --2- 172.21.15.52:0/2638135778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a29a2e7a0 0x564a29a37600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.767+0000 7f8cb0f65640 1 -- 172.21.15.52:0/2638135778 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a297707d0 con 0x564a29a37ca0 2024-04-02T13:57:57.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.767+0000 7f8cab7fe640 1 --2- 172.21.15.52:0/2638135778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a29a2f0c0 0x564a29a34990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:57.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.767+0000 7f8cb0f65640 1 --2- 172.21.15.52:0/2638135778 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a37ca0 0x564a29a34ef0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f8ca402c6d0 tx=0x7f8ca4032b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23077cfef521460 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.767+0000 7f8c8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:57.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.767+0000 7f8c8b7fe640 1 -- 172.21.15.52:0/2638135778 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca40bec50 con 0x564a29a37ca0 2024-04-02T13:57:57.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.767+0000 7f8c8b7fe640 1 -- 172.21.15.52:0/2638135778 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f8ca40c6d30 con 0x564a29a37ca0 2024-04-02T13:57:57.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.767+0000 7f8cb55ac740 1 -- 172.21.15.52:0/2638135778 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ca40348f0 con 0x564a29a37ca0 2024-04-02T13:57:57.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.767+0000 7f8c8b7fe640 1 -- 172.21.15.52:0/2638135778 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8ca40be550 con 0x564a29a37ca0 2024-04-02T13:57:57.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.767+0000 7f8cb55ac740 10 client.16914 fetch_fsmap learned FSMap version 30 2024-04-02T13:57:57.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.767+0000 7f8cb55ac740 1 -- 172.21.15.52:0/2638135778 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564a29a3a850 con 0x564a29a37ca0 2024-04-02T13:57:57.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.767+0000 7f8c8b7fe640 1 -- 172.21.15.52:0/2638135778 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ca40348f0 con 0x564a29a37ca0 2024-04-02T13:57:57.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.767+0000 7f8c8b7fe640 1 -- 172.21.15.52:0/2638135778 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8ca403c030 con 0x564a29a37ca0 2024-04-02T13:57:57.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.767+0000 7f8cb55ac740 10 client.16914 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:57:57.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.767+0000 7f8cb55ac740 10 client.16914 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T13:57:57.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.768+0000 7f8cb55ac740 20 client.16914 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:57.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.768+0000 7f8cb55ac740 1 --2- 172.21.15.52:0/2638135778 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x564a29a47a10 0x564a29a49e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.768+0000 7f8cb55ac740 4 client.16914 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:57:57.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.768+0000 7f8cb55ac740 1 -- 172.21.15.52:0/2638135778 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a2999fcf0 con 0x564a29a47a10 2024-04-02T13:57:57.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.768+0000 7f8cabfff640 1 --2- 172.21.15.52:0/2638135778 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x564a29a47a10 0x564a29a49e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.768+0000 7f8cabfff640 1 --2- 172.21.15.52:0/2638135778 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x564a29a47a10 0x564a29a49e30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9341615405b680df server_cookie=49cedc9a30b1f21 in_seq=0 out_seq=0 2024-04-02T13:57:57.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.768+0000 7f8c8b7fe640 10 client.16914 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:57:57.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.769+0000 7f8c8b7fe640 1 -- 172.21.15.52:0/2638135778 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564a2999fcf0 con 0x564a29a47a10 2024-04-02T13:57:57.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.769+0000 7f8c8b7fe640 10 client.16914 handle_command_reply: tid=0 2024-04-02T13:57:57.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.769+0000 7f8cb55ac740 1 client.16914 shutdown 2024-04-02T13:57:57.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.769+0000 7f8cb55ac740 1 -- 172.21.15.52:0/2638135778 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a37ca0 msgr2=0x564a29a34ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.769+0000 7f8cb55ac740 1 --2- 172.21.15.52:0/2638135778 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a37ca0 0x564a29a34ef0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f8ca402c6d0 tx=0x7f8ca4032b00 comp rx=0 tx=0).stop 2024-04-02T13:57:57.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.769+0000 7f8cb55ac740 1 -- 172.21.15.52:0/2638135778 shutdown_connections 2024-04-02T13:57:57.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.769+0000 7f8cb55ac740 1 --2- 172.21.15.52:0/2638135778 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x564a29a47a10 0x564a29a49e30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.769+0000 7f8cb55ac740 1 --2- 172.21.15.52:0/2638135778 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a37ca0 0x564a29a34ef0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.769+0000 7f8c8b7fe640 0 client.16914 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:57:57.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.769+0000 7f8cb55ac740 1 --2- 172.21.15.52:0/2638135778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a29a2f0c0 0x564a29a34990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.769+0000 7f8cb55ac740 1 --2- 172.21.15.52:0/2638135778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a29a2e7a0 0x564a29a37600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.769+0000 7f8cb55ac740 1 -- 172.21.15.52:0/2638135778 >> 172.21.15.52:0/2638135778 conn(0x564a29a28ed0 msgr2=0x564a29a2ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.769+0000 7f8cb55ac740 1 -- 172.21.15.52:0/2638135778 shutdown_connections 2024-04-02T13:57:57.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.769+0000 7f8cb55ac740 1 -- 172.21.15.52:0/2638135778 wait complete. 2024-04-02T13:57:57.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.769+0000 7f8cb55ac740 20 client.16914 trim_cache size 0 max 16384 2024-04-02T13:57:57.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.782+0000 7f8cb55ac740 1 Processor -- start 2024-04-02T13:57:57.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.783+0000 7f8cb55ac740 1 -- start start 2024-04-02T13:57:57.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.783+0000 7f8cb55ac740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a29a2e7a0 0x564a29a43680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.783+0000 7f8cb55ac740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a2f0c0 0x564a29a2aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.783+0000 7f8cb55ac740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a29a37ca0 0x564a29a370c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.783+0000 7f8cb55ac740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac1aed50 con 0x564a29a2f0c0 2024-04-02T13:57:57.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.783+0000 7f8cb55ac740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac1b0890 con 0x564a29a37ca0 2024-04-02T13:57:57.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.783+0000 7f8cb55ac740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac1af420 con 0x564a29a2e7a0 2024-04-02T13:57:57.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.783+0000 7f8cab7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a2f0c0 0x564a29a2aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.783+0000 7f8cab7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a2f0c0 0x564a29a2aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44320/0 (socket says 172.21.15.52:44320) 2024-04-02T13:57:57.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.783+0000 7f8cab7fe640 1 -- 172.21.15.52:0/565043081 learned_addr learned my addr 172.21.15.52:0/565043081 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:57.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.783+0000 7f8cabfff640 1 --2- 172.21.15.52:0/565043081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a29a2e7a0 0x564a29a43680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.783+0000 7f8cb0f65640 1 --2- 172.21.15.52:0/565043081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a29a37ca0 0x564a29a370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.783+0000 7f8cab7fe640 1 -- 172.21.15.52:0/565043081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a29a2e7a0 msgr2=0x564a29a43680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.783+0000 7f8cab7fe640 1 --2- 172.21.15.52:0/565043081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a29a2e7a0 0x564a29a43680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.783+0000 7f8cab7fe640 1 -- 172.21.15.52:0/565043081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a29a37ca0 msgr2=0x564a29a370c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.783+0000 7f8cab7fe640 1 --2- 172.21.15.52:0/565043081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a29a37ca0 0x564a29a370c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.783+0000 7f8cab7fe640 1 -- 172.21.15.52:0/565043081 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a29a2eef0 con 0x564a29a2f0c0 2024-04-02T13:57:57.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.783+0000 7f8cb0f65640 1 --2- 172.21.15.52:0/565043081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a29a37ca0 0x564a29a370c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:57.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.783+0000 7f8cab7fe640 1 --2- 172.21.15.52:0/565043081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a2f0c0 0x564a29a2aa30 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00bab0 tx=0x7f8c9c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2477851fb6ad52 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.783+0000 7f8c91ffb640 1 -- 172.21.15.52:0/565043081 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c024070 con 0x564a29a2f0c0 2024-04-02T13:57:57.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.783+0000 7f8c91ffb640 1 -- 172.21.15.52:0/565043081 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c9c020bc0 con 0x564a29a2f0c0 2024-04-02T13:57:57.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.783+0000 7f8c91ffb640 1 -- 172.21.15.52:0/565043081 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c01fbb0 con 0x564a29a2f0c0 2024-04-02T13:57:57.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.784+0000 7f8cb55ac740 1 -- 172.21.15.52:0/565043081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a2f0c0 msgr2=0x564a29a2aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.784+0000 7f8cb55ac740 1 --2- 172.21.15.52:0/565043081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a2f0c0 0x564a29a2aa30 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00bab0 tx=0x7f8c9c01c000 comp rx=0 tx=0).stop 2024-04-02T13:57:57.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.784+0000 7f8cb55ac740 1 -- 172.21.15.52:0/565043081 shutdown_connections 2024-04-02T13:57:57.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.784+0000 7f8cb55ac740 1 --2- 172.21.15.52:0/565043081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a29a37ca0 0x564a29a370c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.784+0000 7f8cb55ac740 1 --2- 172.21.15.52:0/565043081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a2f0c0 0x564a29a2aa30 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.784+0000 7f8cb55ac740 1 --2- 172.21.15.52:0/565043081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a29a2e7a0 0x564a29a43680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.784+0000 7f8cb55ac740 1 -- 172.21.15.52:0/565043081 >> 172.21.15.52:0/565043081 conn(0x564a29a28ed0 msgr2=0x564a29a2a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.784+0000 7f8cb55ac740 1 -- 172.21.15.52:0/565043081 shutdown_connections 2024-04-02T13:57:57.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.784+0000 7f8cb55ac740 1 -- 172.21.15.52:0/565043081 wait complete. 2024-04-02T13:57:57.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.784+0000 7f8cb55ac740 1 -- start start 2024-04-02T13:57:57.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.785+0000 7f8cb55ac740 1 Processor -- start 2024-04-02T13:57:57.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.785+0000 7f8cb55ac740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a29a2e7a0 0x564a29a40000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.785+0000 7f8cb55ac740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a2f0c0 0x564a29a31da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.785+0000 7f8cb55ac740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a29a37ca0 0x564a29a322e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.785+0000 7f8cb55ac740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac1b01c0 con 0x564a29a2f0c0 2024-04-02T13:57:57.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.785+0000 7f8cb55ac740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac1afaf0 con 0x564a29a37ca0 2024-04-02T13:57:57.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.785+0000 7f8cb55ac740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac1acd30 con 0x564a29a2e7a0 2024-04-02T13:57:57.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.785+0000 7f8cb55ac740 10 client.0 _ll_register_callbacks cb 0x7f8ca40efca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:57.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.785+0000 7f8cab7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a2f0c0 0x564a29a31da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.785+0000 7f8cab7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a2f0c0 0x564a29a31da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44330/0 (socket says 172.21.15.52:44330) 2024-04-02T13:57:57.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.785+0000 7f8cab7fe640 1 -- 172.21.15.52:0/3687285336 learned_addr learned my addr 172.21.15.52:0/3687285336 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:57.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.785+0000 7f8cabfff640 1 --2- 172.21.15.52:0/3687285336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a29a2e7a0 0x564a29a40000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.785+0000 7f8cab7fe640 1 -- 172.21.15.52:0/3687285336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a29a2e7a0 msgr2=0x564a29a40000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.785+0000 7f8cab7fe640 1 --2- 172.21.15.52:0/3687285336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a29a2e7a0 0x564a29a40000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.785+0000 7f8cab7fe640 1 -- 172.21.15.52:0/3687285336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a29a37ca0 msgr2=0x564a29a322e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.785+0000 7f8cb0f65640 1 --2- 172.21.15.52:0/3687285336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a29a37ca0 0x564a29a322e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.785+0000 7f8cab7fe640 1 --2- 172.21.15.52:0/3687285336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a29a37ca0 0x564a29a322e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.785+0000 7f8cab7fe640 1 -- 172.21.15.52:0/3687285336 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a29a38900 con 0x564a29a2f0c0 2024-04-02T13:57:57.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.785+0000 7f8cab7fe640 1 --2- 172.21.15.52:0/3687285336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a2f0c0 0x564a29a31da0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c005890 tx=0x7f8c9c020a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db23a34014daa36b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.785+0000 7f8c8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:57.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.786+0000 7f8c8b7fe640 1 -- 172.21.15.52:0/3687285336 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c024040 con 0x564a29a2f0c0 2024-04-02T13:57:57.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.786+0000 7f8c8b7fe640 1 -- 172.21.15.52:0/3687285336 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f8c9c005460 con 0x564a29a2f0c0 2024-04-02T13:57:57.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.786+0000 7f8cb55ac740 1 -- 172.21.15.52:0/3687285336 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c9c020bc0 con 0x564a29a2f0c0 2024-04-02T13:57:57.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.786+0000 7f8c8b7fe640 1 -- 172.21.15.52:0/3687285336 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8c9c03d050 con 0x564a29a2f0c0 2024-04-02T13:57:57.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.786+0000 7f8cb55ac740 10 client.16920 fetch_fsmap learned FSMap version 30 2024-04-02T13:57:57.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.786+0000 7f8cb55ac740 1 -- 172.21.15.52:0/3687285336 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564a29aae0c0 con 0x564a29a2f0c0 2024-04-02T13:57:57.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.786+0000 7f8c8b7fe640 1 -- 172.21.15.52:0/3687285336 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c9c020bc0 con 0x564a29a2f0c0 2024-04-02T13:57:57.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.786+0000 7f8c8b7fe640 1 -- 172.21.15.52:0/3687285336 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8c9c00f050 con 0x564a29a2f0c0 2024-04-02T13:57:57.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.786+0000 7f8cb55ac740 10 client.16920 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:57:57.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.786+0000 7f8cb55ac740 10 client.16920 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T13:57:57.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.786+0000 7f8cb55ac740 20 client.16920 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:57.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.786+0000 7f8cb55ac740 1 --2- 172.21.15.52:0/3687285336 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x564a29aae730 0x564a29ab0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.786+0000 7f8cb55ac740 4 client.16920 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:57:57.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.786+0000 7f8cb55ac740 1 -- 172.21.15.52:0/3687285336 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a2999a040 con 0x564a29aae730 2024-04-02T13:57:57.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.786+0000 7f8cabfff640 1 --2- 172.21.15.52:0/3687285336 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x564a29aae730 0x564a29ab0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.787+0000 7f8c8b7fe640 10 client.16920 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:57:57.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.787+0000 7f8cabfff640 1 --2- 172.21.15.52:0/3687285336 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x564a29aae730 0x564a29ab0b50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f243bf62328842f7 server_cookie=57ab992526f9d5de in_seq=0 out_seq=0 2024-04-02T13:57:57.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.787+0000 7f8c8b7fe640 1 -- 172.21.15.52:0/3687285336 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a2999a040 con 0x564a29aae730 2024-04-02T13:57:57.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.787+0000 7f8c8b7fe640 10 client.16920 handle_command_reply: tid=0 2024-04-02T13:57:57.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.787+0000 7f8cb55ac740 1 client.16920 shutdown 2024-04-02T13:57:57.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.787+0000 7f8cb55ac740 1 -- 172.21.15.52:0/3687285336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a2f0c0 msgr2=0x564a29a31da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.787+0000 7f8cb55ac740 1 --2- 172.21.15.52:0/3687285336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a2f0c0 0x564a29a31da0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c005890 tx=0x7f8c9c020a80 comp rx=0 tx=0).stop 2024-04-02T13:57:57.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.787+0000 7f8cb55ac740 1 -- 172.21.15.52:0/3687285336 shutdown_connections 2024-04-02T13:57:57.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.787+0000 7f8cb55ac740 1 --2- 172.21.15.52:0/3687285336 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x564a29aae730 0x564a29ab0b50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.787+0000 7f8cb55ac740 1 --2- 172.21.15.52:0/3687285336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a29a37ca0 0x564a29a322e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.787+0000 7f8c8b7fe640 0 client.16920 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:57:57.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.787+0000 7f8cb55ac740 1 --2- 172.21.15.52:0/3687285336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a29a2f0c0 0x564a29a31da0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.787+0000 7f8cb55ac740 1 --2- 172.21.15.52:0/3687285336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a29a2e7a0 0x564a29a40000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.787+0000 7f8cb55ac740 1 -- 172.21.15.52:0/3687285336 >> 172.21.15.52:0/3687285336 conn(0x564a29a28ed0 msgr2=0x564a29a349d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.787+0000 7f8cb55ac740 1 -- 172.21.15.52:0/3687285336 shutdown_connections 2024-04-02T13:57:57.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.787+0000 7f8cb55ac740 1 -- 172.21.15.52:0/3687285336 wait complete. 2024-04-02T13:57:57.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.787+0000 7f8cb55ac740 20 client.16920 trim_cache size 0 max 16384 2024-04-02T13:57:57.810 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:57:57.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.788+0000 7f8cb29c8640 1 -- 172.21.15.52:0/752274840 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f8c940a3d20 msgr2=0x7f8c940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.788+0000 7f8cb29c8640 1 --2- 172.21.15.52:0/752274840 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f8c940a3d20 0x7f8c940a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0058c0 tx=0x7f8c9c00bf90 comp rx=0 tx=0).stop 2024-04-02T13:57:57.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.788+0000 7f8cb29c8640 1 -- 172.21.15.52:0/752274840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cac107aa0 msgr2=0x7f8cac19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.788+0000 7f8cb29c8640 1 --2- 172.21.15.52:0/752274840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cac107aa0 0x7f8cac19cc30 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4004af0 tx=0x7f8ca4032ec0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.788+0000 7f8cb29c8640 1 -- 172.21.15.52:0/752274840 shutdown_connections 2024-04-02T13:57:57.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.788+0000 7f8cb29c8640 1 --2- 172.21.15.52:0/752274840 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f8c940a3d20 0x7f8c940a61e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.788+0000 7f8cb29c8640 1 --2- 172.21.15.52:0/752274840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cac107aa0 0x7f8cac19cc30 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.788+0000 7f8cb29c8640 1 --2- 172.21.15.52:0/752274840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cac102450 0x7f8cac1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.788+0000 7f8cb29c8640 1 --2- 172.21.15.52:0/752274840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cac101a90 0x7f8cac198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.788+0000 7f8cb29c8640 1 -- 172.21.15.52:0/752274840 >> 172.21.15.52:0/752274840 conn(0x7f8cac078f00 msgr2=0x7f8cac104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.788+0000 7f8cb29c8640 1 -- 172.21.15.52:0/752274840 shutdown_connections 2024-04-02T13:57:57.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.788+0000 7f8cb29c8640 1 -- 172.21.15.52:0/752274840 wait complete. 2024-04-02T13:57:57.813 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:57:57.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.914+0000 7f902d722640 1 -- 172.21.15.52:0/2735807723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9028101350 msgr2=0x7f902810d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.914+0000 7f902d722640 1 --2- 172.21.15.52:0/2735807723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9028101350 0x7f902810d860 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f9020009cd0 tx=0x7f902002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.914+0000 7f902d722640 1 -- 172.21.15.52:0/2735807723 shutdown_connections 2024-04-02T13:57:57.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.914+0000 7f902d722640 1 --2- 172.21.15.52:0/2735807723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9028101350 0x7f902810d860 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.914+0000 7f902d722640 1 --2- 172.21.15.52:0/2735807723 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9028100990 0x7f9028100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.914+0000 7f902d722640 1 --2- 172.21.15.52:0/2735807723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90281069a0 0x7f9028106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.914+0000 7f902d722640 1 -- 172.21.15.52:0/2735807723 >> 172.21.15.52:0/2735807723 conn(0x7f9028077ad0 msgr2=0x7f9028077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.914+0000 7f902d722640 1 -- 172.21.15.52:0/2735807723 shutdown_connections 2024-04-02T13:57:57.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.914+0000 7f902d722640 1 -- 172.21.15.52:0/2735807723 wait complete. 2024-04-02T13:57:57.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.914+0000 7f902d722640 1 Processor -- start 2024-04-02T13:57:57.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.915+0000 7f902d722640 1 -- start start 2024-04-02T13:57:57.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.915+0000 7f902d722640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9028100990 0x7f9028102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.915+0000 7f902d722640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9028101350 0x7f9028103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.915+0000 7f902d722640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90281069a0 0x7f9028103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.915+0000 7f902d722640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9028110db0 con 0x7f90281069a0 2024-04-02T13:57:57.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.915+0000 7f902d722640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9028110c30 con 0x7f9028101350 2024-04-02T13:57:57.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.915+0000 7f902d722640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9028110f30 con 0x7f9028100990 2024-04-02T13:57:57.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.915+0000 7f90277fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90281069a0 0x7f9028103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.915+0000 7f90277fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90281069a0 0x7f9028103990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44346/0 (socket says 172.21.15.52:44346) 2024-04-02T13:57:57.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.915+0000 7f90277fe640 1 -- 172.21.15.52:0/3579845334 learned_addr learned my addr 172.21.15.52:0/3579845334 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:57.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.915+0000 7f901e7fc640 1 --2- 172.21.15.52:0/3579845334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9028101350 0x7f9028103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.915+0000 7f9026ffd640 1 --2- 172.21.15.52:0/3579845334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9028100990 0x7f9028102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.915+0000 7f90277fe640 1 -- 172.21.15.52:0/3579845334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9028100990 msgr2=0x7f9028102f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.915+0000 7f90277fe640 1 --2- 172.21.15.52:0/3579845334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9028100990 0x7f9028102f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.915+0000 7f90277fe640 1 -- 172.21.15.52:0/3579845334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9028101350 msgr2=0x7f9028103450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.915+0000 7f90277fe640 1 --2- 172.21.15.52:0/3579845334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9028101350 0x7f9028103450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.915+0000 7f90277fe640 1 -- 172.21.15.52:0/3579845334 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90281a68a0 con 0x7f90281069a0 2024-04-02T13:57:57.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.915+0000 7f901e7fc640 1 --2- 172.21.15.52:0/3579845334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9028101350 0x7f9028103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.916+0000 7f90277fe640 1 --2- 172.21.15.52:0/3579845334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90281069a0 0x7f9028103990 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f90200348f0 tx=0x7f9020032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.916+0000 7f9024ff9640 1 -- 172.21.15.52:0/3579845334 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020041020 con 0x7f90281069a0 2024-04-02T13:57:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.916+0000 7f9024ff9640 1 -- 172.21.15.52:0/3579845334 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9020034cd0 con 0x7f90281069a0 2024-04-02T13:57:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.916+0000 7f902d722640 1 -- 172.21.15.52:0/3579845334 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90281a6b30 con 0x7f90281069a0 2024-04-02T13:57:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.916+0000 7f9024ff9640 1 -- 172.21.15.52:0/3579845334 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020039040 con 0x7f90281069a0 2024-04-02T13:57:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.916+0000 7f902d722640 1 -- 172.21.15.52:0/3579845334 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90281a7070 con 0x7f90281069a0 2024-04-02T13:57:57.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.918+0000 7f9024ff9640 1 -- 172.21.15.52:0/3579845334 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9020044020 con 0x7f90281069a0 2024-04-02T13:57:57.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.918+0000 7f90303e0740 1 Processor -- start 2024-04-02T13:57:57.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.918+0000 7f90303e0740 1 -- start start 2024-04-02T13:57:57.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.918+0000 7f9024ff9640 1 --2- 172.21.15.52:0/3579845334 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f90140a3da0 0x7f90140a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.919+0000 7f9024ff9640 1 -- 172.21.15.52:0/3579845334 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f90200f2930 con 0x7f90281069a0 2024-04-02T13:57:57.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.919+0000 7f901e7fc640 1 --2- 172.21.15.52:0/3579845334 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f90140a3da0 0x7f90140a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.919+0000 7f90303e0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557a6e5f7a0 0x5557a6e5fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.919+0000 7f90303e0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557a6e600c0 0x5557a6e68760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.919+0000 7f90303e0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e68ca0 0x5557a6e6b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.919+0000 7f90303e0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5557a6dc37b0 con 0x5557a6e68ca0 2024-04-02T13:57:57.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.919+0000 7f90303e0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5557a6d62570 con 0x5557a6e5f7a0 2024-04-02T13:57:57.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.919+0000 7f90303e0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5557a6dbf760 con 0x5557a6e600c0 2024-04-02T13:57:57.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.919+0000 7f90277fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e68ca0 0x5557a6e6b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.919+0000 7f90277fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e68ca0 0x5557a6e6b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44356/0 (socket says 172.21.15.52:44356) 2024-04-02T13:57:57.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.919+0000 7f90277fe640 1 -- 172.21.15.52:0/2229969147 learned_addr learned my addr 172.21.15.52:0/2229969147 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:57.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.919+0000 7f9026ffd640 1 --2- 172.21.15.52:0/2229969147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557a6e5f7a0 0x5557a6e5fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.919+0000 7f90277fe640 1 -- 172.21.15.52:0/2229969147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557a6e600c0 msgr2=0x5557a6e68760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.919+0000 7f90277fe640 1 --2- 172.21.15.52:0/2229969147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557a6e600c0 0x5557a6e68760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.919+0000 7f901e7fc640 1 --2- 172.21.15.52:0/3579845334 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f90140a3da0 0x7f90140a6260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f90100058c0 tx=0x7f901000f730 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.919+0000 7f90277fe640 1 -- 172.21.15.52:0/2229969147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557a6e5f7a0 msgr2=0x5557a6e5fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.919+0000 7f90277fe640 1 --2- 172.21.15.52:0/2229969147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557a6e5f7a0 0x5557a6e5fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.919+0000 7f90277fe640 1 -- 172.21.15.52:0/2229969147 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557a6ad73f0 con 0x5557a6e68ca0 2024-04-02T13:57:57.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.919+0000 7f90277fe640 1 --2- 172.21.15.52:0/2229969147 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e68ca0 0x5557a6e6b090 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f90200be630 tx=0x7f90200be530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733b270a14152555 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.920+0000 7f901d7fa640 1 -- 172.21.15.52:0/2229969147 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90200a9070 con 0x5557a6e68ca0 2024-04-02T13:57:57.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.920+0000 7f901d7fa640 1 -- 172.21.15.52:0/2229969147 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90200beda0 con 0x5557a6e68ca0 2024-04-02T13:57:57.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.920+0000 7f901d7fa640 1 -- 172.21.15.52:0/2229969147 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90200bad10 con 0x5557a6e68ca0 2024-04-02T13:57:57.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.920+0000 7f90303e0740 1 -- 172.21.15.52:0/2229969147 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e68ca0 msgr2=0x5557a6e6b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.920+0000 7f90303e0740 1 --2- 172.21.15.52:0/2229969147 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e68ca0 0x5557a6e6b090 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f90200be630 tx=0x7f90200be530 comp rx=0 tx=0).stop 2024-04-02T13:57:57.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.920+0000 7f90303e0740 1 -- 172.21.15.52:0/2229969147 shutdown_connections 2024-04-02T13:57:57.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.920+0000 7f90303e0740 1 --2- 172.21.15.52:0/2229969147 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e68ca0 0x5557a6e6b090 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.920+0000 7f90303e0740 1 --2- 172.21.15.52:0/2229969147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557a6e600c0 0x5557a6e68760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.920+0000 7f90303e0740 1 --2- 172.21.15.52:0/2229969147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557a6e5f7a0 0x5557a6e5fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.920+0000 7f90303e0740 1 -- 172.21.15.52:0/2229969147 >> 172.21.15.52:0/2229969147 conn(0x5557a6e59ed0 msgr2=0x5557a6e5a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.920+0000 7f90303e0740 1 -- 172.21.15.52:0/2229969147 shutdown_connections 2024-04-02T13:57:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.920+0000 7f90303e0740 1 -- 172.21.15.52:0/2229969147 wait complete. 2024-04-02T13:57:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.921+0000 7f90303e0740 1 -- start start 2024-04-02T13:57:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.921+0000 7f90303e0740 1 Processor -- start 2024-04-02T13:57:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.921+0000 7f90303e0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557a6e5f7a0 0x5557a6e68660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.921+0000 7f90303e0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557a6e600c0 0x5557a6e65a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.921+0000 7f90303e0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e68ca0 0x5557a6e660e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.921+0000 7f90303e0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5557a6e6b850 con 0x5557a6e68ca0 2024-04-02T13:57:57.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.921+0000 7f90303e0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5557a6d7ee80 con 0x5557a6e5f7a0 2024-04-02T13:57:57.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.921+0000 7f90303e0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5557a6e6b9d0 con 0x5557a6e600c0 2024-04-02T13:57:57.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.921+0000 7f90303e0740 10 client.0 _ll_register_callbacks cb 0x5557a6dbaaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:57.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.921+0000 7f90277fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e68ca0 0x5557a6e660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.921+0000 7f90277fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e68ca0 0x5557a6e660e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44370/0 (socket says 172.21.15.52:44370) 2024-04-02T13:57:57.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.921+0000 7f90277fe640 1 -- 172.21.15.52:0/3508750159 learned_addr learned my addr 172.21.15.52:0/3508750159 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:57.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.921+0000 7f9026ffd640 1 --2- 172.21.15.52:0/3508750159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557a6e5f7a0 0x5557a6e68660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.921+0000 7f901e7fc640 1 --2- 172.21.15.52:0/3508750159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557a6e600c0 0x5557a6e65a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.921+0000 7f90277fe640 1 -- 172.21.15.52:0/3508750159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557a6e600c0 msgr2=0x5557a6e65a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.921+0000 7f90277fe640 1 --2- 172.21.15.52:0/3508750159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557a6e600c0 0x5557a6e65a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.921+0000 7f90277fe640 1 -- 172.21.15.52:0/3508750159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557a6e5f7a0 msgr2=0x5557a6e68660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.921+0000 7f90277fe640 1 --2- 172.21.15.52:0/3508750159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557a6e5f7a0 0x5557a6e68660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.921+0000 7f90277fe640 1 -- 172.21.15.52:0/3508750159 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557a6ba17d0 con 0x5557a6e68ca0 2024-04-02T13:57:57.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.922+0000 7f90277fe640 1 --2- 172.21.15.52:0/3508750159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e68ca0 0x5557a6e660e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9020003fd0 tx=0x7f9020003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46fd43071e834470 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.922+0000 7f900e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:57.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.922+0000 7f900e7fc640 1 -- 172.21.15.52:0/3508750159 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90200dc060 con 0x5557a6e68ca0 2024-04-02T13:57:57.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.922+0000 7f900e7fc640 1 -- 172.21.15.52:0/3508750159 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f90200bad10 con 0x5557a6e68ca0 2024-04-02T13:57:57.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.922+0000 7f90303e0740 1 -- 172.21.15.52:0/3508750159 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90200beda0 con 0x5557a6e68ca0 2024-04-02T13:57:57.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.922+0000 7f900e7fc640 1 -- 172.21.15.52:0/3508750159 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f90200be6a0 con 0x5557a6e68ca0 2024-04-02T13:57:57.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.922+0000 7f90303e0740 10 client.16932 fetch_fsmap learned FSMap version 30 2024-04-02T13:57:57.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.922+0000 7f90303e0740 1 -- 172.21.15.52:0/3508750159 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5557a6e6b850 con 0x5557a6e68ca0 2024-04-02T13:57:57.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.922+0000 7f900e7fc640 1 -- 172.21.15.52:0/3508750159 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90200beda0 con 0x5557a6e68ca0 2024-04-02T13:57:57.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.922+0000 7f900e7fc640 1 -- 172.21.15.52:0/3508750159 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f902003c030 con 0x5557a6e68ca0 2024-04-02T13:57:57.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.922+0000 7f90303e0740 10 client.16932 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:57:57.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.922+0000 7f90303e0740 10 client.16932 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:57:57.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.922+0000 7f90303e0740 20 client.16932 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:57.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.922+0000 7f90303e0740 1 --2- 172.21.15.52:0/3508750159 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5557a6e78970 0x5557a6e7ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.922+0000 7f90303e0740 4 client.16932 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:57:57.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.922+0000 7f90303e0740 1 -- 172.21.15.52:0/3508750159 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557a6dd0cf0 con 0x5557a6e78970 2024-04-02T13:57:57.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.923+0000 7f9026ffd640 1 --2- 172.21.15.52:0/3508750159 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5557a6e78970 0x5557a6e7ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.923+0000 7f900e7fc640 10 client.16932 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:57:57.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.923+0000 7f9026ffd640 1 --2- 172.21.15.52:0/3508750159 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5557a6e78970 0x5557a6e7ad60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=71c1a61efa00f46 server_cookie=2f690441f189b502 in_seq=0 out_seq=0 2024-04-02T13:57:57.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.924+0000 7f900e7fc640 1 -- 172.21.15.52:0/3508750159 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5557a6dd0cf0 con 0x5557a6e78970 2024-04-02T13:57:57.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.924+0000 7f900e7fc640 10 client.16932 handle_command_reply: tid=0 2024-04-02T13:57:57.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.924+0000 7f90303e0740 1 client.16932 shutdown 2024-04-02T13:57:57.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.924+0000 7f90303e0740 1 -- 172.21.15.52:0/3508750159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e68ca0 msgr2=0x5557a6e660e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.924+0000 7f90303e0740 1 --2- 172.21.15.52:0/3508750159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e68ca0 0x5557a6e660e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9020003fd0 tx=0x7f9020003ba0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.924+0000 7f90303e0740 1 -- 172.21.15.52:0/3508750159 shutdown_connections 2024-04-02T13:57:57.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.924+0000 7f90303e0740 1 --2- 172.21.15.52:0/3508750159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e68ca0 0x5557a6e660e0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.924+0000 7f90303e0740 1 --2- 172.21.15.52:0/3508750159 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5557a6e78970 0x5557a6e7ad60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.924+0000 7f90303e0740 1 --2- 172.21.15.52:0/3508750159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557a6e600c0 0x5557a6e65a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.924+0000 7f90303e0740 1 --2- 172.21.15.52:0/3508750159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557a6e5f7a0 0x5557a6e68660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.924+0000 7f90303e0740 1 -- 172.21.15.52:0/3508750159 >> 172.21.15.52:0/3508750159 conn(0x5557a6e59ed0 msgr2=0x5557a6e5bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.924+0000 7f900e7fc640 0 client.16932 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:57:57.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.924+0000 7f90303e0740 1 -- 172.21.15.52:0/3508750159 shutdown_connections 2024-04-02T13:57:57.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.924+0000 7f90303e0740 1 -- 172.21.15.52:0/3508750159 wait complete. 2024-04-02T13:57:57.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.924+0000 7f90303e0740 20 client.16932 trim_cache size 0 max 16384 2024-04-02T13:57:57.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.937+0000 7f90303e0740 1 Processor -- start 2024-04-02T13:57:57.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.937+0000 7f90303e0740 1 -- start start 2024-04-02T13:57:57.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.938+0000 7f90303e0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557a6e5f7a0 0x5557a6e74660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.938+0000 7f90303e0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e600c0 0x5557a6e64fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.938+0000 7f90303e0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557a6e68ca0 0x5557a6e80520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.938+0000 7f90303e0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f90281b0fc0 con 0x5557a6e600c0 2024-04-02T13:57:57.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.938+0000 7f90303e0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f90281b2b00 con 0x5557a6e68ca0 2024-04-02T13:57:57.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.938+0000 7f90303e0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f90281b1690 con 0x5557a6e5f7a0 2024-04-02T13:57:57.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.938+0000 7f901e7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e600c0 0x5557a6e64fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.938+0000 7f901e7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e600c0 0x5557a6e64fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44380/0 (socket says 172.21.15.52:44380) 2024-04-02T13:57:57.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.938+0000 7f901e7fc640 1 -- 172.21.15.52:0/1749328499 learned_addr learned my addr 172.21.15.52:0/1749328499 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:57.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.938+0000 7f90277fe640 1 --2- 172.21.15.52:0/1749328499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557a6e68ca0 0x5557a6e80520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.938+0000 7f9026ffd640 1 --2- 172.21.15.52:0/1749328499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557a6e5f7a0 0x5557a6e74660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.938+0000 7f901e7fc640 1 -- 172.21.15.52:0/1749328499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557a6e5f7a0 msgr2=0x5557a6e74660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.938+0000 7f901e7fc640 1 --2- 172.21.15.52:0/1749328499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557a6e5f7a0 0x5557a6e74660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.938+0000 7f901e7fc640 1 -- 172.21.15.52:0/1749328499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557a6e68ca0 msgr2=0x5557a6e80520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.938+0000 7f901e7fc640 1 --2- 172.21.15.52:0/1749328499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557a6e68ca0 0x5557a6e80520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.938+0000 7f901e7fc640 1 -- 172.21.15.52:0/1749328499 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557a6e66980 con 0x5557a6e600c0 2024-04-02T13:57:57.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.938+0000 7f90277fe640 1 --2- 172.21.15.52:0/1749328499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557a6e68ca0 0x5557a6e80520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:57.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.938+0000 7f901e7fc640 1 --2- 172.21.15.52:0/1749328499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e600c0 0x5557a6e64fc0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f9010005890 tx=0x7f9010014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27ef2f76f0a5adc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.938+0000 7f901cff9640 1 -- 172.21.15.52:0/1749328499 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9010023070 con 0x5557a6e600c0 2024-04-02T13:57:57.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.938+0000 7f901cff9640 1 -- 172.21.15.52:0/1749328499 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f901001f930 con 0x5557a6e600c0 2024-04-02T13:57:57.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.938+0000 7f901cff9640 1 -- 172.21.15.52:0/1749328499 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901001ea80 con 0x5557a6e600c0 2024-04-02T13:57:57.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.939+0000 7f90303e0740 1 -- 172.21.15.52:0/1749328499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e600c0 msgr2=0x5557a6e64fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.939+0000 7f90303e0740 1 --2- 172.21.15.52:0/1749328499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e600c0 0x5557a6e64fc0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f9010005890 tx=0x7f9010014000 comp rx=0 tx=0).stop 2024-04-02T13:57:57.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.939+0000 7f90303e0740 1 -- 172.21.15.52:0/1749328499 shutdown_connections 2024-04-02T13:57:57.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.939+0000 7f90303e0740 1 --2- 172.21.15.52:0/1749328499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557a6e68ca0 0x5557a6e80520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.939+0000 7f90303e0740 1 --2- 172.21.15.52:0/1749328499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e600c0 0x5557a6e64fc0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.939+0000 7f90303e0740 1 --2- 172.21.15.52:0/1749328499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557a6e5f7a0 0x5557a6e74660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.939+0000 7f90303e0740 1 -- 172.21.15.52:0/1749328499 >> 172.21.15.52:0/1749328499 conn(0x5557a6e59ed0 msgr2=0x5557a6e5a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.939+0000 7f90303e0740 1 -- 172.21.15.52:0/1749328499 shutdown_connections 2024-04-02T13:57:57.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.939+0000 7f90303e0740 1 -- 172.21.15.52:0/1749328499 wait complete. 2024-04-02T13:57:57.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.939+0000 7f90303e0740 1 -- start start 2024-04-02T13:57:57.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.940+0000 7f90303e0740 1 Processor -- start 2024-04-02T13:57:57.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.940+0000 7f90303e0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557a6e5f7a0 0x5557a6e69790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.940+0000 7f90303e0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e600c0 0x5557a6e69cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.940+0000 7f90303e0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557a6e68ca0 0x5557a6e6fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.940+0000 7f90303e0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f90281b2430 con 0x5557a6e600c0 2024-04-02T13:57:57.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.940+0000 7f90303e0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f90281b1d60 con 0x5557a6e68ca0 2024-04-02T13:57:57.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.940+0000 7f90303e0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f90281aefa0 con 0x5557a6e5f7a0 2024-04-02T13:57:57.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.940+0000 7f90303e0740 10 client.0 _ll_register_callbacks cb 0x7f90200c7310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:57.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.940+0000 7f901e7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e600c0 0x5557a6e69cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.940+0000 7f901e7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e600c0 0x5557a6e69cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44386/0 (socket says 172.21.15.52:44386) 2024-04-02T13:57:57.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.940+0000 7f901e7fc640 1 -- 172.21.15.52:0/3751864541 learned_addr learned my addr 172.21.15.52:0/3751864541 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:57.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.940+0000 7f9026ffd640 1 --2- 172.21.15.52:0/3751864541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557a6e5f7a0 0x5557a6e69790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.940+0000 7f90277fe640 1 --2- 172.21.15.52:0/3751864541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557a6e68ca0 0x5557a6e6fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.940+0000 7f901e7fc640 1 -- 172.21.15.52:0/3751864541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557a6e5f7a0 msgr2=0x5557a6e69790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.940+0000 7f901e7fc640 1 --2- 172.21.15.52:0/3751864541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557a6e5f7a0 0x5557a6e69790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.940+0000 7f901e7fc640 1 -- 172.21.15.52:0/3751864541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557a6e68ca0 msgr2=0x5557a6e6fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:57.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.940+0000 7f901e7fc640 1 --2- 172.21.15.52:0/3751864541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557a6e68ca0 0x5557a6e6fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.940+0000 7f901e7fc640 1 -- 172.21.15.52:0/3751864541 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557a6e6ff70 con 0x5557a6e600c0 2024-04-02T13:57:57.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.940+0000 7f90277fe640 1 --2- 172.21.15.52:0/3751864541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557a6e68ca0 0x5557a6e6fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:57.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.940+0000 7f901e7fc640 1 --2- 172.21.15.52:0/3751864541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e600c0 0x5557a6e69cd0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f901000b5b0 tx=0x7f901001f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9bc07eb687b852 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:57.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.940+0000 7f900e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:57.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.941+0000 7f900e7fc640 1 -- 172.21.15.52:0/3751864541 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9010023070 con 0x5557a6e600c0 2024-04-02T13:57:57.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.941+0000 7f900e7fc640 1 -- 172.21.15.52:0/3751864541 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f901001fcf0 con 0x5557a6e600c0 2024-04-02T13:57:57.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.941+0000 7f90303e0740 1 -- 172.21.15.52:0/3751864541 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f901001f930 con 0x5557a6e600c0 2024-04-02T13:57:57.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.941+0000 7f900e7fc640 1 -- 172.21.15.52:0/3751864541 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9010004020 con 0x5557a6e600c0 2024-04-02T13:57:57.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.941+0000 7f90303e0740 10 client.16938 fetch_fsmap learned FSMap version 30 2024-04-02T13:57:57.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.941+0000 7f90303e0740 1 -- 172.21.15.52:0/3751864541 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5557a6e711e0 con 0x5557a6e600c0 2024-04-02T13:57:57.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.941+0000 7f900e7fc640 1 -- 172.21.15.52:0/3751864541 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f901001f930 con 0x5557a6e600c0 2024-04-02T13:57:57.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.941+0000 7f900e7fc640 1 -- 172.21.15.52:0/3751864541 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f901002f020 con 0x5557a6e600c0 2024-04-02T13:57:57.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.941+0000 7f90303e0740 10 client.16938 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:57:57.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.941+0000 7f90303e0740 10 client.16938 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:57:57.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.941+0000 7f90303e0740 20 client.16938 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:57.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.941+0000 7f90303e0740 1 --2- 172.21.15.52:0/3751864541 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5557a6edef70 0x5557a6ee1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:57.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.941+0000 7f90303e0740 4 client.16938 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:57:57.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.941+0000 7f90303e0740 1 -- 172.21.15.52:0/3751864541 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557a6dcb040 con 0x5557a6edef70 2024-04-02T13:57:57.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.941+0000 7f9026ffd640 1 --2- 172.21.15.52:0/3751864541 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5557a6edef70 0x5557a6ee1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:57.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.942+0000 7f900e7fc640 10 client.16938 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:57:57.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.942+0000 7f9026ffd640 1 --2- 172.21.15.52:0/3751864541 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5557a6edef70 0x5557a6ee1390 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e24a052aca0df809 server_cookie=ce996c20bab1474d in_seq=0 out_seq=0 2024-04-02T13:57:57.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.942+0000 7f900e7fc640 1 -- 172.21.15.52:0/3751864541 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557a6dcb040 con 0x5557a6edef70 2024-04-02T13:57:57.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.942+0000 7f900e7fc640 10 client.16938 handle_command_reply: tid=0 2024-04-02T13:57:57.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.942+0000 7f90303e0740 1 client.16938 shutdown 2024-04-02T13:57:57.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.942+0000 7f90303e0740 1 -- 172.21.15.52:0/3751864541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e600c0 msgr2=0x5557a6e69cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.942+0000 7f90303e0740 1 --2- 172.21.15.52:0/3751864541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e600c0 0x5557a6e69cd0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f901000b5b0 tx=0x7f901001f680 comp rx=0 tx=0).stop 2024-04-02T13:57:57.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.942+0000 7f90303e0740 1 -- 172.21.15.52:0/3751864541 shutdown_connections 2024-04-02T13:57:57.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.942+0000 7f90303e0740 1 --2- 172.21.15.52:0/3751864541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557a6e68ca0 0x5557a6e6fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.942+0000 7f90303e0740 1 --2- 172.21.15.52:0/3751864541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5557a6e600c0 0x5557a6e69cd0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.942+0000 7f90303e0740 1 --2- 172.21.15.52:0/3751864541 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5557a6edef70 0x5557a6ee1390 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.942+0000 7f90303e0740 1 --2- 172.21.15.52:0/3751864541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557a6e5f7a0 0x5557a6e69790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.942+0000 7f900e7fc640 0 client.16938 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:57:57.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.942+0000 7f90303e0740 1 -- 172.21.15.52:0/3751864541 >> 172.21.15.52:0/3751864541 conn(0x5557a6e59ed0 msgr2=0x5557a6e5b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.942+0000 7f90303e0740 1 -- 172.21.15.52:0/3751864541 shutdown_connections 2024-04-02T13:57:57.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.942+0000 7f90303e0740 1 -- 172.21.15.52:0/3751864541 wait complete. 2024-04-02T13:57:57.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.942+0000 7f90303e0740 20 client.16938 trim_cache size 0 max 16384 2024-04-02T13:57:57.959 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:57:57.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.943+0000 7f902d722640 1 -- 172.21.15.52:0/3579845334 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f90140a3da0 msgr2=0x7f90140a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.943+0000 7f902d722640 1 --2- 172.21.15.52:0/3579845334 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f90140a3da0 0x7f90140a6260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f90100058c0 tx=0x7f901000f730 comp rx=0 tx=0).stop 2024-04-02T13:57:57.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.943+0000 7f902d722640 1 -- 172.21.15.52:0/3579845334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90281069a0 msgr2=0x7f9028103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:57.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.943+0000 7f902d722640 1 --2- 172.21.15.52:0/3579845334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90281069a0 0x7f9028103990 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f90200348f0 tx=0x7f9020032ec0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.943+0000 7f902d722640 1 -- 172.21.15.52:0/3579845334 shutdown_connections 2024-04-02T13:57:57.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.943+0000 7f902d722640 1 --2- 172.21.15.52:0/3579845334 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f90140a3da0 0x7f90140a6260 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.943+0000 7f902d722640 1 --2- 172.21.15.52:0/3579845334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90281069a0 0x7f9028103990 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.943+0000 7f902d722640 1 --2- 172.21.15.52:0/3579845334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9028101350 0x7f9028103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.943+0000 7f902d722640 1 --2- 172.21.15.52:0/3579845334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9028100990 0x7f9028102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:57.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.943+0000 7f902d722640 1 -- 172.21.15.52:0/3579845334 >> 172.21.15.52:0/3579845334 conn(0x7f9028077ad0 msgr2=0x7f902806a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:57.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.943+0000 7f902d722640 1 -- 172.21.15.52:0/3579845334 shutdown_connections 2024-04-02T13:57:57.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:57.943+0000 7f902d722640 1 -- 172.21.15.52:0/3579845334 wait complete. 2024-04-02T13:57:57.962 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:57:58.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.065+0000 7fd636b50640 1 -- 172.21.15.52:0/3475601229 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd630101330 msgr2=0x7fd63010d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:58.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.065+0000 7fd636b50640 1 --2- 172.21.15.52:0/3475601229 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd630101330 0x7fd63010d840 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fd620009930 tx=0x7fd62002bc30 comp rx=0 tx=0).stop 2024-04-02T13:57:58.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.065+0000 7fd636b50640 1 -- 172.21.15.52:0/3475601229 shutdown_connections 2024-04-02T13:57:58.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.065+0000 7fd636b50640 1 --2- 172.21.15.52:0/3475601229 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd630101330 0x7fd63010d840 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.065+0000 7fd636b50640 1 --2- 172.21.15.52:0/3475601229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd630100970 0x7fd630100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.065+0000 7fd636b50640 1 --2- 172.21.15.52:0/3475601229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd630106980 0x7fd630106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.065+0000 7fd636b50640 1 -- 172.21.15.52:0/3475601229 >> 172.21.15.52:0/3475601229 conn(0x7fd630077b00 msgr2=0x7fd630077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:58.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.065+0000 7fd636b50640 1 -- 172.21.15.52:0/3475601229 shutdown_connections 2024-04-02T13:57:58.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.065+0000 7fd636b50640 1 -- 172.21.15.52:0/3475601229 wait complete. 2024-04-02T13:57:58.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.065+0000 7fd636b50640 1 Processor -- start 2024-04-02T13:57:58.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.065+0000 7fd636b50640 1 -- start start 2024-04-02T13:57:58.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.065+0000 7fd636b50640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd630100970 0x7fd630198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:58.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.065+0000 7fd636b50640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd630101330 0x7fd6301988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:58.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.065+0000 7fd636b50640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd630106980 0x7fd63019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:58.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.065+0000 7fd636b50640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd630110d90 con 0x7fd630100970 2024-04-02T13:57:58.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.066+0000 7fd636b50640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd630110c10 con 0x7fd630106980 2024-04-02T13:57:58.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.066+0000 7fd636b50640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd630110f10 con 0x7fd630101330 2024-04-02T13:57:58.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.066+0000 7fd6348ec640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd630100970 0x7fd630198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:58.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.066+0000 7fd6348ec640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd630100970 0x7fd630198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44408/0 (socket says 172.21.15.52:44408) 2024-04-02T13:57:58.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.066+0000 7fd6348ec640 1 -- 172.21.15.52:0/318557563 learned_addr learned my addr 172.21.15.52:0/318557563 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:58.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.066+0000 7fd6350ed640 1 --2- 172.21.15.52:0/318557563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd630106980 0x7fd63019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:58.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.066+0000 7fd62ffff640 1 --2- 172.21.15.52:0/318557563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd630101330 0x7fd6301988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:58.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.066+0000 7fd6348ec640 1 -- 172.21.15.52:0/318557563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd630101330 msgr2=0x7fd6301988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:58.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.066+0000 7fd6348ec640 1 --2- 172.21.15.52:0/318557563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd630101330 0x7fd6301988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.066+0000 7fd6348ec640 1 -- 172.21.15.52:0/318557563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd630106980 msgr2=0x7fd63019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:58.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.066+0000 7fd6348ec640 1 --2- 172.21.15.52:0/318557563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd630106980 0x7fd63019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.066+0000 7fd6348ec640 1 -- 172.21.15.52:0/318557563 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63019d350 con 0x7fd630100970 2024-04-02T13:57:58.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.066+0000 7fd6348ec640 1 --2- 172.21.15.52:0/318557563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd630100970 0x7fd630198360 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fd62800ea80 tx=0x7fd62800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:58.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.066+0000 7fd62dffb640 1 -- 172.21.15.52:0/318557563 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62800ce40 con 0x7fd630100970 2024-04-02T13:57:58.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.066+0000 7fd62dffb640 1 -- 172.21.15.52:0/318557563 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd628005020 con 0x7fd630100970 2024-04-02T13:57:58.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.066+0000 7fd636b50640 1 -- 172.21.15.52:0/318557563 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63019d640 con 0x7fd630100970 2024-04-02T13:57:58.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.066+0000 7fd62dffb640 1 -- 172.21.15.52:0/318557563 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628011760 con 0x7fd630100970 2024-04-02T13:57:58.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.066+0000 7fd636b50640 1 -- 172.21.15.52:0/318557563 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6301a4ed0 con 0x7fd630100970 2024-04-02T13:57:58.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.069+0000 7fd62dffb640 1 -- 172.21.15.52:0/318557563 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd6280051d0 con 0x7fd630100970 2024-04-02T13:57:58.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.069+0000 7fd639814740 1 Processor -- start 2024-04-02T13:57:58.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.069+0000 7fd639814740 1 -- start start 2024-04-02T13:57:58.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.069+0000 7fd62dffb640 1 --2- 172.21.15.52:0/318557563 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd6180a3da0 0x7fd6180a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:58.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.069+0000 7fd639814740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576ba62d7a0 0x5576ba62db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:58.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.069+0000 7fd639814740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576ba62e0c0 0x5576ba636760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:58.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.069+0000 7fd639814740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba636ca0 0x5576ba639090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:58.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.069+0000 7fd639814740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5576ba5917b0 con 0x5576ba636ca0 2024-04-02T13:57:58.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.069+0000 7fd639814740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5576ba530570 con 0x5576ba62d7a0 2024-04-02T13:57:58.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.069+0000 7fd639814740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5576ba58d760 con 0x5576ba62e0c0 2024-04-02T13:57:58.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.069+0000 7fd62dffb640 1 -- 172.21.15.52:0/318557563 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd6280d1140 con 0x7fd630100970 2024-04-02T13:57:58.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.069+0000 7fd6350ed640 1 --2- 172.21.15.52:0/318557563 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd6180a3da0 0x7fd6180a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:58.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.069+0000 7fd6350ed640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba636ca0 0x5576ba639090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:58.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.069+0000 7fd6350ed640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba636ca0 0x5576ba639090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44418/0 (socket says 172.21.15.52:44418) 2024-04-02T13:57:58.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.070+0000 7fd6350ed640 1 -- 172.21.15.52:0/3084139989 learned_addr learned my addr 172.21.15.52:0/3084139989 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:58.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.070+0000 7fd6348ec640 1 --2- 172.21.15.52:0/3084139989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576ba62d7a0 0x5576ba62db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:58.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.070+0000 7fd62ffff640 1 --2- 172.21.15.52:0/3084139989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576ba62e0c0 0x5576ba636760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:58.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.070+0000 7fd6350ed640 1 -- 172.21.15.52:0/3084139989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576ba62e0c0 msgr2=0x5576ba636760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:58.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.070+0000 7fd6350ed640 1 --2- 172.21.15.52:0/3084139989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576ba62e0c0 0x5576ba636760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.070+0000 7fd6350ed640 1 -- 172.21.15.52:0/3084139989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576ba62d7a0 msgr2=0x5576ba62db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:58.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.070+0000 7fd6350ed640 1 --2- 172.21.15.52:0/3084139989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576ba62d7a0 0x5576ba62db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.070+0000 7fd6350ed640 1 -- 172.21.15.52:0/3084139989 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576ba2a53f0 con 0x5576ba636ca0 2024-04-02T13:57:58.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.070+0000 7fd6350ed640 1 --2- 172.21.15.52:0/318557563 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd6180a3da0 0x7fd6180a6260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd620005ec0 tx=0x7fd620005dd0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:58.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.070+0000 7fd6350ed640 1 --2- 172.21.15.52:0/3084139989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba636ca0 0x5576ba639090 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fd620005ab0 tx=0x7fd620046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92bf06822bc14e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:58.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.070+0000 7fd616ffd640 1 -- 172.21.15.52:0/3084139989 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620005310 con 0x5576ba636ca0 2024-04-02T13:57:58.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.070+0000 7fd616ffd640 1 -- 172.21.15.52:0/3084139989 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd620050070 con 0x5576ba636ca0 2024-04-02T13:57:58.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.070+0000 7fd616ffd640 1 -- 172.21.15.52:0/3084139989 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620003810 con 0x5576ba636ca0 2024-04-02T13:57:58.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.071+0000 7fd639814740 1 -- 172.21.15.52:0/3084139989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba636ca0 msgr2=0x5576ba639090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:58.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.071+0000 7fd639814740 1 --2- 172.21.15.52:0/3084139989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba636ca0 0x5576ba639090 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fd620005ab0 tx=0x7fd620046000 comp rx=0 tx=0).stop 2024-04-02T13:57:58.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.071+0000 7fd639814740 1 -- 172.21.15.52:0/3084139989 shutdown_connections 2024-04-02T13:57:58.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.071+0000 7fd639814740 1 --2- 172.21.15.52:0/3084139989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba636ca0 0x5576ba639090 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.071+0000 7fd639814740 1 --2- 172.21.15.52:0/3084139989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576ba62e0c0 0x5576ba636760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.071+0000 7fd639814740 1 --2- 172.21.15.52:0/3084139989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576ba62d7a0 0x5576ba62db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.071+0000 7fd639814740 1 -- 172.21.15.52:0/3084139989 >> 172.21.15.52:0/3084139989 conn(0x5576ba627ed0 msgr2=0x5576ba6282b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:58.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.071+0000 7fd639814740 1 -- 172.21.15.52:0/3084139989 shutdown_connections 2024-04-02T13:57:58.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.071+0000 7fd639814740 1 -- 172.21.15.52:0/3084139989 wait complete. 2024-04-02T13:57:58.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.071+0000 7fd639814740 1 -- start start 2024-04-02T13:57:58.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.071+0000 7fd639814740 1 Processor -- start 2024-04-02T13:57:58.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.072+0000 7fd639814740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba62d7a0 0x5576ba6366b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:58.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.072+0000 7fd639814740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576ba62e0c0 0x5576ba633a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:58.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.072+0000 7fd639814740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576ba636ca0 0x5576ba634050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:58.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.072+0000 7fd639814740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5576ba63de40 con 0x5576ba62d7a0 2024-04-02T13:57:58.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.072+0000 7fd639814740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5576ba54ce80 con 0x5576ba62e0c0 2024-04-02T13:57:58.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.072+0000 7fd639814740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5576ba63dfc0 con 0x5576ba636ca0 2024-04-02T13:57:58.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.072+0000 7fd639814740 10 client.0 _ll_register_callbacks cb 0x5576ba588aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:58.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.072+0000 7fd6348ec640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba62d7a0 0x5576ba6366b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:58.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.072+0000 7fd6348ec640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba62d7a0 0x5576ba6366b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44434/0 (socket says 172.21.15.52:44434) 2024-04-02T13:57:58.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.072+0000 7fd6348ec640 1 -- 172.21.15.52:0/1752594195 learned_addr learned my addr 172.21.15.52:0/1752594195 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:58.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.072+0000 7fd62ffff640 1 --2- 172.21.15.52:0/1752594195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576ba62e0c0 0x5576ba633a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:58.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.072+0000 7fd6350ed640 1 --2- 172.21.15.52:0/1752594195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576ba636ca0 0x5576ba634050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:58.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.072+0000 7fd6348ec640 1 -- 172.21.15.52:0/1752594195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576ba636ca0 msgr2=0x5576ba634050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:58.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.072+0000 7fd6348ec640 1 --2- 172.21.15.52:0/1752594195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576ba636ca0 0x5576ba634050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.072+0000 7fd6348ec640 1 -- 172.21.15.52:0/1752594195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576ba62e0c0 msgr2=0x5576ba633a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:58.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.072+0000 7fd6348ec640 1 --2- 172.21.15.52:0/1752594195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576ba62e0c0 0x5576ba633a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.072+0000 7fd6348ec640 1 -- 172.21.15.52:0/1752594195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576ba36f7d0 con 0x5576ba62d7a0 2024-04-02T13:57:58.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.072+0000 7fd62ffff640 1 --2- 172.21.15.52:0/1752594195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576ba62e0c0 0x5576ba633a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:58.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.072+0000 7fd6348ec640 1 --2- 172.21.15.52:0/1752594195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba62d7a0 0x5576ba6366b0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd6280ce5b0 tx=0x7fd62809d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf3dfbf49fe1b9e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:58.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.072+0000 7fd60ffff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:58.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.072+0000 7fd60ffff640 1 -- 172.21.15.52:0/1752594195 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62801f050 con 0x5576ba62d7a0 2024-04-02T13:57:58.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.072+0000 7fd60ffff640 1 -- 172.21.15.52:0/1752594195 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd6280ce7d0 con 0x5576ba62d7a0 2024-04-02T13:57:58.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.072+0000 7fd639814740 1 -- 172.21.15.52:0/1752594195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd620050070 con 0x5576ba62d7a0 2024-04-02T13:57:58.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.073+0000 7fd60ffff640 1 -- 172.21.15.52:0/1752594195 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd6280a5a10 con 0x5576ba62d7a0 2024-04-02T13:57:58.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.073+0000 7fd639814740 10 client.16950 fetch_fsmap learned FSMap version 30 2024-04-02T13:57:58.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.073+0000 7fd639814740 1 -- 172.21.15.52:0/1752594195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5576ba649f80 con 0x5576ba62d7a0 2024-04-02T13:57:58.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.073+0000 7fd60ffff640 1 -- 172.21.15.52:0/1752594195 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd620050070 con 0x5576ba62d7a0 2024-04-02T13:57:58.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.073+0000 7fd60ffff640 1 -- 172.21.15.52:0/1752594195 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd62800f040 con 0x5576ba62d7a0 2024-04-02T13:57:58.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.073+0000 7fd639814740 10 client.16950 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:57:58.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.073+0000 7fd639814740 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:57:58.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.073+0000 7fd639814740 20 client.16950 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:58.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.073+0000 7fd639814740 1 --2- 172.21.15.52:0/1752594195 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5576ba64b430 0x5576ba64d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:58.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.073+0000 7fd639814740 4 client.16950 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:57:58.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.073+0000 7fd639814740 1 -- 172.21.15.52:0/1752594195 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576ba59ecf0 con 0x5576ba64b430 2024-04-02T13:57:58.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.073+0000 7fd62ffff640 1 --2- 172.21.15.52:0/1752594195 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5576ba64b430 0x5576ba64d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:58.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.074+0000 7fd62ffff640 1 --2- 172.21.15.52:0/1752594195 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5576ba64b430 0x5576ba64d880 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74c716b3a22c2a6a server_cookie=233ea135bc99a16a in_seq=0 out_seq=0 2024-04-02T13:57:58.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.074+0000 7fd60ffff640 10 client.16950 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:58.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.074+0000 7fd60ffff640 1 -- 172.21.15.52:0/1752594195 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5576ba59ecf0 con 0x5576ba64b430 2024-04-02T13:57:58.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.074+0000 7fd60ffff640 10 client.16950 handle_command_reply: tid=0 2024-04-02T13:57:58.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.074+0000 7fd639814740 1 client.16950 shutdown 2024-04-02T13:57:58.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.075+0000 7fd639814740 1 -- 172.21.15.52:0/1752594195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba62d7a0 msgr2=0x5576ba6366b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:58.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.075+0000 7fd639814740 1 --2- 172.21.15.52:0/1752594195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba62d7a0 0x5576ba6366b0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd6280ce5b0 tx=0x7fd62809d560 comp rx=0 tx=0).stop 2024-04-02T13:57:58.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.075+0000 7fd639814740 1 -- 172.21.15.52:0/1752594195 shutdown_connections 2024-04-02T13:57:58.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.075+0000 7fd639814740 1 --2- 172.21.15.52:0/1752594195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576ba636ca0 0x5576ba634050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.075+0000 7fd639814740 1 --2- 172.21.15.52:0/1752594195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576ba62e0c0 0x5576ba633a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.075+0000 7fd639814740 1 --2- 172.21.15.52:0/1752594195 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5576ba64b430 0x5576ba64d880 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.075+0000 7fd639814740 1 --2- 172.21.15.52:0/1752594195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba62d7a0 0x5576ba6366b0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.075+0000 7fd60ffff640 0 client.16950 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:58.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.075+0000 7fd639814740 1 -- 172.21.15.52:0/1752594195 >> 172.21.15.52:0/1752594195 conn(0x5576ba627ed0 msgr2=0x5576ba629c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:58.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.075+0000 7fd639814740 1 -- 172.21.15.52:0/1752594195 shutdown_connections 2024-04-02T13:57:58.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.075+0000 7fd639814740 1 -- 172.21.15.52:0/1752594195 wait complete. 2024-04-02T13:57:58.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.075+0000 7fd639814740 20 client.16950 trim_cache size 0 max 16384 2024-04-02T13:57:58.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.088+0000 7fd639814740 1 Processor -- start 2024-04-02T13:57:58.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.088+0000 7fd639814740 1 -- start start 2024-04-02T13:57:58.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.089+0000 7fd639814740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576ba62d7a0 0x5576ba646db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:58.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.089+0000 7fd639814740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba62e0c0 0x5576ba636170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:58.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.089+0000 7fd639814740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576ba636ca0 0x5576ba6366b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:58.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.089+0000 7fd639814740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd6301acc00 con 0x5576ba62e0c0 2024-04-02T13:57:58.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.089+0000 7fd639814740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd6301ae740 con 0x5576ba636ca0 2024-04-02T13:57:58.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.089+0000 7fd639814740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd6301ad2d0 con 0x5576ba62d7a0 2024-04-02T13:57:58.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.089+0000 7fd62ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba62e0c0 0x5576ba636170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:58.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.089+0000 7fd62ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba62e0c0 0x5576ba636170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44442/0 (socket says 172.21.15.52:44442) 2024-04-02T13:57:58.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.089+0000 7fd62ffff640 1 -- 172.21.15.52:0/3986777540 learned_addr learned my addr 172.21.15.52:0/3986777540 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:58.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.089+0000 7fd6348ec640 1 --2- 172.21.15.52:0/3986777540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576ba62d7a0 0x5576ba646db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:58.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.089+0000 7fd62ffff640 1 -- 172.21.15.52:0/3986777540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576ba62d7a0 msgr2=0x5576ba646db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:58.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.089+0000 7fd62ffff640 1 --2- 172.21.15.52:0/3986777540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576ba62d7a0 0x5576ba646db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.089+0000 7fd62ffff640 1 -- 172.21.15.52:0/3986777540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576ba636ca0 msgr2=0x5576ba6366b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:58.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.089+0000 7fd6350ed640 1 --2- 172.21.15.52:0/3986777540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576ba636ca0 0x5576ba6366b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:58.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.089+0000 7fd62ffff640 1 --2- 172.21.15.52:0/3986777540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576ba636ca0 0x5576ba6366b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.089+0000 7fd62ffff640 1 -- 172.21.15.52:0/3986777540 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576ba62def0 con 0x5576ba62e0c0 2024-04-02T13:57:58.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.089+0000 7fd62ffff640 1 --2- 172.21.15.52:0/3986777540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba62e0c0 0x5576ba636170 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x5576ba6277d0 tx=0x7fd624013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffb66d8c44f24f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:58.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.089+0000 7fd6167fc640 1 -- 172.21.15.52:0/3986777540 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62401d430 con 0x5576ba62e0c0 2024-04-02T13:57:58.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.089+0000 7fd6167fc640 1 -- 172.21.15.52:0/3986777540 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd624011440 con 0x5576ba62e0c0 2024-04-02T13:57:58.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.089+0000 7fd6167fc640 1 -- 172.21.15.52:0/3986777540 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62401c630 con 0x5576ba62e0c0 2024-04-02T13:57:58.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.090+0000 7fd639814740 1 -- 172.21.15.52:0/3986777540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba62e0c0 msgr2=0x5576ba636170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:58.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.090+0000 7fd639814740 1 --2- 172.21.15.52:0/3986777540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba62e0c0 0x5576ba636170 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x5576ba6277d0 tx=0x7fd624013830 comp rx=0 tx=0).stop 2024-04-02T13:57:58.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.090+0000 7fd639814740 1 -- 172.21.15.52:0/3986777540 shutdown_connections 2024-04-02T13:57:58.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.090+0000 7fd639814740 1 --2- 172.21.15.52:0/3986777540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576ba636ca0 0x5576ba6366b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.090+0000 7fd639814740 1 --2- 172.21.15.52:0/3986777540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba62e0c0 0x5576ba636170 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.090+0000 7fd639814740 1 --2- 172.21.15.52:0/3986777540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576ba62d7a0 0x5576ba646db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.090+0000 7fd639814740 1 -- 172.21.15.52:0/3986777540 >> 172.21.15.52:0/3986777540 conn(0x5576ba627ed0 msgr2=0x5576ba62ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:58.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.090+0000 7fd639814740 1 -- 172.21.15.52:0/3986777540 shutdown_connections 2024-04-02T13:57:58.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.090+0000 7fd639814740 1 -- 172.21.15.52:0/3986777540 wait complete. 2024-04-02T13:57:58.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.090+0000 7fd639814740 1 -- start start 2024-04-02T13:57:58.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.090+0000 7fd639814740 1 Processor -- start 2024-04-02T13:57:58.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.091+0000 7fd639814740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576ba62d7a0 0x5576ba643750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:58.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.091+0000 7fd639814740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba62e0c0 0x5576ba631010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:58.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.091+0000 7fd639814740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576ba636ca0 0x5576ba631620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:58.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.091+0000 7fd639814740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd6301ae070 con 0x5576ba62e0c0 2024-04-02T13:57:58.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.091+0000 7fd639814740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd6301ad9a0 con 0x5576ba62d7a0 2024-04-02T13:57:58.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.091+0000 7fd639814740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd6301aabe0 con 0x5576ba636ca0 2024-04-02T13:57:58.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.091+0000 7fd639814740 10 client.0 _ll_register_callbacks cb 0x7fd62809c7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:58.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.091+0000 7fd62ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba62e0c0 0x5576ba631010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:58.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.091+0000 7fd62ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba62e0c0 0x5576ba631010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44450/0 (socket says 172.21.15.52:44450) 2024-04-02T13:57:58.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.091+0000 7fd62ffff640 1 -- 172.21.15.52:0/3407582807 learned_addr learned my addr 172.21.15.52:0/3407582807 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:58.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.091+0000 7fd6350ed640 1 --2- 172.21.15.52:0/3407582807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576ba636ca0 0x5576ba631620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:58.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.091+0000 7fd6348ec640 1 --2- 172.21.15.52:0/3407582807 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576ba62d7a0 0x5576ba643750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:58.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.091+0000 7fd62ffff640 1 -- 172.21.15.52:0/3407582807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576ba636ca0 msgr2=0x5576ba631620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:58.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.091+0000 7fd62ffff640 1 --2- 172.21.15.52:0/3407582807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576ba636ca0 0x5576ba631620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.091+0000 7fd62ffff640 1 -- 172.21.15.52:0/3407582807 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576ba62d7a0 msgr2=0x5576ba643750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:58.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.091+0000 7fd62ffff640 1 --2- 172.21.15.52:0/3407582807 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576ba62d7a0 0x5576ba643750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.091+0000 7fd62ffff640 1 -- 172.21.15.52:0/3407582807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576ba637df0 con 0x5576ba62e0c0 2024-04-02T13:57:58.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.091+0000 7fd62ffff640 1 --2- 172.21.15.52:0/3407582807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba62e0c0 0x5576ba631010 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd624020040 tx=0x7fd624012dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b1d5a0e3bbee2e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:58.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.091+0000 7fd60ffff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:58.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.092+0000 7fd60ffff640 1 -- 172.21.15.52:0/3407582807 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62401d640 con 0x5576ba62e0c0 2024-04-02T13:57:58.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.092+0000 7fd60ffff640 1 -- 172.21.15.52:0/3407582807 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd62401d7e0 con 0x5576ba62e0c0 2024-04-02T13:57:58.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.092+0000 7fd639814740 1 -- 172.21.15.52:0/3407582807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd624011440 con 0x5576ba62e0c0 2024-04-02T13:57:58.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.092+0000 7fd60ffff640 1 -- 172.21.15.52:0/3407582807 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd624039050 con 0x5576ba62e0c0 2024-04-02T13:57:58.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.092+0000 7fd639814740 10 client.16956 fetch_fsmap learned FSMap version 30 2024-04-02T13:57:58.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.092+0000 7fd639814740 1 -- 172.21.15.52:0/3407582807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5576ba6b1e20 con 0x5576ba62e0c0 2024-04-02T13:57:58.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.092+0000 7fd60ffff640 1 -- 172.21.15.52:0/3407582807 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd624011440 con 0x5576ba62e0c0 2024-04-02T13:57:58.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.092+0000 7fd60ffff640 1 -- 172.21.15.52:0/3407582807 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd62402e9e0 con 0x5576ba62e0c0 2024-04-02T13:57:58.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.092+0000 7fd639814740 10 client.16956 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:57:58.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.092+0000 7fd639814740 10 client.16956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:57:58.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.092+0000 7fd639814740 20 client.16956 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:58.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.092+0000 7fd639814740 1 --2- 172.21.15.52:0/3407582807 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5576ba6b2570 0x5576ba6b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:58.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.092+0000 7fd639814740 4 client.16956 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:57:58.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.092+0000 7fd639814740 1 -- 172.21.15.52:0/3407582807 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5576ba599040 con 0x5576ba6b2570 2024-04-02T13:57:58.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.092+0000 7fd6348ec640 1 --2- 172.21.15.52:0/3407582807 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5576ba6b2570 0x5576ba6b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:58.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.093+0000 7fd60ffff640 10 client.16956 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:58.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.093+0000 7fd6348ec640 1 --2- 172.21.15.52:0/3407582807 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5576ba6b2570 0x5576ba6b4990 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbb111227cf6fce6 server_cookie=7cfe513a451632de in_seq=0 out_seq=0 2024-04-02T13:57:58.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.093+0000 7fd60ffff640 1 -- 172.21.15.52:0/3407582807 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5576ba599040 con 0x5576ba6b2570 2024-04-02T13:57:58.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.093+0000 7fd60ffff640 10 client.16956 handle_command_reply: tid=0 2024-04-02T13:57:58.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.093+0000 7fd639814740 1 client.16956 shutdown 2024-04-02T13:57:58.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.093+0000 7fd639814740 1 -- 172.21.15.52:0/3407582807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba62e0c0 msgr2=0x5576ba631010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:58.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.093+0000 7fd639814740 1 --2- 172.21.15.52:0/3407582807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba62e0c0 0x5576ba631010 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd624020040 tx=0x7fd624012dd0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.093+0000 7fd639814740 1 -- 172.21.15.52:0/3407582807 shutdown_connections 2024-04-02T13:57:58.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.093+0000 7fd639814740 1 --2- 172.21.15.52:0/3407582807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576ba636ca0 0x5576ba631620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.093+0000 7fd639814740 1 --2- 172.21.15.52:0/3407582807 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5576ba6b2570 0x5576ba6b4990 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.093+0000 7fd639814740 1 --2- 172.21.15.52:0/3407582807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5576ba62e0c0 0x5576ba631010 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.093+0000 7fd639814740 1 --2- 172.21.15.52:0/3407582807 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576ba62d7a0 0x5576ba643750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.093+0000 7fd60ffff640 0 client.16956 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:58.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.093+0000 7fd639814740 1 -- 172.21.15.52:0/3407582807 >> 172.21.15.52:0/3407582807 conn(0x5576ba627ed0 msgr2=0x5576ba629300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:58.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.093+0000 7fd639814740 1 -- 172.21.15.52:0/3407582807 shutdown_connections 2024-04-02T13:57:58.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.093+0000 7fd639814740 1 -- 172.21.15.52:0/3407582807 wait complete. 2024-04-02T13:57:58.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.093+0000 7fd639814740 20 client.16956 trim_cache size 0 max 16384 2024-04-02T13:57:58.109 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:57:58.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.094+0000 7fd636b50640 1 -- 172.21.15.52:0/318557563 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd6180a3da0 msgr2=0x7fd6180a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:58.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.094+0000 7fd636b50640 1 --2- 172.21.15.52:0/318557563 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd6180a3da0 0x7fd6180a6260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd620005ec0 tx=0x7fd620005dd0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.094+0000 7fd636b50640 1 -- 172.21.15.52:0/318557563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd630100970 msgr2=0x7fd630198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:58.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.094+0000 7fd636b50640 1 --2- 172.21.15.52:0/318557563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd630100970 0x7fd630198360 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fd62800ea80 tx=0x7fd62800ef40 comp rx=0 tx=0).stop 2024-04-02T13:57:58.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.094+0000 7fd636b50640 1 -- 172.21.15.52:0/318557563 shutdown_connections 2024-04-02T13:57:58.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.094+0000 7fd636b50640 1 --2- 172.21.15.52:0/318557563 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd6180a3da0 0x7fd6180a6260 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.094+0000 7fd636b50640 1 --2- 172.21.15.52:0/318557563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd630106980 0x7fd63019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.094+0000 7fd636b50640 1 --2- 172.21.15.52:0/318557563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd630101330 0x7fd6301988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.094+0000 7fd636b50640 1 --2- 172.21.15.52:0/318557563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd630100970 0x7fd630198360 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:58.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.094+0000 7fd636b50640 1 -- 172.21.15.52:0/318557563 >> 172.21.15.52:0/318557563 conn(0x7fd630077b00 msgr2=0x7fd6300ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:58.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.094+0000 7fd636b50640 1 -- 172.21.15.52:0/318557563 shutdown_connections 2024-04-02T13:57:58.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:58.094+0000 7fd636b50640 1 -- 172.21.15.52:0/318557563 wait complete. 2024-04-02T13:57:58.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:58 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3919768972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:57:58.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:58 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3919768972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:57:58.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:58 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3919768972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:57:59.113 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:57:59.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:57:59.114 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:57:59.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.229+0000 7f5c12da8640 1 -- 172.21.15.52:0/2844922061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c0c107ab0 msgr2=0x7f5c0c107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:59.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.229+0000 7f5c12da8640 1 --2- 172.21.15.52:0/2844922061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c0c107ab0 0x7f5c0c107e90 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009930 tx=0x7f5c0002bc30 comp rx=0 tx=0).stop 2024-04-02T13:57:59.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.229+0000 7f5c12da8640 1 -- 172.21.15.52:0/2844922061 shutdown_connections 2024-04-02T13:57:59.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.229+0000 7f5c12da8640 1 --2- 172.21.15.52:0/2844922061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c0c102460 0x7f5c0c10e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.229+0000 7f5c12da8640 1 --2- 172.21.15.52:0/2844922061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c0c101aa0 0x7f5c0c101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.229+0000 7f5c12da8640 1 --2- 172.21.15.52:0/2844922061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c0c107ab0 0x7f5c0c107e90 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.229+0000 7f5c12da8640 1 -- 172.21.15.52:0/2844922061 >> 172.21.15.52:0/2844922061 conn(0x7f5c0c0fd530 msgr2=0x7f5c0c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:59.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.229+0000 7f5c12da8640 1 -- 172.21.15.52:0/2844922061 shutdown_connections 2024-04-02T13:57:59.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.229+0000 7f5c12da8640 1 -- 172.21.15.52:0/2844922061 wait complete. 2024-04-02T13:57:59.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.230+0000 7f5c12da8640 1 Processor -- start 2024-04-02T13:57:59.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.230+0000 7f5c12da8640 1 -- start start 2024-04-02T13:57:59.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.230+0000 7f5c12da8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c0c101aa0 0x7f5c0c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.230+0000 7f5c12da8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c0c102460 0x7f5c0c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.230+0000 7f5c12da8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c0c107ab0 0x7f5c0c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.230+0000 7f5c12da8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c111e90 con 0x7f5c0c101aa0 2024-04-02T13:57:59.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.230+0000 7f5c12da8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c111d10 con 0x7f5c0c107ab0 2024-04-02T13:57:59.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.230+0000 7f5c12da8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c112010 con 0x7f5c0c102460 2024-04-02T13:57:59.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.230+0000 7f5c10b44640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c0c101aa0 0x7f5c0c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.230+0000 7f5c10b44640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c0c101aa0 0x7f5c0c198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44464/0 (socket says 172.21.15.52:44464) 2024-04-02T13:57:59.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.230+0000 7f5c10b44640 1 -- 172.21.15.52:0/2926019722 learned_addr learned my addr 172.21.15.52:0/2926019722 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:59.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.231+0000 7f5c11345640 1 --2- 172.21.15.52:0/2926019722 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c0c107ab0 0x7f5c0c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.231+0000 7f5c10b44640 1 -- 172.21.15.52:0/2926019722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c0c102460 msgr2=0x7f5c0c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:59.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.231+0000 7f5c10b44640 1 --2- 172.21.15.52:0/2926019722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c0c102460 0x7f5c0c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.231+0000 7f5c10b44640 1 -- 172.21.15.52:0/2926019722 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c0c107ab0 msgr2=0x7f5c0c19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:59.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.231+0000 7f5c10b44640 1 --2- 172.21.15.52:0/2926019722 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c0c107ab0 0x7f5c0c19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.231+0000 7f5c10b44640 1 -- 172.21.15.52:0/2926019722 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c0c19d330 con 0x7f5c0c101aa0 2024-04-02T13:57:59.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.231+0000 7f5c10b44640 1 --2- 172.21.15.52:0/2926019722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c0c101aa0 0x7f5c0c198390 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009230 tx=0x7f5c0002cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:59.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.231+0000 7f5c09ffb640 1 -- 172.21.15.52:0/2926019722 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c00041070 con 0x7f5c0c101aa0 2024-04-02T13:57:59.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.231+0000 7f5c09ffb640 1 -- 172.21.15.52:0/2926019722 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c00004450 con 0x7f5c0c101aa0 2024-04-02T13:57:59.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.231+0000 7f5c12da8640 1 -- 172.21.15.52:0/2926019722 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c0c19d5c0 con 0x7f5c0c101aa0 2024-04-02T13:57:59.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.231+0000 7f5c09ffb640 1 -- 172.21.15.52:0/2926019722 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c00034440 con 0x7f5c0c101aa0 2024-04-02T13:57:59.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.231+0000 7f5c12da8640 1 -- 172.21.15.52:0/2926019722 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c0c1a4eb0 con 0x7f5c0c101aa0 2024-04-02T13:57:59.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.234+0000 7f5c09ffb640 1 -- 172.21.15.52:0/2926019722 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5c00044070 con 0x7f5c0c101aa0 2024-04-02T13:57:59.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.234+0000 7f5c1598d740 1 Processor -- start 2024-04-02T13:57:59.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.234+0000 7f5c1598d740 1 -- start start 2024-04-02T13:57:59.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.234+0000 7f5c1598d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7d6ba2c0 0x55cd7d6ba6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.234+0000 7f5c1598d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6babe0 0x55cd7d6c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.234+0000 7f5c09ffb640 1 --2- 172.21.15.52:0/2926019722 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5bfc0a3da0 0x7f5bfc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.234+0000 7f5c1598d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7d6c37c0 0x55cd7d6c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.234+0000 7f5c1598d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cd7d61e770 con 0x55cd7d6babe0 2024-04-02T13:57:59.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.234+0000 7f5c1598d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cd7d5c6960 con 0x55cd7d6ba2c0 2024-04-02T13:57:59.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.234+0000 7f5c1598d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cd7d61a8c0 con 0x55cd7d6c37c0 2024-04-02T13:57:59.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.234+0000 7f5c10b44640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6babe0 0x55cd7d6c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.234+0000 7f5c0bfff640 1 --2- 172.21.15.52:0/2926019722 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5bfc0a3da0 0x7f5bfc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.234+0000 7f5c10b44640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6babe0 0x55cd7d6c3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44470/0 (socket says 172.21.15.52:44470) 2024-04-02T13:57:59.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.234+0000 7f5c10b44640 1 -- 172.21.15.52:0/2104572347 learned_addr learned my addr 172.21.15.52:0/2104572347 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:59.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.234+0000 7f5c09ffb640 1 -- 172.21.15.52:0/2926019722 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f5c000f2400 con 0x7f5c0c101aa0 2024-04-02T13:57:59.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.234+0000 7f5c11345640 1 --2- 172.21.15.52:0/2104572347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7d6ba2c0 0x55cd7d6ba6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.235+0000 7f5c11345640 1 --2- 172.21.15.52:0/2104572347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7d6c37c0 0x55cd7d6c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.235+0000 7f5c10b44640 1 -- 172.21.15.52:0/2104572347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7d6c37c0 msgr2=0x55cd7d6c5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:59.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.235+0000 7f5c10b44640 1 --2- 172.21.15.52:0/2104572347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7d6c37c0 0x55cd7d6c5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.235+0000 7f5c10b44640 1 -- 172.21.15.52:0/2104572347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7d6ba2c0 msgr2=0x55cd7d6ba6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:59.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.235+0000 7f5c10b44640 1 --2- 172.21.15.52:0/2104572347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7d6ba2c0 0x55cd7d6ba6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.235+0000 7f5c10b44640 1 -- 172.21.15.52:0/2104572347 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd7d3fa7e0 con 0x55cd7d6babe0 2024-04-02T13:57:59.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.235+0000 7f5c11345640 1 --2- 172.21.15.52:0/2104572347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7d6c37c0 0x55cd7d6c5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:59.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.235+0000 7f5c0bfff640 1 --2- 172.21.15.52:0/2926019722 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5bfc0a3da0 0x7f5bfc0a6260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c1071a0 tx=0x7f5bf8005c70 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:59.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.235+0000 7f5c10b44640 1 --2- 172.21.15.52:0/2104572347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6babe0 0x55cd7d6c3280 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f5c000be1a0 tx=0x7f5c0008daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba74608f7333465 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:59.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.235+0000 7f5bf6ffd640 1 -- 172.21.15.52:0/2104572347 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0004b040 con 0x55cd7d6babe0 2024-04-02T13:57:59.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.235+0000 7f5bf6ffd640 1 -- 172.21.15.52:0/2104572347 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c00004c80 con 0x55cd7d6babe0 2024-04-02T13:57:59.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.235+0000 7f5bf6ffd640 1 -- 172.21.15.52:0/2104572347 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c000be210 con 0x55cd7d6babe0 2024-04-02T13:57:59.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.236+0000 7f5c1598d740 1 -- 172.21.15.52:0/2104572347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6babe0 msgr2=0x55cd7d6c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:59.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.236+0000 7f5c1598d740 1 --2- 172.21.15.52:0/2104572347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6babe0 0x55cd7d6c3280 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f5c000be1a0 tx=0x7f5c0008daf0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.236+0000 7f5c1598d740 1 -- 172.21.15.52:0/2104572347 shutdown_connections 2024-04-02T13:57:59.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.236+0000 7f5c1598d740 1 --2- 172.21.15.52:0/2104572347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7d6c37c0 0x55cd7d6c5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.236+0000 7f5c1598d740 1 --2- 172.21.15.52:0/2104572347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6babe0 0x55cd7d6c3280 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.236+0000 7f5c1598d740 1 --2- 172.21.15.52:0/2104572347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7d6ba2c0 0x55cd7d6ba6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.236+0000 7f5c1598d740 1 -- 172.21.15.52:0/2104572347 >> 172.21.15.52:0/2104572347 conn(0x55cd7d6b49f0 msgr2=0x55cd7d6b4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:59.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.236+0000 7f5c1598d740 1 -- 172.21.15.52:0/2104572347 shutdown_connections 2024-04-02T13:57:59.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.236+0000 7f5c1598d740 1 -- 172.21.15.52:0/2104572347 wait complete. 2024-04-02T13:57:59.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.236+0000 7f5c1598d740 1 -- start start 2024-04-02T13:57:59.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.236+0000 7f5c1598d740 1 Processor -- start 2024-04-02T13:57:59.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.237+0000 7f5c1598d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6ba2c0 0x55cd7d6c07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.237+0000 7f5c1598d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7d6babe0 0x55cd7d6c0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.237+0000 7f5c1598d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7d6c37c0 0x55cd7d6c1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.237+0000 7f5c1598d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cd7d6c6370 con 0x55cd7d6ba2c0 2024-04-02T13:57:59.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.237+0000 7f5c1598d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cd7d5d9d20 con 0x55cd7d6c37c0 2024-04-02T13:57:59.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.237+0000 7f5c1598d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cd7d6c64f0 con 0x55cd7d6babe0 2024-04-02T13:57:59.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.237+0000 7f5c1598d740 10 client.0 _ll_register_callbacks cb 0x55cd7d5a6a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:59.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.237+0000 7f5c10b44640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6ba2c0 0x55cd7d6c07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.237+0000 7f5c10b44640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6ba2c0 0x55cd7d6c07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44484/0 (socket says 172.21.15.52:44484) 2024-04-02T13:57:59.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.237+0000 7f5c10b44640 1 -- 172.21.15.52:0/385904345 learned_addr learned my addr 172.21.15.52:0/385904345 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:59.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.237+0000 7f5c11345640 1 --2- 172.21.15.52:0/385904345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7d6c37c0 0x55cd7d6c1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.237+0000 7f5c0bfff640 1 --2- 172.21.15.52:0/385904345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7d6babe0 0x55cd7d6c0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.237+0000 7f5c10b44640 1 -- 172.21.15.52:0/385904345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7d6babe0 msgr2=0x55cd7d6c0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:59.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.237+0000 7f5c10b44640 1 --2- 172.21.15.52:0/385904345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7d6babe0 0x55cd7d6c0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.237+0000 7f5c10b44640 1 -- 172.21.15.52:0/385904345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7d6c37c0 msgr2=0x55cd7d6c1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:59.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.237+0000 7f5c10b44640 1 --2- 172.21.15.52:0/385904345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7d6c37c0 0x55cd7d6c1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.237+0000 7f5c10b44640 1 -- 172.21.15.52:0/385904345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd7d45af70 con 0x55cd7d6ba2c0 2024-04-02T13:57:59.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.237+0000 7f5c11345640 1 --2- 172.21.15.52:0/385904345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7d6c37c0 0x55cd7d6c1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:59.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.237+0000 7f5c10b44640 1 --2- 172.21.15.52:0/385904345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6ba2c0 0x55cd7d6c07c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f5c000bdf10 tx=0x7f5c00003990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d34709fe6cb1feb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:59.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.237+0000 7f5bebfff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:59.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.237+0000 7f5bebfff640 1 -- 172.21.15.52:0/385904345 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0004b050 con 0x55cd7d6ba2c0 2024-04-02T13:57:59.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.237+0000 7f5bebfff640 1 -- 172.21.15.52:0/385904345 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f5c000035f0 con 0x55cd7d6ba2c0 2024-04-02T13:57:59.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.237+0000 7f5c1598d740 1 -- 172.21.15.52:0/385904345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c00004c80 con 0x55cd7d6ba2c0 2024-04-02T13:57:59.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.238+0000 7f5bebfff640 1 -- 172.21.15.52:0/385904345 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5c000bdd50 con 0x55cd7d6ba2c0 2024-04-02T13:57:59.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.238+0000 7f5c1598d740 10 client.16968 fetch_fsmap learned FSMap version 30 2024-04-02T13:57:59.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.238+0000 7f5c1598d740 1 -- 172.21.15.52:0/385904345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cd7d2f2170 con 0x55cd7d6ba2c0 2024-04-02T13:57:59.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.238+0000 7f5bebfff640 1 -- 172.21.15.52:0/385904345 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c00004c80 con 0x55cd7d6ba2c0 2024-04-02T13:57:59.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.238+0000 7f5bebfff640 1 -- 172.21.15.52:0/385904345 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5c000c2160 con 0x55cd7d6ba2c0 2024-04-02T13:57:59.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.238+0000 7f5c1598d740 10 client.16968 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:57:59.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.238+0000 7f5c1598d740 10 client.16968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:57:59.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.238+0000 7f5c1598d740 20 client.16968 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:59.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.238+0000 7f5c1598d740 1 --2- 172.21.15.52:0/385904345 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cd7d6d3630 0x55cd7d6d5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.238+0000 7f5c1598d740 4 client.16968 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:57:59.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.238+0000 7f5c1598d740 1 -- 172.21.15.52:0/385904345 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd7d62bcb0 con 0x55cd7d6d3630 2024-04-02T13:57:59.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.238+0000 7f5c0bfff640 1 --2- 172.21.15.52:0/385904345 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cd7d6d3630 0x55cd7d6d5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.239+0000 7f5bebfff640 10 client.16968 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:59.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.239+0000 7f5c0bfff640 1 --2- 172.21.15.52:0/385904345 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cd7d6d3630 0x55cd7d6d5a20 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5106a9bdd4caa4b server_cookie=6334f2024526a895 in_seq=0 out_seq=0 2024-04-02T13:57:59.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.240+0000 7f5bebfff640 1 -- 172.21.15.52:0/385904345 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cd7d62bcb0 con 0x55cd7d6d3630 2024-04-02T13:57:59.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.240+0000 7f5bebfff640 10 client.16968 handle_command_reply: tid=0 2024-04-02T13:57:59.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.240+0000 7f5c1598d740 1 client.16968 shutdown 2024-04-02T13:57:59.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.240+0000 7f5c1598d740 1 -- 172.21.15.52:0/385904345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6ba2c0 msgr2=0x55cd7d6c07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:59.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.240+0000 7f5c1598d740 1 --2- 172.21.15.52:0/385904345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6ba2c0 0x55cd7d6c07c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f5c000bdf10 tx=0x7f5c00003990 comp rx=0 tx=0).stop 2024-04-02T13:57:59.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.240+0000 7f5c1598d740 1 -- 172.21.15.52:0/385904345 shutdown_connections 2024-04-02T13:57:59.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.240+0000 7f5c1598d740 1 --2- 172.21.15.52:0/385904345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7d6c37c0 0x55cd7d6c1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.240+0000 7f5c1598d740 1 --2- 172.21.15.52:0/385904345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7d6babe0 0x55cd7d6c0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.240+0000 7f5c1598d740 1 --2- 172.21.15.52:0/385904345 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cd7d6d3630 0x55cd7d6d5a20 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.240+0000 7f5c1598d740 1 --2- 172.21.15.52:0/385904345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6ba2c0 0x55cd7d6c07c0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.240+0000 7f5bebfff640 0 client.16968 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:59.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.240+0000 7f5c1598d740 1 -- 172.21.15.52:0/385904345 >> 172.21.15.52:0/385904345 conn(0x55cd7d6b49f0 msgr2=0x55cd7d6b6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:59.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.240+0000 7f5c1598d740 1 -- 172.21.15.52:0/385904345 shutdown_connections 2024-04-02T13:57:59.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.240+0000 7f5c1598d740 1 -- 172.21.15.52:0/385904345 wait complete. 2024-04-02T13:57:59.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.240+0000 7f5c1598d740 20 client.16968 trim_cache size 0 max 16384 2024-04-02T13:57:59.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.258+0000 7f5c1598d740 1 Processor -- start 2024-04-02T13:57:59.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.258+0000 7f5c1598d740 1 -- start start 2024-04-02T13:57:59.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.258+0000 7f5c1598d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7d6ba2c0 0x55cd7d6c2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.258+0000 7f5c1598d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6babe0 0x55cd7d6cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.258+0000 7f5c1598d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7d6c37c0 0x55cd7d6db2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.258+0000 7f5c1598d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c1aed90 con 0x55cd7d6babe0 2024-04-02T13:57:59.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.258+0000 7f5c1598d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c1b08d0 con 0x55cd7d6c37c0 2024-04-02T13:57:59.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.258+0000 7f5c1598d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c1af460 con 0x55cd7d6ba2c0 2024-04-02T13:57:59.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.258+0000 7f5c0bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6babe0 0x55cd7d6cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.258+0000 7f5c0bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6babe0 0x55cd7d6cf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44496/0 (socket says 172.21.15.52:44496) 2024-04-02T13:57:59.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.258+0000 7f5c0bfff640 1 -- 172.21.15.52:0/251940796 learned_addr learned my addr 172.21.15.52:0/251940796 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:59.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.258+0000 7f5c11345640 1 --2- 172.21.15.52:0/251940796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7d6c37c0 0x55cd7d6db2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.258+0000 7f5c10b44640 1 --2- 172.21.15.52:0/251940796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7d6ba2c0 0x55cd7d6c2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.258+0000 7f5c0bfff640 1 -- 172.21.15.52:0/251940796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7d6ba2c0 msgr2=0x55cd7d6c2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:59.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.258+0000 7f5c0bfff640 1 --2- 172.21.15.52:0/251940796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7d6ba2c0 0x55cd7d6c2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.258+0000 7f5c0bfff640 1 -- 172.21.15.52:0/251940796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7d6c37c0 msgr2=0x55cd7d6db2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:59.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.258+0000 7f5c0bfff640 1 --2- 172.21.15.52:0/251940796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7d6c37c0 0x55cd7d6db2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.258+0000 7f5c0bfff640 1 -- 172.21.15.52:0/251940796 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd7d6b4820 con 0x55cd7d6babe0 2024-04-02T13:57:59.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.258+0000 7f5c10b44640 1 --2- 172.21.15.52:0/251940796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7d6ba2c0 0x55cd7d6c2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:59.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.258+0000 7f5c11345640 1 --2- 172.21.15.52:0/251940796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7d6c37c0 0x55cd7d6db2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:59.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.259+0000 7f5c0bfff640 1 --2- 172.21.15.52:0/251940796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6babe0 0x55cd7d6cf1a0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8011040 tx=0x7f5bf800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05ffe98bb0af766 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:59.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.259+0000 7f5bf67fc640 1 -- 172.21.15.52:0/251940796 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf801dc70 con 0x55cd7d6babe0 2024-04-02T13:57:59.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.259+0000 7f5bf67fc640 1 -- 172.21.15.52:0/251940796 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bf8015040 con 0x55cd7d6babe0 2024-04-02T13:57:59.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.259+0000 7f5bf67fc640 1 -- 172.21.15.52:0/251940796 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8004db0 con 0x55cd7d6babe0 2024-04-02T13:57:59.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.259+0000 7f5c1598d740 1 -- 172.21.15.52:0/251940796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6babe0 msgr2=0x55cd7d6cf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:59.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.259+0000 7f5c1598d740 1 --2- 172.21.15.52:0/251940796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6babe0 0x55cd7d6cf1a0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8011040 tx=0x7f5bf800f000 comp rx=0 tx=0).stop 2024-04-02T13:57:59.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.259+0000 7f5c1598d740 1 -- 172.21.15.52:0/251940796 shutdown_connections 2024-04-02T13:57:59.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.259+0000 7f5c1598d740 1 --2- 172.21.15.52:0/251940796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7d6c37c0 0x55cd7d6db2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.259+0000 7f5c1598d740 1 --2- 172.21.15.52:0/251940796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6babe0 0x55cd7d6cf1a0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.259+0000 7f5c1598d740 1 --2- 172.21.15.52:0/251940796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7d6ba2c0 0x55cd7d6c2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.259+0000 7f5c1598d740 1 -- 172.21.15.52:0/251940796 >> 172.21.15.52:0/251940796 conn(0x55cd7d6b49f0 msgr2=0x55cd7d6b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:59.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.260+0000 7f5c1598d740 1 -- 172.21.15.52:0/251940796 shutdown_connections 2024-04-02T13:57:59.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.260+0000 7f5c1598d740 1 -- 172.21.15.52:0/251940796 wait complete. 2024-04-02T13:57:59.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.260+0000 7f5c1598d740 1 -- start start 2024-04-02T13:57:59.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.260+0000 7f5c1598d740 1 Processor -- start 2024-04-02T13:57:59.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.260+0000 7f5c1598d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7d6ba2c0 0x55cd7d6c7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.260+0000 7f5c1598d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7d6babe0 0x55cd7d6c7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.260+0000 7f5c1598d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6c37c0 0x55cd7d738b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.260+0000 7f5c1598d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c1b0200 con 0x55cd7d6c37c0 2024-04-02T13:57:59.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.260+0000 7f5c1598d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c1afb30 con 0x55cd7d6babe0 2024-04-02T13:57:59.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.260+0000 7f5c1598d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c1acd70 con 0x55cd7d6ba2c0 2024-04-02T13:57:59.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.260+0000 7f5c1598d740 10 client.0 _ll_register_callbacks cb 0x7f5c00005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:59.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.260+0000 7f5c11345640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6c37c0 0x55cd7d738b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.260+0000 7f5c11345640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6c37c0 0x55cd7d738b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44506/0 (socket says 172.21.15.52:44506) 2024-04-02T13:57:59.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.260+0000 7f5c11345640 1 -- 172.21.15.52:0/3296828152 learned_addr learned my addr 172.21.15.52:0/3296828152 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:59.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.261+0000 7f5c0bfff640 1 --2- 172.21.15.52:0/3296828152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7d6babe0 0x55cd7d6c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.261+0000 7f5c10b44640 1 --2- 172.21.15.52:0/3296828152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7d6ba2c0 0x55cd7d6c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.261+0000 7f5c11345640 1 -- 172.21.15.52:0/3296828152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7d6ba2c0 msgr2=0x55cd7d6c7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:59.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.261+0000 7f5c11345640 1 --2- 172.21.15.52:0/3296828152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7d6ba2c0 0x55cd7d6c7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.261+0000 7f5c11345640 1 -- 172.21.15.52:0/3296828152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7d6babe0 msgr2=0x55cd7d6c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:59.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.261+0000 7f5c11345640 1 --2- 172.21.15.52:0/3296828152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7d6babe0 0x55cd7d6c7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.261+0000 7f5c11345640 1 -- 172.21.15.52:0/3296828152 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd7d6c8250 con 0x55cd7d6c37c0 2024-04-02T13:57:59.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.261+0000 7f5c11345640 1 --2- 172.21.15.52:0/3296828152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6c37c0 0x55cd7d738b10 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f5c04011be0 tx=0x7f5c04009b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b87246b9cb4291c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:59.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.261+0000 7f5bebfff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:59.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.261+0000 7f5bebfff640 1 -- 172.21.15.52:0/3296828152 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c04004b20 con 0x55cd7d6c37c0 2024-04-02T13:57:59.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.261+0000 7f5bebfff640 1 -- 172.21.15.52:0/3296828152 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f5c04004cc0 con 0x55cd7d6c37c0 2024-04-02T13:57:59.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.261+0000 7f5c1598d740 1 -- 172.21.15.52:0/3296828152 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bf8015040 con 0x55cd7d6c37c0 2024-04-02T13:57:59.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.261+0000 7f5bebfff640 1 -- 172.21.15.52:0/3296828152 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5c04028050 con 0x55cd7d6c37c0 2024-04-02T13:57:59.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.261+0000 7f5c1598d740 10 client.16974 fetch_fsmap learned FSMap version 30 2024-04-02T13:57:59.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.261+0000 7f5c1598d740 1 -- 172.21.15.52:0/3296828152 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cd7d73a160 con 0x55cd7d6c37c0 2024-04-02T13:57:59.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.261+0000 7f5bebfff640 1 -- 172.21.15.52:0/3296828152 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bf8015040 con 0x55cd7d6c37c0 2024-04-02T13:57:59.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.262+0000 7f5bebfff640 1 -- 172.21.15.52:0/3296828152 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5c0400e3b0 con 0x55cd7d6c37c0 2024-04-02T13:57:59.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.262+0000 7f5c1598d740 10 client.16974 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:57:59.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.262+0000 7f5c1598d740 10 client.16974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:57:59.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.262+0000 7f5c1598d740 20 client.16974 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:59.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.262+0000 7f5c1598d740 1 --2- 172.21.15.52:0/3296828152 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cd7d73a8d0 0x55cd7d73ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.262+0000 7f5c1598d740 4 client.16974 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:57:59.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.262+0000 7f5c1598d740 1 -- 172.21.15.52:0/3296828152 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cd7d626000 con 0x55cd7d73a8d0 2024-04-02T13:57:59.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.262+0000 7f5c10b44640 1 --2- 172.21.15.52:0/3296828152 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cd7d73a8d0 0x55cd7d73ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.262+0000 7f5c10b44640 1 --2- 172.21.15.52:0/3296828152 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cd7d73a8d0 0x55cd7d73ccf0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d25ed6ef576a1802 server_cookie=26a9bcfc88c68172 in_seq=0 out_seq=0 2024-04-02T13:57:59.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.262+0000 7f5bebfff640 10 client.16974 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:59.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.264+0000 7f5bebfff640 1 -- 172.21.15.52:0/3296828152 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cd7d626000 con 0x55cd7d73a8d0 2024-04-02T13:57:59.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.264+0000 7f5bebfff640 10 client.16974 handle_command_reply: tid=0 2024-04-02T13:57:59.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.264+0000 7f5c1598d740 1 client.16974 shutdown 2024-04-02T13:57:59.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.264+0000 7f5c1598d740 1 -- 172.21.15.52:0/3296828152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6c37c0 msgr2=0x55cd7d738b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:59.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.264+0000 7f5c1598d740 1 --2- 172.21.15.52:0/3296828152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6c37c0 0x55cd7d738b10 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f5c04011be0 tx=0x7f5c04009b40 comp rx=0 tx=0).stop 2024-04-02T13:57:59.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.264+0000 7f5c1598d740 1 -- 172.21.15.52:0/3296828152 shutdown_connections 2024-04-02T13:57:59.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.264+0000 7f5c1598d740 1 --2- 172.21.15.52:0/3296828152 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cd7d73a8d0 0x55cd7d73ccf0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.264+0000 7f5c1598d740 1 --2- 172.21.15.52:0/3296828152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd7d6c37c0 0x55cd7d738b10 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.264+0000 7f5c1598d740 1 --2- 172.21.15.52:0/3296828152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd7d6babe0 0x55cd7d6c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.264+0000 7f5bebfff640 0 client.16974 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:59.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.264+0000 7f5c1598d740 1 --2- 172.21.15.52:0/3296828152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd7d6ba2c0 0x55cd7d6c7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.264+0000 7f5c1598d740 1 -- 172.21.15.52:0/3296828152 >> 172.21.15.52:0/3296828152 conn(0x55cd7d6b49f0 msgr2=0x55cd7d6b5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:59.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.264+0000 7f5c1598d740 1 -- 172.21.15.52:0/3296828152 shutdown_connections 2024-04-02T13:57:59.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.264+0000 7f5c1598d740 1 -- 172.21.15.52:0/3296828152 wait complete. 2024-04-02T13:57:59.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.264+0000 7f5c1598d740 20 client.16974 trim_cache size 0 max 16384 2024-04-02T13:57:59.277 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:57:59.277 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-04-02T13:57:59.277 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "3e36fec1-a689-47e7-89e6-f3e086025120", 2024-04-02T13:57:59.277 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-04-02T13:57:59.278 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:57:59.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.265+0000 7f5c12da8640 1 -- 172.21.15.52:0/2926019722 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5bfc0a3da0 msgr2=0x7f5bfc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:59.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.265+0000 7f5c12da8640 1 --2- 172.21.15.52:0/2926019722 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5bfc0a3da0 0x7f5bfc0a6260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c1071a0 tx=0x7f5bf8005c70 comp rx=0 tx=0).stop 2024-04-02T13:57:59.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.265+0000 7f5c12da8640 1 -- 172.21.15.52:0/2926019722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c0c101aa0 msgr2=0x7f5c0c198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:59.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.265+0000 7f5c12da8640 1 --2- 172.21.15.52:0/2926019722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c0c101aa0 0x7f5c0c198390 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009230 tx=0x7f5c0002cba0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.265+0000 7f5c12da8640 1 -- 172.21.15.52:0/2926019722 shutdown_connections 2024-04-02T13:57:59.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.265+0000 7f5c12da8640 1 --2- 172.21.15.52:0/2926019722 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5bfc0a3da0 0x7f5bfc0a6260 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.265+0000 7f5c12da8640 1 --2- 172.21.15.52:0/2926019722 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c0c107ab0 0x7f5c0c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.265+0000 7f5c12da8640 1 --2- 172.21.15.52:0/2926019722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c0c102460 0x7f5c0c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.265+0000 7f5c12da8640 1 --2- 172.21.15.52:0/2926019722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c0c101aa0 0x7f5c0c198390 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.265+0000 7f5c12da8640 1 -- 172.21.15.52:0/2926019722 >> 172.21.15.52:0/2926019722 conn(0x7f5c0c0fd530 msgr2=0x7f5c0c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:59.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.265+0000 7f5c12da8640 1 -- 172.21.15.52:0/2926019722 shutdown_connections 2024-04-02T13:57:59.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.265+0000 7f5c12da8640 1 -- 172.21.15.52:0/2926019722 wait complete. 2024-04-02T13:57:59.281 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:57:59.281 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:57:59.307 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:57:59 smithi052 ceph-mon[32104]: pgmap v137: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:57:59.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.386+0000 7f9cedb8f640 1 -- 172.21.15.52:0/1077542982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ce8103f10 msgr2=0x7f9ce81042f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:59.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.386+0000 7f9cedb8f640 1 --2- 172.21.15.52:0/1077542982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ce8103f10 0x7f9ce81042f0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8009900 tx=0x7f9cd802bc30 comp rx=0 tx=0).stop 2024-04-02T13:57:59.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.386+0000 7f9cedb8f640 1 -- 172.21.15.52:0/1077542982 shutdown_connections 2024-04-02T13:57:59.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.386+0000 7f9cedb8f640 1 --2- 172.21.15.52:0/1077542982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ce8108fe0 0x7f9ce8110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.386+0000 7f9cedb8f640 1 --2- 172.21.15.52:0/1077542982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ce81048c0 0x7f9ce8108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.386+0000 7f9cedb8f640 1 --2- 172.21.15.52:0/1077542982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ce8103f10 0x7f9ce81042f0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.386+0000 7f9cedb8f640 1 -- 172.21.15.52:0/1077542982 >> 172.21.15.52:0/1077542982 conn(0x7f9ce80ff770 msgr2=0x7f9ce8101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:59.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.386+0000 7f9cedb8f640 1 -- 172.21.15.52:0/1077542982 shutdown_connections 2024-04-02T13:57:59.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.386+0000 7f9cedb8f640 1 -- 172.21.15.52:0/1077542982 wait complete. 2024-04-02T13:57:59.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.386+0000 7f9cedb8f640 1 Processor -- start 2024-04-02T13:57:59.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.386+0000 7f9cedb8f640 1 -- start start 2024-04-02T13:57:59.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.387+0000 7f9cedb8f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ce8103f10 0x7f9ce819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.387+0000 7f9cedb8f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ce8108fe0 0x7f9ce819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.387+0000 7f9cedb8f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ce819eef0 0x7f9ce819f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.387+0000 7f9cedb8f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce8114090 con 0x7f9ce8108fe0 2024-04-02T13:57:59.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.387+0000 7f9cedb8f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce8113f10 con 0x7f9ce819eef0 2024-04-02T13:57:59.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.387+0000 7f9cedb8f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce8114210 con 0x7f9ce8103f10 2024-04-02T13:57:59.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.387+0000 7f9ce6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ce8108fe0 0x7f9ce819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.387+0000 7f9ce6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ce8108fe0 0x7f9ce819ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44520/0 (socket says 172.21.15.52:44520) 2024-04-02T13:57:59.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.387+0000 7f9ce6ffd640 1 -- 172.21.15.52:0/3300797618 learned_addr learned my addr 172.21.15.52:0/3300797618 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:59.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.387+0000 7f9ce7fff640 1 --2- 172.21.15.52:0/3300797618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ce819eef0 0x7f9ce819f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.387+0000 7f9ce77fe640 1 --2- 172.21.15.52:0/3300797618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ce8103f10 0x7f9ce819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.387+0000 7f9ce6ffd640 1 -- 172.21.15.52:0/3300797618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ce8103f10 msgr2=0x7f9ce819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:59.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.387+0000 7f9ce6ffd640 1 --2- 172.21.15.52:0/3300797618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ce8103f10 0x7f9ce819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.387+0000 7f9ce6ffd640 1 -- 172.21.15.52:0/3300797618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ce819eef0 msgr2=0x7f9ce819f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:59.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.387+0000 7f9ce6ffd640 1 --2- 172.21.15.52:0/3300797618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ce819eef0 0x7f9ce819f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.387+0000 7f9ce6ffd640 1 -- 172.21.15.52:0/3300797618 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce819f9e0 con 0x7f9ce8108fe0 2024-04-02T13:57:59.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.387+0000 7f9ce6ffd640 1 --2- 172.21.15.52:0/3300797618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ce8108fe0 0x7f9ce819ab70 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00e9f0 tx=0x7f9cdc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:59.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.387+0000 7f9ce4ff9640 1 -- 172.21.15.52:0/3300797618 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc00ccc0 con 0x7f9ce8108fe0 2024-04-02T13:57:59.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.387+0000 7f9ce4ff9640 1 -- 172.21.15.52:0/3300797618 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cdc005500 con 0x7f9ce8108fe0 2024-04-02T13:57:59.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.388+0000 7f9cedb8f640 1 -- 172.21.15.52:0/3300797618 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce819fcd0 con 0x7f9ce8108fe0 2024-04-02T13:57:59.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.388+0000 7f9ce4ff9640 1 -- 172.21.15.52:0/3300797618 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc010690 con 0x7f9ce8108fe0 2024-04-02T13:57:59.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.388+0000 7f9cedb8f640 1 -- 172.21.15.52:0/3300797618 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce81a7470 con 0x7f9ce8108fe0 2024-04-02T13:57:59.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.390+0000 7f9ce4ff9640 1 -- 172.21.15.52:0/3300797618 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9cdc0046c0 con 0x7f9ce8108fe0 2024-04-02T13:57:59.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.390+0000 7f9cf077a740 1 Processor -- start 2024-04-02T13:57:59.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.390+0000 7f9cf077a740 1 -- start start 2024-04-02T13:57:59.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.390+0000 7f9cf077a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061248d0 0x55ce06124cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.390+0000 7f9cf077a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ce061251f0 0x55ce0612d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.390+0000 7f9cf077a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0612ddd0 0x55ce061301c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.390+0000 7f9cf077a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ce06088830 con 0x55ce061248d0 2024-04-02T13:57:59.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.390+0000 7f9cf077a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ce06030820 con 0x55ce061251f0 2024-04-02T13:57:59.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.390+0000 7f9cf077a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ce060847e0 con 0x55ce0612ddd0 2024-04-02T13:57:59.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.391+0000 7f9ce7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061248d0 0x55ce06124cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.391+0000 7f9ce4ff9640 1 --2- 172.21.15.52:0/3300797618 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9cd00a3d20 0x7f9cd00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.391+0000 7f9ce7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061248d0 0x55ce06124cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44530/0 (socket says 172.21.15.52:44530) 2024-04-02T13:57:59.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.391+0000 7f9ce7fff640 1 -- 172.21.15.52:0/3643179540 learned_addr learned my addr 172.21.15.52:0/3643179540 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:59.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.391+0000 7f9ce77fe640 1 --2- 172.21.15.52:0/3643179540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ce061251f0 0x55ce0612d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.391+0000 7f9ce6ffd640 1 --2- 172.21.15.52:0/3643179540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0612ddd0 0x55ce061301c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.391+0000 7f9ce4ff9640 1 -- 172.21.15.52:0/3300797618 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f9cdc0d0ef0 con 0x7f9ce8108fe0 2024-04-02T13:57:59.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.391+0000 7f9ce77fe640 1 -- 172.21.15.52:0/3643179540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0612ddd0 msgr2=0x55ce061301c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:59.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.391+0000 7f9ce77fe640 1 --2- 172.21.15.52:0/3643179540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0612ddd0 0x55ce061301c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.391+0000 7f9ce77fe640 1 -- 172.21.15.52:0/3643179540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061248d0 msgr2=0x55ce06124cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:59.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.391+0000 7f9ce77fe640 1 --2- 172.21.15.52:0/3643179540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061248d0 0x55ce06124cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.391+0000 7f9ce77fe640 1 -- 172.21.15.52:0/3643179540 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce05e667d0 con 0x55ce061251f0 2024-04-02T13:57:59.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.391+0000 7f9ce7fff640 1 --2- 172.21.15.52:0/3643179540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061248d0 0x55ce06124cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:57:59.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.391+0000 7f9ce7fff640 1 --2- 172.21.15.52:0/3300797618 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9cd00a3d20 0x7f9cd00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.391+0000 7f9ce77fe640 1 --2- 172.21.15.52:0/3643179540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ce061251f0 0x55ce0612d890 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8000c00 tx=0x7f9cd8005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=671949d7be1a50ca server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:59.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.392+0000 7f9ce7fff640 1 --2- 172.21.15.52:0/3300797618 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9cd00a3d20 0x7f9cd00a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9ce00049d0 tx=0x7f9ce0015040 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:59.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.392+0000 7f9ccdffb640 1 -- 172.21.15.52:0/3643179540 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8042070 con 0x55ce061251f0 2024-04-02T13:57:59.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.392+0000 7f9ccdffb640 1 -- 172.21.15.52:0/3643179540 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cd802ca80 con 0x55ce061251f0 2024-04-02T13:57:59.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.392+0000 7f9ccdffb640 1 -- 172.21.15.52:0/3643179540 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8004f00 con 0x55ce061251f0 2024-04-02T13:57:59.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.392+0000 7f9cf077a740 1 -- 172.21.15.52:0/3643179540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ce061251f0 msgr2=0x55ce0612d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:59.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.392+0000 7f9cf077a740 1 --2- 172.21.15.52:0/3643179540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ce061251f0 0x55ce0612d890 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8000c00 tx=0x7f9cd8005be0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.392+0000 7f9cf077a740 1 -- 172.21.15.52:0/3643179540 shutdown_connections 2024-04-02T13:57:59.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.392+0000 7f9cf077a740 1 --2- 172.21.15.52:0/3643179540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce0612ddd0 0x55ce061301c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.392+0000 7f9cf077a740 1 --2- 172.21.15.52:0/3643179540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ce061251f0 0x55ce0612d890 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.392+0000 7f9cf077a740 1 --2- 172.21.15.52:0/3643179540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061248d0 0x55ce06124cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.392+0000 7f9cf077a740 1 -- 172.21.15.52:0/3643179540 >> 172.21.15.52:0/3643179540 conn(0x55ce0611f000 msgr2=0x55ce0611f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:59.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.392+0000 7f9cf077a740 1 -- 172.21.15.52:0/3643179540 shutdown_connections 2024-04-02T13:57:59.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.392+0000 7f9cf077a740 1 -- 172.21.15.52:0/3643179540 wait complete. 2024-04-02T13:57:59.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.393+0000 7f9cf077a740 1 -- start start 2024-04-02T13:57:59.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.393+0000 7f9cf077a740 1 Processor -- start 2024-04-02T13:57:59.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.393+0000 7f9cf077a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061248d0 0x55ce0612d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.393+0000 7f9cf077a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce061251f0 0x55ce0612aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.393+0000 7f9cf077a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ce0612ddd0 0x55ce0612afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.393+0000 7f9cf077a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ce06130b60 con 0x55ce061248d0 2024-04-02T13:57:59.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.393+0000 7f9cf077a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ce06043d30 con 0x55ce0612ddd0 2024-04-02T13:57:59.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.393+0000 7f9cf077a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ce06130ce0 con 0x55ce061251f0 2024-04-02T13:57:59.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.393+0000 7f9cf077a740 10 client.0 _ll_register_callbacks cb 0x55ce06043720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:59.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.393+0000 7f9ce77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061248d0 0x55ce0612d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.393+0000 7f9ce77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061248d0 0x55ce0612d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44542/0 (socket says 172.21.15.52:44542) 2024-04-02T13:57:59.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.393+0000 7f9ce77fe640 1 -- 172.21.15.52:0/3489322273 learned_addr learned my addr 172.21.15.52:0/3489322273 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:59.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.393+0000 7f9ce7fff640 1 --2- 172.21.15.52:0/3489322273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ce0612ddd0 0x55ce0612afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.393+0000 7f9ce6ffd640 1 --2- 172.21.15.52:0/3489322273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce061251f0 0x55ce0612aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.394+0000 7f9ce7fff640 1 -- 172.21.15.52:0/3489322273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce061251f0 msgr2=0x55ce0612aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.394+0000 7f9ce7fff640 1 --2- 172.21.15.52:0/3489322273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce061251f0 0x55ce0612aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.394+0000 7f9ce7fff640 1 -- 172.21.15.52:0/3489322273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061248d0 msgr2=0x55ce0612d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.394+0000 7f9ce7fff640 1 --2- 172.21.15.52:0/3489322273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061248d0 0x55ce0612d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.394+0000 7f9ce7fff640 1 -- 172.21.15.52:0/3489322273 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce05d9c3f0 con 0x55ce0612ddd0 2024-04-02T13:57:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.394+0000 7f9ce7fff640 1 --2- 172.21.15.52:0/3489322273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ce0612ddd0 0x55ce0612afd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0005c40 tx=0x7f9ce0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da4804d1f34cce7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.394+0000 7f9cc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-02T13:57:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.394+0000 7f9cc6ffd640 1 -- 172.21.15.52:0/3489322273 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0010bb0 con 0x55ce0612ddd0 2024-04-02T13:57:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.394+0000 7f9cc6ffd640 1 -- 172.21.15.52:0/3489322273 <== mon.2 v2:172.21.15.88:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f9ce0010d50 con 0x55ce0612ddd0 2024-04-02T13:57:59.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.394+0000 7f9cf077a740 1 -- 172.21.15.52:0/3489322273 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cd802ca80 con 0x55ce0612ddd0 2024-04-02T13:57:59.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.394+0000 7f9cc6ffd640 1 -- 172.21.15.52:0/3489322273 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9ce001d480 con 0x55ce0612ddd0 2024-04-02T13:57:59.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.394+0000 7f9cf077a740 10 client.26327 fetch_fsmap learned FSMap version 30 2024-04-02T13:57:59.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.394+0000 7f9cf077a740 1 -- 172.21.15.52:0/3489322273 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ce05d5c0f0 con 0x55ce0612ddd0 2024-04-02T13:57:59.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.395+0000 7f9cc6ffd640 1 -- 172.21.15.52:0/3489322273 <== mon.2 v2:172.21.15.88:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cd802ca80 con 0x55ce0612ddd0 2024-04-02T13:57:59.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.395+0000 7f9cc6ffd640 1 -- 172.21.15.52:0/3489322273 <== mon.2 v2:172.21.15.88:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9ce0026020 con 0x55ce0612ddd0 2024-04-02T13:57:59.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.395+0000 7f9cf077a740 10 client.26327 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:57:59.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.395+0000 7f9cf077a740 10 client.26327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:57:59.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.395+0000 7f9cf077a740 20 client.26327 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:59.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.395+0000 7f9cf077a740 1 --2- 172.21.15.52:0/3489322273 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55ce0613de50 0x55ce06140280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.395+0000 7f9cf077a740 4 client.26327 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:57:59.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.395+0000 7f9cf077a740 1 -- 172.21.15.52:0/3489322273 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce06095d70 con 0x55ce0613de50 2024-04-02T13:57:59.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.395+0000 7f9ce77fe640 1 --2- 172.21.15.52:0/3489322273 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55ce0613de50 0x55ce06140280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.396+0000 7f9ce77fe640 1 --2- 172.21.15.52:0/3489322273 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55ce0613de50 0x55ce06140280 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40af6933758acab0 server_cookie=cf69e003a23ff0c1 in_seq=0 out_seq=0 2024-04-02T13:57:59.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.396+0000 7f9cc6ffd640 10 client.26327 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:59.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.396+0000 7f9cc6ffd640 1 -- 172.21.15.52:0/3489322273 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ce06095d70 con 0x55ce0613de50 2024-04-02T13:57:59.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.396+0000 7f9cc6ffd640 10 client.26327 handle_command_reply: tid=0 2024-04-02T13:57:59.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.396+0000 7f9cf077a740 1 client.26327 shutdown 2024-04-02T13:57:59.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.397+0000 7f9cf077a740 1 -- 172.21.15.52:0/3489322273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ce0612ddd0 msgr2=0x55ce0612afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:59.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.397+0000 7f9cf077a740 1 --2- 172.21.15.52:0/3489322273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ce0612ddd0 0x55ce0612afd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0005c40 tx=0x7f9ce0004a40 comp rx=0 tx=0).stop 2024-04-02T13:57:59.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.397+0000 7f9cf077a740 1 -- 172.21.15.52:0/3489322273 shutdown_connections 2024-04-02T13:57:59.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.397+0000 7f9cf077a740 1 --2- 172.21.15.52:0/3489322273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ce0612ddd0 0x55ce0612afd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.397+0000 7f9cf077a740 1 --2- 172.21.15.52:0/3489322273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce061251f0 0x55ce0612aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.397+0000 7f9cf077a740 1 --2- 172.21.15.52:0/3489322273 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55ce0613de50 0x55ce06140280 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.397+0000 7f9cf077a740 1 --2- 172.21.15.52:0/3489322273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061248d0 0x55ce0612d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.397+0000 7f9cc6ffd640 0 client.26327 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:59.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.397+0000 7f9cf077a740 1 -- 172.21.15.52:0/3489322273 >> 172.21.15.52:0/3489322273 conn(0x55ce0611f000 msgr2=0x55ce06120d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:59.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.397+0000 7f9cf077a740 1 -- 172.21.15.52:0/3489322273 shutdown_connections 2024-04-02T13:57:59.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.397+0000 7f9cf077a740 1 -- 172.21.15.52:0/3489322273 wait complete. 2024-04-02T13:57:59.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.397+0000 7f9cf077a740 20 client.26327 trim_cache size 0 max 16384 2024-04-02T13:57:59.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:57:59 smithi088 ceph-mon[36528]: pgmap v137: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:57:59.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.416+0000 7f9cf077a740 1 Processor -- start 2024-04-02T13:57:59.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.416+0000 7f9cf077a740 1 -- start start 2024-04-02T13:57:59.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.417+0000 7f9cf077a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce061248d0 0x55ce0611e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.417+0000 7f9cf077a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061251f0 0x55ce06120b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.417+0000 7f9cf077a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ce0612ddd0 0x55ce0612d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.417+0000 7f9cf077a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81acc00 con 0x55ce061251f0 2024-04-02T13:57:59.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.417+0000 7f9cf077a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81ae740 con 0x55ce0612ddd0 2024-04-02T13:57:59.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.417+0000 7f9cf077a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81ad2d0 con 0x55ce061248d0 2024-04-02T13:57:59.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.417+0000 7f9ce6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061251f0 0x55ce06120b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.417+0000 7f9ce77fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce061248d0 0x55ce0611e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.417+0000 7f9ce7fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ce0612ddd0 0x55ce0612d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.417+0000 7f9ce6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061251f0 0x55ce06120b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44554/0 (socket says 172.21.15.52:44554) 2024-04-02T13:57:59.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.417+0000 7f9ce77fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce061248d0 0x55ce0611e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.52:53136/0 (socket says 172.21.15.52:53136) 2024-04-02T13:57:59.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.417+0000 7f9ce6ffd640 1 -- 172.21.15.52:0/617676680 learned_addr learned my addr 172.21.15.52:0/617676680 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:59.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.417+0000 7f9ce6ffd640 1 -- 172.21.15.52:0/617676680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce061248d0 msgr2=0x55ce0611e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:59.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.417+0000 7f9ce6ffd640 1 --2- 172.21.15.52:0/617676680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce061248d0 0x55ce0611e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.417+0000 7f9ce6ffd640 1 -- 172.21.15.52:0/617676680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ce0612ddd0 msgr2=0x55ce0612d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:59.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.417+0000 7f9ce6ffd640 1 --2- 172.21.15.52:0/617676680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ce0612ddd0 0x55ce0612d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.417+0000 7f9ce6ffd640 1 -- 172.21.15.52:0/617676680 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce0611ac50 con 0x55ce061251f0 2024-04-02T13:57:59.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.417+0000 7f9ce7fff640 1 --2- 172.21.15.52:0/617676680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ce0612ddd0 0x55ce0612d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:57:59.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.417+0000 7f9ce6ffd640 1 --2- 172.21.15.52:0/617676680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061251f0 0x55ce06120b40 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc098f50 tx=0x7f9cdc0671a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda5a9ce97cbd927 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:59.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.418+0000 7f9ccd7fa640 1 -- 172.21.15.52:0/617676680 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc09c450 con 0x55ce061251f0 2024-04-02T13:57:59.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.418+0000 7f9ccd7fa640 1 -- 172.21.15.52:0/617676680 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cdc09d1f0 con 0x55ce061251f0 2024-04-02T13:57:59.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.418+0000 7f9ccd7fa640 1 -- 172.21.15.52:0/617676680 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc0ce520 con 0x55ce061251f0 2024-04-02T13:57:59.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.418+0000 7f9cf077a740 1 -- 172.21.15.52:0/617676680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061251f0 msgr2=0x55ce06120b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:59.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.418+0000 7f9cf077a740 1 --2- 172.21.15.52:0/617676680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061251f0 0x55ce06120b40 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc098f50 tx=0x7f9cdc0671a0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.418+0000 7f9cf077a740 1 -- 172.21.15.52:0/617676680 shutdown_connections 2024-04-02T13:57:59.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.418+0000 7f9cf077a740 1 --2- 172.21.15.52:0/617676680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ce0612ddd0 0x55ce0612d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.418+0000 7f9cf077a740 1 --2- 172.21.15.52:0/617676680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061251f0 0x55ce06120b40 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.418+0000 7f9cf077a740 1 --2- 172.21.15.52:0/617676680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce061248d0 0x55ce0611e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.418+0000 7f9cf077a740 1 -- 172.21.15.52:0/617676680 >> 172.21.15.52:0/617676680 conn(0x55ce0611f000 msgr2=0x55ce06120430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:59.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.419+0000 7f9cf077a740 1 -- 172.21.15.52:0/617676680 shutdown_connections 2024-04-02T13:57:59.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.419+0000 7f9cf077a740 1 -- 172.21.15.52:0/617676680 wait complete. 2024-04-02T13:57:59.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.419+0000 7f9cf077a740 1 -- start start 2024-04-02T13:57:59.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.419+0000 7f9cf077a740 1 Processor -- start 2024-04-02T13:57:59.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.419+0000 7f9cf077a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061248d0 0x55ce06131d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.419+0000 7f9cf077a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ce0612ddd0 0x55ce061322b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.419+0000 7f9cf077a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce06132950 0x55ce061a33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.419+0000 7f9cf077a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81ae070 con 0x55ce061248d0 2024-04-02T13:57:59.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.419+0000 7f9cf077a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81ad9a0 con 0x55ce0612ddd0 2024-04-02T13:57:59.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.419+0000 7f9cf077a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81aabe0 con 0x55ce06132950 2024-04-02T13:57:59.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.419+0000 7f9cf077a740 10 client.0 _ll_register_callbacks cb 0x7f9ce0009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:59.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.419+0000 7f9ce77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061248d0 0x55ce06131d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.419+0000 7f9ce77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061248d0 0x55ce06131d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44558/0 (socket says 172.21.15.52:44558) 2024-04-02T13:57:59.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.420+0000 7f9ce77fe640 1 -- 172.21.15.52:0/2106443622 learned_addr learned my addr 172.21.15.52:0/2106443622 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:57:59.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.420+0000 7f9ce6ffd640 1 --2- 172.21.15.52:0/2106443622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ce0612ddd0 0x55ce061322b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.420+0000 7f9ce7fff640 1 --2- 172.21.15.52:0/2106443622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce06132950 0x55ce061a33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.420+0000 7f9ce77fe640 1 -- 172.21.15.52:0/2106443622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce06132950 msgr2=0x55ce061a33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:59.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.420+0000 7f9ce77fe640 1 --2- 172.21.15.52:0/2106443622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce06132950 0x55ce061a33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.420+0000 7f9ce77fe640 1 -- 172.21.15.52:0/2106443622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ce0612ddd0 msgr2=0x55ce061322b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:59.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.420+0000 7f9ce77fe640 1 --2- 172.21.15.52:0/2106443622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ce0612ddd0 0x55ce061322b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.420+0000 7f9ce77fe640 1 -- 172.21.15.52:0/2106443622 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce061a3bd0 con 0x55ce061248d0 2024-04-02T13:57:59.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.420+0000 7f9ce6ffd640 1 --2- 172.21.15.52:0/2106443622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ce0612ddd0 0x55ce061322b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:57:59.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.420+0000 7f9ce77fe640 1 --2- 172.21.15.52:0/2106443622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061248d0 0x55ce06131d70 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8009850 tx=0x7f9cd80095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f979036b89bed19b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:59.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.420+0000 7f9cc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:57:59.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.420+0000 7f9cc6ffd640 1 -- 172.21.15.52:0/2106443622 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8055070 con 0x55ce061248d0 2024-04-02T13:57:59.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.420+0000 7f9cf077a740 1 -- 172.21.15.52:0/2106443622 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cdc09d1f0 con 0x55ce061248d0 2024-04-02T13:57:59.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.420+0000 7f9cc6ffd640 1 -- 172.21.15.52:0/2106443622 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f9cd8004040 con 0x55ce061248d0 2024-04-02T13:57:59.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.420+0000 7f9cc6ffd640 1 -- 172.21.15.52:0/2106443622 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9cd804b040 con 0x55ce061248d0 2024-04-02T13:57:59.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.420+0000 7f9cf077a740 10 client.16992 fetch_fsmap learned FSMap version 30 2024-04-02T13:57:59.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.420+0000 7f9cf077a740 1 -- 172.21.15.52:0/2106443622 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ce061a4f40 con 0x55ce061248d0 2024-04-02T13:57:59.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.421+0000 7f9cc6ffd640 1 -- 172.21.15.52:0/2106443622 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cdc09d1f0 con 0x55ce061248d0 2024-04-02T13:57:59.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.421+0000 7f9cc6ffd640 1 -- 172.21.15.52:0/2106443622 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9cd80035f0 con 0x55ce061248d0 2024-04-02T13:57:59.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.421+0000 7f9cf077a740 10 client.16992 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:57:59.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.421+0000 7f9cf077a740 10 client.16992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:57:59.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.421+0000 7f9cf077a740 20 client.16992 populate_metadata read hostname 'smithi052' 2024-04-02T13:57:59.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.421+0000 7f9cf077a740 1 --2- 172.21.15.52:0/2106443622 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55ce061a5230 0x55ce061a7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:59.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.421+0000 7f9cf077a740 4 client.16992 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:57:59.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.421+0000 7f9cf077a740 1 -- 172.21.15.52:0/2106443622 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ce060900c0 con 0x55ce061a5230 2024-04-02T13:57:59.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.421+0000 7f9ce6ffd640 1 --2- 172.21.15.52:0/2106443622 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55ce061a5230 0x55ce061a7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:59.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.421+0000 7f9ce6ffd640 1 --2- 172.21.15.52:0/2106443622 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55ce061a5230 0x55ce061a7620 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=379ea5a0429dbe3f server_cookie=e61126a947c165fa in_seq=0 out_seq=0 2024-04-02T13:57:59.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.421+0000 7f9cc6ffd640 10 client.16992 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:59.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.422+0000 7f9cc6ffd640 1 -- 172.21.15.52:0/2106443622 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55ce060900c0 con 0x55ce061a5230 2024-04-02T13:57:59.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.422+0000 7f9cc6ffd640 10 client.16992 handle_command_reply: tid=0 2024-04-02T13:57:59.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.422+0000 7f9cf077a740 1 client.16992 shutdown 2024-04-02T13:57:59.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.422+0000 7f9cf077a740 1 -- 172.21.15.52:0/2106443622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061248d0 msgr2=0x55ce06131d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:59.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.422+0000 7f9cf077a740 1 --2- 172.21.15.52:0/2106443622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061248d0 0x55ce06131d70 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8009850 tx=0x7f9cd80095e0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.422+0000 7f9cf077a740 1 -- 172.21.15.52:0/2106443622 shutdown_connections 2024-04-02T13:57:59.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.422+0000 7f9cf077a740 1 --2- 172.21.15.52:0/2106443622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ce06132950 0x55ce061a33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.422+0000 7f9cf077a740 1 --2- 172.21.15.52:0/2106443622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ce0612ddd0 0x55ce061322b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.422+0000 7f9cf077a740 1 --2- 172.21.15.52:0/2106443622 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55ce061a5230 0x55ce061a7620 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.422+0000 7f9cf077a740 1 --2- 172.21.15.52:0/2106443622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce061248d0 0x55ce06131d70 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.422+0000 7f9cc6ffd640 0 client.16992 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:57:59.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.422+0000 7f9cf077a740 1 -- 172.21.15.52:0/2106443622 >> 172.21.15.52:0/2106443622 conn(0x55ce0611f000 msgr2=0x55ce06120cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:59.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.422+0000 7f9cf077a740 1 -- 172.21.15.52:0/2106443622 shutdown_connections 2024-04-02T13:57:59.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.422+0000 7f9cf077a740 1 -- 172.21.15.52:0/2106443622 wait complete. 2024-04-02T13:57:59.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.422+0000 7f9cf077a740 20 client.16992 trim_cache size 0 max 16384 2024-04-02T13:57:59.440 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:57:59.440 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "scrub active (2735 inodes in the stack)", 2024-04-02T13:57:59.440 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-04-02T13:57:59.440 INFO:teuthology.orchestra.run.smithi052.stdout: "3e36fec1-a689-47e7-89e6-f3e086025120": { 2024-04-02T13:57:59.440 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-04-02T13:57:59.440 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "3e36fec1-a689-47e7-89e6-f3e086025120", 2024-04-02T13:57:59.441 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-04-02T13:57:59.441 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:57:59.441 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:57:59.441 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:57:59.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.423+0000 7f9cedb8f640 1 -- 172.21.15.52:0/3300797618 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9cd00a3d20 msgr2=0x7f9cd00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:59.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.423+0000 7f9cedb8f640 1 --2- 172.21.15.52:0/3300797618 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9cd00a3d20 0x7f9cd00a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9ce00049d0 tx=0x7f9ce0015040 comp rx=0 tx=0).stop 2024-04-02T13:57:59.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.423+0000 7f9cedb8f640 1 -- 172.21.15.52:0/3300797618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ce8108fe0 msgr2=0x7f9ce819ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:59.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.423+0000 7f9cedb8f640 1 --2- 172.21.15.52:0/3300797618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ce8108fe0 0x7f9ce819ab70 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00e9f0 tx=0x7f9cdc00eeb0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.423+0000 7f9cedb8f640 1 -- 172.21.15.52:0/3300797618 shutdown_connections 2024-04-02T13:57:59.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.423+0000 7f9cedb8f640 1 --2- 172.21.15.52:0/3300797618 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9cd00a3d20 0x7f9cd00a61e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.423+0000 7f9cedb8f640 1 --2- 172.21.15.52:0/3300797618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ce819eef0 0x7f9ce819f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.423+0000 7f9cedb8f640 1 --2- 172.21.15.52:0/3300797618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ce8108fe0 0x7f9ce819ab70 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.423+0000 7f9cedb8f640 1 --2- 172.21.15.52:0/3300797618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ce8103f10 0x7f9ce819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:59.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.423+0000 7f9cedb8f640 1 -- 172.21.15.52:0/3300797618 >> 172.21.15.52:0/3300797618 conn(0x7f9ce80ff770 msgr2=0x7f9ce8100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:59.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.423+0000 7f9cedb8f640 1 -- 172.21.15.52:0/3300797618 shutdown_connections 2024-04-02T13:57:59.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:57:59.423+0000 7f9cedb8f640 1 -- 172.21.15.52:0/3300797618 wait complete. 2024-04-02T13:57:59.444 INFO:tasks.cephfs.filesystem:scrub status for tag:3e36fec1-a689-47e7-89e6-f3e086025120 - {'path': '/', 'tag': '3e36fec1-a689-47e7-89e6-f3e086025120', 'options': 'recursive,force'} 2024-04-02T13:57:59.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:57:59 smithi189 ceph-mon[36683]: pgmap v137: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:00.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:00 smithi052 ceph-mon[32104]: scrub queued for path: / 2024-04-02T13:58:00.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:00 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:58:00.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:00 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:58:00.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:00 smithi088 ceph-mon[36528]: scrub queued for path: / 2024-04-02T13:58:00.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:00 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:58:00.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:00 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:58:00.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:00 smithi189 ceph-mon[36683]: scrub queued for path: / 2024-04-02T13:58:00.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:00 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:58:00.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:00 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:58:01.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:01 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:58:01.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:01 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:58:01.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:01 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:58:01.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:01 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:58:01.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:01 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:58:01.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:01 smithi052 ceph-mon[32104]: pgmap v138: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:01.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:01 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:58:01.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:01 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:58:01.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:01 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:58:01.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:01 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:58:01.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:01 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:58:01.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:01 smithi088 ceph-mon[36528]: pgmap v138: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:01.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:01 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:58:01.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:01 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:58:01.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:01 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:58:01.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:01 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:58:01.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:01 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:58:01.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:01 smithi189 ceph-mon[36683]: pgmap v138: 129 pgs: 129 active+clean; 73 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:03.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:03 smithi052 ceph-mon[32104]: pgmap v139: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T13:58:03.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:03 smithi088 ceph-mon[36528]: pgmap v139: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T13:58:03.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:03 smithi189 ceph-mon[36683]: pgmap v139: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T13:58:05.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:05 smithi052 ceph-mon[32104]: pgmap v140: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:58:05.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:05 smithi088 ceph-mon[36528]: pgmap v140: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:58:05.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:05 smithi189 ceph-mon[36683]: pgmap v140: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:58:06.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:06 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T13:58:06.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:06 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T13:58:06.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:06 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T13:58:06.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:06 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T13:58:06.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:06 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T13:58:06.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:06 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T13:58:07.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:07 smithi052 ceph-mon[32104]: pgmap v141: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T13:58:07.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:07 smithi088 ceph-mon[36528]: pgmap v141: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T13:58:07.439 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:07 smithi189 ceph-mon[36683]: pgmap v141: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T13:58:09.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:09 smithi052 ceph-mon[32104]: pgmap v142: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:58:09.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:09 smithi088 ceph-mon[36528]: pgmap v142: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:58:09.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:09 smithi189 ceph-mon[36683]: pgmap v142: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:58:11.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:11 smithi052 ceph-mon[32104]: pgmap v143: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:58:11.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:11 smithi088 ceph-mon[36528]: pgmap v143: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:58:11.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:11 smithi189 ceph-mon[36683]: pgmap v143: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:58:13.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:13 smithi052 ceph-mon[32104]: pgmap v144: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T13:58:13.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:13 smithi088 ceph-mon[36528]: pgmap v144: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T13:58:13.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:13 smithi189 ceph-mon[36683]: pgmap v144: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T13:58:15.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:15 smithi052 ceph-mon[32104]: pgmap v145: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T13:58:15.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:15 smithi088 ceph-mon[36528]: pgmap v145: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T13:58:15.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:15 smithi189 ceph-mon[36683]: pgmap v145: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T13:58:17.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:17 smithi088 ceph-mon[36528]: pgmap v146: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-02T13:58:17.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:17 smithi189 ceph-mon[36683]: pgmap v146: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-02T13:58:17.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:17 smithi052 ceph-mon[32104]: pgmap v146: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-02T13:58:19.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:19 smithi088 ceph-mon[36528]: pgmap v147: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:19.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:19 smithi189 ceph-mon[36683]: pgmap v147: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:19.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:19 smithi052 ceph-mon[32104]: pgmap v147: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:21.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:21 smithi088 ceph-mon[36528]: pgmap v148: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:21.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:21 smithi189 ceph-mon[36683]: pgmap v148: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:21.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:21 smithi052 ceph-mon[32104]: pgmap v148: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:23.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:23 smithi088 ceph-mon[36528]: pgmap v149: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:58:23.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:23 smithi189 ceph-mon[36683]: pgmap v149: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:58:23.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:23 smithi052 ceph-mon[32104]: pgmap v149: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:58:25.391 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:25 smithi052 ceph-mon[32104]: pgmap v150: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:25.419 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:25 smithi088 ceph-mon[36528]: pgmap v150: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:25.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:25 smithi189 ceph-mon[36683]: pgmap v150: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:27.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:27 smithi088 ceph-mon[36528]: pgmap v151: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:58:27.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:27 smithi189 ceph-mon[36683]: pgmap v151: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:58:27.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:27 smithi052 ceph-mon[32104]: pgmap v151: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:58:29.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:29 smithi088 ceph-mon[36528]: pgmap v152: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:29.445 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:58:29.445 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:58:29.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:29 smithi052 ceph-mon[32104]: pgmap v152: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:29.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:29 smithi189 ceph-mon[36683]: pgmap v152: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:29.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.559+0000 7f522f888640 1 -- 172.21.15.52:0/694318400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5228100270 msgr2=0x7f5228100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:29.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.559+0000 7f522f888640 1 --2- 172.21.15.52:0/694318400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5228100270 0x7f5228100650 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f5224009900 tx=0x7f522402bc30 comp rx=0 tx=0).stop 2024-04-02T13:58:29.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.559+0000 7f522f888640 1 -- 172.21.15.52:0/694318400 shutdown_connections 2024-04-02T13:58:29.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.559+0000 7f522f888640 1 --2- 172.21.15.52:0/694318400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5228106e80 0x7f522810ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.559+0000 7f522f888640 1 --2- 172.21.15.52:0/694318400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5228100c20 0x7f5228106750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.559+0000 7f522f888640 1 --2- 172.21.15.52:0/694318400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5228100270 0x7f5228100650 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.559+0000 7f522f888640 1 -- 172.21.15.52:0/694318400 >> 172.21.15.52:0/694318400 conn(0x7f5228076fc0 msgr2=0x7f52280fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:29.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.559+0000 7f522f888640 1 -- 172.21.15.52:0/694318400 shutdown_connections 2024-04-02T13:58:29.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.559+0000 7f522f888640 1 -- 172.21.15.52:0/694318400 wait complete. 2024-04-02T13:58:29.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.560+0000 7f522f888640 1 Processor -- start 2024-04-02T13:58:29.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.560+0000 7f522f888640 1 -- start start 2024-04-02T13:58:29.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.560+0000 7f522f888640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5228100270 0x7f52281983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:29.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.560+0000 7f522f888640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5228106e80 0x7f52281988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:29.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.560+0000 7f522f888640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f522819cc60 0x7f522819d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:29.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.560+0000 7f522f888640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5228111ee0 con 0x7f522819cc60 2024-04-02T13:58:29.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.560+0000 7f522f888640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5228111d60 con 0x7f5228106e80 2024-04-02T13:58:29.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.560+0000 7f522f888640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5228112060 con 0x7f5228100270 2024-04-02T13:58:29.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.560+0000 7f522de25640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f522819cc60 0x7f522819d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:29.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.560+0000 7f522de25640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f522819cc60 0x7f522819d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53530/0 (socket says 172.21.15.52:53530) 2024-04-02T13:58:29.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.560+0000 7f522de25640 1 -- 172.21.15.52:0/56438917 learned_addr learned my addr 172.21.15.52:0/56438917 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:29.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.560+0000 7f522d624640 1 --2- 172.21.15.52:0/56438917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5228100270 0x7f52281983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:29.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.560+0000 7f522ce23640 1 --2- 172.21.15.52:0/56438917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5228106e80 0x7f52281988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:29.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.561+0000 7f522de25640 1 -- 172.21.15.52:0/56438917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5228100270 msgr2=0x7f52281983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:29.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.561+0000 7f522de25640 1 --2- 172.21.15.52:0/56438917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5228100270 0x7f52281983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.561+0000 7f522de25640 1 -- 172.21.15.52:0/56438917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5228106e80 msgr2=0x7f52281988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:29.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.561+0000 7f522de25640 1 --2- 172.21.15.52:0/56438917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5228106e80 0x7f52281988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.561+0000 7f522de25640 1 -- 172.21.15.52:0/56438917 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f522819d750 con 0x7f522819cc60 2024-04-02T13:58:29.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.561+0000 7f522de25640 1 --2- 172.21.15.52:0/56438917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f522819cc60 0x7f522819d110 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f522000cce0 tx=0x7f522000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:29.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.561+0000 7f521e7fc640 1 -- 172.21.15.52:0/56438917 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522000f450 con 0x7f522819cc60 2024-04-02T13:58:29.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.561+0000 7f522f888640 1 -- 172.21.15.52:0/56438917 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f522819da40 con 0x7f522819cc60 2024-04-02T13:58:29.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.561+0000 7f521e7fc640 1 -- 172.21.15.52:0/56438917 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f522000fad0 con 0x7f522819cc60 2024-04-02T13:58:29.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.561+0000 7f521e7fc640 1 -- 172.21.15.52:0/56438917 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5220018c10 con 0x7f522819cc60 2024-04-02T13:58:29.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.561+0000 7f522f888640 1 -- 172.21.15.52:0/56438917 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52281a51e0 con 0x7f522819cc60 2024-04-02T13:58:29.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.564+0000 7f521e7fc640 1 -- 172.21.15.52:0/56438917 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f522000f5f0 con 0x7f522819cc60 2024-04-02T13:58:29.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.564+0000 7f523247c740 1 Processor -- start 2024-04-02T13:58:29.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.564+0000 7f523247c740 1 -- start start 2024-04-02T13:58:29.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.564+0000 7f523247c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22c93600 0x55dd22c939e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:29.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.564+0000 7f523247c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55dd22c93f20 0x55dd22ca0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:29.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.564+0000 7f523247c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd22ca11e0 0x55dd22ca35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:29.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.564+0000 7f523247c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55dd22bf7830 con 0x55dd22c93600 2024-04-02T13:58:29.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.564+0000 7f523247c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55dd22b9f820 con 0x55dd22c93f20 2024-04-02T13:58:29.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.564+0000 7f523247c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dd22bf37e0 con 0x55dd22ca11e0 2024-04-02T13:58:29.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.565+0000 7f522ce23640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22c93600 0x55dd22c939e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:29.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.565+0000 7f522ce23640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22c93600 0x55dd22c939e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53546/0 (socket says 172.21.15.52:53546) 2024-04-02T13:58:29.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.565+0000 7f522ce23640 1 -- 172.21.15.52:0/1558940630 learned_addr learned my addr 172.21.15.52:0/1558940630 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:29.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.565+0000 7f522d624640 1 --2- 172.21.15.52:0/1558940630 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55dd22c93f20 0x55dd22ca0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:29.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.565+0000 7f521e7fc640 1 --2- 172.21.15.52:0/56438917 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f52140a3d20 0x7f52140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:29.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.565+0000 7f522ce23640 1 --2- 172.21.15.52:0/1558940630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd22ca11e0 0x55dd22ca35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:29.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.565+0000 7f522ce23640 1 -- 172.21.15.52:0/1558940630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd22ca11e0 msgr2=0x55dd22ca35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:29.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.565+0000 7f522ce23640 1 --2- 172.21.15.52:0/1558940630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd22ca11e0 0x55dd22ca35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.565+0000 7f522ce23640 1 -- 172.21.15.52:0/1558940630 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55dd22c93f20 msgr2=0x55dd22ca0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:29.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.565+0000 7f522ce23640 1 --2- 172.21.15.52:0/1558940630 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55dd22c93f20 0x55dd22ca0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.565+0000 7f522ce23640 1 -- 172.21.15.52:0/1558940630 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd229d57d0 con 0x55dd22c93600 2024-04-02T13:58:29.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.565+0000 7f522de25640 1 --2- 172.21.15.52:0/56438917 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f52140a3d20 0x7f52140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:29.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.565+0000 7f521e7fc640 1 -- 172.21.15.52:0/56438917 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f52200d12a0 con 0x7f522819cc60 2024-04-02T13:58:29.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.565+0000 7f522ce23640 1 --2- 172.21.15.52:0/1558940630 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22c93600 0x55dd22c939e0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f521800df60 tx=0x7f521800d610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a61c40aa7d4001 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:29.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.565+0000 7f52137fe640 1 -- 172.21.15.52:0/1558940630 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218004030 con 0x55dd22c93600 2024-04-02T13:58:29.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.565+0000 7f52137fe640 1 -- 172.21.15.52:0/1558940630 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5218005500 con 0x55dd22c93600 2024-04-02T13:58:29.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.565+0000 7f52137fe640 1 -- 172.21.15.52:0/1558940630 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218004c60 con 0x55dd22c93600 2024-04-02T13:58:29.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.566+0000 7f523247c740 1 -- 172.21.15.52:0/1558940630 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22c93600 msgr2=0x55dd22c939e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:29.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.566+0000 7f523247c740 1 --2- 172.21.15.52:0/1558940630 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22c93600 0x55dd22c939e0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f521800df60 tx=0x7f521800d610 comp rx=0 tx=0).stop 2024-04-02T13:58:29.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.566+0000 7f522de25640 1 --2- 172.21.15.52:0/56438917 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f52140a3d20 0x7f52140a61e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f522009cba0 tx=0x7f522000af30 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:29.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.566+0000 7f523247c740 1 -- 172.21.15.52:0/1558940630 shutdown_connections 2024-04-02T13:58:29.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.566+0000 7f523247c740 1 --2- 172.21.15.52:0/1558940630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd22ca11e0 0x55dd22ca35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.566+0000 7f523247c740 1 --2- 172.21.15.52:0/1558940630 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55dd22c93f20 0x55dd22ca0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.566+0000 7f523247c740 1 --2- 172.21.15.52:0/1558940630 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22c93600 0x55dd22c939e0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.566+0000 7f523247c740 1 -- 172.21.15.52:0/1558940630 >> 172.21.15.52:0/1558940630 conn(0x55dd22c8e000 msgr2=0x55dd22c8e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:29.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.566+0000 7f523247c740 1 -- 172.21.15.52:0/1558940630 shutdown_connections 2024-04-02T13:58:29.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.566+0000 7f523247c740 1 -- 172.21.15.52:0/1558940630 wait complete. 2024-04-02T13:58:29.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.566+0000 7f523247c740 1 -- start start 2024-04-02T13:58:29.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.567+0000 7f523247c740 1 Processor -- start 2024-04-02T13:58:29.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.567+0000 7f523247c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55dd22c93600 0x55dd22ca0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:29.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.567+0000 7f523247c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22c93f20 0x55dd22c9de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:29.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.567+0000 7f523247c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd22ca11e0 0x55dd22c9e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:29.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.567+0000 7f523247c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55dd22ca3fa0 con 0x55dd22c93f20 2024-04-02T13:58:29.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.567+0000 7f523247c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55dd22bb2d30 con 0x55dd22c93600 2024-04-02T13:58:29.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.567+0000 7f523247c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dd22ca4120 con 0x55dd22ca11e0 2024-04-02T13:58:29.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.567+0000 7f523247c740 10 client.0 _ll_register_callbacks cb 0x55dd22bb2720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:58:29.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.567+0000 7f522d624640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22c93f20 0x55dd22c9de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:29.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.567+0000 7f522d624640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22c93f20 0x55dd22c9de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53562/0 (socket says 172.21.15.52:53562) 2024-04-02T13:58:29.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.567+0000 7f522d624640 1 -- 172.21.15.52:0/3179233911 learned_addr learned my addr 172.21.15.52:0/3179233911 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:29.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.567+0000 7f522ce23640 1 --2- 172.21.15.52:0/3179233911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55dd22c93600 0x55dd22ca0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:29.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.567+0000 7f522d624640 1 -- 172.21.15.52:0/3179233911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd22ca11e0 msgr2=0x55dd22c9e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:29.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.567+0000 7f522d624640 1 --2- 172.21.15.52:0/3179233911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd22ca11e0 0x55dd22c9e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.567+0000 7f522d624640 1 -- 172.21.15.52:0/3179233911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55dd22c93600 msgr2=0x55dd22ca0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:29.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.567+0000 7f522d624640 1 --2- 172.21.15.52:0/3179233911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55dd22c93600 0x55dd22ca0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.567+0000 7f522d624640 1 -- 172.21.15.52:0/3179233911 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd2290b3f0 con 0x55dd22c93f20 2024-04-02T13:58:29.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.568+0000 7f522d624640 1 --2- 172.21.15.52:0/3179233911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22c93f20 0x55dd22c9de40 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f5224009900 tx=0x7f5224005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e284d053742e91 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:29.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.568+0000 7f5210ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:58:29.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.568+0000 7f5210ff9640 1 -- 172.21.15.52:0/3179233911 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224005230 con 0x55dd22c93f20 2024-04-02T13:58:29.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.568+0000 7f5210ff9640 1 -- 172.21.15.52:0/3179233911 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f52240053d0 con 0x55dd22c93f20 2024-04-02T13:58:29.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.568+0000 7f523247c740 1 -- 172.21.15.52:0/3179233911 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5218005500 con 0x55dd22c93f20 2024-04-02T13:58:29.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.568+0000 7f5210ff9640 1 -- 172.21.15.52:0/3179233911 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5224004490 con 0x55dd22c93f20 2024-04-02T13:58:29.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.568+0000 7f523247c740 10 client.17004 fetch_fsmap learned FSMap version 30 2024-04-02T13:58:29.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.568+0000 7f523247c740 1 -- 172.21.15.52:0/3179233911 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dd228cb0f0 con 0x55dd22c93f20 2024-04-02T13:58:29.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.568+0000 7f5210ff9640 1 -- 172.21.15.52:0/3179233911 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5218005500 con 0x55dd22c93f20 2024-04-02T13:58:29.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.568+0000 7f5210ff9640 1 -- 172.21.15.52:0/3179233911 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5224003950 con 0x55dd22c93f20 2024-04-02T13:58:29.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.568+0000 7f523247c740 10 client.17004 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:58:29.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.568+0000 7f523247c740 10 client.17004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:58:29.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.568+0000 7f523247c740 20 client.17004 populate_metadata read hostname 'smithi052' 2024-04-02T13:58:29.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.568+0000 7f523247c740 1 --2- 172.21.15.52:0/3179233911 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55dd22cb1200 0x55dd22cb3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:29.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.568+0000 7f523247c740 4 client.17004 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:58:29.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.568+0000 7f523247c740 1 -- 172.21.15.52:0/3179233911 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd22c04d70 con 0x55dd22cb1200 2024-04-02T13:58:29.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.569+0000 7f522ce23640 1 --2- 172.21.15.52:0/3179233911 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55dd22cb1200 0x55dd22cb3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:29.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.569+0000 7f5210ff9640 10 client.17004 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:58:29.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.569+0000 7f522ce23640 1 --2- 172.21.15.52:0/3179233911 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55dd22cb1200 0x55dd22cb3650 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d42ad4bb3f40dad2 server_cookie=4f1b881e749e4c0f in_seq=0 out_seq=0 2024-04-02T13:58:29.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.570+0000 7f5210ff9640 1 -- 172.21.15.52:0/3179233911 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dd22c04d70 con 0x55dd22cb1200 2024-04-02T13:58:29.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.570+0000 7f5210ff9640 10 client.17004 handle_command_reply: tid=0 2024-04-02T13:58:29.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.570+0000 7f523247c740 1 client.17004 shutdown 2024-04-02T13:58:29.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.570+0000 7f523247c740 1 -- 172.21.15.52:0/3179233911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22c93f20 msgr2=0x55dd22c9de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:29.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.570+0000 7f523247c740 1 --2- 172.21.15.52:0/3179233911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22c93f20 0x55dd22c9de40 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f5224009900 tx=0x7f5224005e20 comp rx=0 tx=0).stop 2024-04-02T13:58:29.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.570+0000 7f523247c740 1 -- 172.21.15.52:0/3179233911 shutdown_connections 2024-04-02T13:58:29.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.570+0000 7f523247c740 1 --2- 172.21.15.52:0/3179233911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd22ca11e0 0x55dd22c9e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.570+0000 7f523247c740 1 --2- 172.21.15.52:0/3179233911 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55dd22cb1200 0x55dd22cb3650 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.570+0000 7f523247c740 1 --2- 172.21.15.52:0/3179233911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22c93f20 0x55dd22c9de40 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.570+0000 7f523247c740 1 --2- 172.21.15.52:0/3179233911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55dd22c93600 0x55dd22ca0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.570+0000 7f5210ff9640 0 client.17004 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:58:29.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.570+0000 7f523247c740 1 -- 172.21.15.52:0/3179233911 >> 172.21.15.52:0/3179233911 conn(0x55dd22c8e000 msgr2=0x55dd22c8fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:29.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.570+0000 7f523247c740 1 -- 172.21.15.52:0/3179233911 shutdown_connections 2024-04-02T13:58:29.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.570+0000 7f523247c740 1 -- 172.21.15.52:0/3179233911 wait complete. 2024-04-02T13:58:29.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.570+0000 7f523247c740 20 client.17004 trim_cache size 0 max 16384 2024-04-02T13:58:29.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.588+0000 7f523247c740 1 Processor -- start 2024-04-02T13:58:29.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.588+0000 7f523247c740 1 -- start start 2024-04-02T13:58:29.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.588+0000 7f523247c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22c93600 0x55dd22cacea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:29.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.588+0000 7f523247c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55dd22c93f20 0x55dd22ca0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:29.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.588+0000 7f523247c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd22ca11e0 0x55dd22ca0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:29.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.588+0000 7f523247c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f52281acb50 con 0x55dd22c93600 2024-04-02T13:58:29.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.588+0000 7f523247c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f52281ae690 con 0x55dd22c93f20 2024-04-02T13:58:29.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.588+0000 7f523247c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f52281ad220 con 0x55dd22ca11e0 2024-04-02T13:58:29.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.588+0000 7f522ce23640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22c93600 0x55dd22cacea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:29.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.588+0000 7f522ce23640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22c93600 0x55dd22cacea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53570/0 (socket says 172.21.15.52:53570) 2024-04-02T13:58:29.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.588+0000 7f522ce23640 1 -- 172.21.15.52:0/1212996344 learned_addr learned my addr 172.21.15.52:0/1212996344 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:29.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.588+0000 7f522d624640 1 --2- 172.21.15.52:0/1212996344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55dd22c93f20 0x55dd22ca0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:29.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.588+0000 7f522ce23640 1 -- 172.21.15.52:0/1212996344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd22ca11e0 msgr2=0x55dd22ca0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:29.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.588+0000 7f522ce23640 1 --2- 172.21.15.52:0/1212996344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd22ca11e0 0x55dd22ca0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.588+0000 7f522ce23640 1 -- 172.21.15.52:0/1212996344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55dd22c93f20 msgr2=0x55dd22ca0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:29.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.588+0000 7f522ce23640 1 --2- 172.21.15.52:0/1212996344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55dd22c93f20 0x55dd22ca0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.588+0000 7f522ce23640 1 -- 172.21.15.52:0/1212996344 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd22c8de30 con 0x55dd22c93600 2024-04-02T13:58:29.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.588+0000 7f522d624640 1 --2- 172.21.15.52:0/1212996344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55dd22c93f20 0x55dd22ca0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:58:29.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.588+0000 7f522ce23640 1 --2- 172.21.15.52:0/1212996344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22c93600 0x55dd22cacea0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f52180134f0 tx=0x7f521800d350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f399de8da7608969 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:29.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.589+0000 7f5212ffd640 1 -- 172.21.15.52:0/1212996344 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521802f020 con 0x55dd22c93600 2024-04-02T13:58:29.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.589+0000 7f5212ffd640 1 -- 172.21.15.52:0/1212996344 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f521802b900 con 0x55dd22c93600 2024-04-02T13:58:29.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.589+0000 7f5212ffd640 1 -- 172.21.15.52:0/1212996344 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218011e80 con 0x55dd22c93600 2024-04-02T13:58:29.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.589+0000 7f523247c740 1 -- 172.21.15.52:0/1212996344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22c93600 msgr2=0x55dd22cacea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:29.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.589+0000 7f523247c740 1 --2- 172.21.15.52:0/1212996344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22c93600 0x55dd22cacea0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f52180134f0 tx=0x7f521800d350 comp rx=0 tx=0).stop 2024-04-02T13:58:29.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.589+0000 7f523247c740 1 -- 172.21.15.52:0/1212996344 shutdown_connections 2024-04-02T13:58:29.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.589+0000 7f523247c740 1 --2- 172.21.15.52:0/1212996344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd22ca11e0 0x55dd22ca0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.589+0000 7f523247c740 1 --2- 172.21.15.52:0/1212996344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55dd22c93f20 0x55dd22ca0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.589+0000 7f523247c740 1 --2- 172.21.15.52:0/1212996344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22c93600 0x55dd22cacea0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.589+0000 7f523247c740 1 -- 172.21.15.52:0/1212996344 >> 172.21.15.52:0/1212996344 conn(0x55dd22c8e000 msgr2=0x55dd22c994d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:29.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.589+0000 7f523247c740 1 -- 172.21.15.52:0/1212996344 shutdown_connections 2024-04-02T13:58:29.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.589+0000 7f523247c740 1 -- 172.21.15.52:0/1212996344 wait complete. 2024-04-02T13:58:29.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.589+0000 7f523247c740 1 -- start start 2024-04-02T13:58:29.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.590+0000 7f523247c740 1 Processor -- start 2024-04-02T13:58:29.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.590+0000 7f523247c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd22c93600 0x55dd22c9aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:29.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.590+0000 7f523247c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55dd22c93f20 0x55dd22c9afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:29.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.590+0000 7f523247c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22ca11e0 0x55dd22c9b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:29.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.590+0000 7f523247c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f52281adfc0 con 0x55dd22ca11e0 2024-04-02T13:58:29.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.590+0000 7f523247c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f52281ad8f0 con 0x55dd22c93f20 2024-04-02T13:58:29.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.590+0000 7f523247c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f52281aab30 con 0x55dd22c93600 2024-04-02T13:58:29.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.590+0000 7f523247c740 10 client.0 _ll_register_callbacks cb 0x7f522402cd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:58:29.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.590+0000 7f522ce23640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22ca11e0 0x55dd22c9b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:29.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.590+0000 7f522ce23640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22ca11e0 0x55dd22c9b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53578/0 (socket says 172.21.15.52:53578) 2024-04-02T13:58:29.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.590+0000 7f522ce23640 1 -- 172.21.15.52:0/3856634601 learned_addr learned my addr 172.21.15.52:0/3856634601 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:29.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.590+0000 7f522ce23640 1 --2- 172.21.15.52:0/3856634601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd22c93600 0x55dd22c9aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:29.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.590+0000 7f522d624640 1 --2- 172.21.15.52:0/3856634601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55dd22c93f20 0x55dd22c9afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:29.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.590+0000 7f522ce23640 1 -- 172.21.15.52:0/3856634601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd22c93600 msgr2=0x55dd22c9aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:29.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.590+0000 7f522ce23640 1 --2- 172.21.15.52:0/3856634601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd22c93600 0x55dd22c9aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.590+0000 7f522ce23640 1 -- 172.21.15.52:0/3856634601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55dd22c93f20 msgr2=0x55dd22c9afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.590+0000 7f522ce23640 1 --2- 172.21.15.52:0/3856634601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55dd22c93f20 0x55dd22c9afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.590+0000 7f522ce23640 1 -- 172.21.15.52:0/3856634601 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd22ca4110 con 0x55dd22ca11e0 2024-04-02T13:58:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.591+0000 7f522ce23640 1 --2- 172.21.15.52:0/3856634601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22ca11e0 0x55dd22c9b510 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f5218016040 tx=0x7f5218005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9f9bca9cde04d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.591+0000 7f5210ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:58:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.591+0000 7f5210ff9640 1 -- 172.21.15.52:0/3856634601 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521802f050 con 0x55dd22ca11e0 2024-04-02T13:58:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.591+0000 7f5210ff9640 1 -- 172.21.15.52:0/3856634601 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f52180114c0 con 0x55dd22ca11e0 2024-04-02T13:58:29.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.591+0000 7f523247c740 1 -- 172.21.15.52:0/3856634601 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f521802b900 con 0x55dd22ca11e0 2024-04-02T13:58:29.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.591+0000 7f5210ff9640 1 -- 172.21.15.52:0/3856634601 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5218047050 con 0x55dd22ca11e0 2024-04-02T13:58:29.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.591+0000 7f523247c740 10 client.17010 fetch_fsmap learned FSMap version 30 2024-04-02T13:58:29.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.591+0000 7f523247c740 1 -- 172.21.15.52:0/3856634601 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dd22d177a0 con 0x55dd22ca11e0 2024-04-02T13:58:29.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.591+0000 7f5210ff9640 1 -- 172.21.15.52:0/3856634601 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f521802b900 con 0x55dd22ca11e0 2024-04-02T13:58:29.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.591+0000 7f5210ff9640 1 -- 172.21.15.52:0/3856634601 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5218003790 con 0x55dd22ca11e0 2024-04-02T13:58:29.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.591+0000 7f523247c740 10 client.17010 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:58:29.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.591+0000 7f523247c740 10 client.17010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:58:29.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.591+0000 7f523247c740 20 client.17010 populate_metadata read hostname 'smithi052' 2024-04-02T13:58:29.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.591+0000 7f523247c740 1 --2- 172.21.15.52:0/3856634601 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55dd22d17c10 0x55dd22d1a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:29.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.591+0000 7f523247c740 4 client.17010 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:58:29.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.591+0000 7f523247c740 1 -- 172.21.15.52:0/3856634601 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dd22bff0c0 con 0x55dd22d17c10 2024-04-02T13:58:29.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.592+0000 7f522d624640 1 --2- 172.21.15.52:0/3856634601 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55dd22d17c10 0x55dd22d1a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:29.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.592+0000 7f522d624640 1 --2- 172.21.15.52:0/3856634601 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55dd22d17c10 0x55dd22d1a000 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a2f40981fb7c6a1 server_cookie=48cde3d26719e3a0 in_seq=0 out_seq=0 2024-04-02T13:58:29.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.592+0000 7f5210ff9640 10 client.17010 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:58:29.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.592+0000 7f5210ff9640 1 -- 172.21.15.52:0/3856634601 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dd22bff0c0 con 0x55dd22d17c10 2024-04-02T13:58:29.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.592+0000 7f5210ff9640 10 client.17010 handle_command_reply: tid=0 2024-04-02T13:58:29.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.592+0000 7f523247c740 1 client.17010 shutdown 2024-04-02T13:58:29.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.592+0000 7f523247c740 1 -- 172.21.15.52:0/3856634601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22ca11e0 msgr2=0x55dd22c9b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:29.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.592+0000 7f523247c740 1 --2- 172.21.15.52:0/3856634601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22ca11e0 0x55dd22c9b510 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f5218016040 tx=0x7f5218005970 comp rx=0 tx=0).stop 2024-04-02T13:58:29.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.592+0000 7f523247c740 1 -- 172.21.15.52:0/3856634601 shutdown_connections 2024-04-02T13:58:29.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.592+0000 7f523247c740 1 --2- 172.21.15.52:0/3856634601 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55dd22d17c10 0x55dd22d1a000 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.592+0000 7f523247c740 1 --2- 172.21.15.52:0/3856634601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd22ca11e0 0x55dd22c9b510 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.592+0000 7f523247c740 1 --2- 172.21.15.52:0/3856634601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55dd22c93f20 0x55dd22c9afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.592+0000 7f523247c740 1 --2- 172.21.15.52:0/3856634601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd22c93600 0x55dd22c9aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.592+0000 7f5210ff9640 0 client.17010 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:58:29.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.592+0000 7f523247c740 1 -- 172.21.15.52:0/3856634601 >> 172.21.15.52:0/3856634601 conn(0x55dd22c8e000 msgr2=0x55dd22c8c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:29.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.592+0000 7f523247c740 1 -- 172.21.15.52:0/3856634601 shutdown_connections 2024-04-02T13:58:29.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.593+0000 7f523247c740 1 -- 172.21.15.52:0/3856634601 wait complete. 2024-04-02T13:58:29.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.593+0000 7f523247c740 20 client.17010 trim_cache size 0 max 16384 2024-04-02T13:58:29.611 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:58:29.611 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-04-02T13:58:29.611 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-04-02T13:58:29.611 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:58:29.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.593+0000 7f522f888640 1 -- 172.21.15.52:0/56438917 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f52140a3d20 msgr2=0x7f52140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:29.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.593+0000 7f522f888640 1 --2- 172.21.15.52:0/56438917 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f52140a3d20 0x7f52140a61e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f522009cba0 tx=0x7f522000af30 comp rx=0 tx=0).stop 2024-04-02T13:58:29.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.593+0000 7f522f888640 1 -- 172.21.15.52:0/56438917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f522819cc60 msgr2=0x7f522819d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:29.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.593+0000 7f522f888640 1 --2- 172.21.15.52:0/56438917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f522819cc60 0x7f522819d110 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f522000cce0 tx=0x7f522000a620 comp rx=0 tx=0).stop 2024-04-02T13:58:29.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.593+0000 7f522f888640 1 -- 172.21.15.52:0/56438917 shutdown_connections 2024-04-02T13:58:29.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.593+0000 7f522f888640 1 --2- 172.21.15.52:0/56438917 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f52140a3d20 0x7f52140a61e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.593+0000 7f522f888640 1 --2- 172.21.15.52:0/56438917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f522819cc60 0x7f522819d110 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.593+0000 7f522f888640 1 --2- 172.21.15.52:0/56438917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5228106e80 0x7f52281988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.593+0000 7f522f888640 1 --2- 172.21.15.52:0/56438917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5228100270 0x7f52281983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.593+0000 7f522f888640 1 -- 172.21.15.52:0/56438917 >> 172.21.15.52:0/56438917 conn(0x7f5228076fc0 msgr2=0x7f5228104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:29.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.593+0000 7f522f888640 1 -- 172.21.15.52:0/56438917 shutdown_connections 2024-04-02T13:58:29.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.593+0000 7f522f888640 1 -- 172.21.15.52:0/56438917 wait complete. 2024-04-02T13:58:29.616 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:58:29.616 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:58:29.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.754+0000 7f70dc2f2640 1 -- 172.21.15.52:0/1825079153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70d4102450 msgr2=0x7f70d410e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:29.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.754+0000 7f70dc2f2640 1 --2- 172.21.15.52:0/1825079153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70d4102450 0x7f70d410e960 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f70cc009cd0 tx=0x7f70cc02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.754+0000 7f70dc2f2640 1 -- 172.21.15.52:0/1825079153 shutdown_connections 2024-04-02T13:58:29.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.754+0000 7f70dc2f2640 1 --2- 172.21.15.52:0/1825079153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70d4102450 0x7f70d410e960 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.754+0000 7f70dc2f2640 1 --2- 172.21.15.52:0/1825079153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70d4101a90 0x7f70d4101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.754+0000 7f70dc2f2640 1 --2- 172.21.15.52:0/1825079153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70d4107aa0 0x7f70d4107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.754+0000 7f70dc2f2640 1 -- 172.21.15.52:0/1825079153 >> 172.21.15.52:0/1825079153 conn(0x7f70d40fd5a0 msgr2=0x7f70d40ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:29.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.754+0000 7f70dc2f2640 1 -- 172.21.15.52:0/1825079153 shutdown_connections 2024-04-02T13:58:29.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.754+0000 7f70dc2f2640 1 -- 172.21.15.52:0/1825079153 wait complete. 2024-04-02T13:58:29.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.755+0000 7f70dc2f2640 1 Processor -- start 2024-04-02T13:58:29.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.755+0000 7f70dc2f2640 1 -- start start 2024-04-02T13:58:29.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.755+0000 7f70dc2f2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70d4101a90 0x7f70d4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:29.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.755+0000 7f70dc2f2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70d4102450 0x7f70d41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:29.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.755+0000 7f70dc2f2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70d4107aa0 0x7f70d419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:29.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.755+0000 7f70dc2f2640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f70d4111eb0 con 0x7f70d4107aa0 2024-04-02T13:58:29.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.755+0000 7f70dc2f2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f70d4111d30 con 0x7f70d4102450 2024-04-02T13:58:29.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.755+0000 7f70dc2f2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f70d4112030 con 0x7f70d4101a90 2024-04-02T13:58:29.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.755+0000 7f70da88f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70d4107aa0 0x7f70d419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:29.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.755+0000 7f70da88f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70d4107aa0 0x7f70d419cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53600/0 (socket says 172.21.15.52:53600) 2024-04-02T13:58:29.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.755+0000 7f70da88f640 1 -- 172.21.15.52:0/3556182963 learned_addr learned my addr 172.21.15.52:0/3556182963 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:29.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.755+0000 7f70d988d640 1 --2- 172.21.15.52:0/3556182963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70d4102450 0x7f70d41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:29.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.755+0000 7f70da08e640 1 --2- 172.21.15.52:0/3556182963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70d4101a90 0x7f70d4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:29.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.755+0000 7f70da88f640 1 -- 172.21.15.52:0/3556182963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70d4101a90 msgr2=0x7f70d4198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:29.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.755+0000 7f70da88f640 1 --2- 172.21.15.52:0/3556182963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70d4101a90 0x7f70d4198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.755+0000 7f70da88f640 1 -- 172.21.15.52:0/3556182963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70d4102450 msgr2=0x7f70d41988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:29.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.755+0000 7f70da88f640 1 --2- 172.21.15.52:0/3556182963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70d4102450 0x7f70d41988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.755+0000 7f70da88f640 1 -- 172.21.15.52:0/3556182963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d419d360 con 0x7f70d4107aa0 2024-04-02T13:58:29.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.755+0000 7f70da08e640 1 --2- 172.21.15.52:0/3556182963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70d4101a90 0x7f70d4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:58:29.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.755+0000 7f70da88f640 1 --2- 172.21.15.52:0/3556182963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70d4107aa0 0x7f70d419cc30 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f70cc004af0 tx=0x7f70cc032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:29.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.756+0000 7f70cb7fe640 1 -- 172.21.15.52:0/3556182963 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc041020 con 0x7f70d4107aa0 2024-04-02T13:58:29.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.756+0000 7f70dc2f2640 1 -- 172.21.15.52:0/3556182963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d419d5f0 con 0x7f70d4107aa0 2024-04-02T13:58:29.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.756+0000 7f70cb7fe640 1 -- 172.21.15.52:0/3556182963 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70cc034cd0 con 0x7f70d4107aa0 2024-04-02T13:58:29.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.756+0000 7f70cb7fe640 1 -- 172.21.15.52:0/3556182963 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc039040 con 0x7f70d4107aa0 2024-04-02T13:58:29.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.756+0000 7f70dc2f2640 1 -- 172.21.15.52:0/3556182963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d41a4ee0 con 0x7f70d4107aa0 2024-04-02T13:58:29.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.757+0000 7f70dc2f2640 1 -- 172.21.15.52:0/3556182963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d4068d40 con 0x7f70d4107aa0 2024-04-02T13:58:29.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.759+0000 7f70cb7fe640 1 -- 172.21.15.52:0/3556182963 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f70cc044020 con 0x7f70d4107aa0 2024-04-02T13:58:29.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.759+0000 7f70cb7fe640 1 --2- 172.21.15.52:0/3556182963 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f70c00a3d20 0x7f70c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:29.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.759+0000 7f70cb7fe640 1 -- 172.21.15.52:0/3556182963 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f70cc0f2c20 con 0x7f70d4107aa0 2024-04-02T13:58:29.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.759+0000 7f70da08e640 1 --2- 172.21.15.52:0/3556182963 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f70c00a3d20 0x7f70c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:29.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.760+0000 7f70da08e640 1 --2- 172.21.15.52:0/3556182963 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f70c00a3d20 0x7f70c00a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009ad0 tx=0x7f70d0005f90 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:29.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.763+0000 7f70cb7fe640 1 -- 172.21.15.52:0/3556182963 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f70cc04b020 con 0x7f70d4107aa0 2024-04-02T13:58:29.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.988+0000 7f70dc2f2640 1 -- 172.21.15.52:0/3556182963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70d410a020 con 0x7f70d4107aa0 2024-04-02T13:58:29.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.989+0000 7f70cb7fe640 1 -- 172.21.15.52:0/3556182963 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11923 (secure 0 0 0) 0x7f70cc0bf030 con 0x7f70d4107aa0 2024-04-02T13:58:29.993 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:58:29.993 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:57:45.240902+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24467,24457,24490,24475,24503]},"id":1}]} 2024-04-02T13:58:29.994 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-04-02T13:58:29.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.991+0000 7f70dc2f2640 1 -- 172.21.15.52:0/3556182963 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f70c00a3d20 msgr2=0x7f70c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:29.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.991+0000 7f70dc2f2640 1 --2- 172.21.15.52:0/3556182963 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f70c00a3d20 0x7f70c00a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009ad0 tx=0x7f70d0005f90 comp rx=0 tx=0).stop 2024-04-02T13:58:29.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.991+0000 7f70dc2f2640 1 -- 172.21.15.52:0/3556182963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70d4107aa0 msgr2=0x7f70d419cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:29.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.991+0000 7f70dc2f2640 1 --2- 172.21.15.52:0/3556182963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70d4107aa0 0x7f70d419cc30 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f70cc004af0 tx=0x7f70cc032ec0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.991+0000 7f70dc2f2640 1 -- 172.21.15.52:0/3556182963 shutdown_connections 2024-04-02T13:58:29.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.991+0000 7f70dc2f2640 1 --2- 172.21.15.52:0/3556182963 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f70c00a3d20 0x7f70c00a61e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.991+0000 7f70dc2f2640 1 --2- 172.21.15.52:0/3556182963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70d4107aa0 0x7f70d419cc30 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.991+0000 7f70dc2f2640 1 --2- 172.21.15.52:0/3556182963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70d4102450 0x7f70d41988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.991+0000 7f70dc2f2640 1 --2- 172.21.15.52:0/3556182963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70d4101a90 0x7f70d4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:29.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.991+0000 7f70dc2f2640 1 -- 172.21.15.52:0/3556182963 >> 172.21.15.52:0/3556182963 conn(0x7f70d40fd5a0 msgr2=0x7f70d40fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:29.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.991+0000 7f70dc2f2640 1 -- 172.21.15.52:0/3556182963 shutdown_connections 2024-04-02T13:58:29.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:29.991+0000 7f70dc2f2640 1 -- 172.21.15.52:0/3556182963 wait complete. 2024-04-02T13:58:30.006 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:58:30.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.109+0000 7fe121cd9640 1 -- 172.21.15.52:0/3479299160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe11c068d00 msgr2=0x7fe11c069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.109+0000 7fe121cd9640 1 --2- 172.21.15.52:0/3479299160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe11c068d00 0x7fe11c069180 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fe110009930 tx=0x7fe11002bc30 comp rx=0 tx=0).stop 2024-04-02T13:58:30.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.109+0000 7fe121cd9640 1 -- 172.21.15.52:0/3479299160 shutdown_connections 2024-04-02T13:58:30.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.109+0000 7fe121cd9640 1 --2- 172.21.15.52:0/3479299160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe11c0696c0 0x7fe11c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.109+0000 7fe121cd9640 1 --2- 172.21.15.52:0/3479299160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe11c068d00 0x7fe11c069180 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.109+0000 7fe121cd9640 1 --2- 172.21.15.52:0/3479299160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe11c107b50 0x7fe11c107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.109+0000 7fe121cd9640 1 -- 172.21.15.52:0/3479299160 >> 172.21.15.52:0/3479299160 conn(0x7fe11c077b00 msgr2=0x7fe11c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.109+0000 7fe121cd9640 1 -- 172.21.15.52:0/3479299160 shutdown_connections 2024-04-02T13:58:30.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.109+0000 7fe121cd9640 1 -- 172.21.15.52:0/3479299160 wait complete. 2024-04-02T13:58:30.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.110+0000 7fe121cd9640 1 Processor -- start 2024-04-02T13:58:30.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.110+0000 7fe121cd9640 1 -- start start 2024-04-02T13:58:30.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.110+0000 7fe121cd9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe11c068d00 0x7fe11c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.110+0000 7fe121cd9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe11c0696c0 0x7fe11c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.110+0000 7fe121cd9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe11c107b50 0x7fe11c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.110+0000 7fe121cd9640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c111ea0 con 0x7fe11c068d00 2024-04-02T13:58:30.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.110+0000 7fe121cd9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c111d20 con 0x7fe11c107b50 2024-04-02T13:58:30.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.110+0000 7fe121cd9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c112020 con 0x7fe11c0696c0 2024-04-02T13:58:30.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.110+0000 7fe11b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe11c068d00 0x7fe11c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.110+0000 7fe11b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe11c068d00 0x7fe11c198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53606/0 (socket says 172.21.15.52:53606) 2024-04-02T13:58:30.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.110+0000 7fe11b7fe640 1 -- 172.21.15.52:0/3158150942 learned_addr learned my addr 172.21.15.52:0/3158150942 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:30.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.110+0000 7fe11affd640 1 --2- 172.21.15.52:0/3158150942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe11c0696c0 0x7fe11c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.110+0000 7fe11bfff640 1 --2- 172.21.15.52:0/3158150942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe11c107b50 0x7fe11c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.110+0000 7fe11b7fe640 1 -- 172.21.15.52:0/3158150942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe11c0696c0 msgr2=0x7fe11c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.110+0000 7fe11b7fe640 1 --2- 172.21.15.52:0/3158150942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe11c0696c0 0x7fe11c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.110+0000 7fe11b7fe640 1 -- 172.21.15.52:0/3158150942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe11c107b50 msgr2=0x7fe11c19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.110+0000 7fe11b7fe640 1 --2- 172.21.15.52:0/3158150942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe11c107b50 0x7fe11c19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.110+0000 7fe11b7fe640 1 -- 172.21.15.52:0/3158150942 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe11c19d350 con 0x7fe11c068d00 2024-04-02T13:58:30.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.111+0000 7fe11b7fe640 1 --2- 172.21.15.52:0/3158150942 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe11c068d00 0x7fe11c198360 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00e9f0 tx=0x7fe10c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.111+0000 7fe118ff9640 1 -- 172.21.15.52:0/3158150942 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c00cd80 con 0x7fe11c068d00 2024-04-02T13:58:30.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.111+0000 7fe118ff9640 1 -- 172.21.15.52:0/3158150942 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe10c010910 con 0x7fe11c068d00 2024-04-02T13:58:30.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.111+0000 7fe118ff9640 1 -- 172.21.15.52:0/3158150942 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c018990 con 0x7fe11c068d00 2024-04-02T13:58:30.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.111+0000 7fe121cd9640 1 -- 172.21.15.52:0/3158150942 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe11c19d640 con 0x7fe11c068d00 2024-04-02T13:58:30.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.111+0000 7fe121cd9640 1 -- 172.21.15.52:0/3158150942 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe11c1a4ed0 con 0x7fe11c068d00 2024-04-02T13:58:30.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.114+0000 7fe118ff9640 1 -- 172.21.15.52:0/3158150942 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe10c010ac0 con 0x7fe11c068d00 2024-04-02T13:58:30.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.114+0000 7fe1249a0740 1 Processor -- start 2024-04-02T13:58:30.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.114+0000 7fe1249a0740 1 -- start start 2024-04-02T13:58:30.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.114+0000 7fe1249a0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd929bf7a0 0x55cd929bfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.114+0000 7fe1249a0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd929c00c0 0x55cd929c8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.114+0000 7fe1249a0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929c8ca0 0x55cd929cb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.114+0000 7fe1249a0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cd929237b0 con 0x55cd929c8ca0 2024-04-02T13:58:30.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.114+0000 7fe1249a0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cd928c2570 con 0x55cd929c00c0 2024-04-02T13:58:30.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.114+0000 7fe1249a0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cd9291f760 con 0x55cd929bf7a0 2024-04-02T13:58:30.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.114+0000 7fe118ff9640 1 --2- 172.21.15.52:0/3158150942 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe1040a3da0 0x7fe1040a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.114+0000 7fe11affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929c8ca0 0x55cd929cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.114+0000 7fe11bfff640 1 --2- 172.21.15.52:0/3158150942 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe1040a3da0 0x7fe1040a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.114+0000 7fe11b7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd929c00c0 0x55cd929c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.114+0000 7fe11affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929c8ca0 0x55cd929cb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53620/0 (socket says 172.21.15.52:53620) 2024-04-02T13:58:30.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.115+0000 7fe11affd640 1 -- 172.21.15.52:0/3757901944 learned_addr learned my addr 172.21.15.52:0/3757901944 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:30.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.115+0000 7fe11bfff640 1 --2- 172.21.15.52:0/3757901944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd929bf7a0 0x55cd929bfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.115+0000 7fe118ff9640 1 -- 172.21.15.52:0/3158150942 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe10c0d0dc0 con 0x7fe11c068d00 2024-04-02T13:58:30.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.115+0000 7fe11b7fe640 1 -- 172.21.15.52:0/3757901944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd929bf7a0 msgr2=0x55cd929bfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.115+0000 7fe11b7fe640 1 --2- 172.21.15.52:0/3757901944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd929bf7a0 0x55cd929bfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.115+0000 7fe11b7fe640 1 -- 172.21.15.52:0/3757901944 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929c8ca0 msgr2=0x55cd929cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.115+0000 7fe11b7fe640 1 --2- 172.21.15.52:0/3757901944 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929c8ca0 0x55cd929cb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.115+0000 7fe11b7fe640 1 -- 172.21.15.52:0/3757901944 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd926373f0 con 0x55cd929c00c0 2024-04-02T13:58:30.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.115+0000 7fe11affd640 1 --2- 172.21.15.52:0/3757901944 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929c8ca0 0x55cd929cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:58:30.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.115+0000 7fe11bfff640 1 --2- 172.21.15.52:0/3158150942 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe1040a3da0 0x7fe1040a6260 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe114005fd0 tx=0x7fe114009f00 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.115+0000 7fe11b7fe640 1 --2- 172.21.15.52:0/3757901944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd929c00c0 0x55cd929c8760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe10c0ce1d0 tx=0x7fe10c018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2729539377f59f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.117+0000 7fe101ffb640 1 -- 172.21.15.52:0/3757901944 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c026040 con 0x55cd929c00c0 2024-04-02T13:58:30.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.117+0000 7fe101ffb640 1 -- 172.21.15.52:0/3757901944 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe10c09cad0 con 0x55cd929c00c0 2024-04-02T13:58:30.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.117+0000 7fe101ffb640 1 -- 172.21.15.52:0/3757901944 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c0bbe00 con 0x55cd929c00c0 2024-04-02T13:58:30.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.117+0000 7fe1249a0740 1 -- 172.21.15.52:0/3757901944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd929c00c0 msgr2=0x55cd929c8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.117+0000 7fe1249a0740 1 --2- 172.21.15.52:0/3757901944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd929c00c0 0x55cd929c8760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe10c0ce1d0 tx=0x7fe10c018420 comp rx=0 tx=0).stop 2024-04-02T13:58:30.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.117+0000 7fe1249a0740 1 -- 172.21.15.52:0/3757901944 shutdown_connections 2024-04-02T13:58:30.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.117+0000 7fe1249a0740 1 --2- 172.21.15.52:0/3757901944 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929c8ca0 0x55cd929cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.117+0000 7fe1249a0740 1 --2- 172.21.15.52:0/3757901944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd929c00c0 0x55cd929c8760 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.117+0000 7fe1249a0740 1 --2- 172.21.15.52:0/3757901944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd929bf7a0 0x55cd929bfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.117+0000 7fe1249a0740 1 -- 172.21.15.52:0/3757901944 >> 172.21.15.52:0/3757901944 conn(0x55cd929b9ed0 msgr2=0x55cd929ba2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.117+0000 7fe1249a0740 1 -- 172.21.15.52:0/3757901944 shutdown_connections 2024-04-02T13:58:30.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.117+0000 7fe1249a0740 1 -- 172.21.15.52:0/3757901944 wait complete. 2024-04-02T13:58:30.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.118+0000 7fe1249a0740 1 -- start start 2024-04-02T13:58:30.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.118+0000 7fe1249a0740 1 Processor -- start 2024-04-02T13:58:30.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.118+0000 7fe1249a0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd929bf7a0 0x55cd929c8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.118+0000 7fe1249a0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd929c00c0 0x55cd929c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.118+0000 7fe1249a0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929c8ca0 0x55cd929c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.118+0000 7fe1249a0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cd929cb9d0 con 0x55cd929c8ca0 2024-04-02T13:58:30.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.118+0000 7fe1249a0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cd928dee80 con 0x55cd929c00c0 2024-04-02T13:58:30.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.118+0000 7fe1249a0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cd929cbb50 con 0x55cd929bf7a0 2024-04-02T13:58:30.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.118+0000 7fe1249a0740 10 client.0 _ll_register_callbacks cb 0x55cd9291aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:58:30.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.118+0000 7fe11bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929c8ca0 0x55cd929c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.118+0000 7fe11bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929c8ca0 0x55cd929c61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53630/0 (socket says 172.21.15.52:53630) 2024-04-02T13:58:30.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.118+0000 7fe11bfff640 1 -- 172.21.15.52:0/1802558859 learned_addr learned my addr 172.21.15.52:0/1802558859 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:30.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.118+0000 7fe11b7fe640 1 --2- 172.21.15.52:0/1802558859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd929bf7a0 0x55cd929c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.118+0000 7fe11affd640 1 --2- 172.21.15.52:0/1802558859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd929c00c0 0x55cd929c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.119+0000 7fe11bfff640 1 -- 172.21.15.52:0/1802558859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd929bf7a0 msgr2=0x55cd929c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.119+0000 7fe11bfff640 1 --2- 172.21.15.52:0/1802558859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd929bf7a0 0x55cd929c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.119+0000 7fe11bfff640 1 -- 172.21.15.52:0/1802558859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd929c00c0 msgr2=0x55cd929c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.119+0000 7fe11bfff640 1 --2- 172.21.15.52:0/1802558859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd929c00c0 0x55cd929c5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.119+0000 7fe11bfff640 1 -- 172.21.15.52:0/1802558859 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd927017d0 con 0x55cd929c8ca0 2024-04-02T13:58:30.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.119+0000 7fe11bfff640 1 --2- 172.21.15.52:0/1802558859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929c8ca0 0x55cd929c61f0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fe114003b60 tx=0x7fe11401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210c33af4bfa22c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.119+0000 7fe0faffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:58:30.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.119+0000 7fe0faffd640 1 -- 172.21.15.52:0/1802558859 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe114024070 con 0x55cd929c8ca0 2024-04-02T13:58:30.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.119+0000 7fe0faffd640 1 -- 172.21.15.52:0/1802558859 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe11401f450 con 0x55cd929c8ca0 2024-04-02T13:58:30.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.119+0000 7fe1249a0740 1 -- 172.21.15.52:0/1802558859 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe10c09cad0 con 0x55cd929c8ca0 2024-04-02T13:58:30.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.119+0000 7fe0faffd640 1 -- 172.21.15.52:0/1802558859 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe114033050 con 0x55cd929c8ca0 2024-04-02T13:58:30.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.119+0000 7fe1249a0740 10 client.17028 fetch_fsmap learned FSMap version 30 2024-04-02T13:58:30.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.119+0000 7fe1249a0740 1 -- 172.21.15.52:0/1802558859 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cd929d7a60 con 0x55cd929c8ca0 2024-04-02T13:58:30.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.119+0000 7fe0faffd640 1 -- 172.21.15.52:0/1802558859 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe10c09cad0 con 0x55cd929c8ca0 2024-04-02T13:58:30.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.119+0000 7fe0faffd640 1 -- 172.21.15.52:0/1802558859 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe114028490 con 0x55cd929c8ca0 2024-04-02T13:58:30.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.119+0000 7fe1249a0740 10 client.17028 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:58:30.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.119+0000 7fe1249a0740 10 client.17028 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T13:58:30.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.120+0000 7fe1249a0740 20 client.17028 populate_metadata read hostname 'smithi052' 2024-04-02T13:58:30.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.120+0000 7fe1249a0740 1 --2- 172.21.15.52:0/1802558859 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55cd929d8f70 0x55cd929db380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.120+0000 7fe1249a0740 4 client.17028 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:58:30.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.120+0000 7fe1249a0740 1 -- 172.21.15.52:0/1802558859 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd92930cf0 con 0x55cd929d8f70 2024-04-02T13:58:30.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.120+0000 7fe11b7fe640 1 --2- 172.21.15.52:0/1802558859 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55cd929d8f70 0x55cd929db380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.120+0000 7fe11b7fe640 1 --2- 172.21.15.52:0/1802558859 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55cd929d8f70 0x55cd929db380 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=552c4ba049825bdc server_cookie=525218c34e688e0d in_seq=0 out_seq=0 2024-04-02T13:58:30.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.120+0000 7fe0faffd640 10 client.17028 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:58:30.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.121+0000 7fe0faffd640 1 -- 172.21.15.52:0/1802558859 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cd92930cf0 con 0x55cd929d8f70 2024-04-02T13:58:30.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.121+0000 7fe0faffd640 10 client.17028 handle_command_reply: tid=0 2024-04-02T13:58:30.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.121+0000 7fe1249a0740 1 client.17028 shutdown 2024-04-02T13:58:30.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.121+0000 7fe1249a0740 1 -- 172.21.15.52:0/1802558859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929c8ca0 msgr2=0x55cd929c61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.121+0000 7fe1249a0740 1 --2- 172.21.15.52:0/1802558859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929c8ca0 0x55cd929c61f0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fe114003b60 tx=0x7fe11401d000 comp rx=0 tx=0).stop 2024-04-02T13:58:30.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.121+0000 7fe1249a0740 1 -- 172.21.15.52:0/1802558859 shutdown_connections 2024-04-02T13:58:30.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.121+0000 7fe1249a0740 1 --2- 172.21.15.52:0/1802558859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929c8ca0 0x55cd929c61f0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.121+0000 7fe1249a0740 1 --2- 172.21.15.52:0/1802558859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd929c00c0 0x55cd929c5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.121+0000 7fe1249a0740 1 --2- 172.21.15.52:0/1802558859 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55cd929d8f70 0x55cd929db380 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.121+0000 7fe1249a0740 1 --2- 172.21.15.52:0/1802558859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd929bf7a0 0x55cd929c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.121+0000 7fe0faffd640 0 client.17028 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:58:30.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.121+0000 7fe1249a0740 1 -- 172.21.15.52:0/1802558859 >> 172.21.15.52:0/1802558859 conn(0x55cd929b9ed0 msgr2=0x55cd929bbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.121+0000 7fe1249a0740 1 -- 172.21.15.52:0/1802558859 shutdown_connections 2024-04-02T13:58:30.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.121+0000 7fe1249a0740 1 -- 172.21.15.52:0/1802558859 wait complete. 2024-04-02T13:58:30.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.121+0000 7fe1249a0740 20 client.17028 trim_cache size 0 max 16384 2024-04-02T13:58:30.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.135+0000 7fe1249a0740 1 Processor -- start 2024-04-02T13:58:30.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.135+0000 7fe1249a0740 1 -- start start 2024-04-02T13:58:30.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.135+0000 7fe1249a0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929bf7a0 0x55cd929c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.135+0000 7fe1249a0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd929c00c0 0x55cd929c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.135+0000 7fe1249a0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd929c8ca0 0x55cd929c8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.135+0000 7fe1249a0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c1aee70 con 0x55cd929bf7a0 2024-04-02T13:58:30.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.135+0000 7fe1249a0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c1b09b0 con 0x55cd929c8ca0 2024-04-02T13:58:30.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.135+0000 7fe1249a0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c1af540 con 0x55cd929c00c0 2024-04-02T13:58:30.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.135+0000 7fe11b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929bf7a0 0x55cd929c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.135+0000 7fe11b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929bf7a0 0x55cd929c4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53640/0 (socket says 172.21.15.52:53640) 2024-04-02T13:58:30.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.135+0000 7fe11b7fe640 1 -- 172.21.15.52:0/3596178589 learned_addr learned my addr 172.21.15.52:0/3596178589 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:30.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.135+0000 7fe11affd640 1 --2- 172.21.15.52:0/3596178589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd929c00c0 0x55cd929c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.135+0000 7fe11bfff640 1 --2- 172.21.15.52:0/3596178589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd929c8ca0 0x55cd929c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.135+0000 7fe11b7fe640 1 -- 172.21.15.52:0/3596178589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd929c00c0 msgr2=0x55cd929c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.135+0000 7fe11b7fe640 1 --2- 172.21.15.52:0/3596178589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd929c00c0 0x55cd929c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.135+0000 7fe11b7fe640 1 -- 172.21.15.52:0/3596178589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd929c8ca0 msgr2=0x55cd929c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.135+0000 7fe11b7fe640 1 --2- 172.21.15.52:0/3596178589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd929c8ca0 0x55cd929c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.135+0000 7fe11b7fe640 1 -- 172.21.15.52:0/3596178589 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd929c4000 con 0x55cd929bf7a0 2024-04-02T13:58:30.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.135+0000 7fe11affd640 1 --2- 172.21.15.52:0/3596178589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd929c00c0 0x55cd929c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:58:30.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.135+0000 7fe11bfff640 1 --2- 172.21.15.52:0/3596178589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd929c8ca0 0x55cd929c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:58:30.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.135+0000 7fe11b7fe640 1 --2- 172.21.15.52:0/3596178589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929bf7a0 0x55cd929c4c10 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fe10c0ce7d0 tx=0x7fe10c023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb79bcec1ce55f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.135+0000 7fe1017fa640 1 -- 172.21.15.52:0/3596178589 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c099220 con 0x55cd929bf7a0 2024-04-02T13:58:30.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.136+0000 7fe1017fa640 1 -- 172.21.15.52:0/3596178589 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe10c0ce450 con 0x55cd929bf7a0 2024-04-02T13:58:30.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.136+0000 7fe1017fa640 1 -- 172.21.15.52:0/3596178589 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c018d90 con 0x55cd929bf7a0 2024-04-02T13:58:30.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.136+0000 7fe1249a0740 1 -- 172.21.15.52:0/3596178589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929bf7a0 msgr2=0x55cd929c4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.136+0000 7fe1249a0740 1 --2- 172.21.15.52:0/3596178589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929bf7a0 0x55cd929c4c10 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fe10c0ce7d0 tx=0x7fe10c023040 comp rx=0 tx=0).stop 2024-04-02T13:58:30.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.136+0000 7fe1249a0740 1 -- 172.21.15.52:0/3596178589 shutdown_connections 2024-04-02T13:58:30.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.136+0000 7fe1249a0740 1 --2- 172.21.15.52:0/3596178589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd929c8ca0 0x55cd929c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.136+0000 7fe1249a0740 1 --2- 172.21.15.52:0/3596178589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd929c00c0 0x55cd929c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.136+0000 7fe1249a0740 1 --2- 172.21.15.52:0/3596178589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929bf7a0 0x55cd929c4c10 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.136+0000 7fe1249a0740 1 -- 172.21.15.52:0/3596178589 >> 172.21.15.52:0/3596178589 conn(0x55cd929b9ed0 msgr2=0x55cd929bb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.136+0000 7fe1249a0740 1 -- 172.21.15.52:0/3596178589 shutdown_connections 2024-04-02T13:58:30.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.136+0000 7fe1249a0740 1 -- 172.21.15.52:0/3596178589 wait complete. 2024-04-02T13:58:30.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.136+0000 7fe1249a0740 1 -- start start 2024-04-02T13:58:30.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.137+0000 7fe1249a0740 1 Processor -- start 2024-04-02T13:58:30.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.137+0000 7fe1249a0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd929bf7a0 0x55cd929d1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.137+0000 7fe1249a0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd929c00c0 0x55cd929c2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.137+0000 7fe1249a0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929c8ca0 0x55cd929c2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.137+0000 7fe1249a0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c1b02e0 con 0x55cd929c8ca0 2024-04-02T13:58:30.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.137+0000 7fe1249a0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c1afc10 con 0x55cd929c00c0 2024-04-02T13:58:30.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.137+0000 7fe1249a0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c1ace50 con 0x55cd929bf7a0 2024-04-02T13:58:30.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.137+0000 7fe1249a0740 10 client.0 _ll_register_callbacks cb 0x7fe114009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:58:30.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.137+0000 7fe11bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929c8ca0 0x55cd929c2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.137+0000 7fe11bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929c8ca0 0x55cd929c2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53652/0 (socket says 172.21.15.52:53652) 2024-04-02T13:58:30.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.137+0000 7fe11bfff640 1 -- 172.21.15.52:0/3811032601 learned_addr learned my addr 172.21.15.52:0/3811032601 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:30.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.137+0000 7fe11b7fe640 1 --2- 172.21.15.52:0/3811032601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd929bf7a0 0x55cd929d1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.137+0000 7fe11affd640 1 --2- 172.21.15.52:0/3811032601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd929c00c0 0x55cd929c2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.137+0000 7fe11bfff640 1 -- 172.21.15.52:0/3811032601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd929bf7a0 msgr2=0x55cd929d1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.137+0000 7fe11bfff640 1 --2- 172.21.15.52:0/3811032601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd929bf7a0 0x55cd929d1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.137+0000 7fe11bfff640 1 -- 172.21.15.52:0/3811032601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd929c00c0 msgr2=0x55cd929c2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.137+0000 7fe11bfff640 1 --2- 172.21.15.52:0/3811032601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd929c00c0 0x55cd929c2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.137+0000 7fe11bfff640 1 -- 172.21.15.52:0/3811032601 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd929c31a0 con 0x55cd929c8ca0 2024-04-02T13:58:30.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.137+0000 7fe11affd640 1 --2- 172.21.15.52:0/3811032601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd929c00c0 0x55cd929c2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:58:30.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.137+0000 7fe11bfff640 1 --2- 172.21.15.52:0/3811032601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929c8ca0 0x55cd929c2970 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fe114003fd0 tx=0x7fe114009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2acbcdc229346f46 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.137+0000 7fe0faffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:58:30.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.138+0000 7fe0faffd640 1 -- 172.21.15.52:0/3811032601 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11403b020 con 0x55cd929c8ca0 2024-04-02T13:58:30.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.138+0000 7fe0faffd640 1 -- 172.21.15.52:0/3811032601 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fe114028400 con 0x55cd929c8ca0 2024-04-02T13:58:30.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.138+0000 7fe1249a0740 1 -- 172.21.15.52:0/3811032601 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe10c0ce450 con 0x55cd929c8ca0 2024-04-02T13:58:30.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.138+0000 7fe0faffd640 1 -- 172.21.15.52:0/3811032601 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe114046050 con 0x55cd929c8ca0 2024-04-02T13:58:30.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.138+0000 7fe1249a0740 10 client.17034 fetch_fsmap learned FSMap version 30 2024-04-02T13:58:30.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.138+0000 7fe1249a0740 1 -- 172.21.15.52:0/3811032601 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cd92a3f3d0 con 0x55cd929c8ca0 2024-04-02T13:58:30.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.138+0000 7fe0faffd640 1 -- 172.21.15.52:0/3811032601 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe10c0ce450 con 0x55cd929c8ca0 2024-04-02T13:58:30.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.138+0000 7fe0faffd640 1 -- 172.21.15.52:0/3811032601 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe11401f730 con 0x55cd929c8ca0 2024-04-02T13:58:30.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.138+0000 7fe1249a0740 10 client.17034 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:58:30.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.138+0000 7fe1249a0740 10 client.17034 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T13:58:30.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.138+0000 7fe1249a0740 20 client.17034 populate_metadata read hostname 'smithi052' 2024-04-02T13:58:30.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.138+0000 7fe1249a0740 1 --2- 172.21.15.52:0/3811032601 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55cd92a3fb10 0x55cd92a41f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.138+0000 7fe1249a0740 4 client.17034 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:58:30.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.138+0000 7fe1249a0740 1 -- 172.21.15.52:0/3811032601 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd9292b040 con 0x55cd92a3fb10 2024-04-02T13:58:30.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.138+0000 7fe11b7fe640 1 --2- 172.21.15.52:0/3811032601 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55cd92a3fb10 0x55cd92a41f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.139+0000 7fe0faffd640 10 client.17034 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:58:30.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.139+0000 7fe11b7fe640 1 --2- 172.21.15.52:0/3811032601 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55cd92a3fb10 0x55cd92a41f30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bb59e701a93599f2 server_cookie=f39de47fb7955907 in_seq=0 out_seq=0 2024-04-02T13:58:30.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.139+0000 7fe0faffd640 1 -- 172.21.15.52:0/3811032601 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd9292b040 con 0x55cd92a3fb10 2024-04-02T13:58:30.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.139+0000 7fe0faffd640 10 client.17034 handle_command_reply: tid=0 2024-04-02T13:58:30.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.139+0000 7fe1249a0740 1 client.17034 shutdown 2024-04-02T13:58:30.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.139+0000 7fe1249a0740 1 -- 172.21.15.52:0/3811032601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929c8ca0 msgr2=0x55cd929c2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.139+0000 7fe1249a0740 1 --2- 172.21.15.52:0/3811032601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929c8ca0 0x55cd929c2970 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fe114003fd0 tx=0x7fe114009500 comp rx=0 tx=0).stop 2024-04-02T13:58:30.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.139+0000 7fe1249a0740 1 -- 172.21.15.52:0/3811032601 shutdown_connections 2024-04-02T13:58:30.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.139+0000 7fe1249a0740 1 --2- 172.21.15.52:0/3811032601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd929c8ca0 0x55cd929c2970 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.139+0000 7fe1249a0740 1 --2- 172.21.15.52:0/3811032601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cd929c00c0 0x55cd929c2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.139+0000 7fe1249a0740 1 --2- 172.21.15.52:0/3811032601 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55cd92a3fb10 0x55cd92a41f30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.139+0000 7fe1249a0740 1 --2- 172.21.15.52:0/3811032601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd929bf7a0 0x55cd929d1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.139+0000 7fe1249a0740 1 -- 172.21.15.52:0/3811032601 >> 172.21.15.52:0/3811032601 conn(0x55cd929b9ed0 msgr2=0x55cd929bb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.139+0000 7fe0faffd640 0 client.17034 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:58:30.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.139+0000 7fe1249a0740 1 -- 172.21.15.52:0/3811032601 shutdown_connections 2024-04-02T13:58:30.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.139+0000 7fe1249a0740 1 -- 172.21.15.52:0/3811032601 wait complete. 2024-04-02T13:58:30.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.139+0000 7fe1249a0740 20 client.17034 trim_cache size 0 max 16384 2024-04-02T13:58:30.159 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:58:30.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.140+0000 7fe121cd9640 1 -- 172.21.15.52:0/3158150942 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe1040a3da0 msgr2=0x7fe1040a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.140+0000 7fe121cd9640 1 --2- 172.21.15.52:0/3158150942 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe1040a3da0 0x7fe1040a6260 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe114005fd0 tx=0x7fe114009f00 comp rx=0 tx=0).stop 2024-04-02T13:58:30.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.140+0000 7fe121cd9640 1 -- 172.21.15.52:0/3158150942 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe11c068d00 msgr2=0x7fe11c198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.140+0000 7fe121cd9640 1 --2- 172.21.15.52:0/3158150942 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe11c068d00 0x7fe11c198360 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00e9f0 tx=0x7fe10c00eeb0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.140+0000 7fe121cd9640 1 -- 172.21.15.52:0/3158150942 shutdown_connections 2024-04-02T13:58:30.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.140+0000 7fe121cd9640 1 --2- 172.21.15.52:0/3158150942 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe1040a3da0 0x7fe1040a6260 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.140+0000 7fe121cd9640 1 --2- 172.21.15.52:0/3158150942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe11c107b50 0x7fe11c19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.140+0000 7fe121cd9640 1 --2- 172.21.15.52:0/3158150942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe11c0696c0 0x7fe11c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.140+0000 7fe121cd9640 1 --2- 172.21.15.52:0/3158150942 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe11c068d00 0x7fe11c198360 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.140+0000 7fe121cd9640 1 -- 172.21.15.52:0/3158150942 >> 172.21.15.52:0/3158150942 conn(0x7fe11c077b00 msgr2=0x7fe11c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.140+0000 7fe121cd9640 1 -- 172.21.15.52:0/3158150942 shutdown_connections 2024-04-02T13:58:30.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.140+0000 7fe121cd9640 1 -- 172.21.15.52:0/3158150942 wait complete. 2024-04-02T13:58:30.162 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:58:30.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.264+0000 7f71ea50e640 1 -- 172.21.15.52:0/2413949052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e410e340 msgr2=0x7f71e4110750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.264+0000 7f71ea50e640 1 --2- 172.21.15.52:0/2413949052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e410e340 0x7f71e4110750 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009ca0 tx=0x7f71dc02c180 comp rx=0 tx=0).stop 2024-04-02T13:58:30.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.264+0000 7f71ea50e640 1 -- 172.21.15.52:0/2413949052 shutdown_connections 2024-04-02T13:58:30.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.264+0000 7f71ea50e640 1 --2- 172.21.15.52:0/2413949052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e410e340 0x7f71e4110750 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.264+0000 7f71ea50e640 1 --2- 172.21.15.52:0/2413949052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71e40747f0 0x7f71e410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.264+0000 7f71ea50e640 1 --2- 172.21.15.52:0/2413949052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71e4073ed0 0x7f71e40742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.264+0000 7f71ea50e640 1 -- 172.21.15.52:0/2413949052 >> 172.21.15.52:0/2413949052 conn(0x7f71e40fd530 msgr2=0x7f71e40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.264+0000 7f71ea50e640 1 -- 172.21.15.52:0/2413949052 shutdown_connections 2024-04-02T13:58:30.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.264+0000 7f71ea50e640 1 -- 172.21.15.52:0/2413949052 wait complete. 2024-04-02T13:58:30.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.264+0000 7f71ea50e640 1 Processor -- start 2024-04-02T13:58:30.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.264+0000 7f71ea50e640 1 -- start start 2024-04-02T13:58:30.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.265+0000 7f71ea50e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71e4073ed0 0x7f71e419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.265+0000 7f71ea50e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e40747f0 0x7f71e419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.265+0000 7f71ea50e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71e410e340 0x7f71e419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.265+0000 7f71ea50e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f71e41140a0 con 0x7f71e40747f0 2024-04-02T13:58:30.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.265+0000 7f71ea50e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f71e4113f20 con 0x7f71e410e340 2024-04-02T13:58:30.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.265+0000 7f71ea50e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71e4114220 con 0x7f71e4073ed0 2024-04-02T13:58:30.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.265+0000 7f71e37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e40747f0 0x7f71e419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.265+0000 7f71e37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e40747f0 0x7f71e419aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53664/0 (socket says 172.21.15.52:53664) 2024-04-02T13:58:30.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.265+0000 7f71e37fe640 1 -- 172.21.15.52:0/1476705061 learned_addr learned my addr 172.21.15.52:0/1476705061 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:30.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.265+0000 7f71e8aab640 1 --2- 172.21.15.52:0/1476705061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71e410e340 0x7f71e419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.265+0000 7f71e3fff640 1 --2- 172.21.15.52:0/1476705061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71e4073ed0 0x7f71e419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.265+0000 7f71e37fe640 1 -- 172.21.15.52:0/1476705061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71e4073ed0 msgr2=0x7f71e419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.265+0000 7f71e37fe640 1 --2- 172.21.15.52:0/1476705061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71e4073ed0 0x7f71e419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.265+0000 7f71e37fe640 1 -- 172.21.15.52:0/1476705061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71e410e340 msgr2=0x7f71e419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.265+0000 7f71e37fe640 1 --2- 172.21.15.52:0/1476705061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71e410e340 0x7f71e419edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.265+0000 7f71e37fe640 1 -- 172.21.15.52:0/1476705061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e419f500 con 0x7f71e40747f0 2024-04-02T13:58:30.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.265+0000 7f71e8aab640 1 --2- 172.21.15.52:0/1476705061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71e410e340 0x7f71e419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:58:30.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.265+0000 7f71e37fe640 1 --2- 172.21.15.52:0/1476705061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e40747f0 0x7f71e419aaa0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f71d800c890 tx=0x7f71d800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.266+0000 7f71e17fa640 1 -- 172.21.15.52:0/1476705061 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8004220 con 0x7f71e40747f0 2024-04-02T13:58:30.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.266+0000 7f71e17fa640 1 -- 172.21.15.52:0/1476705061 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71d80043c0 con 0x7f71e40747f0 2024-04-02T13:58:30.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.266+0000 7f71ea50e640 1 -- 172.21.15.52:0/1476705061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e419f7f0 con 0x7f71e40747f0 2024-04-02T13:58:30.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.266+0000 7f71e17fa640 1 -- 172.21.15.52:0/1476705061 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d80049e0 con 0x7f71e40747f0 2024-04-02T13:58:30.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.266+0000 7f71ea50e640 1 -- 172.21.15.52:0/1476705061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e41a7080 con 0x7f71e40747f0 2024-04-02T13:58:30.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.268+0000 7f71e17fa640 1 -- 172.21.15.52:0/1476705061 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f71d80079b0 con 0x7f71e40747f0 2024-04-02T13:58:30.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.268+0000 7f71ed1cd740 1 Processor -- start 2024-04-02T13:58:30.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.268+0000 7f71ed1cd740 1 -- start start 2024-04-02T13:58:30.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.268+0000 7f71e17fa640 1 --2- 172.21.15.52:0/1476705061 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f71cc0a3da0 0x7f71cc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.268+0000 7f71e8aab640 1 --2- 172.21.15.52:0/1476705061 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f71cc0a3da0 0x7f71cc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.268+0000 7f71ed1cd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccc7a0 0x55aeaccccb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.268+0000 7f71e17fa640 1 -- 172.21.15.52:0/1476705061 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f71d80d0b40 con 0x7f71e40747f0 2024-04-02T13:58:30.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.268+0000 7f71ed1cd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aeacccd0c0 0x55aeaccd5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.268+0000 7f71ed1cd740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aeaccd5ca0 0x55aeaccd8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.268+0000 7f71ed1cd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55aeacc307b0 con 0x55aeacccc7a0 2024-04-02T13:58:30.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.268+0000 7f71ed1cd740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55aeacbcf570 con 0x55aeaccd5ca0 2024-04-02T13:58:30.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.268+0000 7f71ed1cd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55aeacc2c760 con 0x55aeacccd0c0 2024-04-02T13:58:30.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.269+0000 7f71e3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccc7a0 0x55aeaccccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.269+0000 7f71e3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccc7a0 0x55aeaccccb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53668/0 (socket says 172.21.15.52:53668) 2024-04-02T13:58:30.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.269+0000 7f71e3fff640 1 -- 172.21.15.52:0/3683648609 learned_addr learned my addr 172.21.15.52:0/3683648609 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:30.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.269+0000 7f71e37fe640 1 --2- 172.21.15.52:0/3683648609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aeacccd0c0 0x55aeaccd5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.269+0000 7f71e3fff640 1 -- 172.21.15.52:0/3683648609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aeacccd0c0 msgr2=0x55aeaccd5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.269+0000 7f71e3fff640 1 --2- 172.21.15.52:0/3683648609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aeacccd0c0 0x55aeaccd5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.269+0000 7f71e3fff640 1 -- 172.21.15.52:0/3683648609 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aeaccd5ca0 msgr2=0x55aeaccd8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.269+0000 7f71e8aab640 1 --2- 172.21.15.52:0/3683648609 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aeaccd5ca0 0x55aeaccd8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.269+0000 7f71e3fff640 1 --2- 172.21.15.52:0/3683648609 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aeaccd5ca0 0x55aeaccd8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.269+0000 7f71e3fff640 1 -- 172.21.15.52:0/3683648609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aeac9443f0 con 0x55aeacccc7a0 2024-04-02T13:58:30.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.269+0000 7f71e8aab640 1 --2- 172.21.15.52:0/3683648609 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aeaccd5ca0 0x55aeaccd8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:58:30.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.269+0000 7f71e8aab640 1 --2- 172.21.15.52:0/1476705061 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f71cc0a3da0 0x7f71cc0a6260 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f71dc02c790 tx=0x7f71dc032520 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.269+0000 7f71e3fff640 1 --2- 172.21.15.52:0/3683648609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccc7a0 0x55aeaccccb80 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f71d400cc60 tx=0x7f71d400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df263fed13c8788a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.269+0000 7f71ca7fc640 1 -- 172.21.15.52:0/3683648609 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4014070 con 0x55aeacccc7a0 2024-04-02T13:58:30.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.269+0000 7f71ca7fc640 1 -- 172.21.15.52:0/3683648609 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71d40047a0 con 0x55aeacccc7a0 2024-04-02T13:58:30.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.269+0000 7f71ca7fc640 1 -- 172.21.15.52:0/3683648609 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4010730 con 0x55aeacccc7a0 2024-04-02T13:58:30.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.270+0000 7f71ed1cd740 1 -- 172.21.15.52:0/3683648609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccc7a0 msgr2=0x55aeaccccb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.270+0000 7f71ed1cd740 1 --2- 172.21.15.52:0/3683648609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccc7a0 0x55aeaccccb80 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f71d400cc60 tx=0x7f71d400e720 comp rx=0 tx=0).stop 2024-04-02T13:58:30.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.270+0000 7f71ed1cd740 1 -- 172.21.15.52:0/3683648609 shutdown_connections 2024-04-02T13:58:30.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.270+0000 7f71ed1cd740 1 --2- 172.21.15.52:0/3683648609 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aeaccd5ca0 0x55aeaccd8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.270+0000 7f71ed1cd740 1 --2- 172.21.15.52:0/3683648609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aeacccd0c0 0x55aeaccd5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.270+0000 7f71ed1cd740 1 --2- 172.21.15.52:0/3683648609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccc7a0 0x55aeaccccb80 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.270+0000 7f71ed1cd740 1 -- 172.21.15.52:0/3683648609 >> 172.21.15.52:0/3683648609 conn(0x55aeaccc6ed0 msgr2=0x55aeaccc72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.270+0000 7f71ed1cd740 1 -- 172.21.15.52:0/3683648609 shutdown_connections 2024-04-02T13:58:30.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.270+0000 7f71ed1cd740 1 -- 172.21.15.52:0/3683648609 wait complete. 2024-04-02T13:58:30.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.270+0000 7f71ed1cd740 1 -- start start 2024-04-02T13:58:30.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.270+0000 7f71ed1cd740 1 Processor -- start 2024-04-02T13:58:30.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.271+0000 7f71ed1cd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccc7a0 0x55aeaccd5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.271+0000 7f71ed1cd740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aeacccd0c0 0x55aeaccd29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.271+0000 7f71ed1cd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aeaccd5ca0 0x55aeaccd2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.271+0000 7f71ed1cd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55aeaccd8a90 con 0x55aeacccc7a0 2024-04-02T13:58:30.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.271+0000 7f71ed1cd740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55aeacbebe80 con 0x55aeacccd0c0 2024-04-02T13:58:30.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.271+0000 7f71ed1cd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55aeaccd8c10 con 0x55aeaccd5ca0 2024-04-02T13:58:30.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.271+0000 7f71ed1cd740 10 client.0 _ll_register_callbacks cb 0x55aeacc27aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:58:30.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.271+0000 7f71e3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccc7a0 0x55aeaccd5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.271+0000 7f71e3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccc7a0 0x55aeaccd5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53684/0 (socket says 172.21.15.52:53684) 2024-04-02T13:58:30.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.271+0000 7f71e3fff640 1 -- 172.21.15.52:0/3174531587 learned_addr learned my addr 172.21.15.52:0/3174531587 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:30.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.271+0000 7f71e8aab640 1 --2- 172.21.15.52:0/3174531587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aeaccd5ca0 0x55aeaccd2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.271+0000 7f71e37fe640 1 --2- 172.21.15.52:0/3174531587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aeacccd0c0 0x55aeaccd29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.271+0000 7f71e3fff640 1 -- 172.21.15.52:0/3174531587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aeaccd5ca0 msgr2=0x55aeaccd2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.271+0000 7f71e3fff640 1 --2- 172.21.15.52:0/3174531587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aeaccd5ca0 0x55aeaccd2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.271+0000 7f71e3fff640 1 -- 172.21.15.52:0/3174531587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aeacccd0c0 msgr2=0x55aeaccd29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.271+0000 7f71e3fff640 1 --2- 172.21.15.52:0/3174531587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aeacccd0c0 0x55aeaccd29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.271+0000 7f71e3fff640 1 -- 172.21.15.52:0/3174531587 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aeaca0e7d0 con 0x55aeacccc7a0 2024-04-02T13:58:30.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.271+0000 7f71e8aab640 1 --2- 172.21.15.52:0/3174531587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aeaccd5ca0 0x55aeaccd2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:58:30.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.271+0000 7f71e37fe640 1 --2- 172.21.15.52:0/3174531587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aeacccd0c0 0x55aeaccd29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:58:30.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.271+0000 7f71e3fff640 1 --2- 172.21.15.52:0/3174531587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccc7a0 0x55aeaccd5630 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f71d400cc30 tx=0x7f71d4005210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ee85df8377e295 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.271+0000 7f71c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:58:30.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.271+0000 7f71c37fe640 1 -- 172.21.15.52:0/3174531587 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4027060 con 0x55aeacccc7a0 2024-04-02T13:58:30.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.272+0000 7f71c37fe640 1 -- 172.21.15.52:0/3174531587 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f71d4010510 con 0x55aeacccc7a0 2024-04-02T13:58:30.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.272+0000 7f71ed1cd740 1 -- 172.21.15.52:0/3174531587 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71d40047a0 con 0x55aeacccc7a0 2024-04-02T13:58:30.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.272+0000 7f71c37fe640 1 -- 172.21.15.52:0/3174531587 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f71d402e050 con 0x55aeacccc7a0 2024-04-02T13:58:30.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.272+0000 7f71ed1cd740 10 client.17046 fetch_fsmap learned FSMap version 30 2024-04-02T13:58:30.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.272+0000 7f71ed1cd740 1 -- 172.21.15.52:0/3174531587 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55aeacce4990 con 0x55aeacccc7a0 2024-04-02T13:58:30.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.272+0000 7f71c37fe640 1 -- 172.21.15.52:0/3174531587 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71d40047a0 con 0x55aeacccc7a0 2024-04-02T13:58:30.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.272+0000 7f71c37fe640 1 -- 172.21.15.52:0/3174531587 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f71d40184a0 con 0x55aeacccc7a0 2024-04-02T13:58:30.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.272+0000 7f71ed1cd740 10 client.17046 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:58:30.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.272+0000 7f71ed1cd740 10 client.17046 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T13:58:30.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.272+0000 7f71ed1cd740 20 client.17046 populate_metadata read hostname 'smithi052' 2024-04-02T13:58:30.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.272+0000 7f71ed1cd740 1 --2- 172.21.15.52:0/3174531587 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55aeacce5df0 0x55aeacce81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.272+0000 7f71ed1cd740 4 client.17046 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:58:30.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.272+0000 7f71ed1cd740 1 -- 172.21.15.52:0/3174531587 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aeacc3dcf0 con 0x55aeacce5df0 2024-04-02T13:58:30.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.272+0000 7f71e37fe640 1 --2- 172.21.15.52:0/3174531587 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55aeacce5df0 0x55aeacce81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.273+0000 7f71e37fe640 1 --2- 172.21.15.52:0/3174531587 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55aeacce5df0 0x55aeacce81e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a1419b41af6c1808 server_cookie=248b79056f3f74e0 in_seq=0 out_seq=0 2024-04-02T13:58:30.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.273+0000 7f71c37fe640 10 client.17046 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:58:30.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.273+0000 7f71c37fe640 1 -- 172.21.15.52:0/3174531587 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aeacc3dcf0 con 0x55aeacce5df0 2024-04-02T13:58:30.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.273+0000 7f71c37fe640 10 client.17046 handle_command_reply: tid=0 2024-04-02T13:58:30.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.273+0000 7f71ed1cd740 1 client.17046 shutdown 2024-04-02T13:58:30.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.274+0000 7f71ed1cd740 1 -- 172.21.15.52:0/3174531587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccc7a0 msgr2=0x55aeaccd5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.274+0000 7f71ed1cd740 1 --2- 172.21.15.52:0/3174531587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccc7a0 0x55aeaccd5630 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f71d400cc30 tx=0x7f71d4005210 comp rx=0 tx=0).stop 2024-04-02T13:58:30.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.274+0000 7f71ed1cd740 1 -- 172.21.15.52:0/3174531587 shutdown_connections 2024-04-02T13:58:30.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.274+0000 7f71ed1cd740 1 --2- 172.21.15.52:0/3174531587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aeaccd5ca0 0x55aeaccd2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.274+0000 7f71ed1cd740 1 --2- 172.21.15.52:0/3174531587 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55aeacce5df0 0x55aeacce81e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.274+0000 7f71ed1cd740 1 --2- 172.21.15.52:0/3174531587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aeacccd0c0 0x55aeaccd29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.274+0000 7f71ed1cd740 1 --2- 172.21.15.52:0/3174531587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccc7a0 0x55aeaccd5630 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.274+0000 7f71c37fe640 0 client.17046 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:58:30.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.274+0000 7f71ed1cd740 1 -- 172.21.15.52:0/3174531587 >> 172.21.15.52:0/3174531587 conn(0x55aeaccc6ed0 msgr2=0x55aeaccc8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.274+0000 7f71ed1cd740 1 -- 172.21.15.52:0/3174531587 shutdown_connections 2024-04-02T13:58:30.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.274+0000 7f71ed1cd740 1 -- 172.21.15.52:0/3174531587 wait complete. 2024-04-02T13:58:30.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.274+0000 7f71ed1cd740 20 client.17046 trim_cache size 0 max 16384 2024-04-02T13:58:30.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.287+0000 7f71ed1cd740 1 Processor -- start 2024-04-02T13:58:30.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.287+0000 7f71ed1cd740 1 -- start start 2024-04-02T13:58:30.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.287+0000 7f71ed1cd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccc7a0 0x55aeacce1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.287+0000 7f71ed1cd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aeacccd0c0 0x55aeaccd19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.287+0000 7f71ed1cd740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aeaccd5ca0 0x55aeaccd50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.287+0000 7f71ed1cd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f71e41b1030 con 0x55aeacccc7a0 2024-04-02T13:58:30.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.287+0000 7f71ed1cd740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f71e41b2b70 con 0x55aeaccd5ca0 2024-04-02T13:58:30.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.287+0000 7f71ed1cd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71e41b1700 con 0x55aeacccd0c0 2024-04-02T13:58:30.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.287+0000 7f71e3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccc7a0 0x55aeacce1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.287+0000 7f71e3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccc7a0 0x55aeacce1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53686/0 (socket says 172.21.15.52:53686) 2024-04-02T13:58:30.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.287+0000 7f71e3fff640 1 -- 172.21.15.52:0/2318881237 learned_addr learned my addr 172.21.15.52:0/2318881237 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:30.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.288+0000 7f71e3fff640 1 -- 172.21.15.52:0/2318881237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aeacccd0c0 msgr2=0x55aeaccd19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.288+0000 7f71e3fff640 1 --2- 172.21.15.52:0/2318881237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aeacccd0c0 0x55aeaccd19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.288+0000 7f71e3fff640 1 -- 172.21.15.52:0/2318881237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aeaccd5ca0 msgr2=0x55aeaccd50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.288+0000 7f71e3fff640 1 --2- 172.21.15.52:0/2318881237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aeaccd5ca0 0x55aeaccd50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.288+0000 7f71e3fff640 1 -- 172.21.15.52:0/2318881237 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aeaccd32b0 con 0x55aeacccc7a0 2024-04-02T13:58:30.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.288+0000 7f71e3fff640 1 --2- 172.21.15.52:0/2318881237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccc7a0 0x55aeacce1950 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f71d400cb80 tx=0x7f71d400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3d1ed644757dd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.288+0000 7f71c9ffb640 1 -- 172.21.15.52:0/2318881237 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4027070 con 0x55aeacccc7a0 2024-04-02T13:58:30.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.288+0000 7f71c9ffb640 1 -- 172.21.15.52:0/2318881237 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71d4004510 con 0x55aeacccc7a0 2024-04-02T13:58:30.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.288+0000 7f71c9ffb640 1 -- 172.21.15.52:0/2318881237 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4014070 con 0x55aeacccc7a0 2024-04-02T13:58:30.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.288+0000 7f71ed1cd740 1 -- 172.21.15.52:0/2318881237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccc7a0 msgr2=0x55aeacce1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.288+0000 7f71ed1cd740 1 --2- 172.21.15.52:0/2318881237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccc7a0 0x55aeacce1950 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f71d400cb80 tx=0x7f71d400ed50 comp rx=0 tx=0).stop 2024-04-02T13:58:30.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.288+0000 7f71ed1cd740 1 -- 172.21.15.52:0/2318881237 shutdown_connections 2024-04-02T13:58:30.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.288+0000 7f71ed1cd740 1 --2- 172.21.15.52:0/2318881237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aeaccd5ca0 0x55aeaccd50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.288+0000 7f71ed1cd740 1 --2- 172.21.15.52:0/2318881237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aeacccd0c0 0x55aeaccd19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.288+0000 7f71ed1cd740 1 --2- 172.21.15.52:0/2318881237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccc7a0 0x55aeacce1950 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.288+0000 7f71ed1cd740 1 -- 172.21.15.52:0/2318881237 >> 172.21.15.52:0/2318881237 conn(0x55aeaccc6ed0 msgr2=0x55aeaccd12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.288+0000 7f71ed1cd740 1 -- 172.21.15.52:0/2318881237 shutdown_connections 2024-04-02T13:58:30.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.288+0000 7f71ed1cd740 1 -- 172.21.15.52:0/2318881237 wait complete. 2024-04-02T13:58:30.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.289+0000 7f71ed1cd740 1 -- start start 2024-04-02T13:58:30.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.289+0000 7f71ed1cd740 1 Processor -- start 2024-04-02T13:58:30.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.289+0000 7f71ed1cd740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aeacccc7a0 0x55aeacccf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.289+0000 7f71ed1cd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccd0c0 0x55aeacccf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.289+0000 7f71ed1cd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aeaccd5ca0 0x55aeaccdce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.289+0000 7f71ed1cd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f71e41b24a0 con 0x55aeacccd0c0 2024-04-02T13:58:30.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.289+0000 7f71ed1cd740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f71e41b1dd0 con 0x55aeacccc7a0 2024-04-02T13:58:30.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.289+0000 7f71ed1cd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71e41af010 con 0x55aeaccd5ca0 2024-04-02T13:58:30.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.289+0000 7f71ed1cd740 10 client.0 _ll_register_callbacks cb 0x7f71d400cab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:58:30.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.289+0000 7f71e37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccd0c0 0x55aeacccf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.289+0000 7f71e37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccd0c0 0x55aeacccf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53700/0 (socket says 172.21.15.52:53700) 2024-04-02T13:58:30.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.289+0000 7f71e37fe640 1 -- 172.21.15.52:0/2037073450 learned_addr learned my addr 172.21.15.52:0/2037073450 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:30.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.289+0000 7f71e3fff640 1 --2- 172.21.15.52:0/2037073450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aeacccc7a0 0x55aeacccf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.290+0000 7f71e37fe640 1 -- 172.21.15.52:0/2037073450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aeaccd5ca0 msgr2=0x55aeaccdce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.290+0000 7f71e8aab640 1 --2- 172.21.15.52:0/2037073450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aeaccd5ca0 0x55aeaccdce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.290+0000 7f71e37fe640 1 --2- 172.21.15.52:0/2037073450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aeaccd5ca0 0x55aeaccdce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.290+0000 7f71e37fe640 1 -- 172.21.15.52:0/2037073450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aeacccc7a0 msgr2=0x55aeacccf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.290+0000 7f71e37fe640 1 --2- 172.21.15.52:0/2037073450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aeacccc7a0 0x55aeacccf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.290+0000 7f71e37fe640 1 -- 172.21.15.52:0/2037073450 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aeaccdd3a0 con 0x55aeacccd0c0 2024-04-02T13:58:30.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.290+0000 7f71e8aab640 1 --2- 172.21.15.52:0/2037073450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aeaccd5ca0 0x55aeaccdce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:58:30.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.290+0000 7f71e37fe640 1 --2- 172.21.15.52:0/2037073450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccd0c0 0x55aeacccf900 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f71d8005fd0 tx=0x7f71d8004720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6a79f2df305f0e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.290+0000 7f71c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:58:30.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.290+0000 7f71c37fe640 1 -- 172.21.15.52:0/2037073450 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d801d040 con 0x55aeacccd0c0 2024-04-02T13:58:30.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.290+0000 7f71c37fe640 1 -- 172.21.15.52:0/2037073450 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f71d80ce1d0 con 0x55aeacccd0c0 2024-04-02T13:58:30.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.290+0000 7f71ed1cd740 1 -- 172.21.15.52:0/2037073450 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71d4004510 con 0x55aeacccd0c0 2024-04-02T13:58:30.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.290+0000 7f71c37fe640 1 -- 172.21.15.52:0/2037073450 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f71d80a0b50 con 0x55aeacccd0c0 2024-04-02T13:58:30.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.290+0000 7f71ed1cd740 10 client.17052 fetch_fsmap learned FSMap version 30 2024-04-02T13:58:30.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.290+0000 7f71ed1cd740 1 -- 172.21.15.52:0/2037073450 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55aeaccde700 con 0x55aeacccd0c0 2024-04-02T13:58:30.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.290+0000 7f71c37fe640 1 -- 172.21.15.52:0/2037073450 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71d4004510 con 0x55aeacccd0c0 2024-04-02T13:58:30.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.290+0000 7f71c37fe640 1 -- 172.21.15.52:0/2037073450 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f71d8085ce0 con 0x55aeacccd0c0 2024-04-02T13:58:30.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.290+0000 7f71ed1cd740 10 client.17052 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:58:30.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.290+0000 7f71ed1cd740 10 client.17052 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T13:58:30.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.290+0000 7f71ed1cd740 20 client.17052 populate_metadata read hostname 'smithi052' 2024-04-02T13:58:30.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.290+0000 7f71ed1cd740 1 --2- 172.21.15.52:0/2037073450 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55aeacd4c4d0 0x55aeacd4e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.291+0000 7f71ed1cd740 4 client.17052 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:58:30.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.291+0000 7f71ed1cd740 1 -- 172.21.15.52:0/2037073450 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aeacc38040 con 0x55aeacd4c4d0 2024-04-02T13:58:30.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.291+0000 7f71e3fff640 1 --2- 172.21.15.52:0/2037073450 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55aeacd4c4d0 0x55aeacd4e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.291+0000 7f71e3fff640 1 --2- 172.21.15.52:0/2037073450 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55aeacd4c4d0 0x55aeacd4e8f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=272c9d55a768e6b2 server_cookie=7ecc80044ca1191c in_seq=0 out_seq=0 2024-04-02T13:58:30.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.291+0000 7f71c37fe640 10 client.17052 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:58:30.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.291+0000 7f71c37fe640 1 -- 172.21.15.52:0/2037073450 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aeacc38040 con 0x55aeacd4c4d0 2024-04-02T13:58:30.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.291+0000 7f71c37fe640 10 client.17052 handle_command_reply: tid=0 2024-04-02T13:58:30.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.291+0000 7f71ed1cd740 1 client.17052 shutdown 2024-04-02T13:58:30.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.292+0000 7f71ed1cd740 1 -- 172.21.15.52:0/2037073450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccd0c0 msgr2=0x55aeacccf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.292+0000 7f71ed1cd740 1 --2- 172.21.15.52:0/2037073450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccd0c0 0x55aeacccf900 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f71d8005fd0 tx=0x7f71d8004720 comp rx=0 tx=0).stop 2024-04-02T13:58:30.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.292+0000 7f71ed1cd740 1 -- 172.21.15.52:0/2037073450 shutdown_connections 2024-04-02T13:58:30.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.292+0000 7f71ed1cd740 1 --2- 172.21.15.52:0/2037073450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aeaccd5ca0 0x55aeaccdce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.292+0000 7f71ed1cd740 1 --2- 172.21.15.52:0/2037073450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aeacccd0c0 0x55aeacccf900 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.292+0000 7f71ed1cd740 1 --2- 172.21.15.52:0/2037073450 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55aeacd4c4d0 0x55aeacd4e8f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.292+0000 7f71ed1cd740 1 --2- 172.21.15.52:0/2037073450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55aeacccc7a0 0x55aeacccf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.292+0000 7f71c37fe640 0 client.17052 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:58:30.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.292+0000 7f71ed1cd740 1 -- 172.21.15.52:0/2037073450 >> 172.21.15.52:0/2037073450 conn(0x55aeaccc6ed0 msgr2=0x55aeaccc5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.292+0000 7f71ed1cd740 1 -- 172.21.15.52:0/2037073450 shutdown_connections 2024-04-02T13:58:30.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.292+0000 7f71ed1cd740 1 -- 172.21.15.52:0/2037073450 wait complete. 2024-04-02T13:58:30.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.292+0000 7f71ed1cd740 20 client.17052 trim_cache size 0 max 16384 2024-04-02T13:58:30.310 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:58:30.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.292+0000 7f71ea50e640 1 -- 172.21.15.52:0/1476705061 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f71cc0a3da0 msgr2=0x7f71cc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.292+0000 7f71ea50e640 1 --2- 172.21.15.52:0/1476705061 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f71cc0a3da0 0x7f71cc0a6260 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f71dc02c790 tx=0x7f71dc032520 comp rx=0 tx=0).stop 2024-04-02T13:58:30.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.292+0000 7f71ea50e640 1 -- 172.21.15.52:0/1476705061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e40747f0 msgr2=0x7f71e419aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.292+0000 7f71ea50e640 1 --2- 172.21.15.52:0/1476705061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e40747f0 0x7f71e419aaa0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f71d800c890 tx=0x7f71d800cd50 comp rx=0 tx=0).stop 2024-04-02T13:58:30.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.292+0000 7f71ea50e640 1 -- 172.21.15.52:0/1476705061 shutdown_connections 2024-04-02T13:58:30.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.292+0000 7f71ea50e640 1 --2- 172.21.15.52:0/1476705061 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f71cc0a3da0 0x7f71cc0a6260 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.292+0000 7f71ea50e640 1 --2- 172.21.15.52:0/1476705061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71e410e340 0x7f71e419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.292+0000 7f71ea50e640 1 --2- 172.21.15.52:0/1476705061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71e40747f0 0x7f71e419aaa0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.292+0000 7f71ea50e640 1 --2- 172.21.15.52:0/1476705061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71e4073ed0 0x7f71e419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.292+0000 7f71ea50e640 1 -- 172.21.15.52:0/1476705061 >> 172.21.15.52:0/1476705061 conn(0x7f71e40fd530 msgr2=0x7f71e40fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.292+0000 7f71ea50e640 1 -- 172.21.15.52:0/1476705061 shutdown_connections 2024-04-02T13:58:30.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.292+0000 7f71ea50e640 1 -- 172.21.15.52:0/1476705061 wait complete. 2024-04-02T13:58:30.313 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:30 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3556182963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:58:30.313 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:58:30.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.414+0000 7fcf808b7640 1 -- 172.21.15.52:0/1421586248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf7c101330 msgr2=0x7fcf7c10d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.414+0000 7fcf808b7640 1 --2- 172.21.15.52:0/1421586248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf7c101330 0x7fcf7c10d840 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009cd0 tx=0x7fcf6c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.415+0000 7fcf808b7640 1 -- 172.21.15.52:0/1421586248 shutdown_connections 2024-04-02T13:58:30.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.415+0000 7fcf808b7640 1 --2- 172.21.15.52:0/1421586248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf7c101330 0x7fcf7c10d840 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.415+0000 7fcf808b7640 1 --2- 172.21.15.52:0/1421586248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf7c100970 0x7fcf7c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.415+0000 7fcf808b7640 1 --2- 172.21.15.52:0/1421586248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf7c106980 0x7fcf7c106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.415+0000 7fcf808b7640 1 -- 172.21.15.52:0/1421586248 >> 172.21.15.52:0/1421586248 conn(0x7fcf7c077b00 msgr2=0x7fcf7c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.415+0000 7fcf808b7640 1 -- 172.21.15.52:0/1421586248 shutdown_connections 2024-04-02T13:58:30.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.415+0000 7fcf808b7640 1 -- 172.21.15.52:0/1421586248 wait complete. 2024-04-02T13:58:30.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.415+0000 7fcf808b7640 1 Processor -- start 2024-04-02T13:58:30.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.415+0000 7fcf808b7640 1 -- start start 2024-04-02T13:58:30.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.416+0000 7fcf808b7640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf7c100970 0x7fcf7c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.416+0000 7fcf808b7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf7c101330 0x7fcf7c198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.416+0000 7fcf808b7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf7c106980 0x7fcf7c19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.416+0000 7fcf808b7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c111ea0 con 0x7fcf7c106980 2024-04-02T13:58:30.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.416+0000 7fcf808b7640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c111d20 con 0x7fcf7c100970 2024-04-02T13:58:30.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.416+0000 7fcf808b7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c112020 con 0x7fcf7c101330 2024-04-02T13:58:30.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.416+0000 7fcf7ad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf7c106980 0x7fcf7c19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.416+0000 7fcf7ad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf7c106980 0x7fcf7c19cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53716/0 (socket says 172.21.15.52:53716) 2024-04-02T13:58:30.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.416+0000 7fcf7ad9d640 1 -- 172.21.15.52:0/268863286 learned_addr learned my addr 172.21.15.52:0/268863286 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:30.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.416+0000 7fcf7a59c640 1 --2- 172.21.15.52:0/268863286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf7c100970 0x7fcf7c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.416+0000 7fcf79d9b640 1 --2- 172.21.15.52:0/268863286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf7c101330 0x7fcf7c198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.416+0000 7fcf7ad9d640 1 -- 172.21.15.52:0/268863286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf7c101330 msgr2=0x7fcf7c198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.416+0000 7fcf7ad9d640 1 --2- 172.21.15.52:0/268863286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf7c101330 0x7fcf7c198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.416+0000 7fcf7ad9d640 1 -- 172.21.15.52:0/268863286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf7c100970 msgr2=0x7fcf7c198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.416+0000 7fcf7ad9d640 1 --2- 172.21.15.52:0/268863286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf7c100970 0x7fcf7c198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.416+0000 7fcf7ad9d640 1 -- 172.21.15.52:0/268863286 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7c19d2a0 con 0x7fcf7c106980 2024-04-02T13:58:30.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.416+0000 7fcf7ad9d640 1 --2- 172.21.15.52:0/268863286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf7c106980 0x7fcf7c19cb70 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0348f0 tx=0x7fcf6c032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.416+0000 7fcf737fe640 1 -- 172.21.15.52:0/268863286 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c041020 con 0x7fcf7c106980 2024-04-02T13:58:30.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.416+0000 7fcf737fe640 1 -- 172.21.15.52:0/268863286 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf6c034cd0 con 0x7fcf7c106980 2024-04-02T13:58:30.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.416+0000 7fcf808b7640 1 -- 172.21.15.52:0/268863286 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf7c19d530 con 0x7fcf7c106980 2024-04-02T13:58:30.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.416+0000 7fcf737fe640 1 -- 172.21.15.52:0/268863286 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c039040 con 0x7fcf7c106980 2024-04-02T13:58:30.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.416+0000 7fcf808b7640 1 -- 172.21.15.52:0/268863286 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf7c1a4e20 con 0x7fcf7c106980 2024-04-02T13:58:30.426 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:30 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3556182963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:58:30.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.418+0000 7fcf737fe640 1 -- 172.21.15.52:0/268863286 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcf6c044020 con 0x7fcf7c106980 2024-04-02T13:58:30.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.419+0000 7fcf8357a740 1 Processor -- start 2024-04-02T13:58:30.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.419+0000 7fcf8357a740 1 -- start start 2024-04-02T13:58:30.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.419+0000 7fcf737fe640 1 --2- 172.21.15.52:0/268863286 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fcf680a3da0 0x7fcf680a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.419+0000 7fcf8357a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4139fa450 0x55b4139fa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.419+0000 7fcf8357a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4139fad70 0x55b413a07b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.419+0000 7fcf8357a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b413a080b0 0x55b413a0a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.419+0000 7fcf8357a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b41395e7b0 con 0x55b4139fa450 2024-04-02T13:58:30.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.419+0000 7fcf8357a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b4138fd570 con 0x55b4139fad70 2024-04-02T13:58:30.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.419+0000 7fcf8357a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b41395a760 con 0x55b413a080b0 2024-04-02T13:58:30.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.419+0000 7fcf79d9b640 1 --2- 172.21.15.52:0/268863286 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fcf680a3da0 0x7fcf680a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.419+0000 7fcf737fe640 1 -- 172.21.15.52:0/268863286 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fcf6c0f2970 con 0x7fcf7c106980 2024-04-02T13:58:30.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.419+0000 7fcf7a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4139fa450 0x55b4139fa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.419+0000 7fcf7a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4139fa450 0x55b4139fa830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53726/0 (socket says 172.21.15.52:53726) 2024-04-02T13:58:30.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.419+0000 7fcf7a59c640 1 -- 172.21.15.52:0/2718609491 learned_addr learned my addr 172.21.15.52:0/2718609491 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:30.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.419+0000 7fcf79d9b640 1 --2- 172.21.15.52:0/2718609491 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4139fad70 0x55b413a07b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.420+0000 7fcf7ad9d640 1 --2- 172.21.15.52:0/2718609491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b413a080b0 0x55b413a0a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.420+0000 7fcf7a59c640 1 -- 172.21.15.52:0/2718609491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b413a080b0 msgr2=0x55b413a0a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.420+0000 7fcf7a59c640 1 --2- 172.21.15.52:0/2718609491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b413a080b0 0x55b413a0a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.420+0000 7fcf7a59c640 1 -- 172.21.15.52:0/2718609491 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4139fad70 msgr2=0x55b413a07b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.420+0000 7fcf7a59c640 1 --2- 172.21.15.52:0/2718609491 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4139fad70 0x55b413a07b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.420+0000 7fcf7a59c640 1 -- 172.21.15.52:0/2718609491 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4136723f0 con 0x55b4139fa450 2024-04-02T13:58:30.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.420+0000 7fcf79d9b640 1 --2- 172.21.15.52:0/268863286 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fcf680a3da0 0x7fcf680a6260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c104f80 tx=0x7fcf64014840 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.420+0000 7fcf7a59c640 1 --2- 172.21.15.52:0/2718609491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4139fa450 0x55b4139fa830 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400dc30 tx=0x7fcf7400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e088c2c2e40cf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.420+0000 7fcf70ff9640 1 -- 172.21.15.52:0/2718609491 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74011440 con 0x55b4139fa450 2024-04-02T13:58:30.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.420+0000 7fcf70ff9640 1 -- 172.21.15.52:0/2718609491 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf74017070 con 0x55b4139fa450 2024-04-02T13:58:30.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.420+0000 7fcf70ff9640 1 -- 172.21.15.52:0/2718609491 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74010930 con 0x55b4139fa450 2024-04-02T13:58:30.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.421+0000 7fcf8357a740 1 -- 172.21.15.52:0/2718609491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4139fa450 msgr2=0x55b4139fa830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.421+0000 7fcf8357a740 1 --2- 172.21.15.52:0/2718609491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4139fa450 0x55b4139fa830 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400dc30 tx=0x7fcf7400f730 comp rx=0 tx=0).stop 2024-04-02T13:58:30.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.421+0000 7fcf8357a740 1 -- 172.21.15.52:0/2718609491 shutdown_connections 2024-04-02T13:58:30.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.421+0000 7fcf8357a740 1 --2- 172.21.15.52:0/2718609491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b413a080b0 0x55b413a0a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.421+0000 7fcf8357a740 1 --2- 172.21.15.52:0/2718609491 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4139fad70 0x55b413a07b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.421+0000 7fcf8357a740 1 --2- 172.21.15.52:0/2718609491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4139fa450 0x55b4139fa830 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.421+0000 7fcf8357a740 1 -- 172.21.15.52:0/2718609491 >> 172.21.15.52:0/2718609491 conn(0x55b4139f4ed0 msgr2=0x55b4139f52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.421+0000 7fcf8357a740 1 -- 172.21.15.52:0/2718609491 shutdown_connections 2024-04-02T13:58:30.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.421+0000 7fcf8357a740 1 -- 172.21.15.52:0/2718609491 wait complete. 2024-04-02T13:58:30.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.421+0000 7fcf8357a740 1 -- start start 2024-04-02T13:58:30.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.421+0000 7fcf8357a740 1 Processor -- start 2024-04-02T13:58:30.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.421+0000 7fcf8357a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4139fa450 0x55b413a07aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.421+0000 7fcf8357a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4139fad70 0x55b413a04e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.421+0000 7fcf8357a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b413a080b0 0x55b413a05440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.421+0000 7fcf8357a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b413a0aef0 con 0x55b4139fa450 2024-04-02T13:58:30.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.421+0000 7fcf8357a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b413919e80 con 0x55b4139fad70 2024-04-02T13:58:30.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.421+0000 7fcf8357a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b413a0b070 con 0x55b413a080b0 2024-04-02T13:58:30.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.422+0000 7fcf8357a740 10 client.0 _ll_register_callbacks cb 0x55b413955aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:58:30.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.422+0000 7fcf7a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4139fa450 0x55b413a07aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.422+0000 7fcf7a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4139fa450 0x55b413a07aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53728/0 (socket says 172.21.15.52:53728) 2024-04-02T13:58:30.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.422+0000 7fcf7a59c640 1 -- 172.21.15.52:0/2368060979 learned_addr learned my addr 172.21.15.52:0/2368060979 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:30.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.422+0000 7fcf7ad9d640 1 --2- 172.21.15.52:0/2368060979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b413a080b0 0x55b413a05440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.422+0000 7fcf7a59c640 1 -- 172.21.15.52:0/2368060979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b413a080b0 msgr2=0x55b413a05440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.422+0000 7fcf7a59c640 1 --2- 172.21.15.52:0/2368060979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b413a080b0 0x55b413a05440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.422+0000 7fcf7a59c640 1 -- 172.21.15.52:0/2368060979 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4139fad70 msgr2=0x55b413a04e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.422+0000 7fcf79d9b640 1 --2- 172.21.15.52:0/2368060979 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4139fad70 0x55b413a04e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.422+0000 7fcf7a59c640 1 --2- 172.21.15.52:0/2368060979 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4139fad70 0x55b413a04e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.422+0000 7fcf7a59c640 1 -- 172.21.15.52:0/2368060979 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b41373c7d0 con 0x55b4139fa450 2024-04-02T13:58:30.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.422+0000 7fcf7ad9d640 1 --2- 172.21.15.52:0/2368060979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b413a080b0 0x55b413a05440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:58:30.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.422+0000 7fcf7a59c640 1 --2- 172.21.15.52:0/2368060979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4139fa450 0x55b413a07aa0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400ff90 tx=0x7fcf74011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2ba45419b73a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.422+0000 7fcf61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:58:30.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.422+0000 7fcf61ffb640 1 -- 172.21.15.52:0/2368060979 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74004240 con 0x55b4139fa450 2024-04-02T13:58:30.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.422+0000 7fcf61ffb640 1 -- 172.21.15.52:0/2368060979 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fcf740043e0 con 0x55b4139fa450 2024-04-02T13:58:30.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.422+0000 7fcf8357a740 1 -- 172.21.15.52:0/2368060979 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf74017070 con 0x55b4139fa450 2024-04-02T13:58:30.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.422+0000 7fcf61ffb640 1 -- 172.21.15.52:0/2368060979 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcf74005040 con 0x55b4139fa450 2024-04-02T13:58:30.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.422+0000 7fcf8357a740 10 client.17064 fetch_fsmap learned FSMap version 30 2024-04-02T13:58:30.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.422+0000 7fcf8357a740 1 -- 172.21.15.52:0/2368060979 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b413a16fb0 con 0x55b4139fa450 2024-04-02T13:58:30.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.423+0000 7fcf61ffb640 1 -- 172.21.15.52:0/2368060979 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf74017070 con 0x55b4139fa450 2024-04-02T13:58:30.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.423+0000 7fcf61ffb640 1 -- 172.21.15.52:0/2368060979 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fcf740046c0 con 0x55b4139fa450 2024-04-02T13:58:30.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.423+0000 7fcf8357a740 10 client.17064 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:58:30.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.423+0000 7fcf8357a740 10 client.17064 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T13:58:30.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.423+0000 7fcf8357a740 20 client.17064 populate_metadata read hostname 'smithi052' 2024-04-02T13:58:30.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.423+0000 7fcf8357a740 1 --2- 172.21.15.52:0/2368060979 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55b413a18410 0x55b413a1a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.423+0000 7fcf8357a740 4 client.17064 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:58:30.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.423+0000 7fcf8357a740 1 -- 172.21.15.52:0/2368060979 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b41396bcf0 con 0x55b413a18410 2024-04-02T13:58:30.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.423+0000 7fcf79d9b640 1 --2- 172.21.15.52:0/2368060979 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55b413a18410 0x55b413a1a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.424+0000 7fcf79d9b640 1 --2- 172.21.15.52:0/2368060979 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55b413a18410 0x55b413a1a800 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=56ee7ea1abbee099 server_cookie=6ce3b94a01a62554 in_seq=0 out_seq=0 2024-04-02T13:58:30.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.424+0000 7fcf61ffb640 10 client.17064 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:58:30.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.424+0000 7fcf61ffb640 1 -- 172.21.15.52:0/2368060979 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b41396bcf0 con 0x55b413a18410 2024-04-02T13:58:30.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.424+0000 7fcf61ffb640 10 client.17064 handle_command_reply: tid=0 2024-04-02T13:58:30.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.424+0000 7fcf8357a740 1 client.17064 shutdown 2024-04-02T13:58:30.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.425+0000 7fcf8357a740 1 -- 172.21.15.52:0/2368060979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4139fa450 msgr2=0x55b413a07aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.425+0000 7fcf8357a740 1 --2- 172.21.15.52:0/2368060979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4139fa450 0x55b413a07aa0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400ff90 tx=0x7fcf74011e00 comp rx=0 tx=0).stop 2024-04-02T13:58:30.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.425+0000 7fcf8357a740 1 -- 172.21.15.52:0/2368060979 shutdown_connections 2024-04-02T13:58:30.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.425+0000 7fcf8357a740 1 --2- 172.21.15.52:0/2368060979 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55b413a18410 0x55b413a1a800 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.425+0000 7fcf8357a740 1 --2- 172.21.15.52:0/2368060979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b413a080b0 0x55b413a05440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.425+0000 7fcf61ffb640 0 client.17064 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:58:30.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.425+0000 7fcf8357a740 1 --2- 172.21.15.52:0/2368060979 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4139fad70 0x55b413a04e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.425+0000 7fcf8357a740 1 --2- 172.21.15.52:0/2368060979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4139fa450 0x55b413a07aa0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.425+0000 7fcf8357a740 1 -- 172.21.15.52:0/2368060979 >> 172.21.15.52:0/2368060979 conn(0x55b4139f4ed0 msgr2=0x55b4139f6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.425+0000 7fcf8357a740 1 -- 172.21.15.52:0/2368060979 shutdown_connections 2024-04-02T13:58:30.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.425+0000 7fcf8357a740 1 -- 172.21.15.52:0/2368060979 wait complete. 2024-04-02T13:58:30.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.425+0000 7fcf8357a740 20 client.17064 trim_cache size 0 max 16384 2024-04-02T13:58:30.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.438+0000 7fcf8357a740 1 Processor -- start 2024-04-02T13:58:30.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.438+0000 7fcf8357a740 1 -- start start 2024-04-02T13:58:30.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.438+0000 7fcf8357a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4139fa450 0x55b413a13e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.438+0000 7fcf8357a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4139fad70 0x55b413a0a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.438+0000 7fcf8357a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b413a080b0 0x55b413a07560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.438+0000 7fcf8357a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1acb70 con 0x55b413a080b0 2024-04-02T13:58:30.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.438+0000 7fcf8357a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1ae6b0 con 0x55b4139fad70 2024-04-02T13:58:30.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.438+0000 7fcf8357a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1ad240 con 0x55b4139fa450 2024-04-02T13:58:30.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.438+0000 7fcf7ad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b413a080b0 0x55b413a07560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.438+0000 7fcf7ad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b413a080b0 0x55b413a07560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53736/0 (socket says 172.21.15.52:53736) 2024-04-02T13:58:30.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.438+0000 7fcf7ad9d640 1 -- 172.21.15.52:0/744415650 learned_addr learned my addr 172.21.15.52:0/744415650 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:30.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.439+0000 7fcf7a59c640 1 --2- 172.21.15.52:0/744415650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4139fa450 0x55b413a13e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.439+0000 7fcf79d9b640 1 --2- 172.21.15.52:0/744415650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4139fad70 0x55b413a0a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.439+0000 7fcf7ad9d640 1 -- 172.21.15.52:0/744415650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4139fa450 msgr2=0x55b413a13e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.439+0000 7fcf7ad9d640 1 --2- 172.21.15.52:0/744415650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4139fa450 0x55b413a13e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.439+0000 7fcf7ad9d640 1 -- 172.21.15.52:0/744415650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4139fad70 msgr2=0x55b413a0a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.439+0000 7fcf7ad9d640 1 --2- 172.21.15.52:0/744415650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4139fad70 0x55b413a0a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.439+0000 7fcf7ad9d640 1 -- 172.21.15.52:0/744415650 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b413a057b0 con 0x55b413a080b0 2024-04-02T13:58:30.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.439+0000 7fcf7ad9d640 1 --2- 172.21.15.52:0/744415650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b413a080b0 0x55b413a07560 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009430 tx=0x7fcf6c02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5afa060a3fc4af server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.439+0000 7fcf63fff640 1 -- 172.21.15.52:0/744415650 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c035050 con 0x55b413a080b0 2024-04-02T13:58:30.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.439+0000 7fcf63fff640 1 -- 172.21.15.52:0/744415650 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf6c0a9070 con 0x55b413a080b0 2024-04-02T13:58:30.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.439+0000 7fcf63fff640 1 -- 172.21.15.52:0/744415650 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c0c72f0 con 0x55b413a080b0 2024-04-02T13:58:30.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.439+0000 7fcf8357a740 1 -- 172.21.15.52:0/744415650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b413a080b0 msgr2=0x55b413a07560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.439+0000 7fcf8357a740 1 --2- 172.21.15.52:0/744415650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b413a080b0 0x55b413a07560 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009430 tx=0x7fcf6c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.440+0000 7fcf8357a740 1 -- 172.21.15.52:0/744415650 shutdown_connections 2024-04-02T13:58:30.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.440+0000 7fcf8357a740 1 --2- 172.21.15.52:0/744415650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b413a080b0 0x55b413a07560 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.440+0000 7fcf8357a740 1 --2- 172.21.15.52:0/744415650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4139fad70 0x55b413a0a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.440+0000 7fcf8357a740 1 --2- 172.21.15.52:0/744415650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4139fa450 0x55b413a13e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.440+0000 7fcf8357a740 1 -- 172.21.15.52:0/744415650 >> 172.21.15.52:0/744415650 conn(0x55b4139f4ed0 msgr2=0x55b413a03700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.440+0000 7fcf8357a740 1 -- 172.21.15.52:0/744415650 shutdown_connections 2024-04-02T13:58:30.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.440+0000 7fcf8357a740 1 -- 172.21.15.52:0/744415650 wait complete. 2024-04-02T13:58:30.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.440+0000 7fcf8357a740 1 -- start start 2024-04-02T13:58:30.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.440+0000 7fcf8357a740 1 Processor -- start 2024-04-02T13:58:30.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.440+0000 7fcf8357a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4139fa450 0x55b413a08c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.440+0000 7fcf8357a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4139fad70 0x55b413a09160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.440+0000 7fcf8357a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b413a080b0 0x55b413a096a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.440+0000 7fcf8357a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1adfe0 con 0x55b4139fad70 2024-04-02T13:58:30.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.440+0000 7fcf8357a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1ad910 con 0x55b4139fa450 2024-04-02T13:58:30.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.440+0000 7fcf8357a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1aab70 con 0x55b413a080b0 2024-04-02T13:58:30.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.440+0000 7fcf8357a740 10 client.0 _ll_register_callbacks cb 0x7fcf74015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:58:30.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.440+0000 7fcf79d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4139fad70 0x55b413a09160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.440+0000 7fcf79d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4139fad70 0x55b413a09160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53752/0 (socket says 172.21.15.52:53752) 2024-04-02T13:58:30.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.440+0000 7fcf79d9b640 1 -- 172.21.15.52:0/1141694988 learned_addr learned my addr 172.21.15.52:0/1141694988 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:30.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.440+0000 7fcf7a59c640 1 --2- 172.21.15.52:0/1141694988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4139fa450 0x55b413a08c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.441+0000 7fcf7ad9d640 1 --2- 172.21.15.52:0/1141694988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b413a080b0 0x55b413a096a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.441+0000 7fcf79d9b640 1 -- 172.21.15.52:0/1141694988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b413a080b0 msgr2=0x55b413a096a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.441+0000 7fcf79d9b640 1 --2- 172.21.15.52:0/1141694988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b413a080b0 0x55b413a096a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.441+0000 7fcf79d9b640 1 -- 172.21.15.52:0/1141694988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4139fa450 msgr2=0x55b413a08c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.441+0000 7fcf79d9b640 1 --2- 172.21.15.52:0/1141694988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4139fa450 0x55b413a08c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.441+0000 7fcf79d9b640 1 -- 172.21.15.52:0/1141694988 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b413a0f680 con 0x55b4139fad70 2024-04-02T13:58:30.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.441+0000 7fcf7ad9d640 1 --2- 172.21.15.52:0/1141694988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b413a080b0 0x55b413a096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:58:30.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.441+0000 7fcf79d9b640 1 --2- 172.21.15.52:0/1141694988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4139fad70 0x55b413a09160 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x55b413a04690 tx=0x7fcf64016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0b6b927d5bdef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.441+0000 7fcf61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:58:30.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.441+0000 7fcf61ffb640 1 -- 172.21.15.52:0/1141694988 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6402c470 con 0x55b4139fad70 2024-04-02T13:58:30.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.441+0000 7fcf61ffb640 1 -- 172.21.15.52:0/1141694988 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fcf6402c610 con 0x55b4139fad70 2024-04-02T13:58:30.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.441+0000 7fcf8357a740 1 -- 172.21.15.52:0/1141694988 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf6c0a9070 con 0x55b4139fad70 2024-04-02T13:58:30.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.441+0000 7fcf61ffb640 1 -- 172.21.15.52:0/1141694988 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcf64037050 con 0x55b4139fad70 2024-04-02T13:58:30.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.441+0000 7fcf8357a740 10 client.17070 fetch_fsmap learned FSMap version 30 2024-04-02T13:58:30.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.441+0000 7fcf8357a740 1 -- 172.21.15.52:0/1141694988 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b413a108e0 con 0x55b4139fad70 2024-04-02T13:58:30.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.441+0000 7fcf61ffb640 1 -- 172.21.15.52:0/1141694988 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf6c0a9070 con 0x55b4139fad70 2024-04-02T13:58:30.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.441+0000 7fcf61ffb640 1 -- 172.21.15.52:0/1141694988 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fcf6401d040 con 0x55b4139fad70 2024-04-02T13:58:30.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.441+0000 7fcf8357a740 10 client.17070 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:58:30.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.441+0000 7fcf8357a740 10 client.17070 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T13:58:30.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.441+0000 7fcf8357a740 20 client.17070 populate_metadata read hostname 'smithi052' 2024-04-02T13:58:30.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.441+0000 7fcf8357a740 1 --2- 172.21.15.52:0/1141694988 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55b413a7eb60 0x55b413a80f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.441+0000 7fcf8357a740 4 client.17070 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:58:30.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.441+0000 7fcf8357a740 1 -- 172.21.15.52:0/1141694988 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b413966040 con 0x55b413a7eb60 2024-04-02T13:58:30.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.442+0000 7fcf7a59c640 1 --2- 172.21.15.52:0/1141694988 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55b413a7eb60 0x55b413a80f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.442+0000 7fcf61ffb640 10 client.17070 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:58:30.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.442+0000 7fcf7a59c640 1 --2- 172.21.15.52:0/1141694988 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55b413a7eb60 0x55b413a80f80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4bce1903e0619b83 server_cookie=457e036d922f5408 in_seq=0 out_seq=0 2024-04-02T13:58:30.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.442+0000 7fcf61ffb640 1 -- 172.21.15.52:0/1141694988 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b413966040 con 0x55b413a7eb60 2024-04-02T13:58:30.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.442+0000 7fcf61ffb640 10 client.17070 handle_command_reply: tid=0 2024-04-02T13:58:30.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.442+0000 7fcf8357a740 1 client.17070 shutdown 2024-04-02T13:58:30.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.442+0000 7fcf8357a740 1 -- 172.21.15.52:0/1141694988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4139fad70 msgr2=0x55b413a09160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.442+0000 7fcf8357a740 1 --2- 172.21.15.52:0/1141694988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4139fad70 0x55b413a09160 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x55b413a04690 tx=0x7fcf64016040 comp rx=0 tx=0).stop 2024-04-02T13:58:30.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.442+0000 7fcf8357a740 1 -- 172.21.15.52:0/1141694988 shutdown_connections 2024-04-02T13:58:30.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.442+0000 7fcf8357a740 1 --2- 172.21.15.52:0/1141694988 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55b413a7eb60 0x55b413a80f80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.442+0000 7fcf8357a740 1 --2- 172.21.15.52:0/1141694988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b413a080b0 0x55b413a096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.442+0000 7fcf61ffb640 0 client.17070 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:58:30.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.442+0000 7fcf8357a740 1 --2- 172.21.15.52:0/1141694988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4139fad70 0x55b413a09160 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.442+0000 7fcf8357a740 1 --2- 172.21.15.52:0/1141694988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4139fa450 0x55b413a08c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.442+0000 7fcf8357a740 1 -- 172.21.15.52:0/1141694988 >> 172.21.15.52:0/1141694988 conn(0x55b4139f4ed0 msgr2=0x55b4139f6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.443+0000 7fcf8357a740 1 -- 172.21.15.52:0/1141694988 shutdown_connections 2024-04-02T13:58:30.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.443+0000 7fcf8357a740 1 -- 172.21.15.52:0/1141694988 wait complete. 2024-04-02T13:58:30.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.443+0000 7fcf8357a740 20 client.17070 trim_cache size 0 max 16384 2024-04-02T13:58:30.466 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:58:30.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.443+0000 7fcf808b7640 1 -- 172.21.15.52:0/268863286 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fcf680a3da0 msgr2=0x7fcf680a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.443+0000 7fcf808b7640 1 --2- 172.21.15.52:0/268863286 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fcf680a3da0 0x7fcf680a6260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c104f80 tx=0x7fcf64014840 comp rx=0 tx=0).stop 2024-04-02T13:58:30.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.443+0000 7fcf808b7640 1 -- 172.21.15.52:0/268863286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf7c106980 msgr2=0x7fcf7c19cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.443+0000 7fcf808b7640 1 --2- 172.21.15.52:0/268863286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf7c106980 0x7fcf7c19cb70 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0348f0 tx=0x7fcf6c032ec0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.443+0000 7fcf808b7640 1 -- 172.21.15.52:0/268863286 shutdown_connections 2024-04-02T13:58:30.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.443+0000 7fcf808b7640 1 --2- 172.21.15.52:0/268863286 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fcf680a3da0 0x7fcf680a6260 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.443+0000 7fcf808b7640 1 --2- 172.21.15.52:0/268863286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf7c106980 0x7fcf7c19cb70 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.443+0000 7fcf808b7640 1 --2- 172.21.15.52:0/268863286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf7c101330 0x7fcf7c198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.443+0000 7fcf808b7640 1 --2- 172.21.15.52:0/268863286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf7c100970 0x7fcf7c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.443+0000 7fcf808b7640 1 -- 172.21.15.52:0/268863286 >> 172.21.15.52:0/268863286 conn(0x7fcf7c077b00 msgr2=0x7fcf7c103010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.443+0000 7fcf808b7640 1 -- 172.21.15.52:0/268863286 shutdown_connections 2024-04-02T13:58:30.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.443+0000 7fcf808b7640 1 -- 172.21.15.52:0/268863286 wait complete. 2024-04-02T13:58:30.470 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:58:30.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:30 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3556182963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:58:30.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.572+0000 7f6afa093640 1 -- 172.21.15.52:0/2784481566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6af41009b0 msgr2=0x7f6af4100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.572+0000 7f6afa093640 1 --2- 172.21.15.52:0/2784481566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6af41009b0 0x7f6af4100e30 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009930 tx=0x7f6ae802bc30 comp rx=0 tx=0).stop 2024-04-02T13:58:30.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.572+0000 7f6afa093640 1 -- 172.21.15.52:0/2784481566 shutdown_connections 2024-04-02T13:58:30.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.572+0000 7f6afa093640 1 --2- 172.21.15.52:0/2784481566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6af4101370 0x7f6af410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.572+0000 7f6afa093640 1 --2- 172.21.15.52:0/2784481566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6af41009b0 0x7f6af4100e30 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.572+0000 7f6afa093640 1 --2- 172.21.15.52:0/2784481566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6af41069c0 0x7f6af4106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.572+0000 7f6afa093640 1 -- 172.21.15.52:0/2784481566 >> 172.21.15.52:0/2784481566 conn(0x7f6af4076d80 msgr2=0x7f6af4077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.572+0000 7f6afa093640 1 -- 172.21.15.52:0/2784481566 shutdown_connections 2024-04-02T13:58:30.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.572+0000 7f6afa093640 1 -- 172.21.15.52:0/2784481566 wait complete. 2024-04-02T13:58:30.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.572+0000 7f6afa093640 1 Processor -- start 2024-04-02T13:58:30.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.572+0000 7f6afa093640 1 -- start start 2024-04-02T13:58:30.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.573+0000 7f6afa093640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6af41009b0 0x7f6af41984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.573+0000 7f6afa093640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6af4101370 0x7f6af41989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.573+0000 7f6afa093640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6af41069c0 0x7f6af419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.573+0000 7f6afa093640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6af4110dd0 con 0x7f6af41009b0 2024-04-02T13:58:30.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.573+0000 7f6afa093640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6af4110c50 con 0x7f6af4101370 2024-04-02T13:58:30.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.573+0000 7f6afa093640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6af4110f50 con 0x7f6af41069c0 2024-04-02T13:58:30.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.573+0000 7f6af37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6af41009b0 0x7f6af41984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.573+0000 7f6af37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6af41009b0 0x7f6af41984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53772/0 (socket says 172.21.15.52:53772) 2024-04-02T13:58:30.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.573+0000 7f6af37fe640 1 -- 172.21.15.52:0/526318538 learned_addr learned my addr 172.21.15.52:0/526318538 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:30.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.573+0000 7f6af3fff640 1 --2- 172.21.15.52:0/526318538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6af41069c0 0x7f6af419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.573+0000 7f6af2ffd640 1 --2- 172.21.15.52:0/526318538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6af4101370 0x7f6af41989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.573+0000 7f6af37fe640 1 -- 172.21.15.52:0/526318538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6af41069c0 msgr2=0x7f6af419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.573+0000 7f6af37fe640 1 --2- 172.21.15.52:0/526318538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6af41069c0 0x7f6af419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.573+0000 7f6af37fe640 1 -- 172.21.15.52:0/526318538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6af4101370 msgr2=0x7f6af41989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.573+0000 7f6af37fe640 1 --2- 172.21.15.52:0/526318538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6af4101370 0x7f6af41989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.573+0000 7f6af37fe640 1 -- 172.21.15.52:0/526318538 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af419d440 con 0x7f6af41009b0 2024-04-02T13:58:30.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.573+0000 7f6af3fff640 1 --2- 172.21.15.52:0/526318538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6af41069c0 0x7f6af419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:58:30.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.573+0000 7f6af37fe640 1 --2- 172.21.15.52:0/526318538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6af41009b0 0x7f6af41984a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400d950 tx=0x7f6ae400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.573+0000 7f6af0ff9640 1 -- 172.21.15.52:0/526318538 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4017070 con 0x7f6af41009b0 2024-04-02T13:58:30.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.573+0000 7f6af0ff9640 1 -- 172.21.15.52:0/526318538 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ae4009c30 con 0x7f6af41009b0 2024-04-02T13:58:30.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.573+0000 7f6afa093640 1 -- 172.21.15.52:0/526318538 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af419d730 con 0x7f6af41009b0 2024-04-02T13:58:30.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.573+0000 7f6af0ff9640 1 -- 172.21.15.52:0/526318538 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae400f6b0 con 0x7f6af41009b0 2024-04-02T13:58:30.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.574+0000 7f6afa093640 1 -- 172.21.15.52:0/526318538 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af41a4fc0 con 0x7f6af41009b0 2024-04-02T13:58:30.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.576+0000 7f6af0ff9640 1 -- 172.21.15.52:0/526318538 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6ae40109d0 con 0x7f6af41009b0 2024-04-02T13:58:30.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.576+0000 7f6afcc83740 1 Processor -- start 2024-04-02T13:58:30.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.576+0000 7f6af0ff9640 1 --2- 172.21.15.52:0/526318538 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6adc0a3d20 0x7f6adc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.576+0000 7f6afcc83740 1 -- start start 2024-04-02T13:58:30.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.576+0000 7f6af3fff640 1 --2- 172.21.15.52:0/526318538 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6adc0a3d20 0x7f6adc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.576+0000 7f6afcc83740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdc74de7a0 0x55cdc74deb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.576+0000 7f6afcc83740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74df0c0 0x55cdc74e7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.576+0000 7f6af0ff9640 1 -- 172.21.15.52:0/526318538 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f6ae40d07a0 con 0x7f6af41009b0 2024-04-02T13:58:30.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.576+0000 7f6afcc83740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cdc74e7ca0 0x55cdc74ea090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.576+0000 7f6afcc83740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cdc74427b0 con 0x55cdc74df0c0 2024-04-02T13:58:30.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.576+0000 7f6afcc83740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cdc73e1570 con 0x55cdc74de7a0 2024-04-02T13:58:30.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.576+0000 7f6afcc83740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cdc743e760 con 0x55cdc74e7ca0 2024-04-02T13:58:30.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.576+0000 7f6af2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74df0c0 0x55cdc74e7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.577+0000 7f6af2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74df0c0 0x55cdc74e7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53776/0 (socket says 172.21.15.52:53776) 2024-04-02T13:58:30.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.577+0000 7f6af2ffd640 1 -- 172.21.15.52:0/2089407378 learned_addr learned my addr 172.21.15.52:0/2089407378 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:30.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.577+0000 7f6af37fe640 1 --2- 172.21.15.52:0/2089407378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdc74de7a0 0x55cdc74deb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.577+0000 7f6af3fff640 1 --2- 172.21.15.52:0/526318538 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6adc0a3d20 0x7f6adc0a61e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00ae10 tx=0x7f6aec00e720 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.577+0000 7f6af3fff640 1 --2- 172.21.15.52:0/2089407378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cdc74e7ca0 0x55cdc74ea090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.577+0000 7f6af2ffd640 1 -- 172.21.15.52:0/2089407378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cdc74e7ca0 msgr2=0x55cdc74ea090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.577+0000 7f6af2ffd640 1 --2- 172.21.15.52:0/2089407378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cdc74e7ca0 0x55cdc74ea090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.577+0000 7f6af2ffd640 1 -- 172.21.15.52:0/2089407378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdc74de7a0 msgr2=0x55cdc74deb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.577+0000 7f6af2ffd640 1 --2- 172.21.15.52:0/2089407378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdc74de7a0 0x55cdc74deb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.577+0000 7f6af2ffd640 1 -- 172.21.15.52:0/2089407378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdc71563f0 con 0x55cdc74df0c0 2024-04-02T13:58:30.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.577+0000 7f6af3fff640 1 --2- 172.21.15.52:0/2089407378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cdc74e7ca0 0x55cdc74ea090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:58:30.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.577+0000 7f6af2ffd640 1 --2- 172.21.15.52:0/2089407378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74df0c0 0x55cdc74e7760 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8036040 tx=0x7f6ae802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4adf76b2fe573c24 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.577+0000 7f6ad9ffb640 1 -- 172.21.15.52:0/2089407378 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae802cb00 con 0x55cdc74df0c0 2024-04-02T13:58:30.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.577+0000 7f6ad9ffb640 1 -- 172.21.15.52:0/2089407378 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ae802cca0 con 0x55cdc74df0c0 2024-04-02T13:58:30.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.577+0000 7f6ad9ffb640 1 -- 172.21.15.52:0/2089407378 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8004920 con 0x55cdc74df0c0 2024-04-02T13:58:30.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.578+0000 7f6afcc83740 1 -- 172.21.15.52:0/2089407378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74df0c0 msgr2=0x55cdc74e7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.578+0000 7f6afcc83740 1 --2- 172.21.15.52:0/2089407378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74df0c0 0x55cdc74e7760 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8036040 tx=0x7f6ae802c650 comp rx=0 tx=0).stop 2024-04-02T13:58:30.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.578+0000 7f6afcc83740 1 -- 172.21.15.52:0/2089407378 shutdown_connections 2024-04-02T13:58:30.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.578+0000 7f6afcc83740 1 --2- 172.21.15.52:0/2089407378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cdc74e7ca0 0x55cdc74ea090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.578+0000 7f6afcc83740 1 --2- 172.21.15.52:0/2089407378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74df0c0 0x55cdc74e7760 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.578+0000 7f6afcc83740 1 --2- 172.21.15.52:0/2089407378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdc74de7a0 0x55cdc74deb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.578+0000 7f6afcc83740 1 -- 172.21.15.52:0/2089407378 >> 172.21.15.52:0/2089407378 conn(0x55cdc74d8ed0 msgr2=0x55cdc74d92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.578+0000 7f6afcc83740 1 -- 172.21.15.52:0/2089407378 shutdown_connections 2024-04-02T13:58:30.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.578+0000 7f6afcc83740 1 -- 172.21.15.52:0/2089407378 wait complete. 2024-04-02T13:58:30.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.578+0000 7f6afcc83740 1 -- start start 2024-04-02T13:58:30.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.578+0000 7f6afcc83740 1 Processor -- start 2024-04-02T13:58:30.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.579+0000 7f6afcc83740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdc74de7a0 0x55cdc74e7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.579+0000 7f6afcc83740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cdc74df0c0 0x55cdc74e4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.579+0000 7f6afcc83740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74e7ca0 0x55cdc74e4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.579+0000 7f6afcc83740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cdc74eaa30 con 0x55cdc74e7ca0 2024-04-02T13:58:30.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.579+0000 7f6afcc83740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cdc73fde80 con 0x55cdc74de7a0 2024-04-02T13:58:30.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.579+0000 7f6afcc83740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cdc74eabb0 con 0x55cdc74df0c0 2024-04-02T13:58:30.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.579+0000 7f6afcc83740 10 client.0 _ll_register_callbacks cb 0x55cdc7439aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:58:30.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.579+0000 7f6af3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74e7ca0 0x55cdc74e4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.579+0000 7f6af3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74e7ca0 0x55cdc74e4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53790/0 (socket says 172.21.15.52:53790) 2024-04-02T13:58:30.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.579+0000 7f6af3fff640 1 -- 172.21.15.52:0/3209019958 learned_addr learned my addr 172.21.15.52:0/3209019958 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:30.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.579+0000 7f6af37fe640 1 --2- 172.21.15.52:0/3209019958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdc74de7a0 0x55cdc74e7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.579+0000 7f6af2ffd640 1 --2- 172.21.15.52:0/3209019958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cdc74df0c0 0x55cdc74e4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.579+0000 7f6af3fff640 1 -- 172.21.15.52:0/3209019958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cdc74df0c0 msgr2=0x55cdc74e4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.579+0000 7f6af3fff640 1 --2- 172.21.15.52:0/3209019958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cdc74df0c0 0x55cdc74e4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.579+0000 7f6af3fff640 1 -- 172.21.15.52:0/3209019958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdc74de7a0 msgr2=0x55cdc74e7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.579+0000 7f6af3fff640 1 --2- 172.21.15.52:0/3209019958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdc74de7a0 0x55cdc74e7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.579+0000 7f6af3fff640 1 -- 172.21.15.52:0/3209019958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdc72207d0 con 0x55cdc74e7ca0 2024-04-02T13:58:30.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.579+0000 7f6af3fff640 1 --2- 172.21.15.52:0/3209019958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74e7ca0 0x55cdc74e4ef0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f6aec012e00 tx=0x7f6aec004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b11289e1abf36bc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.579+0000 7f6ad2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:58:30.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.579+0000 7f6ad2ffd640 1 -- 172.21.15.52:0/3209019958 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec022030 con 0x55cdc74e7ca0 2024-04-02T13:58:30.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.579+0000 7f6ad2ffd640 1 -- 172.21.15.52:0/3209019958 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f6aec004b20 con 0x55cdc74e7ca0 2024-04-02T13:58:30.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.579+0000 7f6afcc83740 1 -- 172.21.15.52:0/3209019958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ae802cca0 con 0x55cdc74e7ca0 2024-04-02T13:58:30.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.580+0000 7f6ad2ffd640 1 -- 172.21.15.52:0/3209019958 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6aec02e050 con 0x55cdc74e7ca0 2024-04-02T13:58:30.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.580+0000 7f6afcc83740 10 client.17082 fetch_fsmap learned FSMap version 30 2024-04-02T13:58:30.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.580+0000 7f6afcc83740 1 -- 172.21.15.52:0/3209019958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cdc74f69a0 con 0x55cdc74e7ca0 2024-04-02T13:58:30.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.580+0000 7f6ad2ffd640 1 -- 172.21.15.52:0/3209019958 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ae802cca0 con 0x55cdc74e7ca0 2024-04-02T13:58:30.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.580+0000 7f6ad2ffd640 1 -- 172.21.15.52:0/3209019958 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6aec005020 con 0x55cdc74e7ca0 2024-04-02T13:58:30.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.580+0000 7f6afcc83740 10 client.17082 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:58:30.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.580+0000 7f6afcc83740 10 client.17082 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:58:30.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.580+0000 7f6afcc83740 20 client.17082 populate_metadata read hostname 'smithi052' 2024-04-02T13:58:30.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.580+0000 7f6afcc83740 1 --2- 172.21.15.52:0/3209019958 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55cdc74f7e00 0x55cdc74fa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.580+0000 7f6afcc83740 4 client.17082 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:58:30.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.580+0000 7f6afcc83740 1 -- 172.21.15.52:0/3209019958 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdc744fcf0 con 0x55cdc74f7e00 2024-04-02T13:58:30.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.580+0000 7f6af37fe640 1 --2- 172.21.15.52:0/3209019958 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55cdc74f7e00 0x55cdc74fa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.581+0000 7f6af37fe640 1 --2- 172.21.15.52:0/3209019958 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55cdc74f7e00 0x55cdc74fa260 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=27cf99e1bdc18bd1 server_cookie=af9fe5369685e697 in_seq=0 out_seq=0 2024-04-02T13:58:30.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.581+0000 7f6ad2ffd640 10 client.17082 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:58:30.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.581+0000 7f6ad2ffd640 1 -- 172.21.15.52:0/3209019958 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cdc744fcf0 con 0x55cdc74f7e00 2024-04-02T13:58:30.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.581+0000 7f6ad2ffd640 10 client.17082 handle_command_reply: tid=0 2024-04-02T13:58:30.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.581+0000 7f6afcc83740 1 client.17082 shutdown 2024-04-02T13:58:30.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.581+0000 7f6afcc83740 1 -- 172.21.15.52:0/3209019958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74e7ca0 msgr2=0x55cdc74e4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.581+0000 7f6afcc83740 1 --2- 172.21.15.52:0/3209019958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74e7ca0 0x55cdc74e4ef0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f6aec012e00 tx=0x7f6aec004ab0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.582+0000 7f6afcc83740 1 -- 172.21.15.52:0/3209019958 shutdown_connections 2024-04-02T13:58:30.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.582+0000 7f6afcc83740 1 --2- 172.21.15.52:0/3209019958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74e7ca0 0x55cdc74e4ef0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.582+0000 7f6afcc83740 1 --2- 172.21.15.52:0/3209019958 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55cdc74f7e00 0x55cdc74fa260 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.582+0000 7f6afcc83740 1 --2- 172.21.15.52:0/3209019958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cdc74df0c0 0x55cdc74e4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.582+0000 7f6afcc83740 1 --2- 172.21.15.52:0/3209019958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdc74de7a0 0x55cdc74e7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.582+0000 7f6afcc83740 1 -- 172.21.15.52:0/3209019958 >> 172.21.15.52:0/3209019958 conn(0x55cdc74d8ed0 msgr2=0x55cdc74dac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.582+0000 7f6ad2ffd640 0 client.17082 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:58:30.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.582+0000 7f6afcc83740 1 -- 172.21.15.52:0/3209019958 shutdown_connections 2024-04-02T13:58:30.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.582+0000 7f6afcc83740 1 -- 172.21.15.52:0/3209019958 wait complete. 2024-04-02T13:58:30.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.582+0000 7f6afcc83740 20 client.17082 trim_cache size 0 max 16384 2024-04-02T13:58:30.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.595+0000 7f6afcc83740 1 Processor -- start 2024-04-02T13:58:30.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.595+0000 7f6afcc83740 1 -- start start 2024-04-02T13:58:30.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.595+0000 7f6afcc83740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cdc74de7a0 0x55cdc74f3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.595+0000 7f6afcc83740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74df0c0 0x55cdc74d3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.595+0000 7f6afcc83740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdc74e7ca0 0x55cdc74e70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.595+0000 7f6afcc83740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6af41aee50 con 0x55cdc74df0c0 2024-04-02T13:58:30.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.595+0000 7f6afcc83740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6af41b0990 con 0x55cdc74e7ca0 2024-04-02T13:58:30.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.595+0000 7f6afcc83740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6af41af520 con 0x55cdc74de7a0 2024-04-02T13:58:30.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.595+0000 7f6af2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74df0c0 0x55cdc74d3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.595+0000 7f6af2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74df0c0 0x55cdc74d3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53806/0 (socket says 172.21.15.52:53806) 2024-04-02T13:58:30.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.595+0000 7f6af2ffd640 1 -- 172.21.15.52:0/440656871 learned_addr learned my addr 172.21.15.52:0/440656871 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:30.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.595+0000 7f6af3fff640 1 --2- 172.21.15.52:0/440656871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdc74e7ca0 0x55cdc74e70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.596+0000 7f6af37fe640 1 --2- 172.21.15.52:0/440656871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cdc74de7a0 0x55cdc74f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.596+0000 7f6af2ffd640 1 -- 172.21.15.52:0/440656871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cdc74de7a0 msgr2=0x55cdc74f3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.596+0000 7f6af2ffd640 1 --2- 172.21.15.52:0/440656871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cdc74de7a0 0x55cdc74f3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.596+0000 7f6af2ffd640 1 -- 172.21.15.52:0/440656871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdc74e7ca0 msgr2=0x55cdc74e70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.596+0000 7f6af2ffd640 1 --2- 172.21.15.52:0/440656871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdc74e7ca0 0x55cdc74e70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.596+0000 7f6af2ffd640 1 -- 172.21.15.52:0/440656871 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdc74ddf40 con 0x55cdc74df0c0 2024-04-02T13:58:30.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.596+0000 7f6af2ffd640 1 --2- 172.21.15.52:0/440656871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74df0c0 0x55cdc74d3d90 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x55cdc74de270 tx=0x7f6ae802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7801399e50d886a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.596+0000 7f6ad97fa640 1 -- 172.21.15.52:0/440656871 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae804e070 con 0x55cdc74df0c0 2024-04-02T13:58:30.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.596+0000 7f6ad97fa640 1 -- 172.21.15.52:0/440656871 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ae8005040 con 0x55cdc74df0c0 2024-04-02T13:58:30.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.596+0000 7f6ad97fa640 1 -- 172.21.15.52:0/440656871 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae80049f0 con 0x55cdc74df0c0 2024-04-02T13:58:30.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.596+0000 7f6afcc83740 1 -- 172.21.15.52:0/440656871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74df0c0 msgr2=0x55cdc74d3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.596+0000 7f6afcc83740 1 --2- 172.21.15.52:0/440656871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74df0c0 0x55cdc74d3d90 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x55cdc74de270 tx=0x7f6ae802bc30 comp rx=0 tx=0).stop 2024-04-02T13:58:30.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.597+0000 7f6afcc83740 1 -- 172.21.15.52:0/440656871 shutdown_connections 2024-04-02T13:58:30.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.597+0000 7f6afcc83740 1 --2- 172.21.15.52:0/440656871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdc74e7ca0 0x55cdc74e70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.597+0000 7f6afcc83740 1 --2- 172.21.15.52:0/440656871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74df0c0 0x55cdc74d3d90 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.597+0000 7f6afcc83740 1 --2- 172.21.15.52:0/440656871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cdc74de7a0 0x55cdc74f3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.597+0000 7f6afcc83740 1 -- 172.21.15.52:0/440656871 >> 172.21.15.52:0/440656871 conn(0x55cdc74d8ed0 msgr2=0x55cdc74e3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.597+0000 7f6afcc83740 1 -- 172.21.15.52:0/440656871 shutdown_connections 2024-04-02T13:58:30.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.597+0000 7f6afcc83740 1 -- 172.21.15.52:0/440656871 wait complete. 2024-04-02T13:58:30.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.597+0000 7f6afcc83740 1 -- start start 2024-04-02T13:58:30.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.597+0000 7f6afcc83740 1 Processor -- start 2024-04-02T13:58:30.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.597+0000 7f6afcc83740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74de7a0 0x55cdc74e8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.597+0000 7f6afcc83740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdc74df0c0 0x55cdc74e9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.597+0000 7f6afcc83740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cdc74e7ca0 0x55cdc74e9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.597+0000 7f6afcc83740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6af41b02c0 con 0x55cdc74de7a0 2024-04-02T13:58:30.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.597+0000 7f6afcc83740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6af41afbf0 con 0x55cdc74df0c0 2024-04-02T13:58:30.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.597+0000 7f6afcc83740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6af41ace30 con 0x55cdc74e7ca0 2024-04-02T13:58:30.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.597+0000 7f6afcc83740 10 client.0 _ll_register_callbacks cb 0x7f6aec016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:58:30.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.597+0000 7f6af37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74de7a0 0x55cdc74e8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.597+0000 7f6af37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74de7a0 0x55cdc74e8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53816/0 (socket says 172.21.15.52:53816) 2024-04-02T13:58:30.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.597+0000 7f6af37fe640 1 -- 172.21.15.52:0/882477669 learned_addr learned my addr 172.21.15.52:0/882477669 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:30.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.598+0000 7f6af3fff640 1 --2- 172.21.15.52:0/882477669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cdc74e7ca0 0x55cdc74e9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.598+0000 7f6af37fe640 1 -- 172.21.15.52:0/882477669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cdc74e7ca0 msgr2=0x55cdc74e9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.598+0000 7f6af2ffd640 1 --2- 172.21.15.52:0/882477669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdc74df0c0 0x55cdc74e9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.598+0000 7f6af37fe640 1 --2- 172.21.15.52:0/882477669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cdc74e7ca0 0x55cdc74e9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.598+0000 7f6af37fe640 1 -- 172.21.15.52:0/882477669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdc74df0c0 msgr2=0x55cdc74e9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.598+0000 7f6af37fe640 1 --2- 172.21.15.52:0/882477669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdc74df0c0 0x55cdc74e9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.598+0000 7f6af37fe640 1 -- 172.21.15.52:0/882477669 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdc74ee440 con 0x55cdc74de7a0 2024-04-02T13:58:30.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.598+0000 7f6af37fe640 1 --2- 172.21.15.52:0/882477669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74de7a0 0x55cdc74e8c00 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4010fd0 tx=0x7f6ae4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d41a561ec9c4a2c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.598+0000 7f6ad2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:58:30.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.598+0000 7f6ad2ffd640 1 -- 172.21.15.52:0/882477669 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae40b4020 con 0x55cdc74de7a0 2024-04-02T13:58:30.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.598+0000 7f6ad2ffd640 1 -- 172.21.15.52:0/882477669 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f6ae4098a70 con 0x55cdc74de7a0 2024-04-02T13:58:30.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.598+0000 7f6afcc83740 1 -- 172.21.15.52:0/882477669 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ae8005040 con 0x55cdc74de7a0 2024-04-02T13:58:30.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.598+0000 7f6ad2ffd640 1 -- 172.21.15.52:0/882477669 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6ae4020040 con 0x55cdc74de7a0 2024-04-02T13:58:30.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.598+0000 7f6afcc83740 10 client.17088 fetch_fsmap learned FSMap version 30 2024-04-02T13:58:30.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.598+0000 7f6afcc83740 1 -- 172.21.15.52:0/882477669 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cdc7562cf0 con 0x55cdc74de7a0 2024-04-02T13:58:30.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.598+0000 7f6ad2ffd640 1 -- 172.21.15.52:0/882477669 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ae8005040 con 0x55cdc74de7a0 2024-04-02T13:58:30.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.598+0000 7f6ad2ffd640 1 -- 172.21.15.52:0/882477669 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6ae40a4be0 con 0x55cdc74de7a0 2024-04-02T13:58:30.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.598+0000 7f6afcc83740 10 client.17088 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:58:30.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.598+0000 7f6afcc83740 10 client.17088 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:58:30.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.598+0000 7f6afcc83740 20 client.17088 populate_metadata read hostname 'smithi052' 2024-04-02T13:58:30.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.598+0000 7f6afcc83740 1 --2- 172.21.15.52:0/882477669 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55cdc7563410 0x55cdc7565830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.598+0000 7f6afcc83740 4 client.17088 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:58:30.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.598+0000 7f6afcc83740 1 -- 172.21.15.52:0/882477669 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cdc744a040 con 0x55cdc7563410 2024-04-02T13:58:30.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.599+0000 7f6af2ffd640 1 --2- 172.21.15.52:0/882477669 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55cdc7563410 0x55cdc7565830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.599+0000 7f6ad2ffd640 10 client.17088 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:58:30.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.599+0000 7f6af2ffd640 1 --2- 172.21.15.52:0/882477669 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55cdc7563410 0x55cdc7565830 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3886247e7b740558 server_cookie=de120c3eed7f3b7d in_seq=0 out_seq=0 2024-04-02T13:58:30.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.599+0000 7f6ad2ffd640 1 -- 172.21.15.52:0/882477669 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cdc744a040 con 0x55cdc7563410 2024-04-02T13:58:30.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.599+0000 7f6ad2ffd640 10 client.17088 handle_command_reply: tid=0 2024-04-02T13:58:30.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.599+0000 7f6afcc83740 1 client.17088 shutdown 2024-04-02T13:58:30.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.599+0000 7f6afcc83740 1 -- 172.21.15.52:0/882477669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74de7a0 msgr2=0x55cdc74e8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.600+0000 7f6afcc83740 1 --2- 172.21.15.52:0/882477669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74de7a0 0x55cdc74e8c00 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4010fd0 tx=0x7f6ae4005f90 comp rx=0 tx=0).stop 2024-04-02T13:58:30.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.600+0000 7f6afcc83740 1 -- 172.21.15.52:0/882477669 shutdown_connections 2024-04-02T13:58:30.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.600+0000 7f6afcc83740 1 --2- 172.21.15.52:0/882477669 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55cdc7563410 0x55cdc7565830 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.600+0000 7f6afcc83740 1 --2- 172.21.15.52:0/882477669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cdc74e7ca0 0x55cdc74e9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.600+0000 7f6afcc83740 1 --2- 172.21.15.52:0/882477669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdc74df0c0 0x55cdc74e9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.600+0000 7f6ad2ffd640 0 client.17088 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:58:30.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.600+0000 7f6afcc83740 1 --2- 172.21.15.52:0/882477669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdc74de7a0 0x55cdc74e8c00 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.600+0000 7f6afcc83740 1 -- 172.21.15.52:0/882477669 >> 172.21.15.52:0/882477669 conn(0x55cdc74d8ed0 msgr2=0x55cdc74d76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.600+0000 7f6afcc83740 1 -- 172.21.15.52:0/882477669 shutdown_connections 2024-04-02T13:58:30.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.600+0000 7f6afcc83740 1 -- 172.21.15.52:0/882477669 wait complete. 2024-04-02T13:58:30.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.600+0000 7f6afcc83740 20 client.17088 trim_cache size 0 max 16384 2024-04-02T13:58:30.616 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:58:30.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.600+0000 7f6afa093640 1 -- 172.21.15.52:0/526318538 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6adc0a3d20 msgr2=0x7f6adc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.600+0000 7f6afa093640 1 --2- 172.21.15.52:0/526318538 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6adc0a3d20 0x7f6adc0a61e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00ae10 tx=0x7f6aec00e720 comp rx=0 tx=0).stop 2024-04-02T13:58:30.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.600+0000 7f6afa093640 1 -- 172.21.15.52:0/526318538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6af41009b0 msgr2=0x7f6af41984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.600+0000 7f6afa093640 1 --2- 172.21.15.52:0/526318538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6af41009b0 0x7f6af41984a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400d950 tx=0x7f6ae400de10 comp rx=0 tx=0).stop 2024-04-02T13:58:30.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.600+0000 7f6afa093640 1 -- 172.21.15.52:0/526318538 shutdown_connections 2024-04-02T13:58:30.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.600+0000 7f6afa093640 1 --2- 172.21.15.52:0/526318538 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6adc0a3d20 0x7f6adc0a61e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.600+0000 7f6afa093640 1 --2- 172.21.15.52:0/526318538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6af41069c0 0x7f6af419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.600+0000 7f6afa093640 1 --2- 172.21.15.52:0/526318538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6af4101370 0x7f6af41989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.600+0000 7f6afa093640 1 --2- 172.21.15.52:0/526318538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6af41009b0 0x7f6af41984a0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.600+0000 7f6afa093640 1 -- 172.21.15.52:0/526318538 >> 172.21.15.52:0/526318538 conn(0x7f6af4076d80 msgr2=0x7f6af406a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.600+0000 7f6afa093640 1 -- 172.21.15.52:0/526318538 shutdown_connections 2024-04-02T13:58:30.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.600+0000 7f6afa093640 1 -- 172.21.15.52:0/526318538 wait complete. 2024-04-02T13:58:30.620 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:58:30.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.721+0000 7f07dba33640 1 -- 172.21.15.52:0/891653031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d4101330 msgr2=0x7f07d410d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.721+0000 7f07dba33640 1 --2- 172.21.15.52:0/891653031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d4101330 0x7f07d410d840 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009cd0 tx=0x7f07cc02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.721+0000 7f07dba33640 1 -- 172.21.15.52:0/891653031 shutdown_connections 2024-04-02T13:58:30.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.721+0000 7f07dba33640 1 --2- 172.21.15.52:0/891653031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d4101330 0x7f07d410d840 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.721+0000 7f07dba33640 1 --2- 172.21.15.52:0/891653031 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07d4100970 0x7f07d4100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.721+0000 7f07dba33640 1 --2- 172.21.15.52:0/891653031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07d4106980 0x7f07d4106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.721+0000 7f07dba33640 1 -- 172.21.15.52:0/891653031 >> 172.21.15.52:0/891653031 conn(0x7f07d4077b00 msgr2=0x7f07d4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.721+0000 7f07dba33640 1 -- 172.21.15.52:0/891653031 shutdown_connections 2024-04-02T13:58:30.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.721+0000 7f07dba33640 1 -- 172.21.15.52:0/891653031 wait complete. 2024-04-02T13:58:30.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.722+0000 7f07dba33640 1 Processor -- start 2024-04-02T13:58:30.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.722+0000 7f07dba33640 1 -- start start 2024-04-02T13:58:30.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.722+0000 7f07dba33640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07d4100970 0x7f07d4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.722+0000 7f07dba33640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d4101330 0x7f07d41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.722+0000 7f07dba33640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07d4106980 0x7f07d419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.722+0000 7f07dba33640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f07d4111e90 con 0x7f07d4101330 2024-04-02T13:58:30.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.722+0000 7f07dba33640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f07d4111d10 con 0x7f07d4106980 2024-04-02T13:58:30.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.722+0000 7f07dba33640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f07d4112010 con 0x7f07d4100970 2024-04-02T13:58:30.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.722+0000 7f07d8fce640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d4101330 0x7f07d41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.722+0000 7f07d8fce640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d4101330 0x7f07d41988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53844/0 (socket says 172.21.15.52:53844) 2024-04-02T13:58:30.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.722+0000 7f07d8fce640 1 -- 172.21.15.52:0/2665673638 learned_addr learned my addr 172.21.15.52:0/2665673638 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:30.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.722+0000 7f07d9fd0640 1 --2- 172.21.15.52:0/2665673638 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07d4106980 0x7f07d419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.722+0000 7f07d97cf640 1 --2- 172.21.15.52:0/2665673638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07d4100970 0x7f07d4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.722+0000 7f07d8fce640 1 -- 172.21.15.52:0/2665673638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07d4100970 msgr2=0x7f07d4198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.722+0000 7f07d8fce640 1 --2- 172.21.15.52:0/2665673638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07d4100970 0x7f07d4198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.722+0000 7f07d8fce640 1 -- 172.21.15.52:0/2665673638 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07d4106980 msgr2=0x7f07d419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.722+0000 7f07d8fce640 1 --2- 172.21.15.52:0/2665673638 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07d4106980 0x7f07d419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.722+0000 7f07d8fce640 1 -- 172.21.15.52:0/2665673638 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d419d360 con 0x7f07d4101330 2024-04-02T13:58:30.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.722+0000 7f07d97cf640 1 --2- 172.21.15.52:0/2665673638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07d4100970 0x7f07d4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:58:30.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.722+0000 7f07d8fce640 1 --2- 172.21.15.52:0/2665673638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d4101330 0x7f07d41988b0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f07c400d950 tx=0x7f07c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.723+0000 7f07ca7fc640 1 -- 172.21.15.52:0/2665673638 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4017030 con 0x7f07d4101330 2024-04-02T13:58:30.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.723+0000 7f07ca7fc640 1 -- 172.21.15.52:0/2665673638 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07c4004020 con 0x7f07d4101330 2024-04-02T13:58:30.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.723+0000 7f07dba33640 1 -- 172.21.15.52:0/2665673638 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07d419d650 con 0x7f07d4101330 2024-04-02T13:58:30.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.723+0000 7f07ca7fc640 1 -- 172.21.15.52:0/2665673638 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c40047a0 con 0x7f07d4101330 2024-04-02T13:58:30.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.723+0000 7f07dba33640 1 -- 172.21.15.52:0/2665673638 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d41a4ee0 con 0x7f07d4101330 2024-04-02T13:58:30.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.725+0000 7f07ca7fc640 1 -- 172.21.15.52:0/2665673638 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f07c40041d0 con 0x7f07d4101330 2024-04-02T13:58:30.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.725+0000 7f07de616740 1 Processor -- start 2024-04-02T13:58:30.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.725+0000 7f07de616740 1 -- start start 2024-04-02T13:58:30.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.726+0000 7f07de616740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8467277a0 0x55a846727b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.726+0000 7f07de616740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467280c0 0x55a846730760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.726+0000 7f07de616740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a846730ca0 0x55a846733090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.726+0000 7f07de616740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a84668b7b0 con 0x55a8467280c0 2024-04-02T13:58:30.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.726+0000 7f07de616740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a84662a570 con 0x55a846730ca0 2024-04-02T13:58:30.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.726+0000 7f07de616740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a846687760 con 0x55a8467277a0 2024-04-02T13:58:30.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.726+0000 7f07ca7fc640 1 --2- 172.21.15.52:0/2665673638 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f07c00a3d20 0x7f07c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.726+0000 7f07d97cf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467280c0 0x55a846730760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.726+0000 7f07d97cf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467280c0 0x55a846730760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53856/0 (socket says 172.21.15.52:53856) 2024-04-02T13:58:30.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.726+0000 7f07d97cf640 1 -- 172.21.15.52:0/232161832 learned_addr learned my addr 172.21.15.52:0/232161832 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:30.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.726+0000 7f07d9fd0640 1 --2- 172.21.15.52:0/2665673638 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f07c00a3d20 0x7f07c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.726+0000 7f07d8fce640 1 --2- 172.21.15.52:0/232161832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a846730ca0 0x55a846733090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.726+0000 7f07d9fd0640 1 --2- 172.21.15.52:0/232161832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8467277a0 0x55a846727b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.726+0000 7f07ca7fc640 1 -- 172.21.15.52:0/2665673638 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f07c40d05b0 con 0x7f07d4101330 2024-04-02T13:58:30.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.726+0000 7f07d97cf640 1 -- 172.21.15.52:0/232161832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8467277a0 msgr2=0x55a846727b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.726+0000 7f07d97cf640 1 --2- 172.21.15.52:0/232161832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8467277a0 0x55a846727b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.726+0000 7f07d97cf640 1 -- 172.21.15.52:0/232161832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a846730ca0 msgr2=0x55a846733090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.726+0000 7f07d97cf640 1 --2- 172.21.15.52:0/232161832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a846730ca0 0x55a846733090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.726+0000 7f07d97cf640 1 -- 172.21.15.52:0/232161832 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a84639f3f0 con 0x55a8467280c0 2024-04-02T13:58:30.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.726+0000 7f07d97cf640 1 --2- 172.21.15.52:0/232161832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467280c0 0x55a846730760 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f07d0009ca0 tx=0x7f07d00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c6e496a8786426 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.726+0000 7f07d9fd0640 1 --2- 172.21.15.52:0/2665673638 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f07c00a3d20 0x7f07c00a61e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f07cc005cc0 tx=0x7f07cc02e040 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.726+0000 7f07bf7fe640 1 -- 172.21.15.52:0/232161832 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0012450 con 0x55a8467280c0 2024-04-02T13:58:30.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.727+0000 7f07bf7fe640 1 -- 172.21.15.52:0/232161832 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07d0012ad0 con 0x55a8467280c0 2024-04-02T13:58:30.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.727+0000 7f07bf7fe640 1 -- 172.21.15.52:0/232161832 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d000fbc0 con 0x55a8467280c0 2024-04-02T13:58:30.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.727+0000 7f07de616740 1 -- 172.21.15.52:0/232161832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467280c0 msgr2=0x55a846730760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.727+0000 7f07de616740 1 --2- 172.21.15.52:0/232161832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467280c0 0x55a846730760 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f07d0009ca0 tx=0x7f07d00117b0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.727+0000 7f07de616740 1 -- 172.21.15.52:0/232161832 shutdown_connections 2024-04-02T13:58:30.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.727+0000 7f07de616740 1 --2- 172.21.15.52:0/232161832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a846730ca0 0x55a846733090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.727+0000 7f07de616740 1 --2- 172.21.15.52:0/232161832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467280c0 0x55a846730760 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.727+0000 7f07de616740 1 --2- 172.21.15.52:0/232161832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8467277a0 0x55a846727b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.727+0000 7f07de616740 1 -- 172.21.15.52:0/232161832 >> 172.21.15.52:0/232161832 conn(0x55a846721ed0 msgr2=0x55a8467222b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.727+0000 7f07de616740 1 -- 172.21.15.52:0/232161832 shutdown_connections 2024-04-02T13:58:30.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.727+0000 7f07de616740 1 -- 172.21.15.52:0/232161832 wait complete. 2024-04-02T13:58:30.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.728+0000 7f07de616740 1 -- start start 2024-04-02T13:58:30.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.728+0000 7f07de616740 1 Processor -- start 2024-04-02T13:58:30.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.728+0000 7f07de616740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467277a0 0x55a846730690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.728+0000 7f07de616740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a8467280c0 0x55a84672da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.728+0000 7f07de616740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a846730ca0 0x55a84672e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.728+0000 7f07de616740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a846733a90 con 0x55a8467277a0 2024-04-02T13:58:30.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.728+0000 7f07de616740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a846646e80 con 0x55a8467280c0 2024-04-02T13:58:30.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.728+0000 7f07de616740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a846733c10 con 0x55a846730ca0 2024-04-02T13:58:30.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.728+0000 7f07de616740 10 client.0 _ll_register_callbacks cb 0x55a846682aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:58:30.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.728+0000 7f07d97cf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467277a0 0x55a846730690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.728+0000 7f07d97cf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467277a0 0x55a846730690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53858/0 (socket says 172.21.15.52:53858) 2024-04-02T13:58:30.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.728+0000 7f07d97cf640 1 -- 172.21.15.52:0/2917439285 learned_addr learned my addr 172.21.15.52:0/2917439285 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:30.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.728+0000 7f07d8fce640 1 --2- 172.21.15.52:0/2917439285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a8467280c0 0x55a84672da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.728+0000 7f07d97cf640 1 -- 172.21.15.52:0/2917439285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a846730ca0 msgr2=0x55a84672e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.728+0000 7f07d97cf640 1 --2- 172.21.15.52:0/2917439285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a846730ca0 0x55a84672e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.728+0000 7f07d97cf640 1 -- 172.21.15.52:0/2917439285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a8467280c0 msgr2=0x55a84672da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.728+0000 7f07d97cf640 1 --2- 172.21.15.52:0/2917439285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a8467280c0 0x55a84672da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.728+0000 7f07d97cf640 1 -- 172.21.15.52:0/2917439285 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8464697d0 con 0x55a8467277a0 2024-04-02T13:58:30.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.728+0000 7f07d8fce640 1 --2- 172.21.15.52:0/2917439285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a8467280c0 0x55a84672da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:58:30.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.729+0000 7f07d97cf640 1 --2- 172.21.15.52:0/2917439285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467277a0 0x55a846730690 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f07d0005910 tx=0x7f07d00129e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17b5e924f11c5a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.729+0000 7f07bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:58:30.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.729+0000 7f07bcff9640 1 -- 172.21.15.52:0/2917439285 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0028020 con 0x55a8467277a0 2024-04-02T13:58:30.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.729+0000 7f07bcff9640 1 -- 172.21.15.52:0/2917439285 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f07d000fb00 con 0x55a8467277a0 2024-04-02T13:58:30.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.729+0000 7f07de616740 1 -- 172.21.15.52:0/2917439285 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07d0012ad0 con 0x55a8467277a0 2024-04-02T13:58:30.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.729+0000 7f07bcff9640 1 -- 172.21.15.52:0/2917439285 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f07d0033050 con 0x55a8467277a0 2024-04-02T13:58:30.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.729+0000 7f07de616740 10 client.17100 fetch_fsmap learned FSMap version 30 2024-04-02T13:58:30.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.729+0000 7f07de616740 1 -- 172.21.15.52:0/2917439285 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a84673fa80 con 0x55a8467277a0 2024-04-02T13:58:30.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.729+0000 7f07bcff9640 1 -- 172.21.15.52:0/2917439285 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07d0012ad0 con 0x55a8467277a0 2024-04-02T13:58:30.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.729+0000 7f07bcff9640 1 -- 172.21.15.52:0/2917439285 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f07d0004920 con 0x55a8467277a0 2024-04-02T13:58:30.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.729+0000 7f07de616740 10 client.17100 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:58:30.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.729+0000 7f07de616740 10 client.17100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:58:30.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.729+0000 7f07de616740 20 client.17100 populate_metadata read hostname 'smithi052' 2024-04-02T13:58:30.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.729+0000 7f07de616740 1 --2- 172.21.15.52:0/2917439285 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55a846740f40 0x55a846743330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.729+0000 7f07de616740 4 client.17100 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:58:30.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.729+0000 7f07de616740 1 -- 172.21.15.52:0/2917439285 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a846698cf0 con 0x55a846740f40 2024-04-02T13:58:30.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.730+0000 7f07d8fce640 1 --2- 172.21.15.52:0/2917439285 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55a846740f40 0x55a846743330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.730+0000 7f07bcff9640 10 client.17100 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:58:30.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.730+0000 7f07d8fce640 1 --2- 172.21.15.52:0/2917439285 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55a846740f40 0x55a846743330 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5218efd07dee5e94 server_cookie=93ab6a3f66ed97df in_seq=0 out_seq=0 2024-04-02T13:58:30.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.731+0000 7f07bcff9640 1 -- 172.21.15.52:0/2917439285 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a846698cf0 con 0x55a846740f40 2024-04-02T13:58:30.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.731+0000 7f07bcff9640 10 client.17100 handle_command_reply: tid=0 2024-04-02T13:58:30.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.731+0000 7f07de616740 1 client.17100 shutdown 2024-04-02T13:58:30.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.731+0000 7f07de616740 1 -- 172.21.15.52:0/2917439285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467277a0 msgr2=0x55a846730690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.731+0000 7f07de616740 1 --2- 172.21.15.52:0/2917439285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467277a0 0x55a846730690 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f07d0005910 tx=0x7f07d00129e0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.731+0000 7f07de616740 1 -- 172.21.15.52:0/2917439285 shutdown_connections 2024-04-02T13:58:30.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.731+0000 7f07de616740 1 --2- 172.21.15.52:0/2917439285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a846730ca0 0x55a84672e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.731+0000 7f07de616740 1 --2- 172.21.15.52:0/2917439285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a8467280c0 0x55a84672da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.731+0000 7f07de616740 1 --2- 172.21.15.52:0/2917439285 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55a846740f40 0x55a846743330 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.731+0000 7f07de616740 1 --2- 172.21.15.52:0/2917439285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467277a0 0x55a846730690 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.731+0000 7f07de616740 1 -- 172.21.15.52:0/2917439285 >> 172.21.15.52:0/2917439285 conn(0x55a846721ed0 msgr2=0x55a846723c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.731+0000 7f07bcff9640 0 client.17100 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:58:30.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.731+0000 7f07de616740 1 -- 172.21.15.52:0/2917439285 shutdown_connections 2024-04-02T13:58:30.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.731+0000 7f07de616740 1 -- 172.21.15.52:0/2917439285 wait complete. 2024-04-02T13:58:30.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.731+0000 7f07de616740 20 client.17100 trim_cache size 0 max 16384 2024-04-02T13:58:30.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.745+0000 7f07de616740 1 Processor -- start 2024-04-02T13:58:30.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.745+0000 7f07de616740 1 -- start start 2024-04-02T13:58:30.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.745+0000 7f07de616740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a846721ed0 0x55a84673c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.745+0000 7f07de616740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467277a0 0x55a84672cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.745+0000 7f07de616740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8467280c0 0x55a846730150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.745+0000 7f07de616740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f07d41aeea0 con 0x55a8467277a0 2024-04-02T13:58:30.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.745+0000 7f07de616740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f07d41b09e0 con 0x55a846721ed0 2024-04-02T13:58:30.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.745+0000 7f07de616740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f07d41af570 con 0x55a8467280c0 2024-04-02T13:58:30.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.745+0000 7f07d8fce640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467277a0 0x55a84672cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.745+0000 7f07d8fce640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467277a0 0x55a84672cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53872/0 (socket says 172.21.15.52:53872) 2024-04-02T13:58:30.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.745+0000 7f07d8fce640 1 -- 172.21.15.52:0/2991961066 learned_addr learned my addr 172.21.15.52:0/2991961066 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:30.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.745+0000 7f07d97cf640 1 --2- 172.21.15.52:0/2991961066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a846721ed0 0x55a84673c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.745+0000 7f07d9fd0640 1 --2- 172.21.15.52:0/2991961066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8467280c0 0x55a846730150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.745+0000 7f07d8fce640 1 -- 172.21.15.52:0/2991961066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8467280c0 msgr2=0x55a846730150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.745+0000 7f07d8fce640 1 --2- 172.21.15.52:0/2991961066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8467280c0 0x55a846730150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.745+0000 7f07d8fce640 1 -- 172.21.15.52:0/2991961066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a846721ed0 msgr2=0x55a84673c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.745+0000 7f07d8fce640 1 --2- 172.21.15.52:0/2991961066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a846721ed0 0x55a84673c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.745+0000 7f07d8fce640 1 -- 172.21.15.52:0/2991961066 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a84672e3a0 con 0x55a8467277a0 2024-04-02T13:58:30.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.745+0000 7f07d9fd0640 1 --2- 172.21.15.52:0/2991961066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8467280c0 0x55a846730150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:58:30.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.745+0000 7f07d8fce640 1 --2- 172.21.15.52:0/2991961066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467277a0 0x55a84672cb80 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f07c401b000 tx=0x7f07c4020350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169d77679d9ed344 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.745+0000 7f07beffd640 1 -- 172.21.15.52:0/2991961066 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c40a0560 con 0x55a8467277a0 2024-04-02T13:58:30.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.745+0000 7f07beffd640 1 -- 172.21.15.52:0/2991961066 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07c40a4e20 con 0x55a8467277a0 2024-04-02T13:58:30.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.745+0000 7f07beffd640 1 -- 172.21.15.52:0/2991961066 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c40cdbe0 con 0x55a8467277a0 2024-04-02T13:58:30.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.746+0000 7f07de616740 1 -- 172.21.15.52:0/2991961066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467277a0 msgr2=0x55a84672cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.746+0000 7f07de616740 1 --2- 172.21.15.52:0/2991961066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467277a0 0x55a84672cb80 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f07c401b000 tx=0x7f07c4020350 comp rx=0 tx=0).stop 2024-04-02T13:58:30.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.746+0000 7f07de616740 1 -- 172.21.15.52:0/2991961066 shutdown_connections 2024-04-02T13:58:30.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.746+0000 7f07de616740 1 --2- 172.21.15.52:0/2991961066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8467280c0 0x55a846730150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.746+0000 7f07de616740 1 --2- 172.21.15.52:0/2991961066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467277a0 0x55a84672cb80 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.746+0000 7f07de616740 1 --2- 172.21.15.52:0/2991961066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a846721ed0 0x55a84673c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.746+0000 7f07de616740 1 -- 172.21.15.52:0/2991961066 >> 172.21.15.52:0/2991961066 conn(0x55a8467333b0 msgr2=0x55a8467222b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.746+0000 7f07de616740 1 -- 172.21.15.52:0/2991961066 shutdown_connections 2024-04-02T13:58:30.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.746+0000 7f07de616740 1 -- 172.21.15.52:0/2991961066 wait complete. 2024-04-02T13:58:30.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.746+0000 7f07de616740 1 -- start start 2024-04-02T13:58:30.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.746+0000 7f07de616740 1 Processor -- start 2024-04-02T13:58:30.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.747+0000 7f07de616740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a846721ed0 0x55a846731a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.747+0000 7f07de616740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467277a0 0x55a846731fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.747+0000 7f07de616740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a8467280c0 0x55a846737e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.747+0000 7f07de616740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f07d41b0310 con 0x55a8467277a0 2024-04-02T13:58:30.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.747+0000 7f07de616740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f07d41afc40 con 0x55a8467280c0 2024-04-02T13:58:30.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.747+0000 7f07de616740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f07d41ace80 con 0x55a846721ed0 2024-04-02T13:58:30.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.747+0000 7f07de616740 10 client.0 _ll_register_callbacks cb 0x7f07d0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:58:30.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.747+0000 7f07d8fce640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467277a0 0x55a846731fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.747+0000 7f07d8fce640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467277a0 0x55a846731fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53878/0 (socket says 172.21.15.52:53878) 2024-04-02T13:58:30.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.747+0000 7f07d8fce640 1 -- 172.21.15.52:0/1484878779 learned_addr learned my addr 172.21.15.52:0/1484878779 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:30.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.747+0000 7f07d9fd0640 1 --2- 172.21.15.52:0/1484878779 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a8467280c0 0x55a846737e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.747+0000 7f07d97cf640 1 --2- 172.21.15.52:0/1484878779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a846721ed0 0x55a846731a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.747+0000 7f07d8fce640 1 -- 172.21.15.52:0/1484878779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a846721ed0 msgr2=0x55a846731a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.747+0000 7f07d8fce640 1 --2- 172.21.15.52:0/1484878779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a846721ed0 0x55a846731a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.747+0000 7f07d8fce640 1 -- 172.21.15.52:0/1484878779 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a8467280c0 msgr2=0x55a846737e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:30.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.747+0000 7f07d8fce640 1 --2- 172.21.15.52:0/1484878779 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a8467280c0 0x55a846737e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.747+0000 7f07d8fce640 1 -- 172.21.15.52:0/1484878779 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8467383a0 con 0x55a8467277a0 2024-04-02T13:58:30.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.747+0000 7f07d9fd0640 1 --2- 172.21.15.52:0/1484878779 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a8467280c0 0x55a846737e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:58:30.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.747+0000 7f07d8fce640 1 --2- 172.21.15.52:0/1484878779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467277a0 0x55a846731fa0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f07c409beb0 tx=0x7f07c40a05d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74178164960393e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:30.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.747+0000 7f07bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:58:30.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.747+0000 7f07bcff9640 1 -- 172.21.15.52:0/1484878779 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c409a250 con 0x55a8467277a0 2024-04-02T13:58:30.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.747+0000 7f07bcff9640 1 -- 172.21.15.52:0/1484878779 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f07c409a4b0 con 0x55a8467277a0 2024-04-02T13:58:30.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.747+0000 7f07de616740 1 -- 172.21.15.52:0/1484878779 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07c40a4e20 con 0x55a8467277a0 2024-04-02T13:58:30.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.747+0000 7f07bcff9640 1 -- 172.21.15.52:0/1484878779 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f07c4098820 con 0x55a8467277a0 2024-04-02T13:58:30.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.747+0000 7f07de616740 10 client.17106 fetch_fsmap learned FSMap version 30 2024-04-02T13:58:30.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.748+0000 7f07de616740 1 -- 172.21.15.52:0/1484878779 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a8467395f0 con 0x55a8467277a0 2024-04-02T13:58:30.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.748+0000 7f07bcff9640 1 -- 172.21.15.52:0/1484878779 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07c40a4e20 con 0x55a8467277a0 2024-04-02T13:58:30.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.748+0000 7f07bcff9640 1 -- 172.21.15.52:0/1484878779 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f07c409b600 con 0x55a8467277a0 2024-04-02T13:58:30.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.748+0000 7f07de616740 10 client.17106 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:58:30.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.748+0000 7f07de616740 10 client.17106 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:58:30.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.748+0000 7f07de616740 20 client.17106 populate_metadata read hostname 'smithi052' 2024-04-02T13:58:30.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.748+0000 7f07de616740 1 --2- 172.21.15.52:0/1484878779 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55a846730ca0 0x55a8467adac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:30.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.748+0000 7f07de616740 4 client.17106 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:58:30.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.748+0000 7f07de616740 1 -- 172.21.15.52:0/1484878779 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a846693040 con 0x55a846730ca0 2024-04-02T13:58:30.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.748+0000 7f07d97cf640 1 --2- 172.21.15.52:0/1484878779 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55a846730ca0 0x55a8467adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:30.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.748+0000 7f07bcff9640 10 client.17106 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:58:30.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.748+0000 7f07d97cf640 1 --2- 172.21.15.52:0/1484878779 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55a846730ca0 0x55a8467adac0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dcd8e36cd33ae4f server_cookie=dd2b8cf1b7ba074f in_seq=0 out_seq=0 2024-04-02T13:58:30.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.749+0000 7f07bcff9640 1 -- 172.21.15.52:0/1484878779 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a846693040 con 0x55a846730ca0 2024-04-02T13:58:30.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.749+0000 7f07bcff9640 10 client.17106 handle_command_reply: tid=0 2024-04-02T13:58:30.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.749+0000 7f07de616740 1 client.17106 shutdown 2024-04-02T13:58:30.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.749+0000 7f07de616740 1 -- 172.21.15.52:0/1484878779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467277a0 msgr2=0x55a846731fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.749+0000 7f07de616740 1 --2- 172.21.15.52:0/1484878779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467277a0 0x55a846731fa0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f07c409beb0 tx=0x7f07c40a05d0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.749+0000 7f07de616740 1 -- 172.21.15.52:0/1484878779 shutdown_connections 2024-04-02T13:58:30.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.749+0000 7f07de616740 1 --2- 172.21.15.52:0/1484878779 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a8467280c0 0x55a846737e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.749+0000 7f07de616740 1 --2- 172.21.15.52:0/1484878779 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55a846730ca0 0x55a8467adac0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.749+0000 7f07de616740 1 --2- 172.21.15.52:0/1484878779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a8467277a0 0x55a846731fa0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.749+0000 7f07de616740 1 --2- 172.21.15.52:0/1484878779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a846721ed0 0x55a846731a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.749+0000 7f07bcff9640 0 client.17106 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:58:30.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.749+0000 7f07de616740 1 -- 172.21.15.52:0/1484878779 >> 172.21.15.52:0/1484878779 conn(0x55a8467333b0 msgr2=0x55a846720f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.749+0000 7f07de616740 1 -- 172.21.15.52:0/1484878779 shutdown_connections 2024-04-02T13:58:30.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.749+0000 7f07de616740 1 -- 172.21.15.52:0/1484878779 wait complete. 2024-04-02T13:58:30.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.749+0000 7f07de616740 20 client.17106 trim_cache size 0 max 16384 2024-04-02T13:58:30.772 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:58:30.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.749+0000 7f07dba33640 1 -- 172.21.15.52:0/2665673638 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f07c00a3d20 msgr2=0x7f07c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.749+0000 7f07dba33640 1 --2- 172.21.15.52:0/2665673638 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f07c00a3d20 0x7f07c00a61e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f07cc005cc0 tx=0x7f07cc02e040 comp rx=0 tx=0).stop 2024-04-02T13:58:30.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.750+0000 7f07dba33640 1 -- 172.21.15.52:0/2665673638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d4101330 msgr2=0x7f07d41988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:30.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.750+0000 7f07dba33640 1 --2- 172.21.15.52:0/2665673638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d4101330 0x7f07d41988b0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f07c400d950 tx=0x7f07c400de10 comp rx=0 tx=0).stop 2024-04-02T13:58:30.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.750+0000 7f07dba33640 1 -- 172.21.15.52:0/2665673638 shutdown_connections 2024-04-02T13:58:30.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.750+0000 7f07dba33640 1 --2- 172.21.15.52:0/2665673638 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f07c00a3d20 0x7f07c00a61e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.750+0000 7f07dba33640 1 --2- 172.21.15.52:0/2665673638 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07d4106980 0x7f07d419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.750+0000 7f07dba33640 1 --2- 172.21.15.52:0/2665673638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07d4101330 0x7f07d41988b0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.750+0000 7f07dba33640 1 --2- 172.21.15.52:0/2665673638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07d4100970 0x7f07d4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:30.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.750+0000 7f07dba33640 1 -- 172.21.15.52:0/2665673638 >> 172.21.15.52:0/2665673638 conn(0x7f07d4077b00 msgr2=0x7f07d4103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:30.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.750+0000 7f07dba33640 1 -- 172.21.15.52:0/2665673638 shutdown_connections 2024-04-02T13:58:30.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:30.750+0000 7f07dba33640 1 -- 172.21.15.52:0/2665673638 wait complete. 2024-04-02T13:58:31.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:31 smithi088 ceph-mon[36528]: pgmap v153: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:31.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:31 smithi189 ceph-mon[36683]: pgmap v153: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:31.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:31 smithi052 ceph-mon[32104]: pgmap v153: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:31.778 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:58:31.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:58:31.779 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:58:31.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.894+0000 7f14aa63c640 1 -- 172.21.15.52:0/3423778701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14a4101020 msgr2=0x7f14a4106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:31.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.894+0000 7f14aa63c640 1 --2- 172.21.15.52:0/3423778701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14a4101020 0x7f14a4106750 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f1498009900 tx=0x7f149802bc30 comp rx=0 tx=0).stop 2024-04-02T13:58:31.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.894+0000 7f14aa63c640 1 -- 172.21.15.52:0/3423778701 shutdown_connections 2024-04-02T13:58:31.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.894+0000 7f14aa63c640 1 --2- 172.21.15.52:0/3423778701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14a4106e30 0x7f14a410ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.894+0000 7f14aa63c640 1 --2- 172.21.15.52:0/3423778701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14a4101020 0x7f14a4106750 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.894+0000 7f14aa63c640 1 --2- 172.21.15.52:0/3423778701 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14a4100670 0x7f14a4100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.894+0000 7f14aa63c640 1 -- 172.21.15.52:0/3423778701 >> 172.21.15.52:0/3423778701 conn(0x7f14a4075c60 msgr2=0x7f14a4076070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:31.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.894+0000 7f14aa63c640 1 -- 172.21.15.52:0/3423778701 shutdown_connections 2024-04-02T13:58:31.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.894+0000 7f14aa63c640 1 -- 172.21.15.52:0/3423778701 wait complete. 2024-04-02T13:58:31.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.895+0000 7f14aa63c640 1 Processor -- start 2024-04-02T13:58:31.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.895+0000 7f14aa63c640 1 -- start start 2024-04-02T13:58:31.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.895+0000 7f14aa63c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14a4100670 0x7f14a4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:31.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.895+0000 7f14aa63c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14a4106e30 0x7f14a41989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:31.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.895+0000 7f14aa63c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14a419cd20 0x7f14a419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:31.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.895+0000 7f14aa63c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f14a4111f20 con 0x7f14a4106e30 2024-04-02T13:58:31.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.895+0000 7f14aa63c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f14a4111da0 con 0x7f14a419cd20 2024-04-02T13:58:31.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.895+0000 7f14aa63c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f14a41120a0 con 0x7f14a4100670 2024-04-02T13:58:31.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.895+0000 7f14a37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14a4106e30 0x7f14a41989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:31.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.895+0000 7f14a37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14a4106e30 0x7f14a41989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53026/0 (socket says 172.21.15.52:53026) 2024-04-02T13:58:31.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.895+0000 7f14a37fe640 1 -- 172.21.15.52:0/2109659347 learned_addr learned my addr 172.21.15.52:0/2109659347 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:31.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.896+0000 7f14a3fff640 1 --2- 172.21.15.52:0/2109659347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14a4100670 0x7f14a4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:31.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.896+0000 7f14a8bd9640 1 --2- 172.21.15.52:0/2109659347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14a419cd20 0x7f14a419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:31.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.896+0000 7f14a37fe640 1 -- 172.21.15.52:0/2109659347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14a4100670 msgr2=0x7f14a4198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:31.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.896+0000 7f14a37fe640 1 --2- 172.21.15.52:0/2109659347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14a4100670 0x7f14a4198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.896+0000 7f14a37fe640 1 -- 172.21.15.52:0/2109659347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14a419cd20 msgr2=0x7f14a419d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:31.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.896+0000 7f14a37fe640 1 --2- 172.21.15.52:0/2109659347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14a419cd20 0x7f14a419d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.896+0000 7f14a37fe640 1 -- 172.21.15.52:0/2109659347 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a419d810 con 0x7f14a4106e30 2024-04-02T13:58:31.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.896+0000 7f14a8bd9640 1 --2- 172.21.15.52:0/2109659347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14a419cd20 0x7f14a419d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:58:31.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.896+0000 7f14a37fe640 1 --2- 172.21.15.52:0/2109659347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14a4106e30 0x7f14a41989a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f1498036040 tx=0x7f149802ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:31.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.896+0000 7f14a17fa640 1 -- 172.21.15.52:0/2109659347 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1498041070 con 0x7f14a4106e30 2024-04-02T13:58:31.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.896+0000 7f14a17fa640 1 -- 172.21.15.52:0/2109659347 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14980054e0 con 0x7f14a4106e30 2024-04-02T13:58:31.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.896+0000 7f14aa63c640 1 -- 172.21.15.52:0/2109659347 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a419daa0 con 0x7f14a4106e30 2024-04-02T13:58:31.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.896+0000 7f14a17fa640 1 -- 172.21.15.52:0/2109659347 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1498034440 con 0x7f14a4106e30 2024-04-02T13:58:31.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.896+0000 7f14aa63c640 1 -- 172.21.15.52:0/2109659347 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a41a5260 con 0x7f14a4106e30 2024-04-02T13:58:31.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.899+0000 7f14a17fa640 1 -- 172.21.15.52:0/2109659347 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1498044070 con 0x7f14a4106e30 2024-04-02T13:58:31.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.899+0000 7f14ad223740 1 Processor -- start 2024-04-02T13:58:31.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.899+0000 7f14ad223740 1 -- start start 2024-04-02T13:58:31.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.899+0000 7f14a17fa640 1 --2- 172.21.15.52:0/2109659347 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f148c0a3da0 0x7f148c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:31.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.899+0000 7f14ad223740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0882c0 0x55965a0886a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:31.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.899+0000 7f14ad223740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55965a088be0 0x55965a091280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:31.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.899+0000 7f14ad223740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55965a0917c0 0x55965a093bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:31.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.899+0000 7f14ad223740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559659fec770 con 0x55965a0882c0 2024-04-02T13:58:31.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.899+0000 7f14ad223740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559659f94960 con 0x55965a0917c0 2024-04-02T13:58:31.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.899+0000 7f14ad223740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559659fe88c0 con 0x55965a088be0 2024-04-02T13:58:31.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.899+0000 7f14a8bd9640 1 --2- 172.21.15.52:0/2109659347 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f148c0a3da0 0x7f148c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:31.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.899+0000 7f14a17fa640 1 -- 172.21.15.52:0/2109659347 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f14980f2680 con 0x7f14a4106e30 2024-04-02T13:58:31.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.899+0000 7f14a3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0882c0 0x55965a0886a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:31.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.899+0000 7f14a3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0882c0 0x55965a0886a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53040/0 (socket says 172.21.15.52:53040) 2024-04-02T13:58:31.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.899+0000 7f14a3fff640 1 -- 172.21.15.52:0/640541159 learned_addr learned my addr 172.21.15.52:0/640541159 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:31.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.900+0000 7f14a37fe640 1 --2- 172.21.15.52:0/640541159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55965a088be0 0x55965a091280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:31.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.900+0000 7f14a3fff640 1 -- 172.21.15.52:0/640541159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55965a088be0 msgr2=0x55965a091280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:31.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.900+0000 7f14a3fff640 1 --2- 172.21.15.52:0/640541159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55965a088be0 0x55965a091280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.900+0000 7f14a3fff640 1 -- 172.21.15.52:0/640541159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55965a0917c0 msgr2=0x55965a093bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:31.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.900+0000 7f14a3fff640 1 --2- 172.21.15.52:0/640541159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55965a0917c0 0x55965a093bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.900+0000 7f14a3fff640 1 -- 172.21.15.52:0/640541159 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559659dc87e0 con 0x55965a0882c0 2024-04-02T13:58:31.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.900+0000 7f14a8bd9640 1 --2- 172.21.15.52:0/2109659347 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f148c0a3da0 0x7f148c0a6260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f149c005fd0 tx=0x7f149c008730 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:31.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.900+0000 7f14a3fff640 1 --2- 172.21.15.52:0/640541159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0882c0 0x55965a0886a0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f149400bcb0 tx=0x7f149400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b0206e5384ffef server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:31.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.900+0000 7f148a7fc640 1 -- 172.21.15.52:0/640541159 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494015070 con 0x55965a0882c0 2024-04-02T13:58:31.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.900+0000 7f148a7fc640 1 -- 172.21.15.52:0/640541159 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f149401a940 con 0x55965a0882c0 2024-04-02T13:58:31.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.900+0000 7f148a7fc640 1 -- 172.21.15.52:0/640541159 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14940119e0 con 0x55965a0882c0 2024-04-02T13:58:31.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.901+0000 7f14ad223740 1 -- 172.21.15.52:0/640541159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0882c0 msgr2=0x55965a0886a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:31.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.901+0000 7f14ad223740 1 --2- 172.21.15.52:0/640541159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0882c0 0x55965a0886a0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f149400bcb0 tx=0x7f149400f730 comp rx=0 tx=0).stop 2024-04-02T13:58:31.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.901+0000 7f14ad223740 1 -- 172.21.15.52:0/640541159 shutdown_connections 2024-04-02T13:58:31.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.901+0000 7f14ad223740 1 --2- 172.21.15.52:0/640541159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55965a0917c0 0x55965a093bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.901+0000 7f14ad223740 1 --2- 172.21.15.52:0/640541159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55965a088be0 0x55965a091280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.901+0000 7f14ad223740 1 --2- 172.21.15.52:0/640541159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0882c0 0x55965a0886a0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.901+0000 7f14ad223740 1 -- 172.21.15.52:0/640541159 >> 172.21.15.52:0/640541159 conn(0x55965a0829f0 msgr2=0x55965a082dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:31.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.901+0000 7f14ad223740 1 -- 172.21.15.52:0/640541159 shutdown_connections 2024-04-02T13:58:31.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.901+0000 7f14ad223740 1 -- 172.21.15.52:0/640541159 wait complete. 2024-04-02T13:58:31.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.901+0000 7f14ad223740 1 -- start start 2024-04-02T13:58:31.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.901+0000 7f14ad223740 1 Processor -- start 2024-04-02T13:58:31.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.902+0000 7f14ad223740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55965a0882c0 0x55965a08e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:31.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.902+0000 7f14ad223740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55965a088be0 0x55965a08eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:31.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.902+0000 7f14ad223740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0917c0 0x55965a08f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:31.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.902+0000 7f14ad223740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55965a0945b0 con 0x55965a0917c0 2024-04-02T13:58:31.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.902+0000 7f14ad223740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559659fa7d20 con 0x55965a088be0 2024-04-02T13:58:31.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.902+0000 7f14ad223740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55965a094730 con 0x55965a0882c0 2024-04-02T13:58:31.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.902+0000 7f14ad223740 10 client.0 _ll_register_callbacks cb 0x559659f74a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:58:31.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.902+0000 7f14a8bd9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0917c0 0x55965a08f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:31.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.902+0000 7f14a8bd9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0917c0 0x55965a08f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53042/0 (socket says 172.21.15.52:53042) 2024-04-02T13:58:31.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.902+0000 7f14a8bd9640 1 -- 172.21.15.52:0/101679912 learned_addr learned my addr 172.21.15.52:0/101679912 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:31.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.902+0000 7f14a37fe640 1 --2- 172.21.15.52:0/101679912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55965a088be0 0x55965a08eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:31.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.902+0000 7f14a3fff640 1 --2- 172.21.15.52:0/101679912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55965a0882c0 0x55965a08e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:31.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.902+0000 7f14a8bd9640 1 -- 172.21.15.52:0/101679912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55965a0882c0 msgr2=0x55965a08e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:31.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.902+0000 7f14a8bd9640 1 --2- 172.21.15.52:0/101679912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55965a0882c0 0x55965a08e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.902+0000 7f14a8bd9640 1 -- 172.21.15.52:0/101679912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55965a088be0 msgr2=0x55965a08eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:31.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.902+0000 7f14a8bd9640 1 --2- 172.21.15.52:0/101679912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55965a088be0 0x55965a08eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.902+0000 7f14a8bd9640 1 -- 172.21.15.52:0/101679912 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559659e28f70 con 0x55965a0917c0 2024-04-02T13:58:31.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.902+0000 7f14a37fe640 1 --2- 172.21.15.52:0/101679912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55965a088be0 0x55965a08eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:58:31.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.902+0000 7f14a8bd9640 1 --2- 172.21.15.52:0/101679912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0917c0 0x55965a08f1e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f149c011df0 tx=0x7f149c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475a5e0b5e401ff7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:31.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.902+0000 7f14837fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:58:31.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.902+0000 7f14837fe640 1 -- 172.21.15.52:0/101679912 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c022020 con 0x55965a0917c0 2024-04-02T13:58:31.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.902+0000 7f14837fe640 1 -- 172.21.15.52:0/101679912 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f149c01e450 con 0x55965a0917c0 2024-04-02T13:58:31.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.902+0000 7f14ad223740 1 -- 172.21.15.52:0/101679912 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f149401a940 con 0x55965a0917c0 2024-04-02T13:58:31.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.903+0000 7f14837fe640 1 -- 172.21.15.52:0/101679912 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f149c02d050 con 0x55965a0917c0 2024-04-02T13:58:31.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.903+0000 7f14ad223740 10 client.17118 fetch_fsmap learned FSMap version 30 2024-04-02T13:58:31.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.903+0000 7f14ad223740 1 -- 172.21.15.52:0/101679912 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559659cc0170 con 0x55965a0917c0 2024-04-02T13:58:31.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.903+0000 7f14837fe640 1 -- 172.21.15.52:0/101679912 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f149401a940 con 0x55965a0917c0 2024-04-02T13:58:31.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.903+0000 7f14837fe640 1 -- 172.21.15.52:0/101679912 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f149c01da60 con 0x55965a0917c0 2024-04-02T13:58:31.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.903+0000 7f14ad223740 10 client.17118 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:58:31.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.903+0000 7f14ad223740 10 client.17118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:58:31.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.903+0000 7f14ad223740 20 client.17118 populate_metadata read hostname 'smithi052' 2024-04-02T13:58:31.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.903+0000 7f14ad223740 1 --2- 172.21.15.52:0/101679912 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55965a0a1bd0 0x55965a0a4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:31.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.903+0000 7f14ad223740 4 client.17118 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:58:31.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.903+0000 7f14ad223740 1 -- 172.21.15.52:0/101679912 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559659ff9cb0 con 0x55965a0a1bd0 2024-04-02T13:58:31.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.903+0000 7f14a3fff640 1 --2- 172.21.15.52:0/101679912 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55965a0a1bd0 0x55965a0a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:31.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.904+0000 7f14a3fff640 1 --2- 172.21.15.52:0/101679912 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55965a0a1bd0 0x55965a0a4000 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23eaca7607799040 server_cookie=befe310d4896e781 in_seq=0 out_seq=0 2024-04-02T13:58:31.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.904+0000 7f14837fe640 10 client.17118 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:58:31.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.904+0000 7f14837fe640 1 -- 172.21.15.52:0/101679912 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559659ff9cb0 con 0x55965a0a1bd0 2024-04-02T13:58:31.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.904+0000 7f14837fe640 10 client.17118 handle_command_reply: tid=0 2024-04-02T13:58:31.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.905+0000 7f14ad223740 1 client.17118 shutdown 2024-04-02T13:58:31.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.905+0000 7f14ad223740 1 -- 172.21.15.52:0/101679912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0917c0 msgr2=0x55965a08f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:31.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.905+0000 7f14ad223740 1 --2- 172.21.15.52:0/101679912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0917c0 0x55965a08f1e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f149c011df0 tx=0x7f149c004a40 comp rx=0 tx=0).stop 2024-04-02T13:58:31.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.905+0000 7f14ad223740 1 -- 172.21.15.52:0/101679912 shutdown_connections 2024-04-02T13:58:31.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.905+0000 7f14ad223740 1 --2- 172.21.15.52:0/101679912 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55965a0a1bd0 0x55965a0a4000 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.905+0000 7f14ad223740 1 --2- 172.21.15.52:0/101679912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0917c0 0x55965a08f1e0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.905+0000 7f14ad223740 1 --2- 172.21.15.52:0/101679912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55965a088be0 0x55965a08eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.905+0000 7f14837fe640 0 client.17118 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:58:31.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.905+0000 7f14ad223740 1 --2- 172.21.15.52:0/101679912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55965a0882c0 0x55965a08e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.905+0000 7f14ad223740 1 -- 172.21.15.52:0/101679912 >> 172.21.15.52:0/101679912 conn(0x55965a0829f0 msgr2=0x55965a084740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:31.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.905+0000 7f14ad223740 1 -- 172.21.15.52:0/101679912 shutdown_connections 2024-04-02T13:58:31.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.905+0000 7f14ad223740 1 -- 172.21.15.52:0/101679912 wait complete. 2024-04-02T13:58:31.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.905+0000 7f14ad223740 20 client.17118 trim_cache size 0 max 16384 2024-04-02T13:58:31.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.923+0000 7f14ad223740 1 Processor -- start 2024-04-02T13:58:31.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.923+0000 7f14ad223740 1 -- start start 2024-04-02T13:58:31.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.923+0000 7f14ad223740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0882c0 0x55965a090e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:31.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.923+0000 7f14ad223740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55965a088be0 0x55965a09d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:31.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.923+0000 7f14ad223740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55965a0917c0 0x55965a0a98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:31.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.923+0000 7f14ad223740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f14a41aee00 con 0x55965a0882c0 2024-04-02T13:58:31.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.923+0000 7f14ad223740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f14a41b0940 con 0x55965a088be0 2024-04-02T13:58:31.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.923+0000 7f14ad223740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f14a41af4d0 con 0x55965a0917c0 2024-04-02T13:58:31.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.923+0000 7f14a3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0882c0 0x55965a090e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:31.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.923+0000 7f14a3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0882c0 0x55965a090e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53048/0 (socket says 172.21.15.52:53048) 2024-04-02T13:58:31.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.923+0000 7f14a3fff640 1 -- 172.21.15.52:0/4057661808 learned_addr learned my addr 172.21.15.52:0/4057661808 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:31.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.923+0000 7f14a37fe640 1 --2- 172.21.15.52:0/4057661808 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55965a088be0 0x55965a09d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:31.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.923+0000 7f14a3fff640 1 -- 172.21.15.52:0/4057661808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55965a0917c0 msgr2=0x55965a0a98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:31.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.923+0000 7f14a3fff640 1 --2- 172.21.15.52:0/4057661808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55965a0917c0 0x55965a0a98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.923+0000 7f14a3fff640 1 -- 172.21.15.52:0/4057661808 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55965a088be0 msgr2=0x55965a09d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:31.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.923+0000 7f14a3fff640 1 --2- 172.21.15.52:0/4057661808 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55965a088be0 0x55965a09d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.923+0000 7f14a3fff640 1 -- 172.21.15.52:0/4057661808 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55965a08d710 con 0x55965a0882c0 2024-04-02T13:58:31.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.923+0000 7f14a37fe640 1 --2- 172.21.15.52:0/4057661808 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55965a088be0 0x55965a09d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:58:31.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.923+0000 7f14a3fff640 1 --2- 172.21.15.52:0/4057661808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0882c0 0x55965a090e90 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f14940114b0 tx=0x7f1494011620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e815f53a534e83 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:31.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.923+0000 7f1489ffb640 1 -- 172.21.15.52:0/4057661808 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494015040 con 0x55965a0882c0 2024-04-02T13:58:31.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.924+0000 7f1489ffb640 1 -- 172.21.15.52:0/4057661808 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1494005580 con 0x55965a0882c0 2024-04-02T13:58:31.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.924+0000 7f1489ffb640 1 -- 172.21.15.52:0/4057661808 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14940118f0 con 0x55965a0882c0 2024-04-02T13:58:31.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.924+0000 7f14ad223740 1 -- 172.21.15.52:0/4057661808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0882c0 msgr2=0x55965a090e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:31.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.924+0000 7f14ad223740 1 --2- 172.21.15.52:0/4057661808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0882c0 0x55965a090e90 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f14940114b0 tx=0x7f1494011620 comp rx=0 tx=0).stop 2024-04-02T13:58:31.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.924+0000 7f14ad223740 1 -- 172.21.15.52:0/4057661808 shutdown_connections 2024-04-02T13:58:31.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.924+0000 7f14ad223740 1 --2- 172.21.15.52:0/4057661808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55965a0917c0 0x55965a0a98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.924+0000 7f14ad223740 1 --2- 172.21.15.52:0/4057661808 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55965a088be0 0x55965a09d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.924+0000 7f14ad223740 1 --2- 172.21.15.52:0/4057661808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0882c0 0x55965a090e90 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.924+0000 7f14ad223740 1 -- 172.21.15.52:0/4057661808 >> 172.21.15.52:0/4057661808 conn(0x55965a0829f0 msgr2=0x55965a08efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:31.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.924+0000 7f14ad223740 1 -- 172.21.15.52:0/4057661808 shutdown_connections 2024-04-02T13:58:31.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.924+0000 7f14ad223740 1 -- 172.21.15.52:0/4057661808 wait complete. 2024-04-02T13:58:31.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.924+0000 7f14ad223740 1 -- start start 2024-04-02T13:58:31.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.925+0000 7f14ad223740 1 Processor -- start 2024-04-02T13:58:31.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.925+0000 7f14ad223740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0882c0 0x55965a099da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:31.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.925+0000 7f14ad223740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55965a088be0 0x55965a096120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:31.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.925+0000 7f14ad223740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55965a0917c0 0x55965a096660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:31.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.925+0000 7f14ad223740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f14a41b0270 con 0x55965a0882c0 2024-04-02T13:58:31.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.925+0000 7f14ad223740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f14a41afba0 con 0x55965a0917c0 2024-04-02T13:58:31.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.925+0000 7f14ad223740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f14a41acde0 con 0x55965a088be0 2024-04-02T13:58:31.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.925+0000 7f14ad223740 10 client.0 _ll_register_callbacks cb 0x7f149c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:58:31.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.925+0000 7f14a3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0882c0 0x55965a099da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:31.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.925+0000 7f14a3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0882c0 0x55965a099da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53062/0 (socket says 172.21.15.52:53062) 2024-04-02T13:58:31.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.925+0000 7f14a3fff640 1 -- 172.21.15.52:0/3656382751 learned_addr learned my addr 172.21.15.52:0/3656382751 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:31.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.925+0000 7f14a8bd9640 1 --2- 172.21.15.52:0/3656382751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55965a0917c0 0x55965a096660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:31.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.925+0000 7f14a3fff640 1 -- 172.21.15.52:0/3656382751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55965a088be0 msgr2=0x55965a096120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:31.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.925+0000 7f14a3fff640 1 --2- 172.21.15.52:0/3656382751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55965a088be0 0x55965a096120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.925+0000 7f14a3fff640 1 -- 172.21.15.52:0/3656382751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55965a0917c0 msgr2=0x55965a096660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:31.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.925+0000 7f14a3fff640 1 --2- 172.21.15.52:0/3656382751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55965a0917c0 0x55965a096660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.925+0000 7f14a3fff640 1 -- 172.21.15.52:0/3656382751 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55965a096e60 con 0x55965a0882c0 2024-04-02T13:58:31.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.925+0000 7f14a3fff640 1 --2- 172.21.15.52:0/3656382751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0882c0 0x55965a099da0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f1494006010 tx=0x7f149400b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ec6c2ce5b3accb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:31.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.925+0000 7f14837fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:58:31.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.926+0000 7f14837fe640 1 -- 172.21.15.52:0/3656382751 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149402c070 con 0x55965a0882c0 2024-04-02T13:58:31.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.926+0000 7f14837fe640 1 -- 172.21.15.52:0/3656382751 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f1494028380 con 0x55965a0882c0 2024-04-02T13:58:31.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.926+0000 7f14ad223740 1 -- 172.21.15.52:0/3656382751 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1494005580 con 0x55965a0882c0 2024-04-02T13:58:31.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.926+0000 7f14837fe640 1 -- 172.21.15.52:0/3656382751 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f149403d050 con 0x55965a0882c0 2024-04-02T13:58:31.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.926+0000 7f14ad223740 10 client.17124 fetch_fsmap learned FSMap version 30 2024-04-02T13:58:31.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.926+0000 7f14ad223740 1 -- 172.21.15.52:0/3656382751 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55965a10d540 con 0x55965a0882c0 2024-04-02T13:58:31.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.926+0000 7f14837fe640 1 -- 172.21.15.52:0/3656382751 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1494005580 con 0x55965a0882c0 2024-04-02T13:58:31.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.926+0000 7f14837fe640 1 -- 172.21.15.52:0/3656382751 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1494004680 con 0x55965a0882c0 2024-04-02T13:58:31.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.926+0000 7f14ad223740 10 client.17124 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:58:31.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.926+0000 7f14ad223740 10 client.17124 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:58:31.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.926+0000 7f14ad223740 20 client.17124 populate_metadata read hostname 'smithi052' 2024-04-02T13:58:31.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.926+0000 7f14ad223740 1 --2- 172.21.15.52:0/3656382751 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55965a10da50 0x55965a10fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:31.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.926+0000 7f14ad223740 4 client.17124 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:58:31.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.926+0000 7f14ad223740 1 -- 172.21.15.52:0/3656382751 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559659ff4000 con 0x55965a10da50 2024-04-02T13:58:31.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.927+0000 7f14a37fe640 1 --2- 172.21.15.52:0/3656382751 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55965a10da50 0x55965a10fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:31.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.927+0000 7f14a37fe640 1 --2- 172.21.15.52:0/3656382751 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55965a10da50 0x55965a10fe40 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b695dfe678e8749b server_cookie=a2428ff276e92594 in_seq=0 out_seq=0 2024-04-02T13:58:31.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.927+0000 7f14837fe640 10 client.17124 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:58:31.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.928+0000 7f14837fe640 1 -- 172.21.15.52:0/3656382751 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559659ff4000 con 0x55965a10da50 2024-04-02T13:58:31.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.928+0000 7f14837fe640 10 client.17124 handle_command_reply: tid=0 2024-04-02T13:58:31.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.928+0000 7f14ad223740 1 client.17124 shutdown 2024-04-02T13:58:31.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.928+0000 7f14ad223740 1 -- 172.21.15.52:0/3656382751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0882c0 msgr2=0x55965a099da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:31.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.928+0000 7f14ad223740 1 --2- 172.21.15.52:0/3656382751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0882c0 0x55965a099da0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f1494006010 tx=0x7f149400b420 comp rx=0 tx=0).stop 2024-04-02T13:58:31.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.928+0000 7f14ad223740 1 -- 172.21.15.52:0/3656382751 shutdown_connections 2024-04-02T13:58:31.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.928+0000 7f14ad223740 1 --2- 172.21.15.52:0/3656382751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55965a0917c0 0x55965a096660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.928+0000 7f14ad223740 1 --2- 172.21.15.52:0/3656382751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55965a088be0 0x55965a096120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.928+0000 7f14ad223740 1 --2- 172.21.15.52:0/3656382751 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55965a10da50 0x55965a10fe40 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.928+0000 7f14ad223740 1 --2- 172.21.15.52:0/3656382751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55965a0882c0 0x55965a099da0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.928+0000 7f14ad223740 1 -- 172.21.15.52:0/3656382751 >> 172.21.15.52:0/3656382751 conn(0x55965a0829f0 msgr2=0x55965a081220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:31.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.928+0000 7f14837fe640 0 client.17124 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:58:31.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.928+0000 7f14ad223740 1 -- 172.21.15.52:0/3656382751 shutdown_connections 2024-04-02T13:58:31.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.928+0000 7f14ad223740 1 -- 172.21.15.52:0/3656382751 wait complete. 2024-04-02T13:58:31.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.928+0000 7f14ad223740 20 client.17124 trim_cache size 0 max 16384 2024-04-02T13:58:31.942 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:58:31.943 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-04-02T13:58:31.943 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "5dcc19b5-b27f-48b2-a4cc-459894905b0c", 2024-04-02T13:58:31.943 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-04-02T13:58:31.943 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:58:31.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.929+0000 7f14aa63c640 1 -- 172.21.15.52:0/2109659347 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f148c0a3da0 msgr2=0x7f148c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:31.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.929+0000 7f14aa63c640 1 --2- 172.21.15.52:0/2109659347 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f148c0a3da0 0x7f148c0a6260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f149c005fd0 tx=0x7f149c008730 comp rx=0 tx=0).stop 2024-04-02T13:58:31.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.929+0000 7f14aa63c640 1 -- 172.21.15.52:0/2109659347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14a4106e30 msgr2=0x7f14a41989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:31.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.929+0000 7f14aa63c640 1 --2- 172.21.15.52:0/2109659347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14a4106e30 0x7f14a41989a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f1498036040 tx=0x7f149802ce20 comp rx=0 tx=0).stop 2024-04-02T13:58:31.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.929+0000 7f14aa63c640 1 -- 172.21.15.52:0/2109659347 shutdown_connections 2024-04-02T13:58:31.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.929+0000 7f14aa63c640 1 --2- 172.21.15.52:0/2109659347 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f148c0a3da0 0x7f148c0a6260 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.929+0000 7f14aa63c640 1 --2- 172.21.15.52:0/2109659347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14a419cd20 0x7f14a419d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.929+0000 7f14aa63c640 1 --2- 172.21.15.52:0/2109659347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14a4106e30 0x7f14a41989a0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.929+0000 7f14aa63c640 1 --2- 172.21.15.52:0/2109659347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14a4100670 0x7f14a4198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:31.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.929+0000 7f14aa63c640 1 -- 172.21.15.52:0/2109659347 >> 172.21.15.52:0/2109659347 conn(0x7f14a4075c60 msgr2=0x7f14a40fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:31.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.929+0000 7f14aa63c640 1 -- 172.21.15.52:0/2109659347 shutdown_connections 2024-04-02T13:58:31.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:31.929+0000 7f14aa63c640 1 -- 172.21.15.52:0/2109659347 wait complete. 2024-04-02T13:58:31.946 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:58:31.946 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:58:32.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.050+0000 7f4f78d40640 1 -- 172.21.15.52:0/3610051788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f74106980 msgr2=0x7f4f74106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:32.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.050+0000 7f4f78d40640 1 --2- 172.21.15.52:0/3610051788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f74106980 0x7f4f74106d60 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f4f64009cd0 tx=0x7f4f6402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.051+0000 7f4f78d40640 1 -- 172.21.15.52:0/3610051788 shutdown_connections 2024-04-02T13:58:32.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.051+0000 7f4f78d40640 1 --2- 172.21.15.52:0/3610051788 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f74101330 0x7f4f7410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.051+0000 7f4f78d40640 1 --2- 172.21.15.52:0/3610051788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f74100970 0x7f4f74100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.051+0000 7f4f78d40640 1 --2- 172.21.15.52:0/3610051788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f74106980 0x7f4f74106d60 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.051+0000 7f4f78d40640 1 -- 172.21.15.52:0/3610051788 >> 172.21.15.52:0/3610051788 conn(0x7f4f74077b00 msgr2=0x7f4f74077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:32.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.051+0000 7f4f78d40640 1 -- 172.21.15.52:0/3610051788 shutdown_connections 2024-04-02T13:58:32.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.051+0000 7f4f78d40640 1 -- 172.21.15.52:0/3610051788 wait complete. 2024-04-02T13:58:32.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.051+0000 7f4f78d40640 1 Processor -- start 2024-04-02T13:58:32.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.052+0000 7f4f78d40640 1 -- start start 2024-04-02T13:58:32.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.052+0000 7f4f78d40640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f74100970 0x7f4f741972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:32.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.052+0000 7f4f78d40640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f74101330 0x7f4f74197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:32.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.052+0000 7f4f78d40640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f74106980 0x7f4f7419bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:32.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.052+0000 7f4f78d40640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4f74110d60 con 0x7f4f74100970 2024-04-02T13:58:32.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.052+0000 7f4f78d40640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4f74110be0 con 0x7f4f74106980 2024-04-02T13:58:32.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.052+0000 7f4f78d40640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4f74110ee0 con 0x7f4f74101330 2024-04-02T13:58:32.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.052+0000 7f4f7259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f74100970 0x7f4f741972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:32.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.052+0000 7f4f7259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f74100970 0x7f4f741972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53086/0 (socket says 172.21.15.52:53086) 2024-04-02T13:58:32.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.052+0000 7f4f71d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f74101330 0x7f4f74197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:32.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.052+0000 7f4f7259c640 1 -- 172.21.15.52:0/3026492251 learned_addr learned my addr 172.21.15.52:0/3026492251 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:32.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.052+0000 7f4f72d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f74106980 0x7f4f7419bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:32.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.052+0000 7f4f7259c640 1 -- 172.21.15.52:0/3026492251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f74101330 msgr2=0x7f4f74197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:32.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.052+0000 7f4f7259c640 1 --2- 172.21.15.52:0/3026492251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f74101330 0x7f4f74197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.052+0000 7f4f7259c640 1 -- 172.21.15.52:0/3026492251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f74106980 msgr2=0x7f4f7419bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:32.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.052+0000 7f4f7259c640 1 --2- 172.21.15.52:0/3026492251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f74106980 0x7f4f7419bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.052+0000 7f4f7259c640 1 -- 172.21.15.52:0/3026492251 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7419c2b0 con 0x7f4f74100970 2024-04-02T13:58:32.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.052+0000 7f4f71d9b640 1 --2- 172.21.15.52:0/3026492251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f74101330 0x7f4f74197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:58:32.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.052+0000 7f4f72d9d640 1 --2- 172.21.15.52:0/3026492251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f74106980 0x7f4f7419bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:58:32.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.053+0000 7f4f7259c640 1 --2- 172.21.15.52:0/3026492251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f74100970 0x7f4f741972c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f4f64004af0 tx=0x7f4f64032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:32.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.053+0000 7f4f637fe640 1 -- 172.21.15.52:0/3026492251 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f64041020 con 0x7f4f74100970 2024-04-02T13:58:32.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.053+0000 7f4f637fe640 1 -- 172.21.15.52:0/3026492251 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f64034cd0 con 0x7f4f74100970 2024-04-02T13:58:32.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.053+0000 7f4f78d40640 1 -- 172.21.15.52:0/3026492251 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f7419c540 con 0x7f4f74100970 2024-04-02T13:58:32.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.053+0000 7f4f637fe640 1 -- 172.21.15.52:0/3026492251 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f64039040 con 0x7f4f74100970 2024-04-02T13:58:32.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.053+0000 7f4f78d40640 1 -- 172.21.15.52:0/3026492251 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f741a3e30 con 0x7f4f74100970 2024-04-02T13:58:32.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.055+0000 7f4f637fe640 1 -- 172.21.15.52:0/3026492251 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4f64044020 con 0x7f4f74100970 2024-04-02T13:58:32.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.056+0000 7f4f7ba08740 1 Processor -- start 2024-04-02T13:58:32.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.056+0000 7f4f7ba08740 1 -- start start 2024-04-02T13:58:32.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.056+0000 7f4f7ba08740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573b9f958d0 0x5573b9f95cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:32.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.056+0000 7f4f637fe640 1 --2- 172.21.15.52:0/3026492251 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4f5c0a3d20 0x7f4f5c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:32.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.056+0000 7f4f7ba08740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f961f0 0x5573b9f9e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:32.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.056+0000 7f4f7ba08740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b9f9edd0 0x5573b9fa11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:32.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.056+0000 7f4f7ba08740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5573b9ef9830 con 0x5573b9f958d0 2024-04-02T13:58:32.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.056+0000 7f4f7ba08740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5573b9ea1820 con 0x5573b9f961f0 2024-04-02T13:58:32.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.056+0000 7f4f7ba08740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5573b9ef57e0 con 0x5573b9f9edd0 2024-04-02T13:58:32.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.057+0000 7f4f71d9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f961f0 0x5573b9f9e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:32.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.057+0000 7f4f71d9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f961f0 0x5573b9f9e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:42186/0 (socket says 172.21.15.52:42186) 2024-04-02T13:58:32.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.057+0000 7f4f71d9b640 1 -- 172.21.15.52:0/682632712 learned_addr learned my addr 172.21.15.52:0/682632712 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:32.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.057+0000 7f4f72d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573b9f958d0 0x5573b9f95cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:32.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.057+0000 7f4f7259c640 1 --2- 172.21.15.52:0/3026492251 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4f5c0a3d20 0x7f4f5c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:32.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.057+0000 7f4f637fe640 1 -- 172.21.15.52:0/3026492251 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f4f640f2730 con 0x7f4f74100970 2024-04-02T13:58:32.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.057+0000 7f4f72d9d640 1 --2- 172.21.15.52:0/682632712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b9f9edd0 0x5573b9fa11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:32.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.057+0000 7f4f71d9b640 1 -- 172.21.15.52:0/682632712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b9f9edd0 msgr2=0x5573b9fa11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:32.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.057+0000 7f4f71d9b640 1 --2- 172.21.15.52:0/682632712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b9f9edd0 0x5573b9fa11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.057+0000 7f4f71d9b640 1 -- 172.21.15.52:0/682632712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573b9f958d0 msgr2=0x5573b9f95cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:32.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.057+0000 7f4f71d9b640 1 --2- 172.21.15.52:0/682632712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573b9f958d0 0x5573b9f95cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.057+0000 7f4f71d9b640 1 -- 172.21.15.52:0/682632712 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573b9cd77d0 con 0x5573b9f961f0 2024-04-02T13:58:32.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.057+0000 7f4f72d9d640 1 --2- 172.21.15.52:0/682632712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b9f9edd0 0x5573b9fa11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:58:32.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.057+0000 7f4f7259c640 1 --2- 172.21.15.52:0/3026492251 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4f5c0a3d20 0x7f4f5c0a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4f640efd00 tx=0x7f4f64039bb0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:32.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.057+0000 7f4f71d9b640 1 --2- 172.21.15.52:0/682632712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f961f0 0x5573b9f9e890 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4f680114f0 tx=0x7f4f680119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=796ca55f320970e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:32.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.057+0000 7f4f60ff9640 1 -- 172.21.15.52:0/682632712 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6801e070 con 0x5573b9f961f0 2024-04-02T13:58:32.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.058+0000 7f4f60ff9640 1 -- 172.21.15.52:0/682632712 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f680035f0 con 0x5573b9f961f0 2024-04-02T13:58:32.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.058+0000 7f4f60ff9640 1 -- 172.21.15.52:0/682632712 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6800f820 con 0x5573b9f961f0 2024-04-02T13:58:32.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.058+0000 7f4f7ba08740 1 -- 172.21.15.52:0/682632712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f961f0 msgr2=0x5573b9f9e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:32.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.058+0000 7f4f7ba08740 1 --2- 172.21.15.52:0/682632712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f961f0 0x5573b9f9e890 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4f680114f0 tx=0x7f4f680119b0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.058+0000 7f4f7ba08740 1 -- 172.21.15.52:0/682632712 shutdown_connections 2024-04-02T13:58:32.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.058+0000 7f4f7ba08740 1 --2- 172.21.15.52:0/682632712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b9f9edd0 0x5573b9fa11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.058+0000 7f4f7ba08740 1 --2- 172.21.15.52:0/682632712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f961f0 0x5573b9f9e890 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.058+0000 7f4f7ba08740 1 --2- 172.21.15.52:0/682632712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573b9f958d0 0x5573b9f95cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.058+0000 7f4f7ba08740 1 -- 172.21.15.52:0/682632712 >> 172.21.15.52:0/682632712 conn(0x5573b9f90000 msgr2=0x5573b9f903e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:32.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.058+0000 7f4f7ba08740 1 -- 172.21.15.52:0/682632712 shutdown_connections 2024-04-02T13:58:32.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.058+0000 7f4f7ba08740 1 -- 172.21.15.52:0/682632712 wait complete. 2024-04-02T13:58:32.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.059+0000 7f4f7ba08740 1 -- start start 2024-04-02T13:58:32.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.059+0000 7f4f7ba08740 1 Processor -- start 2024-04-02T13:58:32.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.059+0000 7f4f7ba08740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f958d0 0x5573b9f9e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:32.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.059+0000 7f4f7ba08740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b9f961f0 0x5573b9f9ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:32.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.059+0000 7f4f7ba08740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573b9f9edd0 0x5573b9f9bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:32.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.059+0000 7f4f7ba08740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5573b9fa1bc0 con 0x5573b9f9edd0 2024-04-02T13:58:32.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.059+0000 7f4f7ba08740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5573b9eb4d30 con 0x5573b9f958d0 2024-04-02T13:58:32.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.059+0000 7f4f7ba08740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5573b9fa1d40 con 0x5573b9f961f0 2024-04-02T13:58:32.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.059+0000 7f4f7ba08740 10 client.0 _ll_register_callbacks cb 0x5573b9eb4720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:58:32.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.059+0000 7f4f72d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f958d0 0x5573b9f9e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:32.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.059+0000 7f4f72d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f958d0 0x5573b9f9e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:42196/0 (socket says 172.21.15.52:42196) 2024-04-02T13:58:32.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.059+0000 7f4f72d9d640 1 -- 172.21.15.52:0/1151102786 learned_addr learned my addr 172.21.15.52:0/1151102786 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:32.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.059+0000 7f4f71d9b640 1 --2- 172.21.15.52:0/1151102786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b9f961f0 0x5573b9f9ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:32.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.059+0000 7f4f72d9d640 1 --2- 172.21.15.52:0/1151102786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573b9f9edd0 0x5573b9f9bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:32.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.060+0000 7f4f72d9d640 1 -- 172.21.15.52:0/1151102786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b9f961f0 msgr2=0x5573b9f9ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:32.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.060+0000 7f4f72d9d640 1 --2- 172.21.15.52:0/1151102786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b9f961f0 0x5573b9f9ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.060+0000 7f4f72d9d640 1 -- 172.21.15.52:0/1151102786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573b9f9edd0 msgr2=0x5573b9f9bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:32.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.060+0000 7f4f72d9d640 1 --2- 172.21.15.52:0/1151102786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573b9f9edd0 0x5573b9f9bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.060+0000 7f4f72d9d640 1 -- 172.21.15.52:0/1151102786 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573b9c0d3f0 con 0x5573b9f958d0 2024-04-02T13:58:32.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.060+0000 7f4f72d9d640 1 --2- 172.21.15.52:0/1151102786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f958d0 0x5573b9f9e6a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c012f00 tx=0x7f4f6c005cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0ba1aff51a0556b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:32.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.060+0000 7f4f59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-02T13:58:32.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.060+0000 7f4f59ffb640 1 -- 172.21.15.52:0/1151102786 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c004030 con 0x5573b9f958d0 2024-04-02T13:58:32.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.060+0000 7f4f7ba08740 1 -- 172.21.15.52:0/1151102786 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f680035f0 con 0x5573b9f958d0 2024-04-02T13:58:32.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.060+0000 7f4f59ffb640 1 -- 172.21.15.52:0/1151102786 <== mon.2 v2:172.21.15.88:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f4f6c0041d0 con 0x5573b9f958d0 2024-04-02T13:58:32.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.060+0000 7f4f59ffb640 1 -- 172.21.15.52:0/1151102786 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4f6c028050 con 0x5573b9f958d0 2024-04-02T13:58:32.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.060+0000 7f4f7ba08740 10 client.26438 fetch_fsmap learned FSMap version 30 2024-04-02T13:58:32.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.060+0000 7f4f7ba08740 1 -- 172.21.15.52:0/1151102786 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5573b9bcd0f0 con 0x5573b9f958d0 2024-04-02T13:58:32.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.061+0000 7f4f59ffb640 1 -- 172.21.15.52:0/1151102786 <== mon.2 v2:172.21.15.88:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f680035f0 con 0x5573b9f958d0 2024-04-02T13:58:32.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.061+0000 7f4f59ffb640 1 -- 172.21.15.52:0/1151102786 <== mon.2 v2:172.21.15.88:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4f6c010430 con 0x5573b9f958d0 2024-04-02T13:58:32.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.061+0000 7f4f7ba08740 10 client.26438 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:58:32.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.061+0000 7f4f7ba08740 10 client.26438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:58:32.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.061+0000 7f4f7ba08740 20 client.26438 populate_metadata read hostname 'smithi052' 2024-04-02T13:58:32.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.061+0000 7f4f7ba08740 1 --2- 172.21.15.52:0/1151102786 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5573b9fb0fe0 0x5573b9fb3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:32.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.061+0000 7f4f7ba08740 4 client.26438 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:58:32.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.061+0000 7f4f7ba08740 1 -- 172.21.15.52:0/1151102786 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573b9f06d70 con 0x5573b9fb0fe0 2024-04-02T13:58:32.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.061+0000 7f4f71d9b640 1 --2- 172.21.15.52:0/1151102786 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5573b9fb0fe0 0x5573b9fb3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:32.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.061+0000 7f4f71d9b640 1 --2- 172.21.15.52:0/1151102786 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5573b9fb0fe0 0x5573b9fb3410 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1498ed5f35acd2e3 server_cookie=a69a11c07fa15b3a in_seq=0 out_seq=0 2024-04-02T13:58:32.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.061+0000 7f4f59ffb640 10 client.26438 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:58:32.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.062+0000 7f4f59ffb640 1 -- 172.21.15.52:0/1151102786 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5573b9f06d70 con 0x5573b9fb0fe0 2024-04-02T13:58:32.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.062+0000 7f4f59ffb640 10 client.26438 handle_command_reply: tid=0 2024-04-02T13:58:32.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.062+0000 7f4f7ba08740 1 client.26438 shutdown 2024-04-02T13:58:32.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.062+0000 7f4f7ba08740 1 -- 172.21.15.52:0/1151102786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f958d0 msgr2=0x5573b9f9e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:32.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.062+0000 7f4f7ba08740 1 --2- 172.21.15.52:0/1151102786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f958d0 0x5573b9f9e6a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c012f00 tx=0x7f4f6c005cc0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.062+0000 7f4f7ba08740 1 -- 172.21.15.52:0/1151102786 shutdown_connections 2024-04-02T13:58:32.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.062+0000 7f4f7ba08740 1 --2- 172.21.15.52:0/1151102786 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5573b9fb0fe0 0x5573b9fb3410 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.062+0000 7f4f7ba08740 1 --2- 172.21.15.52:0/1151102786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573b9f9edd0 0x5573b9f9bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.062+0000 7f4f7ba08740 1 --2- 172.21.15.52:0/1151102786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b9f961f0 0x5573b9f9ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.062+0000 7f4f7ba08740 1 --2- 172.21.15.52:0/1151102786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f958d0 0x5573b9f9e6a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.062+0000 7f4f59ffb640 0 client.26438 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:58:32.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.062+0000 7f4f7ba08740 1 -- 172.21.15.52:0/1151102786 >> 172.21.15.52:0/1151102786 conn(0x5573b9f90000 msgr2=0x5573b9f91d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:32.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.062+0000 7f4f7ba08740 1 -- 172.21.15.52:0/1151102786 shutdown_connections 2024-04-02T13:58:32.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.062+0000 7f4f7ba08740 1 -- 172.21.15.52:0/1151102786 wait complete. 2024-04-02T13:58:32.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.063+0000 7f4f7ba08740 20 client.26438 trim_cache size 0 max 16384 2024-04-02T13:58:32.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.084+0000 7f4f7ba08740 1 Processor -- start 2024-04-02T13:58:32.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.084+0000 7f4f7ba08740 1 -- start start 2024-04-02T13:58:32.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.084+0000 7f4f7ba08740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573b9f958d0 0x5573b9f8f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:32.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.084+0000 7f4f7ba08740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f961f0 0x5573b9f9e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:32.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.084+0000 7f4f7ba08740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b9f9edd0 0x5573b9f9e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:32.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.084+0000 7f4f7ba08740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4f741acc30 con 0x5573b9f958d0 2024-04-02T13:58:32.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.084+0000 7f4f7ba08740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4f741ae770 con 0x5573b9f961f0 2024-04-02T13:58:32.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.084+0000 7f4f7ba08740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4f741ad300 con 0x5573b9f9edd0 2024-04-02T13:58:32.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.084+0000 7f4f72d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573b9f958d0 0x5573b9f8f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:32.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.084+0000 7f4f71d9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f961f0 0x5573b9f9e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:32.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.084+0000 7f4f71d9b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f961f0 0x5573b9f9e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:42210/0 (socket says 172.21.15.52:42210) 2024-04-02T13:58:32.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.084+0000 7f4f72d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573b9f958d0 0x5573b9f8f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53118/0 (socket says 172.21.15.52:53118) 2024-04-02T13:58:32.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.084+0000 7f4f71d9b640 1 -- 172.21.15.52:0/2064802201 learned_addr learned my addr 172.21.15.52:0/2064802201 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:32.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.084+0000 7f4f72d9d640 1 --2- 172.21.15.52:0/2064802201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b9f9edd0 0x5573b9f9e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:32.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.084+0000 7f4f71d9b640 1 -- 172.21.15.52:0/2064802201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b9f9edd0 msgr2=0x5573b9f9e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:32.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.084+0000 7f4f71d9b640 1 --2- 172.21.15.52:0/2064802201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b9f9edd0 0x5573b9f9e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.084+0000 7f4f71d9b640 1 -- 172.21.15.52:0/2064802201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573b9f958d0 msgr2=0x5573b9f8f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:32.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.084+0000 7f4f71d9b640 1 --2- 172.21.15.52:0/2064802201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573b9f958d0 0x5573b9f8f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.084+0000 7f4f71d9b640 1 -- 172.21.15.52:0/2064802201 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573b9f90bd0 con 0x5573b9f961f0 2024-04-02T13:58:32.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.084+0000 7f4f72d9d640 1 --2- 172.21.15.52:0/2064802201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b9f9edd0 0x5573b9f9e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:58:32.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.085+0000 7f4f71d9b640 1 --2- 172.21.15.52:0/2064802201 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f961f0 0x5573b9f9e160 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4f68014040 tx=0x7f4f68005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74ef7fed5bb34358 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:32.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.085+0000 7f4f5bfff640 1 -- 172.21.15.52:0/2064802201 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68029e60 con 0x5573b9f961f0 2024-04-02T13:58:32.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.085+0000 7f4f5bfff640 1 -- 172.21.15.52:0/2064802201 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f6801f040 con 0x5573b9f961f0 2024-04-02T13:58:32.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.085+0000 7f4f5bfff640 1 -- 172.21.15.52:0/2064802201 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6801e040 con 0x5573b9f961f0 2024-04-02T13:58:32.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.086+0000 7f4f7ba08740 1 -- 172.21.15.52:0/2064802201 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f961f0 msgr2=0x5573b9f9e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:32.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.086+0000 7f4f7ba08740 1 --2- 172.21.15.52:0/2064802201 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f961f0 0x5573b9f9e160 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4f68014040 tx=0x7f4f68005850 comp rx=0 tx=0).stop 2024-04-02T13:58:32.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.086+0000 7f4f7ba08740 1 -- 172.21.15.52:0/2064802201 shutdown_connections 2024-04-02T13:58:32.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.086+0000 7f4f7ba08740 1 --2- 172.21.15.52:0/2064802201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b9f9edd0 0x5573b9f9e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.086+0000 7f4f7ba08740 1 --2- 172.21.15.52:0/2064802201 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f961f0 0x5573b9f9e160 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.086+0000 7f4f7ba08740 1 --2- 172.21.15.52:0/2064802201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573b9f958d0 0x5573b9f8f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.086+0000 7f4f7ba08740 1 -- 172.21.15.52:0/2064802201 >> 172.21.15.52:0/2064802201 conn(0x5573b9f90000 msgr2=0x5573b9f965d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:32.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.086+0000 7f4f7ba08740 1 -- 172.21.15.52:0/2064802201 shutdown_connections 2024-04-02T13:58:32.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.086+0000 7f4f7ba08740 1 -- 172.21.15.52:0/2064802201 wait complete. 2024-04-02T13:58:32.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.086+0000 7f4f7ba08740 1 -- start start 2024-04-02T13:58:32.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.086+0000 7f4f7ba08740 1 Processor -- start 2024-04-02T13:58:32.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.087+0000 7f4f7ba08740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f958d0 0x5573b9fa74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:32.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.087+0000 7f4f7ba08740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b9f961f0 0x5573b9fa2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:32.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.087+0000 7f4f7ba08740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573b9f9edd0 0x5573b9fa2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:32.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.087+0000 7f4f7ba08740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4f741ae0a0 con 0x5573b9f9edd0 2024-04-02T13:58:32.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.087+0000 7f4f7ba08740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4f741ad9d0 con 0x5573b9f958d0 2024-04-02T13:58:32.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.087+0000 7f4f7ba08740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4f741aac10 con 0x5573b9f961f0 2024-04-02T13:58:32.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.087+0000 7f4f7ba08740 10 client.0 _ll_register_callbacks cb 0x7f4f6c012d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:58:32.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.087+0000 7f4f72d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573b9f9edd0 0x5573b9fa2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:32.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.087+0000 7f4f71d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b9f961f0 0x5573b9fa2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:32.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.087+0000 7f4f72d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573b9f9edd0 0x5573b9fa2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53120/0 (socket says 172.21.15.52:53120) 2024-04-02T13:58:32.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.087+0000 7f4f71d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b9f961f0 0x5573b9fa2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.52:47332/0 (socket says 172.21.15.52:47332) 2024-04-02T13:58:32.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.087+0000 7f4f72d9d640 1 -- 172.21.15.52:0/2338186206 learned_addr learned my addr 172.21.15.52:0/2338186206 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:58:32.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.087+0000 7f4f72d9d640 1 --2- 172.21.15.52:0/2338186206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f958d0 0x5573b9fa74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:32.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.087+0000 7f4f72d9d640 1 -- 172.21.15.52:0/2338186206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b9f961f0 msgr2=0x5573b9fa2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:32.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.087+0000 7f4f72d9d640 1 --2- 172.21.15.52:0/2338186206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b9f961f0 0x5573b9fa2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.087+0000 7f4f72d9d640 1 -- 172.21.15.52:0/2338186206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f958d0 msgr2=0x5573b9fa74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:58:32.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.087+0000 7f4f72d9d640 1 --2- 172.21.15.52:0/2338186206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f958d0 0x5573b9fa74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.087+0000 7f4f72d9d640 1 -- 172.21.15.52:0/2338186206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573b9fa3680 con 0x5573b9f9edd0 2024-04-02T13:58:32.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.087+0000 7f4f72d9d640 1 --2- 172.21.15.52:0/2338186206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573b9f9edd0 0x5573b9fa2e80 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c012cf0 tx=0x7f4f6c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6995c056a6b7910d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:32.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.087+0000 7f4f59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:58:32.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.088+0000 7f4f59ffb640 1 -- 172.21.15.52:0/2338186206 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c010bd0 con 0x5573b9f9edd0 2024-04-02T13:58:32.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.088+0000 7f4f59ffb640 1 -- 172.21.15.52:0/2338186206 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f4f6c0050b0 con 0x5573b9f9edd0 2024-04-02T13:58:32.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.088+0000 7f4f7ba08740 1 -- 172.21.15.52:0/2338186206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f6801f040 con 0x5573b9f9edd0 2024-04-02T13:58:32.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.088+0000 7f4f59ffb640 1 -- 172.21.15.52:0/2338186206 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4f6c0052b0 con 0x5573b9f9edd0 2024-04-02T13:58:32.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.088+0000 7f4f7ba08740 10 client.17142 fetch_fsmap learned FSMap version 30 2024-04-02T13:58:32.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.088+0000 7f4f7ba08740 1 -- 172.21.15.52:0/2338186206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5573ba017660 con 0x5573b9f9edd0 2024-04-02T13:58:32.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.088+0000 7f4f59ffb640 1 -- 172.21.15.52:0/2338186206 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f6801f040 con 0x5573b9f9edd0 2024-04-02T13:58:32.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.088+0000 7f4f59ffb640 1 -- 172.21.15.52:0/2338186206 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4f6c022430 con 0x5573b9f9edd0 2024-04-02T13:58:32.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.088+0000 7f4f7ba08740 10 client.17142 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:58:32.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.088+0000 7f4f7ba08740 10 client.17142 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:58:32.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.088+0000 7f4f7ba08740 20 client.17142 populate_metadata read hostname 'smithi052' 2024-04-02T13:58:32.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.088+0000 7f4f7ba08740 1 --2- 172.21.15.52:0/2338186206 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5573ba017cd0 0x5573ba01a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:32.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.088+0000 7f4f7ba08740 4 client.17142 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:58:32.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.088+0000 7f4f7ba08740 1 -- 172.21.15.52:0/2338186206 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5573b9f010c0 con 0x5573ba017cd0 2024-04-02T13:58:32.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.088+0000 7f4f71d9b640 1 --2- 172.21.15.52:0/2338186206 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5573ba017cd0 0x5573ba01a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:32.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.089+0000 7f4f71d9b640 1 --2- 172.21.15.52:0/2338186206 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5573ba017cd0 0x5573ba01a0c0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6748fa923352d664 server_cookie=ee4f8c8717eb7a9b in_seq=0 out_seq=0 2024-04-02T13:58:32.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.089+0000 7f4f59ffb640 10 client.17142 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:58:32.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.089+0000 7f4f59ffb640 1 -- 172.21.15.52:0/2338186206 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x5573b9f010c0 con 0x5573ba017cd0 2024-04-02T13:58:32.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.089+0000 7f4f59ffb640 10 client.17142 handle_command_reply: tid=0 2024-04-02T13:58:32.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.089+0000 7f4f7ba08740 1 client.17142 shutdown 2024-04-02T13:58:32.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.090+0000 7f4f7ba08740 1 -- 172.21.15.52:0/2338186206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573b9f9edd0 msgr2=0x5573b9fa2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:32.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.090+0000 7f4f7ba08740 1 --2- 172.21.15.52:0/2338186206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573b9f9edd0 0x5573b9fa2e80 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c012cf0 tx=0x7f4f6c000bc0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.090+0000 7f4f7ba08740 1 -- 172.21.15.52:0/2338186206 shutdown_connections 2024-04-02T13:58:32.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.090+0000 7f4f7ba08740 1 --2- 172.21.15.52:0/2338186206 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5573ba017cd0 0x5573ba01a0c0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.090+0000 7f4f7ba08740 1 --2- 172.21.15.52:0/2338186206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5573b9f9edd0 0x5573b9fa2e80 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.090+0000 7f4f7ba08740 1 --2- 172.21.15.52:0/2338186206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b9f961f0 0x5573b9fa2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.090+0000 7f4f7ba08740 1 --2- 172.21.15.52:0/2338186206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5573b9f958d0 0x5573b9fa74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.090+0000 7f4f7ba08740 1 -- 172.21.15.52:0/2338186206 >> 172.21.15.52:0/2338186206 conn(0x5573b9f90000 msgr2=0x5573b9f8f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:32.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.090+0000 7f4f59ffb640 0 client.17142 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:58:32.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.090+0000 7f4f7ba08740 1 -- 172.21.15.52:0/2338186206 shutdown_connections 2024-04-02T13:58:32.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.090+0000 7f4f7ba08740 1 -- 172.21.15.52:0/2338186206 wait complete. 2024-04-02T13:58:32.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.090+0000 7f4f7ba08740 20 client.17142 trim_cache size 0 max 16384 2024-04-02T13:58:32.104 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:58:32.104 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "scrub active (2599 inodes in the stack)", 2024-04-02T13:58:32.104 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-04-02T13:58:32.104 INFO:teuthology.orchestra.run.smithi052.stdout: "5dcc19b5-b27f-48b2-a4cc-459894905b0c": { 2024-04-02T13:58:32.104 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-04-02T13:58:32.105 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "5dcc19b5-b27f-48b2-a4cc-459894905b0c", 2024-04-02T13:58:32.105 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-04-02T13:58:32.105 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:58:32.105 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:58:32.105 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:58:32.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.090+0000 7f4f78d40640 1 -- 172.21.15.52:0/3026492251 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4f5c0a3d20 msgr2=0x7f4f5c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:32.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.090+0000 7f4f78d40640 1 --2- 172.21.15.52:0/3026492251 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4f5c0a3d20 0x7f4f5c0a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4f640efd00 tx=0x7f4f64039bb0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.090+0000 7f4f78d40640 1 -- 172.21.15.52:0/3026492251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f74100970 msgr2=0x7f4f741972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:32.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.090+0000 7f4f78d40640 1 --2- 172.21.15.52:0/3026492251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f74100970 0x7f4f741972c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f4f64004af0 tx=0x7f4f64032ec0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.090+0000 7f4f78d40640 1 -- 172.21.15.52:0/3026492251 shutdown_connections 2024-04-02T13:58:32.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.090+0000 7f4f78d40640 1 --2- 172.21.15.52:0/3026492251 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4f5c0a3d20 0x7f4f5c0a61e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.090+0000 7f4f78d40640 1 --2- 172.21.15.52:0/3026492251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f74106980 0x7f4f7419bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.090+0000 7f4f78d40640 1 --2- 172.21.15.52:0/3026492251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f74101330 0x7f4f74197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.090+0000 7f4f78d40640 1 --2- 172.21.15.52:0/3026492251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f74100970 0x7f4f741972c0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:32.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.090+0000 7f4f78d40640 1 -- 172.21.15.52:0/3026492251 >> 172.21.15.52:0/3026492251 conn(0x7f4f74077b00 msgr2=0x7f4f74103290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:32.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.090+0000 7f4f78d40640 1 -- 172.21.15.52:0/3026492251 shutdown_connections 2024-04-02T13:58:32.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:58:32.090+0000 7f4f78d40640 1 -- 172.21.15.52:0/3026492251 wait complete. 2024-04-02T13:58:32.108 INFO:tasks.cephfs.filesystem:scrub status for tag:5dcc19b5-b27f-48b2-a4cc-459894905b0c - {'path': '/', 'tag': '5dcc19b5-b27f-48b2-a4cc-459894905b0c', 'options': 'recursive,force'} 2024-04-02T13:58:32.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:32 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:58:32.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:32 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:58:32.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:32 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:58:32.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:32 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:58:32.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:32 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:58:32.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:32 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:58:33.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:33 smithi088 ceph-mon[36528]: scrub queued for path: / 2024-04-02T13:58:33.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:33 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:58:33.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:33 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:58:33.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:33 smithi088 ceph-mon[36528]: pgmap v154: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:58:33.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:33 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:58:33.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:33 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:58:33.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:33 smithi189 ceph-mon[36683]: scrub queued for path: / 2024-04-02T13:58:33.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:33 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:58:33.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:33 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:58:33.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:33 smithi189 ceph-mon[36683]: pgmap v154: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:58:33.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:33 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:58:33.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:33 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:58:33.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:33 smithi052 ceph-mon[32104]: scrub queued for path: / 2024-04-02T13:58:33.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:33 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:58:33.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:33 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:58:33.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:33 smithi052 ceph-mon[32104]: pgmap v154: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:58:33.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:33 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:58:33.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:33 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:58:34.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:34 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:58:34.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:34 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:58:34.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:34 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:58:35.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:35 smithi088 ceph-mon[36528]: pgmap v155: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:35.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:35 smithi189 ceph-mon[36683]: pgmap v155: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:35.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:35 smithi052 ceph-mon[32104]: pgmap v155: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:37.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:37 smithi088 ceph-mon[36528]: pgmap v156: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:58:37.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:37 smithi189 ceph-mon[36683]: pgmap v156: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:58:37.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:37 smithi052 ceph-mon[32104]: pgmap v156: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:58:39.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:39 smithi088 ceph-mon[36528]: pgmap v157: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:58:39.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:39 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:58:39.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:39 smithi189 ceph-mon[36683]: pgmap v157: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:58:39.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:39 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:58:39.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:39 smithi052 ceph-mon[32104]: pgmap v157: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:58:39.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:39 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:58:40.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:40 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:58:40.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:40 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:58:40.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:40 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:58:40.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:40 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:58:40.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:40 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:58:40.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:40 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:58:40.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:40 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:58:40.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:40 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:58:40.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:40 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:58:41.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:41 smithi088 ceph-mon[36528]: pgmap v158: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:58:41.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:41 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T13:58:41.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:41 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T13:58:41.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:41 smithi189 ceph-mon[36683]: pgmap v158: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:58:41.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:41 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T13:58:41.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:41 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T13:58:41.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:41 smithi052 ceph-mon[32104]: pgmap v158: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:58:41.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:41 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T13:58:41.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:41 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T13:58:43.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:43 smithi088 ceph-mon[36528]: pgmap v159: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:58:43.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:43 smithi189 ceph-mon[36683]: pgmap v159: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:58:43.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:43 smithi052 ceph-mon[32104]: pgmap v159: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:58:45.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:45 smithi088 ceph-mon[36528]: pgmap v160: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:58:45.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:45 smithi189 ceph-mon[36683]: pgmap v160: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:58:45.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:45 smithi052 ceph-mon[32104]: pgmap v160: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:58:47.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:47 smithi088 ceph-mon[36528]: pgmap v161: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:58:47.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:47 smithi189 ceph-mon[36683]: pgmap v161: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:58:47.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:47 smithi052 ceph-mon[32104]: pgmap v161: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:58:49.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:49 smithi088 ceph-mon[36528]: pgmap v162: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:49.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:49 smithi189 ceph-mon[36683]: pgmap v162: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:49.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:49 smithi052 ceph-mon[32104]: pgmap v162: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:51.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:51 smithi088 ceph-mon[36528]: pgmap v163: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:51.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:51 smithi189 ceph-mon[36683]: pgmap v163: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:51.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:51 smithi052 ceph-mon[32104]: pgmap v163: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:53.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:53 smithi088 ceph-mon[36528]: pgmap v164: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:58:53.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:53 smithi189 ceph-mon[36683]: pgmap v164: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:58:53.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:53 smithi052 ceph-mon[32104]: pgmap v164: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:58:55.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:55 smithi088 ceph-mon[36528]: pgmap v165: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:55.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:55 smithi189 ceph-mon[36683]: pgmap v165: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:55.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:55 smithi052 ceph-mon[32104]: pgmap v165: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:57.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:57 smithi088 ceph-mon[36528]: pgmap v166: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:58:57.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:57 smithi189 ceph-mon[36683]: pgmap v166: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:58:57.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:57 smithi052 ceph-mon[32104]: pgmap v166: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:58:59.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:58:59 smithi088 ceph-mon[36528]: pgmap v167: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:59.538 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:58:59 smithi189 ceph-mon[36683]: pgmap v167: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:58:59.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:58:59 smithi052 ceph-mon[32104]: pgmap v167: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:01.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:01 smithi088 ceph-mon[36528]: pgmap v168: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:01.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:01 smithi189 ceph-mon[36683]: pgmap v168: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:01.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:01 smithi052 ceph-mon[32104]: pgmap v168: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:02.109 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:59:02.110 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:59:02.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.224+0000 7f8dafed3640 1 -- 172.21.15.52:0/2409294284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8da8102450 msgr2=0x7f8da810e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.224+0000 7f8dafed3640 1 --2- 172.21.15.52:0/2409294284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8da8102450 0x7f8da810e960 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8da0009cd0 tx=0x7f8da002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.224+0000 7f8dafed3640 1 -- 172.21.15.52:0/2409294284 shutdown_connections 2024-04-02T13:59:02.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.224+0000 7f8dafed3640 1 --2- 172.21.15.52:0/2409294284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8da8102450 0x7f8da810e960 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.224+0000 7f8dafed3640 1 --2- 172.21.15.52:0/2409294284 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8da8101a90 0x7f8da8101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.224+0000 7f8dafed3640 1 --2- 172.21.15.52:0/2409294284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8da8107aa0 0x7f8da8107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.224+0000 7f8dafed3640 1 -- 172.21.15.52:0/2409294284 >> 172.21.15.52:0/2409294284 conn(0x7f8da80fd5a0 msgr2=0x7f8da80ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:02.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.224+0000 7f8dafed3640 1 -- 172.21.15.52:0/2409294284 shutdown_connections 2024-04-02T13:59:02.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.224+0000 7f8dafed3640 1 -- 172.21.15.52:0/2409294284 wait complete. 2024-04-02T13:59:02.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.224+0000 7f8dafed3640 1 Processor -- start 2024-04-02T13:59:02.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.225+0000 7f8dafed3640 1 -- start start 2024-04-02T13:59:02.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.225+0000 7f8dafed3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8da8101a90 0x7f8da8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.225+0000 7f8dafed3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8da8102450 0x7f8da8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.225+0000 7f8dafed3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8da8107aa0 0x7f8da819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.225+0000 7f8dafed3640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8da8111eb0 con 0x7f8da8107aa0 2024-04-02T13:59:02.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.225+0000 7f8dafed3640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8da8111d30 con 0x7f8da8101a90 2024-04-02T13:59:02.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.225+0000 7f8dafed3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8da8112030 con 0x7f8da8102450 2024-04-02T13:59:02.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.225+0000 7f8dae470640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8da8107aa0 0x7f8da819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.225+0000 7f8dae470640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8da8107aa0 0x7f8da819ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58566/0 (socket says 172.21.15.52:58566) 2024-04-02T13:59:02.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.225+0000 7f8dae470640 1 -- 172.21.15.52:0/2985844202 learned_addr learned my addr 172.21.15.52:0/2985844202 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:02.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.225+0000 7f8dad46e640 1 --2- 172.21.15.52:0/2985844202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8da8102450 0x7f8da8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.225+0000 7f8dadc6f640 1 --2- 172.21.15.52:0/2985844202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8da8101a90 0x7f8da8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.225+0000 7f8dae470640 1 -- 172.21.15.52:0/2985844202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8da8102450 msgr2=0x7f8da8198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.225+0000 7f8dae470640 1 --2- 172.21.15.52:0/2985844202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8da8102450 0x7f8da8198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.225+0000 7f8dae470640 1 -- 172.21.15.52:0/2985844202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8da8101a90 msgr2=0x7f8da8198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.225+0000 7f8dae470640 1 --2- 172.21.15.52:0/2985844202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8da8101a90 0x7f8da8198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.225+0000 7f8dae470640 1 -- 172.21.15.52:0/2985844202 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da819d400 con 0x7f8da8107aa0 2024-04-02T13:59:02.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.225+0000 7f8dad46e640 1 --2- 172.21.15.52:0/2985844202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8da8102450 0x7f8da8198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:59:02.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.225+0000 7f8dadc6f640 1 --2- 172.21.15.52:0/2985844202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8da8101a90 0x7f8da8198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:59:02.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.225+0000 7f8dae470640 1 --2- 172.21.15.52:0/2985844202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8da8107aa0 0x7f8da819ccd0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f8da0037040 tx=0x7f8da0035ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:02.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.226+0000 7f8d9effd640 1 -- 172.21.15.52:0/2985844202 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da0039040 con 0x7f8da8107aa0 2024-04-02T13:59:02.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.226+0000 7f8d9effd640 1 -- 172.21.15.52:0/2985844202 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8da0044070 con 0x7f8da8107aa0 2024-04-02T13:59:02.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.226+0000 7f8dafed3640 1 -- 172.21.15.52:0/2985844202 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da819d690 con 0x7f8da8107aa0 2024-04-02T13:59:02.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.226+0000 7f8d9effd640 1 -- 172.21.15.52:0/2985844202 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da0004330 con 0x7f8da8107aa0 2024-04-02T13:59:02.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.226+0000 7f8dafed3640 1 -- 172.21.15.52:0/2985844202 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da81a4f80 con 0x7f8da8107aa0 2024-04-02T13:59:02.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.228+0000 7f8d9effd640 1 -- 172.21.15.52:0/2985844202 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8da0003970 con 0x7f8da8107aa0 2024-04-02T13:59:02.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.228+0000 7f8db2b91740 1 Processor -- start 2024-04-02T13:59:02.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.228+0000 7f8db2b91740 1 -- start start 2024-04-02T13:59:02.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.229+0000 7f8db2b91740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6f5338d0 0x559e6f533cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.229+0000 7f8db2b91740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5341f0 0x559e6f53c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.229+0000 7f8db2b91740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6f53cdd0 0x559e6f53f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.229+0000 7f8db2b91740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559e6f497830 con 0x559e6f5341f0 2024-04-02T13:59:02.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.229+0000 7f8db2b91740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559e6f43f820 con 0x559e6f53cdd0 2024-04-02T13:59:02.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.229+0000 7f8d9effd640 1 --2- 172.21.15.52:0/2985844202 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f8d940a3d20 0x7f8d940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.229+0000 7f8db2b91740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559e6f4937e0 con 0x559e6f5338d0 2024-04-02T13:59:02.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.229+0000 7f8dae470640 1 --2- 172.21.15.52:0/2985844202 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f8d940a3d20 0x7f8d940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.229+0000 7f8dadc6f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5341f0 0x559e6f53c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.229+0000 7f8dadc6f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5341f0 0x559e6f53c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58568/0 (socket says 172.21.15.52:58568) 2024-04-02T13:59:02.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.229+0000 7f8dadc6f640 1 -- 172.21.15.52:0/470367240 learned_addr learned my addr 172.21.15.52:0/470367240 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:02.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.229+0000 7f8dad46e640 1 --2- 172.21.15.52:0/470367240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6f53cdd0 0x559e6f53f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.229+0000 7f8d9effd640 1 -- 172.21.15.52:0/2985844202 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f8da00f2fe0 con 0x7f8da8107aa0 2024-04-02T13:59:02.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.229+0000 7f8dad46e640 1 --2- 172.21.15.52:0/470367240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6f5338d0 0x559e6f533cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.229+0000 7f8dadc6f640 1 -- 172.21.15.52:0/470367240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6f5338d0 msgr2=0x559e6f533cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:02.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.229+0000 7f8dadc6f640 1 --2- 172.21.15.52:0/470367240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6f5338d0 0x559e6f533cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.229+0000 7f8dadc6f640 1 -- 172.21.15.52:0/470367240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6f53cdd0 msgr2=0x559e6f53f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:02.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.229+0000 7f8dadc6f640 1 --2- 172.21.15.52:0/470367240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6f53cdd0 0x559e6f53f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.229+0000 7f8dadc6f640 1 -- 172.21.15.52:0/470367240 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e6f2757d0 con 0x559e6f5341f0 2024-04-02T13:59:02.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.229+0000 7f8dae470640 1 --2- 172.21.15.52:0/2985844202 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f8d940a3d20 0x7f8d940a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8da00bea50 tx=0x7f8da0090790 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:02.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.229+0000 7f8dadc6f640 1 --2- 172.21.15.52:0/470367240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5341f0 0x559e6f53c890 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f8da400cbf0 tx=0x7f8da400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22100f36906def7e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:02.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.230+0000 7f8d93fff640 1 -- 172.21.15.52:0/470367240 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da4005220 con 0x559e6f5341f0 2024-04-02T13:59:02.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.230+0000 7f8d93fff640 1 -- 172.21.15.52:0/470367240 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8da4016070 con 0x559e6f5341f0 2024-04-02T13:59:02.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.230+0000 7f8d93fff640 1 -- 172.21.15.52:0/470367240 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da4010740 con 0x559e6f5341f0 2024-04-02T13:59:02.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.230+0000 7f8db2b91740 1 -- 172.21.15.52:0/470367240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5341f0 msgr2=0x559e6f53c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.230+0000 7f8db2b91740 1 --2- 172.21.15.52:0/470367240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5341f0 0x559e6f53c890 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f8da400cbf0 tx=0x7f8da400e720 comp rx=0 tx=0).stop 2024-04-02T13:59:02.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.230+0000 7f8db2b91740 1 -- 172.21.15.52:0/470367240 shutdown_connections 2024-04-02T13:59:02.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.230+0000 7f8db2b91740 1 --2- 172.21.15.52:0/470367240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6f53cdd0 0x559e6f53f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.230+0000 7f8db2b91740 1 --2- 172.21.15.52:0/470367240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5341f0 0x559e6f53c890 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.230+0000 7f8db2b91740 1 --2- 172.21.15.52:0/470367240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6f5338d0 0x559e6f533cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.230+0000 7f8db2b91740 1 -- 172.21.15.52:0/470367240 >> 172.21.15.52:0/470367240 conn(0x559e6f52e000 msgr2=0x559e6f52e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:02.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.230+0000 7f8db2b91740 1 -- 172.21.15.52:0/470367240 shutdown_connections 2024-04-02T13:59:02.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.231+0000 7f8db2b91740 1 -- 172.21.15.52:0/470367240 wait complete. 2024-04-02T13:59:02.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.231+0000 7f8db2b91740 1 -- start start 2024-04-02T13:59:02.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.231+0000 7f8db2b91740 1 Processor -- start 2024-04-02T13:59:02.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.231+0000 7f8db2b91740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5338d0 0x559e6f53c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.231+0000 7f8db2b91740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6f5341f0 0x559e6f539a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.231+0000 7f8db2b91740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6f53cdd0 0x559e6f539f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.231+0000 7f8db2b91740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559e6f53fbc0 con 0x559e6f5338d0 2024-04-02T13:59:02.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.231+0000 7f8db2b91740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559e6f452d30 con 0x559e6f53cdd0 2024-04-02T13:59:02.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.231+0000 7f8db2b91740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559e6f53fd40 con 0x559e6f5341f0 2024-04-02T13:59:02.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.231+0000 7f8db2b91740 10 client.0 _ll_register_callbacks cb 0x559e6f452720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:02.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.231+0000 7f8dad46e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5338d0 0x559e6f53c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.231+0000 7f8dad46e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5338d0 0x559e6f53c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58570/0 (socket says 172.21.15.52:58570) 2024-04-02T13:59:02.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.231+0000 7f8dad46e640 1 -- 172.21.15.52:0/2999082450 learned_addr learned my addr 172.21.15.52:0/2999082450 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:02.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.232+0000 7f8dad46e640 1 --2- 172.21.15.52:0/2999082450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6f53cdd0 0x559e6f539f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.232+0000 7f8dadc6f640 1 --2- 172.21.15.52:0/2999082450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6f5341f0 0x559e6f539a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.232+0000 7f8dad46e640 1 -- 172.21.15.52:0/2999082450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6f5341f0 msgr2=0x559e6f539a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:02.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.232+0000 7f8dad46e640 1 --2- 172.21.15.52:0/2999082450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6f5341f0 0x559e6f539a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.232+0000 7f8dad46e640 1 -- 172.21.15.52:0/2999082450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6f53cdd0 msgr2=0x559e6f539f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:02.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.232+0000 7f8dad46e640 1 --2- 172.21.15.52:0/2999082450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6f53cdd0 0x559e6f539f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.232+0000 7f8dad46e640 1 -- 172.21.15.52:0/2999082450 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e6f1ab3f0 con 0x559e6f5338d0 2024-04-02T13:59:02.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.232+0000 7f8dad46e640 1 --2- 172.21.15.52:0/2999082450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5338d0 0x559e6f53c690 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f8d980058c0 tx=0x7f8d980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bcef79ff28ad26d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:02.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.232+0000 7f8d917fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:02.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.232+0000 7f8d917fa640 1 -- 172.21.15.52:0/2999082450 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d98004020 con 0x559e6f5338d0 2024-04-02T13:59:02.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.232+0000 7f8d917fa640 1 -- 172.21.15.52:0/2999082450 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f8d980041c0 con 0x559e6f5338d0 2024-04-02T13:59:02.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.232+0000 7f8db2b91740 1 -- 172.21.15.52:0/2999082450 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8da4016070 con 0x559e6f5338d0 2024-04-02T13:59:02.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.232+0000 7f8d917fa640 1 -- 172.21.15.52:0/2999082450 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8d980043c0 con 0x559e6f5338d0 2024-04-02T13:59:02.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.232+0000 7f8db2b91740 10 client.17154 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:02.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.232+0000 7f8db2b91740 1 -- 172.21.15.52:0/2999082450 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559e6f16b0f0 con 0x559e6f5338d0 2024-04-02T13:59:02.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.232+0000 7f8d917fa640 1 -- 172.21.15.52:0/2999082450 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8da4016070 con 0x559e6f5338d0 2024-04-02T13:59:02.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.233+0000 7f8d917fa640 1 -- 172.21.15.52:0/2999082450 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8d98004bb0 con 0x559e6f5338d0 2024-04-02T13:59:02.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.233+0000 7f8db2b91740 10 client.17154 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:02.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.233+0000 7f8db2b91740 10 client.17154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:59:02.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.233+0000 7f8db2b91740 20 client.17154 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:02.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.233+0000 7f8db2b91740 1 --2- 172.21.15.52:0/2999082450 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x559e6f54cd60 0x559e6f54f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.233+0000 7f8db2b91740 4 client.17154 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:59:02.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.233+0000 7f8db2b91740 1 -- 172.21.15.52:0/2999082450 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e6f4a4d70 con 0x559e6f54cd60 2024-04-02T13:59:02.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.233+0000 7f8dadc6f640 1 --2- 172.21.15.52:0/2999082450 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x559e6f54cd60 0x559e6f54f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.233+0000 7f8dadc6f640 1 --2- 172.21.15.52:0/2999082450 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x559e6f54cd60 0x559e6f54f1b0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f43f9c63276daa4a server_cookie=105daa0ca917df8d in_seq=0 out_seq=0 2024-04-02T13:59:02.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.233+0000 7f8d917fa640 10 client.17154 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:02.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.234+0000 7f8d917fa640 1 -- 172.21.15.52:0/2999082450 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e6f4a4d70 con 0x559e6f54cd60 2024-04-02T13:59:02.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.234+0000 7f8d917fa640 10 client.17154 handle_command_reply: tid=0 2024-04-02T13:59:02.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.234+0000 7f8db2b91740 1 client.17154 shutdown 2024-04-02T13:59:02.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.234+0000 7f8db2b91740 1 -- 172.21.15.52:0/2999082450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5338d0 msgr2=0x559e6f53c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.234+0000 7f8db2b91740 1 --2- 172.21.15.52:0/2999082450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5338d0 0x559e6f53c690 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f8d980058c0 tx=0x7f8d980046c0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.234+0000 7f8db2b91740 1 -- 172.21.15.52:0/2999082450 shutdown_connections 2024-04-02T13:59:02.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.234+0000 7f8db2b91740 1 --2- 172.21.15.52:0/2999082450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6f53cdd0 0x559e6f539f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.234+0000 7f8db2b91740 1 --2- 172.21.15.52:0/2999082450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6f5341f0 0x559e6f539a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.234+0000 7f8db2b91740 1 --2- 172.21.15.52:0/2999082450 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x559e6f54cd60 0x559e6f54f1b0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.234+0000 7f8db2b91740 1 --2- 172.21.15.52:0/2999082450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5338d0 0x559e6f53c690 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.234+0000 7f8db2b91740 1 -- 172.21.15.52:0/2999082450 >> 172.21.15.52:0/2999082450 conn(0x559e6f52e000 msgr2=0x559e6f52fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:02.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.234+0000 7f8d917fa640 0 client.17154 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:02.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.235+0000 7f8db2b91740 1 -- 172.21.15.52:0/2999082450 shutdown_connections 2024-04-02T13:59:02.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.235+0000 7f8db2b91740 1 -- 172.21.15.52:0/2999082450 wait complete. 2024-04-02T13:59:02.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.235+0000 7f8db2b91740 20 client.17154 trim_cache size 0 max 16384 2024-04-02T13:59:02.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.252+0000 7f8db2b91740 1 Processor -- start 2024-04-02T13:59:02.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.252+0000 7f8db2b91740 1 -- start start 2024-04-02T13:59:02.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.252+0000 7f8db2b91740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5338d0 0x559e6f548ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.252+0000 7f8db2b91740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6f5341f0 0x559e6f538960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.252+0000 7f8db2b91740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6f53cdd0 0x559e6f539c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.252+0000 7f8db2b91740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8da81aee70 con 0x559e6f5338d0 2024-04-02T13:59:02.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.252+0000 7f8db2b91740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8da81b09b0 con 0x559e6f53cdd0 2024-04-02T13:59:02.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.252+0000 7f8db2b91740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8da81af540 con 0x559e6f5341f0 2024-04-02T13:59:02.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.253+0000 7f8dad46e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5338d0 0x559e6f548ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.253+0000 7f8dad46e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5338d0 0x559e6f548ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58580/0 (socket says 172.21.15.52:58580) 2024-04-02T13:59:02.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.253+0000 7f8dad46e640 1 -- 172.21.15.52:0/3934264085 learned_addr learned my addr 172.21.15.52:0/3934264085 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:02.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.253+0000 7f8dadc6f640 1 --2- 172.21.15.52:0/3934264085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6f5341f0 0x559e6f538960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.253+0000 7f8dad46e640 1 --2- 172.21.15.52:0/3934264085 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6f53cdd0 0x559e6f539c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.253+0000 7f8dad46e640 1 -- 172.21.15.52:0/3934264085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6f5341f0 msgr2=0x559e6f538960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:02.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.253+0000 7f8dad46e640 1 --2- 172.21.15.52:0/3934264085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6f5341f0 0x559e6f538960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.253+0000 7f8dad46e640 1 -- 172.21.15.52:0/3934264085 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6f53cdd0 msgr2=0x559e6f539c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:02.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.253+0000 7f8dad46e640 1 --2- 172.21.15.52:0/3934264085 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6f53cdd0 0x559e6f539c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.253+0000 7f8dad46e640 1 -- 172.21.15.52:0/3934264085 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e6f52de30 con 0x559e6f5338d0 2024-04-02T13:59:02.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.253+0000 7f8dadc6f640 1 --2- 172.21.15.52:0/3934264085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6f5341f0 0x559e6f538960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:59:02.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.253+0000 7f8dad46e640 1 --2- 172.21.15.52:0/3934264085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5338d0 0x559e6f548ab0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f8d98005cd0 tx=0x7f8d98004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da69e5c3d333272f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:02.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.253+0000 7f8d937fe640 1 -- 172.21.15.52:0/3934264085 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d98010d90 con 0x559e6f5338d0 2024-04-02T13:59:02.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.253+0000 7f8d937fe640 1 -- 172.21.15.52:0/3934264085 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d980054e0 con 0x559e6f5338d0 2024-04-02T13:59:02.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.253+0000 7f8d937fe640 1 -- 172.21.15.52:0/3934264085 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d98004c60 con 0x559e6f5338d0 2024-04-02T13:59:02.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.254+0000 7f8db2b91740 1 -- 172.21.15.52:0/3934264085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5338d0 msgr2=0x559e6f548ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.254+0000 7f8db2b91740 1 --2- 172.21.15.52:0/3934264085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5338d0 0x559e6f548ab0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f8d98005cd0 tx=0x7f8d98004560 comp rx=0 tx=0).stop 2024-04-02T13:59:02.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.254+0000 7f8db2b91740 1 -- 172.21.15.52:0/3934264085 shutdown_connections 2024-04-02T13:59:02.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.254+0000 7f8db2b91740 1 --2- 172.21.15.52:0/3934264085 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6f53cdd0 0x559e6f539c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.254+0000 7f8db2b91740 1 --2- 172.21.15.52:0/3934264085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6f5341f0 0x559e6f538960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.254+0000 7f8db2b91740 1 --2- 172.21.15.52:0/3934264085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5338d0 0x559e6f548ab0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.254+0000 7f8db2b91740 1 -- 172.21.15.52:0/3934264085 >> 172.21.15.52:0/3934264085 conn(0x559e6f52e000 msgr2=0x559e6f52d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:02.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.254+0000 7f8db2b91740 1 -- 172.21.15.52:0/3934264085 shutdown_connections 2024-04-02T13:59:02.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.254+0000 7f8db2b91740 1 -- 172.21.15.52:0/3934264085 wait complete. 2024-04-02T13:59:02.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.254+0000 7f8db2b91740 1 -- start start 2024-04-02T13:59:02.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.254+0000 7f8db2b91740 1 Processor -- start 2024-04-02T13:59:02.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.255+0000 7f8db2b91740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5338d0 0x559e6f5ab650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.255+0000 7f8db2b91740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6f5341f0 0x559e6f5abb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.255+0000 7f8db2b91740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6f53cdd0 0x559e6f540390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.255+0000 7f8db2b91740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8da81b02e0 con 0x559e6f5338d0 2024-04-02T13:59:02.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.255+0000 7f8db2b91740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8da81afc10 con 0x559e6f53cdd0 2024-04-02T13:59:02.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.255+0000 7f8db2b91740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8da81ace50 con 0x559e6f5341f0 2024-04-02T13:59:02.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.255+0000 7f8db2b91740 10 client.0 _ll_register_callbacks cb 0x7f8d98013610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:02.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.255+0000 7f8dad46e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5338d0 0x559e6f5ab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.255+0000 7f8dad46e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5338d0 0x559e6f5ab650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58584/0 (socket says 172.21.15.52:58584) 2024-04-02T13:59:02.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.255+0000 7f8dad46e640 1 -- 172.21.15.52:0/1029737159 learned_addr learned my addr 172.21.15.52:0/1029737159 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:02.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.255+0000 7f8dadc6f640 1 --2- 172.21.15.52:0/1029737159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6f5341f0 0x559e6f5abb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.255+0000 7f8dad46e640 1 -- 172.21.15.52:0/1029737159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6f5341f0 msgr2=0x559e6f5abb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:02.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.255+0000 7f8dad46e640 1 --2- 172.21.15.52:0/1029737159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6f5341f0 0x559e6f5abb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.255+0000 7f8dad46e640 1 -- 172.21.15.52:0/1029737159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6f53cdd0 msgr2=0x559e6f540390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:02.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.255+0000 7f8dad46e640 1 --2- 172.21.15.52:0/1029737159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6f53cdd0 0x559e6f540390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.255+0000 7f8dad46e640 1 -- 172.21.15.52:0/1029737159 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e6f53f1c0 con 0x559e6f5338d0 2024-04-02T13:59:02.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.255+0000 7f8dad46e640 1 --2- 172.21.15.52:0/1029737159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5338d0 0x559e6f5ab650 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f8d98005ad0 tx=0x7f8d98013030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def18b8d8c3f7639 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:02.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.255+0000 7f8d917fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:02.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.255+0000 7f8d917fa640 1 -- 172.21.15.52:0/1029737159 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d98040060 con 0x559e6f5338d0 2024-04-02T13:59:02.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.255+0000 7f8d917fa640 1 -- 172.21.15.52:0/1029737159 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f8d980048e0 con 0x559e6f5338d0 2024-04-02T13:59:02.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.255+0000 7f8db2b91740 1 -- 172.21.15.52:0/1029737159 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d980054e0 con 0x559e6f5338d0 2024-04-02T13:59:02.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.255+0000 7f8d917fa640 1 -- 172.21.15.52:0/1029737159 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8d98004660 con 0x559e6f5338d0 2024-04-02T13:59:02.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.256+0000 7f8db2b91740 10 client.17160 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:02.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.256+0000 7f8db2b91740 1 -- 172.21.15.52:0/1029737159 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559e6f5b2c40 con 0x559e6f5338d0 2024-04-02T13:59:02.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.256+0000 7f8d917fa640 1 -- 172.21.15.52:0/1029737159 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d980054e0 con 0x559e6f5338d0 2024-04-02T13:59:02.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.256+0000 7f8d917fa640 1 -- 172.21.15.52:0/1029737159 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8d9801c040 con 0x559e6f5338d0 2024-04-02T13:59:02.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.256+0000 7f8db2b91740 10 client.17160 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:02.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.256+0000 7f8db2b91740 10 client.17160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:59:02.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.256+0000 7f8db2b91740 20 client.17160 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:02.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.256+0000 7f8db2b91740 1 --2- 172.21.15.52:0/1029737159 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x559e6f5b32a0 0x559e6f5b5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.256+0000 7f8db2b91740 4 client.17160 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:59:02.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.256+0000 7f8db2b91740 1 -- 172.21.15.52:0/1029737159 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559e6f49f0c0 con 0x559e6f5b32a0 2024-04-02T13:59:02.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.256+0000 7f8dadc6f640 1 --2- 172.21.15.52:0/1029737159 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x559e6f5b32a0 0x559e6f5b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.256+0000 7f8d917fa640 10 client.17160 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:02.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.256+0000 7f8dadc6f640 1 --2- 172.21.15.52:0/1029737159 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x559e6f5b32a0 0x559e6f5b5690 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e437ca7884cc3bb server_cookie=894e9c8cead7c025 in_seq=0 out_seq=0 2024-04-02T13:59:02.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.257+0000 7f8d917fa640 1 -- 172.21.15.52:0/1029737159 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559e6f49f0c0 con 0x559e6f5b32a0 2024-04-02T13:59:02.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.257+0000 7f8d917fa640 10 client.17160 handle_command_reply: tid=0 2024-04-02T13:59:02.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.257+0000 7f8db2b91740 1 client.17160 shutdown 2024-04-02T13:59:02.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.257+0000 7f8db2b91740 1 -- 172.21.15.52:0/1029737159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5338d0 msgr2=0x559e6f5ab650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.257+0000 7f8db2b91740 1 --2- 172.21.15.52:0/1029737159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5338d0 0x559e6f5ab650 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f8d98005ad0 tx=0x7f8d98013030 comp rx=0 tx=0).stop 2024-04-02T13:59:02.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.257+0000 7f8db2b91740 1 -- 172.21.15.52:0/1029737159 shutdown_connections 2024-04-02T13:59:02.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.257+0000 7f8db2b91740 1 --2- 172.21.15.52:0/1029737159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559e6f53cdd0 0x559e6f540390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.257+0000 7f8db2b91740 1 --2- 172.21.15.52:0/1029737159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e6f5341f0 0x559e6f5abb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.257+0000 7f8db2b91740 1 --2- 172.21.15.52:0/1029737159 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x559e6f5b32a0 0x559e6f5b5690 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.257+0000 7f8db2b91740 1 --2- 172.21.15.52:0/1029737159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e6f5338d0 0x559e6f5ab650 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.257+0000 7f8d917fa640 0 client.17160 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:02.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.257+0000 7f8db2b91740 1 -- 172.21.15.52:0/1029737159 >> 172.21.15.52:0/1029737159 conn(0x559e6f52e000 msgr2=0x559e6f534710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:02.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.257+0000 7f8db2b91740 1 -- 172.21.15.52:0/1029737159 shutdown_connections 2024-04-02T13:59:02.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.257+0000 7f8db2b91740 1 -- 172.21.15.52:0/1029737159 wait complete. 2024-04-02T13:59:02.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.257+0000 7f8db2b91740 20 client.17160 trim_cache size 0 max 16384 2024-04-02T13:59:02.271 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:59:02.272 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-04-02T13:59:02.272 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-04-02T13:59:02.272 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:59:02.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.258+0000 7f8dafed3640 1 -- 172.21.15.52:0/2985844202 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f8d940a3d20 msgr2=0x7f8d940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.258+0000 7f8dafed3640 1 --2- 172.21.15.52:0/2985844202 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f8d940a3d20 0x7f8d940a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8da00bea50 tx=0x7f8da0090790 comp rx=0 tx=0).stop 2024-04-02T13:59:02.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.258+0000 7f8dafed3640 1 -- 172.21.15.52:0/2985844202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8da8107aa0 msgr2=0x7f8da819ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.258+0000 7f8dafed3640 1 --2- 172.21.15.52:0/2985844202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8da8107aa0 0x7f8da819ccd0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f8da0037040 tx=0x7f8da0035ae0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.258+0000 7f8dafed3640 1 -- 172.21.15.52:0/2985844202 shutdown_connections 2024-04-02T13:59:02.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.258+0000 7f8dafed3640 1 --2- 172.21.15.52:0/2985844202 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f8d940a3d20 0x7f8d940a61e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.258+0000 7f8dafed3640 1 --2- 172.21.15.52:0/2985844202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8da8107aa0 0x7f8da819ccd0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.258+0000 7f8dafed3640 1 --2- 172.21.15.52:0/2985844202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8da8102450 0x7f8da8198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.258+0000 7f8dafed3640 1 --2- 172.21.15.52:0/2985844202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8da8101a90 0x7f8da8198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.258+0000 7f8dafed3640 1 -- 172.21.15.52:0/2985844202 >> 172.21.15.52:0/2985844202 conn(0x7f8da80fd5a0 msgr2=0x7f8da80fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:02.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.258+0000 7f8dafed3640 1 -- 172.21.15.52:0/2985844202 shutdown_connections 2024-04-02T13:59:02.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.258+0000 7f8dafed3640 1 -- 172.21.15.52:0/2985844202 wait complete. 2024-04-02T13:59:02.275 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:59:02.275 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:59:02.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.374+0000 7fd6181af640 1 -- 172.21.15.52:0/2671687429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd610107b90 msgr2=0x7fd610107f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.374+0000 7fd6181af640 1 --2- 172.21.15.52:0/2671687429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd610107b90 0x7fd610107f70 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009900 tx=0x7fd60c02bcd0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.374+0000 7fd6181af640 1 -- 172.21.15.52:0/2671687429 shutdown_connections 2024-04-02T13:59:02.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.374+0000 7fd6181af640 1 --2- 172.21.15.52:0/2671687429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd61006ab30 0x7fd61010e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.374+0000 7fd6181af640 1 --2- 172.21.15.52:0/2671687429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd61006a190 0x7fd61006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.374+0000 7fd6181af640 1 --2- 172.21.15.52:0/2671687429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd610107b90 0x7fd610107f70 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.374+0000 7fd6181af640 1 -- 172.21.15.52:0/2671687429 >> 172.21.15.52:0/2671687429 conn(0x7fd610077aa0 msgr2=0x7fd610077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:02.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.374+0000 7fd6181af640 1 -- 172.21.15.52:0/2671687429 shutdown_connections 2024-04-02T13:59:02.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.374+0000 7fd6181af640 1 -- 172.21.15.52:0/2671687429 wait complete. 2024-04-02T13:59:02.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.375+0000 7fd6181af640 1 Processor -- start 2024-04-02T13:59:02.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.375+0000 7fd6181af640 1 -- start start 2024-04-02T13:59:02.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.375+0000 7fd6181af640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd61006a190 0x7fd610198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.375+0000 7fd6181af640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd61006ab30 0x7fd610198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.375+0000 7fd6181af640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd610107b90 0x7fd61019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.375+0000 7fd6181af640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd610111ec0 con 0x7fd61006ab30 2024-04-02T13:59:02.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.375+0000 7fd6181af640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd610111d40 con 0x7fd610107b90 2024-04-02T13:59:02.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.375+0000 7fd6181af640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd610112040 con 0x7fd61006a190 2024-04-02T13:59:02.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.375+0000 7fd61574a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd61006ab30 0x7fd610198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.375+0000 7fd61574a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd61006ab30 0x7fd610198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58596/0 (socket says 172.21.15.52:58596) 2024-04-02T13:59:02.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.375+0000 7fd61574a640 1 -- 172.21.15.52:0/654397767 learned_addr learned my addr 172.21.15.52:0/654397767 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:02.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.375+0000 7fd615f4b640 1 --2- 172.21.15.52:0/654397767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd61006a190 0x7fd610198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.375+0000 7fd61674c640 1 --2- 172.21.15.52:0/654397767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd610107b90 0x7fd61019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.375+0000 7fd61574a640 1 -- 172.21.15.52:0/654397767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd61006a190 msgr2=0x7fd610198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.375+0000 7fd61574a640 1 --2- 172.21.15.52:0/654397767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd61006a190 0x7fd610198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.375+0000 7fd61574a640 1 -- 172.21.15.52:0/654397767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd610107b90 msgr2=0x7fd61019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.375+0000 7fd61574a640 1 --2- 172.21.15.52:0/654397767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd610107b90 0x7fd61019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.376+0000 7fd61574a640 1 -- 172.21.15.52:0/654397767 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61019d3a0 con 0x7fd61006ab30 2024-04-02T13:59:02.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.376+0000 7fd615f4b640 1 --2- 172.21.15.52:0/654397767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd61006a190 0x7fd610198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:59:02.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.376+0000 7fd61674c640 1 --2- 172.21.15.52:0/654397767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd610107b90 0x7fd61019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:59:02.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.376+0000 7fd61574a640 1 --2- 172.21.15.52:0/654397767 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd61006ab30 0x7fd610198940 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fd60000e9f0 tx=0x7fd60000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:02.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.376+0000 7fd606ffd640 1 -- 172.21.15.52:0/654397767 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60000cd80 con 0x7fd61006ab30 2024-04-02T13:59:02.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.376+0000 7fd6181af640 1 -- 172.21.15.52:0/654397767 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61019d690 con 0x7fd61006ab30 2024-04-02T13:59:02.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.376+0000 7fd606ffd640 1 -- 172.21.15.52:0/654397767 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd600010910 con 0x7fd61006ab30 2024-04-02T13:59:02.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.376+0000 7fd606ffd640 1 -- 172.21.15.52:0/654397767 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6000188f0 con 0x7fd61006ab30 2024-04-02T13:59:02.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.376+0000 7fd6181af640 1 -- 172.21.15.52:0/654397767 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6101a4f20 con 0x7fd61006ab30 2024-04-02T13:59:02.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.377+0000 7fd6181af640 1 -- 172.21.15.52:0/654397767 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd610068d40 con 0x7fd61006ab30 2024-04-02T13:59:02.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.378+0000 7fd606ffd640 1 -- 172.21.15.52:0/654397767 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd600010ac0 con 0x7fd61006ab30 2024-04-02T13:59:02.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.378+0000 7fd606ffd640 1 --2- 172.21.15.52:0/654397767 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd5fc0a3d20 0x7fd5fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.379+0000 7fd615f4b640 1 --2- 172.21.15.52:0/654397767 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd5fc0a3d20 0x7fd5fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.379+0000 7fd606ffd640 1 -- 172.21.15.52:0/654397767 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd6000d1430 con 0x7fd61006ab30 2024-04-02T13:59:02.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.379+0000 7fd615f4b640 1 --2- 172.21.15.52:0/654397767 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd5fc0a3d20 0x7fd5fc0a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd60c005ab0 tx=0x7fd60c005a20 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:02.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.382+0000 7fd606ffd640 1 -- 172.21.15.52:0/654397767 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd60009d730 con 0x7fd61006ab30 2024-04-02T13:59:02.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.614+0000 7fd6181af640 1 -- 172.21.15.52:0/654397767 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd61002d370 con 0x7fd61006ab30 2024-04-02T13:59:02.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.615+0000 7fd606ffd640 1 -- 172.21.15.52:0/654397767 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11923 (secure 0 0 0) 0x7fd60009d030 con 0x7fd61006ab30 2024-04-02T13:59:02.619 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:59:02.619 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:57:45.240902+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24467,24457,24490,24475,24503]},"id":1}]} 2024-04-02T13:59:02.619 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-04-02T13:59:02.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.617+0000 7fd6181af640 1 -- 172.21.15.52:0/654397767 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd5fc0a3d20 msgr2=0x7fd5fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.617+0000 7fd6181af640 1 --2- 172.21.15.52:0/654397767 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd5fc0a3d20 0x7fd5fc0a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd60c005ab0 tx=0x7fd60c005a20 comp rx=0 tx=0).stop 2024-04-02T13:59:02.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.617+0000 7fd6181af640 1 -- 172.21.15.52:0/654397767 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd61006ab30 msgr2=0x7fd610198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.617+0000 7fd6181af640 1 --2- 172.21.15.52:0/654397767 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd61006ab30 0x7fd610198940 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fd60000e9f0 tx=0x7fd60000eeb0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.617+0000 7fd6181af640 1 -- 172.21.15.52:0/654397767 shutdown_connections 2024-04-02T13:59:02.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.617+0000 7fd6181af640 1 --2- 172.21.15.52:0/654397767 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd5fc0a3d20 0x7fd5fc0a61e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.617+0000 7fd6181af640 1 --2- 172.21.15.52:0/654397767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd610107b90 0x7fd61019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.617+0000 7fd6181af640 1 --2- 172.21.15.52:0/654397767 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd61006ab30 0x7fd610198940 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.617+0000 7fd6181af640 1 --2- 172.21.15.52:0/654397767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd61006a190 0x7fd610198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.617+0000 7fd6181af640 1 -- 172.21.15.52:0/654397767 >> 172.21.15.52:0/654397767 conn(0x7fd610077aa0 msgr2=0x7fd6100ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:02.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.617+0000 7fd6181af640 1 -- 172.21.15.52:0/654397767 shutdown_connections 2024-04-02T13:59:02.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.617+0000 7fd6181af640 1 -- 172.21.15.52:0/654397767 wait complete. 2024-04-02T13:59:02.632 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:59:02.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.733+0000 7eff78bcd640 1 -- 172.21.15.52:0/1862613408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff74102460 msgr2=0x7eff7410e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.733+0000 7eff78bcd640 1 --2- 172.21.15.52:0/1862613408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff74102460 0x7eff7410e970 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7eff64009cd0 tx=0x7eff6402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.733+0000 7eff78bcd640 1 -- 172.21.15.52:0/1862613408 shutdown_connections 2024-04-02T13:59:02.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.733+0000 7eff78bcd640 1 --2- 172.21.15.52:0/1862613408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff74102460 0x7eff7410e970 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.733+0000 7eff78bcd640 1 --2- 172.21.15.52:0/1862613408 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff74101aa0 0x7eff74101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.734+0000 7eff78bcd640 1 --2- 172.21.15.52:0/1862613408 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff74107ab0 0x7eff74107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.734+0000 7eff78bcd640 1 -- 172.21.15.52:0/1862613408 >> 172.21.15.52:0/1862613408 conn(0x7eff740fd530 msgr2=0x7eff740ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:02.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.734+0000 7eff78bcd640 1 -- 172.21.15.52:0/1862613408 shutdown_connections 2024-04-02T13:59:02.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.734+0000 7eff78bcd640 1 -- 172.21.15.52:0/1862613408 wait complete. 2024-04-02T13:59:02.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.734+0000 7eff78bcd640 1 Processor -- start 2024-04-02T13:59:02.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.734+0000 7eff78bcd640 1 -- start start 2024-04-02T13:59:02.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.734+0000 7eff78bcd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff74101aa0 0x7eff74198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.734+0000 7eff78bcd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff74102460 0x7eff741988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.734+0000 7eff78bcd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff74107ab0 0x7eff7419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.734+0000 7eff78bcd640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7eff74111ec0 con 0x7eff74107ab0 2024-04-02T13:59:02.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.734+0000 7eff78bcd640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7eff74111d40 con 0x7eff74101aa0 2024-04-02T13:59:02.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.734+0000 7eff78bcd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7eff74112040 con 0x7eff74102460 2024-04-02T13:59:02.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.735+0000 7eff72d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff74107ab0 0x7eff7419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.735+0000 7eff72d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff74107ab0 0x7eff7419cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58616/0 (socket says 172.21.15.52:58616) 2024-04-02T13:59:02.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.735+0000 7eff72d9d640 1 -- 172.21.15.52:0/3266373224 learned_addr learned my addr 172.21.15.52:0/3266373224 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:02.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.735+0000 7eff71d9b640 1 --2- 172.21.15.52:0/3266373224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff74102460 0x7eff741988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.735+0000 7eff7259c640 1 --2- 172.21.15.52:0/3266373224 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff74101aa0 0x7eff74198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.735+0000 7eff72d9d640 1 -- 172.21.15.52:0/3266373224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff74102460 msgr2=0x7eff741988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.735+0000 7eff72d9d640 1 --2- 172.21.15.52:0/3266373224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff74102460 0x7eff741988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.735+0000 7eff72d9d640 1 -- 172.21.15.52:0/3266373224 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff74101aa0 msgr2=0x7eff74198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.735+0000 7eff72d9d640 1 --2- 172.21.15.52:0/3266373224 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff74101aa0 0x7eff74198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.735+0000 7eff72d9d640 1 -- 172.21.15.52:0/3266373224 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff7419d370 con 0x7eff74107ab0 2024-04-02T13:59:02.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.735+0000 7eff7259c640 1 --2- 172.21.15.52:0/3266373224 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff74101aa0 0x7eff74198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:59:02.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.735+0000 7eff72d9d640 1 --2- 172.21.15.52:0/3266373224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff74107ab0 0x7eff7419cc40 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7eff64004af0 tx=0x7eff64032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:02.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.735+0000 7eff6b7fe640 1 -- 172.21.15.52:0/3266373224 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff64041020 con 0x7eff74107ab0 2024-04-02T13:59:02.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.735+0000 7eff78bcd640 1 -- 172.21.15.52:0/3266373224 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff7419d600 con 0x7eff74107ab0 2024-04-02T13:59:02.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.735+0000 7eff6b7fe640 1 -- 172.21.15.52:0/3266373224 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff64034cd0 con 0x7eff74107ab0 2024-04-02T13:59:02.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.735+0000 7eff6b7fe640 1 -- 172.21.15.52:0/3266373224 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff64039040 con 0x7eff74107ab0 2024-04-02T13:59:02.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.735+0000 7eff78bcd640 1 -- 172.21.15.52:0/3266373224 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff741a4ef0 con 0x7eff74107ab0 2024-04-02T13:59:02.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.737+0000 7eff6b7fe640 1 -- 172.21.15.52:0/3266373224 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7eff64044020 con 0x7eff74107ab0 2024-04-02T13:59:02.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.738+0000 7eff7b7b1740 1 Processor -- start 2024-04-02T13:59:02.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.738+0000 7eff7b7b1740 1 -- start start 2024-04-02T13:59:02.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.738+0000 7eff7b7b1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da3c9237a0 0x55da3c923b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.738+0000 7eff7b7b1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3c9240c0 0x55da3c92c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.738+0000 7eff7b7b1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c92cca0 0x55da3c92f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.738+0000 7eff7b7b1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55da3c8877b0 con 0x55da3c92cca0 2024-04-02T13:59:02.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.738+0000 7eff7b7b1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55da3c826570 con 0x55da3c9237a0 2024-04-02T13:59:02.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.738+0000 7eff7b7b1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55da3c883760 con 0x55da3c9240c0 2024-04-02T13:59:02.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.738+0000 7eff71d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c92cca0 0x55da3c92f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.738+0000 7eff71d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c92cca0 0x55da3c92f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58620/0 (socket says 172.21.15.52:58620) 2024-04-02T13:59:02.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.738+0000 7eff71d9b640 1 -- 172.21.15.52:0/1674238371 learned_addr learned my addr 172.21.15.52:0/1674238371 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:02.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.738+0000 7eff6b7fe640 1 --2- 172.21.15.52:0/3266373224 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7eff600a3d20 0x7eff600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.738+0000 7eff71d9b640 1 --2- 172.21.15.52:0/1674238371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da3c9237a0 0x55da3c923b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.739+0000 7eff7259c640 1 --2- 172.21.15.52:0/1674238371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3c9240c0 0x55da3c92c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.739+0000 7eff72d9d640 1 --2- 172.21.15.52:0/3266373224 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7eff600a3d20 0x7eff600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.739+0000 7eff6b7fe640 1 -- 172.21.15.52:0/3266373224 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7eff640f2840 con 0x7eff74107ab0 2024-04-02T13:59:02.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.739+0000 7eff71d9b640 1 -- 172.21.15.52:0/1674238371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3c9240c0 msgr2=0x55da3c92c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:02.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.739+0000 7eff71d9b640 1 --2- 172.21.15.52:0/1674238371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3c9240c0 0x55da3c92c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.739+0000 7eff71d9b640 1 -- 172.21.15.52:0/1674238371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da3c9237a0 msgr2=0x55da3c923b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:02.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.739+0000 7eff71d9b640 1 --2- 172.21.15.52:0/1674238371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da3c9237a0 0x55da3c923b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.739+0000 7eff71d9b640 1 -- 172.21.15.52:0/1674238371 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da3c59b3f0 con 0x55da3c92cca0 2024-04-02T13:59:02.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.739+0000 7eff71d9b640 1 --2- 172.21.15.52:0/1674238371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c92cca0 0x55da3c92f090 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0127c0 tx=0x7eff5c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc72d6fdbd74ae0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:02.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.739+0000 7eff72d9d640 1 --2- 172.21.15.52:0/3266373224 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7eff600a3d20 0x7eff600a61e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7eff640ba9b0 tx=0x7eff64032b00 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:02.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.739+0000 7eff68ff9640 1 -- 172.21.15.52:0/1674238371 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c012e00 con 0x55da3c92cca0 2024-04-02T13:59:02.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.739+0000 7eff68ff9640 1 -- 172.21.15.52:0/1674238371 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff5c010c50 con 0x55da3c92cca0 2024-04-02T13:59:02.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.739+0000 7eff68ff9640 1 -- 172.21.15.52:0/1674238371 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c01ec40 con 0x55da3c92cca0 2024-04-02T13:59:02.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.740+0000 7eff7b7b1740 1 -- 172.21.15.52:0/1674238371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c92cca0 msgr2=0x55da3c92f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.740+0000 7eff7b7b1740 1 --2- 172.21.15.52:0/1674238371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c92cca0 0x55da3c92f090 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0127c0 tx=0x7eff5c00b7c0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.740+0000 7eff7b7b1740 1 -- 172.21.15.52:0/1674238371 shutdown_connections 2024-04-02T13:59:02.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.740+0000 7eff7b7b1740 1 --2- 172.21.15.52:0/1674238371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c92cca0 0x55da3c92f090 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.740+0000 7eff7b7b1740 1 --2- 172.21.15.52:0/1674238371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3c9240c0 0x55da3c92c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.740+0000 7eff7b7b1740 1 --2- 172.21.15.52:0/1674238371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da3c9237a0 0x55da3c923b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.740+0000 7eff7b7b1740 1 -- 172.21.15.52:0/1674238371 >> 172.21.15.52:0/1674238371 conn(0x55da3c91ded0 msgr2=0x55da3c91e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:02.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.740+0000 7eff7b7b1740 1 -- 172.21.15.52:0/1674238371 shutdown_connections 2024-04-02T13:59:02.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.740+0000 7eff7b7b1740 1 -- 172.21.15.52:0/1674238371 wait complete. 2024-04-02T13:59:02.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.740+0000 7eff7b7b1740 1 -- start start 2024-04-02T13:59:02.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.740+0000 7eff7b7b1740 1 Processor -- start 2024-04-02T13:59:02.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.741+0000 7eff7b7b1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3c9237a0 0x55da3c92c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.741+0000 7eff7b7b1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da3c9240c0 0x55da3c929af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.741+0000 7eff7b7b1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c92cca0 0x55da3c92a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.741+0000 7eff7b7b1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55da3c92fa90 con 0x55da3c92cca0 2024-04-02T13:59:02.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.741+0000 7eff7b7b1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55da3c842e80 con 0x55da3c9240c0 2024-04-02T13:59:02.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.741+0000 7eff7b7b1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55da3c92fc10 con 0x55da3c9237a0 2024-04-02T13:59:02.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.741+0000 7eff7b7b1740 10 client.0 _ll_register_callbacks cb 0x55da3c87eaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:02.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.741+0000 7eff71d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c92cca0 0x55da3c92a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.741+0000 7eff71d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c92cca0 0x55da3c92a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58624/0 (socket says 172.21.15.52:58624) 2024-04-02T13:59:02.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.741+0000 7eff71d9b640 1 -- 172.21.15.52:0/1039224483 learned_addr learned my addr 172.21.15.52:0/1039224483 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:02.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.741+0000 7eff71d9b640 1 --2- 172.21.15.52:0/1039224483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3c9237a0 0x55da3c92c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.741+0000 7eff7259c640 1 --2- 172.21.15.52:0/1039224483 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da3c9240c0 0x55da3c929af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.741+0000 7eff71d9b640 1 -- 172.21.15.52:0/1039224483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3c9237a0 msgr2=0x55da3c92c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:02.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.741+0000 7eff71d9b640 1 --2- 172.21.15.52:0/1039224483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3c9237a0 0x55da3c92c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.741+0000 7eff71d9b640 1 -- 172.21.15.52:0/1039224483 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da3c9240c0 msgr2=0x55da3c929af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:02.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.741+0000 7eff71d9b640 1 --2- 172.21.15.52:0/1039224483 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da3c9240c0 0x55da3c929af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.741+0000 7eff71d9b640 1 -- 172.21.15.52:0/1039224483 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da3c6657d0 con 0x55da3c92cca0 2024-04-02T13:59:02.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.741+0000 7eff71d9b640 1 --2- 172.21.15.52:0/1039224483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c92cca0 0x55da3c92a230 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7eff5c011010 tx=0x7eff5c01e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78776e6370049713 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:02.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.741+0000 7eff59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:02.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.741+0000 7eff59ffb640 1 -- 172.21.15.52:0/1039224483 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c02f060 con 0x55da3c92cca0 2024-04-02T13:59:02.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.741+0000 7eff59ffb640 1 -- 172.21.15.52:0/1039224483 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7eff5c012e00 con 0x55da3c92cca0 2024-04-02T13:59:02.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.741+0000 7eff7b7b1740 1 -- 172.21.15.52:0/1039224483 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff5c010c50 con 0x55da3c92cca0 2024-04-02T13:59:02.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.742+0000 7eff59ffb640 1 -- 172.21.15.52:0/1039224483 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7eff5c036050 con 0x55da3c92cca0 2024-04-02T13:59:02.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.742+0000 7eff7b7b1740 10 client.17178 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:02.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.742+0000 7eff7b7b1740 1 -- 172.21.15.52:0/1039224483 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55da3c93bb40 con 0x55da3c92cca0 2024-04-02T13:59:02.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.742+0000 7eff59ffb640 1 -- 172.21.15.52:0/1039224483 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff5c010c50 con 0x55da3c92cca0 2024-04-02T13:59:02.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.742+0000 7eff59ffb640 1 -- 172.21.15.52:0/1039224483 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7eff5c01f5e0 con 0x55da3c92cca0 2024-04-02T13:59:02.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.742+0000 7eff7b7b1740 10 client.17178 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:02.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.742+0000 7eff7b7b1740 10 client.17178 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T13:59:02.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.742+0000 7eff7b7b1740 20 client.17178 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:02.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.742+0000 7eff7b7b1740 1 --2- 172.21.15.52:0/1039224483 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55da3c93cef0 0x55da3c93f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.742+0000 7eff7b7b1740 4 client.17178 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:59:02.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.742+0000 7eff7b7b1740 1 -- 172.21.15.52:0/1039224483 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da3c894cf0 con 0x55da3c93cef0 2024-04-02T13:59:02.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.742+0000 7eff7259c640 1 --2- 172.21.15.52:0/1039224483 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55da3c93cef0 0x55da3c93f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.743+0000 7eff7259c640 1 --2- 172.21.15.52:0/1039224483 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55da3c93cef0 0x55da3c93f2e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=28c06451baf36128 server_cookie=b9e2e73f57ef22b8 in_seq=0 out_seq=0 2024-04-02T13:59:02.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.743+0000 7eff59ffb640 10 client.17178 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:59:02.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.743+0000 7eff59ffb640 1 -- 172.21.15.52:0/1039224483 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da3c894cf0 con 0x55da3c93cef0 2024-04-02T13:59:02.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.743+0000 7eff59ffb640 10 client.17178 handle_command_reply: tid=0 2024-04-02T13:59:02.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.743+0000 7eff7b7b1740 1 client.17178 shutdown 2024-04-02T13:59:02.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.744+0000 7eff7b7b1740 1 -- 172.21.15.52:0/1039224483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c92cca0 msgr2=0x55da3c92a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.744+0000 7eff7b7b1740 1 --2- 172.21.15.52:0/1039224483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c92cca0 0x55da3c92a230 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7eff5c011010 tx=0x7eff5c01e820 comp rx=0 tx=0).stop 2024-04-02T13:59:02.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.744+0000 7eff7b7b1740 1 -- 172.21.15.52:0/1039224483 shutdown_connections 2024-04-02T13:59:02.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.744+0000 7eff7b7b1740 1 --2- 172.21.15.52:0/1039224483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c92cca0 0x55da3c92a230 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.744+0000 7eff7b7b1740 1 --2- 172.21.15.52:0/1039224483 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da3c9240c0 0x55da3c929af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.744+0000 7eff7b7b1740 1 --2- 172.21.15.52:0/1039224483 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55da3c93cef0 0x55da3c93f2e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.744+0000 7eff7b7b1740 1 --2- 172.21.15.52:0/1039224483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3c9237a0 0x55da3c92c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.744+0000 7eff59ffb640 0 client.17178 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:59:02.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.744+0000 7eff7b7b1740 1 -- 172.21.15.52:0/1039224483 >> 172.21.15.52:0/1039224483 conn(0x55da3c91ded0 msgr2=0x55da3c91fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:02.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.744+0000 7eff7b7b1740 1 -- 172.21.15.52:0/1039224483 shutdown_connections 2024-04-02T13:59:02.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.744+0000 7eff7b7b1740 1 -- 172.21.15.52:0/1039224483 wait complete. 2024-04-02T13:59:02.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.744+0000 7eff7b7b1740 20 client.17178 trim_cache size 0 max 16384 2024-04-02T13:59:02.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.757+0000 7eff7b7b1740 1 Processor -- start 2024-04-02T13:59:02.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.757+0000 7eff7b7b1740 1 -- start start 2024-04-02T13:59:02.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.757+0000 7eff7b7b1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3c91ded0 0x55da3c938950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.757+0000 7eff7b7b1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c9237a0 0x55da3c91fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.757+0000 7eff7b7b1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da3c9240c0 0x55da3c92c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.757+0000 7eff7b7b1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7eff741acc90 con 0x55da3c9237a0 2024-04-02T13:59:02.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.757+0000 7eff7b7b1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7eff741ae7d0 con 0x55da3c9240c0 2024-04-02T13:59:02.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.757+0000 7eff7b7b1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7eff741ad360 con 0x55da3c91ded0 2024-04-02T13:59:02.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.757+0000 7eff7259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c9237a0 0x55da3c91fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.757+0000 7eff7259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c9237a0 0x55da3c91fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58636/0 (socket says 172.21.15.52:58636) 2024-04-02T13:59:02.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.757+0000 7eff7259c640 1 -- 172.21.15.52:0/3815111865 learned_addr learned my addr 172.21.15.52:0/3815111865 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:02.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.757+0000 7eff7259c640 1 -- 172.21.15.52:0/3815111865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3c91ded0 msgr2=0x55da3c938950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:02.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.757+0000 7eff7259c640 1 --2- 172.21.15.52:0/3815111865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3c91ded0 0x55da3c938950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.757+0000 7eff7259c640 1 -- 172.21.15.52:0/3815111865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da3c9240c0 msgr2=0x55da3c92c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:02.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.757+0000 7eff7259c640 1 --2- 172.21.15.52:0/3815111865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da3c9240c0 0x55da3c92c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.757+0000 7eff7259c640 1 -- 172.21.15.52:0/3815111865 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da3c923ef0 con 0x55da3c9237a0 2024-04-02T13:59:02.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.758+0000 7eff7259c640 1 --2- 172.21.15.52:0/3815111865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c9237a0 0x55da3c91fbb0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7eff6c015010 tx=0x7eff6c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e11f240802b103 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:02.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.758+0000 7eff5bfff640 1 -- 172.21.15.52:0/3815111865 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c005020 con 0x55da3c9237a0 2024-04-02T13:59:02.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.758+0000 7eff5bfff640 1 -- 172.21.15.52:0/3815111865 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff6c028030 con 0x55da3c9237a0 2024-04-02T13:59:02.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.758+0000 7eff5bfff640 1 -- 172.21.15.52:0/3815111865 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c0243d0 con 0x55da3c9237a0 2024-04-02T13:59:02.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.758+0000 7eff7b7b1740 1 -- 172.21.15.52:0/3815111865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c9237a0 msgr2=0x55da3c91fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.758+0000 7eff7b7b1740 1 --2- 172.21.15.52:0/3815111865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c9237a0 0x55da3c91fbb0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7eff6c015010 tx=0x7eff6c005f90 comp rx=0 tx=0).stop 2024-04-02T13:59:02.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.758+0000 7eff7b7b1740 1 -- 172.21.15.52:0/3815111865 shutdown_connections 2024-04-02T13:59:02.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.758+0000 7eff7b7b1740 1 --2- 172.21.15.52:0/3815111865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da3c9240c0 0x55da3c92c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.758+0000 7eff7b7b1740 1 --2- 172.21.15.52:0/3815111865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c9237a0 0x55da3c91fbb0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.758+0000 7eff7b7b1740 1 --2- 172.21.15.52:0/3815111865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3c91ded0 0x55da3c938950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.758+0000 7eff7b7b1740 1 -- 172.21.15.52:0/3815111865 >> 172.21.15.52:0/3815111865 conn(0x55da3c9254c0 msgr2=0x55da3c91e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:02.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.758+0000 7eff7b7b1740 1 -- 172.21.15.52:0/3815111865 shutdown_connections 2024-04-02T13:59:02.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.758+0000 7eff7b7b1740 1 -- 172.21.15.52:0/3815111865 wait complete. 2024-04-02T13:59:02.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.759+0000 7eff7b7b1740 1 -- start start 2024-04-02T13:59:02.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.759+0000 7eff7b7b1740 1 Processor -- start 2024-04-02T13:59:02.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.759+0000 7eff7b7b1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3c91ded0 0x55da3c9309e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.759+0000 7eff7b7b1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c9237a0 0x55da3c930f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.759+0000 7eff7b7b1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da3c9240c0 0x55da3c931460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.759+0000 7eff7b7b1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7eff741ae100 con 0x55da3c9237a0 2024-04-02T13:59:02.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.759+0000 7eff7b7b1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7eff741ada30 con 0x55da3c9240c0 2024-04-02T13:59:02.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.759+0000 7eff7b7b1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7eff741aac70 con 0x55da3c91ded0 2024-04-02T13:59:02.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.759+0000 7eff7b7b1740 10 client.0 _ll_register_callbacks cb 0x7eff5c00be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:02.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.759+0000 7eff7259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c9237a0 0x55da3c930f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.759+0000 7eff7259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c9237a0 0x55da3c930f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58642/0 (socket says 172.21.15.52:58642) 2024-04-02T13:59:02.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.759+0000 7eff7259c640 1 -- 172.21.15.52:0/2880273457 learned_addr learned my addr 172.21.15.52:0/2880273457 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:02.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.759+0000 7eff71d9b640 1 --2- 172.21.15.52:0/2880273457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3c91ded0 0x55da3c9309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.759+0000 7eff71d9b640 1 --2- 172.21.15.52:0/2880273457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da3c9240c0 0x55da3c931460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.759+0000 7eff7259c640 1 -- 172.21.15.52:0/2880273457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3c91ded0 msgr2=0x55da3c9309e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:02.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.759+0000 7eff7259c640 1 --2- 172.21.15.52:0/2880273457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3c91ded0 0x55da3c9309e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.759+0000 7eff7259c640 1 -- 172.21.15.52:0/2880273457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da3c9240c0 msgr2=0x55da3c931460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:02.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.759+0000 7eff7259c640 1 --2- 172.21.15.52:0/2880273457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da3c9240c0 0x55da3c931460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.759+0000 7eff7259c640 1 -- 172.21.15.52:0/2880273457 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da3c9a2420 con 0x55da3c9237a0 2024-04-02T13:59:02.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.759+0000 7eff71d9b640 1 --2- 172.21.15.52:0/2880273457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da3c9240c0 0x55da3c931460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:59:02.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.759+0000 7eff7259c640 1 --2- 172.21.15.52:0/2880273457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c9237a0 0x55da3c930f20 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7eff6c013e30 tx=0x7eff6c012e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de1cfc9d09c2759 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:02.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.759+0000 7eff59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:02.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.760+0000 7eff59ffb640 1 -- 172.21.15.52:0/2880273457 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c03a060 con 0x55da3c9237a0 2024-04-02T13:59:02.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.760+0000 7eff59ffb640 1 -- 172.21.15.52:0/2880273457 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7eff6c0048c0 con 0x55da3c9237a0 2024-04-02T13:59:02.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.760+0000 7eff7b7b1740 1 -- 172.21.15.52:0/2880273457 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff6c028030 con 0x55da3c9237a0 2024-04-02T13:59:02.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.760+0000 7eff59ffb640 1 -- 172.21.15.52:0/2880273457 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7eff6c041050 con 0x55da3c9237a0 2024-04-02T13:59:02.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.760+0000 7eff7b7b1740 10 client.17184 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:02.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.760+0000 7eff7b7b1740 1 -- 172.21.15.52:0/2880273457 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55da3c9a3700 con 0x55da3c9237a0 2024-04-02T13:59:02.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.760+0000 7eff59ffb640 1 -- 172.21.15.52:0/2880273457 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff6c028030 con 0x55da3c9237a0 2024-04-02T13:59:02.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.760+0000 7eff59ffb640 1 -- 172.21.15.52:0/2880273457 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7eff6c0245b0 con 0x55da3c9237a0 2024-04-02T13:59:02.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.760+0000 7eff7b7b1740 10 client.17184 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:02.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.760+0000 7eff7b7b1740 10 client.17184 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T13:59:02.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.760+0000 7eff7b7b1740 20 client.17184 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:02.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.760+0000 7eff7b7b1740 1 --2- 172.21.15.52:0/2880273457 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55da3c92cca0 0x55da3c9a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.760+0000 7eff7b7b1740 4 client.17184 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:59:02.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.760+0000 7eff7b7b1740 1 -- 172.21.15.52:0/2880273457 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da3c88f040 con 0x55da3c92cca0 2024-04-02T13:59:02.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.761+0000 7eff71d9b640 1 --2- 172.21.15.52:0/2880273457 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55da3c92cca0 0x55da3c9a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.761+0000 7eff71d9b640 1 --2- 172.21.15.52:0/2880273457 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55da3c92cca0 0x55da3c9a5e00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e1598ca8f6e09b7e server_cookie=326ccd1440657c73 in_seq=0 out_seq=0 2024-04-02T13:59:02.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.761+0000 7eff59ffb640 10 client.17184 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:59:02.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.761+0000 7eff59ffb640 1 -- 172.21.15.52:0/2880273457 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da3c88f040 con 0x55da3c92cca0 2024-04-02T13:59:02.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.761+0000 7eff59ffb640 10 client.17184 handle_command_reply: tid=0 2024-04-02T13:59:02.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.761+0000 7eff7b7b1740 1 client.17184 shutdown 2024-04-02T13:59:02.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.761+0000 7eff7b7b1740 1 -- 172.21.15.52:0/2880273457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c9237a0 msgr2=0x55da3c930f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.761+0000 7eff7b7b1740 1 --2- 172.21.15.52:0/2880273457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c9237a0 0x55da3c930f20 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7eff6c013e30 tx=0x7eff6c012e40 comp rx=0 tx=0).stop 2024-04-02T13:59:02.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.761+0000 7eff7b7b1740 1 -- 172.21.15.52:0/2880273457 shutdown_connections 2024-04-02T13:59:02.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.761+0000 7eff7b7b1740 1 --2- 172.21.15.52:0/2880273457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da3c9240c0 0x55da3c931460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.761+0000 7eff7b7b1740 1 --2- 172.21.15.52:0/2880273457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55da3c9237a0 0x55da3c930f20 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.761+0000 7eff7b7b1740 1 --2- 172.21.15.52:0/2880273457 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55da3c92cca0 0x55da3c9a5e00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.761+0000 7eff7b7b1740 1 --2- 172.21.15.52:0/2880273457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3c91ded0 0x55da3c9309e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.761+0000 7eff59ffb640 0 client.17184 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:59:02.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.761+0000 7eff7b7b1740 1 -- 172.21.15.52:0/2880273457 >> 172.21.15.52:0/2880273457 conn(0x55da3c9254c0 msgr2=0x55da3c928b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:02.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.761+0000 7eff7b7b1740 1 -- 172.21.15.52:0/2880273457 shutdown_connections 2024-04-02T13:59:02.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.761+0000 7eff7b7b1740 1 -- 172.21.15.52:0/2880273457 wait complete. 2024-04-02T13:59:02.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.762+0000 7eff7b7b1740 20 client.17184 trim_cache size 0 max 16384 2024-04-02T13:59:02.780 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:59:02.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.762+0000 7eff78bcd640 1 -- 172.21.15.52:0/3266373224 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7eff600a3d20 msgr2=0x7eff600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.762+0000 7eff78bcd640 1 --2- 172.21.15.52:0/3266373224 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7eff600a3d20 0x7eff600a61e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7eff640ba9b0 tx=0x7eff64032b00 comp rx=0 tx=0).stop 2024-04-02T13:59:02.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.762+0000 7eff78bcd640 1 -- 172.21.15.52:0/3266373224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff74107ab0 msgr2=0x7eff7419cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.762+0000 7eff78bcd640 1 --2- 172.21.15.52:0/3266373224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff74107ab0 0x7eff7419cc40 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7eff64004af0 tx=0x7eff64032ec0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.762+0000 7eff78bcd640 1 -- 172.21.15.52:0/3266373224 shutdown_connections 2024-04-02T13:59:02.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.762+0000 7eff78bcd640 1 --2- 172.21.15.52:0/3266373224 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7eff600a3d20 0x7eff600a61e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.762+0000 7eff78bcd640 1 --2- 172.21.15.52:0/3266373224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff74107ab0 0x7eff7419cc40 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.762+0000 7eff78bcd640 1 --2- 172.21.15.52:0/3266373224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff74102460 0x7eff741988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.762+0000 7eff78bcd640 1 --2- 172.21.15.52:0/3266373224 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff74101aa0 0x7eff74198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.762+0000 7eff78bcd640 1 -- 172.21.15.52:0/3266373224 >> 172.21.15.52:0/3266373224 conn(0x7eff740fd530 msgr2=0x7eff740fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:02.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.762+0000 7eff78bcd640 1 -- 172.21.15.52:0/3266373224 shutdown_connections 2024-04-02T13:59:02.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.762+0000 7eff78bcd640 1 -- 172.21.15.52:0/3266373224 wait complete. 2024-04-02T13:59:02.783 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:59:02.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.884+0000 7fa21510e640 1 -- 172.21.15.52:0/3207611468 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2101069c0 msgr2=0x7fa210106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.884+0000 7fa21510e640 1 --2- 172.21.15.52:0/3207611468 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2101069c0 0x7fa210106da0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fa200009930 tx=0x7fa20002bc30 comp rx=0 tx=0).stop 2024-04-02T13:59:02.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.884+0000 7fa21510e640 1 -- 172.21.15.52:0/3207611468 shutdown_connections 2024-04-02T13:59:02.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.884+0000 7fa21510e640 1 --2- 172.21.15.52:0/3207611468 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa210101370 0x7fa21010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.884+0000 7fa21510e640 1 --2- 172.21.15.52:0/3207611468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2101009b0 0x7fa210100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.884+0000 7fa21510e640 1 --2- 172.21.15.52:0/3207611468 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2101069c0 0x7fa210106da0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.884+0000 7fa21510e640 1 -- 172.21.15.52:0/3207611468 >> 172.21.15.52:0/3207611468 conn(0x7fa210076d80 msgr2=0x7fa210077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:02.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.884+0000 7fa21510e640 1 -- 172.21.15.52:0/3207611468 shutdown_connections 2024-04-02T13:59:02.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.884+0000 7fa21510e640 1 -- 172.21.15.52:0/3207611468 wait complete. 2024-04-02T13:59:02.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.885+0000 7fa21510e640 1 Processor -- start 2024-04-02T13:59:02.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.885+0000 7fa21510e640 1 -- start start 2024-04-02T13:59:02.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.885+0000 7fa21510e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2101009b0 0x7fa210198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.885+0000 7fa21510e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa210101370 0x7fa210198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.885+0000 7fa21510e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2101069c0 0x7fa21019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.885+0000 7fa21510e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa210110da0 con 0x7fa210101370 2024-04-02T13:59:02.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.885+0000 7fa21510e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa210110c20 con 0x7fa2101009b0 2024-04-02T13:59:02.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.885+0000 7fa21510e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa210110f20 con 0x7fa2101069c0 2024-04-02T13:59:02.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.885+0000 7fa20659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa210101370 0x7fa210198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.885+0000 7fa20659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa210101370 0x7fa210198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58660/0 (socket says 172.21.15.52:58660) 2024-04-02T13:59:02.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.885+0000 7fa20659c640 1 -- 172.21.15.52:0/224939872 learned_addr learned my addr 172.21.15.52:0/224939872 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:02.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.885+0000 7fa20ed9d640 1 --2- 172.21.15.52:0/224939872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2101009b0 0x7fa210198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.885+0000 7fa20f59e640 1 --2- 172.21.15.52:0/224939872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2101069c0 0x7fa21019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.885+0000 7fa20659c640 1 -- 172.21.15.52:0/224939872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2101069c0 msgr2=0x7fa21019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.886+0000 7fa20659c640 1 --2- 172.21.15.52:0/224939872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2101069c0 0x7fa21019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.886+0000 7fa20659c640 1 -- 172.21.15.52:0/224939872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2101009b0 msgr2=0x7fa210198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.886+0000 7fa20659c640 1 --2- 172.21.15.52:0/224939872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2101009b0 0x7fa210198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.886+0000 7fa20659c640 1 -- 172.21.15.52:0/224939872 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa21019d310 con 0x7fa210101370 2024-04-02T13:59:02.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.886+0000 7fa20ed9d640 1 --2- 172.21.15.52:0/224939872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2101009b0 0x7fa210198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:59:02.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.886+0000 7fa20659c640 1 --2- 172.21.15.52:0/224939872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa210101370 0x7fa210198860 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800d950 tx=0x7fa1f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:02.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.886+0000 7fa20cd99640 1 -- 172.21.15.52:0/224939872 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8017070 con 0x7fa210101370 2024-04-02T13:59:02.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.886+0000 7fa20cd99640 1 -- 172.21.15.52:0/224939872 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f8009c30 con 0x7fa210101370 2024-04-02T13:59:02.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.886+0000 7fa21510e640 1 -- 172.21.15.52:0/224939872 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa21019d600 con 0x7fa210101370 2024-04-02T13:59:02.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.886+0000 7fa20cd99640 1 -- 172.21.15.52:0/224939872 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f800f6b0 con 0x7fa210101370 2024-04-02T13:59:02.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.886+0000 7fa21510e640 1 -- 172.21.15.52:0/224939872 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2101a4e90 con 0x7fa210101370 2024-04-02T13:59:02.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.889+0000 7fa217dd2740 1 Processor -- start 2024-04-02T13:59:02.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.889+0000 7fa20cd99640 1 -- 172.21.15.52:0/224939872 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa1f80109d0 con 0x7fa210101370 2024-04-02T13:59:02.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.889+0000 7fa217dd2740 1 -- start start 2024-04-02T13:59:02.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.889+0000 7fa217dd2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559bd15cc7a0 0x559bd15ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.889+0000 7fa217dd2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bd15cd0c0 0x559bd15d5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.889+0000 7fa217dd2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15d5ca0 0x559bd15d8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.889+0000 7fa217dd2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559bd15307b0 con 0x559bd15d5ca0 2024-04-02T13:59:02.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.889+0000 7fa217dd2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559bd14cf570 con 0x559bd15cc7a0 2024-04-02T13:59:02.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.889+0000 7fa217dd2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559bd152c760 con 0x559bd15cd0c0 2024-04-02T13:59:02.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.889+0000 7fa20659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15d5ca0 0x559bd15d8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.889+0000 7fa20659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15d5ca0 0x559bd15d8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58662/0 (socket says 172.21.15.52:58662) 2024-04-02T13:59:02.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.889+0000 7fa20659c640 1 -- 172.21.15.52:0/1701343316 learned_addr learned my addr 172.21.15.52:0/1701343316 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:02.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.889+0000 7fa20f59e640 1 --2- 172.21.15.52:0/1701343316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559bd15cc7a0 0x559bd15ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.890+0000 7fa20ed9d640 1 --2- 172.21.15.52:0/1701343316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bd15cd0c0 0x559bd15d5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.890+0000 7fa20cd99640 1 --2- 172.21.15.52:0/224939872 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa1fc0a3d20 0x7fa1fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.890+0000 7fa20659c640 1 -- 172.21.15.52:0/1701343316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bd15cd0c0 msgr2=0x559bd15d5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:02.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.890+0000 7fa20659c640 1 --2- 172.21.15.52:0/1701343316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bd15cd0c0 0x559bd15d5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.890+0000 7fa20659c640 1 -- 172.21.15.52:0/1701343316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559bd15cc7a0 msgr2=0x559bd15ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:02.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.890+0000 7fa20659c640 1 --2- 172.21.15.52:0/1701343316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559bd15cc7a0 0x559bd15ccb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.890+0000 7fa20659c640 1 -- 172.21.15.52:0/1701343316 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bd12443f0 con 0x559bd15d5ca0 2024-04-02T13:59:02.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.890+0000 7fa20f59e640 1 --2- 172.21.15.52:0/224939872 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa1fc0a3d20 0x7fa1fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.890+0000 7fa20cd99640 1 -- 172.21.15.52:0/224939872 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fa1f80d0820 con 0x7fa210101370 2024-04-02T13:59:02.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.890+0000 7fa20659c640 1 --2- 172.21.15.52:0/1701343316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15d5ca0 0x559bd15d8090 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80cdba0 tx=0x7fa1f8091910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd25cd1284e08e0e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:02.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.890+0000 7fa20559a640 1 -- 172.21.15.52:0/1701343316 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f809bfc0 con 0x559bd15d5ca0 2024-04-02T13:59:02.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.890+0000 7fa20559a640 1 -- 172.21.15.52:0/1701343316 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f8009de0 con 0x559bd15d5ca0 2024-04-02T13:59:02.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.890+0000 7fa20f59e640 1 --2- 172.21.15.52:0/224939872 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa1fc0a3d20 0x7fa1fc0a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa208005fd0 tx=0x7fa208004a40 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:02.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.890+0000 7fa20559a640 1 -- 172.21.15.52:0/1701343316 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f809bde0 con 0x559bd15d5ca0 2024-04-02T13:59:02.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.891+0000 7fa217dd2740 1 -- 172.21.15.52:0/1701343316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15d5ca0 msgr2=0x559bd15d8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.891+0000 7fa217dd2740 1 --2- 172.21.15.52:0/1701343316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15d5ca0 0x559bd15d8090 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80cdba0 tx=0x7fa1f8091910 comp rx=0 tx=0).stop 2024-04-02T13:59:02.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.891+0000 7fa217dd2740 1 -- 172.21.15.52:0/1701343316 shutdown_connections 2024-04-02T13:59:02.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.891+0000 7fa217dd2740 1 --2- 172.21.15.52:0/1701343316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15d5ca0 0x559bd15d8090 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.891+0000 7fa217dd2740 1 --2- 172.21.15.52:0/1701343316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bd15cd0c0 0x559bd15d5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.891+0000 7fa217dd2740 1 --2- 172.21.15.52:0/1701343316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559bd15cc7a0 0x559bd15ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.891+0000 7fa217dd2740 1 -- 172.21.15.52:0/1701343316 >> 172.21.15.52:0/1701343316 conn(0x559bd15c6ed0 msgr2=0x559bd15c72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:02.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.891+0000 7fa217dd2740 1 -- 172.21.15.52:0/1701343316 shutdown_connections 2024-04-02T13:59:02.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.891+0000 7fa217dd2740 1 -- 172.21.15.52:0/1701343316 wait complete. 2024-04-02T13:59:02.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.891+0000 7fa217dd2740 1 -- start start 2024-04-02T13:59:02.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.892+0000 7fa217dd2740 1 Processor -- start 2024-04-02T13:59:02.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.892+0000 7fa217dd2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559bd15cc7a0 0x559bd15d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.892+0000 7fa217dd2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15cd0c0 0x559bd15d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.892+0000 7fa217dd2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bd15d5ca0 0x559bd15d31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.892+0000 7fa217dd2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559bd15d89d0 con 0x559bd15cd0c0 2024-04-02T13:59:02.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.892+0000 7fa217dd2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559bd14ebe80 con 0x559bd15cc7a0 2024-04-02T13:59:02.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.892+0000 7fa217dd2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559bd15d8b50 con 0x559bd15d5ca0 2024-04-02T13:59:02.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.892+0000 7fa217dd2740 10 client.0 _ll_register_callbacks cb 0x559bd1527aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:02.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.892+0000 7fa20659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15cd0c0 0x559bd15d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.892+0000 7fa20659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15cd0c0 0x559bd15d2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58676/0 (socket says 172.21.15.52:58676) 2024-04-02T13:59:02.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.892+0000 7fa20659c640 1 -- 172.21.15.52:0/1969058460 learned_addr learned my addr 172.21.15.52:0/1969058460 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:02.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.892+0000 7fa20f59e640 1 --2- 172.21.15.52:0/1969058460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bd15d5ca0 0x559bd15d31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.892+0000 7fa20ed9d640 1 --2- 172.21.15.52:0/1969058460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559bd15cc7a0 0x559bd15d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.892+0000 7fa20659c640 1 -- 172.21.15.52:0/1969058460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bd15d5ca0 msgr2=0x559bd15d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:02.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.892+0000 7fa20659c640 1 --2- 172.21.15.52:0/1969058460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bd15d5ca0 0x559bd15d31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.892+0000 7fa20659c640 1 -- 172.21.15.52:0/1969058460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559bd15cc7a0 msgr2=0x559bd15d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:02.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.892+0000 7fa20659c640 1 --2- 172.21.15.52:0/1969058460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559bd15cc7a0 0x559bd15d56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.892+0000 7fa20659c640 1 -- 172.21.15.52:0/1969058460 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bd130e7d0 con 0x559bd15cd0c0 2024-04-02T13:59:02.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.892+0000 7fa20659c640 1 --2- 172.21.15.52:0/1969058460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15cd0c0 0x559bd15d2a80 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fa1f809c180 tx=0x7fa1f8010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41ec87bb180e254 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:02.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.892+0000 7fa1f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:02.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.892+0000 7fa1f67fc640 1 -- 172.21.15.52:0/1969058460 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f809bde0 con 0x559bd15cd0c0 2024-04-02T13:59:02.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.892+0000 7fa1f67fc640 1 -- 172.21.15.52:0/1969058460 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fa1f809a780 con 0x559bd15cd0c0 2024-04-02T13:59:02.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.892+0000 7fa217dd2740 1 -- 172.21.15.52:0/1969058460 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1f8009de0 con 0x559bd15cd0c0 2024-04-02T13:59:02.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.893+0000 7fa1f67fc640 1 -- 172.21.15.52:0/1969058460 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa1f8020030 con 0x559bd15cd0c0 2024-04-02T13:59:02.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.893+0000 7fa217dd2740 10 client.17196 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:02.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.893+0000 7fa217dd2740 1 -- 172.21.15.52:0/1969058460 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559bd15d89d0 con 0x559bd15cd0c0 2024-04-02T13:59:02.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.893+0000 7fa1f67fc640 1 -- 172.21.15.52:0/1969058460 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f8009de0 con 0x559bd15cd0c0 2024-04-02T13:59:02.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.893+0000 7fa1f67fc640 1 -- 172.21.15.52:0/1969058460 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa1f8098af0 con 0x559bd15cd0c0 2024-04-02T13:59:02.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.893+0000 7fa217dd2740 10 client.17196 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:02.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.893+0000 7fa217dd2740 10 client.17196 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T13:59:02.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.893+0000 7fa217dd2740 20 client.17196 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:02.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.893+0000 7fa217dd2740 1 --2- 172.21.15.52:0/1969058460 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x559bd15e5c00 0x559bd15e7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.893+0000 7fa217dd2740 4 client.17196 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:59:02.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.893+0000 7fa217dd2740 1 -- 172.21.15.52:0/1969058460 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bd153dcf0 con 0x559bd15e5c00 2024-04-02T13:59:02.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.893+0000 7fa20ed9d640 1 --2- 172.21.15.52:0/1969058460 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x559bd15e5c00 0x559bd15e7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.894+0000 7fa20ed9d640 1 --2- 172.21.15.52:0/1969058460 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x559bd15e5c00 0x559bd15e7ff0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9affe9bf4298fa64 server_cookie=5ba9ff637cd521e2 in_seq=0 out_seq=0 2024-04-02T13:59:02.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.894+0000 7fa1f67fc640 10 client.17196 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:59:02.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.895+0000 7fa1f67fc640 1 -- 172.21.15.52:0/1969058460 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559bd153dcf0 con 0x559bd15e5c00 2024-04-02T13:59:02.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.895+0000 7fa1f67fc640 10 client.17196 handle_command_reply: tid=0 2024-04-02T13:59:02.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.895+0000 7fa217dd2740 1 client.17196 shutdown 2024-04-02T13:59:02.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.895+0000 7fa217dd2740 1 -- 172.21.15.52:0/1969058460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15cd0c0 msgr2=0x559bd15d2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.895+0000 7fa217dd2740 1 --2- 172.21.15.52:0/1969058460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15cd0c0 0x559bd15d2a80 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fa1f809c180 tx=0x7fa1f8010d70 comp rx=0 tx=0).stop 2024-04-02T13:59:02.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.895+0000 7fa217dd2740 1 -- 172.21.15.52:0/1969058460 shutdown_connections 2024-04-02T13:59:02.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.895+0000 7fa217dd2740 1 --2- 172.21.15.52:0/1969058460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bd15d5ca0 0x559bd15d31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.895+0000 7fa217dd2740 1 --2- 172.21.15.52:0/1969058460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15cd0c0 0x559bd15d2a80 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.895+0000 7fa217dd2740 1 --2- 172.21.15.52:0/1969058460 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x559bd15e5c00 0x559bd15e7ff0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.895+0000 7fa217dd2740 1 --2- 172.21.15.52:0/1969058460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559bd15cc7a0 0x559bd15d56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.895+0000 7fa217dd2740 1 -- 172.21.15.52:0/1969058460 >> 172.21.15.52:0/1969058460 conn(0x559bd15c6ed0 msgr2=0x559bd15c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:02.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.895+0000 7fa1f67fc640 0 client.17196 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:59:02.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.895+0000 7fa217dd2740 1 -- 172.21.15.52:0/1969058460 shutdown_connections 2024-04-02T13:59:02.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.895+0000 7fa217dd2740 1 -- 172.21.15.52:0/1969058460 wait complete. 2024-04-02T13:59:02.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.895+0000 7fa217dd2740 20 client.17196 trim_cache size 0 max 16384 2024-04-02T13:59:02.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.908+0000 7fa217dd2740 1 Processor -- start 2024-04-02T13:59:02.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.908+0000 7fa217dd2740 1 -- start start 2024-04-02T13:59:02.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.908+0000 7fa217dd2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15cc7a0 0x559bd15e18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.908+0000 7fa217dd2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559bd15cd0c0 0x559bd15de1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.908+0000 7fa217dd2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bd15d5ca0 0x559bd15c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.908+0000 7fa217dd2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa2101aedd0 con 0x559bd15cc7a0 2024-04-02T13:59:02.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.908+0000 7fa217dd2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa2101b0910 con 0x559bd15cd0c0 2024-04-02T13:59:02.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.908+0000 7fa217dd2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa2101af4a0 con 0x559bd15d5ca0 2024-04-02T13:59:02.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.909+0000 7fa20ed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15cc7a0 0x559bd15e18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.909+0000 7fa20ed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15cc7a0 0x559bd15e18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58686/0 (socket says 172.21.15.52:58686) 2024-04-02T13:59:02.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.909+0000 7fa20ed9d640 1 -- 172.21.15.52:0/3913445308 learned_addr learned my addr 172.21.15.52:0/3913445308 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:02.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.909+0000 7fa20659c640 1 --2- 172.21.15.52:0/3913445308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559bd15cd0c0 0x559bd15de1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.909+0000 7fa20f59e640 1 --2- 172.21.15.52:0/3913445308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bd15d5ca0 0x559bd15c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.909+0000 7fa20ed9d640 1 -- 172.21.15.52:0/3913445308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bd15d5ca0 msgr2=0x559bd15c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:02.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.909+0000 7fa20ed9d640 1 --2- 172.21.15.52:0/3913445308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bd15d5ca0 0x559bd15c8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.909+0000 7fa20ed9d640 1 -- 172.21.15.52:0/3913445308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559bd15cd0c0 msgr2=0x559bd15de1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:02.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.909+0000 7fa20ed9d640 1 --2- 172.21.15.52:0/3913445308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559bd15cd0c0 0x559bd15de1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.909+0000 7fa20ed9d640 1 -- 172.21.15.52:0/3913445308 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bd15d3530 con 0x559bd15cc7a0 2024-04-02T13:59:02.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.909+0000 7fa20f59e640 1 --2- 172.21.15.52:0/3913445308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bd15d5ca0 0x559bd15c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:59:02.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.909+0000 7fa20659c640 1 --2- 172.21.15.52:0/3913445308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559bd15cd0c0 0x559bd15de1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:59:02.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.909+0000 7fa20ed9d640 1 --2- 172.21.15.52:0/3913445308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15cc7a0 0x559bd15e18f0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fa20002cfd0 tx=0x7fa2000058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3580f666755e674f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:02.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.909+0000 7fa204d99640 1 -- 172.21.15.52:0/3913445308 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa200045dc0 con 0x559bd15cc7a0 2024-04-02T13:59:02.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.909+0000 7fa204d99640 1 -- 172.21.15.52:0/3913445308 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa200004040 con 0x559bd15cc7a0 2024-04-02T13:59:02.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.909+0000 7fa204d99640 1 -- 172.21.15.52:0/3913445308 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa200005040 con 0x559bd15cc7a0 2024-04-02T13:59:02.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.910+0000 7fa217dd2740 1 -- 172.21.15.52:0/3913445308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15cc7a0 msgr2=0x559bd15e18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.910+0000 7fa217dd2740 1 --2- 172.21.15.52:0/3913445308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15cc7a0 0x559bd15e18f0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fa20002cfd0 tx=0x7fa2000058b0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.910+0000 7fa217dd2740 1 -- 172.21.15.52:0/3913445308 shutdown_connections 2024-04-02T13:59:02.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.910+0000 7fa217dd2740 1 --2- 172.21.15.52:0/3913445308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bd15d5ca0 0x559bd15c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.910+0000 7fa217dd2740 1 --2- 172.21.15.52:0/3913445308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559bd15cd0c0 0x559bd15de1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.910+0000 7fa217dd2740 1 --2- 172.21.15.52:0/3913445308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15cc7a0 0x559bd15e18f0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.910+0000 7fa217dd2740 1 -- 172.21.15.52:0/3913445308 >> 172.21.15.52:0/3913445308 conn(0x559bd15c6ed0 msgr2=0x559bd15c8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:02.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.910+0000 7fa217dd2740 1 -- 172.21.15.52:0/3913445308 shutdown_connections 2024-04-02T13:59:02.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.910+0000 7fa217dd2740 1 -- 172.21.15.52:0/3913445308 wait complete. 2024-04-02T13:59:02.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.910+0000 7fa217dd2740 1 -- start start 2024-04-02T13:59:02.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.910+0000 7fa217dd2740 1 Processor -- start 2024-04-02T13:59:02.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.911+0000 7fa217dd2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15cc7a0 0x559bd15cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.911+0000 7fa217dd2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bd15cd0c0 0x559bd15d0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.911+0000 7fa217dd2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559bd15d5ca0 0x559bd15d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.911+0000 7fa217dd2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa2101b0240 con 0x559bd15cc7a0 2024-04-02T13:59:02.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.911+0000 7fa217dd2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa2101afb70 con 0x559bd15d5ca0 2024-04-02T13:59:02.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.911+0000 7fa217dd2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa2101acdb0 con 0x559bd15cd0c0 2024-04-02T13:59:02.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.911+0000 7fa217dd2740 10 client.0 _ll_register_callbacks cb 0x7fa208009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:02.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.911+0000 7fa20ed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15cc7a0 0x559bd15cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.911+0000 7fa20ed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15cc7a0 0x559bd15cfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58688/0 (socket says 172.21.15.52:58688) 2024-04-02T13:59:02.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.911+0000 7fa20ed9d640 1 -- 172.21.15.52:0/1239452570 learned_addr learned my addr 172.21.15.52:0/1239452570 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:02.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.911+0000 7fa20659c640 1 --2- 172.21.15.52:0/1239452570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bd15cd0c0 0x559bd15d0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.911+0000 7fa20ed9d640 1 -- 172.21.15.52:0/1239452570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bd15cd0c0 msgr2=0x559bd15d0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:02.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.911+0000 7fa20ed9d640 1 --2- 172.21.15.52:0/1239452570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bd15cd0c0 0x559bd15d0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.911+0000 7fa20ed9d640 1 -- 172.21.15.52:0/1239452570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559bd15d5ca0 msgr2=0x559bd15d07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:02.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.911+0000 7fa20ed9d640 1 --2- 172.21.15.52:0/1239452570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559bd15d5ca0 0x559bd15d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.911+0000 7fa20ed9d640 1 -- 172.21.15.52:0/1239452570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bd15d62e0 con 0x559bd15cc7a0 2024-04-02T13:59:02.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.911+0000 7fa20ed9d640 1 --2- 172.21.15.52:0/1239452570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15cc7a0 0x559bd15cfd20 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa200005940 tx=0x7fa200045590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b8b73d4f2b8043 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:02.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.911+0000 7fa1f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:02.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.911+0000 7fa1f67fc640 1 -- 172.21.15.52:0/1239452570 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa200052070 con 0x559bd15cc7a0 2024-04-02T13:59:02.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.911+0000 7fa1f67fc640 1 -- 172.21.15.52:0/1239452570 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fa20002cc20 con 0x559bd15cc7a0 2024-04-02T13:59:02.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.911+0000 7fa217dd2740 1 -- 172.21.15.52:0/1239452570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa200004040 con 0x559bd15cc7a0 2024-04-02T13:59:02.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.911+0000 7fa1f67fc640 1 -- 172.21.15.52:0/1239452570 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa200045720 con 0x559bd15cc7a0 2024-04-02T13:59:02.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.911+0000 7fa217dd2740 10 client.17202 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:02.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.911+0000 7fa217dd2740 1 -- 172.21.15.52:0/1239452570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559bd15d7550 con 0x559bd15cc7a0 2024-04-02T13:59:02.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.912+0000 7fa1f67fc640 1 -- 172.21.15.52:0/1239452570 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa200004040 con 0x559bd15cc7a0 2024-04-02T13:59:02.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.912+0000 7fa1f67fc640 1 -- 172.21.15.52:0/1239452570 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa200003cb0 con 0x559bd15cc7a0 2024-04-02T13:59:02.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.912+0000 7fa217dd2740 10 client.17202 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:02.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.912+0000 7fa217dd2740 10 client.17202 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T13:59:02.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.912+0000 7fa217dd2740 20 client.17202 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:02.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.912+0000 7fa217dd2740 1 --2- 172.21.15.52:0/1239452570 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x559bd1651010 0x559bd1653400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:02.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.912+0000 7fa217dd2740 4 client.17202 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:59:02.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.912+0000 7fa217dd2740 1 -- 172.21.15.52:0/1239452570 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559bd1538040 con 0x559bd1651010 2024-04-02T13:59:02.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.912+0000 7fa20659c640 1 --2- 172.21.15.52:0/1239452570 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x559bd1651010 0x559bd1653400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:02.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.912+0000 7fa20659c640 1 --2- 172.21.15.52:0/1239452570 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x559bd1651010 0x559bd1653400 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2d640fc85300db51 server_cookie=d1833762b548d15e in_seq=0 out_seq=0 2024-04-02T13:59:02.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.912+0000 7fa1f67fc640 10 client.17202 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:59:02.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.912+0000 7fa1f67fc640 1 -- 172.21.15.52:0/1239452570 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559bd1538040 con 0x559bd1651010 2024-04-02T13:59:02.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.912+0000 7fa1f67fc640 10 client.17202 handle_command_reply: tid=0 2024-04-02T13:59:02.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.913+0000 7fa217dd2740 1 client.17202 shutdown 2024-04-02T13:59:02.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.913+0000 7fa217dd2740 1 -- 172.21.15.52:0/1239452570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15cc7a0 msgr2=0x559bd15cfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.913+0000 7fa217dd2740 1 --2- 172.21.15.52:0/1239452570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15cc7a0 0x559bd15cfd20 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa200005940 tx=0x7fa200045590 comp rx=0 tx=0).stop 2024-04-02T13:59:02.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.913+0000 7fa217dd2740 1 -- 172.21.15.52:0/1239452570 shutdown_connections 2024-04-02T13:59:02.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.913+0000 7fa217dd2740 1 --2- 172.21.15.52:0/1239452570 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x559bd1651010 0x559bd1653400 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.913+0000 7fa217dd2740 1 --2- 172.21.15.52:0/1239452570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559bd15d5ca0 0x559bd15d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.913+0000 7fa217dd2740 1 --2- 172.21.15.52:0/1239452570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559bd15cd0c0 0x559bd15d0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.913+0000 7fa1f67fc640 0 client.17202 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:59:02.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.913+0000 7fa217dd2740 1 --2- 172.21.15.52:0/1239452570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559bd15cc7a0 0x559bd15cfd20 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.913+0000 7fa217dd2740 1 -- 172.21.15.52:0/1239452570 >> 172.21.15.52:0/1239452570 conn(0x559bd15c6ed0 msgr2=0x559bd15c8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:02.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.913+0000 7fa217dd2740 1 -- 172.21.15.52:0/1239452570 shutdown_connections 2024-04-02T13:59:02.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.913+0000 7fa217dd2740 1 -- 172.21.15.52:0/1239452570 wait complete. 2024-04-02T13:59:02.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.913+0000 7fa217dd2740 20 client.17202 trim_cache size 0 max 16384 2024-04-02T13:59:02.928 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:59:02.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.913+0000 7fa21510e640 1 -- 172.21.15.52:0/224939872 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa1fc0a3d20 msgr2=0x7fa1fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.913+0000 7fa21510e640 1 --2- 172.21.15.52:0/224939872 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa1fc0a3d20 0x7fa1fc0a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa208005fd0 tx=0x7fa208004a40 comp rx=0 tx=0).stop 2024-04-02T13:59:02.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.913+0000 7fa21510e640 1 -- 172.21.15.52:0/224939872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa210101370 msgr2=0x7fa210198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:02.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.913+0000 7fa21510e640 1 --2- 172.21.15.52:0/224939872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa210101370 0x7fa210198860 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800d950 tx=0x7fa1f800de10 comp rx=0 tx=0).stop 2024-04-02T13:59:02.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.913+0000 7fa21510e640 1 -- 172.21.15.52:0/224939872 shutdown_connections 2024-04-02T13:59:02.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.913+0000 7fa21510e640 1 --2- 172.21.15.52:0/224939872 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa1fc0a3d20 0x7fa1fc0a61e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.913+0000 7fa21510e640 1 --2- 172.21.15.52:0/224939872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2101069c0 0x7fa21019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.913+0000 7fa21510e640 1 --2- 172.21.15.52:0/224939872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa210101370 0x7fa210198860 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.913+0000 7fa21510e640 1 --2- 172.21.15.52:0/224939872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2101009b0 0x7fa210198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:02.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.913+0000 7fa21510e640 1 -- 172.21.15.52:0/224939872 >> 172.21.15.52:0/224939872 conn(0x7fa210076d80 msgr2=0x7fa21006a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:02.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.913+0000 7fa21510e640 1 -- 172.21.15.52:0/224939872 shutdown_connections 2024-04-02T13:59:02.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:02.913+0000 7fa21510e640 1 -- 172.21.15.52:0/224939872 wait complete. 2024-04-02T13:59:02.932 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:59:03.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.033+0000 7fa9f739d640 1 -- 172.21.15.52:0/3821552684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9f0101a90 msgr2=0x7fa9f0101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.033+0000 7fa9f739d640 1 --2- 172.21.15.52:0/3821552684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9f0101a90 0x7fa9f0101f10 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009930 tx=0x7fa9e002bc30 comp rx=0 tx=0).stop 2024-04-02T13:59:03.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.033+0000 7fa9f739d640 1 -- 172.21.15.52:0/3821552684 shutdown_connections 2024-04-02T13:59:03.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.033+0000 7fa9f739d640 1 --2- 172.21.15.52:0/3821552684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9f0102450 0x7fa9f010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.033+0000 7fa9f739d640 1 --2- 172.21.15.52:0/3821552684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9f0101a90 0x7fa9f0101f10 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.033+0000 7fa9f739d640 1 --2- 172.21.15.52:0/3821552684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9f0107aa0 0x7fa9f0107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.033+0000 7fa9f739d640 1 -- 172.21.15.52:0/3821552684 >> 172.21.15.52:0/3821552684 conn(0x7fa9f00fd5a0 msgr2=0x7fa9f00ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:03.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.033+0000 7fa9f739d640 1 -- 172.21.15.52:0/3821552684 shutdown_connections 2024-04-02T13:59:03.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.033+0000 7fa9f739d640 1 -- 172.21.15.52:0/3821552684 wait complete. 2024-04-02T13:59:03.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.034+0000 7fa9f739d640 1 Processor -- start 2024-04-02T13:59:03.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.034+0000 7fa9f739d640 1 -- start start 2024-04-02T13:59:03.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.034+0000 7fa9f739d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9f0101a90 0x7fa9f0198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.034+0000 7fa9f739d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9f0102450 0x7fa9f01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.034+0000 7fa9f739d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9f0107aa0 0x7fa9f019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.034+0000 7fa9f739d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f0111eb0 con 0x7fa9f0102450 2024-04-02T13:59:03.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.034+0000 7fa9f739d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f0111d30 con 0x7fa9f0107aa0 2024-04-02T13:59:03.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.034+0000 7fa9f739d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f0112030 con 0x7fa9f0101a90 2024-04-02T13:59:03.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.034+0000 7fa9f4938640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9f0102450 0x7fa9f01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.034+0000 7fa9f4938640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9f0102450 0x7fa9f01989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58694/0 (socket says 172.21.15.52:58694) 2024-04-02T13:59:03.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.034+0000 7fa9f4938640 1 -- 172.21.15.52:0/2899897146 learned_addr learned my addr 172.21.15.52:0/2899897146 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:03.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.034+0000 7fa9f593a640 1 --2- 172.21.15.52:0/2899897146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9f0107aa0 0x7fa9f019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.034+0000 7fa9f5139640 1 --2- 172.21.15.52:0/2899897146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9f0101a90 0x7fa9f0198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.034+0000 7fa9f4938640 1 -- 172.21.15.52:0/2899897146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9f0101a90 msgr2=0x7fa9f0198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.034+0000 7fa9f4938640 1 --2- 172.21.15.52:0/2899897146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9f0101a90 0x7fa9f0198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.034+0000 7fa9f4938640 1 -- 172.21.15.52:0/2899897146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9f0107aa0 msgr2=0x7fa9f019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.034+0000 7fa9f4938640 1 --2- 172.21.15.52:0/2899897146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9f0107aa0 0x7fa9f019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.034+0000 7fa9f4938640 1 -- 172.21.15.52:0/2899897146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f019d460 con 0x7fa9f0102450 2024-04-02T13:59:03.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.034+0000 7fa9f4938640 1 --2- 172.21.15.52:0/2899897146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9f0102450 0x7fa9f01989b0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0004f50 tx=0x7fa9e002ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:03.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.035+0000 7fa9e67fc640 1 -- 172.21.15.52:0/2899897146 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0041070 con 0x7fa9f0102450 2024-04-02T13:59:03.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.035+0000 7fa9e67fc640 1 -- 172.21.15.52:0/2899897146 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e00044d0 con 0x7fa9f0102450 2024-04-02T13:59:03.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.035+0000 7fa9f739d640 1 -- 172.21.15.52:0/2899897146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f019d6f0 con 0x7fa9f0102450 2024-04-02T13:59:03.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.035+0000 7fa9e67fc640 1 -- 172.21.15.52:0/2899897146 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0034440 con 0x7fa9f0102450 2024-04-02T13:59:03.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.035+0000 7fa9f739d640 1 -- 172.21.15.52:0/2899897146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f01a4fe0 con 0x7fa9f0102450 2024-04-02T13:59:03.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.037+0000 7fa9e67fc640 1 -- 172.21.15.52:0/2899897146 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa9e0044070 con 0x7fa9f0102450 2024-04-02T13:59:03.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.037+0000 7fa9f9f86740 1 Processor -- start 2024-04-02T13:59:03.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.037+0000 7fa9f9f86740 1 -- start start 2024-04-02T13:59:03.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.037+0000 7fa9f9f86740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559edbc3d7a0 0x559edbc3db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.037+0000 7fa9e67fc640 1 --2- 172.21.15.52:0/2899897146 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa9dc0a3d20 0x7fa9dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.037+0000 7fa9f9f86740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc3e0c0 0x559edbc46760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.037+0000 7fa9f9f86740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559edbc46ca0 0x559edbc49090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.037+0000 7fa9f9f86740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559edbba17b0 con 0x559edbc3e0c0 2024-04-02T13:59:03.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.037+0000 7fa9f9f86740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559edbb40570 con 0x559edbc46ca0 2024-04-02T13:59:03.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.037+0000 7fa9f9f86740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559edbb9d760 con 0x559edbc3d7a0 2024-04-02T13:59:03.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.038+0000 7fa9f4938640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc3e0c0 0x559edbc46760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.038+0000 7fa9f5139640 1 --2- 172.21.15.52:0/2899897146 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa9dc0a3d20 0x7fa9dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.038+0000 7fa9f4938640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc3e0c0 0x559edbc46760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58706/0 (socket says 172.21.15.52:58706) 2024-04-02T13:59:03.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.038+0000 7fa9f4938640 1 -- 172.21.15.52:0/2298529200 learned_addr learned my addr 172.21.15.52:0/2298529200 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:03.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.038+0000 7fa9f593a640 1 --2- 172.21.15.52:0/2298529200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559edbc3d7a0 0x559edbc3db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.038+0000 7fa9e67fc640 1 -- 172.21.15.52:0/2899897146 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fa9e00f2660 con 0x7fa9f0102450 2024-04-02T13:59:03.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.038+0000 7fa9f593a640 1 --2- 172.21.15.52:0/2298529200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559edbc46ca0 0x559edbc49090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.038+0000 7fa9f4938640 1 -- 172.21.15.52:0/2298529200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559edbc3d7a0 msgr2=0x559edbc3db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:03.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.038+0000 7fa9f4938640 1 --2- 172.21.15.52:0/2298529200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559edbc3d7a0 0x559edbc3db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.038+0000 7fa9f4938640 1 -- 172.21.15.52:0/2298529200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559edbc46ca0 msgr2=0x559edbc49090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:03.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.038+0000 7fa9f4938640 1 --2- 172.21.15.52:0/2298529200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559edbc46ca0 0x559edbc49090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.038+0000 7fa9f4938640 1 -- 172.21.15.52:0/2298529200 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559edb8b53f0 con 0x559edbc3e0c0 2024-04-02T13:59:03.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.038+0000 7fa9f593a640 1 --2- 172.21.15.52:0/2298529200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559edbc3d7a0 0x559edbc3db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:59:03.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.038+0000 7fa9f5139640 1 --2- 172.21.15.52:0/2899897146 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa9dc0a3d20 0x7fa9dc0a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec0058c0 tx=0x7fa9ec00a4c0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:03.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.038+0000 7fa9f4938640 1 --2- 172.21.15.52:0/2298529200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc3e0c0 0x559edbc46760 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00bada0 tx=0x7fa9e0003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed2b616dc1a2ec2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:03.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.038+0000 7fa9db7fe640 1 -- 172.21.15.52:0/2298529200 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e004b070 con 0x559edbc3e0c0 2024-04-02T13:59:03.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.038+0000 7fa9db7fe640 1 -- 172.21.15.52:0/2298529200 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e00bead0 con 0x559edbc3e0c0 2024-04-02T13:59:03.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.038+0000 7fa9db7fe640 1 -- 172.21.15.52:0/2298529200 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e00c3c90 con 0x559edbc3e0c0 2024-04-02T13:59:03.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.039+0000 7fa9f9f86740 1 -- 172.21.15.52:0/2298529200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc3e0c0 msgr2=0x559edbc46760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.039+0000 7fa9f9f86740 1 --2- 172.21.15.52:0/2298529200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc3e0c0 0x559edbc46760 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00bada0 tx=0x7fa9e0003ab0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.039+0000 7fa9f9f86740 1 -- 172.21.15.52:0/2298529200 shutdown_connections 2024-04-02T13:59:03.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.039+0000 7fa9f9f86740 1 --2- 172.21.15.52:0/2298529200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559edbc46ca0 0x559edbc49090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.039+0000 7fa9f9f86740 1 --2- 172.21.15.52:0/2298529200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc3e0c0 0x559edbc46760 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.039+0000 7fa9f9f86740 1 --2- 172.21.15.52:0/2298529200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559edbc3d7a0 0x559edbc3db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.039+0000 7fa9f9f86740 1 -- 172.21.15.52:0/2298529200 >> 172.21.15.52:0/2298529200 conn(0x559edbc37ed0 msgr2=0x559edbc382b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:03.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.039+0000 7fa9f9f86740 1 -- 172.21.15.52:0/2298529200 shutdown_connections 2024-04-02T13:59:03.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.039+0000 7fa9f9f86740 1 -- 172.21.15.52:0/2298529200 wait complete. 2024-04-02T13:59:03.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.039+0000 7fa9f9f86740 1 -- start start 2024-04-02T13:59:03.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.039+0000 7fa9f9f86740 1 Processor -- start 2024-04-02T13:59:03.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.040+0000 7fa9f9f86740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559edbc3d7a0 0x559edbc43ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.040+0000 7fa9f9f86740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559edbc3e0c0 0x559edbc440e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.040+0000 7fa9f9f86740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc46ca0 0x559edbc44620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.040+0000 7fa9f9f86740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559edbc49850 con 0x559edbc46ca0 2024-04-02T13:59:03.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.040+0000 7fa9f9f86740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559edbb5ce80 con 0x559edbc3e0c0 2024-04-02T13:59:03.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.040+0000 7fa9f9f86740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559edbc499d0 con 0x559edbc3d7a0 2024-04-02T13:59:03.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.040+0000 7fa9f9f86740 10 client.0 _ll_register_callbacks cb 0x559edbb98aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:03.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.040+0000 7fa9f593a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc46ca0 0x559edbc44620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.040+0000 7fa9f593a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc46ca0 0x559edbc44620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58716/0 (socket says 172.21.15.52:58716) 2024-04-02T13:59:03.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.040+0000 7fa9f593a640 1 -- 172.21.15.52:0/1017640970 learned_addr learned my addr 172.21.15.52:0/1017640970 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:03.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.040+0000 7fa9f4938640 1 --2- 172.21.15.52:0/1017640970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559edbc3e0c0 0x559edbc440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.040+0000 7fa9f5139640 1 --2- 172.21.15.52:0/1017640970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559edbc3d7a0 0x559edbc43ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.040+0000 7fa9f593a640 1 -- 172.21.15.52:0/1017640970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559edbc3d7a0 msgr2=0x559edbc43ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:03.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.040+0000 7fa9f593a640 1 --2- 172.21.15.52:0/1017640970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559edbc3d7a0 0x559edbc43ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.040+0000 7fa9f593a640 1 -- 172.21.15.52:0/1017640970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559edbc3e0c0 msgr2=0x559edbc440e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:03.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.040+0000 7fa9f593a640 1 --2- 172.21.15.52:0/1017640970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559edbc3e0c0 0x559edbc440e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.040+0000 7fa9f593a640 1 -- 172.21.15.52:0/1017640970 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559edb97f7d0 con 0x559edbc46ca0 2024-04-02T13:59:03.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.041+0000 7fa9f593a640 1 --2- 172.21.15.52:0/1017640970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc46ca0 0x559edbc44620 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8008d30 tx=0x7fa9e8007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc985e5c7c8e8d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:03.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.041+0000 7fa9d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:03.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.041+0000 7fa9d8ff9640 1 -- 172.21.15.52:0/1017640970 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e801e060 con 0x559edbc46ca0 2024-04-02T13:59:03.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.041+0000 7fa9d8ff9640 1 -- 172.21.15.52:0/1017640970 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fa9e800b670 con 0x559edbc46ca0 2024-04-02T13:59:03.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.041+0000 7fa9f9f86740 1 -- 172.21.15.52:0/1017640970 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9e00bead0 con 0x559edbc46ca0 2024-04-02T13:59:03.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.041+0000 7fa9d8ff9640 1 -- 172.21.15.52:0/1017640970 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa9e8025050 con 0x559edbc46ca0 2024-04-02T13:59:03.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.041+0000 7fa9f9f86740 10 client.17214 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:03.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.041+0000 7fa9f9f86740 1 -- 172.21.15.52:0/1017640970 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559edbc55990 con 0x559edbc46ca0 2024-04-02T13:59:03.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.041+0000 7fa9d8ff9640 1 -- 172.21.15.52:0/1017640970 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e00bead0 con 0x559edbc46ca0 2024-04-02T13:59:03.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.041+0000 7fa9d8ff9640 1 -- 172.21.15.52:0/1017640970 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa9e80188c0 con 0x559edbc46ca0 2024-04-02T13:59:03.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.041+0000 7fa9f9f86740 10 client.17214 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:03.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.041+0000 7fa9f9f86740 10 client.17214 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T13:59:03.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.041+0000 7fa9f9f86740 20 client.17214 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:03.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.041+0000 7fa9f9f86740 1 --2- 172.21.15.52:0/1017640970 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x559edbc56e50 0x559edbc59240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.041+0000 7fa9f9f86740 4 client.17214 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:59:03.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.041+0000 7fa9f9f86740 1 -- 172.21.15.52:0/1017640970 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559edbbaecf0 con 0x559edbc56e50 2024-04-02T13:59:03.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.041+0000 7fa9f5139640 1 --2- 172.21.15.52:0/1017640970 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x559edbc56e50 0x559edbc59240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.042+0000 7fa9d8ff9640 10 client.17214 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:59:03.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.042+0000 7fa9f5139640 1 --2- 172.21.15.52:0/1017640970 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x559edbc56e50 0x559edbc59240 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=edbce54d61f4b6e6 server_cookie=4b568e1511bb24f2 in_seq=0 out_seq=0 2024-04-02T13:59:03.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.043+0000 7fa9d8ff9640 1 -- 172.21.15.52:0/1017640970 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559edbbaecf0 con 0x559edbc56e50 2024-04-02T13:59:03.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.043+0000 7fa9d8ff9640 10 client.17214 handle_command_reply: tid=0 2024-04-02T13:59:03.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.043+0000 7fa9f9f86740 1 client.17214 shutdown 2024-04-02T13:59:03.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.043+0000 7fa9f9f86740 1 -- 172.21.15.52:0/1017640970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc46ca0 msgr2=0x559edbc44620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.043+0000 7fa9f9f86740 1 --2- 172.21.15.52:0/1017640970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc46ca0 0x559edbc44620 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8008d30 tx=0x7fa9e8007680 comp rx=0 tx=0).stop 2024-04-02T13:59:03.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.043+0000 7fa9f9f86740 1 -- 172.21.15.52:0/1017640970 shutdown_connections 2024-04-02T13:59:03.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.043+0000 7fa9f9f86740 1 --2- 172.21.15.52:0/1017640970 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x559edbc56e50 0x559edbc59240 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.043+0000 7fa9f9f86740 1 --2- 172.21.15.52:0/1017640970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc46ca0 0x559edbc44620 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.043+0000 7fa9d8ff9640 0 client.17214 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:59:03.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.043+0000 7fa9f9f86740 1 --2- 172.21.15.52:0/1017640970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559edbc3e0c0 0x559edbc440e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.043+0000 7fa9f9f86740 1 --2- 172.21.15.52:0/1017640970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559edbc3d7a0 0x559edbc43ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.043+0000 7fa9f9f86740 1 -- 172.21.15.52:0/1017640970 >> 172.21.15.52:0/1017640970 conn(0x559edbc37ed0 msgr2=0x559edbc39c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:03.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.043+0000 7fa9f9f86740 1 -- 172.21.15.52:0/1017640970 shutdown_connections 2024-04-02T13:59:03.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.043+0000 7fa9f9f86740 1 -- 172.21.15.52:0/1017640970 wait complete. 2024-04-02T13:59:03.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.043+0000 7fa9f9f86740 20 client.17214 trim_cache size 0 max 16384 2024-04-02T13:59:03.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.056+0000 7fa9f9f86740 1 Processor -- start 2024-04-02T13:59:03.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.056+0000 7fa9f9f86740 1 -- start start 2024-04-02T13:59:03.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.056+0000 7fa9f9f86740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559edbc3d7a0 0x559edbc52680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.056+0000 7fa9f9f86740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559edbc3e0c0 0x559edbc42d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.056+0000 7fa9f9f86740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc46ca0 0x559edbc44440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.056+0000 7fa9f9f86740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f01aee20 con 0x559edbc46ca0 2024-04-02T13:59:03.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.056+0000 7fa9f9f86740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f01b0960 con 0x559edbc3e0c0 2024-04-02T13:59:03.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.056+0000 7fa9f9f86740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f01af4f0 con 0x559edbc3d7a0 2024-04-02T13:59:03.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.057+0000 7fa9f593a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc46ca0 0x559edbc44440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.057+0000 7fa9f593a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc46ca0 0x559edbc44440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58720/0 (socket says 172.21.15.52:58720) 2024-04-02T13:59:03.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.057+0000 7fa9f593a640 1 -- 172.21.15.52:0/3707585185 learned_addr learned my addr 172.21.15.52:0/3707585185 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:03.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.057+0000 7fa9f4938640 1 --2- 172.21.15.52:0/3707585185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559edbc3e0c0 0x559edbc42d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.057+0000 7fa9f5139640 1 --2- 172.21.15.52:0/3707585185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559edbc3d7a0 0x559edbc52680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.057+0000 7fa9f593a640 1 -- 172.21.15.52:0/3707585185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559edbc3d7a0 msgr2=0x559edbc52680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:03.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.057+0000 7fa9f593a640 1 --2- 172.21.15.52:0/3707585185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559edbc3d7a0 0x559edbc52680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.057+0000 7fa9f593a640 1 -- 172.21.15.52:0/3707585185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559edbc3e0c0 msgr2=0x559edbc42d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:03.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.057+0000 7fa9f593a640 1 --2- 172.21.15.52:0/3707585185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559edbc3e0c0 0x559edbc42d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.057+0000 7fa9f593a640 1 -- 172.21.15.52:0/3707585185 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559edbc420f0 con 0x559edbc46ca0 2024-04-02T13:59:03.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.057+0000 7fa9f4938640 1 --2- 172.21.15.52:0/3707585185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559edbc3e0c0 0x559edbc42d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:59:03.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.057+0000 7fa9f593a640 1 --2- 172.21.15.52:0/3707585185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc46ca0 0x559edbc44440 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8000f80 tx=0x7fa9e8009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9d7356d78a41bf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:03.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.057+0000 7fa9daffd640 1 -- 172.21.15.52:0/3707585185 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e802e070 con 0x559edbc46ca0 2024-04-02T13:59:03.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.057+0000 7fa9daffd640 1 -- 172.21.15.52:0/3707585185 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e800f040 con 0x559edbc46ca0 2024-04-02T13:59:03.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.057+0000 7fa9daffd640 1 -- 172.21.15.52:0/3707585185 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e800b760 con 0x559edbc46ca0 2024-04-02T13:59:03.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.058+0000 7fa9f9f86740 1 -- 172.21.15.52:0/3707585185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc46ca0 msgr2=0x559edbc44440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.058+0000 7fa9f9f86740 1 --2- 172.21.15.52:0/3707585185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc46ca0 0x559edbc44440 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8000f80 tx=0x7fa9e8009850 comp rx=0 tx=0).stop 2024-04-02T13:59:03.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.058+0000 7fa9f9f86740 1 -- 172.21.15.52:0/3707585185 shutdown_connections 2024-04-02T13:59:03.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.058+0000 7fa9f9f86740 1 --2- 172.21.15.52:0/3707585185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc46ca0 0x559edbc44440 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.058+0000 7fa9f9f86740 1 --2- 172.21.15.52:0/3707585185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559edbc3e0c0 0x559edbc42d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.058+0000 7fa9f9f86740 1 --2- 172.21.15.52:0/3707585185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559edbc3d7a0 0x559edbc52680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.058+0000 7fa9f9f86740 1 -- 172.21.15.52:0/3707585185 >> 172.21.15.52:0/3707585185 conn(0x559edbc37ed0 msgr2=0x559edbc366a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:03.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.058+0000 7fa9f9f86740 1 -- 172.21.15.52:0/3707585185 shutdown_connections 2024-04-02T13:59:03.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.058+0000 7fa9f9f86740 1 -- 172.21.15.52:0/3707585185 wait complete. 2024-04-02T13:59:03.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.058+0000 7fa9f9f86740 1 -- start start 2024-04-02T13:59:03.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.058+0000 7fa9f9f86740 1 Processor -- start 2024-04-02T13:59:03.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.059+0000 7fa9f9f86740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559edbc3d7a0 0x559edbc4ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.059+0000 7fa9f9f86740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc3e0c0 0x559edbc4a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.059+0000 7fa9f9f86740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559edbc46ca0 0x559edbc4a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.059+0000 7fa9f9f86740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f01b0290 con 0x559edbc3e0c0 2024-04-02T13:59:03.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.059+0000 7fa9f9f86740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f01afbc0 con 0x559edbc46ca0 2024-04-02T13:59:03.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.059+0000 7fa9f9f86740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f01ace00 con 0x559edbc3d7a0 2024-04-02T13:59:03.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.059+0000 7fa9f9f86740 10 client.0 _ll_register_callbacks cb 0x7fa9e8009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:03.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.059+0000 7fa9f4938640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc3e0c0 0x559edbc4a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.059+0000 7fa9f4938640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc3e0c0 0x559edbc4a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58730/0 (socket says 172.21.15.52:58730) 2024-04-02T13:59:03.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.059+0000 7fa9f4938640 1 -- 172.21.15.52:0/2647019059 learned_addr learned my addr 172.21.15.52:0/2647019059 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:03.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.059+0000 7fa9f5139640 1 --2- 172.21.15.52:0/2647019059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559edbc3d7a0 0x559edbc4ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.059+0000 7fa9f4938640 1 -- 172.21.15.52:0/2647019059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559edbc3d7a0 msgr2=0x559edbc4ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:03.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.059+0000 7fa9f4938640 1 --2- 172.21.15.52:0/2647019059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559edbc3d7a0 0x559edbc4ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.059+0000 7fa9f4938640 1 -- 172.21.15.52:0/2647019059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559edbc46ca0 msgr2=0x559edbc4a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:03.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.059+0000 7fa9f4938640 1 --2- 172.21.15.52:0/2647019059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559edbc46ca0 0x559edbc4a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.059+0000 7fa9f4938640 1 -- 172.21.15.52:0/2647019059 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559edbc4abb0 con 0x559edbc3e0c0 2024-04-02T13:59:03.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.059+0000 7fa9f4938640 1 --2- 172.21.15.52:0/2647019059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc3e0c0 0x559edbc4a130 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00f0070 tx=0x7fa9e0009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d9033a7d6495b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:03.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.059+0000 7fa9d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:03.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.059+0000 7fa9d8ff9640 1 -- 172.21.15.52:0/2647019059 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e004b040 con 0x559edbc3e0c0 2024-04-02T13:59:03.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.059+0000 7fa9d8ff9640 1 -- 172.21.15.52:0/2647019059 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fa9e0003710 con 0x559edbc3e0c0 2024-04-02T13:59:03.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.059+0000 7fa9f9f86740 1 -- 172.21.15.52:0/2647019059 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9e800f040 con 0x559edbc3e0c0 2024-04-02T13:59:03.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.059+0000 7fa9d8ff9640 1 -- 172.21.15.52:0/2647019059 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa9e00ba670 con 0x559edbc3e0c0 2024-04-02T13:59:03.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.060+0000 7fa9f9f86740 10 client.17220 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:03.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.060+0000 7fa9f9f86740 1 -- 172.21.15.52:0/2647019059 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559edbcbd0b0 con 0x559edbc3e0c0 2024-04-02T13:59:03.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.060+0000 7fa9d8ff9640 1 -- 172.21.15.52:0/2647019059 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e800f040 con 0x559edbc3e0c0 2024-04-02T13:59:03.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.060+0000 7fa9d8ff9640 1 -- 172.21.15.52:0/2647019059 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa9e00f22c0 con 0x559edbc3e0c0 2024-04-02T13:59:03.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.060+0000 7fa9f9f86740 10 client.17220 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:03.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.060+0000 7fa9f9f86740 10 client.17220 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T13:59:03.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.060+0000 7fa9f9f86740 20 client.17220 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:03.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.060+0000 7fa9f9f86740 1 --2- 172.21.15.52:0/2647019059 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x559edbcbd660 0x559edbcbfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.060+0000 7fa9f9f86740 4 client.17220 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:59:03.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.060+0000 7fa9f9f86740 1 -- 172.21.15.52:0/2647019059 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559edbba9040 con 0x559edbcbd660 2024-04-02T13:59:03.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.060+0000 7fa9f5139640 1 --2- 172.21.15.52:0/2647019059 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x559edbcbd660 0x559edbcbfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.060+0000 7fa9d8ff9640 10 client.17220 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:59:03.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.060+0000 7fa9f5139640 1 --2- 172.21.15.52:0/2647019059 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x559edbcbd660 0x559edbcbfa80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b129f86e3044dcb6 server_cookie=2f33eb8ab68aa940 in_seq=0 out_seq=0 2024-04-02T13:59:03.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.061+0000 7fa9d8ff9640 1 -- 172.21.15.52:0/2647019059 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559edbba9040 con 0x559edbcbd660 2024-04-02T13:59:03.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.061+0000 7fa9d8ff9640 10 client.17220 handle_command_reply: tid=0 2024-04-02T13:59:03.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.061+0000 7fa9f9f86740 1 client.17220 shutdown 2024-04-02T13:59:03.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.061+0000 7fa9f9f86740 1 -- 172.21.15.52:0/2647019059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc3e0c0 msgr2=0x559edbc4a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.061+0000 7fa9f9f86740 1 --2- 172.21.15.52:0/2647019059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc3e0c0 0x559edbc4a130 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00f0070 tx=0x7fa9e0009300 comp rx=0 tx=0).stop 2024-04-02T13:59:03.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.061+0000 7fa9f9f86740 1 -- 172.21.15.52:0/2647019059 shutdown_connections 2024-04-02T13:59:03.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.061+0000 7fa9f9f86740 1 --2- 172.21.15.52:0/2647019059 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x559edbcbd660 0x559edbcbfa80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.061+0000 7fa9f9f86740 1 --2- 172.21.15.52:0/2647019059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559edbc46ca0 0x559edbc4a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.061+0000 7fa9f9f86740 1 --2- 172.21.15.52:0/2647019059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559edbc3e0c0 0x559edbc4a130 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.061+0000 7fa9d8ff9640 0 client.17220 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:59:03.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.061+0000 7fa9f9f86740 1 --2- 172.21.15.52:0/2647019059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559edbc3d7a0 0x559edbc4ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.061+0000 7fa9f9f86740 1 -- 172.21.15.52:0/2647019059 >> 172.21.15.52:0/2647019059 conn(0x559edbc37ed0 msgr2=0x559edbc36f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:03.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.061+0000 7fa9f9f86740 1 -- 172.21.15.52:0/2647019059 shutdown_connections 2024-04-02T13:59:03.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.061+0000 7fa9f9f86740 1 -- 172.21.15.52:0/2647019059 wait complete. 2024-04-02T13:59:03.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.061+0000 7fa9f9f86740 20 client.17220 trim_cache size 0 max 16384 2024-04-02T13:59:03.078 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:59:03.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.061+0000 7fa9f739d640 1 -- 172.21.15.52:0/2899897146 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa9dc0a3d20 msgr2=0x7fa9dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.062+0000 7fa9f739d640 1 --2- 172.21.15.52:0/2899897146 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa9dc0a3d20 0x7fa9dc0a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec0058c0 tx=0x7fa9ec00a4c0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.062+0000 7fa9f739d640 1 -- 172.21.15.52:0/2899897146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9f0102450 msgr2=0x7fa9f01989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.062+0000 7fa9f739d640 1 --2- 172.21.15.52:0/2899897146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9f0102450 0x7fa9f01989b0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0004f50 tx=0x7fa9e002ce20 comp rx=0 tx=0).stop 2024-04-02T13:59:03.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.062+0000 7fa9f739d640 1 -- 172.21.15.52:0/2899897146 shutdown_connections 2024-04-02T13:59:03.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.062+0000 7fa9f739d640 1 --2- 172.21.15.52:0/2899897146 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa9dc0a3d20 0x7fa9dc0a61e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.062+0000 7fa9f739d640 1 --2- 172.21.15.52:0/2899897146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9f0107aa0 0x7fa9f019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.062+0000 7fa9f739d640 1 --2- 172.21.15.52:0/2899897146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9f0102450 0x7fa9f01989b0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.062+0000 7fa9f739d640 1 --2- 172.21.15.52:0/2899897146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9f0101a90 0x7fa9f0198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.062+0000 7fa9f739d640 1 -- 172.21.15.52:0/2899897146 >> 172.21.15.52:0/2899897146 conn(0x7fa9f00fd5a0 msgr2=0x7fa9f00fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:03.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.062+0000 7fa9f739d640 1 -- 172.21.15.52:0/2899897146 shutdown_connections 2024-04-02T13:59:03.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.062+0000 7fa9f739d640 1 -- 172.21.15.52:0/2899897146 wait complete. 2024-04-02T13:59:03.081 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:59:03.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.182+0000 7fd739f60640 1 -- 172.21.15.52:0/2658863223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd734101330 msgr2=0x7fd73410d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.182+0000 7fd739f60640 1 --2- 172.21.15.52:0/2658863223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd734101330 0x7fd73410d840 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009cd0 tx=0x7fd72c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.183+0000 7fd739f60640 1 -- 172.21.15.52:0/2658863223 shutdown_connections 2024-04-02T13:59:03.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.183+0000 7fd739f60640 1 --2- 172.21.15.52:0/2658863223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd734101330 0x7fd73410d840 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.183+0000 7fd739f60640 1 --2- 172.21.15.52:0/2658863223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd734100970 0x7fd734100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.183+0000 7fd739f60640 1 --2- 172.21.15.52:0/2658863223 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd734106980 0x7fd734106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.183+0000 7fd739f60640 1 -- 172.21.15.52:0/2658863223 >> 172.21.15.52:0/2658863223 conn(0x7fd734077b00 msgr2=0x7fd734077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:03.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.183+0000 7fd739f60640 1 -- 172.21.15.52:0/2658863223 shutdown_connections 2024-04-02T13:59:03.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.183+0000 7fd739f60640 1 -- 172.21.15.52:0/2658863223 wait complete. 2024-04-02T13:59:03.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.183+0000 7fd739f60640 1 Processor -- start 2024-04-02T13:59:03.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.183+0000 7fd739f60640 1 -- start start 2024-04-02T13:59:03.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.184+0000 7fd739f60640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd734100970 0x7fd734199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.184+0000 7fd739f60640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd734101330 0x7fd734199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.184+0000 7fd739f60640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd734106980 0x7fd7341a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.184+0000 7fd739f60640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd734110d60 con 0x7fd734101330 2024-04-02T13:59:03.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.184+0000 7fd739f60640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd734110be0 con 0x7fd734106980 2024-04-02T13:59:03.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.184+0000 7fd739f60640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd734110ee0 con 0x7fd734100970 2024-04-02T13:59:03.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.184+0000 7fd732ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd734101330 0x7fd734199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.184+0000 7fd732ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd734101330 0x7fd734199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58742/0 (socket says 172.21.15.52:58742) 2024-04-02T13:59:03.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.184+0000 7fd732ffd640 1 -- 172.21.15.52:0/424674187 learned_addr learned my addr 172.21.15.52:0/424674187 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:03.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.184+0000 7fd733fff640 1 --2- 172.21.15.52:0/424674187 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd734106980 0x7fd7341a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.184+0000 7fd7337fe640 1 --2- 172.21.15.52:0/424674187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd734100970 0x7fd734199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.184+0000 7fd732ffd640 1 -- 172.21.15.52:0/424674187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd734100970 msgr2=0x7fd734199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.184+0000 7fd732ffd640 1 --2- 172.21.15.52:0/424674187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd734100970 0x7fd734199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.184+0000 7fd732ffd640 1 -- 172.21.15.52:0/424674187 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd734106980 msgr2=0x7fd7341a4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.184+0000 7fd732ffd640 1 --2- 172.21.15.52:0/424674187 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd734106980 0x7fd7341a4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.184+0000 7fd732ffd640 1 -- 172.21.15.52:0/424674187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7341a46c0 con 0x7fd734101330 2024-04-02T13:59:03.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.184+0000 7fd732ffd640 1 --2- 172.21.15.52:0/424674187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd734101330 0x7fd734199640 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fd72800c890 tx=0x7fd72800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:03.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.184+0000 7fd730ff9640 1 -- 172.21.15.52:0/424674187 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728004220 con 0x7fd734101330 2024-04-02T13:59:03.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.184+0000 7fd739f60640 1 -- 172.21.15.52:0/424674187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7341a4890 con 0x7fd734101330 2024-04-02T13:59:03.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.184+0000 7fd730ff9640 1 -- 172.21.15.52:0/424674187 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7280043c0 con 0x7fd734101330 2024-04-02T13:59:03.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.184+0000 7fd730ff9640 1 -- 172.21.15.52:0/424674187 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7280049e0 con 0x7fd734101330 2024-04-02T13:59:03.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.184+0000 7fd739f60640 1 -- 172.21.15.52:0/424674187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7341a4de0 con 0x7fd734101330 2024-04-02T13:59:03.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.187+0000 7fd730ff9640 1 -- 172.21.15.52:0/424674187 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd7280079b0 con 0x7fd734101330 2024-04-02T13:59:03.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.187+0000 7fd73cc29740 1 Processor -- start 2024-04-02T13:59:03.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.187+0000 7fd73cc29740 1 -- start start 2024-04-02T13:59:03.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.187+0000 7fd730ff9640 1 --2- 172.21.15.52:0/424674187 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd71c0a3d20 0x7fd71c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.187+0000 7fd73cc29740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb037a0 0x55f38fb03b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.187+0000 7fd73cc29740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f38fb040c0 0x55f38fb0c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.187+0000 7fd73cc29740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f38fb0cca0 0x55f38fb0f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.187+0000 7fd73cc29740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f38fa677b0 con 0x55f38fb037a0 2024-04-02T13:59:03.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.187+0000 7fd73cc29740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f38fa06570 con 0x55f38fb040c0 2024-04-02T13:59:03.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.187+0000 7fd73cc29740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f38fa63760 con 0x55f38fb0cca0 2024-04-02T13:59:03.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.187+0000 7fd7337fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb037a0 0x55f38fb03b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.187+0000 7fd733fff640 1 --2- 172.21.15.52:0/424674187 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd71c0a3d20 0x7fd71c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.187+0000 7fd7337fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb037a0 0x55f38fb03b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58752/0 (socket says 172.21.15.52:58752) 2024-04-02T13:59:03.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.187+0000 7fd7337fe640 1 -- 172.21.15.52:0/3855203895 learned_addr learned my addr 172.21.15.52:0/3855203895 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:03.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.187+0000 7fd730ff9640 1 -- 172.21.15.52:0/424674187 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd7280d0be0 con 0x7fd734101330 2024-04-02T13:59:03.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.188+0000 7fd732ffd640 1 --2- 172.21.15.52:0/3855203895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f38fb040c0 0x55f38fb0c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.188+0000 7fd7337fe640 1 -- 172.21.15.52:0/3855203895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f38fb0cca0 msgr2=0x55f38fb0f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:03.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.188+0000 7fd7337fe640 1 --2- 172.21.15.52:0/3855203895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f38fb0cca0 0x55f38fb0f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.188+0000 7fd7337fe640 1 -- 172.21.15.52:0/3855203895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f38fb040c0 msgr2=0x55f38fb0c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:03.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.188+0000 7fd7337fe640 1 --2- 172.21.15.52:0/3855203895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f38fb040c0 0x55f38fb0c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.188+0000 7fd7337fe640 1 -- 172.21.15.52:0/3855203895 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f38f77b3f0 con 0x55f38fb037a0 2024-04-02T13:59:03.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.188+0000 7fd732ffd640 1 --2- 172.21.15.52:0/3855203895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f38fb040c0 0x55f38fb0c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:59:03.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.188+0000 7fd7337fe640 1 --2- 172.21.15.52:0/3855203895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb037a0 0x55f38fb03b80 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd72400cbe0 tx=0x7fd72400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc0cdfeaf287a9d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:03.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.188+0000 7fd733fff640 1 --2- 172.21.15.52:0/424674187 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd71c0a3d20 0x7fd71c0a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd72c005cc0 tx=0x7fd72c0098d0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:03.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.188+0000 7fd719ffb640 1 -- 172.21.15.52:0/3855203895 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724005220 con 0x55f38fb037a0 2024-04-02T13:59:03.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.188+0000 7fd719ffb640 1 -- 172.21.15.52:0/3855203895 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd724004c80 con 0x55f38fb037a0 2024-04-02T13:59:03.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.188+0000 7fd719ffb640 1 -- 172.21.15.52:0/3855203895 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724010730 con 0x55f38fb037a0 2024-04-02T13:59:03.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.189+0000 7fd73cc29740 1 -- 172.21.15.52:0/3855203895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb037a0 msgr2=0x55f38fb03b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.189+0000 7fd73cc29740 1 --2- 172.21.15.52:0/3855203895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb037a0 0x55f38fb03b80 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd72400cbe0 tx=0x7fd72400e720 comp rx=0 tx=0).stop 2024-04-02T13:59:03.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.189+0000 7fd73cc29740 1 -- 172.21.15.52:0/3855203895 shutdown_connections 2024-04-02T13:59:03.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.189+0000 7fd73cc29740 1 --2- 172.21.15.52:0/3855203895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f38fb0cca0 0x55f38fb0f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.189+0000 7fd73cc29740 1 --2- 172.21.15.52:0/3855203895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f38fb040c0 0x55f38fb0c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.189+0000 7fd73cc29740 1 --2- 172.21.15.52:0/3855203895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb037a0 0x55f38fb03b80 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.189+0000 7fd73cc29740 1 -- 172.21.15.52:0/3855203895 >> 172.21.15.52:0/3855203895 conn(0x55f38fafded0 msgr2=0x55f38fafe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:03.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.189+0000 7fd73cc29740 1 -- 172.21.15.52:0/3855203895 shutdown_connections 2024-04-02T13:59:03.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.189+0000 7fd73cc29740 1 -- 172.21.15.52:0/3855203895 wait complete. 2024-04-02T13:59:03.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.189+0000 7fd73cc29740 1 -- start start 2024-04-02T13:59:03.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.189+0000 7fd73cc29740 1 Processor -- start 2024-04-02T13:59:03.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.189+0000 7fd73cc29740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb037a0 0x55f38fb0c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.189+0000 7fd73cc29740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f38fb040c0 0x55f38fb09a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.189+0000 7fd73cc29740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f38fb0cca0 0x55f38fb0a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.189+0000 7fd73cc29740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f38fb0fa90 con 0x55f38fb037a0 2024-04-02T13:59:03.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.189+0000 7fd73cc29740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f38fa22e80 con 0x55f38fb0cca0 2024-04-02T13:59:03.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.189+0000 7fd73cc29740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f38fb0fc10 con 0x55f38fb040c0 2024-04-02T13:59:03.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.190+0000 7fd73cc29740 10 client.0 _ll_register_callbacks cb 0x55f38fa5eaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:03.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.190+0000 7fd7337fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb037a0 0x55f38fb0c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.190+0000 7fd7337fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb037a0 0x55f38fb0c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58764/0 (socket says 172.21.15.52:58764) 2024-04-02T13:59:03.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.190+0000 7fd7337fe640 1 -- 172.21.15.52:0/1975222814 learned_addr learned my addr 172.21.15.52:0/1975222814 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:03.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.190+0000 7fd732ffd640 1 --2- 172.21.15.52:0/1975222814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f38fb040c0 0x55f38fb09a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.190+0000 7fd733fff640 1 --2- 172.21.15.52:0/1975222814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f38fb0cca0 0x55f38fb0a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.190+0000 7fd7337fe640 1 -- 172.21.15.52:0/1975222814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f38fb040c0 msgr2=0x55f38fb09a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:03.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.190+0000 7fd7337fe640 1 --2- 172.21.15.52:0/1975222814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f38fb040c0 0x55f38fb09a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.190+0000 7fd7337fe640 1 -- 172.21.15.52:0/1975222814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f38fb0cca0 msgr2=0x55f38fb0a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:03.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.190+0000 7fd7337fe640 1 --2- 172.21.15.52:0/1975222814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f38fb0cca0 0x55f38fb0a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.190+0000 7fd7337fe640 1 -- 172.21.15.52:0/1975222814 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f38f8457d0 con 0x55f38fb037a0 2024-04-02T13:59:03.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.190+0000 7fd733fff640 1 --2- 172.21.15.52:0/1975222814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f38fb0cca0 0x55f38fb0a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:59:03.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.190+0000 7fd7337fe640 1 --2- 172.21.15.52:0/1975222814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb037a0 0x55f38fb0c6b0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fd724010470 tx=0x7fd72400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92464cd1c1a40c4a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:03.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.190+0000 7fd712ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:03.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.190+0000 7fd712ffd640 1 -- 172.21.15.52:0/1975222814 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7240048d0 con 0x55f38fb037a0 2024-04-02T13:59:03.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.190+0000 7fd712ffd640 1 -- 172.21.15.52:0/1975222814 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd724004a70 con 0x55f38fb037a0 2024-04-02T13:59:03.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.190+0000 7fd73cc29740 1 -- 172.21.15.52:0/1975222814 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd724004c80 con 0x55f38fb037a0 2024-04-02T13:59:03.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.190+0000 7fd712ffd640 1 -- 172.21.15.52:0/1975222814 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd724004050 con 0x55f38fb037a0 2024-04-02T13:59:03.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.190+0000 7fd73cc29740 10 client.17232 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:03.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.190+0000 7fd73cc29740 1 -- 172.21.15.52:0/1975222814 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f38fb1ba30 con 0x55f38fb037a0 2024-04-02T13:59:03.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.191+0000 7fd712ffd640 1 -- 172.21.15.52:0/1975222814 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd724004c80 con 0x55f38fb037a0 2024-04-02T13:59:03.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.191+0000 7fd712ffd640 1 -- 172.21.15.52:0/1975222814 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd724010510 con 0x55f38fb037a0 2024-04-02T13:59:03.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.191+0000 7fd73cc29740 10 client.17232 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:03.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.191+0000 7fd73cc29740 10 client.17232 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:59:03.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.191+0000 7fd73cc29740 20 client.17232 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:03.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.191+0000 7fd73cc29740 1 --2- 172.21.15.52:0/1975222814 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55f38fb1ce30 0x55f38fb1f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.191+0000 7fd73cc29740 4 client.17232 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:59:03.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.191+0000 7fd73cc29740 1 -- 172.21.15.52:0/1975222814 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f38fa74cf0 con 0x55f38fb1ce30 2024-04-02T13:59:03.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.191+0000 7fd732ffd640 1 --2- 172.21.15.52:0/1975222814 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55f38fb1ce30 0x55f38fb1f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.192+0000 7fd712ffd640 10 client.17232 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:59:03.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.192+0000 7fd732ffd640 1 --2- 172.21.15.52:0/1975222814 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55f38fb1ce30 0x55f38fb1f220 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e809ce10174b80c6 server_cookie=9f24885db25d497 in_seq=0 out_seq=0 2024-04-02T13:59:03.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.192+0000 7fd712ffd640 1 -- 172.21.15.52:0/1975222814 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f38fa74cf0 con 0x55f38fb1ce30 2024-04-02T13:59:03.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.192+0000 7fd712ffd640 10 client.17232 handle_command_reply: tid=0 2024-04-02T13:59:03.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.192+0000 7fd73cc29740 1 client.17232 shutdown 2024-04-02T13:59:03.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.192+0000 7fd73cc29740 1 -- 172.21.15.52:0/1975222814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb037a0 msgr2=0x55f38fb0c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.192+0000 7fd73cc29740 1 --2- 172.21.15.52:0/1975222814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb037a0 0x55f38fb0c6b0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fd724010470 tx=0x7fd72400c920 comp rx=0 tx=0).stop 2024-04-02T13:59:03.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.192+0000 7fd73cc29740 1 -- 172.21.15.52:0/1975222814 shutdown_connections 2024-04-02T13:59:03.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.192+0000 7fd73cc29740 1 --2- 172.21.15.52:0/1975222814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f38fb0cca0 0x55f38fb0a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.192+0000 7fd73cc29740 1 --2- 172.21.15.52:0/1975222814 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55f38fb1ce30 0x55f38fb1f220 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.192+0000 7fd73cc29740 1 --2- 172.21.15.52:0/1975222814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f38fb040c0 0x55f38fb09a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.192+0000 7fd73cc29740 1 --2- 172.21.15.52:0/1975222814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb037a0 0x55f38fb0c6b0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.192+0000 7fd712ffd640 0 client.17232 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:59:03.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.192+0000 7fd73cc29740 1 -- 172.21.15.52:0/1975222814 >> 172.21.15.52:0/1975222814 conn(0x55f38fafded0 msgr2=0x55f38faffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:03.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.192+0000 7fd73cc29740 1 -- 172.21.15.52:0/1975222814 shutdown_connections 2024-04-02T13:59:03.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.193+0000 7fd73cc29740 1 -- 172.21.15.52:0/1975222814 wait complete. 2024-04-02T13:59:03.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.193+0000 7fd73cc29740 20 client.17232 trim_cache size 0 max 16384 2024-04-02T13:59:03.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.206+0000 7fd73cc29740 1 Processor -- start 2024-04-02T13:59:03.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.206+0000 7fd73cc29740 1 -- start start 2024-04-02T13:59:03.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.206+0000 7fd73cc29740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f38fb037a0 0x55f38fb18950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.206+0000 7fd73cc29740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb040c0 0x55f38fafcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.206+0000 7fd73cc29740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f38fb0cca0 0x55f38fb0c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.206+0000 7fd73cc29740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd7341aee70 con 0x55f38fb040c0 2024-04-02T13:59:03.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.206+0000 7fd73cc29740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd7341b09b0 con 0x55f38fb0cca0 2024-04-02T13:59:03.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.206+0000 7fd73cc29740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd7341af540 con 0x55f38fb037a0 2024-04-02T13:59:03.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.206+0000 7fd732ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb040c0 0x55f38fafcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.206+0000 7fd732ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb040c0 0x55f38fafcdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58772/0 (socket says 172.21.15.52:58772) 2024-04-02T13:59:03.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.206+0000 7fd732ffd640 1 -- 172.21.15.52:0/3103909204 learned_addr learned my addr 172.21.15.52:0/3103909204 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:03.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.206+0000 7fd733fff640 1 --2- 172.21.15.52:0/3103909204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f38fb0cca0 0x55f38fb0c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.206+0000 7fd732ffd640 1 -- 172.21.15.52:0/3103909204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f38fb037a0 msgr2=0x55f38fb18950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:03.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.206+0000 7fd7337fe640 1 --2- 172.21.15.52:0/3103909204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f38fb037a0 0x55f38fb18950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.206+0000 7fd732ffd640 1 --2- 172.21.15.52:0/3103909204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f38fb037a0 0x55f38fb18950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.206+0000 7fd732ffd640 1 -- 172.21.15.52:0/3103909204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f38fb0cca0 msgr2=0x55f38fb0c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:03.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.206+0000 7fd732ffd640 1 --2- 172.21.15.52:0/3103909204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f38fb0cca0 0x55f38fb0c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.206+0000 7fd732ffd640 1 -- 172.21.15.52:0/3103909204 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f38fb03eb0 con 0x55f38fb040c0 2024-04-02T13:59:03.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.206+0000 7fd733fff640 1 --2- 172.21.15.52:0/3103909204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f38fb0cca0 0x55f38fb0c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:59:03.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.206+0000 7fd7337fe640 1 --2- 172.21.15.52:0/3103909204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f38fb037a0 0x55f38fb18950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:59:03.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.207+0000 7fd732ffd640 1 --2- 172.21.15.52:0/3103909204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb040c0 0x55f38fafcdb0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fd7280ce050 tx=0x7fd7280054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba84f471f38df4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:03.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.207+0000 7fd7197fa640 1 -- 172.21.15.52:0/3103909204 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72801d040 con 0x55f38fb040c0 2024-04-02T13:59:03.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.207+0000 7fd7197fa640 1 -- 172.21.15.52:0/3103909204 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7280be040 con 0x55f38fb040c0 2024-04-02T13:59:03.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.207+0000 7fd7197fa640 1 -- 172.21.15.52:0/3103909204 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7280ce270 con 0x55f38fb040c0 2024-04-02T13:59:03.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.207+0000 7fd73cc29740 1 -- 172.21.15.52:0/3103909204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb040c0 msgr2=0x55f38fafcdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.207+0000 7fd73cc29740 1 --2- 172.21.15.52:0/3103909204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb040c0 0x55f38fafcdb0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fd7280ce050 tx=0x7fd7280054a0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.207+0000 7fd73cc29740 1 -- 172.21.15.52:0/3103909204 shutdown_connections 2024-04-02T13:59:03.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.207+0000 7fd73cc29740 1 --2- 172.21.15.52:0/3103909204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f38fb0cca0 0x55f38fb0c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.207+0000 7fd73cc29740 1 --2- 172.21.15.52:0/3103909204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb040c0 0x55f38fafcdb0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.207+0000 7fd73cc29740 1 --2- 172.21.15.52:0/3103909204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f38fb037a0 0x55f38fb18950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.207+0000 7fd73cc29740 1 -- 172.21.15.52:0/3103909204 >> 172.21.15.52:0/3103909204 conn(0x55f38fafded0 msgr2=0x55f38fafc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:03.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.207+0000 7fd73cc29740 1 -- 172.21.15.52:0/3103909204 shutdown_connections 2024-04-02T13:59:03.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.207+0000 7fd73cc29740 1 -- 172.21.15.52:0/3103909204 wait complete. 2024-04-02T13:59:03.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.208+0000 7fd73cc29740 1 -- start start 2024-04-02T13:59:03.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.208+0000 7fd73cc29740 1 Processor -- start 2024-04-02T13:59:03.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.208+0000 7fd73cc29740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb037a0 0x55f38fb15360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.208+0000 7fd73cc29740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f38fb040c0 0x55f38fb0d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.208+0000 7fd73cc29740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f38fb0cca0 0x55f38fb0df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.208+0000 7fd73cc29740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd7341b02e0 con 0x55f38fb037a0 2024-04-02T13:59:03.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.208+0000 7fd73cc29740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd7341afc10 con 0x55f38fb0cca0 2024-04-02T13:59:03.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.208+0000 7fd73cc29740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd7341ace50 con 0x55f38fb040c0 2024-04-02T13:59:03.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.208+0000 7fd73cc29740 10 client.0 _ll_register_callbacks cb 0x7fd724014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:03.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.208+0000 7fd7337fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb037a0 0x55f38fb15360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.208+0000 7fd7337fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb037a0 0x55f38fb15360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58786/0 (socket says 172.21.15.52:58786) 2024-04-02T13:59:03.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.208+0000 7fd7337fe640 1 -- 172.21.15.52:0/1902097465 learned_addr learned my addr 172.21.15.52:0/1902097465 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:03.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.208+0000 7fd732ffd640 1 --2- 172.21.15.52:0/1902097465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f38fb040c0 0x55f38fb0d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.208+0000 7fd733fff640 1 --2- 172.21.15.52:0/1902097465 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f38fb0cca0 0x55f38fb0df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.208+0000 7fd7337fe640 1 -- 172.21.15.52:0/1902097465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f38fb040c0 msgr2=0x55f38fb0d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:03.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.208+0000 7fd7337fe640 1 --2- 172.21.15.52:0/1902097465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f38fb040c0 0x55f38fb0d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.208+0000 7fd7337fe640 1 -- 172.21.15.52:0/1902097465 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f38fb0cca0 msgr2=0x55f38fb0df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:03.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.208+0000 7fd7337fe640 1 --2- 172.21.15.52:0/1902097465 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f38fb0cca0 0x55f38fb0df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.208+0000 7fd7337fe640 1 -- 172.21.15.52:0/1902097465 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f38fb12e50 con 0x55f38fb037a0 2024-04-02T13:59:03.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.209+0000 7fd7337fe640 1 --2- 172.21.15.52:0/1902097465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb037a0 0x55f38fb15360 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fd724005730 tx=0x7fd724005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b620be9b6709dcb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:03.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.209+0000 7fd712ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:03.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.209+0000 7fd712ffd640 1 -- 172.21.15.52:0/1902097465 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72400ca30 con 0x55f38fb037a0 2024-04-02T13:59:03.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.209+0000 7fd73cc29740 1 -- 172.21.15.52:0/1902097465 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7280be040 con 0x55f38fb037a0 2024-04-02T13:59:03.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.209+0000 7fd712ffd640 1 -- 172.21.15.52:0/1902097465 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd7240106b0 con 0x55f38fb037a0 2024-04-02T13:59:03.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.209+0000 7fd712ffd640 1 -- 172.21.15.52:0/1902097465 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd72400ec50 con 0x55f38fb037a0 2024-04-02T13:59:03.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.209+0000 7fd73cc29740 10 client.17238 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:03.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.209+0000 7fd73cc29740 1 -- 172.21.15.52:0/1902097465 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f38fb83260 con 0x55f38fb037a0 2024-04-02T13:59:03.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.209+0000 7fd712ffd640 1 -- 172.21.15.52:0/1902097465 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7280be040 con 0x55f38fb037a0 2024-04-02T13:59:03.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.209+0000 7fd712ffd640 1 -- 172.21.15.52:0/1902097465 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd724003960 con 0x55f38fb037a0 2024-04-02T13:59:03.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.209+0000 7fd73cc29740 10 client.17238 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:03.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.209+0000 7fd73cc29740 10 client.17238 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:59:03.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.209+0000 7fd73cc29740 20 client.17238 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:03.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.209+0000 7fd73cc29740 1 --2- 172.21.15.52:0/1902097465 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55f38fb837f0 0x55f38fb85be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.209+0000 7fd73cc29740 4 client.17238 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:59:03.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.209+0000 7fd73cc29740 1 -- 172.21.15.52:0/1902097465 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f38fa6f040 con 0x55f38fb837f0 2024-04-02T13:59:03.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.210+0000 7fd732ffd640 1 --2- 172.21.15.52:0/1902097465 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55f38fb837f0 0x55f38fb85be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.210+0000 7fd712ffd640 10 client.17238 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:59:03.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.210+0000 7fd732ffd640 1 --2- 172.21.15.52:0/1902097465 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55f38fb837f0 0x55f38fb85be0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8b0bf8c7d1c65a56 server_cookie=6c35438e88fcc11 in_seq=0 out_seq=0 2024-04-02T13:59:03.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.210+0000 7fd712ffd640 1 -- 172.21.15.52:0/1902097465 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f38fa6f040 con 0x55f38fb837f0 2024-04-02T13:59:03.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.210+0000 7fd712ffd640 10 client.17238 handle_command_reply: tid=0 2024-04-02T13:59:03.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.210+0000 7fd73cc29740 1 client.17238 shutdown 2024-04-02T13:59:03.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.210+0000 7fd73cc29740 1 -- 172.21.15.52:0/1902097465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb037a0 msgr2=0x55f38fb15360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.210+0000 7fd73cc29740 1 --2- 172.21.15.52:0/1902097465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb037a0 0x55f38fb15360 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fd724005730 tx=0x7fd724005850 comp rx=0 tx=0).stop 2024-04-02T13:59:03.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.210+0000 7fd73cc29740 1 -- 172.21.15.52:0/1902097465 shutdown_connections 2024-04-02T13:59:03.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.210+0000 7fd73cc29740 1 --2- 172.21.15.52:0/1902097465 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f38fb0cca0 0x55f38fb0df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.210+0000 7fd73cc29740 1 --2- 172.21.15.52:0/1902097465 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55f38fb837f0 0x55f38fb85be0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.210+0000 7fd73cc29740 1 --2- 172.21.15.52:0/1902097465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f38fb040c0 0x55f38fb0d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.210+0000 7fd73cc29740 1 --2- 172.21.15.52:0/1902097465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f38fb037a0 0x55f38fb15360 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.210+0000 7fd712ffd640 0 client.17238 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:59:03.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.210+0000 7fd73cc29740 1 -- 172.21.15.52:0/1902097465 >> 172.21.15.52:0/1902097465 conn(0x55f38fafded0 msgr2=0x55f38faffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:03.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.210+0000 7fd73cc29740 1 -- 172.21.15.52:0/1902097465 shutdown_connections 2024-04-02T13:59:03.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.211+0000 7fd73cc29740 1 -- 172.21.15.52:0/1902097465 wait complete. 2024-04-02T13:59:03.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.211+0000 7fd73cc29740 20 client.17238 trim_cache size 0 max 16384 2024-04-02T13:59:03.225 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:59:03.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.211+0000 7fd739f60640 1 -- 172.21.15.52:0/424674187 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd71c0a3d20 msgr2=0x7fd71c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.211+0000 7fd739f60640 1 --2- 172.21.15.52:0/424674187 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd71c0a3d20 0x7fd71c0a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd72c005cc0 tx=0x7fd72c0098d0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.211+0000 7fd739f60640 1 -- 172.21.15.52:0/424674187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd734101330 msgr2=0x7fd734199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.211+0000 7fd739f60640 1 --2- 172.21.15.52:0/424674187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd734101330 0x7fd734199640 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fd72800c890 tx=0x7fd72800cd50 comp rx=0 tx=0).stop 2024-04-02T13:59:03.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.211+0000 7fd739f60640 1 -- 172.21.15.52:0/424674187 shutdown_connections 2024-04-02T13:59:03.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.211+0000 7fd739f60640 1 --2- 172.21.15.52:0/424674187 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd71c0a3d20 0x7fd71c0a61e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.211+0000 7fd739f60640 1 --2- 172.21.15.52:0/424674187 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd734106980 0x7fd7341a4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.211+0000 7fd739f60640 1 --2- 172.21.15.52:0/424674187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd734101330 0x7fd734199640 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.211+0000 7fd739f60640 1 --2- 172.21.15.52:0/424674187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd734100970 0x7fd734199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.211+0000 7fd739f60640 1 -- 172.21.15.52:0/424674187 >> 172.21.15.52:0/424674187 conn(0x7fd734077b00 msgr2=0x7fd734102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:03.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.211+0000 7fd739f60640 1 -- 172.21.15.52:0/424674187 shutdown_connections 2024-04-02T13:59:03.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.211+0000 7fd739f60640 1 -- 172.21.15.52:0/424674187 wait complete. 2024-04-02T13:59:03.228 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:59:03.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.329+0000 7f053bfaf640 1 -- 172.21.15.52:0/2781823680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0534101aa0 msgr2=0x7f0534101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.329+0000 7f053bfaf640 1 --2- 172.21.15.52:0/2781823680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0534101aa0 0x7f0534101f20 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f0528009930 tx=0x7f052802bc30 comp rx=0 tx=0).stop 2024-04-02T13:59:03.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.329+0000 7f053bfaf640 1 -- 172.21.15.52:0/2781823680 shutdown_connections 2024-04-02T13:59:03.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.329+0000 7f053bfaf640 1 --2- 172.21.15.52:0/2781823680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0534102460 0x7f053410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.329+0000 7f053bfaf640 1 --2- 172.21.15.52:0/2781823680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0534101aa0 0x7f0534101f20 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.329+0000 7f053bfaf640 1 --2- 172.21.15.52:0/2781823680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0534107ab0 0x7f0534107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.329+0000 7f053bfaf640 1 -- 172.21.15.52:0/2781823680 >> 172.21.15.52:0/2781823680 conn(0x7f05340fd530 msgr2=0x7f05340ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:03.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.330+0000 7f053bfaf640 1 -- 172.21.15.52:0/2781823680 shutdown_connections 2024-04-02T13:59:03.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.330+0000 7f053bfaf640 1 -- 172.21.15.52:0/2781823680 wait complete. 2024-04-02T13:59:03.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.330+0000 7f053bfaf640 1 Processor -- start 2024-04-02T13:59:03.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.330+0000 7f053bfaf640 1 -- start start 2024-04-02T13:59:03.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.330+0000 7f053bfaf640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0534101aa0 0x7f05341983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.330+0000 7f053bfaf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0534102460 0x7f0534198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.330+0000 7f053bfaf640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0534107ab0 0x7f053419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.330+0000 7f053bfaf640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0534111ec0 con 0x7f0534102460 2024-04-02T13:59:03.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.330+0000 7f053bfaf640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0534111d40 con 0x7f0534101aa0 2024-04-02T13:59:03.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.330+0000 7f053bfaf640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0534112040 con 0x7f0534107ab0 2024-04-02T13:59:03.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.330+0000 7f053954a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0534102460 0x7f0534198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.330+0000 7f053954a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0534102460 0x7f0534198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58804/0 (socket says 172.21.15.52:58804) 2024-04-02T13:59:03.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.330+0000 7f053954a640 1 -- 172.21.15.52:0/3756715654 learned_addr learned my addr 172.21.15.52:0/3756715654 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:03.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.331+0000 7f0539d4b640 1 --2- 172.21.15.52:0/3756715654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0534101aa0 0x7f05341983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.331+0000 7f053a54c640 1 --2- 172.21.15.52:0/3756715654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0534107ab0 0x7f053419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.331+0000 7f053954a640 1 -- 172.21.15.52:0/3756715654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0534107ab0 msgr2=0x7f053419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.331+0000 7f053954a640 1 --2- 172.21.15.52:0/3756715654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0534107ab0 0x7f053419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.331+0000 7f053954a640 1 -- 172.21.15.52:0/3756715654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0534101aa0 msgr2=0x7f05341983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.331+0000 7f053954a640 1 --2- 172.21.15.52:0/3756715654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0534101aa0 0x7f05341983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.331+0000 7f053954a640 1 -- 172.21.15.52:0/3756715654 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053419d380 con 0x7f0534102460 2024-04-02T13:59:03.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.331+0000 7f053954a640 1 --2- 172.21.15.52:0/3756715654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0534102460 0x7f0534198920 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0528009230 tx=0x7f052802cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:03.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.331+0000 7f0532ffd640 1 -- 172.21.15.52:0/3756715654 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528041070 con 0x7f0534102460 2024-04-02T13:59:03.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.331+0000 7f053bfaf640 1 -- 172.21.15.52:0/3756715654 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f053419d610 con 0x7f0534102460 2024-04-02T13:59:03.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.331+0000 7f0532ffd640 1 -- 172.21.15.52:0/3756715654 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0528004450 con 0x7f0534102460 2024-04-02T13:59:03.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.331+0000 7f0532ffd640 1 -- 172.21.15.52:0/3756715654 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528034440 con 0x7f0534102460 2024-04-02T13:59:03.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.331+0000 7f053bfaf640 1 -- 172.21.15.52:0/3756715654 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05341a4f00 con 0x7f0534102460 2024-04-02T13:59:03.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.333+0000 7f0532ffd640 1 -- 172.21.15.52:0/3756715654 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0528044070 con 0x7f0534102460 2024-04-02T13:59:03.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.334+0000 7f053eb96740 1 Processor -- start 2024-04-02T13:59:03.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.334+0000 7f053eb96740 1 -- start start 2024-04-02T13:59:03.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.334+0000 7f053eb96740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c0c7a0 0x55b4d2c0cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.334+0000 7f0532ffd640 1 --2- 172.21.15.52:0/3756715654 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f051c0a3d20 0x7f051c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.334+0000 7f053eb96740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4d2c0d0c0 0x55b4d2c15760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.334+0000 7f053eb96740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4d2c15ca0 0x55b4d2c18090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.334+0000 7f053eb96740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b4d2b707b0 con 0x55b4d2c0c7a0 2024-04-02T13:59:03.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.334+0000 7f053eb96740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b4d2b0f570 con 0x55b4d2c0d0c0 2024-04-02T13:59:03.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.334+0000 7f053eb96740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b4d2b6c760 con 0x55b4d2c15ca0 2024-04-02T13:59:03.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.334+0000 7f0539d4b640 1 --2- 172.21.15.52:0/3756715654 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f051c0a3d20 0x7f051c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.334+0000 7f053a54c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c0c7a0 0x55b4d2c0cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.334+0000 7f053a54c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c0c7a0 0x55b4d2c0cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58812/0 (socket says 172.21.15.52:58812) 2024-04-02T13:59:03.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.334+0000 7f053a54c640 1 -- 172.21.15.52:0/79688555 learned_addr learned my addr 172.21.15.52:0/79688555 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:03.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.334+0000 7f0532ffd640 1 -- 172.21.15.52:0/3756715654 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f05280f24c0 con 0x7f0534102460 2024-04-02T13:59:03.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.334+0000 7f053954a640 1 --2- 172.21.15.52:0/79688555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4d2c0d0c0 0x55b4d2c15760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.334+0000 7f053a54c640 1 --2- 172.21.15.52:0/79688555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4d2c15ca0 0x55b4d2c18090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.334+0000 7f053a54c640 1 -- 172.21.15.52:0/79688555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4d2c15ca0 msgr2=0x55b4d2c18090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:03.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.334+0000 7f053a54c640 1 --2- 172.21.15.52:0/79688555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4d2c15ca0 0x55b4d2c18090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.334+0000 7f053a54c640 1 -- 172.21.15.52:0/79688555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4d2c0d0c0 msgr2=0x55b4d2c15760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:03.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.334+0000 7f053a54c640 1 --2- 172.21.15.52:0/79688555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4d2c0d0c0 0x55b4d2c15760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.335+0000 7f053a54c640 1 -- 172.21.15.52:0/79688555 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4d28843f0 con 0x55b4d2c0c7a0 2024-04-02T13:59:03.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.335+0000 7f0539d4b640 1 --2- 172.21.15.52:0/3756715654 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f051c0a3d20 0x7f051c0a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f052c005d00 tx=0x7f052c00b410 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:03.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.335+0000 7f053a54c640 1 --2- 172.21.15.52:0/79688555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c0c7a0 0x55b4d2c0cb80 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f05240128d0 tx=0x7f0524009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3795856395ef4e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:03.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.335+0000 7f051bfff640 1 -- 172.21.15.52:0/79688555 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0524003970 con 0x55b4d2c0c7a0 2024-04-02T13:59:03.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.335+0000 7f051bfff640 1 -- 172.21.15.52:0/79688555 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0524012e50 con 0x55b4d2c0c7a0 2024-04-02T13:59:03.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.335+0000 7f051bfff640 1 -- 172.21.15.52:0/79688555 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0524004290 con 0x55b4d2c0c7a0 2024-04-02T13:59:03.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.335+0000 7f053eb96740 1 -- 172.21.15.52:0/79688555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c0c7a0 msgr2=0x55b4d2c0cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.335+0000 7f053eb96740 1 --2- 172.21.15.52:0/79688555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c0c7a0 0x55b4d2c0cb80 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f05240128d0 tx=0x7f0524009f90 comp rx=0 tx=0).stop 2024-04-02T13:59:03.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.336+0000 7f053eb96740 1 -- 172.21.15.52:0/79688555 shutdown_connections 2024-04-02T13:59:03.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.336+0000 7f053eb96740 1 --2- 172.21.15.52:0/79688555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4d2c15ca0 0x55b4d2c18090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.336+0000 7f053eb96740 1 --2- 172.21.15.52:0/79688555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4d2c0d0c0 0x55b4d2c15760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.336+0000 7f053eb96740 1 --2- 172.21.15.52:0/79688555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c0c7a0 0x55b4d2c0cb80 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.336+0000 7f053eb96740 1 -- 172.21.15.52:0/79688555 >> 172.21.15.52:0/79688555 conn(0x55b4d2c06ed0 msgr2=0x55b4d2c072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:03.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.336+0000 7f053eb96740 1 -- 172.21.15.52:0/79688555 shutdown_connections 2024-04-02T13:59:03.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.336+0000 7f053eb96740 1 -- 172.21.15.52:0/79688555 wait complete. 2024-04-02T13:59:03.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.336+0000 7f053eb96740 1 -- start start 2024-04-02T13:59:03.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.336+0000 7f053eb96740 1 Processor -- start 2024-04-02T13:59:03.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.336+0000 7f053eb96740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c0c7a0 0x55b4d2c12b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.336+0000 7f053eb96740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4d2c0d0c0 0x55b4d2c13040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.336+0000 7f053eb96740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4d2c15ca0 0x55b4d2c13580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.336+0000 7f053eb96740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b4d2c18a90 con 0x55b4d2c0c7a0 2024-04-02T13:59:03.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.336+0000 7f053eb96740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b4d2b2be80 con 0x55b4d2c0d0c0 2024-04-02T13:59:03.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.336+0000 7f053eb96740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b4d2c18c10 con 0x55b4d2c15ca0 2024-04-02T13:59:03.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.336+0000 7f053eb96740 10 client.0 _ll_register_callbacks cb 0x55b4d2b67aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:03.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.337+0000 7f0539d4b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c0c7a0 0x55b4d2c12b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.337+0000 7f0539d4b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c0c7a0 0x55b4d2c12b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58818/0 (socket says 172.21.15.52:58818) 2024-04-02T13:59:03.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.337+0000 7f0539d4b640 1 -- 172.21.15.52:0/3120642582 learned_addr learned my addr 172.21.15.52:0/3120642582 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:03.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.337+0000 7f053954a640 1 --2- 172.21.15.52:0/3120642582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4d2c0d0c0 0x55b4d2c13040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.337+0000 7f053a54c640 1 --2- 172.21.15.52:0/3120642582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4d2c15ca0 0x55b4d2c13580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.337+0000 7f0539d4b640 1 -- 172.21.15.52:0/3120642582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4d2c15ca0 msgr2=0x55b4d2c13580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:03.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.337+0000 7f0539d4b640 1 --2- 172.21.15.52:0/3120642582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4d2c15ca0 0x55b4d2c13580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.337+0000 7f0539d4b640 1 -- 172.21.15.52:0/3120642582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4d2c0d0c0 msgr2=0x55b4d2c13040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:03.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.337+0000 7f0539d4b640 1 --2- 172.21.15.52:0/3120642582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4d2c0d0c0 0x55b4d2c13040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.337+0000 7f0539d4b640 1 -- 172.21.15.52:0/3120642582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4d294e7d0 con 0x55b4d2c0c7a0 2024-04-02T13:59:03.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.337+0000 7f053954a640 1 --2- 172.21.15.52:0/3120642582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4d2c0d0c0 0x55b4d2c13040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:59:03.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.337+0000 7f053a54c640 1 --2- 172.21.15.52:0/3120642582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4d2c15ca0 0x55b4d2c13580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:59:03.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.337+0000 7f0539d4b640 1 --2- 172.21.15.52:0/3120642582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c0c7a0 0x55b4d2c12b00 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f052c0116c0 tx=0x7f052c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1190339a5e174dd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:03.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.337+0000 7f05197fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:03.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.337+0000 7f05197fa640 1 -- 172.21.15.52:0/3120642582 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c01e8f0 con 0x55b4d2c0c7a0 2024-04-02T13:59:03.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.337+0000 7f053eb96740 1 -- 172.21.15.52:0/3120642582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0524012e50 con 0x55b4d2c0c7a0 2024-04-02T13:59:03.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.337+0000 7f05197fa640 1 -- 172.21.15.52:0/3120642582 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f052c01ea90 con 0x55b4d2c0c7a0 2024-04-02T13:59:03.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.337+0000 7f05197fa640 1 -- 172.21.15.52:0/3120642582 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f052c02f050 con 0x55b4d2c0c7a0 2024-04-02T13:59:03.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.337+0000 7f053eb96740 10 client.17250 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:03.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.337+0000 7f053eb96740 1 -- 172.21.15.52:0/3120642582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b4d2c24bc0 con 0x55b4d2c0c7a0 2024-04-02T13:59:03.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.338+0000 7f05197fa640 1 -- 172.21.15.52:0/3120642582 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0524012e50 con 0x55b4d2c0c7a0 2024-04-02T13:59:03.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.338+0000 7f05197fa640 1 -- 172.21.15.52:0/3120642582 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f052c01c3b0 con 0x55b4d2c0c7a0 2024-04-02T13:59:03.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.338+0000 7f053eb96740 10 client.17250 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:03.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.338+0000 7f053eb96740 10 client.17250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:59:03.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.338+0000 7f053eb96740 20 client.17250 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:03.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.338+0000 7f053eb96740 1 --2- 172.21.15.52:0/3120642582 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55b4d2c26080 0x55b4d2c284f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.338+0000 7f053eb96740 4 client.17250 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:59:03.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.338+0000 7f053eb96740 1 -- 172.21.15.52:0/3120642582 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4d2b7dcf0 con 0x55b4d2c26080 2024-04-02T13:59:03.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.338+0000 7f053954a640 1 --2- 172.21.15.52:0/3120642582 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55b4d2c26080 0x55b4d2c284f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.339+0000 7f05197fa640 10 client.17250 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:03.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.339+0000 7f053954a640 1 --2- 172.21.15.52:0/3120642582 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55b4d2c26080 0x55b4d2c284f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f7a73b7029d1df5 server_cookie=77df49b469446ee1 in_seq=0 out_seq=0 2024-04-02T13:59:03.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.339+0000 7f05197fa640 1 -- 172.21.15.52:0/3120642582 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b4d2b7dcf0 con 0x55b4d2c26080 2024-04-02T13:59:03.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.339+0000 7f05197fa640 10 client.17250 handle_command_reply: tid=0 2024-04-02T13:59:03.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.339+0000 7f053eb96740 1 client.17250 shutdown 2024-04-02T13:59:03.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.339+0000 7f053eb96740 1 -- 172.21.15.52:0/3120642582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c0c7a0 msgr2=0x55b4d2c12b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.339+0000 7f053eb96740 1 --2- 172.21.15.52:0/3120642582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c0c7a0 0x55b4d2c12b00 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f052c0116c0 tx=0x7f052c004730 comp rx=0 tx=0).stop 2024-04-02T13:59:03.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.340+0000 7f053eb96740 1 -- 172.21.15.52:0/3120642582 shutdown_connections 2024-04-02T13:59:03.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.340+0000 7f053eb96740 1 --2- 172.21.15.52:0/3120642582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4d2c15ca0 0x55b4d2c13580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.340+0000 7f053eb96740 1 --2- 172.21.15.52:0/3120642582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4d2c0d0c0 0x55b4d2c13040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.340+0000 7f053eb96740 1 --2- 172.21.15.52:0/3120642582 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55b4d2c26080 0x55b4d2c284f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.340+0000 7f053eb96740 1 --2- 172.21.15.52:0/3120642582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c0c7a0 0x55b4d2c12b00 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.340+0000 7f05197fa640 0 client.17250 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:03.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.340+0000 7f053eb96740 1 -- 172.21.15.52:0/3120642582 >> 172.21.15.52:0/3120642582 conn(0x55b4d2c06ed0 msgr2=0x55b4d2c08c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:03.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.340+0000 7f053eb96740 1 -- 172.21.15.52:0/3120642582 shutdown_connections 2024-04-02T13:59:03.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.340+0000 7f053eb96740 1 -- 172.21.15.52:0/3120642582 wait complete. 2024-04-02T13:59:03.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.340+0000 7f053eb96740 20 client.17250 trim_cache size 0 max 16384 2024-04-02T13:59:03.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.353+0000 7f053eb96740 1 Processor -- start 2024-04-02T13:59:03.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.353+0000 7f053eb96740 1 -- start start 2024-04-02T13:59:03.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.353+0000 7f053eb96740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4d2c0c7a0 0x55b4d2c15230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.353+0000 7f053eb96740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4d2c0d0c0 0x55b4d2c21950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.353+0000 7f053eb96740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c15ca0 0x55b4d2c119b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.353+0000 7f053eb96740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f05341aedd0 con 0x55b4d2c15ca0 2024-04-02T13:59:03.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.353+0000 7f053eb96740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f05341b0910 con 0x55b4d2c0d0c0 2024-04-02T13:59:03.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.353+0000 7f053eb96740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f05341af4a0 con 0x55b4d2c0c7a0 2024-04-02T13:59:03.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.353+0000 7f053a54c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c15ca0 0x55b4d2c119b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.353+0000 7f053a54c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c15ca0 0x55b4d2c119b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58820/0 (socket says 172.21.15.52:58820) 2024-04-02T13:59:03.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.353+0000 7f053a54c640 1 -- 172.21.15.52:0/678587340 learned_addr learned my addr 172.21.15.52:0/678587340 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:03.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.353+0000 7f0539d4b640 1 --2- 172.21.15.52:0/678587340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4d2c0c7a0 0x55b4d2c15230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.353+0000 7f053954a640 1 --2- 172.21.15.52:0/678587340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4d2c0d0c0 0x55b4d2c21950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.353+0000 7f053a54c640 1 -- 172.21.15.52:0/678587340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4d2c0c7a0 msgr2=0x55b4d2c15230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:03.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.353+0000 7f053a54c640 1 --2- 172.21.15.52:0/678587340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4d2c0c7a0 0x55b4d2c15230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.353+0000 7f053a54c640 1 -- 172.21.15.52:0/678587340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4d2c0d0c0 msgr2=0x55b4d2c21950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:03.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.353+0000 7f053a54c640 1 --2- 172.21.15.52:0/678587340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4d2c0d0c0 0x55b4d2c21950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.353+0000 7f053a54c640 1 -- 172.21.15.52:0/678587340 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4d2c0e6d0 con 0x55b4d2c15ca0 2024-04-02T13:59:03.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.353+0000 7f0539d4b640 1 --2- 172.21.15.52:0/678587340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4d2c0c7a0 0x55b4d2c15230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:59:03.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.354+0000 7f053a54c640 1 --2- 172.21.15.52:0/678587340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c15ca0 0x55b4d2c119b0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f0524006010 tx=0x7f05240092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90585b7c705d8718 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:03.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.354+0000 7f051b7fe640 1 -- 172.21.15.52:0/678587340 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05240122b0 con 0x55b4d2c15ca0 2024-04-02T13:59:03.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.354+0000 7f051b7fe640 1 -- 172.21.15.52:0/678587340 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0524022070 con 0x55b4d2c15ca0 2024-04-02T13:59:03.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.354+0000 7f051b7fe640 1 -- 172.21.15.52:0/678587340 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05240050c0 con 0x55b4d2c15ca0 2024-04-02T13:59:03.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.354+0000 7f053eb96740 1 -- 172.21.15.52:0/678587340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c15ca0 msgr2=0x55b4d2c119b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.354+0000 7f053eb96740 1 --2- 172.21.15.52:0/678587340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c15ca0 0x55b4d2c119b0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f0524006010 tx=0x7f05240092e0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.354+0000 7f053eb96740 1 -- 172.21.15.52:0/678587340 shutdown_connections 2024-04-02T13:59:03.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.354+0000 7f053eb96740 1 --2- 172.21.15.52:0/678587340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c15ca0 0x55b4d2c119b0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.354+0000 7f053eb96740 1 --2- 172.21.15.52:0/678587340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4d2c0d0c0 0x55b4d2c21950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.354+0000 7f053eb96740 1 --2- 172.21.15.52:0/678587340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4d2c0c7a0 0x55b4d2c15230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.354+0000 7f053eb96740 1 -- 172.21.15.52:0/678587340 >> 172.21.15.52:0/678587340 conn(0x55b4d2c06ed0 msgr2=0x55b4d2c112a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:03.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.354+0000 7f053eb96740 1 -- 172.21.15.52:0/678587340 shutdown_connections 2024-04-02T13:59:03.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.354+0000 7f053eb96740 1 -- 172.21.15.52:0/678587340 wait complete. 2024-04-02T13:59:03.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.355+0000 7f053eb96740 1 -- start start 2024-04-02T13:59:03.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.355+0000 7f053eb96740 1 Processor -- start 2024-04-02T13:59:03.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.355+0000 7f053eb96740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4d2c0c7a0 0x55b4d2c1dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.355+0000 7f053eb96740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c0d0c0 0x55b4d2c1e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.355+0000 7f053eb96740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4d2c15ca0 0x55b4d2c18f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.355+0000 7f053eb96740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f05341b0240 con 0x55b4d2c0d0c0 2024-04-02T13:59:03.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.355+0000 7f053eb96740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f05341afb70 con 0x55b4d2c15ca0 2024-04-02T13:59:03.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.355+0000 7f053eb96740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f05341acdb0 con 0x55b4d2c0c7a0 2024-04-02T13:59:03.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.355+0000 7f053eb96740 10 client.0 _ll_register_callbacks cb 0x7f052c00b8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:03.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.355+0000 7f053954a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c0d0c0 0x55b4d2c1e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.355+0000 7f053954a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c0d0c0 0x55b4d2c1e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58834/0 (socket says 172.21.15.52:58834) 2024-04-02T13:59:03.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.355+0000 7f053954a640 1 -- 172.21.15.52:0/2506027336 learned_addr learned my addr 172.21.15.52:0/2506027336 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:03.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.355+0000 7f0539d4b640 1 --2- 172.21.15.52:0/2506027336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4d2c0c7a0 0x55b4d2c1dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.356+0000 7f053a54c640 1 --2- 172.21.15.52:0/2506027336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4d2c15ca0 0x55b4d2c18f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.356+0000 7f053954a640 1 -- 172.21.15.52:0/2506027336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4d2c0c7a0 msgr2=0x55b4d2c1dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:03.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.356+0000 7f053954a640 1 --2- 172.21.15.52:0/2506027336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4d2c0c7a0 0x55b4d2c1dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.356+0000 7f053954a640 1 -- 172.21.15.52:0/2506027336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4d2c15ca0 msgr2=0x55b4d2c18f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:03.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.356+0000 7f053954a640 1 --2- 172.21.15.52:0/2506027336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4d2c15ca0 0x55b4d2c18f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.356+0000 7f053954a640 1 -- 172.21.15.52:0/2506027336 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4d2c196c0 con 0x55b4d2c0d0c0 2024-04-02T13:59:03.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.356+0000 7f053954a640 1 --2- 172.21.15.52:0/2506027336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c0d0c0 0x55b4d2c1e000 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f05280be160 tx=0x7f0528031ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506eb4aee67720b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:03.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.356+0000 7f05197fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:03.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.356+0000 7f05197fa640 1 -- 172.21.15.52:0/2506027336 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052804b040 con 0x55b4d2c0d0c0 2024-04-02T13:59:03.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.356+0000 7f05197fa640 1 -- 172.21.15.52:0/2506027336 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f05280ba840 con 0x55b4d2c0d0c0 2024-04-02T13:59:03.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.356+0000 7f053eb96740 1 -- 172.21.15.52:0/2506027336 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0524022070 con 0x55b4d2c0d0c0 2024-04-02T13:59:03.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.356+0000 7f05197fa640 1 -- 172.21.15.52:0/2506027336 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0528004c80 con 0x55b4d2c0d0c0 2024-04-02T13:59:03.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.356+0000 7f053eb96740 10 client.17256 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:03.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.356+0000 7f053eb96740 1 -- 172.21.15.52:0/2506027336 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b4d2c8c000 con 0x55b4d2c0d0c0 2024-04-02T13:59:03.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.356+0000 7f05197fa640 1 -- 172.21.15.52:0/2506027336 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0524022070 con 0x55b4d2c0d0c0 2024-04-02T13:59:03.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.356+0000 7f05197fa640 1 -- 172.21.15.52:0/2506027336 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f05280bcd00 con 0x55b4d2c0d0c0 2024-04-02T13:59:03.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.356+0000 7f053eb96740 10 client.17256 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:03.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.356+0000 7f053eb96740 10 client.17256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:59:03.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.356+0000 7f053eb96740 20 client.17256 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:03.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.356+0000 7f053eb96740 1 --2- 172.21.15.52:0/2506027336 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55b4d2c8c730 0x55b4d2c8eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:03.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.356+0000 7f053eb96740 4 client.17256 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:59:03.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.356+0000 7f053eb96740 1 -- 172.21.15.52:0/2506027336 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b4d2c0d950 con 0x55b4d2c8c730 2024-04-02T13:59:03.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.357+0000 7f0539d4b640 1 --2- 172.21.15.52:0/2506027336 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55b4d2c8c730 0x55b4d2c8eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:03.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.357+0000 7f05197fa640 10 client.17256 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:03.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.357+0000 7f0539d4b640 1 --2- 172.21.15.52:0/2506027336 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55b4d2c8c730 0x55b4d2c8eb50 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7214cb261df8e83f server_cookie=4e30ea8122252317 in_seq=0 out_seq=0 2024-04-02T13:59:03.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.357+0000 7f05197fa640 1 -- 172.21.15.52:0/2506027336 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b4d2c0d950 con 0x55b4d2c8c730 2024-04-02T13:59:03.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.357+0000 7f05197fa640 10 client.17256 handle_command_reply: tid=0 2024-04-02T13:59:03.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.357+0000 7f053eb96740 1 client.17256 shutdown 2024-04-02T13:59:03.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.357+0000 7f053eb96740 1 -- 172.21.15.52:0/2506027336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c0d0c0 msgr2=0x55b4d2c1e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.357+0000 7f053eb96740 1 --2- 172.21.15.52:0/2506027336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c0d0c0 0x55b4d2c1e000 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f05280be160 tx=0x7f0528031ba0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.357+0000 7f053eb96740 1 -- 172.21.15.52:0/2506027336 shutdown_connections 2024-04-02T13:59:03.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.357+0000 7f053eb96740 1 --2- 172.21.15.52:0/2506027336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b4d2c15ca0 0x55b4d2c18f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.357+0000 7f053eb96740 1 --2- 172.21.15.52:0/2506027336 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55b4d2c8c730 0x55b4d2c8eb50 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.357+0000 7f053eb96740 1 --2- 172.21.15.52:0/2506027336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4d2c0d0c0 0x55b4d2c1e000 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.357+0000 7f053eb96740 1 --2- 172.21.15.52:0/2506027336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4d2c0c7a0 0x55b4d2c1dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.357+0000 7f05197fa640 0 client.17256 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:03.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.357+0000 7f053eb96740 1 -- 172.21.15.52:0/2506027336 >> 172.21.15.52:0/2506027336 conn(0x55b4d2c06ed0 msgr2=0x55b4d2c08b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:03.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.357+0000 7f053eb96740 1 -- 172.21.15.52:0/2506027336 shutdown_connections 2024-04-02T13:59:03.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.358+0000 7f053eb96740 1 -- 172.21.15.52:0/2506027336 wait complete. 2024-04-02T13:59:03.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.358+0000 7f053eb96740 20 client.17256 trim_cache size 0 max 16384 2024-04-02T13:59:03.373 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:59:03.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.358+0000 7f053bfaf640 1 -- 172.21.15.52:0/3756715654 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f051c0a3d20 msgr2=0x7f051c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.358+0000 7f053bfaf640 1 --2- 172.21.15.52:0/3756715654 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f051c0a3d20 0x7f051c0a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f052c005d00 tx=0x7f052c00b410 comp rx=0 tx=0).stop 2024-04-02T13:59:03.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.358+0000 7f053bfaf640 1 -- 172.21.15.52:0/3756715654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0534102460 msgr2=0x7f0534198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:03.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.358+0000 7f053bfaf640 1 --2- 172.21.15.52:0/3756715654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0534102460 0x7f0534198920 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0528009230 tx=0x7f052802cba0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.358+0000 7f053bfaf640 1 -- 172.21.15.52:0/3756715654 shutdown_connections 2024-04-02T13:59:03.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.358+0000 7f053bfaf640 1 --2- 172.21.15.52:0/3756715654 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f051c0a3d20 0x7f051c0a61e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.358+0000 7f053bfaf640 1 --2- 172.21.15.52:0/3756715654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0534107ab0 0x7f053419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.358+0000 7f053bfaf640 1 --2- 172.21.15.52:0/3756715654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0534102460 0x7f0534198920 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.358+0000 7f053bfaf640 1 --2- 172.21.15.52:0/3756715654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0534101aa0 0x7f05341983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:03.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.358+0000 7f053bfaf640 1 -- 172.21.15.52:0/3756715654 >> 172.21.15.52:0/3756715654 conn(0x7f05340fd530 msgr2=0x7f05340fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:03.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.358+0000 7f053bfaf640 1 -- 172.21.15.52:0/3756715654 shutdown_connections 2024-04-02T13:59:03.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:03.358+0000 7f053bfaf640 1 -- 172.21.15.52:0/3756715654 wait complete. 2024-04-02T13:59:03.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:03 smithi052 ceph-mon[32104]: pgmap v169: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T13:59:03.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:03 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/654397767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:59:03.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:03 smithi189 ceph-mon[36683]: pgmap v169: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T13:59:03.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:03 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/654397767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:59:03.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:03 smithi088 ceph-mon[36528]: pgmap v169: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T13:59:03.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:03 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/654397767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:59:04.377 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:59:04.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:59:04.378 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:59:04.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.494+0000 7fc859a63640 1 -- 172.21.15.52:0/881133896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc854100970 msgr2=0x7fc854100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:04.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.494+0000 7fc859a63640 1 --2- 172.21.15.52:0/881133896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc854100970 0x7fc854100df0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fc848009930 tx=0x7fc84802bc30 comp rx=0 tx=0).stop 2024-04-02T13:59:04.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.494+0000 7fc859a63640 1 -- 172.21.15.52:0/881133896 shutdown_connections 2024-04-02T13:59:04.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.494+0000 7fc859a63640 1 --2- 172.21.15.52:0/881133896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc854101330 0x7fc85410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.494+0000 7fc859a63640 1 --2- 172.21.15.52:0/881133896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc854100970 0x7fc854100df0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.494+0000 7fc859a63640 1 --2- 172.21.15.52:0/881133896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc854106980 0x7fc854106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.494+0000 7fc859a63640 1 -- 172.21.15.52:0/881133896 >> 172.21.15.52:0/881133896 conn(0x7fc854077b00 msgr2=0x7fc854077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:04.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.494+0000 7fc859a63640 1 -- 172.21.15.52:0/881133896 shutdown_connections 2024-04-02T13:59:04.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.494+0000 7fc859a63640 1 -- 172.21.15.52:0/881133896 wait complete. 2024-04-02T13:59:04.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.495+0000 7fc859a63640 1 Processor -- start 2024-04-02T13:59:04.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.495+0000 7fc859a63640 1 -- start start 2024-04-02T13:59:04.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.495+0000 7fc859a63640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc854100970 0x7fc854102f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.495+0000 7fc859a63640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc854101330 0x7fc854103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.495+0000 7fc859a63640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc854106980 0x7fc8541039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.495+0000 7fc859a63640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc854110d90 con 0x7fc854106980 2024-04-02T13:59:04.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.495+0000 7fc859a63640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc854110c10 con 0x7fc854101330 2024-04-02T13:59:04.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.495+0000 7fc859a63640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc854110f10 con 0x7fc854100970 2024-04-02T13:59:04.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.495+0000 7fc853fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc854106980 0x7fc8541039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.495+0000 7fc853fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc854106980 0x7fc8541039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58838/0 (socket says 172.21.15.52:58838) 2024-04-02T13:59:04.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.495+0000 7fc853fff640 1 -- 172.21.15.52:0/4021880948 learned_addr learned my addr 172.21.15.52:0/4021880948 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:04.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.495+0000 7fc8537fe640 1 --2- 172.21.15.52:0/4021880948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc854100970 0x7fc854102f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.495+0000 7fc852ffd640 1 --2- 172.21.15.52:0/4021880948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc854101330 0x7fc854103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.496+0000 7fc853fff640 1 -- 172.21.15.52:0/4021880948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc854100970 msgr2=0x7fc854102f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:04.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.496+0000 7fc853fff640 1 --2- 172.21.15.52:0/4021880948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc854100970 0x7fc854102f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.496+0000 7fc853fff640 1 -- 172.21.15.52:0/4021880948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc854101330 msgr2=0x7fc854103480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:04.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.496+0000 7fc853fff640 1 --2- 172.21.15.52:0/4021880948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc854101330 0x7fc854103480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.496+0000 7fc853fff640 1 -- 172.21.15.52:0/4021880948 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8541a6920 con 0x7fc854106980 2024-04-02T13:59:04.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.496+0000 7fc852ffd640 1 --2- 172.21.15.52:0/4021880948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc854101330 0x7fc854103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:59:04.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.496+0000 7fc853fff640 1 --2- 172.21.15.52:0/4021880948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc854106980 0x7fc8541039c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00b970 tx=0x7fc84c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:04.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.496+0000 7fc850ff9640 1 -- 172.21.15.52:0/4021880948 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c019070 con 0x7fc854106980 2024-04-02T13:59:04.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.496+0000 7fc859a63640 1 -- 172.21.15.52:0/4021880948 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8541a6c10 con 0x7fc854106980 2024-04-02T13:59:04.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.496+0000 7fc850ff9640 1 -- 172.21.15.52:0/4021880948 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc84c00cb40 con 0x7fc854106980 2024-04-02T13:59:04.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.496+0000 7fc850ff9640 1 -- 172.21.15.52:0/4021880948 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c007650 con 0x7fc854106980 2024-04-02T13:59:04.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.496+0000 7fc859a63640 1 -- 172.21.15.52:0/4021880948 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8541a70e0 con 0x7fc854106980 2024-04-02T13:59:04.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.499+0000 7fc850ff9640 1 -- 172.21.15.52:0/4021880948 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc84c01c070 con 0x7fc854106980 2024-04-02T13:59:04.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.499+0000 7fc85c653740 1 Processor -- start 2024-04-02T13:59:04.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.499+0000 7fc85c653740 1 -- start start 2024-04-02T13:59:04.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.499+0000 7fc85c653740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629376712c0 0x5629376716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.499+0000 7fc85c653740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562937671be0 0x56293767a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.499+0000 7fc85c653740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56293767a7c0 0x56293767cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.499+0000 7fc85c653740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5629375d5770 con 0x562937671be0 2024-04-02T13:59:04.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.499+0000 7fc85c653740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56293757d960 con 0x5629376712c0 2024-04-02T13:59:04.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.499+0000 7fc85c653740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5629375d18c0 con 0x56293767a7c0 2024-04-02T13:59:04.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.499+0000 7fc8537fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562937671be0 0x56293767a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.499+0000 7fc8537fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562937671be0 0x56293767a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58854/0 (socket says 172.21.15.52:58854) 2024-04-02T13:59:04.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.499+0000 7fc8537fe640 1 -- 172.21.15.52:0/1131140144 learned_addr learned my addr 172.21.15.52:0/1131140144 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:04.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.499+0000 7fc850ff9640 1 --2- 172.21.15.52:0/4021880948 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc83c0a3d20 0x7fc83c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.499+0000 7fc852ffd640 1 --2- 172.21.15.52:0/1131140144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629376712c0 0x5629376716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.500+0000 7fc852ffd640 1 --2- 172.21.15.52:0/1131140144 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56293767a7c0 0x56293767cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.500+0000 7fc8537fe640 1 -- 172.21.15.52:0/1131140144 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56293767a7c0 msgr2=0x56293767cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:04.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.500+0000 7fc8537fe640 1 --2- 172.21.15.52:0/1131140144 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56293767a7c0 0x56293767cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.500+0000 7fc853fff640 1 --2- 172.21.15.52:0/4021880948 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc83c0a3d20 0x7fc83c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.500+0000 7fc8537fe640 1 -- 172.21.15.52:0/1131140144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629376712c0 msgr2=0x5629376716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:04.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.500+0000 7fc8537fe640 1 --2- 172.21.15.52:0/1131140144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629376712c0 0x5629376716a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.500+0000 7fc8537fe640 1 -- 172.21.15.52:0/1131140144 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629373b17e0 con 0x562937671be0 2024-04-02T13:59:04.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.500+0000 7fc852ffd640 1 --2- 172.21.15.52:0/1131140144 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56293767a7c0 0x56293767cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:59:04.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.500+0000 7fc850ff9640 1 -- 172.21.15.52:0/4021880948 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc84c0cffe0 con 0x7fc854106980 2024-04-02T13:59:04.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.500+0000 7fc8537fe640 1 --2- 172.21.15.52:0/1131140144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562937671be0 0x56293767a280 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fc84400bd10 tx=0x7fc8440107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fea2760abe0fb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:04.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.500+0000 7fc839ffb640 1 -- 172.21.15.52:0/1131140144 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc844004030 con 0x562937671be0 2024-04-02T13:59:04.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.500+0000 7fc839ffb640 1 -- 172.21.15.52:0/1131140144 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8440041d0 con 0x562937671be0 2024-04-02T13:59:04.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.500+0000 7fc839ffb640 1 -- 172.21.15.52:0/1131140144 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8440117e0 con 0x562937671be0 2024-04-02T13:59:04.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.500+0000 7fc853fff640 1 --2- 172.21.15.52:0/4021880948 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc83c0a3d20 0x7fc83c0a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc84c098260 tx=0x7fc84c06a7b0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:04.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.501+0000 7fc85c653740 1 -- 172.21.15.52:0/1131140144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562937671be0 msgr2=0x56293767a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:04.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.501+0000 7fc85c653740 1 --2- 172.21.15.52:0/1131140144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562937671be0 0x56293767a280 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fc84400bd10 tx=0x7fc8440107a0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.501+0000 7fc85c653740 1 -- 172.21.15.52:0/1131140144 shutdown_connections 2024-04-02T13:59:04.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.501+0000 7fc85c653740 1 --2- 172.21.15.52:0/1131140144 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56293767a7c0 0x56293767cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.501+0000 7fc85c653740 1 --2- 172.21.15.52:0/1131140144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562937671be0 0x56293767a280 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.501+0000 7fc85c653740 1 --2- 172.21.15.52:0/1131140144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629376712c0 0x5629376716a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.501+0000 7fc85c653740 1 -- 172.21.15.52:0/1131140144 >> 172.21.15.52:0/1131140144 conn(0x56293766b9f0 msgr2=0x56293766bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:04.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.501+0000 7fc85c653740 1 -- 172.21.15.52:0/1131140144 shutdown_connections 2024-04-02T13:59:04.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.501+0000 7fc85c653740 1 -- 172.21.15.52:0/1131140144 wait complete. 2024-04-02T13:59:04.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.501+0000 7fc85c653740 1 -- start start 2024-04-02T13:59:04.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.501+0000 7fc85c653740 1 Processor -- start 2024-04-02T13:59:04.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.502+0000 7fc85c653740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629376712c0 0x5629376777b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.502+0000 7fc85c653740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562937671be0 0x562937677cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.502+0000 7fc85c653740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56293767a7c0 0x562937678230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.502+0000 7fc85c653740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56293767d5b0 con 0x56293767a7c0 2024-04-02T13:59:04.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.502+0000 7fc85c653740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x562937590d20 con 0x5629376712c0 2024-04-02T13:59:04.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.502+0000 7fc85c653740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56293767d730 con 0x562937671be0 2024-04-02T13:59:04.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.502+0000 7fc85c653740 10 client.0 _ll_register_callbacks cb 0x56293755da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:04.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.502+0000 7fc852ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56293767a7c0 0x562937678230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.502+0000 7fc852ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56293767a7c0 0x562937678230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58860/0 (socket says 172.21.15.52:58860) 2024-04-02T13:59:04.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.502+0000 7fc852ffd640 1 -- 172.21.15.52:0/2430923771 learned_addr learned my addr 172.21.15.52:0/2430923771 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:04.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.502+0000 7fc852ffd640 1 --2- 172.21.15.52:0/2430923771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629376712c0 0x5629376777b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.502+0000 7fc8537fe640 1 --2- 172.21.15.52:0/2430923771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562937671be0 0x562937677cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.502+0000 7fc852ffd640 1 -- 172.21.15.52:0/2430923771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562937671be0 msgr2=0x562937677cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:04.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.502+0000 7fc852ffd640 1 --2- 172.21.15.52:0/2430923771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562937671be0 0x562937677cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.502+0000 7fc852ffd640 1 -- 172.21.15.52:0/2430923771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629376712c0 msgr2=0x5629376777b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:04.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.502+0000 7fc852ffd640 1 --2- 172.21.15.52:0/2430923771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629376712c0 0x5629376777b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.502+0000 7fc852ffd640 1 -- 172.21.15.52:0/2430923771 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562937411f70 con 0x56293767a7c0 2024-04-02T13:59:04.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.502+0000 7fc852ffd640 1 --2- 172.21.15.52:0/2430923771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56293767a7c0 0x562937678230 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fc848036040 tx=0x7fc8480056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8090575d093ce7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:04.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.502+0000 7fc832ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:04.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.502+0000 7fc832ffd640 1 -- 172.21.15.52:0/2430923771 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848044070 con 0x56293767a7c0 2024-04-02T13:59:04.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.502+0000 7fc832ffd640 1 -- 172.21.15.52:0/2430923771 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc8480035f0 con 0x56293767a7c0 2024-04-02T13:59:04.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.502+0000 7fc85c653740 1 -- 172.21.15.52:0/2430923771 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8440041d0 con 0x56293767a7c0 2024-04-02T13:59:04.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.503+0000 7fc832ffd640 1 -- 172.21.15.52:0/2430923771 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc848050050 con 0x56293767a7c0 2024-04-02T13:59:04.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.503+0000 7fc85c653740 10 client.17268 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:04.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.503+0000 7fc85c653740 1 -- 172.21.15.52:0/2430923771 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5629372a9170 con 0x56293767a7c0 2024-04-02T13:59:04.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.503+0000 7fc832ffd640 1 -- 172.21.15.52:0/2430923771 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8440041d0 con 0x56293767a7c0 2024-04-02T13:59:04.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.503+0000 7fc832ffd640 1 -- 172.21.15.52:0/2430923771 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc848034ab0 con 0x56293767a7c0 2024-04-02T13:59:04.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.503+0000 7fc85c653740 10 client.17268 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:04.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.503+0000 7fc85c653740 10 client.17268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:59:04.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.503+0000 7fc85c653740 20 client.17268 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:04.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.503+0000 7fc85c653740 1 --2- 172.21.15.52:0/2430923771 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x56293768a9b0 0x56293768ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.503+0000 7fc85c653740 4 client.17268 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:59:04.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.503+0000 7fc85c653740 1 -- 172.21.15.52:0/2430923771 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629375e2cb0 con 0x56293768a9b0 2024-04-02T13:59:04.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.503+0000 7fc8537fe640 1 --2- 172.21.15.52:0/2430923771 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x56293768a9b0 0x56293768ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.504+0000 7fc832ffd640 10 client.17268 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:04.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.504+0000 7fc8537fe640 1 --2- 172.21.15.52:0/2430923771 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x56293768a9b0 0x56293768ce30 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3db03cdf04187809 server_cookie=859738acffb93d95 in_seq=0 out_seq=0 2024-04-02T13:59:04.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.504+0000 7fc832ffd640 1 -- 172.21.15.52:0/2430923771 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629375e2cb0 con 0x56293768a9b0 2024-04-02T13:59:04.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.504+0000 7fc832ffd640 10 client.17268 handle_command_reply: tid=0 2024-04-02T13:59:04.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.504+0000 7fc85c653740 1 client.17268 shutdown 2024-04-02T13:59:04.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.505+0000 7fc85c653740 1 -- 172.21.15.52:0/2430923771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56293767a7c0 msgr2=0x562937678230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:04.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.505+0000 7fc85c653740 1 --2- 172.21.15.52:0/2430923771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56293767a7c0 0x562937678230 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fc848036040 tx=0x7fc8480056d0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.505+0000 7fc85c653740 1 -- 172.21.15.52:0/2430923771 shutdown_connections 2024-04-02T13:59:04.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.505+0000 7fc85c653740 1 --2- 172.21.15.52:0/2430923771 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x56293768a9b0 0x56293768ce30 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.505+0000 7fc85c653740 1 --2- 172.21.15.52:0/2430923771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56293767a7c0 0x562937678230 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.505+0000 7fc85c653740 1 --2- 172.21.15.52:0/2430923771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562937671be0 0x562937677cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.505+0000 7fc85c653740 1 --2- 172.21.15.52:0/2430923771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629376712c0 0x5629376777b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.505+0000 7fc832ffd640 0 client.17268 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:04.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.505+0000 7fc85c653740 1 -- 172.21.15.52:0/2430923771 >> 172.21.15.52:0/2430923771 conn(0x56293766b9f0 msgr2=0x56293766d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:04.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.505+0000 7fc85c653740 1 -- 172.21.15.52:0/2430923771 shutdown_connections 2024-04-02T13:59:04.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.505+0000 7fc85c653740 1 -- 172.21.15.52:0/2430923771 wait complete. 2024-04-02T13:59:04.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.505+0000 7fc85c653740 20 client.17268 trim_cache size 0 max 16384 2024-04-02T13:59:04.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.523+0000 7fc85c653740 1 Processor -- start 2024-04-02T13:59:04.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.523+0000 7fc85c653740 1 -- start start 2024-04-02T13:59:04.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.523+0000 7fc85c653740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629376712c0 0x562937679ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.523+0000 7fc85c653740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562937671be0 0x562937686470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.523+0000 7fc85c653740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56293767a7c0 0x562937692700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.523+0000 7fc85c653740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541b1020 con 0x56293767a7c0 2024-04-02T13:59:04.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.523+0000 7fc85c653740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541b2b60 con 0x562937671be0 2024-04-02T13:59:04.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.523+0000 7fc85c653740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541b16f0 con 0x5629376712c0 2024-04-02T13:59:04.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.523+0000 7fc852ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56293767a7c0 0x562937692700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.523+0000 7fc852ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56293767a7c0 0x562937692700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58868/0 (socket says 172.21.15.52:58868) 2024-04-02T13:59:04.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.523+0000 7fc852ffd640 1 -- 172.21.15.52:0/1517861913 learned_addr learned my addr 172.21.15.52:0/1517861913 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:04.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.523+0000 7fc8537fe640 1 --2- 172.21.15.52:0/1517861913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562937671be0 0x562937686470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.523+0000 7fc852ffd640 1 -- 172.21.15.52:0/1517861913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629376712c0 msgr2=0x562937679ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:04.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.523+0000 7fc852ffd640 1 --2- 172.21.15.52:0/1517861913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629376712c0 0x562937679ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.523+0000 7fc852ffd640 1 -- 172.21.15.52:0/1517861913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562937671be0 msgr2=0x562937686470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:04.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.523+0000 7fc852ffd640 1 --2- 172.21.15.52:0/1517861913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562937671be0 0x562937686470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.523+0000 7fc852ffd640 1 -- 172.21.15.52:0/1517861913 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562937665ec0 con 0x56293767a7c0 2024-04-02T13:59:04.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.523+0000 7fc852ffd640 1 --2- 172.21.15.52:0/1517861913 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56293767a7c0 0x562937692700 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fc848006010 tx=0x7fc848009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1110139828b09de server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:04.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.524+0000 7fc8397fa640 1 -- 172.21.15.52:0/1517861913 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848058070 con 0x56293767a7c0 2024-04-02T13:59:04.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.524+0000 7fc8397fa640 1 -- 172.21.15.52:0/1517861913 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc848040540 con 0x56293767a7c0 2024-04-02T13:59:04.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.524+0000 7fc8397fa640 1 -- 172.21.15.52:0/1517861913 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848044040 con 0x56293767a7c0 2024-04-02T13:59:04.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.524+0000 7fc85c653740 1 -- 172.21.15.52:0/1517861913 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56293767a7c0 msgr2=0x562937692700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:04.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.524+0000 7fc85c653740 1 --2- 172.21.15.52:0/1517861913 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56293767a7c0 0x562937692700 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fc848006010 tx=0x7fc848009830 comp rx=0 tx=0).stop 2024-04-02T13:59:04.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.524+0000 7fc85c653740 1 -- 172.21.15.52:0/1517861913 shutdown_connections 2024-04-02T13:59:04.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.524+0000 7fc85c653740 1 --2- 172.21.15.52:0/1517861913 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56293767a7c0 0x562937692700 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.524+0000 7fc85c653740 1 --2- 172.21.15.52:0/1517861913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562937671be0 0x562937686470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.524+0000 7fc85c653740 1 --2- 172.21.15.52:0/1517861913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629376712c0 0x562937679ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.524+0000 7fc85c653740 1 -- 172.21.15.52:0/1517861913 >> 172.21.15.52:0/1517861913 conn(0x56293766b9f0 msgr2=0x56293766d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:04.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.524+0000 7fc85c653740 1 -- 172.21.15.52:0/1517861913 shutdown_connections 2024-04-02T13:59:04.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.524+0000 7fc85c653740 1 -- 172.21.15.52:0/1517861913 wait complete. 2024-04-02T13:59:04.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.524+0000 7fc85c653740 1 -- start start 2024-04-02T13:59:04.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.525+0000 7fc85c653740 1 Processor -- start 2024-04-02T13:59:04.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.525+0000 7fc85c653740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629376712c0 0x5629376829c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.525+0000 7fc85c653740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562937671be0 0x562937682f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.525+0000 7fc85c653740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56293767a7c0 0x56293767ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.525+0000 7fc85c653740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541b2490 con 0x5629376712c0 2024-04-02T13:59:04.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.525+0000 7fc85c653740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541b1dc0 con 0x562937671be0 2024-04-02T13:59:04.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.525+0000 7fc85c653740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541af000 con 0x56293767a7c0 2024-04-02T13:59:04.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.525+0000 7fc85c653740 10 client.0 _ll_register_callbacks cb 0x7fc848009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:04.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.525+0000 7fc852ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629376712c0 0x5629376829c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.525+0000 7fc852ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629376712c0 0x5629376829c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58872/0 (socket says 172.21.15.52:58872) 2024-04-02T13:59:04.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.525+0000 7fc852ffd640 1 -- 172.21.15.52:0/1043364949 learned_addr learned my addr 172.21.15.52:0/1043364949 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:04.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.525+0000 7fc8537fe640 1 --2- 172.21.15.52:0/1043364949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562937671be0 0x562937682f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.525+0000 7fc852ffd640 1 -- 172.21.15.52:0/1043364949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56293767a7c0 msgr2=0x56293767ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:04.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.525+0000 7fc852ffd640 1 --2- 172.21.15.52:0/1043364949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56293767a7c0 0x56293767ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.525+0000 7fc852ffd640 1 -- 172.21.15.52:0/1043364949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562937671be0 msgr2=0x562937682f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:04.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.525+0000 7fc852ffd640 1 --2- 172.21.15.52:0/1043364949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562937671be0 0x562937682f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.525+0000 7fc852ffd640 1 -- 172.21.15.52:0/1043364949 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56293767f740 con 0x5629376712c0 2024-04-02T13:59:04.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.525+0000 7fc8537fe640 1 --2- 172.21.15.52:0/1043364949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562937671be0 0x562937682f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:59:04.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.526+0000 7fc852ffd640 1 --2- 172.21.15.52:0/1043364949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629376712c0 0x5629376829c0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fc84802c170 tx=0x7fc84802cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fec20302ebb98ec server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:04.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.526+0000 7fc832ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:04.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.526+0000 7fc832ffd640 1 -- 172.21.15.52:0/1043364949 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848058070 con 0x5629376712c0 2024-04-02T13:59:04.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.526+0000 7fc832ffd640 1 -- 172.21.15.52:0/1043364949 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc848034c10 con 0x5629376712c0 2024-04-02T13:59:04.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.526+0000 7fc85c653740 1 -- 172.21.15.52:0/1043364949 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc848040540 con 0x5629376712c0 2024-04-02T13:59:04.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.526+0000 7fc832ffd640 1 -- 172.21.15.52:0/1043364949 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc84804d070 con 0x5629376712c0 2024-04-02T13:59:04.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.526+0000 7fc85c653740 10 client.17274 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:04.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.526+0000 7fc85c653740 1 -- 172.21.15.52:0/1043364949 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5629376f1c50 con 0x5629376712c0 2024-04-02T13:59:04.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.526+0000 7fc832ffd640 1 -- 172.21.15.52:0/1043364949 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc848040540 con 0x5629376712c0 2024-04-02T13:59:04.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.526+0000 7fc832ffd640 1 -- 172.21.15.52:0/1043364949 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc848049a50 con 0x5629376712c0 2024-04-02T13:59:04.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.526+0000 7fc85c653740 10 client.17274 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:04.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.526+0000 7fc85c653740 10 client.17274 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:59:04.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.526+0000 7fc85c653740 20 client.17274 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:04.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.526+0000 7fc85c653740 1 --2- 172.21.15.52:0/1043364949 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5629376f2200 0x5629376f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.526+0000 7fc85c653740 4 client.17274 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:59:04.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.526+0000 7fc85c653740 1 -- 172.21.15.52:0/1043364949 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5629375dd000 con 0x5629376f2200 2024-04-02T13:59:04.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.527+0000 7fc8537fe640 1 --2- 172.21.15.52:0/1043364949 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5629376f2200 0x5629376f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.527+0000 7fc8537fe640 1 --2- 172.21.15.52:0/1043364949 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5629376f2200 0x5629376f45f0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e606ccc578c3ffad server_cookie=326240b8d7861775 in_seq=0 out_seq=0 2024-04-02T13:59:04.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.527+0000 7fc832ffd640 10 client.17274 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:04.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.528+0000 7fc832ffd640 1 -- 172.21.15.52:0/1043364949 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5629375dd000 con 0x5629376f2200 2024-04-02T13:59:04.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.528+0000 7fc832ffd640 10 client.17274 handle_command_reply: tid=0 2024-04-02T13:59:04.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.528+0000 7fc85c653740 1 client.17274 shutdown 2024-04-02T13:59:04.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.528+0000 7fc85c653740 1 -- 172.21.15.52:0/1043364949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629376712c0 msgr2=0x5629376829c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:04.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.528+0000 7fc85c653740 1 --2- 172.21.15.52:0/1043364949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629376712c0 0x5629376829c0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fc84802c170 tx=0x7fc84802cd30 comp rx=0 tx=0).stop 2024-04-02T13:59:04.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.528+0000 7fc85c653740 1 -- 172.21.15.52:0/1043364949 shutdown_connections 2024-04-02T13:59:04.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.528+0000 7fc85c653740 1 --2- 172.21.15.52:0/1043364949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56293767a7c0 0x56293767ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.528+0000 7fc85c653740 1 --2- 172.21.15.52:0/1043364949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562937671be0 0x562937682f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.528+0000 7fc85c653740 1 --2- 172.21.15.52:0/1043364949 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5629376f2200 0x5629376f45f0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.528+0000 7fc85c653740 1 --2- 172.21.15.52:0/1043364949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629376712c0 0x5629376829c0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.528+0000 7fc85c653740 1 -- 172.21.15.52:0/1043364949 >> 172.21.15.52:0/1043364949 conn(0x56293766b9f0 msgr2=0x562937677770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:04.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.528+0000 7fc832ffd640 0 client.17274 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:04.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.528+0000 7fc85c653740 1 -- 172.21.15.52:0/1043364949 shutdown_connections 2024-04-02T13:59:04.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.528+0000 7fc85c653740 1 -- 172.21.15.52:0/1043364949 wait complete. 2024-04-02T13:59:04.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.528+0000 7fc85c653740 20 client.17274 trim_cache size 0 max 16384 2024-04-02T13:59:04.546 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:59:04.546 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-04-02T13:59:04.546 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "bc33a038-db2f-4019-8d64-56957e432b0d", 2024-04-02T13:59:04.546 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-04-02T13:59:04.546 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:59:04.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.529+0000 7fc859a63640 1 -- 172.21.15.52:0/4021880948 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc83c0a3d20 msgr2=0x7fc83c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:04.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.529+0000 7fc859a63640 1 --2- 172.21.15.52:0/4021880948 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc83c0a3d20 0x7fc83c0a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc84c098260 tx=0x7fc84c06a7b0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.529+0000 7fc859a63640 1 -- 172.21.15.52:0/4021880948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc854106980 msgr2=0x7fc8541039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:04.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.529+0000 7fc859a63640 1 --2- 172.21.15.52:0/4021880948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc854106980 0x7fc8541039c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00b970 tx=0x7fc84c00be30 comp rx=0 tx=0).stop 2024-04-02T13:59:04.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.529+0000 7fc859a63640 1 -- 172.21.15.52:0/4021880948 shutdown_connections 2024-04-02T13:59:04.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.529+0000 7fc859a63640 1 --2- 172.21.15.52:0/4021880948 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc83c0a3d20 0x7fc83c0a61e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.529+0000 7fc859a63640 1 --2- 172.21.15.52:0/4021880948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc854106980 0x7fc8541039c0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.529+0000 7fc859a63640 1 --2- 172.21.15.52:0/4021880948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc854101330 0x7fc854103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.529+0000 7fc859a63640 1 --2- 172.21.15.52:0/4021880948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc854100970 0x7fc854102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.529+0000 7fc859a63640 1 -- 172.21.15.52:0/4021880948 >> 172.21.15.52:0/4021880948 conn(0x7fc854077b00 msgr2=0x7fc854068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:04.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.529+0000 7fc859a63640 1 -- 172.21.15.52:0/4021880948 shutdown_connections 2024-04-02T13:59:04.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.529+0000 7fc859a63640 1 -- 172.21.15.52:0/4021880948 wait complete. 2024-04-02T13:59:04.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:59:04.550 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:59:04.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.658+0000 7ff3a2663640 1 -- 172.21.15.52:0/3521077288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff39c103f10 msgr2=0x7ff39c1042f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:04.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.658+0000 7ff3a2663640 1 --2- 172.21.15.52:0/3521077288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff39c103f10 0x7ff39c1042f0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009900 tx=0x7ff38c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:59:04.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.658+0000 7ff3a2663640 1 -- 172.21.15.52:0/3521077288 shutdown_connections 2024-04-02T13:59:04.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.658+0000 7ff3a2663640 1 --2- 172.21.15.52:0/3521077288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff39c108fe0 0x7ff39c110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.658+0000 7ff3a2663640 1 --2- 172.21.15.52:0/3521077288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff39c1048c0 0x7ff39c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.658+0000 7ff3a2663640 1 --2- 172.21.15.52:0/3521077288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff39c103f10 0x7ff39c1042f0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.658+0000 7ff3a2663640 1 -- 172.21.15.52:0/3521077288 >> 172.21.15.52:0/3521077288 conn(0x7ff39c0ff790 msgr2=0x7ff39c101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:04.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.658+0000 7ff3a2663640 1 -- 172.21.15.52:0/3521077288 shutdown_connections 2024-04-02T13:59:04.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.658+0000 7ff3a2663640 1 -- 172.21.15.52:0/3521077288 wait complete. 2024-04-02T13:59:04.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.659+0000 7ff3a2663640 1 Processor -- start 2024-04-02T13:59:04.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.659+0000 7ff3a2663640 1 -- start start 2024-04-02T13:59:04.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.659+0000 7ff3a2663640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff39c103f10 0x7ff39c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.659+0000 7ff3a2663640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff39c108fe0 0x7ff39c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.659+0000 7ff3a2663640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff39c19ef10 0x7ff39c19f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.659+0000 7ff3a2663640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c114090 con 0x7ff39c108fe0 2024-04-02T13:59:04.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.659+0000 7ff3a2663640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c113f10 con 0x7ff39c103f10 2024-04-02T13:59:04.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.659+0000 7ff3a2663640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c114210 con 0x7ff39c19ef10 2024-04-02T13:59:04.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.659+0000 7ff3937fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff39c108fe0 0x7ff39c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.659+0000 7ff39bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff39c103f10 0x7ff39c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.659+0000 7ff3937fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff39c108fe0 0x7ff39c19ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58898/0 (socket says 172.21.15.52:58898) 2024-04-02T13:59:04.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.659+0000 7ff3937fe640 1 -- 172.21.15.52:0/727511567 learned_addr learned my addr 172.21.15.52:0/727511567 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:04.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.660+0000 7ff3a0c00640 1 --2- 172.21.15.52:0/727511567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff39c19ef10 0x7ff39c19f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.660+0000 7ff3937fe640 1 -- 172.21.15.52:0/727511567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff39c19ef10 msgr2=0x7ff39c19f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:04.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.660+0000 7ff3937fe640 1 --2- 172.21.15.52:0/727511567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff39c19ef10 0x7ff39c19f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.660+0000 7ff3937fe640 1 -- 172.21.15.52:0/727511567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff39c103f10 msgr2=0x7ff39c19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:04.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.660+0000 7ff3937fe640 1 --2- 172.21.15.52:0/727511567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff39c103f10 0x7ff39c19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.660+0000 7ff3937fe640 1 -- 172.21.15.52:0/727511567 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39c19fa00 con 0x7ff39c108fe0 2024-04-02T13:59:04.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.660+0000 7ff39bfff640 1 --2- 172.21.15.52:0/727511567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff39c103f10 0x7ff39c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:59:04.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.660+0000 7ff3a0c00640 1 --2- 172.21.15.52:0/727511567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff39c19ef10 0x7ff39c19f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:59:04.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.660+0000 7ff3937fe640 1 --2- 172.21.15.52:0/727511567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff39c108fe0 0x7ff39c19ab90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7ff38400e9c0 tx=0x7ff38400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:04.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.660+0000 7ff399ffb640 1 -- 172.21.15.52:0/727511567 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38400cd80 con 0x7ff39c108fe0 2024-04-02T13:59:04.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.660+0000 7ff3a2663640 1 -- 172.21.15.52:0/727511567 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff39c19fcf0 con 0x7ff39c108fe0 2024-04-02T13:59:04.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.660+0000 7ff399ffb640 1 -- 172.21.15.52:0/727511567 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff384005500 con 0x7ff39c108fe0 2024-04-02T13:59:04.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.661+0000 7ff399ffb640 1 -- 172.21.15.52:0/727511567 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384010690 con 0x7ff39c108fe0 2024-04-02T13:59:04.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.661+0000 7ff3a2663640 1 -- 172.21.15.52:0/727511567 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff39c1a74e0 con 0x7ff39c108fe0 2024-04-02T13:59:04.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.663+0000 7ff399ffb640 1 -- 172.21.15.52:0/727511567 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff3840046c0 con 0x7ff39c108fe0 2024-04-02T13:59:04.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.663+0000 7ff399ffb640 1 --2- 172.21.15.52:0/727511567 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff3880a3d20 0x7ff3880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.663+0000 7ff39bfff640 1 --2- 172.21.15.52:0/727511567 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff3880a3d20 0x7ff3880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.663+0000 7ff399ffb640 1 -- 172.21.15.52:0/727511567 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7ff3840d0f60 con 0x7ff39c108fe0 2024-04-02T13:59:04.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.664+0000 7ff3a5254740 1 Processor -- start 2024-04-02T13:59:04.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.664+0000 7ff3a5254740 1 -- start start 2024-04-02T13:59:04.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.664+0000 7ff3a5254740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556225d9b0e0 0x556225d9b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.664+0000 7ff3a5254740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9ba00 0x556225da40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.664+0000 7ff3a5254740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556225da45e0 0x556225da69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.664+0000 7ff39bfff640 1 --2- 172.21.15.52:0/727511567 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff3880a3d20 0x7ff3880a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff38c006010 tx=0x7ff38c005b50 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:04.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.664+0000 7ff3a5254740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556225cf6830 con 0x556225d9ba00 2024-04-02T13:59:04.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.664+0000 7ff3a5254740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556225c9e820 con 0x556225d9b0e0 2024-04-02T13:59:04.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.664+0000 7ff3a5254740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556225cf27e0 con 0x556225da45e0 2024-04-02T13:59:04.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.664+0000 7ff3937fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9ba00 0x556225da40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.664+0000 7ff3937fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9ba00 0x556225da40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58906/0 (socket says 172.21.15.52:58906) 2024-04-02T13:59:04.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.664+0000 7ff3937fe640 1 -- 172.21.15.52:0/2051861338 learned_addr learned my addr 172.21.15.52:0/2051861338 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:04.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.664+0000 7ff39bfff640 1 --2- 172.21.15.52:0/2051861338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556225d9b0e0 0x556225d9b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.664+0000 7ff3a0c00640 1 --2- 172.21.15.52:0/2051861338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556225da45e0 0x556225da69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.664+0000 7ff3937fe640 1 -- 172.21.15.52:0/2051861338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556225da45e0 msgr2=0x556225da69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:04.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.664+0000 7ff3937fe640 1 --2- 172.21.15.52:0/2051861338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556225da45e0 0x556225da69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.665+0000 7ff3937fe640 1 -- 172.21.15.52:0/2051861338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556225d9b0e0 msgr2=0x556225d9b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:04.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.665+0000 7ff3937fe640 1 --2- 172.21.15.52:0/2051861338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556225d9b0e0 0x556225d9b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.665+0000 7ff3937fe640 1 -- 172.21.15.52:0/2051861338 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556225ad47d0 con 0x556225d9ba00 2024-04-02T13:59:04.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.665+0000 7ff3937fe640 1 --2- 172.21.15.52:0/2051861338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9ba00 0x556225da40a0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7ff3840a5300 tx=0x7ff38409c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c973c59af998ceef server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:04.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.665+0000 7ff3927fc640 1 -- 172.21.15.52:0/2051861338 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3840a0f70 con 0x556225d9ba00 2024-04-02T13:59:04.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.665+0000 7ff3927fc640 1 -- 172.21.15.52:0/2051861338 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3840a5830 con 0x556225d9ba00 2024-04-02T13:59:04.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.665+0000 7ff3927fc640 1 -- 172.21.15.52:0/2051861338 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3840a53a0 con 0x556225d9ba00 2024-04-02T13:59:04.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.665+0000 7ff3a5254740 1 -- 172.21.15.52:0/2051861338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9ba00 msgr2=0x556225da40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:04.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.666+0000 7ff3a5254740 1 --2- 172.21.15.52:0/2051861338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9ba00 0x556225da40a0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7ff3840a5300 tx=0x7ff38409c520 comp rx=0 tx=0).stop 2024-04-02T13:59:04.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.666+0000 7ff3a5254740 1 -- 172.21.15.52:0/2051861338 shutdown_connections 2024-04-02T13:59:04.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.666+0000 7ff3a5254740 1 --2- 172.21.15.52:0/2051861338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556225da45e0 0x556225da69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.666+0000 7ff3a5254740 1 --2- 172.21.15.52:0/2051861338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9ba00 0x556225da40a0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.666+0000 7ff3a5254740 1 --2- 172.21.15.52:0/2051861338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556225d9b0e0 0x556225d9b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.666+0000 7ff3a5254740 1 -- 172.21.15.52:0/2051861338 >> 172.21.15.52:0/2051861338 conn(0x556225d95810 msgr2=0x556225d95bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:04.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.666+0000 7ff3a5254740 1 -- 172.21.15.52:0/2051861338 shutdown_connections 2024-04-02T13:59:04.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.666+0000 7ff3a5254740 1 -- 172.21.15.52:0/2051861338 wait complete. 2024-04-02T13:59:04.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.666+0000 7ff3a5254740 1 -- start start 2024-04-02T13:59:04.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.666+0000 7ff3a5254740 1 Processor -- start 2024-04-02T13:59:04.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.666+0000 7ff3a5254740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556225d9b0e0 0x556225da3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.666+0000 7ff3a5254740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9ba00 0x556225da1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.666+0000 7ff3a5254740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556225da45e0 0x556225da1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.666+0000 7ff3a5254740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556225da73d0 con 0x556225d9ba00 2024-04-02T13:59:04.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.666+0000 7ff3a5254740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556225cb1d30 con 0x556225da45e0 2024-04-02T13:59:04.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.666+0000 7ff3a5254740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556225da7550 con 0x556225d9b0e0 2024-04-02T13:59:04.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.666+0000 7ff3a5254740 10 client.0 _ll_register_callbacks cb 0x556225cb1720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:04.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.667+0000 7ff3937fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9ba00 0x556225da1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.667+0000 7ff3937fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9ba00 0x556225da1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58910/0 (socket says 172.21.15.52:58910) 2024-04-02T13:59:04.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.667+0000 7ff3937fe640 1 -- 172.21.15.52:0/4054043975 learned_addr learned my addr 172.21.15.52:0/4054043975 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:04.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.667+0000 7ff39bfff640 1 --2- 172.21.15.52:0/4054043975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556225d9b0e0 0x556225da3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.667+0000 7ff3a0c00640 1 --2- 172.21.15.52:0/4054043975 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556225da45e0 0x556225da1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.667+0000 7ff3937fe640 1 -- 172.21.15.52:0/4054043975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556225d9b0e0 msgr2=0x556225da3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:04.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.667+0000 7ff3937fe640 1 --2- 172.21.15.52:0/4054043975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556225d9b0e0 0x556225da3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.667+0000 7ff3937fe640 1 -- 172.21.15.52:0/4054043975 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556225da45e0 msgr2=0x556225da1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:04.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.667+0000 7ff3937fe640 1 --2- 172.21.15.52:0/4054043975 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556225da45e0 0x556225da1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.667+0000 7ff3937fe640 1 -- 172.21.15.52:0/4054043975 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556225a0a3f0 con 0x556225d9ba00 2024-04-02T13:59:04.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.667+0000 7ff3937fe640 1 --2- 172.21.15.52:0/4054043975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9ba00 0x556225da1230 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7ff38409c8c0 tx=0x7ff38409cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4698c1fa136c4f47 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:04.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.667+0000 7ff37b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:04.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.667+0000 7ff37b7fe640 1 -- 172.21.15.52:0/4054043975 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3840a0f70 con 0x556225d9ba00 2024-04-02T13:59:04.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.667+0000 7ff37b7fe640 1 -- 172.21.15.52:0/4054043975 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7ff3840a53a0 con 0x556225d9ba00 2024-04-02T13:59:04.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.667+0000 7ff3a5254740 1 -- 172.21.15.52:0/4054043975 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3840a5830 con 0x556225d9ba00 2024-04-02T13:59:04.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.668+0000 7ff37b7fe640 1 -- 172.21.15.52:0/4054043975 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff3840a55a0 con 0x556225d9ba00 2024-04-02T13:59:04.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.668+0000 7ff3a5254740 10 client.17286 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:04.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.668+0000 7ff3a5254740 1 -- 172.21.15.52:0/4054043975 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5562259ca0f0 con 0x556225d9ba00 2024-04-02T13:59:04.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.668+0000 7ff37b7fe640 1 -- 172.21.15.52:0/4054043975 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff38409c010 con 0x556225d9ba00 2024-04-02T13:59:04.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.668+0000 7ff37b7fe640 1 -- 172.21.15.52:0/4054043975 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff38409ba80 con 0x556225d9ba00 2024-04-02T13:59:04.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.668+0000 7ff3a5254740 10 client.17286 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:04.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.668+0000 7ff3a5254740 10 client.17286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:59:04.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.668+0000 7ff3a5254740 20 client.17286 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:04.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.668+0000 7ff3a5254740 1 --2- 172.21.15.52:0/4054043975 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556225db4420 0x556225db6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.668+0000 7ff3a5254740 4 client.17286 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:59:04.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.668+0000 7ff3a5254740 1 -- 172.21.15.52:0/4054043975 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff3840a5830 con 0x556225db4420 2024-04-02T13:59:04.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.668+0000 7ff39bfff640 1 --2- 172.21.15.52:0/4054043975 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556225db4420 0x556225db6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.669+0000 7ff39bfff640 1 --2- 172.21.15.52:0/4054043975 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556225db4420 0x556225db6810 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f71843e257c45691 server_cookie=b89b98a9fc1be9b1 in_seq=0 out_seq=0 2024-04-02T13:59:04.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.669+0000 7ff37b7fe640 10 client.17286 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:04.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.669+0000 7ff37b7fe640 1 -- 172.21.15.52:0/4054043975 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7ff3840a5830 con 0x556225db4420 2024-04-02T13:59:04.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.670+0000 7ff37b7fe640 10 client.17286 handle_command_reply: tid=0 2024-04-02T13:59:04.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.670+0000 7ff3a5254740 1 client.17286 shutdown 2024-04-02T13:59:04.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.670+0000 7ff3a5254740 1 -- 172.21.15.52:0/4054043975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9ba00 msgr2=0x556225da1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:04.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.670+0000 7ff3a5254740 1 --2- 172.21.15.52:0/4054043975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9ba00 0x556225da1230 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7ff38409c8c0 tx=0x7ff38409cd40 comp rx=0 tx=0).stop 2024-04-02T13:59:04.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.670+0000 7ff3a5254740 1 -- 172.21.15.52:0/4054043975 shutdown_connections 2024-04-02T13:59:04.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.670+0000 7ff3a5254740 1 --2- 172.21.15.52:0/4054043975 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556225da45e0 0x556225da1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.670+0000 7ff3a5254740 1 --2- 172.21.15.52:0/4054043975 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556225db4420 0x556225db6810 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.670+0000 7ff3a5254740 1 --2- 172.21.15.52:0/4054043975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9ba00 0x556225da1230 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.670+0000 7ff3a5254740 1 --2- 172.21.15.52:0/4054043975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556225d9b0e0 0x556225da3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.670+0000 7ff37b7fe640 0 client.17286 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:04.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.670+0000 7ff3a5254740 1 -- 172.21.15.52:0/4054043975 >> 172.21.15.52:0/4054043975 conn(0x556225d95810 msgr2=0x556225d96cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:04.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.670+0000 7ff3a5254740 1 -- 172.21.15.52:0/4054043975 shutdown_connections 2024-04-02T13:59:04.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.670+0000 7ff3a5254740 1 -- 172.21.15.52:0/4054043975 wait complete. 2024-04-02T13:59:04.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.670+0000 7ff3a5254740 20 client.17286 trim_cache size 0 max 16384 2024-04-02T13:59:04.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.691+0000 7ff3a5254740 1 Processor -- start 2024-04-02T13:59:04.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.691+0000 7ff3a5254740 1 -- start start 2024-04-02T13:59:04.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.691+0000 7ff3a5254740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556225d9b0e0 0x556225da0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.691+0000 7ff3a5254740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9ba00 0x556225dafeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.691+0000 7ff3a5254740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556225da45e0 0x556225da3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.691+0000 7ff3a5254740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c1acca0 con 0x556225d9ba00 2024-04-02T13:59:04.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.691+0000 7ff3a5254740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c1ae7e0 con 0x556225da45e0 2024-04-02T13:59:04.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.691+0000 7ff3a5254740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c1ad370 con 0x556225d9b0e0 2024-04-02T13:59:04.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.692+0000 7ff3937fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9ba00 0x556225dafeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.692+0000 7ff3937fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9ba00 0x556225dafeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58920/0 (socket says 172.21.15.52:58920) 2024-04-02T13:59:04.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.692+0000 7ff3937fe640 1 -- 172.21.15.52:0/461398768 learned_addr learned my addr 172.21.15.52:0/461398768 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:04.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.692+0000 7ff39bfff640 1 --2- 172.21.15.52:0/461398768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556225d9b0e0 0x556225da0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.692+0000 7ff3a0c00640 1 --2- 172.21.15.52:0/461398768 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556225da45e0 0x556225da3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.692+0000 7ff3937fe640 1 -- 172.21.15.52:0/461398768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556225d9b0e0 msgr2=0x556225da0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:04.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.692+0000 7ff3937fe640 1 --2- 172.21.15.52:0/461398768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556225d9b0e0 0x556225da0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.692+0000 7ff3937fe640 1 -- 172.21.15.52:0/461398768 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556225da45e0 msgr2=0x556225da3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:04.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.692+0000 7ff3937fe640 1 --2- 172.21.15.52:0/461398768 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556225da45e0 0x556225da3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.692+0000 7ff3937fe640 1 -- 172.21.15.52:0/461398768 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556225d88c50 con 0x556225d9ba00 2024-04-02T13:59:04.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.692+0000 7ff3937fe640 1 --2- 172.21.15.52:0/461398768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9ba00 0x556225dafeb0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7ff3840ce970 tx=0x7ff3840bd050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e5572163e20068 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:04.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.692+0000 7ff391ffb640 1 -- 172.21.15.52:0/461398768 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3840a0f70 con 0x556225d9ba00 2024-04-02T13:59:04.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.692+0000 7ff391ffb640 1 -- 172.21.15.52:0/461398768 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff38409d2c0 con 0x556225d9ba00 2024-04-02T13:59:04.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.692+0000 7ff391ffb640 1 -- 172.21.15.52:0/461398768 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384061b40 con 0x556225d9ba00 2024-04-02T13:59:04.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.693+0000 7ff3a5254740 1 -- 172.21.15.52:0/461398768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9ba00 msgr2=0x556225dafeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:04.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.693+0000 7ff3a5254740 1 --2- 172.21.15.52:0/461398768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9ba00 0x556225dafeb0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7ff3840ce970 tx=0x7ff3840bd050 comp rx=0 tx=0).stop 2024-04-02T13:59:04.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.693+0000 7ff3a5254740 1 -- 172.21.15.52:0/461398768 shutdown_connections 2024-04-02T13:59:04.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.693+0000 7ff3a5254740 1 --2- 172.21.15.52:0/461398768 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556225da45e0 0x556225da3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.693+0000 7ff3a5254740 1 --2- 172.21.15.52:0/461398768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9ba00 0x556225dafeb0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.693+0000 7ff3a5254740 1 --2- 172.21.15.52:0/461398768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556225d9b0e0 0x556225da0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.693+0000 7ff3a5254740 1 -- 172.21.15.52:0/461398768 >> 172.21.15.52:0/461398768 conn(0x556225d95810 msgr2=0x556225daf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:04.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.693+0000 7ff3a5254740 1 -- 172.21.15.52:0/461398768 shutdown_connections 2024-04-02T13:59:04.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.693+0000 7ff3a5254740 1 -- 172.21.15.52:0/461398768 wait complete. 2024-04-02T13:59:04.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.694+0000 7ff3a5254740 1 -- start start 2024-04-02T13:59:04.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.694+0000 7ff3a5254740 1 Processor -- start 2024-04-02T13:59:04.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.694+0000 7ff3a5254740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9b0e0 0x556225db9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.694+0000 7ff3a5254740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556225d9ba00 0x556225db6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.694+0000 7ff3a5254740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556225da45e0 0x556225db74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.694+0000 7ff3a5254740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c1ae110 con 0x556225d9b0e0 2024-04-02T13:59:04.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.694+0000 7ff3a5254740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c1ada40 con 0x556225d9ba00 2024-04-02T13:59:04.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.694+0000 7ff3a5254740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c1aac80 con 0x556225da45e0 2024-04-02T13:59:04.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.694+0000 7ff3a5254740 10 client.0 _ll_register_callbacks cb 0x7ff38409ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:04.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.694+0000 7ff39bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9b0e0 0x556225db9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.694+0000 7ff39bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9b0e0 0x556225db9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58934/0 (socket says 172.21.15.52:58934) 2024-04-02T13:59:04.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.694+0000 7ff39bfff640 1 -- 172.21.15.52:0/523601071 learned_addr learned my addr 172.21.15.52:0/523601071 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:04.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.694+0000 7ff3937fe640 1 --2- 172.21.15.52:0/523601071 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556225d9ba00 0x556225db6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.694+0000 7ff3a0c00640 1 --2- 172.21.15.52:0/523601071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556225da45e0 0x556225db74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.695+0000 7ff39bfff640 1 -- 172.21.15.52:0/523601071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556225da45e0 msgr2=0x556225db74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:04.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.695+0000 7ff39bfff640 1 --2- 172.21.15.52:0/523601071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556225da45e0 0x556225db74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.695+0000 7ff39bfff640 1 -- 172.21.15.52:0/523601071 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556225d9ba00 msgr2=0x556225db6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:04.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.695+0000 7ff39bfff640 1 --2- 172.21.15.52:0/523601071 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556225d9ba00 0x556225db6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.695+0000 7ff39bfff640 1 -- 172.21.15.52:0/523601071 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556225db7cf0 con 0x556225d9b0e0 2024-04-02T13:59:04.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.695+0000 7ff3937fe640 1 --2- 172.21.15.52:0/523601071 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556225d9ba00 0x556225db6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:59:04.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.695+0000 7ff39bfff640 1 --2- 172.21.15.52:0/523601071 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9b0e0 0x556225db9bf0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7ff38c02c170 tx=0x7ff38c02e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6cfa75f1ae9c25 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:04.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.695+0000 7ff37b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:04.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.695+0000 7ff37b7fe640 1 -- 172.21.15.52:0/523601071 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c0043d0 con 0x556225d9b0e0 2024-04-02T13:59:04.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.695+0000 7ff37b7fe640 1 -- 172.21.15.52:0/523601071 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7ff38c054340 con 0x556225d9b0e0 2024-04-02T13:59:04.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.695+0000 7ff3a5254740 1 -- 172.21.15.52:0/523601071 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff38409d2c0 con 0x556225d9b0e0 2024-04-02T13:59:04.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.695+0000 7ff37b7fe640 1 -- 172.21.15.52:0/523601071 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff38c064050 con 0x556225d9b0e0 2024-04-02T13:59:04.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.695+0000 7ff3a5254740 10 client.17292 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:04.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.695+0000 7ff3a5254740 1 -- 172.21.15.52:0/523601071 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556225e1a5d0 con 0x556225d9b0e0 2024-04-02T13:59:04.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.696+0000 7ff37b7fe640 1 -- 172.21.15.52:0/523601071 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff38409d2c0 con 0x556225d9b0e0 2024-04-02T13:59:04.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.696+0000 7ff37b7fe640 1 -- 172.21.15.52:0/523601071 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff3840bd0c0 con 0x556225d9b0e0 2024-04-02T13:59:04.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.696+0000 7ff3a5254740 10 client.17292 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:04.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.696+0000 7ff3a5254740 10 client.17292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:59:04.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.696+0000 7ff3a5254740 20 client.17292 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:04.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.696+0000 7ff3a5254740 1 --2- 172.21.15.52:0/523601071 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556225e1ad00 0x556225e1d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:04.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.696+0000 7ff3a5254740 4 client.17292 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:59:04.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.696+0000 7ff3a5254740 1 -- 172.21.15.52:0/523601071 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556225d03d70 con 0x556225e1ad00 2024-04-02T13:59:04.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.696+0000 7ff3937fe640 1 --2- 172.21.15.52:0/523601071 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556225e1ad00 0x556225e1d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:04.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.697+0000 7ff3937fe640 1 --2- 172.21.15.52:0/523601071 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556225e1ad00 0x556225e1d120 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90eac1f0051c3f91 server_cookie=b658edba375defa5 in_seq=0 out_seq=0 2024-04-02T13:59:04.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.697+0000 7ff37b7fe640 10 client.17292 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:04.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.697+0000 7ff37b7fe640 1 -- 172.21.15.52:0/523601071 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x556225d03d70 con 0x556225e1ad00 2024-04-02T13:59:04.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.697+0000 7ff37b7fe640 10 client.17292 handle_command_reply: tid=0 2024-04-02T13:59:04.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.697+0000 7ff3a5254740 1 client.17292 shutdown 2024-04-02T13:59:04.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.697+0000 7ff3a5254740 1 -- 172.21.15.52:0/523601071 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9b0e0 msgr2=0x556225db9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:04.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.697+0000 7ff3a5254740 1 --2- 172.21.15.52:0/523601071 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9b0e0 0x556225db9bf0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7ff38c02c170 tx=0x7ff38c02e050 comp rx=0 tx=0).stop 2024-04-02T13:59:04.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.697+0000 7ff3a5254740 1 -- 172.21.15.52:0/523601071 shutdown_connections 2024-04-02T13:59:04.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.697+0000 7ff3a5254740 1 --2- 172.21.15.52:0/523601071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556225da45e0 0x556225db74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.697+0000 7ff3a5254740 1 --2- 172.21.15.52:0/523601071 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556225d9ba00 0x556225db6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.697+0000 7ff3a5254740 1 --2- 172.21.15.52:0/523601071 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556225e1ad00 0x556225e1d120 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.697+0000 7ff3a5254740 1 --2- 172.21.15.52:0/523601071 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556225d9b0e0 0x556225db9bf0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.697+0000 7ff37b7fe640 0 client.17292 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:04.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.697+0000 7ff3a5254740 1 -- 172.21.15.52:0/523601071 >> 172.21.15.52:0/523601071 conn(0x556225d95810 msgr2=0x556225d9bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:04.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.697+0000 7ff3a5254740 1 -- 172.21.15.52:0/523601071 shutdown_connections 2024-04-02T13:59:04.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.697+0000 7ff3a5254740 1 -- 172.21.15.52:0/523601071 wait complete. 2024-04-02T13:59:04.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.697+0000 7ff3a5254740 20 client.17292 trim_cache size 0 max 16384 2024-04-02T13:59:04.713 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:59:04.713 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "scrub active (2375 inodes in the stack)", 2024-04-02T13:59:04.714 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-04-02T13:59:04.714 INFO:teuthology.orchestra.run.smithi052.stdout: "bc33a038-db2f-4019-8d64-56957e432b0d": { 2024-04-02T13:59:04.714 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-04-02T13:59:04.714 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "bc33a038-db2f-4019-8d64-56957e432b0d", 2024-04-02T13:59:04.714 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-04-02T13:59:04.714 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:59:04.714 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:59:04.715 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:59:04.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.698+0000 7ff3a2663640 1 -- 172.21.15.52:0/727511567 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff3880a3d20 msgr2=0x7ff3880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:04.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.698+0000 7ff3a2663640 1 --2- 172.21.15.52:0/727511567 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff3880a3d20 0x7ff3880a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff38c006010 tx=0x7ff38c005b50 comp rx=0 tx=0).stop 2024-04-02T13:59:04.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.698+0000 7ff3a2663640 1 -- 172.21.15.52:0/727511567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff39c108fe0 msgr2=0x7ff39c19ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:04.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.698+0000 7ff3a2663640 1 --2- 172.21.15.52:0/727511567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff39c108fe0 0x7ff39c19ab90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7ff38400e9c0 tx=0x7ff38400ee80 comp rx=0 tx=0).stop 2024-04-02T13:59:04.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.698+0000 7ff3a2663640 1 -- 172.21.15.52:0/727511567 shutdown_connections 2024-04-02T13:59:04.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.698+0000 7ff3a2663640 1 --2- 172.21.15.52:0/727511567 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff3880a3d20 0x7ff3880a61e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.698+0000 7ff3a2663640 1 --2- 172.21.15.52:0/727511567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff39c19ef10 0x7ff39c19f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.698+0000 7ff3a2663640 1 --2- 172.21.15.52:0/727511567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff39c108fe0 0x7ff39c19ab90 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.698+0000 7ff3a2663640 1 --2- 172.21.15.52:0/727511567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff39c103f10 0x7ff39c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:04.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.698+0000 7ff3a2663640 1 -- 172.21.15.52:0/727511567 >> 172.21.15.52:0/727511567 conn(0x7ff39c0ff790 msgr2=0x7ff39c100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:04.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.698+0000 7ff3a2663640 1 -- 172.21.15.52:0/727511567 shutdown_connections 2024-04-02T13:59:04.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:04.698+0000 7ff3a2663640 1 -- 172.21.15.52:0/727511567 wait complete. 2024-04-02T13:59:04.718 INFO:tasks.cephfs.filesystem:scrub status for tag:bc33a038-db2f-4019-8d64-56957e432b0d - {'path': '/', 'tag': 'bc33a038-db2f-4019-8d64-56957e432b0d', 'options': 'recursive,force'} 2024-04-02T13:59:05.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:05 smithi189 ceph-mon[36683]: pgmap v170: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:59:05.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:05 smithi189 ceph-mon[36683]: scrub queued for path: / 2024-04-02T13:59:05.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:05 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:59:05.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:05 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:59:05.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:05 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:59:05.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:05 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:59:05.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:05 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:59:05.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:05 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:59:05.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:05 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:59:05.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:05 smithi052 ceph-mon[32104]: pgmap v170: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:59:05.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:05 smithi052 ceph-mon[32104]: scrub queued for path: / 2024-04-02T13:59:05.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:05 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:59:05.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:05 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:59:05.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:05 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:59:05.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:05 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:59:05.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:05 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:59:05.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:05 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:59:05.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:05 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:59:05.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:05 smithi088 ceph-mon[36528]: pgmap v170: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:59:05.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:05 smithi088 ceph-mon[36528]: scrub queued for path: / 2024-04-02T13:59:05.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:05 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:59:05.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:05 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:59:05.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:05 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:59:05.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:05 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:59:05.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:05 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:59:05.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:05 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:59:05.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:05 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:59:07.722 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:07 smithi088 ceph-mon[36528]: pgmap v171: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T13:59:07.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:07 smithi189 ceph-mon[36683]: pgmap v171: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T13:59:07.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:07 smithi052 ceph-mon[32104]: pgmap v171: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T13:59:09.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:09 smithi189 ceph-mon[36683]: pgmap v172: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:59:09.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:09 smithi052 ceph-mon[32104]: pgmap v172: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:59:09.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:09 smithi088 ceph-mon[36528]: pgmap v172: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:59:10.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:10 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T13:59:10.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:10 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T13:59:10.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:10 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T13:59:11.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:11 smithi189 ceph-mon[36683]: pgmap v173: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:59:11.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:11 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T13:59:11.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:11 smithi052 ceph-mon[32104]: pgmap v173: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:59:11.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:11 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T13:59:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:11 smithi088 ceph-mon[36528]: pgmap v173: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:59:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:11 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T13:59:13.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:13 smithi189 ceph-mon[36683]: pgmap v174: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T13:59:13.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:13 smithi052 ceph-mon[32104]: pgmap v174: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T13:59:13.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:13 smithi088 ceph-mon[36528]: pgmap v174: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T13:59:15.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:15 smithi189 ceph-mon[36683]: pgmap v175: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-02T13:59:15.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:15 smithi052 ceph-mon[32104]: pgmap v175: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-02T13:59:15.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:15 smithi088 ceph-mon[36528]: pgmap v175: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-02T13:59:17.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:17 smithi189 ceph-mon[36683]: pgmap v176: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-02T13:59:17.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:17 smithi052 ceph-mon[32104]: pgmap v176: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-02T13:59:17.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:17 smithi088 ceph-mon[36528]: pgmap v176: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-02T13:59:19.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:19 smithi189 ceph-mon[36683]: pgmap v177: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T13:59:19.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:19 smithi052 ceph-mon[32104]: pgmap v177: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T13:59:19.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:19 smithi088 ceph-mon[36528]: pgmap v177: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T13:59:21.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:21 smithi189 ceph-mon[36683]: pgmap v178: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T13:59:21.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:21 smithi052 ceph-mon[32104]: pgmap v178: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T13:59:21.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:21 smithi088 ceph-mon[36528]: pgmap v178: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T13:59:23.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:23 smithi189 ceph-mon[36683]: pgmap v179: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-02T13:59:23.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:23 smithi052 ceph-mon[32104]: pgmap v179: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-02T13:59:23.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:23 smithi088 ceph-mon[36528]: pgmap v179: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-02T13:59:25.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:25 smithi189 ceph-mon[36683]: pgmap v180: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:25.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:25 smithi052 ceph-mon[32104]: pgmap v180: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:25.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:25 smithi088 ceph-mon[36528]: pgmap v180: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:27.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:27 smithi189 ceph-mon[36683]: pgmap v181: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:59:27.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:27 smithi052 ceph-mon[32104]: pgmap v181: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:59:27.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:27 smithi088 ceph-mon[36528]: pgmap v181: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:59:29.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:29 smithi189 ceph-mon[36683]: pgmap v182: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:29.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:29 smithi052 ceph-mon[32104]: pgmap v182: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:29.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:29 smithi088 ceph-mon[36528]: pgmap v182: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:31.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:31 smithi189 ceph-mon[36683]: pgmap v183: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:31.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:31 smithi052 ceph-mon[32104]: pgmap v183: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:31.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:31 smithi088 ceph-mon[36528]: pgmap v183: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:33.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:33 smithi189 ceph-mon[36683]: pgmap v184: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:59:33.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:33 smithi052 ceph-mon[32104]: pgmap v184: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:59:33.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:33 smithi088 ceph-mon[36528]: pgmap v184: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:59:34.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:59:34.720 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:59:34.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.835+0000 7f7089b0e640 1 -- 172.21.15.52:0/1186820241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70841009b0 msgr2=0x7f7084100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:34.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.835+0000 7f7089b0e640 1 --2- 172.21.15.52:0/1186820241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70841009b0 0x7f7084100e30 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f7078009930 tx=0x7f707802bc30 comp rx=0 tx=0).stop 2024-04-02T13:59:34.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.835+0000 7f7089b0e640 1 -- 172.21.15.52:0/1186820241 shutdown_connections 2024-04-02T13:59:34.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.835+0000 7f7089b0e640 1 --2- 172.21.15.52:0/1186820241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7084101370 0x7f708410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.835+0000 7f7089b0e640 1 --2- 172.21.15.52:0/1186820241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70841009b0 0x7f7084100e30 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.835+0000 7f7089b0e640 1 --2- 172.21.15.52:0/1186820241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70841069c0 0x7f7084106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.835+0000 7f7089b0e640 1 -- 172.21.15.52:0/1186820241 >> 172.21.15.52:0/1186820241 conn(0x7f7084076d80 msgr2=0x7f7084077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:34.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.835+0000 7f7089b0e640 1 -- 172.21.15.52:0/1186820241 shutdown_connections 2024-04-02T13:59:34.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.835+0000 7f7089b0e640 1 -- 172.21.15.52:0/1186820241 wait complete. 2024-04-02T13:59:34.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.836+0000 7f7089b0e640 1 Processor -- start 2024-04-02T13:59:34.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.836+0000 7f7089b0e640 1 -- start start 2024-04-02T13:59:34.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.836+0000 7f7089b0e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70841009b0 0x7f70841972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:34.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.836+0000 7f7089b0e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7084101370 0x7f7084197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:34.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.836+0000 7f7089b0e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70841069c0 0x7f708419bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:34.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.836+0000 7f7089b0e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7084110dd0 con 0x7f7084101370 2024-04-02T13:59:34.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.836+0000 7f7089b0e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7084110c50 con 0x7f70841009b0 2024-04-02T13:59:34.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.836+0000 7f7089b0e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7084110f50 con 0x7f70841069c0 2024-04-02T13:59:34.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.836+0000 7f7082ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7084101370 0x7f7084197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:34.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.836+0000 7f7082ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7084101370 0x7f7084197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51902/0 (socket says 172.21.15.52:51902) 2024-04-02T13:59:34.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.836+0000 7f7082ffd640 1 -- 172.21.15.52:0/2216930612 learned_addr learned my addr 172.21.15.52:0/2216930612 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:34.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.836+0000 7f7083fff640 1 --2- 172.21.15.52:0/2216930612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70841069c0 0x7f708419bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:34.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.836+0000 7f70837fe640 1 --2- 172.21.15.52:0/2216930612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70841009b0 0x7f70841972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:34.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.836+0000 7f7082ffd640 1 -- 172.21.15.52:0/2216930612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70841069c0 msgr2=0x7f708419bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:34.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.836+0000 7f7082ffd640 1 --2- 172.21.15.52:0/2216930612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70841069c0 0x7f708419bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.836+0000 7f7082ffd640 1 -- 172.21.15.52:0/2216930612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70841009b0 msgr2=0x7f70841972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:34.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.836+0000 7f7082ffd640 1 --2- 172.21.15.52:0/2216930612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70841009b0 0x7f70841972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.836+0000 7f7082ffd640 1 -- 172.21.15.52:0/2216930612 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708419c2c0 con 0x7f7084101370 2024-04-02T13:59:34.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.837+0000 7f7082ffd640 1 --2- 172.21.15.52:0/2216930612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7084101370 0x7f7084197810 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f7078009230 tx=0x7f707802cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:34.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.837+0000 7f7080ff9640 1 -- 172.21.15.52:0/2216930612 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7078041070 con 0x7f7084101370 2024-04-02T13:59:34.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.837+0000 7f7080ff9640 1 -- 172.21.15.52:0/2216930612 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7078004450 con 0x7f7084101370 2024-04-02T13:59:34.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.837+0000 7f7089b0e640 1 -- 172.21.15.52:0/2216930612 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f708419c550 con 0x7f7084101370 2024-04-02T13:59:34.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.837+0000 7f7080ff9640 1 -- 172.21.15.52:0/2216930612 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7078034440 con 0x7f7084101370 2024-04-02T13:59:34.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.837+0000 7f7089b0e640 1 -- 172.21.15.52:0/2216930612 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f708406a0d0 con 0x7f7084101370 2024-04-02T13:59:34.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.840+0000 7f7080ff9640 1 -- 172.21.15.52:0/2216930612 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7078044070 con 0x7f7084101370 2024-04-02T13:59:34.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.840+0000 7f708c7d5740 1 Processor -- start 2024-04-02T13:59:34.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.840+0000 7f708c7d5740 1 -- start start 2024-04-02T13:59:34.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.840+0000 7f708c7d5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d32d7208d0 0x55d32d720cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:34.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.840+0000 7f708c7d5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d32d7211f0 0x55d32d729890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:34.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.840+0000 7f708c7d5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d729dd0 0x55d32d72c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:34.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.840+0000 7f708c7d5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d32d684830 con 0x55d32d729dd0 2024-04-02T13:59:34.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.840+0000 7f708c7d5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d32d62c820 con 0x55d32d7211f0 2024-04-02T13:59:34.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.840+0000 7f708c7d5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d32d6807e0 con 0x55d32d7208d0 2024-04-02T13:59:34.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.840+0000 7f7080ff9640 1 --2- 172.21.15.52:0/2216930612 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f706c0a3d20 0x7f706c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:34.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.840+0000 7f7082ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d729dd0 0x55d32d72c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:34.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.840+0000 7f70837fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d32d7211f0 0x55d32d729890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:34.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.840+0000 7f7082ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d729dd0 0x55d32d72c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51914/0 (socket says 172.21.15.52:51914) 2024-04-02T13:59:34.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.840+0000 7f7082ffd640 1 -- 172.21.15.52:0/2049920252 learned_addr learned my addr 172.21.15.52:0/2049920252 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:34.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.841+0000 7f7083fff640 1 --2- 172.21.15.52:0/2216930612 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f706c0a3d20 0x7f706c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:34.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.841+0000 7f7083fff640 1 --2- 172.21.15.52:0/2049920252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d32d7208d0 0x55d32d720cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:34.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.841+0000 7f7080ff9640 1 -- 172.21.15.52:0/2216930612 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f70780f2530 con 0x7f7084101370 2024-04-02T13:59:34.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.841+0000 7f70837fe640 1 -- 172.21.15.52:0/2049920252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d32d7208d0 msgr2=0x55d32d720cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:34.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.841+0000 7f70837fe640 1 --2- 172.21.15.52:0/2049920252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d32d7208d0 0x55d32d720cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.841+0000 7f70837fe640 1 -- 172.21.15.52:0/2049920252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d729dd0 msgr2=0x55d32d72c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:34.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.841+0000 7f70837fe640 1 --2- 172.21.15.52:0/2049920252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d729dd0 0x55d32d72c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.841+0000 7f70837fe640 1 -- 172.21.15.52:0/2049920252 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d32d4627d0 con 0x55d32d7211f0 2024-04-02T13:59:34.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.841+0000 7f7082ffd640 1 --2- 172.21.15.52:0/2049920252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d729dd0 0x55d32d72c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:59:34.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.841+0000 7f7083fff640 1 --2- 172.21.15.52:0/2216930612 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f706c0a3d20 0x7f706c0a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f707c0077f0 tx=0x7f707c007de0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:34.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.841+0000 7f70837fe640 1 --2- 172.21.15.52:0/2049920252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d32d7211f0 0x55d32d729890 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f707400bb60 tx=0x7f707400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=879144ad0624b79c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:34.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.841+0000 7f7069ffb640 1 -- 172.21.15.52:0/2049920252 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074015070 con 0x55d32d7211f0 2024-04-02T13:59:34.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.841+0000 7f7069ffb640 1 -- 172.21.15.52:0/2049920252 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f707400be50 con 0x55d32d7211f0 2024-04-02T13:59:34.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.841+0000 7f7069ffb640 1 -- 172.21.15.52:0/2049920252 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074011440 con 0x55d32d7211f0 2024-04-02T13:59:34.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.842+0000 7f708c7d5740 1 -- 172.21.15.52:0/2049920252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d32d7211f0 msgr2=0x55d32d729890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:34.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.842+0000 7f708c7d5740 1 --2- 172.21.15.52:0/2049920252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d32d7211f0 0x55d32d729890 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f707400bb60 tx=0x7f707400f730 comp rx=0 tx=0).stop 2024-04-02T13:59:34.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.842+0000 7f708c7d5740 1 -- 172.21.15.52:0/2049920252 shutdown_connections 2024-04-02T13:59:34.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.842+0000 7f708c7d5740 1 --2- 172.21.15.52:0/2049920252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d729dd0 0x55d32d72c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.842+0000 7f708c7d5740 1 --2- 172.21.15.52:0/2049920252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d32d7211f0 0x55d32d729890 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.842+0000 7f708c7d5740 1 --2- 172.21.15.52:0/2049920252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d32d7208d0 0x55d32d720cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.842+0000 7f708c7d5740 1 -- 172.21.15.52:0/2049920252 >> 172.21.15.52:0/2049920252 conn(0x55d32d71b000 msgr2=0x55d32d71b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:34.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.842+0000 7f708c7d5740 1 -- 172.21.15.52:0/2049920252 shutdown_connections 2024-04-02T13:59:34.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.842+0000 7f708c7d5740 1 -- 172.21.15.52:0/2049920252 wait complete. 2024-04-02T13:59:34.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.842+0000 7f708c7d5740 1 -- start start 2024-04-02T13:59:34.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.843+0000 7f708c7d5740 1 Processor -- start 2024-04-02T13:59:34.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.843+0000 7f708c7d5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d32d7208d0 0x55d32d729630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:34.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.843+0000 7f708c7d5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d7211f0 0x55d32d7269c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:34.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.843+0000 7f708c7d5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d32d729dd0 0x55d32d726f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:34.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.843+0000 7f708c7d5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d32d72cbc0 con 0x55d32d7211f0 2024-04-02T13:59:34.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.843+0000 7f708c7d5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d32d63fd30 con 0x55d32d729dd0 2024-04-02T13:59:34.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.843+0000 7f708c7d5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d32d72cd40 con 0x55d32d7208d0 2024-04-02T13:59:34.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.843+0000 7f708c7d5740 10 client.0 _ll_register_callbacks cb 0x55d32d63f720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:34.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.843+0000 7f7082ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d7211f0 0x55d32d7269c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:34.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.843+0000 7f7082ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d7211f0 0x55d32d7269c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51918/0 (socket says 172.21.15.52:51918) 2024-04-02T13:59:34.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.843+0000 7f7082ffd640 1 -- 172.21.15.52:0/4145782649 learned_addr learned my addr 172.21.15.52:0/4145782649 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:34.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.843+0000 7f7083fff640 1 --2- 172.21.15.52:0/4145782649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d32d729dd0 0x55d32d726f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:34.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.843+0000 7f70837fe640 1 --2- 172.21.15.52:0/4145782649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d32d7208d0 0x55d32d729630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:34.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.843+0000 7f7082ffd640 1 -- 172.21.15.52:0/4145782649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d32d7208d0 msgr2=0x55d32d729630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:34.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.843+0000 7f7082ffd640 1 --2- 172.21.15.52:0/4145782649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d32d7208d0 0x55d32d729630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.843+0000 7f7082ffd640 1 -- 172.21.15.52:0/4145782649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d32d729dd0 msgr2=0x55d32d726f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:34.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.843+0000 7f7082ffd640 1 --2- 172.21.15.52:0/4145782649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d32d729dd0 0x55d32d726f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.843+0000 7f7082ffd640 1 -- 172.21.15.52:0/4145782649 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d32d3983f0 con 0x55d32d7211f0 2024-04-02T13:59:34.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.844+0000 7f7082ffd640 1 --2- 172.21.15.52:0/4145782649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d7211f0 0x55d32d7269c0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f707802cfd0 tx=0x7f7078000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb03f76996c49642 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:34.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.844+0000 7f7062ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:34.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.844+0000 7f7062ffd640 1 -- 172.21.15.52:0/4145782649 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707804b040 con 0x55d32d7211f0 2024-04-02T13:59:34.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.844+0000 7f7062ffd640 1 -- 172.21.15.52:0/4145782649 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f70780f21d0 con 0x55d32d7211f0 2024-04-02T13:59:34.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.844+0000 7f708c7d5740 1 -- 172.21.15.52:0/4145782649 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f707400be50 con 0x55d32d7211f0 2024-04-02T13:59:34.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.844+0000 7f7062ffd640 1 -- 172.21.15.52:0/4145782649 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f70780bdde0 con 0x55d32d7211f0 2024-04-02T13:59:34.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.844+0000 7f708c7d5740 10 client.17304 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:34.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.844+0000 7f708c7d5740 1 -- 172.21.15.52:0/4145782649 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d32d3580f0 con 0x55d32d7211f0 2024-04-02T13:59:34.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.844+0000 7f7062ffd640 1 -- 172.21.15.52:0/4145782649 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f707400be50 con 0x55d32d7211f0 2024-04-02T13:59:34.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.844+0000 7f7062ffd640 1 -- 172.21.15.52:0/4145782649 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f70780036f0 con 0x55d32d7211f0 2024-04-02T13:59:34.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.844+0000 7f708c7d5740 10 client.17304 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:34.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.844+0000 7f708c7d5740 10 client.17304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:59:34.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.844+0000 7f708c7d5740 20 client.17304 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:34.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.844+0000 7f708c7d5740 1 --2- 172.21.15.52:0/4145782649 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55d32d739bf0 0x55d32d73c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:34.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.844+0000 7f708c7d5740 4 client.17304 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:59:34.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.844+0000 7f708c7d5740 1 -- 172.21.15.52:0/4145782649 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d32d691d70 con 0x55d32d739bf0 2024-04-02T13:59:34.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.845+0000 7f70837fe640 1 --2- 172.21.15.52:0/4145782649 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55d32d739bf0 0x55d32d73c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:34.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.845+0000 7f70837fe640 1 --2- 172.21.15.52:0/4145782649 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55d32d739bf0 0x55d32d73c020 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74e4cbdf5e780d41 server_cookie=7e1360275bcfcb7c in_seq=0 out_seq=0 2024-04-02T13:59:34.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.845+0000 7f7062ffd640 10 client.17304 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:34.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.846+0000 7f7062ffd640 1 -- 172.21.15.52:0/4145782649 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d32d691d70 con 0x55d32d739bf0 2024-04-02T13:59:34.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.846+0000 7f7062ffd640 10 client.17304 handle_command_reply: tid=0 2024-04-02T13:59:34.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.846+0000 7f708c7d5740 1 client.17304 shutdown 2024-04-02T13:59:34.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.846+0000 7f708c7d5740 1 -- 172.21.15.52:0/4145782649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d7211f0 msgr2=0x55d32d7269c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:34.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.846+0000 7f708c7d5740 1 --2- 172.21.15.52:0/4145782649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d7211f0 0x55d32d7269c0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f707802cfd0 tx=0x7f7078000bc0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.846+0000 7f708c7d5740 1 -- 172.21.15.52:0/4145782649 shutdown_connections 2024-04-02T13:59:34.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.846+0000 7f708c7d5740 1 --2- 172.21.15.52:0/4145782649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d32d729dd0 0x55d32d726f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.846+0000 7f708c7d5740 1 --2- 172.21.15.52:0/4145782649 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55d32d739bf0 0x55d32d73c020 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.846+0000 7f708c7d5740 1 --2- 172.21.15.52:0/4145782649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d7211f0 0x55d32d7269c0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.846+0000 7f708c7d5740 1 --2- 172.21.15.52:0/4145782649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d32d7208d0 0x55d32d729630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.846+0000 7f7062ffd640 0 client.17304 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:34.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.846+0000 7f708c7d5740 1 -- 172.21.15.52:0/4145782649 >> 172.21.15.52:0/4145782649 conn(0x55d32d71b000 msgr2=0x55d32d71cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:34.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.846+0000 7f708c7d5740 1 -- 172.21.15.52:0/4145782649 shutdown_connections 2024-04-02T13:59:34.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.846+0000 7f708c7d5740 1 -- 172.21.15.52:0/4145782649 wait complete. 2024-04-02T13:59:34.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.846+0000 7f708c7d5740 20 client.17304 trim_cache size 0 max 16384 2024-04-02T13:59:34.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.863+0000 7f708c7d5740 1 Processor -- start 2024-04-02T13:59:34.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.864+0000 7f708c7d5740 1 -- start start 2024-04-02T13:59:34.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.864+0000 7f708c7d5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d32d7208d0 0x55d32d721fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:34.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.864+0000 7f708c7d5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d32d7211f0 0x55d32d71a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:34.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.864+0000 7f708c7d5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d729dd0 0x55d32d7290f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:34.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.864+0000 7f708c7d5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f70841aedc0 con 0x55d32d729dd0 2024-04-02T13:59:34.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.864+0000 7f708c7d5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f70841b0900 con 0x55d32d7208d0 2024-04-02T13:59:34.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.864+0000 7f708c7d5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f70841af490 con 0x55d32d7211f0 2024-04-02T13:59:34.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.864+0000 7f7083fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d729dd0 0x55d32d7290f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:34.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.864+0000 7f7083fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d729dd0 0x55d32d7290f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51932/0 (socket says 172.21.15.52:51932) 2024-04-02T13:59:34.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.864+0000 7f7083fff640 1 -- 172.21.15.52:0/484906234 learned_addr learned my addr 172.21.15.52:0/484906234 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:34.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.864+0000 7f70837fe640 1 --2- 172.21.15.52:0/484906234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d32d7208d0 0x55d32d721fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:34.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.864+0000 7f7082ffd640 1 --2- 172.21.15.52:0/484906234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d32d7211f0 0x55d32d71a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:34.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.864+0000 7f7083fff640 1 -- 172.21.15.52:0/484906234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d32d7211f0 msgr2=0x55d32d71a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:34.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.864+0000 7f7083fff640 1 --2- 172.21.15.52:0/484906234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d32d7211f0 0x55d32d71a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.864+0000 7f7083fff640 1 -- 172.21.15.52:0/484906234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d32d7208d0 msgr2=0x55d32d721fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:34.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.864+0000 7f7083fff640 1 --2- 172.21.15.52:0/484906234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d32d7208d0 0x55d32d721fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.864+0000 7f7083fff640 1 -- 172.21.15.52:0/484906234 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d32d71ae30 con 0x55d32d729dd0 2024-04-02T13:59:34.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.864+0000 7f7083fff640 1 --2- 172.21.15.52:0/484906234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d729dd0 0x55d32d7290f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f707c014190 tx=0x7f707c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945de0dced367a9d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:34.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.864+0000 7f70697fa640 1 -- 172.21.15.52:0/484906234 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c0129d0 con 0x55d32d729dd0 2024-04-02T13:59:34.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.865+0000 7f70697fa640 1 -- 172.21.15.52:0/484906234 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f707c012b70 con 0x55d32d729dd0 2024-04-02T13:59:34.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.865+0000 7f70697fa640 1 -- 172.21.15.52:0/484906234 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c020430 con 0x55d32d729dd0 2024-04-02T13:59:34.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.865+0000 7f708c7d5740 1 -- 172.21.15.52:0/484906234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d729dd0 msgr2=0x55d32d7290f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:34.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.865+0000 7f708c7d5740 1 --2- 172.21.15.52:0/484906234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d729dd0 0x55d32d7290f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f707c014190 tx=0x7f707c004ab0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.865+0000 7f708c7d5740 1 -- 172.21.15.52:0/484906234 shutdown_connections 2024-04-02T13:59:34.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.865+0000 7f708c7d5740 1 --2- 172.21.15.52:0/484906234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d729dd0 0x55d32d7290f0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.865+0000 7f708c7d5740 1 --2- 172.21.15.52:0/484906234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d32d7211f0 0x55d32d71a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.865+0000 7f708c7d5740 1 --2- 172.21.15.52:0/484906234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d32d7208d0 0x55d32d721fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.865+0000 7f708c7d5740 1 -- 172.21.15.52:0/484906234 >> 172.21.15.52:0/484906234 conn(0x55d32d71b000 msgr2=0x55d32d7199b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:34.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.865+0000 7f708c7d5740 1 -- 172.21.15.52:0/484906234 shutdown_connections 2024-04-02T13:59:34.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.865+0000 7f708c7d5740 1 -- 172.21.15.52:0/484906234 wait complete. 2024-04-02T13:59:34.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.865+0000 7f708c7d5740 1 -- start start 2024-04-02T13:59:34.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.866+0000 7f708c7d5740 1 Processor -- start 2024-04-02T13:59:34.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.866+0000 7f708c7d5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d32d7208d0 0x55d32d72dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:34.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.866+0000 7f708c7d5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d32d7211f0 0x55d32d72e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:34.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.866+0000 7f708c7d5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d729dd0 0x55d32d79f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:34.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.866+0000 7f708c7d5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f70841b0230 con 0x55d32d729dd0 2024-04-02T13:59:34.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.866+0000 7f708c7d5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f70841afb60 con 0x55d32d7211f0 2024-04-02T13:59:34.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.866+0000 7f708c7d5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f70841acda0 con 0x55d32d7208d0 2024-04-02T13:59:34.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.866+0000 7f708c7d5740 10 client.0 _ll_register_callbacks cb 0x7f7078005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:34.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.866+0000 7f7083fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d729dd0 0x55d32d79f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:34.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.866+0000 7f7083fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d729dd0 0x55d32d79f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51942/0 (socket says 172.21.15.52:51942) 2024-04-02T13:59:34.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.866+0000 7f7083fff640 1 -- 172.21.15.52:0/3526851576 learned_addr learned my addr 172.21.15.52:0/3526851576 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:34.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.866+0000 7f7082ffd640 1 --2- 172.21.15.52:0/3526851576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d32d7211f0 0x55d32d72e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:34.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.866+0000 7f70837fe640 1 --2- 172.21.15.52:0/3526851576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d32d7208d0 0x55d32d72dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:34.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.866+0000 7f7083fff640 1 -- 172.21.15.52:0/3526851576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d32d7208d0 msgr2=0x55d32d72dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:34.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.866+0000 7f7083fff640 1 --2- 172.21.15.52:0/3526851576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d32d7208d0 0x55d32d72dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.866+0000 7f7083fff640 1 -- 172.21.15.52:0/3526851576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d32d7211f0 msgr2=0x55d32d72e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:34.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.866+0000 7f7083fff640 1 --2- 172.21.15.52:0/3526851576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d32d7211f0 0x55d32d72e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.866+0000 7f7083fff640 1 -- 172.21.15.52:0/3526851576 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d32d72eb30 con 0x55d32d729dd0 2024-04-02T13:59:34.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.866+0000 7f7082ffd640 1 --2- 172.21.15.52:0/3526851576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d32d7211f0 0x55d32d72e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:59:34.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.867+0000 7f7083fff640 1 --2- 172.21.15.52:0/3526851576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d729dd0 0x55d32d79f160 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f707c021b70 tx=0x7f707c0146a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce96578d2645da27 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:34.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.867+0000 7f7062ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:34.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.867+0000 7f7062ffd640 1 -- 172.21.15.52:0/3526851576 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c038020 con 0x55d32d729dd0 2024-04-02T13:59:34.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.867+0000 7f7062ffd640 1 -- 172.21.15.52:0/3526851576 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f707c0078b0 con 0x55d32d729dd0 2024-04-02T13:59:34.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.867+0000 7f708c7d5740 1 -- 172.21.15.52:0/3526851576 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f707c012b70 con 0x55d32d729dd0 2024-04-02T13:59:34.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.867+0000 7f7062ffd640 1 -- 172.21.15.52:0/3526851576 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f707c040050 con 0x55d32d729dd0 2024-04-02T13:59:34.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.867+0000 7f708c7d5740 10 client.17310 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:34.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.867+0000 7f708c7d5740 1 -- 172.21.15.52:0/3526851576 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d32d7a07b0 con 0x55d32d729dd0 2024-04-02T13:59:34.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.867+0000 7f7062ffd640 1 -- 172.21.15.52:0/3526851576 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f707c012b70 con 0x55d32d729dd0 2024-04-02T13:59:34.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.867+0000 7f7062ffd640 1 -- 172.21.15.52:0/3526851576 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f707c034420 con 0x55d32d729dd0 2024-04-02T13:59:34.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.867+0000 7f708c7d5740 10 client.17310 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:34.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.867+0000 7f708c7d5740 10 client.17310 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:59:34.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.867+0000 7f708c7d5740 20 client.17310 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:34.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.867+0000 7f708c7d5740 1 --2- 172.21.15.52:0/3526851576 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55d32d7a0e70 0x55d32d7a3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:34.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.867+0000 7f708c7d5740 4 client.17310 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:59:34.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.867+0000 7f708c7d5740 1 -- 172.21.15.52:0/3526851576 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d32d68c0c0 con 0x55d32d7a0e70 2024-04-02T13:59:34.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.868+0000 7f70837fe640 1 --2- 172.21.15.52:0/3526851576 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55d32d7a0e70 0x55d32d7a3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:34.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.868+0000 7f70837fe640 1 --2- 172.21.15.52:0/3526851576 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55d32d7a0e70 0x55d32d7a3290 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9b1db95d0ca5917 server_cookie=c0522da72520fd09 in_seq=0 out_seq=0 2024-04-02T13:59:34.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.868+0000 7f7062ffd640 10 client.17310 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:34.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.868+0000 7f7062ffd640 1 -- 172.21.15.52:0/3526851576 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d32d68c0c0 con 0x55d32d7a0e70 2024-04-02T13:59:34.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.868+0000 7f7062ffd640 10 client.17310 handle_command_reply: tid=0 2024-04-02T13:59:34.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.868+0000 7f708c7d5740 1 client.17310 shutdown 2024-04-02T13:59:34.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.868+0000 7f708c7d5740 1 -- 172.21.15.52:0/3526851576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d729dd0 msgr2=0x55d32d79f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:34.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.868+0000 7f708c7d5740 1 --2- 172.21.15.52:0/3526851576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d729dd0 0x55d32d79f160 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f707c021b70 tx=0x7f707c0146a0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.869+0000 7f708c7d5740 1 -- 172.21.15.52:0/3526851576 shutdown_connections 2024-04-02T13:59:34.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.869+0000 7f708c7d5740 1 --2- 172.21.15.52:0/3526851576 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55d32d7a0e70 0x55d32d7a3290 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.869+0000 7f708c7d5740 1 --2- 172.21.15.52:0/3526851576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d32d729dd0 0x55d32d79f160 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.869+0000 7f708c7d5740 1 --2- 172.21.15.52:0/3526851576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d32d7211f0 0x55d32d72e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.869+0000 7f7062ffd640 0 client.17310 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:34.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.869+0000 7f708c7d5740 1 --2- 172.21.15.52:0/3526851576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d32d7208d0 0x55d32d72dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.869+0000 7f708c7d5740 1 -- 172.21.15.52:0/3526851576 >> 172.21.15.52:0/3526851576 conn(0x55d32d71b000 msgr2=0x55d32d721e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:34.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.869+0000 7f708c7d5740 1 -- 172.21.15.52:0/3526851576 shutdown_connections 2024-04-02T13:59:34.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.869+0000 7f708c7d5740 1 -- 172.21.15.52:0/3526851576 wait complete. 2024-04-02T13:59:34.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.869+0000 7f708c7d5740 20 client.17310 trim_cache size 0 max 16384 2024-04-02T13:59:34.883 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:59:34.883 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-04-02T13:59:34.883 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-04-02T13:59:34.884 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:59:34.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.869+0000 7f7089b0e640 1 -- 172.21.15.52:0/2216930612 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f706c0a3d20 msgr2=0x7f706c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:34.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.869+0000 7f7089b0e640 1 --2- 172.21.15.52:0/2216930612 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f706c0a3d20 0x7f706c0a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f707c0077f0 tx=0x7f707c007de0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.869+0000 7f7089b0e640 1 -- 172.21.15.52:0/2216930612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7084101370 msgr2=0x7f7084197810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:34.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.869+0000 7f7089b0e640 1 --2- 172.21.15.52:0/2216930612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7084101370 0x7f7084197810 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f7078009230 tx=0x7f707802cc70 comp rx=0 tx=0).stop 2024-04-02T13:59:34.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.869+0000 7f7089b0e640 1 -- 172.21.15.52:0/2216930612 shutdown_connections 2024-04-02T13:59:34.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.869+0000 7f7089b0e640 1 --2- 172.21.15.52:0/2216930612 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f706c0a3d20 0x7f706c0a61e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.869+0000 7f7089b0e640 1 --2- 172.21.15.52:0/2216930612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70841069c0 0x7f708419bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.869+0000 7f7089b0e640 1 --2- 172.21.15.52:0/2216930612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7084101370 0x7f7084197810 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.869+0000 7f7089b0e640 1 --2- 172.21.15.52:0/2216930612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70841009b0 0x7f70841972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.869+0000 7f7089b0e640 1 -- 172.21.15.52:0/2216930612 >> 172.21.15.52:0/2216930612 conn(0x7f7084076d80 msgr2=0x7f70840fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:34.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.869+0000 7f7089b0e640 1 -- 172.21.15.52:0/2216930612 shutdown_connections 2024-04-02T13:59:34.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.869+0000 7f7089b0e640 1 -- 172.21.15.52:0/2216930612 wait complete. 2024-04-02T13:59:34.887 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:59:34.887 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:59:34.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.986+0000 7f556d904640 1 -- 172.21.15.52:0/4167700021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5568107aa0 msgr2=0x7f5568107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:34.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.986+0000 7f556d904640 1 --2- 172.21.15.52:0/4167700021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5568107aa0 0x7f5568107e80 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f5558009930 tx=0x7f555802bc30 comp rx=0 tx=0).stop 2024-04-02T13:59:34.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.986+0000 7f556d904640 1 -- 172.21.15.52:0/4167700021 shutdown_connections 2024-04-02T13:59:34.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.986+0000 7f556d904640 1 --2- 172.21.15.52:0/4167700021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5568102450 0x7f556810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.986+0000 7f556d904640 1 --2- 172.21.15.52:0/4167700021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5568101a90 0x7f5568101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.986+0000 7f556d904640 1 --2- 172.21.15.52:0/4167700021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5568107aa0 0x7f5568107e80 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.986+0000 7f556d904640 1 -- 172.21.15.52:0/4167700021 >> 172.21.15.52:0/4167700021 conn(0x7f55680fd5a0 msgr2=0x7f55680ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:34.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.986+0000 7f556d904640 1 -- 172.21.15.52:0/4167700021 shutdown_connections 2024-04-02T13:59:34.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.986+0000 7f556d904640 1 -- 172.21.15.52:0/4167700021 wait complete. 2024-04-02T13:59:34.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.987+0000 7f556d904640 1 Processor -- start 2024-04-02T13:59:34.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.987+0000 7f556d904640 1 -- start start 2024-04-02T13:59:34.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.987+0000 7f556d904640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5568101a90 0x7f5568198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:34.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.987+0000 7f556d904640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5568102450 0x7f55681988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:34.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.987+0000 7f556d904640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5568107aa0 0x7f556819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:34.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.987+0000 7f556d904640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5568111eb0 con 0x7f5568102450 2024-04-02T13:59:34.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.987+0000 7f556d904640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5568111d30 con 0x7f5568101a90 2024-04-02T13:59:34.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.987+0000 7f556d904640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5568112030 con 0x7f5568107aa0 2024-04-02T13:59:34.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.987+0000 7f55667fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5568102450 0x7f55681988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:34.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.987+0000 7f55667fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5568102450 0x7f55681988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51968/0 (socket says 172.21.15.52:51968) 2024-04-02T13:59:34.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.987+0000 7f55667fc640 1 -- 172.21.15.52:0/479183649 learned_addr learned my addr 172.21.15.52:0/479183649 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:34.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.987+0000 7f5566ffd640 1 --2- 172.21.15.52:0/479183649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5568101a90 0x7f5568198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:34.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.988+0000 7f55677fe640 1 --2- 172.21.15.52:0/479183649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5568107aa0 0x7f556819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:34.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.988+0000 7f55667fc640 1 -- 172.21.15.52:0/479183649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5568107aa0 msgr2=0x7f556819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:34.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.988+0000 7f55667fc640 1 --2- 172.21.15.52:0/479183649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5568107aa0 0x7f556819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.988+0000 7f55667fc640 1 -- 172.21.15.52:0/479183649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5568101a90 msgr2=0x7f5568198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:34.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.988+0000 7f55667fc640 1 --2- 172.21.15.52:0/479183649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5568101a90 0x7f5568198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:34.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.988+0000 7f55667fc640 1 -- 172.21.15.52:0/479183649 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556819d380 con 0x7f5568102450 2024-04-02T13:59:34.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.988+0000 7f55677fe640 1 --2- 172.21.15.52:0/479183649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5568107aa0 0x7f556819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:59:34.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.988+0000 7f55667fc640 1 --2- 172.21.15.52:0/479183649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5568102450 0x7f55681988d0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f555c00c970 tx=0x7f555c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:34.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.988+0000 7f556c902640 1 -- 172.21.15.52:0/479183649 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c007bb0 con 0x7f5568102450 2024-04-02T13:59:34.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.988+0000 7f556c902640 1 -- 172.21.15.52:0/479183649 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f555c007d50 con 0x7f5568102450 2024-04-02T13:59:34.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.988+0000 7f556c902640 1 -- 172.21.15.52:0/479183649 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c00f450 con 0x7f5568102450 2024-04-02T13:59:34.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.988+0000 7f556d904640 1 -- 172.21.15.52:0/479183649 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556819d670 con 0x7f5568102450 2024-04-02T13:59:34.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.988+0000 7f556d904640 1 -- 172.21.15.52:0/479183649 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55681a4f00 con 0x7f5568102450 2024-04-02T13:59:34.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.989+0000 7f556d904640 1 -- 172.21.15.52:0/479183649 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5568068d40 con 0x7f5568102450 2024-04-02T13:59:34.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.990+0000 7f556c902640 1 -- 172.21.15.52:0/479183649 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f555c004fe0 con 0x7f5568102450 2024-04-02T13:59:34.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.991+0000 7f556c902640 1 --2- 172.21.15.52:0/479183649 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f55500a3da0 0x7f55500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:34.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.991+0000 7f5566ffd640 1 --2- 172.21.15.52:0/479183649 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f55500a3da0 0x7f55500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:34.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.991+0000 7f556c902640 1 -- 172.21.15.52:0/479183649 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f555c0d0d00 con 0x7f5568102450 2024-04-02T13:59:34.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.991+0000 7f5566ffd640 1 --2- 172.21.15.52:0/479183649 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f55500a3da0 0x7f55500a6260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5558006010 tx=0x7f5558000bc0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:34.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:34.994+0000 7f556c902640 1 -- 172.21.15.52:0/479183649 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f555c014030 con 0x7f5568102450 2024-04-02T13:59:35.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.221+0000 7f556d904640 1 -- 172.21.15.52:0/479183649 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55681996e0 con 0x7f5568102450 2024-04-02T13:59:35.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.222+0000 7f556c902640 1 -- 172.21.15.52:0/479183649 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11923 (secure 0 0 0) 0x7f55681996e0 con 0x7f5568102450 2024-04-02T13:59:35.225 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T13:59:35.226 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:57:45.240902+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24467,24457,24490,24475,24503]},"id":1}]} 2024-04-02T13:59:35.226 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-04-02T13:59:35.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.224+0000 7f556d904640 1 -- 172.21.15.52:0/479183649 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f55500a3da0 msgr2=0x7f55500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.224+0000 7f556d904640 1 --2- 172.21.15.52:0/479183649 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f55500a3da0 0x7f55500a6260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5558006010 tx=0x7f5558000bc0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.224+0000 7f556d904640 1 -- 172.21.15.52:0/479183649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5568102450 msgr2=0x7f55681988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.224+0000 7f556d904640 1 --2- 172.21.15.52:0/479183649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5568102450 0x7f55681988d0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f555c00c970 tx=0x7f555c00ce30 comp rx=0 tx=0).stop 2024-04-02T13:59:35.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.224+0000 7f556d904640 1 -- 172.21.15.52:0/479183649 shutdown_connections 2024-04-02T13:59:35.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.224+0000 7f556d904640 1 --2- 172.21.15.52:0/479183649 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f55500a3da0 0x7f55500a6260 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.224+0000 7f556d904640 1 --2- 172.21.15.52:0/479183649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5568107aa0 0x7f556819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.224+0000 7f556d904640 1 --2- 172.21.15.52:0/479183649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5568102450 0x7f55681988d0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.224+0000 7f556d904640 1 --2- 172.21.15.52:0/479183649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5568101a90 0x7f5568198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.224+0000 7f556d904640 1 -- 172.21.15.52:0/479183649 >> 172.21.15.52:0/479183649 conn(0x7f55680fd5a0 msgr2=0x7f55680fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.224+0000 7f556d904640 1 -- 172.21.15.52:0/479183649 shutdown_connections 2024-04-02T13:59:35.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.224+0000 7f556d904640 1 -- 172.21.15.52:0/479183649 wait complete. 2024-04-02T13:59:35.238 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:59:35.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.340+0000 7f2ee36a5640 1 -- 172.21.15.52:0/1016680895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2edc100990 msgr2=0x7f2edc100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.340+0000 7f2ee36a5640 1 --2- 172.21.15.52:0/1016680895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2edc100990 0x7f2edc100e10 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009930 tx=0x7f2ecc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:59:35.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.340+0000 7f2ee36a5640 1 -- 172.21.15.52:0/1016680895 shutdown_connections 2024-04-02T13:59:35.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.340+0000 7f2ee36a5640 1 --2- 172.21.15.52:0/1016680895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2edc101350 0x7f2edc10d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.340+0000 7f2ee36a5640 1 --2- 172.21.15.52:0/1016680895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2edc100990 0x7f2edc100e10 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.340+0000 7f2ee36a5640 1 --2- 172.21.15.52:0/1016680895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2edc1069a0 0x7f2edc106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.340+0000 7f2ee36a5640 1 -- 172.21.15.52:0/1016680895 >> 172.21.15.52:0/1016680895 conn(0x7f2edc077ad0 msgr2=0x7f2edc077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.340+0000 7f2ee36a5640 1 -- 172.21.15.52:0/1016680895 shutdown_connections 2024-04-02T13:59:35.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.340+0000 7f2ee36a5640 1 -- 172.21.15.52:0/1016680895 wait complete. 2024-04-02T13:59:35.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.340+0000 7f2ee36a5640 1 Processor -- start 2024-04-02T13:59:35.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.340+0000 7f2ee36a5640 1 -- start start 2024-04-02T13:59:35.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.341+0000 7f2ee36a5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2edc100990 0x7f2edc1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.341+0000 7f2ee36a5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2edc101350 0x7f2edc197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.341+0000 7f2ee36a5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2edc1069a0 0x7f2edc19bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.341+0000 7f2ee36a5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc110d80 con 0x7f2edc1069a0 2024-04-02T13:59:35.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.341+0000 7f2ee36a5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc110c00 con 0x7f2edc100990 2024-04-02T13:59:35.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.341+0000 7f2ee36a5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc110f00 con 0x7f2edc101350 2024-04-02T13:59:35.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.341+0000 7f2ee1c42640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2edc1069a0 0x7f2edc19bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.341+0000 7f2ee1c42640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2edc1069a0 0x7f2edc19bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51976/0 (socket says 172.21.15.52:51976) 2024-04-02T13:59:35.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.341+0000 7f2ee1c42640 1 -- 172.21.15.52:0/644546524 learned_addr learned my addr 172.21.15.52:0/644546524 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:35.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.341+0000 7f2ee0c40640 1 --2- 172.21.15.52:0/644546524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2edc101350 0x7f2edc197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.341+0000 7f2ee1441640 1 --2- 172.21.15.52:0/644546524 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2edc100990 0x7f2edc1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.341+0000 7f2ee1c42640 1 -- 172.21.15.52:0/644546524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2edc101350 msgr2=0x7f2edc197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.341+0000 7f2ee1c42640 1 --2- 172.21.15.52:0/644546524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2edc101350 0x7f2edc197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.341+0000 7f2ee1c42640 1 -- 172.21.15.52:0/644546524 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2edc100990 msgr2=0x7f2edc1972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.341+0000 7f2ee1c42640 1 --2- 172.21.15.52:0/644546524 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2edc100990 0x7f2edc1972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.341+0000 7f2ee1c42640 1 -- 172.21.15.52:0/644546524 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2edc19c2b0 con 0x7f2edc1069a0 2024-04-02T13:59:35.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.341+0000 7f2ee1441640 1 --2- 172.21.15.52:0/644546524 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2edc100990 0x7f2edc1972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:59:35.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.341+0000 7f2ee1c42640 1 --2- 172.21.15.52:0/644546524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2edc1069a0 0x7f2edc19bb80 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400eaa0 tx=0x7f2ed400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.341+0000 7f2ed27fc640 1 -- 172.21.15.52:0/644546524 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed400cdb0 con 0x7f2edc1069a0 2024-04-02T13:59:35.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.341+0000 7f2ed27fc640 1 -- 172.21.15.52:0/644546524 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed4005020 con 0x7f2edc1069a0 2024-04-02T13:59:35.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.341+0000 7f2ee36a5640 1 -- 172.21.15.52:0/644546524 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2edc19c5a0 con 0x7f2edc1069a0 2024-04-02T13:59:35.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.341+0000 7f2ed27fc640 1 -- 172.21.15.52:0/644546524 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed40116b0 con 0x7f2edc1069a0 2024-04-02T13:59:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.341+0000 7f2ee36a5640 1 -- 172.21.15.52:0/644546524 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2edc1a3e30 con 0x7f2edc1069a0 2024-04-02T13:59:35.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.344+0000 7f2ed27fc640 1 -- 172.21.15.52:0/644546524 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2ed4005560 con 0x7f2edc1069a0 2024-04-02T13:59:35.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.344+0000 7f2ee6369740 1 Processor -- start 2024-04-02T13:59:35.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.344+0000 7f2ee6369740 1 -- start start 2024-04-02T13:59:35.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.344+0000 7f2ee6369740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202490 0x55fb93202870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.344+0000 7f2ee6369740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb93202db0 0x55fb9320d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.344+0000 7f2ee6369740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb9320deb0 0x55fb932102a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.344+0000 7f2ee6369740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fb931667b0 con 0x55fb93202490 2024-04-02T13:59:35.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.344+0000 7f2ee6369740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55fb93105570 con 0x55fb9320deb0 2024-04-02T13:59:35.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.344+0000 7f2ee6369740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fb93162760 con 0x55fb93202db0 2024-04-02T13:59:35.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.344+0000 7f2ed27fc640 1 --2- 172.21.15.52:0/644546524 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2ec80a3d20 0x7f2ec80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.344+0000 7f2ee0c40640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202490 0x55fb93202870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.345+0000 7f2ee1c42640 1 --2- 172.21.15.52:0/644546524 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2ec80a3d20 0x7f2ec80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.345+0000 7f2ee0c40640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202490 0x55fb93202870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51978/0 (socket says 172.21.15.52:51978) 2024-04-02T13:59:35.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.345+0000 7f2ee0c40640 1 -- 172.21.15.52:0/3585343961 learned_addr learned my addr 172.21.15.52:0/3585343961 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:35.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.345+0000 7f2ed27fc640 1 -- 172.21.15.52:0/644546524 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f2ed40d0d90 con 0x7f2edc1069a0 2024-04-02T13:59:35.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.345+0000 7f2ee1441640 1 --2- 172.21.15.52:0/3585343961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb93202db0 0x55fb9320d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.345+0000 7f2ee0c40640 1 --2- 172.21.15.52:0/3585343961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb9320deb0 0x55fb932102a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.345+0000 7f2ee0c40640 1 -- 172.21.15.52:0/3585343961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb93202db0 msgr2=0x55fb9320d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.345+0000 7f2ee0c40640 1 --2- 172.21.15.52:0/3585343961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb93202db0 0x55fb9320d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.345+0000 7f2ee0c40640 1 -- 172.21.15.52:0/3585343961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb9320deb0 msgr2=0x55fb932102a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.345+0000 7f2ee0c40640 1 --2- 172.21.15.52:0/3585343961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb9320deb0 0x55fb932102a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.345+0000 7f2ee0c40640 1 -- 172.21.15.52:0/3585343961 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb92e7a3f0 con 0x55fb93202490 2024-04-02T13:59:35.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.345+0000 7f2ee1441640 1 --2- 172.21.15.52:0/3585343961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb93202db0 0x55fb9320d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:59:35.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.345+0000 7f2ee1c42640 1 --2- 172.21.15.52:0/644546524 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2ec80a3d20 0x7f2ec80a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40ce200 tx=0x7f2ed40bd040 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.345+0000 7f2ee0c40640 1 --2- 172.21.15.52:0/3585343961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202490 0x55fb93202870 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009900 tx=0x7f2ecc005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af004376b89be473 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.345+0000 7f2ec77fe640 1 -- 172.21.15.52:0/3585343961 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc02cb40 con 0x55fb93202490 2024-04-02T13:59:35.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.345+0000 7f2ec77fe640 1 -- 172.21.15.52:0/3585343961 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ecc02cce0 con 0x55fb93202490 2024-04-02T13:59:35.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.345+0000 7f2ec77fe640 1 -- 172.21.15.52:0/3585343961 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc0051b0 con 0x55fb93202490 2024-04-02T13:59:35.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.346+0000 7f2ee6369740 1 -- 172.21.15.52:0/3585343961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202490 msgr2=0x55fb93202870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.346+0000 7f2ee6369740 1 --2- 172.21.15.52:0/3585343961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202490 0x55fb93202870 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009900 tx=0x7f2ecc005d50 comp rx=0 tx=0).stop 2024-04-02T13:59:35.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.346+0000 7f2ee6369740 1 -- 172.21.15.52:0/3585343961 shutdown_connections 2024-04-02T13:59:35.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.346+0000 7f2ee6369740 1 --2- 172.21.15.52:0/3585343961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb9320deb0 0x55fb932102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.346+0000 7f2ee6369740 1 --2- 172.21.15.52:0/3585343961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb93202db0 0x55fb9320d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.346+0000 7f2ee6369740 1 --2- 172.21.15.52:0/3585343961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202490 0x55fb93202870 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.346+0000 7f2ee6369740 1 -- 172.21.15.52:0/3585343961 >> 172.21.15.52:0/3585343961 conn(0x55fb931fced0 msgr2=0x55fb931fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.346+0000 7f2ee6369740 1 -- 172.21.15.52:0/3585343961 shutdown_connections 2024-04-02T13:59:35.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.346+0000 7f2ee6369740 1 -- 172.21.15.52:0/3585343961 wait complete. 2024-04-02T13:59:35.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.346+0000 7f2ee6369740 1 -- start start 2024-04-02T13:59:35.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.346+0000 7f2ee6369740 1 Processor -- start 2024-04-02T13:59:35.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.347+0000 7f2ee6369740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb93202490 0x55fb9320d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.347+0000 7f2ee6369740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202db0 0x55fb9320acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.347+0000 7f2ee6369740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb9320deb0 0x55fb9320b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.347+0000 7f2ee6369740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fb93210c40 con 0x55fb93202db0 2024-04-02T13:59:35.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.347+0000 7f2ee6369740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55fb93121e80 con 0x55fb93202490 2024-04-02T13:59:35.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.347+0000 7f2ee6369740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fb93210dc0 con 0x55fb9320deb0 2024-04-02T13:59:35.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.347+0000 7f2ee6369740 10 client.0 _ll_register_callbacks cb 0x55fb9315daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:35.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.347+0000 7f2ee1441640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202db0 0x55fb9320acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.347+0000 7f2ee1441640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202db0 0x55fb9320acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51982/0 (socket says 172.21.15.52:51982) 2024-04-02T13:59:35.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.347+0000 7f2ee1441640 1 -- 172.21.15.52:0/3788499993 learned_addr learned my addr 172.21.15.52:0/3788499993 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:35.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.347+0000 7f2ee0c40640 1 --2- 172.21.15.52:0/3788499993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb93202490 0x55fb9320d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.347+0000 7f2ee0c40640 1 --2- 172.21.15.52:0/3788499993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb9320deb0 0x55fb9320b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.347+0000 7f2ee1441640 1 -- 172.21.15.52:0/3788499993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb9320deb0 msgr2=0x55fb9320b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.347+0000 7f2ee1441640 1 --2- 172.21.15.52:0/3788499993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb9320deb0 0x55fb9320b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.347+0000 7f2ee1441640 1 -- 172.21.15.52:0/3788499993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb93202490 msgr2=0x55fb9320d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.347+0000 7f2ee1441640 1 --2- 172.21.15.52:0/3788499993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb93202490 0x55fb9320d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.347+0000 7f2ee1441640 1 -- 172.21.15.52:0/3788499993 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb92f447d0 con 0x55fb93202db0 2024-04-02T13:59:35.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.347+0000 7f2ee0c40640 1 --2- 172.21.15.52:0/3788499993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb9320deb0 0x55fb9320b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:59:35.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.347+0000 7f2ee1441640 1 --2- 172.21.15.52:0/3788499993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202db0 0x55fb9320acb0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800bf80 tx=0x7f2ed8011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18064b9a2a84d50 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.347+0000 7f2ec4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:35.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.348+0000 7f2ec4ff9640 1 -- 172.21.15.52:0/3788499993 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8012710 con 0x55fb93202db0 2024-04-02T13:59:35.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.348+0000 7f2ec4ff9640 1 -- 172.21.15.52:0/3788499993 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f2ed80128b0 con 0x55fb93202db0 2024-04-02T13:59:35.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.348+0000 7f2ee6369740 1 -- 172.21.15.52:0/3788499993 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fb93173cf0 con 0x55fb93202db0 2024-04-02T13:59:35.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.348+0000 7f2ec4ff9640 1 -- 172.21.15.52:0/3788499993 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2ed8012ab0 con 0x55fb93202db0 2024-04-02T13:59:35.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.348+0000 7f2ee6369740 10 client.17328 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:35.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.348+0000 7f2ee6369740 1 -- 172.21.15.52:0/3788499993 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fb9321cda0 con 0x55fb93202db0 2024-04-02T13:59:35.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.348+0000 7f2ec4ff9640 1 -- 172.21.15.52:0/3788499993 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55fb93173cf0 con 0x55fb93202db0 2024-04-02T13:59:35.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.348+0000 7f2ec4ff9640 1 -- 172.21.15.52:0/3788499993 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2ed801a3d0 con 0x55fb93202db0 2024-04-02T13:59:35.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.348+0000 7f2ee6369740 10 client.17328 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:35.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.348+0000 7f2ee6369740 10 client.17328 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T13:59:35.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.348+0000 7f2ee6369740 20 client.17328 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:35.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.348+0000 7f2ee6369740 1 --2- 172.21.15.52:0/3788499993 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55fb9321e260 0x55fb93220700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.348+0000 7f2ee6369740 4 client.17328 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:59:35.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.348+0000 7f2ee6369740 1 -- 172.21.15.52:0/3788499993 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb9316e040 con 0x55fb9321e260 2024-04-02T13:59:35.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.348+0000 7f2ee0c40640 1 --2- 172.21.15.52:0/3788499993 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55fb9321e260 0x55fb93220700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.349+0000 7f2ee0c40640 1 --2- 172.21.15.52:0/3788499993 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55fb9321e260 0x55fb93220700 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1f0413830d2a4ffb server_cookie=e87cbea789a29387 in_seq=0 out_seq=0 2024-04-02T13:59:35.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.349+0000 7f2ec4ff9640 10 client.17328 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:59:35.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.350+0000 7f2ec4ff9640 1 -- 172.21.15.52:0/3788499993 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fb9316e040 con 0x55fb9321e260 2024-04-02T13:59:35.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.350+0000 7f2ec4ff9640 10 client.17328 handle_command_reply: tid=0 2024-04-02T13:59:35.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.350+0000 7f2ee6369740 1 client.17328 shutdown 2024-04-02T13:59:35.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.350+0000 7f2ee6369740 1 -- 172.21.15.52:0/3788499993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202db0 msgr2=0x55fb9320acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.350+0000 7f2ee6369740 1 --2- 172.21.15.52:0/3788499993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202db0 0x55fb9320acb0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800bf80 tx=0x7f2ed8011870 comp rx=0 tx=0).stop 2024-04-02T13:59:35.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.350+0000 7f2ee6369740 1 -- 172.21.15.52:0/3788499993 shutdown_connections 2024-04-02T13:59:35.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.350+0000 7f2ee6369740 1 --2- 172.21.15.52:0/3788499993 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55fb9321e260 0x55fb93220700 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.350+0000 7f2ee6369740 1 --2- 172.21.15.52:0/3788499993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb9320deb0 0x55fb9320b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.350+0000 7f2ee6369740 1 --2- 172.21.15.52:0/3788499993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202db0 0x55fb9320acb0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.350+0000 7f2ec4ff9640 0 client.17328 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:59:35.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.350+0000 7f2ee6369740 1 --2- 172.21.15.52:0/3788499993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb93202490 0x55fb9320d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.350+0000 7f2ee6369740 1 -- 172.21.15.52:0/3788499993 >> 172.21.15.52:0/3788499993 conn(0x55fb931fced0 msgr2=0x55fb931fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.350+0000 7f2ee6369740 1 -- 172.21.15.52:0/3788499993 shutdown_connections 2024-04-02T13:59:35.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.350+0000 7f2ee6369740 1 -- 172.21.15.52:0/3788499993 wait complete. 2024-04-02T13:59:35.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.350+0000 7f2ee6369740 20 client.17328 trim_cache size 0 max 16384 2024-04-02T13:59:35.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.363+0000 7f2ee6369740 1 Processor -- start 2024-04-02T13:59:35.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.363+0000 7f2ee6369740 1 -- start start 2024-04-02T13:59:35.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.363+0000 7f2ee6369740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202490 0x55fb93219b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.364+0000 7f2ee6369740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb93202db0 0x55fb93209c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.364+0000 7f2ee6369740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb9320deb0 0x55fb9320d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.364+0000 7f2ee6369740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc1acb80 con 0x55fb93202490 2024-04-02T13:59:35.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.364+0000 7f2ee6369740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc1ae6c0 con 0x55fb93202db0 2024-04-02T13:59:35.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.364+0000 7f2ee6369740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc1ad250 con 0x55fb9320deb0 2024-04-02T13:59:35.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.364+0000 7f2ee0c40640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202490 0x55fb93219b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.364+0000 7f2ee0c40640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202490 0x55fb93219b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51992/0 (socket says 172.21.15.52:51992) 2024-04-02T13:59:35.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.364+0000 7f2ee0c40640 1 -- 172.21.15.52:0/1673710658 learned_addr learned my addr 172.21.15.52:0/1673710658 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:35.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.364+0000 7f2ee1441640 1 --2- 172.21.15.52:0/1673710658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb93202db0 0x55fb93209c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.364+0000 7f2ee0c40640 1 -- 172.21.15.52:0/1673710658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb9320deb0 msgr2=0x55fb9320d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.364+0000 7f2ee0c40640 1 --2- 172.21.15.52:0/1673710658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb9320deb0 0x55fb9320d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.364+0000 7f2ee0c40640 1 -- 172.21.15.52:0/1673710658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb93202db0 msgr2=0x55fb93209c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.364+0000 7f2ee0c40640 1 --2- 172.21.15.52:0/1673710658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb93202db0 0x55fb93209c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.364+0000 7f2ee0c40640 1 -- 172.21.15.52:0/1673710658 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb931f87a0 con 0x55fb93202490 2024-04-02T13:59:35.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.364+0000 7f2ee0c40640 1 --2- 172.21.15.52:0/1673710658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202490 0x55fb93219b70 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0096c0 tx=0x7f2ecc004b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccdd3dc4ee930c93 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.364+0000 7f2ec6ffd640 1 -- 172.21.15.52:0/1673710658 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc02caa0 con 0x55fb93202490 2024-04-02T13:59:35.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.364+0000 7f2ec6ffd640 1 -- 172.21.15.52:0/1673710658 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ecc053bd0 con 0x55fb93202490 2024-04-02T13:59:35.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.364+0000 7f2ec6ffd640 1 -- 172.21.15.52:0/1673710658 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc053650 con 0x55fb93202490 2024-04-02T13:59:35.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.365+0000 7f2ee6369740 1 -- 172.21.15.52:0/1673710658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202490 msgr2=0x55fb93219b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.365+0000 7f2ee6369740 1 --2- 172.21.15.52:0/1673710658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202490 0x55fb93219b70 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0096c0 tx=0x7f2ecc004b90 comp rx=0 tx=0).stop 2024-04-02T13:59:35.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.365+0000 7f2ee6369740 1 -- 172.21.15.52:0/1673710658 shutdown_connections 2024-04-02T13:59:35.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.365+0000 7f2ee6369740 1 --2- 172.21.15.52:0/1673710658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb9320deb0 0x55fb9320d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.365+0000 7f2ee6369740 1 --2- 172.21.15.52:0/1673710658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb93202db0 0x55fb93209c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.365+0000 7f2ee6369740 1 --2- 172.21.15.52:0/1673710658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202490 0x55fb93219b70 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.365+0000 7f2ee6369740 1 -- 172.21.15.52:0/1673710658 >> 172.21.15.52:0/1673710658 conn(0x55fb931fced0 msgr2=0x55fb93209580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.365+0000 7f2ee6369740 1 -- 172.21.15.52:0/1673710658 shutdown_connections 2024-04-02T13:59:35.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.365+0000 7f2ee6369740 1 -- 172.21.15.52:0/1673710658 wait complete. 2024-04-02T13:59:35.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.365+0000 7f2ee6369740 1 -- start start 2024-04-02T13:59:35.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.365+0000 7f2ee6369740 1 Processor -- start 2024-04-02T13:59:35.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.366+0000 7f2ee6369740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb93202490 0x55fb9320e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.366+0000 7f2ee6369740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202db0 0x55fb9320e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.366+0000 7f2ee6369740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb9320deb0 0x55fb9320eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.366+0000 7f2ee6369740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc1adff0 con 0x55fb93202db0 2024-04-02T13:59:35.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.366+0000 7f2ee6369740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc1ad920 con 0x55fb9320deb0 2024-04-02T13:59:35.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.366+0000 7f2ee6369740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc1aab60 con 0x55fb93202490 2024-04-02T13:59:35.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.366+0000 7f2ee6369740 10 client.0 _ll_register_callbacks cb 0x7f2ed800b500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:35.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.366+0000 7f2ee1441640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202db0 0x55fb9320e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.366+0000 7f2ee1441640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202db0 0x55fb9320e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52008/0 (socket says 172.21.15.52:52008) 2024-04-02T13:59:35.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.366+0000 7f2ee1441640 1 -- 172.21.15.52:0/2670923662 learned_addr learned my addr 172.21.15.52:0/2670923662 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:35.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.366+0000 7f2ee0c40640 1 --2- 172.21.15.52:0/2670923662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb93202490 0x55fb9320e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.366+0000 7f2ee1441640 1 -- 172.21.15.52:0/2670923662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb93202490 msgr2=0x55fb9320e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.366+0000 7f2ee0c40640 1 --2- 172.21.15.52:0/2670923662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb9320deb0 0x55fb9320eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.366+0000 7f2ee1441640 1 --2- 172.21.15.52:0/2670923662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb93202490 0x55fb9320e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.366+0000 7f2ee1441640 1 -- 172.21.15.52:0/2670923662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb9320deb0 msgr2=0x55fb9320eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.366+0000 7f2ee1441640 1 --2- 172.21.15.52:0/2670923662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb9320deb0 0x55fb9320eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.366+0000 7f2ee1441640 1 -- 172.21.15.52:0/2670923662 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb93215320 con 0x55fb93202db0 2024-04-02T13:59:35.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.366+0000 7f2ee1441640 1 --2- 172.21.15.52:0/2670923662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202db0 0x55fb9320e9b0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x55fb9320c280 tx=0x7f2ed800b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761d949f479c5602 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.366+0000 7f2ec4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:35.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.366+0000 7f2ec4ff9640 1 -- 172.21.15.52:0/2670923662 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8012710 con 0x55fb93202db0 2024-04-02T13:59:35.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.366+0000 7f2ee6369740 1 -- 172.21.15.52:0/2670923662 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ecc053bd0 con 0x55fb93202db0 2024-04-02T13:59:35.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.366+0000 7f2ec4ff9640 1 -- 172.21.15.52:0/2670923662 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f2ed80128b0 con 0x55fb93202db0 2024-04-02T13:59:35.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.366+0000 7f2ec4ff9640 1 -- 172.21.15.52:0/2670923662 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2ed801a610 con 0x55fb93202db0 2024-04-02T13:59:35.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.366+0000 7f2ee6369740 10 client.17334 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:35.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.366+0000 7f2ee6369740 1 -- 172.21.15.52:0/2670923662 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fb93216680 con 0x55fb93202db0 2024-04-02T13:59:35.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.367+0000 7f2ec4ff9640 1 -- 172.21.15.52:0/2670923662 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ecc053bd0 con 0x55fb93202db0 2024-04-02T13:59:35.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.367+0000 7f2ec4ff9640 1 -- 172.21.15.52:0/2670923662 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2ed8004680 con 0x55fb93202db0 2024-04-02T13:59:35.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.367+0000 7f2ee6369740 10 client.17334 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:35.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.367+0000 7f2ee6369740 10 client.17334 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T13:59:35.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.367+0000 7f2ee6369740 20 client.17334 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:35.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.367+0000 7f2ee6369740 1 --2- 172.21.15.52:0/2670923662 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55fb93284b80 0x55fb93286fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.367+0000 7f2ee6369740 4 client.17334 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:59:35.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.367+0000 7f2ee6369740 1 -- 172.21.15.52:0/2670923662 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fb930eb6b0 con 0x55fb93284b80 2024-04-02T13:59:35.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.367+0000 7f2ee0c40640 1 --2- 172.21.15.52:0/2670923662 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55fb93284b80 0x55fb93286fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.368+0000 7f2ec4ff9640 10 client.17334 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T13:59:35.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.367+0000 7f2ee0c40640 1 --2- 172.21.15.52:0/2670923662 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55fb93284b80 0x55fb93286fa0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=933eff80ccf740df server_cookie=eaa0112f3b13c763 in_seq=0 out_seq=0 2024-04-02T13:59:35.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.368+0000 7f2ec4ff9640 1 -- 172.21.15.52:0/2670923662 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fb930eb6b0 con 0x55fb93284b80 2024-04-02T13:59:35.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.368+0000 7f2ec4ff9640 10 client.17334 handle_command_reply: tid=0 2024-04-02T13:59:35.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.368+0000 7f2ee6369740 1 client.17334 shutdown 2024-04-02T13:59:35.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.368+0000 7f2ee6369740 1 -- 172.21.15.52:0/2670923662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202db0 msgr2=0x55fb9320e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.368+0000 7f2ee6369740 1 --2- 172.21.15.52:0/2670923662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202db0 0x55fb9320e9b0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x55fb9320c280 tx=0x7f2ed800b7e0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.368+0000 7f2ee6369740 1 -- 172.21.15.52:0/2670923662 shutdown_connections 2024-04-02T13:59:35.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.368+0000 7f2ee6369740 1 --2- 172.21.15.52:0/2670923662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb9320deb0 0x55fb9320eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.368+0000 7f2ee6369740 1 --2- 172.21.15.52:0/2670923662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb93202db0 0x55fb9320e9b0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.368+0000 7f2ee6369740 1 --2- 172.21.15.52:0/2670923662 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55fb93284b80 0x55fb93286fa0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.368+0000 7f2ee6369740 1 --2- 172.21.15.52:0/2670923662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb93202490 0x55fb9320e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.368+0000 7f2ec4ff9640 0 client.17334 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T13:59:35.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.368+0000 7f2ee6369740 1 -- 172.21.15.52:0/2670923662 >> 172.21.15.52:0/2670923662 conn(0x55fb931fced0 msgr2=0x55fb931fb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.368+0000 7f2ee6369740 1 -- 172.21.15.52:0/2670923662 shutdown_connections 2024-04-02T13:59:35.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.368+0000 7f2ee6369740 1 -- 172.21.15.52:0/2670923662 wait complete. 2024-04-02T13:59:35.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.368+0000 7f2ee6369740 20 client.17334 trim_cache size 0 max 16384 2024-04-02T13:59:35.382 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:59:35.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.368+0000 7f2ee36a5640 1 -- 172.21.15.52:0/644546524 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2ec80a3d20 msgr2=0x7f2ec80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.368+0000 7f2ee36a5640 1 --2- 172.21.15.52:0/644546524 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2ec80a3d20 0x7f2ec80a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40ce200 tx=0x7f2ed40bd040 comp rx=0 tx=0).stop 2024-04-02T13:59:35.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.369+0000 7f2ee36a5640 1 -- 172.21.15.52:0/644546524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2edc1069a0 msgr2=0x7f2edc19bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.369+0000 7f2ee36a5640 1 --2- 172.21.15.52:0/644546524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2edc1069a0 0x7f2edc19bb80 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400eaa0 tx=0x7f2ed400ef60 comp rx=0 tx=0).stop 2024-04-02T13:59:35.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.369+0000 7f2ee36a5640 1 -- 172.21.15.52:0/644546524 shutdown_connections 2024-04-02T13:59:35.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.369+0000 7f2ee36a5640 1 --2- 172.21.15.52:0/644546524 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2ec80a3d20 0x7f2ec80a61e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.369+0000 7f2ee36a5640 1 --2- 172.21.15.52:0/644546524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2edc1069a0 0x7f2edc19bb80 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.369+0000 7f2ee36a5640 1 --2- 172.21.15.52:0/644546524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2edc101350 0x7f2edc197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.369+0000 7f2ee36a5640 1 --2- 172.21.15.52:0/644546524 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2edc100990 0x7f2edc1972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.369+0000 7f2ee36a5640 1 -- 172.21.15.52:0/644546524 >> 172.21.15.52:0/644546524 conn(0x7f2edc077ad0 msgr2=0x7f2edc103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.369+0000 7f2ee36a5640 1 -- 172.21.15.52:0/644546524 shutdown_connections 2024-04-02T13:59:35.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.369+0000 7f2ee36a5640 1 -- 172.21.15.52:0/644546524 wait complete. 2024-04-02T13:59:35.385 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:59:35.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.488+0000 7f9fc6900640 1 -- 172.21.15.52:0/1033424764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc0107aa0 msgr2=0x7f9fc0107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.488+0000 7f9fc6900640 1 --2- 172.21.15.52:0/1033424764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc0107aa0 0x7f9fc0107e80 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009930 tx=0x7f9fb002bc30 comp rx=0 tx=0).stop 2024-04-02T13:59:35.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.488+0000 7f9fc6900640 1 -- 172.21.15.52:0/1033424764 shutdown_connections 2024-04-02T13:59:35.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.488+0000 7f9fc6900640 1 --2- 172.21.15.52:0/1033424764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fc0102450 0x7f9fc010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.488+0000 7f9fc6900640 1 --2- 172.21.15.52:0/1033424764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fc0101a90 0x7f9fc0101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.488+0000 7f9fc6900640 1 --2- 172.21.15.52:0/1033424764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc0107aa0 0x7f9fc0107e80 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.488+0000 7f9fc6900640 1 -- 172.21.15.52:0/1033424764 >> 172.21.15.52:0/1033424764 conn(0x7f9fc0078f00 msgr2=0x7f9fc0079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.488+0000 7f9fc6900640 1 -- 172.21.15.52:0/1033424764 shutdown_connections 2024-04-02T13:59:35.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.488+0000 7f9fc6900640 1 -- 172.21.15.52:0/1033424764 wait complete. 2024-04-02T13:59:35.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.489+0000 7f9fc6900640 1 Processor -- start 2024-04-02T13:59:35.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.489+0000 7f9fc6900640 1 -- start start 2024-04-02T13:59:35.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.489+0000 7f9fc6900640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fc0101a90 0x7f9fc01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.489+0000 7f9fc6900640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc0102450 0x7f9fc0198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.489+0000 7f9fc6900640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fc0107aa0 0x7f9fc019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.489+0000 7f9fc6900640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc0111e80 con 0x7f9fc0102450 2024-04-02T13:59:35.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.489+0000 7f9fc6900640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc0111d00 con 0x7f9fc0101a90 2024-04-02T13:59:35.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.489+0000 7f9fc6900640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc0112000 con 0x7f9fc0107aa0 2024-04-02T13:59:35.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.489+0000 7f9fbf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc0102450 0x7f9fc0198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.489+0000 7f9fbf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc0102450 0x7f9fc0198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52030/0 (socket says 172.21.15.52:52030) 2024-04-02T13:59:35.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.489+0000 7f9fbf7fe640 1 -- 172.21.15.52:0/2709975272 learned_addr learned my addr 172.21.15.52:0/2709975272 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:35.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.489+0000 7f9fbffff640 1 --2- 172.21.15.52:0/2709975272 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fc0101a90 0x7f9fc01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.489+0000 7f9fc4e9d640 1 --2- 172.21.15.52:0/2709975272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fc0107aa0 0x7f9fc019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.489+0000 7f9fbf7fe640 1 -- 172.21.15.52:0/2709975272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fc0107aa0 msgr2=0x7f9fc019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.489+0000 7f9fbf7fe640 1 --2- 172.21.15.52:0/2709975272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fc0107aa0 0x7f9fc019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.489+0000 7f9fbf7fe640 1 -- 172.21.15.52:0/2709975272 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fc0101a90 msgr2=0x7f9fc01983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.489+0000 7f9fbf7fe640 1 --2- 172.21.15.52:0/2709975272 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fc0101a90 0x7f9fc01983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.490+0000 7f9fbf7fe640 1 -- 172.21.15.52:0/2709975272 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc019d360 con 0x7f9fc0102450 2024-04-02T13:59:35.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.490+0000 7f9fbf7fe640 1 --2- 172.21.15.52:0/2709975272 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc0102450 0x7f9fc0198900 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400c920 tx=0x7f9fb400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.490+0000 7f9fbd7fa640 1 -- 172.21.15.52:0/2709975272 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4007c70 con 0x7f9fc0102450 2024-04-02T13:59:35.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.490+0000 7f9fbd7fa640 1 -- 172.21.15.52:0/2709975272 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb4007e10 con 0x7f9fc0102450 2024-04-02T13:59:35.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.490+0000 7f9fc6900640 1 -- 172.21.15.52:0/2709975272 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc019d650 con 0x7f9fc0102450 2024-04-02T13:59:35.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.490+0000 7f9fbd7fa640 1 -- 172.21.15.52:0/2709975272 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4018890 con 0x7f9fc0102450 2024-04-02T13:59:35.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.490+0000 7f9fc6900640 1 -- 172.21.15.52:0/2709975272 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc01a4ee0 con 0x7f9fc0102450 2024-04-02T13:59:35.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.492+0000 7f9fbd7fa640 1 -- 172.21.15.52:0/2709975272 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9fb4018a70 con 0x7f9fc0102450 2024-04-02T13:59:35.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.492+0000 7f9fc95bf740 1 Processor -- start 2024-04-02T13:59:35.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.492+0000 7f9fc95bf740 1 -- start start 2024-04-02T13:59:35.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.492+0000 7f9fc95bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e460a897a0 0x55e460a89b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.492+0000 7f9fc95bf740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e460a8a0c0 0x55e460a92760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.493+0000 7f9fc95bf740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a92ca0 0x55e460a95090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.493+0000 7f9fc95bf740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e4609ed7b0 con 0x55e460a92ca0 2024-04-02T13:59:35.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.493+0000 7f9fc95bf740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e46098c570 con 0x55e460a8a0c0 2024-04-02T13:59:35.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.493+0000 7f9fc95bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e4609e9760 con 0x55e460a897a0 2024-04-02T13:59:35.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.493+0000 7f9fbf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a92ca0 0x55e460a95090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.493+0000 7f9fbf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a92ca0 0x55e460a95090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52036/0 (socket says 172.21.15.52:52036) 2024-04-02T13:59:35.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.493+0000 7f9fbd7fa640 1 --2- 172.21.15.52:0/2709975272 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9fa80a3d20 0x7f9fa80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.493+0000 7f9fbf7fe640 1 -- 172.21.15.52:0/2315520509 learned_addr learned my addr 172.21.15.52:0/2315520509 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:35.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.493+0000 7f9fc4e9d640 1 --2- 172.21.15.52:0/2315520509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e460a897a0 0x55e460a89b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.493+0000 7f9fbffff640 1 --2- 172.21.15.52:0/2315520509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e460a8a0c0 0x55e460a92760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.493+0000 7f9fc4e9d640 1 --2- 172.21.15.52:0/2709975272 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9fa80a3d20 0x7f9fa80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.493+0000 7f9fbd7fa640 1 -- 172.21.15.52:0/2709975272 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f9fb40d0c70 con 0x7f9fc0102450 2024-04-02T13:59:35.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.493+0000 7f9fbf7fe640 1 -- 172.21.15.52:0/2315520509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e460a897a0 msgr2=0x55e460a89b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.493+0000 7f9fbf7fe640 1 --2- 172.21.15.52:0/2315520509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e460a897a0 0x55e460a89b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.493+0000 7f9fbf7fe640 1 -- 172.21.15.52:0/2315520509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e460a8a0c0 msgr2=0x55e460a92760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.493+0000 7f9fbf7fe640 1 --2- 172.21.15.52:0/2315520509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e460a8a0c0 0x55e460a92760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.493+0000 7f9fbf7fe640 1 -- 172.21.15.52:0/2315520509 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4607013f0 con 0x55e460a92ca0 2024-04-02T13:59:35.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.494+0000 7f9fbf7fe640 1 --2- 172.21.15.52:0/2315520509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a92ca0 0x55e460a95090 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f9fb409c5d0 tx=0x7f9fb4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6109180bb3f0819a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.494+0000 7f9fc4e9d640 1 --2- 172.21.15.52:0/2709975272 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9fa80a3d20 0x7f9fa80a61e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9fb800bfd0 tx=0x7f9fb8005c20 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.494+0000 7f9fa67fc640 1 -- 172.21.15.52:0/2315520509 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb40a0c80 con 0x55e460a92ca0 2024-04-02T13:59:35.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.494+0000 7f9fa67fc640 1 -- 172.21.15.52:0/2315520509 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb40a5540 con 0x55e460a92ca0 2024-04-02T13:59:35.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.494+0000 7f9fa67fc640 1 -- 172.21.15.52:0/2315520509 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb40870d0 con 0x55e460a92ca0 2024-04-02T13:59:35.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.494+0000 7f9fc95bf740 1 -- 172.21.15.52:0/2315520509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a92ca0 msgr2=0x55e460a95090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.494+0000 7f9fc95bf740 1 --2- 172.21.15.52:0/2315520509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a92ca0 0x55e460a95090 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f9fb409c5d0 tx=0x7f9fb4000bc0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.494+0000 7f9fc95bf740 1 -- 172.21.15.52:0/2315520509 shutdown_connections 2024-04-02T13:59:35.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.494+0000 7f9fc95bf740 1 --2- 172.21.15.52:0/2315520509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a92ca0 0x55e460a95090 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.494+0000 7f9fc95bf740 1 --2- 172.21.15.52:0/2315520509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e460a8a0c0 0x55e460a92760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.494+0000 7f9fc95bf740 1 --2- 172.21.15.52:0/2315520509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e460a897a0 0x55e460a89b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.494+0000 7f9fc95bf740 1 -- 172.21.15.52:0/2315520509 >> 172.21.15.52:0/2315520509 conn(0x55e460a83ed0 msgr2=0x55e460a842b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.494+0000 7f9fc95bf740 1 -- 172.21.15.52:0/2315520509 shutdown_connections 2024-04-02T13:59:35.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.494+0000 7f9fc95bf740 1 -- 172.21.15.52:0/2315520509 wait complete. 2024-04-02T13:59:35.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.495+0000 7f9fc95bf740 1 -- start start 2024-04-02T13:59:35.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.495+0000 7f9fc95bf740 1 Processor -- start 2024-04-02T13:59:35.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.495+0000 7f9fc95bf740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a897a0 0x55e460a926c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.495+0000 7f9fc95bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e460a8a0c0 0x55e460a8fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.495+0000 7f9fc95bf740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e460a92ca0 0x55e460a90060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.495+0000 7f9fc95bf740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e460a95a30 con 0x55e460a897a0 2024-04-02T13:59:35.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.495+0000 7f9fc95bf740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e4609a8e80 con 0x55e460a92ca0 2024-04-02T13:59:35.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.495+0000 7f9fc95bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e460a95bb0 con 0x55e460a8a0c0 2024-04-02T13:59:35.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.495+0000 7f9fbffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a897a0 0x55e460a926c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.495+0000 7f9fc95bf740 10 client.0 _ll_register_callbacks cb 0x55e4609e4aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:35.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.495+0000 7f9fbffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a897a0 0x55e460a926c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52042/0 (socket says 172.21.15.52:52042) 2024-04-02T13:59:35.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.495+0000 7f9fbffff640 1 -- 172.21.15.52:0/3594195762 learned_addr learned my addr 172.21.15.52:0/3594195762 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:35.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.495+0000 7f9fbf7fe640 1 --2- 172.21.15.52:0/3594195762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e460a8a0c0 0x55e460a8fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.496+0000 7f9fc4e9d640 1 --2- 172.21.15.52:0/3594195762 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e460a92ca0 0x55e460a90060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.496+0000 7f9fbffff640 1 -- 172.21.15.52:0/3594195762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e460a8a0c0 msgr2=0x55e460a8fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.496+0000 7f9fbffff640 1 --2- 172.21.15.52:0/3594195762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e460a8a0c0 0x55e460a8fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.496+0000 7f9fbffff640 1 -- 172.21.15.52:0/3594195762 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e460a92ca0 msgr2=0x55e460a90060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.496+0000 7f9fbffff640 1 --2- 172.21.15.52:0/3594195762 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e460a92ca0 0x55e460a90060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.496+0000 7f9fbffff640 1 -- 172.21.15.52:0/3594195762 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4607cb7d0 con 0x55e460a897a0 2024-04-02T13:59:35.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.496+0000 7f9fc4e9d640 1 --2- 172.21.15.52:0/3594195762 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e460a92ca0 0x55e460a90060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:59:35.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.496+0000 7f9fbffff640 1 --2- 172.21.15.52:0/3594195762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a897a0 0x55e460a926c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f9fb002cfd0 tx=0x7f9fb0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee357e2d3b0b467c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.496+0000 7f9f9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:35.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.496+0000 7f9f9f7fe640 1 -- 172.21.15.52:0/3594195762 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0004430 con 0x55e460a897a0 2024-04-02T13:59:35.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.496+0000 7f9f9f7fe640 1 -- 172.21.15.52:0/3594195762 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f9fb00046c0 con 0x55e460a897a0 2024-04-02T13:59:35.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.496+0000 7f9fc95bf740 1 -- 172.21.15.52:0/3594195762 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fb40a5540 con 0x55e460a897a0 2024-04-02T13:59:35.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.496+0000 7f9f9f7fe640 1 -- 172.21.15.52:0/3594195762 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9fb00048c0 con 0x55e460a897a0 2024-04-02T13:59:35.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.496+0000 7f9fc95bf740 10 client.17346 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:35.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.496+0000 7f9fc95bf740 1 -- 172.21.15.52:0/3594195762 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e460aa1a90 con 0x55e460a897a0 2024-04-02T13:59:35.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.496+0000 7f9f9f7fe640 1 -- 172.21.15.52:0/3594195762 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb40a5540 con 0x55e460a897a0 2024-04-02T13:59:35.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.496+0000 7f9f9f7fe640 1 -- 172.21.15.52:0/3594195762 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9fb0003850 con 0x55e460a897a0 2024-04-02T13:59:35.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.496+0000 7f9fc95bf740 10 client.17346 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:35.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.497+0000 7f9fc95bf740 10 client.17346 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T13:59:35.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.497+0000 7f9fc95bf740 20 client.17346 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:35.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.497+0000 7f9fc95bf740 1 --2- 172.21.15.52:0/3594195762 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55e460aa2f50 0x55e460aa53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.497+0000 7f9fc95bf740 4 client.17346 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:59:35.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.497+0000 7f9fc95bf740 1 -- 172.21.15.52:0/3594195762 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4609facf0 con 0x55e460aa2f50 2024-04-02T13:59:35.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.497+0000 7f9fbf7fe640 1 --2- 172.21.15.52:0/3594195762 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55e460aa2f50 0x55e460aa53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.497+0000 7f9fbf7fe640 1 --2- 172.21.15.52:0/3594195762 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55e460aa2f50 0x55e460aa53d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7b29e6780b0dcd44 server_cookie=8abf0eca5fce3fe6 in_seq=0 out_seq=0 2024-04-02T13:59:35.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.497+0000 7f9f9f7fe640 10 client.17346 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:59:35.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.498+0000 7f9f9f7fe640 1 -- 172.21.15.52:0/3594195762 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e4609facf0 con 0x55e460aa2f50 2024-04-02T13:59:35.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.498+0000 7f9f9f7fe640 10 client.17346 handle_command_reply: tid=0 2024-04-02T13:59:35.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.498+0000 7f9fc95bf740 1 client.17346 shutdown 2024-04-02T13:59:35.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.498+0000 7f9fc95bf740 1 -- 172.21.15.52:0/3594195762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a897a0 msgr2=0x55e460a926c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.498+0000 7f9fc95bf740 1 --2- 172.21.15.52:0/3594195762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a897a0 0x55e460a926c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f9fb002cfd0 tx=0x7f9fb0004040 comp rx=0 tx=0).stop 2024-04-02T13:59:35.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.498+0000 7f9fc95bf740 1 -- 172.21.15.52:0/3594195762 shutdown_connections 2024-04-02T13:59:35.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.498+0000 7f9fc95bf740 1 --2- 172.21.15.52:0/3594195762 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55e460aa2f50 0x55e460aa53d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.498+0000 7f9fc95bf740 1 --2- 172.21.15.52:0/3594195762 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e460a92ca0 0x55e460a90060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.498+0000 7f9fc95bf740 1 --2- 172.21.15.52:0/3594195762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e460a8a0c0 0x55e460a8fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.498+0000 7f9f9f7fe640 0 client.17346 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:59:35.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.498+0000 7f9fc95bf740 1 --2- 172.21.15.52:0/3594195762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a897a0 0x55e460a926c0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.498+0000 7f9fc95bf740 1 -- 172.21.15.52:0/3594195762 >> 172.21.15.52:0/3594195762 conn(0x55e460a83ed0 msgr2=0x55e460a85c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.498+0000 7f9fc95bf740 1 -- 172.21.15.52:0/3594195762 shutdown_connections 2024-04-02T13:59:35.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.498+0000 7f9fc95bf740 1 -- 172.21.15.52:0/3594195762 wait complete. 2024-04-02T13:59:35.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.498+0000 7f9fc95bf740 20 client.17346 trim_cache size 0 max 16384 2024-04-02T13:59:35.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.511+0000 7f9fc95bf740 1 Processor -- start 2024-04-02T13:59:35.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.511+0000 7f9fc95bf740 1 -- start start 2024-04-02T13:59:35.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.512+0000 7f9fc95bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e460a897a0 0x55e460a9e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.512+0000 7f9fc95bf740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e460a8a0c0 0x55e460a92180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.512+0000 7f9fc95bf740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a92ca0 0x55e460a926c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.512+0000 7f9fc95bf740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01aee40 con 0x55e460a92ca0 2024-04-02T13:59:35.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.512+0000 7f9fc95bf740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01b0980 con 0x55e460a8a0c0 2024-04-02T13:59:35.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.512+0000 7f9fc95bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01af510 con 0x55e460a897a0 2024-04-02T13:59:35.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.512+0000 7f9fc4e9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a92ca0 0x55e460a926c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.512+0000 7f9fc4e9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a92ca0 0x55e460a926c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52048/0 (socket says 172.21.15.52:52048) 2024-04-02T13:59:35.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.512+0000 7f9fc4e9d640 1 -- 172.21.15.52:0/3245313802 learned_addr learned my addr 172.21.15.52:0/3245313802 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:35.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.512+0000 7f9fbffff640 1 --2- 172.21.15.52:0/3245313802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e460a897a0 0x55e460a9e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.512+0000 7f9fc4e9d640 1 -- 172.21.15.52:0/3245313802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e460a897a0 msgr2=0x55e460a9e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.512+0000 7f9fc4e9d640 1 --2- 172.21.15.52:0/3245313802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e460a897a0 0x55e460a9e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.512+0000 7f9fc4e9d640 1 -- 172.21.15.52:0/3245313802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e460a8a0c0 msgr2=0x55e460a92180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.512+0000 7f9fc4e9d640 1 --2- 172.21.15.52:0/3245313802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e460a8a0c0 0x55e460a92180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.512+0000 7f9fc4e9d640 1 -- 172.21.15.52:0/3245313802 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e460a89ef0 con 0x55e460a92ca0 2024-04-02T13:59:35.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.512+0000 7f9fbffff640 1 --2- 172.21.15.52:0/3245313802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e460a897a0 0x55e460a9e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:59:35.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.512+0000 7f9fc4e9d640 1 --2- 172.21.15.52:0/3245313802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a92ca0 0x55e460a926c0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f9fb800bc10 tx=0x7f9fb800b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861867353fe04969 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.512+0000 7f9fa5ffb640 1 -- 172.21.15.52:0/3245313802 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8004d20 con 0x55e460a92ca0 2024-04-02T13:59:35.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.512+0000 7f9fa5ffb640 1 -- 172.21.15.52:0/3245313802 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb8005020 con 0x55e460a92ca0 2024-04-02T13:59:35.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.512+0000 7f9fa5ffb640 1 -- 172.21.15.52:0/3245313802 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb801f660 con 0x55e460a92ca0 2024-04-02T13:59:35.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.513+0000 7f9fc95bf740 1 -- 172.21.15.52:0/3245313802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a92ca0 msgr2=0x55e460a926c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.513+0000 7f9fc95bf740 1 --2- 172.21.15.52:0/3245313802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a92ca0 0x55e460a926c0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f9fb800bc10 tx=0x7f9fb800b540 comp rx=0 tx=0).stop 2024-04-02T13:59:35.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.513+0000 7f9fc95bf740 1 -- 172.21.15.52:0/3245313802 shutdown_connections 2024-04-02T13:59:35.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.513+0000 7f9fc95bf740 1 --2- 172.21.15.52:0/3245313802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a92ca0 0x55e460a926c0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.513+0000 7f9fc95bf740 1 --2- 172.21.15.52:0/3245313802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e460a8a0c0 0x55e460a92180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.513+0000 7f9fc95bf740 1 --2- 172.21.15.52:0/3245313802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e460a897a0 0x55e460a9e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.513+0000 7f9fc95bf740 1 -- 172.21.15.52:0/3245313802 >> 172.21.15.52:0/3245313802 conn(0x55e460a83ed0 msgr2=0x55e460a8e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.513+0000 7f9fc95bf740 1 -- 172.21.15.52:0/3245313802 shutdown_connections 2024-04-02T13:59:35.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.513+0000 7f9fc95bf740 1 -- 172.21.15.52:0/3245313802 wait complete. 2024-04-02T13:59:35.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.513+0000 7f9fc95bf740 1 -- start start 2024-04-02T13:59:35.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.514+0000 7f9fc95bf740 1 Processor -- start 2024-04-02T13:59:35.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.514+0000 7f9fc95bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e460a897a0 0x55e460a9b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.514+0000 7f9fc95bf740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a8a0c0 0x55e460a96690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.514+0000 7f9fc95bf740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e460a92ca0 0x55e460a96ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.514+0000 7f9fc95bf740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01b02b0 con 0x55e460a8a0c0 2024-04-02T13:59:35.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.514+0000 7f9fc95bf740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01afbe0 con 0x55e460a92ca0 2024-04-02T13:59:35.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.514+0000 7f9fc95bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01ace20 con 0x55e460a897a0 2024-04-02T13:59:35.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.514+0000 7f9fc95bf740 10 client.0 _ll_register_callbacks cb 0x7f9fb0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:35.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.514+0000 7f9fbf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a8a0c0 0x55e460a96690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.514+0000 7f9fbf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a8a0c0 0x55e460a96690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52056/0 (socket says 172.21.15.52:52056) 2024-04-02T13:59:35.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.514+0000 7f9fbf7fe640 1 -- 172.21.15.52:0/1920952285 learned_addr learned my addr 172.21.15.52:0/1920952285 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:35.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.514+0000 7f9fc4e9d640 1 --2- 172.21.15.52:0/1920952285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e460a92ca0 0x55e460a96ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.514+0000 7f9fbf7fe640 1 -- 172.21.15.52:0/1920952285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e460a897a0 msgr2=0x55e460a9b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.514+0000 7f9fbf7fe640 1 --2- 172.21.15.52:0/1920952285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e460a897a0 0x55e460a9b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.514+0000 7f9fbf7fe640 1 -- 172.21.15.52:0/1920952285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e460a92ca0 msgr2=0x55e460a96ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.514+0000 7f9fbf7fe640 1 --2- 172.21.15.52:0/1920952285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e460a92ca0 0x55e460a96ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.514+0000 7f9fbf7fe640 1 -- 172.21.15.52:0/1920952285 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e460a97470 con 0x55e460a8a0c0 2024-04-02T13:59:35.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.514+0000 7f9fbf7fe640 1 --2- 172.21.15.52:0/1920952285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a8a0c0 0x55e460a96690 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4098d30 tx=0x7f9fb400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced72392e167e626 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.514+0000 7f9f9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:35.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.514+0000 7f9f9f7fe640 1 -- 172.21.15.52:0/1920952285 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb40a5540 con 0x55e460a8a0c0 2024-04-02T13:59:35.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.514+0000 7f9f9f7fe640 1 -- 172.21.15.52:0/1920952285 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f9fb400fb30 con 0x55e460a8a0c0 2024-04-02T13:59:35.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.514+0000 7f9fc95bf740 1 -- 172.21.15.52:0/1920952285 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fb8005020 con 0x55e460a8a0c0 2024-04-02T13:59:35.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.515+0000 7f9f9f7fe640 1 -- 172.21.15.52:0/1920952285 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9fb40672d0 con 0x55e460a8a0c0 2024-04-02T13:59:35.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.515+0000 7f9fc95bf740 10 client.17352 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:35.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.515+0000 7f9fc95bf740 1 -- 172.21.15.52:0/1920952285 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e460b09550 con 0x55e460a8a0c0 2024-04-02T13:59:35.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.515+0000 7f9f9f7fe640 1 -- 172.21.15.52:0/1920952285 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb8005020 con 0x55e460a8a0c0 2024-04-02T13:59:35.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.515+0000 7f9f9f7fe640 1 -- 172.21.15.52:0/1920952285 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9fb40870b0 con 0x55e460a8a0c0 2024-04-02T13:59:35.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.515+0000 7f9fc95bf740 10 client.17352 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:35.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.515+0000 7f9fc95bf740 10 client.17352 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T13:59:35.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.515+0000 7f9fc95bf740 20 client.17352 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:35.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.515+0000 7f9fc95bf740 1 --2- 172.21.15.52:0/1920952285 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55e460b09900 0x55e460b0bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.515+0000 7f9fc95bf740 4 client.17352 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:59:35.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.515+0000 7f9fc95bf740 1 -- 172.21.15.52:0/1920952285 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4609f5040 con 0x55e460b09900 2024-04-02T13:59:35.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.515+0000 7f9fbffff640 1 --2- 172.21.15.52:0/1920952285 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55e460b09900 0x55e460b0bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.516+0000 7f9f9f7fe640 10 client.17352 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T13:59:35.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.516+0000 7f9fbffff640 1 --2- 172.21.15.52:0/1920952285 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55e460b09900 0x55e460b0bcf0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b639579684681d1c server_cookie=9ff42a82ae57c27c in_seq=0 out_seq=0 2024-04-02T13:59:35.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.516+0000 7f9f9f7fe640 1 -- 172.21.15.52:0/1920952285 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4609f5040 con 0x55e460b09900 2024-04-02T13:59:35.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.516+0000 7f9f9f7fe640 10 client.17352 handle_command_reply: tid=0 2024-04-02T13:59:35.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.516+0000 7f9fc95bf740 1 client.17352 shutdown 2024-04-02T13:59:35.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.516+0000 7f9fc95bf740 1 -- 172.21.15.52:0/1920952285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a8a0c0 msgr2=0x55e460a96690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.516+0000 7f9fc95bf740 1 --2- 172.21.15.52:0/1920952285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a8a0c0 0x55e460a96690 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4098d30 tx=0x7f9fb400a430 comp rx=0 tx=0).stop 2024-04-02T13:59:35.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.516+0000 7f9fc95bf740 1 -- 172.21.15.52:0/1920952285 shutdown_connections 2024-04-02T13:59:35.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.516+0000 7f9fc95bf740 1 --2- 172.21.15.52:0/1920952285 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55e460b09900 0x55e460b0bcf0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.516+0000 7f9fc95bf740 1 --2- 172.21.15.52:0/1920952285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e460a92ca0 0x55e460a96ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.516+0000 7f9fc95bf740 1 --2- 172.21.15.52:0/1920952285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e460a8a0c0 0x55e460a96690 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.516+0000 7f9f9f7fe640 0 client.17352 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T13:59:35.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.516+0000 7f9fc95bf740 1 --2- 172.21.15.52:0/1920952285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e460a897a0 0x55e460a9b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.516+0000 7f9fc95bf740 1 -- 172.21.15.52:0/1920952285 >> 172.21.15.52:0/1920952285 conn(0x55e460a83ed0 msgr2=0x55e460a826a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.516+0000 7f9fc95bf740 1 -- 172.21.15.52:0/1920952285 shutdown_connections 2024-04-02T13:59:35.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.516+0000 7f9fc95bf740 1 -- 172.21.15.52:0/1920952285 wait complete. 2024-04-02T13:59:35.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.516+0000 7f9fc95bf740 20 client.17352 trim_cache size 0 max 16384 2024-04-02T13:59:35.536 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:59:35.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.517+0000 7f9fc6900640 1 -- 172.21.15.52:0/2709975272 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9fa80a3d20 msgr2=0x7f9fa80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.517+0000 7f9fc6900640 1 --2- 172.21.15.52:0/2709975272 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9fa80a3d20 0x7f9fa80a61e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9fb800bfd0 tx=0x7f9fb8005c20 comp rx=0 tx=0).stop 2024-04-02T13:59:35.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.517+0000 7f9fc6900640 1 -- 172.21.15.52:0/2709975272 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc0102450 msgr2=0x7f9fc0198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.517+0000 7f9fc6900640 1 --2- 172.21.15.52:0/2709975272 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc0102450 0x7f9fc0198900 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400c920 tx=0x7f9fb400cde0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.517+0000 7f9fc6900640 1 -- 172.21.15.52:0/2709975272 shutdown_connections 2024-04-02T13:59:35.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.517+0000 7f9fc6900640 1 --2- 172.21.15.52:0/2709975272 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9fa80a3d20 0x7f9fa80a61e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.517+0000 7f9fc6900640 1 --2- 172.21.15.52:0/2709975272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fc0107aa0 0x7f9fc019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.517+0000 7f9fc6900640 1 --2- 172.21.15.52:0/2709975272 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fc0102450 0x7f9fc0198900 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.517+0000 7f9fc6900640 1 --2- 172.21.15.52:0/2709975272 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fc0101a90 0x7f9fc01983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.517+0000 7f9fc6900640 1 -- 172.21.15.52:0/2709975272 >> 172.21.15.52:0/2709975272 conn(0x7f9fc0078f00 msgr2=0x7f9fc0104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.517+0000 7f9fc6900640 1 -- 172.21.15.52:0/2709975272 shutdown_connections 2024-04-02T13:59:35.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.517+0000 7f9fc6900640 1 -- 172.21.15.52:0/2709975272 wait complete. 2024-04-02T13:59:35.539 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:35 smithi052 ceph-mon[32104]: pgmap v185: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:35.539 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:35 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/479183649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:59:35.540 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:59:35.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.641+0000 7fc237700640 1 -- 172.21.15.52:0/433323972 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc23010e350 msgr2=0x7fc230110740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.641+0000 7fc237700640 1 --2- 172.21.15.52:0/433323972 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc23010e350 0x7fc230110740 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fc220009ca0 tx=0x7fc22002c180 comp rx=0 tx=0).stop 2024-04-02T13:59:35.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.641+0000 7fc237700640 1 -- 172.21.15.52:0/433323972 shutdown_connections 2024-04-02T13:59:35.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.641+0000 7fc237700640 1 --2- 172.21.15.52:0/433323972 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc23010e350 0x7fc230110740 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.641+0000 7fc237700640 1 --2- 172.21.15.52:0/433323972 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2300747d0 0x7fc23010de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.641+0000 7fc237700640 1 --2- 172.21.15.52:0/433323972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc230073eb0 0x7fc230074290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.641+0000 7fc237700640 1 -- 172.21.15.52:0/433323972 >> 172.21.15.52:0/433323972 conn(0x7fc2300fd5a0 msgr2=0x7fc2300ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.642+0000 7fc237700640 1 -- 172.21.15.52:0/433323972 shutdown_connections 2024-04-02T13:59:35.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.642+0000 7fc237700640 1 -- 172.21.15.52:0/433323972 wait complete. 2024-04-02T13:59:35.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.642+0000 7fc237700640 1 Processor -- start 2024-04-02T13:59:35.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.642+0000 7fc237700640 1 -- start start 2024-04-02T13:59:35.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.642+0000 7fc237700640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc230073eb0 0x7fc23019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.642+0000 7fc237700640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2300747d0 0x7fc23019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.642+0000 7fc237700640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc23010e350 0x7fc23019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.642+0000 7fc237700640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc230114070 con 0x7fc23010e350 2024-04-02T13:59:35.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.642+0000 7fc237700640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc230113ef0 con 0x7fc230073eb0 2024-04-02T13:59:35.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.642+0000 7fc237700640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc2301141f0 con 0x7fc2300747d0 2024-04-02T13:59:35.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.642+0000 7fc235c9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc23010e350 0x7fc23019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.642+0000 7fc235c9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc23010e350 0x7fc23019ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52074/0 (socket says 172.21.15.52:52074) 2024-04-02T13:59:35.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.642+0000 7fc235c9d640 1 -- 172.21.15.52:0/2027782115 learned_addr learned my addr 172.21.15.52:0/2027782115 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:35.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.643+0000 7fc234c9b640 1 --2- 172.21.15.52:0/2027782115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2300747d0 0x7fc23019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.643+0000 7fc23549c640 1 --2- 172.21.15.52:0/2027782115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc230073eb0 0x7fc23019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.643+0000 7fc235c9d640 1 -- 172.21.15.52:0/2027782115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2300747d0 msgr2=0x7fc23019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.643+0000 7fc235c9d640 1 --2- 172.21.15.52:0/2027782115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2300747d0 0x7fc23019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.643+0000 7fc235c9d640 1 -- 172.21.15.52:0/2027782115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc230073eb0 msgr2=0x7fc23019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.643+0000 7fc235c9d640 1 --2- 172.21.15.52:0/2027782115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc230073eb0 0x7fc23019a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.643+0000 7fc235c9d640 1 -- 172.21.15.52:0/2027782115 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23019f490 con 0x7fc23010e350 2024-04-02T13:59:35.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.643+0000 7fc234c9b640 1 --2- 172.21.15.52:0/2027782115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2300747d0 0x7fc23019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:59:35.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.643+0000 7fc235c9d640 1 --2- 172.21.15.52:0/2027782115 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc23010e350 0x7fc23019ed60 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fc22002c790 tx=0x7fc220035ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.643+0000 7fc22e7fc640 1 -- 172.21.15.52:0/2027782115 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc220039040 con 0x7fc23010e350 2024-04-02T13:59:35.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.643+0000 7fc22e7fc640 1 -- 172.21.15.52:0/2027782115 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc220044070 con 0x7fc23010e350 2024-04-02T13:59:35.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.643+0000 7fc237700640 1 -- 172.21.15.52:0/2027782115 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc23019f720 con 0x7fc23010e350 2024-04-02T13:59:35.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.643+0000 7fc22e7fc640 1 -- 172.21.15.52:0/2027782115 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2200329f0 con 0x7fc23010e350 2024-04-02T13:59:35.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.643+0000 7fc237700640 1 -- 172.21.15.52:0/2027782115 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2301a7010 con 0x7fc23010e350 2024-04-02T13:59:35.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.645+0000 7fc22e7fc640 1 -- 172.21.15.52:0/2027782115 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc220032b90 con 0x7fc23010e350 2024-04-02T13:59:35.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.646+0000 7fc23a3c3740 1 Processor -- start 2024-04-02T13:59:35.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.646+0000 7fc23a3c3740 1 -- start start 2024-04-02T13:59:35.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.646+0000 7fc23a3c3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb7c8707a0 0x55cb7c870b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.646+0000 7fc23a3c3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cb7c8710c0 0x55cb7c879760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.646+0000 7fc23a3c3740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c879ca0 0x55cb7c87c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.646+0000 7fc23a3c3740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cb7c7d47b0 con 0x55cb7c879ca0 2024-04-02T13:59:35.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.646+0000 7fc23a3c3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cb7c773570 con 0x55cb7c8710c0 2024-04-02T13:59:35.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.646+0000 7fc23a3c3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cb7c7d0760 con 0x55cb7c8707a0 2024-04-02T13:59:35.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.646+0000 7fc22e7fc640 1 --2- 172.21.15.52:0/2027782115 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc2180a3d20 0x7fc2180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.646+0000 7fc234c9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c879ca0 0x55cb7c87c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.646+0000 7fc234c9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c879ca0 0x55cb7c87c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52084/0 (socket says 172.21.15.52:52084) 2024-04-02T13:59:35.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.646+0000 7fc234c9b640 1 -- 172.21.15.52:0/383920335 learned_addr learned my addr 172.21.15.52:0/383920335 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:35.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.646+0000 7fc23549c640 1 --2- 172.21.15.52:0/383920335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cb7c8710c0 0x55cb7c879760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.646+0000 7fc235c9d640 1 --2- 172.21.15.52:0/2027782115 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc2180a3d20 0x7fc2180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.646+0000 7fc234c9b640 1 --2- 172.21.15.52:0/383920335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb7c8707a0 0x55cb7c870b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.646+0000 7fc22e7fc640 1 -- 172.21.15.52:0/2027782115 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc2200f3170 con 0x7fc23010e350 2024-04-02T13:59:35.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.646+0000 7fc23549c640 1 -- 172.21.15.52:0/383920335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb7c8707a0 msgr2=0x55cb7c870b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.646+0000 7fc23549c640 1 --2- 172.21.15.52:0/383920335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb7c8707a0 0x55cb7c870b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.646+0000 7fc23549c640 1 -- 172.21.15.52:0/383920335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c879ca0 msgr2=0x55cb7c87c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.646+0000 7fc23549c640 1 --2- 172.21.15.52:0/383920335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c879ca0 0x55cb7c87c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.646+0000 7fc23549c640 1 -- 172.21.15.52:0/383920335 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb7c4e83f0 con 0x55cb7c8710c0 2024-04-02T13:59:35.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.647+0000 7fc235c9d640 1 --2- 172.21.15.52:0/2027782115 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc2180a3d20 0x7fc2180a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc2200bb2e0 tx=0x7fc22002c180 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.647+0000 7fc23549c640 1 --2- 172.21.15.52:0/383920335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cb7c8710c0 0x55cb7c879760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc22800ba70 tx=0x7fc22800bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=178f5408276efdc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.647+0000 7fc2177fe640 1 -- 172.21.15.52:0/383920335 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc228017420 con 0x55cb7c8710c0 2024-04-02T13:59:35.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.647+0000 7fc2177fe640 1 -- 172.21.15.52:0/383920335 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc228017aa0 con 0x55cb7c8710c0 2024-04-02T13:59:35.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.647+0000 7fc2177fe640 1 -- 172.21.15.52:0/383920335 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22800fa50 con 0x55cb7c8710c0 2024-04-02T13:59:35.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.648+0000 7fc23a3c3740 1 -- 172.21.15.52:0/383920335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cb7c8710c0 msgr2=0x55cb7c879760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.648+0000 7fc23a3c3740 1 --2- 172.21.15.52:0/383920335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cb7c8710c0 0x55cb7c879760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc22800ba70 tx=0x7fc22800bf30 comp rx=0 tx=0).stop 2024-04-02T13:59:35.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.648+0000 7fc23a3c3740 1 -- 172.21.15.52:0/383920335 shutdown_connections 2024-04-02T13:59:35.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.648+0000 7fc23a3c3740 1 --2- 172.21.15.52:0/383920335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c879ca0 0x55cb7c87c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.648+0000 7fc23a3c3740 1 --2- 172.21.15.52:0/383920335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cb7c8710c0 0x55cb7c879760 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.648+0000 7fc23a3c3740 1 --2- 172.21.15.52:0/383920335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb7c8707a0 0x55cb7c870b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.648+0000 7fc23a3c3740 1 -- 172.21.15.52:0/383920335 >> 172.21.15.52:0/383920335 conn(0x55cb7c86aed0 msgr2=0x55cb7c86b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.648+0000 7fc23a3c3740 1 -- 172.21.15.52:0/383920335 shutdown_connections 2024-04-02T13:59:35.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.648+0000 7fc23a3c3740 1 -- 172.21.15.52:0/383920335 wait complete. 2024-04-02T13:59:35.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.648+0000 7fc23a3c3740 1 -- start start 2024-04-02T13:59:35.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.648+0000 7fc23a3c3740 1 Processor -- start 2024-04-02T13:59:35.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.648+0000 7fc23a3c3740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c8707a0 0x55cb7c8796f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.648+0000 7fc23a3c3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cb7c8710c0 0x55cb7c876a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.648+0000 7fc23a3c3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb7c879ca0 0x55cb7c8771c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.648+0000 7fc23a3c3740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cb7c87ca90 con 0x55cb7c8707a0 2024-04-02T13:59:35.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.648+0000 7fc23a3c3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cb7c78fe80 con 0x55cb7c8710c0 2024-04-02T13:59:35.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.648+0000 7fc23a3c3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cb7c87cc10 con 0x55cb7c879ca0 2024-04-02T13:59:35.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.648+0000 7fc23a3c3740 10 client.0 _ll_register_callbacks cb 0x55cb7c7cbaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:35.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.649+0000 7fc234c9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c8707a0 0x55cb7c8796f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.649+0000 7fc234c9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c8707a0 0x55cb7c8796f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52096/0 (socket says 172.21.15.52:52096) 2024-04-02T13:59:35.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.649+0000 7fc234c9b640 1 -- 172.21.15.52:0/1341000876 learned_addr learned my addr 172.21.15.52:0/1341000876 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:35.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.649+0000 7fc23549c640 1 --2- 172.21.15.52:0/1341000876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cb7c8710c0 0x55cb7c876a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.649+0000 7fc234c9b640 1 --2- 172.21.15.52:0/1341000876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb7c879ca0 0x55cb7c8771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.649+0000 7fc234c9b640 1 -- 172.21.15.52:0/1341000876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb7c879ca0 msgr2=0x55cb7c8771c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.649+0000 7fc234c9b640 1 --2- 172.21.15.52:0/1341000876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb7c879ca0 0x55cb7c8771c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.649+0000 7fc234c9b640 1 -- 172.21.15.52:0/1341000876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cb7c8710c0 msgr2=0x55cb7c876a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.649+0000 7fc234c9b640 1 --2- 172.21.15.52:0/1341000876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cb7c8710c0 0x55cb7c876a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.649+0000 7fc234c9b640 1 -- 172.21.15.52:0/1341000876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb7c5b27d0 con 0x55cb7c8707a0 2024-04-02T13:59:35.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.649+0000 7fc234c9b640 1 --2- 172.21.15.52:0/1341000876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c8707a0 0x55cb7c8796f0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fc22400ef70 tx=0x7fc224004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b550115c621a495 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.649+0000 7fc214ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:35.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.649+0000 7fc214ff9640 1 -- 172.21.15.52:0/1341000876 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22401b040 con 0x55cb7c8707a0 2024-04-02T13:59:35.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.649+0000 7fc214ff9640 1 -- 172.21.15.52:0/1341000876 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc224005220 con 0x55cb7c8707a0 2024-04-02T13:59:35.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.649+0000 7fc23a3c3740 1 -- 172.21.15.52:0/1341000876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc228017aa0 con 0x55cb7c8707a0 2024-04-02T13:59:35.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.649+0000 7fc214ff9640 1 -- 172.21.15.52:0/1341000876 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc224029050 con 0x55cb7c8707a0 2024-04-02T13:59:35.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.649+0000 7fc23a3c3740 10 client.17364 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:35.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.649+0000 7fc23a3c3740 1 -- 172.21.15.52:0/1341000876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cb7c888b70 con 0x55cb7c8707a0 2024-04-02T13:59:35.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.650+0000 7fc214ff9640 1 -- 172.21.15.52:0/1341000876 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc228017aa0 con 0x55cb7c8707a0 2024-04-02T13:59:35.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.650+0000 7fc214ff9640 1 -- 172.21.15.52:0/1341000876 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc22400f040 con 0x55cb7c8707a0 2024-04-02T13:59:35.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.650+0000 7fc23a3c3740 10 client.17364 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:35.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.650+0000 7fc23a3c3740 10 client.17364 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T13:59:35.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.650+0000 7fc23a3c3740 20 client.17364 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:35.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.650+0000 7fc23a3c3740 1 --2- 172.21.15.52:0/1341000876 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55cb7c88a030 0x55cb7c88c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.650+0000 7fc23a3c3740 4 client.17364 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:59:35.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.650+0000 7fc23a3c3740 1 -- 172.21.15.52:0/1341000876 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb7c7e1cf0 con 0x55cb7c88a030 2024-04-02T13:59:35.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.650+0000 7fc23549c640 1 --2- 172.21.15.52:0/1341000876 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55cb7c88a030 0x55cb7c88c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.651+0000 7fc23549c640 1 --2- 172.21.15.52:0/1341000876 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55cb7c88a030 0x55cb7c88c420 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6e35db406ac51b70 server_cookie=79a78fdd690fb38f in_seq=0 out_seq=0 2024-04-02T13:59:35.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.651+0000 7fc214ff9640 10 client.17364 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:59:35.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.651+0000 7fc214ff9640 1 -- 172.21.15.52:0/1341000876 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb7c7e1cf0 con 0x55cb7c88a030 2024-04-02T13:59:35.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.651+0000 7fc214ff9640 10 client.17364 handle_command_reply: tid=0 2024-04-02T13:59:35.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.651+0000 7fc23a3c3740 1 client.17364 shutdown 2024-04-02T13:59:35.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.652+0000 7fc23a3c3740 1 -- 172.21.15.52:0/1341000876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c8707a0 msgr2=0x55cb7c8796f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.652+0000 7fc23a3c3740 1 --2- 172.21.15.52:0/1341000876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c8707a0 0x55cb7c8796f0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fc22400ef70 tx=0x7fc224004730 comp rx=0 tx=0).stop 2024-04-02T13:59:35.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.652+0000 7fc23a3c3740 1 -- 172.21.15.52:0/1341000876 shutdown_connections 2024-04-02T13:59:35.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.652+0000 7fc23a3c3740 1 --2- 172.21.15.52:0/1341000876 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55cb7c88a030 0x55cb7c88c420 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.652+0000 7fc23a3c3740 1 --2- 172.21.15.52:0/1341000876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb7c879ca0 0x55cb7c8771c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.652+0000 7fc23a3c3740 1 --2- 172.21.15.52:0/1341000876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cb7c8710c0 0x55cb7c876a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.652+0000 7fc23a3c3740 1 --2- 172.21.15.52:0/1341000876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c8707a0 0x55cb7c8796f0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.652+0000 7fc214ff9640 0 client.17364 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:59:35.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.652+0000 7fc23a3c3740 1 -- 172.21.15.52:0/1341000876 >> 172.21.15.52:0/1341000876 conn(0x55cb7c86aed0 msgr2=0x55cb7c86cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.652+0000 7fc23a3c3740 1 -- 172.21.15.52:0/1341000876 shutdown_connections 2024-04-02T13:59:35.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.652+0000 7fc23a3c3740 1 -- 172.21.15.52:0/1341000876 wait complete. 2024-04-02T13:59:35.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.652+0000 7fc23a3c3740 20 client.17364 trim_cache size 0 max 16384 2024-04-02T13:59:35.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.665+0000 7fc23a3c3740 1 Processor -- start 2024-04-02T13:59:35.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.665+0000 7fc23a3c3740 1 -- start start 2024-04-02T13:59:35.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.665+0000 7fc23a3c3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb7c8707a0 0x55cb7c885980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.665+0000 7fc23a3c3740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c8710c0 0x55cb7c875a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.665+0000 7fc23a3c3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cb7c879ca0 0x55cb7c891cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.665+0000 7fc23a3c3740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc2301b1020 con 0x55cb7c8710c0 2024-04-02T13:59:35.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.665+0000 7fc23a3c3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc2301b2b60 con 0x55cb7c879ca0 2024-04-02T13:59:35.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.665+0000 7fc23a3c3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc2301b16f0 con 0x55cb7c8707a0 2024-04-02T13:59:35.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.666+0000 7fc23549c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c8710c0 0x55cb7c875a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.666+0000 7fc23549c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c8710c0 0x55cb7c875a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52104/0 (socket says 172.21.15.52:52104) 2024-04-02T13:59:35.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.666+0000 7fc23549c640 1 -- 172.21.15.52:0/2567907675 learned_addr learned my addr 172.21.15.52:0/2567907675 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:35.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.666+0000 7fc234c9b640 1 --2- 172.21.15.52:0/2567907675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb7c8707a0 0x55cb7c885980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.666+0000 7fc23549c640 1 -- 172.21.15.52:0/2567907675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb7c8707a0 msgr2=0x55cb7c885980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.666+0000 7fc23549c640 1 --2- 172.21.15.52:0/2567907675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb7c8707a0 0x55cb7c885980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.666+0000 7fc23549c640 1 -- 172.21.15.52:0/2567907675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cb7c879ca0 msgr2=0x55cb7c891cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.666+0000 7fc23549c640 1 --2- 172.21.15.52:0/2567907675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cb7c879ca0 0x55cb7c891cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.666+0000 7fc23549c640 1 -- 172.21.15.52:0/2567907675 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb7c8865f0 con 0x55cb7c8710c0 2024-04-02T13:59:35.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.666+0000 7fc23549c640 1 --2- 172.21.15.52:0/2567907675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c8710c0 0x55cb7c875a60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fc22800ba70 tx=0x7fc22800bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08f49aefc22edf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.666+0000 7fc216ffd640 1 -- 172.21.15.52:0/2567907675 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22802f070 con 0x55cb7c8710c0 2024-04-02T13:59:35.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.666+0000 7fc216ffd640 1 -- 172.21.15.52:0/2567907675 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc228005520 con 0x55cb7c8710c0 2024-04-02T13:59:35.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.666+0000 7fc216ffd640 1 -- 172.21.15.52:0/2567907675 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc228029670 con 0x55cb7c8710c0 2024-04-02T13:59:35.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.667+0000 7fc23a3c3740 1 -- 172.21.15.52:0/2567907675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c8710c0 msgr2=0x55cb7c875a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.667+0000 7fc23a3c3740 1 --2- 172.21.15.52:0/2567907675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c8710c0 0x55cb7c875a60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fc22800ba70 tx=0x7fc22800bf30 comp rx=0 tx=0).stop 2024-04-02T13:59:35.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.667+0000 7fc23a3c3740 1 -- 172.21.15.52:0/2567907675 shutdown_connections 2024-04-02T13:59:35.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.667+0000 7fc23a3c3740 1 --2- 172.21.15.52:0/2567907675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cb7c879ca0 0x55cb7c891cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.667+0000 7fc23a3c3740 1 --2- 172.21.15.52:0/2567907675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c8710c0 0x55cb7c875a60 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.667+0000 7fc23a3c3740 1 --2- 172.21.15.52:0/2567907675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb7c8707a0 0x55cb7c885980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.667+0000 7fc23a3c3740 1 -- 172.21.15.52:0/2567907675 >> 172.21.15.52:0/2567907675 conn(0x55cb7c86aed0 msgr2=0x55cb7c875350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.667+0000 7fc23a3c3740 1 -- 172.21.15.52:0/2567907675 shutdown_connections 2024-04-02T13:59:35.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.667+0000 7fc23a3c3740 1 -- 172.21.15.52:0/2567907675 wait complete. 2024-04-02T13:59:35.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.667+0000 7fc23a3c3740 1 -- start start 2024-04-02T13:59:35.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.667+0000 7fc23a3c3740 1 Processor -- start 2024-04-02T13:59:35.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.668+0000 7fc23a3c3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb7c8707a0 0x55cb7c87a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.668+0000 7fc23a3c3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cb7c8710c0 0x55cb7c87ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.668+0000 7fc23a3c3740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c879ca0 0x55cb7c880e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.668+0000 7fc23a3c3740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc2301b2490 con 0x55cb7c879ca0 2024-04-02T13:59:35.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.668+0000 7fc23a3c3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc2301b1dc0 con 0x55cb7c8710c0 2024-04-02T13:59:35.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.668+0000 7fc23a3c3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc2301af000 con 0x55cb7c8707a0 2024-04-02T13:59:35.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.668+0000 7fc23a3c3740 10 client.0 _ll_register_callbacks cb 0x7fc224005e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:35.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.668+0000 7fc234c9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c879ca0 0x55cb7c880e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.668+0000 7fc234c9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c879ca0 0x55cb7c880e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52110/0 (socket says 172.21.15.52:52110) 2024-04-02T13:59:35.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.668+0000 7fc234c9b640 1 -- 172.21.15.52:0/3391779589 learned_addr learned my addr 172.21.15.52:0/3391779589 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:35.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.668+0000 7fc234c9b640 1 --2- 172.21.15.52:0/3391779589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb7c8707a0 0x55cb7c87a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.668+0000 7fc23549c640 1 --2- 172.21.15.52:0/3391779589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cb7c8710c0 0x55cb7c87ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.668+0000 7fc234c9b640 1 -- 172.21.15.52:0/3391779589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb7c8707a0 msgr2=0x55cb7c87a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.668+0000 7fc234c9b640 1 --2- 172.21.15.52:0/3391779589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb7c8707a0 0x55cb7c87a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.668+0000 7fc234c9b640 1 -- 172.21.15.52:0/3391779589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cb7c8710c0 msgr2=0x55cb7c87ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.668+0000 7fc234c9b640 1 --2- 172.21.15.52:0/3391779589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cb7c8710c0 0x55cb7c87ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.668+0000 7fc234c9b640 1 -- 172.21.15.52:0/3391779589 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb7c8813d0 con 0x55cb7c879ca0 2024-04-02T13:59:35.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.668+0000 7fc234c9b640 1 --2- 172.21.15.52:0/3391779589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c879ca0 0x55cb7c880e90 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fc22400ead0 tx=0x7fc2240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecfdd131636e920a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.668+0000 7fc214ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:35.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.668+0000 7fc214ff9640 1 -- 172.21.15.52:0/3391779589 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc224005bb0 con 0x55cb7c879ca0 2024-04-02T13:59:35.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.668+0000 7fc214ff9640 1 -- 172.21.15.52:0/3391779589 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc224012760 con 0x55cb7c879ca0 2024-04-02T13:59:35.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.668+0000 7fc23a3c3740 1 -- 172.21.15.52:0/3391779589 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc228005520 con 0x55cb7c879ca0 2024-04-02T13:59:35.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.668+0000 7fc214ff9640 1 -- 172.21.15.52:0/3391779589 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc224020b60 con 0x55cb7c879ca0 2024-04-02T13:59:35.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.668+0000 7fc23a3c3740 10 client.17370 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:35.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.668+0000 7fc23a3c3740 1 -- 172.21.15.52:0/3391779589 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cb7c8826d0 con 0x55cb7c879ca0 2024-04-02T13:59:35.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.669+0000 7fc214ff9640 1 -- 172.21.15.52:0/3391779589 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc228005520 con 0x55cb7c879ca0 2024-04-02T13:59:35.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.669+0000 7fc214ff9640 1 -- 172.21.15.52:0/3391779589 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc2240047a0 con 0x55cb7c879ca0 2024-04-02T13:59:35.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.669+0000 7fc23a3c3740 10 client.17370 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:35.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.669+0000 7fc23a3c3740 10 client.17370 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T13:59:35.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.669+0000 7fc23a3c3740 20 client.17370 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:35.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.669+0000 7fc23a3c3740 1 --2- 172.21.15.52:0/3391779589 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55cb7c8f0920 0x55cb7c8f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.669+0000 7fc23a3c3740 4 client.17370 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:59:35.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.669+0000 7fc23a3c3740 1 -- 172.21.15.52:0/3391779589 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cb7c7dc040 con 0x55cb7c8f0920 2024-04-02T13:59:35.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.669+0000 7fc23549c640 1 --2- 172.21.15.52:0/3391779589 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55cb7c8f0920 0x55cb7c8f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.670+0000 7fc214ff9640 10 client.17370 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T13:59:35.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.670+0000 7fc23549c640 1 --2- 172.21.15.52:0/3391779589 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55cb7c8f0920 0x55cb7c8f2d40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cfca5b4b1d392a4f server_cookie=68bfb29c5fd16a6e in_seq=0 out_seq=0 2024-04-02T13:59:35.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.670+0000 7fc214ff9640 1 -- 172.21.15.52:0/3391779589 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cb7c7dc040 con 0x55cb7c8f0920 2024-04-02T13:59:35.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.670+0000 7fc214ff9640 10 client.17370 handle_command_reply: tid=0 2024-04-02T13:59:35.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.670+0000 7fc23a3c3740 1 client.17370 shutdown 2024-04-02T13:59:35.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.670+0000 7fc23a3c3740 1 -- 172.21.15.52:0/3391779589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c879ca0 msgr2=0x55cb7c880e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.670+0000 7fc23a3c3740 1 --2- 172.21.15.52:0/3391779589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c879ca0 0x55cb7c880e90 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fc22400ead0 tx=0x7fc2240046c0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.670+0000 7fc23a3c3740 1 -- 172.21.15.52:0/3391779589 shutdown_connections 2024-04-02T13:59:35.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.670+0000 7fc23a3c3740 1 --2- 172.21.15.52:0/3391779589 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55cb7c8f0920 0x55cb7c8f2d40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.670+0000 7fc23a3c3740 1 --2- 172.21.15.52:0/3391779589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb7c879ca0 0x55cb7c880e90 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.670+0000 7fc23a3c3740 1 --2- 172.21.15.52:0/3391779589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cb7c8710c0 0x55cb7c87ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.670+0000 7fc214ff9640 0 client.17370 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T13:59:35.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.670+0000 7fc23a3c3740 1 --2- 172.21.15.52:0/3391779589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb7c8707a0 0x55cb7c87a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.670+0000 7fc23a3c3740 1 -- 172.21.15.52:0/3391779589 >> 172.21.15.52:0/3391779589 conn(0x55cb7c86aed0 msgr2=0x55cb7c875be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.670+0000 7fc23a3c3740 1 -- 172.21.15.52:0/3391779589 shutdown_connections 2024-04-02T13:59:35.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.670+0000 7fc23a3c3740 1 -- 172.21.15.52:0/3391779589 wait complete. 2024-04-02T13:59:35.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.670+0000 7fc23a3c3740 20 client.17370 trim_cache size 0 max 16384 2024-04-02T13:59:35.685 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:59:35.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.671+0000 7fc237700640 1 -- 172.21.15.52:0/2027782115 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc2180a3d20 msgr2=0x7fc2180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.671+0000 7fc237700640 1 --2- 172.21.15.52:0/2027782115 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc2180a3d20 0x7fc2180a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc2200bb2e0 tx=0x7fc22002c180 comp rx=0 tx=0).stop 2024-04-02T13:59:35.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.671+0000 7fc237700640 1 -- 172.21.15.52:0/2027782115 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc23010e350 msgr2=0x7fc23019ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.671+0000 7fc237700640 1 --2- 172.21.15.52:0/2027782115 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc23010e350 0x7fc23019ed60 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fc22002c790 tx=0x7fc220035ed0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.671+0000 7fc237700640 1 -- 172.21.15.52:0/2027782115 shutdown_connections 2024-04-02T13:59:35.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.671+0000 7fc237700640 1 --2- 172.21.15.52:0/2027782115 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc2180a3d20 0x7fc2180a61e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.671+0000 7fc237700640 1 --2- 172.21.15.52:0/2027782115 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc23010e350 0x7fc23019ed60 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.671+0000 7fc237700640 1 --2- 172.21.15.52:0/2027782115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2300747d0 0x7fc23019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.671+0000 7fc237700640 1 --2- 172.21.15.52:0/2027782115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc230073eb0 0x7fc23019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.671+0000 7fc237700640 1 -- 172.21.15.52:0/2027782115 >> 172.21.15.52:0/2027782115 conn(0x7fc2300fd5a0 msgr2=0x7fc2300fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.671+0000 7fc237700640 1 -- 172.21.15.52:0/2027782115 shutdown_connections 2024-04-02T13:59:35.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.671+0000 7fc237700640 1 -- 172.21.15.52:0/2027782115 wait complete. 2024-04-02T13:59:35.688 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:59:35.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:35 smithi189 ceph-mon[36683]: pgmap v185: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:35.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:35 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/479183649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:59:35.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.789+0000 7f3beb277640 1 -- 172.21.15.52:0/53973418 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be4107ab0 msgr2=0x7f3be4107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.789+0000 7f3beb277640 1 --2- 172.21.15.52:0/53973418 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be4107ab0 0x7f3be4107e90 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009930 tx=0x7f3be002bc30 comp rx=0 tx=0).stop 2024-04-02T13:59:35.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.789+0000 7f3beb277640 1 -- 172.21.15.52:0/53973418 shutdown_connections 2024-04-02T13:59:35.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.789+0000 7f3beb277640 1 --2- 172.21.15.52:0/53973418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3be4102460 0x7f3be410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.789+0000 7f3beb277640 1 --2- 172.21.15.52:0/53973418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3be4101aa0 0x7f3be4101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.789+0000 7f3beb277640 1 --2- 172.21.15.52:0/53973418 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be4107ab0 0x7f3be4107e90 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.789+0000 7f3beb277640 1 -- 172.21.15.52:0/53973418 >> 172.21.15.52:0/53973418 conn(0x7f3be40fd530 msgr2=0x7f3be40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.789+0000 7f3beb277640 1 -- 172.21.15.52:0/53973418 shutdown_connections 2024-04-02T13:59:35.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.789+0000 7f3beb277640 1 -- 172.21.15.52:0/53973418 wait complete. 2024-04-02T13:59:35.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.790+0000 7f3beb277640 1 Processor -- start 2024-04-02T13:59:35.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.790+0000 7f3beb277640 1 -- start start 2024-04-02T13:59:35.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.790+0000 7f3beb277640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3be4101aa0 0x7f3be41983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.790+0000 7f3beb277640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be4102460 0x7f3be41988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.790+0000 7f3beb277640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3be4107ab0 0x7f3be419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.790+0000 7f3beb277640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3be4111e90 con 0x7f3be4102460 2024-04-02T13:59:35.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.790+0000 7f3beb277640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3be4111d10 con 0x7f3be4101aa0 2024-04-02T13:59:35.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.790+0000 7f3beb277640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3be4112010 con 0x7f3be4107ab0 2024-04-02T13:59:35.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.790+0000 7f3be8812640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be4102460 0x7f3be41988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.790+0000 7f3be8812640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be4102460 0x7f3be41988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52138/0 (socket says 172.21.15.52:52138) 2024-04-02T13:59:35.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.790+0000 7f3be8812640 1 -- 172.21.15.52:0/2833352777 learned_addr learned my addr 172.21.15.52:0/2833352777 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:35.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.790+0000 7f3be9013640 1 --2- 172.21.15.52:0/2833352777 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3be4101aa0 0x7f3be41983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.790+0000 7f3be9814640 1 --2- 172.21.15.52:0/2833352777 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3be4107ab0 0x7f3be419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.790+0000 7f3be8812640 1 -- 172.21.15.52:0/2833352777 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3be4107ab0 msgr2=0x7f3be419cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.790+0000 7f3be8812640 1 --2- 172.21.15.52:0/2833352777 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3be4107ab0 0x7f3be419cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.790+0000 7f3be8812640 1 -- 172.21.15.52:0/2833352777 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3be4101aa0 msgr2=0x7f3be41983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.790+0000 7f3be8812640 1 --2- 172.21.15.52:0/2833352777 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3be4101aa0 0x7f3be41983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.790+0000 7f3be8812640 1 -- 172.21.15.52:0/2833352777 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be419d3a0 con 0x7f3be4102460 2024-04-02T13:59:35.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.790+0000 7f3be8812640 1 --2- 172.21.15.52:0/2833352777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be4102460 0x7f3be41988f0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400e9f0 tx=0x7f3bd400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.790+0000 7f3bda7fc640 1 -- 172.21.15.52:0/2833352777 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd400cd80 con 0x7f3be4102460 2024-04-02T13:59:35.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.791+0000 7f3bda7fc640 1 -- 172.21.15.52:0/2833352777 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd4010910 con 0x7f3be4102460 2024-04-02T13:59:35.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.791+0000 7f3beb277640 1 -- 172.21.15.52:0/2833352777 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be419d690 con 0x7f3be4102460 2024-04-02T13:59:35.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.791+0000 7f3bda7fc640 1 -- 172.21.15.52:0/2833352777 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4018990 con 0x7f3be4102460 2024-04-02T13:59:35.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.791+0000 7f3beb277640 1 -- 172.21.15.52:0/2833352777 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be41a4f20 con 0x7f3be4102460 2024-04-02T13:59:35.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.793+0000 7f3bda7fc640 1 -- 172.21.15.52:0/2833352777 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3bd4010ac0 con 0x7f3be4102460 2024-04-02T13:59:35.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.793+0000 7f3bede5c740 1 Processor -- start 2024-04-02T13:59:35.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.793+0000 7f3bede5c740 1 -- start start 2024-04-02T13:59:35.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.793+0000 7f3bede5c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c574d0 0x5577a3c578b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.793+0000 7f3bede5c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577a3c57df0 0x5577a3c64b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.793+0000 7f3bda7fc640 1 --2- 172.21.15.52:0/2833352777 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3bd00a3d20 0x7f3bd00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.793+0000 7f3bede5c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577a3c650b0 0x5577a3c674a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.793+0000 7f3bede5c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5577a3bbb7b0 con 0x5577a3c574d0 2024-04-02T13:59:35.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.793+0000 7f3bede5c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5577a3b5a570 con 0x5577a3c57df0 2024-04-02T13:59:35.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.793+0000 7f3bede5c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5577a3bb7760 con 0x5577a3c650b0 2024-04-02T13:59:35.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.794+0000 7f3be9814640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c574d0 0x5577a3c578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.794+0000 7f3be9814640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c574d0 0x5577a3c578b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52146/0 (socket says 172.21.15.52:52146) 2024-04-02T13:59:35.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.794+0000 7f3be9814640 1 -- 172.21.15.52:0/3756669512 learned_addr learned my addr 172.21.15.52:0/3756669512 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:35.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.794+0000 7f3be8812640 1 --2- 172.21.15.52:0/2833352777 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3bd00a3d20 0x7f3bd00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.794+0000 7f3be9814640 1 --2- 172.21.15.52:0/3756669512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577a3c650b0 0x5577a3c674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.794+0000 7f3be9013640 1 --2- 172.21.15.52:0/3756669512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577a3c57df0 0x5577a3c64b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.794+0000 7f3bda7fc640 1 -- 172.21.15.52:0/2833352777 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f3bd40d0e60 con 0x7f3be4102460 2024-04-02T13:59:35.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.794+0000 7f3be9814640 1 -- 172.21.15.52:0/3756669512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577a3c650b0 msgr2=0x5577a3c674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.794+0000 7f3be9814640 1 --2- 172.21.15.52:0/3756669512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577a3c650b0 0x5577a3c674a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.794+0000 7f3be9814640 1 -- 172.21.15.52:0/3756669512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577a3c57df0 msgr2=0x5577a3c64b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.794+0000 7f3be9814640 1 --2- 172.21.15.52:0/3756669512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577a3c57df0 0x5577a3c64b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.794+0000 7f3be9814640 1 -- 172.21.15.52:0/3756669512 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577a38cf3f0 con 0x5577a3c574d0 2024-04-02T13:59:35.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.794+0000 7f3be8812640 1 --2- 172.21.15.52:0/2833352777 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3bd00a3d20 0x7f3bd00a61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400e9c0 tx=0x7f3bd4092000 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.794+0000 7f3be9814640 1 --2- 172.21.15.52:0/3756669512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c574d0 0x5577a3c578b0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc005c20 tx=0x7f3bdc009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515f8cdb6ab5ac5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.794+0000 7f3bcf7fe640 1 -- 172.21.15.52:0/3756669512 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc00ed80 con 0x5577a3c574d0 2024-04-02T13:59:35.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.794+0000 7f3bcf7fe640 1 -- 172.21.15.52:0/3756669512 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bdc005020 con 0x5577a3c574d0 2024-04-02T13:59:35.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.794+0000 7f3bcf7fe640 1 -- 172.21.15.52:0/3756669512 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc012670 con 0x5577a3c574d0 2024-04-02T13:59:35.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.795+0000 7f3bede5c740 1 -- 172.21.15.52:0/3756669512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c574d0 msgr2=0x5577a3c578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.795+0000 7f3bede5c740 1 --2- 172.21.15.52:0/3756669512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c574d0 0x5577a3c578b0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc005c20 tx=0x7f3bdc009a00 comp rx=0 tx=0).stop 2024-04-02T13:59:35.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.795+0000 7f3bede5c740 1 -- 172.21.15.52:0/3756669512 shutdown_connections 2024-04-02T13:59:35.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.795+0000 7f3bede5c740 1 --2- 172.21.15.52:0/3756669512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577a3c650b0 0x5577a3c674a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.795+0000 7f3bede5c740 1 --2- 172.21.15.52:0/3756669512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577a3c57df0 0x5577a3c64b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.795+0000 7f3bede5c740 1 --2- 172.21.15.52:0/3756669512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c574d0 0x5577a3c578b0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.795+0000 7f3bede5c740 1 -- 172.21.15.52:0/3756669512 >> 172.21.15.52:0/3756669512 conn(0x5577a3c51ed0 msgr2=0x5577a3c522b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.795+0000 7f3bede5c740 1 -- 172.21.15.52:0/3756669512 shutdown_connections 2024-04-02T13:59:35.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.795+0000 7f3bede5c740 1 -- 172.21.15.52:0/3756669512 wait complete. 2024-04-02T13:59:35.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.795+0000 7f3bede5c740 1 -- start start 2024-04-02T13:59:35.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.795+0000 7f3bede5c740 1 Processor -- start 2024-04-02T13:59:35.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.796+0000 7f3bede5c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577a3c574d0 0x5577a3c64aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.796+0000 7f3bede5c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577a3c57df0 0x5577a3c61e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.796+0000 7f3bede5c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c650b0 0x5577a3c62440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.796+0000 7f3bede5c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5577a3c67e10 con 0x5577a3c650b0 2024-04-02T13:59:35.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.796+0000 7f3bede5c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5577a3b76e80 con 0x5577a3c574d0 2024-04-02T13:59:35.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.796+0000 7f3bede5c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5577a3c67f90 con 0x5577a3c57df0 2024-04-02T13:59:35.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.796+0000 7f3bede5c740 10 client.0 _ll_register_callbacks cb 0x5577a3bb2aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:35.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.796+0000 7f3be9814640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c650b0 0x5577a3c62440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.796+0000 7f3be9814640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c650b0 0x5577a3c62440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52156/0 (socket says 172.21.15.52:52156) 2024-04-02T13:59:35.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.796+0000 7f3be9814640 1 -- 172.21.15.52:0/942670217 learned_addr learned my addr 172.21.15.52:0/942670217 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:35.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.796+0000 7f3be9814640 1 --2- 172.21.15.52:0/942670217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577a3c574d0 0x5577a3c64aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.796+0000 7f3be9013640 1 --2- 172.21.15.52:0/942670217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577a3c57df0 0x5577a3c61e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.796+0000 7f3be9814640 1 -- 172.21.15.52:0/942670217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577a3c57df0 msgr2=0x5577a3c61e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.796+0000 7f3be9814640 1 --2- 172.21.15.52:0/942670217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577a3c57df0 0x5577a3c61e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.796+0000 7f3be9814640 1 -- 172.21.15.52:0/942670217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577a3c574d0 msgr2=0x5577a3c64aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.796+0000 7f3be9814640 1 --2- 172.21.15.52:0/942670217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577a3c574d0 0x5577a3c64aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.796+0000 7f3be9814640 1 -- 172.21.15.52:0/942670217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577a39997d0 con 0x5577a3c650b0 2024-04-02T13:59:35.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.796+0000 7f3be9013640 1 --2- 172.21.15.52:0/942670217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577a3c57df0 0x5577a3c61e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:59:35.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.796+0000 7f3be9814640 1 --2- 172.21.15.52:0/942670217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c650b0 0x5577a3c62440 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc005cc0 tx=0x7f3bdc005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cfae111a1b35fb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.796+0000 7f3bccff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:35.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.796+0000 7f3bccff9640 1 -- 172.21.15.52:0/942670217 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc02c060 con 0x5577a3c650b0 2024-04-02T13:59:35.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.797+0000 7f3bccff9640 1 -- 172.21.15.52:0/942670217 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f3bdc00ed50 con 0x5577a3c650b0 2024-04-02T13:59:35.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.797+0000 7f3bede5c740 1 -- 172.21.15.52:0/942670217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bdc005020 con 0x5577a3c650b0 2024-04-02T13:59:35.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.797+0000 7f3bccff9640 1 -- 172.21.15.52:0/942670217 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3bdc0148e0 con 0x5577a3c650b0 2024-04-02T13:59:35.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.797+0000 7f3bede5c740 10 client.17382 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:35.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.797+0000 7f3bede5c740 1 -- 172.21.15.52:0/942670217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5577a3c73e70 con 0x5577a3c650b0 2024-04-02T13:59:35.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.797+0000 7f3bccff9640 1 -- 172.21.15.52:0/942670217 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bdc005020 con 0x5577a3c650b0 2024-04-02T13:59:35.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.797+0000 7f3bccff9640 1 -- 172.21.15.52:0/942670217 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3bdc01e020 con 0x5577a3c650b0 2024-04-02T13:59:35.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.797+0000 7f3bede5c740 10 client.17382 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:35.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.797+0000 7f3bede5c740 10 client.17382 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:59:35.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.797+0000 7f3bede5c740 20 client.17382 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:35.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.797+0000 7f3bede5c740 1 --2- 172.21.15.52:0/942670217 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5577a3c752d0 0x5577a3c776c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.797+0000 7f3bede5c740 4 client.17382 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:59:35.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.797+0000 7f3bede5c740 1 -- 172.21.15.52:0/942670217 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577a3bc8cf0 con 0x5577a3c752d0 2024-04-02T13:59:35.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.797+0000 7f3be9013640 1 --2- 172.21.15.52:0/942670217 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5577a3c752d0 0x5577a3c776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.798+0000 7f3be9013640 1 --2- 172.21.15.52:0/942670217 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5577a3c752d0 0x5577a3c776c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d93aaad7d066e3b server_cookie=825fba48ac0143bb in_seq=0 out_seq=0 2024-04-02T13:59:35.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.798+0000 7f3bccff9640 10 client.17382 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:59:35.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.799+0000 7f3bccff9640 1 -- 172.21.15.52:0/942670217 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5577a3bc8cf0 con 0x5577a3c752d0 2024-04-02T13:59:35.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.799+0000 7f3bccff9640 10 client.17382 handle_command_reply: tid=0 2024-04-02T13:59:35.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.799+0000 7f3bede5c740 1 client.17382 shutdown 2024-04-02T13:59:35.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.799+0000 7f3bede5c740 1 -- 172.21.15.52:0/942670217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c650b0 msgr2=0x5577a3c62440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.799+0000 7f3bede5c740 1 --2- 172.21.15.52:0/942670217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c650b0 0x5577a3c62440 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc005cc0 tx=0x7f3bdc005c50 comp rx=0 tx=0).stop 2024-04-02T13:59:35.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.799+0000 7f3bede5c740 1 -- 172.21.15.52:0/942670217 shutdown_connections 2024-04-02T13:59:35.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.799+0000 7f3bede5c740 1 --2- 172.21.15.52:0/942670217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c650b0 0x5577a3c62440 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.799+0000 7f3bede5c740 1 --2- 172.21.15.52:0/942670217 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5577a3c752d0 0x5577a3c776c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.799+0000 7f3bede5c740 1 --2- 172.21.15.52:0/942670217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577a3c57df0 0x5577a3c61e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.799+0000 7f3bccff9640 0 client.17382 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:59:35.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.799+0000 7f3bede5c740 1 --2- 172.21.15.52:0/942670217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577a3c574d0 0x5577a3c64aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.799+0000 7f3bede5c740 1 -- 172.21.15.52:0/942670217 >> 172.21.15.52:0/942670217 conn(0x5577a3c51ed0 msgr2=0x5577a3c53c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.799+0000 7f3bede5c740 1 -- 172.21.15.52:0/942670217 shutdown_connections 2024-04-02T13:59:35.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.799+0000 7f3bede5c740 1 -- 172.21.15.52:0/942670217 wait complete. 2024-04-02T13:59:35.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.799+0000 7f3bede5c740 20 client.17382 trim_cache size 0 max 16384 2024-04-02T13:59:35.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.813+0000 7f3bede5c740 1 Processor -- start 2024-04-02T13:59:35.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.813+0000 7f3bede5c740 1 -- start start 2024-04-02T13:59:35.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.813+0000 7f3bede5c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577a3c574d0 0x5577a3c70d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.813+0000 7f3bede5c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c57df0 0x5577a3c60ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.813+0000 7f3bede5c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577a3c650b0 0x5577a3c64560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.813+0000 7f3bede5c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3be41acce0 con 0x5577a3c57df0 2024-04-02T13:59:35.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.813+0000 7f3bede5c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3be41ae820 con 0x5577a3c650b0 2024-04-02T13:59:35.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.813+0000 7f3bede5c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3be41ad3b0 con 0x5577a3c574d0 2024-04-02T13:59:35.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.813+0000 7f3be9013640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c57df0 0x5577a3c60ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.813+0000 7f3be9814640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577a3c574d0 0x5577a3c70d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.813+0000 7f3be9013640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c57df0 0x5577a3c60ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52172/0 (socket says 172.21.15.52:52172) 2024-04-02T13:59:35.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.813+0000 7f3be9013640 1 -- 172.21.15.52:0/991004387 learned_addr learned my addr 172.21.15.52:0/991004387 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:35.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.813+0000 7f3be9814640 1 --2- 172.21.15.52:0/991004387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577a3c650b0 0x5577a3c64560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.813+0000 7f3be9013640 1 -- 172.21.15.52:0/991004387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577a3c574d0 msgr2=0x5577a3c70d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.813+0000 7f3be9013640 1 --2- 172.21.15.52:0/991004387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577a3c574d0 0x5577a3c70d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.813+0000 7f3be9013640 1 -- 172.21.15.52:0/991004387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577a3c650b0 msgr2=0x5577a3c64560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.813+0000 7f3be9013640 1 --2- 172.21.15.52:0/991004387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577a3c650b0 0x5577a3c64560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.813+0000 7f3be9013640 1 -- 172.21.15.52:0/991004387 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577a3c57c20 con 0x5577a3c57df0 2024-04-02T13:59:35.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.813+0000 7f3be9814640 1 --2- 172.21.15.52:0/991004387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577a3c650b0 0x5577a3c64560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:59:35.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.813+0000 7f3be9013640 1 --2- 172.21.15.52:0/991004387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c57df0 0x5577a3c60ce0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009900 tx=0x7f3be0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c28b617791e5e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.814+0000 7f3bceffd640 1 -- 172.21.15.52:0/991004387 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0047dc0 con 0x5577a3c57df0 2024-04-02T13:59:35.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.814+0000 7f3bceffd640 1 -- 172.21.15.52:0/991004387 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be004c070 con 0x5577a3c57df0 2024-04-02T13:59:35.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.814+0000 7f3bceffd640 1 -- 172.21.15.52:0/991004387 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be0047940 con 0x5577a3c57df0 2024-04-02T13:59:35.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.814+0000 7f3bede5c740 1 -- 172.21.15.52:0/991004387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c57df0 msgr2=0x5577a3c60ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.814+0000 7f3bede5c740 1 --2- 172.21.15.52:0/991004387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c57df0 0x5577a3c60ce0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009900 tx=0x7f3be0005fd0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.814+0000 7f3bede5c740 1 -- 172.21.15.52:0/991004387 shutdown_connections 2024-04-02T13:59:35.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.814+0000 7f3bede5c740 1 --2- 172.21.15.52:0/991004387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577a3c650b0 0x5577a3c64560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.814+0000 7f3bede5c740 1 --2- 172.21.15.52:0/991004387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c57df0 0x5577a3c60ce0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.814+0000 7f3bede5c740 1 --2- 172.21.15.52:0/991004387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577a3c574d0 0x5577a3c70d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.814+0000 7f3bede5c740 1 -- 172.21.15.52:0/991004387 >> 172.21.15.52:0/991004387 conn(0x5577a3c51ed0 msgr2=0x5577a3c605d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.814+0000 7f3bede5c740 1 -- 172.21.15.52:0/991004387 shutdown_connections 2024-04-02T13:59:35.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.814+0000 7f3bede5c740 1 -- 172.21.15.52:0/991004387 wait complete. 2024-04-02T13:59:35.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.815+0000 7f3bede5c740 1 -- start start 2024-04-02T13:59:35.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.815+0000 7f3bede5c740 1 Processor -- start 2024-04-02T13:59:35.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.815+0000 7f3bede5c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c574d0 0x5577a3c65c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.815+0000 7f3bede5c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577a3c57df0 0x5577a3c661c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.815+0000 7f3bede5c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577a3c650b0 0x5577a3c6c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.815+0000 7f3bede5c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3be41ae150 con 0x5577a3c574d0 2024-04-02T13:59:35.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.815+0000 7f3bede5c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3be41ada80 con 0x5577a3c650b0 2024-04-02T13:59:35.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.815+0000 7f3bede5c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3be41aacc0 con 0x5577a3c57df0 2024-04-02T13:59:35.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.815+0000 7f3bede5c740 10 client.0 _ll_register_callbacks cb 0x7f3be0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:35.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.815+0000 7f3be9814640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c574d0 0x5577a3c65c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.815+0000 7f3be9814640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c574d0 0x5577a3c65c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52186/0 (socket says 172.21.15.52:52186) 2024-04-02T13:59:35.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.815+0000 7f3be9814640 1 -- 172.21.15.52:0/3488835775 learned_addr learned my addr 172.21.15.52:0/3488835775 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:35.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.815+0000 7f3be9814640 1 --2- 172.21.15.52:0/3488835775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577a3c650b0 0x5577a3c6c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.815+0000 7f3be9814640 1 -- 172.21.15.52:0/3488835775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577a3c57df0 msgr2=0x5577a3c661c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.815+0000 7f3be9814640 1 --2- 172.21.15.52:0/3488835775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577a3c57df0 0x5577a3c661c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.815+0000 7f3be9814640 1 -- 172.21.15.52:0/3488835775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577a3c650b0 msgr2=0x5577a3c6c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.815+0000 7f3be9814640 1 --2- 172.21.15.52:0/3488835775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577a3c650b0 0x5577a3c6c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.815+0000 7f3be9814640 1 -- 172.21.15.52:0/3488835775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577a3c6c810 con 0x5577a3c574d0 2024-04-02T13:59:35.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.815+0000 7f3be9814640 1 --2- 172.21.15.52:0/3488835775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c574d0 0x5577a3c65c80 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc014c10 tx=0x7f3bdc005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7909d0b5038bc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.815+0000 7f3bccff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:35.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.816+0000 7f3bccff9640 1 -- 172.21.15.52:0/3488835775 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc02c040 con 0x5577a3c574d0 2024-04-02T13:59:35.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.816+0000 7f3bccff9640 1 -- 172.21.15.52:0/3488835775 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f3bdc005990 con 0x5577a3c574d0 2024-04-02T13:59:35.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.816+0000 7f3bede5c740 1 -- 172.21.15.52:0/3488835775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3be004c070 con 0x5577a3c574d0 2024-04-02T13:59:35.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.816+0000 7f3bccff9640 1 -- 172.21.15.52:0/3488835775 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3bdc00eb50 con 0x5577a3c574d0 2024-04-02T13:59:35.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.816+0000 7f3bede5c740 10 client.17388 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:35.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.816+0000 7f3bede5c740 1 -- 172.21.15.52:0/3488835775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5577a3c6da70 con 0x5577a3c574d0 2024-04-02T13:59:35.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.816+0000 7f3bccff9640 1 -- 172.21.15.52:0/3488835775 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be004c070 con 0x5577a3c574d0 2024-04-02T13:59:35.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.816+0000 7f3bccff9640 1 -- 172.21.15.52:0/3488835775 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3bdc01e040 con 0x5577a3c574d0 2024-04-02T13:59:35.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.816+0000 7f3bede5c740 10 client.17388 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:35.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.816+0000 7f3bede5c740 10 client.17388 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:59:35.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.816+0000 7f3bede5c740 20 client.17388 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:35.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.816+0000 7f3bede5c740 1 --2- 172.21.15.52:0/3488835775 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5577a3cdbae0 0x5577a3cddf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.816+0000 7f3bede5c740 4 client.17388 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:59:35.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.816+0000 7f3bede5c740 1 -- 172.21.15.52:0/3488835775 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5577a3bc3040 con 0x5577a3cdbae0 2024-04-02T13:59:35.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.817+0000 7f3be9013640 1 --2- 172.21.15.52:0/3488835775 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5577a3cdbae0 0x5577a3cddf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.817+0000 7f3be9013640 1 --2- 172.21.15.52:0/3488835775 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5577a3cdbae0 0x5577a3cddf00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6a5e9f8d2b00fb1c server_cookie=ca6d1f505aaa6c in_seq=0 out_seq=0 2024-04-02T13:59:35.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.817+0000 7f3bccff9640 10 client.17388 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T13:59:35.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.817+0000 7f3bccff9640 1 -- 172.21.15.52:0/3488835775 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5577a3bc3040 con 0x5577a3cdbae0 2024-04-02T13:59:35.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.817+0000 7f3bccff9640 10 client.17388 handle_command_reply: tid=0 2024-04-02T13:59:35.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.817+0000 7f3bede5c740 1 client.17388 shutdown 2024-04-02T13:59:35.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.817+0000 7f3bede5c740 1 -- 172.21.15.52:0/3488835775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c574d0 msgr2=0x5577a3c65c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.817+0000 7f3bede5c740 1 --2- 172.21.15.52:0/3488835775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c574d0 0x5577a3c65c80 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc014c10 tx=0x7f3bdc005c50 comp rx=0 tx=0).stop 2024-04-02T13:59:35.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.817+0000 7f3bede5c740 1 -- 172.21.15.52:0/3488835775 shutdown_connections 2024-04-02T13:59:35.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.817+0000 7f3bede5c740 1 --2- 172.21.15.52:0/3488835775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577a3c650b0 0x5577a3c6c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.817+0000 7f3bede5c740 1 --2- 172.21.15.52:0/3488835775 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5577a3cdbae0 0x5577a3cddf00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.817+0000 7f3bede5c740 1 --2- 172.21.15.52:0/3488835775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577a3c57df0 0x5577a3c661c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.817+0000 7f3bede5c740 1 --2- 172.21.15.52:0/3488835775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577a3c574d0 0x5577a3c65c80 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.817+0000 7f3bccff9640 0 client.17388 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T13:59:35.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.817+0000 7f3bede5c740 1 -- 172.21.15.52:0/3488835775 >> 172.21.15.52:0/3488835775 conn(0x5577a3c51ed0 msgr2=0x5577a3c60e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.817+0000 7f3bede5c740 1 -- 172.21.15.52:0/3488835775 shutdown_connections 2024-04-02T13:59:35.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.817+0000 7f3bede5c740 1 -- 172.21.15.52:0/3488835775 wait complete. 2024-04-02T13:59:35.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.818+0000 7f3bede5c740 20 client.17388 trim_cache size 0 max 16384 2024-04-02T13:59:35.835 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:59:35.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.818+0000 7f3beb277640 1 -- 172.21.15.52:0/2833352777 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3bd00a3d20 msgr2=0x7f3bd00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.818+0000 7f3beb277640 1 --2- 172.21.15.52:0/2833352777 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3bd00a3d20 0x7f3bd00a61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400e9c0 tx=0x7f3bd4092000 comp rx=0 tx=0).stop 2024-04-02T13:59:35.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.818+0000 7f3beb277640 1 -- 172.21.15.52:0/2833352777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be4102460 msgr2=0x7f3be41988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.818+0000 7f3beb277640 1 --2- 172.21.15.52:0/2833352777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be4102460 0x7f3be41988f0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400e9f0 tx=0x7f3bd400eeb0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.818+0000 7f3beb277640 1 -- 172.21.15.52:0/2833352777 shutdown_connections 2024-04-02T13:59:35.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.818+0000 7f3beb277640 1 --2- 172.21.15.52:0/2833352777 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3bd00a3d20 0x7f3bd00a61e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.818+0000 7f3beb277640 1 --2- 172.21.15.52:0/2833352777 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3be4107ab0 0x7f3be419cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.818+0000 7f3beb277640 1 --2- 172.21.15.52:0/2833352777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3be4102460 0x7f3be41988f0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.818+0000 7f3beb277640 1 --2- 172.21.15.52:0/2833352777 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3be4101aa0 0x7f3be41983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.818+0000 7f3beb277640 1 -- 172.21.15.52:0/2833352777 >> 172.21.15.52:0/2833352777 conn(0x7f3be40fd530 msgr2=0x7f3be40fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.818+0000 7f3beb277640 1 -- 172.21.15.52:0/2833352777 shutdown_connections 2024-04-02T13:59:35.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.818+0000 7f3beb277640 1 -- 172.21.15.52:0/2833352777 wait complete. 2024-04-02T13:59:35.839 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:59:35.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:35 smithi088 ceph-mon[36528]: pgmap v185: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:35.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:35 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/479183649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:59:35.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.941+0000 7f6024450640 1 -- 172.21.15.52:0/3975644484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f601c101330 msgr2=0x7f601c10d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.941+0000 7f6024450640 1 --2- 172.21.15.52:0/3975644484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f601c101330 0x7f601c10d840 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f600c009cd0 tx=0x7f600c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.941+0000 7f6024450640 1 -- 172.21.15.52:0/3975644484 shutdown_connections 2024-04-02T13:59:35.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.941+0000 7f6024450640 1 --2- 172.21.15.52:0/3975644484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f601c101330 0x7f601c10d840 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.941+0000 7f6024450640 1 --2- 172.21.15.52:0/3975644484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f601c100970 0x7f601c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.941+0000 7f6024450640 1 --2- 172.21.15.52:0/3975644484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f601c106980 0x7f601c106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.941+0000 7f6024450640 1 -- 172.21.15.52:0/3975644484 >> 172.21.15.52:0/3975644484 conn(0x7f601c077b00 msgr2=0x7f601c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.941+0000 7f6024450640 1 -- 172.21.15.52:0/3975644484 shutdown_connections 2024-04-02T13:59:35.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.941+0000 7f6024450640 1 -- 172.21.15.52:0/3975644484 wait complete. 2024-04-02T13:59:35.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.941+0000 7f6024450640 1 Processor -- start 2024-04-02T13:59:35.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.941+0000 7f6024450640 1 -- start start 2024-04-02T13:59:35.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.942+0000 7f6024450640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f601c100970 0x7f601c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.942+0000 7f6024450640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f601c101330 0x7f601c1996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.942+0000 7f6024450640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f601c106980 0x7f601c1a4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.942+0000 7f6024450640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f601c110d90 con 0x7f601c100970 2024-04-02T13:59:35.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.942+0000 7f6024450640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f601c110c10 con 0x7f601c106980 2024-04-02T13:59:35.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.942+0000 7f6024450640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f601c110f10 con 0x7f601c101330 2024-04-02T13:59:35.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.942+0000 7f60221ec640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f601c100970 0x7f601c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.942+0000 7f60221ec640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f601c100970 0x7f601c1991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52202/0 (socket says 172.21.15.52:52202) 2024-04-02T13:59:35.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.942+0000 7f60221ec640 1 -- 172.21.15.52:0/2573679171 learned_addr learned my addr 172.21.15.52:0/2573679171 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:35.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.942+0000 7f60219eb640 1 --2- 172.21.15.52:0/2573679171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f601c101330 0x7f601c1996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.942+0000 7f60229ed640 1 --2- 172.21.15.52:0/2573679171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f601c106980 0x7f601c1a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.942+0000 7f60221ec640 1 -- 172.21.15.52:0/2573679171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f601c101330 msgr2=0x7f601c1996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.942+0000 7f60221ec640 1 --2- 172.21.15.52:0/2573679171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f601c101330 0x7f601c1996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.942+0000 7f60221ec640 1 -- 172.21.15.52:0/2573679171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f601c106980 msgr2=0x7f601c1a4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.942+0000 7f60221ec640 1 --2- 172.21.15.52:0/2573679171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f601c106980 0x7f601c1a4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.942+0000 7f60221ec640 1 -- 172.21.15.52:0/2573679171 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601c1a4770 con 0x7f601c100970 2024-04-02T13:59:35.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.942+0000 7f60221ec640 1 --2- 172.21.15.52:0/2573679171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f601c100970 0x7f601c1991b0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f601400d950 tx=0x7f601400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.942+0000 7f601b7fe640 1 -- 172.21.15.52:0/2573679171 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6014017070 con 0x7f601c100970 2024-04-02T13:59:35.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.943+0000 7f6024450640 1 -- 172.21.15.52:0/2573679171 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601c1a4940 con 0x7f601c100970 2024-04-02T13:59:35.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.943+0000 7f601b7fe640 1 -- 172.21.15.52:0/2573679171 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6014009c30 con 0x7f601c100970 2024-04-02T13:59:35.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.943+0000 7f601b7fe640 1 -- 172.21.15.52:0/2573679171 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601400f6b0 con 0x7f601c100970 2024-04-02T13:59:35.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.943+0000 7f6024450640 1 -- 172.21.15.52:0/2573679171 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f601c1a4eb0 con 0x7f601c100970 2024-04-02T13:59:35.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.945+0000 7f601b7fe640 1 -- 172.21.15.52:0/2573679171 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f60140109d0 con 0x7f601c100970 2024-04-02T13:59:35.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.946+0000 7f6027030740 1 Processor -- start 2024-04-02T13:59:35.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.946+0000 7f6027030740 1 -- start start 2024-04-02T13:59:35.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.946+0000 7f6027030740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c417024180 0x55c417024560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.946+0000 7f6027030740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c417024b30 0x55c417031b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.946+0000 7f6027030740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c4170320b0 0x55c4170344a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.946+0000 7f6027030740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c416f887b0 con 0x55c4170320b0 2024-04-02T13:59:35.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.946+0000 7f6027030740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c416f27570 con 0x55c417024b30 2024-04-02T13:59:35.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.946+0000 7f6027030740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c416f84760 con 0x55c417024180 2024-04-02T13:59:35.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.946+0000 7f60219eb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c4170320b0 0x55c4170344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.946+0000 7f601b7fe640 1 --2- 172.21.15.52:0/2573679171 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f60040a3d20 0x7f60040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.946+0000 7f60219eb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c4170320b0 0x55c4170344a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52214/0 (socket says 172.21.15.52:52214) 2024-04-02T13:59:35.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.946+0000 7f60219eb640 1 -- 172.21.15.52:0/892068911 learned_addr learned my addr 172.21.15.52:0/892068911 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:35.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.947+0000 7f60229ed640 1 --2- 172.21.15.52:0/892068911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c417024180 0x55c417024560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.947+0000 7f60221ec640 1 --2- 172.21.15.52:0/892068911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c417024b30 0x55c417031b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.947+0000 7f60229ed640 1 --2- 172.21.15.52:0/2573679171 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f60040a3d20 0x7f60040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.947+0000 7f60219eb640 1 -- 172.21.15.52:0/892068911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c417024180 msgr2=0x55c417024560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.947+0000 7f60219eb640 1 --2- 172.21.15.52:0/892068911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c417024180 0x55c417024560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.947+0000 7f60219eb640 1 -- 172.21.15.52:0/892068911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c417024b30 msgr2=0x55c417031b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.947+0000 7f60219eb640 1 --2- 172.21.15.52:0/892068911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c417024b30 0x55c417031b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.947+0000 7f60219eb640 1 -- 172.21.15.52:0/892068911 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c416c9c3f0 con 0x55c4170320b0 2024-04-02T13:59:35.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.947+0000 7f60221ec640 1 --2- 172.21.15.52:0/892068911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c417024b30 0x55c417031b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:59:35.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.947+0000 7f601b7fe640 1 -- 172.21.15.52:0/2573679171 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f60140d0b60 con 0x7f601c100970 2024-04-02T13:59:35.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.947+0000 7f60219eb640 1 --2- 172.21.15.52:0/892068911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c4170320b0 0x55c4170344a0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f601000dc80 tx=0x7f601000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d82dd51bf50ad4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.947+0000 7f60229ed640 1 --2- 172.21.15.52:0/2573679171 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f60040a3d20 0x7f60040a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f600c02c630 tx=0x7f600c0095c0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.947+0000 7f6018bfe640 1 -- 172.21.15.52:0/892068911 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010015070 con 0x55c4170320b0 2024-04-02T13:59:35.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.947+0000 7f6018bfe640 1 -- 172.21.15.52:0/892068911 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f601001a940 con 0x55c4170320b0 2024-04-02T13:59:35.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.947+0000 7f6018bfe640 1 -- 172.21.15.52:0/892068911 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010011a60 con 0x55c4170320b0 2024-04-02T13:59:35.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.948+0000 7f6027030740 1 -- 172.21.15.52:0/892068911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c4170320b0 msgr2=0x55c4170344a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.948+0000 7f6027030740 1 --2- 172.21.15.52:0/892068911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c4170320b0 0x55c4170344a0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f601000dc80 tx=0x7f601000f730 comp rx=0 tx=0).stop 2024-04-02T13:59:35.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.948+0000 7f6027030740 1 -- 172.21.15.52:0/892068911 shutdown_connections 2024-04-02T13:59:35.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.948+0000 7f6027030740 1 --2- 172.21.15.52:0/892068911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c4170320b0 0x55c4170344a0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.948+0000 7f6027030740 1 --2- 172.21.15.52:0/892068911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c417024b30 0x55c417031b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.948+0000 7f6027030740 1 --2- 172.21.15.52:0/892068911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c417024180 0x55c417024560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.948+0000 7f6027030740 1 -- 172.21.15.52:0/892068911 >> 172.21.15.52:0/892068911 conn(0x55c41701eed0 msgr2=0x55c41701f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.948+0000 7f6027030740 1 -- 172.21.15.52:0/892068911 shutdown_connections 2024-04-02T13:59:35.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.948+0000 7f6027030740 1 -- 172.21.15.52:0/892068911 wait complete. 2024-04-02T13:59:35.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.948+0000 7f6027030740 1 -- start start 2024-04-02T13:59:35.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.949+0000 7f6027030740 1 Processor -- start 2024-04-02T13:59:35.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.949+0000 7f6027030740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c417024180 0x55c417031ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.949+0000 7f6027030740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c417024b30 0x55c41702ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.949+0000 7f6027030740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4170320b0 0x55c41702f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.949+0000 7f6027030740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c417034e70 con 0x55c417024180 2024-04-02T13:59:35.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.949+0000 7f6027030740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c416f43e80 con 0x55c4170320b0 2024-04-02T13:59:35.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.949+0000 7f6027030740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c417034ff0 con 0x55c417024b30 2024-04-02T13:59:35.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.949+0000 7f60221ec640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c417024180 0x55c417031ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.949+0000 7f6027030740 10 client.0 _ll_register_callbacks cb 0x55c416f7faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:35.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.949+0000 7f60221ec640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c417024180 0x55c417031ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52216/0 (socket says 172.21.15.52:52216) 2024-04-02T13:59:35.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.949+0000 7f60221ec640 1 -- 172.21.15.52:0/3643099806 learned_addr learned my addr 172.21.15.52:0/3643099806 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:35.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.949+0000 7f60219eb640 1 --2- 172.21.15.52:0/3643099806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c417024b30 0x55c41702ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.949+0000 7f60229ed640 1 --2- 172.21.15.52:0/3643099806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4170320b0 0x55c41702f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.949+0000 7f60221ec640 1 -- 172.21.15.52:0/3643099806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c417024b30 msgr2=0x55c41702ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.949+0000 7f60221ec640 1 --2- 172.21.15.52:0/3643099806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c417024b30 0x55c41702ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.949+0000 7f60221ec640 1 -- 172.21.15.52:0/3643099806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4170320b0 msgr2=0x55c41702f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.949+0000 7f60221ec640 1 --2- 172.21.15.52:0/3643099806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4170320b0 0x55c41702f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.949+0000 7f60221ec640 1 -- 172.21.15.52:0/3643099806 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c416d667d0 con 0x55c417024180 2024-04-02T13:59:35.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.949+0000 7f60229ed640 1 --2- 172.21.15.52:0/3643099806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4170320b0 0x55c41702f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:59:35.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.949+0000 7f60221ec640 1 --2- 172.21.15.52:0/3643099806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c417024180 0x55c417031ac0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f6014098bc0 tx=0x7f60140be6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0963e986d96a32 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.949+0000 7f6001ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:35.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.949+0000 7f6001ffb640 1 -- 172.21.15.52:0/3643099806 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6014020070 con 0x55c417024180 2024-04-02T13:59:35.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.950+0000 7f6001ffb640 1 -- 172.21.15.52:0/3643099806 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f60140ce190 con 0x55c417024180 2024-04-02T13:59:35.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.950+0000 7f6027030740 1 -- 172.21.15.52:0/3643099806 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f601001a940 con 0x55c417024180 2024-04-02T13:59:35.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.950+0000 7f6001ffb640 1 -- 172.21.15.52:0/3643099806 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f60140ce390 con 0x55c417024180 2024-04-02T13:59:35.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.950+0000 7f6027030740 10 client.17400 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:35.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.950+0000 7f6027030740 1 -- 172.21.15.52:0/3643099806 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c417040f10 con 0x55c417024180 2024-04-02T13:59:35.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.950+0000 7f6001ffb640 1 -- 172.21.15.52:0/3643099806 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f601001a940 con 0x55c417024180 2024-04-02T13:59:35.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.950+0000 7f6001ffb640 1 -- 172.21.15.52:0/3643099806 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f60140a4f40 con 0x55c417024180 2024-04-02T13:59:35.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.950+0000 7f6027030740 10 client.17400 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:35.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.950+0000 7f6027030740 10 client.17400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:59:35.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.950+0000 7f6027030740 20 client.17400 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:35.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.950+0000 7f6027030740 1 --2- 172.21.15.52:0/3643099806 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c417042260 0x55c417044650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.950+0000 7f6027030740 4 client.17400 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:59:35.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.950+0000 7f6027030740 1 -- 172.21.15.52:0/3643099806 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c416f95cf0 con 0x55c417042260 2024-04-02T13:59:35.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.950+0000 7f60219eb640 1 --2- 172.21.15.52:0/3643099806 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c417042260 0x55c417044650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.951+0000 7f60219eb640 1 --2- 172.21.15.52:0/3643099806 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c417042260 0x55c417044650 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6733ca155594596 server_cookie=f39de238366a19dc in_seq=0 out_seq=0 2024-04-02T13:59:35.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.951+0000 7f6001ffb640 10 client.17400 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:35.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.951+0000 7f6001ffb640 1 -- 172.21.15.52:0/3643099806 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c416f95cf0 con 0x55c417042260 2024-04-02T13:59:35.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.951+0000 7f6001ffb640 10 client.17400 handle_command_reply: tid=0 2024-04-02T13:59:35.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.952+0000 7f6027030740 1 client.17400 shutdown 2024-04-02T13:59:35.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.952+0000 7f6027030740 1 -- 172.21.15.52:0/3643099806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c417024180 msgr2=0x55c417031ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.952+0000 7f6027030740 1 --2- 172.21.15.52:0/3643099806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c417024180 0x55c417031ac0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f6014098bc0 tx=0x7f60140be6c0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.952+0000 7f6027030740 1 -- 172.21.15.52:0/3643099806 shutdown_connections 2024-04-02T13:59:35.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.952+0000 7f6027030740 1 --2- 172.21.15.52:0/3643099806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4170320b0 0x55c41702f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.952+0000 7f6027030740 1 --2- 172.21.15.52:0/3643099806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c417024b30 0x55c41702ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.952+0000 7f6027030740 1 --2- 172.21.15.52:0/3643099806 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c417042260 0x55c417044650 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.952+0000 7f6001ffb640 0 client.17400 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:35.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.952+0000 7f6027030740 1 --2- 172.21.15.52:0/3643099806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c417024180 0x55c417031ac0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.952+0000 7f6027030740 1 -- 172.21.15.52:0/3643099806 >> 172.21.15.52:0/3643099806 conn(0x55c41701eed0 msgr2=0x55c417020c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.952+0000 7f6027030740 1 -- 172.21.15.52:0/3643099806 shutdown_connections 2024-04-02T13:59:35.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.952+0000 7f6027030740 1 -- 172.21.15.52:0/3643099806 wait complete. 2024-04-02T13:59:35.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.952+0000 7f6027030740 20 client.17400 trim_cache size 0 max 16384 2024-04-02T13:59:35.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.965+0000 7f6027030740 1 Processor -- start 2024-04-02T13:59:35.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.965+0000 7f6027030740 1 -- start start 2024-04-02T13:59:35.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.965+0000 7f6027030740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c417024180 0x55c41703dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.965+0000 7f6027030740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c417024b30 0x55c41702ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.965+0000 7f6027030740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4170320b0 0x55c417031580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.965+0000 7f6027030740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f601c1accf0 con 0x55c417024180 2024-04-02T13:59:35.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.965+0000 7f6027030740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f601c1ae830 con 0x55c4170320b0 2024-04-02T13:59:35.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.965+0000 7f6027030740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f601c1ad3c0 con 0x55c417024b30 2024-04-02T13:59:35.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.966+0000 7f60221ec640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c417024180 0x55c41703dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.966+0000 7f60221ec640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c417024180 0x55c41703dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52232/0 (socket says 172.21.15.52:52232) 2024-04-02T13:59:35.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.966+0000 7f60221ec640 1 -- 172.21.15.52:0/3557200305 learned_addr learned my addr 172.21.15.52:0/3557200305 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:35.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.966+0000 7f60221ec640 1 -- 172.21.15.52:0/3557200305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c417024b30 msgr2=0x55c41702ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.966+0000 7f60221ec640 1 --2- 172.21.15.52:0/3557200305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c417024b30 0x55c41702ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.966+0000 7f60221ec640 1 -- 172.21.15.52:0/3557200305 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4170320b0 msgr2=0x55c417031580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.966+0000 7f60221ec640 1 --2- 172.21.15.52:0/3557200305 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4170320b0 0x55c417031580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.966+0000 7f60221ec640 1 -- 172.21.15.52:0/3557200305 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c41702f7d0 con 0x55c417024180 2024-04-02T13:59:35.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.966+0000 7f60221ec640 1 --2- 172.21.15.52:0/3557200305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c417024180 0x55c41703dda0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f60140d0f20 tx=0x7f601400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc9512c94bea1fd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.966+0000 7f6003fff640 1 -- 172.21.15.52:0/3557200305 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601409c760 con 0x55c417024180 2024-04-02T13:59:35.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.966+0000 7f6003fff640 1 -- 172.21.15.52:0/3557200305 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60140a0b10 con 0x55c417024180 2024-04-02T13:59:35.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.966+0000 7f6003fff640 1 -- 172.21.15.52:0/3557200305 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60140ce350 con 0x55c417024180 2024-04-02T13:59:35.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.967+0000 7f6027030740 1 -- 172.21.15.52:0/3557200305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c417024180 msgr2=0x55c41703dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.967+0000 7f6027030740 1 --2- 172.21.15.52:0/3557200305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c417024180 0x55c41703dda0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f60140d0f20 tx=0x7f601400bf60 comp rx=0 tx=0).stop 2024-04-02T13:59:35.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.967+0000 7f6027030740 1 -- 172.21.15.52:0/3557200305 shutdown_connections 2024-04-02T13:59:35.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.967+0000 7f6027030740 1 --2- 172.21.15.52:0/3557200305 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4170320b0 0x55c417031580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.967+0000 7f6027030740 1 --2- 172.21.15.52:0/3557200305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c417024b30 0x55c41702ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.967+0000 7f6027030740 1 --2- 172.21.15.52:0/3557200305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c417024180 0x55c41703dda0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.967+0000 7f6027030740 1 -- 172.21.15.52:0/3557200305 >> 172.21.15.52:0/3557200305 conn(0x55c41701eed0 msgr2=0x55c41702d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.967+0000 7f6027030740 1 -- 172.21.15.52:0/3557200305 shutdown_connections 2024-04-02T13:59:35.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.967+0000 7f6027030740 1 -- 172.21.15.52:0/3557200305 wait complete. 2024-04-02T13:59:35.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.967+0000 7f6027030740 1 -- start start 2024-04-02T13:59:35.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.967+0000 7f6027030740 1 Processor -- start 2024-04-02T13:59:35.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.967+0000 7f6027030740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c417024180 0x55c41703a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.967+0000 7f6027030740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c417024b30 0x55c41702c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.967+0000 7f6027030740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4170320b0 0x55c41702c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.967+0000 7f6027030740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f601c1ae160 con 0x55c417024b30 2024-04-02T13:59:35.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.967+0000 7f6027030740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f601c1ada90 con 0x55c417024180 2024-04-02T13:59:35.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.967+0000 7f6027030740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f601c1aacd0 con 0x55c4170320b0 2024-04-02T13:59:35.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.967+0000 7f6027030740 10 client.0 _ll_register_callbacks cb 0x7f601409c310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:35.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.967+0000 7f60219eb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c417024b30 0x55c41702c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.967+0000 7f60219eb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c417024b30 0x55c41702c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52236/0 (socket says 172.21.15.52:52236) 2024-04-02T13:59:35.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.967+0000 7f60219eb640 1 -- 172.21.15.52:0/1035324008 learned_addr learned my addr 172.21.15.52:0/1035324008 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:35.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.968+0000 7f60221ec640 1 --2- 172.21.15.52:0/1035324008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c417024180 0x55c41703a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.968+0000 7f60229ed640 1 --2- 172.21.15.52:0/1035324008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4170320b0 0x55c41702c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.968+0000 7f60219eb640 1 -- 172.21.15.52:0/1035324008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4170320b0 msgr2=0x55c41702c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.968+0000 7f60219eb640 1 --2- 172.21.15.52:0/1035324008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4170320b0 0x55c41702c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.968+0000 7f60219eb640 1 -- 172.21.15.52:0/1035324008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c417024180 msgr2=0x55c41703a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:35.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.968+0000 7f60219eb640 1 --2- 172.21.15.52:0/1035324008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c417024180 0x55c41703a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.968+0000 7f60219eb640 1 -- 172.21.15.52:0/1035324008 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c41702ce30 con 0x55c417024b30 2024-04-02T13:59:35.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.968+0000 7f60229ed640 1 --2- 172.21.15.52:0/1035324008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4170320b0 0x55c41702c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:59:35.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.968+0000 7f60219eb640 1 --2- 172.21.15.52:0/1035324008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c417024b30 0x55c41702c1b0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f6010006010 tx=0x7f601000ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b04938cc6d43b89 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:35.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.968+0000 7f6001ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:35.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.968+0000 7f6001ffb640 1 -- 172.21.15.52:0/1035324008 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010015040 con 0x55c417024b30 2024-04-02T13:59:35.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.968+0000 7f6001ffb640 1 -- 172.21.15.52:0/1035324008 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f6010004450 con 0x55c417024b30 2024-04-02T13:59:35.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.968+0000 7f6027030740 1 -- 172.21.15.52:0/1035324008 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60140a0b10 con 0x55c417024b30 2024-04-02T13:59:35.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.968+0000 7f6001ffb640 1 -- 172.21.15.52:0/1035324008 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6010031050 con 0x55c417024b30 2024-04-02T13:59:35.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.968+0000 7f6027030740 10 client.17406 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:35.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.968+0000 7f6027030740 1 -- 172.21.15.52:0/1035324008 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c4170a87a0 con 0x55c417024b30 2024-04-02T13:59:35.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.968+0000 7f6001ffb640 1 -- 172.21.15.52:0/1035324008 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60140a0b10 con 0x55c417024b30 2024-04-02T13:59:35.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.968+0000 7f6001ffb640 1 -- 172.21.15.52:0/1035324008 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6010011860 con 0x55c417024b30 2024-04-02T13:59:35.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.968+0000 7f6027030740 10 client.17406 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:35.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.968+0000 7f6027030740 10 client.17406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:59:35.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.968+0000 7f6027030740 20 client.17406 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:35.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.968+0000 7f6027030740 1 --2- 172.21.15.52:0/1035324008 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c4170a8e10 0x55c4170ab230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:35.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.968+0000 7f6027030740 4 client.17406 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:59:35.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.968+0000 7f6027030740 1 -- 172.21.15.52:0/1035324008 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c416f90040 con 0x55c4170a8e10 2024-04-02T13:59:35.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.969+0000 7f60221ec640 1 --2- 172.21.15.52:0/1035324008 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c4170a8e10 0x55c4170ab230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:35.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.969+0000 7f6001ffb640 10 client.17406 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:35.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.969+0000 7f60221ec640 1 --2- 172.21.15.52:0/1035324008 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c4170a8e10 0x55c4170ab230 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b96bc439be9959c server_cookie=5a8fb55eb75b6df4 in_seq=0 out_seq=0 2024-04-02T13:59:35.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.969+0000 7f6001ffb640 1 -- 172.21.15.52:0/1035324008 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c416f90040 con 0x55c4170a8e10 2024-04-02T13:59:35.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.969+0000 7f6001ffb640 10 client.17406 handle_command_reply: tid=0 2024-04-02T13:59:35.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.969+0000 7f6027030740 1 client.17406 shutdown 2024-04-02T13:59:35.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.970+0000 7f6027030740 1 -- 172.21.15.52:0/1035324008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c417024b30 msgr2=0x55c41702c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.970+0000 7f6027030740 1 --2- 172.21.15.52:0/1035324008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c417024b30 0x55c41702c1b0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f6010006010 tx=0x7f601000ff60 comp rx=0 tx=0).stop 2024-04-02T13:59:35.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.970+0000 7f6027030740 1 -- 172.21.15.52:0/1035324008 shutdown_connections 2024-04-02T13:59:35.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.970+0000 7f6027030740 1 --2- 172.21.15.52:0/1035324008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c4170320b0 0x55c41702c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.970+0000 7f6027030740 1 --2- 172.21.15.52:0/1035324008 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55c4170a8e10 0x55c4170ab230 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.970+0000 7f6027030740 1 --2- 172.21.15.52:0/1035324008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c417024b30 0x55c41702c1b0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.970+0000 7f6027030740 1 --2- 172.21.15.52:0/1035324008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c417024180 0x55c41703a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.970+0000 7f6027030740 1 -- 172.21.15.52:0/1035324008 >> 172.21.15.52:0/1035324008 conn(0x55c41701eed0 msgr2=0x55c417020320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.970+0000 7f6001ffb640 0 client.17406 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:35.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.970+0000 7f6027030740 1 -- 172.21.15.52:0/1035324008 shutdown_connections 2024-04-02T13:59:35.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.970+0000 7f6027030740 1 -- 172.21.15.52:0/1035324008 wait complete. 2024-04-02T13:59:35.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.970+0000 7f6027030740 20 client.17406 trim_cache size 0 max 16384 2024-04-02T13:59:35.984 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T13:59:35.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.970+0000 7f6024450640 1 -- 172.21.15.52:0/2573679171 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f60040a3d20 msgr2=0x7f60040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.970+0000 7f6024450640 1 --2- 172.21.15.52:0/2573679171 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f60040a3d20 0x7f60040a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f600c02c630 tx=0x7f600c0095c0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.970+0000 7f6024450640 1 -- 172.21.15.52:0/2573679171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f601c100970 msgr2=0x7f601c1991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:35.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.970+0000 7f6024450640 1 --2- 172.21.15.52:0/2573679171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f601c100970 0x7f601c1991b0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f601400d950 tx=0x7f601400de10 comp rx=0 tx=0).stop 2024-04-02T13:59:35.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.970+0000 7f6024450640 1 -- 172.21.15.52:0/2573679171 shutdown_connections 2024-04-02T13:59:35.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.970+0000 7f6024450640 1 --2- 172.21.15.52:0/2573679171 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f60040a3d20 0x7f60040a61e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.970+0000 7f6024450640 1 --2- 172.21.15.52:0/2573679171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f601c106980 0x7f601c1a4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.970+0000 7f6024450640 1 --2- 172.21.15.52:0/2573679171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f601c101330 0x7f601c1996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.970+0000 7f6024450640 1 --2- 172.21.15.52:0/2573679171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f601c100970 0x7f601c1991b0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:35.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.970+0000 7f6024450640 1 -- 172.21.15.52:0/2573679171 >> 172.21.15.52:0/2573679171 conn(0x7f601c077b00 msgr2=0x7f601c103170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:35.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.970+0000 7f6024450640 1 -- 172.21.15.52:0/2573679171 shutdown_connections 2024-04-02T13:59:35.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:35.970+0000 7f6024450640 1 -- 172.21.15.52:0/2573679171 wait complete. 2024-04-02T13:59:36.988 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:59:36.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:59:36.989 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:59:37.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.104+0000 7ffb9e300640 1 -- 172.21.15.52:0/3376025641 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb98103690 msgr2=0x7ffb9810bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:37.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.104+0000 7ffb9e300640 1 --2- 172.21.15.52:0/3376025641 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb98103690 0x7ffb9810bb70 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7ffb88009900 tx=0x7ffb8802bcd0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.104+0000 7ffb9e300640 1 -- 172.21.15.52:0/3376025641 shutdown_connections 2024-04-02T13:59:37.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.104+0000 7ffb9e300640 1 --2- 172.21.15.52:0/3376025641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb9810c140 0x7ffb9810e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.104+0000 7ffb9e300640 1 --2- 172.21.15.52:0/3376025641 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb98103690 0x7ffb9810bb70 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.104+0000 7ffb9e300640 1 --2- 172.21.15.52:0/3376025641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb98102d70 0x7ffb98103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.104+0000 7ffb9e300640 1 -- 172.21.15.52:0/3376025641 >> 172.21.15.52:0/3376025641 conn(0x7ffb98077b60 msgr2=0x7ffb98077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:37.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.104+0000 7ffb9e300640 1 -- 172.21.15.52:0/3376025641 shutdown_connections 2024-04-02T13:59:37.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.104+0000 7ffb9e300640 1 -- 172.21.15.52:0/3376025641 wait complete. 2024-04-02T13:59:37.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.105+0000 7ffb9e300640 1 Processor -- start 2024-04-02T13:59:37.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.105+0000 7ffb9e300640 1 -- start start 2024-04-02T13:59:37.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.105+0000 7ffb9e300640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb98102d70 0x7ffb98198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.105+0000 7ffb9e300640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb98103690 0x7ffb98198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.105+0000 7ffb9e300640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb9810c140 0x7ffb9819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.106+0000 7ffb9e300640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffb98111f10 con 0x7ffb98102d70 2024-04-02T13:59:37.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.106+0000 7ffb9e300640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffb98111d90 con 0x7ffb9810c140 2024-04-02T13:59:37.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.106+0000 7ffb9e300640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffb98112090 con 0x7ffb98103690 2024-04-02T13:59:37.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.106+0000 7ffb97fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb98102d70 0x7ffb98198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.106+0000 7ffb97fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb98102d70 0x7ffb98198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52258/0 (socket says 172.21.15.52:52258) 2024-04-02T13:59:37.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.106+0000 7ffb97fff640 1 -- 172.21.15.52:0/1396820772 learned_addr learned my addr 172.21.15.52:0/1396820772 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:37.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.106+0000 7ffb977fe640 1 --2- 172.21.15.52:0/1396820772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb98103690 0x7ffb98198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.106+0000 7ffb9c89d640 1 --2- 172.21.15.52:0/1396820772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb9810c140 0x7ffb9819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.106+0000 7ffb97fff640 1 -- 172.21.15.52:0/1396820772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb98103690 msgr2=0x7ffb98198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:37.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.106+0000 7ffb97fff640 1 --2- 172.21.15.52:0/1396820772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb98103690 0x7ffb98198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.106+0000 7ffb97fff640 1 -- 172.21.15.52:0/1396820772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb9810c140 msgr2=0x7ffb9819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:37.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.106+0000 7ffb97fff640 1 --2- 172.21.15.52:0/1396820772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb9810c140 0x7ffb9819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.106+0000 7ffb97fff640 1 -- 172.21.15.52:0/1396820772 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb9819d420 con 0x7ffb98102d70 2024-04-02T13:59:37.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.106+0000 7ffb97fff640 1 --2- 172.21.15.52:0/1396820772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb98102d70 0x7ffb98198430 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c00d770 tx=0x7ffb8c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:37.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.106+0000 7ffb957fa640 1 -- 172.21.15.52:0/1396820772 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c017070 con 0x7ffb98102d70 2024-04-02T13:59:37.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.106+0000 7ffb957fa640 1 -- 172.21.15.52:0/1396820772 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb8c005500 con 0x7ffb98102d70 2024-04-02T13:59:37.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.106+0000 7ffb9e300640 1 -- 172.21.15.52:0/1396820772 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb9819d710 con 0x7ffb98102d70 2024-04-02T13:59:37.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.106+0000 7ffb957fa640 1 -- 172.21.15.52:0/1396820772 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c004e00 con 0x7ffb98102d70 2024-04-02T13:59:37.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.106+0000 7ffb9e300640 1 -- 172.21.15.52:0/1396820772 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb981a4fa0 con 0x7ffb98102d70 2024-04-02T13:59:37.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.109+0000 7ffb957fa640 1 -- 172.21.15.52:0/1396820772 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffb8c004fa0 con 0x7ffb98102d70 2024-04-02T13:59:37.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.109+0000 7ffba0fc1740 1 Processor -- start 2024-04-02T13:59:37.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.109+0000 7ffba0fc1740 1 -- start start 2024-04-02T13:59:37.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.109+0000 7ffba0fc1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cf6e1f2c0 0x556cf6e1f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.109+0000 7ffba0fc1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556cf6e1fbe0 0x556cf6e28280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.109+0000 7ffba0fc1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e287c0 0x556cf6e2abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.109+0000 7ffba0fc1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556cf6d83770 con 0x556cf6e287c0 2024-04-02T13:59:37.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.109+0000 7ffba0fc1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556cf6d2b960 con 0x556cf6e1fbe0 2024-04-02T13:59:37.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.109+0000 7ffba0fc1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556cf6d7f8c0 con 0x556cf6e1f2c0 2024-04-02T13:59:37.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.109+0000 7ffb957fa640 1 --2- 172.21.15.52:0/1396820772 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ffb800a3da0 0x7ffb800a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.109+0000 7ffb977fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e287c0 0x556cf6e2abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.109+0000 7ffb977fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e287c0 0x556cf6e2abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52260/0 (socket says 172.21.15.52:52260) 2024-04-02T13:59:37.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.109+0000 7ffb977fe640 1 -- 172.21.15.52:0/737551074 learned_addr learned my addr 172.21.15.52:0/737551074 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:37.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.110+0000 7ffb97fff640 1 --2- 172.21.15.52:0/737551074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556cf6e1fbe0 0x556cf6e28280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.110+0000 7ffb9c89d640 1 --2- 172.21.15.52:0/737551074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cf6e1f2c0 0x556cf6e1f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.110+0000 7ffb9c89d640 1 --2- 172.21.15.52:0/1396820772 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ffb800a3da0 0x7ffb800a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.110+0000 7ffb957fa640 1 -- 172.21.15.52:0/1396820772 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7ffb8c0d0540 con 0x7ffb98102d70 2024-04-02T13:59:37.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.110+0000 7ffb977fe640 1 -- 172.21.15.52:0/737551074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cf6e1f2c0 msgr2=0x556cf6e1f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:37.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.110+0000 7ffb977fe640 1 --2- 172.21.15.52:0/737551074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cf6e1f2c0 0x556cf6e1f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.110+0000 7ffb977fe640 1 -- 172.21.15.52:0/737551074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556cf6e1fbe0 msgr2=0x556cf6e28280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:37.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.110+0000 7ffb977fe640 1 --2- 172.21.15.52:0/737551074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556cf6e1fbe0 0x556cf6e28280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.110+0000 7ffb977fe640 1 -- 172.21.15.52:0/737551074 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cf6b5f7e0 con 0x556cf6e287c0 2024-04-02T13:59:37.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.110+0000 7ffb97fff640 1 --2- 172.21.15.52:0/737551074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556cf6e1fbe0 0x556cf6e28280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:59:37.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.110+0000 7ffb977fe640 1 --2- 172.21.15.52:0/737551074 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e287c0 0x556cf6e2abb0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7ffb88009230 tx=0x7ffb88005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8565ba001d981860 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:37.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.110+0000 7ffb9c89d640 1 --2- 172.21.15.52:0/1396820772 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ffb800a3da0 0x7ffb800a6260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffb90005c40 tx=0x7ffb90014040 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:37.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.110+0000 7ffb7e7fc640 1 -- 172.21.15.52:0/737551074 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb88042070 con 0x556cf6e287c0 2024-04-02T13:59:37.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.110+0000 7ffb7e7fc640 1 -- 172.21.15.52:0/737551074 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb8802c730 con 0x556cf6e287c0 2024-04-02T13:59:37.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.110+0000 7ffb7e7fc640 1 -- 172.21.15.52:0/737551074 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb88004f10 con 0x556cf6e287c0 2024-04-02T13:59:37.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.111+0000 7ffba0fc1740 1 -- 172.21.15.52:0/737551074 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e287c0 msgr2=0x556cf6e2abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:37.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.111+0000 7ffba0fc1740 1 --2- 172.21.15.52:0/737551074 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e287c0 0x556cf6e2abb0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7ffb88009230 tx=0x7ffb88005bf0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.111+0000 7ffba0fc1740 1 -- 172.21.15.52:0/737551074 shutdown_connections 2024-04-02T13:59:37.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.111+0000 7ffba0fc1740 1 --2- 172.21.15.52:0/737551074 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e287c0 0x556cf6e2abb0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.111+0000 7ffba0fc1740 1 --2- 172.21.15.52:0/737551074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556cf6e1fbe0 0x556cf6e28280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.111+0000 7ffba0fc1740 1 --2- 172.21.15.52:0/737551074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cf6e1f2c0 0x556cf6e1f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.111+0000 7ffba0fc1740 1 -- 172.21.15.52:0/737551074 >> 172.21.15.52:0/737551074 conn(0x556cf6e199f0 msgr2=0x556cf6e19dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:37.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.111+0000 7ffba0fc1740 1 -- 172.21.15.52:0/737551074 shutdown_connections 2024-04-02T13:59:37.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.111+0000 7ffba0fc1740 1 -- 172.21.15.52:0/737551074 wait complete. 2024-04-02T13:59:37.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.111+0000 7ffba0fc1740 1 -- start start 2024-04-02T13:59:37.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.112+0000 7ffba0fc1740 1 Processor -- start 2024-04-02T13:59:37.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.112+0000 7ffba0fc1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e1f2c0 0x556cf6e25780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.112+0000 7ffba0fc1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cf6e1fbe0 0x556cf6e25cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.112+0000 7ffba0fc1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556cf6e287c0 0x556cf6e26200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.112+0000 7ffba0fc1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556cf6e2b550 con 0x556cf6e1f2c0 2024-04-02T13:59:37.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.112+0000 7ffba0fc1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556cf6d3ed20 con 0x556cf6e287c0 2024-04-02T13:59:37.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.112+0000 7ffba0fc1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556cf6e2b6d0 con 0x556cf6e1fbe0 2024-04-02T13:59:37.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.112+0000 7ffba0fc1740 10 client.0 _ll_register_callbacks cb 0x556cf6d0ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:37.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.112+0000 7ffb97fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e1f2c0 0x556cf6e25780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.112+0000 7ffb97fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e1f2c0 0x556cf6e25780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52266/0 (socket says 172.21.15.52:52266) 2024-04-02T13:59:37.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.112+0000 7ffb97fff640 1 -- 172.21.15.52:0/2993274712 learned_addr learned my addr 172.21.15.52:0/2993274712 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:37.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.112+0000 7ffb977fe640 1 --2- 172.21.15.52:0/2993274712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cf6e1fbe0 0x556cf6e25cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.112+0000 7ffb9c89d640 1 --2- 172.21.15.52:0/2993274712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556cf6e287c0 0x556cf6e26200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.112+0000 7ffb97fff640 1 -- 172.21.15.52:0/2993274712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cf6e1fbe0 msgr2=0x556cf6e25cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:37.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.112+0000 7ffb97fff640 1 --2- 172.21.15.52:0/2993274712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cf6e1fbe0 0x556cf6e25cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.112+0000 7ffb97fff640 1 -- 172.21.15.52:0/2993274712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556cf6e287c0 msgr2=0x556cf6e26200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:37.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.112+0000 7ffb97fff640 1 --2- 172.21.15.52:0/2993274712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556cf6e287c0 0x556cf6e26200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.112+0000 7ffb97fff640 1 -- 172.21.15.52:0/2993274712 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cf6bbff70 con 0x556cf6e1f2c0 2024-04-02T13:59:37.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.113+0000 7ffb97fff640 1 --2- 172.21.15.52:0/2993274712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e1f2c0 0x556cf6e25780 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0cda00 tx=0x7ffb8c066cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de85ffece4d061b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:37.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.113+0000 7ffb777fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:37.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.113+0000 7ffb777fe640 1 -- 172.21.15.52:0/2993274712 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c020040 con 0x556cf6e1f2c0 2024-04-02T13:59:37.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.113+0000 7ffb777fe640 1 -- 172.21.15.52:0/2993274712 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7ffb8c0cdbd0 con 0x556cf6e1f2c0 2024-04-02T13:59:37.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.113+0000 7ffba0fc1740 1 -- 172.21.15.52:0/2993274712 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556cf6d90cb0 con 0x556cf6e1f2c0 2024-04-02T13:59:37.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.113+0000 7ffb777fe640 1 -- 172.21.15.52:0/2993274712 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffb8c09c8a0 con 0x556cf6e1f2c0 2024-04-02T13:59:37.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.113+0000 7ffba0fc1740 10 client.17418 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:37.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.113+0000 7ffba0fc1740 1 -- 172.21.15.52:0/2993274712 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556cf6a57170 con 0x556cf6e1f2c0 2024-04-02T13:59:37.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.113+0000 7ffb777fe640 1 -- 172.21.15.52:0/2993274712 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556cf6d90cb0 con 0x556cf6e1f2c0 2024-04-02T13:59:37.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.113+0000 7ffb777fe640 1 -- 172.21.15.52:0/2993274712 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ffb8c0d0190 con 0x556cf6e1f2c0 2024-04-02T13:59:37.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.113+0000 7ffba0fc1740 10 client.17418 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:37.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.113+0000 7ffba0fc1740 10 client.17418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:59:37.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.113+0000 7ffba0fc1740 20 client.17418 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:37.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.113+0000 7ffba0fc1740 1 --2- 172.21.15.52:0/2993274712 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556cf6e38c10 0x556cf6e3b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.113+0000 7ffba0fc1740 4 client.17418 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:59:37.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.113+0000 7ffba0fc1740 1 -- 172.21.15.52:0/2993274712 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cf6d8b000 con 0x556cf6e38c10 2024-04-02T13:59:37.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.114+0000 7ffb977fe640 1 --2- 172.21.15.52:0/2993274712 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556cf6e38c10 0x556cf6e3b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.114+0000 7ffb777fe640 10 client.17418 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:37.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.114+0000 7ffb977fe640 1 --2- 172.21.15.52:0/2993274712 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556cf6e38c10 0x556cf6e3b080 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=332ab332240b1a3c server_cookie=1e72a701789a6bd1 in_seq=0 out_seq=0 2024-04-02T13:59:37.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.115+0000 7ffb777fe640 1 -- 172.21.15.52:0/2993274712 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556cf6d8b000 con 0x556cf6e38c10 2024-04-02T13:59:37.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.115+0000 7ffb777fe640 10 client.17418 handle_command_reply: tid=0 2024-04-02T13:59:37.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.115+0000 7ffba0fc1740 1 client.17418 shutdown 2024-04-02T13:59:37.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.115+0000 7ffba0fc1740 1 -- 172.21.15.52:0/2993274712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e1f2c0 msgr2=0x556cf6e25780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:37.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.115+0000 7ffba0fc1740 1 --2- 172.21.15.52:0/2993274712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e1f2c0 0x556cf6e25780 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0cda00 tx=0x7ffb8c066cf0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.115+0000 7ffba0fc1740 1 -- 172.21.15.52:0/2993274712 shutdown_connections 2024-04-02T13:59:37.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.115+0000 7ffba0fc1740 1 --2- 172.21.15.52:0/2993274712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556cf6e287c0 0x556cf6e26200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.115+0000 7ffba0fc1740 1 --2- 172.21.15.52:0/2993274712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cf6e1fbe0 0x556cf6e25cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.115+0000 7ffba0fc1740 1 --2- 172.21.15.52:0/2993274712 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556cf6e38c10 0x556cf6e3b080 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.115+0000 7ffba0fc1740 1 --2- 172.21.15.52:0/2993274712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e1f2c0 0x556cf6e25780 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.115+0000 7ffb777fe640 0 client.17418 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:37.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.115+0000 7ffba0fc1740 1 -- 172.21.15.52:0/2993274712 >> 172.21.15.52:0/2993274712 conn(0x556cf6e199f0 msgr2=0x556cf6e1b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:37.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.115+0000 7ffba0fc1740 1 -- 172.21.15.52:0/2993274712 shutdown_connections 2024-04-02T13:59:37.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.115+0000 7ffba0fc1740 1 -- 172.21.15.52:0/2993274712 wait complete. 2024-04-02T13:59:37.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.115+0000 7ffba0fc1740 20 client.17418 trim_cache size 0 max 16384 2024-04-02T13:59:37.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.133+0000 7ffba0fc1740 1 Processor -- start 2024-04-02T13:59:37.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.133+0000 7ffba0fc1740 1 -- start start 2024-04-02T13:59:37.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.133+0000 7ffba0fc1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e1f2c0 0x556cf6e27eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.133+0000 7ffba0fc1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cf6e1fbe0 0x556cf6e344a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.133+0000 7ffba0fc1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556cf6e287c0 0x556cf6e40950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.133+0000 7ffba0fc1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffb981aed10 con 0x556cf6e1f2c0 2024-04-02T13:59:37.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.133+0000 7ffba0fc1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffb981b0850 con 0x556cf6e287c0 2024-04-02T13:59:37.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.133+0000 7ffba0fc1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffb981af3e0 con 0x556cf6e1fbe0 2024-04-02T13:59:37.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.133+0000 7ffb97fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e1f2c0 0x556cf6e27eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.133+0000 7ffb97fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e1f2c0 0x556cf6e27eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52280/0 (socket says 172.21.15.52:52280) 2024-04-02T13:59:37.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.133+0000 7ffb97fff640 1 -- 172.21.15.52:0/1289630044 learned_addr learned my addr 172.21.15.52:0/1289630044 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:37.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.133+0000 7ffb977fe640 1 --2- 172.21.15.52:0/1289630044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cf6e1fbe0 0x556cf6e344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.133+0000 7ffb9c89d640 1 --2- 172.21.15.52:0/1289630044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556cf6e287c0 0x556cf6e40950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.133+0000 7ffb97fff640 1 -- 172.21.15.52:0/1289630044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cf6e1fbe0 msgr2=0x556cf6e344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:37.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.133+0000 7ffb97fff640 1 --2- 172.21.15.52:0/1289630044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cf6e1fbe0 0x556cf6e344a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.133+0000 7ffb97fff640 1 -- 172.21.15.52:0/1289630044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556cf6e287c0 msgr2=0x556cf6e40950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:37.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.133+0000 7ffb97fff640 1 --2- 172.21.15.52:0/1289630044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556cf6e287c0 0x556cf6e40950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.133+0000 7ffb97fff640 1 -- 172.21.15.52:0/1289630044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cf6e19820 con 0x556cf6e1f2c0 2024-04-02T13:59:37.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.133+0000 7ffb9c89d640 1 --2- 172.21.15.52:0/1289630044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556cf6e287c0 0x556cf6e40950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:59:37.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.134+0000 7ffb97fff640 1 --2- 172.21.15.52:0/1289630044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e1f2c0 0x556cf6e27eb0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c09f040 tx=0x7ffb8c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e6db430ceff7a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:37.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.134+0000 7ffb7dffb640 1 -- 172.21.15.52:0/1289630044 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c0ba070 con 0x556cf6e1f2c0 2024-04-02T13:59:37.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.134+0000 7ffb7dffb640 1 -- 172.21.15.52:0/1289630044 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb8c09c1a0 con 0x556cf6e1f2c0 2024-04-02T13:59:37.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.134+0000 7ffb7dffb640 1 -- 172.21.15.52:0/1289630044 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c014070 con 0x556cf6e1f2c0 2024-04-02T13:59:37.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.134+0000 7ffba0fc1740 1 -- 172.21.15.52:0/1289630044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e1f2c0 msgr2=0x556cf6e27eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:37.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.134+0000 7ffba0fc1740 1 --2- 172.21.15.52:0/1289630044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e1f2c0 0x556cf6e27eb0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c09f040 tx=0x7ffb8c009c30 comp rx=0 tx=0).stop 2024-04-02T13:59:37.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.134+0000 7ffba0fc1740 1 -- 172.21.15.52:0/1289630044 shutdown_connections 2024-04-02T13:59:37.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.134+0000 7ffba0fc1740 1 --2- 172.21.15.52:0/1289630044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556cf6e287c0 0x556cf6e40950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.134+0000 7ffba0fc1740 1 --2- 172.21.15.52:0/1289630044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cf6e1fbe0 0x556cf6e344a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.134+0000 7ffba0fc1740 1 --2- 172.21.15.52:0/1289630044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e1f2c0 0x556cf6e27eb0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.134+0000 7ffba0fc1740 1 -- 172.21.15.52:0/1289630044 >> 172.21.15.52:0/1289630044 conn(0x556cf6e199f0 msgr2=0x556cf6e18ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:37.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.134+0000 7ffba0fc1740 1 -- 172.21.15.52:0/1289630044 shutdown_connections 2024-04-02T13:59:37.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.134+0000 7ffba0fc1740 1 -- 172.21.15.52:0/1289630044 wait complete. 2024-04-02T13:59:37.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.134+0000 7ffba0fc1740 1 -- start start 2024-04-02T13:59:37.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.135+0000 7ffba0fc1740 1 Processor -- start 2024-04-02T13:59:37.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.135+0000 7ffba0fc1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556cf6e1f2c0 0x556cf6e309b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.135+0000 7ffba0fc1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e1fbe0 0x556cf6e30ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.135+0000 7ffba0fc1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cf6e287c0 0x556cf6e2d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.135+0000 7ffba0fc1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffb981b0180 con 0x556cf6e1fbe0 2024-04-02T13:59:37.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.135+0000 7ffba0fc1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffb981afab0 con 0x556cf6e1f2c0 2024-04-02T13:59:37.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.135+0000 7ffba0fc1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffb981accf0 con 0x556cf6e287c0 2024-04-02T13:59:37.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.135+0000 7ffba0fc1740 10 client.0 _ll_register_callbacks cb 0x7ffb8c00d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:37.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.135+0000 7ffb977fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e1fbe0 0x556cf6e30ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.135+0000 7ffb977fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e1fbe0 0x556cf6e30ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52290/0 (socket says 172.21.15.52:52290) 2024-04-02T13:59:37.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.135+0000 7ffb977fe640 1 -- 172.21.15.52:0/709830045 learned_addr learned my addr 172.21.15.52:0/709830045 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:37.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.135+0000 7ffb97fff640 1 --2- 172.21.15.52:0/709830045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556cf6e1f2c0 0x556cf6e309b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.135+0000 7ffb9c89d640 1 --2- 172.21.15.52:0/709830045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cf6e287c0 0x556cf6e2d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.135+0000 7ffb977fe640 1 -- 172.21.15.52:0/709830045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cf6e287c0 msgr2=0x556cf6e2d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:37.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.135+0000 7ffb977fe640 1 --2- 172.21.15.52:0/709830045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cf6e287c0 0x556cf6e2d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.135+0000 7ffb977fe640 1 -- 172.21.15.52:0/709830045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556cf6e1f2c0 msgr2=0x556cf6e309b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:37.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.135+0000 7ffb977fe640 1 --2- 172.21.15.52:0/709830045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556cf6e1f2c0 0x556cf6e309b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.135+0000 7ffb977fe640 1 -- 172.21.15.52:0/709830045 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cf6e2e0c0 con 0x556cf6e1fbe0 2024-04-02T13:59:37.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.136+0000 7ffb977fe640 1 --2- 172.21.15.52:0/709830045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e1fbe0 0x556cf6e30ef0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x556cf6e1ab80 tx=0x7ffb88009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5605d660da64f9e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:37.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.136+0000 7ffb777fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:37.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.136+0000 7ffb777fe640 1 -- 172.21.15.52:0/709830045 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb88042040 con 0x556cf6e1fbe0 2024-04-02T13:59:37.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.136+0000 7ffb777fe640 1 -- 172.21.15.52:0/709830045 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7ffb8802c920 con 0x556cf6e1fbe0 2024-04-02T13:59:37.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.136+0000 7ffba0fc1740 1 -- 172.21.15.52:0/709830045 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb8c09c1a0 con 0x556cf6e1fbe0 2024-04-02T13:59:37.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.136+0000 7ffb777fe640 1 -- 172.21.15.52:0/709830045 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffb8802cb20 con 0x556cf6e1fbe0 2024-04-02T13:59:37.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.136+0000 7ffba0fc1740 10 client.17424 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:37.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.136+0000 7ffba0fc1740 1 -- 172.21.15.52:0/709830045 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556cf6e9fbf0 con 0x556cf6e1fbe0 2024-04-02T13:59:37.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.136+0000 7ffb777fe640 1 -- 172.21.15.52:0/709830045 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb8c09c1a0 con 0x556cf6e1fbe0 2024-04-02T13:59:37.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.136+0000 7ffb777fe640 1 -- 172.21.15.52:0/709830045 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ffb8804f070 con 0x556cf6e1fbe0 2024-04-02T13:59:37.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.136+0000 7ffba0fc1740 10 client.17424 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:37.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.136+0000 7ffba0fc1740 10 client.17424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:59:37.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.136+0000 7ffba0fc1740 20 client.17424 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:37.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.136+0000 7ffba0fc1740 1 --2- 172.21.15.52:0/709830045 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556cf6e9fff0 0x556cf6ea23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.136+0000 7ffba0fc1740 4 client.17424 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:59:37.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.136+0000 7ffba0fc1740 1 -- 172.21.15.52:0/709830045 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556cf6d08be0 con 0x556cf6e9fff0 2024-04-02T13:59:37.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.137+0000 7ffb97fff640 1 --2- 172.21.15.52:0/709830045 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556cf6e9fff0 0x556cf6ea23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.137+0000 7ffb97fff640 1 --2- 172.21.15.52:0/709830045 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556cf6e9fff0 0x556cf6ea23e0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc42612862ea9d81 server_cookie=90d496f7a6b6aa25 in_seq=0 out_seq=0 2024-04-02T13:59:37.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.137+0000 7ffb777fe640 10 client.17424 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:37.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.138+0000 7ffb777fe640 1 -- 172.21.15.52:0/709830045 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556cf6d08be0 con 0x556cf6e9fff0 2024-04-02T13:59:37.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.138+0000 7ffb777fe640 10 client.17424 handle_command_reply: tid=0 2024-04-02T13:59:37.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.138+0000 7ffba0fc1740 1 client.17424 shutdown 2024-04-02T13:59:37.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.138+0000 7ffba0fc1740 1 -- 172.21.15.52:0/709830045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e1fbe0 msgr2=0x556cf6e30ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:37.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.138+0000 7ffba0fc1740 1 --2- 172.21.15.52:0/709830045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e1fbe0 0x556cf6e30ef0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x556cf6e1ab80 tx=0x7ffb88009260 comp rx=0 tx=0).stop 2024-04-02T13:59:37.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.139+0000 7ffba0fc1740 1 -- 172.21.15.52:0/709830045 shutdown_connections 2024-04-02T13:59:37.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.139+0000 7ffba0fc1740 1 --2- 172.21.15.52:0/709830045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556cf6e287c0 0x556cf6e2d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.139+0000 7ffba0fc1740 1 --2- 172.21.15.52:0/709830045 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556cf6e9fff0 0x556cf6ea23e0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.139+0000 7ffba0fc1740 1 --2- 172.21.15.52:0/709830045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556cf6e1fbe0 0x556cf6e30ef0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.139+0000 7ffba0fc1740 1 --2- 172.21.15.52:0/709830045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556cf6e1f2c0 0x556cf6e309b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.139+0000 7ffba0fc1740 1 -- 172.21.15.52:0/709830045 >> 172.21.15.52:0/709830045 conn(0x556cf6e199f0 msgr2=0x556cf6e25740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.139+0000 7ffb777fe640 0 client.17424 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.139+0000 7ffba0fc1740 1 -- 172.21.15.52:0/709830045 shutdown_connections 2024-04-02T13:59:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.139+0000 7ffba0fc1740 1 -- 172.21.15.52:0/709830045 wait complete. 2024-04-02T13:59:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.139+0000 7ffba0fc1740 20 client.17424 trim_cache size 0 max 16384 2024-04-02T13:59:37.154 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:59:37.154 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-04-02T13:59:37.154 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "d59ebd72-eab2-41d2-91a2-f97892a645b3", 2024-04-02T13:59:37.154 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-04-02T13:59:37.154 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:59:37.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.139+0000 7ffb9e300640 1 -- 172.21.15.52:0/1396820772 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ffb800a3da0 msgr2=0x7ffb800a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:37.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.139+0000 7ffb9e300640 1 --2- 172.21.15.52:0/1396820772 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ffb800a3da0 0x7ffb800a6260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffb90005c40 tx=0x7ffb90014040 comp rx=0 tx=0).stop 2024-04-02T13:59:37.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.139+0000 7ffb9e300640 1 -- 172.21.15.52:0/1396820772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb98102d70 msgr2=0x7ffb98198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:37.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.139+0000 7ffb9e300640 1 --2- 172.21.15.52:0/1396820772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb98102d70 0x7ffb98198430 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c00d770 tx=0x7ffb8c00dc30 comp rx=0 tx=0).stop 2024-04-02T13:59:37.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.139+0000 7ffb9e300640 1 -- 172.21.15.52:0/1396820772 shutdown_connections 2024-04-02T13:59:37.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.139+0000 7ffb9e300640 1 --2- 172.21.15.52:0/1396820772 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ffb800a3da0 0x7ffb800a6260 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.139+0000 7ffb9e300640 1 --2- 172.21.15.52:0/1396820772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb9810c140 0x7ffb9819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.139+0000 7ffb9e300640 1 --2- 172.21.15.52:0/1396820772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb98103690 0x7ffb98198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.139+0000 7ffb9e300640 1 --2- 172.21.15.52:0/1396820772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb98102d70 0x7ffb98198430 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.139+0000 7ffb9e300640 1 -- 172.21.15.52:0/1396820772 >> 172.21.15.52:0/1396820772 conn(0x7ffb98077b60 msgr2=0x7ffb980fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:37.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.139+0000 7ffb9e300640 1 -- 172.21.15.52:0/1396820772 shutdown_connections 2024-04-02T13:59:37.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.139+0000 7ffb9e300640 1 -- 172.21.15.52:0/1396820772 wait complete. 2024-04-02T13:59:37.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:59:37.158 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:59:37.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.265+0000 7feb9800d640 1 -- 172.21.15.52:0/2457965866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb90068d00 msgr2=0x7feb90069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:37.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.265+0000 7feb9800d640 1 --2- 172.21.15.52:0/2457965866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb90068d00 0x7feb90069180 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7feb80009930 tx=0x7feb8002bc30 comp rx=0 tx=0).stop 2024-04-02T13:59:37.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.265+0000 7feb9800d640 1 -- 172.21.15.52:0/2457965866 shutdown_connections 2024-04-02T13:59:37.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.265+0000 7feb9800d640 1 --2- 172.21.15.52:0/2457965866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb900696c0 0x7feb9010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.265+0000 7feb9800d640 1 --2- 172.21.15.52:0/2457965866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb90068d00 0x7feb90069180 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.265+0000 7feb9800d640 1 --2- 172.21.15.52:0/2457965866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb90107b50 0x7feb90107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.265+0000 7feb9800d640 1 -- 172.21.15.52:0/2457965866 >> 172.21.15.52:0/2457965866 conn(0x7feb90077b00 msgr2=0x7feb90077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:37.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.265+0000 7feb9800d640 1 -- 172.21.15.52:0/2457965866 shutdown_connections 2024-04-02T13:59:37.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.265+0000 7feb9800d640 1 -- 172.21.15.52:0/2457965866 wait complete. 2024-04-02T13:59:37.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.266+0000 7feb9800d640 1 Processor -- start 2024-04-02T13:59:37.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.266+0000 7feb9800d640 1 -- start start 2024-04-02T13:59:37.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.266+0000 7feb9800d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb90068d00 0x7feb901983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.266+0000 7feb9800d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb900696c0 0x7feb901988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.266+0000 7feb9800d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb90107b50 0x7feb9019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.266+0000 7feb9800d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feb90111ec0 con 0x7feb900696c0 2024-04-02T13:59:37.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.266+0000 7feb9800d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feb90111d40 con 0x7feb90068d00 2024-04-02T13:59:37.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.266+0000 7feb9800d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feb90112040 con 0x7feb90107b50 2024-04-02T13:59:37.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.266+0000 7feb95da9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb90068d00 0x7feb901983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.266+0000 7feb955a8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb900696c0 0x7feb901988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.267+0000 7feb965aa640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb90107b50 0x7feb9019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.267+0000 7feb95da9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb90068d00 0x7feb901983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:57338/0 (socket says 172.21.15.52:57338) 2024-04-02T13:59:37.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.267+0000 7feb95da9640 1 -- 172.21.15.52:0/3461986790 learned_addr learned my addr 172.21.15.52:0/3461986790 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:37.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.267+0000 7feb955a8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb900696c0 0x7feb901988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52304/0 (socket says 172.21.15.52:52304) 2024-04-02T13:59:37.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.267+0000 7feb95da9640 1 -- 172.21.15.52:0/3461986790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb90107b50 msgr2=0x7feb9019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:37.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.267+0000 7feb95da9640 1 --2- 172.21.15.52:0/3461986790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb90107b50 0x7feb9019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.267+0000 7feb95da9640 1 -- 172.21.15.52:0/3461986790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb900696c0 msgr2=0x7feb901988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:37.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.267+0000 7feb95da9640 1 --2- 172.21.15.52:0/3461986790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb900696c0 0x7feb901988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.267+0000 7feb95da9640 1 -- 172.21.15.52:0/3461986790 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9019d310 con 0x7feb90068d00 2024-04-02T13:59:37.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.267+0000 7feb955a8640 1 --2- 172.21.15.52:0/3461986790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb900696c0 0x7feb901988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:59:37.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.267+0000 7feb95da9640 1 --2- 172.21.15.52:0/3461986790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb90068d00 0x7feb901983a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00b550 tx=0x7feb8c00ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:37.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.267+0000 7feb86ffd640 1 -- 172.21.15.52:0/3461986790 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c004020 con 0x7feb90068d00 2024-04-02T13:59:37.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.267+0000 7feb9800d640 1 -- 172.21.15.52:0/3461986790 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb9019d600 con 0x7feb90068d00 2024-04-02T13:59:37.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.267+0000 7feb86ffd640 1 -- 172.21.15.52:0/3461986790 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb8c0041c0 con 0x7feb90068d00 2024-04-02T13:59:37.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.267+0000 7feb86ffd640 1 -- 172.21.15.52:0/3461986790 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c00a910 con 0x7feb90068d00 2024-04-02T13:59:37.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.267+0000 7feb9800d640 1 -- 172.21.15.52:0/3461986790 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb901008f0 con 0x7feb90068d00 2024-04-02T13:59:37.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.270+0000 7feb9abf9740 1 Processor -- start 2024-04-02T13:59:37.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.270+0000 7feb9abf9740 1 -- start start 2024-04-02T13:59:37.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.270+0000 7feb86ffd640 1 -- 172.21.15.52:0/3461986790 <== mon.2 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feb8c00aaf0 con 0x7feb90068d00 2024-04-02T13:59:37.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.270+0000 7feb9abf9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b1385b8d0 0x564b1385bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.270+0000 7feb9abf9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b1385c1f0 0x564b13864890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.270+0000 7feb9abf9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b13864dd0 0x564b138671c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.270+0000 7feb9abf9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564b137bf830 con 0x564b1385b8d0 2024-04-02T13:59:37.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.270+0000 7feb9abf9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564b13767820 con 0x564b13864dd0 2024-04-02T13:59:37.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.270+0000 7feb9abf9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564b137bb7e0 con 0x564b1385c1f0 2024-04-02T13:59:37.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.270+0000 7feb965aa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b1385b8d0 0x564b1385bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.270+0000 7feb965aa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b1385b8d0 0x564b1385bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52314/0 (socket says 172.21.15.52:52314) 2024-04-02T13:59:37.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.271+0000 7feb965aa640 1 -- 172.21.15.52:0/1558239611 learned_addr learned my addr 172.21.15.52:0/1558239611 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:37.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.271+0000 7feb95da9640 1 --2- 172.21.15.52:0/1558239611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b1385c1f0 0x564b13864890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.271+0000 7feb955a8640 1 --2- 172.21.15.52:0/1558239611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b13864dd0 0x564b138671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.271+0000 7feb965aa640 1 -- 172.21.15.52:0/1558239611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b1385c1f0 msgr2=0x564b13864890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:37.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.271+0000 7feb965aa640 1 --2- 172.21.15.52:0/1558239611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b1385c1f0 0x564b13864890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.271+0000 7feb965aa640 1 -- 172.21.15.52:0/1558239611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b13864dd0 msgr2=0x564b138671c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:37.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.271+0000 7feb965aa640 1 --2- 172.21.15.52:0/1558239611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b13864dd0 0x564b138671c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.271+0000 7feb965aa640 1 -- 172.21.15.52:0/1558239611 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b1359d7d0 con 0x564b1385b8d0 2024-04-02T13:59:37.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.271+0000 7feb86ffd640 1 --2- 172.21.15.52:0/3461986790 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7feb7c0a3d20 0x7feb7c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.271+0000 7feb965aa640 1 --2- 172.21.15.52:0/1558239611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b1385b8d0 0x564b1385bcb0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7feb88009fd0 tx=0x7feb880118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90956bf8c17400d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:37.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.271+0000 7feb95da9640 1 --2- 172.21.15.52:0/3461986790 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7feb7c0a3d20 0x7feb7c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.271+0000 7feb7bfff640 1 -- 172.21.15.52:0/1558239611 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88004d20 con 0x564b1385b8d0 2024-04-02T13:59:37.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.271+0000 7feb7bfff640 1 -- 172.21.15.52:0/1558239611 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb88004050 con 0x564b1385b8d0 2024-04-02T13:59:37.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.271+0000 7feb7bfff640 1 -- 172.21.15.52:0/1558239611 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88005240 con 0x564b1385b8d0 2024-04-02T13:59:37.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.271+0000 7feb86ffd640 1 -- 172.21.15.52:0/3461986790 <== mon.2 v2:172.21.15.88:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7feb8c0d88e0 con 0x7feb90068d00 2024-04-02T13:59:37.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.272+0000 7feb9abf9740 1 -- 172.21.15.52:0/1558239611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b1385b8d0 msgr2=0x564b1385bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:37.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.272+0000 7feb95da9640 1 --2- 172.21.15.52:0/3461986790 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7feb7c0a3d20 0x7feb7c0a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7feb8c0d5da0 tx=0x7feb8c08f040 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:37.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.272+0000 7feb9abf9740 1 --2- 172.21.15.52:0/1558239611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b1385b8d0 0x564b1385bcb0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7feb88009fd0 tx=0x7feb880118d0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.272+0000 7feb9abf9740 1 -- 172.21.15.52:0/1558239611 shutdown_connections 2024-04-02T13:59:37.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.272+0000 7feb9abf9740 1 --2- 172.21.15.52:0/1558239611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b13864dd0 0x564b138671c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.272+0000 7feb9abf9740 1 --2- 172.21.15.52:0/1558239611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b1385c1f0 0x564b13864890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.272+0000 7feb9abf9740 1 --2- 172.21.15.52:0/1558239611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b1385b8d0 0x564b1385bcb0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.272+0000 7feb9abf9740 1 -- 172.21.15.52:0/1558239611 >> 172.21.15.52:0/1558239611 conn(0x564b13856000 msgr2=0x564b138563e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:37.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.272+0000 7feb9abf9740 1 -- 172.21.15.52:0/1558239611 shutdown_connections 2024-04-02T13:59:37.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.272+0000 7feb9abf9740 1 -- 172.21.15.52:0/1558239611 wait complete. 2024-04-02T13:59:37.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.273+0000 7feb9abf9740 1 -- start start 2024-04-02T13:59:37.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.273+0000 7feb9abf9740 1 Processor -- start 2024-04-02T13:59:37.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.273+0000 7feb9abf9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b1385b8d0 0x564b13864690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.273+0000 7feb9abf9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b1385c1f0 0x564b13861a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.273+0000 7feb9abf9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b13864dd0 0x564b13861f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.273+0000 7feb9abf9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564b13867bc0 con 0x564b13864dd0 2024-04-02T13:59:37.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.273+0000 7feb9abf9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564b1377ad30 con 0x564b1385c1f0 2024-04-02T13:59:37.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.273+0000 7feb9abf9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564b13867d40 con 0x564b1385b8d0 2024-04-02T13:59:37.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.273+0000 7feb9abf9740 10 client.0 _ll_register_callbacks cb 0x564b1377a720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:37.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.274+0000 7feb955a8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b1385c1f0 0x564b13861a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.274+0000 7feb965aa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b13864dd0 0x564b13861f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.274+0000 7feb955a8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b1385c1f0 0x564b13861a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:57364/0 (socket says 172.21.15.52:57364) 2024-04-02T13:59:37.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.274+0000 7feb965aa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b13864dd0 0x564b13861f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52326/0 (socket says 172.21.15.52:52326) 2024-04-02T13:59:37.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.274+0000 7feb955a8640 1 -- 172.21.15.52:0/3898772888 learned_addr learned my addr 172.21.15.52:0/3898772888 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:37.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.274+0000 7feb965aa640 1 --2- 172.21.15.52:0/3898772888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b1385b8d0 0x564b13864690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.274+0000 7feb955a8640 1 -- 172.21.15.52:0/3898772888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b1385b8d0 msgr2=0x564b13864690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:37.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.274+0000 7feb955a8640 1 --2- 172.21.15.52:0/3898772888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b1385b8d0 0x564b13864690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.274+0000 7feb955a8640 1 -- 172.21.15.52:0/3898772888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b13864dd0 msgr2=0x564b13861f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:37.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.274+0000 7feb955a8640 1 --2- 172.21.15.52:0/3898772888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b13864dd0 0x564b13861f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.274+0000 7feb955a8640 1 -- 172.21.15.52:0/3898772888 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b134d33f0 con 0x564b1385c1f0 2024-04-02T13:59:37.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.274+0000 7feb965aa640 1 --2- 172.21.15.52:0/3898772888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b13864dd0 0x564b13861f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:59:37.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.274+0000 7feb955a8640 1 --2- 172.21.15.52:0/3898772888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b1385c1f0 0x564b13861a20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7feb8002c240 tx=0x7feb8002c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3549cf45ac677632 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:37.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.274+0000 7feb797fa640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-02T13:59:37.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.274+0000 7feb797fa640 1 -- 172.21.15.52:0/3898772888 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8002c8b0 con 0x564b1385c1f0 2024-04-02T13:59:37.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.275+0000 7feb797fa640 1 -- 172.21.15.52:0/3898772888 <== mon.2 v2:172.21.15.88:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7feb8002ca50 con 0x564b1385c1f0 2024-04-02T13:59:37.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.275+0000 7feb9abf9740 1 -- 172.21.15.52:0/3898772888 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb88004050 con 0x564b1385c1f0 2024-04-02T13:59:37.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.275+0000 7feb797fa640 1 -- 172.21.15.52:0/3898772888 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feb8002cc50 con 0x564b1385c1f0 2024-04-02T13:59:37.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.275+0000 7feb9abf9740 10 client.26654 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:37.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.275+0000 7feb9abf9740 1 -- 172.21.15.52:0/3898772888 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564b134930f0 con 0x564b1385c1f0 2024-04-02T13:59:37.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.275+0000 7feb797fa640 1 -- 172.21.15.52:0/3898772888 <== mon.2 v2:172.21.15.88:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb88004050 con 0x564b1385c1f0 2024-04-02T13:59:37.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.275+0000 7feb797fa640 1 -- 172.21.15.52:0/3898772888 <== mon.2 v2:172.21.15.88:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7feb80046070 con 0x564b1385c1f0 2024-04-02T13:59:37.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.275+0000 7feb9abf9740 10 client.26654 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:37.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.275+0000 7feb9abf9740 10 client.26654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:59:37.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.275+0000 7feb9abf9740 20 client.26654 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:37.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.275+0000 7feb9abf9740 1 --2- 172.21.15.52:0/3898772888 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564b13874d20 0x564b13877130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.275+0000 7feb9abf9740 4 client.26654 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:59:37.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.275+0000 7feb9abf9740 1 -- 172.21.15.52:0/3898772888 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b137ccd70 con 0x564b13874d20 2024-04-02T13:59:37.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.275+0000 7feb965aa640 1 --2- 172.21.15.52:0/3898772888 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564b13874d20 0x564b13877130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.276+0000 7feb965aa640 1 --2- 172.21.15.52:0/3898772888 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564b13874d20 0x564b13877130 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bd3342fdec2668a server_cookie=a31649ab8c26c28d in_seq=0 out_seq=0 2024-04-02T13:59:37.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.276+0000 7feb797fa640 10 client.26654 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:37.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.277+0000 7feb797fa640 1 -- 172.21.15.52:0/3898772888 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564b137ccd70 con 0x564b13874d20 2024-04-02T13:59:37.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.277+0000 7feb797fa640 10 client.26654 handle_command_reply: tid=0 2024-04-02T13:59:37.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.277+0000 7feb9abf9740 1 client.26654 shutdown 2024-04-02T13:59:37.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.277+0000 7feb9abf9740 1 -- 172.21.15.52:0/3898772888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b1385c1f0 msgr2=0x564b13861a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:37.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.277+0000 7feb9abf9740 1 --2- 172.21.15.52:0/3898772888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b1385c1f0 0x564b13861a20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7feb8002c240 tx=0x7feb8002c800 comp rx=0 tx=0).stop 2024-04-02T13:59:37.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.277+0000 7feb9abf9740 1 -- 172.21.15.52:0/3898772888 shutdown_connections 2024-04-02T13:59:37.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.277+0000 7feb9abf9740 1 --2- 172.21.15.52:0/3898772888 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564b13874d20 0x564b13877130 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.277+0000 7feb9abf9740 1 --2- 172.21.15.52:0/3898772888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b13864dd0 0x564b13861f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.277+0000 7feb9abf9740 1 --2- 172.21.15.52:0/3898772888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b1385c1f0 0x564b13861a20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.277+0000 7feb9abf9740 1 --2- 172.21.15.52:0/3898772888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b1385b8d0 0x564b13864690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.277+0000 7feb9abf9740 1 -- 172.21.15.52:0/3898772888 >> 172.21.15.52:0/3898772888 conn(0x564b13856000 msgr2=0x564b13857d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:37.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.277+0000 7feb797fa640 0 client.26654 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:37.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.277+0000 7feb9abf9740 1 -- 172.21.15.52:0/3898772888 shutdown_connections 2024-04-02T13:59:37.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.278+0000 7feb9abf9740 1 -- 172.21.15.52:0/3898772888 wait complete. 2024-04-02T13:59:37.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.278+0000 7feb9abf9740 20 client.26654 trim_cache size 0 max 16384 2024-04-02T13:59:37.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.297+0000 7feb9abf9740 1 Processor -- start 2024-04-02T13:59:37.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.297+0000 7feb9abf9740 1 -- start start 2024-04-02T13:59:37.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.297+0000 7feb9abf9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b13856000 0x564b13870a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.297+0000 7feb9abf9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b1385b8d0 0x564b13860310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.297+0000 7feb9abf9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b1385c1f0 0x564b13864150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.298+0000 7feb9abf9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feb901acb90 con 0x564b1385c1f0 2024-04-02T13:59:37.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.298+0000 7feb9abf9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feb901ae6d0 con 0x564b1385b8d0 2024-04-02T13:59:37.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.298+0000 7feb9abf9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feb901ad260 con 0x564b13856000 2024-04-02T13:59:37.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.298+0000 7feb965aa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b1385c1f0 0x564b13864150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.298+0000 7feb955a8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b1385b8d0 0x564b13860310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.298+0000 7feb965aa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b1385c1f0 0x564b13864150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52338/0 (socket says 172.21.15.52:52338) 2024-04-02T13:59:37.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.298+0000 7feb965aa640 1 -- 172.21.15.52:0/1913067879 learned_addr learned my addr 172.21.15.52:0/1913067879 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:37.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.298+0000 7feb965aa640 1 --2- 172.21.15.52:0/1913067879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b13856000 0x564b13870a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.298+0000 7feb965aa640 1 -- 172.21.15.52:0/1913067879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b13856000 msgr2=0x564b13870a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:37.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.298+0000 7feb965aa640 1 --2- 172.21.15.52:0/1913067879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b13856000 0x564b13870a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.298+0000 7feb965aa640 1 -- 172.21.15.52:0/1913067879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b1385b8d0 msgr2=0x564b13860310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:37.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.298+0000 7feb965aa640 1 --2- 172.21.15.52:0/1913067879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b1385b8d0 0x564b13860310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.298+0000 7feb965aa640 1 -- 172.21.15.52:0/1913067879 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b13855e30 con 0x564b1385c1f0 2024-04-02T13:59:37.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.298+0000 7feb965aa640 1 --2- 172.21.15.52:0/1913067879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b1385c1f0 0x564b13864150 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7feb88005c40 tx=0x7feb880118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82af641f86f16133 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:37.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.298+0000 7feb7b7fe640 1 -- 172.21.15.52:0/1913067879 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88004320 con 0x564b1385c1f0 2024-04-02T13:59:37.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.298+0000 7feb7b7fe640 1 -- 172.21.15.52:0/1913067879 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb8800f4e0 con 0x564b1385c1f0 2024-04-02T13:59:37.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.298+0000 7feb7b7fe640 1 -- 172.21.15.52:0/1913067879 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88019670 con 0x564b1385c1f0 2024-04-02T13:59:37.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.299+0000 7feb9abf9740 1 -- 172.21.15.52:0/1913067879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b1385c1f0 msgr2=0x564b13864150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:37.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.299+0000 7feb9abf9740 1 --2- 172.21.15.52:0/1913067879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b1385c1f0 0x564b13864150 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7feb88005c40 tx=0x7feb880118d0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.299+0000 7feb9abf9740 1 -- 172.21.15.52:0/1913067879 shutdown_connections 2024-04-02T13:59:37.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.299+0000 7feb9abf9740 1 --2- 172.21.15.52:0/1913067879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b1385c1f0 0x564b13864150 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.299+0000 7feb9abf9740 1 --2- 172.21.15.52:0/1913067879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b1385b8d0 0x564b13860310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.299+0000 7feb9abf9740 1 --2- 172.21.15.52:0/1913067879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b13856000 0x564b13870a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.299+0000 7feb9abf9740 1 -- 172.21.15.52:0/1913067879 >> 172.21.15.52:0/1913067879 conn(0x564b138620c0 msgr2=0x564b1385d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:37.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.299+0000 7feb9abf9740 1 -- 172.21.15.52:0/1913067879 shutdown_connections 2024-04-02T13:59:37.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.299+0000 7feb9abf9740 1 -- 172.21.15.52:0/1913067879 wait complete. 2024-04-02T13:59:37.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.300+0000 7feb9abf9740 1 -- start start 2024-04-02T13:59:37.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.300+0000 7feb9abf9740 1 Processor -- start 2024-04-02T13:59:37.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.300+0000 7feb9abf9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b13856000 0x564b1385e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.300+0000 7feb9abf9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b1385b8d0 0x564b1385ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.300+0000 7feb9abf9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b1385c1f0 0x564b1385f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.300+0000 7feb9abf9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feb901ae000 con 0x564b1385b8d0 2024-04-02T13:59:37.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.300+0000 7feb9abf9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feb901ad930 con 0x564b1385c1f0 2024-04-02T13:59:37.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.300+0000 7feb9abf9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feb901aab70 con 0x564b13856000 2024-04-02T13:59:37.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.300+0000 7feb9abf9740 10 client.0 _ll_register_callbacks cb 0x7feb80005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:59:37.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.300+0000 7feb955a8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b1385b8d0 0x564b1385ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.300+0000 7feb965aa640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b13856000 0x564b1385e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.300+0000 7feb955a8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b1385b8d0 0x564b1385ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52352/0 (socket says 172.21.15.52:52352) 2024-04-02T13:59:37.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.300+0000 7feb955a8640 1 -- 172.21.15.52:0/1773709553 learned_addr learned my addr 172.21.15.52:0/1773709553 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T13:59:37.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.300+0000 7feb965aa640 1 --2- 172.21.15.52:0/1773709553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b1385c1f0 0x564b1385f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.301+0000 7feb955a8640 1 -- 172.21.15.52:0/1773709553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b13856000 msgr2=0x564b1385e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:37.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.301+0000 7feb955a8640 1 --2- 172.21.15.52:0/1773709553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b13856000 0x564b1385e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.301+0000 7feb955a8640 1 -- 172.21.15.52:0/1773709553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b1385c1f0 msgr2=0x564b1385f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:59:37.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.301+0000 7feb955a8640 1 --2- 172.21.15.52:0/1773709553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b1385c1f0 0x564b1385f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.301+0000 7feb955a8640 1 -- 172.21.15.52:0/1773709553 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b138da560 con 0x564b1385b8d0 2024-04-02T13:59:37.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.301+0000 7feb955a8640 1 --2- 172.21.15.52:0/1773709553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b1385b8d0 0x564b1385ef30 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x564b13862ff0 tx=0x7feb80005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26ee3b05294ed55 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:59:37.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.301+0000 7feb797fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T13:59:37.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.301+0000 7feb797fa640 1 -- 172.21.15.52:0/1773709553 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80009520 con 0x564b1385b8d0 2024-04-02T13:59:37.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.301+0000 7feb797fa640 1 -- 172.21.15.52:0/1773709553 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7feb80003c20 con 0x564b1385b8d0 2024-04-02T13:59:37.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.301+0000 7feb9abf9740 1 -- 172.21.15.52:0/1773709553 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb8800f4e0 con 0x564b1385b8d0 2024-04-02T13:59:37.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.301+0000 7feb797fa640 1 -- 172.21.15.52:0/1773709553 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feb8005e050 con 0x564b1385b8d0 2024-04-02T13:59:37.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.301+0000 7feb9abf9740 10 client.17442 fetch_fsmap learned FSMap version 30 2024-04-02T13:59:37.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.301+0000 7feb9abf9740 1 -- 172.21.15.52:0/1773709553 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564b138db870 con 0x564b1385b8d0 2024-04-02T13:59:37.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.302+0000 7feb797fa640 1 -- 172.21.15.52:0/1773709553 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb8800f4e0 con 0x564b1385b8d0 2024-04-02T13:59:37.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.302+0000 7feb797fa640 1 -- 172.21.15.52:0/1773709553 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7feb80046040 con 0x564b1385b8d0 2024-04-02T13:59:37.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.302+0000 7feb9abf9740 10 client.17442 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:59:37.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.302+0000 7feb9abf9740 10 client.17442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:59:37.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.302+0000 7feb9abf9740 20 client.17442 populate_metadata read hostname 'smithi052' 2024-04-02T13:59:37.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.302+0000 7feb9abf9740 1 --2- 172.21.15.52:0/1773709553 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564b13864dd0 0x564b138ddfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:59:37.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.302+0000 7feb9abf9740 4 client.17442 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:59:37.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.302+0000 7feb9abf9740 1 -- 172.21.15.52:0/1773709553 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564b137c70c0 con 0x564b13864dd0 2024-04-02T13:59:37.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.302+0000 7feb965aa640 1 --2- 172.21.15.52:0/1773709553 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564b13864dd0 0x564b138ddfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:59:37.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.303+0000 7feb797fa640 10 client.17442 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:37.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.303+0000 7feb965aa640 1 --2- 172.21.15.52:0/1773709553 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564b13864dd0 0x564b138ddfa0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbb87306a631a95d server_cookie=dbbbd371a181e6d6 in_seq=0 out_seq=0 2024-04-02T13:59:37.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.303+0000 7feb797fa640 1 -- 172.21.15.52:0/1773709553 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x564b137c70c0 con 0x564b13864dd0 2024-04-02T13:59:37.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.303+0000 7feb797fa640 10 client.17442 handle_command_reply: tid=0 2024-04-02T13:59:37.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.303+0000 7feb9abf9740 1 client.17442 shutdown 2024-04-02T13:59:37.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.303+0000 7feb9abf9740 1 -- 172.21.15.52:0/1773709553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b1385b8d0 msgr2=0x564b1385ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:37.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.303+0000 7feb9abf9740 1 --2- 172.21.15.52:0/1773709553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b1385b8d0 0x564b1385ef30 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x564b13862ff0 tx=0x7feb80005e70 comp rx=0 tx=0).stop 2024-04-02T13:59:37.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.303+0000 7feb9abf9740 1 -- 172.21.15.52:0/1773709553 shutdown_connections 2024-04-02T13:59:37.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.303+0000 7feb9abf9740 1 --2- 172.21.15.52:0/1773709553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564b1385c1f0 0x564b1385f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.303+0000 7feb9abf9740 1 --2- 172.21.15.52:0/1773709553 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564b13864dd0 0x564b138ddfa0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.303+0000 7feb797fa640 0 client.17442 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T13:59:37.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.303+0000 7feb9abf9740 1 --2- 172.21.15.52:0/1773709553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b1385b8d0 0x564b1385ef30 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.303+0000 7feb9abf9740 1 --2- 172.21.15.52:0/1773709553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b13856000 0x564b1385e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.303+0000 7feb9abf9740 1 -- 172.21.15.52:0/1773709553 >> 172.21.15.52:0/1773709553 conn(0x564b138620c0 msgr2=0x564b138549b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:37.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.303+0000 7feb9abf9740 1 -- 172.21.15.52:0/1773709553 shutdown_connections 2024-04-02T13:59:37.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.304+0000 7feb9abf9740 1 -- 172.21.15.52:0/1773709553 wait complete. 2024-04-02T13:59:37.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.304+0000 7feb9abf9740 20 client.17442 trim_cache size 0 max 16384 2024-04-02T13:59:37.316 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T13:59:37.316 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "scrub active (2421 inodes in the stack)", 2024-04-02T13:59:37.317 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-04-02T13:59:37.317 INFO:teuthology.orchestra.run.smithi052.stdout: "d59ebd72-eab2-41d2-91a2-f97892a645b3": { 2024-04-02T13:59:37.317 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-04-02T13:59:37.317 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "d59ebd72-eab2-41d2-91a2-f97892a645b3", 2024-04-02T13:59:37.317 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-04-02T13:59:37.317 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:59:37.317 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T13:59:37.317 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T13:59:37.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.304+0000 7feb9800d640 1 -- 172.21.15.52:0/3461986790 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7feb7c0a3d20 msgr2=0x7feb7c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:37.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.304+0000 7feb9800d640 1 --2- 172.21.15.52:0/3461986790 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7feb7c0a3d20 0x7feb7c0a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7feb8c0d5da0 tx=0x7feb8c08f040 comp rx=0 tx=0).stop 2024-04-02T13:59:37.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.304+0000 7feb9800d640 1 -- 172.21.15.52:0/3461986790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb90068d00 msgr2=0x7feb901983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:59:37.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.304+0000 7feb9800d640 1 --2- 172.21.15.52:0/3461986790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb90068d00 0x7feb901983a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00b550 tx=0x7feb8c00ba10 comp rx=0 tx=0).stop 2024-04-02T13:59:37.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.304+0000 7feb9800d640 1 -- 172.21.15.52:0/3461986790 shutdown_connections 2024-04-02T13:59:37.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.304+0000 7feb9800d640 1 --2- 172.21.15.52:0/3461986790 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7feb7c0a3d20 0x7feb7c0a61e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.304+0000 7feb9800d640 1 --2- 172.21.15.52:0/3461986790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb90107b50 0x7feb9019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.304+0000 7feb9800d640 1 --2- 172.21.15.52:0/3461986790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb900696c0 0x7feb901988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.304+0000 7feb9800d640 1 --2- 172.21.15.52:0/3461986790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb90068d00 0x7feb901983a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:59:37.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.304+0000 7feb9800d640 1 -- 172.21.15.52:0/3461986790 >> 172.21.15.52:0/3461986790 conn(0x7feb90077b00 msgr2=0x7feb900fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:59:37.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.304+0000 7feb9800d640 1 -- 172.21.15.52:0/3461986790 shutdown_connections 2024-04-02T13:59:37.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T13:59:37.304+0000 7feb9800d640 1 -- 172.21.15.52:0/3461986790 wait complete. 2024-04-02T13:59:37.321 INFO:tasks.cephfs.filesystem:scrub status for tag:d59ebd72-eab2-41d2-91a2-f97892a645b3 - {'path': '/', 'tag': 'd59ebd72-eab2-41d2-91a2-f97892a645b3', 'options': 'recursive,force'} 2024-04-02T13:59:37.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:37 smithi052 ceph-mon[32104]: pgmap v186: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:59:37.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:37 smithi052 ceph-mon[32104]: scrub queued for path: / 2024-04-02T13:59:37.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:37 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:59:37.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:37 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:59:37.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:37 smithi189 ceph-mon[36683]: pgmap v186: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:59:37.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:37 smithi189 ceph-mon[36683]: scrub queued for path: / 2024-04-02T13:59:37.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:37 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:59:37.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:37 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:59:37.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:37 smithi088 ceph-mon[36528]: pgmap v186: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:59:37.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:37 smithi088 ceph-mon[36528]: scrub queued for path: / 2024-04-02T13:59:37.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:37 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:59:37.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:37 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:59:38.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:38 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:59:38.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:38 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:59:38.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:38 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T13:59:38.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:38 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:59:38.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:38 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T13:59:38.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:38 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:59:38.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:38 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:59:38.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:38 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T13:59:38.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:38 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:59:38.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:38 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T13:59:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:38 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:59:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:38 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:59:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:38 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T13:59:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:38 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:59:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:38 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T13:59:39.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:39 smithi189 ceph-mon[36683]: pgmap v187: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:39.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:39 smithi052 ceph-mon[32104]: pgmap v187: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:39.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:39 smithi088 ceph-mon[36528]: pgmap v187: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:40.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:40 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:59:40.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:40 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:59:40.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:40 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:59:40.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:40 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:59:40.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:40 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:59:40.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:40 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:59:40.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:40 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:59:40.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:40 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:59:40.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:40 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:59:40.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:40 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:59:40.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:40 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:59:40.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:40 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T13:59:41.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:41 smithi189 ceph-mon[36683]: pgmap v188: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:41.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:41 smithi052 ceph-mon[32104]: pgmap v188: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:41.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:41 smithi088 ceph-mon[36528]: pgmap v188: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:43.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:43 smithi189 ceph-mon[36683]: pgmap v189: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:59:43.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:43 smithi052 ceph-mon[32104]: pgmap v189: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:59:43.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:43 smithi088 ceph-mon[36528]: pgmap v189: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:59:45.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:45 smithi189 ceph-mon[36683]: pgmap v190: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:59:45.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:45 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T13:59:45.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:45 smithi052 ceph-mon[32104]: pgmap v190: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:59:45.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:45 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T13:59:45.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:45 smithi088 ceph-mon[36528]: pgmap v190: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:59:45.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:45 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T13:59:46.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:46 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T13:59:46.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:46 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T13:59:46.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:46 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T13:59:47.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:47 smithi189 ceph-mon[36683]: pgmap v191: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:59:47.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:47 smithi052 ceph-mon[32104]: pgmap v191: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:59:47.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:47 smithi088 ceph-mon[36528]: pgmap v191: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:59:49.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:49 smithi189 ceph-mon[36683]: pgmap v192: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:59:49.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:49 smithi052 ceph-mon[32104]: pgmap v192: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:59:49.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:49 smithi088 ceph-mon[36528]: pgmap v192: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:59:51.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:51 smithi189 ceph-mon[36683]: pgmap v193: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:59:51.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:51 smithi052 ceph-mon[32104]: pgmap v193: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:59:51.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:51 smithi088 ceph-mon[36528]: pgmap v193: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T13:59:53.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:53 smithi189 ceph-mon[36683]: pgmap v194: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:59:53.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:53 smithi052 ceph-mon[32104]: pgmap v194: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:59:53.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:53 smithi088 ceph-mon[36528]: pgmap v194: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T13:59:55.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:55 smithi189 ceph-mon[36683]: pgmap v195: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:55.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:55 smithi052 ceph-mon[32104]: pgmap v195: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:55 smithi088 ceph-mon[36528]: pgmap v195: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:57.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:57 smithi189 ceph-mon[36683]: pgmap v196: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:59:57.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:57 smithi052 ceph-mon[32104]: pgmap v196: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:59:57.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:57 smithi088 ceph-mon[36528]: pgmap v196: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:59:59.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 13:59:59 smithi189 ceph-mon[36683]: pgmap v197: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:59.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 13:59:59 smithi052 ceph-mon[32104]: pgmap v197: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:59:59.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 13:59:59 smithi088 ceph-mon[36528]: pgmap v197: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:00:00.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:00 smithi189 ceph-mon[36683]: overall HEALTH_OK 2024-04-02T14:00:00.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:00 smithi052 ceph-mon[32104]: overall HEALTH_OK 2024-04-02T14:00:00.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:00 smithi088 ceph-mon[36528]: overall HEALTH_OK 2024-04-02T14:00:01.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:01 smithi189 ceph-mon[36683]: pgmap v198: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:00:01.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:01 smithi052 ceph-mon[32104]: pgmap v198: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:00:01.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:01 smithi088 ceph-mon[36528]: pgmap v198: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:00:03.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:03 smithi189 ceph-mon[36683]: pgmap v199: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-02T14:00:03.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:03 smithi052 ceph-mon[32104]: pgmap v199: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-02T14:00:03.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:03 smithi088 ceph-mon[36528]: pgmap v199: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-02T14:00:05.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:05 smithi189 ceph-mon[36683]: pgmap v200: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-02T14:00:05.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:05 smithi052 ceph-mon[32104]: pgmap v200: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-02T14:00:05.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:05 smithi088 ceph-mon[36528]: pgmap v200: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-02T14:00:07.322 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:00:07.323 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:00:07.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.439+0000 7f401a567640 1 -- 172.21.15.52:0/143682379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4014101330 msgr2=0x7f401410d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:07.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.439+0000 7f401a567640 1 --2- 172.21.15.52:0/143682379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4014101330 0x7f401410d840 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f400c009cd0 tx=0x7f400c02c0f0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.439+0000 7f401a567640 1 -- 172.21.15.52:0/143682379 shutdown_connections 2024-04-02T14:00:07.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.439+0000 7f401a567640 1 --2- 172.21.15.52:0/143682379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4014101330 0x7f401410d840 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.439+0000 7f401a567640 1 --2- 172.21.15.52:0/143682379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4014100970 0x7f4014100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.439+0000 7f401a567640 1 --2- 172.21.15.52:0/143682379 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4014106980 0x7f4014106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.439+0000 7f401a567640 1 -- 172.21.15.52:0/143682379 >> 172.21.15.52:0/143682379 conn(0x7f4014077b00 msgr2=0x7f4014077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:07.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.439+0000 7f401a567640 1 -- 172.21.15.52:0/143682379 shutdown_connections 2024-04-02T14:00:07.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.439+0000 7f401a567640 1 -- 172.21.15.52:0/143682379 wait complete. 2024-04-02T14:00:07.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.440+0000 7f401a567640 1 Processor -- start 2024-04-02T14:00:07.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.440+0000 7f401a567640 1 -- start start 2024-04-02T14:00:07.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.440+0000 7f401a567640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4014100970 0x7f4014199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.440+0000 7f401a567640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4014101330 0x7f40141996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.440+0000 7f401a567640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4014106980 0x7f40141a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.440+0000 7f401a567640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4014110d90 con 0x7f4014100970 2024-04-02T14:00:07.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.440+0000 7f401a567640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4014110c10 con 0x7f4014106980 2024-04-02T14:00:07.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.440+0000 7f401a567640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4014110f10 con 0x7f4014101330 2024-04-02T14:00:07.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.440+0000 7f4013fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4014100970 0x7f4014199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.440+0000 7f4013fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4014100970 0x7f4014199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34312/0 (socket says 172.21.15.52:34312) 2024-04-02T14:00:07.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.440+0000 7f4013fff640 1 -- 172.21.15.52:0/989564966 learned_addr learned my addr 172.21.15.52:0/989564966 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:07.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.440+0000 7f4018b04640 1 --2- 172.21.15.52:0/989564966 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4014106980 0x7f40141a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.440+0000 7f40137fe640 1 --2- 172.21.15.52:0/989564966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4014101330 0x7f40141996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.440+0000 7f4013fff640 1 -- 172.21.15.52:0/989564966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4014101330 msgr2=0x7f40141996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:07.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.440+0000 7f4013fff640 1 --2- 172.21.15.52:0/989564966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4014101330 0x7f40141996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.440+0000 7f4013fff640 1 -- 172.21.15.52:0/989564966 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4014106980 msgr2=0x7f40141a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:07.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.441+0000 7f4013fff640 1 --2- 172.21.15.52:0/989564966 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4014106980 0x7f40141a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.441+0000 7f4013fff640 1 -- 172.21.15.52:0/989564966 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40141a4750 con 0x7f4014100970 2024-04-02T14:00:07.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.441+0000 7f4018b04640 1 --2- 172.21.15.52:0/989564966 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4014106980 0x7f40141a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:00:07.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.441+0000 7f4013fff640 1 --2- 172.21.15.52:0/989564966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4014100970 0x7f4014199190 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00e9f0 tx=0x7f3ffc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:07.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.441+0000 7f40117fa640 1 -- 172.21.15.52:0/989564966 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc00cd80 con 0x7f4014100970 2024-04-02T14:00:07.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.441+0000 7f401a567640 1 -- 172.21.15.52:0/989564966 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40141a4920 con 0x7f4014100970 2024-04-02T14:00:07.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.441+0000 7f40117fa640 1 -- 172.21.15.52:0/989564966 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ffc010910 con 0x7f4014100970 2024-04-02T14:00:07.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.441+0000 7f40117fa640 1 -- 172.21.15.52:0/989564966 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc018990 con 0x7f4014100970 2024-04-02T14:00:07.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.441+0000 7f401a567640 1 -- 172.21.15.52:0/989564966 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40141a4e70 con 0x7f4014100970 2024-04-02T14:00:07.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.444+0000 7f40117fa640 1 -- 172.21.15.52:0/989564966 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3ffc010ac0 con 0x7f4014100970 2024-04-02T14:00:07.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.444+0000 7f401d22b740 1 Processor -- start 2024-04-02T14:00:07.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.444+0000 7f401d22b740 1 -- start start 2024-04-02T14:00:07.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.444+0000 7f401d22b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53ae8d0 0x562bc53aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.444+0000 7f401d22b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bc53af1f0 0x562bc53b7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.444+0000 7f401d22b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bc53b7dd0 0x562bc53ba1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.444+0000 7f401d22b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562bc5312830 con 0x562bc53ae8d0 2024-04-02T14:00:07.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.444+0000 7f401d22b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x562bc52ba820 con 0x562bc53b7dd0 2024-04-02T14:00:07.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.444+0000 7f401d22b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562bc530e7e0 con 0x562bc53af1f0 2024-04-02T14:00:07.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.444+0000 7f4018b04640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53ae8d0 0x562bc53aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.444+0000 7f4018b04640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53ae8d0 0x562bc53aecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34318/0 (socket says 172.21.15.52:34318) 2024-04-02T14:00:07.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.444+0000 7f4018b04640 1 -- 172.21.15.52:0/1387708416 learned_addr learned my addr 172.21.15.52:0/1387708416 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:07.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.445+0000 7f4013fff640 1 --2- 172.21.15.52:0/1387708416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bc53af1f0 0x562bc53b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.445+0000 7f40117fa640 1 --2- 172.21.15.52:0/989564966 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f40000a3d20 0x7f40000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.445+0000 7f40137fe640 1 --2- 172.21.15.52:0/1387708416 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bc53b7dd0 0x562bc53ba1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.445+0000 7f4018b04640 1 --2- 172.21.15.52:0/989564966 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f40000a3d20 0x7f40000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.445+0000 7f4018b04640 1 -- 172.21.15.52:0/1387708416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bc53af1f0 msgr2=0x562bc53b7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:07.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.445+0000 7f4018b04640 1 --2- 172.21.15.52:0/1387708416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bc53af1f0 0x562bc53b7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.445+0000 7f4018b04640 1 -- 172.21.15.52:0/1387708416 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bc53b7dd0 msgr2=0x562bc53ba1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:07.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.445+0000 7f4018b04640 1 --2- 172.21.15.52:0/1387708416 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bc53b7dd0 0x562bc53ba1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.445+0000 7f4018b04640 1 -- 172.21.15.52:0/1387708416 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bc50f07d0 con 0x562bc53ae8d0 2024-04-02T14:00:07.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.445+0000 7f40137fe640 1 --2- 172.21.15.52:0/1387708416 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bc53b7dd0 0x562bc53ba1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:00:07.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.445+0000 7f40117fa640 1 -- 172.21.15.52:0/989564966 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f3ffc0d0f20 con 0x7f4014100970 2024-04-02T14:00:07.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.445+0000 7f4018b04640 1 --2- 172.21.15.52:0/1387708416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53ae8d0 0x562bc53aecb0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f400c009430 tx=0x7f400c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f792cf2db09223e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:07.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.445+0000 7f400a7fc640 1 -- 172.21.15.52:0/1387708416 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c0328f0 con 0x562bc53ae8d0 2024-04-02T14:00:07.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.445+0000 7f400a7fc640 1 -- 172.21.15.52:0/1387708416 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f400c004520 con 0x562bc53ae8d0 2024-04-02T14:00:07.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.445+0000 7f400a7fc640 1 -- 172.21.15.52:0/1387708416 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c034970 con 0x562bc53ae8d0 2024-04-02T14:00:07.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.446+0000 7f401d22b740 1 -- 172.21.15.52:0/1387708416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53ae8d0 msgr2=0x562bc53aecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:07.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.446+0000 7f4018b04640 1 --2- 172.21.15.52:0/989564966 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f40000a3d20 0x7f40000a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f400c04c000 tx=0x7f400c048000 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:07.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.446+0000 7f401d22b740 1 --2- 172.21.15.52:0/1387708416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53ae8d0 0x562bc53aecb0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f400c009430 tx=0x7f400c032f40 comp rx=0 tx=0).stop 2024-04-02T14:00:07.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.446+0000 7f401d22b740 1 -- 172.21.15.52:0/1387708416 shutdown_connections 2024-04-02T14:00:07.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.446+0000 7f401d22b740 1 --2- 172.21.15.52:0/1387708416 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bc53b7dd0 0x562bc53ba1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.446+0000 7f401d22b740 1 --2- 172.21.15.52:0/1387708416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bc53af1f0 0x562bc53b7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.446+0000 7f401d22b740 1 --2- 172.21.15.52:0/1387708416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53ae8d0 0x562bc53aecb0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.446+0000 7f401d22b740 1 -- 172.21.15.52:0/1387708416 >> 172.21.15.52:0/1387708416 conn(0x562bc53a9000 msgr2=0x562bc53a93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:07.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.446+0000 7f401d22b740 1 -- 172.21.15.52:0/1387708416 shutdown_connections 2024-04-02T14:00:07.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.446+0000 7f401d22b740 1 -- 172.21.15.52:0/1387708416 wait complete. 2024-04-02T14:00:07.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.446+0000 7f401d22b740 1 -- start start 2024-04-02T14:00:07.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.447+0000 7f401d22b740 1 Processor -- start 2024-04-02T14:00:07.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.447+0000 7f401d22b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53ae8d0 0x562bc53b7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.447+0000 7f401d22b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bc53af1f0 0x562bc53b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.447+0000 7f401d22b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bc53b7dd0 0x562bc53b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.447+0000 7f401d22b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562bc53babc0 con 0x562bc53ae8d0 2024-04-02T14:00:07.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.447+0000 7f401d22b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x562bc52cdd30 con 0x562bc53b7dd0 2024-04-02T14:00:07.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.447+0000 7f401d22b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562bc53bad40 con 0x562bc53af1f0 2024-04-02T14:00:07.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.447+0000 7f401d22b740 10 client.0 _ll_register_callbacks cb 0x562bc52cd720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:07.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.447+0000 7f4013fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53ae8d0 0x562bc53b7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.447+0000 7f4013fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53ae8d0 0x562bc53b7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34330/0 (socket says 172.21.15.52:34330) 2024-04-02T14:00:07.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.447+0000 7f4013fff640 1 -- 172.21.15.52:0/490455168 learned_addr learned my addr 172.21.15.52:0/490455168 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:07.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.447+0000 7f4018b04640 1 --2- 172.21.15.52:0/490455168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bc53b7dd0 0x562bc53b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.447+0000 7f40137fe640 1 --2- 172.21.15.52:0/490455168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bc53af1f0 0x562bc53b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.447+0000 7f4013fff640 1 -- 172.21.15.52:0/490455168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bc53af1f0 msgr2=0x562bc53b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:07.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.447+0000 7f4013fff640 1 --2- 172.21.15.52:0/490455168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bc53af1f0 0x562bc53b49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.447+0000 7f4013fff640 1 -- 172.21.15.52:0/490455168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bc53b7dd0 msgr2=0x562bc53b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:07.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.447+0000 7f4013fff640 1 --2- 172.21.15.52:0/490455168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bc53b7dd0 0x562bc53b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.447+0000 7f4013fff640 1 -- 172.21.15.52:0/490455168 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bc50263f0 con 0x562bc53ae8d0 2024-04-02T14:00:07.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.447+0000 7f4018b04640 1 --2- 172.21.15.52:0/490455168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bc53b7dd0 0x562bc53b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:00:07.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.447+0000 7f4013fff640 1 --2- 172.21.15.52:0/490455168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53ae8d0 0x562bc53b7630 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00e9c0 tx=0x7f3ffc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d51770a89b2130 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.447+0000 7f3ff37fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.448+0000 7f3ff37fe640 1 -- 172.21.15.52:0/490455168 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc0a57f0 con 0x562bc53ae8d0 2024-04-02T14:00:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.448+0000 7f3ff37fe640 1 -- 172.21.15.52:0/490455168 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f3ffc018b30 con 0x562bc53ae8d0 2024-04-02T14:00:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.448+0000 7f401d22b740 1 -- 172.21.15.52:0/490455168 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f400c004520 con 0x562bc53ae8d0 2024-04-02T14:00:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.448+0000 7f3ff37fe640 1 -- 172.21.15.52:0/490455168 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3ffc0b60d0 con 0x562bc53ae8d0 2024-04-02T14:00:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.448+0000 7f401d22b740 10 client.17454 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.448+0000 7f401d22b740 1 -- 172.21.15.52:0/490455168 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562bc4fe60f0 con 0x562bc53ae8d0 2024-04-02T14:00:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.448+0000 7f3ff37fe640 1 -- 172.21.15.52:0/490455168 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f400c004520 con 0x562bc53ae8d0 2024-04-02T14:00:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.448+0000 7f3ff37fe640 1 -- 172.21.15.52:0/490455168 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3ffc0988d0 con 0x562bc53ae8d0 2024-04-02T14:00:07.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.448+0000 7f401d22b740 10 client.17454 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:07.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.448+0000 7f401d22b740 10 client.17454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:00:07.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.448+0000 7f401d22b740 20 client.17454 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:07.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.448+0000 7f401d22b740 1 --2- 172.21.15.52:0/490455168 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x562bc53c7d60 0x562bc53ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.448+0000 7f401d22b740 4 client.17454 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:00:07.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.448+0000 7f401d22b740 1 -- 172.21.15.52:0/490455168 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bc531fd70 con 0x562bc53c7d60 2024-04-02T14:00:07.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.449+0000 7f40137fe640 1 --2- 172.21.15.52:0/490455168 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x562bc53c7d60 0x562bc53ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.449+0000 7f40137fe640 1 --2- 172.21.15.52:0/490455168 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x562bc53c7d60 0x562bc53ca1d0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc9956e17bcb5054 server_cookie=b6fc6de8113a533 in_seq=0 out_seq=0 2024-04-02T14:00:07.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.449+0000 7f3ff37fe640 10 client.17454 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:07.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.450+0000 7f3ff37fe640 1 -- 172.21.15.52:0/490455168 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562bc531fd70 con 0x562bc53c7d60 2024-04-02T14:00:07.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.450+0000 7f3ff37fe640 10 client.17454 handle_command_reply: tid=0 2024-04-02T14:00:07.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.450+0000 7f401d22b740 1 client.17454 shutdown 2024-04-02T14:00:07.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.450+0000 7f401d22b740 1 -- 172.21.15.52:0/490455168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53ae8d0 msgr2=0x562bc53b7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:07.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.450+0000 7f401d22b740 1 --2- 172.21.15.52:0/490455168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53ae8d0 0x562bc53b7630 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00e9c0 tx=0x7f3ffc009310 comp rx=0 tx=0).stop 2024-04-02T14:00:07.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.450+0000 7f401d22b740 1 -- 172.21.15.52:0/490455168 shutdown_connections 2024-04-02T14:00:07.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.450+0000 7f401d22b740 1 --2- 172.21.15.52:0/490455168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bc53b7dd0 0x562bc53b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.450+0000 7f401d22b740 1 --2- 172.21.15.52:0/490455168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bc53af1f0 0x562bc53b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.450+0000 7f401d22b740 1 --2- 172.21.15.52:0/490455168 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x562bc53c7d60 0x562bc53ca1d0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.450+0000 7f401d22b740 1 --2- 172.21.15.52:0/490455168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53ae8d0 0x562bc53b7630 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.450+0000 7f3ff37fe640 0 client.17454 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:07.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.450+0000 7f401d22b740 1 -- 172.21.15.52:0/490455168 >> 172.21.15.52:0/490455168 conn(0x562bc53a9000 msgr2=0x562bc53aad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:07.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.450+0000 7f401d22b740 1 -- 172.21.15.52:0/490455168 shutdown_connections 2024-04-02T14:00:07.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.450+0000 7f401d22b740 1 -- 172.21.15.52:0/490455168 wait complete. 2024-04-02T14:00:07.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.450+0000 7f401d22b740 20 client.17454 trim_cache size 0 max 16384 2024-04-02T14:00:07.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.468+0000 7f401d22b740 1 Processor -- start 2024-04-02T14:00:07.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.468+0000 7f401d22b740 1 -- start start 2024-04-02T14:00:07.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.468+0000 7f401d22b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bc53ae8d0 0x562bc53c3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.468+0000 7f401d22b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53af1f0 0x562bc53affa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.468+0000 7f401d22b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bc53b7dd0 0x562bc53b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.468+0000 7f401d22b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f40141aee70 con 0x562bc53af1f0 2024-04-02T14:00:07.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.468+0000 7f401d22b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f40141b09b0 con 0x562bc53ae8d0 2024-04-02T14:00:07.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.468+0000 7f401d22b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f40141af540 con 0x562bc53b7dd0 2024-04-02T14:00:07.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.468+0000 7f40137fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53af1f0 0x562bc53affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.468+0000 7f40137fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53af1f0 0x562bc53affa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34336/0 (socket says 172.21.15.52:34336) 2024-04-02T14:00:07.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.468+0000 7f40137fe640 1 -- 172.21.15.52:0/2437342847 learned_addr learned my addr 172.21.15.52:0/2437342847 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:07.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.468+0000 7f4013fff640 1 --2- 172.21.15.52:0/2437342847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bc53ae8d0 0x562bc53c3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.468+0000 7f4018b04640 1 --2- 172.21.15.52:0/2437342847 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bc53b7dd0 0x562bc53b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.468+0000 7f40137fe640 1 -- 172.21.15.52:0/2437342847 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bc53b7dd0 msgr2=0x562bc53b70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:07.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.468+0000 7f40137fe640 1 --2- 172.21.15.52:0/2437342847 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bc53b7dd0 0x562bc53b70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.468+0000 7f40137fe640 1 -- 172.21.15.52:0/2437342847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bc53ae8d0 msgr2=0x562bc53c3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:07.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.468+0000 7f40137fe640 1 --2- 172.21.15.52:0/2437342847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bc53ae8d0 0x562bc53c3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.468+0000 7f40137fe640 1 -- 172.21.15.52:0/2437342847 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bc53a8e30 con 0x562bc53af1f0 2024-04-02T14:00:07.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.468+0000 7f4018b04640 1 --2- 172.21.15.52:0/2437342847 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bc53b7dd0 0x562bc53b70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:00:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.469+0000 7f40137fe640 1 --2- 172.21.15.52:0/2437342847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53af1f0 0x562bc53affa0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f4004005d00 tx=0x7f4004005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b877f6d4b5ff22fb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.469+0000 7f4009ffb640 1 -- 172.21.15.52:0/2437342847 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004013d10 con 0x562bc53af1f0 2024-04-02T14:00:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.469+0000 7f4009ffb640 1 -- 172.21.15.52:0/2437342847 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f400401e030 con 0x562bc53af1f0 2024-04-02T14:00:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.469+0000 7f4009ffb640 1 -- 172.21.15.52:0/2437342847 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004011ac0 con 0x562bc53af1f0 2024-04-02T14:00:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.469+0000 7f401d22b740 1 -- 172.21.15.52:0/2437342847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53af1f0 msgr2=0x562bc53affa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.469+0000 7f401d22b740 1 --2- 172.21.15.52:0/2437342847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53af1f0 0x562bc53affa0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f4004005d00 tx=0x7f4004005c50 comp rx=0 tx=0).stop 2024-04-02T14:00:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.469+0000 7f401d22b740 1 -- 172.21.15.52:0/2437342847 shutdown_connections 2024-04-02T14:00:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.469+0000 7f401d22b740 1 --2- 172.21.15.52:0/2437342847 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bc53b7dd0 0x562bc53b70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.469+0000 7f401d22b740 1 --2- 172.21.15.52:0/2437342847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53af1f0 0x562bc53affa0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.469+0000 7f401d22b740 1 --2- 172.21.15.52:0/2437342847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bc53ae8d0 0x562bc53c3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.469+0000 7f401d22b740 1 -- 172.21.15.52:0/2437342847 >> 172.21.15.52:0/2437342847 conn(0x562bc53a9000 msgr2=0x562bc53b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.469+0000 7f401d22b740 1 -- 172.21.15.52:0/2437342847 shutdown_connections 2024-04-02T14:00:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.469+0000 7f401d22b740 1 -- 172.21.15.52:0/2437342847 wait complete. 2024-04-02T14:00:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.470+0000 7f401d22b740 1 -- start start 2024-04-02T14:00:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.470+0000 7f401d22b740 1 Processor -- start 2024-04-02T14:00:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.470+0000 7f401d22b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bc53ae8d0 0x562bc53c0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.470+0000 7f401d22b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bc53af1f0 0x562bc53bb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.470+0000 7f401d22b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53b7dd0 0x562bc53bbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.470+0000 7f401d22b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f40141b02e0 con 0x562bc53b7dd0 2024-04-02T14:00:07.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.470+0000 7f401d22b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f40141afc10 con 0x562bc53af1f0 2024-04-02T14:00:07.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.470+0000 7f401d22b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f40141ace50 con 0x562bc53ae8d0 2024-04-02T14:00:07.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.470+0000 7f401d22b740 10 client.0 _ll_register_callbacks cb 0x7f3ffc09c4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:07.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.470+0000 7f4018b04640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53b7dd0 0x562bc53bbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.470+0000 7f4018b04640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53b7dd0 0x562bc53bbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34340/0 (socket says 172.21.15.52:34340) 2024-04-02T14:00:07.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.470+0000 7f4018b04640 1 -- 172.21.15.52:0/3545799059 learned_addr learned my addr 172.21.15.52:0/3545799059 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:07.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.471+0000 7f40137fe640 1 --2- 172.21.15.52:0/3545799059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bc53af1f0 0x562bc53bb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.471+0000 7f4013fff640 1 --2- 172.21.15.52:0/3545799059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bc53ae8d0 0x562bc53c0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.471+0000 7f4018b04640 1 -- 172.21.15.52:0/3545799059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bc53ae8d0 msgr2=0x562bc53c0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:07.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.471+0000 7f4018b04640 1 --2- 172.21.15.52:0/3545799059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bc53ae8d0 0x562bc53c0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.471+0000 7f4018b04640 1 -- 172.21.15.52:0/3545799059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bc53af1f0 msgr2=0x562bc53bb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:07.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.471+0000 7f4018b04640 1 --2- 172.21.15.52:0/3545799059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bc53af1f0 0x562bc53bb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.471+0000 7f4018b04640 1 -- 172.21.15.52:0/3545799059 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bc53bc410 con 0x562bc53b7dd0 2024-04-02T14:00:07.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.471+0000 7f40137fe640 1 --2- 172.21.15.52:0/3545799059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bc53af1f0 0x562bc53bb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:00:07.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.471+0000 7f4018b04640 1 --2- 172.21.15.52:0/3545799059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53b7dd0 0x562bc53bbbe0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f400c009430 tx=0x7f400c0349a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f125b2bdff746d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:07.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.471+0000 7f3ff37fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:07.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.471+0000 7f3ff37fe640 1 -- 172.21.15.52:0/3545799059 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c039070 con 0x562bc53b7dd0 2024-04-02T14:00:07.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.471+0000 7f3ff37fe640 1 -- 172.21.15.52:0/3545799059 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f400c0328f0 con 0x562bc53b7dd0 2024-04-02T14:00:07.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.471+0000 7f401d22b740 1 -- 172.21.15.52:0/3545799059 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f400401e030 con 0x562bc53b7dd0 2024-04-02T14:00:07.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.471+0000 7f3ff37fe640 1 -- 172.21.15.52:0/3545799059 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f400c060050 con 0x562bc53b7dd0 2024-04-02T14:00:07.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.471+0000 7f401d22b740 10 client.17460 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:07.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.471+0000 7f401d22b740 1 -- 172.21.15.52:0/3545799059 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562bc542e280 con 0x562bc53b7dd0 2024-04-02T14:00:07.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.471+0000 7f3ff37fe640 1 -- 172.21.15.52:0/3545799059 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f400401e030 con 0x562bc53b7dd0 2024-04-02T14:00:07.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.472+0000 7f3ff37fe640 1 -- 172.21.15.52:0/3545799059 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f400c043430 con 0x562bc53b7dd0 2024-04-02T14:00:07.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.472+0000 7f401d22b740 10 client.17460 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:07.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.472+0000 7f401d22b740 10 client.17460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:00:07.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.472+0000 7f401d22b740 20 client.17460 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:07.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.472+0000 7f401d22b740 1 --2- 172.21.15.52:0/3545799059 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x562bc542e580 0x562bc54309a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.472+0000 7f401d22b740 4 client.17460 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:00:07.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.472+0000 7f401d22b740 1 -- 172.21.15.52:0/3545799059 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562bc531a0c0 con 0x562bc542e580 2024-04-02T14:00:07.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.472+0000 7f4013fff640 1 --2- 172.21.15.52:0/3545799059 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x562bc542e580 0x562bc54309a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.472+0000 7f4013fff640 1 --2- 172.21.15.52:0/3545799059 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x562bc542e580 0x562bc54309a0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bb792795306de7b server_cookie=1d6aa56d137b39fb in_seq=0 out_seq=0 2024-04-02T14:00:07.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.472+0000 7f3ff37fe640 10 client.17460 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:07.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.472+0000 7f3ff37fe640 1 -- 172.21.15.52:0/3545799059 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562bc531a0c0 con 0x562bc542e580 2024-04-02T14:00:07.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.472+0000 7f3ff37fe640 10 client.17460 handle_command_reply: tid=0 2024-04-02T14:00:07.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.472+0000 7f401d22b740 1 client.17460 shutdown 2024-04-02T14:00:07.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.473+0000 7f401d22b740 1 -- 172.21.15.52:0/3545799059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53b7dd0 msgr2=0x562bc53bbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:07.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.473+0000 7f401d22b740 1 --2- 172.21.15.52:0/3545799059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53b7dd0 0x562bc53bbbe0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f400c009430 tx=0x7f400c0349a0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.473+0000 7f401d22b740 1 -- 172.21.15.52:0/3545799059 shutdown_connections 2024-04-02T14:00:07.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.473+0000 7f401d22b740 1 --2- 172.21.15.52:0/3545799059 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x562bc542e580 0x562bc54309a0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.473+0000 7f401d22b740 1 --2- 172.21.15.52:0/3545799059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bc53b7dd0 0x562bc53bbbe0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.473+0000 7f401d22b740 1 --2- 172.21.15.52:0/3545799059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bc53af1f0 0x562bc53bb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.473+0000 7f3ff37fe640 0 client.17460 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:07.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.473+0000 7f401d22b740 1 --2- 172.21.15.52:0/3545799059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bc53ae8d0 0x562bc53c0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.473+0000 7f401d22b740 1 -- 172.21.15.52:0/3545799059 >> 172.21.15.52:0/3545799059 conn(0x562bc53a9000 msgr2=0x562bc53affa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:07.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.473+0000 7f401d22b740 1 -- 172.21.15.52:0/3545799059 shutdown_connections 2024-04-02T14:00:07.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.473+0000 7f401d22b740 1 -- 172.21.15.52:0/3545799059 wait complete. 2024-04-02T14:00:07.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.473+0000 7f401d22b740 20 client.17460 trim_cache size 0 max 16384 2024-04-02T14:00:07.488 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:00:07.488 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-04-02T14:00:07.488 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-04-02T14:00:07.488 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:00:07.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.473+0000 7f401a567640 1 -- 172.21.15.52:0/989564966 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f40000a3d20 msgr2=0x7f40000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:07.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.473+0000 7f401a567640 1 --2- 172.21.15.52:0/989564966 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f40000a3d20 0x7f40000a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f400c04c000 tx=0x7f400c048000 comp rx=0 tx=0).stop 2024-04-02T14:00:07.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.473+0000 7f401a567640 1 -- 172.21.15.52:0/989564966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4014100970 msgr2=0x7f4014199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:07.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.473+0000 7f401a567640 1 --2- 172.21.15.52:0/989564966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4014100970 0x7f4014199190 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00e9f0 tx=0x7f3ffc00eeb0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.473+0000 7f401a567640 1 -- 172.21.15.52:0/989564966 shutdown_connections 2024-04-02T14:00:07.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.473+0000 7f401a567640 1 --2- 172.21.15.52:0/989564966 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f40000a3d20 0x7f40000a61e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.473+0000 7f401a567640 1 --2- 172.21.15.52:0/989564966 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4014106980 0x7f40141a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.473+0000 7f401a567640 1 --2- 172.21.15.52:0/989564966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4014101330 0x7f40141996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.473+0000 7f401a567640 1 --2- 172.21.15.52:0/989564966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4014100970 0x7f4014199190 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.473+0000 7f401a567640 1 -- 172.21.15.52:0/989564966 >> 172.21.15.52:0/989564966 conn(0x7f4014077b00 msgr2=0x7f4014103170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:07.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.474+0000 7f401a567640 1 -- 172.21.15.52:0/989564966 shutdown_connections 2024-04-02T14:00:07.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.474+0000 7f401a567640 1 -- 172.21.15.52:0/989564966 wait complete. 2024-04-02T14:00:07.491 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T14:00:07.492 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:00:07.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.592+0000 7fed8a206640 1 -- 172.21.15.52:0/825629836 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed841003d0 msgr2=0x7fed84100850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:07.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.592+0000 7fed8a206640 1 --2- 172.21.15.52:0/825629836 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed841003d0 0x7fed84100850 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fed6c009930 tx=0x7fed6c02bcd0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.592+0000 7fed8a206640 1 -- 172.21.15.52:0/825629836 shutdown_connections 2024-04-02T14:00:07.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.592+0000 7fed8a206640 1 --2- 172.21.15.52:0/825629836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed84100d90 0x7fed8410e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.592+0000 7fed8a206640 1 --2- 172.21.15.52:0/825629836 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed841003d0 0x7fed84100850 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.592+0000 7fed8a206640 1 --2- 172.21.15.52:0/825629836 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed84107d10 0x7fed841080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.592+0000 7fed8a206640 1 -- 172.21.15.52:0/825629836 >> 172.21.15.52:0/825629836 conn(0x7fed84075d50 msgr2=0x7fed84076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:07.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.592+0000 7fed8a206640 1 -- 172.21.15.52:0/825629836 shutdown_connections 2024-04-02T14:00:07.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.592+0000 7fed8a206640 1 -- 172.21.15.52:0/825629836 wait complete. 2024-04-02T14:00:07.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.593+0000 7fed8a206640 1 Processor -- start 2024-04-02T14:00:07.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.593+0000 7fed8a206640 1 -- start start 2024-04-02T14:00:07.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.593+0000 7fed8a206640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed841003d0 0x7fed84198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.593+0000 7fed8a206640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed84100d90 0x7fed84198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.593+0000 7fed8a206640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed84107d10 0x7fed8419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.593+0000 7fed8a206640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fed84111f90 con 0x7fed841003d0 2024-04-02T14:00:07.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.593+0000 7fed8a206640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fed84111e10 con 0x7fed84107d10 2024-04-02T14:00:07.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.593+0000 7fed8a206640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fed84112110 con 0x7fed84100d90 2024-04-02T14:00:07.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.593+0000 7fed837fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed841003d0 0x7fed84198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.593+0000 7fed837fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed841003d0 0x7fed84198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34352/0 (socket says 172.21.15.52:34352) 2024-04-02T14:00:07.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.593+0000 7fed837fe640 1 -- 172.21.15.52:0/3578003971 learned_addr learned my addr 172.21.15.52:0/3578003971 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:07.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.593+0000 7fed83fff640 1 --2- 172.21.15.52:0/3578003971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed84107d10 0x7fed8419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.593+0000 7fed82ffd640 1 --2- 172.21.15.52:0/3578003971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed84100d90 0x7fed84198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.593+0000 7fed837fe640 1 -- 172.21.15.52:0/3578003971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed84100d90 msgr2=0x7fed84198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:07.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.593+0000 7fed837fe640 1 --2- 172.21.15.52:0/3578003971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed84100d90 0x7fed84198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.593+0000 7fed837fe640 1 -- 172.21.15.52:0/3578003971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed84107d10 msgr2=0x7fed8419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:07.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.593+0000 7fed837fe640 1 --2- 172.21.15.52:0/3578003971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed84107d10 0x7fed8419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.593+0000 7fed837fe640 1 -- 172.21.15.52:0/3578003971 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8419d400 con 0x7fed841003d0 2024-04-02T14:00:07.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.594+0000 7fed837fe640 1 --2- 172.21.15.52:0/3578003971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed841003d0 0x7fed84198410 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fed7400c890 tx=0x7fed7400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:07.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.594+0000 7fed80ff9640 1 -- 172.21.15.52:0/3578003971 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed74004220 con 0x7fed841003d0 2024-04-02T14:00:07.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.594+0000 7fed8a206640 1 -- 172.21.15.52:0/3578003971 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8419d6f0 con 0x7fed841003d0 2024-04-02T14:00:07.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.594+0000 7fed80ff9640 1 -- 172.21.15.52:0/3578003971 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed740043c0 con 0x7fed841003d0 2024-04-02T14:00:07.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.594+0000 7fed80ff9640 1 -- 172.21.15.52:0/3578003971 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed74004a00 con 0x7fed841003d0 2024-04-02T14:00:07.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.594+0000 7fed8a206640 1 -- 172.21.15.52:0/3578003971 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed841a4f80 con 0x7fed841003d0 2024-04-02T14:00:07.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.595+0000 7fed8a206640 1 -- 172.21.15.52:0/3578003971 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed84068d40 con 0x7fed841003d0 2024-04-02T14:00:07.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.596+0000 7fed80ff9640 1 -- 172.21.15.52:0/3578003971 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fed740079b0 con 0x7fed841003d0 2024-04-02T14:00:07.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.597+0000 7fed80ff9640 1 --2- 172.21.15.52:0/3578003971 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fed700a3d20 0x7fed700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.597+0000 7fed80ff9640 1 -- 172.21.15.52:0/3578003971 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fed740d0ca0 con 0x7fed841003d0 2024-04-02T14:00:07.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.597+0000 7fed82ffd640 1 --2- 172.21.15.52:0/3578003971 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fed700a3d20 0x7fed700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.597+0000 7fed82ffd640 1 --2- 172.21.15.52:0/3578003971 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fed700a3d20 0x7fed700a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fed6c006010 tx=0x7fed6c004f10 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:07.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.600+0000 7fed80ff9640 1 -- 172.21.15.52:0/3578003971 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fed74014030 con 0x7fed841003d0 2024-04-02T14:00:07.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:07 smithi189 ceph-mon[36683]: pgmap v201: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-02T14:00:07.821 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:07 smithi088 ceph-mon[36528]: pgmap v201: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-02T14:00:07.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.825+0000 7fed8a206640 1 -- 172.21.15.52:0/3578003971 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed8410a050 con 0x7fed841003d0 2024-04-02T14:00:07.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.825+0000 7fed80ff9640 1 -- 172.21.15.52:0/3578003971 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11923 (secure 0 0 0) 0x7fed7409d000 con 0x7fed841003d0 2024-04-02T14:00:07.829 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T14:00:07.829 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:57:45.240902+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24467,24457,24490,24475,24503]},"id":1}]} 2024-04-02T14:00:07.830 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-04-02T14:00:07.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.828+0000 7fed8a206640 1 -- 172.21.15.52:0/3578003971 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fed700a3d20 msgr2=0x7fed700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:07.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.828+0000 7fed8a206640 1 --2- 172.21.15.52:0/3578003971 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fed700a3d20 0x7fed700a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fed6c006010 tx=0x7fed6c004f10 comp rx=0 tx=0).stop 2024-04-02T14:00:07.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.828+0000 7fed8a206640 1 -- 172.21.15.52:0/3578003971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed841003d0 msgr2=0x7fed84198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:07.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.828+0000 7fed8a206640 1 --2- 172.21.15.52:0/3578003971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed841003d0 0x7fed84198410 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fed7400c890 tx=0x7fed7400cd50 comp rx=0 tx=0).stop 2024-04-02T14:00:07.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.828+0000 7fed8a206640 1 -- 172.21.15.52:0/3578003971 shutdown_connections 2024-04-02T14:00:07.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.828+0000 7fed8a206640 1 --2- 172.21.15.52:0/3578003971 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fed700a3d20 0x7fed700a61e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.828+0000 7fed8a206640 1 --2- 172.21.15.52:0/3578003971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed84107d10 0x7fed8419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.828+0000 7fed8a206640 1 --2- 172.21.15.52:0/3578003971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed84100d90 0x7fed84198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.828+0000 7fed8a206640 1 --2- 172.21.15.52:0/3578003971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed841003d0 0x7fed84198410 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.828+0000 7fed8a206640 1 -- 172.21.15.52:0/3578003971 >> 172.21.15.52:0/3578003971 conn(0x7fed84075d50 msgr2=0x7fed840feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:07.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.828+0000 7fed8a206640 1 -- 172.21.15.52:0/3578003971 shutdown_connections 2024-04-02T14:00:07.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.828+0000 7fed8a206640 1 -- 172.21.15.52:0/3578003971 wait complete. 2024-04-02T14:00:07.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:07 smithi052 ceph-mon[32104]: pgmap v201: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-02T14:00:07.842 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T14:00:07.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.944+0000 7fda0358f640 1 -- 172.21.15.52:0/897440518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9fc106a30 msgr2=0x7fd9fc106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:07.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.944+0000 7fda0358f640 1 --2- 172.21.15.52:0/897440518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9fc106a30 0x7fd9fc106e10 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009930 tx=0x7fd9f802bc30 comp rx=0 tx=0).stop 2024-04-02T14:00:07.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.944+0000 7fda0358f640 1 -- 172.21.15.52:0/897440518 shutdown_connections 2024-04-02T14:00:07.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.944+0000 7fda0358f640 1 --2- 172.21.15.52:0/897440518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9fc1013e0 0x7fd9fc10d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.944+0000 7fda0358f640 1 --2- 172.21.15.52:0/897440518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9fc100a20 0x7fd9fc100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.944+0000 7fda0358f640 1 --2- 172.21.15.52:0/897440518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9fc106a30 0x7fd9fc106e10 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.944+0000 7fda0358f640 1 -- 172.21.15.52:0/897440518 >> 172.21.15.52:0/897440518 conn(0x7fd9fc077b60 msgr2=0x7fd9fc077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:07.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.944+0000 7fda0358f640 1 -- 172.21.15.52:0/897440518 shutdown_connections 2024-04-02T14:00:07.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.944+0000 7fda0358f640 1 -- 172.21.15.52:0/897440518 wait complete. 2024-04-02T14:00:07.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.945+0000 7fda0358f640 1 Processor -- start 2024-04-02T14:00:07.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.945+0000 7fda0358f640 1 -- start start 2024-04-02T14:00:07.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.945+0000 7fda0358f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9fc100a20 0x7fd9fc197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.945+0000 7fda0358f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9fc1013e0 0x7fd9fc197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.945+0000 7fda0358f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9fc106a30 0x7fd9fc197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.945+0000 7fda0358f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc110e40 con 0x7fd9fc100a20 2024-04-02T14:00:07.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.945+0000 7fda0358f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc110cc0 con 0x7fd9fc1013e0 2024-04-02T14:00:07.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.945+0000 7fda0358f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc110fc0 con 0x7fd9fc106a30 2024-04-02T14:00:07.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.945+0000 7fda0132b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9fc100a20 0x7fd9fc197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.945+0000 7fda0132b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9fc100a20 0x7fd9fc197030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34364/0 (socket says 172.21.15.52:34364) 2024-04-02T14:00:07.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.945+0000 7fda0132b640 1 -- 172.21.15.52:0/1046794242 learned_addr learned my addr 172.21.15.52:0/1046794242 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:07.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.945+0000 7fda00b2a640 1 --2- 172.21.15.52:0/1046794242 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9fc1013e0 0x7fd9fc197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.945+0000 7fda0132b640 1 -- 172.21.15.52:0/1046794242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9fc106a30 msgr2=0x7fd9fc197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:07.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.945+0000 7fda01b2c640 1 --2- 172.21.15.52:0/1046794242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9fc106a30 0x7fd9fc197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.945+0000 7fda0132b640 1 --2- 172.21.15.52:0/1046794242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9fc106a30 0x7fd9fc197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.945+0000 7fda0132b640 1 -- 172.21.15.52:0/1046794242 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9fc1013e0 msgr2=0x7fd9fc197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:07.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.945+0000 7fda0132b640 1 --2- 172.21.15.52:0/1046794242 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9fc1013e0 0x7fd9fc197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.945+0000 7fda0132b640 1 -- 172.21.15.52:0/1046794242 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9fc1a68a0 con 0x7fd9fc100a20 2024-04-02T14:00:07.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.945+0000 7fda0132b640 1 --2- 172.21.15.52:0/1046794242 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9fc100a20 0x7fd9fc197030 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009230 tx=0x7fd9f802cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:07.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.946+0000 7fd9ee7fc640 1 -- 172.21.15.52:0/1046794242 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8041070 con 0x7fd9fc100a20 2024-04-02T14:00:07.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.946+0000 7fd9ee7fc640 1 -- 172.21.15.52:0/1046794242 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9f8004450 con 0x7fd9fc100a20 2024-04-02T14:00:07.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.946+0000 7fd9ee7fc640 1 -- 172.21.15.52:0/1046794242 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8034440 con 0x7fd9fc100a20 2024-04-02T14:00:07.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.946+0000 7fda0358f640 1 -- 172.21.15.52:0/1046794242 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9fc1a6b30 con 0x7fd9fc100a20 2024-04-02T14:00:07.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.946+0000 7fda0358f640 1 -- 172.21.15.52:0/1046794242 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9fc1a6fd0 con 0x7fd9fc100a20 2024-04-02T14:00:07.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.948+0000 7fd9ee7fc640 1 -- 172.21.15.52:0/1046794242 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd9f8044070 con 0x7fd9fc100a20 2024-04-02T14:00:07.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.948+0000 7fda06170740 1 Processor -- start 2024-04-02T14:00:07.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.948+0000 7fda06170740 1 -- start start 2024-04-02T14:00:07.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.948+0000 7fda06170740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55927eee07a0 0x55927eee0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.948+0000 7fd9ee7fc640 1 --2- 172.21.15.52:0/1046794242 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd9e80a3d20 0x7fd9e80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.948+0000 7fda06170740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927eee10c0 0x55927eee9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.948+0000 7fda06170740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee9ca0 0x55927eeec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.948+0000 7fda06170740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55927ee447b0 con 0x55927eee9ca0 2024-04-02T14:00:07.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.948+0000 7fda06170740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55927ede3570 con 0x55927eee07a0 2024-04-02T14:00:07.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.948+0000 7fda06170740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55927ee40760 con 0x55927eee10c0 2024-04-02T14:00:07.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.949+0000 7fda01b2c640 1 --2- 172.21.15.52:0/1046794242 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd9e80a3d20 0x7fd9e80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.949+0000 7fd9ee7fc640 1 -- 172.21.15.52:0/1046794242 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd9f80f25b0 con 0x7fd9fc100a20 2024-04-02T14:00:07.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.949+0000 7fda01b2c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee9ca0 0x55927eeec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.949+0000 7fda01b2c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee9ca0 0x55927eeec090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34366/0 (socket says 172.21.15.52:34366) 2024-04-02T14:00:07.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.949+0000 7fda01b2c640 1 -- 172.21.15.52:0/3544472205 learned_addr learned my addr 172.21.15.52:0/3544472205 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:07.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.949+0000 7fda0132b640 1 --2- 172.21.15.52:0/3544472205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55927eee07a0 0x55927eee0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.949+0000 7fda00b2a640 1 --2- 172.21.15.52:0/3544472205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927eee10c0 0x55927eee9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.949+0000 7fda01b2c640 1 -- 172.21.15.52:0/3544472205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927eee10c0 msgr2=0x55927eee9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:07.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.949+0000 7fda01b2c640 1 --2- 172.21.15.52:0/3544472205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927eee10c0 0x55927eee9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.949+0000 7fda01b2c640 1 -- 172.21.15.52:0/3544472205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55927eee07a0 msgr2=0x55927eee0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:07.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.949+0000 7fda01b2c640 1 --2- 172.21.15.52:0/3544472205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55927eee07a0 0x55927eee0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.949+0000 7fda01b2c640 1 -- 172.21.15.52:0/3544472205 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55927eb583f0 con 0x55927eee9ca0 2024-04-02T14:00:07.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.949+0000 7fda0132b640 1 --2- 172.21.15.52:0/3544472205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55927eee07a0 0x55927eee0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:00:07.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.949+0000 7fda01b2c640 1 --2- 172.21.15.52:0/1046794242 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd9e80a3d20 0x7fd9e80a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4014a40 tx=0x7fd9f4009c70 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:07.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.949+0000 7fda01b2c640 1 --2- 172.21.15.52:0/3544472205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee9ca0 0x55927eeec090 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4011c80 tx=0x7fd9f401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7309bbf4bc007211 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:07.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.949+0000 7fd9e77fe640 1 -- 172.21.15.52:0/3544472205 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4011d70 con 0x55927eee9ca0 2024-04-02T14:00:07.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.949+0000 7fd9e77fe640 1 -- 172.21.15.52:0/3544472205 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9f4014110 con 0x55927eee9ca0 2024-04-02T14:00:07.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.949+0000 7fd9e77fe640 1 -- 172.21.15.52:0/3544472205 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4003bd0 con 0x55927eee9ca0 2024-04-02T14:00:07.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.950+0000 7fda06170740 1 -- 172.21.15.52:0/3544472205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee9ca0 msgr2=0x55927eeec090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:07.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.950+0000 7fda06170740 1 --2- 172.21.15.52:0/3544472205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee9ca0 0x55927eeec090 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4011c80 tx=0x7fd9f401b000 comp rx=0 tx=0).stop 2024-04-02T14:00:07.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.950+0000 7fda06170740 1 -- 172.21.15.52:0/3544472205 shutdown_connections 2024-04-02T14:00:07.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.950+0000 7fda06170740 1 --2- 172.21.15.52:0/3544472205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee9ca0 0x55927eeec090 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.950+0000 7fda06170740 1 --2- 172.21.15.52:0/3544472205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927eee10c0 0x55927eee9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.950+0000 7fda06170740 1 --2- 172.21.15.52:0/3544472205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55927eee07a0 0x55927eee0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.950+0000 7fda06170740 1 -- 172.21.15.52:0/3544472205 >> 172.21.15.52:0/3544472205 conn(0x55927eedaed0 msgr2=0x55927eedb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:07.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.950+0000 7fda06170740 1 -- 172.21.15.52:0/3544472205 shutdown_connections 2024-04-02T14:00:07.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.950+0000 7fda06170740 1 -- 172.21.15.52:0/3544472205 wait complete. 2024-04-02T14:00:07.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.950+0000 7fda06170740 1 -- start start 2024-04-02T14:00:07.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.950+0000 7fda06170740 1 Processor -- start 2024-04-02T14:00:07.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.951+0000 7fda06170740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee07a0 0x55927eee9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.951+0000 7fda06170740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927eee10c0 0x55927eee6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.951+0000 7fda06170740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55927eee9ca0 0x55927eee7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.951+0000 7fda06170740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55927eeeca30 con 0x55927eee07a0 2024-04-02T14:00:07.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.951+0000 7fda06170740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55927edffe80 con 0x55927eee9ca0 2024-04-02T14:00:07.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.951+0000 7fda06170740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55927eeecbb0 con 0x55927eee10c0 2024-04-02T14:00:07.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.951+0000 7fda06170740 10 client.0 _ll_register_callbacks cb 0x55927ee3baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:07.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.951+0000 7fda0132b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee07a0 0x55927eee9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.951+0000 7fda0132b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee07a0 0x55927eee9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34380/0 (socket says 172.21.15.52:34380) 2024-04-02T14:00:07.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.951+0000 7fda0132b640 1 -- 172.21.15.52:0/340819226 learned_addr learned my addr 172.21.15.52:0/340819226 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:07.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.951+0000 7fda00b2a640 1 --2- 172.21.15.52:0/340819226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927eee10c0 0x55927eee6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.951+0000 7fda01b2c640 1 --2- 172.21.15.52:0/340819226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55927eee9ca0 0x55927eee7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.951+0000 7fda0132b640 1 -- 172.21.15.52:0/340819226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927eee10c0 msgr2=0x55927eee6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:07.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.951+0000 7fda0132b640 1 --2- 172.21.15.52:0/340819226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927eee10c0 0x55927eee6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.951+0000 7fda0132b640 1 -- 172.21.15.52:0/340819226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55927eee9ca0 msgr2=0x55927eee7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:07.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.951+0000 7fda0132b640 1 --2- 172.21.15.52:0/340819226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55927eee9ca0 0x55927eee7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.951+0000 7fda0132b640 1 -- 172.21.15.52:0/340819226 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55927ec227d0 con 0x55927eee07a0 2024-04-02T14:00:07.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.951+0000 7fda0132b640 1 --2- 172.21.15.52:0/340819226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee07a0 0x55927eee9760 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009430 tx=0x7fd9f80be0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804f2e2f9b87f763 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:07.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.951+0000 7fd9e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:07.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.951+0000 7fd9e4ff9640 1 -- 172.21.15.52:0/340819226 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f804b040 con 0x55927eee07a0 2024-04-02T14:00:07.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.952+0000 7fd9e4ff9640 1 -- 172.21.15.52:0/340819226 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd9f80f21f0 con 0x55927eee07a0 2024-04-02T14:00:07.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.952+0000 7fda06170740 1 -- 172.21.15.52:0/340819226 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9f4014110 con 0x55927eee07a0 2024-04-02T14:00:07.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.952+0000 7fd9e4ff9640 1 -- 172.21.15.52:0/340819226 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd9f80f23f0 con 0x55927eee07a0 2024-04-02T14:00:07.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.952+0000 7fda06170740 10 client.17478 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:07.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.952+0000 7fda06170740 1 -- 172.21.15.52:0/340819226 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55927eef8b30 con 0x55927eee07a0 2024-04-02T14:00:07.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.952+0000 7fd9e4ff9640 1 -- 172.21.15.52:0/340819226 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9f4014110 con 0x55927eee07a0 2024-04-02T14:00:07.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.952+0000 7fd9e4ff9640 1 -- 172.21.15.52:0/340819226 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd9f80dd140 con 0x55927eee07a0 2024-04-02T14:00:07.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.952+0000 7fda06170740 10 client.17478 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:07.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.952+0000 7fda06170740 10 client.17478 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T14:00:07.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.952+0000 7fda06170740 20 client.17478 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:07.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.952+0000 7fda06170740 1 --2- 172.21.15.52:0/340819226 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55927eef9ed0 0x55927eefc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.952+0000 7fda06170740 4 client.17478 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:00:07.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.952+0000 7fda06170740 1 -- 172.21.15.52:0/340819226 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55927ee51cf0 con 0x55927eef9ed0 2024-04-02T14:00:07.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.952+0000 7fda00b2a640 1 --2- 172.21.15.52:0/340819226 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55927eef9ed0 0x55927eefc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.953+0000 7fd9e4ff9640 10 client.17478 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T14:00:07.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.953+0000 7fda00b2a640 1 --2- 172.21.15.52:0/340819226 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55927eef9ed0 0x55927eefc2e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dac784b55698adde server_cookie=db0e1610076001cf in_seq=0 out_seq=0 2024-04-02T14:00:07.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.954+0000 7fd9e4ff9640 1 -- 172.21.15.52:0/340819226 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55927ee51cf0 con 0x55927eef9ed0 2024-04-02T14:00:07.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.954+0000 7fd9e4ff9640 10 client.17478 handle_command_reply: tid=0 2024-04-02T14:00:07.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.954+0000 7fda06170740 1 client.17478 shutdown 2024-04-02T14:00:07.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.954+0000 7fda06170740 1 -- 172.21.15.52:0/340819226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee07a0 msgr2=0x55927eee9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:07.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.954+0000 7fda06170740 1 --2- 172.21.15.52:0/340819226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee07a0 0x55927eee9760 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009430 tx=0x7fd9f80be0f0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.954+0000 7fda06170740 1 -- 172.21.15.52:0/340819226 shutdown_connections 2024-04-02T14:00:07.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.954+0000 7fda06170740 1 --2- 172.21.15.52:0/340819226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55927eee9ca0 0x55927eee7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.954+0000 7fda06170740 1 --2- 172.21.15.52:0/340819226 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55927eef9ed0 0x55927eefc2e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.954+0000 7fda06170740 1 --2- 172.21.15.52:0/340819226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927eee10c0 0x55927eee6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.954+0000 7fda06170740 1 --2- 172.21.15.52:0/340819226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee07a0 0x55927eee9760 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.954+0000 7fd9e4ff9640 0 client.17478 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T14:00:07.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.954+0000 7fda06170740 1 -- 172.21.15.52:0/340819226 >> 172.21.15.52:0/340819226 conn(0x55927eedaed0 msgr2=0x55927eedcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:07.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.954+0000 7fda06170740 1 -- 172.21.15.52:0/340819226 shutdown_connections 2024-04-02T14:00:07.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.954+0000 7fda06170740 1 -- 172.21.15.52:0/340819226 wait complete. 2024-04-02T14:00:07.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.954+0000 7fda06170740 20 client.17478 trim_cache size 0 max 16384 2024-04-02T14:00:07.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.967+0000 7fda06170740 1 Processor -- start 2024-04-02T14:00:07.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.968+0000 7fda06170740 1 -- start start 2024-04-02T14:00:07.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.968+0000 7fda06170740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927eee07a0 0x55927eefc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.968+0000 7fda06170740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55927eee10c0 0x55927eef5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.968+0000 7fda06170740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee9ca0 0x55927eee5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.968+0000 7fda06170740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc1b1020 con 0x55927eee9ca0 2024-04-02T14:00:07.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.968+0000 7fda06170740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc1b2b60 con 0x55927eee10c0 2024-04-02T14:00:07.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.968+0000 7fda06170740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc1b16f0 con 0x55927eee07a0 2024-04-02T14:00:07.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.968+0000 7fda01b2c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee9ca0 0x55927eee5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.968+0000 7fda01b2c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee9ca0 0x55927eee5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34388/0 (socket says 172.21.15.52:34388) 2024-04-02T14:00:07.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.968+0000 7fda01b2c640 1 -- 172.21.15.52:0/1204268473 learned_addr learned my addr 172.21.15.52:0/1204268473 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:07.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.968+0000 7fda01b2c640 1 -- 172.21.15.52:0/1204268473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927eee07a0 msgr2=0x55927eefc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:07.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.968+0000 7fda01b2c640 1 --2- 172.21.15.52:0/1204268473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927eee07a0 0x55927eefc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.968+0000 7fda00b2a640 1 --2- 172.21.15.52:0/1204268473 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55927eee10c0 0x55927eef5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.968+0000 7fda01b2c640 1 -- 172.21.15.52:0/1204268473 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55927eee10c0 msgr2=0x55927eef5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:07.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.968+0000 7fda01b2c640 1 --2- 172.21.15.52:0/1204268473 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55927eee10c0 0x55927eef5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.968+0000 7fda01b2c640 1 -- 172.21.15.52:0/1204268473 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55927eee5040 con 0x55927eee9ca0 2024-04-02T14:00:07.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.968+0000 7fda01b2c640 1 --2- 172.21.15.52:0/1204268473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee9ca0 0x55927eee5c50 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x55927eedc060 tx=0x7fd9f4014f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca8e68bf6ce1095 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:07.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.968+0000 7fd9e6ffd640 1 -- 172.21.15.52:0/1204268473 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f40094b0 con 0x55927eee9ca0 2024-04-02T14:00:07.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.968+0000 7fd9e6ffd640 1 -- 172.21.15.52:0/1204268473 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9f4003e50 con 0x55927eee9ca0 2024-04-02T14:00:07.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.968+0000 7fd9e6ffd640 1 -- 172.21.15.52:0/1204268473 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4005040 con 0x55927eee9ca0 2024-04-02T14:00:07.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.969+0000 7fda06170740 1 -- 172.21.15.52:0/1204268473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee9ca0 msgr2=0x55927eee5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:07.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.969+0000 7fda06170740 1 --2- 172.21.15.52:0/1204268473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee9ca0 0x55927eee5c50 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x55927eedc060 tx=0x7fd9f4014f60 comp rx=0 tx=0).stop 2024-04-02T14:00:07.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.969+0000 7fda06170740 1 -- 172.21.15.52:0/1204268473 shutdown_connections 2024-04-02T14:00:07.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.969+0000 7fda06170740 1 --2- 172.21.15.52:0/1204268473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee9ca0 0x55927eee5c50 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.969+0000 7fda06170740 1 --2- 172.21.15.52:0/1204268473 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55927eee10c0 0x55927eef5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.969+0000 7fda06170740 1 --2- 172.21.15.52:0/1204268473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927eee07a0 0x55927eefc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.969+0000 7fda06170740 1 -- 172.21.15.52:0/1204268473 >> 172.21.15.52:0/1204268473 conn(0x55927eedaed0 msgr2=0x55927eedb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:07.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.969+0000 7fda06170740 1 -- 172.21.15.52:0/1204268473 shutdown_connections 2024-04-02T14:00:07.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.969+0000 7fda06170740 1 -- 172.21.15.52:0/1204268473 wait complete. 2024-04-02T14:00:07.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.969+0000 7fda06170740 1 -- start start 2024-04-02T14:00:07.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.969+0000 7fda06170740 1 Processor -- start 2024-04-02T14:00:07.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.970+0000 7fda06170740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927eee07a0 0x55927eef2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.970+0000 7fda06170740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55927eee10c0 0x55927eeed510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.970+0000 7fda06170740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee9ca0 0x55927eeeda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.970+0000 7fda06170740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc1b2490 con 0x55927eee9ca0 2024-04-02T14:00:07.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.970+0000 7fda06170740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc1b1dc0 con 0x55927eee10c0 2024-04-02T14:00:07.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.970+0000 7fda06170740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc1af000 con 0x55927eee07a0 2024-04-02T14:00:07.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.970+0000 7fda06170740 10 client.0 _ll_register_callbacks cb 0x7fd9f8009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:07.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.970+0000 7fda01b2c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee9ca0 0x55927eeeda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.970+0000 7fda01b2c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee9ca0 0x55927eeeda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34404/0 (socket says 172.21.15.52:34404) 2024-04-02T14:00:07.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.970+0000 7fda01b2c640 1 -- 172.21.15.52:0/2087378090 learned_addr learned my addr 172.21.15.52:0/2087378090 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:07.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.970+0000 7fda0132b640 1 --2- 172.21.15.52:0/2087378090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927eee07a0 0x55927eef2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.970+0000 7fda00b2a640 1 --2- 172.21.15.52:0/2087378090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55927eee10c0 0x55927eeed510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.970+0000 7fda01b2c640 1 -- 172.21.15.52:0/2087378090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927eee07a0 msgr2=0x55927eef2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:07.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.970+0000 7fda01b2c640 1 --2- 172.21.15.52:0/2087378090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927eee07a0 0x55927eef2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.970+0000 7fda01b2c640 1 -- 172.21.15.52:0/2087378090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55927eee10c0 msgr2=0x55927eeed510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:07.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.970+0000 7fda01b2c640 1 --2- 172.21.15.52:0/2087378090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55927eee10c0 0x55927eeed510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.970+0000 7fda01b2c640 1 -- 172.21.15.52:0/2087378090 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55927eeee250 con 0x55927eee9ca0 2024-04-02T14:00:07.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.970+0000 7fda01b2c640 1 --2- 172.21.15.52:0/2087378090 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee9ca0 0x55927eeeda50 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4005eb0 tx=0x7fd9f4003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ec17c034667082 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:07.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.970+0000 7fd9e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:07.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.971+0000 7fd9e4ff9640 1 -- 172.21.15.52:0/2087378090 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f40140d0 con 0x55927eee9ca0 2024-04-02T14:00:07.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.971+0000 7fd9e4ff9640 1 -- 172.21.15.52:0/2087378090 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fd9f4030480 con 0x55927eee9ca0 2024-04-02T14:00:07.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.971+0000 7fda06170740 1 -- 172.21.15.52:0/2087378090 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9f4003e50 con 0x55927eee9ca0 2024-04-02T14:00:07.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.971+0000 7fd9e4ff9640 1 -- 172.21.15.52:0/2087378090 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd9f4014270 con 0x55927eee9ca0 2024-04-02T14:00:07.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.971+0000 7fda06170740 10 client.17484 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:07.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.971+0000 7fda06170740 1 -- 172.21.15.52:0/2087378090 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55927ef60360 con 0x55927eee9ca0 2024-04-02T14:00:07.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.971+0000 7fd9e4ff9640 1 -- 172.21.15.52:0/2087378090 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9f4003e50 con 0x55927eee9ca0 2024-04-02T14:00:07.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.971+0000 7fd9e4ff9640 1 -- 172.21.15.52:0/2087378090 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd9f4038070 con 0x55927eee9ca0 2024-04-02T14:00:07.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.971+0000 7fda06170740 10 client.17484 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:07.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.971+0000 7fda06170740 10 client.17484 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T14:00:07.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.971+0000 7fda06170740 20 client.17484 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:07.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.971+0000 7fda06170740 1 --2- 172.21.15.52:0/2087378090 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55927ef60830 0x55927ef62c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:07.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.971+0000 7fda06170740 4 client.17484 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:00:07.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.971+0000 7fda06170740 1 -- 172.21.15.52:0/2087378090 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55927eee1950 con 0x55927ef60830 2024-04-02T14:00:07.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.972+0000 7fda0132b640 1 --2- 172.21.15.52:0/2087378090 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55927ef60830 0x55927ef62c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:07.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.972+0000 7fda0132b640 1 --2- 172.21.15.52:0/2087378090 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55927ef60830 0x55927ef62c20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bd90ee5475dcf190 server_cookie=344eba4e6818bf25 in_seq=0 out_seq=0 2024-04-02T14:00:07.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.972+0000 7fd9e4ff9640 10 client.17484 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T14:00:07.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.972+0000 7fd9e4ff9640 1 -- 172.21.15.52:0/2087378090 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55927eee1950 con 0x55927ef60830 2024-04-02T14:00:07.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.972+0000 7fd9e4ff9640 10 client.17484 handle_command_reply: tid=0 2024-04-02T14:00:07.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.972+0000 7fda06170740 1 client.17484 shutdown 2024-04-02T14:00:07.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.972+0000 7fda06170740 1 -- 172.21.15.52:0/2087378090 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee9ca0 msgr2=0x55927eeeda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:07.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.972+0000 7fda06170740 1 --2- 172.21.15.52:0/2087378090 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee9ca0 0x55927eeeda50 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4005eb0 tx=0x7fd9f4003dc0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.972+0000 7fda06170740 1 -- 172.21.15.52:0/2087378090 shutdown_connections 2024-04-02T14:00:07.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.972+0000 7fda06170740 1 --2- 172.21.15.52:0/2087378090 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55927eee9ca0 0x55927eeeda50 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.972+0000 7fda06170740 1 --2- 172.21.15.52:0/2087378090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55927eee10c0 0x55927eeed510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.972+0000 7fda06170740 1 --2- 172.21.15.52:0/2087378090 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55927ef60830 0x55927ef62c20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.972+0000 7fda06170740 1 --2- 172.21.15.52:0/2087378090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927eee07a0 0x55927eef2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.972+0000 7fda06170740 1 -- 172.21.15.52:0/2087378090 >> 172.21.15.52:0/2087378090 conn(0x55927eedaed0 msgr2=0x55927eedc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:07.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.972+0000 7fd9e4ff9640 0 client.17484 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T14:00:07.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.972+0000 7fda06170740 1 -- 172.21.15.52:0/2087378090 shutdown_connections 2024-04-02T14:00:07.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.972+0000 7fda06170740 1 -- 172.21.15.52:0/2087378090 wait complete. 2024-04-02T14:00:07.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.972+0000 7fda06170740 20 client.17484 trim_cache size 0 max 16384 2024-04-02T14:00:07.989 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:00:07.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.973+0000 7fda0358f640 1 -- 172.21.15.52:0/1046794242 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd9e80a3d20 msgr2=0x7fd9e80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:07.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.973+0000 7fda0358f640 1 --2- 172.21.15.52:0/1046794242 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd9e80a3d20 0x7fd9e80a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4014a40 tx=0x7fd9f4009c70 comp rx=0 tx=0).stop 2024-04-02T14:00:07.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.973+0000 7fda0358f640 1 -- 172.21.15.52:0/1046794242 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9fc100a20 msgr2=0x7fd9fc197030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:07.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.973+0000 7fda0358f640 1 --2- 172.21.15.52:0/1046794242 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9fc100a20 0x7fd9fc197030 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009230 tx=0x7fd9f802cc70 comp rx=0 tx=0).stop 2024-04-02T14:00:07.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.973+0000 7fda0358f640 1 -- 172.21.15.52:0/1046794242 shutdown_connections 2024-04-02T14:00:07.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.973+0000 7fda0358f640 1 --2- 172.21.15.52:0/1046794242 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd9e80a3d20 0x7fd9e80a61e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.973+0000 7fda0358f640 1 --2- 172.21.15.52:0/1046794242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9fc106a30 0x7fd9fc197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.973+0000 7fda0358f640 1 --2- 172.21.15.52:0/1046794242 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9fc1013e0 0x7fd9fc197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.973+0000 7fda0358f640 1 --2- 172.21.15.52:0/1046794242 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9fc100a20 0x7fd9fc197030 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:07.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.973+0000 7fda0358f640 1 -- 172.21.15.52:0/1046794242 >> 172.21.15.52:0/1046794242 conn(0x7fd9fc077b60 msgr2=0x7fd9fc103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:07.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.973+0000 7fda0358f640 1 -- 172.21.15.52:0/1046794242 shutdown_connections 2024-04-02T14:00:07.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:07.973+0000 7fda0358f640 1 -- 172.21.15.52:0/1046794242 wait complete. 2024-04-02T14:00:07.992 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T14:00:08.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.094+0000 7f9d9ca2b640 1 -- 172.21.15.52:0/1524110851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d98107aa0 msgr2=0x7f9d98107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.094+0000 7f9d9ca2b640 1 --2- 172.21.15.52:0/1524110851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d98107aa0 0x7f9d98107e80 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009930 tx=0x7f9d8802bc30 comp rx=0 tx=0).stop 2024-04-02T14:00:08.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.094+0000 7f9d9ca2b640 1 -- 172.21.15.52:0/1524110851 shutdown_connections 2024-04-02T14:00:08.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.094+0000 7f9d9ca2b640 1 --2- 172.21.15.52:0/1524110851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d98102450 0x7f9d9810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.094+0000 7f9d9ca2b640 1 --2- 172.21.15.52:0/1524110851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d98101a90 0x7f9d98101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.094+0000 7f9d9ca2b640 1 --2- 172.21.15.52:0/1524110851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d98107aa0 0x7f9d98107e80 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.094+0000 7f9d9ca2b640 1 -- 172.21.15.52:0/1524110851 >> 172.21.15.52:0/1524110851 conn(0x7f9d980fd5a0 msgr2=0x7f9d980ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:08.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.094+0000 7f9d9ca2b640 1 -- 172.21.15.52:0/1524110851 shutdown_connections 2024-04-02T14:00:08.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.094+0000 7f9d9ca2b640 1 -- 172.21.15.52:0/1524110851 wait complete. 2024-04-02T14:00:08.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.095+0000 7f9d9ca2b640 1 Processor -- start 2024-04-02T14:00:08.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.095+0000 7f9d9ca2b640 1 -- start start 2024-04-02T14:00:08.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.095+0000 7f9d9ca2b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d98101a90 0x7f9d981983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.095+0000 7f9d9ca2b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d98102450 0x7f9d981988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.095+0000 7f9d9ca2b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d98107aa0 0x7f9d9819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.095+0000 7f9d9ca2b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9d98111e80 con 0x7f9d98102450 2024-04-02T14:00:08.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.095+0000 7f9d9ca2b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9d98111d00 con 0x7f9d98101a90 2024-04-02T14:00:08.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.095+0000 7f9d9ca2b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9d98112000 con 0x7f9d98107aa0 2024-04-02T14:00:08.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.095+0000 7f9d95d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d98102450 0x7f9d981988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.095+0000 7f9d95d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d98102450 0x7f9d981988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34428/0 (socket says 172.21.15.52:34428) 2024-04-02T14:00:08.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.095+0000 7f9d95d9b640 1 -- 172.21.15.52:0/2404302595 learned_addr learned my addr 172.21.15.52:0/2404302595 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:08.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.095+0000 7f9d9659c640 1 --2- 172.21.15.52:0/2404302595 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d98101a90 0x7f9d981983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.095+0000 7f9d96d9d640 1 --2- 172.21.15.52:0/2404302595 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d98107aa0 0x7f9d9819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.095+0000 7f9d95d9b640 1 -- 172.21.15.52:0/2404302595 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d98107aa0 msgr2=0x7f9d9819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.095+0000 7f9d95d9b640 1 --2- 172.21.15.52:0/2404302595 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d98107aa0 0x7f9d9819cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.095+0000 7f9d95d9b640 1 -- 172.21.15.52:0/2404302595 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d98101a90 msgr2=0x7f9d981983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.095+0000 7f9d95d9b640 1 --2- 172.21.15.52:0/2404302595 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d98101a90 0x7f9d981983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.095+0000 7f9d95d9b640 1 -- 172.21.15.52:0/2404302595 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9819d340 con 0x7f9d98102450 2024-04-02T14:00:08.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.095+0000 7f9d95d9b640 1 --2- 172.21.15.52:0/2404302595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d98102450 0x7f9d981988e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00c920 tx=0x7f9d8c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:08.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.096+0000 7f9d877fe640 1 -- 172.21.15.52:0/2404302595 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c007c70 con 0x7f9d98102450 2024-04-02T14:00:08.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.096+0000 7f9d877fe640 1 -- 172.21.15.52:0/2404302595 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d8c007e10 con 0x7f9d98102450 2024-04-02T14:00:08.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.096+0000 7f9d9ca2b640 1 -- 172.21.15.52:0/2404302595 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d9819d630 con 0x7f9d98102450 2024-04-02T14:00:08.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.096+0000 7f9d877fe640 1 -- 172.21.15.52:0/2404302595 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c018890 con 0x7f9d98102450 2024-04-02T14:00:08.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.096+0000 7f9d9ca2b640 1 -- 172.21.15.52:0/2404302595 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d981a4ec0 con 0x7f9d98102450 2024-04-02T14:00:08.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.098+0000 7f9d877fe640 1 -- 172.21.15.52:0/2404302595 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9d8c018a70 con 0x7f9d98102450 2024-04-02T14:00:08.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.098+0000 7f9d9f60c740 1 Processor -- start 2024-04-02T14:00:08.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.098+0000 7f9d9f60c740 1 -- start start 2024-04-02T14:00:08.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.099+0000 7f9d9f60c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7ba7a0 0x560c9d7bab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.099+0000 7f9d9f60c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c9d7bb0c0 0x560c9d7c3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.099+0000 7f9d9f60c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c9d7c3ca0 0x560c9d7c6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.099+0000 7f9d9f60c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560c9d71e7b0 con 0x560c9d7ba7a0 2024-04-02T14:00:08.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.099+0000 7f9d9f60c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560c9d6bd570 con 0x560c9d7c3ca0 2024-04-02T14:00:08.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.099+0000 7f9d9f60c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560c9d71a760 con 0x560c9d7bb0c0 2024-04-02T14:00:08.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.099+0000 7f9d96d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7ba7a0 0x560c9d7bab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.099+0000 7f9d96d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7ba7a0 0x560c9d7bab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34444/0 (socket says 172.21.15.52:34444) 2024-04-02T14:00:08.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.099+0000 7f9d96d9d640 1 -- 172.21.15.52:0/2575736066 learned_addr learned my addr 172.21.15.52:0/2575736066 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:08.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.099+0000 7f9d877fe640 1 --2- 172.21.15.52:0/2404302595 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9d800a3d20 0x7f9d800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.099+0000 7f9d9659c640 1 --2- 172.21.15.52:0/2575736066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c9d7bb0c0 0x560c9d7c3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.099+0000 7f9d96d9d640 1 --2- 172.21.15.52:0/2404302595 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9d800a3d20 0x7f9d800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.099+0000 7f9d877fe640 1 -- 172.21.15.52:0/2404302595 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f9d8c0d0c70 con 0x7f9d98102450 2024-04-02T14:00:08.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.099+0000 7f9d95d9b640 1 --2- 172.21.15.52:0/2575736066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c9d7c3ca0 0x560c9d7c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.099+0000 7f9d96d9d640 1 -- 172.21.15.52:0/2575736066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c9d7bb0c0 msgr2=0x560c9d7c3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.099+0000 7f9d96d9d640 1 --2- 172.21.15.52:0/2575736066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c9d7bb0c0 0x560c9d7c3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.099+0000 7f9d96d9d640 1 -- 172.21.15.52:0/2575736066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c9d7c3ca0 msgr2=0x560c9d7c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.099+0000 7f9d96d9d640 1 --2- 172.21.15.52:0/2575736066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c9d7c3ca0 0x560c9d7c6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.099+0000 7f9d96d9d640 1 -- 172.21.15.52:0/2575736066 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c9d4323f0 con 0x560c9d7ba7a0 2024-04-02T14:00:08.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.099+0000 7f9d9659c640 1 --2- 172.21.15.52:0/2575736066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c9d7bb0c0 0x560c9d7c3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:00:08.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.099+0000 7f9d96d9d640 1 --2- 172.21.15.52:0/2575736066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7ba7a0 0x560c9d7bab80 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f9d900129f0 tx=0x7f9d90012eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3bc7473a4142d90 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:08.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.100+0000 7f9d84ff9640 1 -- 172.21.15.52:0/2575736066 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90023070 con 0x560c9d7ba7a0 2024-04-02T14:00:08.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.100+0000 7f9d84ff9640 1 -- 172.21.15.52:0/2575736066 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d90004710 con 0x560c9d7ba7a0 2024-04-02T14:00:08.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.100+0000 7f9d84ff9640 1 -- 172.21.15.52:0/2575736066 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9001fa90 con 0x560c9d7ba7a0 2024-04-02T14:00:08.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.100+0000 7f9d9f60c740 1 -- 172.21.15.52:0/2575736066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7ba7a0 msgr2=0x560c9d7bab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.100+0000 7f9d96d9d640 1 --2- 172.21.15.52:0/2404302595 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9d800a3d20 0x7f9d800a61e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9d90022000 tx=0x7f9d90021000 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:08.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.100+0000 7f9d9f60c740 1 --2- 172.21.15.52:0/2575736066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7ba7a0 0x560c9d7bab80 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f9d900129f0 tx=0x7f9d90012eb0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.100+0000 7f9d9f60c740 1 -- 172.21.15.52:0/2575736066 shutdown_connections 2024-04-02T14:00:08.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.100+0000 7f9d9f60c740 1 --2- 172.21.15.52:0/2575736066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c9d7c3ca0 0x560c9d7c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.100+0000 7f9d9f60c740 1 --2- 172.21.15.52:0/2575736066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c9d7bb0c0 0x560c9d7c3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.100+0000 7f9d9f60c740 1 --2- 172.21.15.52:0/2575736066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7ba7a0 0x560c9d7bab80 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.100+0000 7f9d9f60c740 1 -- 172.21.15.52:0/2575736066 >> 172.21.15.52:0/2575736066 conn(0x560c9d7b4ed0 msgr2=0x560c9d7b52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:08.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.100+0000 7f9d9f60c740 1 -- 172.21.15.52:0/2575736066 shutdown_connections 2024-04-02T14:00:08.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.100+0000 7f9d9f60c740 1 -- 172.21.15.52:0/2575736066 wait complete. 2024-04-02T14:00:08.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.101+0000 7f9d9f60c740 1 -- start start 2024-04-02T14:00:08.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.101+0000 7f9d9f60c740 1 Processor -- start 2024-04-02T14:00:08.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.101+0000 7f9d9f60c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c9d7ba7a0 0x560c9d7c3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.101+0000 7f9d9f60c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c9d7bb0c0 0x560c9d7c0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.101+0000 7f9d9f60c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7c3ca0 0x560c9d7c1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.101+0000 7f9d9f60c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560c9d7c6a90 con 0x560c9d7c3ca0 2024-04-02T14:00:08.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.101+0000 7f9d9f60c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560c9d6d9e80 con 0x560c9d7ba7a0 2024-04-02T14:00:08.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.101+0000 7f9d9f60c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560c9d7c6c10 con 0x560c9d7bb0c0 2024-04-02T14:00:08.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.101+0000 7f9d9f60c740 10 client.0 _ll_register_callbacks cb 0x560c9d715aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:08.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.101+0000 7f9d96d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7c3ca0 0x560c9d7c1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.101+0000 7f9d96d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7c3ca0 0x560c9d7c1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34460/0 (socket says 172.21.15.52:34460) 2024-04-02T14:00:08.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.101+0000 7f9d96d9d640 1 -- 172.21.15.52:0/3758225483 learned_addr learned my addr 172.21.15.52:0/3758225483 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:08.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.101+0000 7f9d9659c640 1 --2- 172.21.15.52:0/3758225483 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c9d7ba7a0 0x560c9d7c3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.101+0000 7f9d96d9d640 1 -- 172.21.15.52:0/3758225483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c9d7bb0c0 msgr2=0x560c9d7c0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.101+0000 7f9d95d9b640 1 --2- 172.21.15.52:0/3758225483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c9d7bb0c0 0x560c9d7c0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.101+0000 7f9d96d9d640 1 --2- 172.21.15.52:0/3758225483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c9d7bb0c0 0x560c9d7c0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.101+0000 7f9d96d9d640 1 -- 172.21.15.52:0/3758225483 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c9d7ba7a0 msgr2=0x560c9d7c3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.101+0000 7f9d96d9d640 1 --2- 172.21.15.52:0/3758225483 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c9d7ba7a0 0x560c9d7c3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.101+0000 7f9d96d9d640 1 -- 172.21.15.52:0/3758225483 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c9d4fc7d0 con 0x560c9d7c3ca0 2024-04-02T14:00:08.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.101+0000 7f9d9659c640 1 --2- 172.21.15.52:0/3758225483 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c9d7ba7a0 0x560c9d7c3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:00:08.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.102+0000 7f9d96d9d640 1 --2- 172.21.15.52:0/3758225483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7c3ca0 0x560c9d7c1010 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f9d9001f610 tx=0x7f9d90004b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9bc9fd7b66d681 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:08.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.102+0000 7f9d7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:08.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.102+0000 7f9d7dffb640 1 -- 172.21.15.52:0/3758225483 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90023030 con 0x560c9d7c3ca0 2024-04-02T14:00:08.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.102+0000 7f9d7dffb640 1 -- 172.21.15.52:0/3758225483 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f9d900040c0 con 0x560c9d7c3ca0 2024-04-02T14:00:08.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.102+0000 7f9d9f60c740 1 -- 172.21.15.52:0/3758225483 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d90004710 con 0x560c9d7c3ca0 2024-04-02T14:00:08.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.102+0000 7f9d7dffb640 1 -- 172.21.15.52:0/3758225483 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9d900042c0 con 0x560c9d7c3ca0 2024-04-02T14:00:08.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.102+0000 7f9d9f60c740 10 client.17496 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:08.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.102+0000 7f9d9f60c740 1 -- 172.21.15.52:0/3758225483 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560c9d7d2a90 con 0x560c9d7c3ca0 2024-04-02T14:00:08.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.102+0000 7f9d7dffb640 1 -- 172.21.15.52:0/3758225483 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d90004710 con 0x560c9d7c3ca0 2024-04-02T14:00:08.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.102+0000 7f9d7dffb640 1 -- 172.21.15.52:0/3758225483 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9d9002e020 con 0x560c9d7c3ca0 2024-04-02T14:00:08.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.102+0000 7f9d9f60c740 10 client.17496 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:08.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.102+0000 7f9d9f60c740 10 client.17496 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T14:00:08.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.102+0000 7f9d9f60c740 20 client.17496 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:08.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.102+0000 7f9d9f60c740 1 --2- 172.21.15.52:0/3758225483 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x560c9d7d3de0 0x560c9d7d61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.102+0000 7f9d9f60c740 4 client.17496 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:00:08.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.102+0000 7f9d9f60c740 1 -- 172.21.15.52:0/3758225483 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c9d72bcf0 con 0x560c9d7d3de0 2024-04-02T14:00:08.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.103+0000 7f9d9659c640 1 --2- 172.21.15.52:0/3758225483 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x560c9d7d3de0 0x560c9d7d61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.103+0000 7f9d9659c640 1 --2- 172.21.15.52:0/3758225483 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x560c9d7d3de0 0x560c9d7d61d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=830028bc72e64c42 server_cookie=f18c90f4c0d30fba in_seq=0 out_seq=0 2024-04-02T14:00:08.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.103+0000 7f9d7dffb640 10 client.17496 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T14:00:08.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.104+0000 7f9d7dffb640 1 -- 172.21.15.52:0/3758225483 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560c9d72bcf0 con 0x560c9d7d3de0 2024-04-02T14:00:08.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.104+0000 7f9d7dffb640 10 client.17496 handle_command_reply: tid=0 2024-04-02T14:00:08.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.104+0000 7f9d9f60c740 1 client.17496 shutdown 2024-04-02T14:00:08.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.104+0000 7f9d9f60c740 1 -- 172.21.15.52:0/3758225483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7c3ca0 msgr2=0x560c9d7c1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.104+0000 7f9d9f60c740 1 --2- 172.21.15.52:0/3758225483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7c3ca0 0x560c9d7c1010 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f9d9001f610 tx=0x7f9d90004b70 comp rx=0 tx=0).stop 2024-04-02T14:00:08.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.104+0000 7f9d9f60c740 1 -- 172.21.15.52:0/3758225483 shutdown_connections 2024-04-02T14:00:08.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.104+0000 7f9d9f60c740 1 --2- 172.21.15.52:0/3758225483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7c3ca0 0x560c9d7c1010 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.104+0000 7f9d9f60c740 1 --2- 172.21.15.52:0/3758225483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c9d7bb0c0 0x560c9d7c0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.104+0000 7f9d9f60c740 1 --2- 172.21.15.52:0/3758225483 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x560c9d7d3de0 0x560c9d7d61d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.104+0000 7f9d9f60c740 1 --2- 172.21.15.52:0/3758225483 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c9d7ba7a0 0x560c9d7c3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.104+0000 7f9d7dffb640 0 client.17496 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T14:00:08.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.104+0000 7f9d9f60c740 1 -- 172.21.15.52:0/3758225483 >> 172.21.15.52:0/3758225483 conn(0x560c9d7b4ed0 msgr2=0x560c9d7b6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:08.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.104+0000 7f9d9f60c740 1 -- 172.21.15.52:0/3758225483 shutdown_connections 2024-04-02T14:00:08.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.104+0000 7f9d9f60c740 1 -- 172.21.15.52:0/3758225483 wait complete. 2024-04-02T14:00:08.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.104+0000 7f9d9f60c740 20 client.17496 trim_cache size 0 max 16384 2024-04-02T14:00:08.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.117+0000 7f9d9f60c740 1 Processor -- start 2024-04-02T14:00:08.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.117+0000 7f9d9f60c740 1 -- start start 2024-04-02T14:00:08.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.118+0000 7f9d9f60c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c9d7ba7a0 0x560c9d7cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.118+0000 7f9d9f60c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c9d7bb0c0 0x560c9d7c3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.118+0000 7f9d9f60c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7c3ca0 0x560c9d7c3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.118+0000 7f9d9f60c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9d981aed80 con 0x560c9d7c3ca0 2024-04-02T14:00:08.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.118+0000 7f9d9f60c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9d981b08c0 con 0x560c9d7bb0c0 2024-04-02T14:00:08.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.118+0000 7f9d9f60c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9d981af450 con 0x560c9d7ba7a0 2024-04-02T14:00:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.118+0000 7f9d96d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7c3ca0 0x560c9d7c3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.118+0000 7f9d96d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7c3ca0 0x560c9d7c3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34472/0 (socket says 172.21.15.52:34472) 2024-04-02T14:00:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.118+0000 7f9d96d9d640 1 -- 172.21.15.52:0/3596094769 learned_addr learned my addr 172.21.15.52:0/3596094769 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.118+0000 7f9d96d9d640 1 -- 172.21.15.52:0/3596094769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c9d7ba7a0 msgr2=0x560c9d7cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.118+0000 7f9d9659c640 1 --2- 172.21.15.52:0/3596094769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c9d7ba7a0 0x560c9d7cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.118+0000 7f9d95d9b640 1 --2- 172.21.15.52:0/3596094769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c9d7bb0c0 0x560c9d7c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.118+0000 7f9d96d9d640 1 --2- 172.21.15.52:0/3596094769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c9d7ba7a0 0x560c9d7cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.118+0000 7f9d96d9d640 1 -- 172.21.15.52:0/3596094769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c9d7bb0c0 msgr2=0x560c9d7c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.118+0000 7f9d96d9d640 1 --2- 172.21.15.52:0/3596094769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c9d7bb0c0 0x560c9d7c3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.118+0000 7f9d96d9d640 1 -- 172.21.15.52:0/3596094769 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c9d7bb810 con 0x560c9d7c3ca0 2024-04-02T14:00:08.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.118+0000 7f9d95d9b640 1 --2- 172.21.15.52:0/3596094769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c9d7bb0c0 0x560c9d7c3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:00:08.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.118+0000 7f9d96d9d640 1 --2- 172.21.15.52:0/3596094769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7c3ca0 0x560c9d7c3670 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f9d900129f0 tx=0x7f9d90009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f029de39d8d5b340 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:08.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.118+0000 7f9d7ffff640 1 -- 172.21.15.52:0/3596094769 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90023070 con 0x560c9d7c3ca0 2024-04-02T14:00:08.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.118+0000 7f9d7ffff640 1 -- 172.21.15.52:0/3596094769 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d9000e720 con 0x560c9d7c3ca0 2024-04-02T14:00:08.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.118+0000 7f9d7ffff640 1 -- 172.21.15.52:0/3596094769 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90009a30 con 0x560c9d7c3ca0 2024-04-02T14:00:08.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.119+0000 7f9d9f60c740 1 -- 172.21.15.52:0/3596094769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7c3ca0 msgr2=0x560c9d7c3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.119+0000 7f9d9f60c740 1 --2- 172.21.15.52:0/3596094769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7c3ca0 0x560c9d7c3670 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f9d900129f0 tx=0x7f9d90009480 comp rx=0 tx=0).stop 2024-04-02T14:00:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.119+0000 7f9d9f60c740 1 -- 172.21.15.52:0/3596094769 shutdown_connections 2024-04-02T14:00:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.119+0000 7f9d9f60c740 1 --2- 172.21.15.52:0/3596094769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7c3ca0 0x560c9d7c3670 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.119+0000 7f9d9f60c740 1 --2- 172.21.15.52:0/3596094769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c9d7bb0c0 0x560c9d7c3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.119+0000 7f9d9f60c740 1 --2- 172.21.15.52:0/3596094769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c9d7ba7a0 0x560c9d7cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.119+0000 7f9d9f60c740 1 -- 172.21.15.52:0/3596094769 >> 172.21.15.52:0/3596094769 conn(0x560c9d7b4ed0 msgr2=0x560c9d7b6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.119+0000 7f9d9f60c740 1 -- 172.21.15.52:0/3596094769 shutdown_connections 2024-04-02T14:00:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.119+0000 7f9d9f60c740 1 -- 172.21.15.52:0/3596094769 wait complete. 2024-04-02T14:00:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.119+0000 7f9d9f60c740 1 -- start start 2024-04-02T14:00:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.119+0000 7f9d9f60c740 1 Processor -- start 2024-04-02T14:00:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.119+0000 7f9d9f60c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c9d7ba7a0 0x560c9d7cc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.119+0000 7f9d9f60c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7bb0c0 0x560c9d7c4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.119+0000 7f9d9f60c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c9d7c3ca0 0x560c9d7c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.120+0000 7f9d9f60c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9d981b01f0 con 0x560c9d7bb0c0 2024-04-02T14:00:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.120+0000 7f9d9f60c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9d981afb20 con 0x560c9d7ba7a0 2024-04-02T14:00:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.120+0000 7f9d9f60c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9d981acd60 con 0x560c9d7c3ca0 2024-04-02T14:00:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.120+0000 7f9d9f60c740 10 client.0 _ll_register_callbacks cb 0x7f9d88009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.120+0000 7f9d95d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7bb0c0 0x560c9d7c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.120+0000 7f9d95d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7bb0c0 0x560c9d7c4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34484/0 (socket says 172.21.15.52:34484) 2024-04-02T14:00:08.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.120+0000 7f9d95d9b640 1 -- 172.21.15.52:0/1486151511 learned_addr learned my addr 172.21.15.52:0/1486151511 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:08.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.120+0000 7f9d9659c640 1 --2- 172.21.15.52:0/1486151511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c9d7ba7a0 0x560c9d7cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.120+0000 7f9d95d9b640 1 -- 172.21.15.52:0/1486151511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c9d7c3ca0 msgr2=0x560c9d7c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.120+0000 7f9d96d9d640 1 --2- 172.21.15.52:0/1486151511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c9d7c3ca0 0x560c9d7c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.120+0000 7f9d95d9b640 1 --2- 172.21.15.52:0/1486151511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c9d7c3ca0 0x560c9d7c4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.120+0000 7f9d95d9b640 1 -- 172.21.15.52:0/1486151511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c9d7ba7a0 msgr2=0x560c9d7cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.120+0000 7f9d95d9b640 1 --2- 172.21.15.52:0/1486151511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c9d7ba7a0 0x560c9d7cc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.120+0000 7f9d95d9b640 1 -- 172.21.15.52:0/1486151511 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c9d7c5300 con 0x560c9d7bb0c0 2024-04-02T14:00:08.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.120+0000 7f9d9659c640 1 --2- 172.21.15.52:0/1486151511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c9d7ba7a0 0x560c9d7cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:00:08.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.120+0000 7f9d95d9b640 1 --2- 172.21.15.52:0/1486151511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7bb0c0 0x560c9d7c4680 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c09c860 tx=0x7f9d8c09c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6833fa70382f6781 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:08.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.120+0000 7f9d7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:08.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.120+0000 7f9d7dffb640 1 -- 172.21.15.52:0/1486151511 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c0a0c80 con 0x560c9d7bb0c0 2024-04-02T14:00:08.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.120+0000 7f9d7dffb640 1 -- 172.21.15.52:0/1486151511 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f9d8c098f40 con 0x560c9d7bb0c0 2024-04-02T14:00:08.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.120+0000 7f9d9f60c740 1 -- 172.21.15.52:0/1486151511 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d9000e720 con 0x560c9d7bb0c0 2024-04-02T14:00:08.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.120+0000 7f9d7dffb640 1 -- 172.21.15.52:0/1486151511 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9d8c0a5540 con 0x560c9d7bb0c0 2024-04-02T14:00:08.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.120+0000 7f9d9f60c740 10 client.17502 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:08.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.120+0000 7f9d9f60c740 1 -- 172.21.15.52:0/1486151511 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560c9d83a310 con 0x560c9d7bb0c0 2024-04-02T14:00:08.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.120+0000 7f9d7dffb640 1 -- 172.21.15.52:0/1486151511 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d9000e720 con 0x560c9d7bb0c0 2024-04-02T14:00:08.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.121+0000 7f9d7dffb640 1 -- 172.21.15.52:0/1486151511 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9d8c0a50b0 con 0x560c9d7bb0c0 2024-04-02T14:00:08.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.121+0000 7f9d9f60c740 10 client.17502 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:08.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.121+0000 7f9d9f60c740 10 client.17502 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T14:00:08.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.121+0000 7f9d9f60c740 20 client.17502 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:08.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.121+0000 7f9d9f60c740 1 --2- 172.21.15.52:0/1486151511 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x560c9d83a9e0 0x560c9d83cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.121+0000 7f9d9f60c740 4 client.17502 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:00:08.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.121+0000 7f9d9f60c740 1 -- 172.21.15.52:0/1486151511 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560c9d726040 con 0x560c9d83a9e0 2024-04-02T14:00:08.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.121+0000 7f9d9659c640 1 --2- 172.21.15.52:0/1486151511 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x560c9d83a9e0 0x560c9d83cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.121+0000 7f9d7dffb640 10 client.17502 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T14:00:08.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.121+0000 7f9d9659c640 1 --2- 172.21.15.52:0/1486151511 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x560c9d83a9e0 0x560c9d83cdd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d3a80d3dad5db5df server_cookie=efa7afcf9573c871 in_seq=0 out_seq=0 2024-04-02T14:00:08.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.121+0000 7f9d7dffb640 1 -- 172.21.15.52:0/1486151511 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560c9d726040 con 0x560c9d83a9e0 2024-04-02T14:00:08.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.121+0000 7f9d7dffb640 10 client.17502 handle_command_reply: tid=0 2024-04-02T14:00:08.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.121+0000 7f9d9f60c740 1 client.17502 shutdown 2024-04-02T14:00:08.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.122+0000 7f9d9f60c740 1 -- 172.21.15.52:0/1486151511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7bb0c0 msgr2=0x560c9d7c4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.122+0000 7f9d9f60c740 1 --2- 172.21.15.52:0/1486151511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7bb0c0 0x560c9d7c4680 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c09c860 tx=0x7f9d8c09c4d0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.122+0000 7f9d9f60c740 1 -- 172.21.15.52:0/1486151511 shutdown_connections 2024-04-02T14:00:08.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.122+0000 7f9d9f60c740 1 --2- 172.21.15.52:0/1486151511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c9d7c3ca0 0x560c9d7c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.122+0000 7f9d9f60c740 1 --2- 172.21.15.52:0/1486151511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c9d7bb0c0 0x560c9d7c4680 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.122+0000 7f9d9f60c740 1 --2- 172.21.15.52:0/1486151511 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x560c9d83a9e0 0x560c9d83cdd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.122+0000 7f9d9f60c740 1 --2- 172.21.15.52:0/1486151511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c9d7ba7a0 0x560c9d7cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.122+0000 7f9d7dffb640 0 client.17502 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T14:00:08.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.122+0000 7f9d9f60c740 1 -- 172.21.15.52:0/1486151511 >> 172.21.15.52:0/1486151511 conn(0x560c9d7b4ed0 msgr2=0x560c9d7bf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:08.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.122+0000 7f9d9f60c740 1 -- 172.21.15.52:0/1486151511 shutdown_connections 2024-04-02T14:00:08.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.122+0000 7f9d9f60c740 1 -- 172.21.15.52:0/1486151511 wait complete. 2024-04-02T14:00:08.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.122+0000 7f9d9f60c740 20 client.17502 trim_cache size 0 max 16384 2024-04-02T14:00:08.142 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:00:08.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.122+0000 7f9d9ca2b640 1 -- 172.21.15.52:0/2404302595 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9d800a3d20 msgr2=0x7f9d800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.122+0000 7f9d9ca2b640 1 --2- 172.21.15.52:0/2404302595 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9d800a3d20 0x7f9d800a61e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9d90022000 tx=0x7f9d90021000 comp rx=0 tx=0).stop 2024-04-02T14:00:08.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.122+0000 7f9d9ca2b640 1 -- 172.21.15.52:0/2404302595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d98102450 msgr2=0x7f9d981988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.122+0000 7f9d9ca2b640 1 --2- 172.21.15.52:0/2404302595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d98102450 0x7f9d981988e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00c920 tx=0x7f9d8c00cde0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.122+0000 7f9d9ca2b640 1 -- 172.21.15.52:0/2404302595 shutdown_connections 2024-04-02T14:00:08.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.122+0000 7f9d9ca2b640 1 --2- 172.21.15.52:0/2404302595 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9d800a3d20 0x7f9d800a61e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.122+0000 7f9d9ca2b640 1 --2- 172.21.15.52:0/2404302595 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d98107aa0 0x7f9d9819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.122+0000 7f9d9ca2b640 1 --2- 172.21.15.52:0/2404302595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d98102450 0x7f9d981988e0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.122+0000 7f9d9ca2b640 1 --2- 172.21.15.52:0/2404302595 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d98101a90 0x7f9d981983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.122+0000 7f9d9ca2b640 1 -- 172.21.15.52:0/2404302595 >> 172.21.15.52:0/2404302595 conn(0x7f9d980fd5a0 msgr2=0x7f9d980fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:08.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.122+0000 7f9d9ca2b640 1 -- 172.21.15.52:0/2404302595 shutdown_connections 2024-04-02T14:00:08.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.122+0000 7f9d9ca2b640 1 -- 172.21.15.52:0/2404302595 wait complete. 2024-04-02T14:00:08.145 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T14:00:08.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.246+0000 7f2dae711640 1 -- 172.21.15.52:0/3615945222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2da8107aa0 msgr2=0x7f2da8107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.246+0000 7f2dae711640 1 --2- 172.21.15.52:0/3615945222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2da8107aa0 0x7f2da8107e80 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009930 tx=0x7f2d9802bc30 comp rx=0 tx=0).stop 2024-04-02T14:00:08.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.246+0000 7f2dae711640 1 -- 172.21.15.52:0/3615945222 shutdown_connections 2024-04-02T14:00:08.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.246+0000 7f2dae711640 1 --2- 172.21.15.52:0/3615945222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2da8102450 0x7f2da810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.246+0000 7f2dae711640 1 --2- 172.21.15.52:0/3615945222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2da8101a90 0x7f2da8101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.246+0000 7f2dae711640 1 --2- 172.21.15.52:0/3615945222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2da8107aa0 0x7f2da8107e80 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.246+0000 7f2dae711640 1 -- 172.21.15.52:0/3615945222 >> 172.21.15.52:0/3615945222 conn(0x7f2da8078f00 msgr2=0x7f2da8079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:08.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.246+0000 7f2dae711640 1 -- 172.21.15.52:0/3615945222 shutdown_connections 2024-04-02T14:00:08.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.246+0000 7f2dae711640 1 -- 172.21.15.52:0/3615945222 wait complete. 2024-04-02T14:00:08.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.247+0000 7f2dae711640 1 Processor -- start 2024-04-02T14:00:08.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.247+0000 7f2dae711640 1 -- start start 2024-04-02T14:00:08.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.247+0000 7f2dae711640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2da8101a90 0x7f2da8073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.247+0000 7f2dae711640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2da8102450 0x7f2da8074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.247+0000 7f2dae711640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2da8107aa0 0x7f2da8074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.247+0000 7f2dae711640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2da8111eb0 con 0x7f2da8107aa0 2024-04-02T14:00:08.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.247+0000 7f2dae711640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2da8111d30 con 0x7f2da8101a90 2024-04-02T14:00:08.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.247+0000 7f2dae711640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2da8112030 con 0x7f2da8102450 2024-04-02T14:00:08.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.247+0000 7f2daccae640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2da8107aa0 0x7f2da8074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.247+0000 7f2daccae640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2da8107aa0 0x7f2da8074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34506/0 (socket says 172.21.15.52:34506) 2024-04-02T14:00:08.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.247+0000 7f2daccae640 1 -- 172.21.15.52:0/918272579 learned_addr learned my addr 172.21.15.52:0/918272579 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:08.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.247+0000 7f2da7fff640 1 --2- 172.21.15.52:0/918272579 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2da8101a90 0x7f2da8073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.247+0000 7f2da77fe640 1 --2- 172.21.15.52:0/918272579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2da8102450 0x7f2da8074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.247+0000 7f2daccae640 1 -- 172.21.15.52:0/918272579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2da8102450 msgr2=0x7f2da8074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.247+0000 7f2daccae640 1 --2- 172.21.15.52:0/918272579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2da8102450 0x7f2da8074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.247+0000 7f2daccae640 1 -- 172.21.15.52:0/918272579 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2da8101a90 msgr2=0x7f2da8073e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.247+0000 7f2daccae640 1 --2- 172.21.15.52:0/918272579 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2da8101a90 0x7f2da8073e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.247+0000 7f2daccae640 1 -- 172.21.15.52:0/918272579 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da81a6870 con 0x7f2da8107aa0 2024-04-02T14:00:08.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.247+0000 7f2da77fe640 1 --2- 172.21.15.52:0/918272579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2da8102450 0x7f2da8074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:00:08.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.248+0000 7f2daccae640 1 --2- 172.21.15.52:0/918272579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2da8107aa0 0x7f2da8074880 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f2da000eaa0 tx=0x7f2da000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:08.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.248+0000 7f2da57fa640 1 -- 172.21.15.52:0/918272579 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da000cdb0 con 0x7f2da8107aa0 2024-04-02T14:00:08.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.248+0000 7f2dae711640 1 -- 172.21.15.52:0/918272579 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da81a6b60 con 0x7f2da8107aa0 2024-04-02T14:00:08.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.248+0000 7f2da57fa640 1 -- 172.21.15.52:0/918272579 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2da0005020 con 0x7f2da8107aa0 2024-04-02T14:00:08.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.248+0000 7f2da57fa640 1 -- 172.21.15.52:0/918272579 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da00116d0 con 0x7f2da8107aa0 2024-04-02T14:00:08.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.248+0000 7f2dae711640 1 -- 172.21.15.52:0/918272579 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da81a70b0 con 0x7f2da8107aa0 2024-04-02T14:00:08.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.250+0000 7f2da57fa640 1 -- 172.21.15.52:0/918272579 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2da00054d0 con 0x7f2da8107aa0 2024-04-02T14:00:08.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.250+0000 7f2db13d0740 1 Processor -- start 2024-04-02T14:00:08.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.250+0000 7f2db13d0740 1 -- start start 2024-04-02T14:00:08.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.250+0000 7f2db13d0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5580a8efa7a0 0x5580a8efab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.250+0000 7f2db13d0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580a8efb0c0 0x5580a8f03760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.250+0000 7f2db13d0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8f03ca0 0x5580a8f06090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.251+0000 7f2db13d0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5580a8e5e7b0 con 0x5580a8f03ca0 2024-04-02T14:00:08.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.251+0000 7f2db13d0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5580a8dfd570 con 0x5580a8efa7a0 2024-04-02T14:00:08.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.251+0000 7f2db13d0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5580a8e5a760 con 0x5580a8efb0c0 2024-04-02T14:00:08.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.251+0000 7f2da57fa640 1 --2- 172.21.15.52:0/918272579 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2d900a3d20 0x7f2d900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.251+0000 7f2da77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8f03ca0 0x5580a8f06090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.251+0000 7f2daccae640 1 --2- 172.21.15.52:0/918272579 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2d900a3d20 0x7f2d900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.251+0000 7f2da7fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580a8efb0c0 0x5580a8f03760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.251+0000 7f2da77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8f03ca0 0x5580a8f06090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34512/0 (socket says 172.21.15.52:34512) 2024-04-02T14:00:08.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.251+0000 7f2da77fe640 1 -- 172.21.15.52:0/3993662463 learned_addr learned my addr 172.21.15.52:0/3993662463 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:08.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.251+0000 7f2da57fa640 1 -- 172.21.15.52:0/918272579 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f2da00d0d70 con 0x7f2da8107aa0 2024-04-02T14:00:08.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.251+0000 7f2da77fe640 1 --2- 172.21.15.52:0/3993662463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5580a8efa7a0 0x5580a8efab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.251+0000 7f2da7fff640 1 -- 172.21.15.52:0/3993662463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5580a8efa7a0 msgr2=0x5580a8efab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.251+0000 7f2da7fff640 1 --2- 172.21.15.52:0/3993662463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5580a8efa7a0 0x5580a8efab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.251+0000 7f2da7fff640 1 -- 172.21.15.52:0/3993662463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8f03ca0 msgr2=0x5580a8f06090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.251+0000 7f2da7fff640 1 --2- 172.21.15.52:0/3993662463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8f03ca0 0x5580a8f06090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.251+0000 7f2da7fff640 1 -- 172.21.15.52:0/3993662463 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580a8b723f0 con 0x5580a8efb0c0 2024-04-02T14:00:08.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.251+0000 7f2da77fe640 1 --2- 172.21.15.52:0/3993662463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8f03ca0 0x5580a8f06090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:00:08.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.251+0000 7f2daccae640 1 --2- 172.21.15.52:0/918272579 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2d900a3d20 0x7f2d900a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2da00a50b0 tx=0x7f2da0087040 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:08.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.251+0000 7f2da7fff640 1 --2- 172.21.15.52:0/3993662463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580a8efb0c0 0x5580a8f03760 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009230 tx=0x7f2d98005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a9d549dd3c8c3a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:08.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.251+0000 7f2d8e7fc640 1 -- 172.21.15.52:0/3993662463 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98042070 con 0x5580a8efb0c0 2024-04-02T14:00:08.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.252+0000 7f2d8e7fc640 1 -- 172.21.15.52:0/3993662463 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d9802ca80 con 0x5580a8efb0c0 2024-04-02T14:00:08.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.252+0000 7f2d8e7fc640 1 -- 172.21.15.52:0/3993662463 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98004f50 con 0x5580a8efb0c0 2024-04-02T14:00:08.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.252+0000 7f2db13d0740 1 -- 172.21.15.52:0/3993662463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580a8efb0c0 msgr2=0x5580a8f03760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.252+0000 7f2db13d0740 1 --2- 172.21.15.52:0/3993662463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580a8efb0c0 0x5580a8f03760 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009230 tx=0x7f2d98005c00 comp rx=0 tx=0).stop 2024-04-02T14:00:08.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.252+0000 7f2db13d0740 1 -- 172.21.15.52:0/3993662463 shutdown_connections 2024-04-02T14:00:08.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.252+0000 7f2db13d0740 1 --2- 172.21.15.52:0/3993662463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8f03ca0 0x5580a8f06090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.252+0000 7f2db13d0740 1 --2- 172.21.15.52:0/3993662463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580a8efb0c0 0x5580a8f03760 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.252+0000 7f2db13d0740 1 --2- 172.21.15.52:0/3993662463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5580a8efa7a0 0x5580a8efab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.252+0000 7f2db13d0740 1 -- 172.21.15.52:0/3993662463 >> 172.21.15.52:0/3993662463 conn(0x5580a8ef4ed0 msgr2=0x5580a8ef52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:08.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.252+0000 7f2db13d0740 1 -- 172.21.15.52:0/3993662463 shutdown_connections 2024-04-02T14:00:08.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.252+0000 7f2db13d0740 1 -- 172.21.15.52:0/3993662463 wait complete. 2024-04-02T14:00:08.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.252+0000 7f2db13d0740 1 -- start start 2024-04-02T14:00:08.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.253+0000 7f2db13d0740 1 Processor -- start 2024-04-02T14:00:08.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.253+0000 7f2db13d0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5580a8efa7a0 0x5580a8f036c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.253+0000 7f2db13d0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8efb0c0 0x5580a8f00a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.253+0000 7f2db13d0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580a8f03ca0 0x5580a8f01060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.253+0000 7f2db13d0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5580a8f06a30 con 0x5580a8efb0c0 2024-04-02T14:00:08.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.253+0000 7f2db13d0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5580a8e19e80 con 0x5580a8efa7a0 2024-04-02T14:00:08.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.253+0000 7f2db13d0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5580a8f06bb0 con 0x5580a8f03ca0 2024-04-02T14:00:08.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.253+0000 7f2db13d0740 10 client.0 _ll_register_callbacks cb 0x5580a8e55aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:08.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.253+0000 7f2da7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8efb0c0 0x5580a8f00a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.253+0000 7f2da7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8efb0c0 0x5580a8f00a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34526/0 (socket says 172.21.15.52:34526) 2024-04-02T14:00:08.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.253+0000 7f2da7fff640 1 -- 172.21.15.52:0/1250080996 learned_addr learned my addr 172.21.15.52:0/1250080996 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:08.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.253+0000 7f2da77fe640 1 --2- 172.21.15.52:0/1250080996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580a8f03ca0 0x5580a8f01060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.253+0000 7f2da77fe640 1 --2- 172.21.15.52:0/1250080996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5580a8efa7a0 0x5580a8f036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.253+0000 7f2da7fff640 1 -- 172.21.15.52:0/1250080996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580a8f03ca0 msgr2=0x5580a8f01060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.253+0000 7f2da7fff640 1 --2- 172.21.15.52:0/1250080996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580a8f03ca0 0x5580a8f01060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.253+0000 7f2da7fff640 1 -- 172.21.15.52:0/1250080996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5580a8efa7a0 msgr2=0x5580a8f036c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.253+0000 7f2da7fff640 1 --2- 172.21.15.52:0/1250080996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5580a8efa7a0 0x5580a8f036c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.253+0000 7f2da7fff640 1 -- 172.21.15.52:0/1250080996 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580a8c3c7d0 con 0x5580a8efb0c0 2024-04-02T14:00:08.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.253+0000 7f2da77fe640 1 --2- 172.21.15.52:0/1250080996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580a8f03ca0 0x5580a8f01060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:00:08.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.253+0000 7f2da7fff640 1 --2- 172.21.15.52:0/1250080996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8efb0c0 0x5580a8f00a50 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f2d98005ab0 tx=0x7f2d9802c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18127d1ceb5e5bba server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:08.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.254+0000 7f2d877fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:08.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.254+0000 7f2d877fe640 1 -- 172.21.15.52:0/1250080996 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98042040 con 0x5580a8efb0c0 2024-04-02T14:00:08.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.254+0000 7f2d877fe640 1 -- 172.21.15.52:0/1250080996 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f2d98004240 con 0x5580a8efb0c0 2024-04-02T14:00:08.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.254+0000 7f2db13d0740 1 -- 172.21.15.52:0/1250080996 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d9802ca80 con 0x5580a8efb0c0 2024-04-02T14:00:08.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.254+0000 7f2d877fe640 1 -- 172.21.15.52:0/1250080996 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2d98004440 con 0x5580a8efb0c0 2024-04-02T14:00:08.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.254+0000 7f2db13d0740 10 client.17514 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:08.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.254+0000 7f2db13d0740 1 -- 172.21.15.52:0/1250080996 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5580a8f12ac0 con 0x5580a8efb0c0 2024-04-02T14:00:08.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.254+0000 7f2d877fe640 1 -- 172.21.15.52:0/1250080996 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d9802ca80 con 0x5580a8efb0c0 2024-04-02T14:00:08.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.254+0000 7f2d877fe640 1 -- 172.21.15.52:0/1250080996 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2d98003b20 con 0x5580a8efb0c0 2024-04-02T14:00:08.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.254+0000 7f2db13d0740 10 client.17514 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:08.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.254+0000 7f2db13d0740 10 client.17514 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T14:00:08.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.254+0000 7f2db13d0740 20 client.17514 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:08.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.254+0000 7f2db13d0740 1 --2- 172.21.15.52:0/1250080996 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5580a8f13e10 0x5580a8f16200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.254+0000 7f2db13d0740 4 client.17514 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:00:08.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.254+0000 7f2db13d0740 1 -- 172.21.15.52:0/1250080996 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580a8e6bcf0 con 0x5580a8f13e10 2024-04-02T14:00:08.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.255+0000 7f2da77fe640 1 --2- 172.21.15.52:0/1250080996 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5580a8f13e10 0x5580a8f16200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.255+0000 7f2da77fe640 1 --2- 172.21.15.52:0/1250080996 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5580a8f13e10 0x5580a8f16200 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5216bbf95cc45d6c server_cookie=b550523e03504718 in_seq=0 out_seq=0 2024-04-02T14:00:08.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.255+0000 7f2d877fe640 10 client.17514 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T14:00:08.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.256+0000 7f2d877fe640 1 -- 172.21.15.52:0/1250080996 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5580a8e6bcf0 con 0x5580a8f13e10 2024-04-02T14:00:08.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.256+0000 7f2d877fe640 10 client.17514 handle_command_reply: tid=0 2024-04-02T14:00:08.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.256+0000 7f2db13d0740 1 client.17514 shutdown 2024-04-02T14:00:08.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.256+0000 7f2db13d0740 1 -- 172.21.15.52:0/1250080996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8efb0c0 msgr2=0x5580a8f00a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.256+0000 7f2db13d0740 1 --2- 172.21.15.52:0/1250080996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8efb0c0 0x5580a8f00a50 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f2d98005ab0 tx=0x7f2d9802c900 comp rx=0 tx=0).stop 2024-04-02T14:00:08.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.256+0000 7f2db13d0740 1 -- 172.21.15.52:0/1250080996 shutdown_connections 2024-04-02T14:00:08.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.256+0000 7f2db13d0740 1 --2- 172.21.15.52:0/1250080996 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5580a8f13e10 0x5580a8f16200 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.256+0000 7f2db13d0740 1 --2- 172.21.15.52:0/1250080996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580a8f03ca0 0x5580a8f01060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.256+0000 7f2db13d0740 1 --2- 172.21.15.52:0/1250080996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8efb0c0 0x5580a8f00a50 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.256+0000 7f2d877fe640 0 client.17514 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T14:00:08.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.256+0000 7f2db13d0740 1 --2- 172.21.15.52:0/1250080996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5580a8efa7a0 0x5580a8f036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.256+0000 7f2db13d0740 1 -- 172.21.15.52:0/1250080996 >> 172.21.15.52:0/1250080996 conn(0x5580a8ef4ed0 msgr2=0x5580a8ef6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:08.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.256+0000 7f2db13d0740 1 -- 172.21.15.52:0/1250080996 shutdown_connections 2024-04-02T14:00:08.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.256+0000 7f2db13d0740 1 -- 172.21.15.52:0/1250080996 wait complete. 2024-04-02T14:00:08.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.256+0000 7f2db13d0740 20 client.17514 trim_cache size 0 max 16384 2024-04-02T14:00:08.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.269+0000 7f2db13d0740 1 Processor -- start 2024-04-02T14:00:08.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.269+0000 7f2db13d0740 1 -- start start 2024-04-02T14:00:08.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.270+0000 7f2db13d0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580a8efa7a0 0x5580a8effa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.270+0000 7f2db13d0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5580a8efb0c0 0x5580a8f03180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.270+0000 7f2db13d0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8f03ca0 0x5580a8f036c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.270+0000 7f2db13d0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2da81b0f30 con 0x5580a8f03ca0 2024-04-02T14:00:08.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.270+0000 7f2db13d0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2da81b2a70 con 0x5580a8efb0c0 2024-04-02T14:00:08.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.270+0000 7f2db13d0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2da81b1600 con 0x5580a8efa7a0 2024-04-02T14:00:08.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.270+0000 7f2da77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8f03ca0 0x5580a8f036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.270+0000 7f2da77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8f03ca0 0x5580a8f036c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34534/0 (socket says 172.21.15.52:34534) 2024-04-02T14:00:08.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.270+0000 7f2da77fe640 1 -- 172.21.15.52:0/176657493 learned_addr learned my addr 172.21.15.52:0/176657493 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:08.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.270+0000 7f2da77fe640 1 --2- 172.21.15.52:0/176657493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580a8efa7a0 0x5580a8effa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.270+0000 7f2da77fe640 1 -- 172.21.15.52:0/176657493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580a8efa7a0 msgr2=0x5580a8effa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.270+0000 7f2da77fe640 1 --2- 172.21.15.52:0/176657493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580a8efa7a0 0x5580a8effa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.270+0000 7f2da77fe640 1 -- 172.21.15.52:0/176657493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5580a8efb0c0 msgr2=0x5580a8f03180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.270+0000 7f2da7fff640 1 --2- 172.21.15.52:0/176657493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5580a8efb0c0 0x5580a8f03180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.270+0000 7f2da77fe640 1 --2- 172.21.15.52:0/176657493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5580a8efb0c0 0x5580a8f03180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.270+0000 7f2da77fe640 1 -- 172.21.15.52:0/176657493 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580a8efaef0 con 0x5580a8f03ca0 2024-04-02T14:00:08.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.270+0000 7f2da77fe640 1 --2- 172.21.15.52:0/176657493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8f03ca0 0x5580a8f036c0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c000c00 tx=0x7f2d9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b565f5819ee525 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:08.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.270+0000 7f2d8dffb640 1 -- 172.21.15.52:0/176657493 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c026030 con 0x5580a8f03ca0 2024-04-02T14:00:08.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.270+0000 7f2d8dffb640 1 -- 172.21.15.52:0/176657493 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d9c005500 con 0x5580a8f03ca0 2024-04-02T14:00:08.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.270+0000 7f2d8dffb640 1 -- 172.21.15.52:0/176657493 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c020940 con 0x5580a8f03ca0 2024-04-02T14:00:08.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.271+0000 7f2db13d0740 1 -- 172.21.15.52:0/176657493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8f03ca0 msgr2=0x5580a8f036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.271+0000 7f2db13d0740 1 --2- 172.21.15.52:0/176657493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8f03ca0 0x5580a8f036c0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c000c00 tx=0x7f2d9c0046c0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.271+0000 7f2db13d0740 1 -- 172.21.15.52:0/176657493 shutdown_connections 2024-04-02T14:00:08.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.271+0000 7f2db13d0740 1 --2- 172.21.15.52:0/176657493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8f03ca0 0x5580a8f036c0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.271+0000 7f2db13d0740 1 --2- 172.21.15.52:0/176657493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5580a8efb0c0 0x5580a8f03180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.271+0000 7f2db13d0740 1 --2- 172.21.15.52:0/176657493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580a8efa7a0 0x5580a8effa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.271+0000 7f2db13d0740 1 -- 172.21.15.52:0/176657493 >> 172.21.15.52:0/176657493 conn(0x5580a8ef4ed0 msgr2=0x5580a8eff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:08.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.271+0000 7f2db13d0740 1 -- 172.21.15.52:0/176657493 shutdown_connections 2024-04-02T14:00:08.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.271+0000 7f2db13d0740 1 -- 172.21.15.52:0/176657493 wait complete. 2024-04-02T14:00:08.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.271+0000 7f2db13d0740 1 -- start start 2024-04-02T14:00:08.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.271+0000 7f2db13d0740 1 Processor -- start 2024-04-02T14:00:08.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.272+0000 7f2db13d0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580a8efa7a0 0x5580a8f19190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.272+0000 7f2db13d0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5580a8efb0c0 0x5580a8f196d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.272+0000 7f2db13d0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8f03ca0 0x5580a8f14660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.272+0000 7f2db13d0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2da81b23a0 con 0x5580a8f03ca0 2024-04-02T14:00:08.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.272+0000 7f2db13d0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2da81b1cd0 con 0x5580a8efb0c0 2024-04-02T14:00:08.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.272+0000 7f2db13d0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2da81aef10 con 0x5580a8efa7a0 2024-04-02T14:00:08.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.272+0000 7f2db13d0740 10 client.0 _ll_register_callbacks cb 0x7f2d9c00c5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:08.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.272+0000 7f2da77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8f03ca0 0x5580a8f14660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.272+0000 7f2da77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8f03ca0 0x5580a8f14660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34536/0 (socket says 172.21.15.52:34536) 2024-04-02T14:00:08.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.272+0000 7f2da77fe640 1 -- 172.21.15.52:0/2492855942 learned_addr learned my addr 172.21.15.52:0/2492855942 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:08.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.272+0000 7f2da77fe640 1 --2- 172.21.15.52:0/2492855942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580a8efa7a0 0x5580a8f19190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.272+0000 7f2da7fff640 1 --2- 172.21.15.52:0/2492855942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5580a8efb0c0 0x5580a8f196d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.272+0000 7f2da77fe640 1 -- 172.21.15.52:0/2492855942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580a8efa7a0 msgr2=0x5580a8f19190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.272+0000 7f2da77fe640 1 --2- 172.21.15.52:0/2492855942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580a8efa7a0 0x5580a8f19190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.272+0000 7f2da77fe640 1 -- 172.21.15.52:0/2492855942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5580a8efb0c0 msgr2=0x5580a8f196d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.272+0000 7f2da77fe640 1 --2- 172.21.15.52:0/2492855942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5580a8efb0c0 0x5580a8f196d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.272+0000 7f2da77fe640 1 -- 172.21.15.52:0/2492855942 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580a8f14e90 con 0x5580a8f03ca0 2024-04-02T14:00:08.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.272+0000 7f2da7fff640 1 --2- 172.21.15.52:0/2492855942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5580a8efb0c0 0x5580a8f196d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:00:08.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.272+0000 7f2da77fe640 1 --2- 172.21.15.52:0/2492855942 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8f03ca0 0x5580a8f14660 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0125d0 tx=0x7f2d9c020720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c32db23c4d6d9209 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:08.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.272+0000 7f2d877fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:08.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.273+0000 7f2d877fe640 1 -- 172.21.15.52:0/2492855942 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c01c040 con 0x5580a8f03ca0 2024-04-02T14:00:08.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.273+0000 7f2d877fe640 1 -- 172.21.15.52:0/2492855942 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f2d9c0204e0 con 0x5580a8f03ca0 2024-04-02T14:00:08.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.273+0000 7f2db13d0740 1 -- 172.21.15.52:0/2492855942 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d9c005500 con 0x5580a8f03ca0 2024-04-02T14:00:08.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.273+0000 7f2d877fe640 1 -- 172.21.15.52:0/2492855942 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2d9c026040 con 0x5580a8f03ca0 2024-04-02T14:00:08.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.273+0000 7f2db13d0740 10 client.17520 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:08.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.273+0000 7f2db13d0740 1 -- 172.21.15.52:0/2492855942 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5580a8f79b60 con 0x5580a8f03ca0 2024-04-02T14:00:08.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.273+0000 7f2d877fe640 1 -- 172.21.15.52:0/2492855942 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d9c005500 con 0x5580a8f03ca0 2024-04-02T14:00:08.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.273+0000 7f2d877fe640 1 -- 172.21.15.52:0/2492855942 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2d9c02c7e0 con 0x5580a8f03ca0 2024-04-02T14:00:08.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.273+0000 7f2db13d0740 10 client.17520 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:08.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.273+0000 7f2db13d0740 10 client.17520 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T14:00:08.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.273+0000 7f2db13d0740 20 client.17520 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:08.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.273+0000 7f2db13d0740 1 --2- 172.21.15.52:0/2492855942 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5580a8f7a140 0x5580a8f7c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.273+0000 7f2db13d0740 4 client.17520 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:00:08.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.273+0000 7f2db13d0740 1 -- 172.21.15.52:0/2492855942 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5580a8e66040 con 0x5580a8f7a140 2024-04-02T14:00:08.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.273+0000 7f2da7fff640 1 --2- 172.21.15.52:0/2492855942 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5580a8f7a140 0x5580a8f7c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.274+0000 7f2d877fe640 10 client.17520 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T14:00:08.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.274+0000 7f2da7fff640 1 --2- 172.21.15.52:0/2492855942 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5580a8f7a140 0x5580a8f7c560 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3af3336bfd9de0fc server_cookie=be887d4beb94ec99 in_seq=0 out_seq=0 2024-04-02T14:00:08.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.274+0000 7f2d877fe640 1 -- 172.21.15.52:0/2492855942 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5580a8e66040 con 0x5580a8f7a140 2024-04-02T14:00:08.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.274+0000 7f2d877fe640 10 client.17520 handle_command_reply: tid=0 2024-04-02T14:00:08.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.274+0000 7f2db13d0740 1 client.17520 shutdown 2024-04-02T14:00:08.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.274+0000 7f2db13d0740 1 -- 172.21.15.52:0/2492855942 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8f03ca0 msgr2=0x5580a8f14660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.274+0000 7f2db13d0740 1 --2- 172.21.15.52:0/2492855942 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8f03ca0 0x5580a8f14660 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0125d0 tx=0x7f2d9c020720 comp rx=0 tx=0).stop 2024-04-02T14:00:08.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.274+0000 7f2db13d0740 1 -- 172.21.15.52:0/2492855942 shutdown_connections 2024-04-02T14:00:08.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.274+0000 7f2db13d0740 1 --2- 172.21.15.52:0/2492855942 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5580a8f7a140 0x5580a8f7c560 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.274+0000 7f2db13d0740 1 --2- 172.21.15.52:0/2492855942 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5580a8f03ca0 0x5580a8f14660 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.274+0000 7f2db13d0740 1 --2- 172.21.15.52:0/2492855942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5580a8efb0c0 0x5580a8f196d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.274+0000 7f2db13d0740 1 --2- 172.21.15.52:0/2492855942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580a8efa7a0 0x5580a8f19190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.274+0000 7f2d877fe640 0 client.17520 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T14:00:08.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.274+0000 7f2db13d0740 1 -- 172.21.15.52:0/2492855942 >> 172.21.15.52:0/2492855942 conn(0x5580a8ef4ed0 msgr2=0x5580a8f0ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:08.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.274+0000 7f2db13d0740 1 -- 172.21.15.52:0/2492855942 shutdown_connections 2024-04-02T14:00:08.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.274+0000 7f2db13d0740 1 -- 172.21.15.52:0/2492855942 wait complete. 2024-04-02T14:00:08.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.274+0000 7f2db13d0740 20 client.17520 trim_cache size 0 max 16384 2024-04-02T14:00:08.287 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:00:08.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.275+0000 7f2dae711640 1 -- 172.21.15.52:0/918272579 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2d900a3d20 msgr2=0x7f2d900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.275+0000 7f2dae711640 1 --2- 172.21.15.52:0/918272579 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2d900a3d20 0x7f2d900a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2da00a50b0 tx=0x7f2da0087040 comp rx=0 tx=0).stop 2024-04-02T14:00:08.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.275+0000 7f2dae711640 1 -- 172.21.15.52:0/918272579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2da8107aa0 msgr2=0x7f2da8074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.275+0000 7f2dae711640 1 --2- 172.21.15.52:0/918272579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2da8107aa0 0x7f2da8074880 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f2da000eaa0 tx=0x7f2da000ef60 comp rx=0 tx=0).stop 2024-04-02T14:00:08.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.275+0000 7f2dae711640 1 -- 172.21.15.52:0/918272579 shutdown_connections 2024-04-02T14:00:08.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.275+0000 7f2dae711640 1 --2- 172.21.15.52:0/918272579 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2d900a3d20 0x7f2d900a61e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.275+0000 7f2dae711640 1 --2- 172.21.15.52:0/918272579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2da8107aa0 0x7f2da8074880 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.275+0000 7f2dae711640 1 --2- 172.21.15.52:0/918272579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2da8102450 0x7f2da8074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.275+0000 7f2dae711640 1 --2- 172.21.15.52:0/918272579 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2da8101a90 0x7f2da8073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.275+0000 7f2dae711640 1 -- 172.21.15.52:0/918272579 >> 172.21.15.52:0/918272579 conn(0x7f2da8078f00 msgr2=0x7f2da81043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:08.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.275+0000 7f2dae711640 1 -- 172.21.15.52:0/918272579 shutdown_connections 2024-04-02T14:00:08.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.275+0000 7f2dae711640 1 -- 172.21.15.52:0/918272579 wait complete. 2024-04-02T14:00:08.290 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T14:00:08.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.392+0000 7fa76ab40640 1 -- 172.21.15.52:0/3814466126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa76406a190 msgr2=0x7fa76406a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.392+0000 7fa76ab40640 1 --2- 172.21.15.52:0/3814466126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa76406a190 0x7fa76406a610 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fa754009930 tx=0x7fa75402bc30 comp rx=0 tx=0).stop 2024-04-02T14:00:08.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.392+0000 7fa76ab40640 1 -- 172.21.15.52:0/3814466126 shutdown_connections 2024-04-02T14:00:08.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.392+0000 7fa76ab40640 1 --2- 172.21.15.52:0/3814466126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa76406ab50 0x7fa76410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.392+0000 7fa76ab40640 1 --2- 172.21.15.52:0/3814466126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa76406a190 0x7fa76406a610 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.392+0000 7fa76ab40640 1 --2- 172.21.15.52:0/3814466126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa764107aa0 0x7fa764107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.392+0000 7fa76ab40640 1 -- 172.21.15.52:0/3814466126 >> 172.21.15.52:0/3814466126 conn(0x7fa764077ad0 msgr2=0x7fa764077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:08.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.392+0000 7fa76ab40640 1 -- 172.21.15.52:0/3814466126 shutdown_connections 2024-04-02T14:00:08.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.392+0000 7fa76ab40640 1 -- 172.21.15.52:0/3814466126 wait complete. 2024-04-02T14:00:08.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.393+0000 7fa76ab40640 1 Processor -- start 2024-04-02T14:00:08.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.393+0000 7fa76ab40640 1 -- start start 2024-04-02T14:00:08.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.393+0000 7fa76ab40640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa76406a190 0x7fa764198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.393+0000 7fa76ab40640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa76406ab50 0x7fa7641989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.393+0000 7fa76ab40640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa764107aa0 0x7fa76419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.393+0000 7fa76ab40640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa764111eb0 con 0x7fa764107aa0 2024-04-02T14:00:08.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.393+0000 7fa76ab40640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa764111d30 con 0x7fa76406ab50 2024-04-02T14:00:08.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.393+0000 7fa76ab40640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa764112030 con 0x7fa76406a190 2024-04-02T14:00:08.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.393+0000 7fa7690dd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa764107aa0 0x7fa76419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.393+0000 7fa7690dd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa764107aa0 0x7fa76419cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34556/0 (socket says 172.21.15.52:34556) 2024-04-02T14:00:08.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.393+0000 7fa7690dd640 1 -- 172.21.15.52:0/200634323 learned_addr learned my addr 172.21.15.52:0/200634323 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:08.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.393+0000 7fa75bfff640 1 --2- 172.21.15.52:0/200634323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa76406ab50 0x7fa7641989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.393+0000 7fa7688dc640 1 --2- 172.21.15.52:0/200634323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa76406a190 0x7fa764198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.393+0000 7fa7690dd640 1 -- 172.21.15.52:0/200634323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa76406a190 msgr2=0x7fa764198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.393+0000 7fa7690dd640 1 --2- 172.21.15.52:0/200634323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa76406a190 0x7fa764198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.393+0000 7fa7690dd640 1 -- 172.21.15.52:0/200634323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa76406ab50 msgr2=0x7fa7641989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.393+0000 7fa7690dd640 1 --2- 172.21.15.52:0/200634323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa76406ab50 0x7fa7641989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.393+0000 7fa7690dd640 1 -- 172.21.15.52:0/200634323 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76419d460 con 0x7fa764107aa0 2024-04-02T14:00:08.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.394+0000 7fa7690dd640 1 --2- 172.21.15.52:0/200634323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa764107aa0 0x7fa76419cd30 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00b970 tx=0x7fa75c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:08.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.394+0000 7fa759ffb640 1 -- 172.21.15.52:0/200634323 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c019070 con 0x7fa764107aa0 2024-04-02T14:00:08.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.394+0000 7fa759ffb640 1 -- 172.21.15.52:0/200634323 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa75c00cb40 con 0x7fa764107aa0 2024-04-02T14:00:08.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.394+0000 7fa76ab40640 1 -- 172.21.15.52:0/200634323 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa76419d750 con 0x7fa764107aa0 2024-04-02T14:00:08.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.394+0000 7fa759ffb640 1 -- 172.21.15.52:0/200634323 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c007650 con 0x7fa764107aa0 2024-04-02T14:00:08.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.394+0000 7fa76ab40640 1 -- 172.21.15.52:0/200634323 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7641a4fe0 con 0x7fa764107aa0 2024-04-02T14:00:08.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.397+0000 7fa759ffb640 1 -- 172.21.15.52:0/200634323 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa75c01c070 con 0x7fa764107aa0 2024-04-02T14:00:08.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.397+0000 7fa76d806740 1 Processor -- start 2024-04-02T14:00:08.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.397+0000 7fa76d806740 1 -- start start 2024-04-02T14:00:08.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.397+0000 7fa76d806740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559873ec87a0 0x559873ec8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.397+0000 7fa76d806740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559873ec90c0 0x559873ed1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.397+0000 7fa76d806740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ed1ca0 0x559873ed4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.397+0000 7fa76d806740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559873e2c7b0 con 0x559873ed1ca0 2024-04-02T14:00:08.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.397+0000 7fa76d806740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559873dcb570 con 0x559873ec90c0 2024-04-02T14:00:08.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.397+0000 7fa76d806740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559873e28760 con 0x559873ec87a0 2024-04-02T14:00:08.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.397+0000 7fa759ffb640 1 --2- 172.21.15.52:0/200634323 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa74c0a3d20 0x7fa74c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.397+0000 7fa75bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ed1ca0 0x559873ed4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.397+0000 7fa75bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ed1ca0 0x559873ed4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34560/0 (socket says 172.21.15.52:34560) 2024-04-02T14:00:08.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.397+0000 7fa75bfff640 1 -- 172.21.15.52:0/904093405 learned_addr learned my addr 172.21.15.52:0/904093405 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:08.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.397+0000 7fa7690dd640 1 --2- 172.21.15.52:0/200634323 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa74c0a3d20 0x7fa74c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.397+0000 7fa7688dc640 1 --2- 172.21.15.52:0/904093405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559873ec90c0 0x559873ed1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.397+0000 7fa75bfff640 1 --2- 172.21.15.52:0/904093405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559873ec87a0 0x559873ec8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.397+0000 7fa759ffb640 1 -- 172.21.15.52:0/200634323 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fa75c0cffe0 con 0x7fa764107aa0 2024-04-02T14:00:08.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.398+0000 7fa75bfff640 1 -- 172.21.15.52:0/904093405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559873ec87a0 msgr2=0x559873ec8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.398+0000 7fa75bfff640 1 --2- 172.21.15.52:0/904093405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559873ec87a0 0x559873ec8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.398+0000 7fa75bfff640 1 -- 172.21.15.52:0/904093405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559873ec90c0 msgr2=0x559873ed1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.398+0000 7fa75bfff640 1 --2- 172.21.15.52:0/904093405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559873ec90c0 0x559873ed1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.398+0000 7fa75bfff640 1 -- 172.21.15.52:0/904093405 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559873b403f0 con 0x559873ed1ca0 2024-04-02T14:00:08.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.398+0000 7fa75bfff640 1 --2- 172.21.15.52:0/904093405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ed1ca0 0x559873ed4090 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fa754004ee0 tx=0x7fa754004e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d4c793b1a3cb6c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:08.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.398+0000 7fa7690dd640 1 --2- 172.21.15.52:0/200634323 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa74c0a3d20 0x7fa74c0a61e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa75c0cd5b0 tx=0x7fa75c086930 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:08.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.398+0000 7fa74affd640 1 -- 172.21.15.52:0/904093405 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa754049070 con 0x559873ed1ca0 2024-04-02T14:00:08.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.398+0000 7fa74affd640 1 -- 172.21.15.52:0/904093405 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7540046c0 con 0x559873ed1ca0 2024-04-02T14:00:08.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.398+0000 7fa74affd640 1 -- 172.21.15.52:0/904093405 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7540052d0 con 0x559873ed1ca0 2024-04-02T14:00:08.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.399+0000 7fa76d806740 1 -- 172.21.15.52:0/904093405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ed1ca0 msgr2=0x559873ed4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.399+0000 7fa76d806740 1 --2- 172.21.15.52:0/904093405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ed1ca0 0x559873ed4090 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fa754004ee0 tx=0x7fa754004e70 comp rx=0 tx=0).stop 2024-04-02T14:00:08.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.399+0000 7fa76d806740 1 -- 172.21.15.52:0/904093405 shutdown_connections 2024-04-02T14:00:08.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.399+0000 7fa76d806740 1 --2- 172.21.15.52:0/904093405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ed1ca0 0x559873ed4090 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.399+0000 7fa76d806740 1 --2- 172.21.15.52:0/904093405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559873ec90c0 0x559873ed1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.399+0000 7fa76d806740 1 --2- 172.21.15.52:0/904093405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559873ec87a0 0x559873ec8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.399+0000 7fa76d806740 1 -- 172.21.15.52:0/904093405 >> 172.21.15.52:0/904093405 conn(0x559873ec2ed0 msgr2=0x559873ec32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:08.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.399+0000 7fa76d806740 1 -- 172.21.15.52:0/904093405 shutdown_connections 2024-04-02T14:00:08.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.399+0000 7fa76d806740 1 -- 172.21.15.52:0/904093405 wait complete. 2024-04-02T14:00:08.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.399+0000 7fa76d806740 1 -- start start 2024-04-02T14:00:08.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.399+0000 7fa76d806740 1 Processor -- start 2024-04-02T14:00:08.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.399+0000 7fa76d806740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559873ec87a0 0x559873ed1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.400+0000 7fa76d806740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559873ec90c0 0x559873eceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.400+0000 7fa76d806740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ed1ca0 0x559873ecf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.400+0000 7fa76d806740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559873ed4a30 con 0x559873ed1ca0 2024-04-02T14:00:08.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.400+0000 7fa76d806740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559873de7e80 con 0x559873ec90c0 2024-04-02T14:00:08.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.400+0000 7fa76d806740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559873ed4bb0 con 0x559873ec87a0 2024-04-02T14:00:08.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.400+0000 7fa76d806740 10 client.0 _ll_register_callbacks cb 0x559873e23aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:08.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.400+0000 7fa75bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ed1ca0 0x559873ecf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.400+0000 7fa75bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ed1ca0 0x559873ecf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34566/0 (socket says 172.21.15.52:34566) 2024-04-02T14:00:08.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.400+0000 7fa75bfff640 1 -- 172.21.15.52:0/98349884 learned_addr learned my addr 172.21.15.52:0/98349884 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:08.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.400+0000 7fa75bfff640 1 --2- 172.21.15.52:0/98349884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559873ec87a0 0x559873ed1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.400+0000 7fa7688dc640 1 --2- 172.21.15.52:0/98349884 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559873ec90c0 0x559873eceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.400+0000 7fa75bfff640 1 -- 172.21.15.52:0/98349884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559873ec87a0 msgr2=0x559873ed1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.400+0000 7fa75bfff640 1 --2- 172.21.15.52:0/98349884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559873ec87a0 0x559873ed1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.400+0000 7fa75bfff640 1 -- 172.21.15.52:0/98349884 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559873ec90c0 msgr2=0x559873eceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.400+0000 7fa75bfff640 1 --2- 172.21.15.52:0/98349884 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559873ec90c0 0x559873eceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.400+0000 7fa75bfff640 1 -- 172.21.15.52:0/98349884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559873c0a7d0 con 0x559873ed1ca0 2024-04-02T14:00:08.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.400+0000 7fa75bfff640 1 --2- 172.21.15.52:0/98349884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ed1ca0 0x559873ecf1e0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fa754004fb0 tx=0x7fa75402c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1904cdfe30cd2262 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:08.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.400+0000 7fa743fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:08.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.400+0000 7fa743fff640 1 -- 172.21.15.52:0/98349884 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa754057060 con 0x559873ed1ca0 2024-04-02T14:00:08.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.400+0000 7fa743fff640 1 -- 172.21.15.52:0/98349884 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fa754005460 con 0x559873ed1ca0 2024-04-02T14:00:08.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.400+0000 7fa76d806740 1 -- 172.21.15.52:0/98349884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7540046c0 con 0x559873ed1ca0 2024-04-02T14:00:08.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.400+0000 7fa743fff640 1 -- 172.21.15.52:0/98349884 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa75405e050 con 0x559873ed1ca0 2024-04-02T14:00:08.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.401+0000 7fa76d806740 10 client.17532 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:08.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.401+0000 7fa76d806740 1 -- 172.21.15.52:0/98349884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559873ee0ac0 con 0x559873ed1ca0 2024-04-02T14:00:08.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.401+0000 7fa743fff640 1 -- 172.21.15.52:0/98349884 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7540046c0 con 0x559873ed1ca0 2024-04-02T14:00:08.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.401+0000 7fa743fff640 1 -- 172.21.15.52:0/98349884 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa75404c070 con 0x559873ed1ca0 2024-04-02T14:00:08.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.401+0000 7fa76d806740 10 client.17532 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:08.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.401+0000 7fa76d806740 10 client.17532 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T14:00:08.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.401+0000 7fa76d806740 20 client.17532 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:08.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.401+0000 7fa76d806740 1 --2- 172.21.15.52:0/98349884 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x559873ee1e10 0x559873ee4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.401+0000 7fa76d806740 4 client.17532 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:00:08.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.401+0000 7fa76d806740 1 -- 172.21.15.52:0/98349884 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559873e39cf0 con 0x559873ee1e10 2024-04-02T14:00:08.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.401+0000 7fa7688dc640 1 --2- 172.21.15.52:0/98349884 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x559873ee1e10 0x559873ee4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.402+0000 7fa743fff640 10 client.17532 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T14:00:08.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.402+0000 7fa7688dc640 1 --2- 172.21.15.52:0/98349884 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x559873ee1e10 0x559873ee4200 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=64a1b621d6f379a6 server_cookie=3f051e02d5d6f9cf in_seq=0 out_seq=0 2024-04-02T14:00:08.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.402+0000 7fa743fff640 1 -- 172.21.15.52:0/98349884 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559873e39cf0 con 0x559873ee1e10 2024-04-02T14:00:08.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.402+0000 7fa743fff640 10 client.17532 handle_command_reply: tid=0 2024-04-02T14:00:08.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.402+0000 7fa76d806740 1 client.17532 shutdown 2024-04-02T14:00:08.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.402+0000 7fa76d806740 1 -- 172.21.15.52:0/98349884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ed1ca0 msgr2=0x559873ecf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.402+0000 7fa76d806740 1 --2- 172.21.15.52:0/98349884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ed1ca0 0x559873ecf1e0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fa754004fb0 tx=0x7fa75402c6b0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.402+0000 7fa76d806740 1 -- 172.21.15.52:0/98349884 shutdown_connections 2024-04-02T14:00:08.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.402+0000 7fa76d806740 1 --2- 172.21.15.52:0/98349884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ed1ca0 0x559873ecf1e0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.402+0000 7fa76d806740 1 --2- 172.21.15.52:0/98349884 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559873ec90c0 0x559873eceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.402+0000 7fa76d806740 1 --2- 172.21.15.52:0/98349884 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x559873ee1e10 0x559873ee4200 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.402+0000 7fa76d806740 1 --2- 172.21.15.52:0/98349884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559873ec87a0 0x559873ed1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.402+0000 7fa76d806740 1 -- 172.21.15.52:0/98349884 >> 172.21.15.52:0/98349884 conn(0x559873ec2ed0 msgr2=0x559873ec4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:08.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.402+0000 7fa743fff640 0 client.17532 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T14:00:08.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.403+0000 7fa76d806740 1 -- 172.21.15.52:0/98349884 shutdown_connections 2024-04-02T14:00:08.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.403+0000 7fa76d806740 1 -- 172.21.15.52:0/98349884 wait complete. 2024-04-02T14:00:08.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.403+0000 7fa76d806740 20 client.17532 trim_cache size 0 max 16384 2024-04-02T14:00:08.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.416+0000 7fa76d806740 1 Processor -- start 2024-04-02T14:00:08.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.416+0000 7fa76d806740 1 -- start start 2024-04-02T14:00:08.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.416+0000 7fa76d806740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ec87a0 0x559873ec94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.416+0000 7fa76d806740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559873ec90c0 0x559873ecda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.416+0000 7fa76d806740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559873ed1ca0 0x559873ed11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.416+0000 7fa76d806740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa7641aee20 con 0x559873ec87a0 2024-04-02T14:00:08.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.416+0000 7fa76d806740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa7641b0960 con 0x559873ec90c0 2024-04-02T14:00:08.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.416+0000 7fa76d806740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa7641af4f0 con 0x559873ed1ca0 2024-04-02T14:00:08.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.416+0000 7fa75bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ec87a0 0x559873ec94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.416+0000 7fa75bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ec87a0 0x559873ec94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34576/0 (socket says 172.21.15.52:34576) 2024-04-02T14:00:08.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.416+0000 7fa75bfff640 1 -- 172.21.15.52:0/20696452 learned_addr learned my addr 172.21.15.52:0/20696452 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:08.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.416+0000 7fa75bfff640 1 -- 172.21.15.52:0/20696452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559873ed1ca0 msgr2=0x559873ed11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.416+0000 7fa75bfff640 1 --2- 172.21.15.52:0/20696452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559873ed1ca0 0x559873ed11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.416+0000 7fa75bfff640 1 -- 172.21.15.52:0/20696452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559873ec90c0 msgr2=0x559873ecda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.416+0000 7fa75bfff640 1 --2- 172.21.15.52:0/20696452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559873ec90c0 0x559873ecda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.416+0000 7fa75bfff640 1 -- 172.21.15.52:0/20696452 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559873eea040 con 0x559873ec87a0 2024-04-02T14:00:08.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.417+0000 7fa75bfff640 1 --2- 172.21.15.52:0/20696452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ec87a0 0x559873ec94a0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fa754005ec0 tx=0x7fa75402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a0793f8c713ccf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:08.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.417+0000 7fa74a7fc640 1 -- 172.21.15.52:0/20696452 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa754057070 con 0x559873ec87a0 2024-04-02T14:00:08.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.417+0000 7fa74a7fc640 1 -- 172.21.15.52:0/20696452 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7540046c0 con 0x559873ec87a0 2024-04-02T14:00:08.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.417+0000 7fa74a7fc640 1 -- 172.21.15.52:0/20696452 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa754049040 con 0x559873ec87a0 2024-04-02T14:00:08.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.417+0000 7fa76d806740 1 -- 172.21.15.52:0/20696452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ec87a0 msgr2=0x559873ec94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.417+0000 7fa76d806740 1 --2- 172.21.15.52:0/20696452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ec87a0 0x559873ec94a0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fa754005ec0 tx=0x7fa75402c7e0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.417+0000 7fa76d806740 1 -- 172.21.15.52:0/20696452 shutdown_connections 2024-04-02T14:00:08.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.417+0000 7fa76d806740 1 --2- 172.21.15.52:0/20696452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559873ed1ca0 0x559873ed11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.417+0000 7fa76d806740 1 --2- 172.21.15.52:0/20696452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559873ec90c0 0x559873ecda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.417+0000 7fa76d806740 1 --2- 172.21.15.52:0/20696452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ec87a0 0x559873ec94a0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.417+0000 7fa76d806740 1 -- 172.21.15.52:0/20696452 >> 172.21.15.52:0/20696452 conn(0x559873ec2ed0 msgr2=0x559873ecd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:08.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.417+0000 7fa76d806740 1 -- 172.21.15.52:0/20696452 shutdown_connections 2024-04-02T14:00:08.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.417+0000 7fa76d806740 1 -- 172.21.15.52:0/20696452 wait complete. 2024-04-02T14:00:08.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.418+0000 7fa76d806740 1 -- start start 2024-04-02T14:00:08.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.418+0000 7fa76d806740 1 Processor -- start 2024-04-02T14:00:08.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.418+0000 7fa76d806740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559873ec87a0 0x559873ee24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.418+0000 7fa76d806740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559873ec90c0 0x559873ee2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.418+0000 7fa76d806740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ed1ca0 0x559873f40010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.418+0000 7fa76d806740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa7641b0290 con 0x559873ed1ca0 2024-04-02T14:00:08.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.418+0000 7fa76d806740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa7641afbc0 con 0x559873ec87a0 2024-04-02T14:00:08.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.418+0000 7fa76d806740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa7641ace00 con 0x559873ec90c0 2024-04-02T14:00:08.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.418+0000 7fa76d806740 10 client.0 _ll_register_callbacks cb 0x7fa754047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:08.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.418+0000 7fa75bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ed1ca0 0x559873f40010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.418+0000 7fa75bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ed1ca0 0x559873f40010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34582/0 (socket says 172.21.15.52:34582) 2024-04-02T14:00:08.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.418+0000 7fa75bfff640 1 -- 172.21.15.52:0/1033597079 learned_addr learned my addr 172.21.15.52:0/1033597079 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:08.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.418+0000 7fa75bfff640 1 --2- 172.21.15.52:0/1033597079 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559873ec87a0 0x559873ee24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.419+0000 7fa7688dc640 1 --2- 172.21.15.52:0/1033597079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559873ec90c0 0x559873ee2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.419+0000 7fa75bfff640 1 -- 172.21.15.52:0/1033597079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559873ec90c0 msgr2=0x559873ee2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.419+0000 7fa75bfff640 1 --2- 172.21.15.52:0/1033597079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559873ec90c0 0x559873ee2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.419+0000 7fa75bfff640 1 -- 172.21.15.52:0/1033597079 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559873ec87a0 msgr2=0x559873ee24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.419+0000 7fa75bfff640 1 --2- 172.21.15.52:0/1033597079 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559873ec87a0 0x559873ee24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.419+0000 7fa75bfff640 1 -- 172.21.15.52:0/1033597079 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559873f40650 con 0x559873ed1ca0 2024-04-02T14:00:08.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.419+0000 7fa75bfff640 1 --2- 172.21.15.52:0/1033597079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ed1ca0 0x559873f40010 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fa754041730 tx=0x7fa754036040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e648402e09bdd5a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:08.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.419+0000 7fa743fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:08.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.419+0000 7fa743fff640 1 -- 172.21.15.52:0/1033597079 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa754049070 con 0x559873ed1ca0 2024-04-02T14:00:08.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.419+0000 7fa76d806740 1 -- 172.21.15.52:0/1033597079 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7540046c0 con 0x559873ed1ca0 2024-04-02T14:00:08.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.419+0000 7fa743fff640 1 -- 172.21.15.52:0/1033597079 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fa754005460 con 0x559873ed1ca0 2024-04-02T14:00:08.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.419+0000 7fa743fff640 1 -- 172.21.15.52:0/1033597079 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa754057040 con 0x559873ed1ca0 2024-04-02T14:00:08.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.419+0000 7fa76d806740 10 client.17538 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:08.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.419+0000 7fa76d806740 1 -- 172.21.15.52:0/1033597079 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559873f47a00 con 0x559873ed1ca0 2024-04-02T14:00:08.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.419+0000 7fa743fff640 1 -- 172.21.15.52:0/1033597079 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7540046c0 con 0x559873ed1ca0 2024-04-02T14:00:08.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.419+0000 7fa743fff640 1 -- 172.21.15.52:0/1033597079 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa75404c040 con 0x559873ed1ca0 2024-04-02T14:00:08.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.420+0000 7fa76d806740 10 client.17538 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:08.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.420+0000 7fa76d806740 10 client.17538 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T14:00:08.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.420+0000 7fa76d806740 20 client.17538 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:08.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.420+0000 7fa76d806740 1 --2- 172.21.15.52:0/1033597079 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x559873f47eb0 0x559873f4a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.420+0000 7fa76d806740 4 client.17538 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:00:08.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.420+0000 7fa76d806740 1 -- 172.21.15.52:0/1033597079 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559873e34040 con 0x559873f47eb0 2024-04-02T14:00:08.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.420+0000 7fa7688dc640 1 --2- 172.21.15.52:0/1033597079 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x559873f47eb0 0x559873f4a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.420+0000 7fa743fff640 10 client.17538 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T14:00:08.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.420+0000 7fa7688dc640 1 --2- 172.21.15.52:0/1033597079 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x559873f47eb0 0x559873f4a2d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f339442e9c754c90 server_cookie=bffb664c240789bb in_seq=0 out_seq=0 2024-04-02T14:00:08.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.420+0000 7fa743fff640 1 -- 172.21.15.52:0/1033597079 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559873e34040 con 0x559873f47eb0 2024-04-02T14:00:08.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.420+0000 7fa743fff640 10 client.17538 handle_command_reply: tid=0 2024-04-02T14:00:08.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.420+0000 7fa76d806740 1 client.17538 shutdown 2024-04-02T14:00:08.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.420+0000 7fa76d806740 1 -- 172.21.15.52:0/1033597079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ed1ca0 msgr2=0x559873f40010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.420+0000 7fa76d806740 1 --2- 172.21.15.52:0/1033597079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ed1ca0 0x559873f40010 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fa754041730 tx=0x7fa754036040 comp rx=0 tx=0).stop 2024-04-02T14:00:08.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.420+0000 7fa76d806740 1 -- 172.21.15.52:0/1033597079 shutdown_connections 2024-04-02T14:00:08.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.420+0000 7fa76d806740 1 --2- 172.21.15.52:0/1033597079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559873ed1ca0 0x559873f40010 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.420+0000 7fa76d806740 1 --2- 172.21.15.52:0/1033597079 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x559873f47eb0 0x559873f4a2d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.420+0000 7fa76d806740 1 --2- 172.21.15.52:0/1033597079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559873ec90c0 0x559873ee2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.420+0000 7fa743fff640 0 client.17538 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T14:00:08.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.420+0000 7fa76d806740 1 --2- 172.21.15.52:0/1033597079 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559873ec87a0 0x559873ee24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.421+0000 7fa76d806740 1 -- 172.21.15.52:0/1033597079 >> 172.21.15.52:0/1033597079 conn(0x559873ec2ed0 msgr2=0x559873edce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:08.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.421+0000 7fa76d806740 1 -- 172.21.15.52:0/1033597079 shutdown_connections 2024-04-02T14:00:08.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.421+0000 7fa76d806740 1 -- 172.21.15.52:0/1033597079 wait complete. 2024-04-02T14:00:08.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.421+0000 7fa76d806740 20 client.17538 trim_cache size 0 max 16384 2024-04-02T14:00:08.437 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:00:08.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.421+0000 7fa76ab40640 1 -- 172.21.15.52:0/200634323 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa74c0a3d20 msgr2=0x7fa74c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.421+0000 7fa76ab40640 1 --2- 172.21.15.52:0/200634323 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa74c0a3d20 0x7fa74c0a61e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa75c0cd5b0 tx=0x7fa75c086930 comp rx=0 tx=0).stop 2024-04-02T14:00:08.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.421+0000 7fa76ab40640 1 -- 172.21.15.52:0/200634323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa764107aa0 msgr2=0x7fa76419cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.421+0000 7fa76ab40640 1 --2- 172.21.15.52:0/200634323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa764107aa0 0x7fa76419cd30 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00b970 tx=0x7fa75c00be30 comp rx=0 tx=0).stop 2024-04-02T14:00:08.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.421+0000 7fa76ab40640 1 -- 172.21.15.52:0/200634323 shutdown_connections 2024-04-02T14:00:08.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.421+0000 7fa76ab40640 1 --2- 172.21.15.52:0/200634323 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa74c0a3d20 0x7fa74c0a61e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.421+0000 7fa76ab40640 1 --2- 172.21.15.52:0/200634323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa764107aa0 0x7fa76419cd30 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.421+0000 7fa76ab40640 1 --2- 172.21.15.52:0/200634323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa76406ab50 0x7fa7641989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.421+0000 7fa76ab40640 1 --2- 172.21.15.52:0/200634323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa76406a190 0x7fa764198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.421+0000 7fa76ab40640 1 -- 172.21.15.52:0/200634323 >> 172.21.15.52:0/200634323 conn(0x7fa764077ad0 msgr2=0x7fa7640fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:08.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.421+0000 7fa76ab40640 1 -- 172.21.15.52:0/200634323 shutdown_connections 2024-04-02T14:00:08.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.421+0000 7fa76ab40640 1 -- 172.21.15.52:0/200634323 wait complete. 2024-04-02T14:00:08.440 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T14:00:08.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.541+0000 7f7a8dcc5640 1 -- 172.21.15.52:0/1047012994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a88068ce0 msgr2=0x7f7a88069160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.541+0000 7f7a8dcc5640 1 --2- 172.21.15.52:0/1047012994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a88068ce0 0x7f7a88069160 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009930 tx=0x7f7a7c02bc30 comp rx=0 tx=0).stop 2024-04-02T14:00:08.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.541+0000 7f7a8dcc5640 1 -- 172.21.15.52:0/1047012994 shutdown_connections 2024-04-02T14:00:08.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.541+0000 7f7a8dcc5640 1 --2- 172.21.15.52:0/1047012994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a880696a0 0x7f7a8810e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.541+0000 7f7a8dcc5640 1 --2- 172.21.15.52:0/1047012994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a88068ce0 0x7f7a88069160 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.541+0000 7f7a8dcc5640 1 --2- 172.21.15.52:0/1047012994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a88107a80 0x7f7a88107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.541+0000 7f7a8dcc5640 1 -- 172.21.15.52:0/1047012994 >> 172.21.15.52:0/1047012994 conn(0x7f7a88077b00 msgr2=0x7f7a88077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:08.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.541+0000 7f7a8dcc5640 1 -- 172.21.15.52:0/1047012994 shutdown_connections 2024-04-02T14:00:08.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.541+0000 7f7a8dcc5640 1 -- 172.21.15.52:0/1047012994 wait complete. 2024-04-02T14:00:08.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.542+0000 7f7a8dcc5640 1 Processor -- start 2024-04-02T14:00:08.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.542+0000 7f7a8dcc5640 1 -- start start 2024-04-02T14:00:08.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.542+0000 7f7a8dcc5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a88068ce0 0x7f7a88198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.542+0000 7f7a8dcc5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a880696a0 0x7f7a88198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.542+0000 7f7a8dcc5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a88107a80 0x7f7a8819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.542+0000 7f7a8dcc5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7a88111ee0 con 0x7f7a88107a80 2024-04-02T14:00:08.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.542+0000 7f7a8dcc5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7a88111d60 con 0x7f7a88068ce0 2024-04-02T14:00:08.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.542+0000 7f7a8dcc5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7a88112060 con 0x7f7a880696a0 2024-04-02T14:00:08.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.542+0000 7f7a87fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a88107a80 0x7f7a8819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.542+0000 7f7a87fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a88107a80 0x7f7a8819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34606/0 (socket says 172.21.15.52:34606) 2024-04-02T14:00:08.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.542+0000 7f7a87fff640 1 -- 172.21.15.52:0/2448398640 learned_addr learned my addr 172.21.15.52:0/2448398640 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:08.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.543+0000 7f7a877fe640 1 --2- 172.21.15.52:0/2448398640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a88068ce0 0x7f7a88198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.543+0000 7f7a86ffd640 1 --2- 172.21.15.52:0/2448398640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a880696a0 0x7f7a88198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.543+0000 7f7a87fff640 1 -- 172.21.15.52:0/2448398640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a880696a0 msgr2=0x7f7a88198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.543+0000 7f7a87fff640 1 --2- 172.21.15.52:0/2448398640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a880696a0 0x7f7a88198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.543+0000 7f7a87fff640 1 -- 172.21.15.52:0/2448398640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a88068ce0 msgr2=0x7f7a88198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.543+0000 7f7a87fff640 1 --2- 172.21.15.52:0/2448398640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a88068ce0 0x7f7a88198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.543+0000 7f7a87fff640 1 -- 172.21.15.52:0/2448398640 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8819d410 con 0x7f7a88107a80 2024-04-02T14:00:08.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.543+0000 7f7a86ffd640 1 --2- 172.21.15.52:0/2448398640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a880696a0 0x7f7a88198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:00:08.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.543+0000 7f7a87fff640 1 --2- 172.21.15.52:0/2448398640 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a88107a80 0x7f7a8819cce0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f7a8000eb10 tx=0x7f7a8000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:08.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.543+0000 7f7a84ff9640 1 -- 172.21.15.52:0/2448398640 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80004b20 con 0x7f7a88107a80 2024-04-02T14:00:08.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.543+0000 7f7a84ff9640 1 -- 172.21.15.52:0/2448398640 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a80004cc0 con 0x7f7a88107a80 2024-04-02T14:00:08.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.543+0000 7f7a8dcc5640 1 -- 172.21.15.52:0/2448398640 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8819d700 con 0x7f7a88107a80 2024-04-02T14:00:08.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.543+0000 7f7a84ff9640 1 -- 172.21.15.52:0/2448398640 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80010760 con 0x7f7a88107a80 2024-04-02T14:00:08.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.543+0000 7f7a8dcc5640 1 -- 172.21.15.52:0/2448398640 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a881a4f90 con 0x7f7a88107a80 2024-04-02T14:00:08.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.545+0000 7f7a84ff9640 1 -- 172.21.15.52:0/2448398640 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7a80005020 con 0x7f7a88107a80 2024-04-02T14:00:08.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.545+0000 7f7a90986740 1 Processor -- start 2024-04-02T14:00:08.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.546+0000 7f7a90986740 1 -- start start 2024-04-02T14:00:08.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.546+0000 7f7a84ff9640 1 --2- 172.21.15.52:0/2448398640 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f7a700a3d20 0x7f7a700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.546+0000 7f7a90986740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fc87e57a0 0x564fc87e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.546+0000 7f7a90986740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564fc87e60c0 0x564fc87ee760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.546+0000 7f7a90986740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87eeca0 0x564fc87f1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.546+0000 7f7a90986740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564fc87497b0 con 0x564fc87eeca0 2024-04-02T14:00:08.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.546+0000 7f7a90986740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564fc86e8570 con 0x564fc87e60c0 2024-04-02T14:00:08.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.546+0000 7f7a90986740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564fc8745760 con 0x564fc87e57a0 2024-04-02T14:00:08.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.546+0000 7f7a86ffd640 1 --2- 172.21.15.52:0/2448398640 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f7a700a3d20 0x7f7a700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.546+0000 7f7a84ff9640 1 -- 172.21.15.52:0/2448398640 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f7a800d1170 con 0x7f7a88107a80 2024-04-02T14:00:08.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.546+0000 7f7a87fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87eeca0 0x564fc87f1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.546+0000 7f7a87fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87eeca0 0x564fc87f1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34610/0 (socket says 172.21.15.52:34610) 2024-04-02T14:00:08.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.546+0000 7f7a87fff640 1 -- 172.21.15.52:0/3066145241 learned_addr learned my addr 172.21.15.52:0/3066145241 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:08.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.546+0000 7f7a86ffd640 1 --2- 172.21.15.52:0/3066145241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564fc87e60c0 0x564fc87ee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.546+0000 7f7a877fe640 1 --2- 172.21.15.52:0/3066145241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fc87e57a0 0x564fc87e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.546+0000 7f7a87fff640 1 -- 172.21.15.52:0/3066145241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fc87e57a0 msgr2=0x564fc87e5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.546+0000 7f7a87fff640 1 --2- 172.21.15.52:0/3066145241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fc87e57a0 0x564fc87e5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.546+0000 7f7a87fff640 1 -- 172.21.15.52:0/3066145241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564fc87e60c0 msgr2=0x564fc87ee760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.546+0000 7f7a87fff640 1 --2- 172.21.15.52:0/3066145241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564fc87e60c0 0x564fc87ee760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.546+0000 7f7a87fff640 1 -- 172.21.15.52:0/3066145241 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fc845d3f0 con 0x564fc87eeca0 2024-04-02T14:00:08.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.546+0000 7f7a87fff640 1 --2- 172.21.15.52:0/3066145241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87eeca0 0x564fc87f1090 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f7a800a5510 tx=0x7f7a800990f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846a012a053f2782 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:08.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.546+0000 7f7a86ffd640 1 --2- 172.21.15.52:0/2448398640 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f7a700a3d20 0x7f7a700a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009900 tx=0x7f7a7c02c760 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:08.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.547+0000 7f7a6dffb640 1 -- 172.21.15.52:0/3066145241 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8009d4d0 con 0x564fc87eeca0 2024-04-02T14:00:08.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.547+0000 7f7a6dffb640 1 -- 172.21.15.52:0/3066145241 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a8009cdd0 con 0x564fc87eeca0 2024-04-02T14:00:08.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.547+0000 7f7a6dffb640 1 -- 172.21.15.52:0/3066145241 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a800ce990 con 0x564fc87eeca0 2024-04-02T14:00:08.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.547+0000 7f7a90986740 1 -- 172.21.15.52:0/3066145241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87eeca0 msgr2=0x564fc87f1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.547+0000 7f7a90986740 1 --2- 172.21.15.52:0/3066145241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87eeca0 0x564fc87f1090 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f7a800a5510 tx=0x7f7a800990f0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.547+0000 7f7a90986740 1 -- 172.21.15.52:0/3066145241 shutdown_connections 2024-04-02T14:00:08.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.547+0000 7f7a90986740 1 --2- 172.21.15.52:0/3066145241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87eeca0 0x564fc87f1090 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.547+0000 7f7a90986740 1 --2- 172.21.15.52:0/3066145241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564fc87e60c0 0x564fc87ee760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.547+0000 7f7a90986740 1 --2- 172.21.15.52:0/3066145241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fc87e57a0 0x564fc87e5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.547+0000 7f7a90986740 1 -- 172.21.15.52:0/3066145241 >> 172.21.15.52:0/3066145241 conn(0x564fc87dfed0 msgr2=0x564fc87e02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:08.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.547+0000 7f7a90986740 1 -- 172.21.15.52:0/3066145241 shutdown_connections 2024-04-02T14:00:08.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.547+0000 7f7a90986740 1 -- 172.21.15.52:0/3066145241 wait complete. 2024-04-02T14:00:08.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.548+0000 7f7a90986740 1 -- start start 2024-04-02T14:00:08.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.548+0000 7f7a90986740 1 Processor -- start 2024-04-02T14:00:08.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.548+0000 7f7a90986740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fc87e57a0 0x564fc87ee600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.548+0000 7f7a90986740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564fc87e60c0 0x564fc87eb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.548+0000 7f7a90986740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87eeca0 0x564fc87ebef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.548+0000 7f7a90986740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564fc87f1a90 con 0x564fc87eeca0 2024-04-02T14:00:08.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.548+0000 7f7a90986740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564fc8704e80 con 0x564fc87e60c0 2024-04-02T14:00:08.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.548+0000 7f7a90986740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564fc87f1c10 con 0x564fc87e57a0 2024-04-02T14:00:08.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.548+0000 7f7a90986740 10 client.0 _ll_register_callbacks cb 0x564fc8740aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:08.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.548+0000 7f7a87fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87eeca0 0x564fc87ebef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.548+0000 7f7a87fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87eeca0 0x564fc87ebef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34624/0 (socket says 172.21.15.52:34624) 2024-04-02T14:00:08.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.548+0000 7f7a87fff640 1 -- 172.21.15.52:0/453896790 learned_addr learned my addr 172.21.15.52:0/453896790 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:08.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.548+0000 7f7a877fe640 1 --2- 172.21.15.52:0/453896790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fc87e57a0 0x564fc87ee600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.548+0000 7f7a86ffd640 1 --2- 172.21.15.52:0/453896790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564fc87e60c0 0x564fc87eb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.549+0000 7f7a87fff640 1 -- 172.21.15.52:0/453896790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fc87e57a0 msgr2=0x564fc87ee600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.549+0000 7f7a87fff640 1 --2- 172.21.15.52:0/453896790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fc87e57a0 0x564fc87ee600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.549+0000 7f7a87fff640 1 -- 172.21.15.52:0/453896790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564fc87e60c0 msgr2=0x564fc87eb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.549+0000 7f7a87fff640 1 --2- 172.21.15.52:0/453896790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564fc87e60c0 0x564fc87eb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.549+0000 7f7a87fff640 1 -- 172.21.15.52:0/453896790 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fc85277d0 con 0x564fc87eeca0 2024-04-02T14:00:08.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.549+0000 7f7a87fff640 1 --2- 172.21.15.52:0/453896790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87eeca0 0x564fc87ebef0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f7a800049d0 tx=0x7f7a800ce520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e0f214eee91e5a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:08.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.549+0000 7f7a66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:08.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.549+0000 7f7a66ffd640 1 -- 172.21.15.52:0/453896790 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8007f020 con 0x564fc87eeca0 2024-04-02T14:00:08.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.549+0000 7f7a66ffd640 1 -- 172.21.15.52:0/453896790 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f7a8005dd10 con 0x564fc87eeca0 2024-04-02T14:00:08.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.549+0000 7f7a90986740 1 -- 172.21.15.52:0/453896790 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a8009cdd0 con 0x564fc87eeca0 2024-04-02T14:00:08.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.549+0000 7f7a66ffd640 1 -- 172.21.15.52:0/453896790 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7a800a5a40 con 0x564fc87eeca0 2024-04-02T14:00:08.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.549+0000 7f7a90986740 10 client.17550 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:08.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.549+0000 7f7a90986740 1 -- 172.21.15.52:0/453896790 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564fc87f1a90 con 0x564fc87eeca0 2024-04-02T14:00:08.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.549+0000 7f7a66ffd640 1 -- 172.21.15.52:0/453896790 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a8009cdd0 con 0x564fc87eeca0 2024-04-02T14:00:08.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.549+0000 7f7a66ffd640 1 -- 172.21.15.52:0/453896790 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7a80005310 con 0x564fc87eeca0 2024-04-02T14:00:08.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.549+0000 7f7a90986740 10 client.17550 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:08.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.549+0000 7f7a90986740 10 client.17550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:00:08.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.549+0000 7f7a90986740 20 client.17550 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:08.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.549+0000 7f7a90986740 1 --2- 172.21.15.52:0/453896790 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564fc87feb80 0x564fc8800f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.549+0000 7f7a90986740 4 client.17550 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:00:08.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.549+0000 7f7a90986740 1 -- 172.21.15.52:0/453896790 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fc8756cf0 con 0x564fc87feb80 2024-04-02T14:00:08.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.550+0000 7f7a877fe640 1 --2- 172.21.15.52:0/453896790 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564fc87feb80 0x564fc8800f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.550+0000 7f7a877fe640 1 --2- 172.21.15.52:0/453896790 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564fc87feb80 0x564fc8800f70 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd90f4baf9253eb1 server_cookie=2e685480cd672331 in_seq=0 out_seq=0 2024-04-02T14:00:08.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.550+0000 7f7a66ffd640 10 client.17550 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:08.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.551+0000 7f7a66ffd640 1 -- 172.21.15.52:0/453896790 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564fc8756cf0 con 0x564fc87feb80 2024-04-02T14:00:08.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.551+0000 7f7a66ffd640 10 client.17550 handle_command_reply: tid=0 2024-04-02T14:00:08.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.551+0000 7f7a90986740 1 client.17550 shutdown 2024-04-02T14:00:08.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.551+0000 7f7a90986740 1 -- 172.21.15.52:0/453896790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87eeca0 msgr2=0x564fc87ebef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.551+0000 7f7a90986740 1 --2- 172.21.15.52:0/453896790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87eeca0 0x564fc87ebef0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f7a800049d0 tx=0x7f7a800ce520 comp rx=0 tx=0).stop 2024-04-02T14:00:08.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.551+0000 7f7a90986740 1 -- 172.21.15.52:0/453896790 shutdown_connections 2024-04-02T14:00:08.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.551+0000 7f7a90986740 1 --2- 172.21.15.52:0/453896790 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564fc87feb80 0x564fc8800f70 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.551+0000 7f7a90986740 1 --2- 172.21.15.52:0/453896790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87eeca0 0x564fc87ebef0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.551+0000 7f7a90986740 1 --2- 172.21.15.52:0/453896790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564fc87e60c0 0x564fc87eb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.551+0000 7f7a90986740 1 --2- 172.21.15.52:0/453896790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fc87e57a0 0x564fc87ee600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.551+0000 7f7a66ffd640 0 client.17550 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:08.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.551+0000 7f7a90986740 1 -- 172.21.15.52:0/453896790 >> 172.21.15.52:0/453896790 conn(0x564fc87dfed0 msgr2=0x564fc87e1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:08.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.551+0000 7f7a90986740 1 -- 172.21.15.52:0/453896790 shutdown_connections 2024-04-02T14:00:08.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.551+0000 7f7a90986740 1 -- 172.21.15.52:0/453896790 wait complete. 2024-04-02T14:00:08.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.551+0000 7f7a90986740 20 client.17550 trim_cache size 0 max 16384 2024-04-02T14:00:08.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.565+0000 7f7a90986740 1 Processor -- start 2024-04-02T14:00:08.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.565+0000 7f7a90986740 1 -- start start 2024-04-02T14:00:08.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.565+0000 7f7a90986740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87e57a0 0x564fc87fa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.565+0000 7f7a90986740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fc87e60c0 0x564fc87e1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.565+0000 7f7a90986740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564fc87eeca0 0x564fc87ee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.565+0000 7f7a90986740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7a881aef10 con 0x564fc87e57a0 2024-04-02T14:00:08.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.565+0000 7f7a90986740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7a881b0a50 con 0x564fc87eeca0 2024-04-02T14:00:08.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.565+0000 7f7a90986740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7a881af5e0 con 0x564fc87e60c0 2024-04-02T14:00:08.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.565+0000 7f7a877fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87e57a0 0x564fc87fa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.565+0000 7f7a877fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87e57a0 0x564fc87fa950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34632/0 (socket says 172.21.15.52:34632) 2024-04-02T14:00:08.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.565+0000 7f7a877fe640 1 -- 172.21.15.52:0/3900025820 learned_addr learned my addr 172.21.15.52:0/3900025820 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:08.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.565+0000 7f7a877fe640 1 -- 172.21.15.52:0/3900025820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fc87e60c0 msgr2=0x564fc87e1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.565+0000 7f7a877fe640 1 --2- 172.21.15.52:0/3900025820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fc87e60c0 0x564fc87e1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.565+0000 7f7a877fe640 1 -- 172.21.15.52:0/3900025820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564fc87eeca0 msgr2=0x564fc87ee0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.565+0000 7f7a877fe640 1 --2- 172.21.15.52:0/3900025820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564fc87eeca0 0x564fc87ee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.565+0000 7f7a877fe640 1 -- 172.21.15.52:0/3900025820 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fc87e9ee0 con 0x564fc87e57a0 2024-04-02T14:00:08.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.565+0000 7f7a877fe640 1 --2- 172.21.15.52:0/3900025820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87e57a0 0x564fc87fa950 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x564fc87498e0 tx=0x7f7a78005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a5b78bfad61576 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:08.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.565+0000 7f7a6d7fa640 1 -- 172.21.15.52:0/3900025820 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78004250 con 0x564fc87e57a0 2024-04-02T14:00:08.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.565+0000 7f7a6d7fa640 1 -- 172.21.15.52:0/3900025820 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a780043f0 con 0x564fc87e57a0 2024-04-02T14:00:08.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.565+0000 7f7a6d7fa640 1 -- 172.21.15.52:0/3900025820 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78020110 con 0x564fc87e57a0 2024-04-02T14:00:08.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.566+0000 7f7a90986740 1 -- 172.21.15.52:0/3900025820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87e57a0 msgr2=0x564fc87fa950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.566+0000 7f7a90986740 1 --2- 172.21.15.52:0/3900025820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87e57a0 0x564fc87fa950 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x564fc87498e0 tx=0x7f7a78005f90 comp rx=0 tx=0).stop 2024-04-02T14:00:08.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.566+0000 7f7a90986740 1 -- 172.21.15.52:0/3900025820 shutdown_connections 2024-04-02T14:00:08.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.566+0000 7f7a90986740 1 --2- 172.21.15.52:0/3900025820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564fc87eeca0 0x564fc87ee0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.566+0000 7f7a90986740 1 --2- 172.21.15.52:0/3900025820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fc87e60c0 0x564fc87e1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.566+0000 7f7a90986740 1 --2- 172.21.15.52:0/3900025820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87e57a0 0x564fc87fa950 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.566+0000 7f7a90986740 1 -- 172.21.15.52:0/3900025820 >> 172.21.15.52:0/3900025820 conn(0x564fc87dfed0 msgr2=0x564fc87e1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:08.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.566+0000 7f7a90986740 1 -- 172.21.15.52:0/3900025820 shutdown_connections 2024-04-02T14:00:08.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.566+0000 7f7a90986740 1 -- 172.21.15.52:0/3900025820 wait complete. 2024-04-02T14:00:08.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.566+0000 7f7a90986740 1 -- start start 2024-04-02T14:00:08.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.567+0000 7f7a90986740 1 Processor -- start 2024-04-02T14:00:08.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.567+0000 7f7a90986740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87e57a0 0x564fc87ef8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.567+0000 7f7a90986740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564fc87e60c0 0x564fc87efe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.567+0000 7f7a90986740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fc87eeca0 0x564fc87f0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.567+0000 7f7a90986740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7a881b0380 con 0x564fc87e57a0 2024-04-02T14:00:08.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.567+0000 7f7a90986740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7a881afcb0 con 0x564fc87e60c0 2024-04-02T14:00:08.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.567+0000 7f7a90986740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7a881acef0 con 0x564fc87eeca0 2024-04-02T14:00:08.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.567+0000 7f7a90986740 10 client.0 _ll_register_callbacks cb 0x7f7a8009c7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:08.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.567+0000 7f7a877fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87e57a0 0x564fc87ef8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.567+0000 7f7a877fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87e57a0 0x564fc87ef8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34640/0 (socket says 172.21.15.52:34640) 2024-04-02T14:00:08.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.567+0000 7f7a877fe640 1 -- 172.21.15.52:0/3771876205 learned_addr learned my addr 172.21.15.52:0/3771876205 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:08.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.567+0000 7f7a87fff640 1 --2- 172.21.15.52:0/3771876205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fc87eeca0 0x564fc87f0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.567+0000 7f7a86ffd640 1 --2- 172.21.15.52:0/3771876205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564fc87e60c0 0x564fc87efe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.567+0000 7f7a877fe640 1 -- 172.21.15.52:0/3771876205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fc87eeca0 msgr2=0x564fc87f0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.567+0000 7f7a877fe640 1 --2- 172.21.15.52:0/3771876205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fc87eeca0 0x564fc87f0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.567+0000 7f7a877fe640 1 -- 172.21.15.52:0/3771876205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564fc87e60c0 msgr2=0x564fc87efe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:08.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.567+0000 7f7a877fe640 1 --2- 172.21.15.52:0/3771876205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564fc87e60c0 0x564fc87efe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.567+0000 7f7a877fe640 1 -- 172.21.15.52:0/3771876205 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fc87f3f20 con 0x564fc87e57a0 2024-04-02T14:00:08.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.567+0000 7f7a86ffd640 1 --2- 172.21.15.52:0/3771876205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564fc87e60c0 0x564fc87efe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:00:08.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.567+0000 7f7a87fff640 1 --2- 172.21.15.52:0/3771876205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fc87eeca0 0x564fc87f0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:00:08.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.567+0000 7f7a877fe640 1 --2- 172.21.15.52:0/3771876205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87e57a0 0x564fc87ef8e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009e00 tx=0x7f7a78004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d625b61433168b94 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:08.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.567+0000 7f7a66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:08.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.567+0000 7f7a66ffd640 1 -- 172.21.15.52:0/3771876205 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78030020 con 0x564fc87e57a0 2024-04-02T14:00:08.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.568+0000 7f7a66ffd640 1 -- 172.21.15.52:0/3771876205 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7f7a78004ac0 con 0x564fc87e57a0 2024-04-02T14:00:08.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.568+0000 7f7a90986740 1 -- 172.21.15.52:0/3771876205 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a780043f0 con 0x564fc87e57a0 2024-04-02T14:00:08.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.568+0000 7f7a66ffd640 1 -- 172.21.15.52:0/3771876205 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7a7803b050 con 0x564fc87e57a0 2024-04-02T14:00:08.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.568+0000 7f7a90986740 10 client.17556 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:08.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.568+0000 7f7a90986740 1 -- 172.21.15.52:0/3771876205 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564fc8865030 con 0x564fc87e57a0 2024-04-02T14:00:08.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.568+0000 7f7a66ffd640 1 -- 172.21.15.52:0/3771876205 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a780043f0 con 0x564fc87e57a0 2024-04-02T14:00:08.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.568+0000 7f7a66ffd640 1 -- 172.21.15.52:0/3771876205 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7a78020520 con 0x564fc87e57a0 2024-04-02T14:00:08.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.568+0000 7f7a90986740 10 client.17556 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:08.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.568+0000 7f7a90986740 10 client.17556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:00:08.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.568+0000 7f7a90986740 20 client.17556 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:08.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.568+0000 7f7a90986740 1 --2- 172.21.15.52:0/3771876205 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564fc8865750 0x564fc8867b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:08.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.568+0000 7f7a90986740 4 client.17556 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:00:08.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.568+0000 7f7a90986740 1 -- 172.21.15.52:0/3771876205 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564fc8751040 con 0x564fc8865750 2024-04-02T14:00:08.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.568+0000 7f7a86ffd640 1 --2- 172.21.15.52:0/3771876205 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564fc8865750 0x564fc8867b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:08.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.569+0000 7f7a66ffd640 10 client.17556 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:08.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.569+0000 7f7a86ffd640 1 --2- 172.21.15.52:0/3771876205 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564fc8865750 0x564fc8867b70 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e087f24ef84deea3 server_cookie=bb35d161a7b6798d in_seq=0 out_seq=0 2024-04-02T14:00:08.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.569+0000 7f7a66ffd640 1 -- 172.21.15.52:0/3771876205 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564fc8751040 con 0x564fc8865750 2024-04-02T14:00:08.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.569+0000 7f7a66ffd640 10 client.17556 handle_command_reply: tid=0 2024-04-02T14:00:08.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.569+0000 7f7a90986740 1 client.17556 shutdown 2024-04-02T14:00:08.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.569+0000 7f7a90986740 1 -- 172.21.15.52:0/3771876205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87e57a0 msgr2=0x564fc87ef8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.569+0000 7f7a90986740 1 --2- 172.21.15.52:0/3771876205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87e57a0 0x564fc87ef8e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009e00 tx=0x7f7a78004820 comp rx=0 tx=0).stop 2024-04-02T14:00:08.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.569+0000 7f7a90986740 1 -- 172.21.15.52:0/3771876205 shutdown_connections 2024-04-02T14:00:08.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.569+0000 7f7a90986740 1 --2- 172.21.15.52:0/3771876205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564fc87eeca0 0x564fc87f0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.569+0000 7f7a90986740 1 --2- 172.21.15.52:0/3771876205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564fc87e60c0 0x564fc87efe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.569+0000 7f7a90986740 1 --2- 172.21.15.52:0/3771876205 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x564fc8865750 0x564fc8867b70 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.569+0000 7f7a90986740 1 --2- 172.21.15.52:0/3771876205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fc87e57a0 0x564fc87ef8e0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.569+0000 7f7a66ffd640 0 client.17556 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:08.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.569+0000 7f7a90986740 1 -- 172.21.15.52:0/3771876205 >> 172.21.15.52:0/3771876205 conn(0x564fc87dfed0 msgr2=0x564fc87e1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:08.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.569+0000 7f7a90986740 1 -- 172.21.15.52:0/3771876205 shutdown_connections 2024-04-02T14:00:08.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.569+0000 7f7a90986740 1 -- 172.21.15.52:0/3771876205 wait complete. 2024-04-02T14:00:08.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.569+0000 7f7a90986740 20 client.17556 trim_cache size 0 max 16384 2024-04-02T14:00:08.587 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:00:08.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.570+0000 7f7a8dcc5640 1 -- 172.21.15.52:0/2448398640 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f7a700a3d20 msgr2=0x7f7a700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.570+0000 7f7a8dcc5640 1 --2- 172.21.15.52:0/2448398640 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f7a700a3d20 0x7f7a700a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009900 tx=0x7f7a7c02c760 comp rx=0 tx=0).stop 2024-04-02T14:00:08.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.570+0000 7f7a8dcc5640 1 -- 172.21.15.52:0/2448398640 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a88107a80 msgr2=0x7f7a8819cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:08.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.570+0000 7f7a8dcc5640 1 --2- 172.21.15.52:0/2448398640 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a88107a80 0x7f7a8819cce0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f7a8000eb10 tx=0x7f7a8000c2d0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.570+0000 7f7a8dcc5640 1 -- 172.21.15.52:0/2448398640 shutdown_connections 2024-04-02T14:00:08.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.570+0000 7f7a8dcc5640 1 --2- 172.21.15.52:0/2448398640 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f7a700a3d20 0x7f7a700a61e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.570+0000 7f7a8dcc5640 1 --2- 172.21.15.52:0/2448398640 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a88107a80 0x7f7a8819cce0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.570+0000 7f7a8dcc5640 1 --2- 172.21.15.52:0/2448398640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a880696a0 0x7f7a88198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.570+0000 7f7a8dcc5640 1 --2- 172.21.15.52:0/2448398640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a88068ce0 0x7f7a88198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:08.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.570+0000 7f7a8dcc5640 1 -- 172.21.15.52:0/2448398640 >> 172.21.15.52:0/2448398640 conn(0x7f7a88077b00 msgr2=0x7f7a880fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:08.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.570+0000 7f7a8dcc5640 1 -- 172.21.15.52:0/2448398640 shutdown_connections 2024-04-02T14:00:08.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:08.570+0000 7f7a8dcc5640 1 -- 172.21.15.52:0/2448398640 wait complete. 2024-04-02T14:00:08.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:08 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3578003971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:00:08.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:08 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3578003971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:00:09.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:08 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3578003971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:00:09.590 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T14:00:09.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:00:09.591 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T14:00:09.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.707+0000 7fc77d83c640 1 -- 172.21.15.52:0/271423879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7781069c0 msgr2=0x7fc778106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:09.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.707+0000 7fc77d83c640 1 --2- 172.21.15.52:0/271423879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7781069c0 0x7fc778106da0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fc768009930 tx=0x7fc76802bc30 comp rx=0 tx=0).stop 2024-04-02T14:00:09.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.707+0000 7fc77d83c640 1 -- 172.21.15.52:0/271423879 shutdown_connections 2024-04-02T14:00:09.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.707+0000 7fc77d83c640 1 --2- 172.21.15.52:0/271423879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc778101370 0x7fc77810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.707+0000 7fc77d83c640 1 --2- 172.21.15.52:0/271423879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7781009b0 0x7fc778100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.707+0000 7fc77d83c640 1 --2- 172.21.15.52:0/271423879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7781069c0 0x7fc778106da0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.707+0000 7fc77d83c640 1 -- 172.21.15.52:0/271423879 >> 172.21.15.52:0/271423879 conn(0x7fc778076d80 msgr2=0x7fc778077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:09.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.707+0000 7fc77d83c640 1 -- 172.21.15.52:0/271423879 shutdown_connections 2024-04-02T14:00:09.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.707+0000 7fc77d83c640 1 -- 172.21.15.52:0/271423879 wait complete. 2024-04-02T14:00:09.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.708+0000 7fc77d83c640 1 Processor -- start 2024-04-02T14:00:09.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.708+0000 7fc77d83c640 1 -- start start 2024-04-02T14:00:09.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.708+0000 7fc77d83c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7781009b0 0x7fc778199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.708+0000 7fc77d83c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc778101370 0x7fc778199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.708+0000 7fc77d83c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7781069c0 0x7fc7781a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.708+0000 7fc77d83c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc778110dd0 con 0x7fc7781069c0 2024-04-02T14:00:09.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.708+0000 7fc77d83c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc778110c50 con 0x7fc7781009b0 2024-04-02T14:00:09.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.708+0000 7fc77d83c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc778110f50 con 0x7fc778101370 2024-04-02T14:00:09.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.708+0000 7fc7777fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7781069c0 0x7fc7781a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.708+0000 7fc7777fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7781069c0 0x7fc7781a42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34664/0 (socket says 172.21.15.52:34664) 2024-04-02T14:00:09.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.708+0000 7fc7777fe640 1 -- 172.21.15.52:0/4267409699 learned_addr learned my addr 172.21.15.52:0/4267409699 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:09.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.708+0000 7fc776ffd640 1 --2- 172.21.15.52:0/4267409699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7781009b0 0x7fc778199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.708+0000 7fc7767fc640 1 --2- 172.21.15.52:0/4267409699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc778101370 0x7fc778199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.708+0000 7fc7777fe640 1 -- 172.21.15.52:0/4267409699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc778101370 msgr2=0x7fc778199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:09.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.708+0000 7fc7777fe640 1 --2- 172.21.15.52:0/4267409699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc778101370 0x7fc778199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.708+0000 7fc7777fe640 1 -- 172.21.15.52:0/4267409699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7781009b0 msgr2=0x7fc778199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:09.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.708+0000 7fc7777fe640 1 --2- 172.21.15.52:0/4267409699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7781009b0 0x7fc778199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.708+0000 7fc7777fe640 1 -- 172.21.15.52:0/4267409699 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7781a4810 con 0x7fc7781069c0 2024-04-02T14:00:09.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.709+0000 7fc7777fe640 1 --2- 172.21.15.52:0/4267409699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7781069c0 0x7fc7781a42d0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fc77000eaa0 tx=0x7fc77000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:09.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.709+0000 7fc77c83a640 1 -- 172.21.15.52:0/4267409699 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77000cdb0 con 0x7fc7781069c0 2024-04-02T14:00:09.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.709+0000 7fc77c83a640 1 -- 172.21.15.52:0/4267409699 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc770005020 con 0x7fc7781069c0 2024-04-02T14:00:09.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.709+0000 7fc77d83c640 1 -- 172.21.15.52:0/4267409699 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7781a49e0 con 0x7fc7781069c0 2024-04-02T14:00:09.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.709+0000 7fc77c83a640 1 -- 172.21.15.52:0/4267409699 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7700116b0 con 0x7fc7781069c0 2024-04-02T14:00:09.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.709+0000 7fc77d83c640 1 -- 172.21.15.52:0/4267409699 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7781a4f00 con 0x7fc7781069c0 2024-04-02T14:00:09.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.712+0000 7fc77c83a640 1 -- 172.21.15.52:0/4267409699 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc770005560 con 0x7fc7781069c0 2024-04-02T14:00:09.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.712+0000 7fc780429740 1 Processor -- start 2024-04-02T14:00:09.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.712+0000 7fc780429740 1 -- start start 2024-04-02T14:00:09.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.712+0000 7fc780429740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8071f4ca0 0x55d8071f5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.712+0000 7fc780429740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8071f5650 0x55d807202690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.712+0000 7fc780429740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d807202bd0 0x55d807204fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.712+0000 7fc780429740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d807159770 con 0x55d8071f4ca0 2024-04-02T14:00:09.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.712+0000 7fc780429740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d807101960 con 0x55d8071f5650 2024-04-02T14:00:09.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.712+0000 7fc780429740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d8071558c0 con 0x55d807202bd0 2024-04-02T14:00:09.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.712+0000 7fc7767fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8071f4ca0 0x55d8071f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.712+0000 7fc7767fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8071f4ca0 0x55d8071f5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34666/0 (socket says 172.21.15.52:34666) 2024-04-02T14:00:09.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.712+0000 7fc7767fc640 1 -- 172.21.15.52:0/268030530 learned_addr learned my addr 172.21.15.52:0/268030530 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:09.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.712+0000 7fc77c83a640 1 --2- 172.21.15.52:0/4267409699 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc7600a3d20 0x7fc7600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.713+0000 7fc776ffd640 1 --2- 172.21.15.52:0/268030530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8071f5650 0x55d807202690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.713+0000 7fc7777fe640 1 --2- 172.21.15.52:0/4267409699 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc7600a3d20 0x7fc7600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.713+0000 7fc7767fc640 1 -- 172.21.15.52:0/268030530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d807202bd0 msgr2=0x55d807204fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:09.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.713+0000 7fc7767fc640 1 --2- 172.21.15.52:0/268030530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d807202bd0 0x55d807204fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.713+0000 7fc7767fc640 1 -- 172.21.15.52:0/268030530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8071f5650 msgr2=0x55d807202690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:09.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.713+0000 7fc7767fc640 1 --2- 172.21.15.52:0/268030530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8071f5650 0x55d807202690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.713+0000 7fc77c83a640 1 -- 172.21.15.52:0/4267409699 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc7700d0d70 con 0x7fc7781069c0 2024-04-02T14:00:09.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.713+0000 7fc7767fc640 1 -- 172.21.15.52:0/268030530 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d806f357e0 con 0x55d8071f4ca0 2024-04-02T14:00:09.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.713+0000 7fc776ffd640 1 --2- 172.21.15.52:0/268030530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8071f5650 0x55d807202690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:00:09.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.713+0000 7fc7767fc640 1 --2- 172.21.15.52:0/268030530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8071f4ca0 0x55d8071f5080 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00bd30 tx=0x7fc76c011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06be88fd735fdd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:09.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.713+0000 7fc7777fe640 1 --2- 172.21.15.52:0/4267409699 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc7600a3d20 0x7fc7600a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc77009c670 tx=0x7fc77006e3b0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:09.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.713+0000 7fc75dffb640 1 -- 172.21.15.52:0/268030530 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c01a400 con 0x55d8071f4ca0 2024-04-02T14:00:09.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.713+0000 7fc75dffb640 1 -- 172.21.15.52:0/268030530 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc76c01aa80 con 0x55d8071f4ca0 2024-04-02T14:00:09.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.713+0000 7fc75dffb640 1 -- 172.21.15.52:0/268030530 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c012ae0 con 0x55d8071f4ca0 2024-04-02T14:00:09.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.714+0000 7fc780429740 1 -- 172.21.15.52:0/268030530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8071f4ca0 msgr2=0x55d8071f5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:09.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.714+0000 7fc780429740 1 --2- 172.21.15.52:0/268030530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8071f4ca0 0x55d8071f5080 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00bd30 tx=0x7fc76c011750 comp rx=0 tx=0).stop 2024-04-02T14:00:09.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.714+0000 7fc780429740 1 -- 172.21.15.52:0/268030530 shutdown_connections 2024-04-02T14:00:09.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.714+0000 7fc780429740 1 --2- 172.21.15.52:0/268030530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d807202bd0 0x55d807204fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.714+0000 7fc780429740 1 --2- 172.21.15.52:0/268030530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8071f5650 0x55d807202690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.714+0000 7fc780429740 1 --2- 172.21.15.52:0/268030530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8071f4ca0 0x55d8071f5080 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.714+0000 7fc780429740 1 -- 172.21.15.52:0/268030530 >> 172.21.15.52:0/268030530 conn(0x55d8071ef9f0 msgr2=0x55d8071efdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:09.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.714+0000 7fc780429740 1 -- 172.21.15.52:0/268030530 shutdown_connections 2024-04-02T14:00:09.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.714+0000 7fc780429740 1 -- 172.21.15.52:0/268030530 wait complete. 2024-04-02T14:00:09.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.714+0000 7fc780429740 1 -- start start 2024-04-02T14:00:09.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.714+0000 7fc780429740 1 Processor -- start 2024-04-02T14:00:09.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.715+0000 7fc780429740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8071f4ca0 0x55d8071ffb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.715+0000 7fc780429740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8071f5650 0x55d8072000b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.715+0000 7fc780429740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d807202bd0 0x55d8072005f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.715+0000 7fc780429740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d807205990 con 0x55d807202bd0 2024-04-02T14:00:09.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.715+0000 7fc780429740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d807114d20 con 0x55d8071f4ca0 2024-04-02T14:00:09.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.715+0000 7fc780429740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d807205b10 con 0x55d8071f5650 2024-04-02T14:00:09.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.715+0000 7fc780429740 10 client.0 _ll_register_callbacks cb 0x55d8070e1a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:09.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.715+0000 7fc7767fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d807202bd0 0x55d8072005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.715+0000 7fc7767fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d807202bd0 0x55d8072005f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34678/0 (socket says 172.21.15.52:34678) 2024-04-02T14:00:09.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.715+0000 7fc7767fc640 1 -- 172.21.15.52:0/2578545701 learned_addr learned my addr 172.21.15.52:0/2578545701 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:09.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.715+0000 7fc776ffd640 1 --2- 172.21.15.52:0/2578545701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8071f5650 0x55d8072000b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.715+0000 7fc7767fc640 1 --2- 172.21.15.52:0/2578545701 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8071f4ca0 0x55d8071ffb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.715+0000 7fc7767fc640 1 -- 172.21.15.52:0/2578545701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8071f5650 msgr2=0x55d8072000b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:09.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.715+0000 7fc7767fc640 1 --2- 172.21.15.52:0/2578545701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8071f5650 0x55d8072000b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.715+0000 7fc7767fc640 1 -- 172.21.15.52:0/2578545701 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8071f4ca0 msgr2=0x55d8071ffb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:09.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.715+0000 7fc7767fc640 1 --2- 172.21.15.52:0/2578545701 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8071f4ca0 0x55d8071ffb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.715+0000 7fc7767fc640 1 -- 172.21.15.52:0/2578545701 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d806f95f70 con 0x55d807202bd0 2024-04-02T14:00:09.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.715+0000 7fc776ffd640 1 --2- 172.21.15.52:0/2578545701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8071f5650 0x55d8072000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:00:09.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.715+0000 7fc7767fc640 1 --2- 172.21.15.52:0/2578545701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d807202bd0 0x55d8072005f0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0058c0 tx=0x7fc76c00b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b54c2e0a643c5a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:09.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.715+0000 7fc756ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:09.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.716+0000 7fc756ffd640 1 -- 172.21.15.52:0/2578545701 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c01a400 con 0x55d807202bd0 2024-04-02T14:00:09.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.716+0000 7fc756ffd640 1 -- 172.21.15.52:0/2578545701 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc76c01a5a0 con 0x55d807202bd0 2024-04-02T14:00:09.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.716+0000 7fc780429740 1 -- 172.21.15.52:0/2578545701 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc76c01aa80 con 0x55d807202bd0 2024-04-02T14:00:09.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.716+0000 7fc756ffd640 1 -- 172.21.15.52:0/2578545701 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc76c01a7a0 con 0x55d807202bd0 2024-04-02T14:00:09.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.716+0000 7fc780429740 10 client.17568 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:09.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.716+0000 7fc780429740 1 -- 172.21.15.52:0/2578545701 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d806e2d170 con 0x55d807202bd0 2024-04-02T14:00:09.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.716+0000 7fc756ffd640 1 -- 172.21.15.52:0/2578545701 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc76c01aa80 con 0x55d807202bd0 2024-04-02T14:00:09.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.716+0000 7fc756ffd640 1 -- 172.21.15.52:0/2578545701 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc76c00f3c0 con 0x55d807202bd0 2024-04-02T14:00:09.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.716+0000 7fc780429740 10 client.17568 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:09.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.716+0000 7fc780429740 10 client.17568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:00:09.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.716+0000 7fc780429740 20 client.17568 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:09.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.716+0000 7fc780429740 1 --2- 172.21.15.52:0/2578545701 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55d807212de0 0x55d8072151d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.716+0000 7fc780429740 4 client.17568 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:00:09.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.716+0000 7fc780429740 1 -- 172.21.15.52:0/2578545701 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d807166cb0 con 0x55d807212de0 2024-04-02T14:00:09.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.717+0000 7fc776ffd640 1 --2- 172.21.15.52:0/2578545701 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55d807212de0 0x55d8072151d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.717+0000 7fc776ffd640 1 --2- 172.21.15.52:0/2578545701 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55d807212de0 0x55d8072151d0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b4eca5a76549463 server_cookie=d6ae083c1c88820a in_seq=0 out_seq=0 2024-04-02T14:00:09.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.717+0000 7fc756ffd640 10 client.17568 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:09.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.718+0000 7fc756ffd640 1 -- 172.21.15.52:0/2578545701 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d807166cb0 con 0x55d807212de0 2024-04-02T14:00:09.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.718+0000 7fc756ffd640 10 client.17568 handle_command_reply: tid=0 2024-04-02T14:00:09.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.718+0000 7fc780429740 1 client.17568 shutdown 2024-04-02T14:00:09.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.718+0000 7fc780429740 1 -- 172.21.15.52:0/2578545701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d807202bd0 msgr2=0x55d8072005f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:09.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.718+0000 7fc780429740 1 --2- 172.21.15.52:0/2578545701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d807202bd0 0x55d8072005f0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0058c0 tx=0x7fc76c00b3a0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.718+0000 7fc780429740 1 -- 172.21.15.52:0/2578545701 shutdown_connections 2024-04-02T14:00:09.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.718+0000 7fc780429740 1 --2- 172.21.15.52:0/2578545701 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55d807212de0 0x55d8072151d0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.718+0000 7fc780429740 1 --2- 172.21.15.52:0/2578545701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d807202bd0 0x55d8072005f0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.718+0000 7fc780429740 1 --2- 172.21.15.52:0/2578545701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8071f5650 0x55d8072000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.718+0000 7fc756ffd640 0 client.17568 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:09.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.718+0000 7fc780429740 1 --2- 172.21.15.52:0/2578545701 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8071f4ca0 0x55d8071ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.718+0000 7fc780429740 1 -- 172.21.15.52:0/2578545701 >> 172.21.15.52:0/2578545701 conn(0x55d8071ef9f0 msgr2=0x55d8071f1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:09.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.718+0000 7fc780429740 1 -- 172.21.15.52:0/2578545701 shutdown_connections 2024-04-02T14:00:09.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.718+0000 7fc780429740 1 -- 172.21.15.52:0/2578545701 wait complete. 2024-04-02T14:00:09.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.718+0000 7fc780429740 20 client.17568 trim_cache size 0 max 16384 2024-04-02T14:00:09.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.736+0000 7fc780429740 1 Processor -- start 2024-04-02T14:00:09.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.736+0000 7fc780429740 1 -- start start 2024-04-02T14:00:09.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.736+0000 7fc780429740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8071f4ca0 0x55d80720e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.736+0000 7fc780429740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8071f5650 0x55d8071ff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.736+0000 7fc780429740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d807202bd0 0x55d80721a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.736+0000 7fc780429740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc7781acd60 con 0x55d8071f4ca0 2024-04-02T14:00:09.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.736+0000 7fc780429740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc7781ae8a0 con 0x55d8071f5650 2024-04-02T14:00:09.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.736+0000 7fc780429740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc7781ad430 con 0x55d807202bd0 2024-04-02T14:00:09.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.736+0000 7fc7767fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8071f4ca0 0x55d80720e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.736+0000 7fc7767fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8071f4ca0 0x55d80720e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34692/0 (socket says 172.21.15.52:34692) 2024-04-02T14:00:09.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.736+0000 7fc7767fc640 1 -- 172.21.15.52:0/1428271800 learned_addr learned my addr 172.21.15.52:0/1428271800 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:09.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.736+0000 7fc776ffd640 1 --2- 172.21.15.52:0/1428271800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8071f5650 0x55d8071ff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.736+0000 7fc7767fc640 1 -- 172.21.15.52:0/1428271800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d807202bd0 msgr2=0x55d80721a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:09.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.736+0000 7fc7767fc640 1 --2- 172.21.15.52:0/1428271800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d807202bd0 0x55d80721a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.736+0000 7fc7767fc640 1 -- 172.21.15.52:0/1428271800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8071f5650 msgr2=0x55d8071ff0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:09.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.737+0000 7fc7767fc640 1 --2- 172.21.15.52:0/1428271800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8071f5650 0x55d8071ff0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.737+0000 7fc7767fc640 1 -- 172.21.15.52:0/1428271800 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8071ef820 con 0x55d8071f4ca0 2024-04-02T14:00:09.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.737+0000 7fc776ffd640 1 --2- 172.21.15.52:0/1428271800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8071f5650 0x55d8071ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:00:09.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.737+0000 7fc7767fc640 1 --2- 172.21.15.52:0/1428271800 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8071f4ca0 0x55d80720e920 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00f810 tx=0x7fc76c011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e687616eea01b5b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:09.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.737+0000 7fc75d7fa640 1 -- 172.21.15.52:0/1428271800 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c01a400 con 0x55d8071f4ca0 2024-04-02T14:00:09.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.737+0000 7fc75d7fa640 1 -- 172.21.15.52:0/1428271800 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc76c01a5a0 con 0x55d8071f4ca0 2024-04-02T14:00:09.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.737+0000 7fc75d7fa640 1 -- 172.21.15.52:0/1428271800 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c01a940 con 0x55d8071f4ca0 2024-04-02T14:00:09.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.737+0000 7fc780429740 1 -- 172.21.15.52:0/1428271800 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8071f4ca0 msgr2=0x55d80720e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:09.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.737+0000 7fc780429740 1 --2- 172.21.15.52:0/1428271800 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8071f4ca0 0x55d80720e920 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00f810 tx=0x7fc76c011d00 comp rx=0 tx=0).stop 2024-04-02T14:00:09.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.738+0000 7fc780429740 1 -- 172.21.15.52:0/1428271800 shutdown_connections 2024-04-02T14:00:09.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.738+0000 7fc780429740 1 --2- 172.21.15.52:0/1428271800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d807202bd0 0x55d80721a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.738+0000 7fc780429740 1 --2- 172.21.15.52:0/1428271800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8071f5650 0x55d8071ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.738+0000 7fc780429740 1 --2- 172.21.15.52:0/1428271800 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8071f4ca0 0x55d80720e920 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.738+0000 7fc780429740 1 -- 172.21.15.52:0/1428271800 >> 172.21.15.52:0/1428271800 conn(0x55d8071ef9f0 msgr2=0x55d8072003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:09.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.738+0000 7fc780429740 1 -- 172.21.15.52:0/1428271800 shutdown_connections 2024-04-02T14:00:09.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.738+0000 7fc780429740 1 -- 172.21.15.52:0/1428271800 wait complete. 2024-04-02T14:00:09.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.738+0000 7fc780429740 1 -- start start 2024-04-02T14:00:09.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.738+0000 7fc780429740 1 Processor -- start 2024-04-02T14:00:09.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.738+0000 7fc780429740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8071f4ca0 0x55d807207120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.738+0000 7fc780429740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8071f5650 0x55d807207660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.738+0000 7fc780429740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d807202bd0 0x55d807272210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.738+0000 7fc780429740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc7781ae1d0 con 0x55d8071f5650 2024-04-02T14:00:09.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.738+0000 7fc780429740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc7781adb00 con 0x55d8071f4ca0 2024-04-02T14:00:09.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.738+0000 7fc780429740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc7781aad40 con 0x55d807202bd0 2024-04-02T14:00:09.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.738+0000 7fc780429740 10 client.0 _ll_register_callbacks cb 0x7fc76c00bc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:09.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.739+0000 7fc776ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8071f5650 0x55d807207660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.739+0000 7fc776ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8071f5650 0x55d807207660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34694/0 (socket says 172.21.15.52:34694) 2024-04-02T14:00:09.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.739+0000 7fc776ffd640 1 -- 172.21.15.52:0/616328293 learned_addr learned my addr 172.21.15.52:0/616328293 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:09.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.739+0000 7fc7767fc640 1 --2- 172.21.15.52:0/616328293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d807202bd0 0x55d807272210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.739+0000 7fc7767fc640 1 --2- 172.21.15.52:0/616328293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8071f4ca0 0x55d807207120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.739+0000 7fc776ffd640 1 -- 172.21.15.52:0/616328293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d807202bd0 msgr2=0x55d807272210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:09.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.739+0000 7fc776ffd640 1 --2- 172.21.15.52:0/616328293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d807202bd0 0x55d807272210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.739+0000 7fc776ffd640 1 -- 172.21.15.52:0/616328293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8071f4ca0 msgr2=0x55d807207120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:09.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.739+0000 7fc776ffd640 1 --2- 172.21.15.52:0/616328293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8071f4ca0 0x55d807207120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.739+0000 7fc776ffd640 1 -- 172.21.15.52:0/616328293 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d807272800 con 0x55d8071f5650 2024-04-02T14:00:09.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.739+0000 7fc7767fc640 1 --2- 172.21.15.52:0/616328293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8071f4ca0 0x55d807207120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:00:09.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.739+0000 7fc776ffd640 1 --2- 172.21.15.52:0/616328293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8071f5650 0x55d807207660 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fc76802cfd0 tx=0x7fc76802ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b430a1cdf8ddf591 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:09.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.739+0000 7fc756ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:09.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.739+0000 7fc756ffd640 1 -- 172.21.15.52:0/616328293 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76804a070 con 0x55d8071f5650 2024-04-02T14:00:09.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.739+0000 7fc756ffd640 1 -- 172.21.15.52:0/616328293 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7fc768046dc0 con 0x55d8071f5650 2024-04-02T14:00:09.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.739+0000 7fc780429740 1 -- 172.21.15.52:0/616328293 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc76c01a5a0 con 0x55d8071f5650 2024-04-02T14:00:09.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.739+0000 7fc756ffd640 1 -- 172.21.15.52:0/616328293 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc768056050 con 0x55d8071f5650 2024-04-02T14:00:09.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.739+0000 7fc780429740 10 client.17574 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:09.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.739+0000 7fc780429740 1 -- 172.21.15.52:0/616328293 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d807279c70 con 0x55d8071f5650 2024-04-02T14:00:09.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.739+0000 7fc756ffd640 1 -- 172.21.15.52:0/616328293 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc76c01a5a0 con 0x55d8071f5650 2024-04-02T14:00:09.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.740+0000 7fc756ffd640 1 -- 172.21.15.52:0/616328293 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc768004750 con 0x55d8071f5650 2024-04-02T14:00:09.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.740+0000 7fc780429740 10 client.17574 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:09.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.740+0000 7fc780429740 10 client.17574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:00:09.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.740+0000 7fc780429740 20 client.17574 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:09.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.740+0000 7fc780429740 1 --2- 172.21.15.52:0/616328293 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55d80727a420 0x55d80727c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.740+0000 7fc780429740 4 client.17574 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T14:00:09.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.740+0000 7fc780429740 1 -- 172.21.15.52:0/616328293 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d807161000 con 0x55d80727a420 2024-04-02T14:00:09.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.740+0000 7fc7767fc640 1 --2- 172.21.15.52:0/616328293 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55d80727a420 0x55d80727c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.740+0000 7fc756ffd640 10 client.17574 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:09.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.740+0000 7fc7767fc640 1 --2- 172.21.15.52:0/616328293 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55d80727a420 0x55d80727c840 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbf64dae1ef6320f server_cookie=8fbe3ffbe7bed1d5 in_seq=0 out_seq=0 2024-04-02T14:00:09.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.742+0000 7fc756ffd640 1 -- 172.21.15.52:0/616328293 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d807161000 con 0x55d80727a420 2024-04-02T14:00:09.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.742+0000 7fc756ffd640 10 client.17574 handle_command_reply: tid=0 2024-04-02T14:00:09.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.742+0000 7fc780429740 1 client.17574 shutdown 2024-04-02T14:00:09.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.742+0000 7fc780429740 1 -- 172.21.15.52:0/616328293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8071f5650 msgr2=0x55d807207660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:09.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.742+0000 7fc780429740 1 --2- 172.21.15.52:0/616328293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8071f5650 0x55d807207660 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fc76802cfd0 tx=0x7fc76802ca90 comp rx=0 tx=0).stop 2024-04-02T14:00:09.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.742+0000 7fc780429740 1 -- 172.21.15.52:0/616328293 shutdown_connections 2024-04-02T14:00:09.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.742+0000 7fc780429740 1 --2- 172.21.15.52:0/616328293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d807202bd0 0x55d807272210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.742+0000 7fc780429740 1 --2- 172.21.15.52:0/616328293 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55d80727a420 0x55d80727c840 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.742+0000 7fc780429740 1 --2- 172.21.15.52:0/616328293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8071f5650 0x55d807207660 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.742+0000 7fc756ffd640 0 client.17574 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:09.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.742+0000 7fc780429740 1 --2- 172.21.15.52:0/616328293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8071f4ca0 0x55d807207120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.742+0000 7fc780429740 1 -- 172.21.15.52:0/616328293 >> 172.21.15.52:0/616328293 conn(0x55d8071ef9f0 msgr2=0x55d8071ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:09.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.742+0000 7fc780429740 1 -- 172.21.15.52:0/616328293 shutdown_connections 2024-04-02T14:00:09.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.742+0000 7fc780429740 1 -- 172.21.15.52:0/616328293 wait complete. 2024-04-02T14:00:09.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.742+0000 7fc780429740 20 client.17574 trim_cache size 0 max 16384 2024-04-02T14:00:09.755 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:00:09.755 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-04-02T14:00:09.756 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "54474adf-879d-4612-bd3c-40cb8efbffb2", 2024-04-02T14:00:09.756 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-04-02T14:00:09.756 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:00:09.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.743+0000 7fc77d83c640 1 -- 172.21.15.52:0/4267409699 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc7600a3d20 msgr2=0x7fc7600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:09.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.743+0000 7fc77d83c640 1 --2- 172.21.15.52:0/4267409699 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc7600a3d20 0x7fc7600a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc77009c670 tx=0x7fc77006e3b0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.743+0000 7fc77d83c640 1 -- 172.21.15.52:0/4267409699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7781069c0 msgr2=0x7fc7781a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:09.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.743+0000 7fc77d83c640 1 --2- 172.21.15.52:0/4267409699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7781069c0 0x7fc7781a42d0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fc77000eaa0 tx=0x7fc77000ef60 comp rx=0 tx=0).stop 2024-04-02T14:00:09.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.743+0000 7fc77d83c640 1 -- 172.21.15.52:0/4267409699 shutdown_connections 2024-04-02T14:00:09.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.743+0000 7fc77d83c640 1 --2- 172.21.15.52:0/4267409699 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc7600a3d20 0x7fc7600a61e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.743+0000 7fc77d83c640 1 --2- 172.21.15.52:0/4267409699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7781069c0 0x7fc7781a42d0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.743+0000 7fc77d83c640 1 --2- 172.21.15.52:0/4267409699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc778101370 0x7fc778199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.743+0000 7fc77d83c640 1 --2- 172.21.15.52:0/4267409699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7781009b0 0x7fc778199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.743+0000 7fc77d83c640 1 -- 172.21.15.52:0/4267409699 >> 172.21.15.52:0/4267409699 conn(0x7fc778076d80 msgr2=0x7fc7780fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:09.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.743+0000 7fc77d83c640 1 -- 172.21.15.52:0/4267409699 shutdown_connections 2024-04-02T14:00:09.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.743+0000 7fc77d83c640 1 -- 172.21.15.52:0/4267409699 wait complete. 2024-04-02T14:00:09.759 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:00:09.759 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:00:09.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:09 smithi052 ceph-mon[32104]: pgmap v202: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-02T14:00:09.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.865+0000 7fec04f04640 1 -- 172.21.15.52:0/429877352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec00107ab0 msgr2=0x7fec00107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:09.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.865+0000 7fec04f04640 1 --2- 172.21.15.52:0/429877352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec00107ab0 0x7fec00107e90 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7febf0009930 tx=0x7febf002bc30 comp rx=0 tx=0).stop 2024-04-02T14:00:09.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.865+0000 7fec04f04640 1 -- 172.21.15.52:0/429877352 shutdown_connections 2024-04-02T14:00:09.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.865+0000 7fec04f04640 1 --2- 172.21.15.52:0/429877352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec00102460 0x7fec0010e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.865+0000 7fec04f04640 1 --2- 172.21.15.52:0/429877352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec00101aa0 0x7fec00101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.865+0000 7fec04f04640 1 --2- 172.21.15.52:0/429877352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec00107ab0 0x7fec00107e90 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.865+0000 7fec04f04640 1 -- 172.21.15.52:0/429877352 >> 172.21.15.52:0/429877352 conn(0x7fec000fd590 msgr2=0x7fec000ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:09.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.865+0000 7fec04f04640 1 -- 172.21.15.52:0/429877352 shutdown_connections 2024-04-02T14:00:09.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.866+0000 7fec04f04640 1 -- 172.21.15.52:0/429877352 wait complete. 2024-04-02T14:00:09.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.866+0000 7fec04f04640 1 Processor -- start 2024-04-02T14:00:09.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.866+0000 7fec04f04640 1 -- start start 2024-04-02T14:00:09.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.866+0000 7fec04f04640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec00101aa0 0x7fec00073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.867+0000 7fec04f04640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec00102460 0x7fec00074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.867+0000 7fec04f04640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec00107ab0 0x7fec00074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.867+0000 7fec04f04640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fec00111ec0 con 0x7fec00107ab0 2024-04-02T14:00:09.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.867+0000 7fec04f04640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fec00111d40 con 0x7fec00101aa0 2024-04-02T14:00:09.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.867+0000 7fec04f04640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fec00112040 con 0x7fec00102460 2024-04-02T14:00:09.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.867+0000 7febfed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec00107ab0 0x7fec00074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.867+0000 7febfed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec00107ab0 0x7fec00074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34710/0 (socket says 172.21.15.52:34710) 2024-04-02T14:00:09.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.867+0000 7febfed9d640 1 -- 172.21.15.52:0/2661673962 learned_addr learned my addr 172.21.15.52:0/2661673962 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:09.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.867+0000 7febfdd9b640 1 --2- 172.21.15.52:0/2661673962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec00102460 0x7fec00074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.867+0000 7febfe59c640 1 --2- 172.21.15.52:0/2661673962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec00101aa0 0x7fec00073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.867+0000 7febfed9d640 1 -- 172.21.15.52:0/2661673962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec00102460 msgr2=0x7fec00074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:09.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.867+0000 7febfed9d640 1 --2- 172.21.15.52:0/2661673962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec00102460 0x7fec00074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.867+0000 7febfed9d640 1 -- 172.21.15.52:0/2661673962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec00101aa0 msgr2=0x7fec00073e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:09.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.867+0000 7febfed9d640 1 --2- 172.21.15.52:0/2661673962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec00101aa0 0x7fec00073e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.867+0000 7febfed9d640 1 -- 172.21.15.52:0/2661673962 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec001a6900 con 0x7fec00107ab0 2024-04-02T14:00:09.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.867+0000 7febfed9d640 1 --2- 172.21.15.52:0/2661673962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec00107ab0 0x7fec00074880 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7febf800eb10 tx=0x7febf800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:09.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.867+0000 7febef7fe640 1 -- 172.21.15.52:0/2661673962 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8004b20 con 0x7fec00107ab0 2024-04-02T14:00:09.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.867+0000 7fec04f04640 1 -- 172.21.15.52:0/2661673962 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec001a6bf0 con 0x7fec00107ab0 2024-04-02T14:00:09.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.867+0000 7febef7fe640 1 -- 172.21.15.52:0/2661673962 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febf8004cc0 con 0x7fec00107ab0 2024-04-02T14:00:09.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.867+0000 7febef7fe640 1 -- 172.21.15.52:0/2661673962 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8010760 con 0x7fec00107ab0 2024-04-02T14:00:09.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.867+0000 7fec04f04640 1 -- 172.21.15.52:0/2661673962 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec001a7140 con 0x7fec00107ab0 2024-04-02T14:00:09.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.870+0000 7febef7fe640 1 -- 172.21.15.52:0/2661673962 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7febf8005020 con 0x7fec00107ab0 2024-04-02T14:00:09.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.871+0000 7fec07bc9740 1 Processor -- start 2024-04-02T14:00:09.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.871+0000 7fec07bc9740 1 -- start start 2024-04-02T14:00:09.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.871+0000 7fec07bc9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607faf8f8d0 0x5607faf8fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.871+0000 7fec07bc9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf901f0 0x5607faf98890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.871+0000 7fec07bc9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5607faf98dd0 0x5607faf9b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.871+0000 7fec07bc9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5607faef3830 con 0x5607faf901f0 2024-04-02T14:00:09.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.871+0000 7fec07bc9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5607fae9b820 con 0x5607faf98dd0 2024-04-02T14:00:09.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.871+0000 7fec07bc9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5607faeef7e0 con 0x5607faf8f8d0 2024-04-02T14:00:09.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.871+0000 7febef7fe640 1 --2- 172.21.15.52:0/2661673962 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7febe80a3d20 0x7febe80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.871+0000 7febfe59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf901f0 0x5607faf98890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.871+0000 7febfe59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf901f0 0x5607faf98890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34722/0 (socket says 172.21.15.52:34722) 2024-04-02T14:00:09.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.871+0000 7febfe59c640 1 -- 172.21.15.52:0/2291185299 learned_addr learned my addr 172.21.15.52:0/2291185299 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:09.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.871+0000 7febfdd9b640 1 --2- 172.21.15.52:0/2291185299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607faf8f8d0 0x5607faf8fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.871+0000 7febfed9d640 1 --2- 172.21.15.52:0/2661673962 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7febe80a3d20 0x7febe80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.871+0000 7febfdd9b640 1 --2- 172.21.15.52:0/2291185299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5607faf98dd0 0x5607faf9b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.871+0000 7febef7fe640 1 -- 172.21.15.52:0/2661673962 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7febf80d1030 con 0x7fec00107ab0 2024-04-02T14:00:09.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.872+0000 7febfdd9b640 1 -- 172.21.15.52:0/2291185299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5607faf98dd0 msgr2=0x5607faf9b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:09.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.872+0000 7febfdd9b640 1 --2- 172.21.15.52:0/2291185299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5607faf98dd0 0x5607faf9b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.872+0000 7febfdd9b640 1 -- 172.21.15.52:0/2291185299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf901f0 msgr2=0x5607faf98890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:09.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.872+0000 7febfdd9b640 1 --2- 172.21.15.52:0/2291185299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf901f0 0x5607faf98890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.872+0000 7febfdd9b640 1 -- 172.21.15.52:0/2291185299 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607facd17d0 con 0x5607faf8f8d0 2024-04-02T14:00:09.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.872+0000 7febfdd9b640 1 --2- 172.21.15.52:0/2291185299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607faf8f8d0 0x5607faf8fcb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fec00075890 tx=0x7febf4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bfefc0112ac087e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:09.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.872+0000 7febfed9d640 1 --2- 172.21.15.52:0/2661673962 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7febe80a3d20 0x7febe80a61e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7febf80990f0 tx=0x7febf8092120 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:09.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.872+0000 7febecff9640 1 -- 172.21.15.52:0/2291185299 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4004050 con 0x5607faf8f8d0 2024-04-02T14:00:09.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.872+0000 7febecff9640 1 -- 172.21.15.52:0/2291185299 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febf400fca0 con 0x5607faf8f8d0 2024-04-02T14:00:09.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.872+0000 7febecff9640 1 -- 172.21.15.52:0/2291185299 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4003ab0 con 0x5607faf8f8d0 2024-04-02T14:00:09.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.873+0000 7fec07bc9740 1 -- 172.21.15.52:0/2291185299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607faf8f8d0 msgr2=0x5607faf8fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:09.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.873+0000 7fec07bc9740 1 --2- 172.21.15.52:0/2291185299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607faf8f8d0 0x5607faf8fcb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fec00075890 tx=0x7febf4004730 comp rx=0 tx=0).stop 2024-04-02T14:00:09.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.873+0000 7fec07bc9740 1 -- 172.21.15.52:0/2291185299 shutdown_connections 2024-04-02T14:00:09.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.873+0000 7fec07bc9740 1 --2- 172.21.15.52:0/2291185299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5607faf98dd0 0x5607faf9b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.873+0000 7fec07bc9740 1 --2- 172.21.15.52:0/2291185299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf901f0 0x5607faf98890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.873+0000 7fec07bc9740 1 --2- 172.21.15.52:0/2291185299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607faf8f8d0 0x5607faf8fcb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.873+0000 7fec07bc9740 1 -- 172.21.15.52:0/2291185299 >> 172.21.15.52:0/2291185299 conn(0x5607faf8a000 msgr2=0x5607faf8a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:09.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.873+0000 7fec07bc9740 1 -- 172.21.15.52:0/2291185299 shutdown_connections 2024-04-02T14:00:09.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.873+0000 7fec07bc9740 1 -- 172.21.15.52:0/2291185299 wait complete. 2024-04-02T14:00:09.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.874+0000 7fec07bc9740 1 -- start start 2024-04-02T14:00:09.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.874+0000 7fec07bc9740 1 Processor -- start 2024-04-02T14:00:09.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.874+0000 7fec07bc9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5607faf8f8d0 0x5607faf98680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.874+0000 7fec07bc9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607faf901f0 0x5607faf95a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.874+0000 7fec07bc9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf98dd0 0x5607faf95f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.874+0000 7fec07bc9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5607faf9bbc0 con 0x5607faf98dd0 2024-04-02T14:00:09.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.874+0000 7fec07bc9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5607faeaed30 con 0x5607faf8f8d0 2024-04-02T14:00:09.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.874+0000 7fec07bc9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5607faf9bd40 con 0x5607faf901f0 2024-04-02T14:00:09.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.874+0000 7fec07bc9740 10 client.0 _ll_register_callbacks cb 0x5607faeae720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:09.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.874+0000 7febfdd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf98dd0 0x5607faf95f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.874+0000 7febfdd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf98dd0 0x5607faf95f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34726/0 (socket says 172.21.15.52:34726) 2024-04-02T14:00:09.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.874+0000 7febfdd9b640 1 -- 172.21.15.52:0/1177267507 learned_addr learned my addr 172.21.15.52:0/1177267507 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:09.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.874+0000 7febfdd9b640 1 --2- 172.21.15.52:0/1177267507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5607faf8f8d0 0x5607faf98680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.874+0000 7febfe59c640 1 --2- 172.21.15.52:0/1177267507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607faf901f0 0x5607faf95a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.874+0000 7febfdd9b640 1 -- 172.21.15.52:0/1177267507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607faf901f0 msgr2=0x5607faf95a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:09.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.874+0000 7febfdd9b640 1 --2- 172.21.15.52:0/1177267507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607faf901f0 0x5607faf95a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.874+0000 7febfdd9b640 1 -- 172.21.15.52:0/1177267507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5607faf8f8d0 msgr2=0x5607faf98680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:09.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.874+0000 7febfdd9b640 1 --2- 172.21.15.52:0/1177267507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5607faf8f8d0 0x5607faf98680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.874+0000 7febfdd9b640 1 -- 172.21.15.52:0/1177267507 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607fac073f0 con 0x5607faf98dd0 2024-04-02T14:00:09.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.875+0000 7febfdd9b640 1 --2- 172.21.15.52:0/1177267507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf98dd0 0x5607faf95f50 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7febf40097a0 tx=0x7febf40035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b7914dac6b6246 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:09.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.875+0000 7febe5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:09.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.875+0000 7febe5ffb640 1 -- 172.21.15.52:0/1177267507 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf401f440 con 0x5607faf98dd0 2024-04-02T14:00:09.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.875+0000 7febe5ffb640 1 -- 172.21.15.52:0/1177267507 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7febf401f5e0 con 0x5607faf98dd0 2024-04-02T14:00:09.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.875+0000 7fec07bc9740 1 -- 172.21.15.52:0/1177267507 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febf400fca0 con 0x5607faf98dd0 2024-04-02T14:00:09.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.875+0000 7febe5ffb640 1 -- 172.21.15.52:0/1177267507 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7febf4036050 con 0x5607faf98dd0 2024-04-02T14:00:09.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.875+0000 7fec07bc9740 10 client.17586 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:09.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.875+0000 7fec07bc9740 1 -- 172.21.15.52:0/1177267507 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5607fabc70f0 con 0x5607faf98dd0 2024-04-02T14:00:09.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.875+0000 7febe5ffb640 1 -- 172.21.15.52:0/1177267507 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febf400fca0 con 0x5607faf98dd0 2024-04-02T14:00:09.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.875+0000 7febe5ffb640 1 -- 172.21.15.52:0/1177267507 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7febf4003eb0 con 0x5607faf98dd0 2024-04-02T14:00:09.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.875+0000 7fec07bc9740 10 client.17586 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:09.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.875+0000 7fec07bc9740 10 client.17586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:00:09.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.875+0000 7fec07bc9740 20 client.17586 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:09.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.875+0000 7fec07bc9740 1 --2- 172.21.15.52:0/1177267507 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5607fafa8c20 0x5607fafab010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.875+0000 7fec07bc9740 4 client.17586 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:00:09.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.875+0000 7fec07bc9740 1 -- 172.21.15.52:0/1177267507 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607faf00d70 con 0x5607fafa8c20 2024-04-02T14:00:09.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.876+0000 7febfe59c640 1 --2- 172.21.15.52:0/1177267507 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5607fafa8c20 0x5607fafab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.876+0000 7febfe59c640 1 --2- 172.21.15.52:0/1177267507 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5607fafa8c20 0x5607fafab010 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=188ed15fae6d0098 server_cookie=a8d88b229dcd99db in_seq=0 out_seq=0 2024-04-02T14:00:09.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.876+0000 7febe5ffb640 10 client.17586 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:09.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.877+0000 7febe5ffb640 1 -- 172.21.15.52:0/1177267507 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5607faf00d70 con 0x5607fafa8c20 2024-04-02T14:00:09.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.877+0000 7febe5ffb640 10 client.17586 handle_command_reply: tid=0 2024-04-02T14:00:09.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.877+0000 7fec07bc9740 1 client.17586 shutdown 2024-04-02T14:00:09.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.877+0000 7fec07bc9740 1 -- 172.21.15.52:0/1177267507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf98dd0 msgr2=0x5607faf95f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:09.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.877+0000 7fec07bc9740 1 --2- 172.21.15.52:0/1177267507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf98dd0 0x5607faf95f50 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7febf40097a0 tx=0x7febf40035f0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.877+0000 7fec07bc9740 1 -- 172.21.15.52:0/1177267507 shutdown_connections 2024-04-02T14:00:09.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.877+0000 7fec07bc9740 1 --2- 172.21.15.52:0/1177267507 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5607fafa8c20 0x5607fafab010 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.877+0000 7fec07bc9740 1 --2- 172.21.15.52:0/1177267507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf98dd0 0x5607faf95f50 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.877+0000 7fec07bc9740 1 --2- 172.21.15.52:0/1177267507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607faf901f0 0x5607faf95a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.877+0000 7fec07bc9740 1 --2- 172.21.15.52:0/1177267507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5607faf8f8d0 0x5607faf98680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.877+0000 7fec07bc9740 1 -- 172.21.15.52:0/1177267507 >> 172.21.15.52:0/1177267507 conn(0x5607faf8a000 msgr2=0x5607faf8bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:09.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.877+0000 7febe5ffb640 0 client.17586 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:09.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.878+0000 7fec07bc9740 1 -- 172.21.15.52:0/1177267507 shutdown_connections 2024-04-02T14:00:09.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.878+0000 7fec07bc9740 1 -- 172.21.15.52:0/1177267507 wait complete. 2024-04-02T14:00:09.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.878+0000 7fec07bc9740 20 client.17586 trim_cache size 0 max 16384 2024-04-02T14:00:09.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.899+0000 7fec07bc9740 1 Processor -- start 2024-04-02T14:00:09.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.899+0000 7fec07bc9740 1 -- start start 2024-04-02T14:00:09.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.899+0000 7fec07bc9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf8f8d0 0x5607faf889b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.899+0000 7fec07bc9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607faf901f0 0x5607faf951c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.899+0000 7fec07bc9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5607faf98dd0 0x5607faf98140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.899+0000 7fec07bc9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fec001b1100 con 0x5607faf8f8d0 2024-04-02T14:00:09.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.899+0000 7fec07bc9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fec001b2c40 con 0x5607faf98dd0 2024-04-02T14:00:09.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.899+0000 7fec07bc9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fec001b17d0 con 0x5607faf901f0 2024-04-02T14:00:09.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.899+0000 7febfdd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf8f8d0 0x5607faf889b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.899+0000 7febfdd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf8f8d0 0x5607faf889b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34730/0 (socket says 172.21.15.52:34730) 2024-04-02T14:00:09.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.899+0000 7febfe59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607faf901f0 0x5607faf951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.899+0000 7febfdd9b640 1 -- 172.21.15.52:0/2020267298 learned_addr learned my addr 172.21.15.52:0/2020267298 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:09.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.899+0000 7febfdd9b640 1 --2- 172.21.15.52:0/2020267298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5607faf98dd0 0x5607faf98140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.900+0000 7febfdd9b640 1 -- 172.21.15.52:0/2020267298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607faf901f0 msgr2=0x5607faf951c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:09.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.900+0000 7febfdd9b640 1 --2- 172.21.15.52:0/2020267298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607faf901f0 0x5607faf951c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.900+0000 7febfdd9b640 1 -- 172.21.15.52:0/2020267298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5607faf98dd0 msgr2=0x5607faf98140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:09.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.900+0000 7febfdd9b640 1 --2- 172.21.15.52:0/2020267298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5607faf98dd0 0x5607faf98140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.900+0000 7febfdd9b640 1 -- 172.21.15.52:0/2020267298 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607faf89e30 con 0x5607faf8f8d0 2024-04-02T14:00:09.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.900+0000 7febfdd9b640 1 --2- 172.21.15.52:0/2020267298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf8f8d0 0x5607faf889b0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7febf4006010 tx=0x7febf40059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8858d2d0ddb0cd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:09.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.900+0000 7febe7fff640 1 -- 172.21.15.52:0/2020267298 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4003660 con 0x5607faf8f8d0 2024-04-02T14:00:09.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.900+0000 7febe7fff640 1 -- 172.21.15.52:0/2020267298 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febf40044a0 con 0x5607faf8f8d0 2024-04-02T14:00:09.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.900+0000 7febe7fff640 1 -- 172.21.15.52:0/2020267298 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4005040 con 0x5607faf8f8d0 2024-04-02T14:00:09.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.901+0000 7fec07bc9740 1 -- 172.21.15.52:0/2020267298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf8f8d0 msgr2=0x5607faf889b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:09.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.901+0000 7fec07bc9740 1 --2- 172.21.15.52:0/2020267298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf8f8d0 0x5607faf889b0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7febf4006010 tx=0x7febf40059b0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.901+0000 7fec07bc9740 1 -- 172.21.15.52:0/2020267298 shutdown_connections 2024-04-02T14:00:09.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.901+0000 7fec07bc9740 1 --2- 172.21.15.52:0/2020267298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5607faf98dd0 0x5607faf98140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.901+0000 7fec07bc9740 1 --2- 172.21.15.52:0/2020267298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607faf901f0 0x5607faf951c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.901+0000 7fec07bc9740 1 --2- 172.21.15.52:0/2020267298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf8f8d0 0x5607faf889b0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.901+0000 7fec07bc9740 1 -- 172.21.15.52:0/2020267298 >> 172.21.15.52:0/2020267298 conn(0x5607faf8a000 msgr2=0x5607faf90fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:09.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.901+0000 7fec07bc9740 1 -- 172.21.15.52:0/2020267298 shutdown_connections 2024-04-02T14:00:09.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.901+0000 7fec07bc9740 1 -- 172.21.15.52:0/2020267298 wait complete. 2024-04-02T14:00:09.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.901+0000 7fec07bc9740 1 -- start start 2024-04-02T14:00:09.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.901+0000 7fec07bc9740 1 Processor -- start 2024-04-02T14:00:09.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.902+0000 7fec07bc9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607faf8f8d0 0x5607fafa0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.902+0000 7fec07bc9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5607faf901f0 0x5607fafa1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.902+0000 7fec07bc9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf98dd0 0x5607faf91c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.902+0000 7fec07bc9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fec001b2570 con 0x5607faf98dd0 2024-04-02T14:00:09.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.902+0000 7fec07bc9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fec001b1ea0 con 0x5607faf901f0 2024-04-02T14:00:09.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.902+0000 7fec07bc9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fec001af0e0 con 0x5607faf8f8d0 2024-04-02T14:00:09.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.902+0000 7fec07bc9740 10 client.0 _ll_register_callbacks cb 0x7febf400fb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:09.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.902+0000 7febfdd9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607faf8f8d0 0x5607fafa0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.902+0000 7febfdd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf98dd0 0x5607faf91c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.902+0000 7febfdd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf98dd0 0x5607faf91c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34736/0 (socket says 172.21.15.52:34736) 2024-04-02T14:00:09.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.902+0000 7febfe59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5607faf901f0 0x5607fafa1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.902+0000 7febfdd9b640 1 -- 172.21.15.52:0/2821193643 learned_addr learned my addr 172.21.15.52:0/2821193643 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:09.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.902+0000 7febfdd9b640 1 -- 172.21.15.52:0/2821193643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607faf8f8d0 msgr2=0x5607fafa0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:09.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.902+0000 7febfdd9b640 1 --2- 172.21.15.52:0/2821193643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607faf8f8d0 0x5607fafa0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.902+0000 7febfdd9b640 1 -- 172.21.15.52:0/2821193643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5607faf901f0 msgr2=0x5607fafa1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:09.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.902+0000 7febfdd9b640 1 --2- 172.21.15.52:0/2821193643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5607faf901f0 0x5607fafa1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.902+0000 7febfdd9b640 1 -- 172.21.15.52:0/2821193643 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607faf92270 con 0x5607faf98dd0 2024-04-02T14:00:09.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.902+0000 7febfdd9b640 1 --2- 172.21.15.52:0/2821193643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf98dd0 0x5607faf91c80 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7febf403c040 tx=0x7febf400ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836e9a9e971a1fad server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:09.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.902+0000 7febe5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:09.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.902+0000 7febe5ffb640 1 -- 172.21.15.52:0/2821193643 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4030040 con 0x5607faf98dd0 2024-04-02T14:00:09.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.902+0000 7febe5ffb640 1 -- 172.21.15.52:0/2821193643 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9350+0+0 (secure 0 0 0) 0x7febf40301e0 con 0x5607faf98dd0 2024-04-02T14:00:09.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.902+0000 7fec07bc9740 1 -- 172.21.15.52:0/2821193643 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febf40044a0 con 0x5607faf98dd0 2024-04-02T14:00:09.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.903+0000 7febe5ffb640 1 -- 172.21.15.52:0/2821193643 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7febf40303e0 con 0x5607faf98dd0 2024-04-02T14:00:09.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.903+0000 7fec07bc9740 10 client.17592 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:09.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.903+0000 7fec07bc9740 1 -- 172.21.15.52:0/2821193643 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5607fb00e880 con 0x5607faf98dd0 2024-04-02T14:00:09.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.903+0000 7febe5ffb640 1 -- 172.21.15.52:0/2821193643 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febf40044a0 con 0x5607faf98dd0 2024-04-02T14:00:09.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.903+0000 7febe5ffb640 1 -- 172.21.15.52:0/2821193643 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7febf4030990 con 0x5607faf98dd0 2024-04-02T14:00:09.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.903+0000 7fec07bc9740 10 client.17592 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:09.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.903+0000 7fec07bc9740 10 client.17592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:00:09.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.903+0000 7fec07bc9740 20 client.17592 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:09.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.903+0000 7fec07bc9740 1 --2- 172.21.15.52:0/2821193643 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5607fb00ef30 0x5607fb011350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:09.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.903+0000 7fec07bc9740 4 client.17592 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:00:09.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.903+0000 7fec07bc9740 1 -- 172.21.15.52:0/2821193643 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5607faefb0c0 con 0x5607fb00ef30 2024-04-02T14:00:09.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.903+0000 7febfe59c640 1 --2- 172.21.15.52:0/2821193643 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5607fb00ef30 0x5607fb011350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:09.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.903+0000 7febe5ffb640 10 client.17592 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:09.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.903+0000 7febfe59c640 1 --2- 172.21.15.52:0/2821193643 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5607fb00ef30 0x5607fb011350 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e019a09a649af285 server_cookie=45260395f0d418e3 in_seq=0 out_seq=0 2024-04-02T14:00:09.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.904+0000 7febe5ffb640 1 -- 172.21.15.52:0/2821193643 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x5607faefb0c0 con 0x5607fb00ef30 2024-04-02T14:00:09.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.904+0000 7febe5ffb640 10 client.17592 handle_command_reply: tid=0 2024-04-02T14:00:09.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.904+0000 7fec07bc9740 1 client.17592 shutdown 2024-04-02T14:00:09.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.904+0000 7fec07bc9740 1 -- 172.21.15.52:0/2821193643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf98dd0 msgr2=0x5607faf91c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:09.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.904+0000 7fec07bc9740 1 --2- 172.21.15.52:0/2821193643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf98dd0 0x5607faf91c80 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7febf403c040 tx=0x7febf400ffa0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.904+0000 7fec07bc9740 1 -- 172.21.15.52:0/2821193643 shutdown_connections 2024-04-02T14:00:09.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.904+0000 7fec07bc9740 1 --2- 172.21.15.52:0/2821193643 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5607fb00ef30 0x5607fb011350 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.904+0000 7febe5ffb640 0 client.17592 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:09.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.904+0000 7fec07bc9740 1 --2- 172.21.15.52:0/2821193643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607faf98dd0 0x5607faf91c80 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.904+0000 7fec07bc9740 1 --2- 172.21.15.52:0/2821193643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5607faf901f0 0x5607fafa1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.904+0000 7fec07bc9740 1 --2- 172.21.15.52:0/2821193643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607faf8f8d0 0x5607fafa0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.904+0000 7fec07bc9740 1 -- 172.21.15.52:0/2821193643 >> 172.21.15.52:0/2821193643 conn(0x5607faf8a000 msgr2=0x5607faf8b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:09.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.904+0000 7fec07bc9740 1 -- 172.21.15.52:0/2821193643 shutdown_connections 2024-04-02T14:00:09.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.904+0000 7fec07bc9740 1 -- 172.21.15.52:0/2821193643 wait complete. 2024-04-02T14:00:09.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.904+0000 7fec07bc9740 20 client.17592 trim_cache size 0 max 16384 2024-04-02T14:00:09.920 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:00:09.920 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "scrub active (2506 inodes in the stack)", 2024-04-02T14:00:09.920 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-04-02T14:00:09.920 INFO:teuthology.orchestra.run.smithi052.stdout: "54474adf-879d-4612-bd3c-40cb8efbffb2": { 2024-04-02T14:00:09.921 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-04-02T14:00:09.921 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "54474adf-879d-4612-bd3c-40cb8efbffb2", 2024-04-02T14:00:09.921 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-04-02T14:00:09.921 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:00:09.921 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:00:09.921 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:00:09.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.905+0000 7fec04f04640 1 -- 172.21.15.52:0/2661673962 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7febe80a3d20 msgr2=0x7febe80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:09.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.905+0000 7fec04f04640 1 --2- 172.21.15.52:0/2661673962 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7febe80a3d20 0x7febe80a61e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7febf80990f0 tx=0x7febf8092120 comp rx=0 tx=0).stop 2024-04-02T14:00:09.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.905+0000 7fec04f04640 1 -- 172.21.15.52:0/2661673962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec00107ab0 msgr2=0x7fec00074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:09.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.905+0000 7fec04f04640 1 --2- 172.21.15.52:0/2661673962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec00107ab0 0x7fec00074880 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7febf800eb10 tx=0x7febf800c2d0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.905+0000 7fec04f04640 1 -- 172.21.15.52:0/2661673962 shutdown_connections 2024-04-02T14:00:09.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.905+0000 7fec04f04640 1 --2- 172.21.15.52:0/2661673962 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7febe80a3d20 0x7febe80a61e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.905+0000 7fec04f04640 1 --2- 172.21.15.52:0/2661673962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec00107ab0 0x7fec00074880 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.905+0000 7fec04f04640 1 --2- 172.21.15.52:0/2661673962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec00102460 0x7fec00074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.905+0000 7fec04f04640 1 --2- 172.21.15.52:0/2661673962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec00101aa0 0x7fec00073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:09.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.905+0000 7fec04f04640 1 -- 172.21.15.52:0/2661673962 >> 172.21.15.52:0/2661673962 conn(0x7fec000fd590 msgr2=0x7fec000fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:09.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.905+0000 7fec04f04640 1 -- 172.21.15.52:0/2661673962 shutdown_connections 2024-04-02T14:00:09.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:09.905+0000 7fec04f04640 1 -- 172.21.15.52:0/2661673962 wait complete. 2024-04-02T14:00:09.925 INFO:tasks.cephfs.filesystem:scrub status for tag:54474adf-879d-4612-bd3c-40cb8efbffb2 - {'path': '/', 'tag': '54474adf-879d-4612-bd3c-40cb8efbffb2', 'options': 'recursive,force'} 2024-04-02T14:00:09.925 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:09 smithi088 ceph-mon[36528]: pgmap v202: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-02T14:00:10.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:09 smithi189 ceph-mon[36683]: pgmap v202: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-02T14:00:10.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:10 smithi052 ceph-mon[32104]: scrub queued for path: / 2024-04-02T14:00:10.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:10 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:00:10.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:10 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T14:00:10.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:10 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T14:00:10.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:10 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:00:10.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:10 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:00:10.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:10 smithi088 ceph-mon[36528]: scrub queued for path: / 2024-04-02T14:00:10.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:10 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:00:10.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:10 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T14:00:10.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:10 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T14:00:10.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:10 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:00:10.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:10 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:00:11.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:10 smithi189 ceph-mon[36683]: scrub queued for path: / 2024-04-02T14:00:11.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:10 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:00:11.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:10 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T14:00:11.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:10 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T14:00:11.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:10 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:00:11.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:10 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:00:11.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:11 smithi052 ceph-mon[32104]: pgmap v203: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-02T14:00:11.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:11 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T14:00:11.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:11 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:00:11.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:11 smithi052 ceph-mon[32104]: osdmap e94: 12 total, 12 up, 12 in 2024-04-02T14:00:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:11 smithi088 ceph-mon[36528]: pgmap v203: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-02T14:00:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:11 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T14:00:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:11 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:00:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:11 smithi088 ceph-mon[36528]: osdmap e94: 12 total, 12 up, 12 in 2024-04-02T14:00:12.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:11 smithi189 ceph-mon[36683]: pgmap v203: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-02T14:00:12.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:11 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T14:00:12.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:11 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:00:12.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:11 smithi189 ceph-mon[36683]: osdmap e94: 12 total, 12 up, 12 in 2024-04-02T14:00:13.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:13 smithi052 ceph-mon[32104]: pgmap v205: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:00:13.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:13 smithi088 ceph-mon[36528]: pgmap v205: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:00:14.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:13 smithi189 ceph-mon[36683]: pgmap v205: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:00:15.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:15 smithi052 ceph-mon[32104]: pgmap v206: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:00:15.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:15 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T14:00:15.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:15 smithi088 ceph-mon[36528]: pgmap v206: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:00:15.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:15 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T14:00:16.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:15 smithi189 ceph-mon[36683]: pgmap v206: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:00:16.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:15 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T14:00:16.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:16 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T14:00:16.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:16 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T14:00:17.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:16 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T14:00:17.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:17 smithi052 ceph-mon[32104]: pgmap v207: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:00:17.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:17 smithi088 ceph-mon[36528]: pgmap v207: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:00:18.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:17 smithi189 ceph-mon[36683]: pgmap v207: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:00:19.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:19 smithi052 ceph-mon[32104]: pgmap v208: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:00:19.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:19 smithi088 ceph-mon[36528]: pgmap v208: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:00:20.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:19 smithi189 ceph-mon[36683]: pgmap v208: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:00:21.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:21 smithi189 ceph-mon[36683]: pgmap v209: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:00:21.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:21 smithi189 ceph-mon[36683]: osdmap e95: 12 total, 12 up, 12 in 2024-04-02T14:00:21.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:21 smithi052 ceph-mon[32104]: pgmap v209: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:00:21.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:21 smithi052 ceph-mon[32104]: osdmap e95: 12 total, 12 up, 12 in 2024-04-02T14:00:21.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:21 smithi088 ceph-mon[36528]: pgmap v209: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:00:21.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:21 smithi088 ceph-mon[36528]: osdmap e95: 12 total, 12 up, 12 in 2024-04-02T14:00:23.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:23 smithi189 ceph-mon[36683]: pgmap v211: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:00:23.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:23 smithi052 ceph-mon[32104]: pgmap v211: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:00:23.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:23 smithi088 ceph-mon[36528]: pgmap v211: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:00:25.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:25 smithi189 ceph-mon[36683]: pgmap v212: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:00:25.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:25 smithi052 ceph-mon[32104]: pgmap v212: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:00:25.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:25 smithi088 ceph-mon[36528]: pgmap v212: 129 pgs: 129 active+clean; 73 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:00:27.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:27 smithi189 ceph-mon[36683]: pgmap v213: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T14:00:27.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:27 smithi052 ceph-mon[32104]: pgmap v213: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T14:00:27.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:27 smithi088 ceph-mon[36528]: pgmap v213: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T14:00:29.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:29 smithi189 ceph-mon[36683]: pgmap v214: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T14:00:29.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:29 smithi052 ceph-mon[32104]: pgmap v214: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T14:00:29.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:29 smithi088 ceph-mon[36528]: pgmap v214: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T14:00:31.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:31 smithi189 ceph-mon[36683]: pgmap v215: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T14:00:31.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:31 smithi052 ceph-mon[32104]: pgmap v215: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T14:00:31.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:31 smithi088 ceph-mon[36528]: pgmap v215: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T14:00:33.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:33 smithi189 ceph-mon[36683]: pgmap v216: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T14:00:33.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:33 smithi052 ceph-mon[32104]: pgmap v216: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T14:00:33.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:33 smithi088 ceph-mon[36528]: pgmap v216: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T14:00:35.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:35 smithi189 ceph-mon[36683]: pgmap v217: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:00:35.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:35 smithi052 ceph-mon[32104]: pgmap v217: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:00:35.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:35 smithi088 ceph-mon[36528]: pgmap v217: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:00:37.722 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:37 smithi088 ceph-mon[36528]: pgmap v218: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:00:37.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:37 smithi189 ceph-mon[36683]: pgmap v218: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:00:37.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:37 smithi052 ceph-mon[32104]: pgmap v218: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:00:39.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:39 smithi189 ceph-mon[36683]: pgmap v219: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:00:39.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:39 smithi052 ceph-mon[32104]: pgmap v219: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:00:39.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:39 smithi088 ceph-mon[36528]: pgmap v219: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:00:39.925 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:00:39.926 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:00:40.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.040+0000 7ff1b6bf6640 1 -- 172.21.15.52:0/361510940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1b01009b0 msgr2=0x7ff1b0100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.040+0000 7ff1b6bf6640 1 --2- 172.21.15.52:0/361510940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1b01009b0 0x7ff1b0100e30 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009930 tx=0x7ff19c02bc30 comp rx=0 tx=0).stop 2024-04-02T14:00:40.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.040+0000 7ff1b6bf6640 1 -- 172.21.15.52:0/361510940 shutdown_connections 2024-04-02T14:00:40.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.040+0000 7ff1b6bf6640 1 --2- 172.21.15.52:0/361510940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1b0101370 0x7ff1b010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.040+0000 7ff1b6bf6640 1 --2- 172.21.15.52:0/361510940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1b01009b0 0x7ff1b0100e30 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.040+0000 7ff1b6bf6640 1 --2- 172.21.15.52:0/361510940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1b01069c0 0x7ff1b0106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.040+0000 7ff1b6bf6640 1 -- 172.21.15.52:0/361510940 >> 172.21.15.52:0/361510940 conn(0x7ff1b0076d80 msgr2=0x7ff1b0077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.040+0000 7ff1b6bf6640 1 -- 172.21.15.52:0/361510940 shutdown_connections 2024-04-02T14:00:40.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.040+0000 7ff1b6bf6640 1 -- 172.21.15.52:0/361510940 wait complete. 2024-04-02T14:00:40.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.041+0000 7ff1b6bf6640 1 Processor -- start 2024-04-02T14:00:40.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.041+0000 7ff1b6bf6640 1 -- start start 2024-04-02T14:00:40.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.041+0000 7ff1b6bf6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1b01009b0 0x7ff1b00fe0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.041+0000 7ff1b6bf6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1b0101370 0x7ff1b00fe5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.041+0000 7ff1b6bf6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1b01069c0 0x7ff1b00feb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.041+0000 7ff1b6bf6640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b0110dd0 con 0x7ff1b01069c0 2024-04-02T14:00:40.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.041+0000 7ff1b6bf6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b0110c50 con 0x7ff1b01009b0 2024-04-02T14:00:40.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.041+0000 7ff1b6bf6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b0110f50 con 0x7ff1b0101370 2024-04-02T14:00:40.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.041+0000 7ff1b5193640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1b01069c0 0x7ff1b00feb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.041+0000 7ff1b5193640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1b01069c0 0x7ff1b00feb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33990/0 (socket says 172.21.15.52:33990) 2024-04-02T14:00:40.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.041+0000 7ff1b5193640 1 -- 172.21.15.52:0/3735925991 learned_addr learned my addr 172.21.15.52:0/3735925991 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:40.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.041+0000 7ff1b4992640 1 --2- 172.21.15.52:0/3735925991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1b01009b0 0x7ff1b00fe0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.041+0000 7ff1affff640 1 --2- 172.21.15.52:0/3735925991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1b0101370 0x7ff1b00fe5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.041+0000 7ff1b5193640 1 -- 172.21.15.52:0/3735925991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1b0101370 msgr2=0x7ff1b00fe5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.041+0000 7ff1b5193640 1 --2- 172.21.15.52:0/3735925991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1b0101370 0x7ff1b00fe5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.042+0000 7ff1b5193640 1 -- 172.21.15.52:0/3735925991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1b01009b0 msgr2=0x7ff1b00fe0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.042+0000 7ff1b5193640 1 --2- 172.21.15.52:0/3735925991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1b01009b0 0x7ff1b00fe0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.042+0000 7ff1b5193640 1 -- 172.21.15.52:0/3735925991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b01a6910 con 0x7ff1b01069c0 2024-04-02T14:00:40.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.042+0000 7ff1b5193640 1 --2- 172.21.15.52:0/3735925991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1b01069c0 0x7ff1b00feb30 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800b970 tx=0x7ff1a800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.042+0000 7ff1adffb640 1 -- 172.21.15.52:0/3735925991 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8019070 con 0x7ff1b01069c0 2024-04-02T14:00:40.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.042+0000 7ff1adffb640 1 -- 172.21.15.52:0/3735925991 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1a800cb40 con 0x7ff1b01069c0 2024-04-02T14:00:40.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.042+0000 7ff1b6bf6640 1 -- 172.21.15.52:0/3735925991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b01a6c00 con 0x7ff1b01069c0 2024-04-02T14:00:40.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.042+0000 7ff1adffb640 1 -- 172.21.15.52:0/3735925991 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8007650 con 0x7ff1b01069c0 2024-04-02T14:00:40.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.042+0000 7ff1b6bf6640 1 -- 172.21.15.52:0/3735925991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b01a7120 con 0x7ff1b01069c0 2024-04-02T14:00:40.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.045+0000 7ff1adffb640 1 -- 172.21.15.52:0/3735925991 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff1a801c070 con 0x7ff1b01069c0 2024-04-02T14:00:40.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.045+0000 7ff1b97db740 1 Processor -- start 2024-04-02T14:00:40.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.045+0000 7ff1b97db740 1 -- start start 2024-04-02T14:00:40.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.045+0000 7ff1b97db740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b26c47f8d0 0x55b26c47fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.045+0000 7ff1adffb640 1 --2- 172.21.15.52:0/3735925991 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff1a00a3da0 0x7ff1a00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.045+0000 7ff1b97db740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c4801f0 0x55b26c488890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.045+0000 7ff1b97db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b26c488dd0 0x55b26c48b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.045+0000 7ff1b97db740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b26c3e3830 con 0x55b26c4801f0 2024-04-02T14:00:40.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.045+0000 7ff1b97db740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b26c38b820 con 0x55b26c47f8d0 2024-04-02T14:00:40.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.045+0000 7ff1b97db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b26c3df7e0 con 0x55b26c488dd0 2024-04-02T14:00:40.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.046+0000 7ff1b4992640 1 --2- 172.21.15.52:0/3735925991 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff1a00a3da0 0x7ff1a00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.046+0000 7ff1affff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c4801f0 0x55b26c488890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.046+0000 7ff1adffb640 1 -- 172.21.15.52:0/3735925991 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7ff1a80cff80 con 0x7ff1b01069c0 2024-04-02T14:00:40.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.046+0000 7ff1b5193640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b26c488dd0 0x55b26c48b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.046+0000 7ff1affff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c4801f0 0x55b26c488890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34004/0 (socket says 172.21.15.52:34004) 2024-04-02T14:00:40.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.046+0000 7ff1affff640 1 -- 172.21.15.52:0/1368901580 learned_addr learned my addr 172.21.15.52:0/1368901580 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:40.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.046+0000 7ff1affff640 1 --2- 172.21.15.52:0/1368901580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b26c47f8d0 0x55b26c47fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.046+0000 7ff1b5193640 1 -- 172.21.15.52:0/1368901580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b26c47f8d0 msgr2=0x55b26c47fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.046+0000 7ff1b5193640 1 --2- 172.21.15.52:0/1368901580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b26c47f8d0 0x55b26c47fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.046+0000 7ff1b5193640 1 -- 172.21.15.52:0/1368901580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c4801f0 msgr2=0x55b26c488890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.046+0000 7ff1b5193640 1 --2- 172.21.15.52:0/1368901580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c4801f0 0x55b26c488890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.046+0000 7ff1b5193640 1 -- 172.21.15.52:0/1368901580 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b26c1c17d0 con 0x55b26c488dd0 2024-04-02T14:00:40.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.046+0000 7ff1b4992640 1 --2- 172.21.15.52:0/3735925991 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff1a00a3da0 0x7ff1a00a6260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff1a40058c0 tx=0x7ff1a400a2d0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.046+0000 7ff1b5193640 1 --2- 172.21.15.52:0/1368901580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b26c488dd0 0x55b26c48b1c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80cd450 tx=0x7ff1a8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16143803ba829575 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.046+0000 7ff19affd640 1 -- 172.21.15.52:0/1368901580 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a80bae60 con 0x55b26c488dd0 2024-04-02T14:00:40.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.046+0000 7ff19affd640 1 -- 172.21.15.52:0/1368901580 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1a809bd50 con 0x55b26c488dd0 2024-04-02T14:00:40.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.046+0000 7ff19affd640 1 -- 172.21.15.52:0/1368901580 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a80983c0 con 0x55b26c488dd0 2024-04-02T14:00:40.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.047+0000 7ff1b97db740 1 -- 172.21.15.52:0/1368901580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b26c488dd0 msgr2=0x55b26c48b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.047+0000 7ff1b97db740 1 --2- 172.21.15.52:0/1368901580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b26c488dd0 0x55b26c48b1c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80cd450 tx=0x7ff1a8004a40 comp rx=0 tx=0).stop 2024-04-02T14:00:40.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.047+0000 7ff1b97db740 1 -- 172.21.15.52:0/1368901580 shutdown_connections 2024-04-02T14:00:40.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.047+0000 7ff1b97db740 1 --2- 172.21.15.52:0/1368901580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b26c488dd0 0x55b26c48b1c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.047+0000 7ff1b97db740 1 --2- 172.21.15.52:0/1368901580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c4801f0 0x55b26c488890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.047+0000 7ff1b97db740 1 --2- 172.21.15.52:0/1368901580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b26c47f8d0 0x55b26c47fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.047+0000 7ff1b97db740 1 -- 172.21.15.52:0/1368901580 >> 172.21.15.52:0/1368901580 conn(0x55b26c47a000 msgr2=0x55b26c47a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.047+0000 7ff1b97db740 1 -- 172.21.15.52:0/1368901580 shutdown_connections 2024-04-02T14:00:40.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.047+0000 7ff1b97db740 1 -- 172.21.15.52:0/1368901580 wait complete. 2024-04-02T14:00:40.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.047+0000 7ff1b97db740 1 -- start start 2024-04-02T14:00:40.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.048+0000 7ff1b97db740 1 Processor -- start 2024-04-02T14:00:40.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.048+0000 7ff1b97db740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b26c47f8d0 0x55b26c4886b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.048+0000 7ff1b97db740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c4801f0 0x55b26c485a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.048+0000 7ff1b97db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b26c488dd0 0x55b26c485f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.048+0000 7ff1b97db740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b26c48bbc0 con 0x55b26c4801f0 2024-04-02T14:00:40.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.048+0000 7ff1b97db740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b26c39ed30 con 0x55b26c47f8d0 2024-04-02T14:00:40.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.048+0000 7ff1b97db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b26c48bd40 con 0x55b26c488dd0 2024-04-02T14:00:40.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.048+0000 7ff1b97db740 10 client.0 _ll_register_callbacks cb 0x55b26c39e720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:40.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.048+0000 7ff1affff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c4801f0 0x55b26c485a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.048+0000 7ff1affff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c4801f0 0x55b26c485a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34018/0 (socket says 172.21.15.52:34018) 2024-04-02T14:00:40.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.048+0000 7ff1affff640 1 -- 172.21.15.52:0/3931431700 learned_addr learned my addr 172.21.15.52:0/3931431700 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:40.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.048+0000 7ff1b4992640 1 --2- 172.21.15.52:0/3931431700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b26c47f8d0 0x55b26c4886b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.048+0000 7ff1b5193640 1 --2- 172.21.15.52:0/3931431700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b26c488dd0 0x55b26c485f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.048+0000 7ff1affff640 1 -- 172.21.15.52:0/3931431700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b26c488dd0 msgr2=0x55b26c485f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.048+0000 7ff1affff640 1 --2- 172.21.15.52:0/3931431700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b26c488dd0 0x55b26c485f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.048+0000 7ff1affff640 1 -- 172.21.15.52:0/3931431700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b26c47f8d0 msgr2=0x55b26c4886b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.048+0000 7ff1affff640 1 --2- 172.21.15.52:0/3931431700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b26c47f8d0 0x55b26c4886b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.048+0000 7ff1affff640 1 -- 172.21.15.52:0/3931431700 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b26c0f73f0 con 0x55b26c4801f0 2024-04-02T14:00:40.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.048+0000 7ff1b5193640 1 --2- 172.21.15.52:0/3931431700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b26c488dd0 0x55b26c485f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:00:40.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.048+0000 7ff1b4992640 1 --2- 172.21.15.52:0/3931431700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b26c47f8d0 0x55b26c4886b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:00:40.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.049+0000 7ff1affff640 1 --2- 172.21.15.52:0/3931431700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c4801f0 0x55b26c485a40 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7ff19c02cfd0 tx=0x7ff19c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5050673c5d49b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.049+0000 7ff18ffff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:40.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.049+0000 7ff18ffff640 1 -- 172.21.15.52:0/3931431700 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c0048c0 con 0x55b26c4801f0 2024-04-02T14:00:40.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.049+0000 7ff18ffff640 1 -- 172.21.15.52:0/3931431700 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7ff19c004a60 con 0x55b26c4801f0 2024-04-02T14:00:40.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.049+0000 7ff1b97db740 1 -- 172.21.15.52:0/3931431700 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1a809bd50 con 0x55b26c4801f0 2024-04-02T14:00:40.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.049+0000 7ff18ffff640 1 -- 172.21.15.52:0/3931431700 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff19c004c60 con 0x55b26c4801f0 2024-04-02T14:00:40.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.049+0000 7ff1b97db740 10 client.17604 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:40.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.049+0000 7ff1b97db740 1 -- 172.21.15.52:0/3931431700 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b26c0b70f0 con 0x55b26c4801f0 2024-04-02T14:00:40.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.049+0000 7ff18ffff640 1 -- 172.21.15.52:0/3931431700 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1a809bd50 con 0x55b26c4801f0 2024-04-02T14:00:40.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.049+0000 7ff18ffff640 1 -- 172.21.15.52:0/3931431700 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff19c003960 con 0x55b26c4801f0 2024-04-02T14:00:40.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.049+0000 7ff1b97db740 10 client.17604 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:40.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.049+0000 7ff1b97db740 10 client.17604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:00:40.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.049+0000 7ff1b97db740 20 client.17604 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:40.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.049+0000 7ff1b97db740 1 --2- 172.21.15.52:0/3931431700 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55b26c498e10 0x55b26c49b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.049+0000 7ff1b97db740 4 client.17604 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:00:40.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.049+0000 7ff1b97db740 1 -- 172.21.15.52:0/3931431700 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b26c3f0d70 con 0x55b26c498e10 2024-04-02T14:00:40.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.050+0000 7ff1b4992640 1 --2- 172.21.15.52:0/3931431700 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55b26c498e10 0x55b26c49b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.050+0000 7ff18ffff640 10 client.17604 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:40.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.050+0000 7ff1b4992640 1 --2- 172.21.15.52:0/3931431700 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55b26c498e10 0x55b26c49b220 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2c7754c7eb1764f server_cookie=d34a210b8259d90b in_seq=0 out_seq=0 2024-04-02T14:00:40.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.051+0000 7ff18ffff640 1 -- 172.21.15.52:0/3931431700 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b26c3f0d70 con 0x55b26c498e10 2024-04-02T14:00:40.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.051+0000 7ff18ffff640 10 client.17604 handle_command_reply: tid=0 2024-04-02T14:00:40.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.051+0000 7ff1b97db740 1 client.17604 shutdown 2024-04-02T14:00:40.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.051+0000 7ff1b97db740 1 -- 172.21.15.52:0/3931431700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c4801f0 msgr2=0x55b26c485a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.051+0000 7ff1b97db740 1 --2- 172.21.15.52:0/3931431700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c4801f0 0x55b26c485a40 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7ff19c02cfd0 tx=0x7ff19c004270 comp rx=0 tx=0).stop 2024-04-02T14:00:40.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.051+0000 7ff1b97db740 1 -- 172.21.15.52:0/3931431700 shutdown_connections 2024-04-02T14:00:40.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.051+0000 7ff1b97db740 1 --2- 172.21.15.52:0/3931431700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b26c488dd0 0x55b26c485f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.051+0000 7ff1b97db740 1 --2- 172.21.15.52:0/3931431700 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55b26c498e10 0x55b26c49b220 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.051+0000 7ff1b97db740 1 --2- 172.21.15.52:0/3931431700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c4801f0 0x55b26c485a40 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.051+0000 7ff1b97db740 1 --2- 172.21.15.52:0/3931431700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b26c47f8d0 0x55b26c4886b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.051+0000 7ff18ffff640 0 client.17604 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:40.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.051+0000 7ff1b97db740 1 -- 172.21.15.52:0/3931431700 >> 172.21.15.52:0/3931431700 conn(0x55b26c47a000 msgr2=0x55b26c47bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.051+0000 7ff1b97db740 1 -- 172.21.15.52:0/3931431700 shutdown_connections 2024-04-02T14:00:40.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.051+0000 7ff1b97db740 1 -- 172.21.15.52:0/3931431700 wait complete. 2024-04-02T14:00:40.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.051+0000 7ff1b97db740 20 client.17604 trim_cache size 0 max 16384 2024-04-02T14:00:40.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.069+0000 7ff1b97db740 1 Processor -- start 2024-04-02T14:00:40.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.069+0000 7ff1b97db740 1 -- start start 2024-04-02T14:00:40.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.069+0000 7ff1b97db740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b26c47f8d0 0x55b26c494ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.069+0000 7ff1b97db740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c4801f0 0x55b26c480f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.069+0000 7ff1b97db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b26c488dd0 0x55b26c488170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.069+0000 7ff1b97db740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b01b1100 con 0x55b26c4801f0 2024-04-02T14:00:40.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.069+0000 7ff1b97db740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b01b2c40 con 0x55b26c47f8d0 2024-04-02T14:00:40.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.069+0000 7ff1b97db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b01b17d0 con 0x55b26c488dd0 2024-04-02T14:00:40.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.069+0000 7ff1affff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c4801f0 0x55b26c480f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.069+0000 7ff1affff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c4801f0 0x55b26c480f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34028/0 (socket says 172.21.15.52:34028) 2024-04-02T14:00:40.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.069+0000 7ff1affff640 1 -- 172.21.15.52:0/117271234 learned_addr learned my addr 172.21.15.52:0/117271234 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:40.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.069+0000 7ff1affff640 1 -- 172.21.15.52:0/117271234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b26c488dd0 msgr2=0x55b26c488170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.069+0000 7ff1b4992640 1 --2- 172.21.15.52:0/117271234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b26c47f8d0 0x55b26c494ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.069+0000 7ff1b5193640 1 --2- 172.21.15.52:0/117271234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b26c488dd0 0x55b26c488170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.069+0000 7ff1affff640 1 --2- 172.21.15.52:0/117271234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b26c488dd0 0x55b26c488170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.069+0000 7ff1affff640 1 -- 172.21.15.52:0/117271234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b26c47f8d0 msgr2=0x55b26c494ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.069+0000 7ff1affff640 1 --2- 172.21.15.52:0/117271234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b26c47f8d0 0x55b26c494ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.069+0000 7ff1affff640 1 -- 172.21.15.52:0/117271234 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b26c479e30 con 0x55b26c4801f0 2024-04-02T14:00:40.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.069+0000 7ff1b4992640 1 --2- 172.21.15.52:0/117271234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b26c47f8d0 0x55b26c494ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:00:40.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.069+0000 7ff1b5193640 1 --2- 172.21.15.52:0/117271234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b26c488dd0 0x55b26c488170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:00:40.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.070+0000 7ff1affff640 1 --2- 172.21.15.52:0/117271234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c4801f0 0x55b26c480f40 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7ff19c02c170 tx=0x7ff19c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9a90fa5ae1dfd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.070+0000 7ff19a7fc640 1 -- 172.21.15.52:0/117271234 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c0048c0 con 0x55b26c4801f0 2024-04-02T14:00:40.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.070+0000 7ff19a7fc640 1 -- 172.21.15.52:0/117271234 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff19c05b070 con 0x55b26c4801f0 2024-04-02T14:00:40.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.070+0000 7ff19a7fc640 1 -- 172.21.15.52:0/117271234 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c005500 con 0x55b26c4801f0 2024-04-02T14:00:40.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.070+0000 7ff1b97db740 1 -- 172.21.15.52:0/117271234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c4801f0 msgr2=0x55b26c480f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.070+0000 7ff1b97db740 1 --2- 172.21.15.52:0/117271234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c4801f0 0x55b26c480f40 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7ff19c02c170 tx=0x7ff19c000bc0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.070+0000 7ff1b97db740 1 -- 172.21.15.52:0/117271234 shutdown_connections 2024-04-02T14:00:40.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.070+0000 7ff1b97db740 1 --2- 172.21.15.52:0/117271234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b26c488dd0 0x55b26c488170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.070+0000 7ff1b97db740 1 --2- 172.21.15.52:0/117271234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c4801f0 0x55b26c480f40 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.070+0000 7ff1b97db740 1 --2- 172.21.15.52:0/117271234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b26c47f8d0 0x55b26c494ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.070+0000 7ff1b97db740 1 -- 172.21.15.52:0/117271234 >> 172.21.15.52:0/117271234 conn(0x55b26c47a000 msgr2=0x55b26c480830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.070+0000 7ff1b97db740 1 -- 172.21.15.52:0/117271234 shutdown_connections 2024-04-02T14:00:40.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.070+0000 7ff1b97db740 1 -- 172.21.15.52:0/117271234 wait complete. 2024-04-02T14:00:40.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.071+0000 7ff1b97db740 1 -- start start 2024-04-02T14:00:40.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.071+0000 7ff1b97db740 1 Processor -- start 2024-04-02T14:00:40.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.071+0000 7ff1b97db740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c47f8d0 0x55b26c490fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.071+0000 7ff1b97db740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b26c4801f0 0x55b26c4914e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.071+0000 7ff1b97db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b26c488dd0 0x55b26c48c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.071+0000 7ff1b97db740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b01b2570 con 0x55b26c47f8d0 2024-04-02T14:00:40.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.071+0000 7ff1b97db740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b01b1ea0 con 0x55b26c4801f0 2024-04-02T14:00:40.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.071+0000 7ff1b97db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b01af100 con 0x55b26c488dd0 2024-04-02T14:00:40.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.071+0000 7ff1b97db740 10 client.0 _ll_register_callbacks cb 0x7ff19c02cb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:40.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.071+0000 7ff1b4992640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c47f8d0 0x55b26c490fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.071+0000 7ff1b4992640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c47f8d0 0x55b26c490fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34034/0 (socket says 172.21.15.52:34034) 2024-04-02T14:00:40.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.071+0000 7ff1b4992640 1 -- 172.21.15.52:0/1563174713 learned_addr learned my addr 172.21.15.52:0/1563174713 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:40.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.071+0000 7ff1b5193640 1 --2- 172.21.15.52:0/1563174713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b26c488dd0 0x55b26c48c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.071+0000 7ff1affff640 1 --2- 172.21.15.52:0/1563174713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b26c4801f0 0x55b26c4914e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.072+0000 7ff1b4992640 1 -- 172.21.15.52:0/1563174713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b26c488dd0 msgr2=0x55b26c48c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.072+0000 7ff1b4992640 1 --2- 172.21.15.52:0/1563174713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b26c488dd0 0x55b26c48c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.072+0000 7ff1b4992640 1 -- 172.21.15.52:0/1563174713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b26c4801f0 msgr2=0x55b26c4914e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.072+0000 7ff1b4992640 1 --2- 172.21.15.52:0/1563174713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b26c4801f0 0x55b26c4914e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.072+0000 7ff1b4992640 1 -- 172.21.15.52:0/1563174713 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b26c48cd90 con 0x55b26c47f8d0 2024-04-02T14:00:40.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.072+0000 7ff1b4992640 1 --2- 172.21.15.52:0/1563174713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c47f8d0 0x55b26c490fa0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x55b26c49ba90 tx=0x7ff1a4012070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddcaf9edfa947cca server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.072+0000 7ff18ffff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:40.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.072+0000 7ff18ffff640 1 -- 172.21.15.52:0/1563174713 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4029030 con 0x55b26c47f8d0 2024-04-02T14:00:40.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.072+0000 7ff18ffff640 1 -- 172.21.15.52:0/1563174713 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7ff1a4025b50 con 0x55b26c47f8d0 2024-04-02T14:00:40.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.072+0000 7ff1b97db740 1 -- 172.21.15.52:0/1563174713 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff19c05b070 con 0x55b26c47f8d0 2024-04-02T14:00:40.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.072+0000 7ff18ffff640 1 -- 172.21.15.52:0/1563174713 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff1a4035050 con 0x55b26c47f8d0 2024-04-02T14:00:40.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.072+0000 7ff1b97db740 10 client.17610 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:40.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.072+0000 7ff1b97db740 1 -- 172.21.15.52:0/1563174713 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b26c4fef70 con 0x55b26c47f8d0 2024-04-02T14:00:40.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.072+0000 7ff18ffff640 1 -- 172.21.15.52:0/1563174713 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff19c05b070 con 0x55b26c47f8d0 2024-04-02T14:00:40.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.072+0000 7ff18ffff640 1 -- 172.21.15.52:0/1563174713 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ff1a40047a0 con 0x55b26c47f8d0 2024-04-02T14:00:40.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.073+0000 7ff1b97db740 10 client.17610 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:40.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.073+0000 7ff1b97db740 10 client.17610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:00:40.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.073+0000 7ff1b97db740 20 client.17610 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:40.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.073+0000 7ff1b97db740 1 --2- 172.21.15.52:0/1563174713 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55b26c4ff640 0x55b26c501a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.073+0000 7ff1b97db740 4 client.17610 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:00:40.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.073+0000 7ff1b97db740 1 -- 172.21.15.52:0/1563174713 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b26c3eb0c0 con 0x55b26c4ff640 2024-04-02T14:00:40.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.073+0000 7ff1affff640 1 --2- 172.21.15.52:0/1563174713 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55b26c4ff640 0x55b26c501a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.073+0000 7ff1affff640 1 --2- 172.21.15.52:0/1563174713 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55b26c4ff640 0x55b26c501a30 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf7ca843c426d4fa server_cookie=b00188b8a96ab08f in_seq=0 out_seq=0 2024-04-02T14:00:40.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.073+0000 7ff18ffff640 10 client.17610 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:40.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.073+0000 7ff18ffff640 1 -- 172.21.15.52:0/1563174713 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b26c3eb0c0 con 0x55b26c4ff640 2024-04-02T14:00:40.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.073+0000 7ff18ffff640 10 client.17610 handle_command_reply: tid=0 2024-04-02T14:00:40.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.073+0000 7ff1b97db740 1 client.17610 shutdown 2024-04-02T14:00:40.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.073+0000 7ff1b97db740 1 -- 172.21.15.52:0/1563174713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c47f8d0 msgr2=0x55b26c490fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.073+0000 7ff1b97db740 1 --2- 172.21.15.52:0/1563174713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c47f8d0 0x55b26c490fa0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x55b26c49ba90 tx=0x7ff1a4012070 comp rx=0 tx=0).stop 2024-04-02T14:00:40.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.074+0000 7ff1b97db740 1 -- 172.21.15.52:0/1563174713 shutdown_connections 2024-04-02T14:00:40.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.074+0000 7ff1b97db740 1 --2- 172.21.15.52:0/1563174713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b26c488dd0 0x55b26c48c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.074+0000 7ff1b97db740 1 --2- 172.21.15.52:0/1563174713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b26c4801f0 0x55b26c4914e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.074+0000 7ff1b97db740 1 --2- 172.21.15.52:0/1563174713 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55b26c4ff640 0x55b26c501a30 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.074+0000 7ff1b97db740 1 --2- 172.21.15.52:0/1563174713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26c47f8d0 0x55b26c490fa0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.074+0000 7ff18ffff640 0 client.17610 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:40.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.074+0000 7ff1b97db740 1 -- 172.21.15.52:0/1563174713 >> 172.21.15.52:0/1563174713 conn(0x55b26c47a000 msgr2=0x55b26c481060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.074+0000 7ff1b97db740 1 -- 172.21.15.52:0/1563174713 shutdown_connections 2024-04-02T14:00:40.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.074+0000 7ff1b97db740 1 -- 172.21.15.52:0/1563174713 wait complete. 2024-04-02T14:00:40.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.074+0000 7ff1b97db740 20 client.17610 trim_cache size 0 max 16384 2024-04-02T14:00:40.088 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:00:40.088 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-04-02T14:00:40.088 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-04-02T14:00:40.088 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:00:40.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.074+0000 7ff1b6bf6640 1 -- 172.21.15.52:0/3735925991 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff1a00a3da0 msgr2=0x7ff1a00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.074+0000 7ff1b6bf6640 1 --2- 172.21.15.52:0/3735925991 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff1a00a3da0 0x7ff1a00a6260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff1a40058c0 tx=0x7ff1a400a2d0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.074+0000 7ff1b6bf6640 1 -- 172.21.15.52:0/3735925991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1b01069c0 msgr2=0x7ff1b00feb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.074+0000 7ff1b6bf6640 1 --2- 172.21.15.52:0/3735925991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1b01069c0 0x7ff1b00feb30 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800b970 tx=0x7ff1a800be30 comp rx=0 tx=0).stop 2024-04-02T14:00:40.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.074+0000 7ff1b6bf6640 1 -- 172.21.15.52:0/3735925991 shutdown_connections 2024-04-02T14:00:40.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.074+0000 7ff1b6bf6640 1 --2- 172.21.15.52:0/3735925991 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff1a00a3da0 0x7ff1a00a6260 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.074+0000 7ff1b6bf6640 1 --2- 172.21.15.52:0/3735925991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1b01069c0 0x7ff1b00feb30 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.074+0000 7ff1b6bf6640 1 --2- 172.21.15.52:0/3735925991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1b0101370 0x7ff1b00fe5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.074+0000 7ff1b6bf6640 1 --2- 172.21.15.52:0/3735925991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1b01009b0 0x7ff1b00fe0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.074+0000 7ff1b6bf6640 1 -- 172.21.15.52:0/3735925991 >> 172.21.15.52:0/3735925991 conn(0x7ff1b0076d80 msgr2=0x7ff1b006a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.074+0000 7ff1b6bf6640 1 -- 172.21.15.52:0/3735925991 shutdown_connections 2024-04-02T14:00:40.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.075+0000 7ff1b6bf6640 1 -- 172.21.15.52:0/3735925991 wait complete. 2024-04-02T14:00:40.091 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T14:00:40.092 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:00:40.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.191+0000 7fa4317f6640 1 -- 172.21.15.52:0/462616347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa42c1036b0 msgr2=0x7fa42c10bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.191+0000 7fa4317f6640 1 --2- 172.21.15.52:0/462616347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa42c1036b0 0x7fa42c10bb90 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fa420009900 tx=0x7fa42002bc30 comp rx=0 tx=0).stop 2024-04-02T14:00:40.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.191+0000 7fa4317f6640 1 -- 172.21.15.52:0/462616347 shutdown_connections 2024-04-02T14:00:40.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.191+0000 7fa4317f6640 1 --2- 172.21.15.52:0/462616347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa42c10c160 0x7fa42c10e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.191+0000 7fa4317f6640 1 --2- 172.21.15.52:0/462616347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa42c1036b0 0x7fa42c10bb90 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.191+0000 7fa4317f6640 1 --2- 172.21.15.52:0/462616347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa42c102d90 0x7fa42c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.191+0000 7fa4317f6640 1 -- 172.21.15.52:0/462616347 >> 172.21.15.52:0/462616347 conn(0x7fa42c077b00 msgr2=0x7fa42c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.192+0000 7fa4317f6640 1 -- 172.21.15.52:0/462616347 shutdown_connections 2024-04-02T14:00:40.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.192+0000 7fa4317f6640 1 -- 172.21.15.52:0/462616347 wait complete. 2024-04-02T14:00:40.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.192+0000 7fa4317f6640 1 Processor -- start 2024-04-02T14:00:40.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.192+0000 7fa4317f6640 1 -- start start 2024-04-02T14:00:40.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.192+0000 7fa4317f6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa42c102d90 0x7fa42c1009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.192+0000 7fa4317f6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa42c1036b0 0x7fa42c100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.192+0000 7fa4317f6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa42c10c160 0x7fa42c101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.192+0000 7fa4317f6640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c111f40 con 0x7fa42c1036b0 2024-04-02T14:00:40.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.192+0000 7fa4317f6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c111dc0 con 0x7fa42c10c160 2024-04-02T14:00:40.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.192+0000 7fa4317f6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c1120c0 con 0x7fa42c102d90 2024-04-02T14:00:40.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.193+0000 7fa42a7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa42c1036b0 0x7fa42c100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.193+0000 7fa42a7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa42c1036b0 0x7fa42c100f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34054/0 (socket says 172.21.15.52:34054) 2024-04-02T14:00:40.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.193+0000 7fa42a7fc640 1 -- 172.21.15.52:0/369407938 learned_addr learned my addr 172.21.15.52:0/369407938 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:40.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.193+0000 7fa42affd640 1 --2- 172.21.15.52:0/369407938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa42c102d90 0x7fa42c1009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.193+0000 7fa42b7fe640 1 --2- 172.21.15.52:0/369407938 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa42c10c160 0x7fa42c101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.193+0000 7fa42a7fc640 1 -- 172.21.15.52:0/369407938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa42c102d90 msgr2=0x7fa42c1009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.193+0000 7fa42a7fc640 1 --2- 172.21.15.52:0/369407938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa42c102d90 0x7fa42c1009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.193+0000 7fa42a7fc640 1 -- 172.21.15.52:0/369407938 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa42c10c160 msgr2=0x7fa42c101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.193+0000 7fa42a7fc640 1 --2- 172.21.15.52:0/369407938 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa42c10c160 0x7fa42c101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.193+0000 7fa42a7fc640 1 -- 172.21.15.52:0/369407938 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42c1a68e0 con 0x7fa42c1036b0 2024-04-02T14:00:40.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.193+0000 7fa42b7fe640 1 --2- 172.21.15.52:0/369407938 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa42c10c160 0x7fa42c101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:00:40.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.193+0000 7fa42a7fc640 1 --2- 172.21.15.52:0/369407938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa42c1036b0 0x7fa42c100f20 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fa420006010 tx=0x7fa42002cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.193+0000 7fa413fff640 1 -- 172.21.15.52:0/369407938 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420041070 con 0x7fa42c1036b0 2024-04-02T14:00:40.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.193+0000 7fa4317f6640 1 -- 172.21.15.52:0/369407938 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa42c1a6b70 con 0x7fa42c1036b0 2024-04-02T14:00:40.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.193+0000 7fa413fff640 1 -- 172.21.15.52:0/369407938 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa420004450 con 0x7fa42c1036b0 2024-04-02T14:00:40.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.193+0000 7fa413fff640 1 -- 172.21.15.52:0/369407938 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420003e00 con 0x7fa42c1036b0 2024-04-02T14:00:40.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.193+0000 7fa4317f6640 1 -- 172.21.15.52:0/369407938 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa42c1a7060 con 0x7fa42c1036b0 2024-04-02T14:00:40.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.194+0000 7fa4317f6640 1 -- 172.21.15.52:0/369407938 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa42c0688f0 con 0x7fa42c1036b0 2024-04-02T14:00:40.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.195+0000 7fa413fff640 1 -- 172.21.15.52:0/369407938 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa420044070 con 0x7fa42c1036b0 2024-04-02T14:00:40.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.196+0000 7fa413fff640 1 --2- 172.21.15.52:0/369407938 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa4140a3d20 0x7fa4140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.196+0000 7fa413fff640 1 -- 172.21.15.52:0/369407938 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7fa4200f25e0 con 0x7fa42c1036b0 2024-04-02T14:00:40.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.196+0000 7fa42affd640 1 --2- 172.21.15.52:0/369407938 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa4140a3d20 0x7fa4140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.197+0000 7fa42affd640 1 --2- 172.21.15.52:0/369407938 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa4140a3d20 0x7fa4140a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa42400bbe0 tx=0x7fa42400b390 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.201+0000 7fa413fff640 1 -- 172.21.15.52:0/369407938 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa4200bea00 con 0x7fa42c1036b0 2024-04-02T14:00:40.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.426+0000 7fa4317f6640 1 -- 172.21.15.52:0/369407938 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa42c107250 con 0x7fa42c1036b0 2024-04-02T14:00:40.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.427+0000 7fa413fff640 1 -- 172.21.15.52:0/369407938 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11923 (secure 0 0 0) 0x7fa4200be300 con 0x7fa42c1036b0 2024-04-02T14:00:40.431 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T14:00:40.431 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:57:45.240902+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24467,24457,24490,24475,24503]},"id":1}]} 2024-04-02T14:00:40.432 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-04-02T14:00:40.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.429+0000 7fa4317f6640 1 -- 172.21.15.52:0/369407938 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa4140a3d20 msgr2=0x7fa4140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.429+0000 7fa4317f6640 1 --2- 172.21.15.52:0/369407938 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa4140a3d20 0x7fa4140a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa42400bbe0 tx=0x7fa42400b390 comp rx=0 tx=0).stop 2024-04-02T14:00:40.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.429+0000 7fa4317f6640 1 -- 172.21.15.52:0/369407938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa42c1036b0 msgr2=0x7fa42c100f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.429+0000 7fa4317f6640 1 --2- 172.21.15.52:0/369407938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa42c1036b0 0x7fa42c100f20 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fa420006010 tx=0x7fa42002cb30 comp rx=0 tx=0).stop 2024-04-02T14:00:40.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.429+0000 7fa4317f6640 1 -- 172.21.15.52:0/369407938 shutdown_connections 2024-04-02T14:00:40.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.429+0000 7fa4317f6640 1 --2- 172.21.15.52:0/369407938 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa4140a3d20 0x7fa4140a61e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.429+0000 7fa4317f6640 1 --2- 172.21.15.52:0/369407938 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa42c10c160 0x7fa42c101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.429+0000 7fa4317f6640 1 --2- 172.21.15.52:0/369407938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa42c1036b0 0x7fa42c100f20 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.429+0000 7fa4317f6640 1 --2- 172.21.15.52:0/369407938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa42c102d90 0x7fa42c1009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.429+0000 7fa4317f6640 1 -- 172.21.15.52:0/369407938 >> 172.21.15.52:0/369407938 conn(0x7fa42c077b00 msgr2=0x7fa42c0fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.430+0000 7fa4317f6640 1 -- 172.21.15.52:0/369407938 shutdown_connections 2024-04-02T14:00:40.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.430+0000 7fa4317f6640 1 -- 172.21.15.52:0/369407938 wait complete. 2024-04-02T14:00:40.445 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T14:00:40.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.548+0000 7f83e517c640 1 -- 172.21.15.52:0/2314561285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83e0106980 msgr2=0x7f83e0106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.548+0000 7f83e517c640 1 --2- 172.21.15.52:0/2314561285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83e0106980 0x7f83e0106d60 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f83d4009930 tx=0x7f83d402bc30 comp rx=0 tx=0).stop 2024-04-02T14:00:40.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.548+0000 7f83e517c640 1 -- 172.21.15.52:0/2314561285 shutdown_connections 2024-04-02T14:00:40.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.548+0000 7f83e517c640 1 --2- 172.21.15.52:0/2314561285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83e0101330 0x7f83e010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.548+0000 7f83e517c640 1 --2- 172.21.15.52:0/2314561285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83e0100970 0x7f83e0100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.548+0000 7f83e517c640 1 --2- 172.21.15.52:0/2314561285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83e0106980 0x7f83e0106d60 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.548+0000 7f83e517c640 1 -- 172.21.15.52:0/2314561285 >> 172.21.15.52:0/2314561285 conn(0x7f83e0077b00 msgr2=0x7f83e0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.548+0000 7f83e517c640 1 -- 172.21.15.52:0/2314561285 shutdown_connections 2024-04-02T14:00:40.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.548+0000 7f83e517c640 1 -- 172.21.15.52:0/2314561285 wait complete. 2024-04-02T14:00:40.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.549+0000 7f83e517c640 1 Processor -- start 2024-04-02T14:00:40.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.549+0000 7f83e517c640 1 -- start start 2024-04-02T14:00:40.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.549+0000 7f83e517c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83e0100970 0x7f83e0197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.549+0000 7f83e517c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83e0101330 0x7f83e0197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.549+0000 7f83e517c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83e0106980 0x7f83e019bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.549+0000 7f83e517c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f83e0110db0 con 0x7f83e0100970 2024-04-02T14:00:40.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.549+0000 7f83e517c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f83e0110c30 con 0x7f83e0101330 2024-04-02T14:00:40.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.549+0000 7f83e517c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f83e0110f30 con 0x7f83e0106980 2024-04-02T14:00:40.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.550+0000 7f83ded9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83e0100970 0x7f83e0197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.550+0000 7f83ded9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83e0100970 0x7f83e0197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34068/0 (socket says 172.21.15.52:34068) 2024-04-02T14:00:40.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.550+0000 7f83ded9d640 1 -- 172.21.15.52:0/2011165479 learned_addr learned my addr 172.21.15.52:0/2011165479 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:40.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.550+0000 7f83df59e640 1 --2- 172.21.15.52:0/2011165479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83e0106980 0x7f83e019bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.550+0000 7f83de59c640 1 --2- 172.21.15.52:0/2011165479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83e0101330 0x7f83e0197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.550+0000 7f83ded9d640 1 -- 172.21.15.52:0/2011165479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83e0106980 msgr2=0x7f83e019bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.550+0000 7f83ded9d640 1 --2- 172.21.15.52:0/2011165479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83e0106980 0x7f83e019bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.550+0000 7f83ded9d640 1 -- 172.21.15.52:0/2011165479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83e0101330 msgr2=0x7f83e0197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.550+0000 7f83ded9d640 1 --2- 172.21.15.52:0/2011165479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83e0101330 0x7f83e0197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.550+0000 7f83ded9d640 1 -- 172.21.15.52:0/2011165479 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e019c300 con 0x7f83e0100970 2024-04-02T14:00:40.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.550+0000 7f83de59c640 1 --2- 172.21.15.52:0/2011165479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83e0101330 0x7f83e0197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:00:40.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.550+0000 7f83ded9d640 1 --2- 172.21.15.52:0/2011165479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83e0100970 0x7f83e0197310 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f83d4009900 tx=0x7f83d4004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.550+0000 7f83c7fff640 1 -- 172.21.15.52:0/2011165479 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d4043020 con 0x7f83e0100970 2024-04-02T14:00:40.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.550+0000 7f83e517c640 1 -- 172.21.15.52:0/2011165479 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83e019c590 con 0x7f83e0100970 2024-04-02T14:00:40.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.550+0000 7f83c7fff640 1 -- 172.21.15.52:0/2011165479 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d40054e0 con 0x7f83e0100970 2024-04-02T14:00:40.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.550+0000 7f83c7fff640 1 -- 172.21.15.52:0/2011165479 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d4003d00 con 0x7f83e0100970 2024-04-02T14:00:40.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.550+0000 7f83e517c640 1 -- 172.21.15.52:0/2011165479 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83e01a3e80 con 0x7f83e0100970 2024-04-02T14:00:40.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.553+0000 7f83c7fff640 1 -- 172.21.15.52:0/2011165479 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f83d4046020 con 0x7f83e0100970 2024-04-02T14:00:40.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.553+0000 7f83e7e37740 1 Processor -- start 2024-04-02T14:00:40.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.553+0000 7f83e7e37740 1 -- start start 2024-04-02T14:00:40.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.553+0000 7f83e7e37740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b73a277a0 0x556b73a27b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.553+0000 7f83e7e37740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a280c0 0x556b73a30760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.553+0000 7f83e7e37740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b73a30ca0 0x556b73a33090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.553+0000 7f83e7e37740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556b7398b7b0 con 0x556b73a280c0 2024-04-02T14:00:40.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.553+0000 7f83e7e37740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556b7392a570 con 0x556b73a30ca0 2024-04-02T14:00:40.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.553+0000 7f83e7e37740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556b73987760 con 0x556b73a277a0 2024-04-02T14:00:40.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.553+0000 7f83c7fff640 1 --2- 172.21.15.52:0/2011165479 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f83c80a3d20 0x7f83c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.554+0000 7f83df59e640 1 --2- 172.21.15.52:0/2011165479 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f83c80a3d20 0x7f83c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.554+0000 7f83ded9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a280c0 0x556b73a30760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.554+0000 7f83de59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b73a30ca0 0x556b73a33090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.554+0000 7f83df59e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b73a277a0 0x556b73a27b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.554+0000 7f83de59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b73a30ca0 0x556b73a33090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:38286/0 (socket says 172.21.15.52:38286) 2024-04-02T14:00:40.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.554+0000 7f83de59c640 1 -- 172.21.15.52:0/4046891897 learned_addr learned my addr 172.21.15.52:0/4046891897 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:40.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.554+0000 7f83df59e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b73a277a0 0x556b73a27b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.52:39152/0 (socket says 172.21.15.52:39152) 2024-04-02T14:00:40.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.554+0000 7f83c7fff640 1 -- 172.21.15.52:0/2011165479 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f83d40f2bf0 con 0x7f83e0100970 2024-04-02T14:00:40.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.554+0000 7f83de59c640 1 -- 172.21.15.52:0/4046891897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b73a277a0 msgr2=0x556b73a27b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.554+0000 7f83de59c640 1 --2- 172.21.15.52:0/4046891897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b73a277a0 0x556b73a27b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.554+0000 7f83de59c640 1 -- 172.21.15.52:0/4046891897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a280c0 msgr2=0x556b73a30760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.554+0000 7f83de59c640 1 --2- 172.21.15.52:0/4046891897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a280c0 0x556b73a30760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.554+0000 7f83de59c640 1 -- 172.21.15.52:0/4046891897 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b7369f3f0 con 0x556b73a30ca0 2024-04-02T14:00:40.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.554+0000 7f83df59e640 1 --2- 172.21.15.52:0/2011165479 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f83c80a3d20 0x7f83c80a61e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f83d80098b0 tx=0x7f83d8009d90 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.554+0000 7f83de59c640 1 --2- 172.21.15.52:0/4046891897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b73a30ca0 0x556b73a33090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f83d000bfd0 tx=0x7f83d0010930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ef9fe95818bd1cb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.554+0000 7f83c57fa640 1 -- 172.21.15.52:0/4046891897 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d001e070 con 0x556b73a30ca0 2024-04-02T14:00:40.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.554+0000 7f83c57fa640 1 -- 172.21.15.52:0/4046891897 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d000d9a0 con 0x556b73a30ca0 2024-04-02T14:00:40.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.554+0000 7f83c57fa640 1 -- 172.21.15.52:0/4046891897 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d0019990 con 0x556b73a30ca0 2024-04-02T14:00:40.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.555+0000 7f83e7e37740 1 -- 172.21.15.52:0/4046891897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b73a30ca0 msgr2=0x556b73a33090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.555+0000 7f83e7e37740 1 --2- 172.21.15.52:0/4046891897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b73a30ca0 0x556b73a33090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f83d000bfd0 tx=0x7f83d0010930 comp rx=0 tx=0).stop 2024-04-02T14:00:40.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.555+0000 7f83e7e37740 1 -- 172.21.15.52:0/4046891897 shutdown_connections 2024-04-02T14:00:40.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.555+0000 7f83e7e37740 1 --2- 172.21.15.52:0/4046891897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b73a30ca0 0x556b73a33090 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.555+0000 7f83e7e37740 1 --2- 172.21.15.52:0/4046891897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a280c0 0x556b73a30760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.555+0000 7f83e7e37740 1 --2- 172.21.15.52:0/4046891897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b73a277a0 0x556b73a27b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.555+0000 7f83e7e37740 1 -- 172.21.15.52:0/4046891897 >> 172.21.15.52:0/4046891897 conn(0x556b73a21ed0 msgr2=0x556b73a222b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.555+0000 7f83e7e37740 1 -- 172.21.15.52:0/4046891897 shutdown_connections 2024-04-02T14:00:40.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.555+0000 7f83e7e37740 1 -- 172.21.15.52:0/4046891897 wait complete. 2024-04-02T14:00:40.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.555+0000 7f83e7e37740 1 -- start start 2024-04-02T14:00:40.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.556+0000 7f83e7e37740 1 Processor -- start 2024-04-02T14:00:40.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.556+0000 7f83e7e37740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b73a277a0 0x556b73a306f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.556+0000 7f83e7e37740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a280c0 0x556b73a2da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.556+0000 7f83e7e37740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b73a30ca0 0x556b73a2e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.556+0000 7f83e7e37740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556b73a33a90 con 0x556b73a280c0 2024-04-02T14:00:40.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.556+0000 7f83e7e37740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556b73946e80 con 0x556b73a277a0 2024-04-02T14:00:40.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.556+0000 7f83e7e37740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556b73a33c10 con 0x556b73a30ca0 2024-04-02T14:00:40.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.556+0000 7f83e7e37740 10 client.0 _ll_register_callbacks cb 0x556b73982aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:40.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.556+0000 7f83de59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a280c0 0x556b73a2da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.556+0000 7f83ded9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b73a277a0 0x556b73a306f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.556+0000 7f83de59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a280c0 0x556b73a2da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34092/0 (socket says 172.21.15.52:34092) 2024-04-02T14:00:40.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.556+0000 7f83de59c640 1 -- 172.21.15.52:0/3551702501 learned_addr learned my addr 172.21.15.52:0/3551702501 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:40.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.556+0000 7f83df59e640 1 --2- 172.21.15.52:0/3551702501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b73a30ca0 0x556b73a2e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.556+0000 7f83de59c640 1 -- 172.21.15.52:0/3551702501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b73a30ca0 msgr2=0x556b73a2e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.556+0000 7f83de59c640 1 --2- 172.21.15.52:0/3551702501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b73a30ca0 0x556b73a2e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.556+0000 7f83de59c640 1 -- 172.21.15.52:0/3551702501 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b73a277a0 msgr2=0x556b73a306f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.556+0000 7f83de59c640 1 --2- 172.21.15.52:0/3551702501 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b73a277a0 0x556b73a306f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.556+0000 7f83de59c640 1 -- 172.21.15.52:0/3551702501 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b737697d0 con 0x556b73a280c0 2024-04-02T14:00:40.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.556+0000 7f83ded9d640 1 --2- 172.21.15.52:0/3551702501 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b73a277a0 0x556b73a306f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:00:40.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.557+0000 7f83de59c640 1 --2- 172.21.15.52:0/3551702501 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a280c0 0x556b73a2da80 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f83d0019510 tx=0x7f83d000de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4c7dc4707ab9b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.557+0000 7f83c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:40.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.557+0000 7f83c2ff5640 1 -- 172.21.15.52:0/3551702501 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d001e070 con 0x556b73a280c0 2024-04-02T14:00:40.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.557+0000 7f83c2ff5640 1 -- 172.21.15.52:0/3551702501 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f83d000d790 con 0x556b73a280c0 2024-04-02T14:00:40.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.557+0000 7f83e7e37740 1 -- 172.21.15.52:0/3551702501 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83d000d9a0 con 0x556b73a280c0 2024-04-02T14:00:40.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.557+0000 7f83c2ff5640 1 -- 172.21.15.52:0/3551702501 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f83d002f050 con 0x556b73a280c0 2024-04-02T14:00:40.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.557+0000 7f83e7e37740 10 client.17628 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:40.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.557+0000 7f83e7e37740 1 -- 172.21.15.52:0/3551702501 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556b73a3fb10 con 0x556b73a280c0 2024-04-02T14:00:40.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.557+0000 7f83c2ff5640 1 -- 172.21.15.52:0/3551702501 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d000d9a0 con 0x556b73a280c0 2024-04-02T14:00:40.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.557+0000 7f83c2ff5640 1 -- 172.21.15.52:0/3551702501 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f83d002a030 con 0x556b73a280c0 2024-04-02T14:00:40.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.557+0000 7f83e7e37740 10 client.17628 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:40.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.557+0000 7f83e7e37740 10 client.17628 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T14:00:40.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.557+0000 7f83e7e37740 20 client.17628 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:40.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.557+0000 7f83e7e37740 1 --2- 172.21.15.52:0/3551702501 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x556b73a40f20 0x556b73a43310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.557+0000 7f83e7e37740 4 client.17628 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:00:40.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.557+0000 7f83e7e37740 1 -- 172.21.15.52:0/3551702501 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b73998cf0 con 0x556b73a40f20 2024-04-02T14:00:40.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.558+0000 7f83ded9d640 1 --2- 172.21.15.52:0/3551702501 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x556b73a40f20 0x556b73a43310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.558+0000 7f83c2ff5640 10 client.17628 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T14:00:40.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.558+0000 7f83ded9d640 1 --2- 172.21.15.52:0/3551702501 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x556b73a40f20 0x556b73a43310 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b509e61e65a0e735 server_cookie=6243f24d2ca31e4c in_seq=0 out_seq=0 2024-04-02T14:00:40.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.559+0000 7f83c2ff5640 1 -- 172.21.15.52:0/3551702501 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556b73998cf0 con 0x556b73a40f20 2024-04-02T14:00:40.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.559+0000 7f83c2ff5640 10 client.17628 handle_command_reply: tid=0 2024-04-02T14:00:40.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.559+0000 7f83e7e37740 1 client.17628 shutdown 2024-04-02T14:00:40.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.559+0000 7f83e7e37740 1 -- 172.21.15.52:0/3551702501 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a280c0 msgr2=0x556b73a2da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.559+0000 7f83e7e37740 1 --2- 172.21.15.52:0/3551702501 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a280c0 0x556b73a2da80 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f83d0019510 tx=0x7f83d000de00 comp rx=0 tx=0).stop 2024-04-02T14:00:40.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.559+0000 7f83e7e37740 1 -- 172.21.15.52:0/3551702501 shutdown_connections 2024-04-02T14:00:40.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.559+0000 7f83e7e37740 1 --2- 172.21.15.52:0/3551702501 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x556b73a40f20 0x556b73a43310 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.559+0000 7f83e7e37740 1 --2- 172.21.15.52:0/3551702501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b73a30ca0 0x556b73a2e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.559+0000 7f83e7e37740 1 --2- 172.21.15.52:0/3551702501 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a280c0 0x556b73a2da80 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.559+0000 7f83c2ff5640 0 client.17628 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T14:00:40.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.559+0000 7f83e7e37740 1 --2- 172.21.15.52:0/3551702501 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b73a277a0 0x556b73a306f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.559+0000 7f83e7e37740 1 -- 172.21.15.52:0/3551702501 >> 172.21.15.52:0/3551702501 conn(0x556b73a21ed0 msgr2=0x556b73a23c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.559+0000 7f83e7e37740 1 -- 172.21.15.52:0/3551702501 shutdown_connections 2024-04-02T14:00:40.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.559+0000 7f83e7e37740 1 -- 172.21.15.52:0/3551702501 wait complete. 2024-04-02T14:00:40.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.559+0000 7f83e7e37740 20 client.17628 trim_cache size 0 max 16384 2024-04-02T14:00:40.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.572+0000 7f83e7e37740 1 Processor -- start 2024-04-02T14:00:40.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.572+0000 7f83e7e37740 1 -- start start 2024-04-02T14:00:40.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.572+0000 7f83e7e37740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b73a277a0 0x556b73a3c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.572+0000 7f83e7e37740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b73a280c0 0x556b73a32fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.572+0000 7f83e7e37740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a30ca0 0x556b73a489a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.572+0000 7f83e7e37740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f83e01acdf0 con 0x556b73a30ca0 2024-04-02T14:00:40.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.572+0000 7f83e7e37740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f83e01ae930 con 0x556b73a280c0 2024-04-02T14:00:40.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.572+0000 7f83e7e37740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f83e01ad4c0 con 0x556b73a277a0 2024-04-02T14:00:40.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.572+0000 7f83df59e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a30ca0 0x556b73a489a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.573+0000 7f83df59e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a30ca0 0x556b73a489a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34102/0 (socket says 172.21.15.52:34102) 2024-04-02T14:00:40.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.573+0000 7f83df59e640 1 -- 172.21.15.52:0/638558095 learned_addr learned my addr 172.21.15.52:0/638558095 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:40.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.573+0000 7f83ded9d640 1 --2- 172.21.15.52:0/638558095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b73a277a0 0x556b73a3c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.573+0000 7f83de59c640 1 --2- 172.21.15.52:0/638558095 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b73a280c0 0x556b73a32fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.573+0000 7f83df59e640 1 -- 172.21.15.52:0/638558095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b73a277a0 msgr2=0x556b73a3c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.573+0000 7f83df59e640 1 --2- 172.21.15.52:0/638558095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b73a277a0 0x556b73a3c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.573+0000 7f83df59e640 1 -- 172.21.15.52:0/638558095 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b73a280c0 msgr2=0x556b73a32fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.573+0000 7f83df59e640 1 --2- 172.21.15.52:0/638558095 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b73a280c0 0x556b73a32fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.573+0000 7f83df59e640 1 -- 172.21.15.52:0/638558095 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b73a2e530 con 0x556b73a30ca0 2024-04-02T14:00:40.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.573+0000 7f83df59e640 1 --2- 172.21.15.52:0/638558095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a30ca0 0x556b73a489a0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009a90 tx=0x7f83d8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149574e66e0614e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.573+0000 7f83c4ff9640 1 -- 172.21.15.52:0/638558095 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8004b20 con 0x556b73a30ca0 2024-04-02T14:00:40.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.573+0000 7f83c4ff9640 1 -- 172.21.15.52:0/638558095 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d8004cc0 con 0x556b73a30ca0 2024-04-02T14:00:40.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.573+0000 7f83c4ff9640 1 -- 172.21.15.52:0/638558095 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8005410 con 0x556b73a30ca0 2024-04-02T14:00:40.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.574+0000 7f83e7e37740 1 -- 172.21.15.52:0/638558095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a30ca0 msgr2=0x556b73a489a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.574+0000 7f83e7e37740 1 --2- 172.21.15.52:0/638558095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a30ca0 0x556b73a489a0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009a90 tx=0x7f83d8004ab0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.574+0000 7f83e7e37740 1 -- 172.21.15.52:0/638558095 shutdown_connections 2024-04-02T14:00:40.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.574+0000 7f83e7e37740 1 --2- 172.21.15.52:0/638558095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a30ca0 0x556b73a489a0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.574+0000 7f83e7e37740 1 --2- 172.21.15.52:0/638558095 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b73a280c0 0x556b73a32fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.574+0000 7f83e7e37740 1 --2- 172.21.15.52:0/638558095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b73a277a0 0x556b73a3c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.574+0000 7f83e7e37740 1 -- 172.21.15.52:0/638558095 >> 172.21.15.52:0/638558095 conn(0x556b73a21ed0 msgr2=0x556b73a222b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.574+0000 7f83e7e37740 1 -- 172.21.15.52:0/638558095 shutdown_connections 2024-04-02T14:00:40.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.574+0000 7f83e7e37740 1 -- 172.21.15.52:0/638558095 wait complete. 2024-04-02T14:00:40.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.574+0000 7f83e7e37740 1 -- start start 2024-04-02T14:00:40.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.574+0000 7f83e7e37740 1 Processor -- start 2024-04-02T14:00:40.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.575+0000 7f83e7e37740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a277a0 0x556b73a32030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.575+0000 7f83e7e37740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b73a280c0 0x556b73a32570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.575+0000 7f83e7e37740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b73a30ca0 0x556b73a32ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.575+0000 7f83e7e37740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f83e01ae260 con 0x556b73a277a0 2024-04-02T14:00:40.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.575+0000 7f83e7e37740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f83e01adb90 con 0x556b73a280c0 2024-04-02T14:00:40.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.575+0000 7f83e7e37740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f83e01aadf0 con 0x556b73a30ca0 2024-04-02T14:00:40.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.575+0000 7f83e7e37740 10 client.0 _ll_register_callbacks cb 0x7f83d001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:40.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.575+0000 7f83ded9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a277a0 0x556b73a32030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.575+0000 7f83ded9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a277a0 0x556b73a32030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34106/0 (socket says 172.21.15.52:34106) 2024-04-02T14:00:40.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.575+0000 7f83ded9d640 1 -- 172.21.15.52:0/622453404 learned_addr learned my addr 172.21.15.52:0/622453404 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:40.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.575+0000 7f83df59e640 1 --2- 172.21.15.52:0/622453404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b73a30ca0 0x556b73a32ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.575+0000 7f83de59c640 1 --2- 172.21.15.52:0/622453404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b73a280c0 0x556b73a32570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.575+0000 7f83ded9d640 1 -- 172.21.15.52:0/622453404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b73a30ca0 msgr2=0x556b73a32ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.575+0000 7f83ded9d640 1 --2- 172.21.15.52:0/622453404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b73a30ca0 0x556b73a32ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.575+0000 7f83ded9d640 1 -- 172.21.15.52:0/622453404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b73a280c0 msgr2=0x556b73a32570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.575+0000 7f83ded9d640 1 --2- 172.21.15.52:0/622453404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b73a280c0 0x556b73a32570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.575+0000 7f83ded9d640 1 -- 172.21.15.52:0/622453404 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b73a27ef0 con 0x556b73a277a0 2024-04-02T14:00:40.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.575+0000 7f83ded9d640 1 --2- 172.21.15.52:0/622453404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a277a0 0x556b73a32030 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f83d40f05b0 tx=0x7f83d4090350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f791aac39a813717 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.575+0000 7f83c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:40.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.575+0000 7f83c2ff5640 1 -- 172.21.15.52:0/622453404 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d40035f0 con 0x556b73a277a0 2024-04-02T14:00:40.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.575+0000 7f83c2ff5640 1 -- 172.21.15.52:0/622453404 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f83d40baf80 con 0x556b73a277a0 2024-04-02T14:00:40.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.575+0000 7f83e7e37740 1 -- 172.21.15.52:0/622453404 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83d8004cc0 con 0x556b73a277a0 2024-04-02T14:00:40.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.575+0000 7f83c2ff5640 1 -- 172.21.15.52:0/622453404 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f83d4005690 con 0x556b73a277a0 2024-04-02T14:00:40.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.575+0000 7f83e7e37740 10 client.17634 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:40.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.575+0000 7f83e7e37740 1 -- 172.21.15.52:0/622453404 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556b73aa7210 con 0x556b73a277a0 2024-04-02T14:00:40.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.576+0000 7f83c2ff5640 1 -- 172.21.15.52:0/622453404 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d8004cc0 con 0x556b73a277a0 2024-04-02T14:00:40.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.576+0000 7f83c2ff5640 1 -- 172.21.15.52:0/622453404 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f83d40a90b0 con 0x556b73a277a0 2024-04-02T14:00:40.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.576+0000 7f83e7e37740 10 client.17634 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:40.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.576+0000 7f83e7e37740 10 client.17634 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T14:00:40.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.576+0000 7f83e7e37740 20 client.17634 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:40.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.576+0000 7f83e7e37740 1 --2- 172.21.15.52:0/622453404 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x556b73aa78e0 0x556b73aa9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.576+0000 7f83e7e37740 4 client.17634 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:00:40.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.576+0000 7f83e7e37740 1 -- 172.21.15.52:0/622453404 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b73993040 con 0x556b73aa78e0 2024-04-02T14:00:40.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.576+0000 7f83de59c640 1 --2- 172.21.15.52:0/622453404 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x556b73aa78e0 0x556b73aa9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.576+0000 7f83de59c640 1 --2- 172.21.15.52:0/622453404 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x556b73aa78e0 0x556b73aa9cd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9e61115173ee240f server_cookie=6c49a479fc75b096 in_seq=0 out_seq=0 2024-04-02T14:00:40.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.576+0000 7f83c2ff5640 10 client.17634 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T14:00:40.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.577+0000 7f83c2ff5640 1 -- 172.21.15.52:0/622453404 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b73993040 con 0x556b73aa78e0 2024-04-02T14:00:40.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.577+0000 7f83c2ff5640 10 client.17634 handle_command_reply: tid=0 2024-04-02T14:00:40.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.577+0000 7f83e7e37740 1 client.17634 shutdown 2024-04-02T14:00:40.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.577+0000 7f83e7e37740 1 -- 172.21.15.52:0/622453404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a277a0 msgr2=0x556b73a32030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.577+0000 7f83e7e37740 1 --2- 172.21.15.52:0/622453404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a277a0 0x556b73a32030 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f83d40f05b0 tx=0x7f83d4090350 comp rx=0 tx=0).stop 2024-04-02T14:00:40.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.577+0000 7f83e7e37740 1 -- 172.21.15.52:0/622453404 shutdown_connections 2024-04-02T14:00:40.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.577+0000 7f83e7e37740 1 --2- 172.21.15.52:0/622453404 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x556b73aa78e0 0x556b73aa9cd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.577+0000 7f83e7e37740 1 --2- 172.21.15.52:0/622453404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b73a30ca0 0x556b73a32ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.577+0000 7f83e7e37740 1 --2- 172.21.15.52:0/622453404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b73a280c0 0x556b73a32570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.577+0000 7f83c2ff5640 0 client.17634 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T14:00:40.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.577+0000 7f83e7e37740 1 --2- 172.21.15.52:0/622453404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b73a277a0 0x556b73a32030 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.577+0000 7f83e7e37740 1 -- 172.21.15.52:0/622453404 >> 172.21.15.52:0/622453404 conn(0x556b73a21ed0 msgr2=0x556b73a206a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.577+0000 7f83e7e37740 1 -- 172.21.15.52:0/622453404 shutdown_connections 2024-04-02T14:00:40.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.577+0000 7f83e7e37740 1 -- 172.21.15.52:0/622453404 wait complete. 2024-04-02T14:00:40.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.577+0000 7f83e7e37740 20 client.17634 trim_cache size 0 max 16384 2024-04-02T14:00:40.593 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:00:40.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.577+0000 7f83e517c640 1 -- 172.21.15.52:0/2011165479 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f83c80a3d20 msgr2=0x7f83c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.577+0000 7f83e517c640 1 --2- 172.21.15.52:0/2011165479 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f83c80a3d20 0x7f83c80a61e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f83d80098b0 tx=0x7f83d8009d90 comp rx=0 tx=0).stop 2024-04-02T14:00:40.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.577+0000 7f83e517c640 1 -- 172.21.15.52:0/2011165479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83e0100970 msgr2=0x7f83e0197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.577+0000 7f83e517c640 1 --2- 172.21.15.52:0/2011165479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83e0100970 0x7f83e0197310 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f83d4009900 tx=0x7f83d4004430 comp rx=0 tx=0).stop 2024-04-02T14:00:40.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.577+0000 7f83e517c640 1 -- 172.21.15.52:0/2011165479 shutdown_connections 2024-04-02T14:00:40.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.577+0000 7f83e517c640 1 --2- 172.21.15.52:0/2011165479 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f83c80a3d20 0x7f83c80a61e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.577+0000 7f83e517c640 1 --2- 172.21.15.52:0/2011165479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83e0106980 0x7f83e019bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.577+0000 7f83e517c640 1 --2- 172.21.15.52:0/2011165479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83e0101330 0x7f83e0197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.577+0000 7f83e517c640 1 --2- 172.21.15.52:0/2011165479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83e0100970 0x7f83e0197310 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.577+0000 7f83e517c640 1 -- 172.21.15.52:0/2011165479 >> 172.21.15.52:0/2011165479 conn(0x7f83e0077b00 msgr2=0x7f83e01031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.578+0000 7f83e517c640 1 -- 172.21.15.52:0/2011165479 shutdown_connections 2024-04-02T14:00:40.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.578+0000 7f83e517c640 1 -- 172.21.15.52:0/2011165479 wait complete. 2024-04-02T14:00:40.596 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T14:00:40.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.698+0000 7f229a6d9640 1 -- 172.21.15.52:0/1745101610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2294068d00 msgr2=0x7f2294069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.698+0000 7f229a6d9640 1 --2- 172.21.15.52:0/1745101610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2294068d00 0x7f2294069180 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f2284009930 tx=0x7f228402bc30 comp rx=0 tx=0).stop 2024-04-02T14:00:40.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.698+0000 7f229a6d9640 1 -- 172.21.15.52:0/1745101610 shutdown_connections 2024-04-02T14:00:40.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.698+0000 7f229a6d9640 1 --2- 172.21.15.52:0/1745101610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22940696c0 0x7f229410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.699+0000 7f229a6d9640 1 --2- 172.21.15.52:0/1745101610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2294068d00 0x7f2294069180 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.699+0000 7f229a6d9640 1 --2- 172.21.15.52:0/1745101610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2294107b50 0x7f2294107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.699+0000 7f229a6d9640 1 -- 172.21.15.52:0/1745101610 >> 172.21.15.52:0/1745101610 conn(0x7f2294077b00 msgr2=0x7f2294077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.699+0000 7f229a6d9640 1 -- 172.21.15.52:0/1745101610 shutdown_connections 2024-04-02T14:00:40.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.699+0000 7f229a6d9640 1 -- 172.21.15.52:0/1745101610 wait complete. 2024-04-02T14:00:40.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.699+0000 7f229a6d9640 1 Processor -- start 2024-04-02T14:00:40.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.699+0000 7f229a6d9640 1 -- start start 2024-04-02T14:00:40.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.699+0000 7f229a6d9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2294068d00 0x7f22941009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.700+0000 7f229a6d9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22940696c0 0x7f2294100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.700+0000 7f229a6d9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2294107b50 0x7f2294101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.700+0000 7f229a6d9640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2294111e70 con 0x7f22940696c0 2024-04-02T14:00:40.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.700+0000 7f229a6d9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2294111cf0 con 0x7f2294107b50 2024-04-02T14:00:40.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.700+0000 7f229a6d9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2294111ff0 con 0x7f2294068d00 2024-04-02T14:00:40.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.700+0000 7f22937fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22940696c0 0x7f2294100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.700+0000 7f22937fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22940696c0 0x7f2294100f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34126/0 (socket says 172.21.15.52:34126) 2024-04-02T14:00:40.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.700+0000 7f22937fe640 1 -- 172.21.15.52:0/3645770680 learned_addr learned my addr 172.21.15.52:0/3645770680 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:40.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.700+0000 7f2293fff640 1 --2- 172.21.15.52:0/3645770680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2294068d00 0x7f22941009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.700+0000 7f2298c76640 1 --2- 172.21.15.52:0/3645770680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2294107b50 0x7f2294101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.700+0000 7f22937fe640 1 -- 172.21.15.52:0/3645770680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2294068d00 msgr2=0x7f22941009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.700+0000 7f22937fe640 1 --2- 172.21.15.52:0/3645770680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2294068d00 0x7f22941009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.700+0000 7f22937fe640 1 -- 172.21.15.52:0/3645770680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2294107b50 msgr2=0x7f2294101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.700+0000 7f22937fe640 1 --2- 172.21.15.52:0/3645770680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2294107b50 0x7f2294101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.700+0000 7f22937fe640 1 -- 172.21.15.52:0/3645770680 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22941a6850 con 0x7f22940696c0 2024-04-02T14:00:40.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.700+0000 7f22937fe640 1 --2- 172.21.15.52:0/3645770680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22940696c0 0x7f2294100f00 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f2284009230 tx=0x7f2284004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.700+0000 7f22917fa640 1 -- 172.21.15.52:0/3645770680 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284041070 con 0x7f22940696c0 2024-04-02T14:00:40.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.700+0000 7f229a6d9640 1 -- 172.21.15.52:0/3645770680 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22941a6ae0 con 0x7f22940696c0 2024-04-02T14:00:40.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.700+0000 7f22917fa640 1 -- 172.21.15.52:0/3645770680 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22840054e0 con 0x7f22940696c0 2024-04-02T14:00:40.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.700+0000 7f22917fa640 1 -- 172.21.15.52:0/3645770680 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284003c90 con 0x7f22940696c0 2024-04-02T14:00:40.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.700+0000 7f229a6d9640 1 -- 172.21.15.52:0/3645770680 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22941a7020 con 0x7f22940696c0 2024-04-02T14:00:40.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.703+0000 7f22917fa640 1 -- 172.21.15.52:0/3645770680 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2284044070 con 0x7f22940696c0 2024-04-02T14:00:40.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.703+0000 7f229d3a0740 1 Processor -- start 2024-04-02T14:00:40.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.703+0000 7f229d3a0740 1 -- start start 2024-04-02T14:00:40.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.703+0000 7f22917fa640 1 --2- 172.21.15.52:0/3645770680 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f227c0a3d20 0x7f227c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.703+0000 7f229d3a0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a2bf3ea7a0 0x55a2bf3eab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.703+0000 7f229d3a0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3eb0c0 0x55a2bf3f3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.703+0000 7f229d3a0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2bf3f3ca0 0x55a2bf3f6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.703+0000 7f229d3a0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a2bf34e7b0 con 0x55a2bf3ea7a0 2024-04-02T14:00:40.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.703+0000 7f229d3a0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a2bf2ed570 con 0x55a2bf3eb0c0 2024-04-02T14:00:40.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.703+0000 7f229d3a0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a2bf34a760 con 0x55a2bf3f3ca0 2024-04-02T14:00:40.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.703+0000 7f2293fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a2bf3ea7a0 0x55a2bf3eab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.703+0000 7f2298c76640 1 --2- 172.21.15.52:0/3645770680 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f227c0a3d20 0x7f227c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.703+0000 7f22937fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3eb0c0 0x55a2bf3f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.703+0000 7f2293fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a2bf3ea7a0 0x55a2bf3eab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34140/0 (socket says 172.21.15.52:34140) 2024-04-02T14:00:40.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.703+0000 7f2293fff640 1 -- 172.21.15.52:0/3502517511 learned_addr learned my addr 172.21.15.52:0/3502517511 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:40.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.703+0000 7f22917fa640 1 -- 172.21.15.52:0/3645770680 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f22840f26c0 con 0x7f22940696c0 2024-04-02T14:00:40.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.703+0000 7f2298c76640 1 --2- 172.21.15.52:0/3502517511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2bf3f3ca0 0x55a2bf3f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.704+0000 7f22937fe640 1 -- 172.21.15.52:0/3502517511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2bf3f3ca0 msgr2=0x55a2bf3f6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.704+0000 7f22937fe640 1 --2- 172.21.15.52:0/3502517511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2bf3f3ca0 0x55a2bf3f6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.704+0000 7f22937fe640 1 -- 172.21.15.52:0/3502517511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a2bf3ea7a0 msgr2=0x55a2bf3eab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.704+0000 7f22937fe640 1 --2- 172.21.15.52:0/3502517511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a2bf3ea7a0 0x55a2bf3eab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.704+0000 7f22937fe640 1 -- 172.21.15.52:0/3502517511 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2bf0623f0 con 0x55a2bf3eb0c0 2024-04-02T14:00:40.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.704+0000 7f2293fff640 1 --2- 172.21.15.52:0/3502517511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a2bf3ea7a0 0x55a2bf3eab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:00:40.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.704+0000 7f2298c76640 1 --2- 172.21.15.52:0/3645770680 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f227c0a3d20 0x7f227c0a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f228c005fd0 tx=0x7f228c009f10 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.704+0000 7f22937fe640 1 --2- 172.21.15.52:0/3502517511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3eb0c0 0x55a2bf3f3760 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f22840efb90 tx=0x7f2284003760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e08ab9ea65403419 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.704+0000 7f227a7fc640 1 -- 172.21.15.52:0/3502517511 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228404b070 con 0x55a2bf3eb0c0 2024-04-02T14:00:40.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.704+0000 7f227a7fc640 1 -- 172.21.15.52:0/3502517511 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f228402ccd0 con 0x55a2bf3eb0c0 2024-04-02T14:00:40.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.704+0000 7f227a7fc640 1 -- 172.21.15.52:0/3502517511 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22840a26d0 con 0x55a2bf3eb0c0 2024-04-02T14:00:40.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.705+0000 7f229d3a0740 1 -- 172.21.15.52:0/3502517511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3eb0c0 msgr2=0x55a2bf3f3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.705+0000 7f229d3a0740 1 --2- 172.21.15.52:0/3502517511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3eb0c0 0x55a2bf3f3760 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f22840efb90 tx=0x7f2284003760 comp rx=0 tx=0).stop 2024-04-02T14:00:40.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.705+0000 7f229d3a0740 1 -- 172.21.15.52:0/3502517511 shutdown_connections 2024-04-02T14:00:40.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.705+0000 7f229d3a0740 1 --2- 172.21.15.52:0/3502517511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2bf3f3ca0 0x55a2bf3f6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.705+0000 7f229d3a0740 1 --2- 172.21.15.52:0/3502517511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3eb0c0 0x55a2bf3f3760 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.705+0000 7f229d3a0740 1 --2- 172.21.15.52:0/3502517511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a2bf3ea7a0 0x55a2bf3eab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.705+0000 7f229d3a0740 1 -- 172.21.15.52:0/3502517511 >> 172.21.15.52:0/3502517511 conn(0x55a2bf3e4ed0 msgr2=0x55a2bf3e52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.705+0000 7f229d3a0740 1 -- 172.21.15.52:0/3502517511 shutdown_connections 2024-04-02T14:00:40.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.705+0000 7f229d3a0740 1 -- 172.21.15.52:0/3502517511 wait complete. 2024-04-02T14:00:40.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.705+0000 7f229d3a0740 1 -- start start 2024-04-02T14:00:40.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.705+0000 7f229d3a0740 1 Processor -- start 2024-04-02T14:00:40.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.706+0000 7f229d3a0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3ea7a0 0x55a2bf3f0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.706+0000 7f229d3a0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2bf3eb0c0 0x55a2bf3f10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.706+0000 7f229d3a0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a2bf3f3ca0 0x55a2bf3f15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.706+0000 7f229d3a0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a2bf3f6850 con 0x55a2bf3f3ca0 2024-04-02T14:00:40.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.706+0000 7f229d3a0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a2bf309e80 con 0x55a2bf3ea7a0 2024-04-02T14:00:40.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.706+0000 7f229d3a0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a2bf3f69d0 con 0x55a2bf3eb0c0 2024-04-02T14:00:40.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.706+0000 7f229d3a0740 10 client.0 _ll_register_callbacks cb 0x55a2bf345aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:40.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.706+0000 7f2293fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3ea7a0 0x55a2bf3f0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.706+0000 7f2293fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3ea7a0 0x55a2bf3f0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:38338/0 (socket says 172.21.15.52:38338) 2024-04-02T14:00:40.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.706+0000 7f2298c76640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a2bf3f3ca0 0x55a2bf3f15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.706+0000 7f2293fff640 1 -- 172.21.15.52:0/270765197 learned_addr learned my addr 172.21.15.52:0/270765197 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:40.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.706+0000 7f22937fe640 1 --2- 172.21.15.52:0/270765197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2bf3eb0c0 0x55a2bf3f10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.706+0000 7f2293fff640 1 -- 172.21.15.52:0/270765197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2bf3eb0c0 msgr2=0x55a2bf3f10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.706+0000 7f2293fff640 1 --2- 172.21.15.52:0/270765197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2bf3eb0c0 0x55a2bf3f10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.706+0000 7f2293fff640 1 -- 172.21.15.52:0/270765197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a2bf3f3ca0 msgr2=0x55a2bf3f15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.706+0000 7f2293fff640 1 --2- 172.21.15.52:0/270765197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a2bf3f3ca0 0x55a2bf3f15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.706+0000 7f2293fff640 1 -- 172.21.15.52:0/270765197 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2bf12c7d0 con 0x55a2bf3ea7a0 2024-04-02T14:00:40.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.706+0000 7f2293fff640 1 --2- 172.21.15.52:0/270765197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3ea7a0 0x55a2bf3f0b60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2288012740 tx=0x7f2288012c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75051155eb57c523 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.706+0000 7f22737fe640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-02T14:00:40.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.706+0000 7f22737fe640 1 -- 172.21.15.52:0/270765197 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228800f8a0 con 0x55a2bf3ea7a0 2024-04-02T14:00:40.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.707+0000 7f229d3a0740 1 -- 172.21.15.52:0/270765197 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a2bf35bcf0 con 0x55a2bf3ea7a0 2024-04-02T14:00:40.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.707+0000 7f22737fe640 1 -- 172.21.15.52:0/270765197 <== mon.2 v2:172.21.15.88:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f228800fa40 con 0x55a2bf3ea7a0 2024-04-02T14:00:40.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.707+0000 7f22737fe640 1 -- 172.21.15.52:0/270765197 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f228800fc40 con 0x55a2bf3ea7a0 2024-04-02T14:00:40.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.707+0000 7f229d3a0740 10 client.26828 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:40.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.707+0000 7f229d3a0740 1 -- 172.21.15.52:0/270765197 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a2bf4028d0 con 0x55a2bf3ea7a0 2024-04-02T14:00:40.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.707+0000 7f22737fe640 1 -- 172.21.15.52:0/270765197 <== mon.2 v2:172.21.15.88:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a2bf35bcf0 con 0x55a2bf3ea7a0 2024-04-02T14:00:40.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.707+0000 7f22737fe640 1 -- 172.21.15.52:0/270765197 <== mon.2 v2:172.21.15.88:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f228801c420 con 0x55a2bf3ea7a0 2024-04-02T14:00:40.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.707+0000 7f229d3a0740 10 client.26828 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:40.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.707+0000 7f229d3a0740 10 client.26828 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T14:00:40.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.707+0000 7f229d3a0740 20 client.26828 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:40.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.707+0000 7f229d3a0740 1 --2- 172.21.15.52:0/270765197 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55a2bf403d90 0x55a2bf4061f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.707+0000 7f229d3a0740 4 client.26828 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:00:40.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.707+0000 7f229d3a0740 1 -- 172.21.15.52:0/270765197 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2bf356040 con 0x55a2bf403d90 2024-04-02T14:00:40.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.707+0000 7f22937fe640 1 --2- 172.21.15.52:0/270765197 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55a2bf403d90 0x55a2bf4061f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.708+0000 7f22737fe640 10 client.26828 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T14:00:40.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.708+0000 7f22937fe640 1 --2- 172.21.15.52:0/270765197 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55a2bf403d90 0x55a2bf4061f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8d7e95102ce0a34e server_cookie=159fd0ca880a51de in_seq=0 out_seq=0 2024-04-02T14:00:40.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.708+0000 7f22737fe640 1 -- 172.21.15.52:0/270765197 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a2bf356040 con 0x55a2bf403d90 2024-04-02T14:00:40.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.708+0000 7f22737fe640 10 client.26828 handle_command_reply: tid=0 2024-04-02T14:00:40.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.708+0000 7f229d3a0740 1 client.26828 shutdown 2024-04-02T14:00:40.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.709+0000 7f229d3a0740 1 -- 172.21.15.52:0/270765197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3ea7a0 msgr2=0x55a2bf3f0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.709+0000 7f229d3a0740 1 --2- 172.21.15.52:0/270765197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3ea7a0 0x55a2bf3f0b60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2288012740 tx=0x7f2288012c00 comp rx=0 tx=0).stop 2024-04-02T14:00:40.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.709+0000 7f229d3a0740 1 -- 172.21.15.52:0/270765197 shutdown_connections 2024-04-02T14:00:40.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.709+0000 7f229d3a0740 1 --2- 172.21.15.52:0/270765197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a2bf3f3ca0 0x55a2bf3f15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.709+0000 7f229d3a0740 1 --2- 172.21.15.52:0/270765197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2bf3eb0c0 0x55a2bf3f10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.709+0000 7f229d3a0740 1 --2- 172.21.15.52:0/270765197 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55a2bf403d90 0x55a2bf4061f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.709+0000 7f229d3a0740 1 --2- 172.21.15.52:0/270765197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3ea7a0 0x55a2bf3f0b60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.709+0000 7f22737fe640 0 client.26828 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T14:00:40.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.709+0000 7f229d3a0740 1 -- 172.21.15.52:0/270765197 >> 172.21.15.52:0/270765197 conn(0x55a2bf3e4ed0 msgr2=0x55a2bf3e6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.709+0000 7f229d3a0740 1 -- 172.21.15.52:0/270765197 shutdown_connections 2024-04-02T14:00:40.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.709+0000 7f229d3a0740 1 -- 172.21.15.52:0/270765197 wait complete. 2024-04-02T14:00:40.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.709+0000 7f229d3a0740 20 client.26828 trim_cache size 0 max 16384 2024-04-02T14:00:40.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.722+0000 7f229d3a0740 1 Processor -- start 2024-04-02T14:00:40.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.722+0000 7f229d3a0740 1 -- start start 2024-04-02T14:00:40.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.722+0000 7f229d3a0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2bf3ea7a0 0x55a2bf3f3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.723+0000 7f229d3a0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a2bf3eb0c0 0x55a2bf40bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.723+0000 7f229d3a0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3f3ca0 0x55a2bf40c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.723+0000 7f229d3a0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f22941b1170 con 0x55a2bf3eb0c0 2024-04-02T14:00:40.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.723+0000 7f229d3a0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f22941b2cb0 con 0x55a2bf3f3ca0 2024-04-02T14:00:40.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.723+0000 7f229d3a0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f22941b1840 con 0x55a2bf3ea7a0 2024-04-02T14:00:40.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.723+0000 7f22937fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a2bf3eb0c0 0x55a2bf40bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.723+0000 7f22937fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a2bf3eb0c0 0x55a2bf40bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34162/0 (socket says 172.21.15.52:34162) 2024-04-02T14:00:40.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.723+0000 7f22937fe640 1 -- 172.21.15.52:0/1399237380 learned_addr learned my addr 172.21.15.52:0/1399237380 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:40.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.723+0000 7f2293fff640 1 --2- 172.21.15.52:0/1399237380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2bf3ea7a0 0x55a2bf3f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.723+0000 7f2298c76640 1 --2- 172.21.15.52:0/1399237380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3f3ca0 0x55a2bf40c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.723+0000 7f22937fe640 1 -- 172.21.15.52:0/1399237380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2bf3ea7a0 msgr2=0x55a2bf3f3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.723+0000 7f22937fe640 1 --2- 172.21.15.52:0/1399237380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2bf3ea7a0 0x55a2bf3f3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.723+0000 7f22937fe640 1 -- 172.21.15.52:0/1399237380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3f3ca0 msgr2=0x55a2bf40c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.723+0000 7f22937fe640 1 --2- 172.21.15.52:0/1399237380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3f3ca0 0x55a2bf40c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.723+0000 7f22937fe640 1 -- 172.21.15.52:0/1399237380 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2bf406870 con 0x55a2bf3eb0c0 2024-04-02T14:00:40.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.723+0000 7f2293fff640 1 --2- 172.21.15.52:0/1399237380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2bf3ea7a0 0x55a2bf3f3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:00:40.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.723+0000 7f2298c76640 1 --2- 172.21.15.52:0/1399237380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3f3ca0 0x55a2bf40c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:00:40.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.723+0000 7f22937fe640 1 --2- 172.21.15.52:0/1399237380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a2bf3eb0c0 0x55a2bf40bb10 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f22840bc740 tx=0x7f2284003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5547569da7f30019 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.723+0000 7f2279ffb640 1 -- 172.21.15.52:0/1399237380 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228404b040 con 0x55a2bf3eb0c0 2024-04-02T14:00:40.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.723+0000 7f2279ffb640 1 -- 172.21.15.52:0/1399237380 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22840c2790 con 0x55a2bf3eb0c0 2024-04-02T14:00:40.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.723+0000 7f2279ffb640 1 -- 172.21.15.52:0/1399237380 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22840bc790 con 0x55a2bf3eb0c0 2024-04-02T14:00:40.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.724+0000 7f229d3a0740 1 -- 172.21.15.52:0/1399237380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a2bf3eb0c0 msgr2=0x55a2bf40bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.724+0000 7f229d3a0740 1 --2- 172.21.15.52:0/1399237380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a2bf3eb0c0 0x55a2bf40bb10 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f22840bc740 tx=0x7f2284003760 comp rx=0 tx=0).stop 2024-04-02T14:00:40.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.724+0000 7f229d3a0740 1 -- 172.21.15.52:0/1399237380 shutdown_connections 2024-04-02T14:00:40.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.724+0000 7f229d3a0740 1 --2- 172.21.15.52:0/1399237380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3f3ca0 0x55a2bf40c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.724+0000 7f229d3a0740 1 --2- 172.21.15.52:0/1399237380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a2bf3eb0c0 0x55a2bf40bb10 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.724+0000 7f229d3a0740 1 --2- 172.21.15.52:0/1399237380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2bf3ea7a0 0x55a2bf3f3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.724+0000 7f229d3a0740 1 -- 172.21.15.52:0/1399237380 >> 172.21.15.52:0/1399237380 conn(0x55a2bf3e4ed0 msgr2=0x55a2bf3e3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.724+0000 7f229d3a0740 1 -- 172.21.15.52:0/1399237380 shutdown_connections 2024-04-02T14:00:40.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.724+0000 7f229d3a0740 1 -- 172.21.15.52:0/1399237380 wait complete. 2024-04-02T14:00:40.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.724+0000 7f229d3a0740 1 -- start start 2024-04-02T14:00:40.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.725+0000 7f229d3a0740 1 Processor -- start 2024-04-02T14:00:40.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.725+0000 7f229d3a0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2bf3ea7a0 0x55a2bf3fbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.725+0000 7f229d3a0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3eb0c0 0x55a2bf3ed490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.725+0000 7f229d3a0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a2bf3f3ca0 0x55a2bf3ed9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.725+0000 7f229d3a0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f22941b25e0 con 0x55a2bf3f3ca0 2024-04-02T14:00:40.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.725+0000 7f229d3a0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f22941b1f10 con 0x55a2bf3eb0c0 2024-04-02T14:00:40.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.725+0000 7f229d3a0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f22941af170 con 0x55a2bf3ea7a0 2024-04-02T14:00:40.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.725+0000 7f229d3a0740 10 client.0 _ll_register_callbacks cb 0x7f2288009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:40.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.725+0000 7f22937fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3eb0c0 0x55a2bf3ed490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.725+0000 7f2298c76640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a2bf3f3ca0 0x55a2bf3ed9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.725+0000 7f22937fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3eb0c0 0x55a2bf3ed490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:38362/0 (socket says 172.21.15.52:38362) 2024-04-02T14:00:40.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.725+0000 7f22937fe640 1 -- 172.21.15.52:0/3757572190 learned_addr learned my addr 172.21.15.52:0/3757572190 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:40.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.725+0000 7f2298c76640 1 --2- 172.21.15.52:0/3757572190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a2bf3f3ca0 0x55a2bf3ed9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34178/0 (socket says 172.21.15.52:34178) 2024-04-02T14:00:40.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.725+0000 7f2293fff640 1 --2- 172.21.15.52:0/3757572190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2bf3ea7a0 0x55a2bf3fbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.725+0000 7f22937fe640 1 -- 172.21.15.52:0/3757572190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2bf3ea7a0 msgr2=0x55a2bf3fbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.725+0000 7f22937fe640 1 --2- 172.21.15.52:0/3757572190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2bf3ea7a0 0x55a2bf3fbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.725+0000 7f22937fe640 1 -- 172.21.15.52:0/3757572190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a2bf3f3ca0 msgr2=0x55a2bf3ed9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.725+0000 7f22937fe640 1 --2- 172.21.15.52:0/3757572190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a2bf3f3ca0 0x55a2bf3ed9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.725+0000 7f22937fe640 1 -- 172.21.15.52:0/3757572190 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2bf3ee010 con 0x55a2bf3eb0c0 2024-04-02T14:00:40.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.726+0000 7f22937fe640 1 --2- 172.21.15.52:0/3757572190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3eb0c0 0x55a2bf3ed490 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2284009430 tx=0x7f22840bdfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebe8aba2c1198897 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.726+0000 7f22737fe640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-02T14:00:40.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.726+0000 7f22737fe640 1 -- 172.21.15.52:0/3757572190 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228404b070 con 0x55a2bf3eb0c0 2024-04-02T14:00:40.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.726+0000 7f229d3a0740 1 -- 172.21.15.52:0/3757572190 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22840c2790 con 0x55a2bf3eb0c0 2024-04-02T14:00:40.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.726+0000 7f22737fe640 1 -- 172.21.15.52:0/3757572190 <== mon.2 v2:172.21.15.88:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f228402cab0 con 0x55a2bf3eb0c0 2024-04-02T14:00:40.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.726+0000 7f22737fe640 1 -- 172.21.15.52:0/3757572190 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f22840a7040 con 0x55a2bf3eb0c0 2024-04-02T14:00:40.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.726+0000 7f229d3a0740 10 client.26834 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:40.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.726+0000 7f229d3a0740 1 -- 172.21.15.52:0/3757572190 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a2bf46a1b0 con 0x55a2bf3eb0c0 2024-04-02T14:00:40.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.726+0000 7f22737fe640 1 -- 172.21.15.52:0/3757572190 <== mon.2 v2:172.21.15.88:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22840c2790 con 0x55a2bf3eb0c0 2024-04-02T14:00:40.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.726+0000 7f22737fe640 1 -- 172.21.15.52:0/3757572190 <== mon.2 v2:172.21.15.88:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f22840efc70 con 0x55a2bf3eb0c0 2024-04-02T14:00:40.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.726+0000 7f229d3a0740 10 client.26834 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:40.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.726+0000 7f229d3a0740 10 client.26834 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T14:00:40.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.726+0000 7f229d3a0740 20 client.26834 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:40.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.726+0000 7f229d3a0740 1 --2- 172.21.15.52:0/3757572190 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55a2bf46a4f0 0x55a2bf46c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.726+0000 7f229d3a0740 4 client.26834 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:00:40.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.726+0000 7f229d3a0740 1 -- 172.21.15.52:0/3757572190 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a2bf2d36b0 con 0x55a2bf46a4f0 2024-04-02T14:00:40.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.726+0000 7f2293fff640 1 --2- 172.21.15.52:0/3757572190 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55a2bf46a4f0 0x55a2bf46c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.727+0000 7f22737fe640 10 client.26834 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T14:00:40.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.727+0000 7f2293fff640 1 --2- 172.21.15.52:0/3757572190 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55a2bf46a4f0 0x55a2bf46c910 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5002042328b473c8 server_cookie=663444584a167a84 in_seq=0 out_seq=0 2024-04-02T14:00:40.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.727+0000 7f22737fe640 1 -- 172.21.15.52:0/3757572190 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a2bf2d36b0 con 0x55a2bf46a4f0 2024-04-02T14:00:40.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.727+0000 7f22737fe640 10 client.26834 handle_command_reply: tid=0 2024-04-02T14:00:40.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.727+0000 7f229d3a0740 1 client.26834 shutdown 2024-04-02T14:00:40.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.727+0000 7f229d3a0740 1 -- 172.21.15.52:0/3757572190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3eb0c0 msgr2=0x55a2bf3ed490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.727+0000 7f229d3a0740 1 --2- 172.21.15.52:0/3757572190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3eb0c0 0x55a2bf3ed490 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2284009430 tx=0x7f22840bdfd0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.727+0000 7f229d3a0740 1 -- 172.21.15.52:0/3757572190 shutdown_connections 2024-04-02T14:00:40.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.727+0000 7f229d3a0740 1 --2- 172.21.15.52:0/3757572190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a2bf3f3ca0 0x55a2bf3ed9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.727+0000 7f229d3a0740 1 --2- 172.21.15.52:0/3757572190 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55a2bf46a4f0 0x55a2bf46c910 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.727+0000 7f229d3a0740 1 --2- 172.21.15.52:0/3757572190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a2bf3eb0c0 0x55a2bf3ed490 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.727+0000 7f22737fe640 0 client.26834 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T14:00:40.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.727+0000 7f229d3a0740 1 --2- 172.21.15.52:0/3757572190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2bf3ea7a0 0x55a2bf3fbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.727+0000 7f229d3a0740 1 -- 172.21.15.52:0/3757572190 >> 172.21.15.52:0/3757572190 conn(0x55a2bf3e4ed0 msgr2=0x55a2bf3f0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.727+0000 7f229d3a0740 1 -- 172.21.15.52:0/3757572190 shutdown_connections 2024-04-02T14:00:40.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.727+0000 7f229d3a0740 1 -- 172.21.15.52:0/3757572190 wait complete. 2024-04-02T14:00:40.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.727+0000 7f229d3a0740 20 client.26834 trim_cache size 0 max 16384 2024-04-02T14:00:40.741 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:00:40.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.728+0000 7f229a6d9640 1 -- 172.21.15.52:0/3645770680 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f227c0a3d20 msgr2=0x7f227c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.728+0000 7f229a6d9640 1 --2- 172.21.15.52:0/3645770680 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f227c0a3d20 0x7f227c0a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f228c005fd0 tx=0x7f228c009f10 comp rx=0 tx=0).stop 2024-04-02T14:00:40.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.728+0000 7f229a6d9640 1 -- 172.21.15.52:0/3645770680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22940696c0 msgr2=0x7f2294100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.728+0000 7f229a6d9640 1 --2- 172.21.15.52:0/3645770680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22940696c0 0x7f2294100f00 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f2284009230 tx=0x7f2284004270 comp rx=0 tx=0).stop 2024-04-02T14:00:40.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.728+0000 7f229a6d9640 1 -- 172.21.15.52:0/3645770680 shutdown_connections 2024-04-02T14:00:40.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.728+0000 7f229a6d9640 1 --2- 172.21.15.52:0/3645770680 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f227c0a3d20 0x7f227c0a61e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.728+0000 7f229a6d9640 1 --2- 172.21.15.52:0/3645770680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2294107b50 0x7f2294101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.728+0000 7f229a6d9640 1 --2- 172.21.15.52:0/3645770680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22940696c0 0x7f2294100f00 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.728+0000 7f229a6d9640 1 --2- 172.21.15.52:0/3645770680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2294068d00 0x7f22941009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.728+0000 7f229a6d9640 1 -- 172.21.15.52:0/3645770680 >> 172.21.15.52:0/3645770680 conn(0x7f2294077b00 msgr2=0x7f22940fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.728+0000 7f229a6d9640 1 -- 172.21.15.52:0/3645770680 shutdown_connections 2024-04-02T14:00:40.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.728+0000 7f229a6d9640 1 -- 172.21.15.52:0/3645770680 wait complete. 2024-04-02T14:00:40.744 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:40 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:00:40.744 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:40 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/369407938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:00:40.745 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T14:00:40.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:40 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:00:40.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:40 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/369407938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:00:40.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.849+0000 7f46bc01b640 1 -- 172.21.15.52:0/122920388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46b4106980 msgr2=0x7f46b4106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.849+0000 7f46bc01b640 1 --2- 172.21.15.52:0/122920388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46b4106980 0x7f46b4106d60 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f46b0009930 tx=0x7f46b002bc30 comp rx=0 tx=0).stop 2024-04-02T14:00:40.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.849+0000 7f46bc01b640 1 -- 172.21.15.52:0/122920388 shutdown_connections 2024-04-02T14:00:40.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.849+0000 7f46bc01b640 1 --2- 172.21.15.52:0/122920388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46b4101330 0x7f46b410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.849+0000 7f46bc01b640 1 --2- 172.21.15.52:0/122920388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46b4100970 0x7f46b4100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.849+0000 7f46bc01b640 1 --2- 172.21.15.52:0/122920388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46b4106980 0x7f46b4106d60 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.849+0000 7f46bc01b640 1 -- 172.21.15.52:0/122920388 >> 172.21.15.52:0/122920388 conn(0x7f46b4077b00 msgr2=0x7f46b4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.849+0000 7f46bc01b640 1 -- 172.21.15.52:0/122920388 shutdown_connections 2024-04-02T14:00:40.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.849+0000 7f46bc01b640 1 -- 172.21.15.52:0/122920388 wait complete. 2024-04-02T14:00:40.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.849+0000 7f46bc01b640 1 Processor -- start 2024-04-02T14:00:40.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.849+0000 7f46bc01b640 1 -- start start 2024-04-02T14:00:40.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.850+0000 7f46bc01b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46b4100970 0x7f46b4102f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.850+0000 7f46bc01b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46b4101330 0x7f46b4103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.850+0000 7f46bc01b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46b4106980 0x7f46b41039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.850+0000 7f46bc01b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f46b4110d60 con 0x7f46b4101330 2024-04-02T14:00:40.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.850+0000 7f46bc01b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f46b4110be0 con 0x7f46b4100970 2024-04-02T14:00:40.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.850+0000 7f46bc01b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f46b4110ee0 con 0x7f46b4106980 2024-04-02T14:00:40.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.850+0000 7f46b95b6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46b4101330 0x7f46b4103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.850+0000 7f46b95b6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46b4101330 0x7f46b4103480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34188/0 (socket says 172.21.15.52:34188) 2024-04-02T14:00:40.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.850+0000 7f46b95b6640 1 -- 172.21.15.52:0/3914916971 learned_addr learned my addr 172.21.15.52:0/3914916971 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:40.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.850+0000 7f46b9db7640 1 --2- 172.21.15.52:0/3914916971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46b4100970 0x7f46b4102f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.850+0000 7f46ba5b8640 1 --2- 172.21.15.52:0/3914916971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46b4106980 0x7f46b41039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.850+0000 7f46b95b6640 1 -- 172.21.15.52:0/3914916971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46b4106980 msgr2=0x7f46b41039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.850+0000 7f46b95b6640 1 --2- 172.21.15.52:0/3914916971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46b4106980 0x7f46b41039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.850+0000 7f46b95b6640 1 -- 172.21.15.52:0/3914916971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46b4100970 msgr2=0x7f46b4102f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.850+0000 7f46b95b6640 1 --2- 172.21.15.52:0/3914916971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46b4100970 0x7f46b4102f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.850+0000 7f46b95b6640 1 -- 172.21.15.52:0/3914916971 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b41a68b0 con 0x7f46b4101330 2024-04-02T14:00:40.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.850+0000 7f46b95b6640 1 --2- 172.21.15.52:0/3914916971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46b4101330 0x7f46b4103480 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f46a400e9f0 tx=0x7f46a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.850+0000 7f46aaffd640 1 -- 172.21.15.52:0/3914916971 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a400cd80 con 0x7f46b4101330 2024-04-02T14:00:40.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.850+0000 7f46aaffd640 1 -- 172.21.15.52:0/3914916971 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46a4010910 con 0x7f46b4101330 2024-04-02T14:00:40.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.850+0000 7f46aaffd640 1 -- 172.21.15.52:0/3914916971 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4018990 con 0x7f46b4101330 2024-04-02T14:00:40.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.850+0000 7f46bc01b640 1 -- 172.21.15.52:0/3914916971 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b41a6ba0 con 0x7f46b4101330 2024-04-02T14:00:40.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.850+0000 7f46bc01b640 1 -- 172.21.15.52:0/3914916971 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b41a7140 con 0x7f46b4101330 2024-04-02T14:00:40.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.853+0000 7f46aaffd640 1 -- 172.21.15.52:0/3914916971 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f46a4010ac0 con 0x7f46b4101330 2024-04-02T14:00:40.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.853+0000 7f46bec04740 1 Processor -- start 2024-04-02T14:00:40.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.853+0000 7f46bec04740 1 -- start start 2024-04-02T14:00:40.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.853+0000 7f46bec04740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba57a0 0x56053aba5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.853+0000 7f46bec04740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053aba60c0 0x56053abae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.853+0000 7f46aaffd640 1 --2- 172.21.15.52:0/3914916971 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f46a00a3da0 0x7f46a00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.853+0000 7f46bec04740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56053abaeca0 0x56053abb1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.853+0000 7f46bec04740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56053ab097b0 con 0x56053aba57a0 2024-04-02T14:00:40.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.853+0000 7f46bec04740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56053aaa8570 con 0x56053abaeca0 2024-04-02T14:00:40.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.853+0000 7f46bec04740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56053ab05760 con 0x56053aba60c0 2024-04-02T14:00:40.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.853+0000 7f46ba5b8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba57a0 0x56053aba5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.853+0000 7f46b95b6640 1 --2- 172.21.15.52:0/3914916971 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f46a00a3da0 0x7f46a00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.854+0000 7f46ba5b8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba57a0 0x56053aba5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34198/0 (socket says 172.21.15.52:34198) 2024-04-02T14:00:40.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.854+0000 7f46ba5b8640 1 -- 172.21.15.52:0/3723897113 learned_addr learned my addr 172.21.15.52:0/3723897113 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:40.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.854+0000 7f46b9db7640 1 --2- 172.21.15.52:0/3723897113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053aba60c0 0x56053abae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.854+0000 7f46ba5b8640 1 --2- 172.21.15.52:0/3723897113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56053abaeca0 0x56053abb1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.854+0000 7f46aaffd640 1 -- 172.21.15.52:0/3914916971 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f46a40d0e10 con 0x7f46b4101330 2024-04-02T14:00:40.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.854+0000 7f46ba5b8640 1 -- 172.21.15.52:0/3723897113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053aba60c0 msgr2=0x56053abae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.854+0000 7f46ba5b8640 1 --2- 172.21.15.52:0/3723897113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053aba60c0 0x56053abae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.854+0000 7f46ba5b8640 1 -- 172.21.15.52:0/3723897113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56053abaeca0 msgr2=0x56053abb1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.854+0000 7f46ba5b8640 1 --2- 172.21.15.52:0/3723897113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56053abaeca0 0x56053abb1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.854+0000 7f46ba5b8640 1 -- 172.21.15.52:0/3723897113 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56053a81d3f0 con 0x56053aba57a0 2024-04-02T14:00:40.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.854+0000 7f46ba5b8640 1 --2- 172.21.15.52:0/3723897113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba57a0 0x56053aba5b80 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f46ac013ad0 tx=0x7f46ac005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b530966a3f720f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.854+0000 7f46b95b6640 1 --2- 172.21.15.52:0/3914916971 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f46a00a3da0 0x7f46a00a6260 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f46a40ce280 tx=0x7f46a406f680 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.854+0000 7f469ffff640 1 -- 172.21.15.52:0/3723897113 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac013130 con 0x56053aba57a0 2024-04-02T14:00:40.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.854+0000 7f469ffff640 1 -- 172.21.15.52:0/3723897113 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46ac00f940 con 0x56053aba57a0 2024-04-02T14:00:40.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.854+0000 7f469ffff640 1 -- 172.21.15.52:0/3723897113 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac005310 con 0x56053aba57a0 2024-04-02T14:00:40.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.855+0000 7f46bec04740 1 -- 172.21.15.52:0/3723897113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba57a0 msgr2=0x56053aba5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.855+0000 7f46bec04740 1 --2- 172.21.15.52:0/3723897113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba57a0 0x56053aba5b80 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f46ac013ad0 tx=0x7f46ac005bd0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.855+0000 7f46bec04740 1 -- 172.21.15.52:0/3723897113 shutdown_connections 2024-04-02T14:00:40.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.855+0000 7f46bec04740 1 --2- 172.21.15.52:0/3723897113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56053abaeca0 0x56053abb1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.855+0000 7f46bec04740 1 --2- 172.21.15.52:0/3723897113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053aba60c0 0x56053abae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.855+0000 7f46bec04740 1 --2- 172.21.15.52:0/3723897113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba57a0 0x56053aba5b80 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.855+0000 7f46bec04740 1 -- 172.21.15.52:0/3723897113 >> 172.21.15.52:0/3723897113 conn(0x56053ab9fed0 msgr2=0x56053aba02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.855+0000 7f46bec04740 1 -- 172.21.15.52:0/3723897113 shutdown_connections 2024-04-02T14:00:40.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.855+0000 7f46bec04740 1 -- 172.21.15.52:0/3723897113 wait complete. 2024-04-02T14:00:40.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.855+0000 7f46bec04740 1 -- start start 2024-04-02T14:00:40.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.855+0000 7f46bec04740 1 Processor -- start 2024-04-02T14:00:40.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.856+0000 7f46bec04740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba57a0 0x56053abae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.856+0000 7f46bec04740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56053aba60c0 0x56053ababa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.856+0000 7f46bec04740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053abaeca0 0x56053abac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.856+0000 7f46bec04740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56053abb1a30 con 0x56053aba57a0 2024-04-02T14:00:40.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.856+0000 7f46bec04740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56053aac4e80 con 0x56053aba60c0 2024-04-02T14:00:40.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.856+0000 7f46bec04740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56053abb1bb0 con 0x56053abaeca0 2024-04-02T14:00:40.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.856+0000 7f46bec04740 10 client.0 _ll_register_callbacks cb 0x56053ab00aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:40.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.856+0000 7f46ba5b8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba57a0 0x56053abae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.856+0000 7f46b9db7640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56053aba60c0 0x56053ababa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.856+0000 7f46ba5b8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba57a0 0x56053abae6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34204/0 (socket says 172.21.15.52:34204) 2024-04-02T14:00:40.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.856+0000 7f46ba5b8640 1 -- 172.21.15.52:0/1765779591 learned_addr learned my addr 172.21.15.52:0/1765779591 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:40.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.856+0000 7f46ba5b8640 1 --2- 172.21.15.52:0/1765779591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053abaeca0 0x56053abac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.856+0000 7f46b9db7640 1 -- 172.21.15.52:0/1765779591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053abaeca0 msgr2=0x56053abac1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.856+0000 7f46b9db7640 1 --2- 172.21.15.52:0/1765779591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053abaeca0 0x56053abac1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.856+0000 7f46b9db7640 1 -- 172.21.15.52:0/1765779591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba57a0 msgr2=0x56053abae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.856+0000 7f46b9db7640 1 --2- 172.21.15.52:0/1765779591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba57a0 0x56053abae6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.856+0000 7f46b9db7640 1 -- 172.21.15.52:0/1765779591 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56053a8e77d0 con 0x56053aba60c0 2024-04-02T14:00:40.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.856+0000 7f46ba5b8640 1 --2- 172.21.15.52:0/1765779591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba57a0 0x56053abae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:00:40.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.856+0000 7f46b9db7640 1 --2- 172.21.15.52:0/1765779591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56053aba60c0 0x56053ababa60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f46b0009230 tx=0x7f46b002cdc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e543ee08087c137 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.856+0000 7f469d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-02T14:00:40.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.857+0000 7f469d7fa640 1 -- 172.21.15.52:0/1765779591 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b0004300 con 0x56053aba60c0 2024-04-02T14:00:40.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.857+0000 7f469d7fa640 1 -- 172.21.15.52:0/1765779591 <== mon.2 v2:172.21.15.88:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f46b00046c0 con 0x56053aba60c0 2024-04-02T14:00:40.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.857+0000 7f46bec04740 1 -- 172.21.15.52:0/1765779591 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46ac00f940 con 0x56053aba60c0 2024-04-02T14:00:40.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.857+0000 7f469d7fa640 1 -- 172.21.15.52:0/1765779591 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f46b0051050 con 0x56053aba60c0 2024-04-02T14:00:40.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.857+0000 7f46bec04740 10 client.26846 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:40.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.857+0000 7f46bec04740 1 -- 172.21.15.52:0/1765779591 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56053abbdad0 con 0x56053aba60c0 2024-04-02T14:00:40.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.857+0000 7f469d7fa640 1 -- 172.21.15.52:0/1765779591 <== mon.2 v2:172.21.15.88:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46ac00f940 con 0x56053aba60c0 2024-04-02T14:00:40.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.857+0000 7f469d7fa640 1 -- 172.21.15.52:0/1765779591 <== mon.2 v2:172.21.15.88:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f46b0005470 con 0x56053aba60c0 2024-04-02T14:00:40.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.857+0000 7f46bec04740 10 client.26846 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:40.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.857+0000 7f46bec04740 10 client.26846 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T14:00:40.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.857+0000 7f46bec04740 20 client.26846 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:40.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.857+0000 7f46bec04740 1 --2- 172.21.15.52:0/1765779591 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x56053abbef90 0x56053abc1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.857+0000 7f46bec04740 4 client.26846 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:00:40.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.857+0000 7f46bec04740 1 -- 172.21.15.52:0/1765779591 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56053ab16cf0 con 0x56053abbef90 2024-04-02T14:00:40.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.857+0000 7f46ba5b8640 1 --2- 172.21.15.52:0/1765779591 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x56053abbef90 0x56053abc1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.858+0000 7f46ba5b8640 1 --2- 172.21.15.52:0/1765779591 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x56053abbef90 0x56053abc1380 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ce4a7e73c040ca6 server_cookie=9a0bbf360a40512a in_seq=0 out_seq=0 2024-04-02T14:00:40.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.858+0000 7f469d7fa640 10 client.26846 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T14:00:40.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.859+0000 7f469d7fa640 1 -- 172.21.15.52:0/1765779591 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56053ab16cf0 con 0x56053abbef90 2024-04-02T14:00:40.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.859+0000 7f469d7fa640 10 client.26846 handle_command_reply: tid=0 2024-04-02T14:00:40.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.859+0000 7f46bec04740 1 client.26846 shutdown 2024-04-02T14:00:40.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.859+0000 7f46bec04740 1 -- 172.21.15.52:0/1765779591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56053aba60c0 msgr2=0x56053ababa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.859+0000 7f46bec04740 1 --2- 172.21.15.52:0/1765779591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56053aba60c0 0x56053ababa60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f46b0009230 tx=0x7f46b002cdc0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.859+0000 7f46bec04740 1 -- 172.21.15.52:0/1765779591 shutdown_connections 2024-04-02T14:00:40.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.859+0000 7f46bec04740 1 --2- 172.21.15.52:0/1765779591 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x56053abbef90 0x56053abc1380 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.859+0000 7f46bec04740 1 --2- 172.21.15.52:0/1765779591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053abaeca0 0x56053abac1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.859+0000 7f46bec04740 1 --2- 172.21.15.52:0/1765779591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56053aba60c0 0x56053ababa60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.859+0000 7f469d7fa640 0 client.26846 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T14:00:40.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.859+0000 7f46bec04740 1 --2- 172.21.15.52:0/1765779591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba57a0 0x56053abae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.859+0000 7f46bec04740 1 -- 172.21.15.52:0/1765779591 >> 172.21.15.52:0/1765779591 conn(0x56053ab9fed0 msgr2=0x56053aba1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.859+0000 7f46bec04740 1 -- 172.21.15.52:0/1765779591 shutdown_connections 2024-04-02T14:00:40.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.859+0000 7f46bec04740 1 -- 172.21.15.52:0/1765779591 wait complete. 2024-04-02T14:00:40.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.859+0000 7f46bec04740 20 client.26846 trim_cache size 0 max 16384 2024-04-02T14:00:40.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.875+0000 7f46bec04740 1 Processor -- start 2024-04-02T14:00:40.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.875+0000 7f46bec04740 1 -- start start 2024-04-02T14:00:40.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.875+0000 7f46bec04740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba57a0 0x56053abb1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.875+0000 7f46bec04740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56053aba60c0 0x56053abba920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.875+0000 7f46bec04740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053abaeca0 0x56053abc6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.875+0000 7f46bec04740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f46b41b1130 con 0x56053aba57a0 2024-04-02T14:00:40.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.875+0000 7f46bec04740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f46b41b2c70 con 0x56053aba60c0 2024-04-02T14:00:40.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.875+0000 7f46bec04740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f46b41b1800 con 0x56053abaeca0 2024-04-02T14:00:40.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.875+0000 7f46ba5b8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba57a0 0x56053abb1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.875+0000 7f46ba5b8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba57a0 0x56053abb1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34212/0 (socket says 172.21.15.52:34212) 2024-04-02T14:00:40.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.875+0000 7f46ba5b8640 1 -- 172.21.15.52:0/2586632217 learned_addr learned my addr 172.21.15.52:0/2586632217 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:40.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.875+0000 7f46b9db7640 1 --2- 172.21.15.52:0/2586632217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56053aba60c0 0x56053abba920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.875+0000 7f46ba5b8640 1 --2- 172.21.15.52:0/2586632217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053abaeca0 0x56053abc6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.876+0000 7f46b9db7640 1 -- 172.21.15.52:0/2586632217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053abaeca0 msgr2=0x56053abc6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.876+0000 7f46b9db7640 1 --2- 172.21.15.52:0/2586632217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053abaeca0 0x56053abc6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.876+0000 7f46b9db7640 1 -- 172.21.15.52:0/2586632217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba57a0 msgr2=0x56053abb1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.876+0000 7f46b9db7640 1 --2- 172.21.15.52:0/2586632217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba57a0 0x56053abb1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.876+0000 7f46b9db7640 1 -- 172.21.15.52:0/2586632217 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56053abac510 con 0x56053aba60c0 2024-04-02T14:00:40.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.876+0000 7f46b9db7640 1 --2- 172.21.15.52:0/2586632217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56053aba60c0 0x56053abba920 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f46b002c820 tx=0x7f46b0005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ccb100f3beaf22b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.876+0000 7f469f7fe640 1 -- 172.21.15.52:0/2586632217 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b002cc20 con 0x56053aba60c0 2024-04-02T14:00:40.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.876+0000 7f469f7fe640 1 -- 172.21.15.52:0/2586632217 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46b0009520 con 0x56053aba60c0 2024-04-02T14:00:40.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.876+0000 7f469f7fe640 1 -- 172.21.15.52:0/2586632217 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b0005d50 con 0x56053aba60c0 2024-04-02T14:00:40.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.877+0000 7f46bec04740 1 -- 172.21.15.52:0/2586632217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56053aba60c0 msgr2=0x56053abba920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.877+0000 7f46bec04740 1 --2- 172.21.15.52:0/2586632217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56053aba60c0 0x56053abba920 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f46b002c820 tx=0x7f46b0005a20 comp rx=0 tx=0).stop 2024-04-02T14:00:40.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.877+0000 7f46bec04740 1 -- 172.21.15.52:0/2586632217 shutdown_connections 2024-04-02T14:00:40.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.877+0000 7f46bec04740 1 --2- 172.21.15.52:0/2586632217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053abaeca0 0x56053abc6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.877+0000 7f46bec04740 1 --2- 172.21.15.52:0/2586632217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56053aba60c0 0x56053abba920 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.877+0000 7f46bec04740 1 --2- 172.21.15.52:0/2586632217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba57a0 0x56053abb1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.877+0000 7f46bec04740 1 -- 172.21.15.52:0/2586632217 >> 172.21.15.52:0/2586632217 conn(0x56053ab9fed0 msgr2=0x56053abaa330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.877+0000 7f46bec04740 1 -- 172.21.15.52:0/2586632217 shutdown_connections 2024-04-02T14:00:40.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.877+0000 7f46bec04740 1 -- 172.21.15.52:0/2586632217 wait complete. 2024-04-02T14:00:40.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.877+0000 7f46bec04740 1 -- start start 2024-04-02T14:00:40.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.877+0000 7f46bec04740 1 Processor -- start 2024-04-02T14:00:40.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.878+0000 7f46bec04740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56053aba57a0 0x56053abaf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.878+0000 7f46bec04740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba60c0 0x56053abafc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.878+0000 7f46bec04740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053abaeca0 0x56053abb0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.878+0000 7f46bec04740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f46b41b25a0 con 0x56053aba60c0 2024-04-02T14:00:40.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.878+0000 7f46bec04740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f46b41b1ed0 con 0x56053aba57a0 2024-04-02T14:00:40.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.878+0000 7f46bec04740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f46b41af130 con 0x56053abaeca0 2024-04-02T14:00:40.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.878+0000 7f46bec04740 10 client.0 _ll_register_callbacks cb 0x7f46b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:40.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.878+0000 7f46b9db7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba60c0 0x56053abafc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.878+0000 7f46b9db7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba60c0 0x56053abafc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34228/0 (socket says 172.21.15.52:34228) 2024-04-02T14:00:40.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.878+0000 7f46b9db7640 1 -- 172.21.15.52:0/3038001225 learned_addr learned my addr 172.21.15.52:0/3038001225 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:40.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.878+0000 7f46ba5b8640 1 --2- 172.21.15.52:0/3038001225 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56053aba57a0 0x56053abaf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.878+0000 7f46ba5b8640 1 --2- 172.21.15.52:0/3038001225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053abaeca0 0x56053abb0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.878+0000 7f46b9db7640 1 -- 172.21.15.52:0/3038001225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053abaeca0 msgr2=0x56053abb0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.878+0000 7f46b9db7640 1 --2- 172.21.15.52:0/3038001225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053abaeca0 0x56053abb0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.878+0000 7f46b9db7640 1 -- 172.21.15.52:0/3038001225 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56053aba57a0 msgr2=0x56053abaf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:40.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.878+0000 7f46b9db7640 1 --2- 172.21.15.52:0/3038001225 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56053aba57a0 0x56053abaf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.878+0000 7f46b9db7640 1 -- 172.21.15.52:0/3038001225 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56053abc16f0 con 0x56053aba60c0 2024-04-02T14:00:40.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.878+0000 7f46b9db7640 1 --2- 172.21.15.52:0/3038001225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba60c0 0x56053abafc40 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f46b002bc70 tx=0x7f46b002c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10022b87bd1d9b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:40.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.878+0000 7f469d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:40.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.879+0000 7f469d7fa640 1 -- 172.21.15.52:0/3038001225 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b0004300 con 0x56053aba60c0 2024-04-02T14:00:40.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.879+0000 7f469d7fa640 1 -- 172.21.15.52:0/3038001225 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f46b00046c0 con 0x56053aba60c0 2024-04-02T14:00:40.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.879+0000 7f46bec04740 1 -- 172.21.15.52:0/3038001225 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46b0009520 con 0x56053aba60c0 2024-04-02T14:00:40.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.879+0000 7f469d7fa640 1 -- 172.21.15.52:0/3038001225 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f46b0067050 con 0x56053aba60c0 2024-04-02T14:00:40.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.879+0000 7f46bec04740 10 client.17670 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:40.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.879+0000 7f46bec04740 1 -- 172.21.15.52:0/3038001225 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56053abb7090 con 0x56053aba60c0 2024-04-02T14:00:40.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.879+0000 7f469d7fa640 1 -- 172.21.15.52:0/3038001225 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46b0009520 con 0x56053aba60c0 2024-04-02T14:00:40.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.879+0000 7f469d7fa640 1 -- 172.21.15.52:0/3038001225 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f46b0005310 con 0x56053aba60c0 2024-04-02T14:00:40.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.879+0000 7f46bec04740 10 client.17670 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:40.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.879+0000 7f46bec04740 10 client.17670 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T14:00:40.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.879+0000 7f46bec04740 20 client.17670 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:40.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.879+0000 7f46bec04740 1 --2- 172.21.15.52:0/3038001225 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x56053ac25480 0x56053ac27870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:40.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.879+0000 7f46bec04740 4 client.17670 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:00:40.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.879+0000 7f46bec04740 1 -- 172.21.15.52:0/3038001225 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56053ab11040 con 0x56053ac25480 2024-04-02T14:00:40.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.880+0000 7f46ba5b8640 1 --2- 172.21.15.52:0/3038001225 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x56053ac25480 0x56053ac27870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:40.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.880+0000 7f469d7fa640 10 client.17670 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T14:00:40.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.880+0000 7f46ba5b8640 1 --2- 172.21.15.52:0/3038001225 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x56053ac25480 0x56053ac27870 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d00a12354a3e09da server_cookie=51bcc89fbb3d38b1 in_seq=0 out_seq=0 2024-04-02T14:00:40.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.880+0000 7f469d7fa640 1 -- 172.21.15.52:0/3038001225 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56053ab11040 con 0x56053ac25480 2024-04-02T14:00:40.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.880+0000 7f469d7fa640 10 client.17670 handle_command_reply: tid=0 2024-04-02T14:00:40.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.880+0000 7f46bec04740 1 client.17670 shutdown 2024-04-02T14:00:40.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.880+0000 7f46bec04740 1 -- 172.21.15.52:0/3038001225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba60c0 msgr2=0x56053abafc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.880+0000 7f46bec04740 1 --2- 172.21.15.52:0/3038001225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba60c0 0x56053abafc40 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f46b002bc70 tx=0x7f46b002c330 comp rx=0 tx=0).stop 2024-04-02T14:00:40.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.880+0000 7f46bec04740 1 -- 172.21.15.52:0/3038001225 shutdown_connections 2024-04-02T14:00:40.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.880+0000 7f46bec04740 1 --2- 172.21.15.52:0/3038001225 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x56053ac25480 0x56053ac27870 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.880+0000 7f46bec04740 1 --2- 172.21.15.52:0/3038001225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053abaeca0 0x56053abb0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.880+0000 7f469d7fa640 0 client.17670 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T14:00:40.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.880+0000 7f46bec04740 1 --2- 172.21.15.52:0/3038001225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56053aba60c0 0x56053abafc40 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.880+0000 7f46bec04740 1 --2- 172.21.15.52:0/3038001225 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56053aba57a0 0x56053abaf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.881+0000 7f46bec04740 1 -- 172.21.15.52:0/3038001225 >> 172.21.15.52:0/3038001225 conn(0x56053ab9fed0 msgr2=0x56053ab9e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.881+0000 7f46bec04740 1 -- 172.21.15.52:0/3038001225 shutdown_connections 2024-04-02T14:00:40.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.881+0000 7f46bec04740 1 -- 172.21.15.52:0/3038001225 wait complete. 2024-04-02T14:00:40.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.881+0000 7f46bec04740 20 client.17670 trim_cache size 0 max 16384 2024-04-02T14:00:40.894 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:00:40.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.881+0000 7f46bc01b640 1 -- 172.21.15.52:0/3914916971 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f46a00a3da0 msgr2=0x7f46a00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.881+0000 7f46bc01b640 1 --2- 172.21.15.52:0/3914916971 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f46a00a3da0 0x7f46a00a6260 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f46a40ce280 tx=0x7f46a406f680 comp rx=0 tx=0).stop 2024-04-02T14:00:40.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.881+0000 7f46bc01b640 1 -- 172.21.15.52:0/3914916971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46b4101330 msgr2=0x7f46b4103480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:40.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.881+0000 7f46bc01b640 1 --2- 172.21.15.52:0/3914916971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46b4101330 0x7f46b4103480 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f46a400e9f0 tx=0x7f46a400eeb0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.881+0000 7f46bc01b640 1 -- 172.21.15.52:0/3914916971 shutdown_connections 2024-04-02T14:00:40.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.881+0000 7f46bc01b640 1 --2- 172.21.15.52:0/3914916971 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f46a00a3da0 0x7f46a00a6260 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.881+0000 7f46bc01b640 1 --2- 172.21.15.52:0/3914916971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46b4106980 0x7f46b41039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.881+0000 7f46bc01b640 1 --2- 172.21.15.52:0/3914916971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46b4101330 0x7f46b4103480 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.881+0000 7f46bc01b640 1 --2- 172.21.15.52:0/3914916971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46b4100970 0x7f46b4102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:40.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.881+0000 7f46bc01b640 1 -- 172.21.15.52:0/3914916971 >> 172.21.15.52:0/3914916971 conn(0x7f46b4077b00 msgr2=0x7f46b4068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:40.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.881+0000 7f46bc01b640 1 -- 172.21.15.52:0/3914916971 shutdown_connections 2024-04-02T14:00:40.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:40.881+0000 7f46bc01b640 1 -- 172.21.15.52:0/3914916971 wait complete. 2024-04-02T14:00:40.897 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T14:00:40.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:40 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:00:40.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:40 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/369407938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:00:41.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.002+0000 7fb1dfa3a640 1 -- 172.21.15.52:0/668791040 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d8068ce0 msgr2=0x7fb1d8069160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:41.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.002+0000 7fb1dfa3a640 1 --2- 172.21.15.52:0/668791040 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d8068ce0 0x7fb1d8069160 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc009930 tx=0x7fb1cc02bc30 comp rx=0 tx=0).stop 2024-04-02T14:00:41.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.002+0000 7fb1dfa3a640 1 -- 172.21.15.52:0/668791040 shutdown_connections 2024-04-02T14:00:41.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.002+0000 7fb1dfa3a640 1 --2- 172.21.15.52:0/668791040 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1d80696a0 0x7fb1d810e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.002+0000 7fb1dfa3a640 1 --2- 172.21.15.52:0/668791040 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d8068ce0 0x7fb1d8069160 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.002+0000 7fb1dfa3a640 1 --2- 172.21.15.52:0/668791040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1d8107a80 0x7fb1d8107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.002+0000 7fb1dfa3a640 1 -- 172.21.15.52:0/668791040 >> 172.21.15.52:0/668791040 conn(0x7fb1d8077b00 msgr2=0x7fb1d8077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:41.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.002+0000 7fb1dfa3a640 1 -- 172.21.15.52:0/668791040 shutdown_connections 2024-04-02T14:00:41.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.002+0000 7fb1dfa3a640 1 -- 172.21.15.52:0/668791040 wait complete. 2024-04-02T14:00:41.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.002+0000 7fb1dfa3a640 1 Processor -- start 2024-04-02T14:00:41.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.002+0000 7fb1dfa3a640 1 -- start start 2024-04-02T14:00:41.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.002+0000 7fb1dfa3a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1d8068ce0 0x7fb1d81009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.002+0000 7fb1dfa3a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1d80696a0 0x7fb1d8100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.002+0000 7fb1dfa3a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d8107a80 0x7fb1d8101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.002+0000 7fb1dfa3a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d8111e90 con 0x7fb1d8107a80 2024-04-02T14:00:41.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.003+0000 7fb1dfa3a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d8111d10 con 0x7fb1d80696a0 2024-04-02T14:00:41.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.003+0000 7fb1dfa3a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d8112010 con 0x7fb1d8068ce0 2024-04-02T14:00:41.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.003+0000 7fb1ddfd7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d8107a80 0x7fb1d8101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.003+0000 7fb1dd7d6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1d8068ce0 0x7fb1d81009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.003+0000 7fb1dd7d6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1d8068ce0 0x7fb1d81009c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.52:39300/0 (socket says 172.21.15.52:39300) 2024-04-02T14:00:41.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.003+0000 7fb1ddfd7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d8107a80 0x7fb1d8101440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34242/0 (socket says 172.21.15.52:34242) 2024-04-02T14:00:41.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.003+0000 7fb1dd7d6640 1 -- 172.21.15.52:0/4136852273 learned_addr learned my addr 172.21.15.52:0/4136852273 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:41.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.003+0000 7fb1dcfd5640 1 --2- 172.21.15.52:0/4136852273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1d80696a0 0x7fb1d8100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.003+0000 7fb1dd7d6640 1 -- 172.21.15.52:0/4136852273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1d80696a0 msgr2=0x7fb1d8100f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:41.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.003+0000 7fb1dd7d6640 1 --2- 172.21.15.52:0/4136852273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1d80696a0 0x7fb1d8100f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.003+0000 7fb1dd7d6640 1 -- 172.21.15.52:0/4136852273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d8107a80 msgr2=0x7fb1d8101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:41.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.003+0000 7fb1dd7d6640 1 --2- 172.21.15.52:0/4136852273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d8107a80 0x7fb1d8101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.003+0000 7fb1dd7d6640 1 -- 172.21.15.52:0/4136852273 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d81a68f0 con 0x7fb1d8068ce0 2024-04-02T14:00:41.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.003+0000 7fb1dcfd5640 1 --2- 172.21.15.52:0/4136852273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1d80696a0 0x7fb1d8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:00:41.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.003+0000 7fb1dd7d6640 1 --2- 172.21.15.52:0/4136852273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1d8068ce0 0x7fb1d81009c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400e9f0 tx=0x7fb1d400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:41.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.003+0000 7fb1ca7fc640 1 -- 172.21.15.52:0/4136852273 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d400ccc0 con 0x7fb1d8068ce0 2024-04-02T14:00:41.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.003+0000 7fb1ca7fc640 1 -- 172.21.15.52:0/4136852273 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1d400ce60 con 0x7fb1d8068ce0 2024-04-02T14:00:41.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.003+0000 7fb1dfa3a640 1 -- 172.21.15.52:0/4136852273 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d81a6be0 con 0x7fb1d8068ce0 2024-04-02T14:00:41.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.003+0000 7fb1ca7fc640 1 -- 172.21.15.52:0/4136852273 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4010690 con 0x7fb1d8068ce0 2024-04-02T14:00:41.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.003+0000 7fb1dfa3a640 1 -- 172.21.15.52:0/4136852273 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d81a7100 con 0x7fb1d8068ce0 2024-04-02T14:00:41.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.005+0000 7fb1ca7fc640 1 -- 172.21.15.52:0/4136852273 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb1d4004ea0 con 0x7fb1d8068ce0 2024-04-02T14:00:41.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.006+0000 7fb1e2619740 1 Processor -- start 2024-04-02T14:00:41.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.006+0000 7fb1e2619740 1 -- start start 2024-04-02T14:00:41.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.006+0000 7fb1ca7fc640 1 --2- 172.21.15.52:0/4136852273 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb1c40a3da0 0x7fb1c40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.006+0000 7fb1e2619740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632961f27a0 0x5632961f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.006+0000 7fb1ddfd7640 1 --2- 172.21.15.52:0/4136852273 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb1c40a3da0 0x7fb1c40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.006+0000 7fb1e2619740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961f30c0 0x5632961fb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.006+0000 7fb1e2619740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5632961fbca0 0x5632961fe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.006+0000 7fb1e2619740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5632961567b0 con 0x5632961f30c0 2024-04-02T14:00:41.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.006+0000 7fb1e2619740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5632960f5570 con 0x5632961fbca0 2024-04-02T14:00:41.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.006+0000 7fb1e2619740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563296152760 con 0x5632961f27a0 2024-04-02T14:00:41.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.007+0000 7fb1ca7fc640 1 -- 172.21.15.52:0/4136852273 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7fb1d40d0f90 con 0x7fb1d8068ce0 2024-04-02T14:00:41.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.007+0000 7fb1dcfd5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961f30c0 0x5632961fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.007+0000 7fb1dcfd5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961f30c0 0x5632961fb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34254/0 (socket says 172.21.15.52:34254) 2024-04-02T14:00:41.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.007+0000 7fb1dcfd5640 1 -- 172.21.15.52:0/703427010 learned_addr learned my addr 172.21.15.52:0/703427010 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:41.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.007+0000 7fb1dd7d6640 1 --2- 172.21.15.52:0/703427010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632961f27a0 0x5632961f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.007+0000 7fb1ddfd7640 1 --2- 172.21.15.52:0/703427010 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5632961fbca0 0x5632961fe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.007+0000 7fb1dd7d6640 1 -- 172.21.15.52:0/703427010 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5632961fbca0 msgr2=0x5632961fe090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:41.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.007+0000 7fb1dd7d6640 1 --2- 172.21.15.52:0/703427010 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5632961fbca0 0x5632961fe090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.007+0000 7fb1dd7d6640 1 -- 172.21.15.52:0/703427010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961f30c0 msgr2=0x5632961fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:41.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.007+0000 7fb1dd7d6640 1 --2- 172.21.15.52:0/703427010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961f30c0 0x5632961fb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.007+0000 7fb1dd7d6640 1 -- 172.21.15.52:0/703427010 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563295e6a3f0 con 0x5632961f27a0 2024-04-02T14:00:41.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.007+0000 7fb1dcfd5640 1 --2- 172.21.15.52:0/703427010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961f30c0 0x5632961fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:00:41.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.007+0000 7fb1ddfd7640 1 --2- 172.21.15.52:0/4136852273 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb1c40a3da0 0x7fb1c40a6260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0005fd0 tx=0x7fb1d0011be0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:41.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.007+0000 7fb1dd7d6640 1 --2- 172.21.15.52:0/703427010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632961f27a0 0x5632961f2b80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40ce2f0 tx=0x7fb1d409c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcc0707d49342fb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:41.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.007+0000 7fb1c37fe640 1 -- 172.21.15.52:0/703427010 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d40a0ef0 con 0x5632961f27a0 2024-04-02T14:00:41.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.007+0000 7fb1c37fe640 1 -- 172.21.15.52:0/703427010 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1d40a57b0 con 0x5632961f27a0 2024-04-02T14:00:41.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.007+0000 7fb1c37fe640 1 -- 172.21.15.52:0/703427010 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d40ce700 con 0x5632961f27a0 2024-04-02T14:00:41.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.008+0000 7fb1e2619740 1 -- 172.21.15.52:0/703427010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632961f27a0 msgr2=0x5632961f2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:41.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.008+0000 7fb1e2619740 1 --2- 172.21.15.52:0/703427010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632961f27a0 0x5632961f2b80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40ce2f0 tx=0x7fb1d409c560 comp rx=0 tx=0).stop 2024-04-02T14:00:41.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.008+0000 7fb1e2619740 1 -- 172.21.15.52:0/703427010 shutdown_connections 2024-04-02T14:00:41.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.008+0000 7fb1e2619740 1 --2- 172.21.15.52:0/703427010 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5632961fbca0 0x5632961fe090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.008+0000 7fb1e2619740 1 --2- 172.21.15.52:0/703427010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961f30c0 0x5632961fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.008+0000 7fb1e2619740 1 --2- 172.21.15.52:0/703427010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632961f27a0 0x5632961f2b80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.008+0000 7fb1e2619740 1 -- 172.21.15.52:0/703427010 >> 172.21.15.52:0/703427010 conn(0x5632961eced0 msgr2=0x5632961ed2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:41.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.008+0000 7fb1e2619740 1 -- 172.21.15.52:0/703427010 shutdown_connections 2024-04-02T14:00:41.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.008+0000 7fb1e2619740 1 -- 172.21.15.52:0/703427010 wait complete. 2024-04-02T14:00:41.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.008+0000 7fb1e2619740 1 -- start start 2024-04-02T14:00:41.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.009+0000 7fb1e2619740 1 Processor -- start 2024-04-02T14:00:41.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.009+0000 7fb1e2619740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632961f27a0 0x5632961f8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.009+0000 7fb1e2619740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5632961f30c0 0x5632961f90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.009+0000 7fb1e2619740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961fbca0 0x5632961f95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.009+0000 7fb1e2619740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5632961fea90 con 0x5632961fbca0 2024-04-02T14:00:41.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.009+0000 7fb1e2619740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563296111e80 con 0x5632961f30c0 2024-04-02T14:00:41.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.009+0000 7fb1e2619740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5632961fec10 con 0x5632961f27a0 2024-04-02T14:00:41.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.009+0000 7fb1e2619740 10 client.0 _ll_register_callbacks cb 0x56329614daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:41.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.009+0000 7fb1ddfd7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961fbca0 0x5632961f95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.009+0000 7fb1ddfd7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961fbca0 0x5632961f95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34260/0 (socket says 172.21.15.52:34260) 2024-04-02T14:00:41.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.009+0000 7fb1ddfd7640 1 -- 172.21.15.52:0/1819549533 learned_addr learned my addr 172.21.15.52:0/1819549533 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:41.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.009+0000 7fb1dd7d6640 1 --2- 172.21.15.52:0/1819549533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632961f27a0 0x5632961f8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.009+0000 7fb1dcfd5640 1 --2- 172.21.15.52:0/1819549533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5632961f30c0 0x5632961f90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.009+0000 7fb1ddfd7640 1 -- 172.21.15.52:0/1819549533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632961f27a0 msgr2=0x5632961f8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:41.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.009+0000 7fb1ddfd7640 1 --2- 172.21.15.52:0/1819549533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632961f27a0 0x5632961f8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.009+0000 7fb1ddfd7640 1 -- 172.21.15.52:0/1819549533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5632961f30c0 msgr2=0x5632961f90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:41.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.009+0000 7fb1ddfd7640 1 --2- 172.21.15.52:0/1819549533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5632961f30c0 0x5632961f90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.009+0000 7fb1ddfd7640 1 -- 172.21.15.52:0/1819549533 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563295f347d0 con 0x5632961fbca0 2024-04-02T14:00:41.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.009+0000 7fb1dcfd5640 1 --2- 172.21.15.52:0/1819549533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5632961f30c0 0x5632961f90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:00:41.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.010+0000 7fb1ddfd7640 1 --2- 172.21.15.52:0/1819549533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961fbca0 0x5632961f95f0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000fef0 tx=0x7fb1d0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb14f1b04bf76ce server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:41.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.010+0000 7fb1c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:41.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.010+0000 7fb1c0ff9640 1 -- 172.21.15.52:0/1819549533 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0024070 con 0x5632961fbca0 2024-04-02T14:00:41.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.010+0000 7fb1c0ff9640 1 -- 172.21.15.52:0/1819549533 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7fb1d0029460 con 0x5632961fbca0 2024-04-02T14:00:41.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.010+0000 7fb1e2619740 1 -- 172.21.15.52:0/1819549533 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1d40a57b0 con 0x5632961fbca0 2024-04-02T14:00:41.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.010+0000 7fb1c0ff9640 1 -- 172.21.15.52:0/1819549533 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb1d0032050 con 0x5632961fbca0 2024-04-02T14:00:41.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.010+0000 7fb1e2619740 10 client.17682 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:41.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.010+0000 7fb1e2619740 1 -- 172.21.15.52:0/1819549533 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56329620ac10 con 0x5632961fbca0 2024-04-02T14:00:41.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.010+0000 7fb1c0ff9640 1 -- 172.21.15.52:0/1819549533 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1d40a57b0 con 0x5632961fbca0 2024-04-02T14:00:41.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.011+0000 7fb1c0ff9640 1 -- 172.21.15.52:0/1819549533 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb1d001d3c0 con 0x5632961fbca0 2024-04-02T14:00:41.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.011+0000 7fb1e2619740 10 client.17682 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:41.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.011+0000 7fb1e2619740 10 client.17682 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T14:00:41.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.011+0000 7fb1e2619740 20 client.17682 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:41.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.011+0000 7fb1e2619740 1 --2- 172.21.15.52:0/1819549533 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x56329620c0d0 0x56329620e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.011+0000 7fb1e2619740 4 client.17682 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:00:41.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.011+0000 7fb1e2619740 1 -- 172.21.15.52:0/1819549533 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563296163cf0 con 0x56329620c0d0 2024-04-02T14:00:41.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.011+0000 7fb1dd7d6640 1 --2- 172.21.15.52:0/1819549533 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x56329620c0d0 0x56329620e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.012+0000 7fb1dd7d6640 1 --2- 172.21.15.52:0/1819549533 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x56329620c0d0 0x56329620e4e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7313a7d2d2913a47 server_cookie=497359a34237a40e in_seq=0 out_seq=0 2024-04-02T14:00:41.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.012+0000 7fb1c0ff9640 10 client.17682 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T14:00:41.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.012+0000 7fb1c0ff9640 1 -- 172.21.15.52:0/1819549533 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563296163cf0 con 0x56329620c0d0 2024-04-02T14:00:41.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.012+0000 7fb1c0ff9640 10 client.17682 handle_command_reply: tid=0 2024-04-02T14:00:41.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.012+0000 7fb1e2619740 1 client.17682 shutdown 2024-04-02T14:00:41.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.012+0000 7fb1e2619740 1 -- 172.21.15.52:0/1819549533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961fbca0 msgr2=0x5632961f95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:41.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.012+0000 7fb1e2619740 1 --2- 172.21.15.52:0/1819549533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961fbca0 0x5632961f95f0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000fef0 tx=0x7fb1d0004ab0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.012+0000 7fb1e2619740 1 -- 172.21.15.52:0/1819549533 shutdown_connections 2024-04-02T14:00:41.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.012+0000 7fb1e2619740 1 --2- 172.21.15.52:0/1819549533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961fbca0 0x5632961f95f0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.012+0000 7fb1e2619740 1 --2- 172.21.15.52:0/1819549533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5632961f30c0 0x5632961f90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.012+0000 7fb1e2619740 1 --2- 172.21.15.52:0/1819549533 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x56329620c0d0 0x56329620e4e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.012+0000 7fb1e2619740 1 --2- 172.21.15.52:0/1819549533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632961f27a0 0x5632961f8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.013+0000 7fb1e2619740 1 -- 172.21.15.52:0/1819549533 >> 172.21.15.52:0/1819549533 conn(0x5632961eced0 msgr2=0x5632961eec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:41.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.013+0000 7fb1c0ff9640 0 client.17682 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T14:00:41.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.013+0000 7fb1e2619740 1 -- 172.21.15.52:0/1819549533 shutdown_connections 2024-04-02T14:00:41.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.013+0000 7fb1e2619740 1 -- 172.21.15.52:0/1819549533 wait complete. 2024-04-02T14:00:41.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.013+0000 7fb1e2619740 20 client.17682 trim_cache size 0 max 16384 2024-04-02T14:00:41.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.027+0000 7fb1e2619740 1 Processor -- start 2024-04-02T14:00:41.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.027+0000 7fb1e2619740 1 -- start start 2024-04-02T14:00:41.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.027+0000 7fb1e2619740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961f27a0 0x5632961fb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.027+0000 7fb1e2619740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5632961f30c0 0x563296207950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.027+0000 7fb1e2619740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632961fbca0 0x563296213db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.027+0000 7fb1e2619740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d81b11b0 con 0x5632961f27a0 2024-04-02T14:00:41.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.027+0000 7fb1e2619740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d81b2cf0 con 0x5632961f30c0 2024-04-02T14:00:41.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.027+0000 7fb1e2619740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d81b1880 con 0x5632961fbca0 2024-04-02T14:00:41.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.028+0000 7fb1dd7d6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961f27a0 0x5632961fb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.028+0000 7fb1dd7d6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961f27a0 0x5632961fb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34268/0 (socket says 172.21.15.52:34268) 2024-04-02T14:00:41.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.028+0000 7fb1dd7d6640 1 -- 172.21.15.52:0/1714137375 learned_addr learned my addr 172.21.15.52:0/1714137375 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:41.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.028+0000 7fb1dcfd5640 1 --2- 172.21.15.52:0/1714137375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5632961f30c0 0x563296207950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.028+0000 7fb1ddfd7640 1 --2- 172.21.15.52:0/1714137375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632961fbca0 0x563296213db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.028+0000 7fb1dd7d6640 1 -- 172.21.15.52:0/1714137375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632961fbca0 msgr2=0x563296213db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:41.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.028+0000 7fb1dd7d6640 1 --2- 172.21.15.52:0/1714137375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632961fbca0 0x563296213db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.028+0000 7fb1dd7d6640 1 -- 172.21.15.52:0/1714137375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5632961f30c0 msgr2=0x563296207950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:41.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.028+0000 7fb1dd7d6640 1 --2- 172.21.15.52:0/1714137375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5632961f30c0 0x563296207950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.028+0000 7fb1dd7d6640 1 -- 172.21.15.52:0/1714137375 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632961f2ef0 con 0x5632961f27a0 2024-04-02T14:00:41.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.028+0000 7fb1dcfd5640 1 --2- 172.21.15.52:0/1714137375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5632961f30c0 0x563296207950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:00:41.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.028+0000 7fb1dd7d6640 1 --2- 172.21.15.52:0/1714137375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961f27a0 0x5632961fb2a0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40a5280 tx=0x7fb1d4098f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4781a63c838c52ef server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:41.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.028+0000 7fb1c2ffd640 1 -- 172.21.15.52:0/1714137375 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d40a0ef0 con 0x5632961f27a0 2024-04-02T14:00:41.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.028+0000 7fb1c2ffd640 1 -- 172.21.15.52:0/1714137375 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1d40a57b0 con 0x5632961f27a0 2024-04-02T14:00:41.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.028+0000 7fb1c2ffd640 1 -- 172.21.15.52:0/1714137375 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4016330 con 0x5632961f27a0 2024-04-02T14:00:41.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.029+0000 7fb1e2619740 1 -- 172.21.15.52:0/1714137375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961f27a0 msgr2=0x5632961fb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:41.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.029+0000 7fb1e2619740 1 --2- 172.21.15.52:0/1714137375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961f27a0 0x5632961fb2a0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40a5280 tx=0x7fb1d4098f10 comp rx=0 tx=0).stop 2024-04-02T14:00:41.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.029+0000 7fb1e2619740 1 -- 172.21.15.52:0/1714137375 shutdown_connections 2024-04-02T14:00:41.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.029+0000 7fb1e2619740 1 --2- 172.21.15.52:0/1714137375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632961fbca0 0x563296213db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.029+0000 7fb1e2619740 1 --2- 172.21.15.52:0/1714137375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5632961f30c0 0x563296207950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.029+0000 7fb1e2619740 1 --2- 172.21.15.52:0/1714137375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961f27a0 0x5632961fb2a0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.029+0000 7fb1e2619740 1 -- 172.21.15.52:0/1714137375 >> 172.21.15.52:0/1714137375 conn(0x5632961eced0 msgr2=0x5632961f3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:41.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.029+0000 7fb1e2619740 1 -- 172.21.15.52:0/1714137375 shutdown_connections 2024-04-02T14:00:41.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.029+0000 7fb1e2619740 1 -- 172.21.15.52:0/1714137375 wait complete. 2024-04-02T14:00:41.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.029+0000 7fb1e2619740 1 -- start start 2024-04-02T14:00:41.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.029+0000 7fb1e2619740 1 Processor -- start 2024-04-02T14:00:41.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.030+0000 7fb1e2619740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632961f27a0 0x563296204050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.030+0000 7fb1e2619740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961f30c0 0x5632961f5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.030+0000 7fb1e2619740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5632961fbca0 0x5632961f5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.030+0000 7fb1e2619740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d81b2620 con 0x5632961f30c0 2024-04-02T14:00:41.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.030+0000 7fb1e2619740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d81b1f50 con 0x5632961fbca0 2024-04-02T14:00:41.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.030+0000 7fb1e2619740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d81af1b0 con 0x5632961f27a0 2024-04-02T14:00:41.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.030+0000 7fb1e2619740 10 client.0 _ll_register_callbacks cb 0x7fb1d000fe00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:41.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.030+0000 7fb1dcfd5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961f30c0 0x5632961f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.030+0000 7fb1dcfd5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961f30c0 0x5632961f5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34276/0 (socket says 172.21.15.52:34276) 2024-04-02T14:00:41.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.030+0000 7fb1dcfd5640 1 -- 172.21.15.52:0/2029262495 learned_addr learned my addr 172.21.15.52:0/2029262495 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:41.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.030+0000 7fb1ddfd7640 1 --2- 172.21.15.52:0/2029262495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5632961fbca0 0x5632961f5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.030+0000 7fb1dd7d6640 1 --2- 172.21.15.52:0/2029262495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632961f27a0 0x563296204050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.030+0000 7fb1dcfd5640 1 -- 172.21.15.52:0/2029262495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632961f27a0 msgr2=0x563296204050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:41.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.030+0000 7fb1dcfd5640 1 --2- 172.21.15.52:0/2029262495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632961f27a0 0x563296204050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.030+0000 7fb1dcfd5640 1 -- 172.21.15.52:0/2029262495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5632961fbca0 msgr2=0x5632961f5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:41.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.030+0000 7fb1dcfd5640 1 --2- 172.21.15.52:0/2029262495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5632961fbca0 0x5632961f5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.030+0000 7fb1dcfd5640 1 -- 172.21.15.52:0/2029262495 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632961f5df0 con 0x5632961f30c0 2024-04-02T14:00:41.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.030+0000 7fb1dcfd5640 1 --2- 172.21.15.52:0/2029262495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961f30c0 0x5632961f5340 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc02c170 tx=0x7fb1cc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c074063303f1f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:41.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.030+0000 7fb1c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:41.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.031+0000 7fb1c0ff9640 1 -- 172.21.15.52:0/2029262495 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc02c6b0 con 0x5632961f30c0 2024-04-02T14:00:41.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.031+0000 7fb1c0ff9640 1 -- 172.21.15.52:0/2029262495 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7fb1cc005030 con 0x5632961f30c0 2024-04-02T14:00:41.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.031+0000 7fb1e2619740 1 -- 172.21.15.52:0/2029262495 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1d40a57b0 con 0x5632961f30c0 2024-04-02T14:00:41.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.031+0000 7fb1c0ff9640 1 -- 172.21.15.52:0/2029262495 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb1cc051050 con 0x5632961f30c0 2024-04-02T14:00:41.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.031+0000 7fb1e2619740 10 client.17688 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:41.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.031+0000 7fb1e2619740 1 -- 172.21.15.52:0/2029262495 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5632962723a0 con 0x5632961f30c0 2024-04-02T14:00:41.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.031+0000 7fb1c0ff9640 1 -- 172.21.15.52:0/2029262495 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1d40a57b0 con 0x5632961f30c0 2024-04-02T14:00:41.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.031+0000 7fb1c0ff9640 1 -- 172.21.15.52:0/2029262495 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb1cc0035f0 con 0x5632961f30c0 2024-04-02T14:00:41.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.031+0000 7fb1e2619740 10 client.17688 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:41.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.031+0000 7fb1e2619740 10 client.17688 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T14:00:41.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.031+0000 7fb1e2619740 20 client.17688 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:41.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.031+0000 7fb1e2619740 1 --2- 172.21.15.52:0/2029262495 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x563296272b50 0x563296274f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.031+0000 7fb1e2619740 4 client.17688 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:00:41.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.031+0000 7fb1e2619740 1 -- 172.21.15.52:0/2029262495 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56329615e040 con 0x563296272b50 2024-04-02T14:00:41.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.032+0000 7fb1dd7d6640 1 --2- 172.21.15.52:0/2029262495 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x563296272b50 0x563296274f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.032+0000 7fb1dd7d6640 1 --2- 172.21.15.52:0/2029262495 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x563296272b50 0x563296274f70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4ffa4a7bb27812d5 server_cookie=6f74c22f77202acd in_seq=0 out_seq=0 2024-04-02T14:00:41.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.032+0000 7fb1c0ff9640 10 client.17688 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T14:00:41.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.032+0000 7fb1c0ff9640 1 -- 172.21.15.52:0/2029262495 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56329615e040 con 0x563296272b50 2024-04-02T14:00:41.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.032+0000 7fb1c0ff9640 10 client.17688 handle_command_reply: tid=0 2024-04-02T14:00:41.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.032+0000 7fb1e2619740 1 client.17688 shutdown 2024-04-02T14:00:41.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.032+0000 7fb1e2619740 1 -- 172.21.15.52:0/2029262495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961f30c0 msgr2=0x5632961f5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:41.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.032+0000 7fb1e2619740 1 --2- 172.21.15.52:0/2029262495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961f30c0 0x5632961f5340 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc02c170 tx=0x7fb1cc004270 comp rx=0 tx=0).stop 2024-04-02T14:00:41.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.032+0000 7fb1e2619740 1 -- 172.21.15.52:0/2029262495 shutdown_connections 2024-04-02T14:00:41.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.032+0000 7fb1e2619740 1 --2- 172.21.15.52:0/2029262495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5632961fbca0 0x5632961f5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.032+0000 7fb1e2619740 1 --2- 172.21.15.52:0/2029262495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5632961f30c0 0x5632961f5340 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.032+0000 7fb1e2619740 1 --2- 172.21.15.52:0/2029262495 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x563296272b50 0x563296274f70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.032+0000 7fb1e2619740 1 --2- 172.21.15.52:0/2029262495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632961f27a0 0x563296204050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.032+0000 7fb1e2619740 1 -- 172.21.15.52:0/2029262495 >> 172.21.15.52:0/2029262495 conn(0x5632961eced0 msgr2=0x5632961eb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:41.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.032+0000 7fb1c0ff9640 0 client.17688 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T14:00:41.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.032+0000 7fb1e2619740 1 -- 172.21.15.52:0/2029262495 shutdown_connections 2024-04-02T14:00:41.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.032+0000 7fb1e2619740 1 -- 172.21.15.52:0/2029262495 wait complete. 2024-04-02T14:00:41.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.032+0000 7fb1e2619740 20 client.17688 trim_cache size 0 max 16384 2024-04-02T14:00:41.046 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:00:41.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.033+0000 7fb1dfa3a640 1 -- 172.21.15.52:0/4136852273 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb1c40a3da0 msgr2=0x7fb1c40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:41.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.033+0000 7fb1dfa3a640 1 --2- 172.21.15.52:0/4136852273 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb1c40a3da0 0x7fb1c40a6260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0005fd0 tx=0x7fb1d0011be0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.033+0000 7fb1dfa3a640 1 -- 172.21.15.52:0/4136852273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1d8068ce0 msgr2=0x7fb1d81009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:41.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.033+0000 7fb1dfa3a640 1 --2- 172.21.15.52:0/4136852273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1d8068ce0 0x7fb1d81009c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400e9f0 tx=0x7fb1d400edf0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.033+0000 7fb1dfa3a640 1 -- 172.21.15.52:0/4136852273 shutdown_connections 2024-04-02T14:00:41.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.033+0000 7fb1dfa3a640 1 --2- 172.21.15.52:0/4136852273 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb1c40a3da0 0x7fb1c40a6260 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.033+0000 7fb1dfa3a640 1 --2- 172.21.15.52:0/4136852273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d8107a80 0x7fb1d8101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.033+0000 7fb1dfa3a640 1 --2- 172.21.15.52:0/4136852273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1d80696a0 0x7fb1d8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.033+0000 7fb1dfa3a640 1 --2- 172.21.15.52:0/4136852273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1d8068ce0 0x7fb1d81009c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.033+0000 7fb1dfa3a640 1 -- 172.21.15.52:0/4136852273 >> 172.21.15.52:0/4136852273 conn(0x7fb1d8077b00 msgr2=0x7fb1d80fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:41.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.033+0000 7fb1dfa3a640 1 -- 172.21.15.52:0/4136852273 shutdown_connections 2024-04-02T14:00:41.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.033+0000 7fb1dfa3a640 1 -- 172.21.15.52:0/4136852273 wait complete. 2024-04-02T14:00:41.049 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T14:00:41.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.151+0000 7fb15e00d640 1 -- 172.21.15.52:0/3512662437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1580696c0 msgr2=0x7fb15810e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:41.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.151+0000 7fb15e00d640 1 --2- 172.21.15.52:0/3512662437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1580696c0 0x7fb15810e960 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fb150009cd0 tx=0x7fb15002c0f0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.152+0000 7fb15e00d640 1 -- 172.21.15.52:0/3512662437 shutdown_connections 2024-04-02T14:00:41.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.152+0000 7fb15e00d640 1 --2- 172.21.15.52:0/3512662437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1580696c0 0x7fb15810e960 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.152+0000 7fb15e00d640 1 --2- 172.21.15.52:0/3512662437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb158068d00 0x7fb158069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.152+0000 7fb15e00d640 1 --2- 172.21.15.52:0/3512662437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb158107b50 0x7fb158107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.152+0000 7fb15e00d640 1 -- 172.21.15.52:0/3512662437 >> 172.21.15.52:0/3512662437 conn(0x7fb158077b00 msgr2=0x7fb158077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:41.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.152+0000 7fb15e00d640 1 -- 172.21.15.52:0/3512662437 shutdown_connections 2024-04-02T14:00:41.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.152+0000 7fb15e00d640 1 -- 172.21.15.52:0/3512662437 wait complete. 2024-04-02T14:00:41.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.152+0000 7fb15e00d640 1 Processor -- start 2024-04-02T14:00:41.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.152+0000 7fb15e00d640 1 -- start start 2024-04-02T14:00:41.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.152+0000 7fb15e00d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb158068d00 0x7fb158198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.152+0000 7fb15e00d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1580696c0 0x7fb1581988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.152+0000 7fb15e00d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb158107b50 0x7fb15819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.152+0000 7fb15e00d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb158111e70 con 0x7fb158107b50 2024-04-02T14:00:41.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.152+0000 7fb15e00d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb158111cf0 con 0x7fb158068d00 2024-04-02T14:00:41.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.152+0000 7fb15e00d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb158111ff0 con 0x7fb1580696c0 2024-04-02T14:00:41.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.153+0000 7fb157fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb158107b50 0x7fb15819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.153+0000 7fb157fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb158107b50 0x7fb15819cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34292/0 (socket says 172.21.15.52:34292) 2024-04-02T14:00:41.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.153+0000 7fb157fff640 1 -- 172.21.15.52:0/2062797560 learned_addr learned my addr 172.21.15.52:0/2062797560 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:41.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.153+0000 7fb1577fe640 1 --2- 172.21.15.52:0/2062797560 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb158068d00 0x7fb158198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.153+0000 7fb156ffd640 1 --2- 172.21.15.52:0/2062797560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1580696c0 0x7fb1581988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.153+0000 7fb157fff640 1 -- 172.21.15.52:0/2062797560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1580696c0 msgr2=0x7fb1581988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:41.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.153+0000 7fb157fff640 1 --2- 172.21.15.52:0/2062797560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1580696c0 0x7fb1581988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.153+0000 7fb157fff640 1 -- 172.21.15.52:0/2062797560 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb158068d00 msgr2=0x7fb158198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:41.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.153+0000 7fb157fff640 1 --2- 172.21.15.52:0/2062797560 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb158068d00 0x7fb158198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.153+0000 7fb157fff640 1 -- 172.21.15.52:0/2062797560 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb15819d380 con 0x7fb158107b50 2024-04-02T14:00:41.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.153+0000 7fb157fff640 1 --2- 172.21.15.52:0/2062797560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb158107b50 0x7fb15819cc50 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fb150009ca0 tx=0x7fb150032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:41.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.153+0000 7fb154ff9640 1 -- 172.21.15.52:0/2062797560 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb150042020 con 0x7fb158107b50 2024-04-02T14:00:41.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.153+0000 7fb154ff9640 1 -- 172.21.15.52:0/2062797560 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb150034640 con 0x7fb158107b50 2024-04-02T14:00:41.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.153+0000 7fb154ff9640 1 -- 172.21.15.52:0/2062797560 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb150039040 con 0x7fb158107b50 2024-04-02T14:00:41.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.153+0000 7fb15e00d640 1 -- 172.21.15.52:0/2062797560 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb15819d610 con 0x7fb158107b50 2024-04-02T14:00:41.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.153+0000 7fb15e00d640 1 -- 172.21.15.52:0/2062797560 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1581a4f00 con 0x7fb158107b50 2024-04-02T14:00:41.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.156+0000 7fb154ff9640 1 -- 172.21.15.52:0/2062797560 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb150045020 con 0x7fb158107b50 2024-04-02T14:00:41.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.156+0000 7fb160bfa740 1 Processor -- start 2024-04-02T14:00:41.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.156+0000 7fb160bfa740 1 -- start start 2024-04-02T14:00:41.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.156+0000 7fb160bfa740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e757a0 0x55add1e75b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.156+0000 7fb160bfa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55add1e760c0 0x55add1e7e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.156+0000 7fb160bfa740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55add1e7eca0 0x55add1e81090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.156+0000 7fb160bfa740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55add1dd97b0 con 0x55add1e757a0 2024-04-02T14:00:41.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.156+0000 7fb160bfa740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55add1d78570 con 0x55add1e7eca0 2024-04-02T14:00:41.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.156+0000 7fb160bfa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55add1dd5760 con 0x55add1e760c0 2024-04-02T14:00:41.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.157+0000 7fb156ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e757a0 0x55add1e75b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.157+0000 7fb156ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e757a0 0x55add1e75b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34296/0 (socket says 172.21.15.52:34296) 2024-04-02T14:00:41.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.157+0000 7fb156ffd640 1 -- 172.21.15.52:0/1853078929 learned_addr learned my addr 172.21.15.52:0/1853078929 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:41.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.157+0000 7fb1577fe640 1 --2- 172.21.15.52:0/1853078929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55add1e760c0 0x55add1e7e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.157+0000 7fb154ff9640 1 --2- 172.21.15.52:0/2062797560 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb1400a3d20 0x7fb1400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.157+0000 7fb156ffd640 1 -- 172.21.15.52:0/1853078929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55add1e760c0 msgr2=0x55add1e7e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:41.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.157+0000 7fb156ffd640 1 --2- 172.21.15.52:0/1853078929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55add1e760c0 0x55add1e7e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.157+0000 7fb156ffd640 1 -- 172.21.15.52:0/1853078929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55add1e7eca0 msgr2=0x55add1e81090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:41.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.157+0000 7fb156ffd640 1 --2- 172.21.15.52:0/1853078929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55add1e7eca0 0x55add1e81090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.157+0000 7fb156ffd640 1 -- 172.21.15.52:0/1853078929 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55add1aed3f0 con 0x55add1e757a0 2024-04-02T14:00:41.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.157+0000 7fb157fff640 1 --2- 172.21.15.52:0/2062797560 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb1400a3d20 0x7fb1400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.157+0000 7fb154ff9640 1 -- 172.21.15.52:0/2062797560 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7fb1500f27f0 con 0x7fb158107b50 2024-04-02T14:00:41.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.157+0000 7fb156ffd640 1 --2- 172.21.15.52:0/1853078929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e757a0 0x55add1e75b80 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb14c009cc0 tx=0x7fb14c0137d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d871153a6eb6b69 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:41.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.157+0000 7fb13dffb640 1 -- 172.21.15.52:0/1853078929 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c013070 con 0x55add1e757a0 2024-04-02T14:00:41.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.157+0000 7fb13dffb640 1 -- 172.21.15.52:0/1853078929 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb14c017030 con 0x55add1e757a0 2024-04-02T14:00:41.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.157+0000 7fb13dffb640 1 -- 172.21.15.52:0/1853078929 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c01b5f0 con 0x55add1e757a0 2024-04-02T14:00:41.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.158+0000 7fb157fff640 1 --2- 172.21.15.52:0/2062797560 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb1400a3d20 0x7fb1400a61e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb1500be1b0 tx=0x7fb15002c0f0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:41.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.158+0000 7fb160bfa740 1 -- 172.21.15.52:0/1853078929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e757a0 msgr2=0x55add1e75b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:41.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.158+0000 7fb160bfa740 1 --2- 172.21.15.52:0/1853078929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e757a0 0x55add1e75b80 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb14c009cc0 tx=0x7fb14c0137d0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.158+0000 7fb160bfa740 1 -- 172.21.15.52:0/1853078929 shutdown_connections 2024-04-02T14:00:41.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.158+0000 7fb160bfa740 1 --2- 172.21.15.52:0/1853078929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55add1e7eca0 0x55add1e81090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.158+0000 7fb160bfa740 1 --2- 172.21.15.52:0/1853078929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55add1e760c0 0x55add1e7e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.158+0000 7fb160bfa740 1 --2- 172.21.15.52:0/1853078929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e757a0 0x55add1e75b80 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.158+0000 7fb160bfa740 1 -- 172.21.15.52:0/1853078929 >> 172.21.15.52:0/1853078929 conn(0x55add1e6fed0 msgr2=0x55add1e702b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:41.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.158+0000 7fb160bfa740 1 -- 172.21.15.52:0/1853078929 shutdown_connections 2024-04-02T14:00:41.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.158+0000 7fb160bfa740 1 -- 172.21.15.52:0/1853078929 wait complete. 2024-04-02T14:00:41.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.158+0000 7fb160bfa740 1 -- start start 2024-04-02T14:00:41.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.159+0000 7fb160bfa740 1 Processor -- start 2024-04-02T14:00:41.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.159+0000 7fb160bfa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55add1e757a0 0x55add1e7e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.159+0000 7fb160bfa740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e760c0 0x55add1e7bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.159+0000 7fb160bfa740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55add1e7eca0 0x55add1e7c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.159+0000 7fb160bfa740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55add1e81a90 con 0x55add1e760c0 2024-04-02T14:00:41.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.159+0000 7fb160bfa740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55add1d94e80 con 0x55add1e7eca0 2024-04-02T14:00:41.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.159+0000 7fb160bfa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55add1e81c10 con 0x55add1e757a0 2024-04-02T14:00:41.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.159+0000 7fb160bfa740 10 client.0 _ll_register_callbacks cb 0x55add1dd0aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:41.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.159+0000 7fb1577fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e760c0 0x55add1e7bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.159+0000 7fb1577fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e760c0 0x55add1e7bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34304/0 (socket says 172.21.15.52:34304) 2024-04-02T14:00:41.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.159+0000 7fb1577fe640 1 -- 172.21.15.52:0/1382590530 learned_addr learned my addr 172.21.15.52:0/1382590530 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:41.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.159+0000 7fb156ffd640 1 --2- 172.21.15.52:0/1382590530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55add1e757a0 0x55add1e7e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.159+0000 7fb156ffd640 1 --2- 172.21.15.52:0/1382590530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55add1e7eca0 0x55add1e7c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.159+0000 7fb1577fe640 1 -- 172.21.15.52:0/1382590530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55add1e757a0 msgr2=0x55add1e7e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:41.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.159+0000 7fb1577fe640 1 --2- 172.21.15.52:0/1382590530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55add1e757a0 0x55add1e7e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.159+0000 7fb1577fe640 1 -- 172.21.15.52:0/1382590530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55add1e7eca0 msgr2=0x55add1e7c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:41.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.159+0000 7fb1577fe640 1 --2- 172.21.15.52:0/1382590530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55add1e7eca0 0x55add1e7c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.159+0000 7fb1577fe640 1 -- 172.21.15.52:0/1382590530 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55add1bb77d0 con 0x55add1e760c0 2024-04-02T14:00:41.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.159+0000 7fb1577fe640 1 --2- 172.21.15.52:0/1382590530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e760c0 0x55add1e7bac0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fb14800bf40 tx=0x7fb1480118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499772d2c58a3a6e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:41.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.159+0000 7fb136ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:41.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.160+0000 7fb136ffd640 1 -- 172.21.15.52:0/1382590530 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb148004020 con 0x55add1e760c0 2024-04-02T14:00:41.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.160+0000 7fb136ffd640 1 -- 172.21.15.52:0/1382590530 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7fb1480041c0 con 0x55add1e760c0 2024-04-02T14:00:41.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.160+0000 7fb160bfa740 1 -- 172.21.15.52:0/1382590530 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb14c017030 con 0x55add1e760c0 2024-04-02T14:00:41.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.160+0000 7fb136ffd640 1 -- 172.21.15.52:0/1382590530 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb148024050 con 0x55add1e760c0 2024-04-02T14:00:41.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.160+0000 7fb160bfa740 10 client.17700 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:41.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.160+0000 7fb160bfa740 1 -- 172.21.15.52:0/1382590530 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55add1e8db50 con 0x55add1e760c0 2024-04-02T14:00:41.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.160+0000 7fb136ffd640 1 -- 172.21.15.52:0/1382590530 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb14c017030 con 0x55add1e760c0 2024-04-02T14:00:41.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.160+0000 7fb136ffd640 1 -- 172.21.15.52:0/1382590530 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb148004cd0 con 0x55add1e760c0 2024-04-02T14:00:41.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.160+0000 7fb160bfa740 10 client.17700 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:41.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.160+0000 7fb160bfa740 10 client.17700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:00:41.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.160+0000 7fb160bfa740 20 client.17700 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:41.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.160+0000 7fb160bfa740 1 --2- 172.21.15.52:0/1382590530 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55add1e8f0b0 0x55add1e914d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.160+0000 7fb160bfa740 4 client.17700 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:00:41.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.160+0000 7fb160bfa740 1 -- 172.21.15.52:0/1382590530 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55add1de6cf0 con 0x55add1e8f0b0 2024-04-02T14:00:41.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.160+0000 7fb156ffd640 1 --2- 172.21.15.52:0/1382590530 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55add1e8f0b0 0x55add1e914d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.161+0000 7fb136ffd640 10 client.17700 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:41.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.161+0000 7fb156ffd640 1 --2- 172.21.15.52:0/1382590530 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55add1e8f0b0 0x55add1e914d0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3b855f9c74b9466 server_cookie=955fdcc43bb0c311 in_seq=0 out_seq=0 2024-04-02T14:00:41.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.161+0000 7fb136ffd640 1 -- 172.21.15.52:0/1382590530 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55add1de6cf0 con 0x55add1e8f0b0 2024-04-02T14:00:41.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.161+0000 7fb136ffd640 10 client.17700 handle_command_reply: tid=0 2024-04-02T14:00:41.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.161+0000 7fb160bfa740 1 client.17700 shutdown 2024-04-02T14:00:41.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.162+0000 7fb160bfa740 1 -- 172.21.15.52:0/1382590530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e760c0 msgr2=0x55add1e7bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:41.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.162+0000 7fb160bfa740 1 --2- 172.21.15.52:0/1382590530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e760c0 0x55add1e7bac0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fb14800bf40 tx=0x7fb1480118d0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.162+0000 7fb160bfa740 1 -- 172.21.15.52:0/1382590530 shutdown_connections 2024-04-02T14:00:41.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.162+0000 7fb160bfa740 1 --2- 172.21.15.52:0/1382590530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55add1e7eca0 0x55add1e7c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.162+0000 7fb160bfa740 1 --2- 172.21.15.52:0/1382590530 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55add1e8f0b0 0x55add1e914d0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.162+0000 7fb160bfa740 1 --2- 172.21.15.52:0/1382590530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e760c0 0x55add1e7bac0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.162+0000 7fb160bfa740 1 --2- 172.21.15.52:0/1382590530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55add1e757a0 0x55add1e7e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.162+0000 7fb136ffd640 0 client.17700 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:41.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.162+0000 7fb160bfa740 1 -- 172.21.15.52:0/1382590530 >> 172.21.15.52:0/1382590530 conn(0x55add1e6fed0 msgr2=0x55add1e71c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:41.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.162+0000 7fb160bfa740 1 -- 172.21.15.52:0/1382590530 shutdown_connections 2024-04-02T14:00:41.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.162+0000 7fb160bfa740 1 -- 172.21.15.52:0/1382590530 wait complete. 2024-04-02T14:00:41.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.162+0000 7fb160bfa740 20 client.17700 trim_cache size 0 max 16384 2024-04-02T14:00:41.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.175+0000 7fb160bfa740 1 Processor -- start 2024-04-02T14:00:41.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.175+0000 7fb160bfa740 1 -- start start 2024-04-02T14:00:41.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.175+0000 7fb160bfa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55add1e757a0 0x55add1e8b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.175+0000 7fb160bfa740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55add1e760c0 0x55add1e7ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.175+0000 7fb160bfa740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e7eca0 0x55add1e8a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.175+0000 7fb160bfa740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb1581aefa0 con 0x55add1e7eca0 2024-04-02T14:00:41.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.175+0000 7fb160bfa740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb1581b0ae0 con 0x55add1e760c0 2024-04-02T14:00:41.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.175+0000 7fb160bfa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb1581af670 con 0x55add1e757a0 2024-04-02T14:00:41.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.175+0000 7fb156ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e7eca0 0x55add1e8a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.175+0000 7fb1577fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55add1e760c0 0x55add1e7ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.175+0000 7fb156ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e7eca0 0x55add1e8a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34314/0 (socket says 172.21.15.52:34314) 2024-04-02T14:00:41.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.175+0000 7fb156ffd640 1 -- 172.21.15.52:0/2804564779 learned_addr learned my addr 172.21.15.52:0/2804564779 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:41.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.176+0000 7fb156ffd640 1 --2- 172.21.15.52:0/2804564779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55add1e757a0 0x55add1e8b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.176+0000 7fb156ffd640 1 -- 172.21.15.52:0/2804564779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55add1e757a0 msgr2=0x55add1e8b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:41.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.176+0000 7fb156ffd640 1 --2- 172.21.15.52:0/2804564779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55add1e757a0 0x55add1e8b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.176+0000 7fb156ffd640 1 -- 172.21.15.52:0/2804564779 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55add1e760c0 msgr2=0x55add1e7ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:41.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.176+0000 7fb156ffd640 1 --2- 172.21.15.52:0/2804564779 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55add1e760c0 0x55add1e7ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.176+0000 7fb156ffd640 1 -- 172.21.15.52:0/2804564779 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55add1e91840 con 0x55add1e7eca0 2024-04-02T14:00:41.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.176+0000 7fb1577fe640 1 --2- 172.21.15.52:0/2804564779 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55add1e760c0 0x55add1e7ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:00:41.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.176+0000 7fb156ffd640 1 --2- 172.21.15.52:0/2804564779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e7eca0 0x55add1e8a980 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fb14c013cd0 tx=0x7fb14c009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c99229e80fd7914 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:41.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.176+0000 7fb13d7fa640 1 -- 172.21.15.52:0/2804564779 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c009660 con 0x55add1e7eca0 2024-04-02T14:00:41.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.176+0000 7fb13d7fa640 1 -- 172.21.15.52:0/2804564779 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb14c017070 con 0x55add1e7eca0 2024-04-02T14:00:41.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.176+0000 7fb13d7fa640 1 -- 172.21.15.52:0/2804564779 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c00f8d0 con 0x55add1e7eca0 2024-04-02T14:00:41.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.176+0000 7fb160bfa740 1 -- 172.21.15.52:0/2804564779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e7eca0 msgr2=0x55add1e8a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:41.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.176+0000 7fb160bfa740 1 --2- 172.21.15.52:0/2804564779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e7eca0 0x55add1e8a980 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fb14c013cd0 tx=0x7fb14c009530 comp rx=0 tx=0).stop 2024-04-02T14:00:41.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.176+0000 7fb160bfa740 1 -- 172.21.15.52:0/2804564779 shutdown_connections 2024-04-02T14:00:41.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.176+0000 7fb160bfa740 1 --2- 172.21.15.52:0/2804564779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e7eca0 0x55add1e8a980 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.176+0000 7fb160bfa740 1 --2- 172.21.15.52:0/2804564779 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55add1e760c0 0x55add1e7ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.176+0000 7fb160bfa740 1 --2- 172.21.15.52:0/2804564779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55add1e757a0 0x55add1e8b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.176+0000 7fb160bfa740 1 -- 172.21.15.52:0/2804564779 >> 172.21.15.52:0/2804564779 conn(0x55add1e6fed0 msgr2=0x55add1e7b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:41.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.176+0000 7fb160bfa740 1 -- 172.21.15.52:0/2804564779 shutdown_connections 2024-04-02T14:00:41.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.177+0000 7fb160bfa740 1 -- 172.21.15.52:0/2804564779 wait complete. 2024-04-02T14:00:41.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.177+0000 7fb160bfa740 1 -- start start 2024-04-02T14:00:41.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.177+0000 7fb160bfa740 1 Processor -- start 2024-04-02T14:00:41.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.177+0000 7fb160bfa740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e757a0 0x55add1e86ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.177+0000 7fb160bfa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55add1e760c0 0x55add1e87410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.177+0000 7fb160bfa740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55add1e7eca0 0x55add1e82440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.177+0000 7fb160bfa740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb1581b0410 con 0x55add1e757a0 2024-04-02T14:00:41.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.177+0000 7fb160bfa740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb1581afd40 con 0x55add1e7eca0 2024-04-02T14:00:41.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.177+0000 7fb160bfa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb1581acfa0 con 0x55add1e760c0 2024-04-02T14:00:41.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.177+0000 7fb160bfa740 10 client.0 _ll_register_callbacks cb 0x7fb14800bb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:41.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.177+0000 7fb156ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e757a0 0x55add1e86ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.177+0000 7fb156ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e757a0 0x55add1e86ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34320/0 (socket says 172.21.15.52:34320) 2024-04-02T14:00:41.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.177+0000 7fb156ffd640 1 -- 172.21.15.52:0/3024289597 learned_addr learned my addr 172.21.15.52:0/3024289597 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:41.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.177+0000 7fb156ffd640 1 --2- 172.21.15.52:0/3024289597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55add1e7eca0 0x55add1e82440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.178+0000 7fb156ffd640 1 -- 172.21.15.52:0/3024289597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55add1e760c0 msgr2=0x55add1e87410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:41.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.178+0000 7fb1577fe640 1 --2- 172.21.15.52:0/3024289597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55add1e760c0 0x55add1e87410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.178+0000 7fb156ffd640 1 --2- 172.21.15.52:0/3024289597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55add1e760c0 0x55add1e87410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.178+0000 7fb156ffd640 1 -- 172.21.15.52:0/3024289597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55add1e7eca0 msgr2=0x55add1e82440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:41.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.178+0000 7fb156ffd640 1 --2- 172.21.15.52:0/3024289597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55add1e7eca0 0x55add1e82440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.178+0000 7fb156ffd640 1 -- 172.21.15.52:0/3024289597 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55add1e82c40 con 0x55add1e757a0 2024-04-02T14:00:41.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.178+0000 7fb156ffd640 1 --2- 172.21.15.52:0/3024289597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e757a0 0x55add1e86ed0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fb14c009b00 tx=0x7fb14c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688486f0075d84d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:41.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.178+0000 7fb136ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:41.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.178+0000 7fb136ffd640 1 -- 172.21.15.52:0/3024289597 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c009660 con 0x55add1e757a0 2024-04-02T14:00:41.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.178+0000 7fb136ffd640 1 -- 172.21.15.52:0/3024289597 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7fb14c005040 con 0x55add1e757a0 2024-04-02T14:00:41.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.178+0000 7fb160bfa740 1 -- 172.21.15.52:0/3024289597 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb14c017070 con 0x55add1e757a0 2024-04-02T14:00:41.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.178+0000 7fb136ffd640 1 -- 172.21.15.52:0/3024289597 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb14c03c050 con 0x55add1e757a0 2024-04-02T14:00:41.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.178+0000 7fb160bfa740 10 client.17706 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:41.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.178+0000 7fb160bfa740 1 -- 172.21.15.52:0/3024289597 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55add1ef52a0 con 0x55add1e757a0 2024-04-02T14:00:41.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.178+0000 7fb136ffd640 1 -- 172.21.15.52:0/3024289597 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb14c017070 con 0x55add1e757a0 2024-04-02T14:00:41.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.178+0000 7fb136ffd640 1 -- 172.21.15.52:0/3024289597 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb14c0035f0 con 0x55add1e757a0 2024-04-02T14:00:41.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.178+0000 7fb160bfa740 10 client.17706 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:41.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.178+0000 7fb160bfa740 10 client.17706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:00:41.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.178+0000 7fb160bfa740 20 client.17706 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:41.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.178+0000 7fb160bfa740 1 --2- 172.21.15.52:0/3024289597 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55add1ef56d0 0x55add1ef7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:41.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.178+0000 7fb160bfa740 4 client.17706 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:00:41.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.178+0000 7fb160bfa740 1 -- 172.21.15.52:0/3024289597 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55add1de1040 con 0x55add1ef56d0 2024-04-02T14:00:41.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.179+0000 7fb1577fe640 1 --2- 172.21.15.52:0/3024289597 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55add1ef56d0 0x55add1ef7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:41.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.179+0000 7fb136ffd640 10 client.17706 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:41.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.179+0000 7fb1577fe640 1 --2- 172.21.15.52:0/3024289597 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55add1ef56d0 0x55add1ef7ac0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6459572d3970e92f server_cookie=d35e07ad371f69f4 in_seq=0 out_seq=0 2024-04-02T14:00:41.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.179+0000 7fb136ffd640 1 -- 172.21.15.52:0/3024289597 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55add1de1040 con 0x55add1ef56d0 2024-04-02T14:00:41.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.179+0000 7fb136ffd640 10 client.17706 handle_command_reply: tid=0 2024-04-02T14:00:41.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.179+0000 7fb160bfa740 1 client.17706 shutdown 2024-04-02T14:00:41.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.179+0000 7fb160bfa740 1 -- 172.21.15.52:0/3024289597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e757a0 msgr2=0x55add1e86ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:41.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.179+0000 7fb160bfa740 1 --2- 172.21.15.52:0/3024289597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e757a0 0x55add1e86ed0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fb14c009b00 tx=0x7fb14c001e50 comp rx=0 tx=0).stop 2024-04-02T14:00:41.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.179+0000 7fb160bfa740 1 -- 172.21.15.52:0/3024289597 shutdown_connections 2024-04-02T14:00:41.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.179+0000 7fb160bfa740 1 --2- 172.21.15.52:0/3024289597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55add1e7eca0 0x55add1e82440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.179+0000 7fb160bfa740 1 --2- 172.21.15.52:0/3024289597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55add1e760c0 0x55add1e87410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.179+0000 7fb160bfa740 1 --2- 172.21.15.52:0/3024289597 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55add1ef56d0 0x55add1ef7ac0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.179+0000 7fb160bfa740 1 --2- 172.21.15.52:0/3024289597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55add1e757a0 0x55add1e86ed0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.179+0000 7fb136ffd640 0 client.17706 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:41.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.179+0000 7fb160bfa740 1 -- 172.21.15.52:0/3024289597 >> 172.21.15.52:0/3024289597 conn(0x55add1e6fed0 msgr2=0x55add1e711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:41.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.179+0000 7fb160bfa740 1 -- 172.21.15.52:0/3024289597 shutdown_connections 2024-04-02T14:00:41.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.179+0000 7fb160bfa740 1 -- 172.21.15.52:0/3024289597 wait complete. 2024-04-02T14:00:41.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.179+0000 7fb160bfa740 20 client.17706 trim_cache size 0 max 16384 2024-04-02T14:00:41.194 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:00:41.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.180+0000 7fb15e00d640 1 -- 172.21.15.52:0/2062797560 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb1400a3d20 msgr2=0x7fb1400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:41.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.180+0000 7fb15e00d640 1 --2- 172.21.15.52:0/2062797560 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb1400a3d20 0x7fb1400a61e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb1500be1b0 tx=0x7fb15002c0f0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.180+0000 7fb15e00d640 1 -- 172.21.15.52:0/2062797560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb158107b50 msgr2=0x7fb15819cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:41.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.180+0000 7fb15e00d640 1 --2- 172.21.15.52:0/2062797560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb158107b50 0x7fb15819cc50 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fb150009ca0 tx=0x7fb150032ec0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.180+0000 7fb15e00d640 1 -- 172.21.15.52:0/2062797560 shutdown_connections 2024-04-02T14:00:41.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.180+0000 7fb15e00d640 1 --2- 172.21.15.52:0/2062797560 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb1400a3d20 0x7fb1400a61e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.180+0000 7fb15e00d640 1 --2- 172.21.15.52:0/2062797560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb158107b50 0x7fb15819cc50 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.180+0000 7fb15e00d640 1 --2- 172.21.15.52:0/2062797560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1580696c0 0x7fb1581988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.180+0000 7fb15e00d640 1 --2- 172.21.15.52:0/2062797560 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb158068d00 0x7fb158198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:41.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.180+0000 7fb15e00d640 1 -- 172.21.15.52:0/2062797560 >> 172.21.15.52:0/2062797560 conn(0x7fb158077b00 msgr2=0x7fb1580fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:41.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.180+0000 7fb15e00d640 1 -- 172.21.15.52:0/2062797560 shutdown_connections 2024-04-02T14:00:41.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:41.180+0000 7fb15e00d640 1 -- 172.21.15.52:0/2062797560 wait complete. 2024-04-02T14:00:41.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:41 smithi189 ceph-mon[36683]: pgmap v220: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:00:41.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:41 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:00:41.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:41 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:00:41.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:41 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:00:41.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:41 smithi052 ceph-mon[32104]: pgmap v220: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:00:41.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:41 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:00:41.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:41 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:00:41.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:41 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:00:41.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:41 smithi088 ceph-mon[36528]: pgmap v220: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:00:41.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:41 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:00:41.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:41 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:00:41.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:41 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:00:42.198 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T14:00:42.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:00:42.199 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T14:00:42.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.315+0000 7f1ece904640 1 -- 172.21.15.52:0/2637528285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ec8107aa0 msgr2=0x7f1ec8107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:42.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.315+0000 7f1ece904640 1 --2- 172.21.15.52:0/2637528285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ec8107aa0 0x7f1ec8107e80 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009930 tx=0x7f1eb002bc30 comp rx=0 tx=0).stop 2024-04-02T14:00:42.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.315+0000 7f1ece904640 1 -- 172.21.15.52:0/2637528285 shutdown_connections 2024-04-02T14:00:42.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.315+0000 7f1ece904640 1 --2- 172.21.15.52:0/2637528285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec8102450 0x7f1ec810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.315+0000 7f1ece904640 1 --2- 172.21.15.52:0/2637528285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ec8101a90 0x7f1ec8101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.315+0000 7f1ece904640 1 --2- 172.21.15.52:0/2637528285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ec8107aa0 0x7f1ec8107e80 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.315+0000 7f1ece904640 1 -- 172.21.15.52:0/2637528285 >> 172.21.15.52:0/2637528285 conn(0x7f1ec8078f00 msgr2=0x7f1ec8079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:42.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.315+0000 7f1ece904640 1 -- 172.21.15.52:0/2637528285 shutdown_connections 2024-04-02T14:00:42.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.315+0000 7f1ece904640 1 -- 172.21.15.52:0/2637528285 wait complete. 2024-04-02T14:00:42.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.315+0000 7f1ece904640 1 Processor -- start 2024-04-02T14:00:42.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.315+0000 7f1ece904640 1 -- start start 2024-04-02T14:00:42.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.316+0000 7f1ece904640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec8101a90 0x7f1ec8198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.316+0000 7f1ece904640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ec8102450 0x7f1ec8198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.316+0000 7f1ece904640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ec8107aa0 0x7f1ec819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.316+0000 7f1ece904640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec8111eb0 con 0x7f1ec8107aa0 2024-04-02T14:00:42.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.316+0000 7f1ece904640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec8111d30 con 0x7f1ec8101a90 2024-04-02T14:00:42.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.316+0000 7f1ece904640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec8112030 con 0x7f1ec8102450 2024-04-02T14:00:42.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.316+0000 7f1eccea1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ec8107aa0 0x7f1ec819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.316+0000 7f1eccea1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ec8107aa0 0x7f1ec819cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56016/0 (socket says 172.21.15.52:56016) 2024-04-02T14:00:42.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.316+0000 7f1eccea1640 1 -- 172.21.15.52:0/64177983 learned_addr learned my addr 172.21.15.52:0/64177983 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:42.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.316+0000 7f1ec7fff640 1 --2- 172.21.15.52:0/64177983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec8101a90 0x7f1ec8198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.316+0000 7f1ec77fe640 1 --2- 172.21.15.52:0/64177983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ec8102450 0x7f1ec8198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.316+0000 7f1eccea1640 1 -- 172.21.15.52:0/64177983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ec8102450 msgr2=0x7f1ec8198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:42.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.316+0000 7f1eccea1640 1 --2- 172.21.15.52:0/64177983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ec8102450 0x7f1ec8198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.316+0000 7f1eccea1640 1 -- 172.21.15.52:0/64177983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec8101a90 msgr2=0x7f1ec8198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:42.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.316+0000 7f1eccea1640 1 --2- 172.21.15.52:0/64177983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec8101a90 0x7f1ec8198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.316+0000 7f1eccea1640 1 -- 172.21.15.52:0/64177983 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec819d320 con 0x7f1ec8107aa0 2024-04-02T14:00:42.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.317+0000 7f1eccea1640 1 --2- 172.21.15.52:0/64177983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ec8107aa0 0x7f1ec819cbf0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000cce0 tx=0x7f1ec000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:42.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.317+0000 7f1ec57fa640 1 -- 172.21.15.52:0/64177983 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec000f450 con 0x7f1ec8107aa0 2024-04-02T14:00:42.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.317+0000 7f1ec57fa640 1 -- 172.21.15.52:0/64177983 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ec000fad0 con 0x7f1ec8107aa0 2024-04-02T14:00:42.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.317+0000 7f1ece904640 1 -- 172.21.15.52:0/64177983 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec819d610 con 0x7f1ec8107aa0 2024-04-02T14:00:42.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.317+0000 7f1ec57fa640 1 -- 172.21.15.52:0/64177983 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0018c10 con 0x7f1ec8107aa0 2024-04-02T14:00:42.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.317+0000 7f1ece904640 1 -- 172.21.15.52:0/64177983 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec81a4ea0 con 0x7f1ec8107aa0 2024-04-02T14:00:42.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.319+0000 7f1ec57fa640 1 -- 172.21.15.52:0/64177983 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1ec000f5f0 con 0x7f1ec8107aa0 2024-04-02T14:00:42.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.320+0000 7f1ed15c9740 1 Processor -- start 2024-04-02T14:00:42.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.320+0000 7f1ed15c9740 1 -- start start 2024-04-02T14:00:42.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.320+0000 7f1ed15c9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5624b86f92c0 0x5624b86f96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.320+0000 7f1ed15c9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624b86f9be0 0x5624b8702280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.320+0000 7f1ed15c9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b87027c0 0x5624b8704bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.320+0000 7f1ed15c9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5624b865d770 con 0x5624b87027c0 2024-04-02T14:00:42.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.320+0000 7f1ed15c9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5624b8605960 con 0x5624b86f92c0 2024-04-02T14:00:42.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.320+0000 7f1ed15c9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5624b86598c0 con 0x5624b86f9be0 2024-04-02T14:00:42.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.320+0000 7f1ec77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b87027c0 0x5624b8704bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.320+0000 7f1ec57fa640 1 --2- 172.21.15.52:0/64177983 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1eb40a3da0 0x7f1eb40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.320+0000 7f1ec77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b87027c0 0x5624b8704bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56028/0 (socket says 172.21.15.52:56028) 2024-04-02T14:00:42.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.320+0000 7f1ec77fe640 1 -- 172.21.15.52:0/4274143105 learned_addr learned my addr 172.21.15.52:0/4274143105 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:42.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.320+0000 7f1ec7fff640 1 --2- 172.21.15.52:0/4274143105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624b86f9be0 0x5624b8702280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.320+0000 7f1ec77fe640 1 --2- 172.21.15.52:0/4274143105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5624b86f92c0 0x5624b86f96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.320+0000 7f1eccea1640 1 --2- 172.21.15.52:0/64177983 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1eb40a3da0 0x7f1eb40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.320+0000 7f1ec57fa640 1 -- 172.21.15.52:0/64177983 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f1ec00d1060 con 0x7f1ec8107aa0 2024-04-02T14:00:42.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.320+0000 7f1ec77fe640 1 -- 172.21.15.52:0/4274143105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624b86f9be0 msgr2=0x5624b8702280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:42.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.320+0000 7f1ec77fe640 1 --2- 172.21.15.52:0/4274143105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624b86f9be0 0x5624b8702280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.320+0000 7f1ec77fe640 1 -- 172.21.15.52:0/4274143105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5624b86f92c0 msgr2=0x5624b86f96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:42.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.320+0000 7f1ec77fe640 1 --2- 172.21.15.52:0/4274143105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5624b86f92c0 0x5624b86f96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.320+0000 7f1ec77fe640 1 -- 172.21.15.52:0/4274143105 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624b84397e0 con 0x5624b87027c0 2024-04-02T14:00:42.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.321+0000 7f1ec77fe640 1 --2- 172.21.15.52:0/4274143105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b87027c0 0x5624b8704bb0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800bc10 tx=0x7f1eb80041f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96db84d1f0a6ed5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:42.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.321+0000 7f1eccea1640 1 --2- 172.21.15.52:0/64177983 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1eb40a3da0 0x7f1eb40a6260 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00ce4d0 tx=0x7f1ec001baf0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:42.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.321+0000 7f1ebe7fc640 1 -- 172.21.15.52:0/4274143105 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8005020 con 0x5624b87027c0 2024-04-02T14:00:42.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.321+0000 7f1ebe7fc640 1 -- 172.21.15.52:0/4274143105 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eb80047a0 con 0x5624b87027c0 2024-04-02T14:00:42.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.321+0000 7f1ebe7fc640 1 -- 172.21.15.52:0/4274143105 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8010710 con 0x5624b87027c0 2024-04-02T14:00:42.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.321+0000 7f1ed15c9740 1 -- 172.21.15.52:0/4274143105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b87027c0 msgr2=0x5624b8704bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:42.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.321+0000 7f1ed15c9740 1 --2- 172.21.15.52:0/4274143105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b87027c0 0x5624b8704bb0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800bc10 tx=0x7f1eb80041f0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.321+0000 7f1ed15c9740 1 -- 172.21.15.52:0/4274143105 shutdown_connections 2024-04-02T14:00:42.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.321+0000 7f1ed15c9740 1 --2- 172.21.15.52:0/4274143105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b87027c0 0x5624b8704bb0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.321+0000 7f1ed15c9740 1 --2- 172.21.15.52:0/4274143105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624b86f9be0 0x5624b8702280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.321+0000 7f1ed15c9740 1 --2- 172.21.15.52:0/4274143105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5624b86f92c0 0x5624b86f96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.321+0000 7f1ed15c9740 1 -- 172.21.15.52:0/4274143105 >> 172.21.15.52:0/4274143105 conn(0x5624b86f39f0 msgr2=0x5624b86f3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:42.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.322+0000 7f1ed15c9740 1 -- 172.21.15.52:0/4274143105 shutdown_connections 2024-04-02T14:00:42.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.322+0000 7f1ed15c9740 1 -- 172.21.15.52:0/4274143105 wait complete. 2024-04-02T14:00:42.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.322+0000 7f1ed15c9740 1 -- start start 2024-04-02T14:00:42.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.322+0000 7f1ed15c9740 1 Processor -- start 2024-04-02T14:00:42.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.322+0000 7f1ed15c9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624b86f92c0 0x5624b86ff860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.322+0000 7f1ed15c9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b86f9be0 0x5624b86ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.322+0000 7f1ed15c9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5624b87027c0 0x5624b87002e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.322+0000 7f1ed15c9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5624b87055b0 con 0x5624b86f9be0 2024-04-02T14:00:42.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.322+0000 7f1ed15c9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5624b8618d20 con 0x5624b87027c0 2024-04-02T14:00:42.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.322+0000 7f1ed15c9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5624b8705730 con 0x5624b86f92c0 2024-04-02T14:00:42.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.322+0000 7f1ed15c9740 10 client.0 _ll_register_callbacks cb 0x5624b85e5a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:42.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.322+0000 7f1ec7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b86f9be0 0x5624b86ffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.323+0000 7f1ec7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b86f9be0 0x5624b86ffda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56038/0 (socket says 172.21.15.52:56038) 2024-04-02T14:00:42.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.323+0000 7f1ec7fff640 1 -- 172.21.15.52:0/4067207055 learned_addr learned my addr 172.21.15.52:0/4067207055 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:42.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.323+0000 7f1ec77fe640 1 --2- 172.21.15.52:0/4067207055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624b86f92c0 0x5624b86ff860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.323+0000 7f1ec77fe640 1 --2- 172.21.15.52:0/4067207055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5624b87027c0 0x5624b87002e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.323+0000 7f1ec7fff640 1 -- 172.21.15.52:0/4067207055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624b86f92c0 msgr2=0x5624b86ff860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:42.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.323+0000 7f1ec7fff640 1 --2- 172.21.15.52:0/4067207055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624b86f92c0 0x5624b86ff860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.323+0000 7f1ec7fff640 1 -- 172.21.15.52:0/4067207055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5624b87027c0 msgr2=0x5624b87002e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:42.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.323+0000 7f1ec7fff640 1 --2- 172.21.15.52:0/4067207055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5624b87027c0 0x5624b87002e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.323+0000 7f1ec7fff640 1 -- 172.21.15.52:0/4067207055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624b8499f70 con 0x5624b86f9be0 2024-04-02T14:00:42.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.323+0000 7f1ec77fe640 1 --2- 172.21.15.52:0/4067207055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5624b87027c0 0x5624b87002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:00:42.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.323+0000 7f1ec7fff640 1 --2- 172.21.15.52:0/4067207055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b86f9be0 0x5624b86ffda0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f1eb002cfd0 tx=0x7f1eb002ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa223854b0e0fa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:42.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.323+0000 7f1ea77fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:42.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.323+0000 7f1ea77fe640 1 -- 172.21.15.52:0/4067207055 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0004380 con 0x5624b86f9be0 2024-04-02T14:00:42.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.323+0000 7f1ea77fe640 1 -- 172.21.15.52:0/4067207055 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f1eb00046c0 con 0x5624b86f9be0 2024-04-02T14:00:42.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.323+0000 7f1ed15c9740 1 -- 172.21.15.52:0/4067207055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1eb80047a0 con 0x5624b86f9be0 2024-04-02T14:00:42.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.323+0000 7f1ea77fe640 1 -- 172.21.15.52:0/4067207055 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1eb00048c0 con 0x5624b86f9be0 2024-04-02T14:00:42.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.323+0000 7f1ed15c9740 10 client.17718 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:42.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.323+0000 7f1ed15c9740 1 -- 172.21.15.52:0/4067207055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5624b8331170 con 0x5624b86f9be0 2024-04-02T14:00:42.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.324+0000 7f1ea77fe640 1 -- 172.21.15.52:0/4067207055 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eb80047a0 con 0x5624b86f9be0 2024-04-02T14:00:42.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.324+0000 7f1ea77fe640 1 -- 172.21.15.52:0/4067207055 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1eb0003660 con 0x5624b86f9be0 2024-04-02T14:00:42.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.324+0000 7f1ed15c9740 10 client.17718 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.324+0000 7f1ed15c9740 10 client.17718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:00:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.324+0000 7f1ed15c9740 20 client.17718 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.324+0000 7f1ed15c9740 1 --2- 172.21.15.52:0/4067207055 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5624b8712cf0 0x5624b8715110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.324+0000 7f1ed15c9740 4 client.17718 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:00:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.324+0000 7f1ed15c9740 1 -- 172.21.15.52:0/4067207055 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624b866acb0 con 0x5624b8712cf0 2024-04-02T14:00:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.324+0000 7f1ec77fe640 1 --2- 172.21.15.52:0/4067207055 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5624b8712cf0 0x5624b8715110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.324+0000 7f1ec77fe640 1 --2- 172.21.15.52:0/4067207055 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5624b8712cf0 0x5624b8715110 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad9bb0b2da6f6c03 server_cookie=61070f3fa8d992a0 in_seq=0 out_seq=0 2024-04-02T14:00:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.324+0000 7f1ea77fe640 10 client.17718 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:42.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.325+0000 7f1ea77fe640 1 -- 172.21.15.52:0/4067207055 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5624b866acb0 con 0x5624b8712cf0 2024-04-02T14:00:42.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.325+0000 7f1ea77fe640 10 client.17718 handle_command_reply: tid=0 2024-04-02T14:00:42.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.325+0000 7f1ed15c9740 1 client.17718 shutdown 2024-04-02T14:00:42.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.325+0000 7f1ed15c9740 1 -- 172.21.15.52:0/4067207055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b86f9be0 msgr2=0x5624b86ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:42.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.325+0000 7f1ed15c9740 1 --2- 172.21.15.52:0/4067207055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b86f9be0 0x5624b86ffda0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f1eb002cfd0 tx=0x7f1eb002ce30 comp rx=0 tx=0).stop 2024-04-02T14:00:42.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.325+0000 7f1ed15c9740 1 -- 172.21.15.52:0/4067207055 shutdown_connections 2024-04-02T14:00:42.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.325+0000 7f1ed15c9740 1 --2- 172.21.15.52:0/4067207055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5624b87027c0 0x5624b87002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.325+0000 7f1ed15c9740 1 --2- 172.21.15.52:0/4067207055 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5624b8712cf0 0x5624b8715110 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.325+0000 7f1ed15c9740 1 --2- 172.21.15.52:0/4067207055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b86f9be0 0x5624b86ffda0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.325+0000 7f1ed15c9740 1 --2- 172.21.15.52:0/4067207055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624b86f92c0 0x5624b86ff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.325+0000 7f1ea77fe640 0 client.17718 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:42.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.325+0000 7f1ed15c9740 1 -- 172.21.15.52:0/4067207055 >> 172.21.15.52:0/4067207055 conn(0x5624b86f39f0 msgr2=0x5624b86f5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:42.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.325+0000 7f1ed15c9740 1 -- 172.21.15.52:0/4067207055 shutdown_connections 2024-04-02T14:00:42.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.326+0000 7f1ed15c9740 1 -- 172.21.15.52:0/4067207055 wait complete. 2024-04-02T14:00:42.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.326+0000 7f1ed15c9740 20 client.17718 trim_cache size 0 max 16384 2024-04-02T14:00:42.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.343+0000 7f1ed15c9740 1 Processor -- start 2024-04-02T14:00:42.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.343+0000 7f1ed15c9740 1 -- start start 2024-04-02T14:00:42.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.344+0000 7f1ed15c9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624b86f92c0 0x5624b870e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.344+0000 7f1ed15c9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5624b86f9be0 0x5624b8700100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.344+0000 7f1ed15c9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b87027c0 0x5624b871aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.344+0000 7f1ed15c9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81aee90 con 0x5624b87027c0 2024-04-02T14:00:42.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.344+0000 7f1ed15c9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81b09d0 con 0x5624b86f9be0 2024-04-02T14:00:42.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.344+0000 7f1ed15c9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81af560 con 0x5624b86f92c0 2024-04-02T14:00:42.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.344+0000 7f1ec77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b87027c0 0x5624b871aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.344+0000 7f1ec77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b87027c0 0x5624b871aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56050/0 (socket says 172.21.15.52:56050) 2024-04-02T14:00:42.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.344+0000 7f1ec77fe640 1 -- 172.21.15.52:0/1469149477 learned_addr learned my addr 172.21.15.52:0/1469149477 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:42.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.344+0000 7f1ec77fe640 1 -- 172.21.15.52:0/1469149477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624b86f92c0 msgr2=0x5624b870e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:42.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.344+0000 7f1ec77fe640 1 --2- 172.21.15.52:0/1469149477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624b86f92c0 0x5624b870e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.344+0000 7f1ec77fe640 1 -- 172.21.15.52:0/1469149477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5624b86f9be0 msgr2=0x5624b8700100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:42.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.344+0000 7f1ec7fff640 1 --2- 172.21.15.52:0/1469149477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5624b86f9be0 0x5624b8700100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.344+0000 7f1ec77fe640 1 --2- 172.21.15.52:0/1469149477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5624b86f9be0 0x5624b8700100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.344+0000 7f1ec77fe640 1 -- 172.21.15.52:0/1469149477 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624b86f9a10 con 0x5624b87027c0 2024-04-02T14:00:42.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.344+0000 7f1ec7fff640 1 --2- 172.21.15.52:0/1469149477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5624b86f9be0 0x5624b8700100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:00:42.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.344+0000 7f1ec77fe640 1 --2- 172.21.15.52:0/1469149477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b87027c0 0x5624b871aa20 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x5624b86f4b80 tx=0x7f1eb8005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf18369fc0f6cd85 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:42.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.344+0000 7f1ebdffb640 1 -- 172.21.15.52:0/1469149477 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb802b040 con 0x5624b87027c0 2024-04-02T14:00:42.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.344+0000 7f1ebdffb640 1 -- 172.21.15.52:0/1469149477 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eb802e810 con 0x5624b87027c0 2024-04-02T14:00:42.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.344+0000 7f1ebdffb640 1 -- 172.21.15.52:0/1469149477 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8023c60 con 0x5624b87027c0 2024-04-02T14:00:42.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.345+0000 7f1ed15c9740 1 -- 172.21.15.52:0/1469149477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b87027c0 msgr2=0x5624b871aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:42.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.345+0000 7f1ed15c9740 1 --2- 172.21.15.52:0/1469149477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b87027c0 0x5624b871aa20 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x5624b86f4b80 tx=0x7f1eb8005bb0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.345+0000 7f1ed15c9740 1 -- 172.21.15.52:0/1469149477 shutdown_connections 2024-04-02T14:00:42.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.345+0000 7f1ed15c9740 1 --2- 172.21.15.52:0/1469149477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b87027c0 0x5624b871aa20 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.345+0000 7f1ed15c9740 1 --2- 172.21.15.52:0/1469149477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5624b86f9be0 0x5624b8700100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.345+0000 7f1ed15c9740 1 --2- 172.21.15.52:0/1469149477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624b86f92c0 0x5624b870e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.345+0000 7f1ed15c9740 1 -- 172.21.15.52:0/1469149477 >> 172.21.15.52:0/1469149477 conn(0x5624b86f39f0 msgr2=0x5624b86f2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:42.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.345+0000 7f1ed15c9740 1 -- 172.21.15.52:0/1469149477 shutdown_connections 2024-04-02T14:00:42.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.345+0000 7f1ed15c9740 1 -- 172.21.15.52:0/1469149477 wait complete. 2024-04-02T14:00:42.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.345+0000 7f1ed15c9740 1 -- start start 2024-04-02T14:00:42.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.346+0000 7f1ed15c9740 1 Processor -- start 2024-04-02T14:00:42.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.346+0000 7f1ed15c9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b86f92c0 0x5624b8706670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.346+0000 7f1ed15c9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624b86f9be0 0x5624b8706bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.346+0000 7f1ed15c9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5624b87027c0 0x5624b87070f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.346+0000 7f1ed15c9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81b0300 con 0x5624b86f92c0 2024-04-02T14:00:42.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.346+0000 7f1ed15c9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81afc30 con 0x5624b87027c0 2024-04-02T14:00:42.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.346+0000 7f1ed15c9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81ace90 con 0x5624b86f9be0 2024-04-02T14:00:42.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.346+0000 7f1ed15c9740 10 client.0 _ll_register_callbacks cb 0x7f1eb002ccf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:42.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.346+0000 7f1ec77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b86f92c0 0x5624b8706670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.346+0000 7f1ec77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b86f92c0 0x5624b8706670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56060/0 (socket says 172.21.15.52:56060) 2024-04-02T14:00:42.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.346+0000 7f1ec77fe640 1 -- 172.21.15.52:0/3552869097 learned_addr learned my addr 172.21.15.52:0/3552869097 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:42.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.346+0000 7f1ec77fe640 1 --2- 172.21.15.52:0/3552869097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5624b87027c0 0x5624b87070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.346+0000 7f1ec7fff640 1 --2- 172.21.15.52:0/3552869097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624b86f9be0 0x5624b8706bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.346+0000 7f1ec77fe640 1 -- 172.21.15.52:0/3552869097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624b86f9be0 msgr2=0x5624b8706bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:42.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.346+0000 7f1ec77fe640 1 --2- 172.21.15.52:0/3552869097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624b86f9be0 0x5624b8706bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.346+0000 7f1ec77fe640 1 -- 172.21.15.52:0/3552869097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5624b87027c0 msgr2=0x5624b87070f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:42.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.346+0000 7f1ec77fe640 1 --2- 172.21.15.52:0/3552869097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5624b87027c0 0x5624b87070f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.346+0000 7f1ec77fe640 1 -- 172.21.15.52:0/3552869097 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624b8778530 con 0x5624b86f92c0 2024-04-02T14:00:42.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.346+0000 7f1ec7fff640 1 --2- 172.21.15.52:0/3552869097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624b86f9be0 0x5624b8706bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:00:42.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.346+0000 7f1ec77fe640 1 --2- 172.21.15.52:0/3552869097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b86f92c0 0x5624b8706670 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x5624b866ad60 tx=0x7f1eb80238e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca62890cecb528c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:42.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.346+0000 7f1ea77fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:42.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.347+0000 7f1ea77fe640 1 -- 172.21.15.52:0/3552869097 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb803f060 con 0x5624b86f92c0 2024-04-02T14:00:42.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.347+0000 7f1ea77fe640 1 -- 172.21.15.52:0/3552869097 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7f1eb80239e0 con 0x5624b86f92c0 2024-04-02T14:00:42.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.347+0000 7f1ed15c9740 1 -- 172.21.15.52:0/3552869097 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1eb802e810 con 0x5624b86f92c0 2024-04-02T14:00:42.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.347+0000 7f1ea77fe640 1 -- 172.21.15.52:0/3552869097 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1eb8033040 con 0x5624b86f92c0 2024-04-02T14:00:42.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.347+0000 7f1ed15c9740 10 client.17724 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:42.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.347+0000 7f1ed15c9740 1 -- 172.21.15.52:0/3552869097 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5624b87797c0 con 0x5624b86f92c0 2024-04-02T14:00:42.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.347+0000 7f1ea77fe640 1 -- 172.21.15.52:0/3552869097 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eb802e810 con 0x5624b86f92c0 2024-04-02T14:00:42.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.347+0000 7f1ea77fe640 1 -- 172.21.15.52:0/3552869097 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1eb801d040 con 0x5624b86f92c0 2024-04-02T14:00:42.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.347+0000 7f1ed15c9740 10 client.17724 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:42.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.347+0000 7f1ed15c9740 10 client.17724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:00:42.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.347+0000 7f1ed15c9740 20 client.17724 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:42.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.347+0000 7f1ed15c9740 1 --2- 172.21.15.52:0/3552869097 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5624b8779c40 0x5624b877c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.347+0000 7f1ed15c9740 4 client.17724 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T14:00:42.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.347+0000 7f1ed15c9740 1 -- 172.21.15.52:0/3552869097 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5624b8665000 con 0x5624b8779c40 2024-04-02T14:00:42.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.347+0000 7f1ec7fff640 1 --2- 172.21.15.52:0/3552869097 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5624b8779c40 0x5624b877c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.348+0000 7f1ea77fe640 10 client.17724 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:42.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.348+0000 7f1ec7fff640 1 --2- 172.21.15.52:0/3552869097 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5624b8779c40 0x5624b877c030 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59de642347887f8b server_cookie=aebacd7c9928dbe8 in_seq=0 out_seq=0 2024-04-02T14:00:42.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.349+0000 7f1ea77fe640 1 -- 172.21.15.52:0/3552869097 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5624b8665000 con 0x5624b8779c40 2024-04-02T14:00:42.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.349+0000 7f1ea77fe640 10 client.17724 handle_command_reply: tid=0 2024-04-02T14:00:42.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.349+0000 7f1ed15c9740 1 client.17724 shutdown 2024-04-02T14:00:42.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.349+0000 7f1ed15c9740 1 -- 172.21.15.52:0/3552869097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b86f92c0 msgr2=0x5624b8706670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:42.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.349+0000 7f1ed15c9740 1 --2- 172.21.15.52:0/3552869097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b86f92c0 0x5624b8706670 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x5624b866ad60 tx=0x7f1eb80238e0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.349+0000 7f1ed15c9740 1 -- 172.21.15.52:0/3552869097 shutdown_connections 2024-04-02T14:00:42.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.349+0000 7f1ed15c9740 1 --2- 172.21.15.52:0/3552869097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5624b87027c0 0x5624b87070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.349+0000 7f1ed15c9740 1 --2- 172.21.15.52:0/3552869097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624b86f9be0 0x5624b8706bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.349+0000 7f1ed15c9740 1 --2- 172.21.15.52:0/3552869097 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5624b8779c40 0x5624b877c030 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.349+0000 7f1ed15c9740 1 --2- 172.21.15.52:0/3552869097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624b86f92c0 0x5624b8706670 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.349+0000 7f1ea77fe640 0 client.17724 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:42.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.349+0000 7f1ed15c9740 1 -- 172.21.15.52:0/3552869097 >> 172.21.15.52:0/3552869097 conn(0x5624b86f39f0 msgr2=0x5624b86f4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:42.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.349+0000 7f1ed15c9740 1 -- 172.21.15.52:0/3552869097 shutdown_connections 2024-04-02T14:00:42.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.349+0000 7f1ed15c9740 1 -- 172.21.15.52:0/3552869097 wait complete. 2024-04-02T14:00:42.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.349+0000 7f1ed15c9740 20 client.17724 trim_cache size 0 max 16384 2024-04-02T14:00:42.363 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:00:42.363 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-04-02T14:00:42.363 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "1e34f335-e306-4f37-9a0b-d19dfdec426a", 2024-04-02T14:00:42.363 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-04-02T14:00:42.364 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:00:42.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.350+0000 7f1ece904640 1 -- 172.21.15.52:0/64177983 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1eb40a3da0 msgr2=0x7f1eb40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:42.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.350+0000 7f1ece904640 1 --2- 172.21.15.52:0/64177983 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1eb40a3da0 0x7f1eb40a6260 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00ce4d0 tx=0x7f1ec001baf0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.350+0000 7f1ece904640 1 -- 172.21.15.52:0/64177983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ec8107aa0 msgr2=0x7f1ec819cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:42.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.350+0000 7f1ece904640 1 --2- 172.21.15.52:0/64177983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ec8107aa0 0x7f1ec819cbf0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000cce0 tx=0x7f1ec000a620 comp rx=0 tx=0).stop 2024-04-02T14:00:42.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.350+0000 7f1ece904640 1 -- 172.21.15.52:0/64177983 shutdown_connections 2024-04-02T14:00:42.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.350+0000 7f1ece904640 1 --2- 172.21.15.52:0/64177983 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1eb40a3da0 0x7f1eb40a6260 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.350+0000 7f1ece904640 1 --2- 172.21.15.52:0/64177983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ec8107aa0 0x7f1ec819cbf0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.350+0000 7f1ece904640 1 --2- 172.21.15.52:0/64177983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ec8102450 0x7f1ec8198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.350+0000 7f1ece904640 1 --2- 172.21.15.52:0/64177983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec8101a90 0x7f1ec8198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.350+0000 7f1ece904640 1 -- 172.21.15.52:0/64177983 >> 172.21.15.52:0/64177983 conn(0x7f1ec8078f00 msgr2=0x7f1ec8104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:42.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.350+0000 7f1ece904640 1 -- 172.21.15.52:0/64177983 shutdown_connections 2024-04-02T14:00:42.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.350+0000 7f1ece904640 1 -- 172.21.15.52:0/64177983 wait complete. 2024-04-02T14:00:42.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:00:42.367 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:00:42.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.477+0000 7fdf8c9a1640 1 -- 172.21.15.52:0/2599502097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf88102b90 msgr2=0x7fdf88102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:42.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.477+0000 7fdf8c9a1640 1 --2- 172.21.15.52:0/2599502097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf88102b90 0x7fdf88102f70 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009900 tx=0x7fdf7802bcd0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.477+0000 7fdf8c9a1640 1 -- 172.21.15.52:0/2599502097 shutdown_connections 2024-04-02T14:00:42.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.477+0000 7fdf8c9a1640 1 --2- 172.21.15.52:0/2599502097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf8810bf60 0x7fdf8810e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.477+0000 7fdf8c9a1640 1 --2- 172.21.15.52:0/2599502097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf881034b0 0x7fdf8810b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.477+0000 7fdf8c9a1640 1 --2- 172.21.15.52:0/2599502097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf88102b90 0x7fdf88102f70 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.477+0000 7fdf8c9a1640 1 -- 172.21.15.52:0/2599502097 >> 172.21.15.52:0/2599502097 conn(0x7fdf880fc350 msgr2=0x7fdf880fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:42.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.477+0000 7fdf8c9a1640 1 -- 172.21.15.52:0/2599502097 shutdown_connections 2024-04-02T14:00:42.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.477+0000 7fdf8c9a1640 1 -- 172.21.15.52:0/2599502097 wait complete. 2024-04-02T14:00:42.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.477+0000 7fdf8c9a1640 1 Processor -- start 2024-04-02T14:00:42.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.478+0000 7fdf8c9a1640 1 -- start start 2024-04-02T14:00:42.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.478+0000 7fdf8c9a1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf88102b90 0x7fdf881982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.478+0000 7fdf8c9a1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf881034b0 0x7fdf881987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.478+0000 7fdf8c9a1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf8810bf60 0x7fdf8819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.478+0000 7fdf8c9a1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdf88111d40 con 0x7fdf8810bf60 2024-04-02T14:00:42.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.478+0000 7fdf8c9a1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdf88111bc0 con 0x7fdf881034b0 2024-04-02T14:00:42.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.478+0000 7fdf8c9a1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdf88111ec0 con 0x7fdf88102b90 2024-04-02T14:00:42.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.478+0000 7fdf877fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf88102b90 0x7fdf881982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.478+0000 7fdf87fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf8810bf60 0x7fdf8819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.478+0000 7fdf877fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf88102b90 0x7fdf881982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.52:43138/0 (socket says 172.21.15.52:43138) 2024-04-02T14:00:42.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.478+0000 7fdf877fe640 1 -- 172.21.15.52:0/608883669 learned_addr learned my addr 172.21.15.52:0/608883669 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:42.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.478+0000 7fdf86ffd640 1 --2- 172.21.15.52:0/608883669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf881034b0 0x7fdf881987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.479+0000 7fdf877fe640 1 -- 172.21.15.52:0/608883669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf881034b0 msgr2=0x7fdf881987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:42.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.479+0000 7fdf877fe640 1 --2- 172.21.15.52:0/608883669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf881034b0 0x7fdf881987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.479+0000 7fdf877fe640 1 -- 172.21.15.52:0/608883669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf8810bf60 msgr2=0x7fdf8819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:42.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.479+0000 7fdf877fe640 1 --2- 172.21.15.52:0/608883669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf8810bf60 0x7fdf8819cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.479+0000 7fdf877fe640 1 -- 172.21.15.52:0/608883669 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8819d1f0 con 0x7fdf88102b90 2024-04-02T14:00:42.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.479+0000 7fdf877fe640 1 --2- 172.21.15.52:0/608883669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf88102b90 0x7fdf881982a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdf78005940 tx=0x7fdf7802cbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:42.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.479+0000 7fdf84ff9640 1 -- 172.21.15.52:0/608883669 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf78041070 con 0x7fdf88102b90 2024-04-02T14:00:42.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.479+0000 7fdf84ff9640 1 -- 172.21.15.52:0/608883669 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf78004450 con 0x7fdf88102b90 2024-04-02T14:00:42.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.479+0000 7fdf84ff9640 1 -- 172.21.15.52:0/608883669 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf78003d80 con 0x7fdf88102b90 2024-04-02T14:00:42.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.479+0000 7fdf8c9a1640 1 -- 172.21.15.52:0/608883669 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf8819d480 con 0x7fdf88102b90 2024-04-02T14:00:42.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.479+0000 7fdf8c9a1640 1 -- 172.21.15.52:0/608883669 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf88100710 con 0x7fdf88102b90 2024-04-02T14:00:42.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.481+0000 7fdf84ff9640 1 -- 172.21.15.52:0/608883669 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdf78044070 con 0x7fdf88102b90 2024-04-02T14:00:42.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.482+0000 7fdf907e0740 1 Processor -- start 2024-04-02T14:00:42.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.482+0000 7fdf907e0740 1 -- start start 2024-04-02T14:00:42.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.482+0000 7fdf84ff9640 1 --2- 172.21.15.52:0/608883669 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdf700a3da0 0x7fdf700a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.482+0000 7fdf907e0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbf2f89ae0 0x55cbf2f89ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.482+0000 7fdf907e0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f8a400 0x55cbf2f92aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.482+0000 7fdf907e0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbf2f92fe0 0x55cbf2f953d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.482+0000 7fdf907e0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cbf2eed830 con 0x55cbf2f8a400 2024-04-02T14:00:42.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.482+0000 7fdf84ff9640 1 -- 172.21.15.52:0/608883669 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7fdf780f2470 con 0x7fdf88102b90 2024-04-02T14:00:42.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.482+0000 7fdf907e0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cbf2e95820 con 0x55cbf2f89ae0 2024-04-02T14:00:42.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.482+0000 7fdf907e0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cbf2ee97e0 con 0x55cbf2f92fe0 2024-04-02T14:00:42.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.482+0000 7fdf87fff640 1 --2- 172.21.15.52:0/608883669 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdf700a3da0 0x7fdf700a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.482+0000 7fdf86ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f8a400 0x55cbf2f92aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.482+0000 7fdf877fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbf2f89ae0 0x55cbf2f89ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.482+0000 7fdf86ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f8a400 0x55cbf2f92aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56080/0 (socket says 172.21.15.52:56080) 2024-04-02T14:00:42.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.482+0000 7fdf86ffd640 1 -- 172.21.15.52:0/1608444081 learned_addr learned my addr 172.21.15.52:0/1608444081 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:42.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.482+0000 7fdf87fff640 1 --2- 172.21.15.52:0/1608444081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbf2f92fe0 0x55cbf2f953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.483+0000 7fdf877fe640 1 -- 172.21.15.52:0/1608444081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbf2f92fe0 msgr2=0x55cbf2f953d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:42.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.483+0000 7fdf877fe640 1 --2- 172.21.15.52:0/1608444081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbf2f92fe0 0x55cbf2f953d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.483+0000 7fdf877fe640 1 -- 172.21.15.52:0/1608444081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f8a400 msgr2=0x55cbf2f92aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:42.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.483+0000 7fdf877fe640 1 --2- 172.21.15.52:0/1608444081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f8a400 0x55cbf2f92aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.483+0000 7fdf877fe640 1 -- 172.21.15.52:0/1608444081 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbf2ccb7d0 con 0x55cbf2f89ae0 2024-04-02T14:00:42.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.483+0000 7fdf87fff640 1 --2- 172.21.15.52:0/1608444081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbf2f92fe0 0x55cbf2f953d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:00:42.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.483+0000 7fdf87fff640 1 --2- 172.21.15.52:0/608883669 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdf700a3da0 0x7fdf700a6260 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009fd0 tx=0x7fdf80013b00 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:42.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.483+0000 7fdf877fe640 1 --2- 172.21.15.52:0/1608444081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbf2f89ae0 0x55cbf2f89ec0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdf780ef9e0 tx=0x7fdf780ba580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8b35a6aab1fae94 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:42.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.483+0000 7fdf6dffb640 1 -- 172.21.15.52:0/1608444081 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7804b040 con 0x55cbf2f89ae0 2024-04-02T14:00:42.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.483+0000 7fdf6dffb640 1 -- 172.21.15.52:0/1608444081 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf78004c80 con 0x55cbf2f89ae0 2024-04-02T14:00:42.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.483+0000 7fdf6dffb640 1 -- 172.21.15.52:0/1608444081 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf780f2170 con 0x55cbf2f89ae0 2024-04-02T14:00:42.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.484+0000 7fdf907e0740 1 -- 172.21.15.52:0/1608444081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbf2f89ae0 msgr2=0x55cbf2f89ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:42.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.484+0000 7fdf907e0740 1 --2- 172.21.15.52:0/1608444081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbf2f89ae0 0x55cbf2f89ec0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdf780ef9e0 tx=0x7fdf780ba580 comp rx=0 tx=0).stop 2024-04-02T14:00:42.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.484+0000 7fdf907e0740 1 -- 172.21.15.52:0/1608444081 shutdown_connections 2024-04-02T14:00:42.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.484+0000 7fdf907e0740 1 --2- 172.21.15.52:0/1608444081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbf2f92fe0 0x55cbf2f953d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.484+0000 7fdf907e0740 1 --2- 172.21.15.52:0/1608444081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f8a400 0x55cbf2f92aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.484+0000 7fdf907e0740 1 --2- 172.21.15.52:0/1608444081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbf2f89ae0 0x55cbf2f89ec0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.484+0000 7fdf907e0740 1 -- 172.21.15.52:0/1608444081 >> 172.21.15.52:0/1608444081 conn(0x55cbf2f84210 msgr2=0x55cbf2f845f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:42.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.484+0000 7fdf907e0740 1 -- 172.21.15.52:0/1608444081 shutdown_connections 2024-04-02T14:00:42.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.484+0000 7fdf907e0740 1 -- 172.21.15.52:0/1608444081 wait complete. 2024-04-02T14:00:42.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.484+0000 7fdf907e0740 1 -- start start 2024-04-02T14:00:42.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.484+0000 7fdf907e0740 1 Processor -- start 2024-04-02T14:00:42.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.485+0000 7fdf907e0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbf2f89ae0 0x55cbf2f92930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.485+0000 7fdf907e0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f8a400 0x55cbf2f8fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.485+0000 7fdf907e0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbf2f92fe0 0x55cbf2f90220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.485+0000 7fdf907e0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cbf2f95b90 con 0x55cbf2f8a400 2024-04-02T14:00:42.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.485+0000 7fdf907e0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cbf2ea8d30 con 0x55cbf2f92fe0 2024-04-02T14:00:42.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.485+0000 7fdf907e0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cbf2f95d10 con 0x55cbf2f89ae0 2024-04-02T14:00:42.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.485+0000 7fdf907e0740 10 client.0 _ll_register_callbacks cb 0x55cbf2ea8720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:42.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.485+0000 7fdf86ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f8a400 0x55cbf2f8fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.485+0000 7fdf86ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f8a400 0x55cbf2f8fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56092/0 (socket says 172.21.15.52:56092) 2024-04-02T14:00:42.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.485+0000 7fdf86ffd640 1 -- 172.21.15.52:0/1123278245 learned_addr learned my addr 172.21.15.52:0/1123278245 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:42.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.485+0000 7fdf877fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbf2f89ae0 0x55cbf2f92930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.485+0000 7fdf87fff640 1 --2- 172.21.15.52:0/1123278245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbf2f92fe0 0x55cbf2f90220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.485+0000 7fdf86ffd640 1 -- 172.21.15.52:0/1123278245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbf2f89ae0 msgr2=0x55cbf2f92930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:42.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.485+0000 7fdf86ffd640 1 --2- 172.21.15.52:0/1123278245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbf2f89ae0 0x55cbf2f92930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.485+0000 7fdf86ffd640 1 -- 172.21.15.52:0/1123278245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbf2f92fe0 msgr2=0x55cbf2f90220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:42.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.485+0000 7fdf86ffd640 1 --2- 172.21.15.52:0/1123278245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbf2f92fe0 0x55cbf2f90220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.485+0000 7fdf86ffd640 1 -- 172.21.15.52:0/1123278245 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbf2c013f0 con 0x55cbf2f8a400 2024-04-02T14:00:42.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.485+0000 7fdf86ffd640 1 --2- 172.21.15.52:0/1123278245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f8a400 0x55cbf2f8fcc0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c012740 tx=0x7fdf7c012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9816a1348b0ade39 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:42.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.485+0000 7fdf66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:42.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.486+0000 7fdf66ffd640 1 -- 172.21.15.52:0/1123278245 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c004050 con 0x55cbf2f8a400 2024-04-02T14:00:42.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.486+0000 7fdf66ffd640 1 -- 172.21.15.52:0/1123278245 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7fdf7c0041f0 con 0x55cbf2f8a400 2024-04-02T14:00:42.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.486+0000 7fdf907e0740 1 -- 172.21.15.52:0/1123278245 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf78004c80 con 0x55cbf2f8a400 2024-04-02T14:00:42.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.486+0000 7fdf66ffd640 1 -- 172.21.15.52:0/1123278245 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdf7c025050 con 0x55cbf2f8a400 2024-04-02T14:00:42.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.486+0000 7fdf907e0740 10 client.17736 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:42.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.486+0000 7fdf907e0740 1 -- 172.21.15.52:0/1123278245 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cbf2bc10f0 con 0x55cbf2f8a400 2024-04-02T14:00:42.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.486+0000 7fdf66ffd640 1 -- 172.21.15.52:0/1123278245 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf78004c80 con 0x55cbf2f8a400 2024-04-02T14:00:42.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.486+0000 7fdf66ffd640 1 -- 172.21.15.52:0/1123278245 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fdf7c003a70 con 0x55cbf2f8a400 2024-04-02T14:00:42.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.486+0000 7fdf907e0740 10 client.17736 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:42.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.486+0000 7fdf907e0740 10 client.17736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:00:42.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.486+0000 7fdf907e0740 20 client.17736 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:42.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.486+0000 7fdf907e0740 1 --2- 172.21.15.52:0/1123278245 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cbf2fa2d40 0x55cbf2fa5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.486+0000 7fdf907e0740 4 client.17736 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:00:42.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.486+0000 7fdf907e0740 1 -- 172.21.15.52:0/1123278245 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbf2efad70 con 0x55cbf2fa2d40 2024-04-02T14:00:42.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.487+0000 7fdf877fe640 1 --2- 172.21.15.52:0/1123278245 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cbf2fa2d40 0x55cbf2fa5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.487+0000 7fdf66ffd640 10 client.17736 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:42.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.487+0000 7fdf877fe640 1 --2- 172.21.15.52:0/1123278245 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cbf2fa2d40 0x55cbf2fa5180 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f22c4c93ac0167ae server_cookie=9872d1ec529f1bc8 in_seq=0 out_seq=0 2024-04-02T14:00:42.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.488+0000 7fdf66ffd640 1 -- 172.21.15.52:0/1123278245 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cbf2efad70 con 0x55cbf2fa2d40 2024-04-02T14:00:42.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.488+0000 7fdf66ffd640 10 client.17736 handle_command_reply: tid=0 2024-04-02T14:00:42.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.488+0000 7fdf907e0740 1 client.17736 shutdown 2024-04-02T14:00:42.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.488+0000 7fdf907e0740 1 -- 172.21.15.52:0/1123278245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f8a400 msgr2=0x55cbf2f8fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:42.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.488+0000 7fdf907e0740 1 --2- 172.21.15.52:0/1123278245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f8a400 0x55cbf2f8fcc0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c012740 tx=0x7fdf7c012c00 comp rx=0 tx=0).stop 2024-04-02T14:00:42.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.488+0000 7fdf907e0740 1 -- 172.21.15.52:0/1123278245 shutdown_connections 2024-04-02T14:00:42.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.488+0000 7fdf907e0740 1 --2- 172.21.15.52:0/1123278245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbf2f92fe0 0x55cbf2f90220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.488+0000 7fdf907e0740 1 --2- 172.21.15.52:0/1123278245 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cbf2fa2d40 0x55cbf2fa5180 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.488+0000 7fdf907e0740 1 --2- 172.21.15.52:0/1123278245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f8a400 0x55cbf2f8fcc0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.488+0000 7fdf66ffd640 0 client.17736 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:42.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.488+0000 7fdf907e0740 1 --2- 172.21.15.52:0/1123278245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbf2f89ae0 0x55cbf2f92930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.488+0000 7fdf907e0740 1 -- 172.21.15.52:0/1123278245 >> 172.21.15.52:0/1123278245 conn(0x55cbf2f84210 msgr2=0x55cbf2f83490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:42.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.488+0000 7fdf907e0740 1 -- 172.21.15.52:0/1123278245 shutdown_connections 2024-04-02T14:00:42.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.488+0000 7fdf907e0740 1 -- 172.21.15.52:0/1123278245 wait complete. 2024-04-02T14:00:42.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.488+0000 7fdf907e0740 20 client.17736 trim_cache size 0 max 16384 2024-04-02T14:00:42.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.509+0000 7fdf907e0740 1 Processor -- start 2024-04-02T14:00:42.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.510+0000 7fdf907e0740 1 -- start start 2024-04-02T14:00:42.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.510+0000 7fdf907e0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f89ae0 0x55cbf2f8b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.510+0000 7fdf907e0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbf2f8a400 0x55cbf2f923f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.510+0000 7fdf907e0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbf2f92fe0 0x55cbf2f92930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.510+0000 7fdf907e0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdf881b0f20 con 0x55cbf2f89ae0 2024-04-02T14:00:42.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.510+0000 7fdf907e0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdf881b2a60 con 0x55cbf2f8a400 2024-04-02T14:00:42.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.510+0000 7fdf907e0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdf881b15f0 con 0x55cbf2f92fe0 2024-04-02T14:00:42.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.510+0000 7fdf877fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f89ae0 0x55cbf2f8b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.510+0000 7fdf877fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f89ae0 0x55cbf2f8b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56094/0 (socket says 172.21.15.52:56094) 2024-04-02T14:00:42.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.510+0000 7fdf877fe640 1 -- 172.21.15.52:0/3378851148 learned_addr learned my addr 172.21.15.52:0/3378851148 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:42.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.510+0000 7fdf86ffd640 1 --2- 172.21.15.52:0/3378851148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbf2f8a400 0x55cbf2f923f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.510+0000 7fdf87fff640 1 --2- 172.21.15.52:0/3378851148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbf2f92fe0 0x55cbf2f92930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.510+0000 7fdf877fe640 1 -- 172.21.15.52:0/3378851148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbf2f92fe0 msgr2=0x55cbf2f92930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:42.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.510+0000 7fdf877fe640 1 --2- 172.21.15.52:0/3378851148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbf2f92fe0 0x55cbf2f92930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.510+0000 7fdf877fe640 1 -- 172.21.15.52:0/3378851148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbf2f8a400 msgr2=0x55cbf2f923f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:42.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.510+0000 7fdf877fe640 1 --2- 172.21.15.52:0/3378851148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbf2f8a400 0x55cbf2f923f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.510+0000 7fdf877fe640 1 -- 172.21.15.52:0/3378851148 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbf2f953d0 con 0x55cbf2f89ae0 2024-04-02T14:00:42.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.511+0000 7fdf877fe640 1 --2- 172.21.15.52:0/3378851148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f89ae0 0x55cbf2f8b150 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fdf780bc590 tx=0x7fdf780ba6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e97a6d6f241657 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:42.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.511+0000 7fdf6d7fa640 1 -- 172.21.15.52:0/3378851148 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7804b040 con 0x55cbf2f89ae0 2024-04-02T14:00:42.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.511+0000 7fdf6d7fa640 1 -- 172.21.15.52:0/3378851148 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf78004c80 con 0x55cbf2f89ae0 2024-04-02T14:00:42.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.511+0000 7fdf6d7fa640 1 -- 172.21.15.52:0/3378851148 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf780efb50 con 0x55cbf2f89ae0 2024-04-02T14:00:42.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.511+0000 7fdf907e0740 1 -- 172.21.15.52:0/3378851148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f89ae0 msgr2=0x55cbf2f8b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:42.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.511+0000 7fdf907e0740 1 --2- 172.21.15.52:0/3378851148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f89ae0 0x55cbf2f8b150 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fdf780bc590 tx=0x7fdf780ba6b0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.512+0000 7fdf907e0740 1 -- 172.21.15.52:0/3378851148 shutdown_connections 2024-04-02T14:00:42.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.512+0000 7fdf907e0740 1 --2- 172.21.15.52:0/3378851148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbf2f92fe0 0x55cbf2f92930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.512+0000 7fdf907e0740 1 --2- 172.21.15.52:0/3378851148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbf2f8a400 0x55cbf2f923f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.512+0000 7fdf907e0740 1 --2- 172.21.15.52:0/3378851148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f89ae0 0x55cbf2f8b150 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.512+0000 7fdf907e0740 1 -- 172.21.15.52:0/3378851148 >> 172.21.15.52:0/3378851148 conn(0x55cbf2f84210 msgr2=0x55cbf2f8aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:42.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.512+0000 7fdf907e0740 1 -- 172.21.15.52:0/3378851148 shutdown_connections 2024-04-02T14:00:42.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.512+0000 7fdf907e0740 1 -- 172.21.15.52:0/3378851148 wait complete. 2024-04-02T14:00:42.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.512+0000 7fdf907e0740 1 -- start start 2024-04-02T14:00:42.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.512+0000 7fdf907e0740 1 Processor -- start 2024-04-02T14:00:42.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.512+0000 7fdf907e0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbf2f89ae0 0x55cbf2f9af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.512+0000 7fdf907e0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f8a400 0x55cbf2f8c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.512+0000 7fdf907e0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbf2f92fe0 0x55cbf2f8ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.512+0000 7fdf907e0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdf881b2390 con 0x55cbf2f8a400 2024-04-02T14:00:42.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.512+0000 7fdf907e0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdf881b1cc0 con 0x55cbf2f89ae0 2024-04-02T14:00:42.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.513+0000 7fdf907e0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdf881aef20 con 0x55cbf2f92fe0 2024-04-02T14:00:42.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.513+0000 7fdf907e0740 10 client.0 _ll_register_callbacks cb 0x7fdf7c009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:00:42.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.513+0000 7fdf86ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f8a400 0x55cbf2f8c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.513+0000 7fdf877fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbf2f89ae0 0x55cbf2f9af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.513+0000 7fdf86ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f8a400 0x55cbf2f8c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56100/0 (socket says 172.21.15.52:56100) 2024-04-02T14:00:42.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.513+0000 7fdf86ffd640 1 -- 172.21.15.52:0/620097345 learned_addr learned my addr 172.21.15.52:0/620097345 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:00:42.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.513+0000 7fdf87fff640 1 --2- 172.21.15.52:0/620097345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbf2f92fe0 0x55cbf2f8ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.513+0000 7fdf86ffd640 1 -- 172.21.15.52:0/620097345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbf2f92fe0 msgr2=0x55cbf2f8ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:42.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.513+0000 7fdf86ffd640 1 --2- 172.21.15.52:0/620097345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbf2f92fe0 0x55cbf2f8ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.513+0000 7fdf86ffd640 1 -- 172.21.15.52:0/620097345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbf2f89ae0 msgr2=0x55cbf2f9af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:00:42.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.513+0000 7fdf86ffd640 1 --2- 172.21.15.52:0/620097345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbf2f89ae0 0x55cbf2f9af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.513+0000 7fdf86ffd640 1 -- 172.21.15.52:0/620097345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbf2f90030 con 0x55cbf2f8a400 2024-04-02T14:00:42.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.513+0000 7fdf877fe640 1 --2- 172.21.15.52:0/620097345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbf2f89ae0 0x55cbf2f9af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:00:42.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.513+0000 7fdf86ffd640 1 --2- 172.21.15.52:0/620097345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f8a400 0x55cbf2f8c550 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x55cbf2f91290 tx=0x7fdf7c00fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836f6e9983d0d73c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:00:42.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.513+0000 7fdf66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:00:42.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.513+0000 7fdf66ffd640 1 -- 172.21.15.52:0/620097345 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c02d070 con 0x55cbf2f8a400 2024-04-02T14:00:42.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.513+0000 7fdf66ffd640 1 -- 172.21.15.52:0/620097345 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9438+0+0 (secure 0 0 0) 0x7fdf7c0035f0 con 0x55cbf2f8a400 2024-04-02T14:00:42.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.513+0000 7fdf907e0740 1 -- 172.21.15.52:0/620097345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf78004c80 con 0x55cbf2f8a400 2024-04-02T14:00:42.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.514+0000 7fdf66ffd640 1 -- 172.21.15.52:0/620097345 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdf7c03a050 con 0x55cbf2f8a400 2024-04-02T14:00:42.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.514+0000 7fdf907e0740 10 client.17742 fetch_fsmap learned FSMap version 30 2024-04-02T14:00:42.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.514+0000 7fdf907e0740 1 -- 172.21.15.52:0/620097345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cbf3008c10 con 0x55cbf2f8a400 2024-04-02T14:00:42.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.514+0000 7fdf66ffd640 1 -- 172.21.15.52:0/620097345 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf78004c80 con 0x55cbf2f8a400 2024-04-02T14:00:42.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.514+0000 7fdf66ffd640 1 -- 172.21.15.52:0/620097345 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fdf7c0055c0 con 0x55cbf2f8a400 2024-04-02T14:00:42.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.514+0000 7fdf907e0740 10 client.17742 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:00:42.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.514+0000 7fdf907e0740 10 client.17742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:00:42.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.514+0000 7fdf907e0740 20 client.17742 populate_metadata read hostname 'smithi052' 2024-04-02T14:00:42.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.514+0000 7fdf907e0740 1 --2- 172.21.15.52:0/620097345 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cbf2fa2d40 0x55cbf300b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:00:42.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.514+0000 7fdf907e0740 4 client.17742 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:00:42.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.514+0000 7fdf907e0740 1 -- 172.21.15.52:0/620097345 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cbf2ef50c0 con 0x55cbf2fa2d40 2024-04-02T14:00:42.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.515+0000 7fdf877fe640 1 --2- 172.21.15.52:0/620097345 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cbf2fa2d40 0x55cbf300b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:00:42.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.515+0000 7fdf66ffd640 10 client.17742 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:42.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.515+0000 7fdf877fe640 1 --2- 172.21.15.52:0/620097345 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cbf2fa2d40 0x55cbf300b370 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d8dd9d9c17bf765 server_cookie=a42213e4e20a6319 in_seq=0 out_seq=0 2024-04-02T14:00:42.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.515+0000 7fdf66ffd640 1 -- 172.21.15.52:0/620097345 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55cbf2ef50c0 con 0x55cbf2fa2d40 2024-04-02T14:00:42.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.515+0000 7fdf66ffd640 10 client.17742 handle_command_reply: tid=0 2024-04-02T14:00:42.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.515+0000 7fdf907e0740 1 client.17742 shutdown 2024-04-02T14:00:42.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.516+0000 7fdf907e0740 1 -- 172.21.15.52:0/620097345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f8a400 msgr2=0x55cbf2f8c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:42.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.516+0000 7fdf907e0740 1 --2- 172.21.15.52:0/620097345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f8a400 0x55cbf2f8c550 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x55cbf2f91290 tx=0x7fdf7c00fbd0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.516+0000 7fdf907e0740 1 -- 172.21.15.52:0/620097345 shutdown_connections 2024-04-02T14:00:42.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.516+0000 7fdf907e0740 1 --2- 172.21.15.52:0/620097345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbf2f92fe0 0x55cbf2f8ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.516+0000 7fdf907e0740 1 --2- 172.21.15.52:0/620097345 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cbf2fa2d40 0x55cbf300b370 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.516+0000 7fdf907e0740 1 --2- 172.21.15.52:0/620097345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbf2f8a400 0x55cbf2f8c550 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.516+0000 7fdf907e0740 1 --2- 172.21.15.52:0/620097345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cbf2f89ae0 0x55cbf2f9af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.516+0000 7fdf907e0740 1 -- 172.21.15.52:0/620097345 >> 172.21.15.52:0/620097345 conn(0x55cbf2f84210 msgr2=0x55cbf2f83490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:42.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.516+0000 7fdf66ffd640 0 client.17742 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:00:42.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.516+0000 7fdf907e0740 1 -- 172.21.15.52:0/620097345 shutdown_connections 2024-04-02T14:00:42.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.516+0000 7fdf907e0740 1 -- 172.21.15.52:0/620097345 wait complete. 2024-04-02T14:00:42.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.516+0000 7fdf907e0740 20 client.17742 trim_cache size 0 max 16384 2024-04-02T14:00:42.531 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:00:42.532 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "scrub active (2409 inodes in the stack)", 2024-04-02T14:00:42.532 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-04-02T14:00:42.532 INFO:teuthology.orchestra.run.smithi052.stdout: "1e34f335-e306-4f37-9a0b-d19dfdec426a": { 2024-04-02T14:00:42.532 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-04-02T14:00:42.532 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "1e34f335-e306-4f37-9a0b-d19dfdec426a", 2024-04-02T14:00:42.532 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-04-02T14:00:42.532 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:00:42.532 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:00:42.533 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:00:42.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.516+0000 7fdf8c9a1640 1 -- 172.21.15.52:0/608883669 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdf700a3da0 msgr2=0x7fdf700a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:42.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.516+0000 7fdf8c9a1640 1 --2- 172.21.15.52:0/608883669 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdf700a3da0 0x7fdf700a6260 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009fd0 tx=0x7fdf80013b00 comp rx=0 tx=0).stop 2024-04-02T14:00:42.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.517+0000 7fdf8c9a1640 1 -- 172.21.15.52:0/608883669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf88102b90 msgr2=0x7fdf881982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:00:42.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.517+0000 7fdf8c9a1640 1 --2- 172.21.15.52:0/608883669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf88102b90 0x7fdf881982a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdf78005940 tx=0x7fdf7802cbd0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.517+0000 7fdf8c9a1640 1 -- 172.21.15.52:0/608883669 shutdown_connections 2024-04-02T14:00:42.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.517+0000 7fdf8c9a1640 1 --2- 172.21.15.52:0/608883669 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdf700a3da0 0x7fdf700a6260 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.517+0000 7fdf8c9a1640 1 --2- 172.21.15.52:0/608883669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf8810bf60 0x7fdf8819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.517+0000 7fdf8c9a1640 1 --2- 172.21.15.52:0/608883669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf881034b0 0x7fdf881987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.517+0000 7fdf8c9a1640 1 --2- 172.21.15.52:0/608883669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf88102b90 0x7fdf881982a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:00:42.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.517+0000 7fdf8c9a1640 1 -- 172.21.15.52:0/608883669 >> 172.21.15.52:0/608883669 conn(0x7fdf880fc350 msgr2=0x7fdf880fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:00:42.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.517+0000 7fdf8c9a1640 1 -- 172.21.15.52:0/608883669 shutdown_connections 2024-04-02T14:00:42.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:00:42.517+0000 7fdf8c9a1640 1 -- 172.21.15.52:0/608883669 wait complete. 2024-04-02T14:00:42.535 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:42 smithi052 ceph-mon[32104]: scrub queued for path: / 2024-04-02T14:00:42.535 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:42 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:00:42.536 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:42 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T14:00:42.537 INFO:tasks.cephfs.filesystem:scrub status for tag:1e34f335-e306-4f37-9a0b-d19dfdec426a - {'path': '/', 'tag': '1e34f335-e306-4f37-9a0b-d19dfdec426a', 'options': 'recursive,force'} 2024-04-02T14:00:42.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:42 smithi189 ceph-mon[36683]: scrub queued for path: / 2024-04-02T14:00:42.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:42 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:00:42.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:42 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T14:00:42.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:42 smithi088 ceph-mon[36528]: scrub queued for path: / 2024-04-02T14:00:42.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:42 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:00:42.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:42 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T14:00:43.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:43 smithi189 ceph-mon[36683]: pgmap v221: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:00:43.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:43 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T14:00:43.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:43 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:00:43.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:43 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T14:00:43.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:43 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:00:43.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:43 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:00:43.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:43 smithi052 ceph-mon[32104]: pgmap v221: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:00:43.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:43 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T14:00:43.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:43 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:00:43.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:43 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T14:00:43.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:43 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:00:43.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:43 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:00:43.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:43 smithi088 ceph-mon[36528]: pgmap v221: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:00:43.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:43 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T14:00:43.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:43 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:00:43.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:43 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T14:00:43.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:43 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:00:43.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:43 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:00:45.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:45 smithi189 ceph-mon[36683]: pgmap v222: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:00:45.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:45 smithi052 ceph-mon[32104]: pgmap v222: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:00:45.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:45 smithi088 ceph-mon[36528]: pgmap v222: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:00:47.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:47 smithi189 ceph-mon[36683]: pgmap v223: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T14:00:47.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:47 smithi052 ceph-mon[32104]: pgmap v223: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T14:00:47.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:47 smithi088 ceph-mon[36528]: pgmap v223: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T14:00:49.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:49 smithi189 ceph-mon[36683]: pgmap v224: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:00:49.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:49 smithi052 ceph-mon[32104]: pgmap v224: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:00:49.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:49 smithi088 ceph-mon[36528]: pgmap v224: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:00:50.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:50 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T14:00:50.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:50 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T14:00:50.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:50 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T14:00:51.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:51 smithi189 ceph-mon[36683]: pgmap v225: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:00:51.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:51 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T14:00:51.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:51 smithi052 ceph-mon[32104]: pgmap v225: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:00:51.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:51 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T14:00:51.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:51 smithi088 ceph-mon[36528]: pgmap v225: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:00:51.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:51 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T14:00:53.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:53 smithi189 ceph-mon[36683]: pgmap v226: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T14:00:53.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:53 smithi052 ceph-mon[32104]: pgmap v226: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T14:00:53.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:53 smithi088 ceph-mon[36528]: pgmap v226: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T14:00:55.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:55 smithi189 ceph-mon[36683]: pgmap v227: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:00:55.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:55 smithi052 ceph-mon[32104]: pgmap v227: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:00:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:55 smithi088 ceph-mon[36528]: pgmap v227: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:00:57.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:57 smithi189 ceph-mon[36683]: pgmap v228: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T14:00:57.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:57 smithi052 ceph-mon[32104]: pgmap v228: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T14:00:57.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:57 smithi088 ceph-mon[36528]: pgmap v228: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T14:00:59.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:00:59 smithi189 ceph-mon[36683]: pgmap v229: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:00:59.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:00:59 smithi052 ceph-mon[32104]: pgmap v229: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:00:59.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:00:59 smithi088 ceph-mon[36528]: pgmap v229: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:01.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:01 smithi052 ceph-mon[32104]: pgmap v230: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:01.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:01 smithi088 ceph-mon[36528]: pgmap v230: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:01.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:01 smithi189 ceph-mon[36683]: pgmap v230: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:03.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:03 smithi189 ceph-mon[36683]: pgmap v231: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:01:03.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:03 smithi052 ceph-mon[32104]: pgmap v231: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:01:03.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:03 smithi088 ceph-mon[36528]: pgmap v231: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:01:05.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:05 smithi189 ceph-mon[36683]: pgmap v232: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:01:05.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:05 smithi052 ceph-mon[32104]: pgmap v232: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:01:05.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:05 smithi088 ceph-mon[36528]: pgmap v232: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:01:07.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:07 smithi189 ceph-mon[36683]: pgmap v233: 129 pgs: 129 active+clean; 73 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:01:07.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:07 smithi052 ceph-mon[32104]: pgmap v233: 129 pgs: 129 active+clean; 73 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:01:07.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:07 smithi088 ceph-mon[36528]: pgmap v233: 129 pgs: 129 active+clean; 73 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:01:09.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:09 smithi189 ceph-mon[36683]: pgmap v234: 129 pgs: 129 active+clean; 73 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:01:09.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:09 smithi052 ceph-mon[32104]: pgmap v234: 129 pgs: 129 active+clean; 73 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:01:09.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:09 smithi088 ceph-mon[36528]: pgmap v234: 129 pgs: 129 active+clean; 73 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:01:11.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:11 smithi189 ceph-mon[36683]: pgmap v235: 129 pgs: 129 active+clean; 73 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:01:11.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:11 smithi189 ceph-mon[36683]: osdmap e96: 12 total, 12 up, 12 in 2024-04-02T14:01:11.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:11 smithi052 ceph-mon[32104]: pgmap v235: 129 pgs: 129 active+clean; 73 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:01:11.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:11 smithi052 ceph-mon[32104]: osdmap e96: 12 total, 12 up, 12 in 2024-04-02T14:01:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:11 smithi088 ceph-mon[36528]: pgmap v235: 129 pgs: 129 active+clean; 73 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:01:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:11 smithi088 ceph-mon[36528]: osdmap e96: 12 total, 12 up, 12 in 2024-04-02T14:01:12.538 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:01:12.538 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:01:12.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27bb2e4640 1 -- 172.21.15.52:0/1845013006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b4100970 msgr2=0x7f27b4100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:12.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27bb2e4640 1 --2- 172.21.15.52:0/1845013006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b4100970 0x7f27b4100df0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f27a4009930 tx=0x7f27a402bc30 comp rx=0 tx=0).stop 2024-04-02T14:01:12.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27bb2e4640 1 -- 172.21.15.52:0/1845013006 shutdown_connections 2024-04-02T14:01:12.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27bb2e4640 1 --2- 172.21.15.52:0/1845013006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27b4101330 0x7f27b410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27bb2e4640 1 --2- 172.21.15.52:0/1845013006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b4100970 0x7f27b4100df0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27bb2e4640 1 --2- 172.21.15.52:0/1845013006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27b4106980 0x7f27b4106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27bb2e4640 1 -- 172.21.15.52:0/1845013006 >> 172.21.15.52:0/1845013006 conn(0x7f27b4077b00 msgr2=0x7f27b4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:12.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27bb2e4640 1 -- 172.21.15.52:0/1845013006 shutdown_connections 2024-04-02T14:01:12.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27bb2e4640 1 -- 172.21.15.52:0/1845013006 wait complete. 2024-04-02T14:01:12.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27bb2e4640 1 Processor -- start 2024-04-02T14:01:12.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27bb2e4640 1 -- start start 2024-04-02T14:01:12.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27bb2e4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27b4100970 0x7f27b41972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:12.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27bb2e4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b4101330 0x7f27b41977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:12.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27bb2e4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27b4106980 0x7f27b419bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:12.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27bb2e4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f27b4110d90 con 0x7f27b4101330 2024-04-02T14:01:12.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27bb2e4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f27b4110c10 con 0x7f27b4106980 2024-04-02T14:01:12.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27bb2e4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f27b4110f10 con 0x7f27b4100970 2024-04-02T14:01:12.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27b887f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b4101330 0x7f27b41977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:12.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27b887f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b4101330 0x7f27b41977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55976/0 (socket says 172.21.15.52:55976) 2024-04-02T14:01:12.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27b887f640 1 -- 172.21.15.52:0/1209934209 learned_addr learned my addr 172.21.15.52:0/1209934209 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:12.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27b9080640 1 --2- 172.21.15.52:0/1209934209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27b4100970 0x7f27b41972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:12.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27b9881640 1 --2- 172.21.15.52:0/1209934209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27b4106980 0x7f27b419bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:12.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27b887f640 1 -- 172.21.15.52:0/1209934209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27b4100970 msgr2=0x7f27b41972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:12.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27b887f640 1 --2- 172.21.15.52:0/1209934209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27b4100970 0x7f27b41972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27b887f640 1 -- 172.21.15.52:0/1209934209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27b4106980 msgr2=0x7f27b419bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:12.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27b887f640 1 --2- 172.21.15.52:0/1209934209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27b4106980 0x7f27b419bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27b887f640 1 -- 172.21.15.52:0/1209934209 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b419c2a0 con 0x7f27b4101330 2024-04-02T14:01:12.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27b9881640 1 --2- 172.21.15.52:0/1209934209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27b4106980 0x7f27b419bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:12.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27b9080640 1 --2- 172.21.15.52:0/1209934209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27b4100970 0x7f27b41972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:12.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27b887f640 1 --2- 172.21.15.52:0/1209934209 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b4101330 0x7f27b41977f0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f27a4009230 tx=0x7f27a402cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:12.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27aa7fc640 1 -- 172.21.15.52:0/1209934209 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a4041070 con 0x7f27b4101330 2024-04-02T14:01:12.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27aa7fc640 1 -- 172.21.15.52:0/1209934209 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27a4004450 con 0x7f27b4101330 2024-04-02T14:01:12.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27bb2e4640 1 -- 172.21.15.52:0/1209934209 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b419c530 con 0x7f27b4101330 2024-04-02T14:01:12.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27aa7fc640 1 -- 172.21.15.52:0/1209934209 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a4034440 con 0x7f27b4101330 2024-04-02T14:01:12.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.654+0000 7f27bb2e4640 1 -- 172.21.15.52:0/1209934209 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b41a3e20 con 0x7f27b4101330 2024-04-02T14:01:12.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.658+0000 7f27aa7fc640 1 -- 172.21.15.52:0/1209934209 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f27a4044070 con 0x7f27b4101330 2024-04-02T14:01:12.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.658+0000 7f27bdfa5740 1 Processor -- start 2024-04-02T14:01:12.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.658+0000 7f27bdfa5740 1 -- start start 2024-04-02T14:01:12.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.658+0000 7f27bdfa5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcc98d0 0x55707bcc9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:12.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.658+0000 7f27aa7fc640 1 --2- 172.21.15.52:0/1209934209 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f27a00a3da0 0x7f27a00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:12.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.658+0000 7f27bdfa5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55707bcca1f0 0x55707bcd2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:12.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.658+0000 7f27bdfa5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55707bcd2dd0 0x55707bcd51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:12.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.658+0000 7f27bdfa5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55707bc2d830 con 0x55707bcc98d0 2024-04-02T14:01:12.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.658+0000 7f27bdfa5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55707bbd5820 con 0x55707bcd2dd0 2024-04-02T14:01:12.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.658+0000 7f27bdfa5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55707bc297e0 con 0x55707bcca1f0 2024-04-02T14:01:12.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.658+0000 7f27b9881640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcc98d0 0x55707bcc9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:12.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.658+0000 7f27b9080640 1 --2- 172.21.15.52:0/1209934209 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f27a00a3da0 0x7f27a00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:12.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.658+0000 7f27b9881640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcc98d0 0x55707bcc9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55986/0 (socket says 172.21.15.52:55986) 2024-04-02T14:01:12.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.658+0000 7f27b9881640 1 -- 172.21.15.52:0/1166729226 learned_addr learned my addr 172.21.15.52:0/1166729226 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:12.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.658+0000 7f27aa7fc640 1 -- 172.21.15.52:0/1209934209 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9526+0+0 (secure 0 0 0) 0x7f27a40f25e0 con 0x7f27b4101330 2024-04-02T14:01:12.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.658+0000 7f27b887f640 1 --2- 172.21.15.52:0/1166729226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55707bcca1f0 0x55707bcd2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:12.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.658+0000 7f27b9881640 1 --2- 172.21.15.52:0/1166729226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55707bcd2dd0 0x55707bcd51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:12.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.658+0000 7f27b9881640 1 -- 172.21.15.52:0/1166729226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55707bcca1f0 msgr2=0x55707bcd2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:12.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.658+0000 7f27b9881640 1 --2- 172.21.15.52:0/1166729226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55707bcca1f0 0x55707bcd2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.658+0000 7f27b9881640 1 -- 172.21.15.52:0/1166729226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55707bcd2dd0 msgr2=0x55707bcd51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:12.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.658+0000 7f27b9881640 1 --2- 172.21.15.52:0/1166729226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55707bcd2dd0 0x55707bcd51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.658+0000 7f27b9881640 1 -- 172.21.15.52:0/1166729226 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55707ba0b7d0 con 0x55707bcc98d0 2024-04-02T14:01:12.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.658+0000 7f27b9080640 1 --2- 172.21.15.52:0/1209934209 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f27a00a3da0 0x7f27a00a6260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f27b0005fd0 tx=0x7f27b000b5d0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:12.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.658+0000 7f27b9881640 1 --2- 172.21.15.52:0/1166729226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcc98d0 0x55707bcc9cb0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f27ac012810 tx=0x7f27ac00a900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca8441b437ed608 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:12.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.658+0000 7f279f7fe640 1 -- 172.21.15.52:0/1166729226 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac018070 con 0x55707bcc98d0 2024-04-02T14:01:12.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.659+0000 7f279f7fe640 1 -- 172.21.15.52:0/1166729226 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27ac004d20 con 0x55707bcc98d0 2024-04-02T14:01:12.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.659+0000 7f279f7fe640 1 -- 172.21.15.52:0/1166729226 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac005690 con 0x55707bcc98d0 2024-04-02T14:01:12.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.659+0000 7f27bdfa5740 1 -- 172.21.15.52:0/1166729226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcc98d0 msgr2=0x55707bcc9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:12.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.659+0000 7f27bdfa5740 1 --2- 172.21.15.52:0/1166729226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcc98d0 0x55707bcc9cb0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f27ac012810 tx=0x7f27ac00a900 comp rx=0 tx=0).stop 2024-04-02T14:01:12.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.659+0000 7f27bdfa5740 1 -- 172.21.15.52:0/1166729226 shutdown_connections 2024-04-02T14:01:12.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.659+0000 7f27bdfa5740 1 --2- 172.21.15.52:0/1166729226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55707bcd2dd0 0x55707bcd51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.659+0000 7f27bdfa5740 1 --2- 172.21.15.52:0/1166729226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55707bcca1f0 0x55707bcd2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.659+0000 7f27bdfa5740 1 --2- 172.21.15.52:0/1166729226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcc98d0 0x55707bcc9cb0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.659+0000 7f27bdfa5740 1 -- 172.21.15.52:0/1166729226 >> 172.21.15.52:0/1166729226 conn(0x55707bcc4000 msgr2=0x55707bcc43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:12.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.659+0000 7f27bdfa5740 1 -- 172.21.15.52:0/1166729226 shutdown_connections 2024-04-02T14:01:12.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.659+0000 7f27bdfa5740 1 -- 172.21.15.52:0/1166729226 wait complete. 2024-04-02T14:01:12.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.660+0000 7f27bdfa5740 1 -- start start 2024-04-02T14:01:12.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.660+0000 7f27bdfa5740 1 Processor -- start 2024-04-02T14:01:12.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.660+0000 7f27bdfa5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcc98d0 0x55707bcd25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:12.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.660+0000 7f27bdfa5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55707bcca1f0 0x55707bccf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:12.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.660+0000 7f27bdfa5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55707bcd2dd0 0x55707bccfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:12.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.660+0000 7f27bdfa5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55707bcd5bc0 con 0x55707bcc98d0 2024-04-02T14:01:12.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.660+0000 7f27bdfa5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55707bbe8d30 con 0x55707bcd2dd0 2024-04-02T14:01:12.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.660+0000 7f27bdfa5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55707bcd5d40 con 0x55707bcca1f0 2024-04-02T14:01:12.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.660+0000 7f27bdfa5740 10 client.0 _ll_register_callbacks cb 0x55707bbe8720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:12.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.660+0000 7f27b9080640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcc98d0 0x55707bcd25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:12.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.660+0000 7f27b9080640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcc98d0 0x55707bcd25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55994/0 (socket says 172.21.15.52:55994) 2024-04-02T14:01:12.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.660+0000 7f27b9080640 1 -- 172.21.15.52:0/317005592 learned_addr learned my addr 172.21.15.52:0/317005592 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:12.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.660+0000 7f27b9881640 1 --2- 172.21.15.52:0/317005592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55707bcd2dd0 0x55707bccfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:12.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.661+0000 7f27b9080640 1 -- 172.21.15.52:0/317005592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55707bcca1f0 msgr2=0x55707bccf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:12.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.661+0000 7f27b9080640 1 --2- 172.21.15.52:0/317005592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55707bcca1f0 0x55707bccf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.661+0000 7f27b9080640 1 -- 172.21.15.52:0/317005592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55707bcd2dd0 msgr2=0x55707bccfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:12.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.661+0000 7f27b9080640 1 --2- 172.21.15.52:0/317005592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55707bcd2dd0 0x55707bccfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.661+0000 7f27b9080640 1 -- 172.21.15.52:0/317005592 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55707b9413f0 con 0x55707bcc98d0 2024-04-02T14:01:12.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.661+0000 7f27b9080640 1 --2- 172.21.15.52:0/317005592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcc98d0 0x55707bcd25c0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f27b0011490 tx=0x7f27b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae7398379453759 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:12.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.661+0000 7f279cff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:12.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.661+0000 7f279cff9640 1 -- 172.21.15.52:0/317005592 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b0022070 con 0x55707bcc98d0 2024-04-02T14:01:12.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.661+0000 7f279cff9640 1 -- 172.21.15.52:0/317005592 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9526+0+0 (secure 0 0 0) 0x7f27b001e860 con 0x55707bcc98d0 2024-04-02T14:01:12.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.661+0000 7f27bdfa5740 1 -- 172.21.15.52:0/317005592 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27ac004d20 con 0x55707bcc98d0 2024-04-02T14:01:12.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.661+0000 7f279cff9640 1 -- 172.21.15.52:0/317005592 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f27b002e050 con 0x55707bcc98d0 2024-04-02T14:01:12.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.661+0000 7f27bdfa5740 10 client.17754 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:12.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.661+0000 7f27bdfa5740 1 -- 172.21.15.52:0/317005592 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55707b9010f0 con 0x55707bcc98d0 2024-04-02T14:01:12.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.661+0000 7f279cff9640 1 -- 172.21.15.52:0/317005592 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27ac004d20 con 0x55707bcc98d0 2024-04-02T14:01:12.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.661+0000 7f279cff9640 1 -- 172.21.15.52:0/317005592 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f27b0010c10 con 0x55707bcc98d0 2024-04-02T14:01:12.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.662+0000 7f27bdfa5740 10 client.17754 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:12.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.662+0000 7f27bdfa5740 10 client.17754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:01:12.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.662+0000 7f27bdfa5740 20 client.17754 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:12.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.662+0000 7f27bdfa5740 1 --2- 172.21.15.52:0/317005592 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55707bce2c50 0x55707bce50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:12.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.662+0000 7f27bdfa5740 4 client.17754 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:01:12.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.662+0000 7f27bdfa5740 1 -- 172.21.15.52:0/317005592 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55707bc3ad70 con 0x55707bce2c50 2024-04-02T14:01:12.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.662+0000 7f27b887f640 1 --2- 172.21.15.52:0/317005592 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55707bce2c50 0x55707bce50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:12.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.662+0000 7f279cff9640 10 client.17754 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:12.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.662+0000 7f27b887f640 1 --2- 172.21.15.52:0/317005592 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55707bce2c50 0x55707bce50d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e5d5b074c92f088 server_cookie=5e7baa7d17fee84b in_seq=0 out_seq=0 2024-04-02T14:01:12.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.663+0000 7f279cff9640 1 -- 172.21.15.52:0/317005592 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55707bc3ad70 con 0x55707bce2c50 2024-04-02T14:01:12.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.663+0000 7f279cff9640 10 client.17754 handle_command_reply: tid=0 2024-04-02T14:01:12.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.663+0000 7f27bdfa5740 1 client.17754 shutdown 2024-04-02T14:01:12.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.663+0000 7f27bdfa5740 1 -- 172.21.15.52:0/317005592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcc98d0 msgr2=0x55707bcd25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:12.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.663+0000 7f27bdfa5740 1 --2- 172.21.15.52:0/317005592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcc98d0 0x55707bcd25c0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f27b0011490 tx=0x7f27b00046c0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.663+0000 7f27bdfa5740 1 -- 172.21.15.52:0/317005592 shutdown_connections 2024-04-02T14:01:12.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.663+0000 7f27bdfa5740 1 --2- 172.21.15.52:0/317005592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55707bcd2dd0 0x55707bccfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.663+0000 7f27bdfa5740 1 --2- 172.21.15.52:0/317005592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55707bcca1f0 0x55707bccf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.663+0000 7f27bdfa5740 1 --2- 172.21.15.52:0/317005592 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55707bce2c50 0x55707bce50d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.663+0000 7f27bdfa5740 1 --2- 172.21.15.52:0/317005592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcc98d0 0x55707bcd25c0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.663+0000 7f279cff9640 0 client.17754 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:12.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.663+0000 7f27bdfa5740 1 -- 172.21.15.52:0/317005592 >> 172.21.15.52:0/317005592 conn(0x55707bcc4000 msgr2=0x55707bcc5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:12.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.663+0000 7f27bdfa5740 1 -- 172.21.15.52:0/317005592 shutdown_connections 2024-04-02T14:01:12.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.663+0000 7f27bdfa5740 1 -- 172.21.15.52:0/317005592 wait complete. 2024-04-02T14:01:12.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.663+0000 7f27bdfa5740 20 client.17754 trim_cache size 0 max 16384 2024-04-02T14:01:12.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.681+0000 7f27bdfa5740 1 Processor -- start 2024-04-02T14:01:12.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.681+0000 7f27bdfa5740 1 -- start start 2024-04-02T14:01:12.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.681+0000 7f27bdfa5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55707bcc98d0 0x55707bcdea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:12.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.681+0000 7f27bdfa5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55707bcca1f0 0x55707bcdb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:12.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.681+0000 7f27bdfa5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcd2dd0 0x55707bcd2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:12.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.681+0000 7f27bdfa5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f27b41adf60 con 0x55707bcd2dd0 2024-04-02T14:01:12.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.681+0000 7f27bdfa5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f27b41afaa0 con 0x55707bcca1f0 2024-04-02T14:01:12.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.681+0000 7f27bdfa5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f27b41ae630 con 0x55707bcc98d0 2024-04-02T14:01:12.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.681+0000 7f27b9881640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcd2dd0 0x55707bcd2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:12.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.681+0000 7f27b9881640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcd2dd0 0x55707bcd2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56004/0 (socket says 172.21.15.52:56004) 2024-04-02T14:01:12.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.681+0000 7f27b9881640 1 -- 172.21.15.52:0/2504775273 learned_addr learned my addr 172.21.15.52:0/2504775273 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:12.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.682+0000 7f27b9080640 1 --2- 172.21.15.52:0/2504775273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55707bcc98d0 0x55707bcdea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:12.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.682+0000 7f27b887f640 1 --2- 172.21.15.52:0/2504775273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55707bcca1f0 0x55707bcdb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:12.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.682+0000 7f27b9881640 1 -- 172.21.15.52:0/2504775273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55707bcc98d0 msgr2=0x55707bcdea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:12.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.682+0000 7f27b9881640 1 --2- 172.21.15.52:0/2504775273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55707bcc98d0 0x55707bcdea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.682+0000 7f27b9881640 1 -- 172.21.15.52:0/2504775273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55707bcca1f0 msgr2=0x55707bcdb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:12.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.682+0000 7f27b9881640 1 --2- 172.21.15.52:0/2504775273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55707bcca1f0 0x55707bcdb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.682+0000 7f27b9881640 1 -- 172.21.15.52:0/2504775273 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55707bcc3e30 con 0x55707bcd2dd0 2024-04-02T14:01:12.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.682+0000 7f27b9080640 1 --2- 172.21.15.52:0/2504775273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55707bcc98d0 0x55707bcdea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:12.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.682+0000 7f27b887f640 1 --2- 172.21.15.52:0/2504775273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55707bcca1f0 0x55707bcdb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:12.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.682+0000 7f27b9881640 1 --2- 172.21.15.52:0/2504775273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcd2dd0 0x55707bcd2080 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f27ac01b000 tx=0x7f27ac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e24d817713bd40c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:12.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.682+0000 7f279effd640 1 -- 172.21.15.52:0/2504775273 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac018040 con 0x55707bcd2dd0 2024-04-02T14:01:12.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.682+0000 7f279effd640 1 -- 172.21.15.52:0/2504775273 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27ac003e50 con 0x55707bcd2dd0 2024-04-02T14:01:12.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.682+0000 7f279effd640 1 -- 172.21.15.52:0/2504775273 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac004cf0 con 0x55707bcd2dd0 2024-04-02T14:01:12.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.683+0000 7f27bdfa5740 1 -- 172.21.15.52:0/2504775273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcd2dd0 msgr2=0x55707bcd2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:12.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.683+0000 7f27bdfa5740 1 --2- 172.21.15.52:0/2504775273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcd2dd0 0x55707bcd2080 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f27ac01b000 tx=0x7f27ac00a620 comp rx=0 tx=0).stop 2024-04-02T14:01:12.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.683+0000 7f27bdfa5740 1 -- 172.21.15.52:0/2504775273 shutdown_connections 2024-04-02T14:01:12.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.683+0000 7f27bdfa5740 1 --2- 172.21.15.52:0/2504775273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcd2dd0 0x55707bcd2080 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.683+0000 7f27bdfa5740 1 --2- 172.21.15.52:0/2504775273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55707bcca1f0 0x55707bcdb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.683+0000 7f27bdfa5740 1 --2- 172.21.15.52:0/2504775273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55707bcc98d0 0x55707bcdea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.683+0000 7f27bdfa5740 1 -- 172.21.15.52:0/2504775273 >> 172.21.15.52:0/2504775273 conn(0x55707bcc4000 msgr2=0x55707bcc3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:12.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.683+0000 7f27bdfa5740 1 -- 172.21.15.52:0/2504775273 shutdown_connections 2024-04-02T14:01:12.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.683+0000 7f27bdfa5740 1 -- 172.21.15.52:0/2504775273 wait complete. 2024-04-02T14:01:12.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.683+0000 7f27bdfa5740 1 -- start start 2024-04-02T14:01:12.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.683+0000 7f27bdfa5740 1 Processor -- start 2024-04-02T14:01:12.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.683+0000 7f27bdfa5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55707bcc98d0 0x55707bcd91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:12.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.683+0000 7f27bdfa5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcca1f0 0x55707bcd9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:12.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.683+0000 7f27bdfa5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55707bcd2dd0 0x55707bd4a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:12.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.683+0000 7f27bdfa5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f27b41af3d0 con 0x55707bcca1f0 2024-04-02T14:01:12.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.683+0000 7f27bdfa5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f27b41aed00 con 0x55707bcd2dd0 2024-04-02T14:01:12.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.683+0000 7f27bdfa5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f27b41abf60 con 0x55707bcc98d0 2024-04-02T14:01:12.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.683+0000 7f27bdfa5740 10 client.0 _ll_register_callbacks cb 0x7f27b000fd20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:12.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.684+0000 7f27b887f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcca1f0 0x55707bcd9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:12.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.684+0000 7f27b887f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcca1f0 0x55707bcd9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56012/0 (socket says 172.21.15.52:56012) 2024-04-02T14:01:12.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.684+0000 7f27b887f640 1 -- 172.21.15.52:0/1439043110 learned_addr learned my addr 172.21.15.52:0/1439043110 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:12.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.684+0000 7f27b9881640 1 --2- 172.21.15.52:0/1439043110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55707bcd2dd0 0x55707bd4a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:12.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.684+0000 7f27b9080640 1 --2- 172.21.15.52:0/1439043110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55707bcc98d0 0x55707bcd91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:12.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.684+0000 7f27b887f640 1 -- 172.21.15.52:0/1439043110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55707bcc98d0 msgr2=0x55707bcd91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:12.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.684+0000 7f27b887f640 1 --2- 172.21.15.52:0/1439043110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55707bcc98d0 0x55707bcd91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.684+0000 7f27b887f640 1 -- 172.21.15.52:0/1439043110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55707bcd2dd0 msgr2=0x55707bd4a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:12.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.684+0000 7f27b887f640 1 --2- 172.21.15.52:0/1439043110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55707bcd2dd0 0x55707bd4a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.684+0000 7f27b887f640 1 -- 172.21.15.52:0/1439043110 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55707bd4a890 con 0x55707bcca1f0 2024-04-02T14:01:12.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.684+0000 7f27b9881640 1 --2- 172.21.15.52:0/1439043110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55707bcd2dd0 0x55707bd4a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:12.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.684+0000 7f27b887f640 1 --2- 172.21.15.52:0/1439043110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcca1f0 0x55707bcd9710 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f27a402bc70 tx=0x7f27a402cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8e83f2fabadd81 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:12.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.684+0000 7f279cff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:12.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.684+0000 7f279cff9640 1 -- 172.21.15.52:0/1439043110 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a404b040 con 0x55707bcca1f0 2024-04-02T14:01:12.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.684+0000 7f279cff9640 1 -- 172.21.15.52:0/1439043110 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9526+0+0 (secure 0 0 0) 0x7f27a40ba910 con 0x55707bcca1f0 2024-04-02T14:01:12.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.684+0000 7f27bdfa5740 1 -- 172.21.15.52:0/1439043110 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27ac003e50 con 0x55707bcca1f0 2024-04-02T14:01:12.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.684+0000 7f279cff9640 1 -- 172.21.15.52:0/1439043110 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f27a40bec30 con 0x55707bcca1f0 2024-04-02T14:01:12.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.684+0000 7f27bdfa5740 10 client.17760 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:12.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.684+0000 7f27bdfa5740 1 -- 172.21.15.52:0/1439043110 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55707bd4bb10 con 0x55707bcca1f0 2024-04-02T14:01:12.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.684+0000 7f279cff9640 1 -- 172.21.15.52:0/1439043110 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27ac003e50 con 0x55707bcca1f0 2024-04-02T14:01:12.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.685+0000 7f279cff9640 1 -- 172.21.15.52:0/1439043110 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f27a40bf5d0 con 0x55707bcca1f0 2024-04-02T14:01:12.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.685+0000 7f27bdfa5740 10 client.17760 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:12.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.685+0000 7f27bdfa5740 10 client.17760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:01:12.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.685+0000 7f27bdfa5740 20 client.17760 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:12.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.685+0000 7f27bdfa5740 1 --2- 172.21.15.52:0/1439043110 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55707bd4c290 0x55707bd4e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:12.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.685+0000 7f27bdfa5740 4 client.17760 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:01:12.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.685+0000 7f27bdfa5740 1 -- 172.21.15.52:0/1439043110 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55707bc350c0 con 0x55707bd4c290 2024-04-02T14:01:12.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.685+0000 7f27b9080640 1 --2- 172.21.15.52:0/1439043110 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55707bd4c290 0x55707bd4e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:12.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.685+0000 7f279cff9640 10 client.17760 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:12.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.685+0000 7f27b9080640 1 --2- 172.21.15.52:0/1439043110 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55707bd4c290 0x55707bd4e6b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c75b2e2f492ce0 server_cookie=25ea0456b5de2eb1 in_seq=0 out_seq=0 2024-04-02T14:01:12.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.686+0000 7f279cff9640 1 -- 172.21.15.52:0/1439043110 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55707bc350c0 con 0x55707bd4c290 2024-04-02T14:01:12.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.686+0000 7f279cff9640 10 client.17760 handle_command_reply: tid=0 2024-04-02T14:01:12.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.686+0000 7f27bdfa5740 1 client.17760 shutdown 2024-04-02T14:01:12.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.686+0000 7f27bdfa5740 1 -- 172.21.15.52:0/1439043110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcca1f0 msgr2=0x55707bcd9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:12.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.686+0000 7f27bdfa5740 1 --2- 172.21.15.52:0/1439043110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcca1f0 0x55707bcd9710 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f27a402bc70 tx=0x7f27a402cde0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.686+0000 7f27bdfa5740 1 -- 172.21.15.52:0/1439043110 shutdown_connections 2024-04-02T14:01:12.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.686+0000 7f27bdfa5740 1 --2- 172.21.15.52:0/1439043110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55707bcd2dd0 0x55707bd4a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.686+0000 7f27bdfa5740 1 --2- 172.21.15.52:0/1439043110 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55707bd4c290 0x55707bd4e6b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.686+0000 7f27bdfa5740 1 --2- 172.21.15.52:0/1439043110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55707bcca1f0 0x55707bcd9710 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.686+0000 7f27bdfa5740 1 --2- 172.21.15.52:0/1439043110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55707bcc98d0 0x55707bcd91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.686+0000 7f27bdfa5740 1 -- 172.21.15.52:0/1439043110 >> 172.21.15.52:0/1439043110 conn(0x55707bcc4000 msgr2=0x55707bcc53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:12.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.686+0000 7f279cff9640 0 client.17760 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:12.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.686+0000 7f27bdfa5740 1 -- 172.21.15.52:0/1439043110 shutdown_connections 2024-04-02T14:01:12.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.686+0000 7f27bdfa5740 1 -- 172.21.15.52:0/1439043110 wait complete. 2024-04-02T14:01:12.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.686+0000 7f27bdfa5740 20 client.17760 trim_cache size 0 max 16384 2024-04-02T14:01:12.704 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:01:12.704 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-04-02T14:01:12.704 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-04-02T14:01:12.704 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:01:12.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.686+0000 7f27bb2e4640 1 -- 172.21.15.52:0/1209934209 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f27a00a3da0 msgr2=0x7f27a00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:12.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.686+0000 7f27bb2e4640 1 --2- 172.21.15.52:0/1209934209 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f27a00a3da0 0x7f27a00a6260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f27b0005fd0 tx=0x7f27b000b5d0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.686+0000 7f27bb2e4640 1 -- 172.21.15.52:0/1209934209 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b4101330 msgr2=0x7f27b41977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:12.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.686+0000 7f27bb2e4640 1 --2- 172.21.15.52:0/1209934209 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b4101330 0x7f27b41977f0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f27a4009230 tx=0x7f27a402cc70 comp rx=0 tx=0).stop 2024-04-02T14:01:12.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.686+0000 7f27bb2e4640 1 -- 172.21.15.52:0/1209934209 shutdown_connections 2024-04-02T14:01:12.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.686+0000 7f27bb2e4640 1 --2- 172.21.15.52:0/1209934209 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f27a00a3da0 0x7f27a00a6260 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.686+0000 7f27bb2e4640 1 --2- 172.21.15.52:0/1209934209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27b4106980 0x7f27b419bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.686+0000 7f27bb2e4640 1 --2- 172.21.15.52:0/1209934209 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b4101330 0x7f27b41977f0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.686+0000 7f27bb2e4640 1 --2- 172.21.15.52:0/1209934209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27b4100970 0x7f27b41972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.686+0000 7f27bb2e4640 1 -- 172.21.15.52:0/1209934209 >> 172.21.15.52:0/1209934209 conn(0x7f27b4077b00 msgr2=0x7f27b4103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:12.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.687+0000 7f27bb2e4640 1 -- 172.21.15.52:0/1209934209 shutdown_connections 2024-04-02T14:01:12.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.687+0000 7f27bb2e4640 1 -- 172.21.15.52:0/1209934209 wait complete. 2024-04-02T14:01:12.707 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T14:01:12.707 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:01:12.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.806+0000 7efd6227b640 1 -- 172.21.15.52:0/1880967217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd5c10c100 msgr2=0x7efd5c10e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:12.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.806+0000 7efd6227b640 1 --2- 172.21.15.52:0/1880967217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd5c10c100 0x7efd5c10e530 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7efd54009ca0 tx=0x7efd5402c180 comp rx=0 tx=0).stop 2024-04-02T14:01:12.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.806+0000 7efd6227b640 1 -- 172.21.15.52:0/1880967217 shutdown_connections 2024-04-02T14:01:12.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.806+0000 7efd6227b640 1 --2- 172.21.15.52:0/1880967217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd5c10c100 0x7efd5c10e530 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.806+0000 7efd6227b640 1 --2- 172.21.15.52:0/1880967217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd5c103650 0x7efd5c10bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.806+0000 7efd6227b640 1 --2- 172.21.15.52:0/1880967217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd5c102d30 0x7efd5c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.806+0000 7efd6227b640 1 -- 172.21.15.52:0/1880967217 >> 172.21.15.52:0/1880967217 conn(0x7efd5c076d80 msgr2=0x7efd5c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:12.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.806+0000 7efd6227b640 1 -- 172.21.15.52:0/1880967217 shutdown_connections 2024-04-02T14:01:12.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.806+0000 7efd6227b640 1 -- 172.21.15.52:0/1880967217 wait complete. 2024-04-02T14:01:12.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.807+0000 7efd6227b640 1 Processor -- start 2024-04-02T14:01:12.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.807+0000 7efd6227b640 1 -- start start 2024-04-02T14:01:12.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.807+0000 7efd6227b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd5c102d30 0x7efd5c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:12.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.807+0000 7efd6227b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd5c103650 0x7efd5c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:12.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.807+0000 7efd6227b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd5c10c100 0x7efd5c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:12.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.807+0000 7efd6227b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c111e90 con 0x7efd5c103650 2024-04-02T14:01:12.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.807+0000 7efd6227b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c111d10 con 0x7efd5c102d30 2024-04-02T14:01:12.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.807+0000 7efd6227b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c112010 con 0x7efd5c10c100 2024-04-02T14:01:12.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.807+0000 7efd5b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd5c103650 0x7efd5c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:12.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.807+0000 7efd5b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd5c103650 0x7efd5c1988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56018/0 (socket says 172.21.15.52:56018) 2024-04-02T14:01:12.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.807+0000 7efd5b7fe640 1 -- 172.21.15.52:0/288998813 learned_addr learned my addr 172.21.15.52:0/288998813 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:12.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.807+0000 7efd5bfff640 1 --2- 172.21.15.52:0/288998813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd5c102d30 0x7efd5c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:12.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.807+0000 7efd60818640 1 --2- 172.21.15.52:0/288998813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd5c10c100 0x7efd5c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:12.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.808+0000 7efd5b7fe640 1 -- 172.21.15.52:0/288998813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd5c10c100 msgr2=0x7efd5c19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:12.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.808+0000 7efd5b7fe640 1 --2- 172.21.15.52:0/288998813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd5c10c100 0x7efd5c19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.808+0000 7efd5b7fe640 1 -- 172.21.15.52:0/288998813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd5c102d30 msgr2=0x7efd5c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:12.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.808+0000 7efd5b7fe640 1 --2- 172.21.15.52:0/288998813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd5c102d30 0x7efd5c198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:12.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.808+0000 7efd5b7fe640 1 -- 172.21.15.52:0/288998813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5c19d330 con 0x7efd5c103650 2024-04-02T14:01:12.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.808+0000 7efd60818640 1 --2- 172.21.15.52:0/288998813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd5c10c100 0x7efd5c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:12.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.808+0000 7efd5b7fe640 1 --2- 172.21.15.52:0/288998813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd5c103650 0x7efd5c1988d0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7efd4c00d770 tx=0x7efd4c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:12.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.808+0000 7efd597fa640 1 -- 172.21.15.52:0/288998813 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c017070 con 0x7efd5c103650 2024-04-02T14:01:12.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.808+0000 7efd6227b640 1 -- 172.21.15.52:0/288998813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd5c19d620 con 0x7efd5c103650 2024-04-02T14:01:12.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.808+0000 7efd597fa640 1 -- 172.21.15.52:0/288998813 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd4c005500 con 0x7efd5c103650 2024-04-02T14:01:12.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.808+0000 7efd597fa640 1 -- 172.21.15.52:0/288998813 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c004e00 con 0x7efd5c103650 2024-04-02T14:01:12.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.808+0000 7efd6227b640 1 -- 172.21.15.52:0/288998813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd5c1a4eb0 con 0x7efd5c103650 2024-04-02T14:01:12.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.809+0000 7efd6227b640 1 -- 172.21.15.52:0/288998813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd5c06a950 con 0x7efd5c103650 2024-04-02T14:01:12.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.810+0000 7efd597fa640 1 -- 172.21.15.52:0/288998813 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efd4c004fa0 con 0x7efd5c103650 2024-04-02T14:01:12.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.811+0000 7efd597fa640 1 --2- 172.21.15.52:0/288998813 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efd440a3d20 0x7efd440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:12.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.811+0000 7efd597fa640 1 -- 172.21.15.52:0/288998813 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9526+0+0 (secure 0 0 0) 0x7efd4c0d09d0 con 0x7efd5c103650 2024-04-02T14:01:12.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.811+0000 7efd5bfff640 1 --2- 172.21.15.52:0/288998813 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efd440a3d20 0x7efd440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:12.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.811+0000 7efd5bfff640 1 --2- 172.21.15.52:0/288998813 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efd440a3d20 0x7efd440a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efd5000bbe0 tx=0x7efd5000b390 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:12.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:12.815+0000 7efd597fa640 1 -- 172.21.15.52:0/288998813 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7efd4c09cc80 con 0x7efd5c103650 2024-04-02T14:01:13.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.041+0000 7efd6227b640 1 -- 172.21.15.52:0/288998813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd5c068d40 con 0x7efd5c103650 2024-04-02T14:01:13.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.042+0000 7efd597fa640 1 -- 172.21.15.52:0/288998813 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11923 (secure 0 0 0) 0x7efd5c068d40 con 0x7efd5c103650 2024-04-02T14:01:13.047 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T14:01:13.047 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:57:45.240902+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24467,24457,24490,24475,24503]},"id":1}]} 2024-04-02T14:01:13.048 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-04-02T14:01:13.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.044+0000 7efd6227b640 1 -- 172.21.15.52:0/288998813 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efd440a3d20 msgr2=0x7efd440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.044+0000 7efd6227b640 1 --2- 172.21.15.52:0/288998813 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efd440a3d20 0x7efd440a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efd5000bbe0 tx=0x7efd5000b390 comp rx=0 tx=0).stop 2024-04-02T14:01:13.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.044+0000 7efd6227b640 1 -- 172.21.15.52:0/288998813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd5c103650 msgr2=0x7efd5c1988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.044+0000 7efd6227b640 1 --2- 172.21.15.52:0/288998813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd5c103650 0x7efd5c1988d0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7efd4c00d770 tx=0x7efd4c00dc30 comp rx=0 tx=0).stop 2024-04-02T14:01:13.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.044+0000 7efd6227b640 1 -- 172.21.15.52:0/288998813 shutdown_connections 2024-04-02T14:01:13.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.044+0000 7efd6227b640 1 --2- 172.21.15.52:0/288998813 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efd440a3d20 0x7efd440a61e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.044+0000 7efd6227b640 1 --2- 172.21.15.52:0/288998813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd5c10c100 0x7efd5c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.044+0000 7efd6227b640 1 --2- 172.21.15.52:0/288998813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd5c103650 0x7efd5c1988d0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.044+0000 7efd6227b640 1 --2- 172.21.15.52:0/288998813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd5c102d30 0x7efd5c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.044+0000 7efd6227b640 1 -- 172.21.15.52:0/288998813 >> 172.21.15.52:0/288998813 conn(0x7efd5c076d80 msgr2=0x7efd5c0fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.044+0000 7efd6227b640 1 -- 172.21.15.52:0/288998813 shutdown_connections 2024-04-02T14:01:13.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.044+0000 7efd6227b640 1 -- 172.21.15.52:0/288998813 wait complete. 2024-04-02T14:01:13.060 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T14:01:13.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.160+0000 7fd16bf72640 1 -- 172.21.15.52:0/914747494 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd164102450 msgr2=0x7fd16410e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.160+0000 7fd16bf72640 1 --2- 172.21.15.52:0/914747494 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd164102450 0x7fd16410e960 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fd15c009cd0 tx=0x7fd15c02c0f0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.160+0000 7fd16bf72640 1 -- 172.21.15.52:0/914747494 shutdown_connections 2024-04-02T14:01:13.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.160+0000 7fd16bf72640 1 --2- 172.21.15.52:0/914747494 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd164102450 0x7fd16410e960 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.160+0000 7fd16bf72640 1 --2- 172.21.15.52:0/914747494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd164101a90 0x7fd164101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.160+0000 7fd16bf72640 1 --2- 172.21.15.52:0/914747494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd164107aa0 0x7fd164107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.160+0000 7fd16bf72640 1 -- 172.21.15.52:0/914747494 >> 172.21.15.52:0/914747494 conn(0x7fd164078f00 msgr2=0x7fd164079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.160+0000 7fd16bf72640 1 -- 172.21.15.52:0/914747494 shutdown_connections 2024-04-02T14:01:13.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.160+0000 7fd16bf72640 1 -- 172.21.15.52:0/914747494 wait complete. 2024-04-02T14:01:13.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.160+0000 7fd16bf72640 1 Processor -- start 2024-04-02T14:01:13.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.160+0000 7fd16bf72640 1 -- start start 2024-04-02T14:01:13.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.161+0000 7fd16bf72640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd164101a90 0x7fd164198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.161+0000 7fd16bf72640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd164102450 0x7fd1641988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.161+0000 7fd16bf72640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd164107aa0 0x7fd16419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.161+0000 7fd16bf72640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd164111e80 con 0x7fd164107aa0 2024-04-02T14:01:13.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.161+0000 7fd16bf72640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd164111d00 con 0x7fd164102450 2024-04-02T14:01:13.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.161+0000 7fd16bf72640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd164112000 con 0x7fd164101a90 2024-04-02T14:01:13.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.161+0000 7fd16a50f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd164107aa0 0x7fd16419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.161+0000 7fd16a50f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd164107aa0 0x7fd16419cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56030/0 (socket says 172.21.15.52:56030) 2024-04-02T14:01:13.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.161+0000 7fd16a50f640 1 -- 172.21.15.52:0/961310813 learned_addr learned my addr 172.21.15.52:0/961310813 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:13.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.161+0000 7fd16950d640 1 --2- 172.21.15.52:0/961310813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd164102450 0x7fd1641988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.161+0000 7fd169d0e640 1 --2- 172.21.15.52:0/961310813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd164101a90 0x7fd164198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.161+0000 7fd16a50f640 1 -- 172.21.15.52:0/961310813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd164101a90 msgr2=0x7fd164198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.161+0000 7fd16a50f640 1 --2- 172.21.15.52:0/961310813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd164101a90 0x7fd164198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.161+0000 7fd16a50f640 1 -- 172.21.15.52:0/961310813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd164102450 msgr2=0x7fd1641988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.161+0000 7fd16a50f640 1 --2- 172.21.15.52:0/961310813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd164102450 0x7fd1641988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.161+0000 7fd16a50f640 1 -- 172.21.15.52:0/961310813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16419d370 con 0x7fd164107aa0 2024-04-02T14:01:13.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.161+0000 7fd16950d640 1 --2- 172.21.15.52:0/961310813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd164102450 0x7fd1641988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:13.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.161+0000 7fd16a50f640 1 --2- 172.21.15.52:0/961310813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd164107aa0 0x7fd16419cc40 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0348f0 tx=0x7fd15c032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.161+0000 7fd15affd640 1 -- 172.21.15.52:0/961310813 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c041020 con 0x7fd164107aa0 2024-04-02T14:01:13.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.161+0000 7fd15affd640 1 -- 172.21.15.52:0/961310813 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd15c034cd0 con 0x7fd164107aa0 2024-04-02T14:01:13.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.161+0000 7fd16bf72640 1 -- 172.21.15.52:0/961310813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd16419d600 con 0x7fd164107aa0 2024-04-02T14:01:13.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.161+0000 7fd15affd640 1 -- 172.21.15.52:0/961310813 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c039040 con 0x7fd164107aa0 2024-04-02T14:01:13.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.161+0000 7fd16bf72640 1 -- 172.21.15.52:0/961310813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1641a4ef0 con 0x7fd164107aa0 2024-04-02T14:01:13.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.164+0000 7fd15affd640 1 -- 172.21.15.52:0/961310813 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd15c044020 con 0x7fd164107aa0 2024-04-02T14:01:13.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.164+0000 7fd16ec32740 1 Processor -- start 2024-04-02T14:01:13.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.164+0000 7fd16ec32740 1 -- start start 2024-04-02T14:01:13.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.164+0000 7fd16ec32740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c901ffc7a0 0x55c901ffcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.164+0000 7fd16ec32740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffd0c0 0x55c902005760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.164+0000 7fd16ec32740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c902005ca0 0x55c902008090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.164+0000 7fd16ec32740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c901f607b0 con 0x55c901ffd0c0 2024-04-02T14:01:13.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.164+0000 7fd16ec32740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c901eff570 con 0x55c901ffc7a0 2024-04-02T14:01:13.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.164+0000 7fd16ec32740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c901f5c760 con 0x55c902005ca0 2024-04-02T14:01:13.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.164+0000 7fd15affd640 1 --2- 172.21.15.52:0/961310813 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd1500a3d20 0x7fd1500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.164+0000 7fd169d0e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffd0c0 0x55c902005760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.164+0000 7fd169d0e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffd0c0 0x55c902005760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56044/0 (socket says 172.21.15.52:56044) 2024-04-02T14:01:13.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.164+0000 7fd169d0e640 1 -- 172.21.15.52:0/457030325 learned_addr learned my addr 172.21.15.52:0/457030325 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:13.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.164+0000 7fd16a50f640 1 --2- 172.21.15.52:0/961310813 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd1500a3d20 0x7fd1500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.165+0000 7fd15affd640 1 -- 172.21.15.52:0/961310813 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9526+0+0 (secure 0 0 0) 0x7fd15c0f2a50 con 0x7fd164107aa0 2024-04-02T14:01:13.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.165+0000 7fd16950d640 1 --2- 172.21.15.52:0/457030325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c902005ca0 0x55c902008090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.165+0000 7fd16950d640 1 --2- 172.21.15.52:0/457030325 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c901ffc7a0 0x55c901ffcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.165+0000 7fd169d0e640 1 -- 172.21.15.52:0/457030325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c902005ca0 msgr2=0x55c902008090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.165+0000 7fd169d0e640 1 --2- 172.21.15.52:0/457030325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c902005ca0 0x55c902008090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.165+0000 7fd169d0e640 1 -- 172.21.15.52:0/457030325 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c901ffc7a0 msgr2=0x55c901ffcb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.165+0000 7fd169d0e640 1 --2- 172.21.15.52:0/457030325 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c901ffc7a0 0x55c901ffcb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.165+0000 7fd169d0e640 1 -- 172.21.15.52:0/457030325 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c901c743f0 con 0x55c901ffd0c0 2024-04-02T14:01:13.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.165+0000 7fd16950d640 1 --2- 172.21.15.52:0/457030325 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c901ffc7a0 0x55c901ffcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:13.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.165+0000 7fd169d0e640 1 --2- 172.21.15.52:0/457030325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffd0c0 0x55c902005760 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fd160009af0 tx=0x7fd16000e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505eba3b34af5f17 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.165+0000 7fd16a50f640 1 --2- 172.21.15.52:0/961310813 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd1500a3d20 0x7fd1500a61e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0eff70 tx=0x7fd15c0891b0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.165+0000 7fd14ffff640 1 -- 172.21.15.52:0/457030325 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160016070 con 0x55c901ffd0c0 2024-04-02T14:01:13.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.165+0000 7fd14ffff640 1 -- 172.21.15.52:0/457030325 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd160005500 con 0x55c901ffd0c0 2024-04-02T14:01:13.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.165+0000 7fd14ffff640 1 -- 172.21.15.52:0/457030325 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160011470 con 0x55c901ffd0c0 2024-04-02T14:01:13.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.166+0000 7fd16ec32740 1 -- 172.21.15.52:0/457030325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffd0c0 msgr2=0x55c902005760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.166+0000 7fd16ec32740 1 --2- 172.21.15.52:0/457030325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffd0c0 0x55c902005760 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fd160009af0 tx=0x7fd16000e2f0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.166+0000 7fd16ec32740 1 -- 172.21.15.52:0/457030325 shutdown_connections 2024-04-02T14:01:13.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.166+0000 7fd16ec32740 1 --2- 172.21.15.52:0/457030325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c902005ca0 0x55c902008090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.166+0000 7fd16ec32740 1 --2- 172.21.15.52:0/457030325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffd0c0 0x55c902005760 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.166+0000 7fd16ec32740 1 --2- 172.21.15.52:0/457030325 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c901ffc7a0 0x55c901ffcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.166+0000 7fd16ec32740 1 -- 172.21.15.52:0/457030325 >> 172.21.15.52:0/457030325 conn(0x55c901ff6ed0 msgr2=0x55c901ff72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.166+0000 7fd16ec32740 1 -- 172.21.15.52:0/457030325 shutdown_connections 2024-04-02T14:01:13.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.166+0000 7fd16ec32740 1 -- 172.21.15.52:0/457030325 wait complete. 2024-04-02T14:01:13.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.166+0000 7fd16ec32740 1 -- start start 2024-04-02T14:01:13.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.166+0000 7fd16ec32740 1 Processor -- start 2024-04-02T14:01:13.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.167+0000 7fd16ec32740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffc7a0 0x55c902005660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.167+0000 7fd16ec32740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c901ffd0c0 0x55c9020029f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.167+0000 7fd16ec32740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c902005ca0 0x55c902003000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.167+0000 7fd16ec32740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c902008a90 con 0x55c901ffc7a0 2024-04-02T14:01:13.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.167+0000 7fd16ec32740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c901f1be80 con 0x55c902005ca0 2024-04-02T14:01:13.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.167+0000 7fd16ec32740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c902008c10 con 0x55c901ffd0c0 2024-04-02T14:01:13.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.167+0000 7fd16ec32740 10 client.0 _ll_register_callbacks cb 0x55c901f57aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:13.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.167+0000 7fd16950d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffc7a0 0x55c902005660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.167+0000 7fd16950d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffc7a0 0x55c902005660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56056/0 (socket says 172.21.15.52:56056) 2024-04-02T14:01:13.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.167+0000 7fd16950d640 1 -- 172.21.15.52:0/850513572 learned_addr learned my addr 172.21.15.52:0/850513572 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:13.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.167+0000 7fd169d0e640 1 --2- 172.21.15.52:0/850513572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c901ffd0c0 0x55c9020029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.167+0000 7fd16950d640 1 -- 172.21.15.52:0/850513572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c901ffd0c0 msgr2=0x55c9020029f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.167+0000 7fd16950d640 1 --2- 172.21.15.52:0/850513572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c901ffd0c0 0x55c9020029f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.167+0000 7fd16950d640 1 -- 172.21.15.52:0/850513572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c902005ca0 msgr2=0x55c902003000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.167+0000 7fd16950d640 1 --2- 172.21.15.52:0/850513572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c902005ca0 0x55c902003000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.167+0000 7fd16950d640 1 -- 172.21.15.52:0/850513572 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c901d3e7d0 con 0x55c901ffc7a0 2024-04-02T14:01:13.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.167+0000 7fd16950d640 1 --2- 172.21.15.52:0/850513572 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffc7a0 0x55c902005660 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fd154014860 tx=0x7fd154014d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9560266961dc93 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.167+0000 7fd14d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:13.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.168+0000 7fd14d7fa640 1 -- 172.21.15.52:0/850513572 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd154011c30 con 0x55c901ffc7a0 2024-04-02T14:01:13.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.168+0000 7fd14d7fa640 1 -- 172.21.15.52:0/850513572 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9526+0+0 (secure 0 0 0) 0x7fd154011dd0 con 0x55c901ffc7a0 2024-04-02T14:01:13.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.168+0000 7fd16ec32740 1 -- 172.21.15.52:0/850513572 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd160005500 con 0x55c901ffc7a0 2024-04-02T14:01:13.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.168+0000 7fd14d7fa640 1 -- 172.21.15.52:0/850513572 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd154024050 con 0x55c901ffc7a0 2024-04-02T14:01:13.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.168+0000 7fd16ec32740 10 client.17778 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:13.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.168+0000 7fd16ec32740 1 -- 172.21.15.52:0/850513572 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c902014b10 con 0x55c901ffc7a0 2024-04-02T14:01:13.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.168+0000 7fd14d7fa640 1 -- 172.21.15.52:0/850513572 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd160005500 con 0x55c901ffc7a0 2024-04-02T14:01:13.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.168+0000 7fd14d7fa640 1 -- 172.21.15.52:0/850513572 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd154003cd0 con 0x55c901ffc7a0 2024-04-02T14:01:13.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.168+0000 7fd16ec32740 10 client.17778 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:13.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.168+0000 7fd16ec32740 10 client.17778 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T14:01:13.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.168+0000 7fd16ec32740 20 client.17778 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:13.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.168+0000 7fd16ec32740 1 --2- 172.21.15.52:0/850513572 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55c902015f70 0x55c9020183b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.168+0000 7fd16ec32740 4 client.17778 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:01:13.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.168+0000 7fd16ec32740 1 -- 172.21.15.52:0/850513572 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c901f6dcf0 con 0x55c902015f70 2024-04-02T14:01:13.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.168+0000 7fd169d0e640 1 --2- 172.21.15.52:0/850513572 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55c902015f70 0x55c9020183b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.169+0000 7fd14d7fa640 10 client.17778 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T14:01:13.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.169+0000 7fd169d0e640 1 --2- 172.21.15.52:0/850513572 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55c902015f70 0x55c9020183b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fa07834a1e5ee5b server_cookie=e98035849a760773 in_seq=0 out_seq=0 2024-04-02T14:01:13.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.171+0000 7fd14d7fa640 1 -- 172.21.15.52:0/850513572 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c901f6dcf0 con 0x55c902015f70 2024-04-02T14:01:13.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.171+0000 7fd14d7fa640 10 client.17778 handle_command_reply: tid=0 2024-04-02T14:01:13.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.171+0000 7fd16ec32740 1 client.17778 shutdown 2024-04-02T14:01:13.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.171+0000 7fd16ec32740 1 -- 172.21.15.52:0/850513572 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffc7a0 msgr2=0x55c902005660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.171+0000 7fd16ec32740 1 --2- 172.21.15.52:0/850513572 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffc7a0 0x55c902005660 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fd154014860 tx=0x7fd154014d20 comp rx=0 tx=0).stop 2024-04-02T14:01:13.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.171+0000 7fd16ec32740 1 -- 172.21.15.52:0/850513572 shutdown_connections 2024-04-02T14:01:13.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.171+0000 7fd16ec32740 1 --2- 172.21.15.52:0/850513572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c902005ca0 0x55c902003000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.171+0000 7fd16ec32740 1 --2- 172.21.15.52:0/850513572 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55c902015f70 0x55c9020183b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.171+0000 7fd16ec32740 1 --2- 172.21.15.52:0/850513572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c901ffd0c0 0x55c9020029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.171+0000 7fd16ec32740 1 --2- 172.21.15.52:0/850513572 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffc7a0 0x55c902005660 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.171+0000 7fd14d7fa640 0 client.17778 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T14:01:13.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.171+0000 7fd16ec32740 1 -- 172.21.15.52:0/850513572 >> 172.21.15.52:0/850513572 conn(0x55c901ff6ed0 msgr2=0x55c901ff8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.171+0000 7fd16ec32740 1 -- 172.21.15.52:0/850513572 shutdown_connections 2024-04-02T14:01:13.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.171+0000 7fd16ec32740 1 -- 172.21.15.52:0/850513572 wait complete. 2024-04-02T14:01:13.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.171+0000 7fd16ec32740 20 client.17778 trim_cache size 0 max 16384 2024-04-02T14:01:13.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.183+0000 7fd16ec32740 1 Processor -- start 2024-04-02T14:01:13.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.183+0000 7fd16ec32740 1 -- start start 2024-04-02T14:01:13.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.183+0000 7fd16ec32740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffc7a0 0x55c9020119e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.183+0000 7fd16ec32740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c901ffd0c0 0x55c90200e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.183+0000 7fd16ec32740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c902005ca0 0x55c902005120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.183+0000 7fd16ec32740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd1641aeff0 con 0x55c901ffc7a0 2024-04-02T14:01:13.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.183+0000 7fd16ec32740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd1641b0b30 con 0x55c901ffd0c0 2024-04-02T14:01:13.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.183+0000 7fd16ec32740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd1641af6c0 con 0x55c902005ca0 2024-04-02T14:01:13.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.184+0000 7fd16950d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffc7a0 0x55c9020119e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.184+0000 7fd16950d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffc7a0 0x55c9020119e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56072/0 (socket says 172.21.15.52:56072) 2024-04-02T14:01:13.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.184+0000 7fd16950d640 1 -- 172.21.15.52:0/51183303 learned_addr learned my addr 172.21.15.52:0/51183303 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:13.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.184+0000 7fd16950d640 1 --2- 172.21.15.52:0/51183303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c902005ca0 0x55c902005120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.184+0000 7fd169d0e640 1 --2- 172.21.15.52:0/51183303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c901ffd0c0 0x55c90200e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.184+0000 7fd16950d640 1 -- 172.21.15.52:0/51183303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c902005ca0 msgr2=0x55c902005120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.184+0000 7fd16950d640 1 --2- 172.21.15.52:0/51183303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c902005ca0 0x55c902005120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.184+0000 7fd16950d640 1 -- 172.21.15.52:0/51183303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c901ffd0c0 msgr2=0x55c90200e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.184+0000 7fd16950d640 1 --2- 172.21.15.52:0/51183303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c901ffd0c0 0x55c90200e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.184+0000 7fd16950d640 1 -- 172.21.15.52:0/51183303 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c902018a30 con 0x55c901ffc7a0 2024-04-02T14:01:13.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.184+0000 7fd16950d640 1 --2- 172.21.15.52:0/51183303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffc7a0 0x55c9020119e0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x55c902011f60 tx=0x7fd154001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b887ba22cb9fd7f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.184+0000 7fd14f7fe640 1 -- 172.21.15.52:0/51183303 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd154011e30 con 0x55c901ffc7a0 2024-04-02T14:01:13.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.184+0000 7fd14f7fe640 1 -- 172.21.15.52:0/51183303 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd15402e030 con 0x55c901ffc7a0 2024-04-02T14:01:13.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.184+0000 7fd14f7fe640 1 -- 172.21.15.52:0/51183303 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15401d8f0 con 0x55c901ffc7a0 2024-04-02T14:01:13.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.185+0000 7fd16ec32740 1 -- 172.21.15.52:0/51183303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffc7a0 msgr2=0x55c9020119e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.185+0000 7fd16ec32740 1 --2- 172.21.15.52:0/51183303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffc7a0 0x55c9020119e0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x55c902011f60 tx=0x7fd154001e50 comp rx=0 tx=0).stop 2024-04-02T14:01:13.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.185+0000 7fd16ec32740 1 -- 172.21.15.52:0/51183303 shutdown_connections 2024-04-02T14:01:13.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.185+0000 7fd16ec32740 1 --2- 172.21.15.52:0/51183303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c902005ca0 0x55c902005120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.185+0000 7fd16ec32740 1 --2- 172.21.15.52:0/51183303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c901ffd0c0 0x55c90200e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.185+0000 7fd16ec32740 1 --2- 172.21.15.52:0/51183303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffc7a0 0x55c9020119e0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.185+0000 7fd16ec32740 1 -- 172.21.15.52:0/51183303 >> 172.21.15.52:0/51183303 conn(0x55c901ff6ed0 msgr2=0x55c902001a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.185+0000 7fd16ec32740 1 -- 172.21.15.52:0/51183303 shutdown_connections 2024-04-02T14:01:13.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.185+0000 7fd16ec32740 1 -- 172.21.15.52:0/51183303 wait complete. 2024-04-02T14:01:13.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.185+0000 7fd16ec32740 1 -- start start 2024-04-02T14:01:13.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.185+0000 7fd16ec32740 1 Processor -- start 2024-04-02T14:01:13.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.186+0000 7fd16ec32740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffc7a0 0x55c902007480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.186+0000 7fd16ec32740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c901ffd0c0 0x55c9020079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.186+0000 7fd16ec32740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c902005ca0 0x55c902006150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.186+0000 7fd16ec32740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd1641b0460 con 0x55c901ffc7a0 2024-04-02T14:01:13.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.186+0000 7fd16ec32740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd1641afd90 con 0x55c902005ca0 2024-04-02T14:01:13.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.186+0000 7fd16ec32740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd1641acff0 con 0x55c901ffd0c0 2024-04-02T14:01:13.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.186+0000 7fd16ec32740 10 client.0 _ll_register_callbacks cb 0x7fd15400c6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:13.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.186+0000 7fd16950d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffc7a0 0x55c902007480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.186+0000 7fd16950d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffc7a0 0x55c902007480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56086/0 (socket says 172.21.15.52:56086) 2024-04-02T14:01:13.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.186+0000 7fd16950d640 1 -- 172.21.15.52:0/3480403302 learned_addr learned my addr 172.21.15.52:0/3480403302 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:13.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.186+0000 7fd169d0e640 1 --2- 172.21.15.52:0/3480403302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c901ffd0c0 0x55c9020079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.186+0000 7fd16950d640 1 --2- 172.21.15.52:0/3480403302 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c902005ca0 0x55c902006150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.186+0000 7fd16950d640 1 -- 172.21.15.52:0/3480403302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c901ffd0c0 msgr2=0x55c9020079c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.186+0000 7fd16950d640 1 --2- 172.21.15.52:0/3480403302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c901ffd0c0 0x55c9020079c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.186+0000 7fd16950d640 1 -- 172.21.15.52:0/3480403302 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c902005ca0 msgr2=0x55c902006150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.186+0000 7fd16950d640 1 --2- 172.21.15.52:0/3480403302 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c902005ca0 0x55c902006150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.186+0000 7fd16950d640 1 -- 172.21.15.52:0/3480403302 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c902006950 con 0x55c901ffc7a0 2024-04-02T14:01:13.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.186+0000 7fd16950d640 1 --2- 172.21.15.52:0/3480403302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffc7a0 0x55c902007480 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fd154012a30 tx=0x7fd154012c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a59dc3999acba0b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.186+0000 7fd14d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:13.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.186+0000 7fd14d7fa640 1 -- 172.21.15.52:0/3480403302 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15403a020 con 0x55c901ffc7a0 2024-04-02T14:01:13.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.186+0000 7fd16ec32740 1 -- 172.21.15.52:0/3480403302 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd15402e030 con 0x55c901ffc7a0 2024-04-02T14:01:13.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.186+0000 7fd14d7fa640 1 -- 172.21.15.52:0/3480403302 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9526+0+0 (secure 0 0 0) 0x7fd154011c30 con 0x55c901ffc7a0 2024-04-02T14:01:13.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.187+0000 7fd14d7fa640 1 -- 172.21.15.52:0/3480403302 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd154011e30 con 0x55c901ffc7a0 2024-04-02T14:01:13.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.187+0000 7fd16ec32740 10 client.17784 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:13.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.187+0000 7fd16ec32740 1 -- 172.21.15.52:0/3480403302 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c9020810f0 con 0x55c901ffc7a0 2024-04-02T14:01:13.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.187+0000 7fd14d7fa640 1 -- 172.21.15.52:0/3480403302 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd15402e030 con 0x55c901ffc7a0 2024-04-02T14:01:13.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.187+0000 7fd14d7fa640 1 -- 172.21.15.52:0/3480403302 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd154005140 con 0x55c901ffc7a0 2024-04-02T14:01:13.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.187+0000 7fd16ec32740 10 client.17784 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:13.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.187+0000 7fd16ec32740 10 client.17784 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T14:01:13.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.187+0000 7fd16ec32740 20 client.17784 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:13.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.187+0000 7fd16ec32740 1 --2- 172.21.15.52:0/3480403302 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55c9020816a0 0x55c902083ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.187+0000 7fd16ec32740 4 client.17784 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:01:13.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.187+0000 7fd16ec32740 1 -- 172.21.15.52:0/3480403302 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c901f68040 con 0x55c9020816a0 2024-04-02T14:01:13.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.187+0000 7fd169d0e640 1 --2- 172.21.15.52:0/3480403302 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55c9020816a0 0x55c902083ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.188+0000 7fd14d7fa640 10 client.17784 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T14:01:13.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.188+0000 7fd169d0e640 1 --2- 172.21.15.52:0/3480403302 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55c9020816a0 0x55c902083ac0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3c64fbed221c1c67 server_cookie=87bb52bb177df1d3 in_seq=0 out_seq=0 2024-04-02T14:01:13.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.188+0000 7fd14d7fa640 1 -- 172.21.15.52:0/3480403302 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c901f68040 con 0x55c9020816a0 2024-04-02T14:01:13.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.188+0000 7fd14d7fa640 10 client.17784 handle_command_reply: tid=0 2024-04-02T14:01:13.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.188+0000 7fd16ec32740 1 client.17784 shutdown 2024-04-02T14:01:13.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.188+0000 7fd16ec32740 1 -- 172.21.15.52:0/3480403302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffc7a0 msgr2=0x55c902007480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.188+0000 7fd16ec32740 1 --2- 172.21.15.52:0/3480403302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffc7a0 0x55c902007480 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fd154012a30 tx=0x7fd154012c70 comp rx=0 tx=0).stop 2024-04-02T14:01:13.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.188+0000 7fd16ec32740 1 -- 172.21.15.52:0/3480403302 shutdown_connections 2024-04-02T14:01:13.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.188+0000 7fd16ec32740 1 --2- 172.21.15.52:0/3480403302 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c902005ca0 0x55c902006150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.188+0000 7fd16ec32740 1 --2- 172.21.15.52:0/3480403302 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55c9020816a0 0x55c902083ac0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.188+0000 7fd16ec32740 1 --2- 172.21.15.52:0/3480403302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c901ffd0c0 0x55c9020079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.188+0000 7fd14d7fa640 0 client.17784 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T14:01:13.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.188+0000 7fd16ec32740 1 --2- 172.21.15.52:0/3480403302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c901ffc7a0 0x55c902007480 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.188+0000 7fd16ec32740 1 -- 172.21.15.52:0/3480403302 >> 172.21.15.52:0/3480403302 conn(0x55c901ff6ed0 msgr2=0x55c901ff56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.188+0000 7fd16ec32740 1 -- 172.21.15.52:0/3480403302 shutdown_connections 2024-04-02T14:01:13.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.188+0000 7fd16ec32740 1 -- 172.21.15.52:0/3480403302 wait complete. 2024-04-02T14:01:13.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.188+0000 7fd16ec32740 20 client.17784 trim_cache size 0 max 16384 2024-04-02T14:01:13.204 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:01:13.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.189+0000 7fd16bf72640 1 -- 172.21.15.52:0/961310813 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd1500a3d20 msgr2=0x7fd1500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.189+0000 7fd16bf72640 1 --2- 172.21.15.52:0/961310813 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd1500a3d20 0x7fd1500a61e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0eff70 tx=0x7fd15c0891b0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.189+0000 7fd16bf72640 1 -- 172.21.15.52:0/961310813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd164107aa0 msgr2=0x7fd16419cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.189+0000 7fd16bf72640 1 --2- 172.21.15.52:0/961310813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd164107aa0 0x7fd16419cc40 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0348f0 tx=0x7fd15c032ec0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.189+0000 7fd16bf72640 1 -- 172.21.15.52:0/961310813 shutdown_connections 2024-04-02T14:01:13.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.189+0000 7fd16bf72640 1 --2- 172.21.15.52:0/961310813 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd1500a3d20 0x7fd1500a61e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.189+0000 7fd16bf72640 1 --2- 172.21.15.52:0/961310813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd164107aa0 0x7fd16419cc40 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.189+0000 7fd16bf72640 1 --2- 172.21.15.52:0/961310813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd164102450 0x7fd1641988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.189+0000 7fd16bf72640 1 --2- 172.21.15.52:0/961310813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd164101a90 0x7fd164198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.189+0000 7fd16bf72640 1 -- 172.21.15.52:0/961310813 >> 172.21.15.52:0/961310813 conn(0x7fd164078f00 msgr2=0x7fd164104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.189+0000 7fd16bf72640 1 -- 172.21.15.52:0/961310813 shutdown_connections 2024-04-02T14:01:13.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.189+0000 7fd16bf72640 1 -- 172.21.15.52:0/961310813 wait complete. 2024-04-02T14:01:13.207 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T14:01:13.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.307+0000 7fa1ed0d9640 1 -- 172.21.15.52:0/2563027917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1e8107ab0 msgr2=0x7fa1e8107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.307+0000 7fa1ed0d9640 1 --2- 172.21.15.52:0/2563027917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1e8107ab0 0x7fa1e8107e90 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8009930 tx=0x7fa1d802bc30 comp rx=0 tx=0).stop 2024-04-02T14:01:13.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.307+0000 7fa1ed0d9640 1 -- 172.21.15.52:0/2563027917 shutdown_connections 2024-04-02T14:01:13.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.307+0000 7fa1ed0d9640 1 --2- 172.21.15.52:0/2563027917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1e8102460 0x7fa1e810e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.307+0000 7fa1ed0d9640 1 --2- 172.21.15.52:0/2563027917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1e8101aa0 0x7fa1e8101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.307+0000 7fa1ed0d9640 1 --2- 172.21.15.52:0/2563027917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1e8107ab0 0x7fa1e8107e90 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.307+0000 7fa1ed0d9640 1 -- 172.21.15.52:0/2563027917 >> 172.21.15.52:0/2563027917 conn(0x7fa1e80fd530 msgr2=0x7fa1e80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.307+0000 7fa1ed0d9640 1 -- 172.21.15.52:0/2563027917 shutdown_connections 2024-04-02T14:01:13.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.307+0000 7fa1ed0d9640 1 -- 172.21.15.52:0/2563027917 wait complete. 2024-04-02T14:01:13.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.308+0000 7fa1ed0d9640 1 Processor -- start 2024-04-02T14:01:13.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.308+0000 7fa1ed0d9640 1 -- start start 2024-04-02T14:01:13.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.308+0000 7fa1ed0d9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1e8101aa0 0x7fa1e8198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.308+0000 7fa1ed0d9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1e8102460 0x7fa1e81988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.308+0000 7fa1ed0d9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1e8107ab0 0x7fa1e819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.308+0000 7fa1ed0d9640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e8111e90 con 0x7fa1e8107ab0 2024-04-02T14:01:13.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.308+0000 7fa1ed0d9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e8111d10 con 0x7fa1e8102460 2024-04-02T14:01:13.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.308+0000 7fa1ed0d9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e8112010 con 0x7fa1e8101aa0 2024-04-02T14:01:13.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.308+0000 7fa1e759e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1e8107ab0 0x7fa1e819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.308+0000 7fa1e759e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1e8107ab0 0x7fa1e819cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56098/0 (socket says 172.21.15.52:56098) 2024-04-02T14:01:13.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.308+0000 7fa1e759e640 1 -- 172.21.15.52:0/3764182172 learned_addr learned my addr 172.21.15.52:0/3764182172 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:13.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.308+0000 7fa1e6d9d640 1 --2- 172.21.15.52:0/3764182172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1e8101aa0 0x7fa1e8198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.309+0000 7fa1dffff640 1 --2- 172.21.15.52:0/3764182172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1e8102460 0x7fa1e81988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.309+0000 7fa1e759e640 1 -- 172.21.15.52:0/3764182172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1e8101aa0 msgr2=0x7fa1e8198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.309+0000 7fa1e759e640 1 --2- 172.21.15.52:0/3764182172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1e8101aa0 0x7fa1e8198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.309+0000 7fa1e759e640 1 -- 172.21.15.52:0/3764182172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1e8102460 msgr2=0x7fa1e81988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.309+0000 7fa1e759e640 1 --2- 172.21.15.52:0/3764182172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1e8102460 0x7fa1e81988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.309+0000 7fa1e759e640 1 -- 172.21.15.52:0/3764182172 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e819d330 con 0x7fa1e8107ab0 2024-04-02T14:01:13.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.309+0000 7fa1dffff640 1 --2- 172.21.15.52:0/3764182172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1e8102460 0x7fa1e81988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:13.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.309+0000 7fa1e759e640 1 --2- 172.21.15.52:0/3764182172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1e8107ab0 0x7fa1e819cc00 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000cce0 tx=0x7fa1e000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.309+0000 7fa1e4d99640 1 -- 172.21.15.52:0/3764182172 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e000f450 con 0x7fa1e8107ab0 2024-04-02T14:01:13.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.309+0000 7fa1ed0d9640 1 -- 172.21.15.52:0/3764182172 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e819d620 con 0x7fa1e8107ab0 2024-04-02T14:01:13.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.309+0000 7fa1e4d99640 1 -- 172.21.15.52:0/3764182172 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e000fad0 con 0x7fa1e8107ab0 2024-04-02T14:01:13.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.309+0000 7fa1e4d99640 1 -- 172.21.15.52:0/3764182172 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0018c10 con 0x7fa1e8107ab0 2024-04-02T14:01:13.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.309+0000 7fa1ed0d9640 1 -- 172.21.15.52:0/3764182172 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e81a4eb0 con 0x7fa1e8107ab0 2024-04-02T14:01:13.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.311+0000 7fa1e4d99640 1 -- 172.21.15.52:0/3764182172 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa1e000f5f0 con 0x7fa1e8107ab0 2024-04-02T14:01:13.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.311+0000 7fa1efd9e740 1 Processor -- start 2024-04-02T14:01:13.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.312+0000 7fa1efd9e740 1 -- start start 2024-04-02T14:01:13.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.312+0000 7fa1e4d99640 1 --2- 172.21.15.52:0/3764182172 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa1d40a3d20 0x7fa1d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.312+0000 7fa1efd9e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b0cdb07a0 0x562b0cdb0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.312+0000 7fa1efd9e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb10c0 0x562b0cdb9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.312+0000 7fa1efd9e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b0cdb9ca0 0x562b0cdbc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.312+0000 7fa1efd9e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562b0cd147b0 con 0x562b0cdb10c0 2024-04-02T14:01:13.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.312+0000 7fa1efd9e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x562b0ccb3570 con 0x562b0cdb07a0 2024-04-02T14:01:13.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.312+0000 7fa1efd9e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562b0cd10760 con 0x562b0cdb9ca0 2024-04-02T14:01:13.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.312+0000 7fa1e4d99640 1 -- 172.21.15.52:0/3764182172 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9526+0+0 (secure 0 0 0) 0x7fa1e00d12e0 con 0x7fa1e8107ab0 2024-04-02T14:01:13.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.312+0000 7fa1dffff640 1 --2- 172.21.15.52:0/3764182172 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa1d40a3d20 0x7fa1d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.312+0000 7fa1e6d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b0cdb07a0 0x562b0cdb0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.312+0000 7fa1dffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb10c0 0x562b0cdb9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.312+0000 7fa1e6d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b0cdb07a0 0x562b0cdb0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:33270/0 (socket says 172.21.15.52:33270) 2024-04-02T14:01:13.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.312+0000 7fa1e6d9d640 1 -- 172.21.15.52:0/3037402938 learned_addr learned my addr 172.21.15.52:0/3037402938 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:13.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.312+0000 7fa1e759e640 1 --2- 172.21.15.52:0/3037402938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b0cdb9ca0 0x562b0cdbc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.312+0000 7fa1dffff640 1 -- 172.21.15.52:0/3037402938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b0cdb9ca0 msgr2=0x562b0cdbc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.312+0000 7fa1dffff640 1 --2- 172.21.15.52:0/3037402938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b0cdb9ca0 0x562b0cdbc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.312+0000 7fa1dffff640 1 -- 172.21.15.52:0/3037402938 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b0cdb07a0 msgr2=0x562b0cdb0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.312+0000 7fa1dffff640 1 --2- 172.21.15.52:0/3037402938 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b0cdb07a0 0x562b0cdb0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.313+0000 7fa1dffff640 1 -- 172.21.15.52:0/3037402938 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b0ca283f0 con 0x562b0cdb10c0 2024-04-02T14:01:13.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.313+0000 7fa1dffff640 1 --2- 172.21.15.52:0/3764182172 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa1d40a3d20 0x7fa1d40a61e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0005d90 tx=0x7fa1d000bb50 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.313+0000 7fa1dffff640 1 --2- 172.21.15.52:0/3037402938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb10c0 0x562b0cdb9760 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fa1e80ffb90 tx=0x7fa1d001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc98c053ae9562ed server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.313+0000 7fa1dd7fa640 1 -- 172.21.15.52:0/3037402938 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0021070 con 0x562b0cdb10c0 2024-04-02T14:01:13.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.313+0000 7fa1dd7fa640 1 -- 172.21.15.52:0/3037402938 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d0005500 con 0x562b0cdb10c0 2024-04-02T14:01:13.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.313+0000 7fa1dd7fa640 1 -- 172.21.15.52:0/3037402938 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d000e790 con 0x562b0cdb10c0 2024-04-02T14:01:13.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.314+0000 7fa1efd9e740 1 -- 172.21.15.52:0/3037402938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb10c0 msgr2=0x562b0cdb9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.314+0000 7fa1efd9e740 1 --2- 172.21.15.52:0/3037402938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb10c0 0x562b0cdb9760 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fa1e80ffb90 tx=0x7fa1d001a000 comp rx=0 tx=0).stop 2024-04-02T14:01:13.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.314+0000 7fa1efd9e740 1 -- 172.21.15.52:0/3037402938 shutdown_connections 2024-04-02T14:01:13.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.314+0000 7fa1efd9e740 1 --2- 172.21.15.52:0/3037402938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b0cdb9ca0 0x562b0cdbc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.314+0000 7fa1efd9e740 1 --2- 172.21.15.52:0/3037402938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb10c0 0x562b0cdb9760 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.314+0000 7fa1efd9e740 1 --2- 172.21.15.52:0/3037402938 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b0cdb07a0 0x562b0cdb0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.314+0000 7fa1efd9e740 1 -- 172.21.15.52:0/3037402938 >> 172.21.15.52:0/3037402938 conn(0x562b0cdaaed0 msgr2=0x562b0cdab2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.314+0000 7fa1efd9e740 1 -- 172.21.15.52:0/3037402938 shutdown_connections 2024-04-02T14:01:13.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.314+0000 7fa1efd9e740 1 -- 172.21.15.52:0/3037402938 wait complete. 2024-04-02T14:01:13.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.314+0000 7fa1efd9e740 1 -- start start 2024-04-02T14:01:13.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.314+0000 7fa1efd9e740 1 Processor -- start 2024-04-02T14:01:13.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.314+0000 7fa1efd9e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b0cdb07a0 0x562b0cdb96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.314+0000 7fa1efd9e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb10c0 0x562b0cdb6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.314+0000 7fa1efd9e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b0cdb9ca0 0x562b0cdb71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.314+0000 7fa1efd9e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562b0cdbca30 con 0x562b0cdb10c0 2024-04-02T14:01:13.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.314+0000 7fa1efd9e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x562b0cccfe80 con 0x562b0cdb07a0 2024-04-02T14:01:13.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.314+0000 7fa1efd9e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562b0cdbcbb0 con 0x562b0cdb9ca0 2024-04-02T14:01:13.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.314+0000 7fa1efd9e740 10 client.0 _ll_register_callbacks cb 0x562b0cd0baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:13.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.314+0000 7fa1dffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb10c0 0x562b0cdb6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.314+0000 7fa1dffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb10c0 0x562b0cdb6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56116/0 (socket says 172.21.15.52:56116) 2024-04-02T14:01:13.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.314+0000 7fa1dffff640 1 -- 172.21.15.52:0/578855728 learned_addr learned my addr 172.21.15.52:0/578855728 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:13.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.315+0000 7fa1e6d9d640 1 --2- 172.21.15.52:0/578855728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b0cdb07a0 0x562b0cdb96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.315+0000 7fa1e759e640 1 --2- 172.21.15.52:0/578855728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b0cdb9ca0 0x562b0cdb71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.315+0000 7fa1dffff640 1 -- 172.21.15.52:0/578855728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b0cdb9ca0 msgr2=0x562b0cdb71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.315+0000 7fa1dffff640 1 --2- 172.21.15.52:0/578855728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b0cdb9ca0 0x562b0cdb71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.315+0000 7fa1dffff640 1 -- 172.21.15.52:0/578855728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b0cdb07a0 msgr2=0x562b0cdb96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.315+0000 7fa1dffff640 1 --2- 172.21.15.52:0/578855728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b0cdb07a0 0x562b0cdb96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.315+0000 7fa1dffff640 1 -- 172.21.15.52:0/578855728 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b0caf27d0 con 0x562b0cdb10c0 2024-04-02T14:01:13.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.315+0000 7fa1e759e640 1 --2- 172.21.15.52:0/578855728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b0cdb9ca0 0x562b0cdb71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:13.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.315+0000 7fa1e6d9d640 1 --2- 172.21.15.52:0/578855728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b0cdb07a0 0x562b0cdb96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:13.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.315+0000 7fa1dffff640 1 --2- 172.21.15.52:0/578855728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb10c0 0x562b0cdb6a80 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0004ae0 tx=0x7fa1d0004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367cb8561262060c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.315+0000 7fa1ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:13.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.315+0000 7fa1ce7fc640 1 -- 172.21.15.52:0/578855728 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0032060 con 0x562b0cdb10c0 2024-04-02T14:01:13.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.315+0000 7fa1ce7fc640 1 -- 172.21.15.52:0/578855728 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9526+0+0 (secure 0 0 0) 0x7fa1d0005180 con 0x562b0cdb10c0 2024-04-02T14:01:13.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.315+0000 7fa1efd9e740 1 -- 172.21.15.52:0/578855728 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1d0005500 con 0x562b0cdb10c0 2024-04-02T14:01:13.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.315+0000 7fa1ce7fc640 1 -- 172.21.15.52:0/578855728 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa1d0039050 con 0x562b0cdb10c0 2024-04-02T14:01:13.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.315+0000 7fa1efd9e740 10 client.17796 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:13.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.315+0000 7fa1efd9e740 1 -- 172.21.15.52:0/578855728 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562b0cdc8ad0 con 0x562b0cdb10c0 2024-04-02T14:01:13.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.315+0000 7fa1ce7fc640 1 -- 172.21.15.52:0/578855728 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d0005500 con 0x562b0cdb10c0 2024-04-02T14:01:13.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.317+0000 7fa1ce7fc640 1 -- 172.21.15.52:0/578855728 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa1d0011600 con 0x562b0cdb10c0 2024-04-02T14:01:13.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.317+0000 7fa1efd9e740 10 client.17796 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:13.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.317+0000 7fa1efd9e740 10 client.17796 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T14:01:13.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.317+0000 7fa1efd9e740 20 client.17796 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:13.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.317+0000 7fa1efd9e740 1 --2- 172.21.15.52:0/578855728 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x562b0cdc9ee0 0x562b0cdcc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.317+0000 7fa1efd9e740 4 client.17796 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:01:13.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.317+0000 7fa1efd9e740 1 -- 172.21.15.52:0/578855728 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b0cd21cf0 con 0x562b0cdc9ee0 2024-04-02T14:01:13.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.317+0000 7fa1e6d9d640 1 --2- 172.21.15.52:0/578855728 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x562b0cdc9ee0 0x562b0cdcc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.317+0000 7fa1ce7fc640 10 client.17796 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T14:01:13.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.317+0000 7fa1e6d9d640 1 --2- 172.21.15.52:0/578855728 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x562b0cdc9ee0 0x562b0cdcc2d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=60d8cce36f850e33 server_cookie=da261ab18eaffc10 in_seq=0 out_seq=0 2024-04-02T14:01:13.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.317+0000 7fa1ce7fc640 1 -- 172.21.15.52:0/578855728 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562b0cd21cf0 con 0x562b0cdc9ee0 2024-04-02T14:01:13.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.317+0000 7fa1ce7fc640 10 client.17796 handle_command_reply: tid=0 2024-04-02T14:01:13.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.317+0000 7fa1efd9e740 1 client.17796 shutdown 2024-04-02T14:01:13.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.317+0000 7fa1efd9e740 1 -- 172.21.15.52:0/578855728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb10c0 msgr2=0x562b0cdb6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.317+0000 7fa1efd9e740 1 --2- 172.21.15.52:0/578855728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb10c0 0x562b0cdb6a80 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0004ae0 tx=0x7fa1d0004b50 comp rx=0 tx=0).stop 2024-04-02T14:01:13.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.317+0000 7fa1efd9e740 1 -- 172.21.15.52:0/578855728 shutdown_connections 2024-04-02T14:01:13.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.317+0000 7fa1efd9e740 1 --2- 172.21.15.52:0/578855728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b0cdb9ca0 0x562b0cdb71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.317+0000 7fa1efd9e740 1 --2- 172.21.15.52:0/578855728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb10c0 0x562b0cdb6a80 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.317+0000 7fa1efd9e740 1 --2- 172.21.15.52:0/578855728 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x562b0cdc9ee0 0x562b0cdcc2d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.317+0000 7fa1efd9e740 1 --2- 172.21.15.52:0/578855728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b0cdb07a0 0x562b0cdb96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.317+0000 7fa1efd9e740 1 -- 172.21.15.52:0/578855728 >> 172.21.15.52:0/578855728 conn(0x562b0cdaaed0 msgr2=0x562b0cdacc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.317+0000 7fa1ce7fc640 0 client.17796 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T14:01:13.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.317+0000 7fa1efd9e740 1 -- 172.21.15.52:0/578855728 shutdown_connections 2024-04-02T14:01:13.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.317+0000 7fa1efd9e740 1 -- 172.21.15.52:0/578855728 wait complete. 2024-04-02T14:01:13.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.317+0000 7fa1efd9e740 20 client.17796 trim_cache size 0 max 16384 2024-04-02T14:01:13.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.331+0000 7fa1efd9e740 1 Processor -- start 2024-04-02T14:01:13.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.331+0000 7fa1efd9e740 1 -- start start 2024-04-02T14:01:13.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.331+0000 7fa1efd9e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb07a0 0x562b0cdc5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.331+0000 7fa1efd9e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b0cdb10c0 0x562b0cdd1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.331+0000 7fa1efd9e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b0cdb9ca0 0x562b0cdd1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.331+0000 7fa1efd9e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81af020 con 0x562b0cdb07a0 2024-04-02T14:01:13.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.331+0000 7fa1efd9e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81b0b60 con 0x562b0cdb9ca0 2024-04-02T14:01:13.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.331+0000 7fa1efd9e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81af6f0 con 0x562b0cdb10c0 2024-04-02T14:01:13.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.331+0000 7fa1e6d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb07a0 0x562b0cdc5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.331+0000 7fa1e6d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb07a0 0x562b0cdc5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56122/0 (socket says 172.21.15.52:56122) 2024-04-02T14:01:13.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.331+0000 7fa1e6d9d640 1 -- 172.21.15.52:0/724559674 learned_addr learned my addr 172.21.15.52:0/724559674 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:13.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.331+0000 7fa1dffff640 1 --2- 172.21.15.52:0/724559674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b0cdb10c0 0x562b0cdd1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.331+0000 7fa1e6d9d640 1 -- 172.21.15.52:0/724559674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b0cdb10c0 msgr2=0x562b0cdd1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.331+0000 7fa1e6d9d640 1 --2- 172.21.15.52:0/724559674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b0cdb10c0 0x562b0cdd1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.331+0000 7fa1e6d9d640 1 -- 172.21.15.52:0/724559674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b0cdb9ca0 msgr2=0x562b0cdd1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.331+0000 7fa1e6d9d640 1 --2- 172.21.15.52:0/724559674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b0cdb9ca0 0x562b0cdd1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.331+0000 7fa1e6d9d640 1 -- 172.21.15.52:0/724559674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b0cdb7530 con 0x562b0cdb07a0 2024-04-02T14:01:13.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.331+0000 7fa1dffff640 1 --2- 172.21.15.52:0/724559674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b0cdb10c0 0x562b0cdd1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:13.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.331+0000 7fa1e6d9d640 1 --2- 172.21.15.52:0/724559674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb07a0 0x562b0cdc5940 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8006010 tx=0x7fa1d8004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a59d24914948df8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1dcff9640 1 -- 172.21.15.52:0/724559674 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d802cb90 con 0x562b0cdb07a0 2024-04-02T14:01:13.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1dcff9640 1 -- 172.21.15.52:0/724559674 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d802cd30 con 0x562b0cdb07a0 2024-04-02T14:01:13.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1dcff9640 1 -- 172.21.15.52:0/724559674 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d80046c0 con 0x562b0cdb07a0 2024-04-02T14:01:13.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1efd9e740 1 -- 172.21.15.52:0/724559674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb07a0 msgr2=0x562b0cdc5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1efd9e740 1 --2- 172.21.15.52:0/724559674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb07a0 0x562b0cdc5940 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8006010 tx=0x7fa1d8004f10 comp rx=0 tx=0).stop 2024-04-02T14:01:13.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1efd9e740 1 -- 172.21.15.52:0/724559674 shutdown_connections 2024-04-02T14:01:13.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1efd9e740 1 --2- 172.21.15.52:0/724559674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b0cdb9ca0 0x562b0cdd1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1efd9e740 1 --2- 172.21.15.52:0/724559674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b0cdb10c0 0x562b0cdd1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1efd9e740 1 --2- 172.21.15.52:0/724559674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb07a0 0x562b0cdc5940 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1efd9e740 1 -- 172.21.15.52:0/724559674 >> 172.21.15.52:0/724559674 conn(0x562b0cdaaed0 msgr2=0x562b0cdb5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1efd9e740 1 -- 172.21.15.52:0/724559674 shutdown_connections 2024-04-02T14:01:13.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1efd9e740 1 -- 172.21.15.52:0/724559674 wait complete. 2024-04-02T14:01:13.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1efd9e740 1 -- start start 2024-04-02T14:01:13.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1efd9e740 1 Processor -- start 2024-04-02T14:01:13.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1efd9e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b0cdb07a0 0x562b0cdc2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1efd9e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b0cdb10c0 0x562b0cdb3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1efd9e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb9ca0 0x562b0cdb3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1efd9e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81b0490 con 0x562b0cdb9ca0 2024-04-02T14:01:13.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1efd9e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81afdc0 con 0x562b0cdb10c0 2024-04-02T14:01:13.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1efd9e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81ad020 con 0x562b0cdb07a0 2024-04-02T14:01:13.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1efd9e740 10 client.0 _ll_register_callbacks cb 0x7fa1d8005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:13.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1e759e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb9ca0 0x562b0cdb3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1e759e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb9ca0 0x562b0cdb3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56130/0 (socket says 172.21.15.52:56130) 2024-04-02T14:01:13.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1e759e640 1 -- 172.21.15.52:0/2457304620 learned_addr learned my addr 172.21.15.52:0/2457304620 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:13.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1dffff640 1 --2- 172.21.15.52:0/2457304620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b0cdb10c0 0x562b0cdb3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1e759e640 1 -- 172.21.15.52:0/2457304620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b0cdb07a0 msgr2=0x562b0cdc2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1e6d9d640 1 --2- 172.21.15.52:0/2457304620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b0cdb07a0 0x562b0cdc2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1e759e640 1 --2- 172.21.15.52:0/2457304620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b0cdb07a0 0x562b0cdc2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1e759e640 1 -- 172.21.15.52:0/2457304620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b0cdb10c0 msgr2=0x562b0cdb3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1e759e640 1 --2- 172.21.15.52:0/2457304620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b0cdb10c0 0x562b0cdb3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1e759e640 1 -- 172.21.15.52:0/2457304620 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b0cdb0ef0 con 0x562b0cdb9ca0 2024-04-02T14:01:13.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.333+0000 7fa1dffff640 1 --2- 172.21.15.52:0/2457304620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b0cdb10c0 0x562b0cdb3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:13.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.334+0000 7fa1e759e640 1 --2- 172.21.15.52:0/2457304620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb9ca0 0x562b0cdb3a90 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa1e001f000 tx=0x7fa1e00655e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad976244a50e348d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.334+0000 7fa1ce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:13.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.334+0000 7fa1ce7fc640 1 -- 172.21.15.52:0/2457304620 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e009ce30 con 0x562b0cdb9ca0 2024-04-02T14:01:13.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.334+0000 7fa1ce7fc640 1 -- 172.21.15.52:0/2457304620 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9526+0+0 (secure 0 0 0) 0x7fa1e00ce860 con 0x562b0cdb9ca0 2024-04-02T14:01:13.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.334+0000 7fa1efd9e740 1 -- 172.21.15.52:0/2457304620 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1d802cd30 con 0x562b0cdb9ca0 2024-04-02T14:01:13.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.334+0000 7fa1ce7fc640 1 -- 172.21.15.52:0/2457304620 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa1e00651a0 con 0x562b0cdb9ca0 2024-04-02T14:01:13.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.334+0000 7fa1efd9e740 10 client.17802 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:13.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.334+0000 7fa1efd9e740 1 -- 172.21.15.52:0/2457304620 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562b0cdc0c60 con 0x562b0cdb9ca0 2024-04-02T14:01:13.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.334+0000 7fa1ce7fc640 1 -- 172.21.15.52:0/2457304620 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d802cd30 con 0x562b0cdb9ca0 2024-04-02T14:01:13.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.334+0000 7fa1ce7fc640 1 -- 172.21.15.52:0/2457304620 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa1e00a5610 con 0x562b0cdb9ca0 2024-04-02T14:01:13.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.334+0000 7fa1efd9e740 10 client.17802 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:13.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.334+0000 7fa1efd9e740 10 client.17802 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T14:01:13.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.334+0000 7fa1efd9e740 20 client.17802 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:13.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.334+0000 7fa1efd9e740 1 --2- 172.21.15.52:0/2457304620 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x562b0ce306a0 0x562b0ce32a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.334+0000 7fa1efd9e740 4 client.17802 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:01:13.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.334+0000 7fa1efd9e740 1 -- 172.21.15.52:0/2457304620 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b0cd1c040 con 0x562b0ce306a0 2024-04-02T14:01:13.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.334+0000 7fa1e6d9d640 1 --2- 172.21.15.52:0/2457304620 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x562b0ce306a0 0x562b0ce32a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.335+0000 7fa1ce7fc640 10 client.17802 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T14:01:13.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.335+0000 7fa1e6d9d640 1 --2- 172.21.15.52:0/2457304620 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x562b0ce306a0 0x562b0ce32a90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=68bf036885f72965 server_cookie=718a5fc8a105062a in_seq=0 out_seq=0 2024-04-02T14:01:13.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.335+0000 7fa1ce7fc640 1 -- 172.21.15.52:0/2457304620 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b0cd1c040 con 0x562b0ce306a0 2024-04-02T14:01:13.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.335+0000 7fa1ce7fc640 10 client.17802 handle_command_reply: tid=0 2024-04-02T14:01:13.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.335+0000 7fa1efd9e740 1 client.17802 shutdown 2024-04-02T14:01:13.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.335+0000 7fa1efd9e740 1 -- 172.21.15.52:0/2457304620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb9ca0 msgr2=0x562b0cdb3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.335+0000 7fa1efd9e740 1 --2- 172.21.15.52:0/2457304620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb9ca0 0x562b0cdb3a90 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa1e001f000 tx=0x7fa1e00655e0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.335+0000 7fa1efd9e740 1 -- 172.21.15.52:0/2457304620 shutdown_connections 2024-04-02T14:01:13.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.335+0000 7fa1efd9e740 1 --2- 172.21.15.52:0/2457304620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b0cdb9ca0 0x562b0cdb3a90 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.335+0000 7fa1efd9e740 1 --2- 172.21.15.52:0/2457304620 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x562b0ce306a0 0x562b0ce32a90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.335+0000 7fa1efd9e740 1 --2- 172.21.15.52:0/2457304620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b0cdb10c0 0x562b0cdb3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.335+0000 7fa1efd9e740 1 --2- 172.21.15.52:0/2457304620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b0cdb07a0 0x562b0cdc2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.335+0000 7fa1ce7fc640 0 client.17802 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T14:01:13.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.335+0000 7fa1efd9e740 1 -- 172.21.15.52:0/2457304620 >> 172.21.15.52:0/2457304620 conn(0x562b0cdaaed0 msgr2=0x562b0cdac2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.335+0000 7fa1efd9e740 1 -- 172.21.15.52:0/2457304620 shutdown_connections 2024-04-02T14:01:13.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.335+0000 7fa1efd9e740 1 -- 172.21.15.52:0/2457304620 wait complete. 2024-04-02T14:01:13.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.335+0000 7fa1efd9e740 20 client.17802 trim_cache size 0 max 16384 2024-04-02T14:01:13.353 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:01:13.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.337+0000 7fa1ed0d9640 1 -- 172.21.15.52:0/3764182172 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa1d40a3d20 msgr2=0x7fa1d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.337+0000 7fa1ed0d9640 1 --2- 172.21.15.52:0/3764182172 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa1d40a3d20 0x7fa1d40a61e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0005d90 tx=0x7fa1d000bb50 comp rx=0 tx=0).stop 2024-04-02T14:01:13.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.337+0000 7fa1ed0d9640 1 -- 172.21.15.52:0/3764182172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1e8107ab0 msgr2=0x7fa1e819cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.337+0000 7fa1ed0d9640 1 --2- 172.21.15.52:0/3764182172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1e8107ab0 0x7fa1e819cc00 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000cce0 tx=0x7fa1e000a620 comp rx=0 tx=0).stop 2024-04-02T14:01:13.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.337+0000 7fa1ed0d9640 1 -- 172.21.15.52:0/3764182172 shutdown_connections 2024-04-02T14:01:13.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.337+0000 7fa1ed0d9640 1 --2- 172.21.15.52:0/3764182172 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa1d40a3d20 0x7fa1d40a61e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.337+0000 7fa1ed0d9640 1 --2- 172.21.15.52:0/3764182172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1e8107ab0 0x7fa1e819cc00 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.337+0000 7fa1ed0d9640 1 --2- 172.21.15.52:0/3764182172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1e8102460 0x7fa1e81988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.337+0000 7fa1ed0d9640 1 --2- 172.21.15.52:0/3764182172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1e8101aa0 0x7fa1e8198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.337+0000 7fa1ed0d9640 1 -- 172.21.15.52:0/3764182172 >> 172.21.15.52:0/3764182172 conn(0x7fa1e80fd530 msgr2=0x7fa1e80fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.337+0000 7fa1ed0d9640 1 -- 172.21.15.52:0/3764182172 shutdown_connections 2024-04-02T14:01:13.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.337+0000 7fa1ed0d9640 1 -- 172.21.15.52:0/3764182172 wait complete. 2024-04-02T14:01:13.356 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T14:01:13.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.459+0000 7febf259d640 1 -- 172.21.15.52:0/2809197095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febec1003c0 msgr2=0x7febec100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.459+0000 7febf259d640 1 --2- 172.21.15.52:0/2809197095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febec1003c0 0x7febec100840 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7febdc009930 tx=0x7febdc02bcd0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.459+0000 7febf259d640 1 -- 172.21.15.52:0/2809197095 shutdown_connections 2024-04-02T14:01:13.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.459+0000 7febf259d640 1 --2- 172.21.15.52:0/2809197095 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7febec100d80 0x7febec10e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.459+0000 7febf259d640 1 --2- 172.21.15.52:0/2809197095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febec1003c0 0x7febec100840 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.459+0000 7febf259d640 1 --2- 172.21.15.52:0/2809197095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febec107d00 0x7febec1080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.459+0000 7febf259d640 1 -- 172.21.15.52:0/2809197095 >> 172.21.15.52:0/2809197095 conn(0x7febec075d40 msgr2=0x7febec076150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.459+0000 7febf259d640 1 -- 172.21.15.52:0/2809197095 shutdown_connections 2024-04-02T14:01:13.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.459+0000 7febf259d640 1 -- 172.21.15.52:0/2809197095 wait complete. 2024-04-02T14:01:13.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.459+0000 7febf259d640 1 Processor -- start 2024-04-02T14:01:13.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.459+0000 7febf259d640 1 -- start start 2024-04-02T14:01:13.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.460+0000 7febf259d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febec1003c0 0x7febec1015a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.460+0000 7febf259d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febec100d80 0x7febec101ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.460+0000 7febf259d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7febec107d00 0x7febec102020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.460+0000 7febf259d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7febec111f60 con 0x7febec100d80 2024-04-02T14:01:13.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.460+0000 7febf259d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7febec111de0 con 0x7febec107d00 2024-04-02T14:01:13.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.460+0000 7febf259d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7febec1120e0 con 0x7febec1003c0 2024-04-02T14:01:13.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.460+0000 7febeb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febec100d80 0x7febec101ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.460+0000 7febeb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febec100d80 0x7febec101ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56138/0 (socket says 172.21.15.52:56138) 2024-04-02T14:01:13.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.460+0000 7febeb7fe640 1 -- 172.21.15.52:0/3790669033 learned_addr learned my addr 172.21.15.52:0/3790669033 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:13.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.460+0000 7febf0b3a640 1 --2- 172.21.15.52:0/3790669033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7febec107d00 0x7febec102020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.460+0000 7febebfff640 1 --2- 172.21.15.52:0/3790669033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febec1003c0 0x7febec1015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.460+0000 7febeb7fe640 1 -- 172.21.15.52:0/3790669033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febec1003c0 msgr2=0x7febec1015a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.460+0000 7febeb7fe640 1 --2- 172.21.15.52:0/3790669033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febec1003c0 0x7febec1015a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.460+0000 7febeb7fe640 1 -- 172.21.15.52:0/3790669033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7febec107d00 msgr2=0x7febec102020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.460+0000 7febeb7fe640 1 --2- 172.21.15.52:0/3790669033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7febec107d00 0x7febec102020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.460+0000 7febeb7fe640 1 -- 172.21.15.52:0/3790669033 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febec1a68e0 con 0x7febec100d80 2024-04-02T14:01:13.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.460+0000 7febf0b3a640 1 --2- 172.21.15.52:0/3790669033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7febec107d00 0x7febec102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:13.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.461+0000 7febeb7fe640 1 --2- 172.21.15.52:0/3790669033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febec100d80 0x7febec101ae0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7febdc009230 tx=0x7febdc0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.461+0000 7febe97fa640 1 -- 172.21.15.52:0/3790669033 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc042020 con 0x7febec100d80 2024-04-02T14:01:13.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.461+0000 7febf259d640 1 -- 172.21.15.52:0/3790669033 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febec1a6b70 con 0x7febec100d80 2024-04-02T14:01:13.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.461+0000 7febe97fa640 1 -- 172.21.15.52:0/3790669033 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febdc0054e0 con 0x7febec100d80 2024-04-02T14:01:13.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.461+0000 7febe97fa640 1 -- 172.21.15.52:0/3790669033 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc003c90 con 0x7febec100d80 2024-04-02T14:01:13.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.461+0000 7febf259d640 1 -- 172.21.15.52:0/3790669033 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febec1a7110 con 0x7febec100d80 2024-04-02T14:01:13.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.463+0000 7febe97fa640 1 -- 172.21.15.52:0/3790669033 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7febdc045020 con 0x7febec100d80 2024-04-02T14:01:13.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.463+0000 7febf5189740 1 Processor -- start 2024-04-02T14:01:13.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.463+0000 7febf5189740 1 -- start start 2024-04-02T14:01:13.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.464+0000 7febf5189740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e89f2d67a0 0x55e89f2d6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.464+0000 7febf5189740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2d70c0 0x55e89f2df760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.464+0000 7febf5189740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e89f2dfca0 0x55e89f2e2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.464+0000 7febf5189740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e89f23a7b0 con 0x55e89f2d70c0 2024-04-02T14:01:13.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.464+0000 7febf5189740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e89f1d9570 con 0x55e89f2d67a0 2024-04-02T14:01:13.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.464+0000 7febf5189740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e89f236760 con 0x55e89f2dfca0 2024-04-02T14:01:13.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.464+0000 7febe97fa640 1 --2- 172.21.15.52:0/3790669033 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7febd80a3d50 0x7febd80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.464+0000 7febebfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2d70c0 0x55e89f2df760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.464+0000 7febebfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2d70c0 0x55e89f2df760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56146/0 (socket says 172.21.15.52:56146) 2024-04-02T14:01:13.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.464+0000 7febebfff640 1 -- 172.21.15.52:0/2685126954 learned_addr learned my addr 172.21.15.52:0/2685126954 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:13.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.464+0000 7febf0b3a640 1 --2- 172.21.15.52:0/2685126954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e89f2d67a0 0x55e89f2d6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.464+0000 7febe97fa640 1 -- 172.21.15.52:0/3790669033 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9526+0+0 (secure 0 0 0) 0x7febdc0f2930 con 0x7febec100d80 2024-04-02T14:01:13.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.464+0000 7febeb7fe640 1 --2- 172.21.15.52:0/2685126954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e89f2dfca0 0x55e89f2e2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.464+0000 7febf0b3a640 1 --2- 172.21.15.52:0/3790669033 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7febd80a3d50 0x7febd80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.464+0000 7febebfff640 1 -- 172.21.15.52:0/2685126954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e89f2dfca0 msgr2=0x55e89f2e2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.464+0000 7febebfff640 1 --2- 172.21.15.52:0/2685126954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e89f2dfca0 0x55e89f2e2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.464+0000 7febebfff640 1 -- 172.21.15.52:0/2685126954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e89f2d67a0 msgr2=0x55e89f2d6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.464+0000 7febebfff640 1 --2- 172.21.15.52:0/2685126954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e89f2d67a0 0x55e89f2d6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.464+0000 7febebfff640 1 -- 172.21.15.52:0/2685126954 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e89ef4e3f0 con 0x55e89f2d70c0 2024-04-02T14:01:13.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.464+0000 7febeb7fe640 1 --2- 172.21.15.52:0/2685126954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e89f2dfca0 0x55e89f2e2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:13.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.464+0000 7febebfff640 1 --2- 172.21.15.52:0/2685126954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2d70c0 0x55e89f2df760 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7febd400bc20 tx=0x7febd400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e93b597ec9fa6f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.465+0000 7febf0b3a640 1 --2- 172.21.15.52:0/3790669033 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7febd80a3d50 0x7febd80a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7febe4007310 tx=0x7febe40072a0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.465+0000 7febe27fc640 1 -- 172.21.15.52:0/2685126954 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd4015070 con 0x55e89f2d70c0 2024-04-02T14:01:13.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.465+0000 7febe27fc640 1 -- 172.21.15.52:0/2685126954 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febd4005500 con 0x55e89f2d70c0 2024-04-02T14:01:13.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.465+0000 7febe27fc640 1 -- 172.21.15.52:0/2685126954 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd40116a0 con 0x55e89f2d70c0 2024-04-02T14:01:13.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.465+0000 7febf5189740 1 -- 172.21.15.52:0/2685126954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2d70c0 msgr2=0x55e89f2df760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.465+0000 7febf5189740 1 --2- 172.21.15.52:0/2685126954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2d70c0 0x55e89f2df760 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7febd400bc20 tx=0x7febd400f730 comp rx=0 tx=0).stop 2024-04-02T14:01:13.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.465+0000 7febf5189740 1 -- 172.21.15.52:0/2685126954 shutdown_connections 2024-04-02T14:01:13.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.465+0000 7febf5189740 1 --2- 172.21.15.52:0/2685126954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e89f2dfca0 0x55e89f2e2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.465+0000 7febf5189740 1 --2- 172.21.15.52:0/2685126954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2d70c0 0x55e89f2df760 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.465+0000 7febf5189740 1 --2- 172.21.15.52:0/2685126954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e89f2d67a0 0x55e89f2d6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.465+0000 7febf5189740 1 -- 172.21.15.52:0/2685126954 >> 172.21.15.52:0/2685126954 conn(0x55e89f2d0ed0 msgr2=0x55e89f2d12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.465+0000 7febf5189740 1 -- 172.21.15.52:0/2685126954 shutdown_connections 2024-04-02T14:01:13.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.465+0000 7febf5189740 1 -- 172.21.15.52:0/2685126954 wait complete. 2024-04-02T14:01:13.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.466+0000 7febf5189740 1 -- start start 2024-04-02T14:01:13.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.466+0000 7febf5189740 1 Processor -- start 2024-04-02T14:01:13.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.466+0000 7febf5189740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e89f2d67a0 0x55e89f2df6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.466+0000 7febf5189740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2d70c0 0x55e89f2dca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.466+0000 7febf5189740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e89f2dfca0 0x55e89f2dd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.466+0000 7febf5189740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e89f2e2a90 con 0x55e89f2d70c0 2024-04-02T14:01:13.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.466+0000 7febf5189740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e89f1f5e80 con 0x55e89f2dfca0 2024-04-02T14:01:13.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.466+0000 7febf5189740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e89f2e2c10 con 0x55e89f2d67a0 2024-04-02T14:01:13.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.466+0000 7febf5189740 10 client.0 _ll_register_callbacks cb 0x55e89f231aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:13.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.466+0000 7febeb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2d70c0 0x55e89f2dca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.466+0000 7febeb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2d70c0 0x55e89f2dca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56156/0 (socket says 172.21.15.52:56156) 2024-04-02T14:01:13.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.466+0000 7febeb7fe640 1 -- 172.21.15.52:0/2505889191 learned_addr learned my addr 172.21.15.52:0/2505889191 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:13.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.466+0000 7febf0b3a640 1 --2- 172.21.15.52:0/2505889191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e89f2dfca0 0x55e89f2dd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.466+0000 7febebfff640 1 --2- 172.21.15.52:0/2505889191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e89f2d67a0 0x55e89f2df6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.466+0000 7febeb7fe640 1 -- 172.21.15.52:0/2505889191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e89f2d67a0 msgr2=0x55e89f2df6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.466+0000 7febeb7fe640 1 --2- 172.21.15.52:0/2505889191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e89f2d67a0 0x55e89f2df6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.466+0000 7febeb7fe640 1 -- 172.21.15.52:0/2505889191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e89f2dfca0 msgr2=0x55e89f2dd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.466+0000 7febeb7fe640 1 --2- 172.21.15.52:0/2505889191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e89f2dfca0 0x55e89f2dd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.466+0000 7febeb7fe640 1 -- 172.21.15.52:0/2505889191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e89f0187d0 con 0x55e89f2d70c0 2024-04-02T14:01:13.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.467+0000 7febeb7fe640 1 --2- 172.21.15.52:0/2505889191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2d70c0 0x55e89f2dca30 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7febdc009900 tx=0x7febdc0ba950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd418f374152544 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.467+0000 7febcb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:13.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.467+0000 7febcb7fe640 1 -- 172.21.15.52:0/2505889191 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc0f0120 con 0x55e89f2d70c0 2024-04-02T14:01:13.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.467+0000 7febcb7fe640 1 -- 172.21.15.52:0/2505889191 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9526+0+0 (secure 0 0 0) 0x7febdc0be850 con 0x55e89f2d70c0 2024-04-02T14:01:13.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.467+0000 7febf5189740 1 -- 172.21.15.52:0/2505889191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febd4005500 con 0x55e89f2d70c0 2024-04-02T14:01:13.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.467+0000 7febcb7fe640 1 -- 172.21.15.52:0/2505889191 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7febdc004c80 con 0x55e89f2d70c0 2024-04-02T14:01:13.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.467+0000 7febf5189740 10 client.17814 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:13.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.467+0000 7febf5189740 1 -- 172.21.15.52:0/2505889191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e89f2ee9f0 con 0x55e89f2d70c0 2024-04-02T14:01:13.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.467+0000 7febcb7fe640 1 -- 172.21.15.52:0/2505889191 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febd4005500 con 0x55e89f2d70c0 2024-04-02T14:01:13.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.467+0000 7febcb7fe640 1 -- 172.21.15.52:0/2505889191 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7febdc0dd410 con 0x55e89f2d70c0 2024-04-02T14:01:13.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.467+0000 7febf5189740 10 client.17814 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:13.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.467+0000 7febf5189740 10 client.17814 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T14:01:13.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.467+0000 7febf5189740 20 client.17814 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:13.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.467+0000 7febf5189740 1 --2- 172.21.15.52:0/2505889191 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55e89f2efef0 0x55e89f2f2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.467+0000 7febf5189740 4 client.17814 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:01:13.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.467+0000 7febf5189740 1 -- 172.21.15.52:0/2505889191 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e89f247cf0 con 0x55e89f2efef0 2024-04-02T14:01:13.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.468+0000 7febebfff640 1 --2- 172.21.15.52:0/2505889191 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55e89f2efef0 0x55e89f2f2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.468+0000 7febebfff640 1 --2- 172.21.15.52:0/2505889191 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55e89f2efef0 0x55e89f2f2340 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=80558ceadfd90a77 server_cookie=8963ca0ec7402c29 in_seq=0 out_seq=0 2024-04-02T14:01:13.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.468+0000 7febcb7fe640 10 client.17814 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T14:01:13.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.469+0000 7febcb7fe640 1 -- 172.21.15.52:0/2505889191 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e89f247cf0 con 0x55e89f2efef0 2024-04-02T14:01:13.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.469+0000 7febcb7fe640 10 client.17814 handle_command_reply: tid=0 2024-04-02T14:01:13.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.469+0000 7febf5189740 1 client.17814 shutdown 2024-04-02T14:01:13.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.469+0000 7febf5189740 1 -- 172.21.15.52:0/2505889191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2d70c0 msgr2=0x55e89f2dca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.469+0000 7febf5189740 1 --2- 172.21.15.52:0/2505889191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2d70c0 0x55e89f2dca30 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7febdc009900 tx=0x7febdc0ba950 comp rx=0 tx=0).stop 2024-04-02T14:01:13.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.469+0000 7febf5189740 1 -- 172.21.15.52:0/2505889191 shutdown_connections 2024-04-02T14:01:13.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.469+0000 7febf5189740 1 --2- 172.21.15.52:0/2505889191 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55e89f2efef0 0x55e89f2f2340 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.469+0000 7febf5189740 1 --2- 172.21.15.52:0/2505889191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e89f2dfca0 0x55e89f2dd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.469+0000 7febf5189740 1 --2- 172.21.15.52:0/2505889191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2d70c0 0x55e89f2dca30 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.469+0000 7febf5189740 1 --2- 172.21.15.52:0/2505889191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e89f2d67a0 0x55e89f2df6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.469+0000 7febcb7fe640 0 client.17814 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T14:01:13.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.469+0000 7febf5189740 1 -- 172.21.15.52:0/2505889191 >> 172.21.15.52:0/2505889191 conn(0x55e89f2d0ed0 msgr2=0x55e89f2d2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.469+0000 7febf5189740 1 -- 172.21.15.52:0/2505889191 shutdown_connections 2024-04-02T14:01:13.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.469+0000 7febf5189740 1 -- 172.21.15.52:0/2505889191 wait complete. 2024-04-02T14:01:13.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.469+0000 7febf5189740 20 client.17814 trim_cache size 0 max 16384 2024-04-02T14:01:13.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.482+0000 7febf5189740 1 Processor -- start 2024-04-02T14:01:13.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.482+0000 7febf5189740 1 -- start start 2024-04-02T14:01:13.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.483+0000 7febf5189740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e89f2d67a0 0x55e89f2eb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.483+0000 7febf5189740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e89f2d70c0 0x55e89f2d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.483+0000 7febf5189740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2dfca0 0x55e89f2f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.483+0000 7febf5189740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7febec1b11b0 con 0x55e89f2dfca0 2024-04-02T14:01:13.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.483+0000 7febf5189740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7febec1b2cf0 con 0x55e89f2d70c0 2024-04-02T14:01:13.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.483+0000 7febf5189740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7febec1b1880 con 0x55e89f2d67a0 2024-04-02T14:01:13.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.483+0000 7febf0b3a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2dfca0 0x55e89f2f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.483+0000 7febf0b3a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2dfca0 0x55e89f2f7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56158/0 (socket says 172.21.15.52:56158) 2024-04-02T14:01:13.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.483+0000 7febf0b3a640 1 -- 172.21.15.52:0/2756918449 learned_addr learned my addr 172.21.15.52:0/2756918449 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:13.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.483+0000 7febebfff640 1 --2- 172.21.15.52:0/2756918449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e89f2d67a0 0x55e89f2eb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.483+0000 7febeb7fe640 1 --2- 172.21.15.52:0/2756918449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e89f2d70c0 0x55e89f2d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.483+0000 7febf0b3a640 1 -- 172.21.15.52:0/2756918449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e89f2d67a0 msgr2=0x55e89f2eb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.483+0000 7febf0b3a640 1 --2- 172.21.15.52:0/2756918449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e89f2d67a0 0x55e89f2eb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.483+0000 7febf0b3a640 1 -- 172.21.15.52:0/2756918449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e89f2d70c0 msgr2=0x55e89f2d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.483+0000 7febf0b3a640 1 --2- 172.21.15.52:0/2756918449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e89f2d70c0 0x55e89f2d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.483+0000 7febf0b3a640 1 -- 172.21.15.52:0/2756918449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e89f2d6ef0 con 0x55e89f2dfca0 2024-04-02T14:01:13.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.483+0000 7febebfff640 1 --2- 172.21.15.52:0/2756918449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e89f2d67a0 0x55e89f2eb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:13.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.483+0000 7febf0b3a640 1 --2- 172.21.15.52:0/2756918449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2dfca0 0x55e89f2f7c50 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7febe40109f0 tx=0x7febe4007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8bdf03d38fadb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.483+0000 7febe1ffb640 1 -- 172.21.15.52:0/2756918449 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4004d20 con 0x55e89f2dfca0 2024-04-02T14:01:13.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.483+0000 7febe1ffb640 1 -- 172.21.15.52:0/2756918449 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febe4005020 con 0x55e89f2dfca0 2024-04-02T14:01:13.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.483+0000 7febe1ffb640 1 -- 172.21.15.52:0/2756918449 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe400e690 con 0x55e89f2dfca0 2024-04-02T14:01:13.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.484+0000 7febf5189740 1 -- 172.21.15.52:0/2756918449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2dfca0 msgr2=0x55e89f2f7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.484+0000 7febf5189740 1 --2- 172.21.15.52:0/2756918449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2dfca0 0x55e89f2f7c50 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7febe40109f0 tx=0x7febe4007680 comp rx=0 tx=0).stop 2024-04-02T14:01:13.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.484+0000 7febf5189740 1 -- 172.21.15.52:0/2756918449 shutdown_connections 2024-04-02T14:01:13.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.484+0000 7febf5189740 1 --2- 172.21.15.52:0/2756918449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2dfca0 0x55e89f2f7c50 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.484+0000 7febf5189740 1 --2- 172.21.15.52:0/2756918449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e89f2d70c0 0x55e89f2d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.484+0000 7febf5189740 1 --2- 172.21.15.52:0/2756918449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e89f2d67a0 0x55e89f2eb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.484+0000 7febf5189740 1 -- 172.21.15.52:0/2756918449 >> 172.21.15.52:0/2756918449 conn(0x55e89f2d0ed0 msgr2=0x55e89f2dc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.484+0000 7febf5189740 1 -- 172.21.15.52:0/2756918449 shutdown_connections 2024-04-02T14:01:13.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.484+0000 7febf5189740 1 -- 172.21.15.52:0/2756918449 wait complete. 2024-04-02T14:01:13.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.484+0000 7febf5189740 1 -- start start 2024-04-02T14:01:13.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.484+0000 7febf5189740 1 Processor -- start 2024-04-02T14:01:13.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.485+0000 7febf5189740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e89f2d67a0 0x55e89f2e0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.485+0000 7febf5189740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2d70c0 0x55e89f2e0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.485+0000 7febf5189740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e89f2dfca0 0x55e89f2e14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.485+0000 7febf5189740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7febec1b2620 con 0x55e89f2d70c0 2024-04-02T14:01:13.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.485+0000 7febf5189740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7febec1b1f50 con 0x55e89f2dfca0 2024-04-02T14:01:13.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.485+0000 7febf5189740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7febec1af1b0 con 0x55e89f2d67a0 2024-04-02T14:01:13.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.485+0000 7febf5189740 10 client.0 _ll_register_callbacks cb 0x7febdc0035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:13.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.485+0000 7febeb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2d70c0 0x55e89f2e0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.485+0000 7febeb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2d70c0 0x55e89f2e0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56174/0 (socket says 172.21.15.52:56174) 2024-04-02T14:01:13.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.485+0000 7febeb7fe640 1 -- 172.21.15.52:0/3506495119 learned_addr learned my addr 172.21.15.52:0/3506495119 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:13.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.485+0000 7febebfff640 1 --2- 172.21.15.52:0/3506495119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e89f2d67a0 0x55e89f2e0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.485+0000 7febf0b3a640 1 --2- 172.21.15.52:0/3506495119 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e89f2dfca0 0x55e89f2e14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.485+0000 7febeb7fe640 1 -- 172.21.15.52:0/3506495119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e89f2d67a0 msgr2=0x55e89f2e0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.485+0000 7febeb7fe640 1 --2- 172.21.15.52:0/3506495119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e89f2d67a0 0x55e89f2e0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.485+0000 7febeb7fe640 1 -- 172.21.15.52:0/3506495119 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e89f2dfca0 msgr2=0x55e89f2e14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.485+0000 7febeb7fe640 1 --2- 172.21.15.52:0/3506495119 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e89f2dfca0 0x55e89f2e14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.485+0000 7febeb7fe640 1 -- 172.21.15.52:0/3506495119 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e89f2e5fa0 con 0x55e89f2d70c0 2024-04-02T14:01:13.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.485+0000 7febf0b3a640 1 --2- 172.21.15.52:0/3506495119 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e89f2dfca0 0x55e89f2e14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:13.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.485+0000 7febeb7fe640 1 --2- 172.21.15.52:0/3506495119 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2d70c0 0x55e89f2e0fa0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7febdc0c7260 tx=0x7febdc0ba240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5003cdd713e137 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.485+0000 7febcb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:13.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.485+0000 7febcb7fe640 1 -- 172.21.15.52:0/3506495119 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc004c80 con 0x55e89f2d70c0 2024-04-02T14:01:13.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.485+0000 7febcb7fe640 1 -- 172.21.15.52:0/3506495119 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9526+0+0 (secure 0 0 0) 0x7febdc0c6dd0 con 0x55e89f2d70c0 2024-04-02T14:01:13.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.485+0000 7febf5189740 1 -- 172.21.15.52:0/3506495119 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febe4005020 con 0x55e89f2d70c0 2024-04-02T14:01:13.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.486+0000 7febcb7fe640 1 -- 172.21.15.52:0/3506495119 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7febdc0bead0 con 0x55e89f2d70c0 2024-04-02T14:01:13.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.486+0000 7febf5189740 10 client.17820 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:13.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.486+0000 7febf5189740 1 -- 172.21.15.52:0/3506495119 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e89f356750 con 0x55e89f2d70c0 2024-04-02T14:01:13.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.486+0000 7febcb7fe640 1 -- 172.21.15.52:0/3506495119 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febe4005020 con 0x55e89f2d70c0 2024-04-02T14:01:13.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.486+0000 7febcb7fe640 1 -- 172.21.15.52:0/3506495119 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7febdc0a7b30 con 0x55e89f2d70c0 2024-04-02T14:01:13.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.486+0000 7febf5189740 10 client.17820 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:13.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.486+0000 7febf5189740 10 client.17820 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T14:01:13.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.486+0000 7febf5189740 20 client.17820 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:13.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.486+0000 7febf5189740 1 --2- 172.21.15.52:0/3506495119 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55e89f356d40 0x55e89f359160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.486+0000 7febf5189740 4 client.17820 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:01:13.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.486+0000 7febf5189740 1 -- 172.21.15.52:0/3506495119 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e89f242040 con 0x55e89f356d40 2024-04-02T14:01:13.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.486+0000 7febebfff640 1 --2- 172.21.15.52:0/3506495119 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55e89f356d40 0x55e89f359160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.487+0000 7febcb7fe640 10 client.17820 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T14:01:13.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.487+0000 7febebfff640 1 --2- 172.21.15.52:0/3506495119 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55e89f356d40 0x55e89f359160 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e865c2073fb9372 server_cookie=5c085eff6309995b in_seq=0 out_seq=0 2024-04-02T14:01:13.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.487+0000 7febcb7fe640 1 -- 172.21.15.52:0/3506495119 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e89f242040 con 0x55e89f356d40 2024-04-02T14:01:13.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.487+0000 7febcb7fe640 10 client.17820 handle_command_reply: tid=0 2024-04-02T14:01:13.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.487+0000 7febf5189740 1 client.17820 shutdown 2024-04-02T14:01:13.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.487+0000 7febf5189740 1 -- 172.21.15.52:0/3506495119 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2d70c0 msgr2=0x55e89f2e0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.487+0000 7febf5189740 1 --2- 172.21.15.52:0/3506495119 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2d70c0 0x55e89f2e0fa0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7febdc0c7260 tx=0x7febdc0ba240 comp rx=0 tx=0).stop 2024-04-02T14:01:13.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.487+0000 7febf5189740 1 -- 172.21.15.52:0/3506495119 shutdown_connections 2024-04-02T14:01:13.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.487+0000 7febf5189740 1 --2- 172.21.15.52:0/3506495119 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55e89f356d40 0x55e89f359160 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.487+0000 7febf5189740 1 --2- 172.21.15.52:0/3506495119 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e89f2dfca0 0x55e89f2e14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.487+0000 7febf5189740 1 --2- 172.21.15.52:0/3506495119 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e89f2d70c0 0x55e89f2e0fa0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.487+0000 7febcb7fe640 0 client.17820 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T14:01:13.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.487+0000 7febf5189740 1 --2- 172.21.15.52:0/3506495119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e89f2d67a0 0x55e89f2e0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.487+0000 7febf5189740 1 -- 172.21.15.52:0/3506495119 >> 172.21.15.52:0/3506495119 conn(0x55e89f2d0ed0 msgr2=0x55e89f2dc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.487+0000 7febf5189740 1 -- 172.21.15.52:0/3506495119 shutdown_connections 2024-04-02T14:01:13.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.487+0000 7febf5189740 1 -- 172.21.15.52:0/3506495119 wait complete. 2024-04-02T14:01:13.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.487+0000 7febf5189740 20 client.17820 trim_cache size 0 max 16384 2024-04-02T14:01:13.504 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:01:13.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.488+0000 7febf259d640 1 -- 172.21.15.52:0/3790669033 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7febd80a3d50 msgr2=0x7febd80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.488+0000 7febf259d640 1 --2- 172.21.15.52:0/3790669033 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7febd80a3d50 0x7febd80a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7febe4007310 tx=0x7febe40072a0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.488+0000 7febf259d640 1 -- 172.21.15.52:0/3790669033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febec100d80 msgr2=0x7febec101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.488+0000 7febf259d640 1 --2- 172.21.15.52:0/3790669033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febec100d80 0x7febec101ae0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7febdc009230 tx=0x7febdc0043b0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.488+0000 7febf259d640 1 -- 172.21.15.52:0/3790669033 shutdown_connections 2024-04-02T14:01:13.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.488+0000 7febf259d640 1 --2- 172.21.15.52:0/3790669033 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7febd80a3d50 0x7febd80a6210 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.488+0000 7febf259d640 1 --2- 172.21.15.52:0/3790669033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7febec107d00 0x7febec102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.488+0000 7febf259d640 1 --2- 172.21.15.52:0/3790669033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febec100d80 0x7febec101ae0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.488+0000 7febf259d640 1 --2- 172.21.15.52:0/3790669033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febec1003c0 0x7febec1015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.488+0000 7febf259d640 1 -- 172.21.15.52:0/3790669033 >> 172.21.15.52:0/3790669033 conn(0x7febec075d40 msgr2=0x7febec0fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.488+0000 7febf259d640 1 -- 172.21.15.52:0/3790669033 shutdown_connections 2024-04-02T14:01:13.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.488+0000 7febf259d640 1 -- 172.21.15.52:0/3790669033 wait complete. 2024-04-02T14:01:13.507 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T14:01:13.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.609+0000 7ffb89d60640 1 -- 172.21.15.52:0/283385608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb84107aa0 msgr2=0x7ffb84107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.609+0000 7ffb89d60640 1 --2- 172.21.15.52:0/283385608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb84107aa0 0x7ffb84107e80 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009930 tx=0x7ffb7402bc30 comp rx=0 tx=0).stop 2024-04-02T14:01:13.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.609+0000 7ffb89d60640 1 -- 172.21.15.52:0/283385608 shutdown_connections 2024-04-02T14:01:13.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.609+0000 7ffb89d60640 1 --2- 172.21.15.52:0/283385608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb84102450 0x7ffb8410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.609+0000 7ffb89d60640 1 --2- 172.21.15.52:0/283385608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb84101a90 0x7ffb84101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.609+0000 7ffb89d60640 1 --2- 172.21.15.52:0/283385608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb84107aa0 0x7ffb84107e80 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.609+0000 7ffb89d60640 1 -- 172.21.15.52:0/283385608 >> 172.21.15.52:0/283385608 conn(0x7ffb84078f00 msgr2=0x7ffb84079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.609+0000 7ffb89d60640 1 -- 172.21.15.52:0/283385608 shutdown_connections 2024-04-02T14:01:13.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.609+0000 7ffb89d60640 1 -- 172.21.15.52:0/283385608 wait complete. 2024-04-02T14:01:13.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.610+0000 7ffb89d60640 1 Processor -- start 2024-04-02T14:01:13.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.610+0000 7ffb89d60640 1 -- start start 2024-04-02T14:01:13.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.610+0000 7ffb89d60640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb84101a90 0x7ffb84198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.610+0000 7ffb89d60640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb84102450 0x7ffb841988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.610+0000 7ffb89d60640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb84107aa0 0x7ffb8419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.610+0000 7ffb89d60640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffb84111eb0 con 0x7ffb84101a90 2024-04-02T14:01:13.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.610+0000 7ffb89d60640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffb84111d30 con 0x7ffb84107aa0 2024-04-02T14:01:13.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.610+0000 7ffb89d60640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffb84112030 con 0x7ffb84102450 2024-04-02T14:01:13.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.610+0000 7ffb837fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb84101a90 0x7ffb84198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.610+0000 7ffb837fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb84101a90 0x7ffb84198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56196/0 (socket says 172.21.15.52:56196) 2024-04-02T14:01:13.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.610+0000 7ffb837fe640 1 -- 172.21.15.52:0/1895519534 learned_addr learned my addr 172.21.15.52:0/1895519534 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:13.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.610+0000 7ffb83fff640 1 --2- 172.21.15.52:0/1895519534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb84107aa0 0x7ffb8419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.610+0000 7ffb82ffd640 1 --2- 172.21.15.52:0/1895519534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb84102450 0x7ffb841988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.610+0000 7ffb837fe640 1 -- 172.21.15.52:0/1895519534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb84102450 msgr2=0x7ffb841988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.610+0000 7ffb837fe640 1 --2- 172.21.15.52:0/1895519534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb84102450 0x7ffb841988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.610+0000 7ffb837fe640 1 -- 172.21.15.52:0/1895519534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb84107aa0 msgr2=0x7ffb8419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.610+0000 7ffb837fe640 1 --2- 172.21.15.52:0/1895519534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb84107aa0 0x7ffb8419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.610+0000 7ffb837fe640 1 -- 172.21.15.52:0/1895519534 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8419d380 con 0x7ffb84101a90 2024-04-02T14:01:13.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.611+0000 7ffb837fe640 1 --2- 172.21.15.52:0/1895519534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb84101a90 0x7ffb84198390 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009230 tx=0x7ffb7402cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.611+0000 7ffb80ff9640 1 -- 172.21.15.52:0/1895519534 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb74041070 con 0x7ffb84101a90 2024-04-02T14:01:13.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.611+0000 7ffb80ff9640 1 -- 172.21.15.52:0/1895519534 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb74004450 con 0x7ffb84101a90 2024-04-02T14:01:13.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.611+0000 7ffb80ff9640 1 -- 172.21.15.52:0/1895519534 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb74034440 con 0x7ffb84101a90 2024-04-02T14:01:13.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.611+0000 7ffb89d60640 1 -- 172.21.15.52:0/1895519534 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb8419d610 con 0x7ffb84101a90 2024-04-02T14:01:13.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.611+0000 7ffb89d60640 1 -- 172.21.15.52:0/1895519534 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb841a4f00 con 0x7ffb84101a90 2024-04-02T14:01:13.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.613+0000 7ffb80ff9640 1 -- 172.21.15.52:0/1895519534 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffb74044070 con 0x7ffb84101a90 2024-04-02T14:01:13.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.613+0000 7ffb8ca24740 1 Processor -- start 2024-04-02T14:01:13.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.613+0000 7ffb8ca24740 1 -- start start 2024-04-02T14:01:13.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.614+0000 7ffb8ca24740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561362507a0 0x556136250b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.614+0000 7ffb80ff9640 1 --2- 172.21.15.52:0/1895519534 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ffb700a3d20 0x7ffb700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.614+0000 7ffb8ca24740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561362510c0 0x556136259760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.614+0000 7ffb8ca24740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556136259ca0 0x55613625c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.614+0000 7ffb8ca24740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5561361b47b0 con 0x5561362510c0 2024-04-02T14:01:13.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.614+0000 7ffb8ca24740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556136153570 con 0x556136259ca0 2024-04-02T14:01:13.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.614+0000 7ffb8ca24740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5561361b0760 con 0x5561362507a0 2024-04-02T14:01:13.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.614+0000 7ffb82ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561362510c0 0x556136259760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.614+0000 7ffb83fff640 1 --2- 172.21.15.52:0/1895519534 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ffb700a3d20 0x7ffb700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.614+0000 7ffb82ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561362510c0 0x556136259760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56208/0 (socket says 172.21.15.52:56208) 2024-04-02T14:01:13.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.614+0000 7ffb82ffd640 1 -- 172.21.15.52:0/2059342012 learned_addr learned my addr 172.21.15.52:0/2059342012 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:13.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.614+0000 7ffb80ff9640 1 -- 172.21.15.52:0/1895519534 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9526+0+0 (secure 0 0 0) 0x7ffb740f24f0 con 0x7ffb84101a90 2024-04-02T14:01:13.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.614+0000 7ffb837fe640 1 --2- 172.21.15.52:0/2059342012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561362507a0 0x556136250b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.614+0000 7ffb83fff640 1 --2- 172.21.15.52:0/2059342012 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556136259ca0 0x55613625c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.614+0000 7ffb82ffd640 1 -- 172.21.15.52:0/2059342012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561362507a0 msgr2=0x556136250b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.614+0000 7ffb82ffd640 1 --2- 172.21.15.52:0/2059342012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561362507a0 0x556136250b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.614+0000 7ffb82ffd640 1 -- 172.21.15.52:0/2059342012 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556136259ca0 msgr2=0x55613625c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.614+0000 7ffb82ffd640 1 --2- 172.21.15.52:0/2059342012 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556136259ca0 0x55613625c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.614+0000 7ffb82ffd640 1 -- 172.21.15.52:0/2059342012 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556135ec83f0 con 0x5561362510c0 2024-04-02T14:01:13.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.614+0000 7ffb83fff640 1 --2- 172.21.15.52:0/1895519534 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ffb700a3d20 0x7ffb700a61e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c005fd0 tx=0x7ffb7c012ad0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.614+0000 7ffb82ffd640 1 --2- 172.21.15.52:0/2059342012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561362510c0 0x556136259760 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c00b740 tx=0x7ffb6c00bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27cc6f531e44a37 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.614+0000 7ffb79ffb640 1 -- 172.21.15.52:0/2059342012 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c003dd0 con 0x5561362510c0 2024-04-02T14:01:13.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.615+0000 7ffb79ffb640 1 -- 172.21.15.52:0/2059342012 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb6c015070 con 0x5561362510c0 2024-04-02T14:01:13.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.615+0000 7ffb79ffb640 1 -- 172.21.15.52:0/2059342012 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c004da0 con 0x5561362510c0 2024-04-02T14:01:13.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.615+0000 7ffb8ca24740 1 -- 172.21.15.52:0/2059342012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561362510c0 msgr2=0x556136259760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.615+0000 7ffb8ca24740 1 --2- 172.21.15.52:0/2059342012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561362510c0 0x556136259760 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c00b740 tx=0x7ffb6c00bc00 comp rx=0 tx=0).stop 2024-04-02T14:01:13.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.615+0000 7ffb8ca24740 1 -- 172.21.15.52:0/2059342012 shutdown_connections 2024-04-02T14:01:13.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.615+0000 7ffb8ca24740 1 --2- 172.21.15.52:0/2059342012 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556136259ca0 0x55613625c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.615+0000 7ffb8ca24740 1 --2- 172.21.15.52:0/2059342012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561362510c0 0x556136259760 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.615+0000 7ffb8ca24740 1 --2- 172.21.15.52:0/2059342012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561362507a0 0x556136250b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.615+0000 7ffb8ca24740 1 -- 172.21.15.52:0/2059342012 >> 172.21.15.52:0/2059342012 conn(0x55613624aed0 msgr2=0x55613624b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.615+0000 7ffb8ca24740 1 -- 172.21.15.52:0/2059342012 shutdown_connections 2024-04-02T14:01:13.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.615+0000 7ffb8ca24740 1 -- 172.21.15.52:0/2059342012 wait complete. 2024-04-02T14:01:13.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.615+0000 7ffb8ca24740 1 -- start start 2024-04-02T14:01:13.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.616+0000 7ffb8ca24740 1 Processor -- start 2024-04-02T14:01:13.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.616+0000 7ffb8ca24740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561362507a0 0x556136259740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.616+0000 7ffb8ca24740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561362510c0 0x556136256ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.616+0000 7ffb8ca24740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556136259ca0 0x5561362571c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.616+0000 7ffb8ca24740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55613625ca90 con 0x5561362507a0 2024-04-02T14:01:13.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.616+0000 7ffb8ca24740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55613616fe80 con 0x556136259ca0 2024-04-02T14:01:13.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.616+0000 7ffb8ca24740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55613625cc10 con 0x5561362510c0 2024-04-02T14:01:13.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.616+0000 7ffb8ca24740 10 client.0 _ll_register_callbacks cb 0x5561361abaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:13.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.616+0000 7ffb837fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561362507a0 0x556136259740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.616+0000 7ffb837fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561362507a0 0x556136259740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56214/0 (socket says 172.21.15.52:56214) 2024-04-02T14:01:13.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.616+0000 7ffb837fe640 1 -- 172.21.15.52:0/2475937305 learned_addr learned my addr 172.21.15.52:0/2475937305 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:13.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.616+0000 7ffb83fff640 1 --2- 172.21.15.52:0/2475937305 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556136259ca0 0x5561362571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.616+0000 7ffb82ffd640 1 --2- 172.21.15.52:0/2475937305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561362510c0 0x556136256ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.616+0000 7ffb837fe640 1 -- 172.21.15.52:0/2475937305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561362510c0 msgr2=0x556136256ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.616+0000 7ffb837fe640 1 --2- 172.21.15.52:0/2475937305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561362510c0 0x556136256ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.616+0000 7ffb837fe640 1 -- 172.21.15.52:0/2475937305 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556136259ca0 msgr2=0x5561362571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.616+0000 7ffb837fe640 1 --2- 172.21.15.52:0/2475937305 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556136259ca0 0x5561362571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.616+0000 7ffb837fe640 1 -- 172.21.15.52:0/2475937305 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556135f927d0 con 0x5561362507a0 2024-04-02T14:01:13.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.617+0000 7ffb837fe640 1 --2- 172.21.15.52:0/2475937305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561362507a0 0x556136259740 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7ffb740efe50 tx=0x7ffb74009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956f742e3aadc5c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.617+0000 7ffb62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:13.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.617+0000 7ffb62ffd640 1 -- 172.21.15.52:0/2475937305 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7404b040 con 0x5561362507a0 2024-04-02T14:01:13.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.617+0000 7ffb62ffd640 1 -- 172.21.15.52:0/2475937305 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9526+0+0 (secure 0 0 0) 0x7ffb740d76b0 con 0x5561362507a0 2024-04-02T14:01:13.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.617+0000 7ffb8ca24740 1 -- 172.21.15.52:0/2475937305 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb6c015070 con 0x5561362507a0 2024-04-02T14:01:13.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.617+0000 7ffb62ffd640 1 -- 172.21.15.52:0/2475937305 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffb740bdbc0 con 0x5561362507a0 2024-04-02T14:01:13.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.617+0000 7ffb8ca24740 10 client.17832 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:13.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.617+0000 7ffb8ca24740 1 -- 172.21.15.52:0/2475937305 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556136268b50 con 0x5561362507a0 2024-04-02T14:01:13.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.617+0000 7ffb62ffd640 1 -- 172.21.15.52:0/2475937305 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb6c015070 con 0x5561362507a0 2024-04-02T14:01:13.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.617+0000 7ffb62ffd640 1 -- 172.21.15.52:0/2475937305 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ffb740c6990 con 0x5561362507a0 2024-04-02T14:01:13.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.617+0000 7ffb8ca24740 10 client.17832 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:13.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.617+0000 7ffb8ca24740 10 client.17832 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T14:01:13.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.617+0000 7ffb8ca24740 20 client.17832 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:13.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.617+0000 7ffb8ca24740 1 --2- 172.21.15.52:0/2475937305 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55613626a000 0x55613626c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.617+0000 7ffb8ca24740 4 client.17832 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:01:13.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.617+0000 7ffb8ca24740 1 -- 172.21.15.52:0/2475937305 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561361c1cf0 con 0x55613626a000 2024-04-02T14:01:13.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.618+0000 7ffb82ffd640 1 --2- 172.21.15.52:0/2475937305 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55613626a000 0x55613626c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.618+0000 7ffb62ffd640 10 client.17832 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T14:01:13.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.618+0000 7ffb82ffd640 1 --2- 172.21.15.52:0/2475937305 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55613626a000 0x55613626c430 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4352af6545a8add9 server_cookie=9267b572e3d6adf6 in_seq=0 out_seq=0 2024-04-02T14:01:13.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.619+0000 7ffb62ffd640 1 -- 172.21.15.52:0/2475937305 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5561361c1cf0 con 0x55613626a000 2024-04-02T14:01:13.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.619+0000 7ffb62ffd640 10 client.17832 handle_command_reply: tid=0 2024-04-02T14:01:13.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.619+0000 7ffb8ca24740 1 client.17832 shutdown 2024-04-02T14:01:13.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.619+0000 7ffb8ca24740 1 -- 172.21.15.52:0/2475937305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561362507a0 msgr2=0x556136259740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.619+0000 7ffb8ca24740 1 --2- 172.21.15.52:0/2475937305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561362507a0 0x556136259740 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7ffb740efe50 tx=0x7ffb74009260 comp rx=0 tx=0).stop 2024-04-02T14:01:13.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.619+0000 7ffb8ca24740 1 -- 172.21.15.52:0/2475937305 shutdown_connections 2024-04-02T14:01:13.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.619+0000 7ffb8ca24740 1 --2- 172.21.15.52:0/2475937305 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556136259ca0 0x5561362571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.619+0000 7ffb8ca24740 1 --2- 172.21.15.52:0/2475937305 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55613626a000 0x55613626c430 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.619+0000 7ffb8ca24740 1 --2- 172.21.15.52:0/2475937305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561362510c0 0x556136256ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.619+0000 7ffb8ca24740 1 --2- 172.21.15.52:0/2475937305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561362507a0 0x556136259740 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.619+0000 7ffb62ffd640 0 client.17832 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T14:01:13.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.619+0000 7ffb8ca24740 1 -- 172.21.15.52:0/2475937305 >> 172.21.15.52:0/2475937305 conn(0x55613624aed0 msgr2=0x55613624cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.619+0000 7ffb8ca24740 1 -- 172.21.15.52:0/2475937305 shutdown_connections 2024-04-02T14:01:13.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.619+0000 7ffb8ca24740 1 -- 172.21.15.52:0/2475937305 wait complete. 2024-04-02T14:01:13.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.619+0000 7ffb8ca24740 20 client.17832 trim_cache size 0 max 16384 2024-04-02T14:01:13.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.632+0000 7ffb8ca24740 1 Processor -- start 2024-04-02T14:01:13.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.633+0000 7ffb8ca24740 1 -- start start 2024-04-02T14:01:13.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.633+0000 7ffb8ca24740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561362507a0 0x556136265980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.633+0000 7ffb8ca24740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561362510c0 0x5561362514a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.633+0000 7ffb8ca24740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556136259ca0 0x556136259200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.633+0000 7ffb8ca24740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffb841aef50 con 0x556136259ca0 2024-04-02T14:01:13.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.633+0000 7ffb8ca24740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffb841b0a90 con 0x5561362510c0 2024-04-02T14:01:13.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.633+0000 7ffb8ca24740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffb841af620 con 0x5561362507a0 2024-04-02T14:01:13.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.633+0000 7ffb83fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556136259ca0 0x556136259200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.633+0000 7ffb83fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556136259ca0 0x556136259200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56218/0 (socket says 172.21.15.52:56218) 2024-04-02T14:01:13.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.633+0000 7ffb83fff640 1 -- 172.21.15.52:0/2221324190 learned_addr learned my addr 172.21.15.52:0/2221324190 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:13.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.633+0000 7ffb837fe640 1 --2- 172.21.15.52:0/2221324190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561362507a0 0x556136265980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.633+0000 7ffb83fff640 1 -- 172.21.15.52:0/2221324190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561362507a0 msgr2=0x556136265980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.633+0000 7ffb83fff640 1 --2- 172.21.15.52:0/2221324190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561362507a0 0x556136265980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.633+0000 7ffb83fff640 1 -- 172.21.15.52:0/2221324190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561362510c0 msgr2=0x5561362514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.633+0000 7ffb83fff640 1 --2- 172.21.15.52:0/2221324190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561362510c0 0x5561362514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.633+0000 7ffb83fff640 1 -- 172.21.15.52:0/2221324190 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556136250ef0 con 0x556136259ca0 2024-04-02T14:01:13.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.633+0000 7ffb837fe640 1 --2- 172.21.15.52:0/2221324190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561362507a0 0x556136265980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:13.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.633+0000 7ffb83fff640 1 --2- 172.21.15.52:0/2221324190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556136259ca0 0x556136259200 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00ead0 tx=0x7ffb7c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7983347d43f2244 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.633+0000 7ffb797fa640 1 -- 172.21.15.52:0/2221324190 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c020c80 con 0x556136259ca0 2024-04-02T14:01:13.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.633+0000 7ffb797fa640 1 -- 172.21.15.52:0/2221324190 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb7c020e20 con 0x556136259ca0 2024-04-02T14:01:13.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.633+0000 7ffb797fa640 1 -- 172.21.15.52:0/2221324190 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c00f880 con 0x556136259ca0 2024-04-02T14:01:13.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.634+0000 7ffb8ca24740 1 -- 172.21.15.52:0/2221324190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556136259ca0 msgr2=0x556136259200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.634+0000 7ffb8ca24740 1 --2- 172.21.15.52:0/2221324190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556136259ca0 0x556136259200 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00ead0 tx=0x7ffb7c01d000 comp rx=0 tx=0).stop 2024-04-02T14:01:13.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.634+0000 7ffb8ca24740 1 -- 172.21.15.52:0/2221324190 shutdown_connections 2024-04-02T14:01:13.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.634+0000 7ffb8ca24740 1 --2- 172.21.15.52:0/2221324190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556136259ca0 0x556136259200 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.634+0000 7ffb8ca24740 1 --2- 172.21.15.52:0/2221324190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561362510c0 0x5561362514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.634+0000 7ffb8ca24740 1 --2- 172.21.15.52:0/2221324190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561362507a0 0x556136265980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.634+0000 7ffb8ca24740 1 -- 172.21.15.52:0/2221324190 >> 172.21.15.52:0/2221324190 conn(0x55613624aed0 msgr2=0x5561362553a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.634+0000 7ffb8ca24740 1 -- 172.21.15.52:0/2221324190 shutdown_connections 2024-04-02T14:01:13.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.634+0000 7ffb8ca24740 1 -- 172.21.15.52:0/2221324190 wait complete. 2024-04-02T14:01:13.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.634+0000 7ffb8ca24740 1 -- start start 2024-04-02T14:01:13.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.634+0000 7ffb8ca24740 1 Processor -- start 2024-04-02T14:01:13.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.635+0000 7ffb8ca24740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561362507a0 0x556136261f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.635+0000 7ffb8ca24740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561362510c0 0x55613625d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.635+0000 7ffb8ca24740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556136259ca0 0x55613625d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.635+0000 7ffb8ca24740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffb841b03c0 con 0x556136259ca0 2024-04-02T14:01:13.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.635+0000 7ffb8ca24740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffb841afcf0 con 0x5561362510c0 2024-04-02T14:01:13.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.635+0000 7ffb8ca24740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffb841acf50 con 0x5561362507a0 2024-04-02T14:01:13.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.635+0000 7ffb8ca24740 10 client.0 _ll_register_callbacks cb 0x7ffb74009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:13.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.635+0000 7ffb83fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556136259ca0 0x55613625d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.635+0000 7ffb83fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556136259ca0 0x55613625d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56228/0 (socket says 172.21.15.52:56228) 2024-04-02T14:01:13.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.635+0000 7ffb83fff640 1 -- 172.21.15.52:0/4188648338 learned_addr learned my addr 172.21.15.52:0/4188648338 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:13.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.635+0000 7ffb837fe640 1 --2- 172.21.15.52:0/4188648338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561362507a0 0x556136261f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.635+0000 7ffb82ffd640 1 --2- 172.21.15.52:0/4188648338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561362510c0 0x55613625d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.635+0000 7ffb83fff640 1 -- 172.21.15.52:0/4188648338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561362507a0 msgr2=0x556136261f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.635+0000 7ffb83fff640 1 --2- 172.21.15.52:0/4188648338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561362507a0 0x556136261f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.635+0000 7ffb83fff640 1 -- 172.21.15.52:0/4188648338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561362510c0 msgr2=0x55613625d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.635+0000 7ffb83fff640 1 --2- 172.21.15.52:0/4188648338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561362510c0 0x55613625d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.635+0000 7ffb83fff640 1 -- 172.21.15.52:0/4188648338 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55613625def0 con 0x556136259ca0 2024-04-02T14:01:13.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.635+0000 7ffb82ffd640 1 --2- 172.21.15.52:0/4188648338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561362510c0 0x55613625d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:13.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.635+0000 7ffb837fe640 1 --2- 172.21.15.52:0/4188648338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561362507a0 0x556136261f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:13.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.635+0000 7ffb83fff640 1 --2- 172.21.15.52:0/4188648338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556136259ca0 0x55613625d9b0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009aa0 tx=0x7ffb7c020d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a50da2c7303f9904 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.635+0000 7ffb62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:13.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.635+0000 7ffb62ffd640 1 -- 172.21.15.52:0/4188648338 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c004d00 con 0x556136259ca0 2024-04-02T14:01:13.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.635+0000 7ffb62ffd640 1 -- 172.21.15.52:0/4188648338 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9526+0+0 (secure 0 0 0) 0x7ffb7c005040 con 0x556136259ca0 2024-04-02T14:01:13.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.635+0000 7ffb8ca24740 1 -- 172.21.15.52:0/4188648338 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb7c020e20 con 0x556136259ca0 2024-04-02T14:01:13.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.636+0000 7ffb62ffd640 1 -- 172.21.15.52:0/4188648338 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffb7c005260 con 0x556136259ca0 2024-04-02T14:01:13.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.636+0000 7ffb8ca24740 10 client.17838 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:13.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.636+0000 7ffb8ca24740 1 -- 172.21.15.52:0/4188648338 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5561362d01b0 con 0x556136259ca0 2024-04-02T14:01:13.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.636+0000 7ffb62ffd640 1 -- 172.21.15.52:0/4188648338 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb7c020e20 con 0x556136259ca0 2024-04-02T14:01:13.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.636+0000 7ffb62ffd640 1 -- 172.21.15.52:0/4188648338 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7ffb7c0299b0 con 0x556136259ca0 2024-04-02T14:01:13.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.636+0000 7ffb8ca24740 10 client.17838 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:13.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.636+0000 7ffb8ca24740 10 client.17838 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T14:01:13.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.636+0000 7ffb8ca24740 20 client.17838 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:13.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.636+0000 7ffb8ca24740 1 --2- 172.21.15.52:0/4188648338 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5561362d0820 0x5561362d2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.636+0000 7ffb8ca24740 4 client.17838 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:01:13.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.636+0000 7ffb8ca24740 1 -- 172.21.15.52:0/4188648338 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5561361bc040 con 0x5561362d0820 2024-04-02T14:01:13.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.636+0000 7ffb837fe640 1 --2- 172.21.15.52:0/4188648338 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5561362d0820 0x5561362d2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.637+0000 7ffb62ffd640 10 client.17838 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T14:01:13.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.637+0000 7ffb837fe640 1 --2- 172.21.15.52:0/4188648338 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5561362d0820 0x5561362d2c40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e331cab42721aa5 server_cookie=3f1051ab14832210 in_seq=0 out_seq=0 2024-04-02T14:01:13.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.637+0000 7ffb62ffd640 1 -- 172.21.15.52:0/4188648338 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5561361bc040 con 0x5561362d0820 2024-04-02T14:01:13.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.637+0000 7ffb62ffd640 10 client.17838 handle_command_reply: tid=0 2024-04-02T14:01:13.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.637+0000 7ffb8ca24740 1 client.17838 shutdown 2024-04-02T14:01:13.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.637+0000 7ffb8ca24740 1 -- 172.21.15.52:0/4188648338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556136259ca0 msgr2=0x55613625d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.637+0000 7ffb8ca24740 1 --2- 172.21.15.52:0/4188648338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556136259ca0 0x55613625d9b0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009aa0 tx=0x7ffb7c020d30 comp rx=0 tx=0).stop 2024-04-02T14:01:13.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.637+0000 7ffb8ca24740 1 -- 172.21.15.52:0/4188648338 shutdown_connections 2024-04-02T14:01:13.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.637+0000 7ffb8ca24740 1 --2- 172.21.15.52:0/4188648338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556136259ca0 0x55613625d9b0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.637+0000 7ffb8ca24740 1 --2- 172.21.15.52:0/4188648338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561362510c0 0x55613625d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.637+0000 7ffb8ca24740 1 --2- 172.21.15.52:0/4188648338 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5561362d0820 0x5561362d2c40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.637+0000 7ffb8ca24740 1 --2- 172.21.15.52:0/4188648338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561362507a0 0x556136261f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.637+0000 7ffb62ffd640 0 client.17838 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T14:01:13.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.637+0000 7ffb8ca24740 1 -- 172.21.15.52:0/4188648338 >> 172.21.15.52:0/4188648338 conn(0x55613624aed0 msgr2=0x5561362496a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.637+0000 7ffb8ca24740 1 -- 172.21.15.52:0/4188648338 shutdown_connections 2024-04-02T14:01:13.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.637+0000 7ffb8ca24740 1 -- 172.21.15.52:0/4188648338 wait complete. 2024-04-02T14:01:13.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.637+0000 7ffb8ca24740 20 client.17838 trim_cache size 0 max 16384 2024-04-02T14:01:13.652 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:01:13.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.638+0000 7ffb89d60640 1 -- 172.21.15.52:0/1895519534 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ffb700a3d20 msgr2=0x7ffb700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.638+0000 7ffb89d60640 1 --2- 172.21.15.52:0/1895519534 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ffb700a3d20 0x7ffb700a61e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c005fd0 tx=0x7ffb7c012ad0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.638+0000 7ffb89d60640 1 -- 172.21.15.52:0/1895519534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb84101a90 msgr2=0x7ffb84198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.638+0000 7ffb89d60640 1 --2- 172.21.15.52:0/1895519534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb84101a90 0x7ffb84198390 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009230 tx=0x7ffb7402cc70 comp rx=0 tx=0).stop 2024-04-02T14:01:13.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.638+0000 7ffb89d60640 1 -- 172.21.15.52:0/1895519534 shutdown_connections 2024-04-02T14:01:13.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.638+0000 7ffb89d60640 1 --2- 172.21.15.52:0/1895519534 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ffb700a3d20 0x7ffb700a61e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.638+0000 7ffb89d60640 1 --2- 172.21.15.52:0/1895519534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb84107aa0 0x7ffb8419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.638+0000 7ffb89d60640 1 --2- 172.21.15.52:0/1895519534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb84102450 0x7ffb841988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.638+0000 7ffb89d60640 1 --2- 172.21.15.52:0/1895519534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb84101a90 0x7ffb84198390 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.638+0000 7ffb89d60640 1 -- 172.21.15.52:0/1895519534 >> 172.21.15.52:0/1895519534 conn(0x7ffb84078f00 msgr2=0x7ffb84104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.638+0000 7ffb89d60640 1 -- 172.21.15.52:0/1895519534 shutdown_connections 2024-04-02T14:01:13.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.638+0000 7ffb89d60640 1 -- 172.21.15.52:0/1895519534 wait complete. 2024-04-02T14:01:13.655 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:13 smithi052 ceph-mon[32104]: pgmap v237: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:01:13.655 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:13 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/288998813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:01:13.656 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T14:01:13.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.757+0000 7f3ff7293640 1 -- 172.21.15.52:0/4144676917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ff0102d40 msgr2=0x7f3ff0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.757+0000 7f3ff7293640 1 --2- 172.21.15.52:0/4144676917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ff0102d40 0x7f3ff0103120 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009900 tx=0x7f3fe802bcd0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.757+0000 7f3ff7293640 1 -- 172.21.15.52:0/4144676917 shutdown_connections 2024-04-02T14:01:13.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.757+0000 7f3ff7293640 1 --2- 172.21.15.52:0/4144676917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff010c110 0x7f3ff010e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.757+0000 7f3ff7293640 1 --2- 172.21.15.52:0/4144676917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ff0103660 0x7f3ff010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.757+0000 7f3ff7293640 1 --2- 172.21.15.52:0/4144676917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ff0102d40 0x7f3ff0103120 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.757+0000 7f3ff7293640 1 -- 172.21.15.52:0/4144676917 >> 172.21.15.52:0/4144676917 conn(0x7f3ff0076d80 msgr2=0x7f3ff0077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.757+0000 7f3ff7293640 1 -- 172.21.15.52:0/4144676917 shutdown_connections 2024-04-02T14:01:13.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.757+0000 7f3ff7293640 1 -- 172.21.15.52:0/4144676917 wait complete. 2024-04-02T14:01:13.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.758+0000 7f3ff7293640 1 Processor -- start 2024-04-02T14:01:13.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.758+0000 7f3ff7293640 1 -- start start 2024-04-02T14:01:13.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.758+0000 7f3ff7293640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ff0102d40 0x7f3ff0100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.758+0000 7f3ff7293640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ff0103660 0x7f3ff0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.758+0000 7f3ff7293640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff010c110 0x7f3ff0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.758+0000 7f3ff7293640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff0111ea0 con 0x7f3ff0103660 2024-04-02T14:01:13.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.758+0000 7f3ff7293640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff0111d20 con 0x7f3ff0102d40 2024-04-02T14:01:13.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.758+0000 7f3ff7293640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff0112020 con 0x7f3ff010c110 2024-04-02T14:01:13.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.758+0000 7f3ff482e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ff0103660 0x7f3ff0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.758+0000 7f3ff482e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ff0103660 0x7f3ff0100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56256/0 (socket says 172.21.15.52:56256) 2024-04-02T14:01:13.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.758+0000 7f3ff482e640 1 -- 172.21.15.52:0/2093506083 learned_addr learned my addr 172.21.15.52:0/2093506083 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:13.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.758+0000 7f3ff502f640 1 --2- 172.21.15.52:0/2093506083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ff0102d40 0x7f3ff0100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.759+0000 7f3ff482e640 1 -- 172.21.15.52:0/2093506083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff010c110 msgr2=0x7f3ff0101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.759+0000 7f3ff5830640 1 --2- 172.21.15.52:0/2093506083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff010c110 0x7f3ff0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.759+0000 7f3ff482e640 1 --2- 172.21.15.52:0/2093506083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff010c110 0x7f3ff0101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.759+0000 7f3ff482e640 1 -- 172.21.15.52:0/2093506083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ff0102d40 msgr2=0x7f3ff0100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.759+0000 7f3ff482e640 1 --2- 172.21.15.52:0/2093506083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ff0102d40 0x7f3ff0100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.759+0000 7f3ff482e640 1 -- 172.21.15.52:0/2093506083 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff01a6880 con 0x7f3ff0103660 2024-04-02T14:01:13.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.759+0000 7f3ff502f640 1 --2- 172.21.15.52:0/2093506083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ff0102d40 0x7f3ff0100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:13.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.759+0000 7f3ff5830640 1 --2- 172.21.15.52:0/2093506083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff010c110 0x7f3ff0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:13.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.759+0000 7f3ff482e640 1 --2- 172.21.15.52:0/2093506083 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ff0103660 0x7f3ff0100ed0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400e9f0 tx=0x7f3fe400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.759+0000 7f3fee7fc640 1 -- 172.21.15.52:0/2093506083 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe400cd80 con 0x7f3ff0103660 2024-04-02T14:01:13.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.759+0000 7f3fee7fc640 1 -- 172.21.15.52:0/2093506083 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fe4010910 con 0x7f3ff0103660 2024-04-02T14:01:13.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.759+0000 7f3ff7293640 1 -- 172.21.15.52:0/2093506083 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff01a6b70 con 0x7f3ff0103660 2024-04-02T14:01:13.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.759+0000 7f3fee7fc640 1 -- 172.21.15.52:0/2093506083 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4018990 con 0x7f3ff0103660 2024-04-02T14:01:13.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.759+0000 7f3ff7293640 1 -- 172.21.15.52:0/2093506083 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff01a7110 con 0x7f3ff0103660 2024-04-02T14:01:13.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.761+0000 7f3fee7fc640 1 -- 172.21.15.52:0/2093506083 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3fe4010ac0 con 0x7f3ff0103660 2024-04-02T14:01:13.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.761+0000 7f3ff9e81740 1 Processor -- start 2024-04-02T14:01:13.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.761+0000 7f3ff9e81740 1 -- start start 2024-04-02T14:01:13.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.762+0000 7f3fee7fc640 1 --2- 172.21.15.52:0/2093506083 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3fd80a3da0 0x7f3fd80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.762+0000 7f3ff9e81740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e627187a0 0x563e62718b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.762+0000 7f3ff9e81740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e627190c0 0x563e62721760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.762+0000 7f3ff9e81740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e62721ca0 0x563e62724090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.762+0000 7f3ff9e81740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563e6267c7b0 con 0x563e62721ca0 2024-04-02T14:01:13.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.762+0000 7f3ff9e81740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563e6261b570 con 0x563e627187a0 2024-04-02T14:01:13.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.762+0000 7f3ff9e81740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563e62678760 con 0x563e627190c0 2024-04-02T14:01:13.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.762+0000 7f3fee7fc640 1 -- 172.21.15.52:0/2093506083 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9526+0+0 (secure 0 0 0) 0x7f3fe40d0fd0 con 0x7f3ff0103660 2024-04-02T14:01:13.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.762+0000 7f3ff5830640 1 --2- 172.21.15.52:0/2093506083 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3fd80a3da0 0x7f3fd80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.762+0000 7f3ff482e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e627190c0 0x563e62721760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.762+0000 7f3ff5830640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e62721ca0 0x563e62724090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.762+0000 7f3ff482e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e627190c0 0x563e62721760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.52:39568/0 (socket says 172.21.15.52:39568) 2024-04-02T14:01:13.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.762+0000 7f3ff482e640 1 -- 172.21.15.52:0/1046385849 learned_addr learned my addr 172.21.15.52:0/1046385849 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:13.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.762+0000 7f3ff502f640 1 --2- 172.21.15.52:0/1046385849 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e627187a0 0x563e62718b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.762+0000 7f3ff482e640 1 -- 172.21.15.52:0/1046385849 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e627187a0 msgr2=0x563e62718b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.762+0000 7f3ff482e640 1 --2- 172.21.15.52:0/1046385849 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e627187a0 0x563e62718b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.762+0000 7f3ff482e640 1 -- 172.21.15.52:0/1046385849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e62721ca0 msgr2=0x563e62724090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.762+0000 7f3ff482e640 1 --2- 172.21.15.52:0/1046385849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e62721ca0 0x563e62724090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.762+0000 7f3ff482e640 1 -- 172.21.15.52:0/1046385849 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e623903f0 con 0x563e627190c0 2024-04-02T14:01:13.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.762+0000 7f3ff5830640 1 --2- 172.21.15.52:0/2093506083 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3fd80a3da0 0x7f3fd80a6260 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0009810 tx=0x7f3fe0009e20 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.763+0000 7f3ff482e640 1 --2- 172.21.15.52:0/1046385849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e627190c0 0x563e62721760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4099550 tx=0x7f3fe40b6300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=303780d4d6f8f154 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.763+0000 7f3fd77fe640 1 -- 172.21.15.52:0/1046385849 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe409cb80 con 0x563e627190c0 2024-04-02T14:01:13.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.763+0000 7f3fd77fe640 1 -- 172.21.15.52:0/1046385849 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fe40a0f30 con 0x563e627190c0 2024-04-02T14:01:13.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.763+0000 7f3fd77fe640 1 -- 172.21.15.52:0/1046385849 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe40bd840 con 0x563e627190c0 2024-04-02T14:01:13.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.763+0000 7f3ff9e81740 1 -- 172.21.15.52:0/1046385849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e627190c0 msgr2=0x563e62721760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.763+0000 7f3ff9e81740 1 --2- 172.21.15.52:0/1046385849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e627190c0 0x563e62721760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4099550 tx=0x7f3fe40b6300 comp rx=0 tx=0).stop 2024-04-02T14:01:13.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.763+0000 7f3ff9e81740 1 -- 172.21.15.52:0/1046385849 shutdown_connections 2024-04-02T14:01:13.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.763+0000 7f3ff9e81740 1 --2- 172.21.15.52:0/1046385849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e62721ca0 0x563e62724090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.763+0000 7f3ff9e81740 1 --2- 172.21.15.52:0/1046385849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e627190c0 0x563e62721760 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.763+0000 7f3ff9e81740 1 --2- 172.21.15.52:0/1046385849 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e627187a0 0x563e62718b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.763+0000 7f3ff9e81740 1 -- 172.21.15.52:0/1046385849 >> 172.21.15.52:0/1046385849 conn(0x563e62712ed0 msgr2=0x563e627132b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.763+0000 7f3ff9e81740 1 -- 172.21.15.52:0/1046385849 shutdown_connections 2024-04-02T14:01:13.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.764+0000 7f3ff9e81740 1 -- 172.21.15.52:0/1046385849 wait complete. 2024-04-02T14:01:13.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.764+0000 7f3ff9e81740 1 -- start start 2024-04-02T14:01:13.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.764+0000 7f3ff9e81740 1 Processor -- start 2024-04-02T14:01:13.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.764+0000 7f3ff9e81740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e627187a0 0x563e627216d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.764+0000 7f3ff9e81740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e627190c0 0x563e6271ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.764+0000 7f3ff9e81740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e62721ca0 0x563e6271f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.764+0000 7f3ff9e81740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563e62724a90 con 0x563e627190c0 2024-04-02T14:01:13.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.764+0000 7f3ff9e81740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563e62637e80 con 0x563e627187a0 2024-04-02T14:01:13.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.764+0000 7f3ff9e81740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563e62724c10 con 0x563e62721ca0 2024-04-02T14:01:13.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.764+0000 7f3ff9e81740 10 client.0 _ll_register_callbacks cb 0x563e62673aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:13.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.764+0000 7f3ff482e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e627190c0 0x563e6271ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.764+0000 7f3ff482e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e627190c0 0x563e6271ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56266/0 (socket says 172.21.15.52:56266) 2024-04-02T14:01:13.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.764+0000 7f3ff482e640 1 -- 172.21.15.52:0/38472534 learned_addr learned my addr 172.21.15.52:0/38472534 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:13.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.764+0000 7f3ff5830640 1 --2- 172.21.15.52:0/38472534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e62721ca0 0x563e6271f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.765+0000 7f3ff502f640 1 --2- 172.21.15.52:0/38472534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e627187a0 0x563e627216d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.765+0000 7f3ff482e640 1 -- 172.21.15.52:0/38472534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e62721ca0 msgr2=0x563e6271f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.765+0000 7f3ff482e640 1 --2- 172.21.15.52:0/38472534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e62721ca0 0x563e6271f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.765+0000 7f3ff482e640 1 -- 172.21.15.52:0/38472534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e627187a0 msgr2=0x563e627216d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.765+0000 7f3ff482e640 1 --2- 172.21.15.52:0/38472534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e627187a0 0x563e627216d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.765+0000 7f3ff482e640 1 -- 172.21.15.52:0/38472534 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e6245a7d0 con 0x563e627190c0 2024-04-02T14:01:13.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.765+0000 7f3ff5830640 1 --2- 172.21.15.52:0/38472534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e62721ca0 0x563e6271f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:13.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.765+0000 7f3ff482e640 1 --2- 172.21.15.52:0/38472534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e627190c0 0x563e6271ea60 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f3fe409cb10 tx=0x7f3fe4098f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a59c616ae5c3a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.765+0000 7f3fd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:13.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.765+0000 7f3fd4ff9640 1 -- 172.21.15.52:0/38472534 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe40bb070 con 0x563e627190c0 2024-04-02T14:01:13.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.765+0000 7f3fd4ff9640 1 -- 172.21.15.52:0/38472534 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9526+0+0 (secure 0 0 0) 0x7f3fe40bd7a0 con 0x563e627190c0 2024-04-02T14:01:13.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.765+0000 7f3ff9e81740 1 -- 172.21.15.52:0/38472534 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fe40a0f30 con 0x563e627190c0 2024-04-02T14:01:13.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.765+0000 7f3fd4ff9640 1 -- 172.21.15.52:0/38472534 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3fe40a57f0 con 0x563e627190c0 2024-04-02T14:01:13.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.765+0000 7f3ff9e81740 10 client.17850 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:13.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.765+0000 7f3ff9e81740 1 -- 172.21.15.52:0/38472534 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563e62730ab0 con 0x563e627190c0 2024-04-02T14:01:13.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.765+0000 7f3fd4ff9640 1 -- 172.21.15.52:0/38472534 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fe40a0f30 con 0x563e627190c0 2024-04-02T14:01:13.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.765+0000 7f3fd4ff9640 1 -- 172.21.15.52:0/38472534 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3fe409bfd0 con 0x563e627190c0 2024-04-02T14:01:13.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.766+0000 7f3ff9e81740 10 client.17850 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:13.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.766+0000 7f3ff9e81740 10 client.17850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:01:13.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.766+0000 7f3ff9e81740 20 client.17850 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:13.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.766+0000 7f3ff9e81740 1 --2- 172.21.15.52:0/38472534 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x563e62731e00 0x563e627341f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.766+0000 7f3ff9e81740 4 client.17850 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:01:13.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.766+0000 7f3ff9e81740 1 -- 172.21.15.52:0/38472534 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e62689cf0 con 0x563e62731e00 2024-04-02T14:01:13.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.766+0000 7f3ff502f640 1 --2- 172.21.15.52:0/38472534 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x563e62731e00 0x563e627341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.766+0000 7f3fd4ff9640 10 client.17850 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:13.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.766+0000 7f3ff502f640 1 --2- 172.21.15.52:0/38472534 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x563e62731e00 0x563e627341f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1644a88c1218b4f6 server_cookie=657ae32f37c868ec in_seq=0 out_seq=0 2024-04-02T14:01:13.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.767+0000 7f3fd4ff9640 1 -- 172.21.15.52:0/38472534 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e62689cf0 con 0x563e62731e00 2024-04-02T14:01:13.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.767+0000 7f3fd4ff9640 10 client.17850 handle_command_reply: tid=0 2024-04-02T14:01:13.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.767+0000 7f3ff9e81740 1 client.17850 shutdown 2024-04-02T14:01:13.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.767+0000 7f3ff9e81740 1 -- 172.21.15.52:0/38472534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e627190c0 msgr2=0x563e6271ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.767+0000 7f3ff9e81740 1 --2- 172.21.15.52:0/38472534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e627190c0 0x563e6271ea60 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f3fe409cb10 tx=0x7f3fe4098f50 comp rx=0 tx=0).stop 2024-04-02T14:01:13.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.767+0000 7f3ff9e81740 1 -- 172.21.15.52:0/38472534 shutdown_connections 2024-04-02T14:01:13.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.768+0000 7f3ff9e81740 1 --2- 172.21.15.52:0/38472534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e62721ca0 0x563e6271f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.768+0000 7f3ff9e81740 1 --2- 172.21.15.52:0/38472534 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x563e62731e00 0x563e627341f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.768+0000 7f3ff9e81740 1 --2- 172.21.15.52:0/38472534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e627190c0 0x563e6271ea60 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.768+0000 7f3fd4ff9640 0 client.17850 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:13.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.768+0000 7f3ff9e81740 1 --2- 172.21.15.52:0/38472534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e627187a0 0x563e627216d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.768+0000 7f3ff9e81740 1 -- 172.21.15.52:0/38472534 >> 172.21.15.52:0/38472534 conn(0x563e62712ed0 msgr2=0x563e62714c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.768+0000 7f3ff9e81740 1 -- 172.21.15.52:0/38472534 shutdown_connections 2024-04-02T14:01:13.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.768+0000 7f3ff9e81740 1 -- 172.21.15.52:0/38472534 wait complete. 2024-04-02T14:01:13.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.768+0000 7f3ff9e81740 20 client.17850 trim_cache size 0 max 16384 2024-04-02T14:01:13.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:13 smithi189 ceph-mon[36683]: pgmap v237: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:01:13.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:13 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/288998813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:01:13.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.781+0000 7f3ff9e81740 1 Processor -- start 2024-04-02T14:01:13.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.781+0000 7f3ff9e81740 1 -- start start 2024-04-02T14:01:13.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.781+0000 7f3ff9e81740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e62712ed0 0x563e6272d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.781+0000 7f3ff9e81740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e627187a0 0x563e627397e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.781+0000 7f3ff9e81740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e627190c0 0x563e62739d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.781+0000 7f3ff9e81740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01b11b0 con 0x563e627190c0 2024-04-02T14:01:13.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.781+0000 7f3ff9e81740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01b2cf0 con 0x563e62712ed0 2024-04-02T14:01:13.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.781+0000 7f3ff9e81740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01b1880 con 0x563e627187a0 2024-04-02T14:01:13.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.782+0000 7f3ff5830640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e627190c0 0x563e62739d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.782+0000 7f3ff5830640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e627190c0 0x563e62739d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56268/0 (socket says 172.21.15.52:56268) 2024-04-02T14:01:13.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.782+0000 7f3ff5830640 1 -- 172.21.15.52:0/1917007445 learned_addr learned my addr 172.21.15.52:0/1917007445 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:13.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.782+0000 7f3ff482e640 1 --2- 172.21.15.52:0/1917007445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e627187a0 0x563e627397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.782+0000 7f3ff502f640 1 --2- 172.21.15.52:0/1917007445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e62712ed0 0x563e6272d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.782+0000 7f3ff5830640 1 -- 172.21.15.52:0/1917007445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e627187a0 msgr2=0x563e627397e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.782+0000 7f3ff5830640 1 --2- 172.21.15.52:0/1917007445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e627187a0 0x563e627397e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.782+0000 7f3ff5830640 1 -- 172.21.15.52:0/1917007445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e62712ed0 msgr2=0x563e6272d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.782+0000 7f3ff5830640 1 --2- 172.21.15.52:0/1917007445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e62712ed0 0x563e6272d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.782+0000 7f3ff5830640 1 -- 172.21.15.52:0/1917007445 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e6271f510 con 0x563e627190c0 2024-04-02T14:01:13.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.782+0000 7f3ff5830640 1 --2- 172.21.15.52:0/1917007445 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e627190c0 0x563e62739d20 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00141e0 tx=0x7f3fe001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49996f614a5a2721 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.782+0000 7f3fd6ffd640 1 -- 172.21.15.52:0/1917007445 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0004030 con 0x563e627190c0 2024-04-02T14:01:13.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.782+0000 7f3fd6ffd640 1 -- 172.21.15.52:0/1917007445 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fe00046b0 con 0x563e627190c0 2024-04-02T14:01:13.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.782+0000 7f3fd6ffd640 1 -- 172.21.15.52:0/1917007445 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0021d30 con 0x563e627190c0 2024-04-02T14:01:13.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.783+0000 7f3ff9e81740 1 -- 172.21.15.52:0/1917007445 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e627190c0 msgr2=0x563e62739d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.783+0000 7f3ff9e81740 1 --2- 172.21.15.52:0/1917007445 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e627190c0 0x563e62739d20 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00141e0 tx=0x7f3fe001e000 comp rx=0 tx=0).stop 2024-04-02T14:01:13.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.783+0000 7f3ff9e81740 1 -- 172.21.15.52:0/1917007445 shutdown_connections 2024-04-02T14:01:13.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.783+0000 7f3ff9e81740 1 --2- 172.21.15.52:0/1917007445 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e627190c0 0x563e62739d20 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.783+0000 7f3ff9e81740 1 --2- 172.21.15.52:0/1917007445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e627187a0 0x563e627397e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.783+0000 7f3ff9e81740 1 --2- 172.21.15.52:0/1917007445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e62712ed0 0x563e6272d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.783+0000 7f3ff9e81740 1 -- 172.21.15.52:0/1917007445 >> 172.21.15.52:0/1917007445 conn(0x563e627243b0 msgr2=0x563e627132b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.783+0000 7f3ff9e81740 1 -- 172.21.15.52:0/1917007445 shutdown_connections 2024-04-02T14:01:13.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.783+0000 7f3ff9e81740 1 -- 172.21.15.52:0/1917007445 wait complete. 2024-04-02T14:01:13.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.783+0000 7f3ff9e81740 1 -- start start 2024-04-02T14:01:13.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.783+0000 7f3ff9e81740 1 Processor -- start 2024-04-02T14:01:13.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.783+0000 7f3ff9e81740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e62712ed0 0x563e62722450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.784+0000 7f3ff9e81740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e627187a0 0x563e62722990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.784+0000 7f3ff9e81740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e627190c0 0x563e62722ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.784+0000 7f3ff9e81740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01b2620 con 0x563e627190c0 2024-04-02T14:01:13.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.784+0000 7f3ff9e81740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01b1f50 con 0x563e62712ed0 2024-04-02T14:01:13.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.784+0000 7f3ff9e81740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01af1b0 con 0x563e627187a0 2024-04-02T14:01:13.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.784+0000 7f3ff9e81740 10 client.0 _ll_register_callbacks cb 0x7f3fe409c7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:13.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.784+0000 7f3ff5830640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e627190c0 0x563e62722ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.784+0000 7f3ff5830640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e627190c0 0x563e62722ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56272/0 (socket says 172.21.15.52:56272) 2024-04-02T14:01:13.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.784+0000 7f3ff5830640 1 -- 172.21.15.52:0/241526640 learned_addr learned my addr 172.21.15.52:0/241526640 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:13.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.784+0000 7f3ff482e640 1 --2- 172.21.15.52:0/241526640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e627187a0 0x563e62722990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.784+0000 7f3ff502f640 1 --2- 172.21.15.52:0/241526640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e62712ed0 0x563e62722450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.784+0000 7f3ff5830640 1 -- 172.21.15.52:0/241526640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e627187a0 msgr2=0x563e62722990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.784+0000 7f3ff5830640 1 --2- 172.21.15.52:0/241526640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e627187a0 0x563e62722990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.784+0000 7f3ff5830640 1 -- 172.21.15.52:0/241526640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e62712ed0 msgr2=0x563e62722450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:13.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.784+0000 7f3ff5830640 1 --2- 172.21.15.52:0/241526640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e62712ed0 0x563e62722450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.784+0000 7f3ff5830640 1 -- 172.21.15.52:0/241526640 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e62728f90 con 0x563e627190c0 2024-04-02T14:01:13.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.784+0000 7f3ff5830640 1 --2- 172.21.15.52:0/241526640 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e627190c0 0x563e62722ed0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0021870 tx=0x7f3fe0004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6928d4b04aa53efc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:13.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.784+0000 7f3fd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:13.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.784+0000 7f3fd4ff9640 1 -- 172.21.15.52:0/241526640 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe002f040 con 0x563e627190c0 2024-04-02T14:01:13.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.784+0000 7f3fd4ff9640 1 -- 172.21.15.52:0/241526640 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9526+0+0 (secure 0 0 0) 0x7f3fe0014210 con 0x563e627190c0 2024-04-02T14:01:13.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.784+0000 7f3ff9e81740 1 -- 172.21.15.52:0/241526640 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fe00046b0 con 0x563e627190c0 2024-04-02T14:01:13.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.784+0000 7f3fd4ff9640 1 -- 172.21.15.52:0/241526640 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3fe0040050 con 0x563e627190c0 2024-04-02T14:01:13.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.784+0000 7f3ff9e81740 10 client.17856 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:13.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.784+0000 7f3ff9e81740 1 -- 172.21.15.52:0/241526640 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563e6272a290 con 0x563e627190c0 2024-04-02T14:01:13.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.785+0000 7f3fd4ff9640 1 -- 172.21.15.52:0/241526640 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fe00046b0 con 0x563e627190c0 2024-04-02T14:01:13.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.785+0000 7f3fd4ff9640 1 -- 172.21.15.52:0/241526640 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3fe0032070 con 0x563e627190c0 2024-04-02T14:01:13.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.785+0000 7f3ff9e81740 10 client.17856 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:13.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.785+0000 7f3ff9e81740 10 client.17856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:01:13.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.785+0000 7f3ff9e81740 20 client.17856 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:13.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.785+0000 7f3ff9e81740 1 --2- 172.21.15.52:0/241526640 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x563e62721ca0 0x563e6279a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:13.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.785+0000 7f3ff9e81740 4 client.17856 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:01:13.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.785+0000 7f3ff9e81740 1 -- 172.21.15.52:0/241526640 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e62684040 con 0x563e62721ca0 2024-04-02T14:01:13.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.785+0000 7f3ff502f640 1 --2- 172.21.15.52:0/241526640 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x563e62721ca0 0x563e6279a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:13.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.785+0000 7f3fd4ff9640 10 client.17856 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:13.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.785+0000 7f3ff502f640 1 --2- 172.21.15.52:0/241526640 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x563e62721ca0 0x563e6279a090 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e39b41a5bd7c65f1 server_cookie=dae0200e753420b1 in_seq=0 out_seq=0 2024-04-02T14:01:13.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.786+0000 7f3fd4ff9640 1 -- 172.21.15.52:0/241526640 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e62684040 con 0x563e62721ca0 2024-04-02T14:01:13.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.786+0000 7f3fd4ff9640 10 client.17856 handle_command_reply: tid=0 2024-04-02T14:01:13.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.786+0000 7f3ff9e81740 1 client.17856 shutdown 2024-04-02T14:01:13.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.786+0000 7f3ff9e81740 1 -- 172.21.15.52:0/241526640 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e627190c0 msgr2=0x563e62722ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.786+0000 7f3ff9e81740 1 --2- 172.21.15.52:0/241526640 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e627190c0 0x563e62722ed0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0021870 tx=0x7f3fe0004860 comp rx=0 tx=0).stop 2024-04-02T14:01:13.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.786+0000 7f3ff9e81740 1 -- 172.21.15.52:0/241526640 shutdown_connections 2024-04-02T14:01:13.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.786+0000 7f3ff9e81740 1 --2- 172.21.15.52:0/241526640 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x563e62721ca0 0x563e6279a090 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.786+0000 7f3ff9e81740 1 --2- 172.21.15.52:0/241526640 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563e627190c0 0x563e62722ed0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.786+0000 7f3fd4ff9640 0 client.17856 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:13.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.786+0000 7f3ff9e81740 1 --2- 172.21.15.52:0/241526640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e627187a0 0x563e62722990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.786+0000 7f3ff9e81740 1 --2- 172.21.15.52:0/241526640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e62712ed0 0x563e62722450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.786+0000 7f3ff9e81740 1 -- 172.21.15.52:0/241526640 >> 172.21.15.52:0/241526640 conn(0x563e627243b0 msgr2=0x563e6271d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.786+0000 7f3ff9e81740 1 -- 172.21.15.52:0/241526640 shutdown_connections 2024-04-02T14:01:13.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.786+0000 7f3ff9e81740 1 -- 172.21.15.52:0/241526640 wait complete. 2024-04-02T14:01:13.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.786+0000 7f3ff9e81740 20 client.17856 trim_cache size 0 max 16384 2024-04-02T14:01:13.802 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:01:13.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.786+0000 7f3ff7293640 1 -- 172.21.15.52:0/2093506083 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3fd80a3da0 msgr2=0x7f3fd80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.786+0000 7f3ff7293640 1 --2- 172.21.15.52:0/2093506083 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3fd80a3da0 0x7f3fd80a6260 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0009810 tx=0x7f3fe0009e20 comp rx=0 tx=0).stop 2024-04-02T14:01:13.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.786+0000 7f3ff7293640 1 -- 172.21.15.52:0/2093506083 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ff0103660 msgr2=0x7f3ff0100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:13.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.786+0000 7f3ff7293640 1 --2- 172.21.15.52:0/2093506083 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ff0103660 0x7f3ff0100ed0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400e9f0 tx=0x7f3fe400eeb0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.786+0000 7f3ff7293640 1 -- 172.21.15.52:0/2093506083 shutdown_connections 2024-04-02T14:01:13.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.786+0000 7f3ff7293640 1 --2- 172.21.15.52:0/2093506083 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f3fd80a3da0 0x7f3fd80a6260 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.786+0000 7f3ff7293640 1 --2- 172.21.15.52:0/2093506083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff010c110 0x7f3ff0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.786+0000 7f3ff7293640 1 --2- 172.21.15.52:0/2093506083 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ff0103660 0x7f3ff0100ed0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.786+0000 7f3ff7293640 1 --2- 172.21.15.52:0/2093506083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ff0102d40 0x7f3ff0100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:13.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.786+0000 7f3ff7293640 1 -- 172.21.15.52:0/2093506083 >> 172.21.15.52:0/2093506083 conn(0x7f3ff0076d80 msgr2=0x7f3ff00fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:13.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.787+0000 7f3ff7293640 1 -- 172.21.15.52:0/2093506083 shutdown_connections 2024-04-02T14:01:13.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:13.787+0000 7f3ff7293640 1 -- 172.21.15.52:0/2093506083 wait complete. 2024-04-02T14:01:13.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:13 smithi088 ceph-mon[36528]: pgmap v237: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:01:13.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:13 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/288998813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:01:14.806 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T14:01:14.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:01:14.807 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T14:01:14.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.923+0000 7fc46ac3c640 1 -- 172.21.15.52:0/1766396963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc464068ce0 msgr2=0x7fc464069160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:14.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.923+0000 7fc46ac3c640 1 --2- 172.21.15.52:0/1766396963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc464068ce0 0x7fc464069160 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fc454009cd0 tx=0x7fc45402c0f0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.923+0000 7fc46ac3c640 1 -- 172.21.15.52:0/1766396963 shutdown_connections 2024-04-02T14:01:14.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.923+0000 7fc46ac3c640 1 --2- 172.21.15.52:0/1766396963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4640696a0 0x7fc46410e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.923+0000 7fc46ac3c640 1 --2- 172.21.15.52:0/1766396963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc464068ce0 0x7fc464069160 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.923+0000 7fc46ac3c640 1 --2- 172.21.15.52:0/1766396963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc464107a80 0x7fc464107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.923+0000 7fc46ac3c640 1 -- 172.21.15.52:0/1766396963 >> 172.21.15.52:0/1766396963 conn(0x7fc464077b00 msgr2=0x7fc464077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:14.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.923+0000 7fc46ac3c640 1 -- 172.21.15.52:0/1766396963 shutdown_connections 2024-04-02T14:01:14.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.923+0000 7fc46ac3c640 1 -- 172.21.15.52:0/1766396963 wait complete. 2024-04-02T14:01:14.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.923+0000 7fc46ac3c640 1 Processor -- start 2024-04-02T14:01:14.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.923+0000 7fc46ac3c640 1 -- start start 2024-04-02T14:01:14.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.923+0000 7fc46ac3c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc464068ce0 0x7fc4641983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:14.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.923+0000 7fc46ac3c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4640696a0 0x7fc464198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:14.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.923+0000 7fc46ac3c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc464107a80 0x7fc46419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:14.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.923+0000 7fc46ac3c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc464111e90 con 0x7fc464068ce0 2024-04-02T14:01:14.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.923+0000 7fc46ac3c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc464111d10 con 0x7fc4640696a0 2024-04-02T14:01:14.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.924+0000 7fc46ac3c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc464112010 con 0x7fc464107a80 2024-04-02T14:01:14.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.924+0000 7fc4689d8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc464068ce0 0x7fc4641983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:14.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.924+0000 7fc4689d8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc464068ce0 0x7fc4641983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56280/0 (socket says 172.21.15.52:56280) 2024-04-02T14:01:14.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.924+0000 7fc4689d8640 1 -- 172.21.15.52:0/1757310037 learned_addr learned my addr 172.21.15.52:0/1757310037 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:14.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.924+0000 7fc4691d9640 1 --2- 172.21.15.52:0/1757310037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc464107a80 0x7fc46419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:14.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.924+0000 7fc463fff640 1 --2- 172.21.15.52:0/1757310037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4640696a0 0x7fc464198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:14.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.924+0000 7fc4689d8640 1 -- 172.21.15.52:0/1757310037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc464107a80 msgr2=0x7fc46419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:14.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.924+0000 7fc4689d8640 1 --2- 172.21.15.52:0/1757310037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc464107a80 0x7fc46419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.924+0000 7fc4689d8640 1 -- 172.21.15.52:0/1757310037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4640696a0 msgr2=0x7fc464198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:14.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.924+0000 7fc4689d8640 1 --2- 172.21.15.52:0/1757310037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4640696a0 0x7fc464198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.924+0000 7fc4689d8640 1 -- 172.21.15.52:0/1757310037 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46419d3e0 con 0x7fc464068ce0 2024-04-02T14:01:14.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.924+0000 7fc4689d8640 1 --2- 172.21.15.52:0/1757310037 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc464068ce0 0x7fc4641983f0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00d770 tx=0x7fc45c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:14.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.924+0000 7fc461ffb640 1 -- 172.21.15.52:0/1757310037 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c017070 con 0x7fc464068ce0 2024-04-02T14:01:14.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.924+0000 7fc461ffb640 1 -- 172.21.15.52:0/1757310037 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc45c005500 con 0x7fc464068ce0 2024-04-02T14:01:14.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.924+0000 7fc46ac3c640 1 -- 172.21.15.52:0/1757310037 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46419d6d0 con 0x7fc464068ce0 2024-04-02T14:01:14.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.924+0000 7fc461ffb640 1 -- 172.21.15.52:0/1757310037 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c004e00 con 0x7fc464068ce0 2024-04-02T14:01:14.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.925+0000 7fc46ac3c640 1 -- 172.21.15.52:0/1757310037 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4641a4f60 con 0x7fc464068ce0 2024-04-02T14:01:14.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.927+0000 7fc461ffb640 1 -- 172.21.15.52:0/1757310037 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc45c004fa0 con 0x7fc464068ce0 2024-04-02T14:01:14.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.927+0000 7fc46d826740 1 Processor -- start 2024-04-02T14:01:14.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.927+0000 7fc461ffb640 1 --2- 172.21.15.52:0/1757310037 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc4500a3d20 0x7fc4500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:14.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.927+0000 7fc46d826740 1 -- start start 2024-04-02T14:01:14.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.928+0000 7fc4691d9640 1 --2- 172.21.15.52:0/1757310037 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc4500a3d20 0x7fc4500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:14.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.928+0000 7fc461ffb640 1 -- 172.21.15.52:0/1757310037 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9526+0+0 (secure 0 0 0) 0x7fc45c0d0500 con 0x7fc464068ce0 2024-04-02T14:01:14.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.928+0000 7fc46d826740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56219947e2c0 0x56219947e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:14.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.928+0000 7fc46d826740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56219947ebe0 0x562199487280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:14.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.928+0000 7fc46d826740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621994877c0 0x562199489bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:14.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.928+0000 7fc46d826740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5621993e2770 con 0x56219947e2c0 2024-04-02T14:01:14.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.928+0000 7fc46d826740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56219938a960 con 0x56219947ebe0 2024-04-02T14:01:14.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.928+0000 7fc46d826740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5621993de8c0 con 0x5621994877c0 2024-04-02T14:01:14.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.928+0000 7fc4689d8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56219947e2c0 0x56219947e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:14.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.928+0000 7fc4689d8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56219947e2c0 0x56219947e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56296/0 (socket says 172.21.15.52:56296) 2024-04-02T14:01:14.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.928+0000 7fc4689d8640 1 -- 172.21.15.52:0/3488896616 learned_addr learned my addr 172.21.15.52:0/3488896616 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:14.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.928+0000 7fc463fff640 1 --2- 172.21.15.52:0/3488896616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56219947ebe0 0x562199487280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:14.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.928+0000 7fc4691d9640 1 --2- 172.21.15.52:0/3488896616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621994877c0 0x562199489bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:14.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.928+0000 7fc4689d8640 1 -- 172.21.15.52:0/3488896616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621994877c0 msgr2=0x562199489bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:14.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.928+0000 7fc4689d8640 1 --2- 172.21.15.52:0/3488896616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621994877c0 0x562199489bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.928+0000 7fc4689d8640 1 -- 172.21.15.52:0/3488896616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56219947ebe0 msgr2=0x562199487280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:14.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.928+0000 7fc4689d8640 1 --2- 172.21.15.52:0/3488896616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56219947ebe0 0x562199487280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.928+0000 7fc4691d9640 1 --2- 172.21.15.52:0/1757310037 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc4500a3d20 0x7fc4500a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00b600 tx=0x7fc44c00b2e0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:14.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.928+0000 7fc4689d8640 1 -- 172.21.15.52:0/3488896616 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621991be7e0 con 0x56219947e2c0 2024-04-02T14:01:14.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.928+0000 7fc463fff640 1 --2- 172.21.15.52:0/3488896616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56219947ebe0 0x562199487280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:14.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.928+0000 7fc4689d8640 1 --2- 172.21.15.52:0/3488896616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56219947e2c0 0x56219947e6a0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fc45c09be50 tx=0x7fc45c004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8998152d18ce7eb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.929+0000 7fc45affd640 1 -- 172.21.15.52:0/3488896616 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c020040 con 0x56219947e2c0 2024-04-02T14:01:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.929+0000 7fc45affd640 1 -- 172.21.15.52:0/3488896616 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc45c0a0500 con 0x56219947e2c0 2024-04-02T14:01:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.929+0000 7fc45affd640 1 -- 172.21.15.52:0/3488896616 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c0987c0 con 0x56219947e2c0 2024-04-02T14:01:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.929+0000 7fc46d826740 1 -- 172.21.15.52:0/3488896616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56219947e2c0 msgr2=0x56219947e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.929+0000 7fc46d826740 1 --2- 172.21.15.52:0/3488896616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56219947e2c0 0x56219947e6a0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fc45c09be50 tx=0x7fc45c004780 comp rx=0 tx=0).stop 2024-04-02T14:01:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.929+0000 7fc46d826740 1 -- 172.21.15.52:0/3488896616 shutdown_connections 2024-04-02T14:01:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.929+0000 7fc46d826740 1 --2- 172.21.15.52:0/3488896616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621994877c0 0x562199489bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.929+0000 7fc46d826740 1 --2- 172.21.15.52:0/3488896616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56219947ebe0 0x562199487280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.929+0000 7fc46d826740 1 --2- 172.21.15.52:0/3488896616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56219947e2c0 0x56219947e6a0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.929+0000 7fc46d826740 1 -- 172.21.15.52:0/3488896616 >> 172.21.15.52:0/3488896616 conn(0x5621994789f0 msgr2=0x562199478dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.929+0000 7fc46d826740 1 -- 172.21.15.52:0/3488896616 shutdown_connections 2024-04-02T14:01:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.929+0000 7fc46d826740 1 -- 172.21.15.52:0/3488896616 wait complete. 2024-04-02T14:01:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.930+0000 7fc46d826740 1 -- start start 2024-04-02T14:01:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.930+0000 7fc46d826740 1 Processor -- start 2024-04-02T14:01:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.930+0000 7fc46d826740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56219947e2c0 0x562199484880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.930+0000 7fc46d826740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56219947ebe0 0x562199484dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.930+0000 7fc46d826740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5621994877c0 0x562199485300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:14.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.930+0000 7fc46d826740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56219948a5b0 con 0x56219947ebe0 2024-04-02T14:01:14.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.930+0000 7fc46d826740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56219939dd20 con 0x5621994877c0 2024-04-02T14:01:14.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.930+0000 7fc46d826740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56219948a730 con 0x56219947e2c0 2024-04-02T14:01:14.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.930+0000 7fc46d826740 10 client.0 _ll_register_callbacks cb 0x56219936aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:14.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.930+0000 7fc463fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56219947ebe0 0x562199484dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:14.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.930+0000 7fc463fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56219947ebe0 0x562199484dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56312/0 (socket says 172.21.15.52:56312) 2024-04-02T14:01:14.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.930+0000 7fc463fff640 1 -- 172.21.15.52:0/271936885 learned_addr learned my addr 172.21.15.52:0/271936885 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:14.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.930+0000 7fc4689d8640 1 --2- 172.21.15.52:0/271936885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56219947e2c0 0x562199484880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:14.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.931+0000 7fc4691d9640 1 --2- 172.21.15.52:0/271936885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5621994877c0 0x562199485300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:14.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.931+0000 7fc463fff640 1 -- 172.21.15.52:0/271936885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56219947e2c0 msgr2=0x562199484880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:14.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.931+0000 7fc463fff640 1 --2- 172.21.15.52:0/271936885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56219947e2c0 0x562199484880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.931+0000 7fc463fff640 1 -- 172.21.15.52:0/271936885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5621994877c0 msgr2=0x562199485300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:14.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.931+0000 7fc463fff640 1 --2- 172.21.15.52:0/271936885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5621994877c0 0x562199485300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.931+0000 7fc463fff640 1 -- 172.21.15.52:0/271936885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56219921ef70 con 0x56219947ebe0 2024-04-02T14:01:14.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.931+0000 7fc4689d8640 1 --2- 172.21.15.52:0/271936885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56219947e2c0 0x562199484880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:14.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.931+0000 7fc4691d9640 1 --2- 172.21.15.52:0/271936885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5621994877c0 0x562199485300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:14.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.931+0000 7fc463fff640 1 --2- 172.21.15.52:0/271936885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56219947ebe0 0x562199484dc0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fc454037040 tx=0x7fc454032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991148e91750b6e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:14.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.931+0000 7fc443fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:14.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.931+0000 7fc443fff640 1 -- 172.21.15.52:0/271936885 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc454043070 con 0x56219947ebe0 2024-04-02T14:01:14.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.931+0000 7fc443fff640 1 -- 172.21.15.52:0/271936885 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9526+0+0 (secure 0 0 0) 0x7fc454032790 con 0x56219947ebe0 2024-04-02T14:01:14.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.931+0000 7fc46d826740 1 -- 172.21.15.52:0/271936885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc45c0a0500 con 0x56219947ebe0 2024-04-02T14:01:14.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.931+0000 7fc443fff640 1 -- 172.21.15.52:0/271936885 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc45404f050 con 0x56219947ebe0 2024-04-02T14:01:14.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.931+0000 7fc46d826740 10 client.17868 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:14.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.931+0000 7fc46d826740 1 -- 172.21.15.52:0/271936885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5621990b6170 con 0x56219947ebe0 2024-04-02T14:01:14.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.931+0000 7fc443fff640 1 -- 172.21.15.52:0/271936885 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc45c0a0500 con 0x56219947ebe0 2024-04-02T14:01:14.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.932+0000 7fc443fff640 1 -- 172.21.15.52:0/271936885 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc4540044f0 con 0x56219947ebe0 2024-04-02T14:01:14.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.932+0000 7fc46d826740 10 client.17868 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:14.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.932+0000 7fc46d826740 10 client.17868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:01:14.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.932+0000 7fc46d826740 20 client.17868 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:14.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.932+0000 7fc46d826740 1 --2- 172.21.15.52:0/271936885 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x562199497b70 0x562199499fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:14.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.932+0000 7fc46d826740 4 client.17868 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:01:14.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.932+0000 7fc46d826740 1 -- 172.21.15.52:0/271936885 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621993efcb0 con 0x562199497b70 2024-04-02T14:01:14.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.932+0000 7fc4689d8640 1 --2- 172.21.15.52:0/271936885 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x562199497b70 0x562199499fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:14.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.932+0000 7fc443fff640 10 client.17868 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:14.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.932+0000 7fc4689d8640 1 --2- 172.21.15.52:0/271936885 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x562199497b70 0x562199499fd0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dc98d3b1f895a71 server_cookie=e315cf024d3f97a5 in_seq=0 out_seq=0 2024-04-02T14:01:14.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.933+0000 7fc443fff640 1 -- 172.21.15.52:0/271936885 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5621993efcb0 con 0x562199497b70 2024-04-02T14:01:14.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.933+0000 7fc443fff640 10 client.17868 handle_command_reply: tid=0 2024-04-02T14:01:14.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.933+0000 7fc46d826740 1 client.17868 shutdown 2024-04-02T14:01:14.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.933+0000 7fc46d826740 1 -- 172.21.15.52:0/271936885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56219947ebe0 msgr2=0x562199484dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:14.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.933+0000 7fc46d826740 1 --2- 172.21.15.52:0/271936885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56219947ebe0 0x562199484dc0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fc454037040 tx=0x7fc454032d10 comp rx=0 tx=0).stop 2024-04-02T14:01:14.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.933+0000 7fc46d826740 1 -- 172.21.15.52:0/271936885 shutdown_connections 2024-04-02T14:01:14.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.933+0000 7fc46d826740 1 --2- 172.21.15.52:0/271936885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5621994877c0 0x562199485300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.933+0000 7fc46d826740 1 --2- 172.21.15.52:0/271936885 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x562199497b70 0x562199499fd0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.933+0000 7fc46d826740 1 --2- 172.21.15.52:0/271936885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56219947ebe0 0x562199484dc0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.933+0000 7fc46d826740 1 --2- 172.21.15.52:0/271936885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56219947e2c0 0x562199484880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.933+0000 7fc46d826740 1 -- 172.21.15.52:0/271936885 >> 172.21.15.52:0/271936885 conn(0x5621994789f0 msgr2=0x56219947a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:14.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.933+0000 7fc443fff640 0 client.17868 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:14.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.933+0000 7fc46d826740 1 -- 172.21.15.52:0/271936885 shutdown_connections 2024-04-02T14:01:14.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.933+0000 7fc46d826740 1 -- 172.21.15.52:0/271936885 wait complete. 2024-04-02T14:01:14.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.934+0000 7fc46d826740 20 client.17868 trim_cache size 0 max 16384 2024-04-02T14:01:14.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.952+0000 7fc46d826740 1 Processor -- start 2024-04-02T14:01:14.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.952+0000 7fc46d826740 1 -- start start 2024-04-02T14:01:14.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.952+0000 7fc46d826740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56219947e2c0 0x562199486fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:14.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.952+0000 7fc46d826740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56219947ebe0 0x562199493470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:14.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.952+0000 7fc46d826740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621994877c0 0x56219949f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:14.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.952+0000 7fc46d826740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc4641af140 con 0x5621994877c0 2024-04-02T14:01:14.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.952+0000 7fc46d826740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc4641b0c80 con 0x56219947ebe0 2024-04-02T14:01:14.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.952+0000 7fc46d826740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc4641af810 con 0x56219947e2c0 2024-04-02T14:01:14.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.952+0000 7fc4691d9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621994877c0 0x56219949f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:14.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.952+0000 7fc4691d9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621994877c0 0x56219949f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56318/0 (socket says 172.21.15.52:56318) 2024-04-02T14:01:14.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.952+0000 7fc4691d9640 1 -- 172.21.15.52:0/3831689598 learned_addr learned my addr 172.21.15.52:0/3831689598 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:14.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.952+0000 7fc463fff640 1 --2- 172.21.15.52:0/3831689598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56219947ebe0 0x562199493470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:14.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.952+0000 7fc4691d9640 1 -- 172.21.15.52:0/3831689598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56219947e2c0 msgr2=0x562199486fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:14.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.952+0000 7fc4691d9640 1 --2- 172.21.15.52:0/3831689598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56219947e2c0 0x562199486fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.952+0000 7fc4691d9640 1 -- 172.21.15.52:0/3831689598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56219947ebe0 msgr2=0x562199493470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:14.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.952+0000 7fc4691d9640 1 --2- 172.21.15.52:0/3831689598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56219947ebe0 0x562199493470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.952+0000 7fc4691d9640 1 -- 172.21.15.52:0/3831689598 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562199483830 con 0x5621994877c0 2024-04-02T14:01:14.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.952+0000 7fc463fff640 1 --2- 172.21.15.52:0/3831689598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56219947ebe0 0x562199493470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:14.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.952+0000 7fc4691d9640 1 --2- 172.21.15.52:0/3831689598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621994877c0 0x56219949f8a0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fc44c003d00 tx=0x7fc44c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed9492cdb43fef6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:14.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.952+0000 7fc45a7fc640 1 -- 172.21.15.52:0/3831689598 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c005020 con 0x5621994877c0 2024-04-02T14:01:14.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.952+0000 7fc45a7fc640 1 -- 172.21.15.52:0/3831689598 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc44c0047a0 con 0x5621994877c0 2024-04-02T14:01:14.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.952+0000 7fc45a7fc640 1 -- 172.21.15.52:0/3831689598 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c0117c0 con 0x5621994877c0 2024-04-02T14:01:14.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.953+0000 7fc46d826740 1 -- 172.21.15.52:0/3831689598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621994877c0 msgr2=0x56219949f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:14.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.953+0000 7fc46d826740 1 --2- 172.21.15.52:0/3831689598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621994877c0 0x56219949f8a0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fc44c003d00 tx=0x7fc44c018000 comp rx=0 tx=0).stop 2024-04-02T14:01:14.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.953+0000 7fc46d826740 1 -- 172.21.15.52:0/3831689598 shutdown_connections 2024-04-02T14:01:14.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.953+0000 7fc46d826740 1 --2- 172.21.15.52:0/3831689598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621994877c0 0x56219949f8a0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.953+0000 7fc46d826740 1 --2- 172.21.15.52:0/3831689598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56219947ebe0 0x562199493470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.953+0000 7fc46d826740 1 --2- 172.21.15.52:0/3831689598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56219947e2c0 0x562199486fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.953+0000 7fc46d826740 1 -- 172.21.15.52:0/3831689598 >> 172.21.15.52:0/3831689598 conn(0x5621994789f0 msgr2=0x562199477ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:14.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.953+0000 7fc46d826740 1 -- 172.21.15.52:0/3831689598 shutdown_connections 2024-04-02T14:01:14.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.953+0000 7fc46d826740 1 -- 172.21.15.52:0/3831689598 wait complete. 2024-04-02T14:01:14.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.953+0000 7fc46d826740 1 -- start start 2024-04-02T14:01:14.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.953+0000 7fc46d826740 1 Processor -- start 2024-04-02T14:01:14.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.954+0000 7fc46d826740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56219947e2c0 0x56219948c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:14.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.954+0000 7fc46d826740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56219947ebe0 0x56219948cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:14.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.954+0000 7fc46d826740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621994877c0 0x56219948d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:14.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.954+0000 7fc46d826740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc4641b05b0 con 0x56219947ebe0 2024-04-02T14:01:14.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.954+0000 7fc46d826740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc4641afee0 con 0x56219947e2c0 2024-04-02T14:01:14.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.954+0000 7fc46d826740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc4641ad140 con 0x5621994877c0 2024-04-02T14:01:14.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.954+0000 7fc46d826740 10 client.0 _ll_register_callbacks cb 0x7fc454037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:14.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.954+0000 7fc463fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56219947ebe0 0x56219948cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:14.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.954+0000 7fc463fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56219947ebe0 0x56219948cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56322/0 (socket says 172.21.15.52:56322) 2024-04-02T14:01:14.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.954+0000 7fc463fff640 1 -- 172.21.15.52:0/484743014 learned_addr learned my addr 172.21.15.52:0/484743014 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:14.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.954+0000 7fc4691d9640 1 --2- 172.21.15.52:0/484743014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621994877c0 0x56219948d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:14.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.954+0000 7fc4689d8640 1 --2- 172.21.15.52:0/484743014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56219947e2c0 0x56219948c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:14.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.954+0000 7fc463fff640 1 -- 172.21.15.52:0/484743014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621994877c0 msgr2=0x56219948d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:14.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.954+0000 7fc463fff640 1 --2- 172.21.15.52:0/484743014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621994877c0 0x56219948d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.954+0000 7fc463fff640 1 -- 172.21.15.52:0/484743014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56219947e2c0 msgr2=0x56219948c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:14.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.954+0000 7fc463fff640 1 --2- 172.21.15.52:0/484743014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56219947e2c0 0x56219948c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.954+0000 7fc463fff640 1 -- 172.21.15.52:0/484743014 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621994fe3f0 con 0x56219947ebe0 2024-04-02T14:01:14.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.954+0000 7fc4691d9640 1 --2- 172.21.15.52:0/484743014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621994877c0 0x56219948d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:14.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.954+0000 7fc4689d8640 1 --2- 172.21.15.52:0/484743014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56219947e2c0 0x56219948c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:14.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.954+0000 7fc463fff640 1 --2- 172.21.15.52:0/484743014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56219947ebe0 0x56219948cbc0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fc45402c630 tx=0x7fc454032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a95821ed0ff658 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:14.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.954+0000 7fc443fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:14.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.955+0000 7fc443fff640 1 -- 172.21.15.52:0/484743014 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc454043040 con 0x56219947ebe0 2024-04-02T14:01:14.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.955+0000 7fc443fff640 1 -- 172.21.15.52:0/484743014 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9526+0+0 (secure 0 0 0) 0x7fc454034c00 con 0x56219947ebe0 2024-04-02T14:01:14.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.955+0000 7fc46d826740 1 -- 172.21.15.52:0/484743014 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc44c0047a0 con 0x56219947ebe0 2024-04-02T14:01:14.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.955+0000 7fc443fff640 1 -- 172.21.15.52:0/484743014 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc45405c050 con 0x56219947ebe0 2024-04-02T14:01:14.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.955+0000 7fc46d826740 10 client.17874 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:14.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.955+0000 7fc46d826740 1 -- 172.21.15.52:0/484743014 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5621994ff6a0 con 0x56219947ebe0 2024-04-02T14:01:14.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.955+0000 7fc443fff640 1 -- 172.21.15.52:0/484743014 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc44c0047a0 con 0x56219947ebe0 2024-04-02T14:01:14.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.955+0000 7fc443fff640 1 -- 172.21.15.52:0/484743014 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc454048890 con 0x56219947ebe0 2024-04-02T14:01:14.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.955+0000 7fc46d826740 10 client.17874 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:14.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.955+0000 7fc46d826740 10 client.17874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:01:14.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.955+0000 7fc46d826740 20 client.17874 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:14.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.955+0000 7fc46d826740 1 --2- 172.21.15.52:0/484743014 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5621994ffdf0 0x562199502210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:14.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.955+0000 7fc46d826740 4 client.17874 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T14:01:14.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.955+0000 7fc46d826740 1 -- 172.21.15.52:0/484743014 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5621993ea000 con 0x5621994ffdf0 2024-04-02T14:01:14.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.955+0000 7fc4689d8640 1 --2- 172.21.15.52:0/484743014 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5621994ffdf0 0x562199502210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:14.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.956+0000 7fc4689d8640 1 --2- 172.21.15.52:0/484743014 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5621994ffdf0 0x562199502210 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c1f7dd25e7ecd80 server_cookie=9e883e718b58d11d in_seq=0 out_seq=0 2024-04-02T14:01:14.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.956+0000 7fc443fff640 10 client.17874 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:14.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.957+0000 7fc443fff640 1 -- 172.21.15.52:0/484743014 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5621993ea000 con 0x5621994ffdf0 2024-04-02T14:01:14.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.957+0000 7fc443fff640 10 client.17874 handle_command_reply: tid=0 2024-04-02T14:01:14.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.957+0000 7fc46d826740 1 client.17874 shutdown 2024-04-02T14:01:14.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.957+0000 7fc46d826740 1 -- 172.21.15.52:0/484743014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56219947ebe0 msgr2=0x56219948cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:14.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.957+0000 7fc46d826740 1 --2- 172.21.15.52:0/484743014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56219947ebe0 0x56219948cbc0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fc45402c630 tx=0x7fc454032f90 comp rx=0 tx=0).stop 2024-04-02T14:01:14.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.957+0000 7fc46d826740 1 -- 172.21.15.52:0/484743014 shutdown_connections 2024-04-02T14:01:14.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.957+0000 7fc46d826740 1 --2- 172.21.15.52:0/484743014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621994877c0 0x56219948d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.957+0000 7fc46d826740 1 --2- 172.21.15.52:0/484743014 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5621994ffdf0 0x562199502210 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.957+0000 7fc46d826740 1 --2- 172.21.15.52:0/484743014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56219947ebe0 0x56219948cbc0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.957+0000 7fc46d826740 1 --2- 172.21.15.52:0/484743014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56219947e2c0 0x56219948c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.957+0000 7fc443fff640 0 client.17874 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:14.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.957+0000 7fc46d826740 1 -- 172.21.15.52:0/484743014 >> 172.21.15.52:0/484743014 conn(0x5621994789f0 msgr2=0x562199484820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:14.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.957+0000 7fc46d826740 1 -- 172.21.15.52:0/484743014 shutdown_connections 2024-04-02T14:01:14.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.957+0000 7fc46d826740 1 -- 172.21.15.52:0/484743014 wait complete. 2024-04-02T14:01:14.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.958+0000 7fc46d826740 20 client.17874 trim_cache size 0 max 16384 2024-04-02T14:01:14.972 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:01:14.972 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-04-02T14:01:14.972 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "693910b7-7799-4909-b9fc-85035ff4ce06", 2024-04-02T14:01:14.972 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-04-02T14:01:14.972 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:01:14.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.958+0000 7fc46ac3c640 1 -- 172.21.15.52:0/1757310037 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc4500a3d20 msgr2=0x7fc4500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:14.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.958+0000 7fc46ac3c640 1 --2- 172.21.15.52:0/1757310037 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc4500a3d20 0x7fc4500a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00b600 tx=0x7fc44c00b2e0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.958+0000 7fc46ac3c640 1 -- 172.21.15.52:0/1757310037 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc464068ce0 msgr2=0x7fc4641983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:14.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.958+0000 7fc46ac3c640 1 --2- 172.21.15.52:0/1757310037 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc464068ce0 0x7fc4641983f0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00d770 tx=0x7fc45c00dc30 comp rx=0 tx=0).stop 2024-04-02T14:01:14.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.958+0000 7fc46ac3c640 1 -- 172.21.15.52:0/1757310037 shutdown_connections 2024-04-02T14:01:14.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.958+0000 7fc46ac3c640 1 --2- 172.21.15.52:0/1757310037 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc4500a3d20 0x7fc4500a61e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.958+0000 7fc46ac3c640 1 --2- 172.21.15.52:0/1757310037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc464107a80 0x7fc46419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.958+0000 7fc46ac3c640 1 --2- 172.21.15.52:0/1757310037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4640696a0 0x7fc464198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.958+0000 7fc46ac3c640 1 --2- 172.21.15.52:0/1757310037 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc464068ce0 0x7fc4641983f0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:14.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.958+0000 7fc46ac3c640 1 -- 172.21.15.52:0/1757310037 >> 172.21.15.52:0/1757310037 conn(0x7fc464077b00 msgr2=0x7fc4640fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:14.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.958+0000 7fc46ac3c640 1 -- 172.21.15.52:0/1757310037 shutdown_connections 2024-04-02T14:01:14.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:14.958+0000 7fc46ac3c640 1 -- 172.21.15.52:0/1757310037 wait complete. 2024-04-02T14:01:14.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:01:14.975 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:01:15.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.084+0000 7efe73e3c640 1 -- 172.21.15.52:0/1318906660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe6c068d00 msgr2=0x7efe6c069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:15.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.084+0000 7efe73e3c640 1 --2- 172.21.15.52:0/1318906660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe6c068d00 0x7efe6c069180 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009930 tx=0x7efe5c02bc30 comp rx=0 tx=0).stop 2024-04-02T14:01:15.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.084+0000 7efe73e3c640 1 -- 172.21.15.52:0/1318906660 shutdown_connections 2024-04-02T14:01:15.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.084+0000 7efe73e3c640 1 --2- 172.21.15.52:0/1318906660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe6c0696c0 0x7efe6c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.084+0000 7efe73e3c640 1 --2- 172.21.15.52:0/1318906660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe6c068d00 0x7efe6c069180 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.084+0000 7efe73e3c640 1 --2- 172.21.15.52:0/1318906660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe6c107b50 0x7efe6c107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.084+0000 7efe73e3c640 1 -- 172.21.15.52:0/1318906660 >> 172.21.15.52:0/1318906660 conn(0x7efe6c077b00 msgr2=0x7efe6c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:15.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.084+0000 7efe73e3c640 1 -- 172.21.15.52:0/1318906660 shutdown_connections 2024-04-02T14:01:15.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.084+0000 7efe73e3c640 1 -- 172.21.15.52:0/1318906660 wait complete. 2024-04-02T14:01:15.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.085+0000 7efe73e3c640 1 Processor -- start 2024-04-02T14:01:15.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.085+0000 7efe73e3c640 1 -- start start 2024-04-02T14:01:15.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.085+0000 7efe73e3c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe6c068d00 0x7efe6c1009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:15.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.085+0000 7efe73e3c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe6c0696c0 0x7efe6c100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:15.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.085+0000 7efe73e3c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe6c107b50 0x7efe6c101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:15.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.085+0000 7efe73e3c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c111ea0 con 0x7efe6c068d00 2024-04-02T14:01:15.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.085+0000 7efe73e3c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c111d20 con 0x7efe6c0696c0 2024-04-02T14:01:15.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.085+0000 7efe73e3c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c112020 con 0x7efe6c107b50 2024-04-02T14:01:15.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.085+0000 7efe71bd8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe6c068d00 0x7efe6c1009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:15.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.085+0000 7efe71bd8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe6c068d00 0x7efe6c1009c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56334/0 (socket says 172.21.15.52:56334) 2024-04-02T14:01:15.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.085+0000 7efe71bd8640 1 -- 172.21.15.52:0/517206859 learned_addr learned my addr 172.21.15.52:0/517206859 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:15.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.085+0000 7efe713d7640 1 --2- 172.21.15.52:0/517206859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe6c0696c0 0x7efe6c100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:15.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.085+0000 7efe723d9640 1 --2- 172.21.15.52:0/517206859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe6c107b50 0x7efe6c101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.085+0000 7efe713d7640 1 -- 172.21.15.52:0/517206859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe6c107b50 msgr2=0x7efe6c101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.085+0000 7efe713d7640 1 --2- 172.21.15.52:0/517206859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe6c107b50 0x7efe6c101440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.085+0000 7efe713d7640 1 -- 172.21.15.52:0/517206859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe6c068d00 msgr2=0x7efe6c1009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.085+0000 7efe713d7640 1 --2- 172.21.15.52:0/517206859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe6c068d00 0x7efe6c1009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.085+0000 7efe713d7640 1 -- 172.21.15.52:0/517206859 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6c1a6890 con 0x7efe6c0696c0 2024-04-02T14:01:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.086+0000 7efe713d7640 1 --2- 172.21.15.52:0/517206859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe6c0696c0 0x7efe6c100f00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009230 tx=0x7efe5c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.086+0000 7efe62ffd640 1 -- 172.21.15.52:0/517206859 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c041070 con 0x7efe6c0696c0 2024-04-02T14:01:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.086+0000 7efe62ffd640 1 -- 172.21.15.52:0/517206859 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe5c0054e0 con 0x7efe6c0696c0 2024-04-02T14:01:15.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.086+0000 7efe73e3c640 1 -- 172.21.15.52:0/517206859 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe6c1a6b20 con 0x7efe6c0696c0 2024-04-02T14:01:15.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.086+0000 7efe62ffd640 1 -- 172.21.15.52:0/517206859 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c003bf0 con 0x7efe6c0696c0 2024-04-02T14:01:15.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.086+0000 7efe73e3c640 1 -- 172.21.15.52:0/517206859 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe6c1a7030 con 0x7efe6c0696c0 2024-04-02T14:01:15.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.089+0000 7efe62ffd640 1 -- 172.21.15.52:0/517206859 <== mon.2 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efe5c044070 con 0x7efe6c0696c0 2024-04-02T14:01:15.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.089+0000 7efe76a23740 1 Processor -- start 2024-04-02T14:01:15.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.089+0000 7efe76a23740 1 -- start start 2024-04-02T14:01:15.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.089+0000 7efe76a23740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635cd0dc8d0 0x5635cd0dccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:15.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.089+0000 7efe76a23740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 0x5635cd0e5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:15.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.090+0000 7efe76a23740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5635cd0e5dd0 0x5635cd0e81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:15.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.090+0000 7efe76a23740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5635cd040830 con 0x5635cd0dd1f0 2024-04-02T14:01:15.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.090+0000 7efe76a23740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5635ccfe8820 con 0x5635cd0dc8d0 2024-04-02T14:01:15.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.090+0000 7efe76a23740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5635cd03c7e0 con 0x5635cd0e5dd0 2024-04-02T14:01:15.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.090+0000 7efe71bd8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 0x5635cd0e5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:15.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.090+0000 7efe723d9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635cd0dc8d0 0x5635cd0dccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:15.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.090+0000 7efe71bd8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 0x5635cd0e5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56342/0 (socket says 172.21.15.52:56342) 2024-04-02T14:01:15.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.090+0000 7efe713d7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5635cd0e5dd0 0x5635cd0e81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:15.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.090+0000 7efe723d9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635cd0dc8d0 0x5635cd0dccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:33536/0 (socket says 172.21.15.52:33536) 2024-04-02T14:01:15.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.090+0000 7efe71bd8640 1 -- 172.21.15.52:0/2405856405 learned_addr learned my addr 172.21.15.52:0/2405856405 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:15.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.090+0000 7efe62ffd640 1 --2- 172.21.15.52:0/517206859 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efe580a3dd0 0x7efe580a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:15.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.090+0000 7efe723d9640 1 --2- 172.21.15.52:0/517206859 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efe580a3dd0 0x7efe580a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:15.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.090+0000 7efe71bd8640 1 -- 172.21.15.52:0/2405856405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5635cd0e5dd0 msgr2=0x5635cd0e81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:15.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.090+0000 7efe71bd8640 1 --2- 172.21.15.52:0/2405856405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5635cd0e5dd0 0x5635cd0e81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.090+0000 7efe71bd8640 1 -- 172.21.15.52:0/2405856405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635cd0dc8d0 msgr2=0x5635cd0dccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:15.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.090+0000 7efe71bd8640 1 --2- 172.21.15.52:0/2405856405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635cd0dc8d0 0x5635cd0dccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.090+0000 7efe71bd8640 1 -- 172.21.15.52:0/2405856405 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635cce1e7d0 con 0x5635cd0dd1f0 2024-04-02T14:01:15.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.090+0000 7efe62ffd640 1 -- 172.21.15.52:0/517206859 <== mon.2 v2:172.21.15.88:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9526+0+0 (secure 0 0 0) 0x7efe5c0f2380 con 0x7efe6c0696c0 2024-04-02T14:01:15.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.090+0000 7efe723d9640 1 --2- 172.21.15.52:0/2405856405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635cd0dc8d0 0x5635cd0dccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:01:15.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.090+0000 7efe71bd8640 1 --2- 172.21.15.52:0/2405856405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 0x5635cd0e5890 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7efe68009d90 tx=0x7efe68011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99252d564af4c723 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:15.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.090+0000 7efe723d9640 1 --2- 172.21.15.52:0/517206859 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efe580a3dd0 0x7efe580a6290 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efe64009fd0 tx=0x7efe64004ab0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:15.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.091+0000 7efe57fff640 1 -- 172.21.15.52:0/2405856405 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6801e070 con 0x5635cd0dd1f0 2024-04-02T14:01:15.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.091+0000 7efe57fff640 1 -- 172.21.15.52:0/2405856405 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe68019b10 con 0x5635cd0dd1f0 2024-04-02T14:01:15.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.091+0000 7efe57fff640 1 -- 172.21.15.52:0/2405856405 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6800fbe0 con 0x5635cd0dd1f0 2024-04-02T14:01:15.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.091+0000 7efe76a23740 1 -- 172.21.15.52:0/2405856405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 msgr2=0x5635cd0e5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:15.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.091+0000 7efe76a23740 1 --2- 172.21.15.52:0/2405856405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 0x5635cd0e5890 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7efe68009d90 tx=0x7efe68011810 comp rx=0 tx=0).stop 2024-04-02T14:01:15.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.091+0000 7efe76a23740 1 -- 172.21.15.52:0/2405856405 shutdown_connections 2024-04-02T14:01:15.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.091+0000 7efe76a23740 1 --2- 172.21.15.52:0/2405856405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5635cd0e5dd0 0x5635cd0e81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.091+0000 7efe76a23740 1 --2- 172.21.15.52:0/2405856405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 0x5635cd0e5890 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.091+0000 7efe76a23740 1 --2- 172.21.15.52:0/2405856405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635cd0dc8d0 0x5635cd0dccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.091+0000 7efe76a23740 1 -- 172.21.15.52:0/2405856405 >> 172.21.15.52:0/2405856405 conn(0x5635cd0d7000 msgr2=0x5635cd0d73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:15.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.092+0000 7efe76a23740 1 -- 172.21.15.52:0/2405856405 shutdown_connections 2024-04-02T14:01:15.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.092+0000 7efe76a23740 1 -- 172.21.15.52:0/2405856405 wait complete. 2024-04-02T14:01:15.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.092+0000 7efe76a23740 1 -- start start 2024-04-02T14:01:15.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.092+0000 7efe76a23740 1 Processor -- start 2024-04-02T14:01:15.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.092+0000 7efe76a23740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5635cd0dc8d0 0x5635cd0e5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:15.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.092+0000 7efe76a23740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 0x5635cd0e29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:15.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.092+0000 7efe76a23740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635cd0e5dd0 0x5635cd0e2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:15.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.093+0000 7efe76a23740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5635cd0e8bc0 con 0x5635cd0dd1f0 2024-04-02T14:01:15.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.093+0000 7efe76a23740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5635ccffbd30 con 0x5635cd0e5dd0 2024-04-02T14:01:15.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.093+0000 7efe76a23740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5635cd0e8d40 con 0x5635cd0dc8d0 2024-04-02T14:01:15.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.093+0000 7efe713d7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 0x5635cd0e29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:15.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.093+0000 7efe76a23740 10 client.0 _ll_register_callbacks cb 0x5635ccffb720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:15.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.093+0000 7efe713d7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 0x5635cd0e29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56352/0 (socket says 172.21.15.52:56352) 2024-04-02T14:01:15.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.093+0000 7efe713d7640 1 -- 172.21.15.52:0/4205031985 learned_addr learned my addr 172.21.15.52:0/4205031985 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:15.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.093+0000 7efe723d9640 1 --2- 172.21.15.52:0/4205031985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635cd0e5dd0 0x5635cd0e2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:15.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.093+0000 7efe71bd8640 1 --2- 172.21.15.52:0/4205031985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5635cd0dc8d0 0x5635cd0e5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:15.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.093+0000 7efe713d7640 1 -- 172.21.15.52:0/4205031985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5635cd0dc8d0 msgr2=0x5635cd0e5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:15.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.093+0000 7efe713d7640 1 --2- 172.21.15.52:0/4205031985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5635cd0dc8d0 0x5635cd0e5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.093+0000 7efe713d7640 1 -- 172.21.15.52:0/4205031985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635cd0e5dd0 msgr2=0x5635cd0e2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:15.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.093+0000 7efe713d7640 1 --2- 172.21.15.52:0/4205031985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635cd0e5dd0 0x5635cd0e2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.093+0000 7efe713d7640 1 -- 172.21.15.52:0/4205031985 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635ccd543f0 con 0x5635cd0dd1f0 2024-04-02T14:01:15.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.093+0000 7efe71bd8640 1 --2- 172.21.15.52:0/4205031985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5635cd0dc8d0 0x5635cd0e5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:15.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.093+0000 7efe713d7640 1 --2- 172.21.15.52:0/4205031985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 0x5635cd0e29f0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0baa10 tx=0x7efe5c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4bda1efe8b1b806 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:15.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.093+0000 7efe557fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:15.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.093+0000 7efe557fa640 1 -- 172.21.15.52:0/4205031985 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c04b040 con 0x5635cd0dd1f0 2024-04-02T14:01:15.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.093+0000 7efe557fa640 1 -- 172.21.15.52:0/4205031985 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9526+0+0 (secure 0 0 0) 0x7efe5c0ba6b0 con 0x5635cd0dd1f0 2024-04-02T14:01:15.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.093+0000 7efe76a23740 1 -- 172.21.15.52:0/4205031985 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe68019b10 con 0x5635cd0dd1f0 2024-04-02T14:01:15.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.093+0000 7efe557fa640 1 -- 172.21.15.52:0/4205031985 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efe5c0bd9a0 con 0x5635cd0dd1f0 2024-04-02T14:01:15.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.093+0000 7efe76a23740 10 client.17886 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:15.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.093+0000 7efe76a23740 1 -- 172.21.15.52:0/4205031985 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5635ccd140f0 con 0x5635cd0dd1f0 2024-04-02T14:01:15.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.094+0000 7efe557fa640 1 -- 172.21.15.52:0/4205031985 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe68019b10 con 0x5635cd0dd1f0 2024-04-02T14:01:15.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.094+0000 7efe557fa640 1 -- 172.21.15.52:0/4205031985 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7efe5c0c6820 con 0x5635cd0dd1f0 2024-04-02T14:01:15.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.094+0000 7efe76a23740 10 client.17886 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:15.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.094+0000 7efe76a23740 10 client.17886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:01:15.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.094+0000 7efe76a23740 20 client.17886 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:15.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.094+0000 7efe76a23740 1 --2- 172.21.15.52:0/4205031985 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5635cd0f5c40 0x5635cd0f8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:15.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.094+0000 7efe76a23740 4 client.17886 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:01:15.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.094+0000 7efe76a23740 1 -- 172.21.15.52:0/4205031985 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635cd04dd70 con 0x5635cd0f5c40 2024-04-02T14:01:15.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.094+0000 7efe71bd8640 1 --2- 172.21.15.52:0/4205031985 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5635cd0f5c40 0x5635cd0f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:15.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.095+0000 7efe557fa640 10 client.17886 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:15.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.095+0000 7efe71bd8640 1 --2- 172.21.15.52:0/4205031985 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5635cd0f5c40 0x5635cd0f8070 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=401d99942e266925 server_cookie=565aa54bd8cb0b25 in_seq=0 out_seq=0 2024-04-02T14:01:15.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.096+0000 7efe557fa640 1 -- 172.21.15.52:0/4205031985 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5635cd04dd70 con 0x5635cd0f5c40 2024-04-02T14:01:15.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.096+0000 7efe557fa640 10 client.17886 handle_command_reply: tid=0 2024-04-02T14:01:15.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.096+0000 7efe76a23740 1 client.17886 shutdown 2024-04-02T14:01:15.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.096+0000 7efe76a23740 1 -- 172.21.15.52:0/4205031985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 msgr2=0x5635cd0e29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:15.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.096+0000 7efe76a23740 1 --2- 172.21.15.52:0/4205031985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 0x5635cd0e29f0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0baa10 tx=0x7efe5c005c90 comp rx=0 tx=0).stop 2024-04-02T14:01:15.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.096+0000 7efe76a23740 1 -- 172.21.15.52:0/4205031985 shutdown_connections 2024-04-02T14:01:15.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.096+0000 7efe76a23740 1 --2- 172.21.15.52:0/4205031985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635cd0e5dd0 0x5635cd0e2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.096+0000 7efe76a23740 1 --2- 172.21.15.52:0/4205031985 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5635cd0f5c40 0x5635cd0f8070 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.096+0000 7efe76a23740 1 --2- 172.21.15.52:0/4205031985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 0x5635cd0e29f0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.096+0000 7efe76a23740 1 --2- 172.21.15.52:0/4205031985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5635cd0dc8d0 0x5635cd0e5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.096+0000 7efe557fa640 0 client.17886 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:15.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.096+0000 7efe76a23740 1 -- 172.21.15.52:0/4205031985 >> 172.21.15.52:0/4205031985 conn(0x5635cd0d7000 msgr2=0x5635cd0d8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:15.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.096+0000 7efe76a23740 1 -- 172.21.15.52:0/4205031985 shutdown_connections 2024-04-02T14:01:15.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.096+0000 7efe76a23740 1 -- 172.21.15.52:0/4205031985 wait complete. 2024-04-02T14:01:15.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.096+0000 7efe76a23740 20 client.17886 trim_cache size 0 max 16384 2024-04-02T14:01:15.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.116+0000 7efe76a23740 1 Processor -- start 2024-04-02T14:01:15.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.116+0000 7efe76a23740 1 -- start start 2024-04-02T14:01:15.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.117+0000 7efe76a23740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5635cd0d7000 0x5635cd0d60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:15.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.117+0000 7efe76a23740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635cd0dc8d0 0x5635cd0e5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:15.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.117+0000 7efe76a23740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 0x5635cd0e5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:15.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.117+0000 7efe76a23740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1b11f0 con 0x5635cd0dd1f0 2024-04-02T14:01:15.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.117+0000 7efe76a23740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1b2d30 con 0x5635cd0dc8d0 2024-04-02T14:01:15.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.117+0000 7efe76a23740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1b18c0 con 0x5635cd0d7000 2024-04-02T14:01:15.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.117+0000 7efe723d9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 0x5635cd0e5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:15.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.117+0000 7efe713d7640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635cd0dc8d0 0x5635cd0e5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:15.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.117+0000 7efe723d9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 0x5635cd0e5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56358/0 (socket says 172.21.15.52:56358) 2024-04-02T14:01:15.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.117+0000 7efe723d9640 1 -- 172.21.15.52:0/1041644155 learned_addr learned my addr 172.21.15.52:0/1041644155 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:15.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.117+0000 7efe713d7640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635cd0dc8d0 0x5635cd0e5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:33558/0 (socket says 172.21.15.52:33558) 2024-04-02T14:01:15.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.117+0000 7efe71bd8640 1 --2- 172.21.15.52:0/1041644155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5635cd0d7000 0x5635cd0d60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:15.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.117+0000 7efe723d9640 1 -- 172.21.15.52:0/1041644155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5635cd0d7000 msgr2=0x5635cd0d60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:15.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.117+0000 7efe723d9640 1 --2- 172.21.15.52:0/1041644155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5635cd0d7000 0x5635cd0d60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.117+0000 7efe723d9640 1 -- 172.21.15.52:0/1041644155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635cd0dc8d0 msgr2=0x5635cd0e5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:15.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.117+0000 7efe723d9640 1 --2- 172.21.15.52:0/1041644155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635cd0dc8d0 0x5635cd0e5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.117+0000 7efe723d9640 1 -- 172.21.15.52:0/1041644155 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635cd0d6e30 con 0x5635cd0dd1f0 2024-04-02T14:01:15.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.117+0000 7efe71bd8640 1 --2- 172.21.15.52:0/1041644155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5635cd0d7000 0x5635cd0d60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:01:15.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.117+0000 7efe723d9640 1 --2- 172.21.15.52:0/1041644155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 0x5635cd0e5660 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7efe640039c0 tx=0x7efe64009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745690826cfa4ccd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:15.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.118+0000 7efe577fe640 1 -- 172.21.15.52:0/1041644155 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64019070 con 0x5635cd0dd1f0 2024-04-02T14:01:15.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.118+0000 7efe577fe640 1 -- 172.21.15.52:0/1041644155 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe64025070 con 0x5635cd0dd1f0 2024-04-02T14:01:15.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.118+0000 7efe577fe640 1 -- 172.21.15.52:0/1041644155 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe640217d0 con 0x5635cd0dd1f0 2024-04-02T14:01:15.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.118+0000 7efe76a23740 1 -- 172.21.15.52:0/1041644155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 msgr2=0x5635cd0e5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:15.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.118+0000 7efe76a23740 1 --2- 172.21.15.52:0/1041644155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 0x5635cd0e5660 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7efe640039c0 tx=0x7efe64009a20 comp rx=0 tx=0).stop 2024-04-02T14:01:15.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.118+0000 7efe76a23740 1 -- 172.21.15.52:0/1041644155 shutdown_connections 2024-04-02T14:01:15.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.118+0000 7efe76a23740 1 --2- 172.21.15.52:0/1041644155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 0x5635cd0e5660 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.118+0000 7efe76a23740 1 --2- 172.21.15.52:0/1041644155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635cd0dc8d0 0x5635cd0e5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.118+0000 7efe76a23740 1 --2- 172.21.15.52:0/1041644155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5635cd0d7000 0x5635cd0d60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.118+0000 7efe76a23740 1 -- 172.21.15.52:0/1041644155 >> 172.21.15.52:0/1041644155 conn(0x5635cd0e1630 msgr2=0x5635cd0d59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:15.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.119+0000 7efe76a23740 1 -- 172.21.15.52:0/1041644155 shutdown_connections 2024-04-02T14:01:15.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.119+0000 7efe76a23740 1 -- 172.21.15.52:0/1041644155 wait complete. 2024-04-02T14:01:15.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.119+0000 7efe76a23740 1 -- start start 2024-04-02T14:01:15.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.119+0000 7efe76a23740 1 Processor -- start 2024-04-02T14:01:15.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.119+0000 7efe76a23740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635cd0d7000 0x5635cd0ee270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:15.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.119+0000 7efe76a23740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5635cd0dc8d0 0x5635cd0ea5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:15.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.119+0000 7efe76a23740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 0x5635cd0eab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:15.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.119+0000 7efe76a23740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1b2660 con 0x5635cd0dd1f0 2024-04-02T14:01:15.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.119+0000 7efe76a23740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1b1f90 con 0x5635cd0d7000 2024-04-02T14:01:15.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.119+0000 7efe76a23740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1af1f0 con 0x5635cd0dc8d0 2024-04-02T14:01:15.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.119+0000 7efe76a23740 10 client.0 _ll_register_callbacks cb 0x7efe5c009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:15.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.119+0000 7efe71bd8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635cd0d7000 0x5635cd0ee270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:15.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.119+0000 7efe723d9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 0x5635cd0eab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:15.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.119+0000 7efe71bd8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635cd0d7000 0x5635cd0ee270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:33574/0 (socket says 172.21.15.52:33574) 2024-04-02T14:01:15.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.119+0000 7efe723d9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 0x5635cd0eab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56362/0 (socket says 172.21.15.52:56362) 2024-04-02T14:01:15.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.119+0000 7efe71bd8640 1 -- 172.21.15.52:0/788431598 learned_addr learned my addr 172.21.15.52:0/788431598 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:15.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.120+0000 7efe713d7640 1 --2- 172.21.15.52:0/788431598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5635cd0dc8d0 0x5635cd0ea5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:15.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.120+0000 7efe71bd8640 1 -- 172.21.15.52:0/788431598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5635cd0dc8d0 msgr2=0x5635cd0ea5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:15.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.120+0000 7efe71bd8640 1 --2- 172.21.15.52:0/788431598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5635cd0dc8d0 0x5635cd0ea5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.120+0000 7efe71bd8640 1 -- 172.21.15.52:0/788431598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 msgr2=0x5635cd0eab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:15.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.120+0000 7efe71bd8640 1 --2- 172.21.15.52:0/788431598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 0x5635cd0eab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.120+0000 7efe71bd8640 1 -- 172.21.15.52:0/788431598 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635cd0eb220 con 0x5635cd0d7000 2024-04-02T14:01:15.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.120+0000 7efe71bd8640 1 --2- 172.21.15.52:0/788431598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635cd0d7000 0x5635cd0ee270 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efe68009250 tx=0x7efe68009470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fe16f4c7320b0be server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:15.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.120+0000 7efe557fa640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-02T14:01:15.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.120+0000 7efe557fa640 1 -- 172.21.15.52:0/788431598 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6801e040 con 0x5635cd0d7000 2024-04-02T14:01:15.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.120+0000 7efe557fa640 1 -- 172.21.15.52:0/788431598 <== mon.2 v2:172.21.15.88:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9526+0+0 (secure 0 0 0) 0x7efe680050b0 con 0x5635cd0d7000 2024-04-02T14:01:15.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.120+0000 7efe76a23740 1 -- 172.21.15.52:0/788431598 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe64025070 con 0x5635cd0d7000 2024-04-02T14:01:15.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.120+0000 7efe557fa640 1 -- 172.21.15.52:0/788431598 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efe68038050 con 0x5635cd0d7000 2024-04-02T14:01:15.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.120+0000 7efe76a23740 10 client.27020 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:15.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.121+0000 7efe76a23740 1 -- 172.21.15.52:0/788431598 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5635cd15d0e0 con 0x5635cd0d7000 2024-04-02T14:01:15.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.121+0000 7efe557fa640 1 -- 172.21.15.52:0/788431598 <== mon.2 v2:172.21.15.88:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe64025070 con 0x5635cd0d7000 2024-04-02T14:01:15.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.121+0000 7efe557fa640 1 -- 172.21.15.52:0/788431598 <== mon.2 v2:172.21.15.88:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7efe6800fd10 con 0x5635cd0d7000 2024-04-02T14:01:15.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.121+0000 7efe76a23740 10 client.27020 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:15.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.121+0000 7efe76a23740 10 client.27020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:01:15.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.121+0000 7efe76a23740 20 client.27020 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:15.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.121+0000 7efe76a23740 1 --2- 172.21.15.52:0/788431598 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5635cd0e5dd0 0x5635cd15f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:15.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.121+0000 7efe76a23740 4 client.27020 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:01:15.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.121+0000 7efe76a23740 1 -- 172.21.15.52:0/788431598 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5635cd0480c0 con 0x5635cd0e5dd0 2024-04-02T14:01:15.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.121+0000 7efe713d7640 1 --2- 172.21.15.52:0/788431598 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5635cd0e5dd0 0x5635cd15f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:15.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.122+0000 7efe713d7640 1 --2- 172.21.15.52:0/788431598 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5635cd0e5dd0 0x5635cd15f6e0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f217f7ed3ecffd9e server_cookie=c5e86d47251e4704 in_seq=0 out_seq=0 2024-04-02T14:01:15.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.122+0000 7efe557fa640 10 client.27020 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:15.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.122+0000 7efe557fa640 1 -- 172.21.15.52:0/788431598 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x5635cd0480c0 con 0x5635cd0e5dd0 2024-04-02T14:01:15.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.122+0000 7efe557fa640 10 client.27020 handle_command_reply: tid=0 2024-04-02T14:01:15.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.122+0000 7efe76a23740 1 client.27020 shutdown 2024-04-02T14:01:15.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.122+0000 7efe76a23740 1 -- 172.21.15.52:0/788431598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635cd0d7000 msgr2=0x5635cd0ee270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:15.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.122+0000 7efe76a23740 1 --2- 172.21.15.52:0/788431598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635cd0d7000 0x5635cd0ee270 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efe68009250 tx=0x7efe68009470 comp rx=0 tx=0).stop 2024-04-02T14:01:15.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.122+0000 7efe76a23740 1 -- 172.21.15.52:0/788431598 shutdown_connections 2024-04-02T14:01:15.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.122+0000 7efe76a23740 1 --2- 172.21.15.52:0/788431598 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5635cd0e5dd0 0x5635cd15f6e0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.122+0000 7efe76a23740 1 --2- 172.21.15.52:0/788431598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635cd0dd1f0 0x5635cd0eab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.122+0000 7efe557fa640 0 client.27020 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:15.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.122+0000 7efe76a23740 1 --2- 172.21.15.52:0/788431598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5635cd0dc8d0 0x5635cd0ea5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.122+0000 7efe76a23740 1 --2- 172.21.15.52:0/788431598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635cd0d7000 0x5635cd0ee270 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.122+0000 7efe76a23740 1 -- 172.21.15.52:0/788431598 >> 172.21.15.52:0/788431598 conn(0x5635cd0e1630 msgr2=0x5635cd0d6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:15.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.122+0000 7efe76a23740 1 -- 172.21.15.52:0/788431598 shutdown_connections 2024-04-02T14:01:15.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.123+0000 7efe76a23740 1 -- 172.21.15.52:0/788431598 wait complete. 2024-04-02T14:01:15.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.123+0000 7efe76a23740 20 client.27020 trim_cache size 0 max 16384 2024-04-02T14:01:15.140 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:01:15.140 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "scrub active (2458 inodes in the stack)", 2024-04-02T14:01:15.140 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-04-02T14:01:15.140 INFO:teuthology.orchestra.run.smithi052.stdout: "693910b7-7799-4909-b9fc-85035ff4ce06": { 2024-04-02T14:01:15.141 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-04-02T14:01:15.141 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "693910b7-7799-4909-b9fc-85035ff4ce06", 2024-04-02T14:01:15.141 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-04-02T14:01:15.141 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:01:15.141 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:01:15.141 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:01:15.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.123+0000 7efe73e3c640 1 -- 172.21.15.52:0/517206859 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efe580a3dd0 msgr2=0x7efe580a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:15.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.123+0000 7efe73e3c640 1 --2- 172.21.15.52:0/517206859 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efe580a3dd0 0x7efe580a6290 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efe64009fd0 tx=0x7efe64004ab0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.123+0000 7efe73e3c640 1 -- 172.21.15.52:0/517206859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe6c0696c0 msgr2=0x7efe6c100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:15.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.123+0000 7efe73e3c640 1 --2- 172.21.15.52:0/517206859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe6c0696c0 0x7efe6c100f00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009230 tx=0x7efe5c004270 comp rx=0 tx=0).stop 2024-04-02T14:01:15.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.123+0000 7efe73e3c640 1 -- 172.21.15.52:0/517206859 shutdown_connections 2024-04-02T14:01:15.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.123+0000 7efe73e3c640 1 --2- 172.21.15.52:0/517206859 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efe580a3dd0 0x7efe580a6290 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.123+0000 7efe73e3c640 1 --2- 172.21.15.52:0/517206859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe6c107b50 0x7efe6c101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.123+0000 7efe73e3c640 1 --2- 172.21.15.52:0/517206859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe6c0696c0 0x7efe6c100f00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.123+0000 7efe73e3c640 1 --2- 172.21.15.52:0/517206859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe6c068d00 0x7efe6c1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:15.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.123+0000 7efe73e3c640 1 -- 172.21.15.52:0/517206859 >> 172.21.15.52:0/517206859 conn(0x7efe6c077b00 msgr2=0x7efe6c0fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:15.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.123+0000 7efe73e3c640 1 -- 172.21.15.52:0/517206859 shutdown_connections 2024-04-02T14:01:15.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:15.123+0000 7efe73e3c640 1 -- 172.21.15.52:0/517206859 wait complete. 2024-04-02T14:01:15.144 INFO:tasks.cephfs.filesystem:scrub status for tag:693910b7-7799-4909-b9fc-85035ff4ce06 - {'path': '/', 'tag': '693910b7-7799-4909-b9fc-85035ff4ce06', 'options': 'recursive,force'} 2024-04-02T14:01:15.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:15 smithi189 ceph-mon[36683]: pgmap v238: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:01:15.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:15 smithi189 ceph-mon[36683]: scrub queued for path: / 2024-04-02T14:01:15.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:15 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:01:15.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:15 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T14:01:15.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:15 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T14:01:15.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:15 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:01:15.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:15 smithi189 ceph-mon[36683]: osdmap e97: 12 total, 12 up, 12 in 2024-04-02T14:01:15.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:15 smithi052 ceph-mon[32104]: pgmap v238: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:01:15.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:15 smithi052 ceph-mon[32104]: scrub queued for path: / 2024-04-02T14:01:15.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:15 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:01:15.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:15 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T14:01:15.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:15 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T14:01:15.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:15 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:01:15.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:15 smithi052 ceph-mon[32104]: osdmap e97: 12 total, 12 up, 12 in 2024-04-02T14:01:15.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:15 smithi088 ceph-mon[36528]: pgmap v238: 129 pgs: 129 active+clean; 73 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:01:15.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:15 smithi088 ceph-mon[36528]: scrub queued for path: / 2024-04-02T14:01:15.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:15 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:01:15.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:15 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T14:01:15.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:15 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T14:01:15.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:15 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:01:15.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:15 smithi088 ceph-mon[36528]: osdmap e97: 12 total, 12 up, 12 in 2024-04-02T14:01:16.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:16 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T14:01:16.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:16 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:01:16.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:16 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:01:16.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:16 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T14:01:16.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:16 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:01:16.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:16 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:01:16.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:16 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T14:01:16.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:16 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:01:16.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:16 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:01:17.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:17 smithi189 ceph-mon[36683]: pgmap v240: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-02T14:01:17.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:17 smithi052 ceph-mon[32104]: pgmap v240: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-02T14:01:17.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:17 smithi088 ceph-mon[36528]: pgmap v240: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-02T14:01:19.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:19 smithi052 ceph-mon[32104]: pgmap v241: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-02T14:01:19.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:19 smithi088 ceph-mon[36528]: pgmap v241: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-02T14:01:20.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:19 smithi189 ceph-mon[36683]: pgmap v241: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-02T14:01:21.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:21 smithi052 ceph-mon[32104]: pgmap v242: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 215 B/s wr, 11 op/s 2024-04-02T14:01:21.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:21 smithi088 ceph-mon[36528]: pgmap v242: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 215 B/s wr, 11 op/s 2024-04-02T14:01:22.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:21 smithi189 ceph-mon[36683]: pgmap v242: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 215 B/s wr, 11 op/s 2024-04-02T14:01:23.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:23 smithi052 ceph-mon[32104]: pgmap v243: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:01:23.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:23 smithi088 ceph-mon[36528]: pgmap v243: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:01:24.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:23 smithi189 ceph-mon[36683]: pgmap v243: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:01:25.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:25 smithi052 ceph-mon[32104]: pgmap v244: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:01:25.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:25 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T14:01:25.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:25 smithi088 ceph-mon[36528]: pgmap v244: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:01:25.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:25 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T14:01:26.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:25 smithi189 ceph-mon[36683]: pgmap v244: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:01:26.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:25 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T14:01:26.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:26 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T14:01:26.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:26 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T14:01:27.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:26 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T14:01:27.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:27 smithi052 ceph-mon[32104]: pgmap v245: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 96 B/s wr, 14 op/s 2024-04-02T14:01:27.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:27 smithi088 ceph-mon[36528]: pgmap v245: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 96 B/s wr, 14 op/s 2024-04-02T14:01:28.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:27 smithi189 ceph-mon[36683]: pgmap v245: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 96 B/s wr, 14 op/s 2024-04-02T14:01:29.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:29 smithi052 ceph-mon[32104]: pgmap v246: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:29.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:29 smithi088 ceph-mon[36528]: pgmap v246: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:30.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:29 smithi189 ceph-mon[36683]: pgmap v246: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:31.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:31 smithi052 ceph-mon[32104]: pgmap v247: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:31.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:31 smithi088 ceph-mon[36528]: pgmap v247: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:32.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:31 smithi189 ceph-mon[36683]: pgmap v247: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:33.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:33 smithi052 ceph-mon[32104]: pgmap v248: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:01:33.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:33 smithi088 ceph-mon[36528]: pgmap v248: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:01:34.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:33 smithi189 ceph-mon[36683]: pgmap v248: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:01:35.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:35 smithi052 ceph-mon[32104]: pgmap v249: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:35.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:35 smithi088 ceph-mon[36528]: pgmap v249: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:36.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:35 smithi189 ceph-mon[36683]: pgmap v249: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:37.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:37 smithi189 ceph-mon[36683]: pgmap v250: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:01:37.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:37 smithi052 ceph-mon[32104]: pgmap v250: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:01:37.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:37 smithi088 ceph-mon[36528]: pgmap v250: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:01:39.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:39 smithi052 ceph-mon[32104]: pgmap v251: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:39.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:39 smithi088 ceph-mon[36528]: pgmap v251: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:40.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:39 smithi189 ceph-mon[36683]: pgmap v251: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:41.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:41 smithi052 ceph-mon[32104]: pgmap v252: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:41.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:41 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:01:41.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:41 smithi088 ceph-mon[36528]: pgmap v252: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:41.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:41 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:01:42.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:41 smithi189 ceph-mon[36683]: pgmap v252: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:42.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:41 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:01:42.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:42 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:01:42.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:42 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:01:42.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:42 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:01:42.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:42 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:01:42.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:42 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:01:42.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:42 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:01:43.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:42 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:01:43.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:42 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:01:43.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:42 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:01:43.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:43 smithi052 ceph-mon[32104]: pgmap v253: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:01:43.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:43 smithi088 ceph-mon[36528]: pgmap v253: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:01:44.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:43 smithi189 ceph-mon[36683]: pgmap v253: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:01:45.146 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:01:45.146 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:01:45.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.262+0000 7f9458655640 1 -- 172.21.15.52:0/2266110885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9450107b50 msgr2=0x7f9450107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.262+0000 7f9458655640 1 --2- 172.21.15.52:0/2266110885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9450107b50 0x7f9450107f30 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f9448009930 tx=0x7f944802bc30 comp rx=0 tx=0).stop 2024-04-02T14:01:45.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.263+0000 7f9458655640 1 -- 172.21.15.52:0/2266110885 shutdown_connections 2024-04-02T14:01:45.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.263+0000 7f9458655640 1 --2- 172.21.15.52:0/2266110885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94500696c0 0x7f945010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.263+0000 7f9458655640 1 --2- 172.21.15.52:0/2266110885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9450068d00 0x7f9450069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.263+0000 7f9458655640 1 --2- 172.21.15.52:0/2266110885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9450107b50 0x7f9450107f30 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.263+0000 7f9458655640 1 -- 172.21.15.52:0/2266110885 >> 172.21.15.52:0/2266110885 conn(0x7f9450077b00 msgr2=0x7f9450077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:45.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.263+0000 7f9458655640 1 -- 172.21.15.52:0/2266110885 shutdown_connections 2024-04-02T14:01:45.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.263+0000 7f9458655640 1 -- 172.21.15.52:0/2266110885 wait complete. 2024-04-02T14:01:45.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.263+0000 7f9458655640 1 Processor -- start 2024-04-02T14:01:45.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.263+0000 7f9458655640 1 -- start start 2024-04-02T14:01:45.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.263+0000 7f9458655640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9450068d00 0x7f9450198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.263+0000 7f9458655640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94500696c0 0x7f94501988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.264+0000 7f9458655640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9450107b50 0x7f945019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.264+0000 7f9458655640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9450111e70 con 0x7f9450107b50 2024-04-02T14:01:45.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.264+0000 7f9458655640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9450111cf0 con 0x7f9450068d00 2024-04-02T14:01:45.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.264+0000 7f9458655640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9450111ff0 con 0x7f94500696c0 2024-04-02T14:01:45.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.264+0000 7f9456bf2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9450107b50 0x7f945019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.264+0000 7f9456bf2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9450107b50 0x7f945019cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60146/0 (socket says 172.21.15.52:60146) 2024-04-02T14:01:45.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.264+0000 7f9456bf2640 1 -- 172.21.15.52:0/2315928712 learned_addr learned my addr 172.21.15.52:0/2315928712 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:45.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.264+0000 7f94563f1640 1 --2- 172.21.15.52:0/2315928712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9450068d00 0x7f9450198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.264+0000 7f9455bf0640 1 --2- 172.21.15.52:0/2315928712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94500696c0 0x7f94501988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.264+0000 7f9456bf2640 1 -- 172.21.15.52:0/2315928712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94500696c0 msgr2=0x7f94501988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.264+0000 7f9456bf2640 1 --2- 172.21.15.52:0/2315928712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94500696c0 0x7f94501988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.264+0000 7f9456bf2640 1 -- 172.21.15.52:0/2315928712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9450068d00 msgr2=0x7f9450198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.264+0000 7f9456bf2640 1 --2- 172.21.15.52:0/2315928712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9450068d00 0x7f9450198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.264+0000 7f9456bf2640 1 -- 172.21.15.52:0/2315928712 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945019d360 con 0x7f9450107b50 2024-04-02T14:01:45.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.264+0000 7f9456bf2640 1 --2- 172.21.15.52:0/2315928712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9450107b50 0x7f945019cc30 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f9440009fd0 tx=0x7f944000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:45.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.264+0000 7f944f7fe640 1 -- 172.21.15.52:0/2315928712 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440017070 con 0x7f9450107b50 2024-04-02T14:01:45.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.264+0000 7f9458655640 1 -- 172.21.15.52:0/2315928712 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f945019d650 con 0x7f9450107b50 2024-04-02T14:01:45.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.264+0000 7f944f7fe640 1 -- 172.21.15.52:0/2315928712 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9440003e50 con 0x7f9450107b50 2024-04-02T14:01:45.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.264+0000 7f944f7fe640 1 -- 172.21.15.52:0/2315928712 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94400106c0 con 0x7f9450107b50 2024-04-02T14:01:45.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.264+0000 7f9458655640 1 -- 172.21.15.52:0/2315928712 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94501a4ee0 con 0x7f9450107b50 2024-04-02T14:01:45.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.267+0000 7f944f7fe640 1 -- 172.21.15.52:0/2315928712 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9440003970 con 0x7f9450107b50 2024-04-02T14:01:45.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.268+0000 7f945b242740 1 Processor -- start 2024-04-02T14:01:45.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.268+0000 7f945b242740 1 -- start start 2024-04-02T14:01:45.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.268+0000 7f945b242740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347c8d0 0x556c1347ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.268+0000 7f945b242740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c1347d1f0 0x556c13485890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.268+0000 7f945b242740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c13485dd0 0x556c134881c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.268+0000 7f945b242740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556c133e0830 con 0x556c1347c8d0 2024-04-02T14:01:45.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.268+0000 7f945b242740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556c13388820 con 0x556c13485dd0 2024-04-02T14:01:45.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.268+0000 7f945b242740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556c133dc7e0 con 0x556c1347d1f0 2024-04-02T14:01:45.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.268+0000 7f9455bf0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347c8d0 0x556c1347ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.268+0000 7f9455bf0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347c8d0 0x556c1347ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60156/0 (socket says 172.21.15.52:60156) 2024-04-02T14:01:45.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.268+0000 7f9455bf0640 1 -- 172.21.15.52:0/3011247675 learned_addr learned my addr 172.21.15.52:0/3011247675 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:45.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.268+0000 7f944f7fe640 1 --2- 172.21.15.52:0/2315928712 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f94380a3d20 0x7f94380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.268+0000 7f94563f1640 1 --2- 172.21.15.52:0/3011247675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c1347d1f0 0x556c13485890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.268+0000 7f9455bf0640 1 --2- 172.21.15.52:0/3011247675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c13485dd0 0x556c134881c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.268+0000 7f9456bf2640 1 --2- 172.21.15.52:0/2315928712 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f94380a3d20 0x7f94380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.269+0000 7f9455bf0640 1 -- 172.21.15.52:0/3011247675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c1347d1f0 msgr2=0x556c13485890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:45.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.269+0000 7f9455bf0640 1 --2- 172.21.15.52:0/3011247675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c1347d1f0 0x556c13485890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.269+0000 7f9455bf0640 1 -- 172.21.15.52:0/3011247675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c13485dd0 msgr2=0x556c134881c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:45.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.269+0000 7f9455bf0640 1 --2- 172.21.15.52:0/3011247675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c13485dd0 0x556c134881c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.269+0000 7f9455bf0640 1 -- 172.21.15.52:0/3011247675 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c131be7d0 con 0x556c1347c8d0 2024-04-02T14:01:45.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.269+0000 7f944f7fe640 1 -- 172.21.15.52:0/2315928712 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f94400d0800 con 0x7f9450107b50 2024-04-02T14:01:45.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.269+0000 7f9455bf0640 1 --2- 172.21.15.52:0/3011247675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347c8d0 0x556c1347ccb0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f9444012630 tx=0x7f944400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e921767b8356d7c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:45.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.269+0000 7f9456bf2640 1 --2- 172.21.15.52:0/2315928712 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f94380a3d20 0x7f94380a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f944009c450 tx=0x7f9440089040 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:45.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.269+0000 7f944cbfe640 1 -- 172.21.15.52:0/3011247675 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444005020 con 0x556c1347c8d0 2024-04-02T14:01:45.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.269+0000 7f944cbfe640 1 -- 172.21.15.52:0/3011247675 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94440056a0 con 0x556c1347c8d0 2024-04-02T14:01:45.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.269+0000 7f944cbfe640 1 -- 172.21.15.52:0/3011247675 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944400e770 con 0x556c1347c8d0 2024-04-02T14:01:45.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.270+0000 7f945b242740 1 -- 172.21.15.52:0/3011247675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347c8d0 msgr2=0x556c1347ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.270+0000 7f945b242740 1 --2- 172.21.15.52:0/3011247675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347c8d0 0x556c1347ccb0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f9444012630 tx=0x7f944400b8c0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.270+0000 7f945b242740 1 -- 172.21.15.52:0/3011247675 shutdown_connections 2024-04-02T14:01:45.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.270+0000 7f945b242740 1 --2- 172.21.15.52:0/3011247675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c13485dd0 0x556c134881c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.270+0000 7f945b242740 1 --2- 172.21.15.52:0/3011247675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c1347d1f0 0x556c13485890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.270+0000 7f945b242740 1 --2- 172.21.15.52:0/3011247675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347c8d0 0x556c1347ccb0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.270+0000 7f945b242740 1 -- 172.21.15.52:0/3011247675 >> 172.21.15.52:0/3011247675 conn(0x556c13477000 msgr2=0x556c134773e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:45.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.270+0000 7f945b242740 1 -- 172.21.15.52:0/3011247675 shutdown_connections 2024-04-02T14:01:45.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.270+0000 7f945b242740 1 -- 172.21.15.52:0/3011247675 wait complete. 2024-04-02T14:01:45.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.270+0000 7f945b242740 1 -- start start 2024-04-02T14:01:45.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.270+0000 7f945b242740 1 Processor -- start 2024-04-02T14:01:45.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.270+0000 7f945b242740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347c8d0 0x556c134855b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.271+0000 7f945b242740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c1347d1f0 0x556c13482940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.271+0000 7f945b242740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c13485dd0 0x556c13482e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.271+0000 7f945b242740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556c13488bc0 con 0x556c1347c8d0 2024-04-02T14:01:45.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.271+0000 7f945b242740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556c1339bd30 con 0x556c1347d1f0 2024-04-02T14:01:45.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.271+0000 7f945b242740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556c13488d40 con 0x556c13485dd0 2024-04-02T14:01:45.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.271+0000 7f945b242740 10 client.0 _ll_register_callbacks cb 0x556c1339b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:45.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.271+0000 7f9455bf0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347c8d0 0x556c134855b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.271+0000 7f9455bf0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347c8d0 0x556c134855b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60160/0 (socket says 172.21.15.52:60160) 2024-04-02T14:01:45.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.271+0000 7f9455bf0640 1 -- 172.21.15.52:0/1406598395 learned_addr learned my addr 172.21.15.52:0/1406598395 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:45.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.271+0000 7f94563f1640 1 --2- 172.21.15.52:0/1406598395 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c1347d1f0 0x556c13482940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.271+0000 7f9455bf0640 1 --2- 172.21.15.52:0/1406598395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c13485dd0 0x556c13482e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.271+0000 7f9455bf0640 1 -- 172.21.15.52:0/1406598395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c13485dd0 msgr2=0x556c13482e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:45.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.271+0000 7f9455bf0640 1 --2- 172.21.15.52:0/1406598395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c13485dd0 0x556c13482e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.271+0000 7f9455bf0640 1 -- 172.21.15.52:0/1406598395 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c1347d1f0 msgr2=0x556c13482940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:45.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.271+0000 7f9455bf0640 1 --2- 172.21.15.52:0/1406598395 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c1347d1f0 0x556c13482940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.271+0000 7f9455bf0640 1 -- 172.21.15.52:0/1406598395 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c130f43f0 con 0x556c1347c8d0 2024-04-02T14:01:45.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.271+0000 7f9455bf0640 1 --2- 172.21.15.52:0/1406598395 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347c8d0 0x556c134855b0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f9444004db0 tx=0x7f944400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb6ba2f421b431c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:45.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.271+0000 7f9435ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:45.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.271+0000 7f9435ffb640 1 -- 172.21.15.52:0/1406598395 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444005050 con 0x556c1347c8d0 2024-04-02T14:01:45.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.271+0000 7f9435ffb640 1 -- 172.21.15.52:0/1406598395 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f94440051f0 con 0x556c1347c8d0 2024-04-02T14:01:45.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.271+0000 7f945b242740 1 -- 172.21.15.52:0/1406598395 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94440056a0 con 0x556c1347c8d0 2024-04-02T14:01:45.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.272+0000 7f9435ffb640 1 -- 172.21.15.52:0/1406598395 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9444036050 con 0x556c1347c8d0 2024-04-02T14:01:45.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.272+0000 7f945b242740 10 client.17904 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:45.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.272+0000 7f945b242740 1 -- 172.21.15.52:0/1406598395 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556c130b40f0 con 0x556c1347c8d0 2024-04-02T14:01:45.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.272+0000 7f9435ffb640 1 -- 172.21.15.52:0/1406598395 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94440056a0 con 0x556c1347c8d0 2024-04-02T14:01:45.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.272+0000 7f9435ffb640 1 -- 172.21.15.52:0/1406598395 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f94440304f0 con 0x556c1347c8d0 2024-04-02T14:01:45.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.272+0000 7f945b242740 10 client.17904 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:45.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.272+0000 7f945b242740 10 client.17904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:01:45.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.272+0000 7f945b242740 20 client.17904 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:45.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.272+0000 7f945b242740 1 --2- 172.21.15.52:0/1406598395 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556c13495b60 0x556c13497f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.272+0000 7f945b242740 4 client.17904 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:01:45.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.272+0000 7f945b242740 1 -- 172.21.15.52:0/1406598395 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c133edd70 con 0x556c13495b60 2024-04-02T14:01:45.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.272+0000 7f94563f1640 1 --2- 172.21.15.52:0/1406598395 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556c13495b60 0x556c13497f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.273+0000 7f94563f1640 1 --2- 172.21.15.52:0/1406598395 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556c13495b60 0x556c13497f50 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ab960951603cf60 server_cookie=e25204eb643d6736 in_seq=0 out_seq=0 2024-04-02T14:01:45.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.273+0000 7f9435ffb640 10 client.17904 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:45.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.273+0000 7f9435ffb640 1 -- 172.21.15.52:0/1406598395 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556c133edd70 con 0x556c13495b60 2024-04-02T14:01:45.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.273+0000 7f9435ffb640 10 client.17904 handle_command_reply: tid=0 2024-04-02T14:01:45.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.274+0000 7f945b242740 1 client.17904 shutdown 2024-04-02T14:01:45.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.274+0000 7f945b242740 1 -- 172.21.15.52:0/1406598395 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347c8d0 msgr2=0x556c134855b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.274+0000 7f945b242740 1 --2- 172.21.15.52:0/1406598395 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347c8d0 0x556c134855b0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f9444004db0 tx=0x7f944400b630 comp rx=0 tx=0).stop 2024-04-02T14:01:45.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.274+0000 7f945b242740 1 -- 172.21.15.52:0/1406598395 shutdown_connections 2024-04-02T14:01:45.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.274+0000 7f945b242740 1 --2- 172.21.15.52:0/1406598395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c13485dd0 0x556c13482e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.274+0000 7f945b242740 1 --2- 172.21.15.52:0/1406598395 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c1347d1f0 0x556c13482940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.274+0000 7f945b242740 1 --2- 172.21.15.52:0/1406598395 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556c13495b60 0x556c13497f50 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.274+0000 7f945b242740 1 --2- 172.21.15.52:0/1406598395 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347c8d0 0x556c134855b0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.274+0000 7f945b242740 1 -- 172.21.15.52:0/1406598395 >> 172.21.15.52:0/1406598395 conn(0x556c13477000 msgr2=0x556c13478d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:45.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.274+0000 7f9435ffb640 0 client.17904 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:45.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.274+0000 7f945b242740 1 -- 172.21.15.52:0/1406598395 shutdown_connections 2024-04-02T14:01:45.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.274+0000 7f945b242740 1 -- 172.21.15.52:0/1406598395 wait complete. 2024-04-02T14:01:45.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.274+0000 7f945b242740 20 client.17904 trim_cache size 0 max 16384 2024-04-02T14:01:45.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.291+0000 7f945b242740 1 Processor -- start 2024-04-02T14:01:45.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.291+0000 7f945b242740 1 -- start start 2024-04-02T14:01:45.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.292+0000 7f945b242740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c1347c8d0 0x556c13491ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.292+0000 7f945b242740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347d1f0 0x556c13476240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.292+0000 7f945b242740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c13485dd0 0x556c134820f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.292+0000 7f945b242740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f94501aef10 con 0x556c1347d1f0 2024-04-02T14:01:45.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.292+0000 7f945b242740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f94501b0a50 con 0x556c13485dd0 2024-04-02T14:01:45.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.292+0000 7f945b242740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f94501af5e0 con 0x556c1347c8d0 2024-04-02T14:01:45.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.292+0000 7f94563f1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347d1f0 0x556c13476240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.292+0000 7f94563f1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347d1f0 0x556c13476240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60164/0 (socket says 172.21.15.52:60164) 2024-04-02T14:01:45.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.292+0000 7f94563f1640 1 -- 172.21.15.52:0/1275175490 learned_addr learned my addr 172.21.15.52:0/1275175490 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:45.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.292+0000 7f9455bf0640 1 --2- 172.21.15.52:0/1275175490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c1347c8d0 0x556c13491ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.292+0000 7f94563f1640 1 -- 172.21.15.52:0/1275175490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c1347c8d0 msgr2=0x556c13491ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:45.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.292+0000 7f94563f1640 1 --2- 172.21.15.52:0/1275175490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c1347c8d0 0x556c13491ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.292+0000 7f94563f1640 1 -- 172.21.15.52:0/1275175490 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c13485dd0 msgr2=0x556c134820f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:45.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.292+0000 7f94563f1640 1 --2- 172.21.15.52:0/1275175490 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c13485dd0 0x556c134820f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.292+0000 7f94563f1640 1 -- 172.21.15.52:0/1275175490 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c13476e30 con 0x556c1347d1f0 2024-04-02T14:01:45.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.292+0000 7f9455bf0640 1 --2- 172.21.15.52:0/1275175490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c1347c8d0 0x556c13491ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:45.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.292+0000 7f94563f1640 1 --2- 172.21.15.52:0/1275175490 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347d1f0 0x556c13476240 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f9448005ab0 tx=0x7f944802e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267ee23e0e73cab5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:45.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.292+0000 7f9437fff640 1 -- 172.21.15.52:0/1275175490 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448004430 con 0x556c1347d1f0 2024-04-02T14:01:45.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.292+0000 7f9437fff640 1 -- 172.21.15.52:0/1275175490 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f944804f030 con 0x556c1347d1f0 2024-04-02T14:01:45.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.292+0000 7f9437fff640 1 -- 172.21.15.52:0/1275175490 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448004d30 con 0x556c1347d1f0 2024-04-02T14:01:45.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.293+0000 7f945b242740 1 -- 172.21.15.52:0/1275175490 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347d1f0 msgr2=0x556c13476240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.293+0000 7f945b242740 1 --2- 172.21.15.52:0/1275175490 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347d1f0 0x556c13476240 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f9448005ab0 tx=0x7f944802e040 comp rx=0 tx=0).stop 2024-04-02T14:01:45.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.293+0000 7f945b242740 1 -- 172.21.15.52:0/1275175490 shutdown_connections 2024-04-02T14:01:45.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.293+0000 7f945b242740 1 --2- 172.21.15.52:0/1275175490 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c13485dd0 0x556c134820f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.293+0000 7f945b242740 1 --2- 172.21.15.52:0/1275175490 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347d1f0 0x556c13476240 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.293+0000 7f945b242740 1 --2- 172.21.15.52:0/1275175490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c1347c8d0 0x556c13491ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.293+0000 7f945b242740 1 -- 172.21.15.52:0/1275175490 >> 172.21.15.52:0/1275175490 conn(0x556c13477000 msgr2=0x556c13484060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:45.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.293+0000 7f945b242740 1 -- 172.21.15.52:0/1275175490 shutdown_connections 2024-04-02T14:01:45.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.293+0000 7f945b242740 1 -- 172.21.15.52:0/1275175490 wait complete. 2024-04-02T14:01:45.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.293+0000 7f945b242740 1 -- start start 2024-04-02T14:01:45.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.293+0000 7f945b242740 1 Processor -- start 2024-04-02T14:01:45.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.293+0000 7f945b242740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347c8d0 0x556c1348e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.293+0000 7f945b242740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c1347d1f0 0x556c13489520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.293+0000 7f945b242740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c13485dd0 0x556c13489a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.293+0000 7f945b242740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f94501b0380 con 0x556c1347c8d0 2024-04-02T14:01:45.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.293+0000 7f945b242740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f94501afcb0 con 0x556c13485dd0 2024-04-02T14:01:45.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.293+0000 7f945b242740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f94501acf10 con 0x556c1347d1f0 2024-04-02T14:01:45.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.293+0000 7f945b242740 10 client.0 _ll_register_callbacks cb 0x7f94440120d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:45.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.294+0000 7f9455bf0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347c8d0 0x556c1348e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.294+0000 7f9455bf0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347c8d0 0x556c1348e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60166/0 (socket says 172.21.15.52:60166) 2024-04-02T14:01:45.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.294+0000 7f9455bf0640 1 -- 172.21.15.52:0/1428078415 learned_addr learned my addr 172.21.15.52:0/1428078415 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:45.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.294+0000 7f94563f1640 1 --2- 172.21.15.52:0/1428078415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c1347d1f0 0x556c13489520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.294+0000 7f9455bf0640 1 --2- 172.21.15.52:0/1428078415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c13485dd0 0x556c13489a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.294+0000 7f9455bf0640 1 -- 172.21.15.52:0/1428078415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c1347d1f0 msgr2=0x556c13489520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:45.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.294+0000 7f9455bf0640 1 --2- 172.21.15.52:0/1428078415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c1347d1f0 0x556c13489520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.294+0000 7f9455bf0640 1 -- 172.21.15.52:0/1428078415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c13485dd0 msgr2=0x556c13489a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:45.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.294+0000 7f9455bf0640 1 --2- 172.21.15.52:0/1428078415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c13485dd0 0x556c13489a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.294+0000 7f9455bf0640 1 -- 172.21.15.52:0/1428078415 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c13489fa0 con 0x556c1347c8d0 2024-04-02T14:01:45.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.294+0000 7f9455bf0640 1 --2- 172.21.15.52:0/1428078415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347c8d0 0x556c1348e030 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f9444012430 tx=0x7f9444012510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1720921c5a61f19 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:45.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.294+0000 7f9435ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:45.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.294+0000 7f9435ffb640 1 -- 172.21.15.52:0/1428078415 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444004220 con 0x556c1347c8d0 2024-04-02T14:01:45.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.294+0000 7f9435ffb640 1 -- 172.21.15.52:0/1428078415 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f94440043c0 con 0x556c1347c8d0 2024-04-02T14:01:45.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.294+0000 7f945b242740 1 -- 172.21.15.52:0/1428078415 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f944804f030 con 0x556c1347c8d0 2024-04-02T14:01:45.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.294+0000 7f9435ffb640 1 -- 172.21.15.52:0/1428078415 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9444004b90 con 0x556c1347c8d0 2024-04-02T14:01:45.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.294+0000 7f945b242740 10 client.17910 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:45.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.294+0000 7f945b242740 1 -- 172.21.15.52:0/1428078415 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556c134fff70 con 0x556c1347c8d0 2024-04-02T14:01:45.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.295+0000 7f9435ffb640 1 -- 172.21.15.52:0/1428078415 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f944804f030 con 0x556c1347c8d0 2024-04-02T14:01:45.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.295+0000 7f9435ffb640 1 -- 172.21.15.52:0/1428078415 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9444003860 con 0x556c1347c8d0 2024-04-02T14:01:45.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.295+0000 7f945b242740 10 client.17910 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:45.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.295+0000 7f945b242740 10 client.17910 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:01:45.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.295+0000 7f945b242740 20 client.17910 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:45.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.295+0000 7f945b242740 1 --2- 172.21.15.52:0/1428078415 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556c135005e0 0x556c135029d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.295+0000 7f945b242740 4 client.17910 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:01:45.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.295+0000 7f945b242740 1 -- 172.21.15.52:0/1428078415 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556c133e80c0 con 0x556c135005e0 2024-04-02T14:01:45.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.295+0000 7f94563f1640 1 --2- 172.21.15.52:0/1428078415 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556c135005e0 0x556c135029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.295+0000 7f9435ffb640 10 client.17910 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:45.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.295+0000 7f94563f1640 1 --2- 172.21.15.52:0/1428078415 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556c135005e0 0x556c135029d0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99b2386fa0680993 server_cookie=d5c58c6c61f988a1 in_seq=0 out_seq=0 2024-04-02T14:01:45.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.296+0000 7f9435ffb640 1 -- 172.21.15.52:0/1428078415 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556c133e80c0 con 0x556c135005e0 2024-04-02T14:01:45.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.296+0000 7f9435ffb640 10 client.17910 handle_command_reply: tid=0 2024-04-02T14:01:45.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.296+0000 7f945b242740 1 client.17910 shutdown 2024-04-02T14:01:45.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.296+0000 7f945b242740 1 -- 172.21.15.52:0/1428078415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347c8d0 msgr2=0x556c1348e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.296+0000 7f945b242740 1 --2- 172.21.15.52:0/1428078415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347c8d0 0x556c1348e030 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f9444012430 tx=0x7f9444012510 comp rx=0 tx=0).stop 2024-04-02T14:01:45.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.296+0000 7f945b242740 1 -- 172.21.15.52:0/1428078415 shutdown_connections 2024-04-02T14:01:45.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.296+0000 7f945b242740 1 --2- 172.21.15.52:0/1428078415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c13485dd0 0x556c13489a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.296+0000 7f945b242740 1 --2- 172.21.15.52:0/1428078415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c1347d1f0 0x556c13489520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.296+0000 7f945b242740 1 --2- 172.21.15.52:0/1428078415 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556c135005e0 0x556c135029d0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.296+0000 7f945b242740 1 --2- 172.21.15.52:0/1428078415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c1347c8d0 0x556c1348e030 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.296+0000 7f945b242740 1 -- 172.21.15.52:0/1428078415 >> 172.21.15.52:0/1428078415 conn(0x556c13477000 msgr2=0x556c134760c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:45.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.296+0000 7f9435ffb640 0 client.17910 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:45.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.296+0000 7f945b242740 1 -- 172.21.15.52:0/1428078415 shutdown_connections 2024-04-02T14:01:45.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.296+0000 7f945b242740 1 -- 172.21.15.52:0/1428078415 wait complete. 2024-04-02T14:01:45.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.296+0000 7f945b242740 20 client.17910 trim_cache size 0 max 16384 2024-04-02T14:01:45.311 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:01:45.312 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-04-02T14:01:45.312 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-04-02T14:01:45.312 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:01:45.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.296+0000 7f9458655640 1 -- 172.21.15.52:0/2315928712 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f94380a3d20 msgr2=0x7f94380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.296+0000 7f9458655640 1 --2- 172.21.15.52:0/2315928712 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f94380a3d20 0x7f94380a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f944009c450 tx=0x7f9440089040 comp rx=0 tx=0).stop 2024-04-02T14:01:45.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.296+0000 7f9458655640 1 -- 172.21.15.52:0/2315928712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9450107b50 msgr2=0x7f945019cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.296+0000 7f9458655640 1 --2- 172.21.15.52:0/2315928712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9450107b50 0x7f945019cc30 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f9440009fd0 tx=0x7f944000b260 comp rx=0 tx=0).stop 2024-04-02T14:01:45.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.296+0000 7f9458655640 1 -- 172.21.15.52:0/2315928712 shutdown_connections 2024-04-02T14:01:45.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.296+0000 7f9458655640 1 --2- 172.21.15.52:0/2315928712 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f94380a3d20 0x7f94380a61e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.296+0000 7f9458655640 1 --2- 172.21.15.52:0/2315928712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9450107b50 0x7f945019cc30 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.296+0000 7f9458655640 1 --2- 172.21.15.52:0/2315928712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94500696c0 0x7f94501988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.296+0000 7f9458655640 1 --2- 172.21.15.52:0/2315928712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9450068d00 0x7f9450198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.296+0000 7f9458655640 1 -- 172.21.15.52:0/2315928712 >> 172.21.15.52:0/2315928712 conn(0x7f9450077b00 msgr2=0x7f94500fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:45.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.297+0000 7f9458655640 1 -- 172.21.15.52:0/2315928712 shutdown_connections 2024-04-02T14:01:45.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.297+0000 7f9458655640 1 -- 172.21.15.52:0/2315928712 wait complete. 2024-04-02T14:01:45.315 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T14:01:45.315 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:01:45.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.416+0000 7f54d5433640 1 -- 172.21.15.52:0/1314471515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54d0102450 msgr2=0x7f54d010e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.416+0000 7f54d5433640 1 --2- 172.21.15.52:0/1314471515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54d0102450 0x7f54d010e960 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009cd0 tx=0x7f54c802c0f0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.416+0000 7f54d5433640 1 -- 172.21.15.52:0/1314471515 shutdown_connections 2024-04-02T14:01:45.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.416+0000 7f54d5433640 1 --2- 172.21.15.52:0/1314471515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54d0102450 0x7f54d010e960 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.416+0000 7f54d5433640 1 --2- 172.21.15.52:0/1314471515 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54d0101a90 0x7f54d0101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.416+0000 7f54d5433640 1 --2- 172.21.15.52:0/1314471515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54d0107aa0 0x7f54d0107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.416+0000 7f54d5433640 1 -- 172.21.15.52:0/1314471515 >> 172.21.15.52:0/1314471515 conn(0x7f54d00fd5a0 msgr2=0x7f54d00ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:45.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.416+0000 7f54d5433640 1 -- 172.21.15.52:0/1314471515 shutdown_connections 2024-04-02T14:01:45.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.416+0000 7f54d5433640 1 -- 172.21.15.52:0/1314471515 wait complete. 2024-04-02T14:01:45.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.417+0000 7f54d5433640 1 Processor -- start 2024-04-02T14:01:45.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.417+0000 7f54d5433640 1 -- start start 2024-04-02T14:01:45.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.417+0000 7f54d5433640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54d0101a90 0x7f54d0198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.417+0000 7f54d5433640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54d0102450 0x7f54d01988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.417+0000 7f54d5433640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54d0107aa0 0x7f54d019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.417+0000 7f54d5433640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f54d0111eb0 con 0x7f54d0107aa0 2024-04-02T14:01:45.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.417+0000 7f54d5433640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f54d0111d30 con 0x7f54d0102450 2024-04-02T14:01:45.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.417+0000 7f54d5433640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f54d0112030 con 0x7f54d0101a90 2024-04-02T14:01:45.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.417+0000 7f54cf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54d0107aa0 0x7f54d019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.417+0000 7f54cf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54d0107aa0 0x7f54d019cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60180/0 (socket says 172.21.15.52:60180) 2024-04-02T14:01:45.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.417+0000 7f54cf7fe640 1 -- 172.21.15.52:0/207356327 learned_addr learned my addr 172.21.15.52:0/207356327 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:45.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.417+0000 7f54ceffd640 1 --2- 172.21.15.52:0/207356327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54d0101a90 0x7f54d0198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.417+0000 7f54c7fff640 1 --2- 172.21.15.52:0/207356327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54d0102450 0x7f54d01988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.417+0000 7f54cf7fe640 1 -- 172.21.15.52:0/207356327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54d0101a90 msgr2=0x7f54d0198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.417+0000 7f54cf7fe640 1 --2- 172.21.15.52:0/207356327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54d0101a90 0x7f54d0198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.417+0000 7f54cf7fe640 1 -- 172.21.15.52:0/207356327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54d0102450 msgr2=0x7f54d01988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.417+0000 7f54cf7fe640 1 --2- 172.21.15.52:0/207356327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54d0102450 0x7f54d01988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.417+0000 7f54cf7fe640 1 -- 172.21.15.52:0/207356327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d019d360 con 0x7f54d0107aa0 2024-04-02T14:01:45.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.417+0000 7f54cf7fe640 1 --2- 172.21.15.52:0/207356327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54d0107aa0 0x7f54d019cc30 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f54c8004af0 tx=0x7f54c8032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:45.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.418+0000 7f54ccff9640 1 -- 172.21.15.52:0/207356327 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8041020 con 0x7f54d0107aa0 2024-04-02T14:01:45.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.418+0000 7f54ccff9640 1 -- 172.21.15.52:0/207356327 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54c8034cd0 con 0x7f54d0107aa0 2024-04-02T14:01:45.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.418+0000 7f54ccff9640 1 -- 172.21.15.52:0/207356327 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8039040 con 0x7f54d0107aa0 2024-04-02T14:01:45.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.418+0000 7f54d5433640 1 -- 172.21.15.52:0/207356327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d019d5f0 con 0x7f54d0107aa0 2024-04-02T14:01:45.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.418+0000 7f54d5433640 1 -- 172.21.15.52:0/207356327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d01a4ee0 con 0x7f54d0107aa0 2024-04-02T14:01:45.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.419+0000 7f54d5433640 1 -- 172.21.15.52:0/207356327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54d0068d40 con 0x7f54d0107aa0 2024-04-02T14:01:45.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.420+0000 7f54ccff9640 1 -- 172.21.15.52:0/207356327 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f54c8044020 con 0x7f54d0107aa0 2024-04-02T14:01:45.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.420+0000 7f54ccff9640 1 --2- 172.21.15.52:0/207356327 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f54bc0a3d20 0x7f54bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.420+0000 7f54ccff9640 1 -- 172.21.15.52:0/207356327 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f54c80f2c10 con 0x7f54d0107aa0 2024-04-02T14:01:45.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.420+0000 7f54ceffd640 1 --2- 172.21.15.52:0/207356327 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f54bc0a3d20 0x7f54bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.421+0000 7f54ceffd640 1 --2- 172.21.15.52:0/207356327 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f54bc0a3d20 0x7f54bc0a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f54c000bb60 tx=0x7f54c000b390 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.424+0000 7f54ccff9640 1 -- 172.21.15.52:0/207356327 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f54c804b020 con 0x7f54d0107aa0 2024-04-02T14:01:45.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.651+0000 7f54d5433640 1 -- 172.21.15.52:0/207356327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54d010a050 con 0x7f54d0107aa0 2024-04-02T14:01:45.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.652+0000 7f54ccff9640 1 -- 172.21.15.52:0/207356327 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11923 (secure 0 0 0) 0x7f54c80bf030 con 0x7f54d0107aa0 2024-04-02T14:01:45.656 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T14:01:45.656 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:57:45.240902+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24467,24457,24490,24475,24503]},"id":1}]} 2024-04-02T14:01:45.656 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-04-02T14:01:45.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.654+0000 7f54d5433640 1 -- 172.21.15.52:0/207356327 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f54bc0a3d20 msgr2=0x7f54bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.654+0000 7f54d5433640 1 --2- 172.21.15.52:0/207356327 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f54bc0a3d20 0x7f54bc0a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f54c000bb60 tx=0x7f54c000b390 comp rx=0 tx=0).stop 2024-04-02T14:01:45.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.654+0000 7f54d5433640 1 -- 172.21.15.52:0/207356327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54d0107aa0 msgr2=0x7f54d019cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.654+0000 7f54d5433640 1 --2- 172.21.15.52:0/207356327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54d0107aa0 0x7f54d019cc30 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f54c8004af0 tx=0x7f54c8032ec0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.654+0000 7f54d5433640 1 -- 172.21.15.52:0/207356327 shutdown_connections 2024-04-02T14:01:45.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.654+0000 7f54d5433640 1 --2- 172.21.15.52:0/207356327 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f54bc0a3d20 0x7f54bc0a61e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.654+0000 7f54d5433640 1 --2- 172.21.15.52:0/207356327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54d0107aa0 0x7f54d019cc30 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.654+0000 7f54d5433640 1 --2- 172.21.15.52:0/207356327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54d0102450 0x7f54d01988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.654+0000 7f54d5433640 1 --2- 172.21.15.52:0/207356327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54d0101a90 0x7f54d0198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.654+0000 7f54d5433640 1 -- 172.21.15.52:0/207356327 >> 172.21.15.52:0/207356327 conn(0x7f54d00fd5a0 msgr2=0x7f54d00fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:45.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.654+0000 7f54d5433640 1 -- 172.21.15.52:0/207356327 shutdown_connections 2024-04-02T14:01:45.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.654+0000 7f54d5433640 1 -- 172.21.15.52:0/207356327 wait complete. 2024-04-02T14:01:45.669 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T14:01:45.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.770+0000 7fc281ab1640 1 -- 172.21.15.52:0/1201585708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc27c06a190 msgr2=0x7fc27c06a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.770+0000 7fc281ab1640 1 --2- 172.21.15.52:0/1201585708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc27c06a190 0x7fc27c06a610 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009930 tx=0x7fc26c02bc30 comp rx=0 tx=0).stop 2024-04-02T14:01:45.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.770+0000 7fc281ab1640 1 -- 172.21.15.52:0/1201585708 shutdown_connections 2024-04-02T14:01:45.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.770+0000 7fc281ab1640 1 --2- 172.21.15.52:0/1201585708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc27c06ab50 0x7fc27c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.770+0000 7fc281ab1640 1 --2- 172.21.15.52:0/1201585708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc27c06a190 0x7fc27c06a610 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.770+0000 7fc281ab1640 1 --2- 172.21.15.52:0/1201585708 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc27c107aa0 0x7fc27c107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.770+0000 7fc281ab1640 1 -- 172.21.15.52:0/1201585708 >> 172.21.15.52:0/1201585708 conn(0x7fc27c077ad0 msgr2=0x7fc27c077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:45.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.770+0000 7fc281ab1640 1 -- 172.21.15.52:0/1201585708 shutdown_connections 2024-04-02T14:01:45.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.770+0000 7fc281ab1640 1 -- 172.21.15.52:0/1201585708 wait complete. 2024-04-02T14:01:45.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.771+0000 7fc281ab1640 1 Processor -- start 2024-04-02T14:01:45.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.771+0000 7fc281ab1640 1 -- start start 2024-04-02T14:01:45.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.771+0000 7fc281ab1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc27c06a190 0x7fc27c1009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.771+0000 7fc281ab1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc27c06ab50 0x7fc27c100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.771+0000 7fc281ab1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc27c107aa0 0x7fc27c101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.771+0000 7fc281ab1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c111ed0 con 0x7fc27c06a190 2024-04-02T14:01:45.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.771+0000 7fc281ab1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c111d50 con 0x7fc27c107aa0 2024-04-02T14:01:45.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.771+0000 7fc281ab1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c112050 con 0x7fc27c06ab50 2024-04-02T14:01:45.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.771+0000 7fc27b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc27c06a190 0x7fc27c1009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.771+0000 7fc27b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc27c06a190 0x7fc27c1009e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60206/0 (socket says 172.21.15.52:60206) 2024-04-02T14:01:45.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.771+0000 7fc27b7fe640 1 -- 172.21.15.52:0/3804205324 learned_addr learned my addr 172.21.15.52:0/3804205324 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:45.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.771+0000 7fc27bfff640 1 --2- 172.21.15.52:0/3804205324 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc27c107aa0 0x7fc27c101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.771+0000 7fc27b7fe640 1 -- 172.21.15.52:0/3804205324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc27c06ab50 msgr2=0x7fc27c100f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.771+0000 7fc27b7fe640 1 --2- 172.21.15.52:0/3804205324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc27c06ab50 0x7fc27c100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.771+0000 7fc27b7fe640 1 -- 172.21.15.52:0/3804205324 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc27c107aa0 msgr2=0x7fc27c101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.771+0000 7fc27b7fe640 1 --2- 172.21.15.52:0/3804205324 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc27c107aa0 0x7fc27c101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.771+0000 7fc27b7fe640 1 -- 172.21.15.52:0/3804205324 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27c1a6930 con 0x7fc27c06a190 2024-04-02T14:01:45.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.772+0000 7fc27b7fe640 1 --2- 172.21.15.52:0/3804205324 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc27c06a190 0x7fc27c1009e0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fc27000b4a0 tx=0x7fc27000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:45.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.772+0000 7fc278ff9640 1 -- 172.21.15.52:0/3804205324 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc270014070 con 0x7fc27c06a190 2024-04-02T14:01:45.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.772+0000 7fc281ab1640 1 -- 172.21.15.52:0/3804205324 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc27c1a6c20 con 0x7fc27c06a190 2024-04-02T14:01:45.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.772+0000 7fc278ff9640 1 -- 172.21.15.52:0/3804205324 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc270010af0 con 0x7fc27c06a190 2024-04-02T14:01:45.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.772+0000 7fc278ff9640 1 -- 172.21.15.52:0/3804205324 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc270017d10 con 0x7fc27c06a190 2024-04-02T14:01:45.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.772+0000 7fc281ab1640 1 -- 172.21.15.52:0/3804205324 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc27c1a7170 con 0x7fc27c06a190 2024-04-02T14:01:45.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.774+0000 7fc278ff9640 1 -- 172.21.15.52:0/3804205324 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc270017070 con 0x7fc27c06a190 2024-04-02T14:01:45.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.774+0000 7fc284775740 1 Processor -- start 2024-04-02T14:01:45.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.774+0000 7fc284775740 1 -- start start 2024-04-02T14:01:45.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.774+0000 7fc284775740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562412aab7a0 0x562412aabb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.774+0000 7fc284775740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562412aac0c0 0x562412ab4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.774+0000 7fc284775740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412ab4ca0 0x562412ab7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.774+0000 7fc284775740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562412a0f7b0 con 0x562412ab4ca0 2024-04-02T14:01:45.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.774+0000 7fc284775740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5624129ae570 con 0x562412aab7a0 2024-04-02T14:01:45.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.774+0000 7fc284775740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562412a0b760 con 0x562412aac0c0 2024-04-02T14:01:45.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.775+0000 7fc27affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412ab4ca0 0x562412ab7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.775+0000 7fc27affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412ab4ca0 0x562412ab7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60222/0 (socket says 172.21.15.52:60222) 2024-04-02T14:01:45.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.775+0000 7fc27affd640 1 -- 172.21.15.52:0/1277164556 learned_addr learned my addr 172.21.15.52:0/1277164556 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:45.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.775+0000 7fc278ff9640 1 --2- 172.21.15.52:0/3804205324 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc2640a3d20 0x7fc2640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.775+0000 7fc27bfff640 1 --2- 172.21.15.52:0/1277164556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562412aab7a0 0x562412aabb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.775+0000 7fc27affd640 1 -- 172.21.15.52:0/1277164556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562412aac0c0 msgr2=0x562412ab4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:45.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.775+0000 7fc278ff9640 1 -- 172.21.15.52:0/3804205324 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7fc2700cfe00 con 0x7fc27c06a190 2024-04-02T14:01:45.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.775+0000 7fc27b7fe640 1 --2- 172.21.15.52:0/1277164556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562412aac0c0 0x562412ab4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.775+0000 7fc27affd640 1 --2- 172.21.15.52:0/1277164556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562412aac0c0 0x562412ab4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.775+0000 7fc27affd640 1 -- 172.21.15.52:0/1277164556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562412aab7a0 msgr2=0x562412aabb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:45.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.775+0000 7fc27affd640 1 --2- 172.21.15.52:0/1277164556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562412aab7a0 0x562412aabb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.775+0000 7fc27affd640 1 -- 172.21.15.52:0/1277164556 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624127233f0 con 0x562412ab4ca0 2024-04-02T14:01:45.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.775+0000 7fc27bfff640 1 --2- 172.21.15.52:0/1277164556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562412aab7a0 0x562412aabb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:45.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.775+0000 7fc27b7fe640 1 --2- 172.21.15.52:0/1277164556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562412aac0c0 0x562412ab4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:45.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.775+0000 7fc27bfff640 1 --2- 172.21.15.52:0/3804205324 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc2640a3d20 0x7fc2640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.775+0000 7fc27affd640 1 --2- 172.21.15.52:0/1277164556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412ab4ca0 0x562412ab7090 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fc26c036040 tx=0x7fc26c005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277125e2cd1886ba server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:45.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.775+0000 7fc261ffb640 1 -- 172.21.15.52:0/1277164556 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c041020 con 0x562412ab4ca0 2024-04-02T14:01:45.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.775+0000 7fc27bfff640 1 --2- 172.21.15.52:0/3804205324 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc2640a3d20 0x7fc2640a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc2740098b0 tx=0x7fc274004a40 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:45.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.775+0000 7fc261ffb640 1 -- 172.21.15.52:0/1277164556 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc26c004040 con 0x562412ab4ca0 2024-04-02T14:01:45.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.775+0000 7fc261ffb640 1 -- 172.21.15.52:0/1277164556 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c003b50 con 0x562412ab4ca0 2024-04-02T14:01:45.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.776+0000 7fc284775740 1 -- 172.21.15.52:0/1277164556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412ab4ca0 msgr2=0x562412ab7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.776+0000 7fc284775740 1 --2- 172.21.15.52:0/1277164556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412ab4ca0 0x562412ab7090 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fc26c036040 tx=0x7fc26c005760 comp rx=0 tx=0).stop 2024-04-02T14:01:45.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.776+0000 7fc284775740 1 -- 172.21.15.52:0/1277164556 shutdown_connections 2024-04-02T14:01:45.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.776+0000 7fc284775740 1 --2- 172.21.15.52:0/1277164556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412ab4ca0 0x562412ab7090 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.776+0000 7fc284775740 1 --2- 172.21.15.52:0/1277164556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562412aac0c0 0x562412ab4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.776+0000 7fc284775740 1 --2- 172.21.15.52:0/1277164556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562412aab7a0 0x562412aabb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.776+0000 7fc284775740 1 -- 172.21.15.52:0/1277164556 >> 172.21.15.52:0/1277164556 conn(0x562412aa5ed0 msgr2=0x562412aa62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:45.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.776+0000 7fc284775740 1 -- 172.21.15.52:0/1277164556 shutdown_connections 2024-04-02T14:01:45.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.776+0000 7fc284775740 1 -- 172.21.15.52:0/1277164556 wait complete. 2024-04-02T14:01:45.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.776+0000 7fc284775740 1 -- start start 2024-04-02T14:01:45.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.777+0000 7fc284775740 1 Processor -- start 2024-04-02T14:01:45.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.777+0000 7fc284775740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562412aab7a0 0x562412ab4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.777+0000 7fc284775740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562412aac0c0 0x562412ab1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.777+0000 7fc284775740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412ab4ca0 0x562412ab2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.777+0000 7fc284775740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562412ab7a30 con 0x562412ab4ca0 2024-04-02T14:01:45.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.777+0000 7fc284775740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5624129cae80 con 0x562412aac0c0 2024-04-02T14:01:45.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.777+0000 7fc284775740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562412ab7bb0 con 0x562412aab7a0 2024-04-02T14:01:45.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.777+0000 7fc284775740 10 client.0 _ll_register_callbacks cb 0x562412a06aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:45.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.777+0000 7fc27bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412ab4ca0 0x562412ab2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.777+0000 7fc27bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412ab4ca0 0x562412ab2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60224/0 (socket says 172.21.15.52:60224) 2024-04-02T14:01:45.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.777+0000 7fc27bfff640 1 -- 172.21.15.52:0/2692905945 learned_addr learned my addr 172.21.15.52:0/2692905945 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:45.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.777+0000 7fc27affd640 1 --2- 172.21.15.52:0/2692905945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562412aac0c0 0x562412ab1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.777+0000 7fc27b7fe640 1 --2- 172.21.15.52:0/2692905945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562412aab7a0 0x562412ab4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.777+0000 7fc27bfff640 1 -- 172.21.15.52:0/2692905945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562412aab7a0 msgr2=0x562412ab4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:45.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.777+0000 7fc27bfff640 1 --2- 172.21.15.52:0/2692905945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562412aab7a0 0x562412ab4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.777+0000 7fc27bfff640 1 -- 172.21.15.52:0/2692905945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562412aac0c0 msgr2=0x562412ab1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:45.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.777+0000 7fc27bfff640 1 --2- 172.21.15.52:0/2692905945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562412aac0c0 0x562412ab1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.777+0000 7fc27bfff640 1 -- 172.21.15.52:0/2692905945 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624127ed7d0 con 0x562412ab4ca0 2024-04-02T14:01:45.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.777+0000 7fc27affd640 1 --2- 172.21.15.52:0/2692905945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562412aac0c0 0x562412ab1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:45.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.777+0000 7fc27bfff640 1 --2- 172.21.15.52:0/2692905945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412ab4ca0 0x562412ab2030 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fc274013960 tx=0x7fc274004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3fab69291e927c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:45.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.777+0000 7fc25affd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:45.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.778+0000 7fc25affd640 1 -- 172.21.15.52:0/2692905945 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27401ed70 con 0x562412ab4ca0 2024-04-02T14:01:45.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.778+0000 7fc25affd640 1 -- 172.21.15.52:0/2692905945 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7fc274028470 con 0x562412ab4ca0 2024-04-02T14:01:45.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.778+0000 7fc284775740 1 -- 172.21.15.52:0/2692905945 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc26c004040 con 0x562412ab4ca0 2024-04-02T14:01:45.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.778+0000 7fc25affd640 1 -- 172.21.15.52:0/2692905945 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc274031050 con 0x562412ab4ca0 2024-04-02T14:01:45.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.778+0000 7fc284775740 10 client.17928 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:45.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.778+0000 7fc284775740 1 -- 172.21.15.52:0/2692905945 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562412ac3a50 con 0x562412ab4ca0 2024-04-02T14:01:45.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.778+0000 7fc25affd640 1 -- 172.21.15.52:0/2692905945 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc26c004040 con 0x562412ab4ca0 2024-04-02T14:01:45.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.778+0000 7fc25affd640 1 -- 172.21.15.52:0/2692905945 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc27400f040 con 0x562412ab4ca0 2024-04-02T14:01:45.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.778+0000 7fc284775740 10 client.17928 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:45.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.778+0000 7fc284775740 10 client.17928 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T14:01:45.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.778+0000 7fc284775740 20 client.17928 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:45.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.778+0000 7fc284775740 1 --2- 172.21.15.52:0/2692905945 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x562412ac4f10 0x562412ac7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.778+0000 7fc284775740 4 client.17928 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:01:45.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.778+0000 7fc284775740 1 -- 172.21.15.52:0/2692905945 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562412a1ccf0 con 0x562412ac4f10 2024-04-02T14:01:45.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.779+0000 7fc27b7fe640 1 --2- 172.21.15.52:0/2692905945 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x562412ac4f10 0x562412ac7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.779+0000 7fc27b7fe640 1 --2- 172.21.15.52:0/2692905945 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x562412ac4f10 0x562412ac7360 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da348f9178ac64df server_cookie=a61de49f50a42a8b in_seq=0 out_seq=0 2024-04-02T14:01:45.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.779+0000 7fc25affd640 10 client.17928 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T14:01:45.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.780+0000 7fc25affd640 1 -- 172.21.15.52:0/2692905945 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562412a1ccf0 con 0x562412ac4f10 2024-04-02T14:01:45.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.780+0000 7fc25affd640 10 client.17928 handle_command_reply: tid=0 2024-04-02T14:01:45.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.780+0000 7fc284775740 1 client.17928 shutdown 2024-04-02T14:01:45.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.780+0000 7fc284775740 1 -- 172.21.15.52:0/2692905945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412ab4ca0 msgr2=0x562412ab2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.780+0000 7fc284775740 1 --2- 172.21.15.52:0/2692905945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412ab4ca0 0x562412ab2030 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fc274013960 tx=0x7fc274004ab0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.780+0000 7fc284775740 1 -- 172.21.15.52:0/2692905945 shutdown_connections 2024-04-02T14:01:45.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.780+0000 7fc284775740 1 --2- 172.21.15.52:0/2692905945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412ab4ca0 0x562412ab2030 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.780+0000 7fc284775740 1 --2- 172.21.15.52:0/2692905945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562412aac0c0 0x562412ab1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.780+0000 7fc284775740 1 --2- 172.21.15.52:0/2692905945 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x562412ac4f10 0x562412ac7360 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.780+0000 7fc284775740 1 --2- 172.21.15.52:0/2692905945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562412aab7a0 0x562412ab4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.780+0000 7fc25affd640 0 client.17928 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T14:01:45.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.780+0000 7fc284775740 1 -- 172.21.15.52:0/2692905945 >> 172.21.15.52:0/2692905945 conn(0x562412aa5ed0 msgr2=0x562412aa7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:45.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.780+0000 7fc284775740 1 -- 172.21.15.52:0/2692905945 shutdown_connections 2024-04-02T14:01:45.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.780+0000 7fc284775740 1 -- 172.21.15.52:0/2692905945 wait complete. 2024-04-02T14:01:45.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.780+0000 7fc284775740 20 client.17928 trim_cache size 0 max 16384 2024-04-02T14:01:45.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.793+0000 7fc284775740 1 Processor -- start 2024-04-02T14:01:45.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.793+0000 7fc284775740 1 -- start start 2024-04-02T14:01:45.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.793+0000 7fc284775740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562412aab7a0 0x562412ac0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.793+0000 7fc284775740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412aac0c0 0x562412ab4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.793+0000 7fc284775740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562412ab4ca0 0x562412ab4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.793+0000 7fc284775740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c1b11b0 con 0x562412aac0c0 2024-04-02T14:01:45.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.793+0000 7fc284775740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c1b2cf0 con 0x562412ab4ca0 2024-04-02T14:01:45.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.793+0000 7fc284775740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c1b1880 con 0x562412aab7a0 2024-04-02T14:01:45.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.793+0000 7fc27affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412aac0c0 0x562412ab4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.794+0000 7fc27affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412aac0c0 0x562412ab4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60236/0 (socket says 172.21.15.52:60236) 2024-04-02T14:01:45.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.794+0000 7fc27affd640 1 -- 172.21.15.52:0/1983003962 learned_addr learned my addr 172.21.15.52:0/1983003962 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:45.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.794+0000 7fc27b7fe640 1 --2- 172.21.15.52:0/1983003962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562412aab7a0 0x562412ac0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.794+0000 7fc27bfff640 1 --2- 172.21.15.52:0/1983003962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562412ab4ca0 0x562412ab4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.794+0000 7fc27affd640 1 -- 172.21.15.52:0/1983003962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562412aab7a0 msgr2=0x562412ac0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:45.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.794+0000 7fc27affd640 1 --2- 172.21.15.52:0/1983003962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562412aab7a0 0x562412ac0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.794+0000 7fc27affd640 1 -- 172.21.15.52:0/1983003962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562412ab4ca0 msgr2=0x562412ab4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:45.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.794+0000 7fc27affd640 1 --2- 172.21.15.52:0/1983003962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562412ab4ca0 0x562412ab4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.794+0000 7fc27affd640 1 -- 172.21.15.52:0/1983003962 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562412aad610 con 0x562412aac0c0 2024-04-02T14:01:45.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.794+0000 7fc27bfff640 1 --2- 172.21.15.52:0/1983003962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562412ab4ca0 0x562412ab4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:45.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.794+0000 7fc27affd640 1 --2- 172.21.15.52:0/1983003962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412aac0c0 0x562412ab4150 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x562412aa7060 tx=0x7fc26c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a131b65b0c936c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:45.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.794+0000 7fc2617fa640 1 -- 172.21.15.52:0/1983003962 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c041030 con 0x562412aac0c0 2024-04-02T14:01:45.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.794+0000 7fc2617fa640 1 -- 172.21.15.52:0/1983003962 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc26c04a020 con 0x562412aac0c0 2024-04-02T14:01:45.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.794+0000 7fc2617fa640 1 -- 172.21.15.52:0/1983003962 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c003880 con 0x562412aac0c0 2024-04-02T14:01:45.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.795+0000 7fc284775740 1 -- 172.21.15.52:0/1983003962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412aac0c0 msgr2=0x562412ab4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.795+0000 7fc284775740 1 --2- 172.21.15.52:0/1983003962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412aac0c0 0x562412ab4150 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x562412aa7060 tx=0x7fc26c005bd0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.795+0000 7fc284775740 1 -- 172.21.15.52:0/1983003962 shutdown_connections 2024-04-02T14:01:45.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.795+0000 7fc284775740 1 --2- 172.21.15.52:0/1983003962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562412ab4ca0 0x562412ab4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.795+0000 7fc284775740 1 --2- 172.21.15.52:0/1983003962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412aac0c0 0x562412ab4150 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.795+0000 7fc284775740 1 --2- 172.21.15.52:0/1983003962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562412aab7a0 0x562412ac0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.795+0000 7fc284775740 1 -- 172.21.15.52:0/1983003962 >> 172.21.15.52:0/1983003962 conn(0x562412aa5ed0 msgr2=0x562412ab19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:45.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.795+0000 7fc284775740 1 -- 172.21.15.52:0/1983003962 shutdown_connections 2024-04-02T14:01:45.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.795+0000 7fc284775740 1 -- 172.21.15.52:0/1983003962 wait complete. 2024-04-02T14:01:45.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.795+0000 7fc284775740 1 -- start start 2024-04-02T14:01:45.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.795+0000 7fc284775740 1 Processor -- start 2024-04-02T14:01:45.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc284775740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562412aab7a0 0x562412abd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc284775740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412aac0c0 0x562412aaeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc284775740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562412ab4ca0 0x562412aaf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc284775740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c1b2620 con 0x562412aac0c0 2024-04-02T14:01:45.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc284775740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c1b1f50 con 0x562412aab7a0 2024-04-02T14:01:45.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc284775740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c1af1b0 con 0x562412ab4ca0 2024-04-02T14:01:45.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc284775740 10 client.0 _ll_register_callbacks cb 0x7fc274009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:45.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc27affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412aac0c0 0x562412aaeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc27affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412aac0c0 0x562412aaeca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60244/0 (socket says 172.21.15.52:60244) 2024-04-02T14:01:45.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc27affd640 1 -- 172.21.15.52:0/3619520996 learned_addr learned my addr 172.21.15.52:0/3619520996 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:45.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc27b7fe640 1 --2- 172.21.15.52:0/3619520996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562412aab7a0 0x562412abd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc27bfff640 1 --2- 172.21.15.52:0/3619520996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562412ab4ca0 0x562412aaf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc27affd640 1 -- 172.21.15.52:0/3619520996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562412ab4ca0 msgr2=0x562412aaf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:45.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc27affd640 1 --2- 172.21.15.52:0/3619520996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562412ab4ca0 0x562412aaf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc27affd640 1 -- 172.21.15.52:0/3619520996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562412aab7a0 msgr2=0x562412abd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:45.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc27affd640 1 --2- 172.21.15.52:0/3619520996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562412aab7a0 0x562412abd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc27affd640 1 -- 172.21.15.52:0/3619520996 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562412aaf850 con 0x562412aac0c0 2024-04-02T14:01:45.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc27b7fe640 1 --2- 172.21.15.52:0/3619520996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562412aab7a0 0x562412abd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:45.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc27bfff640 1 --2- 172.21.15.52:0/3619520996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562412ab4ca0 0x562412aaf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:45.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc27affd640 1 --2- 172.21.15.52:0/3619520996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412aac0c0 0x562412aaeca0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fc26c02c170 tx=0x7fc26c004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a425ba3ec3cff0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:45.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc25affd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:45.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc25affd640 1 -- 172.21.15.52:0/3619520996 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c053070 con 0x562412aac0c0 2024-04-02T14:01:45.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc25affd640 1 -- 172.21.15.52:0/3619520996 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7fc26c003870 con 0x562412aac0c0 2024-04-02T14:01:45.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc284775740 1 -- 172.21.15.52:0/3619520996 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc26c04a020 con 0x562412aac0c0 2024-04-02T14:01:45.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc25affd640 1 -- 172.21.15.52:0/3619520996 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc26c05f050 con 0x562412aac0c0 2024-04-02T14:01:45.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc284775740 10 client.17934 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:45.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.796+0000 7fc284775740 1 -- 172.21.15.52:0/3619520996 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562412ab64c0 con 0x562412aac0c0 2024-04-02T14:01:45.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.797+0000 7fc25affd640 1 -- 172.21.15.52:0/3619520996 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc26c04a020 con 0x562412aac0c0 2024-04-02T14:01:45.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.797+0000 7fc25affd640 1 -- 172.21.15.52:0/3619520996 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc26c03da00 con 0x562412aac0c0 2024-04-02T14:01:45.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.797+0000 7fc284775740 10 client.17934 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:45.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.797+0000 7fc284775740 10 client.17934 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T14:01:45.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.797+0000 7fc284775740 20 client.17934 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:45.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.797+0000 7fc284775740 1 --2- 172.21.15.52:0/3619520996 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x562412b2b5b0 0x562412b2d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.797+0000 7fc284775740 4 client.17934 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:01:45.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.797+0000 7fc284775740 1 -- 172.21.15.52:0/3619520996 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562412a17040 con 0x562412b2b5b0 2024-04-02T14:01:45.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.797+0000 7fc27b7fe640 1 --2- 172.21.15.52:0/3619520996 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x562412b2b5b0 0x562412b2d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.797+0000 7fc27b7fe640 1 --2- 172.21.15.52:0/3619520996 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x562412b2b5b0 0x562412b2d9a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=807cde0e09bba963 server_cookie=72fe1cb14624de02 in_seq=0 out_seq=0 2024-04-02T14:01:45.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.797+0000 7fc25affd640 10 client.17934 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T14:01:45.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.798+0000 7fc25affd640 1 -- 172.21.15.52:0/3619520996 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562412a17040 con 0x562412b2b5b0 2024-04-02T14:01:45.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.798+0000 7fc25affd640 10 client.17934 handle_command_reply: tid=0 2024-04-02T14:01:45.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.798+0000 7fc284775740 1 client.17934 shutdown 2024-04-02T14:01:45.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.798+0000 7fc284775740 1 -- 172.21.15.52:0/3619520996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412aac0c0 msgr2=0x562412aaeca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.798+0000 7fc284775740 1 --2- 172.21.15.52:0/3619520996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412aac0c0 0x562412aaeca0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fc26c02c170 tx=0x7fc26c004280 comp rx=0 tx=0).stop 2024-04-02T14:01:45.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.798+0000 7fc284775740 1 -- 172.21.15.52:0/3619520996 shutdown_connections 2024-04-02T14:01:45.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.798+0000 7fc284775740 1 --2- 172.21.15.52:0/3619520996 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x562412b2b5b0 0x562412b2d9a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.798+0000 7fc284775740 1 --2- 172.21.15.52:0/3619520996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562412ab4ca0 0x562412aaf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.798+0000 7fc284775740 1 --2- 172.21.15.52:0/3619520996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562412aac0c0 0x562412aaeca0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.798+0000 7fc25affd640 0 client.17934 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T14:01:45.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.798+0000 7fc284775740 1 --2- 172.21.15.52:0/3619520996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562412aab7a0 0x562412abd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.798+0000 7fc284775740 1 -- 172.21.15.52:0/3619520996 >> 172.21.15.52:0/3619520996 conn(0x562412aa5ed0 msgr2=0x562412ab0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:45.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.798+0000 7fc284775740 1 -- 172.21.15.52:0/3619520996 shutdown_connections 2024-04-02T14:01:45.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.798+0000 7fc284775740 1 -- 172.21.15.52:0/3619520996 wait complete. 2024-04-02T14:01:45.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.798+0000 7fc284775740 20 client.17934 trim_cache size 0 max 16384 2024-04-02T14:01:45.815 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:01:45.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.798+0000 7fc281ab1640 1 -- 172.21.15.52:0/3804205324 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc2640a3d20 msgr2=0x7fc2640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.798+0000 7fc281ab1640 1 --2- 172.21.15.52:0/3804205324 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc2640a3d20 0x7fc2640a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc2740098b0 tx=0x7fc274004a40 comp rx=0 tx=0).stop 2024-04-02T14:01:45.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.798+0000 7fc281ab1640 1 -- 172.21.15.52:0/3804205324 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc27c06a190 msgr2=0x7fc27c1009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.798+0000 7fc281ab1640 1 --2- 172.21.15.52:0/3804205324 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc27c06a190 0x7fc27c1009e0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fc27000b4a0 tx=0x7fc27000b960 comp rx=0 tx=0).stop 2024-04-02T14:01:45.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.798+0000 7fc281ab1640 1 -- 172.21.15.52:0/3804205324 shutdown_connections 2024-04-02T14:01:45.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.798+0000 7fc281ab1640 1 --2- 172.21.15.52:0/3804205324 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc2640a3d20 0x7fc2640a61e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.798+0000 7fc281ab1640 1 --2- 172.21.15.52:0/3804205324 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc27c107aa0 0x7fc27c101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.798+0000 7fc281ab1640 1 --2- 172.21.15.52:0/3804205324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc27c06ab50 0x7fc27c100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.798+0000 7fc281ab1640 1 --2- 172.21.15.52:0/3804205324 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc27c06a190 0x7fc27c1009e0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.798+0000 7fc281ab1640 1 -- 172.21.15.52:0/3804205324 >> 172.21.15.52:0/3804205324 conn(0x7fc27c077ad0 msgr2=0x7fc27c0fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:45.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.799+0000 7fc281ab1640 1 -- 172.21.15.52:0/3804205324 shutdown_connections 2024-04-02T14:01:45.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.799+0000 7fc281ab1640 1 -- 172.21.15.52:0/3804205324 wait complete. 2024-04-02T14:01:45.818 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T14:01:45.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:45 smithi052 ceph-mon[32104]: pgmap v254: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:45.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:45 smithi088 ceph-mon[36528]: pgmap v254: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:45.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.919+0000 7f8b51fcd640 1 -- 172.21.15.52:0/2974630377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b4c1069c0 msgr2=0x7f8b4c106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.919+0000 7f8b51fcd640 1 --2- 172.21.15.52:0/2974630377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b4c1069c0 0x7f8b4c106da0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c009930 tx=0x7f8b3c02bc30 comp rx=0 tx=0).stop 2024-04-02T14:01:45.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.919+0000 7f8b51fcd640 1 -- 172.21.15.52:0/2974630377 shutdown_connections 2024-04-02T14:01:45.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.919+0000 7f8b51fcd640 1 --2- 172.21.15.52:0/2974630377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b4c101370 0x7f8b4c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.919+0000 7f8b51fcd640 1 --2- 172.21.15.52:0/2974630377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b4c1009b0 0x7f8b4c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.919+0000 7f8b51fcd640 1 --2- 172.21.15.52:0/2974630377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b4c1069c0 0x7f8b4c106da0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.919+0000 7f8b51fcd640 1 -- 172.21.15.52:0/2974630377 >> 172.21.15.52:0/2974630377 conn(0x7f8b4c076d80 msgr2=0x7f8b4c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:45.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.919+0000 7f8b51fcd640 1 -- 172.21.15.52:0/2974630377 shutdown_connections 2024-04-02T14:01:45.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.919+0000 7f8b51fcd640 1 -- 172.21.15.52:0/2974630377 wait complete. 2024-04-02T14:01:45.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.919+0000 7f8b51fcd640 1 Processor -- start 2024-04-02T14:01:45.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.919+0000 7f8b51fcd640 1 -- start start 2024-04-02T14:01:45.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.920+0000 7f8b51fcd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b4c1009b0 0x7f8b4c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.920+0000 7f8b51fcd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b4c101370 0x7f8b4c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.920+0000 7f8b51fcd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b4c1069c0 0x7f8b4c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.920+0000 7f8b51fcd640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c110dd0 con 0x7f8b4c1009b0 2024-04-02T14:01:45.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.920+0000 7f8b51fcd640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c110c50 con 0x7f8b4c101370 2024-04-02T14:01:45.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.920+0000 7f8b51fcd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c110f50 con 0x7f8b4c1069c0 2024-04-02T14:01:45.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.920+0000 7f8b4b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b4c1009b0 0x7f8b4c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.920+0000 7f8b4b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b4c1009b0 0x7f8b4c1983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60270/0 (socket says 172.21.15.52:60270) 2024-04-02T14:01:45.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.920+0000 7f8b4b7fe640 1 -- 172.21.15.52:0/338018641 learned_addr learned my addr 172.21.15.52:0/338018641 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:45.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.920+0000 7f8b4bfff640 1 --2- 172.21.15.52:0/338018641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b4c1069c0 0x7f8b4c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.920+0000 7f8b4affd640 1 --2- 172.21.15.52:0/338018641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b4c101370 0x7f8b4c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.920+0000 7f8b4b7fe640 1 -- 172.21.15.52:0/338018641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b4c1069c0 msgr2=0x7f8b4c19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.920+0000 7f8b4b7fe640 1 --2- 172.21.15.52:0/338018641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b4c1069c0 0x7f8b4c19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.920+0000 7f8b4b7fe640 1 -- 172.21.15.52:0/338018641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b4c101370 msgr2=0x7f8b4c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.920+0000 7f8b4b7fe640 1 --2- 172.21.15.52:0/338018641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b4c101370 0x7f8b4c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.920+0000 7f8b4b7fe640 1 -- 172.21.15.52:0/338018641 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4c19d3a0 con 0x7f8b4c1009b0 2024-04-02T14:01:45.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.920+0000 7f8b4bfff640 1 --2- 172.21.15.52:0/338018641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b4c1069c0 0x7f8b4c19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:45.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.920+0000 7f8b4affd640 1 --2- 172.21.15.52:0/338018641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b4c101370 0x7f8b4c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:45.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.920+0000 7f8b4b7fe640 1 --2- 172.21.15.52:0/338018641 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b4c1009b0 0x7f8b4c1983b0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c009230 tx=0x7f8b3c02cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:45.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.920+0000 7f8b48ff9640 1 -- 172.21.15.52:0/338018641 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c041070 con 0x7f8b4c1009b0 2024-04-02T14:01:45.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.920+0000 7f8b51fcd640 1 -- 172.21.15.52:0/338018641 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b4c19d630 con 0x7f8b4c1009b0 2024-04-02T14:01:45.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.920+0000 7f8b48ff9640 1 -- 172.21.15.52:0/338018641 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b3c004450 con 0x7f8b4c1009b0 2024-04-02T14:01:45.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.920+0000 7f8b48ff9640 1 -- 172.21.15.52:0/338018641 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c034440 con 0x7f8b4c1009b0 2024-04-02T14:01:45.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.921+0000 7f8b51fcd640 1 -- 172.21.15.52:0/338018641 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b4c1a4f20 con 0x7f8b4c1009b0 2024-04-02T14:01:45.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.923+0000 7f8b48ff9640 1 -- 172.21.15.52:0/338018641 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8b3c044070 con 0x7f8b4c1009b0 2024-04-02T14:01:45.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.923+0000 7f8b54bb3740 1 Processor -- start 2024-04-02T14:01:45.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.923+0000 7f8b54bb3740 1 -- start start 2024-04-02T14:01:45.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.923+0000 7f8b54bb3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588e5aaf7a0 0x5588e5aafb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.923+0000 7f8b48ff9640 1 --2- 172.21.15.52:0/338018641 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f8b340a3d20 0x7f8b340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.923+0000 7f8b54bb3740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 0x5588e5ab8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.923+0000 7f8b54bb3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588e5ab8ca0 0x5588e5abb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.923+0000 7f8b54bb3740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5588e5a137b0 con 0x5588e5ab00c0 2024-04-02T14:01:45.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.923+0000 7f8b54bb3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5588e59b2570 con 0x5588e5ab8ca0 2024-04-02T14:01:45.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.923+0000 7f8b54bb3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5588e5a0f760 con 0x5588e5aaf7a0 2024-04-02T14:01:45.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.923+0000 7f8b4affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 0x5588e5ab8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.923+0000 7f8b4b7fe640 1 --2- 172.21.15.52:0/338018641 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f8b340a3d20 0x7f8b340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.923+0000 7f8b4affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 0x5588e5ab8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60278/0 (socket says 172.21.15.52:60278) 2024-04-02T14:01:45.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.924+0000 7f8b4affd640 1 -- 172.21.15.52:0/2846424821 learned_addr learned my addr 172.21.15.52:0/2846424821 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:45.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.924+0000 7f8b48ff9640 1 -- 172.21.15.52:0/338018641 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f8b3c0f2600 con 0x7f8b4c1009b0 2024-04-02T14:01:45.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.924+0000 7f8b4bfff640 1 --2- 172.21.15.52:0/2846424821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588e5aaf7a0 0x5588e5aafb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.924+0000 7f8b4bfff640 1 --2- 172.21.15.52:0/2846424821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588e5ab8ca0 0x5588e5abb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.924+0000 7f8b4affd640 1 -- 172.21.15.52:0/2846424821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588e5aaf7a0 msgr2=0x5588e5aafb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:45.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.924+0000 7f8b4affd640 1 --2- 172.21.15.52:0/2846424821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588e5aaf7a0 0x5588e5aafb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.924+0000 7f8b4affd640 1 -- 172.21.15.52:0/2846424821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588e5ab8ca0 msgr2=0x5588e5abb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:45.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.924+0000 7f8b4affd640 1 --2- 172.21.15.52:0/2846424821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588e5ab8ca0 0x5588e5abb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.924+0000 7f8b4affd640 1 -- 172.21.15.52:0/2846424821 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588e57273f0 con 0x5588e5ab00c0 2024-04-02T14:01:45.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.924+0000 7f8b4bfff640 1 --2- 172.21.15.52:0/2846424821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588e5aaf7a0 0x5588e5aafb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:45.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.924+0000 7f8b4affd640 1 --2- 172.21.15.52:0/2846424821 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 0x5588e5ab8760 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009c20 tx=0x7f8b4000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486a118cb5a548fd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:45.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.924+0000 7f8b4b7fe640 1 --2- 172.21.15.52:0/338018641 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f8b340a3d20 0x7f8b340a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0effc0 tx=0x7f8b3c000bc0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:45.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.924+0000 7f8b31ffb640 1 -- 172.21.15.52:0/2846424821 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40016070 con 0x5588e5ab00c0 2024-04-02T14:01:45.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.924+0000 7f8b31ffb640 1 -- 172.21.15.52:0/2846424821 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b40011950 con 0x5588e5ab00c0 2024-04-02T14:01:45.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.924+0000 7f8b31ffb640 1 -- 172.21.15.52:0/2846424821 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40010910 con 0x5588e5ab00c0 2024-04-02T14:01:45.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.925+0000 7f8b54bb3740 1 -- 172.21.15.52:0/2846424821 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 msgr2=0x5588e5ab8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.925+0000 7f8b54bb3740 1 --2- 172.21.15.52:0/2846424821 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 0x5588e5ab8760 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009c20 tx=0x7f8b4000e6c0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.925+0000 7f8b54bb3740 1 -- 172.21.15.52:0/2846424821 shutdown_connections 2024-04-02T14:01:45.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.925+0000 7f8b54bb3740 1 --2- 172.21.15.52:0/2846424821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588e5ab8ca0 0x5588e5abb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.925+0000 7f8b54bb3740 1 --2- 172.21.15.52:0/2846424821 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 0x5588e5ab8760 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.925+0000 7f8b54bb3740 1 --2- 172.21.15.52:0/2846424821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588e5aaf7a0 0x5588e5aafb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.925+0000 7f8b54bb3740 1 -- 172.21.15.52:0/2846424821 >> 172.21.15.52:0/2846424821 conn(0x5588e5aa9ed0 msgr2=0x5588e5aaa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:45.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.925+0000 7f8b54bb3740 1 -- 172.21.15.52:0/2846424821 shutdown_connections 2024-04-02T14:01:45.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.925+0000 7f8b54bb3740 1 -- 172.21.15.52:0/2846424821 wait complete. 2024-04-02T14:01:45.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.925+0000 7f8b54bb3740 1 -- start start 2024-04-02T14:01:45.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.925+0000 7f8b54bb3740 1 Processor -- start 2024-04-02T14:01:45.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.926+0000 7f8b54bb3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588e5aaf7a0 0x5588e5ab5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.926+0000 7f8b54bb3740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 0x5588e5ab6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.926+0000 7f8b54bb3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588e5ab8ca0 0x5588e5ab65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.926+0000 7f8b54bb3740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5588e5abba90 con 0x5588e5ab00c0 2024-04-02T14:01:45.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.926+0000 7f8b54bb3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5588e59cee80 con 0x5588e5ab8ca0 2024-04-02T14:01:45.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.926+0000 7f8b54bb3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5588e5abbc10 con 0x5588e5aaf7a0 2024-04-02T14:01:45.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.926+0000 7f8b54bb3740 10 client.0 _ll_register_callbacks cb 0x5588e5a0aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:45.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.926+0000 7f8b4affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 0x5588e5ab6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.926+0000 7f8b4affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 0x5588e5ab6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60290/0 (socket says 172.21.15.52:60290) 2024-04-02T14:01:45.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.926+0000 7f8b4affd640 1 -- 172.21.15.52:0/1425899598 learned_addr learned my addr 172.21.15.52:0/1425899598 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:45.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.926+0000 7f8b4bfff640 1 --2- 172.21.15.52:0/1425899598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588e5aaf7a0 0x5588e5ab5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.926+0000 7f8b4affd640 1 -- 172.21.15.52:0/1425899598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588e5aaf7a0 msgr2=0x5588e5ab5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:45.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.926+0000 7f8b4affd640 1 --2- 172.21.15.52:0/1425899598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588e5aaf7a0 0x5588e5ab5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.926+0000 7f8b4affd640 1 -- 172.21.15.52:0/1425899598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588e5ab8ca0 msgr2=0x5588e5ab65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:45.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.926+0000 7f8b4affd640 1 --2- 172.21.15.52:0/1425899598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588e5ab8ca0 0x5588e5ab65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.926+0000 7f8b4affd640 1 -- 172.21.15.52:0/1425899598 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588e57f17d0 con 0x5588e5ab00c0 2024-04-02T14:01:45.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.926+0000 7f8b4bfff640 1 --2- 172.21.15.52:0/1425899598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588e5aaf7a0 0x5588e5ab5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:45.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.926+0000 7f8b4affd640 1 --2- 172.21.15.52:0/1425899598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 0x5588e5ab6070 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f8b400104b0 tx=0x7f8b40010510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987cc514eced1f07 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:45.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.926+0000 7f8b2affd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:45.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.926+0000 7f8b2affd640 1 -- 172.21.15.52:0/1425899598 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40027020 con 0x5588e5ab00c0 2024-04-02T14:01:45.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.926+0000 7f8b2affd640 1 -- 172.21.15.52:0/1425899598 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f8b40004020 con 0x5588e5ab00c0 2024-04-02T14:01:45.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.926+0000 7f8b54bb3740 1 -- 172.21.15.52:0/1425899598 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b40011950 con 0x5588e5ab00c0 2024-04-02T14:01:45.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.926+0000 7f8b2affd640 1 -- 172.21.15.52:0/1425899598 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8b4002f050 con 0x5588e5ab00c0 2024-04-02T14:01:45.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.927+0000 7f8b54bb3740 10 client.17946 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:45.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.927+0000 7f8b54bb3740 1 -- 172.21.15.52:0/1425899598 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5588e5ac7bc0 con 0x5588e5ab00c0 2024-04-02T14:01:45.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.927+0000 7f8b2affd640 1 -- 172.21.15.52:0/1425899598 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b40011950 con 0x5588e5ab00c0 2024-04-02T14:01:45.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.927+0000 7f8b2affd640 1 -- 172.21.15.52:0/1425899598 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8b4000f040 con 0x5588e5ab00c0 2024-04-02T14:01:45.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.927+0000 7f8b54bb3740 10 client.17946 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:45.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.927+0000 7f8b54bb3740 10 client.17946 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T14:01:45.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.927+0000 7f8b54bb3740 20 client.17946 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:45.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.927+0000 7f8b54bb3740 1 --2- 172.21.15.52:0/1425899598 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5588e5ac9020 0x5588e5acb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.927+0000 7f8b54bb3740 4 client.17946 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:01:45.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.927+0000 7f8b54bb3740 1 -- 172.21.15.52:0/1425899598 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588e5a20cf0 con 0x5588e5ac9020 2024-04-02T14:01:45.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.927+0000 7f8b4bfff640 1 --2- 172.21.15.52:0/1425899598 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5588e5ac9020 0x5588e5acb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.928+0000 7f8b2affd640 10 client.17946 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T14:01:45.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.928+0000 7f8b4bfff640 1 --2- 172.21.15.52:0/1425899598 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5588e5ac9020 0x5588e5acb410 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4452ffb738d9dc94 server_cookie=668f4c0ff959de12 in_seq=0 out_seq=0 2024-04-02T14:01:45.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.928+0000 7f8b2affd640 1 -- 172.21.15.52:0/1425899598 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5588e5a20cf0 con 0x5588e5ac9020 2024-04-02T14:01:45.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.928+0000 7f8b2affd640 10 client.17946 handle_command_reply: tid=0 2024-04-02T14:01:45.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.928+0000 7f8b54bb3740 1 client.17946 shutdown 2024-04-02T14:01:45.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.928+0000 7f8b54bb3740 1 -- 172.21.15.52:0/1425899598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 msgr2=0x5588e5ab6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.928+0000 7f8b54bb3740 1 --2- 172.21.15.52:0/1425899598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 0x5588e5ab6070 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f8b400104b0 tx=0x7f8b40010510 comp rx=0 tx=0).stop 2024-04-02T14:01:45.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.928+0000 7f8b54bb3740 1 -- 172.21.15.52:0/1425899598 shutdown_connections 2024-04-02T14:01:45.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.928+0000 7f8b54bb3740 1 --2- 172.21.15.52:0/1425899598 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5588e5ac9020 0x5588e5acb410 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.928+0000 7f8b54bb3740 1 --2- 172.21.15.52:0/1425899598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588e5ab8ca0 0x5588e5ab65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.929+0000 7f8b54bb3740 1 --2- 172.21.15.52:0/1425899598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 0x5588e5ab6070 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.929+0000 7f8b54bb3740 1 --2- 172.21.15.52:0/1425899598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588e5aaf7a0 0x5588e5ab5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.929+0000 7f8b54bb3740 1 -- 172.21.15.52:0/1425899598 >> 172.21.15.52:0/1425899598 conn(0x5588e5aa9ed0 msgr2=0x5588e5aabc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:45.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.929+0000 7f8b2affd640 0 client.17946 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T14:01:45.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.929+0000 7f8b54bb3740 1 -- 172.21.15.52:0/1425899598 shutdown_connections 2024-04-02T14:01:45.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.929+0000 7f8b54bb3740 1 -- 172.21.15.52:0/1425899598 wait complete. 2024-04-02T14:01:45.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.929+0000 7f8b54bb3740 20 client.17946 trim_cache size 0 max 16384 2024-04-02T14:01:45.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.942+0000 7f8b54bb3740 1 Processor -- start 2024-04-02T14:01:45.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.942+0000 7f8b54bb3740 1 -- start start 2024-04-02T14:01:45.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.942+0000 7f8b54bb3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588e5aaf7a0 0x5588e5ab8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.942+0000 7f8b54bb3740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 0x5588e5ac4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.942+0000 7f8b54bb3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588e5ab8ca0 0x5588e5ab49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.942+0000 7f8b54bb3740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1aef40 con 0x5588e5ab00c0 2024-04-02T14:01:45.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.942+0000 7f8b54bb3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1b0a80 con 0x5588e5aaf7a0 2024-04-02T14:01:45.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.942+0000 7f8b54bb3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1af610 con 0x5588e5ab8ca0 2024-04-02T14:01:45.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.942+0000 7f8b4affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 0x5588e5ac4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.942+0000 7f8b4affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 0x5588e5ac4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60306/0 (socket says 172.21.15.52:60306) 2024-04-02T14:01:45.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.942+0000 7f8b4affd640 1 -- 172.21.15.52:0/733771905 learned_addr learned my addr 172.21.15.52:0/733771905 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:45.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.943+0000 7f8b4affd640 1 -- 172.21.15.52:0/733771905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588e5ab8ca0 msgr2=0x5588e5ab49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:45.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.943+0000 7f8b4bfff640 1 --2- 172.21.15.52:0/733771905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588e5ab8ca0 0x5588e5ab49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.943+0000 7f8b4affd640 1 --2- 172.21.15.52:0/733771905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588e5ab8ca0 0x5588e5ab49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.943+0000 7f8b4affd640 1 -- 172.21.15.52:0/733771905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588e5aaf7a0 msgr2=0x5588e5ab8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:45.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.943+0000 7f8b4affd640 1 --2- 172.21.15.52:0/733771905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588e5aaf7a0 0x5588e5ab8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.943+0000 7f8b4affd640 1 -- 172.21.15.52:0/733771905 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588e5abbbd0 con 0x5588e5ab00c0 2024-04-02T14:01:45.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.943+0000 7f8b4bfff640 1 --2- 172.21.15.52:0/733771905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588e5ab8ca0 0x5588e5ab49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:45.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.943+0000 7f8b4affd640 1 --2- 172.21.15.52:0/733771905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 0x5588e5ac4980 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009b70 tx=0x7f8b40005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf474db6e7648101 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:45.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.943+0000 7f8b317fa640 1 -- 172.21.15.52:0/733771905 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40027070 con 0x5588e5ab00c0 2024-04-02T14:01:45.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.943+0000 7f8b317fa640 1 -- 172.21.15.52:0/733771905 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b40011780 con 0x5588e5ab00c0 2024-04-02T14:01:45.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.943+0000 7f8b317fa640 1 -- 172.21.15.52:0/733771905 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40016050 con 0x5588e5ab00c0 2024-04-02T14:01:45.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.944+0000 7f8b54bb3740 1 -- 172.21.15.52:0/733771905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 msgr2=0x5588e5ac4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.944+0000 7f8b54bb3740 1 --2- 172.21.15.52:0/733771905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 0x5588e5ac4980 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009b70 tx=0x7f8b40005850 comp rx=0 tx=0).stop 2024-04-02T14:01:45.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.944+0000 7f8b54bb3740 1 -- 172.21.15.52:0/733771905 shutdown_connections 2024-04-02T14:01:45.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.944+0000 7f8b54bb3740 1 --2- 172.21.15.52:0/733771905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588e5ab8ca0 0x5588e5ab49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.944+0000 7f8b54bb3740 1 --2- 172.21.15.52:0/733771905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 0x5588e5ac4980 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.944+0000 7f8b54bb3740 1 --2- 172.21.15.52:0/733771905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588e5aaf7a0 0x5588e5ab8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.944+0000 7f8b54bb3740 1 -- 172.21.15.52:0/733771905 >> 172.21.15.52:0/733771905 conn(0x5588e5aa9ed0 msgr2=0x5588e5ab42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:45.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.944+0000 7f8b54bb3740 1 -- 172.21.15.52:0/733771905 shutdown_connections 2024-04-02T14:01:45.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.944+0000 7f8b54bb3740 1 -- 172.21.15.52:0/733771905 wait complete. 2024-04-02T14:01:45.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.944+0000 7f8b54bb3740 1 -- start start 2024-04-02T14:01:45.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.944+0000 7f8b54bb3740 1 Processor -- start 2024-04-02T14:01:45.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.944+0000 7f8b54bb3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588e5aaf7a0 0x5588e5abbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.944+0000 7f8b54bb3740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 0x5588e5abc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.944+0000 7f8b54bb3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588e5ab8ca0 0x5588e5abc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.944+0000 7f8b54bb3740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1b03b0 con 0x5588e5ab00c0 2024-04-02T14:01:45.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.944+0000 7f8b54bb3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1afce0 con 0x5588e5aaf7a0 2024-04-02T14:01:45.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.944+0000 7f8b54bb3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1acf40 con 0x5588e5ab8ca0 2024-04-02T14:01:45.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.944+0000 7f8b54bb3740 10 client.0 _ll_register_callbacks cb 0x7f8b40009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:45.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.945+0000 7f8b4affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 0x5588e5abc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.945+0000 7f8b4affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 0x5588e5abc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60320/0 (socket says 172.21.15.52:60320) 2024-04-02T14:01:45.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.945+0000 7f8b4affd640 1 -- 172.21.15.52:0/4117073414 learned_addr learned my addr 172.21.15.52:0/4117073414 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:45.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.945+0000 7f8b4bfff640 1 --2- 172.21.15.52:0/4117073414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588e5ab8ca0 0x5588e5abc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.945+0000 7f8b4affd640 1 -- 172.21.15.52:0/4117073414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588e5ab8ca0 msgr2=0x5588e5abc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:45.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.945+0000 7f8b4affd640 1 --2- 172.21.15.52:0/4117073414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588e5ab8ca0 0x5588e5abc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.945+0000 7f8b4affd640 1 -- 172.21.15.52:0/4117073414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588e5aaf7a0 msgr2=0x5588e5abbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:45.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.945+0000 7f8b4affd640 1 --2- 172.21.15.52:0/4117073414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588e5aaf7a0 0x5588e5abbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.945+0000 7f8b4affd640 1 -- 172.21.15.52:0/4117073414 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588e5b2d300 con 0x5588e5ab00c0 2024-04-02T14:01:45.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.945+0000 7f8b4affd640 1 --2- 172.21.15.52:0/4117073414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 0x5588e5abc390 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f8b400099e0 tx=0x7f8b400119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85d8a7b88eafaaf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:45.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.945+0000 7f8b2affd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:45.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.945+0000 7f8b2affd640 1 -- 172.21.15.52:0/4117073414 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40016070 con 0x5588e5ab00c0 2024-04-02T14:01:45.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.945+0000 7f8b2affd640 1 -- 172.21.15.52:0/4117073414 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f8b40004020 con 0x5588e5ab00c0 2024-04-02T14:01:45.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.945+0000 7f8b54bb3740 1 -- 172.21.15.52:0/4117073414 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b40011780 con 0x5588e5ab00c0 2024-04-02T14:01:45.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.945+0000 7f8b2affd640 1 -- 172.21.15.52:0/4117073414 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8b40027040 con 0x5588e5ab00c0 2024-04-02T14:01:45.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.945+0000 7f8b54bb3740 10 client.17952 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:45.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.945+0000 7f8b54bb3740 1 -- 172.21.15.52:0/4117073414 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5588e5b2e540 con 0x5588e5ab00c0 2024-04-02T14:01:45.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.946+0000 7f8b2affd640 1 -- 172.21.15.52:0/4117073414 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b40011780 con 0x5588e5ab00c0 2024-04-02T14:01:45.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.946+0000 7f8b2affd640 1 -- 172.21.15.52:0/4117073414 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8b40003850 con 0x5588e5ab00c0 2024-04-02T14:01:45.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.946+0000 7f8b54bb3740 10 client.17952 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:45.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.946+0000 7f8b54bb3740 10 client.17952 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T14:01:45.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.946+0000 7f8b54bb3740 20 client.17952 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:45.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.946+0000 7f8b54bb3740 1 --2- 172.21.15.52:0/4117073414 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5588e5b2eb40 0x5588e5b30f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:45.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.946+0000 7f8b54bb3740 4 client.17952 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:01:45.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.946+0000 7f8b54bb3740 1 -- 172.21.15.52:0/4117073414 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5588e5a1b040 con 0x5588e5b2eb40 2024-04-02T14:01:45.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.946+0000 7f8b4bfff640 1 --2- 172.21.15.52:0/4117073414 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5588e5b2eb40 0x5588e5b30f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:45.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.946+0000 7f8b4bfff640 1 --2- 172.21.15.52:0/4117073414 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5588e5b2eb40 0x5588e5b30f60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c1c2bc8028c3543d server_cookie=554c0e1793337762 in_seq=0 out_seq=0 2024-04-02T14:01:45.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.946+0000 7f8b2affd640 10 client.17952 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T14:01:45.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.947+0000 7f8b2affd640 1 -- 172.21.15.52:0/4117073414 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5588e5a1b040 con 0x5588e5b2eb40 2024-04-02T14:01:45.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.947+0000 7f8b2affd640 10 client.17952 handle_command_reply: tid=0 2024-04-02T14:01:45.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.947+0000 7f8b54bb3740 1 client.17952 shutdown 2024-04-02T14:01:45.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.947+0000 7f8b54bb3740 1 -- 172.21.15.52:0/4117073414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 msgr2=0x5588e5abc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.947+0000 7f8b54bb3740 1 --2- 172.21.15.52:0/4117073414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 0x5588e5abc390 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f8b400099e0 tx=0x7f8b400119b0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.947+0000 7f8b54bb3740 1 -- 172.21.15.52:0/4117073414 shutdown_connections 2024-04-02T14:01:45.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.947+0000 7f8b54bb3740 1 --2- 172.21.15.52:0/4117073414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588e5ab8ca0 0x5588e5abc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.947+0000 7f8b54bb3740 1 --2- 172.21.15.52:0/4117073414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e5ab00c0 0x5588e5abc390 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.947+0000 7f8b54bb3740 1 --2- 172.21.15.52:0/4117073414 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5588e5b2eb40 0x5588e5b30f60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.947+0000 7f8b54bb3740 1 --2- 172.21.15.52:0/4117073414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588e5aaf7a0 0x5588e5abbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.947+0000 7f8b54bb3740 1 -- 172.21.15.52:0/4117073414 >> 172.21.15.52:0/4117073414 conn(0x5588e5aa9ed0 msgr2=0x5588e5ab4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:45.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.947+0000 7f8b2affd640 0 client.17952 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T14:01:45.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.947+0000 7f8b54bb3740 1 -- 172.21.15.52:0/4117073414 shutdown_connections 2024-04-02T14:01:45.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.947+0000 7f8b54bb3740 1 -- 172.21.15.52:0/4117073414 wait complete. 2024-04-02T14:01:45.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.947+0000 7f8b54bb3740 20 client.17952 trim_cache size 0 max 16384 2024-04-02T14:01:45.963 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:01:45.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.947+0000 7f8b51fcd640 1 -- 172.21.15.52:0/338018641 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f8b340a3d20 msgr2=0x7f8b340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.947+0000 7f8b51fcd640 1 --2- 172.21.15.52:0/338018641 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f8b340a3d20 0x7f8b340a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0effc0 tx=0x7f8b3c000bc0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.947+0000 7f8b51fcd640 1 -- 172.21.15.52:0/338018641 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b4c1009b0 msgr2=0x7f8b4c1983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:45.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.947+0000 7f8b51fcd640 1 --2- 172.21.15.52:0/338018641 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b4c1009b0 0x7f8b4c1983b0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c009230 tx=0x7f8b3c02cc70 comp rx=0 tx=0).stop 2024-04-02T14:01:45.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.948+0000 7f8b51fcd640 1 -- 172.21.15.52:0/338018641 shutdown_connections 2024-04-02T14:01:45.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.948+0000 7f8b51fcd640 1 --2- 172.21.15.52:0/338018641 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f8b340a3d20 0x7f8b340a61e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.948+0000 7f8b51fcd640 1 --2- 172.21.15.52:0/338018641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b4c1069c0 0x7f8b4c19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.948+0000 7f8b51fcd640 1 --2- 172.21.15.52:0/338018641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b4c101370 0x7f8b4c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.948+0000 7f8b51fcd640 1 --2- 172.21.15.52:0/338018641 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b4c1009b0 0x7f8b4c1983b0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:45.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.948+0000 7f8b51fcd640 1 -- 172.21.15.52:0/338018641 >> 172.21.15.52:0/338018641 conn(0x7f8b4c076d80 msgr2=0x7f8b4c06a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:45.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.948+0000 7f8b51fcd640 1 -- 172.21.15.52:0/338018641 shutdown_connections 2024-04-02T14:01:45.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:45.948+0000 7f8b51fcd640 1 -- 172.21.15.52:0/338018641 wait complete. 2024-04-02T14:01:45.966 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T14:01:46.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:45 smithi189 ceph-mon[36683]: pgmap v254: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:46.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.106+0000 7f4566938640 1 -- 172.21.15.52:0/701269287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4560101330 msgr2=0x7f456010d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.106+0000 7f4566938640 1 --2- 172.21.15.52:0/701269287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4560101330 0x7f456010d840 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f4558009cd0 tx=0x7f455802c0f0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.106+0000 7f4566938640 1 -- 172.21.15.52:0/701269287 shutdown_connections 2024-04-02T14:01:46.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.106+0000 7f4566938640 1 --2- 172.21.15.52:0/701269287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4560101330 0x7f456010d840 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.106+0000 7f4566938640 1 --2- 172.21.15.52:0/701269287 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4560100970 0x7f4560100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.106+0000 7f4566938640 1 --2- 172.21.15.52:0/701269287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4560106980 0x7f4560106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.106+0000 7f4566938640 1 -- 172.21.15.52:0/701269287 >> 172.21.15.52:0/701269287 conn(0x7f4560077b00 msgr2=0x7f4560077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:46.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.107+0000 7f4566938640 1 -- 172.21.15.52:0/701269287 shutdown_connections 2024-04-02T14:01:46.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.107+0000 7f4566938640 1 -- 172.21.15.52:0/701269287 wait complete. 2024-04-02T14:01:46.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.107+0000 7f4566938640 1 Processor -- start 2024-04-02T14:01:46.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.107+0000 7f4566938640 1 -- start start 2024-04-02T14:01:46.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.107+0000 7f4566938640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4560100970 0x7f4560198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.107+0000 7f4566938640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4560101330 0x7f4560198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.107+0000 7f4566938640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4560106980 0x7f456019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.107+0000 7f4566938640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4560110d90 con 0x7f4560106980 2024-04-02T14:01:46.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.107+0000 7f4566938640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4560110c10 con 0x7f4560100970 2024-04-02T14:01:46.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.107+0000 7f4566938640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4560110f10 con 0x7f4560101330 2024-04-02T14:01:46.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.108+0000 7f4564ed5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4560106980 0x7f456019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.108+0000 7f4564ed5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4560106980 0x7f456019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60350/0 (socket says 172.21.15.52:60350) 2024-04-02T14:01:46.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.108+0000 7f4564ed5640 1 -- 172.21.15.52:0/81401627 learned_addr learned my addr 172.21.15.52:0/81401627 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:46.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.108+0000 7f455ffff640 1 --2- 172.21.15.52:0/81401627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4560100970 0x7f4560198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.108+0000 7f455f7fe640 1 --2- 172.21.15.52:0/81401627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4560101330 0x7f4560198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.108+0000 7f4564ed5640 1 -- 172.21.15.52:0/81401627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4560101330 msgr2=0x7f4560198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.108+0000 7f4564ed5640 1 --2- 172.21.15.52:0/81401627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4560101330 0x7f4560198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.108+0000 7f4564ed5640 1 -- 172.21.15.52:0/81401627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4560100970 msgr2=0x7f4560198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.108+0000 7f4564ed5640 1 --2- 172.21.15.52:0/81401627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4560100970 0x7f4560198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.108+0000 7f4564ed5640 1 -- 172.21.15.52:0/81401627 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456019d330 con 0x7f4560106980 2024-04-02T14:01:46.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.108+0000 7f4564ed5640 1 --2- 172.21.15.52:0/81401627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4560106980 0x7f456019cc00 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f45580348f0 tx=0x7f4558032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:46.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.108+0000 7f455d7fa640 1 -- 172.21.15.52:0/81401627 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4558041020 con 0x7f4560106980 2024-04-02T14:01:46.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.108+0000 7f455d7fa640 1 -- 172.21.15.52:0/81401627 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4558034cd0 con 0x7f4560106980 2024-04-02T14:01:46.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.108+0000 7f4566938640 1 -- 172.21.15.52:0/81401627 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456019d5c0 con 0x7f4560106980 2024-04-02T14:01:46.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.108+0000 7f455d7fa640 1 -- 172.21.15.52:0/81401627 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4558039040 con 0x7f4560106980 2024-04-02T14:01:46.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.108+0000 7f4566938640 1 -- 172.21.15.52:0/81401627 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45601a4eb0 con 0x7f4560106980 2024-04-02T14:01:46.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.110+0000 7f455d7fa640 1 -- 172.21.15.52:0/81401627 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4558044020 con 0x7f4560106980 2024-04-02T14:01:46.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.111+0000 7f45695f0740 1 Processor -- start 2024-04-02T14:01:46.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.111+0000 7f45695f0740 1 -- start start 2024-04-02T14:01:46.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.111+0000 7f45695f0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bb28aa7a0 0x562bb28aab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.111+0000 7f45695f0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28ab0c0 0x562bb28b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.111+0000 7f45695f0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bb28b3ca0 0x562bb28b6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.111+0000 7f45695f0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562bb280e7b0 con 0x562bb28ab0c0 2024-04-02T14:01:46.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.111+0000 7f45695f0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x562bb27ad570 con 0x562bb28b3ca0 2024-04-02T14:01:46.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.111+0000 7f45695f0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562bb280a760 con 0x562bb28aa7a0 2024-04-02T14:01:46.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.111+0000 7f455d7fa640 1 --2- 172.21.15.52:0/81401627 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f45480a3d20 0x7f45480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.111+0000 7f455ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28ab0c0 0x562bb28b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.111+0000 7f455ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28ab0c0 0x562bb28b3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60362/0 (socket says 172.21.15.52:60362) 2024-04-02T14:01:46.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.111+0000 7f455ffff640 1 -- 172.21.15.52:0/2283541875 learned_addr learned my addr 172.21.15.52:0/2283541875 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:46.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.111+0000 7f455f7fe640 1 --2- 172.21.15.52:0/2283541875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bb28aa7a0 0x562bb28aab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.111+0000 7f4564ed5640 1 --2- 172.21.15.52:0/81401627 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f45480a3d20 0x7f45480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.111+0000 7f455f7fe640 1 --2- 172.21.15.52:0/2283541875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bb28b3ca0 0x562bb28b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.112+0000 7f455d7fa640 1 -- 172.21.15.52:0/81401627 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f45580f2920 con 0x7f4560106980 2024-04-02T14:01:46.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.112+0000 7f455ffff640 1 -- 172.21.15.52:0/2283541875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bb28aa7a0 msgr2=0x562bb28aab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:46.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.112+0000 7f455ffff640 1 --2- 172.21.15.52:0/2283541875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bb28aa7a0 0x562bb28aab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.112+0000 7f455ffff640 1 -- 172.21.15.52:0/2283541875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bb28b3ca0 msgr2=0x562bb28b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:46.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.112+0000 7f455ffff640 1 --2- 172.21.15.52:0/2283541875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bb28b3ca0 0x562bb28b6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.112+0000 7f455ffff640 1 -- 172.21.15.52:0/2283541875 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bb25223f0 con 0x562bb28ab0c0 2024-04-02T14:01:46.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.112+0000 7f455ffff640 1 --2- 172.21.15.52:0/2283541875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28ab0c0 0x562bb28b3760 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f455000bc90 tx=0x7f45500107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0688528fa5d3afe server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:46.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.112+0000 7f4564ed5640 1 --2- 172.21.15.52:0/81401627 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f45480a3d20 0x7f45480a61e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f45580be340 tx=0x7f4558090080 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:46.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.112+0000 7f45467fc640 1 -- 172.21.15.52:0/2283541875 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455001c480 con 0x562bb28ab0c0 2024-04-02T14:01:46.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.112+0000 7f45467fc640 1 -- 172.21.15.52:0/2283541875 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f455001cb00 con 0x562bb28ab0c0 2024-04-02T14:01:46.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.112+0000 7f45467fc640 1 -- 172.21.15.52:0/2283541875 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4550011b90 con 0x562bb28ab0c0 2024-04-02T14:01:46.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.113+0000 7f45695f0740 1 -- 172.21.15.52:0/2283541875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28ab0c0 msgr2=0x562bb28b3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.113+0000 7f45695f0740 1 --2- 172.21.15.52:0/2283541875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28ab0c0 0x562bb28b3760 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f455000bc90 tx=0x7f45500107a0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.113+0000 7f45695f0740 1 -- 172.21.15.52:0/2283541875 shutdown_connections 2024-04-02T14:01:46.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.113+0000 7f45695f0740 1 --2- 172.21.15.52:0/2283541875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bb28b3ca0 0x562bb28b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.113+0000 7f45695f0740 1 --2- 172.21.15.52:0/2283541875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28ab0c0 0x562bb28b3760 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.113+0000 7f45695f0740 1 --2- 172.21.15.52:0/2283541875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bb28aa7a0 0x562bb28aab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.113+0000 7f45695f0740 1 -- 172.21.15.52:0/2283541875 >> 172.21.15.52:0/2283541875 conn(0x562bb28a4ed0 msgr2=0x562bb28a52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:46.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.113+0000 7f45695f0740 1 -- 172.21.15.52:0/2283541875 shutdown_connections 2024-04-02T14:01:46.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.113+0000 7f45695f0740 1 -- 172.21.15.52:0/2283541875 wait complete. 2024-04-02T14:01:46.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.113+0000 7f45695f0740 1 -- start start 2024-04-02T14:01:46.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.113+0000 7f45695f0740 1 Processor -- start 2024-04-02T14:01:46.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.113+0000 7f45695f0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bb28aa7a0 0x562bb28b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.113+0000 7f45695f0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bb28ab0c0 0x562bb28b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.113+0000 7f45695f0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28b3ca0 0x562bb28b1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.113+0000 7f45695f0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562bb28b6a90 con 0x562bb28b3ca0 2024-04-02T14:01:46.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.113+0000 7f45695f0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x562bb27c9e80 con 0x562bb28aa7a0 2024-04-02T14:01:46.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.113+0000 7f45695f0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562bb28b6c10 con 0x562bb28ab0c0 2024-04-02T14:01:46.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.113+0000 7f45695f0740 10 client.0 _ll_register_callbacks cb 0x562bb2805aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:46.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.114+0000 7f455f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28b3ca0 0x562bb28b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.114+0000 7f455ffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bb28ab0c0 0x562bb28b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.114+0000 7f455f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28b3ca0 0x562bb28b1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60364/0 (socket says 172.21.15.52:60364) 2024-04-02T14:01:46.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.114+0000 7f455f7fe640 1 -- 172.21.15.52:0/3565796396 learned_addr learned my addr 172.21.15.52:0/3565796396 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:46.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.114+0000 7f455f7fe640 1 --2- 172.21.15.52:0/3565796396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bb28aa7a0 0x562bb28b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.114+0000 7f455f7fe640 1 -- 172.21.15.52:0/3565796396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bb28ab0c0 msgr2=0x562bb28b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:46.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.114+0000 7f455f7fe640 1 --2- 172.21.15.52:0/3565796396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bb28ab0c0 0x562bb28b0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.114+0000 7f455f7fe640 1 -- 172.21.15.52:0/3565796396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bb28aa7a0 msgr2=0x562bb28b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:46.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.114+0000 7f455f7fe640 1 --2- 172.21.15.52:0/3565796396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bb28aa7a0 0x562bb28b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.114+0000 7f455f7fe640 1 -- 172.21.15.52:0/3565796396 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bb25ec7d0 con 0x562bb28b3ca0 2024-04-02T14:01:46.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.114+0000 7f455f7fe640 1 --2- 172.21.15.52:0/3565796396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28b3ca0 0x562bb28b1030 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f4554012ea0 tx=0x7f455400cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9db9e6dbb9133b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:46.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.114+0000 7f453f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:46.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.114+0000 7f453f7fe640 1 -- 172.21.15.52:0/3565796396 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554004030 con 0x562bb28b3ca0 2024-04-02T14:01:46.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.114+0000 7f453f7fe640 1 -- 172.21.15.52:0/3565796396 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f45540041d0 con 0x562bb28b3ca0 2024-04-02T14:01:46.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.114+0000 7f45695f0740 1 -- 172.21.15.52:0/3565796396 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f455001cb00 con 0x562bb28b3ca0 2024-04-02T14:01:46.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.114+0000 7f453f7fe640 1 -- 172.21.15.52:0/3565796396 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4554027050 con 0x562bb28b3ca0 2024-04-02T14:01:46.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.114+0000 7f45695f0740 10 client.17964 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:46.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.114+0000 7f45695f0740 1 -- 172.21.15.52:0/3565796396 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562bb28c2ae0 con 0x562bb28b3ca0 2024-04-02T14:01:46.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.115+0000 7f453f7fe640 1 -- 172.21.15.52:0/3565796396 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f455001cb00 con 0x562bb28b3ca0 2024-04-02T14:01:46.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.115+0000 7f453f7fe640 1 -- 172.21.15.52:0/3565796396 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f45540094c0 con 0x562bb28b3ca0 2024-04-02T14:01:46.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.115+0000 7f45695f0740 10 client.17964 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:46.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.115+0000 7f45695f0740 10 client.17964 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T14:01:46.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.115+0000 7f45695f0740 20 client.17964 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:46.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.115+0000 7f45695f0740 1 --2- 172.21.15.52:0/3565796396 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x562bb28c3f40 0x562bb28c6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.115+0000 7f45695f0740 4 client.17964 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:01:46.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.115+0000 7f45695f0740 1 -- 172.21.15.52:0/3565796396 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bb281bcf0 con 0x562bb28c3f40 2024-04-02T14:01:46.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.115+0000 7f455ffff640 1 --2- 172.21.15.52:0/3565796396 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x562bb28c3f40 0x562bb28c6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.115+0000 7f455ffff640 1 --2- 172.21.15.52:0/3565796396 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x562bb28c3f40 0x562bb28c6390 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b6599e3891e9613 server_cookie=628a61c34c08b8b0 in_seq=0 out_seq=0 2024-04-02T14:01:46.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.115+0000 7f453f7fe640 10 client.17964 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T14:01:46.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.116+0000 7f453f7fe640 1 -- 172.21.15.52:0/3565796396 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562bb281bcf0 con 0x562bb28c3f40 2024-04-02T14:01:46.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.116+0000 7f453f7fe640 10 client.17964 handle_command_reply: tid=0 2024-04-02T14:01:46.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.116+0000 7f45695f0740 1 client.17964 shutdown 2024-04-02T14:01:46.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.116+0000 7f45695f0740 1 -- 172.21.15.52:0/3565796396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28b3ca0 msgr2=0x562bb28b1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.116+0000 7f45695f0740 1 --2- 172.21.15.52:0/3565796396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28b3ca0 0x562bb28b1030 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f4554012ea0 tx=0x7f455400cb50 comp rx=0 tx=0).stop 2024-04-02T14:01:46.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.116+0000 7f45695f0740 1 -- 172.21.15.52:0/3565796396 shutdown_connections 2024-04-02T14:01:46.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.116+0000 7f45695f0740 1 --2- 172.21.15.52:0/3565796396 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x562bb28c3f40 0x562bb28c6390 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.116+0000 7f45695f0740 1 --2- 172.21.15.52:0/3565796396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28b3ca0 0x562bb28b1030 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.116+0000 7f45695f0740 1 --2- 172.21.15.52:0/3565796396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bb28ab0c0 0x562bb28b0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.116+0000 7f45695f0740 1 --2- 172.21.15.52:0/3565796396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bb28aa7a0 0x562bb28b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.116+0000 7f453f7fe640 0 client.17964 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T14:01:46.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.116+0000 7f45695f0740 1 -- 172.21.15.52:0/3565796396 >> 172.21.15.52:0/3565796396 conn(0x562bb28a4ed0 msgr2=0x562bb28a6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:46.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.116+0000 7f45695f0740 1 -- 172.21.15.52:0/3565796396 shutdown_connections 2024-04-02T14:01:46.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.116+0000 7f45695f0740 1 -- 172.21.15.52:0/3565796396 wait complete. 2024-04-02T14:01:46.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.116+0000 7f45695f0740 20 client.17964 trim_cache size 0 max 16384 2024-04-02T14:01:46.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.130+0000 7f45695f0740 1 Processor -- start 2024-04-02T14:01:46.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.130+0000 7f45695f0740 1 -- start start 2024-04-02T14:01:46.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.130+0000 7f45695f0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bb28aa7a0 0x562bb28bf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.130+0000 7f45695f0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28ab0c0 0x562bb28b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.130+0000 7f45695f0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bb28b3ca0 0x562bb28b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.130+0000 7f45695f0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f45601aef70 con 0x562bb28ab0c0 2024-04-02T14:01:46.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.130+0000 7f45695f0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f45601b0ab0 con 0x562bb28aa7a0 2024-04-02T14:01:46.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.130+0000 7f45695f0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f45601af640 con 0x562bb28b3ca0 2024-04-02T14:01:46.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.130+0000 7f455ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28ab0c0 0x562bb28b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.130+0000 7f455ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28ab0c0 0x562bb28b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60366/0 (socket says 172.21.15.52:60366) 2024-04-02T14:01:46.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.130+0000 7f455ffff640 1 -- 172.21.15.52:0/2881533755 learned_addr learned my addr 172.21.15.52:0/2881533755 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:46.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.130+0000 7f455ffff640 1 -- 172.21.15.52:0/2881533755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bb28b3ca0 msgr2=0x562bb28b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:46.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.130+0000 7f455ffff640 1 --2- 172.21.15.52:0/2881533755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bb28b3ca0 0x562bb28b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.130+0000 7f455ffff640 1 -- 172.21.15.52:0/2881533755 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bb28aa7a0 msgr2=0x562bb28bf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:46.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.130+0000 7f455f7fe640 1 --2- 172.21.15.52:0/2881533755 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bb28aa7a0 0x562bb28bf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.130+0000 7f455ffff640 1 --2- 172.21.15.52:0/2881533755 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bb28aa7a0 0x562bb28bf9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.130+0000 7f455ffff640 1 -- 172.21.15.52:0/2881533755 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bb28aaef0 con 0x562bb28ab0c0 2024-04-02T14:01:46.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.130+0000 7f455ffff640 1 --2- 172.21.15.52:0/2881533755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28ab0c0 0x562bb28b3150 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x562bb28aa270 tx=0x7f4550011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf485c17720e0a67 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:46.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.131+0000 7f4545ffb640 1 -- 172.21.15.52:0/2881533755 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4550004020 con 0x562bb28ab0c0 2024-04-02T14:01:46.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.131+0000 7f4545ffb640 1 -- 172.21.15.52:0/2881533755 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4550011d60 con 0x562bb28ab0c0 2024-04-02T14:01:46.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.131+0000 7f4545ffb640 1 -- 172.21.15.52:0/2881533755 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4550005290 con 0x562bb28ab0c0 2024-04-02T14:01:46.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.131+0000 7f45695f0740 1 -- 172.21.15.52:0/2881533755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28ab0c0 msgr2=0x562bb28b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.131+0000 7f45695f0740 1 --2- 172.21.15.52:0/2881533755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28ab0c0 0x562bb28b3150 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x562bb28aa270 tx=0x7f4550011810 comp rx=0 tx=0).stop 2024-04-02T14:01:46.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.131+0000 7f45695f0740 1 -- 172.21.15.52:0/2881533755 shutdown_connections 2024-04-02T14:01:46.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.131+0000 7f45695f0740 1 --2- 172.21.15.52:0/2881533755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bb28b3ca0 0x562bb28b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.131+0000 7f45695f0740 1 --2- 172.21.15.52:0/2881533755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28ab0c0 0x562bb28b3150 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.131+0000 7f45695f0740 1 --2- 172.21.15.52:0/2881533755 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bb28aa7a0 0x562bb28bf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.131+0000 7f45695f0740 1 -- 172.21.15.52:0/2881533755 >> 172.21.15.52:0/2881533755 conn(0x562bb28a4ed0 msgr2=0x562bb28af310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:46.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.131+0000 7f45695f0740 1 -- 172.21.15.52:0/2881533755 shutdown_connections 2024-04-02T14:01:46.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.131+0000 7f45695f0740 1 -- 172.21.15.52:0/2881533755 wait complete. 2024-04-02T14:01:46.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.131+0000 7f45695f0740 1 -- start start 2024-04-02T14:01:46.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.132+0000 7f45695f0740 1 Processor -- start 2024-04-02T14:01:46.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.132+0000 7f45695f0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bb28aa7a0 0x562bb28ad420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.132+0000 7f45695f0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28ab0c0 0x562bb28ad960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.132+0000 7f45695f0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bb28b3ca0 0x562bb28baec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.132+0000 7f45695f0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f45601b03e0 con 0x562bb28ab0c0 2024-04-02T14:01:46.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.132+0000 7f45695f0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f45601afd10 con 0x562bb28aa7a0 2024-04-02T14:01:46.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.132+0000 7f45695f0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f45601acf70 con 0x562bb28b3ca0 2024-04-02T14:01:46.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.132+0000 7f45695f0740 10 client.0 _ll_register_callbacks cb 0x7f455400ced0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:46.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.132+0000 7f455ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28ab0c0 0x562bb28ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.132+0000 7f455ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28ab0c0 0x562bb28ad960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60374/0 (socket says 172.21.15.52:60374) 2024-04-02T14:01:46.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.132+0000 7f455ffff640 1 -- 172.21.15.52:0/2208813872 learned_addr learned my addr 172.21.15.52:0/2208813872 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:46.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.132+0000 7f455f7fe640 1 --2- 172.21.15.52:0/2208813872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bb28aa7a0 0x562bb28ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.132+0000 7f455ffff640 1 -- 172.21.15.52:0/2208813872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bb28b3ca0 msgr2=0x562bb28baec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:46.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.132+0000 7f455ffff640 1 --2- 172.21.15.52:0/2208813872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bb28b3ca0 0x562bb28baec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.132+0000 7f455ffff640 1 -- 172.21.15.52:0/2208813872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bb28aa7a0 msgr2=0x562bb28ad420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:46.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.132+0000 7f455ffff640 1 --2- 172.21.15.52:0/2208813872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bb28aa7a0 0x562bb28ad420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.132+0000 7f455ffff640 1 -- 172.21.15.52:0/2208813872 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bb28bb4b0 con 0x562bb28ab0c0 2024-04-02T14:01:46.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.132+0000 7f455f7fe640 1 --2- 172.21.15.52:0/2208813872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bb28aa7a0 0x562bb28ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:46.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.132+0000 7f455ffff640 1 --2- 172.21.15.52:0/2208813872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28ab0c0 0x562bb28ad960 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f455000bbb0 tx=0x7f455000b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8306291586d032a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:46.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.132+0000 7f453f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:46.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.133+0000 7f453f7fe640 1 -- 172.21.15.52:0/2208813872 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4550038060 con 0x562bb28ab0c0 2024-04-02T14:01:46.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.133+0000 7f453f7fe640 1 -- 172.21.15.52:0/2208813872 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f45500053f0 con 0x562bb28ab0c0 2024-04-02T14:01:46.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.133+0000 7f45695f0740 1 -- 172.21.15.52:0/2208813872 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4550011d60 con 0x562bb28ab0c0 2024-04-02T14:01:46.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.133+0000 7f453f7fe640 1 -- 172.21.15.52:0/2208813872 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f45500055f0 con 0x562bb28ab0c0 2024-04-02T14:01:46.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.133+0000 7f45695f0740 10 client.17970 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:46.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.133+0000 7f45695f0740 1 -- 172.21.15.52:0/2208813872 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562bb28bc7b0 con 0x562bb28ab0c0 2024-04-02T14:01:46.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.133+0000 7f453f7fe640 1 -- 172.21.15.52:0/2208813872 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4550011d60 con 0x562bb28ab0c0 2024-04-02T14:01:46.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.133+0000 7f453f7fe640 1 -- 172.21.15.52:0/2208813872 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f45500035f0 con 0x562bb28ab0c0 2024-04-02T14:01:46.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.133+0000 7f45695f0740 10 client.17970 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:46.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.133+0000 7f45695f0740 10 client.17970 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T14:01:46.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.133+0000 7f45695f0740 20 client.17970 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:46.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.133+0000 7f45695f0740 1 --2- 172.21.15.52:0/2208813872 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x562bb292a890 0x562bb292ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.133+0000 7f45695f0740 4 client.17970 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:01:46.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.133+0000 7f45695f0740 1 -- 172.21.15.52:0/2208813872 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562bb2816040 con 0x562bb292a890 2024-04-02T14:01:46.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.133+0000 7f455f7fe640 1 --2- 172.21.15.52:0/2208813872 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x562bb292a890 0x562bb292ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.134+0000 7f453f7fe640 10 client.17970 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T14:01:46.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.134+0000 7f455f7fe640 1 --2- 172.21.15.52:0/2208813872 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x562bb292a890 0x562bb292ccb0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b3ecc2dcaa29eefc server_cookie=757c4f9592aa4278 in_seq=0 out_seq=0 2024-04-02T14:01:46.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.134+0000 7f453f7fe640 1 -- 172.21.15.52:0/2208813872 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562bb2816040 con 0x562bb292a890 2024-04-02T14:01:46.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.134+0000 7f453f7fe640 10 client.17970 handle_command_reply: tid=0 2024-04-02T14:01:46.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.134+0000 7f45695f0740 1 client.17970 shutdown 2024-04-02T14:01:46.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.134+0000 7f45695f0740 1 -- 172.21.15.52:0/2208813872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28ab0c0 msgr2=0x562bb28ad960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.134+0000 7f45695f0740 1 --2- 172.21.15.52:0/2208813872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28ab0c0 0x562bb28ad960 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f455000bbb0 tx=0x7f455000b690 comp rx=0 tx=0).stop 2024-04-02T14:01:46.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.134+0000 7f45695f0740 1 -- 172.21.15.52:0/2208813872 shutdown_connections 2024-04-02T14:01:46.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.134+0000 7f45695f0740 1 --2- 172.21.15.52:0/2208813872 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x562bb292a890 0x562bb292ccb0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.134+0000 7f45695f0740 1 --2- 172.21.15.52:0/2208813872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bb28b3ca0 0x562bb28baec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.134+0000 7f45695f0740 1 --2- 172.21.15.52:0/2208813872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562bb28ab0c0 0x562bb28ad960 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.134+0000 7f453f7fe640 0 client.17970 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T14:01:46.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.134+0000 7f45695f0740 1 --2- 172.21.15.52:0/2208813872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562bb28aa7a0 0x562bb28ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.134+0000 7f45695f0740 1 -- 172.21.15.52:0/2208813872 >> 172.21.15.52:0/2208813872 conn(0x562bb28a4ed0 msgr2=0x562bb28a6200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:46.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.134+0000 7f45695f0740 1 -- 172.21.15.52:0/2208813872 shutdown_connections 2024-04-02T14:01:46.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.134+0000 7f45695f0740 1 -- 172.21.15.52:0/2208813872 wait complete. 2024-04-02T14:01:46.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.135+0000 7f45695f0740 20 client.17970 trim_cache size 0 max 16384 2024-04-02T14:01:46.150 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:01:46.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.135+0000 7f4566938640 1 -- 172.21.15.52:0/81401627 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f45480a3d20 msgr2=0x7f45480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.135+0000 7f4566938640 1 --2- 172.21.15.52:0/81401627 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f45480a3d20 0x7f45480a61e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f45580be340 tx=0x7f4558090080 comp rx=0 tx=0).stop 2024-04-02T14:01:46.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.135+0000 7f4566938640 1 -- 172.21.15.52:0/81401627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4560106980 msgr2=0x7f456019cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.135+0000 7f4566938640 1 --2- 172.21.15.52:0/81401627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4560106980 0x7f456019cc00 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f45580348f0 tx=0x7f4558032ec0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.135+0000 7f4566938640 1 -- 172.21.15.52:0/81401627 shutdown_connections 2024-04-02T14:01:46.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.135+0000 7f4566938640 1 --2- 172.21.15.52:0/81401627 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f45480a3d20 0x7f45480a61e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.135+0000 7f4566938640 1 --2- 172.21.15.52:0/81401627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4560106980 0x7f456019cc00 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.135+0000 7f4566938640 1 --2- 172.21.15.52:0/81401627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4560101330 0x7f4560198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.135+0000 7f4566938640 1 --2- 172.21.15.52:0/81401627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4560100970 0x7f4560198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.135+0000 7f4566938640 1 -- 172.21.15.52:0/81401627 >> 172.21.15.52:0/81401627 conn(0x7f4560077b00 msgr2=0x7f45600ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:46.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.135+0000 7f4566938640 1 -- 172.21.15.52:0/81401627 shutdown_connections 2024-04-02T14:01:46.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.135+0000 7f4566938640 1 -- 172.21.15.52:0/81401627 wait complete. 2024-04-02T14:01:46.153 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T14:01:46.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.254+0000 7fbfa360d640 1 -- 172.21.15.52:0/4149440425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf9c102d70 msgr2=0x7fbf9c103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.254+0000 7fbfa360d640 1 --2- 172.21.15.52:0/4149440425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf9c102d70 0x7fbf9c103150 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009900 tx=0x7fbf9802bcd0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.254+0000 7fbfa360d640 1 -- 172.21.15.52:0/4149440425 shutdown_connections 2024-04-02T14:01:46.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.254+0000 7fbfa360d640 1 --2- 172.21.15.52:0/4149440425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf9c10c140 0x7fbf9c10e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.254+0000 7fbfa360d640 1 --2- 172.21.15.52:0/4149440425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf9c103690 0x7fbf9c10bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.254+0000 7fbfa360d640 1 --2- 172.21.15.52:0/4149440425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf9c102d70 0x7fbf9c103150 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.254+0000 7fbfa360d640 1 -- 172.21.15.52:0/4149440425 >> 172.21.15.52:0/4149440425 conn(0x7fbf9c077b60 msgr2=0x7fbf9c077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:46.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.254+0000 7fbfa360d640 1 -- 172.21.15.52:0/4149440425 shutdown_connections 2024-04-02T14:01:46.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.254+0000 7fbfa360d640 1 -- 172.21.15.52:0/4149440425 wait complete. 2024-04-02T14:01:46.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.255+0000 7fbfa360d640 1 Processor -- start 2024-04-02T14:01:46.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.255+0000 7fbfa360d640 1 -- start start 2024-04-02T14:01:46.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.255+0000 7fbfa360d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf9c102d70 0x7fbf9c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.255+0000 7fbfa360d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf9c103690 0x7fbf9c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.255+0000 7fbfa360d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf9c10c140 0x7fbf9c19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.255+0000 7fbfa360d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c111f10 con 0x7fbf9c102d70 2024-04-02T14:01:46.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.255+0000 7fbfa360d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c111d90 con 0x7fbf9c10c140 2024-04-02T14:01:46.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.255+0000 7fbfa360d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c112090 con 0x7fbf9c103690 2024-04-02T14:01:46.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.255+0000 7fbfa13a9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf9c102d70 0x7fbf9c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.255+0000 7fbfa13a9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf9c102d70 0x7fbf9c198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60400/0 (socket says 172.21.15.52:60400) 2024-04-02T14:01:46.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.255+0000 7fbfa13a9640 1 -- 172.21.15.52:0/4138503313 learned_addr learned my addr 172.21.15.52:0/4138503313 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:46.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.255+0000 7fbfa1baa640 1 --2- 172.21.15.52:0/4138503313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf9c10c140 0x7fbf9c19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.255+0000 7fbfa0ba8640 1 --2- 172.21.15.52:0/4138503313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf9c103690 0x7fbf9c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.255+0000 7fbfa13a9640 1 -- 172.21.15.52:0/4138503313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf9c103690 msgr2=0x7fbf9c198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.255+0000 7fbfa13a9640 1 --2- 172.21.15.52:0/4138503313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf9c103690 0x7fbf9c198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.255+0000 7fbfa13a9640 1 -- 172.21.15.52:0/4138503313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf9c10c140 msgr2=0x7fbf9c19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.255+0000 7fbfa13a9640 1 --2- 172.21.15.52:0/4138503313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf9c10c140 0x7fbf9c19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.255+0000 7fbfa13a9640 1 -- 172.21.15.52:0/4138503313 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9c19d320 con 0x7fbf9c102d70 2024-04-02T14:01:46.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.255+0000 7fbfa0ba8640 1 --2- 172.21.15.52:0/4138503313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf9c103690 0x7fbf9c198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:46.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.256+0000 7fbfa13a9640 1 --2- 172.21.15.52:0/4138503313 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf9c102d70 0x7fbf9c198330 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fbf98005940 tx=0x7fbf9802cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:46.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.256+0000 7fbf927fc640 1 -- 172.21.15.52:0/4138503313 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf98041070 con 0x7fbf9c102d70 2024-04-02T14:01:46.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.256+0000 7fbf927fc640 1 -- 172.21.15.52:0/4138503313 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf98004450 con 0x7fbf9c102d70 2024-04-02T14:01:46.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.256+0000 7fbfa360d640 1 -- 172.21.15.52:0/4138503313 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf9c19d5b0 con 0x7fbf9c102d70 2024-04-02T14:01:46.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.256+0000 7fbf927fc640 1 -- 172.21.15.52:0/4138503313 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf98003df0 con 0x7fbf9c102d70 2024-04-02T14:01:46.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.256+0000 7fbfa360d640 1 -- 172.21.15.52:0/4138503313 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf9c1a4ea0 con 0x7fbf9c102d70 2024-04-02T14:01:46.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.258+0000 7fbf927fc640 1 -- 172.21.15.52:0/4138503313 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbf98044070 con 0x7fbf9c102d70 2024-04-02T14:01:46.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.258+0000 7fbfa61f2740 1 Processor -- start 2024-04-02T14:01:46.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.258+0000 7fbfa61f2740 1 -- start start 2024-04-02T14:01:46.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.259+0000 7fbfa61f2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 0x5603db9f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.259+0000 7fbfa61f2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603db9f30c0 0x5603db9fb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.259+0000 7fbfa61f2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603db9fbca0 0x5603db9fe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.259+0000 7fbfa61f2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5603db9567b0 con 0x5603db9f27a0 2024-04-02T14:01:46.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.259+0000 7fbfa61f2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5603db8f5570 con 0x5603db9f30c0 2024-04-02T14:01:46.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.259+0000 7fbfa61f2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5603db952760 con 0x5603db9fbca0 2024-04-02T14:01:46.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.259+0000 7fbf927fc640 1 --2- 172.21.15.52:0/4138503313 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fbf880a3d20 0x7fbf880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.259+0000 7fbfa1baa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 0x5603db9f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.259+0000 7fbfa1baa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 0x5603db9f2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60402/0 (socket says 172.21.15.52:60402) 2024-04-02T14:01:46.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.259+0000 7fbfa1baa640 1 -- 172.21.15.52:0/1156853169 learned_addr learned my addr 172.21.15.52:0/1156853169 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:46.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.259+0000 7fbfa1baa640 1 --2- 172.21.15.52:0/4138503313 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fbf880a3d20 0x7fbf880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.259+0000 7fbf927fc640 1 -- 172.21.15.52:0/4138503313 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7fbf980f2660 con 0x7fbf9c102d70 2024-04-02T14:01:46.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.259+0000 7fbfa13a9640 1 --2- 172.21.15.52:0/1156853169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603db9f30c0 0x5603db9fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.259+0000 7fbfa0ba8640 1 --2- 172.21.15.52:0/1156853169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603db9fbca0 0x5603db9fe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.259+0000 7fbfa1baa640 1 -- 172.21.15.52:0/1156853169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603db9fbca0 msgr2=0x5603db9fe090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:46.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.259+0000 7fbfa1baa640 1 --2- 172.21.15.52:0/1156853169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603db9fbca0 0x5603db9fe090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.259+0000 7fbfa1baa640 1 -- 172.21.15.52:0/1156853169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603db9f30c0 msgr2=0x5603db9fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:46.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.259+0000 7fbfa1baa640 1 --2- 172.21.15.52:0/1156853169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603db9f30c0 0x5603db9fb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.259+0000 7fbfa1baa640 1 -- 172.21.15.52:0/1156853169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603db66a3f0 con 0x5603db9f27a0 2024-04-02T14:01:46.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.259+0000 7fbfa1baa640 1 --2- 172.21.15.52:0/1156853169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 0x5603db9f2b80 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fbf94014e80 tx=0x7fbf940117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9553915c6d8cbef4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:46.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.259+0000 7fbfa1baa640 1 --2- 172.21.15.52:0/4138503313 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fbf880a3d20 0x7fbf880a61e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbf94011cb0 tx=0x7fbf94016000 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:46.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.259+0000 7fbf877fe640 1 -- 172.21.15.52:0/1156853169 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf94024070 con 0x5603db9f27a0 2024-04-02T14:01:46.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.260+0000 7fbf877fe640 1 -- 172.21.15.52:0/1156853169 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf94007c60 con 0x5603db9f27a0 2024-04-02T14:01:46.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.260+0000 7fbf877fe640 1 -- 172.21.15.52:0/1156853169 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf940058d0 con 0x5603db9f27a0 2024-04-02T14:01:46.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.260+0000 7fbfa61f2740 1 -- 172.21.15.52:0/1156853169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 msgr2=0x5603db9f2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.260+0000 7fbfa61f2740 1 --2- 172.21.15.52:0/1156853169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 0x5603db9f2b80 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fbf94014e80 tx=0x7fbf940117b0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.260+0000 7fbfa61f2740 1 -- 172.21.15.52:0/1156853169 shutdown_connections 2024-04-02T14:01:46.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.260+0000 7fbfa61f2740 1 --2- 172.21.15.52:0/1156853169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603db9fbca0 0x5603db9fe090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.260+0000 7fbfa61f2740 1 --2- 172.21.15.52:0/1156853169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603db9f30c0 0x5603db9fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.260+0000 7fbfa61f2740 1 --2- 172.21.15.52:0/1156853169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 0x5603db9f2b80 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.260+0000 7fbfa61f2740 1 -- 172.21.15.52:0/1156853169 >> 172.21.15.52:0/1156853169 conn(0x5603db9eced0 msgr2=0x5603db9ed2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:46.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.260+0000 7fbfa61f2740 1 -- 172.21.15.52:0/1156853169 shutdown_connections 2024-04-02T14:01:46.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.260+0000 7fbfa61f2740 1 -- 172.21.15.52:0/1156853169 wait complete. 2024-04-02T14:01:46.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.261+0000 7fbfa61f2740 1 -- start start 2024-04-02T14:01:46.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.261+0000 7fbfa61f2740 1 Processor -- start 2024-04-02T14:01:46.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.261+0000 7fbfa61f2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 0x5603db9fb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.261+0000 7fbfa61f2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603db9f30c0 0x5603db9f8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.261+0000 7fbfa61f2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603db9fbca0 0x5603db9f9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.261+0000 7fbfa61f2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5603db9fea30 con 0x5603db9f27a0 2024-04-02T14:01:46.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.261+0000 7fbfa61f2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5603db911e80 con 0x5603db9f30c0 2024-04-02T14:01:46.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.261+0000 7fbfa61f2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5603db9febb0 con 0x5603db9fbca0 2024-04-02T14:01:46.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.261+0000 7fbfa61f2740 10 client.0 _ll_register_callbacks cb 0x5603db94daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:46.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.261+0000 7fbfa13a9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 0x5603db9fb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.261+0000 7fbfa13a9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 0x5603db9fb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60410/0 (socket says 172.21.15.52:60410) 2024-04-02T14:01:46.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.261+0000 7fbfa13a9640 1 -- 172.21.15.52:0/440992150 learned_addr learned my addr 172.21.15.52:0/440992150 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:46.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.261+0000 7fbfa0ba8640 1 --2- 172.21.15.52:0/440992150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603db9f30c0 0x5603db9f8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.261+0000 7fbfa1baa640 1 --2- 172.21.15.52:0/440992150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603db9fbca0 0x5603db9f9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.261+0000 7fbfa13a9640 1 -- 172.21.15.52:0/440992150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603db9fbca0 msgr2=0x5603db9f9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:46.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.261+0000 7fbfa13a9640 1 --2- 172.21.15.52:0/440992150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603db9fbca0 0x5603db9f9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.261+0000 7fbfa13a9640 1 -- 172.21.15.52:0/440992150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603db9f30c0 msgr2=0x5603db9f8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:46.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.262+0000 7fbfa13a9640 1 --2- 172.21.15.52:0/440992150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603db9f30c0 0x5603db9f8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.262+0000 7fbfa13a9640 1 -- 172.21.15.52:0/440992150 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603db7347d0 con 0x5603db9f27a0 2024-04-02T14:01:46.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.262+0000 7fbfa1baa640 1 --2- 172.21.15.52:0/440992150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603db9fbca0 0x5603db9f9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:46.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.262+0000 7fbfa0ba8640 1 --2- 172.21.15.52:0/440992150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603db9f30c0 0x5603db9f8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:01:46.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.262+0000 7fbfa13a9640 1 --2- 172.21.15.52:0/440992150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 0x5603db9fb750 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fbf980be080 tx=0x7fbf980bdda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36389b6ba3e5aa7d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:46.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.262+0000 7fbf84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:46.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.262+0000 7fbf84ff9640 1 -- 172.21.15.52:0/440992150 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf980f1070 con 0x5603db9f27a0 2024-04-02T14:01:46.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.262+0000 7fbf84ff9640 1 -- 172.21.15.52:0/440992150 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7fbf98041210 con 0x5603db9f27a0 2024-04-02T14:01:46.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.262+0000 7fbfa61f2740 1 -- 172.21.15.52:0/440992150 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf94007c60 con 0x5603db9f27a0 2024-04-02T14:01:46.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.262+0000 7fbf84ff9640 1 -- 172.21.15.52:0/440992150 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbf9804b040 con 0x5603db9f27a0 2024-04-02T14:01:46.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.262+0000 7fbfa61f2740 10 client.17982 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:46.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.262+0000 7fbfa61f2740 1 -- 172.21.15.52:0/440992150 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5603dba0aae0 con 0x5603db9f27a0 2024-04-02T14:01:46.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.262+0000 7fbf84ff9640 1 -- 172.21.15.52:0/440992150 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf94007c60 con 0x5603db9f27a0 2024-04-02T14:01:46.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.262+0000 7fbf84ff9640 1 -- 172.21.15.52:0/440992150 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fbf980dd160 con 0x5603db9f27a0 2024-04-02T14:01:46.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.262+0000 7fbfa61f2740 10 client.17982 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:46.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.262+0000 7fbfa61f2740 10 client.17982 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T14:01:46.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.262+0000 7fbfa61f2740 20 client.17982 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:46.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.262+0000 7fbfa61f2740 1 --2- 172.21.15.52:0/440992150 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5603dba0bed0 0x5603dba0e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.262+0000 7fbfa61f2740 4 client.17982 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:01:46.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.262+0000 7fbfa61f2740 1 -- 172.21.15.52:0/440992150 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603db963cf0 con 0x5603dba0bed0 2024-04-02T14:01:46.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.263+0000 7fbfa0ba8640 1 --2- 172.21.15.52:0/440992150 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5603dba0bed0 0x5603dba0e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.263+0000 7fbfa0ba8640 1 --2- 172.21.15.52:0/440992150 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5603dba0bed0 0x5603dba0e2c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=82fb4f3676301b27 server_cookie=edab5cf0641bb4f2 in_seq=0 out_seq=0 2024-04-02T14:01:46.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.263+0000 7fbf84ff9640 10 client.17982 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T14:01:46.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.264+0000 7fbf84ff9640 1 -- 172.21.15.52:0/440992150 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5603db963cf0 con 0x5603dba0bed0 2024-04-02T14:01:46.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.264+0000 7fbf84ff9640 10 client.17982 handle_command_reply: tid=0 2024-04-02T14:01:46.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.264+0000 7fbfa61f2740 1 client.17982 shutdown 2024-04-02T14:01:46.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.264+0000 7fbfa61f2740 1 -- 172.21.15.52:0/440992150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 msgr2=0x5603db9fb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.264+0000 7fbfa61f2740 1 --2- 172.21.15.52:0/440992150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 0x5603db9fb750 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fbf980be080 tx=0x7fbf980bdda0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.264+0000 7fbfa61f2740 1 -- 172.21.15.52:0/440992150 shutdown_connections 2024-04-02T14:01:46.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.264+0000 7fbfa61f2740 1 --2- 172.21.15.52:0/440992150 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5603dba0bed0 0x5603dba0e2c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.264+0000 7fbfa61f2740 1 --2- 172.21.15.52:0/440992150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603db9fbca0 0x5603db9f9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.264+0000 7fbfa61f2740 1 --2- 172.21.15.52:0/440992150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603db9f30c0 0x5603db9f8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.264+0000 7fbfa61f2740 1 --2- 172.21.15.52:0/440992150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 0x5603db9fb750 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.264+0000 7fbf84ff9640 0 client.17982 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T14:01:46.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.264+0000 7fbfa61f2740 1 -- 172.21.15.52:0/440992150 >> 172.21.15.52:0/440992150 conn(0x5603db9eced0 msgr2=0x5603db9eec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:46.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.264+0000 7fbfa61f2740 1 -- 172.21.15.52:0/440992150 shutdown_connections 2024-04-02T14:01:46.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.264+0000 7fbfa61f2740 1 -- 172.21.15.52:0/440992150 wait complete. 2024-04-02T14:01:46.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.264+0000 7fbfa61f2740 20 client.17982 trim_cache size 0 max 16384 2024-04-02T14:01:46.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.278+0000 7fbfa61f2740 1 Processor -- start 2024-04-02T14:01:46.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.278+0000 7fbfa61f2740 1 -- start start 2024-04-02T14:01:46.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.278+0000 7fbfa61f2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 0x5603db9ebdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.278+0000 7fbfa61f2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603db9f30c0 0x5603db9fb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.278+0000 7fbfa61f2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603db9fbca0 0x5603db9fb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.278+0000 7fbfa61f2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c1aed40 con 0x5603db9f27a0 2024-04-02T14:01:46.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.278+0000 7fbfa61f2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c1b0880 con 0x5603db9fbca0 2024-04-02T14:01:46.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.278+0000 7fbfa61f2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c1af410 con 0x5603db9f30c0 2024-04-02T14:01:46.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.278+0000 7fbfa13a9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 0x5603db9ebdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.278+0000 7fbfa13a9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 0x5603db9ebdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60422/0 (socket says 172.21.15.52:60422) 2024-04-02T14:01:46.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.278+0000 7fbfa13a9640 1 -- 172.21.15.52:0/2747382172 learned_addr learned my addr 172.21.15.52:0/2747382172 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:46.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.278+0000 7fbfa1baa640 1 --2- 172.21.15.52:0/2747382172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603db9fbca0 0x5603db9fb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.278+0000 7fbfa0ba8640 1 --2- 172.21.15.52:0/2747382172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603db9f30c0 0x5603db9fb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.278+0000 7fbfa13a9640 1 -- 172.21.15.52:0/2747382172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603db9f30c0 msgr2=0x5603db9fb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:46.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.278+0000 7fbfa13a9640 1 --2- 172.21.15.52:0/2747382172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603db9f30c0 0x5603db9fb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.278+0000 7fbfa13a9640 1 -- 172.21.15.52:0/2747382172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603db9fbca0 msgr2=0x5603db9fb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:46.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.278+0000 7fbfa13a9640 1 --2- 172.21.15.52:0/2747382172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603db9fbca0 0x5603db9fb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.278+0000 7fbfa13a9640 1 -- 172.21.15.52:0/2747382172 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603db9f9590 con 0x5603db9f27a0 2024-04-02T14:01:46.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.278+0000 7fbfa1baa640 1 --2- 172.21.15.52:0/2747382172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603db9fbca0 0x5603db9fb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:46.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.279+0000 7fbfa13a9640 1 --2- 172.21.15.52:0/2747382172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 0x5603db9ebdb0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fbf980ba7e0 tx=0x7fbf980bde10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=defe1b117d513cc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:46.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.279+0000 7fbf86ffd640 1 -- 172.21.15.52:0/2747382172 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf980f1040 con 0x5603db9f27a0 2024-04-02T14:01:46.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.279+0000 7fbf86ffd640 1 -- 172.21.15.52:0/2747382172 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf980f2040 con 0x5603db9f27a0 2024-04-02T14:01:46.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.279+0000 7fbf86ffd640 1 -- 172.21.15.52:0/2747382172 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf980c2730 con 0x5603db9f27a0 2024-04-02T14:01:46.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.279+0000 7fbfa61f2740 1 -- 172.21.15.52:0/2747382172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 msgr2=0x5603db9ebdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.279+0000 7fbfa61f2740 1 --2- 172.21.15.52:0/2747382172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 0x5603db9ebdb0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fbf980ba7e0 tx=0x7fbf980bde10 comp rx=0 tx=0).stop 2024-04-02T14:01:46.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.279+0000 7fbfa61f2740 1 -- 172.21.15.52:0/2747382172 shutdown_connections 2024-04-02T14:01:46.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.279+0000 7fbfa61f2740 1 --2- 172.21.15.52:0/2747382172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603db9fbca0 0x5603db9fb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.279+0000 7fbfa61f2740 1 --2- 172.21.15.52:0/2747382172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603db9f30c0 0x5603db9fb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.279+0000 7fbfa61f2740 1 --2- 172.21.15.52:0/2747382172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 0x5603db9ebdb0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.279+0000 7fbfa61f2740 1 -- 172.21.15.52:0/2747382172 >> 172.21.15.52:0/2747382172 conn(0x5603db9eced0 msgr2=0x5603db9eb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:46.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.279+0000 7fbfa61f2740 1 -- 172.21.15.52:0/2747382172 shutdown_connections 2024-04-02T14:01:46.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.279+0000 7fbfa61f2740 1 -- 172.21.15.52:0/2747382172 wait complete. 2024-04-02T14:01:46.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.280+0000 7fbfa61f2740 1 -- start start 2024-04-02T14:01:46.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.280+0000 7fbfa61f2740 1 Processor -- start 2024-04-02T14:01:46.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.280+0000 7fbfa61f2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 0x5603dba04260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.280+0000 7fbfa61f2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603db9f30c0 0x5603db9f5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.280+0000 7fbfa61f2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603db9fbca0 0x5603db9f5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.280+0000 7fbfa61f2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c1b01b0 con 0x5603db9f27a0 2024-04-02T14:01:46.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.280+0000 7fbfa61f2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c1afae0 con 0x5603db9fbca0 2024-04-02T14:01:46.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.280+0000 7fbfa61f2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c1acd40 con 0x5603db9f30c0 2024-04-02T14:01:46.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.280+0000 7fbfa61f2740 10 client.0 _ll_register_callbacks cb 0x7fbf98009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:46.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.280+0000 7fbfa13a9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 0x5603dba04260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.280+0000 7fbfa13a9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 0x5603dba04260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60436/0 (socket says 172.21.15.52:60436) 2024-04-02T14:01:46.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.280+0000 7fbfa13a9640 1 -- 172.21.15.52:0/2099622565 learned_addr learned my addr 172.21.15.52:0/2099622565 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:46.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.280+0000 7fbfa0ba8640 1 --2- 172.21.15.52:0/2099622565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603db9f30c0 0x5603db9f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.280+0000 7fbfa13a9640 1 -- 172.21.15.52:0/2099622565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603db9f30c0 msgr2=0x5603db9f5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:46.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.280+0000 7fbfa1baa640 1 --2- 172.21.15.52:0/2099622565 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603db9fbca0 0x5603db9f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.280+0000 7fbfa13a9640 1 --2- 172.21.15.52:0/2099622565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603db9f30c0 0x5603db9f5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.280+0000 7fbfa13a9640 1 -- 172.21.15.52:0/2099622565 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603db9fbca0 msgr2=0x5603db9f5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:46.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.280+0000 7fbfa13a9640 1 --2- 172.21.15.52:0/2099622565 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603db9fbca0 0x5603db9f5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.281+0000 7fbfa13a9640 1 -- 172.21.15.52:0/2099622565 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603db9f2ef0 con 0x5603db9f27a0 2024-04-02T14:01:46.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.281+0000 7fbfa1baa640 1 --2- 172.21.15.52:0/2099622565 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603db9fbca0 0x5603db9f5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:46.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.281+0000 7fbfa13a9640 1 --2- 172.21.15.52:0/2099622565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 0x5603dba04260 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fbf980be310 tx=0x7fbf980c6ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f0d98ed8f375d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:46.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.281+0000 7fbf84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:46.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.281+0000 7fbf84ff9640 1 -- 172.21.15.52:0/2099622565 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf980a7020 con 0x5603db9f27a0 2024-04-02T14:01:46.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.281+0000 7fbf84ff9640 1 -- 172.21.15.52:0/2099622565 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7fbf980f23f0 con 0x5603db9f27a0 2024-04-02T14:01:46.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.281+0000 7fbfa61f2740 1 -- 172.21.15.52:0/2099622565 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf980f2040 con 0x5603db9f27a0 2024-04-02T14:01:46.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.281+0000 7fbf84ff9640 1 -- 172.21.15.52:0/2099622565 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbf980cf050 con 0x5603db9f27a0 2024-04-02T14:01:46.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.281+0000 7fbfa61f2740 10 client.17988 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:46.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.281+0000 7fbfa61f2740 1 -- 172.21.15.52:0/2099622565 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5603db9f2ef0 con 0x5603db9f27a0 2024-04-02T14:01:46.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.281+0000 7fbf84ff9640 1 -- 172.21.15.52:0/2099622565 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf980f2040 con 0x5603db9f27a0 2024-04-02T14:01:46.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.281+0000 7fbf84ff9640 1 -- 172.21.15.52:0/2099622565 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fbf980a1020 con 0x5603db9f27a0 2024-04-02T14:01:46.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.281+0000 7fbfa61f2740 10 client.17988 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:46.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.281+0000 7fbfa61f2740 10 client.17988 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T14:01:46.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.281+0000 7fbfa61f2740 20 client.17988 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:46.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.281+0000 7fbfa61f2740 1 --2- 172.21.15.52:0/2099622565 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5603dba72620 0x5603dba74a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.281+0000 7fbfa61f2740 4 client.17988 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:01:46.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.281+0000 7fbfa61f2740 1 -- 172.21.15.52:0/2099622565 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5603db95e040 con 0x5603dba72620 2024-04-02T14:01:46.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.282+0000 7fbfa0ba8640 1 --2- 172.21.15.52:0/2099622565 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5603dba72620 0x5603dba74a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.282+0000 7fbfa0ba8640 1 --2- 172.21.15.52:0/2099622565 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5603dba72620 0x5603dba74a10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5af8fba3b4f7780 server_cookie=26a62fd2369a09 in_seq=0 out_seq=0 2024-04-02T14:01:46.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.282+0000 7fbf84ff9640 10 client.17988 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T14:01:46.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.282+0000 7fbf84ff9640 1 -- 172.21.15.52:0/2099622565 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5603db95e040 con 0x5603dba72620 2024-04-02T14:01:46.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.282+0000 7fbf84ff9640 10 client.17988 handle_command_reply: tid=0 2024-04-02T14:01:46.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.282+0000 7fbfa61f2740 1 client.17988 shutdown 2024-04-02T14:01:46.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.282+0000 7fbfa61f2740 1 -- 172.21.15.52:0/2099622565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 msgr2=0x5603dba04260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.282+0000 7fbfa61f2740 1 --2- 172.21.15.52:0/2099622565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 0x5603dba04260 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fbf980be310 tx=0x7fbf980c6ed0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.282+0000 7fbfa61f2740 1 -- 172.21.15.52:0/2099622565 shutdown_connections 2024-04-02T14:01:46.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.282+0000 7fbfa61f2740 1 --2- 172.21.15.52:0/2099622565 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603db9fbca0 0x5603db9f5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.282+0000 7fbfa61f2740 1 --2- 172.21.15.52:0/2099622565 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x5603dba72620 0x5603dba74a10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.282+0000 7fbfa61f2740 1 --2- 172.21.15.52:0/2099622565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603db9f30c0 0x5603db9f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.282+0000 7fbf84ff9640 0 client.17988 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T14:01:46.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.282+0000 7fbfa61f2740 1 --2- 172.21.15.52:0/2099622565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603db9f27a0 0x5603dba04260 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.282+0000 7fbfa61f2740 1 -- 172.21.15.52:0/2099622565 >> 172.21.15.52:0/2099622565 conn(0x5603db9eced0 msgr2=0x5603db9ebf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:46.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.282+0000 7fbfa61f2740 1 -- 172.21.15.52:0/2099622565 shutdown_connections 2024-04-02T14:01:46.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.282+0000 7fbfa61f2740 1 -- 172.21.15.52:0/2099622565 wait complete. 2024-04-02T14:01:46.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.283+0000 7fbfa61f2740 20 client.17988 trim_cache size 0 max 16384 2024-04-02T14:01:46.299 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:01:46.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.283+0000 7fbfa360d640 1 -- 172.21.15.52:0/4138503313 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fbf880a3d20 msgr2=0x7fbf880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.283+0000 7fbfa360d640 1 --2- 172.21.15.52:0/4138503313 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fbf880a3d20 0x7fbf880a61e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbf94011cb0 tx=0x7fbf94016000 comp rx=0 tx=0).stop 2024-04-02T14:01:46.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.283+0000 7fbfa360d640 1 -- 172.21.15.52:0/4138503313 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf9c102d70 msgr2=0x7fbf9c198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.283+0000 7fbfa360d640 1 --2- 172.21.15.52:0/4138503313 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf9c102d70 0x7fbf9c198330 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fbf98005940 tx=0x7fbf9802cc40 comp rx=0 tx=0).stop 2024-04-02T14:01:46.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.283+0000 7fbfa360d640 1 -- 172.21.15.52:0/4138503313 shutdown_connections 2024-04-02T14:01:46.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.283+0000 7fbfa360d640 1 --2- 172.21.15.52:0/4138503313 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fbf880a3d20 0x7fbf880a61e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.283+0000 7fbfa360d640 1 --2- 172.21.15.52:0/4138503313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf9c10c140 0x7fbf9c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.283+0000 7fbfa360d640 1 --2- 172.21.15.52:0/4138503313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf9c103690 0x7fbf9c198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.283+0000 7fbfa360d640 1 --2- 172.21.15.52:0/4138503313 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf9c102d70 0x7fbf9c198330 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.283+0000 7fbfa360d640 1 -- 172.21.15.52:0/4138503313 >> 172.21.15.52:0/4138503313 conn(0x7fbf9c077b60 msgr2=0x7fbf9c0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:46.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.283+0000 7fbfa360d640 1 -- 172.21.15.52:0/4138503313 shutdown_connections 2024-04-02T14:01:46.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.283+0000 7fbfa360d640 1 -- 172.21.15.52:0/4138503313 wait complete. 2024-04-02T14:01:46.302 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T14:01:46.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.404+0000 7f591cba3640 1 -- 172.21.15.52:0/2008311664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5918101ed0 msgr2=0x7f591810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.404+0000 7f591cba3640 1 --2- 172.21.15.52:0/2008311664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5918101ed0 0x7f591810d880 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f5908009ca0 tx=0x7f590802c0f0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.404+0000 7f591cba3640 1 -- 172.21.15.52:0/2008311664 shutdown_connections 2024-04-02T14:01:46.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.404+0000 7f591cba3640 1 --2- 172.21.15.52:0/2008311664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5918101ed0 0x7f591810d880 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.404+0000 7f591cba3640 1 --2- 172.21.15.52:0/2008311664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5918101510 0x7f5918101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.404+0000 7f591cba3640 1 --2- 172.21.15.52:0/2008311664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59181059a0 0x7f5918105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.404+0000 7f591cba3640 1 -- 172.21.15.52:0/2008311664 >> 172.21.15.52:0/2008311664 conn(0x7f5918075c60 msgr2=0x7f5918076070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:46.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.404+0000 7f591cba3640 1 -- 172.21.15.52:0/2008311664 shutdown_connections 2024-04-02T14:01:46.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.404+0000 7f591cba3640 1 -- 172.21.15.52:0/2008311664 wait complete. 2024-04-02T14:01:46.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.405+0000 7f591cba3640 1 Processor -- start 2024-04-02T14:01:46.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.405+0000 7f591cba3640 1 -- start start 2024-04-02T14:01:46.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.405+0000 7f591cba3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5918101510 0x7f59181982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.405+0000 7f591cba3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5918101ed0 0x7f59181987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.405+0000 7f591cba3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59181059a0 0x7f591819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.405+0000 7f591cba3640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5918111ed0 con 0x7f5918101510 2024-04-02T14:01:46.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.405+0000 7f591cba3640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5918111d50 con 0x7f5918101ed0 2024-04-02T14:01:46.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.405+0000 7f591cba3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5918112050 con 0x7f59181059a0 2024-04-02T14:01:46.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.405+0000 7f591659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5918101510 0x7f59181982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.405+0000 7f591659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5918101510 0x7f59181982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60452/0 (socket says 172.21.15.52:60452) 2024-04-02T14:01:46.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.405+0000 7f591659c640 1 -- 172.21.15.52:0/1359733604 learned_addr learned my addr 172.21.15.52:0/1359733604 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:46.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.405+0000 7f5916d9d640 1 --2- 172.21.15.52:0/1359733604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59181059a0 0x7f591819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.405+0000 7f5915d9b640 1 --2- 172.21.15.52:0/1359733604 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5918101ed0 0x7f59181987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.405+0000 7f591659c640 1 -- 172.21.15.52:0/1359733604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59181059a0 msgr2=0x7f591819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.405+0000 7f591659c640 1 --2- 172.21.15.52:0/1359733604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59181059a0 0x7f591819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.405+0000 7f591659c640 1 -- 172.21.15.52:0/1359733604 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5918101ed0 msgr2=0x7f59181987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.405+0000 7f591659c640 1 --2- 172.21.15.52:0/1359733604 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5918101ed0 0x7f59181987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.405+0000 7f591659c640 1 -- 172.21.15.52:0/1359733604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f591819d2a0 con 0x7f5918101510 2024-04-02T14:01:46.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.405+0000 7f5915d9b640 1 --2- 172.21.15.52:0/1359733604 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5918101ed0 0x7f59181987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:46.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.405+0000 7f5916d9d640 1 --2- 172.21.15.52:0/1359733604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59181059a0 0x7f591819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:46.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.405+0000 7f591659c640 1 --2- 172.21.15.52:0/1359733604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5918101510 0x7f59181982b0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f591000d770 tx=0x7f591000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:46.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.406+0000 7f590f7fe640 1 -- 172.21.15.52:0/1359733604 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910017070 con 0x7f5918101510 2024-04-02T14:01:46.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.406+0000 7f591cba3640 1 -- 172.21.15.52:0/1359733604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f591819d590 con 0x7f5918101510 2024-04-02T14:01:46.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.406+0000 7f590f7fe640 1 -- 172.21.15.52:0/1359733604 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5910009c30 con 0x7f5918101510 2024-04-02T14:01:46.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.406+0000 7f591cba3640 1 -- 172.21.15.52:0/1359733604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59181a4e20 con 0x7f5918101510 2024-04-02T14:01:46.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.406+0000 7f590f7fe640 1 -- 172.21.15.52:0/1359733604 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910020060 con 0x7f5918101510 2024-04-02T14:01:46.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.408+0000 7f590f7fe640 1 -- 172.21.15.52:0/1359733604 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5910003d70 con 0x7f5918101510 2024-04-02T14:01:46.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.408+0000 7f591f782740 1 Processor -- start 2024-04-02T14:01:46.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.408+0000 7f591f782740 1 -- start start 2024-04-02T14:01:46.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.408+0000 7f590f7fe640 1 --2- 172.21.15.52:0/1359733604 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f59040a3d20 0x7f59040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.408+0000 7f591f782740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559aad5767a0 0x559aad576b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.408+0000 7f591f782740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aad5770c0 0x559aad57f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.408+0000 7f591f782740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad57fca0 0x559aad582090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.408+0000 7f591f782740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559aad4da7b0 con 0x559aad57fca0 2024-04-02T14:01:46.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.408+0000 7f591f782740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559aad479570 con 0x559aad5767a0 2024-04-02T14:01:46.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.408+0000 7f591f782740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559aad4d6760 con 0x559aad5770c0 2024-04-02T14:01:46.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.409+0000 7f590f7fe640 1 -- 172.21.15.52:0/1359733604 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f59100d0930 con 0x7f5918101510 2024-04-02T14:01:46.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.409+0000 7f5916d9d640 1 --2- 172.21.15.52:0/1359733604 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f59040a3d20 0x7f59040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.409+0000 7f5916d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad57fca0 0x559aad582090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.409+0000 7f5916d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad57fca0 0x559aad582090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60456/0 (socket says 172.21.15.52:60456) 2024-04-02T14:01:46.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.409+0000 7f5916d9d640 1 -- 172.21.15.52:0/3088991881 learned_addr learned my addr 172.21.15.52:0/3088991881 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:46.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.409+0000 7f591659c640 1 --2- 172.21.15.52:0/3088991881 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559aad5767a0 0x559aad576b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.409+0000 7f5915d9b640 1 --2- 172.21.15.52:0/3088991881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aad5770c0 0x559aad57f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.409+0000 7f5916d9d640 1 -- 172.21.15.52:0/3088991881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aad5770c0 msgr2=0x559aad57f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:46.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.409+0000 7f5916d9d640 1 --2- 172.21.15.52:0/3088991881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aad5770c0 0x559aad57f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.409+0000 7f5916d9d640 1 -- 172.21.15.52:0/3088991881 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559aad5767a0 msgr2=0x559aad576b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:46.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.409+0000 7f5916d9d640 1 --2- 172.21.15.52:0/3088991881 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559aad5767a0 0x559aad576b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.409+0000 7f5916d9d640 1 -- 172.21.15.52:0/3088991881 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aad1ee3f0 con 0x559aad57fca0 2024-04-02T14:01:46.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.409+0000 7f5916d9d640 1 --2- 172.21.15.52:0/3088991881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad57fca0 0x559aad582090 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f5908009ca0 tx=0x7f59080329a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d84fdd17654e92c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:46.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.409+0000 7f5916d9d640 1 --2- 172.21.15.52:0/1359733604 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f59040a3d20 0x7f59040a61e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5908009a60 tx=0x7f590802e000 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:46.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.409+0000 7f590cff9640 1 -- 172.21.15.52:0/3088991881 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908050070 con 0x559aad57fca0 2024-04-02T14:01:46.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.409+0000 7f590cff9640 1 -- 172.21.15.52:0/3088991881 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5908004460 con 0x559aad57fca0 2024-04-02T14:01:46.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.409+0000 7f590cff9640 1 -- 172.21.15.52:0/3088991881 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908039040 con 0x559aad57fca0 2024-04-02T14:01:46.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.410+0000 7f591f782740 1 -- 172.21.15.52:0/3088991881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad57fca0 msgr2=0x559aad582090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.410+0000 7f591f782740 1 --2- 172.21.15.52:0/3088991881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad57fca0 0x559aad582090 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f5908009ca0 tx=0x7f59080329a0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.410+0000 7f591f782740 1 -- 172.21.15.52:0/3088991881 shutdown_connections 2024-04-02T14:01:46.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.410+0000 7f591f782740 1 --2- 172.21.15.52:0/3088991881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad57fca0 0x559aad582090 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.410+0000 7f591f782740 1 --2- 172.21.15.52:0/3088991881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aad5770c0 0x559aad57f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.410+0000 7f591f782740 1 --2- 172.21.15.52:0/3088991881 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559aad5767a0 0x559aad576b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.410+0000 7f591f782740 1 -- 172.21.15.52:0/3088991881 >> 172.21.15.52:0/3088991881 conn(0x559aad570ed0 msgr2=0x559aad5712b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:46.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.410+0000 7f591f782740 1 -- 172.21.15.52:0/3088991881 shutdown_connections 2024-04-02T14:01:46.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.410+0000 7f591f782740 1 -- 172.21.15.52:0/3088991881 wait complete. 2024-04-02T14:01:46.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.410+0000 7f591f782740 1 -- start start 2024-04-02T14:01:46.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.411+0000 7f591f782740 1 Processor -- start 2024-04-02T14:01:46.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.411+0000 7f591f782740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad5767a0 0x559aad57f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.411+0000 7f591f782740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aad5770c0 0x559aad57ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.411+0000 7f591f782740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559aad57fca0 0x559aad57d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.411+0000 7f591f782740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559aad582a30 con 0x559aad5767a0 2024-04-02T14:01:46.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.411+0000 7f591f782740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559aad495e80 con 0x559aad57fca0 2024-04-02T14:01:46.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.411+0000 7f591f782740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559aad582bb0 con 0x559aad5770c0 2024-04-02T14:01:46.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.411+0000 7f591f782740 10 client.0 _ll_register_callbacks cb 0x559aad4d1aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:46.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.411+0000 7f591659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad5767a0 0x559aad57f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.411+0000 7f591659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad5767a0 0x559aad57f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60470/0 (socket says 172.21.15.52:60470) 2024-04-02T14:01:46.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.411+0000 7f591659c640 1 -- 172.21.15.52:0/2028928634 learned_addr learned my addr 172.21.15.52:0/2028928634 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:46.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.411+0000 7f5916d9d640 1 --2- 172.21.15.52:0/2028928634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559aad57fca0 0x559aad57d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.411+0000 7f5915d9b640 1 --2- 172.21.15.52:0/2028928634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aad5770c0 0x559aad57ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.411+0000 7f591659c640 1 -- 172.21.15.52:0/2028928634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aad5770c0 msgr2=0x559aad57ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:46.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.411+0000 7f591659c640 1 --2- 172.21.15.52:0/2028928634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aad5770c0 0x559aad57ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.411+0000 7f591659c640 1 -- 172.21.15.52:0/2028928634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559aad57fca0 msgr2=0x559aad57d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:46.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.411+0000 7f591659c640 1 --2- 172.21.15.52:0/2028928634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559aad57fca0 0x559aad57d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.411+0000 7f591659c640 1 -- 172.21.15.52:0/2028928634 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aad2b87d0 con 0x559aad5767a0 2024-04-02T14:01:46.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.411+0000 7f5915d9b640 1 --2- 172.21.15.52:0/2028928634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aad5770c0 0x559aad57ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:46.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.411+0000 7f591659c640 1 --2- 172.21.15.52:0/2028928634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad5767a0 0x559aad57f6b0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f5910009610 tx=0x7f591009c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d0448837492427 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:46.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.411+0000 7f58fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:46.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.412+0000 7f58fdffb640 1 -- 172.21.15.52:0/2028928634 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591009c540 con 0x559aad5767a0 2024-04-02T14:01:46.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.412+0000 7f58fdffb640 1 -- 172.21.15.52:0/2028928634 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f59100cdf70 con 0x559aad5767a0 2024-04-02T14:01:46.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.412+0000 7f591f782740 1 -- 172.21.15.52:0/2028928634 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5908004460 con 0x559aad5767a0 2024-04-02T14:01:46.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.412+0000 7f58fdffb640 1 -- 172.21.15.52:0/2028928634 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f59100a08f0 con 0x559aad5767a0 2024-04-02T14:01:46.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.412+0000 7f591f782740 10 client.18000 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:46.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.412+0000 7f591f782740 1 -- 172.21.15.52:0/2028928634 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559aad58eac0 con 0x559aad5767a0 2024-04-02T14:01:46.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.412+0000 7f58fdffb640 1 -- 172.21.15.52:0/2028928634 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5908004460 con 0x559aad5767a0 2024-04-02T14:01:46.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.412+0000 7f58fdffb640 1 -- 172.21.15.52:0/2028928634 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5910098bb0 con 0x559aad5767a0 2024-04-02T14:01:46.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.412+0000 7f591f782740 10 client.18000 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:46.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.412+0000 7f591f782740 10 client.18000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:01:46.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.412+0000 7f591f782740 20 client.18000 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:46.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.412+0000 7f591f782740 1 --2- 172.21.15.52:0/2028928634 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x559aad58ff70 0x559aad5923c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.412+0000 7f591f782740 4 client.18000 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:01:46.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.412+0000 7f591f782740 1 -- 172.21.15.52:0/2028928634 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559aad4e7cf0 con 0x559aad58ff70 2024-04-02T14:01:46.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.412+0000 7f5915d9b640 1 --2- 172.21.15.52:0/2028928634 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x559aad58ff70 0x559aad5923c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.413+0000 7f58fdffb640 10 client.18000 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:46.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.413+0000 7f5915d9b640 1 --2- 172.21.15.52:0/2028928634 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x559aad58ff70 0x559aad5923c0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c51642dadca87d4f server_cookie=8579b8a25eac9948 in_seq=0 out_seq=0 2024-04-02T14:01:46.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.413+0000 7f58fdffb640 1 -- 172.21.15.52:0/2028928634 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559aad4e7cf0 con 0x559aad58ff70 2024-04-02T14:01:46.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.413+0000 7f58fdffb640 10 client.18000 handle_command_reply: tid=0 2024-04-02T14:01:46.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.413+0000 7f591f782740 1 client.18000 shutdown 2024-04-02T14:01:46.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.414+0000 7f591f782740 1 -- 172.21.15.52:0/2028928634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad5767a0 msgr2=0x559aad57f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.414+0000 7f591f782740 1 --2- 172.21.15.52:0/2028928634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad5767a0 0x559aad57f6b0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f5910009610 tx=0x7f591009c140 comp rx=0 tx=0).stop 2024-04-02T14:01:46.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.414+0000 7f591f782740 1 -- 172.21.15.52:0/2028928634 shutdown_connections 2024-04-02T14:01:46.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.414+0000 7f591f782740 1 --2- 172.21.15.52:0/2028928634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559aad57fca0 0x559aad57d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.414+0000 7f591f782740 1 --2- 172.21.15.52:0/2028928634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aad5770c0 0x559aad57ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.414+0000 7f591f782740 1 --2- 172.21.15.52:0/2028928634 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x559aad58ff70 0x559aad5923c0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.414+0000 7f591f782740 1 --2- 172.21.15.52:0/2028928634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad5767a0 0x559aad57f6b0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.414+0000 7f58fdffb640 0 client.18000 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:46.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.414+0000 7f591f782740 1 -- 172.21.15.52:0/2028928634 >> 172.21.15.52:0/2028928634 conn(0x559aad570ed0 msgr2=0x559aad572c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:46.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.414+0000 7f591f782740 1 -- 172.21.15.52:0/2028928634 shutdown_connections 2024-04-02T14:01:46.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.414+0000 7f591f782740 1 -- 172.21.15.52:0/2028928634 wait complete. 2024-04-02T14:01:46.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.414+0000 7f591f782740 20 client.18000 trim_cache size 0 max 16384 2024-04-02T14:01:46.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.427+0000 7f591f782740 1 Processor -- start 2024-04-02T14:01:46.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.427+0000 7f591f782740 1 -- start start 2024-04-02T14:01:46.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.427+0000 7f591f782740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad5767a0 0x559aad58b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.427+0000 7f591f782740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aad5770c0 0x559aad57ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.427+0000 7f591f782740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559aad57fca0 0x559aad57f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.427+0000 7f591f782740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f59181aee10 con 0x559aad5767a0 2024-04-02T14:01:46.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.427+0000 7f591f782740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f59181b0950 con 0x559aad57fca0 2024-04-02T14:01:46.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.427+0000 7f591f782740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59181af4e0 con 0x559aad5770c0 2024-04-02T14:01:46.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.427+0000 7f591659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad5767a0 0x559aad58b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.427+0000 7f591659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad5767a0 0x559aad58b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60482/0 (socket says 172.21.15.52:60482) 2024-04-02T14:01:46.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.427+0000 7f591659c640 1 -- 172.21.15.52:0/1964848368 learned_addr learned my addr 172.21.15.52:0/1964848368 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:46.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.427+0000 7f591659c640 1 -- 172.21.15.52:0/1964848368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aad5770c0 msgr2=0x559aad57ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:46.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.427+0000 7f591659c640 1 --2- 172.21.15.52:0/1964848368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aad5770c0 0x559aad57ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.427+0000 7f5916d9d640 1 --2- 172.21.15.52:0/1964848368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559aad57fca0 0x559aad57f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.427+0000 7f591659c640 1 -- 172.21.15.52:0/1964848368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559aad57fca0 msgr2=0x559aad57f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:46.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.427+0000 7f591659c640 1 --2- 172.21.15.52:0/1964848368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559aad57fca0 0x559aad57f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.427+0000 7f591659c640 1 -- 172.21.15.52:0/1964848368 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aad57d3c0 con 0x559aad5767a0 2024-04-02T14:01:46.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.427+0000 7f5916d9d640 1 --2- 172.21.15.52:0/1964848368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559aad57fca0 0x559aad57f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:46.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.428+0000 7f591659c640 1 --2- 172.21.15.52:0/1964848368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad5767a0 0x559aad58b950 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f591009c4d0 tx=0x7f591009aa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2480dd657e8b9895 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:46.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.428+0000 7f58fffff640 1 -- 172.21.15.52:0/1964848368 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59100a51b0 con 0x559aad5767a0 2024-04-02T14:01:46.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.428+0000 7f58fffff640 1 -- 172.21.15.52:0/1964848368 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5910015070 con 0x559aad5767a0 2024-04-02T14:01:46.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.428+0000 7f58fffff640 1 -- 172.21.15.52:0/1964848368 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59100a3840 con 0x559aad5767a0 2024-04-02T14:01:46.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.428+0000 7f591f782740 1 -- 172.21.15.52:0/1964848368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad5767a0 msgr2=0x559aad58b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.428+0000 7f591f782740 1 --2- 172.21.15.52:0/1964848368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad5767a0 0x559aad58b950 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f591009c4d0 tx=0x7f591009aa50 comp rx=0 tx=0).stop 2024-04-02T14:01:46.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.428+0000 7f591f782740 1 -- 172.21.15.52:0/1964848368 shutdown_connections 2024-04-02T14:01:46.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.428+0000 7f591f782740 1 --2- 172.21.15.52:0/1964848368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559aad57fca0 0x559aad57f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.428+0000 7f591f782740 1 --2- 172.21.15.52:0/1964848368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aad5770c0 0x559aad57ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.428+0000 7f591f782740 1 --2- 172.21.15.52:0/1964848368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad5767a0 0x559aad58b950 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.428+0000 7f591f782740 1 -- 172.21.15.52:0/1964848368 >> 172.21.15.52:0/1964848368 conn(0x559aad570ed0 msgr2=0x559aad57b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:46.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.428+0000 7f591f782740 1 -- 172.21.15.52:0/1964848368 shutdown_connections 2024-04-02T14:01:46.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.428+0000 7f591f782740 1 -- 172.21.15.52:0/1964848368 wait complete. 2024-04-02T14:01:46.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.429+0000 7f591f782740 1 -- start start 2024-04-02T14:01:46.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.429+0000 7f591f782740 1 Processor -- start 2024-04-02T14:01:46.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.429+0000 7f591f782740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559aad5767a0 0x559aad587e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.429+0000 7f591f782740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad5770c0 0x559aad5883a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.429+0000 7f591f782740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aad57fca0 0x559aad579d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.429+0000 7f591f782740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f59181b0280 con 0x559aad5770c0 2024-04-02T14:01:46.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.429+0000 7f591f782740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f59181afbb0 con 0x559aad5767a0 2024-04-02T14:01:46.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.429+0000 7f591f782740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59181ace10 con 0x559aad57fca0 2024-04-02T14:01:46.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.429+0000 7f591f782740 10 client.0 _ll_register_callbacks cb 0x7f591000de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:46.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.429+0000 7f5915d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad5770c0 0x559aad5883a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.429+0000 7f5915d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad5770c0 0x559aad5883a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60486/0 (socket says 172.21.15.52:60486) 2024-04-02T14:01:46.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.429+0000 7f5915d9b640 1 -- 172.21.15.52:0/1885035289 learned_addr learned my addr 172.21.15.52:0/1885035289 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:46.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.429+0000 7f591659c640 1 --2- 172.21.15.52:0/1885035289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559aad5767a0 0x559aad587e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.429+0000 7f5916d9d640 1 --2- 172.21.15.52:0/1885035289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aad57fca0 0x559aad579d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.429+0000 7f5915d9b640 1 -- 172.21.15.52:0/1885035289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aad57fca0 msgr2=0x559aad579d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:46.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.429+0000 7f5915d9b640 1 --2- 172.21.15.52:0/1885035289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aad57fca0 0x559aad579d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.429+0000 7f5915d9b640 1 -- 172.21.15.52:0/1885035289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559aad5767a0 msgr2=0x559aad587e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:46.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.429+0000 7f5915d9b640 1 --2- 172.21.15.52:0/1885035289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559aad5767a0 0x559aad587e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.429+0000 7f5915d9b640 1 -- 172.21.15.52:0/1885035289 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aad57a530 con 0x559aad5770c0 2024-04-02T14:01:46.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.429+0000 7f591659c640 1 --2- 172.21.15.52:0/1885035289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559aad5767a0 0x559aad587e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:46.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.429+0000 7f5916d9d640 1 --2- 172.21.15.52:0/1885035289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aad57fca0 0x559aad579d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:46.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.430+0000 7f5915d9b640 1 --2- 172.21.15.52:0/1885035289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad5770c0 0x559aad5883a0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f59180762d0 tx=0x7f5900012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbafe237e1b067a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:46.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.430+0000 7f58fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:46.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.430+0000 7f58fdffb640 1 -- 172.21.15.52:0/1885035289 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900019330 con 0x559aad5770c0 2024-04-02T14:01:46.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.430+0000 7f58fdffb640 1 -- 172.21.15.52:0/1885035289 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f59000194d0 con 0x559aad5770c0 2024-04-02T14:01:46.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.430+0000 7f591f782740 1 -- 172.21.15.52:0/1885035289 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5910015070 con 0x559aad5770c0 2024-04-02T14:01:46.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.430+0000 7f58fdffb640 1 -- 172.21.15.52:0/1885035289 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5900029050 con 0x559aad5770c0 2024-04-02T14:01:46.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.430+0000 7f591f782740 10 client.18006 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:46.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.430+0000 7f591f782740 1 -- 172.21.15.52:0/1885035289 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559aad580d50 con 0x559aad5770c0 2024-04-02T14:01:46.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.430+0000 7f58fdffb640 1 -- 172.21.15.52:0/1885035289 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5910015070 con 0x559aad5770c0 2024-04-02T14:01:46.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.430+0000 7f58fdffb640 1 -- 172.21.15.52:0/1885035289 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5900010c80 con 0x559aad5770c0 2024-04-02T14:01:46.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.430+0000 7f591f782740 10 client.18006 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:46.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.430+0000 7f591f782740 10 client.18006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:01:46.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.430+0000 7f591f782740 20 client.18006 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:46.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.430+0000 7f591f782740 1 --2- 172.21.15.52:0/1885035289 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x559aad5f66e0 0x559aad5f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:46.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.430+0000 7f591f782740 4 client.18006 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:01:46.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.430+0000 7f591f782740 1 -- 172.21.15.52:0/1885035289 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559aad4e2040 con 0x559aad5f66e0 2024-04-02T14:01:46.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.431+0000 7f591659c640 1 --2- 172.21.15.52:0/1885035289 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x559aad5f66e0 0x559aad5f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:46.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.431+0000 7f591659c640 1 --2- 172.21.15.52:0/1885035289 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x559aad5f66e0 0x559aad5f8b00 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55d711745e1b07d3 server_cookie=9f981492f47f9abd in_seq=0 out_seq=0 2024-04-02T14:01:46.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.431+0000 7f58fdffb640 10 client.18006 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:46.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.431+0000 7f58fdffb640 1 -- 172.21.15.52:0/1885035289 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559aad4e2040 con 0x559aad5f66e0 2024-04-02T14:01:46.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.431+0000 7f58fdffb640 10 client.18006 handle_command_reply: tid=0 2024-04-02T14:01:46.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.431+0000 7f591f782740 1 client.18006 shutdown 2024-04-02T14:01:46.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.431+0000 7f591f782740 1 -- 172.21.15.52:0/1885035289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad5770c0 msgr2=0x559aad5883a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.431+0000 7f591f782740 1 --2- 172.21.15.52:0/1885035289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad5770c0 0x559aad5883a0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f59180762d0 tx=0x7f5900012820 comp rx=0 tx=0).stop 2024-04-02T14:01:46.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.431+0000 7f591f782740 1 -- 172.21.15.52:0/1885035289 shutdown_connections 2024-04-02T14:01:46.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.431+0000 7f591f782740 1 --2- 172.21.15.52:0/1885035289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559aad57fca0 0x559aad579d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.431+0000 7f591f782740 1 --2- 172.21.15.52:0/1885035289 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x559aad5f66e0 0x559aad5f8b00 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.431+0000 7f591f782740 1 --2- 172.21.15.52:0/1885035289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559aad5770c0 0x559aad5883a0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.431+0000 7f58fdffb640 0 client.18006 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:46.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.431+0000 7f591f782740 1 --2- 172.21.15.52:0/1885035289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559aad5767a0 0x559aad587e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.431+0000 7f591f782740 1 -- 172.21.15.52:0/1885035289 >> 172.21.15.52:0/1885035289 conn(0x559aad570ed0 msgr2=0x559aad572300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:46.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.431+0000 7f591f782740 1 -- 172.21.15.52:0/1885035289 shutdown_connections 2024-04-02T14:01:46.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.431+0000 7f591f782740 1 -- 172.21.15.52:0/1885035289 wait complete. 2024-04-02T14:01:46.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.431+0000 7f591f782740 20 client.18006 trim_cache size 0 max 16384 2024-04-02T14:01:46.451 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:01:46.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.432+0000 7f591cba3640 1 -- 172.21.15.52:0/1359733604 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f59040a3d20 msgr2=0x7f59040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.432+0000 7f591cba3640 1 --2- 172.21.15.52:0/1359733604 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f59040a3d20 0x7f59040a61e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5908009a60 tx=0x7f590802e000 comp rx=0 tx=0).stop 2024-04-02T14:01:46.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.432+0000 7f591cba3640 1 -- 172.21.15.52:0/1359733604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5918101510 msgr2=0x7f59181982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:46.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.432+0000 7f591cba3640 1 --2- 172.21.15.52:0/1359733604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5918101510 0x7f59181982b0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f591000d770 tx=0x7f591000dc30 comp rx=0 tx=0).stop 2024-04-02T14:01:46.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.432+0000 7f591cba3640 1 -- 172.21.15.52:0/1359733604 shutdown_connections 2024-04-02T14:01:46.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.432+0000 7f591cba3640 1 --2- 172.21.15.52:0/1359733604 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f59040a3d20 0x7f59040a61e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.432+0000 7f591cba3640 1 --2- 172.21.15.52:0/1359733604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59181059a0 0x7f591819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.432+0000 7f591cba3640 1 --2- 172.21.15.52:0/1359733604 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5918101ed0 0x7f59181987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.432+0000 7f591cba3640 1 --2- 172.21.15.52:0/1359733604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5918101510 0x7f59181982b0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:46.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.432+0000 7f591cba3640 1 -- 172.21.15.52:0/1359733604 >> 172.21.15.52:0/1359733604 conn(0x7f5918075c60 msgr2=0x7f59180ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:46.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.432+0000 7f591cba3640 1 -- 172.21.15.52:0/1359733604 shutdown_connections 2024-04-02T14:01:46.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:46.432+0000 7f591cba3640 1 -- 172.21.15.52:0/1359733604 wait complete. 2024-04-02T14:01:46.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:46 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/207356327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:01:46.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:46 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/207356327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:01:47.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:46 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/207356327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:01:47.455 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T14:01:47.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:01:47.455 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T14:01:47.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.570+0000 7f00de2c5640 1 -- 172.21.15.52:0/506992770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00d810e470 msgr2=0x7f00d8110860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:47.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.570+0000 7f00de2c5640 1 --2- 172.21.15.52:0/506992770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00d810e470 0x7f00d8110860 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009ca0 tx=0x7f00d002c0f0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.570+0000 7f00de2c5640 1 -- 172.21.15.52:0/506992770 shutdown_connections 2024-04-02T14:01:47.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.570+0000 7f00de2c5640 1 --2- 172.21.15.52:0/506992770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00d810e470 0x7f00d8110860 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.570+0000 7f00de2c5640 1 --2- 172.21.15.52:0/506992770 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00d81011e0 0x7f00d810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.570+0000 7f00de2c5640 1 --2- 172.21.15.52:0/506992770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00d81008c0 0x7f00d8100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.570+0000 7f00de2c5640 1 -- 172.21.15.52:0/506992770 >> 172.21.15.52:0/506992770 conn(0x7f00d8076d80 msgr2=0x7f00d8077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:47.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.570+0000 7f00de2c5640 1 -- 172.21.15.52:0/506992770 shutdown_connections 2024-04-02T14:01:47.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.570+0000 7f00de2c5640 1 -- 172.21.15.52:0/506992770 wait complete. 2024-04-02T14:01:47.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.571+0000 7f00de2c5640 1 Processor -- start 2024-04-02T14:01:47.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.571+0000 7f00de2c5640 1 -- start start 2024-04-02T14:01:47.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.571+0000 7f00de2c5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00d81008c0 0x7f00d819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.571+0000 7f00de2c5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00d81011e0 0x7f00d819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.571+0000 7f00de2c5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00d810e470 0x7f00d819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.571+0000 7f00de2c5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f00d8114070 con 0x7f00d81008c0 2024-04-02T14:01:47.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.571+0000 7f00de2c5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f00d8113ef0 con 0x7f00d810e470 2024-04-02T14:01:47.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.571+0000 7f00de2c5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81141f0 con 0x7f00d81011e0 2024-04-02T14:01:47.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.571+0000 7f00d7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00d81008c0 0x7f00d819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.571+0000 7f00d7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00d81008c0 0x7f00d819a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60512/0 (socket says 172.21.15.52:60512) 2024-04-02T14:01:47.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.571+0000 7f00d7fff640 1 -- 172.21.15.52:0/1750610923 learned_addr learned my addr 172.21.15.52:0/1750610923 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:47.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.572+0000 7f00dc862640 1 --2- 172.21.15.52:0/1750610923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00d810e470 0x7f00d819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.572+0000 7f00d77fe640 1 --2- 172.21.15.52:0/1750610923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00d81011e0 0x7f00d819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.572+0000 7f00d7fff640 1 -- 172.21.15.52:0/1750610923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00d81011e0 msgr2=0x7f00d819aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:47.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.572+0000 7f00d7fff640 1 --2- 172.21.15.52:0/1750610923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00d81011e0 0x7f00d819aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.572+0000 7f00d7fff640 1 -- 172.21.15.52:0/1750610923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00d810e470 msgr2=0x7f00d819edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:47.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.572+0000 7f00d7fff640 1 --2- 172.21.15.52:0/1750610923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00d810e470 0x7f00d819edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.572+0000 7f00d7fff640 1 -- 172.21.15.52:0/1750610923 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d819f500 con 0x7f00d81008c0 2024-04-02T14:01:47.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.572+0000 7f00d77fe640 1 --2- 172.21.15.52:0/1750610923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00d81011e0 0x7f00d819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:47.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.572+0000 7f00dc862640 1 --2- 172.21.15.52:0/1750610923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00d810e470 0x7f00d819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:47.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.572+0000 7f00d7fff640 1 --2- 172.21.15.52:0/1750610923 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00d81008c0 0x7f00d819a510 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f00c800c920 tx=0x7f00c800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:47.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.572+0000 7f00d57fa640 1 -- 172.21.15.52:0/1750610923 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8007c70 con 0x7f00d81008c0 2024-04-02T14:01:47.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.572+0000 7f00d57fa640 1 -- 172.21.15.52:0/1750610923 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00c8007e10 con 0x7f00d81008c0 2024-04-02T14:01:47.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.572+0000 7f00de2c5640 1 -- 172.21.15.52:0/1750610923 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00d819f7f0 con 0x7f00d81008c0 2024-04-02T14:01:47.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.572+0000 7f00d57fa640 1 -- 172.21.15.52:0/1750610923 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8018890 con 0x7f00d81008c0 2024-04-02T14:01:47.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.572+0000 7f00de2c5640 1 -- 172.21.15.52:0/1750610923 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00d81a7080 con 0x7f00d81008c0 2024-04-02T14:01:47.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.574+0000 7f00d57fa640 1 -- 172.21.15.52:0/1750610923 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f00c8018a70 con 0x7f00d81008c0 2024-04-02T14:01:47.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.575+0000 7f00e0f86740 1 Processor -- start 2024-04-02T14:01:47.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.575+0000 7f00e0f86740 1 -- start start 2024-04-02T14:01:47.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.575+0000 7f00e0f86740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e1a2c0 0x556a60e1a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.575+0000 7f00e0f86740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a60e1abe0 0x556a60e23280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.575+0000 7f00e0f86740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556a60e237c0 0x556a60e25bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.575+0000 7f00e0f86740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556a60d7e770 con 0x556a60e1a2c0 2024-04-02T14:01:47.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.575+0000 7f00e0f86740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556a60d26960 con 0x556a60e1abe0 2024-04-02T14:01:47.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.575+0000 7f00e0f86740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556a60d7a8c0 con 0x556a60e237c0 2024-04-02T14:01:47.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.575+0000 7f00d57fa640 1 --2- 172.21.15.52:0/1750610923 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f00c00a3d20 0x7f00c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.575+0000 7f00dc862640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e1a2c0 0x556a60e1a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.575+0000 7f00dc862640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e1a2c0 0x556a60e1a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60520/0 (socket says 172.21.15.52:60520) 2024-04-02T14:01:47.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.575+0000 7f00dc862640 1 -- 172.21.15.52:0/3882435165 learned_addr learned my addr 172.21.15.52:0/3882435165 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:47.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.575+0000 7f00d57fa640 1 -- 172.21.15.52:0/1750610923 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f00c80d0dc0 con 0x7f00d81008c0 2024-04-02T14:01:47.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.575+0000 7f00dc862640 1 --2- 172.21.15.52:0/1750610923 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f00c00a3d20 0x7f00c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.575+0000 7f00d7fff640 1 --2- 172.21.15.52:0/3882435165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a60e1abe0 0x556a60e23280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.575+0000 7f00d77fe640 1 --2- 172.21.15.52:0/3882435165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556a60e237c0 0x556a60e25bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.575+0000 7f00dc862640 1 -- 172.21.15.52:0/3882435165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556a60e237c0 msgr2=0x556a60e25bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:47.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.575+0000 7f00dc862640 1 --2- 172.21.15.52:0/3882435165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556a60e237c0 0x556a60e25bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.575+0000 7f00dc862640 1 -- 172.21.15.52:0/3882435165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a60e1abe0 msgr2=0x556a60e23280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:47.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.575+0000 7f00dc862640 1 --2- 172.21.15.52:0/3882435165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a60e1abe0 0x556a60e23280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.575+0000 7f00dc862640 1 -- 172.21.15.52:0/3882435165 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a60b5a7e0 con 0x556a60e1a2c0 2024-04-02T14:01:47.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.576+0000 7f00d77fe640 1 --2- 172.21.15.52:0/3882435165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556a60e237c0 0x556a60e25bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:47.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.576+0000 7f00dc862640 1 --2- 172.21.15.52:0/3882435165 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e1a2c0 0x556a60e1a6a0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f00d0032530 tx=0x7f00d0032690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef555483d908aee server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:47.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.576+0000 7f00dc862640 1 --2- 172.21.15.52:0/1750610923 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f00c00a3d20 0x7f00c00a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f00d8102aa0 tx=0x7f00d0041000 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:47.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.576+0000 7f00be7fc640 1 -- 172.21.15.52:0/3882435165 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d0051070 con 0x556a60e1a2c0 2024-04-02T14:01:47.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.576+0000 7f00be7fc640 1 -- 172.21.15.52:0/3882435165 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00d0048070 con 0x556a60e1a2c0 2024-04-02T14:01:47.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.576+0000 7f00be7fc640 1 -- 172.21.15.52:0/3882435165 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d0039040 con 0x556a60e1a2c0 2024-04-02T14:01:47.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.576+0000 7f00e0f86740 1 -- 172.21.15.52:0/3882435165 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e1a2c0 msgr2=0x556a60e1a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:47.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.576+0000 7f00e0f86740 1 --2- 172.21.15.52:0/3882435165 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e1a2c0 0x556a60e1a6a0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f00d0032530 tx=0x7f00d0032690 comp rx=0 tx=0).stop 2024-04-02T14:01:47.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.577+0000 7f00e0f86740 1 -- 172.21.15.52:0/3882435165 shutdown_connections 2024-04-02T14:01:47.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.577+0000 7f00e0f86740 1 --2- 172.21.15.52:0/3882435165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556a60e237c0 0x556a60e25bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.577+0000 7f00e0f86740 1 --2- 172.21.15.52:0/3882435165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a60e1abe0 0x556a60e23280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.577+0000 7f00e0f86740 1 --2- 172.21.15.52:0/3882435165 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e1a2c0 0x556a60e1a6a0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.577+0000 7f00e0f86740 1 -- 172.21.15.52:0/3882435165 >> 172.21.15.52:0/3882435165 conn(0x556a60e149f0 msgr2=0x556a60e14dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:47.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.577+0000 7f00e0f86740 1 -- 172.21.15.52:0/3882435165 shutdown_connections 2024-04-02T14:01:47.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.577+0000 7f00e0f86740 1 -- 172.21.15.52:0/3882435165 wait complete. 2024-04-02T14:01:47.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.577+0000 7f00e0f86740 1 -- start start 2024-04-02T14:01:47.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.577+0000 7f00e0f86740 1 Processor -- start 2024-04-02T14:01:47.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.577+0000 7f00e0f86740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e1a2c0 0x556a60e20760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.577+0000 7f00e0f86740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556a60e1abe0 0x556a60e20ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.577+0000 7f00e0f86740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a60e237c0 0x556a60e211e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.577+0000 7f00e0f86740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556a60e26550 con 0x556a60e1a2c0 2024-04-02T14:01:47.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.577+0000 7f00e0f86740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556a60d39d20 con 0x556a60e237c0 2024-04-02T14:01:47.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.577+0000 7f00e0f86740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556a60e266d0 con 0x556a60e1abe0 2024-04-02T14:01:47.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.577+0000 7f00e0f86740 10 client.0 _ll_register_callbacks cb 0x556a60d06a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:47.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.578+0000 7f00d7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e1a2c0 0x556a60e20760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.578+0000 7f00d7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e1a2c0 0x556a60e20760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60524/0 (socket says 172.21.15.52:60524) 2024-04-02T14:01:47.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.578+0000 7f00d7fff640 1 -- 172.21.15.52:0/3139437991 learned_addr learned my addr 172.21.15.52:0/3139437991 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:47.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.578+0000 7f00d77fe640 1 --2- 172.21.15.52:0/3139437991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556a60e1abe0 0x556a60e20ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.578+0000 7f00dc862640 1 --2- 172.21.15.52:0/3139437991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a60e237c0 0x556a60e211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.578+0000 7f00d7fff640 1 -- 172.21.15.52:0/3139437991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556a60e1abe0 msgr2=0x556a60e20ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:47.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.578+0000 7f00d7fff640 1 --2- 172.21.15.52:0/3139437991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556a60e1abe0 0x556a60e20ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.578+0000 7f00d7fff640 1 -- 172.21.15.52:0/3139437991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a60e237c0 msgr2=0x556a60e211e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:47.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.578+0000 7f00d7fff640 1 --2- 172.21.15.52:0/3139437991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a60e237c0 0x556a60e211e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.578+0000 7f00d7fff640 1 -- 172.21.15.52:0/3139437991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a60bbaf70 con 0x556a60e1a2c0 2024-04-02T14:01:47.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.578+0000 7f00d7fff640 1 --2- 172.21.15.52:0/3139437991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e1a2c0 0x556a60e20760 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f00c809c6d0 tx=0x7f00c809aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367137072be379d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:47.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.578+0000 7f00b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:47.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.578+0000 7f00b77fe640 1 -- 172.21.15.52:0/3139437991 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c800de20 con 0x556a60e1a2c0 2024-04-02T14:01:47.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.578+0000 7f00b77fe640 1 -- 172.21.15.52:0/3139437991 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f00c801dd40 con 0x556a60e1a2c0 2024-04-02T14:01:47.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.578+0000 7f00e0f86740 1 -- 172.21.15.52:0/3139437991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00d0048070 con 0x556a60e1a2c0 2024-04-02T14:01:47.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.578+0000 7f00b77fe640 1 -- 172.21.15.52:0/3139437991 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f00c809c9d0 con 0x556a60e1a2c0 2024-04-02T14:01:47.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.578+0000 7f00e0f86740 10 client.18018 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:47.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.578+0000 7f00e0f86740 1 -- 172.21.15.52:0/3139437991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556a60a52170 con 0x556a60e1a2c0 2024-04-02T14:01:47.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.579+0000 7f00b77fe640 1 -- 172.21.15.52:0/3139437991 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00d0048070 con 0x556a60e1a2c0 2024-04-02T14:01:47.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.579+0000 7f00b77fe640 1 -- 172.21.15.52:0/3139437991 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f00c8099040 con 0x556a60e1a2c0 2024-04-02T14:01:47.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.579+0000 7f00e0f86740 10 client.18018 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:47.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.579+0000 7f00e0f86740 10 client.18018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:01:47.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.579+0000 7f00e0f86740 20 client.18018 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:47.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.579+0000 7f00e0f86740 1 --2- 172.21.15.52:0/3139437991 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556a60e338a0 0x556a60e35cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.579+0000 7f00e0f86740 4 client.18018 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:01:47.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.579+0000 7f00e0f86740 1 -- 172.21.15.52:0/3139437991 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a60d8bcb0 con 0x556a60e338a0 2024-04-02T14:01:47.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.579+0000 7f00d77fe640 1 --2- 172.21.15.52:0/3139437991 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556a60e338a0 0x556a60e35cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.579+0000 7f00b77fe640 10 client.18018 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:47.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.579+0000 7f00d77fe640 1 --2- 172.21.15.52:0/3139437991 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556a60e338a0 0x556a60e35cf0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e65d73b1e93b45e3 server_cookie=95eb23efa95e98b3 in_seq=0 out_seq=0 2024-04-02T14:01:47.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.580+0000 7f00b77fe640 1 -- 172.21.15.52:0/3139437991 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556a60d8bcb0 con 0x556a60e338a0 2024-04-02T14:01:47.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.580+0000 7f00b77fe640 10 client.18018 handle_command_reply: tid=0 2024-04-02T14:01:47.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.580+0000 7f00e0f86740 1 client.18018 shutdown 2024-04-02T14:01:47.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.580+0000 7f00e0f86740 1 -- 172.21.15.52:0/3139437991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e1a2c0 msgr2=0x556a60e20760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:47.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.580+0000 7f00e0f86740 1 --2- 172.21.15.52:0/3139437991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e1a2c0 0x556a60e20760 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f00c809c6d0 tx=0x7f00c809aee0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.580+0000 7f00e0f86740 1 -- 172.21.15.52:0/3139437991 shutdown_connections 2024-04-02T14:01:47.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.580+0000 7f00e0f86740 1 --2- 172.21.15.52:0/3139437991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a60e237c0 0x556a60e211e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.580+0000 7f00e0f86740 1 --2- 172.21.15.52:0/3139437991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556a60e1abe0 0x556a60e20ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.580+0000 7f00e0f86740 1 --2- 172.21.15.52:0/3139437991 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556a60e338a0 0x556a60e35cf0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.580+0000 7f00e0f86740 1 --2- 172.21.15.52:0/3139437991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e1a2c0 0x556a60e20760 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.580+0000 7f00e0f86740 1 -- 172.21.15.52:0/3139437991 >> 172.21.15.52:0/3139437991 conn(0x556a60e149f0 msgr2=0x556a60e16740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:47.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.580+0000 7f00b77fe640 0 client.18018 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:47.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.580+0000 7f00e0f86740 1 -- 172.21.15.52:0/3139437991 shutdown_connections 2024-04-02T14:01:47.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.580+0000 7f00e0f86740 1 -- 172.21.15.52:0/3139437991 wait complete. 2024-04-02T14:01:47.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.581+0000 7f00e0f86740 20 client.18018 trim_cache size 0 max 16384 2024-04-02T14:01:47.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.598+0000 7f00e0f86740 1 Processor -- start 2024-04-02T14:01:47.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.598+0000 7f00e0f86740 1 -- start start 2024-04-02T14:01:47.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.599+0000 7f00e0f86740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e1a2c0 0x556a60e22e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.599+0000 7f00e0f86740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a60e1abe0 0x556a60e2f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.599+0000 7f00e0f86740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556a60e237c0 0x556a60e13930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.599+0000 7f00e0f86740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81b1090 con 0x556a60e1a2c0 2024-04-02T14:01:47.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.599+0000 7f00e0f86740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81b2bd0 con 0x556a60e1abe0 2024-04-02T14:01:47.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.599+0000 7f00e0f86740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81b1760 con 0x556a60e237c0 2024-04-02T14:01:47.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.599+0000 7f00d7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e1a2c0 0x556a60e22e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.599+0000 7f00d7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e1a2c0 0x556a60e22e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60536/0 (socket says 172.21.15.52:60536) 2024-04-02T14:01:47.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.599+0000 7f00d7fff640 1 -- 172.21.15.52:0/359168569 learned_addr learned my addr 172.21.15.52:0/359168569 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:47.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.599+0000 7f00dc862640 1 --2- 172.21.15.52:0/359168569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556a60e237c0 0x556a60e13930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.599+0000 7f00d77fe640 1 --2- 172.21.15.52:0/359168569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a60e1abe0 0x556a60e2f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.599+0000 7f00d7fff640 1 -- 172.21.15.52:0/359168569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556a60e237c0 msgr2=0x556a60e13930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:47.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.599+0000 7f00d7fff640 1 --2- 172.21.15.52:0/359168569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556a60e237c0 0x556a60e13930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.599+0000 7f00d7fff640 1 -- 172.21.15.52:0/359168569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a60e1abe0 msgr2=0x556a60e2f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:47.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.599+0000 7f00d7fff640 1 --2- 172.21.15.52:0/359168569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a60e1abe0 0x556a60e2f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.599+0000 7f00d7fff640 1 -- 172.21.15.52:0/359168569 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a60e0eec0 con 0x556a60e1a2c0 2024-04-02T14:01:47.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.599+0000 7f00d77fe640 1 --2- 172.21.15.52:0/359168569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a60e1abe0 0x556a60e2f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:01:47.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.599+0000 7f00d7fff640 1 --2- 172.21.15.52:0/359168569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e1a2c0 0x556a60e22e90 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f00c80993a0 tx=0x7f00c800fe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d3e2b3a44262a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:47.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.599+0000 7f00bdffb640 1 -- 172.21.15.52:0/359168569 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c80a1f10 con 0x556a60e1a2c0 2024-04-02T14:01:47.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.599+0000 7f00bdffb640 1 -- 172.21.15.52:0/359168569 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00c800de20 con 0x556a60e1a2c0 2024-04-02T14:01:47.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.599+0000 7f00bdffb640 1 -- 172.21.15.52:0/359168569 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c809cc30 con 0x556a60e1a2c0 2024-04-02T14:01:47.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.600+0000 7f00e0f86740 1 -- 172.21.15.52:0/359168569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e1a2c0 msgr2=0x556a60e22e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:47.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.600+0000 7f00e0f86740 1 --2- 172.21.15.52:0/359168569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e1a2c0 0x556a60e22e90 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f00c80993a0 tx=0x7f00c800fe90 comp rx=0 tx=0).stop 2024-04-02T14:01:47.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.600+0000 7f00e0f86740 1 -- 172.21.15.52:0/359168569 shutdown_connections 2024-04-02T14:01:47.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.600+0000 7f00e0f86740 1 --2- 172.21.15.52:0/359168569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556a60e237c0 0x556a60e13930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.600+0000 7f00e0f86740 1 --2- 172.21.15.52:0/359168569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a60e1abe0 0x556a60e2f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.600+0000 7f00e0f86740 1 --2- 172.21.15.52:0/359168569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e1a2c0 0x556a60e22e90 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.600+0000 7f00e0f86740 1 -- 172.21.15.52:0/359168569 >> 172.21.15.52:0/359168569 conn(0x556a60e149f0 msgr2=0x556a60e13220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:47.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.600+0000 7f00e0f86740 1 -- 172.21.15.52:0/359168569 shutdown_connections 2024-04-02T14:01:47.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.600+0000 7f00e0f86740 1 -- 172.21.15.52:0/359168569 wait complete. 2024-04-02T14:01:47.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.600+0000 7f00e0f86740 1 -- start start 2024-04-02T14:01:47.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.601+0000 7f00e0f86740 1 Processor -- start 2024-04-02T14:01:47.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.601+0000 7f00e0f86740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a60e1a2c0 0x556a60e27b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.601+0000 7f00e0f86740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556a60e1abe0 0x556a60e28060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.601+0000 7f00e0f86740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e237c0 0x556a60e98e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.601+0000 7f00e0f86740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81b2500 con 0x556a60e237c0 2024-04-02T14:01:47.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.601+0000 7f00e0f86740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81b1e30 con 0x556a60e1a2c0 2024-04-02T14:01:47.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.601+0000 7f00e0f86740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81af090 con 0x556a60e1abe0 2024-04-02T14:01:47.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.601+0000 7f00e0f86740 10 client.0 _ll_register_callbacks cb 0x7f00c800c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:47.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.601+0000 7f00dc862640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e237c0 0x556a60e98e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.601+0000 7f00dc862640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e237c0 0x556a60e98e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60548/0 (socket says 172.21.15.52:60548) 2024-04-02T14:01:47.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.601+0000 7f00dc862640 1 -- 172.21.15.52:0/1552844393 learned_addr learned my addr 172.21.15.52:0/1552844393 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:47.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.601+0000 7f00d77fe640 1 --2- 172.21.15.52:0/1552844393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556a60e1abe0 0x556a60e28060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.601+0000 7f00d7fff640 1 --2- 172.21.15.52:0/1552844393 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a60e1a2c0 0x556a60e27b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.601+0000 7f00dc862640 1 -- 172.21.15.52:0/1552844393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556a60e1abe0 msgr2=0x556a60e28060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:47.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.601+0000 7f00dc862640 1 --2- 172.21.15.52:0/1552844393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556a60e1abe0 0x556a60e28060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.601+0000 7f00dc862640 1 -- 172.21.15.52:0/1552844393 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a60e1a2c0 msgr2=0x556a60e27b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:47.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.601+0000 7f00dc862640 1 --2- 172.21.15.52:0/1552844393 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a60e1a2c0 0x556a60e27b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.601+0000 7f00dc862640 1 -- 172.21.15.52:0/1552844393 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a60e993c0 con 0x556a60e237c0 2024-04-02T14:01:47.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.602+0000 7f00dc862640 1 --2- 172.21.15.52:0/1552844393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e237c0 0x556a60e98e30 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009a60 tx=0x7f00d0003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34d218a6c78a8b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:47.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.602+0000 7f00b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:47.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.602+0000 7f00b77fe640 1 -- 172.21.15.52:0/1552844393 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d005a060 con 0x556a60e237c0 2024-04-02T14:01:47.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.602+0000 7f00b77fe640 1 -- 172.21.15.52:0/1552844393 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f00d0004240 con 0x556a60e237c0 2024-04-02T14:01:47.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.602+0000 7f00e0f86740 1 -- 172.21.15.52:0/1552844393 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00c800de20 con 0x556a60e237c0 2024-04-02T14:01:47.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.602+0000 7f00b77fe640 1 -- 172.21.15.52:0/1552844393 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f00d0061050 con 0x556a60e237c0 2024-04-02T14:01:47.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.602+0000 7f00e0f86740 10 client.18024 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:47.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.602+0000 7f00e0f86740 1 -- 172.21.15.52:0/1552844393 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556a60e9a670 con 0x556a60e237c0 2024-04-02T14:01:47.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.602+0000 7f00b77fe640 1 -- 172.21.15.52:0/1552844393 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00c800de20 con 0x556a60e237c0 2024-04-02T14:01:47.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.602+0000 7f00b77fe640 1 -- 172.21.15.52:0/1552844393 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f00d0034440 con 0x556a60e237c0 2024-04-02T14:01:47.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.602+0000 7f00e0f86740 10 client.18024 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:47.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.602+0000 7f00e0f86740 10 client.18024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:01:47.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.602+0000 7f00e0f86740 20 client.18024 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:47.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.602+0000 7f00e0f86740 1 --2- 172.21.15.52:0/1552844393 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556a60e9a9a0 0x556a60e9cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.602+0000 7f00e0f86740 4 client.18024 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T14:01:47.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.602+0000 7f00e0f86740 1 -- 172.21.15.52:0/1552844393 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556a60d86000 con 0x556a60e9a9a0 2024-04-02T14:01:47.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.603+0000 7f00d7fff640 1 --2- 172.21.15.52:0/1552844393 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556a60e9a9a0 0x556a60e9cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.603+0000 7f00d7fff640 1 --2- 172.21.15.52:0/1552844393 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556a60e9a9a0 0x556a60e9cd90 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c01ac7833be452f server_cookie=4d10efe5f1a9603 in_seq=0 out_seq=0 2024-04-02T14:01:47.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.603+0000 7f00b77fe640 10 client.18024 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:47.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.604+0000 7f00b77fe640 1 -- 172.21.15.52:0/1552844393 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556a60d86000 con 0x556a60e9a9a0 2024-04-02T14:01:47.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.604+0000 7f00b77fe640 10 client.18024 handle_command_reply: tid=0 2024-04-02T14:01:47.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.604+0000 7f00e0f86740 1 client.18024 shutdown 2024-04-02T14:01:47.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.604+0000 7f00e0f86740 1 -- 172.21.15.52:0/1552844393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e237c0 msgr2=0x556a60e98e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:47.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.604+0000 7f00e0f86740 1 --2- 172.21.15.52:0/1552844393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e237c0 0x556a60e98e30 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009a60 tx=0x7f00d0003de0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.604+0000 7f00e0f86740 1 -- 172.21.15.52:0/1552844393 shutdown_connections 2024-04-02T14:01:47.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.604+0000 7f00e0f86740 1 --2- 172.21.15.52:0/1552844393 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556a60e9a9a0 0x556a60e9cd90 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.604+0000 7f00e0f86740 1 --2- 172.21.15.52:0/1552844393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a60e237c0 0x556a60e98e30 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.604+0000 7f00e0f86740 1 --2- 172.21.15.52:0/1552844393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556a60e1abe0 0x556a60e28060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.604+0000 7f00b77fe640 0 client.18024 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:47.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.604+0000 7f00e0f86740 1 --2- 172.21.15.52:0/1552844393 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a60e1a2c0 0x556a60e27b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.604+0000 7f00e0f86740 1 -- 172.21.15.52:0/1552844393 >> 172.21.15.52:0/1552844393 conn(0x556a60e149f0 msgr2=0x556a60e16600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:47.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.604+0000 7f00e0f86740 1 -- 172.21.15.52:0/1552844393 shutdown_connections 2024-04-02T14:01:47.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.604+0000 7f00e0f86740 1 -- 172.21.15.52:0/1552844393 wait complete. 2024-04-02T14:01:47.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.605+0000 7f00e0f86740 20 client.18024 trim_cache size 0 max 16384 2024-04-02T14:01:47.617 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:01:47.617 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-04-02T14:01:47.618 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "02d561e8-e8d2-4569-8e8f-599bb3000c26", 2024-04-02T14:01:47.618 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-04-02T14:01:47.618 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:01:47.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.605+0000 7f00de2c5640 1 -- 172.21.15.52:0/1750610923 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f00c00a3d20 msgr2=0x7f00c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:47.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.605+0000 7f00de2c5640 1 --2- 172.21.15.52:0/1750610923 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f00c00a3d20 0x7f00c00a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f00d8102aa0 tx=0x7f00d0041000 comp rx=0 tx=0).stop 2024-04-02T14:01:47.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.605+0000 7f00de2c5640 1 -- 172.21.15.52:0/1750610923 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00d81008c0 msgr2=0x7f00d819a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:47.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.605+0000 7f00de2c5640 1 --2- 172.21.15.52:0/1750610923 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00d81008c0 0x7f00d819a510 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f00c800c920 tx=0x7f00c800cde0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.605+0000 7f00de2c5640 1 -- 172.21.15.52:0/1750610923 shutdown_connections 2024-04-02T14:01:47.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.605+0000 7f00de2c5640 1 --2- 172.21.15.52:0/1750610923 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f00c00a3d20 0x7f00c00a61e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.605+0000 7f00de2c5640 1 --2- 172.21.15.52:0/1750610923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00d810e470 0x7f00d819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.605+0000 7f00de2c5640 1 --2- 172.21.15.52:0/1750610923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00d81011e0 0x7f00d819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.605+0000 7f00de2c5640 1 --2- 172.21.15.52:0/1750610923 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00d81008c0 0x7f00d819a510 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.605+0000 7f00de2c5640 1 -- 172.21.15.52:0/1750610923 >> 172.21.15.52:0/1750610923 conn(0x7f00d8076d80 msgr2=0x7f00d810f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:47.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.605+0000 7f00de2c5640 1 -- 172.21.15.52:0/1750610923 shutdown_connections 2024-04-02T14:01:47.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.605+0000 7f00de2c5640 1 -- 172.21.15.52:0/1750610923 wait complete. 2024-04-02T14:01:47.621 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:01:47.621 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:01:47.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.729+0000 7f1f596e4640 1 -- 172.21.15.52:0/933027484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f54107a90 msgr2=0x7f1f54107e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:47.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.729+0000 7f1f596e4640 1 --2- 172.21.15.52:0/933027484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f54107a90 0x7f1f54107e70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1f44009890 tx=0x7f1f4402bb70 comp rx=0 tx=0).stop 2024-04-02T14:01:47.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.729+0000 7f1f596e4640 1 -- 172.21.15.52:0/933027484 shutdown_connections 2024-04-02T14:01:47.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.729+0000 7f1f596e4640 1 --2- 172.21.15.52:0/933027484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f54102440 0x7f1f5410e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.729+0000 7f1f596e4640 1 --2- 172.21.15.52:0/933027484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f54101a80 0x7f1f54101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.729+0000 7f1f596e4640 1 --2- 172.21.15.52:0/933027484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f54107a90 0x7f1f54107e70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.729+0000 7f1f596e4640 1 -- 172.21.15.52:0/933027484 >> 172.21.15.52:0/933027484 conn(0x7f1f540fd590 msgr2=0x7f1f540ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:47.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.729+0000 7f1f596e4640 1 -- 172.21.15.52:0/933027484 shutdown_connections 2024-04-02T14:01:47.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.729+0000 7f1f596e4640 1 -- 172.21.15.52:0/933027484 wait complete. 2024-04-02T14:01:47.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.729+0000 7f1f596e4640 1 Processor -- start 2024-04-02T14:01:47.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.730+0000 7f1f596e4640 1 -- start start 2024-04-02T14:01:47.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.730+0000 7f1f596e4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f54101a80 0x7f1f54073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.730+0000 7f1f596e4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f54102440 0x7f1f54074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.730+0000 7f1f596e4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f54107a90 0x7f1f54074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.730+0000 7f1f596e4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1f54111e70 con 0x7f1f54101a80 2024-04-02T14:01:47.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.730+0000 7f1f596e4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1f54111cf0 con 0x7f1f54107a90 2024-04-02T14:01:47.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.730+0000 7f1f596e4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1f54111ff0 con 0x7f1f54102440 2024-04-02T14:01:47.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.730+0000 7f1f52ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f54101a80 0x7f1f54073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.730+0000 7f1f52ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f54101a80 0x7f1f54073e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60562/0 (socket says 172.21.15.52:60562) 2024-04-02T14:01:47.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.730+0000 7f1f52ffd640 1 -- 172.21.15.52:0/2332525229 learned_addr learned my addr 172.21.15.52:0/2332525229 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:47.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.730+0000 7f1f527fc640 1 --2- 172.21.15.52:0/2332525229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f54102440 0x7f1f54074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.730+0000 7f1f537fe640 1 --2- 172.21.15.52:0/2332525229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f54107a90 0x7f1f54074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.730+0000 7f1f52ffd640 1 -- 172.21.15.52:0/2332525229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f54102440 msgr2=0x7f1f54074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:47.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.730+0000 7f1f52ffd640 1 --2- 172.21.15.52:0/2332525229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f54102440 0x7f1f54074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.730+0000 7f1f52ffd640 1 -- 172.21.15.52:0/2332525229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f54107a90 msgr2=0x7f1f54074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:47.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.730+0000 7f1f52ffd640 1 --2- 172.21.15.52:0/2332525229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f54107a90 0x7f1f54074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.730+0000 7f1f52ffd640 1 -- 172.21.15.52:0/2332525229 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f541a6900 con 0x7f1f54101a80 2024-04-02T14:01:47.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.730+0000 7f1f527fc640 1 --2- 172.21.15.52:0/2332525229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f54102440 0x7f1f54074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:47.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.730+0000 7f1f537fe640 1 --2- 172.21.15.52:0/2332525229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f54107a90 0x7f1f54074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:47.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.731+0000 7f1f52ffd640 1 --2- 172.21.15.52:0/2332525229 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f54101a80 0x7f1f54073e00 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f1f4402c180 tx=0x7f1f44004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:47.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.731+0000 7f1f3bfff640 1 -- 172.21.15.52:0/2332525229 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f44041020 con 0x7f1f54101a80 2024-04-02T14:01:47.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.731+0000 7f1f596e4640 1 -- 172.21.15.52:0/2332525229 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f541a6b90 con 0x7f1f54101a80 2024-04-02T14:01:47.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.731+0000 7f1f3bfff640 1 -- 172.21.15.52:0/2332525229 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f4402c9f0 con 0x7f1f54101a80 2024-04-02T14:01:47.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.731+0000 7f1f3bfff640 1 -- 172.21.15.52:0/2332525229 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f44039040 con 0x7f1f54101a80 2024-04-02T14:01:47.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.731+0000 7f1f596e4640 1 -- 172.21.15.52:0/2332525229 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f541a70d0 con 0x7f1f54101a80 2024-04-02T14:01:47.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.733+0000 7f1f3bfff640 1 -- 172.21.15.52:0/2332525229 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1f44044020 con 0x7f1f54101a80 2024-04-02T14:01:47.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.734+0000 7f1f5c3a5740 1 Processor -- start 2024-04-02T14:01:47.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.734+0000 7f1f3bfff640 1 --2- 172.21.15.52:0/2332525229 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1f3c0a3d20 0x7f1f3c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.734+0000 7f1f5c3a5740 1 -- start start 2024-04-02T14:01:47.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.734+0000 7f1f537fe640 1 --2- 172.21.15.52:0/2332525229 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1f3c0a3d20 0x7f1f3c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.734+0000 7f1f3bfff640 1 -- 172.21.15.52:0/2332525229 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f1f440f2400 con 0x7f1f54101a80 2024-04-02T14:01:47.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.734+0000 7f1f5c3a5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b568d0 0x562484b56cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.734+0000 7f1f5c3a5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562484b571f0 0x562484b5f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.734+0000 7f1f5c3a5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b5fdd0 0x562484b621c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.734+0000 7f1f5c3a5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562484aba830 con 0x562484b5fdd0 2024-04-02T14:01:47.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.734+0000 7f1f5c3a5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x562484a62820 con 0x562484b568d0 2024-04-02T14:01:47.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.734+0000 7f1f5c3a5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562484ab67e0 con 0x562484b571f0 2024-04-02T14:01:47.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.734+0000 7f1f537fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b5fdd0 0x562484b621c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.734+0000 7f1f52ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b568d0 0x562484b56cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.734+0000 7f1f537fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b5fdd0 0x562484b621c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60578/0 (socket says 172.21.15.52:60578) 2024-04-02T14:01:47.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.734+0000 7f1f52ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b568d0 0x562484b56cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:33142/0 (socket says 172.21.15.52:33142) 2024-04-02T14:01:47.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.734+0000 7f1f537fe640 1 -- 172.21.15.52:0/1643460372 learned_addr learned my addr 172.21.15.52:0/1643460372 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:47.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.734+0000 7f1f527fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562484b571f0 0x562484b5f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.735+0000 7f1f52ffd640 1 -- 172.21.15.52:0/1643460372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562484b571f0 msgr2=0x562484b5f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:47.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.735+0000 7f1f52ffd640 1 --2- 172.21.15.52:0/1643460372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562484b571f0 0x562484b5f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.735+0000 7f1f52ffd640 1 -- 172.21.15.52:0/1643460372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b5fdd0 msgr2=0x562484b621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:47.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.735+0000 7f1f52ffd640 1 --2- 172.21.15.52:0/1643460372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b5fdd0 0x562484b621c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.735+0000 7f1f52ffd640 1 -- 172.21.15.52:0/1643460372 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624848987d0 con 0x562484b568d0 2024-04-02T14:01:47.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.735+0000 7f1f537fe640 1 --2- 172.21.15.52:0/2332525229 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1f3c0a3d20 0x7f1f3c0a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0127e0 tx=0x7f1f4c012be0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:47.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.735+0000 7f1f52ffd640 1 --2- 172.21.15.52:0/1643460372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b568d0 0x562484b56cb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1f440ef8d0 tx=0x7f1f440327b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=904a5e0c43a1f4aa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:47.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.735+0000 7f1f397fa640 1 -- 172.21.15.52:0/1643460372 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f44034640 con 0x562484b568d0 2024-04-02T14:01:47.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.735+0000 7f1f397fa640 1 -- 172.21.15.52:0/1643460372 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f440be820 con 0x562484b568d0 2024-04-02T14:01:47.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.735+0000 7f1f397fa640 1 -- 172.21.15.52:0/1643460372 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f440efa40 con 0x562484b568d0 2024-04-02T14:01:47.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.736+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/1643460372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b568d0 msgr2=0x562484b56cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:47.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.736+0000 7f1f5c3a5740 1 --2- 172.21.15.52:0/1643460372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b568d0 0x562484b56cb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1f440ef8d0 tx=0x7f1f440327b0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.736+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/1643460372 shutdown_connections 2024-04-02T14:01:47.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.736+0000 7f1f5c3a5740 1 --2- 172.21.15.52:0/1643460372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b5fdd0 0x562484b621c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.736+0000 7f1f5c3a5740 1 --2- 172.21.15.52:0/1643460372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562484b571f0 0x562484b5f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.736+0000 7f1f5c3a5740 1 --2- 172.21.15.52:0/1643460372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b568d0 0x562484b56cb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.736+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/1643460372 >> 172.21.15.52:0/1643460372 conn(0x562484b51000 msgr2=0x562484b513e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:47.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.736+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/1643460372 shutdown_connections 2024-04-02T14:01:47.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.736+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/1643460372 wait complete. 2024-04-02T14:01:47.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.736+0000 7f1f5c3a5740 1 -- start start 2024-04-02T14:01:47.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.736+0000 7f1f5c3a5740 1 Processor -- start 2024-04-02T14:01:47.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.737+0000 7f1f5c3a5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b568d0 0x562484b5f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.737+0000 7f1f5c3a5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b571f0 0x562484b5ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.737+0000 7f1f5c3a5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562484b5fdd0 0x562484b5cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.737+0000 7f1f5c3a5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562484b62980 con 0x562484b568d0 2024-04-02T14:01:47.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.737+0000 7f1f5c3a5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x562484a75d30 con 0x562484b571f0 2024-04-02T14:01:47.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.737+0000 7f1f5c3a5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562484b62b00 con 0x562484b5fdd0 2024-04-02T14:01:47.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.737+0000 7f1f527fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b571f0 0x562484b5ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.737+0000 7f1f527fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b571f0 0x562484b5ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:33148/0 (socket says 172.21.15.52:33148) 2024-04-02T14:01:47.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.737+0000 7f1f527fc640 1 -- 172.21.15.52:0/419419422 learned_addr learned my addr 172.21.15.52:0/419419422 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:47.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.737+0000 7f1f52ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b568d0 0x562484b5f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.737+0000 7f1f5c3a5740 10 client.0 _ll_register_callbacks cb 0x562484a75720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:47.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.737+0000 7f1f537fe640 1 --2- 172.21.15.52:0/419419422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562484b5fdd0 0x562484b5cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.737+0000 7f1f52ffd640 1 -- 172.21.15.52:0/419419422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562484b5fdd0 msgr2=0x562484b5cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:47.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.737+0000 7f1f52ffd640 1 --2- 172.21.15.52:0/419419422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562484b5fdd0 0x562484b5cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.737+0000 7f1f52ffd640 1 -- 172.21.15.52:0/419419422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b571f0 msgr2=0x562484b5ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:47.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.737+0000 7f1f52ffd640 1 --2- 172.21.15.52:0/419419422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b571f0 0x562484b5ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.737+0000 7f1f52ffd640 1 -- 172.21.15.52:0/419419422 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624847ce3f0 con 0x562484b568d0 2024-04-02T14:01:47.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.737+0000 7f1f527fc640 1 --2- 172.21.15.52:0/419419422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b571f0 0x562484b5ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:01:47.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.737+0000 7f1f537fe640 1 --2- 172.21.15.52:0/419419422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562484b5fdd0 0x562484b5cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:01:47.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.737+0000 7f1f52ffd640 1 --2- 172.21.15.52:0/419419422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b568d0 0x562484b5f6e0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f1f440be9d0 tx=0x7f1f4402c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16f5fd80840a3c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:47.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.737+0000 7f1f36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:47.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.737+0000 7f1f36ff5640 1 -- 172.21.15.52:0/419419422 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f440bc420 con 0x562484b568d0 2024-04-02T14:01:47.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.738+0000 7f1f36ff5640 1 -- 172.21.15.52:0/419419422 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f1f440ba790 con 0x562484b568d0 2024-04-02T14:01:47.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.738+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/419419422 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f440be820 con 0x562484b568d0 2024-04-02T14:01:47.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.738+0000 7f1f36ff5640 1 -- 172.21.15.52:0/419419422 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1f440be120 con 0x562484b568d0 2024-04-02T14:01:47.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.738+0000 7f1f5c3a5740 10 client.18036 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:47.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.738+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/419419422 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56248478e0f0 con 0x562484b568d0 2024-04-02T14:01:47.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.738+0000 7f1f36ff5640 1 -- 172.21.15.52:0/419419422 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f440be820 con 0x562484b568d0 2024-04-02T14:01:47.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.738+0000 7f1f36ff5640 1 -- 172.21.15.52:0/419419422 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1f4403c030 con 0x562484b568d0 2024-04-02T14:01:47.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.738+0000 7f1f5c3a5740 10 client.18036 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:47.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.738+0000 7f1f5c3a5740 10 client.18036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:01:47.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.738+0000 7f1f5c3a5740 20 client.18036 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:47.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.738+0000 7f1f5c3a5740 1 --2- 172.21.15.52:0/419419422 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x562484b6f970 0x562484b71d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.738+0000 7f1f5c3a5740 4 client.18036 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:01:47.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.738+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/419419422 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562484ac7d70 con 0x562484b6f970 2024-04-02T14:01:47.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.738+0000 7f1f527fc640 1 --2- 172.21.15.52:0/419419422 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x562484b6f970 0x562484b71d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.739+0000 7f1f36ff5640 10 client.18036 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:47.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.739+0000 7f1f527fc640 1 --2- 172.21.15.52:0/419419422 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x562484b6f970 0x562484b71d60 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df3de7de358f574c server_cookie=76d9e85a5ce53bd1 in_seq=0 out_seq=0 2024-04-02T14:01:47.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.740+0000 7f1f36ff5640 1 -- 172.21.15.52:0/419419422 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562484ac7d70 con 0x562484b6f970 2024-04-02T14:01:47.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.740+0000 7f1f36ff5640 10 client.18036 handle_command_reply: tid=0 2024-04-02T14:01:47.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.740+0000 7f1f5c3a5740 1 client.18036 shutdown 2024-04-02T14:01:47.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.740+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/419419422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b568d0 msgr2=0x562484b5f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:47.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.740+0000 7f1f5c3a5740 1 --2- 172.21.15.52:0/419419422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b568d0 0x562484b5f6e0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f1f440be9d0 tx=0x7f1f4402c390 comp rx=0 tx=0).stop 2024-04-02T14:01:47.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.740+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/419419422 shutdown_connections 2024-04-02T14:01:47.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.740+0000 7f1f5c3a5740 1 --2- 172.21.15.52:0/419419422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562484b5fdd0 0x562484b5cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.740+0000 7f1f5c3a5740 1 --2- 172.21.15.52:0/419419422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b571f0 0x562484b5ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.740+0000 7f1f5c3a5740 1 --2- 172.21.15.52:0/419419422 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x562484b6f970 0x562484b71d60 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.740+0000 7f1f5c3a5740 1 --2- 172.21.15.52:0/419419422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b568d0 0x562484b5f6e0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.740+0000 7f1f36ff5640 0 client.18036 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:47.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.740+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/419419422 >> 172.21.15.52:0/419419422 conn(0x562484b51000 msgr2=0x562484b52d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:47.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.740+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/419419422 shutdown_connections 2024-04-02T14:01:47.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.740+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/419419422 wait complete. 2024-04-02T14:01:47.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.740+0000 7f1f5c3a5740 20 client.18036 trim_cache size 0 max 16384 2024-04-02T14:01:47.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.761+0000 7f1f5c3a5740 1 Processor -- start 2024-04-02T14:01:47.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.761+0000 7f1f5c3a5740 1 -- start start 2024-04-02T14:01:47.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.761+0000 7f1f5c3a5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b568d0 0x562484b50240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.761+0000 7f1f5c3a5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b571f0 0x562484b57e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.761+0000 7f1f5c3a5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562484b5fdd0 0x562484b5f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.761+0000 7f1f5c3a5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1f541b1160 con 0x562484b571f0 2024-04-02T14:01:47.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.761+0000 7f1f5c3a5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1f541b2ca0 con 0x562484b568d0 2024-04-02T14:01:47.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.761+0000 7f1f5c3a5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1f541b1830 con 0x562484b5fdd0 2024-04-02T14:01:47.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.761+0000 7f1f527fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b571f0 0x562484b57e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.761+0000 7f1f52ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b568d0 0x562484b50240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.761+0000 7f1f527fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b571f0 0x562484b57e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60586/0 (socket says 172.21.15.52:60586) 2024-04-02T14:01:47.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.761+0000 7f1f527fc640 1 -- 172.21.15.52:0/2016726346 learned_addr learned my addr 172.21.15.52:0/2016726346 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:47.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.761+0000 7f1f52ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b568d0 0x562484b50240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:33156/0 (socket says 172.21.15.52:33156) 2024-04-02T14:01:47.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.761+0000 7f1f537fe640 1 --2- 172.21.15.52:0/2016726346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562484b5fdd0 0x562484b5f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.761+0000 7f1f527fc640 1 -- 172.21.15.52:0/2016726346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562484b5fdd0 msgr2=0x562484b5f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:47.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.761+0000 7f1f527fc640 1 --2- 172.21.15.52:0/2016726346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562484b5fdd0 0x562484b5f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.761+0000 7f1f527fc640 1 -- 172.21.15.52:0/2016726346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b568d0 msgr2=0x562484b50240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:47.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.761+0000 7f1f527fc640 1 --2- 172.21.15.52:0/2016726346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b568d0 0x562484b50240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.761+0000 7f1f527fc640 1 -- 172.21.15.52:0/2016726346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562484b4cc50 con 0x562484b571f0 2024-04-02T14:01:47.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.762+0000 7f1f52ffd640 1 --2- 172.21.15.52:0/2016726346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b568d0 0x562484b50240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:01:47.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.762+0000 7f1f527fc640 1 --2- 172.21.15.52:0/2016726346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b571f0 0x562484b57e20 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x562484b5c2d0 tx=0x7f1f48009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a20b63ea2b6e530c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:47.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.762+0000 7f1f38ff9640 1 -- 172.21.15.52:0/2016726346 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48027070 con 0x562484b571f0 2024-04-02T14:01:47.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.762+0000 7f1f38ff9640 1 -- 172.21.15.52:0/2016726346 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f48011e80 con 0x562484b571f0 2024-04-02T14:01:47.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.762+0000 7f1f38ff9640 1 -- 172.21.15.52:0/2016726346 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48021770 con 0x562484b571f0 2024-04-02T14:01:47.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.763+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/2016726346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b571f0 msgr2=0x562484b57e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:47.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.763+0000 7f1f5c3a5740 1 --2- 172.21.15.52:0/2016726346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b571f0 0x562484b57e20 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x562484b5c2d0 tx=0x7f1f48009260 comp rx=0 tx=0).stop 2024-04-02T14:01:47.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.763+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/2016726346 shutdown_connections 2024-04-02T14:01:47.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.763+0000 7f1f5c3a5740 1 --2- 172.21.15.52:0/2016726346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562484b5fdd0 0x562484b5f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.763+0000 7f1f5c3a5740 1 --2- 172.21.15.52:0/2016726346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b571f0 0x562484b57e20 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.763+0000 7f1f5c3a5740 1 --2- 172.21.15.52:0/2016726346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b568d0 0x562484b50240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.763+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/2016726346 >> 172.21.15.52:0/2016726346 conn(0x562484b51000 msgr2=0x562484b57710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:47.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.763+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/2016726346 shutdown_connections 2024-04-02T14:01:47.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.763+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/2016726346 wait complete. 2024-04-02T14:01:47.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.763+0000 7f1f5c3a5740 1 -- start start 2024-04-02T14:01:47.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.763+0000 7f1f5c3a5740 1 Processor -- start 2024-04-02T14:01:47.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.763+0000 7f1f5c3a5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562484b568d0 0x562484b67cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.763+0000 7f1f5c3a5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b571f0 0x562484b68200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.763+0000 7f1f5c3a5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b5fdd0 0x562484b64c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.763+0000 7f1f5c3a5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1f541b25d0 con 0x562484b571f0 2024-04-02T14:01:47.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.763+0000 7f1f5c3a5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1f541b1f00 con 0x562484b5fdd0 2024-04-02T14:01:47.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.763+0000 7f1f5c3a5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1f541af160 con 0x562484b568d0 2024-04-02T14:01:47.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.764+0000 7f1f5c3a5740 10 client.0 _ll_register_callbacks cb 0x7f1f48009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:01:47.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.764+0000 7f1f527fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b571f0 0x562484b68200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.764+0000 7f1f527fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b571f0 0x562484b68200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60594/0 (socket says 172.21.15.52:60594) 2024-04-02T14:01:47.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.764+0000 7f1f527fc640 1 -- 172.21.15.52:0/2537737463 learned_addr learned my addr 172.21.15.52:0/2537737463 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:01:47.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.764+0000 7f1f537fe640 1 --2- 172.21.15.52:0/2537737463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b5fdd0 0x562484b64c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.764+0000 7f1f52ffd640 1 --2- 172.21.15.52:0/2537737463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562484b568d0 0x562484b67cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.764+0000 7f1f527fc640 1 -- 172.21.15.52:0/2537737463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562484b568d0 msgr2=0x562484b67cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:47.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.764+0000 7f1f527fc640 1 --2- 172.21.15.52:0/2537737463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562484b568d0 0x562484b67cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.764+0000 7f1f527fc640 1 -- 172.21.15.52:0/2537737463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b5fdd0 msgr2=0x562484b64c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:01:47.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.764+0000 7f1f527fc640 1 --2- 172.21.15.52:0/2537737463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b5fdd0 0x562484b64c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.764+0000 7f1f527fc640 1 -- 172.21.15.52:0/2537737463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562484b652d0 con 0x562484b571f0 2024-04-02T14:01:47.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.764+0000 7f1f537fe640 1 --2- 172.21.15.52:0/2537737463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b5fdd0 0x562484b64c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:01:47.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.764+0000 7f1f527fc640 1 --2- 172.21.15.52:0/2537737463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b571f0 0x562484b68200 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f1f48006010 tx=0x7f1f48012810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0de948f8682862 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:01:47.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.764+0000 7f1f36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:01:47.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.764+0000 7f1f36ff5640 1 -- 172.21.15.52:0/2537737463 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48030070 con 0x562484b571f0 2024-04-02T14:01:47.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.764+0000 7f1f36ff5640 1 -- 172.21.15.52:0/2537737463 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f1f48011ba0 con 0x562484b571f0 2024-04-02T14:01:47.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.764+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/2537737463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562484ac20c0 con 0x562484b571f0 2024-04-02T14:01:47.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.765+0000 7f1f36ff5640 1 -- 172.21.15.52:0/2537737463 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1f48039050 con 0x562484b571f0 2024-04-02T14:01:47.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.765+0000 7f1f5c3a5740 10 client.18042 fetch_fsmap learned FSMap version 30 2024-04-02T14:01:47.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.765+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/2537737463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562484bd6720 con 0x562484b571f0 2024-04-02T14:01:47.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.765+0000 7f1f36ff5640 1 -- 172.21.15.52:0/2537737463 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562484ac20c0 con 0x562484b571f0 2024-04-02T14:01:47.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.765+0000 7f1f36ff5640 1 -- 172.21.15.52:0/2537737463 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1f48033020 con 0x562484b571f0 2024-04-02T14:01:47.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.766+0000 7f1f5c3a5740 10 client.18042 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:01:47.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.766+0000 7f1f5c3a5740 10 client.18042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:01:47.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.766+0000 7f1f5c3a5740 20 client.18042 populate_metadata read hostname 'smithi052' 2024-04-02T14:01:47.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.766+0000 7f1f5c3a5740 1 --2- 172.21.15.52:0/2537737463 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x562484b6f970 0x562484bd8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:01:47.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.766+0000 7f1f5c3a5740 4 client.18042 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:01:47.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.766+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/2537737463 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562484a3f560 con 0x562484b6f970 2024-04-02T14:01:47.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.766+0000 7f1f52ffd640 1 --2- 172.21.15.52:0/2537737463 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x562484b6f970 0x562484bd8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:01:47.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.766+0000 7f1f36ff5640 10 client.18042 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:47.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.766+0000 7f1f52ffd640 1 --2- 172.21.15.52:0/2537737463 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x562484b6f970 0x562484bd8e20 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=955bbeb81f1fadef server_cookie=bb1d6d1e255af371 in_seq=0 out_seq=0 2024-04-02T14:01:47.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.767+0000 7f1f36ff5640 1 -- 172.21.15.52:0/2537737463 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x562484a3f560 con 0x562484b6f970 2024-04-02T14:01:47.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.767+0000 7f1f36ff5640 10 client.18042 handle_command_reply: tid=0 2024-04-02T14:01:47.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.767+0000 7f1f5c3a5740 1 client.18042 shutdown 2024-04-02T14:01:47.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.767+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/2537737463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b571f0 msgr2=0x562484b68200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:47.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.767+0000 7f1f5c3a5740 1 --2- 172.21.15.52:0/2537737463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b571f0 0x562484b68200 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f1f48006010 tx=0x7f1f48012810 comp rx=0 tx=0).stop 2024-04-02T14:01:47.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.767+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/2537737463 shutdown_connections 2024-04-02T14:01:47.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.767+0000 7f1f5c3a5740 1 --2- 172.21.15.52:0/2537737463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562484b5fdd0 0x562484b64c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.767+0000 7f1f5c3a5740 1 --2- 172.21.15.52:0/2537737463 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x562484b6f970 0x562484bd8e20 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.767+0000 7f1f5c3a5740 1 --2- 172.21.15.52:0/2537737463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562484b571f0 0x562484b68200 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.767+0000 7f1f5c3a5740 1 --2- 172.21.15.52:0/2537737463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562484b568d0 0x562484b67cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.767+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/2537737463 >> 172.21.15.52:0/2537737463 conn(0x562484b51000 msgr2=0x562484b500c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:47.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.767+0000 7f1f36ff5640 0 client.18042 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:01:47.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.767+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/2537737463 shutdown_connections 2024-04-02T14:01:47.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.767+0000 7f1f5c3a5740 1 -- 172.21.15.52:0/2537737463 wait complete. 2024-04-02T14:01:47.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.767+0000 7f1f5c3a5740 20 client.18042 trim_cache size 0 max 16384 2024-04-02T14:01:47.783 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:01:47.783 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "scrub active (2405 inodes in the stack)", 2024-04-02T14:01:47.783 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-04-02T14:01:47.783 INFO:teuthology.orchestra.run.smithi052.stdout: "02d561e8-e8d2-4569-8e8f-599bb3000c26": { 2024-04-02T14:01:47.783 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-04-02T14:01:47.783 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "02d561e8-e8d2-4569-8e8f-599bb3000c26", 2024-04-02T14:01:47.783 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-04-02T14:01:47.784 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:01:47.784 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:01:47.784 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:01:47.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.767+0000 7f1f596e4640 1 -- 172.21.15.52:0/2332525229 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1f3c0a3d20 msgr2=0x7f1f3c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:47.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.767+0000 7f1f596e4640 1 --2- 172.21.15.52:0/2332525229 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1f3c0a3d20 0x7f1f3c0a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0127e0 tx=0x7f1f4c012be0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.767+0000 7f1f596e4640 1 -- 172.21.15.52:0/2332525229 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f54101a80 msgr2=0x7f1f54073e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:01:47.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.767+0000 7f1f596e4640 1 --2- 172.21.15.52:0/2332525229 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f54101a80 0x7f1f54073e00 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f1f4402c180 tx=0x7f1f44004550 comp rx=0 tx=0).stop 2024-04-02T14:01:47.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.768+0000 7f1f596e4640 1 -- 172.21.15.52:0/2332525229 shutdown_connections 2024-04-02T14:01:47.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.768+0000 7f1f596e4640 1 --2- 172.21.15.52:0/2332525229 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1f3c0a3d20 0x7f1f3c0a61e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.768+0000 7f1f596e4640 1 --2- 172.21.15.52:0/2332525229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f54107a90 0x7f1f54074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.768+0000 7f1f596e4640 1 --2- 172.21.15.52:0/2332525229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f54102440 0x7f1f54074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.768+0000 7f1f596e4640 1 --2- 172.21.15.52:0/2332525229 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f54101a80 0x7f1f54073e00 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:01:47.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.768+0000 7f1f596e4640 1 -- 172.21.15.52:0/2332525229 >> 172.21.15.52:0/2332525229 conn(0x7f1f540fd590 msgr2=0x7f1f540fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:01:47.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.768+0000 7f1f596e4640 1 -- 172.21.15.52:0/2332525229 shutdown_connections 2024-04-02T14:01:47.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:01:47.768+0000 7f1f596e4640 1 -- 172.21.15.52:0/2332525229 wait complete. 2024-04-02T14:01:47.787 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:47 smithi052 ceph-mon[32104]: pgmap v255: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:01:47.788 INFO:tasks.cephfs.filesystem:scrub status for tag:02d561e8-e8d2-4569-8e8f-599bb3000c26 - {'path': '/', 'tag': '02d561e8-e8d2-4569-8e8f-599bb3000c26', 'options': 'recursive,force'} 2024-04-02T14:01:47.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:47 smithi088 ceph-mon[36528]: pgmap v255: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:01:48.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:47 smithi189 ceph-mon[36683]: pgmap v255: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:01:48.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:48 smithi088 ceph-mon[36528]: scrub queued for path: / 2024-04-02T14:01:48.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:48 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:01:48.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:48 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T14:01:48.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:48 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T14:01:48.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:48 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:01:48.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:48 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T14:01:48.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:48 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:01:48.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:48 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:01:49.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:48 smithi189 ceph-mon[36683]: scrub queued for path: / 2024-04-02T14:01:49.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:48 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:01:49.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:48 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T14:01:49.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:48 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T14:01:49.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:48 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:01:49.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:48 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T14:01:49.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:48 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:01:49.040 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:48 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:01:49.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:48 smithi052 ceph-mon[32104]: scrub queued for path: / 2024-04-02T14:01:49.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:48 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:01:49.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:48 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T14:01:49.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:48 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T14:01:49.091 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:48 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:01:49.091 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:48 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T14:01:49.091 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:48 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:01:49.091 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:48 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:01:49.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:49 smithi088 ceph-mon[36528]: pgmap v256: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:50.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:49 smithi189 ceph-mon[36683]: pgmap v256: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:50.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:49 smithi052 ceph-mon[32104]: pgmap v256: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:51.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:51 smithi088 ceph-mon[36528]: pgmap v257: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:52.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:51 smithi189 ceph-mon[36683]: pgmap v257: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:52.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:51 smithi052 ceph-mon[32104]: pgmap v257: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:01:53.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:53 smithi088 ceph-mon[36528]: pgmap v258: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T14:01:54.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:53 smithi189 ceph-mon[36683]: pgmap v258: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T14:01:54.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:53 smithi052 ceph-mon[32104]: pgmap v258: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T14:01:55.892 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:55 smithi052 ceph-mon[32104]: pgmap v259: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:01:55.892 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:55 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T14:01:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:55 smithi088 ceph-mon[36528]: pgmap v259: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:01:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:55 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T14:01:56.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:55 smithi189 ceph-mon[36683]: pgmap v259: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:01:56.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:55 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T14:01:56.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:56 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T14:01:57.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:56 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T14:01:57.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:56 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T14:01:57.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:57 smithi088 ceph-mon[36528]: pgmap v260: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T14:01:58.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:57 smithi189 ceph-mon[36683]: pgmap v260: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T14:01:58.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:57 smithi052 ceph-mon[32104]: pgmap v260: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T14:01:59.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:01:59 smithi088 ceph-mon[36528]: pgmap v261: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:02:00.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:01:59 smithi189 ceph-mon[36683]: pgmap v261: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:02:00.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:01:59 smithi052 ceph-mon[32104]: pgmap v261: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:02:01.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:01 smithi088 ceph-mon[36528]: pgmap v262: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:02:02.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:01 smithi189 ceph-mon[36683]: pgmap v262: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:02:02.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:01 smithi052 ceph-mon[32104]: pgmap v262: 129 pgs: 129 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:02:03.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:03 smithi088 ceph-mon[36528]: pgmap v263: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:02:04.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:03 smithi189 ceph-mon[36683]: pgmap v263: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:02:04.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:03 smithi052 ceph-mon[32104]: pgmap v263: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:02:05.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:05 smithi088 ceph-mon[36528]: pgmap v264: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:02:06.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:05 smithi189 ceph-mon[36683]: pgmap v264: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:02:06.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:05 smithi052 ceph-mon[32104]: pgmap v264: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:02:07.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:07 smithi088 ceph-mon[36528]: pgmap v265: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:02:08.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:07 smithi189 ceph-mon[36683]: pgmap v265: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:02:08.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:07 smithi052 ceph-mon[32104]: pgmap v265: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:02:09.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:09 smithi088 ceph-mon[36528]: pgmap v266: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:02:10.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:09 smithi189 ceph-mon[36683]: pgmap v266: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:02:10.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:09 smithi052 ceph-mon[32104]: pgmap v266: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:02:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:11 smithi088 ceph-mon[36528]: pgmap v267: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:02:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:11 smithi088 ceph-mon[36528]: osdmap e98: 12 total, 12 up, 12 in 2024-04-02T14:02:12.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:11 smithi189 ceph-mon[36683]: pgmap v267: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:02:12.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:11 smithi189 ceph-mon[36683]: osdmap e98: 12 total, 12 up, 12 in 2024-04-02T14:02:12.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:11 smithi052 ceph-mon[32104]: pgmap v267: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:02:12.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:11 smithi052 ceph-mon[32104]: osdmap e98: 12 total, 12 up, 12 in 2024-04-02T14:02:13.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:13 smithi088 ceph-mon[36528]: pgmap v269: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:02:14.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:13 smithi189 ceph-mon[36683]: pgmap v269: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:02:14.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:13 smithi052 ceph-mon[32104]: pgmap v269: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:02:15.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:15 smithi088 ceph-mon[36528]: pgmap v270: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:02:16.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:15 smithi189 ceph-mon[36683]: pgmap v270: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:02:16.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:15 smithi052 ceph-mon[32104]: pgmap v270: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:02:17.789 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:02:17.790 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:02:17.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.903+0000 7f4397e69640 1 -- 172.21.15.52:0/2435789844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43901011e0 msgr2=0x7f439010df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:17.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.903+0000 7f4397e69640 1 --2- 172.21.15.52:0/2435789844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43901011e0 0x7f439010df30 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f4380009900 tx=0x7f438002bc30 comp rx=0 tx=0).stop 2024-04-02T14:02:17.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.904+0000 7f4397e69640 1 -- 172.21.15.52:0/2435789844 shutdown_connections 2024-04-02T14:02:17.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.904+0000 7f4397e69640 1 --2- 172.21.15.52:0/2435789844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f439010e470 0x7f4390110860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.904+0000 7f4397e69640 1 --2- 172.21.15.52:0/2435789844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43901011e0 0x7f439010df30 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.904+0000 7f4397e69640 1 --2- 172.21.15.52:0/2435789844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43901008c0 0x7f4390100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.904+0000 7f4397e69640 1 -- 172.21.15.52:0/2435789844 >> 172.21.15.52:0/2435789844 conn(0x7f4390076d80 msgr2=0x7f4390077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:17.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.904+0000 7f4397e69640 1 -- 172.21.15.52:0/2435789844 shutdown_connections 2024-04-02T14:02:17.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.904+0000 7f4397e69640 1 -- 172.21.15.52:0/2435789844 wait complete. 2024-04-02T14:02:17.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.904+0000 7f4397e69640 1 Processor -- start 2024-04-02T14:02:17.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.904+0000 7f4397e69640 1 -- start start 2024-04-02T14:02:17.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.904+0000 7f4397e69640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43901008c0 0x7f439019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:17.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.904+0000 7f4397e69640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43901011e0 0x7f439019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:17.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.904+0000 7f4397e69640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f439010e470 0x7f439019ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:17.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.904+0000 7f4397e69640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f43901140d0 con 0x7f43901008c0 2024-04-02T14:02:17.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.904+0000 7f4397e69640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4390113f50 con 0x7f43901011e0 2024-04-02T14:02:17.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.904+0000 7f4397e69640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4390114250 con 0x7f439010e470 2024-04-02T14:02:17.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.905+0000 7f4395c05640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43901008c0 0x7f439019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:17.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.905+0000 7f4395c05640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43901008c0 0x7f439019a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53120/0 (socket says 172.21.15.52:53120) 2024-04-02T14:02:17.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.905+0000 7f4395c05640 1 -- 172.21.15.52:0/3453569801 learned_addr learned my addr 172.21.15.52:0/3453569801 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:17.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.905+0000 7f4396406640 1 --2- 172.21.15.52:0/3453569801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f439010e470 0x7f439019ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:17.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.905+0000 7f4395404640 1 --2- 172.21.15.52:0/3453569801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43901011e0 0x7f439019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:17.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.905+0000 7f4395c05640 1 -- 172.21.15.52:0/3453569801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f439010e470 msgr2=0x7f439019ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:17.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.905+0000 7f4395c05640 1 --2- 172.21.15.52:0/3453569801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f439010e470 0x7f439019ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.905+0000 7f4395c05640 1 -- 172.21.15.52:0/3453569801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43901011e0 msgr2=0x7f439019ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:17.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.905+0000 7f4395c05640 1 --2- 172.21.15.52:0/3453569801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43901011e0 0x7f439019ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.905+0000 7f4395c05640 1 -- 172.21.15.52:0/3453569801 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439019f5c0 con 0x7f43901008c0 2024-04-02T14:02:17.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.905+0000 7f4395404640 1 --2- 172.21.15.52:0/3453569801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43901011e0 0x7f439019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:02:17.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.905+0000 7f4395c05640 1 --2- 172.21.15.52:0/3453569801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43901008c0 0x7f439019a5d0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f438c00e9f0 tx=0x7f438c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:17.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.905+0000 7f4386ffd640 1 -- 172.21.15.52:0/3453569801 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c00cd80 con 0x7f43901008c0 2024-04-02T14:02:17.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.905+0000 7f4386ffd640 1 -- 172.21.15.52:0/3453569801 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f438c010910 con 0x7f43901008c0 2024-04-02T14:02:17.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.905+0000 7f4397e69640 1 -- 172.21.15.52:0/3453569801 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f439019f8b0 con 0x7f43901008c0 2024-04-02T14:02:17.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.905+0000 7f4386ffd640 1 -- 172.21.15.52:0/3453569801 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c018990 con 0x7f43901008c0 2024-04-02T14:02:17.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.905+0000 7f4397e69640 1 -- 172.21.15.52:0/3453569801 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43901a7140 con 0x7f43901008c0 2024-04-02T14:02:17.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.908+0000 7f4386ffd640 1 -- 172.21.15.52:0/3453569801 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f438c010ac0 con 0x7f43901008c0 2024-04-02T14:02:17.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.908+0000 7f439aa4c740 1 Processor -- start 2024-04-02T14:02:17.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.909+0000 7f439aa4c740 1 -- start start 2024-04-02T14:02:17.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.909+0000 7f439aa4c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cf204ab8d0 0x55cf204abcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:17.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.909+0000 7f439aa4c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204ac1f0 0x55cf204b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:17.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.909+0000 7f439aa4c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf204b4dd0 0x55cf204b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:17.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.909+0000 7f439aa4c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cf2040f830 con 0x55cf204ac1f0 2024-04-02T14:02:17.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.909+0000 7f439aa4c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cf203b7820 con 0x55cf204ab8d0 2024-04-02T14:02:17.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.909+0000 7f439aa4c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cf2040b7e0 con 0x55cf204b4dd0 2024-04-02T14:02:17.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.909+0000 7f4386ffd640 1 --2- 172.21.15.52:0/3453569801 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f437c0a3d20 0x7f437c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:17.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.909+0000 7f4395c05640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204ac1f0 0x55cf204b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:17.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.909+0000 7f4396406640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cf204ab8d0 0x55cf204abcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:17.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.909+0000 7f4396406640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cf204ab8d0 0x55cf204abcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:45578/0 (socket says 172.21.15.52:45578) 2024-04-02T14:02:17.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.909+0000 7f4395c05640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204ac1f0 0x55cf204b4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53128/0 (socket says 172.21.15.52:53128) 2024-04-02T14:02:17.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.909+0000 7f4396406640 1 -- 172.21.15.52:0/556869109 learned_addr learned my addr 172.21.15.52:0/556869109 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:17.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.909+0000 7f4396406640 1 --2- 172.21.15.52:0/3453569801 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f437c0a3d20 0x7f437c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:17.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.909+0000 7f4395404640 1 --2- 172.21.15.52:0/556869109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf204b4dd0 0x55cf204b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:17.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.909+0000 7f4386ffd640 1 -- 172.21.15.52:0/3453569801 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9526+0+0 (secure 0 0 0) 0x7f438c0d0fc0 con 0x7f43901008c0 2024-04-02T14:02:17.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.910+0000 7f4395c05640 1 -- 172.21.15.52:0/556869109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf204b4dd0 msgr2=0x55cf204b71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:17.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.910+0000 7f4395c05640 1 --2- 172.21.15.52:0/556869109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf204b4dd0 0x55cf204b71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.910+0000 7f4395c05640 1 -- 172.21.15.52:0/556869109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cf204ab8d0 msgr2=0x55cf204abcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:17.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.910+0000 7f4395c05640 1 --2- 172.21.15.52:0/556869109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cf204ab8d0 0x55cf204abcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.910+0000 7f4395c05640 1 -- 172.21.15.52:0/556869109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf201ed7d0 con 0x55cf204ac1f0 2024-04-02T14:02:17.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.910+0000 7f4396406640 1 --2- 172.21.15.52:0/3453569801 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f437c0a3d20 0x7f437c0a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4388005c40 tx=0x7f4388009b70 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:17.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.910+0000 7f4395c05640 1 --2- 172.21.15.52:0/556869109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204ac1f0 0x55cf204b4890 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f438c0ce320 tx=0x7f438c09c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2443740a7fe19308 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:17.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.910+0000 7f437bfff640 1 -- 172.21.15.52:0/556869109 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c0a57e0 con 0x55cf204ac1f0 2024-04-02T14:02:17.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.910+0000 7f437bfff640 1 -- 172.21.15.52:0/556869109 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f438c0bd0f0 con 0x55cf204ac1f0 2024-04-02T14:02:17.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.910+0000 7f437bfff640 1 -- 172.21.15.52:0/556869109 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c018b30 con 0x55cf204ac1f0 2024-04-02T14:02:17.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.911+0000 7f439aa4c740 1 -- 172.21.15.52:0/556869109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204ac1f0 msgr2=0x55cf204b4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:17.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.911+0000 7f439aa4c740 1 --2- 172.21.15.52:0/556869109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204ac1f0 0x55cf204b4890 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f438c0ce320 tx=0x7f438c09c7e0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.911+0000 7f439aa4c740 1 -- 172.21.15.52:0/556869109 shutdown_connections 2024-04-02T14:02:17.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.911+0000 7f439aa4c740 1 --2- 172.21.15.52:0/556869109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf204b4dd0 0x55cf204b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.911+0000 7f439aa4c740 1 --2- 172.21.15.52:0/556869109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204ac1f0 0x55cf204b4890 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.911+0000 7f439aa4c740 1 --2- 172.21.15.52:0/556869109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cf204ab8d0 0x55cf204abcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.911+0000 7f439aa4c740 1 -- 172.21.15.52:0/556869109 >> 172.21.15.52:0/556869109 conn(0x55cf204a6000 msgr2=0x55cf204a63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:17.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.911+0000 7f439aa4c740 1 -- 172.21.15.52:0/556869109 shutdown_connections 2024-04-02T14:02:17.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.911+0000 7f439aa4c740 1 -- 172.21.15.52:0/556869109 wait complete. 2024-04-02T14:02:17.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.911+0000 7f439aa4c740 1 -- start start 2024-04-02T14:02:17.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.911+0000 7f439aa4c740 1 Processor -- start 2024-04-02T14:02:17.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.912+0000 7f439aa4c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf204ab8d0 0x55cf204b4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:17.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.912+0000 7f439aa4c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cf204ac1f0 0x55cf204b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:17.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.912+0000 7f439aa4c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204b4dd0 0x55cf204b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:17.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.912+0000 7f439aa4c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cf204b7b00 con 0x55cf204b4dd0 2024-04-02T14:02:17.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.912+0000 7f439aa4c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cf203cad30 con 0x55cf204ac1f0 2024-04-02T14:02:17.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.912+0000 7f439aa4c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cf204b7c80 con 0x55cf204ab8d0 2024-04-02T14:02:17.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.912+0000 7f439aa4c740 10 client.0 _ll_register_callbacks cb 0x55cf203ca720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:17.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.912+0000 7f4396406640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204b4dd0 0x55cf204b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:17.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.912+0000 7f4396406640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204b4dd0 0x55cf204b1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53142/0 (socket says 172.21.15.52:53142) 2024-04-02T14:02:17.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.912+0000 7f4396406640 1 -- 172.21.15.52:0/4231300540 learned_addr learned my addr 172.21.15.52:0/4231300540 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:17.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.912+0000 7f4395404640 1 --2- 172.21.15.52:0/4231300540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cf204ac1f0 0x55cf204b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:17.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.912+0000 7f4395c05640 1 --2- 172.21.15.52:0/4231300540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf204ab8d0 0x55cf204b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:17.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.912+0000 7f4396406640 1 -- 172.21.15.52:0/4231300540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf204ab8d0 msgr2=0x55cf204b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:17.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.912+0000 7f4396406640 1 --2- 172.21.15.52:0/4231300540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf204ab8d0 0x55cf204b4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.912+0000 7f4396406640 1 -- 172.21.15.52:0/4231300540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cf204ac1f0 msgr2=0x55cf204b1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:17.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.912+0000 7f4396406640 1 --2- 172.21.15.52:0/4231300540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cf204ac1f0 0x55cf204b1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.912+0000 7f4396406640 1 -- 172.21.15.52:0/4231300540 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf201233f0 con 0x55cf204b4dd0 2024-04-02T14:02:17.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.913+0000 7f4396406640 1 --2- 172.21.15.52:0/4231300540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204b4dd0 0x55cf204b1fd0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f438801f890 tx=0x7f4388005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4c3545b74c0101 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:17.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.913+0000 7f43797fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:17.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.913+0000 7f43797fa640 1 -- 172.21.15.52:0/4231300540 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438800ecd0 con 0x55cf204b4dd0 2024-04-02T14:02:17.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.913+0000 7f43797fa640 1 -- 172.21.15.52:0/4231300540 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9526+0+0 (secure 0 0 0) 0x7f438802b480 con 0x55cf204b4dd0 2024-04-02T14:02:17.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.913+0000 7f439aa4c740 1 -- 172.21.15.52:0/4231300540 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f438c0bd0f0 con 0x55cf204b4dd0 2024-04-02T14:02:17.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.913+0000 7f43797fa640 1 -- 172.21.15.52:0/4231300540 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4388034050 con 0x55cf204b4dd0 2024-04-02T14:02:17.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.913+0000 7f439aa4c740 10 client.18054 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:17.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.913+0000 7f439aa4c740 1 -- 172.21.15.52:0/4231300540 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cf200e30f0 con 0x55cf204b4dd0 2024-04-02T14:02:17.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.913+0000 7f43797fa640 1 -- 172.21.15.52:0/4231300540 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f438c0bd0f0 con 0x55cf204b4dd0 2024-04-02T14:02:17.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.913+0000 7f43797fa640 1 -- 172.21.15.52:0/4231300540 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4388020450 con 0x55cf204b4dd0 2024-04-02T14:02:17.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.913+0000 7f439aa4c740 10 client.18054 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:17.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.913+0000 7f439aa4c740 10 client.18054 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:02:17.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.913+0000 7f439aa4c740 20 client.18054 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:17.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.913+0000 7f439aa4c740 1 --2- 172.21.15.52:0/4231300540 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cf204c4d40 0x55cf204c7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:17.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.913+0000 7f439aa4c740 4 client.18054 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:02:17.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.913+0000 7f439aa4c740 1 -- 172.21.15.52:0/4231300540 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf2041cd70 con 0x55cf204c4d40 2024-04-02T14:02:17.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.914+0000 7f4395c05640 1 --2- 172.21.15.52:0/4231300540 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cf204c4d40 0x55cf204c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:17.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.914+0000 7f43797fa640 10 client.18054 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:17.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.914+0000 7f4395c05640 1 --2- 172.21.15.52:0/4231300540 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cf204c4d40 0x55cf204c7150 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eec43828ed78fde7 server_cookie=6250c2d53f0cfda9 in_seq=0 out_seq=0 2024-04-02T14:02:17.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.915+0000 7f43797fa640 1 -- 172.21.15.52:0/4231300540 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf2041cd70 con 0x55cf204c4d40 2024-04-02T14:02:17.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.915+0000 7f43797fa640 10 client.18054 handle_command_reply: tid=0 2024-04-02T14:02:17.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.915+0000 7f439aa4c740 1 client.18054 shutdown 2024-04-02T14:02:17.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.915+0000 7f439aa4c740 1 -- 172.21.15.52:0/4231300540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204b4dd0 msgr2=0x55cf204b1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:17.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.915+0000 7f439aa4c740 1 --2- 172.21.15.52:0/4231300540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204b4dd0 0x55cf204b1fd0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f438801f890 tx=0x7f4388005bd0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.915+0000 7f439aa4c740 1 -- 172.21.15.52:0/4231300540 shutdown_connections 2024-04-02T14:02:17.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.915+0000 7f439aa4c740 1 --2- 172.21.15.52:0/4231300540 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cf204c4d40 0x55cf204c7150 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.915+0000 7f439aa4c740 1 --2- 172.21.15.52:0/4231300540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204b4dd0 0x55cf204b1fd0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.915+0000 7f439aa4c740 1 --2- 172.21.15.52:0/4231300540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cf204ac1f0 0x55cf204b1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.915+0000 7f439aa4c740 1 --2- 172.21.15.52:0/4231300540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf204ab8d0 0x55cf204b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.915+0000 7f43797fa640 0 client.18054 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:17.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.915+0000 7f439aa4c740 1 -- 172.21.15.52:0/4231300540 >> 172.21.15.52:0/4231300540 conn(0x55cf204a6000 msgr2=0x55cf204a7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:17.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.915+0000 7f439aa4c740 1 -- 172.21.15.52:0/4231300540 shutdown_connections 2024-04-02T14:02:17.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.915+0000 7f439aa4c740 1 -- 172.21.15.52:0/4231300540 wait complete. 2024-04-02T14:02:17.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.915+0000 7f439aa4c740 20 client.18054 trim_cache size 0 max 16384 2024-04-02T14:02:17.931 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:17 smithi088 ceph-mon[36528]: pgmap v271: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:02:17.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.933+0000 7f439aa4c740 1 Processor -- start 2024-04-02T14:02:17.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.933+0000 7f439aa4c740 1 -- start start 2024-04-02T14:02:17.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.933+0000 7f439aa4c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204ab8d0 0x55cf204ad0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:17.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.933+0000 7f439aa4c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf204ac1f0 0x55cf204a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:17.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.933+0000 7f439aa4c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cf204b4dd0 0x55cf204b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:17.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.933+0000 7f439aa4c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f43901b1310 con 0x55cf204ab8d0 2024-04-02T14:02:17.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.933+0000 7f439aa4c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f43901b2e50 con 0x55cf204b4dd0 2024-04-02T14:02:17.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.933+0000 7f439aa4c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f43901b19e0 con 0x55cf204ac1f0 2024-04-02T14:02:17.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.933+0000 7f4395c05640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204ab8d0 0x55cf204ad0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:17.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.933+0000 7f4395c05640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204ab8d0 0x55cf204ad0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53148/0 (socket says 172.21.15.52:53148) 2024-04-02T14:02:17.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.933+0000 7f4395c05640 1 -- 172.21.15.52:0/3883287790 learned_addr learned my addr 172.21.15.52:0/3883287790 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:17.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.933+0000 7f4395404640 1 --2- 172.21.15.52:0/3883287790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf204ac1f0 0x55cf204a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:17.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.933+0000 7f4396406640 1 --2- 172.21.15.52:0/3883287790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cf204b4dd0 0x55cf204b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:17.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.933+0000 7f4395c05640 1 -- 172.21.15.52:0/3883287790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf204ac1f0 msgr2=0x55cf204a50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:17.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.933+0000 7f4395c05640 1 --2- 172.21.15.52:0/3883287790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf204ac1f0 0x55cf204a50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.934+0000 7f4395c05640 1 -- 172.21.15.52:0/3883287790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cf204b4dd0 msgr2=0x55cf204b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:17.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.934+0000 7f4395c05640 1 --2- 172.21.15.52:0/3883287790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cf204b4dd0 0x55cf204b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.934+0000 7f4395c05640 1 -- 172.21.15.52:0/3883287790 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf204a6bd0 con 0x55cf204ab8d0 2024-04-02T14:02:17.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.934+0000 7f4396406640 1 --2- 172.21.15.52:0/3883287790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cf204b4dd0 0x55cf204b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:02:17.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.934+0000 7f4395c05640 1 --2- 172.21.15.52:0/3883287790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204ab8d0 0x55cf204ad0c0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f438c0ce3d0 tx=0x7f438c09ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58f412f49c97e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:17.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.934+0000 7f437b7fe640 1 -- 172.21.15.52:0/3883287790 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c0b5030 con 0x55cf204ab8d0 2024-04-02T14:02:17.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.934+0000 7f437b7fe640 1 -- 172.21.15.52:0/3883287790 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f438c0991e0 con 0x55cf204ab8d0 2024-04-02T14:02:17.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.934+0000 7f437b7fe640 1 -- 172.21.15.52:0/3883287790 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c018e20 con 0x55cf204ab8d0 2024-04-02T14:02:17.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.934+0000 7f439aa4c740 1 -- 172.21.15.52:0/3883287790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204ab8d0 msgr2=0x55cf204ad0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:17.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.934+0000 7f439aa4c740 1 --2- 172.21.15.52:0/3883287790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204ab8d0 0x55cf204ad0c0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f438c0ce3d0 tx=0x7f438c09ccb0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.934+0000 7f439aa4c740 1 -- 172.21.15.52:0/3883287790 shutdown_connections 2024-04-02T14:02:17.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.935+0000 7f439aa4c740 1 --2- 172.21.15.52:0/3883287790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cf204b4dd0 0x55cf204b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.935+0000 7f439aa4c740 1 --2- 172.21.15.52:0/3883287790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf204ac1f0 0x55cf204a50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.935+0000 7f439aa4c740 1 --2- 172.21.15.52:0/3883287790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204ab8d0 0x55cf204ad0c0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.935+0000 7f439aa4c740 1 -- 172.21.15.52:0/3883287790 >> 172.21.15.52:0/3883287790 conn(0x55cf204a6000 msgr2=0x55cf204a49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:17.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.935+0000 7f439aa4c740 1 -- 172.21.15.52:0/3883287790 shutdown_connections 2024-04-02T14:02:17.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.935+0000 7f439aa4c740 1 -- 172.21.15.52:0/3883287790 wait complete. 2024-04-02T14:02:17.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.935+0000 7f439aa4c740 1 -- start start 2024-04-02T14:02:17.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.935+0000 7f439aa4c740 1 Processor -- start 2024-04-02T14:02:17.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.935+0000 7f439aa4c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204ab8d0 0x55cf204b8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:17.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.935+0000 7f439aa4c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf204ac1f0 0x55cf204b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:17.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.935+0000 7f439aa4c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cf204b4dd0 0x55cf204b92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.935+0000 7f439aa4c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f43901b2780 con 0x55cf204ab8d0 2024-04-02T14:02:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.935+0000 7f439aa4c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f43901b20b0 con 0x55cf204b4dd0 2024-04-02T14:02:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.935+0000 7f439aa4c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f43901af310 con 0x55cf204ac1f0 2024-04-02T14:02:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.935+0000 7f439aa4c740 10 client.0 _ll_register_callbacks cb 0x7f438801f6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.935+0000 7f4395c05640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204ab8d0 0x55cf204b8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.935+0000 7f4395c05640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204ab8d0 0x55cf204b8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53164/0 (socket says 172.21.15.52:53164) 2024-04-02T14:02:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.935+0000 7f4395c05640 1 -- 172.21.15.52:0/1013982872 learned_addr learned my addr 172.21.15.52:0/1013982872 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.936+0000 7f4395404640 1 --2- 172.21.15.52:0/1013982872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf204ac1f0 0x55cf204b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.936+0000 7f4396406640 1 --2- 172.21.15.52:0/1013982872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cf204b4dd0 0x55cf204b92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.936+0000 7f4395c05640 1 -- 172.21.15.52:0/1013982872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf204ac1f0 msgr2=0x55cf204b8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.936+0000 7f4395c05640 1 --2- 172.21.15.52:0/1013982872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf204ac1f0 0x55cf204b8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.936+0000 7f4395c05640 1 -- 172.21.15.52:0/1013982872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cf204b4dd0 msgr2=0x55cf204b92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.936+0000 7f4395c05640 1 --2- 172.21.15.52:0/1013982872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cf204b4dd0 0x55cf204b92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.936+0000 7f4395c05640 1 -- 172.21.15.52:0/1013982872 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf2052a290 con 0x55cf204ab8d0 2024-04-02T14:02:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.936+0000 7f4395404640 1 --2- 172.21.15.52:0/1013982872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf204ac1f0 0x55cf204b8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:02:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.936+0000 7f4396406640 1 --2- 172.21.15.52:0/1013982872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cf204b4dd0 0x55cf204b92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:02:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.936+0000 7f4395c05640 1 --2- 172.21.15.52:0/1013982872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204ab8d0 0x55cf204b8820 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f438c000c00 tx=0x7f438c09d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b2892041c73e6e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.936+0000 7f43797fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.936+0000 7f43797fa640 1 -- 172.21.15.52:0/1013982872 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c0b5070 con 0x55cf204ab8d0 2024-04-02T14:02:17.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.936+0000 7f43797fa640 1 -- 172.21.15.52:0/1013982872 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9526+0+0 (secure 0 0 0) 0x7f438c018b30 con 0x55cf204ab8d0 2024-04-02T14:02:17.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.936+0000 7f439aa4c740 1 -- 172.21.15.52:0/1013982872 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f438c0991e0 con 0x55cf204ab8d0 2024-04-02T14:02:17.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.936+0000 7f43797fa640 1 -- 172.21.15.52:0/1013982872 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f438c09bfc0 con 0x55cf204ab8d0 2024-04-02T14:02:17.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.936+0000 7f439aa4c740 10 client.18060 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:17.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.936+0000 7f439aa4c740 1 -- 172.21.15.52:0/1013982872 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cf2052b4c0 con 0x55cf204ab8d0 2024-04-02T14:02:17.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.936+0000 7f43797fa640 1 -- 172.21.15.52:0/1013982872 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f438c0991e0 con 0x55cf204ab8d0 2024-04-02T14:02:17.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.937+0000 7f43797fa640 1 -- 172.21.15.52:0/1013982872 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f438c0b7070 con 0x55cf204ab8d0 2024-04-02T14:02:17.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.937+0000 7f439aa4c740 10 client.18060 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:17.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.937+0000 7f439aa4c740 10 client.18060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:02:17.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.937+0000 7f439aa4c740 20 client.18060 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:17.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.937+0000 7f439aa4c740 1 --2- 172.21.15.52:0/1013982872 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cf2052b740 0x55cf2052db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:17.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.937+0000 7f439aa4c740 4 client.18060 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:02:17.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.937+0000 7f439aa4c740 1 -- 172.21.15.52:0/1013982872 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cf204170c0 con 0x55cf2052b740 2024-04-02T14:02:17.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.937+0000 7f4395404640 1 --2- 172.21.15.52:0/1013982872 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cf2052b740 0x55cf2052db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:17.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.937+0000 7f4395404640 1 --2- 172.21.15.52:0/1013982872 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cf2052b740 0x55cf2052db30 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4360304ab28560be server_cookie=e5eb95488380fe6f in_seq=0 out_seq=0 2024-04-02T14:02:17.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.937+0000 7f43797fa640 10 client.18060 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:17.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.937+0000 7f43797fa640 1 -- 172.21.15.52:0/1013982872 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cf204170c0 con 0x55cf2052b740 2024-04-02T14:02:17.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.937+0000 7f43797fa640 10 client.18060 handle_command_reply: tid=0 2024-04-02T14:02:17.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.937+0000 7f439aa4c740 1 client.18060 shutdown 2024-04-02T14:02:17.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.938+0000 7f439aa4c740 1 -- 172.21.15.52:0/1013982872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204ab8d0 msgr2=0x55cf204b8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:17.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.938+0000 7f439aa4c740 1 --2- 172.21.15.52:0/1013982872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204ab8d0 0x55cf204b8820 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f438c000c00 tx=0x7f438c09d3b0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.938+0000 7f439aa4c740 1 -- 172.21.15.52:0/1013982872 shutdown_connections 2024-04-02T14:02:17.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.938+0000 7f439aa4c740 1 --2- 172.21.15.52:0/1013982872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cf204b4dd0 0x55cf204b92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.938+0000 7f439aa4c740 1 --2- 172.21.15.52:0/1013982872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf204ac1f0 0x55cf204b8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.938+0000 7f439aa4c740 1 --2- 172.21.15.52:0/1013982872 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55cf2052b740 0x55cf2052db30 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.938+0000 7f439aa4c740 1 --2- 172.21.15.52:0/1013982872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cf204ab8d0 0x55cf204b8820 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.938+0000 7f43797fa640 0 client.18060 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:17.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.938+0000 7f439aa4c740 1 -- 172.21.15.52:0/1013982872 >> 172.21.15.52:0/1013982872 conn(0x55cf204a6000 msgr2=0x55cf204ace90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:17.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.938+0000 7f439aa4c740 1 -- 172.21.15.52:0/1013982872 shutdown_connections 2024-04-02T14:02:17.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.938+0000 7f439aa4c740 1 -- 172.21.15.52:0/1013982872 wait complete. 2024-04-02T14:02:17.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.938+0000 7f439aa4c740 20 client.18060 trim_cache size 0 max 16384 2024-04-02T14:02:17.952 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:02:17.953 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-04-02T14:02:17.953 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-04-02T14:02:17.953 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:02:17.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.938+0000 7f4397e69640 1 -- 172.21.15.52:0/3453569801 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f437c0a3d20 msgr2=0x7f437c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:17.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.938+0000 7f4397e69640 1 --2- 172.21.15.52:0/3453569801 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f437c0a3d20 0x7f437c0a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4388005c40 tx=0x7f4388009b70 comp rx=0 tx=0).stop 2024-04-02T14:02:17.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.938+0000 7f4397e69640 1 -- 172.21.15.52:0/3453569801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43901008c0 msgr2=0x7f439019a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:17.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.938+0000 7f4397e69640 1 --2- 172.21.15.52:0/3453569801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43901008c0 0x7f439019a5d0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f438c00e9f0 tx=0x7f438c00eeb0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.938+0000 7f4397e69640 1 -- 172.21.15.52:0/3453569801 shutdown_connections 2024-04-02T14:02:17.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.938+0000 7f4397e69640 1 --2- 172.21.15.52:0/3453569801 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f437c0a3d20 0x7f437c0a61e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.938+0000 7f4397e69640 1 --2- 172.21.15.52:0/3453569801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f439010e470 0x7f439019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.938+0000 7f4397e69640 1 --2- 172.21.15.52:0/3453569801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43901011e0 0x7f439019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.938+0000 7f4397e69640 1 --2- 172.21.15.52:0/3453569801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43901008c0 0x7f439019a5d0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:17.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.938+0000 7f4397e69640 1 -- 172.21.15.52:0/3453569801 >> 172.21.15.52:0/3453569801 conn(0x7f4390076d80 msgr2=0x7f439010fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:17.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.938+0000 7f4397e69640 1 -- 172.21.15.52:0/3453569801 shutdown_connections 2024-04-02T14:02:17.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:17.938+0000 7f4397e69640 1 -- 172.21.15.52:0/3453569801 wait complete. 2024-04-02T14:02:17.955 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:17 smithi052 ceph-mon[32104]: pgmap v271: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:02:17.956 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T14:02:17.957 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:02:18.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:17 smithi189 ceph-mon[36683]: pgmap v271: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:02:18.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.056+0000 7f66722b8640 1 -- 172.21.15.52:0/1050952630 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f666c101360 msgr2=0x7f666c10d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.056+0000 7f66722b8640 1 --2- 172.21.15.52:0/1050952630 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f666c101360 0x7f666c10d870 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f6664009cd0 tx=0x7f666402c0f0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.056+0000 7f66722b8640 1 -- 172.21.15.52:0/1050952630 shutdown_connections 2024-04-02T14:02:18.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.056+0000 7f66722b8640 1 --2- 172.21.15.52:0/1050952630 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f666c101360 0x7f666c10d870 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.056+0000 7f66722b8640 1 --2- 172.21.15.52:0/1050952630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f666c1009a0 0x7f666c100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.056+0000 7f66722b8640 1 --2- 172.21.15.52:0/1050952630 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f666c1069b0 0x7f666c106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.056+0000 7f66722b8640 1 -- 172.21.15.52:0/1050952630 >> 172.21.15.52:0/1050952630 conn(0x7f666c076d80 msgr2=0x7f666c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.056+0000 7f66722b8640 1 -- 172.21.15.52:0/1050952630 shutdown_connections 2024-04-02T14:02:18.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.056+0000 7f66722b8640 1 -- 172.21.15.52:0/1050952630 wait complete. 2024-04-02T14:02:18.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.057+0000 7f66722b8640 1 Processor -- start 2024-04-02T14:02:18.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.057+0000 7f66722b8640 1 -- start start 2024-04-02T14:02:18.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.057+0000 7f66722b8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f666c1009a0 0x7f666c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.057+0000 7f66722b8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f666c101360 0x7f666c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.057+0000 7f66722b8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f666c1069b0 0x7f666c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.057+0000 7f66722b8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f666c110dc0 con 0x7f666c1009a0 2024-04-02T14:02:18.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.057+0000 7f66722b8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f666c110c40 con 0x7f666c101360 2024-04-02T14:02:18.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.057+0000 7f66722b8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f666c110f40 con 0x7f666c1069b0 2024-04-02T14:02:18.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.057+0000 7f666bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f666c1009a0 0x7f666c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.058+0000 7f666bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f666c1009a0 0x7f666c198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53174/0 (socket says 172.21.15.52:53174) 2024-04-02T14:02:18.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.058+0000 7f666bfff640 1 -- 172.21.15.52:0/3450540165 learned_addr learned my addr 172.21.15.52:0/3450540165 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:18.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.058+0000 7f6670855640 1 --2- 172.21.15.52:0/3450540165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f666c1069b0 0x7f666c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.058+0000 7f666b7fe640 1 --2- 172.21.15.52:0/3450540165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f666c101360 0x7f666c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.058+0000 7f666bfff640 1 -- 172.21.15.52:0/3450540165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f666c1069b0 msgr2=0x7f666c19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.058+0000 7f666bfff640 1 --2- 172.21.15.52:0/3450540165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f666c1069b0 0x7f666c19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.058+0000 7f666bfff640 1 -- 172.21.15.52:0/3450540165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f666c101360 msgr2=0x7f666c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.058+0000 7f666bfff640 1 --2- 172.21.15.52:0/3450540165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f666c101360 0x7f666c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.058+0000 7f666bfff640 1 -- 172.21.15.52:0/3450540165 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666c19d380 con 0x7f666c1009a0 2024-04-02T14:02:18.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.058+0000 7f666bfff640 1 --2- 172.21.15.52:0/3450540165 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f666c1009a0 0x7f666c198390 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f665c00e9f0 tx=0x7f665c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.058+0000 7f66697fa640 1 -- 172.21.15.52:0/3450540165 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c00cd80 con 0x7f666c1009a0 2024-04-02T14:02:18.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.058+0000 7f66722b8640 1 -- 172.21.15.52:0/3450540165 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666c19d670 con 0x7f666c1009a0 2024-04-02T14:02:18.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.058+0000 7f66697fa640 1 -- 172.21.15.52:0/3450540165 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f665c010910 con 0x7f666c1009a0 2024-04-02T14:02:18.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.058+0000 7f66697fa640 1 -- 172.21.15.52:0/3450540165 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c018990 con 0x7f666c1009a0 2024-04-02T14:02:18.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.058+0000 7f66722b8640 1 -- 172.21.15.52:0/3450540165 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f666c1a4f00 con 0x7f666c1009a0 2024-04-02T14:02:18.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.059+0000 7f66722b8640 1 -- 172.21.15.52:0/3450540165 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f666c1020e0 con 0x7f666c1009a0 2024-04-02T14:02:18.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.060+0000 7f66697fa640 1 -- 172.21.15.52:0/3450540165 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f665c010ac0 con 0x7f666c1009a0 2024-04-02T14:02:18.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.061+0000 7f66697fa640 1 --2- 172.21.15.52:0/3450540165 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f66540a3d20 0x7f66540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.061+0000 7f66697fa640 1 -- 172.21.15.52:0/3450540165 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9526+0+0 (secure 0 0 0) 0x7f665c0d0f20 con 0x7f666c1009a0 2024-04-02T14:02:18.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.061+0000 7f666b7fe640 1 --2- 172.21.15.52:0/3450540165 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f66540a3d20 0x7f66540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.062+0000 7f666b7fe640 1 --2- 172.21.15.52:0/3450540165 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f66540a3d20 0x7f66540a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6660005fd0 tx=0x7f6660005e20 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.066+0000 7f66697fa640 1 -- 172.21.15.52:0/3450540165 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f665c09d1d0 con 0x7f666c1009a0 2024-04-02T14:02:18.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.291+0000 7f66722b8640 1 -- 172.21.15.52:0/3450540165 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f666c068d40 con 0x7f666c1009a0 2024-04-02T14:02:18.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.292+0000 7f66697fa640 1 -- 172.21.15.52:0/3450540165 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11923 (secure 0 0 0) 0x7f665c09cad0 con 0x7f666c1009a0 2024-04-02T14:02:18.296 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T14:02:18.296 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:57:45.240902+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24467,24457,24490,24475,24503]},"id":1}]} 2024-04-02T14:02:18.296 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-04-02T14:02:18.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.294+0000 7f66722b8640 1 -- 172.21.15.52:0/3450540165 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f66540a3d20 msgr2=0x7f66540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.294+0000 7f66722b8640 1 --2- 172.21.15.52:0/3450540165 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f66540a3d20 0x7f66540a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6660005fd0 tx=0x7f6660005e20 comp rx=0 tx=0).stop 2024-04-02T14:02:18.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.294+0000 7f66722b8640 1 -- 172.21.15.52:0/3450540165 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f666c1009a0 msgr2=0x7f666c198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.294+0000 7f66722b8640 1 --2- 172.21.15.52:0/3450540165 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f666c1009a0 0x7f666c198390 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f665c00e9f0 tx=0x7f665c00eeb0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.294+0000 7f66722b8640 1 -- 172.21.15.52:0/3450540165 shutdown_connections 2024-04-02T14:02:18.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.294+0000 7f66722b8640 1 --2- 172.21.15.52:0/3450540165 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f66540a3d20 0x7f66540a61e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.294+0000 7f66722b8640 1 --2- 172.21.15.52:0/3450540165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f666c1069b0 0x7f666c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.294+0000 7f66722b8640 1 --2- 172.21.15.52:0/3450540165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f666c101360 0x7f666c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.294+0000 7f66722b8640 1 --2- 172.21.15.52:0/3450540165 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f666c1009a0 0x7f666c198390 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.294+0000 7f66722b8640 1 -- 172.21.15.52:0/3450540165 >> 172.21.15.52:0/3450540165 conn(0x7f666c076d80 msgr2=0x7f666c1047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.294+0000 7f66722b8640 1 -- 172.21.15.52:0/3450540165 shutdown_connections 2024-04-02T14:02:18.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.294+0000 7f66722b8640 1 -- 172.21.15.52:0/3450540165 wait complete. 2024-04-02T14:02:18.309 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T14:02:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.410+0000 7fb677888640 1 -- 172.21.15.52:0/133233991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6701069c0 msgr2=0x7fb670106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.410+0000 7fb677888640 1 --2- 172.21.15.52:0/133233991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6701069c0 0x7fb670106da0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fb66c009930 tx=0x7fb66c02bc30 comp rx=0 tx=0).stop 2024-04-02T14:02:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.410+0000 7fb677888640 1 -- 172.21.15.52:0/133233991 shutdown_connections 2024-04-02T14:02:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.410+0000 7fb677888640 1 --2- 172.21.15.52:0/133233991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb670101370 0x7fb67010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.410+0000 7fb677888640 1 --2- 172.21.15.52:0/133233991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6701009b0 0x7fb670100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.410+0000 7fb677888640 1 --2- 172.21.15.52:0/133233991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6701069c0 0x7fb670106da0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.410+0000 7fb677888640 1 -- 172.21.15.52:0/133233991 >> 172.21.15.52:0/133233991 conn(0x7fb670076d80 msgr2=0x7fb670077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.410+0000 7fb677888640 1 -- 172.21.15.52:0/133233991 shutdown_connections 2024-04-02T14:02:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.410+0000 7fb677888640 1 -- 172.21.15.52:0/133233991 wait complete. 2024-04-02T14:02:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.411+0000 7fb677888640 1 Processor -- start 2024-04-02T14:02:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.411+0000 7fb677888640 1 -- start start 2024-04-02T14:02:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.411+0000 7fb677888640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6701009b0 0x7fb670197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.411+0000 7fb677888640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb670101370 0x7fb670197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.411+0000 7fb677888640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6701069c0 0x7fb67019bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.411+0000 7fb677888640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb670110dd0 con 0x7fb670101370 2024-04-02T14:02:18.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.411+0000 7fb677888640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb670110c50 con 0x7fb6701009b0 2024-04-02T14:02:18.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.411+0000 7fb677888640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb670110f50 con 0x7fb6701069c0 2024-04-02T14:02:18.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.411+0000 7fb674e23640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb670101370 0x7fb670197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.411+0000 7fb674e23640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb670101370 0x7fb670197850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53196/0 (socket says 172.21.15.52:53196) 2024-04-02T14:02:18.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.411+0000 7fb674e23640 1 -- 172.21.15.52:0/4024418650 learned_addr learned my addr 172.21.15.52:0/4024418650 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:18.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.411+0000 7fb675624640 1 --2- 172.21.15.52:0/4024418650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6701009b0 0x7fb670197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.411+0000 7fb675e25640 1 --2- 172.21.15.52:0/4024418650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6701069c0 0x7fb67019bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.411+0000 7fb674e23640 1 -- 172.21.15.52:0/4024418650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6701069c0 msgr2=0x7fb67019bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.411+0000 7fb674e23640 1 --2- 172.21.15.52:0/4024418650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6701069c0 0x7fb67019bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.411+0000 7fb674e23640 1 -- 172.21.15.52:0/4024418650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6701009b0 msgr2=0x7fb670197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.411+0000 7fb674e23640 1 --2- 172.21.15.52:0/4024418650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6701009b0 0x7fb670197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.411+0000 7fb674e23640 1 -- 172.21.15.52:0/4024418650 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb67019c300 con 0x7fb670101370 2024-04-02T14:02:18.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.411+0000 7fb675e25640 1 --2- 172.21.15.52:0/4024418650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6701069c0 0x7fb67019bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:02:18.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.412+0000 7fb674e23640 1 --2- 172.21.15.52:0/4024418650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb670101370 0x7fb670197850 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fb66000d770 tx=0x7fb66000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.412+0000 7fb6667fc640 1 -- 172.21.15.52:0/4024418650 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb660017070 con 0x7fb670101370 2024-04-02T14:02:18.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.412+0000 7fb6667fc640 1 -- 172.21.15.52:0/4024418650 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb660005500 con 0x7fb670101370 2024-04-02T14:02:18.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.412+0000 7fb6667fc640 1 -- 172.21.15.52:0/4024418650 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb660004e00 con 0x7fb670101370 2024-04-02T14:02:18.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.412+0000 7fb677888640 1 -- 172.21.15.52:0/4024418650 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb67019c5f0 con 0x7fb670101370 2024-04-02T14:02:18.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.412+0000 7fb677888640 1 -- 172.21.15.52:0/4024418650 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6701a3e80 con 0x7fb670101370 2024-04-02T14:02:18.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.414+0000 7fb6667fc640 1 -- 172.21.15.52:0/4024418650 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb660004fa0 con 0x7fb670101370 2024-04-02T14:02:18.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.414+0000 7fb67a46a740 1 Processor -- start 2024-04-02T14:02:18.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.415+0000 7fb67a46a740 1 -- start start 2024-04-02T14:02:18.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.415+0000 7fb67a46a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557976d72460 0x557976d72840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.415+0000 7fb6667fc640 1 --2- 172.21.15.52:0/4024418650 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb65c0a3d20 0x7fb65c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.415+0000 7fb67a46a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557976d72d80 0x557976d7d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.415+0000 7fb67a46a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d7deb0 0x557976d802a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.415+0000 7fb67a46a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557976cd67b0 con 0x557976d7deb0 2024-04-02T14:02:18.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.415+0000 7fb67a46a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557976c75570 con 0x557976d72d80 2024-04-02T14:02:18.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.415+0000 7fb67a46a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557976cd2760 con 0x557976d72460 2024-04-02T14:02:18.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.415+0000 7fb675e25640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d7deb0 0x557976d802a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.415+0000 7fb675624640 1 --2- 172.21.15.52:0/4024418650 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb65c0a3d20 0x7fb65c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.415+0000 7fb675e25640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d7deb0 0x557976d802a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53204/0 (socket says 172.21.15.52:53204) 2024-04-02T14:02:18.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.415+0000 7fb675e25640 1 -- 172.21.15.52:0/1294021864 learned_addr learned my addr 172.21.15.52:0/1294021864 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:18.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.415+0000 7fb6667fc640 1 -- 172.21.15.52:0/4024418650 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9526+0+0 (secure 0 0 0) 0x7fb6600d0720 con 0x7fb670101370 2024-04-02T14:02:18.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.415+0000 7fb675e25640 1 --2- 172.21.15.52:0/1294021864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557976d72460 0x557976d72840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.415+0000 7fb674e23640 1 --2- 172.21.15.52:0/1294021864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557976d72d80 0x557976d7d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.415+0000 7fb675e25640 1 -- 172.21.15.52:0/1294021864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557976d72460 msgr2=0x557976d72840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.415+0000 7fb675e25640 1 --2- 172.21.15.52:0/1294021864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557976d72460 0x557976d72840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.415+0000 7fb675e25640 1 -- 172.21.15.52:0/1294021864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557976d72d80 msgr2=0x557976d7d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.415+0000 7fb675e25640 1 --2- 172.21.15.52:0/1294021864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557976d72d80 0x557976d7d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.415+0000 7fb675e25640 1 -- 172.21.15.52:0/1294021864 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579769ea3f0 con 0x557976d7deb0 2024-04-02T14:02:18.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.415+0000 7fb675624640 1 --2- 172.21.15.52:0/4024418650 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb65c0a3d20 0x7fb65c0a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb66c006010 tx=0x7fb66c000bc0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.415+0000 7fb675e25640 1 --2- 172.21.15.52:0/1294021864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d7deb0 0x557976d802a0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fb668013840 tx=0x7fb6680099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633e739e2538f3b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.416+0000 7fb65b7fe640 1 -- 172.21.15.52:0/1294021864 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb668019030 con 0x557976d7deb0 2024-04-02T14:02:18.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.416+0000 7fb65b7fe640 1 -- 172.21.15.52:0/1294021864 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb668003e50 con 0x557976d7deb0 2024-04-02T14:02:18.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.416+0000 7fb65b7fe640 1 -- 172.21.15.52:0/1294021864 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb668011b60 con 0x557976d7deb0 2024-04-02T14:02:18.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.416+0000 7fb67a46a740 1 -- 172.21.15.52:0/1294021864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d7deb0 msgr2=0x557976d802a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.416+0000 7fb67a46a740 1 --2- 172.21.15.52:0/1294021864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d7deb0 0x557976d802a0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fb668013840 tx=0x7fb6680099c0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.416+0000 7fb67a46a740 1 -- 172.21.15.52:0/1294021864 shutdown_connections 2024-04-02T14:02:18.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.416+0000 7fb67a46a740 1 --2- 172.21.15.52:0/1294021864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d7deb0 0x557976d802a0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.416+0000 7fb67a46a740 1 --2- 172.21.15.52:0/1294021864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557976d72d80 0x557976d7d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.416+0000 7fb67a46a740 1 --2- 172.21.15.52:0/1294021864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557976d72460 0x557976d72840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.416+0000 7fb67a46a740 1 -- 172.21.15.52:0/1294021864 >> 172.21.15.52:0/1294021864 conn(0x557976d6ced0 msgr2=0x557976d6d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.416+0000 7fb67a46a740 1 -- 172.21.15.52:0/1294021864 shutdown_connections 2024-04-02T14:02:18.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.416+0000 7fb67a46a740 1 -- 172.21.15.52:0/1294021864 wait complete. 2024-04-02T14:02:18.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.417+0000 7fb67a46a740 1 -- start start 2024-04-02T14:02:18.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.417+0000 7fb67a46a740 1 Processor -- start 2024-04-02T14:02:18.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.417+0000 7fb67a46a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d72460 0x557976d7d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.417+0000 7fb67a46a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557976d72d80 0x557976d7ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.417+0000 7fb67a46a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557976d7deb0 0x557976d7b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.417+0000 7fb67a46a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557976d80ca0 con 0x557976d72460 2024-04-02T14:02:18.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.417+0000 7fb67a46a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557976c91e80 con 0x557976d72d80 2024-04-02T14:02:18.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.417+0000 7fb67a46a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557976d80e20 con 0x557976d7deb0 2024-04-02T14:02:18.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.417+0000 7fb67a46a740 10 client.0 _ll_register_callbacks cb 0x557976ccdaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:18.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.417+0000 7fb675624640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d72460 0x557976d7d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.417+0000 7fb675624640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d72460 0x557976d7d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53210/0 (socket says 172.21.15.52:53210) 2024-04-02T14:02:18.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.417+0000 7fb675624640 1 -- 172.21.15.52:0/4217500754 learned_addr learned my addr 172.21.15.52:0/4217500754 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:18.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.418+0000 7fb675624640 1 -- 172.21.15.52:0/4217500754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557976d7deb0 msgr2=0x557976d7b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.418+0000 7fb675624640 1 --2- 172.21.15.52:0/4217500754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557976d7deb0 0x557976d7b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.418+0000 7fb675624640 1 -- 172.21.15.52:0/4217500754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557976d72d80 msgr2=0x557976d7ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.418+0000 7fb674e23640 1 --2- 172.21.15.52:0/4217500754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557976d72d80 0x557976d7ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.418+0000 7fb675624640 1 --2- 172.21.15.52:0/4217500754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557976d72d80 0x557976d7ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.418+0000 7fb675624640 1 -- 172.21.15.52:0/4217500754 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557976ab47d0 con 0x557976d72460 2024-04-02T14:02:18.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.418+0000 7fb675624640 1 --2- 172.21.15.52:0/4217500754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d72460 0x557976d7d8e0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fb66c035000 tx=0x7fb66c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6e7c8161a9c50 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.418+0000 7fb658ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:18.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.418+0000 7fb658ff9640 1 -- 172.21.15.52:0/4217500754 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c04d020 con 0x557976d72460 2024-04-02T14:02:18.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.418+0000 7fb658ff9640 1 -- 172.21.15.52:0/4217500754 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9526+0+0 (secure 0 0 0) 0x7fb66c003cf0 con 0x557976d72460 2024-04-02T14:02:18.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.418+0000 7fb67a46a740 1 -- 172.21.15.52:0/4217500754 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb668003e50 con 0x557976d72460 2024-04-02T14:02:18.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.418+0000 7fb658ff9640 1 -- 172.21.15.52:0/4217500754 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb66c058050 con 0x557976d72460 2024-04-02T14:02:18.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.418+0000 7fb67a46a740 10 client.18078 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:18.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.418+0000 7fb67a46a740 1 -- 172.21.15.52:0/4217500754 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557976d8cd80 con 0x557976d72460 2024-04-02T14:02:18.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.418+0000 7fb658ff9640 1 -- 172.21.15.52:0/4217500754 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb668003e50 con 0x557976d72460 2024-04-02T14:02:18.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.418+0000 7fb658ff9640 1 -- 172.21.15.52:0/4217500754 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb66c004b30 con 0x557976d72460 2024-04-02T14:02:18.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.418+0000 7fb67a46a740 10 client.18078 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:18.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.418+0000 7fb67a46a740 10 client.18078 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T14:02:18.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.419+0000 7fb67a46a740 20 client.18078 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:18.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.419+0000 7fb67a46a740 1 --2- 172.21.15.52:0/4217500754 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x557976d8e290 0x557976d906e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.419+0000 7fb67a46a740 4 client.18078 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:02:18.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.419+0000 7fb67a46a740 1 -- 172.21.15.52:0/4217500754 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557976ce3cf0 con 0x557976d8e290 2024-04-02T14:02:18.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.419+0000 7fb674e23640 1 --2- 172.21.15.52:0/4217500754 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x557976d8e290 0x557976d906e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.419+0000 7fb658ff9640 10 client.18078 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T14:02:18.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.419+0000 7fb674e23640 1 --2- 172.21.15.52:0/4217500754 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x557976d8e290 0x557976d906e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=adf93919d2458f29 server_cookie=903edcac5d1a6ede in_seq=0 out_seq=0 2024-04-02T14:02:18.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.420+0000 7fb658ff9640 1 -- 172.21.15.52:0/4217500754 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557976ce3cf0 con 0x557976d8e290 2024-04-02T14:02:18.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.420+0000 7fb658ff9640 10 client.18078 handle_command_reply: tid=0 2024-04-02T14:02:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.420+0000 7fb67a46a740 1 client.18078 shutdown 2024-04-02T14:02:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.420+0000 7fb67a46a740 1 -- 172.21.15.52:0/4217500754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d72460 msgr2=0x557976d7d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.420+0000 7fb67a46a740 1 --2- 172.21.15.52:0/4217500754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d72460 0x557976d7d8e0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fb66c035000 tx=0x7fb66c009260 comp rx=0 tx=0).stop 2024-04-02T14:02:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.420+0000 7fb67a46a740 1 -- 172.21.15.52:0/4217500754 shutdown_connections 2024-04-02T14:02:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.420+0000 7fb67a46a740 1 --2- 172.21.15.52:0/4217500754 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x557976d8e290 0x557976d906e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.420+0000 7fb67a46a740 1 --2- 172.21.15.52:0/4217500754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557976d7deb0 0x557976d7b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.420+0000 7fb67a46a740 1 --2- 172.21.15.52:0/4217500754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557976d72d80 0x557976d7ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.420+0000 7fb67a46a740 1 --2- 172.21.15.52:0/4217500754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d72460 0x557976d7d8e0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.420+0000 7fb658ff9640 0 client.18078 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T14:02:18.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.420+0000 7fb67a46a740 1 -- 172.21.15.52:0/4217500754 >> 172.21.15.52:0/4217500754 conn(0x557976d6ced0 msgr2=0x557976d6ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.421+0000 7fb67a46a740 1 -- 172.21.15.52:0/4217500754 shutdown_connections 2024-04-02T14:02:18.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.421+0000 7fb67a46a740 1 -- 172.21.15.52:0/4217500754 wait complete. 2024-04-02T14:02:18.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.421+0000 7fb67a46a740 20 client.18078 trim_cache size 0 max 16384 2024-04-02T14:02:18.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.434+0000 7fb67a46a740 1 Processor -- start 2024-04-02T14:02:18.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.434+0000 7fb67a46a740 1 -- start start 2024-04-02T14:02:18.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.434+0000 7fb67a46a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557976d72460 0x557976d89bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.434+0000 7fb67a46a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d72d80 0x557976d95fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.434+0000 7fb67a46a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557976d7deb0 0x557976d964f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.434+0000 7fb67a46a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb6701ace70 con 0x557976d72d80 2024-04-02T14:02:18.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.434+0000 7fb67a46a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb6701ae9b0 con 0x557976d72460 2024-04-02T14:02:18.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.434+0000 7fb67a46a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb6701ad540 con 0x557976d7deb0 2024-04-02T14:02:18.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.434+0000 7fb674e23640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d72d80 0x557976d95fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.434+0000 7fb674e23640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d72d80 0x557976d95fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53226/0 (socket says 172.21.15.52:53226) 2024-04-02T14:02:18.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.434+0000 7fb674e23640 1 -- 172.21.15.52:0/2135200944 learned_addr learned my addr 172.21.15.52:0/2135200944 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:18.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.435+0000 7fb675624640 1 --2- 172.21.15.52:0/2135200944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557976d72460 0x557976d89bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.435+0000 7fb675e25640 1 --2- 172.21.15.52:0/2135200944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557976d7deb0 0x557976d964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.435+0000 7fb674e23640 1 -- 172.21.15.52:0/2135200944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557976d7deb0 msgr2=0x557976d964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.435+0000 7fb674e23640 1 --2- 172.21.15.52:0/2135200944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557976d7deb0 0x557976d964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.435+0000 7fb674e23640 1 -- 172.21.15.52:0/2135200944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557976d72460 msgr2=0x557976d89bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.435+0000 7fb674e23640 1 --2- 172.21.15.52:0/2135200944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557976d72460 0x557976d89bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.435+0000 7fb674e23640 1 -- 172.21.15.52:0/2135200944 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557976d7aa40 con 0x557976d72d80 2024-04-02T14:02:18.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.435+0000 7fb675e25640 1 --2- 172.21.15.52:0/2135200944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557976d7deb0 0x557976d964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:02:18.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.435+0000 7fb674e23640 1 --2- 172.21.15.52:0/2135200944 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d72d80 0x557976d95fb0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fb66000d740 tx=0x7fb66009c1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24af4649cba296ac server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.435+0000 7fb65affd640 1 -- 172.21.15.52:0/2135200944 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb660020040 con 0x557976d72d80 2024-04-02T14:02:18.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.435+0000 7fb65affd640 1 -- 172.21.15.52:0/2135200944 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb66009c9d0 con 0x557976d72d80 2024-04-02T14:02:18.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.435+0000 7fb65affd640 1 -- 172.21.15.52:0/2135200944 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66009a370 con 0x557976d72d80 2024-04-02T14:02:18.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.436+0000 7fb67a46a740 1 -- 172.21.15.52:0/2135200944 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d72d80 msgr2=0x557976d95fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.436+0000 7fb67a46a740 1 --2- 172.21.15.52:0/2135200944 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d72d80 0x557976d95fb0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fb66000d740 tx=0x7fb66009c1c0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.436+0000 7fb67a46a740 1 -- 172.21.15.52:0/2135200944 shutdown_connections 2024-04-02T14:02:18.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.436+0000 7fb67a46a740 1 --2- 172.21.15.52:0/2135200944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557976d7deb0 0x557976d964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.436+0000 7fb67a46a740 1 --2- 172.21.15.52:0/2135200944 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d72d80 0x557976d95fb0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.436+0000 7fb67a46a740 1 --2- 172.21.15.52:0/2135200944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557976d72460 0x557976d89bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.436+0000 7fb67a46a740 1 -- 172.21.15.52:0/2135200944 >> 172.21.15.52:0/2135200944 conn(0x557976d6ced0 msgr2=0x557976d75f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.436+0000 7fb67a46a740 1 -- 172.21.15.52:0/2135200944 shutdown_connections 2024-04-02T14:02:18.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.436+0000 7fb67a46a740 1 -- 172.21.15.52:0/2135200944 wait complete. 2024-04-02T14:02:18.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.436+0000 7fb67a46a740 1 -- start start 2024-04-02T14:02:18.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.436+0000 7fb67a46a740 1 Processor -- start 2024-04-02T14:02:18.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.436+0000 7fb67a46a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d72460 0x557976d7e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.436+0000 7fb67a46a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557976d72d80 0x557976d7e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.436+0000 7fb67a46a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557976d7deb0 0x557976d7edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.436+0000 7fb67a46a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb6701ae2e0 con 0x557976d72460 2024-04-02T14:02:18.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.436+0000 7fb67a46a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb6701adc10 con 0x557976d72d80 2024-04-02T14:02:18.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.436+0000 7fb67a46a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb6701aae70 con 0x557976d7deb0 2024-04-02T14:02:18.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.436+0000 7fb67a46a740 10 client.0 _ll_register_callbacks cb 0x7fb66c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:18.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.436+0000 7fb675624640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d72460 0x557976d7e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.437+0000 7fb675624640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d72460 0x557976d7e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53242/0 (socket says 172.21.15.52:53242) 2024-04-02T14:02:18.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.437+0000 7fb675624640 1 -- 172.21.15.52:0/1998449484 learned_addr learned my addr 172.21.15.52:0/1998449484 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:18.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.437+0000 7fb674e23640 1 --2- 172.21.15.52:0/1998449484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557976d72d80 0x557976d7e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.437+0000 7fb675e25640 1 --2- 172.21.15.52:0/1998449484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557976d7deb0 0x557976d7edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.437+0000 7fb675624640 1 -- 172.21.15.52:0/1998449484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557976d7deb0 msgr2=0x557976d7edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.437+0000 7fb675624640 1 --2- 172.21.15.52:0/1998449484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557976d7deb0 0x557976d7edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.437+0000 7fb675624640 1 -- 172.21.15.52:0/1998449484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557976d72d80 msgr2=0x557976d7e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.437+0000 7fb675624640 1 --2- 172.21.15.52:0/1998449484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557976d72d80 0x557976d7e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.437+0000 7fb675624640 1 -- 172.21.15.52:0/1998449484 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557976d850d0 con 0x557976d72460 2024-04-02T14:02:18.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.437+0000 7fb674e23640 1 --2- 172.21.15.52:0/1998449484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557976d72d80 0x557976d7e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:02:18.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.437+0000 7fb675624640 1 --2- 172.21.15.52:0/1998449484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d72460 0x557976d7e350 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fb66c02c170 tx=0x7fb66c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2bd213472ad41cd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.437+0000 7fb658ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:18.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.437+0000 7fb658ff9640 1 -- 172.21.15.52:0/1998449484 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c060020 con 0x557976d72460 2024-04-02T14:02:18.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.437+0000 7fb658ff9640 1 -- 172.21.15.52:0/1998449484 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9526+0+0 (secure 0 0 0) 0x7fb66c004ad0 con 0x557976d72460 2024-04-02T14:02:18.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.437+0000 7fb67a46a740 1 -- 172.21.15.52:0/1998449484 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb66009c9d0 con 0x557976d72460 2024-04-02T14:02:18.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.437+0000 7fb658ff9640 1 -- 172.21.15.52:0/1998449484 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb66c06b050 con 0x557976d72460 2024-04-02T14:02:18.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.437+0000 7fb67a46a740 10 client.18084 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:18.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.437+0000 7fb67a46a740 1 -- 172.21.15.52:0/1998449484 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557976d86340 con 0x557976d72460 2024-04-02T14:02:18.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.437+0000 7fb658ff9640 1 -- 172.21.15.52:0/1998449484 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb66009c9d0 con 0x557976d72460 2024-04-02T14:02:18.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.438+0000 7fb658ff9640 1 -- 172.21.15.52:0/1998449484 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fb66c034840 con 0x557976d72460 2024-04-02T14:02:18.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.438+0000 7fb67a46a740 10 client.18084 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:18.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.438+0000 7fb67a46a740 10 client.18084 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T14:02:18.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.438+0000 7fb67a46a740 20 client.18084 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:18.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.438+0000 7fb67a46a740 1 --2- 172.21.15.52:0/1998449484 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x557976df6c20 0x557976df9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.438+0000 7fb67a46a740 4 client.18084 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:02:18.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.438+0000 7fb67a46a740 1 -- 172.21.15.52:0/1998449484 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557976cde040 con 0x557976df6c20 2024-04-02T14:02:18.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.438+0000 7fb674e23640 1 --2- 172.21.15.52:0/1998449484 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x557976df6c20 0x557976df9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.438+0000 7fb674e23640 1 --2- 172.21.15.52:0/1998449484 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x557976df6c20 0x557976df9010 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e6c63bf7a600bae5 server_cookie=705d7359cd0c2185 in_seq=0 out_seq=0 2024-04-02T14:02:18.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.438+0000 7fb658ff9640 10 client.18084 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T14:02:18.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.438+0000 7fb658ff9640 1 -- 172.21.15.52:0/1998449484 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557976cde040 con 0x557976df6c20 2024-04-02T14:02:18.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.438+0000 7fb658ff9640 10 client.18084 handle_command_reply: tid=0 2024-04-02T14:02:18.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.438+0000 7fb67a46a740 1 client.18084 shutdown 2024-04-02T14:02:18.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.439+0000 7fb67a46a740 1 -- 172.21.15.52:0/1998449484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d72460 msgr2=0x557976d7e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.439+0000 7fb67a46a740 1 --2- 172.21.15.52:0/1998449484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d72460 0x557976d7e350 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fb66c02c170 tx=0x7fb66c003ce0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.439+0000 7fb67a46a740 1 -- 172.21.15.52:0/1998449484 shutdown_connections 2024-04-02T14:02:18.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.439+0000 7fb67a46a740 1 --2- 172.21.15.52:0/1998449484 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x557976df6c20 0x557976df9010 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.439+0000 7fb67a46a740 1 --2- 172.21.15.52:0/1998449484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557976d7deb0 0x557976d7edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.439+0000 7fb67a46a740 1 --2- 172.21.15.52:0/1998449484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557976d72d80 0x557976d7e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.439+0000 7fb658ff9640 0 client.18084 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T14:02:18.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.439+0000 7fb67a46a740 1 --2- 172.21.15.52:0/1998449484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557976d72460 0x557976d7e350 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.439+0000 7fb67a46a740 1 -- 172.21.15.52:0/1998449484 >> 172.21.15.52:0/1998449484 conn(0x557976d6ced0 msgr2=0x557976d79500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.439+0000 7fb67a46a740 1 -- 172.21.15.52:0/1998449484 shutdown_connections 2024-04-02T14:02:18.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.439+0000 7fb67a46a740 1 -- 172.21.15.52:0/1998449484 wait complete. 2024-04-02T14:02:18.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.439+0000 7fb67a46a740 20 client.18084 trim_cache size 0 max 16384 2024-04-02T14:02:18.455 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:02:18.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.439+0000 7fb677888640 1 -- 172.21.15.52:0/4024418650 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb65c0a3d20 msgr2=0x7fb65c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.439+0000 7fb677888640 1 --2- 172.21.15.52:0/4024418650 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb65c0a3d20 0x7fb65c0a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb66c006010 tx=0x7fb66c000bc0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.439+0000 7fb677888640 1 -- 172.21.15.52:0/4024418650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb670101370 msgr2=0x7fb670197850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.439+0000 7fb677888640 1 --2- 172.21.15.52:0/4024418650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb670101370 0x7fb670197850 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fb66000d770 tx=0x7fb66000dc30 comp rx=0 tx=0).stop 2024-04-02T14:02:18.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.439+0000 7fb677888640 1 -- 172.21.15.52:0/4024418650 shutdown_connections 2024-04-02T14:02:18.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.439+0000 7fb677888640 1 --2- 172.21.15.52:0/4024418650 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb65c0a3d20 0x7fb65c0a61e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.439+0000 7fb677888640 1 --2- 172.21.15.52:0/4024418650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6701069c0 0x7fb67019bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.439+0000 7fb677888640 1 --2- 172.21.15.52:0/4024418650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb670101370 0x7fb670197850 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.439+0000 7fb677888640 1 --2- 172.21.15.52:0/4024418650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6701009b0 0x7fb670197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.439+0000 7fb677888640 1 -- 172.21.15.52:0/4024418650 >> 172.21.15.52:0/4024418650 conn(0x7fb670076d80 msgr2=0x7fb6700fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.439+0000 7fb677888640 1 -- 172.21.15.52:0/4024418650 shutdown_connections 2024-04-02T14:02:18.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.440+0000 7fb677888640 1 -- 172.21.15.52:0/4024418650 wait complete. 2024-04-02T14:02:18.458 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T14:02:18.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.560+0000 7facfe006640 1 -- 172.21.15.52:0/1916361709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facf8102d40 msgr2=0x7facf8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.560+0000 7facfe006640 1 --2- 172.21.15.52:0/1916361709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facf8102d40 0x7facf8103120 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7face8009900 tx=0x7face802bcd0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.560+0000 7facfe006640 1 -- 172.21.15.52:0/1916361709 shutdown_connections 2024-04-02T14:02:18.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.560+0000 7facfe006640 1 --2- 172.21.15.52:0/1916361709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facf810c110 0x7facf810e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.560+0000 7facfe006640 1 --2- 172.21.15.52:0/1916361709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facf8103660 0x7facf810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.560+0000 7facfe006640 1 --2- 172.21.15.52:0/1916361709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facf8102d40 0x7facf8103120 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.560+0000 7facfe006640 1 -- 172.21.15.52:0/1916361709 >> 172.21.15.52:0/1916361709 conn(0x7facf8076d80 msgr2=0x7facf8077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.560+0000 7facfe006640 1 -- 172.21.15.52:0/1916361709 shutdown_connections 2024-04-02T14:02:18.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.560+0000 7facfe006640 1 -- 172.21.15.52:0/1916361709 wait complete. 2024-04-02T14:02:18.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.560+0000 7facfe006640 1 Processor -- start 2024-04-02T14:02:18.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.561+0000 7facfe006640 1 -- start start 2024-04-02T14:02:18.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.561+0000 7facfe006640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facf8102d40 0x7facf81983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.561+0000 7facfe006640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facf8103660 0x7facf8198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.561+0000 7facfe006640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facf810c110 0x7facf819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.561+0000 7facfe006640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7facf8111ea0 con 0x7facf8102d40 2024-04-02T14:02:18.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.561+0000 7facfe006640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7facf8111d20 con 0x7facf8103660 2024-04-02T14:02:18.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.561+0000 7facfe006640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7facf8112020 con 0x7facf810c110 2024-04-02T14:02:18.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.561+0000 7facf77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facf8102d40 0x7facf81983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.561+0000 7facf77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facf8102d40 0x7facf81983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53264/0 (socket says 172.21.15.52:53264) 2024-04-02T14:02:18.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.561+0000 7facf77fe640 1 -- 172.21.15.52:0/2276244330 learned_addr learned my addr 172.21.15.52:0/2276244330 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:18.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.561+0000 7facf7fff640 1 --2- 172.21.15.52:0/2276244330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facf810c110 0x7facf819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.561+0000 7facf6ffd640 1 --2- 172.21.15.52:0/2276244330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facf8103660 0x7facf8198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.561+0000 7facf77fe640 1 -- 172.21.15.52:0/2276244330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facf810c110 msgr2=0x7facf819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.561+0000 7facf77fe640 1 --2- 172.21.15.52:0/2276244330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facf810c110 0x7facf819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.561+0000 7facf77fe640 1 -- 172.21.15.52:0/2276244330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facf8103660 msgr2=0x7facf8198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.561+0000 7facf77fe640 1 --2- 172.21.15.52:0/2276244330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facf8103660 0x7facf8198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.561+0000 7facf77fe640 1 -- 172.21.15.52:0/2276244330 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf819d3b0 con 0x7facf8102d40 2024-04-02T14:02:18.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.561+0000 7facf6ffd640 1 --2- 172.21.15.52:0/2276244330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facf8103660 0x7facf8198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:02:18.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.562+0000 7facf77fe640 1 --2- 172.21.15.52:0/2276244330 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facf8102d40 0x7facf81983c0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7face8005940 tx=0x7face802cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.562+0000 7facf4ff9640 1 -- 172.21.15.52:0/2276244330 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face8041070 con 0x7facf8102d40 2024-04-02T14:02:18.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.562+0000 7facfe006640 1 -- 172.21.15.52:0/2276244330 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf819d640 con 0x7facf8102d40 2024-04-02T14:02:18.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.562+0000 7facf4ff9640 1 -- 172.21.15.52:0/2276244330 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7face8004450 con 0x7facf8102d40 2024-04-02T14:02:18.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.562+0000 7facf4ff9640 1 -- 172.21.15.52:0/2276244330 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face8034440 con 0x7facf8102d40 2024-04-02T14:02:18.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.562+0000 7facfe006640 1 -- 172.21.15.52:0/2276244330 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf81a4f30 con 0x7facf8102d40 2024-04-02T14:02:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.565+0000 7facf4ff9640 1 -- 172.21.15.52:0/2276244330 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7face8044070 con 0x7facf8102d40 2024-04-02T14:02:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.565+0000 7fad00bec740 1 Processor -- start 2024-04-02T14:02:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.565+0000 7fad00bec740 1 -- start start 2024-04-02T14:02:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.565+0000 7fad00bec740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5602288507a0 0x560228850b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.565+0000 7fad00bec740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602288510c0 0x560228859760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.565+0000 7fad00bec740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 0x56022885c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.565+0000 7fad00bec740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5602287b47b0 con 0x560228859ca0 2024-04-02T14:02:18.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.565+0000 7fad00bec740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560228753570 con 0x5602288507a0 2024-04-02T14:02:18.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.565+0000 7fad00bec740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5602287b0760 con 0x5602288510c0 2024-04-02T14:02:18.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.565+0000 7facf6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 0x56022885c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.565+0000 7facf6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 0x56022885c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53278/0 (socket says 172.21.15.52:53278) 2024-04-02T14:02:18.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.565+0000 7facf6ffd640 1 -- 172.21.15.52:0/1642779936 learned_addr learned my addr 172.21.15.52:0/1642779936 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:18.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.565+0000 7facf4ff9640 1 --2- 172.21.15.52:0/2276244330 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7face00a3d20 0x7face00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.565+0000 7facf7fff640 1 --2- 172.21.15.52:0/1642779936 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5602288507a0 0x560228850b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.565+0000 7facf7fff640 1 --2- 172.21.15.52:0/2276244330 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7face00a3d20 0x7face00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.566+0000 7facf4ff9640 1 -- 172.21.15.52:0/2276244330 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9526+0+0 (secure 0 0 0) 0x7face80f2720 con 0x7facf8102d40 2024-04-02T14:02:18.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.566+0000 7facf77fe640 1 --2- 172.21.15.52:0/1642779936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602288510c0 0x560228859760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.566+0000 7facf6ffd640 1 -- 172.21.15.52:0/1642779936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602288510c0 msgr2=0x560228859760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.566+0000 7facf6ffd640 1 --2- 172.21.15.52:0/1642779936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602288510c0 0x560228859760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.566+0000 7facf6ffd640 1 -- 172.21.15.52:0/1642779936 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5602288507a0 msgr2=0x560228850b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.566+0000 7facf6ffd640 1 --2- 172.21.15.52:0/1642779936 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5602288507a0 0x560228850b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.566+0000 7facf6ffd640 1 -- 172.21.15.52:0/1642779936 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602284c83f0 con 0x560228859ca0 2024-04-02T14:02:18.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.566+0000 7facf77fe640 1 --2- 172.21.15.52:0/1642779936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602288510c0 0x560228859760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:02:18.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.566+0000 7facf6ffd640 1 --2- 172.21.15.52:0/1642779936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 0x56022885c090 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7facec00bd30 tx=0x7facec00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb270a7093a3a4d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.566+0000 7facf7fff640 1 --2- 172.21.15.52:0/2276244330 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7face00a3d20 0x7face00a61e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7facf806b1d0 tx=0x7facf0009600 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.566+0000 7facddffb640 1 -- 172.21.15.52:0/1642779936 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec015070 con 0x560228859ca0 2024-04-02T14:02:18.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.566+0000 7facddffb640 1 -- 172.21.15.52:0/1642779936 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facec00be20 con 0x560228859ca0 2024-04-02T14:02:18.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.566+0000 7facddffb640 1 -- 172.21.15.52:0/1642779936 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec011aa0 con 0x560228859ca0 2024-04-02T14:02:18.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.567+0000 7fad00bec740 1 -- 172.21.15.52:0/1642779936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 msgr2=0x56022885c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.567+0000 7fad00bec740 1 --2- 172.21.15.52:0/1642779936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 0x56022885c090 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7facec00bd30 tx=0x7facec00f730 comp rx=0 tx=0).stop 2024-04-02T14:02:18.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.567+0000 7fad00bec740 1 -- 172.21.15.52:0/1642779936 shutdown_connections 2024-04-02T14:02:18.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.567+0000 7fad00bec740 1 --2- 172.21.15.52:0/1642779936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 0x56022885c090 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.567+0000 7fad00bec740 1 --2- 172.21.15.52:0/1642779936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602288510c0 0x560228859760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.567+0000 7fad00bec740 1 --2- 172.21.15.52:0/1642779936 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5602288507a0 0x560228850b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.567+0000 7fad00bec740 1 -- 172.21.15.52:0/1642779936 >> 172.21.15.52:0/1642779936 conn(0x56022884aed0 msgr2=0x56022884b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.567+0000 7fad00bec740 1 -- 172.21.15.52:0/1642779936 shutdown_connections 2024-04-02T14:02:18.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.567+0000 7fad00bec740 1 -- 172.21.15.52:0/1642779936 wait complete. 2024-04-02T14:02:18.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.567+0000 7fad00bec740 1 -- start start 2024-04-02T14:02:18.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.567+0000 7fad00bec740 1 Processor -- start 2024-04-02T14:02:18.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.568+0000 7fad00bec740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5602288507a0 0x560228856b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.568+0000 7fad00bec740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602288510c0 0x560228857040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.568+0000 7fad00bec740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 0x560228857580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.568+0000 7fad00bec740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56022885ca90 con 0x560228859ca0 2024-04-02T14:02:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.568+0000 7fad00bec740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56022876fe80 con 0x5602288507a0 2024-04-02T14:02:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.568+0000 7fad00bec740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56022885cc10 con 0x5602288510c0 2024-04-02T14:02:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.568+0000 7fad00bec740 10 client.0 _ll_register_callbacks cb 0x5602287abaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.568+0000 7facf7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 0x560228857580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.568+0000 7facf7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 0x560228857580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53288/0 (socket says 172.21.15.52:53288) 2024-04-02T14:02:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.568+0000 7facf7fff640 1 -- 172.21.15.52:0/2225583696 learned_addr learned my addr 172.21.15.52:0/2225583696 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.568+0000 7facf6ffd640 1 --2- 172.21.15.52:0/2225583696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602288510c0 0x560228857040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.568+0000 7facf77fe640 1 --2- 172.21.15.52:0/2225583696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5602288507a0 0x560228856b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.568+0000 7facf7fff640 1 -- 172.21.15.52:0/2225583696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602288510c0 msgr2=0x560228857040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.568+0000 7facf7fff640 1 --2- 172.21.15.52:0/2225583696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602288510c0 0x560228857040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.568+0000 7facf7fff640 1 -- 172.21.15.52:0/2225583696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5602288507a0 msgr2=0x560228856b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.568+0000 7facf7fff640 1 --2- 172.21.15.52:0/2225583696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5602288507a0 0x560228856b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.568+0000 7facf7fff640 1 -- 172.21.15.52:0/2225583696 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602285927d0 con 0x560228859ca0 2024-04-02T14:02:18.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.568+0000 7facf7fff640 1 --2- 172.21.15.52:0/2225583696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 0x560228857580 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7facf0010ce0 tx=0x7facf0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd09cabe2797ec42 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.568+0000 7facd6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:18.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.568+0000 7facd6ffd640 1 -- 172.21.15.52:0/2225583696 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf0004d20 con 0x560228859ca0 2024-04-02T14:02:18.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.569+0000 7facd6ffd640 1 -- 172.21.15.52:0/2225583696 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9526+0+0 (secure 0 0 0) 0x7facf0005020 con 0x560228859ca0 2024-04-02T14:02:18.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.569+0000 7fad00bec740 1 -- 172.21.15.52:0/2225583696 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facec00be20 con 0x560228859ca0 2024-04-02T14:02:18.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.569+0000 7facd6ffd640 1 -- 172.21.15.52:0/2225583696 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7facf0005220 con 0x560228859ca0 2024-04-02T14:02:18.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.569+0000 7fad00bec740 10 client.18096 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:18.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.569+0000 7fad00bec740 1 -- 172.21.15.52:0/2225583696 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560228868b30 con 0x560228859ca0 2024-04-02T14:02:18.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.569+0000 7facd6ffd640 1 -- 172.21.15.52:0/2225583696 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facec00be20 con 0x560228859ca0 2024-04-02T14:02:18.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.569+0000 7facd6ffd640 1 -- 172.21.15.52:0/2225583696 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7facf000f040 con 0x560228859ca0 2024-04-02T14:02:18.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.569+0000 7fad00bec740 10 client.18096 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:18.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.569+0000 7fad00bec740 10 client.18096 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T14:02:18.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.569+0000 7fad00bec740 20 client.18096 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:18.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.569+0000 7fad00bec740 1 --2- 172.21.15.52:0/2225583696 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x560228869ff0 0x56022886c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.569+0000 7fad00bec740 4 client.18096 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:02:18.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.569+0000 7fad00bec740 1 -- 172.21.15.52:0/2225583696 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602287c1cf0 con 0x560228869ff0 2024-04-02T14:02:18.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.569+0000 7facf77fe640 1 --2- 172.21.15.52:0/2225583696 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x560228869ff0 0x56022886c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.570+0000 7facd6ffd640 10 client.18096 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T14:02:18.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.570+0000 7facf77fe640 1 --2- 172.21.15.52:0/2225583696 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x560228869ff0 0x56022886c440 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1ced18a5b026714f server_cookie=4b80cf53bd54fcae in_seq=0 out_seq=0 2024-04-02T14:02:18.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.570+0000 7facd6ffd640 1 -- 172.21.15.52:0/2225583696 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5602287c1cf0 con 0x560228869ff0 2024-04-02T14:02:18.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.570+0000 7facd6ffd640 10 client.18096 handle_command_reply: tid=0 2024-04-02T14:02:18.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.570+0000 7fad00bec740 1 client.18096 shutdown 2024-04-02T14:02:18.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.571+0000 7fad00bec740 1 -- 172.21.15.52:0/2225583696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 msgr2=0x560228857580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.571+0000 7fad00bec740 1 --2- 172.21.15.52:0/2225583696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 0x560228857580 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7facf0010ce0 tx=0x7facf0004a40 comp rx=0 tx=0).stop 2024-04-02T14:02:18.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.571+0000 7fad00bec740 1 -- 172.21.15.52:0/2225583696 shutdown_connections 2024-04-02T14:02:18.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.571+0000 7fad00bec740 1 --2- 172.21.15.52:0/2225583696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 0x560228857580 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.571+0000 7fad00bec740 1 --2- 172.21.15.52:0/2225583696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602288510c0 0x560228857040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.571+0000 7fad00bec740 1 --2- 172.21.15.52:0/2225583696 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x560228869ff0 0x56022886c440 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.571+0000 7fad00bec740 1 --2- 172.21.15.52:0/2225583696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5602288507a0 0x560228856b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.571+0000 7facd6ffd640 0 client.18096 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T14:02:18.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.571+0000 7fad00bec740 1 -- 172.21.15.52:0/2225583696 >> 172.21.15.52:0/2225583696 conn(0x56022884aed0 msgr2=0x56022884cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.571+0000 7fad00bec740 1 -- 172.21.15.52:0/2225583696 shutdown_connections 2024-04-02T14:02:18.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.571+0000 7fad00bec740 1 -- 172.21.15.52:0/2225583696 wait complete. 2024-04-02T14:02:18.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.571+0000 7fad00bec740 20 client.18096 trim_cache size 0 max 16384 2024-04-02T14:02:18.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.584+0000 7fad00bec740 1 Processor -- start 2024-04-02T14:02:18.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.584+0000 7fad00bec740 1 -- start start 2024-04-02T14:02:18.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.584+0000 7fad00bec740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602288507a0 0x560228859230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.584+0000 7fad00bec740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5602288510c0 0x560228865950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.584+0000 7fad00bec740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 0x560228855ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.584+0000 7fad00bec740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7facf81af0a0 con 0x560228859ca0 2024-04-02T14:02:18.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.584+0000 7fad00bec740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7facf81b0be0 con 0x5602288510c0 2024-04-02T14:02:18.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.584+0000 7fad00bec740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7facf81af770 con 0x5602288507a0 2024-04-02T14:02:18.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.584+0000 7facf7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 0x560228855ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.584+0000 7facf7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 0x560228855ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53296/0 (socket says 172.21.15.52:53296) 2024-04-02T14:02:18.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.584+0000 7facf7fff640 1 -- 172.21.15.52:0/2431036662 learned_addr learned my addr 172.21.15.52:0/2431036662 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:18.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.585+0000 7facf6ffd640 1 --2- 172.21.15.52:0/2431036662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5602288510c0 0x560228865950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.585+0000 7facf7fff640 1 -- 172.21.15.52:0/2431036662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602288507a0 msgr2=0x560228859230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.585+0000 7facf7fff640 1 --2- 172.21.15.52:0/2431036662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602288507a0 0x560228859230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.585+0000 7facf7fff640 1 -- 172.21.15.52:0/2431036662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5602288510c0 msgr2=0x560228865950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.585+0000 7facf7fff640 1 --2- 172.21.15.52:0/2431036662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5602288510c0 0x560228865950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.585+0000 7facf7fff640 1 -- 172.21.15.52:0/2431036662 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560228850ef0 con 0x560228859ca0 2024-04-02T14:02:18.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.585+0000 7facf6ffd640 1 --2- 172.21.15.52:0/2431036662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5602288510c0 0x560228865950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:02:18.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.585+0000 7facf7fff640 1 --2- 172.21.15.52:0/2431036662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 0x560228855ae0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7facf0009b40 tx=0x7facf0010f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae51026b57aa7592 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.585+0000 7facdd7fa640 1 -- 172.21.15.52:0/2431036662 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf003b070 con 0x560228859ca0 2024-04-02T14:02:18.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.585+0000 7facdd7fa640 1 -- 172.21.15.52:0/2431036662 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facf001ebf0 con 0x560228859ca0 2024-04-02T14:02:18.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.585+0000 7facdd7fa640 1 -- 172.21.15.52:0/2431036662 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf0029430 con 0x560228859ca0 2024-04-02T14:02:18.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.585+0000 7fad00bec740 1 -- 172.21.15.52:0/2431036662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 msgr2=0x560228855ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.585+0000 7fad00bec740 1 --2- 172.21.15.52:0/2431036662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 0x560228855ae0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7facf0009b40 tx=0x7facf0010f20 comp rx=0 tx=0).stop 2024-04-02T14:02:18.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.585+0000 7fad00bec740 1 -- 172.21.15.52:0/2431036662 shutdown_connections 2024-04-02T14:02:18.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.585+0000 7fad00bec740 1 --2- 172.21.15.52:0/2431036662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 0x560228855ae0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.585+0000 7fad00bec740 1 --2- 172.21.15.52:0/2431036662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5602288510c0 0x560228865950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.585+0000 7fad00bec740 1 --2- 172.21.15.52:0/2431036662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602288507a0 0x560228859230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.585+0000 7fad00bec740 1 -- 172.21.15.52:0/2431036662 >> 172.21.15.52:0/2431036662 conn(0x56022884aed0 msgr2=0x5602288553d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.585+0000 7fad00bec740 1 -- 172.21.15.52:0/2431036662 shutdown_connections 2024-04-02T14:02:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.585+0000 7fad00bec740 1 -- 172.21.15.52:0/2431036662 wait complete. 2024-04-02T14:02:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.586+0000 7fad00bec740 1 -- start start 2024-04-02T14:02:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.586+0000 7fad00bec740 1 Processor -- start 2024-04-02T14:02:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.586+0000 7fad00bec740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5602288507a0 0x56022885d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.586+0000 7fad00bec740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602288510c0 0x56022885d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.586+0000 7fad00bec740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 0x5602288ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.586+0000 7fad00bec740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7facf81b0510 con 0x560228859ca0 2024-04-02T14:02:18.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.586+0000 7fad00bec740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7facf81afe40 con 0x5602288507a0 2024-04-02T14:02:18.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.586+0000 7fad00bec740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7facf81ad0a0 con 0x5602288510c0 2024-04-02T14:02:18.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.586+0000 7fad00bec740 10 client.0 _ll_register_callbacks cb 0x7facf0009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:18.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.586+0000 7facf7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 0x5602288ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.586+0000 7facf7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 0x5602288ce570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53300/0 (socket says 172.21.15.52:53300) 2024-04-02T14:02:18.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.586+0000 7facf7fff640 1 -- 172.21.15.52:0/4276997423 learned_addr learned my addr 172.21.15.52:0/4276997423 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:18.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.586+0000 7facf77fe640 1 --2- 172.21.15.52:0/4276997423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5602288507a0 0x56022885d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.587+0000 7facf7fff640 1 -- 172.21.15.52:0/4276997423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602288510c0 msgr2=0x56022885d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.587+0000 7facf7fff640 1 --2- 172.21.15.52:0/4276997423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602288510c0 0x56022885d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.587+0000 7facf7fff640 1 -- 172.21.15.52:0/4276997423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5602288507a0 msgr2=0x56022885d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.587+0000 7facf7fff640 1 --2- 172.21.15.52:0/4276997423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5602288507a0 0x56022885d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.587+0000 7facf7fff640 1 -- 172.21.15.52:0/4276997423 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602288ceab0 con 0x560228859ca0 2024-04-02T14:02:18.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.587+0000 7facf77fe640 1 --2- 172.21.15.52:0/4276997423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5602288507a0 0x56022885d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:02:18.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.587+0000 7facf7fff640 1 --2- 172.21.15.52:0/4276997423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 0x5602288ce570 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7facf00099c0 tx=0x7facf0009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9224e85ea195e844 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.587+0000 7facd6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:18.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.587+0000 7facd6ffd640 1 -- 172.21.15.52:0/4276997423 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf0044060 con 0x560228859ca0 2024-04-02T14:02:18.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.587+0000 7facd6ffd640 1 -- 172.21.15.52:0/4276997423 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9526+0+0 (secure 0 0 0) 0x7facf0029630 con 0x560228859ca0 2024-04-02T14:02:18.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.587+0000 7fad00bec740 1 -- 172.21.15.52:0/4276997423 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facf001ebf0 con 0x560228859ca0 2024-04-02T14:02:18.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.587+0000 7facd6ffd640 1 -- 172.21.15.52:0/4276997423 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7facf003b070 con 0x560228859ca0 2024-04-02T14:02:18.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.587+0000 7fad00bec740 10 client.18102 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:18.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.587+0000 7fad00bec740 1 -- 172.21.15.52:0/4276997423 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5602288cfc20 con 0x560228859ca0 2024-04-02T14:02:18.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.587+0000 7facd6ffd640 1 -- 172.21.15.52:0/4276997423 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facf001ebf0 con 0x560228859ca0 2024-04-02T14:02:18.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.587+0000 7facd6ffd640 1 -- 172.21.15.52:0/4276997423 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7facf0036480 con 0x560228859ca0 2024-04-02T14:02:18.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.587+0000 7fad00bec740 10 client.18102 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:18.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.587+0000 7fad00bec740 10 client.18102 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T14:02:18.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.587+0000 7fad00bec740 20 client.18102 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:18.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.587+0000 7fad00bec740 1 --2- 172.21.15.52:0/4276997423 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5602288d0280 0x5602288d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.587+0000 7fad00bec740 4 client.18102 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:02:18.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.587+0000 7fad00bec740 1 -- 172.21.15.52:0/4276997423 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5602287bc040 con 0x5602288d0280 2024-04-02T14:02:18.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.588+0000 7facf77fe640 1 --2- 172.21.15.52:0/4276997423 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5602288d0280 0x5602288d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.588+0000 7facf77fe640 1 --2- 172.21.15.52:0/4276997423 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5602288d0280 0x5602288d26a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6e4d3b5de7b5fc8f server_cookie=3936a99d82995e03 in_seq=0 out_seq=0 2024-04-02T14:02:18.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.588+0000 7facd6ffd640 10 client.18102 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T14:02:18.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.588+0000 7facd6ffd640 1 -- 172.21.15.52:0/4276997423 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5602287bc040 con 0x5602288d0280 2024-04-02T14:02:18.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.588+0000 7facd6ffd640 10 client.18102 handle_command_reply: tid=0 2024-04-02T14:02:18.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.588+0000 7fad00bec740 1 client.18102 shutdown 2024-04-02T14:02:18.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.588+0000 7fad00bec740 1 -- 172.21.15.52:0/4276997423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 msgr2=0x5602288ce570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.588+0000 7fad00bec740 1 --2- 172.21.15.52:0/4276997423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 0x5602288ce570 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7facf00099c0 tx=0x7facf0009b90 comp rx=0 tx=0).stop 2024-04-02T14:02:18.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.588+0000 7fad00bec740 1 -- 172.21.15.52:0/4276997423 shutdown_connections 2024-04-02T14:02:18.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.588+0000 7fad00bec740 1 --2- 172.21.15.52:0/4276997423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560228859ca0 0x5602288ce570 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.588+0000 7fad00bec740 1 --2- 172.21.15.52:0/4276997423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602288510c0 0x56022885d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.588+0000 7fad00bec740 1 --2- 172.21.15.52:0/4276997423 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x5602288d0280 0x5602288d26a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.588+0000 7fad00bec740 1 --2- 172.21.15.52:0/4276997423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5602288507a0 0x56022885d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.588+0000 7facd6ffd640 0 client.18102 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T14:02:18.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.588+0000 7fad00bec740 1 -- 172.21.15.52:0/4276997423 >> 172.21.15.52:0/4276997423 conn(0x56022884aed0 msgr2=0x560228855c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.588+0000 7fad00bec740 1 -- 172.21.15.52:0/4276997423 shutdown_connections 2024-04-02T14:02:18.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.589+0000 7fad00bec740 1 -- 172.21.15.52:0/4276997423 wait complete. 2024-04-02T14:02:18.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.589+0000 7fad00bec740 20 client.18102 trim_cache size 0 max 16384 2024-04-02T14:02:18.605 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:02:18.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.589+0000 7facfe006640 1 -- 172.21.15.52:0/2276244330 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7face00a3d20 msgr2=0x7face00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.589+0000 7facfe006640 1 --2- 172.21.15.52:0/2276244330 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7face00a3d20 0x7face00a61e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7facf806b1d0 tx=0x7facf0009600 comp rx=0 tx=0).stop 2024-04-02T14:02:18.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.589+0000 7facfe006640 1 -- 172.21.15.52:0/2276244330 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facf8102d40 msgr2=0x7facf81983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.589+0000 7facfe006640 1 --2- 172.21.15.52:0/2276244330 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facf8102d40 0x7facf81983c0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7face8005940 tx=0x7face802cd10 comp rx=0 tx=0).stop 2024-04-02T14:02:18.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.589+0000 7facfe006640 1 -- 172.21.15.52:0/2276244330 shutdown_connections 2024-04-02T14:02:18.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.589+0000 7facfe006640 1 --2- 172.21.15.52:0/2276244330 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7face00a3d20 0x7face00a61e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.589+0000 7facfe006640 1 --2- 172.21.15.52:0/2276244330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facf810c110 0x7facf819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.589+0000 7facfe006640 1 --2- 172.21.15.52:0/2276244330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facf8103660 0x7facf8198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.589+0000 7facfe006640 1 --2- 172.21.15.52:0/2276244330 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facf8102d40 0x7facf81983c0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.589+0000 7facfe006640 1 -- 172.21.15.52:0/2276244330 >> 172.21.15.52:0/2276244330 conn(0x7facf8076d80 msgr2=0x7facf80fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.589+0000 7facfe006640 1 -- 172.21.15.52:0/2276244330 shutdown_connections 2024-04-02T14:02:18.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.589+0000 7facfe006640 1 -- 172.21.15.52:0/2276244330 wait complete. 2024-04-02T14:02:18.608 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T14:02:18.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.713+0000 7f565719f640 1 -- 172.21.15.52:0/1405104160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5650106870 msgr2=0x7f5650106c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.713+0000 7f565719f640 1 --2- 172.21.15.52:0/1405104160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5650106870 0x7f5650106c50 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f564c009930 tx=0x7f564c02bc30 comp rx=0 tx=0).stop 2024-04-02T14:02:18.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.713+0000 7f565719f640 1 -- 172.21.15.52:0/1405104160 shutdown_connections 2024-04-02T14:02:18.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.713+0000 7f565719f640 1 --2- 172.21.15.52:0/1405104160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5650101220 0x7f565010d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.713+0000 7f565719f640 1 --2- 172.21.15.52:0/1405104160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5650100860 0x7f5650100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.713+0000 7f565719f640 1 --2- 172.21.15.52:0/1405104160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5650106870 0x7f5650106c50 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.713+0000 7f565719f640 1 -- 172.21.15.52:0/1405104160 >> 172.21.15.52:0/1405104160 conn(0x7f56500fc350 msgr2=0x7f56500fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.713+0000 7f565719f640 1 -- 172.21.15.52:0/1405104160 shutdown_connections 2024-04-02T14:02:18.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.713+0000 7f565719f640 1 -- 172.21.15.52:0/1405104160 wait complete. 2024-04-02T14:02:18.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.714+0000 7f565719f640 1 Processor -- start 2024-04-02T14:02:18.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.714+0000 7f565719f640 1 -- start start 2024-04-02T14:02:18.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.714+0000 7f565719f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5650100860 0x7f56501981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.714+0000 7f565719f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5650101220 0x7f5650198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.714+0000 7f565719f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5650106870 0x7f565019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.714+0000 7f565719f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5650110c40 con 0x7f5650106870 2024-04-02T14:02:18.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.714+0000 7f565719f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5650110ac0 con 0x7f5650100860 2024-04-02T14:02:18.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.714+0000 7f565719f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5650110dc0 con 0x7f5650101220 2024-04-02T14:02:18.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.715+0000 7f565699e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5650106870 0x7f565019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.715+0000 7f565699e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5650106870 0x7f565019cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53330/0 (socket says 172.21.15.52:53330) 2024-04-02T14:02:18.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.715+0000 7f565699e640 1 -- 172.21.15.52:0/3675444748 learned_addr learned my addr 172.21.15.52:0/3675444748 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:18.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.715+0000 7f565599c640 1 --2- 172.21.15.52:0/3675444748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5650101220 0x7f5650198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.715+0000 7f565619d640 1 --2- 172.21.15.52:0/3675444748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5650100860 0x7f56501981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.715+0000 7f565699e640 1 -- 172.21.15.52:0/3675444748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5650101220 msgr2=0x7f5650198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.715+0000 7f565699e640 1 --2- 172.21.15.52:0/3675444748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5650101220 0x7f5650198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.715+0000 7f565699e640 1 -- 172.21.15.52:0/3675444748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5650100860 msgr2=0x7f56501981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.715+0000 7f565699e640 1 --2- 172.21.15.52:0/3675444748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5650100860 0x7f56501981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.715+0000 7f565699e640 1 -- 172.21.15.52:0/3675444748 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565019d1e0 con 0x7f5650106870 2024-04-02T14:02:18.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.715+0000 7f565699e640 1 --2- 172.21.15.52:0/3675444748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5650106870 0x7f565019cab0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f564800cce0 tx=0x7f564800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.715+0000 7f56477fe640 1 -- 172.21.15.52:0/3675444748 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564800f450 con 0x7f5650106870 2024-04-02T14:02:18.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.715+0000 7f56477fe640 1 -- 172.21.15.52:0/3675444748 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f564800fad0 con 0x7f5650106870 2024-04-02T14:02:18.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.715+0000 7f565719f640 1 -- 172.21.15.52:0/3675444748 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f565019d4d0 con 0x7f5650106870 2024-04-02T14:02:18.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.715+0000 7f56477fe640 1 -- 172.21.15.52:0/3675444748 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5648018c10 con 0x7f5650106870 2024-04-02T14:02:18.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.715+0000 7f565719f640 1 -- 172.21.15.52:0/3675444748 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56501a4d60 con 0x7f5650106870 2024-04-02T14:02:18.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.718+0000 7f56477fe640 1 -- 172.21.15.52:0/3675444748 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f564800f5f0 con 0x7f5650106870 2024-04-02T14:02:18.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.718+0000 7f565afe2740 1 Processor -- start 2024-04-02T14:02:18.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.718+0000 7f565afe2740 1 -- start start 2024-04-02T14:02:18.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.719+0000 7f565afe2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d590c980 0x5565d590cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.719+0000 7f565afe2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5565d590d2a0 0x5565d5915940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.719+0000 7f565afe2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d5915e80 0x5565d5918270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.719+0000 7f565afe2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5565d58707b0 con 0x5565d5915e80 2024-04-02T14:02:18.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.719+0000 7f565afe2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5565d580f570 con 0x5565d590d2a0 2024-04-02T14:02:18.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.719+0000 7f565afe2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5565d586c760 con 0x5565d590c980 2024-04-02T14:02:18.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.719+0000 7f56477fe640 1 --2- 172.21.15.52:0/3675444748 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f563c0a3d20 0x7f563c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.719+0000 7f565599c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d5915e80 0x5565d5918270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.719+0000 7f565599c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d5915e80 0x5565d5918270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53336/0 (socket says 172.21.15.52:53336) 2024-04-02T14:02:18.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.719+0000 7f565599c640 1 -- 172.21.15.52:0/1079668295 learned_addr learned my addr 172.21.15.52:0/1079668295 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:18.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.719+0000 7f565699e640 1 --2- 172.21.15.52:0/3675444748 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f563c0a3d20 0x7f563c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.719+0000 7f565619d640 1 --2- 172.21.15.52:0/1079668295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5565d590d2a0 0x5565d5915940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.719+0000 7f565599c640 1 --2- 172.21.15.52:0/1079668295 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d590c980 0x5565d590cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.719+0000 7f56477fe640 1 -- 172.21.15.52:0/3675444748 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9526+0+0 (secure 0 0 0) 0x7f56480d12f0 con 0x7f5650106870 2024-04-02T14:02:18.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.719+0000 7f565599c640 1 -- 172.21.15.52:0/1079668295 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d590c980 msgr2=0x5565d590cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.719+0000 7f565599c640 1 --2- 172.21.15.52:0/1079668295 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d590c980 0x5565d590cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.719+0000 7f565599c640 1 -- 172.21.15.52:0/1079668295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5565d590d2a0 msgr2=0x5565d5915940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.719+0000 7f565599c640 1 --2- 172.21.15.52:0/1079668295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5565d590d2a0 0x5565d5915940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.719+0000 7f565599c640 1 -- 172.21.15.52:0/1079668295 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565d55843f0 con 0x5565d5915e80 2024-04-02T14:02:18.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.719+0000 7f565699e640 1 --2- 172.21.15.52:0/3675444748 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f563c0a3d20 0x7f563c0a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f564800ccb0 tx=0x7f564806e8e0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.719+0000 7f565599c640 1 --2- 172.21.15.52:0/1079668295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d5915e80 0x5565d5918270 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f56400135b0 tx=0x7f564000c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ae629f56e39baa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.720+0000 7f5644bfe640 1 -- 172.21.15.52:0/1079668295 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5640019030 con 0x5565d5915e80 2024-04-02T14:02:18.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.720+0000 7f5644bfe640 1 -- 172.21.15.52:0/1079668295 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5640005500 con 0x5565d5915e80 2024-04-02T14:02:18.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.720+0000 7f5644bfe640 1 -- 172.21.15.52:0/1079668295 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56400115a0 con 0x5565d5915e80 2024-04-02T14:02:18.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.720+0000 7f565afe2740 1 -- 172.21.15.52:0/1079668295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d5915e80 msgr2=0x5565d5918270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.720+0000 7f565afe2740 1 --2- 172.21.15.52:0/1079668295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d5915e80 0x5565d5918270 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f56400135b0 tx=0x7f564000c690 comp rx=0 tx=0).stop 2024-04-02T14:02:18.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.720+0000 7f565afe2740 1 -- 172.21.15.52:0/1079668295 shutdown_connections 2024-04-02T14:02:18.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.720+0000 7f565afe2740 1 --2- 172.21.15.52:0/1079668295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d5915e80 0x5565d5918270 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.720+0000 7f565afe2740 1 --2- 172.21.15.52:0/1079668295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5565d590d2a0 0x5565d5915940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.720+0000 7f565afe2740 1 --2- 172.21.15.52:0/1079668295 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d590c980 0x5565d590cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.720+0000 7f565afe2740 1 -- 172.21.15.52:0/1079668295 >> 172.21.15.52:0/1079668295 conn(0x5565d59070b0 msgr2=0x5565d5907490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.721+0000 7f565afe2740 1 -- 172.21.15.52:0/1079668295 shutdown_connections 2024-04-02T14:02:18.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.721+0000 7f565afe2740 1 -- 172.21.15.52:0/1079668295 wait complete. 2024-04-02T14:02:18.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.721+0000 7f565afe2740 1 -- start start 2024-04-02T14:02:18.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.721+0000 7f565afe2740 1 Processor -- start 2024-04-02T14:02:18.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.721+0000 7f565afe2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d590c980 0x5565d5912ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.721+0000 7f565afe2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d590d2a0 0x5565d5913220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.721+0000 7f565afe2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5565d5915e80 0x5565d5913760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.721+0000 7f565afe2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5565d5918c70 con 0x5565d590c980 2024-04-02T14:02:18.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.721+0000 7f565afe2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5565d582be80 con 0x5565d5915e80 2024-04-02T14:02:18.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.721+0000 7f565afe2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5565d5918df0 con 0x5565d590d2a0 2024-04-02T14:02:18.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.721+0000 7f565afe2740 10 client.0 _ll_register_callbacks cb 0x5565d5867aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:18.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.721+0000 7f565599c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d590c980 0x5565d5912ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.721+0000 7f565599c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d590c980 0x5565d5912ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53350/0 (socket says 172.21.15.52:53350) 2024-04-02T14:02:18.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.721+0000 7f565599c640 1 -- 172.21.15.52:0/706012462 learned_addr learned my addr 172.21.15.52:0/706012462 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:18.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.721+0000 7f565619d640 1 --2- 172.21.15.52:0/706012462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d590d2a0 0x5565d5913220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.721+0000 7f565599c640 1 --2- 172.21.15.52:0/706012462 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5565d5915e80 0x5565d5913760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.722+0000 7f565599c640 1 -- 172.21.15.52:0/706012462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d590d2a0 msgr2=0x5565d5913220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.722+0000 7f565599c640 1 --2- 172.21.15.52:0/706012462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d590d2a0 0x5565d5913220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.722+0000 7f565599c640 1 -- 172.21.15.52:0/706012462 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5565d5915e80 msgr2=0x5565d5913760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.722+0000 7f565599c640 1 --2- 172.21.15.52:0/706012462 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5565d5915e80 0x5565d5913760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.722+0000 7f565599c640 1 -- 172.21.15.52:0/706012462 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565d564e7d0 con 0x5565d590c980 2024-04-02T14:02:18.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.722+0000 7f565599c640 1 --2- 172.21.15.52:0/706012462 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d590c980 0x5565d5912ce0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f5640013580 tx=0x7f5640004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682a8a1fe0fc0058 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.722+0000 7f5639ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:18.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.722+0000 7f5639ffb640 1 -- 172.21.15.52:0/706012462 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5640019040 con 0x5565d590c980 2024-04-02T14:02:18.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.722+0000 7f5639ffb640 1 -- 172.21.15.52:0/706012462 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9526+0+0 (secure 0 0 0) 0x7f5640004030 con 0x5565d590c980 2024-04-02T14:02:18.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.722+0000 7f565afe2740 1 -- 172.21.15.52:0/706012462 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5640005500 con 0x5565d590c980 2024-04-02T14:02:18.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.722+0000 7f5639ffb640 1 -- 172.21.15.52:0/706012462 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5640031050 con 0x5565d590c980 2024-04-02T14:02:18.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.722+0000 7f565afe2740 10 client.18114 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:18.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.722+0000 7f565afe2740 1 -- 172.21.15.52:0/706012462 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5565d5924d40 con 0x5565d590c980 2024-04-02T14:02:18.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.722+0000 7f5639ffb640 1 -- 172.21.15.52:0/706012462 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5640005500 con 0x5565d590c980 2024-04-02T14:02:18.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.722+0000 7f5639ffb640 1 -- 172.21.15.52:0/706012462 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5640003bd0 con 0x5565d590c980 2024-04-02T14:02:18.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.723+0000 7f565afe2740 10 client.18114 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:18.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.723+0000 7f565afe2740 10 client.18114 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T14:02:18.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.723+0000 7f565afe2740 20 client.18114 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:18.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.723+0000 7f565afe2740 1 --2- 172.21.15.52:0/706012462 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5565d59261a0 0x5565d5928590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.723+0000 7f565afe2740 4 client.18114 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:02:18.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.723+0000 7f565afe2740 1 -- 172.21.15.52:0/706012462 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565d587dcf0 con 0x5565d59261a0 2024-04-02T14:02:18.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.723+0000 7f565619d640 1 --2- 172.21.15.52:0/706012462 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5565d59261a0 0x5565d5928590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.723+0000 7f565619d640 1 --2- 172.21.15.52:0/706012462 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5565d59261a0 0x5565d5928590 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f139756d21b68214 server_cookie=9a4769e5a3d7b848 in_seq=0 out_seq=0 2024-04-02T14:02:18.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.723+0000 7f5639ffb640 10 client.18114 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T14:02:18.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.724+0000 7f5639ffb640 1 -- 172.21.15.52:0/706012462 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5565d587dcf0 con 0x5565d59261a0 2024-04-02T14:02:18.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.724+0000 7f5639ffb640 10 client.18114 handle_command_reply: tid=0 2024-04-02T14:02:18.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.724+0000 7f565afe2740 1 client.18114 shutdown 2024-04-02T14:02:18.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.724+0000 7f565afe2740 1 -- 172.21.15.52:0/706012462 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d590c980 msgr2=0x5565d5912ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.724+0000 7f565afe2740 1 --2- 172.21.15.52:0/706012462 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d590c980 0x5565d5912ce0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f5640013580 tx=0x7f5640004c10 comp rx=0 tx=0).stop 2024-04-02T14:02:18.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.724+0000 7f565afe2740 1 -- 172.21.15.52:0/706012462 shutdown_connections 2024-04-02T14:02:18.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.724+0000 7f565afe2740 1 --2- 172.21.15.52:0/706012462 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5565d59261a0 0x5565d5928590 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.724+0000 7f565afe2740 1 --2- 172.21.15.52:0/706012462 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5565d5915e80 0x5565d5913760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.724+0000 7f5639ffb640 0 client.18114 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T14:02:18.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.724+0000 7f565afe2740 1 --2- 172.21.15.52:0/706012462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d590d2a0 0x5565d5913220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.724+0000 7f565afe2740 1 --2- 172.21.15.52:0/706012462 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d590c980 0x5565d5912ce0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.724+0000 7f565afe2740 1 -- 172.21.15.52:0/706012462 >> 172.21.15.52:0/706012462 conn(0x5565d59070b0 msgr2=0x5565d5908e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.724+0000 7f565afe2740 1 -- 172.21.15.52:0/706012462 shutdown_connections 2024-04-02T14:02:18.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.724+0000 7f565afe2740 1 -- 172.21.15.52:0/706012462 wait complete. 2024-04-02T14:02:18.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.724+0000 7f565afe2740 20 client.18114 trim_cache size 0 max 16384 2024-04-02T14:02:18.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.738+0000 7f565afe2740 1 Processor -- start 2024-04-02T14:02:18.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.738+0000 7f565afe2740 1 -- start start 2024-04-02T14:02:18.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.738+0000 7f565afe2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d590c980 0x5565d5915410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.738+0000 7f565afe2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5565d590d2a0 0x5565d5921b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.738+0000 7f565afe2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d5915e80 0x5565d592dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.738+0000 7f565afe2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f56501aee70 con 0x5565d5915e80 2024-04-02T14:02:18.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.738+0000 7f565afe2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f56501b09b0 con 0x5565d590d2a0 2024-04-02T14:02:18.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.738+0000 7f565afe2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f56501af540 con 0x5565d590c980 2024-04-02T14:02:18.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.738+0000 7f565599c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d5915e80 0x5565d592dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.738+0000 7f565599c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d5915e80 0x5565d592dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53356/0 (socket says 172.21.15.52:53356) 2024-04-02T14:02:18.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.738+0000 7f565599c640 1 -- 172.21.15.52:0/670032103 learned_addr learned my addr 172.21.15.52:0/670032103 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:18.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.738+0000 7f565619d640 1 --2- 172.21.15.52:0/670032103 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5565d590d2a0 0x5565d5921b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.738+0000 7f565599c640 1 --2- 172.21.15.52:0/670032103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d590c980 0x5565d5915410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.738+0000 7f565599c640 1 -- 172.21.15.52:0/670032103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d590c980 msgr2=0x5565d5915410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.738+0000 7f565599c640 1 --2- 172.21.15.52:0/670032103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d590c980 0x5565d5915410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.738+0000 7f565599c640 1 -- 172.21.15.52:0/670032103 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5565d590d2a0 msgr2=0x5565d5921b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.738+0000 7f565599c640 1 --2- 172.21.15.52:0/670032103 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5565d590d2a0 0x5565d5921b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.738+0000 7f565599c640 1 -- 172.21.15.52:0/670032103 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565d5912240 con 0x5565d5915e80 2024-04-02T14:02:18.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.738+0000 7f565599c640 1 --2- 172.21.15.52:0/670032103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d5915e80 0x5565d592dd00 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f56400117e0 tx=0x7f5640005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47b2e2fb440b032 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.738+0000 7f563bfff640 1 -- 172.21.15.52:0/670032103 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5640039070 con 0x5565d5915e80 2024-04-02T14:02:18.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.738+0000 7f563bfff640 1 -- 172.21.15.52:0/670032103 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f564000c430 con 0x5565d5915e80 2024-04-02T14:02:18.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.739+0000 7f563bfff640 1 -- 172.21.15.52:0/670032103 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5640013290 con 0x5565d5915e80 2024-04-02T14:02:18.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.739+0000 7f565afe2740 1 -- 172.21.15.52:0/670032103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d5915e80 msgr2=0x5565d592dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.739+0000 7f565afe2740 1 --2- 172.21.15.52:0/670032103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d5915e80 0x5565d592dd00 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f56400117e0 tx=0x7f5640005970 comp rx=0 tx=0).stop 2024-04-02T14:02:18.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.739+0000 7f565afe2740 1 -- 172.21.15.52:0/670032103 shutdown_connections 2024-04-02T14:02:18.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.739+0000 7f565afe2740 1 --2- 172.21.15.52:0/670032103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d5915e80 0x5565d592dd00 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.739+0000 7f565afe2740 1 --2- 172.21.15.52:0/670032103 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5565d590d2a0 0x5565d5921b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.739+0000 7f565afe2740 1 --2- 172.21.15.52:0/670032103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d590c980 0x5565d5915410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.739+0000 7f565afe2740 1 -- 172.21.15.52:0/670032103 >> 172.21.15.52:0/670032103 conn(0x5565d59070b0 msgr2=0x5565d5906150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.739+0000 7f565afe2740 1 -- 172.21.15.52:0/670032103 shutdown_connections 2024-04-02T14:02:18.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.739+0000 7f565afe2740 1 -- 172.21.15.52:0/670032103 wait complete. 2024-04-02T14:02:18.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.739+0000 7f565afe2740 1 -- start start 2024-04-02T14:02:18.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.739+0000 7f565afe2740 1 Processor -- start 2024-04-02T14:02:18.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.740+0000 7f565afe2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d590c980 0x5565d5916860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.740+0000 7f565afe2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5565d590d2a0 0x5565d5916da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.740+0000 7f565afe2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d5915e80 0x5565d591d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.740+0000 7f565afe2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f56501b02e0 con 0x5565d590c980 2024-04-02T14:02:18.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.740+0000 7f565afe2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f56501afc10 con 0x5565d590d2a0 2024-04-02T14:02:18.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.740+0000 7f565afe2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f56501ace70 con 0x5565d5915e80 2024-04-02T14:02:18.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.740+0000 7f565afe2740 10 client.0 _ll_register_callbacks cb 0x7f564c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:18.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.740+0000 7f565599c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d590c980 0x5565d5916860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.740+0000 7f565599c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d590c980 0x5565d5916860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53362/0 (socket says 172.21.15.52:53362) 2024-04-02T14:02:18.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.740+0000 7f565599c640 1 -- 172.21.15.52:0/1673090411 learned_addr learned my addr 172.21.15.52:0/1673090411 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:18.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.740+0000 7f565599c640 1 -- 172.21.15.52:0/1673090411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d5915e80 msgr2=0x5565d591d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.740+0000 7f565599c640 1 --2- 172.21.15.52:0/1673090411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d5915e80 0x5565d591d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.740+0000 7f565599c640 1 -- 172.21.15.52:0/1673090411 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5565d590d2a0 msgr2=0x5565d5916da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.740+0000 7f565599c640 1 --2- 172.21.15.52:0/1673090411 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5565d590d2a0 0x5565d5916da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.740+0000 7f565599c640 1 -- 172.21.15.52:0/1673090411 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565d591d550 con 0x5565d590c980 2024-04-02T14:02:18.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.740+0000 7f565599c640 1 --2- 172.21.15.52:0/1673090411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d590c980 0x5565d5916860 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f5640011690 tx=0x7f564000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de1611a17d711e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.740+0000 7f5639ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:18.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.740+0000 7f5639ffb640 1 -- 172.21.15.52:0/1673090411 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5640019070 con 0x5565d590c980 2024-04-02T14:02:18.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.740+0000 7f5639ffb640 1 -- 172.21.15.52:0/1673090411 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9526+0+0 (secure 0 0 0) 0x7f5640013040 con 0x5565d590c980 2024-04-02T14:02:18.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.740+0000 7f565afe2740 1 -- 172.21.15.52:0/1673090411 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f564000c430 con 0x5565d590c980 2024-04-02T14:02:18.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.740+0000 7f5639ffb640 1 -- 172.21.15.52:0/1673090411 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5640039040 con 0x5565d590c980 2024-04-02T14:02:18.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.740+0000 7f565afe2740 10 client.18120 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:18.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.740+0000 7f565afe2740 1 -- 172.21.15.52:0/1673090411 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5565d591e850 con 0x5565d590c980 2024-04-02T14:02:18.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.741+0000 7f5639ffb640 1 -- 172.21.15.52:0/1673090411 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f564000c430 con 0x5565d590c980 2024-04-02T14:02:18.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.741+0000 7f5639ffb640 1 -- 172.21.15.52:0/1673090411 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f564003c070 con 0x5565d590c980 2024-04-02T14:02:18.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.741+0000 7f565afe2740 10 client.18120 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:18.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.741+0000 7f565afe2740 10 client.18120 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T14:02:18.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.741+0000 7f565afe2740 20 client.18120 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:18.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.741+0000 7f565afe2740 1 --2- 172.21.15.52:0/1673090411 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5565d598c8b0 0x5565d598eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.741+0000 7f565afe2740 4 client.18120 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:02:18.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.741+0000 7f565afe2740 1 -- 172.21.15.52:0/1673090411 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565d5878040 con 0x5565d598c8b0 2024-04-02T14:02:18.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.741+0000 7f565619d640 1 --2- 172.21.15.52:0/1673090411 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5565d598c8b0 0x5565d598eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.741+0000 7f565619d640 1 --2- 172.21.15.52:0/1673090411 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5565d598c8b0 0x5565d598eca0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6fbc3a069e1bd43c server_cookie=bc2f1ac7005460be in_seq=0 out_seq=0 2024-04-02T14:02:18.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.741+0000 7f5639ffb640 10 client.18120 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T14:02:18.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f5639ffb640 1 -- 172.21.15.52:0/1673090411 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565d5878040 con 0x5565d598c8b0 2024-04-02T14:02:18.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f5639ffb640 10 client.18120 handle_command_reply: tid=0 2024-04-02T14:02:18.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f565afe2740 1 client.18120 shutdown 2024-04-02T14:02:18.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f565afe2740 1 -- 172.21.15.52:0/1673090411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d590c980 msgr2=0x5565d5916860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f565afe2740 1 --2- 172.21.15.52:0/1673090411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d590c980 0x5565d5916860 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f5640011690 tx=0x7f564000cc00 comp rx=0 tx=0).stop 2024-04-02T14:02:18.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f565afe2740 1 -- 172.21.15.52:0/1673090411 shutdown_connections 2024-04-02T14:02:18.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f565afe2740 1 --2- 172.21.15.52:0/1673090411 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5565d598c8b0 0x5565d598eca0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f565afe2740 1 --2- 172.21.15.52:0/1673090411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565d5915e80 0x5565d591d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f5639ffb640 0 client.18120 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T14:02:18.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f565afe2740 1 --2- 172.21.15.52:0/1673090411 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5565d590d2a0 0x5565d5916da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f565afe2740 1 --2- 172.21.15.52:0/1673090411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565d590c980 0x5565d5916860 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f565afe2740 1 -- 172.21.15.52:0/1673090411 >> 172.21.15.52:0/1673090411 conn(0x5565d59070b0 msgr2=0x5565d5912c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f565afe2740 1 -- 172.21.15.52:0/1673090411 shutdown_connections 2024-04-02T14:02:18.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f565afe2740 1 -- 172.21.15.52:0/1673090411 wait complete. 2024-04-02T14:02:18.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f565afe2740 20 client.18120 trim_cache size 0 max 16384 2024-04-02T14:02:18.759 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:02:18.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f565719f640 1 -- 172.21.15.52:0/3675444748 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f563c0a3d20 msgr2=0x7f563c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f565719f640 1 --2- 172.21.15.52:0/3675444748 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f563c0a3d20 0x7f563c0a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f564800ccb0 tx=0x7f564806e8e0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f565719f640 1 -- 172.21.15.52:0/3675444748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5650106870 msgr2=0x7f565019cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f565719f640 1 --2- 172.21.15.52:0/3675444748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5650106870 0x7f565019cab0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f564800cce0 tx=0x7f564800a620 comp rx=0 tx=0).stop 2024-04-02T14:02:18.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f565719f640 1 -- 172.21.15.52:0/3675444748 shutdown_connections 2024-04-02T14:02:18.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f565719f640 1 --2- 172.21.15.52:0/3675444748 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f563c0a3d20 0x7f563c0a61e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f565719f640 1 --2- 172.21.15.52:0/3675444748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5650106870 0x7f565019cab0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f565719f640 1 --2- 172.21.15.52:0/3675444748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5650101220 0x7f5650198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f565719f640 1 --2- 172.21.15.52:0/3675444748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5650100860 0x7f56501981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f565719f640 1 -- 172.21.15.52:0/3675444748 >> 172.21.15.52:0/3675444748 conn(0x7f56500fc350 msgr2=0x7f565006a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f565719f640 1 -- 172.21.15.52:0/3675444748 shutdown_connections 2024-04-02T14:02:18.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.742+0000 7f565719f640 1 -- 172.21.15.52:0/3675444748 wait complete. 2024-04-02T14:02:18.762 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T14:02:18.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.864+0000 7fd63c92b640 1 -- 172.21.15.52:0/1813100841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd63810c160 msgr2=0x7fd63810e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.864+0000 7fd63c92b640 1 --2- 172.21.15.52:0/1813100841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd63810c160 0x7fd63810e5b0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fd630009ca0 tx=0x7fd63002c180 comp rx=0 tx=0).stop 2024-04-02T14:02:18.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.864+0000 7fd63c92b640 1 -- 172.21.15.52:0/1813100841 shutdown_connections 2024-04-02T14:02:18.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.864+0000 7fd63c92b640 1 --2- 172.21.15.52:0/1813100841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd63810c160 0x7fd63810e5b0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.864+0000 7fd63c92b640 1 --2- 172.21.15.52:0/1813100841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6381036b0 0x7fd63810bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.864+0000 7fd63c92b640 1 --2- 172.21.15.52:0/1813100841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd638102d90 0x7fd638103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.864+0000 7fd63c92b640 1 -- 172.21.15.52:0/1813100841 >> 172.21.15.52:0/1813100841 conn(0x7fd638077b60 msgr2=0x7fd638077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.864+0000 7fd63c92b640 1 -- 172.21.15.52:0/1813100841 shutdown_connections 2024-04-02T14:02:18.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.864+0000 7fd63c92b640 1 -- 172.21.15.52:0/1813100841 wait complete. 2024-04-02T14:02:18.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.864+0000 7fd63c92b640 1 Processor -- start 2024-04-02T14:02:18.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.865+0000 7fd63c92b640 1 -- start start 2024-04-02T14:02:18.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.865+0000 7fd63c92b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd638102d90 0x7fd6381983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.865+0000 7fd63c92b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6381036b0 0x7fd638198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.865+0000 7fd63c92b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd63810c160 0x7fd63819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.865+0000 7fd63c92b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd638111f00 con 0x7fd638102d90 2024-04-02T14:02:18.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.865+0000 7fd63c92b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd638111d80 con 0x7fd63810c160 2024-04-02T14:02:18.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.865+0000 7fd63c92b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd638112080 con 0x7fd6381036b0 2024-04-02T14:02:18.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.865+0000 7fd63659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd638102d90 0x7fd6381983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.865+0000 7fd63659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd638102d90 0x7fd6381983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53380/0 (socket says 172.21.15.52:53380) 2024-04-02T14:02:18.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.865+0000 7fd63659c640 1 -- 172.21.15.52:0/3344525826 learned_addr learned my addr 172.21.15.52:0/3344525826 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:18.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.865+0000 7fd62ffff640 1 --2- 172.21.15.52:0/3344525826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6381036b0 0x7fd638198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.865+0000 7fd636d9d640 1 --2- 172.21.15.52:0/3344525826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd63810c160 0x7fd63819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.865+0000 7fd63659c640 1 -- 172.21.15.52:0/3344525826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6381036b0 msgr2=0x7fd638198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.865+0000 7fd63659c640 1 --2- 172.21.15.52:0/3344525826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6381036b0 0x7fd638198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.865+0000 7fd63659c640 1 -- 172.21.15.52:0/3344525826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd63810c160 msgr2=0x7fd63819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.865+0000 7fd63659c640 1 --2- 172.21.15.52:0/3344525826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd63810c160 0x7fd63819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.865+0000 7fd63659c640 1 -- 172.21.15.52:0/3344525826 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63819d380 con 0x7fd638102d90 2024-04-02T14:02:18.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.865+0000 7fd63659c640 1 --2- 172.21.15.52:0/3344525826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd638102d90 0x7fd6381983e0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fd62800c890 tx=0x7fd62800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.865+0000 7fd62f7fe640 1 -- 172.21.15.52:0/3344525826 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628004220 con 0x7fd638102d90 2024-04-02T14:02:18.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.866+0000 7fd62f7fe640 1 -- 172.21.15.52:0/3344525826 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6280043c0 con 0x7fd638102d90 2024-04-02T14:02:18.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.866+0000 7fd63c92b640 1 -- 172.21.15.52:0/3344525826 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63819d670 con 0x7fd638102d90 2024-04-02T14:02:18.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.866+0000 7fd62f7fe640 1 -- 172.21.15.52:0/3344525826 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6280049e0 con 0x7fd638102d90 2024-04-02T14:02:18.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.866+0000 7fd63c92b640 1 -- 172.21.15.52:0/3344525826 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6381a4f00 con 0x7fd638102d90 2024-04-02T14:02:18.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.868+0000 7fd62f7fe640 1 -- 172.21.15.52:0/3344525826 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd6280079b0 con 0x7fd638102d90 2024-04-02T14:02:18.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.868+0000 7fd63f5ef740 1 Processor -- start 2024-04-02T14:02:18.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.868+0000 7fd63f5ef740 1 -- start start 2024-04-02T14:02:18.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.868+0000 7fd63f5ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6d38504d0 0x55c6d38508b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.868+0000 7fd63f5ef740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 0x55c6d385db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.868+0000 7fd63f5ef740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6d385e0b0 0x55c6d38604a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.869+0000 7fd63f5ef740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c6d37b47b0 con 0x55c6d3850df0 2024-04-02T14:02:18.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.869+0000 7fd63f5ef740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c6d3753570 con 0x55c6d385e0b0 2024-04-02T14:02:18.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.869+0000 7fd63f5ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c6d37b0760 con 0x55c6d38504d0 2024-04-02T14:02:18.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.869+0000 7fd62f7fe640 1 --2- 172.21.15.52:0/3344525826 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd6240a3d20 0x7fd6240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.869+0000 7fd63659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 0x55c6d385db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.869+0000 7fd63659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 0x55c6d385db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53390/0 (socket says 172.21.15.52:53390) 2024-04-02T14:02:18.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.869+0000 7fd63659c640 1 -- 172.21.15.52:0/2822846844 learned_addr learned my addr 172.21.15.52:0/2822846844 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:18.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.869+0000 7fd636d9d640 1 --2- 172.21.15.52:0/3344525826 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd6240a3d20 0x7fd6240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.869+0000 7fd62f7fe640 1 -- 172.21.15.52:0/3344525826 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9526+0+0 (secure 0 0 0) 0x7fd6280d0cd0 con 0x7fd638102d90 2024-04-02T14:02:18.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.869+0000 7fd62ffff640 1 --2- 172.21.15.52:0/2822846844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6d385e0b0 0x55c6d38604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.869+0000 7fd636d9d640 1 --2- 172.21.15.52:0/2822846844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6d38504d0 0x55c6d38508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.869+0000 7fd63659c640 1 -- 172.21.15.52:0/2822846844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6d38504d0 msgr2=0x55c6d38508b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.869+0000 7fd63659c640 1 --2- 172.21.15.52:0/2822846844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6d38504d0 0x55c6d38508b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.869+0000 7fd63659c640 1 -- 172.21.15.52:0/2822846844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6d385e0b0 msgr2=0x55c6d38604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.869+0000 7fd63659c640 1 --2- 172.21.15.52:0/2822846844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6d385e0b0 0x55c6d38604a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.869+0000 7fd63659c640 1 -- 172.21.15.52:0/2822846844 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6d34c83f0 con 0x55c6d3850df0 2024-04-02T14:02:18.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.869+0000 7fd63659c640 1 --2- 172.21.15.52:0/2822846844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 0x55c6d385db70 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fd628098ce0 tx=0x7fd6280c7990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66bb015963c0a058 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.869+0000 7fd636d9d640 1 --2- 172.21.15.52:0/3344525826 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd6240a3d20 0x7fd6240a61e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd630009430 tx=0x7fd63002e040 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.869+0000 7fd62cff9640 1 -- 172.21.15.52:0/2822846844 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62801d050 con 0x55c6d3850df0 2024-04-02T14:02:18.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.869+0000 7fd62cff9640 1 -- 172.21.15.52:0/2822846844 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6280d2070 con 0x55c6d3850df0 2024-04-02T14:02:18.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.869+0000 7fd62cff9640 1 -- 172.21.15.52:0/2822846844 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628098ef0 con 0x55c6d3850df0 2024-04-02T14:02:18.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.870+0000 7fd63f5ef740 1 -- 172.21.15.52:0/2822846844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 msgr2=0x55c6d385db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.870+0000 7fd63f5ef740 1 --2- 172.21.15.52:0/2822846844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 0x55c6d385db70 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fd628098ce0 tx=0x7fd6280c7990 comp rx=0 tx=0).stop 2024-04-02T14:02:18.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.870+0000 7fd63f5ef740 1 -- 172.21.15.52:0/2822846844 shutdown_connections 2024-04-02T14:02:18.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.870+0000 7fd63f5ef740 1 --2- 172.21.15.52:0/2822846844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6d385e0b0 0x55c6d38604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.870+0000 7fd63f5ef740 1 --2- 172.21.15.52:0/2822846844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 0x55c6d385db70 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.870+0000 7fd63f5ef740 1 --2- 172.21.15.52:0/2822846844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6d38504d0 0x55c6d38508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.870+0000 7fd63f5ef740 1 -- 172.21.15.52:0/2822846844 >> 172.21.15.52:0/2822846844 conn(0x55c6d384aed0 msgr2=0x55c6d384b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.870+0000 7fd63f5ef740 1 -- 172.21.15.52:0/2822846844 shutdown_connections 2024-04-02T14:02:18.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.870+0000 7fd63f5ef740 1 -- 172.21.15.52:0/2822846844 wait complete. 2024-04-02T14:02:18.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.871+0000 7fd63f5ef740 1 -- start start 2024-04-02T14:02:18.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.871+0000 7fd63f5ef740 1 Processor -- start 2024-04-02T14:02:18.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.871+0000 7fd63f5ef740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6d38504d0 0x55c6d385af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.871+0000 7fd63f5ef740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 0x55c6d385b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.871+0000 7fd63f5ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6d385e0b0 0x55c6d385b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.871+0000 7fd63f5ef740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c6d3860e70 con 0x55c6d3850df0 2024-04-02T14:02:18.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.871+0000 7fd63f5ef740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c6d376fe80 con 0x55c6d38504d0 2024-04-02T14:02:18.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.871+0000 7fd63f5ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c6d3860ff0 con 0x55c6d385e0b0 2024-04-02T14:02:18.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.871+0000 7fd63f5ef740 10 client.0 _ll_register_callbacks cb 0x55c6d37abaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:18.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.871+0000 7fd62ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 0x55c6d385b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.871+0000 7fd62ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 0x55c6d385b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53394/0 (socket says 172.21.15.52:53394) 2024-04-02T14:02:18.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.871+0000 7fd62ffff640 1 -- 172.21.15.52:0/1672063932 learned_addr learned my addr 172.21.15.52:0/1672063932 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:18.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.871+0000 7fd63659c640 1 --2- 172.21.15.52:0/1672063932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6d38504d0 0x55c6d385af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.871+0000 7fd636d9d640 1 --2- 172.21.15.52:0/1672063932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6d385e0b0 0x55c6d385b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.871+0000 7fd62ffff640 1 -- 172.21.15.52:0/1672063932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6d385e0b0 msgr2=0x55c6d385b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.871+0000 7fd62ffff640 1 --2- 172.21.15.52:0/1672063932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6d385e0b0 0x55c6d385b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.871+0000 7fd62ffff640 1 -- 172.21.15.52:0/1672063932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6d38504d0 msgr2=0x55c6d385af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.871+0000 7fd62ffff640 1 --2- 172.21.15.52:0/1672063932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6d38504d0 0x55c6d385af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.871+0000 7fd62ffff640 1 -- 172.21.15.52:0/1672063932 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6d35927d0 con 0x55c6d3850df0 2024-04-02T14:02:18.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.871+0000 7fd636d9d640 1 --2- 172.21.15.52:0/1672063932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6d385e0b0 0x55c6d385b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:02:18.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.871+0000 7fd62ffff640 1 --2- 172.21.15.52:0/1672063932 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 0x55c6d385b460 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fd62000bec0 tx=0x7fd620012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e4b25cb1639a91 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.871+0000 7fd61dffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:18.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.872+0000 7fd61dffb640 1 -- 172.21.15.52:0/1672063932 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6200108a0 con 0x55c6d3850df0 2024-04-02T14:02:18.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.872+0000 7fd61dffb640 1 -- 172.21.15.52:0/1672063932 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9526+0+0 (secure 0 0 0) 0x7fd620010a40 con 0x55c6d3850df0 2024-04-02T14:02:18.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.872+0000 7fd63f5ef740 1 -- 172.21.15.52:0/1672063932 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6280d2070 con 0x55c6d3850df0 2024-04-02T14:02:18.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.872+0000 7fd61dffb640 1 -- 172.21.15.52:0/1672063932 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd620010c40 con 0x55c6d3850df0 2024-04-02T14:02:18.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.872+0000 7fd63f5ef740 10 client.18132 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:18.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.872+0000 7fd63f5ef740 1 -- 172.21.15.52:0/1672063932 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c6d386d000 con 0x55c6d3850df0 2024-04-02T14:02:18.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.872+0000 7fd61dffb640 1 -- 172.21.15.52:0/1672063932 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6280d2070 con 0x55c6d3850df0 2024-04-02T14:02:18.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.872+0000 7fd61dffb640 1 -- 172.21.15.52:0/1672063932 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd620019070 con 0x55c6d3850df0 2024-04-02T14:02:18.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.872+0000 7fd63f5ef740 10 client.18132 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:18.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.872+0000 7fd63f5ef740 10 client.18132 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T14:02:18.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.872+0000 7fd63f5ef740 20 client.18132 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:18.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.872+0000 7fd63f5ef740 1 --2- 172.21.15.52:0/1672063932 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55c6d386e560 0x55c6d38709c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.872+0000 7fd63f5ef740 4 client.18132 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:02:18.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.872+0000 7fd63f5ef740 1 -- 172.21.15.52:0/1672063932 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6d37c1cf0 con 0x55c6d386e560 2024-04-02T14:02:18.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.873+0000 7fd63659c640 1 --2- 172.21.15.52:0/1672063932 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55c6d386e560 0x55c6d38709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.873+0000 7fd63659c640 1 --2- 172.21.15.52:0/1672063932 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55c6d386e560 0x55c6d38709c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=196854d5bb926393 server_cookie=64d852c6d8e53090 in_seq=0 out_seq=0 2024-04-02T14:02:18.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.873+0000 7fd61dffb640 10 client.18132 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T14:02:18.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.874+0000 7fd61dffb640 1 -- 172.21.15.52:0/1672063932 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c6d37c1cf0 con 0x55c6d386e560 2024-04-02T14:02:18.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.874+0000 7fd61dffb640 10 client.18132 handle_command_reply: tid=0 2024-04-02T14:02:18.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.874+0000 7fd63f5ef740 1 client.18132 shutdown 2024-04-02T14:02:18.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.874+0000 7fd63f5ef740 1 -- 172.21.15.52:0/1672063932 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 msgr2=0x55c6d385b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.874+0000 7fd63f5ef740 1 --2- 172.21.15.52:0/1672063932 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 0x55c6d385b460 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fd62000bec0 tx=0x7fd620012820 comp rx=0 tx=0).stop 2024-04-02T14:02:18.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.874+0000 7fd63f5ef740 1 -- 172.21.15.52:0/1672063932 shutdown_connections 2024-04-02T14:02:18.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.874+0000 7fd63f5ef740 1 --2- 172.21.15.52:0/1672063932 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55c6d386e560 0x55c6d38709c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.874+0000 7fd63f5ef740 1 --2- 172.21.15.52:0/1672063932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6d385e0b0 0x55c6d385b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.874+0000 7fd63f5ef740 1 --2- 172.21.15.52:0/1672063932 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 0x55c6d385b460 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.874+0000 7fd61dffb640 0 client.18132 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T14:02:18.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.874+0000 7fd63f5ef740 1 --2- 172.21.15.52:0/1672063932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6d38504d0 0x55c6d385af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.874+0000 7fd63f5ef740 1 -- 172.21.15.52:0/1672063932 >> 172.21.15.52:0/1672063932 conn(0x55c6d384aed0 msgr2=0x55c6d384cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.874+0000 7fd63f5ef740 1 -- 172.21.15.52:0/1672063932 shutdown_connections 2024-04-02T14:02:18.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.874+0000 7fd63f5ef740 1 -- 172.21.15.52:0/1672063932 wait complete. 2024-04-02T14:02:18.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.874+0000 7fd63f5ef740 20 client.18132 trim_cache size 0 max 16384 2024-04-02T14:02:18.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.887+0000 7fd63f5ef740 1 Processor -- start 2024-04-02T14:02:18.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.887+0000 7fd63f5ef740 1 -- start start 2024-04-02T14:02:18.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.887+0000 7fd63f5ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6d38504d0 0x55c6d385d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.887+0000 7fd63f5ef740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 0x55c6d3869d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.887+0000 7fd63f5ef740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6d385e0b0 0x55c6d385a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.887+0000 7fd63f5ef740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd6381acf10 con 0x55c6d3850df0 2024-04-02T14:02:18.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.887+0000 7fd63f5ef740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd6381aea50 con 0x55c6d385e0b0 2024-04-02T14:02:18.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.887+0000 7fd63f5ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd6381ad5e0 con 0x55c6d38504d0 2024-04-02T14:02:18.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.887+0000 7fd62ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 0x55c6d3869d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.887+0000 7fd62ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 0x55c6d3869d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53396/0 (socket says 172.21.15.52:53396) 2024-04-02T14:02:18.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.887+0000 7fd62ffff640 1 -- 172.21.15.52:0/1199033513 learned_addr learned my addr 172.21.15.52:0/1199033513 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:18.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.888+0000 7fd62ffff640 1 -- 172.21.15.52:0/1199033513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6d38504d0 msgr2=0x55c6d385d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.888+0000 7fd62ffff640 1 --2- 172.21.15.52:0/1199033513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6d38504d0 0x55c6d385d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.888+0000 7fd62ffff640 1 -- 172.21.15.52:0/1199033513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6d385e0b0 msgr2=0x55c6d385a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.888+0000 7fd62ffff640 1 --2- 172.21.15.52:0/1199033513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6d385e0b0 0x55c6d385a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.888+0000 7fd62ffff640 1 -- 172.21.15.52:0/1199033513 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6d3850c20 con 0x55c6d3850df0 2024-04-02T14:02:18.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.888+0000 7fd62ffff640 1 --2- 172.21.15.52:0/1199033513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 0x55c6d3869d70 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fd620005bc0 tx=0x7fd620005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ef63f5395b439b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.888+0000 7fd61ffff640 1 -- 172.21.15.52:0/1199033513 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620010de0 con 0x55c6d3850df0 2024-04-02T14:02:18.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.888+0000 7fd61ffff640 1 -- 172.21.15.52:0/1199033513 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd620004040 con 0x55c6d3850df0 2024-04-02T14:02:18.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.888+0000 7fd61ffff640 1 -- 172.21.15.52:0/1199033513 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62001ab20 con 0x55c6d3850df0 2024-04-02T14:02:18.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.888+0000 7fd63f5ef740 1 -- 172.21.15.52:0/1199033513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 msgr2=0x55c6d3869d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.888+0000 7fd63f5ef740 1 --2- 172.21.15.52:0/1199033513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 0x55c6d3869d70 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fd620005bc0 tx=0x7fd620005c50 comp rx=0 tx=0).stop 2024-04-02T14:02:18.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.889+0000 7fd63f5ef740 1 -- 172.21.15.52:0/1199033513 shutdown_connections 2024-04-02T14:02:18.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.889+0000 7fd63f5ef740 1 --2- 172.21.15.52:0/1199033513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6d385e0b0 0x55c6d385a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.889+0000 7fd63f5ef740 1 --2- 172.21.15.52:0/1199033513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 0x55c6d3869d70 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.889+0000 7fd63f5ef740 1 --2- 172.21.15.52:0/1199033513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6d38504d0 0x55c6d385d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.889+0000 7fd63f5ef740 1 -- 172.21.15.52:0/1199033513 >> 172.21.15.52:0/1199033513 conn(0x55c6d384aed0 msgr2=0x55c6d385aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.889+0000 7fd63f5ef740 1 -- 172.21.15.52:0/1199033513 shutdown_connections 2024-04-02T14:02:18.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.889+0000 7fd63f5ef740 1 -- 172.21.15.52:0/1199033513 wait complete. 2024-04-02T14:02:18.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.889+0000 7fd63f5ef740 1 -- start start 2024-04-02T14:02:18.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.889+0000 7fd63f5ef740 1 Processor -- start 2024-04-02T14:02:18.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.889+0000 7fd63f5ef740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6d38504d0 0x55c6d3866590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.889+0000 7fd63f5ef740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 0x55c6d38578c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.889+0000 7fd63f5ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6d385e0b0 0x55c6d3857e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.889+0000 7fd63f5ef740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd6381ae380 con 0x55c6d3850df0 2024-04-02T14:02:18.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.889+0000 7fd63f5ef740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd6381adcb0 con 0x55c6d38504d0 2024-04-02T14:02:18.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.889+0000 7fd63f5ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd6381aaf10 con 0x55c6d385e0b0 2024-04-02T14:02:18.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.889+0000 7fd63f5ef740 10 client.0 _ll_register_callbacks cb 0x55c6d385e000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:18.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.889+0000 7fd62ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 0x55c6d38578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.889+0000 7fd62ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 0x55c6d38578c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53410/0 (socket says 172.21.15.52:53410) 2024-04-02T14:02:18.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.889+0000 7fd62ffff640 1 -- 172.21.15.52:0/1291387182 learned_addr learned my addr 172.21.15.52:0/1291387182 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:18.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.889+0000 7fd636d9d640 1 --2- 172.21.15.52:0/1291387182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6d385e0b0 0x55c6d3857e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.889+0000 7fd63659c640 1 --2- 172.21.15.52:0/1291387182 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6d38504d0 0x55c6d3866590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.889+0000 7fd62ffff640 1 -- 172.21.15.52:0/1291387182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6d385e0b0 msgr2=0x55c6d3857e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.890+0000 7fd62ffff640 1 --2- 172.21.15.52:0/1291387182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6d385e0b0 0x55c6d3857e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.890+0000 7fd62ffff640 1 -- 172.21.15.52:0/1291387182 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6d38504d0 msgr2=0x55c6d3866590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:18.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.890+0000 7fd62ffff640 1 --2- 172.21.15.52:0/1291387182 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6d38504d0 0x55c6d3866590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.890+0000 7fd62ffff640 1 -- 172.21.15.52:0/1291387182 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6d3858570 con 0x55c6d3850df0 2024-04-02T14:02:18.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.890+0000 7fd63659c640 1 --2- 172.21.15.52:0/1291387182 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6d38504d0 0x55c6d3866590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:02:18.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.890+0000 7fd62ffff640 1 --2- 172.21.15.52:0/1291387182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 0x55c6d38578c0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fd620000c00 tx=0x7fd620012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee7687b195c2a06 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:18.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.890+0000 7fd61dffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:18.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.890+0000 7fd61dffb640 1 -- 172.21.15.52:0/1291387182 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620039070 con 0x55c6d3850df0 2024-04-02T14:02:18.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.890+0000 7fd61dffb640 1 -- 172.21.15.52:0/1291387182 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9526+0+0 (secure 0 0 0) 0x7fd620004400 con 0x55c6d3850df0 2024-04-02T14:02:18.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.890+0000 7fd63f5ef740 1 -- 172.21.15.52:0/1291387182 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd620004040 con 0x55c6d3850df0 2024-04-02T14:02:18.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.890+0000 7fd61dffb640 1 -- 172.21.15.52:0/1291387182 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd620045050 con 0x55c6d3850df0 2024-04-02T14:02:18.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.890+0000 7fd63f5ef740 10 client.18138 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:18.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.890+0000 7fd63f5ef740 1 -- 172.21.15.52:0/1291387182 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c6d38d4aa0 con 0x55c6d3850df0 2024-04-02T14:02:18.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.890+0000 7fd61dffb640 1 -- 172.21.15.52:0/1291387182 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd620004040 con 0x55c6d3850df0 2024-04-02T14:02:18.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.890+0000 7fd61dffb640 1 -- 172.21.15.52:0/1291387182 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd620029070 con 0x55c6d3850df0 2024-04-02T14:02:18.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.890+0000 7fd63f5ef740 10 client.18138 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:18.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.890+0000 7fd63f5ef740 10 client.18138 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T14:02:18.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.890+0000 7fd63f5ef740 20 client.18138 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:18.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.890+0000 7fd63f5ef740 1 --2- 172.21.15.52:0/1291387182 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55c6d38d5150 0x55c6d38d7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:18.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.890+0000 7fd63f5ef740 4 client.18138 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:02:18.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.890+0000 7fd63f5ef740 1 -- 172.21.15.52:0/1291387182 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6d37bc040 con 0x55c6d38d5150 2024-04-02T14:02:18.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.891+0000 7fd63659c640 1 --2- 172.21.15.52:0/1291387182 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55c6d38d5150 0x55c6d38d7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:18.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.891+0000 7fd61dffb640 10 client.18138 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T14:02:18.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.891+0000 7fd63659c640 1 --2- 172.21.15.52:0/1291387182 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55c6d38d5150 0x55c6d38d7570 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8af83ce7b2ee6b82 server_cookie=2cdf2beb2bfca441 in_seq=0 out_seq=0 2024-04-02T14:02:18.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.891+0000 7fd61dffb640 1 -- 172.21.15.52:0/1291387182 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6d37bc040 con 0x55c6d38d5150 2024-04-02T14:02:18.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.891+0000 7fd61dffb640 10 client.18138 handle_command_reply: tid=0 2024-04-02T14:02:18.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.891+0000 7fd63f5ef740 1 client.18138 shutdown 2024-04-02T14:02:18.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.891+0000 7fd63f5ef740 1 -- 172.21.15.52:0/1291387182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 msgr2=0x55c6d38578c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.891+0000 7fd63f5ef740 1 --2- 172.21.15.52:0/1291387182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 0x55c6d38578c0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fd620000c00 tx=0x7fd620012820 comp rx=0 tx=0).stop 2024-04-02T14:02:18.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.891+0000 7fd63f5ef740 1 -- 172.21.15.52:0/1291387182 shutdown_connections 2024-04-02T14:02:18.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.891+0000 7fd63f5ef740 1 --2- 172.21.15.52:0/1291387182 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55c6d38d5150 0x55c6d38d7570 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.891+0000 7fd63f5ef740 1 --2- 172.21.15.52:0/1291387182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6d385e0b0 0x55c6d3857e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.891+0000 7fd61dffb640 0 client.18138 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T14:02:18.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.891+0000 7fd63f5ef740 1 --2- 172.21.15.52:0/1291387182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c6d3850df0 0x55c6d38578c0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.891+0000 7fd63f5ef740 1 --2- 172.21.15.52:0/1291387182 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6d38504d0 0x55c6d3866590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.891+0000 7fd63f5ef740 1 -- 172.21.15.52:0/1291387182 >> 172.21.15.52:0/1291387182 conn(0x55c6d384aed0 msgr2=0x55c6d3849f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.891+0000 7fd63f5ef740 1 -- 172.21.15.52:0/1291387182 shutdown_connections 2024-04-02T14:02:18.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.891+0000 7fd63f5ef740 1 -- 172.21.15.52:0/1291387182 wait complete. 2024-04-02T14:02:18.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.892+0000 7fd63f5ef740 20 client.18138 trim_cache size 0 max 16384 2024-04-02T14:02:18.907 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:02:18.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.892+0000 7fd63c92b640 1 -- 172.21.15.52:0/3344525826 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd6240a3d20 msgr2=0x7fd6240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.892+0000 7fd63c92b640 1 --2- 172.21.15.52:0/3344525826 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd6240a3d20 0x7fd6240a61e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd630009430 tx=0x7fd63002e040 comp rx=0 tx=0).stop 2024-04-02T14:02:18.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.892+0000 7fd63c92b640 1 -- 172.21.15.52:0/3344525826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd638102d90 msgr2=0x7fd6381983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:18.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.892+0000 7fd63c92b640 1 --2- 172.21.15.52:0/3344525826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd638102d90 0x7fd6381983e0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fd62800c890 tx=0x7fd62800cd50 comp rx=0 tx=0).stop 2024-04-02T14:02:18.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.892+0000 7fd63c92b640 1 -- 172.21.15.52:0/3344525826 shutdown_connections 2024-04-02T14:02:18.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.892+0000 7fd63c92b640 1 --2- 172.21.15.52:0/3344525826 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd6240a3d20 0x7fd6240a61e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.892+0000 7fd63c92b640 1 --2- 172.21.15.52:0/3344525826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd63810c160 0x7fd63819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.892+0000 7fd63c92b640 1 --2- 172.21.15.52:0/3344525826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6381036b0 0x7fd638198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.892+0000 7fd63c92b640 1 --2- 172.21.15.52:0/3344525826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd638102d90 0x7fd6381983e0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:18.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.892+0000 7fd63c92b640 1 -- 172.21.15.52:0/3344525826 >> 172.21.15.52:0/3344525826 conn(0x7fd638077b60 msgr2=0x7fd6380fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:18.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.892+0000 7fd63c92b640 1 -- 172.21.15.52:0/3344525826 shutdown_connections 2024-04-02T14:02:18.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:18.892+0000 7fd63c92b640 1 -- 172.21.15.52:0/3344525826 wait complete. 2024-04-02T14:02:18.910 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:18 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3450540165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:02:18.911 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T14:02:18.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:18 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3450540165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:02:19.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.011+0000 7fa0e0b11640 1 -- 172.21.15.52:0/3501647542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0dc100970 msgr2=0x7fa0dc100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:19.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.011+0000 7fa0e0b11640 1 --2- 172.21.15.52:0/3501647542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0dc100970 0x7fa0dc100df0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4009930 tx=0x7fa0c402bc30 comp rx=0 tx=0).stop 2024-04-02T14:02:19.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.012+0000 7fa0e0b11640 1 -- 172.21.15.52:0/3501647542 shutdown_connections 2024-04-02T14:02:19.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.012+0000 7fa0e0b11640 1 --2- 172.21.15.52:0/3501647542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0dc101330 0x7fa0dc10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.012+0000 7fa0e0b11640 1 --2- 172.21.15.52:0/3501647542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0dc100970 0x7fa0dc100df0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.012+0000 7fa0e0b11640 1 --2- 172.21.15.52:0/3501647542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0dc106980 0x7fa0dc106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.012+0000 7fa0e0b11640 1 -- 172.21.15.52:0/3501647542 >> 172.21.15.52:0/3501647542 conn(0x7fa0dc077b00 msgr2=0x7fa0dc077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:19.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.012+0000 7fa0e0b11640 1 -- 172.21.15.52:0/3501647542 shutdown_connections 2024-04-02T14:02:19.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.012+0000 7fa0e0b11640 1 -- 172.21.15.52:0/3501647542 wait complete. 2024-04-02T14:02:19.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.012+0000 7fa0e0b11640 1 Processor -- start 2024-04-02T14:02:19.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.012+0000 7fa0e0b11640 1 -- start start 2024-04-02T14:02:19.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.013+0000 7fa0e0b11640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0dc100970 0x7fa0dc0fd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:19.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.013+0000 7fa0e0b11640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0dc101330 0x7fa0dc0fdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:19.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.013+0000 7fa0e0b11640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0dc106980 0x7fa0dc0fe420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:19.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.013+0000 7fa0e0b11640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc110d90 con 0x7fa0dc106980 2024-04-02T14:02:19.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.013+0000 7fa0e0b11640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc110c10 con 0x7fa0dc100970 2024-04-02T14:02:19.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.013+0000 7fa0e0b11640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc110f10 con 0x7fa0dc101330 2024-04-02T14:02:19.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.013+0000 7fa0dad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0dc106980 0x7fa0dc0fe420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:19.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.013+0000 7fa0dad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0dc106980 0x7fa0dc0fe420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53438/0 (socket says 172.21.15.52:53438) 2024-04-02T14:02:19.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.013+0000 7fa0dad9d640 1 -- 172.21.15.52:0/4074031795 learned_addr learned my addr 172.21.15.52:0/4074031795 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:19.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.013+0000 7fa0d3fff640 1 --2- 172.21.15.52:0/4074031795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0dc101330 0x7fa0dc0fdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:19.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.013+0000 7fa0da59c640 1 --2- 172.21.15.52:0/4074031795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0dc100970 0x7fa0dc0fd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:19.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.013+0000 7fa0dad9d640 1 -- 172.21.15.52:0/4074031795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0dc101330 msgr2=0x7fa0dc0fdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:19.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.013+0000 7fa0dad9d640 1 --2- 172.21.15.52:0/4074031795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0dc101330 0x7fa0dc0fdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.013+0000 7fa0dad9d640 1 -- 172.21.15.52:0/4074031795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0dc100970 msgr2=0x7fa0dc0fd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:19.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.013+0000 7fa0dad9d640 1 --2- 172.21.15.52:0/4074031795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0dc100970 0x7fa0dc0fd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.013+0000 7fa0dad9d640 1 -- 172.21.15.52:0/4074031795 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0dc1aad80 con 0x7fa0dc106980 2024-04-02T14:02:19.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.013+0000 7fa0dad9d640 1 --2- 172.21.15.52:0/4074031795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0dc106980 0x7fa0dc0fe420 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400cad0 tx=0x7fa0d400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:19.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.013+0000 7fa0d37fe640 1 -- 172.21.15.52:0/4074031795 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d400ae20 con 0x7fa0dc106980 2024-04-02T14:02:19.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.013+0000 7fa0e0b11640 1 -- 172.21.15.52:0/4074031795 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0dc1aafb0 con 0x7fa0dc106980 2024-04-02T14:02:19.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.013+0000 7fa0d37fe640 1 -- 172.21.15.52:0/4074031795 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0d4005020 con 0x7fa0dc106980 2024-04-02T14:02:19.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.013+0000 7fa0d37fe640 1 -- 172.21.15.52:0/4074031795 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d400f6f0 con 0x7fa0dc106980 2024-04-02T14:02:19.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.013+0000 7fa0e0b11640 1 -- 172.21.15.52:0/4074031795 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0dc1ab480 con 0x7fa0dc106980 2024-04-02T14:02:19.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.016+0000 7fa0d37fe640 1 -- 172.21.15.52:0/4074031795 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa0d40051d0 con 0x7fa0dc106980 2024-04-02T14:02:19.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.016+0000 7fa0e37ce740 1 Processor -- start 2024-04-02T14:02:19.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.016+0000 7fa0e37ce740 1 -- start start 2024-04-02T14:02:19.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.016+0000 7fa0d37fe640 1 --2- 172.21.15.52:0/4074031795 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa0c80a3d20 0x7fa0c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:19.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.016+0000 7fa0e37ce740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556eb999b7a0 0x556eb999bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:19.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.016+0000 7fa0e37ce740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb999c0c0 0x556eb99a4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:19.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.016+0000 7fa0e37ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eb99a4ca0 0x556eb99a7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:19.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.016+0000 7fa0e37ce740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556eb98ff7b0 con 0x556eb999c0c0 2024-04-02T14:02:19.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.016+0000 7fa0e37ce740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556eb989e570 con 0x556eb999b7a0 2024-04-02T14:02:19.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.016+0000 7fa0e37ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556eb98fb760 con 0x556eb99a4ca0 2024-04-02T14:02:19.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.017+0000 7fa0d3fff640 1 --2- 172.21.15.52:0/4074031795 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa0c80a3d20 0x7fa0c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:19.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.017+0000 7fa0d37fe640 1 -- 172.21.15.52:0/4074031795 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9526+0+0 (secure 0 0 0) 0x7fa0d40d0cc0 con 0x7fa0dc106980 2024-04-02T14:02:19.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.017+0000 7fa0d3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb999c0c0 0x556eb99a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:19.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.017+0000 7fa0d3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb999c0c0 0x556eb99a4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53454/0 (socket says 172.21.15.52:53454) 2024-04-02T14:02:19.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.017+0000 7fa0d3fff640 1 -- 172.21.15.52:0/2514860266 learned_addr learned my addr 172.21.15.52:0/2514860266 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:19.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.017+0000 7fa0dad9d640 1 --2- 172.21.15.52:0/2514860266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eb99a4ca0 0x556eb99a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:19.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.017+0000 7fa0da59c640 1 --2- 172.21.15.52:0/2514860266 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556eb999b7a0 0x556eb999bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:19.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.017+0000 7fa0dad9d640 1 -- 172.21.15.52:0/2514860266 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556eb999b7a0 msgr2=0x556eb999bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:19.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.017+0000 7fa0dad9d640 1 --2- 172.21.15.52:0/2514860266 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556eb999b7a0 0x556eb999bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.017+0000 7fa0dad9d640 1 -- 172.21.15.52:0/2514860266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb999c0c0 msgr2=0x556eb99a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:19.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.017+0000 7fa0dad9d640 1 --2- 172.21.15.52:0/2514860266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb999c0c0 0x556eb99a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.017+0000 7fa0dad9d640 1 -- 172.21.15.52:0/2514860266 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eb96133f0 con 0x556eb99a4ca0 2024-04-02T14:02:19.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.017+0000 7fa0da59c640 1 --2- 172.21.15.52:0/2514860266 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556eb999b7a0 0x556eb999bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:02:19.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.017+0000 7fa0d3fff640 1 --2- 172.21.15.52:0/2514860266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb999c0c0 0x556eb99a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:02:19.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.017+0000 7fa0d3fff640 1 --2- 172.21.15.52:0/4074031795 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa0c80a3d20 0x7fa0c80a61e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4004ee0 tx=0x7fa0c402c660 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:19.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.017+0000 7fa0dad9d640 1 --2- 172.21.15.52:0/2514860266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eb99a4ca0 0x556eb99a7090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40ce020 tx=0x7fa0d409cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c32ac9314f5bbb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:19.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.017+0000 7fa0d0ff9640 1 -- 172.21.15.52:0/2514860266 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d401d040 con 0x556eb99a4ca0 2024-04-02T14:02:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.018+0000 7fa0d0ff9640 1 -- 172.21.15.52:0/2514860266 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0d40a54e0 con 0x556eb99a4ca0 2024-04-02T14:02:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.018+0000 7fa0d0ff9640 1 -- 172.21.15.52:0/2514860266 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d409bcc0 con 0x556eb99a4ca0 2024-04-02T14:02:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.018+0000 7fa0e37ce740 1 -- 172.21.15.52:0/2514860266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eb99a4ca0 msgr2=0x556eb99a7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.018+0000 7fa0e37ce740 1 --2- 172.21.15.52:0/2514860266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eb99a4ca0 0x556eb99a7090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40ce020 tx=0x7fa0d409cf70 comp rx=0 tx=0).stop 2024-04-02T14:02:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.018+0000 7fa0e37ce740 1 -- 172.21.15.52:0/2514860266 shutdown_connections 2024-04-02T14:02:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.018+0000 7fa0e37ce740 1 --2- 172.21.15.52:0/2514860266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eb99a4ca0 0x556eb99a7090 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.018+0000 7fa0e37ce740 1 --2- 172.21.15.52:0/2514860266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb999c0c0 0x556eb99a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.018+0000 7fa0e37ce740 1 --2- 172.21.15.52:0/2514860266 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556eb999b7a0 0x556eb999bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.018+0000 7fa0e37ce740 1 -- 172.21.15.52:0/2514860266 >> 172.21.15.52:0/2514860266 conn(0x556eb9995ed0 msgr2=0x556eb99962b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:19.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.018+0000 7fa0e37ce740 1 -- 172.21.15.52:0/2514860266 shutdown_connections 2024-04-02T14:02:19.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.018+0000 7fa0e37ce740 1 -- 172.21.15.52:0/2514860266 wait complete. 2024-04-02T14:02:19.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.019+0000 7fa0e37ce740 1 -- start start 2024-04-02T14:02:19.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.019+0000 7fa0e37ce740 1 Processor -- start 2024-04-02T14:02:19.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.019+0000 7fa0e37ce740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556eb999b7a0 0x556eb99a4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:19.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.019+0000 7fa0e37ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eb999c0c0 0x556eb99a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:19.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.019+0000 7fa0e37ce740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb99a4ca0 0x556eb99a2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:19.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.019+0000 7fa0e37ce740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556eb99a7a90 con 0x556eb99a4ca0 2024-04-02T14:02:19.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.019+0000 7fa0e37ce740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556eb98bae80 con 0x556eb999b7a0 2024-04-02T14:02:19.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.019+0000 7fa0e37ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556eb99a7c10 con 0x556eb999c0c0 2024-04-02T14:02:19.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.019+0000 7fa0e37ce740 10 client.0 _ll_register_callbacks cb 0x556eb98f6aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:19.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.019+0000 7fa0dad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb99a4ca0 0x556eb99a2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:19.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.019+0000 7fa0dad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb99a4ca0 0x556eb99a2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53466/0 (socket says 172.21.15.52:53466) 2024-04-02T14:02:19.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.019+0000 7fa0dad9d640 1 -- 172.21.15.52:0/118554674 learned_addr learned my addr 172.21.15.52:0/118554674 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:19.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.019+0000 7fa0d3fff640 1 --2- 172.21.15.52:0/118554674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eb999c0c0 0x556eb99a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:19.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.019+0000 7fa0da59c640 1 --2- 172.21.15.52:0/118554674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556eb999b7a0 0x556eb99a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:19.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.019+0000 7fa0dad9d640 1 -- 172.21.15.52:0/118554674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eb999c0c0 msgr2=0x556eb99a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:19.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.019+0000 7fa0dad9d640 1 --2- 172.21.15.52:0/118554674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eb999c0c0 0x556eb99a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.019+0000 7fa0dad9d640 1 -- 172.21.15.52:0/118554674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556eb999b7a0 msgr2=0x556eb99a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:19.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.019+0000 7fa0dad9d640 1 --2- 172.21.15.52:0/118554674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556eb999b7a0 0x556eb99a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.019+0000 7fa0dad9d640 1 -- 172.21.15.52:0/118554674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eb96dd7d0 con 0x556eb99a4ca0 2024-04-02T14:02:19.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.020+0000 7fa0dad9d640 1 --2- 172.21.15.52:0/118554674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb99a4ca0 0x556eb99a2230 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40ce0d0 tx=0x7fa0d40ce480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03173ebbca0cd16 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:19.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.020+0000 7fa0c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:19.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.020+0000 7fa0c1ffb640 1 -- 172.21.15.52:0/118554674 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d401d070 con 0x556eb99a4ca0 2024-04-02T14:02:19.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.020+0000 7fa0c1ffb640 1 -- 172.21.15.52:0/118554674 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9526+0+0 (secure 0 0 0) 0x7fa0d409dba0 con 0x556eb99a4ca0 2024-04-02T14:02:19.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.020+0000 7fa0e37ce740 1 -- 172.21.15.52:0/118554674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0d40a54e0 con 0x556eb99a4ca0 2024-04-02T14:02:19.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.020+0000 7fa0c1ffb640 1 -- 172.21.15.52:0/118554674 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa0d40800d0 con 0x556eb99a4ca0 2024-04-02T14:02:19.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.020+0000 7fa0e37ce740 10 client.18150 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:19.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.020+0000 7fa0e37ce740 1 -- 172.21.15.52:0/118554674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556eb99b3b40 con 0x556eb99a4ca0 2024-04-02T14:02:19.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.020+0000 7fa0c1ffb640 1 -- 172.21.15.52:0/118554674 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0d40a54e0 con 0x556eb99a4ca0 2024-04-02T14:02:19.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.020+0000 7fa0c1ffb640 1 -- 172.21.15.52:0/118554674 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa0d409b730 con 0x556eb99a4ca0 2024-04-02T14:02:19.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.020+0000 7fa0e37ce740 10 client.18150 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:19.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.020+0000 7fa0e37ce740 10 client.18150 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:02:19.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.020+0000 7fa0e37ce740 20 client.18150 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:19.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.020+0000 7fa0e37ce740 1 --2- 172.21.15.52:0/118554674 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556eb99b4e90 0x556eb99b7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:19.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.020+0000 7fa0e37ce740 4 client.18150 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:02:19.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.020+0000 7fa0e37ce740 1 -- 172.21.15.52:0/118554674 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556eb990ccf0 con 0x556eb99b4e90 2024-04-02T14:02:19.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.021+0000 7fa0da59c640 1 --2- 172.21.15.52:0/118554674 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556eb99b4e90 0x556eb99b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:19.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.021+0000 7fa0da59c640 1 --2- 172.21.15.52:0/118554674 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556eb99b4e90 0x556eb99b7280 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cb56f3b045fb3ee server_cookie=957e945efc85e559 in_seq=0 out_seq=0 2024-04-02T14:02:19.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.021+0000 7fa0c1ffb640 10 client.18150 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:19.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.022+0000 7fa0c1ffb640 1 -- 172.21.15.52:0/118554674 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556eb990ccf0 con 0x556eb99b4e90 2024-04-02T14:02:19.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.022+0000 7fa0c1ffb640 10 client.18150 handle_command_reply: tid=0 2024-04-02T14:02:19.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.022+0000 7fa0e37ce740 1 client.18150 shutdown 2024-04-02T14:02:19.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.022+0000 7fa0e37ce740 1 -- 172.21.15.52:0/118554674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb99a4ca0 msgr2=0x556eb99a2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:19.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.022+0000 7fa0e37ce740 1 --2- 172.21.15.52:0/118554674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb99a4ca0 0x556eb99a2230 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40ce0d0 tx=0x7fa0d40ce480 comp rx=0 tx=0).stop 2024-04-02T14:02:19.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.022+0000 7fa0e37ce740 1 -- 172.21.15.52:0/118554674 shutdown_connections 2024-04-02T14:02:19.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.022+0000 7fa0e37ce740 1 --2- 172.21.15.52:0/118554674 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556eb99b4e90 0x556eb99b7280 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.022+0000 7fa0e37ce740 1 --2- 172.21.15.52:0/118554674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb99a4ca0 0x556eb99a2230 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.022+0000 7fa0c1ffb640 0 client.18150 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:19.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.022+0000 7fa0e37ce740 1 --2- 172.21.15.52:0/118554674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eb999c0c0 0x556eb99a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.022+0000 7fa0e37ce740 1 --2- 172.21.15.52:0/118554674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556eb999b7a0 0x556eb99a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.022+0000 7fa0e37ce740 1 -- 172.21.15.52:0/118554674 >> 172.21.15.52:0/118554674 conn(0x556eb9995ed0 msgr2=0x556eb9997c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:19.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.022+0000 7fa0e37ce740 1 -- 172.21.15.52:0/118554674 shutdown_connections 2024-04-02T14:02:19.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.022+0000 7fa0e37ce740 1 -- 172.21.15.52:0/118554674 wait complete. 2024-04-02T14:02:19.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.022+0000 7fa0e37ce740 20 client.18150 trim_cache size 0 max 16384 2024-04-02T14:02:19.043 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:18 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3450540165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:02:19.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.035+0000 7fa0e37ce740 1 Processor -- start 2024-04-02T14:02:19.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.035+0000 7fa0e37ce740 1 -- start start 2024-04-02T14:02:19.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.035+0000 7fa0e37ce740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb999b7a0 0x556eb99a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:19.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.035+0000 7fa0e37ce740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556eb999c0c0 0x556eb99a4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:19.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.035+0000 7fa0e37ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eb99a4ca0 0x556eb99a4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:19.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.035+0000 7fa0e37ce740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1b5660 con 0x556eb999b7a0 2024-04-02T14:02:19.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.035+0000 7fa0e37ce740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1b71a0 con 0x556eb999c0c0 2024-04-02T14:02:19.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.035+0000 7fa0e37ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1b5d30 con 0x556eb99a4ca0 2024-04-02T14:02:19.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.035+0000 7fa0da59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb999b7a0 0x556eb99a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:19.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.036+0000 7fa0da59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb999b7a0 0x556eb99a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53478/0 (socket says 172.21.15.52:53478) 2024-04-02T14:02:19.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.036+0000 7fa0da59c640 1 -- 172.21.15.52:0/2435861686 learned_addr learned my addr 172.21.15.52:0/2435861686 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:19.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.036+0000 7fa0dad9d640 1 --2- 172.21.15.52:0/2435861686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eb99a4ca0 0x556eb99a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:19.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.036+0000 7fa0d3fff640 1 --2- 172.21.15.52:0/2435861686 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556eb999c0c0 0x556eb99a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:19.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.036+0000 7fa0da59c640 1 -- 172.21.15.52:0/2435861686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eb99a4ca0 msgr2=0x556eb99a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:19.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.036+0000 7fa0da59c640 1 --2- 172.21.15.52:0/2435861686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eb99a4ca0 0x556eb99a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.036+0000 7fa0da59c640 1 -- 172.21.15.52:0/2435861686 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556eb999c0c0 msgr2=0x556eb99a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:19.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.036+0000 7fa0da59c640 1 --2- 172.21.15.52:0/2435861686 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556eb999c0c0 0x556eb99a4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.036+0000 7fa0da59c640 1 -- 172.21.15.52:0/2435861686 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eb99a0040 con 0x556eb999b7a0 2024-04-02T14:02:19.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.036+0000 7fa0da59c640 1 --2- 172.21.15.52:0/2435861686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb999b7a0 0x556eb99a09a0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x556eb99a7cb0 tx=0x7fa0cc00b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec05b281241dd78 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:19.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.036+0000 7fa0c3fff640 1 -- 172.21.15.52:0/2435861686 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc01d430 con 0x556eb999b7a0 2024-04-02T14:02:19.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.036+0000 7fa0c3fff640 1 -- 172.21.15.52:0/2435861686 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0cc005020 con 0x556eb999b7a0 2024-04-02T14:02:19.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.036+0000 7fa0c3fff640 1 -- 172.21.15.52:0/2435861686 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc004b20 con 0x556eb999b7a0 2024-04-02T14:02:19.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.037+0000 7fa0e37ce740 1 -- 172.21.15.52:0/2435861686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb999b7a0 msgr2=0x556eb99a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:19.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.037+0000 7fa0e37ce740 1 --2- 172.21.15.52:0/2435861686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb999b7a0 0x556eb99a09a0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x556eb99a7cb0 tx=0x7fa0cc00b460 comp rx=0 tx=0).stop 2024-04-02T14:02:19.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.037+0000 7fa0e37ce740 1 -- 172.21.15.52:0/2435861686 shutdown_connections 2024-04-02T14:02:19.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.037+0000 7fa0e37ce740 1 --2- 172.21.15.52:0/2435861686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eb99a4ca0 0x556eb99a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.037+0000 7fa0e37ce740 1 --2- 172.21.15.52:0/2435861686 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556eb999c0c0 0x556eb99a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.037+0000 7fa0e37ce740 1 --2- 172.21.15.52:0/2435861686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb999b7a0 0x556eb99a09a0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.037+0000 7fa0e37ce740 1 -- 172.21.15.52:0/2435861686 >> 172.21.15.52:0/2435861686 conn(0x556eb9995ed0 msgr2=0x556eb99a0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:19.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.037+0000 7fa0e37ce740 1 -- 172.21.15.52:0/2435861686 shutdown_connections 2024-04-02T14:02:19.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.037+0000 7fa0e37ce740 1 -- 172.21.15.52:0/2435861686 wait complete. 2024-04-02T14:02:19.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.037+0000 7fa0e37ce740 1 -- start start 2024-04-02T14:02:19.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.037+0000 7fa0e37ce740 1 Processor -- start 2024-04-02T14:02:19.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.037+0000 7fa0e37ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eb999b7a0 0x556eb99ba220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:19.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.037+0000 7fa0e37ce740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556eb999c0c0 0x556eb99ba760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:19.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.037+0000 7fa0e37ce740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb99a4ca0 0x556eb99b5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:19.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.037+0000 7fa0e37ce740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1b6ad0 con 0x556eb99a4ca0 2024-04-02T14:02:19.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.037+0000 7fa0e37ce740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1b6400 con 0x556eb999c0c0 2024-04-02T14:02:19.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.037+0000 7fa0e37ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1b3660 con 0x556eb999b7a0 2024-04-02T14:02:19.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.037+0000 7fa0e37ce740 10 client.0 _ll_register_callbacks cb 0x7fa0c402cac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:19.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.038+0000 7fa0dad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb99a4ca0 0x556eb99b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:19.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.038+0000 7fa0dad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb99a4ca0 0x556eb99b5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53484/0 (socket says 172.21.15.52:53484) 2024-04-02T14:02:19.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.038+0000 7fa0dad9d640 1 -- 172.21.15.52:0/3256688243 learned_addr learned my addr 172.21.15.52:0/3256688243 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:19.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.038+0000 7fa0d3fff640 1 --2- 172.21.15.52:0/3256688243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556eb999c0c0 0x556eb99ba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:19.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.038+0000 7fa0da59c640 1 --2- 172.21.15.52:0/3256688243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eb999b7a0 0x556eb99ba220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:19.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.038+0000 7fa0dad9d640 1 -- 172.21.15.52:0/3256688243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eb999b7a0 msgr2=0x556eb99ba220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:19.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.038+0000 7fa0dad9d640 1 --2- 172.21.15.52:0/3256688243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eb999b7a0 0x556eb99ba220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.038+0000 7fa0dad9d640 1 -- 172.21.15.52:0/3256688243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556eb999c0c0 msgr2=0x556eb99ba760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:19.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.038+0000 7fa0dad9d640 1 --2- 172.21.15.52:0/3256688243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556eb999c0c0 0x556eb99ba760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.038+0000 7fa0dad9d640 1 -- 172.21.15.52:0/3256688243 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eb99b5f80 con 0x556eb99a4ca0 2024-04-02T14:02:19.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.038+0000 7fa0dad9d640 1 --2- 172.21.15.52:0/3256688243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb99a4ca0 0x556eb99b5750 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fa0d409ab20 tx=0x7fa0d409c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98272787848e2b48 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:19.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.038+0000 7fa0c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:19.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.038+0000 7fa0c1ffb640 1 -- 172.21.15.52:0/3256688243 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d401d040 con 0x556eb99a4ca0 2024-04-02T14:02:19.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.038+0000 7fa0c1ffb640 1 -- 172.21.15.52:0/3256688243 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9526+0+0 (secure 0 0 0) 0x7fa0d409dba0 con 0x556eb99a4ca0 2024-04-02T14:02:19.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.038+0000 7fa0e37ce740 1 -- 172.21.15.52:0/3256688243 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0cc005020 con 0x556eb99a4ca0 2024-04-02T14:02:19.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.038+0000 7fa0c1ffb640 1 -- 172.21.15.52:0/3256688243 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa0d40ce2a0 con 0x556eb99a4ca0 2024-04-02T14:02:19.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.038+0000 7fa0e37ce740 10 client.18156 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:19.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.038+0000 7fa0e37ce740 1 -- 172.21.15.52:0/3256688243 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556eb9a1ac20 con 0x556eb99a4ca0 2024-04-02T14:02:19.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.038+0000 7fa0c1ffb640 1 -- 172.21.15.52:0/3256688243 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0cc005020 con 0x556eb99a4ca0 2024-04-02T14:02:19.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.038+0000 7fa0c1ffb640 1 -- 172.21.15.52:0/3256688243 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa0d409bcc0 con 0x556eb99a4ca0 2024-04-02T14:02:19.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.039+0000 7fa0e37ce740 10 client.18156 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:19.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.039+0000 7fa0e37ce740 10 client.18156 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:02:19.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.039+0000 7fa0e37ce740 20 client.18156 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:19.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.039+0000 7fa0e37ce740 1 --2- 172.21.15.52:0/3256688243 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556eb9a1b1a0 0x556eb9a1d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:19.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.039+0000 7fa0e37ce740 4 client.18156 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:02:19.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.039+0000 7fa0e37ce740 1 -- 172.21.15.52:0/3256688243 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556eb9907040 con 0x556eb9a1b1a0 2024-04-02T14:02:19.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.039+0000 7fa0da59c640 1 --2- 172.21.15.52:0/3256688243 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556eb9a1b1a0 0x556eb9a1d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:19.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.039+0000 7fa0c1ffb640 10 client.18156 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:19.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.039+0000 7fa0da59c640 1 --2- 172.21.15.52:0/3256688243 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556eb9a1b1a0 0x556eb9a1d590 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6acb5334ea5c23d server_cookie=f75ce9998c68e2bd in_seq=0 out_seq=0 2024-04-02T14:02:19.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.039+0000 7fa0c1ffb640 1 -- 172.21.15.52:0/3256688243 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556eb9907040 con 0x556eb9a1b1a0 2024-04-02T14:02:19.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.039+0000 7fa0c1ffb640 10 client.18156 handle_command_reply: tid=0 2024-04-02T14:02:19.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.039+0000 7fa0e37ce740 1 client.18156 shutdown 2024-04-02T14:02:19.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.040+0000 7fa0e37ce740 1 -- 172.21.15.52:0/3256688243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb99a4ca0 msgr2=0x556eb99b5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:19.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.040+0000 7fa0e37ce740 1 --2- 172.21.15.52:0/3256688243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb99a4ca0 0x556eb99b5750 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fa0d409ab20 tx=0x7fa0d409c4d0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.040+0000 7fa0e37ce740 1 -- 172.21.15.52:0/3256688243 shutdown_connections 2024-04-02T14:02:19.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.040+0000 7fa0e37ce740 1 --2- 172.21.15.52:0/3256688243 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x556eb9a1b1a0 0x556eb9a1d590 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.040+0000 7fa0e37ce740 1 --2- 172.21.15.52:0/3256688243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556eb99a4ca0 0x556eb99b5750 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.040+0000 7fa0c1ffb640 0 client.18156 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:19.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.040+0000 7fa0e37ce740 1 --2- 172.21.15.52:0/3256688243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556eb999c0c0 0x556eb99ba760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.040+0000 7fa0e37ce740 1 --2- 172.21.15.52:0/3256688243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eb999b7a0 0x556eb99ba220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.040+0000 7fa0e37ce740 1 -- 172.21.15.52:0/3256688243 >> 172.21.15.52:0/3256688243 conn(0x556eb9995ed0 msgr2=0x556eb99b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:19.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.040+0000 7fa0e37ce740 1 -- 172.21.15.52:0/3256688243 shutdown_connections 2024-04-02T14:02:19.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.040+0000 7fa0e37ce740 1 -- 172.21.15.52:0/3256688243 wait complete. 2024-04-02T14:02:19.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.040+0000 7fa0e37ce740 20 client.18156 trim_cache size 0 max 16384 2024-04-02T14:02:19.061 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:02:19.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.040+0000 7fa0e0b11640 1 -- 172.21.15.52:0/4074031795 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa0c80a3d20 msgr2=0x7fa0c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:19.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.040+0000 7fa0e0b11640 1 --2- 172.21.15.52:0/4074031795 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa0c80a3d20 0x7fa0c80a61e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4004ee0 tx=0x7fa0c402c660 comp rx=0 tx=0).stop 2024-04-02T14:02:19.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.040+0000 7fa0e0b11640 1 -- 172.21.15.52:0/4074031795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0dc106980 msgr2=0x7fa0dc0fe420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:19.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.040+0000 7fa0e0b11640 1 --2- 172.21.15.52:0/4074031795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0dc106980 0x7fa0dc0fe420 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400cad0 tx=0x7fa0d400cf90 comp rx=0 tx=0).stop 2024-04-02T14:02:19.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.040+0000 7fa0e0b11640 1 -- 172.21.15.52:0/4074031795 shutdown_connections 2024-04-02T14:02:19.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.040+0000 7fa0e0b11640 1 --2- 172.21.15.52:0/4074031795 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa0c80a3d20 0x7fa0c80a61e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.040+0000 7fa0e0b11640 1 --2- 172.21.15.52:0/4074031795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0dc106980 0x7fa0dc0fe420 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.040+0000 7fa0e0b11640 1 --2- 172.21.15.52:0/4074031795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0dc101330 0x7fa0dc0fdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.040+0000 7fa0e0b11640 1 --2- 172.21.15.52:0/4074031795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0dc100970 0x7fa0dc0fd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:19.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.040+0000 7fa0e0b11640 1 -- 172.21.15.52:0/4074031795 >> 172.21.15.52:0/4074031795 conn(0x7fa0dc077b00 msgr2=0x7fa0dc0ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:19.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.040+0000 7fa0e0b11640 1 -- 172.21.15.52:0/4074031795 shutdown_connections 2024-04-02T14:02:19.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:19.040+0000 7fa0e0b11640 1 -- 172.21.15.52:0/4074031795 wait complete. 2024-04-02T14:02:19.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:19 smithi088 ceph-mon[36528]: pgmap v272: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:02:20.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:19 smithi189 ceph-mon[36683]: pgmap v272: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:02:20.065 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T14:02:20.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:02:20.215 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T14:02:20.216 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:19 smithi052 ceph-mon[32104]: pgmap v272: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:02:20.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.332+0000 7f529399d640 1 -- 172.21.15.52:0/3632344656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f528c107d00 msgr2=0x7f528c1080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:20.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.332+0000 7f529399d640 1 --2- 172.21.15.52:0/3632344656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f528c107d00 0x7f528c1080e0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f5288009930 tx=0x7f528802bcd0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.332+0000 7f529399d640 1 -- 172.21.15.52:0/3632344656 shutdown_connections 2024-04-02T14:02:20.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.332+0000 7f529399d640 1 --2- 172.21.15.52:0/3632344656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f528c100d80 0x7f528c10e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.332+0000 7f529399d640 1 --2- 172.21.15.52:0/3632344656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f528c1003c0 0x7f528c100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.332+0000 7f529399d640 1 --2- 172.21.15.52:0/3632344656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f528c107d00 0x7f528c1080e0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.332+0000 7f529399d640 1 -- 172.21.15.52:0/3632344656 >> 172.21.15.52:0/3632344656 conn(0x7f528c075d40 msgr2=0x7f528c076150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:20.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.332+0000 7f529399d640 1 -- 172.21.15.52:0/3632344656 shutdown_connections 2024-04-02T14:02:20.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.332+0000 7f529399d640 1 -- 172.21.15.52:0/3632344656 wait complete. 2024-04-02T14:02:20.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.333+0000 7f529399d640 1 Processor -- start 2024-04-02T14:02:20.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.333+0000 7f529399d640 1 -- start start 2024-04-02T14:02:20.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.333+0000 7f529399d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f528c1003c0 0x7f528c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.333+0000 7f529399d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f528c100d80 0x7f528c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.333+0000 7f529399d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f528c107d00 0x7f528c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.333+0000 7f529399d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f528c111f20 con 0x7f528c100d80 2024-04-02T14:02:20.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.333+0000 7f529399d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f528c111da0 con 0x7f528c1003c0 2024-04-02T14:02:20.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.333+0000 7f529399d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f528c1120a0 con 0x7f528c107d00 2024-04-02T14:02:20.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.333+0000 7f5290f38640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f528c100d80 0x7f528c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.333+0000 7f5290f38640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f528c100d80 0x7f528c1988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53508/0 (socket says 172.21.15.52:53508) 2024-04-02T14:02:20.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.333+0000 7f5290f38640 1 -- 172.21.15.52:0/4244727940 learned_addr learned my addr 172.21.15.52:0/4244727940 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:20.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.333+0000 7f5291739640 1 --2- 172.21.15.52:0/4244727940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f528c1003c0 0x7f528c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.333+0000 7f5291f3a640 1 --2- 172.21.15.52:0/4244727940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f528c107d00 0x7f528c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.333+0000 7f5290f38640 1 -- 172.21.15.52:0/4244727940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f528c107d00 msgr2=0x7f528c19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:20.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.333+0000 7f5290f38640 1 --2- 172.21.15.52:0/4244727940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f528c107d00 0x7f528c19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.333+0000 7f5290f38640 1 -- 172.21.15.52:0/4244727940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f528c1003c0 msgr2=0x7f528c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:20.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.333+0000 7f5290f38640 1 --2- 172.21.15.52:0/4244727940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f528c1003c0 0x7f528c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.333+0000 7f5290f38640 1 -- 172.21.15.52:0/4244727940 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f528c19d350 con 0x7f528c100d80 2024-04-02T14:02:20.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.334+0000 7f5290f38640 1 --2- 172.21.15.52:0/4244727940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f528c100d80 0x7f528c1988a0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f527c00c890 tx=0x7f527c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:20.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.334+0000 7f52827fc640 1 -- 172.21.15.52:0/4244727940 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c004220 con 0x7f528c100d80 2024-04-02T14:02:20.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.334+0000 7f52827fc640 1 -- 172.21.15.52:0/4244727940 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f527c0043c0 con 0x7f528c100d80 2024-04-02T14:02:20.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.334+0000 7f529399d640 1 -- 172.21.15.52:0/4244727940 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f528c19d640 con 0x7f528c100d80 2024-04-02T14:02:20.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.334+0000 7f52827fc640 1 -- 172.21.15.52:0/4244727940 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c0049e0 con 0x7f528c100d80 2024-04-02T14:02:20.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.334+0000 7f529399d640 1 -- 172.21.15.52:0/4244727940 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f528c1a4ed0 con 0x7f528c100d80 2024-04-02T14:02:20.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.336+0000 7f52827fc640 1 -- 172.21.15.52:0/4244727940 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f527c0079b0 con 0x7f528c100d80 2024-04-02T14:02:20.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.337+0000 7f5296589740 1 Processor -- start 2024-04-02T14:02:20.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.337+0000 7f5296589740 1 -- start start 2024-04-02T14:02:20.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.337+0000 7f5296589740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9eb2c0 0x55bc1a9eb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.337+0000 7f5296589740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bc1a9ebbe0 0x55bc1a9f4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.337+0000 7f5296589740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1a9f47c0 0x55bc1a9f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.337+0000 7f5296589740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55bc1a94f770 con 0x55bc1a9eb2c0 2024-04-02T14:02:20.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.337+0000 7f5296589740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55bc1a8f7960 con 0x55bc1a9ebbe0 2024-04-02T14:02:20.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.337+0000 7f5296589740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bc1a94b8c0 con 0x55bc1a9f47c0 2024-04-02T14:02:20.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.337+0000 7f52827fc640 1 --2- 172.21.15.52:0/4244727940 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f52780a3da0 0x7f52780a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.337+0000 7f5291f3a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9eb2c0 0x55bc1a9eb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.337+0000 7f5291f3a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9eb2c0 0x55bc1a9eb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53510/0 (socket says 172.21.15.52:53510) 2024-04-02T14:02:20.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.337+0000 7f5291f3a640 1 -- 172.21.15.52:0/4198592605 learned_addr learned my addr 172.21.15.52:0/4198592605 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:20.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.337+0000 7f5291739640 1 --2- 172.21.15.52:0/4198592605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bc1a9ebbe0 0x55bc1a9f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.337+0000 7f5291f3a640 1 --2- 172.21.15.52:0/4244727940 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f52780a3da0 0x7f52780a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.337+0000 7f52827fc640 1 -- 172.21.15.52:0/4244727940 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9526+0+0 (secure 0 0 0) 0x7f527c0d0cd0 con 0x7f528c100d80 2024-04-02T14:02:20.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.337+0000 7f5290f38640 1 --2- 172.21.15.52:0/4198592605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1a9f47c0 0x55bc1a9f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.337+0000 7f5291f3a640 1 -- 172.21.15.52:0/4198592605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1a9f47c0 msgr2=0x55bc1a9f6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:20.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.337+0000 7f5291f3a640 1 --2- 172.21.15.52:0/4198592605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1a9f47c0 0x55bc1a9f6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.337+0000 7f5291f3a640 1 -- 172.21.15.52:0/4198592605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bc1a9ebbe0 msgr2=0x55bc1a9f4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:20.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.337+0000 7f5291f3a640 1 --2- 172.21.15.52:0/4198592605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bc1a9ebbe0 0x55bc1a9f4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.338+0000 7f5291f3a640 1 -- 172.21.15.52:0/4198592605 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc1a72b7e0 con 0x55bc1a9eb2c0 2024-04-02T14:02:20.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.338+0000 7f5291f3a640 1 --2- 172.21.15.52:0/4198592605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9eb2c0 0x55bc1a9eb6a0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f5284012e00 tx=0x7f528400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41afe786007fe44 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:20.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.338+0000 7f5291f3a640 1 --2- 172.21.15.52:0/4244727940 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f52780a3da0 0x7f52780a6260 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f528400fc30 tx=0x7f528401a000 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:20.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.338+0000 7f52777fe640 1 -- 172.21.15.52:0/4198592605 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284020070 con 0x55bc1a9eb2c0 2024-04-02T14:02:20.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.338+0000 7f52777fe640 1 -- 172.21.15.52:0/4198592605 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f528400ab30 con 0x55bc1a9eb2c0 2024-04-02T14:02:20.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.338+0000 7f52777fe640 1 -- 172.21.15.52:0/4198592605 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528401ce60 con 0x55bc1a9eb2c0 2024-04-02T14:02:20.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.339+0000 7f5296589740 1 -- 172.21.15.52:0/4198592605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9eb2c0 msgr2=0x55bc1a9eb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:20.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.339+0000 7f5296589740 1 --2- 172.21.15.52:0/4198592605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9eb2c0 0x55bc1a9eb6a0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f5284012e00 tx=0x7f528400f730 comp rx=0 tx=0).stop 2024-04-02T14:02:20.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.339+0000 7f5296589740 1 -- 172.21.15.52:0/4198592605 shutdown_connections 2024-04-02T14:02:20.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.339+0000 7f5296589740 1 --2- 172.21.15.52:0/4198592605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1a9f47c0 0x55bc1a9f6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.339+0000 7f5296589740 1 --2- 172.21.15.52:0/4198592605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bc1a9ebbe0 0x55bc1a9f4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.339+0000 7f5296589740 1 --2- 172.21.15.52:0/4198592605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9eb2c0 0x55bc1a9eb6a0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.339+0000 7f5296589740 1 -- 172.21.15.52:0/4198592605 >> 172.21.15.52:0/4198592605 conn(0x55bc1a9e59f0 msgr2=0x55bc1a9e5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:20.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.339+0000 7f5296589740 1 -- 172.21.15.52:0/4198592605 shutdown_connections 2024-04-02T14:02:20.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.339+0000 7f5296589740 1 -- 172.21.15.52:0/4198592605 wait complete. 2024-04-02T14:02:20.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.339+0000 7f5296589740 1 -- start start 2024-04-02T14:02:20.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.339+0000 7f5296589740 1 Processor -- start 2024-04-02T14:02:20.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.340+0000 7f5296589740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bc1a9eb2c0 0x55bc1a9f17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.340+0000 7f5296589740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9ebbe0 0x55bc1a9f1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.340+0000 7f5296589740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1a9f47c0 0x55bc1a9f2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.340+0000 7f5296589740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55bc1a9f7550 con 0x55bc1a9ebbe0 2024-04-02T14:02:20.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.340+0000 7f5296589740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55bc1a90ad20 con 0x55bc1a9eb2c0 2024-04-02T14:02:20.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.340+0000 7f5296589740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bc1a9f76d0 con 0x55bc1a9f47c0 2024-04-02T14:02:20.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.340+0000 7f5296589740 10 client.0 _ll_register_callbacks cb 0x55bc1a8d7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:20.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.340+0000 7f5290f38640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9ebbe0 0x55bc1a9f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.340+0000 7f5290f38640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9ebbe0 0x55bc1a9f1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53518/0 (socket says 172.21.15.52:53518) 2024-04-02T14:02:20.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.340+0000 7f5290f38640 1 -- 172.21.15.52:0/2263327672 learned_addr learned my addr 172.21.15.52:0/2263327672 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:20.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.340+0000 7f5291f3a640 1 --2- 172.21.15.52:0/2263327672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1a9f47c0 0x55bc1a9f2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.340+0000 7f5291739640 1 --2- 172.21.15.52:0/2263327672 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bc1a9eb2c0 0x55bc1a9f17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.340+0000 7f5290f38640 1 -- 172.21.15.52:0/2263327672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1a9f47c0 msgr2=0x55bc1a9f2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:20.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.340+0000 7f5290f38640 1 --2- 172.21.15.52:0/2263327672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1a9f47c0 0x55bc1a9f2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.340+0000 7f5290f38640 1 -- 172.21.15.52:0/2263327672 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bc1a9eb2c0 msgr2=0x55bc1a9f17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:20.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.340+0000 7f5290f38640 1 --2- 172.21.15.52:0/2263327672 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bc1a9eb2c0 0x55bc1a9f17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.340+0000 7f5290f38640 1 -- 172.21.15.52:0/2263327672 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc1a78bf70 con 0x55bc1a9ebbe0 2024-04-02T14:02:20.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.340+0000 7f5290f38640 1 --2- 172.21.15.52:0/2263327672 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9ebbe0 0x55bc1a9f1cf0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f527c09c7b0 tx=0x7f527c09c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e809530358304803 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:20.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.340+0000 7f5274ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:20.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.340+0000 7f5274ff9640 1 -- 172.21.15.52:0/2263327672 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c01d040 con 0x55bc1a9ebbe0 2024-04-02T14:02:20.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.340+0000 7f5274ff9640 1 -- 172.21.15.52:0/2263327672 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9526+0+0 (secure 0 0 0) 0x7f527c0ce250 con 0x55bc1a9ebbe0 2024-04-02T14:02:20.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.340+0000 7f5296589740 1 -- 172.21.15.52:0/2263327672 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f528400ab30 con 0x55bc1a9ebbe0 2024-04-02T14:02:20.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.341+0000 7f5274ff9640 1 -- 172.21.15.52:0/2263327672 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f527c0ce450 con 0x55bc1a9ebbe0 2024-04-02T14:02:20.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.341+0000 7f5296589740 10 client.18168 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:20.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.341+0000 7f5296589740 1 -- 172.21.15.52:0/2263327672 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bc1a623170 con 0x55bc1a9ebbe0 2024-04-02T14:02:20.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.341+0000 7f5274ff9640 1 -- 172.21.15.52:0/2263327672 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f528400ab30 con 0x55bc1a9ebbe0 2024-04-02T14:02:20.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.341+0000 7f5274ff9640 1 -- 172.21.15.52:0/2263327672 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f527c09ca80 con 0x55bc1a9ebbe0 2024-04-02T14:02:20.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.341+0000 7f5296589740 10 client.18168 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:20.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.341+0000 7f5296589740 10 client.18168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:02:20.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.341+0000 7f5296589740 20 client.18168 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:20.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.341+0000 7f5296589740 1 --2- 172.21.15.52:0/2263327672 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55bc1aa04a10 0x55bc1aa06e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.341+0000 7f5296589740 4 client.18168 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:02:20.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.341+0000 7f5296589740 1 -- 172.21.15.52:0/2263327672 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc1a95ccb0 con 0x55bc1aa04a10 2024-04-02T14:02:20.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.341+0000 7f5291739640 1 --2- 172.21.15.52:0/2263327672 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55bc1aa04a10 0x55bc1aa06e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.342+0000 7f5274ff9640 10 client.18168 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:20.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.342+0000 7f5291739640 1 --2- 172.21.15.52:0/2263327672 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55bc1aa04a10 0x55bc1aa06e60 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acc4c776a2f60a29 server_cookie=364a7fdc50f7863a in_seq=0 out_seq=0 2024-04-02T14:02:20.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.342+0000 7f5274ff9640 1 -- 172.21.15.52:0/2263327672 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bc1a95ccb0 con 0x55bc1aa04a10 2024-04-02T14:02:20.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.342+0000 7f5274ff9640 10 client.18168 handle_command_reply: tid=0 2024-04-02T14:02:20.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.342+0000 7f5296589740 1 client.18168 shutdown 2024-04-02T14:02:20.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.343+0000 7f5296589740 1 -- 172.21.15.52:0/2263327672 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9ebbe0 msgr2=0x55bc1a9f1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:20.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.343+0000 7f5296589740 1 --2- 172.21.15.52:0/2263327672 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9ebbe0 0x55bc1a9f1cf0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f527c09c7b0 tx=0x7f527c09c240 comp rx=0 tx=0).stop 2024-04-02T14:02:20.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.343+0000 7f5296589740 1 -- 172.21.15.52:0/2263327672 shutdown_connections 2024-04-02T14:02:20.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.343+0000 7f5296589740 1 --2- 172.21.15.52:0/2263327672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1a9f47c0 0x55bc1a9f2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.343+0000 7f5296589740 1 --2- 172.21.15.52:0/2263327672 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55bc1aa04a10 0x55bc1aa06e60 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.343+0000 7f5296589740 1 --2- 172.21.15.52:0/2263327672 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9ebbe0 0x55bc1a9f1cf0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.343+0000 7f5296589740 1 --2- 172.21.15.52:0/2263327672 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bc1a9eb2c0 0x55bc1a9f17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.343+0000 7f5274ff9640 0 client.18168 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:20.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.343+0000 7f5296589740 1 -- 172.21.15.52:0/2263327672 >> 172.21.15.52:0/2263327672 conn(0x55bc1a9e59f0 msgr2=0x55bc1a9e7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:20.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.343+0000 7f5296589740 1 -- 172.21.15.52:0/2263327672 shutdown_connections 2024-04-02T14:02:20.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.343+0000 7f5296589740 1 -- 172.21.15.52:0/2263327672 wait complete. 2024-04-02T14:02:20.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.343+0000 7f5296589740 20 client.18168 trim_cache size 0 max 16384 2024-04-02T14:02:20.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.361+0000 7f5296589740 1 Processor -- start 2024-04-02T14:02:20.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.361+0000 7f5296589740 1 -- start start 2024-04-02T14:02:20.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.361+0000 7f5296589740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bc1a9eb2c0 0x55bc1a9f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.361+0000 7f5296589740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1a9ebbe0 0x55bc1aa00440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.361+0000 7f5296589740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9f47c0 0x55bc1aa0c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.361+0000 7f5296589740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f528c1aef20 con 0x55bc1a9f47c0 2024-04-02T14:02:20.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.361+0000 7f5296589740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f528c1b0a60 con 0x55bc1a9eb2c0 2024-04-02T14:02:20.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.361+0000 7f5296589740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f528c1af5f0 con 0x55bc1a9ebbe0 2024-04-02T14:02:20.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.361+0000 7f5291f3a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9f47c0 0x55bc1aa0c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.361+0000 7f5291f3a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9f47c0 0x55bc1aa0c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53528/0 (socket says 172.21.15.52:53528) 2024-04-02T14:02:20.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.361+0000 7f5291f3a640 1 -- 172.21.15.52:0/2841730412 learned_addr learned my addr 172.21.15.52:0/2841730412 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:20.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.361+0000 7f5291739640 1 --2- 172.21.15.52:0/2841730412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bc1a9eb2c0 0x55bc1a9f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.361+0000 7f5291f3a640 1 -- 172.21.15.52:0/2841730412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1a9ebbe0 msgr2=0x55bc1aa00440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:20.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.361+0000 7f5290f38640 1 --2- 172.21.15.52:0/2841730412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1a9ebbe0 0x55bc1aa00440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.361+0000 7f5291f3a640 1 --2- 172.21.15.52:0/2841730412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1a9ebbe0 0x55bc1aa00440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.361+0000 7f5291f3a640 1 -- 172.21.15.52:0/2841730412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bc1a9eb2c0 msgr2=0x55bc1a9f3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:20.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.361+0000 7f5291f3a640 1 --2- 172.21.15.52:0/2841730412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bc1a9eb2c0 0x55bc1a9f3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.361+0000 7f5291f3a640 1 -- 172.21.15.52:0/2841730412 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc1a9f0760 con 0x55bc1a9f47c0 2024-04-02T14:02:20.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.362+0000 7f5291f3a640 1 --2- 172.21.15.52:0/2841730412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9f47c0 0x55bc1aa0c730 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f528401cd10 tx=0x7f528400add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754f46cdfa27a21f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:20.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.362+0000 7f5276ffd640 1 -- 172.21.15.52:0/2841730412 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284033070 con 0x55bc1a9f47c0 2024-04-02T14:02:20.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.362+0000 7f5276ffd640 1 -- 172.21.15.52:0/2841730412 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f528401de50 con 0x55bc1a9f47c0 2024-04-02T14:02:20.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.362+0000 7f5276ffd640 1 -- 172.21.15.52:0/2841730412 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284020040 con 0x55bc1a9f47c0 2024-04-02T14:02:20.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.362+0000 7f5296589740 1 -- 172.21.15.52:0/2841730412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9f47c0 msgr2=0x55bc1aa0c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:20.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.362+0000 7f5296589740 1 --2- 172.21.15.52:0/2841730412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9f47c0 0x55bc1aa0c730 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f528401cd10 tx=0x7f528400add0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.362+0000 7f5296589740 1 -- 172.21.15.52:0/2841730412 shutdown_connections 2024-04-02T14:02:20.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.362+0000 7f5296589740 1 --2- 172.21.15.52:0/2841730412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9f47c0 0x55bc1aa0c730 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.362+0000 7f5296589740 1 --2- 172.21.15.52:0/2841730412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1a9ebbe0 0x55bc1aa00440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.362+0000 7f5296589740 1 --2- 172.21.15.52:0/2841730412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bc1a9eb2c0 0x55bc1a9f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.362+0000 7f5296589740 1 -- 172.21.15.52:0/2841730412 >> 172.21.15.52:0/2841730412 conn(0x55bc1a9e59f0 msgr2=0x55bc1a9e7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:20.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.363+0000 7f5296589740 1 -- 172.21.15.52:0/2841730412 shutdown_connections 2024-04-02T14:02:20.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.363+0000 7f5296589740 1 -- 172.21.15.52:0/2841730412 wait complete. 2024-04-02T14:02:20.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.363+0000 7f5296589740 1 -- start start 2024-04-02T14:02:20.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.363+0000 7f5296589740 1 Processor -- start 2024-04-02T14:02:20.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.363+0000 7f5296589740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bc1a9eb2c0 0x55bc1a9fcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.363+0000 7f5296589740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9ebbe0 0x55bc1a9f98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.363+0000 7f5296589740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1a9f47c0 0x55bc1a9f9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.363+0000 7f5296589740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f528c1b0390 con 0x55bc1a9ebbe0 2024-04-02T14:02:20.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.363+0000 7f5296589740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f528c1afcc0 con 0x55bc1a9eb2c0 2024-04-02T14:02:20.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.363+0000 7f5296589740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f528c1acf20 con 0x55bc1a9f47c0 2024-04-02T14:02:20.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.363+0000 7f5296589740 10 client.0 _ll_register_callbacks cb 0x7f527c098ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:20.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.363+0000 7f5290f38640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9ebbe0 0x55bc1a9f98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.363+0000 7f5290f38640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9ebbe0 0x55bc1a9f98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53532/0 (socket says 172.21.15.52:53532) 2024-04-02T14:02:20.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.364+0000 7f5290f38640 1 -- 172.21.15.52:0/36620155 learned_addr learned my addr 172.21.15.52:0/36620155 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:20.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.364+0000 7f5291739640 1 --2- 172.21.15.52:0/36620155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bc1a9eb2c0 0x55bc1a9fcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.364+0000 7f5291f3a640 1 --2- 172.21.15.52:0/36620155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1a9f47c0 0x55bc1a9f9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.364+0000 7f5290f38640 1 -- 172.21.15.52:0/36620155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1a9f47c0 msgr2=0x55bc1a9f9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:20.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.364+0000 7f5290f38640 1 --2- 172.21.15.52:0/36620155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1a9f47c0 0x55bc1a9f9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.364+0000 7f5290f38640 1 -- 172.21.15.52:0/36620155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bc1a9eb2c0 msgr2=0x55bc1a9fcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:20.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.364+0000 7f5290f38640 1 --2- 172.21.15.52:0/36620155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bc1a9eb2c0 0x55bc1a9fcb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.364+0000 7f5290f38640 1 -- 172.21.15.52:0/36620155 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc1a9fa370 con 0x55bc1a9ebbe0 2024-04-02T14:02:20.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.364+0000 7f5290f38640 1 --2- 172.21.15.52:0/36620155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9ebbe0 0x55bc1a9f98f0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f527c0991a0 tx=0x7f527c00a520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aff92d3f6054767 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:20.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.364+0000 7f5274ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:20.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.364+0000 7f5274ff9640 1 -- 172.21.15.52:0/36620155 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c0ce450 con 0x55bc1a9ebbe0 2024-04-02T14:02:20.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.364+0000 7f5274ff9640 1 -- 172.21.15.52:0/36620155 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9526+0+0 (secure 0 0 0) 0x7f527c0a5050 con 0x55bc1a9ebbe0 2024-04-02T14:02:20.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.364+0000 7f5296589740 1 -- 172.21.15.52:0/36620155 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f528401de50 con 0x55bc1a9ebbe0 2024-04-02T14:02:20.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.364+0000 7f5274ff9640 1 -- 172.21.15.52:0/36620155 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f527c0a5250 con 0x55bc1a9ebbe0 2024-04-02T14:02:20.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.364+0000 7f5296589740 10 client.18174 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:20.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.364+0000 7f5296589740 1 -- 172.21.15.52:0/36620155 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bc1aa6bd20 con 0x55bc1a9ebbe0 2024-04-02T14:02:20.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.364+0000 7f5274ff9640 1 -- 172.21.15.52:0/36620155 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f528401de50 con 0x55bc1a9ebbe0 2024-04-02T14:02:20.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.365+0000 7f5274ff9640 1 -- 172.21.15.52:0/36620155 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f527c09bc70 con 0x55bc1a9ebbe0 2024-04-02T14:02:20.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.365+0000 7f5296589740 10 client.18174 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:20.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.365+0000 7f5296589740 10 client.18174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:02:20.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.365+0000 7f5296589740 20 client.18174 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:20.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.365+0000 7f5296589740 1 --2- 172.21.15.52:0/36620155 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55bc1aa6c3c0 0x55bc1aa6e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.365+0000 7f5296589740 4 client.18174 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T14:02:20.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.365+0000 7f5296589740 1 -- 172.21.15.52:0/36620155 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bc1a957000 con 0x55bc1aa6c3c0 2024-04-02T14:02:20.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.365+0000 7f5291739640 1 --2- 172.21.15.52:0/36620155 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55bc1aa6c3c0 0x55bc1aa6e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.365+0000 7f5274ff9640 10 client.18174 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:20.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.365+0000 7f5291739640 1 --2- 172.21.15.52:0/36620155 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55bc1aa6c3c0 0x55bc1aa6e7e0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11b4c931a136d09f server_cookie=7f8e6f3fc21bb396 in_seq=0 out_seq=0 2024-04-02T14:02:20.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.367+0000 7f5274ff9640 1 -- 172.21.15.52:0/36620155 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bc1a957000 con 0x55bc1aa6c3c0 2024-04-02T14:02:20.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.367+0000 7f5274ff9640 10 client.18174 handle_command_reply: tid=0 2024-04-02T14:02:20.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.367+0000 7f5296589740 1 client.18174 shutdown 2024-04-02T14:02:20.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.367+0000 7f5296589740 1 -- 172.21.15.52:0/36620155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9ebbe0 msgr2=0x55bc1a9f98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:20.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.367+0000 7f5296589740 1 --2- 172.21.15.52:0/36620155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9ebbe0 0x55bc1a9f98f0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f527c0991a0 tx=0x7f527c00a520 comp rx=0 tx=0).stop 2024-04-02T14:02:20.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.367+0000 7f5296589740 1 -- 172.21.15.52:0/36620155 shutdown_connections 2024-04-02T14:02:20.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.367+0000 7f5296589740 1 --2- 172.21.15.52:0/36620155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1a9f47c0 0x55bc1a9f9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.367+0000 7f5296589740 1 --2- 172.21.15.52:0/36620155 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55bc1aa6c3c0 0x55bc1aa6e7e0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.367+0000 7f5296589740 1 --2- 172.21.15.52:0/36620155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc1a9ebbe0 0x55bc1a9f98f0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.367+0000 7f5296589740 1 --2- 172.21.15.52:0/36620155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bc1a9eb2c0 0x55bc1a9fcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.367+0000 7f5274ff9640 0 client.18174 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:20.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.367+0000 7f5296589740 1 -- 172.21.15.52:0/36620155 >> 172.21.15.52:0/36620155 conn(0x55bc1a9e59f0 msgr2=0x55bc1a9f1770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:20.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.367+0000 7f5296589740 1 -- 172.21.15.52:0/36620155 shutdown_connections 2024-04-02T14:02:20.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.367+0000 7f5296589740 1 -- 172.21.15.52:0/36620155 wait complete. 2024-04-02T14:02:20.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.367+0000 7f5296589740 20 client.18174 trim_cache size 0 max 16384 2024-04-02T14:02:20.381 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:02:20.382 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-04-02T14:02:20.382 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "cd207d03-2399-47e5-a7d2-9824fa434c25", 2024-04-02T14:02:20.382 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-04-02T14:02:20.382 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:02:20.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.368+0000 7f529399d640 1 -- 172.21.15.52:0/4244727940 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f52780a3da0 msgr2=0x7f52780a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:20.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.368+0000 7f529399d640 1 --2- 172.21.15.52:0/4244727940 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f52780a3da0 0x7f52780a6260 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f528400fc30 tx=0x7f528401a000 comp rx=0 tx=0).stop 2024-04-02T14:02:20.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.368+0000 7f529399d640 1 -- 172.21.15.52:0/4244727940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f528c100d80 msgr2=0x7f528c1988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:20.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.368+0000 7f529399d640 1 --2- 172.21.15.52:0/4244727940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f528c100d80 0x7f528c1988a0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f527c00c890 tx=0x7f527c00cd50 comp rx=0 tx=0).stop 2024-04-02T14:02:20.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.368+0000 7f529399d640 1 -- 172.21.15.52:0/4244727940 shutdown_connections 2024-04-02T14:02:20.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.368+0000 7f529399d640 1 --2- 172.21.15.52:0/4244727940 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f52780a3da0 0x7f52780a6260 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.368+0000 7f529399d640 1 --2- 172.21.15.52:0/4244727940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f528c107d00 0x7f528c19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.368+0000 7f529399d640 1 --2- 172.21.15.52:0/4244727940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f528c100d80 0x7f528c1988a0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.368+0000 7f529399d640 1 --2- 172.21.15.52:0/4244727940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f528c1003c0 0x7f528c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.368+0000 7f529399d640 1 -- 172.21.15.52:0/4244727940 >> 172.21.15.52:0/4244727940 conn(0x7f528c075d40 msgr2=0x7f528c0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:20.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.368+0000 7f529399d640 1 -- 172.21.15.52:0/4244727940 shutdown_connections 2024-04-02T14:02:20.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.368+0000 7f529399d640 1 -- 172.21.15.52:0/4244727940 wait complete. 2024-04-02T14:02:20.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:02:20.386 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:02:20.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.498+0000 7fd94719d640 1 -- 172.21.15.52:0/1865288945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd940068d00 msgr2=0x7fd9400690e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:20.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.498+0000 7fd94719d640 1 --2- 172.21.15.52:0/1865288945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd940068d00 0x7fd9400690e0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fd934009900 tx=0x7fd93402bcd0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.498+0000 7fd94719d640 1 -- 172.21.15.52:0/1865288945 shutdown_connections 2024-04-02T14:02:20.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.498+0000 7fd94719d640 1 --2- 172.21.15.52:0/1865288945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd94010e4a0 0x7fd940110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.498+0000 7fd94719d640 1 --2- 172.21.15.52:0/1865288945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd940069620 0x7fd94010df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.498+0000 7fd94719d640 1 --2- 172.21.15.52:0/1865288945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd940068d00 0x7fd9400690e0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.498+0000 7fd94719d640 1 -- 172.21.15.52:0/1865288945 >> 172.21.15.52:0/1865288945 conn(0x7fd940077b00 msgr2=0x7fd940077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:20.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.498+0000 7fd94719d640 1 -- 172.21.15.52:0/1865288945 shutdown_connections 2024-04-02T14:02:20.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.498+0000 7fd94719d640 1 -- 172.21.15.52:0/1865288945 wait complete. 2024-04-02T14:02:20.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.498+0000 7fd94719d640 1 Processor -- start 2024-04-02T14:02:20.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.498+0000 7fd94719d640 1 -- start start 2024-04-02T14:02:20.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.499+0000 7fd94719d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd940068d00 0x7fd94019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.499+0000 7fd94719d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd940069620 0x7fd94019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.499+0000 7fd94719d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd94010e4a0 0x7fd94019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.499+0000 7fd94719d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd940114070 con 0x7fd940068d00 2024-04-02T14:02:20.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.499+0000 7fd94719d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd940113ef0 con 0x7fd940069620 2024-04-02T14:02:20.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.499+0000 7fd94719d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd9401141f0 con 0x7fd94010e4a0 2024-04-02T14:02:20.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.499+0000 7fd944f39640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd940068d00 0x7fd94019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.499+0000 7fd944f39640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd940068d00 0x7fd94019a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53542/0 (socket says 172.21.15.52:53542) 2024-04-02T14:02:20.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.499+0000 7fd944f39640 1 -- 172.21.15.52:0/14479732 learned_addr learned my addr 172.21.15.52:0/14479732 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:20.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.499+0000 7fd93ffff640 1 --2- 172.21.15.52:0/14479732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd940069620 0x7fd94019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.499+0000 7fd94573a640 1 --2- 172.21.15.52:0/14479732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd94010e4a0 0x7fd94019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.499+0000 7fd944f39640 1 -- 172.21.15.52:0/14479732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd94010e4a0 msgr2=0x7fd94019edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:20.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.499+0000 7fd944f39640 1 --2- 172.21.15.52:0/14479732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd94010e4a0 0x7fd94019edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.499+0000 7fd944f39640 1 -- 172.21.15.52:0/14479732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd940069620 msgr2=0x7fd94019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:20.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.499+0000 7fd944f39640 1 --2- 172.21.15.52:0/14479732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd940069620 0x7fd94019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.499+0000 7fd944f39640 1 -- 172.21.15.52:0/14479732 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94019f4e0 con 0x7fd940068d00 2024-04-02T14:02:20.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.499+0000 7fd944f39640 1 --2- 172.21.15.52:0/14479732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd940068d00 0x7fd94019a4f0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fd934006010 tx=0x7fd93402cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:20.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.499+0000 7fd93dffb640 1 -- 172.21.15.52:0/14479732 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934043020 con 0x7fd940068d00 2024-04-02T14:02:20.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.499+0000 7fd93dffb640 1 -- 172.21.15.52:0/14479732 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9340054e0 con 0x7fd940068d00 2024-04-02T14:02:20.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.499+0000 7fd94719d640 1 -- 172.21.15.52:0/14479732 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94019f770 con 0x7fd940068d00 2024-04-02T14:02:20.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.499+0000 7fd93dffb640 1 -- 172.21.15.52:0/14479732 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934003c60 con 0x7fd940068d00 2024-04-02T14:02:20.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.500+0000 7fd94719d640 1 -- 172.21.15.52:0/14479732 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9401a7060 con 0x7fd940068d00 2024-04-02T14:02:20.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.502+0000 7fd93dffb640 1 -- 172.21.15.52:0/14479732 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd934046020 con 0x7fd940068d00 2024-04-02T14:02:20.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.502+0000 7fd949d87740 1 Processor -- start 2024-04-02T14:02:20.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.502+0000 7fd949d87740 1 -- start start 2024-04-02T14:02:20.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.503+0000 7fd949d87740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55608ab788d0 0x55608ab78cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.503+0000 7fd949d87740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab791f0 0x55608ab81890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.503+0000 7fd949d87740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55608ab81dd0 0x55608ab841c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.503+0000 7fd949d87740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55608aadc830 con 0x55608ab791f0 2024-04-02T14:02:20.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.503+0000 7fd949d87740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55608aa84820 con 0x55608ab788d0 2024-04-02T14:02:20.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.503+0000 7fd949d87740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55608aad87e0 con 0x55608ab81dd0 2024-04-02T14:02:20.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.503+0000 7fd93dffb640 1 --2- 172.21.15.52:0/14479732 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd9300a3d20 0x7fd9300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.503+0000 7fd94573a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55608ab788d0 0x55608ab78cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.503+0000 7fd94573a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55608ab788d0 0x55608ab78cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:45960/0 (socket says 172.21.15.52:45960) 2024-04-02T14:02:20.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.503+0000 7fd94573a640 1 -- 172.21.15.52:0/3182211653 learned_addr learned my addr 172.21.15.52:0/3182211653 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:20.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.503+0000 7fd94573a640 1 --2- 172.21.15.52:0/14479732 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd9300a3d20 0x7fd9300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.503+0000 7fd93ffff640 1 --2- 172.21.15.52:0/3182211653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55608ab81dd0 0x55608ab841c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.503+0000 7fd93dffb640 1 -- 172.21.15.52:0/14479732 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7fd9340f2b50 con 0x7fd940068d00 2024-04-02T14:02:20.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.503+0000 7fd944f39640 1 --2- 172.21.15.52:0/3182211653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab791f0 0x55608ab81890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.503+0000 7fd94573a640 1 -- 172.21.15.52:0/3182211653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55608ab81dd0 msgr2=0x55608ab841c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:20.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.503+0000 7fd94573a640 1 --2- 172.21.15.52:0/3182211653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55608ab81dd0 0x55608ab841c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.503+0000 7fd94573a640 1 -- 172.21.15.52:0/3182211653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab791f0 msgr2=0x55608ab81890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:20.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.503+0000 7fd94573a640 1 --2- 172.21.15.52:0/3182211653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab791f0 0x55608ab81890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.503+0000 7fd94573a640 1 -- 172.21.15.52:0/3182211653 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55608a8ba7d0 con 0x55608ab788d0 2024-04-02T14:02:20.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.504+0000 7fd94573a640 1 --2- 172.21.15.52:0/3182211653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55608ab788d0 0x55608ab78cb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd938012d80 tx=0x7fd938010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25fa4d148fcc6fae server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:20.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.504+0000 7fd94573a640 1 --2- 172.21.15.52:0/14479732 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd9300a3d20 0x7fd9300a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd938010c40 tx=0x7fd938017000 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:20.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.504+0000 7fd92affd640 1 -- 172.21.15.52:0/3182211653 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938023070 con 0x55608ab788d0 2024-04-02T14:02:20.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.504+0000 7fd92affd640 1 -- 172.21.15.52:0/3182211653 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd938009d80 con 0x55608ab788d0 2024-04-02T14:02:20.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.504+0000 7fd92affd640 1 -- 172.21.15.52:0/3182211653 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938003ce0 con 0x55608ab788d0 2024-04-02T14:02:20.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.505+0000 7fd949d87740 1 -- 172.21.15.52:0/3182211653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55608ab788d0 msgr2=0x55608ab78cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:20.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.505+0000 7fd949d87740 1 --2- 172.21.15.52:0/3182211653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55608ab788d0 0x55608ab78cb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd938012d80 tx=0x7fd938010740 comp rx=0 tx=0).stop 2024-04-02T14:02:20.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.505+0000 7fd949d87740 1 -- 172.21.15.52:0/3182211653 shutdown_connections 2024-04-02T14:02:20.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.505+0000 7fd949d87740 1 --2- 172.21.15.52:0/3182211653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55608ab81dd0 0x55608ab841c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.505+0000 7fd949d87740 1 --2- 172.21.15.52:0/3182211653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab791f0 0x55608ab81890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.505+0000 7fd949d87740 1 --2- 172.21.15.52:0/3182211653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55608ab788d0 0x55608ab78cb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.505+0000 7fd949d87740 1 -- 172.21.15.52:0/3182211653 >> 172.21.15.52:0/3182211653 conn(0x55608ab73000 msgr2=0x55608ab733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:20.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.505+0000 7fd949d87740 1 -- 172.21.15.52:0/3182211653 shutdown_connections 2024-04-02T14:02:20.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.505+0000 7fd949d87740 1 -- 172.21.15.52:0/3182211653 wait complete. 2024-04-02T14:02:20.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.505+0000 7fd949d87740 1 -- start start 2024-04-02T14:02:20.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.506+0000 7fd949d87740 1 Processor -- start 2024-04-02T14:02:20.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.506+0000 7fd949d87740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55608ab788d0 0x55608ab81760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.506+0000 7fd949d87740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab791f0 0x55608ab7eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.506+0000 7fd949d87740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55608ab81dd0 0x55608ab7f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.506+0000 7fd949d87740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55608ab84b60 con 0x55608ab791f0 2024-04-02T14:02:20.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.506+0000 7fd949d87740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55608aa97d30 con 0x55608ab81dd0 2024-04-02T14:02:20.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.506+0000 7fd949d87740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55608ab84ce0 con 0x55608ab788d0 2024-04-02T14:02:20.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.506+0000 7fd949d87740 10 client.0 _ll_register_callbacks cb 0x55608aa97720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:20.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.506+0000 7fd93ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab791f0 0x55608ab7eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.506+0000 7fd944f39640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55608ab788d0 0x55608ab81760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.506+0000 7fd93ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab791f0 0x55608ab7eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53548/0 (socket says 172.21.15.52:53548) 2024-04-02T14:02:20.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.506+0000 7fd93ffff640 1 -- 172.21.15.52:0/1673705307 learned_addr learned my addr 172.21.15.52:0/1673705307 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:20.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.506+0000 7fd94573a640 1 --2- 172.21.15.52:0/1673705307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55608ab81dd0 0x55608ab7f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.507+0000 7fd93ffff640 1 -- 172.21.15.52:0/1673705307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55608ab788d0 msgr2=0x55608ab81760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:20.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.507+0000 7fd93ffff640 1 --2- 172.21.15.52:0/1673705307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55608ab788d0 0x55608ab81760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.507+0000 7fd93ffff640 1 -- 172.21.15.52:0/1673705307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55608ab81dd0 msgr2=0x55608ab7f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:20.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.507+0000 7fd93ffff640 1 --2- 172.21.15.52:0/1673705307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55608ab81dd0 0x55608ab7f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.507+0000 7fd93ffff640 1 -- 172.21.15.52:0/1673705307 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55608a7f03f0 con 0x55608ab791f0 2024-04-02T14:02:20.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.507+0000 7fd93ffff640 1 --2- 172.21.15.52:0/1673705307 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab791f0 0x55608ab7eaf0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fd92c012700 tx=0x7fd92c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a266ecef9e67ff server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:20.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.507+0000 7fd91ffff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:20.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.507+0000 7fd91ffff640 1 -- 172.21.15.52:0/1673705307 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92c004050 con 0x55608ab791f0 2024-04-02T14:02:20.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.507+0000 7fd91ffff640 1 -- 172.21.15.52:0/1673705307 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7fd92c0041f0 con 0x55608ab791f0 2024-04-02T14:02:20.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.507+0000 7fd949d87740 1 -- 172.21.15.52:0/1673705307 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd938009d80 con 0x55608ab791f0 2024-04-02T14:02:20.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.507+0000 7fd91ffff640 1 -- 172.21.15.52:0/1673705307 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd92c025050 con 0x55608ab791f0 2024-04-02T14:02:20.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.507+0000 7fd949d87740 10 client.18186 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:20.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.507+0000 7fd949d87740 1 -- 172.21.15.52:0/1673705307 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55608a7b00f0 con 0x55608ab791f0 2024-04-02T14:02:20.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.508+0000 7fd91ffff640 1 -- 172.21.15.52:0/1673705307 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd938009d80 con 0x55608ab791f0 2024-04-02T14:02:20.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.508+0000 7fd91ffff640 1 -- 172.21.15.52:0/1673705307 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd92c003a30 con 0x55608ab791f0 2024-04-02T14:02:20.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.508+0000 7fd949d87740 10 client.18186 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:20.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.508+0000 7fd949d87740 10 client.18186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:02:20.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.508+0000 7fd949d87740 20 client.18186 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:20.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.508+0000 7fd949d87740 1 --2- 172.21.15.52:0/1673705307 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55608ab91e40 0x55608ab942a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.508+0000 7fd949d87740 4 client.18186 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:02:20.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.508+0000 7fd949d87740 1 -- 172.21.15.52:0/1673705307 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55608aae9d70 con 0x55608ab91e40 2024-04-02T14:02:20.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.508+0000 7fd944f39640 1 --2- 172.21.15.52:0/1673705307 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55608ab91e40 0x55608ab942a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.509+0000 7fd91ffff640 10 client.18186 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:20.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.509+0000 7fd944f39640 1 --2- 172.21.15.52:0/1673705307 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55608ab91e40 0x55608ab942a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba81ca97d4332470 server_cookie=6b685ab072943ef0 in_seq=0 out_seq=0 2024-04-02T14:02:20.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.509+0000 7fd91ffff640 1 -- 172.21.15.52:0/1673705307 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55608aae9d70 con 0x55608ab91e40 2024-04-02T14:02:20.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.509+0000 7fd91ffff640 10 client.18186 handle_command_reply: tid=0 2024-04-02T14:02:20.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.509+0000 7fd949d87740 1 client.18186 shutdown 2024-04-02T14:02:20.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.510+0000 7fd949d87740 1 -- 172.21.15.52:0/1673705307 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab791f0 msgr2=0x55608ab7eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:20.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.510+0000 7fd949d87740 1 --2- 172.21.15.52:0/1673705307 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab791f0 0x55608ab7eaf0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fd92c012700 tx=0x7fd92c012bc0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.510+0000 7fd949d87740 1 -- 172.21.15.52:0/1673705307 shutdown_connections 2024-04-02T14:02:20.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.510+0000 7fd949d87740 1 --2- 172.21.15.52:0/1673705307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55608ab81dd0 0x55608ab7f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.510+0000 7fd949d87740 1 --2- 172.21.15.52:0/1673705307 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55608ab91e40 0x55608ab942a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.510+0000 7fd949d87740 1 --2- 172.21.15.52:0/1673705307 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab791f0 0x55608ab7eaf0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.510+0000 7fd91ffff640 0 client.18186 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:20.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.510+0000 7fd949d87740 1 --2- 172.21.15.52:0/1673705307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55608ab788d0 0x55608ab81760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.510+0000 7fd949d87740 1 -- 172.21.15.52:0/1673705307 >> 172.21.15.52:0/1673705307 conn(0x55608ab73000 msgr2=0x55608ab74d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:20.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.510+0000 7fd949d87740 1 -- 172.21.15.52:0/1673705307 shutdown_connections 2024-04-02T14:02:20.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.510+0000 7fd949d87740 1 -- 172.21.15.52:0/1673705307 wait complete. 2024-04-02T14:02:20.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.510+0000 7fd949d87740 20 client.18186 trim_cache size 0 max 16384 2024-04-02T14:02:20.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.531+0000 7fd949d87740 1 Processor -- start 2024-04-02T14:02:20.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.531+0000 7fd949d87740 1 -- start start 2024-04-02T14:02:20.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.531+0000 7fd949d87740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55608ab788d0 0x55608ab8da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.531+0000 7fd949d87740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55608ab791f0 0x55608ab72240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.531+0000 7fd949d87740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab81dd0 0x55608ab81220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.531+0000 7fd949d87740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd9401b10c0 con 0x55608ab81dd0 2024-04-02T14:02:20.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.531+0000 7fd949d87740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd9401b2c00 con 0x55608ab791f0 2024-04-02T14:02:20.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.531+0000 7fd949d87740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd9401b1790 con 0x55608ab788d0 2024-04-02T14:02:20.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.531+0000 7fd94573a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab81dd0 0x55608ab81220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.531+0000 7fd94573a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab81dd0 0x55608ab81220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53550/0 (socket says 172.21.15.52:53550) 2024-04-02T14:02:20.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.531+0000 7fd93ffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55608ab791f0 0x55608ab72240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.531+0000 7fd94573a640 1 -- 172.21.15.52:0/427819081 learned_addr learned my addr 172.21.15.52:0/427819081 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:20.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.531+0000 7fd944f39640 1 --2- 172.21.15.52:0/427819081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55608ab788d0 0x55608ab8da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.531+0000 7fd944f39640 1 -- 172.21.15.52:0/427819081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55608ab791f0 msgr2=0x55608ab72240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:20.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.531+0000 7fd944f39640 1 --2- 172.21.15.52:0/427819081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55608ab791f0 0x55608ab72240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.531+0000 7fd944f39640 1 -- 172.21.15.52:0/427819081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab81dd0 msgr2=0x55608ab81220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:20.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.531+0000 7fd944f39640 1 --2- 172.21.15.52:0/427819081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab81dd0 0x55608ab81220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.531+0000 7fd944f39640 1 -- 172.21.15.52:0/427819081 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55608ab6ec50 con 0x55608ab788d0 2024-04-02T14:02:20.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.531+0000 7fd93ffff640 1 --2- 172.21.15.52:0/427819081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55608ab791f0 0x55608ab72240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:02:20.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.531+0000 7fd94573a640 1 --2- 172.21.15.52:0/427819081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab81dd0 0x55608ab81220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:02:20.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.532+0000 7fd944f39640 1 --2- 172.21.15.52:0/427819081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55608ab788d0 0x55608ab8da20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd9340f0020 tx=0x7fd9340bab50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=746d830cb1b9486e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:20.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.532+0000 7fd92a7fc640 1 -- 172.21.15.52:0/427819081 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934005690 con 0x55608ab788d0 2024-04-02T14:02:20.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.532+0000 7fd92a7fc640 1 -- 172.21.15.52:0/427819081 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9340bef70 con 0x55608ab788d0 2024-04-02T14:02:20.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.532+0000 7fd92a7fc640 1 -- 172.21.15.52:0/427819081 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9340f0190 con 0x55608ab788d0 2024-04-02T14:02:20.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.533+0000 7fd949d87740 1 -- 172.21.15.52:0/427819081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55608ab788d0 msgr2=0x55608ab8da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:20.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.533+0000 7fd949d87740 1 --2- 172.21.15.52:0/427819081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55608ab788d0 0x55608ab8da20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd9340f0020 tx=0x7fd9340bab50 comp rx=0 tx=0).stop 2024-04-02T14:02:20.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.533+0000 7fd949d87740 1 -- 172.21.15.52:0/427819081 shutdown_connections 2024-04-02T14:02:20.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.533+0000 7fd949d87740 1 --2- 172.21.15.52:0/427819081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab81dd0 0x55608ab81220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.533+0000 7fd949d87740 1 --2- 172.21.15.52:0/427819081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55608ab791f0 0x55608ab72240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.533+0000 7fd949d87740 1 --2- 172.21.15.52:0/427819081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55608ab788d0 0x55608ab8da20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.533+0000 7fd949d87740 1 -- 172.21.15.52:0/427819081 >> 172.21.15.52:0/427819081 conn(0x55608ab73000 msgr2=0x55608ab7da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:20.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.533+0000 7fd949d87740 1 -- 172.21.15.52:0/427819081 shutdown_connections 2024-04-02T14:02:20.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.533+0000 7fd949d87740 1 -- 172.21.15.52:0/427819081 wait complete. 2024-04-02T14:02:20.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.533+0000 7fd949d87740 1 -- start start 2024-04-02T14:02:20.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.533+0000 7fd949d87740 1 Processor -- start 2024-04-02T14:02:20.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.534+0000 7fd949d87740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55608ab788d0 0x55608ab85890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.534+0000 7fd949d87740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55608ab791f0 0x55608ab85dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.534+0000 7fd949d87740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab81dd0 0x55608ab86310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.534+0000 7fd949d87740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd9401b2530 con 0x55608ab81dd0 2024-04-02T14:02:20.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.534+0000 7fd949d87740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd9401b1e60 con 0x55608ab788d0 2024-04-02T14:02:20.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.534+0000 7fd949d87740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd9401af0c0 con 0x55608ab791f0 2024-04-02T14:02:20.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.534+0000 7fd949d87740 10 client.0 _ll_register_callbacks cb 0x7fd92c009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:20.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.534+0000 7fd94573a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab81dd0 0x55608ab86310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.534+0000 7fd94573a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab81dd0 0x55608ab86310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53562/0 (socket says 172.21.15.52:53562) 2024-04-02T14:02:20.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.534+0000 7fd93ffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55608ab791f0 0x55608ab85dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.534+0000 7fd94573a640 1 -- 172.21.15.52:0/4260897031 learned_addr learned my addr 172.21.15.52:0/4260897031 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:20.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.534+0000 7fd944f39640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55608ab788d0 0x55608ab85890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.534+0000 7fd94573a640 1 -- 172.21.15.52:0/4260897031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55608ab791f0 msgr2=0x55608ab85dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:20.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.534+0000 7fd94573a640 1 --2- 172.21.15.52:0/4260897031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55608ab791f0 0x55608ab85dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.534+0000 7fd94573a640 1 -- 172.21.15.52:0/4260897031 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55608ab788d0 msgr2=0x55608ab85890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:20.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.534+0000 7fd94573a640 1 --2- 172.21.15.52:0/4260897031 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55608ab788d0 0x55608ab85890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.534+0000 7fd94573a640 1 -- 172.21.15.52:0/4260897031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55608abf73e0 con 0x55608ab81dd0 2024-04-02T14:02:20.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.534+0000 7fd94573a640 1 --2- 172.21.15.52:0/4260897031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab81dd0 0x55608ab86310 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fd9380096e0 tx=0x7fd93800de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834b904d9eb1b11f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:20.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.534+0000 7fd91ffff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:20.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.534+0000 7fd91ffff640 1 -- 172.21.15.52:0/4260897031 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938023040 con 0x55608ab81dd0 2024-04-02T14:02:20.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.534+0000 7fd91ffff640 1 -- 172.21.15.52:0/4260897031 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7fd938003970 con 0x55608ab81dd0 2024-04-02T14:02:20.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.534+0000 7fd949d87740 1 -- 172.21.15.52:0/4260897031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9340bef70 con 0x55608ab81dd0 2024-04-02T14:02:20.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.535+0000 7fd91ffff640 1 -- 172.21.15.52:0/4260897031 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd938012890 con 0x55608ab81dd0 2024-04-02T14:02:20.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.535+0000 7fd949d87740 10 client.18192 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:20.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.535+0000 7fd949d87740 1 -- 172.21.15.52:0/4260897031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55608abf8630 con 0x55608ab81dd0 2024-04-02T14:02:20.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.535+0000 7fd91ffff640 1 -- 172.21.15.52:0/4260897031 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9340bef70 con 0x55608ab81dd0 2024-04-02T14:02:20.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.535+0000 7fd91ffff640 1 -- 172.21.15.52:0/4260897031 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd938005820 con 0x55608ab81dd0 2024-04-02T14:02:20.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.535+0000 7fd949d87740 10 client.18192 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:20.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.535+0000 7fd949d87740 10 client.18192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:02:20.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.535+0000 7fd949d87740 20 client.18192 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:20.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.535+0000 7fd949d87740 1 --2- 172.21.15.52:0/4260897031 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55608abf8b50 0x55608abfaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:20.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.535+0000 7fd949d87740 4 client.18192 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:02:20.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.535+0000 7fd949d87740 1 -- 172.21.15.52:0/4260897031 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55608aae40c0 con 0x55608abf8b50 2024-04-02T14:02:20.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.535+0000 7fd944f39640 1 --2- 172.21.15.52:0/4260897031 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55608abf8b50 0x55608abfaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:20.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.536+0000 7fd944f39640 1 --2- 172.21.15.52:0/4260897031 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55608abf8b50 0x55608abfaf70 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9504c7e585ae9d3d server_cookie=6f8289fa8858cc89 in_seq=0 out_seq=0 2024-04-02T14:02:20.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.536+0000 7fd91ffff640 10 client.18192 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:20.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.536+0000 7fd91ffff640 1 -- 172.21.15.52:0/4260897031 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55608aae40c0 con 0x55608abf8b50 2024-04-02T14:02:20.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.536+0000 7fd91ffff640 10 client.18192 handle_command_reply: tid=0 2024-04-02T14:02:20.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.536+0000 7fd949d87740 1 client.18192 shutdown 2024-04-02T14:02:20.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.536+0000 7fd949d87740 1 -- 172.21.15.52:0/4260897031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab81dd0 msgr2=0x55608ab86310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:20.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.536+0000 7fd949d87740 1 --2- 172.21.15.52:0/4260897031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab81dd0 0x55608ab86310 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fd9380096e0 tx=0x7fd93800de20 comp rx=0 tx=0).stop 2024-04-02T14:02:20.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.536+0000 7fd949d87740 1 -- 172.21.15.52:0/4260897031 shutdown_connections 2024-04-02T14:02:20.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.536+0000 7fd949d87740 1 --2- 172.21.15.52:0/4260897031 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55608abf8b50 0x55608abfaf70 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.536+0000 7fd949d87740 1 --2- 172.21.15.52:0/4260897031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55608ab81dd0 0x55608ab86310 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.536+0000 7fd949d87740 1 --2- 172.21.15.52:0/4260897031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55608ab791f0 0x55608ab85dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.536+0000 7fd949d87740 1 --2- 172.21.15.52:0/4260897031 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55608ab788d0 0x55608ab85890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.536+0000 7fd91ffff640 0 client.18192 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:20.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.536+0000 7fd949d87740 1 -- 172.21.15.52:0/4260897031 >> 172.21.15.52:0/4260897031 conn(0x55608ab73000 msgr2=0x55608ab72240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:20.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.537+0000 7fd949d87740 1 -- 172.21.15.52:0/4260897031 shutdown_connections 2024-04-02T14:02:20.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.537+0000 7fd949d87740 1 -- 172.21.15.52:0/4260897031 wait complete. 2024-04-02T14:02:20.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.537+0000 7fd949d87740 20 client.18192 trim_cache size 0 max 16384 2024-04-02T14:02:20.551 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:02:20.552 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "scrub active (2677 inodes in the stack)", 2024-04-02T14:02:20.552 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-04-02T14:02:20.552 INFO:teuthology.orchestra.run.smithi052.stdout: "cd207d03-2399-47e5-a7d2-9824fa434c25": { 2024-04-02T14:02:20.552 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-04-02T14:02:20.552 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "cd207d03-2399-47e5-a7d2-9824fa434c25", 2024-04-02T14:02:20.552 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-04-02T14:02:20.552 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:02:20.552 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:02:20.553 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:02:20.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.537+0000 7fd94719d640 1 -- 172.21.15.52:0/14479732 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd9300a3d20 msgr2=0x7fd9300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:20.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.537+0000 7fd94719d640 1 --2- 172.21.15.52:0/14479732 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd9300a3d20 0x7fd9300a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd938010c40 tx=0x7fd938017000 comp rx=0 tx=0).stop 2024-04-02T14:02:20.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.537+0000 7fd94719d640 1 -- 172.21.15.52:0/14479732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd940068d00 msgr2=0x7fd94019a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:20.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.537+0000 7fd94719d640 1 --2- 172.21.15.52:0/14479732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd940068d00 0x7fd94019a4f0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fd934006010 tx=0x7fd93402cbd0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.537+0000 7fd94719d640 1 -- 172.21.15.52:0/14479732 shutdown_connections 2024-04-02T14:02:20.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.537+0000 7fd94719d640 1 --2- 172.21.15.52:0/14479732 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd9300a3d20 0x7fd9300a61e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.537+0000 7fd94719d640 1 --2- 172.21.15.52:0/14479732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd94010e4a0 0x7fd94019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.537+0000 7fd94719d640 1 --2- 172.21.15.52:0/14479732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd940069620 0x7fd94019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.537+0000 7fd94719d640 1 --2- 172.21.15.52:0/14479732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd940068d00 0x7fd94019a4f0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:20.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.537+0000 7fd94719d640 1 -- 172.21.15.52:0/14479732 >> 172.21.15.52:0/14479732 conn(0x7fd940077b00 msgr2=0x7fd94010f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:20.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.538+0000 7fd94719d640 1 -- 172.21.15.52:0/14479732 shutdown_connections 2024-04-02T14:02:20.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:20.538+0000 7fd94719d640 1 -- 172.21.15.52:0/14479732 wait complete. 2024-04-02T14:02:20.557 INFO:tasks.cephfs.filesystem:scrub status for tag:cd207d03-2399-47e5-a7d2-9824fa434c25 - {'path': '/', 'tag': 'cd207d03-2399-47e5-a7d2-9824fa434c25', 'options': 'recursive,force'} 2024-04-02T14:02:21.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:21 smithi189 ceph-mon[36683]: pgmap v273: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:02:21.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:21 smithi189 ceph-mon[36683]: scrub queued for path: / 2024-04-02T14:02:21.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:21 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:02:21.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:21 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T14:02:21.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:21 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T14:02:21.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:21 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:02:21.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:21 smithi189 ceph-mon[36683]: osdmap e99: 12 total, 12 up, 12 in 2024-04-02T14:02:21.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:21 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T14:02:21.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:21 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:02:21.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:21 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:02:21.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:21 smithi052 ceph-mon[32104]: pgmap v273: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:02:21.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:21 smithi052 ceph-mon[32104]: scrub queued for path: / 2024-04-02T14:02:21.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:21 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:02:21.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:21 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T14:02:21.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:21 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T14:02:21.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:21 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:02:21.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:21 smithi052 ceph-mon[32104]: osdmap e99: 12 total, 12 up, 12 in 2024-04-02T14:02:21.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:21 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T14:02:21.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:21 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:02:21.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:21 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:02:21.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:21 smithi088 ceph-mon[36528]: pgmap v273: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:02:21.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:21 smithi088 ceph-mon[36528]: scrub queued for path: / 2024-04-02T14:02:21.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:21 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:02:21.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:21 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T14:02:21.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:21 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T14:02:21.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:21 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:02:21.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:21 smithi088 ceph-mon[36528]: osdmap e99: 12 total, 12 up, 12 in 2024-04-02T14:02:21.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:21 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T14:02:21.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:21 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:02:21.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:21 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:02:23.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:23 smithi189 ceph-mon[36683]: pgmap v275: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:02:23.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:23 smithi052 ceph-mon[32104]: pgmap v275: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:02:23.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:23 smithi088 ceph-mon[36528]: pgmap v275: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:02:25.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:25 smithi189 ceph-mon[36683]: pgmap v276: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:02:25.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:25 smithi052 ceph-mon[32104]: pgmap v276: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:02:25.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:25 smithi088 ceph-mon[36528]: pgmap v276: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:02:27.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:27 smithi189 ceph-mon[36683]: pgmap v277: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-02T14:02:27.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:27 smithi052 ceph-mon[32104]: pgmap v277: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-02T14:02:27.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:27 smithi088 ceph-mon[36528]: pgmap v277: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-02T14:02:29.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:29 smithi189 ceph-mon[36683]: pgmap v278: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-02T14:02:29.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:29 smithi052 ceph-mon[32104]: pgmap v278: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-02T14:02:29.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:29 smithi088 ceph-mon[36528]: pgmap v278: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-02T14:02:30.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:30 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T14:02:30.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:30 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T14:02:30.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:30 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T14:02:31.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:31 smithi189 ceph-mon[36683]: pgmap v279: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-02T14:02:31.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:31 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T14:02:31.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:31 smithi052 ceph-mon[32104]: pgmap v279: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-02T14:02:31.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:31 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T14:02:31.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:31 smithi088 ceph-mon[36528]: pgmap v279: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-02T14:02:31.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:31 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T14:02:33.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:33 smithi189 ceph-mon[36683]: pgmap v280: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 13 op/s 2024-04-02T14:02:33.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:33 smithi052 ceph-mon[32104]: pgmap v280: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 13 op/s 2024-04-02T14:02:33.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:33 smithi088 ceph-mon[36528]: pgmap v280: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 13 op/s 2024-04-02T14:02:35.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:35 smithi189 ceph-mon[36683]: pgmap v281: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:02:35.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:35 smithi052 ceph-mon[32104]: pgmap v281: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:02:35.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:35 smithi088 ceph-mon[36528]: pgmap v281: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:02:37.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:37 smithi189 ceph-mon[36683]: pgmap v282: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:02:37.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:37 smithi052 ceph-mon[32104]: pgmap v282: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:02:37.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:37 smithi088 ceph-mon[36528]: pgmap v282: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:02:39.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:39 smithi189 ceph-mon[36683]: pgmap v283: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:02:39.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:39 smithi052 ceph-mon[32104]: pgmap v283: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:02:39.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:39 smithi088 ceph-mon[36528]: pgmap v283: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:02:41.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:41 smithi189 ceph-mon[36683]: pgmap v284: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:02:41.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:41 smithi052 ceph-mon[32104]: pgmap v284: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:02:41.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:41 smithi088 ceph-mon[36528]: pgmap v284: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:02:42.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:42 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:02:42.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:42 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:02:42.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:42 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:02:43.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:43 smithi189 ceph-mon[36683]: pgmap v285: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:02:43.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:43 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:02:43.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:43 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:02:43.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:43 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:02:43.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:43 smithi052 ceph-mon[32104]: pgmap v285: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:02:43.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:43 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:02:43.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:43 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:02:43.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:43 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:02:43.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:43 smithi088 ceph-mon[36528]: pgmap v285: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:02:43.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:43 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:02:43.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:43 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:02:43.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:43 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:02:45.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:45 smithi189 ceph-mon[36683]: pgmap v286: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:02:45.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:45 smithi052 ceph-mon[32104]: pgmap v286: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:02:45.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:45 smithi088 ceph-mon[36528]: pgmap v286: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:02:47.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:47 smithi189 ceph-mon[36683]: pgmap v287: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:02:47.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:47 smithi052 ceph-mon[32104]: pgmap v287: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:02:47.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:47 smithi088 ceph-mon[36528]: pgmap v287: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:02:49.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:49 smithi189 ceph-mon[36683]: pgmap v288: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:02:49.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:49 smithi052 ceph-mon[32104]: pgmap v288: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:02:49.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:49 smithi088 ceph-mon[36528]: pgmap v288: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:02:50.558 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:02:50.558 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:02:50.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.673+0000 7fa45702a640 1 -- 172.21.15.52:0/1621239048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa450101710 msgr2=0x7fa45010d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:50.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.673+0000 7fa45702a640 1 --2- 172.21.15.52:0/1621239048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa450101710 0x7fa45010d7e0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fa448009cd0 tx=0x7fa44802c0f0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.673+0000 7fa45702a640 1 -- 172.21.15.52:0/1621239048 shutdown_connections 2024-04-02T14:02:50.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.673+0000 7fa45702a640 1 --2- 172.21.15.52:0/1621239048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa450101710 0x7fa45010d7e0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.673+0000 7fa45702a640 1 --2- 172.21.15.52:0/1621239048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa450100d50 0x7fa4501011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.673+0000 7fa45702a640 1 --2- 172.21.15.52:0/1621239048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4501059e0 0x7fa450105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.673+0000 7fa45702a640 1 -- 172.21.15.52:0/1621239048 >> 172.21.15.52:0/1621239048 conn(0x7fa450076800 msgr2=0x7fa450076c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:50.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.673+0000 7fa45702a640 1 -- 172.21.15.52:0/1621239048 shutdown_connections 2024-04-02T14:02:50.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.673+0000 7fa45702a640 1 -- 172.21.15.52:0/1621239048 wait complete. 2024-04-02T14:02:50.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.674+0000 7fa45702a640 1 Processor -- start 2024-04-02T14:02:50.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.674+0000 7fa45702a640 1 -- start start 2024-04-02T14:02:50.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.674+0000 7fa45702a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa450100d50 0x7fa4501991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:50.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.674+0000 7fa45702a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa450101710 0x7fa4501996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:50.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.674+0000 7fa45702a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4501059e0 0x7fa4501a4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:50.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.674+0000 7fa45702a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa450110e10 con 0x7fa450101710 2024-04-02T14:02:50.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.674+0000 7fa45702a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa450110c90 con 0x7fa450100d50 2024-04-02T14:02:50.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.674+0000 7fa45702a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa450110f90 con 0x7fa4501059e0 2024-04-02T14:02:50.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.674+0000 7fa447fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa450101710 0x7fa4501996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:50.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.674+0000 7fa447fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa450101710 0x7fa4501996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45616/0 (socket says 172.21.15.52:45616) 2024-04-02T14:02:50.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.674+0000 7fa447fff640 1 -- 172.21.15.52:0/1520414763 learned_addr learned my addr 172.21.15.52:0/1520414763 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:50.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.674+0000 7fa454dc6640 1 --2- 172.21.15.52:0/1520414763 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa450100d50 0x7fa4501991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:50.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.674+0000 7fa4555c7640 1 --2- 172.21.15.52:0/1520414763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4501059e0 0x7fa4501a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:50.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.674+0000 7fa447fff640 1 -- 172.21.15.52:0/1520414763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4501059e0 msgr2=0x7fa4501a4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:50.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.674+0000 7fa447fff640 1 --2- 172.21.15.52:0/1520414763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4501059e0 0x7fa4501a4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.674+0000 7fa447fff640 1 -- 172.21.15.52:0/1520414763 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa450100d50 msgr2=0x7fa4501991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:50.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.674+0000 7fa447fff640 1 --2- 172.21.15.52:0/1520414763 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa450100d50 0x7fa4501991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.674+0000 7fa447fff640 1 -- 172.21.15.52:0/1520414763 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4501a4770 con 0x7fa450101710 2024-04-02T14:02:50.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.674+0000 7fa4555c7640 1 --2- 172.21.15.52:0/1520414763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4501059e0 0x7fa4501a4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:02:50.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.674+0000 7fa447fff640 1 --2- 172.21.15.52:0/1520414763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa450101710 0x7fa4501996f0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fa44000c920 tx=0x7fa44000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:50.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.675+0000 7fa445ffb640 1 -- 172.21.15.52:0/1520414763 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa440007c70 con 0x7fa450101710 2024-04-02T14:02:50.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.675+0000 7fa45702a640 1 -- 172.21.15.52:0/1520414763 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4501a4940 con 0x7fa450101710 2024-04-02T14:02:50.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.675+0000 7fa445ffb640 1 -- 172.21.15.52:0/1520414763 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa440007e10 con 0x7fa450101710 2024-04-02T14:02:50.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.675+0000 7fa445ffb640 1 -- 172.21.15.52:0/1520414763 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa440018890 con 0x7fa450101710 2024-04-02T14:02:50.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.675+0000 7fa45702a640 1 -- 172.21.15.52:0/1520414763 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4501a4e30 con 0x7fa450101710 2024-04-02T14:02:50.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.677+0000 7fa445ffb640 1 -- 172.21.15.52:0/1520414763 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa440018a70 con 0x7fa450101710 2024-04-02T14:02:50.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.678+0000 7fa459c12740 1 Processor -- start 2024-04-02T14:02:50.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.678+0000 7fa445ffb640 1 --2- 172.21.15.52:0/1520414763 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa4380a3d20 0x7fa4380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:50.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.678+0000 7fa459c12740 1 -- start start 2024-04-02T14:02:50.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.678+0000 7fa454dc6640 1 --2- 172.21.15.52:0/1520414763 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa4380a3d20 0x7fa4380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:50.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.678+0000 7fa445ffb640 1 -- 172.21.15.52:0/1520414763 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7fa4400d0c90 con 0x7fa450101710 2024-04-02T14:02:50.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.678+0000 7fa459c12740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f5ffc8d0 0x55a0f5ffccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:50.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.678+0000 7fa459c12740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a0f5ffd1f0 0x55a0f6005890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:50.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.678+0000 7fa459c12740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f6005dd0 0x55a0f60081c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:50.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.678+0000 7fa459c12740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a0f5f60830 con 0x55a0f6005dd0 2024-04-02T14:02:50.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.678+0000 7fa459c12740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a0f5f08820 con 0x55a0f5ffd1f0 2024-04-02T14:02:50.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.678+0000 7fa459c12740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a0f5f5c7e0 con 0x55a0f5ffc8d0 2024-04-02T14:02:50.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.679+0000 7fa4555c7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f6005dd0 0x55a0f60081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:50.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.679+0000 7fa4555c7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f6005dd0 0x55a0f60081c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45632/0 (socket says 172.21.15.52:45632) 2024-04-02T14:02:50.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.679+0000 7fa4555c7640 1 -- 172.21.15.52:0/2400398162 learned_addr learned my addr 172.21.15.52:0/2400398162 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:50.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.679+0000 7fa454dc6640 1 --2- 172.21.15.52:0/1520414763 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa4380a3d20 0x7fa4380a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa44c005d00 tx=0x7fa44c00b410 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:50.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.679+0000 7fa454dc6640 1 --2- 172.21.15.52:0/2400398162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f5ffc8d0 0x55a0f5ffccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:50.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.679+0000 7fa447fff640 1 --2- 172.21.15.52:0/2400398162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a0f5ffd1f0 0x55a0f6005890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:50.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.679+0000 7fa4555c7640 1 -- 172.21.15.52:0/2400398162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f5ffc8d0 msgr2=0x55a0f5ffccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:50.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.679+0000 7fa4555c7640 1 --2- 172.21.15.52:0/2400398162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f5ffc8d0 0x55a0f5ffccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.679+0000 7fa4555c7640 1 -- 172.21.15.52:0/2400398162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a0f5ffd1f0 msgr2=0x55a0f6005890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:50.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.679+0000 7fa4555c7640 1 --2- 172.21.15.52:0/2400398162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a0f5ffd1f0 0x55a0f6005890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.679+0000 7fa4555c7640 1 -- 172.21.15.52:0/2400398162 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0f5d3e7d0 con 0x55a0f6005dd0 2024-04-02T14:02:50.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.679+0000 7fa447fff640 1 --2- 172.21.15.52:0/2400398162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a0f5ffd1f0 0x55a0f6005890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:02:50.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.679+0000 7fa4555c7640 1 --2- 172.21.15.52:0/2400398162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f6005dd0 0x55a0f60081c0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fa448004af0 tx=0x7fa4480323d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c60be9375a5539 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:50.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.679+0000 7fa436ffd640 1 -- 172.21.15.52:0/2400398162 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa448039050 con 0x55a0f6005dd0 2024-04-02T14:02:50.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.679+0000 7fa436ffd640 1 -- 172.21.15.52:0/2400398162 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa448045070 con 0x55a0f6005dd0 2024-04-02T14:02:50.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.679+0000 7fa436ffd640 1 -- 172.21.15.52:0/2400398162 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa448004040 con 0x55a0f6005dd0 2024-04-02T14:02:50.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.680+0000 7fa459c12740 1 -- 172.21.15.52:0/2400398162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f6005dd0 msgr2=0x55a0f60081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:50.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.680+0000 7fa459c12740 1 --2- 172.21.15.52:0/2400398162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f6005dd0 0x55a0f60081c0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fa448004af0 tx=0x7fa4480323d0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.680+0000 7fa459c12740 1 -- 172.21.15.52:0/2400398162 shutdown_connections 2024-04-02T14:02:50.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.680+0000 7fa459c12740 1 --2- 172.21.15.52:0/2400398162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f6005dd0 0x55a0f60081c0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.680+0000 7fa459c12740 1 --2- 172.21.15.52:0/2400398162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a0f5ffd1f0 0x55a0f6005890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.680+0000 7fa459c12740 1 --2- 172.21.15.52:0/2400398162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f5ffc8d0 0x55a0f5ffccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.680+0000 7fa459c12740 1 -- 172.21.15.52:0/2400398162 >> 172.21.15.52:0/2400398162 conn(0x55a0f5ff7000 msgr2=0x55a0f5ff73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:50.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.680+0000 7fa459c12740 1 -- 172.21.15.52:0/2400398162 shutdown_connections 2024-04-02T14:02:50.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.680+0000 7fa459c12740 1 -- 172.21.15.52:0/2400398162 wait complete. 2024-04-02T14:02:50.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.680+0000 7fa459c12740 1 -- start start 2024-04-02T14:02:50.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.681+0000 7fa459c12740 1 Processor -- start 2024-04-02T14:02:50.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.681+0000 7fa459c12740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f5ffc8d0 0x55a0f6005460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:50.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.681+0000 7fa459c12740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a0f5ffd1f0 0x55a0f6002a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:50.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.681+0000 7fa459c12740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f6005dd0 0x55a0f6002f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:50.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.681+0000 7fa459c12740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a0f6008b60 con 0x55a0f5ffc8d0 2024-04-02T14:02:50.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.681+0000 7fa459c12740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a0f5f1bd30 con 0x55a0f5ffd1f0 2024-04-02T14:02:50.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.681+0000 7fa459c12740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a0f6008ce0 con 0x55a0f6005dd0 2024-04-02T14:02:50.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.681+0000 7fa459c12740 10 client.0 _ll_register_callbacks cb 0x55a0f5f1b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:50.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.681+0000 7fa454dc6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f5ffc8d0 0x55a0f6005460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:50.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.681+0000 7fa454dc6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f5ffc8d0 0x55a0f6005460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45642/0 (socket says 172.21.15.52:45642) 2024-04-02T14:02:50.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.681+0000 7fa447fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a0f5ffd1f0 0x55a0f6002a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:50.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.681+0000 7fa454dc6640 1 -- 172.21.15.52:0/1131963761 learned_addr learned my addr 172.21.15.52:0/1131963761 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:50.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.681+0000 7fa4555c7640 1 --2- 172.21.15.52:0/1131963761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f6005dd0 0x55a0f6002f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:50.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.681+0000 7fa454dc6640 1 -- 172.21.15.52:0/1131963761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f6005dd0 msgr2=0x55a0f6002f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:50.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.681+0000 7fa454dc6640 1 --2- 172.21.15.52:0/1131963761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f6005dd0 0x55a0f6002f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.681+0000 7fa454dc6640 1 -- 172.21.15.52:0/1131963761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a0f5ffd1f0 msgr2=0x55a0f6002a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:50.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.681+0000 7fa454dc6640 1 --2- 172.21.15.52:0/1131963761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a0f5ffd1f0 0x55a0f6002a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.681+0000 7fa454dc6640 1 -- 172.21.15.52:0/1131963761 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0f5c743f0 con 0x55a0f5ffc8d0 2024-04-02T14:02:50.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.681+0000 7fa447fff640 1 --2- 172.21.15.52:0/1131963761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a0f5ffd1f0 0x55a0f6002a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:02:50.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.681+0000 7fa4555c7640 1 --2- 172.21.15.52:0/1131963761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f6005dd0 0x55a0f6002f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:02:50.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.682+0000 7fa454dc6640 1 --2- 172.21.15.52:0/1131963761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f5ffc8d0 0x55a0f6005460 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fa44c011770 tx=0x7fa44c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f240a4c39b0375a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:50.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.682+0000 7fa42ffff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:50.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.682+0000 7fa42ffff640 1 -- 172.21.15.52:0/1131963761 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44c01fb40 con 0x55a0f5ffc8d0 2024-04-02T14:02:50.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.682+0000 7fa42ffff640 1 -- 172.21.15.52:0/1131963761 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7fa44c01fce0 con 0x55a0f5ffc8d0 2024-04-02T14:02:50.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.682+0000 7fa459c12740 1 -- 172.21.15.52:0/1131963761 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa448045070 con 0x55a0f5ffc8d0 2024-04-02T14:02:50.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.682+0000 7fa42ffff640 1 -- 172.21.15.52:0/1131963761 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa44c028440 con 0x55a0f5ffc8d0 2024-04-02T14:02:50.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.682+0000 7fa459c12740 10 client.18204 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:50.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.682+0000 7fa459c12740 1 -- 172.21.15.52:0/1131963761 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a0f5c340f0 con 0x55a0f5ffc8d0 2024-04-02T14:02:50.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.682+0000 7fa42ffff640 1 -- 172.21.15.52:0/1131963761 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa448045070 con 0x55a0f5ffc8d0 2024-04-02T14:02:50.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.682+0000 7fa42ffff640 1 -- 172.21.15.52:0/1131963761 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa44c026070 con 0x55a0f5ffc8d0 2024-04-02T14:02:50.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.682+0000 7fa459c12740 10 client.18204 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:50.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.682+0000 7fa459c12740 10 client.18204 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:02:50.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.682+0000 7fa459c12740 20 client.18204 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:50.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.683+0000 7fa459c12740 1 --2- 172.21.15.52:0/1131963761 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55a0f601e5c0 0x55a0f6020a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:50.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.683+0000 7fa459c12740 4 client.18204 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:02:50.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.683+0000 7fa459c12740 1 -- 172.21.15.52:0/1131963761 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0f5f6dd70 con 0x55a0f601e5c0 2024-04-02T14:02:50.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.683+0000 7fa447fff640 1 --2- 172.21.15.52:0/1131963761 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55a0f601e5c0 0x55a0f6020a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:50.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.683+0000 7fa447fff640 1 --2- 172.21.15.52:0/1131963761 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55a0f601e5c0 0x55a0f6020a50 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=731c6f28531b9987 server_cookie=1279de28b3e945cf in_seq=0 out_seq=0 2024-04-02T14:02:50.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.683+0000 7fa42ffff640 10 client.18204 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:50.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.684+0000 7fa42ffff640 1 -- 172.21.15.52:0/1131963761 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a0f5f6dd70 con 0x55a0f601e5c0 2024-04-02T14:02:50.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.684+0000 7fa42ffff640 10 client.18204 handle_command_reply: tid=0 2024-04-02T14:02:50.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.684+0000 7fa459c12740 1 client.18204 shutdown 2024-04-02T14:02:50.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.684+0000 7fa459c12740 1 -- 172.21.15.52:0/1131963761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f5ffc8d0 msgr2=0x55a0f6005460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:50.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.684+0000 7fa459c12740 1 --2- 172.21.15.52:0/1131963761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f5ffc8d0 0x55a0f6005460 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fa44c011770 tx=0x7fa44c004730 comp rx=0 tx=0).stop 2024-04-02T14:02:50.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.684+0000 7fa459c12740 1 -- 172.21.15.52:0/1131963761 shutdown_connections 2024-04-02T14:02:50.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.684+0000 7fa459c12740 1 --2- 172.21.15.52:0/1131963761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f6005dd0 0x55a0f6002f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.684+0000 7fa459c12740 1 --2- 172.21.15.52:0/1131963761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a0f5ffd1f0 0x55a0f6002a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.684+0000 7fa459c12740 1 --2- 172.21.15.52:0/1131963761 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55a0f601e5c0 0x55a0f6020a50 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.684+0000 7fa459c12740 1 --2- 172.21.15.52:0/1131963761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f5ffc8d0 0x55a0f6005460 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.684+0000 7fa42ffff640 0 client.18204 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:50.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.684+0000 7fa459c12740 1 -- 172.21.15.52:0/1131963761 >> 172.21.15.52:0/1131963761 conn(0x55a0f5ff7000 msgr2=0x55a0f5ff8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:50.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.684+0000 7fa459c12740 1 -- 172.21.15.52:0/1131963761 shutdown_connections 2024-04-02T14:02:50.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.684+0000 7fa459c12740 1 -- 172.21.15.52:0/1131963761 wait complete. 2024-04-02T14:02:50.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.684+0000 7fa459c12740 20 client.18204 trim_cache size 0 max 16384 2024-04-02T14:02:50.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.702+0000 7fa459c12740 1 Processor -- start 2024-04-02T14:02:50.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.702+0000 7fa459c12740 1 -- start start 2024-04-02T14:02:50.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.702+0000 7fa459c12740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a0f5ffc8d0 0x55a0f6005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:50.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.702+0000 7fa459c12740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f5ffd1f0 0x55a0f6026370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:50.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.702+0000 7fa459c12740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f6005dd0 0x55a0f60268b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:50.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.702+0000 7fa459c12740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa4501acbe0 con 0x55a0f5ffd1f0 2024-04-02T14:02:50.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.702+0000 7fa459c12740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa4501ae720 con 0x55a0f5ffc8d0 2024-04-02T14:02:50.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.702+0000 7fa459c12740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa4501ad2b0 con 0x55a0f6005dd0 2024-04-02T14:02:50.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.702+0000 7fa447fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f5ffd1f0 0x55a0f6026370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:50.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.702+0000 7fa447fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f5ffd1f0 0x55a0f6026370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45654/0 (socket says 172.21.15.52:45654) 2024-04-02T14:02:50.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.702+0000 7fa447fff640 1 -- 172.21.15.52:0/1691043232 learned_addr learned my addr 172.21.15.52:0/1691043232 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:50.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.702+0000 7fa4555c7640 1 --2- 172.21.15.52:0/1691043232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f6005dd0 0x55a0f60268b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:50.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.702+0000 7fa447fff640 1 -- 172.21.15.52:0/1691043232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f6005dd0 msgr2=0x55a0f60268b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:50.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.702+0000 7fa447fff640 1 --2- 172.21.15.52:0/1691043232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f6005dd0 0x55a0f60268b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.702+0000 7fa447fff640 1 -- 172.21.15.52:0/1691043232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a0f5ffc8d0 msgr2=0x55a0f6005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:50.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.703+0000 7fa454dc6640 1 --2- 172.21.15.52:0/1691043232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a0f5ffc8d0 0x55a0f6005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:50.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.703+0000 7fa447fff640 1 --2- 172.21.15.52:0/1691043232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a0f5ffc8d0 0x55a0f6005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.703+0000 7fa447fff640 1 -- 172.21.15.52:0/1691043232 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0f5ff7bd0 con 0x55a0f5ffd1f0 2024-04-02T14:02:50.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.703+0000 7fa447fff640 1 --2- 172.21.15.52:0/1691043232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f5ffd1f0 0x55a0f6026370 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fa440007590 tx=0x7fa440087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67bae4339ece0408 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:50.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.703+0000 7fa4367fc640 1 -- 172.21.15.52:0/1691043232 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44009ac60 con 0x55a0f5ffd1f0 2024-04-02T14:02:50.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.703+0000 7fa4367fc640 1 -- 172.21.15.52:0/1691043232 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa44000fb30 con 0x55a0f5ffd1f0 2024-04-02T14:02:50.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.703+0000 7fa4367fc640 1 -- 172.21.15.52:0/1691043232 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44009de30 con 0x55a0f5ffd1f0 2024-04-02T14:02:50.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.703+0000 7fa459c12740 1 -- 172.21.15.52:0/1691043232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f5ffd1f0 msgr2=0x55a0f6026370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:50.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.703+0000 7fa459c12740 1 --2- 172.21.15.52:0/1691043232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f5ffd1f0 0x55a0f6026370 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fa440007590 tx=0x7fa440087040 comp rx=0 tx=0).stop 2024-04-02T14:02:50.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.703+0000 7fa459c12740 1 -- 172.21.15.52:0/1691043232 shutdown_connections 2024-04-02T14:02:50.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.703+0000 7fa459c12740 1 --2- 172.21.15.52:0/1691043232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f6005dd0 0x55a0f60268b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.703+0000 7fa459c12740 1 --2- 172.21.15.52:0/1691043232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f5ffd1f0 0x55a0f6026370 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.703+0000 7fa459c12740 1 --2- 172.21.15.52:0/1691043232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a0f5ffc8d0 0x55a0f6005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.703+0000 7fa459c12740 1 -- 172.21.15.52:0/1691043232 >> 172.21.15.52:0/1691043232 conn(0x55a0f5ff7000 msgr2=0x55a0f60022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:50.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.704+0000 7fa459c12740 1 -- 172.21.15.52:0/1691043232 shutdown_connections 2024-04-02T14:02:50.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.704+0000 7fa459c12740 1 -- 172.21.15.52:0/1691043232 wait complete. 2024-04-02T14:02:50.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.704+0000 7fa459c12740 1 -- start start 2024-04-02T14:02:50.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.704+0000 7fa459c12740 1 Processor -- start 2024-04-02T14:02:50.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.704+0000 7fa459c12740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a0f5ffc8d0 0x55a0f601b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:50.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.704+0000 7fa459c12740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f5ffd1f0 0x55a0f60166a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:50.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.704+0000 7fa459c12740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f6005dd0 0x55a0f6016be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:50.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.704+0000 7fa459c12740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa4501ae050 con 0x55a0f5ffd1f0 2024-04-02T14:02:50.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.704+0000 7fa459c12740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa4501ad980 con 0x55a0f5ffc8d0 2024-04-02T14:02:50.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.704+0000 7fa459c12740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa4501aabe0 con 0x55a0f6005dd0 2024-04-02T14:02:50.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.704+0000 7fa459c12740 10 client.0 _ll_register_callbacks cb 0x7fa44c011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:50.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.704+0000 7fa447fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f5ffd1f0 0x55a0f60166a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:50.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.704+0000 7fa447fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f5ffd1f0 0x55a0f60166a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45668/0 (socket says 172.21.15.52:45668) 2024-04-02T14:02:50.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.704+0000 7fa447fff640 1 -- 172.21.15.52:0/703977990 learned_addr learned my addr 172.21.15.52:0/703977990 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:50.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.704+0000 7fa4555c7640 1 --2- 172.21.15.52:0/703977990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f6005dd0 0x55a0f6016be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:50.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.704+0000 7fa454dc6640 1 --2- 172.21.15.52:0/703977990 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a0f5ffc8d0 0x55a0f601b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:50.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.705+0000 7fa447fff640 1 -- 172.21.15.52:0/703977990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f6005dd0 msgr2=0x55a0f6016be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:50.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.705+0000 7fa447fff640 1 --2- 172.21.15.52:0/703977990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f6005dd0 0x55a0f6016be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.705+0000 7fa447fff640 1 -- 172.21.15.52:0/703977990 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a0f5ffc8d0 msgr2=0x55a0f601b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:50.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.705+0000 7fa447fff640 1 --2- 172.21.15.52:0/703977990 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a0f5ffc8d0 0x55a0f601b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.705+0000 7fa447fff640 1 -- 172.21.15.52:0/703977990 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0f6017410 con 0x55a0f5ffd1f0 2024-04-02T14:02:50.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.705+0000 7fa447fff640 1 --2- 172.21.15.52:0/703977990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f5ffd1f0 0x55a0f60166a0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fa44000cfd0 tx=0x7fa44009ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852b62abb20b1f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:50.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.705+0000 7fa42ffff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:50.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.705+0000 7fa42ffff640 1 -- 172.21.15.52:0/703977990 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44009dbd0 con 0x55a0f5ffd1f0 2024-04-02T14:02:50.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.705+0000 7fa42ffff640 1 -- 172.21.15.52:0/703977990 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7fa44009dd70 con 0x55a0f5ffd1f0 2024-04-02T14:02:50.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.705+0000 7fa459c12740 1 -- 172.21.15.52:0/703977990 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa44000fb30 con 0x55a0f5ffd1f0 2024-04-02T14:02:50.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.705+0000 7fa42ffff640 1 -- 172.21.15.52:0/703977990 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa4400a5080 con 0x55a0f5ffd1f0 2024-04-02T14:02:50.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.705+0000 7fa459c12740 10 client.18210 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:50.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.705+0000 7fa459c12740 1 -- 172.21.15.52:0/703977990 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a0f6084be0 con 0x55a0f5ffd1f0 2024-04-02T14:02:50.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.705+0000 7fa42ffff640 1 -- 172.21.15.52:0/703977990 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa44000fb30 con 0x55a0f5ffd1f0 2024-04-02T14:02:50.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.706+0000 7fa42ffff640 1 -- 172.21.15.52:0/703977990 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa44009bcf0 con 0x55a0f5ffd1f0 2024-04-02T14:02:50.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.706+0000 7fa459c12740 10 client.18210 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:50.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.706+0000 7fa459c12740 10 client.18210 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:02:50.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.706+0000 7fa459c12740 20 client.18210 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:50.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.706+0000 7fa459c12740 1 --2- 172.21.15.52:0/703977990 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55a0f6085240 0x55a0f6087630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:50.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.706+0000 7fa459c12740 4 client.18210 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:02:50.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.706+0000 7fa459c12740 1 -- 172.21.15.52:0/703977990 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a0f601ba10 con 0x55a0f6085240 2024-04-02T14:02:50.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.706+0000 7fa454dc6640 1 --2- 172.21.15.52:0/703977990 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55a0f6085240 0x55a0f6087630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:50.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.706+0000 7fa42ffff640 10 client.18210 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:50.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.706+0000 7fa454dc6640 1 --2- 172.21.15.52:0/703977990 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55a0f6085240 0x55a0f6087630 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc950426043617bc server_cookie=7d5c68e286658bc in_seq=0 out_seq=0 2024-04-02T14:02:50.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.707+0000 7fa42ffff640 1 -- 172.21.15.52:0/703977990 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a0f601ba10 con 0x55a0f6085240 2024-04-02T14:02:50.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.707+0000 7fa42ffff640 10 client.18210 handle_command_reply: tid=0 2024-04-02T14:02:50.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.707+0000 7fa459c12740 1 client.18210 shutdown 2024-04-02T14:02:50.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.707+0000 7fa459c12740 1 -- 172.21.15.52:0/703977990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f5ffd1f0 msgr2=0x55a0f60166a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:50.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.707+0000 7fa459c12740 1 --2- 172.21.15.52:0/703977990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f5ffd1f0 0x55a0f60166a0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fa44000cfd0 tx=0x7fa44009ac60 comp rx=0 tx=0).stop 2024-04-02T14:02:50.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.707+0000 7fa459c12740 1 -- 172.21.15.52:0/703977990 shutdown_connections 2024-04-02T14:02:50.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.707+0000 7fa459c12740 1 --2- 172.21.15.52:0/703977990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0f6005dd0 0x55a0f6016be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.707+0000 7fa459c12740 1 --2- 172.21.15.52:0/703977990 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55a0f6085240 0x55a0f6087630 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.707+0000 7fa459c12740 1 --2- 172.21.15.52:0/703977990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a0f5ffd1f0 0x55a0f60166a0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.707+0000 7fa42ffff640 0 client.18210 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:50.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.707+0000 7fa459c12740 1 --2- 172.21.15.52:0/703977990 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a0f5ffc8d0 0x55a0f601b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.707+0000 7fa459c12740 1 -- 172.21.15.52:0/703977990 >> 172.21.15.52:0/703977990 conn(0x55a0f5ff7000 msgr2=0x55a0f6002b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:50.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.707+0000 7fa459c12740 1 -- 172.21.15.52:0/703977990 shutdown_connections 2024-04-02T14:02:50.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.707+0000 7fa459c12740 1 -- 172.21.15.52:0/703977990 wait complete. 2024-04-02T14:02:50.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.707+0000 7fa459c12740 20 client.18210 trim_cache size 0 max 16384 2024-04-02T14:02:50.724 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:02:50.724 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-04-02T14:02:50.724 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-04-02T14:02:50.724 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:02:50.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.708+0000 7fa45702a640 1 -- 172.21.15.52:0/1520414763 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa4380a3d20 msgr2=0x7fa4380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:50.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.708+0000 7fa45702a640 1 --2- 172.21.15.52:0/1520414763 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa4380a3d20 0x7fa4380a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa44c005d00 tx=0x7fa44c00b410 comp rx=0 tx=0).stop 2024-04-02T14:02:50.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.708+0000 7fa45702a640 1 -- 172.21.15.52:0/1520414763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa450101710 msgr2=0x7fa4501996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:50.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.708+0000 7fa45702a640 1 --2- 172.21.15.52:0/1520414763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa450101710 0x7fa4501996f0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fa44000c920 tx=0x7fa44000cde0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.708+0000 7fa45702a640 1 -- 172.21.15.52:0/1520414763 shutdown_connections 2024-04-02T14:02:50.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.708+0000 7fa45702a640 1 --2- 172.21.15.52:0/1520414763 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fa4380a3d20 0x7fa4380a61e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.708+0000 7fa45702a640 1 --2- 172.21.15.52:0/1520414763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4501059e0 0x7fa4501a4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.708+0000 7fa45702a640 1 --2- 172.21.15.52:0/1520414763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa450101710 0x7fa4501996f0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.708+0000 7fa45702a640 1 --2- 172.21.15.52:0/1520414763 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa450100d50 0x7fa4501991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.708+0000 7fa45702a640 1 -- 172.21.15.52:0/1520414763 >> 172.21.15.52:0/1520414763 conn(0x7fa450076800 msgr2=0x7fa4500fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:50.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.708+0000 7fa45702a640 1 -- 172.21.15.52:0/1520414763 shutdown_connections 2024-04-02T14:02:50.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.708+0000 7fa45702a640 1 -- 172.21.15.52:0/1520414763 wait complete. 2024-04-02T14:02:50.728 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T14:02:50.728 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:02:50.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.827+0000 7f6f96e0d640 1 -- 172.21.15.52:0/3245479091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f90100a80 msgr2=0x7f6f90100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:50.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.827+0000 7f6f96e0d640 1 --2- 172.21.15.52:0/3245479091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f90100a80 0x7f6f90100f00 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009930 tx=0x7f6f7c02bc30 comp rx=0 tx=0).stop 2024-04-02T14:02:50.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.827+0000 7f6f96e0d640 1 -- 172.21.15.52:0/3245479091 shutdown_connections 2024-04-02T14:02:50.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.827+0000 7f6f96e0d640 1 --2- 172.21.15.52:0/3245479091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f90101440 0x7f6f9010d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.827+0000 7f6f96e0d640 1 --2- 172.21.15.52:0/3245479091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f90100a80 0x7f6f90100f00 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.827+0000 7f6f96e0d640 1 --2- 172.21.15.52:0/3245479091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f90106a40 0x7f6f90106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.827+0000 7f6f96e0d640 1 -- 172.21.15.52:0/3245479091 >> 172.21.15.52:0/3245479091 conn(0x7f6f90077b60 msgr2=0x7f6f90077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:50.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.827+0000 7f6f96e0d640 1 -- 172.21.15.52:0/3245479091 shutdown_connections 2024-04-02T14:02:50.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.827+0000 7f6f96e0d640 1 -- 172.21.15.52:0/3245479091 wait complete. 2024-04-02T14:02:50.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.828+0000 7f6f96e0d640 1 Processor -- start 2024-04-02T14:02:50.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.828+0000 7f6f96e0d640 1 -- start start 2024-04-02T14:02:50.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.828+0000 7f6f96e0d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f90100a80 0x7f6f90198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:50.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.828+0000 7f6f96e0d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f90101440 0x7f6f90198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:50.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.828+0000 7f6f96e0d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f90106a40 0x7f6f9019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:50.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.828+0000 7f6f96e0d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6f90110e50 con 0x7f6f90100a80 2024-04-02T14:02:50.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.828+0000 7f6f96e0d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6f90110cd0 con 0x7f6f90106a40 2024-04-02T14:02:50.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.828+0000 7f6f96e0d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6f90110fd0 con 0x7f6f90101440 2024-04-02T14:02:50.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.828+0000 7f6f94ba9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f90100a80 0x7f6f90198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:50.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.828+0000 7f6f94ba9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f90100a80 0x7f6f90198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45692/0 (socket says 172.21.15.52:45692) 2024-04-02T14:02:50.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.828+0000 7f6f94ba9640 1 -- 172.21.15.52:0/4220841741 learned_addr learned my addr 172.21.15.52:0/4220841741 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:50.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.829+0000 7f6f8ffff640 1 --2- 172.21.15.52:0/4220841741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f90101440 0x7f6f90198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:50.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.829+0000 7f6f953aa640 1 --2- 172.21.15.52:0/4220841741 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f90106a40 0x7f6f9019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:50.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.829+0000 7f6f94ba9640 1 -- 172.21.15.52:0/4220841741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f90101440 msgr2=0x7f6f90198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:50.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.829+0000 7f6f94ba9640 1 --2- 172.21.15.52:0/4220841741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f90101440 0x7f6f90198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.829+0000 7f6f94ba9640 1 -- 172.21.15.52:0/4220841741 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f90106a40 msgr2=0x7f6f9019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:50.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.829+0000 7f6f94ba9640 1 --2- 172.21.15.52:0/4220841741 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f90106a40 0x7f6f9019ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:50.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.829+0000 7f6f94ba9640 1 -- 172.21.15.52:0/4220841741 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f9019d3f0 con 0x7f6f90100a80 2024-04-02T14:02:50.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.829+0000 7f6f8ffff640 1 --2- 172.21.15.52:0/4220841741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f90101440 0x7f6f90198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:02:50.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.829+0000 7f6f94ba9640 1 --2- 172.21.15.52:0/4220841741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f90100a80 0x7f6f90198400 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400c970 tx=0x7f6f8400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:50.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.829+0000 7f6f8dffb640 1 -- 172.21.15.52:0/4220841741 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f84007bb0 con 0x7f6f90100a80 2024-04-02T14:02:50.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.829+0000 7f6f8dffb640 1 -- 172.21.15.52:0/4220841741 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f84007d50 con 0x7f6f90100a80 2024-04-02T14:02:50.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.829+0000 7f6f96e0d640 1 -- 172.21.15.52:0/4220841741 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f9019d6e0 con 0x7f6f90100a80 2024-04-02T14:02:50.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.829+0000 7f6f8dffb640 1 -- 172.21.15.52:0/4220841741 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8400f450 con 0x7f6f90100a80 2024-04-02T14:02:50.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.829+0000 7f6f96e0d640 1 -- 172.21.15.52:0/4220841741 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f901a4f70 con 0x7f6f90100a80 2024-04-02T14:02:50.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.830+0000 7f6f96e0d640 1 -- 172.21.15.52:0/4220841741 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f90068d40 con 0x7f6f90100a80 2024-04-02T14:02:50.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.831+0000 7f6f8dffb640 1 -- 172.21.15.52:0/4220841741 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6f84005020 con 0x7f6f90100a80 2024-04-02T14:02:50.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.832+0000 7f6f8dffb640 1 --2- 172.21.15.52:0/4220841741 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6f800a3d20 0x7f6f800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:50.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.832+0000 7f6f8ffff640 1 --2- 172.21.15.52:0/4220841741 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6f800a3d20 0x7f6f800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:50.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.832+0000 7f6f8dffb640 1 -- 172.21.15.52:0/4220841741 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f6f840d09e0 con 0x7f6f90100a80 2024-04-02T14:02:50.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.832+0000 7f6f8ffff640 1 --2- 172.21.15.52:0/4220841741 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6f800a3d20 0x7f6f800a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c036040 tx=0x7f6f7c000bc0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:50.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:50.836+0000 7f6f8dffb640 1 -- 172.21.15.52:0/4220841741 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6f84014030 con 0x7f6f90100a80 2024-04-02T14:02:51.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.063+0000 7f6f96e0d640 1 -- 172.21.15.52:0/4220841741 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f90109fe0 con 0x7f6f90100a80 2024-04-02T14:02:51.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.063+0000 7f6f8dffb640 1 -- 172.21.15.52:0/4220841741 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11923 (secure 0 0 0) 0x7f6f8409ccf0 con 0x7f6f90100a80 2024-04-02T14:02:51.067 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T14:02:51.068 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:57:45.240902+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24467,24457,24490,24475,24503]},"id":1}]} 2024-04-02T14:02:51.068 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-04-02T14:02:51.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.065+0000 7f6f96e0d640 1 -- 172.21.15.52:0/4220841741 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6f800a3d20 msgr2=0x7f6f800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.065+0000 7f6f96e0d640 1 --2- 172.21.15.52:0/4220841741 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6f800a3d20 0x7f6f800a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c036040 tx=0x7f6f7c000bc0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.065+0000 7f6f96e0d640 1 -- 172.21.15.52:0/4220841741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f90100a80 msgr2=0x7f6f90198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.065+0000 7f6f96e0d640 1 --2- 172.21.15.52:0/4220841741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f90100a80 0x7f6f90198400 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400c970 tx=0x7f6f8400ce30 comp rx=0 tx=0).stop 2024-04-02T14:02:51.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.065+0000 7f6f96e0d640 1 -- 172.21.15.52:0/4220841741 shutdown_connections 2024-04-02T14:02:51.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.065+0000 7f6f96e0d640 1 --2- 172.21.15.52:0/4220841741 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6f800a3d20 0x7f6f800a61e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.065+0000 7f6f96e0d640 1 --2- 172.21.15.52:0/4220841741 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f90106a40 0x7f6f9019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.065+0000 7f6f96e0d640 1 --2- 172.21.15.52:0/4220841741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f90101440 0x7f6f90198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.065+0000 7f6f96e0d640 1 --2- 172.21.15.52:0/4220841741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f90100a80 0x7f6f90198400 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.065+0000 7f6f96e0d640 1 -- 172.21.15.52:0/4220841741 >> 172.21.15.52:0/4220841741 conn(0x7f6f90077b60 msgr2=0x7f6f9006a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.066+0000 7f6f96e0d640 1 -- 172.21.15.52:0/4220841741 shutdown_connections 2024-04-02T14:02:51.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.066+0000 7f6f96e0d640 1 -- 172.21.15.52:0/4220841741 wait complete. 2024-04-02T14:02:51.080 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T14:02:51.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.182+0000 7efef110e640 1 -- 172.21.15.52:0/850228010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efeec101370 msgr2=0x7efeec10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.182+0000 7efef110e640 1 --2- 172.21.15.52:0/850228010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efeec101370 0x7efeec10d880 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7efee4009cd0 tx=0x7efee402c0f0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.182+0000 7efef110e640 1 -- 172.21.15.52:0/850228010 shutdown_connections 2024-04-02T14:02:51.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.182+0000 7efef110e640 1 --2- 172.21.15.52:0/850228010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efeec101370 0x7efeec10d880 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.182+0000 7efef110e640 1 --2- 172.21.15.52:0/850228010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efeec1009b0 0x7efeec100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.182+0000 7efef110e640 1 --2- 172.21.15.52:0/850228010 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efeec1069c0 0x7efeec106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.182+0000 7efef110e640 1 -- 172.21.15.52:0/850228010 >> 172.21.15.52:0/850228010 conn(0x7efeec076d80 msgr2=0x7efeec077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.182+0000 7efef110e640 1 -- 172.21.15.52:0/850228010 shutdown_connections 2024-04-02T14:02:51.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.182+0000 7efef110e640 1 -- 172.21.15.52:0/850228010 wait complete. 2024-04-02T14:02:51.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.182+0000 7efef110e640 1 Processor -- start 2024-04-02T14:02:51.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.183+0000 7efef110e640 1 -- start start 2024-04-02T14:02:51.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.183+0000 7efef110e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efeec1009b0 0x7efeec1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.183+0000 7efef110e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efeec101370 0x7efeec198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.183+0000 7efef110e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efeec1069c0 0x7efeec19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.183+0000 7efef110e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efeec110da0 con 0x7efeec101370 2024-04-02T14:02:51.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.183+0000 7efef110e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efeec110c20 con 0x7efeec1009b0 2024-04-02T14:02:51.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.183+0000 7efef110e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efeec110f20 con 0x7efeec1069c0 2024-04-02T14:02:51.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.183+0000 7efeea59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efeec101370 0x7efeec198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.183+0000 7efeea59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efeec101370 0x7efeec198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45708/0 (socket says 172.21.15.52:45708) 2024-04-02T14:02:51.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.183+0000 7efeea59c640 1 -- 172.21.15.52:0/1315167835 learned_addr learned my addr 172.21.15.52:0/1315167835 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:51.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.183+0000 7efeead9d640 1 --2- 172.21.15.52:0/1315167835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efeec1009b0 0x7efeec1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.183+0000 7efeeb59e640 1 --2- 172.21.15.52:0/1315167835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efeec1069c0 0x7efeec19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.183+0000 7efeea59c640 1 -- 172.21.15.52:0/1315167835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efeec1069c0 msgr2=0x7efeec19cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.183+0000 7efeea59c640 1 --2- 172.21.15.52:0/1315167835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efeec1069c0 0x7efeec19cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.183+0000 7efeea59c640 1 -- 172.21.15.52:0/1315167835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efeec1009b0 msgr2=0x7efeec1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.183+0000 7efeea59c640 1 --2- 172.21.15.52:0/1315167835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efeec1009b0 0x7efeec1982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.183+0000 7efeea59c640 1 -- 172.21.15.52:0/1315167835 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeec19d2d0 con 0x7efeec101370 2024-04-02T14:02:51.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.183+0000 7efeead9d640 1 --2- 172.21.15.52:0/1315167835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efeec1009b0 0x7efeec1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:02:51.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.183+0000 7efeea59c640 1 --2- 172.21.15.52:0/1315167835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efeec101370 0x7efeec198820 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7efee000d950 tx=0x7efee000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.184+0000 7efed3fff640 1 -- 172.21.15.52:0/1315167835 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0017030 con 0x7efeec101370 2024-04-02T14:02:51.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.184+0000 7efed3fff640 1 -- 172.21.15.52:0/1315167835 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efee0004020 con 0x7efeec101370 2024-04-02T14:02:51.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.184+0000 7efef110e640 1 -- 172.21.15.52:0/1315167835 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeec19d5c0 con 0x7efeec101370 2024-04-02T14:02:51.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.184+0000 7efed3fff640 1 -- 172.21.15.52:0/1315167835 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee00047a0 con 0x7efeec101370 2024-04-02T14:02:51.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.184+0000 7efef110e640 1 -- 172.21.15.52:0/1315167835 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeec1a4e50 con 0x7efeec101370 2024-04-02T14:02:51.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.186+0000 7efed3fff640 1 -- 172.21.15.52:0/1315167835 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efee0004940 con 0x7efeec101370 2024-04-02T14:02:51.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.186+0000 7efef3dd4740 1 Processor -- start 2024-04-02T14:02:51.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.186+0000 7efef3dd4740 1 -- start start 2024-04-02T14:02:51.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.186+0000 7efef3dd4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567ab57a0 0x563567ab5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.186+0000 7efef3dd4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563567ab60c0 0x563567abe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.186+0000 7efef3dd4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563567abeca0 0x563567ac1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.186+0000 7efef3dd4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563567a197b0 con 0x563567ab57a0 2024-04-02T14:02:51.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.186+0000 7efef3dd4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5635679b8570 con 0x563567abeca0 2024-04-02T14:02:51.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.186+0000 7efef3dd4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563567a15760 con 0x563567ab60c0 2024-04-02T14:02:51.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.186+0000 7efeeb59e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567ab57a0 0x563567ab5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.186+0000 7efeeb59e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567ab57a0 0x563567ab5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45716/0 (socket says 172.21.15.52:45716) 2024-04-02T14:02:51.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.187+0000 7efeeb59e640 1 -- 172.21.15.52:0/414290854 learned_addr learned my addr 172.21.15.52:0/414290854 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:51.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.187+0000 7efeead9d640 1 --2- 172.21.15.52:0/414290854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563567ab60c0 0x563567abe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.187+0000 7efed3fff640 1 --2- 172.21.15.52:0/1315167835 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efed40a3d20 0x7efed40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.187+0000 7efeea59c640 1 --2- 172.21.15.52:0/414290854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563567abeca0 0x563567ac1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.187+0000 7efeeb59e640 1 -- 172.21.15.52:0/414290854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563567ab60c0 msgr2=0x563567abe760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.187+0000 7efeeb59e640 1 --2- 172.21.15.52:0/414290854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563567ab60c0 0x563567abe760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.187+0000 7efeeb59e640 1 -- 172.21.15.52:0/414290854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563567abeca0 msgr2=0x563567ac1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.187+0000 7efeeb59e640 1 --2- 172.21.15.52:0/414290854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563567abeca0 0x563567ac1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.187+0000 7efeeb59e640 1 -- 172.21.15.52:0/414290854 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56356772d3f0 con 0x563567ab57a0 2024-04-02T14:02:51.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.187+0000 7efeea59c640 1 --2- 172.21.15.52:0/414290854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563567abeca0 0x563567ac1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:02:51.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.187+0000 7efed3fff640 1 -- 172.21.15.52:0/1315167835 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7efee00d0660 con 0x7efeec101370 2024-04-02T14:02:51.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.187+0000 7efeeb59e640 1 --2- 172.21.15.52:0/1315167835 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efed40a3d20 0x7efed40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.187+0000 7efeeb59e640 1 --2- 172.21.15.52:0/414290854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567ab57a0 0x563567ab5b80 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7efee4005e30 tx=0x7efee4032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c8c3eb084d4eb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.187+0000 7efed17fa640 1 -- 172.21.15.52:0/414290854 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4039050 con 0x563567ab57a0 2024-04-02T14:02:51.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.187+0000 7efed17fa640 1 -- 172.21.15.52:0/414290854 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efee4046030 con 0x563567ab57a0 2024-04-02T14:02:51.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.187+0000 7efed17fa640 1 -- 172.21.15.52:0/414290854 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4034440 con 0x563567ab57a0 2024-04-02T14:02:51.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.188+0000 7efef3dd4740 1 -- 172.21.15.52:0/414290854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567ab57a0 msgr2=0x563567ab5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.188+0000 7efeeb59e640 1 --2- 172.21.15.52:0/1315167835 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efed40a3d20 0x7efed40a61e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efee4004af0 tx=0x7efee404b000 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.188+0000 7efef3dd4740 1 --2- 172.21.15.52:0/414290854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567ab57a0 0x563567ab5b80 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7efee4005e30 tx=0x7efee4032420 comp rx=0 tx=0).stop 2024-04-02T14:02:51.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.188+0000 7efef3dd4740 1 -- 172.21.15.52:0/414290854 shutdown_connections 2024-04-02T14:02:51.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.188+0000 7efef3dd4740 1 --2- 172.21.15.52:0/414290854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563567abeca0 0x563567ac1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.188+0000 7efef3dd4740 1 --2- 172.21.15.52:0/414290854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563567ab60c0 0x563567abe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.188+0000 7efef3dd4740 1 --2- 172.21.15.52:0/414290854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567ab57a0 0x563567ab5b80 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.188+0000 7efef3dd4740 1 -- 172.21.15.52:0/414290854 >> 172.21.15.52:0/414290854 conn(0x563567aafed0 msgr2=0x563567ab02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.188+0000 7efef3dd4740 1 -- 172.21.15.52:0/414290854 shutdown_connections 2024-04-02T14:02:51.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.188+0000 7efef3dd4740 1 -- 172.21.15.52:0/414290854 wait complete. 2024-04-02T14:02:51.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.188+0000 7efef3dd4740 1 -- start start 2024-04-02T14:02:51.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.188+0000 7efef3dd4740 1 Processor -- start 2024-04-02T14:02:51.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.189+0000 7efef3dd4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567ab57a0 0x563567abe660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.189+0000 7efef3dd4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563567ab60c0 0x563567abb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.189+0000 7efef3dd4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563567abeca0 0x563567abc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.189+0000 7efef3dd4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563567ac1a90 con 0x563567ab57a0 2024-04-02T14:02:51.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.189+0000 7efef3dd4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5635679d4e80 con 0x563567ab60c0 2024-04-02T14:02:51.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.189+0000 7efef3dd4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563567ac1c10 con 0x563567abeca0 2024-04-02T14:02:51.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.189+0000 7efeead9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567ab57a0 0x563567abe660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.189+0000 7efef3dd4740 10 client.0 _ll_register_callbacks cb 0x563567a10aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:51.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.189+0000 7efeead9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567ab57a0 0x563567abe660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45720/0 (socket says 172.21.15.52:45720) 2024-04-02T14:02:51.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.189+0000 7efeead9d640 1 -- 172.21.15.52:0/3098896146 learned_addr learned my addr 172.21.15.52:0/3098896146 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:51.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.189+0000 7efeeb59e640 1 --2- 172.21.15.52:0/3098896146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563567abeca0 0x563567abc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.189+0000 7efeea59c640 1 --2- 172.21.15.52:0/3098896146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563567ab60c0 0x563567abb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.189+0000 7efeead9d640 1 -- 172.21.15.52:0/3098896146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563567abeca0 msgr2=0x563567abc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.189+0000 7efeead9d640 1 --2- 172.21.15.52:0/3098896146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563567abeca0 0x563567abc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.189+0000 7efeead9d640 1 -- 172.21.15.52:0/3098896146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563567ab60c0 msgr2=0x563567abb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.189+0000 7efeead9d640 1 --2- 172.21.15.52:0/3098896146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563567ab60c0 0x563567abb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.189+0000 7efeead9d640 1 -- 172.21.15.52:0/3098896146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635677f77d0 con 0x563567ab57a0 2024-04-02T14:02:51.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.189+0000 7efeea59c640 1 --2- 172.21.15.52:0/3098896146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563567ab60c0 0x563567abb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:02:51.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.189+0000 7efeead9d640 1 --2- 172.21.15.52:0/3098896146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567ab57a0 0x563567abe660 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7efedc011a30 tx=0x7efedc011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224d6bee889ce4ed server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.189+0000 7efeceff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:51.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.189+0000 7efeceff5640 1 -- 172.21.15.52:0/3098896146 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc00fdb0 con 0x563567ab57a0 2024-04-02T14:02:51.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.189+0000 7efeceff5640 1 -- 172.21.15.52:0/3098896146 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7efedc004020 con 0x563567ab57a0 2024-04-02T14:02:51.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.189+0000 7efef3dd4740 1 -- 172.21.15.52:0/3098896146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efee4046030 con 0x563567ab57a0 2024-04-02T14:02:51.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.190+0000 7efeceff5640 1 -- 172.21.15.52:0/3098896146 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efedc024050 con 0x563567ab57a0 2024-04-02T14:02:51.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.190+0000 7efef3dd4740 10 client.18228 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:51.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.190+0000 7efef3dd4740 1 -- 172.21.15.52:0/3098896146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563567acda90 con 0x563567ab57a0 2024-04-02T14:02:51.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.190+0000 7efeceff5640 1 -- 172.21.15.52:0/3098896146 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efee4046030 con 0x563567ab57a0 2024-04-02T14:02:51.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.190+0000 7efeceff5640 1 -- 172.21.15.52:0/3098896146 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7efedc018030 con 0x563567ab57a0 2024-04-02T14:02:51.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.190+0000 7efef3dd4740 10 client.18228 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:51.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.190+0000 7efef3dd4740 10 client.18228 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T14:02:51.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.190+0000 7efef3dd4740 20 client.18228 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:51.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.190+0000 7efef3dd4740 1 --2- 172.21.15.52:0/3098896146 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x563567acefa0 0x563567ad1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.190+0000 7efef3dd4740 4 client.18228 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:02:51.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.190+0000 7efef3dd4740 1 -- 172.21.15.52:0/3098896146 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563567a26cf0 con 0x563567acefa0 2024-04-02T14:02:51.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.190+0000 7efeea59c640 1 --2- 172.21.15.52:0/3098896146 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x563567acefa0 0x563567ad1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.191+0000 7efeceff5640 10 client.18228 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T14:02:51.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.191+0000 7efeea59c640 1 --2- 172.21.15.52:0/3098896146 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x563567acefa0 0x563567ad1410 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c1c8b4973870cfe9 server_cookie=3f7d7e9e9212bb63 in_seq=0 out_seq=0 2024-04-02T14:02:51.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.191+0000 7efeceff5640 1 -- 172.21.15.52:0/3098896146 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563567a26cf0 con 0x563567acefa0 2024-04-02T14:02:51.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.191+0000 7efeceff5640 10 client.18228 handle_command_reply: tid=0 2024-04-02T14:02:51.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.191+0000 7efef3dd4740 1 client.18228 shutdown 2024-04-02T14:02:51.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.192+0000 7efef3dd4740 1 -- 172.21.15.52:0/3098896146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567ab57a0 msgr2=0x563567abe660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.192+0000 7efef3dd4740 1 --2- 172.21.15.52:0/3098896146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567ab57a0 0x563567abe660 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7efedc011a30 tx=0x7efedc011ef0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.192+0000 7efef3dd4740 1 -- 172.21.15.52:0/3098896146 shutdown_connections 2024-04-02T14:02:51.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.192+0000 7efef3dd4740 1 --2- 172.21.15.52:0/3098896146 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x563567acefa0 0x563567ad1410 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.192+0000 7efef3dd4740 1 --2- 172.21.15.52:0/3098896146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563567abeca0 0x563567abc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.192+0000 7efef3dd4740 1 --2- 172.21.15.52:0/3098896146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563567ab60c0 0x563567abb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.192+0000 7efeceff5640 0 client.18228 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T14:02:51.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.192+0000 7efef3dd4740 1 --2- 172.21.15.52:0/3098896146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567ab57a0 0x563567abe660 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.192+0000 7efef3dd4740 1 -- 172.21.15.52:0/3098896146 >> 172.21.15.52:0/3098896146 conn(0x563567aafed0 msgr2=0x563567ab1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.192+0000 7efef3dd4740 1 -- 172.21.15.52:0/3098896146 shutdown_connections 2024-04-02T14:02:51.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.192+0000 7efef3dd4740 1 -- 172.21.15.52:0/3098896146 wait complete. 2024-04-02T14:02:51.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.192+0000 7efef3dd4740 20 client.18228 trim_cache size 0 max 16384 2024-04-02T14:02:51.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.205+0000 7efef3dd4740 1 Processor -- start 2024-04-02T14:02:51.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.205+0000 7efef3dd4740 1 -- start start 2024-04-02T14:02:51.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.205+0000 7efef3dd4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563567ab57a0 0x563567aca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.205+0000 7efef3dd4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567ab60c0 0x563567abe120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.205+0000 7efef3dd4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563567abeca0 0x563567abe660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.205+0000 7efef3dd4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efeec1aee80 con 0x563567ab60c0 2024-04-02T14:02:51.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.205+0000 7efef3dd4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efeec1b09c0 con 0x563567abeca0 2024-04-02T14:02:51.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.205+0000 7efef3dd4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efeec1af550 con 0x563567ab57a0 2024-04-02T14:02:51.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.205+0000 7efeea59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567ab60c0 0x563567abe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.205+0000 7efeea59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567ab60c0 0x563567abe120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45722/0 (socket says 172.21.15.52:45722) 2024-04-02T14:02:51.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.205+0000 7efeea59c640 1 -- 172.21.15.52:0/3249565589 learned_addr learned my addr 172.21.15.52:0/3249565589 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:51.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.206+0000 7efeead9d640 1 --2- 172.21.15.52:0/3249565589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563567ab57a0 0x563567aca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.206+0000 7efeea59c640 1 -- 172.21.15.52:0/3249565589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563567ab57a0 msgr2=0x563567aca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.206+0000 7efeea59c640 1 --2- 172.21.15.52:0/3249565589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563567ab57a0 0x563567aca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.206+0000 7efeea59c640 1 -- 172.21.15.52:0/3249565589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563567abeca0 msgr2=0x563567abe660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.206+0000 7efeeb59e640 1 --2- 172.21.15.52:0/3249565589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563567abeca0 0x563567abe660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.206+0000 7efeea59c640 1 --2- 172.21.15.52:0/3249565589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563567abeca0 0x563567abe660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.206+0000 7efeea59c640 1 -- 172.21.15.52:0/3249565589 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563567ab5ef0 con 0x563567ab60c0 2024-04-02T14:02:51.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.206+0000 7efeead9d640 1 --2- 172.21.15.52:0/3249565589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563567ab57a0 0x563567aca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:02:51.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.206+0000 7efeea59c640 1 --2- 172.21.15.52:0/3249565589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567ab60c0 0x563567abe120 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7efee009c200 tx=0x7efee006dcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9648531e74b833 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.206+0000 7efed0ff9640 1 -- 172.21.15.52:0/3249565589 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee00cf070 con 0x563567ab60c0 2024-04-02T14:02:51.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.206+0000 7efed0ff9640 1 -- 172.21.15.52:0/3249565589 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efee009c270 con 0x563567ab60c0 2024-04-02T14:02:51.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.206+0000 7efed0ff9640 1 -- 172.21.15.52:0/3249565589 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee00cde30 con 0x563567ab60c0 2024-04-02T14:02:51.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.206+0000 7efef3dd4740 1 -- 172.21.15.52:0/3249565589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567ab60c0 msgr2=0x563567abe120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.206+0000 7efef3dd4740 1 --2- 172.21.15.52:0/3249565589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567ab60c0 0x563567abe120 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7efee009c200 tx=0x7efee006dcb0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.207+0000 7efef3dd4740 1 -- 172.21.15.52:0/3249565589 shutdown_connections 2024-04-02T14:02:51.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.207+0000 7efef3dd4740 1 --2- 172.21.15.52:0/3249565589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563567abeca0 0x563567abe660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.207+0000 7efef3dd4740 1 --2- 172.21.15.52:0/3249565589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567ab60c0 0x563567abe120 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.207+0000 7efef3dd4740 1 --2- 172.21.15.52:0/3249565589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563567ab57a0 0x563567aca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.207+0000 7efef3dd4740 1 -- 172.21.15.52:0/3249565589 >> 172.21.15.52:0/3249565589 conn(0x563567aafed0 msgr2=0x563567abaa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.207+0000 7efef3dd4740 1 -- 172.21.15.52:0/3249565589 shutdown_connections 2024-04-02T14:02:51.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.207+0000 7efef3dd4740 1 -- 172.21.15.52:0/3249565589 wait complete. 2024-04-02T14:02:51.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.207+0000 7efef3dd4740 1 -- start start 2024-04-02T14:02:51.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.207+0000 7efef3dd4740 1 Processor -- start 2024-04-02T14:02:51.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.207+0000 7efef3dd4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563567ab57a0 0x563567ac7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.207+0000 7efef3dd4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563567ab60c0 0x563567abfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.207+0000 7efef3dd4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567abeca0 0x563567ac0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.207+0000 7efef3dd4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efeec1b02f0 con 0x563567abeca0 2024-04-02T14:02:51.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.207+0000 7efef3dd4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efeec1afc20 con 0x563567ab57a0 2024-04-02T14:02:51.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.207+0000 7efef3dd4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efeec1ace80 con 0x563567ab60c0 2024-04-02T14:02:51.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.207+0000 7efef3dd4740 10 client.0 _ll_register_callbacks cb 0x7efedc008760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:51.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.207+0000 7efeeb59e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567abeca0 0x563567ac0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.207+0000 7efeeb59e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567abeca0 0x563567ac0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45728/0 (socket says 172.21.15.52:45728) 2024-04-02T14:02:51.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.207+0000 7efeeb59e640 1 -- 172.21.15.52:0/2918488868 learned_addr learned my addr 172.21.15.52:0/2918488868 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:51.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.207+0000 7efeea59c640 1 --2- 172.21.15.52:0/2918488868 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563567ab60c0 0x563567abfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.207+0000 7efeead9d640 1 --2- 172.21.15.52:0/2918488868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563567ab57a0 0x563567ac7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.207+0000 7efeeb59e640 1 -- 172.21.15.52:0/2918488868 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563567ab60c0 msgr2=0x563567abfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.207+0000 7efeeb59e640 1 --2- 172.21.15.52:0/2918488868 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563567ab60c0 0x563567abfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.208+0000 7efeeb59e640 1 -- 172.21.15.52:0/2918488868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563567ab57a0 msgr2=0x563567ac7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.208+0000 7efeeb59e640 1 --2- 172.21.15.52:0/2918488868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563567ab57a0 0x563567ac7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.208+0000 7efeeb59e640 1 -- 172.21.15.52:0/2918488868 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563567ac4f20 con 0x563567abeca0 2024-04-02T14:02:51.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.208+0000 7efeeb59e640 1 --2- 172.21.15.52:0/2918488868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567abeca0 0x563567ac0330 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7efee4033000 tx=0x7efee40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd8d658425d4f56d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.208+0000 7efeceff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:51.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.208+0000 7efeceff5640 1 -- 172.21.15.52:0/2918488868 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4051040 con 0x563567abeca0 2024-04-02T14:02:51.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.208+0000 7efeceff5640 1 -- 172.21.15.52:0/2918488868 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7efee4009800 con 0x563567abeca0 2024-04-02T14:02:51.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.208+0000 7efef3dd4740 1 -- 172.21.15.52:0/2918488868 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efee009c270 con 0x563567abeca0 2024-04-02T14:02:51.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.208+0000 7efeceff5640 1 -- 172.21.15.52:0/2918488868 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efee4064050 con 0x563567abeca0 2024-04-02T14:02:51.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.208+0000 7efef3dd4740 10 client.18234 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:51.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.208+0000 7efef3dd4740 1 -- 172.21.15.52:0/2918488868 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563567b35640 con 0x563567abeca0 2024-04-02T14:02:51.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.208+0000 7efeceff5640 1 -- 172.21.15.52:0/2918488868 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efee009c270 con 0x563567abeca0 2024-04-02T14:02:51.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.208+0000 7efeceff5640 1 -- 172.21.15.52:0/2918488868 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7efee4054030 con 0x563567abeca0 2024-04-02T14:02:51.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.208+0000 7efef3dd4740 10 client.18234 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:51.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.208+0000 7efef3dd4740 10 client.18234 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T14:02:51.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.208+0000 7efef3dd4740 20 client.18234 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:51.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.208+0000 7efef3dd4740 1 --2- 172.21.15.52:0/2918488868 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x563567b35940 0x563567b37d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.208+0000 7efef3dd4740 4 client.18234 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:02:51.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.208+0000 7efef3dd4740 1 -- 172.21.15.52:0/2918488868 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563567a21040 con 0x563567b35940 2024-04-02T14:02:51.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.209+0000 7efeead9d640 1 --2- 172.21.15.52:0/2918488868 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x563567b35940 0x563567b37d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.209+0000 7efeead9d640 1 --2- 172.21.15.52:0/2918488868 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x563567b35940 0x563567b37d60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2a4531f8a09349fd server_cookie=56cd0fbd0655a344 in_seq=0 out_seq=0 2024-04-02T14:02:51.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.209+0000 7efeceff5640 10 client.18234 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T14:02:51.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.209+0000 7efeceff5640 1 -- 172.21.15.52:0/2918488868 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563567a21040 con 0x563567b35940 2024-04-02T14:02:51.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.209+0000 7efeceff5640 10 client.18234 handle_command_reply: tid=0 2024-04-02T14:02:51.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.209+0000 7efef3dd4740 1 client.18234 shutdown 2024-04-02T14:02:51.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.209+0000 7efef3dd4740 1 -- 172.21.15.52:0/2918488868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567abeca0 msgr2=0x563567ac0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.209+0000 7efef3dd4740 1 --2- 172.21.15.52:0/2918488868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567abeca0 0x563567ac0330 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7efee4033000 tx=0x7efee40095c0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.209+0000 7efef3dd4740 1 -- 172.21.15.52:0/2918488868 shutdown_connections 2024-04-02T14:02:51.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.209+0000 7efef3dd4740 1 --2- 172.21.15.52:0/2918488868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563567abeca0 0x563567ac0330 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.209+0000 7efef3dd4740 1 --2- 172.21.15.52:0/2918488868 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x563567b35940 0x563567b37d60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.209+0000 7efef3dd4740 1 --2- 172.21.15.52:0/2918488868 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563567ab60c0 0x563567abfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.209+0000 7efef3dd4740 1 --2- 172.21.15.52:0/2918488868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563567ab57a0 0x563567ac7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.209+0000 7efeceff5640 0 client.18234 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T14:02:51.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.209+0000 7efef3dd4740 1 -- 172.21.15.52:0/2918488868 >> 172.21.15.52:0/2918488868 conn(0x563567aafed0 msgr2=0x563567ab11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.209+0000 7efef3dd4740 1 -- 172.21.15.52:0/2918488868 shutdown_connections 2024-04-02T14:02:51.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.209+0000 7efef3dd4740 1 -- 172.21.15.52:0/2918488868 wait complete. 2024-04-02T14:02:51.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.210+0000 7efef3dd4740 20 client.18234 trim_cache size 0 max 16384 2024-04-02T14:02:51.230 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:02:51.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.210+0000 7efef110e640 1 -- 172.21.15.52:0/1315167835 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efed40a3d20 msgr2=0x7efed40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.210+0000 7efef110e640 1 --2- 172.21.15.52:0/1315167835 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efed40a3d20 0x7efed40a61e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efee4004af0 tx=0x7efee404b000 comp rx=0 tx=0).stop 2024-04-02T14:02:51.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.210+0000 7efef110e640 1 -- 172.21.15.52:0/1315167835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efeec101370 msgr2=0x7efeec198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.210+0000 7efef110e640 1 --2- 172.21.15.52:0/1315167835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efeec101370 0x7efeec198820 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7efee000d950 tx=0x7efee000de10 comp rx=0 tx=0).stop 2024-04-02T14:02:51.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.210+0000 7efef110e640 1 -- 172.21.15.52:0/1315167835 shutdown_connections 2024-04-02T14:02:51.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.210+0000 7efef110e640 1 --2- 172.21.15.52:0/1315167835 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efed40a3d20 0x7efed40a61e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.210+0000 7efef110e640 1 --2- 172.21.15.52:0/1315167835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efeec1069c0 0x7efeec19cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.210+0000 7efef110e640 1 --2- 172.21.15.52:0/1315167835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efeec101370 0x7efeec198820 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.210+0000 7efef110e640 1 --2- 172.21.15.52:0/1315167835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efeec1009b0 0x7efeec1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.210+0000 7efef110e640 1 -- 172.21.15.52:0/1315167835 >> 172.21.15.52:0/1315167835 conn(0x7efeec076d80 msgr2=0x7efeec06a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.210+0000 7efef110e640 1 -- 172.21.15.52:0/1315167835 shutdown_connections 2024-04-02T14:02:51.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.210+0000 7efef110e640 1 -- 172.21.15.52:0/1315167835 wait complete. 2024-04-02T14:02:51.233 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T14:02:51.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.334+0000 7f437d57b640 1 -- 172.21.15.52:0/3864759151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4378101370 msgr2=0x7f437810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.334+0000 7f437d57b640 1 --2- 172.21.15.52:0/3864759151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4378101370 0x7f437810d880 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f4370009cd0 tx=0x7f437002c0f0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.334+0000 7f437d57b640 1 -- 172.21.15.52:0/3864759151 shutdown_connections 2024-04-02T14:02:51.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.334+0000 7f437d57b640 1 --2- 172.21.15.52:0/3864759151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4378101370 0x7f437810d880 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.334+0000 7f437d57b640 1 --2- 172.21.15.52:0/3864759151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43781009b0 0x7f4378100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.334+0000 7f437d57b640 1 --2- 172.21.15.52:0/3864759151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43781069c0 0x7f4378106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.334+0000 7f437d57b640 1 -- 172.21.15.52:0/3864759151 >> 172.21.15.52:0/3864759151 conn(0x7f4378076d80 msgr2=0x7f4378077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.334+0000 7f437d57b640 1 -- 172.21.15.52:0/3864759151 shutdown_connections 2024-04-02T14:02:51.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.334+0000 7f437d57b640 1 -- 172.21.15.52:0/3864759151 wait complete. 2024-04-02T14:02:51.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.334+0000 7f437d57b640 1 Processor -- start 2024-04-02T14:02:51.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.335+0000 7f437d57b640 1 -- start start 2024-04-02T14:02:51.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.335+0000 7f437d57b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43781009b0 0x7f4378198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.335+0000 7f437d57b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4378101370 0x7f4378198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.335+0000 7f437d57b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43781069c0 0x7f437819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.335+0000 7f437d57b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4378111ed0 con 0x7f43781009b0 2024-04-02T14:02:51.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.335+0000 7f437d57b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4378111d50 con 0x7f4378101370 2024-04-02T14:02:51.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.335+0000 7f437d57b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4378112050 con 0x7f43781069c0 2024-04-02T14:02:51.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.335+0000 7f4376ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43781009b0 0x7f4378198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.335+0000 7f4376ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43781009b0 0x7f4378198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45744/0 (socket says 172.21.15.52:45744) 2024-04-02T14:02:51.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.335+0000 7f4376ffd640 1 -- 172.21.15.52:0/351066846 learned_addr learned my addr 172.21.15.52:0/351066846 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:51.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.335+0000 7f43767fc640 1 --2- 172.21.15.52:0/351066846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4378101370 0x7f4378198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.335+0000 7f43777fe640 1 --2- 172.21.15.52:0/351066846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43781069c0 0x7f437819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.335+0000 7f4376ffd640 1 -- 172.21.15.52:0/351066846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43781069c0 msgr2=0x7f437819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.335+0000 7f4376ffd640 1 --2- 172.21.15.52:0/351066846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43781069c0 0x7f437819cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.335+0000 7f4376ffd640 1 -- 172.21.15.52:0/351066846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4378101370 msgr2=0x7f4378198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.335+0000 7f4376ffd640 1 --2- 172.21.15.52:0/351066846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4378101370 0x7f4378198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.335+0000 7f4376ffd640 1 -- 172.21.15.52:0/351066846 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437819d340 con 0x7f43781009b0 2024-04-02T14:02:51.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.336+0000 7f4376ffd640 1 --2- 172.21.15.52:0/351066846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43781009b0 0x7f4378198350 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f436800d950 tx=0x7f436800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.336+0000 7f435ffff640 1 -- 172.21.15.52:0/351066846 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4368017030 con 0x7f43781009b0 2024-04-02T14:02:51.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.336+0000 7f437d57b640 1 -- 172.21.15.52:0/351066846 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f437819d630 con 0x7f43781009b0 2024-04-02T14:02:51.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.336+0000 7f435ffff640 1 -- 172.21.15.52:0/351066846 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4368004020 con 0x7f43781009b0 2024-04-02T14:02:51.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.336+0000 7f435ffff640 1 -- 172.21.15.52:0/351066846 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43680047a0 con 0x7f43781009b0 2024-04-02T14:02:51.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.336+0000 7f437d57b640 1 -- 172.21.15.52:0/351066846 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43781a4ec0 con 0x7f43781009b0 2024-04-02T14:02:51.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.338+0000 7f435ffff640 1 -- 172.21.15.52:0/351066846 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f43680041d0 con 0x7f43781009b0 2024-04-02T14:02:51.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.338+0000 7f4380165740 1 Processor -- start 2024-04-02T14:02:51.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.338+0000 7f4380165740 1 -- start start 2024-04-02T14:02:51.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.338+0000 7f4380165740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d66980a7a0 0x55d66980ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.338+0000 7f4380165740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d66980b0c0 0x55d669813760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.338+0000 7f4380165740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d669813ca0 0x55d669816090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.338+0000 7f4380165740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d66976e7b0 con 0x55d669813ca0 2024-04-02T14:02:51.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.338+0000 7f4380165740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d66970d570 con 0x55d66980b0c0 2024-04-02T14:02:51.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.338+0000 7f4380165740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d66976a760 con 0x55d66980a7a0 2024-04-02T14:02:51.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.338+0000 7f435ffff640 1 --2- 172.21.15.52:0/351066846 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f43600a3d20 0x7f43600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.339+0000 7f43767fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d669813ca0 0x55d669816090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.339+0000 7f43767fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d669813ca0 0x55d669816090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45756/0 (socket says 172.21.15.52:45756) 2024-04-02T14:02:51.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.339+0000 7f43767fc640 1 -- 172.21.15.52:0/1223903833 learned_addr learned my addr 172.21.15.52:0/1223903833 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:51.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.339+0000 7f43777fe640 1 --2- 172.21.15.52:0/1223903833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d66980a7a0 0x55d66980ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.339+0000 7f4376ffd640 1 --2- 172.21.15.52:0/1223903833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d66980b0c0 0x55d669813760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.339+0000 7f43777fe640 1 --2- 172.21.15.52:0/351066846 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f43600a3d20 0x7f43600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.339+0000 7f435ffff640 1 -- 172.21.15.52:0/351066846 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f43680d05a0 con 0x7f43781009b0 2024-04-02T14:02:51.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.339+0000 7f43767fc640 1 -- 172.21.15.52:0/1223903833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d66980a7a0 msgr2=0x55d66980ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.339+0000 7f43767fc640 1 --2- 172.21.15.52:0/1223903833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d66980a7a0 0x55d66980ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.339+0000 7f43767fc640 1 -- 172.21.15.52:0/1223903833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d66980b0c0 msgr2=0x55d669813760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.339+0000 7f43767fc640 1 --2- 172.21.15.52:0/1223903833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d66980b0c0 0x55d669813760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.339+0000 7f43767fc640 1 -- 172.21.15.52:0/1223903833 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6694823f0 con 0x55d669813ca0 2024-04-02T14:02:51.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.339+0000 7f43777fe640 1 --2- 172.21.15.52:0/351066846 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f43600a3d20 0x7f43600a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f437002c630 tx=0x7f437002c0f0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.339+0000 7f43767fc640 1 --2- 172.21.15.52:0/1223903833 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d669813ca0 0x55d669816090 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f436c009ca0 tx=0x7f436c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99cd5d260c09c84a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.339+0000 7f435d7fa640 1 -- 172.21.15.52:0/1223903833 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c01e070 con 0x55d669813ca0 2024-04-02T14:02:51.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.339+0000 7f435d7fa640 1 -- 172.21.15.52:0/1223903833 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f436c019b10 con 0x55d669813ca0 2024-04-02T14:02:51.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.339+0000 7f435d7fa640 1 -- 172.21.15.52:0/1223903833 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c00ea30 con 0x55d669813ca0 2024-04-02T14:02:51.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.340+0000 7f4380165740 1 -- 172.21.15.52:0/1223903833 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d669813ca0 msgr2=0x55d669816090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.340+0000 7f4380165740 1 --2- 172.21.15.52:0/1223903833 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d669813ca0 0x55d669816090 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f436c009ca0 tx=0x7f436c011810 comp rx=0 tx=0).stop 2024-04-02T14:02:51.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.340+0000 7f4380165740 1 -- 172.21.15.52:0/1223903833 shutdown_connections 2024-04-02T14:02:51.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.340+0000 7f4380165740 1 --2- 172.21.15.52:0/1223903833 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d669813ca0 0x55d669816090 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.340+0000 7f4380165740 1 --2- 172.21.15.52:0/1223903833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d66980b0c0 0x55d669813760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.340+0000 7f4380165740 1 --2- 172.21.15.52:0/1223903833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d66980a7a0 0x55d66980ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.340+0000 7f4380165740 1 -- 172.21.15.52:0/1223903833 >> 172.21.15.52:0/1223903833 conn(0x55d669804ed0 msgr2=0x55d6698052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.340+0000 7f4380165740 1 -- 172.21.15.52:0/1223903833 shutdown_connections 2024-04-02T14:02:51.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.340+0000 7f4380165740 1 -- 172.21.15.52:0/1223903833 wait complete. 2024-04-02T14:02:51.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.340+0000 7f4380165740 1 -- start start 2024-04-02T14:02:51.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.340+0000 7f4380165740 1 Processor -- start 2024-04-02T14:02:51.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.341+0000 7f4380165740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d66980a7a0 0x55d6698136b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.341+0000 7f4380165740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d66980b0c0 0x55d669810a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.341+0000 7f4380165740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d669813ca0 0x55d669811050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.341+0000 7f4380165740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d669816a90 con 0x55d66980b0c0 2024-04-02T14:02:51.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.341+0000 7f4380165740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d669729e80 con 0x55d66980a7a0 2024-04-02T14:02:51.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.341+0000 7f4380165740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d669816c10 con 0x55d669813ca0 2024-04-02T14:02:51.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.341+0000 7f4380165740 10 client.0 _ll_register_callbacks cb 0x55d669765aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:51.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.341+0000 7f43767fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d66980b0c0 0x55d669810a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.341+0000 7f43767fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d66980b0c0 0x55d669810a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45758/0 (socket says 172.21.15.52:45758) 2024-04-02T14:02:51.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.341+0000 7f43767fc640 1 -- 172.21.15.52:0/3211225743 learned_addr learned my addr 172.21.15.52:0/3211225743 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:51.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.341+0000 7f4376ffd640 1 --2- 172.21.15.52:0/3211225743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d66980a7a0 0x55d6698136b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.341+0000 7f43777fe640 1 --2- 172.21.15.52:0/3211225743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d669813ca0 0x55d669811050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.341+0000 7f43767fc640 1 -- 172.21.15.52:0/3211225743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d669813ca0 msgr2=0x55d669811050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.341+0000 7f43767fc640 1 --2- 172.21.15.52:0/3211225743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d669813ca0 0x55d669811050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.341+0000 7f43767fc640 1 -- 172.21.15.52:0/3211225743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d66980a7a0 msgr2=0x55d6698136b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.341+0000 7f43767fc640 1 --2- 172.21.15.52:0/3211225743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d66980a7a0 0x55d6698136b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.341+0000 7f43767fc640 1 -- 172.21.15.52:0/3211225743 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d66954c7d0 con 0x55d66980b0c0 2024-04-02T14:02:51.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.341+0000 7f4376ffd640 1 --2- 172.21.15.52:0/3211225743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d66980a7a0 0x55d6698136b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:02:51.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.341+0000 7f43767fc640 1 --2- 172.21.15.52:0/3211225743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d66980b0c0 0x55d669810a40 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f436c009c70 tx=0x7f436c00e5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646c484c704dbade server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.341+0000 7f435aff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:51.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.341+0000 7f435aff5640 1 -- 172.21.15.52:0/3211225743 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c028020 con 0x55d66980b0c0 2024-04-02T14:02:51.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.341+0000 7f435aff5640 1 -- 172.21.15.52:0/3211225743 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f436c005050 con 0x55d66980b0c0 2024-04-02T14:02:51.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.341+0000 7f4380165740 1 -- 172.21.15.52:0/3211225743 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f436c019b10 con 0x55d66980b0c0 2024-04-02T14:02:51.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.342+0000 7f435aff5640 1 -- 172.21.15.52:0/3211225743 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f436c033050 con 0x55d66980b0c0 2024-04-02T14:02:51.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.342+0000 7f4380165740 10 client.18246 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:51.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.342+0000 7f4380165740 1 -- 172.21.15.52:0/3211225743 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d669822ac0 con 0x55d66980b0c0 2024-04-02T14:02:51.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.342+0000 7f435aff5640 1 -- 172.21.15.52:0/3211225743 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f436c019b10 con 0x55d66980b0c0 2024-04-02T14:02:51.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.342+0000 7f435aff5640 1 -- 172.21.15.52:0/3211225743 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f436c0046c0 con 0x55d66980b0c0 2024-04-02T14:02:51.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.342+0000 7f4380165740 10 client.18246 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:51.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.342+0000 7f4380165740 10 client.18246 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T14:02:51.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.342+0000 7f4380165740 20 client.18246 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:51.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.342+0000 7f4380165740 1 --2- 172.21.15.52:0/3211225743 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55d669823e70 0x55d669826260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.342+0000 7f4380165740 4 client.18246 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:02:51.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.342+0000 7f4380165740 1 -- 172.21.15.52:0/3211225743 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d66977bcf0 con 0x55d669823e70 2024-04-02T14:02:51.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.342+0000 7f4376ffd640 1 --2- 172.21.15.52:0/3211225743 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55d669823e70 0x55d669826260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.343+0000 7f435aff5640 10 client.18246 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T14:02:51.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.343+0000 7f4376ffd640 1 --2- 172.21.15.52:0/3211225743 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55d669823e70 0x55d669826260 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=595c7befa82905a3 server_cookie=90d43307f376386a in_seq=0 out_seq=0 2024-04-02T14:02:51.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.343+0000 7f435aff5640 1 -- 172.21.15.52:0/3211225743 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d66977bcf0 con 0x55d669823e70 2024-04-02T14:02:51.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.343+0000 7f435aff5640 10 client.18246 handle_command_reply: tid=0 2024-04-02T14:02:51.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.343+0000 7f4380165740 1 client.18246 shutdown 2024-04-02T14:02:51.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.343+0000 7f4380165740 1 -- 172.21.15.52:0/3211225743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d66980b0c0 msgr2=0x55d669810a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.343+0000 7f4380165740 1 --2- 172.21.15.52:0/3211225743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d66980b0c0 0x55d669810a40 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f436c009c70 tx=0x7f436c00e5d0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.343+0000 7f4380165740 1 -- 172.21.15.52:0/3211225743 shutdown_connections 2024-04-02T14:02:51.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.343+0000 7f4380165740 1 --2- 172.21.15.52:0/3211225743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d669813ca0 0x55d669811050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.343+0000 7f4380165740 1 --2- 172.21.15.52:0/3211225743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d66980b0c0 0x55d669810a40 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.343+0000 7f4380165740 1 --2- 172.21.15.52:0/3211225743 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55d669823e70 0x55d669826260 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.344+0000 7f4380165740 1 --2- 172.21.15.52:0/3211225743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d66980a7a0 0x55d6698136b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.344+0000 7f435aff5640 0 client.18246 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T14:02:51.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.344+0000 7f4380165740 1 -- 172.21.15.52:0/3211225743 >> 172.21.15.52:0/3211225743 conn(0x55d669804ed0 msgr2=0x55d669806c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.344+0000 7f4380165740 1 -- 172.21.15.52:0/3211225743 shutdown_connections 2024-04-02T14:02:51.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.344+0000 7f4380165740 1 -- 172.21.15.52:0/3211225743 wait complete. 2024-04-02T14:02:51.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.344+0000 7f4380165740 20 client.18246 trim_cache size 0 max 16384 2024-04-02T14:02:51.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.357+0000 7f4380165740 1 Processor -- start 2024-04-02T14:02:51.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.357+0000 7f4380165740 1 -- start start 2024-04-02T14:02:51.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.357+0000 7f4380165740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d66980a7a0 0x55d66981f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.357+0000 7f4380165740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d66980b0c0 0x55d669813170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.357+0000 7f4380165740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d669813ca0 0x55d6698136b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.357+0000 7f4380165740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f43781aef80 con 0x55d66980a7a0 2024-04-02T14:02:51.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.357+0000 7f4380165740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f43781b0ac0 con 0x55d669813ca0 2024-04-02T14:02:51.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.357+0000 7f4380165740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f43781af650 con 0x55d66980b0c0 2024-04-02T14:02:51.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.357+0000 7f4376ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d66980a7a0 0x55d66981f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.357+0000 7f4376ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d66980a7a0 0x55d66981f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45768/0 (socket says 172.21.15.52:45768) 2024-04-02T14:02:51.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.357+0000 7f4376ffd640 1 -- 172.21.15.52:0/1865822747 learned_addr learned my addr 172.21.15.52:0/1865822747 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:51.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.357+0000 7f4376ffd640 1 -- 172.21.15.52:0/1865822747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d66980b0c0 msgr2=0x55d669813170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.357+0000 7f4376ffd640 1 --2- 172.21.15.52:0/1865822747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d66980b0c0 0x55d669813170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.357+0000 7f4376ffd640 1 -- 172.21.15.52:0/1865822747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d669813ca0 msgr2=0x55d6698136b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.357+0000 7f43777fe640 1 --2- 172.21.15.52:0/1865822747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d669813ca0 0x55d6698136b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.357+0000 7f4376ffd640 1 --2- 172.21.15.52:0/1865822747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d669813ca0 0x55d6698136b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.357+0000 7f4376ffd640 1 -- 172.21.15.52:0/1865822747 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6698113c0 con 0x55d66980a7a0 2024-04-02T14:02:51.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.357+0000 7f4376ffd640 1 --2- 172.21.15.52:0/1865822747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d66980a7a0 0x55d66981f950 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f4368098b20 tx=0x7f436806ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c509ed39894659b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.358+0000 7f435cff9640 1 -- 172.21.15.52:0/1865822747 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4368066c60 con 0x55d66980a7a0 2024-04-02T14:02:51.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.358+0000 7f435cff9640 1 -- 172.21.15.52:0/1865822747 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f436809c150 con 0x55d66980a7a0 2024-04-02T14:02:51.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.358+0000 7f435cff9640 1 -- 172.21.15.52:0/1865822747 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43680a0120 con 0x55d66980a7a0 2024-04-02T14:02:51.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.358+0000 7f4380165740 1 -- 172.21.15.52:0/1865822747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d66980a7a0 msgr2=0x55d66981f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.358+0000 7f4380165740 1 --2- 172.21.15.52:0/1865822747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d66980a7a0 0x55d66981f950 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f4368098b20 tx=0x7f436806ba30 comp rx=0 tx=0).stop 2024-04-02T14:02:51.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.358+0000 7f4380165740 1 -- 172.21.15.52:0/1865822747 shutdown_connections 2024-04-02T14:02:51.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.358+0000 7f4380165740 1 --2- 172.21.15.52:0/1865822747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d669813ca0 0x55d6698136b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.358+0000 7f4380165740 1 --2- 172.21.15.52:0/1865822747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d66980b0c0 0x55d669813170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.358+0000 7f4380165740 1 --2- 172.21.15.52:0/1865822747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d66980a7a0 0x55d66981f950 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.358+0000 7f4380165740 1 -- 172.21.15.52:0/1865822747 >> 172.21.15.52:0/1865822747 conn(0x55d669804ed0 msgr2=0x55d6698052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.358+0000 7f4380165740 1 -- 172.21.15.52:0/1865822747 shutdown_connections 2024-04-02T14:02:51.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.358+0000 7f4380165740 1 -- 172.21.15.52:0/1865822747 wait complete. 2024-04-02T14:02:51.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.358+0000 7f4380165740 1 -- start start 2024-04-02T14:02:51.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.359+0000 7f4380165740 1 Processor -- start 2024-04-02T14:02:51.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.359+0000 7f4380165740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d66980a7a0 0x55d66980ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.359+0000 7f4380165740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d66980b0c0 0x55d66980d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.359+0000 7f4380165740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d669813ca0 0x55d66980d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.359+0000 7f4380165740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f43781b03f0 con 0x55d66980b0c0 2024-04-02T14:02:51.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.359+0000 7f4380165740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f43781afd20 con 0x55d669813ca0 2024-04-02T14:02:51.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.359+0000 7f4380165740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f43781acf80 con 0x55d66980a7a0 2024-04-02T14:02:51.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.359+0000 7f4380165740 10 client.0 _ll_register_callbacks cb 0x7f436c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:51.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.359+0000 7f43767fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d66980b0c0 0x55d66980d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.359+0000 7f43767fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d66980b0c0 0x55d66980d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45770/0 (socket says 172.21.15.52:45770) 2024-04-02T14:02:51.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.359+0000 7f43767fc640 1 -- 172.21.15.52:0/2994765435 learned_addr learned my addr 172.21.15.52:0/2994765435 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:51.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.359+0000 7f4376ffd640 1 --2- 172.21.15.52:0/2994765435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d66980a7a0 0x55d66980ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.359+0000 7f43777fe640 1 --2- 172.21.15.52:0/2994765435 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d669813ca0 0x55d66980d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.359+0000 7f43767fc640 1 -- 172.21.15.52:0/2994765435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d66980a7a0 msgr2=0x55d66980ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.359+0000 7f43767fc640 1 --2- 172.21.15.52:0/2994765435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d66980a7a0 0x55d66980ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.359+0000 7f43767fc640 1 -- 172.21.15.52:0/2994765435 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d669813ca0 msgr2=0x55d66980d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.359+0000 7f43767fc640 1 --2- 172.21.15.52:0/2994765435 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d669813ca0 0x55d66980d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.359+0000 7f43767fc640 1 -- 172.21.15.52:0/2994765435 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d66980aef0 con 0x55d66980b0c0 2024-04-02T14:02:51.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.359+0000 7f43767fc640 1 --2- 172.21.15.52:0/2994765435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d66980b0c0 0x55d66980d390 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f436c004c00 tx=0x7f436c00e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d602d43b4f6f37b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.359+0000 7f435aff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:51.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.359+0000 7f435aff5640 1 -- 172.21.15.52:0/2994765435 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c028070 con 0x55d66980b0c0 2024-04-02T14:02:51.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.359+0000 7f435aff5640 1 -- 172.21.15.52:0/2994765435 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f436c019a70 con 0x55d66980b0c0 2024-04-02T14:02:51.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.359+0000 7f4380165740 1 -- 172.21.15.52:0/2994765435 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f436809c150 con 0x55d66980b0c0 2024-04-02T14:02:51.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.360+0000 7f435aff5640 1 -- 172.21.15.52:0/2994765435 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f436c01e040 con 0x55d66980b0c0 2024-04-02T14:02:51.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.360+0000 7f4380165740 10 client.18252 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:51.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.360+0000 7f4380165740 1 -- 172.21.15.52:0/2994765435 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d66981c170 con 0x55d66980b0c0 2024-04-02T14:02:51.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.360+0000 7f435aff5640 1 -- 172.21.15.52:0/2994765435 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f436809c150 con 0x55d66980b0c0 2024-04-02T14:02:51.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.360+0000 7f435aff5640 1 -- 172.21.15.52:0/2994765435 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f436c004030 con 0x55d66980b0c0 2024-04-02T14:02:51.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.360+0000 7f4380165740 10 client.18252 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:51.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.360+0000 7f4380165740 10 client.18252 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T14:02:51.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.360+0000 7f4380165740 20 client.18252 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:51.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.360+0000 7f4380165740 1 --2- 172.21.15.52:0/2994765435 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55d66988a290 0x55d66988c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.360+0000 7f4380165740 4 client.18252 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:02:51.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.360+0000 7f4380165740 1 -- 172.21.15.52:0/2994765435 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d669776040 con 0x55d66988a290 2024-04-02T14:02:51.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.360+0000 7f4376ffd640 1 --2- 172.21.15.52:0/2994765435 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55d66988a290 0x55d66988c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.361+0000 7f4376ffd640 1 --2- 172.21.15.52:0/2994765435 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55d66988a290 0x55d66988c680 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=74644af6775b8e63 server_cookie=90d277f8103cf5c8 in_seq=0 out_seq=0 2024-04-02T14:02:51.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.361+0000 7f435aff5640 10 client.18252 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T14:02:51.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.361+0000 7f435aff5640 1 -- 172.21.15.52:0/2994765435 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d669776040 con 0x55d66988a290 2024-04-02T14:02:51.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.361+0000 7f435aff5640 10 client.18252 handle_command_reply: tid=0 2024-04-02T14:02:51.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.361+0000 7f4380165740 1 client.18252 shutdown 2024-04-02T14:02:51.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.361+0000 7f4380165740 1 -- 172.21.15.52:0/2994765435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d66980b0c0 msgr2=0x55d66980d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.361+0000 7f4380165740 1 --2- 172.21.15.52:0/2994765435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d66980b0c0 0x55d66980d390 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f436c004c00 tx=0x7f436c00e3b0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.361+0000 7f4380165740 1 -- 172.21.15.52:0/2994765435 shutdown_connections 2024-04-02T14:02:51.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.361+0000 7f4380165740 1 --2- 172.21.15.52:0/2994765435 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x55d66988a290 0x55d66988c680 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.361+0000 7f4380165740 1 --2- 172.21.15.52:0/2994765435 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d669813ca0 0x55d66980d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.361+0000 7f4380165740 1 --2- 172.21.15.52:0/2994765435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d66980b0c0 0x55d66980d390 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.361+0000 7f4380165740 1 --2- 172.21.15.52:0/2994765435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d66980a7a0 0x55d66980ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.361+0000 7f435aff5640 0 client.18252 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T14:02:51.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.361+0000 7f4380165740 1 -- 172.21.15.52:0/2994765435 >> 172.21.15.52:0/2994765435 conn(0x55d669804ed0 msgr2=0x55d66980f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.361+0000 7f4380165740 1 -- 172.21.15.52:0/2994765435 shutdown_connections 2024-04-02T14:02:51.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.361+0000 7f4380165740 1 -- 172.21.15.52:0/2994765435 wait complete. 2024-04-02T14:02:51.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.361+0000 7f4380165740 20 client.18252 trim_cache size 0 max 16384 2024-04-02T14:02:51.380 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:02:51.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.361+0000 7f437d57b640 1 -- 172.21.15.52:0/351066846 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f43600a3d20 msgr2=0x7f43600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.361+0000 7f437d57b640 1 --2- 172.21.15.52:0/351066846 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f43600a3d20 0x7f43600a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f437002c630 tx=0x7f437002c0f0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.361+0000 7f437d57b640 1 -- 172.21.15.52:0/351066846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43781009b0 msgr2=0x7f4378198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.361+0000 7f437d57b640 1 --2- 172.21.15.52:0/351066846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43781009b0 0x7f4378198350 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f436800d950 tx=0x7f436800de10 comp rx=0 tx=0).stop 2024-04-02T14:02:51.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.361+0000 7f437d57b640 1 -- 172.21.15.52:0/351066846 shutdown_connections 2024-04-02T14:02:51.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.362+0000 7f437d57b640 1 --2- 172.21.15.52:0/351066846 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f43600a3d20 0x7f43600a61e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.362+0000 7f437d57b640 1 --2- 172.21.15.52:0/351066846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43781069c0 0x7f437819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.362+0000 7f437d57b640 1 --2- 172.21.15.52:0/351066846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4378101370 0x7f4378198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.362+0000 7f437d57b640 1 --2- 172.21.15.52:0/351066846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43781009b0 0x7f4378198350 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.362+0000 7f437d57b640 1 -- 172.21.15.52:0/351066846 >> 172.21.15.52:0/351066846 conn(0x7f4378076d80 msgr2=0x7f43780fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.362+0000 7f437d57b640 1 -- 172.21.15.52:0/351066846 shutdown_connections 2024-04-02T14:02:51.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.362+0000 7f437d57b640 1 -- 172.21.15.52:0/351066846 wait complete. 2024-04-02T14:02:51.386 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T14:02:51.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.487+0000 7f711fa2c640 1 -- 172.21.15.52:0/902965568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71181069c0 msgr2=0x7f7118106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.487+0000 7f711fa2c640 1 --2- 172.21.15.52:0/902965568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71181069c0 0x7f7118106da0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f7114009930 tx=0x7f711402bc30 comp rx=0 tx=0).stop 2024-04-02T14:02:51.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.487+0000 7f711fa2c640 1 -- 172.21.15.52:0/902965568 shutdown_connections 2024-04-02T14:02:51.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.487+0000 7f711fa2c640 1 --2- 172.21.15.52:0/902965568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7118101370 0x7f711810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.487+0000 7f711fa2c640 1 --2- 172.21.15.52:0/902965568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71181009b0 0x7f7118100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.487+0000 7f711fa2c640 1 --2- 172.21.15.52:0/902965568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71181069c0 0x7f7118106da0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.487+0000 7f711fa2c640 1 -- 172.21.15.52:0/902965568 >> 172.21.15.52:0/902965568 conn(0x7f7118076d80 msgr2=0x7f7118077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.487+0000 7f711fa2c640 1 -- 172.21.15.52:0/902965568 shutdown_connections 2024-04-02T14:02:51.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.487+0000 7f711fa2c640 1 -- 172.21.15.52:0/902965568 wait complete. 2024-04-02T14:02:51.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.488+0000 7f711fa2c640 1 Processor -- start 2024-04-02T14:02:51.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.488+0000 7f711fa2c640 1 -- start start 2024-04-02T14:02:51.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.488+0000 7f711fa2c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71181009b0 0x7f7118198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.488+0000 7f711fa2c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7118101370 0x7f7118198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.488+0000 7f711fa2c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71181069c0 0x7f711819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.488+0000 7f711fa2c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7118110da0 con 0x7f7118101370 2024-04-02T14:02:51.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.488+0000 7f711fa2c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7118110c20 con 0x7f71181009b0 2024-04-02T14:02:51.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.488+0000 7f711fa2c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7118110f20 con 0x7f71181069c0 2024-04-02T14:02:51.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.488+0000 7f711cfc7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7118101370 0x7f7118198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.488+0000 7f711cfc7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7118101370 0x7f7118198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45790/0 (socket says 172.21.15.52:45790) 2024-04-02T14:02:51.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.488+0000 7f711cfc7640 1 -- 172.21.15.52:0/1026140799 learned_addr learned my addr 172.21.15.52:0/1026140799 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:51.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.488+0000 7f711d7c8640 1 --2- 172.21.15.52:0/1026140799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71181009b0 0x7f7118198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.488+0000 7f711dfc9640 1 --2- 172.21.15.52:0/1026140799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71181069c0 0x7f711819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.488+0000 7f711cfc7640 1 -- 172.21.15.52:0/1026140799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71181069c0 msgr2=0x7f711819cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.488+0000 7f711cfc7640 1 --2- 172.21.15.52:0/1026140799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71181069c0 0x7f711819cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.488+0000 7f711cfc7640 1 -- 172.21.15.52:0/1026140799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71181009b0 msgr2=0x7f7118198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.488+0000 7f711cfc7640 1 --2- 172.21.15.52:0/1026140799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71181009b0 0x7f7118198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.488+0000 7f711cfc7640 1 -- 172.21.15.52:0/1026140799 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711819d320 con 0x7f7118101370 2024-04-02T14:02:51.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.488+0000 7f711cfc7640 1 --2- 172.21.15.52:0/1026140799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7118101370 0x7f7118198870 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f710800d950 tx=0x7f710800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.489+0000 7f710e7fc640 1 -- 172.21.15.52:0/1026140799 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108017070 con 0x7f7118101370 2024-04-02T14:02:51.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.489+0000 7f711fa2c640 1 -- 172.21.15.52:0/1026140799 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f711819d610 con 0x7f7118101370 2024-04-02T14:02:51.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.489+0000 7f710e7fc640 1 -- 172.21.15.52:0/1026140799 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7108009c30 con 0x7f7118101370 2024-04-02T14:02:51.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.489+0000 7f710e7fc640 1 -- 172.21.15.52:0/1026140799 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710800f6b0 con 0x7f7118101370 2024-04-02T14:02:51.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.489+0000 7f711fa2c640 1 -- 172.21.15.52:0/1026140799 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71181a4ea0 con 0x7f7118101370 2024-04-02T14:02:51.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.491+0000 7f710e7fc640 1 -- 172.21.15.52:0/1026140799 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f71080109d0 con 0x7f7118101370 2024-04-02T14:02:51.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.491+0000 7f712260f740 1 Processor -- start 2024-04-02T14:02:51.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.492+0000 7f712260f740 1 -- start start 2024-04-02T14:02:51.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.492+0000 7f710e7fc640 1 --2- 172.21.15.52:0/1026140799 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f71040a3d20 0x7f71040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.492+0000 7f712260f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56376aed67a0 0x56376aed6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.492+0000 7f712260f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56376aed70c0 0x56376aedf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.492+0000 7f712260f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aedfca0 0x56376aee2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.492+0000 7f712260f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56376ae3a7b0 con 0x56376aedfca0 2024-04-02T14:02:51.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.492+0000 7f712260f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56376add9570 con 0x56376aed70c0 2024-04-02T14:02:51.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.492+0000 7f712260f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56376ae36760 con 0x56376aed67a0 2024-04-02T14:02:51.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.492+0000 7f711dfc9640 1 --2- 172.21.15.52:0/1026140799 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f71040a3d20 0x7f71040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.492+0000 7f710e7fc640 1 -- 172.21.15.52:0/1026140799 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f71080d0800 con 0x7f7118101370 2024-04-02T14:02:51.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.492+0000 7f711dfc9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aedfca0 0x56376aee2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.492+0000 7f711dfc9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aedfca0 0x56376aee2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45796/0 (socket says 172.21.15.52:45796) 2024-04-02T14:02:51.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.492+0000 7f711dfc9640 1 -- 172.21.15.52:0/1152858866 learned_addr learned my addr 172.21.15.52:0/1152858866 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:51.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.492+0000 7f711cfc7640 1 --2- 172.21.15.52:0/1152858866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56376aed70c0 0x56376aedf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.492+0000 7f711d7c8640 1 --2- 172.21.15.52:0/1152858866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56376aed67a0 0x56376aed6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.492+0000 7f711dfc9640 1 -- 172.21.15.52:0/1152858866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56376aed67a0 msgr2=0x56376aed6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.492+0000 7f711dfc9640 1 --2- 172.21.15.52:0/1152858866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56376aed67a0 0x56376aed6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.492+0000 7f711dfc9640 1 -- 172.21.15.52:0/1152858866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56376aed70c0 msgr2=0x56376aedf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.492+0000 7f711dfc9640 1 --2- 172.21.15.52:0/1152858866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56376aed70c0 0x56376aedf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.492+0000 7f711dfc9640 1 -- 172.21.15.52:0/1152858866 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56376ab4e3f0 con 0x56376aedfca0 2024-04-02T14:02:51.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.492+0000 7f711d7c8640 1 --2- 172.21.15.52:0/1152858866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56376aed67a0 0x56376aed6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:02:51.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.492+0000 7f711cfc7640 1 --2- 172.21.15.52:0/1152858866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56376aed70c0 0x56376aedf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:02:51.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.492+0000 7f711dfc9640 1 --2- 172.21.15.52:0/1026140799 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f71040a3d20 0x7f71040a61e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f711000bfd0 tx=0x7f711000bd40 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.492+0000 7f711dfc9640 1 --2- 172.21.15.52:0/1152858866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aedfca0 0x56376aee2090 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f7110010d90 tx=0x7f711001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9cc8762be9625d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.493+0000 7f71037fe640 1 -- 172.21.15.52:0/1152858866 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711000ee60 con 0x56376aedfca0 2024-04-02T14:02:51.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.493+0000 7f71037fe640 1 -- 172.21.15.52:0/1152858866 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7110010e80 con 0x56376aedfca0 2024-04-02T14:02:51.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.493+0000 7f71037fe640 1 -- 172.21.15.52:0/1152858866 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110003e00 con 0x56376aedfca0 2024-04-02T14:02:51.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.493+0000 7f712260f740 1 -- 172.21.15.52:0/1152858866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aedfca0 msgr2=0x56376aee2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.493+0000 7f712260f740 1 --2- 172.21.15.52:0/1152858866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aedfca0 0x56376aee2090 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f7110010d90 tx=0x7f711001a000 comp rx=0 tx=0).stop 2024-04-02T14:02:51.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.493+0000 7f712260f740 1 -- 172.21.15.52:0/1152858866 shutdown_connections 2024-04-02T14:02:51.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.493+0000 7f712260f740 1 --2- 172.21.15.52:0/1152858866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aedfca0 0x56376aee2090 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.493+0000 7f712260f740 1 --2- 172.21.15.52:0/1152858866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56376aed70c0 0x56376aedf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.493+0000 7f712260f740 1 --2- 172.21.15.52:0/1152858866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56376aed67a0 0x56376aed6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.493+0000 7f712260f740 1 -- 172.21.15.52:0/1152858866 >> 172.21.15.52:0/1152858866 conn(0x56376aed0ed0 msgr2=0x56376aed12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.493+0000 7f712260f740 1 -- 172.21.15.52:0/1152858866 shutdown_connections 2024-04-02T14:02:51.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.493+0000 7f712260f740 1 -- 172.21.15.52:0/1152858866 wait complete. 2024-04-02T14:02:51.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.494+0000 7f712260f740 1 -- start start 2024-04-02T14:02:51.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.494+0000 7f712260f740 1 Processor -- start 2024-04-02T14:02:51.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.494+0000 7f712260f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56376aed67a0 0x56376aedf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.494+0000 7f712260f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56376aed70c0 0x56376aedca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.494+0000 7f712260f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aedfca0 0x56376aedd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.494+0000 7f712260f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56376aee2a30 con 0x56376aedfca0 2024-04-02T14:02:51.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.494+0000 7f712260f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56376adf5e80 con 0x56376aed70c0 2024-04-02T14:02:51.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.494+0000 7f712260f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56376aee2bb0 con 0x56376aed67a0 2024-04-02T14:02:51.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.494+0000 7f712260f740 10 client.0 _ll_register_callbacks cb 0x56376ae31aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:51.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.494+0000 7f711dfc9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aedfca0 0x56376aedd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.494+0000 7f711dfc9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aedfca0 0x56376aedd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45806/0 (socket says 172.21.15.52:45806) 2024-04-02T14:02:51.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.494+0000 7f711dfc9640 1 -- 172.21.15.52:0/1336840077 learned_addr learned my addr 172.21.15.52:0/1336840077 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:51.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.494+0000 7f711cfc7640 1 --2- 172.21.15.52:0/1336840077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56376aed70c0 0x56376aedca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.495+0000 7f711d7c8640 1 --2- 172.21.15.52:0/1336840077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56376aed67a0 0x56376aedf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.495+0000 7f711dfc9640 1 -- 172.21.15.52:0/1336840077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56376aed67a0 msgr2=0x56376aedf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.495+0000 7f711dfc9640 1 --2- 172.21.15.52:0/1336840077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56376aed67a0 0x56376aedf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.495+0000 7f711dfc9640 1 -- 172.21.15.52:0/1336840077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56376aed70c0 msgr2=0x56376aedca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.495+0000 7f711dfc9640 1 --2- 172.21.15.52:0/1336840077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56376aed70c0 0x56376aedca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.495+0000 7f711dfc9640 1 -- 172.21.15.52:0/1336840077 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56376ac187d0 con 0x56376aedfca0 2024-04-02T14:02:51.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.495+0000 7f711d7c8640 1 --2- 172.21.15.52:0/1336840077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56376aed67a0 0x56376aedf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:02:51.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.495+0000 7f711dfc9640 1 --2- 172.21.15.52:0/1336840077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aedfca0 0x56376aedd1c0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f711000ec20 tx=0x7f71100039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef03adabf9130d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.495+0000 7f7100ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:51.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.495+0000 7f7100ff9640 1 -- 172.21.15.52:0/1336840077 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711002a030 con 0x56376aedfca0 2024-04-02T14:02:51.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.495+0000 7f7100ff9640 1 -- 172.21.15.52:0/1336840077 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f7110022070 con 0x56376aedfca0 2024-04-02T14:02:51.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.495+0000 7f712260f740 1 -- 172.21.15.52:0/1336840077 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56376ae47cf0 con 0x56376aedfca0 2024-04-02T14:02:51.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.495+0000 7f7100ff9640 1 -- 172.21.15.52:0/1336840077 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7110037050 con 0x56376aedfca0 2024-04-02T14:02:51.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.495+0000 7f712260f740 10 client.18264 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:51.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.495+0000 7f712260f740 1 -- 172.21.15.52:0/1336840077 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56376aeeead0 con 0x56376aedfca0 2024-04-02T14:02:51.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.495+0000 7f7100ff9640 1 -- 172.21.15.52:0/1336840077 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56376ae47cf0 con 0x56376aedfca0 2024-04-02T14:02:51.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.495+0000 7f7100ff9640 1 -- 172.21.15.52:0/1336840077 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7110026600 con 0x56376aedfca0 2024-04-02T14:02:51.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.495+0000 7f712260f740 10 client.18264 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:51.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.495+0000 7f712260f740 10 client.18264 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T14:02:51.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.496+0000 7f712260f740 20 client.18264 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:51.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.496+0000 7f712260f740 1 --2- 172.21.15.52:0/1336840077 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x56376aeefe80 0x56376aef2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.496+0000 7f712260f740 4 client.18264 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:02:51.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.496+0000 7f712260f740 1 -- 172.21.15.52:0/1336840077 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56376ae42040 con 0x56376aeefe80 2024-04-02T14:02:51.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.496+0000 7f711d7c8640 1 --2- 172.21.15.52:0/1336840077 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x56376aeefe80 0x56376aef2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.496+0000 7f711d7c8640 1 --2- 172.21.15.52:0/1336840077 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x56376aeefe80 0x56376aef2270 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=10718a51da9a4439 server_cookie=3f3c77a138b2fd59 in_seq=0 out_seq=0 2024-04-02T14:02:51.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.496+0000 7f7100ff9640 10 client.18264 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T14:02:51.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.497+0000 7f7100ff9640 1 -- 172.21.15.52:0/1336840077 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56376ae42040 con 0x56376aeefe80 2024-04-02T14:02:51.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.497+0000 7f7100ff9640 10 client.18264 handle_command_reply: tid=0 2024-04-02T14:02:51.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.497+0000 7f712260f740 1 client.18264 shutdown 2024-04-02T14:02:51.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.497+0000 7f712260f740 1 -- 172.21.15.52:0/1336840077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aedfca0 msgr2=0x56376aedd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.497+0000 7f712260f740 1 --2- 172.21.15.52:0/1336840077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aedfca0 0x56376aedd1c0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f711000ec20 tx=0x7f71100039e0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.497+0000 7f712260f740 1 -- 172.21.15.52:0/1336840077 shutdown_connections 2024-04-02T14:02:51.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.497+0000 7f712260f740 1 --2- 172.21.15.52:0/1336840077 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x56376aeefe80 0x56376aef2270 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.497+0000 7f712260f740 1 --2- 172.21.15.52:0/1336840077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aedfca0 0x56376aedd1c0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.497+0000 7f712260f740 1 --2- 172.21.15.52:0/1336840077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56376aed70c0 0x56376aedca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.497+0000 7f7100ff9640 0 client.18264 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T14:02:51.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.497+0000 7f712260f740 1 --2- 172.21.15.52:0/1336840077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56376aed67a0 0x56376aedf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.497+0000 7f712260f740 1 -- 172.21.15.52:0/1336840077 >> 172.21.15.52:0/1336840077 conn(0x56376aed0ed0 msgr2=0x56376aed2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.497+0000 7f712260f740 1 -- 172.21.15.52:0/1336840077 shutdown_connections 2024-04-02T14:02:51.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.497+0000 7f712260f740 1 -- 172.21.15.52:0/1336840077 wait complete. 2024-04-02T14:02:51.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.497+0000 7f712260f740 20 client.18264 trim_cache size 0 max 16384 2024-04-02T14:02:51.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.511+0000 7f712260f740 1 Processor -- start 2024-04-02T14:02:51.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.511+0000 7f712260f740 1 -- start start 2024-04-02T14:02:51.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.511+0000 7f712260f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56376aed67a0 0x56376aedc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.511+0000 7f712260f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56376aed70c0 0x56376aef78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.511+0000 7f712260f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aedfca0 0x56376aef7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.511+0000 7f712260f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f71181aef10 con 0x56376aedfca0 2024-04-02T14:02:51.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.511+0000 7f712260f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f71181b0a50 con 0x56376aed67a0 2024-04-02T14:02:51.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.511+0000 7f712260f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71181af5e0 con 0x56376aed70c0 2024-04-02T14:02:51.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.511+0000 7f711dfc9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aedfca0 0x56376aef7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.511+0000 7f711dfc9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aedfca0 0x56376aef7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45816/0 (socket says 172.21.15.52:45816) 2024-04-02T14:02:51.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.511+0000 7f711dfc9640 1 -- 172.21.15.52:0/576641985 learned_addr learned my addr 172.21.15.52:0/576641985 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:51.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.511+0000 7f711d7c8640 1 --2- 172.21.15.52:0/576641985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56376aed67a0 0x56376aedc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.511+0000 7f711cfc7640 1 --2- 172.21.15.52:0/576641985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56376aed70c0 0x56376aef78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.511+0000 7f711dfc9640 1 -- 172.21.15.52:0/576641985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56376aed70c0 msgr2=0x56376aef78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.511+0000 7f711dfc9640 1 --2- 172.21.15.52:0/576641985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56376aed70c0 0x56376aef78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.511+0000 7f711dfc9640 1 -- 172.21.15.52:0/576641985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56376aed67a0 msgr2=0x56376aedc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.511+0000 7f711dfc9640 1 --2- 172.21.15.52:0/576641985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56376aed67a0 0x56376aedc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.511+0000 7f711dfc9640 1 -- 172.21.15.52:0/576641985 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56376aed6ef0 con 0x56376aedfca0 2024-04-02T14:02:51.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.511+0000 7f711cfc7640 1 --2- 172.21.15.52:0/576641985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56376aed70c0 0x56376aef78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:02:51.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.511+0000 7f711d7c8640 1 --2- 172.21.15.52:0/576641985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56376aed67a0 0x56376aedc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:02:51.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.511+0000 7f711dfc9640 1 --2- 172.21.15.52:0/576641985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aedfca0 0x56376aef7e10 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f71100269d0 tx=0x7f7110005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=464f3de1df939666 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.514+0000 7f7102ffd640 1 -- 172.21.15.52:0/576641985 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110034070 con 0x56376aedfca0 2024-04-02T14:02:51.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.514+0000 7f7102ffd640 1 -- 172.21.15.52:0/576641985 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f711000ba50 con 0x56376aedfca0 2024-04-02T14:02:51.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.514+0000 7f7102ffd640 1 -- 172.21.15.52:0/576641985 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711002a070 con 0x56376aedfca0 2024-04-02T14:02:51.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.515+0000 7f712260f740 1 -- 172.21.15.52:0/576641985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aedfca0 msgr2=0x56376aef7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.515+0000 7f712260f740 1 --2- 172.21.15.52:0/576641985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aedfca0 0x56376aef7e10 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f71100269d0 tx=0x7f7110005c50 comp rx=0 tx=0).stop 2024-04-02T14:02:51.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.515+0000 7f712260f740 1 -- 172.21.15.52:0/576641985 shutdown_connections 2024-04-02T14:02:51.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.515+0000 7f712260f740 1 --2- 172.21.15.52:0/576641985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aedfca0 0x56376aef7e10 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.515+0000 7f712260f740 1 --2- 172.21.15.52:0/576641985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56376aed70c0 0x56376aef78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.515+0000 7f712260f740 1 --2- 172.21.15.52:0/576641985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56376aed67a0 0x56376aedc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.515+0000 7f712260f740 1 -- 172.21.15.52:0/576641985 >> 172.21.15.52:0/576641985 conn(0x56376aed0ed0 msgr2=0x56376aecf6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.515+0000 7f712260f740 1 -- 172.21.15.52:0/576641985 shutdown_connections 2024-04-02T14:02:51.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.515+0000 7f712260f740 1 -- 172.21.15.52:0/576641985 wait complete. 2024-04-02T14:02:51.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.515+0000 7f712260f740 1 -- start start 2024-04-02T14:02:51.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.515+0000 7f712260f740 1 Processor -- start 2024-04-02T14:02:51.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.516+0000 7f712260f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56376aed67a0 0x56376aee7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.516+0000 7f712260f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aed70c0 0x56376aee8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.516+0000 7f712260f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56376aedfca0 0x56376aed9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.516+0000 7f712260f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f71181b0380 con 0x56376aed70c0 2024-04-02T14:02:51.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.516+0000 7f712260f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f71181afcb0 con 0x56376aedfca0 2024-04-02T14:02:51.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.516+0000 7f712260f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71181acf10 con 0x56376aed67a0 2024-04-02T14:02:51.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.516+0000 7f712260f740 10 client.0 _ll_register_callbacks cb 0x7f710800df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:51.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.516+0000 7f711cfc7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aed70c0 0x56376aee8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.516+0000 7f711cfc7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aed70c0 0x56376aee8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45832/0 (socket says 172.21.15.52:45832) 2024-04-02T14:02:51.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.516+0000 7f711cfc7640 1 -- 172.21.15.52:0/2386934945 learned_addr learned my addr 172.21.15.52:0/2386934945 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:51.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.516+0000 7f711dfc9640 1 --2- 172.21.15.52:0/2386934945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56376aedfca0 0x56376aed9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.516+0000 7f711d7c8640 1 --2- 172.21.15.52:0/2386934945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56376aed67a0 0x56376aee7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.516+0000 7f711cfc7640 1 -- 172.21.15.52:0/2386934945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56376aed67a0 msgr2=0x56376aee7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.516+0000 7f711cfc7640 1 --2- 172.21.15.52:0/2386934945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56376aed67a0 0x56376aee7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.516+0000 7f711cfc7640 1 -- 172.21.15.52:0/2386934945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56376aedfca0 msgr2=0x56376aed9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.516+0000 7f711cfc7640 1 --2- 172.21.15.52:0/2386934945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56376aedfca0 0x56376aed9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.516+0000 7f711cfc7640 1 -- 172.21.15.52:0/2386934945 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56376aeda410 con 0x56376aed70c0 2024-04-02T14:02:51.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.516+0000 7f711d7c8640 1 --2- 172.21.15.52:0/2386934945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56376aed67a0 0x56376aee7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:02:51.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.516+0000 7f711cfc7640 1 --2- 172.21.15.52:0/2386934945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aed70c0 0x56376aee8390 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f7108009610 tx=0x7f7108009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421a1bdcf7cf6f49 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.516+0000 7f7100ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:51.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.517+0000 7f7100ff9640 1 -- 172.21.15.52:0/2386934945 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108020040 con 0x56376aed70c0 2024-04-02T14:02:51.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.517+0000 7f7100ff9640 1 -- 172.21.15.52:0/2386934945 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f7108076b20 con 0x56376aed70c0 2024-04-02T14:02:51.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.517+0000 7f712260f740 1 -- 172.21.15.52:0/2386934945 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f711000ba50 con 0x56376aed70c0 2024-04-02T14:02:51.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.517+0000 7f7100ff9640 1 -- 172.21.15.52:0/2386934945 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f71080cdfe0 con 0x56376aed70c0 2024-04-02T14:02:51.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.517+0000 7f712260f740 10 client.18270 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:51.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.517+0000 7f712260f740 1 -- 172.21.15.52:0/2386934945 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56376aee0b70 con 0x56376aed70c0 2024-04-02T14:02:51.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.517+0000 7f7100ff9640 1 -- 172.21.15.52:0/2386934945 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f711000ba50 con 0x56376aed70c0 2024-04-02T14:02:51.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.517+0000 7f7100ff9640 1 -- 172.21.15.52:0/2386934945 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f71080a4b90 con 0x56376aed70c0 2024-04-02T14:02:51.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.517+0000 7f712260f740 10 client.18270 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:51.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.517+0000 7f712260f740 10 client.18270 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T14:02:51.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.517+0000 7f712260f740 20 client.18270 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:51.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.517+0000 7f712260f740 1 --2- 172.21.15.52:0/2386934945 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x56376af562a0 0x56376af586c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.517+0000 7f712260f740 4 client.18270 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:02:51.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.517+0000 7f712260f740 1 -- 172.21.15.52:0/2386934945 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56376adbf6b0 con 0x56376af562a0 2024-04-02T14:02:51.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.517+0000 7f711d7c8640 1 --2- 172.21.15.52:0/2386934945 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x56376af562a0 0x56376af586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.518+0000 7f711d7c8640 1 --2- 172.21.15.52:0/2386934945 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x56376af562a0 0x56376af586c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=30166be95fb4629f server_cookie=df942e4974887392 in_seq=0 out_seq=0 2024-04-02T14:02:51.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.518+0000 7f7100ff9640 10 client.18270 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T14:02:51.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.518+0000 7f7100ff9640 1 -- 172.21.15.52:0/2386934945 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56376adbf6b0 con 0x56376af562a0 2024-04-02T14:02:51.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.518+0000 7f7100ff9640 10 client.18270 handle_command_reply: tid=0 2024-04-02T14:02:51.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.518+0000 7f712260f740 1 client.18270 shutdown 2024-04-02T14:02:51.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.518+0000 7f712260f740 1 -- 172.21.15.52:0/2386934945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aed70c0 msgr2=0x56376aee8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.518+0000 7f712260f740 1 --2- 172.21.15.52:0/2386934945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aed70c0 0x56376aee8390 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f7108009610 tx=0x7f7108009f20 comp rx=0 tx=0).stop 2024-04-02T14:02:51.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.518+0000 7f712260f740 1 -- 172.21.15.52:0/2386934945 shutdown_connections 2024-04-02T14:02:51.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.518+0000 7f712260f740 1 --2- 172.21.15.52:0/2386934945 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x56376af562a0 0x56376af586c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.518+0000 7f712260f740 1 --2- 172.21.15.52:0/2386934945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56376aedfca0 0x56376aed9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.518+0000 7f7100ff9640 0 client.18270 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T14:02:51.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.518+0000 7f712260f740 1 --2- 172.21.15.52:0/2386934945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56376aed70c0 0x56376aee8390 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.518+0000 7f712260f740 1 --2- 172.21.15.52:0/2386934945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56376aed67a0 0x56376aee7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.518+0000 7f712260f740 1 -- 172.21.15.52:0/2386934945 >> 172.21.15.52:0/2386934945 conn(0x56376aed0ed0 msgr2=0x56376aecff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.518+0000 7f712260f740 1 -- 172.21.15.52:0/2386934945 shutdown_connections 2024-04-02T14:02:51.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.518+0000 7f712260f740 1 -- 172.21.15.52:0/2386934945 wait complete. 2024-04-02T14:02:51.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.519+0000 7f712260f740 20 client.18270 trim_cache size 0 max 16384 2024-04-02T14:02:51.531 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:02:51.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.519+0000 7f711fa2c640 1 -- 172.21.15.52:0/1026140799 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f71040a3d20 msgr2=0x7f71040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.519+0000 7f711fa2c640 1 --2- 172.21.15.52:0/1026140799 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f71040a3d20 0x7f71040a61e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f711000bfd0 tx=0x7f711000bd40 comp rx=0 tx=0).stop 2024-04-02T14:02:51.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.519+0000 7f711fa2c640 1 -- 172.21.15.52:0/1026140799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7118101370 msgr2=0x7f7118198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.519+0000 7f711fa2c640 1 --2- 172.21.15.52:0/1026140799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7118101370 0x7f7118198870 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f710800d950 tx=0x7f710800de10 comp rx=0 tx=0).stop 2024-04-02T14:02:51.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.519+0000 7f711fa2c640 1 -- 172.21.15.52:0/1026140799 shutdown_connections 2024-04-02T14:02:51.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.519+0000 7f711fa2c640 1 --2- 172.21.15.52:0/1026140799 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f71040a3d20 0x7f71040a61e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.519+0000 7f711fa2c640 1 --2- 172.21.15.52:0/1026140799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71181069c0 0x7f711819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.519+0000 7f711fa2c640 1 --2- 172.21.15.52:0/1026140799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7118101370 0x7f7118198870 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.519+0000 7f711fa2c640 1 --2- 172.21.15.52:0/1026140799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71181009b0 0x7f7118198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.519+0000 7f711fa2c640 1 -- 172.21.15.52:0/1026140799 >> 172.21.15.52:0/1026140799 conn(0x7f7118076d80 msgr2=0x7f7118104760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.519+0000 7f711fa2c640 1 -- 172.21.15.52:0/1026140799 shutdown_connections 2024-04-02T14:02:51.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.519+0000 7f711fa2c640 1 -- 172.21.15.52:0/1026140799 wait complete. 2024-04-02T14:02:51.536 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T14:02:51.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.637+0000 7fc6fb3e2640 1 -- 172.21.15.52:0/1740489497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f4107aa0 msgr2=0x7fc6f4107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.637+0000 7fc6fb3e2640 1 --2- 172.21.15.52:0/1740489497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f4107aa0 0x7fc6f4107e80 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0009930 tx=0x7fc6f002bc30 comp rx=0 tx=0).stop 2024-04-02T14:02:51.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.637+0000 7fc6fb3e2640 1 -- 172.21.15.52:0/1740489497 shutdown_connections 2024-04-02T14:02:51.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.637+0000 7fc6fb3e2640 1 --2- 172.21.15.52:0/1740489497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6f4102450 0x7fc6f410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.637+0000 7fc6fb3e2640 1 --2- 172.21.15.52:0/1740489497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6f4101a90 0x7fc6f4101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.637+0000 7fc6fb3e2640 1 --2- 172.21.15.52:0/1740489497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f4107aa0 0x7fc6f4107e80 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.637+0000 7fc6fb3e2640 1 -- 172.21.15.52:0/1740489497 >> 172.21.15.52:0/1740489497 conn(0x7fc6f4078f00 msgr2=0x7fc6f4079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.638+0000 7fc6fb3e2640 1 -- 172.21.15.52:0/1740489497 shutdown_connections 2024-04-02T14:02:51.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.638+0000 7fc6fb3e2640 1 -- 172.21.15.52:0/1740489497 wait complete. 2024-04-02T14:02:51.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.638+0000 7fc6fb3e2640 1 Processor -- start 2024-04-02T14:02:51.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.638+0000 7fc6fb3e2640 1 -- start start 2024-04-02T14:02:51.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.638+0000 7fc6fb3e2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6f4101a90 0x7fc6f41982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.638+0000 7fc6fb3e2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f4102450 0x7fc6f4198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.638+0000 7fc6fb3e2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6f4107aa0 0x7fc6f419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.638+0000 7fc6fb3e2640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4111e80 con 0x7fc6f4102450 2024-04-02T14:02:51.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.638+0000 7fc6fb3e2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4111d00 con 0x7fc6f4101a90 2024-04-02T14:02:51.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.638+0000 7fc6fb3e2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4112000 con 0x7fc6f4107aa0 2024-04-02T14:02:51.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.638+0000 7fc6f897d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f4102450 0x7fc6f4198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.638+0000 7fc6f897d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f4102450 0x7fc6f4198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46336/0 (socket says 172.21.15.52:46336) 2024-04-02T14:02:51.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.638+0000 7fc6f897d640 1 -- 172.21.15.52:0/3391481866 learned_addr learned my addr 172.21.15.52:0/3391481866 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:51.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.639+0000 7fc6f997f640 1 --2- 172.21.15.52:0/3391481866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6f4107aa0 0x7fc6f419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.639+0000 7fc6f917e640 1 --2- 172.21.15.52:0/3391481866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6f4101a90 0x7fc6f41982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.639+0000 7fc6f897d640 1 -- 172.21.15.52:0/3391481866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6f4107aa0 msgr2=0x7fc6f419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.639+0000 7fc6f897d640 1 --2- 172.21.15.52:0/3391481866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6f4107aa0 0x7fc6f419cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.639+0000 7fc6f897d640 1 -- 172.21.15.52:0/3391481866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6f4101a90 msgr2=0x7fc6f41982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.639+0000 7fc6f897d640 1 --2- 172.21.15.52:0/3391481866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6f4101a90 0x7fc6f41982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.639+0000 7fc6f897d640 1 -- 172.21.15.52:0/3391481866 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f419d2b0 con 0x7fc6f4102450 2024-04-02T14:02:51.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.639+0000 7fc6f897d640 1 --2- 172.21.15.52:0/3391481866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f4102450 0x7fc6f4198800 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400c920 tx=0x7fc6e400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.639+0000 7fc6ea7fc640 1 -- 172.21.15.52:0/3391481866 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4007c70 con 0x7fc6f4102450 2024-04-02T14:02:51.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.639+0000 7fc6ea7fc640 1 -- 172.21.15.52:0/3391481866 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6e4007e10 con 0x7fc6f4102450 2024-04-02T14:02:51.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.639+0000 7fc6fb3e2640 1 -- 172.21.15.52:0/3391481866 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f419d5a0 con 0x7fc6f4102450 2024-04-02T14:02:51.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.639+0000 7fc6ea7fc640 1 -- 172.21.15.52:0/3391481866 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4018890 con 0x7fc6f4102450 2024-04-02T14:02:51.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.639+0000 7fc6fb3e2640 1 -- 172.21.15.52:0/3391481866 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f41a4e30 con 0x7fc6f4102450 2024-04-02T14:02:51.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.641+0000 7fc6ea7fc640 1 -- 172.21.15.52:0/3391481866 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc6e4018a70 con 0x7fc6f4102450 2024-04-02T14:02:51.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.642+0000 7fc6fdfc6740 1 Processor -- start 2024-04-02T14:02:51.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.642+0000 7fc6fdfc6740 1 -- start start 2024-04-02T14:02:51.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.642+0000 7fc6fdfc6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 0x55ae95b91b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.642+0000 7fc6fdfc6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae95b920c0 0x55ae95b9a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.642+0000 7fc6ea7fc640 1 --2- 172.21.15.52:0/3391481866 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc6e00a3d20 0x7fc6e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.642+0000 7fc6fdfc6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae95b9aca0 0x55ae95b9d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.642+0000 7fc6fdfc6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ae95af57b0 con 0x55ae95b917a0 2024-04-02T14:02:51.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.642+0000 7fc6fdfc6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ae95a94570 con 0x55ae95b920c0 2024-04-02T14:02:51.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.642+0000 7fc6fdfc6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ae95af1760 con 0x55ae95b9aca0 2024-04-02T14:02:51.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.642+0000 7fc6f997f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 0x55ae95b91b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.642+0000 7fc6f897d640 1 --2- 172.21.15.52:0/3391481866 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc6e00a3d20 0x7fc6e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.642+0000 7fc6f997f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 0x55ae95b91b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46344/0 (socket says 172.21.15.52:46344) 2024-04-02T14:02:51.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.642+0000 7fc6f997f640 1 -- 172.21.15.52:0/2792677219 learned_addr learned my addr 172.21.15.52:0/2792677219 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:51.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.642+0000 7fc6ea7fc640 1 -- 172.21.15.52:0/3391481866 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7fc6e40d0c10 con 0x7fc6f4102450 2024-04-02T14:02:51.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.642+0000 7fc6f917e640 1 --2- 172.21.15.52:0/2792677219 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae95b920c0 0x55ae95b9a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.642+0000 7fc6f997f640 1 -- 172.21.15.52:0/2792677219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae95b9aca0 msgr2=0x55ae95b9d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.642+0000 7fc6f997f640 1 --2- 172.21.15.52:0/2792677219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae95b9aca0 0x55ae95b9d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.642+0000 7fc6f997f640 1 -- 172.21.15.52:0/2792677219 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae95b920c0 msgr2=0x55ae95b9a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.642+0000 7fc6f997f640 1 --2- 172.21.15.52:0/2792677219 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae95b920c0 0x55ae95b9a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.642+0000 7fc6f997f640 1 -- 172.21.15.52:0/2792677219 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae958093f0 con 0x55ae95b917a0 2024-04-02T14:02:51.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.642+0000 7fc6f917e640 1 --2- 172.21.15.52:0/2792677219 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae95b920c0 0x55ae95b9a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:02:51.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.643+0000 7fc6f897d640 1 --2- 172.21.15.52:0/3391481866 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc6e00a3d20 0x7fc6e00a61e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc6e40ce5d0 tx=0x7fc6e4087040 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.643+0000 7fc6f997f640 1 --2- 172.21.15.52:0/2792677219 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 0x55ae95b91b80 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec007f60 tx=0x7fc6ec007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ccb275787dcee0b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.643+0000 7fc6df7fe640 1 -- 172.21.15.52:0/2792677219 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec016070 con 0x55ae95b917a0 2024-04-02T14:02:51.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.643+0000 7fc6df7fe640 1 -- 172.21.15.52:0/2792677219 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6ec012bf0 con 0x55ae95b917a0 2024-04-02T14:02:51.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.643+0000 7fc6df7fe640 1 -- 172.21.15.52:0/2792677219 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec00ea40 con 0x55ae95b917a0 2024-04-02T14:02:51.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.643+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/2792677219 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 msgr2=0x55ae95b91b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.643+0000 7fc6fdfc6740 1 --2- 172.21.15.52:0/2792677219 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 0x55ae95b91b80 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec007f60 tx=0x7fc6ec007f90 comp rx=0 tx=0).stop 2024-04-02T14:02:51.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.643+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/2792677219 shutdown_connections 2024-04-02T14:02:51.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.643+0000 7fc6fdfc6740 1 --2- 172.21.15.52:0/2792677219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae95b9aca0 0x55ae95b9d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.643+0000 7fc6fdfc6740 1 --2- 172.21.15.52:0/2792677219 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae95b920c0 0x55ae95b9a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.643+0000 7fc6fdfc6740 1 --2- 172.21.15.52:0/2792677219 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 0x55ae95b91b80 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.643+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/2792677219 >> 172.21.15.52:0/2792677219 conn(0x55ae95b8bed0 msgr2=0x55ae95b8c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.644+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/2792677219 shutdown_connections 2024-04-02T14:02:51.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.644+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/2792677219 wait complete. 2024-04-02T14:02:51.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.644+0000 7fc6fdfc6740 1 -- start start 2024-04-02T14:02:51.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.644+0000 7fc6fdfc6740 1 Processor -- start 2024-04-02T14:02:51.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.644+0000 7fc6fdfc6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 0x55ae95b9a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.644+0000 7fc6fdfc6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae95b920c0 0x55ae95b97a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.644+0000 7fc6fdfc6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae95b9aca0 0x55ae95b97fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.644+0000 7fc6fdfc6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ae95b9da90 con 0x55ae95b917a0 2024-04-02T14:02:51.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.644+0000 7fc6fdfc6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ae95ab0e80 con 0x55ae95b920c0 2024-04-02T14:02:51.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.644+0000 7fc6fdfc6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ae95b9dc10 con 0x55ae95b9aca0 2024-04-02T14:02:51.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.644+0000 7fc6fdfc6740 10 client.0 _ll_register_callbacks cb 0x55ae95aecaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:51.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.644+0000 7fc6f997f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 0x55ae95b9a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.644+0000 7fc6f997f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 0x55ae95b9a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46354/0 (socket says 172.21.15.52:46354) 2024-04-02T14:02:51.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.644+0000 7fc6f997f640 1 -- 172.21.15.52:0/2728914545 learned_addr learned my addr 172.21.15.52:0/2728914545 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:51.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.645+0000 7fc6f997f640 1 -- 172.21.15.52:0/2728914545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae95b9aca0 msgr2=0x55ae95b97fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.645+0000 7fc6f997f640 1 --2- 172.21.15.52:0/2728914545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae95b9aca0 0x55ae95b97fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.645+0000 7fc6f917e640 1 --2- 172.21.15.52:0/2728914545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae95b920c0 0x55ae95b97a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.645+0000 7fc6f997f640 1 -- 172.21.15.52:0/2728914545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae95b920c0 msgr2=0x55ae95b97a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.645+0000 7fc6f997f640 1 --2- 172.21.15.52:0/2728914545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae95b920c0 0x55ae95b97a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.645+0000 7fc6f997f640 1 -- 172.21.15.52:0/2728914545 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae958d37d0 con 0x55ae95b917a0 2024-04-02T14:02:51.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.645+0000 7fc6f917e640 1 --2- 172.21.15.52:0/2728914545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae95b920c0 0x55ae95b97a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:02:51.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.645+0000 7fc6f997f640 1 --2- 172.21.15.52:0/2728914545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 0x55ae95b9a490 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec007580 tx=0x7fc6ec007310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da881bdcd1a1aff server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.645+0000 7fc6dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:51.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.645+0000 7fc6dcff9640 1 -- 172.21.15.52:0/2728914545 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec016070 con 0x55ae95b917a0 2024-04-02T14:02:51.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.645+0000 7fc6dcff9640 1 -- 172.21.15.52:0/2728914545 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7fc6ec012920 con 0x55ae95b917a0 2024-04-02T14:02:51.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.645+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/2728914545 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6ec012bf0 con 0x55ae95b917a0 2024-04-02T14:02:51.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.645+0000 7fc6dcff9640 1 -- 172.21.15.52:0/2728914545 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc6ec007be0 con 0x55ae95b917a0 2024-04-02T14:02:51.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.645+0000 7fc6fdfc6740 10 client.18282 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:51.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.645+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/2728914545 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ae95bb2370 con 0x55ae95b917a0 2024-04-02T14:02:51.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.645+0000 7fc6dcff9640 1 -- 172.21.15.52:0/2728914545 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6ec012bf0 con 0x55ae95b917a0 2024-04-02T14:02:51.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.646+0000 7fc6dcff9640 1 -- 172.21.15.52:0/2728914545 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc6ec005640 con 0x55ae95b917a0 2024-04-02T14:02:51.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.646+0000 7fc6fdfc6740 10 client.18282 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:51.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.646+0000 7fc6fdfc6740 10 client.18282 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T14:02:51.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.646+0000 7fc6fdfc6740 20 client.18282 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:51.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.646+0000 7fc6fdfc6740 1 --2- 172.21.15.52:0/2728914545 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55ae95bb3aa0 0x55ae95bb5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.646+0000 7fc6fdfc6740 4 client.18282 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:02:51.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.646+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/2728914545 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae95b02cf0 con 0x55ae95bb3aa0 2024-04-02T14:02:51.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.646+0000 7fc6f917e640 1 --2- 172.21.15.52:0/2728914545 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55ae95bb3aa0 0x55ae95bb5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.646+0000 7fc6f917e640 1 --2- 172.21.15.52:0/2728914545 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55ae95bb3aa0 0x55ae95bb5e90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd6a43d8c172c946 server_cookie=24a8fa5d10f7a57 in_seq=0 out_seq=0 2024-04-02T14:02:51.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.646+0000 7fc6dcff9640 10 client.18282 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T14:02:51.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.647+0000 7fc6dcff9640 1 -- 172.21.15.52:0/2728914545 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ae95b02cf0 con 0x55ae95bb3aa0 2024-04-02T14:02:51.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.647+0000 7fc6dcff9640 10 client.18282 handle_command_reply: tid=0 2024-04-02T14:02:51.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.647+0000 7fc6fdfc6740 1 client.18282 shutdown 2024-04-02T14:02:51.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.647+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/2728914545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 msgr2=0x55ae95b9a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.647+0000 7fc6fdfc6740 1 --2- 172.21.15.52:0/2728914545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 0x55ae95b9a490 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec007580 tx=0x7fc6ec007310 comp rx=0 tx=0).stop 2024-04-02T14:02:51.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.647+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/2728914545 shutdown_connections 2024-04-02T14:02:51.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.647+0000 7fc6fdfc6740 1 --2- 172.21.15.52:0/2728914545 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55ae95bb3aa0 0x55ae95bb5e90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.647+0000 7fc6dcff9640 0 client.18282 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T14:02:51.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.647+0000 7fc6fdfc6740 1 --2- 172.21.15.52:0/2728914545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae95b9aca0 0x55ae95b97fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.647+0000 7fc6fdfc6740 1 --2- 172.21.15.52:0/2728914545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae95b920c0 0x55ae95b97a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.647+0000 7fc6fdfc6740 1 --2- 172.21.15.52:0/2728914545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 0x55ae95b9a490 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.647+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/2728914545 >> 172.21.15.52:0/2728914545 conn(0x55ae95b8bed0 msgr2=0x55ae95b8dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.647+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/2728914545 shutdown_connections 2024-04-02T14:02:51.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.647+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/2728914545 wait complete. 2024-04-02T14:02:51.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.647+0000 7fc6fdfc6740 20 client.18282 trim_cache size 0 max 16384 2024-04-02T14:02:51.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.661+0000 7fc6fdfc6740 1 Processor -- start 2024-04-02T14:02:51.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.661+0000 7fc6fdfc6740 1 -- start start 2024-04-02T14:02:51.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.661+0000 7fc6fdfc6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 0x55ae95ba0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.661+0000 7fc6fdfc6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae95b920c0 0x55ae95b9a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.661+0000 7fc6fdfc6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae95b9aca0 0x55ae95b94ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.661+0000 7fc6fdfc6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f41acc80 con 0x55ae95b917a0 2024-04-02T14:02:51.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.661+0000 7fc6fdfc6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f41ae7c0 con 0x55ae95b9aca0 2024-04-02T14:02:51.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.661+0000 7fc6fdfc6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f41ad350 con 0x55ae95b920c0 2024-04-02T14:02:51.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.661+0000 7fc6f997f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 0x55ae95ba0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.661+0000 7fc6f997f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 0x55ae95ba0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46360/0 (socket says 172.21.15.52:46360) 2024-04-02T14:02:51.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.661+0000 7fc6f997f640 1 -- 172.21.15.52:0/2511238035 learned_addr learned my addr 172.21.15.52:0/2511238035 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:51.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.661+0000 7fc6f997f640 1 --2- 172.21.15.52:0/2511238035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae95b9aca0 0x55ae95b94ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.661+0000 7fc6f917e640 1 --2- 172.21.15.52:0/2511238035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae95b920c0 0x55ae95b9a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.661+0000 7fc6f997f640 1 -- 172.21.15.52:0/2511238035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae95b920c0 msgr2=0x55ae95b9a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.661+0000 7fc6f997f640 1 --2- 172.21.15.52:0/2511238035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae95b920c0 0x55ae95b9a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.661+0000 7fc6f997f640 1 -- 172.21.15.52:0/2511238035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae95b9aca0 msgr2=0x55ae95b94ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.661+0000 7fc6f997f640 1 --2- 172.21.15.52:0/2511238035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae95b9aca0 0x55ae95b94ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.661+0000 7fc6f997f640 1 -- 172.21.15.52:0/2511238035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae95b91ef0 con 0x55ae95b917a0 2024-04-02T14:02:51.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.661+0000 7fc6f997f640 1 --2- 172.21.15.52:0/2511238035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 0x55ae95ba0ea0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec004f70 tx=0x7fc6ec011300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f8a9ca4ec71efb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.662+0000 7fc6deffd640 1 -- 172.21.15.52:0/2511238035 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec016070 con 0x55ae95b917a0 2024-04-02T14:02:51.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.662+0000 7fc6deffd640 1 -- 172.21.15.52:0/2511238035 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6ec022070 con 0x55ae95b917a0 2024-04-02T14:02:51.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.662+0000 7fc6deffd640 1 -- 172.21.15.52:0/2511238035 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec005000 con 0x55ae95b917a0 2024-04-02T14:02:51.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.662+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/2511238035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 msgr2=0x55ae95ba0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.662+0000 7fc6fdfc6740 1 --2- 172.21.15.52:0/2511238035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 0x55ae95ba0ea0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec004f70 tx=0x7fc6ec011300 comp rx=0 tx=0).stop 2024-04-02T14:02:51.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.662+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/2511238035 shutdown_connections 2024-04-02T14:02:51.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.662+0000 7fc6fdfc6740 1 --2- 172.21.15.52:0/2511238035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae95b9aca0 0x55ae95b94ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.662+0000 7fc6fdfc6740 1 --2- 172.21.15.52:0/2511238035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae95b920c0 0x55ae95b9a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.662+0000 7fc6fdfc6740 1 --2- 172.21.15.52:0/2511238035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 0x55ae95ba0ea0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.662+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/2511238035 >> 172.21.15.52:0/2511238035 conn(0x55ae95b8bed0 msgr2=0x55ae95b92d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.662+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/2511238035 shutdown_connections 2024-04-02T14:02:51.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.662+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/2511238035 wait complete. 2024-04-02T14:02:51.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.662+0000 7fc6fdfc6740 1 -- start start 2024-04-02T14:02:51.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.663+0000 7fc6fdfc6740 1 Processor -- start 2024-04-02T14:02:51.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.663+0000 7fc6fdfc6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 0x55ae95bba010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.663+0000 7fc6fdfc6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae95b920c0 0x55ae95bba550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.663+0000 7fc6fdfc6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae95b9aca0 0x55ae95bbaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.663+0000 7fc6fdfc6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f41ae0f0 con 0x55ae95b917a0 2024-04-02T14:02:51.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.663+0000 7fc6fdfc6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f41ada20 con 0x55ae95b9aca0 2024-04-02T14:02:51.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.663+0000 7fc6fdfc6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f41aac80 con 0x55ae95b920c0 2024-04-02T14:02:51.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.663+0000 7fc6fdfc6740 10 client.0 _ll_register_callbacks cb 0x7fc6ec007e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:51.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.663+0000 7fc6f997f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 0x55ae95bba010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.663+0000 7fc6f997f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 0x55ae95bba010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46370/0 (socket says 172.21.15.52:46370) 2024-04-02T14:02:51.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.663+0000 7fc6f997f640 1 -- 172.21.15.52:0/133797898 learned_addr learned my addr 172.21.15.52:0/133797898 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:51.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.663+0000 7fc6f997f640 1 --2- 172.21.15.52:0/133797898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae95b9aca0 0x55ae95bbaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.663+0000 7fc6f917e640 1 --2- 172.21.15.52:0/133797898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae95b920c0 0x55ae95bba550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.663+0000 7fc6f997f640 1 -- 172.21.15.52:0/133797898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae95b920c0 msgr2=0x55ae95bba550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.663+0000 7fc6f997f640 1 --2- 172.21.15.52:0/133797898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae95b920c0 0x55ae95bba550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.663+0000 7fc6f997f640 1 -- 172.21.15.52:0/133797898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae95b9aca0 msgr2=0x55ae95bbaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.663+0000 7fc6f997f640 1 --2- 172.21.15.52:0/133797898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae95b9aca0 0x55ae95bbaa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.663+0000 7fc6f997f640 1 -- 172.21.15.52:0/133797898 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae95baedd0 con 0x55ae95b917a0 2024-04-02T14:02:51.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.664+0000 7fc6f997f640 1 --2- 172.21.15.52:0/133797898 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 0x55ae95bba010 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec007390 tx=0x7fc6ec01bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e26ef5d03db95a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.664+0000 7fc6dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:51.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.664+0000 7fc6dcff9640 1 -- 172.21.15.52:0/133797898 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec016070 con 0x55ae95b917a0 2024-04-02T14:02:51.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.664+0000 7fc6dcff9640 1 -- 172.21.15.52:0/133797898 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7fc6ec012920 con 0x55ae95b917a0 2024-04-02T14:02:51.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.664+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/133797898 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6ec022070 con 0x55ae95b917a0 2024-04-02T14:02:51.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.664+0000 7fc6dcff9640 1 -- 172.21.15.52:0/133797898 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc6ec012b20 con 0x55ae95b917a0 2024-04-02T14:02:51.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.664+0000 7fc6fdfc6740 10 client.18288 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:51.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.664+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/133797898 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ae95bb0090 con 0x55ae95b917a0 2024-04-02T14:02:51.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.664+0000 7fc6dcff9640 1 -- 172.21.15.52:0/133797898 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6ec022070 con 0x55ae95b917a0 2024-04-02T14:02:51.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.664+0000 7fc6dcff9640 1 -- 172.21.15.52:0/133797898 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc6ec038040 con 0x55ae95b917a0 2024-04-02T14:02:51.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.664+0000 7fc6fdfc6740 10 client.18288 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:51.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.664+0000 7fc6fdfc6740 10 client.18288 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T14:02:51.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.664+0000 7fc6fdfc6740 20 client.18288 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:51.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.664+0000 7fc6fdfc6740 1 --2- 172.21.15.52:0/133797898 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55ae95c19c70 0x55ae95c1c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.664+0000 7fc6fdfc6740 4 client.18288 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:02:51.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.664+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/133797898 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae95afd040 con 0x55ae95c19c70 2024-04-02T14:02:51.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.665+0000 7fc6f917e640 1 --2- 172.21.15.52:0/133797898 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55ae95c19c70 0x55ae95c1c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.665+0000 7fc6dcff9640 10 client.18288 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T14:02:51.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.665+0000 7fc6f917e640 1 --2- 172.21.15.52:0/133797898 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55ae95c19c70 0x55ae95c1c060 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b3394ef3a1fa31ea server_cookie=3b1c81222c52bb68 in_seq=0 out_seq=0 2024-04-02T14:02:51.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.665+0000 7fc6dcff9640 1 -- 172.21.15.52:0/133797898 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae95afd040 con 0x55ae95c19c70 2024-04-02T14:02:51.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.665+0000 7fc6dcff9640 10 client.18288 handle_command_reply: tid=0 2024-04-02T14:02:51.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.665+0000 7fc6fdfc6740 1 client.18288 shutdown 2024-04-02T14:02:51.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.665+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/133797898 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 msgr2=0x55ae95bba010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.665+0000 7fc6fdfc6740 1 --2- 172.21.15.52:0/133797898 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 0x55ae95bba010 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec007390 tx=0x7fc6ec01bc70 comp rx=0 tx=0).stop 2024-04-02T14:02:51.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.665+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/133797898 shutdown_connections 2024-04-02T14:02:51.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.665+0000 7fc6fdfc6740 1 --2- 172.21.15.52:0/133797898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae95b9aca0 0x55ae95bbaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.665+0000 7fc6fdfc6740 1 --2- 172.21.15.52:0/133797898 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55ae95c19c70 0x55ae95c1c060 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.665+0000 7fc6fdfc6740 1 --2- 172.21.15.52:0/133797898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae95b920c0 0x55ae95bba550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.665+0000 7fc6fdfc6740 1 --2- 172.21.15.52:0/133797898 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae95b917a0 0x55ae95bba010 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.665+0000 7fc6dcff9640 0 client.18288 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T14:02:51.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.665+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/133797898 >> 172.21.15.52:0/133797898 conn(0x55ae95b8bed0 msgr2=0x55ae95b8d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.665+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/133797898 shutdown_connections 2024-04-02T14:02:51.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.665+0000 7fc6fdfc6740 1 -- 172.21.15.52:0/133797898 wait complete. 2024-04-02T14:02:51.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.666+0000 7fc6fdfc6740 20 client.18288 trim_cache size 0 max 16384 2024-04-02T14:02:51.681 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:02:51.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.666+0000 7fc6fb3e2640 1 -- 172.21.15.52:0/3391481866 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc6e00a3d20 msgr2=0x7fc6e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.666+0000 7fc6fb3e2640 1 --2- 172.21.15.52:0/3391481866 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc6e00a3d20 0x7fc6e00a61e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc6e40ce5d0 tx=0x7fc6e4087040 comp rx=0 tx=0).stop 2024-04-02T14:02:51.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.666+0000 7fc6fb3e2640 1 -- 172.21.15.52:0/3391481866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f4102450 msgr2=0x7fc6f4198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.666+0000 7fc6fb3e2640 1 --2- 172.21.15.52:0/3391481866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f4102450 0x7fc6f4198800 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400c920 tx=0x7fc6e400cde0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.666+0000 7fc6fb3e2640 1 -- 172.21.15.52:0/3391481866 shutdown_connections 2024-04-02T14:02:51.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.666+0000 7fc6fb3e2640 1 --2- 172.21.15.52:0/3391481866 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc6e00a3d20 0x7fc6e00a61e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.666+0000 7fc6fb3e2640 1 --2- 172.21.15.52:0/3391481866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6f4107aa0 0x7fc6f419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.666+0000 7fc6fb3e2640 1 --2- 172.21.15.52:0/3391481866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f4102450 0x7fc6f4198800 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.666+0000 7fc6fb3e2640 1 --2- 172.21.15.52:0/3391481866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6f4101a90 0x7fc6f41982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.666+0000 7fc6fb3e2640 1 -- 172.21.15.52:0/3391481866 >> 172.21.15.52:0/3391481866 conn(0x7fc6f4078f00 msgr2=0x7fc6f41042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.666+0000 7fc6fb3e2640 1 -- 172.21.15.52:0/3391481866 shutdown_connections 2024-04-02T14:02:51.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.666+0000 7fc6fb3e2640 1 -- 172.21.15.52:0/3391481866 wait complete. 2024-04-02T14:02:51.683 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:51 smithi052 ceph-mon[32104]: pgmap v289: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:02:51.683 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:51 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/4220841741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:02:51.684 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T14:02:51.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:51 smithi189 ceph-mon[36683]: pgmap v289: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:02:51.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:51 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/4220841741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:02:51.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.785+0000 7f5e7c904640 1 -- 172.21.15.52:0/2667261762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e78107aa0 msgr2=0x7f5e78107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.785+0000 7f5e7c904640 1 --2- 172.21.15.52:0/2667261762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e78107aa0 0x7f5e78107e80 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f5e68009930 tx=0x7f5e6802bc30 comp rx=0 tx=0).stop 2024-04-02T14:02:51.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.785+0000 7f5e7c904640 1 -- 172.21.15.52:0/2667261762 shutdown_connections 2024-04-02T14:02:51.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.785+0000 7f5e7c904640 1 --2- 172.21.15.52:0/2667261762 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e78102450 0x7f5e7810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.785+0000 7f5e7c904640 1 --2- 172.21.15.52:0/2667261762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e78101a90 0x7f5e78101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.785+0000 7f5e7c904640 1 --2- 172.21.15.52:0/2667261762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e78107aa0 0x7f5e78107e80 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.785+0000 7f5e7c904640 1 -- 172.21.15.52:0/2667261762 >> 172.21.15.52:0/2667261762 conn(0x7f5e78078f00 msgr2=0x7f5e78079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.785+0000 7f5e7c904640 1 -- 172.21.15.52:0/2667261762 shutdown_connections 2024-04-02T14:02:51.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.785+0000 7f5e7c904640 1 -- 172.21.15.52:0/2667261762 wait complete. 2024-04-02T14:02:51.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.786+0000 7f5e7c904640 1 Processor -- start 2024-04-02T14:02:51.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.786+0000 7f5e7c904640 1 -- start start 2024-04-02T14:02:51.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.786+0000 7f5e7c904640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e78101a90 0x7f5e781983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.786+0000 7f5e7c904640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e78102450 0x7f5e781988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.786+0000 7f5e7c904640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e78107aa0 0x7f5e7819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.786+0000 7f5e7c904640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5e78111eb0 con 0x7f5e78101a90 2024-04-02T14:02:51.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.786+0000 7f5e7c904640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5e78111d30 con 0x7f5e78107aa0 2024-04-02T14:02:51.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.786+0000 7f5e7c904640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5e78112030 con 0x7f5e78102450 2024-04-02T14:02:51.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.786+0000 7f5e7659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e78101a90 0x7f5e781983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.786+0000 7f5e7659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e78101a90 0x7f5e781983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46384/0 (socket says 172.21.15.52:46384) 2024-04-02T14:02:51.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.786+0000 7f5e7659c640 1 -- 172.21.15.52:0/1547094900 learned_addr learned my addr 172.21.15.52:0/1547094900 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:51.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.786+0000 7f5e76d9d640 1 --2- 172.21.15.52:0/1547094900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e78107aa0 0x7f5e7819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.786+0000 7f5e6ffff640 1 --2- 172.21.15.52:0/1547094900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e78102450 0x7f5e781988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.786+0000 7f5e7659c640 1 -- 172.21.15.52:0/1547094900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e78102450 msgr2=0x7f5e781988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.786+0000 7f5e7659c640 1 --2- 172.21.15.52:0/1547094900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e78102450 0x7f5e781988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.786+0000 7f5e7659c640 1 -- 172.21.15.52:0/1547094900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e78107aa0 msgr2=0x7f5e7819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.786+0000 7f5e7659c640 1 --2- 172.21.15.52:0/1547094900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e78107aa0 0x7f5e7819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.786+0000 7f5e7659c640 1 -- 172.21.15.52:0/1547094900 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7819d3a0 con 0x7f5e78101a90 2024-04-02T14:02:51.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.787+0000 7f5e7659c640 1 --2- 172.21.15.52:0/1547094900 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e78101a90 0x7f5e781983b0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f5e68009230 tx=0x7f5e6802cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.787+0000 7f5e6f7fe640 1 -- 172.21.15.52:0/1547094900 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e68041070 con 0x7f5e78101a90 2024-04-02T14:02:51.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.787+0000 7f5e6f7fe640 1 -- 172.21.15.52:0/1547094900 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e68004450 con 0x7f5e78101a90 2024-04-02T14:02:51.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.787+0000 7f5e7c904640 1 -- 172.21.15.52:0/1547094900 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e7819d630 con 0x7f5e78101a90 2024-04-02T14:02:51.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.787+0000 7f5e6f7fe640 1 -- 172.21.15.52:0/1547094900 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e68034440 con 0x7f5e78101a90 2024-04-02T14:02:51.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.787+0000 7f5e7c904640 1 -- 172.21.15.52:0/1547094900 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e781a4f20 con 0x7f5e78101a90 2024-04-02T14:02:51.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.790+0000 7f5e6f7fe640 1 -- 172.21.15.52:0/1547094900 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5e68044070 con 0x7f5e78101a90 2024-04-02T14:02:51.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.790+0000 7f5e7f5c8740 1 Processor -- start 2024-04-02T14:02:51.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.790+0000 7f5e7f5c8740 1 -- start start 2024-04-02T14:02:51.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.790+0000 7f5e7f5c8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c17b817a0 0x558c17b81b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.790+0000 7f5e7f5c8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c17b820c0 0x558c17b8a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.790+0000 7f5e7f5c8740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b8aca0 0x558c17b8d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.790+0000 7f5e7f5c8740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558c17ae57b0 con 0x558c17b8aca0 2024-04-02T14:02:51.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.790+0000 7f5e7f5c8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x558c17a84570 con 0x558c17b817a0 2024-04-02T14:02:51.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.790+0000 7f5e7f5c8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558c17ae1760 con 0x558c17b820c0 2024-04-02T14:02:51.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.790+0000 7f5e6ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b8aca0 0x558c17b8d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.790+0000 7f5e6ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b8aca0 0x558c17b8d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46396/0 (socket says 172.21.15.52:46396) 2024-04-02T14:02:51.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.790+0000 7f5e6ffff640 1 -- 172.21.15.52:0/2308675274 learned_addr learned my addr 172.21.15.52:0/2308675274 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:51.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.791+0000 7f5e6f7fe640 1 --2- 172.21.15.52:0/1547094900 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5e640a3d20 0x7f5e640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.791+0000 7f5e7659c640 1 --2- 172.21.15.52:0/2308675274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c17b820c0 0x558c17b8a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.791+0000 7f5e76d9d640 1 --2- 172.21.15.52:0/2308675274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c17b817a0 0x558c17b81b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.791+0000 7f5e6ffff640 1 -- 172.21.15.52:0/2308675274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c17b820c0 msgr2=0x558c17b8a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.791+0000 7f5e6ffff640 1 --2- 172.21.15.52:0/2308675274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c17b820c0 0x558c17b8a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.791+0000 7f5e6ffff640 1 -- 172.21.15.52:0/2308675274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c17b817a0 msgr2=0x558c17b81b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.791+0000 7f5e6ffff640 1 --2- 172.21.15.52:0/2308675274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c17b817a0 0x558c17b81b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.791+0000 7f5e6ffff640 1 -- 172.21.15.52:0/2308675274 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c177f93f0 con 0x558c17b8aca0 2024-04-02T14:02:51.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.791+0000 7f5e76d9d640 1 --2- 172.21.15.52:0/2308675274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c17b817a0 0x558c17b81b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:02:51.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.791+0000 7f5e76d9d640 1 --2- 172.21.15.52:0/1547094900 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5e640a3d20 0x7f5e640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.791+0000 7f5e6f7fe640 1 -- 172.21.15.52:0/1547094900 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f5e680f2420 con 0x7f5e78101a90 2024-04-02T14:02:51.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.791+0000 7f5e6ffff640 1 --2- 172.21.15.52:0/2308675274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b8aca0 0x558c17b8d090 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000bcb0 tx=0x7f5e600107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2dfa2f1050a2784 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.791+0000 7f5e6cff9640 1 -- 172.21.15.52:0/2308675274 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e60004030 con 0x558c17b8aca0 2024-04-02T14:02:51.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.791+0000 7f5e6cff9640 1 -- 172.21.15.52:0/2308675274 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e600041d0 con 0x558c17b8aca0 2024-04-02T14:02:51.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.791+0000 7f5e6cff9640 1 -- 172.21.15.52:0/2308675274 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e60011840 con 0x558c17b8aca0 2024-04-02T14:02:51.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.791+0000 7f5e76d9d640 1 --2- 172.21.15.52:0/1547094900 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5e640a3d20 0x7f5e640a61e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5e70005fd0 tx=0x7f5e70004a40 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.792+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/2308675274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b8aca0 msgr2=0x558c17b8d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.792+0000 7f5e7f5c8740 1 --2- 172.21.15.52:0/2308675274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b8aca0 0x558c17b8d090 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000bcb0 tx=0x7f5e600107a0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.792+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/2308675274 shutdown_connections 2024-04-02T14:02:51.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.792+0000 7f5e7f5c8740 1 --2- 172.21.15.52:0/2308675274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b8aca0 0x558c17b8d090 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.792+0000 7f5e7f5c8740 1 --2- 172.21.15.52:0/2308675274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c17b820c0 0x558c17b8a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.792+0000 7f5e7f5c8740 1 --2- 172.21.15.52:0/2308675274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c17b817a0 0x558c17b81b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.792+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/2308675274 >> 172.21.15.52:0/2308675274 conn(0x558c17b7bed0 msgr2=0x558c17b7c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.792+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/2308675274 shutdown_connections 2024-04-02T14:02:51.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.792+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/2308675274 wait complete. 2024-04-02T14:02:51.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.792+0000 7f5e7f5c8740 1 -- start start 2024-04-02T14:02:51.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.792+0000 7f5e7f5c8740 1 Processor -- start 2024-04-02T14:02:51.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.793+0000 7f5e7f5c8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c17b817a0 0x558c17b8a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.793+0000 7f5e7f5c8740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b820c0 0x558c17b879f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.793+0000 7f5e7f5c8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c17b8aca0 0x558c17b88000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.793+0000 7f5e7f5c8740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558c17b8da90 con 0x558c17b820c0 2024-04-02T14:02:51.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.793+0000 7f5e7f5c8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x558c17aa0e80 con 0x558c17b817a0 2024-04-02T14:02:51.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.793+0000 7f5e7f5c8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558c17b8dc10 con 0x558c17b8aca0 2024-04-02T14:02:51.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.793+0000 7f5e7f5c8740 10 client.0 _ll_register_callbacks cb 0x558c17adcaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:51.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.793+0000 7f5e6ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b820c0 0x558c17b879f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.793+0000 7f5e6ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b820c0 0x558c17b879f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46400/0 (socket says 172.21.15.52:46400) 2024-04-02T14:02:51.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.793+0000 7f5e6ffff640 1 -- 172.21.15.52:0/1037428861 learned_addr learned my addr 172.21.15.52:0/1037428861 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:51.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.793+0000 7f5e76d9d640 1 --2- 172.21.15.52:0/1037428861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c17b8aca0 0x558c17b88000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.793+0000 7f5e7659c640 1 --2- 172.21.15.52:0/1037428861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c17b817a0 0x558c17b8a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.793+0000 7f5e6ffff640 1 -- 172.21.15.52:0/1037428861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c17b8aca0 msgr2=0x558c17b88000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.793+0000 7f5e6ffff640 1 --2- 172.21.15.52:0/1037428861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c17b8aca0 0x558c17b88000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.793+0000 7f5e6ffff640 1 -- 172.21.15.52:0/1037428861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c17b817a0 msgr2=0x558c17b8a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.793+0000 7f5e6ffff640 1 --2- 172.21.15.52:0/1037428861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c17b817a0 0x558c17b8a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.793+0000 7f5e6ffff640 1 -- 172.21.15.52:0/1037428861 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c178c37d0 con 0x558c17b820c0 2024-04-02T14:02:51.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.793+0000 7f5e7659c640 1 --2- 172.21.15.52:0/1037428861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c17b817a0 0x558c17b8a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:02:51.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.793+0000 7f5e76d9d640 1 --2- 172.21.15.52:0/1037428861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c17b8aca0 0x558c17b88000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:02:51.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.793+0000 7f5e6ffff640 1 --2- 172.21.15.52:0/1037428861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b820c0 0x558c17b879f0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000b8e0 tx=0x7f5e6000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b487dcedb559810 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.793+0000 7f5e5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:51.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.793+0000 7f5e5dffb640 1 -- 172.21.15.52:0/1037428861 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6002a060 con 0x558c17b820c0 2024-04-02T14:02:51.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.793+0000 7f5e5dffb640 1 -- 172.21.15.52:0/1037428861 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f5e600035f0 con 0x558c17b820c0 2024-04-02T14:02:51.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.793+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/1037428861 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e600041d0 con 0x558c17b820c0 2024-04-02T14:02:51.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.794+0000 7f5e5dffb640 1 -- 172.21.15.52:0/1037428861 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5e60031050 con 0x558c17b820c0 2024-04-02T14:02:51.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.794+0000 7f5e7f5c8740 10 client.18300 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:51.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.794+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/1037428861 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558c17b99a30 con 0x558c17b820c0 2024-04-02T14:02:51.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.794+0000 7f5e5dffb640 1 -- 172.21.15.52:0/1037428861 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e600041d0 con 0x558c17b820c0 2024-04-02T14:02:51.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.794+0000 7f5e5dffb640 1 -- 172.21.15.52:0/1037428861 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5e60004880 con 0x558c17b820c0 2024-04-02T14:02:51.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.794+0000 7f5e7f5c8740 10 client.18300 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:51.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.794+0000 7f5e7f5c8740 10 client.18300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:02:51.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.794+0000 7f5e7f5c8740 20 client.18300 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:51.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.794+0000 7f5e7f5c8740 1 --2- 172.21.15.52:0/1037428861 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x558c17b9ade0 0x558c17b9d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.794+0000 7f5e7f5c8740 4 client.18300 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:02:51.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.794+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/1037428861 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c17af2cf0 con 0x558c17b9ade0 2024-04-02T14:02:51.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.794+0000 7f5e7659c640 1 --2- 172.21.15.52:0/1037428861 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x558c17b9ade0 0x558c17b9d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.795+0000 7f5e7659c640 1 --2- 172.21.15.52:0/1037428861 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x558c17b9ade0 0x558c17b9d1d0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5d844bb9bb38e99 server_cookie=e8a96f9b751a963d in_seq=0 out_seq=0 2024-04-02T14:02:51.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.795+0000 7f5e5dffb640 10 client.18300 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:51.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.795+0000 7f5e5dffb640 1 -- 172.21.15.52:0/1037428861 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558c17af2cf0 con 0x558c17b9ade0 2024-04-02T14:02:51.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.795+0000 7f5e5dffb640 10 client.18300 handle_command_reply: tid=0 2024-04-02T14:02:51.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.795+0000 7f5e7f5c8740 1 client.18300 shutdown 2024-04-02T14:02:51.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.796+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/1037428861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b820c0 msgr2=0x558c17b879f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.796+0000 7f5e7f5c8740 1 --2- 172.21.15.52:0/1037428861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b820c0 0x558c17b879f0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000b8e0 tx=0x7f5e6000b910 comp rx=0 tx=0).stop 2024-04-02T14:02:51.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.796+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/1037428861 shutdown_connections 2024-04-02T14:02:51.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.796+0000 7f5e7f5c8740 1 --2- 172.21.15.52:0/1037428861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c17b8aca0 0x558c17b88000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.796+0000 7f5e7f5c8740 1 --2- 172.21.15.52:0/1037428861 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x558c17b9ade0 0x558c17b9d1d0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.796+0000 7f5e7f5c8740 1 --2- 172.21.15.52:0/1037428861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b820c0 0x558c17b879f0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.796+0000 7f5e5dffb640 0 client.18300 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:51.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.796+0000 7f5e7f5c8740 1 --2- 172.21.15.52:0/1037428861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c17b817a0 0x558c17b8a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.796+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/1037428861 >> 172.21.15.52:0/1037428861 conn(0x558c17b7bed0 msgr2=0x558c17b7dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.796+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/1037428861 shutdown_connections 2024-04-02T14:02:51.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.796+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/1037428861 wait complete. 2024-04-02T14:02:51.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.796+0000 7f5e7f5c8740 20 client.18300 trim_cache size 0 max 16384 2024-04-02T14:02:51.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.809+0000 7f5e7f5c8740 1 Processor -- start 2024-04-02T14:02:51.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.809+0000 7f5e7f5c8740 1 -- start start 2024-04-02T14:02:51.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.809+0000 7f5e7f5c8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c17b817a0 0x558c17b969b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.809+0000 7f5e7f5c8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c17b820c0 0x558c17b869d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.809+0000 7f5e7f5c8740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b8aca0 0x558c17b8a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.809+0000 7f5e7f5c8740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5e781aef90 con 0x558c17b8aca0 2024-04-02T14:02:51.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.809+0000 7f5e7f5c8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5e781b0ad0 con 0x558c17b820c0 2024-04-02T14:02:51.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.809+0000 7f5e7f5c8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5e781af660 con 0x558c17b817a0 2024-04-02T14:02:51.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.809+0000 7f5e76d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b8aca0 0x558c17b8a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.809+0000 7f5e76d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b8aca0 0x558c17b8a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46408/0 (socket says 172.21.15.52:46408) 2024-04-02T14:02:51.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.809+0000 7f5e76d9d640 1 -- 172.21.15.52:0/1954010903 learned_addr learned my addr 172.21.15.52:0/1954010903 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:51.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.809+0000 7f5e76d9d640 1 -- 172.21.15.52:0/1954010903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c17b817a0 msgr2=0x558c17b969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.809+0000 7f5e76d9d640 1 --2- 172.21.15.52:0/1954010903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c17b817a0 0x558c17b969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.809+0000 7f5e76d9d640 1 -- 172.21.15.52:0/1954010903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c17b820c0 msgr2=0x558c17b869d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.809+0000 7f5e76d9d640 1 --2- 172.21.15.52:0/1954010903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c17b820c0 0x558c17b869d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.809+0000 7f5e76d9d640 1 -- 172.21.15.52:0/1954010903 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c17b88370 con 0x558c17b8aca0 2024-04-02T14:02:51.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.810+0000 7f5e76d9d640 1 --2- 172.21.15.52:0/1954010903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b8aca0 0x558c17b8a120 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f5e7001e7e0 tx=0x7f5e70016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56905f57e686f6d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.810+0000 7f5e5ffff640 1 -- 172.21.15.52:0/1954010903 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7002c070 con 0x558c17b8aca0 2024-04-02T14:02:51.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.810+0000 7f5e5ffff640 1 -- 172.21.15.52:0/1954010903 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e700273f0 con 0x558c17b8aca0 2024-04-02T14:02:51.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.810+0000 7f5e5ffff640 1 -- 172.21.15.52:0/1954010903 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7000f3c0 con 0x558c17b8aca0 2024-04-02T14:02:51.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.810+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/1954010903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b8aca0 msgr2=0x558c17b8a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.810+0000 7f5e7f5c8740 1 --2- 172.21.15.52:0/1954010903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b8aca0 0x558c17b8a120 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f5e7001e7e0 tx=0x7f5e70016000 comp rx=0 tx=0).stop 2024-04-02T14:02:51.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.810+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/1954010903 shutdown_connections 2024-04-02T14:02:51.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.810+0000 7f5e7f5c8740 1 --2- 172.21.15.52:0/1954010903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b8aca0 0x558c17b8a120 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.810+0000 7f5e7f5c8740 1 --2- 172.21.15.52:0/1954010903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c17b820c0 0x558c17b869d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.810+0000 7f5e7f5c8740 1 --2- 172.21.15.52:0/1954010903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c17b817a0 0x558c17b969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.810+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/1954010903 >> 172.21.15.52:0/1954010903 conn(0x558c17b7bed0 msgr2=0x558c17b862c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.810+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/1954010903 shutdown_connections 2024-04-02T14:02:51.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.810+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/1954010903 wait complete. 2024-04-02T14:02:51.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.811+0000 7f5e7f5c8740 1 -- start start 2024-04-02T14:02:51.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.811+0000 7f5e7f5c8740 1 Processor -- start 2024-04-02T14:02:51.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.811+0000 7f5e7f5c8740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b817a0 0x558c17b8bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.811+0000 7f5e7f5c8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c17b820c0 0x558c17b8c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.811+0000 7f5e7f5c8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c17b8aca0 0x558c17b8c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.811+0000 7f5e7f5c8740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5e781b0400 con 0x558c17b817a0 2024-04-02T14:02:51.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.811+0000 7f5e7f5c8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5e781afd30 con 0x558c17b820c0 2024-04-02T14:02:51.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.811+0000 7f5e7f5c8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5e781acf90 con 0x558c17b8aca0 2024-04-02T14:02:51.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.811+0000 7f5e7f5c8740 10 client.0 _ll_register_callbacks cb 0x7f5e6000bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:51.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.811+0000 7f5e7659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b817a0 0x558c17b8bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.811+0000 7f5e7659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b817a0 0x558c17b8bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46422/0 (socket says 172.21.15.52:46422) 2024-04-02T14:02:51.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.811+0000 7f5e7659c640 1 -- 172.21.15.52:0/3007101792 learned_addr learned my addr 172.21.15.52:0/3007101792 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:51.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.811+0000 7f5e6ffff640 1 --2- 172.21.15.52:0/3007101792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c17b820c0 0x558c17b8c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.811+0000 7f5e76d9d640 1 --2- 172.21.15.52:0/3007101792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c17b8aca0 0x558c17b8c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.811+0000 7f5e7659c640 1 -- 172.21.15.52:0/3007101792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c17b8aca0 msgr2=0x558c17b8c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.811+0000 7f5e7659c640 1 --2- 172.21.15.52:0/3007101792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c17b8aca0 0x558c17b8c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.811+0000 7f5e7659c640 1 -- 172.21.15.52:0/3007101792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c17b820c0 msgr2=0x558c17b8c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:51.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.811+0000 7f5e7659c640 1 --2- 172.21.15.52:0/3007101792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c17b820c0 0x558c17b8c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.811+0000 7f5e7659c640 1 -- 172.21.15.52:0/3007101792 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c17b81ef0 con 0x558c17b817a0 2024-04-02T14:02:51.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.811+0000 7f5e7659c640 1 --2- 172.21.15.52:0/3007101792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b817a0 0x558c17b8bda0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f5e680bdef0 tx=0x7f5e680f4930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ac958a3c6a7c1e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:51.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.811+0000 7f5e5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:51.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.812+0000 7f5e5dffb640 1 -- 172.21.15.52:0/3007101792 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6804b040 con 0x558c17b817a0 2024-04-02T14:02:51.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.812+0000 7f5e5dffb640 1 -- 172.21.15.52:0/3007101792 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f5e680efa60 con 0x558c17b817a0 2024-04-02T14:02:51.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.812+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/3007101792 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e700273f0 con 0x558c17b817a0 2024-04-02T14:02:51.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.812+0000 7f5e5dffb640 1 -- 172.21.15.52:0/3007101792 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5e680bdd30 con 0x558c17b817a0 2024-04-02T14:02:51.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.812+0000 7f5e7f5c8740 10 client.18306 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:51.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.812+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/3007101792 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558c17c010d0 con 0x558c17b817a0 2024-04-02T14:02:51.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.812+0000 7f5e5dffb640 1 -- 172.21.15.52:0/3007101792 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e700273f0 con 0x558c17b817a0 2024-04-02T14:02:51.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.812+0000 7f5e5dffb640 1 -- 172.21.15.52:0/3007101792 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5e680ba860 con 0x558c17b817a0 2024-04-02T14:02:51.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.812+0000 7f5e7f5c8740 10 client.18306 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:51.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.812+0000 7f5e7f5c8740 10 client.18306 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:02:51.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.812+0000 7f5e7f5c8740 20 client.18306 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:51.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.812+0000 7f5e7f5c8740 1 --2- 172.21.15.52:0/3007101792 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x558c17c017f0 0x558c17c03be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:51.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.812+0000 7f5e7f5c8740 4 client.18306 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:02:51.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.812+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/3007101792 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558c17aed040 con 0x558c17c017f0 2024-04-02T14:02:51.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.812+0000 7f5e6ffff640 1 --2- 172.21.15.52:0/3007101792 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x558c17c017f0 0x558c17c03be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:51.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.813+0000 7f5e5dffb640 10 client.18306 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:51.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.813+0000 7f5e6ffff640 1 --2- 172.21.15.52:0/3007101792 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x558c17c017f0 0x558c17c03be0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=331a734ecea7798e server_cookie=ea83bb97b31682fb in_seq=0 out_seq=0 2024-04-02T14:02:51.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.813+0000 7f5e5dffb640 1 -- 172.21.15.52:0/3007101792 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558c17aed040 con 0x558c17c017f0 2024-04-02T14:02:51.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.813+0000 7f5e5dffb640 10 client.18306 handle_command_reply: tid=0 2024-04-02T14:02:51.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.813+0000 7f5e7f5c8740 1 client.18306 shutdown 2024-04-02T14:02:51.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.813+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/3007101792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b817a0 msgr2=0x558c17b8bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.813+0000 7f5e7f5c8740 1 --2- 172.21.15.52:0/3007101792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b817a0 0x558c17b8bda0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f5e680bdef0 tx=0x7f5e680f4930 comp rx=0 tx=0).stop 2024-04-02T14:02:51.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.813+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/3007101792 shutdown_connections 2024-04-02T14:02:51.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.813+0000 7f5e7f5c8740 1 --2- 172.21.15.52:0/3007101792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c17b8aca0 0x558c17b8c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.813+0000 7f5e7f5c8740 1 --2- 172.21.15.52:0/3007101792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558c17b820c0 0x558c17b8c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.813+0000 7f5e7f5c8740 1 --2- 172.21.15.52:0/3007101792 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x558c17c017f0 0x558c17c03be0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.813+0000 7f5e7f5c8740 1 --2- 172.21.15.52:0/3007101792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558c17b817a0 0x558c17b8bda0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.813+0000 7f5e5dffb640 0 client.18306 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:51.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.813+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/3007101792 >> 172.21.15.52:0/3007101792 conn(0x558c17b7bed0 msgr2=0x558c17b7a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.813+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/3007101792 shutdown_connections 2024-04-02T14:02:51.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.813+0000 7f5e7f5c8740 1 -- 172.21.15.52:0/3007101792 wait complete. 2024-04-02T14:02:51.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.813+0000 7f5e7f5c8740 20 client.18306 trim_cache size 0 max 16384 2024-04-02T14:02:51.828 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:02:51.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.814+0000 7f5e7c904640 1 -- 172.21.15.52:0/1547094900 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5e640a3d20 msgr2=0x7f5e640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.814+0000 7f5e7c904640 1 --2- 172.21.15.52:0/1547094900 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5e640a3d20 0x7f5e640a61e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5e70005fd0 tx=0x7f5e70004a40 comp rx=0 tx=0).stop 2024-04-02T14:02:51.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.814+0000 7f5e7c904640 1 -- 172.21.15.52:0/1547094900 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e78101a90 msgr2=0x7f5e781983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:51.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.814+0000 7f5e7c904640 1 --2- 172.21.15.52:0/1547094900 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e78101a90 0x7f5e781983b0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f5e68009230 tx=0x7f5e6802cba0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.814+0000 7f5e7c904640 1 -- 172.21.15.52:0/1547094900 shutdown_connections 2024-04-02T14:02:51.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.814+0000 7f5e7c904640 1 --2- 172.21.15.52:0/1547094900 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f5e640a3d20 0x7f5e640a61e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.814+0000 7f5e7c904640 1 --2- 172.21.15.52:0/1547094900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e78107aa0 0x7f5e7819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.814+0000 7f5e7c904640 1 --2- 172.21.15.52:0/1547094900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e78102450 0x7f5e781988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.814+0000 7f5e7c904640 1 --2- 172.21.15.52:0/1547094900 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e78101a90 0x7f5e781983b0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:51.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.814+0000 7f5e7c904640 1 -- 172.21.15.52:0/1547094900 >> 172.21.15.52:0/1547094900 conn(0x7f5e78078f00 msgr2=0x7f5e78104520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:51.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.814+0000 7f5e7c904640 1 -- 172.21.15.52:0/1547094900 shutdown_connections 2024-04-02T14:02:51.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:51.814+0000 7f5e7c904640 1 -- 172.21.15.52:0/1547094900 wait complete. 2024-04-02T14:02:51.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:51 smithi088 ceph-mon[36528]: pgmap v289: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:02:51.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:51 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/4220841741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:02:52.831 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T14:02:52.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:02:52.832 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T14:02:52.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.948+0000 7f97aedc8640 1 -- 172.21.15.52:0/1334657813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97a8106a30 msgr2=0x7f97a8106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:52.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.948+0000 7f97aedc8640 1 --2- 172.21.15.52:0/1334657813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97a8106a30 0x7f97a8106e10 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f979c009930 tx=0x7f979c02bc30 comp rx=0 tx=0).stop 2024-04-02T14:02:52.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.948+0000 7f97aedc8640 1 -- 172.21.15.52:0/1334657813 shutdown_connections 2024-04-02T14:02:52.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.948+0000 7f97aedc8640 1 --2- 172.21.15.52:0/1334657813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97a81013e0 0x7f97a810d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.948+0000 7f97aedc8640 1 --2- 172.21.15.52:0/1334657813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f97a8100a20 0x7f97a8100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.948+0000 7f97aedc8640 1 --2- 172.21.15.52:0/1334657813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97a8106a30 0x7f97a8106e10 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.948+0000 7f97aedc8640 1 -- 172.21.15.52:0/1334657813 >> 172.21.15.52:0/1334657813 conn(0x7f97a8077b60 msgr2=0x7f97a8077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:52.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.948+0000 7f97aedc8640 1 -- 172.21.15.52:0/1334657813 shutdown_connections 2024-04-02T14:02:52.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.948+0000 7f97aedc8640 1 -- 172.21.15.52:0/1334657813 wait complete. 2024-04-02T14:02:52.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.949+0000 7f97aedc8640 1 Processor -- start 2024-04-02T14:02:52.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.949+0000 7f97aedc8640 1 -- start start 2024-04-02T14:02:52.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.949+0000 7f97aedc8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97a8100a20 0x7f97a81991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:52.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.949+0000 7f97aedc8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97a81013e0 0x7f97a8199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:52.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.949+0000 7f97aedc8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f97a8106a30 0x7f97a81a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:52.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.949+0000 7f97aedc8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f97a8110e10 con 0x7f97a8100a20 2024-04-02T14:02:52.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.949+0000 7f97aedc8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f97a8110c90 con 0x7f97a81013e0 2024-04-02T14:02:52.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.949+0000 7f97aedc8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f97a8110f90 con 0x7f97a8106a30 2024-04-02T14:02:52.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.949+0000 7f97acb64640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97a8100a20 0x7f97a81991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:52.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.949+0000 7f97acb64640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97a8100a20 0x7f97a81991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46452/0 (socket says 172.21.15.52:46452) 2024-04-02T14:02:52.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.949+0000 7f97acb64640 1 -- 172.21.15.52:0/555642855 learned_addr learned my addr 172.21.15.52:0/555642855 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:52.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.949+0000 7f97ad365640 1 --2- 172.21.15.52:0/555642855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f97a8106a30 0x7f97a81a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:52.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.949+0000 7f97a7fff640 1 --2- 172.21.15.52:0/555642855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97a81013e0 0x7f97a8199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:52.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.949+0000 7f97acb64640 1 -- 172.21.15.52:0/555642855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f97a8106a30 msgr2=0x7f97a81a4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:52.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.949+0000 7f97acb64640 1 --2- 172.21.15.52:0/555642855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f97a8106a30 0x7f97a81a4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.949+0000 7f97acb64640 1 -- 172.21.15.52:0/555642855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97a81013e0 msgr2=0x7f97a8199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:52.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.949+0000 7f97acb64640 1 --2- 172.21.15.52:0/555642855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97a81013e0 0x7f97a8199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.949+0000 7f97acb64640 1 -- 172.21.15.52:0/555642855 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a81a4790 con 0x7f97a8100a20 2024-04-02T14:02:52.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.949+0000 7f97a7fff640 1 --2- 172.21.15.52:0/555642855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97a81013e0 0x7f97a8199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:02:52.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.950+0000 7f97acb64640 1 --2- 172.21.15.52:0/555642855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97a8100a20 0x7f97a81991d0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f979c009230 tx=0x7f979c02cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:52.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.950+0000 7f97a5ffb640 1 -- 172.21.15.52:0/555642855 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c041070 con 0x7f97a8100a20 2024-04-02T14:02:52.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.950+0000 7f97a5ffb640 1 -- 172.21.15.52:0/555642855 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f979c004450 con 0x7f97a8100a20 2024-04-02T14:02:52.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.950+0000 7f97aedc8640 1 -- 172.21.15.52:0/555642855 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a81a4960 con 0x7f97a8100a20 2024-04-02T14:02:52.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.950+0000 7f97a5ffb640 1 -- 172.21.15.52:0/555642855 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c034440 con 0x7f97a8100a20 2024-04-02T14:02:52.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.950+0000 7f97aedc8640 1 -- 172.21.15.52:0/555642855 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a81a4e20 con 0x7f97a8100a20 2024-04-02T14:02:52.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.952+0000 7f97a5ffb640 1 -- 172.21.15.52:0/555642855 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f979c044070 con 0x7f97a8100a20 2024-04-02T14:02:52.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.953+0000 7f97b19ad740 1 Processor -- start 2024-04-02T14:02:52.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.953+0000 7f97b19ad740 1 -- start start 2024-04-02T14:02:52.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.953+0000 7f97b19ad740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f04e83b2c0 0x55f04e83b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:52.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.953+0000 7f97b19ad740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e83bbe0 0x55f04e844280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:52.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.953+0000 7f97b19ad740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f04e8447c0 0x55f04e846bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:52.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.953+0000 7f97a5ffb640 1 --2- 172.21.15.52:0/555642855 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f97980a3d20 0x7f97980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:52.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.953+0000 7f97b19ad740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f04e79f770 con 0x55f04e83bbe0 2024-04-02T14:02:52.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.953+0000 7f97b19ad740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f04e747960 con 0x55f04e83b2c0 2024-04-02T14:02:52.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.953+0000 7f97b19ad740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f04e79b8c0 con 0x55f04e8447c0 2024-04-02T14:02:52.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.953+0000 7f97acb64640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e83bbe0 0x55f04e844280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:52.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.953+0000 7f97ad365640 1 --2- 172.21.15.52:0/555642855 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f97980a3d20 0x7f97980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:52.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.953+0000 7f97acb64640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e83bbe0 0x55f04e844280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46458/0 (socket says 172.21.15.52:46458) 2024-04-02T14:02:52.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.953+0000 7f97acb64640 1 -- 172.21.15.52:0/3750201274 learned_addr learned my addr 172.21.15.52:0/3750201274 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:52.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.954+0000 7f97a5ffb640 1 -- 172.21.15.52:0/555642855 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f979c0f2650 con 0x7f97a8100a20 2024-04-02T14:02:52.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.954+0000 7f97ad365640 1 --2- 172.21.15.52:0/3750201274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f04e83b2c0 0x55f04e83b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:52.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.954+0000 7f97a7fff640 1 --2- 172.21.15.52:0/3750201274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f04e8447c0 0x55f04e846bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:52.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.954+0000 7f97acb64640 1 -- 172.21.15.52:0/3750201274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f04e8447c0 msgr2=0x55f04e846bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:52.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.954+0000 7f97acb64640 1 --2- 172.21.15.52:0/3750201274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f04e8447c0 0x55f04e846bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.954+0000 7f97acb64640 1 -- 172.21.15.52:0/3750201274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f04e83b2c0 msgr2=0x55f04e83b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:52.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.954+0000 7f97acb64640 1 --2- 172.21.15.52:0/3750201274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f04e83b2c0 0x55f04e83b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.954+0000 7f97acb64640 1 -- 172.21.15.52:0/3750201274 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f04e57b7e0 con 0x55f04e83bbe0 2024-04-02T14:02:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.954+0000 7f97acb64640 1 --2- 172.21.15.52:0/3750201274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e83bbe0 0x55f04e844280 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f979c090dd0 tx=0x7f979c090e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560a2ef894fc1c2f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.954+0000 7f97ad365640 1 --2- 172.21.15.52:0/555642855 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f97980a3d20 0x7f97980a61e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f97a0005c40 tx=0x7f97a0009c90 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.954+0000 7f9792ffd640 1 -- 172.21.15.52:0/3750201274 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c04b070 con 0x55f04e83bbe0 2024-04-02T14:02:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.954+0000 7f9792ffd640 1 -- 172.21.15.52:0/3750201274 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f979c0bea10 con 0x55f04e83bbe0 2024-04-02T14:02:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.954+0000 7f9792ffd640 1 -- 172.21.15.52:0/3750201274 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c0c6f80 con 0x55f04e83bbe0 2024-04-02T14:02:52.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.955+0000 7f97b19ad740 1 -- 172.21.15.52:0/3750201274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e83bbe0 msgr2=0x55f04e844280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:52.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.955+0000 7f97b19ad740 1 --2- 172.21.15.52:0/3750201274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e83bbe0 0x55f04e844280 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f979c090dd0 tx=0x7f979c090e00 comp rx=0 tx=0).stop 2024-04-02T14:02:52.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.955+0000 7f97b19ad740 1 -- 172.21.15.52:0/3750201274 shutdown_connections 2024-04-02T14:02:52.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.955+0000 7f97b19ad740 1 --2- 172.21.15.52:0/3750201274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f04e8447c0 0x55f04e846bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.955+0000 7f97b19ad740 1 --2- 172.21.15.52:0/3750201274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e83bbe0 0x55f04e844280 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.955+0000 7f97b19ad740 1 --2- 172.21.15.52:0/3750201274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f04e83b2c0 0x55f04e83b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.955+0000 7f97b19ad740 1 -- 172.21.15.52:0/3750201274 >> 172.21.15.52:0/3750201274 conn(0x55f04e8359f0 msgr2=0x55f04e835dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:52.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.955+0000 7f97b19ad740 1 -- 172.21.15.52:0/3750201274 shutdown_connections 2024-04-02T14:02:52.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.955+0000 7f97b19ad740 1 -- 172.21.15.52:0/3750201274 wait complete. 2024-04-02T14:02:52.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.955+0000 7f97b19ad740 1 -- start start 2024-04-02T14:02:52.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.955+0000 7f97b19ad740 1 Processor -- start 2024-04-02T14:02:52.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.956+0000 7f97b19ad740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e83b2c0 0x55f04e841710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:52.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.956+0000 7f97b19ad740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f04e83bbe0 0x55f04e841c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:52.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.956+0000 7f97b19ad740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f04e8447c0 0x55f04e842190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:52.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.956+0000 7f97b19ad740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f04e79f770 con 0x55f04e83b2c0 2024-04-02T14:02:52.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.956+0000 7f97b19ad740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f04e75ad20 con 0x55f04e8447c0 2024-04-02T14:02:52.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.956+0000 7f97b19ad740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f04e847370 con 0x55f04e83bbe0 2024-04-02T14:02:52.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.956+0000 7f97b19ad740 10 client.0 _ll_register_callbacks cb 0x55f04e727a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:52.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.956+0000 7f97acb64640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e83b2c0 0x55f04e841710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:52.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.956+0000 7f97acb64640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e83b2c0 0x55f04e841710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46460/0 (socket says 172.21.15.52:46460) 2024-04-02T14:02:52.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.956+0000 7f97acb64640 1 -- 172.21.15.52:0/1439196557 learned_addr learned my addr 172.21.15.52:0/1439196557 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:52.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.956+0000 7f97ad365640 1 --2- 172.21.15.52:0/1439196557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f04e8447c0 0x55f04e842190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:52.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.956+0000 7f97a7fff640 1 --2- 172.21.15.52:0/1439196557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f04e83bbe0 0x55f04e841c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:52.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.956+0000 7f97acb64640 1 -- 172.21.15.52:0/1439196557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f04e83bbe0 msgr2=0x55f04e841c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:52.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.956+0000 7f97acb64640 1 --2- 172.21.15.52:0/1439196557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f04e83bbe0 0x55f04e841c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.956+0000 7f97acb64640 1 -- 172.21.15.52:0/1439196557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f04e8447c0 msgr2=0x55f04e842190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:52.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.956+0000 7f97acb64640 1 --2- 172.21.15.52:0/1439196557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f04e8447c0 0x55f04e842190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.956+0000 7f97acb64640 1 -- 172.21.15.52:0/1439196557 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f04e5dbf70 con 0x55f04e83b2c0 2024-04-02T14:02:52.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.956+0000 7f97a7fff640 1 --2- 172.21.15.52:0/1439196557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f04e83bbe0 0x55f04e841c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:02:52.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.956+0000 7f97acb64640 1 --2- 172.21.15.52:0/1439196557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e83b2c0 0x55f04e841710 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f979c0be010 tx=0x7f979c003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6abda7d3e7f2ff0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:52.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.956+0000 7f9787fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:52.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.956+0000 7f9787fff640 1 -- 172.21.15.52:0/1439196557 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c04b040 con 0x55f04e83b2c0 2024-04-02T14:02:52.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.956+0000 7f9787fff640 1 -- 172.21.15.52:0/1439196557 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f979c0f2240 con 0x55f04e83b2c0 2024-04-02T14:02:52.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.957+0000 7f97b19ad740 1 -- 172.21.15.52:0/1439196557 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f979c0bea10 con 0x55f04e83b2c0 2024-04-02T14:02:52.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.957+0000 7f9787fff640 1 -- 172.21.15.52:0/1439196557 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f979c0f2440 con 0x55f04e83b2c0 2024-04-02T14:02:52.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.957+0000 7f97b19ad740 10 client.18318 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:52.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.957+0000 7f97b19ad740 1 -- 172.21.15.52:0/1439196557 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f04e473170 con 0x55f04e83b2c0 2024-04-02T14:02:52.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.957+0000 7f9787fff640 1 -- 172.21.15.52:0/1439196557 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f979c0bea10 con 0x55f04e83b2c0 2024-04-02T14:02:52.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.957+0000 7f9787fff640 1 -- 172.21.15.52:0/1439196557 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f979c0efba0 con 0x55f04e83b2c0 2024-04-02T14:02:52.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.957+0000 7f97b19ad740 10 client.18318 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:52.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.957+0000 7f97b19ad740 10 client.18318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:02:52.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.957+0000 7f97b19ad740 20 client.18318 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:52.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.957+0000 7f97b19ad740 1 --2- 172.21.15.52:0/1439196557 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55f04e854530 0x55f04e856920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:52.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.957+0000 7f97b19ad740 4 client.18318 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:02:52.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.957+0000 7f97b19ad740 1 -- 172.21.15.52:0/1439196557 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f04e7accb0 con 0x55f04e854530 2024-04-02T14:02:52.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.957+0000 7f97a7fff640 1 --2- 172.21.15.52:0/1439196557 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55f04e854530 0x55f04e856920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:52.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.958+0000 7f9787fff640 10 client.18318 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:52.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.958+0000 7f97a7fff640 1 --2- 172.21.15.52:0/1439196557 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55f04e854530 0x55f04e856920 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=693a1184a1c191fb server_cookie=ba25d5cb2b59bb75 in_seq=0 out_seq=0 2024-04-02T14:02:52.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.958+0000 7f9787fff640 1 -- 172.21.15.52:0/1439196557 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f04e7accb0 con 0x55f04e854530 2024-04-02T14:02:52.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.958+0000 7f9787fff640 10 client.18318 handle_command_reply: tid=0 2024-04-02T14:02:52.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.959+0000 7f97b19ad740 1 client.18318 shutdown 2024-04-02T14:02:52.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.959+0000 7f97b19ad740 1 -- 172.21.15.52:0/1439196557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e83b2c0 msgr2=0x55f04e841710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:52.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.959+0000 7f97b19ad740 1 --2- 172.21.15.52:0/1439196557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e83b2c0 0x55f04e841710 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f979c0be010 tx=0x7f979c003a90 comp rx=0 tx=0).stop 2024-04-02T14:02:52.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.959+0000 7f97b19ad740 1 -- 172.21.15.52:0/1439196557 shutdown_connections 2024-04-02T14:02:52.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.959+0000 7f97b19ad740 1 --2- 172.21.15.52:0/1439196557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f04e8447c0 0x55f04e842190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.959+0000 7f97b19ad740 1 --2- 172.21.15.52:0/1439196557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f04e83bbe0 0x55f04e841c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.959+0000 7f97b19ad740 1 --2- 172.21.15.52:0/1439196557 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55f04e854530 0x55f04e856920 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.959+0000 7f97b19ad740 1 --2- 172.21.15.52:0/1439196557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e83b2c0 0x55f04e841710 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.959+0000 7f9787fff640 0 client.18318 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:52.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.959+0000 7f97b19ad740 1 -- 172.21.15.52:0/1439196557 >> 172.21.15.52:0/1439196557 conn(0x55f04e8359f0 msgr2=0x55f04e837740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:52.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.959+0000 7f97b19ad740 1 -- 172.21.15.52:0/1439196557 shutdown_connections 2024-04-02T14:02:52.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.959+0000 7f97b19ad740 1 -- 172.21.15.52:0/1439196557 wait complete. 2024-04-02T14:02:52.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.959+0000 7f97b19ad740 20 client.18318 trim_cache size 0 max 16384 2024-04-02T14:02:52.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.977+0000 7f97b19ad740 1 Processor -- start 2024-04-02T14:02:52.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.977+0000 7f97b19ad740 1 -- start start 2024-04-02T14:02:52.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.977+0000 7f97b19ad740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f04e83b2c0 0x55f04e843e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:52.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.977+0000 7f97b19ad740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e83bbe0 0x55f04e8500b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:52.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.977+0000 7f97b19ad740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f04e8447c0 0x55f04e85c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:52.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.977+0000 7f97b19ad740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f97a81aede0 con 0x55f04e83bbe0 2024-04-02T14:02:52.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.977+0000 7f97b19ad740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f97a81b0920 con 0x55f04e8447c0 2024-04-02T14:02:52.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.977+0000 7f97b19ad740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f97a81af4b0 con 0x55f04e83b2c0 2024-04-02T14:02:52.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.977+0000 7f97a7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e83bbe0 0x55f04e8500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:52.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.977+0000 7f97a7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e83bbe0 0x55f04e8500b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46476/0 (socket says 172.21.15.52:46476) 2024-04-02T14:02:52.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.977+0000 7f97a7fff640 1 -- 172.21.15.52:0/1856994067 learned_addr learned my addr 172.21.15.52:0/1856994067 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:52.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.977+0000 7f97ad365640 1 --2- 172.21.15.52:0/1856994067 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f04e8447c0 0x55f04e85c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:52.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.977+0000 7f97a7fff640 1 -- 172.21.15.52:0/1856994067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f04e83b2c0 msgr2=0x55f04e843e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:52.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.977+0000 7f97a7fff640 1 --2- 172.21.15.52:0/1856994067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f04e83b2c0 0x55f04e843e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.977+0000 7f97a7fff640 1 -- 172.21.15.52:0/1856994067 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f04e8447c0 msgr2=0x55f04e85c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:52.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.977+0000 7f97a7fff640 1 --2- 172.21.15.52:0/1856994067 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f04e8447c0 0x55f04e85c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.977+0000 7f97a7fff640 1 -- 172.21.15.52:0/1856994067 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f04e83ba10 con 0x55f04e83bbe0 2024-04-02T14:02:52.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.978+0000 7f97a7fff640 1 --2- 172.21.15.52:0/1856994067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e83bbe0 0x55f04e8500b0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x55f04e836b80 tx=0x7f979400b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7efddc589ff55e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:52.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.978+0000 7f97927fc640 1 -- 172.21.15.52:0/1856994067 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794013990 con 0x55f04e83bbe0 2024-04-02T14:02:52.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.978+0000 7f97927fc640 1 -- 172.21.15.52:0/1856994067 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f979401a330 con 0x55f04e83bbe0 2024-04-02T14:02:52.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.978+0000 7f97927fc640 1 -- 172.21.15.52:0/1856994067 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794011a00 con 0x55f04e83bbe0 2024-04-02T14:02:52.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.978+0000 7f97b19ad740 1 -- 172.21.15.52:0/1856994067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e83bbe0 msgr2=0x55f04e8500b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:52.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.978+0000 7f97b19ad740 1 --2- 172.21.15.52:0/1856994067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e83bbe0 0x55f04e8500b0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x55f04e836b80 tx=0x7f979400b470 comp rx=0 tx=0).stop 2024-04-02T14:02:52.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.978+0000 7f97b19ad740 1 -- 172.21.15.52:0/1856994067 shutdown_connections 2024-04-02T14:02:52.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.978+0000 7f97b19ad740 1 --2- 172.21.15.52:0/1856994067 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f04e8447c0 0x55f04e85c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.978+0000 7f97b19ad740 1 --2- 172.21.15.52:0/1856994067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e83bbe0 0x55f04e8500b0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.978+0000 7f97b19ad740 1 --2- 172.21.15.52:0/1856994067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f04e83b2c0 0x55f04e843e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.978+0000 7f97b19ad740 1 -- 172.21.15.52:0/1856994067 >> 172.21.15.52:0/1856994067 conn(0x55f04e8359f0 msgr2=0x55f04e837640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:52.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.978+0000 7f97b19ad740 1 -- 172.21.15.52:0/1856994067 shutdown_connections 2024-04-02T14:02:52.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.978+0000 7f97b19ad740 1 -- 172.21.15.52:0/1856994067 wait complete. 2024-04-02T14:02:52.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.979+0000 7f97b19ad740 1 -- start start 2024-04-02T14:02:52.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.979+0000 7f97b19ad740 1 Processor -- start 2024-04-02T14:02:52.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.979+0000 7f97b19ad740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f04e83b2c0 0x55f04e8487a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:52.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.979+0000 7f97b19ad740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f04e83bbe0 0x55f04e848ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:52.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.979+0000 7f97b19ad740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e8447c0 0x55f04e8b9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:52.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.979+0000 7f97b19ad740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f97a81b0250 con 0x55f04e8447c0 2024-04-02T14:02:52.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.979+0000 7f97b19ad740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f97a81afb80 con 0x55f04e83b2c0 2024-04-02T14:02:52.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.979+0000 7f97b19ad740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f97a81acde0 con 0x55f04e83bbe0 2024-04-02T14:02:52.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.979+0000 7f97b19ad740 10 client.0 _ll_register_callbacks cb 0x7f979c0830b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:52.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.979+0000 7f97ad365640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e8447c0 0x55f04e8b9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:52.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.979+0000 7f97ad365640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e8447c0 0x55f04e8b9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46484/0 (socket says 172.21.15.52:46484) 2024-04-02T14:02:52.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.979+0000 7f97ad365640 1 -- 172.21.15.52:0/3337607920 learned_addr learned my addr 172.21.15.52:0/3337607920 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:52.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.979+0000 7f97acb64640 1 --2- 172.21.15.52:0/3337607920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f04e83b2c0 0x55f04e8487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:52.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.979+0000 7f97a7fff640 1 --2- 172.21.15.52:0/3337607920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f04e83bbe0 0x55f04e848ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:52.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.980+0000 7f97ad365640 1 -- 172.21.15.52:0/3337607920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f04e83bbe0 msgr2=0x55f04e848ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:52.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.980+0000 7f97ad365640 1 --2- 172.21.15.52:0/3337607920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f04e83bbe0 0x55f04e848ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.980+0000 7f97ad365640 1 -- 172.21.15.52:0/3337607920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f04e83b2c0 msgr2=0x55f04e8487a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:52.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.980+0000 7f97ad365640 1 --2- 172.21.15.52:0/3337607920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f04e83b2c0 0x55f04e8487a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.980+0000 7f97ad365640 1 -- 172.21.15.52:0/3337607920 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f04e8b9fa0 con 0x55f04e8447c0 2024-04-02T14:02:52.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.980+0000 7f97a7fff640 1 --2- 172.21.15.52:0/3337607920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f04e83bbe0 0x55f04e848ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:02:52.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.980+0000 7f97ad365640 1 --2- 172.21.15.52:0/3337607920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e8447c0 0x55f04e8b9a10 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f97a0012480 tx=0x7f97a0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e163872d14f4d356 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:52.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.980+0000 7f9787fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:52.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.980+0000 7f9787fff640 1 -- 172.21.15.52:0/3337607920 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a002e060 con 0x55f04e8447c0 2024-04-02T14:02:52.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.980+0000 7f9787fff640 1 -- 172.21.15.52:0/3337607920 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f97a001b040 con 0x55f04e8447c0 2024-04-02T14:02:52.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.980+0000 7f97b19ad740 1 -- 172.21.15.52:0/3337607920 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f979401a330 con 0x55f04e8447c0 2024-04-02T14:02:52.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.980+0000 7f9787fff640 1 -- 172.21.15.52:0/3337607920 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f97a0035050 con 0x55f04e8447c0 2024-04-02T14:02:52.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.980+0000 7f97b19ad740 10 client.18324 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:52.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.980+0000 7f97b19ad740 1 -- 172.21.15.52:0/3337607920 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f04e8bb310 con 0x55f04e8447c0 2024-04-02T14:02:52.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.980+0000 7f9787fff640 1 -- 172.21.15.52:0/3337607920 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f979401a330 con 0x55f04e8447c0 2024-04-02T14:02:52.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.980+0000 7f9787fff640 1 -- 172.21.15.52:0/3337607920 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f97a0028070 con 0x55f04e8447c0 2024-04-02T14:02:52.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.980+0000 7f97b19ad740 10 client.18324 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:52.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.980+0000 7f97b19ad740 10 client.18324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:02:52.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.980+0000 7f97b19ad740 20 client.18324 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:52.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.980+0000 7f97b19ad740 1 --2- 172.21.15.52:0/3337607920 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55f04e8bbaf0 0x55f04e8bdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:52.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.980+0000 7f97b19ad740 4 client.18324 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T14:02:52.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.980+0000 7f97b19ad740 1 -- 172.21.15.52:0/3337607920 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f04e7a7000 con 0x55f04e8bbaf0 2024-04-02T14:02:52.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.981+0000 7f97acb64640 1 --2- 172.21.15.52:0/3337607920 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55f04e8bbaf0 0x55f04e8bdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:52.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.981+0000 7f9787fff640 10 client.18324 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:52.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.981+0000 7f97acb64640 1 --2- 172.21.15.52:0/3337607920 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55f04e8bbaf0 0x55f04e8bdf10 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69d5427eb8ada8de server_cookie=a9b5a4b101cae96f in_seq=0 out_seq=0 2024-04-02T14:02:52.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.982+0000 7f9787fff640 1 -- 172.21.15.52:0/3337607920 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f04e7a7000 con 0x55f04e8bbaf0 2024-04-02T14:02:52.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.982+0000 7f9787fff640 10 client.18324 handle_command_reply: tid=0 2024-04-02T14:02:52.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.982+0000 7f97b19ad740 1 client.18324 shutdown 2024-04-02T14:02:52.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.982+0000 7f97b19ad740 1 -- 172.21.15.52:0/3337607920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e8447c0 msgr2=0x55f04e8b9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:52.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.982+0000 7f97b19ad740 1 --2- 172.21.15.52:0/3337607920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e8447c0 0x55f04e8b9a10 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f97a0012480 tx=0x7f97a0005bd0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.983+0000 7f97b19ad740 1 -- 172.21.15.52:0/3337607920 shutdown_connections 2024-04-02T14:02:52.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.983+0000 7f97b19ad740 1 --2- 172.21.15.52:0/3337607920 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55f04e8bbaf0 0x55f04e8bdf10 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.983+0000 7f97b19ad740 1 --2- 172.21.15.52:0/3337607920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f04e8447c0 0x55f04e8b9a10 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.983+0000 7f97b19ad740 1 --2- 172.21.15.52:0/3337607920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f04e83bbe0 0x55f04e848ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.983+0000 7f97b19ad740 1 --2- 172.21.15.52:0/3337607920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f04e83b2c0 0x55f04e8487a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.983+0000 7f97b19ad740 1 -- 172.21.15.52:0/3337607920 >> 172.21.15.52:0/3337607920 conn(0x55f04e8359f0 msgr2=0x55f04e834220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:52.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.983+0000 7f9787fff640 0 client.18324 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:52.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.983+0000 7f97b19ad740 1 -- 172.21.15.52:0/3337607920 shutdown_connections 2024-04-02T14:02:52.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.983+0000 7f97b19ad740 1 -- 172.21.15.52:0/3337607920 wait complete. 2024-04-02T14:02:52.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.983+0000 7f97b19ad740 20 client.18324 trim_cache size 0 max 16384 2024-04-02T14:02:52.997 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:02:52.997 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-04-02T14:02:52.997 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "994d29a7-d20f-4359-b018-0aeefb3a99af", 2024-04-02T14:02:52.997 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-04-02T14:02:52.997 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:02:52.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.983+0000 7f97aedc8640 1 -- 172.21.15.52:0/555642855 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f97980a3d20 msgr2=0x7f97980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:52.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.983+0000 7f97aedc8640 1 --2- 172.21.15.52:0/555642855 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f97980a3d20 0x7f97980a61e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f97a0005c40 tx=0x7f97a0009c90 comp rx=0 tx=0).stop 2024-04-02T14:02:52.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.983+0000 7f97aedc8640 1 -- 172.21.15.52:0/555642855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97a8100a20 msgr2=0x7f97a81991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:52.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.983+0000 7f97aedc8640 1 --2- 172.21.15.52:0/555642855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97a8100a20 0x7f97a81991d0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f979c009230 tx=0x7f979c02cc70 comp rx=0 tx=0).stop 2024-04-02T14:02:52.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.983+0000 7f97aedc8640 1 -- 172.21.15.52:0/555642855 shutdown_connections 2024-04-02T14:02:52.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.983+0000 7f97aedc8640 1 --2- 172.21.15.52:0/555642855 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f97980a3d20 0x7f97980a61e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.983+0000 7f97aedc8640 1 --2- 172.21.15.52:0/555642855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f97a8106a30 0x7f97a81a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:52.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.983+0000 7f97aedc8640 1 --2- 172.21.15.52:0/555642855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97a81013e0 0x7f97a8199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.983+0000 7f97aedc8640 1 --2- 172.21.15.52:0/555642855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97a8100a20 0x7f97a81991d0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.983+0000 7f97aedc8640 1 -- 172.21.15.52:0/555642855 >> 172.21.15.52:0/555642855 conn(0x7f97a8077b60 msgr2=0x7f97a8103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:53.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.983+0000 7f97aedc8640 1 -- 172.21.15.52:0/555642855 shutdown_connections 2024-04-02T14:02:53.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:52.983+0000 7f97aedc8640 1 -- 172.21.15.52:0/555642855 wait complete. 2024-04-02T14:02:53.000 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:02:53.001 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:02:53.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.109+0000 7f4cece0f640 1 -- 172.21.15.52:0/3439353098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ce8106dd0 msgr2=0x7f4ce810e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:53.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.109+0000 7f4cece0f640 1 --2- 172.21.15.52:0/3439353098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ce8106dd0 0x7f4ce810e9a0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009ca0 tx=0x7f4ce002c0f0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.109+0000 7f4cece0f640 1 -- 172.21.15.52:0/3439353098 shutdown_connections 2024-04-02T14:02:53.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.109+0000 7f4cece0f640 1 --2- 172.21.15.52:0/3439353098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ce8106dd0 0x7f4ce810e9a0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.109+0000 7f4cece0f640 1 --2- 172.21.15.52:0/3439353098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ce8100bc0 0x7f4ce81066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.109+0000 7f4cece0f640 1 --2- 172.21.15.52:0/3439353098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ce8100210 0x7f4ce81005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.109+0000 7f4cece0f640 1 -- 172.21.15.52:0/3439353098 >> 172.21.15.52:0/3439353098 conn(0x7f4ce8076f20 msgr2=0x7f4ce80fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:53.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.110+0000 7f4cece0f640 1 -- 172.21.15.52:0/3439353098 shutdown_connections 2024-04-02T14:02:53.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.110+0000 7f4cece0f640 1 -- 172.21.15.52:0/3439353098 wait complete. 2024-04-02T14:02:53.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.110+0000 7f4cece0f640 1 Processor -- start 2024-04-02T14:02:53.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.110+0000 7f4cece0f640 1 -- start start 2024-04-02T14:02:53.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.111+0000 7f4cece0f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ce8100210 0x7f4ce819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:53.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.111+0000 7f4cece0f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ce8106dd0 0x7f4ce819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:53.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.111+0000 7f4cece0f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ce819ee30 0x7f4ce819f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:53.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.111+0000 7f4cece0f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce8103110 con 0x7f4ce819ee30 2024-04-02T14:02:53.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.111+0000 7f4cece0f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce8102f90 con 0x7f4ce8100210 2024-04-02T14:02:53.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.111+0000 7f4cece0f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce8114230 con 0x7f4ce8106dd0 2024-04-02T14:02:53.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.111+0000 7f4ce6d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ce819ee30 0x7f4ce819f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:53.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.111+0000 7f4ce6d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ce819ee30 0x7f4ce819f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46506/0 (socket says 172.21.15.52:46506) 2024-04-02T14:02:53.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.111+0000 7f4cdffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ce8106dd0 0x7f4ce819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:53.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.111+0000 7f4ce6d9d640 1 -- 172.21.15.52:0/3816239616 learned_addr learned my addr 172.21.15.52:0/3816239616 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:53.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.111+0000 7f4ce659c640 1 --2- 172.21.15.52:0/3816239616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ce8100210 0x7f4ce819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:53.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.111+0000 7f4ce6d9d640 1 -- 172.21.15.52:0/3816239616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ce8106dd0 msgr2=0x7f4ce819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:53.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.111+0000 7f4ce6d9d640 1 --2- 172.21.15.52:0/3816239616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ce8106dd0 0x7f4ce819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.111+0000 7f4ce6d9d640 1 -- 172.21.15.52:0/3816239616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ce8100210 msgr2=0x7f4ce819a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:53.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.111+0000 7f4ce6d9d640 1 --2- 172.21.15.52:0/3816239616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ce8100210 0x7f4ce819a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.111+0000 7f4ce6d9d640 1 -- 172.21.15.52:0/3816239616 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce819f920 con 0x7f4ce819ee30 2024-04-02T14:02:53.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.111+0000 7f4ce6d9d640 1 --2- 172.21.15.52:0/3816239616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ce819ee30 0x7f4ce819f2e0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f4ce002c700 tx=0x7f4ce0032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:53.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.112+0000 7f4cdf7fe640 1 -- 172.21.15.52:0/3816239616 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0041020 con 0x7f4ce819ee30 2024-04-02T14:02:53.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.112+0000 7f4cdf7fe640 1 -- 172.21.15.52:0/3816239616 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce0034cd0 con 0x7f4ce819ee30 2024-04-02T14:02:53.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.112+0000 7f4cdf7fe640 1 -- 172.21.15.52:0/3816239616 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0039040 con 0x7f4ce819ee30 2024-04-02T14:02:53.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.112+0000 7f4cece0f640 1 -- 172.21.15.52:0/3816239616 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce819fbb0 con 0x7f4ce819ee30 2024-04-02T14:02:53.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.112+0000 7f4cece0f640 1 -- 172.21.15.52:0/3816239616 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce81a7320 con 0x7f4ce819ee30 2024-04-02T14:02:53.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.115+0000 7f4cdf7fe640 1 -- 172.21.15.52:0/3816239616 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4ce0044020 con 0x7f4ce819ee30 2024-04-02T14:02:53.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.115+0000 7f4cef9f1740 1 Processor -- start 2024-04-02T14:02:53.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.116+0000 7f4cef9f1740 1 -- start start 2024-04-02T14:02:53.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.116+0000 7f4cef9f1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e94e3858d0 0x55e94e385cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:53.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.116+0000 7f4cef9f1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e94e3861f0 0x55e94e38e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:53.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.116+0000 7f4cdf7fe640 1 --2- 172.21.15.52:0/3816239616 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4cd40a3d20 0x7f4cd40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:53.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.116+0000 7f4cef9f1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e94e38edd0 0x55e94e3911c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:53.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.116+0000 7f4cef9f1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e94e2e9830 con 0x55e94e3861f0 2024-04-02T14:02:53.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.116+0000 7f4cef9f1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e94e291820 con 0x55e94e38edd0 2024-04-02T14:02:53.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.116+0000 7f4cef9f1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e94e2e57e0 con 0x55e94e3858d0 2024-04-02T14:02:53.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.116+0000 7f4cdffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e94e3858d0 0x55e94e385cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:53.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.116+0000 7f4ce6d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e94e38edd0 0x55e94e3911c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:53.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.116+0000 7f4cdffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e94e3858d0 0x55e94e385cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.52:49512/0 (socket says 172.21.15.52:49512) 2024-04-02T14:02:53.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.116+0000 7f4cdffff640 1 -- 172.21.15.52:0/3941048899 learned_addr learned my addr 172.21.15.52:0/3941048899 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:53.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.116+0000 7f4ce659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e94e3861f0 0x55e94e38e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:53.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.116+0000 7f4cdffff640 1 --2- 172.21.15.52:0/3816239616 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4cd40a3d20 0x7f4cd40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:53.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.116+0000 7f4cdf7fe640 1 -- 172.21.15.52:0/3816239616 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f4ce00f27c0 con 0x7f4ce819ee30 2024-04-02T14:02:53.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.116+0000 7f4cdffff640 1 -- 172.21.15.52:0/3941048899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e94e38edd0 msgr2=0x55e94e3911c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:53.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.116+0000 7f4cdffff640 1 --2- 172.21.15.52:0/3941048899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e94e38edd0 0x55e94e3911c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.116+0000 7f4cdffff640 1 -- 172.21.15.52:0/3941048899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e94e3861f0 msgr2=0x55e94e38e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:53.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.116+0000 7f4cdffff640 1 --2- 172.21.15.52:0/3941048899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e94e3861f0 0x55e94e38e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.116+0000 7f4cdffff640 1 -- 172.21.15.52:0/3941048899 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e94e0c77d0 con 0x55e94e3858d0 2024-04-02T14:02:53.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.116+0000 7f4ce6d9d640 1 --2- 172.21.15.52:0/3941048899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e94e38edd0 0x55e94e3911c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:02:53.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.117+0000 7f4cdffff640 1 --2- 172.21.15.52:0/3816239616 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4cd40a3d20 0x7f4cd40a61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000f790 tx=0x7f4cd000e000 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:53.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.117+0000 7f4cdffff640 1 --2- 172.21.15.52:0/3941048899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e94e3858d0 0x55e94e385cb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009560 tx=0x7f4cd0011ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c4b48e574124a94 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:53.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.117+0000 7f4cdcff9640 1 -- 172.21.15.52:0/3941048899 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0011560 con 0x55e94e3858d0 2024-04-02T14:02:53.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.117+0000 7f4cdcff9640 1 -- 172.21.15.52:0/3941048899 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cd0003ad0 con 0x55e94e3858d0 2024-04-02T14:02:53.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.117+0000 7f4cdcff9640 1 -- 172.21.15.52:0/3941048899 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd001e940 con 0x55e94e3858d0 2024-04-02T14:02:53.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.118+0000 7f4cef9f1740 1 -- 172.21.15.52:0/3941048899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e94e3858d0 msgr2=0x55e94e385cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:53.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.118+0000 7f4cef9f1740 1 --2- 172.21.15.52:0/3941048899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e94e3858d0 0x55e94e385cb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009560 tx=0x7f4cd0011ec0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.118+0000 7f4cef9f1740 1 -- 172.21.15.52:0/3941048899 shutdown_connections 2024-04-02T14:02:53.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.118+0000 7f4cef9f1740 1 --2- 172.21.15.52:0/3941048899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e94e38edd0 0x55e94e3911c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.118+0000 7f4cef9f1740 1 --2- 172.21.15.52:0/3941048899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e94e3861f0 0x55e94e38e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.118+0000 7f4cef9f1740 1 --2- 172.21.15.52:0/3941048899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e94e3858d0 0x55e94e385cb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.118+0000 7f4cef9f1740 1 -- 172.21.15.52:0/3941048899 >> 172.21.15.52:0/3941048899 conn(0x55e94e380000 msgr2=0x55e94e3803e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:53.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.118+0000 7f4cef9f1740 1 -- 172.21.15.52:0/3941048899 shutdown_connections 2024-04-02T14:02:53.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.118+0000 7f4cef9f1740 1 -- 172.21.15.52:0/3941048899 wait complete. 2024-04-02T14:02:53.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.118+0000 7f4cef9f1740 1 -- start start 2024-04-02T14:02:53.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.118+0000 7f4cef9f1740 1 Processor -- start 2024-04-02T14:02:53.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.118+0000 7f4cef9f1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e94e3858d0 0x55e94e38e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:53.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.118+0000 7f4cef9f1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e94e3861f0 0x55e94e38b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:53.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.119+0000 7f4cef9f1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e94e38edd0 0x55e94e38bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:53.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.119+0000 7f4cef9f1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e94e391b60 con 0x55e94e3858d0 2024-04-02T14:02:53.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.119+0000 7f4cef9f1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e94e2a4d30 con 0x55e94e3861f0 2024-04-02T14:02:53.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.119+0000 7f4cef9f1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e94e391ce0 con 0x55e94e38edd0 2024-04-02T14:02:53.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.119+0000 7f4cef9f1740 10 client.0 _ll_register_callbacks cb 0x55e94e2a4720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:53.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.119+0000 7f4ce659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e94e3858d0 0x55e94e38e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:53.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.119+0000 7f4ce659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e94e3858d0 0x55e94e38e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46538/0 (socket says 172.21.15.52:46538) 2024-04-02T14:02:53.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.119+0000 7f4ce659c640 1 -- 172.21.15.52:0/4225731991 learned_addr learned my addr 172.21.15.52:0/4225731991 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:53.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.119+0000 7f4cdffff640 1 --2- 172.21.15.52:0/4225731991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e94e3861f0 0x55e94e38b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:53.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.119+0000 7f4ce6d9d640 1 --2- 172.21.15.52:0/4225731991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e94e38edd0 0x55e94e38bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:53.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.119+0000 7f4ce659c640 1 -- 172.21.15.52:0/4225731991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e94e38edd0 msgr2=0x55e94e38bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:53.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.119+0000 7f4ce659c640 1 --2- 172.21.15.52:0/4225731991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e94e38edd0 0x55e94e38bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.119+0000 7f4ce659c640 1 -- 172.21.15.52:0/4225731991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e94e3861f0 msgr2=0x55e94e38b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:53.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.119+0000 7f4ce659c640 1 --2- 172.21.15.52:0/4225731991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e94e3861f0 0x55e94e38b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.119+0000 7f4ce659c640 1 -- 172.21.15.52:0/4225731991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e94dffd3f0 con 0x55e94e3858d0 2024-04-02T14:02:53.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.119+0000 7f4ce659c640 1 --2- 172.21.15.52:0/4225731991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e94e3858d0 0x55e94e38e650 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8012740 tx=0x7f4cd8012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af8b15c6ea7f455 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:53.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.119+0000 7f4ccdffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:02:53.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.119+0000 7f4ccdffb640 1 -- 172.21.15.52:0/4225731991 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd800f920 con 0x55e94e3858d0 2024-04-02T14:02:53.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.120+0000 7f4ccdffb640 1 -- 172.21.15.52:0/4225731991 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f4cd800fac0 con 0x55e94e3858d0 2024-04-02T14:02:53.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.120+0000 7f4cef9f1740 1 -- 172.21.15.52:0/4225731991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cd0003ad0 con 0x55e94e3858d0 2024-04-02T14:02:53.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.120+0000 7f4ccdffb640 1 -- 172.21.15.52:0/4225731991 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4cd800fcc0 con 0x55e94e3858d0 2024-04-02T14:02:53.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.120+0000 7f4cef9f1740 10 client.18336 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:53.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.120+0000 7f4cef9f1740 1 -- 172.21.15.52:0/4225731991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e94dfbd0f0 con 0x55e94e3858d0 2024-04-02T14:02:53.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.120+0000 7f4ccdffb640 1 -- 172.21.15.52:0/4225731991 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cd0003ad0 con 0x55e94e3858d0 2024-04-02T14:02:53.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.120+0000 7f4ccdffb640 1 -- 172.21.15.52:0/4225731991 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4cd801c420 con 0x55e94e3858d0 2024-04-02T14:02:53.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.120+0000 7f4cef9f1740 10 client.18336 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:53.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.120+0000 7f4cef9f1740 10 client.18336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:02:53.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.120+0000 7f4cef9f1740 20 client.18336 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:53.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.120+0000 7f4cef9f1740 1 --2- 172.21.15.52:0/4225731991 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55e94e39ed90 0x55e94e3a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:53.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.120+0000 7f4cef9f1740 4 client.18336 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:02:53.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.120+0000 7f4cef9f1740 1 -- 172.21.15.52:0/4225731991 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e94e2f6d70 con 0x55e94e39ed90 2024-04-02T14:02:53.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.121+0000 7f4cdffff640 1 --2- 172.21.15.52:0/4225731991 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55e94e39ed90 0x55e94e3a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:53.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.121+0000 7f4cdffff640 1 --2- 172.21.15.52:0/4225731991 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55e94e39ed90 0x55e94e3a11e0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=278da52b6a77acf2 server_cookie=340fbd2b1a5c756 in_seq=0 out_seq=0 2024-04-02T14:02:53.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.121+0000 7f4ccdffb640 10 client.18336 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:53.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.122+0000 7f4ccdffb640 1 -- 172.21.15.52:0/4225731991 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e94e2f6d70 con 0x55e94e39ed90 2024-04-02T14:02:53.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.122+0000 7f4ccdffb640 10 client.18336 handle_command_reply: tid=0 2024-04-02T14:02:53.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.122+0000 7f4cef9f1740 1 client.18336 shutdown 2024-04-02T14:02:53.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.122+0000 7f4cef9f1740 1 -- 172.21.15.52:0/4225731991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e94e3858d0 msgr2=0x55e94e38e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:53.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.122+0000 7f4cef9f1740 1 --2- 172.21.15.52:0/4225731991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e94e3858d0 0x55e94e38e650 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8012740 tx=0x7f4cd8012c00 comp rx=0 tx=0).stop 2024-04-02T14:02:53.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.122+0000 7f4cef9f1740 1 -- 172.21.15.52:0/4225731991 shutdown_connections 2024-04-02T14:02:53.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.122+0000 7f4cef9f1740 1 --2- 172.21.15.52:0/4225731991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e94e38edd0 0x55e94e38bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.122+0000 7f4cef9f1740 1 --2- 172.21.15.52:0/4225731991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e94e3861f0 0x55e94e38b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.122+0000 7f4cef9f1740 1 --2- 172.21.15.52:0/4225731991 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55e94e39ed90 0x55e94e3a11e0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.122+0000 7f4cef9f1740 1 --2- 172.21.15.52:0/4225731991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e94e3858d0 0x55e94e38e650 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.122+0000 7f4cef9f1740 1 -- 172.21.15.52:0/4225731991 >> 172.21.15.52:0/4225731991 conn(0x55e94e380000 msgr2=0x55e94e381d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:53.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.122+0000 7f4ccdffb640 0 client.18336 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:53.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.122+0000 7f4cef9f1740 1 -- 172.21.15.52:0/4225731991 shutdown_connections 2024-04-02T14:02:53.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.122+0000 7f4cef9f1740 1 -- 172.21.15.52:0/4225731991 wait complete. 2024-04-02T14:02:53.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.122+0000 7f4cef9f1740 20 client.18336 trim_cache size 0 max 16384 2024-04-02T14:02:53.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.143+0000 7f4cef9f1740 1 Processor -- start 2024-04-02T14:02:53.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.143+0000 7f4cef9f1740 1 -- start start 2024-04-02T14:02:53.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.143+0000 7f4cef9f1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e94e3858d0 0x55e94e39aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:53.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.143+0000 7f4cef9f1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e94e3861f0 0x55e94e38ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:53.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.143+0000 7f4cef9f1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e94e38edd0 0x55e94e37f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:53.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.143+0000 7f4cef9f1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81b10e0 con 0x55e94e3861f0 2024-04-02T14:02:53.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.143+0000 7f4cef9f1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81b2c20 con 0x55e94e3858d0 2024-04-02T14:02:53.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.143+0000 7f4cef9f1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81b17b0 con 0x55e94e38edd0 2024-04-02T14:02:53.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.143+0000 7f4cdffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e94e3861f0 0x55e94e38ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:53.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.143+0000 7f4cdffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e94e3861f0 0x55e94e38ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46546/0 (socket says 172.21.15.52:46546) 2024-04-02T14:02:53.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.143+0000 7f4cdffff640 1 -- 172.21.15.52:0/1030210394 learned_addr learned my addr 172.21.15.52:0/1030210394 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:53.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.143+0000 7f4ce659c640 1 --2- 172.21.15.52:0/1030210394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e94e3858d0 0x55e94e39aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:53.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.143+0000 7f4ce6d9d640 1 --2- 172.21.15.52:0/1030210394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e94e38edd0 0x55e94e37f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:53.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.143+0000 7f4cdffff640 1 -- 172.21.15.52:0/1030210394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e94e38edd0 msgr2=0x55e94e37f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:53.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.143+0000 7f4cdffff640 1 --2- 172.21.15.52:0/1030210394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e94e38edd0 0x55e94e37f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.143+0000 7f4cdffff640 1 -- 172.21.15.52:0/1030210394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e94e3858d0 msgr2=0x55e94e39aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:53.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.143+0000 7f4cdffff640 1 --2- 172.21.15.52:0/1030210394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e94e3858d0 0x55e94e39aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.143+0000 7f4cdffff640 1 -- 172.21.15.52:0/1030210394 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e94e386020 con 0x55e94e3861f0 2024-04-02T14:02:53.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.144+0000 7f4ce659c640 1 --2- 172.21.15.52:0/1030210394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e94e3858d0 0x55e94e39aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:02:53.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.144+0000 7f4cdffff640 1 --2- 172.21.15.52:0/1030210394 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e94e3861f0 0x55e94e38ab40 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0026000 tx=0x7f4cd0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c129bebbb2ae6b53 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:53.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.144+0000 7f4ccffff640 1 -- 172.21.15.52:0/1030210394 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd003c070 con 0x55e94e3861f0 2024-04-02T14:02:53.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.144+0000 7f4ccffff640 1 -- 172.21.15.52:0/1030210394 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cd0003ad0 con 0x55e94e3861f0 2024-04-02T14:02:53.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.144+0000 7f4ccffff640 1 -- 172.21.15.52:0/1030210394 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0036030 con 0x55e94e3861f0 2024-04-02T14:02:53.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.145+0000 7f4cef9f1740 1 -- 172.21.15.52:0/1030210394 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e94e3861f0 msgr2=0x55e94e38ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:53.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.145+0000 7f4cef9f1740 1 --2- 172.21.15.52:0/1030210394 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e94e3861f0 0x55e94e38ab40 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0026000 tx=0x7f4cd0005dc0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.145+0000 7f4cef9f1740 1 -- 172.21.15.52:0/1030210394 shutdown_connections 2024-04-02T14:02:53.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.145+0000 7f4cef9f1740 1 --2- 172.21.15.52:0/1030210394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e94e38edd0 0x55e94e37f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.145+0000 7f4cef9f1740 1 --2- 172.21.15.52:0/1030210394 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e94e3861f0 0x55e94e38ab40 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.145+0000 7f4cef9f1740 1 --2- 172.21.15.52:0/1030210394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e94e3858d0 0x55e94e39aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.145+0000 7f4cef9f1740 1 -- 172.21.15.52:0/1030210394 >> 172.21.15.52:0/1030210394 conn(0x55e94e380000 msgr2=0x55e94e37e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:53.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.145+0000 7f4cef9f1740 1 -- 172.21.15.52:0/1030210394 shutdown_connections 2024-04-02T14:02:53.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.145+0000 7f4cef9f1740 1 -- 172.21.15.52:0/1030210394 wait complete. 2024-04-02T14:02:53.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.145+0000 7f4cef9f1740 1 -- start start 2024-04-02T14:02:53.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.145+0000 7f4cef9f1740 1 Processor -- start 2024-04-02T14:02:53.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.146+0000 7f4cef9f1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e94e3858d0 0x55e94e396f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:53.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.146+0000 7f4cef9f1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e94e3861f0 0x55e94e397470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:53.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.146+0000 7f4cef9f1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e94e38edd0 0x55e94e387c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:53.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.146+0000 7f4cef9f1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81b2550 con 0x55e94e38edd0 2024-04-02T14:02:53.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.146+0000 7f4cef9f1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81b1e80 con 0x55e94e3861f0 2024-04-02T14:02:53.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.146+0000 7f4cef9f1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81af0e0 con 0x55e94e3858d0 2024-04-02T14:02:53.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.146+0000 7f4cef9f1740 10 client.0 _ll_register_callbacks cb 0x7f4cd8009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:02:53.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.146+0000 7f4cdffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e94e3861f0 0x55e94e397470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:53.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.146+0000 7f4cdffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e94e3861f0 0x55e94e397470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:40970/0 (socket says 172.21.15.52:40970) 2024-04-02T14:02:53.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.146+0000 7f4ce659c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e94e3858d0 0x55e94e396f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:53.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.146+0000 7f4ce6d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e94e38edd0 0x55e94e387c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:53.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.146+0000 7f4cdffff640 1 -- 172.21.15.52:0/4051430260 learned_addr learned my addr 172.21.15.52:0/4051430260 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:02:53.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.146+0000 7f4ce659c640 1 -- 172.21.15.52:0/4051430260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e94e3861f0 msgr2=0x55e94e397470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:53.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.146+0000 7f4ce659c640 1 --2- 172.21.15.52:0/4051430260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e94e3861f0 0x55e94e397470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.146+0000 7f4ce659c640 1 -- 172.21.15.52:0/4051430260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e94e38edd0 msgr2=0x55e94e387c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:02:53.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.146+0000 7f4ce659c640 1 --2- 172.21.15.52:0/4051430260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e94e38edd0 0x55e94e387c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.146+0000 7f4ce659c640 1 -- 172.21.15.52:0/4051430260 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e94e388190 con 0x55e94e3858d0 2024-04-02T14:02:53.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.146+0000 7f4ce6d9d640 1 --2- 172.21.15.52:0/4051430260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e94e38edd0 0x55e94e387c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:02:53.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.146+0000 7f4ce659c640 1 --2- 172.21.15.52:0/4051430260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e94e3858d0 0x55e94e396f30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4cd801ca80 tx=0x7f4cd801ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60042afb63348849 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:02:53.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.146+0000 7f4ccdffb640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-02T14:02:53.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.147+0000 7f4ccdffb640 1 -- 172.21.15.52:0/4051430260 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd802f060 con 0x55e94e3858d0 2024-04-02T14:02:53.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.147+0000 7f4cef9f1740 1 -- 172.21.15.52:0/4051430260 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cd0003ad0 con 0x55e94e3858d0 2024-04-02T14:02:53.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.147+0000 7f4ccdffb640 1 -- 172.21.15.52:0/4051430260 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f4cd801c630 con 0x55e94e3858d0 2024-04-02T14:02:53.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.147+0000 7f4ccdffb640 1 -- 172.21.15.52:0/4051430260 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4cd800fe60 con 0x55e94e3858d0 2024-04-02T14:02:53.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.147+0000 7f4cef9f1740 10 client.27181 fetch_fsmap learned FSMap version 30 2024-04-02T14:02:53.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.147+0000 7f4cef9f1740 1 -- 172.21.15.52:0/4051430260 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e94e404af0 con 0x55e94e3858d0 2024-04-02T14:02:53.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.147+0000 7f4ccdffb640 1 -- 172.21.15.52:0/4051430260 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cd0003ad0 con 0x55e94e3858d0 2024-04-02T14:02:53.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.147+0000 7f4ccdffb640 1 -- 172.21.15.52:0/4051430260 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4cd8003a60 con 0x55e94e3858d0 2024-04-02T14:02:53.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.147+0000 7f4cef9f1740 10 client.27181 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:02:53.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.147+0000 7f4cef9f1740 10 client.27181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:02:53.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.147+0000 7f4cef9f1740 20 client.27181 populate_metadata read hostname 'smithi052' 2024-04-02T14:02:53.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.147+0000 7f4cef9f1740 1 --2- 172.21.15.52:0/4051430260 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55e94e405270 0x55e94e407690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:02:53.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.147+0000 7f4cef9f1740 4 client.27181 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:02:53.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.147+0000 7f4cef9f1740 1 -- 172.21.15.52:0/4051430260 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e94e2f10c0 con 0x55e94e405270 2024-04-02T14:02:53.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.147+0000 7f4cdffff640 1 --2- 172.21.15.52:0/4051430260 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55e94e405270 0x55e94e407690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:02:53.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.148+0000 7f4ccdffb640 10 client.27181 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:53.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.148+0000 7f4cdffff640 1 --2- 172.21.15.52:0/4051430260 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55e94e405270 0x55e94e407690 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef743765aff4bb98 server_cookie=dc227f71c9339f3f in_seq=0 out_seq=0 2024-04-02T14:02:53.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.148+0000 7f4ccdffb640 1 -- 172.21.15.52:0/4051430260 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55e94e2f10c0 con 0x55e94e405270 2024-04-02T14:02:53.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.148+0000 7f4ccdffb640 10 client.27181 handle_command_reply: tid=0 2024-04-02T14:02:53.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.148+0000 7f4cef9f1740 1 client.27181 shutdown 2024-04-02T14:02:53.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.148+0000 7f4cef9f1740 1 -- 172.21.15.52:0/4051430260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e94e3858d0 msgr2=0x55e94e396f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:53.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.148+0000 7f4cef9f1740 1 --2- 172.21.15.52:0/4051430260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e94e3858d0 0x55e94e396f30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4cd801ca80 tx=0x7f4cd801ce30 comp rx=0 tx=0).stop 2024-04-02T14:02:53.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.148+0000 7f4cef9f1740 1 -- 172.21.15.52:0/4051430260 shutdown_connections 2024-04-02T14:02:53.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.148+0000 7f4cef9f1740 1 --2- 172.21.15.52:0/4051430260 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55e94e405270 0x55e94e407690 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.148+0000 7f4cef9f1740 1 --2- 172.21.15.52:0/4051430260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e94e38edd0 0x55e94e387c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.148+0000 7f4cef9f1740 1 --2- 172.21.15.52:0/4051430260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e94e3861f0 0x55e94e397470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.148+0000 7f4ccdffb640 0 client.27181 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:02:53.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.148+0000 7f4cef9f1740 1 --2- 172.21.15.52:0/4051430260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e94e3858d0 0x55e94e396f30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.148+0000 7f4cef9f1740 1 -- 172.21.15.52:0/4051430260 >> 172.21.15.52:0/4051430260 conn(0x55e94e380000 msgr2=0x55e94e386710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:53.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.149+0000 7f4cef9f1740 1 -- 172.21.15.52:0/4051430260 shutdown_connections 2024-04-02T14:02:53.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.149+0000 7f4cef9f1740 1 -- 172.21.15.52:0/4051430260 wait complete. 2024-04-02T14:02:53.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.149+0000 7f4cef9f1740 20 client.27181 trim_cache size 0 max 16384 2024-04-02T14:02:53.163 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:02:53.164 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "scrub active (2405 inodes in the stack)", 2024-04-02T14:02:53.164 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-04-02T14:02:53.164 INFO:teuthology.orchestra.run.smithi052.stdout: "994d29a7-d20f-4359-b018-0aeefb3a99af": { 2024-04-02T14:02:53.164 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-04-02T14:02:53.164 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "994d29a7-d20f-4359-b018-0aeefb3a99af", 2024-04-02T14:02:53.164 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-04-02T14:02:53.165 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:02:53.165 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:02:53.165 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:02:53.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.149+0000 7f4cece0f640 1 -- 172.21.15.52:0/3816239616 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4cd40a3d20 msgr2=0x7f4cd40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:53.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.149+0000 7f4cece0f640 1 --2- 172.21.15.52:0/3816239616 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4cd40a3d20 0x7f4cd40a61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000f790 tx=0x7f4cd000e000 comp rx=0 tx=0).stop 2024-04-02T14:02:53.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.149+0000 7f4cece0f640 1 -- 172.21.15.52:0/3816239616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ce819ee30 msgr2=0x7f4ce819f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:02:53.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.149+0000 7f4cece0f640 1 --2- 172.21.15.52:0/3816239616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ce819ee30 0x7f4ce819f2e0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f4ce002c700 tx=0x7f4ce0032f20 comp rx=0 tx=0).stop 2024-04-02T14:02:53.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.149+0000 7f4cece0f640 1 -- 172.21.15.52:0/3816239616 shutdown_connections 2024-04-02T14:02:53.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.149+0000 7f4cece0f640 1 --2- 172.21.15.52:0/3816239616 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f4cd40a3d20 0x7f4cd40a61e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.149+0000 7f4cece0f640 1 --2- 172.21.15.52:0/3816239616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ce819ee30 0x7f4ce819f2e0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.149+0000 7f4cece0f640 1 --2- 172.21.15.52:0/3816239616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ce8106dd0 0x7f4ce819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.149+0000 7f4cece0f640 1 --2- 172.21.15.52:0/3816239616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ce8100210 0x7f4ce819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:02:53.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.149+0000 7f4cece0f640 1 -- 172.21.15.52:0/3816239616 >> 172.21.15.52:0/3816239616 conn(0x7f4ce8076f20 msgr2=0x7f4ce8105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:02:53.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.149+0000 7f4cece0f640 1 -- 172.21.15.52:0/3816239616 shutdown_connections 2024-04-02T14:02:53.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:02:53.150+0000 7f4cece0f640 1 -- 172.21.15.52:0/3816239616 wait complete. 2024-04-02T14:02:53.168 INFO:tasks.cephfs.filesystem:scrub status for tag:994d29a7-d20f-4359-b018-0aeefb3a99af - {'path': '/', 'tag': '994d29a7-d20f-4359-b018-0aeefb3a99af', 'options': 'recursive,force'} 2024-04-02T14:02:53.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:53 smithi189 ceph-mon[36683]: pgmap v290: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:02:53.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:53 smithi189 ceph-mon[36683]: scrub queued for path: / 2024-04-02T14:02:53.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:53 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:02:53.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:53 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T14:02:53.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:53 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T14:02:53.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:53 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:02:53.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:53 smithi052 ceph-mon[32104]: pgmap v290: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:02:53.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:53 smithi052 ceph-mon[32104]: scrub queued for path: / 2024-04-02T14:02:53.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:53 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:02:53.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:53 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T14:02:53.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:53 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T14:02:53.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:53 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:02:53.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:53 smithi088 ceph-mon[36528]: pgmap v290: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:02:53.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:53 smithi088 ceph-mon[36528]: scrub queued for path: / 2024-04-02T14:02:53.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:53 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:02:53.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:53 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T14:02:53.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:53 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T14:02:53.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:53 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:02:54.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:54 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T14:02:54.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:54 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:02:54.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:54 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:02:54.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:54 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T14:02:54.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:54 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:02:54.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:54 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:02:54.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:54 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T14:02:54.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:54 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:02:54.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:54 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:02:55.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:55 smithi189 ceph-mon[36683]: pgmap v291: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:02:55.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:55 smithi052 ceph-mon[32104]: pgmap v291: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:02:55.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:55 smithi088 ceph-mon[36528]: pgmap v291: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:02:57.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:57 smithi189 ceph-mon[36683]: pgmap v292: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T14:02:57.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:57 smithi052 ceph-mon[32104]: pgmap v292: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T14:02:57.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:57 smithi088 ceph-mon[36528]: pgmap v292: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T14:02:59.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:02:59 smithi189 ceph-mon[36683]: pgmap v293: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:02:59.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:02:59 smithi052 ceph-mon[32104]: pgmap v293: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:02:59.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:02:59 smithi088 ceph-mon[36528]: pgmap v293: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:03:00.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:00 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T14:03:00.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:00 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T14:03:00.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:00 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T14:03:01.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:01 smithi052 ceph-mon[32104]: pgmap v294: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:03:01.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:01 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T14:03:01.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:01 smithi088 ceph-mon[36528]: pgmap v294: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:03:01.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:01 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T14:03:02.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:01 smithi189 ceph-mon[36683]: pgmap v294: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:03:02.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:01 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T14:03:03.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:03 smithi052 ceph-mon[32104]: pgmap v295: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:03:03.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:03 smithi088 ceph-mon[36528]: pgmap v295: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:03:04.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:03 smithi189 ceph-mon[36683]: pgmap v295: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:03:05.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:05 smithi052 ceph-mon[32104]: pgmap v296: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:03:05.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:05 smithi088 ceph-mon[36528]: pgmap v296: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:03:06.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:05 smithi189 ceph-mon[36683]: pgmap v296: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:03:07.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:07 smithi052 ceph-mon[32104]: pgmap v297: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:03:07.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:07 smithi189 ceph-mon[36683]: pgmap v297: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:03:07.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:07 smithi088 ceph-mon[36528]: pgmap v297: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:03:09.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:09 smithi052 ceph-mon[32104]: pgmap v298: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:03:09.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:09 smithi088 ceph-mon[36528]: pgmap v298: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:03:10.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:09 smithi189 ceph-mon[36683]: pgmap v298: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:03:11.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:11 smithi052 ceph-mon[32104]: pgmap v299: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:03:11.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:11 smithi052 ceph-mon[32104]: osdmap e100: 12 total, 12 up, 12 in 2024-04-02T14:03:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:11 smithi088 ceph-mon[36528]: pgmap v299: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:03:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:11 smithi088 ceph-mon[36528]: osdmap e100: 12 total, 12 up, 12 in 2024-04-02T14:03:12.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:11 smithi189 ceph-mon[36683]: pgmap v299: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:03:12.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:11 smithi189 ceph-mon[36683]: osdmap e100: 12 total, 12 up, 12 in 2024-04-02T14:03:13.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:13 smithi052 ceph-mon[32104]: pgmap v301: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:03:13.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:13 smithi088 ceph-mon[36528]: pgmap v301: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:03:14.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:13 smithi189 ceph-mon[36683]: pgmap v301: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:03:15.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:15 smithi052 ceph-mon[32104]: pgmap v302: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:03:15.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:15 smithi088 ceph-mon[36528]: pgmap v302: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:03:16.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:15 smithi189 ceph-mon[36683]: pgmap v302: 129 pgs: 129 active+clean; 73 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:03:17.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:17 smithi052 ceph-mon[32104]: pgmap v303: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:03:17.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:17 smithi088 ceph-mon[36528]: pgmap v303: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:03:18.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:17 smithi189 ceph-mon[36683]: pgmap v303: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:03:19.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:19 smithi052 ceph-mon[32104]: pgmap v304: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:03:19.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:19 smithi088 ceph-mon[36528]: pgmap v304: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:03:20.038 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:19 smithi189 ceph-mon[36683]: pgmap v304: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:03:21.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:21 smithi189 ceph-mon[36683]: pgmap v305: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:03:21.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:21 smithi189 ceph-mon[36683]: osdmap e101: 12 total, 12 up, 12 in 2024-04-02T14:03:21.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:21 smithi052 ceph-mon[32104]: pgmap v305: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:03:21.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:21 smithi052 ceph-mon[32104]: osdmap e101: 12 total, 12 up, 12 in 2024-04-02T14:03:21.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:21 smithi088 ceph-mon[36528]: pgmap v305: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:03:21.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:21 smithi088 ceph-mon[36528]: osdmap e101: 12 total, 12 up, 12 in 2024-04-02T14:03:23.169 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:03:23.170 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:03:23.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.284+0000 7f825c02a640 1 -- 172.21.15.52:0/2018036587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8254100970 msgr2=0x7f8254100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.284+0000 7f825c02a640 1 --2- 172.21.15.52:0/2018036587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8254100970 0x7f8254100df0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f8244009930 tx=0x7f824402bc30 comp rx=0 tx=0).stop 2024-04-02T14:03:23.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.284+0000 7f825c02a640 1 -- 172.21.15.52:0/2018036587 shutdown_connections 2024-04-02T14:03:23.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.284+0000 7f825c02a640 1 --2- 172.21.15.52:0/2018036587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8254101330 0x7f825410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.284+0000 7f825c02a640 1 --2- 172.21.15.52:0/2018036587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8254100970 0x7f8254100df0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.284+0000 7f825c02a640 1 --2- 172.21.15.52:0/2018036587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8254106980 0x7f8254106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.284+0000 7f825c02a640 1 -- 172.21.15.52:0/2018036587 >> 172.21.15.52:0/2018036587 conn(0x7f8254077b00 msgr2=0x7f8254077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:23.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.284+0000 7f825c02a640 1 -- 172.21.15.52:0/2018036587 shutdown_connections 2024-04-02T14:03:23.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.284+0000 7f825c02a640 1 -- 172.21.15.52:0/2018036587 wait complete. 2024-04-02T14:03:23.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.285+0000 7f825c02a640 1 Processor -- start 2024-04-02T14:03:23.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.285+0000 7f825c02a640 1 -- start start 2024-04-02T14:03:23.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.285+0000 7f825c02a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8254100970 0x7f8254199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.285+0000 7f825c02a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8254101330 0x7f82541996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.285+0000 7f825c02a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8254106980 0x7f82541a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.285+0000 7f825c02a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8254110d90 con 0x7f8254106980 2024-04-02T14:03:23.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.285+0000 7f825c02a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8254110c10 con 0x7f8254101330 2024-04-02T14:03:23.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.285+0000 7f825c02a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8254110f10 con 0x7f8254100970 2024-04-02T14:03:23.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.285+0000 7f825a5c7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8254106980 0x7f82541a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.285+0000 7f825a5c7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8254106980 0x7f82541a41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56964/0 (socket says 172.21.15.52:56964) 2024-04-02T14:03:23.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.285+0000 7f825a5c7640 1 -- 172.21.15.52:0/2694413889 learned_addr learned my addr 172.21.15.52:0/2694413889 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:23.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.285+0000 7f82595c5640 1 --2- 172.21.15.52:0/2694413889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8254101330 0x7f82541996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.285+0000 7f8259dc6640 1 --2- 172.21.15.52:0/2694413889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8254100970 0x7f8254199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.285+0000 7f825a5c7640 1 -- 172.21.15.52:0/2694413889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8254100970 msgr2=0x7f8254199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.285+0000 7f825a5c7640 1 --2- 172.21.15.52:0/2694413889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8254100970 0x7f8254199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.285+0000 7f825a5c7640 1 -- 172.21.15.52:0/2694413889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8254101330 msgr2=0x7f82541996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.285+0000 7f825a5c7640 1 --2- 172.21.15.52:0/2694413889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8254101330 0x7f82541996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.285+0000 7f825a5c7640 1 -- 172.21.15.52:0/2694413889 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82541a4730 con 0x7f8254106980 2024-04-02T14:03:23.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.285+0000 7f825a5c7640 1 --2- 172.21.15.52:0/2694413889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8254106980 0x7f82541a41f0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f824c00eaa0 tx=0x7f824c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:23.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.286+0000 7f824affd640 1 -- 172.21.15.52:0/2694413889 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c00cdb0 con 0x7f8254106980 2024-04-02T14:03:23.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.286+0000 7f824affd640 1 -- 172.21.15.52:0/2694413889 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f824c005020 con 0x7f8254106980 2024-04-02T14:03:23.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.286+0000 7f825c02a640 1 -- 172.21.15.52:0/2694413889 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82541a4900 con 0x7f8254106980 2024-04-02T14:03:23.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.286+0000 7f824affd640 1 -- 172.21.15.52:0/2694413889 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c0116b0 con 0x7f8254106980 2024-04-02T14:03:23.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.286+0000 7f825c02a640 1 -- 172.21.15.52:0/2694413889 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82541a4e20 con 0x7f8254106980 2024-04-02T14:03:23.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.288+0000 7f824affd640 1 -- 172.21.15.52:0/2694413889 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f824c005560 con 0x7f8254106980 2024-04-02T14:03:23.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.288+0000 7f825ec11740 1 Processor -- start 2024-04-02T14:03:23.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.288+0000 7f825ec11740 1 -- start start 2024-04-02T14:03:23.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.289+0000 7f825ec11740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6e619b8d0 0x55a6e619bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.289+0000 7f825ec11740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619c1f0 0x55a6e61a4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.289+0000 7f825ec11740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6e61a4dd0 0x55a6e61a71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.289+0000 7f825ec11740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a6e60ff830 con 0x55a6e619c1f0 2024-04-02T14:03:23.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.289+0000 7f825ec11740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a6e60a7820 con 0x55a6e61a4dd0 2024-04-02T14:03:23.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.289+0000 7f825ec11740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a6e60fb7e0 con 0x55a6e619b8d0 2024-04-02T14:03:23.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.289+0000 7f824affd640 1 --2- 172.21.15.52:0/2694413889 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f82400a3d20 0x7f82400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.289+0000 7f8259dc6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619c1f0 0x55a6e61a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.289+0000 7f8259dc6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619c1f0 0x55a6e61a4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56974/0 (socket says 172.21.15.52:56974) 2024-04-02T14:03:23.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.289+0000 7f8259dc6640 1 -- 172.21.15.52:0/921272640 learned_addr learned my addr 172.21.15.52:0/921272640 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:23.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.289+0000 7f825a5c7640 1 --2- 172.21.15.52:0/2694413889 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f82400a3d20 0x7f82400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.289+0000 7f82595c5640 1 --2- 172.21.15.52:0/921272640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6e619b8d0 0x55a6e619bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.289+0000 7f82595c5640 1 --2- 172.21.15.52:0/921272640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6e61a4dd0 0x55a6e61a71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.289+0000 7f824affd640 1 -- 172.21.15.52:0/2694413889 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f824c0d0ef0 con 0x7f8254106980 2024-04-02T14:03:23.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.289+0000 7f8259dc6640 1 -- 172.21.15.52:0/921272640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6e619b8d0 msgr2=0x55a6e619bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:23.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.289+0000 7f8259dc6640 1 --2- 172.21.15.52:0/921272640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6e619b8d0 0x55a6e619bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.289+0000 7f8259dc6640 1 -- 172.21.15.52:0/921272640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6e61a4dd0 msgr2=0x55a6e61a71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:23.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.289+0000 7f8259dc6640 1 --2- 172.21.15.52:0/921272640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6e61a4dd0 0x55a6e61a71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.289+0000 7f8259dc6640 1 -- 172.21.15.52:0/921272640 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6e5edd7d0 con 0x55a6e619c1f0 2024-04-02T14:03:23.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.289+0000 7f82595c5640 1 --2- 172.21.15.52:0/921272640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6e61a4dd0 0x55a6e61a71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:03:23.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.289+0000 7f825a5c7640 1 --2- 172.21.15.52:0/2694413889 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f82400a3d20 0x7f82400a61e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f824c099470 tx=0x7f824c06e040 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:23.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.290+0000 7f8259dc6640 1 --2- 172.21.15.52:0/921272640 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619c1f0 0x55a6e61a4890 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f825000bac0 tx=0x7f825000bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a283baf6657a18 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:23.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.290+0000 7f823ffff640 1 -- 172.21.15.52:0/921272640 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250017420 con 0x55a6e619c1f0 2024-04-02T14:03:23.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.290+0000 7f823ffff640 1 -- 172.21.15.52:0/921272640 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8250017aa0 con 0x55a6e619c1f0 2024-04-02T14:03:23.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.290+0000 7f823ffff640 1 -- 172.21.15.52:0/921272640 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825000fa90 con 0x55a6e619c1f0 2024-04-02T14:03:23.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.290+0000 7f825ec11740 1 -- 172.21.15.52:0/921272640 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619c1f0 msgr2=0x55a6e61a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.290+0000 7f825ec11740 1 --2- 172.21.15.52:0/921272640 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619c1f0 0x55a6e61a4890 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f825000bac0 tx=0x7f825000bf80 comp rx=0 tx=0).stop 2024-04-02T14:03:23.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.290+0000 7f825ec11740 1 -- 172.21.15.52:0/921272640 shutdown_connections 2024-04-02T14:03:23.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.290+0000 7f825ec11740 1 --2- 172.21.15.52:0/921272640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6e61a4dd0 0x55a6e61a71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.290+0000 7f825ec11740 1 --2- 172.21.15.52:0/921272640 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619c1f0 0x55a6e61a4890 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.290+0000 7f825ec11740 1 --2- 172.21.15.52:0/921272640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6e619b8d0 0x55a6e619bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.290+0000 7f825ec11740 1 -- 172.21.15.52:0/921272640 >> 172.21.15.52:0/921272640 conn(0x55a6e6196000 msgr2=0x55a6e61963e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:23.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.291+0000 7f825ec11740 1 -- 172.21.15.52:0/921272640 shutdown_connections 2024-04-02T14:03:23.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.291+0000 7f825ec11740 1 -- 172.21.15.52:0/921272640 wait complete. 2024-04-02T14:03:23.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.291+0000 7f825ec11740 1 -- start start 2024-04-02T14:03:23.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.291+0000 7f825ec11740 1 Processor -- start 2024-04-02T14:03:23.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.291+0000 7f825ec11740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619b8d0 0x55a6e61a4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.291+0000 7f825ec11740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6e619c1f0 0x55a6e61a1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.291+0000 7f825ec11740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6e61a4dd0 0x55a6e61a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.291+0000 7f825ec11740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a6e61a7bc0 con 0x55a6e619b8d0 2024-04-02T14:03:23.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.291+0000 7f825ec11740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a6e60bad30 con 0x55a6e619c1f0 2024-04-02T14:03:23.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.291+0000 7f825ec11740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a6e61a7d40 con 0x55a6e61a4dd0 2024-04-02T14:03:23.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.291+0000 7f825ec11740 10 client.0 _ll_register_callbacks cb 0x55a6e60ba720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:23.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.292+0000 7f82595c5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619b8d0 0x55a6e61a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.292+0000 7f82595c5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619b8d0 0x55a6e61a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56984/0 (socket says 172.21.15.52:56984) 2024-04-02T14:03:23.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.292+0000 7f82595c5640 1 -- 172.21.15.52:0/903265198 learned_addr learned my addr 172.21.15.52:0/903265198 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:23.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.292+0000 7f82595c5640 1 --2- 172.21.15.52:0/903265198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6e61a4dd0 0x55a6e61a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.292+0000 7f8259dc6640 1 --2- 172.21.15.52:0/903265198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6e619c1f0 0x55a6e61a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.292+0000 7f82595c5640 1 -- 172.21.15.52:0/903265198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6e61a4dd0 msgr2=0x55a6e61a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:23.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.292+0000 7f82595c5640 1 --2- 172.21.15.52:0/903265198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6e61a4dd0 0x55a6e61a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.292+0000 7f82595c5640 1 -- 172.21.15.52:0/903265198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6e619c1f0 msgr2=0x55a6e61a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:23.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.292+0000 7f82595c5640 1 --2- 172.21.15.52:0/903265198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6e619c1f0 0x55a6e61a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.292+0000 7f82595c5640 1 -- 172.21.15.52:0/903265198 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6e5e133f0 con 0x55a6e619b8d0 2024-04-02T14:03:23.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.292+0000 7f8259dc6640 1 --2- 172.21.15.52:0/903265198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6e619c1f0 0x55a6e61a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:03:23.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.292+0000 7f82595c5640 1 --2- 172.21.15.52:0/903265198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619b8d0 0x55a6e61a4600 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f8244005ec0 tx=0x7f8244005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a60f4affd3e90a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:23.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.292+0000 7f823d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:23.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.292+0000 7f823d7fa640 1 -- 172.21.15.52:0/903265198 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8244005020 con 0x55a6e619b8d0 2024-04-02T14:03:23.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.292+0000 7f823d7fa640 1 -- 172.21.15.52:0/903265198 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f82440051c0 con 0x55a6e619b8d0 2024-04-02T14:03:23.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.292+0000 7f825ec11740 1 -- 172.21.15.52:0/903265198 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8250017aa0 con 0x55a6e619b8d0 2024-04-02T14:03:23.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.292+0000 7f823d7fa640 1 -- 172.21.15.52:0/903265198 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8244053050 con 0x55a6e619b8d0 2024-04-02T14:03:23.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.292+0000 7f825ec11740 10 client.18354 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:23.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.292+0000 7f825ec11740 1 -- 172.21.15.52:0/903265198 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a6e5dd30f0 con 0x55a6e619b8d0 2024-04-02T14:03:23.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.293+0000 7f823d7fa640 1 -- 172.21.15.52:0/903265198 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8250017aa0 con 0x55a6e619b8d0 2024-04-02T14:03:23.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.293+0000 7f823d7fa640 1 -- 172.21.15.52:0/903265198 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f82440036b0 con 0x55a6e619b8d0 2024-04-02T14:03:23.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.293+0000 7f825ec11740 10 client.18354 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:23.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.293+0000 7f825ec11740 10 client.18354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:03:23.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.293+0000 7f825ec11740 20 client.18354 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:23.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.293+0000 7f825ec11740 1 --2- 172.21.15.52:0/903265198 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55a6e61b4d00 0x55a6e61b7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.293+0000 7f825ec11740 4 client.18354 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:03:23.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.293+0000 7f825ec11740 1 -- 172.21.15.52:0/903265198 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6e610cd70 con 0x55a6e61b4d00 2024-04-02T14:03:23.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.293+0000 7f8259dc6640 1 --2- 172.21.15.52:0/903265198 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55a6e61b4d00 0x55a6e61b7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.294+0000 7f8259dc6640 1 --2- 172.21.15.52:0/903265198 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55a6e61b4d00 0x55a6e61b7130 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e73b306220e84ae server_cookie=eb48c2083e55f1ca in_seq=0 out_seq=0 2024-04-02T14:03:23.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.294+0000 7f823d7fa640 10 client.18354 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:03:23.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.294+0000 7f823d7fa640 1 -- 172.21.15.52:0/903265198 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a6e610cd70 con 0x55a6e61b4d00 2024-04-02T14:03:23.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.294+0000 7f823d7fa640 10 client.18354 handle_command_reply: tid=0 2024-04-02T14:03:23.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.294+0000 7f825ec11740 1 client.18354 shutdown 2024-04-02T14:03:23.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.294+0000 7f825ec11740 1 -- 172.21.15.52:0/903265198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619b8d0 msgr2=0x55a6e61a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.294+0000 7f825ec11740 1 --2- 172.21.15.52:0/903265198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619b8d0 0x55a6e61a4600 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f8244005ec0 tx=0x7f8244005e50 comp rx=0 tx=0).stop 2024-04-02T14:03:23.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.295+0000 7f825ec11740 1 -- 172.21.15.52:0/903265198 shutdown_connections 2024-04-02T14:03:23.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.295+0000 7f825ec11740 1 --2- 172.21.15.52:0/903265198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6e61a4dd0 0x55a6e61a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.295+0000 7f825ec11740 1 --2- 172.21.15.52:0/903265198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6e619c1f0 0x55a6e61a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.295+0000 7f825ec11740 1 --2- 172.21.15.52:0/903265198 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55a6e61b4d00 0x55a6e61b7130 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.295+0000 7f825ec11740 1 --2- 172.21.15.52:0/903265198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619b8d0 0x55a6e61a4600 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.295+0000 7f825ec11740 1 -- 172.21.15.52:0/903265198 >> 172.21.15.52:0/903265198 conn(0x55a6e6196000 msgr2=0x55a6e6197d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:23.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.295+0000 7f823d7fa640 0 client.18354 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:03:23.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.295+0000 7f825ec11740 1 -- 172.21.15.52:0/903265198 shutdown_connections 2024-04-02T14:03:23.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.295+0000 7f825ec11740 1 -- 172.21.15.52:0/903265198 wait complete. 2024-04-02T14:03:23.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.295+0000 7f825ec11740 20 client.18354 trim_cache size 0 max 16384 2024-04-02T14:03:23.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.312+0000 7f825ec11740 1 Processor -- start 2024-04-02T14:03:23.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.312+0000 7f825ec11740 1 -- start start 2024-04-02T14:03:23.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.312+0000 7f825ec11740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6e619b8d0 0x55a6e61b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.312+0000 7f825ec11740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619c1f0 0x55a6e61a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.312+0000 7f825ec11740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6e61a4dd0 0x55a6e61a4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.312+0000 7f825ec11740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f82541aefc0 con 0x55a6e619c1f0 2024-04-02T14:03:23.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.312+0000 7f825ec11740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f82541b0b00 con 0x55a6e61a4dd0 2024-04-02T14:03:23.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.312+0000 7f825ec11740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f82541af690 con 0x55a6e619b8d0 2024-04-02T14:03:23.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.312+0000 7f8259dc6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619c1f0 0x55a6e61a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.313+0000 7f8259dc6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619c1f0 0x55a6e61a40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57000/0 (socket says 172.21.15.52:57000) 2024-04-02T14:03:23.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.313+0000 7f8259dc6640 1 -- 172.21.15.52:0/3116299084 learned_addr learned my addr 172.21.15.52:0/3116299084 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:23.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.313+0000 7f8259dc6640 1 -- 172.21.15.52:0/3116299084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6e619b8d0 msgr2=0x55a6e61b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:23.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.313+0000 7f82595c5640 1 --2- 172.21.15.52:0/3116299084 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6e61a4dd0 0x55a6e61a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.313+0000 7f8259dc6640 1 --2- 172.21.15.52:0/3116299084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6e619b8d0 0x55a6e61b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.313+0000 7f8259dc6640 1 -- 172.21.15.52:0/3116299084 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6e61a4dd0 msgr2=0x55a6e61a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:23.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.313+0000 7f8259dc6640 1 --2- 172.21.15.52:0/3116299084 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6e61a4dd0 0x55a6e61a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.313+0000 7f8259dc6640 1 -- 172.21.15.52:0/3116299084 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6e6195e30 con 0x55a6e619c1f0 2024-04-02T14:03:23.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.313+0000 7f82595c5640 1 --2- 172.21.15.52:0/3116299084 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6e61a4dd0 0x55a6e61a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:03:23.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.313+0000 7f8259dc6640 1 --2- 172.21.15.52:0/3116299084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619c1f0 0x55a6e61a40c0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f825000f660 tx=0x7f825000f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112d9f72cfc84384 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:23.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.313+0000 7f823f7fe640 1 -- 172.21.15.52:0/3116299084 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825002e070 con 0x55a6e619c1f0 2024-04-02T14:03:23.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.313+0000 7f823f7fe640 1 -- 172.21.15.52:0/3116299084 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8250028d50 con 0x55a6e619c1f0 2024-04-02T14:03:23.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.313+0000 7f823f7fe640 1 -- 172.21.15.52:0/3116299084 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250005470 con 0x55a6e619c1f0 2024-04-02T14:03:23.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.314+0000 7f825ec11740 1 -- 172.21.15.52:0/3116299084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619c1f0 msgr2=0x55a6e61a40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.314+0000 7f825ec11740 1 --2- 172.21.15.52:0/3116299084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619c1f0 0x55a6e61a40c0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f825000f660 tx=0x7f825000f5f0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.314+0000 7f825ec11740 1 -- 172.21.15.52:0/3116299084 shutdown_connections 2024-04-02T14:03:23.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.314+0000 7f825ec11740 1 --2- 172.21.15.52:0/3116299084 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6e61a4dd0 0x55a6e61a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.314+0000 7f825ec11740 1 --2- 172.21.15.52:0/3116299084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619c1f0 0x55a6e61a40c0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.314+0000 7f825ec11740 1 --2- 172.21.15.52:0/3116299084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6e619b8d0 0x55a6e61b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.314+0000 7f825ec11740 1 -- 172.21.15.52:0/3116299084 >> 172.21.15.52:0/3116299084 conn(0x55a6e6196000 msgr2=0x55a6e6197230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:23.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.314+0000 7f825ec11740 1 -- 172.21.15.52:0/3116299084 shutdown_connections 2024-04-02T14:03:23.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.314+0000 7f825ec11740 1 -- 172.21.15.52:0/3116299084 wait complete. 2024-04-02T14:03:23.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.314+0000 7f825ec11740 1 -- start start 2024-04-02T14:03:23.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.314+0000 7f825ec11740 1 Processor -- start 2024-04-02T14:03:23.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.315+0000 7f825ec11740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6e619b8d0 0x55a6e61ad390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.315+0000 7f825ec11740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619c1f0 0x55a6e61a8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.315+0000 7f825ec11740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6e61a4dd0 0x55a6e61a8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.315+0000 7f825ec11740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f82541b0430 con 0x55a6e619c1f0 2024-04-02T14:03:23.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.315+0000 7f825ec11740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f82541afd60 con 0x55a6e619b8d0 2024-04-02T14:03:23.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.315+0000 7f825ec11740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f82541acfc0 con 0x55a6e61a4dd0 2024-04-02T14:03:23.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.315+0000 7f825ec11740 10 client.0 _ll_register_callbacks cb 0x7f824402cda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:23.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.315+0000 7f8259dc6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619c1f0 0x55a6e61a8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.315+0000 7f8259dc6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619c1f0 0x55a6e61a8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57010/0 (socket says 172.21.15.52:57010) 2024-04-02T14:03:23.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.315+0000 7f8259dc6640 1 -- 172.21.15.52:0/1158667553 learned_addr learned my addr 172.21.15.52:0/1158667553 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:23.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.315+0000 7f82595c5640 1 --2- 172.21.15.52:0/1158667553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6e619b8d0 0x55a6e61ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.315+0000 7f82595c5640 1 --2- 172.21.15.52:0/1158667553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6e61a4dd0 0x55a6e61a8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.315+0000 7f8259dc6640 1 -- 172.21.15.52:0/1158667553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6e61a4dd0 msgr2=0x55a6e61a8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:23.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.315+0000 7f8259dc6640 1 --2- 172.21.15.52:0/1158667553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6e61a4dd0 0x55a6e61a8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.315+0000 7f8259dc6640 1 -- 172.21.15.52:0/1158667553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6e619b8d0 msgr2=0x55a6e61ad390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:23.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.315+0000 7f8259dc6640 1 --2- 172.21.15.52:0/1158667553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6e619b8d0 0x55a6e61ad390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.315+0000 7f8259dc6640 1 -- 172.21.15.52:0/1158667553 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6e61a9470 con 0x55a6e619c1f0 2024-04-02T14:03:23.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.315+0000 7f82595c5640 1 --2- 172.21.15.52:0/1158667553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6e61a4dd0 0x55a6e61a8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:03:23.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.315+0000 7f8259dc6640 1 --2- 172.21.15.52:0/1158667553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619c1f0 0x55a6e61a8750 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f8250006010 tx=0x7f8250005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e4d39a206eab54 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:23.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.315+0000 7f823d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:23.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.315+0000 7f823d7fa640 1 -- 172.21.15.52:0/1158667553 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250036020 con 0x55a6e619c1f0 2024-04-02T14:03:23.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.315+0000 7f823d7fa640 1 -- 172.21.15.52:0/1158667553 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f82500051b0 con 0x55a6e619c1f0 2024-04-02T14:03:23.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.315+0000 7f825ec11740 1 -- 172.21.15.52:0/1158667553 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8250028d50 con 0x55a6e619c1f0 2024-04-02T14:03:23.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.315+0000 7f823d7fa640 1 -- 172.21.15.52:0/1158667553 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f825002e040 con 0x55a6e619c1f0 2024-04-02T14:03:23.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.315+0000 7f825ec11740 10 client.18360 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:23.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.316+0000 7f825ec11740 1 -- 172.21.15.52:0/1158667553 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a6e621b200 con 0x55a6e619c1f0 2024-04-02T14:03:23.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.316+0000 7f823d7fa640 1 -- 172.21.15.52:0/1158667553 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8250028d50 con 0x55a6e619c1f0 2024-04-02T14:03:23.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.316+0000 7f823d7fa640 1 -- 172.21.15.52:0/1158667553 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8250028030 con 0x55a6e619c1f0 2024-04-02T14:03:23.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.316+0000 7f825ec11740 10 client.18360 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:23.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.316+0000 7f825ec11740 10 client.18360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:03:23.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.316+0000 7f825ec11740 20 client.18360 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:23.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.316+0000 7f825ec11740 1 --2- 172.21.15.52:0/1158667553 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55a6e621b790 0x55a6e621dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.316+0000 7f825ec11740 4 client.18360 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:03:23.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.316+0000 7f825ec11740 1 -- 172.21.15.52:0/1158667553 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a6e61070c0 con 0x55a6e621b790 2024-04-02T14:03:23.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.316+0000 7f82595c5640 1 --2- 172.21.15.52:0/1158667553 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55a6e621b790 0x55a6e621dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.316+0000 7f82595c5640 1 --2- 172.21.15.52:0/1158667553 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55a6e621b790 0x55a6e621dbb0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8a92cadb2c76307 server_cookie=cc7c40b4b10a73ea in_seq=0 out_seq=0 2024-04-02T14:03:23.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.316+0000 7f823d7fa640 10 client.18360 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:03:23.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.317+0000 7f823d7fa640 1 -- 172.21.15.52:0/1158667553 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a6e61070c0 con 0x55a6e621b790 2024-04-02T14:03:23.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.317+0000 7f823d7fa640 10 client.18360 handle_command_reply: tid=0 2024-04-02T14:03:23.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.317+0000 7f825ec11740 1 client.18360 shutdown 2024-04-02T14:03:23.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.317+0000 7f825ec11740 1 -- 172.21.15.52:0/1158667553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619c1f0 msgr2=0x55a6e61a8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.317+0000 7f825ec11740 1 --2- 172.21.15.52:0/1158667553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619c1f0 0x55a6e61a8750 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f8250006010 tx=0x7f8250005b10 comp rx=0 tx=0).stop 2024-04-02T14:03:23.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.317+0000 7f825ec11740 1 -- 172.21.15.52:0/1158667553 shutdown_connections 2024-04-02T14:03:23.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.317+0000 7f825ec11740 1 --2- 172.21.15.52:0/1158667553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6e61a4dd0 0x55a6e61a8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.317+0000 7f825ec11740 1 --2- 172.21.15.52:0/1158667553 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55a6e621b790 0x55a6e621dbb0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.317+0000 7f825ec11740 1 --2- 172.21.15.52:0/1158667553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a6e619c1f0 0x55a6e61a8750 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.317+0000 7f825ec11740 1 --2- 172.21.15.52:0/1158667553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6e619b8d0 0x55a6e61ad390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.317+0000 7f823d7fa640 0 client.18360 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:03:23.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.317+0000 7f825ec11740 1 -- 172.21.15.52:0/1158667553 >> 172.21.15.52:0/1158667553 conn(0x55a6e6196000 msgr2=0x55a6e6197890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:23.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.317+0000 7f825ec11740 1 -- 172.21.15.52:0/1158667553 shutdown_connections 2024-04-02T14:03:23.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.317+0000 7f825ec11740 1 -- 172.21.15.52:0/1158667553 wait complete. 2024-04-02T14:03:23.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.317+0000 7f825ec11740 20 client.18360 trim_cache size 0 max 16384 2024-04-02T14:03:23.334 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:03:23.335 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-04-02T14:03:23.335 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-04-02T14:03:23.335 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:03:23.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.318+0000 7f825c02a640 1 -- 172.21.15.52:0/2694413889 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f82400a3d20 msgr2=0x7f82400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.318+0000 7f825c02a640 1 --2- 172.21.15.52:0/2694413889 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f82400a3d20 0x7f82400a61e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f824c099470 tx=0x7f824c06e040 comp rx=0 tx=0).stop 2024-04-02T14:03:23.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.318+0000 7f825c02a640 1 -- 172.21.15.52:0/2694413889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8254106980 msgr2=0x7f82541a41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.318+0000 7f825c02a640 1 --2- 172.21.15.52:0/2694413889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8254106980 0x7f82541a41f0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f824c00eaa0 tx=0x7f824c00ef60 comp rx=0 tx=0).stop 2024-04-02T14:03:23.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.318+0000 7f825c02a640 1 -- 172.21.15.52:0/2694413889 shutdown_connections 2024-04-02T14:03:23.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.318+0000 7f825c02a640 1 --2- 172.21.15.52:0/2694413889 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f82400a3d20 0x7f82400a61e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.318+0000 7f825c02a640 1 --2- 172.21.15.52:0/2694413889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8254106980 0x7f82541a41f0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.318+0000 7f825c02a640 1 --2- 172.21.15.52:0/2694413889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8254101330 0x7f82541996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.318+0000 7f825c02a640 1 --2- 172.21.15.52:0/2694413889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8254100970 0x7f8254199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.318+0000 7f825c02a640 1 -- 172.21.15.52:0/2694413889 >> 172.21.15.52:0/2694413889 conn(0x7f8254077b00 msgr2=0x7f8254103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:23.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.318+0000 7f825c02a640 1 -- 172.21.15.52:0/2694413889 shutdown_connections 2024-04-02T14:03:23.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.318+0000 7f825c02a640 1 -- 172.21.15.52:0/2694413889 wait complete. 2024-04-02T14:03:23.338 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T14:03:23.338 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:03:23.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.438+0000 7f9a2437b640 1 -- 172.21.15.52:0/4168649198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a1c100a00 msgr2=0x7f9a1c100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.438+0000 7f9a2437b640 1 --2- 172.21.15.52:0/4168649198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a1c100a00 0x7f9a1c100e80 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c009930 tx=0x7f9a0c02bc30 comp rx=0 tx=0).stop 2024-04-02T14:03:23.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.438+0000 7f9a2437b640 1 -- 172.21.15.52:0/4168649198 shutdown_connections 2024-04-02T14:03:23.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.438+0000 7f9a2437b640 1 --2- 172.21.15.52:0/4168649198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a1c1013c0 0x7f9a1c10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.438+0000 7f9a2437b640 1 --2- 172.21.15.52:0/4168649198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a1c100a00 0x7f9a1c100e80 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.438+0000 7f9a2437b640 1 --2- 172.21.15.52:0/4168649198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a1c106a10 0x7f9a1c106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.438+0000 7f9a2437b640 1 -- 172.21.15.52:0/4168649198 >> 172.21.15.52:0/4168649198 conn(0x7f9a1c077b00 msgr2=0x7f9a1c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:23.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.438+0000 7f9a2437b640 1 -- 172.21.15.52:0/4168649198 shutdown_connections 2024-04-02T14:03:23.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.438+0000 7f9a2437b640 1 -- 172.21.15.52:0/4168649198 wait complete. 2024-04-02T14:03:23.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.438+0000 7f9a2437b640 1 Processor -- start 2024-04-02T14:03:23.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.438+0000 7f9a2437b640 1 -- start start 2024-04-02T14:03:23.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.438+0000 7f9a2437b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a1c100a00 0x7f9a1c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.438+0000 7f9a2437b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a1c1013c0 0x7f9a1c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.438+0000 7f9a2437b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a1c106a10 0x7f9a1c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.438+0000 7f9a2437b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c110df0 con 0x7f9a1c1013c0 2024-04-02T14:03:23.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.438+0000 7f9a2437b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c110c70 con 0x7f9a1c106a10 2024-04-02T14:03:23.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.438+0000 7f9a2437b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c110f70 con 0x7f9a1c100a00 2024-04-02T14:03:23.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.439+0000 7f9a21916640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a1c1013c0 0x7f9a1c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.439+0000 7f9a21916640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a1c1013c0 0x7f9a1c1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57036/0 (socket says 172.21.15.52:57036) 2024-04-02T14:03:23.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.439+0000 7f9a21916640 1 -- 172.21.15.52:0/83273045 learned_addr learned my addr 172.21.15.52:0/83273045 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:23.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.439+0000 7f9a22117640 1 --2- 172.21.15.52:0/83273045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a1c100a00 0x7f9a1c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.439+0000 7f9a22918640 1 --2- 172.21.15.52:0/83273045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a1c106a10 0x7f9a1c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.439+0000 7f9a21916640 1 -- 172.21.15.52:0/83273045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a1c100a00 msgr2=0x7f9a1c198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.439+0000 7f9a21916640 1 --2- 172.21.15.52:0/83273045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a1c100a00 0x7f9a1c198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.439+0000 7f9a21916640 1 -- 172.21.15.52:0/83273045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a1c106a10 msgr2=0x7f9a1c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.439+0000 7f9a21916640 1 --2- 172.21.15.52:0/83273045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a1c106a10 0x7f9a1c19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.439+0000 7f9a21916640 1 -- 172.21.15.52:0/83273045 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1c19d470 con 0x7f9a1c1013c0 2024-04-02T14:03:23.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.439+0000 7f9a21916640 1 --2- 172.21.15.52:0/83273045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a1c1013c0 0x7f9a1c1989c0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c009230 tx=0x7f9a0c02cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:23.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.439+0000 7f9a137fe640 1 -- 172.21.15.52:0/83273045 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c041070 con 0x7f9a1c1013c0 2024-04-02T14:03:23.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.439+0000 7f9a2437b640 1 -- 172.21.15.52:0/83273045 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a1c19d700 con 0x7f9a1c1013c0 2024-04-02T14:03:23.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.439+0000 7f9a137fe640 1 -- 172.21.15.52:0/83273045 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a0c004450 con 0x7f9a1c1013c0 2024-04-02T14:03:23.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.439+0000 7f9a137fe640 1 -- 172.21.15.52:0/83273045 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c034440 con 0x7f9a1c1013c0 2024-04-02T14:03:23.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.439+0000 7f9a2437b640 1 -- 172.21.15.52:0/83273045 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a1c1a4ff0 con 0x7f9a1c1013c0 2024-04-02T14:03:23.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.440+0000 7f9a2437b640 1 -- 172.21.15.52:0/83273045 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a1c0688f0 con 0x7f9a1c1013c0 2024-04-02T14:03:23.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.441+0000 7f9a137fe640 1 -- 172.21.15.52:0/83273045 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9a0c044070 con 0x7f9a1c1013c0 2024-04-02T14:03:23.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.442+0000 7f9a137fe640 1 --2- 172.21.15.52:0/83273045 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9a080a3d20 0x7f9a080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.442+0000 7f9a22117640 1 --2- 172.21.15.52:0/83273045 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9a080a3d20 0x7f9a080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.442+0000 7f9a137fe640 1 -- 172.21.15.52:0/83273045 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f9a0c0f2750 con 0x7f9a1c1013c0 2024-04-02T14:03:23.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.442+0000 7f9a22117640 1 --2- 172.21.15.52:0/83273045 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9a080a3d20 0x7f9a080a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9a18005d00 tx=0x7f9a1800b410 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:23.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.446+0000 7f9a137fe640 1 -- 172.21.15.52:0/83273045 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9a0c0be520 con 0x7f9a1c1013c0 2024-04-02T14:03:23.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.672+0000 7f9a2437b640 1 -- 172.21.15.52:0/83273045 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a1c1997d0 con 0x7f9a1c1013c0 2024-04-02T14:03:23.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.673+0000 7f9a137fe640 1 -- 172.21.15.52:0/83273045 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11923 (secure 0 0 0) 0x7f9a0c0c2820 con 0x7f9a1c1013c0 2024-04-02T14:03:23.677 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T14:03:23.677 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:57:45.240902+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24467,24457,24490,24475,24503]},"id":1}]} 2024-04-02T14:03:23.677 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-04-02T14:03:23.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.675+0000 7f9a2437b640 1 -- 172.21.15.52:0/83273045 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9a080a3d20 msgr2=0x7f9a080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.675+0000 7f9a2437b640 1 --2- 172.21.15.52:0/83273045 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9a080a3d20 0x7f9a080a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9a18005d00 tx=0x7f9a1800b410 comp rx=0 tx=0).stop 2024-04-02T14:03:23.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.675+0000 7f9a2437b640 1 -- 172.21.15.52:0/83273045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a1c1013c0 msgr2=0x7f9a1c1989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.675+0000 7f9a2437b640 1 --2- 172.21.15.52:0/83273045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a1c1013c0 0x7f9a1c1989c0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c009230 tx=0x7f9a0c02cba0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.675+0000 7f9a2437b640 1 -- 172.21.15.52:0/83273045 shutdown_connections 2024-04-02T14:03:23.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.675+0000 7f9a2437b640 1 --2- 172.21.15.52:0/83273045 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9a080a3d20 0x7f9a080a61e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.675+0000 7f9a2437b640 1 --2- 172.21.15.52:0/83273045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a1c106a10 0x7f9a1c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.675+0000 7f9a2437b640 1 --2- 172.21.15.52:0/83273045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a1c1013c0 0x7f9a1c1989c0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.675+0000 7f9a2437b640 1 --2- 172.21.15.52:0/83273045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a1c100a00 0x7f9a1c198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.675+0000 7f9a2437b640 1 -- 172.21.15.52:0/83273045 >> 172.21.15.52:0/83273045 conn(0x7f9a1c077b00 msgr2=0x7f9a1c069000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:23.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.675+0000 7f9a2437b640 1 -- 172.21.15.52:0/83273045 shutdown_connections 2024-04-02T14:03:23.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.675+0000 7f9a2437b640 1 -- 172.21.15.52:0/83273045 wait complete. 2024-04-02T14:03:23.690 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T14:03:23.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:23 smithi189 ceph-mon[36683]: pgmap v307: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:03:23.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.790+0000 7f46cfe86640 1 -- 172.21.15.52:0/926506419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46c81009b0 msgr2=0x7f46c8100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.790+0000 7f46cfe86640 1 --2- 172.21.15.52:0/926506419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46c81009b0 0x7f46c8100e30 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009930 tx=0x7f46bc02bc30 comp rx=0 tx=0).stop 2024-04-02T14:03:23.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.790+0000 7f46cfe86640 1 -- 172.21.15.52:0/926506419 shutdown_connections 2024-04-02T14:03:23.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.790+0000 7f46cfe86640 1 --2- 172.21.15.52:0/926506419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46c8101370 0x7f46c810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.790+0000 7f46cfe86640 1 --2- 172.21.15.52:0/926506419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46c81009b0 0x7f46c8100e30 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.790+0000 7f46cfe86640 1 --2- 172.21.15.52:0/926506419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46c81069c0 0x7f46c8106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.790+0000 7f46cfe86640 1 -- 172.21.15.52:0/926506419 >> 172.21.15.52:0/926506419 conn(0x7f46c8076d80 msgr2=0x7f46c8077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:23.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.791+0000 7f46cfe86640 1 -- 172.21.15.52:0/926506419 shutdown_connections 2024-04-02T14:03:23.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.791+0000 7f46cfe86640 1 -- 172.21.15.52:0/926506419 wait complete. 2024-04-02T14:03:23.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.791+0000 7f46cfe86640 1 Processor -- start 2024-04-02T14:03:23.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.791+0000 7f46cfe86640 1 -- start start 2024-04-02T14:03:23.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.791+0000 7f46cfe86640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46c81009b0 0x7f46c81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.791+0000 7f46cfe86640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46c8101370 0x7f46c8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.791+0000 7f46cfe86640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46c81069c0 0x7f46c819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.791+0000 7f46cfe86640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f46c8110dd0 con 0x7f46c8101370 2024-04-02T14:03:23.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.791+0000 7f46cfe86640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f46c8110c50 con 0x7f46c81009b0 2024-04-02T14:03:23.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.791+0000 7f46cfe86640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f46c8110f50 con 0x7f46c81069c0 2024-04-02T14:03:23.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.791+0000 7f46cd421640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46c8101370 0x7f46c8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.791+0000 7f46cd421640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46c8101370 0x7f46c8198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57046/0 (socket says 172.21.15.52:57046) 2024-04-02T14:03:23.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.791+0000 7f46cd421640 1 -- 172.21.15.52:0/1049381068 learned_addr learned my addr 172.21.15.52:0/1049381068 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:23.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.792+0000 7f46cdc22640 1 --2- 172.21.15.52:0/1049381068 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46c81009b0 0x7f46c81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.792+0000 7f46ce423640 1 --2- 172.21.15.52:0/1049381068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46c81069c0 0x7f46c819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.792+0000 7f46cd421640 1 -- 172.21.15.52:0/1049381068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46c81069c0 msgr2=0x7f46c819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.792+0000 7f46cd421640 1 --2- 172.21.15.52:0/1049381068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46c81069c0 0x7f46c819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.792+0000 7f46cd421640 1 -- 172.21.15.52:0/1049381068 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46c81009b0 msgr2=0x7f46c81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.792+0000 7f46cd421640 1 --2- 172.21.15.52:0/1049381068 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46c81009b0 0x7f46c81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.792+0000 7f46cd421640 1 -- 172.21.15.52:0/1049381068 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c819d3d0 con 0x7f46c8101370 2024-04-02T14:03:23.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.792+0000 7f46cd421640 1 --2- 172.21.15.52:0/1049381068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46c8101370 0x7f46c8198920 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009230 tx=0x7f46bc02cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:23.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.792+0000 7f46c6ffd640 1 -- 172.21.15.52:0/1049381068 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc041070 con 0x7f46c8101370 2024-04-02T14:03:23.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.792+0000 7f46c6ffd640 1 -- 172.21.15.52:0/1049381068 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46bc004450 con 0x7f46c8101370 2024-04-02T14:03:23.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.792+0000 7f46cfe86640 1 -- 172.21.15.52:0/1049381068 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c819d660 con 0x7f46c8101370 2024-04-02T14:03:23.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.792+0000 7f46c6ffd640 1 -- 172.21.15.52:0/1049381068 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc034440 con 0x7f46c8101370 2024-04-02T14:03:23.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.792+0000 7f46cfe86640 1 -- 172.21.15.52:0/1049381068 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c81a4f50 con 0x7f46c8101370 2024-04-02T14:03:23.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.794+0000 7f46c6ffd640 1 -- 172.21.15.52:0/1049381068 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f46bc044070 con 0x7f46c8101370 2024-04-02T14:03:23.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.795+0000 7f46d2a6e740 1 Processor -- start 2024-04-02T14:03:23.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.795+0000 7f46d2a6e740 1 -- start start 2024-04-02T14:03:23.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.795+0000 7f46c6ffd640 1 --2- 172.21.15.52:0/1049381068 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f46b00a3da0 0x7f46b00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.795+0000 7f46d2a6e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4f9647a0 0x559a4f964b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.795+0000 7f46d2a6e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a4f9650c0 0x559a4f96d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.795+0000 7f46d2a6e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f96dca0 0x559a4f970090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.795+0000 7f46ce423640 1 --2- 172.21.15.52:0/1049381068 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f46b00a3da0 0x7f46b00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.795+0000 7f46d2a6e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559a4f8c87b0 con 0x559a4f96dca0 2024-04-02T14:03:23.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.795+0000 7f46d2a6e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559a4f867570 con 0x559a4f9647a0 2024-04-02T14:03:23.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.795+0000 7f46d2a6e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559a4f8c4760 con 0x559a4f9650c0 2024-04-02T14:03:23.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.795+0000 7f46c6ffd640 1 -- 172.21.15.52:0/1049381068 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f46bc0f23f0 con 0x7f46c8101370 2024-04-02T14:03:23.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.795+0000 7f46ce423640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f96dca0 0x559a4f970090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.795+0000 7f46ce423640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f96dca0 0x559a4f970090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57058/0 (socket says 172.21.15.52:57058) 2024-04-02T14:03:23.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.795+0000 7f46ce423640 1 -- 172.21.15.52:0/172740240 learned_addr learned my addr 172.21.15.52:0/172740240 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:23.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.796+0000 7f46cdc22640 1 --2- 172.21.15.52:0/172740240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4f9647a0 0x559a4f964b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.796+0000 7f46cd421640 1 --2- 172.21.15.52:0/172740240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a4f9650c0 0x559a4f96d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.796+0000 7f46ce423640 1 --2- 172.21.15.52:0/1049381068 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f46b00a3da0 0x7f46b00a6260 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009fd0 tx=0x7f46b8004410 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:23.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.796+0000 7f46ce423640 1 -- 172.21.15.52:0/172740240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a4f9650c0 msgr2=0x559a4f96d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:23.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.796+0000 7f46ce423640 1 --2- 172.21.15.52:0/172740240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a4f9650c0 0x559a4f96d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.796+0000 7f46ce423640 1 -- 172.21.15.52:0/172740240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4f9647a0 msgr2=0x559a4f964b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:23.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.796+0000 7f46ce423640 1 --2- 172.21.15.52:0/172740240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4f9647a0 0x559a4f964b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.796+0000 7f46ce423640 1 -- 172.21.15.52:0/172740240 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a4f5dc3f0 con 0x559a4f96dca0 2024-04-02T14:03:23.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.796+0000 7f46cdc22640 1 --2- 172.21.15.52:0/172740240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4f9647a0 0x559a4f964b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:03:23.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.796+0000 7f46ce423640 1 --2- 172.21.15.52:0/172740240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f96dca0 0x559a4f970090 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f46b8004910 tx=0x7f46b801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5bf3481c1262923 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:23.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.796+0000 7f46affff640 1 -- 172.21.15.52:0/172740240 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b801e9b0 con 0x559a4f96dca0 2024-04-02T14:03:23.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.796+0000 7f46affff640 1 -- 172.21.15.52:0/172740240 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46b801eb50 con 0x559a4f96dca0 2024-04-02T14:03:23.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.796+0000 7f46affff640 1 -- 172.21.15.52:0/172740240 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b800f620 con 0x559a4f96dca0 2024-04-02T14:03:23.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.797+0000 7f46d2a6e740 1 -- 172.21.15.52:0/172740240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f96dca0 msgr2=0x559a4f970090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.797+0000 7f46d2a6e740 1 --2- 172.21.15.52:0/172740240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f96dca0 0x559a4f970090 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f46b8004910 tx=0x7f46b801e000 comp rx=0 tx=0).stop 2024-04-02T14:03:23.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.797+0000 7f46d2a6e740 1 -- 172.21.15.52:0/172740240 shutdown_connections 2024-04-02T14:03:23.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.797+0000 7f46d2a6e740 1 --2- 172.21.15.52:0/172740240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f96dca0 0x559a4f970090 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.797+0000 7f46d2a6e740 1 --2- 172.21.15.52:0/172740240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a4f9650c0 0x559a4f96d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.797+0000 7f46d2a6e740 1 --2- 172.21.15.52:0/172740240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4f9647a0 0x559a4f964b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.797+0000 7f46d2a6e740 1 -- 172.21.15.52:0/172740240 >> 172.21.15.52:0/172740240 conn(0x559a4f95eed0 msgr2=0x559a4f95f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:23.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.797+0000 7f46d2a6e740 1 -- 172.21.15.52:0/172740240 shutdown_connections 2024-04-02T14:03:23.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.797+0000 7f46d2a6e740 1 -- 172.21.15.52:0/172740240 wait complete. 2024-04-02T14:03:23.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.797+0000 7f46d2a6e740 1 -- start start 2024-04-02T14:03:23.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.797+0000 7f46d2a6e740 1 Processor -- start 2024-04-02T14:03:23.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.798+0000 7f46d2a6e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a4f9647a0 0x559a4f96d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.798+0000 7f46d2a6e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4f9650c0 0x559a4f96aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.798+0000 7f46d2a6e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f96dca0 0x559a4f96b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.798+0000 7f46d2a6e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559a4f970a30 con 0x559a4f96dca0 2024-04-02T14:03:23.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.798+0000 7f46d2a6e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559a4f883e80 con 0x559a4f9650c0 2024-04-02T14:03:23.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.798+0000 7f46d2a6e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559a4f970bb0 con 0x559a4f9647a0 2024-04-02T14:03:23.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.798+0000 7f46d2a6e740 10 client.0 _ll_register_callbacks cb 0x559a4f8bfaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:23.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.798+0000 7f46ce423640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f96dca0 0x559a4f96b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.798+0000 7f46ce423640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f96dca0 0x559a4f96b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57068/0 (socket says 172.21.15.52:57068) 2024-04-02T14:03:23.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.798+0000 7f46ce423640 1 -- 172.21.15.52:0/2681248459 learned_addr learned my addr 172.21.15.52:0/2681248459 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:23.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.798+0000 7f46cdc22640 1 --2- 172.21.15.52:0/2681248459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a4f9647a0 0x559a4f96d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.798+0000 7f46cd421640 1 --2- 172.21.15.52:0/2681248459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4f9650c0 0x559a4f96aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.798+0000 7f46ce423640 1 -- 172.21.15.52:0/2681248459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a4f9647a0 msgr2=0x559a4f96d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:23.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.798+0000 7f46ce423640 1 --2- 172.21.15.52:0/2681248459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a4f9647a0 0x559a4f96d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.798+0000 7f46ce423640 1 -- 172.21.15.52:0/2681248459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4f9650c0 msgr2=0x559a4f96aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:23.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.798+0000 7f46ce423640 1 --2- 172.21.15.52:0/2681248459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4f9650c0 0x559a4f96aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.798+0000 7f46ce423640 1 -- 172.21.15.52:0/2681248459 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a4f6a67d0 con 0x559a4f96dca0 2024-04-02T14:03:23.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.798+0000 7f46cdc22640 1 --2- 172.21.15.52:0/2681248459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a4f9647a0 0x559a4f96d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:03:23.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.798+0000 7f46cd421640 1 --2- 172.21.15.52:0/2681248459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4f9650c0 0x559a4f96aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:03:23.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.798+0000 7f46ce423640 1 --2- 172.21.15.52:0/2681248459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f96dca0 0x559a4f96b1a0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f46b8003af0 tx=0x7f46b801ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717f5a407a00d4cf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:23.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.798+0000 7f46ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:23.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.798+0000 7f46ad7fa640 1 -- 172.21.15.52:0/2681248459 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b801f650 con 0x559a4f96dca0 2024-04-02T14:03:23.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.799+0000 7f46ad7fa640 1 -- 172.21.15.52:0/2681248459 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f46b801f7f0 con 0x559a4f96dca0 2024-04-02T14:03:23.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.799+0000 7f46d2a6e740 1 -- 172.21.15.52:0/2681248459 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46b801eb50 con 0x559a4f96dca0 2024-04-02T14:03:23.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.799+0000 7f46ad7fa640 1 -- 172.21.15.52:0/2681248459 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f46b803b050 con 0x559a4f96dca0 2024-04-02T14:03:23.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.799+0000 7f46d2a6e740 10 client.18378 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:23.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.799+0000 7f46d2a6e740 1 -- 172.21.15.52:0/2681248459 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559a4f97ca50 con 0x559a4f96dca0 2024-04-02T14:03:23.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.799+0000 7f46ad7fa640 1 -- 172.21.15.52:0/2681248459 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46b801eb50 con 0x559a4f96dca0 2024-04-02T14:03:23.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.799+0000 7f46ad7fa640 1 -- 172.21.15.52:0/2681248459 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f46b8029c60 con 0x559a4f96dca0 2024-04-02T14:03:23.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.799+0000 7f46d2a6e740 10 client.18378 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:23.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.799+0000 7f46d2a6e740 10 client.18378 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T14:03:23.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.799+0000 7f46d2a6e740 20 client.18378 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:23.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.799+0000 7f46d2a6e740 1 --2- 172.21.15.52:0/2681248459 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559a4f97de60 0x559a4f980250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.799+0000 7f46d2a6e740 4 client.18378 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:03:23.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.799+0000 7f46d2a6e740 1 -- 172.21.15.52:0/2681248459 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a4f8d5cf0 con 0x559a4f97de60 2024-04-02T14:03:23.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.799+0000 7f46cdc22640 1 --2- 172.21.15.52:0/2681248459 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559a4f97de60 0x559a4f980250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.800+0000 7f46cdc22640 1 --2- 172.21.15.52:0/2681248459 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559a4f97de60 0x559a4f980250 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a87262e039abeeb9 server_cookie=7d8607165e605901 in_seq=0 out_seq=0 2024-04-02T14:03:23.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.800+0000 7f46ad7fa640 10 client.18378 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T14:03:23.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.800+0000 7f46ad7fa640 1 -- 172.21.15.52:0/2681248459 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a4f8d5cf0 con 0x559a4f97de60 2024-04-02T14:03:23.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.800+0000 7f46ad7fa640 10 client.18378 handle_command_reply: tid=0 2024-04-02T14:03:23.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.801+0000 7f46d2a6e740 1 client.18378 shutdown 2024-04-02T14:03:23.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.801+0000 7f46d2a6e740 1 -- 172.21.15.52:0/2681248459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f96dca0 msgr2=0x559a4f96b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.801+0000 7f46d2a6e740 1 --2- 172.21.15.52:0/2681248459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f96dca0 0x559a4f96b1a0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f46b8003af0 tx=0x7f46b801ef90 comp rx=0 tx=0).stop 2024-04-02T14:03:23.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.801+0000 7f46d2a6e740 1 -- 172.21.15.52:0/2681248459 shutdown_connections 2024-04-02T14:03:23.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.801+0000 7f46d2a6e740 1 --2- 172.21.15.52:0/2681248459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f96dca0 0x559a4f96b1a0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.801+0000 7f46d2a6e740 1 --2- 172.21.15.52:0/2681248459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4f9650c0 0x559a4f96aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.801+0000 7f46d2a6e740 1 --2- 172.21.15.52:0/2681248459 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559a4f97de60 0x559a4f980250 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.801+0000 7f46d2a6e740 1 --2- 172.21.15.52:0/2681248459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a4f9647a0 0x559a4f96d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.801+0000 7f46d2a6e740 1 -- 172.21.15.52:0/2681248459 >> 172.21.15.52:0/2681248459 conn(0x559a4f95eed0 msgr2=0x559a4f960c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:23.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.801+0000 7f46ad7fa640 0 client.18378 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T14:03:23.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.801+0000 7f46d2a6e740 1 -- 172.21.15.52:0/2681248459 shutdown_connections 2024-04-02T14:03:23.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.801+0000 7f46d2a6e740 1 -- 172.21.15.52:0/2681248459 wait complete. 2024-04-02T14:03:23.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.801+0000 7f46d2a6e740 20 client.18378 trim_cache size 0 max 16384 2024-04-02T14:03:23.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.814+0000 7f46d2a6e740 1 Processor -- start 2024-04-02T14:03:23.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.814+0000 7f46d2a6e740 1 -- start start 2024-04-02T14:03:23.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.814+0000 7f46d2a6e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f9647a0 0x559a4f9798f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.814+0000 7f46d2a6e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4f9650c0 0x559a4f985940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.814+0000 7f46d2a6e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a4f96dca0 0x559a4f985e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.814+0000 7f46d2a6e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f46c81aef20 con 0x559a4f9647a0 2024-04-02T14:03:23.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.814+0000 7f46d2a6e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f46c81b0a60 con 0x559a4f9650c0 2024-04-02T14:03:23.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.814+0000 7f46d2a6e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f46c81af5f0 con 0x559a4f96dca0 2024-04-02T14:03:23.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.814+0000 7f46cdc22640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f9647a0 0x559a4f9798f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.814+0000 7f46cdc22640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f9647a0 0x559a4f9798f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57072/0 (socket says 172.21.15.52:57072) 2024-04-02T14:03:23.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.814+0000 7f46cdc22640 1 -- 172.21.15.52:0/1364184935 learned_addr learned my addr 172.21.15.52:0/1364184935 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:23.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.815+0000 7f46ce423640 1 --2- 172.21.15.52:0/1364184935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a4f96dca0 0x559a4f985e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.815+0000 7f46cdc22640 1 -- 172.21.15.52:0/1364184935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a4f96dca0 msgr2=0x559a4f985e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:23.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.815+0000 7f46cdc22640 1 --2- 172.21.15.52:0/1364184935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a4f96dca0 0x559a4f985e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.815+0000 7f46cdc22640 1 -- 172.21.15.52:0/1364184935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4f9650c0 msgr2=0x559a4f985940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:23.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.815+0000 7f46cd421640 1 --2- 172.21.15.52:0/1364184935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4f9650c0 0x559a4f985940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.815+0000 7f46cdc22640 1 --2- 172.21.15.52:0/1364184935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4f9650c0 0x559a4f985940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.815+0000 7f46cdc22640 1 -- 172.21.15.52:0/1364184935 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a4f964eb0 con 0x559a4f9647a0 2024-04-02T14:03:23.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.815+0000 7f46ce423640 1 --2- 172.21.15.52:0/1364184935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a4f96dca0 0x559a4f985e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:03:23.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.815+0000 7f46cd421640 1 --2- 172.21.15.52:0/1364184935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4f9650c0 0x559a4f985940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:03:23.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.815+0000 7f46cdc22640 1 --2- 172.21.15.52:0/1364184935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f9647a0 0x559a4f9798f0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009aa0 tx=0x7f46c0004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a746667060c7a525 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:23.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.815+0000 7f46af7fe640 1 -- 172.21.15.52:0/1364184935 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0003b50 con 0x559a4f9647a0 2024-04-02T14:03:23.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.815+0000 7f46af7fe640 1 -- 172.21.15.52:0/1364184935 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46c0005500 con 0x559a4f9647a0 2024-04-02T14:03:23.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.815+0000 7f46af7fe640 1 -- 172.21.15.52:0/1364184935 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c001e050 con 0x559a4f9647a0 2024-04-02T14:03:23.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.816+0000 7f46d2a6e740 1 -- 172.21.15.52:0/1364184935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f9647a0 msgr2=0x559a4f9798f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.816+0000 7f46d2a6e740 1 --2- 172.21.15.52:0/1364184935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f9647a0 0x559a4f9798f0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009aa0 tx=0x7f46c0004410 comp rx=0 tx=0).stop 2024-04-02T14:03:23.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.816+0000 7f46d2a6e740 1 -- 172.21.15.52:0/1364184935 shutdown_connections 2024-04-02T14:03:23.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.816+0000 7f46d2a6e740 1 --2- 172.21.15.52:0/1364184935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a4f96dca0 0x559a4f985e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.816+0000 7f46d2a6e740 1 --2- 172.21.15.52:0/1364184935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4f9650c0 0x559a4f985940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.816+0000 7f46d2a6e740 1 --2- 172.21.15.52:0/1364184935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f9647a0 0x559a4f9798f0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.816+0000 7f46d2a6e740 1 -- 172.21.15.52:0/1364184935 >> 172.21.15.52:0/1364184935 conn(0x559a4f95eed0 msgr2=0x559a4f965d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:23.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.816+0000 7f46d2a6e740 1 -- 172.21.15.52:0/1364184935 shutdown_connections 2024-04-02T14:03:23.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.816+0000 7f46d2a6e740 1 -- 172.21.15.52:0/1364184935 wait complete. 2024-04-02T14:03:23.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.816+0000 7f46d2a6e740 1 -- start start 2024-04-02T14:03:23.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.816+0000 7f46d2a6e740 1 Processor -- start 2024-04-02T14:03:23.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.816+0000 7f46d2a6e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a4f9647a0 0x559a4f96e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.816+0000 7f46d2a6e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4f9650c0 0x559a4f96e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.817+0000 7f46d2a6e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f96dca0 0x559a4f96eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.817+0000 7f46d2a6e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f46c81b0390 con 0x559a4f96dca0 2024-04-02T14:03:23.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.817+0000 7f46d2a6e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f46c81afcc0 con 0x559a4f9650c0 2024-04-02T14:03:23.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.817+0000 7f46d2a6e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f46c81acf20 con 0x559a4f9647a0 2024-04-02T14:03:23.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.817+0000 7f46d2a6e740 10 client.0 _ll_register_callbacks cb 0x7f46c00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:23.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.817+0000 7f46ce423640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f96dca0 0x559a4f96eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.817+0000 7f46ce423640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f96dca0 0x559a4f96eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57078/0 (socket says 172.21.15.52:57078) 2024-04-02T14:03:23.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.817+0000 7f46ce423640 1 -- 172.21.15.52:0/21916423 learned_addr learned my addr 172.21.15.52:0/21916423 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:23.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.817+0000 7f46cd421640 1 --2- 172.21.15.52:0/21916423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4f9650c0 0x559a4f96e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.817+0000 7f46cdc22640 1 --2- 172.21.15.52:0/21916423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a4f9647a0 0x559a4f96e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.817+0000 7f46ce423640 1 -- 172.21.15.52:0/21916423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a4f9647a0 msgr2=0x559a4f96e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:23.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.817+0000 7f46ce423640 1 --2- 172.21.15.52:0/21916423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a4f9647a0 0x559a4f96e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.817+0000 7f46ce423640 1 -- 172.21.15.52:0/21916423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4f9650c0 msgr2=0x559a4f96e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:23.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.817+0000 7f46ce423640 1 --2- 172.21.15.52:0/21916423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4f9650c0 0x559a4f96e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.817+0000 7f46ce423640 1 -- 172.21.15.52:0/21916423 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a4f974eb0 con 0x559a4f96dca0 2024-04-02T14:03:23.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.817+0000 7f46ce423640 1 --2- 172.21.15.52:0/21916423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f96dca0 0x559a4f96eb00 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009700 tx=0x7f46b8003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238390d21723fa6e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:23.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.817+0000 7f46ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:23.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.817+0000 7f46ad7fa640 1 -- 172.21.15.52:0/21916423 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b8009d00 con 0x559a4f96dca0 2024-04-02T14:03:23.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.817+0000 7f46ad7fa640 1 -- 172.21.15.52:0/21916423 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f46b80350d0 con 0x559a4f96dca0 2024-04-02T14:03:23.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.817+0000 7f46d2a6e740 1 -- 172.21.15.52:0/21916423 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46c0005500 con 0x559a4f96dca0 2024-04-02T14:03:23.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.817+0000 7f46ad7fa640 1 -- 172.21.15.52:0/21916423 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f46b8048050 con 0x559a4f96dca0 2024-04-02T14:03:23.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.817+0000 7f46d2a6e740 10 client.18384 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:23.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.817+0000 7f46d2a6e740 1 -- 172.21.15.52:0/21916423 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559a4f9760e0 con 0x559a4f96dca0 2024-04-02T14:03:23.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.818+0000 7f46ad7fa640 1 -- 172.21.15.52:0/21916423 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46c0005500 con 0x559a4f96dca0 2024-04-02T14:03:23.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.818+0000 7f46ad7fa640 1 -- 172.21.15.52:0/21916423 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f46b8004bc0 con 0x559a4f96dca0 2024-04-02T14:03:23.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.818+0000 7f46d2a6e740 10 client.18384 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:23.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.818+0000 7f46d2a6e740 10 client.18384 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T14:03:23.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.818+0000 7f46d2a6e740 20 client.18384 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:23.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.818+0000 7f46d2a6e740 1 --2- 172.21.15.52:0/21916423 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559a4f9e41b0 0x559a4f9e65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.818+0000 7f46d2a6e740 4 client.18384 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:03:23.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.818+0000 7f46d2a6e740 1 -- 172.21.15.52:0/21916423 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a4f9705e0 con 0x559a4f9e41b0 2024-04-02T14:03:23.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.818+0000 7f46cdc22640 1 --2- 172.21.15.52:0/21916423 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559a4f9e41b0 0x559a4f9e65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.819+0000 7f46cdc22640 1 --2- 172.21.15.52:0/21916423 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559a4f9e41b0 0x559a4f9e65a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1d972e3eea5dfb37 server_cookie=451874a0fcc16703 in_seq=0 out_seq=0 2024-04-02T14:03:23.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.819+0000 7f46ad7fa640 10 client.18384 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T14:03:23.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.819+0000 7f46ad7fa640 1 -- 172.21.15.52:0/21916423 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a4f9705e0 con 0x559a4f9e41b0 2024-04-02T14:03:23.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.819+0000 7f46ad7fa640 10 client.18384 handle_command_reply: tid=0 2024-04-02T14:03:23.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.819+0000 7f46d2a6e740 1 client.18384 shutdown 2024-04-02T14:03:23.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.819+0000 7f46d2a6e740 1 -- 172.21.15.52:0/21916423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f96dca0 msgr2=0x559a4f96eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.819+0000 7f46d2a6e740 1 --2- 172.21.15.52:0/21916423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f96dca0 0x559a4f96eb00 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009700 tx=0x7f46b8003dc0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.819+0000 7f46d2a6e740 1 -- 172.21.15.52:0/21916423 shutdown_connections 2024-04-02T14:03:23.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.819+0000 7f46d2a6e740 1 --2- 172.21.15.52:0/21916423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a4f96dca0 0x559a4f96eb00 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.819+0000 7f46d2a6e740 1 --2- 172.21.15.52:0/21916423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4f9650c0 0x559a4f96e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.819+0000 7f46d2a6e740 1 --2- 172.21.15.52:0/21916423 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x559a4f9e41b0 0x559a4f9e65a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.819+0000 7f46d2a6e740 1 --2- 172.21.15.52:0/21916423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a4f9647a0 0x559a4f96e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.819+0000 7f46d2a6e740 1 -- 172.21.15.52:0/21916423 >> 172.21.15.52:0/21916423 conn(0x559a4f95eed0 msgr2=0x559a4f95d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:23.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.819+0000 7f46ad7fa640 0 client.18384 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T14:03:23.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.819+0000 7f46d2a6e740 1 -- 172.21.15.52:0/21916423 shutdown_connections 2024-04-02T14:03:23.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.819+0000 7f46d2a6e740 1 -- 172.21.15.52:0/21916423 wait complete. 2024-04-02T14:03:23.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.819+0000 7f46d2a6e740 20 client.18384 trim_cache size 0 max 16384 2024-04-02T14:03:23.834 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:03:23.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.820+0000 7f46cfe86640 1 -- 172.21.15.52:0/1049381068 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f46b00a3da0 msgr2=0x7f46b00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.820+0000 7f46cfe86640 1 --2- 172.21.15.52:0/1049381068 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f46b00a3da0 0x7f46b00a6260 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009fd0 tx=0x7f46b8004410 comp rx=0 tx=0).stop 2024-04-02T14:03:23.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.820+0000 7f46cfe86640 1 -- 172.21.15.52:0/1049381068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46c8101370 msgr2=0x7f46c8198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.820+0000 7f46cfe86640 1 --2- 172.21.15.52:0/1049381068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46c8101370 0x7f46c8198920 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009230 tx=0x7f46bc02cba0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.820+0000 7f46cfe86640 1 -- 172.21.15.52:0/1049381068 shutdown_connections 2024-04-02T14:03:23.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.820+0000 7f46cfe86640 1 --2- 172.21.15.52:0/1049381068 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f46b00a3da0 0x7f46b00a6260 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.820+0000 7f46cfe86640 1 --2- 172.21.15.52:0/1049381068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46c81069c0 0x7f46c819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.820+0000 7f46cfe86640 1 --2- 172.21.15.52:0/1049381068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46c8101370 0x7f46c8198920 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.820+0000 7f46cfe86640 1 --2- 172.21.15.52:0/1049381068 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46c81009b0 0x7f46c81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.820+0000 7f46cfe86640 1 -- 172.21.15.52:0/1049381068 >> 172.21.15.52:0/1049381068 conn(0x7f46c8076d80 msgr2=0x7f46c8104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:23.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.820+0000 7f46cfe86640 1 -- 172.21.15.52:0/1049381068 shutdown_connections 2024-04-02T14:03:23.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.820+0000 7f46cfe86640 1 -- 172.21.15.52:0/1049381068 wait complete. 2024-04-02T14:03:23.837 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:23 smithi052 ceph-mon[32104]: pgmap v307: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:03:23.837 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T14:03:23.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:23 smithi088 ceph-mon[36528]: pgmap v307: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:03:23.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.938+0000 7fc692b1c640 1 -- 172.21.15.52:0/3659848243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc68c1069a0 msgr2=0x7fc68c106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.938+0000 7fc692b1c640 1 --2- 172.21.15.52:0/3659848243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc68c1069a0 0x7fc68c106d80 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fc680009930 tx=0x7fc68002bc30 comp rx=0 tx=0).stop 2024-04-02T14:03:23.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.938+0000 7fc692b1c640 1 -- 172.21.15.52:0/3659848243 shutdown_connections 2024-04-02T14:03:23.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.938+0000 7fc692b1c640 1 --2- 172.21.15.52:0/3659848243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc68c101350 0x7fc68c10d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.938+0000 7fc692b1c640 1 --2- 172.21.15.52:0/3659848243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc68c100990 0x7fc68c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.938+0000 7fc692b1c640 1 --2- 172.21.15.52:0/3659848243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc68c1069a0 0x7fc68c106d80 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.938+0000 7fc692b1c640 1 -- 172.21.15.52:0/3659848243 >> 172.21.15.52:0/3659848243 conn(0x7fc68c077ad0 msgr2=0x7fc68c077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:23.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.938+0000 7fc692b1c640 1 -- 172.21.15.52:0/3659848243 shutdown_connections 2024-04-02T14:03:23.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.938+0000 7fc692b1c640 1 -- 172.21.15.52:0/3659848243 wait complete. 2024-04-02T14:03:23.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.939+0000 7fc692b1c640 1 Processor -- start 2024-04-02T14:03:23.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.939+0000 7fc692b1c640 1 -- start start 2024-04-02T14:03:23.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.939+0000 7fc692b1c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc68c100990 0x7fc68c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.939+0000 7fc692b1c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc68c101350 0x7fc68c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.939+0000 7fc692b1c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc68c1069a0 0x7fc68c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.939+0000 7fc692b1c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c111e70 con 0x7fc68c101350 2024-04-02T14:03:23.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.939+0000 7fc692b1c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c111cf0 con 0x7fc68c1069a0 2024-04-02T14:03:23.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.939+0000 7fc692b1c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c111ff0 con 0x7fc68c100990 2024-04-02T14:03:23.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.939+0000 7fc68bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc68c101350 0x7fc68c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.939+0000 7fc68bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc68c101350 0x7fc68c198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57096/0 (socket says 172.21.15.52:57096) 2024-04-02T14:03:23.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.939+0000 7fc68bfff640 1 -- 172.21.15.52:0/3275519705 learned_addr learned my addr 172.21.15.52:0/3275519705 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:23.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.939+0000 7fc6908b8640 1 --2- 172.21.15.52:0/3275519705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc68c100990 0x7fc68c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.939+0000 7fc6910b9640 1 --2- 172.21.15.52:0/3275519705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc68c1069a0 0x7fc68c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.939+0000 7fc68bfff640 1 -- 172.21.15.52:0/3275519705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc68c100990 msgr2=0x7fc68c198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.939+0000 7fc68bfff640 1 --2- 172.21.15.52:0/3275519705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc68c100990 0x7fc68c198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.939+0000 7fc68bfff640 1 -- 172.21.15.52:0/3275519705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc68c1069a0 msgr2=0x7fc68c19cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.939+0000 7fc68bfff640 1 --2- 172.21.15.52:0/3275519705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc68c1069a0 0x7fc68c19cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.939+0000 7fc68bfff640 1 -- 172.21.15.52:0/3275519705 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68c19d2f0 con 0x7fc68c101350 2024-04-02T14:03:23.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.939+0000 7fc6908b8640 1 --2- 172.21.15.52:0/3275519705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc68c100990 0x7fc68c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:03:23.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.939+0000 7fc6910b9640 1 --2- 172.21.15.52:0/3275519705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc68c1069a0 0x7fc68c19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:03:23.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.939+0000 7fc68bfff640 1 --2- 172.21.15.52:0/3275519705 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc68c101350 0x7fc68c198890 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fc67800c920 tx=0x7fc67800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:23.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.940+0000 7fc689ffb640 1 -- 172.21.15.52:0/3275519705 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc678007c70 con 0x7fc68c101350 2024-04-02T14:03:23.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.940+0000 7fc692b1c640 1 -- 172.21.15.52:0/3275519705 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc68c19d5e0 con 0x7fc68c101350 2024-04-02T14:03:23.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.940+0000 7fc689ffb640 1 -- 172.21.15.52:0/3275519705 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc678007e10 con 0x7fc68c101350 2024-04-02T14:03:23.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.940+0000 7fc689ffb640 1 -- 172.21.15.52:0/3275519705 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc678018890 con 0x7fc68c101350 2024-04-02T14:03:23.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.940+0000 7fc692b1c640 1 -- 172.21.15.52:0/3275519705 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc68c1a4e70 con 0x7fc68c101350 2024-04-02T14:03:23.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.942+0000 7fc689ffb640 1 -- 172.21.15.52:0/3275519705 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc678018a70 con 0x7fc68c101350 2024-04-02T14:03:23.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.942+0000 7fc689ffb640 1 --2- 172.21.15.52:0/3275519705 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc67c0a3d20 0x7fc67c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.942+0000 7fc6957e5740 1 Processor -- start 2024-04-02T14:03:23.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.942+0000 7fc6957e5740 1 -- start start 2024-04-02T14:03:23.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.942+0000 7fc6908b8640 1 --2- 172.21.15.52:0/3275519705 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc67c0a3d20 0x7fc67c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.942+0000 7fc689ffb640 1 -- 172.21.15.52:0/3275519705 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fc6780d0dc0 con 0x7fc68c101350 2024-04-02T14:03:23.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.942+0000 7fc6957e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d7d4864d0 0x564d7d4868b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.943+0000 7fc6957e5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d7d486df0 0x564d7d493b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.943+0000 7fc6957e5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d4940b0 0x564d7d4964a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.943+0000 7fc6957e5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564d7d3ea7b0 con 0x564d7d4940b0 2024-04-02T14:03:23.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.943+0000 7fc6957e5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564d7d389570 con 0x564d7d486df0 2024-04-02T14:03:23.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.943+0000 7fc6957e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564d7d3e6760 con 0x564d7d4864d0 2024-04-02T14:03:23.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.943+0000 7fc6910b9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d4940b0 0x564d7d4964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.943+0000 7fc6910b9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d4940b0 0x564d7d4964a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57104/0 (socket says 172.21.15.52:57104) 2024-04-02T14:03:23.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.943+0000 7fc6910b9640 1 -- 172.21.15.52:0/1525316055 learned_addr learned my addr 172.21.15.52:0/1525316055 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:23.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.943+0000 7fc6908b8640 1 --2- 172.21.15.52:0/1525316055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d7d4864d0 0x564d7d4868b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.943+0000 7fc68bfff640 1 --2- 172.21.15.52:0/1525316055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d7d486df0 0x564d7d493b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.943+0000 7fc6908b8640 1 --2- 172.21.15.52:0/3275519705 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc67c0a3d20 0x7fc67c0a61e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc680006010 tx=0x7fc68002c880 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:23.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.943+0000 7fc6910b9640 1 -- 172.21.15.52:0/1525316055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d7d4864d0 msgr2=0x564d7d4868b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:23.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.943+0000 7fc6910b9640 1 --2- 172.21.15.52:0/1525316055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d7d4864d0 0x564d7d4868b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.943+0000 7fc6910b9640 1 -- 172.21.15.52:0/1525316055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d7d486df0 msgr2=0x564d7d493b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:23.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.943+0000 7fc6910b9640 1 --2- 172.21.15.52:0/1525316055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d7d486df0 0x564d7d493b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.943+0000 7fc6910b9640 1 -- 172.21.15.52:0/1525316055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d7d0fe3f0 con 0x564d7d4940b0 2024-04-02T14:03:23.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.943+0000 7fc6908b8640 1 --2- 172.21.15.52:0/1525316055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d7d4864d0 0x564d7d4868b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:03:23.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.943+0000 7fc6910b9640 1 --2- 172.21.15.52:0/1525316055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d4940b0 0x564d7d4964a0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fc684009e40 tx=0x7fc68400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd0432ca49488d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:23.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.943+0000 7fc676ffd640 1 -- 172.21.15.52:0/1525316055 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684004d20 con 0x564d7d4940b0 2024-04-02T14:03:23.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.943+0000 7fc676ffd640 1 -- 172.21.15.52:0/1525316055 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc684005500 con 0x564d7d4940b0 2024-04-02T14:03:23.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.943+0000 7fc676ffd640 1 -- 172.21.15.52:0/1525316055 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684011860 con 0x564d7d4940b0 2024-04-02T14:03:23.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.944+0000 7fc6957e5740 1 -- 172.21.15.52:0/1525316055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d4940b0 msgr2=0x564d7d4964a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.944+0000 7fc6957e5740 1 --2- 172.21.15.52:0/1525316055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d4940b0 0x564d7d4964a0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fc684009e40 tx=0x7fc68400f730 comp rx=0 tx=0).stop 2024-04-02T14:03:23.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.944+0000 7fc6957e5740 1 -- 172.21.15.52:0/1525316055 shutdown_connections 2024-04-02T14:03:23.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.944+0000 7fc6957e5740 1 --2- 172.21.15.52:0/1525316055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d4940b0 0x564d7d4964a0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.944+0000 7fc6957e5740 1 --2- 172.21.15.52:0/1525316055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d7d486df0 0x564d7d493b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.944+0000 7fc6957e5740 1 --2- 172.21.15.52:0/1525316055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d7d4864d0 0x564d7d4868b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.944+0000 7fc6957e5740 1 -- 172.21.15.52:0/1525316055 >> 172.21.15.52:0/1525316055 conn(0x564d7d480ed0 msgr2=0x564d7d4812b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:23.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.944+0000 7fc6957e5740 1 -- 172.21.15.52:0/1525316055 shutdown_connections 2024-04-02T14:03:23.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.944+0000 7fc6957e5740 1 -- 172.21.15.52:0/1525316055 wait complete. 2024-04-02T14:03:23.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.944+0000 7fc6957e5740 1 -- start start 2024-04-02T14:03:23.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.945+0000 7fc6957e5740 1 Processor -- start 2024-04-02T14:03:23.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.945+0000 7fc6957e5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d4864d0 0x564d7d490f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.945+0000 7fc6957e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d7d486df0 0x564d7d4914a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.945+0000 7fc6957e5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d7d4940b0 0x564d7d4919e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.945+0000 7fc6957e5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564d7d496e70 con 0x564d7d4864d0 2024-04-02T14:03:23.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.945+0000 7fc6957e5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564d7d3a5e80 con 0x564d7d4940b0 2024-04-02T14:03:23.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.945+0000 7fc6957e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564d7d496ff0 con 0x564d7d486df0 2024-04-02T14:03:23.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.945+0000 7fc6957e5740 10 client.0 _ll_register_callbacks cb 0x564d7d3e1aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:23.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.945+0000 7fc6908b8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d4864d0 0x564d7d490f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.945+0000 7fc6908b8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d4864d0 0x564d7d490f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57120/0 (socket says 172.21.15.52:57120) 2024-04-02T14:03:23.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.945+0000 7fc6908b8640 1 -- 172.21.15.52:0/3551723404 learned_addr learned my addr 172.21.15.52:0/3551723404 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:23.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.945+0000 7fc6910b9640 1 --2- 172.21.15.52:0/3551723404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d7d4940b0 0x564d7d4919e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.945+0000 7fc68bfff640 1 --2- 172.21.15.52:0/3551723404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d7d486df0 0x564d7d4914a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.945+0000 7fc6908b8640 1 -- 172.21.15.52:0/3551723404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d7d486df0 msgr2=0x564d7d4914a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:23.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.945+0000 7fc6908b8640 1 --2- 172.21.15.52:0/3551723404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d7d486df0 0x564d7d4914a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.945+0000 7fc6908b8640 1 -- 172.21.15.52:0/3551723404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d7d4940b0 msgr2=0x564d7d4919e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:23.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.945+0000 7fc6908b8640 1 --2- 172.21.15.52:0/3551723404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d7d4940b0 0x564d7d4919e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.945+0000 7fc6908b8640 1 -- 172.21.15.52:0/3551723404 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d7d1c87d0 con 0x564d7d4864d0 2024-04-02T14:03:23.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.946+0000 7fc6908b8640 1 --2- 172.21.15.52:0/3551723404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d4864d0 0x564d7d490f60 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fc680035000 tx=0x7fc68002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5468f5dc1c5f66e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:23.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.946+0000 7fc66bfff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:23.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.946+0000 7fc66bfff640 1 -- 172.21.15.52:0/3551723404 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc680046070 con 0x564d7d4864d0 2024-04-02T14:03:23.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.946+0000 7fc66bfff640 1 -- 172.21.15.52:0/3551723404 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fc6800055d0 con 0x564d7d4864d0 2024-04-02T14:03:23.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.946+0000 7fc6957e5740 1 -- 172.21.15.52:0/3551723404 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc684005500 con 0x564d7d4864d0 2024-04-02T14:03:23.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.946+0000 7fc66bfff640 1 -- 172.21.15.52:0/3551723404 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc680059050 con 0x564d7d4864d0 2024-04-02T14:03:23.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.946+0000 7fc6957e5740 10 client.18396 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:23.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.946+0000 7fc6957e5740 1 -- 172.21.15.52:0/3551723404 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564d7d4a2fe0 con 0x564d7d4864d0 2024-04-02T14:03:23.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.946+0000 7fc66bfff640 1 -- 172.21.15.52:0/3551723404 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc684005500 con 0x564d7d4864d0 2024-04-02T14:03:23.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.946+0000 7fc66bfff640 1 -- 172.21.15.52:0/3551723404 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc680049070 con 0x564d7d4864d0 2024-04-02T14:03:23.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.946+0000 7fc6957e5740 10 client.18396 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:23.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.946+0000 7fc6957e5740 10 client.18396 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T14:03:23.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.946+0000 7fc6957e5740 20 client.18396 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:23.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.946+0000 7fc6957e5740 1 --2- 172.21.15.52:0/3551723404 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x564d7d4a44f0 0x564d7d4a6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.946+0000 7fc6957e5740 4 client.18396 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:03:23.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.946+0000 7fc6957e5740 1 -- 172.21.15.52:0/3551723404 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d7d3f7cf0 con 0x564d7d4a44f0 2024-04-02T14:03:23.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.947+0000 7fc68bfff640 1 --2- 172.21.15.52:0/3551723404 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x564d7d4a44f0 0x564d7d4a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.947+0000 7fc66bfff640 10 client.18396 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T14:03:23.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.947+0000 7fc68bfff640 1 --2- 172.21.15.52:0/3551723404 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x564d7d4a44f0 0x564d7d4a6920 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5b232b00d72d555f server_cookie=ad3ad89fafdefb7c in_seq=0 out_seq=0 2024-04-02T14:03:23.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.948+0000 7fc66bfff640 1 -- 172.21.15.52:0/3551723404 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d7d3f7cf0 con 0x564d7d4a44f0 2024-04-02T14:03:23.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.948+0000 7fc66bfff640 10 client.18396 handle_command_reply: tid=0 2024-04-02T14:03:23.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.948+0000 7fc6957e5740 1 client.18396 shutdown 2024-04-02T14:03:23.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.948+0000 7fc6957e5740 1 -- 172.21.15.52:0/3551723404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d4864d0 msgr2=0x564d7d490f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.948+0000 7fc6957e5740 1 --2- 172.21.15.52:0/3551723404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d4864d0 0x564d7d490f60 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fc680035000 tx=0x7fc68002bc30 comp rx=0 tx=0).stop 2024-04-02T14:03:23.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.948+0000 7fc6957e5740 1 -- 172.21.15.52:0/3551723404 shutdown_connections 2024-04-02T14:03:23.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.948+0000 7fc6957e5740 1 --2- 172.21.15.52:0/3551723404 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x564d7d4a44f0 0x564d7d4a6920 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.948+0000 7fc6957e5740 1 --2- 172.21.15.52:0/3551723404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d7d4940b0 0x564d7d4919e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.948+0000 7fc6957e5740 1 --2- 172.21.15.52:0/3551723404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d7d486df0 0x564d7d4914a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.948+0000 7fc6957e5740 1 --2- 172.21.15.52:0/3551723404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d4864d0 0x564d7d490f60 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.948+0000 7fc66bfff640 0 client.18396 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T14:03:23.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.948+0000 7fc6957e5740 1 -- 172.21.15.52:0/3551723404 >> 172.21.15.52:0/3551723404 conn(0x564d7d480ed0 msgr2=0x564d7d482c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:23.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.948+0000 7fc6957e5740 1 -- 172.21.15.52:0/3551723404 shutdown_connections 2024-04-02T14:03:23.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.948+0000 7fc6957e5740 1 -- 172.21.15.52:0/3551723404 wait complete. 2024-04-02T14:03:23.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.948+0000 7fc6957e5740 20 client.18396 trim_cache size 0 max 16384 2024-04-02T14:03:23.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.961+0000 7fc6957e5740 1 Processor -- start 2024-04-02T14:03:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.961+0000 7fc6957e5740 1 -- start start 2024-04-02T14:03:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.961+0000 7fc6957e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d7d4864d0 0x564d7d493690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.961+0000 7fc6957e5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d7d486df0 0x564d7d49fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.961+0000 7fc6957e5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d4940b0 0x564d7d4ac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.961+0000 7fc6957e5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c1acce0 con 0x564d7d4940b0 2024-04-02T14:03:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.961+0000 7fc6957e5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c1ae820 con 0x564d7d486df0 2024-04-02T14:03:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.961+0000 7fc6957e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c1ad3b0 con 0x564d7d4864d0 2024-04-02T14:03:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.962+0000 7fc6910b9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d4940b0 0x564d7d4ac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.962+0000 7fc6910b9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d4940b0 0x564d7d4ac1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57136/0 (socket says 172.21.15.52:57136) 2024-04-02T14:03:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.962+0000 7fc6910b9640 1 -- 172.21.15.52:0/1587179215 learned_addr learned my addr 172.21.15.52:0/1587179215 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.962+0000 7fc68bfff640 1 --2- 172.21.15.52:0/1587179215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d7d486df0 0x564d7d49fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.962+0000 7fc6910b9640 1 -- 172.21.15.52:0/1587179215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d7d4864d0 msgr2=0x564d7d493690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.962+0000 7fc6910b9640 1 --2- 172.21.15.52:0/1587179215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d7d4864d0 0x564d7d493690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.962+0000 7fc6910b9640 1 -- 172.21.15.52:0/1587179215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d7d486df0 msgr2=0x564d7d49fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.962+0000 7fc6910b9640 1 --2- 172.21.15.52:0/1587179215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d7d486df0 0x564d7d49fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.962+0000 7fc6910b9640 1 -- 172.21.15.52:0/1587179215 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d7d48bd40 con 0x564d7d4940b0 2024-04-02T14:03:23.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.962+0000 7fc68bfff640 1 --2- 172.21.15.52:0/1587179215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d7d486df0 0x564d7d49fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:03:23.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.962+0000 7fc6910b9640 1 --2- 172.21.15.52:0/1587179215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d4940b0 0x564d7d4ac1f0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fc684018000 tx=0x7fc684009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa322e9f8060eb61 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:23.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.962+0000 7fc6767fc640 1 -- 172.21.15.52:0/1587179215 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684004bc0 con 0x564d7d4940b0 2024-04-02T14:03:23.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.962+0000 7fc6767fc640 1 -- 172.21.15.52:0/1587179215 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc684009a40 con 0x564d7d4940b0 2024-04-02T14:03:23.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.962+0000 7fc6767fc640 1 -- 172.21.15.52:0/1587179215 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6840052b0 con 0x564d7d4940b0 2024-04-02T14:03:23.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.963+0000 7fc6957e5740 1 -- 172.21.15.52:0/1587179215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d4940b0 msgr2=0x564d7d4ac1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.963+0000 7fc6957e5740 1 --2- 172.21.15.52:0/1587179215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d4940b0 0x564d7d4ac1f0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fc684018000 tx=0x7fc684009d70 comp rx=0 tx=0).stop 2024-04-02T14:03:23.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.963+0000 7fc6957e5740 1 -- 172.21.15.52:0/1587179215 shutdown_connections 2024-04-02T14:03:23.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.963+0000 7fc6957e5740 1 --2- 172.21.15.52:0/1587179215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d4940b0 0x564d7d4ac1f0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.963+0000 7fc6957e5740 1 --2- 172.21.15.52:0/1587179215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d7d486df0 0x564d7d49fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.963+0000 7fc6957e5740 1 --2- 172.21.15.52:0/1587179215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d7d4864d0 0x564d7d493690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.963+0000 7fc6957e5740 1 -- 172.21.15.52:0/1587179215 >> 172.21.15.52:0/1587179215 conn(0x564d7d480ed0 msgr2=0x564d7d47ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:23.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.963+0000 7fc6957e5740 1 -- 172.21.15.52:0/1587179215 shutdown_connections 2024-04-02T14:03:23.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.963+0000 7fc6957e5740 1 -- 172.21.15.52:0/1587179215 wait complete. 2024-04-02T14:03:23.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.963+0000 7fc6957e5740 1 -- start start 2024-04-02T14:03:23.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.963+0000 7fc6957e5740 1 Processor -- start 2024-04-02T14:03:23.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.964+0000 7fc6957e5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d7d4864d0 0x564d7d49c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.964+0000 7fc6957e5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d486df0 0x564d7d48d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.964+0000 7fc6957e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d7d4940b0 0x564d7d48de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.964+0000 7fc6957e5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c1ae150 con 0x564d7d486df0 2024-04-02T14:03:23.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.964+0000 7fc6957e5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c1ada80 con 0x564d7d4864d0 2024-04-02T14:03:23.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.964+0000 7fc6957e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c1aace0 con 0x564d7d4940b0 2024-04-02T14:03:23.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.964+0000 7fc6957e5740 10 client.0 _ll_register_callbacks cb 0x7fc680004110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:23.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.964+0000 7fc68bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d486df0 0x564d7d48d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.964+0000 7fc68bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d486df0 0x564d7d48d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57150/0 (socket says 172.21.15.52:57150) 2024-04-02T14:03:23.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.964+0000 7fc68bfff640 1 -- 172.21.15.52:0/1262786752 learned_addr learned my addr 172.21.15.52:0/1262786752 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:23.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.964+0000 7fc6908b8640 1 --2- 172.21.15.52:0/1262786752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d7d4864d0 0x564d7d49c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.964+0000 7fc68bfff640 1 -- 172.21.15.52:0/1262786752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d7d4940b0 msgr2=0x564d7d48de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:23.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.964+0000 7fc68bfff640 1 --2- 172.21.15.52:0/1262786752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d7d4940b0 0x564d7d48de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.964+0000 7fc68bfff640 1 -- 172.21.15.52:0/1262786752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d7d4864d0 msgr2=0x564d7d49c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:23.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.964+0000 7fc68bfff640 1 --2- 172.21.15.52:0/1262786752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d7d4864d0 0x564d7d49c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.964+0000 7fc68bfff640 1 -- 172.21.15.52:0/1262786752 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d7d48e640 con 0x564d7d486df0 2024-04-02T14:03:23.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.964+0000 7fc6908b8640 1 --2- 172.21.15.52:0/1262786752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d7d4864d0 0x564d7d49c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:03:23.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.964+0000 7fc68bfff640 1 --2- 172.21.15.52:0/1262786752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d486df0 0x564d7d48d920 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fc67809c6d0 tx=0x7fc678000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f57ce5b9e13897 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:23.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.964+0000 7fc66bfff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:23.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.964+0000 7fc66bfff640 1 -- 172.21.15.52:0/1262786752 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6780a5640 con 0x564d7d486df0 2024-04-02T14:03:23.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.964+0000 7fc66bfff640 1 -- 172.21.15.52:0/1262786752 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fc67800fb30 con 0x564d7d486df0 2024-04-02T14:03:23.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.964+0000 7fc6957e5740 1 -- 172.21.15.52:0/1262786752 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc684009a40 con 0x564d7d486df0 2024-04-02T14:03:23.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.964+0000 7fc66bfff640 1 -- 172.21.15.52:0/1262786752 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc6780ce400 con 0x564d7d486df0 2024-04-02T14:03:23.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.964+0000 7fc6957e5740 10 client.18402 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:23.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.964+0000 7fc6957e5740 1 -- 172.21.15.52:0/1262786752 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564d7d50aa40 con 0x564d7d486df0 2024-04-02T14:03:23.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.965+0000 7fc66bfff640 1 -- 172.21.15.52:0/1262786752 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc684009a40 con 0x564d7d486df0 2024-04-02T14:03:23.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.965+0000 7fc66bfff640 1 -- 172.21.15.52:0/1262786752 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc6780bbb90 con 0x564d7d486df0 2024-04-02T14:03:23.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.965+0000 7fc6957e5740 10 client.18402 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:23.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.965+0000 7fc6957e5740 10 client.18402 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T14:03:23.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.965+0000 7fc6957e5740 20 client.18402 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:23.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.965+0000 7fc6957e5740 1 --2- 172.21.15.52:0/1262786752 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x564d7d50b170 0x564d7d50d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:23.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.965+0000 7fc6957e5740 4 client.18402 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:03:23.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.965+0000 7fc6957e5740 1 -- 172.21.15.52:0/1262786752 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d7d3f2040 con 0x564d7d50b170 2024-04-02T14:03:23.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.965+0000 7fc6908b8640 1 --2- 172.21.15.52:0/1262786752 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x564d7d50b170 0x564d7d50d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:23.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.965+0000 7fc66bfff640 10 client.18402 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T14:03:23.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.965+0000 7fc6908b8640 1 --2- 172.21.15.52:0/1262786752 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x564d7d50b170 0x564d7d50d5b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f2120550b1f18535 server_cookie=5034b31f26e5fcd4 in_seq=0 out_seq=0 2024-04-02T14:03:23.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.966+0000 7fc66bfff640 1 -- 172.21.15.52:0/1262786752 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d7d3f2040 con 0x564d7d50b170 2024-04-02T14:03:23.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.966+0000 7fc66bfff640 10 client.18402 handle_command_reply: tid=0 2024-04-02T14:03:23.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.966+0000 7fc6957e5740 1 client.18402 shutdown 2024-04-02T14:03:23.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.966+0000 7fc6957e5740 1 -- 172.21.15.52:0/1262786752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d486df0 msgr2=0x564d7d48d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.966+0000 7fc6957e5740 1 --2- 172.21.15.52:0/1262786752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d486df0 0x564d7d48d920 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fc67809c6d0 tx=0x7fc678000bc0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.966+0000 7fc6957e5740 1 -- 172.21.15.52:0/1262786752 shutdown_connections 2024-04-02T14:03:23.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.966+0000 7fc6957e5740 1 --2- 172.21.15.52:0/1262786752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d7d4940b0 0x564d7d48de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.966+0000 7fc6957e5740 1 --2- 172.21.15.52:0/1262786752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d7d486df0 0x564d7d48d920 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.966+0000 7fc6957e5740 1 --2- 172.21.15.52:0/1262786752 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x564d7d50b170 0x564d7d50d5b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.966+0000 7fc6957e5740 1 --2- 172.21.15.52:0/1262786752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d7d4864d0 0x564d7d49c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.966+0000 7fc66bfff640 0 client.18402 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T14:03:23.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.966+0000 7fc6957e5740 1 -- 172.21.15.52:0/1262786752 >> 172.21.15.52:0/1262786752 conn(0x564d7d480ed0 msgr2=0x564d7d490f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:23.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.966+0000 7fc6957e5740 1 -- 172.21.15.52:0/1262786752 shutdown_connections 2024-04-02T14:03:23.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.966+0000 7fc6957e5740 1 -- 172.21.15.52:0/1262786752 wait complete. 2024-04-02T14:03:23.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.966+0000 7fc6957e5740 20 client.18402 trim_cache size 0 max 16384 2024-04-02T14:03:23.982 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:03:23.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.966+0000 7fc692b1c640 1 -- 172.21.15.52:0/3275519705 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc67c0a3d20 msgr2=0x7fc67c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.966+0000 7fc692b1c640 1 --2- 172.21.15.52:0/3275519705 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc67c0a3d20 0x7fc67c0a61e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc680006010 tx=0x7fc68002c880 comp rx=0 tx=0).stop 2024-04-02T14:03:23.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.966+0000 7fc692b1c640 1 -- 172.21.15.52:0/3275519705 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc68c101350 msgr2=0x7fc68c198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:23.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.966+0000 7fc692b1c640 1 --2- 172.21.15.52:0/3275519705 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc68c101350 0x7fc68c198890 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fc67800c920 tx=0x7fc67800cde0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.966+0000 7fc692b1c640 1 -- 172.21.15.52:0/3275519705 shutdown_connections 2024-04-02T14:03:23.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.966+0000 7fc692b1c640 1 --2- 172.21.15.52:0/3275519705 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc67c0a3d20 0x7fc67c0a61e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.966+0000 7fc692b1c640 1 --2- 172.21.15.52:0/3275519705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc68c1069a0 0x7fc68c19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.966+0000 7fc692b1c640 1 --2- 172.21.15.52:0/3275519705 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc68c101350 0x7fc68c198890 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.966+0000 7fc692b1c640 1 --2- 172.21.15.52:0/3275519705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc68c100990 0x7fc68c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:23.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.966+0000 7fc692b1c640 1 -- 172.21.15.52:0/3275519705 >> 172.21.15.52:0/3275519705 conn(0x7fc68c077ad0 msgr2=0x7fc68c103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:23.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.967+0000 7fc692b1c640 1 -- 172.21.15.52:0/3275519705 shutdown_connections 2024-04-02T14:03:23.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:23.967+0000 7fc692b1c640 1 -- 172.21.15.52:0/3275519705 wait complete. 2024-04-02T14:03:23.985 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T14:03:24.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.085+0000 7fc63e693640 1 -- 172.21.15.52:0/3883707649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc638100970 msgr2=0x7fc638100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.085+0000 7fc63e693640 1 --2- 172.21.15.52:0/3883707649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc638100970 0x7fc638100df0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009930 tx=0x7fc62c02bc30 comp rx=0 tx=0).stop 2024-04-02T14:03:24.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.085+0000 7fc63e693640 1 -- 172.21.15.52:0/3883707649 shutdown_connections 2024-04-02T14:03:24.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.085+0000 7fc63e693640 1 --2- 172.21.15.52:0/3883707649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc638101330 0x7fc63810d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.085+0000 7fc63e693640 1 --2- 172.21.15.52:0/3883707649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc638100970 0x7fc638100df0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.085+0000 7fc63e693640 1 --2- 172.21.15.52:0/3883707649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc638106980 0x7fc638106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.085+0000 7fc63e693640 1 -- 172.21.15.52:0/3883707649 >> 172.21.15.52:0/3883707649 conn(0x7fc638077b00 msgr2=0x7fc638077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:24.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.086+0000 7fc63e693640 1 -- 172.21.15.52:0/3883707649 shutdown_connections 2024-04-02T14:03:24.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.086+0000 7fc63e693640 1 -- 172.21.15.52:0/3883707649 wait complete. 2024-04-02T14:03:24.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.086+0000 7fc63e693640 1 Processor -- start 2024-04-02T14:03:24.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.086+0000 7fc63e693640 1 -- start start 2024-04-02T14:03:24.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.086+0000 7fc63e693640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc638100970 0x7fc6381972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.086+0000 7fc63e693640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc638101330 0x7fc638197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.087+0000 7fc63e693640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc638106980 0x7fc63819bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.087+0000 7fc63e693640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc638110d60 con 0x7fc638100970 2024-04-02T14:03:24.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.087+0000 7fc63e693640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc638110be0 con 0x7fc638101330 2024-04-02T14:03:24.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.087+0000 7fc63e693640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc638110ee0 con 0x7fc638106980 2024-04-02T14:03:24.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.087+0000 7fc637fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc638100970 0x7fc6381972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.087+0000 7fc637fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc638100970 0x7fc6381972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57166/0 (socket says 172.21.15.52:57166) 2024-04-02T14:03:24.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.087+0000 7fc637fff640 1 -- 172.21.15.52:0/2025445629 learned_addr learned my addr 172.21.15.52:0/2025445629 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:24.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.087+0000 7fc63cc30640 1 --2- 172.21.15.52:0/2025445629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc638106980 0x7fc63819bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.087+0000 7fc6377fe640 1 --2- 172.21.15.52:0/2025445629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc638101330 0x7fc638197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.087+0000 7fc637fff640 1 -- 172.21.15.52:0/2025445629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc638106980 msgr2=0x7fc63819bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.087+0000 7fc637fff640 1 --2- 172.21.15.52:0/2025445629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc638106980 0x7fc63819bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.087+0000 7fc637fff640 1 -- 172.21.15.52:0/2025445629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc638101330 msgr2=0x7fc638197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.087+0000 7fc637fff640 1 --2- 172.21.15.52:0/2025445629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc638101330 0x7fc638197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.087+0000 7fc637fff640 1 -- 172.21.15.52:0/2025445629 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63819c2d0 con 0x7fc638100970 2024-04-02T14:03:24.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.087+0000 7fc6377fe640 1 --2- 172.21.15.52:0/2025445629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc638101330 0x7fc638197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:03:24.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.087+0000 7fc637fff640 1 --2- 172.21.15.52:0/2025445629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc638100970 0x7fc6381972e0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fc62800d950 tx=0x7fc62800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:24.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.087+0000 7fc6357fa640 1 -- 172.21.15.52:0/2025445629 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc628017070 con 0x7fc638100970 2024-04-02T14:03:24.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.087+0000 7fc63e693640 1 -- 172.21.15.52:0/2025445629 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63819c5c0 con 0x7fc638100970 2024-04-02T14:03:24.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.087+0000 7fc6357fa640 1 -- 172.21.15.52:0/2025445629 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc628009c30 con 0x7fc638100970 2024-04-02T14:03:24.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.087+0000 7fc6357fa640 1 -- 172.21.15.52:0/2025445629 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62800f6b0 con 0x7fc638100970 2024-04-02T14:03:24.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.087+0000 7fc63e693640 1 -- 172.21.15.52:0/2025445629 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6381a3e50 con 0x7fc638100970 2024-04-02T14:03:24.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.089+0000 7fc6357fa640 1 -- 172.21.15.52:0/2025445629 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc6280109d0 con 0x7fc638100970 2024-04-02T14:03:24.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.090+0000 7fc640f22740 1 Processor -- start 2024-04-02T14:03:24.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.090+0000 7fc640f22740 1 -- start start 2024-04-02T14:03:24.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.090+0000 7fc640f22740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618d44577a0 0x5618d4457b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.090+0000 7fc640f22740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5618d44580c0 0x5618d4460760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.090+0000 7fc6357fa640 1 --2- 172.21.15.52:0/2025445629 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc6200a3d20 0x7fc6200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.090+0000 7fc640f22740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d4460ca0 0x5618d4463090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.090+0000 7fc640f22740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5618d43bb7b0 con 0x5618d4460ca0 2024-04-02T14:03:24.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.090+0000 7fc640f22740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5618d435a570 con 0x5618d44580c0 2024-04-02T14:03:24.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.090+0000 7fc640f22740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5618d43b7760 con 0x5618d44577a0 2024-04-02T14:03:24.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.090+0000 7fc6377fe640 1 --2- 172.21.15.52:0/2025445629 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc6200a3d20 0x7fc6200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.090+0000 7fc63cc30640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d4460ca0 0x5618d4463090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.090+0000 7fc63cc30640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d4460ca0 0x5618d4463090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57176/0 (socket says 172.21.15.52:57176) 2024-04-02T14:03:24.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.090+0000 7fc63cc30640 1 -- 172.21.15.52:0/3071395857 learned_addr learned my addr 172.21.15.52:0/3071395857 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:24.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.090+0000 7fc6357fa640 1 -- 172.21.15.52:0/2025445629 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fc6280d07a0 con 0x7fc638100970 2024-04-02T14:03:24.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.091+0000 7fc63cc30640 1 --2- 172.21.15.52:0/3071395857 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618d44577a0 0x5618d4457b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.091+0000 7fc637fff640 1 --2- 172.21.15.52:0/3071395857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5618d44580c0 0x5618d4460760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.091+0000 7fc63cc30640 1 -- 172.21.15.52:0/3071395857 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618d44577a0 msgr2=0x5618d4457b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:24.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.091+0000 7fc63cc30640 1 --2- 172.21.15.52:0/3071395857 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618d44577a0 0x5618d4457b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.091+0000 7fc63cc30640 1 -- 172.21.15.52:0/3071395857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5618d44580c0 msgr2=0x5618d4460760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:24.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.091+0000 7fc63cc30640 1 --2- 172.21.15.52:0/3071395857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5618d44580c0 0x5618d4460760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.091+0000 7fc63cc30640 1 -- 172.21.15.52:0/3071395857 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618d40cf3f0 con 0x5618d4460ca0 2024-04-02T14:03:24.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.091+0000 7fc6377fe640 1 --2- 172.21.15.52:0/2025445629 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc6200a3d20 0x7fc6200a61e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc62c006010 tx=0x7fc62c02c6f0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:24.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.091+0000 7fc63cc30640 1 --2- 172.21.15.52:0/3071395857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d4460ca0 0x5618d4463090 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fc630012aa0 tx=0x7fc630009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9f3c5a57eb15d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:24.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.091+0000 7fc61e7fc640 1 -- 172.21.15.52:0/3071395857 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630003a30 con 0x5618d4460ca0 2024-04-02T14:03:24.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.091+0000 7fc61e7fc640 1 -- 172.21.15.52:0/3071395857 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc630010d10 con 0x5618d4460ca0 2024-04-02T14:03:24.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.091+0000 7fc61e7fc640 1 -- 172.21.15.52:0/3071395857 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630004450 con 0x5618d4460ca0 2024-04-02T14:03:24.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.092+0000 7fc640f22740 1 -- 172.21.15.52:0/3071395857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d4460ca0 msgr2=0x5618d4463090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.092+0000 7fc640f22740 1 --2- 172.21.15.52:0/3071395857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d4460ca0 0x5618d4463090 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fc630012aa0 tx=0x7fc630009a70 comp rx=0 tx=0).stop 2024-04-02T14:03:24.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.092+0000 7fc640f22740 1 -- 172.21.15.52:0/3071395857 shutdown_connections 2024-04-02T14:03:24.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.092+0000 7fc640f22740 1 --2- 172.21.15.52:0/3071395857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d4460ca0 0x5618d4463090 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.092+0000 7fc640f22740 1 --2- 172.21.15.52:0/3071395857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5618d44580c0 0x5618d4460760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.092+0000 7fc640f22740 1 --2- 172.21.15.52:0/3071395857 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618d44577a0 0x5618d4457b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.092+0000 7fc640f22740 1 -- 172.21.15.52:0/3071395857 >> 172.21.15.52:0/3071395857 conn(0x5618d4451ed0 msgr2=0x5618d44522b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:24.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.092+0000 7fc640f22740 1 -- 172.21.15.52:0/3071395857 shutdown_connections 2024-04-02T14:03:24.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.092+0000 7fc640f22740 1 -- 172.21.15.52:0/3071395857 wait complete. 2024-04-02T14:03:24.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.092+0000 7fc640f22740 1 -- start start 2024-04-02T14:03:24.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.092+0000 7fc640f22740 1 Processor -- start 2024-04-02T14:03:24.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.093+0000 7fc640f22740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5618d44577a0 0x5618d44606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.093+0000 7fc640f22740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d44580c0 0x5618d445da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.093+0000 7fc640f22740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618d4460ca0 0x5618d445e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.093+0000 7fc640f22740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5618d4463a90 con 0x5618d44580c0 2024-04-02T14:03:24.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.093+0000 7fc640f22740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5618d4376e80 con 0x5618d44577a0 2024-04-02T14:03:24.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.093+0000 7fc640f22740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5618d4463c10 con 0x5618d4460ca0 2024-04-02T14:03:24.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.093+0000 7fc640f22740 10 client.0 _ll_register_callbacks cb 0x5618d43b2aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:24.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.093+0000 7fc6377fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d44580c0 0x5618d445da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.093+0000 7fc6377fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d44580c0 0x5618d445da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57180/0 (socket says 172.21.15.52:57180) 2024-04-02T14:03:24.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.093+0000 7fc6377fe640 1 -- 172.21.15.52:0/58604930 learned_addr learned my addr 172.21.15.52:0/58604930 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:24.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.093+0000 7fc63cc30640 1 --2- 172.21.15.52:0/58604930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618d4460ca0 0x5618d445e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.093+0000 7fc637fff640 1 --2- 172.21.15.52:0/58604930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5618d44577a0 0x5618d44606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.093+0000 7fc6377fe640 1 -- 172.21.15.52:0/58604930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618d4460ca0 msgr2=0x5618d445e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:24.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.093+0000 7fc6377fe640 1 --2- 172.21.15.52:0/58604930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618d4460ca0 0x5618d445e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.093+0000 7fc6377fe640 1 -- 172.21.15.52:0/58604930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5618d44577a0 msgr2=0x5618d44606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.093+0000 7fc6377fe640 1 --2- 172.21.15.52:0/58604930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5618d44577a0 0x5618d44606f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.093+0000 7fc6377fe640 1 -- 172.21.15.52:0/58604930 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618d41997d0 con 0x5618d44580c0 2024-04-02T14:03:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.093+0000 7fc6377fe640 1 --2- 172.21.15.52:0/58604930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d44580c0 0x5618d445da80 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009930 tx=0x7fc62c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fef7ac0e70bf9da server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.093+0000 7fc6177fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.093+0000 7fc6177fe640 1 -- 172.21.15.52:0/58604930 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c04d020 con 0x5618d44580c0 2024-04-02T14:03:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.093+0000 7fc6177fe640 1 -- 172.21.15.52:0/58604930 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fc62c003cc0 con 0x5618d44580c0 2024-04-02T14:03:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.093+0000 7fc640f22740 1 -- 172.21.15.52:0/58604930 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc630010d10 con 0x5618d44580c0 2024-04-02T14:03:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.094+0000 7fc6177fe640 1 -- 172.21.15.52:0/58604930 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc62c058050 con 0x5618d44580c0 2024-04-02T14:03:24.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.094+0000 7fc640f22740 10 client.18414 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:24.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.094+0000 7fc640f22740 1 -- 172.21.15.52:0/58604930 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5618d446fa80 con 0x5618d44580c0 2024-04-02T14:03:24.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.094+0000 7fc6177fe640 1 -- 172.21.15.52:0/58604930 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc630010d10 con 0x5618d44580c0 2024-04-02T14:03:24.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.094+0000 7fc6177fe640 1 -- 172.21.15.52:0/58604930 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc62c004c50 con 0x5618d44580c0 2024-04-02T14:03:24.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.094+0000 7fc640f22740 10 client.18414 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:24.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.094+0000 7fc640f22740 10 client.18414 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T14:03:24.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.094+0000 7fc640f22740 20 client.18414 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:24.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.094+0000 7fc640f22740 1 --2- 172.21.15.52:0/58604930 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5618d4470fe0 0x5618d4473420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.094+0000 7fc640f22740 4 client.18414 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:03:24.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.094+0000 7fc640f22740 1 -- 172.21.15.52:0/58604930 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618d43c8cf0 con 0x5618d4470fe0 2024-04-02T14:03:24.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.094+0000 7fc637fff640 1 --2- 172.21.15.52:0/58604930 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5618d4470fe0 0x5618d4473420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.095+0000 7fc637fff640 1 --2- 172.21.15.52:0/58604930 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5618d4470fe0 0x5618d4473420 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6e75b45e2953b52 server_cookie=6c5750a7ff87d22 in_seq=0 out_seq=0 2024-04-02T14:03:24.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.095+0000 7fc6177fe640 10 client.18414 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T14:03:24.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.095+0000 7fc6177fe640 1 -- 172.21.15.52:0/58604930 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5618d43c8cf0 con 0x5618d4470fe0 2024-04-02T14:03:24.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.095+0000 7fc6177fe640 10 client.18414 handle_command_reply: tid=0 2024-04-02T14:03:24.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.095+0000 7fc640f22740 1 client.18414 shutdown 2024-04-02T14:03:24.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.096+0000 7fc640f22740 1 -- 172.21.15.52:0/58604930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d44580c0 msgr2=0x5618d445da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.096+0000 7fc640f22740 1 --2- 172.21.15.52:0/58604930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d44580c0 0x5618d445da80 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009930 tx=0x7fc62c009260 comp rx=0 tx=0).stop 2024-04-02T14:03:24.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.096+0000 7fc640f22740 1 -- 172.21.15.52:0/58604930 shutdown_connections 2024-04-02T14:03:24.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.096+0000 7fc640f22740 1 --2- 172.21.15.52:0/58604930 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5618d4470fe0 0x5618d4473420 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.096+0000 7fc640f22740 1 --2- 172.21.15.52:0/58604930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618d4460ca0 0x5618d445e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.096+0000 7fc640f22740 1 --2- 172.21.15.52:0/58604930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d44580c0 0x5618d445da80 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.096+0000 7fc640f22740 1 --2- 172.21.15.52:0/58604930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5618d44577a0 0x5618d44606f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.096+0000 7fc6177fe640 0 client.18414 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T14:03:24.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.096+0000 7fc640f22740 1 -- 172.21.15.52:0/58604930 >> 172.21.15.52:0/58604930 conn(0x5618d4451ed0 msgr2=0x5618d4453c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:24.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.096+0000 7fc640f22740 1 -- 172.21.15.52:0/58604930 shutdown_connections 2024-04-02T14:03:24.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.096+0000 7fc640f22740 1 -- 172.21.15.52:0/58604930 wait complete. 2024-04-02T14:03:24.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.096+0000 7fc640f22740 20 client.18414 trim_cache size 0 max 16384 2024-04-02T14:03:24.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.109+0000 7fc640f22740 1 Processor -- start 2024-04-02T14:03:24.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.109+0000 7fc640f22740 1 -- start start 2024-04-02T14:03:24.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.109+0000 7fc640f22740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5618d44577a0 0x5618d44584a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.109+0000 7fc640f22740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d44580c0 0x5618d446c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.109+0000 7fc640f22740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618d4460ca0 0x5618d4469080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.109+0000 7fc640f22740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc6381adf40 con 0x5618d44580c0 2024-04-02T14:03:24.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.109+0000 7fc640f22740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc6381afa80 con 0x5618d44577a0 2024-04-02T14:03:24.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.109+0000 7fc640f22740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc6381ae610 con 0x5618d4460ca0 2024-04-02T14:03:24.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.109+0000 7fc6377fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d44580c0 0x5618d446c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.109+0000 7fc6377fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d44580c0 0x5618d446c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57188/0 (socket says 172.21.15.52:57188) 2024-04-02T14:03:24.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.109+0000 7fc6377fe640 1 -- 172.21.15.52:0/508611655 learned_addr learned my addr 172.21.15.52:0/508611655 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:24.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.110+0000 7fc637fff640 1 --2- 172.21.15.52:0/508611655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5618d44577a0 0x5618d44584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.110+0000 7fc6377fe640 1 -- 172.21.15.52:0/508611655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618d4460ca0 msgr2=0x5618d4469080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:24.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.110+0000 7fc63cc30640 1 --2- 172.21.15.52:0/508611655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618d4460ca0 0x5618d4469080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.110+0000 7fc6377fe640 1 --2- 172.21.15.52:0/508611655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618d4460ca0 0x5618d4469080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.110+0000 7fc6377fe640 1 -- 172.21.15.52:0/508611655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5618d44577a0 msgr2=0x5618d44584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:24.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.110+0000 7fc6377fe640 1 --2- 172.21.15.52:0/508611655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5618d44577a0 0x5618d44584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.110+0000 7fc6377fe640 1 -- 172.21.15.52:0/508611655 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618d4457ef0 con 0x5618d44580c0 2024-04-02T14:03:24.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.110+0000 7fc6377fe640 1 --2- 172.21.15.52:0/508611655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d44580c0 0x5618d446c980 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fc62c02f020 tx=0x7fc62c005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb9c51b9d942377 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:24.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.110+0000 7fc61dffb640 1 -- 172.21.15.52:0/508611655 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c061070 con 0x5618d44580c0 2024-04-02T14:03:24.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.110+0000 7fc61dffb640 1 -- 172.21.15.52:0/508611655 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc62c044070 con 0x5618d44580c0 2024-04-02T14:03:24.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.110+0000 7fc61dffb640 1 -- 172.21.15.52:0/508611655 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c04d040 con 0x5618d44580c0 2024-04-02T14:03:24.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.111+0000 7fc640f22740 1 -- 172.21.15.52:0/508611655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d44580c0 msgr2=0x5618d446c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.111+0000 7fc640f22740 1 --2- 172.21.15.52:0/508611655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d44580c0 0x5618d446c980 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fc62c02f020 tx=0x7fc62c005a40 comp rx=0 tx=0).stop 2024-04-02T14:03:24.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.111+0000 7fc640f22740 1 -- 172.21.15.52:0/508611655 shutdown_connections 2024-04-02T14:03:24.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.111+0000 7fc640f22740 1 --2- 172.21.15.52:0/508611655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618d4460ca0 0x5618d4469080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.111+0000 7fc640f22740 1 --2- 172.21.15.52:0/508611655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d44580c0 0x5618d446c980 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.111+0000 7fc640f22740 1 --2- 172.21.15.52:0/508611655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5618d44577a0 0x5618d44584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.111+0000 7fc640f22740 1 -- 172.21.15.52:0/508611655 >> 172.21.15.52:0/508611655 conn(0x5618d4451ed0 msgr2=0x5618d445c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:24.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.111+0000 7fc640f22740 1 -- 172.21.15.52:0/508611655 shutdown_connections 2024-04-02T14:03:24.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.111+0000 7fc640f22740 1 -- 172.21.15.52:0/508611655 wait complete. 2024-04-02T14:03:24.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.111+0000 7fc640f22740 1 -- start start 2024-04-02T14:03:24.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.111+0000 7fc640f22740 1 Processor -- start 2024-04-02T14:03:24.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.111+0000 7fc640f22740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618d44577a0 0x5618d4462750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.111+0000 7fc640f22740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d44580c0 0x5618d445ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.111+0000 7fc640f22740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5618d4460ca0 0x5618d445b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.111+0000 7fc640f22740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc6381af3b0 con 0x5618d44580c0 2024-04-02T14:03:24.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.111+0000 7fc640f22740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc6381aece0 con 0x5618d4460ca0 2024-04-02T14:03:24.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.111+0000 7fc640f22740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc6381abf40 con 0x5618d44577a0 2024-04-02T14:03:24.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.111+0000 7fc640f22740 10 client.0 _ll_register_callbacks cb 0x7fc62c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:24.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.111+0000 7fc6377fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d44580c0 0x5618d445ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.111+0000 7fc6377fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d44580c0 0x5618d445ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57200/0 (socket says 172.21.15.52:57200) 2024-04-02T14:03:24.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.111+0000 7fc6377fe640 1 -- 172.21.15.52:0/1487816960 learned_addr learned my addr 172.21.15.52:0/1487816960 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:24.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.112+0000 7fc6377fe640 1 -- 172.21.15.52:0/1487816960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618d44577a0 msgr2=0x5618d4462750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:24.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.112+0000 7fc6377fe640 1 --2- 172.21.15.52:0/1487816960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618d44577a0 0x5618d4462750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.112+0000 7fc6377fe640 1 -- 172.21.15.52:0/1487816960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5618d4460ca0 msgr2=0x5618d445b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:24.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.112+0000 7fc6377fe640 1 --2- 172.21.15.52:0/1487816960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5618d4460ca0 0x5618d445b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.112+0000 7fc6377fe640 1 -- 172.21.15.52:0/1487816960 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618d445b7b0 con 0x5618d44580c0 2024-04-02T14:03:24.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.112+0000 7fc6377fe640 1 --2- 172.21.15.52:0/1487816960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d44580c0 0x5618d445ad00 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009930 tx=0x7fc62c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231fb4f760c55bb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:24.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.112+0000 7fc6177fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:24.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.112+0000 7fc6177fe640 1 -- 172.21.15.52:0/1487816960 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c061030 con 0x5618d44580c0 2024-04-02T14:03:24.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.112+0000 7fc6177fe640 1 -- 172.21.15.52:0/1487816960 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fc62c034a50 con 0x5618d44580c0 2024-04-02T14:03:24.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.112+0000 7fc640f22740 1 -- 172.21.15.52:0/1487816960 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc62c044070 con 0x5618d44580c0 2024-04-02T14:03:24.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.112+0000 7fc6177fe640 1 -- 172.21.15.52:0/1487816960 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc62c04d050 con 0x5618d44580c0 2024-04-02T14:03:24.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.112+0000 7fc640f22740 10 client.18420 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:24.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.112+0000 7fc640f22740 1 -- 172.21.15.52:0/1487816960 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5618d4461f70 con 0x5618d44580c0 2024-04-02T14:03:24.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.112+0000 7fc6177fe640 1 -- 172.21.15.52:0/1487816960 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc62c044070 con 0x5618d44580c0 2024-04-02T14:03:24.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.113+0000 7fc6177fe640 1 -- 172.21.15.52:0/1487816960 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc62c0050c0 con 0x5618d44580c0 2024-04-02T14:03:24.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.113+0000 7fc640f22740 10 client.18420 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:24.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.113+0000 7fc640f22740 10 client.18420 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T14:03:24.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.113+0000 7fc640f22740 20 client.18420 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:24.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.113+0000 7fc640f22740 1 --2- 172.21.15.52:0/1487816960 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5618d44d99b0 0x5618d44dbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.113+0000 7fc640f22740 4 client.18420 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:03:24.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.113+0000 7fc640f22740 1 -- 172.21.15.52:0/1487816960 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5618d43c3040 con 0x5618d44d99b0 2024-04-02T14:03:24.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.113+0000 7fc637fff640 1 --2- 172.21.15.52:0/1487816960 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5618d44d99b0 0x5618d44dbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.113+0000 7fc637fff640 1 --2- 172.21.15.52:0/1487816960 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5618d44d99b0 0x5618d44dbdd0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4fb0d2df050669cd server_cookie=b7fe6987df8de33c in_seq=0 out_seq=0 2024-04-02T14:03:24.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.113+0000 7fc6177fe640 10 client.18420 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T14:03:24.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.113+0000 7fc6177fe640 1 -- 172.21.15.52:0/1487816960 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5618d43c3040 con 0x5618d44d99b0 2024-04-02T14:03:24.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.113+0000 7fc6177fe640 10 client.18420 handle_command_reply: tid=0 2024-04-02T14:03:24.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.113+0000 7fc640f22740 1 client.18420 shutdown 2024-04-02T14:03:24.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.114+0000 7fc640f22740 1 -- 172.21.15.52:0/1487816960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d44580c0 msgr2=0x5618d445ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.114+0000 7fc640f22740 1 --2- 172.21.15.52:0/1487816960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d44580c0 0x5618d445ad00 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009930 tx=0x7fc62c0039c0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.114+0000 7fc640f22740 1 -- 172.21.15.52:0/1487816960 shutdown_connections 2024-04-02T14:03:24.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.114+0000 7fc640f22740 1 --2- 172.21.15.52:0/1487816960 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x5618d44d99b0 0x5618d44dbdd0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.114+0000 7fc640f22740 1 --2- 172.21.15.52:0/1487816960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5618d4460ca0 0x5618d445b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.114+0000 7fc640f22740 1 --2- 172.21.15.52:0/1487816960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618d44580c0 0x5618d445ad00 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.114+0000 7fc6177fe640 0 client.18420 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T14:03:24.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.114+0000 7fc640f22740 1 --2- 172.21.15.52:0/1487816960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618d44577a0 0x5618d4462750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.114+0000 7fc640f22740 1 -- 172.21.15.52:0/1487816960 >> 172.21.15.52:0/1487816960 conn(0x5618d4451ed0 msgr2=0x5618d445c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:24.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.114+0000 7fc640f22740 1 -- 172.21.15.52:0/1487816960 shutdown_connections 2024-04-02T14:03:24.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.114+0000 7fc640f22740 1 -- 172.21.15.52:0/1487816960 wait complete. 2024-04-02T14:03:24.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.114+0000 7fc640f22740 20 client.18420 trim_cache size 0 max 16384 2024-04-02T14:03:24.128 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:03:24.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.114+0000 7fc63e693640 1 -- 172.21.15.52:0/2025445629 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc6200a3d20 msgr2=0x7fc6200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.114+0000 7fc63e693640 1 --2- 172.21.15.52:0/2025445629 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc6200a3d20 0x7fc6200a61e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc62c006010 tx=0x7fc62c02c6f0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.114+0000 7fc63e693640 1 -- 172.21.15.52:0/2025445629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc638100970 msgr2=0x7fc6381972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.114+0000 7fc63e693640 1 --2- 172.21.15.52:0/2025445629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc638100970 0x7fc6381972e0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fc62800d950 tx=0x7fc62800de10 comp rx=0 tx=0).stop 2024-04-02T14:03:24.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.114+0000 7fc63e693640 1 -- 172.21.15.52:0/2025445629 shutdown_connections 2024-04-02T14:03:24.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.114+0000 7fc63e693640 1 --2- 172.21.15.52:0/2025445629 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fc6200a3d20 0x7fc6200a61e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.114+0000 7fc63e693640 1 --2- 172.21.15.52:0/2025445629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc638106980 0x7fc63819bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.114+0000 7fc63e693640 1 --2- 172.21.15.52:0/2025445629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc638101330 0x7fc638197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.114+0000 7fc63e693640 1 --2- 172.21.15.52:0/2025445629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc638100970 0x7fc6381972e0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.114+0000 7fc63e693640 1 -- 172.21.15.52:0/2025445629 >> 172.21.15.52:0/2025445629 conn(0x7fc638077b00 msgr2=0x7fc6381031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:24.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.114+0000 7fc63e693640 1 -- 172.21.15.52:0/2025445629 shutdown_connections 2024-04-02T14:03:24.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.114+0000 7fc63e693640 1 -- 172.21.15.52:0/2025445629 wait complete. 2024-04-02T14:03:24.131 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T14:03:24.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.232+0000 7f8522d4d640 1 -- 172.21.15.52:0/4267200701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f851c1009b0 msgr2=0x7f851c100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.232+0000 7f8522d4d640 1 --2- 172.21.15.52:0/4267200701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f851c1009b0 0x7f851c100e30 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f8510009930 tx=0x7f851002bc30 comp rx=0 tx=0).stop 2024-04-02T14:03:24.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.232+0000 7f8522d4d640 1 -- 172.21.15.52:0/4267200701 shutdown_connections 2024-04-02T14:03:24.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.232+0000 7f8522d4d640 1 --2- 172.21.15.52:0/4267200701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f851c101370 0x7f851c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.232+0000 7f8522d4d640 1 --2- 172.21.15.52:0/4267200701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f851c1009b0 0x7f851c100e30 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.232+0000 7f8522d4d640 1 --2- 172.21.15.52:0/4267200701 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f851c1069c0 0x7f851c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.232+0000 7f8522d4d640 1 -- 172.21.15.52:0/4267200701 >> 172.21.15.52:0/4267200701 conn(0x7f851c076d80 msgr2=0x7f851c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:24.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.232+0000 7f8522d4d640 1 -- 172.21.15.52:0/4267200701 shutdown_connections 2024-04-02T14:03:24.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.232+0000 7f8522d4d640 1 -- 172.21.15.52:0/4267200701 wait complete. 2024-04-02T14:03:24.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.233+0000 7f8522d4d640 1 Processor -- start 2024-04-02T14:03:24.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.233+0000 7f8522d4d640 1 -- start start 2024-04-02T14:03:24.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.233+0000 7f8522d4d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f851c1009b0 0x7f851c1970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.233+0000 7f8522d4d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f851c101370 0x7f851c1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.233+0000 7f8522d4d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f851c1069c0 0x7f851c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.233+0000 7f8522d4d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f851c110dd0 con 0x7f851c101370 2024-04-02T14:03:24.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.233+0000 7f8522d4d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f851c110c50 con 0x7f851c1069c0 2024-04-02T14:03:24.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.233+0000 7f8522d4d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f851c110f50 con 0x7f851c1009b0 2024-04-02T14:03:24.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.233+0000 7f851bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f851c101370 0x7f851c1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.233+0000 7f851bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f851c101370 0x7f851c1975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57220/0 (socket says 172.21.15.52:57220) 2024-04-02T14:03:24.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.233+0000 7f851bfff640 1 -- 172.21.15.52:0/3280400160 learned_addr learned my addr 172.21.15.52:0/3280400160 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:24.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.233+0000 7f85212ea640 1 --2- 172.21.15.52:0/3280400160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f851c1069c0 0x7f851c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.233+0000 7f8520ae9640 1 --2- 172.21.15.52:0/3280400160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f851c1009b0 0x7f851c1970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.233+0000 7f851bfff640 1 -- 172.21.15.52:0/3280400160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f851c1009b0 msgr2=0x7f851c1970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.233+0000 7f851bfff640 1 --2- 172.21.15.52:0/3280400160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f851c1009b0 0x7f851c1970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.233+0000 7f851bfff640 1 -- 172.21.15.52:0/3280400160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f851c1069c0 msgr2=0x7f851c197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.233+0000 7f851bfff640 1 --2- 172.21.15.52:0/3280400160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f851c1069c0 0x7f851c197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.233+0000 7f851bfff640 1 -- 172.21.15.52:0/3280400160 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851c1a6910 con 0x7f851c101370 2024-04-02T14:03:24.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.234+0000 7f851bfff640 1 --2- 172.21.15.52:0/3280400160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f851c101370 0x7f851c1975e0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f8510009230 tx=0x7f851002cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:24.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.234+0000 7f8519ffb640 1 -- 172.21.15.52:0/3280400160 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8510041070 con 0x7f851c101370 2024-04-02T14:03:24.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.234+0000 7f8519ffb640 1 -- 172.21.15.52:0/3280400160 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8510004450 con 0x7f851c101370 2024-04-02T14:03:24.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.234+0000 7f8522d4d640 1 -- 172.21.15.52:0/3280400160 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f851c1a6ba0 con 0x7f851c101370 2024-04-02T14:03:24.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.234+0000 7f8519ffb640 1 -- 172.21.15.52:0/3280400160 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8510034440 con 0x7f851c101370 2024-04-02T14:03:24.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.234+0000 7f8522d4d640 1 -- 172.21.15.52:0/3280400160 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f851c1a7090 con 0x7f851c101370 2024-04-02T14:03:24.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.236+0000 7f8519ffb640 1 -- 172.21.15.52:0/3280400160 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8510044070 con 0x7f851c101370 2024-04-02T14:03:24.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.236+0000 7f8525a0b740 1 Processor -- start 2024-04-02T14:03:24.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.236+0000 7f8525a0b740 1 -- start start 2024-04-02T14:03:24.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.237+0000 7f8525a0b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555b8ae247a0 0x555b8ae24b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.237+0000 7f8525a0b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae250c0 0x555b8ae2d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.237+0000 7f8525a0b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b8ae2dca0 0x555b8ae30090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.237+0000 7f8525a0b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555b8ad887b0 con 0x555b8ae250c0 2024-04-02T14:03:24.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.237+0000 7f8525a0b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x555b8ad27570 con 0x555b8ae247a0 2024-04-02T14:03:24.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.237+0000 7f8525a0b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555b8ad84760 con 0x555b8ae2dca0 2024-04-02T14:03:24.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.237+0000 7f8519ffb640 1 --2- 172.21.15.52:0/3280400160 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f85040a3d20 0x7f85040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.237+0000 7f8520ae9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae250c0 0x555b8ae2d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.237+0000 7f85212ea640 1 --2- 172.21.15.52:0/3280400160 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f85040a3d20 0x7f85040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.237+0000 7f8520ae9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae250c0 0x555b8ae2d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57228/0 (socket says 172.21.15.52:57228) 2024-04-02T14:03:24.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.237+0000 7f8520ae9640 1 -- 172.21.15.52:0/1759168656 learned_addr learned my addr 172.21.15.52:0/1759168656 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:24.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.237+0000 7f8519ffb640 1 -- 172.21.15.52:0/3280400160 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f85100f2550 con 0x7f851c101370 2024-04-02T14:03:24.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.237+0000 7f851bfff640 1 --2- 172.21.15.52:0/1759168656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b8ae2dca0 0x555b8ae30090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.237+0000 7f85212ea640 1 --2- 172.21.15.52:0/1759168656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555b8ae247a0 0x555b8ae24b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.237+0000 7f8520ae9640 1 -- 172.21.15.52:0/1759168656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b8ae2dca0 msgr2=0x555b8ae30090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:24.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.237+0000 7f8520ae9640 1 --2- 172.21.15.52:0/1759168656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b8ae2dca0 0x555b8ae30090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.237+0000 7f8520ae9640 1 -- 172.21.15.52:0/1759168656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555b8ae247a0 msgr2=0x555b8ae24b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:24.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.237+0000 7f8520ae9640 1 --2- 172.21.15.52:0/1759168656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555b8ae247a0 0x555b8ae24b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.237+0000 7f8520ae9640 1 -- 172.21.15.52:0/1759168656 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b8aa9c3f0 con 0x555b8ae250c0 2024-04-02T14:03:24.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.237+0000 7f85212ea640 1 --2- 172.21.15.52:0/3280400160 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f85040a3d20 0x7f85040a61e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8514009fd0 tx=0x7f8514009ee0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:24.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.237+0000 7f8520ae9640 1 --2- 172.21.15.52:0/1759168656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae250c0 0x555b8ae2d760 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f850c009bc0 tx=0x7f850c00e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57c2b42600b049b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:24.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.237+0000 7f8502ffd640 1 -- 172.21.15.52:0/1759168656 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c016070 con 0x555b8ae250c0 2024-04-02T14:03:24.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.237+0000 7f8502ffd640 1 -- 172.21.15.52:0/1759168656 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f850c005500 con 0x555b8ae250c0 2024-04-02T14:03:24.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.237+0000 7f8502ffd640 1 -- 172.21.15.52:0/1759168656 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c011470 con 0x555b8ae250c0 2024-04-02T14:03:24.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.238+0000 7f8525a0b740 1 -- 172.21.15.52:0/1759168656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae250c0 msgr2=0x555b8ae2d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.238+0000 7f8525a0b740 1 --2- 172.21.15.52:0/1759168656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae250c0 0x555b8ae2d760 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f850c009bc0 tx=0x7f850c00e2f0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.238+0000 7f8525a0b740 1 -- 172.21.15.52:0/1759168656 shutdown_connections 2024-04-02T14:03:24.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.238+0000 7f8525a0b740 1 --2- 172.21.15.52:0/1759168656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b8ae2dca0 0x555b8ae30090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.238+0000 7f8525a0b740 1 --2- 172.21.15.52:0/1759168656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae250c0 0x555b8ae2d760 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.238+0000 7f8525a0b740 1 --2- 172.21.15.52:0/1759168656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555b8ae247a0 0x555b8ae24b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.238+0000 7f8525a0b740 1 -- 172.21.15.52:0/1759168656 >> 172.21.15.52:0/1759168656 conn(0x555b8ae1eed0 msgr2=0x555b8ae1f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:24.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.238+0000 7f8525a0b740 1 -- 172.21.15.52:0/1759168656 shutdown_connections 2024-04-02T14:03:24.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.238+0000 7f8525a0b740 1 -- 172.21.15.52:0/1759168656 wait complete. 2024-04-02T14:03:24.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.238+0000 7f8525a0b740 1 -- start start 2024-04-02T14:03:24.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.238+0000 7f8525a0b740 1 Processor -- start 2024-04-02T14:03:24.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.239+0000 7f8525a0b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555b8ae247a0 0x555b8ae2d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.239+0000 7f8525a0b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae250c0 0x555b8ae2aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.239+0000 7f8525a0b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b8ae2dca0 0x555b8ae2b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.239+0000 7f8525a0b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555b8ae30a90 con 0x555b8ae250c0 2024-04-02T14:03:24.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.239+0000 7f8525a0b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x555b8ad43e80 con 0x555b8ae247a0 2024-04-02T14:03:24.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.239+0000 7f8525a0b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555b8ae30c10 con 0x555b8ae2dca0 2024-04-02T14:03:24.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.239+0000 7f8525a0b740 10 client.0 _ll_register_callbacks cb 0x555b8ad7faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:24.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.239+0000 7f851bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae250c0 0x555b8ae2aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.239+0000 7f851bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae250c0 0x555b8ae2aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57240/0 (socket says 172.21.15.52:57240) 2024-04-02T14:03:24.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.239+0000 7f851bfff640 1 -- 172.21.15.52:0/3341979239 learned_addr learned my addr 172.21.15.52:0/3341979239 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:24.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.239+0000 7f8520ae9640 1 --2- 172.21.15.52:0/3341979239 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555b8ae247a0 0x555b8ae2d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.239+0000 7f851bfff640 1 -- 172.21.15.52:0/3341979239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b8ae2dca0 msgr2=0x555b8ae2b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:24.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.239+0000 7f85212ea640 1 --2- 172.21.15.52:0/3341979239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b8ae2dca0 0x555b8ae2b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.239+0000 7f851bfff640 1 --2- 172.21.15.52:0/3341979239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b8ae2dca0 0x555b8ae2b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.239+0000 7f851bfff640 1 -- 172.21.15.52:0/3341979239 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555b8ae247a0 msgr2=0x555b8ae2d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:24.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.239+0000 7f851bfff640 1 --2- 172.21.15.52:0/3341979239 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555b8ae247a0 0x555b8ae2d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.239+0000 7f851bfff640 1 -- 172.21.15.52:0/3341979239 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b8ab667d0 con 0x555b8ae250c0 2024-04-02T14:03:24.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.239+0000 7f851bfff640 1 --2- 172.21.15.52:0/3341979239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae250c0 0x555b8ae2aa60 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f85100efad0 tx=0x7f85100be100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551beb2b636a9af9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:24.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.239+0000 7f84fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:24.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.240+0000 7f84fbfff640 1 -- 172.21.15.52:0/3341979239 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85100d6070 con 0x555b8ae250c0 2024-04-02T14:03:24.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.240+0000 7f84fbfff640 1 -- 172.21.15.52:0/3341979239 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f85100f2140 con 0x555b8ae250c0 2024-04-02T14:03:24.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.240+0000 7f8525a0b740 1 -- 172.21.15.52:0/3341979239 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f850c005500 con 0x555b8ae250c0 2024-04-02T14:03:24.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.240+0000 7f84fbfff640 1 -- 172.21.15.52:0/3341979239 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f851004b040 con 0x555b8ae250c0 2024-04-02T14:03:24.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.240+0000 7f8525a0b740 10 client.18432 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:24.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.240+0000 7f8525a0b740 1 -- 172.21.15.52:0/3341979239 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555b8ae3cae0 con 0x555b8ae250c0 2024-04-02T14:03:24.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.240+0000 7f84fbfff640 1 -- 172.21.15.52:0/3341979239 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f850c005500 con 0x555b8ae250c0 2024-04-02T14:03:24.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.240+0000 7f84fbfff640 1 -- 172.21.15.52:0/3341979239 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f85100c6a50 con 0x555b8ae250c0 2024-04-02T14:03:24.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.240+0000 7f8525a0b740 10 client.18432 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:24.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.240+0000 7f8525a0b740 10 client.18432 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T14:03:24.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.240+0000 7f8525a0b740 20 client.18432 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:24.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.240+0000 7f8525a0b740 1 --2- 172.21.15.52:0/3341979239 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x555b8ae3def0 0x555b8ae40360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.240+0000 7f8525a0b740 4 client.18432 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:03:24.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.240+0000 7f8525a0b740 1 -- 172.21.15.52:0/3341979239 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b8ad95cf0 con 0x555b8ae3def0 2024-04-02T14:03:24.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.240+0000 7f8520ae9640 1 --2- 172.21.15.52:0/3341979239 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x555b8ae3def0 0x555b8ae40360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.241+0000 7f84fbfff640 10 client.18432 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T14:03:24.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.241+0000 7f8520ae9640 1 --2- 172.21.15.52:0/3341979239 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x555b8ae3def0 0x555b8ae40360 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0d7a775590aff96 server_cookie=4ef804833e01ae77 in_seq=0 out_seq=0 2024-04-02T14:03:24.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.241+0000 7f84fbfff640 1 -- 172.21.15.52:0/3341979239 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555b8ad95cf0 con 0x555b8ae3def0 2024-04-02T14:03:24.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.241+0000 7f84fbfff640 10 client.18432 handle_command_reply: tid=0 2024-04-02T14:03:24.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.242+0000 7f8525a0b740 1 client.18432 shutdown 2024-04-02T14:03:24.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.242+0000 7f8525a0b740 1 -- 172.21.15.52:0/3341979239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae250c0 msgr2=0x555b8ae2aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.242+0000 7f8525a0b740 1 --2- 172.21.15.52:0/3341979239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae250c0 0x555b8ae2aa60 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f85100efad0 tx=0x7f85100be100 comp rx=0 tx=0).stop 2024-04-02T14:03:24.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.242+0000 7f8525a0b740 1 -- 172.21.15.52:0/3341979239 shutdown_connections 2024-04-02T14:03:24.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.242+0000 7f8525a0b740 1 --2- 172.21.15.52:0/3341979239 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x555b8ae3def0 0x555b8ae40360 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.242+0000 7f8525a0b740 1 --2- 172.21.15.52:0/3341979239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b8ae2dca0 0x555b8ae2b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.242+0000 7f84fbfff640 0 client.18432 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T14:03:24.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.242+0000 7f8525a0b740 1 --2- 172.21.15.52:0/3341979239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae250c0 0x555b8ae2aa60 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.242+0000 7f8525a0b740 1 --2- 172.21.15.52:0/3341979239 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555b8ae247a0 0x555b8ae2d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.242+0000 7f8525a0b740 1 -- 172.21.15.52:0/3341979239 >> 172.21.15.52:0/3341979239 conn(0x555b8ae1eed0 msgr2=0x555b8ae20c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:24.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.242+0000 7f8525a0b740 1 -- 172.21.15.52:0/3341979239 shutdown_connections 2024-04-02T14:03:24.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.242+0000 7f8525a0b740 1 -- 172.21.15.52:0/3341979239 wait complete. 2024-04-02T14:03:24.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.242+0000 7f8525a0b740 20 client.18432 trim_cache size 0 max 16384 2024-04-02T14:03:24.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.255+0000 7f8525a0b740 1 Processor -- start 2024-04-02T14:03:24.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.255+0000 7f8525a0b740 1 -- start start 2024-04-02T14:03:24.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.255+0000 7f8525a0b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae247a0 0x555b8ae39980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.255+0000 7f8525a0b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555b8ae250c0 0x555b8ae402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.255+0000 7f8525a0b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b8ae2dca0 0x555b8ae45c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.255+0000 7f8525a0b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f851c1b10c0 con 0x555b8ae247a0 2024-04-02T14:03:24.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.255+0000 7f8525a0b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f851c1b2c00 con 0x555b8ae250c0 2024-04-02T14:03:24.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.255+0000 7f8525a0b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f851c1b1790 con 0x555b8ae2dca0 2024-04-02T14:03:24.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.255+0000 7f8520ae9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae247a0 0x555b8ae39980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.255+0000 7f8520ae9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae247a0 0x555b8ae39980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57252/0 (socket says 172.21.15.52:57252) 2024-04-02T14:03:24.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.255+0000 7f8520ae9640 1 -- 172.21.15.52:0/734543373 learned_addr learned my addr 172.21.15.52:0/734543373 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:24.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.255+0000 7f851bfff640 1 --2- 172.21.15.52:0/734543373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555b8ae250c0 0x555b8ae402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.255+0000 7f8520ae9640 1 -- 172.21.15.52:0/734543373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b8ae2dca0 msgr2=0x555b8ae45c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:24.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.255+0000 7f8520ae9640 1 --2- 172.21.15.52:0/734543373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b8ae2dca0 0x555b8ae45c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.255+0000 7f8520ae9640 1 -- 172.21.15.52:0/734543373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555b8ae250c0 msgr2=0x555b8ae402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:24.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.255+0000 7f8520ae9640 1 --2- 172.21.15.52:0/734543373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555b8ae250c0 0x555b8ae402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.256+0000 7f8520ae9640 1 -- 172.21.15.52:0/734543373 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b8ae2b4d0 con 0x555b8ae247a0 2024-04-02T14:03:24.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.256+0000 7f8520ae9640 1 --2- 172.21.15.52:0/734543373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae247a0 0x555b8ae39980 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x555b8ae24270 tx=0x7f850c009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1335fbac2d9c422 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:24.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.256+0000 7f85027fc640 1 -- 172.21.15.52:0/734543373 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c016040 con 0x555b8ae247a0 2024-04-02T14:03:24.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.256+0000 7f85027fc640 1 -- 172.21.15.52:0/734543373 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f850c02e070 con 0x555b8ae247a0 2024-04-02T14:03:24.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.256+0000 7f85027fc640 1 -- 172.21.15.52:0/734543373 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c011470 con 0x555b8ae247a0 2024-04-02T14:03:24.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.256+0000 7f8525a0b740 1 -- 172.21.15.52:0/734543373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae247a0 msgr2=0x555b8ae39980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.256+0000 7f8525a0b740 1 --2- 172.21.15.52:0/734543373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae247a0 0x555b8ae39980 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x555b8ae24270 tx=0x7f850c009700 comp rx=0 tx=0).stop 2024-04-02T14:03:24.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.256+0000 7f8525a0b740 1 -- 172.21.15.52:0/734543373 shutdown_connections 2024-04-02T14:03:24.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.256+0000 7f8525a0b740 1 --2- 172.21.15.52:0/734543373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b8ae2dca0 0x555b8ae45c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.256+0000 7f8525a0b740 1 --2- 172.21.15.52:0/734543373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555b8ae250c0 0x555b8ae402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.256+0000 7f8525a0b740 1 --2- 172.21.15.52:0/734543373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae247a0 0x555b8ae39980 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.256+0000 7f8525a0b740 1 -- 172.21.15.52:0/734543373 >> 172.21.15.52:0/734543373 conn(0x555b8ae1eed0 msgr2=0x555b8ae29330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:24.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.256+0000 7f8525a0b740 1 -- 172.21.15.52:0/734543373 shutdown_connections 2024-04-02T14:03:24.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.256+0000 7f8525a0b740 1 -- 172.21.15.52:0/734543373 wait complete. 2024-04-02T14:03:24.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.257+0000 7f8525a0b740 1 -- start start 2024-04-02T14:03:24.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.257+0000 7f8525a0b740 1 Processor -- start 2024-04-02T14:03:24.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.257+0000 7f8525a0b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555b8ae247a0 0x555b8ae363d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.257+0000 7f8525a0b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b8ae250c0 0x555b8ae2e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.257+0000 7f8525a0b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae2dca0 0x555b8ae2ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.257+0000 7f8525a0b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f851c1b2530 con 0x555b8ae2dca0 2024-04-02T14:03:24.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.257+0000 7f8525a0b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f851c1b1e60 con 0x555b8ae247a0 2024-04-02T14:03:24.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.257+0000 7f8525a0b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f851c1af0c0 con 0x555b8ae250c0 2024-04-02T14:03:24.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.257+0000 7f8525a0b740 10 client.0 _ll_register_callbacks cb 0x7f85100ba510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:24.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.257+0000 7f85212ea640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae2dca0 0x555b8ae2ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.257+0000 7f85212ea640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae2dca0 0x555b8ae2ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57268/0 (socket says 172.21.15.52:57268) 2024-04-02T14:03:24.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.257+0000 7f85212ea640 1 -- 172.21.15.52:0/3196013026 learned_addr learned my addr 172.21.15.52:0/3196013026 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:24.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.257+0000 7f8520ae9640 1 --2- 172.21.15.52:0/3196013026 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555b8ae247a0 0x555b8ae363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.258+0000 7f851bfff640 1 --2- 172.21.15.52:0/3196013026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b8ae250c0 0x555b8ae2e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.258+0000 7f85212ea640 1 -- 172.21.15.52:0/3196013026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b8ae250c0 msgr2=0x555b8ae2e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:24.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.258+0000 7f85212ea640 1 --2- 172.21.15.52:0/3196013026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b8ae250c0 0x555b8ae2e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.258+0000 7f85212ea640 1 -- 172.21.15.52:0/3196013026 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555b8ae247a0 msgr2=0x555b8ae363d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:24.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.258+0000 7f85212ea640 1 --2- 172.21.15.52:0/3196013026 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555b8ae247a0 0x555b8ae363d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.258+0000 7f85212ea640 1 -- 172.21.15.52:0/3196013026 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b8ae33e80 con 0x555b8ae2dca0 2024-04-02T14:03:24.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.258+0000 7f85212ea640 1 --2- 172.21.15.52:0/3196013026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae2dca0 0x555b8ae2ef10 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f8514008f50 tx=0x7f8514004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b1fade06d5d6ca server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:24.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.258+0000 7f84fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:24.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.258+0000 7f84fbfff640 1 -- 172.21.15.52:0/3196013026 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514029060 con 0x555b8ae2dca0 2024-04-02T14:03:24.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.258+0000 7f8525a0b740 1 -- 172.21.15.52:0/3196013026 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f850c02e070 con 0x555b8ae2dca0 2024-04-02T14:03:24.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.258+0000 7f84fbfff640 1 -- 172.21.15.52:0/3196013026 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f85140245f0 con 0x555b8ae2dca0 2024-04-02T14:03:24.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.258+0000 7f84fbfff640 1 -- 172.21.15.52:0/3196013026 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8514030050 con 0x555b8ae2dca0 2024-04-02T14:03:24.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.258+0000 7f8525a0b740 10 client.18438 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:24.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.258+0000 7f8525a0b740 1 -- 172.21.15.52:0/3196013026 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555b8aea45a0 con 0x555b8ae2dca0 2024-04-02T14:03:24.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.258+0000 7f84fbfff640 1 -- 172.21.15.52:0/3196013026 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f850c02e070 con 0x555b8ae2dca0 2024-04-02T14:03:24.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.259+0000 7f84fbfff640 1 -- 172.21.15.52:0/3196013026 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8514023070 con 0x555b8ae2dca0 2024-04-02T14:03:24.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.259+0000 7f8525a0b740 10 client.18438 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:24.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.259+0000 7f8525a0b740 10 client.18438 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T14:03:24.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.259+0000 7f8525a0b740 20 client.18438 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:24.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.259+0000 7f8525a0b740 1 --2- 172.21.15.52:0/3196013026 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x555b8aea4d20 0x555b8aea7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.259+0000 7f8525a0b740 4 client.18438 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:03:24.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.259+0000 7f8525a0b740 1 -- 172.21.15.52:0/3196013026 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b8ad90040 con 0x555b8aea4d20 2024-04-02T14:03:24.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.259+0000 7f8520ae9640 1 --2- 172.21.15.52:0/3196013026 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x555b8aea4d20 0x555b8aea7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.260+0000 7f84fbfff640 10 client.18438 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T14:03:24.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.260+0000 7f8520ae9640 1 --2- 172.21.15.52:0/3196013026 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x555b8aea4d20 0x555b8aea7140 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d41b9b0d70562476 server_cookie=ea038553724206d0 in_seq=0 out_seq=0 2024-04-02T14:03:24.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.260+0000 7f84fbfff640 1 -- 172.21.15.52:0/3196013026 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b8ad90040 con 0x555b8aea4d20 2024-04-02T14:03:24.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.260+0000 7f84fbfff640 10 client.18438 handle_command_reply: tid=0 2024-04-02T14:03:24.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.260+0000 7f8525a0b740 1 client.18438 shutdown 2024-04-02T14:03:24.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.260+0000 7f8525a0b740 1 -- 172.21.15.52:0/3196013026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae2dca0 msgr2=0x555b8ae2ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.260+0000 7f8525a0b740 1 --2- 172.21.15.52:0/3196013026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae2dca0 0x555b8ae2ef10 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f8514008f50 tx=0x7f8514004b50 comp rx=0 tx=0).stop 2024-04-02T14:03:24.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.260+0000 7f8525a0b740 1 -- 172.21.15.52:0/3196013026 shutdown_connections 2024-04-02T14:03:24.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.260+0000 7f8525a0b740 1 --2- 172.21.15.52:0/3196013026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b8ae2dca0 0x555b8ae2ef10 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.260+0000 7f8525a0b740 1 --2- 172.21.15.52:0/3196013026 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x555b8aea4d20 0x555b8aea7140 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.260+0000 7f8525a0b740 1 --2- 172.21.15.52:0/3196013026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b8ae250c0 0x555b8ae2e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.260+0000 7f8525a0b740 1 --2- 172.21.15.52:0/3196013026 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555b8ae247a0 0x555b8ae363d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.260+0000 7f84fbfff640 0 client.18438 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T14:03:24.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.260+0000 7f8525a0b740 1 -- 172.21.15.52:0/3196013026 >> 172.21.15.52:0/3196013026 conn(0x555b8ae1eed0 msgr2=0x555b8ae1d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:24.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.260+0000 7f8525a0b740 1 -- 172.21.15.52:0/3196013026 shutdown_connections 2024-04-02T14:03:24.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.260+0000 7f8525a0b740 1 -- 172.21.15.52:0/3196013026 wait complete. 2024-04-02T14:03:24.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.260+0000 7f8525a0b740 20 client.18438 trim_cache size 0 max 16384 2024-04-02T14:03:24.275 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:03:24.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.261+0000 7f8522d4d640 1 -- 172.21.15.52:0/3280400160 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f85040a3d20 msgr2=0x7f85040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.261+0000 7f8522d4d640 1 --2- 172.21.15.52:0/3280400160 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f85040a3d20 0x7f85040a61e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8514009fd0 tx=0x7f8514009ee0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.261+0000 7f8522d4d640 1 -- 172.21.15.52:0/3280400160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f851c101370 msgr2=0x7f851c1975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.261+0000 7f8522d4d640 1 --2- 172.21.15.52:0/3280400160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f851c101370 0x7f851c1975e0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f8510009230 tx=0x7f851002cc70 comp rx=0 tx=0).stop 2024-04-02T14:03:24.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.261+0000 7f8522d4d640 1 -- 172.21.15.52:0/3280400160 shutdown_connections 2024-04-02T14:03:24.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.261+0000 7f8522d4d640 1 --2- 172.21.15.52:0/3280400160 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f85040a3d20 0x7f85040a61e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.261+0000 7f8522d4d640 1 --2- 172.21.15.52:0/3280400160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f851c1069c0 0x7f851c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.261+0000 7f8522d4d640 1 --2- 172.21.15.52:0/3280400160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f851c101370 0x7f851c1975e0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.261+0000 7f8522d4d640 1 --2- 172.21.15.52:0/3280400160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f851c1009b0 0x7f851c1970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.261+0000 7f8522d4d640 1 -- 172.21.15.52:0/3280400160 >> 172.21.15.52:0/3280400160 conn(0x7f851c076d80 msgr2=0x7f851c0fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:24.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.261+0000 7f8522d4d640 1 -- 172.21.15.52:0/3280400160 shutdown_connections 2024-04-02T14:03:24.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.261+0000 7f8522d4d640 1 -- 172.21.15.52:0/3280400160 wait complete. 2024-04-02T14:03:24.278 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T14:03:24.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.379+0000 7fedaa83c640 1 -- 172.21.15.52:0/729768001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feda4100c20 msgr2=0x7feda4106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.379+0000 7fedaa83c640 1 --2- 172.21.15.52:0/729768001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feda4100c20 0x7feda4106750 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fed94009900 tx=0x7fed9402bc30 comp rx=0 tx=0).stop 2024-04-02T14:03:24.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.379+0000 7fedaa83c640 1 -- 172.21.15.52:0/729768001 shutdown_connections 2024-04-02T14:03:24.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.379+0000 7fedaa83c640 1 --2- 172.21.15.52:0/729768001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feda4106e80 0x7feda410ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.379+0000 7fedaa83c640 1 --2- 172.21.15.52:0/729768001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feda4100c20 0x7feda4106750 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.379+0000 7fedaa83c640 1 --2- 172.21.15.52:0/729768001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feda4100270 0x7feda4100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.379+0000 7fedaa83c640 1 -- 172.21.15.52:0/729768001 >> 172.21.15.52:0/729768001 conn(0x7feda4076fc0 msgr2=0x7feda40fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:24.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.379+0000 7fedaa83c640 1 -- 172.21.15.52:0/729768001 shutdown_connections 2024-04-02T14:03:24.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.379+0000 7fedaa83c640 1 -- 172.21.15.52:0/729768001 wait complete. 2024-04-02T14:03:24.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.380+0000 7fedaa83c640 1 Processor -- start 2024-04-02T14:03:24.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.380+0000 7fedaa83c640 1 -- start start 2024-04-02T14:03:24.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.380+0000 7fedaa83c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feda4100270 0x7feda41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.380+0000 7fedaa83c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feda4106e80 0x7feda4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.380+0000 7fedaa83c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feda419cca0 0x7feda419d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.380+0000 7fedaa83c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feda4111ee0 con 0x7feda419cca0 2024-04-02T14:03:24.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.380+0000 7fedaa83c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feda4111d60 con 0x7feda4106e80 2024-04-02T14:03:24.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.380+0000 7fedaa83c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feda4112060 con 0x7feda4100270 2024-04-02T14:03:24.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.380+0000 7feda8dd9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feda419cca0 0x7feda419d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.380+0000 7feda8dd9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feda419cca0 0x7feda419d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57282/0 (socket says 172.21.15.52:57282) 2024-04-02T14:03:24.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.380+0000 7feda8dd9640 1 -- 172.21.15.52:0/454346965 learned_addr learned my addr 172.21.15.52:0/454346965 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:24.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.380+0000 7feda3fff640 1 --2- 172.21.15.52:0/454346965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feda4100270 0x7feda41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.380+0000 7feda37fe640 1 --2- 172.21.15.52:0/454346965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feda4106e80 0x7feda4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.381+0000 7feda8dd9640 1 -- 172.21.15.52:0/454346965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feda4100270 msgr2=0x7feda41983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.381+0000 7feda8dd9640 1 --2- 172.21.15.52:0/454346965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feda4100270 0x7feda41983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.381+0000 7feda8dd9640 1 -- 172.21.15.52:0/454346965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feda4106e80 msgr2=0x7feda4198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.381+0000 7feda8dd9640 1 --2- 172.21.15.52:0/454346965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feda4106e80 0x7feda4198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.381+0000 7feda8dd9640 1 -- 172.21.15.52:0/454346965 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda419d790 con 0x7feda419cca0 2024-04-02T14:03:24.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.381+0000 7feda8dd9640 1 --2- 172.21.15.52:0/454346965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feda419cca0 0x7feda419d150 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00cad0 tx=0x7fed9c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:24.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.381+0000 7feda17fa640 1 -- 172.21.15.52:0/454346965 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c00ae20 con 0x7feda419cca0 2024-04-02T14:03:24.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.381+0000 7feda17fa640 1 -- 172.21.15.52:0/454346965 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed9c005020 con 0x7feda419cca0 2024-04-02T14:03:24.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.381+0000 7fedaa83c640 1 -- 172.21.15.52:0/454346965 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda419da80 con 0x7feda419cca0 2024-04-02T14:03:24.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.381+0000 7feda17fa640 1 -- 172.21.15.52:0/454346965 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c00f6f0 con 0x7feda419cca0 2024-04-02T14:03:24.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.381+0000 7fedaa83c640 1 -- 172.21.15.52:0/454346965 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda41a51f0 con 0x7feda419cca0 2024-04-02T14:03:24.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.383+0000 7feda17fa640 1 -- 172.21.15.52:0/454346965 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fed9c0051d0 con 0x7feda419cca0 2024-04-02T14:03:24.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.383+0000 7fedad426740 1 Processor -- start 2024-04-02T14:03:24.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.383+0000 7fedad426740 1 -- start start 2024-04-02T14:03:24.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.383+0000 7fedad426740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2c97a0 0x557fbe2c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.383+0000 7fedad426740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fbe2ca0c0 0x557fbe2d2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.384+0000 7fedad426740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557fbe2d2ca0 0x557fbe2d5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.384+0000 7fedad426740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557fbe22d7b0 con 0x557fbe2c97a0 2024-04-02T14:03:24.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.384+0000 7fedad426740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557fbe1cc570 con 0x557fbe2d2ca0 2024-04-02T14:03:24.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.384+0000 7fedad426740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557fbe229760 con 0x557fbe2ca0c0 2024-04-02T14:03:24.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.384+0000 7feda17fa640 1 --2- 172.21.15.52:0/454346965 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fed8c0a3d20 0x7fed8c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.384+0000 7feda37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2c97a0 0x557fbe2c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.384+0000 7feda37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2c97a0 0x557fbe2c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57288/0 (socket says 172.21.15.52:57288) 2024-04-02T14:03:24.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.384+0000 7feda37fe640 1 -- 172.21.15.52:0/541553869 learned_addr learned my addr 172.21.15.52:0/541553869 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:24.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.384+0000 7feda3fff640 1 --2- 172.21.15.52:0/541553869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fbe2ca0c0 0x557fbe2d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.384+0000 7feda8dd9640 1 --2- 172.21.15.52:0/454346965 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fed8c0a3d20 0x7fed8c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.384+0000 7feda17fa640 1 -- 172.21.15.52:0/454346965 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fed9c0d0b80 con 0x7feda419cca0 2024-04-02T14:03:24.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.384+0000 7feda37fe640 1 -- 172.21.15.52:0/541553869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fbe2ca0c0 msgr2=0x557fbe2d2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:24.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.384+0000 7feda37fe640 1 --2- 172.21.15.52:0/541553869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fbe2ca0c0 0x557fbe2d2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.384+0000 7feda37fe640 1 -- 172.21.15.52:0/541553869 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557fbe2d2ca0 msgr2=0x557fbe2d5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:24.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.384+0000 7feda37fe640 1 --2- 172.21.15.52:0/541553869 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557fbe2d2ca0 0x557fbe2d5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.384+0000 7feda37fe640 1 -- 172.21.15.52:0/541553869 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fbdf413f0 con 0x557fbe2c97a0 2024-04-02T14:03:24.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.384+0000 7feda37fe640 1 --2- 172.21.15.52:0/541553869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2c97a0 0x557fbe2c9b80 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fed94005940 tx=0x7fed94005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55f03e20c768c8f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:24.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.384+0000 7feda8dd9640 1 --2- 172.21.15.52:0/454346965 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fed8c0a3d20 0x7fed8c0a61e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fed9c09c490 tx=0x7fed9c087040 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:24.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.384+0000 7fed8a7fc640 1 -- 172.21.15.52:0/541553869 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94043070 con 0x557fbe2c97a0 2024-04-02T14:03:24.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.384+0000 7fed8a7fc640 1 -- 172.21.15.52:0/541553869 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed9402cb40 con 0x557fbe2c97a0 2024-04-02T14:03:24.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.384+0000 7fed8a7fc640 1 -- 172.21.15.52:0/541553869 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed940046c0 con 0x557fbe2c97a0 2024-04-02T14:03:24.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.385+0000 7fedad426740 1 -- 172.21.15.52:0/541553869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2c97a0 msgr2=0x557fbe2c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.385+0000 7fedad426740 1 --2- 172.21.15.52:0/541553869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2c97a0 0x557fbe2c9b80 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fed94005940 tx=0x7fed94005e80 comp rx=0 tx=0).stop 2024-04-02T14:03:24.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.385+0000 7fedad426740 1 -- 172.21.15.52:0/541553869 shutdown_connections 2024-04-02T14:03:24.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.385+0000 7fedad426740 1 --2- 172.21.15.52:0/541553869 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557fbe2d2ca0 0x557fbe2d5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.385+0000 7fedad426740 1 --2- 172.21.15.52:0/541553869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fbe2ca0c0 0x557fbe2d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.385+0000 7fedad426740 1 --2- 172.21.15.52:0/541553869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2c97a0 0x557fbe2c9b80 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.385+0000 7fedad426740 1 -- 172.21.15.52:0/541553869 >> 172.21.15.52:0/541553869 conn(0x557fbe2c3ed0 msgr2=0x557fbe2c42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:24.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.385+0000 7fedad426740 1 -- 172.21.15.52:0/541553869 shutdown_connections 2024-04-02T14:03:24.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.385+0000 7fedad426740 1 -- 172.21.15.52:0/541553869 wait complete. 2024-04-02T14:03:24.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.385+0000 7fedad426740 1 -- start start 2024-04-02T14:03:24.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.386+0000 7fedad426740 1 Processor -- start 2024-04-02T14:03:24.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.386+0000 7fedad426740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fbe2c97a0 0x557fbe2d2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.386+0000 7fedad426740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557fbe2ca0c0 0x557fbe2cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.386+0000 7fedad426740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2d2ca0 0x557fbe2d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.386+0000 7fedad426740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557fbe2d5a30 con 0x557fbe2d2ca0 2024-04-02T14:03:24.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.386+0000 7fedad426740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557fbe1e8e80 con 0x557fbe2ca0c0 2024-04-02T14:03:24.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.386+0000 7fedad426740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557fbe2d5bb0 con 0x557fbe2c97a0 2024-04-02T14:03:24.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.386+0000 7fedad426740 10 client.0 _ll_register_callbacks cb 0x557fbe224aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:24.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.386+0000 7feda37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2d2ca0 0x557fbe2d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.386+0000 7feda37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2d2ca0 0x557fbe2d01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57294/0 (socket says 172.21.15.52:57294) 2024-04-02T14:03:24.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.386+0000 7feda37fe640 1 -- 172.21.15.52:0/1919174371 learned_addr learned my addr 172.21.15.52:0/1919174371 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:24.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.386+0000 7feda3fff640 1 --2- 172.21.15.52:0/1919174371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557fbe2ca0c0 0x557fbe2cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.386+0000 7feda37fe640 1 --2- 172.21.15.52:0/1919174371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fbe2c97a0 0x557fbe2d2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.386+0000 7feda3fff640 1 -- 172.21.15.52:0/1919174371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fbe2c97a0 msgr2=0x557fbe2d2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:24.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.386+0000 7feda3fff640 1 --2- 172.21.15.52:0/1919174371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fbe2c97a0 0x557fbe2d2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.386+0000 7feda3fff640 1 -- 172.21.15.52:0/1919174371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2d2ca0 msgr2=0x557fbe2d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:24.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.386+0000 7feda3fff640 1 --2- 172.21.15.52:0/1919174371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2d2ca0 0x557fbe2d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.386+0000 7feda3fff640 1 -- 172.21.15.52:0/1919174371 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fbe00b7d0 con 0x557fbe2ca0c0 2024-04-02T14:03:24.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.386+0000 7feda37fe640 1 --2- 172.21.15.52:0/1919174371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2d2ca0 0x557fbe2d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:03:24.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.387+0000 7feda3fff640 1 --2- 172.21.15.52:0/1919174371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557fbe2ca0c0 0x557fbe2cfab0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fed9800baa0 tx=0x7fed9800bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b76fc58d386da8d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:24.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.387+0000 7fed837fe640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-02T14:03:24.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.387+0000 7fed837fe640 1 -- 172.21.15.52:0/1919174371 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98003df0 con 0x557fbe2ca0c0 2024-04-02T14:03:24.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.387+0000 7fedad426740 1 -- 172.21.15.52:0/1919174371 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed9402cb40 con 0x557fbe2ca0c0 2024-04-02T14:03:24.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.387+0000 7fed837fe640 1 -- 172.21.15.52:0/1919174371 <== mon.2 v2:172.21.15.88:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fed98005020 con 0x557fbe2ca0c0 2024-04-02T14:03:24.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.387+0000 7fed837fe640 1 -- 172.21.15.52:0/1919174371 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed98024050 con 0x557fbe2ca0c0 2024-04-02T14:03:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.387+0000 7fedad426740 10 client.27422 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.387+0000 7fedad426740 1 -- 172.21.15.52:0/1919174371 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557fbe2e1b70 con 0x557fbe2ca0c0 2024-04-02T14:03:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.387+0000 7fed837fe640 1 -- 172.21.15.52:0/1919174371 <== mon.2 v2:172.21.15.88:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed9402cb40 con 0x557fbe2ca0c0 2024-04-02T14:03:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.387+0000 7fed837fe640 1 -- 172.21.15.52:0/1919174371 <== mon.2 v2:172.21.15.88:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fed9801b470 con 0x557fbe2ca0c0 2024-04-02T14:03:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.387+0000 7fedad426740 10 client.27422 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.387+0000 7fedad426740 10 client.27422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:03:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.387+0000 7fedad426740 20 client.27422 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.387+0000 7fedad426740 1 --2- 172.21.15.52:0/1919174371 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557fbe2e2fd0 0x557fbe2e53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.387+0000 7fedad426740 4 client.27422 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:03:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.387+0000 7fedad426740 1 -- 172.21.15.52:0/1919174371 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fbe23acf0 con 0x557fbe2e2fd0 2024-04-02T14:03:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.388+0000 7feda37fe640 1 --2- 172.21.15.52:0/1919174371 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557fbe2e2fd0 0x557fbe2e53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.388+0000 7fed837fe640 10 client.27422 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:03:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.388+0000 7feda37fe640 1 --2- 172.21.15.52:0/1919174371 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557fbe2e2fd0 0x557fbe2e53f0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c82819f861082b0 server_cookie=3fd067ce975db398 in_seq=0 out_seq=0 2024-04-02T14:03:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.389+0000 7fed837fe640 1 -- 172.21.15.52:0/1919174371 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557fbe23acf0 con 0x557fbe2e2fd0 2024-04-02T14:03:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.389+0000 7fed837fe640 10 client.27422 handle_command_reply: tid=0 2024-04-02T14:03:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.389+0000 7fedad426740 1 client.27422 shutdown 2024-04-02T14:03:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.389+0000 7fedad426740 1 -- 172.21.15.52:0/1919174371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557fbe2ca0c0 msgr2=0x557fbe2cfab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.389+0000 7fedad426740 1 --2- 172.21.15.52:0/1919174371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557fbe2ca0c0 0x557fbe2cfab0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fed9800baa0 tx=0x7fed9800bf60 comp rx=0 tx=0).stop 2024-04-02T14:03:24.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.389+0000 7fedad426740 1 -- 172.21.15.52:0/1919174371 shutdown_connections 2024-04-02T14:03:24.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.389+0000 7fedad426740 1 --2- 172.21.15.52:0/1919174371 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557fbe2e2fd0 0x557fbe2e53f0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.389+0000 7fedad426740 1 --2- 172.21.15.52:0/1919174371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2d2ca0 0x557fbe2d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.389+0000 7fedad426740 1 --2- 172.21.15.52:0/1919174371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557fbe2ca0c0 0x557fbe2cfab0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.389+0000 7fedad426740 1 --2- 172.21.15.52:0/1919174371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fbe2c97a0 0x557fbe2d2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.389+0000 7fed837fe640 0 client.27422 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:03:24.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.389+0000 7fedad426740 1 -- 172.21.15.52:0/1919174371 >> 172.21.15.52:0/1919174371 conn(0x557fbe2c3ed0 msgr2=0x557fbe2c5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:24.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.389+0000 7fedad426740 1 -- 172.21.15.52:0/1919174371 shutdown_connections 2024-04-02T14:03:24.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.389+0000 7fedad426740 1 -- 172.21.15.52:0/1919174371 wait complete. 2024-04-02T14:03:24.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.389+0000 7fedad426740 20 client.27422 trim_cache size 0 max 16384 2024-04-02T14:03:24.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.402+0000 7fedad426740 1 Processor -- start 2024-04-02T14:03:24.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.402+0000 7fedad426740 1 -- start start 2024-04-02T14:03:24.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.402+0000 7fedad426740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2c97a0 0x557fbe2d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.402+0000 7fedad426740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fbe2ca0c0 0x557fbe2de8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.402+0000 7fedad426740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557fbe2d2ca0 0x557fbe2eaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.402+0000 7fedad426740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feda41aee40 con 0x557fbe2c97a0 2024-04-02T14:03:24.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.402+0000 7fedad426740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feda41b0980 con 0x557fbe2d2ca0 2024-04-02T14:03:24.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.402+0000 7fedad426740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feda41af510 con 0x557fbe2ca0c0 2024-04-02T14:03:24.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.403+0000 7feda37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2c97a0 0x557fbe2d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.403+0000 7feda37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2c97a0 0x557fbe2d01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57296/0 (socket says 172.21.15.52:57296) 2024-04-02T14:03:24.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.403+0000 7feda37fe640 1 -- 172.21.15.52:0/3037128799 learned_addr learned my addr 172.21.15.52:0/3037128799 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:24.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.403+0000 7feda37fe640 1 -- 172.21.15.52:0/3037128799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fbe2ca0c0 msgr2=0x557fbe2de8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:24.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.403+0000 7feda37fe640 1 --2- 172.21.15.52:0/3037128799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fbe2ca0c0 0x557fbe2de8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.403+0000 7feda37fe640 1 -- 172.21.15.52:0/3037128799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557fbe2d2ca0 msgr2=0x557fbe2eaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:24.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.403+0000 7feda37fe640 1 --2- 172.21.15.52:0/3037128799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557fbe2d2ca0 0x557fbe2eaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.403+0000 7feda37fe640 1 -- 172.21.15.52:0/3037128799 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fbe2c9ef0 con 0x557fbe2c97a0 2024-04-02T14:03:24.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.403+0000 7feda37fe640 1 --2- 172.21.15.52:0/3037128799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2c97a0 0x557fbe2d01d0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fed94005940 tx=0x7fed94005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431a1785bf956884 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:24.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.403+0000 7fed89ffb640 1 -- 172.21.15.52:0/3037128799 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94043040 con 0x557fbe2c97a0 2024-04-02T14:03:24.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.403+0000 7fed89ffb640 1 -- 172.21.15.52:0/3037128799 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed9402ca60 con 0x557fbe2c97a0 2024-04-02T14:03:24.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.403+0000 7fed89ffb640 1 -- 172.21.15.52:0/3037128799 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94004920 con 0x557fbe2c97a0 2024-04-02T14:03:24.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.404+0000 7fedad426740 1 -- 172.21.15.52:0/3037128799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2c97a0 msgr2=0x557fbe2d01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.404+0000 7fedad426740 1 --2- 172.21.15.52:0/3037128799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2c97a0 0x557fbe2d01d0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fed94005940 tx=0x7fed94005d50 comp rx=0 tx=0).stop 2024-04-02T14:03:24.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.404+0000 7fedad426740 1 -- 172.21.15.52:0/3037128799 shutdown_connections 2024-04-02T14:03:24.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.404+0000 7fedad426740 1 --2- 172.21.15.52:0/3037128799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557fbe2d2ca0 0x557fbe2eaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.404+0000 7fedad426740 1 --2- 172.21.15.52:0/3037128799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fbe2ca0c0 0x557fbe2de8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.404+0000 7fedad426740 1 --2- 172.21.15.52:0/3037128799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2c97a0 0x557fbe2d01d0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.404+0000 7fedad426740 1 -- 172.21.15.52:0/3037128799 >> 172.21.15.52:0/3037128799 conn(0x557fbe2c3ed0 msgr2=0x557fbe2c42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:24.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.404+0000 7fedad426740 1 -- 172.21.15.52:0/3037128799 shutdown_connections 2024-04-02T14:03:24.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.404+0000 7fedad426740 1 -- 172.21.15.52:0/3037128799 wait complete. 2024-04-02T14:03:24.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.404+0000 7fedad426740 1 -- start start 2024-04-02T14:03:24.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.404+0000 7fedad426740 1 Processor -- start 2024-04-02T14:03:24.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.404+0000 7fedad426740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2c97a0 0x557fbe2d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.405+0000 7fedad426740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fbe2ca0c0 0x557fbe2d6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.405+0000 7fedad426740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557fbe2d2ca0 0x557fbe2d6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.405+0000 7fedad426740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feda41b02b0 con 0x557fbe2c97a0 2024-04-02T14:03:24.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.405+0000 7fedad426740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feda41afbe0 con 0x557fbe2d2ca0 2024-04-02T14:03:24.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.405+0000 7fedad426740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feda41ace40 con 0x557fbe2ca0c0 2024-04-02T14:03:24.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.405+0000 7fedad426740 10 client.0 _ll_register_callbacks cb 0x7fed98004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:24.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.405+0000 7feda37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2c97a0 0x557fbe2d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.405+0000 7feda37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2c97a0 0x557fbe2d5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57310/0 (socket says 172.21.15.52:57310) 2024-04-02T14:03:24.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.405+0000 7feda37fe640 1 -- 172.21.15.52:0/4115997766 learned_addr learned my addr 172.21.15.52:0/4115997766 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:24.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.405+0000 7feda37fe640 1 --2- 172.21.15.52:0/4115997766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557fbe2d2ca0 0x557fbe2d6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.405+0000 7feda3fff640 1 --2- 172.21.15.52:0/4115997766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fbe2ca0c0 0x557fbe2d6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.405+0000 7feda37fe640 1 -- 172.21.15.52:0/4115997766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fbe2ca0c0 msgr2=0x557fbe2d6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:24.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.405+0000 7feda37fe640 1 --2- 172.21.15.52:0/4115997766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fbe2ca0c0 0x557fbe2d6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.405+0000 7feda37fe640 1 -- 172.21.15.52:0/4115997766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557fbe2d2ca0 msgr2=0x557fbe2d6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:24.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.405+0000 7feda37fe640 1 --2- 172.21.15.52:0/4115997766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557fbe2d2ca0 0x557fbe2d6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.405+0000 7feda37fe640 1 -- 172.21.15.52:0/4115997766 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fbe3476d0 con 0x557fbe2c97a0 2024-04-02T14:03:24.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.405+0000 7feda37fe640 1 --2- 172.21.15.52:0/4115997766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2c97a0 0x557fbe2d5dc0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fed94005ec0 tx=0x7fed94005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924a11e7b0950504 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:24.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.405+0000 7fed837fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:24.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.405+0000 7fed837fe640 1 -- 172.21.15.52:0/4115997766 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94043070 con 0x557fbe2c97a0 2024-04-02T14:03:24.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.405+0000 7fed837fe640 1 -- 172.21.15.52:0/4115997766 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fed94004230 con 0x557fbe2c97a0 2024-04-02T14:03:24.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.405+0000 7fedad426740 1 -- 172.21.15.52:0/4115997766 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed9402ca60 con 0x557fbe2c97a0 2024-04-02T14:03:24.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.405+0000 7fed837fe640 1 -- 172.21.15.52:0/4115997766 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed94052e40 con 0x557fbe2c97a0 2024-04-02T14:03:24.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.405+0000 7fedad426740 10 client.18456 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:24.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.405+0000 7fedad426740 1 -- 172.21.15.52:0/4115997766 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557fbe3489b0 con 0x557fbe2c97a0 2024-04-02T14:03:24.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.406+0000 7fed837fe640 1 -- 172.21.15.52:0/4115997766 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed9402ca60 con 0x557fbe2c97a0 2024-04-02T14:03:24.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.406+0000 7fed837fe640 1 -- 172.21.15.52:0/4115997766 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fed94004be0 con 0x557fbe2c97a0 2024-04-02T14:03:24.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.406+0000 7fedad426740 10 client.18456 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:24.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.406+0000 7fedad426740 10 client.18456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:03:24.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.406+0000 7fedad426740 20 client.18456 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:24.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.406+0000 7fedad426740 1 --2- 172.21.15.52:0/4115997766 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557fbe348c20 0x557fbe34b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:24.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.406+0000 7fedad426740 4 client.18456 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:03:24.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.406+0000 7fedad426740 1 -- 172.21.15.52:0/4115997766 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557fbe235040 con 0x557fbe348c20 2024-04-02T14:03:24.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.406+0000 7feda3fff640 1 --2- 172.21.15.52:0/4115997766 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557fbe348c20 0x557fbe34b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:24.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.406+0000 7fed837fe640 10 client.18456 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:03:24.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.406+0000 7feda3fff640 1 --2- 172.21.15.52:0/4115997766 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557fbe348c20 0x557fbe34b040 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8223304f9fe6f683 server_cookie=834f9f48b38876b1 in_seq=0 out_seq=0 2024-04-02T14:03:24.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.407+0000 7fed837fe640 1 -- 172.21.15.52:0/4115997766 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557fbe235040 con 0x557fbe348c20 2024-04-02T14:03:24.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.407+0000 7fed837fe640 10 client.18456 handle_command_reply: tid=0 2024-04-02T14:03:24.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.407+0000 7fedad426740 1 client.18456 shutdown 2024-04-02T14:03:24.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.407+0000 7fedad426740 1 -- 172.21.15.52:0/4115997766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2c97a0 msgr2=0x557fbe2d5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.407+0000 7fedad426740 1 --2- 172.21.15.52:0/4115997766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2c97a0 0x557fbe2d5dc0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fed94005ec0 tx=0x7fed94005a20 comp rx=0 tx=0).stop 2024-04-02T14:03:24.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.407+0000 7fedad426740 1 -- 172.21.15.52:0/4115997766 shutdown_connections 2024-04-02T14:03:24.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.407+0000 7fedad426740 1 --2- 172.21.15.52:0/4115997766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557fbe2d2ca0 0x557fbe2d6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.407+0000 7fedad426740 1 --2- 172.21.15.52:0/4115997766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fbe2ca0c0 0x557fbe2d6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.407+0000 7fedad426740 1 --2- 172.21.15.52:0/4115997766 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x557fbe348c20 0x557fbe34b040 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.407+0000 7fedad426740 1 --2- 172.21.15.52:0/4115997766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557fbe2c97a0 0x557fbe2d5dc0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.407+0000 7fed837fe640 0 client.18456 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:03:24.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.407+0000 7fedad426740 1 -- 172.21.15.52:0/4115997766 >> 172.21.15.52:0/4115997766 conn(0x557fbe2c3ed0 msgr2=0x557fbe2ce380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:24.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.407+0000 7fedad426740 1 -- 172.21.15.52:0/4115997766 shutdown_connections 2024-04-02T14:03:24.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.407+0000 7fedad426740 1 -- 172.21.15.52:0/4115997766 wait complete. 2024-04-02T14:03:24.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.407+0000 7fedad426740 20 client.18456 trim_cache size 0 max 16384 2024-04-02T14:03:24.422 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:03:24.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.407+0000 7fedaa83c640 1 -- 172.21.15.52:0/454346965 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fed8c0a3d20 msgr2=0x7fed8c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.407+0000 7fedaa83c640 1 --2- 172.21.15.52:0/454346965 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fed8c0a3d20 0x7fed8c0a61e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fed9c09c490 tx=0x7fed9c087040 comp rx=0 tx=0).stop 2024-04-02T14:03:24.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.407+0000 7fedaa83c640 1 -- 172.21.15.52:0/454346965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feda419cca0 msgr2=0x7feda419d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:24.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.407+0000 7fedaa83c640 1 --2- 172.21.15.52:0/454346965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feda419cca0 0x7feda419d150 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00cad0 tx=0x7fed9c00cf90 comp rx=0 tx=0).stop 2024-04-02T14:03:24.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.408+0000 7fedaa83c640 1 -- 172.21.15.52:0/454346965 shutdown_connections 2024-04-02T14:03:24.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.408+0000 7fedaa83c640 1 --2- 172.21.15.52:0/454346965 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fed8c0a3d20 0x7fed8c0a61e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.408+0000 7fedaa83c640 1 --2- 172.21.15.52:0/454346965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feda419cca0 0x7feda419d150 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.408+0000 7fedaa83c640 1 --2- 172.21.15.52:0/454346965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feda4106e80 0x7feda4198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.408+0000 7fedaa83c640 1 --2- 172.21.15.52:0/454346965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feda4100270 0x7feda41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:24.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.408+0000 7fedaa83c640 1 -- 172.21.15.52:0/454346965 >> 172.21.15.52:0/454346965 conn(0x7feda4076fc0 msgr2=0x7feda4105220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:24.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.408+0000 7fedaa83c640 1 -- 172.21.15.52:0/454346965 shutdown_connections 2024-04-02T14:03:24.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:24.408+0000 7fedaa83c640 1 -- 172.21.15.52:0/454346965 wait complete. 2024-04-02T14:03:24.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:24 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/83273045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:03:24.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:24 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/83273045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:03:24.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:24 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/83273045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:03:25.426 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T14:03:25.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:03:25.427 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T14:03:25.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.544+0000 7fcbcee42640 1 -- 172.21.15.52:0/4220316616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbc8106980 msgr2=0x7fcbc8106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:25.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.544+0000 7fcbcee42640 1 --2- 172.21.15.52:0/4220316616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbc8106980 0x7fcbc8106d60 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0009cd0 tx=0x7fcbc002c0f0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.544+0000 7fcbcee42640 1 -- 172.21.15.52:0/4220316616 shutdown_connections 2024-04-02T14:03:25.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.544+0000 7fcbcee42640 1 --2- 172.21.15.52:0/4220316616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcbc8101330 0x7fcbc810d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.544+0000 7fcbcee42640 1 --2- 172.21.15.52:0/4220316616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbc8100970 0x7fcbc8100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.544+0000 7fcbcee42640 1 --2- 172.21.15.52:0/4220316616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbc8106980 0x7fcbc8106d60 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.544+0000 7fcbcee42640 1 -- 172.21.15.52:0/4220316616 >> 172.21.15.52:0/4220316616 conn(0x7fcbc8077b00 msgr2=0x7fcbc8077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:25.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.544+0000 7fcbcee42640 1 -- 172.21.15.52:0/4220316616 shutdown_connections 2024-04-02T14:03:25.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.544+0000 7fcbcee42640 1 -- 172.21.15.52:0/4220316616 wait complete. 2024-04-02T14:03:25.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.544+0000 7fcbcee42640 1 Processor -- start 2024-04-02T14:03:25.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.544+0000 7fcbcee42640 1 -- start start 2024-04-02T14:03:25.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.545+0000 7fcbcee42640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcbc8100970 0x7fcbc8198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.545+0000 7fcbcee42640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbc8101330 0x7fcbc8198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.545+0000 7fcbcee42640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbc8106980 0x7fcbc819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.545+0000 7fcbcee42640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc8111f50 con 0x7fcbc8101330 2024-04-02T14:03:25.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.545+0000 7fcbcee42640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc8111dd0 con 0x7fcbc8106980 2024-04-02T14:03:25.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.545+0000 7fcbcee42640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81120d0 con 0x7fcbc8100970 2024-04-02T14:03:25.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.545+0000 7fcbc7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbc8101330 0x7fcbc8198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.545+0000 7fcbc7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbc8101330 0x7fcbc8198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57334/0 (socket says 172.21.15.52:57334) 2024-04-02T14:03:25.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.545+0000 7fcbc7fff640 1 -- 172.21.15.52:0/805974310 learned_addr learned my addr 172.21.15.52:0/805974310 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:25.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.545+0000 7fcbcd3df640 1 --2- 172.21.15.52:0/805974310 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbc8106980 0x7fcbc819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.545+0000 7fcbccbde640 1 --2- 172.21.15.52:0/805974310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcbc8100970 0x7fcbc8198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.545+0000 7fcbc7fff640 1 -- 172.21.15.52:0/805974310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcbc8100970 msgr2=0x7fcbc8198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:25.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.545+0000 7fcbc7fff640 1 --2- 172.21.15.52:0/805974310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcbc8100970 0x7fcbc8198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.545+0000 7fcbc7fff640 1 -- 172.21.15.52:0/805974310 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbc8106980 msgr2=0x7fcbc819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:25.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.545+0000 7fcbc7fff640 1 --2- 172.21.15.52:0/805974310 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbc8106980 0x7fcbc819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.545+0000 7fcbc7fff640 1 -- 172.21.15.52:0/805974310 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc819d310 con 0x7fcbc8101330 2024-04-02T14:03:25.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.545+0000 7fcbcd3df640 1 --2- 172.21.15.52:0/805974310 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbc8106980 0x7fcbc819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:03:25.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.545+0000 7fcbc7fff640 1 --2- 172.21.15.52:0/805974310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbc8101330 0x7fcbc8198860 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc00e9f0 tx=0x7fcbbc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:25.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.545+0000 7fcbc5ffb640 1 -- 172.21.15.52:0/805974310 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc00cd80 con 0x7fcbc8101330 2024-04-02T14:03:25.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.545+0000 7fcbcee42640 1 -- 172.21.15.52:0/805974310 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc819d600 con 0x7fcbc8101330 2024-04-02T14:03:25.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.545+0000 7fcbc5ffb640 1 -- 172.21.15.52:0/805974310 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbbc010910 con 0x7fcbc8101330 2024-04-02T14:03:25.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.545+0000 7fcbc5ffb640 1 -- 172.21.15.52:0/805974310 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc018990 con 0x7fcbc8101330 2024-04-02T14:03:25.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.546+0000 7fcbcee42640 1 -- 172.21.15.52:0/805974310 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc81a4e90 con 0x7fcbc8101330 2024-04-02T14:03:25.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.548+0000 7fcbc5ffb640 1 -- 172.21.15.52:0/805974310 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcbbc010ac0 con 0x7fcbc8101330 2024-04-02T14:03:25.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.548+0000 7fcbd1a1f740 1 Processor -- start 2024-04-02T14:03:25.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.548+0000 7fcbd1a1f740 1 -- start start 2024-04-02T14:03:25.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.548+0000 7fcbd1a1f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55853f090ff0 0x55853f0913d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.548+0000 7fcbd1a1f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f091910 0x55853f09e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.548+0000 7fcbc5ffb640 1 --2- 172.21.15.52:0/805974310 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fcbb00a3d20 0x7fcbb00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.548+0000 7fcbd1a1f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55853f09ebd0 0x55853f0a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.548+0000 7fcbd1a1f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55853eff5770 con 0x55853f091910 2024-04-02T14:03:25.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.548+0000 7fcbd1a1f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55853ef9d960 con 0x55853f090ff0 2024-04-02T14:03:25.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.548+0000 7fcbd1a1f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55853eff18c0 con 0x55853f09ebd0 2024-04-02T14:03:25.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.549+0000 7fcbccbde640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f091910 0x55853f09e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.549+0000 7fcbc7fff640 1 --2- 172.21.15.52:0/805974310 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fcbb00a3d20 0x7fcbb00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.549+0000 7fcbccbde640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f091910 0x55853f09e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57346/0 (socket says 172.21.15.52:57346) 2024-04-02T14:03:25.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.549+0000 7fcbccbde640 1 -- 172.21.15.52:0/957192179 learned_addr learned my addr 172.21.15.52:0/957192179 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:25.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.549+0000 7fcbcd3df640 1 --2- 172.21.15.52:0/957192179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55853f09ebd0 0x55853f0a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.549+0000 7fcbcd3df640 1 --2- 172.21.15.52:0/957192179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55853f090ff0 0x55853f0913d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.549+0000 7fcbc5ffb640 1 -- 172.21.15.52:0/805974310 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fcbbc0d1070 con 0x7fcbc8101330 2024-04-02T14:03:25.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.549+0000 7fcbccbde640 1 -- 172.21.15.52:0/957192179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55853f09ebd0 msgr2=0x55853f0a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:25.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.549+0000 7fcbccbde640 1 --2- 172.21.15.52:0/957192179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55853f09ebd0 0x55853f0a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.549+0000 7fcbccbde640 1 -- 172.21.15.52:0/957192179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55853f090ff0 msgr2=0x55853f0913d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:25.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.549+0000 7fcbccbde640 1 --2- 172.21.15.52:0/957192179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55853f090ff0 0x55853f0913d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.549+0000 7fcbccbde640 1 -- 172.21.15.52:0/957192179 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55853edd17e0 con 0x55853f091910 2024-04-02T14:03:25.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.549+0000 7fcbc7fff640 1 --2- 172.21.15.52:0/805974310 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fcbb00a3d20 0x7fcbb00a61e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc09c920 tx=0x7fcbbc018420 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:25.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.549+0000 7fcbccbde640 1 --2- 172.21.15.52:0/957192179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f091910 0x55853f09e690 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0037040 tx=0x7fcbc0009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54d5e589fa7e9f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:25.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.549+0000 7fcbaeffd640 1 -- 172.21.15.52:0/957192179 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0043020 con 0x55853f091910 2024-04-02T14:03:25.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.549+0000 7fcbaeffd640 1 -- 172.21.15.52:0/957192179 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbc0032c90 con 0x55853f091910 2024-04-02T14:03:25.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.549+0000 7fcbaeffd640 1 -- 172.21.15.52:0/957192179 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0039040 con 0x55853f091910 2024-04-02T14:03:25.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.550+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/957192179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f091910 msgr2=0x55853f09e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:25.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.550+0000 7fcbd1a1f740 1 --2- 172.21.15.52:0/957192179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f091910 0x55853f09e690 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0037040 tx=0x7fcbc0009910 comp rx=0 tx=0).stop 2024-04-02T14:03:25.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.550+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/957192179 shutdown_connections 2024-04-02T14:03:25.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.550+0000 7fcbd1a1f740 1 --2- 172.21.15.52:0/957192179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55853f09ebd0 0x55853f0a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.550+0000 7fcbd1a1f740 1 --2- 172.21.15.52:0/957192179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f091910 0x55853f09e690 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.550+0000 7fcbd1a1f740 1 --2- 172.21.15.52:0/957192179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55853f090ff0 0x55853f0913d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.550+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/957192179 >> 172.21.15.52:0/957192179 conn(0x55853f08b9f0 msgr2=0x55853f08bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:25.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.550+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/957192179 shutdown_connections 2024-04-02T14:03:25.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.550+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/957192179 wait complete. 2024-04-02T14:03:25.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.550+0000 7fcbd1a1f740 1 -- start start 2024-04-02T14:03:25.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.551+0000 7fcbd1a1f740 1 Processor -- start 2024-04-02T14:03:25.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.551+0000 7fcbd1a1f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f090ff0 0x55853f09bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.551+0000 7fcbd1a1f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55853f091910 0x55853f09c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.551+0000 7fcbd1a1f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55853f09ebd0 0x55853f09c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.551+0000 7fcbd1a1f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55853f0a1930 con 0x55853f090ff0 2024-04-02T14:03:25.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.551+0000 7fcbd1a1f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55853efb0d20 con 0x55853f091910 2024-04-02T14:03:25.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.551+0000 7fcbd1a1f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55853f0a1ab0 con 0x55853f09ebd0 2024-04-02T14:03:25.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.551+0000 7fcbd1a1f740 10 client.0 _ll_register_callbacks cb 0x55853ef7da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:25.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.551+0000 7fcbcd3df640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f090ff0 0x55853f09bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.551+0000 7fcbcd3df640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f090ff0 0x55853f09bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57358/0 (socket says 172.21.15.52:57358) 2024-04-02T14:03:25.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.551+0000 7fcbcd3df640 1 -- 172.21.15.52:0/4279917158 learned_addr learned my addr 172.21.15.52:0/4279917158 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:25.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.551+0000 7fcbccbde640 1 --2- 172.21.15.52:0/4279917158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55853f091910 0x55853f09c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.551+0000 7fcbcd3df640 1 --2- 172.21.15.52:0/4279917158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55853f09ebd0 0x55853f09c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.551+0000 7fcbcd3df640 1 -- 172.21.15.52:0/4279917158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55853f09ebd0 msgr2=0x55853f09c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:25.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.551+0000 7fcbcd3df640 1 --2- 172.21.15.52:0/4279917158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55853f09ebd0 0x55853f09c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.551+0000 7fcbcd3df640 1 -- 172.21.15.52:0/4279917158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55853f091910 msgr2=0x55853f09c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:25.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.551+0000 7fcbcd3df640 1 --2- 172.21.15.52:0/4279917158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55853f091910 0x55853f09c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.551+0000 7fcbcd3df640 1 -- 172.21.15.52:0/4279917158 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55853ee31f70 con 0x55853f090ff0 2024-04-02T14:03:25.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.551+0000 7fcbccbde640 1 --2- 172.21.15.52:0/4279917158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55853f091910 0x55853f09c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:03:25.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.552+0000 7fcbcd3df640 1 --2- 172.21.15.52:0/4279917158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f090ff0 0x55853f09bc00 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8014ea0 tx=0x7fcbb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4bae5f521b64284 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:25.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.552+0000 7fcba7fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:25.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.552+0000 7fcba7fff640 1 -- 172.21.15.52:0/4279917158 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb80130d0 con 0x55853f090ff0 2024-04-02T14:03:25.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.552+0000 7fcba7fff640 1 -- 172.21.15.52:0/4279917158 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fcbb80040f0 con 0x55853f090ff0 2024-04-02T14:03:25.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.552+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/4279917158 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbc0032c90 con 0x55853f090ff0 2024-04-02T14:03:25.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.552+0000 7fcba7fff640 1 -- 172.21.15.52:0/4279917158 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcbb8026050 con 0x55853f090ff0 2024-04-02T14:03:25.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.552+0000 7fcbd1a1f740 10 client.18468 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:25.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.552+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/4279917158 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55853ecc9170 con 0x55853f090ff0 2024-04-02T14:03:25.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.552+0000 7fcba7fff640 1 -- 172.21.15.52:0/4279917158 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbc0032c90 con 0x55853f090ff0 2024-04-02T14:03:25.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.552+0000 7fcba7fff640 1 -- 172.21.15.52:0/4279917158 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fcbb800f580 con 0x55853f090ff0 2024-04-02T14:03:25.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.552+0000 7fcbd1a1f740 10 client.18468 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:25.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.552+0000 7fcbd1a1f740 10 client.18468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:03:25.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.552+0000 7fcbd1a1f740 20 client.18468 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:25.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.552+0000 7fcbd1a1f740 1 --2- 172.21.15.52:0/4279917158 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55853f0af030 0x55853f0b1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.552+0000 7fcbd1a1f740 4 client.18468 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:03:25.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.552+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/4279917158 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55853f002cb0 con 0x55853f0af030 2024-04-02T14:03:25.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.553+0000 7fcbccbde640 1 --2- 172.21.15.52:0/4279917158 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55853f0af030 0x55853f0b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.553+0000 7fcba7fff640 10 client.18468 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:03:25.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.553+0000 7fcbccbde640 1 --2- 172.21.15.52:0/4279917158 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55853f0af030 0x55853f0b1470 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6231d58a62af04db server_cookie=90113c0876b61347 in_seq=0 out_seq=0 2024-04-02T14:03:25.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.554+0000 7fcba7fff640 1 -- 172.21.15.52:0/4279917158 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55853f002cb0 con 0x55853f0af030 2024-04-02T14:03:25.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.554+0000 7fcba7fff640 10 client.18468 handle_command_reply: tid=0 2024-04-02T14:03:25.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.554+0000 7fcbd1a1f740 1 client.18468 shutdown 2024-04-02T14:03:25.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.554+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/4279917158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f090ff0 msgr2=0x55853f09bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:25.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.554+0000 7fcbd1a1f740 1 --2- 172.21.15.52:0/4279917158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f090ff0 0x55853f09bc00 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8014ea0 tx=0x7fcbb80046c0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.554+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/4279917158 shutdown_connections 2024-04-02T14:03:25.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.554+0000 7fcbd1a1f740 1 --2- 172.21.15.52:0/4279917158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55853f09ebd0 0x55853f09c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.554+0000 7fcbd1a1f740 1 --2- 172.21.15.52:0/4279917158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55853f091910 0x55853f09c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.554+0000 7fcbd1a1f740 1 --2- 172.21.15.52:0/4279917158 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55853f0af030 0x55853f0b1470 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.554+0000 7fcbd1a1f740 1 --2- 172.21.15.52:0/4279917158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f090ff0 0x55853f09bc00 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.554+0000 7fcba7fff640 0 client.18468 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:03:25.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.554+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/4279917158 >> 172.21.15.52:0/4279917158 conn(0x55853f08b9f0 msgr2=0x55853f08d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:25.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.554+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/4279917158 shutdown_connections 2024-04-02T14:03:25.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.554+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/4279917158 wait complete. 2024-04-02T14:03:25.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.554+0000 7fcbd1a1f740 20 client.18468 trim_cache size 0 max 16384 2024-04-02T14:03:25.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.572+0000 7fcbd1a1f740 1 Processor -- start 2024-04-02T14:03:25.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.572+0000 7fcbd1a1f740 1 -- start start 2024-04-02T14:03:25.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.572+0000 7fcbd1a1f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55853f090ff0 0x55853f09e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.572+0000 7fcbd1a1f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f091910 0x55853f0aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.572+0000 7fcbd1a1f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55853f09ebd0 0x55853f0b6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.572+0000 7fcbd1a1f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81acbe0 con 0x55853f091910 2024-04-02T14:03:25.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.572+0000 7fcbd1a1f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81ae720 con 0x55853f09ebd0 2024-04-02T14:03:25.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.572+0000 7fcbd1a1f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81ad2b0 con 0x55853f090ff0 2024-04-02T14:03:25.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.572+0000 7fcbccbde640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f091910 0x55853f0aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.572+0000 7fcbccbde640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f091910 0x55853f0aa8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57370/0 (socket says 172.21.15.52:57370) 2024-04-02T14:03:25.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.572+0000 7fcbccbde640 1 -- 172.21.15.52:0/3923414815 learned_addr learned my addr 172.21.15.52:0/3923414815 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:25.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.572+0000 7fcbcd3df640 1 --2- 172.21.15.52:0/3923414815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55853f09ebd0 0x55853f0b6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.572+0000 7fcbcd3df640 1 --2- 172.21.15.52:0/3923414815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55853f090ff0 0x55853f09e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.572+0000 7fcbccbde640 1 -- 172.21.15.52:0/3923414815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55853f090ff0 msgr2=0x55853f09e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:25.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.572+0000 7fcbccbde640 1 --2- 172.21.15.52:0/3923414815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55853f090ff0 0x55853f09e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.572+0000 7fcbccbde640 1 -- 172.21.15.52:0/3923414815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55853f09ebd0 msgr2=0x55853f0b6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:25.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.572+0000 7fcbccbde640 1 --2- 172.21.15.52:0/3923414815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55853f09ebd0 0x55853f0b6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.572+0000 7fcbccbde640 1 -- 172.21.15.52:0/3923414815 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55853f0903b0 con 0x55853f091910 2024-04-02T14:03:25.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.573+0000 7fcbccbde640 1 --2- 172.21.15.52:0/3923414815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f091910 0x55853f0aa8c0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fcbc002e010 tx=0x7fcbc0009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b93a3a2856bdf99 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:25.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.573+0000 7fcbae7fc640 1 -- 172.21.15.52:0/3923414815 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0043050 con 0x55853f091910 2024-04-02T14:03:25.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.573+0000 7fcbae7fc640 1 -- 172.21.15.52:0/3923414815 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbc004d070 con 0x55853f091910 2024-04-02T14:03:25.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.573+0000 7fcbae7fc640 1 -- 172.21.15.52:0/3923414815 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0039040 con 0x55853f091910 2024-04-02T14:03:25.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.573+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/3923414815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f091910 msgr2=0x55853f0aa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:25.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.573+0000 7fcbd1a1f740 1 --2- 172.21.15.52:0/3923414815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f091910 0x55853f0aa8c0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fcbc002e010 tx=0x7fcbc0009910 comp rx=0 tx=0).stop 2024-04-02T14:03:25.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.573+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/3923414815 shutdown_connections 2024-04-02T14:03:25.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.573+0000 7fcbd1a1f740 1 --2- 172.21.15.52:0/3923414815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55853f09ebd0 0x55853f0b6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.573+0000 7fcbd1a1f740 1 --2- 172.21.15.52:0/3923414815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f091910 0x55853f0aa8c0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.573+0000 7fcbd1a1f740 1 --2- 172.21.15.52:0/3923414815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55853f090ff0 0x55853f09e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.573+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/3923414815 >> 172.21.15.52:0/3923414815 conn(0x55853f08b9f0 msgr2=0x55853f08aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:25.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.573+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/3923414815 shutdown_connections 2024-04-02T14:03:25.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.574+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/3923414815 wait complete. 2024-04-02T14:03:25.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.574+0000 7fcbd1a1f740 1 -- start start 2024-04-02T14:03:25.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.574+0000 7fcbd1a1f740 1 Processor -- start 2024-04-02T14:03:25.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.574+0000 7fcbd1a1f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f090ff0 0x55853f0a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.574+0000 7fcbd1a1f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55853f091910 0x55853f0a3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.574+0000 7fcbd1a1f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55853f09ebd0 0x55853f0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.574+0000 7fcbd1a1f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81ae050 con 0x55853f090ff0 2024-04-02T14:03:25.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.574+0000 7fcbd1a1f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81ad980 con 0x55853f09ebd0 2024-04-02T14:03:25.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.574+0000 7fcbd1a1f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc81aabe0 con 0x55853f091910 2024-04-02T14:03:25.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.574+0000 7fcbd1a1f740 10 client.0 _ll_register_callbacks cb 0x7fcbb8010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:25.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.574+0000 7fcbcd3df640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f090ff0 0x55853f0a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.574+0000 7fcbcd3df640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f090ff0 0x55853f0a2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57372/0 (socket says 172.21.15.52:57372) 2024-04-02T14:03:25.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.574+0000 7fcbcd3df640 1 -- 172.21.15.52:0/2273673770 learned_addr learned my addr 172.21.15.52:0/2273673770 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:25.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.574+0000 7fcbcd3df640 1 --2- 172.21.15.52:0/2273673770 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55853f09ebd0 0x55853f0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.575+0000 7fcbccbde640 1 --2- 172.21.15.52:0/2273673770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55853f091910 0x55853f0a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.575+0000 7fcbcd3df640 1 -- 172.21.15.52:0/2273673770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55853f091910 msgr2=0x55853f0a3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:25.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.575+0000 7fcbcd3df640 1 --2- 172.21.15.52:0/2273673770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55853f091910 0x55853f0a3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.575+0000 7fcbcd3df640 1 -- 172.21.15.52:0/2273673770 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55853f09ebd0 msgr2=0x55853f0a35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:25.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.575+0000 7fcbcd3df640 1 --2- 172.21.15.52:0/2273673770 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55853f09ebd0 0x55853f0a35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.575+0000 7fcbcd3df640 1 -- 172.21.15.52:0/2273673770 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55853f114850 con 0x55853f090ff0 2024-04-02T14:03:25.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.575+0000 7fcbcd3df640 1 --2- 172.21.15.52:0/2273673770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f090ff0 0x55853f0a2b20 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fcbb800f950 tx=0x7fcbb800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7cbe7969876526f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:25.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.575+0000 7fcba7fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:25.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.575+0000 7fcba7fff640 1 -- 172.21.15.52:0/2273673770 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8014150 con 0x55853f090ff0 2024-04-02T14:03:25.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.575+0000 7fcba7fff640 1 -- 172.21.15.52:0/2273673770 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fcbb80142f0 con 0x55853f090ff0 2024-04-02T14:03:25.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.575+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/2273673770 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbc004d070 con 0x55853f090ff0 2024-04-02T14:03:25.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.575+0000 7fcba7fff640 1 -- 172.21.15.52:0/2273673770 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcbb800fbc0 con 0x55853f090ff0 2024-04-02T14:03:25.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.575+0000 7fcbd1a1f740 10 client.18474 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:25.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.575+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/2273673770 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55853f115b90 con 0x55853f090ff0 2024-04-02T14:03:25.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.575+0000 7fcba7fff640 1 -- 172.21.15.52:0/2273673770 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbc004d070 con 0x55853f090ff0 2024-04-02T14:03:25.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.576+0000 7fcba7fff640 1 -- 172.21.15.52:0/2273673770 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fcbb8003980 con 0x55853f090ff0 2024-04-02T14:03:25.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.576+0000 7fcbd1a1f740 10 client.18474 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:25.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.576+0000 7fcbd1a1f740 10 client.18474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:03:25.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.576+0000 7fcbd1a1f740 20 client.18474 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:25.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.576+0000 7fcbd1a1f740 1 --2- 172.21.15.52:0/2273673770 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55853f116280 0x55853f1186a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.576+0000 7fcbd1a1f740 4 client.18474 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T14:03:25.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.576+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/2273673770 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55853effd000 con 0x55853f116280 2024-04-02T14:03:25.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.576+0000 7fcbccbde640 1 --2- 172.21.15.52:0/2273673770 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55853f116280 0x55853f1186a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.576+0000 7fcba7fff640 10 client.18474 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:03:25.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.576+0000 7fcbccbde640 1 --2- 172.21.15.52:0/2273673770 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55853f116280 0x55853f1186a0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c351399705f6437c server_cookie=4c639e628ed984ed in_seq=0 out_seq=0 2024-04-02T14:03:25.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.577+0000 7fcba7fff640 1 -- 172.21.15.52:0/2273673770 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55853effd000 con 0x55853f116280 2024-04-02T14:03:25.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.577+0000 7fcba7fff640 10 client.18474 handle_command_reply: tid=0 2024-04-02T14:03:25.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.577+0000 7fcbd1a1f740 1 client.18474 shutdown 2024-04-02T14:03:25.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.577+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/2273673770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f090ff0 msgr2=0x55853f0a2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:25.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.577+0000 7fcbd1a1f740 1 --2- 172.21.15.52:0/2273673770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f090ff0 0x55853f0a2b20 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fcbb800f950 tx=0x7fcbb800eb50 comp rx=0 tx=0).stop 2024-04-02T14:03:25.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.577+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/2273673770 shutdown_connections 2024-04-02T14:03:25.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.577+0000 7fcbd1a1f740 1 --2- 172.21.15.52:0/2273673770 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55853f09ebd0 0x55853f0a35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.577+0000 7fcbd1a1f740 1 --2- 172.21.15.52:0/2273673770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55853f091910 0x55853f0a3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.577+0000 7fcbd1a1f740 1 --2- 172.21.15.52:0/2273673770 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55853f116280 0x55853f1186a0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.577+0000 7fcbd1a1f740 1 --2- 172.21.15.52:0/2273673770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55853f090ff0 0x55853f0a2b20 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.577+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/2273673770 >> 172.21.15.52:0/2273673770 conn(0x55853f08b9f0 msgr2=0x55853f09bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:25.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.578+0000 7fcba7fff640 0 client.18474 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:03:25.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.578+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/2273673770 shutdown_connections 2024-04-02T14:03:25.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.578+0000 7fcbd1a1f740 1 -- 172.21.15.52:0/2273673770 wait complete. 2024-04-02T14:03:25.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.578+0000 7fcbd1a1f740 20 client.18474 trim_cache size 0 max 16384 2024-04-02T14:03:25.592 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:03:25.592 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-04-02T14:03:25.592 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "d7671a4c-1fd0-4cb0-a255-022487119dd6", 2024-04-02T14:03:25.592 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-04-02T14:03:25.592 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:03:25.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.578+0000 7fcbcee42640 1 -- 172.21.15.52:0/805974310 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fcbb00a3d20 msgr2=0x7fcbb00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:25.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.578+0000 7fcbcee42640 1 --2- 172.21.15.52:0/805974310 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fcbb00a3d20 0x7fcbb00a61e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc09c920 tx=0x7fcbbc018420 comp rx=0 tx=0).stop 2024-04-02T14:03:25.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.578+0000 7fcbcee42640 1 -- 172.21.15.52:0/805974310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbc8101330 msgr2=0x7fcbc8198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:25.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.578+0000 7fcbcee42640 1 --2- 172.21.15.52:0/805974310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbc8101330 0x7fcbc8198860 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc00e9f0 tx=0x7fcbbc00eeb0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.578+0000 7fcbcee42640 1 -- 172.21.15.52:0/805974310 shutdown_connections 2024-04-02T14:03:25.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.578+0000 7fcbcee42640 1 --2- 172.21.15.52:0/805974310 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fcbb00a3d20 0x7fcbb00a61e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.578+0000 7fcbcee42640 1 --2- 172.21.15.52:0/805974310 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbc8106980 0x7fcbc819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.578+0000 7fcbcee42640 1 --2- 172.21.15.52:0/805974310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbc8101330 0x7fcbc8198860 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.578+0000 7fcbcee42640 1 --2- 172.21.15.52:0/805974310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcbc8100970 0x7fcbc8198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.578+0000 7fcbcee42640 1 -- 172.21.15.52:0/805974310 >> 172.21.15.52:0/805974310 conn(0x7fcbc8077b00 msgr2=0x7fcbc8103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:25.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.578+0000 7fcbcee42640 1 -- 172.21.15.52:0/805974310 shutdown_connections 2024-04-02T14:03:25.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.578+0000 7fcbcee42640 1 -- 172.21.15.52:0/805974310 wait complete. 2024-04-02T14:03:25.595 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:03:25.596 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:03:25.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.703+0000 7f2986463640 1 -- 172.21.15.52:0/2333633963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2980105ef0 msgr2=0x7f298010dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:25.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.703+0000 7f2986463640 1 --2- 172.21.15.52:0/2333633963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2980105ef0 0x7f298010dac0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f2978009ca0 tx=0x7f297802c0f0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.703+0000 7f2986463640 1 -- 172.21.15.52:0/2333633963 shutdown_connections 2024-04-02T14:03:25.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.703+0000 7f2986463640 1 --2- 172.21.15.52:0/2333633963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2980105ef0 0x7f298010dac0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.703+0000 7f2986463640 1 --2- 172.21.15.52:0/2333633963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2980101cf0 0x7f2980105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.703+0000 7f2986463640 1 --2- 172.21.15.52:0/2333633963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2980101340 0x7f2980101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.703+0000 7f2986463640 1 -- 172.21.15.52:0/2333633963 >> 172.21.15.52:0/2333633963 conn(0x7f2980076f20 msgr2=0x7f29800fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:25.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.703+0000 7f2986463640 1 -- 172.21.15.52:0/2333633963 shutdown_connections 2024-04-02T14:03:25.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.703+0000 7f2986463640 1 -- 172.21.15.52:0/2333633963 wait complete. 2024-04-02T14:03:25.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.704+0000 7f2986463640 1 Processor -- start 2024-04-02T14:03:25.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.704+0000 7f2986463640 1 -- start start 2024-04-02T14:03:25.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.704+0000 7f2986463640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2980101340 0x7f2980199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.704+0000 7f2986463640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2980105ef0 0x7f2980199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.704+0000 7f2986463640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2980199ea0 0x7f29801a44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.704+0000 7f2986463640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2980110fa0 con 0x7f2980199ea0 2024-04-02T14:03:25.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.704+0000 7f2986463640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2980110e20 con 0x7f2980105ef0 2024-04-02T14:03:25.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.704+0000 7f2986463640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2980111120 con 0x7f2980101340 2024-04-02T14:03:25.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.705+0000 7f2984a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2980199ea0 0x7f29801a44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.705+0000 7f297ffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2980101340 0x7f2980199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.705+0000 7f297f7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2980105ef0 0x7f2980199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.705+0000 7f2984a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2980199ea0 0x7f29801a44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57386/0 (socket says 172.21.15.52:57386) 2024-04-02T14:03:25.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.705+0000 7f2984a00640 1 -- 172.21.15.52:0/4014402137 learned_addr learned my addr 172.21.15.52:0/4014402137 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:25.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.705+0000 7f297ffff640 1 -- 172.21.15.52:0/4014402137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2980105ef0 msgr2=0x7f2980199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:25.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.705+0000 7f297ffff640 1 --2- 172.21.15.52:0/4014402137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2980105ef0 0x7f2980199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.705+0000 7f297ffff640 1 -- 172.21.15.52:0/4014402137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2980199ea0 msgr2=0x7f29801a44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:25.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.705+0000 7f297ffff640 1 --2- 172.21.15.52:0/4014402137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2980199ea0 0x7f29801a44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.705+0000 7f297ffff640 1 -- 172.21.15.52:0/4014402137 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29801a4c40 con 0x7f2980101340 2024-04-02T14:03:25.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.705+0000 7f297f7fe640 1 --2- 172.21.15.52:0/4014402137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2980105ef0 0x7f2980199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:03:25.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.705+0000 7f2984a00640 1 --2- 172.21.15.52:0/4014402137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2980199ea0 0x7f29801a44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:03:25.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.705+0000 7f297ffff640 1 --2- 172.21.15.52:0/4014402137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2980101340 0x7f2980199420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f297000e9f0 tx=0x7f297000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:25.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.705+0000 7f297d7fa640 1 -- 172.21.15.52:0/4014402137 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297000ccc0 con 0x7f2980101340 2024-04-02T14:03:25.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.706+0000 7f2986463640 1 -- 172.21.15.52:0/4014402137 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29801a4f30 con 0x7f2980101340 2024-04-02T14:03:25.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.706+0000 7f297d7fa640 1 -- 172.21.15.52:0/4014402137 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2970010910 con 0x7f2980101340 2024-04-02T14:03:25.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.706+0000 7f297d7fa640 1 -- 172.21.15.52:0/4014402137 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2970018960 con 0x7f2980101340 2024-04-02T14:03:25.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.706+0000 7f2986463640 1 -- 172.21.15.52:0/4014402137 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29801a5440 con 0x7f2980101340 2024-04-02T14:03:25.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.708+0000 7f297d7fa640 1 -- 172.21.15.52:0/4014402137 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2970010ac0 con 0x7f2980101340 2024-04-02T14:03:25.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.708+0000 7f2989052740 1 Processor -- start 2024-04-02T14:03:25.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.708+0000 7f2989052740 1 -- start start 2024-04-02T14:03:25.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.709+0000 7f2989052740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625abca18d0 0x5625abca1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.709+0000 7f2989052740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abca21f0 0x5625abcaa890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.709+0000 7f297d7fa640 1 --2- 172.21.15.52:0/4014402137 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f29680a3d50 0x7f29680a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.709+0000 7f2989052740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5625abcaadd0 0x5625abcad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.709+0000 7f2989052740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5625abc05830 con 0x5625abca21f0 2024-04-02T14:03:25.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.709+0000 7f2989052740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5625abbad820 con 0x5625abcaadd0 2024-04-02T14:03:25.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.709+0000 7f2989052740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5625abc017e0 con 0x5625abca18d0 2024-04-02T14:03:25.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.709+0000 7f2984a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625abca18d0 0x5625abca1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.709+0000 7f2984a00640 1 --2- 172.21.15.52:0/4014402137 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f29680a3d50 0x7f29680a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.709+0000 7f297ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abca21f0 0x5625abcaa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.709+0000 7f297ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abca21f0 0x5625abcaa890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57400/0 (socket says 172.21.15.52:57400) 2024-04-02T14:03:25.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.709+0000 7f297ffff640 1 -- 172.21.15.52:0/2062623110 learned_addr learned my addr 172.21.15.52:0/2062623110 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:25.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.709+0000 7f297f7fe640 1 --2- 172.21.15.52:0/2062623110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5625abcaadd0 0x5625abcad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.709+0000 7f297d7fa640 1 -- 172.21.15.52:0/4014402137 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f29700d0f10 con 0x7f2980101340 2024-04-02T14:03:25.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.709+0000 7f297ffff640 1 -- 172.21.15.52:0/2062623110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625abca18d0 msgr2=0x5625abca1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:25.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.709+0000 7f297ffff640 1 --2- 172.21.15.52:0/2062623110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625abca18d0 0x5625abca1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.709+0000 7f297ffff640 1 -- 172.21.15.52:0/2062623110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5625abcaadd0 msgr2=0x5625abcad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:25.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.709+0000 7f297ffff640 1 --2- 172.21.15.52:0/2062623110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5625abcaadd0 0x5625abcad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.709+0000 7f297ffff640 1 -- 172.21.15.52:0/2062623110 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625ab9e37d0 con 0x5625abca21f0 2024-04-02T14:03:25.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.709+0000 7f2984a00640 1 --2- 172.21.15.52:0/4014402137 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f29680a3d50 0x7f29680a6210 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2978009430 tx=0x7f2978009360 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:25.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.709+0000 7f297ffff640 1 --2- 172.21.15.52:0/2062623110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abca21f0 0x5625abcaa890 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f29700a5200 tx=0x7f297006f660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29cbb66f533e3741 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:25.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.710+0000 7f29667fc640 1 -- 172.21.15.52:0/2062623110 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29700a0e70 con 0x5625abca21f0 2024-04-02T14:03:25.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.710+0000 7f29667fc640 1 -- 172.21.15.52:0/2062623110 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29700a5730 con 0x5625abca21f0 2024-04-02T14:03:25.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.710+0000 7f29667fc640 1 -- 172.21.15.52:0/2062623110 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29700c4c00 con 0x5625abca21f0 2024-04-02T14:03:25.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.710+0000 7f2989052740 1 -- 172.21.15.52:0/2062623110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abca21f0 msgr2=0x5625abcaa890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:25.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.710+0000 7f2989052740 1 --2- 172.21.15.52:0/2062623110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abca21f0 0x5625abcaa890 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f29700a5200 tx=0x7f297006f660 comp rx=0 tx=0).stop 2024-04-02T14:03:25.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.710+0000 7f2989052740 1 -- 172.21.15.52:0/2062623110 shutdown_connections 2024-04-02T14:03:25.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.710+0000 7f2989052740 1 --2- 172.21.15.52:0/2062623110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5625abcaadd0 0x5625abcad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.710+0000 7f2989052740 1 --2- 172.21.15.52:0/2062623110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abca21f0 0x5625abcaa890 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.710+0000 7f2989052740 1 --2- 172.21.15.52:0/2062623110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625abca18d0 0x5625abca1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.710+0000 7f2989052740 1 -- 172.21.15.52:0/2062623110 >> 172.21.15.52:0/2062623110 conn(0x5625abc9c000 msgr2=0x5625abc9c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:25.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.711+0000 7f2989052740 1 -- 172.21.15.52:0/2062623110 shutdown_connections 2024-04-02T14:03:25.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.711+0000 7f2989052740 1 -- 172.21.15.52:0/2062623110 wait complete. 2024-04-02T14:03:25.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.711+0000 7f2989052740 1 -- start start 2024-04-02T14:03:25.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.711+0000 7f2989052740 1 Processor -- start 2024-04-02T14:03:25.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.711+0000 7f2989052740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5625abca18d0 0x5625abcaa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.711+0000 7f2989052740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abca21f0 0x5625abca7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.711+0000 7f2989052740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625abcaadd0 0x5625abca7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.711+0000 7f2989052740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5625abcadb60 con 0x5625abca21f0 2024-04-02T14:03:25.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.711+0000 7f2989052740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5625abbc0d30 con 0x5625abca18d0 2024-04-02T14:03:25.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.711+0000 7f2989052740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5625abcadce0 con 0x5625abcaadd0 2024-04-02T14:03:25.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.711+0000 7f2989052740 10 client.0 _ll_register_callbacks cb 0x5625abbc0720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:25.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.711+0000 7f297f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abca21f0 0x5625abca7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.712+0000 7f297ffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5625abca18d0 0x5625abcaa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.712+0000 7f297ffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5625abca18d0 0x5625abcaa700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:35936/0 (socket says 172.21.15.52:35936) 2024-04-02T14:03:25.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.712+0000 7f297f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abca21f0 0x5625abca7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57412/0 (socket says 172.21.15.52:57412) 2024-04-02T14:03:25.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.712+0000 7f297ffff640 1 -- 172.21.15.52:0/615926057 learned_addr learned my addr 172.21.15.52:0/615926057 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:25.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.712+0000 7f2984a00640 1 --2- 172.21.15.52:0/615926057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625abcaadd0 0x5625abca7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.712+0000 7f297ffff640 1 -- 172.21.15.52:0/615926057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625abcaadd0 msgr2=0x5625abca7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:25.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.712+0000 7f297ffff640 1 --2- 172.21.15.52:0/615926057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625abcaadd0 0x5625abca7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.712+0000 7f297ffff640 1 -- 172.21.15.52:0/615926057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abca21f0 msgr2=0x5625abca7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:25.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.712+0000 7f297ffff640 1 --2- 172.21.15.52:0/615926057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abca21f0 0x5625abca7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.712+0000 7f297ffff640 1 -- 172.21.15.52:0/615926057 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625ab9193f0 con 0x5625abca18d0 2024-04-02T14:03:25.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.712+0000 7f297f7fe640 1 --2- 172.21.15.52:0/615926057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abca21f0 0x5625abca7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:03:25.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.712+0000 7f297ffff640 1 --2- 172.21.15.52:0/615926057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5625abca18d0 0x5625abcaa700 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f297009c870 tx=0x7f297000c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b048b460295405c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:25.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.712+0000 7f295f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-02T14:03:25.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.712+0000 7f295f7fe640 1 -- 172.21.15.52:0/615926057 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29700a0e70 con 0x5625abca18d0 2024-04-02T14:03:25.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.712+0000 7f295f7fe640 1 -- 172.21.15.52:0/615926057 <== mon.2 v2:172.21.15.88:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f297009ae70 con 0x5625abca18d0 2024-04-02T14:03:25.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.712+0000 7f2989052740 1 -- 172.21.15.52:0/615926057 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29700a5730 con 0x5625abca18d0 2024-04-02T14:03:25.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.713+0000 7f295f7fe640 1 -- 172.21.15.52:0/615926057 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f29700c4b60 con 0x5625abca18d0 2024-04-02T14:03:25.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.713+0000 7f2989052740 10 client.27455 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:25.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.713+0000 7f2989052740 1 -- 172.21.15.52:0/615926057 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5625ab8d90f0 con 0x5625abca18d0 2024-04-02T14:03:25.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.713+0000 7f295f7fe640 1 -- 172.21.15.52:0/615926057 <== mon.2 v2:172.21.15.88:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29700a5730 con 0x5625abca18d0 2024-04-02T14:03:25.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.713+0000 7f295f7fe640 1 -- 172.21.15.52:0/615926057 <== mon.2 v2:172.21.15.88:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2970018c20 con 0x5625abca18d0 2024-04-02T14:03:25.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.713+0000 7f2989052740 10 client.27455 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:25.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.713+0000 7f2989052740 10 client.27455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:03:25.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.713+0000 7f2989052740 20 client.27455 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:25.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.713+0000 7f2989052740 1 --2- 172.21.15.52:0/615926057 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5625abcbac30 0x5625abcbd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.713+0000 7f2989052740 4 client.27455 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:03:25.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.713+0000 7f2989052740 1 -- 172.21.15.52:0/615926057 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625abc12d70 con 0x5625abcbac30 2024-04-02T14:03:25.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.714+0000 7f297f7fe640 1 --2- 172.21.15.52:0/615926057 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5625abcbac30 0x5625abcbd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.714+0000 7f295f7fe640 10 client.27455 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:03:25.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.714+0000 7f297f7fe640 1 --2- 172.21.15.52:0/615926057 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5625abcbac30 0x5625abcbd020 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64702ac2ee4fd4a3 server_cookie=6cbf5e51dc96f17f in_seq=0 out_seq=0 2024-04-02T14:03:25.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.715+0000 7f295f7fe640 1 -- 172.21.15.52:0/615926057 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5625abc12d70 con 0x5625abcbac30 2024-04-02T14:03:25.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.715+0000 7f295f7fe640 10 client.27455 handle_command_reply: tid=0 2024-04-02T14:03:25.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.715+0000 7f2989052740 1 client.27455 shutdown 2024-04-02T14:03:25.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.715+0000 7f2989052740 1 -- 172.21.15.52:0/615926057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5625abca18d0 msgr2=0x5625abcaa700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:25.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.715+0000 7f2989052740 1 --2- 172.21.15.52:0/615926057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5625abca18d0 0x5625abcaa700 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f297009c870 tx=0x7f297000c7d0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.715+0000 7f2989052740 1 -- 172.21.15.52:0/615926057 shutdown_connections 2024-04-02T14:03:25.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.715+0000 7f2989052740 1 --2- 172.21.15.52:0/615926057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625abcaadd0 0x5625abca7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.715+0000 7f2989052740 1 --2- 172.21.15.52:0/615926057 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5625abcbac30 0x5625abcbd020 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.715+0000 7f2989052740 1 --2- 172.21.15.52:0/615926057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abca21f0 0x5625abca7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.715+0000 7f2989052740 1 --2- 172.21.15.52:0/615926057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5625abca18d0 0x5625abcaa700 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.715+0000 7f2989052740 1 -- 172.21.15.52:0/615926057 >> 172.21.15.52:0/615926057 conn(0x5625abc9c000 msgr2=0x5625abc9dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:25.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.715+0000 7f295f7fe640 0 client.27455 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:03:25.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.715+0000 7f2989052740 1 -- 172.21.15.52:0/615926057 shutdown_connections 2024-04-02T14:03:25.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.715+0000 7f2989052740 1 -- 172.21.15.52:0/615926057 wait complete. 2024-04-02T14:03:25.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.716+0000 7f2989052740 20 client.27455 trim_cache size 0 max 16384 2024-04-02T14:03:25.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.736+0000 7f2989052740 1 Processor -- start 2024-04-02T14:03:25.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.736+0000 7f2989052740 1 -- start start 2024-04-02T14:03:25.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.736+0000 7f2989052740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625abca18d0 0x5625abcb67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.736+0000 7f2989052740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5625abca21f0 0x5625abca6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.736+0000 7f2989052740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abcaadd0 0x5625abcaa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.736+0000 7f2989052740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f29801af050 con 0x5625abcaadd0 2024-04-02T14:03:25.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.736+0000 7f2989052740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f29801b0b90 con 0x5625abca21f0 2024-04-02T14:03:25.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.736+0000 7f2989052740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f29801af720 con 0x5625abca18d0 2024-04-02T14:03:25.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.736+0000 7f297f7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5625abca21f0 0x5625abca6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.736+0000 7f297ffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625abca18d0 0x5625abcb67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.736+0000 7f297f7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5625abca21f0 0x5625abca6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.52:35940/0 (socket says 172.21.15.52:35940) 2024-04-02T14:03:25.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.736+0000 7f297f7fe640 1 -- 172.21.15.52:0/1371714627 learned_addr learned my addr 172.21.15.52:0/1371714627 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:25.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.736+0000 7f297ffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625abca18d0 0x5625abcb67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.52:51912/0 (socket says 172.21.15.52:51912) 2024-04-02T14:03:25.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.736+0000 7f2984a00640 1 --2- 172.21.15.52:0/1371714627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abcaadd0 0x5625abcaa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.737+0000 7f297f7fe640 1 -- 172.21.15.52:0/1371714627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625abca18d0 msgr2=0x5625abcb67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:25.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.737+0000 7f297f7fe640 1 --2- 172.21.15.52:0/1371714627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625abca18d0 0x5625abcb67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.737+0000 7f297f7fe640 1 -- 172.21.15.52:0/1371714627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abcaadd0 msgr2=0x5625abcaa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:25.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.737+0000 7f297f7fe640 1 --2- 172.21.15.52:0/1371714627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abcaadd0 0x5625abcaa1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.737+0000 7f297f7fe640 1 -- 172.21.15.52:0/1371714627 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625abc97c50 con 0x5625abca21f0 2024-04-02T14:03:25.743 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:25 smithi052 ceph-mon[32104]: pgmap v308: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:03:25.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.737+0000 7f2984a00640 1 --2- 172.21.15.52:0/1371714627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abcaadd0 0x5625abcaa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:03:25.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.737+0000 7f297f7fe640 1 --2- 172.21.15.52:0/1371714627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5625abca21f0 0x5625abca6940 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2980100ae0 tx=0x7f2974014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cac8394313375e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:25.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.737+0000 7f2965ffb640 1 -- 172.21.15.52:0/1371714627 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2974004030 con 0x5625abca21f0 2024-04-02T14:03:25.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.737+0000 7f2965ffb640 1 -- 172.21.15.52:0/1371714627 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29740041d0 con 0x5625abca21f0 2024-04-02T14:03:25.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.737+0000 7f2965ffb640 1 -- 172.21.15.52:0/1371714627 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2974022310 con 0x5625abca21f0 2024-04-02T14:03:25.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.738+0000 7f2989052740 1 -- 172.21.15.52:0/1371714627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5625abca21f0 msgr2=0x5625abca6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:25.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.738+0000 7f2989052740 1 --2- 172.21.15.52:0/1371714627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5625abca21f0 0x5625abca6940 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2980100ae0 tx=0x7f2974014040 comp rx=0 tx=0).stop 2024-04-02T14:03:25.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.738+0000 7f2989052740 1 -- 172.21.15.52:0/1371714627 shutdown_connections 2024-04-02T14:03:25.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.738+0000 7f2989052740 1 --2- 172.21.15.52:0/1371714627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abcaadd0 0x5625abcaa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.738+0000 7f2989052740 1 --2- 172.21.15.52:0/1371714627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5625abca21f0 0x5625abca6940 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.738+0000 7f2989052740 1 --2- 172.21.15.52:0/1371714627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625abca18d0 0x5625abcb67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.738+0000 7f2989052740 1 -- 172.21.15.52:0/1371714627 >> 172.21.15.52:0/1371714627 conn(0x5625abc9c000 msgr2=0x5625abca6230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:25.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.738+0000 7f2989052740 1 -- 172.21.15.52:0/1371714627 shutdown_connections 2024-04-02T14:03:25.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.738+0000 7f2989052740 1 -- 172.21.15.52:0/1371714627 wait complete. 2024-04-02T14:03:25.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.739+0000 7f2989052740 1 -- start start 2024-04-02T14:03:25.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.739+0000 7f2989052740 1 Processor -- start 2024-04-02T14:03:25.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.739+0000 7f2989052740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abca18d0 0x5625abcbd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.739+0000 7f2989052740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5625abca21f0 0x5625abcbded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.739+0000 7f2989052740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625abcaadd0 0x5625abcbe410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.739+0000 7f2989052740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f29801b04c0 con 0x5625abca18d0 2024-04-02T14:03:25.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.739+0000 7f2989052740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f29801afdf0 con 0x5625abca21f0 2024-04-02T14:03:25.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.739+0000 7f2989052740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f29801ad050 con 0x5625abcaadd0 2024-04-02T14:03:25.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.739+0000 7f2989052740 10 client.0 _ll_register_callbacks cb 0x7f2970016f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:25.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.739+0000 7f297ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abca18d0 0x5625abcbd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.739+0000 7f297ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abca18d0 0x5625abcbd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57428/0 (socket says 172.21.15.52:57428) 2024-04-02T14:03:25.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.740+0000 7f297ffff640 1 -- 172.21.15.52:0/2920352271 learned_addr learned my addr 172.21.15.52:0/2920352271 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:25.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.740+0000 7f297f7fe640 1 --2- 172.21.15.52:0/2920352271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5625abca21f0 0x5625abcbded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.740+0000 7f2984a00640 1 --2- 172.21.15.52:0/2920352271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625abcaadd0 0x5625abcbe410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.740+0000 7f297ffff640 1 -- 172.21.15.52:0/2920352271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625abcaadd0 msgr2=0x5625abcbe410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:25.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.740+0000 7f297ffff640 1 --2- 172.21.15.52:0/2920352271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625abcaadd0 0x5625abcbe410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.740+0000 7f297ffff640 1 -- 172.21.15.52:0/2920352271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5625abca21f0 msgr2=0x5625abcbded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:25.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.740+0000 7f297ffff640 1 --2- 172.21.15.52:0/2920352271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5625abca21f0 0x5625abcbded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.740+0000 7f297ffff640 1 -- 172.21.15.52:0/2920352271 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625abd1fe10 con 0x5625abca18d0 2024-04-02T14:03:25.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.740+0000 7f297f7fe640 1 --2- 172.21.15.52:0/2920352271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5625abca21f0 0x5625abcbded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:03:25.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.740+0000 7f297ffff640 1 --2- 172.21.15.52:0/2920352271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abca18d0 0x5625abcbd990 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f2970099540 tx=0x7f297000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d1567917c53fcf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:25.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.740+0000 7f295f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:25.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.740+0000 7f295f7fe640 1 -- 172.21.15.52:0/2920352271 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2970018d00 con 0x5625abca18d0 2024-04-02T14:03:25.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.740+0000 7f295f7fe640 1 -- 172.21.15.52:0/2920352271 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f29700c4740 con 0x5625abca18d0 2024-04-02T14:03:25.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.740+0000 7f2989052740 1 -- 172.21.15.52:0/2920352271 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29740041d0 con 0x5625abca18d0 2024-04-02T14:03:25.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.740+0000 7f295f7fe640 1 -- 172.21.15.52:0/2920352271 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f29700a0e70 con 0x5625abca18d0 2024-04-02T14:03:25.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.741+0000 7f2989052740 10 client.18492 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:25.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.741+0000 7f2989052740 1 -- 172.21.15.52:0/2920352271 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5625abd1fe10 con 0x5625abca18d0 2024-04-02T14:03:25.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.741+0000 7f295f7fe640 1 -- 172.21.15.52:0/2920352271 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29740041d0 con 0x5625abca18d0 2024-04-02T14:03:25.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.741+0000 7f295f7fe640 1 -- 172.21.15.52:0/2920352271 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f29700c4350 con 0x5625abca18d0 2024-04-02T14:03:25.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.741+0000 7f2989052740 10 client.18492 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:25.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.741+0000 7f2989052740 10 client.18492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:03:25.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.741+0000 7f2989052740 20 client.18492 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:25.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.741+0000 7f2989052740 1 --2- 172.21.15.52:0/2920352271 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5625abd213e0 0x5625abd23800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:25.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.741+0000 7f2989052740 4 client.18492 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:03:25.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.741+0000 7f2989052740 1 -- 172.21.15.52:0/2920352271 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5625abc0d0c0 con 0x5625abd213e0 2024-04-02T14:03:25.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.741+0000 7f297f7fe640 1 --2- 172.21.15.52:0/2920352271 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5625abd213e0 0x5625abd23800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:25.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.742+0000 7f295f7fe640 10 client.18492 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:03:25.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.742+0000 7f297f7fe640 1 --2- 172.21.15.52:0/2920352271 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5625abd213e0 0x5625abd23800 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cfc54a700424ca8 server_cookie=f22acecb6835b3dd in_seq=0 out_seq=0 2024-04-02T14:03:25.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.742+0000 7f295f7fe640 1 -- 172.21.15.52:0/2920352271 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x5625abc0d0c0 con 0x5625abd213e0 2024-04-02T14:03:25.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.742+0000 7f295f7fe640 10 client.18492 handle_command_reply: tid=0 2024-04-02T14:03:25.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.742+0000 7f2989052740 1 client.18492 shutdown 2024-04-02T14:03:25.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.742+0000 7f2989052740 1 -- 172.21.15.52:0/2920352271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abca18d0 msgr2=0x5625abcbd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:25.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.742+0000 7f2989052740 1 --2- 172.21.15.52:0/2920352271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abca18d0 0x5625abcbd990 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f2970099540 tx=0x7f297000c840 comp rx=0 tx=0).stop 2024-04-02T14:03:25.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.742+0000 7f2989052740 1 -- 172.21.15.52:0/2920352271 shutdown_connections 2024-04-02T14:03:25.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.742+0000 7f2989052740 1 --2- 172.21.15.52:0/2920352271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625abcaadd0 0x5625abcbe410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.742+0000 7f2989052740 1 --2- 172.21.15.52:0/2920352271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5625abca21f0 0x5625abcbded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.742+0000 7f2989052740 1 --2- 172.21.15.52:0/2920352271 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x5625abd213e0 0x5625abd23800 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.742+0000 7f2989052740 1 --2- 172.21.15.52:0/2920352271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5625abca18d0 0x5625abcbd990 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.742+0000 7f2989052740 1 -- 172.21.15.52:0/2920352271 >> 172.21.15.52:0/2920352271 conn(0x5625abc9c000 msgr2=0x5625abca2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:25.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.742+0000 7f295f7fe640 0 client.18492 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:03:25.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.743+0000 7f2989052740 1 -- 172.21.15.52:0/2920352271 shutdown_connections 2024-04-02T14:03:25.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.743+0000 7f2989052740 1 -- 172.21.15.52:0/2920352271 wait complete. 2024-04-02T14:03:25.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.743+0000 7f2989052740 20 client.18492 trim_cache size 0 max 16384 2024-04-02T14:03:25.756 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:03:25.756 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "scrub active (2388 inodes in the stack)", 2024-04-02T14:03:25.756 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-04-02T14:03:25.757 INFO:teuthology.orchestra.run.smithi052.stdout: "d7671a4c-1fd0-4cb0-a255-022487119dd6": { 2024-04-02T14:03:25.757 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-04-02T14:03:25.757 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "d7671a4c-1fd0-4cb0-a255-022487119dd6", 2024-04-02T14:03:25.757 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-04-02T14:03:25.757 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:03:25.757 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:03:25.757 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:03:25.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.743+0000 7f2986463640 1 -- 172.21.15.52:0/4014402137 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f29680a3d50 msgr2=0x7f29680a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:25.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.743+0000 7f2986463640 1 --2- 172.21.15.52:0/4014402137 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f29680a3d50 0x7f29680a6210 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2978009430 tx=0x7f2978009360 comp rx=0 tx=0).stop 2024-04-02T14:03:25.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.743+0000 7f2986463640 1 -- 172.21.15.52:0/4014402137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2980101340 msgr2=0x7f2980199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:25.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.743+0000 7f2986463640 1 --2- 172.21.15.52:0/4014402137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2980101340 0x7f2980199420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f297000e9f0 tx=0x7f297000eeb0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.743+0000 7f2986463640 1 -- 172.21.15.52:0/4014402137 shutdown_connections 2024-04-02T14:03:25.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.743+0000 7f2986463640 1 --2- 172.21.15.52:0/4014402137 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f29680a3d50 0x7f29680a6210 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.743+0000 7f2986463640 1 --2- 172.21.15.52:0/4014402137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2980199ea0 0x7f29801a44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.743+0000 7f2986463640 1 --2- 172.21.15.52:0/4014402137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2980105ef0 0x7f2980199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.743+0000 7f2986463640 1 --2- 172.21.15.52:0/4014402137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2980101340 0x7f2980199420 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:25.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.743+0000 7f2986463640 1 -- 172.21.15.52:0/4014402137 >> 172.21.15.52:0/4014402137 conn(0x7f2980076f20 msgr2=0x7f2980104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:25.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.743+0000 7f2986463640 1 -- 172.21.15.52:0/4014402137 shutdown_connections 2024-04-02T14:03:25.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:25.743+0000 7f2986463640 1 -- 172.21.15.52:0/4014402137 wait complete. 2024-04-02T14:03:25.760 INFO:tasks.cephfs.filesystem:scrub status for tag:d7671a4c-1fd0-4cb0-a255-022487119dd6 - {'path': '/', 'tag': 'd7671a4c-1fd0-4cb0-a255-022487119dd6', 'options': 'recursive,force'} 2024-04-02T14:03:25.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:25 smithi189 ceph-mon[36683]: pgmap v308: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:03:25.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:25 smithi088 ceph-mon[36528]: pgmap v308: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:03:26.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:26 smithi189 ceph-mon[36683]: scrub queued for path: / 2024-04-02T14:03:26.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:26 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:03:26.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:26 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T14:03:26.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:26 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T14:03:26.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:26 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:03:26.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:26 smithi189 ceph-mon[36683]: scrub summary: active paths [/] 2024-04-02T14:03:26.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:26 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:03:26.791 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:26 smithi189 ceph-mon[36683]: scrub summary: idle+waiting paths [/] 2024-04-02T14:03:26.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:26 smithi052 ceph-mon[32104]: scrub queued for path: / 2024-04-02T14:03:26.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:26 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:03:26.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:26 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T14:03:26.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:26 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T14:03:26.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:26 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:03:26.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:26 smithi052 ceph-mon[32104]: scrub summary: active paths [/] 2024-04-02T14:03:26.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:26 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:03:26.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:26 smithi052 ceph-mon[32104]: scrub summary: idle+waiting paths [/] 2024-04-02T14:03:26.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:26 smithi088 ceph-mon[36528]: scrub queued for path: / 2024-04-02T14:03:26.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:26 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:03:26.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:26 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T14:03:26.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:26 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T14:03:26.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:26 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:03:26.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:26 smithi088 ceph-mon[36528]: scrub summary: active paths [/] 2024-04-02T14:03:26.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:26 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:03:26.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:26 smithi088 ceph-mon[36528]: scrub summary: idle+waiting paths [/] 2024-04-02T14:03:27.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:27 smithi189 ceph-mon[36683]: pgmap v309: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-02T14:03:27.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:27 smithi052 ceph-mon[32104]: pgmap v309: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-02T14:03:27.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:27 smithi088 ceph-mon[36528]: pgmap v309: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-02T14:03:29.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:29 smithi189 ceph-mon[36683]: pgmap v310: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-02T14:03:29.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:29 smithi052 ceph-mon[32104]: pgmap v310: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-02T14:03:29.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:29 smithi088 ceph-mon[36528]: pgmap v310: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-02T14:03:31.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:31 smithi189 ceph-mon[36683]: pgmap v311: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-02T14:03:31.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:31 smithi052 ceph-mon[32104]: pgmap v311: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-02T14:03:31.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:31 smithi088 ceph-mon[36528]: pgmap v311: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-02T14:03:33.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:33 smithi189 ceph-mon[36683]: pgmap v312: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 13 op/s 2024-04-02T14:03:33.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:33 smithi052 ceph-mon[32104]: pgmap v312: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 13 op/s 2024-04-02T14:03:33.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:33 smithi088 ceph-mon[36528]: pgmap v312: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 13 op/s 2024-04-02T14:03:35.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:35 smithi189 ceph-mon[36683]: pgmap v313: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:03:35.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:35 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T14:03:35.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:35 smithi052 ceph-mon[32104]: pgmap v313: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:03:35.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:35 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T14:03:35.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:35 smithi088 ceph-mon[36528]: pgmap v313: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-02T14:03:35.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:35 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T14:03:36.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:36 smithi189 ceph-mon[36683]: scrub summary: idle 2024-04-02T14:03:36.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:36 smithi052 ceph-mon[32104]: scrub summary: idle 2024-04-02T14:03:36.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:36 smithi088 ceph-mon[36528]: scrub summary: idle 2024-04-02T14:03:37.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:37 smithi189 ceph-mon[36683]: pgmap v314: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T14:03:37.821 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:37 smithi088 ceph-mon[36528]: pgmap v314: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T14:03:37.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:37 smithi052 ceph-mon[32104]: pgmap v314: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-04-02T14:03:39.614 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-02T14:03:39.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.726+0000 7f0144138640 1 -- 172.21.15.52:0/1246968577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f013c06a860 msgr2=0x7f013c06aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:39.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.726+0000 7f0144138640 1 --2- 172.21.15.52:0/1246968577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f013c06a860 0x7f013c06aca0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f012c009900 tx=0x7f012c02bcd0 comp rx=0 tx=0).stop 2024-04-02T14:03:39.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.726+0000 7f0144138640 1 -- 172.21.15.52:0/1246968577 shutdown_connections 2024-04-02T14:03:39.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.726+0000 7f0144138640 1 --2- 172.21.15.52:0/1246968577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f013c10e2c0 0x7f013c110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:39.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.726+0000 7f0144138640 1 --2- 172.21.15.52:0/1246968577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f013c06a860 0x7f013c06aca0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:39.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.726+0000 7f0144138640 1 --2- 172.21.15.52:0/1246968577 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f013c102a80 0x7f013c06a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:39.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.726+0000 7f0144138640 1 -- 172.21.15.52:0/1246968577 >> 172.21.15.52:0/1246968577 conn(0x7f013c077850 msgr2=0x7f013c077c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:39.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.726+0000 7f0144138640 1 -- 172.21.15.52:0/1246968577 shutdown_connections 2024-04-02T14:03:39.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.726+0000 7f0144138640 1 -- 172.21.15.52:0/1246968577 wait complete. 2024-04-02T14:03:39.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.727+0000 7f0144138640 1 Processor -- start 2024-04-02T14:03:39.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.727+0000 7f0144138640 1 -- start start 2024-04-02T14:03:39.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.727+0000 7f0144138640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f013c06a860 0x7f013c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:39.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.727+0000 7f0144138640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f013c102a80 0x7f013c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:39.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.727+0000 7f0144138640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f013c10e2c0 0x7f013c19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:39.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.727+0000 7f0144138640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f013c114090 con 0x7f013c10e2c0 2024-04-02T14:03:39.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.727+0000 7f0144138640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f013c113f10 con 0x7f013c06a860 2024-04-02T14:03:39.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.727+0000 7f0144138640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f013c114210 con 0x7f013c102a80 2024-04-02T14:03:39.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.727+0000 7f01426d5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f013c10e2c0 0x7f013c19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:39.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.727+0000 7f01426d5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f013c10e2c0 0x7f013c19ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39100/0 (socket says 172.21.15.52:39100) 2024-04-02T14:03:39.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.727+0000 7f01426d5640 1 -- 172.21.15.52:0/1889692105 learned_addr learned my addr 172.21.15.52:0/1889692105 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:39.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.727+0000 7f01416d3640 1 --2- 172.21.15.52:0/1889692105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f013c102a80 0x7f013c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:39.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.727+0000 7f0141ed4640 1 --2- 172.21.15.52:0/1889692105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f013c06a860 0x7f013c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:39.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.727+0000 7f01426d5640 1 -- 172.21.15.52:0/1889692105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f013c102a80 msgr2=0x7f013c19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:39.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.727+0000 7f01426d5640 1 --2- 172.21.15.52:0/1889692105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f013c102a80 0x7f013c19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:39.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.727+0000 7f01426d5640 1 -- 172.21.15.52:0/1889692105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f013c06a860 msgr2=0x7f013c19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:39.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.727+0000 7f01426d5640 1 --2- 172.21.15.52:0/1889692105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f013c06a860 0x7f013c19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:39.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.727+0000 7f01426d5640 1 -- 172.21.15.52:0/1889692105 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013c19f430 con 0x7f013c10e2c0 2024-04-02T14:03:39.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.728+0000 7f01426d5640 1 --2- 172.21.15.52:0/1889692105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f013c10e2c0 0x7f013c19ed00 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f013400e960 tx=0x7f013400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:39.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.728+0000 7f0132ffd640 1 -- 172.21.15.52:0/1889692105 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0134004d20 con 0x7f013c10e2c0 2024-04-02T14:03:39.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.728+0000 7f0132ffd640 1 -- 172.21.15.52:0/1889692105 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0134005020 con 0x7f013c10e2c0 2024-04-02T14:03:39.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.728+0000 7f0144138640 1 -- 172.21.15.52:0/1889692105 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013c19f720 con 0x7f013c10e2c0 2024-04-02T14:03:39.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.728+0000 7f0132ffd640 1 -- 172.21.15.52:0/1889692105 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01340116d0 con 0x7f013c10e2c0 2024-04-02T14:03:39.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.728+0000 7f0144138640 1 -- 172.21.15.52:0/1889692105 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f013c1a6fb0 con 0x7f013c10e2c0 2024-04-02T14:03:39.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.729+0000 7f0144138640 1 -- 172.21.15.52:0/1889692105 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f013c100ff0 con 0x7f013c10e2c0 2024-04-02T14:03:39.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.730+0000 7f0132ffd640 1 -- 172.21.15.52:0/1889692105 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0134005560 con 0x7f013c10e2c0 2024-04-02T14:03:39.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.731+0000 7f0132ffd640 1 --2- 172.21.15.52:0/1889692105 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f01280a3dc0 0x7f01280a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:39.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.731+0000 7f0141ed4640 1 --2- 172.21.15.52:0/1889692105 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f01280a3dc0 0x7f01280a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:39.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.731+0000 7f0132ffd640 1 -- 172.21.15.52:0/1889692105 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f01340d10f0 con 0x7f013c10e2c0 2024-04-02T14:03:39.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.731+0000 7f0141ed4640 1 --2- 172.21.15.52:0/1889692105 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f01280a3dc0 0x7f01280a6280 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f01380058c0 tx=0x7f013800a2d0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:39.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.734+0000 7f0132ffd640 1 -- 172.21.15.52:0/1889692105 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0134016030 con 0x7f013c10e2c0 2024-04-02T14:03:39.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:39 smithi189 ceph-mon[36683]: pgmap v315: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:03:39.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:39 smithi052 ceph-mon[32104]: pgmap v315: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:03:39.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:39 smithi088 ceph-mon[36528]: pgmap v315: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:03:39.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.941+0000 7f0144138640 1 -- 172.21.15.52:0/1889692105 --> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f013c100350 con 0x7f01280a3dc0 2024-04-02T14:03:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.949+0000 7f0132ffd640 1 -- 172.21.15.52:0/1889692105 <== mgr.25160 v2:172.21.15.52:6800/1019901931 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f013c100350 con 0x7f01280a3dc0 2024-04-02T14:03:39.953 INFO:teuthology.orchestra.run.smithi052.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-02T00:00:00", "created": "2024-04-02T13:53:38", "first": "2024-04-02T13:54:00", "last": "2024-04-02T14:03:00", "last_pruned": "2024-04-02T14:03:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-02T14:03:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.951+0000 7f0144138640 1 -- 172.21.15.52:0/1889692105 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f01280a3dc0 msgr2=0x7f01280a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.951+0000 7f0144138640 1 --2- 172.21.15.52:0/1889692105 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f01280a3dc0 0x7f01280a6280 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f01380058c0 tx=0x7f013800a2d0 comp rx=0 tx=0).stop 2024-04-02T14:03:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.951+0000 7f0144138640 1 -- 172.21.15.52:0/1889692105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f013c10e2c0 msgr2=0x7f013c19ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:39.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.951+0000 7f0144138640 1 --2- 172.21.15.52:0/1889692105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f013c10e2c0 0x7f013c19ed00 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f013400e960 tx=0x7f013400ee20 comp rx=0 tx=0).stop 2024-04-02T14:03:39.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.951+0000 7f0144138640 1 -- 172.21.15.52:0/1889692105 shutdown_connections 2024-04-02T14:03:39.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.951+0000 7f0144138640 1 --2- 172.21.15.52:0/1889692105 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f01280a3dc0 0x7f01280a6280 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:39.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.951+0000 7f0144138640 1 --2- 172.21.15.52:0/1889692105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f013c10e2c0 0x7f013c19ed00 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:39.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.951+0000 7f0144138640 1 --2- 172.21.15.52:0/1889692105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f013c102a80 0x7f013c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:39.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.951+0000 7f0144138640 1 --2- 172.21.15.52:0/1889692105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f013c06a860 0x7f013c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:39.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.951+0000 7f0144138640 1 -- 172.21.15.52:0/1889692105 >> 172.21.15.52:0/1889692105 conn(0x7f013c077850 msgr2=0x7f013c0ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:39.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.951+0000 7f0144138640 1 -- 172.21.15.52:0/1889692105 shutdown_connections 2024-04-02T14:03:39.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:39.951+0000 7f0144138640 1 -- 172.21.15.52:0/1889692105 wait complete. 2024-04-02T14:03:39.965 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-02T14:03:40.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.066+0000 7f7a8b44e640 1 -- 172.21.15.52:0/4236406601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a84107bd0 msgr2=0x7f7a84107fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:40.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.066+0000 7f7a8b44e640 1 --2- 172.21.15.52:0/4236406601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a84107bd0 0x7f7a84107fb0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009930 tx=0x7f7a7c02bc30 comp rx=0 tx=0).stop 2024-04-02T14:03:40.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.067+0000 7f7a8b44e640 1 -- 172.21.15.52:0/4236406601 shutdown_connections 2024-04-02T14:03:40.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.067+0000 7f7a8b44e640 1 --2- 172.21.15.52:0/4236406601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a84069b20 0x7f7a8410e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:40.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.067+0000 7f7a8b44e640 1 --2- 172.21.15.52:0/4236406601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a84069180 0x7f7a840695e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:40.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.067+0000 7f7a8b44e640 1 --2- 172.21.15.52:0/4236406601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a84107bd0 0x7f7a84107fb0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:40.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.067+0000 7f7a8b44e640 1 -- 172.21.15.52:0/4236406601 >> 172.21.15.52:0/4236406601 conn(0x7f7a84077850 msgr2=0x7f7a84077c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:40.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.067+0000 7f7a8b44e640 1 -- 172.21.15.52:0/4236406601 shutdown_connections 2024-04-02T14:03:40.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.067+0000 7f7a8b44e640 1 -- 172.21.15.52:0/4236406601 wait complete. 2024-04-02T14:03:40.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.067+0000 7f7a8b44e640 1 Processor -- start 2024-04-02T14:03:40.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.067+0000 7f7a8b44e640 1 -- start start 2024-04-02T14:03:40.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.067+0000 7f7a8b44e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a84069180 0x7f7a841982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:40.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.067+0000 7f7a8b44e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a84069b20 0x7f7a84198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:40.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.067+0000 7f7a8b44e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a84107bd0 0x7f7a8419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:40.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.067+0000 7f7a8b44e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7a84111ec0 con 0x7f7a84069180 2024-04-02T14:03:40.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.067+0000 7f7a8b44e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7a84111d40 con 0x7f7a84069b20 2024-04-02T14:03:40.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.067+0000 7f7a8b44e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7a84112040 con 0x7f7a84107bd0 2024-04-02T14:03:40.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.067+0000 7f7a891ea640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a84069180 0x7f7a841982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:40.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.068+0000 7f7a891ea640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a84069180 0x7f7a841982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39122/0 (socket says 172.21.15.52:39122) 2024-04-02T14:03:40.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.068+0000 7f7a891ea640 1 -- 172.21.15.52:0/1402152206 learned_addr learned my addr 172.21.15.52:0/1402152206 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:40.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.068+0000 7f7a899eb640 1 --2- 172.21.15.52:0/1402152206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a84107bd0 0x7f7a8419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:40.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.068+0000 7f7a891ea640 1 -- 172.21.15.52:0/1402152206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a84107bd0 msgr2=0x7f7a8419cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:40.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.068+0000 7f7a891ea640 1 --2- 172.21.15.52:0/1402152206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a84107bd0 0x7f7a8419cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:40.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.068+0000 7f7a889e9640 1 --2- 172.21.15.52:0/1402152206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a84069b20 0x7f7a84198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:40.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.068+0000 7f7a891ea640 1 -- 172.21.15.52:0/1402152206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a84069b20 msgr2=0x7f7a84198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:40.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.068+0000 7f7a891ea640 1 --2- 172.21.15.52:0/1402152206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a84069b20 0x7f7a84198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:40.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.068+0000 7f7a891ea640 1 -- 172.21.15.52:0/1402152206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8419d2d0 con 0x7f7a84069180 2024-04-02T14:03:40.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.068+0000 7f7a889e9640 1 --2- 172.21.15.52:0/1402152206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a84069b20 0x7f7a84198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:03:40.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.068+0000 7f7a891ea640 1 --2- 172.21.15.52:0/1402152206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a84069180 0x7f7a841982e0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009230 tx=0x7f7a7c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:40.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.068+0000 7f7a7a7fc640 1 -- 172.21.15.52:0/1402152206 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c041070 con 0x7f7a84069180 2024-04-02T14:03:40.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.068+0000 7f7a7a7fc640 1 -- 172.21.15.52:0/1402152206 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a7c02ccc0 con 0x7f7a84069180 2024-04-02T14:03:40.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.068+0000 7f7a7a7fc640 1 -- 172.21.15.52:0/1402152206 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c003de0 con 0x7f7a84069180 2024-04-02T14:03:40.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.068+0000 7f7a8b44e640 1 -- 172.21.15.52:0/1402152206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8419d560 con 0x7f7a84069180 2024-04-02T14:03:40.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.068+0000 7f7a8b44e640 1 -- 172.21.15.52:0/1402152206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a841a4e50 con 0x7f7a84069180 2024-04-02T14:03:40.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.069+0000 7f7a8b44e640 1 -- 172.21.15.52:0/1402152206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a84103200 con 0x7f7a84069180 2024-04-02T14:03:40.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.070+0000 7f7a7a7fc640 1 -- 172.21.15.52:0/1402152206 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7a7c044070 con 0x7f7a84069180 2024-04-02T14:03:40.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.071+0000 7f7a7a7fc640 1 --2- 172.21.15.52:0/1402152206 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f7a700a3dc0 0x7f7a700a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:40.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.071+0000 7f7a889e9640 1 --2- 172.21.15.52:0/1402152206 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f7a700a3dc0 0x7f7a700a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:40.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.071+0000 7f7a7a7fc640 1 -- 172.21.15.52:0/1402152206 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f7a7c0f28d0 con 0x7f7a84069180 2024-04-02T14:03:40.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.071+0000 7f7a889e9640 1 --2- 172.21.15.52:0/1402152206 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f7a700a3dc0 0x7f7a700a6280 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7a84077dd0 tx=0x7f7a74005f90 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:40.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.075+0000 7f7a7a7fc640 1 -- 172.21.15.52:0/1402152206 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7a7c0becf0 con 0x7f7a84069180 2024-04-02T14:03:40.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.291+0000 7f7a8b44e640 1 -- 172.21.15.52:0/1402152206 --> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f7a84061f40 con 0x7f7a700a3dc0 2024-04-02T14:03:40.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.298+0000 7f7a7a7fc640 1 -- 172.21.15.52:0/1402152206 <== mgr.25160 v2:172.21.15.52:6800/1019901931 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f7a84061f40 con 0x7f7a700a3dc0 2024-04-02T14:03:40.302 INFO:teuthology.orchestra.run.smithi052.stdout:/ 1m 6m3h 2024-04-02T14:03:40.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.300+0000 7f7a8b44e640 1 -- 172.21.15.52:0/1402152206 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f7a700a3dc0 msgr2=0x7f7a700a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:40.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.300+0000 7f7a8b44e640 1 --2- 172.21.15.52:0/1402152206 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f7a700a3dc0 0x7f7a700a6280 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7a84077dd0 tx=0x7f7a74005f90 comp rx=0 tx=0).stop 2024-04-02T14:03:40.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.300+0000 7f7a8b44e640 1 -- 172.21.15.52:0/1402152206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a84069180 msgr2=0x7f7a841982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:40.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.300+0000 7f7a8b44e640 1 --2- 172.21.15.52:0/1402152206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a84069180 0x7f7a841982e0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009230 tx=0x7f7a7c0043b0 comp rx=0 tx=0).stop 2024-04-02T14:03:40.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.300+0000 7f7a8b44e640 1 -- 172.21.15.52:0/1402152206 shutdown_connections 2024-04-02T14:03:40.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.300+0000 7f7a8b44e640 1 --2- 172.21.15.52:0/1402152206 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f7a700a3dc0 0x7f7a700a6280 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:40.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.300+0000 7f7a8b44e640 1 --2- 172.21.15.52:0/1402152206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a84107bd0 0x7f7a8419cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:40.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.300+0000 7f7a8b44e640 1 --2- 172.21.15.52:0/1402152206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a84069b20 0x7f7a84198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:40.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.300+0000 7f7a8b44e640 1 --2- 172.21.15.52:0/1402152206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a84069180 0x7f7a841982e0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:40.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.300+0000 7f7a8b44e640 1 -- 172.21.15.52:0/1402152206 >> 172.21.15.52:0/1402152206 conn(0x7f7a84077850 msgr2=0x7f7a840fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:40.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.301+0000 7f7a8b44e640 1 -- 172.21.15.52:0/1402152206 shutdown_connections 2024-04-02T14:03:40.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:40.301+0000 7f7a8b44e640 1 -- 172.21.15.52:0/1402152206 wait complete. 2024-04-02T14:03:40.315 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-02T14:03:40.328 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-02T14:03:41.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:41 smithi189 ceph-mon[36683]: from='client.18498 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:03:41.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:41 smithi189 ceph-mon[36683]: pgmap v316: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:03:41.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:41 smithi189 ceph-mon[36683]: from='client.18504 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:03:41.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:41 smithi052 ceph-mon[32104]: from='client.18498 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:03:41.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:41 smithi052 ceph-mon[32104]: pgmap v316: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:03:41.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:41 smithi052 ceph-mon[32104]: from='client.18504 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:03:41.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:41 smithi088 ceph-mon[36528]: from='client.18498 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:03:41.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:41 smithi088 ceph-mon[36528]: pgmap v316: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:03:41.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:41 smithi088 ceph-mon[36528]: from='client.18504 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T14:03:43.785 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:43 smithi088 ceph-mon[36528]: pgmap v317: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-02T14:03:43.785 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:43 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:03:43.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:43 smithi189 ceph-mon[36683]: pgmap v317: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-02T14:03:43.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:43 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:03:43.801 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:43 smithi052 ceph-mon[32104]: pgmap v317: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-02T14:03:43.801 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:43 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T14:03:45.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:45 smithi189 ceph-mon[36683]: pgmap v318: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-02T14:03:45.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:45 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:03:45.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:45 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:03:45.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:45 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:03:45.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:45 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:03:45.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:45 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:03:45.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:45 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:03:45.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:45 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:45 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:45 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:45 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:45 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:45 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.791 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:45 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.791 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:45 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.791 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:45 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.791 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:45 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.791 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:45 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.791 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:45 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.791 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:45 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:03:45.791 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:45 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:03:45.792 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:45 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:03:45.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:45 smithi052 ceph-mon[32104]: pgmap v318: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-02T14:03:45.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:45 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:03:45.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:45 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:03:45.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:45 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:03:45.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:45 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:03:45.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:45 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:03:45.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:45 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:03:45.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:45 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:45 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:45 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:45 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:45 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:45 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:45 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:45 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:45 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:45 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.843 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:45 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.843 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:45 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.843 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:45 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:03:45.843 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:45 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:03:45.843 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:45 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:03:45.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:45 smithi088 ceph-mon[36528]: pgmap v318: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-02T14:03:45.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:45 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:03:45.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:45 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:03:45.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:45 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:03:45.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:45 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:03:45.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:45 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:03:45.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:45 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:03:45.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:45 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:45 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:45 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:45 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:45 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:45 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:45 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:45 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:45 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.923 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:45 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.923 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:45 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.923 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:45 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T14:03:45.923 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:45 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T14:03:45.923 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:45 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T14:03:45.923 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:45 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' 2024-04-02T14:03:46.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:46 smithi189 ceph-mon[36683]: Adjusting osd_memory_target on smithi189 to 174.1M 2024-04-02T14:03:46.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:46 smithi189 ceph-mon[36683]: Unable to set osd_memory_target on smithi189 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-04-02T14:03:46.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:46 smithi189 ceph-mon[36683]: Adjusting osd_memory_target on smithi088 to 174.1M 2024-04-02T14:03:46.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:46 smithi189 ceph-mon[36683]: Unable to set osd_memory_target on smithi088 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-04-02T14:03:46.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:46 smithi189 ceph-mon[36683]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-04-02T14:03:46.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:46 smithi189 ceph-mon[36683]: Unable to set osd_memory_target on smithi052 to 185898496: error parsing value: Value '185898496' is below minimum 939524096 2024-04-02T14:03:46.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:46 smithi052 ceph-mon[32104]: Adjusting osd_memory_target on smithi189 to 174.1M 2024-04-02T14:03:46.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:46 smithi052 ceph-mon[32104]: Unable to set osd_memory_target on smithi189 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-04-02T14:03:46.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:46 smithi052 ceph-mon[32104]: Adjusting osd_memory_target on smithi088 to 174.1M 2024-04-02T14:03:46.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:46 smithi052 ceph-mon[32104]: Unable to set osd_memory_target on smithi088 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-04-02T14:03:46.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:46 smithi052 ceph-mon[32104]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-04-02T14:03:46.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:46 smithi052 ceph-mon[32104]: Unable to set osd_memory_target on smithi052 to 185898496: error parsing value: Value '185898496' is below minimum 939524096 2024-04-02T14:03:46.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:46 smithi088 ceph-mon[36528]: Adjusting osd_memory_target on smithi189 to 174.1M 2024-04-02T14:03:46.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:46 smithi088 ceph-mon[36528]: Unable to set osd_memory_target on smithi189 to 182635622: error parsing value: Value '182635622' is below minimum 939524096 2024-04-02T14:03:46.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:46 smithi088 ceph-mon[36528]: Adjusting osd_memory_target on smithi088 to 174.1M 2024-04-02T14:03:46.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:46 smithi088 ceph-mon[36528]: Unable to set osd_memory_target on smithi088 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-04-02T14:03:46.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:46 smithi088 ceph-mon[36528]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-04-02T14:03:46.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:46 smithi088 ceph-mon[36528]: Unable to set osd_memory_target on smithi052 to 185898496: error parsing value: Value '185898496' is below minimum 939524096 2024-04-02T14:03:47.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:47 smithi189 ceph-mon[36683]: pgmap v319: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-02T14:03:47.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:47 smithi052 ceph-mon[32104]: pgmap v319: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-02T14:03:47.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:47 smithi088 ceph-mon[36528]: pgmap v319: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-02T14:03:49.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:49 smithi189 ceph-mon[36683]: pgmap v320: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-02T14:03:49.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:49 smithi052 ceph-mon[32104]: pgmap v320: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-02T14:03:49.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:49 smithi088 ceph-mon[36528]: pgmap v320: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-02T14:03:51.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:51 smithi189 ceph-mon[36683]: pgmap v321: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-02T14:03:51.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:51 smithi052 ceph-mon[32104]: pgmap v321: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-02T14:03:51.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:51 smithi088 ceph-mon[36528]: pgmap v321: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-02T14:03:53.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:53 smithi189 ceph-mon[36683]: pgmap v322: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-02T14:03:53.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:53 smithi052 ceph-mon[32104]: pgmap v322: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-02T14:03:53.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:53 smithi088 ceph-mon[36528]: pgmap v322: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-02T14:03:55.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T14:03:55.761 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T14:03:55.788 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:55 smithi052 ceph-mon[32104]: pgmap v323: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:03:55.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:55 smithi189 ceph-mon[36683]: pgmap v323: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:03:55.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.868+0000 7f46d4baf640 1 -- 172.21.15.52:0/2530055513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46d0107b50 msgr2=0x7f46d0107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:55.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.868+0000 7f46d4baf640 1 --2- 172.21.15.52:0/2530055513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46d0107b50 0x7f46d0107f30 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009930 tx=0x7f46c002bc30 comp rx=0 tx=0).stop 2024-04-02T14:03:55.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.868+0000 7f46d4baf640 1 -- 172.21.15.52:0/2530055513 shutdown_connections 2024-04-02T14:03:55.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.868+0000 7f46d4baf640 1 --2- 172.21.15.52:0/2530055513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46d00696c0 0x7f46d010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.868+0000 7f46d4baf640 1 --2- 172.21.15.52:0/2530055513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46d0068d00 0x7f46d0069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.868+0000 7f46d4baf640 1 --2- 172.21.15.52:0/2530055513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46d0107b50 0x7f46d0107f30 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.868+0000 7f46d4baf640 1 -- 172.21.15.52:0/2530055513 >> 172.21.15.52:0/2530055513 conn(0x7f46d0077b00 msgr2=0x7f46d0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:55.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.868+0000 7f46d4baf640 1 -- 172.21.15.52:0/2530055513 shutdown_connections 2024-04-02T14:03:55.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.868+0000 7f46d4baf640 1 -- 172.21.15.52:0/2530055513 wait complete. 2024-04-02T14:03:55.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.869+0000 7f46d4baf640 1 Processor -- start 2024-04-02T14:03:55.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.869+0000 7f46d4baf640 1 -- start start 2024-04-02T14:03:55.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.870+0000 7f46d4baf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46d0068d00 0x7f46d01009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:55.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.870+0000 7f46d4baf640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46d00696c0 0x7f46d0100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:55.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.870+0000 7f46d4baf640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46d0107b50 0x7f46d0101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:55.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.870+0000 7f46d4baf640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f46d0111ea0 con 0x7f46d0068d00 2024-04-02T14:03:55.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.870+0000 7f46d4baf640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f46d0111d20 con 0x7f46d00696c0 2024-04-02T14:03:55.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.870+0000 7f46d4baf640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f46d0112020 con 0x7f46d0107b50 2024-04-02T14:03:55.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.870+0000 7f46ce59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46d0068d00 0x7f46d01009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:55.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.870+0000 7f46ce59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46d0068d00 0x7f46d01009c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33016/0 (socket says 172.21.15.52:33016) 2024-04-02T14:03:55.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.870+0000 7f46ce59c640 1 -- 172.21.15.52:0/925711339 learned_addr learned my addr 172.21.15.52:0/925711339 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:55.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.870+0000 7f46c5d9b640 1 --2- 172.21.15.52:0/925711339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46d00696c0 0x7f46d0100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:55.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.870+0000 7f46ced9d640 1 --2- 172.21.15.52:0/925711339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46d0107b50 0x7f46d0101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:55.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.870+0000 7f46ce59c640 1 -- 172.21.15.52:0/925711339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46d0107b50 msgr2=0x7f46d0101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:55.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.870+0000 7f46ce59c640 1 --2- 172.21.15.52:0/925711339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46d0107b50 0x7f46d0101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.870+0000 7f46ce59c640 1 -- 172.21.15.52:0/925711339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46d00696c0 msgr2=0x7f46d0100f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:55.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.870+0000 7f46ce59c640 1 --2- 172.21.15.52:0/925711339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46d00696c0 0x7f46d0100f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.870+0000 7f46ce59c640 1 -- 172.21.15.52:0/925711339 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d01a6880 con 0x7f46d0068d00 2024-04-02T14:03:55.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.870+0000 7f46ce59c640 1 --2- 172.21.15.52:0/925711339 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46d0068d00 0x7f46d01009c0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009230 tx=0x7f46c0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:55.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.870+0000 7f46c7fff640 1 -- 172.21.15.52:0/925711339 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0041070 con 0x7f46d0068d00 2024-04-02T14:03:55.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.870+0000 7f46d4baf640 1 -- 172.21.15.52:0/925711339 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d01a6b10 con 0x7f46d0068d00 2024-04-02T14:03:55.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.871+0000 7f46c7fff640 1 -- 172.21.15.52:0/925711339 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46c00054e0 con 0x7f46d0068d00 2024-04-02T14:03:55.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.871+0000 7f46c7fff640 1 -- 172.21.15.52:0/925711339 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0003c90 con 0x7f46d0068d00 2024-04-02T14:03:55.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.871+0000 7f46d4baf640 1 -- 172.21.15.52:0/925711339 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d01a7050 con 0x7f46d0068d00 2024-04-02T14:03:55.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.873+0000 7f46c7fff640 1 -- 172.21.15.52:0/925711339 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f46c0044070 con 0x7f46d0068d00 2024-04-02T14:03:55.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.873+0000 7f46d7798740 1 Processor -- start 2024-04-02T14:03:55.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.873+0000 7f46d7798740 1 -- start start 2024-04-02T14:03:55.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.873+0000 7f46d7798740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a67a8e8d0 0x558a67a8ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:55.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.873+0000 7f46d7798740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558a67a8f1f0 0x558a67a97890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:55.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.873+0000 7f46d7798740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a97dd0 0x558a67a9a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:55.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.873+0000 7f46d7798740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558a679f2830 con 0x558a67a97dd0 2024-04-02T14:03:55.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.873+0000 7f46d7798740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x558a6799a820 con 0x558a67a8f1f0 2024-04-02T14:03:55.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.873+0000 7f46d7798740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558a679ee7e0 con 0x558a67a8e8d0 2024-04-02T14:03:55.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.874+0000 7f46c7fff640 1 --2- 172.21.15.52:0/925711339 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f46bc0a3d50 0x7f46bc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:55.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.874+0000 7f46c5d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a97dd0 0x558a67a9a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:55.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.874+0000 7f46c5d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a97dd0 0x558a67a9a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33028/0 (socket says 172.21.15.52:33028) 2024-04-02T14:03:55.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.874+0000 7f46ced9d640 1 --2- 172.21.15.52:0/925711339 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f46bc0a3d50 0x7f46bc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:55.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.874+0000 7f46c5d9b640 1 -- 172.21.15.52:0/3713087659 learned_addr learned my addr 172.21.15.52:0/3713087659 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:55.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.874+0000 7f46c7fff640 1 -- 172.21.15.52:0/925711339 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f46c00f26f0 con 0x7f46d0068d00 2024-04-02T14:03:55.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.874+0000 7f46ce59c640 1 --2- 172.21.15.52:0/3713087659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558a67a8f1f0 0x558a67a97890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:55.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.874+0000 7f46ced9d640 1 --2- 172.21.15.52:0/3713087659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a67a8e8d0 0x558a67a8ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:55.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.874+0000 7f46c5d9b640 1 -- 172.21.15.52:0/3713087659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a67a8e8d0 msgr2=0x558a67a8ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:55.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.874+0000 7f46c5d9b640 1 --2- 172.21.15.52:0/3713087659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a67a8e8d0 0x558a67a8ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.874+0000 7f46c5d9b640 1 -- 172.21.15.52:0/3713087659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558a67a8f1f0 msgr2=0x558a67a97890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:55.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.874+0000 7f46c5d9b640 1 --2- 172.21.15.52:0/3713087659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558a67a8f1f0 0x558a67a97890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.874+0000 7f46c5d9b640 1 -- 172.21.15.52:0/3713087659 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a677d07d0 con 0x558a67a97dd0 2024-04-02T14:03:55.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.874+0000 7f46ced9d640 1 --2- 172.21.15.52:0/925711339 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f46bc0a3d50 0x7f46bc0a6210 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009270 tx=0x7f46c8004ab0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:55.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.874+0000 7f46c5d9b640 1 --2- 172.21.15.52:0/3713087659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a97dd0 0x558a67a9a1c0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009ca0 tx=0x7f46b8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3226ceee5b5959e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:55.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.874+0000 7f46c4d99640 1 -- 172.21.15.52:0/3713087659 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b801e070 con 0x558a67a97dd0 2024-04-02T14:03:55.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.874+0000 7f46c4d99640 1 -- 172.21.15.52:0/3713087659 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46b8019b10 con 0x558a67a97dd0 2024-04-02T14:03:55.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.874+0000 7f46c4d99640 1 -- 172.21.15.52:0/3713087659 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b800eae0 con 0x558a67a97dd0 2024-04-02T14:03:55.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.875+0000 7f46d7798740 1 -- 172.21.15.52:0/3713087659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a97dd0 msgr2=0x558a67a9a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:55.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.875+0000 7f46d7798740 1 --2- 172.21.15.52:0/3713087659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a97dd0 0x558a67a9a1c0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009ca0 tx=0x7f46b8011810 comp rx=0 tx=0).stop 2024-04-02T14:03:55.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.875+0000 7f46d7798740 1 -- 172.21.15.52:0/3713087659 shutdown_connections 2024-04-02T14:03:55.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.875+0000 7f46d7798740 1 --2- 172.21.15.52:0/3713087659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a97dd0 0x558a67a9a1c0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.875+0000 7f46d7798740 1 --2- 172.21.15.52:0/3713087659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558a67a8f1f0 0x558a67a97890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.875+0000 7f46d7798740 1 --2- 172.21.15.52:0/3713087659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a67a8e8d0 0x558a67a8ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.875+0000 7f46d7798740 1 -- 172.21.15.52:0/3713087659 >> 172.21.15.52:0/3713087659 conn(0x558a67a89000 msgr2=0x558a67a893e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:55.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.875+0000 7f46d7798740 1 -- 172.21.15.52:0/3713087659 shutdown_connections 2024-04-02T14:03:55.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.875+0000 7f46d7798740 1 -- 172.21.15.52:0/3713087659 wait complete. 2024-04-02T14:03:55.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.876+0000 7f46d7798740 1 -- start start 2024-04-02T14:03:55.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.876+0000 7f46d7798740 1 Processor -- start 2024-04-02T14:03:55.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.876+0000 7f46d7798740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a67a8e8d0 0x558a67a97660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:55.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.876+0000 7f46d7798740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558a67a8f1f0 0x558a67a949f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:55.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.876+0000 7f46d7798740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a97dd0 0x558a67a94f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:55.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.876+0000 7f46d7798740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558a67a9abc0 con 0x558a67a97dd0 2024-04-02T14:03:55.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.876+0000 7f46d7798740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x558a679add30 con 0x558a67a8f1f0 2024-04-02T14:03:55.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.876+0000 7f46d7798740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558a67a9ad40 con 0x558a67a8e8d0 2024-04-02T14:03:55.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.876+0000 7f46d7798740 10 client.0 _ll_register_callbacks cb 0x558a679ad720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:55.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.876+0000 7f46ced9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a97dd0 0x558a67a94f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:55.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.876+0000 7f46ced9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a97dd0 0x558a67a94f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33044/0 (socket says 172.21.15.52:33044) 2024-04-02T14:03:55.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.876+0000 7f46ced9d640 1 -- 172.21.15.52:0/3019117254 learned_addr learned my addr 172.21.15.52:0/3019117254 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:55.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.876+0000 7f46ce59c640 1 --2- 172.21.15.52:0/3019117254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a67a8e8d0 0x558a67a97660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:55.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.876+0000 7f46c5d9b640 1 --2- 172.21.15.52:0/3019117254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558a67a8f1f0 0x558a67a949f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:55.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.876+0000 7f46ced9d640 1 -- 172.21.15.52:0/3019117254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a67a8e8d0 msgr2=0x558a67a97660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:55.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.876+0000 7f46ced9d640 1 --2- 172.21.15.52:0/3019117254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a67a8e8d0 0x558a67a97660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.876+0000 7f46ced9d640 1 -- 172.21.15.52:0/3019117254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558a67a8f1f0 msgr2=0x558a67a949f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:55.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.876+0000 7f46ced9d640 1 --2- 172.21.15.52:0/3019117254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558a67a8f1f0 0x558a67a949f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.876+0000 7f46ced9d640 1 -- 172.21.15.52:0/3019117254 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a677063f0 con 0x558a67a97dd0 2024-04-02T14:03:55.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.877+0000 7f46ced9d640 1 --2- 172.21.15.52:0/3019117254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a97dd0 0x558a67a94f30 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f46c800db20 tx=0x7f46c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a742d6418dd3f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:55.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.877+0000 7f46b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:55.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.877+0000 7f46b5ffb640 1 -- 172.21.15.52:0/3019117254 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8003c40 con 0x558a67a97dd0 2024-04-02T14:03:55.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.877+0000 7f46d7798740 1 -- 172.21.15.52:0/3019117254 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46b8019b10 con 0x558a67a97dd0 2024-04-02T14:03:55.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.877+0000 7f46b5ffb640 1 -- 172.21.15.52:0/3019117254 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f46c8003de0 con 0x558a67a97dd0 2024-04-02T14:03:55.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.877+0000 7f46b5ffb640 1 -- 172.21.15.52:0/3019117254 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f46c8010930 con 0x558a67a97dd0 2024-04-02T14:03:55.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.877+0000 7f46d7798740 10 client.18516 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:55.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.877+0000 7f46d7798740 1 -- 172.21.15.52:0/3019117254 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558a676c60f0 con 0x558a67a97dd0 2024-04-02T14:03:55.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.877+0000 7f46b5ffb640 1 -- 172.21.15.52:0/3019117254 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46b8019b10 con 0x558a67a97dd0 2024-04-02T14:03:55.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.877+0000 7f46b5ffb640 1 -- 172.21.15.52:0/3019117254 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f46c802c4e0 con 0x558a67a97dd0 2024-04-02T14:03:55.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.877+0000 7f46d7798740 10 client.18516 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:55.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.877+0000 7f46d7798740 10 client.18516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:03:55.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.878+0000 7f46d7798740 20 client.18516 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:55.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.878+0000 7f46d7798740 1 --2- 172.21.15.52:0/3019117254 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x558a67aa7cd0 0x558a67aaa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:55.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.878+0000 7f46d7798740 4 client.18516 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:03:55.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.878+0000 7f46d7798740 1 -- 172.21.15.52:0/3019117254 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a679ffd70 con 0x558a67aa7cd0 2024-04-02T14:03:55.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.878+0000 7f46ce59c640 1 --2- 172.21.15.52:0/3019117254 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x558a67aa7cd0 0x558a67aaa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:55.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.878+0000 7f46ce59c640 1 --2- 172.21.15.52:0/3019117254 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x558a67aa7cd0 0x558a67aaa120 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27eb139e54fe31ea server_cookie=4e4ff3f38cf375d0 in_seq=0 out_seq=0 2024-04-02T14:03:55.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.878+0000 7f46b5ffb640 10 client.18516 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:03:55.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.879+0000 7f46b5ffb640 1 -- 172.21.15.52:0/3019117254 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558a679ffd70 con 0x558a67aa7cd0 2024-04-02T14:03:55.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.879+0000 7f46b5ffb640 10 client.18516 handle_command_reply: tid=0 2024-04-02T14:03:55.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.879+0000 7f46d7798740 1 client.18516 shutdown 2024-04-02T14:03:55.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.879+0000 7f46d7798740 1 -- 172.21.15.52:0/3019117254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a97dd0 msgr2=0x558a67a94f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:55.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.879+0000 7f46d7798740 1 --2- 172.21.15.52:0/3019117254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a97dd0 0x558a67a94f30 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f46c800db20 tx=0x7f46c8004a40 comp rx=0 tx=0).stop 2024-04-02T14:03:55.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.879+0000 7f46d7798740 1 -- 172.21.15.52:0/3019117254 shutdown_connections 2024-04-02T14:03:55.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.879+0000 7f46d7798740 1 --2- 172.21.15.52:0/3019117254 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x558a67aa7cd0 0x558a67aaa120 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.879+0000 7f46d7798740 1 --2- 172.21.15.52:0/3019117254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a97dd0 0x558a67a94f30 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.879+0000 7f46d7798740 1 --2- 172.21.15.52:0/3019117254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558a67a8f1f0 0x558a67a949f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.879+0000 7f46b5ffb640 0 client.18516 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:03:55.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.879+0000 7f46d7798740 1 --2- 172.21.15.52:0/3019117254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a67a8e8d0 0x558a67a97660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.879+0000 7f46d7798740 1 -- 172.21.15.52:0/3019117254 >> 172.21.15.52:0/3019117254 conn(0x558a67a89000 msgr2=0x558a67a8ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:55.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.879+0000 7f46d7798740 1 -- 172.21.15.52:0/3019117254 shutdown_connections 2024-04-02T14:03:55.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.879+0000 7f46d7798740 1 -- 172.21.15.52:0/3019117254 wait complete. 2024-04-02T14:03:55.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.879+0000 7f46d7798740 20 client.18516 trim_cache size 0 max 16384 2024-04-02T14:03:55.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.897+0000 7f46d7798740 1 Processor -- start 2024-04-02T14:03:55.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.897+0000 7f46d7798740 1 -- start start 2024-04-02T14:03:55.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.897+0000 7f46d7798740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558a67a8e8d0 0x558a67a97120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:55.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.897+0000 7f46d7798740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a67a8f1f0 0x558a67a97660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:55.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.897+0000 7f46d7798740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a97dd0 0x558a67aaf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:55.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.897+0000 7f46d7798740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f46d01b11b0 con 0x558a67a97dd0 2024-04-02T14:03:55.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.897+0000 7f46d7798740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f46d01b2cf0 con 0x558a67a8e8d0 2024-04-02T14:03:55.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.897+0000 7f46d7798740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f46d01b1880 con 0x558a67a8f1f0 2024-04-02T14:03:55.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.897+0000 7f46ced9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a97dd0 0x558a67aaf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:55.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.897+0000 7f46ced9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a97dd0 0x558a67aaf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33048/0 (socket says 172.21.15.52:33048) 2024-04-02T14:03:55.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.897+0000 7f46ced9d640 1 -- 172.21.15.52:0/304073870 learned_addr learned my addr 172.21.15.52:0/304073870 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:55.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.897+0000 7f46ce59c640 1 --2- 172.21.15.52:0/304073870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558a67a8e8d0 0x558a67a97120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:55.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.897+0000 7f46c5d9b640 1 --2- 172.21.15.52:0/304073870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a67a8f1f0 0x558a67a97660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:55.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.897+0000 7f46ced9d640 1 -- 172.21.15.52:0/304073870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a67a8f1f0 msgr2=0x558a67a97660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:55.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.897+0000 7f46ced9d640 1 --2- 172.21.15.52:0/304073870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a67a8f1f0 0x558a67a97660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.897+0000 7f46ced9d640 1 -- 172.21.15.52:0/304073870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558a67a8e8d0 msgr2=0x558a67a97120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:55.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.897+0000 7f46ced9d640 1 --2- 172.21.15.52:0/304073870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558a67a8e8d0 0x558a67a97120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.897+0000 7f46ced9d640 1 -- 172.21.15.52:0/304073870 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a67a88e30 con 0x558a67a97dd0 2024-04-02T14:03:55.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.898+0000 7f46ced9d640 1 --2- 172.21.15.52:0/304073870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a97dd0 0x558a67aaf9f0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009d00 tx=0x7f46c8005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c36eff664ad3168 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:55.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.898+0000 7f46b7fff640 1 -- 172.21.15.52:0/304073870 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8012980 con 0x558a67a97dd0 2024-04-02T14:03:55.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.898+0000 7f46b7fff640 1 -- 172.21.15.52:0/304073870 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46c8025030 con 0x558a67a97dd0 2024-04-02T14:03:55.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.898+0000 7f46b7fff640 1 -- 172.21.15.52:0/304073870 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c802c310 con 0x558a67a97dd0 2024-04-02T14:03:55.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.898+0000 7f46d7798740 1 -- 172.21.15.52:0/304073870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a97dd0 msgr2=0x558a67aaf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:55.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.898+0000 7f46d7798740 1 --2- 172.21.15.52:0/304073870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a97dd0 0x558a67aaf9f0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009d00 tx=0x7f46c8005cf0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.899+0000 7f46d7798740 1 -- 172.21.15.52:0/304073870 shutdown_connections 2024-04-02T14:03:55.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.899+0000 7f46d7798740 1 --2- 172.21.15.52:0/304073870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a97dd0 0x558a67aaf9f0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.899+0000 7f46d7798740 1 --2- 172.21.15.52:0/304073870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a67a8f1f0 0x558a67a97660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.899+0000 7f46d7798740 1 --2- 172.21.15.52:0/304073870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558a67a8e8d0 0x558a67a97120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.899+0000 7f46d7798740 1 -- 172.21.15.52:0/304073870 >> 172.21.15.52:0/304073870 conn(0x558a67a89000 msgr2=0x558a67a8f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:55.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.899+0000 7f46d7798740 1 -- 172.21.15.52:0/304073870 shutdown_connections 2024-04-02T14:03:55.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.899+0000 7f46d7798740 1 -- 172.21.15.52:0/304073870 wait complete. 2024-04-02T14:03:55.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.899+0000 7f46d7798740 1 -- start start 2024-04-02T14:03:55.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.899+0000 7f46d7798740 1 Processor -- start 2024-04-02T14:03:55.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.899+0000 7f46d7798740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a67a8e8d0 0x558a67aa02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:55.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.899+0000 7f46d7798740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a8f1f0 0x558a67a9b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:55.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.899+0000 7f46d7798740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558a67a97dd0 0x558a67a9bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:55.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.899+0000 7f46d7798740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f46d01b2620 con 0x558a67a8f1f0 2024-04-02T14:03:55.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.899+0000 7f46d7798740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f46d01b1f50 con 0x558a67a97dd0 2024-04-02T14:03:55.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.899+0000 7f46d7798740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f46d01af1b0 con 0x558a67a8e8d0 2024-04-02T14:03:55.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.899+0000 7f46d7798740 10 client.0 _ll_register_callbacks cb 0x7f46c80127e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:55.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.899+0000 7f46c5d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a8f1f0 0x558a67a9b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:55.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.899+0000 7f46c5d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a8f1f0 0x558a67a9b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33056/0 (socket says 172.21.15.52:33056) 2024-04-02T14:03:55.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.899+0000 7f46c5d9b640 1 -- 172.21.15.52:0/776218076 learned_addr learned my addr 172.21.15.52:0/776218076 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:55.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.900+0000 7f46ce59c640 1 --2- 172.21.15.52:0/776218076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a67a8e8d0 0x558a67aa02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:55.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.900+0000 7f46ced9d640 1 --2- 172.21.15.52:0/776218076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558a67a97dd0 0x558a67a9bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:55.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.900+0000 7f46c5d9b640 1 -- 172.21.15.52:0/776218076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a67a8e8d0 msgr2=0x558a67aa02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:55.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.900+0000 7f46c5d9b640 1 --2- 172.21.15.52:0/776218076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a67a8e8d0 0x558a67aa02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.900+0000 7f46c5d9b640 1 -- 172.21.15.52:0/776218076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558a67a97dd0 msgr2=0x558a67a9bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:55.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.900+0000 7f46c5d9b640 1 --2- 172.21.15.52:0/776218076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558a67a97dd0 0x558a67a9bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.900+0000 7f46c5d9b640 1 -- 172.21.15.52:0/776218076 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a67a9c290 con 0x558a67a8f1f0 2024-04-02T14:03:55.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.900+0000 7f46c5d9b640 1 --2- 172.21.15.52:0/776218076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a8f1f0 0x558a67a9b610 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009250 tx=0x7f46b80054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59af374e15f44923 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:55.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.900+0000 7f46b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:55.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.900+0000 7f46b5ffb640 1 -- 172.21.15.52:0/776218076 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b802a070 con 0x558a67a8f1f0 2024-04-02T14:03:55.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.900+0000 7f46b5ffb640 1 -- 172.21.15.52:0/776218076 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f46b800e720 con 0x558a67a8f1f0 2024-04-02T14:03:55.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.900+0000 7f46d7798740 1 -- 172.21.15.52:0/776218076 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46c8025030 con 0x558a67a8f1f0 2024-04-02T14:03:55.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.900+0000 7f46b5ffb640 1 -- 172.21.15.52:0/776218076 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f46b8033050 con 0x558a67a8f1f0 2024-04-02T14:03:55.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.900+0000 7f46d7798740 10 client.18522 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:55.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.900+0000 7f46d7798740 1 -- 172.21.15.52:0/776218076 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558a67b0e1a0 con 0x558a67a8f1f0 2024-04-02T14:03:55.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.900+0000 7f46b5ffb640 1 -- 172.21.15.52:0/776218076 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46c8025030 con 0x558a67a8f1f0 2024-04-02T14:03:55.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.901+0000 7f46b5ffb640 1 -- 172.21.15.52:0/776218076 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f46b802d020 con 0x558a67a8f1f0 2024-04-02T14:03:55.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.901+0000 7f46d7798740 10 client.18522 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:55.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.901+0000 7f46d7798740 10 client.18522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:03:55.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.901+0000 7f46d7798740 20 client.18522 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:55.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.901+0000 7f46d7798740 1 --2- 172.21.15.52:0/776218076 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x558a67b0e780 0x558a67b10ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:55.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.901+0000 7f46d7798740 4 client.18522 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T14:03:55.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.901+0000 7f46d7798740 1 -- 172.21.15.52:0/776218076 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558a679fa0c0 con 0x558a67b0e780 2024-04-02T14:03:55.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.901+0000 7f46ce59c640 1 --2- 172.21.15.52:0/776218076 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x558a67b0e780 0x558a67b10ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:55.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.901+0000 7f46b5ffb640 10 client.18522 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:03:55.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.901+0000 7f46ce59c640 1 --2- 172.21.15.52:0/776218076 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x558a67b0e780 0x558a67b10ba0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80d449be5f1bda9c server_cookie=c959941a8ab1be2f in_seq=0 out_seq=0 2024-04-02T14:03:55.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.902+0000 7f46b5ffb640 1 -- 172.21.15.52:0/776218076 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558a679fa0c0 con 0x558a67b0e780 2024-04-02T14:03:55.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.902+0000 7f46b5ffb640 10 client.18522 handle_command_reply: tid=0 2024-04-02T14:03:55.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.902+0000 7f46d7798740 1 client.18522 shutdown 2024-04-02T14:03:55.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.902+0000 7f46d7798740 1 -- 172.21.15.52:0/776218076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a8f1f0 msgr2=0x558a67a9b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:55.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.902+0000 7f46d7798740 1 --2- 172.21.15.52:0/776218076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a8f1f0 0x558a67a9b610 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009250 tx=0x7f46b80054e0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.902+0000 7f46d7798740 1 -- 172.21.15.52:0/776218076 shutdown_connections 2024-04-02T14:03:55.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.902+0000 7f46d7798740 1 --2- 172.21.15.52:0/776218076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558a67a97dd0 0x558a67a9bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.902+0000 7f46d7798740 1 --2- 172.21.15.52:0/776218076 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x558a67b0e780 0x558a67b10ba0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.902+0000 7f46d7798740 1 --2- 172.21.15.52:0/776218076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558a67a8f1f0 0x558a67a9b610 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.902+0000 7f46d7798740 1 --2- 172.21.15.52:0/776218076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a67a8e8d0 0x558a67aa02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.902+0000 7f46b5ffb640 0 client.18522 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:03:55.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.902+0000 7f46d7798740 1 -- 172.21.15.52:0/776218076 >> 172.21.15.52:0/776218076 conn(0x558a67a89000 msgr2=0x558a67a8fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:55.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.902+0000 7f46d7798740 1 -- 172.21.15.52:0/776218076 shutdown_connections 2024-04-02T14:03:55.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.902+0000 7f46d7798740 1 -- 172.21.15.52:0/776218076 wait complete. 2024-04-02T14:03:55.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.902+0000 7f46d7798740 20 client.18522 trim_cache size 0 max 16384 2024-04-02T14:03:55.931 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:03:55.931 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-04-02T14:03:55.931 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-04-02T14:03:55.932 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:03:55.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.902+0000 7f46d4baf640 1 -- 172.21.15.52:0/925711339 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f46bc0a3d50 msgr2=0x7f46bc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:55.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.902+0000 7f46d4baf640 1 --2- 172.21.15.52:0/925711339 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f46bc0a3d50 0x7f46bc0a6210 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009270 tx=0x7f46c8004ab0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.902+0000 7f46d4baf640 1 -- 172.21.15.52:0/925711339 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46d0068d00 msgr2=0x7f46d01009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:55.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.902+0000 7f46d4baf640 1 --2- 172.21.15.52:0/925711339 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46d0068d00 0x7f46d01009c0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009230 tx=0x7f46c0004270 comp rx=0 tx=0).stop 2024-04-02T14:03:55.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.902+0000 7f46d4baf640 1 -- 172.21.15.52:0/925711339 shutdown_connections 2024-04-02T14:03:55.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.902+0000 7f46d4baf640 1 --2- 172.21.15.52:0/925711339 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f46bc0a3d50 0x7f46bc0a6210 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.903+0000 7f46d4baf640 1 --2- 172.21.15.52:0/925711339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46d0107b50 0x7f46d0101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.903+0000 7f46d4baf640 1 --2- 172.21.15.52:0/925711339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46d00696c0 0x7f46d0100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.903+0000 7f46d4baf640 1 --2- 172.21.15.52:0/925711339 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46d0068d00 0x7f46d01009c0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:55.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.903+0000 7f46d4baf640 1 -- 172.21.15.52:0/925711339 >> 172.21.15.52:0/925711339 conn(0x7f46d0077b00 msgr2=0x7f46d00fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:55.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.903+0000 7f46d4baf640 1 -- 172.21.15.52:0/925711339 shutdown_connections 2024-04-02T14:03:55.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:55.903+0000 7f46d4baf640 1 -- 172.21.15.52:0/925711339 wait complete. 2024-04-02T14:03:55.936 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T14:03:55.936 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:03:55.936 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:55 smithi088 ceph-mon[36528]: pgmap v323: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:03:56.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.076+0000 7f7785af2640 1 -- 172.21.15.52:0/2698715159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77801016a0 msgr2=0x7f778010d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.076+0000 7f7785af2640 1 --2- 172.21.15.52:0/2698715159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77801016a0 0x7f778010d830 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f7778009ca0 tx=0x7f777802c0f0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.077+0000 7f7785af2640 1 -- 172.21.15.52:0/2698715159 shutdown_connections 2024-04-02T14:03:56.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.077+0000 7f7785af2640 1 --2- 172.21.15.52:0/2698715159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77801016a0 0x7f778010d830 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.077+0000 7f7785af2640 1 --2- 172.21.15.52:0/2698715159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7780100ce0 0x7f7780101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.077+0000 7f7785af2640 1 --2- 172.21.15.52:0/2698715159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7780106b40 0x7f7780106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.077+0000 7f7785af2640 1 -- 172.21.15.52:0/2698715159 >> 172.21.15.52:0/2698715159 conn(0x7f77800779e0 msgr2=0x7f7780077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.077+0000 7f7785af2640 1 -- 172.21.15.52:0/2698715159 shutdown_connections 2024-04-02T14:03:56.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.077+0000 7f7785af2640 1 -- 172.21.15.52:0/2698715159 wait complete. 2024-04-02T14:03:56.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.077+0000 7f7785af2640 1 Processor -- start 2024-04-02T14:03:56.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.077+0000 7f7785af2640 1 -- start start 2024-04-02T14:03:56.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.078+0000 7f7785af2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7780100ce0 0x7f7780102820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.078+0000 7f7785af2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77801016a0 0x7f7780102d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.078+0000 7f7785af2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7780106b40 0x7f77801032a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.078+0000 7f7785af2640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7780110d70 con 0x7f7780100ce0 2024-04-02T14:03:56.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.078+0000 7f7785af2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7780110bf0 con 0x7f7780106b40 2024-04-02T14:03:56.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.078+0000 7f7785af2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7780110ef0 con 0x7f77801016a0 2024-04-02T14:03:56.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.078+0000 7f777f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7780100ce0 0x7f7780102820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.078+0000 7f777f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7780100ce0 0x7f7780102820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33068/0 (socket says 172.21.15.52:33068) 2024-04-02T14:03:56.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.078+0000 7f777f7fe640 1 -- 172.21.15.52:0/482048443 learned_addr learned my addr 172.21.15.52:0/482048443 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:56.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.078+0000 7f777effd640 1 --2- 172.21.15.52:0/482048443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77801016a0 0x7f7780102d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.078+0000 7f777ffff640 1 --2- 172.21.15.52:0/482048443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7780106b40 0x7f77801032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.078+0000 7f777f7fe640 1 -- 172.21.15.52:0/482048443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77801016a0 msgr2=0x7f7780102d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.078+0000 7f777f7fe640 1 --2- 172.21.15.52:0/482048443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77801016a0 0x7f7780102d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.078+0000 7f777f7fe640 1 -- 172.21.15.52:0/482048443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7780106b40 msgr2=0x7f77801032a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.078+0000 7f777f7fe640 1 --2- 172.21.15.52:0/482048443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7780106b40 0x7f77801032a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.078+0000 7f777f7fe640 1 -- 172.21.15.52:0/482048443 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77801a6740 con 0x7f7780100ce0 2024-04-02T14:03:56.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.078+0000 7f777ffff640 1 --2- 172.21.15.52:0/482048443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7780106b40 0x7f77801032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:03:56.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.078+0000 7f777f7fe640 1 --2- 172.21.15.52:0/482048443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7780100ce0 0x7f7780102820 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f777400d950 tx=0x7f777400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.078+0000 7f777cff9640 1 -- 172.21.15.52:0/482048443 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7774017030 con 0x7f7780100ce0 2024-04-02T14:03:56.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.078+0000 7f7785af2640 1 -- 172.21.15.52:0/482048443 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77801a6a30 con 0x7f7780100ce0 2024-04-02T14:03:56.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.078+0000 7f777cff9640 1 -- 172.21.15.52:0/482048443 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7774004020 con 0x7f7780100ce0 2024-04-02T14:03:56.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.078+0000 7f777cff9640 1 -- 172.21.15.52:0/482048443 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7774005650 con 0x7f7780100ce0 2024-04-02T14:03:56.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.079+0000 7f7785af2640 1 -- 172.21.15.52:0/482048443 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77801a6fd0 con 0x7f7780100ce0 2024-04-02T14:03:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.079+0000 7f7785af2640 1 -- 172.21.15.52:0/482048443 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7780102100 con 0x7f7780100ce0 2024-04-02T14:03:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.081+0000 7f777cff9640 1 -- 172.21.15.52:0/482048443 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f77740041d0 con 0x7f7780100ce0 2024-04-02T14:03:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.081+0000 7f777cff9640 1 --2- 172.21.15.52:0/482048443 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f77680a3d20 0x7f77680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.081+0000 7f777effd640 1 --2- 172.21.15.52:0/482048443 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f77680a3d20 0x7f77680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.081+0000 7f777cff9640 1 -- 172.21.15.52:0/482048443 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f77740d0ba0 con 0x7f7780100ce0 2024-04-02T14:03:56.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.082+0000 7f777effd640 1 --2- 172.21.15.52:0/482048443 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f77680a3d20 0x7f77680a61e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f77801042b0 tx=0x7f777000a2d0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.089+0000 7f777cff9640 1 -- 172.21.15.52:0/482048443 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f777400e350 con 0x7f7780100ce0 2024-04-02T14:03:56.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.321+0000 7f7785af2640 1 -- 172.21.15.52:0/482048443 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77801a72c0 con 0x7f7780100ce0 2024-04-02T14:03:56.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.322+0000 7f777cff9640 1 -- 172.21.15.52:0/482048443 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11923 (secure 0 0 0) 0x7f777409ceb0 con 0x7f7780100ce0 2024-04-02T14:03:56.326 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T14:03:56.327 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:57:45.240902+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24467,24457,24490,24475,24503]},"id":1}]} 2024-04-02T14:03:56.327 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-04-02T14:03:56.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.325+0000 7f7785af2640 1 -- 172.21.15.52:0/482048443 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f77680a3d20 msgr2=0x7f77680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.325+0000 7f7785af2640 1 --2- 172.21.15.52:0/482048443 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f77680a3d20 0x7f77680a61e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f77801042b0 tx=0x7f777000a2d0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.325+0000 7f7785af2640 1 -- 172.21.15.52:0/482048443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7780100ce0 msgr2=0x7f7780102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.325+0000 7f7785af2640 1 --2- 172.21.15.52:0/482048443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7780100ce0 0x7f7780102820 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f777400d950 tx=0x7f777400dd50 comp rx=0 tx=0).stop 2024-04-02T14:03:56.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.325+0000 7f7785af2640 1 -- 172.21.15.52:0/482048443 shutdown_connections 2024-04-02T14:03:56.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.325+0000 7f7785af2640 1 --2- 172.21.15.52:0/482048443 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f77680a3d20 0x7f77680a61e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.325+0000 7f7785af2640 1 --2- 172.21.15.52:0/482048443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7780106b40 0x7f77801032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.325+0000 7f7785af2640 1 --2- 172.21.15.52:0/482048443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77801016a0 0x7f7780102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.325+0000 7f7785af2640 1 --2- 172.21.15.52:0/482048443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7780100ce0 0x7f7780102820 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.325+0000 7f7785af2640 1 -- 172.21.15.52:0/482048443 >> 172.21.15.52:0/482048443 conn(0x7f77800779e0 msgr2=0x7f7780104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.325+0000 7f7785af2640 1 -- 172.21.15.52:0/482048443 shutdown_connections 2024-04-02T14:03:56.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.325+0000 7f7785af2640 1 -- 172.21.15.52:0/482048443 wait complete. 2024-04-02T14:03:56.339 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T14:03:56.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.444+0000 7f0ec4c50640 1 -- 172.21.15.52:0/592635077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ec0101330 msgr2=0x7f0ec010d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.444+0000 7f0ec4c50640 1 --2- 172.21.15.52:0/592635077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ec0101330 0x7f0ec010d840 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009cd0 tx=0x7f0eb802c0f0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.445+0000 7f0ec4c50640 1 -- 172.21.15.52:0/592635077 shutdown_connections 2024-04-02T14:03:56.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.445+0000 7f0ec4c50640 1 --2- 172.21.15.52:0/592635077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ec0101330 0x7f0ec010d840 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.445+0000 7f0ec4c50640 1 --2- 172.21.15.52:0/592635077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ec0100970 0x7f0ec0100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.445+0000 7f0ec4c50640 1 --2- 172.21.15.52:0/592635077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ec0106980 0x7f0ec0106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.445+0000 7f0ec4c50640 1 -- 172.21.15.52:0/592635077 >> 172.21.15.52:0/592635077 conn(0x7f0ec0077b00 msgr2=0x7f0ec0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.445+0000 7f0ec4c50640 1 -- 172.21.15.52:0/592635077 shutdown_connections 2024-04-02T14:03:56.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.445+0000 7f0ec4c50640 1 -- 172.21.15.52:0/592635077 wait complete. 2024-04-02T14:03:56.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.446+0000 7f0ec4c50640 1 Processor -- start 2024-04-02T14:03:56.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.446+0000 7f0ec4c50640 1 -- start start 2024-04-02T14:03:56.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.446+0000 7f0ec4c50640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ec0100970 0x7f0ec0198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.446+0000 7f0ec4c50640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ec0101330 0x7f0ec0198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.446+0000 7f0ec4c50640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ec0106980 0x7f0ec019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.446+0000 7f0ec4c50640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec0110d60 con 0x7f0ec0106980 2024-04-02T14:03:56.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.446+0000 7f0ec4c50640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec0110be0 con 0x7f0ec0100970 2024-04-02T14:03:56.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.446+0000 7f0ec4c50640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec0110ee0 con 0x7f0ec0101330 2024-04-02T14:03:56.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.446+0000 7f0ebed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ec0106980 0x7f0ec019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.446+0000 7f0ebed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ec0106980 0x7f0ec019cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33094/0 (socket says 172.21.15.52:33094) 2024-04-02T14:03:56.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.446+0000 7f0ebed9d640 1 -- 172.21.15.52:0/3173538746 learned_addr learned my addr 172.21.15.52:0/3173538746 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:56.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.446+0000 7f0ebdd9b640 1 --2- 172.21.15.52:0/3173538746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ec0101330 0x7f0ec0198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.446+0000 7f0ebe59c640 1 --2- 172.21.15.52:0/3173538746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ec0100970 0x7f0ec0198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.446+0000 7f0ebed9d640 1 -- 172.21.15.52:0/3173538746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ec0101330 msgr2=0x7f0ec0198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.446+0000 7f0ebed9d640 1 --2- 172.21.15.52:0/3173538746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ec0101330 0x7f0ec0198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.446+0000 7f0ebed9d640 1 -- 172.21.15.52:0/3173538746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ec0100970 msgr2=0x7f0ec0198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.446+0000 7f0ebed9d640 1 --2- 172.21.15.52:0/3173538746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ec0100970 0x7f0ec0198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.446+0000 7f0ebed9d640 1 -- 172.21.15.52:0/3173538746 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec019d340 con 0x7f0ec0106980 2024-04-02T14:03:56.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.447+0000 7f0ebed9d640 1 --2- 172.21.15.52:0/3173538746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ec0106980 0x7f0ec019cc10 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8005e30 tx=0x7f0eb8032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.448+0000 7f0eb77fe640 1 -- 172.21.15.52:0/3173538746 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8041020 con 0x7f0ec0106980 2024-04-02T14:03:56.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.448+0000 7f0eb77fe640 1 -- 172.21.15.52:0/3173538746 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb8034cd0 con 0x7f0ec0106980 2024-04-02T14:03:56.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.448+0000 7f0ec4c50640 1 -- 172.21.15.52:0/3173538746 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ec019d5d0 con 0x7f0ec0106980 2024-04-02T14:03:56.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.448+0000 7f0eb77fe640 1 -- 172.21.15.52:0/3173538746 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8039040 con 0x7f0ec0106980 2024-04-02T14:03:56.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.448+0000 7f0ec4c50640 1 -- 172.21.15.52:0/3173538746 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ec01a4ec0 con 0x7f0ec0106980 2024-04-02T14:03:56.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.450+0000 7f0eb77fe640 1 -- 172.21.15.52:0/3173538746 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0eb8044020 con 0x7f0ec0106980 2024-04-02T14:03:56.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.451+0000 7f0eb77fe640 1 --2- 172.21.15.52:0/3173538746 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f0eac0a3da0 0x7f0eac0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.451+0000 7f0ebe59c640 1 --2- 172.21.15.52:0/3173538746 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f0eac0a3da0 0x7f0eac0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.451+0000 7f0eb77fe640 1 -- 172.21.15.52:0/3173538746 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f0eb80f27c0 con 0x7f0ec0106980 2024-04-02T14:03:56.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.451+0000 7f0ebe59c640 1 --2- 172.21.15.52:0/3173538746 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f0eac0a3da0 0x7f0eac0a6260 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000bb60 tx=0x7f0eb000b390 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.452+0000 7f0ec7830740 1 Processor -- start 2024-04-02T14:03:56.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.452+0000 7f0ec7830740 1 -- start start 2024-04-02T14:03:56.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.452+0000 7f0ec7830740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b4d7a0 0x55b767b4db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.452+0000 7f0ec7830740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b767b4e0c0 0x55b767b56760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.452+0000 7f0ec7830740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b767b56ca0 0x55b767b59090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.452+0000 7f0ec7830740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b767ab17b0 con 0x55b767b4d7a0 2024-04-02T14:03:56.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.452+0000 7f0ec7830740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b767a50570 con 0x55b767b56ca0 2024-04-02T14:03:56.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.452+0000 7f0ec7830740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b767aad760 con 0x55b767b4e0c0 2024-04-02T14:03:56.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.452+0000 7f0ebe59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b4d7a0 0x55b767b4db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.452+0000 7f0ebe59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b4d7a0 0x55b767b4db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33104/0 (socket says 172.21.15.52:33104) 2024-04-02T14:03:56.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.452+0000 7f0ebe59c640 1 -- 172.21.15.52:0/1500012332 learned_addr learned my addr 172.21.15.52:0/1500012332 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:56.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.453+0000 7f0ebdd9b640 1 --2- 172.21.15.52:0/1500012332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b767b4e0c0 0x55b767b56760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.453+0000 7f0ebed9d640 1 --2- 172.21.15.52:0/1500012332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b767b56ca0 0x55b767b59090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.453+0000 7f0ebe59c640 1 -- 172.21.15.52:0/1500012332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b767b4e0c0 msgr2=0x55b767b56760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.453+0000 7f0ebe59c640 1 --2- 172.21.15.52:0/1500012332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b767b4e0c0 0x55b767b56760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.453+0000 7f0ebe59c640 1 -- 172.21.15.52:0/1500012332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b767b56ca0 msgr2=0x55b767b59090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.453+0000 7f0ebe59c640 1 --2- 172.21.15.52:0/1500012332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b767b56ca0 0x55b767b59090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.453+0000 7f0ebe59c640 1 -- 172.21.15.52:0/1500012332 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7677c53f0 con 0x55b767b4d7a0 2024-04-02T14:03:56.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.453+0000 7f0ebed9d640 1 --2- 172.21.15.52:0/1500012332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b767b56ca0 0x55b767b59090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:03:56.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.453+0000 7f0ebe59c640 1 --2- 172.21.15.52:0/1500012332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b4d7a0 0x55b767b4db80 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0010fa0 tx=0x7f0eb0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d288b62ad2574c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.453+0000 7f0eb4ff9640 1 -- 172.21.15.52:0/1500012332 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb001e620 con 0x55b767b4d7a0 2024-04-02T14:03:56.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.453+0000 7f0eb4ff9640 1 -- 172.21.15.52:0/1500012332 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb001eca0 con 0x55b767b4d7a0 2024-04-02T14:03:56.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.453+0000 7f0eb4ff9640 1 -- 172.21.15.52:0/1500012332 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0026ca0 con 0x55b767b4d7a0 2024-04-02T14:03:56.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.454+0000 7f0ec7830740 1 -- 172.21.15.52:0/1500012332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b4d7a0 msgr2=0x55b767b4db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.454+0000 7f0ec7830740 1 --2- 172.21.15.52:0/1500012332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b4d7a0 0x55b767b4db80 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0010fa0 tx=0x7f0eb0013000 comp rx=0 tx=0).stop 2024-04-02T14:03:56.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.454+0000 7f0ec7830740 1 -- 172.21.15.52:0/1500012332 shutdown_connections 2024-04-02T14:03:56.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.454+0000 7f0ec7830740 1 --2- 172.21.15.52:0/1500012332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b767b56ca0 0x55b767b59090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.454+0000 7f0ec7830740 1 --2- 172.21.15.52:0/1500012332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b767b4e0c0 0x55b767b56760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.454+0000 7f0ec7830740 1 --2- 172.21.15.52:0/1500012332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b4d7a0 0x55b767b4db80 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.454+0000 7f0ec7830740 1 -- 172.21.15.52:0/1500012332 >> 172.21.15.52:0/1500012332 conn(0x55b767b47ed0 msgr2=0x55b767b482b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.454+0000 7f0ec7830740 1 -- 172.21.15.52:0/1500012332 shutdown_connections 2024-04-02T14:03:56.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.454+0000 7f0ec7830740 1 -- 172.21.15.52:0/1500012332 wait complete. 2024-04-02T14:03:56.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.454+0000 7f0ec7830740 1 -- start start 2024-04-02T14:03:56.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.454+0000 7f0ec7830740 1 Processor -- start 2024-04-02T14:03:56.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.455+0000 7f0ec7830740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b767b4d7a0 0x55b767b53b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.455+0000 7f0ec7830740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b767b4e0c0 0x55b767b54040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.455+0000 7f0ec7830740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b56ca0 0x55b767b54580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.455+0000 7f0ec7830740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b767b59a90 con 0x55b767b56ca0 2024-04-02T14:03:56.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.455+0000 7f0ec7830740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b767a6ce80 con 0x55b767b4d7a0 2024-04-02T14:03:56.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.455+0000 7f0ec7830740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b767b59c10 con 0x55b767b4e0c0 2024-04-02T14:03:56.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.455+0000 7f0ec7830740 10 client.0 _ll_register_callbacks cb 0x55b767aa8aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:56.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.455+0000 7f0ebed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b56ca0 0x55b767b54580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.455+0000 7f0ebed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b56ca0 0x55b767b54580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33118/0 (socket says 172.21.15.52:33118) 2024-04-02T14:03:56.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.455+0000 7f0ebed9d640 1 -- 172.21.15.52:0/1801542599 learned_addr learned my addr 172.21.15.52:0/1801542599 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:56.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.455+0000 7f0ebdd9b640 1 --2- 172.21.15.52:0/1801542599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b767b4e0c0 0x55b767b54040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.455+0000 7f0ebe59c640 1 --2- 172.21.15.52:0/1801542599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b767b4d7a0 0x55b767b53b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.455+0000 7f0ebed9d640 1 -- 172.21.15.52:0/1801542599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b767b4e0c0 msgr2=0x55b767b54040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.455+0000 7f0ebed9d640 1 --2- 172.21.15.52:0/1801542599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b767b4e0c0 0x55b767b54040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.455+0000 7f0ebed9d640 1 -- 172.21.15.52:0/1801542599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b767b4d7a0 msgr2=0x55b767b53b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.455+0000 7f0ebed9d640 1 --2- 172.21.15.52:0/1801542599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b767b4d7a0 0x55b767b53b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.455+0000 7f0ebed9d640 1 -- 172.21.15.52:0/1801542599 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b76788f7d0 con 0x55b767b56ca0 2024-04-02T14:03:56.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.455+0000 7f0ebed9d640 1 --2- 172.21.15.52:0/1801542599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b56ca0 0x55b767b54580 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f0eb80ba940 tx=0x7f0eb80be0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa9512c0b7f14bf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.455+0000 7f0ea5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:56.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.455+0000 7f0ea5ffb640 1 -- 172.21.15.52:0/1801542599 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb80be4e0 con 0x55b767b56ca0 2024-04-02T14:03:56.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.455+0000 7f0ea5ffb640 1 -- 172.21.15.52:0/1801542599 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f0eb80f23f0 con 0x55b767b56ca0 2024-04-02T14:03:56.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.455+0000 7f0ec7830740 1 -- 172.21.15.52:0/1801542599 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0eb001eca0 con 0x55b767b56ca0 2024-04-02T14:03:56.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.456+0000 7f0ea5ffb640 1 -- 172.21.15.52:0/1801542599 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0eb80f25f0 con 0x55b767b56ca0 2024-04-02T14:03:56.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.456+0000 7f0ec7830740 10 client.18540 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:56.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.456+0000 7f0ec7830740 1 -- 172.21.15.52:0/1801542599 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b767b65b00 con 0x55b767b56ca0 2024-04-02T14:03:56.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.456+0000 7f0ea5ffb640 1 -- 172.21.15.52:0/1801542599 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb001eca0 con 0x55b767b56ca0 2024-04-02T14:03:56.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.456+0000 7f0ea5ffb640 1 -- 172.21.15.52:0/1801542599 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0eb803c030 con 0x55b767b56ca0 2024-04-02T14:03:56.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.456+0000 7f0ec7830740 10 client.18540 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:56.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.456+0000 7f0ec7830740 10 client.18540 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T14:03:56.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.456+0000 7f0ec7830740 20 client.18540 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:56.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.456+0000 7f0ec7830740 1 --2- 172.21.15.52:0/1801542599 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55b767b66f10 0x55b767b69320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.456+0000 7f0ec7830740 4 client.18540 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:03:56.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.456+0000 7f0ec7830740 1 -- 172.21.15.52:0/1801542599 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b767abecf0 con 0x55b767b66f10 2024-04-02T14:03:56.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.456+0000 7f0ebe59c640 1 --2- 172.21.15.52:0/1801542599 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55b767b66f10 0x55b767b69320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.457+0000 7f0ebe59c640 1 --2- 172.21.15.52:0/1801542599 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55b767b66f10 0x55b767b69320 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fc255d67019c898c server_cookie=74d2dfe6ad89adda in_seq=0 out_seq=0 2024-04-02T14:03:56.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.457+0000 7f0ea5ffb640 10 client.18540 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T14:03:56.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.457+0000 7f0ea5ffb640 1 -- 172.21.15.52:0/1801542599 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b767abecf0 con 0x55b767b66f10 2024-04-02T14:03:56.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.457+0000 7f0ea5ffb640 10 client.18540 handle_command_reply: tid=0 2024-04-02T14:03:56.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.458+0000 7f0ec7830740 1 client.18540 shutdown 2024-04-02T14:03:56.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.458+0000 7f0ec7830740 1 -- 172.21.15.52:0/1801542599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b56ca0 msgr2=0x55b767b54580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.458+0000 7f0ec7830740 1 --2- 172.21.15.52:0/1801542599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b56ca0 0x55b767b54580 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f0eb80ba940 tx=0x7f0eb80be0e0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.458+0000 7f0ec7830740 1 -- 172.21.15.52:0/1801542599 shutdown_connections 2024-04-02T14:03:56.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.458+0000 7f0ec7830740 1 --2- 172.21.15.52:0/1801542599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b56ca0 0x55b767b54580 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.458+0000 7f0ec7830740 1 --2- 172.21.15.52:0/1801542599 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55b767b66f10 0x55b767b69320 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.458+0000 7f0ec7830740 1 --2- 172.21.15.52:0/1801542599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b767b4e0c0 0x55b767b54040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.458+0000 7f0ec7830740 1 --2- 172.21.15.52:0/1801542599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b767b4d7a0 0x55b767b53b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.458+0000 7f0ea5ffb640 0 client.18540 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T14:03:56.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.458+0000 7f0ec7830740 1 -- 172.21.15.52:0/1801542599 >> 172.21.15.52:0/1801542599 conn(0x55b767b47ed0 msgr2=0x55b767b49c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.458+0000 7f0ec7830740 1 -- 172.21.15.52:0/1801542599 shutdown_connections 2024-04-02T14:03:56.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.458+0000 7f0ec7830740 1 -- 172.21.15.52:0/1801542599 wait complete. 2024-04-02T14:03:56.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.458+0000 7f0ec7830740 20 client.18540 trim_cache size 0 max 16384 2024-04-02T14:03:56.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.471+0000 7f0ec7830740 1 Processor -- start 2024-04-02T14:03:56.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.471+0000 7f0ec7830740 1 -- start start 2024-04-02T14:03:56.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.471+0000 7f0ec7830740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b767b4d7a0 0x55b767b62980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.471+0000 7f0ec7830740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b767b4e0c0 0x55b767b543a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.471+0000 7f0ec7830740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b56ca0 0x55b767b6ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.471+0000 7f0ec7830740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01aef60 con 0x55b767b56ca0 2024-04-02T14:03:56.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.471+0000 7f0ec7830740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01b0aa0 con 0x55b767b4e0c0 2024-04-02T14:03:56.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.471+0000 7f0ec7830740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01af630 con 0x55b767b4d7a0 2024-04-02T14:03:56.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.472+0000 7f0ebed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b56ca0 0x55b767b6ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.472+0000 7f0ebed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b56ca0 0x55b767b6ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33134/0 (socket says 172.21.15.52:33134) 2024-04-02T14:03:56.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.472+0000 7f0ebed9d640 1 -- 172.21.15.52:0/3503389175 learned_addr learned my addr 172.21.15.52:0/3503389175 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:56.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.472+0000 7f0ebe59c640 1 --2- 172.21.15.52:0/3503389175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b767b4d7a0 0x55b767b62980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.472+0000 7f0ebdd9b640 1 --2- 172.21.15.52:0/3503389175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b767b4e0c0 0x55b767b543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.472+0000 7f0ebed9d640 1 -- 172.21.15.52:0/3503389175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b767b4d7a0 msgr2=0x55b767b62980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.472+0000 7f0ebed9d640 1 --2- 172.21.15.52:0/3503389175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b767b4d7a0 0x55b767b62980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.472+0000 7f0ebed9d640 1 -- 172.21.15.52:0/3503389175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b767b4e0c0 msgr2=0x55b767b543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.472+0000 7f0ebed9d640 1 --2- 172.21.15.52:0/3503389175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b767b4e0c0 0x55b767b543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.472+0000 7f0ebed9d640 1 -- 172.21.15.52:0/3503389175 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b767b699a0 con 0x55b767b56ca0 2024-04-02T14:03:56.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.472+0000 7f0ebe59c640 1 --2- 172.21.15.52:0/3503389175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b767b4d7a0 0x55b767b62980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:03:56.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.472+0000 7f0ebed9d640 1 --2- 172.21.15.52:0/3503389175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b56ca0 0x55b767b6ebf0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f0eb80be1e0 tx=0x7f0eb80be150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9228d03b229a0d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.472+0000 7f0ea7fff640 1 -- 172.21.15.52:0/3503389175 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb80be4e0 con 0x55b767b56ca0 2024-04-02T14:03:56.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.472+0000 7f0ea7fff640 1 -- 172.21.15.52:0/3503389175 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb80c2890 con 0x55b767b56ca0 2024-04-02T14:03:56.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.472+0000 7f0ea7fff640 1 -- 172.21.15.52:0/3503389175 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb80bdb80 con 0x55b767b56ca0 2024-04-02T14:03:56.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.473+0000 7f0ec7830740 1 -- 172.21.15.52:0/3503389175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b56ca0 msgr2=0x55b767b6ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.473+0000 7f0ec7830740 1 --2- 172.21.15.52:0/3503389175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b56ca0 0x55b767b6ebf0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f0eb80be1e0 tx=0x7f0eb80be150 comp rx=0 tx=0).stop 2024-04-02T14:03:56.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.473+0000 7f0ec7830740 1 -- 172.21.15.52:0/3503389175 shutdown_connections 2024-04-02T14:03:56.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.473+0000 7f0ec7830740 1 --2- 172.21.15.52:0/3503389175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b56ca0 0x55b767b6ebf0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.473+0000 7f0ec7830740 1 --2- 172.21.15.52:0/3503389175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b767b4e0c0 0x55b767b543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.473+0000 7f0ec7830740 1 --2- 172.21.15.52:0/3503389175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b767b4d7a0 0x55b767b62980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.473+0000 7f0ec7830740 1 -- 172.21.15.52:0/3503389175 >> 172.21.15.52:0/3503389175 conn(0x55b767b47ed0 msgr2=0x55b767b49b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.473+0000 7f0ec7830740 1 -- 172.21.15.52:0/3503389175 shutdown_connections 2024-04-02T14:03:56.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.473+0000 7f0ec7830740 1 -- 172.21.15.52:0/3503389175 wait complete. 2024-04-02T14:03:56.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.473+0000 7f0ec7830740 1 -- start start 2024-04-02T14:03:56.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.473+0000 7f0ec7830740 1 Processor -- start 2024-04-02T14:03:56.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.473+0000 7f0ec7830740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b767b4d7a0 0x55b767b57850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.473+0000 7f0ec7830740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b4e0c0 0x55b767b57d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.473+0000 7f0ec7830740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b767b56ca0 0x55b767b5de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.473+0000 7f0ec7830740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01b03d0 con 0x55b767b4e0c0 2024-04-02T14:03:56.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.473+0000 7f0ec7830740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01afd00 con 0x55b767b4d7a0 2024-04-02T14:03:56.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.473+0000 7f0ec7830740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01acf60 con 0x55b767b56ca0 2024-04-02T14:03:56.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.474+0000 7f0ec7830740 10 client.0 _ll_register_callbacks cb 0x7f0eb80be2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:56.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.474+0000 7f0ebdd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b4e0c0 0x55b767b57d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.474+0000 7f0ebdd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b4e0c0 0x55b767b57d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33144/0 (socket says 172.21.15.52:33144) 2024-04-02T14:03:56.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.474+0000 7f0ebdd9b640 1 -- 172.21.15.52:0/2851955692 learned_addr learned my addr 172.21.15.52:0/2851955692 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:56.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.474+0000 7f0ebe59c640 1 --2- 172.21.15.52:0/2851955692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b767b4d7a0 0x55b767b57850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.474+0000 7f0ebed9d640 1 --2- 172.21.15.52:0/2851955692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b767b56ca0 0x55b767b5de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.474+0000 7f0ebdd9b640 1 -- 172.21.15.52:0/2851955692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b767b56ca0 msgr2=0x55b767b5de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.474+0000 7f0ebdd9b640 1 --2- 172.21.15.52:0/2851955692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b767b56ca0 0x55b767b5de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.474+0000 7f0ebdd9b640 1 -- 172.21.15.52:0/2851955692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b767b4d7a0 msgr2=0x55b767b57850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.474+0000 7f0ebdd9b640 1 --2- 172.21.15.52:0/2851955692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b767b4d7a0 0x55b767b57850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.474+0000 7f0ebdd9b640 1 -- 172.21.15.52:0/2851955692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b767b5e480 con 0x55b767b4e0c0 2024-04-02T14:03:56.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.474+0000 7f0ebdd9b640 1 --2- 172.21.15.52:0/2851955692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b4e0c0 0x55b767b57d90 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x55b767b4d270 tx=0x7f0ea80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf0dd38f4d5d471 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.474+0000 7f0ea5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:56.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.474+0000 7f0ea5ffb640 1 -- 172.21.15.52:0/2851955692 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8005220 con 0x55b767b4e0c0 2024-04-02T14:03:56.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.474+0000 7f0ea5ffb640 1 -- 172.21.15.52:0/2851955692 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f0ea80053c0 con 0x55b767b4e0c0 2024-04-02T14:03:56.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.474+0000 7f0ec7830740 1 -- 172.21.15.52:0/2851955692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0eb80c2890 con 0x55b767b4e0c0 2024-04-02T14:03:56.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.474+0000 7f0ea5ffb640 1 -- 172.21.15.52:0/2851955692 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0ea8027050 con 0x55b767b4e0c0 2024-04-02T14:03:56.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.474+0000 7f0ec7830740 10 client.18546 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:56.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.474+0000 7f0ec7830740 1 -- 172.21.15.52:0/2851955692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b767bcd460 con 0x55b767b4e0c0 2024-04-02T14:03:56.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.475+0000 7f0ea5ffb640 1 -- 172.21.15.52:0/2851955692 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb80c2890 con 0x55b767b4e0c0 2024-04-02T14:03:56.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.475+0000 7f0ea5ffb640 1 -- 172.21.15.52:0/2851955692 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0ea801b070 con 0x55b767b4e0c0 2024-04-02T14:03:56.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.475+0000 7f0ec7830740 10 client.18546 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:56.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.475+0000 7f0ec7830740 10 client.18546 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-04-02T14:03:56.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.475+0000 7f0ec7830740 20 client.18546 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:56.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.475+0000 7f0ec7830740 1 --2- 172.21.15.52:0/2851955692 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55b767bcdb20 0x55b767bcff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.475+0000 7f0ec7830740 4 client.18546 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:03:56.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.475+0000 7f0ec7830740 1 -- 172.21.15.52:0/2851955692 --> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b767ab9040 con 0x55b767bcdb20 2024-04-02T14:03:56.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.475+0000 7f0ebe59c640 1 --2- 172.21.15.52:0/2851955692 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55b767bcdb20 0x55b767bcff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.476+0000 7f0ebe59c640 1 --2- 172.21.15.52:0/2851955692 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55b767bcdb20 0x55b767bcff40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ad964b70160edd9f server_cookie=6d60fa17be0409bb in_seq=0 out_seq=0 2024-04-02T14:03:56.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.476+0000 7f0ea5ffb640 10 client.18546 ms_handle_connect on v2:172.21.15.189:6834/1647960400 2024-04-02T14:03:56.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.476+0000 7f0ea5ffb640 1 -- 172.21.15.52:0/2851955692 <== mds.4 v2:172.21.15.189:6834/1647960400 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b767ab9040 con 0x55b767bcdb20 2024-04-02T14:03:56.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.476+0000 7f0ea5ffb640 10 client.18546 handle_command_reply: tid=0 2024-04-02T14:03:56.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.476+0000 7f0ec7830740 1 client.18546 shutdown 2024-04-02T14:03:56.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.476+0000 7f0ec7830740 1 -- 172.21.15.52:0/2851955692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b4e0c0 msgr2=0x55b767b57d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.476+0000 7f0ec7830740 1 --2- 172.21.15.52:0/2851955692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b4e0c0 0x55b767b57d90 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x55b767b4d270 tx=0x7f0ea80039c0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.476+0000 7f0ec7830740 1 -- 172.21.15.52:0/2851955692 shutdown_connections 2024-04-02T14:03:56.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.476+0000 7f0ec7830740 1 --2- 172.21.15.52:0/2851955692 >> [v2:172.21.15.189:6834/1647960400,v1:172.21.15.189:6835/1647960400] conn(0x55b767bcdb20 0x55b767bcff40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.476+0000 7f0ec7830740 1 --2- 172.21.15.52:0/2851955692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b767b56ca0 0x55b767b5de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.476+0000 7f0ea5ffb640 0 client.18546 ms_handle_reset on v2:172.21.15.189:6834/1647960400 2024-04-02T14:03:56.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.476+0000 7f0ec7830740 1 --2- 172.21.15.52:0/2851955692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b767b4e0c0 0x55b767b57d90 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.476+0000 7f0ec7830740 1 --2- 172.21.15.52:0/2851955692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b767b4d7a0 0x55b767b57850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.476+0000 7f0ec7830740 1 -- 172.21.15.52:0/2851955692 >> 172.21.15.52:0/2851955692 conn(0x55b767b47ed0 msgr2=0x55b767b466a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.476+0000 7f0ec7830740 1 -- 172.21.15.52:0/2851955692 shutdown_connections 2024-04-02T14:03:56.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.476+0000 7f0ec7830740 1 -- 172.21.15.52:0/2851955692 wait complete. 2024-04-02T14:03:56.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.476+0000 7f0ec7830740 20 client.18546 trim_cache size 0 max 16384 2024-04-02T14:03:56.491 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:03:56.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.477+0000 7f0ec4c50640 1 -- 172.21.15.52:0/3173538746 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f0eac0a3da0 msgr2=0x7f0eac0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.477+0000 7f0ec4c50640 1 --2- 172.21.15.52:0/3173538746 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f0eac0a3da0 0x7f0eac0a6260 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000bb60 tx=0x7f0eb000b390 comp rx=0 tx=0).stop 2024-04-02T14:03:56.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.477+0000 7f0ec4c50640 1 -- 172.21.15.52:0/3173538746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ec0106980 msgr2=0x7f0ec019cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.477+0000 7f0ec4c50640 1 --2- 172.21.15.52:0/3173538746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ec0106980 0x7f0ec019cc10 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8005e30 tx=0x7f0eb8032ea0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.477+0000 7f0ec4c50640 1 -- 172.21.15.52:0/3173538746 shutdown_connections 2024-04-02T14:03:56.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.477+0000 7f0ec4c50640 1 --2- 172.21.15.52:0/3173538746 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f0eac0a3da0 0x7f0eac0a6260 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.477+0000 7f0ec4c50640 1 --2- 172.21.15.52:0/3173538746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ec0106980 0x7f0ec019cc10 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.477+0000 7f0ec4c50640 1 --2- 172.21.15.52:0/3173538746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ec0101330 0x7f0ec0198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.477+0000 7f0ec4c50640 1 --2- 172.21.15.52:0/3173538746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ec0100970 0x7f0ec0198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.477+0000 7f0ec4c50640 1 -- 172.21.15.52:0/3173538746 >> 172.21.15.52:0/3173538746 conn(0x7f0ec0077b00 msgr2=0x7f0ec0068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.477+0000 7f0ec4c50640 1 -- 172.21.15.52:0/3173538746 shutdown_connections 2024-04-02T14:03:56.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.477+0000 7f0ec4c50640 1 -- 172.21.15.52:0/3173538746 wait complete. 2024-04-02T14:03:56.494 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T14:03:56.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.596+0000 7f642c5c8640 1 -- 172.21.15.52:0/2629677831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6424106980 msgr2=0x7f6424106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.596+0000 7f642c5c8640 1 --2- 172.21.15.52:0/2629677831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6424106980 0x7f6424106d60 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f6414009930 tx=0x7f641402bc30 comp rx=0 tx=0).stop 2024-04-02T14:03:56.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.596+0000 7f642c5c8640 1 -- 172.21.15.52:0/2629677831 shutdown_connections 2024-04-02T14:03:56.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.596+0000 7f642c5c8640 1 --2- 172.21.15.52:0/2629677831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6424101330 0x7f642410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.596+0000 7f642c5c8640 1 --2- 172.21.15.52:0/2629677831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6424100970 0x7f6424100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.596+0000 7f642c5c8640 1 --2- 172.21.15.52:0/2629677831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6424106980 0x7f6424106d60 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.596+0000 7f642c5c8640 1 -- 172.21.15.52:0/2629677831 >> 172.21.15.52:0/2629677831 conn(0x7f6424077b00 msgr2=0x7f6424077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.596+0000 7f642c5c8640 1 -- 172.21.15.52:0/2629677831 shutdown_connections 2024-04-02T14:03:56.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.596+0000 7f642c5c8640 1 -- 172.21.15.52:0/2629677831 wait complete. 2024-04-02T14:03:56.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.597+0000 7f642c5c8640 1 Processor -- start 2024-04-02T14:03:56.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.597+0000 7f642c5c8640 1 -- start start 2024-04-02T14:03:56.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.597+0000 7f642c5c8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6424100970 0x7f64241983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.597+0000 7f642c5c8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6424101330 0x7f6424198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.597+0000 7f642c5c8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6424106980 0x7f642419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.597+0000 7f642c5c8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6424110d90 con 0x7f6424106980 2024-04-02T14:03:56.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.597+0000 7f642c5c8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6424110c10 con 0x7f6424100970 2024-04-02T14:03:56.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.597+0000 7f642c5c8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6424110f10 con 0x7f6424101330 2024-04-02T14:03:56.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.597+0000 7f642ab65640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6424106980 0x7f642419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.597+0000 7f642ab65640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6424106980 0x7f642419cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33156/0 (socket says 172.21.15.52:33156) 2024-04-02T14:03:56.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.597+0000 7f642ab65640 1 -- 172.21.15.52:0/490182915 learned_addr learned my addr 172.21.15.52:0/490182915 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:56.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.597+0000 7f642a364640 1 --2- 172.21.15.52:0/490182915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6424100970 0x7f64241983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.598+0000 7f6429b63640 1 --2- 172.21.15.52:0/490182915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6424101330 0x7f6424198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.598+0000 7f642ab65640 1 -- 172.21.15.52:0/490182915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6424101330 msgr2=0x7f6424198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.598+0000 7f642ab65640 1 --2- 172.21.15.52:0/490182915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6424101330 0x7f6424198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.598+0000 7f642ab65640 1 -- 172.21.15.52:0/490182915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6424100970 msgr2=0x7f64241983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.598+0000 7f642ab65640 1 --2- 172.21.15.52:0/490182915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6424100970 0x7f64241983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.598+0000 7f642ab65640 1 -- 172.21.15.52:0/490182915 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f642419d3d0 con 0x7f6424106980 2024-04-02T14:03:56.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.598+0000 7f642ab65640 1 --2- 172.21.15.52:0/490182915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6424106980 0x7f642419cca0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f641c00eb10 tx=0x7f641c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.598+0000 7f641b7fe640 1 -- 172.21.15.52:0/490182915 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641c004b20 con 0x7f6424106980 2024-04-02T14:03:56.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.598+0000 7f641b7fe640 1 -- 172.21.15.52:0/490182915 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f641c004cc0 con 0x7f6424106980 2024-04-02T14:03:56.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.598+0000 7f641b7fe640 1 -- 172.21.15.52:0/490182915 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641c010760 con 0x7f6424106980 2024-04-02T14:03:56.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.598+0000 7f642c5c8640 1 -- 172.21.15.52:0/490182915 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f642419d6c0 con 0x7f6424106980 2024-04-02T14:03:56.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.598+0000 7f642c5c8640 1 -- 172.21.15.52:0/490182915 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64241a4f50 con 0x7f6424106980 2024-04-02T14:03:56.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.601+0000 7f642f1ab740 1 Processor -- start 2024-04-02T14:03:56.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.601+0000 7f641b7fe640 1 -- 172.21.15.52:0/490182915 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f641c005020 con 0x7f6424106980 2024-04-02T14:03:56.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.602+0000 7f642f1ab740 1 -- start start 2024-04-02T14:03:56.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.602+0000 7f642f1ab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b6b4fe180 0x556b6b4fe560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.602+0000 7f642f1ab740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b6b4feb30 0x556b6b50bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.602+0000 7f642f1ab740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b50c0b0 0x556b6b50e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.602+0000 7f642f1ab740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556b6b4627b0 con 0x556b6b50c0b0 2024-04-02T14:03:56.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.602+0000 7f642f1ab740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556b6b401570 con 0x556b6b4feb30 2024-04-02T14:03:56.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.602+0000 7f642f1ab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556b6b45e760 con 0x556b6b4fe180 2024-04-02T14:03:56.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.602+0000 7f6429b63640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b50c0b0 0x556b6b50e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.602+0000 7f6429b63640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b50c0b0 0x556b6b50e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33164/0 (socket says 172.21.15.52:33164) 2024-04-02T14:03:56.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.602+0000 7f6429b63640 1 -- 172.21.15.52:0/3336510146 learned_addr learned my addr 172.21.15.52:0/3336510146 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:56.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.602+0000 7f641b7fe640 1 --2- 172.21.15.52:0/490182915 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f64100a3da0 0x7f64100a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.602+0000 7f642a364640 1 --2- 172.21.15.52:0/3336510146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b6b4feb30 0x556b6b50bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.602+0000 7f6429b63640 1 --2- 172.21.15.52:0/3336510146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b6b4fe180 0x556b6b4fe560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.602+0000 7f642ab65640 1 --2- 172.21.15.52:0/490182915 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f64100a3da0 0x7f64100a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.602+0000 7f6429b63640 1 -- 172.21.15.52:0/3336510146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b6b4fe180 msgr2=0x556b6b4fe560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.602+0000 7f6429b63640 1 --2- 172.21.15.52:0/3336510146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b6b4fe180 0x556b6b4fe560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.602+0000 7f6429b63640 1 -- 172.21.15.52:0/3336510146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b6b4feb30 msgr2=0x556b6b50bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.602+0000 7f6429b63640 1 --2- 172.21.15.52:0/3336510146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b6b4feb30 0x556b6b50bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.602+0000 7f6429b63640 1 -- 172.21.15.52:0/3336510146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b6b1763f0 con 0x556b6b50c0b0 2024-04-02T14:03:56.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.602+0000 7f642a364640 1 --2- 172.21.15.52:0/3336510146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b6b4feb30 0x556b6b50bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:03:56.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.602+0000 7f641b7fe640 1 -- 172.21.15.52:0/490182915 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f641c0d10f0 con 0x7f6424106980 2024-04-02T14:03:56.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.603+0000 7f6429b63640 1 --2- 172.21.15.52:0/3336510146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b50c0b0 0x556b6b50e4a0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f64200058c0 tx=0x7f64200097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dcbd7edac6cfe14 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.603+0000 7f642ab65640 1 --2- 172.21.15.52:0/490182915 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f64100a3da0 0x7f64100a6260 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f641c09cc90 tx=0x7f641c00efd0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.603+0000 7f6418bfe640 1 -- 172.21.15.52:0/3336510146 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6420004030 con 0x556b6b50c0b0 2024-04-02T14:03:56.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.603+0000 7f6418bfe640 1 -- 172.21.15.52:0/3336510146 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6420010cb0 con 0x556b6b50c0b0 2024-04-02T14:03:56.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.603+0000 7f6418bfe640 1 -- 172.21.15.52:0/3336510146 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6420003970 con 0x556b6b50c0b0 2024-04-02T14:03:56.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.603+0000 7f642f1ab740 1 -- 172.21.15.52:0/3336510146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b50c0b0 msgr2=0x556b6b50e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.603+0000 7f642f1ab740 1 --2- 172.21.15.52:0/3336510146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b50c0b0 0x556b6b50e4a0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f64200058c0 tx=0x7f64200097b0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.604+0000 7f642f1ab740 1 -- 172.21.15.52:0/3336510146 shutdown_connections 2024-04-02T14:03:56.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.604+0000 7f642f1ab740 1 --2- 172.21.15.52:0/3336510146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b50c0b0 0x556b6b50e4a0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.604+0000 7f642f1ab740 1 --2- 172.21.15.52:0/3336510146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b6b4feb30 0x556b6b50bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.604+0000 7f642f1ab740 1 --2- 172.21.15.52:0/3336510146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b6b4fe180 0x556b6b4fe560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.604+0000 7f642f1ab740 1 -- 172.21.15.52:0/3336510146 >> 172.21.15.52:0/3336510146 conn(0x556b6b4f8ed0 msgr2=0x556b6b4f92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.604+0000 7f642f1ab740 1 -- 172.21.15.52:0/3336510146 shutdown_connections 2024-04-02T14:03:56.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.604+0000 7f642f1ab740 1 -- 172.21.15.52:0/3336510146 wait complete. 2024-04-02T14:03:56.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.604+0000 7f642f1ab740 1 -- start start 2024-04-02T14:03:56.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.604+0000 7f642f1ab740 1 Processor -- start 2024-04-02T14:03:56.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.604+0000 7f642f1ab740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b4fe180 0x556b6b508f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.604+0000 7f642f1ab740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b6b4feb30 0x556b6b5094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.604+0000 7f642f1ab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b6b50c0b0 0x556b6b5099e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.604+0000 7f642f1ab740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556b6b50ee70 con 0x556b6b4fe180 2024-04-02T14:03:56.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.604+0000 7f642f1ab740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556b6b41de80 con 0x556b6b4feb30 2024-04-02T14:03:56.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.604+0000 7f642f1ab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556b6b50eff0 con 0x556b6b50c0b0 2024-04-02T14:03:56.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.604+0000 7f642f1ab740 10 client.0 _ll_register_callbacks cb 0x556b6b459aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:56.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.604+0000 7f6429b63640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b4fe180 0x556b6b508f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.604+0000 7f6429b63640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b4fe180 0x556b6b508f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33166/0 (socket says 172.21.15.52:33166) 2024-04-02T14:03:56.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.604+0000 7f6429b63640 1 -- 172.21.15.52:0/733785340 learned_addr learned my addr 172.21.15.52:0/733785340 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:56.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.605+0000 7f642a364640 1 --2- 172.21.15.52:0/733785340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b6b4feb30 0x556b6b5094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.605+0000 7f6429b63640 1 --2- 172.21.15.52:0/733785340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b6b50c0b0 0x556b6b5099e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.605+0000 7f6429b63640 1 -- 172.21.15.52:0/733785340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b6b50c0b0 msgr2=0x556b6b5099e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.605+0000 7f6429b63640 1 --2- 172.21.15.52:0/733785340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b6b50c0b0 0x556b6b5099e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.605+0000 7f6429b63640 1 -- 172.21.15.52:0/733785340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b6b4feb30 msgr2=0x556b6b5094a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.605+0000 7f6429b63640 1 --2- 172.21.15.52:0/733785340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b6b4feb30 0x556b6b5094a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.605+0000 7f6429b63640 1 -- 172.21.15.52:0/733785340 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b6b2407d0 con 0x556b6b4fe180 2024-04-02T14:03:56.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.605+0000 7f6429b63640 1 --2- 172.21.15.52:0/733785340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b4fe180 0x556b6b508f60 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f64200096a0 tx=0x7f6420009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456042b54ff73514 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.605+0000 7f640dffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:56.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.605+0000 7f640dffb640 1 -- 172.21.15.52:0/733785340 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642001f840 con 0x556b6b4fe180 2024-04-02T14:03:56.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.605+0000 7f640dffb640 1 -- 172.21.15.52:0/733785340 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f642001f9e0 con 0x556b6b4fe180 2024-04-02T14:03:56.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.605+0000 7f642f1ab740 1 -- 172.21.15.52:0/733785340 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6420010cb0 con 0x556b6b4fe180 2024-04-02T14:03:56.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.605+0000 7f640dffb640 1 -- 172.21.15.52:0/733785340 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6420035050 con 0x556b6b4fe180 2024-04-02T14:03:56.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.605+0000 7f642f1ab740 10 client.18558 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:56.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.605+0000 7f642f1ab740 1 -- 172.21.15.52:0/733785340 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556b6b51b010 con 0x556b6b4fe180 2024-04-02T14:03:56.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.605+0000 7f640dffb640 1 -- 172.21.15.52:0/733785340 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6420010cb0 con 0x556b6b4fe180 2024-04-02T14:03:56.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.606+0000 7f640dffb640 1 -- 172.21.15.52:0/733785340 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f642001eb70 con 0x556b6b4fe180 2024-04-02T14:03:56.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.606+0000 7f642f1ab740 10 client.18558 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:56.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.606+0000 7f642f1ab740 10 client.18558 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T14:03:56.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.606+0000 7f642f1ab740 20 client.18558 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:56.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.606+0000 7f642f1ab740 1 --2- 172.21.15.52:0/733785340 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x556b6b51c3c0 0x556b6b51e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.606+0000 7f642f1ab740 4 client.18558 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:03:56.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.606+0000 7f642f1ab740 1 -- 172.21.15.52:0/733785340 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b6b46fcf0 con 0x556b6b51c3c0 2024-04-02T14:03:56.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.606+0000 7f642a364640 1 --2- 172.21.15.52:0/733785340 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x556b6b51c3c0 0x556b6b51e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.606+0000 7f642a364640 1 --2- 172.21.15.52:0/733785340 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x556b6b51c3c0 0x556b6b51e7b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=716aa6dc5c38cdd2 server_cookie=e37a0f893740191 in_seq=0 out_seq=0 2024-04-02T14:03:56.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.606+0000 7f640dffb640 10 client.18558 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T14:03:56.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.607+0000 7f640dffb640 1 -- 172.21.15.52:0/733785340 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556b6b46fcf0 con 0x556b6b51c3c0 2024-04-02T14:03:56.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.607+0000 7f640dffb640 10 client.18558 handle_command_reply: tid=0 2024-04-02T14:03:56.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.607+0000 7f642f1ab740 1 client.18558 shutdown 2024-04-02T14:03:56.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.607+0000 7f642f1ab740 1 -- 172.21.15.52:0/733785340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b4fe180 msgr2=0x556b6b508f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.607+0000 7f642f1ab740 1 --2- 172.21.15.52:0/733785340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b4fe180 0x556b6b508f60 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f64200096a0 tx=0x7f6420009fb0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.607+0000 7f642f1ab740 1 -- 172.21.15.52:0/733785340 shutdown_connections 2024-04-02T14:03:56.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.607+0000 7f642f1ab740 1 --2- 172.21.15.52:0/733785340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b6b50c0b0 0x556b6b5099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.607+0000 7f642f1ab740 1 --2- 172.21.15.52:0/733785340 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x556b6b51c3c0 0x556b6b51e7b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.607+0000 7f642f1ab740 1 --2- 172.21.15.52:0/733785340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b6b4feb30 0x556b6b5094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.607+0000 7f642f1ab740 1 --2- 172.21.15.52:0/733785340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b4fe180 0x556b6b508f60 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.607+0000 7f640dffb640 0 client.18558 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T14:03:56.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.607+0000 7f642f1ab740 1 -- 172.21.15.52:0/733785340 >> 172.21.15.52:0/733785340 conn(0x556b6b4f8ed0 msgr2=0x556b6b4fac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.607+0000 7f642f1ab740 1 -- 172.21.15.52:0/733785340 shutdown_connections 2024-04-02T14:03:56.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.607+0000 7f642f1ab740 1 -- 172.21.15.52:0/733785340 wait complete. 2024-04-02T14:03:56.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.607+0000 7f642f1ab740 20 client.18558 trim_cache size 0 max 16384 2024-04-02T14:03:56.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.621+0000 7f642f1ab740 1 Processor -- start 2024-04-02T14:03:56.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.621+0000 7f642f1ab740 1 -- start start 2024-04-02T14:03:56.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.621+0000 7f642f1ab740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b4fe180 0x556b6b517da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.621+0000 7f642f1ab740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b6b4feb30 0x556b6b523f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.621+0000 7f642f1ab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b6b50c0b0 0x556b6b524460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.621+0000 7f642f1ab740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f64241ace00 con 0x556b6b4fe180 2024-04-02T14:03:56.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.621+0000 7f642f1ab740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f64241ae940 con 0x556b6b4feb30 2024-04-02T14:03:56.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.621+0000 7f642f1ab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f64241ad4d0 con 0x556b6b50c0b0 2024-04-02T14:03:56.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.621+0000 7f6429b63640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b4fe180 0x556b6b517da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.621+0000 7f6429b63640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b4fe180 0x556b6b517da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33172/0 (socket says 172.21.15.52:33172) 2024-04-02T14:03:56.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.621+0000 7f6429b63640 1 -- 172.21.15.52:0/3070521127 learned_addr learned my addr 172.21.15.52:0/3070521127 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:56.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.621+0000 7f642a364640 1 --2- 172.21.15.52:0/3070521127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b6b4feb30 0x556b6b523f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.621+0000 7f6429b63640 1 -- 172.21.15.52:0/3070521127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b6b50c0b0 msgr2=0x556b6b524460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.621+0000 7f6429b63640 1 --2- 172.21.15.52:0/3070521127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b6b50c0b0 0x556b6b524460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.621+0000 7f6429b63640 1 -- 172.21.15.52:0/3070521127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b6b4feb30 msgr2=0x556b6b523f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.621+0000 7f6429b63640 1 --2- 172.21.15.52:0/3070521127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b6b4feb30 0x556b6b523f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.621+0000 7f6429b63640 1 -- 172.21.15.52:0/3070521127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b6b4ff100 con 0x556b6b4fe180 2024-04-02T14:03:56.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.621+0000 7f642a364640 1 --2- 172.21.15.52:0/3070521127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b6b4feb30 0x556b6b523f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:03:56.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.622+0000 7f6429b63640 1 --2- 172.21.15.52:0/3070521127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b4fe180 0x556b6b517da0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f64200059f0 tx=0x7f6420009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7ebaa4d425bad6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.622+0000 7f640ffff640 1 -- 172.21.15.52:0/3070521127 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6420010c40 con 0x556b6b4fe180 2024-04-02T14:03:56.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.622+0000 7f640ffff640 1 -- 172.21.15.52:0/3070521127 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6420029040 con 0x556b6b4fe180 2024-04-02T14:03:56.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.622+0000 7f640ffff640 1 -- 172.21.15.52:0/3070521127 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642000ec90 con 0x556b6b4fe180 2024-04-02T14:03:56.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.622+0000 7f642f1ab740 1 -- 172.21.15.52:0/3070521127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b4fe180 msgr2=0x556b6b517da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.622+0000 7f642f1ab740 1 --2- 172.21.15.52:0/3070521127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b4fe180 0x556b6b517da0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f64200059f0 tx=0x7f6420009550 comp rx=0 tx=0).stop 2024-04-02T14:03:56.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.622+0000 7f642f1ab740 1 -- 172.21.15.52:0/3070521127 shutdown_connections 2024-04-02T14:03:56.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.622+0000 7f642f1ab740 1 --2- 172.21.15.52:0/3070521127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b6b50c0b0 0x556b6b524460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.622+0000 7f642f1ab740 1 --2- 172.21.15.52:0/3070521127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b6b4feb30 0x556b6b523f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.622+0000 7f642f1ab740 1 --2- 172.21.15.52:0/3070521127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b4fe180 0x556b6b517da0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.622+0000 7f642f1ab740 1 -- 172.21.15.52:0/3070521127 >> 172.21.15.52:0/3070521127 conn(0x556b6b4f8ed0 msgr2=0x556b6b4f7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.622+0000 7f642f1ab740 1 -- 172.21.15.52:0/3070521127 shutdown_connections 2024-04-02T14:03:56.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.622+0000 7f642f1ab740 1 -- 172.21.15.52:0/3070521127 wait complete. 2024-04-02T14:03:56.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.623+0000 7f642f1ab740 1 -- start start 2024-04-02T14:03:56.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.623+0000 7f642f1ab740 1 Processor -- start 2024-04-02T14:03:56.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.623+0000 7f642f1ab740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b6b4fe180 0x556b6b5055a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.623+0000 7f642f1ab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b6b4feb30 0x556b6b505ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.623+0000 7f642f1ab740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b50c0b0 0x556b6b506020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.623+0000 7f642f1ab740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f64241ae270 con 0x556b6b50c0b0 2024-04-02T14:03:56.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.623+0000 7f642f1ab740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f64241adba0 con 0x556b6b4fe180 2024-04-02T14:03:56.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.623+0000 7f642f1ab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f64241aae00 con 0x556b6b4feb30 2024-04-02T14:03:56.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.623+0000 7f642f1ab740 10 client.0 _ll_register_callbacks cb 0x7f6420005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:56.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.623+0000 7f6429b63640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b50c0b0 0x556b6b506020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.623+0000 7f6429b63640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b50c0b0 0x556b6b506020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33186/0 (socket says 172.21.15.52:33186) 2024-04-02T14:03:56.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.623+0000 7f6429b63640 1 -- 172.21.15.52:0/299384368 learned_addr learned my addr 172.21.15.52:0/299384368 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:56.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.623+0000 7f6429b63640 1 --2- 172.21.15.52:0/299384368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b6b4fe180 0x556b6b5055a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.623+0000 7f642a364640 1 --2- 172.21.15.52:0/299384368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b6b4feb30 0x556b6b505ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.623+0000 7f6429b63640 1 -- 172.21.15.52:0/299384368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b6b4feb30 msgr2=0x556b6b505ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.623+0000 7f6429b63640 1 --2- 172.21.15.52:0/299384368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b6b4feb30 0x556b6b505ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.623+0000 7f6429b63640 1 -- 172.21.15.52:0/299384368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b6b4fe180 msgr2=0x556b6b5055a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.623+0000 7f6429b63640 1 --2- 172.21.15.52:0/299384368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b6b4fe180 0x556b6b5055a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.623+0000 7f6429b63640 1 -- 172.21.15.52:0/299384368 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b6b513550 con 0x556b6b50c0b0 2024-04-02T14:03:56.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.624+0000 7f6429b63640 1 --2- 172.21.15.52:0/299384368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b50c0b0 0x556b6b506020 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x556b6b5089e0 tx=0x7f6420009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e88724aa611f42 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.624+0000 7f640dffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:56.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.624+0000 7f640dffb640 1 -- 172.21.15.52:0/299384368 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642000ea30 con 0x556b6b50c0b0 2024-04-02T14:03:56.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.624+0000 7f640dffb640 1 -- 172.21.15.52:0/299384368 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f642000ebd0 con 0x556b6b50c0b0 2024-04-02T14:03:56.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.624+0000 7f642f1ab740 1 -- 172.21.15.52:0/299384368 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6420029040 con 0x556b6b50c0b0 2024-04-02T14:03:56.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.624+0000 7f640dffb640 1 -- 172.21.15.52:0/299384368 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6420005110 con 0x556b6b50c0b0 2024-04-02T14:03:56.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.624+0000 7f642f1ab740 10 client.18564 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:56.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.624+0000 7f642f1ab740 1 -- 172.21.15.52:0/299384368 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556b6b514850 con 0x556b6b50c0b0 2024-04-02T14:03:56.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.624+0000 7f640dffb640 1 -- 172.21.15.52:0/299384368 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6420029040 con 0x556b6b50c0b0 2024-04-02T14:03:56.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.624+0000 7f640dffb640 1 -- 172.21.15.52:0/299384368 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f642001fad0 con 0x556b6b50c0b0 2024-04-02T14:03:56.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.624+0000 7f642f1ab740 10 client.18564 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:56.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.624+0000 7f642f1ab740 10 client.18564 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-04-02T14:03:56.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.624+0000 7f642f1ab740 20 client.18564 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:56.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.624+0000 7f642f1ab740 1 --2- 172.21.15.52:0/299384368 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x556b6b582940 0x556b6b584d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.624+0000 7f642f1ab740 4 client.18564 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:03:56.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.624+0000 7f642f1ab740 1 -- 172.21.15.52:0/299384368 --> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b6b46a040 con 0x556b6b582940 2024-04-02T14:03:56.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.625+0000 7f642a364640 1 --2- 172.21.15.52:0/299384368 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x556b6b582940 0x556b6b584d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.625+0000 7f640dffb640 10 client.18564 ms_handle_connect on v2:172.21.15.88:6836/1091039373 2024-04-02T14:03:56.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.625+0000 7f642a364640 1 --2- 172.21.15.52:0/299384368 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x556b6b582940 0x556b6b584d30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=842902052fe5778d server_cookie=aa0bde81ba4d518c in_seq=0 out_seq=0 2024-04-02T14:03:56.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.625+0000 7f640dffb640 1 -- 172.21.15.52:0/299384368 <== mds.3 v2:172.21.15.88:6836/1091039373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b6b46a040 con 0x556b6b582940 2024-04-02T14:03:56.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.625+0000 7f640dffb640 10 client.18564 handle_command_reply: tid=0 2024-04-02T14:03:56.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.625+0000 7f642f1ab740 1 client.18564 shutdown 2024-04-02T14:03:56.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.625+0000 7f642f1ab740 1 -- 172.21.15.52:0/299384368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b50c0b0 msgr2=0x556b6b506020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.625+0000 7f642f1ab740 1 --2- 172.21.15.52:0/299384368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b50c0b0 0x556b6b506020 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x556b6b5089e0 tx=0x7f6420009d90 comp rx=0 tx=0).stop 2024-04-02T14:03:56.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.625+0000 7f642f1ab740 1 -- 172.21.15.52:0/299384368 shutdown_connections 2024-04-02T14:03:56.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.625+0000 7f642f1ab740 1 --2- 172.21.15.52:0/299384368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b6b50c0b0 0x556b6b506020 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.625+0000 7f642f1ab740 1 --2- 172.21.15.52:0/299384368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b6b4feb30 0x556b6b505ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.625+0000 7f642f1ab740 1 --2- 172.21.15.52:0/299384368 >> [v2:172.21.15.88:6836/1091039373,v1:172.21.15.88:6837/1091039373] conn(0x556b6b582940 0x556b6b584d30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.626+0000 7f642f1ab740 1 --2- 172.21.15.52:0/299384368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b6b4fe180 0x556b6b5055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.626+0000 7f640dffb640 0 client.18564 ms_handle_reset on v2:172.21.15.88:6836/1091039373 2024-04-02T14:03:56.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.626+0000 7f642f1ab740 1 -- 172.21.15.52:0/299384368 >> 172.21.15.52:0/299384368 conn(0x556b6b4f8ed0 msgr2=0x556b6b508f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.626+0000 7f642f1ab740 1 -- 172.21.15.52:0/299384368 shutdown_connections 2024-04-02T14:03:56.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.626+0000 7f642f1ab740 1 -- 172.21.15.52:0/299384368 wait complete. 2024-04-02T14:03:56.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.626+0000 7f642f1ab740 20 client.18564 trim_cache size 0 max 16384 2024-04-02T14:03:56.651 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:03:56.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.626+0000 7f642c5c8640 1 -- 172.21.15.52:0/490182915 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f64100a3da0 msgr2=0x7f64100a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.626+0000 7f642c5c8640 1 --2- 172.21.15.52:0/490182915 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f64100a3da0 0x7f64100a6260 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f641c09cc90 tx=0x7f641c00efd0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.626+0000 7f642c5c8640 1 -- 172.21.15.52:0/490182915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6424106980 msgr2=0x7f642419cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.626+0000 7f642c5c8640 1 --2- 172.21.15.52:0/490182915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6424106980 0x7f642419cca0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f641c00eb10 tx=0x7f641c00c2d0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.626+0000 7f642c5c8640 1 -- 172.21.15.52:0/490182915 shutdown_connections 2024-04-02T14:03:56.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.626+0000 7f642c5c8640 1 --2- 172.21.15.52:0/490182915 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f64100a3da0 0x7f64100a6260 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.626+0000 7f642c5c8640 1 --2- 172.21.15.52:0/490182915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6424106980 0x7f642419cca0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.626+0000 7f642c5c8640 1 --2- 172.21.15.52:0/490182915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6424101330 0x7f6424198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.626+0000 7f642c5c8640 1 --2- 172.21.15.52:0/490182915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6424100970 0x7f64241983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.626+0000 7f642c5c8640 1 -- 172.21.15.52:0/490182915 >> 172.21.15.52:0/490182915 conn(0x7f6424077b00 msgr2=0x7f6424068fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.626+0000 7f642c5c8640 1 -- 172.21.15.52:0/490182915 shutdown_connections 2024-04-02T14:03:56.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.626+0000 7f642c5c8640 1 -- 172.21.15.52:0/490182915 wait complete. 2024-04-02T14:03:56.655 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T14:03:56.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.757+0000 7f77b42eb640 1 -- 172.21.15.52:0/1505800920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77ac107a80 msgr2=0x7f77ac107e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.757+0000 7f77b42eb640 1 --2- 172.21.15.52:0/1505800920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77ac107a80 0x7f77ac107e60 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f77a8009930 tx=0x7f77a802bc30 comp rx=0 tx=0).stop 2024-04-02T14:03:56.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.757+0000 7f77b42eb640 1 -- 172.21.15.52:0/1505800920 shutdown_connections 2024-04-02T14:03:56.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.757+0000 7f77b42eb640 1 --2- 172.21.15.52:0/1505800920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77ac0696a0 0x7f77ac10e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.757+0000 7f77b42eb640 1 --2- 172.21.15.52:0/1505800920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77ac068ce0 0x7f77ac069160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.757+0000 7f77b42eb640 1 --2- 172.21.15.52:0/1505800920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77ac107a80 0x7f77ac107e60 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.757+0000 7f77b42eb640 1 -- 172.21.15.52:0/1505800920 >> 172.21.15.52:0/1505800920 conn(0x7f77ac077b00 msgr2=0x7f77ac077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.757+0000 7f77b42eb640 1 -- 172.21.15.52:0/1505800920 shutdown_connections 2024-04-02T14:03:56.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.757+0000 7f77b42eb640 1 -- 172.21.15.52:0/1505800920 wait complete. 2024-04-02T14:03:56.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.758+0000 7f77b42eb640 1 Processor -- start 2024-04-02T14:03:56.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.758+0000 7f77b42eb640 1 -- start start 2024-04-02T14:03:56.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.758+0000 7f77b42eb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77ac068ce0 0x7f77ac1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.758+0000 7f77b42eb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77ac0696a0 0x7f77ac198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.758+0000 7f77b42eb640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77ac107a80 0x7f77ac19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.758+0000 7f77b42eb640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac111e90 con 0x7f77ac0696a0 2024-04-02T14:03:56.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.758+0000 7f77b42eb640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac111d10 con 0x7f77ac107a80 2024-04-02T14:03:56.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.758+0000 7f77b42eb640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac112010 con 0x7f77ac068ce0 2024-04-02T14:03:56.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.758+0000 7f77b1886640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77ac0696a0 0x7f77ac198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.758+0000 7f77b1886640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77ac0696a0 0x7f77ac198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33210/0 (socket says 172.21.15.52:33210) 2024-04-02T14:03:56.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.758+0000 7f77b1886640 1 -- 172.21.15.52:0/377030185 learned_addr learned my addr 172.21.15.52:0/377030185 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:56.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.758+0000 7f77b2087640 1 --2- 172.21.15.52:0/377030185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77ac068ce0 0x7f77ac1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.758+0000 7f77b2888640 1 --2- 172.21.15.52:0/377030185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77ac107a80 0x7f77ac19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.758+0000 7f77b1886640 1 -- 172.21.15.52:0/377030185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77ac068ce0 msgr2=0x7f77ac1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.758+0000 7f77b1886640 1 --2- 172.21.15.52:0/377030185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77ac068ce0 0x7f77ac1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.758+0000 7f77b1886640 1 -- 172.21.15.52:0/377030185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77ac107a80 msgr2=0x7f77ac19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.758+0000 7f77b1886640 1 --2- 172.21.15.52:0/377030185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77ac107a80 0x7f77ac19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.758+0000 7f77b1886640 1 -- 172.21.15.52:0/377030185 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ac19d3e0 con 0x7f77ac0696a0 2024-04-02T14:03:56.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.758+0000 7f77b2888640 1 --2- 172.21.15.52:0/377030185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77ac107a80 0x7f77ac19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:03:56.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.759+0000 7f77b1886640 1 --2- 172.21.15.52:0/377030185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77ac0696a0 0x7f77ac198930 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f779c00d770 tx=0x7f779c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.759+0000 7f77a37fe640 1 -- 172.21.15.52:0/377030185 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c017070 con 0x7f77ac0696a0 2024-04-02T14:03:56.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.759+0000 7f77a37fe640 1 -- 172.21.15.52:0/377030185 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f779c005500 con 0x7f77ac0696a0 2024-04-02T14:03:56.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.759+0000 7f77b42eb640 1 -- 172.21.15.52:0/377030185 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77ac19d6d0 con 0x7f77ac0696a0 2024-04-02T14:03:56.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.759+0000 7f77a37fe640 1 -- 172.21.15.52:0/377030185 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c004e00 con 0x7f77ac0696a0 2024-04-02T14:03:56.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.759+0000 7f77b42eb640 1 -- 172.21.15.52:0/377030185 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77ac1a4f60 con 0x7f77ac0696a0 2024-04-02T14:03:56.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.762+0000 7f77b6fab740 1 Processor -- start 2024-04-02T14:03:56.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.762+0000 7f77a37fe640 1 -- 172.21.15.52:0/377030185 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f779c004fa0 con 0x7f77ac0696a0 2024-04-02T14:03:56.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.762+0000 7f77b6fab740 1 -- start start 2024-04-02T14:03:56.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.762+0000 7f77b6fab740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09d55b0 0x55d0d09d5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.762+0000 7f77b6fab740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0d09d5f60 0x55d0d09e2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.762+0000 7f77b6fab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0d09e30b0 0x55d0d09e54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.762+0000 7f77b6fab740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d0d09397b0 con 0x55d0d09d55b0 2024-04-02T14:03:56.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.762+0000 7f77b6fab740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d0d08d8570 con 0x55d0d09d5f60 2024-04-02T14:03:56.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.762+0000 7f77b6fab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d0d0935760 con 0x55d0d09e30b0 2024-04-02T14:03:56.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.762+0000 7f77b2888640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09d55b0 0x55d0d09d5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.762+0000 7f77b2888640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09d55b0 0x55d0d09d5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33226/0 (socket says 172.21.15.52:33226) 2024-04-02T14:03:56.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.762+0000 7f77b2888640 1 -- 172.21.15.52:0/2840991356 learned_addr learned my addr 172.21.15.52:0/2840991356 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:56.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.763+0000 7f77b2087640 1 --2- 172.21.15.52:0/2840991356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0d09d5f60 0x55d0d09e2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.763+0000 7f77b1886640 1 --2- 172.21.15.52:0/2840991356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0d09e30b0 0x55d0d09e54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.763+0000 7f77a37fe640 1 --2- 172.21.15.52:0/377030185 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f77980a3d20 0x7f77980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.763+0000 7f77b2888640 1 --2- 172.21.15.52:0/377030185 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f77980a3d20 0x7f77980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.763+0000 7f77b2087640 1 -- 172.21.15.52:0/2840991356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0d09e30b0 msgr2=0x55d0d09e54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.763+0000 7f77b2087640 1 --2- 172.21.15.52:0/2840991356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0d09e30b0 0x55d0d09e54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.763+0000 7f77b2087640 1 -- 172.21.15.52:0/2840991356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09d55b0 msgr2=0x55d0d09d5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.763+0000 7f77b2087640 1 --2- 172.21.15.52:0/2840991356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09d55b0 0x55d0d09d5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.763+0000 7f77a37fe640 1 -- 172.21.15.52:0/377030185 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f779c0d05d0 con 0x7f77ac0696a0 2024-04-02T14:03:56.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.763+0000 7f77b2087640 1 -- 172.21.15.52:0/2840991356 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0d064d3f0 con 0x55d0d09d5f60 2024-04-02T14:03:56.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.763+0000 7f77b2888640 1 --2- 172.21.15.52:0/2840991356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09d55b0 0x55d0d09d5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:03:56.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.763+0000 7f77b2087640 1 --2- 172.21.15.52:0/2840991356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0d09d5f60 0x55d0d09e2b70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f77a802c170 tx=0x7f77a8009670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=866375e73e179f2a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.763+0000 7f77b2888640 1 --2- 172.21.15.52:0/377030185 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f77980a3d20 0x7f77980a61e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f77a400afd0 tx=0x7f77a4004ab0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.763+0000 7f77a0bfe640 1 -- 172.21.15.52:0/2840991356 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a802ca80 con 0x55d0d09d5f60 2024-04-02T14:03:56.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.764+0000 7f77a0bfe640 1 -- 172.21.15.52:0/2840991356 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77a802cc20 con 0x55d0d09d5f60 2024-04-02T14:03:56.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.764+0000 7f77a0bfe640 1 -- 172.21.15.52:0/2840991356 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a80051b0 con 0x55d0d09d5f60 2024-04-02T14:03:56.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.764+0000 7f77b6fab740 1 -- 172.21.15.52:0/2840991356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0d09d5f60 msgr2=0x55d0d09e2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.764+0000 7f77b6fab740 1 --2- 172.21.15.52:0/2840991356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0d09d5f60 0x55d0d09e2b70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f77a802c170 tx=0x7f77a8009670 comp rx=0 tx=0).stop 2024-04-02T14:03:56.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.764+0000 7f77b6fab740 1 -- 172.21.15.52:0/2840991356 shutdown_connections 2024-04-02T14:03:56.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.764+0000 7f77b6fab740 1 --2- 172.21.15.52:0/2840991356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0d09e30b0 0x55d0d09e54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.764+0000 7f77b6fab740 1 --2- 172.21.15.52:0/2840991356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0d09d5f60 0x55d0d09e2b70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.764+0000 7f77b6fab740 1 --2- 172.21.15.52:0/2840991356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09d55b0 0x55d0d09d5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.764+0000 7f77b6fab740 1 -- 172.21.15.52:0/2840991356 >> 172.21.15.52:0/2840991356 conn(0x55d0d09cfed0 msgr2=0x55d0d09d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.764+0000 7f77b6fab740 1 -- 172.21.15.52:0/2840991356 shutdown_connections 2024-04-02T14:03:56.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.764+0000 7f77b6fab740 1 -- 172.21.15.52:0/2840991356 wait complete. 2024-04-02T14:03:56.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.765+0000 7f77b6fab740 1 -- start start 2024-04-02T14:03:56.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.765+0000 7f77b6fab740 1 Processor -- start 2024-04-02T14:03:56.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.765+0000 7f77b6fab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0d09d55b0 0x55d0d09e2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.765+0000 7f77b6fab740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09d5f60 0x55d0d09dfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.765+0000 7f77b6fab740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0d09e30b0 0x55d0d09e05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.765+0000 7f77b6fab740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d0d09e5e10 con 0x55d0d09d5f60 2024-04-02T14:03:56.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.765+0000 7f77b6fab740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d0d08f4e80 con 0x55d0d09e30b0 2024-04-02T14:03:56.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.765+0000 7f77b6fab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d0d09e5f90 con 0x55d0d09d55b0 2024-04-02T14:03:56.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.765+0000 7f77b6fab740 10 client.0 _ll_register_callbacks cb 0x55d0d0930aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:56.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.765+0000 7f77b1886640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09d5f60 0x55d0d09dfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.765+0000 7f77b1886640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09d5f60 0x55d0d09dfe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33230/0 (socket says 172.21.15.52:33230) 2024-04-02T14:03:56.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.765+0000 7f77b1886640 1 -- 172.21.15.52:0/1835595411 learned_addr learned my addr 172.21.15.52:0/1835595411 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:56.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.765+0000 7f77b2087640 1 --2- 172.21.15.52:0/1835595411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0d09d55b0 0x55d0d09e2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.765+0000 7f77b2888640 1 --2- 172.21.15.52:0/1835595411 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0d09e30b0 0x55d0d09e05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.766+0000 7f77b1886640 1 -- 172.21.15.52:0/1835595411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0d09d55b0 msgr2=0x55d0d09e2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.766+0000 7f77b1886640 1 --2- 172.21.15.52:0/1835595411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0d09d55b0 0x55d0d09e2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.766+0000 7f77b1886640 1 -- 172.21.15.52:0/1835595411 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0d09e30b0 msgr2=0x55d0d09e05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.766+0000 7f77b1886640 1 --2- 172.21.15.52:0/1835595411 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0d09e30b0 0x55d0d09e05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.766+0000 7f77b1886640 1 -- 172.21.15.52:0/1835595411 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0d07177d0 con 0x55d0d09d5f60 2024-04-02T14:03:56.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.766+0000 7f77b1886640 1 --2- 172.21.15.52:0/1835595411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09d5f60 0x55d0d09dfe80 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f779c09c3a0 tx=0x7f779c09c1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441885c4bb8b84bf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.766+0000 7f7795ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:56.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.766+0000 7f7795ffb640 1 -- 172.21.15.52:0/1835595411 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c020040 con 0x55d0d09d5f60 2024-04-02T14:03:56.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.766+0000 7f77b6fab740 1 -- 172.21.15.52:0/1835595411 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77a802cc20 con 0x55d0d09d5f60 2024-04-02T14:03:56.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.766+0000 7f7795ffb640 1 -- 172.21.15.52:0/1835595411 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f779c0cdc10 con 0x55d0d09d5f60 2024-04-02T14:03:56.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.766+0000 7f7795ffb640 1 -- 172.21.15.52:0/1835595411 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f779c09c8e0 con 0x55d0d09d5f60 2024-04-02T14:03:56.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.766+0000 7f77b6fab740 10 client.18576 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:56.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.766+0000 7f77b6fab740 1 -- 172.21.15.52:0/1835595411 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d0d09f1f30 con 0x55d0d09d5f60 2024-04-02T14:03:56.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.766+0000 7f7795ffb640 1 -- 172.21.15.52:0/1835595411 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77a802cc20 con 0x55d0d09d5f60 2024-04-02T14:03:56.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.766+0000 7f7795ffb640 1 -- 172.21.15.52:0/1835595411 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f779c0d0270 con 0x55d0d09d5f60 2024-04-02T14:03:56.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.766+0000 7f77b6fab740 10 client.18576 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:56.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.766+0000 7f77b6fab740 10 client.18576 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T14:03:56.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.766+0000 7f77b6fab740 20 client.18576 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:56.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.767+0000 7f77b6fab740 1 --2- 172.21.15.52:0/1835595411 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55d0d09f33f0 0x55d0d09f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.767+0000 7f77b6fab740 4 client.18576 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:03:56.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.767+0000 7f77b6fab740 1 -- 172.21.15.52:0/1835595411 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0d0946cf0 con 0x55d0d09f33f0 2024-04-02T14:03:56.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.767+0000 7f77b2087640 1 --2- 172.21.15.52:0/1835595411 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55d0d09f33f0 0x55d0d09f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.767+0000 7f77b2087640 1 --2- 172.21.15.52:0/1835595411 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55d0d09f33f0 0x55d0d09f57e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b2d408ceb1cefbd0 server_cookie=e3ac5cff836d5c07 in_seq=0 out_seq=0 2024-04-02T14:03:56.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.767+0000 7f7795ffb640 10 client.18576 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T14:03:56.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.768+0000 7f7795ffb640 1 -- 172.21.15.52:0/1835595411 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d0d0946cf0 con 0x55d0d09f33f0 2024-04-02T14:03:56.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.768+0000 7f7795ffb640 10 client.18576 handle_command_reply: tid=0 2024-04-02T14:03:56.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.768+0000 7f77b6fab740 1 client.18576 shutdown 2024-04-02T14:03:56.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.768+0000 7f77b6fab740 1 -- 172.21.15.52:0/1835595411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09d5f60 msgr2=0x55d0d09dfe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.768+0000 7f77b6fab740 1 --2- 172.21.15.52:0/1835595411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09d5f60 0x55d0d09dfe80 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f779c09c3a0 tx=0x7f779c09c1e0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.768+0000 7f77b6fab740 1 -- 172.21.15.52:0/1835595411 shutdown_connections 2024-04-02T14:03:56.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.768+0000 7f77b6fab740 1 --2- 172.21.15.52:0/1835595411 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55d0d09f33f0 0x55d0d09f57e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.768+0000 7f77b6fab740 1 --2- 172.21.15.52:0/1835595411 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0d09e30b0 0x55d0d09e05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.768+0000 7f7795ffb640 0 client.18576 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T14:03:56.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.768+0000 7f77b6fab740 1 --2- 172.21.15.52:0/1835595411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09d5f60 0x55d0d09dfe80 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.768+0000 7f77b6fab740 1 --2- 172.21.15.52:0/1835595411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0d09d55b0 0x55d0d09e2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.768+0000 7f77b6fab740 1 -- 172.21.15.52:0/1835595411 >> 172.21.15.52:0/1835595411 conn(0x55d0d09cfed0 msgr2=0x55d0d09d1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.768+0000 7f77b6fab740 1 -- 172.21.15.52:0/1835595411 shutdown_connections 2024-04-02T14:03:56.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.768+0000 7f77b6fab740 1 -- 172.21.15.52:0/1835595411 wait complete. 2024-04-02T14:03:56.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.768+0000 7f77b6fab740 20 client.18576 trim_cache size 0 max 16384 2024-04-02T14:03:56.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:56 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/482048443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:03:56.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.781+0000 7f77b6fab740 1 Processor -- start 2024-04-02T14:03:56.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.781+0000 7f77b6fab740 1 -- start start 2024-04-02T14:03:56.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.782+0000 7f77b6fab740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0d09d55b0 0x55d0d09eed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.782+0000 7f77b6fab740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09d5f60 0x55d0d09dee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.782+0000 7f77b6fab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0d09e30b0 0x55d0d09fafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.782+0000 7f77b6fab740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac1acc70 con 0x55d0d09d5f60 2024-04-02T14:03:56.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.782+0000 7f77b6fab740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac1ae7b0 con 0x55d0d09d55b0 2024-04-02T14:03:56.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.782+0000 7f77b6fab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac1ad340 con 0x55d0d09e30b0 2024-04-02T14:03:56.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.782+0000 7f77b1886640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09d5f60 0x55d0d09dee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.782+0000 7f77b1886640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09d5f60 0x55d0d09dee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33234/0 (socket says 172.21.15.52:33234) 2024-04-02T14:03:56.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.782+0000 7f77b1886640 1 -- 172.21.15.52:0/2941848434 learned_addr learned my addr 172.21.15.52:0/2941848434 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:56.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.782+0000 7f77b2888640 1 --2- 172.21.15.52:0/2941848434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0d09e30b0 0x55d0d09fafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.782+0000 7f77b1886640 1 -- 172.21.15.52:0/2941848434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0d09e30b0 msgr2=0x55d0d09fafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.782+0000 7f77b1886640 1 --2- 172.21.15.52:0/2941848434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0d09e30b0 0x55d0d09fafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.782+0000 7f77b1886640 1 -- 172.21.15.52:0/2941848434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0d09d55b0 msgr2=0x55d0d09eed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.782+0000 7f77b1886640 1 --2- 172.21.15.52:0/2941848434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0d09d55b0 0x55d0d09eed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.782+0000 7f77b1886640 1 -- 172.21.15.52:0/2941848434 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0d09d5d00 con 0x55d0d09d5f60 2024-04-02T14:03:56.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.782+0000 7f77b2888640 1 --2- 172.21.15.52:0/2941848434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0d09e30b0 0x55d0d09fafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:03:56.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.782+0000 7f77b1886640 1 --2- 172.21.15.52:0/2941848434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09d5f60 0x55d0d09dee60 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f779c09bee0 tx=0x7f779c09a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca38ede3964efe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.782+0000 7f7797fff640 1 -- 172.21.15.52:0/2941848434 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c014020 con 0x55d0d09d5f60 2024-04-02T14:03:56.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.782+0000 7f7797fff640 1 -- 172.21.15.52:0/2941848434 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f779c0d0030 con 0x55d0d09d5f60 2024-04-02T14:03:56.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.782+0000 7f7797fff640 1 -- 172.21.15.52:0/2941848434 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c020070 con 0x55d0d09d5f60 2024-04-02T14:03:56.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.783+0000 7f77b6fab740 1 -- 172.21.15.52:0/2941848434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09d5f60 msgr2=0x55d0d09dee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.783+0000 7f77b6fab740 1 --2- 172.21.15.52:0/2941848434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09d5f60 0x55d0d09dee60 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f779c09bee0 tx=0x7f779c09a6f0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.783+0000 7f77b6fab740 1 -- 172.21.15.52:0/2941848434 shutdown_connections 2024-04-02T14:03:56.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.783+0000 7f77b6fab740 1 --2- 172.21.15.52:0/2941848434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0d09e30b0 0x55d0d09fafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.783+0000 7f77b6fab740 1 --2- 172.21.15.52:0/2941848434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09d5f60 0x55d0d09dee60 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.783+0000 7f77b6fab740 1 --2- 172.21.15.52:0/2941848434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0d09d55b0 0x55d0d09eed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.783+0000 7f77b6fab740 1 -- 172.21.15.52:0/2941848434 >> 172.21.15.52:0/2941848434 conn(0x55d0d09cfed0 msgr2=0x55d0d09de750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.783+0000 7f77b6fab740 1 -- 172.21.15.52:0/2941848434 shutdown_connections 2024-04-02T14:03:56.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.783+0000 7f77b6fab740 1 -- 172.21.15.52:0/2941848434 wait complete. 2024-04-02T14:03:56.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.783+0000 7f77b6fab740 1 -- start start 2024-04-02T14:03:56.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.783+0000 7f77b6fab740 1 Processor -- start 2024-04-02T14:03:56.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.784+0000 7f77b6fab740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0d09d55b0 0x55d0d09eb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.784+0000 7f77b6fab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0d09d5f60 0x55d0d09dc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.784+0000 7f77b6fab740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09e30b0 0x55d0d09dcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.784+0000 7f77b6fab740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac1ae0e0 con 0x55d0d09e30b0 2024-04-02T14:03:56.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.784+0000 7f77b6fab740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac1ada10 con 0x55d0d09d55b0 2024-04-02T14:03:56.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.784+0000 7f77b6fab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac1aac70 con 0x55d0d09d5f60 2024-04-02T14:03:56.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.784+0000 7f77b6fab740 10 client.0 _ll_register_callbacks cb 0x7f779c00de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:56.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.784+0000 7f77b2888640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09e30b0 0x55d0d09dcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.784+0000 7f77b2888640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09e30b0 0x55d0d09dcd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33242/0 (socket says 172.21.15.52:33242) 2024-04-02T14:03:56.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.784+0000 7f77b2888640 1 -- 172.21.15.52:0/3338709967 learned_addr learned my addr 172.21.15.52:0/3338709967 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:56.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.784+0000 7f77b2087640 1 --2- 172.21.15.52:0/3338709967 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0d09d55b0 0x55d0d09eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.784+0000 7f77b1886640 1 --2- 172.21.15.52:0/3338709967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0d09d5f60 0x55d0d09dc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.784+0000 7f77b2888640 1 -- 172.21.15.52:0/3338709967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0d09d5f60 msgr2=0x55d0d09dc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.784+0000 7f77b2888640 1 --2- 172.21.15.52:0/3338709967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0d09d5f60 0x55d0d09dc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.784+0000 7f77b2888640 1 -- 172.21.15.52:0/3338709967 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0d09d55b0 msgr2=0x55d0d09eb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.784+0000 7f77b2888640 1 --2- 172.21.15.52:0/3338709967 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0d09d55b0 0x55d0d09eb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.784+0000 7f77b2888640 1 -- 172.21.15.52:0/3338709967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0d09dd550 con 0x55d0d09e30b0 2024-04-02T14:03:56.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.784+0000 7f77b1886640 1 --2- 172.21.15.52:0/3338709967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0d09d5f60 0x55d0d09dc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:03:56.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.784+0000 7f77b2888640 1 --2- 172.21.15.52:0/3338709967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09e30b0 0x55d0d09dcd20 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f77a400ec60 tx=0x7f77a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daed72f34a7945f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.784+0000 7f7795ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:56.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.784+0000 7f7795ffb640 1 -- 172.21.15.52:0/3338709967 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4018070 con 0x55d0d09e30b0 2024-04-02T14:03:56.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.785+0000 7f7795ffb640 1 -- 172.21.15.52:0/3338709967 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f77a4021db0 con 0x55d0d09e30b0 2024-04-02T14:03:56.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.785+0000 7f77b6fab740 1 -- 172.21.15.52:0/3338709967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f779c0d0030 con 0x55d0d09e30b0 2024-04-02T14:03:56.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.785+0000 7f7795ffb640 1 -- 172.21.15.52:0/3338709967 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f77a4004b20 con 0x55d0d09e30b0 2024-04-02T14:03:56.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.785+0000 7f77b6fab740 10 client.18582 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:56.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.785+0000 7f77b6fab740 1 -- 172.21.15.52:0/3338709967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d0d0a59830 con 0x55d0d09e30b0 2024-04-02T14:03:56.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.785+0000 7f7795ffb640 1 -- 172.21.15.52:0/3338709967 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f779c0d0030 con 0x55d0d09e30b0 2024-04-02T14:03:56.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.785+0000 7f7795ffb640 1 -- 172.21.15.52:0/3338709967 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f77a4028070 con 0x55d0d09e30b0 2024-04-02T14:03:56.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.785+0000 7f77b6fab740 10 client.18582 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:56.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.785+0000 7f77b6fab740 10 client.18582 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-02T14:03:56.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.785+0000 7f77b6fab740 20 client.18582 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:56.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.785+0000 7f77b6fab740 1 --2- 172.21.15.52:0/3338709967 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55d0d0a59f50 0x55d0d0a5c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.785+0000 7f77b6fab740 4 client.18582 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:03:56.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.785+0000 7f77b6fab740 1 -- 172.21.15.52:0/3338709967 --> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d0d0941040 con 0x55d0d0a59f50 2024-04-02T14:03:56.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.785+0000 7f77b2087640 1 --2- 172.21.15.52:0/3338709967 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55d0d0a59f50 0x55d0d0a5c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.786+0000 7f7795ffb640 10 client.18582 ms_handle_connect on v2:172.21.15.189:6836/2295541152 2024-04-02T14:03:56.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.786+0000 7f77b2087640 1 --2- 172.21.15.52:0/3338709967 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55d0d0a59f50 0x55d0d0a5c370 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a7411af629a065dc server_cookie=56352ede225c8532 in_seq=0 out_seq=0 2024-04-02T14:03:56.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.786+0000 7f7795ffb640 1 -- 172.21.15.52:0/3338709967 <== mds.2 v2:172.21.15.189:6836/2295541152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d0d0941040 con 0x55d0d0a59f50 2024-04-02T14:03:56.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.786+0000 7f7795ffb640 10 client.18582 handle_command_reply: tid=0 2024-04-02T14:03:56.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.786+0000 7f77b6fab740 1 client.18582 shutdown 2024-04-02T14:03:56.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.786+0000 7f77b6fab740 1 -- 172.21.15.52:0/3338709967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09e30b0 msgr2=0x55d0d09dcd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.786+0000 7f77b6fab740 1 --2- 172.21.15.52:0/3338709967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09e30b0 0x55d0d09dcd20 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f77a400ec60 tx=0x7f77a4004a40 comp rx=0 tx=0).stop 2024-04-02T14:03:56.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.786+0000 7f77b6fab740 1 -- 172.21.15.52:0/3338709967 shutdown_connections 2024-04-02T14:03:56.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.786+0000 7f77b6fab740 1 --2- 172.21.15.52:0/3338709967 >> [v2:172.21.15.189:6836/2295541152,v1:172.21.15.189:6837/2295541152] conn(0x55d0d0a59f50 0x55d0d0a5c370 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.786+0000 7f77b6fab740 1 --2- 172.21.15.52:0/3338709967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d09e30b0 0x55d0d09dcd20 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.786+0000 7f77b6fab740 1 --2- 172.21.15.52:0/3338709967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0d09d5f60 0x55d0d09dc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.786+0000 7f7795ffb640 0 client.18582 ms_handle_reset on v2:172.21.15.189:6836/2295541152 2024-04-02T14:03:56.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.786+0000 7f77b6fab740 1 --2- 172.21.15.52:0/3338709967 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0d09d55b0 0x55d0d09eb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.786+0000 7f77b6fab740 1 -- 172.21.15.52:0/3338709967 >> 172.21.15.52:0/3338709967 conn(0x55d0d09cfed0 msgr2=0x55d0d09defe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.786+0000 7f77b6fab740 1 -- 172.21.15.52:0/3338709967 shutdown_connections 2024-04-02T14:03:56.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.786+0000 7f77b6fab740 1 -- 172.21.15.52:0/3338709967 wait complete. 2024-04-02T14:03:56.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.786+0000 7f77b6fab740 20 client.18582 trim_cache size 0 max 16384 2024-04-02T14:03:56.807 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:03:56.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.787+0000 7f77b42eb640 1 -- 172.21.15.52:0/377030185 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f77980a3d20 msgr2=0x7f77980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.787+0000 7f77b42eb640 1 --2- 172.21.15.52:0/377030185 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f77980a3d20 0x7f77980a61e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f77a400afd0 tx=0x7f77a4004ab0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.787+0000 7f77b42eb640 1 -- 172.21.15.52:0/377030185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77ac0696a0 msgr2=0x7f77ac198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.787+0000 7f77b42eb640 1 --2- 172.21.15.52:0/377030185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77ac0696a0 0x7f77ac198930 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f779c00d770 tx=0x7f779c00dc30 comp rx=0 tx=0).stop 2024-04-02T14:03:56.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.787+0000 7f77b42eb640 1 -- 172.21.15.52:0/377030185 shutdown_connections 2024-04-02T14:03:56.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.787+0000 7f77b42eb640 1 --2- 172.21.15.52:0/377030185 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f77980a3d20 0x7f77980a61e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.787+0000 7f77b42eb640 1 --2- 172.21.15.52:0/377030185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77ac107a80 0x7f77ac19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.787+0000 7f77b42eb640 1 --2- 172.21.15.52:0/377030185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77ac0696a0 0x7f77ac198930 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.787+0000 7f77b42eb640 1 --2- 172.21.15.52:0/377030185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77ac068ce0 0x7f77ac1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.787+0000 7f77b42eb640 1 -- 172.21.15.52:0/377030185 >> 172.21.15.52:0/377030185 conn(0x7f77ac077b00 msgr2=0x7f77ac0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.787+0000 7f77b42eb640 1 -- 172.21.15.52:0/377030185 shutdown_connections 2024-04-02T14:03:56.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.787+0000 7f77b42eb640 1 -- 172.21.15.52:0/377030185 wait complete. 2024-04-02T14:03:56.809 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:56 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/482048443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:03:56.810 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T14:03:56.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.912+0000 7f44a4b9d640 1 -- 172.21.15.52:0/2093271825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44a0106870 msgr2=0x7f44a0106c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.912+0000 7f44a4b9d640 1 --2- 172.21.15.52:0/2093271825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44a0106870 0x7f44a0106c50 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f4490009930 tx=0x7f449002bc30 comp rx=0 tx=0).stop 2024-04-02T14:03:56.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.912+0000 7f44a4b9d640 1 -- 172.21.15.52:0/2093271825 shutdown_connections 2024-04-02T14:03:56.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.912+0000 7f44a4b9d640 1 --2- 172.21.15.52:0/2093271825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44a0101220 0x7f44a010d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.912+0000 7f44a4b9d640 1 --2- 172.21.15.52:0/2093271825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44a0100860 0x7f44a0100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.912+0000 7f44a4b9d640 1 --2- 172.21.15.52:0/2093271825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44a0106870 0x7f44a0106c50 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.912+0000 7f44a4b9d640 1 -- 172.21.15.52:0/2093271825 >> 172.21.15.52:0/2093271825 conn(0x7f44a00fc350 msgr2=0x7f44a00fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.912+0000 7f44a4b9d640 1 -- 172.21.15.52:0/2093271825 shutdown_connections 2024-04-02T14:03:56.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.912+0000 7f44a4b9d640 1 -- 172.21.15.52:0/2093271825 wait complete. 2024-04-02T14:03:56.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.913+0000 7f44a4b9d640 1 Processor -- start 2024-04-02T14:03:56.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.913+0000 7f44a4b9d640 1 -- start start 2024-04-02T14:03:56.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.913+0000 7f44a4b9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44a0100860 0x7f44a019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.913+0000 7f44a4b9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44a0101220 0x7f44a019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.913+0000 7f44a4b9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44a0106870 0x7f44a019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.913+0000 7f44a4b9d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f44a0110c80 con 0x7f44a0106870 2024-04-02T14:03:56.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.913+0000 7f44a4b9d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f44a0110b00 con 0x7f44a0100860 2024-04-02T14:03:56.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.913+0000 7f44a4b9d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f44a0110e00 con 0x7f44a0101220 2024-04-02T14:03:56.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.913+0000 7f449ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44a0106870 0x7f44a019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.913+0000 7f449ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44a0106870 0x7f44a019ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33256/0 (socket says 172.21.15.52:33256) 2024-04-02T14:03:56.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.913+0000 7f449ffff640 1 -- 172.21.15.52:0/1034384793 learned_addr learned my addr 172.21.15.52:0/1034384793 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:56.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.913+0000 7f449f7fe640 1 --2- 172.21.15.52:0/1034384793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44a0100860 0x7f44a019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.914+0000 7f449effd640 1 --2- 172.21.15.52:0/1034384793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44a0101220 0x7f44a019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.914+0000 7f449ffff640 1 -- 172.21.15.52:0/1034384793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44a0101220 msgr2=0x7f44a019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.914+0000 7f449ffff640 1 --2- 172.21.15.52:0/1034384793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44a0101220 0x7f44a019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.914+0000 7f449ffff640 1 -- 172.21.15.52:0/1034384793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44a0100860 msgr2=0x7f44a019a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.914+0000 7f449ffff640 1 --2- 172.21.15.52:0/1034384793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44a0100860 0x7f44a019a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.914+0000 7f449ffff640 1 -- 172.21.15.52:0/1034384793 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a019f3f0 con 0x7f44a0106870 2024-04-02T14:03:56.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.914+0000 7f449ffff640 1 --2- 172.21.15.52:0/1034384793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44a0106870 0x7f44a019ecc0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f449800cce0 tx=0x7f449800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.914+0000 7f449cff9640 1 -- 172.21.15.52:0/1034384793 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449800f450 con 0x7f44a0106870 2024-04-02T14:03:56.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.914+0000 7f44a4b9d640 1 -- 172.21.15.52:0/1034384793 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a019f6e0 con 0x7f44a0106870 2024-04-02T14:03:56.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.914+0000 7f449cff9640 1 -- 172.21.15.52:0/1034384793 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f449800fad0 con 0x7f44a0106870 2024-04-02T14:03:56.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.914+0000 7f449cff9640 1 -- 172.21.15.52:0/1034384793 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498018c10 con 0x7f44a0106870 2024-04-02T14:03:56.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.914+0000 7f44a4b9d640 1 -- 172.21.15.52:0/1034384793 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a01a6f70 con 0x7f44a0106870 2024-04-02T14:03:56.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.916+0000 7f449cff9640 1 -- 172.21.15.52:0/1034384793 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f449800f5f0 con 0x7f44a0106870 2024-04-02T14:03:56.927 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:56 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/482048443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:03:56.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.917+0000 7f449cff9640 1 --2- 172.21.15.52:0/1034384793 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f44880a3d20 0x7f44880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.917+0000 7f449cff9640 1 -- 172.21.15.52:0/1034384793 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f44980d1220 con 0x7f44a0106870 2024-04-02T14:03:56.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.917+0000 7f44a89e5740 1 Processor -- start 2024-04-02T14:03:56.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.917+0000 7f449f7fe640 1 --2- 172.21.15.52:0/1034384793 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f44880a3d20 0x7f44880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.917+0000 7f44a89e5740 1 -- start start 2024-04-02T14:03:56.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.917+0000 7f44a89e5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd9309980 0x55abd9309d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.917+0000 7f44a89e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abd930a2a0 0x55abd9312940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.917+0000 7f44a89e5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55abd9312e80 0x55abd9315270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.917+0000 7f44a89e5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55abd926d7b0 con 0x55abd9309980 2024-04-02T14:03:56.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.917+0000 7f44a89e5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55abd920c570 con 0x55abd9312e80 2024-04-02T14:03:56.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.917+0000 7f44a89e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55abd9269760 con 0x55abd930a2a0 2024-04-02T14:03:56.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.917+0000 7f449f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd9309980 0x55abd9309d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.917+0000 7f449f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd9309980 0x55abd9309d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33264/0 (socket says 172.21.15.52:33264) 2024-04-02T14:03:56.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.917+0000 7f449f7fe640 1 -- 172.21.15.52:0/1882654751 learned_addr learned my addr 172.21.15.52:0/1882654751 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:56.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.917+0000 7f449ffff640 1 --2- 172.21.15.52:0/1882654751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55abd9312e80 0x55abd9315270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.918+0000 7f449effd640 1 --2- 172.21.15.52:0/1882654751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abd930a2a0 0x55abd9312940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.917+0000 7f449f7fe640 1 --2- 172.21.15.52:0/1034384793 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f44880a3d20 0x7f44880a61e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4490005ab0 tx=0x7f449002c6b0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.918+0000 7f449f7fe640 1 -- 172.21.15.52:0/1882654751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abd930a2a0 msgr2=0x55abd9312940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.918+0000 7f449f7fe640 1 --2- 172.21.15.52:0/1882654751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abd930a2a0 0x55abd9312940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.918+0000 7f449f7fe640 1 -- 172.21.15.52:0/1882654751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55abd9312e80 msgr2=0x55abd9315270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.918+0000 7f449f7fe640 1 --2- 172.21.15.52:0/1882654751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55abd9312e80 0x55abd9315270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.918+0000 7f449f7fe640 1 -- 172.21.15.52:0/1882654751 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd8f813f0 con 0x55abd9309980 2024-04-02T14:03:56.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.918+0000 7f449f7fe640 1 --2- 172.21.15.52:0/1882654751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd9309980 0x55abd9309d60 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f449002cfd0 tx=0x7f4490047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e94dd276458e54 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.918+0000 7f4485ffb640 1 -- 172.21.15.52:0/1882654751 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490050070 con 0x55abd9309980 2024-04-02T14:03:56.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.918+0000 7f4485ffb640 1 -- 172.21.15.52:0/1882654751 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4490004ba0 con 0x55abd9309980 2024-04-02T14:03:56.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.918+0000 7f4485ffb640 1 -- 172.21.15.52:0/1882654751 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44900037f0 con 0x55abd9309980 2024-04-02T14:03:56.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.919+0000 7f44a89e5740 1 -- 172.21.15.52:0/1882654751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd9309980 msgr2=0x55abd9309d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.919+0000 7f44a89e5740 1 --2- 172.21.15.52:0/1882654751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd9309980 0x55abd9309d60 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f449002cfd0 tx=0x7f4490047000 comp rx=0 tx=0).stop 2024-04-02T14:03:56.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.919+0000 7f44a89e5740 1 -- 172.21.15.52:0/1882654751 shutdown_connections 2024-04-02T14:03:56.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.919+0000 7f44a89e5740 1 --2- 172.21.15.52:0/1882654751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55abd9312e80 0x55abd9315270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.919+0000 7f44a89e5740 1 --2- 172.21.15.52:0/1882654751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abd930a2a0 0x55abd9312940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.919+0000 7f44a89e5740 1 --2- 172.21.15.52:0/1882654751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd9309980 0x55abd9309d60 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.919+0000 7f44a89e5740 1 -- 172.21.15.52:0/1882654751 >> 172.21.15.52:0/1882654751 conn(0x55abd93040b0 msgr2=0x55abd9304490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.919+0000 7f44a89e5740 1 -- 172.21.15.52:0/1882654751 shutdown_connections 2024-04-02T14:03:56.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.919+0000 7f44a89e5740 1 -- 172.21.15.52:0/1882654751 wait complete. 2024-04-02T14:03:56.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.919+0000 7f44a89e5740 1 -- start start 2024-04-02T14:03:56.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.919+0000 7f44a89e5740 1 Processor -- start 2024-04-02T14:03:56.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.919+0000 7f44a89e5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55abd9309980 0x55abd930fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.919+0000 7f44a89e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abd930a2a0 0x55abd9310230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.919+0000 7f44a89e5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd9312e80 0x55abd9310770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.919+0000 7f44a89e5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55abd9315c10 con 0x55abd9312e80 2024-04-02T14:03:56.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.920+0000 7f44a89e5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55abd9228e80 con 0x55abd9309980 2024-04-02T14:03:56.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.920+0000 7f44a89e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55abd9315d90 con 0x55abd930a2a0 2024-04-02T14:03:56.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.920+0000 7f44a89e5740 10 client.0 _ll_register_callbacks cb 0x55abd9264aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:56.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.920+0000 7f449ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd9312e80 0x55abd9310770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.920+0000 7f449ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd9312e80 0x55abd9310770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33274/0 (socket says 172.21.15.52:33274) 2024-04-02T14:03:56.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.920+0000 7f449ffff640 1 -- 172.21.15.52:0/381678983 learned_addr learned my addr 172.21.15.52:0/381678983 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:56.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.920+0000 7f449effd640 1 --2- 172.21.15.52:0/381678983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abd930a2a0 0x55abd9310230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.920+0000 7f449f7fe640 1 --2- 172.21.15.52:0/381678983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55abd9309980 0x55abd930fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.920+0000 7f449ffff640 1 -- 172.21.15.52:0/381678983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abd930a2a0 msgr2=0x55abd9310230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.920+0000 7f449ffff640 1 --2- 172.21.15.52:0/381678983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abd930a2a0 0x55abd9310230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.920+0000 7f449ffff640 1 -- 172.21.15.52:0/381678983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55abd9309980 msgr2=0x55abd930fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.920+0000 7f449ffff640 1 --2- 172.21.15.52:0/381678983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55abd9309980 0x55abd930fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.920+0000 7f449ffff640 1 -- 172.21.15.52:0/381678983 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd904b7d0 con 0x55abd9312e80 2024-04-02T14:03:56.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.920+0000 7f449f7fe640 1 --2- 172.21.15.52:0/381678983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55abd9309980 0x55abd930fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:03:56.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.920+0000 7f449ffff640 1 --2- 172.21.15.52:0/381678983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd9312e80 0x55abd9310770 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f4498018550 tx=0x7f449809c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f173ef899f1a640d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.920+0000 7f447effd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:56.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.920+0000 7f447effd640 1 -- 172.21.15.52:0/381678983 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44980bb060 con 0x55abd9312e80 2024-04-02T14:03:56.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.920+0000 7f447effd640 1 -- 172.21.15.52:0/381678983 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f44980b6210 con 0x55abd9312e80 2024-04-02T14:03:56.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.920+0000 7f44a89e5740 1 -- 172.21.15.52:0/381678983 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4490004ba0 con 0x55abd9312e80 2024-04-02T14:03:56.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.920+0000 7f447effd640 1 -- 172.21.15.52:0/381678983 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f44980b6070 con 0x55abd9312e80 2024-04-02T14:03:56.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.920+0000 7f44a89e5740 10 client.18594 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:56.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.920+0000 7f44a89e5740 1 -- 172.21.15.52:0/381678983 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55abd9321d60 con 0x55abd9312e80 2024-04-02T14:03:56.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.921+0000 7f447effd640 1 -- 172.21.15.52:0/381678983 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4490004ba0 con 0x55abd9312e80 2024-04-02T14:03:56.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.921+0000 7f447effd640 1 -- 172.21.15.52:0/381678983 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4498066030 con 0x55abd9312e80 2024-04-02T14:03:56.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.921+0000 7f44a89e5740 10 client.18594 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:56.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.921+0000 7f44a89e5740 10 client.18594 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T14:03:56.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.921+0000 7f44a89e5740 20 client.18594 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:56.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.921+0000 7f44a89e5740 1 --2- 172.21.15.52:0/381678983 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55abd9323170 0x55abd9325580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.921+0000 7f44a89e5740 4 client.18594 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:03:56.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.921+0000 7f44a89e5740 1 -- 172.21.15.52:0/381678983 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abd927acf0 con 0x55abd9323170 2024-04-02T14:03:56.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.921+0000 7f449f7fe640 1 --2- 172.21.15.52:0/381678983 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55abd9323170 0x55abd9325580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.922+0000 7f449f7fe640 1 --2- 172.21.15.52:0/381678983 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55abd9323170 0x55abd9325580 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=197c33356c1b89c2 server_cookie=1e05457e60163283 in_seq=0 out_seq=0 2024-04-02T14:03:56.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.922+0000 7f447effd640 10 client.18594 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T14:03:56.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.922+0000 7f447effd640 1 -- 172.21.15.52:0/381678983 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55abd927acf0 con 0x55abd9323170 2024-04-02T14:03:56.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.922+0000 7f447effd640 10 client.18594 handle_command_reply: tid=0 2024-04-02T14:03:56.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.922+0000 7f44a89e5740 1 client.18594 shutdown 2024-04-02T14:03:56.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.922+0000 7f44a89e5740 1 -- 172.21.15.52:0/381678983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd9312e80 msgr2=0x55abd9310770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.922+0000 7f44a89e5740 1 --2- 172.21.15.52:0/381678983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd9312e80 0x55abd9310770 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f4498018550 tx=0x7f449809c850 comp rx=0 tx=0).stop 2024-04-02T14:03:56.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.923+0000 7f44a89e5740 1 -- 172.21.15.52:0/381678983 shutdown_connections 2024-04-02T14:03:56.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.923+0000 7f44a89e5740 1 --2- 172.21.15.52:0/381678983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd9312e80 0x55abd9310770 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.923+0000 7f44a89e5740 1 --2- 172.21.15.52:0/381678983 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55abd9323170 0x55abd9325580 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.923+0000 7f44a89e5740 1 --2- 172.21.15.52:0/381678983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abd930a2a0 0x55abd9310230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.923+0000 7f44a89e5740 1 --2- 172.21.15.52:0/381678983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55abd9309980 0x55abd930fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.923+0000 7f447effd640 0 client.18594 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T14:03:56.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.923+0000 7f44a89e5740 1 -- 172.21.15.52:0/381678983 >> 172.21.15.52:0/381678983 conn(0x55abd93040b0 msgr2=0x55abd9305e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.923+0000 7f44a89e5740 1 -- 172.21.15.52:0/381678983 shutdown_connections 2024-04-02T14:03:56.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.923+0000 7f44a89e5740 1 -- 172.21.15.52:0/381678983 wait complete. 2024-04-02T14:03:56.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.923+0000 7f44a89e5740 20 client.18594 trim_cache size 0 max 16384 2024-04-02T14:03:56.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.944+0000 7f44a89e5740 1 Processor -- start 2024-04-02T14:03:56.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.944+0000 7f44a89e5740 1 -- start start 2024-04-02T14:03:56.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.945+0000 7f44a89e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abd9309980 0x55abd9312420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.945+0000 7f44a89e5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd930a2a0 0x55abd931ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.945+0000 7f44a89e5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55abd9312e80 0x55abd930ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.945+0000 7f44a89e5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01b0e80 con 0x55abd930a2a0 2024-04-02T14:03:56.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.945+0000 7f44a89e5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01b29c0 con 0x55abd9312e80 2024-04-02T14:03:56.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.945+0000 7f44a89e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01b1550 con 0x55abd9309980 2024-04-02T14:03:56.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.945+0000 7f449effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd930a2a0 0x55abd931ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.945+0000 7f449effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd930a2a0 0x55abd931ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33286/0 (socket says 172.21.15.52:33286) 2024-04-02T14:03:56.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.945+0000 7f449effd640 1 -- 172.21.15.52:0/218868157 learned_addr learned my addr 172.21.15.52:0/218868157 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:56.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.945+0000 7f449f7fe640 1 --2- 172.21.15.52:0/218868157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abd9309980 0x55abd9312420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.945+0000 7f449ffff640 1 --2- 172.21.15.52:0/218868157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55abd9312e80 0x55abd930ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.945+0000 7f449effd640 1 -- 172.21.15.52:0/218868157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abd9309980 msgr2=0x55abd9312420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.945+0000 7f449effd640 1 --2- 172.21.15.52:0/218868157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abd9309980 0x55abd9312420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.945+0000 7f449effd640 1 -- 172.21.15.52:0/218868157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55abd9312e80 msgr2=0x55abd930ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.945+0000 7f449effd640 1 --2- 172.21.15.52:0/218868157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55abd9312e80 0x55abd930ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.945+0000 7f449effd640 1 -- 172.21.15.52:0/218868157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd9303910 con 0x55abd930a2a0 2024-04-02T14:03:56.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.945+0000 7f449f7fe640 1 --2- 172.21.15.52:0/218868157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abd9309980 0x55abd9312420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:03:56.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.946+0000 7f449effd640 1 --2- 172.21.15.52:0/218868157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd930a2a0 0x55abd931ead0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f449400bc00 tx=0x7f4494012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bffb838b33232ea server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.946+0000 7f44857fa640 1 -- 172.21.15.52:0/218868157 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449400bcf0 con 0x55abd930a2a0 2024-04-02T14:03:56.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.946+0000 7f44857fa640 1 -- 172.21.15.52:0/218868157 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4494011920 con 0x55abd930a2a0 2024-04-02T14:03:56.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.946+0000 7f44857fa640 1 -- 172.21.15.52:0/218868157 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449400f890 con 0x55abd930a2a0 2024-04-02T14:03:56.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.946+0000 7f44a89e5740 1 -- 172.21.15.52:0/218868157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd930a2a0 msgr2=0x55abd931ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.946+0000 7f44a89e5740 1 --2- 172.21.15.52:0/218868157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd930a2a0 0x55abd931ead0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f449400bc00 tx=0x7f4494012820 comp rx=0 tx=0).stop 2024-04-02T14:03:56.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.946+0000 7f44a89e5740 1 -- 172.21.15.52:0/218868157 shutdown_connections 2024-04-02T14:03:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.946+0000 7f44a89e5740 1 --2- 172.21.15.52:0/218868157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55abd9312e80 0x55abd930ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.946+0000 7f44a89e5740 1 --2- 172.21.15.52:0/218868157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd930a2a0 0x55abd931ead0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.946+0000 7f44a89e5740 1 --2- 172.21.15.52:0/218868157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abd9309980 0x55abd9312420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.946+0000 7f44a89e5740 1 -- 172.21.15.52:0/218868157 >> 172.21.15.52:0/218868157 conn(0x55abd93040b0 msgr2=0x55abd930e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.947+0000 7f44a89e5740 1 -- 172.21.15.52:0/218868157 shutdown_connections 2024-04-02T14:03:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.947+0000 7f44a89e5740 1 -- 172.21.15.52:0/218868157 wait complete. 2024-04-02T14:03:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.947+0000 7f44a89e5740 1 -- start start 2024-04-02T14:03:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.947+0000 7f44a89e5740 1 Processor -- start 2024-04-02T14:03:56.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.948+0000 7f44a89e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abd9309980 0x55abd930c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.948+0000 7f44a89e5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55abd930a2a0 0x55abd930ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.948+0000 7f44a89e5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd9312e80 0x55abd9319fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.948+0000 7f44a89e5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01b22f0 con 0x55abd9312e80 2024-04-02T14:03:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.948+0000 7f44a89e5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01b1c20 con 0x55abd930a2a0 2024-04-02T14:03:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.948+0000 7f44a89e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01aee80 con 0x55abd9309980 2024-04-02T14:03:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.948+0000 7f44a89e5740 10 client.0 _ll_register_callbacks cb 0x55abd9312dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.948+0000 7f449ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd9312e80 0x55abd9319fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.948+0000 7f449ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd9312e80 0x55abd9319fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33298/0 (socket says 172.21.15.52:33298) 2024-04-02T14:03:56.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.948+0000 7f449ffff640 1 -- 172.21.15.52:0/1940555576 learned_addr learned my addr 172.21.15.52:0/1940555576 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:56.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.948+0000 7f449f7fe640 1 --2- 172.21.15.52:0/1940555576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abd9309980 0x55abd930c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.948+0000 7f449effd640 1 --2- 172.21.15.52:0/1940555576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55abd930a2a0 0x55abd930ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.948+0000 7f449ffff640 1 -- 172.21.15.52:0/1940555576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abd9309980 msgr2=0x55abd930c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.948+0000 7f449ffff640 1 --2- 172.21.15.52:0/1940555576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abd9309980 0x55abd930c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.948+0000 7f449ffff640 1 -- 172.21.15.52:0/1940555576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55abd930a2a0 msgr2=0x55abd930ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:56.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.948+0000 7f449ffff640 1 --2- 172.21.15.52:0/1940555576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55abd930a2a0 0x55abd930ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.948+0000 7f449ffff640 1 -- 172.21.15.52:0/1940555576 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd931a520 con 0x55abd9312e80 2024-04-02T14:03:56.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.948+0000 7f449ffff640 1 --2- 172.21.15.52:0/1940555576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd9312e80 0x55abd9319fe0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f449800ab20 tx=0x7f4498098a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d14939d4f7ea049 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:56.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.948+0000 7f447effd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:56.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.948+0000 7f447effd640 1 -- 172.21.15.52:0/1940555576 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44980990d0 con 0x55abd9312e80 2024-04-02T14:03:56.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.948+0000 7f44a89e5740 1 -- 172.21.15.52:0/1940555576 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4494011920 con 0x55abd9312e80 2024-04-02T14:03:56.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.948+0000 7f447effd640 1 -- 172.21.15.52:0/1940555576 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f4498098d80 con 0x55abd9312e80 2024-04-02T14:03:56.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.949+0000 7f447effd640 1 -- 172.21.15.52:0/1940555576 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f449809c970 con 0x55abd9312e80 2024-04-02T14:03:56.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.949+0000 7f44a89e5740 10 client.18600 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:56.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.949+0000 7f44a89e5740 1 -- 172.21.15.52:0/1940555576 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55abd931b7c0 con 0x55abd9312e80 2024-04-02T14:03:56.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.949+0000 7f447effd640 1 -- 172.21.15.52:0/1940555576 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4494011920 con 0x55abd9312e80 2024-04-02T14:03:56.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.949+0000 7f447effd640 1 -- 172.21.15.52:0/1940555576 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4498066040 con 0x55abd9312e80 2024-04-02T14:03:56.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.949+0000 7f44a89e5740 10 client.18600 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:56.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.949+0000 7f44a89e5740 10 client.18600 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T14:03:56.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.949+0000 7f44a89e5740 20 client.18600 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:56.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.949+0000 7f44a89e5740 1 --2- 172.21.15.52:0/1940555576 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55abd93898c0 0x55abd938bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:56.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.949+0000 7f44a89e5740 4 client.18600 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:03:56.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.949+0000 7f44a89e5740 1 -- 172.21.15.52:0/1940555576 --> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55abd9275040 con 0x55abd93898c0 2024-04-02T14:03:56.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.949+0000 7f449f7fe640 1 --2- 172.21.15.52:0/1940555576 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55abd93898c0 0x55abd938bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:56.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.950+0000 7f449f7fe640 1 --2- 172.21.15.52:0/1940555576 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55abd93898c0 0x55abd938bcb0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ca7a2eb60d3c02ca server_cookie=21a144d8bed955e0 in_seq=0 out_seq=0 2024-04-02T14:03:56.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.950+0000 7f447effd640 10 client.18600 ms_handle_connect on v2:172.21.15.189:6838/729001189 2024-04-02T14:03:56.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.950+0000 7f447effd640 1 -- 172.21.15.52:0/1940555576 <== mds.1 v2:172.21.15.189:6838/729001189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55abd9275040 con 0x55abd93898c0 2024-04-02T14:03:56.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.950+0000 7f447effd640 10 client.18600 handle_command_reply: tid=0 2024-04-02T14:03:56.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.950+0000 7f44a89e5740 1 client.18600 shutdown 2024-04-02T14:03:56.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.950+0000 7f44a89e5740 1 -- 172.21.15.52:0/1940555576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd9312e80 msgr2=0x55abd9319fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.950+0000 7f44a89e5740 1 --2- 172.21.15.52:0/1940555576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd9312e80 0x55abd9319fe0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f449800ab20 tx=0x7f4498098a40 comp rx=0 tx=0).stop 2024-04-02T14:03:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.950+0000 7f44a89e5740 1 -- 172.21.15.52:0/1940555576 shutdown_connections 2024-04-02T14:03:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.950+0000 7f44a89e5740 1 --2- 172.21.15.52:0/1940555576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abd9312e80 0x55abd9319fe0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.950+0000 7f44a89e5740 1 --2- 172.21.15.52:0/1940555576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55abd930a2a0 0x55abd930ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.950+0000 7f44a89e5740 1 --2- 172.21.15.52:0/1940555576 >> [v2:172.21.15.189:6838/729001189,v1:172.21.15.189:6839/729001189] conn(0x55abd93898c0 0x55abd938bcb0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.950+0000 7f44a89e5740 1 --2- 172.21.15.52:0/1940555576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55abd9309980 0x55abd930c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.950+0000 7f447effd640 0 client.18600 ms_handle_reset on v2:172.21.15.189:6838/729001189 2024-04-02T14:03:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.950+0000 7f44a89e5740 1 -- 172.21.15.52:0/1940555576 >> 172.21.15.52:0/1940555576 conn(0x55abd93040b0 msgr2=0x55abd930ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.950+0000 7f44a89e5740 1 -- 172.21.15.52:0/1940555576 shutdown_connections 2024-04-02T14:03:56.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.950+0000 7f44a89e5740 1 -- 172.21.15.52:0/1940555576 wait complete. 2024-04-02T14:03:56.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.950+0000 7f44a89e5740 20 client.18600 trim_cache size 0 max 16384 2024-04-02T14:03:56.974 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:03:56.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.951+0000 7f44a4b9d640 1 -- 172.21.15.52:0/1034384793 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f44880a3d20 msgr2=0x7f44880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.951+0000 7f44a4b9d640 1 --2- 172.21.15.52:0/1034384793 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f44880a3d20 0x7f44880a61e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4490005ab0 tx=0x7f449002c6b0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.951+0000 7f44a4b9d640 1 -- 172.21.15.52:0/1034384793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44a0106870 msgr2=0x7f44a019ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:56.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.951+0000 7f44a4b9d640 1 --2- 172.21.15.52:0/1034384793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44a0106870 0x7f44a019ecc0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f449800cce0 tx=0x7f449800a620 comp rx=0 tx=0).stop 2024-04-02T14:03:56.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.951+0000 7f44a4b9d640 1 -- 172.21.15.52:0/1034384793 shutdown_connections 2024-04-02T14:03:56.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.951+0000 7f44a4b9d640 1 --2- 172.21.15.52:0/1034384793 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f44880a3d20 0x7f44880a61e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.951+0000 7f44a4b9d640 1 --2- 172.21.15.52:0/1034384793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44a0106870 0x7f44a019ecc0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.951+0000 7f44a4b9d640 1 --2- 172.21.15.52:0/1034384793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44a0101220 0x7f44a019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.951+0000 7f44a4b9d640 1 --2- 172.21.15.52:0/1034384793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44a0100860 0x7f44a019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:56.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.951+0000 7f44a4b9d640 1 -- 172.21.15.52:0/1034384793 >> 172.21.15.52:0/1034384793 conn(0x7f44a00fc350 msgr2=0x7f44a0068c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:56.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.951+0000 7f44a4b9d640 1 -- 172.21.15.52:0/1034384793 shutdown_connections 2024-04-02T14:03:56.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:56.951+0000 7f44a4b9d640 1 -- 172.21.15.52:0/1034384793 wait complete. 2024-04-02T14:03:56.977 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T14:03:57.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.079+0000 7fd3511e9640 1 -- 172.21.15.52:0/1382963487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd34c100d80 msgr2=0x7fd34c10e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:57.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.079+0000 7fd3511e9640 1 --2- 172.21.15.52:0/1382963487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd34c100d80 0x7fd34c10e980 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fd344009cd0 tx=0x7fd34402c180 comp rx=0 tx=0).stop 2024-04-02T14:03:57.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.079+0000 7fd3511e9640 1 -- 172.21.15.52:0/1382963487 shutdown_connections 2024-04-02T14:03:57.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.079+0000 7fd3511e9640 1 --2- 172.21.15.52:0/1382963487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd34c100d80 0x7fd34c10e980 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.079+0000 7fd3511e9640 1 --2- 172.21.15.52:0/1382963487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd34c1003c0 0x7fd34c100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.079+0000 7fd3511e9640 1 --2- 172.21.15.52:0/1382963487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd34c107d00 0x7fd34c1080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.079+0000 7fd3511e9640 1 -- 172.21.15.52:0/1382963487 >> 172.21.15.52:0/1382963487 conn(0x7fd34c075d40 msgr2=0x7fd34c076150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:57.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.079+0000 7fd3511e9640 1 -- 172.21.15.52:0/1382963487 shutdown_connections 2024-04-02T14:03:57.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.079+0000 7fd3511e9640 1 -- 172.21.15.52:0/1382963487 wait complete. 2024-04-02T14:03:57.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.080+0000 7fd3511e9640 1 Processor -- start 2024-04-02T14:03:57.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.080+0000 7fd3511e9640 1 -- start start 2024-04-02T14:03:57.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.080+0000 7fd3511e9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd34c1003c0 0x7fd34c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:57.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.080+0000 7fd3511e9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd34c100d80 0x7fd34c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:57.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.080+0000 7fd3511e9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd34c107d00 0x7fd34c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:57.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.080+0000 7fd3511e9640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c111f60 con 0x7fd34c107d00 2024-04-02T14:03:57.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.080+0000 7fd3511e9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c111de0 con 0x7fd34c1003c0 2024-04-02T14:03:57.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.080+0000 7fd3511e9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1120e0 con 0x7fd34c100d80 2024-04-02T14:03:57.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.080+0000 7fd34b59e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd34c107d00 0x7fd34c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:57.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.080+0000 7fd34b59e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd34c107d00 0x7fd34c19cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33302/0 (socket says 172.21.15.52:33302) 2024-04-02T14:03:57.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.080+0000 7fd34b59e640 1 -- 172.21.15.52:0/178709354 learned_addr learned my addr 172.21.15.52:0/178709354 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:57.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.081+0000 7fd34ad9d640 1 --2- 172.21.15.52:0/178709354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd34c1003c0 0x7fd34c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:57.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.081+0000 7fd34a59c640 1 --2- 172.21.15.52:0/178709354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd34c100d80 0x7fd34c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:57.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.081+0000 7fd34b59e640 1 -- 172.21.15.52:0/178709354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd34c100d80 msgr2=0x7fd34c1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:57.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.081+0000 7fd34b59e640 1 --2- 172.21.15.52:0/178709354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd34c100d80 0x7fd34c1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.081+0000 7fd34b59e640 1 -- 172.21.15.52:0/178709354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd34c1003c0 msgr2=0x7fd34c198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:57.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.081+0000 7fd34b59e640 1 --2- 172.21.15.52:0/178709354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd34c1003c0 0x7fd34c198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.081+0000 7fd34b59e640 1 -- 172.21.15.52:0/178709354 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34c19d370 con 0x7fd34c107d00 2024-04-02T14:03:57.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.081+0000 7fd34b59e640 1 --2- 172.21.15.52:0/178709354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd34c107d00 0x7fd34c19cc40 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fd344004af0 tx=0x7fd344034720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:57.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.081+0000 7fd333fff640 1 -- 172.21.15.52:0/178709354 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344039050 con 0x7fd34c107d00 2024-04-02T14:03:57.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.081+0000 7fd333fff640 1 -- 172.21.15.52:0/178709354 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd344045070 con 0x7fd34c107d00 2024-04-02T14:03:57.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.081+0000 7fd3511e9640 1 -- 172.21.15.52:0/178709354 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34c19d600 con 0x7fd34c107d00 2024-04-02T14:03:57.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.081+0000 7fd333fff640 1 -- 172.21.15.52:0/178709354 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3440045a0 con 0x7fd34c107d00 2024-04-02T14:03:57.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.081+0000 7fd3511e9640 1 -- 172.21.15.52:0/178709354 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd34c1a4ef0 con 0x7fd34c107d00 2024-04-02T14:03:57.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.083+0000 7fd333fff640 1 -- 172.21.15.52:0/178709354 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd344032ca0 con 0x7fd34c107d00 2024-04-02T14:03:57.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.083+0000 7fd353dd4740 1 Processor -- start 2024-04-02T14:03:57.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.083+0000 7fd353dd4740 1 -- start start 2024-04-02T14:03:57.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.084+0000 7fd353dd4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf588aa7a0 0x55bf588aab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:57.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.084+0000 7fd353dd4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bf588ab0c0 0x55bf588b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:57.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.084+0000 7fd353dd4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 0x55bf588b6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:57.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.084+0000 7fd353dd4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55bf5880e7b0 con 0x55bf588b3ca0 2024-04-02T14:03:57.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.084+0000 7fd353dd4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55bf587ad570 con 0x55bf588ab0c0 2024-04-02T14:03:57.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.084+0000 7fd353dd4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bf5880a760 con 0x55bf588aa7a0 2024-04-02T14:03:57.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.084+0000 7fd34a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 0x55bf588b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:57.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.084+0000 7fd34a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 0x55bf588b6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33316/0 (socket says 172.21.15.52:33316) 2024-04-02T14:03:57.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.084+0000 7fd34a59c640 1 -- 172.21.15.52:0/923049646 learned_addr learned my addr 172.21.15.52:0/923049646 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:57.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.084+0000 7fd333fff640 1 --2- 172.21.15.52:0/178709354 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd3340a3d20 0x7fd3340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:57.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.084+0000 7fd34a59c640 1 --2- 172.21.15.52:0/923049646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf588aa7a0 0x55bf588aab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:57.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.084+0000 7fd34ad9d640 1 --2- 172.21.15.52:0/923049646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bf588ab0c0 0x55bf588b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:57.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.084+0000 7fd34b59e640 1 --2- 172.21.15.52:0/178709354 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd3340a3d20 0x7fd3340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:57.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.084+0000 7fd34a59c640 1 -- 172.21.15.52:0/923049646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf588aa7a0 msgr2=0x55bf588aab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:57.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.084+0000 7fd34a59c640 1 --2- 172.21.15.52:0/923049646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf588aa7a0 0x55bf588aab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.084+0000 7fd34a59c640 1 -- 172.21.15.52:0/923049646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bf588ab0c0 msgr2=0x55bf588b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:57.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.084+0000 7fd34a59c640 1 --2- 172.21.15.52:0/923049646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bf588ab0c0 0x55bf588b3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.084+0000 7fd34a59c640 1 -- 172.21.15.52:0/923049646 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf585223f0 con 0x55bf588b3ca0 2024-04-02T14:03:57.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.084+0000 7fd333fff640 1 -- 172.21.15.52:0/178709354 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fd3440f3460 con 0x7fd34c107d00 2024-04-02T14:03:57.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.084+0000 7fd34a59c640 1 --2- 172.21.15.52:0/923049646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 0x55bf588b6090 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fd3400149c0 tx=0x7fd34000c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359ce589c8d0b22f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:57.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.085+0000 7fd3317fa640 1 -- 172.21.15.52:0/923049646 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3400142d0 con 0x55bf588b3ca0 2024-04-02T14:03:57.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.085+0000 7fd34b59e640 1 --2- 172.21.15.52:0/178709354 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd3340a3d20 0x7fd3340a61e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd3440f0a80 tx=0x7fd344090cf0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:57.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.085+0000 7fd3317fa640 1 -- 172.21.15.52:0/923049646 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd34000fc40 con 0x55bf588b3ca0 2024-04-02T14:03:57.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.085+0000 7fd3317fa640 1 -- 172.21.15.52:0/923049646 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd340012c00 con 0x55bf588b3ca0 2024-04-02T14:03:57.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.085+0000 7fd353dd4740 1 -- 172.21.15.52:0/923049646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 msgr2=0x55bf588b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:57.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.085+0000 7fd353dd4740 1 --2- 172.21.15.52:0/923049646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 0x55bf588b6090 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fd3400149c0 tx=0x7fd34000c4d0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.085+0000 7fd353dd4740 1 -- 172.21.15.52:0/923049646 shutdown_connections 2024-04-02T14:03:57.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.085+0000 7fd353dd4740 1 --2- 172.21.15.52:0/923049646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 0x55bf588b6090 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.085+0000 7fd353dd4740 1 --2- 172.21.15.52:0/923049646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bf588ab0c0 0x55bf588b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.085+0000 7fd353dd4740 1 --2- 172.21.15.52:0/923049646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf588aa7a0 0x55bf588aab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.085+0000 7fd353dd4740 1 -- 172.21.15.52:0/923049646 >> 172.21.15.52:0/923049646 conn(0x55bf588a4ed0 msgr2=0x55bf588a52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:57.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.085+0000 7fd353dd4740 1 -- 172.21.15.52:0/923049646 shutdown_connections 2024-04-02T14:03:57.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.085+0000 7fd353dd4740 1 -- 172.21.15.52:0/923049646 wait complete. 2024-04-02T14:03:57.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.086+0000 7fd353dd4740 1 -- start start 2024-04-02T14:03:57.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.086+0000 7fd353dd4740 1 Processor -- start 2024-04-02T14:03:57.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.086+0000 7fd353dd4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bf588aa7a0 0x55bf588b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:57.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.086+0000 7fd353dd4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf588ab0c0 0x55bf588b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:57.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.086+0000 7fd353dd4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 0x55bf588b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:57.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.086+0000 7fd353dd4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55bf588b6a90 con 0x55bf588b3ca0 2024-04-02T14:03:57.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.086+0000 7fd353dd4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55bf587c9e80 con 0x55bf588aa7a0 2024-04-02T14:03:57.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.086+0000 7fd353dd4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bf588b6c10 con 0x55bf588ab0c0 2024-04-02T14:03:57.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.086+0000 7fd353dd4740 10 client.0 _ll_register_callbacks cb 0x55bf58805aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:57.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.086+0000 7fd34a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 0x55bf588b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:57.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.086+0000 7fd34a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 0x55bf588b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33332/0 (socket says 172.21.15.52:33332) 2024-04-02T14:03:57.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.086+0000 7fd34a59c640 1 -- 172.21.15.52:0/2078177198 learned_addr learned my addr 172.21.15.52:0/2078177198 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:57.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.086+0000 7fd34a59c640 1 --2- 172.21.15.52:0/2078177198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bf588aa7a0 0x55bf588b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:57.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.086+0000 7fd34ad9d640 1 --2- 172.21.15.52:0/2078177198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf588ab0c0 0x55bf588b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:57.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.086+0000 7fd34a59c640 1 -- 172.21.15.52:0/2078177198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf588ab0c0 msgr2=0x55bf588b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:57.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.086+0000 7fd34a59c640 1 --2- 172.21.15.52:0/2078177198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf588ab0c0 0x55bf588b0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.086+0000 7fd34a59c640 1 -- 172.21.15.52:0/2078177198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bf588aa7a0 msgr2=0x55bf588b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.086+0000 7fd34a59c640 1 --2- 172.21.15.52:0/2078177198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bf588aa7a0 0x55bf588b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.086+0000 7fd34a59c640 1 -- 172.21.15.52:0/2078177198 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf585ec7d0 con 0x55bf588b3ca0 2024-04-02T14:03:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.087+0000 7fd34a59c640 1 --2- 172.21.15.52:0/2078177198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 0x55bf588b11c0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fd340014620 tx=0x7fd340012840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f90de3da19c1515 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.087+0000 7fd32eff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.087+0000 7fd32eff5640 1 -- 172.21.15.52:0/2078177198 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3400140d0 con 0x55bf588b3ca0 2024-04-02T14:03:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.087+0000 7fd32eff5640 1 -- 172.21.15.52:0/2078177198 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fd340014270 con 0x55bf588b3ca0 2024-04-02T14:03:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.087+0000 7fd353dd4740 1 -- 172.21.15.52:0/2078177198 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd34000fc40 con 0x55bf588b3ca0 2024-04-02T14:03:57.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.087+0000 7fd32eff5640 1 -- 172.21.15.52:0/2078177198 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd34000f730 con 0x55bf588b3ca0 2024-04-02T14:03:57.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.087+0000 7fd353dd4740 10 client.18612 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:57.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.087+0000 7fd353dd4740 1 -- 172.21.15.52:0/2078177198 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bf588c2b50 con 0x55bf588b3ca0 2024-04-02T14:03:57.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.087+0000 7fd32eff5640 1 -- 172.21.15.52:0/2078177198 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd34000fc40 con 0x55bf588b3ca0 2024-04-02T14:03:57.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.087+0000 7fd32eff5640 1 -- 172.21.15.52:0/2078177198 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd340004300 con 0x55bf588b3ca0 2024-04-02T14:03:57.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.087+0000 7fd353dd4740 10 client.18612 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:57.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.087+0000 7fd353dd4740 10 client.18612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:03:57.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.087+0000 7fd353dd4740 20 client.18612 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:57.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.087+0000 7fd353dd4740 1 --2- 172.21.15.52:0/2078177198 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55bf588c3fb0 0x55bf588c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:57.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.087+0000 7fd353dd4740 4 client.18612 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T14:03:57.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.087+0000 7fd353dd4740 1 -- 172.21.15.52:0/2078177198 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf5881bcf0 con 0x55bf588c3fb0 2024-04-02T14:03:57.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.088+0000 7fd34ad9d640 1 --2- 172.21.15.52:0/2078177198 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55bf588c3fb0 0x55bf588c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:57.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.088+0000 7fd32eff5640 10 client.18612 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:03:57.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.088+0000 7fd34ad9d640 1 --2- 172.21.15.52:0/2078177198 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55bf588c3fb0 0x55bf588c63a0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58ec17288419cd04 server_cookie=fed4e4b09a9485b2 in_seq=0 out_seq=0 2024-04-02T14:03:57.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.089+0000 7fd32eff5640 1 -- 172.21.15.52:0/2078177198 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf5881bcf0 con 0x55bf588c3fb0 2024-04-02T14:03:57.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.089+0000 7fd32eff5640 10 client.18612 handle_command_reply: tid=0 2024-04-02T14:03:57.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.089+0000 7fd353dd4740 1 client.18612 shutdown 2024-04-02T14:03:57.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.089+0000 7fd353dd4740 1 -- 172.21.15.52:0/2078177198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 msgr2=0x55bf588b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:57.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.089+0000 7fd353dd4740 1 --2- 172.21.15.52:0/2078177198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 0x55bf588b11c0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fd340014620 tx=0x7fd340012840 comp rx=0 tx=0).stop 2024-04-02T14:03:57.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.089+0000 7fd353dd4740 1 -- 172.21.15.52:0/2078177198 shutdown_connections 2024-04-02T14:03:57.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.089+0000 7fd353dd4740 1 --2- 172.21.15.52:0/2078177198 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55bf588c3fb0 0x55bf588c63a0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.089+0000 7fd353dd4740 1 --2- 172.21.15.52:0/2078177198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 0x55bf588b11c0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.089+0000 7fd353dd4740 1 --2- 172.21.15.52:0/2078177198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf588ab0c0 0x55bf588b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.089+0000 7fd32eff5640 0 client.18612 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:03:57.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.089+0000 7fd353dd4740 1 --2- 172.21.15.52:0/2078177198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bf588aa7a0 0x55bf588b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.089+0000 7fd353dd4740 1 -- 172.21.15.52:0/2078177198 >> 172.21.15.52:0/2078177198 conn(0x55bf588a4ed0 msgr2=0x55bf588a6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:57.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.089+0000 7fd353dd4740 1 -- 172.21.15.52:0/2078177198 shutdown_connections 2024-04-02T14:03:57.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.089+0000 7fd353dd4740 1 -- 172.21.15.52:0/2078177198 wait complete. 2024-04-02T14:03:57.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.089+0000 7fd353dd4740 20 client.18612 trim_cache size 0 max 16384 2024-04-02T14:03:57.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.102+0000 7fd353dd4740 1 Processor -- start 2024-04-02T14:03:57.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.102+0000 7fd353dd4740 1 -- start start 2024-04-02T14:03:57.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.103+0000 7fd353dd4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bf588aa7a0 0x55bf588bf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:57.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.103+0000 7fd353dd4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf588ab0c0 0x55bf588c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:57.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.103+0000 7fd353dd4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 0x55bf588cba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:57.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.103+0000 7fd353dd4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1aeed0 con 0x55bf588b3ca0 2024-04-02T14:03:57.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.103+0000 7fd353dd4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1b0a10 con 0x55bf588aa7a0 2024-04-02T14:03:57.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.103+0000 7fd353dd4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1af5a0 con 0x55bf588ab0c0 2024-04-02T14:03:57.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.103+0000 7fd34a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 0x55bf588cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:57.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.103+0000 7fd34a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 0x55bf588cba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33342/0 (socket says 172.21.15.52:33342) 2024-04-02T14:03:57.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.103+0000 7fd34a59c640 1 -- 172.21.15.52:0/1313222577 learned_addr learned my addr 172.21.15.52:0/1313222577 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:57.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.103+0000 7fd34ad9d640 1 --2- 172.21.15.52:0/1313222577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf588ab0c0 0x55bf588c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:57.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.103+0000 7fd34a59c640 1 -- 172.21.15.52:0/1313222577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf588ab0c0 msgr2=0x55bf588c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:57.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.103+0000 7fd34a59c640 1 --2- 172.21.15.52:0/1313222577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf588ab0c0 0x55bf588c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.103+0000 7fd34a59c640 1 -- 172.21.15.52:0/1313222577 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bf588aa7a0 msgr2=0x55bf588bf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:57.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.103+0000 7fd34a59c640 1 --2- 172.21.15.52:0/1313222577 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bf588aa7a0 0x55bf588bf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.103+0000 7fd34a59c640 1 -- 172.21.15.52:0/1313222577 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf588aaef0 con 0x55bf588b3ca0 2024-04-02T14:03:57.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.103+0000 7fd34a59c640 1 --2- 172.21.15.52:0/1313222577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 0x55bf588cba90 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fd340014650 tx=0x7fd340005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77665d906579bffd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:57.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.103+0000 7fd330ff9640 1 -- 172.21.15.52:0/1313222577 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd340012450 con 0x55bf588b3ca0 2024-04-02T14:03:57.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.103+0000 7fd330ff9640 1 -- 172.21.15.52:0/1313222577 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd34000c810 con 0x55bf588b3ca0 2024-04-02T14:03:57.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.103+0000 7fd330ff9640 1 -- 172.21.15.52:0/1313222577 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd340014370 con 0x55bf588b3ca0 2024-04-02T14:03:57.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.104+0000 7fd353dd4740 1 -- 172.21.15.52:0/1313222577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 msgr2=0x55bf588cba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:57.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.104+0000 7fd353dd4740 1 --2- 172.21.15.52:0/1313222577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 0x55bf588cba90 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fd340014650 tx=0x7fd340005970 comp rx=0 tx=0).stop 2024-04-02T14:03:57.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.104+0000 7fd353dd4740 1 -- 172.21.15.52:0/1313222577 shutdown_connections 2024-04-02T14:03:57.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.104+0000 7fd353dd4740 1 --2- 172.21.15.52:0/1313222577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 0x55bf588cba90 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.104+0000 7fd353dd4740 1 --2- 172.21.15.52:0/1313222577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf588ab0c0 0x55bf588c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.104+0000 7fd353dd4740 1 --2- 172.21.15.52:0/1313222577 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bf588aa7a0 0x55bf588bf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.104+0000 7fd353dd4740 1 -- 172.21.15.52:0/1313222577 >> 172.21.15.52:0/1313222577 conn(0x55bf588a4ed0 msgr2=0x55bf588af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:57.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.104+0000 7fd353dd4740 1 -- 172.21.15.52:0/1313222577 shutdown_connections 2024-04-02T14:03:57.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.104+0000 7fd353dd4740 1 -- 172.21.15.52:0/1313222577 wait complete. 2024-04-02T14:03:57.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.104+0000 7fd353dd4740 1 -- start start 2024-04-02T14:03:57.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.104+0000 7fd353dd4740 1 Processor -- start 2024-04-02T14:03:57.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.104+0000 7fd353dd4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf588aa7a0 0x55bf588bbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:57.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.104+0000 7fd353dd4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bf588ab0c0 0x55bf588bc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:57.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.104+0000 7fd353dd4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 0x55bf588ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:57.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.104+0000 7fd353dd4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1b0340 con 0x55bf588b3ca0 2024-04-02T14:03:57.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.105+0000 7fd353dd4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1afc70 con 0x55bf588ab0c0 2024-04-02T14:03:57.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.105+0000 7fd353dd4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1aced0 con 0x55bf588aa7a0 2024-04-02T14:03:57.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.105+0000 7fd353dd4740 10 client.0 _ll_register_callbacks cb 0x7fd33c00c2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T14:03:57.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.105+0000 7fd34a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 0x55bf588ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:57.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.105+0000 7fd34a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 0x55bf588ad150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33344/0 (socket says 172.21.15.52:33344) 2024-04-02T14:03:57.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.105+0000 7fd34a59c640 1 -- 172.21.15.52:0/1304443703 learned_addr learned my addr 172.21.15.52:0/1304443703 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:57.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.105+0000 7fd34a59c640 1 --2- 172.21.15.52:0/1304443703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf588aa7a0 0x55bf588bbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:57.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.105+0000 7fd34a59c640 1 -- 172.21.15.52:0/1304443703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf588aa7a0 msgr2=0x55bf588bbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:57.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.105+0000 7fd34a59c640 1 --2- 172.21.15.52:0/1304443703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf588aa7a0 0x55bf588bbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.105+0000 7fd34a59c640 1 -- 172.21.15.52:0/1304443703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bf588ab0c0 msgr2=0x55bf588bc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T14:03:57.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.105+0000 7fd34a59c640 1 --2- 172.21.15.52:0/1304443703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bf588ab0c0 0x55bf588bc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.105+0000 7fd34a59c640 1 -- 172.21.15.52:0/1304443703 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf588ad950 con 0x55bf588b3ca0 2024-04-02T14:03:57.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.105+0000 7fd34a59c640 1 --2- 172.21.15.52:0/1304443703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 0x55bf588ad150 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fd340014570 tx=0x7fd34002d4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2e19899067d029 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:57.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.105+0000 7fd32eff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-04-02T14:03:57.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.105+0000 7fd32eff5640 1 -- 172.21.15.52:0/1304443703 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34002d550 con 0x55bf588b3ca0 2024-04-02T14:03:57.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.105+0000 7fd32eff5640 1 -- 172.21.15.52:0/1304443703 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fd34002a030 con 0x55bf588b3ca0 2024-04-02T14:03:57.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.105+0000 7fd353dd4740 1 -- 172.21.15.52:0/1304443703 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd34000c810 con 0x55bf588b3ca0 2024-04-02T14:03:57.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.105+0000 7fd32eff5640 1 -- 172.21.15.52:0/1304443703 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd340014320 con 0x55bf588b3ca0 2024-04-02T14:03:57.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.105+0000 7fd353dd4740 10 client.18618 fetch_fsmap learned FSMap version 30 2024-04-02T14:03:57.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.105+0000 7fd353dd4740 1 -- 172.21.15.52:0/1304443703 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bf58929e00 con 0x55bf588b3ca0 2024-04-02T14:03:57.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.105+0000 7fd32eff5640 1 -- 172.21.15.52:0/1304443703 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd34000c810 con 0x55bf588b3ca0 2024-04-02T14:03:57.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.106+0000 7fd32eff5640 1 -- 172.21.15.52:0/1304443703 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd340042040 con 0x55bf588b3ca0 2024-04-02T14:03:57.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.106+0000 7fd353dd4740 10 client.18618 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T14:03:57.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.106+0000 7fd353dd4740 10 client.18618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T14:03:57.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.106+0000 7fd353dd4740 20 client.18618 populate_metadata read hostname 'smithi052' 2024-04-02T14:03:57.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.106+0000 7fd353dd4740 1 --2- 172.21.15.52:0/1304443703 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55bf5892a370 0x55bf5892c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:57.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.106+0000 7fd353dd4740 4 client.18618 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T14:03:57.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.106+0000 7fd353dd4740 1 -- 172.21.15.52:0/1304443703 --> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf58816040 con 0x55bf5892a370 2024-04-02T14:03:57.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.106+0000 7fd34ad9d640 1 --2- 172.21.15.52:0/1304443703 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55bf5892a370 0x55bf5892c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:57.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.106+0000 7fd32eff5640 10 client.18618 ms_handle_connect on v2:172.21.15.88:6838/2513527493 2024-04-02T14:03:57.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.106+0000 7fd34ad9d640 1 --2- 172.21.15.52:0/1304443703 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55bf5892a370 0x55bf5892c790 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2425c039c44a77b server_cookie=bb2354b42c91a6d3 in_seq=0 out_seq=0 2024-04-02T14:03:57.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.106+0000 7fd32eff5640 1 -- 172.21.15.52:0/1304443703 <== mds.0 v2:172.21.15.88:6838/2513527493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf58816040 con 0x55bf5892a370 2024-04-02T14:03:57.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.106+0000 7fd32eff5640 10 client.18618 handle_command_reply: tid=0 2024-04-02T14:03:57.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.106+0000 7fd353dd4740 1 client.18618 shutdown 2024-04-02T14:03:57.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.107+0000 7fd353dd4740 1 -- 172.21.15.52:0/1304443703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 msgr2=0x55bf588ad150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:57.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.107+0000 7fd353dd4740 1 --2- 172.21.15.52:0/1304443703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 0x55bf588ad150 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fd340014570 tx=0x7fd34002d4b0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.107+0000 7fd353dd4740 1 -- 172.21.15.52:0/1304443703 shutdown_connections 2024-04-02T14:03:57.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.107+0000 7fd353dd4740 1 --2- 172.21.15.52:0/1304443703 >> [v2:172.21.15.88:6838/2513527493,v1:172.21.15.88:6839/2513527493] conn(0x55bf5892a370 0x55bf5892c790 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.107+0000 7fd353dd4740 1 --2- 172.21.15.52:0/1304443703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf588b3ca0 0x55bf588ad150 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.107+0000 7fd32eff5640 0 client.18618 ms_handle_reset on v2:172.21.15.88:6838/2513527493 2024-04-02T14:03:57.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.107+0000 7fd353dd4740 1 --2- 172.21.15.52:0/1304443703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bf588ab0c0 0x55bf588bc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.107+0000 7fd353dd4740 1 --2- 172.21.15.52:0/1304443703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf588aa7a0 0x55bf588bbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.107+0000 7fd353dd4740 1 -- 172.21.15.52:0/1304443703 >> 172.21.15.52:0/1304443703 conn(0x55bf588a4ed0 msgr2=0x55bf588a6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:57.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.107+0000 7fd353dd4740 1 -- 172.21.15.52:0/1304443703 shutdown_connections 2024-04-02T14:03:57.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.107+0000 7fd353dd4740 1 -- 172.21.15.52:0/1304443703 wait complete. 2024-04-02T14:03:57.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.107+0000 7fd353dd4740 20 client.18618 trim_cache size 0 max 16384 2024-04-02T14:03:57.122 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-04-02T14:03:57.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.107+0000 7fd3511e9640 1 -- 172.21.15.52:0/178709354 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd3340a3d20 msgr2=0x7fd3340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:57.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.107+0000 7fd3511e9640 1 --2- 172.21.15.52:0/178709354 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd3340a3d20 0x7fd3340a61e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd3440f0a80 tx=0x7fd344090cf0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.107+0000 7fd3511e9640 1 -- 172.21.15.52:0/178709354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd34c107d00 msgr2=0x7fd34c19cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:57.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.107+0000 7fd3511e9640 1 --2- 172.21.15.52:0/178709354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd34c107d00 0x7fd34c19cc40 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fd344004af0 tx=0x7fd344034720 comp rx=0 tx=0).stop 2024-04-02T14:03:57.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.107+0000 7fd3511e9640 1 -- 172.21.15.52:0/178709354 shutdown_connections 2024-04-02T14:03:57.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.107+0000 7fd3511e9640 1 --2- 172.21.15.52:0/178709354 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fd3340a3d20 0x7fd3340a61e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.107+0000 7fd3511e9640 1 --2- 172.21.15.52:0/178709354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd34c107d00 0x7fd34c19cc40 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.107+0000 7fd3511e9640 1 --2- 172.21.15.52:0/178709354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd34c100d80 0x7fd34c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.107+0000 7fd3511e9640 1 --2- 172.21.15.52:0/178709354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd34c1003c0 0x7fd34c198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:57.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.107+0000 7fd3511e9640 1 -- 172.21.15.52:0/178709354 >> 172.21.15.52:0/178709354 conn(0x7fd34c075d40 msgr2=0x7fd34c0fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:57.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.107+0000 7fd3511e9640 1 -- 172.21.15.52:0/178709354 shutdown_connections 2024-04-02T14:03:57.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:57.107+0000 7fd3511e9640 1 -- 172.21.15.52:0/178709354 wait complete. 2024-04-02T14:03:57.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:57 smithi189 ceph-mon[36683]: pgmap v324: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:03:57.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:57 smithi052 ceph-mon[32104]: pgmap v324: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:03:57.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:57 smithi088 ceph-mon[36528]: pgmap v324: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:03:58.126 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-02T14:03:58.126 INFO:tasks.fwd_scrub:done joining 2024-04-02T14:03:58.127 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-02T14:03:58.138 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T14:03:58.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.250+0000 7fae46b00640 1 -- 172.21.15.52:0/2742158590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae401069b0 msgr2=0x7fae40106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:58.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.250+0000 7fae46b00640 1 --2- 172.21.15.52:0/2742158590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae401069b0 0x7fae40106d90 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fae30009930 tx=0x7fae3002bc30 comp rx=0 tx=0).stop 2024-04-02T14:03:58.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.251+0000 7fae46b00640 1 -- 172.21.15.52:0/2742158590 shutdown_connections 2024-04-02T14:03:58.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.251+0000 7fae46b00640 1 --2- 172.21.15.52:0/2742158590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae40101360 0x7fae4010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:58.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.251+0000 7fae46b00640 1 --2- 172.21.15.52:0/2742158590 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae401009a0 0x7fae40100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:58.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.251+0000 7fae46b00640 1 --2- 172.21.15.52:0/2742158590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae401069b0 0x7fae40106d90 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:58.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.251+0000 7fae46b00640 1 -- 172.21.15.52:0/2742158590 >> 172.21.15.52:0/2742158590 conn(0x7fae40076d80 msgr2=0x7fae40077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:58.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.251+0000 7fae46b00640 1 -- 172.21.15.52:0/2742158590 shutdown_connections 2024-04-02T14:03:58.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.251+0000 7fae46b00640 1 -- 172.21.15.52:0/2742158590 wait complete. 2024-04-02T14:03:58.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.251+0000 7fae46b00640 1 Processor -- start 2024-04-02T14:03:58.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.251+0000 7fae46b00640 1 -- start start 2024-04-02T14:03:58.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.251+0000 7fae46b00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae401009a0 0x7fae40198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:58.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.251+0000 7fae46b00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae40101360 0x7fae40198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:58.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.251+0000 7fae46b00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae401069b0 0x7fae4019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:58.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.251+0000 7fae46b00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fae40111ec0 con 0x7fae401069b0 2024-04-02T14:03:58.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.251+0000 7fae46b00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fae40111d40 con 0x7fae401009a0 2024-04-02T14:03:58.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.251+0000 7fae46b00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae40112040 con 0x7fae40101360 2024-04-02T14:03:58.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.252+0000 7fae4509d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae401069b0 0x7fae4019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:58.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.252+0000 7fae4509d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae401069b0 0x7fae4019ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33370/0 (socket says 172.21.15.52:33370) 2024-04-02T14:03:58.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.252+0000 7fae4509d640 1 -- 172.21.15.52:0/705133796 learned_addr learned my addr 172.21.15.52:0/705133796 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:58.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.252+0000 7fae3ffff640 1 --2- 172.21.15.52:0/705133796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae40101360 0x7fae40198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:58.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.252+0000 7fae4489c640 1 --2- 172.21.15.52:0/705133796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae401009a0 0x7fae40198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:58.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.252+0000 7fae4509d640 1 -- 172.21.15.52:0/705133796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae40101360 msgr2=0x7fae40198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:58.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.252+0000 7fae4509d640 1 --2- 172.21.15.52:0/705133796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae40101360 0x7fae40198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:58.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.252+0000 7fae4509d640 1 -- 172.21.15.52:0/705133796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae401009a0 msgr2=0x7fae40198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:58.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.252+0000 7fae4509d640 1 --2- 172.21.15.52:0/705133796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae401009a0 0x7fae40198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:58.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.252+0000 7fae4509d640 1 -- 172.21.15.52:0/705133796 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4019d3f0 con 0x7fae401069b0 2024-04-02T14:03:58.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.252+0000 7fae4509d640 1 --2- 172.21.15.52:0/705133796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae401069b0 0x7fae4019ccc0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fae3800cad0 tx=0x7fae3800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:58.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.252+0000 7fae3dffb640 1 -- 172.21.15.52:0/705133796 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3800ae20 con 0x7fae401069b0 2024-04-02T14:03:58.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.252+0000 7fae3dffb640 1 -- 172.21.15.52:0/705133796 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae38005020 con 0x7fae401069b0 2024-04-02T14:03:58.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.252+0000 7fae46b00640 1 -- 172.21.15.52:0/705133796 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae4019d6e0 con 0x7fae401069b0 2024-04-02T14:03:58.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.252+0000 7fae3dffb640 1 -- 172.21.15.52:0/705133796 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3800f6f0 con 0x7fae401069b0 2024-04-02T14:03:58.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.253+0000 7fae46b00640 1 -- 172.21.15.52:0/705133796 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae401a4f70 con 0x7fae401069b0 2024-04-02T14:03:58.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.254+0000 7fae46b00640 1 -- 172.21.15.52:0/705133796 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae401020e0 con 0x7fae401069b0 2024-04-02T14:03:58.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.254+0000 7fae3dffb640 1 -- 172.21.15.52:0/705133796 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fae380051d0 con 0x7fae401069b0 2024-04-02T14:03:58.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.255+0000 7fae3dffb640 1 --2- 172.21.15.52:0/705133796 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fae280a3da0 0x7fae280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:58.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.255+0000 7fae4489c640 1 --2- 172.21.15.52:0/705133796 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fae280a3da0 0x7fae280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:58.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.255+0000 7fae3dffb640 1 -- 172.21.15.52:0/705133796 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fae380d0aa0 con 0x7fae401069b0 2024-04-02T14:03:58.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.256+0000 7fae4489c640 1 --2- 172.21.15.52:0/705133796 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fae280a3da0 0x7fae280a6260 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fae30005ab0 tx=0x7fae30005fd0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:58.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.260+0000 7fae3dffb640 1 -- 172.21.15.52:0/705133796 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fae38014030 con 0x7fae401069b0 2024-04-02T14:03:58.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.450+0000 7fae46b00640 1 -- 172.21.15.52:0/705133796 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae40199750 con 0x7fae401069b0 2024-04-02T14:03:58.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.451+0000 7fae3dffb640 1 -- 172.21.15.52:0/705133796 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+25112 (secure 0 0 0) 0x7fae3809c760 con 0x7fae401069b0 2024-04-02T14:03:58.455 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T14:03:58.456 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":101,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T14:03:20.471602+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:50:58.778690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:50:59.561527+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:51:03.343487+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/713755826":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/1937276046":"2024-04-03T13:53:29.889159+0000","172.21.15.52:6800/3836085837":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/2957262850":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/3836085837":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/2985814329":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T14:03:58.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.453+0000 7fae46b00640 1 -- 172.21.15.52:0/705133796 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fae280a3da0 msgr2=0x7fae280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:58.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.453+0000 7fae46b00640 1 --2- 172.21.15.52:0/705133796 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fae280a3da0 0x7fae280a6260 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fae30005ab0 tx=0x7fae30005fd0 comp rx=0 tx=0).stop 2024-04-02T14:03:58.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.453+0000 7fae46b00640 1 -- 172.21.15.52:0/705133796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae401069b0 msgr2=0x7fae4019ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:58.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.453+0000 7fae46b00640 1 --2- 172.21.15.52:0/705133796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae401069b0 0x7fae4019ccc0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fae3800cad0 tx=0x7fae3800cf90 comp rx=0 tx=0).stop 2024-04-02T14:03:58.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.453+0000 7fae46b00640 1 -- 172.21.15.52:0/705133796 shutdown_connections 2024-04-02T14:03:58.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.453+0000 7fae46b00640 1 --2- 172.21.15.52:0/705133796 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fae280a3da0 0x7fae280a6260 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:58.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.453+0000 7fae46b00640 1 --2- 172.21.15.52:0/705133796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae401069b0 0x7fae4019ccc0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:58.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.453+0000 7fae46b00640 1 --2- 172.21.15.52:0/705133796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae40101360 0x7fae40198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:58.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.453+0000 7fae46b00640 1 --2- 172.21.15.52:0/705133796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae401009a0 0x7fae40198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:58.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.453+0000 7fae46b00640 1 -- 172.21.15.52:0/705133796 >> 172.21.15.52:0/705133796 conn(0x7fae40076d80 msgr2=0x7fae400fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:58.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.453+0000 7fae46b00640 1 -- 172.21.15.52:0/705133796 shutdown_connections 2024-04-02T14:03:58.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.453+0000 7fae46b00640 1 -- 172.21.15.52:0/705133796 wait complete. 2024-04-02T14:03:58.468 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:46:20.746053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:50:58.778690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:50:59.561527+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:51:03.343487+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T14:03:58.469 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T14:03:58.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.570+0000 7f740e7cd640 1 -- 172.21.15.52:0/1680111241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f740810c110 msgr2=0x7f740810e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:58.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.570+0000 7f740e7cd640 1 --2- 172.21.15.52:0/1680111241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f740810c110 0x7f740810e540 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f7400009ca0 tx=0x7f740002c180 comp rx=0 tx=0).stop 2024-04-02T14:03:58.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.570+0000 7f740e7cd640 1 -- 172.21.15.52:0/1680111241 shutdown_connections 2024-04-02T14:03:58.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.570+0000 7f740e7cd640 1 --2- 172.21.15.52:0/1680111241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f740810c110 0x7f740810e540 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:58.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.570+0000 7f740e7cd640 1 --2- 172.21.15.52:0/1680111241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7408103660 0x7f740810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:58.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.570+0000 7f740e7cd640 1 --2- 172.21.15.52:0/1680111241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7408102d40 0x7f7408103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:58.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.570+0000 7f740e7cd640 1 -- 172.21.15.52:0/1680111241 >> 172.21.15.52:0/1680111241 conn(0x7f7408076d80 msgr2=0x7f7408077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:58.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.570+0000 7f740e7cd640 1 -- 172.21.15.52:0/1680111241 shutdown_connections 2024-04-02T14:03:58.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.570+0000 7f740e7cd640 1 -- 172.21.15.52:0/1680111241 wait complete. 2024-04-02T14:03:58.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.571+0000 7f740e7cd640 1 Processor -- start 2024-04-02T14:03:58.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.571+0000 7f740e7cd640 1 -- start start 2024-04-02T14:03:58.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.571+0000 7f740e7cd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7408102d40 0x7f74081983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:58.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.571+0000 7f740e7cd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7408103660 0x7f7408198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:58.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.571+0000 7f740e7cd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f740810c110 0x7f740819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:58.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.571+0000 7f740e7cd640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7408111ea0 con 0x7f7408102d40 2024-04-02T14:03:58.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.571+0000 7f740e7cd640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7408111d20 con 0x7f7408103660 2024-04-02T14:03:58.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.571+0000 7f740e7cd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7408112020 con 0x7f740810c110 2024-04-02T14:03:58.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.571+0000 7f7407fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7408102d40 0x7f74081983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:58.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.572+0000 7f7407fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7408102d40 0x7f74081983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33386/0 (socket says 172.21.15.52:33386) 2024-04-02T14:03:58.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.572+0000 7f7407fff640 1 -- 172.21.15.52:0/3051988091 learned_addr learned my addr 172.21.15.52:0/3051988091 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:58.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.572+0000 7f740cd6a640 1 --2- 172.21.15.52:0/3051988091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f740810c110 0x7f740819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:58.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.572+0000 7f74077fe640 1 --2- 172.21.15.52:0/3051988091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7408103660 0x7f7408198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:58.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.572+0000 7f7407fff640 1 -- 172.21.15.52:0/3051988091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f740810c110 msgr2=0x7f740819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:58.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.572+0000 7f7407fff640 1 --2- 172.21.15.52:0/3051988091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f740810c110 0x7f740819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:58.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.572+0000 7f7407fff640 1 -- 172.21.15.52:0/3051988091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7408103660 msgr2=0x7f7408198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:58.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.572+0000 7f7407fff640 1 --2- 172.21.15.52:0/3051988091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7408103660 0x7f7408198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:58.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.572+0000 7f7407fff640 1 -- 172.21.15.52:0/3051988091 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740819d360 con 0x7f7408102d40 2024-04-02T14:03:58.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.572+0000 7f7407fff640 1 --2- 172.21.15.52:0/3051988091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7408102d40 0x7f74081983c0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f73f800d950 tx=0x7f73f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:58.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.572+0000 7f74057fa640 1 -- 172.21.15.52:0/3051988091 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8017070 con 0x7f7408102d40 2024-04-02T14:03:58.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.572+0000 7f74057fa640 1 -- 172.21.15.52:0/3051988091 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73f8009c30 con 0x7f7408102d40 2024-04-02T14:03:58.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.572+0000 7f74057fa640 1 -- 172.21.15.52:0/3051988091 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f800f6b0 con 0x7f7408102d40 2024-04-02T14:03:58.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.572+0000 7f740e7cd640 1 -- 172.21.15.52:0/3051988091 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f740819d650 con 0x7f7408102d40 2024-04-02T14:03:58.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.572+0000 7f740e7cd640 1 -- 172.21.15.52:0/3051988091 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74081a4ee0 con 0x7f7408102d40 2024-04-02T14:03:58.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.573+0000 7f740e7cd640 1 -- 172.21.15.52:0/3051988091 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f740806a8f0 con 0x7f7408102d40 2024-04-02T14:03:58.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.574+0000 7f74057fa640 1 -- 172.21.15.52:0/3051988091 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f73f80109d0 con 0x7f7408102d40 2024-04-02T14:03:58.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.575+0000 7f74057fa640 1 --2- 172.21.15.52:0/3051988091 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f73f00a3da0 0x7f73f00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:58.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.575+0000 7f74057fa640 1 -- 172.21.15.52:0/3051988091 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f73f80d0790 con 0x7f7408102d40 2024-04-02T14:03:58.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.575+0000 7f74077fe640 1 --2- 172.21.15.52:0/3051988091 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f73f00a3da0 0x7f73f00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:58.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.575+0000 7f74077fe640 1 --2- 172.21.15.52:0/3051988091 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f73f00a3da0 0x7f73f00a6260 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f74081001e0 tx=0x7f73fc005f90 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:58.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.579+0000 7f74057fa640 1 -- 172.21.15.52:0/3051988091 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f73f8009de0 con 0x7f7408102d40 2024-04-02T14:03:58.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.764+0000 7f740e7cd640 1 -- 172.21.15.52:0/3051988091 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f74081996c0 con 0x7f7408102d40 2024-04-02T14:03:58.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.764+0000 7f74057fa640 1 -- 172.21.15.52:0/3051988091 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7f73f809ca40 con 0x7f7408102d40 2024-04-02T14:03:58.769 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-04-02T14:03:58.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.766+0000 7f740e7cd640 1 -- 172.21.15.52:0/3051988091 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f73f00a3da0 msgr2=0x7f73f00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:58.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.766+0000 7f740e7cd640 1 --2- 172.21.15.52:0/3051988091 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f73f00a3da0 0x7f73f00a6260 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f74081001e0 tx=0x7f73fc005f90 comp rx=0 tx=0).stop 2024-04-02T14:03:58.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.766+0000 7f740e7cd640 1 -- 172.21.15.52:0/3051988091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7408102d40 msgr2=0x7f74081983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:58.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.766+0000 7f740e7cd640 1 --2- 172.21.15.52:0/3051988091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7408102d40 0x7f74081983c0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f73f800d950 tx=0x7f73f800de10 comp rx=0 tx=0).stop 2024-04-02T14:03:58.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.766+0000 7f740e7cd640 1 -- 172.21.15.52:0/3051988091 shutdown_connections 2024-04-02T14:03:58.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.766+0000 7f740e7cd640 1 --2- 172.21.15.52:0/3051988091 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f73f00a3da0 0x7f73f00a6260 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:58.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.766+0000 7f740e7cd640 1 --2- 172.21.15.52:0/3051988091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f740810c110 0x7f740819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:58.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.766+0000 7f740e7cd640 1 --2- 172.21.15.52:0/3051988091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7408103660 0x7f7408198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:58.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.766+0000 7f740e7cd640 1 --2- 172.21.15.52:0/3051988091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7408102d40 0x7f74081983c0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:58.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.766+0000 7f740e7cd640 1 -- 172.21.15.52:0/3051988091 >> 172.21.15.52:0/3051988091 conn(0x7f7408076d80 msgr2=0x7f74080fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:58.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.767+0000 7f740e7cd640 1 -- 172.21.15.52:0/3051988091 shutdown_connections 2024-04-02T14:03:58.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.767+0000 7f740e7cd640 1 -- 172.21.15.52:0/3051988091 wait complete. 2024-04-02T14:03:58.781 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T14:03:58.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.884+0000 7f6495f96640 1 -- 172.21.15.52:0/2652495769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6490101330 msgr2=0x7f649010d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:58.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.884+0000 7f6495f96640 1 --2- 172.21.15.52:0/2652495769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6490101330 0x7f649010d840 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f6488009930 tx=0x7f648802bc30 comp rx=0 tx=0).stop 2024-04-02T14:03:58.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.884+0000 7f6495f96640 1 -- 172.21.15.52:0/2652495769 shutdown_connections 2024-04-02T14:03:58.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.884+0000 7f6495f96640 1 --2- 172.21.15.52:0/2652495769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6490101330 0x7f649010d840 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:58.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.884+0000 7f6495f96640 1 --2- 172.21.15.52:0/2652495769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6490100970 0x7f6490100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:58.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.884+0000 7f6495f96640 1 --2- 172.21.15.52:0/2652495769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6490106980 0x7f6490106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:58.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.884+0000 7f6495f96640 1 -- 172.21.15.52:0/2652495769 >> 172.21.15.52:0/2652495769 conn(0x7f6490077b00 msgr2=0x7f6490077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:58.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.884+0000 7f6495f96640 1 -- 172.21.15.52:0/2652495769 shutdown_connections 2024-04-02T14:03:58.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.884+0000 7f6495f96640 1 -- 172.21.15.52:0/2652495769 wait complete. 2024-04-02T14:03:58.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.884+0000 7f6495f96640 1 Processor -- start 2024-04-02T14:03:58.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.884+0000 7f6495f96640 1 -- start start 2024-04-02T14:03:58.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.885+0000 7f6495f96640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6490100970 0x7f649019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:58.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.885+0000 7f6495f96640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6490101330 0x7f649019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:58.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.885+0000 7f6495f96640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6490106980 0x7f649019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:58.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.885+0000 7f6495f96640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6490110d90 con 0x7f6490101330 2024-04-02T14:03:58.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.885+0000 7f6495f96640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6490110c10 con 0x7f6490106980 2024-04-02T14:03:58.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.885+0000 7f6495f96640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6490110f10 con 0x7f6490100970 2024-04-02T14:03:58.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.885+0000 7f648effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6490101330 0x7f649019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:58.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.885+0000 7f648effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6490101330 0x7f649019aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33404/0 (socket says 172.21.15.52:33404) 2024-04-02T14:03:58.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.885+0000 7f648effd640 1 -- 172.21.15.52:0/2386637853 learned_addr learned my addr 172.21.15.52:0/2386637853 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:58.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.885+0000 7f648ffff640 1 --2- 172.21.15.52:0/2386637853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6490106980 0x7f649019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:58.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.885+0000 7f648f7fe640 1 --2- 172.21.15.52:0/2386637853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6490100970 0x7f649019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:58.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.885+0000 7f648effd640 1 -- 172.21.15.52:0/2386637853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6490100970 msgr2=0x7f649019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:58.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.885+0000 7f648effd640 1 --2- 172.21.15.52:0/2386637853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6490100970 0x7f649019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:58.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.885+0000 7f648effd640 1 -- 172.21.15.52:0/2386637853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6490106980 msgr2=0x7f649019ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:58.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.885+0000 7f648effd640 1 --2- 172.21.15.52:0/2386637853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6490106980 0x7f649019ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:58.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.885+0000 7f648effd640 1 -- 172.21.15.52:0/2386637853 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649019f570 con 0x7f6490101330 2024-04-02T14:03:58.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.885+0000 7f648effd640 1 --2- 172.21.15.52:0/2386637853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6490101330 0x7f649019aac0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f647800e9f0 tx=0x7f647800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:58.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.885+0000 7f648cff9640 1 -- 172.21.15.52:0/2386637853 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647800cd80 con 0x7f6490101330 2024-04-02T14:03:58.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.885+0000 7f648cff9640 1 -- 172.21.15.52:0/2386637853 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6478010910 con 0x7f6490101330 2024-04-02T14:03:58.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.885+0000 7f6495f96640 1 -- 172.21.15.52:0/2386637853 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f649019f860 con 0x7f6490101330 2024-04-02T14:03:58.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.885+0000 7f648cff9640 1 -- 172.21.15.52:0/2386637853 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6478018990 con 0x7f6490101330 2024-04-02T14:03:58.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.885+0000 7f6495f96640 1 -- 172.21.15.52:0/2386637853 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64901a70f0 con 0x7f6490101330 2024-04-02T14:03:58.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.886+0000 7f6495f96640 1 -- 172.21.15.52:0/2386637853 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64900688f0 con 0x7f6490101330 2024-04-02T14:03:58.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.888+0000 7f648cff9640 1 -- 172.21.15.52:0/2386637853 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6478010ac0 con 0x7f6490101330 2024-04-02T14:03:58.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.888+0000 7f648cff9640 1 --2- 172.21.15.52:0/2386637853 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f647c0a3d20 0x7f647c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:58.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.888+0000 7f648f7fe640 1 --2- 172.21.15.52:0/2386637853 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f647c0a3d20 0x7f647c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:58.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.888+0000 7f648cff9640 1 -- 172.21.15.52:0/2386637853 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f64780d13e0 con 0x7f6490101330 2024-04-02T14:03:58.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.889+0000 7f648f7fe640 1 --2- 172.21.15.52:0/2386637853 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f647c0a3d20 0x7f647c0a61e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6480005fd0 tx=0x7f6480009500 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:58.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:58.892+0000 7f648cff9640 1 -- 172.21.15.52:0/2386637853 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f647809d6f0 con 0x7f6490101330 2024-04-02T14:03:59.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.076+0000 7f6495f96640 1 -- 172.21.15.52:0/2386637853 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f64901a7390 con 0x7f6490101330 2024-04-02T14:03:59.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.076+0000 7f648cff9640 1 -- 172.21.15.52:0/2386637853 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) ==== 104+0+11 (secure 0 0 0) 0x7f64780bd0a0 con 0x7f6490101330 2024-04-02T14:03:59.080 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-04-02T14:03:59.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.079+0000 7f6495f96640 1 -- 172.21.15.52:0/2386637853 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f647c0a3d20 msgr2=0x7f647c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:59.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.079+0000 7f6495f96640 1 --2- 172.21.15.52:0/2386637853 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f647c0a3d20 0x7f647c0a61e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6480005fd0 tx=0x7f6480009500 comp rx=0 tx=0).stop 2024-04-02T14:03:59.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.079+0000 7f6495f96640 1 -- 172.21.15.52:0/2386637853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6490101330 msgr2=0x7f649019aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:59.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.079+0000 7f6495f96640 1 --2- 172.21.15.52:0/2386637853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6490101330 0x7f649019aac0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f647800e9f0 tx=0x7f647800eeb0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.079+0000 7f6495f96640 1 -- 172.21.15.52:0/2386637853 shutdown_connections 2024-04-02T14:03:59.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.079+0000 7f6495f96640 1 --2- 172.21.15.52:0/2386637853 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f647c0a3d20 0x7f647c0a61e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.079+0000 7f6495f96640 1 --2- 172.21.15.52:0/2386637853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6490106980 0x7f649019ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.079+0000 7f6495f96640 1 --2- 172.21.15.52:0/2386637853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6490101330 0x7f649019aac0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.079+0000 7f6495f96640 1 --2- 172.21.15.52:0/2386637853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6490100970 0x7f649019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.079+0000 7f6495f96640 1 -- 172.21.15.52:0/2386637853 >> 172.21.15.52:0/2386637853 conn(0x7f6490077b00 msgr2=0x7f6490068ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:59.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.079+0000 7f6495f96640 1 -- 172.21.15.52:0/2386637853 shutdown_connections 2024-04-02T14:03:59.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.079+0000 7f6495f96640 1 -- 172.21.15.52:0/2386637853 wait complete. 2024-04-02T14:03:59.093 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T14:03:59.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.195+0000 7f1667ff6640 1 -- 172.21.15.52:0/810636927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1660101330 msgr2=0x7f166010d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:59.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.195+0000 7f1667ff6640 1 --2- 172.21.15.52:0/810636927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1660101330 0x7f166010d840 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f1658009cd0 tx=0x7f165802c0f0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.196+0000 7f1667ff6640 1 -- 172.21.15.52:0/810636927 shutdown_connections 2024-04-02T14:03:59.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.196+0000 7f1667ff6640 1 --2- 172.21.15.52:0/810636927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1660101330 0x7f166010d840 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.196+0000 7f1667ff6640 1 --2- 172.21.15.52:0/810636927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1660100970 0x7f1660100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.196+0000 7f1667ff6640 1 --2- 172.21.15.52:0/810636927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1660106980 0x7f1660106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.196+0000 7f1667ff6640 1 -- 172.21.15.52:0/810636927 >> 172.21.15.52:0/810636927 conn(0x7f1660077b00 msgr2=0x7f1660077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:59.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.196+0000 7f1667ff6640 1 -- 172.21.15.52:0/810636927 shutdown_connections 2024-04-02T14:03:59.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.196+0000 7f1667ff6640 1 -- 172.21.15.52:0/810636927 wait complete. 2024-04-02T14:03:59.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.196+0000 7f1667ff6640 1 Processor -- start 2024-04-02T14:03:59.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.196+0000 7f1667ff6640 1 -- start start 2024-04-02T14:03:59.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.196+0000 7f1667ff6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1660100970 0x7f166019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:59.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.196+0000 7f1667ff6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1660101330 0x7f166019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:59.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.196+0000 7f1667ff6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1660106980 0x7f166019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:59.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.196+0000 7f1667ff6640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1660111e90 con 0x7f1660106980 2024-04-02T14:03:59.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.196+0000 7f1667ff6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1660111d10 con 0x7f1660101330 2024-04-02T14:03:59.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.196+0000 7f1667ff6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1660112010 con 0x7f1660100970 2024-04-02T14:03:59.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.196+0000 7f1666593640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1660106980 0x7f166019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:59.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.197+0000 7f1666593640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1660106980 0x7f166019ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33428/0 (socket says 172.21.15.52:33428) 2024-04-02T14:03:59.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.197+0000 7f1666593640 1 -- 172.21.15.52:0/2356580023 learned_addr learned my addr 172.21.15.52:0/2356580023 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:59.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.197+0000 7f1665591640 1 --2- 172.21.15.52:0/2356580023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1660101330 0x7f166019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:59.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.197+0000 7f1665d92640 1 --2- 172.21.15.52:0/2356580023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1660100970 0x7f166019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:59.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.197+0000 7f1666593640 1 -- 172.21.15.52:0/2356580023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1660100970 msgr2=0x7f166019a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:59.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.197+0000 7f1666593640 1 --2- 172.21.15.52:0/2356580023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1660100970 0x7f166019a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.197+0000 7f1666593640 1 -- 172.21.15.52:0/2356580023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1660101330 msgr2=0x7f166019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:59.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.197+0000 7f1666593640 1 --2- 172.21.15.52:0/2356580023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1660101330 0x7f166019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.197+0000 7f1666593640 1 -- 172.21.15.52:0/2356580023 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166019f540 con 0x7f1660106980 2024-04-02T14:03:59.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.197+0000 7f1665591640 1 --2- 172.21.15.52:0/2356580023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1660101330 0x7f166019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:03:59.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.197+0000 7f1665d92640 1 --2- 172.21.15.52:0/2356580023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1660100970 0x7f166019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:03:59.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.197+0000 7f1666593640 1 --2- 172.21.15.52:0/2356580023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1660106980 0x7f166019ee10 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f1658004af0 tx=0x7f1658032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:59.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.197+0000 7f1656ffd640 1 -- 172.21.15.52:0/2356580023 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1658041020 con 0x7f1660106980 2024-04-02T14:03:59.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.197+0000 7f1656ffd640 1 -- 172.21.15.52:0/2356580023 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1658034cd0 con 0x7f1660106980 2024-04-02T14:03:59.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.197+0000 7f1667ff6640 1 -- 172.21.15.52:0/2356580023 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166019f7d0 con 0x7f1660106980 2024-04-02T14:03:59.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.197+0000 7f1656ffd640 1 -- 172.21.15.52:0/2356580023 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1658039040 con 0x7f1660106980 2024-04-02T14:03:59.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.197+0000 7f1667ff6640 1 -- 172.21.15.52:0/2356580023 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16601a70c0 con 0x7f1660106980 2024-04-02T14:03:59.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.198+0000 7f1667ff6640 1 -- 172.21.15.52:0/2356580023 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16600688f0 con 0x7f1660106980 2024-04-02T14:03:59.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.199+0000 7f1656ffd640 1 -- 172.21.15.52:0/2356580023 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1658044020 con 0x7f1660106980 2024-04-02T14:03:59.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.200+0000 7f1656ffd640 1 --2- 172.21.15.52:0/2356580023 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f164c0a3d20 0x7f164c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:59.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.200+0000 7f1665d92640 1 --2- 172.21.15.52:0/2356580023 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f164c0a3d20 0x7f164c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:59.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.200+0000 7f1656ffd640 1 -- 172.21.15.52:0/2356580023 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f16580f28c0 con 0x7f1660106980 2024-04-02T14:03:59.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.200+0000 7f1665d92640 1 --2- 172.21.15.52:0/2356580023 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f164c0a3d20 0x7f164c0a61e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f165c0058c0 tx=0x7f165c005cb0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:59.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.204+0000 7f1656ffd640 1 -- 172.21.15.52:0/2356580023 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f165804b020 con 0x7f1660106980 2024-04-02T14:03:59.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.390+0000 7f1667ff6640 1 -- 172.21.15.52:0/2356580023 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f16601a7290 con 0x7f1660106980 2024-04-02T14:03:59.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.390+0000 7f1656ffd640 1 -- 172.21.15.52:0/2356580023 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) ==== 100+0+11 (secure 0 0 0) 0x7f16580bece0 con 0x7f1660106980 2024-04-02T14:03:59.394 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-04-02T14:03:59.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.392+0000 7f1667ff6640 1 -- 172.21.15.52:0/2356580023 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f164c0a3d20 msgr2=0x7f164c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:59.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.392+0000 7f1667ff6640 1 --2- 172.21.15.52:0/2356580023 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f164c0a3d20 0x7f164c0a61e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f165c0058c0 tx=0x7f165c005cb0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.392+0000 7f1667ff6640 1 -- 172.21.15.52:0/2356580023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1660106980 msgr2=0x7f166019ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:59.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.392+0000 7f1667ff6640 1 --2- 172.21.15.52:0/2356580023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1660106980 0x7f166019ee10 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f1658004af0 tx=0x7f1658032ec0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.392+0000 7f1667ff6640 1 -- 172.21.15.52:0/2356580023 shutdown_connections 2024-04-02T14:03:59.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.392+0000 7f1667ff6640 1 --2- 172.21.15.52:0/2356580023 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f164c0a3d20 0x7f164c0a61e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.392+0000 7f1667ff6640 1 --2- 172.21.15.52:0/2356580023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1660106980 0x7f166019ee10 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.392+0000 7f1667ff6640 1 --2- 172.21.15.52:0/2356580023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1660101330 0x7f166019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.392+0000 7f1667ff6640 1 --2- 172.21.15.52:0/2356580023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1660100970 0x7f166019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.392+0000 7f1667ff6640 1 -- 172.21.15.52:0/2356580023 >> 172.21.15.52:0/2356580023 conn(0x7f1660077b00 msgr2=0x7f166010c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:59.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.392+0000 7f1667ff6640 1 -- 172.21.15.52:0/2356580023 shutdown_connections 2024-04-02T14:03:59.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.392+0000 7f1667ff6640 1 -- 172.21.15.52:0/2356580023 wait complete. 2024-04-02T14:03:59.406 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T14:03:59.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.509+0000 7f82f984a640 1 -- 172.21.15.52:0/948526952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82f4100c20 msgr2=0x7f82f4106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:59.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.509+0000 7f82f984a640 1 --2- 172.21.15.52:0/948526952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82f4100c20 0x7f82f4106750 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f82dc009900 tx=0x7f82dc02bc30 comp rx=0 tx=0).stop 2024-04-02T14:03:59.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.509+0000 7f82f984a640 1 -- 172.21.15.52:0/948526952 shutdown_connections 2024-04-02T14:03:59.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.509+0000 7f82f984a640 1 --2- 172.21.15.52:0/948526952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82f4106e80 0x7f82f410ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.509+0000 7f82f984a640 1 --2- 172.21.15.52:0/948526952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82f4100c20 0x7f82f4106750 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.509+0000 7f82f984a640 1 --2- 172.21.15.52:0/948526952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f82f4100270 0x7f82f4100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.509+0000 7f82f984a640 1 -- 172.21.15.52:0/948526952 >> 172.21.15.52:0/948526952 conn(0x7f82f4076fc0 msgr2=0x7f82f40fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:59.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.509+0000 7f82f984a640 1 -- 172.21.15.52:0/948526952 shutdown_connections 2024-04-02T14:03:59.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.509+0000 7f82f984a640 1 -- 172.21.15.52:0/948526952 wait complete. 2024-04-02T14:03:59.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.510+0000 7f82f984a640 1 Processor -- start 2024-04-02T14:03:59.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.510+0000 7f82f984a640 1 -- start start 2024-04-02T14:03:59.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.510+0000 7f82f984a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f82f4100270 0x7f82f41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:59.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.510+0000 7f82f984a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82f4106e80 0x7f82f4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:59.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.510+0000 7f82f984a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82f419cca0 0x7f82f419d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:59.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.510+0000 7f82f984a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f82f4111ee0 con 0x7f82f4106e80 2024-04-02T14:03:59.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.510+0000 7f82f984a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f82f4111d60 con 0x7f82f419cca0 2024-04-02T14:03:59.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.510+0000 7f82f984a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f82f4112060 con 0x7f82f4100270 2024-04-02T14:03:59.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.510+0000 7f82ebfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82f4106e80 0x7f82f4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:59.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.510+0000 7f82ebfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82f4106e80 0x7f82f4198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33448/0 (socket says 172.21.15.52:33448) 2024-04-02T14:03:59.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.510+0000 7f82ebfff640 1 -- 172.21.15.52:0/4005001766 learned_addr learned my addr 172.21.15.52:0/4005001766 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:59.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.510+0000 7f82f37fe640 1 --2- 172.21.15.52:0/4005001766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82f419cca0 0x7f82f419d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:59.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.511+0000 7f82ebfff640 1 -- 172.21.15.52:0/4005001766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f82f4100270 msgr2=0x7f82f41983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:59.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.511+0000 7f82ebfff640 1 --2- 172.21.15.52:0/4005001766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f82f4100270 0x7f82f41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.511+0000 7f82ebfff640 1 -- 172.21.15.52:0/4005001766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82f419cca0 msgr2=0x7f82f419d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:59.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.511+0000 7f82ebfff640 1 --2- 172.21.15.52:0/4005001766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82f419cca0 0x7f82f419d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.511+0000 7f82ebfff640 1 -- 172.21.15.52:0/4005001766 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f419d790 con 0x7f82f4106e80 2024-04-02T14:03:59.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.511+0000 7f82ebfff640 1 --2- 172.21.15.52:0/4005001766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82f4106e80 0x7f82f4198920 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f82dc036040 tx=0x7f82dc02cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:59.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.511+0000 7f82f8848640 1 -- 172.21.15.52:0/4005001766 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc041070 con 0x7f82f4106e80 2024-04-02T14:03:59.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.511+0000 7f82f8848640 1 -- 172.21.15.52:0/4005001766 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82dc004450 con 0x7f82f4106e80 2024-04-02T14:03:59.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.511+0000 7f82f984a640 1 -- 172.21.15.52:0/4005001766 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f419da20 con 0x7f82f4106e80 2024-04-02T14:03:59.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.511+0000 7f82f8848640 1 -- 172.21.15.52:0/4005001766 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc003e50 con 0x7f82f4106e80 2024-04-02T14:03:59.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.511+0000 7f82f984a640 1 -- 172.21.15.52:0/4005001766 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f41a5190 con 0x7f82f4106e80 2024-04-02T14:03:59.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.512+0000 7f82f984a640 1 -- 172.21.15.52:0/4005001766 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f4068d40 con 0x7f82f4106e80 2024-04-02T14:03:59.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.513+0000 7f82f8848640 1 -- 172.21.15.52:0/4005001766 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f82dc044070 con 0x7f82f4106e80 2024-04-02T14:03:59.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.514+0000 7f82f8848640 1 --2- 172.21.15.52:0/4005001766 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f82e00a3ec0 0x7f82e00a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:59.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.514+0000 7f82f8848640 1 -- 172.21.15.52:0/4005001766 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f82dc0f2c50 con 0x7f82f4106e80 2024-04-02T14:03:59.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.514+0000 7f82f2ffd640 1 --2- 172.21.15.52:0/4005001766 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f82e00a3ec0 0x7f82e00a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:59.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.514+0000 7f82f2ffd640 1 --2- 172.21.15.52:0/4005001766 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f82e00a3ec0 0x7f82e00a6380 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f82e4005fd0 tx=0x7f82e4005850 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:59.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.517+0000 7f82f8848640 1 -- 172.21.15.52:0/4005001766 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f82dc0bf070 con 0x7f82f4106e80 2024-04-02T14:03:59.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.703+0000 7f82f984a640 1 -- 172.21.15.52:0/4005001766 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f82f4100c20 con 0x7f82f4106e80 2024-04-02T14:03:59.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.704+0000 7f82f8848640 1 -- 172.21.15.52:0/4005001766 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v101) ==== 103+0+11 (secure 0 0 0) 0x7f82dc0be970 con 0x7f82f4106e80 2024-04-02T14:03:59.708 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-04-02T14:03:59.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.705+0000 7f82f984a640 1 -- 172.21.15.52:0/4005001766 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f82e00a3ec0 msgr2=0x7f82e00a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:59.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.705+0000 7f82f984a640 1 --2- 172.21.15.52:0/4005001766 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f82e00a3ec0 0x7f82e00a6380 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f82e4005fd0 tx=0x7f82e4005850 comp rx=0 tx=0).stop 2024-04-02T14:03:59.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.705+0000 7f82f984a640 1 -- 172.21.15.52:0/4005001766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82f4106e80 msgr2=0x7f82f4198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:59.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.705+0000 7f82f984a640 1 --2- 172.21.15.52:0/4005001766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82f4106e80 0x7f82f4198920 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f82dc036040 tx=0x7f82dc02cbf0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.706+0000 7f82f984a640 1 -- 172.21.15.52:0/4005001766 shutdown_connections 2024-04-02T14:03:59.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.706+0000 7f82f984a640 1 --2- 172.21.15.52:0/4005001766 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f82e00a3ec0 0x7f82e00a6380 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.706+0000 7f82f984a640 1 --2- 172.21.15.52:0/4005001766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82f419cca0 0x7f82f419d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.706+0000 7f82f984a640 1 --2- 172.21.15.52:0/4005001766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82f4106e80 0x7f82f4198920 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.706+0000 7f82f984a640 1 --2- 172.21.15.52:0/4005001766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f82f4100270 0x7f82f41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.706+0000 7f82f984a640 1 -- 172.21.15.52:0/4005001766 >> 172.21.15.52:0/4005001766 conn(0x7f82f4076fc0 msgr2=0x7f82f4105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:59.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.706+0000 7f82f984a640 1 -- 172.21.15.52:0/4005001766 shutdown_connections 2024-04-02T14:03:59.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.706+0000 7f82f984a640 1 -- 172.21.15.52:0/4005001766 wait complete. 2024-04-02T14:03:59.719 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:59 smithi052 ceph-mon[32104]: pgmap v325: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:03:59.719 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:59 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/705133796' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:03:59.719 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:59 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3051988091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:03:59.719 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:59 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2386637853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:03:59.719 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:03:59 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2356580023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:03:59.721 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-02T14:03:59.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:59 smithi189 ceph-mon[36683]: pgmap v325: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:03:59.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:59 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/705133796' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:03:59.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:59 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3051988091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:03:59.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:59 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2386637853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:03:59.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:03:59 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2356580023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:03:59.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.820+0000 7f81d6689640 1 -- 172.21.15.52:0/4036905168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81d006a1b0 msgr2=0x7f81d006a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:59.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.820+0000 7f81d6689640 1 --2- 172.21.15.52:0/4036905168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81d006a1b0 0x7f81d006a610 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f81c0009900 tx=0x7f81c002bcd0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.820+0000 7f81d6689640 1 -- 172.21.15.52:0/4036905168 shutdown_connections 2024-04-02T14:03:59.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.820+0000 7f81d6689640 1 --2- 172.21.15.52:0/4036905168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81d006ab50 0x7f81d010e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.820+0000 7f81d6689640 1 --2- 172.21.15.52:0/4036905168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81d006a1b0 0x7f81d006a610 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.820+0000 7f81d6689640 1 --2- 172.21.15.52:0/4036905168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d0107b90 0x7f81d0107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.820+0000 7f81d6689640 1 -- 172.21.15.52:0/4036905168 >> 172.21.15.52:0/4036905168 conn(0x7f81d0077aa0 msgr2=0x7f81d0077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:03:59.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.820+0000 7f81d6689640 1 -- 172.21.15.52:0/4036905168 shutdown_connections 2024-04-02T14:03:59.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.820+0000 7f81d6689640 1 -- 172.21.15.52:0/4036905168 wait complete. 2024-04-02T14:03:59.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.820+0000 7f81d6689640 1 Processor -- start 2024-04-02T14:03:59.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.820+0000 7f81d6689640 1 -- start start 2024-04-02T14:03:59.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.821+0000 7f81d6689640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81d006a1b0 0x7f81d01984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:59.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.821+0000 7f81d6689640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d006ab50 0x7f81d01989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:59.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.821+0000 7f81d6689640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81d0107b90 0x7f81d019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:59.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.821+0000 7f81d6689640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f81d0111f80 con 0x7f81d006a1b0 2024-04-02T14:03:59.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.821+0000 7f81d6689640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f81d0111e00 con 0x7f81d006ab50 2024-04-02T14:03:59.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.821+0000 7f81d6689640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f81d0112100 con 0x7f81d0107b90 2024-04-02T14:03:59.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.821+0000 7f81cffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81d006a1b0 0x7f81d01984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:59.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.821+0000 7f81cffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81d006a1b0 0x7f81d01984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33458/0 (socket says 172.21.15.52:33458) 2024-04-02T14:03:59.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.821+0000 7f81cffff640 1 -- 172.21.15.52:0/1200357950 learned_addr learned my addr 172.21.15.52:0/1200357950 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:03:59.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.821+0000 7f81cf7fe640 1 --2- 172.21.15.52:0/1200357950 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d006ab50 0x7f81d01989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:59.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.821+0000 7f81cffff640 1 -- 172.21.15.52:0/1200357950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81d0107b90 msgr2=0x7f81d019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:59.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.821+0000 7f81cffff640 1 --2- 172.21.15.52:0/1200357950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81d0107b90 0x7f81d019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.821+0000 7f81cffff640 1 -- 172.21.15.52:0/1200357950 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d006ab50 msgr2=0x7f81d01989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:03:59.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.821+0000 7f81cffff640 1 --2- 172.21.15.52:0/1200357950 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d006ab50 0x7f81d01989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:03:59.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.821+0000 7f81cffff640 1 -- 172.21.15.52:0/1200357950 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d019d490 con 0x7f81d006a1b0 2024-04-02T14:03:59.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.821+0000 7f81cf7fe640 1 --2- 172.21.15.52:0/1200357950 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d006ab50 0x7f81d01989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:03:59.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.821+0000 7f81cffff640 1 --2- 172.21.15.52:0/1200357950 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81d006a1b0 0x7f81d01984a0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f81c400b4a0 tx=0x7f81c400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:59.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.822+0000 7f81cd7fa640 1 -- 172.21.15.52:0/1200357950 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c4014070 con 0x7f81d006a1b0 2024-04-02T14:03:59.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.822+0000 7f81d6689640 1 -- 172.21.15.52:0/1200357950 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81d019d780 con 0x7f81d006a1b0 2024-04-02T14:03:59.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.822+0000 7f81cd7fa640 1 -- 172.21.15.52:0/1200357950 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81c4010af0 con 0x7f81d006a1b0 2024-04-02T14:03:59.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.822+0000 7f81cd7fa640 1 -- 172.21.15.52:0/1200357950 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c4017d10 con 0x7f81d006a1b0 2024-04-02T14:03:59.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.822+0000 7f81d6689640 1 -- 172.21.15.52:0/1200357950 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81d01a5010 con 0x7f81d006a1b0 2024-04-02T14:03:59.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.823+0000 7f81d6689640 1 -- 172.21.15.52:0/1200357950 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81d0068d40 con 0x7f81d006a1b0 2024-04-02T14:03:59.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.824+0000 7f81cd7fa640 1 -- 172.21.15.52:0/1200357950 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f81c4017070 con 0x7f81d006a1b0 2024-04-02T14:03:59.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.825+0000 7f81cd7fa640 1 --2- 172.21.15.52:0/1200357950 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f81b80a3d20 0x7f81b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:03:59.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.825+0000 7f81cf7fe640 1 --2- 172.21.15.52:0/1200357950 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f81b80a3d20 0x7f81b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:03:59.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.825+0000 7f81cd7fa640 1 -- 172.21.15.52:0/1200357950 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f81c40cfeb0 con 0x7f81d006a1b0 2024-04-02T14:03:59.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.825+0000 7f81cf7fe640 1 --2- 172.21.15.52:0/1200357950 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f81b80a3d20 0x7f81b80a61e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f81d00780c0 tx=0x7f81c002cf90 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:03:59.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:03:59.829+0000 7f81cd7fa640 1 -- 172.21.15.52:0/1200357950 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f81c409c2d0 con 0x7f81d006a1b0 2024-04-02T14:03:59.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:59 smithi088 ceph-mon[36528]: pgmap v325: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:03:59.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:59 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/705133796' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:03:59.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:59 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3051988091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:03:59.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:59 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2386637853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:03:59.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:03:59 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2356580023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:04:00.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:00.066+0000 7f81d6689640 1 -- 172.21.15.52:0/1200357950 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f81d0109a10 con 0x7f81d006a1b0 2024-04-02T14:04:00.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:00.074+0000 7f81cd7fa640 1 -- 172.21.15.52:0/1200357950 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746059 (secure 0 0 0) 0x7f81c409bbd0 con 0x7f81d006a1b0 2024-04-02T14:04:00.090 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T14:04:00.091 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:04:00.091 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 26, 2024-04-02T14:04:00.091 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.091 INFO:teuthology.orchestra.run.smithi052.stdout: "active_gid": 25160, 2024-04-02T14:04:00.091 INFO:teuthology.orchestra.run.smithi052.stdout: "active_name": "x", 2024-04-02T14:04:00.091 INFO:teuthology.orchestra.run.smithi052.stdout: "active_addrs": { 2024-04-02T14:04:00.091 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2024-04-02T14:04:00.091 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.092 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2024-04-02T14:04:00.092 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:6800", 2024-04-02T14:04:00.092 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 1019901931 2024-04-02T14:04:00.092 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.092 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.092 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-04-02T14:04:00.092 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:6801", 2024-04-02T14:04:00.092 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 1019901931 2024-04-02T14:04:00.092 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.093 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-04-02T14:04:00.093 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.093 INFO:teuthology.orchestra.run.smithi052.stdout: "active_addr": "172.21.15.52:6801/1019901931", 2024-04-02T14:04:00.093 INFO:teuthology.orchestra.run.smithi052.stdout: "active_change": "2024-04-02T13:53:29.889376+0000", 2024-04-02T14:04:00.093 INFO:teuthology.orchestra.run.smithi052.stdout: "active_mgr_features": 4540701547738038271, 2024-04-02T14:04:00.094 INFO:teuthology.orchestra.run.smithi052.stdout: "available": true, 2024-04-02T14:04:00.094 INFO:teuthology.orchestra.run.smithi052.stdout: "standbys": [ 2024-04-02T14:04:00.094 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.094 INFO:teuthology.orchestra.run.smithi052.stdout: "gid": 25087, 2024-04-02T14:04:00.094 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "y", 2024-04-02T14:04:00.094 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr_features": 4540701547738038271, 2024-04-02T14:04:00.094 INFO:teuthology.orchestra.run.smithi052.stdout: "available_modules": [ 2024-04-02T14:04:00.094 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.095 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "alerts", 2024-04-02T14:04:00.095 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.095 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.095 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.095 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-04-02T14:04:00.095 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-04-02T14:04:00.095 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:00.095 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.096 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.096 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-04-02T14:04:00.096 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.096 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.097 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.097 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How frequently to reexamine health status", 2024-04-02T14:04:00.097 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.097 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.097 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.097 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.097 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.097 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.097 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.098 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.098 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.098 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.098 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.098 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.098 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.098 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.099 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.099 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.099 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.099 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.099 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.099 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.099 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.099 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.099 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.100 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.100 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.100 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.100 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.100 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.100 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.100 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.101 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.101 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.101 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.101 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.101 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.101 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.101 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.101 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.102 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.102 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.102 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.102 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.102 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.102 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.102 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.102 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.103 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.103 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.103 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.103 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.103 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.103 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.103 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.103 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.104 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.104 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.104 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.104 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.104 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.104 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.105 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.105 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.105 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.105 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.105 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.105 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.105 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.105 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.106 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.106 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.106 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.106 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.106 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.107 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.107 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_destination": { 2024-04-02T14:04:00.107 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_destination", 2024-04-02T14:04:00.107 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.107 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.107 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.107 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.108 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.108 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.108 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.108 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email address to send alerts to", 2024-04-02T14:04:00.108 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.108 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.108 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.108 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.109 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_from_name": { 2024-04-02T14:04:00.109 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_from_name", 2024-04-02T14:04:00.109 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.109 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.109 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.109 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "Ceph", 2024-04-02T14:04:00.109 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.109 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.109 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.110 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email From: name", 2024-04-02T14:04:00.110 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.110 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.110 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.110 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.110 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_host": { 2024-04-02T14:04:00.110 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_host", 2024-04-02T14:04:00.111 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.111 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.111 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.111 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.111 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.111 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.111 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.111 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP server", 2024-04-02T14:04:00.112 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.112 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.112 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.112 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.112 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_password": { 2024-04-02T14:04:00.112 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_password", 2024-04-02T14:04:00.112 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.113 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.113 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.113 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.113 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.113 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.113 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.113 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Password to authenticate with", 2024-04-02T14:04:00.113 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.113 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.114 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.114 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.114 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_port": { 2024-04-02T14:04:00.114 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_port", 2024-04-02T14:04:00.114 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.115 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.115 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.115 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "465", 2024-04-02T14:04:00.115 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.115 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.115 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.115 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP port", 2024-04-02T14:04:00.115 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.116 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.116 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.116 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.116 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_sender": { 2024-04-02T14:04:00.116 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_sender", 2024-04-02T14:04:00.116 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.116 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.116 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.117 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.117 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.117 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.117 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.117 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP envelope sender", 2024-04-02T14:04:00.117 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.118 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.118 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.118 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.118 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_ssl": { 2024-04-02T14:04:00.118 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_ssl", 2024-04-02T14:04:00.118 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.119 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.119 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.119 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.119 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.119 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.119 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.119 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-02T14:04:00.120 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.120 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.120 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.120 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.120 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_user": { 2024-04-02T14:04:00.120 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_user", 2024-04-02T14:04:00.120 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.120 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.121 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.121 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.121 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.121 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.121 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.121 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "User to authenticate as", 2024-04-02T14:04:00.121 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.121 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.121 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.122 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.122 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.122 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.122 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.122 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "balancer", 2024-04-02T14:04:00.122 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.122 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.122 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.123 INFO:teuthology.orchestra.run.smithi052.stdout: "active": { 2024-04-02T14:04:00.123 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "active", 2024-04-02T14:04:00.123 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.123 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.123 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.123 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.123 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.123 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.123 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.124 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically balance PGs across cluster", 2024-04-02T14:04:00.124 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.124 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.124 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.124 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.124 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_time": { 2024-04-02T14:04:00.124 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_time", 2024-04-02T14:04:00.124 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.125 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.125 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.125 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0000", 2024-04-02T14:04:00.125 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.125 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.126 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.126 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "beginning time of day to automatically balance", 2024-04-02T14:04:00.126 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T14:04:00.126 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.126 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.126 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.126 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_weekday": { 2024-04-02T14:04:00.127 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_weekday", 2024-04-02T14:04:00.127 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-04-02T14:04:00.127 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.127 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.127 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-04-02T14:04:00.127 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2024-04-02T14:04:00.128 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2024-04-02T14:04:00.128 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.128 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-02T14:04:00.128 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T14:04:00.128 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.128 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.128 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.129 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_max_iterations": { 2024-04-02T14:04:00.129 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_max_iterations", 2024-04-02T14:04:00.129 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-04-02T14:04:00.129 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.129 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.129 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "25", 2024-04-02T14:04:00.129 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-04-02T14:04:00.129 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "250", 2024-04-02T14:04:00.130 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.130 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-02T14:04:00.130 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.130 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.130 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.130 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.130 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_metrics": { 2024-04-02T14:04:00.130 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_metrics", 2024-04-02T14:04:00.131 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.131 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.131 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.131 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "pgs,objects,bytes", 2024-04-02T14:04:00.131 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.131 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.131 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.131 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-02T14:04:00.131 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-02T14:04:00.132 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.132 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.132 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.132 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_step": { 2024-04-02T14:04:00.132 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_step", 2024-04-02T14:04:00.132 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-04-02T14:04:00.132 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.132 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.133 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.5", 2024-04-02T14:04:00.133 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0.001", 2024-04-02T14:04:00.133 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "0.999", 2024-04-02T14:04:00.133 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.133 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "aggressiveness of optimization", 2024-04-02T14:04:00.133 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-02T14:04:00.133 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.133 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.134 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.134 INFO:teuthology.orchestra.run.smithi052.stdout: "end_time": { 2024-04-02T14:04:00.134 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_time", 2024-04-02T14:04:00.134 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.134 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.134 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.134 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2359", 2024-04-02T14:04:00.134 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.135 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.135 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.135 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ending time of day to automatically balance", 2024-04-02T14:04:00.135 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T14:04:00.135 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.135 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.136 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.136 INFO:teuthology.orchestra.run.smithi052.stdout: "end_weekday": { 2024-04-02T14:04:00.136 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_weekday", 2024-04-02T14:04:00.136 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-04-02T14:04:00.136 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.136 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.136 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-04-02T14:04:00.136 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2024-04-02T14:04:00.137 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2024-04-02T14:04:00.137 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.137 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-02T14:04:00.137 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T14:04:00.137 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.137 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.137 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.137 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.138 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.138 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.138 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.138 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.138 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.138 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.139 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.139 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.139 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.139 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.139 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.139 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.139 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.139 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.140 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.140 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.140 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.140 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.140 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.140 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.140 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.140 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.141 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.141 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.141 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.141 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.141 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.141 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.141 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.142 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.142 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.142 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.142 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.142 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.142 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.142 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.143 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.143 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.143 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.143 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.143 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.143 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.143 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.143 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.144 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.144 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.144 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.144 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.144 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.144 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.144 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.144 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.144 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.145 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.145 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.145 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.145 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.145 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.145 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.145 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.145 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.146 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.146 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.146 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.146 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.146 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.147 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.147 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.147 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.147 INFO:teuthology.orchestra.run.smithi052.stdout: "min_score": { 2024-04-02T14:04:00.147 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_score", 2024-04-02T14:04:00.147 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-04-02T14:04:00.147 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.147 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.147 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-04-02T14:04:00.148 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.148 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.148 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.148 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-02T14:04:00.148 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.148 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.148 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.149 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.149 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2024-04-02T14:04:00.149 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2024-04-02T14:04:00.149 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.150 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.150 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.150 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "upmap", 2024-04-02T14:04:00.150 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.150 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.150 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.150 INFO:teuthology.orchestra.run.smithi052.stdout: "crush-compat", 2024-04-02T14:04:00.150 INFO:teuthology.orchestra.run.smithi052.stdout: "none", 2024-04-02T14:04:00.151 INFO:teuthology.orchestra.run.smithi052.stdout: "read", 2024-04-02T14:04:00.151 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap", 2024-04-02T14:04:00.151 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap-read" 2024-04-02T14:04:00.151 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.151 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Balancer mode", 2024-04-02T14:04:00.151 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.151 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.151 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.152 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.152 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_ids": { 2024-04-02T14:04:00.152 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_ids", 2024-04-02T14:04:00.152 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.152 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.152 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.152 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.152 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.152 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.153 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.153 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-02T14:04:00.153 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.153 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.153 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.154 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.154 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-04-02T14:04:00.154 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-04-02T14:04:00.154 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:00.154 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.154 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.154 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-04-02T14:04:00.155 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.155 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.155 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.155 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-02T14:04:00.155 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.155 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.155 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.155 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.156 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_deviation": { 2024-04-02T14:04:00.156 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_deviation", 2024-04-02T14:04:00.156 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.156 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.156 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.156 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-04-02T14:04:00.156 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-04-02T14:04:00.157 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.157 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.157 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-02T14:04:00.157 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-02T14:04:00.157 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.157 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.157 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.158 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_optimizations": { 2024-04-02T14:04:00.158 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_optimizations", 2024-04-02T14:04:00.158 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-04-02T14:04:00.158 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.158 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.158 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-04-02T14:04:00.158 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.158 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.159 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.159 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-02T14:04:00.159 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.159 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.159 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.159 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.159 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.159 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.159 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.160 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm", 2024-04-02T14:04:00.160 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.160 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.160 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.160 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_down_multiplier": { 2024-04-02T14:04:00.160 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_down_multiplier", 2024-04-02T14:04:00.161 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-04-02T14:04:00.161 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.161 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.161 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2024-04-02T14:04:00.161 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.161 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.161 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.161 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-02T14:04:00.161 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.162 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.162 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.162 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.162 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_refresh_rate": { 2024-04-02T14:04:00.162 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_refresh_rate", 2024-04-02T14:04:00.162 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:00.162 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.162 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.163 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "20", 2024-04-02T14:04:00.163 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.163 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.163 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.163 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-02T14:04:00.163 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.163 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.163 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.164 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.164 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_starting_port": { 2024-04-02T14:04:00.164 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_starting_port", 2024-04-02T14:04:00.164 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.164 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.164 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.164 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4721", 2024-04-02T14:04:00.164 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.164 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.165 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.165 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-02T14:04:00.165 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.165 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.165 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.165 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.165 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_ptrace": { 2024-04-02T14:04:00.166 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_ptrace", 2024-04-02T14:04:00.166 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.166 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.166 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.166 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.166 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.166 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.166 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.167 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-02T14:04:00.167 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-02T14:04:00.167 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.167 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.167 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.168 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_interval": { 2024-04-02T14:04:00.168 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_interval", 2024-04-02T14:04:00.168 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:00.168 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.168 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.168 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-04-02T14:04:00.168 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.169 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.169 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.169 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-02T14:04:00.169 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.169 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.169 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.169 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.169 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_memory_target_ratio": { 2024-04-02T14:04:00.170 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_memory_target_ratio", 2024-04-02T14:04:00.170 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-04-02T14:04:00.170 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.170 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.170 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.7", 2024-04-02T14:04:00.170 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.171 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.171 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.171 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-02T14:04:00.171 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.171 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.171 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.172 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.172 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm_log_destination": { 2024-04-02T14:04:00.172 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm_log_destination", 2024-04-02T14:04:00.172 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.172 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.172 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.172 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.172 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.173 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.173 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.173 INFO:teuthology.orchestra.run.smithi052.stdout: "file", 2024-04-02T14:04:00.173 INFO:teuthology.orchestra.run.smithi052.stdout: "file,syslog", 2024-04-02T14:04:00.173 INFO:teuthology.orchestra.run.smithi052.stdout: "syslog" 2024-04-02T14:04:00.173 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.173 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-02T14:04:00.173 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.174 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.174 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.174 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.174 INFO:teuthology.orchestra.run.smithi052.stdout: "cgroups_split": { 2024-04-02T14:04:00.174 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cgroups_split", 2024-04-02T14:04:00.174 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.174 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.174 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.175 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.175 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.175 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.175 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.175 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-02T14:04:00.175 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.175 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.175 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.176 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.176 INFO:teuthology.orchestra.run.smithi052.stdout: "config_checks_enabled": { 2024-04-02T14:04:00.176 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_checks_enabled", 2024-04-02T14:04:00.176 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.176 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.176 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.176 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.177 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.177 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.177 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.177 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-02T14:04:00.177 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.178 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.178 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.178 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.178 INFO:teuthology.orchestra.run.smithi052.stdout: "config_dashboard": { 2024-04-02T14:04:00.178 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_dashboard", 2024-04-02T14:04:00.178 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.179 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.179 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.179 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.179 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.179 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.179 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.179 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-02T14:04:00.179 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.180 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.180 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.180 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.180 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_alertmanager": { 2024-04-02T14:04:00.180 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_alertmanager", 2024-04-02T14:04:00.180 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.181 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.181 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.181 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-02T14:04:00.181 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.181 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.181 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.181 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-04-02T14:04:00.182 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.182 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.182 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.182 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.182 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_base": { 2024-04-02T14:04:00.182 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_base", 2024-04-02T14:04:00.183 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.183 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.183 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.183 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-02T14:04:00.183 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.183 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.183 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.183 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Container image name, without the tag", 2024-04-02T14:04:00.183 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.184 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.184 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.184 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.184 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_elasticsearch": { 2024-04-02T14:04:00.184 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_elasticsearch", 2024-04-02T14:04:00.184 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.184 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.184 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.185 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-02T14:04:00.185 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.185 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.185 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.185 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "elasticsearch container image", 2024-04-02T14:04:00.185 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.186 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.186 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.186 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.186 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_grafana": { 2024-04-02T14:04:00.186 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_grafana", 2024-04-02T14:04:00.186 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.186 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.187 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.187 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-02T14:04:00.187 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.187 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.187 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.187 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-04-02T14:04:00.187 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.187 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.188 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.188 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.188 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_haproxy": { 2024-04-02T14:04:00.188 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_haproxy", 2024-04-02T14:04:00.189 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.189 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.189 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.189 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-02T14:04:00.189 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.189 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.189 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.190 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "HAproxy container image", 2024-04-02T14:04:00.190 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.190 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.190 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.190 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.190 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_agent": { 2024-04-02T14:04:00.190 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_agent", 2024-04-02T14:04:00.190 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.191 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.191 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.191 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-02T14:04:00.191 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.191 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.192 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.192 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger agent container image", 2024-04-02T14:04:00.192 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.192 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.192 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.192 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.192 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_collector": { 2024-04-02T14:04:00.192 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_collector", 2024-04-02T14:04:00.192 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.193 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.193 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.193 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-02T14:04:00.193 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.193 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.193 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.193 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger collector container image", 2024-04-02T14:04:00.193 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.194 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.194 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.194 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.194 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_query": { 2024-04-02T14:04:00.194 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_query", 2024-04-02T14:04:00.194 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.194 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.195 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.195 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-02T14:04:00.195 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.195 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.195 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.195 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger query container image", 2024-04-02T14:04:00.195 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.195 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.196 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.196 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.196 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_keepalived": { 2024-04-02T14:04:00.196 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_keepalived", 2024-04-02T14:04:00.196 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.196 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.196 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.197 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-02T14:04:00.197 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.197 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.197 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.197 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Keepalived container image", 2024-04-02T14:04:00.197 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.197 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.197 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.198 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.198 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_loki": { 2024-04-02T14:04:00.198 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_loki", 2024-04-02T14:04:00.198 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.198 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.199 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.199 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-02T14:04:00.199 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.199 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.199 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.199 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Loki container image", 2024-04-02T14:04:00.199 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.199 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.200 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.200 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.200 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_node_exporter": { 2024-04-02T14:04:00.200 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_node_exporter", 2024-04-02T14:04:00.200 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.200 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.200 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.201 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-02T14:04:00.201 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.201 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.201 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.201 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-04-02T14:04:00.201 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.201 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.201 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.202 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.202 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_nvmeof": { 2024-04-02T14:04:00.202 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_nvmeof", 2024-04-02T14:04:00.202 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.202 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.202 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.203 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-04-02T14:04:00.203 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.203 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.203 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.203 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Nvme-of container image", 2024-04-02T14:04:00.203 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.203 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.203 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.204 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.204 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_prometheus": { 2024-04-02T14:04:00.204 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_prometheus", 2024-04-02T14:04:00.204 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.204 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.204 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.204 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-02T14:04:00.204 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.205 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.205 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.205 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-04-02T14:04:00.205 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.205 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.205 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.205 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.205 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_promtail": { 2024-04-02T14:04:00.205 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_promtail", 2024-04-02T14:04:00.206 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.206 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.206 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.206 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-02T14:04:00.206 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.206 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.207 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.207 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Promtail container image", 2024-04-02T14:04:00.207 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.207 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.207 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.207 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.208 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_samba": { 2024-04-02T14:04:00.208 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_samba", 2024-04-02T14:04:00.208 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.208 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.208 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.208 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-02T14:04:00.208 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.208 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.209 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.209 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Samba/SMB container image", 2024-04-02T14:04:00.209 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.209 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.209 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.209 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.209 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_snmp_gateway": { 2024-04-02T14:04:00.209 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_snmp_gateway", 2024-04-02T14:04:00.210 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.210 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.210 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.210 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-02T14:04:00.210 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.210 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.211 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.211 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SNMP Gateway container image", 2024-04-02T14:04:00.211 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.211 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.211 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.211 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.211 INFO:teuthology.orchestra.run.smithi052.stdout: "container_init": { 2024-04-02T14:04:00.212 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_init", 2024-04-02T14:04:00.212 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.212 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.212 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.212 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.213 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.213 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.213 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.213 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Run podman/docker with `--init`", 2024-04-02T14:04:00.213 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.213 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.213 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.214 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.214 INFO:teuthology.orchestra.run.smithi052.stdout: "daemon_cache_timeout": { 2024-04-02T14:04:00.214 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "daemon_cache_timeout", 2024-04-02T14:04:00.214 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:00.214 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.214 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.214 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-04-02T14:04:00.214 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.215 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.215 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.215 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-02T14:04:00.215 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.215 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.215 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.215 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.215 INFO:teuthology.orchestra.run.smithi052.stdout: "default_cephadm_command_timeout": { 2024-04-02T14:04:00.216 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_cephadm_command_timeout", 2024-04-02T14:04:00.216 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.216 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.216 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.216 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "900", 2024-04-02T14:04:00.216 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.216 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.216 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.217 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-02T14:04:00.217 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.217 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.217 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.217 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.217 INFO:teuthology.orchestra.run.smithi052.stdout: "default_registry": { 2024-04-02T14:04:00.217 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_registry", 2024-04-02T14:04:00.217 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.218 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.218 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.218 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io", 2024-04-02T14:04:00.218 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.218 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.218 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.218 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-02T14:04:00.218 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.219 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.219 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.219 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.219 INFO:teuthology.orchestra.run.smithi052.stdout: "device_cache_timeout": { 2024-04-02T14:04:00.219 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_cache_timeout", 2024-04-02T14:04:00.219 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:00.219 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.219 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.219 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1800", 2024-04-02T14:04:00.220 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.220 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.220 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.220 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache device inventory", 2024-04-02T14:04:00.220 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.220 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.220 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.220 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.220 INFO:teuthology.orchestra.run.smithi052.stdout: "device_enhanced_scan": { 2024-04-02T14:04:00.221 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_enhanced_scan", 2024-04-02T14:04:00.221 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.221 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.221 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.221 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.221 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.221 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.221 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.221 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-02T14:04:00.222 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.222 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.222 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.222 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.222 INFO:teuthology.orchestra.run.smithi052.stdout: "facts_cache_timeout": { 2024-04-02T14:04:00.222 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "facts_cache_timeout", 2024-04-02T14:04:00.222 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:00.223 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.223 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.223 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-04-02T14:04:00.223 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.223 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.223 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.223 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache host facts data", 2024-04-02T14:04:00.223 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.224 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.224 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.224 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.224 INFO:teuthology.orchestra.run.smithi052.stdout: "grafana_dashboards_path": { 2024-04-02T14:04:00.224 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "grafana_dashboards_path", 2024-04-02T14:04:00.224 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.224 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.224 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.224 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-02T14:04:00.225 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.225 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.225 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.225 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-02T14:04:00.225 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.225 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.225 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.225 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.225 INFO:teuthology.orchestra.run.smithi052.stdout: "host_check_interval": { 2024-04-02T14:04:00.226 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "host_check_interval", 2024-04-02T14:04:00.226 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:00.226 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.226 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.226 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-04-02T14:04:00.226 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.227 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.227 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.227 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to perform a host check", 2024-04-02T14:04:00.227 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.227 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.227 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.227 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.227 INFO:teuthology.orchestra.run.smithi052.stdout: "hw_monitoring": { 2024-04-02T14:04:00.228 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hw_monitoring", 2024-04-02T14:04:00.228 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.228 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.228 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.228 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.228 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.228 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.229 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.229 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-02T14:04:00.229 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.229 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.229 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.229 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.229 INFO:teuthology.orchestra.run.smithi052.stdout: "inventory_list_all": { 2024-04-02T14:04:00.229 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "inventory_list_all", 2024-04-02T14:04:00.230 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.230 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.230 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.230 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.230 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.230 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.230 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.230 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-02T14:04:00.231 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.231 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.231 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.231 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.231 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.231 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.231 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.232 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.232 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.232 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.232 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.232 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.232 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.232 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.232 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.233 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.233 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.233 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.233 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.233 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.233 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.234 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.234 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.234 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.234 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.234 INFO:teuthology.orchestra.run.smithi052.stdout: "log_refresh_metadata": { 2024-04-02T14:04:00.234 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_refresh_metadata", 2024-04-02T14:04:00.234 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.235 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.235 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.235 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.235 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.235 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.235 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.235 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-02T14:04:00.236 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.236 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.236 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.236 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.236 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.236 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.236 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.236 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.237 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.237 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.237 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.237 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.237 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.237 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-02T14:04:00.237 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.237 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.238 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.238 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.238 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.238 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.238 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.238 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.238 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.238 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.239 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.239 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.239 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.239 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.239 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.239 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.239 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.239 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.239 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.240 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.240 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.240 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.240 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.240 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.240 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.240 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.240 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.241 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.241 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.241 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.241 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.241 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.241 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.241 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.241 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.242 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.242 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.242 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.242 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.242 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-02T14:04:00.242 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-02T14:04:00.242 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.242 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.243 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.243 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.243 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.243 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.243 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.243 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-02T14:04:00.244 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.244 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.244 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.244 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.244 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-02T14:04:00.244 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-02T14:04:00.244 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.245 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.245 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.245 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "*", 2024-04-02T14:04:00.245 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.245 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.245 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.245 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-02T14:04:00.245 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.246 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.246 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.246 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.246 INFO:teuthology.orchestra.run.smithi052.stdout: "max_count_per_host": { 2024-04-02T14:04:00.246 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_count_per_host", 2024-04-02T14:04:00.246 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.246 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.246 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.247 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-04-02T14:04:00.247 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.247 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.247 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.247 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of daemons per service per host", 2024-04-02T14:04:00.247 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.247 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.247 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.248 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.248 INFO:teuthology.orchestra.run.smithi052.stdout: "max_osd_draining_count": { 2024-04-02T14:04:00.248 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_osd_draining_count", 2024-04-02T14:04:00.248 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.248 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.248 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.248 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-04-02T14:04:00.248 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.249 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.249 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.249 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-02T14:04:00.249 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.249 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.249 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.249 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.250 INFO:teuthology.orchestra.run.smithi052.stdout: "migration_current": { 2024-04-02T14:04:00.250 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "migration_current", 2024-04-02T14:04:00.250 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.250 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.250 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.250 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.250 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.250 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.251 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.251 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "internal - do not modify", 2024-04-02T14:04:00.251 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.251 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.251 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.251 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.251 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2024-04-02T14:04:00.251 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2024-04-02T14:04:00.251 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.252 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.252 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.252 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "root", 2024-04-02T14:04:00.252 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.252 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.252 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.252 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm-package", 2024-04-02T14:04:00.252 INFO:teuthology.orchestra.run.smithi052.stdout: "root" 2024-04-02T14:04:00.253 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.253 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "mode for remote execution of cephadm", 2024-04-02T14:04:00.253 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.253 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.253 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.253 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.253 INFO:teuthology.orchestra.run.smithi052.stdout: "oob_default_addr": { 2024-04-02T14:04:00.253 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "oob_default_addr", 2024-04-02T14:04:00.253 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.254 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.254 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.254 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "169.254.1.1", 2024-04-02T14:04:00.254 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.255 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.255 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.255 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-02T14:04:00.255 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.255 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.255 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.255 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.255 INFO:teuthology.orchestra.run.smithi052.stdout: "prometheus_alerts_path": { 2024-04-02T14:04:00.256 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus_alerts_path", 2024-04-02T14:04:00.256 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.256 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.256 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.256 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-02T14:04:00.256 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.256 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.256 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.256 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-02T14:04:00.257 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.257 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.257 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.257 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.257 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_insecure": { 2024-04-02T14:04:00.257 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_insecure", 2024-04-02T14:04:00.257 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.258 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.258 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.258 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.258 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.258 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.258 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.258 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-02T14:04:00.258 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.259 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.259 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.259 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.259 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_password": { 2024-04-02T14:04:00.259 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_password", 2024-04-02T14:04:00.259 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.259 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.259 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.259 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.260 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.260 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.260 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.260 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-02T14:04:00.260 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.260 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.260 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.260 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.261 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_url": { 2024-04-02T14:04:00.261 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_url", 2024-04-02T14:04:00.261 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.261 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.261 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.261 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.261 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.261 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.262 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.262 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-02T14:04:00.262 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.262 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.262 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.262 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.262 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_username": { 2024-04-02T14:04:00.262 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_username", 2024-04-02T14:04:00.263 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.263 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.263 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.263 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.263 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.263 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.263 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.263 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-02T14:04:00.264 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.264 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.264 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.264 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.264 INFO:teuthology.orchestra.run.smithi052.stdout: "secure_monitoring_stack": { 2024-04-02T14:04:00.264 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "secure_monitoring_stack", 2024-04-02T14:04:00.264 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.264 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.265 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.265 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.265 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.265 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.265 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.265 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-02T14:04:00.265 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.265 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.265 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.266 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.266 INFO:teuthology.orchestra.run.smithi052.stdout: "service_discovery_port": { 2024-04-02T14:04:00.266 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "service_discovery_port", 2024-04-02T14:04:00.266 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.266 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.266 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.266 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8765", 2024-04-02T14:04:00.266 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.267 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.267 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.267 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "cephadm service discovery port", 2024-04-02T14:04:00.267 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.267 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.267 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.267 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.267 INFO:teuthology.orchestra.run.smithi052.stdout: "ssh_config_file": { 2024-04-02T14:04:00.268 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssh_config_file", 2024-04-02T14:04:00.268 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.268 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.268 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.268 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.268 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.268 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.268 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.269 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-02T14:04:00.269 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.269 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.269 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.269 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.269 INFO:teuthology.orchestra.run.smithi052.stdout: "use_agent": { 2024-04-02T14:04:00.269 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_agent", 2024-04-02T14:04:00.269 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.270 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.270 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.270 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.270 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.270 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.270 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.270 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-02T14:04:00.270 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.271 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.271 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.271 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.271 INFO:teuthology.orchestra.run.smithi052.stdout: "use_repo_digest": { 2024-04-02T14:04:00.272 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_repo_digest", 2024-04-02T14:04:00.272 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.272 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.272 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.272 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.272 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.273 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.273 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.273 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-02T14:04:00.273 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.273 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.273 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.273 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.273 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_failed_host_check": { 2024-04-02T14:04:00.274 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_failed_host_check", 2024-04-02T14:04:00.274 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.274 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.274 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.274 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.274 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.274 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.274 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.275 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if the host check fails", 2024-04-02T14:04:00.275 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.275 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.275 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.275 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.275 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_daemons": { 2024-04-02T14:04:00.275 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_daemons", 2024-04-02T14:04:00.275 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.276 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.276 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.276 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.276 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.276 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.276 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.276 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-02T14:04:00.276 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.277 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.277 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.277 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.277 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_hosts": { 2024-04-02T14:04:00.277 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_hosts", 2024-04-02T14:04:00.277 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.277 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.277 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.277 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.278 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.278 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.278 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.278 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-02T14:04:00.278 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.278 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.278 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.279 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.279 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.279 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.279 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.279 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crash", 2024-04-02T14:04:00.279 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.280 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.280 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.280 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.280 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.280 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.280 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.280 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.281 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.281 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.281 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.281 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.281 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.281 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.281 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.281 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.281 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.282 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.282 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.282 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.282 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.282 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.282 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.282 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.283 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.283 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.283 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.283 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.283 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.283 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.284 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.284 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.284 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.284 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.284 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.284 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.284 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.284 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.285 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.285 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.285 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.285 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.285 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.285 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.285 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.285 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.285 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.286 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.286 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.286 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.286 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.286 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.286 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.286 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.286 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.287 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.287 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.287 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.287 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.287 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.287 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.287 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.287 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.287 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.288 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.288 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.288 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.288 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.288 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.288 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.288 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.288 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.288 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.289 INFO:teuthology.orchestra.run.smithi052.stdout: "retain_interval": { 2024-04-02T14:04:00.289 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retain_interval", 2024-04-02T14:04:00.289 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:00.289 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.289 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.289 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "31536000", 2024-04-02T14:04:00.289 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.289 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.290 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.290 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-02T14:04:00.290 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.290 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.290 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.290 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.290 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_recent_interval": { 2024-04-02T14:04:00.290 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_recent_interval", 2024-04-02T14:04:00.291 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:00.291 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.291 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.291 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1209600", 2024-04-02T14:04:00.291 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.291 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.291 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.291 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-02T14:04:00.292 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.292 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.292 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.292 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.292 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.292 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.292 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.292 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dashboard", 2024-04-02T14:04:00.293 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.293 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.293 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.293 INFO:teuthology.orchestra.run.smithi052.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-02T14:04:00.293 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-02T14:04:00.294 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.294 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.294 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.294 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-04-02T14:04:00.295 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.295 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.295 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.295 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.295 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.295 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.295 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.295 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.295 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_HOST": { 2024-04-02T14:04:00.296 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-02T14:04:00.296 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.296 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.296 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.296 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.296 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.296 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.296 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.297 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.297 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.297 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.297 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.297 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.297 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-02T14:04:00.298 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-02T14:04:00.298 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.298 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.299 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.299 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.299 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.299 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.300 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.300 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.300 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.301 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.301 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.301 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.301 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_ENABLED": { 2024-04-02T14:04:00.301 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_ENABLED", 2024-04-02T14:04:00.301 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.301 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.302 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.302 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.302 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.302 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.302 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.302 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.302 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.303 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.303 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.303 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.303 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-02T14:04:00.303 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-02T14:04:00.303 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.303 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.304 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.304 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.304 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.304 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.304 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.304 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.304 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.304 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.305 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.305 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.305 INFO:teuthology.orchestra.run.smithi052.stdout: "ENABLE_BROWSABLE_API": { 2024-04-02T14:04:00.305 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-02T14:04:00.305 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.305 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.305 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.305 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.305 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.306 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.306 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.306 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.306 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.306 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.306 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.306 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.307 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-02T14:04:00.307 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-02T14:04:00.307 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.307 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.307 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.307 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.308 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.308 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.308 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.308 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.308 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.308 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.308 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.308 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.309 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-02T14:04:00.309 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-02T14:04:00.309 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.309 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.309 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.309 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.309 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.309 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.310 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.310 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.310 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.310 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.310 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.310 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.310 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-02T14:04:00.310 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-02T14:04:00.311 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.311 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.311 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.311 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.311 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.311 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.311 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.311 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.311 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.312 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.312 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.312 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.312 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-02T14:04:00.312 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-02T14:04:00.312 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.312 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.313 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.313 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.313 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.313 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.313 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.313 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.314 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.314 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.314 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.314 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.314 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-02T14:04:00.314 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-02T14:04:00.314 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.314 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.315 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.315 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.315 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.315 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.315 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.315 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.315 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.315 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.316 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.316 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.316 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-02T14:04:00.316 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-02T14:04:00.316 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.316 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.316 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.316 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.316 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.317 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.317 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.317 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.317 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.317 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.317 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.318 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.318 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-02T14:04:00.318 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-02T14:04:00.319 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.319 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.319 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.319 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.320 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.320 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.320 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.320 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.321 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.321 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.321 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.321 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.321 INFO:teuthology.orchestra.run.smithi052.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-02T14:04:00.321 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-02T14:04:00.321 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.322 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.322 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.322 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.323 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.323 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.323 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.323 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.323 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.323 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.323 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.323 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.324 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_PASSWORD": { 2024-04-02T14:04:00.324 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-02T14:04:00.324 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.324 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.324 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.324 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-04-02T14:04:00.324 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.324 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.325 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.325 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.325 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.325 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.325 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.325 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.325 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-02T14:04:00.325 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-02T14:04:00.326 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.326 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.326 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.326 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.326 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.326 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.326 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.326 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.327 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.327 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.327 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.327 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.327 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_URL": { 2024-04-02T14:04:00.327 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_URL", 2024-04-02T14:04:00.327 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.327 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.328 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.328 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.328 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.328 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.328 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.328 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.328 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.328 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.328 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.329 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.329 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_USERNAME": { 2024-04-02T14:04:00.329 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-02T14:04:00.329 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.329 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.329 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.329 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-04-02T14:04:00.329 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.330 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.330 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.330 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.330 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.330 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.330 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.330 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.330 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-02T14:04:00.331 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-02T14:04:00.331 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.331 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.331 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.331 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.331 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.331 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.331 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.332 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.332 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.332 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.332 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.332 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.332 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-02T14:04:00.332 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-02T14:04:00.332 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.333 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.333 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.333 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.333 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.333 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.333 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.333 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.333 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.334 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.334 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.334 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.334 INFO:teuthology.orchestra.run.smithi052.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-02T14:04:00.334 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-02T14:04:00.334 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.334 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.334 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.335 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.335 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.335 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.335 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.335 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.335 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.335 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.335 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.336 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.336 INFO:teuthology.orchestra.run.smithi052.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-02T14:04:00.336 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-02T14:04:00.336 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.336 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.336 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.336 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.336 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.336 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.337 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.337 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.337 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.337 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.337 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.337 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.337 INFO:teuthology.orchestra.run.smithi052.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-02T14:04:00.337 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-02T14:04:00.337 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.338 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.338 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.338 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "[]", 2024-04-02T14:04:00.338 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.338 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.338 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.338 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.339 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.339 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.339 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.339 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.339 INFO:teuthology.orchestra.run.smithi052.stdout: "MULTICLUSTER_CONFIG": { 2024-04-02T14:04:00.339 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-02T14:04:00.339 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.339 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.340 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.340 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "{}", 2024-04-02T14:04:00.340 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.340 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.340 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.340 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.340 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.340 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.340 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.341 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.341 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_HOST": { 2024-04-02T14:04:00.341 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-02T14:04:00.341 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.341 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.341 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.341 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.341 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.342 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.342 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.342 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.342 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.342 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.342 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.342 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.342 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-02T14:04:00.343 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-02T14:04:00.343 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.343 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.343 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.343 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.343 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.343 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.344 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.344 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.344 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.344 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.344 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.344 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.344 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-02T14:04:00.345 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-02T14:04:00.345 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.345 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.345 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.345 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.345 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.345 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.345 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.345 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.345 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.346 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.346 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.346 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.346 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-02T14:04:00.346 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-02T14:04:00.346 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.347 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.347 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.347 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.347 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.347 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.347 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.347 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.347 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.348 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.348 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.348 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.348 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-02T14:04:00.348 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-02T14:04:00.348 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.348 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.348 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.348 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.349 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.349 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.349 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.349 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.349 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.349 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.349 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.349 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.349 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-02T14:04:00.350 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-02T14:04:00.350 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.350 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.350 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.350 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.350 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.350 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.351 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.351 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.351 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.351 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.351 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.351 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.351 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-02T14:04:00.351 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-02T14:04:00.352 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.352 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.352 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.352 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.352 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.352 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.352 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.352 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.353 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.353 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.353 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.353 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.353 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-02T14:04:00.353 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-02T14:04:00.353 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.353 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.354 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.354 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.354 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.354 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.354 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.354 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.354 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.355 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.355 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.355 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.355 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-02T14:04:00.355 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-02T14:04:00.355 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.356 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.356 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.356 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.356 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.356 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.356 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.356 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.356 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.357 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.357 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.357 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.357 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_ENABLED": { 2024-04-02T14:04:00.357 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-02T14:04:00.357 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.357 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.357 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.357 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.358 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.358 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.358 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.358 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.358 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.358 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.358 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.358 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.359 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-02T14:04:00.359 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-02T14:04:00.359 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.359 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.359 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.359 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-02T14:04:00.359 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.359 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.360 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.360 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.360 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.360 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.360 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.360 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.360 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-02T14:04:00.360 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-02T14:04:00.361 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.361 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.361 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.361 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-04-02T14:04:00.361 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.361 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.361 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.361 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.361 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.362 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.362 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.362 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.362 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-02T14:04:00.362 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-02T14:04:00.362 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.362 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.362 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.363 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8", 2024-04-02T14:04:00.363 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.363 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.363 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.363 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.363 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.363 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.363 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.364 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.364 INFO:teuthology.orchestra.run.smithi052.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-02T14:04:00.364 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-02T14:04:00.364 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.364 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.364 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.364 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "45", 2024-04-02T14:04:00.364 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.364 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.365 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.365 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.365 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.365 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.365 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.365 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.365 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ACCESS_KEY": { 2024-04-02T14:04:00.365 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-02T14:04:00.366 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.366 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.366 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.366 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.366 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.366 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.366 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.367 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.367 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.367 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.367 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.367 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.367 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-02T14:04:00.367 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-02T14:04:00.367 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.367 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.367 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.368 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-04-02T14:04:00.368 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.368 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.368 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.368 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.368 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.368 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.368 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.368 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.369 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SECRET_KEY": { 2024-04-02T14:04:00.369 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-02T14:04:00.369 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.369 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.369 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.369 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.369 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.369 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.369 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.370 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.370 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.370 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.370 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.370 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.370 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SSL_VERIFY": { 2024-04-02T14:04:00.370 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-02T14:04:00.370 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.371 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.371 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.371 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.371 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.371 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.371 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.372 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.372 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.372 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.372 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.373 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.373 INFO:teuthology.orchestra.run.smithi052.stdout: "UNSAFE_TLS_v1_2": { 2024-04-02T14:04:00.373 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-02T14:04:00.373 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.374 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.374 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.374 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.374 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.375 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.375 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.375 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.375 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.376 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.376 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.376 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.376 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-02T14:04:00.377 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-02T14:04:00.377 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.377 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.377 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.378 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-04-02T14:04:00.378 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.378 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.379 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.379 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.379 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.379 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.380 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.380 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.380 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-02T14:04:00.380 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-02T14:04:00.380 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.381 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.381 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.381 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-04-02T14:04:00.381 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.382 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.382 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.382 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.382 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.383 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.383 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.383 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.383 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-02T14:04:00.384 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-02T14:04:00.384 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.384 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.385 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.385 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-04-02T14:04:00.385 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.385 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.386 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.386 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.386 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.387 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.387 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.387 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.387 INFO:teuthology.orchestra.run.smithi052.stdout: "cross_origin_url": { 2024-04-02T14:04:00.388 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cross_origin_url", 2024-04-02T14:04:00.388 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.388 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.388 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.389 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.389 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.389 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.389 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.390 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.390 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.390 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.390 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.391 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.391 INFO:teuthology.orchestra.run.smithi052.stdout: "crt_file": { 2024-04-02T14:04:00.391 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crt_file", 2024-04-02T14:04:00.392 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.392 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.392 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.392 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.393 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.393 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.393 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.393 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.394 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.394 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.394 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.395 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.395 INFO:teuthology.orchestra.run.smithi052.stdout: "debug": { 2024-04-02T14:04:00.395 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "debug", 2024-04-02T14:04:00.395 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.396 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.396 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.396 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.396 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.397 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.397 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.397 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable/disable debug options", 2024-04-02T14:04:00.397 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.398 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.398 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.398 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.398 INFO:teuthology.orchestra.run.smithi052.stdout: "jwt_token_ttl": { 2024-04-02T14:04:00.399 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "jwt_token_ttl", 2024-04-02T14:04:00.399 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.399 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.399 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.400 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "28800", 2024-04-02T14:04:00.400 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.400 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.401 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.401 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.401 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.401 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.402 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.402 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.402 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2024-04-02T14:04:00.402 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2024-04-02T14:04:00.403 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.403 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.403 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.403 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.403 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.404 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.404 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.404 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.404 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.404 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.404 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.405 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.405 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.405 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.405 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.405 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.406 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.406 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.406 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.406 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.407 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.407 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.407 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.407 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.407 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.407 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.407 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.407 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.408 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.408 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.408 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.408 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.408 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.409 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.409 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.409 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.409 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.409 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.410 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.410 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.410 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.410 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.410 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.411 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.411 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.411 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.411 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.411 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.411 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.411 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.411 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.411 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.412 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.412 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.412 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.412 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.412 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.412 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.412 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.413 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.413 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.413 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.413 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.413 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.413 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.413 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.413 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.414 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.414 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.414 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.414 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.417 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.417 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.417 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.417 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.418 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.418 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.418 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.418 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.418 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.418 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.418 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.418 INFO:teuthology.orchestra.run.smithi052.stdout: "motd": { 2024-04-02T14:04:00.419 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "motd", 2024-04-02T14:04:00.419 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.419 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.419 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.419 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.419 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.419 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.419 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "The message of the day", 2024-04-02T14:04:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect_resolve_ip_addr": { 2024-04-02T14:04:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "redirect_resolve_ip_addr", 2024-04-02T14:04:00.420 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.421 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.422 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.422 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.422 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-04-02T14:04:00.422 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-04-02T14:04:00.422 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.422 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.423 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.423 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2024-04-02T14:04:00.423 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.423 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.423 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.423 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.423 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.424 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.424 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.424 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.424 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-04-02T14:04:00.424 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-04-02T14:04:00.424 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.424 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.424 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.425 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8080", 2024-04-02T14:04:00.425 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.425 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.425 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.425 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.425 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.425 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.426 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.426 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.426 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2024-04-02T14:04:00.426 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2024-04-02T14:04:00.426 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.426 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.427 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.427 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.427 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.427 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.427 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.427 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.427 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.427 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.428 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.428 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.428 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl_server_port": { 2024-04-02T14:04:00.428 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl_server_port", 2024-04-02T14:04:00.428 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.428 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.429 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.429 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8443", 2024-04-02T14:04:00.429 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.429 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.429 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.429 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.429 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.429 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.430 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.430 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.430 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2024-04-02T14:04:00.430 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2024-04-02T14:04:00.430 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.431 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.431 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.431 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "redirect", 2024-04-02T14:04:00.431 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.431 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.431 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.431 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.431 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect" 2024-04-02T14:04:00.432 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.432 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.432 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.432 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.432 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.432 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.432 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2024-04-02T14:04:00.432 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2024-04-02T14:04:00.433 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.433 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.433 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.433 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2024-04-02T14:04:00.433 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2024-04-02T14:04:00.433 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2024-04-02T14:04:00.433 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.433 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.433 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.434 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.434 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.434 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.434 INFO:teuthology.orchestra.run.smithi052.stdout: "url_prefix": { 2024-04-02T14:04:00.434 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url_prefix", 2024-04-02T14:04:00.434 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.434 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.435 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.435 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.435 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.435 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.435 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.435 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.435 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.435 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.435 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.436 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.436 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.436 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.436 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.436 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "devicehealth", 2024-04-02T14:04:00.436 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.436 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.436 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.437 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_monitoring": { 2024-04-02T14:04:00.437 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_monitoring", 2024-04-02T14:04:00.437 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.437 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.437 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.437 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.437 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.437 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.438 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.438 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "monitor device health metrics", 2024-04-02T14:04:00.438 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.438 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.438 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.438 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.439 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.439 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.439 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.439 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.439 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.439 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.439 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.440 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.440 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.440 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.440 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.440 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.440 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.440 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.440 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.441 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.441 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.441 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.441 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.441 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.441 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.442 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.442 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.442 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.442 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.442 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.442 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.442 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.443 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.443 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.443 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.443 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.443 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.444 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.444 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.444 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.444 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.445 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.445 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.445 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.445 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.445 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.445 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.446 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.446 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.446 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.446 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.446 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.447 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.447 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.447 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.447 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.448 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.448 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.448 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.448 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.448 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.449 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.449 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.449 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.449 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.449 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.450 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.450 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.450 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.450 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.450 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.451 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.451 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.451 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.451 INFO:teuthology.orchestra.run.smithi052.stdout: "mark_out_threshold": { 2024-04-02T14:04:00.451 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mark_out_threshold", 2024-04-02T14:04:00.452 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:00.452 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.452 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.452 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2419200", 2024-04-02T14:04:00.452 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.452 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.452 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.453 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-02T14:04:00.453 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.453 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.453 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.453 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.453 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_name": { 2024-04-02T14:04:00.453 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_name", 2024-04-02T14:04:00.454 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.454 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.454 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.454 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "device_health_metrics", 2024-04-02T14:04:00.454 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.454 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.454 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.455 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-02T14:04:00.455 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.455 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.455 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.455 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.455 INFO:teuthology.orchestra.run.smithi052.stdout: "retention_period": { 2024-04-02T14:04:00.455 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retention_period", 2024-04-02T14:04:00.456 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:00.456 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.456 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.456 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15552000", 2024-04-02T14:04:00.456 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.456 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.456 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.456 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain device health metrics", 2024-04-02T14:04:00.457 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.457 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.457 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.457 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.457 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_frequency": { 2024-04-02T14:04:00.457 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_frequency", 2024-04-02T14:04:00.457 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:00.457 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.457 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.458 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2024-04-02T14:04:00.458 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.458 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.458 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.458 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-02T14:04:00.458 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.458 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.459 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.459 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.459 INFO:teuthology.orchestra.run.smithi052.stdout: "self_heal": { 2024-04-02T14:04:00.459 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "self_heal", 2024-04-02T14:04:00.459 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.459 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.459 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.459 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.460 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.460 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.460 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.460 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-02T14:04:00.460 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.460 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.460 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.460 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.461 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-04-02T14:04:00.461 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-04-02T14:04:00.461 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:00.461 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.461 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.461 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-04-02T14:04:00.461 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.461 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.462 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.462 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and check device health", 2024-04-02T14:04:00.462 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.462 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.462 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.462 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.463 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_threshold": { 2024-04-02T14:04:00.463 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_threshold", 2024-04-02T14:04:00.463 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:00.463 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.463 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.463 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7257600", 2024-04-02T14:04:00.463 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.463 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.464 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.464 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-02T14:04:00.464 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.464 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.464 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.464 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.464 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.464 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.464 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.465 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "diskprediction_local", 2024-04-02T14:04:00.465 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.465 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.465 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.465 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.465 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.465 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.465 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.466 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.466 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.466 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.466 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.466 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.466 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.466 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.466 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.467 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.467 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.467 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.467 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.467 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.467 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.467 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.467 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.467 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.468 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.468 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.468 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.468 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.468 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.468 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.468 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.468 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.468 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.469 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.469 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.469 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.469 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.469 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.469 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.469 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.469 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.469 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.470 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.470 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.470 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.470 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.470 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.470 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.470 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.470 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.471 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.471 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.471 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.471 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.471 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.471 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.472 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.472 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.472 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.472 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.473 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.473 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.473 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.473 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.473 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.473 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.474 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.474 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.474 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.474 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.474 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.474 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.474 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.475 INFO:teuthology.orchestra.run.smithi052.stdout: "predict_interval": { 2024-04-02T14:04:00.475 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predict_interval", 2024-04-02T14:04:00.475 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.475 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.475 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.475 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2024-04-02T14:04:00.475 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.475 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.475 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.476 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.476 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.476 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.476 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.476 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.476 INFO:teuthology.orchestra.run.smithi052.stdout: "predictor_model": { 2024-04-02T14:04:00.476 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predictor_model", 2024-04-02T14:04:00.476 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.477 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.477 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.477 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "prophetstor", 2024-04-02T14:04:00.477 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.477 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.477 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.477 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.478 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.478 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.478 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.478 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.478 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-04-02T14:04:00.478 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-04-02T14:04:00.478 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.479 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.479 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.479 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-04-02T14:04:00.479 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.479 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.479 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.479 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.479 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.480 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.480 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.480 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.480 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.480 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.480 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.481 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "influx", 2024-04-02T14:04:00.481 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": false, 2024-04-02T14:04:00.481 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "influxdb python module not found", 2024-04-02T14:04:00.481 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.481 INFO:teuthology.orchestra.run.smithi052.stdout: "batch_size": { 2024-04-02T14:04:00.481 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "batch_size", 2024-04-02T14:04:00.482 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.482 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.482 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.482 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5000", 2024-04-02T14:04:00.482 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.482 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.482 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.482 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-02T14:04:00.483 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.483 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.483 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.483 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.483 INFO:teuthology.orchestra.run.smithi052.stdout: "database": { 2024-04-02T14:04:00.483 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "database", 2024-04-02T14:04:00.483 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.483 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.484 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.484 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "ceph", 2024-04-02T14:04:00.484 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.484 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.484 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.484 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-02T14:04:00.484 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.484 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.485 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.485 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.485 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": { 2024-04-02T14:04:00.485 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hostname", 2024-04-02T14:04:00.485 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.485 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.485 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.485 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.485 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.486 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.486 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.486 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server hostname", 2024-04-02T14:04:00.486 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.486 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.486 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.486 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.487 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-04-02T14:04:00.487 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-04-02T14:04:00.487 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:00.487 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.487 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.487 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "30", 2024-04-02T14:04:00.487 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "5", 2024-04-02T14:04:00.487 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.488 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.488 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-02T14:04:00.488 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.488 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.488 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.488 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.488 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.488 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.488 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.489 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.489 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.489 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.489 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.489 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.489 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.489 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.490 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.490 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.490 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.490 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.490 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.490 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.490 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.490 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.491 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.491 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.491 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.491 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.491 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.492 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.492 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.492 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.492 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.492 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.492 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.492 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.492 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.492 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.493 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.493 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.493 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.493 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.493 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.493 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.493 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.493 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.493 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.494 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.494 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.494 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.494 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.494 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.494 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.494 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.494 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.495 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.495 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.495 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.495 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.495 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.495 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.495 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.495 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.496 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.496 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.496 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.496 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.496 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.496 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.496 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.496 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.497 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.497 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.497 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.497 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.497 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.497 INFO:teuthology.orchestra.run.smithi052.stdout: "password": { 2024-04-02T14:04:00.497 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "password", 2024-04-02T14:04:00.497 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.497 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.498 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.498 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.498 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.498 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.498 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.498 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "password of InfluxDB server user", 2024-04-02T14:04:00.498 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.499 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.499 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.499 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.499 INFO:teuthology.orchestra.run.smithi052.stdout: "port": { 2024-04-02T14:04:00.499 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "port", 2024-04-02T14:04:00.499 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.500 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.500 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.500 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8086", 2024-04-02T14:04:00.500 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.500 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.500 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.500 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server port", 2024-04-02T14:04:00.500 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.501 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.501 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.501 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.501 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2024-04-02T14:04:00.501 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2024-04-02T14:04:00.501 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.501 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.502 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.502 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "false", 2024-04-02T14:04:00.502 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.502 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.502 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.502 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T14:04:00.503 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.503 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.503 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.503 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.503 INFO:teuthology.orchestra.run.smithi052.stdout: "threads": { 2024-04-02T14:04:00.503 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threads", 2024-04-02T14:04:00.503 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.503 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.504 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.504 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-04-02T14:04:00.504 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-04-02T14:04:00.504 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "32", 2024-04-02T14:04:00.504 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.504 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-02T14:04:00.504 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.505 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.505 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.505 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.505 INFO:teuthology.orchestra.run.smithi052.stdout: "username": { 2024-04-02T14:04:00.505 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "username", 2024-04-02T14:04:00.505 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.505 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.505 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.505 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.506 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.506 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.506 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.506 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "username of InfluxDB server user", 2024-04-02T14:04:00.506 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.506 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.506 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.507 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.507 INFO:teuthology.orchestra.run.smithi052.stdout: "verify_ssl": { 2024-04-02T14:04:00.507 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "verify_ssl", 2024-04-02T14:04:00.507 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.507 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.507 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.507 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "true", 2024-04-02T14:04:00.507 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.508 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.508 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.508 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T14:04:00.508 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.508 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.508 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.508 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.508 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.509 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.509 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.509 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "insights", 2024-04-02T14:04:00.509 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.509 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.509 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.509 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.509 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.509 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.510 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.510 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.510 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.510 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.510 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.510 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.510 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.511 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.511 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.511 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.511 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.511 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.511 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.511 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.511 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.512 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.512 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.512 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.512 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.512 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.513 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.513 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.513 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.513 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.513 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.513 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.513 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.513 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.514 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.514 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.514 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.514 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.514 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.514 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.514 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.515 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.515 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.515 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.515 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.515 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.515 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.515 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.515 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.515 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.516 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.516 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.516 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.516 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.516 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.516 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.516 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.516 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.517 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.517 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.517 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.517 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.517 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.517 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.517 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.517 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.517 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.518 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.518 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.518 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.518 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.518 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.518 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.518 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.519 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.519 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.519 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "iostat", 2024-04-02T14:04:00.519 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.519 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.519 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.519 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.519 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.520 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.520 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.520 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.520 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.520 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.520 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.520 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.521 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.521 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.521 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.521 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.522 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.522 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.522 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.522 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.522 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.523 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.523 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.523 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.523 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.523 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.523 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.523 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.524 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.524 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.524 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.524 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.524 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.524 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.524 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.524 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.525 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.525 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.525 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.525 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.525 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.525 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.525 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.525 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.526 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.526 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.526 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.526 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.526 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.526 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.526 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.526 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.527 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.527 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.527 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.527 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.527 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.527 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.527 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.528 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.528 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.528 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.528 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.528 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.528 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.528 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.528 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.529 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.529 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.529 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.529 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.529 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.529 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.529 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.529 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.529 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.530 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "k8sevents", 2024-04-02T14:04:00.530 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.530 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.530 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.530 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_event_retention_days": { 2024-04-02T14:04:00.530 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ceph_event_retention_days", 2024-04-02T14:04:00.530 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.531 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.531 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.531 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7", 2024-04-02T14:04:00.531 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.531 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.531 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.531 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-02T14:04:00.531 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.532 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.532 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.532 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.532 INFO:teuthology.orchestra.run.smithi052.stdout: "config_check_secs": { 2024-04-02T14:04:00.532 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_check_secs", 2024-04-02T14:04:00.532 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.532 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.532 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.533 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-04-02T14:04:00.533 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "10", 2024-04-02T14:04:00.533 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.533 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.533 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-02T14:04:00.534 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.534 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.534 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.534 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.534 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.534 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.534 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.535 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.535 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.535 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.535 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.535 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.535 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.535 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.535 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.536 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.536 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.536 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.536 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.536 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.536 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.536 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.536 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.536 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.537 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.537 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.537 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.537 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.537 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.537 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.537 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.537 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.538 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.538 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.538 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.538 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.538 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.538 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.538 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.538 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.539 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.539 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.539 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.539 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.539 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.539 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.539 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.539 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.540 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.540 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.540 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.540 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.540 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.540 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.540 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.540 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.541 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.541 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.541 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.541 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.541 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.541 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.541 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.541 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.541 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.542 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.542 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.542 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.542 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.542 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.542 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.542 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.542 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.543 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.543 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.543 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.543 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.543 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "localpool", 2024-04-02T14:04:00.543 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.543 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.543 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.544 INFO:teuthology.orchestra.run.smithi052.stdout: "failure_domain": { 2024-04-02T14:04:00.544 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "failure_domain", 2024-04-02T14:04:00.544 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.544 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.544 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.544 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "host", 2024-04-02T14:04:00.545 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.545 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.545 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.545 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "failure domain for any created local pool", 2024-04-02T14:04:00.545 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-02T14:04:00.545 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.546 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.546 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.546 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.546 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.546 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.546 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.546 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.547 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.547 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.547 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.547 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.547 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.547 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.547 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.547 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.548 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.548 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.548 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.548 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.548 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.548 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.548 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.548 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.548 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.549 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.549 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.549 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.549 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.549 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.549 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.549 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.549 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.550 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.550 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.550 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.550 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.550 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.550 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.550 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.551 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.551 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.551 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.551 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.551 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.551 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.551 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.551 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.551 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.552 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.552 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.552 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.552 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.552 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.552 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.552 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.552 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.553 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.553 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.553 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.553 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.553 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.553 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.553 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.553 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.553 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.553 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.554 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.554 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.554 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.554 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.554 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.555 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.555 INFO:teuthology.orchestra.run.smithi052.stdout: "min_size": { 2024-04-02T14:04:00.555 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_size", 2024-04-02T14:04:00.555 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.555 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.555 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.555 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.555 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.556 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.556 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.556 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default min_size for any created local pool", 2024-04-02T14:04:00.556 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-02T14:04:00.556 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.556 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.557 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.557 INFO:teuthology.orchestra.run.smithi052.stdout: "num_rep": { 2024-04-02T14:04:00.557 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "num_rep", 2024-04-02T14:04:00.557 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.557 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.557 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.557 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3", 2024-04-02T14:04:00.557 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.558 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.558 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.558 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default replica count for any created local pool", 2024-04-02T14:04:00.558 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.558 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.558 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.558 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.558 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_num": { 2024-04-02T14:04:00.559 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_num", 2024-04-02T14:04:00.559 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.559 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.559 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.559 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "128", 2024-04-02T14:04:00.559 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.559 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.559 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.560 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default pg_num for any created local pool", 2024-04-02T14:04:00.560 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.560 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.560 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.560 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.560 INFO:teuthology.orchestra.run.smithi052.stdout: "prefix": { 2024-04-02T14:04:00.560 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prefix", 2024-04-02T14:04:00.560 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.560 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.561 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.561 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.561 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.561 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.561 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.561 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name prefix for any created local pool", 2024-04-02T14:04:00.561 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.561 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.562 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.562 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.562 INFO:teuthology.orchestra.run.smithi052.stdout: "subtree": { 2024-04-02T14:04:00.562 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "subtree", 2024-04-02T14:04:00.562 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.562 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.562 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.562 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "rack", 2024-04-02T14:04:00.563 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.563 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.563 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.563 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-02T14:04:00.563 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-02T14:04:00.563 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.563 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.563 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.563 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.564 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.564 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.564 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mds_autoscaler", 2024-04-02T14:04:00.564 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.564 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.564 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.564 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.564 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.565 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.565 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.565 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.565 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.565 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.565 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.565 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.566 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.566 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.566 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.566 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.566 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.566 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.567 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.567 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.567 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.567 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.567 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.567 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.567 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.568 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.568 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.568 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.568 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.568 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.568 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.568 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.568 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.569 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.569 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.569 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.569 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.569 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.569 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.569 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.569 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.570 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.570 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.570 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.570 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.570 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.570 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.570 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.570 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.571 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.571 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.571 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.571 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.571 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.571 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.571 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.571 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.571 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.572 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.572 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.572 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.572 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.572 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.572 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.572 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.572 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.573 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.573 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.573 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.573 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.573 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.573 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.573 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.573 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.573 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.574 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.574 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirroring", 2024-04-02T14:04:00.574 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.574 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.574 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.574 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.574 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.574 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.575 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.575 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.575 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.575 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.575 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.575 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.576 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.576 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.576 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.576 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.576 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.576 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.576 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.577 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.577 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.577 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.577 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.577 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.577 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.577 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.577 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.578 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.578 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.578 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.578 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.578 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.578 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.578 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.579 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.579 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.579 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.579 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.579 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.579 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.579 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.579 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.579 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.579 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.580 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.580 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.580 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.580 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.580 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.580 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.580 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.580 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.581 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.581 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.581 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.581 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.581 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.581 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.581 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.582 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.582 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.582 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.582 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.582 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.582 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.582 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.582 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.583 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.583 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.583 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.583 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.583 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.583 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.583 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.583 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.583 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.584 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "nfs", 2024-04-02T14:04:00.584 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.584 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.584 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.584 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.584 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.584 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.584 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.584 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.585 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.585 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.585 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.585 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.585 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.585 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.585 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.586 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.586 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.586 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.586 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.586 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.587 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.587 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.587 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.587 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.587 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.587 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.588 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.588 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.588 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.588 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.588 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.588 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.588 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.588 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.589 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.589 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.589 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.589 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.589 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.589 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.589 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.589 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.590 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.590 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.590 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.590 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.590 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.590 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.590 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.590 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.591 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.591 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.591 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.591 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.591 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.591 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.591 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.591 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.591 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.592 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.592 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.592 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.592 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.592 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.592 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.592 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.592 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.592 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.593 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.593 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.593 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.593 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.593 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.593 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.593 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.593 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.593 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2024-04-02T14:04:00.594 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.594 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.594 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.594 INFO:teuthology.orchestra.run.smithi052.stdout: "fail_fs": { 2024-04-02T14:04:00.594 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "fail_fs", 2024-04-02T14:04:00.594 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.594 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.595 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.595 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.595 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.595 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.595 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.595 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-02T14:04:00.595 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.595 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.595 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.596 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.596 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.596 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.596 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.596 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.596 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.597 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.597 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.597 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.597 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.597 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.598 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.598 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.598 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.598 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.598 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.598 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.598 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.598 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.599 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.599 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.599 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.599 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.599 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.599 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.599 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.600 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.600 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.600 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.600 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.600 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.600 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.601 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.601 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.601 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.601 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.601 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.601 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.602 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.602 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.602 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.602 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.603 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.603 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.603 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.603 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.603 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.603 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.604 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.604 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.604 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.604 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.604 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.604 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.604 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.604 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.605 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.605 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.605 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.605 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.605 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.605 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.605 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.605 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.606 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.606 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.606 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.606 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.606 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.606 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.606 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.606 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator": { 2024-04-02T14:04:00.607 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2024-04-02T14:04:00.607 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.607 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.607 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.607 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.607 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.607 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.608 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.608 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm", 2024-04-02T14:04:00.608 INFO:teuthology.orchestra.run.smithi052.stdout: "rook", 2024-04-02T14:04:00.608 INFO:teuthology.orchestra.run.smithi052.stdout: "test_orchestrator" 2024-04-02T14:04:00.608 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.608 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Orchestrator backend", 2024-04-02T14:04:00.609 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.609 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.609 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.609 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.609 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.609 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.609 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.609 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_perf_query", 2024-04-02T14:04:00.609 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.610 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.610 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.610 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.610 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.610 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.610 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.610 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.611 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.611 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.611 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.611 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.611 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.611 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.611 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.611 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.611 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.612 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.612 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.612 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.612 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.612 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.612 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.612 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.612 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.613 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.613 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.613 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.613 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.613 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.613 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.613 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.613 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.613 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.614 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.614 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.614 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.614 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.614 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.614 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.614 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.614 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.615 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.615 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.615 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.615 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.615 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.615 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.615 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.616 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.616 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.616 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.616 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.617 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.617 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.617 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.617 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.618 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.618 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.618 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.621 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.621 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.621 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.622 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.622 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.622 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.622 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.623 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.623 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.623 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.624 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.624 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.624 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.624 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.625 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.625 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.625 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_support", 2024-04-02T14:04:00.625 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.626 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.626 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.626 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.626 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.627 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.627 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.627 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.627 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.628 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.628 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.628 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.628 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.629 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.629 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.629 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.629 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.630 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.630 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.630 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.630 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.631 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.631 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.631 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.631 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.632 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.632 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.632 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.633 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.633 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.633 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.633 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.634 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.634 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.634 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.634 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.635 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.635 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.635 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.635 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.636 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.636 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.636 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.637 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.637 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.637 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.637 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.638 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.638 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.638 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.638 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.639 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.639 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.639 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.640 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.640 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.640 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.640 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.641 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.641 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.641 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.641 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.642 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.642 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.642 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.642 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.643 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.643 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.643 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.643 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.644 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.644 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.644 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.645 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.645 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.645 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.646 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_autoscaler", 2024-04-02T14:04:00.646 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.646 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.646 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.647 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.647 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.647 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.647 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.648 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.648 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.648 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.648 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.649 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.649 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.649 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.649 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.649 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.650 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.650 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.650 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.650 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.650 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.651 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.651 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.651 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.651 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.651 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.651 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.652 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.652 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.652 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.652 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.653 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.653 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.653 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.653 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.653 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.653 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.653 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.654 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.654 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.654 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.654 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.654 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.654 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.654 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.655 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.655 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.655 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.655 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.655 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.655 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.655 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.655 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.655 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.656 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.656 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.656 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.656 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.656 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.656 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.656 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.656 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.657 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.657 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.657 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.657 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.657 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.657 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.657 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.658 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.658 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.658 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.658 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.658 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-04-02T14:04:00.658 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-04-02T14:04:00.658 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:00.658 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.659 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.659 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-04-02T14:04:00.659 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.659 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.659 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.659 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.659 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.660 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.660 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.660 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.661 INFO:teuthology.orchestra.run.smithi052.stdout: "threshold": { 2024-04-02T14:04:00.661 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threshold", 2024-04-02T14:04:00.661 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-04-02T14:04:00.661 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.661 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.661 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2024-04-02T14:04:00.661 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1.0", 2024-04-02T14:04:00.662 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.662 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.662 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "scaling threshold", 2024-04-02T14:04:00.662 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-02T14:04:00.662 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.662 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.662 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.662 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.663 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.663 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.663 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "progress", 2024-04-02T14:04:00.663 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.663 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.663 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.663 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_pg_recovery_event": { 2024-04-02T14:04:00.663 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_pg_recovery_event", 2024-04-02T14:04:00.664 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.664 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.664 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.664 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.664 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.664 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.664 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.664 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-02T14:04:00.664 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.665 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.665 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.665 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.665 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2024-04-02T14:04:00.665 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2024-04-02T14:04:00.665 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.665 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.665 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.665 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.666 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.666 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.666 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.666 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.666 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.666 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.666 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.666 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.667 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.667 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.667 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.667 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.667 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.667 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.667 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.667 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.667 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.668 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.668 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.668 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.668 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.668 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.668 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.668 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.668 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.669 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.669 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.669 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.669 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.669 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.669 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.669 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.669 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.669 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.670 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.670 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.670 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.670 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.670 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.670 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.670 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.671 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.671 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.671 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.671 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.671 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.671 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.671 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.671 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.672 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.672 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.672 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.672 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.672 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.672 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.672 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.672 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.673 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.673 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.673 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.673 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.673 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.673 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.673 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.674 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.674 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.674 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.674 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.674 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.674 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.674 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.674 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.675 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.675 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.675 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.675 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.675 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.675 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.675 INFO:teuthology.orchestra.run.smithi052.stdout: "max_completed_events": { 2024-04-02T14:04:00.675 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_completed_events", 2024-04-02T14:04:00.676 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.676 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.676 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.676 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "50", 2024-04-02T14:04:00.676 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.676 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.676 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.676 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "number of past completed events to remember", 2024-04-02T14:04:00.677 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.677 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.677 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.677 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.677 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-04-02T14:04:00.677 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-04-02T14:04:00.677 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:00.677 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.678 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.678 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-04-02T14:04:00.678 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.678 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.678 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.678 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long the module is going to sleep", 2024-04-02T14:04:00.678 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.679 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.679 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.679 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.679 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.679 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.679 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.679 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus", 2024-04-02T14:04:00.679 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.679 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.680 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.680 INFO:teuthology.orchestra.run.smithi052.stdout: "cache": { 2024-04-02T14:04:00.680 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cache", 2024-04-02T14:04:00.680 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.680 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.680 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.680 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.680 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.681 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.681 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.681 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.681 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.681 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.682 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.682 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.682 INFO:teuthology.orchestra.run.smithi052.stdout: "exclude_perf_counters": { 2024-04-02T14:04:00.682 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "exclude_perf_counters", 2024-04-02T14:04:00.682 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.682 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.682 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.682 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.683 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.683 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.683 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.683 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-02T14:04:00.683 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-02T14:04:00.683 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.683 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.683 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.684 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.684 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.684 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.684 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.684 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.684 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.684 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.684 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.685 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.685 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.685 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.685 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.685 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.685 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.685 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.686 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.686 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.686 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.686 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.687 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.687 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.687 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.687 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.687 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.687 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.688 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.688 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.688 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.688 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.688 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.688 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.688 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.688 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.689 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.689 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.689 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.689 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.689 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.689 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.689 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.689 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.689 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.690 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.690 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.690 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.690 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.690 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.690 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.690 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.691 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.691 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.691 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.691 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.691 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.691 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.692 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.692 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.692 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.692 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.692 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.692 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.692 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.692 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.692 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.693 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.693 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.693 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.693 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.693 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.693 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.693 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools": { 2024-04-02T14:04:00.693 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools", 2024-04-02T14:04:00.693 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.694 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.694 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.694 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.694 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.694 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.694 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.694 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.695 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.695 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.695 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.695 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.695 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-02T14:04:00.695 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-02T14:04:00.695 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.695 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.695 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.696 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "300", 2024-04-02T14:04:00.696 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.696 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.696 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.696 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.696 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.696 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.696 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.697 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.697 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_interval": { 2024-04-02T14:04:00.697 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_interval", 2024-04-02T14:04:00.697 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-04-02T14:04:00.697 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.697 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.697 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15.0", 2024-04-02T14:04:00.698 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.698 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.698 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.698 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.698 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.698 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.698 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.698 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.699 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-04-02T14:04:00.699 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-04-02T14:04:00.699 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.699 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.699 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.699 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2024-04-02T14:04:00.699 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.699 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.700 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.700 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-02T14:04:00.700 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.700 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.700 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.700 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.700 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-04-02T14:04:00.700 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-04-02T14:04:00.701 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.701 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.701 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.701 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "9283", 2024-04-02T14:04:00.701 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.701 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.701 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.701 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-02T14:04:00.702 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.702 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.702 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.702 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.703 INFO:teuthology.orchestra.run.smithi052.stdout: "stale_cache_strategy": { 2024-04-02T14:04:00.703 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stale_cache_strategy", 2024-04-02T14:04:00.703 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.703 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.703 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.703 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "log", 2024-04-02T14:04:00.703 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.703 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.704 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.704 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.704 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.704 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.704 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.704 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.704 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2024-04-02T14:04:00.705 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2024-04-02T14:04:00.705 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.705 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.705 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.705 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "default", 2024-04-02T14:04:00.705 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.705 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.705 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.706 INFO:teuthology.orchestra.run.smithi052.stdout: "default", 2024-04-02T14:04:00.706 INFO:teuthology.orchestra.run.smithi052.stdout: "error" 2024-04-02T14:04:00.706 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.706 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.706 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.706 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.706 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.706 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.707 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2024-04-02T14:04:00.707 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2024-04-02T14:04:00.707 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.707 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.707 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.707 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2024-04-02T14:04:00.707 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2024-04-02T14:04:00.707 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2024-04-02T14:04:00.707 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.708 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.708 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.708 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.708 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.708 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.708 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.708 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.708 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.709 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_support", 2024-04-02T14:04:00.709 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.709 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.709 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.709 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.709 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.710 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.710 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.710 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.710 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.710 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.710 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.710 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.711 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.711 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.711 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.711 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.711 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.711 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.711 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.711 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.712 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.712 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.712 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.712 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.712 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.712 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.712 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.713 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.713 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.713 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.713 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.713 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.713 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.713 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.714 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.714 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.714 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.714 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.714 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.714 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.714 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.714 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.715 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.715 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.715 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.715 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.715 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.715 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.715 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.715 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.716 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.716 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.716 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.716 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.716 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.716 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.716 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.716 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.716 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.717 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.717 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.717 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.717 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.717 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.718 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.719 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.719 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_snap_create": { 2024-04-02T14:04:00.719 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_snap_create", 2024-04-02T14:04:00.719 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.719 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.719 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.719 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-04-02T14:04:00.719 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.720 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.720 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.720 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.720 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.720 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.720 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.720 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.720 INFO:teuthology.orchestra.run.smithi052.stdout: "mirror_snapshot_schedule": { 2024-04-02T14:04:00.720 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirror_snapshot_schedule", 2024-04-02T14:04:00.721 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.721 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.721 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.721 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.721 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.721 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.721 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.721 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.721 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.722 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.722 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.722 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.722 INFO:teuthology.orchestra.run.smithi052.stdout: "trash_purge_schedule": { 2024-04-02T14:04:00.722 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "trash_purge_schedule", 2024-04-02T14:04:00.723 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.723 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.723 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.723 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.723 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.723 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.724 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.724 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.724 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.724 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.724 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.724 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.724 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.724 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.725 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.725 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "restful", 2024-04-02T14:04:00.725 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.725 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.725 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.725 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_auth": { 2024-04-02T14:04:00.725 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_auth", 2024-04-02T14:04:00.726 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.726 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.726 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.726 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.726 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.726 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.726 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.727 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.727 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.727 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.727 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.727 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.727 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2024-04-02T14:04:00.727 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2024-04-02T14:04:00.727 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.728 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.728 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.728 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.728 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.728 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.728 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.728 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.728 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.729 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.729 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.729 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.729 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.729 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.729 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.729 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.729 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.730 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.730 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.730 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.730 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.730 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.730 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.730 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.730 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.731 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.731 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.731 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.731 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.731 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.731 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.731 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.731 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.732 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.732 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.732 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.732 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.732 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.732 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.732 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.732 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.733 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.733 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.733 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.733 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.733 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.733 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.733 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.734 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.734 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.734 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.734 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.734 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.734 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.735 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.735 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.735 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.735 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.735 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.735 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.735 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.736 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.736 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.736 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.736 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.736 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.736 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.736 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.736 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.737 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.737 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.737 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.737 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.737 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.737 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.737 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.737 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.737 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.738 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.738 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.738 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.738 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.738 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-04-02T14:04:00.738 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-04-02T14:04:00.738 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.739 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.739 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.739 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.739 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.739 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.739 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.739 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.739 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.740 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.740 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.740 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.740 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-04-02T14:04:00.740 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-04-02T14:04:00.740 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.740 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.740 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.740 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.741 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.741 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.741 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.741 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.741 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.741 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.741 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.741 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.742 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.742 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.742 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.742 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rgw", 2024-04-02T14:04:00.742 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.742 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.742 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.742 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.743 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.743 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.743 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.743 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.743 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.743 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.744 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.744 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.744 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.744 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.745 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.745 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.745 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.745 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.745 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.745 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.745 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.745 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.745 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.746 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.746 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.746 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.746 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.746 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.746 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.746 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.747 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.747 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.747 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.747 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.747 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.747 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.747 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.747 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.748 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.748 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.748 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.748 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.748 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.748 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.748 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.748 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.748 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.749 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.749 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.749 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.749 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.749 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.749 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.749 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.749 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.750 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.750 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.750 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.750 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.750 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.750 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.750 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.751 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.751 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.751 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.751 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.751 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.751 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.751 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.751 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.752 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.752 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.752 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.752 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.752 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.752 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.752 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rook", 2024-04-02T14:04:00.752 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.753 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.753 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.753 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.753 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.753 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.753 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.753 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.753 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.754 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.754 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.754 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.754 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.754 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.754 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.754 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.754 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.755 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.755 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.755 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.755 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.755 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.755 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.755 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.755 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.756 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.756 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.756 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.756 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.756 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.756 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.756 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.756 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.756 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.757 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.757 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.757 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.757 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.757 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.757 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.757 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.757 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.758 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.758 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.758 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.758 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.758 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.759 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.759 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.759 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.759 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.759 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.759 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.759 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.759 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.760 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.760 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.760 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.760 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.760 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.760 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.760 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.760 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.761 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.761 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.761 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.761 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.761 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.761 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.761 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.761 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.761 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.762 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.762 INFO:teuthology.orchestra.run.smithi052.stdout: "storage_class": { 2024-04-02T14:04:00.762 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "storage_class", 2024-04-02T14:04:00.762 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.762 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.762 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.762 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "local", 2024-04-02T14:04:00.763 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.763 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.763 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.763 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-02T14:04:00.763 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.763 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.763 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.763 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.763 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.764 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.764 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.764 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "selftest", 2024-04-02T14:04:00.764 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.764 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.764 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.764 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.764 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.765 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.765 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.765 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.765 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.765 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.765 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.765 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.765 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.765 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.766 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.766 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.766 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.766 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.766 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.766 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.766 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.766 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.767 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.767 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.767 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.767 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.767 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.767 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.767 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.767 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.768 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.768 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.768 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.768 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.768 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.768 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.768 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.768 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.769 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.769 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.769 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.769 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.769 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.769 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.769 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.770 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.770 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.770 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.770 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.770 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.770 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.770 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.770 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.771 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.771 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.771 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.771 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.771 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.771 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.771 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.771 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.772 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.772 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.772 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.772 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.772 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.772 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.772 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.772 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.773 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.773 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.773 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.773 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.773 INFO:teuthology.orchestra.run.smithi052.stdout: "roption1": { 2024-04-02T14:04:00.773 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption1", 2024-04-02T14:04:00.773 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.774 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.774 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.774 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.774 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.774 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.774 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.774 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.775 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.775 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.775 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.775 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.775 INFO:teuthology.orchestra.run.smithi052.stdout: "roption2": { 2024-04-02T14:04:00.775 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption2", 2024-04-02T14:04:00.775 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.775 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.776 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.776 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "xyz", 2024-04-02T14:04:00.776 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.776 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.776 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.776 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.776 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.776 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.776 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.777 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.777 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption1": { 2024-04-02T14:04:00.777 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption1", 2024-04-02T14:04:00.777 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.777 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.777 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.777 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.777 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.777 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.777 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.778 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.778 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.778 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.778 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.778 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.778 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption2": { 2024-04-02T14:04:00.778 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption2", 2024-04-02T14:04:00.778 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.779 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.779 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.779 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.779 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.779 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.779 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.779 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.779 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.779 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.780 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.780 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.780 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption3": { 2024-04-02T14:04:00.780 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption3", 2024-04-02T14:04:00.780 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-04-02T14:04:00.780 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.780 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.780 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.780 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.780 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.781 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.781 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.781 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.781 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.781 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.781 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.781 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption4": { 2024-04-02T14:04:00.781 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption4", 2024-04-02T14:04:00.782 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.782 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.782 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.782 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.782 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.782 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.782 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.783 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.783 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.783 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.783 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.783 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.783 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption5": { 2024-04-02T14:04:00.783 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption5", 2024-04-02T14:04:00.783 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.784 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.784 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.784 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.784 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.784 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.784 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.784 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.784 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.785 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.785 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.785 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.785 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption6": { 2024-04-02T14:04:00.785 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption6", 2024-04-02T14:04:00.785 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.785 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.785 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.786 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.786 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.786 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.786 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.786 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.786 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.787 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.787 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.787 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.787 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption7": { 2024-04-02T14:04:00.787 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption7", 2024-04-02T14:04:00.787 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.787 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.788 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.788 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.788 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-04-02T14:04:00.788 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "42", 2024-04-02T14:04:00.788 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.788 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.788 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.788 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.789 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.789 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.789 INFO:teuthology.orchestra.run.smithi052.stdout: "testkey": { 2024-04-02T14:04:00.789 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testkey", 2024-04-02T14:04:00.789 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.789 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.789 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.789 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.790 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.790 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.790 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.790 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.790 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.790 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.790 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.790 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.791 INFO:teuthology.orchestra.run.smithi052.stdout: "testlkey": { 2024-04-02T14:04:00.791 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testlkey", 2024-04-02T14:04:00.791 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.791 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.791 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.791 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.791 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.791 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.792 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.792 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.792 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.792 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.792 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.792 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.792 INFO:teuthology.orchestra.run.smithi052.stdout: "testnewline": { 2024-04-02T14:04:00.792 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testnewline", 2024-04-02T14:04:00.792 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.793 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.793 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.793 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.793 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.793 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.793 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.793 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.793 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.793 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.794 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.794 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.794 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.794 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.794 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.794 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snap_schedule", 2024-04-02T14:04:00.794 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.794 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.795 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.795 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_m_granularity": { 2024-04-02T14:04:00.795 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_m_granularity", 2024-04-02T14:04:00.795 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.795 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.795 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.795 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.795 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.795 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.796 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.796 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow minute scheduled snapshots", 2024-04-02T14:04:00.796 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.796 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.796 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.796 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.796 INFO:teuthology.orchestra.run.smithi052.stdout: "dump_on_update": { 2024-04-02T14:04:00.796 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dump_on_update", 2024-04-02T14:04:00.797 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.797 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.797 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.797 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.797 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.797 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.797 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.797 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "dump database to debug log on update", 2024-04-02T14:04:00.798 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.798 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.798 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.798 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.798 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.798 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.798 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.798 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.799 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.799 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.799 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.799 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.799 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.799 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.799 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.799 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.799 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.800 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.800 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.800 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.800 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.800 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.800 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.800 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.800 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.800 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.801 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.801 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.801 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.801 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.801 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.801 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.801 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.802 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.802 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.802 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.802 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.802 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.802 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.802 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.802 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.803 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.803 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.803 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.803 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.803 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.803 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.803 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.803 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.803 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.804 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.804 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.804 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.804 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.804 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.804 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.804 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.804 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.804 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.805 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.805 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.805 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.805 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.805 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.805 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.806 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.806 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.806 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.806 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.806 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.806 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.806 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.806 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.807 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.807 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.807 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.807 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.807 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stats", 2024-04-02T14:04:00.807 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.807 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.807 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.808 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.808 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.808 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.808 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.808 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.808 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.808 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.808 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.809 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.809 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.809 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.809 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.809 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.809 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.809 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.810 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.810 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.810 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.810 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.810 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.810 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.810 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.811 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.811 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.811 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.811 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.811 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.811 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.811 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.811 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.811 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.812 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.812 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.812 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.812 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.812 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.812 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.812 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.812 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.813 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.813 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.813 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.813 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.813 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.813 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.813 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.814 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.814 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.814 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.814 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.814 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.814 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.814 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.814 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.815 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.815 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.815 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.815 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.815 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.815 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.815 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.815 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.816 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.816 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.816 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.816 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.816 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.816 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.816 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.817 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.817 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.817 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.817 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.817 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "status", 2024-04-02T14:04:00.817 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.817 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.818 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.818 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.818 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.818 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.818 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.818 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.818 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.819 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.819 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.819 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.819 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.819 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.819 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.819 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.820 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.820 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.820 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.820 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.820 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.820 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.820 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.821 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.821 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.821 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.821 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.821 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.821 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.821 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.821 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.821 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.822 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.822 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.822 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.822 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.822 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.822 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.822 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.822 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.823 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.823 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.823 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.823 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.823 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.823 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.823 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.824 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.824 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.824 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.824 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.824 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.824 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.824 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.824 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.824 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.825 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.825 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.825 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.825 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.825 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.825 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.825 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.825 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.826 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.826 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.826 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.826 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.826 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.826 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.826 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.826 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.827 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.827 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.827 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.827 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.827 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telegraf", 2024-04-02T14:04:00.827 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.828 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.828 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.828 INFO:teuthology.orchestra.run.smithi052.stdout: "address": { 2024-04-02T14:04:00.828 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "address", 2024-04-02T14:04:00.828 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.828 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.828 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.828 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-02T14:04:00.829 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.829 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.829 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.829 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.829 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.829 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.829 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.829 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.830 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-04-02T14:04:00.830 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-04-02T14:04:00.830 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:00.830 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.830 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.830 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15", 2024-04-02T14:04:00.830 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.830 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.831 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.831 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.831 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.831 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.831 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.831 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.831 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.831 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.832 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.832 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.832 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.832 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.832 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.832 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.832 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.832 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.833 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.833 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.833 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.833 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.833 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.833 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.833 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.833 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.834 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.834 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.834 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.834 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.834 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.834 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.834 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.834 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.835 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.835 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.835 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.835 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.835 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.835 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.835 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.835 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.836 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.836 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.836 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.836 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.836 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.836 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.836 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.837 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.837 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.837 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.837 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.837 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.837 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.837 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.838 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.838 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.838 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.838 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.838 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.838 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.839 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.839 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.839 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.839 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.839 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.839 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.839 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.839 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.839 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.840 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.840 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.840 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.840 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.840 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.840 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.840 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.840 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.840 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.841 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.841 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telemetry", 2024-04-02T14:04:00.841 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.841 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.841 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.841 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_basic": { 2024-04-02T14:04:00.841 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_basic", 2024-04-02T14:04:00.841 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.842 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.842 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.842 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.842 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.842 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.842 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.842 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-02T14:04:00.842 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.843 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.843 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.843 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.843 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_crash": { 2024-04-02T14:04:00.843 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_crash", 2024-04-02T14:04:00.843 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.843 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.843 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.843 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.844 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.844 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.844 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.844 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-02T14:04:00.844 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.844 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.844 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.844 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.844 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_device": { 2024-04-02T14:04:00.845 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_device", 2024-04-02T14:04:00.845 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.845 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.845 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.845 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.845 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.845 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.845 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.845 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-02T14:04:00.846 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.846 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.846 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.846 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.846 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_ident": { 2024-04-02T14:04:00.846 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_ident", 2024-04-02T14:04:00.846 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.847 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.847 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.847 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.847 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.847 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.847 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.847 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-02T14:04:00.847 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.848 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.848 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.848 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.848 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_perf": { 2024-04-02T14:04:00.848 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_perf", 2024-04-02T14:04:00.848 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.849 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.849 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.849 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.849 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.849 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.849 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.849 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-02T14:04:00.849 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.850 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.850 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.850 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.850 INFO:teuthology.orchestra.run.smithi052.stdout: "contact": { 2024-04-02T14:04:00.850 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "contact", 2024-04-02T14:04:00.850 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.850 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.850 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.851 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.851 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.851 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.851 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.851 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.851 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.851 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.851 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.851 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.852 INFO:teuthology.orchestra.run.smithi052.stdout: "description": { 2024-04-02T14:04:00.852 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "description", 2024-04-02T14:04:00.852 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.852 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.852 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.852 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.853 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.853 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.853 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.853 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.853 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.853 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.853 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.853 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.853 INFO:teuthology.orchestra.run.smithi052.stdout: "device_url": { 2024-04-02T14:04:00.854 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_url", 2024-04-02T14:04:00.854 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.854 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.854 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.854 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-02T14:04:00.854 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.854 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.854 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.855 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.855 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.855 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.855 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.855 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.855 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2024-04-02T14:04:00.855 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2024-04-02T14:04:00.855 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.856 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.856 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.856 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.856 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.856 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.856 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.856 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.856 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.856 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.857 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.857 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.857 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-04-02T14:04:00.857 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-04-02T14:04:00.857 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.857 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.857 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.857 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "24", 2024-04-02T14:04:00.858 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "8", 2024-04-02T14:04:00.858 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.858 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.858 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.858 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.858 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.858 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.859 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.859 INFO:teuthology.orchestra.run.smithi052.stdout: "last_opt_revision": { 2024-04-02T14:04:00.859 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "last_opt_revision", 2024-04-02T14:04:00.859 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.859 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.859 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.860 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1", 2024-04-02T14:04:00.860 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.860 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.860 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.860 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.860 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.860 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.861 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.861 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.861 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard": { 2024-04-02T14:04:00.861 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard", 2024-04-02T14:04:00.861 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.861 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.861 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.861 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.862 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.862 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.862 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.862 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.862 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.862 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.862 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.862 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.863 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard_description": { 2024-04-02T14:04:00.863 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard_description", 2024-04-02T14:04:00.863 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.863 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.863 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.863 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.863 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.863 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.864 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.864 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.864 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.864 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.864 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.864 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.864 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.864 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.865 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.865 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.865 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.865 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.865 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.865 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.865 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.865 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.866 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.866 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.866 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.866 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.866 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.866 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.867 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.867 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.867 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.867 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.868 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.868 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.868 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.868 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.869 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.869 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.869 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.869 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.869 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.870 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.870 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.870 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.870 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.871 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.871 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.871 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.871 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.871 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.871 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.872 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.872 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.872 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.872 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.873 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.873 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.873 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.873 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.873 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.873 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.874 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.874 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.874 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.874 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.874 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.874 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.874 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.875 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.875 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.875 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.875 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.875 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.875 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.875 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.875 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.876 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.876 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.876 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.876 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.876 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.876 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.876 INFO:teuthology.orchestra.run.smithi052.stdout: "organization": { 2024-04-02T14:04:00.876 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "organization", 2024-04-02T14:04:00.877 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.877 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.877 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.877 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.877 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.877 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.877 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.877 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.877 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.878 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.878 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.878 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.878 INFO:teuthology.orchestra.run.smithi052.stdout: "proxy": { 2024-04-02T14:04:00.878 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "proxy", 2024-04-02T14:04:00.878 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.878 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.878 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.879 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.879 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.879 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.879 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.879 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.879 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.879 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.880 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.880 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.880 INFO:teuthology.orchestra.run.smithi052.stdout: "url": { 2024-04-02T14:04:00.880 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url", 2024-04-02T14:04:00.880 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.880 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.881 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.881 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-02T14:04:00.881 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.881 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.881 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.881 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.881 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.882 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.882 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.882 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.882 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.882 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.882 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.882 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "test_orchestrator", 2024-04-02T14:04:00.883 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.883 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.883 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.883 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.883 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.883 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.883 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.883 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.884 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.884 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.884 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.884 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.884 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.884 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.884 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.884 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.885 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.885 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.885 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.885 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.885 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.885 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.885 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.885 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.886 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.886 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.886 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.886 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.886 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.886 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.886 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.887 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.887 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.887 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.887 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.887 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.888 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.888 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.888 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.888 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.888 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.888 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.888 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.888 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.888 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.889 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.889 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.889 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.889 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.889 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.889 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.889 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.889 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.890 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.890 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.890 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.890 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.890 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.891 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.891 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.891 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.891 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.891 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.892 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.892 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.892 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.892 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.892 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.893 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.893 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.893 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.893 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.893 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.893 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.893 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.893 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.894 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "volumes", 2024-04-02T14:04:00.894 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.894 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.894 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.894 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.894 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.894 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.895 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.895 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.895 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.895 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.895 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.895 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.895 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.895 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.896 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.896 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.896 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.896 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.896 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.896 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.896 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.896 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.896 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.897 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.897 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.897 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.897 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.897 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.897 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.897 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.897 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.898 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.898 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.898 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.898 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.898 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.898 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.899 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.899 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.899 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.899 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.899 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.899 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.899 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.899 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.899 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.900 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.900 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.900 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.900 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.900 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.900 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.900 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.900 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.901 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.901 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.901 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.901 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.901 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.902 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.902 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.902 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.902 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.902 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.902 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.902 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.903 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.903 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.903 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.903 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.903 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.903 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.903 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.903 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_clones": { 2024-04-02T14:04:00.903 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_clones", 2024-04-02T14:04:00.904 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.904 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.904 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.904 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4", 2024-04-02T14:04:00.904 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.904 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.904 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.904 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-02T14:04:00.905 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.905 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.905 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.905 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.905 INFO:teuthology.orchestra.run.smithi052.stdout: "periodic_async_work": { 2024-04-02T14:04:00.905 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "periodic_async_work", 2024-04-02T14:04:00.905 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.905 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.906 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.906 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.906 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.906 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.906 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.906 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Periodically check for async work", 2024-04-02T14:04:00.907 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.907 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.907 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.907 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.907 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_delay": { 2024-04-02T14:04:00.907 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_delay", 2024-04-02T14:04:00.907 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.907 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.907 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.908 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-04-02T14:04:00.908 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.908 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.908 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.908 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-02T14:04:00.908 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.908 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.908 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.908 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.909 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_no_wait": { 2024-04-02T14:04:00.909 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_no_wait", 2024-04-02T14:04:00.909 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.909 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.909 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.909 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.909 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.909 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.910 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.910 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-02T14:04:00.910 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.910 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.910 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.910 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.911 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.911 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.911 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.911 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix", 2024-04-02T14:04:00.911 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.911 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.911 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.912 INFO:teuthology.orchestra.run.smithi052.stdout: "discovery_interval": { 2024-04-02T14:04:00.912 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "discovery_interval", 2024-04-02T14:04:00.912 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-04-02T14:04:00.912 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.912 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.912 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "100", 2024-04-02T14:04:00.912 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.913 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.913 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.913 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.913 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.913 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.913 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.913 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.914 INFO:teuthology.orchestra.run.smithi052.stdout: "identifier": { 2024-04-02T14:04:00.914 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "identifier", 2024-04-02T14:04:00.914 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.914 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.914 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.914 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.914 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.915 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.915 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.915 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.915 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.915 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.915 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.915 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.915 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-04-02T14:04:00.915 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-04-02T14:04:00.916 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:00.916 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.916 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.916 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-04-02T14:04:00.916 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.916 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.916 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.916 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.917 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.917 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.917 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.917 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.917 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.917 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.917 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.917 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.918 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.918 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.918 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.918 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.918 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.918 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.918 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.918 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.919 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.919 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.919 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.919 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.919 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.919 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.919 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.919 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.920 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.920 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.920 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.920 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.920 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.920 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.921 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.921 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.921 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.921 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.921 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.921 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.921 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.921 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.921 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.922 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.922 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.922 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.922 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.922 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.923 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.923 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.923 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.923 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.923 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.923 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.923 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.923 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.924 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.924 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.924 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.924 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.924 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.924 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.924 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.924 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.925 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.925 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.925 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.925 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.925 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.925 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.925 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.925 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.926 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.926 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.926 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.926 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.926 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.926 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.927 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_host": { 2024-04-02T14:04:00.927 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_host", 2024-04-02T14:04:00.927 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.927 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.927 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.927 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.927 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.927 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.928 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.928 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.928 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.928 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.928 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.928 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.928 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_port": { 2024-04-02T14:04:00.928 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_port", 2024-04-02T14:04:00.928 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.929 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.929 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.929 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10051", 2024-04-02T14:04:00.929 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.929 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.929 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.929 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.930 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.930 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.930 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.930 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.930 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_sender": { 2024-04-02T14:04:00.930 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_sender", 2024-04-02T14:04:00.930 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.930 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-02T14:04:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.931 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.932 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.932 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.932 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.932 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.932 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-04-02T14:04:00.932 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.932 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.932 INFO:teuthology.orchestra.run.smithi052.stdout: "gid": 25090, 2024-04-02T14:04:00.933 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "z", 2024-04-02T14:04:00.933 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr_features": 4540701547738038271, 2024-04-02T14:04:00.933 INFO:teuthology.orchestra.run.smithi052.stdout: "available_modules": [ 2024-04-02T14:04:00.933 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.933 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "alerts", 2024-04-02T14:04:00.933 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.934 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.934 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.934 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-04-02T14:04:00.934 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-04-02T14:04:00.934 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:00.934 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.935 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.935 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-04-02T14:04:00.935 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.935 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.935 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.935 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How frequently to reexamine health status", 2024-04-02T14:04:00.935 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.935 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.936 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.936 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.936 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.936 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.936 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.936 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.936 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.936 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.937 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.937 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.937 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.937 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.937 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.937 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.937 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.937 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.938 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.938 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.938 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.938 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.938 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.938 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.938 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.939 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.939 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.939 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.939 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.939 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.939 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.939 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.940 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.940 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.940 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.940 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.940 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.940 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.940 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.940 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.940 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.941 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.941 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.941 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.941 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.941 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.941 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.941 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.942 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.942 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.942 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.942 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.942 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.942 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.942 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.942 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.943 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.943 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.943 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.943 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.943 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.944 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.944 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.944 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.944 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.944 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.945 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.945 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.945 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.945 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.945 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.945 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.945 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.945 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.945 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_destination": { 2024-04-02T14:04:00.946 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_destination", 2024-04-02T14:04:00.946 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.946 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.946 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.946 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.946 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.946 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.947 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.947 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email address to send alerts to", 2024-04-02T14:04:00.947 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.947 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.947 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.947 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.947 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_from_name": { 2024-04-02T14:04:00.947 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_from_name", 2024-04-02T14:04:00.948 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.948 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.948 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.948 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "Ceph", 2024-04-02T14:04:00.948 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.948 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.948 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.948 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email From: name", 2024-04-02T14:04:00.949 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.949 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.949 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.949 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.949 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_host": { 2024-04-02T14:04:00.949 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_host", 2024-04-02T14:04:00.949 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.949 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.950 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.950 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.950 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.950 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.950 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.950 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP server", 2024-04-02T14:04:00.950 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.950 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.951 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.951 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.951 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_password": { 2024-04-02T14:04:00.951 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_password", 2024-04-02T14:04:00.951 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.951 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.951 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.951 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.952 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.952 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.952 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.952 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Password to authenticate with", 2024-04-02T14:04:00.952 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.952 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.952 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.952 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.953 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_port": { 2024-04-02T14:04:00.953 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_port", 2024-04-02T14:04:00.953 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.953 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.953 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.953 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "465", 2024-04-02T14:04:00.953 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.953 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.954 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.954 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP port", 2024-04-02T14:04:00.954 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.954 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.954 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.954 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.954 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_sender": { 2024-04-02T14:04:00.955 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_sender", 2024-04-02T14:04:00.955 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.955 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.955 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.955 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.955 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.955 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.956 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.956 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP envelope sender", 2024-04-02T14:04:00.956 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.956 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.956 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.956 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.956 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_ssl": { 2024-04-02T14:04:00.956 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_ssl", 2024-04-02T14:04:00.957 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.957 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.957 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.957 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.957 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.957 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.957 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.957 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-02T14:04:00.958 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.958 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.958 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.958 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.958 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_user": { 2024-04-02T14:04:00.958 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_user", 2024-04-02T14:04:00.958 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.958 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.959 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.959 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.959 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.959 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.959 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.959 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "User to authenticate as", 2024-04-02T14:04:00.959 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.959 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.960 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.960 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.960 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.960 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.960 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.960 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "balancer", 2024-04-02T14:04:00.960 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.960 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.961 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.961 INFO:teuthology.orchestra.run.smithi052.stdout: "active": { 2024-04-02T14:04:00.961 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "active", 2024-04-02T14:04:00.961 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.961 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.961 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.962 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:00.962 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.962 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.962 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.962 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically balance PGs across cluster", 2024-04-02T14:04:00.962 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.962 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.962 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.963 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.963 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_time": { 2024-04-02T14:04:00.963 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_time", 2024-04-02T14:04:00.963 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.963 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.963 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.963 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0000", 2024-04-02T14:04:00.963 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.964 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.964 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.964 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "beginning time of day to automatically balance", 2024-04-02T14:04:00.964 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T14:04:00.964 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.964 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.964 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.964 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_weekday": { 2024-04-02T14:04:00.964 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_weekday", 2024-04-02T14:04:00.965 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-04-02T14:04:00.965 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.965 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.965 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-04-02T14:04:00.965 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2024-04-02T14:04:00.965 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2024-04-02T14:04:00.966 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.966 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-02T14:04:00.966 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T14:04:00.966 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.966 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.966 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.966 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_max_iterations": { 2024-04-02T14:04:00.967 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_max_iterations", 2024-04-02T14:04:00.967 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-04-02T14:04:00.967 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.967 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.967 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "25", 2024-04-02T14:04:00.967 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-04-02T14:04:00.967 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "250", 2024-04-02T14:04:00.967 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.968 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-02T14:04:00.968 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.968 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.968 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.968 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.968 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_metrics": { 2024-04-02T14:04:00.968 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_metrics", 2024-04-02T14:04:00.968 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.969 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.969 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.969 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "pgs,objects,bytes", 2024-04-02T14:04:00.969 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.969 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.969 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.969 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-02T14:04:00.969 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-02T14:04:00.969 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.970 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.970 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.970 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_step": { 2024-04-02T14:04:00.970 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_step", 2024-04-02T14:04:00.970 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-04-02T14:04:00.970 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.970 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.971 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.5", 2024-04-02T14:04:00.971 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0.001", 2024-04-02T14:04:00.971 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "0.999", 2024-04-02T14:04:00.971 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.971 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "aggressiveness of optimization", 2024-04-02T14:04:00.971 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-02T14:04:00.971 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.971 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.972 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.972 INFO:teuthology.orchestra.run.smithi052.stdout: "end_time": { 2024-04-02T14:04:00.972 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_time", 2024-04-02T14:04:00.972 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.972 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.972 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.972 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2359", 2024-04-02T14:04:00.972 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.973 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.973 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.973 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ending time of day to automatically balance", 2024-04-02T14:04:00.973 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T14:04:00.973 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.973 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.973 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.973 INFO:teuthology.orchestra.run.smithi052.stdout: "end_weekday": { 2024-04-02T14:04:00.973 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_weekday", 2024-04-02T14:04:00.974 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-04-02T14:04:00.974 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.974 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.974 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-04-02T14:04:00.974 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2024-04-02T14:04:00.974 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2024-04-02T14:04:00.974 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.975 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-02T14:04:00.975 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T14:04:00.975 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.975 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.975 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.975 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:00.976 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:00.976 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.976 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.976 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.976 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.976 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.977 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.977 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.977 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.977 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.977 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.977 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.977 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.977 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.977 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.978 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.978 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.978 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.978 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.978 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.978 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:00.978 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:00.978 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.979 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.979 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.979 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.979 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.979 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.979 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.979 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.979 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.980 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.980 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.980 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.980 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:00.980 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:00.980 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.980 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.980 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.981 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:00.981 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.981 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.981 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.981 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:00.981 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:00.981 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:00.982 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:00.982 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:00.982 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:00.982 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.982 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.982 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.982 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.983 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.983 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.983 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:00.983 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:00.983 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:00.983 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.983 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.983 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:00.984 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.984 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.984 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.984 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:00.984 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.984 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.984 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.984 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.985 INFO:teuthology.orchestra.run.smithi052.stdout: "min_score": { 2024-04-02T14:04:00.985 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_score", 2024-04-02T14:04:00.985 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-04-02T14:04:00.985 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.985 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.985 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-04-02T14:04:00.985 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.985 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.986 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.986 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-02T14:04:00.986 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.986 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.986 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.987 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.987 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2024-04-02T14:04:00.987 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2024-04-02T14:04:00.987 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.987 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.987 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.988 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "upmap", 2024-04-02T14:04:00.988 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.988 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.988 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:00.988 INFO:teuthology.orchestra.run.smithi052.stdout: "crush-compat", 2024-04-02T14:04:00.988 INFO:teuthology.orchestra.run.smithi052.stdout: "none", 2024-04-02T14:04:00.989 INFO:teuthology.orchestra.run.smithi052.stdout: "read", 2024-04-02T14:04:00.989 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap", 2024-04-02T14:04:00.989 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap-read" 2024-04-02T14:04:00.989 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:00.989 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Balancer mode", 2024-04-02T14:04:00.989 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.989 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.989 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.990 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.990 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_ids": { 2024-04-02T14:04:00.990 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_ids", 2024-04-02T14:04:00.990 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:00.990 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.990 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.990 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:00.991 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.991 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.991 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.991 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-02T14:04:00.991 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.991 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.991 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.991 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.991 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-04-02T14:04:00.992 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-04-02T14:04:00.992 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:00.992 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.992 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.992 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-04-02T14:04:00.992 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.992 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.992 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.993 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-02T14:04:00.993 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.993 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.993 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.993 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.993 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_deviation": { 2024-04-02T14:04:00.993 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_deviation", 2024-04-02T14:04:00.994 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:00.994 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.994 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.994 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-04-02T14:04:00.994 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-04-02T14:04:00.994 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.994 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.994 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-02T14:04:00.995 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-02T14:04:00.995 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.995 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.995 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.995 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_optimizations": { 2024-04-02T14:04:00.995 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_optimizations", 2024-04-02T14:04:00.995 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-04-02T14:04:00.995 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.995 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:00.996 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-04-02T14:04:00.996 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.996 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.996 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.996 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-02T14:04:00.996 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.996 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:00.997 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:00.997 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.997 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:00.997 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:00.997 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:00.997 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm", 2024-04-02T14:04:00.998 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:00.998 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:00.998 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:00.998 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_down_multiplier": { 2024-04-02T14:04:00.998 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_down_multiplier", 2024-04-02T14:04:00.998 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-04-02T14:04:00.998 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:00.998 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:00.999 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2024-04-02T14:04:00.999 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:00.999 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:00.999 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:00.999 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-02T14:04:00.999 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:00.999 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.000 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.000 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.000 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_refresh_rate": { 2024-04-02T14:04:01.000 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_refresh_rate", 2024-04-02T14:04:01.000 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.000 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.000 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.000 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "20", 2024-04-02T14:04:01.001 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.001 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.001 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.001 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-02T14:04:01.001 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.001 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.001 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.001 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.001 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_starting_port": { 2024-04-02T14:04:01.002 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_starting_port", 2024-04-02T14:04:01.002 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.002 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.002 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.002 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4721", 2024-04-02T14:04:01.002 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.002 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.002 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.003 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-02T14:04:01.003 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.003 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.003 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.003 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.003 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_ptrace": { 2024-04-02T14:04:01.003 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_ptrace", 2024-04-02T14:04:01.003 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.004 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.004 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.004 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.004 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.004 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.004 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.004 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-02T14:04:01.004 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-02T14:04:01.005 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.005 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.005 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.005 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_interval": { 2024-04-02T14:04:01.005 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_interval", 2024-04-02T14:04:01.005 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.005 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.005 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.005 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-04-02T14:04:01.006 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.006 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.006 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.006 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-02T14:04:01.006 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.006 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.006 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.007 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.007 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_memory_target_ratio": { 2024-04-02T14:04:01.007 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_memory_target_ratio", 2024-04-02T14:04:01.007 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-04-02T14:04:01.007 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.007 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.007 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.7", 2024-04-02T14:04:01.008 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.008 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.008 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.008 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-02T14:04:01.008 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.008 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.008 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.008 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.009 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm_log_destination": { 2024-04-02T14:04:01.009 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm_log_destination", 2024-04-02T14:04:01.009 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.009 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.009 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.009 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.009 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.009 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.010 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.010 INFO:teuthology.orchestra.run.smithi052.stdout: "file", 2024-04-02T14:04:01.010 INFO:teuthology.orchestra.run.smithi052.stdout: "file,syslog", 2024-04-02T14:04:01.010 INFO:teuthology.orchestra.run.smithi052.stdout: "syslog" 2024-04-02T14:04:01.010 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.010 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-02T14:04:01.011 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.011 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.011 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.011 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.011 INFO:teuthology.orchestra.run.smithi052.stdout: "cgroups_split": { 2024-04-02T14:04:01.011 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cgroups_split", 2024-04-02T14:04:01.011 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.011 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.012 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.012 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.012 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.012 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.012 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.012 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-02T14:04:01.012 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.012 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.012 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.013 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.013 INFO:teuthology.orchestra.run.smithi052.stdout: "config_checks_enabled": { 2024-04-02T14:04:01.013 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_checks_enabled", 2024-04-02T14:04:01.013 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.013 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.013 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.013 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.013 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.014 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.014 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.014 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-02T14:04:01.014 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.014 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.014 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.014 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.015 INFO:teuthology.orchestra.run.smithi052.stdout: "config_dashboard": { 2024-04-02T14:04:01.015 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_dashboard", 2024-04-02T14:04:01.015 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.015 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.015 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.015 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.015 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.015 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.016 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.016 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-02T14:04:01.016 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.016 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.016 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.016 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.016 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_alertmanager": { 2024-04-02T14:04:01.016 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_alertmanager", 2024-04-02T14:04:01.017 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.017 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.017 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.017 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-02T14:04:01.017 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.017 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.017 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.017 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-04-02T14:04:01.018 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.018 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.018 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.018 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.018 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_base": { 2024-04-02T14:04:01.018 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_base", 2024-04-02T14:04:01.018 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.018 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.019 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.019 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-02T14:04:01.019 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.019 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.019 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.019 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Container image name, without the tag", 2024-04-02T14:04:01.019 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.019 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.020 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.020 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.020 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_elasticsearch": { 2024-04-02T14:04:01.020 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_elasticsearch", 2024-04-02T14:04:01.020 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.020 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.021 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.021 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-02T14:04:01.021 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.021 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.021 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.021 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "elasticsearch container image", 2024-04-02T14:04:01.021 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.021 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.022 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.022 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.022 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_grafana": { 2024-04-02T14:04:01.022 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_grafana", 2024-04-02T14:04:01.022 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.022 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.022 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.022 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-02T14:04:01.023 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.023 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.023 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.023 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-04-02T14:04:01.023 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.023 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.023 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.023 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.024 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_haproxy": { 2024-04-02T14:04:01.024 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_haproxy", 2024-04-02T14:04:01.024 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.024 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.024 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.024 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-02T14:04:01.024 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.024 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.025 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.025 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "HAproxy container image", 2024-04-02T14:04:01.025 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.025 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.025 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.025 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.025 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_agent": { 2024-04-02T14:04:01.025 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_agent", 2024-04-02T14:04:01.026 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.026 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.026 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.026 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-02T14:04:01.026 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.026 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.026 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.026 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger agent container image", 2024-04-02T14:04:01.027 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.027 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.027 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.027 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.027 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_collector": { 2024-04-02T14:04:01.027 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_collector", 2024-04-02T14:04:01.027 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.027 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.028 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.028 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-02T14:04:01.028 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.028 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.028 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.028 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger collector container image", 2024-04-02T14:04:01.028 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.028 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.028 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.029 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.029 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_query": { 2024-04-02T14:04:01.029 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_query", 2024-04-02T14:04:01.029 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.029 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.029 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.029 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-02T14:04:01.030 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.030 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.030 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.030 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger query container image", 2024-04-02T14:04:01.030 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.030 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.030 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.030 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.031 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_keepalived": { 2024-04-02T14:04:01.031 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_keepalived", 2024-04-02T14:04:01.031 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.031 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.031 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.031 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-02T14:04:01.031 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.031 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.032 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.032 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Keepalived container image", 2024-04-02T14:04:01.032 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.032 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.032 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.032 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.032 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_loki": { 2024-04-02T14:04:01.032 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_loki", 2024-04-02T14:04:01.032 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.033 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.033 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.033 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-02T14:04:01.033 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.033 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.033 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.033 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Loki container image", 2024-04-02T14:04:01.034 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.034 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.034 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.034 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.034 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_node_exporter": { 2024-04-02T14:04:01.034 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_node_exporter", 2024-04-02T14:04:01.034 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.035 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.035 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.035 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-02T14:04:01.035 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.035 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.035 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.035 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-04-02T14:04:01.036 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.036 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.036 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.036 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.036 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_nvmeof": { 2024-04-02T14:04:01.036 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_nvmeof", 2024-04-02T14:04:01.036 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.036 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.037 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.037 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-04-02T14:04:01.037 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.037 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.037 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.037 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Nvme-of container image", 2024-04-02T14:04:01.037 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.037 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.037 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.038 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.038 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_prometheus": { 2024-04-02T14:04:01.038 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_prometheus", 2024-04-02T14:04:01.038 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.038 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.038 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.038 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-02T14:04:01.038 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.039 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.039 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.039 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-04-02T14:04:01.039 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.039 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.039 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.040 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.040 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_promtail": { 2024-04-02T14:04:01.040 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_promtail", 2024-04-02T14:04:01.040 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.040 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.040 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.040 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-02T14:04:01.040 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.040 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.041 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.041 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Promtail container image", 2024-04-02T14:04:01.041 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.041 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.041 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.041 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.041 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_samba": { 2024-04-02T14:04:01.041 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_samba", 2024-04-02T14:04:01.042 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.042 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.042 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.042 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-02T14:04:01.042 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.042 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.042 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.043 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Samba/SMB container image", 2024-04-02T14:04:01.043 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.043 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.043 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.043 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.043 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_snmp_gateway": { 2024-04-02T14:04:01.043 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_snmp_gateway", 2024-04-02T14:04:01.044 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.044 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.044 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.044 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-02T14:04:01.044 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.044 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.044 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.045 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SNMP Gateway container image", 2024-04-02T14:04:01.045 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.045 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.045 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.045 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.045 INFO:teuthology.orchestra.run.smithi052.stdout: "container_init": { 2024-04-02T14:04:01.045 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_init", 2024-04-02T14:04:01.045 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.045 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.046 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.046 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.046 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.046 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.046 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.046 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Run podman/docker with `--init`", 2024-04-02T14:04:01.046 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.047 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.047 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.047 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.047 INFO:teuthology.orchestra.run.smithi052.stdout: "daemon_cache_timeout": { 2024-04-02T14:04:01.047 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "daemon_cache_timeout", 2024-04-02T14:04:01.047 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.047 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.048 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.048 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-04-02T14:04:01.048 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.048 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.048 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.048 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-02T14:04:01.048 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.049 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.049 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.049 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.049 INFO:teuthology.orchestra.run.smithi052.stdout: "default_cephadm_command_timeout": { 2024-04-02T14:04:01.049 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_cephadm_command_timeout", 2024-04-02T14:04:01.049 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.050 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.050 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.050 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "900", 2024-04-02T14:04:01.050 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.050 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.050 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.050 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-02T14:04:01.050 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.051 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.051 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.051 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.051 INFO:teuthology.orchestra.run.smithi052.stdout: "default_registry": { 2024-04-02T14:04:01.051 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_registry", 2024-04-02T14:04:01.051 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.051 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.051 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.052 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io", 2024-04-02T14:04:01.052 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.052 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.052 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.052 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-02T14:04:01.052 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.052 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.052 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.053 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.053 INFO:teuthology.orchestra.run.smithi052.stdout: "device_cache_timeout": { 2024-04-02T14:04:01.053 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_cache_timeout", 2024-04-02T14:04:01.053 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.053 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.053 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.053 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1800", 2024-04-02T14:04:01.053 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.053 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.054 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.054 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache device inventory", 2024-04-02T14:04:01.054 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.054 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.054 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.054 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.055 INFO:teuthology.orchestra.run.smithi052.stdout: "device_enhanced_scan": { 2024-04-02T14:04:01.055 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_enhanced_scan", 2024-04-02T14:04:01.055 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.055 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.055 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.055 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.055 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.055 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.056 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.056 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-02T14:04:01.056 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.056 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.056 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.056 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.056 INFO:teuthology.orchestra.run.smithi052.stdout: "facts_cache_timeout": { 2024-04-02T14:04:01.056 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "facts_cache_timeout", 2024-04-02T14:04:01.057 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.057 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.057 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.057 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-04-02T14:04:01.057 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.057 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.057 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.057 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache host facts data", 2024-04-02T14:04:01.058 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.058 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.058 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.058 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.058 INFO:teuthology.orchestra.run.smithi052.stdout: "grafana_dashboards_path": { 2024-04-02T14:04:01.058 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "grafana_dashboards_path", 2024-04-02T14:04:01.059 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.059 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.059 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.059 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-02T14:04:01.060 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.060 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.060 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.060 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-02T14:04:01.060 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.060 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.060 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.061 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.061 INFO:teuthology.orchestra.run.smithi052.stdout: "host_check_interval": { 2024-04-02T14:04:01.061 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "host_check_interval", 2024-04-02T14:04:01.061 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.061 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.061 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.062 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-04-02T14:04:01.062 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.062 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.062 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.062 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to perform a host check", 2024-04-02T14:04:01.062 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.062 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.063 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.063 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.063 INFO:teuthology.orchestra.run.smithi052.stdout: "hw_monitoring": { 2024-04-02T14:04:01.063 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hw_monitoring", 2024-04-02T14:04:01.063 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.063 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.063 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.063 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.064 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.064 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.064 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.064 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-02T14:04:01.064 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.064 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.064 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.064 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.065 INFO:teuthology.orchestra.run.smithi052.stdout: "inventory_list_all": { 2024-04-02T14:04:01.065 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "inventory_list_all", 2024-04-02T14:04:01.065 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.065 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.065 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.065 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.065 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.066 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.066 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.066 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-02T14:04:01.066 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.066 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.066 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.066 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.067 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.067 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.067 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.067 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.067 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.067 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.067 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.067 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.068 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.068 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.068 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.068 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.068 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.068 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.068 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.069 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.069 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.069 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.069 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.069 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.069 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.069 INFO:teuthology.orchestra.run.smithi052.stdout: "log_refresh_metadata": { 2024-04-02T14:04:01.069 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_refresh_metadata", 2024-04-02T14:04:01.070 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.070 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.070 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.070 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.071 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.071 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.071 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.071 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-02T14:04:01.071 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.071 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.071 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.072 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.072 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.072 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.072 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.072 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.072 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.072 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.073 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.073 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.073 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.073 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-02T14:04:01.073 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.073 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.073 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.073 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.074 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.074 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.074 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.074 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.074 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.074 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.074 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.074 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.075 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.075 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.075 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.075 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.075 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.075 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.075 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.075 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.076 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.076 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.076 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.076 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.076 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.076 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.076 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.076 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.077 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.077 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.077 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.077 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.077 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.077 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.077 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.077 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.078 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.078 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.078 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.078 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-02T14:04:01.078 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-02T14:04:01.078 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.078 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.078 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.079 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.079 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.079 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.079 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.079 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-02T14:04:01.079 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.079 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.079 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.080 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.080 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-02T14:04:01.080 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-02T14:04:01.080 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.080 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.080 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.081 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "*", 2024-04-02T14:04:01.081 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.081 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.081 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.081 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-02T14:04:01.081 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.081 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.082 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.082 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.082 INFO:teuthology.orchestra.run.smithi052.stdout: "max_count_per_host": { 2024-04-02T14:04:01.082 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_count_per_host", 2024-04-02T14:04:01.082 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.082 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.083 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.083 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-04-02T14:04:01.083 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.083 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.083 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.083 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of daemons per service per host", 2024-04-02T14:04:01.084 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.084 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.084 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.084 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.084 INFO:teuthology.orchestra.run.smithi052.stdout: "max_osd_draining_count": { 2024-04-02T14:04:01.084 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_osd_draining_count", 2024-04-02T14:04:01.084 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.084 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.084 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.085 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-04-02T14:04:01.085 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.085 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.085 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.085 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-02T14:04:01.085 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.085 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.086 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.086 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.086 INFO:teuthology.orchestra.run.smithi052.stdout: "migration_current": { 2024-04-02T14:04:01.086 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "migration_current", 2024-04-02T14:04:01.086 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.086 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.086 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.086 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.087 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.087 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.087 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.087 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "internal - do not modify", 2024-04-02T14:04:01.087 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.087 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.088 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.088 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.088 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2024-04-02T14:04:01.088 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2024-04-02T14:04:01.088 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.088 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.088 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.088 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "root", 2024-04-02T14:04:01.088 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.089 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.089 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.089 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm-package", 2024-04-02T14:04:01.089 INFO:teuthology.orchestra.run.smithi052.stdout: "root" 2024-04-02T14:04:01.089 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.089 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "mode for remote execution of cephadm", 2024-04-02T14:04:01.089 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.090 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.090 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.090 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.090 INFO:teuthology.orchestra.run.smithi052.stdout: "oob_default_addr": { 2024-04-02T14:04:01.090 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "oob_default_addr", 2024-04-02T14:04:01.090 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.090 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.090 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.091 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "169.254.1.1", 2024-04-02T14:04:01.091 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.091 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.091 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.091 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-02T14:04:01.091 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.092 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.092 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.092 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.092 INFO:teuthology.orchestra.run.smithi052.stdout: "prometheus_alerts_path": { 2024-04-02T14:04:01.092 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus_alerts_path", 2024-04-02T14:04:01.092 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.092 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.092 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.093 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-02T14:04:01.093 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.093 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.093 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.093 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-02T14:04:01.093 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.093 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.093 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.094 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.094 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_insecure": { 2024-04-02T14:04:01.094 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_insecure", 2024-04-02T14:04:01.094 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.094 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.094 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.094 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.094 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.095 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.095 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.095 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-02T14:04:01.095 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.095 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.095 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.095 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.095 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_password": { 2024-04-02T14:04:01.095 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_password", 2024-04-02T14:04:01.096 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.096 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.096 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.096 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.096 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.096 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.097 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.097 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-02T14:04:01.097 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.097 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.097 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.097 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.097 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_url": { 2024-04-02T14:04:01.097 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_url", 2024-04-02T14:04:01.098 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.098 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.098 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.098 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.098 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.098 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.098 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.098 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-02T14:04:01.099 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.099 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.099 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.099 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.099 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_username": { 2024-04-02T14:04:01.099 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_username", 2024-04-02T14:04:01.099 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.099 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.100 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.100 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.100 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.100 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.100 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.100 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-02T14:04:01.100 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.100 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.101 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.101 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.101 INFO:teuthology.orchestra.run.smithi052.stdout: "secure_monitoring_stack": { 2024-04-02T14:04:01.101 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "secure_monitoring_stack", 2024-04-02T14:04:01.101 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.101 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.101 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.102 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.102 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.102 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.102 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.102 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-02T14:04:01.102 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.103 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.103 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.103 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.103 INFO:teuthology.orchestra.run.smithi052.stdout: "service_discovery_port": { 2024-04-02T14:04:01.103 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "service_discovery_port", 2024-04-02T14:04:01.103 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.103 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.104 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.104 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8765", 2024-04-02T14:04:01.104 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.104 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.104 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.104 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "cephadm service discovery port", 2024-04-02T14:04:01.104 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.105 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.105 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.105 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.105 INFO:teuthology.orchestra.run.smithi052.stdout: "ssh_config_file": { 2024-04-02T14:04:01.105 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssh_config_file", 2024-04-02T14:04:01.105 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.105 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.105 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.106 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.106 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.106 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.106 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.106 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-02T14:04:01.106 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.106 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.106 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.107 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.107 INFO:teuthology.orchestra.run.smithi052.stdout: "use_agent": { 2024-04-02T14:04:01.107 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_agent", 2024-04-02T14:04:01.107 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.107 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.107 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.108 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.108 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.108 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.108 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.108 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-02T14:04:01.108 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.108 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.109 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.109 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.109 INFO:teuthology.orchestra.run.smithi052.stdout: "use_repo_digest": { 2024-04-02T14:04:01.109 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_repo_digest", 2024-04-02T14:04:01.109 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.109 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.109 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.109 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.110 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.110 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.110 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.110 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-02T14:04:01.110 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.110 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.110 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.111 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.111 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_failed_host_check": { 2024-04-02T14:04:01.111 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_failed_host_check", 2024-04-02T14:04:01.111 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.111 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.111 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.111 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.112 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.112 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.112 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.112 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if the host check fails", 2024-04-02T14:04:01.112 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.112 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.113 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.113 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.113 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_daemons": { 2024-04-02T14:04:01.113 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_daemons", 2024-04-02T14:04:01.113 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.113 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.113 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.114 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.114 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.114 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.114 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.114 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-02T14:04:01.114 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.115 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.115 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.115 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.115 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_hosts": { 2024-04-02T14:04:01.115 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_hosts", 2024-04-02T14:04:01.115 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.115 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.115 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.115 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.116 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.116 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.116 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.116 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-02T14:04:01.116 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.116 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.116 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.116 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.117 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.117 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.117 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.117 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crash", 2024-04-02T14:04:01.117 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.117 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.117 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.117 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.118 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.118 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.118 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.118 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.118 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.118 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.118 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.119 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.119 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.119 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.119 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.119 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.119 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.119 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.120 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.120 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.120 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.120 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.120 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.120 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.120 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.121 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.121 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.121 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.121 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.121 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.121 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.121 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.122 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.122 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.122 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.122 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.122 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.122 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.122 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.123 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.123 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.123 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.123 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.123 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.123 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.124 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.124 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.124 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.124 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.124 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.124 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.124 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.124 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.125 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.125 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.125 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.125 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.125 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.125 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.125 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.125 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.125 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.126 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.126 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.126 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.126 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.126 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.126 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.126 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.127 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.127 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.127 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.127 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.127 INFO:teuthology.orchestra.run.smithi052.stdout: "retain_interval": { 2024-04-02T14:04:01.127 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retain_interval", 2024-04-02T14:04:01.127 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.127 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.128 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.128 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "31536000", 2024-04-02T14:04:01.128 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.128 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.128 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.128 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-02T14:04:01.128 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.128 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.128 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.129 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.129 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_recent_interval": { 2024-04-02T14:04:01.129 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_recent_interval", 2024-04-02T14:04:01.129 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.129 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.129 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.129 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1209600", 2024-04-02T14:04:01.129 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.130 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.130 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.130 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-02T14:04:01.130 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.130 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.130 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.130 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.130 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.131 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.131 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.131 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dashboard", 2024-04-02T14:04:01.131 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.131 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.131 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.131 INFO:teuthology.orchestra.run.smithi052.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-02T14:04:01.132 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-02T14:04:01.132 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.132 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.132 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.133 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-04-02T14:04:01.133 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.133 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.133 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.134 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.134 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.134 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.135 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.135 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.135 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_HOST": { 2024-04-02T14:04:01.135 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-02T14:04:01.136 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.136 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.136 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.136 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.137 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.137 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.137 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.137 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.137 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.137 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.137 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.137 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.138 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-02T14:04:01.138 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-02T14:04:01.138 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.138 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.138 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.138 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.138 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.138 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.139 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.139 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.139 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.139 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.139 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.139 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.139 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_ENABLED": { 2024-04-02T14:04:01.139 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_ENABLED", 2024-04-02T14:04:01.140 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.140 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.140 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.140 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.140 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.140 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.141 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.141 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.141 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.141 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.141 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.141 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.141 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-02T14:04:01.141 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-02T14:04:01.142 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.142 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.142 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.142 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.142 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.143 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.143 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.143 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.144 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.144 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.144 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.144 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.144 INFO:teuthology.orchestra.run.smithi052.stdout: "ENABLE_BROWSABLE_API": { 2024-04-02T14:04:01.144 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-02T14:04:01.144 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.145 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.145 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.145 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.145 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.145 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.145 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.146 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.146 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.146 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.146 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.146 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.146 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-02T14:04:01.146 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-02T14:04:01.147 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.147 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.147 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.147 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.147 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.147 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.147 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.147 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.148 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.148 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.148 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.148 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.148 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-02T14:04:01.148 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-02T14:04:01.148 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.149 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.149 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.149 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.149 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.149 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.149 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.149 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.149 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.149 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.150 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.150 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.150 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-02T14:04:01.150 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-02T14:04:01.150 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.150 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.150 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.151 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.151 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.151 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.151 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.151 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.151 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.151 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.151 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.152 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.152 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-02T14:04:01.152 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-02T14:04:01.152 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.152 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.152 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.152 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.152 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.152 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.153 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.153 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.153 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.153 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.153 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.153 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.153 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-02T14:04:01.153 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-02T14:04:01.154 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.154 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.154 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.154 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.154 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.154 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.155 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.155 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.155 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.155 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.155 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.155 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.156 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-02T14:04:01.156 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-02T14:04:01.156 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.156 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.156 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.156 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.156 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.157 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.157 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.157 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.157 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.157 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.157 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.157 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.157 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-02T14:04:01.158 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-02T14:04:01.158 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.158 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.158 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.158 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.158 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.158 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.159 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.159 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.159 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.159 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.159 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.159 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.159 INFO:teuthology.orchestra.run.smithi052.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-02T14:04:01.159 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-02T14:04:01.160 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.160 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.160 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.160 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.160 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.160 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.160 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.160 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.161 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.161 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.161 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.161 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.161 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_PASSWORD": { 2024-04-02T14:04:01.161 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-02T14:04:01.161 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.162 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.162 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.162 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-04-02T14:04:01.162 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.162 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.162 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.162 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.162 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.163 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.163 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.163 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.163 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-02T14:04:01.163 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-02T14:04:01.163 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.163 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.163 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.164 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.164 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.164 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.164 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.164 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.164 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.164 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.165 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.165 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.165 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_URL": { 2024-04-02T14:04:01.165 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_URL", 2024-04-02T14:04:01.165 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.165 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.165 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.165 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.166 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.166 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.166 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.166 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.166 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.166 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.167 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.167 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.167 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_USERNAME": { 2024-04-02T14:04:01.167 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-02T14:04:01.167 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.167 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.167 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.167 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-04-02T14:04:01.168 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.168 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.168 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.168 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.168 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.168 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.168 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.168 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.169 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-02T14:04:01.169 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-02T14:04:01.169 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.169 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.169 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.169 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.169 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.169 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.169 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.170 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.170 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.170 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.170 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.170 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.170 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-02T14:04:01.170 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-02T14:04:01.170 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.171 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.171 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.171 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.171 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.171 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.171 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.171 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.171 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.172 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.172 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.172 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.172 INFO:teuthology.orchestra.run.smithi052.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-02T14:04:01.172 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-02T14:04:01.172 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.172 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.172 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.172 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.173 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.173 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.173 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.173 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.173 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.173 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.174 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.174 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.174 INFO:teuthology.orchestra.run.smithi052.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-02T14:04:01.174 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-02T14:04:01.174 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.174 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.174 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.174 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.175 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.175 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.175 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.175 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.175 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.176 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.176 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.176 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.176 INFO:teuthology.orchestra.run.smithi052.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-02T14:04:01.176 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-02T14:04:01.176 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.176 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.176 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.177 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "[]", 2024-04-02T14:04:01.177 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.177 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.177 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.177 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.177 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.177 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.178 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.178 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.178 INFO:teuthology.orchestra.run.smithi052.stdout: "MULTICLUSTER_CONFIG": { 2024-04-02T14:04:01.178 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-02T14:04:01.178 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.178 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.178 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.178 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "{}", 2024-04-02T14:04:01.179 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.179 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.179 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.179 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.179 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.179 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.179 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.179 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.180 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_HOST": { 2024-04-02T14:04:01.180 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-02T14:04:01.180 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.180 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.180 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.180 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.180 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.180 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.181 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.181 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.181 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.181 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.181 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.181 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.181 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-02T14:04:01.181 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-02T14:04:01.181 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.182 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.182 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.182 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.182 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.182 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.182 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.183 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.183 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.183 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.183 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.184 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.184 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-02T14:04:01.184 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-02T14:04:01.184 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.184 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.185 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.185 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.185 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.185 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.185 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.185 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.185 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.185 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.186 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.186 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.186 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-02T14:04:01.186 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-02T14:04:01.186 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.187 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.187 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.187 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.187 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.187 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.187 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.187 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.187 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.188 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.188 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.188 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.188 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-02T14:04:01.188 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-02T14:04:01.188 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.189 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.189 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.189 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.189 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.189 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.189 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.189 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.190 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.190 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.190 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.190 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.190 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-02T14:04:01.190 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-02T14:04:01.190 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.190 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.191 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.191 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.191 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.191 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.191 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.191 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.191 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.191 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.192 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.192 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.192 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-02T14:04:01.192 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-02T14:04:01.192 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.193 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.193 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.193 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.193 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.193 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.193 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.194 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.194 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.194 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.194 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.194 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.194 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-02T14:04:01.194 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-02T14:04:01.195 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.195 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.195 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.195 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.195 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.195 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.195 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.195 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.196 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.196 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.196 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.196 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.196 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-02T14:04:01.196 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-02T14:04:01.196 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.197 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.197 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.197 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.197 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.197 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.197 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.197 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.197 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.198 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.198 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.198 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.198 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_ENABLED": { 2024-04-02T14:04:01.198 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-02T14:04:01.198 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.198 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.198 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.199 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.199 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.199 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.199 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.199 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.199 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.199 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.199 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.199 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.200 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-02T14:04:01.200 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-02T14:04:01.200 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.200 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.200 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.200 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-02T14:04:01.200 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.200 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.201 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.201 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.201 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.201 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.201 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.201 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.201 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-02T14:04:01.201 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-02T14:04:01.201 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.202 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.202 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.202 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-04-02T14:04:01.202 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.202 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.202 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.202 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.203 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.203 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.203 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.203 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.203 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-02T14:04:01.203 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-02T14:04:01.203 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.203 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.204 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.204 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8", 2024-04-02T14:04:01.204 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.204 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.204 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.204 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.205 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.205 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.205 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.205 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.205 INFO:teuthology.orchestra.run.smithi052.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-02T14:04:01.205 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-02T14:04:01.205 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.205 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.206 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.206 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "45", 2024-04-02T14:04:01.206 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.206 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.206 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.206 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.206 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.206 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.207 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.207 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.207 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ACCESS_KEY": { 2024-04-02T14:04:01.207 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-02T14:04:01.207 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.207 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.208 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.208 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.208 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.208 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.208 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.208 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.208 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.208 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.209 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.209 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.209 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-02T14:04:01.209 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-02T14:04:01.209 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.209 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.209 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.209 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-04-02T14:04:01.210 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.210 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.210 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.210 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.210 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.210 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.211 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.211 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.211 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SECRET_KEY": { 2024-04-02T14:04:01.211 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-02T14:04:01.211 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.212 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.212 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.212 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.212 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.212 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.213 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.213 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.213 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.213 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.213 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.213 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.213 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SSL_VERIFY": { 2024-04-02T14:04:01.213 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-02T14:04:01.213 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.214 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.214 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.214 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.214 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.214 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.214 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.215 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.215 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.215 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.215 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.215 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.215 INFO:teuthology.orchestra.run.smithi052.stdout: "UNSAFE_TLS_v1_2": { 2024-04-02T14:04:01.215 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-02T14:04:01.215 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.216 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.216 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.216 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.216 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.216 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.216 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.216 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.216 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.217 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.217 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.217 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.217 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-02T14:04:01.217 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-02T14:04:01.217 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.218 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.218 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.218 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-04-02T14:04:01.218 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.218 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.218 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.218 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.219 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.219 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.219 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.219 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.219 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-02T14:04:01.219 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-02T14:04:01.219 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.220 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.220 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.220 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-04-02T14:04:01.220 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.220 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.220 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.220 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.220 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.221 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.221 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.221 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.221 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-02T14:04:01.221 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-02T14:04:01.221 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.221 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.221 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.221 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-04-02T14:04:01.222 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.222 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.222 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.222 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.222 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.222 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.223 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.223 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.223 INFO:teuthology.orchestra.run.smithi052.stdout: "cross_origin_url": { 2024-04-02T14:04:01.223 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cross_origin_url", 2024-04-02T14:04:01.223 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.223 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.223 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.223 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.223 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.224 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.224 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.224 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.224 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.224 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.224 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.224 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.225 INFO:teuthology.orchestra.run.smithi052.stdout: "crt_file": { 2024-04-02T14:04:01.225 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crt_file", 2024-04-02T14:04:01.225 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.225 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.225 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.225 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.225 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.225 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.225 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.226 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.226 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.226 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.226 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.226 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.226 INFO:teuthology.orchestra.run.smithi052.stdout: "debug": { 2024-04-02T14:04:01.226 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "debug", 2024-04-02T14:04:01.226 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.227 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.227 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.227 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.227 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.227 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.227 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.227 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable/disable debug options", 2024-04-02T14:04:01.227 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.227 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.228 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.228 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.228 INFO:teuthology.orchestra.run.smithi052.stdout: "jwt_token_ttl": { 2024-04-02T14:04:01.228 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "jwt_token_ttl", 2024-04-02T14:04:01.228 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.228 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.229 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.229 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "28800", 2024-04-02T14:04:01.229 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.229 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.229 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.229 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.229 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.229 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.229 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.230 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.230 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2024-04-02T14:04:01.230 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2024-04-02T14:04:01.230 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.230 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.230 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.230 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.230 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.231 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.231 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.231 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.231 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.231 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.231 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.231 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.231 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.231 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.233 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.233 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.233 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.233 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.234 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.234 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.234 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.234 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.234 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.234 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.234 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.235 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.235 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.235 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.235 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.235 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.235 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.235 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.235 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.236 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.236 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.236 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.236 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.236 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.236 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.236 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.237 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.237 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.237 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.237 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.237 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.237 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.237 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.238 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.238 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.238 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.238 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.238 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.238 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.239 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.239 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.239 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.239 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.239 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.239 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.239 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.240 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.240 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.240 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.240 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.240 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.240 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.240 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.240 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.241 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.241 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.241 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.241 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.241 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.241 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.241 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.242 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.242 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.242 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.242 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.242 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.242 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.242 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.243 INFO:teuthology.orchestra.run.smithi052.stdout: "motd": { 2024-04-02T14:04:01.243 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "motd", 2024-04-02T14:04:01.243 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.243 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.243 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.243 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.243 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.243 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.243 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.244 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "The message of the day", 2024-04-02T14:04:01.244 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.244 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.244 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.244 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.244 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect_resolve_ip_addr": { 2024-04-02T14:04:01.244 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "redirect_resolve_ip_addr", 2024-04-02T14:04:01.245 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.245 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.245 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.245 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.245 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.245 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.245 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.245 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.245 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.246 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.246 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.246 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.246 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-04-02T14:04:01.246 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-04-02T14:04:01.247 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.247 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.247 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.247 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2024-04-02T14:04:01.248 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.248 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.248 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.248 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.249 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.249 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.249 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.249 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.250 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-04-02T14:04:01.250 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-04-02T14:04:01.250 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.250 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.251 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.251 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8080", 2024-04-02T14:04:01.251 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.251 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.252 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.252 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.252 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.252 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.253 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.253 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.253 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2024-04-02T14:04:01.253 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2024-04-02T14:04:01.254 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.254 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.254 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.255 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.255 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.255 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.255 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.256 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.256 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.256 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.256 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.257 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.257 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl_server_port": { 2024-04-02T14:04:01.257 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl_server_port", 2024-04-02T14:04:01.257 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.258 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.258 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.258 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8443", 2024-04-02T14:04:01.258 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.259 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.259 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.259 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.260 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.260 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.260 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.260 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.261 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2024-04-02T14:04:01.261 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2024-04-02T14:04:01.261 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.261 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.262 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.262 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "redirect", 2024-04-02T14:04:01.262 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.262 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.263 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.263 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.263 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect" 2024-04-02T14:04:01.263 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.264 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.264 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.264 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.265 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.265 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.265 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2024-04-02T14:04:01.265 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2024-04-02T14:04:01.266 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.266 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.266 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.267 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2024-04-02T14:04:01.267 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2024-04-02T14:04:01.267 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2024-04-02T14:04:01.267 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.268 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.268 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.268 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.268 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.268 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.269 INFO:teuthology.orchestra.run.smithi052.stdout: "url_prefix": { 2024-04-02T14:04:01.269 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url_prefix", 2024-04-02T14:04:01.269 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.269 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.269 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.270 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.270 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.270 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.270 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.270 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.271 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.271 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.271 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.271 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.272 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.272 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.272 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.272 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "devicehealth", 2024-04-02T14:04:01.272 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.273 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.273 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.273 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_monitoring": { 2024-04-02T14:04:01.273 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_monitoring", 2024-04-02T14:04:01.273 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.274 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.274 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.274 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.274 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.274 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.275 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.275 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "monitor device health metrics", 2024-04-02T14:04:01.275 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.275 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.275 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.275 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.275 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.275 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.275 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.276 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.276 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.276 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.276 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.276 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.276 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.276 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.276 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.277 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.277 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.277 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.277 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.277 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.277 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.277 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.277 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.278 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.278 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.278 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.278 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.278 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.278 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.278 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.278 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.279 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.279 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.279 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.279 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.279 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.279 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.279 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.279 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.280 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.280 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.280 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.280 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.280 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.280 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.280 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.280 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.281 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.281 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.281 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.281 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.281 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.281 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.281 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.282 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.282 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.282 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.282 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.282 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.283 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.283 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.283 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.283 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.283 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.283 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.283 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.284 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.284 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.284 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.284 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.284 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.284 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.284 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.284 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.285 INFO:teuthology.orchestra.run.smithi052.stdout: "mark_out_threshold": { 2024-04-02T14:04:01.285 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mark_out_threshold", 2024-04-02T14:04:01.285 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.285 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.285 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.285 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2419200", 2024-04-02T14:04:01.285 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.285 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.286 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.286 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-02T14:04:01.286 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.286 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.286 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.286 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.286 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_name": { 2024-04-02T14:04:01.287 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_name", 2024-04-02T14:04:01.287 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.287 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.287 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.287 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "device_health_metrics", 2024-04-02T14:04:01.287 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.287 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.287 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.288 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-02T14:04:01.288 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.288 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.288 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.288 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.288 INFO:teuthology.orchestra.run.smithi052.stdout: "retention_period": { 2024-04-02T14:04:01.288 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retention_period", 2024-04-02T14:04:01.289 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.289 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.289 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.290 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15552000", 2024-04-02T14:04:01.290 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.290 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.290 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.291 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain device health metrics", 2024-04-02T14:04:01.291 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.291 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.291 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.291 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.291 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_frequency": { 2024-04-02T14:04:01.291 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_frequency", 2024-04-02T14:04:01.291 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.291 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.292 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.292 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2024-04-02T14:04:01.292 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.292 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.292 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.292 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-02T14:04:01.292 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.292 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.293 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.293 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.293 INFO:teuthology.orchestra.run.smithi052.stdout: "self_heal": { 2024-04-02T14:04:01.293 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "self_heal", 2024-04-02T14:04:01.293 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.293 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.293 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.293 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.294 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.294 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.294 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.294 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-02T14:04:01.294 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.294 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.294 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.294 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.295 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-04-02T14:04:01.295 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-04-02T14:04:01.295 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.295 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.295 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.295 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-04-02T14:04:01.295 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.295 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.296 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.296 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and check device health", 2024-04-02T14:04:01.296 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.296 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.296 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.296 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.296 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_threshold": { 2024-04-02T14:04:01.296 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_threshold", 2024-04-02T14:04:01.297 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.297 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.297 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.297 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7257600", 2024-04-02T14:04:01.297 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.297 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.297 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.297 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-02T14:04:01.297 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.298 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.298 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.298 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.298 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.298 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.298 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.298 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "diskprediction_local", 2024-04-02T14:04:01.299 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.299 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.299 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.299 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.299 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.299 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.299 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.299 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.299 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.300 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.300 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.300 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.300 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.300 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.300 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.300 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.300 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.301 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.301 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.301 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.301 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.302 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.302 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.302 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.302 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.303 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.303 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.303 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.303 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.304 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.304 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.304 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.305 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.305 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.305 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.305 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.305 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.306 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.306 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.306 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.307 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.307 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.307 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.307 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.308 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.308 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.308 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.308 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.309 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.309 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.309 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.309 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.310 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.310 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.310 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.310 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.311 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.311 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.311 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.311 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.312 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.312 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.312 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.312 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.312 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.312 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.313 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.313 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.313 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.313 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.313 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.313 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.313 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.313 INFO:teuthology.orchestra.run.smithi052.stdout: "predict_interval": { 2024-04-02T14:04:01.314 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predict_interval", 2024-04-02T14:04:01.314 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.314 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.314 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.314 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2024-04-02T14:04:01.314 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.314 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.314 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.315 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.315 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.315 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.315 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.315 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.315 INFO:teuthology.orchestra.run.smithi052.stdout: "predictor_model": { 2024-04-02T14:04:01.315 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predictor_model", 2024-04-02T14:04:01.315 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.316 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.316 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.316 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "prophetstor", 2024-04-02T14:04:01.316 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.316 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.316 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.316 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.316 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.317 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.317 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.317 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.317 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-04-02T14:04:01.317 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-04-02T14:04:01.317 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.317 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.317 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.317 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-04-02T14:04:01.318 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.318 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.318 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.318 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.318 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.318 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.318 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.318 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.319 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.319 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.319 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.319 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "influx", 2024-04-02T14:04:01.319 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": false, 2024-04-02T14:04:01.319 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "influxdb python module not found", 2024-04-02T14:04:01.319 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.319 INFO:teuthology.orchestra.run.smithi052.stdout: "batch_size": { 2024-04-02T14:04:01.320 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "batch_size", 2024-04-02T14:04:01.320 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.320 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.320 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.320 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5000", 2024-04-02T14:04:01.320 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.320 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.320 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.321 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-02T14:04:01.321 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.321 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.321 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.321 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.321 INFO:teuthology.orchestra.run.smithi052.stdout: "database": { 2024-04-02T14:04:01.321 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "database", 2024-04-02T14:04:01.321 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.321 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.322 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.322 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "ceph", 2024-04-02T14:04:01.322 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.322 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.322 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.322 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-02T14:04:01.322 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.322 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.323 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.323 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.323 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": { 2024-04-02T14:04:01.323 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hostname", 2024-04-02T14:04:01.323 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.323 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.323 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.323 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.324 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.324 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.324 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.324 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server hostname", 2024-04-02T14:04:01.324 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.324 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.324 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.324 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.324 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-04-02T14:04:01.325 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-04-02T14:04:01.325 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.325 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.325 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.325 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "30", 2024-04-02T14:04:01.325 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "5", 2024-04-02T14:04:01.325 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.325 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.326 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-02T14:04:01.326 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.326 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.326 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.326 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.326 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.326 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.326 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.327 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.327 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.327 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.327 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.327 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.327 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.327 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.327 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.328 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.328 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.328 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.328 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.328 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.328 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.328 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.328 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.329 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.329 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.329 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.329 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.329 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.329 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.329 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.329 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.330 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.330 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.330 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.330 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.330 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.330 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.330 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.330 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.331 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.331 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.331 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.331 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.331 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.331 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.331 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.331 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.332 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.332 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.332 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.332 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.332 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.332 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.332 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.332 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.333 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.333 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.333 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.333 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.333 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.333 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.333 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.333 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.334 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.334 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.334 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.334 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.334 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.334 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.334 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.334 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.334 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.335 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.335 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.335 INFO:teuthology.orchestra.run.smithi052.stdout: "password": { 2024-04-02T14:04:01.335 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "password", 2024-04-02T14:04:01.335 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.335 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.335 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.336 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.336 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.336 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.336 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.336 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "password of InfluxDB server user", 2024-04-02T14:04:01.336 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.336 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.336 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.337 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.337 INFO:teuthology.orchestra.run.smithi052.stdout: "port": { 2024-04-02T14:04:01.337 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "port", 2024-04-02T14:04:01.337 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.337 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.337 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.337 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8086", 2024-04-02T14:04:01.337 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.338 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.338 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.338 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server port", 2024-04-02T14:04:01.338 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.338 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.338 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.338 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.338 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2024-04-02T14:04:01.338 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2024-04-02T14:04:01.339 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.339 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.339 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.339 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "false", 2024-04-02T14:04:01.339 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.339 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.339 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.339 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T14:04:01.340 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.340 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.340 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.340 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.340 INFO:teuthology.orchestra.run.smithi052.stdout: "threads": { 2024-04-02T14:04:01.340 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threads", 2024-04-02T14:04:01.340 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.340 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.341 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.341 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-04-02T14:04:01.341 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-04-02T14:04:01.341 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "32", 2024-04-02T14:04:01.341 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.341 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-02T14:04:01.341 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.341 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.342 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.342 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.342 INFO:teuthology.orchestra.run.smithi052.stdout: "username": { 2024-04-02T14:04:01.342 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "username", 2024-04-02T14:04:01.342 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.342 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.342 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.342 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.343 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.343 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.343 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.343 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "username of InfluxDB server user", 2024-04-02T14:04:01.343 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.343 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.343 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.343 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.343 INFO:teuthology.orchestra.run.smithi052.stdout: "verify_ssl": { 2024-04-02T14:04:01.344 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "verify_ssl", 2024-04-02T14:04:01.344 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.344 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.344 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.344 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "true", 2024-04-02T14:04:01.344 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.344 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.344 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.345 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T14:04:01.345 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.345 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.345 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.345 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.345 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.345 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.345 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.346 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "insights", 2024-04-02T14:04:01.346 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.346 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.346 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.346 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.346 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.346 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.346 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.347 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.347 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.347 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.347 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.347 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.347 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.347 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.347 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.348 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.348 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.348 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.348 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.348 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.348 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.348 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.348 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.349 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.349 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.349 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.349 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.349 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.349 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.349 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.349 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.350 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.350 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.350 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.350 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.350 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.350 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.350 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.350 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.351 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.351 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.351 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.351 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.351 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.351 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.351 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.351 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.351 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.352 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.352 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.352 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.352 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.352 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.352 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.352 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.352 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.353 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.353 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.353 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.353 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.353 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.353 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.353 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.353 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.353 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.354 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.354 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.354 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.354 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.354 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.354 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.354 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.354 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.354 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.355 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.355 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.355 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "iostat", 2024-04-02T14:04:01.355 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.355 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.355 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.355 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.355 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.356 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.356 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.356 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.356 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.356 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.356 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.356 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.356 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.357 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.357 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.357 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.357 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.357 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.357 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.357 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.357 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.358 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.358 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.358 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.358 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.358 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.358 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.358 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.359 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.359 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.359 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.359 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.359 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.359 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.359 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.359 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.359 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.360 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.360 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.360 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.360 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.360 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.360 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.360 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.360 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.361 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.361 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.361 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.361 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.361 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.361 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.361 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.361 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.362 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.362 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.362 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.362 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.362 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.362 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.362 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.362 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.363 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.363 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.363 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.363 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.363 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.363 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.363 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.363 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.363 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.364 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.364 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.364 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.364 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.364 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.364 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.364 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "k8sevents", 2024-04-02T14:04:01.364 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.365 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.365 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.365 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_event_retention_days": { 2024-04-02T14:04:01.365 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ceph_event_retention_days", 2024-04-02T14:04:01.365 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.365 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.365 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.365 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7", 2024-04-02T14:04:01.366 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.366 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.366 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.366 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-02T14:04:01.366 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.366 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.366 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.366 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.367 INFO:teuthology.orchestra.run.smithi052.stdout: "config_check_secs": { 2024-04-02T14:04:01.367 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_check_secs", 2024-04-02T14:04:01.367 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.367 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.367 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.367 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-04-02T14:04:01.367 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "10", 2024-04-02T14:04:01.367 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.368 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.368 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-02T14:04:01.368 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.368 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.368 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.368 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.368 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.368 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.368 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.369 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.369 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.369 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.369 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.369 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.369 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.369 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.369 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.370 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.370 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.370 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.370 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.370 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.370 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.370 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.370 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.371 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.371 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.371 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.371 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.371 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.371 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.371 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.371 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.371 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.372 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.372 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.372 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.372 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.372 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.372 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.372 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.372 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.373 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.373 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.373 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.373 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.373 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.373 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.373 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.373 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.374 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.374 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.374 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.374 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.374 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.374 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.374 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.374 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.375 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.375 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.375 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.375 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.375 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.375 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.375 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.375 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.375 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.376 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.376 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.376 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.376 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.376 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.376 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.376 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.376 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.377 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.377 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.377 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.377 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.377 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "localpool", 2024-04-02T14:04:01.377 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.377 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.377 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.377 INFO:teuthology.orchestra.run.smithi052.stdout: "failure_domain": { 2024-04-02T14:04:01.378 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "failure_domain", 2024-04-02T14:04:01.378 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.378 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.378 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.378 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "host", 2024-04-02T14:04:01.378 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.378 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.379 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.379 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "failure domain for any created local pool", 2024-04-02T14:04:01.379 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-02T14:04:01.379 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.379 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.379 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.379 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.379 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.379 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.380 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.380 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.380 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.380 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.380 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.380 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.380 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.380 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.381 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.381 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.381 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.381 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.381 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.381 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.381 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.381 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.382 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.382 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.382 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.382 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.382 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.382 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.383 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.383 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.383 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.383 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.383 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.383 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.383 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.383 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.384 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.384 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.384 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.384 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.384 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.384 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.384 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.384 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.385 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.385 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.385 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.385 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.385 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.385 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.385 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.385 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.385 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.386 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.386 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.386 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.386 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.386 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.386 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.386 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.386 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.387 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.387 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.387 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.387 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.387 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.387 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.387 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.387 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.387 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.388 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.388 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.388 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.388 INFO:teuthology.orchestra.run.smithi052.stdout: "min_size": { 2024-04-02T14:04:01.388 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_size", 2024-04-02T14:04:01.388 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.388 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.388 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.388 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.389 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.389 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.389 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.389 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default min_size for any created local pool", 2024-04-02T14:04:01.389 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-02T14:04:01.389 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.389 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.389 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.389 INFO:teuthology.orchestra.run.smithi052.stdout: "num_rep": { 2024-04-02T14:04:01.390 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "num_rep", 2024-04-02T14:04:01.390 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.390 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.390 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.390 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3", 2024-04-02T14:04:01.390 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.390 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.390 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.391 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default replica count for any created local pool", 2024-04-02T14:04:01.391 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.391 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.391 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.391 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.391 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_num": { 2024-04-02T14:04:01.391 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_num", 2024-04-02T14:04:01.391 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.392 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.392 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.392 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "128", 2024-04-02T14:04:01.392 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.392 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.392 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.392 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default pg_num for any created local pool", 2024-04-02T14:04:01.392 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.392 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.393 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.393 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.393 INFO:teuthology.orchestra.run.smithi052.stdout: "prefix": { 2024-04-02T14:04:01.393 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prefix", 2024-04-02T14:04:01.393 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.393 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.393 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.393 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.393 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.394 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.394 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.394 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name prefix for any created local pool", 2024-04-02T14:04:01.394 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.394 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.394 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.394 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.395 INFO:teuthology.orchestra.run.smithi052.stdout: "subtree": { 2024-04-02T14:04:01.395 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "subtree", 2024-04-02T14:04:01.395 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.395 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.395 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.395 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "rack", 2024-04-02T14:04:01.395 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.395 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.395 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.395 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-02T14:04:01.396 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-02T14:04:01.396 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.396 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.396 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.396 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.396 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.396 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.396 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mds_autoscaler", 2024-04-02T14:04:01.397 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.397 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.397 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.397 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.397 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.397 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.397 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.397 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.397 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.397 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.398 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.398 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.398 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.398 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.398 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.398 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.398 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.398 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.399 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.399 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.399 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.399 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.399 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.399 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.399 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.399 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.399 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.399 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.400 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.400 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.400 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.400 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.400 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.400 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.400 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.400 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.400 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.401 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.401 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.401 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.401 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.401 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.401 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.401 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.401 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.401 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.402 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.402 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.402 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.402 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.402 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.402 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.402 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.402 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.403 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.403 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.403 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.403 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.403 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.403 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.403 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.403 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.403 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.404 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.404 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.404 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.404 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.404 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.404 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.404 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.404 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.405 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.405 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.405 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.405 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.405 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.405 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirroring", 2024-04-02T14:04:01.405 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.405 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.405 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.406 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.406 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.406 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.406 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.406 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.406 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.406 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.406 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.407 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.407 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.407 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.407 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.407 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.407 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.407 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.407 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.407 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.408 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.408 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.408 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.408 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.408 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.408 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.408 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.408 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.409 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.409 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.409 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.409 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.409 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.409 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.409 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.409 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.409 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.410 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.410 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.410 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.410 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.410 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.410 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.410 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.411 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.411 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.411 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.411 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.411 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.411 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.411 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.411 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.412 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.412 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.412 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.412 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.412 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.412 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.412 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.412 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.412 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.413 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.413 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.413 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.413 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.413 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.413 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.413 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.413 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.414 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.414 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.414 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.414 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.414 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.414 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.414 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.414 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "nfs", 2024-04-02T14:04:01.415 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.415 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.415 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.415 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.415 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.415 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.415 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.415 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.416 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.416 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.416 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.416 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.416 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.416 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.416 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.416 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.417 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.417 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.417 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.417 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.417 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.417 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.417 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.417 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.417 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.418 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.418 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.418 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.418 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.418 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.418 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.418 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.418 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.419 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.419 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.419 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.419 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.419 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.419 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.419 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.419 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.420 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.420 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.420 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.420 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.420 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.420 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.420 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.420 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.420 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.421 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.421 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.421 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.421 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.421 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.421 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.421 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.421 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.421 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.422 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.422 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.422 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.422 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.422 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.422 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.422 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.422 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.423 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.423 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.423 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.423 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.423 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.423 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.423 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.423 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.423 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.424 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2024-04-02T14:04:01.424 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.424 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.424 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.424 INFO:teuthology.orchestra.run.smithi052.stdout: "fail_fs": { 2024-04-02T14:04:01.424 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "fail_fs", 2024-04-02T14:04:01.424 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.424 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.425 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.425 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.425 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.425 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.425 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.425 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-02T14:04:01.425 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.425 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.426 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.426 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.426 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.426 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.426 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.426 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.426 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.426 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.427 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.427 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.427 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.427 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.427 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.427 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.427 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.427 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.428 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.428 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.428 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.428 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.428 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.428 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.428 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.428 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.429 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.429 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.429 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.429 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.429 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.429 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.429 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.429 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.429 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.430 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.430 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.430 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.430 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.430 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.430 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.430 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.431 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.431 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.431 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.431 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.431 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.431 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.431 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.431 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.431 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.432 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.432 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.432 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.432 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.432 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.432 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.432 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.432 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.433 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.433 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.433 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.433 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.433 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.433 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.433 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.433 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.433 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.434 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.434 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.434 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.434 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.434 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.434 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.434 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator": { 2024-04-02T14:04:01.434 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2024-04-02T14:04:01.435 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.435 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.435 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.435 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.435 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.435 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.435 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.435 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm", 2024-04-02T14:04:01.436 INFO:teuthology.orchestra.run.smithi052.stdout: "rook", 2024-04-02T14:04:01.436 INFO:teuthology.orchestra.run.smithi052.stdout: "test_orchestrator" 2024-04-02T14:04:01.437 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.437 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Orchestrator backend", 2024-04-02T14:04:01.437 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.438 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.438 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.438 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.438 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.438 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.438 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.438 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_perf_query", 2024-04-02T14:04:01.438 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.439 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.439 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.439 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.439 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.439 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.439 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.439 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.439 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.440 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.440 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.440 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.440 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.440 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.440 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.440 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.440 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.441 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.441 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.441 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.441 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.441 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.441 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.441 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.441 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.442 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.442 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.442 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.442 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.442 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.442 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.442 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.442 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.443 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.443 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.443 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.443 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.443 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.443 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.443 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.443 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.443 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.444 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.444 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.444 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.444 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.444 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.444 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.444 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.445 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.445 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.445 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.445 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.445 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.445 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.446 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.446 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.446 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.446 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.446 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.446 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.446 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.446 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.447 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.447 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.447 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.447 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.447 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.447 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.447 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.447 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.448 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.448 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.448 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.448 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.448 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.448 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_support", 2024-04-02T14:04:01.448 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.448 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.449 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.449 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.449 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.449 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.449 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.449 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.449 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.449 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.449 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.450 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.450 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.450 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.450 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.450 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.450 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.450 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.451 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.451 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.451 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.451 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.451 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.451 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.451 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.451 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.451 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.452 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.452 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.452 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.452 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.452 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.452 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.452 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.452 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.453 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.453 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.453 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.453 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.453 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.453 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.453 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.453 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.454 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.454 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.454 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.454 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.454 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.454 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.454 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.454 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.455 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.455 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.455 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.455 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.455 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.455 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.455 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.455 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.456 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.456 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.456 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.456 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.456 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.456 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.456 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.456 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.457 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.457 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.457 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.457 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.457 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.457 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.457 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.457 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.457 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.458 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_autoscaler", 2024-04-02T14:04:01.458 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.458 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.458 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.458 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.458 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.458 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.458 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.459 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.459 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.459 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.459 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.459 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.459 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.459 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.459 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.459 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.459 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.460 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.460 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.460 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.460 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.460 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.460 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.460 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.460 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.460 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.461 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.461 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.461 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.461 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.461 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.461 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.461 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.461 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.462 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.462 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.462 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.462 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.462 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.462 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.462 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.462 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.463 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.463 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.463 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.463 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.463 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.463 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.463 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.463 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.464 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.464 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.464 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.464 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.464 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.464 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.464 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.464 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.464 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.465 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.465 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.465 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.465 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.465 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.465 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.465 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.465 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.465 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.466 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.466 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.466 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.466 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.466 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.466 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-04-02T14:04:01.466 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-04-02T14:04:01.466 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.467 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.467 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.467 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-04-02T14:04:01.467 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.467 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.467 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.467 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.467 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.467 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.468 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.468 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.468 INFO:teuthology.orchestra.run.smithi052.stdout: "threshold": { 2024-04-02T14:04:01.468 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threshold", 2024-04-02T14:04:01.468 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-04-02T14:04:01.468 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.468 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.468 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2024-04-02T14:04:01.468 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1.0", 2024-04-02T14:04:01.469 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.469 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.469 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "scaling threshold", 2024-04-02T14:04:01.469 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-02T14:04:01.469 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.469 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.469 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.469 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.469 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.470 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.470 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "progress", 2024-04-02T14:04:01.470 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.470 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.470 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.470 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_pg_recovery_event": { 2024-04-02T14:04:01.470 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_pg_recovery_event", 2024-04-02T14:04:01.470 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.471 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.471 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.471 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.471 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.471 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.471 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.471 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-02T14:04:01.471 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.471 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.472 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.472 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.472 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2024-04-02T14:04:01.472 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2024-04-02T14:04:01.472 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.472 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.472 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.472 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.473 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.473 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.473 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.473 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.473 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.473 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.473 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.473 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.473 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.474 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.474 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.474 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.474 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.474 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.474 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.474 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.474 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.475 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.475 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.475 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.475 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.475 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.475 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.475 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.475 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.476 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.476 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.476 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.476 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.476 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.476 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.476 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.476 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.477 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.477 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.477 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.477 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.477 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.477 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.477 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.477 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.478 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.478 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.478 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.478 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.478 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.478 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.478 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.478 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.479 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.479 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.479 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.479 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.479 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.479 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.479 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.479 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.479 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.480 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.480 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.480 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.480 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.480 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.480 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.480 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.480 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.481 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.481 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.481 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.481 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.481 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.481 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.481 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.481 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.482 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.482 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.482 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.482 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.482 INFO:teuthology.orchestra.run.smithi052.stdout: "max_completed_events": { 2024-04-02T14:04:01.482 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_completed_events", 2024-04-02T14:04:01.482 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.482 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.483 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.483 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "50", 2024-04-02T14:04:01.483 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.483 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.483 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.483 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "number of past completed events to remember", 2024-04-02T14:04:01.483 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.483 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.484 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.484 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.484 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-04-02T14:04:01.484 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-04-02T14:04:01.484 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.484 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.484 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.484 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-04-02T14:04:01.485 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.485 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.485 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.485 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long the module is going to sleep", 2024-04-02T14:04:01.485 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.485 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.485 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.485 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.486 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.486 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.486 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.486 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus", 2024-04-02T14:04:01.486 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.486 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.486 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.486 INFO:teuthology.orchestra.run.smithi052.stdout: "cache": { 2024-04-02T14:04:01.487 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cache", 2024-04-02T14:04:01.487 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.487 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.487 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.487 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.487 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.487 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.487 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.488 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.488 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.488 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.488 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.488 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.488 INFO:teuthology.orchestra.run.smithi052.stdout: "exclude_perf_counters": { 2024-04-02T14:04:01.488 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "exclude_perf_counters", 2024-04-02T14:04:01.488 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.488 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.489 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.489 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.489 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.489 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.489 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.489 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-02T14:04:01.489 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-02T14:04:01.489 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.490 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.490 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.490 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.490 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.490 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.490 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.490 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.490 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.491 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.491 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.491 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.491 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.491 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.491 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.491 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.492 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.492 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.492 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.492 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.492 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.492 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.492 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.492 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.492 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.493 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.493 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.493 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.493 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.493 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.493 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.493 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.493 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.494 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.494 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.494 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.494 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.494 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.494 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.494 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.494 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.495 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.495 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.495 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.495 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.495 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.495 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.495 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.495 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.495 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.496 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.496 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.496 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.496 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.496 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.496 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.496 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.496 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.497 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.497 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.497 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.497 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.497 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.497 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.497 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.497 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.497 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.498 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.498 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.498 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.498 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.498 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.498 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.498 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools": { 2024-04-02T14:04:01.498 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools", 2024-04-02T14:04:01.499 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.499 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.499 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.499 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.499 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.499 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.499 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.499 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.500 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.500 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.500 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.500 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.500 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-02T14:04:01.500 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-02T14:04:01.500 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.500 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.501 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.501 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "300", 2024-04-02T14:04:01.501 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.501 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.501 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.501 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.501 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.501 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.501 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.502 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.502 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_interval": { 2024-04-02T14:04:01.502 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_interval", 2024-04-02T14:04:01.502 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-04-02T14:04:01.502 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.502 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.502 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15.0", 2024-04-02T14:04:01.502 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.503 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.503 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.503 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.503 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.503 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.503 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.503 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.503 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-04-02T14:04:01.503 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-04-02T14:04:01.504 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.504 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.504 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.504 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2024-04-02T14:04:01.504 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.504 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.504 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.504 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-02T14:04:01.505 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.505 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.505 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.505 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.505 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-04-02T14:04:01.505 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-04-02T14:04:01.505 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.505 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.506 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.506 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "9283", 2024-04-02T14:04:01.506 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.506 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.506 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.506 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-02T14:04:01.506 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.506 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.506 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.507 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.507 INFO:teuthology.orchestra.run.smithi052.stdout: "stale_cache_strategy": { 2024-04-02T14:04:01.507 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stale_cache_strategy", 2024-04-02T14:04:01.507 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.507 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.507 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.507 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "log", 2024-04-02T14:04:01.507 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.507 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.508 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.508 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.508 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.508 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.508 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.508 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.508 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2024-04-02T14:04:01.508 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2024-04-02T14:04:01.509 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.509 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.509 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.509 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "default", 2024-04-02T14:04:01.509 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.509 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.509 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.509 INFO:teuthology.orchestra.run.smithi052.stdout: "default", 2024-04-02T14:04:01.509 INFO:teuthology.orchestra.run.smithi052.stdout: "error" 2024-04-02T14:04:01.510 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.510 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.510 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.510 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.510 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.510 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.510 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2024-04-02T14:04:01.510 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2024-04-02T14:04:01.511 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.511 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.511 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.511 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2024-04-02T14:04:01.511 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2024-04-02T14:04:01.511 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2024-04-02T14:04:01.511 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.511 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.512 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.512 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.512 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.512 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.512 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.512 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.512 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.512 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_support", 2024-04-02T14:04:01.513 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.513 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.513 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.513 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.513 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.513 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.513 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.513 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.514 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.514 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.514 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.514 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.514 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.514 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.514 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.514 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.515 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.515 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.515 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.515 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.515 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.515 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.515 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.515 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.516 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.516 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.516 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.516 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.516 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.516 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.516 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.516 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.517 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.517 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.517 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.517 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.517 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.517 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.517 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.517 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.517 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.518 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.518 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.518 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.518 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.518 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.518 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.518 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.518 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.519 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.519 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.519 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.519 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.519 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.519 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.519 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.519 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.520 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.520 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.520 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.520 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.520 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.520 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.520 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.520 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.521 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.521 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.521 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.521 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.521 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.521 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.521 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.521 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.522 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_snap_create": { 2024-04-02T14:04:01.522 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_snap_create", 2024-04-02T14:04:01.522 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.522 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.522 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.522 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-04-02T14:04:01.522 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.522 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.522 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.523 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.523 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.523 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.523 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.523 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.523 INFO:teuthology.orchestra.run.smithi052.stdout: "mirror_snapshot_schedule": { 2024-04-02T14:04:01.523 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirror_snapshot_schedule", 2024-04-02T14:04:01.523 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.523 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.524 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.524 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.524 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.524 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.524 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.524 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.524 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.524 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.525 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.525 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.525 INFO:teuthology.orchestra.run.smithi052.stdout: "trash_purge_schedule": { 2024-04-02T14:04:01.525 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "trash_purge_schedule", 2024-04-02T14:04:01.525 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.525 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.525 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.525 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.526 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.526 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.526 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.526 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.526 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.526 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.526 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.526 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.527 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.527 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.527 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.527 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "restful", 2024-04-02T14:04:01.527 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.527 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.527 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.527 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_auth": { 2024-04-02T14:04:01.527 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_auth", 2024-04-02T14:04:01.528 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.528 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.528 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.528 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.528 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.528 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.528 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.528 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.528 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.529 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.529 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.529 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.529 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2024-04-02T14:04:01.529 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2024-04-02T14:04:01.529 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.529 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.529 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.529 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.530 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.530 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.530 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.530 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.530 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.530 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.530 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.530 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.531 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.531 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.531 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.531 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.531 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.531 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.531 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.531 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.532 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.532 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.532 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.532 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.532 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.532 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.532 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.532 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.532 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.533 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.533 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.533 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.533 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.533 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.533 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.533 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.533 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.533 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.534 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.534 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.534 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.534 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.534 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.534 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.534 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.535 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.535 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.535 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.535 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.535 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.535 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.535 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.535 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.536 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.536 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.536 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.536 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.536 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.536 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.536 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.536 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.537 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.537 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.537 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.537 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.537 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.537 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.537 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.537 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.538 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.538 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.538 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.538 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.538 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.538 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.538 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.538 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.539 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.539 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.539 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.539 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.539 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.539 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-04-02T14:04:01.539 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-04-02T14:04:01.539 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.540 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.540 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.540 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.540 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.540 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.540 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.540 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.540 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.540 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.541 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.541 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.541 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-04-02T14:04:01.541 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-04-02T14:04:01.541 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.541 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.541 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.541 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.542 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.542 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.542 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.542 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.542 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.542 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.542 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.542 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.543 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.543 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.543 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.543 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rgw", 2024-04-02T14:04:01.543 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.543 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.543 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.544 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.544 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.544 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.544 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.544 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.544 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.544 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.544 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.544 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.545 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.545 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.545 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.545 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.545 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.545 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.545 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.546 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.546 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.546 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.546 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.546 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.546 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.546 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.546 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.547 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.547 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.547 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.547 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.547 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.547 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.547 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.547 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.548 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.548 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.548 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.548 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.548 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.548 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.548 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.548 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.549 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.549 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.549 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.549 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.549 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.549 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.549 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.549 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.549 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.550 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.550 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.550 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.550 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.550 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.550 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.550 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.551 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.551 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.551 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.551 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.551 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.551 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.551 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.551 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.552 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.552 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.552 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.552 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.552 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.552 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.552 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.552 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.553 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.553 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rook", 2024-04-02T14:04:01.553 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.553 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.553 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.553 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.553 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.553 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.553 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.554 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.554 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.554 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.554 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.554 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.554 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.554 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.554 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.555 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.555 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.555 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.555 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.555 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.555 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.555 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.555 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.556 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.556 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.556 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.556 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.556 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.556 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.556 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.556 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.556 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.557 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.557 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.557 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.557 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.557 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.557 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.557 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.557 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.557 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.558 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.558 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.558 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.558 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.558 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.558 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.558 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.558 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.559 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.559 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.559 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.559 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.559 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.559 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.559 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.559 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.560 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.560 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.560 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.560 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.560 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.560 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.560 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.560 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.560 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.561 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.561 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.561 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.561 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.561 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.561 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.561 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.561 INFO:teuthology.orchestra.run.smithi052.stdout: "storage_class": { 2024-04-02T14:04:01.561 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "storage_class", 2024-04-02T14:04:01.562 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.562 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.562 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.562 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "local", 2024-04-02T14:04:01.562 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.562 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.562 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.562 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-02T14:04:01.563 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.563 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.563 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.563 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.563 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.563 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.563 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.563 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "selftest", 2024-04-02T14:04:01.563 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.564 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.564 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.564 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.564 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.564 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.564 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.564 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.564 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.564 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.565 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.565 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.565 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.565 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.565 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.565 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.565 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.565 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.565 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.566 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.566 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.566 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.566 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.566 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.566 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.566 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.566 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.567 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.567 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.567 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.567 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.567 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.567 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.567 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.567 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.568 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.568 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.568 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.568 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.568 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.568 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.568 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.568 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.568 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.568 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.569 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.569 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.569 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.569 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.569 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.569 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.569 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.569 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.570 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.570 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.570 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.570 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.570 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.570 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.570 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.570 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.570 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.571 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.571 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.571 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.571 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.571 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.571 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.571 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.571 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.572 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.572 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.572 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.572 INFO:teuthology.orchestra.run.smithi052.stdout: "roption1": { 2024-04-02T14:04:01.572 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption1", 2024-04-02T14:04:01.572 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.572 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.572 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.572 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.573 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.573 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.573 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.573 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.573 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.573 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.573 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.573 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.573 INFO:teuthology.orchestra.run.smithi052.stdout: "roption2": { 2024-04-02T14:04:01.574 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption2", 2024-04-02T14:04:01.574 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.574 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.574 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.574 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "xyz", 2024-04-02T14:04:01.574 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.574 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.574 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.574 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.575 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.575 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.575 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.575 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.575 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption1": { 2024-04-02T14:04:01.575 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption1", 2024-04-02T14:04:01.575 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.575 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.575 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.576 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.576 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.576 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.576 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.576 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.576 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.576 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.576 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.576 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.577 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption2": { 2024-04-02T14:04:01.577 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption2", 2024-04-02T14:04:01.577 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.577 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.577 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.577 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.577 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.577 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.577 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.578 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.578 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.578 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.578 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.578 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.578 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption3": { 2024-04-02T14:04:01.578 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption3", 2024-04-02T14:04:01.578 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-04-02T14:04:01.579 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.579 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.579 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.579 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.579 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.579 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.579 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.579 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.579 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.580 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.580 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.580 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption4": { 2024-04-02T14:04:01.580 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption4", 2024-04-02T14:04:01.580 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.580 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.580 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.580 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.580 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.581 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.581 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.581 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.581 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.581 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.581 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.581 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.581 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption5": { 2024-04-02T14:04:01.581 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption5", 2024-04-02T14:04:01.582 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.582 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.582 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.582 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.582 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.582 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.582 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.582 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.582 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.583 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.583 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.583 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.583 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption6": { 2024-04-02T14:04:01.583 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption6", 2024-04-02T14:04:01.583 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.583 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.583 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.583 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.584 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.584 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.584 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.584 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.584 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.584 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.584 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.584 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.585 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption7": { 2024-04-02T14:04:01.585 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption7", 2024-04-02T14:04:01.585 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.585 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.585 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.585 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.585 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-04-02T14:04:01.585 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "42", 2024-04-02T14:04:01.585 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.585 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.586 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.586 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.586 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.586 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.586 INFO:teuthology.orchestra.run.smithi052.stdout: "testkey": { 2024-04-02T14:04:01.586 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testkey", 2024-04-02T14:04:01.586 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.586 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.587 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.587 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.587 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.587 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.587 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.587 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.587 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.587 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.587 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.587 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.588 INFO:teuthology.orchestra.run.smithi052.stdout: "testlkey": { 2024-04-02T14:04:01.588 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testlkey", 2024-04-02T14:04:01.588 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.588 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.588 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.588 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.588 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.588 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.588 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.589 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.589 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.589 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.589 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.589 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.589 INFO:teuthology.orchestra.run.smithi052.stdout: "testnewline": { 2024-04-02T14:04:01.589 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testnewline", 2024-04-02T14:04:01.589 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.589 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.590 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.590 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.590 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.590 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.590 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.590 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.590 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.590 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.590 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.591 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.591 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.591 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.591 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.591 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snap_schedule", 2024-04-02T14:04:01.591 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.591 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.591 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.591 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_m_granularity": { 2024-04-02T14:04:01.592 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_m_granularity", 2024-04-02T14:04:01.592 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.592 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.592 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.592 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.592 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.592 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.592 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.592 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow minute scheduled snapshots", 2024-04-02T14:04:01.593 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.593 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.593 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.593 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.593 INFO:teuthology.orchestra.run.smithi052.stdout: "dump_on_update": { 2024-04-02T14:04:01.593 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dump_on_update", 2024-04-02T14:04:01.593 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.593 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.593 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.594 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.594 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.594 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.594 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.594 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "dump database to debug log on update", 2024-04-02T14:04:01.594 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.594 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.594 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.595 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.595 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.595 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.595 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.595 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.595 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.595 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.595 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.595 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.596 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.596 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.596 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.596 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.596 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.596 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.596 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.596 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.596 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.597 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.597 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.597 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.597 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.597 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.597 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.597 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.597 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.597 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.598 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.598 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.598 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.598 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.598 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.598 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.598 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.598 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.599 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.599 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.599 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.599 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.599 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.599 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.599 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.600 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.600 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.600 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.600 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.600 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.600 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.600 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.600 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.600 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.601 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.601 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.601 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.601 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.601 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.601 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.601 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.601 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.602 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.602 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.602 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.602 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.602 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.602 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.602 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.603 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.603 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.603 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.603 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.603 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.603 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.603 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.603 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.604 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stats", 2024-04-02T14:04:01.604 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.604 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.604 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.604 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.604 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.604 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.604 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.605 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.605 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.605 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.605 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.605 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.605 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.605 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.605 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.605 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.606 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.606 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.606 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.606 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.606 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.606 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.606 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.606 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.607 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.607 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.607 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.607 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.607 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.607 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.607 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.607 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.607 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.607 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.608 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.608 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.608 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.608 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.608 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.608 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.608 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.608 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.608 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.609 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.609 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.609 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.609 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.609 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.609 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.609 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.609 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.609 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.610 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.610 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.610 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.610 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.610 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.610 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.610 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.610 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.611 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.611 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.611 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.611 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.611 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.611 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.611 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.611 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.611 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.611 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.612 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.612 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.612 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.612 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.612 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.612 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.612 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "status", 2024-04-02T14:04:01.612 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.612 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.613 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.613 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.613 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.613 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.613 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.613 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.613 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.613 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.614 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.614 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.614 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.614 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.614 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.614 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.614 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.614 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.615 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.615 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.615 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.615 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.615 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.615 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.615 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.615 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.615 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.616 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.616 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.616 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.616 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.616 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.616 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.616 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.616 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.616 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.616 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.617 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.617 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.617 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.617 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.617 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.617 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.617 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.617 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.617 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.618 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.618 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.618 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.618 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.618 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.618 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.618 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.618 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.618 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.619 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.619 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.619 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.619 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.619 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.619 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.619 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.620 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.620 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.620 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.620 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.620 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.620 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.620 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.620 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.620 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.621 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.621 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.621 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.621 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.621 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.621 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telegraf", 2024-04-02T14:04:01.621 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.621 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.621 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.621 INFO:teuthology.orchestra.run.smithi052.stdout: "address": { 2024-04-02T14:04:01.622 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "address", 2024-04-02T14:04:01.622 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.622 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.622 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.622 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-02T14:04:01.622 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.622 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.622 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.623 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.623 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.623 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.623 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.623 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.623 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-04-02T14:04:01.623 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-04-02T14:04:01.623 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.624 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.624 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.624 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15", 2024-04-02T14:04:01.624 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.624 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.624 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.624 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.624 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.625 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.625 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.625 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.625 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.625 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.625 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.625 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.625 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.626 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.626 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.626 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.626 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.626 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.626 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.626 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.626 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.627 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.627 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.627 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.627 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.627 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.627 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.627 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.627 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.628 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.628 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.628 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.628 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.628 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.628 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.628 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.628 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.628 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.629 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.629 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.629 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.629 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.629 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.629 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.629 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.629 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.630 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.630 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.630 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.630 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.630 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.630 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.630 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.630 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.631 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.631 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.631 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.631 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.631 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.631 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.631 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.631 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.632 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.632 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.632 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.632 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.632 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.632 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.632 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.632 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.632 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.633 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.633 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.633 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.633 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.633 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.633 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.633 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.633 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.633 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.634 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.634 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telemetry", 2024-04-02T14:04:01.634 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.634 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.634 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.634 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_basic": { 2024-04-02T14:04:01.634 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_basic", 2024-04-02T14:04:01.634 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.635 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.635 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.635 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.635 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.635 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.635 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.635 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-02T14:04:01.635 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.635 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.636 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.636 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.636 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_crash": { 2024-04-02T14:04:01.636 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_crash", 2024-04-02T14:04:01.636 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.636 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.636 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.636 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.636 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.637 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.637 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.637 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-02T14:04:01.637 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.637 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.637 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.637 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.637 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_device": { 2024-04-02T14:04:01.638 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_device", 2024-04-02T14:04:01.638 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.638 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.638 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.638 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.638 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.638 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.639 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.639 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-02T14:04:01.639 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.639 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.639 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.639 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.639 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_ident": { 2024-04-02T14:04:01.639 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_ident", 2024-04-02T14:04:01.639 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.640 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.640 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.642 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.642 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.642 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.642 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.642 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-02T14:04:01.642 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.642 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.643 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.643 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.643 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_perf": { 2024-04-02T14:04:01.643 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_perf", 2024-04-02T14:04:01.643 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.643 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.643 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.643 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.643 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.644 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.644 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.644 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-02T14:04:01.644 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.644 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.644 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.644 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.644 INFO:teuthology.orchestra.run.smithi052.stdout: "contact": { 2024-04-02T14:04:01.644 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "contact", 2024-04-02T14:04:01.645 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.645 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.645 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.645 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.645 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.645 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.645 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.645 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.645 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.646 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.646 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.646 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.646 INFO:teuthology.orchestra.run.smithi052.stdout: "description": { 2024-04-02T14:04:01.646 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "description", 2024-04-02T14:04:01.646 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.646 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.646 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.647 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.647 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.647 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.647 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.647 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.647 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.647 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.647 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.647 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.647 INFO:teuthology.orchestra.run.smithi052.stdout: "device_url": { 2024-04-02T14:04:01.648 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_url", 2024-04-02T14:04:01.648 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.648 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.648 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.648 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-02T14:04:01.648 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.648 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.648 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.648 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.649 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.649 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.649 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.649 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.649 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2024-04-02T14:04:01.649 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2024-04-02T14:04:01.649 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.649 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.649 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.650 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.650 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.650 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.650 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.650 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.650 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.650 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.650 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.651 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.651 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-04-02T14:04:01.651 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-04-02T14:04:01.651 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.651 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.651 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.651 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "24", 2024-04-02T14:04:01.651 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "8", 2024-04-02T14:04:01.651 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.651 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.652 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.652 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.652 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.652 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.652 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.652 INFO:teuthology.orchestra.run.smithi052.stdout: "last_opt_revision": { 2024-04-02T14:04:01.652 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "last_opt_revision", 2024-04-02T14:04:01.652 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.652 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.653 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.653 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1", 2024-04-02T14:04:01.653 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.653 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.653 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.653 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.653 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.653 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.653 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.654 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.654 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard": { 2024-04-02T14:04:01.654 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard", 2024-04-02T14:04:01.654 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.654 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.654 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.654 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.654 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.655 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.655 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.655 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.655 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.655 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.655 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.655 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.655 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard_description": { 2024-04-02T14:04:01.655 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard_description", 2024-04-02T14:04:01.656 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.656 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.656 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.656 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.656 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.656 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.656 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.656 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.657 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.657 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.657 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.657 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.657 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.657 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.657 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.657 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.657 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.658 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.658 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.658 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.658 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.658 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.658 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.658 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.658 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.658 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.659 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.659 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.659 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.659 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.659 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.659 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.659 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.659 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.659 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.660 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.660 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.660 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.660 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.660 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.660 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.660 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.660 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.660 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.661 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.661 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.661 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.661 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.661 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.661 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.661 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.661 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.661 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.661 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.662 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.662 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.662 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.662 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.662 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.662 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.662 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.662 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.663 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.663 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.663 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.663 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.663 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.663 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.663 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.663 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.663 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.664 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.664 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.664 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.664 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.664 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.664 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.664 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.664 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.665 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.665 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.665 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.665 INFO:teuthology.orchestra.run.smithi052.stdout: "organization": { 2024-04-02T14:04:01.665 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "organization", 2024-04-02T14:04:01.665 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.665 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.665 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.665 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.666 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.666 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.666 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.666 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.666 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.666 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.666 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.666 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.667 INFO:teuthology.orchestra.run.smithi052.stdout: "proxy": { 2024-04-02T14:04:01.667 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "proxy", 2024-04-02T14:04:01.667 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.667 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.667 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.667 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.667 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.667 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.668 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.668 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.668 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.668 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.668 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.668 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.668 INFO:teuthology.orchestra.run.smithi052.stdout: "url": { 2024-04-02T14:04:01.668 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url", 2024-04-02T14:04:01.668 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.669 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.669 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.669 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-02T14:04:01.669 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.669 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.669 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.669 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.669 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.669 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.669 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.670 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.670 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.670 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.670 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.670 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "test_orchestrator", 2024-04-02T14:04:01.670 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.670 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.670 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.671 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.671 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.671 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.671 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.671 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.671 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.671 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.671 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.671 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.672 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.672 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.672 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.672 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.672 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.672 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.672 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.672 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.672 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.673 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.673 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.673 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.673 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.673 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.673 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.673 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.673 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.673 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.673 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.674 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.674 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.674 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.674 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.674 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.674 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.674 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.674 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.675 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.675 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.675 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.675 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.675 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.675 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.675 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.675 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.675 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.676 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.676 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.676 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.676 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.676 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.676 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.676 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.676 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.676 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.676 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.677 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.677 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.677 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.677 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.677 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.677 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.677 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.677 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.677 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.678 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.678 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.678 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.678 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.678 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.678 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.678 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.678 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.679 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.679 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "volumes", 2024-04-02T14:04:01.679 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.679 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.679 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.679 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.679 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.679 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.679 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.679 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.680 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.680 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.680 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.680 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.680 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.680 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.680 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.680 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.680 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.681 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.681 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.681 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.681 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.681 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.681 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.681 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.681 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.681 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.682 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.682 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.682 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.682 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.682 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.682 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.682 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.682 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.683 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.683 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.683 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.683 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.683 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.683 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.683 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.683 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.683 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.684 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.684 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.684 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.684 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.684 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.684 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.684 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.684 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.684 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.685 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.685 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.685 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.685 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.685 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.685 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.685 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.685 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.685 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.686 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.686 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.686 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.686 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.686 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.686 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.686 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.686 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.686 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.687 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.687 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.687 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.687 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_clones": { 2024-04-02T14:04:01.687 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_clones", 2024-04-02T14:04:01.687 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.687 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.687 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.688 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4", 2024-04-02T14:04:01.688 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.688 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.688 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.688 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-02T14:04:01.688 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.688 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.688 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.689 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.689 INFO:teuthology.orchestra.run.smithi052.stdout: "periodic_async_work": { 2024-04-02T14:04:01.689 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "periodic_async_work", 2024-04-02T14:04:01.689 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.689 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.689 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.689 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.689 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.690 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.690 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.690 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Periodically check for async work", 2024-04-02T14:04:01.690 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.690 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.690 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.690 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.690 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_delay": { 2024-04-02T14:04:01.690 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_delay", 2024-04-02T14:04:01.691 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.691 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.691 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.691 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-04-02T14:04:01.691 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.691 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.691 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.691 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-02T14:04:01.692 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.692 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.692 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.692 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.692 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_no_wait": { 2024-04-02T14:04:01.692 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_no_wait", 2024-04-02T14:04:01.692 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.692 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.693 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.693 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.693 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.693 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.693 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.693 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-02T14:04:01.693 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.693 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.694 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.694 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.694 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.694 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.694 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.694 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix", 2024-04-02T14:04:01.694 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.695 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.695 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.695 INFO:teuthology.orchestra.run.smithi052.stdout: "discovery_interval": { 2024-04-02T14:04:01.695 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "discovery_interval", 2024-04-02T14:04:01.695 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-04-02T14:04:01.695 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.695 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.695 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "100", 2024-04-02T14:04:01.695 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.696 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.696 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.696 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.696 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.696 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.696 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.696 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.696 INFO:teuthology.orchestra.run.smithi052.stdout: "identifier": { 2024-04-02T14:04:01.696 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "identifier", 2024-04-02T14:04:01.697 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.697 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.697 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.697 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.697 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.697 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.697 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.697 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.698 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.698 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.698 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.698 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.698 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-04-02T14:04:01.698 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-04-02T14:04:01.698 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.699 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.699 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.699 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-04-02T14:04:01.699 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.699 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.699 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.699 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.699 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.700 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.700 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.700 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.700 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.700 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.700 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.700 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.700 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.700 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.701 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.701 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.701 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.701 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.701 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.701 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.701 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.701 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.702 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.702 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.702 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.702 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.702 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.702 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.702 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.702 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.703 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.703 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.703 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.703 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.703 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.703 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.703 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.703 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.703 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.704 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.704 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.704 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.704 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.704 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.704 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.704 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.704 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.705 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.705 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.705 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.705 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.705 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.705 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.705 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.705 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.706 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.706 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.706 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.706 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.706 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.706 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.706 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.706 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.707 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.707 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.707 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.707 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.707 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.707 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.707 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.708 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.708 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.708 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.708 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.708 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.708 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.708 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.708 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.709 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_host": { 2024-04-02T14:04:01.709 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_host", 2024-04-02T14:04:01.709 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.709 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.709 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.709 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.709 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.709 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.709 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.710 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.710 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.710 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.710 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.710 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.710 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_port": { 2024-04-02T14:04:01.710 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_port", 2024-04-02T14:04:01.711 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.711 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.711 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.711 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10051", 2024-04-02T14:04:01.711 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.711 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.711 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.711 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.711 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.712 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.712 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.712 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.712 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_sender": { 2024-04-02T14:04:01.712 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_sender", 2024-04-02T14:04:01.712 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.712 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.712 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.713 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-02T14:04:01.713 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.713 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.713 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.713 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.713 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.713 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.713 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.714 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.714 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.714 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.714 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-04-02T14:04:01.714 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.714 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.714 INFO:teuthology.orchestra.run.smithi052.stdout: "modules": [ 2024-04-02T14:04:01.714 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm", 2024-04-02T14:04:01.715 INFO:teuthology.orchestra.run.smithi052.stdout: "dashboard", 2024-04-02T14:04:01.715 INFO:teuthology.orchestra.run.smithi052.stdout: "iostat", 2024-04-02T14:04:01.715 INFO:teuthology.orchestra.run.smithi052.stdout: "nfs", 2024-04-02T14:04:01.715 INFO:teuthology.orchestra.run.smithi052.stdout: "restful", 2024-04-02T14:04:01.715 INFO:teuthology.orchestra.run.smithi052.stdout: "snap_schedule" 2024-04-02T14:04:01.715 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.715 INFO:teuthology.orchestra.run.smithi052.stdout: "available_modules": [ 2024-04-02T14:04:01.715 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.715 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "alerts", 2024-04-02T14:04:01.716 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.716 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.716 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.716 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-04-02T14:04:01.716 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-04-02T14:04:01.716 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.716 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.716 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.717 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-04-02T14:04:01.717 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.717 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.717 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.717 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How frequently to reexamine health status", 2024-04-02T14:04:01.717 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.717 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.717 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.718 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.718 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.718 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.718 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.718 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.718 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.718 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.718 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.719 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.719 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.719 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.719 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.719 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.719 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.719 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.719 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.720 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.720 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.720 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.720 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.720 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.720 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.720 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.720 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.721 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.721 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.721 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.721 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.721 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.721 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.721 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.721 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.722 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.722 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.722 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.722 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.722 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.722 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.722 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.722 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.723 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.723 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.723 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.723 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.723 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.723 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.723 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.723 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.724 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.724 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.724 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.724 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.724 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.724 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.724 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.724 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.725 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.725 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.725 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.725 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.725 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.725 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.725 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.725 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.725 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.726 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.726 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.726 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.726 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.726 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.726 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.726 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_destination": { 2024-04-02T14:04:01.727 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_destination", 2024-04-02T14:04:01.727 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.727 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.727 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.727 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.727 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.727 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.727 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.728 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email address to send alerts to", 2024-04-02T14:04:01.728 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.728 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.728 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.728 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.728 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_from_name": { 2024-04-02T14:04:01.728 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_from_name", 2024-04-02T14:04:01.728 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.728 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.729 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.729 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "Ceph", 2024-04-02T14:04:01.729 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.729 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.729 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.729 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email From: name", 2024-04-02T14:04:01.729 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.729 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.730 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.730 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.730 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_host": { 2024-04-02T14:04:01.730 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_host", 2024-04-02T14:04:01.730 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.730 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.730 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.730 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.731 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.731 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.731 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.731 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP server", 2024-04-02T14:04:01.731 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.731 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.731 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.731 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.732 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_password": { 2024-04-02T14:04:01.732 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_password", 2024-04-02T14:04:01.732 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.732 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.732 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.732 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.732 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.732 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.733 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.733 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Password to authenticate with", 2024-04-02T14:04:01.733 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.733 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.733 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.733 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.733 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_port": { 2024-04-02T14:04:01.733 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_port", 2024-04-02T14:04:01.733 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.734 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.734 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.734 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "465", 2024-04-02T14:04:01.734 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.734 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.734 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.734 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP port", 2024-04-02T14:04:01.734 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.735 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.735 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.735 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.735 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_sender": { 2024-04-02T14:04:01.735 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_sender", 2024-04-02T14:04:01.735 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.735 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.735 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.736 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.736 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.736 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.736 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.736 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP envelope sender", 2024-04-02T14:04:01.736 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.736 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.736 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.736 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.737 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_ssl": { 2024-04-02T14:04:01.737 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_ssl", 2024-04-02T14:04:01.737 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.737 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.737 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.737 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.737 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.737 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.738 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.738 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-02T14:04:01.738 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.738 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.738 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.738 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.738 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_user": { 2024-04-02T14:04:01.738 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_user", 2024-04-02T14:04:01.739 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.739 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.739 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.739 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.739 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.739 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.739 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.739 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "User to authenticate as", 2024-04-02T14:04:01.740 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.740 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.740 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.740 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.740 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.740 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.740 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.740 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "balancer", 2024-04-02T14:04:01.740 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.741 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.741 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.741 INFO:teuthology.orchestra.run.smithi052.stdout: "active": { 2024-04-02T14:04:01.741 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "active", 2024-04-02T14:04:01.741 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.741 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.741 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.741 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.741 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.742 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.742 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.742 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically balance PGs across cluster", 2024-04-02T14:04:01.742 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.742 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.742 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.742 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.742 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_time": { 2024-04-02T14:04:01.743 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_time", 2024-04-02T14:04:01.743 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.743 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.743 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.743 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0000", 2024-04-02T14:04:01.743 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.743 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.743 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.744 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "beginning time of day to automatically balance", 2024-04-02T14:04:01.744 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T14:04:01.744 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.744 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.744 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.744 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_weekday": { 2024-04-02T14:04:01.744 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_weekday", 2024-04-02T14:04:01.744 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-04-02T14:04:01.744 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.745 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.745 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-04-02T14:04:01.745 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2024-04-02T14:04:01.745 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2024-04-02T14:04:01.745 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.745 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-02T14:04:01.745 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T14:04:01.746 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.746 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.746 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.746 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_max_iterations": { 2024-04-02T14:04:01.746 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_max_iterations", 2024-04-02T14:04:01.746 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-04-02T14:04:01.746 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.746 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.747 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "25", 2024-04-02T14:04:01.747 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-04-02T14:04:01.747 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "250", 2024-04-02T14:04:01.747 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.747 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-02T14:04:01.747 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.747 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.747 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.747 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.748 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_metrics": { 2024-04-02T14:04:01.748 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_metrics", 2024-04-02T14:04:01.748 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.748 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.748 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.748 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "pgs,objects,bytes", 2024-04-02T14:04:01.748 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.748 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.748 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.749 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-02T14:04:01.749 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-02T14:04:01.749 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.749 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.749 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.749 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_step": { 2024-04-02T14:04:01.749 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_step", 2024-04-02T14:04:01.749 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-04-02T14:04:01.749 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.750 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.750 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.5", 2024-04-02T14:04:01.750 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0.001", 2024-04-02T14:04:01.750 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "0.999", 2024-04-02T14:04:01.750 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.750 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "aggressiveness of optimization", 2024-04-02T14:04:01.750 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-02T14:04:01.750 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.751 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.751 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.751 INFO:teuthology.orchestra.run.smithi052.stdout: "end_time": { 2024-04-02T14:04:01.751 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_time", 2024-04-02T14:04:01.751 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.751 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.751 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.751 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2359", 2024-04-02T14:04:01.751 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.751 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.752 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.752 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ending time of day to automatically balance", 2024-04-02T14:04:01.752 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T14:04:01.752 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.752 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.752 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.752 INFO:teuthology.orchestra.run.smithi052.stdout: "end_weekday": { 2024-04-02T14:04:01.752 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_weekday", 2024-04-02T14:04:01.753 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-04-02T14:04:01.753 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.753 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.753 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-04-02T14:04:01.753 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2024-04-02T14:04:01.753 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2024-04-02T14:04:01.753 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.753 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-02T14:04:01.753 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T14:04:01.754 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.754 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.754 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.754 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.754 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.754 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.754 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.754 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.755 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.755 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.755 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.755 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.755 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.755 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.755 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.755 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.755 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.756 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.756 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.756 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.756 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.756 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.756 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.756 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.756 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.756 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.757 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.757 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.757 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.757 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.757 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.757 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.757 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.757 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.757 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.758 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.758 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.758 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.758 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.758 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.758 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.758 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.758 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.759 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.759 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.759 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.759 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.759 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.759 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.759 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.759 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.760 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.760 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.760 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.760 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.760 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.760 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.760 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.760 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.760 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.761 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.761 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.761 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.761 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.761 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.761 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.761 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.761 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.761 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.762 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.762 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.762 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.762 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.762 INFO:teuthology.orchestra.run.smithi052.stdout: "min_score": { 2024-04-02T14:04:01.762 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_score", 2024-04-02T14:04:01.762 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-04-02T14:04:01.763 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.763 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.763 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-04-02T14:04:01.763 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.763 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.763 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.763 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-02T14:04:01.763 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.763 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.764 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.764 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.764 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2024-04-02T14:04:01.764 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2024-04-02T14:04:01.764 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.764 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.764 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.764 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "upmap", 2024-04-02T14:04:01.765 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.765 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.765 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.765 INFO:teuthology.orchestra.run.smithi052.stdout: "crush-compat", 2024-04-02T14:04:01.765 INFO:teuthology.orchestra.run.smithi052.stdout: "none", 2024-04-02T14:04:01.765 INFO:teuthology.orchestra.run.smithi052.stdout: "read", 2024-04-02T14:04:01.765 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap", 2024-04-02T14:04:01.765 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap-read" 2024-04-02T14:04:01.765 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.766 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Balancer mode", 2024-04-02T14:04:01.766 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.766 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.766 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.766 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.766 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_ids": { 2024-04-02T14:04:01.766 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_ids", 2024-04-02T14:04:01.766 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.767 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.767 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.767 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.767 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.767 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.767 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.767 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-02T14:04:01.767 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.767 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.768 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.768 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.768 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-04-02T14:04:01.768 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-04-02T14:04:01.768 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.768 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.768 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.768 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-04-02T14:04:01.769 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.769 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.769 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.769 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-02T14:04:01.769 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.769 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.769 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.769 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.770 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_deviation": { 2024-04-02T14:04:01.770 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_deviation", 2024-04-02T14:04:01.770 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.770 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.770 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.770 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-04-02T14:04:01.770 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-04-02T14:04:01.770 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.771 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.771 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-02T14:04:01.771 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-02T14:04:01.771 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.771 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.771 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.771 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_optimizations": { 2024-04-02T14:04:01.771 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_optimizations", 2024-04-02T14:04:01.772 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-04-02T14:04:01.772 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.772 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.772 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-04-02T14:04:01.772 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.772 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.772 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.772 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-02T14:04:01.773 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.773 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.773 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.773 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.773 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.773 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.773 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.773 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm", 2024-04-02T14:04:01.773 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.774 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.774 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.774 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_down_multiplier": { 2024-04-02T14:04:01.774 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_down_multiplier", 2024-04-02T14:04:01.774 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-04-02T14:04:01.774 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.774 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.774 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2024-04-02T14:04:01.775 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.775 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.775 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.775 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-02T14:04:01.775 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.775 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.775 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.776 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.776 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_refresh_rate": { 2024-04-02T14:04:01.776 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_refresh_rate", 2024-04-02T14:04:01.776 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.776 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.776 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.776 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "20", 2024-04-02T14:04:01.776 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.777 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.777 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.777 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-02T14:04:01.777 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.777 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.777 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.777 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.777 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_starting_port": { 2024-04-02T14:04:01.778 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_starting_port", 2024-04-02T14:04:01.778 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.778 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.778 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.778 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4721", 2024-04-02T14:04:01.778 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.778 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.778 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.779 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-02T14:04:01.779 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.779 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.779 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.779 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.779 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_ptrace": { 2024-04-02T14:04:01.779 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_ptrace", 2024-04-02T14:04:01.779 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.780 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.780 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.780 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.780 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.780 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.780 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.780 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-02T14:04:01.780 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-02T14:04:01.780 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.781 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.781 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.781 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_interval": { 2024-04-02T14:04:01.781 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_interval", 2024-04-02T14:04:01.781 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.781 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.781 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.781 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-04-02T14:04:01.781 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.782 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.782 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.782 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-02T14:04:01.782 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.782 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.782 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.782 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.782 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_memory_target_ratio": { 2024-04-02T14:04:01.783 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_memory_target_ratio", 2024-04-02T14:04:01.783 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-04-02T14:04:01.783 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.783 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.783 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.7", 2024-04-02T14:04:01.783 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.783 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.784 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.784 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-02T14:04:01.784 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.784 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.784 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.784 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.784 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm_log_destination": { 2024-04-02T14:04:01.784 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm_log_destination", 2024-04-02T14:04:01.785 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.785 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.785 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.785 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.785 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.785 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.785 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.785 INFO:teuthology.orchestra.run.smithi052.stdout: "file", 2024-04-02T14:04:01.785 INFO:teuthology.orchestra.run.smithi052.stdout: "file,syslog", 2024-04-02T14:04:01.786 INFO:teuthology.orchestra.run.smithi052.stdout: "syslog" 2024-04-02T14:04:01.786 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.786 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-02T14:04:01.786 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.786 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.786 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.786 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.786 INFO:teuthology.orchestra.run.smithi052.stdout: "cgroups_split": { 2024-04-02T14:04:01.786 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cgroups_split", 2024-04-02T14:04:01.787 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.787 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.787 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.787 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.787 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.787 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.787 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.787 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-02T14:04:01.788 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.788 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.788 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.788 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.788 INFO:teuthology.orchestra.run.smithi052.stdout: "config_checks_enabled": { 2024-04-02T14:04:01.788 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_checks_enabled", 2024-04-02T14:04:01.788 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.788 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.789 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.789 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.789 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.789 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.789 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.789 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-02T14:04:01.789 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.789 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.790 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.790 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.790 INFO:teuthology.orchestra.run.smithi052.stdout: "config_dashboard": { 2024-04-02T14:04:01.790 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_dashboard", 2024-04-02T14:04:01.790 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.790 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.790 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.790 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.791 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.791 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.791 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.791 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-02T14:04:01.791 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.791 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.791 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.791 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.791 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_alertmanager": { 2024-04-02T14:04:01.792 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_alertmanager", 2024-04-02T14:04:01.792 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.792 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.792 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.792 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-02T14:04:01.792 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.792 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.792 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.792 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-04-02T14:04:01.793 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.793 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.793 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.793 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.793 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_base": { 2024-04-02T14:04:01.793 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_base", 2024-04-02T14:04:01.793 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.793 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.794 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.794 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-02T14:04:01.794 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.794 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.794 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.794 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Container image name, without the tag", 2024-04-02T14:04:01.794 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.795 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.795 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.795 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.795 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_elasticsearch": { 2024-04-02T14:04:01.795 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_elasticsearch", 2024-04-02T14:04:01.795 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.795 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.795 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.795 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-02T14:04:01.796 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.796 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.796 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.796 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "elasticsearch container image", 2024-04-02T14:04:01.796 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.796 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.796 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.796 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.797 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_grafana": { 2024-04-02T14:04:01.797 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_grafana", 2024-04-02T14:04:01.797 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.797 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.797 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.797 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-02T14:04:01.797 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.797 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.797 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.798 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-04-02T14:04:01.798 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.798 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.798 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.798 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.798 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_haproxy": { 2024-04-02T14:04:01.798 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_haproxy", 2024-04-02T14:04:01.799 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.799 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.799 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.799 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-02T14:04:01.799 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.799 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.799 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.799 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "HAproxy container image", 2024-04-02T14:04:01.799 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.800 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.800 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.800 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.800 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_agent": { 2024-04-02T14:04:01.800 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_agent", 2024-04-02T14:04:01.800 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.800 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.800 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.801 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-02T14:04:01.801 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.801 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.801 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.801 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger agent container image", 2024-04-02T14:04:01.801 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.801 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.801 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.802 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.802 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_collector": { 2024-04-02T14:04:01.802 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_collector", 2024-04-02T14:04:01.802 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.802 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.802 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.802 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-02T14:04:01.802 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.803 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.803 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.803 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger collector container image", 2024-04-02T14:04:01.803 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.803 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.803 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.803 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.803 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_query": { 2024-04-02T14:04:01.803 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_query", 2024-04-02T14:04:01.804 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.804 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.804 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.804 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-02T14:04:01.804 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.804 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.804 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.804 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger query container image", 2024-04-02T14:04:01.805 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.805 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.805 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.805 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.805 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_keepalived": { 2024-04-02T14:04:01.805 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_keepalived", 2024-04-02T14:04:01.805 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.805 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.806 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.806 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-02T14:04:01.806 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.806 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.806 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.806 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Keepalived container image", 2024-04-02T14:04:01.806 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.806 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.806 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.807 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.807 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_loki": { 2024-04-02T14:04:01.807 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_loki", 2024-04-02T14:04:01.807 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.807 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.807 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.807 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-02T14:04:01.807 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.808 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.808 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.808 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Loki container image", 2024-04-02T14:04:01.808 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.808 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.808 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.808 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.809 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_node_exporter": { 2024-04-02T14:04:01.809 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_node_exporter", 2024-04-02T14:04:01.809 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.809 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.809 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.809 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-02T14:04:01.809 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.809 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.809 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.810 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-04-02T14:04:01.810 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.810 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.810 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.810 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.810 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_nvmeof": { 2024-04-02T14:04:01.810 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_nvmeof", 2024-04-02T14:04:01.810 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.811 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.811 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.811 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-04-02T14:04:01.811 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.811 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.811 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.811 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Nvme-of container image", 2024-04-02T14:04:01.811 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.811 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.812 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.812 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.812 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_prometheus": { 2024-04-02T14:04:01.812 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_prometheus", 2024-04-02T14:04:01.812 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.812 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.812 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.812 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-02T14:04:01.813 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.813 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.813 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.813 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-04-02T14:04:01.813 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.813 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.813 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.813 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.813 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_promtail": { 2024-04-02T14:04:01.813 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_promtail", 2024-04-02T14:04:01.814 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.814 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.814 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.814 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-02T14:04:01.814 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.814 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.814 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.814 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Promtail container image", 2024-04-02T14:04:01.815 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.815 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.815 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.815 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.815 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_samba": { 2024-04-02T14:04:01.815 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_samba", 2024-04-02T14:04:01.815 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.815 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.816 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.816 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-02T14:04:01.816 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.816 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.816 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.816 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Samba/SMB container image", 2024-04-02T14:04:01.816 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.816 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.816 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.817 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.817 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_snmp_gateway": { 2024-04-02T14:04:01.817 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_snmp_gateway", 2024-04-02T14:04:01.817 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.817 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.817 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.817 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-02T14:04:01.817 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.817 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.818 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.818 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SNMP Gateway container image", 2024-04-02T14:04:01.818 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.818 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.818 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.818 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.818 INFO:teuthology.orchestra.run.smithi052.stdout: "container_init": { 2024-04-02T14:04:01.818 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_init", 2024-04-02T14:04:01.819 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.819 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.819 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.819 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.819 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.819 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.819 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.819 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Run podman/docker with `--init`", 2024-04-02T14:04:01.820 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.820 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.820 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.820 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.820 INFO:teuthology.orchestra.run.smithi052.stdout: "daemon_cache_timeout": { 2024-04-02T14:04:01.820 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "daemon_cache_timeout", 2024-04-02T14:04:01.820 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.820 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.820 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.821 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-04-02T14:04:01.821 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.821 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.821 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.821 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-02T14:04:01.821 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.821 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.821 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.821 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.822 INFO:teuthology.orchestra.run.smithi052.stdout: "default_cephadm_command_timeout": { 2024-04-02T14:04:01.822 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_cephadm_command_timeout", 2024-04-02T14:04:01.822 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.822 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.822 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.822 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "900", 2024-04-02T14:04:01.822 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.822 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.823 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.823 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-02T14:04:01.823 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.823 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.823 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.823 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.823 INFO:teuthology.orchestra.run.smithi052.stdout: "default_registry": { 2024-04-02T14:04:01.823 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_registry", 2024-04-02T14:04:01.824 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.824 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.824 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.824 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io", 2024-04-02T14:04:01.824 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.824 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.824 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.824 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-02T14:04:01.825 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.825 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.825 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.825 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.825 INFO:teuthology.orchestra.run.smithi052.stdout: "device_cache_timeout": { 2024-04-02T14:04:01.825 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_cache_timeout", 2024-04-02T14:04:01.825 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.825 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.826 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.826 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1800", 2024-04-02T14:04:01.826 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.826 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.826 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.826 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache device inventory", 2024-04-02T14:04:01.826 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.826 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.827 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.827 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.827 INFO:teuthology.orchestra.run.smithi052.stdout: "device_enhanced_scan": { 2024-04-02T14:04:01.827 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_enhanced_scan", 2024-04-02T14:04:01.827 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.827 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.827 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.827 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.828 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.828 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.828 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.828 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-02T14:04:01.828 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.828 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.828 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.828 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.828 INFO:teuthology.orchestra.run.smithi052.stdout: "facts_cache_timeout": { 2024-04-02T14:04:01.829 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "facts_cache_timeout", 2024-04-02T14:04:01.829 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.829 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.829 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.829 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-04-02T14:04:01.829 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.829 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.829 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.830 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache host facts data", 2024-04-02T14:04:01.830 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.830 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.830 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.830 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.830 INFO:teuthology.orchestra.run.smithi052.stdout: "grafana_dashboards_path": { 2024-04-02T14:04:01.830 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "grafana_dashboards_path", 2024-04-02T14:04:01.830 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.831 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.831 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.831 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-02T14:04:01.831 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.831 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.831 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.831 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-02T14:04:01.831 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.831 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.832 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.832 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.832 INFO:teuthology.orchestra.run.smithi052.stdout: "host_check_interval": { 2024-04-02T14:04:01.832 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "host_check_interval", 2024-04-02T14:04:01.832 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.832 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.832 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.832 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-04-02T14:04:01.833 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.833 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.833 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.833 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to perform a host check", 2024-04-02T14:04:01.833 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.833 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.833 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.833 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.833 INFO:teuthology.orchestra.run.smithi052.stdout: "hw_monitoring": { 2024-04-02T14:04:01.834 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hw_monitoring", 2024-04-02T14:04:01.834 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.834 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.834 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.834 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.834 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.834 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.834 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.835 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-02T14:04:01.835 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.835 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.835 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.835 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.835 INFO:teuthology.orchestra.run.smithi052.stdout: "inventory_list_all": { 2024-04-02T14:04:01.835 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "inventory_list_all", 2024-04-02T14:04:01.835 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.835 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.836 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.836 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.836 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.836 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.836 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.836 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-02T14:04:01.836 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.836 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.836 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.837 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.837 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.837 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.837 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.837 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.837 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.837 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.837 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.837 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.837 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.838 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.838 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.838 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.838 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.838 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.838 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.838 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.838 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.839 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.839 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.839 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.839 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.839 INFO:teuthology.orchestra.run.smithi052.stdout: "log_refresh_metadata": { 2024-04-02T14:04:01.839 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_refresh_metadata", 2024-04-02T14:04:01.839 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.839 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.839 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.840 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.840 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.840 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.840 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.840 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-02T14:04:01.840 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.840 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.840 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.841 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.841 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.841 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.841 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.841 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.841 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.841 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.841 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.842 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.842 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.842 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-02T14:04:01.842 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.842 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.842 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.842 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.842 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.842 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.843 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.843 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.843 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.843 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.843 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.843 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.843 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.843 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.843 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.844 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.844 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.844 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.846 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.846 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.846 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.846 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.846 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.846 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.847 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.847 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.847 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.847 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.847 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.847 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.847 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.847 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.847 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.847 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.848 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.848 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.848 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.848 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.848 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.848 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-02T14:04:01.848 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-02T14:04:01.848 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.848 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.849 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.849 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.849 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.849 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.849 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.849 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-02T14:04:01.849 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.849 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.850 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.850 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.850 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-02T14:04:01.850 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-02T14:04:01.850 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.850 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.850 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.850 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "*", 2024-04-02T14:04:01.851 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.851 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.851 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.851 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-02T14:04:01.851 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.851 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.851 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.851 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.851 INFO:teuthology.orchestra.run.smithi052.stdout: "max_count_per_host": { 2024-04-02T14:04:01.852 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_count_per_host", 2024-04-02T14:04:01.852 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.852 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.852 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.852 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-04-02T14:04:01.852 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.852 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.852 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.853 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of daemons per service per host", 2024-04-02T14:04:01.853 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.853 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.853 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.853 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.853 INFO:teuthology.orchestra.run.smithi052.stdout: "max_osd_draining_count": { 2024-04-02T14:04:01.853 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_osd_draining_count", 2024-04-02T14:04:01.853 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.853 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.854 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.854 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-04-02T14:04:01.854 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.854 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.854 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.854 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-02T14:04:01.854 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.855 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.855 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.855 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.855 INFO:teuthology.orchestra.run.smithi052.stdout: "migration_current": { 2024-04-02T14:04:01.855 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "migration_current", 2024-04-02T14:04:01.855 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.855 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.855 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.855 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.856 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.856 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.856 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.856 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "internal - do not modify", 2024-04-02T14:04:01.856 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.856 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.856 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.856 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.856 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2024-04-02T14:04:01.857 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2024-04-02T14:04:01.857 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.857 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.857 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.857 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "root", 2024-04-02T14:04:01.857 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.857 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.857 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.858 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm-package", 2024-04-02T14:04:01.858 INFO:teuthology.orchestra.run.smithi052.stdout: "root" 2024-04-02T14:04:01.858 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.858 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "mode for remote execution of cephadm", 2024-04-02T14:04:01.858 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.858 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.858 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.859 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.859 INFO:teuthology.orchestra.run.smithi052.stdout: "oob_default_addr": { 2024-04-02T14:04:01.859 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "oob_default_addr", 2024-04-02T14:04:01.859 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.859 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.859 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.859 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "169.254.1.1", 2024-04-02T14:04:01.859 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.859 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.860 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.860 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-02T14:04:01.860 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.860 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.860 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.860 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.860 INFO:teuthology.orchestra.run.smithi052.stdout: "prometheus_alerts_path": { 2024-04-02T14:04:01.860 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus_alerts_path", 2024-04-02T14:04:01.860 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.860 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.861 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.861 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-02T14:04:01.861 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.861 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.861 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.861 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-02T14:04:01.861 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.861 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.861 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.862 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.862 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_insecure": { 2024-04-02T14:04:01.862 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_insecure", 2024-04-02T14:04:01.862 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.862 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.862 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.862 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.862 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.863 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.863 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.863 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-02T14:04:01.863 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.863 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.863 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.863 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.863 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_password": { 2024-04-02T14:04:01.863 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_password", 2024-04-02T14:04:01.864 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.864 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.864 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.864 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.864 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.864 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.864 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.864 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-02T14:04:01.865 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.865 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.865 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.865 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.865 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_url": { 2024-04-02T14:04:01.865 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_url", 2024-04-02T14:04:01.865 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.865 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.865 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.866 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.866 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.866 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.866 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.866 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-02T14:04:01.866 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.866 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.866 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.867 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.867 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_username": { 2024-04-02T14:04:01.867 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_username", 2024-04-02T14:04:01.867 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.867 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.867 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.867 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.867 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.867 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.867 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.868 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-02T14:04:01.868 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.868 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.868 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.868 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.868 INFO:teuthology.orchestra.run.smithi052.stdout: "secure_monitoring_stack": { 2024-04-02T14:04:01.868 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "secure_monitoring_stack", 2024-04-02T14:04:01.868 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.868 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.869 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.869 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.869 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.869 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.869 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.869 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-02T14:04:01.869 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.869 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.869 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.870 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.870 INFO:teuthology.orchestra.run.smithi052.stdout: "service_discovery_port": { 2024-04-02T14:04:01.870 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "service_discovery_port", 2024-04-02T14:04:01.870 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.870 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.870 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.870 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8765", 2024-04-02T14:04:01.870 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.871 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.871 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.871 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "cephadm service discovery port", 2024-04-02T14:04:01.871 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.871 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.871 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.871 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.871 INFO:teuthology.orchestra.run.smithi052.stdout: "ssh_config_file": { 2024-04-02T14:04:01.871 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssh_config_file", 2024-04-02T14:04:01.872 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.872 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.872 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.872 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.872 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.872 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.872 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.872 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-02T14:04:01.872 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.873 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.873 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.873 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.873 INFO:teuthology.orchestra.run.smithi052.stdout: "use_agent": { 2024-04-02T14:04:01.873 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_agent", 2024-04-02T14:04:01.873 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.873 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.873 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.873 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.874 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.874 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.874 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.874 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-02T14:04:01.874 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.874 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.874 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.874 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.875 INFO:teuthology.orchestra.run.smithi052.stdout: "use_repo_digest": { 2024-04-02T14:04:01.875 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_repo_digest", 2024-04-02T14:04:01.875 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.875 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.875 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.875 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.875 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.875 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.875 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.876 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-02T14:04:01.876 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.876 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.876 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.876 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.876 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_failed_host_check": { 2024-04-02T14:04:01.876 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_failed_host_check", 2024-04-02T14:04:01.876 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.876 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.876 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.877 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.877 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.877 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.877 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.877 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if the host check fails", 2024-04-02T14:04:01.877 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.877 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.877 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.878 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.878 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_daemons": { 2024-04-02T14:04:01.878 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_daemons", 2024-04-02T14:04:01.878 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.878 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.878 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.878 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.878 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.879 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.879 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.879 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-02T14:04:01.879 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.879 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.879 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.879 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.879 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_hosts": { 2024-04-02T14:04:01.880 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_hosts", 2024-04-02T14:04:01.880 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.880 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.880 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.880 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.880 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.880 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.880 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.881 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-02T14:04:01.881 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.881 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.881 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.881 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.881 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.881 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.881 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.881 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crash", 2024-04-02T14:04:01.882 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.882 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.882 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.882 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.882 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.882 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.882 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.882 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.883 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.883 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.883 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.883 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.883 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.883 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.883 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.883 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.884 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.884 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.884 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.884 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.884 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.884 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.884 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.884 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.885 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.885 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.885 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.885 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.885 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.885 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.885 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.885 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.886 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.886 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.886 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.886 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.886 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.886 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.886 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.886 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.887 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.887 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.887 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.887 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.887 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.887 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.887 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.887 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.888 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.888 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.888 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.888 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.888 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.888 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.888 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.888 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.889 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.889 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.889 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.889 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.889 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.889 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.889 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.889 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.889 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.890 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.890 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.890 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.890 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.890 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.890 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.890 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.890 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.891 INFO:teuthology.orchestra.run.smithi052.stdout: "retain_interval": { 2024-04-02T14:04:01.891 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retain_interval", 2024-04-02T14:04:01.891 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.891 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.891 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.891 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "31536000", 2024-04-02T14:04:01.891 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.891 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.892 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.892 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-02T14:04:01.892 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.892 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.892 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.892 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.892 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_recent_interval": { 2024-04-02T14:04:01.892 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_recent_interval", 2024-04-02T14:04:01.892 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:01.893 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.893 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.893 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1209600", 2024-04-02T14:04:01.893 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.893 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.893 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.893 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-02T14:04:01.893 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.894 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.894 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.894 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.894 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:01.894 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.894 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:01.894 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dashboard", 2024-04-02T14:04:01.894 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:01.895 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:01.895 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:01.895 INFO:teuthology.orchestra.run.smithi052.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-02T14:04:01.895 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-02T14:04:01.895 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.895 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.895 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.895 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-04-02T14:04:01.895 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.896 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.896 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.896 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.896 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.896 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.896 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.896 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.896 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_HOST": { 2024-04-02T14:04:01.896 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-02T14:04:01.897 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.897 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.897 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.897 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.897 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.897 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.897 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.897 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.898 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.898 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.898 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.898 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.898 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-02T14:04:01.898 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-02T14:04:01.898 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.898 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.899 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.899 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.899 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.899 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.899 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.899 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.899 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.899 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.900 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.900 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.900 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_ENABLED": { 2024-04-02T14:04:01.900 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_ENABLED", 2024-04-02T14:04:01.900 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.900 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.900 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.900 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.901 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.901 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.901 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.901 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.901 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.901 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.901 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.901 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.901 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-02T14:04:01.902 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-02T14:04:01.902 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.902 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.902 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.902 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.902 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.902 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.903 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.903 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.903 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.903 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.903 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.903 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.903 INFO:teuthology.orchestra.run.smithi052.stdout: "ENABLE_BROWSABLE_API": { 2024-04-02T14:04:01.903 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-02T14:04:01.903 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.904 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.904 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.904 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.904 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.904 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.904 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.904 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.904 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.905 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.905 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.905 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.905 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-02T14:04:01.905 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-02T14:04:01.905 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.905 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.905 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.905 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.906 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.906 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.906 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.906 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.906 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.906 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.906 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.906 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.907 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-02T14:04:01.907 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-02T14:04:01.907 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.907 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.907 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.907 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.907 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.907 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.908 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.908 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.908 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.908 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.908 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.908 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.908 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-02T14:04:01.908 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-02T14:04:01.909 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.909 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.909 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.909 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.909 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.909 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.909 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.909 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.910 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.910 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.910 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.910 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.910 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-02T14:04:01.910 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-02T14:04:01.910 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.911 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.911 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.911 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.911 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.911 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.911 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.911 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.911 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.912 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.912 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.912 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.912 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-02T14:04:01.912 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-02T14:04:01.912 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.912 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.912 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.913 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.913 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.913 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.913 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.913 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.913 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.913 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.913 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.914 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.914 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-02T14:04:01.914 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-02T14:04:01.914 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.914 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.914 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.914 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.914 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.915 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.915 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.915 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.915 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.915 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.915 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.915 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.916 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-02T14:04:01.916 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-02T14:04:01.916 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.916 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.916 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.916 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.916 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.917 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.917 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.917 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.917 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.917 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.917 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.917 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.917 INFO:teuthology.orchestra.run.smithi052.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-02T14:04:01.918 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-02T14:04:01.918 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.918 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.918 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.918 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.918 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.918 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.918 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.919 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.919 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.919 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.919 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.919 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.919 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_PASSWORD": { 2024-04-02T14:04:01.919 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-02T14:04:01.919 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.920 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.920 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.920 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-04-02T14:04:01.920 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.920 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.920 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.920 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.920 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.921 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.921 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.921 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.921 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-02T14:04:01.921 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-02T14:04:01.921 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.921 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.921 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.922 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.922 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.922 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.922 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.922 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.922 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.922 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.922 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.923 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.923 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_URL": { 2024-04-02T14:04:01.923 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_URL", 2024-04-02T14:04:01.923 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.923 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.923 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.923 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.923 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.923 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.924 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.924 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.924 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.924 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.924 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.924 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.924 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_USERNAME": { 2024-04-02T14:04:01.924 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-02T14:04:01.925 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.925 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.925 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.925 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-04-02T14:04:01.925 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.925 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.925 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.925 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.925 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.926 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.926 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.926 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.926 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-02T14:04:01.926 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-02T14:04:01.926 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.926 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.926 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.927 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.927 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.927 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.927 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.927 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.927 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.927 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.927 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.928 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.928 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-02T14:04:01.928 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-02T14:04:01.928 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.928 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.928 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.928 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.928 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.929 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.929 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.929 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.929 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.929 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.929 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.929 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.929 INFO:teuthology.orchestra.run.smithi052.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-02T14:04:01.930 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-02T14:04:01.930 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.930 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.930 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.930 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.930 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.930 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.931 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.931 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.931 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.931 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.931 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.931 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.931 INFO:teuthology.orchestra.run.smithi052.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-02T14:04:01.931 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-02T14:04:01.932 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.932 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.932 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.932 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.932 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.932 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.932 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.932 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.933 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.933 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.933 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.933 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.933 INFO:teuthology.orchestra.run.smithi052.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-02T14:04:01.933 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-02T14:04:01.933 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.933 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.934 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.934 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "[]", 2024-04-02T14:04:01.934 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.934 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.934 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.934 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.934 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.934 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.935 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.935 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.935 INFO:teuthology.orchestra.run.smithi052.stdout: "MULTICLUSTER_CONFIG": { 2024-04-02T14:04:01.935 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-02T14:04:01.935 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.935 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.935 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.935 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "{}", 2024-04-02T14:04:01.936 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.936 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.936 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.936 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.936 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.936 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.936 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.937 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.937 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_HOST": { 2024-04-02T14:04:01.937 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-02T14:04:01.937 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.937 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.937 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.937 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.937 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.938 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.938 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.938 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.938 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.938 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.938 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.938 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.938 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-02T14:04:01.939 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-02T14:04:01.939 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.939 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.939 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.939 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.939 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.939 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.939 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.940 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.940 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.940 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.940 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.940 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.940 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-02T14:04:01.940 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-02T14:04:01.940 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.941 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.941 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.941 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.941 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.941 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.941 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.941 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.941 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.941 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.942 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.942 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.942 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-02T14:04:01.942 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-02T14:04:01.942 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.942 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.942 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.942 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.943 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.943 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.943 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.943 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.943 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.943 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.943 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.943 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.944 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-02T14:04:01.944 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-02T14:04:01.944 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.944 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.944 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.944 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.944 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.944 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.945 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.945 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.945 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.945 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.945 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.945 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.945 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-02T14:04:01.945 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-02T14:04:01.945 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.946 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.946 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.946 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.946 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.946 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.946 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.946 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.946 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.947 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.947 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.947 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.947 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-02T14:04:01.947 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-02T14:04:01.947 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.947 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.947 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.947 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.948 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.948 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.948 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.948 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.948 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.948 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.948 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.948 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.948 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-02T14:04:01.949 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-02T14:04:01.949 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.949 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.949 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.949 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.949 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.949 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.949 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.950 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.950 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.950 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.950 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.950 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.950 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-02T14:04:01.950 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-02T14:04:01.950 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.951 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.951 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.951 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.951 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.951 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.951 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.951 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.952 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.952 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.952 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.952 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.952 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_ENABLED": { 2024-04-02T14:04:01.952 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-02T14:04:01.952 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.952 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.952 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.953 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.953 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.953 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.953 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.953 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.953 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.953 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.953 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.954 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.954 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-02T14:04:01.954 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-02T14:04:01.954 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.954 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.954 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.954 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-02T14:04:01.955 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.955 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.955 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.955 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.955 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.955 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.955 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.955 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.955 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-02T14:04:01.956 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-02T14:04:01.956 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.956 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.956 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.956 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-04-02T14:04:01.956 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.956 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.956 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.957 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.957 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.957 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.957 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.957 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.957 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-02T14:04:01.957 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-02T14:04:01.957 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.958 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.958 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.958 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8", 2024-04-02T14:04:01.958 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.958 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.958 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.958 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.958 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.959 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.959 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.959 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.959 INFO:teuthology.orchestra.run.smithi052.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-02T14:04:01.959 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-02T14:04:01.959 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.959 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.959 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.959 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "45", 2024-04-02T14:04:01.960 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.960 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.960 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.960 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.960 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.960 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.960 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.960 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.961 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ACCESS_KEY": { 2024-04-02T14:04:01.961 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-02T14:04:01.961 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.961 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.961 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.961 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.961 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.961 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.961 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.961 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.962 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.962 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.962 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.962 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.962 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-02T14:04:01.962 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-02T14:04:01.962 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.962 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.963 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.963 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-04-02T14:04:01.963 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.963 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.963 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.963 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.963 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.963 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.963 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.964 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.964 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SECRET_KEY": { 2024-04-02T14:04:01.964 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-02T14:04:01.964 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.964 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.964 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.964 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.964 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.964 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.965 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.965 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.965 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.965 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.965 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.965 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.965 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SSL_VERIFY": { 2024-04-02T14:04:01.965 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-02T14:04:01.965 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.966 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.966 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.966 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.966 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.966 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.966 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.966 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.966 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.967 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.967 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.967 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.967 INFO:teuthology.orchestra.run.smithi052.stdout: "UNSAFE_TLS_v1_2": { 2024-04-02T14:04:01.967 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-02T14:04:01.967 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.967 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.967 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.968 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.968 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.968 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.968 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.968 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.968 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.968 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.968 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.968 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.969 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-02T14:04:01.969 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-02T14:04:01.969 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.969 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.969 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.969 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-04-02T14:04:01.969 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.969 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.970 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.970 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.970 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.970 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.970 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.970 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.970 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-02T14:04:01.970 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-02T14:04:01.971 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.971 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.971 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.971 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-04-02T14:04:01.971 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.971 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.971 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.971 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.971 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.972 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.972 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.972 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.972 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-02T14:04:01.972 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-02T14:04:01.972 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.972 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.972 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.973 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-04-02T14:04:01.973 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.973 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.973 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.973 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.973 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.973 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.973 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.973 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.974 INFO:teuthology.orchestra.run.smithi052.stdout: "cross_origin_url": { 2024-04-02T14:04:01.974 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cross_origin_url", 2024-04-02T14:04:01.974 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.974 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.974 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.974 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.974 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.974 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.975 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.975 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.975 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.975 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.975 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.975 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.975 INFO:teuthology.orchestra.run.smithi052.stdout: "crt_file": { 2024-04-02T14:04:01.975 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crt_file", 2024-04-02T14:04:01.976 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.976 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.976 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.976 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.976 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.976 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.976 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.976 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.976 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.977 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.977 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.977 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.977 INFO:teuthology.orchestra.run.smithi052.stdout: "debug": { 2024-04-02T14:04:01.977 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "debug", 2024-04-02T14:04:01.977 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.977 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.977 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.977 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.978 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.978 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.978 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.978 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable/disable debug options", 2024-04-02T14:04:01.978 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.978 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.978 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.978 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.979 INFO:teuthology.orchestra.run.smithi052.stdout: "jwt_token_ttl": { 2024-04-02T14:04:01.979 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "jwt_token_ttl", 2024-04-02T14:04:01.979 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.979 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.979 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.979 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "28800", 2024-04-02T14:04:01.979 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.979 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.980 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.980 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.980 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.980 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.980 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.980 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.980 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2024-04-02T14:04:01.980 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2024-04-02T14:04:01.980 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.981 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.981 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.981 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.981 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.981 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.981 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.981 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.981 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.982 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.982 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.982 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.982 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:01.982 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:01.982 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.982 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.982 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.983 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.983 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.983 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.983 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.983 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.983 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.983 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.983 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.984 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.984 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.984 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.984 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.984 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.984 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.984 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.984 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.985 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:01.985 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:01.985 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.985 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.985 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.985 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.985 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.985 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.986 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.986 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.986 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.986 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.986 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.986 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.986 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:01.986 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:01.987 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.987 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.987 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.987 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:01.987 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.987 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.987 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:01.987 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:01.988 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:01.988 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:01.988 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:01.988 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:01.988 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:01.988 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:01.988 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.988 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.989 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.989 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.989 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.989 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:01.989 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:01.989 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.989 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.989 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:01.990 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.990 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.990 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.990 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.990 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.990 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.990 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.990 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.991 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.991 INFO:teuthology.orchestra.run.smithi052.stdout: "motd": { 2024-04-02T14:04:01.991 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "motd", 2024-04-02T14:04:01.991 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.991 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.991 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.991 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:01.992 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.992 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.992 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.992 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "The message of the day", 2024-04-02T14:04:01.992 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.992 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.992 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.992 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.993 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect_resolve_ip_addr": { 2024-04-02T14:04:01.993 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "redirect_resolve_ip_addr", 2024-04-02T14:04:01.993 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.993 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.993 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.993 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:01.993 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.993 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.994 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.994 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.994 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.994 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.994 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.994 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.994 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-04-02T14:04:01.995 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-04-02T14:04:01.995 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:01.995 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.995 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.995 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2024-04-02T14:04:01.995 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.995 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.996 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.996 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.996 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.996 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.996 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.996 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.996 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-04-02T14:04:01.996 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-04-02T14:04:01.997 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:01.997 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.997 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.997 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8080", 2024-04-02T14:04:01.997 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.997 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.997 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.997 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.997 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.998 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.998 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.998 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:01.998 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2024-04-02T14:04:01.998 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2024-04-02T14:04:01.998 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:01.998 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:01.998 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:01.999 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:01.999 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:01.999 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:01.999 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:01.999 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:01.999 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:01.999 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:01.999 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:01.999 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.000 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl_server_port": { 2024-04-02T14:04:02.000 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl_server_port", 2024-04-02T14:04:02.000 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:02.000 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.000 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.000 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8443", 2024-04-02T14:04:02.000 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.000 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.000 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.001 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.001 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.001 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.001 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.001 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.001 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2024-04-02T14:04:02.001 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2024-04-02T14:04:02.001 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.002 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.002 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.002 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "redirect", 2024-04-02T14:04:02.002 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.002 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.002 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.002 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.002 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect" 2024-04-02T14:04:02.003 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.003 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.003 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.003 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.003 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.003 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.003 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2024-04-02T14:04:02.003 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2024-04-02T14:04:02.004 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:02.004 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.004 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.004 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2024-04-02T14:04:02.004 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2024-04-02T14:04:02.004 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2024-04-02T14:04:02.004 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.004 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.005 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.005 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.005 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.005 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.005 INFO:teuthology.orchestra.run.smithi052.stdout: "url_prefix": { 2024-04-02T14:04:02.005 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url_prefix", 2024-04-02T14:04:02.005 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.005 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.006 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.006 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.006 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.006 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.006 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.006 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.006 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.007 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.007 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.007 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.007 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.007 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.007 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.007 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "devicehealth", 2024-04-02T14:04:02.007 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.008 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.008 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.008 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_monitoring": { 2024-04-02T14:04:02.008 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_monitoring", 2024-04-02T14:04:02.008 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.008 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.008 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.008 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:02.009 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.009 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.009 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.009 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "monitor device health metrics", 2024-04-02T14:04:02.009 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.009 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.009 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.009 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.009 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.010 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.010 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.010 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.010 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.010 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.010 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.010 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.011 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.011 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.011 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.011 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.011 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.011 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.011 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.011 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.011 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.012 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.012 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.012 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.012 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.012 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.012 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.012 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.012 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.012 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.013 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.013 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.013 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.013 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.013 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.013 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.013 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.013 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.013 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.014 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.014 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.014 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.014 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.014 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.014 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.014 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.015 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.015 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.015 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.015 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.015 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.015 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.015 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.015 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.015 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.016 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.016 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.016 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.016 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.016 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.016 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.016 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.016 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.016 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.017 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.017 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.017 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.017 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.017 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.017 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.017 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.017 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.017 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.018 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.018 INFO:teuthology.orchestra.run.smithi052.stdout: "mark_out_threshold": { 2024-04-02T14:04:02.018 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mark_out_threshold", 2024-04-02T14:04:02.018 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:02.018 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.018 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.018 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2419200", 2024-04-02T14:04:02.018 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.018 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.019 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.019 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-02T14:04:02.019 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.019 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.019 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.019 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.019 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_name": { 2024-04-02T14:04:02.019 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_name", 2024-04-02T14:04:02.020 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.020 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.020 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.020 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "device_health_metrics", 2024-04-02T14:04:02.020 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.020 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.020 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.020 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-02T14:04:02.020 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.021 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.021 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.021 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.021 INFO:teuthology.orchestra.run.smithi052.stdout: "retention_period": { 2024-04-02T14:04:02.021 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retention_period", 2024-04-02T14:04:02.021 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:02.021 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.021 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.021 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15552000", 2024-04-02T14:04:02.021 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.022 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.022 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.022 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain device health metrics", 2024-04-02T14:04:02.022 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.022 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.022 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.023 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.023 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_frequency": { 2024-04-02T14:04:02.023 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_frequency", 2024-04-02T14:04:02.023 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:02.023 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.023 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.023 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2024-04-02T14:04:02.023 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.023 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.024 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.024 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-02T14:04:02.024 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.024 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.024 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.024 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.024 INFO:teuthology.orchestra.run.smithi052.stdout: "self_heal": { 2024-04-02T14:04:02.024 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "self_heal", 2024-04-02T14:04:02.024 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.025 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.025 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.025 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:02.025 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.025 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.025 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.025 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-02T14:04:02.025 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.025 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.026 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.026 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.026 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-04-02T14:04:02.026 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-04-02T14:04:02.026 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:02.026 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.026 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.026 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-04-02T14:04:02.027 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.027 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.027 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.027 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and check device health", 2024-04-02T14:04:02.027 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.027 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.027 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.027 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.027 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_threshold": { 2024-04-02T14:04:02.028 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_threshold", 2024-04-02T14:04:02.028 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:02.028 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.028 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.028 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7257600", 2024-04-02T14:04:02.028 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.028 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.028 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.029 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-02T14:04:02.029 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.029 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.029 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.029 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.029 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.029 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.029 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.030 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "diskprediction_local", 2024-04-02T14:04:02.030 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.030 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.030 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.030 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.030 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.030 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.030 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.031 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.031 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.031 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.031 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.031 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.031 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.031 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.031 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.032 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.032 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.032 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.032 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.032 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.032 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.032 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.032 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.032 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.033 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.033 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.033 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.033 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.033 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.033 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.033 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.033 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.034 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.034 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.034 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.034 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.034 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.034 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.034 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.034 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.034 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.035 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.035 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.035 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.035 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.035 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.035 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.035 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.035 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.036 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.036 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.036 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.036 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.036 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.036 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.036 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.036 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.036 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.037 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.037 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.037 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.037 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.037 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.037 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.037 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.037 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.037 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.038 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.038 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.038 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.038 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.038 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.038 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.038 INFO:teuthology.orchestra.run.smithi052.stdout: "predict_interval": { 2024-04-02T14:04:02.038 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predict_interval", 2024-04-02T14:04:02.039 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.039 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.039 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.039 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2024-04-02T14:04:02.039 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.039 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.039 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.039 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.040 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.040 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.040 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.040 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.040 INFO:teuthology.orchestra.run.smithi052.stdout: "predictor_model": { 2024-04-02T14:04:02.040 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predictor_model", 2024-04-02T14:04:02.040 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.040 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.041 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.041 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "prophetstor", 2024-04-02T14:04:02.041 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.041 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.041 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.041 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.041 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.042 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.042 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.042 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.042 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-04-02T14:04:02.042 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-04-02T14:04:02.042 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.043 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.043 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.043 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-04-02T14:04:02.043 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.043 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.043 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.043 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.044 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.044 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.044 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.044 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.044 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.044 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.044 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.045 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "influx", 2024-04-02T14:04:02.045 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": false, 2024-04-02T14:04:02.045 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "influxdb python module not found", 2024-04-02T14:04:02.045 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.045 INFO:teuthology.orchestra.run.smithi052.stdout: "batch_size": { 2024-04-02T14:04:02.045 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "batch_size", 2024-04-02T14:04:02.045 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:02.045 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.046 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.046 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5000", 2024-04-02T14:04:02.046 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.046 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.046 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.046 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-02T14:04:02.046 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.046 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.046 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.047 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.047 INFO:teuthology.orchestra.run.smithi052.stdout: "database": { 2024-04-02T14:04:02.047 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "database", 2024-04-02T14:04:02.047 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.047 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.047 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.047 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "ceph", 2024-04-02T14:04:02.047 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.047 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.048 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.048 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-02T14:04:02.048 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.048 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.048 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.048 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.048 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": { 2024-04-02T14:04:02.048 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hostname", 2024-04-02T14:04:02.048 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.049 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.049 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.049 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.049 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.049 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.049 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.049 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server hostname", 2024-04-02T14:04:02.049 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.050 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.050 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.050 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.050 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-04-02T14:04:02.050 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-04-02T14:04:02.050 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:02.050 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.050 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.050 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "30", 2024-04-02T14:04:02.051 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "5", 2024-04-02T14:04:02.051 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.051 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.051 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-02T14:04:02.051 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.051 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.051 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.051 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.052 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.052 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.052 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.052 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.052 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.052 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.052 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.052 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.052 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.053 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.053 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.053 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.053 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.053 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.053 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.053 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.053 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.053 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.054 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.054 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.054 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.054 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.054 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.054 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.054 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.054 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.055 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.055 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.055 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.055 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.055 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.055 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.055 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.055 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.055 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.056 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.056 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.056 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.056 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.056 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.056 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.056 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.056 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.056 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.056 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.057 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.057 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.057 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.057 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.057 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.057 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.057 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.057 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.058 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.058 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.058 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.058 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.058 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.058 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.058 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.058 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.059 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.059 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.059 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.059 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.059 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.059 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.059 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.059 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.060 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.060 INFO:teuthology.orchestra.run.smithi052.stdout: "password": { 2024-04-02T14:04:02.060 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "password", 2024-04-02T14:04:02.060 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.060 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.060 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.060 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.060 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.061 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.061 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.061 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "password of InfluxDB server user", 2024-04-02T14:04:02.061 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.061 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.061 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.061 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.061 INFO:teuthology.orchestra.run.smithi052.stdout: "port": { 2024-04-02T14:04:02.062 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "port", 2024-04-02T14:04:02.062 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:02.062 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.062 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.062 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8086", 2024-04-02T14:04:02.062 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.062 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.062 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.063 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server port", 2024-04-02T14:04:02.063 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.063 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.063 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.063 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.063 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2024-04-02T14:04:02.063 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2024-04-02T14:04:02.063 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.064 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.064 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.064 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "false", 2024-04-02T14:04:02.064 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.064 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.064 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.064 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T14:04:02.064 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.065 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.065 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.065 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.065 INFO:teuthology.orchestra.run.smithi052.stdout: "threads": { 2024-04-02T14:04:02.065 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threads", 2024-04-02T14:04:02.065 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:02.065 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.065 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.066 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-04-02T14:04:02.066 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-04-02T14:04:02.066 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "32", 2024-04-02T14:04:02.066 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.066 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-02T14:04:02.066 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.066 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.066 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.067 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.067 INFO:teuthology.orchestra.run.smithi052.stdout: "username": { 2024-04-02T14:04:02.067 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "username", 2024-04-02T14:04:02.067 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.067 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.067 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.067 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.067 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.068 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.068 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.068 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "username of InfluxDB server user", 2024-04-02T14:04:02.068 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.068 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.068 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.068 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.068 INFO:teuthology.orchestra.run.smithi052.stdout: "verify_ssl": { 2024-04-02T14:04:02.068 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "verify_ssl", 2024-04-02T14:04:02.069 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.069 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.069 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.069 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "true", 2024-04-02T14:04:02.069 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.069 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.069 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.069 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T14:04:02.069 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.070 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.070 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.070 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.070 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.070 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.070 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.070 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "insights", 2024-04-02T14:04:02.070 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.071 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.071 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.071 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.071 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.071 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.071 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.071 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.071 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.071 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.072 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.072 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.072 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.072 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.072 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.072 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.072 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.072 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.072 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.072 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.073 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.073 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.073 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.073 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.073 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.073 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.073 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.073 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.074 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.074 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.074 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.074 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.074 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.074 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.074 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.074 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.074 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.075 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.075 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.075 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.075 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.075 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.075 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.075 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.075 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.075 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.076 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.076 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.076 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.076 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.076 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.076 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.076 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.076 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.076 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.077 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.077 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.077 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.077 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.077 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.077 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.077 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.077 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.077 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.077 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.078 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.078 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.078 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.078 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.078 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.078 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.078 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.078 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.079 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.079 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.079 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.079 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "iostat", 2024-04-02T14:04:02.079 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.079 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.079 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.079 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.079 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.079 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.080 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.080 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.080 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.080 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.080 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.080 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.080 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.080 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.080 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.081 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.081 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.081 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.081 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.081 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.081 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.081 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.081 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.081 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.082 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.082 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.082 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.082 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.082 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.082 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.082 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.082 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.082 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.083 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.083 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.083 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.083 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.083 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.083 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.083 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.083 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.084 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.084 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.084 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.084 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.084 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.084 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.084 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.084 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.085 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.085 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.085 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.085 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.085 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.085 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.085 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.085 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.085 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.086 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.086 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.086 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.086 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.086 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.086 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.086 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.086 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.086 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.087 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.087 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.087 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.087 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.087 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.087 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.087 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.087 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.088 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.088 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "k8sevents", 2024-04-02T14:04:02.088 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.088 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.088 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.088 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_event_retention_days": { 2024-04-02T14:04:02.088 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ceph_event_retention_days", 2024-04-02T14:04:02.088 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:02.088 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.089 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.089 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7", 2024-04-02T14:04:02.089 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.089 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.089 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.089 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-02T14:04:02.089 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.089 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.089 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.090 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.090 INFO:teuthology.orchestra.run.smithi052.stdout: "config_check_secs": { 2024-04-02T14:04:02.090 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_check_secs", 2024-04-02T14:04:02.090 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:02.090 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.090 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.090 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-04-02T14:04:02.090 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "10", 2024-04-02T14:04:02.091 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.091 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.091 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-02T14:04:02.091 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.091 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.091 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.091 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.091 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.091 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.092 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.092 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.092 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.092 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.092 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.092 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.092 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.092 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.092 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.093 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.093 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.093 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.093 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.093 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.093 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.093 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.093 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.094 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.094 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.094 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.094 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.094 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.094 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.094 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.094 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.095 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.095 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.095 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.095 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.095 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.095 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.095 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.095 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.095 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.096 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.096 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.096 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.096 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.096 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.096 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.096 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.096 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.097 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.097 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.097 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.097 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.097 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.097 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.097 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.097 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.097 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.098 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.098 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.098 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.098 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.098 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.098 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.098 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.098 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.099 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.099 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.099 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.099 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.099 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.099 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.099 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.099 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.099 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.099 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.100 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.100 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.100 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "localpool", 2024-04-02T14:04:02.100 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.100 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.100 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.100 INFO:teuthology.orchestra.run.smithi052.stdout: "failure_domain": { 2024-04-02T14:04:02.100 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "failure_domain", 2024-04-02T14:04:02.100 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.101 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.101 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.101 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "host", 2024-04-02T14:04:02.101 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.101 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.101 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.101 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "failure domain for any created local pool", 2024-04-02T14:04:02.101 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-02T14:04:02.101 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.102 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.102 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.102 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.102 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.102 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.102 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.102 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.102 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.102 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.103 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.103 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.103 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.103 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.103 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.103 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.103 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.103 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.104 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.104 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.104 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.104 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.104 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.104 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.104 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.104 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.104 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.105 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.105 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.105 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.105 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.105 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.105 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.105 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.105 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.106 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.106 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.106 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.106 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.106 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.106 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.106 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.106 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.106 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.107 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.107 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.107 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.107 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.107 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.107 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.107 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.107 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.108 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.108 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.108 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.108 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.108 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.108 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.108 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.108 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.109 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.109 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.109 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.109 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.109 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.109 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.109 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.109 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.109 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.110 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.110 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.110 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.110 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.110 INFO:teuthology.orchestra.run.smithi052.stdout: "min_size": { 2024-04-02T14:04:02.110 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_size", 2024-04-02T14:04:02.110 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:02.110 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.111 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.111 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.111 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.111 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.111 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.111 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default min_size for any created local pool", 2024-04-02T14:04:02.111 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-02T14:04:02.111 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.111 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.112 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.112 INFO:teuthology.orchestra.run.smithi052.stdout: "num_rep": { 2024-04-02T14:04:02.112 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "num_rep", 2024-04-02T14:04:02.112 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:02.112 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.112 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.112 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3", 2024-04-02T14:04:02.112 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.113 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.113 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.113 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default replica count for any created local pool", 2024-04-02T14:04:02.113 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.113 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.113 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.113 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.113 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_num": { 2024-04-02T14:04:02.113 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_num", 2024-04-02T14:04:02.114 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:02.114 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.114 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.114 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "128", 2024-04-02T14:04:02.114 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.114 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.114 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.114 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default pg_num for any created local pool", 2024-04-02T14:04:02.115 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.115 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.115 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.115 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.115 INFO:teuthology.orchestra.run.smithi052.stdout: "prefix": { 2024-04-02T14:04:02.115 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prefix", 2024-04-02T14:04:02.115 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.115 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.115 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.116 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.116 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.116 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.116 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.116 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name prefix for any created local pool", 2024-04-02T14:04:02.116 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.116 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.116 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.117 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.117 INFO:teuthology.orchestra.run.smithi052.stdout: "subtree": { 2024-04-02T14:04:02.117 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "subtree", 2024-04-02T14:04:02.117 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.117 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.117 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.117 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "rack", 2024-04-02T14:04:02.117 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.117 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.118 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.118 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-02T14:04:02.118 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-02T14:04:02.118 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.118 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.118 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.118 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.118 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.119 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.119 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mds_autoscaler", 2024-04-02T14:04:02.119 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.119 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.119 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.119 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.119 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.120 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.120 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.120 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.120 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.120 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.120 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.120 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.120 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.121 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.121 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.121 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.121 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.121 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.121 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.121 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.121 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.121 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.122 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.122 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.122 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.122 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.122 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.122 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.122 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.122 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.123 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.123 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.123 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.123 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.123 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.123 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.123 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.124 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.124 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.124 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.124 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.124 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.124 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.124 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.124 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.125 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.125 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.125 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.125 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.125 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.125 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.125 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.125 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.125 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.126 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.126 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.126 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.126 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.126 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.126 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.126 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.126 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.127 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.127 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.127 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.127 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.127 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.127 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.127 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.127 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.128 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.128 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.128 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.128 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.128 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.128 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.128 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirroring", 2024-04-02T14:04:02.128 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.128 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.129 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.129 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.129 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.129 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.129 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.129 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.129 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.129 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.130 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.130 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.130 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.130 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.130 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.130 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.130 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.130 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.131 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.131 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.131 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.131 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.131 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.131 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.131 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.131 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.132 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.132 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.132 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.132 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.132 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.132 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.132 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.132 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.133 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.133 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.133 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.133 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.133 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.133 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.133 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.133 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.134 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.134 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.134 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.134 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.134 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.134 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.134 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.134 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.135 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.135 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.135 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.135 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.135 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.135 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.135 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.135 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.135 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.136 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.136 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.136 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.136 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.136 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.136 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.136 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.136 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.137 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.137 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.137 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.137 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.137 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.137 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.137 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.137 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.138 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.138 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "nfs", 2024-04-02T14:04:02.138 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.138 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.138 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.138 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.138 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.138 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.138 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.139 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.139 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.139 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.139 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.139 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.139 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.139 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.139 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.139 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.140 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.140 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.140 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.140 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.140 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.140 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.140 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.140 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.140 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.141 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.141 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.141 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.141 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.141 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.141 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.141 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.141 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.142 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.142 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.142 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.142 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.142 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.142 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.142 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.142 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.142 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.143 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.143 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.143 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.143 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.143 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.143 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.143 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.143 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.144 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.144 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.144 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.144 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.144 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.144 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.144 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.144 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.145 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.145 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.145 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.145 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.145 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.145 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.145 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.145 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.145 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.146 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.146 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.146 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.146 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.146 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.146 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.146 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.146 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.147 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.147 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2024-04-02T14:04:02.147 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.147 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.147 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.147 INFO:teuthology.orchestra.run.smithi052.stdout: "fail_fs": { 2024-04-02T14:04:02.147 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "fail_fs", 2024-04-02T14:04:02.147 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.147 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.148 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.148 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.148 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.148 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.148 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.148 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-02T14:04:02.148 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.148 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.148 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.149 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.149 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.149 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.149 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.149 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.149 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.149 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.149 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.149 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.150 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.150 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.150 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.150 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.150 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.150 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.150 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.150 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.151 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.151 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.151 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.151 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.151 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.151 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.151 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.151 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.152 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.152 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.152 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.152 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.152 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.152 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.152 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.152 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.152 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.153 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.153 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.153 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.153 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.153 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.153 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.153 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.153 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.154 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.154 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.154 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.154 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.154 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.154 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.154 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.155 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.155 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.155 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.155 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.155 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.155 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.155 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.155 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.156 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.156 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.156 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.156 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.156 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.156 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.156 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.156 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.157 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.157 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.157 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.157 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.157 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.157 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.157 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator": { 2024-04-02T14:04:02.157 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2024-04-02T14:04:02.158 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.158 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.158 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.158 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.158 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.158 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.158 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.158 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm", 2024-04-02T14:04:02.159 INFO:teuthology.orchestra.run.smithi052.stdout: "rook", 2024-04-02T14:04:02.159 INFO:teuthology.orchestra.run.smithi052.stdout: "test_orchestrator" 2024-04-02T14:04:02.159 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.159 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Orchestrator backend", 2024-04-02T14:04:02.159 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.159 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.159 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.159 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.160 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.160 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.160 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.160 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_perf_query", 2024-04-02T14:04:02.160 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.160 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.160 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.160 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.160 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.161 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.161 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.161 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.161 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.161 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.161 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.161 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.161 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.161 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.162 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.162 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.162 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.162 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.162 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.162 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.162 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.162 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.163 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.163 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.163 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.163 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.163 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.163 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.163 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.163 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.163 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.164 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.164 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.164 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.164 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.164 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.164 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.164 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.164 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.164 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.165 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.165 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.165 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.165 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.165 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.165 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.165 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.165 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.165 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.166 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.166 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.166 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.166 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.166 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.166 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.166 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.166 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.166 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.167 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.167 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.167 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.167 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.167 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.167 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.167 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.167 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.167 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.168 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.168 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.168 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.168 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.168 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.168 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.168 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.168 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.168 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.169 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_support", 2024-04-02T14:04:02.169 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.169 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.169 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.169 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.169 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.169 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.169 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.170 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.170 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.170 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.170 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.170 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.170 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.170 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.170 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.170 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.171 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.171 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.171 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.171 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.171 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.171 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.171 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.171 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.171 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.172 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.172 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.172 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.172 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.172 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.172 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.172 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.172 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.172 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.173 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.173 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.173 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.173 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.173 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.173 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.173 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.173 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.173 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.174 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.174 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.174 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.174 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.174 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.174 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.174 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.174 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.175 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.175 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.175 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.175 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.175 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.175 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.175 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.175 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.176 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.176 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.176 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.176 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.176 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.176 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.176 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.176 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.176 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.176 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.177 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.177 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.177 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.177 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.177 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.177 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.177 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.177 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_autoscaler", 2024-04-02T14:04:02.177 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.178 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.178 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.178 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.178 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.178 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.178 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.178 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.178 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.179 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.179 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.179 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.179 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.179 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.179 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.179 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.179 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.180 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.180 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.180 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.180 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.180 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.180 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.180 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.180 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.180 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.181 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.181 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.181 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.181 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.181 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.181 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.181 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.181 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.181 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.182 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.182 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.182 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.182 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.182 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.182 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.182 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.182 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.182 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.183 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.183 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.183 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.183 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.183 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.183 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.183 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.183 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.183 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.184 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.184 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.184 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.184 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.184 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.184 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.184 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.184 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.185 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.185 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.185 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.185 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.185 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.185 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.185 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.185 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.185 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.186 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.186 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.186 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.186 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-04-02T14:04:02.186 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-04-02T14:04:02.186 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:02.186 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.186 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.187 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-04-02T14:04:02.187 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.187 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.187 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.187 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.187 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.187 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.187 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.188 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.188 INFO:teuthology.orchestra.run.smithi052.stdout: "threshold": { 2024-04-02T14:04:02.188 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threshold", 2024-04-02T14:04:02.188 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-04-02T14:04:02.188 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.188 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.188 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2024-04-02T14:04:02.188 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1.0", 2024-04-02T14:04:02.188 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.189 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.189 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "scaling threshold", 2024-04-02T14:04:02.189 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-02T14:04:02.189 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.189 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.189 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.189 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.189 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.190 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.190 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "progress", 2024-04-02T14:04:02.190 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.190 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.190 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.190 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_pg_recovery_event": { 2024-04-02T14:04:02.190 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_pg_recovery_event", 2024-04-02T14:04:02.191 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.191 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.191 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.191 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.191 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.191 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.191 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.191 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-02T14:04:02.192 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.192 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.192 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.192 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.192 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2024-04-02T14:04:02.192 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2024-04-02T14:04:02.192 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.192 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.192 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.193 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:02.193 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.193 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.193 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.193 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.193 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.193 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.193 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.194 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.194 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.194 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.194 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.194 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.194 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.194 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.194 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.195 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.195 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.195 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.195 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.195 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.195 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.195 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.195 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.196 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.196 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.196 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.196 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.196 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.196 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.196 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.196 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.196 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.197 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.197 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.197 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.197 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.197 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.197 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.197 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.197 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.198 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.198 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.198 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.198 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.198 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.198 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.198 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.198 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.199 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.199 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.199 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.199 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.199 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.199 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.199 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.199 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.200 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.200 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.200 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.200 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.200 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.200 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.200 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.200 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.201 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.201 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.201 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.201 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.201 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.201 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.201 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.201 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.202 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.202 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.202 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.202 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.202 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.202 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.202 INFO:teuthology.orchestra.run.smithi052.stdout: "max_completed_events": { 2024-04-02T14:04:02.202 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_completed_events", 2024-04-02T14:04:02.203 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:02.203 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.203 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.203 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "50", 2024-04-02T14:04:02.203 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.203 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.203 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.204 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "number of past completed events to remember", 2024-04-02T14:04:02.204 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.204 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.204 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.204 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.204 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-04-02T14:04:02.204 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-04-02T14:04:02.204 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:02.205 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.205 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.205 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-04-02T14:04:02.205 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.205 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.205 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.205 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long the module is going to sleep", 2024-04-02T14:04:02.205 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.206 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.206 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.206 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.206 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.206 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.206 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.206 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus", 2024-04-02T14:04:02.206 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.207 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.207 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.207 INFO:teuthology.orchestra.run.smithi052.stdout: "cache": { 2024-04-02T14:04:02.207 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cache", 2024-04-02T14:04:02.207 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.207 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.207 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.207 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:02.208 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.208 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.208 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.208 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.208 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.208 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.208 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.208 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.209 INFO:teuthology.orchestra.run.smithi052.stdout: "exclude_perf_counters": { 2024-04-02T14:04:02.209 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "exclude_perf_counters", 2024-04-02T14:04:02.209 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.209 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.209 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.209 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:02.209 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.209 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.210 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.210 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-02T14:04:02.210 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-02T14:04:02.210 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.210 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.210 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.210 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.210 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.211 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.211 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.211 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.211 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.211 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.211 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.211 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.211 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.211 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.211 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.212 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.212 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.212 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.212 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.212 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.212 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.212 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.212 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.212 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.213 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.213 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.213 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.213 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.213 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.213 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.213 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.213 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.213 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.214 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.214 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.214 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.214 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.214 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.214 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.214 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.215 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.215 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.215 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.215 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.215 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.215 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.215 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.215 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.216 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.216 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.216 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.216 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.216 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.216 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.216 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.216 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.217 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.217 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.217 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.217 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.217 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.217 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.217 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.217 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.217 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.218 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.218 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.218 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.218 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.218 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.218 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.218 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.218 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.219 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools": { 2024-04-02T14:04:02.219 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools", 2024-04-02T14:04:02.219 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.219 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.219 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.219 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.219 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.219 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.220 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.220 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.220 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.220 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.220 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.220 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.220 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-02T14:04:02.220 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-02T14:04:02.220 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:02.221 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.221 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.221 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "300", 2024-04-02T14:04:02.221 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.221 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.221 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.221 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.221 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.221 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.222 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.222 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.222 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_interval": { 2024-04-02T14:04:02.222 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_interval", 2024-04-02T14:04:02.222 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-04-02T14:04:02.222 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.222 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.222 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15.0", 2024-04-02T14:04:02.223 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.223 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.223 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.223 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.223 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.223 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.223 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.223 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.224 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-04-02T14:04:02.224 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-04-02T14:04:02.224 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.224 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.224 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.224 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2024-04-02T14:04:02.224 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.224 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.225 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.225 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-02T14:04:02.225 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.225 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.225 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.225 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.225 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-04-02T14:04:02.225 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-04-02T14:04:02.226 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:02.226 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.226 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.226 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "9283", 2024-04-02T14:04:02.226 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.226 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.226 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.226 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-02T14:04:02.227 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.227 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.227 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.227 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.227 INFO:teuthology.orchestra.run.smithi052.stdout: "stale_cache_strategy": { 2024-04-02T14:04:02.227 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stale_cache_strategy", 2024-04-02T14:04:02.227 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.227 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.228 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.228 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "log", 2024-04-02T14:04:02.228 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.228 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.228 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.228 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.228 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.228 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.229 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.229 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.229 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2024-04-02T14:04:02.229 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2024-04-02T14:04:02.229 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.229 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.229 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.229 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "default", 2024-04-02T14:04:02.229 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.230 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.230 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.230 INFO:teuthology.orchestra.run.smithi052.stdout: "default", 2024-04-02T14:04:02.230 INFO:teuthology.orchestra.run.smithi052.stdout: "error" 2024-04-02T14:04:02.230 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.230 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.230 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.231 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.231 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.231 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.231 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2024-04-02T14:04:02.231 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2024-04-02T14:04:02.231 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:02.231 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.231 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.231 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2024-04-02T14:04:02.232 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2024-04-02T14:04:02.232 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2024-04-02T14:04:02.232 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.232 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.232 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.232 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.232 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.232 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.232 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.233 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.233 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.233 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_support", 2024-04-02T14:04:02.233 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.233 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.233 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.233 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.233 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.233 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.234 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.234 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.234 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.234 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.234 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.234 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.234 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.234 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.235 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.235 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.235 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.235 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.235 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.235 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.235 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.235 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.236 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.236 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.236 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.236 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.236 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.236 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.236 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.236 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.236 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.237 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.237 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.237 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.237 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.237 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.237 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.237 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.237 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.237 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.238 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.238 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.238 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.238 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.238 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.238 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.238 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.238 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.239 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.239 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.239 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.239 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.239 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.239 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.239 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.239 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.239 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.240 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.240 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.240 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.240 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.240 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.240 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.240 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.240 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.240 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.240 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.241 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.241 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.241 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.241 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.241 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.241 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.241 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_snap_create": { 2024-04-02T14:04:02.241 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_snap_create", 2024-04-02T14:04:02.241 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:02.242 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.242 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.242 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-04-02T14:04:02.242 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.242 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.242 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.242 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.242 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.242 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.243 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.243 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.243 INFO:teuthology.orchestra.run.smithi052.stdout: "mirror_snapshot_schedule": { 2024-04-02T14:04:02.243 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirror_snapshot_schedule", 2024-04-02T14:04:02.243 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.243 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.243 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.243 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.243 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.244 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.244 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.244 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.244 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.244 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.244 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.244 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.244 INFO:teuthology.orchestra.run.smithi052.stdout: "trash_purge_schedule": { 2024-04-02T14:04:02.244 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "trash_purge_schedule", 2024-04-02T14:04:02.245 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.245 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.245 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.245 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.245 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.245 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.245 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.245 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.245 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.246 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.246 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.246 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.246 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.246 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.246 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.246 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "restful", 2024-04-02T14:04:02.246 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.247 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.247 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.247 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_auth": { 2024-04-02T14:04:02.247 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_auth", 2024-04-02T14:04:02.247 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.247 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.247 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.247 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:02.247 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.247 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.248 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.248 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.248 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.248 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.248 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.248 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.248 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2024-04-02T14:04:02.248 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2024-04-02T14:04:02.248 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.249 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.249 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.249 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.249 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.249 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.249 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.249 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.249 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.249 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.250 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.250 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.250 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.250 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.250 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.250 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.250 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.250 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.251 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.251 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.251 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.251 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.251 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.251 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.251 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.251 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.251 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.251 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.252 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.252 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.252 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.252 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.252 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.252 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.252 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.252 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.252 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.253 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.253 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.253 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.253 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.253 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.253 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.253 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.253 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.253 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.254 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.254 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.254 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.254 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.254 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.254 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.254 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.254 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.254 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.255 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.255 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.255 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.255 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.255 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.255 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.255 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.255 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.255 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.256 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.256 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.256 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.256 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.256 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.256 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.256 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.256 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.256 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.256 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.257 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.257 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.257 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.257 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.257 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.257 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.257 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.257 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.257 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-04-02T14:04:02.258 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-04-02T14:04:02.258 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.258 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.258 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.258 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.258 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.258 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.259 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.259 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.259 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.259 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.259 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.259 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.259 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-04-02T14:04:02.259 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-04-02T14:04:02.259 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.260 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.260 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.260 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.260 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.260 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.260 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.260 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.260 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.260 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.260 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.261 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.261 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.261 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.261 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.261 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rgw", 2024-04-02T14:04:02.261 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.261 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.261 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.261 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.262 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.262 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.262 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.262 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.262 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.262 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.262 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.262 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.263 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.263 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.263 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.263 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.263 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.263 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.263 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.263 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.263 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.264 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.264 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.264 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.264 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.264 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.264 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.264 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.264 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.264 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.265 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.265 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.265 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.265 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.265 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.265 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.265 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.265 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.265 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.265 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.266 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.266 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.266 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.266 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.266 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.266 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.266 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.266 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.266 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.267 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.267 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.267 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.267 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.267 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.267 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.267 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.267 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.267 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.268 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.268 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.268 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.268 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.268 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.268 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.268 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.268 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.268 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.269 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.269 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.269 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.269 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.269 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.269 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.269 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.269 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.269 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.270 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rook", 2024-04-02T14:04:02.270 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.270 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.270 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.270 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.270 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.270 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.270 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.270 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.271 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.271 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.271 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.271 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.271 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.271 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.271 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.271 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.271 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.272 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.272 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.272 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.272 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.272 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.272 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.272 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.272 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.272 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.272 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.273 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.273 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.273 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.273 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.273 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.273 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.273 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.273 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.274 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.274 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.274 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.274 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.274 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.274 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.274 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.274 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.275 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.275 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.275 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.275 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.275 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.275 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.275 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.275 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.275 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.276 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.276 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.276 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.276 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.276 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.276 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.276 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.276 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.276 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.276 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.277 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.277 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.277 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.277 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.277 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.277 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.277 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.277 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.277 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.278 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.278 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.278 INFO:teuthology.orchestra.run.smithi052.stdout: "storage_class": { 2024-04-02T14:04:02.278 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "storage_class", 2024-04-02T14:04:02.278 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.278 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.278 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.279 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "local", 2024-04-02T14:04:02.279 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.279 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.279 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.279 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-02T14:04:02.279 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.279 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.279 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.279 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.280 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.280 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.280 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.280 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "selftest", 2024-04-02T14:04:02.280 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.280 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.280 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.280 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.280 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.281 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.281 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.281 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.281 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.281 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.281 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.281 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.281 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.281 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.282 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.282 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.282 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.282 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.282 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.282 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.282 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.282 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.283 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.283 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.283 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.283 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.283 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.283 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.283 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.283 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.283 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.284 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.284 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.284 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.284 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.284 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.284 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.284 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.284 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.285 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.285 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.285 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.285 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.285 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.285 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.285 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.285 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.285 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.286 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.286 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.286 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.286 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.286 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.286 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.286 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.287 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.287 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.287 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.287 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.287 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.287 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.287 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.287 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.287 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.288 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.288 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.288 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.288 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.288 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.288 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.288 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.288 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.289 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.289 INFO:teuthology.orchestra.run.smithi052.stdout: "roption1": { 2024-04-02T14:04:02.289 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption1", 2024-04-02T14:04:02.289 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.289 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.289 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.289 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.289 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.289 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.290 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.290 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.290 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.290 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.290 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.290 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.290 INFO:teuthology.orchestra.run.smithi052.stdout: "roption2": { 2024-04-02T14:04:02.291 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption2", 2024-04-02T14:04:02.291 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.291 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.291 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.291 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "xyz", 2024-04-02T14:04:02.291 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.291 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.291 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.291 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.292 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.292 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.292 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.292 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.292 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption1": { 2024-04-02T14:04:02.292 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption1", 2024-04-02T14:04:02.292 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.292 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.293 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.293 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.293 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.293 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.293 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.293 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.293 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.293 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.294 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.294 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.294 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption2": { 2024-04-02T14:04:02.294 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption2", 2024-04-02T14:04:02.294 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:02.294 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.294 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.294 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.295 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.295 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.295 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.295 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.295 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.295 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.295 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.295 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.295 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption3": { 2024-04-02T14:04:02.295 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption3", 2024-04-02T14:04:02.296 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-04-02T14:04:02.296 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.296 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.296 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.296 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.296 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.296 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.296 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.296 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.297 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.297 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.297 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.297 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption4": { 2024-04-02T14:04:02.297 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption4", 2024-04-02T14:04:02.297 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.297 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.297 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.297 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.297 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.298 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.298 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.298 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.298 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.298 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.298 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.298 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.298 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption5": { 2024-04-02T14:04:02.299 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption5", 2024-04-02T14:04:02.299 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.299 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.299 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.299 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.299 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.299 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.299 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.299 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.300 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.300 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.300 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.300 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.300 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption6": { 2024-04-02T14:04:02.300 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption6", 2024-04-02T14:04:02.300 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.300 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.301 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.301 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:02.301 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.301 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.301 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.301 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.301 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.301 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.302 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.302 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.302 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption7": { 2024-04-02T14:04:02.302 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption7", 2024-04-02T14:04:02.302 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:02.302 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.302 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.302 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.303 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-04-02T14:04:02.303 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "42", 2024-04-02T14:04:02.303 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.303 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.303 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.303 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.303 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.303 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.304 INFO:teuthology.orchestra.run.smithi052.stdout: "testkey": { 2024-04-02T14:04:02.304 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testkey", 2024-04-02T14:04:02.304 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.304 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.304 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.304 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.304 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.304 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.305 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.305 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.305 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.305 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.305 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.305 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.305 INFO:teuthology.orchestra.run.smithi052.stdout: "testlkey": { 2024-04-02T14:04:02.305 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testlkey", 2024-04-02T14:04:02.306 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.306 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.306 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.306 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.306 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.306 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.306 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.306 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.307 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.307 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.307 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.307 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.307 INFO:teuthology.orchestra.run.smithi052.stdout: "testnewline": { 2024-04-02T14:04:02.307 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testnewline", 2024-04-02T14:04:02.307 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.307 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.307 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.308 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.308 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.308 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.308 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.308 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.308 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.308 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.309 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.309 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.309 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.309 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.309 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.309 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snap_schedule", 2024-04-02T14:04:02.309 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.309 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.309 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.310 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_m_granularity": { 2024-04-02T14:04:02.310 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_m_granularity", 2024-04-02T14:04:02.310 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.310 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.310 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.310 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.310 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.311 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.311 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.311 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow minute scheduled snapshots", 2024-04-02T14:04:02.311 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.311 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.311 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.311 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.311 INFO:teuthology.orchestra.run.smithi052.stdout: "dump_on_update": { 2024-04-02T14:04:02.311 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dump_on_update", 2024-04-02T14:04:02.312 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.312 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.312 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.312 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.312 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.312 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.312 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.313 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "dump database to debug log on update", 2024-04-02T14:04:02.313 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.313 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.313 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.313 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.313 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.313 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.313 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.314 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.314 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.314 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.314 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.314 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.314 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.314 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.314 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.315 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.315 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.315 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.315 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.315 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.315 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.315 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.315 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.315 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.316 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.316 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.316 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.316 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.316 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.316 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.316 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.316 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.316 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.317 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.317 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.317 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.317 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.317 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.317 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.317 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.317 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.317 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.318 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.318 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.318 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.318 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.318 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.318 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.318 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.318 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.319 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.319 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.319 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.319 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.319 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.319 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.319 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.319 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.319 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.320 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.320 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.320 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.320 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.320 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.320 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.320 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.320 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.320 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.321 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.321 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.321 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.321 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.321 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.321 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.321 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.321 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.321 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.321 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stats", 2024-04-02T14:04:02.322 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.322 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.322 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.322 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.322 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.322 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.322 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.322 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.323 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.323 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.323 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.323 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.323 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.323 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.323 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.323 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.323 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.324 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.324 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.324 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.324 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.324 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.324 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.324 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.324 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.325 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.325 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.325 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.325 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.325 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.325 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.325 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.325 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.325 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.326 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.326 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.326 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.326 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.326 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.326 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.326 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.326 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.327 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.327 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.327 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.327 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.327 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.327 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.327 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.327 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.327 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.328 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.328 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.328 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.328 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.328 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.328 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.328 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.328 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.329 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.329 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.329 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.329 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.329 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.329 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.329 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.329 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.329 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.330 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.330 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.330 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.330 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.330 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.330 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.330 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.330 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.331 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "status", 2024-04-02T14:04:02.331 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.331 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.331 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.331 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.331 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.331 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.331 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.331 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.332 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.332 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.332 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.332 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.332 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.332 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.332 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.332 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.333 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.333 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.333 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.333 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.333 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.333 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.333 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.333 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.334 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.334 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.334 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.334 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.334 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.334 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.334 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.334 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.335 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.335 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.335 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.335 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.335 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.335 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.335 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.335 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.335 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.336 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.336 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.336 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.336 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.336 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.336 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.336 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.336 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.337 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.337 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.337 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.337 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.337 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.337 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.337 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.337 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.337 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.338 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.338 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.338 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.338 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.338 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.338 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.338 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.339 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.339 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.339 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.339 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.339 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.339 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.339 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.339 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.339 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.340 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.340 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.340 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telegraf", 2024-04-02T14:04:02.340 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.340 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.340 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.340 INFO:teuthology.orchestra.run.smithi052.stdout: "address": { 2024-04-02T14:04:02.340 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "address", 2024-04-02T14:04:02.340 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.341 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.341 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.341 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-02T14:04:02.341 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.341 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.341 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.341 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.341 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.341 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.342 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.342 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.342 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-04-02T14:04:02.342 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-04-02T14:04:02.342 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:02.342 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.342 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.343 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15", 2024-04-02T14:04:02.343 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.343 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.343 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.343 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.343 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.343 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.343 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.344 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.344 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.344 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.344 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.344 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.344 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.344 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.344 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.344 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.345 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.345 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.345 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.345 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.345 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.345 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.345 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.345 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.345 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.346 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.346 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.346 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.346 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.346 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.346 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.346 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.346 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.347 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.347 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.347 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.347 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.347 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.347 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.347 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.347 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.347 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.348 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.348 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.348 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.348 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.348 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.348 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.348 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.348 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.348 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.349 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.349 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.349 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.349 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.349 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.349 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.349 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.349 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.349 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.350 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.350 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.350 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.350 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.350 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.350 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.350 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.351 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.351 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.351 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.351 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.351 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.351 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.351 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.351 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.351 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.351 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.352 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.352 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.352 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.352 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.352 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telemetry", 2024-04-02T14:04:02.352 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.352 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.352 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.353 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_basic": { 2024-04-02T14:04:02.353 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_basic", 2024-04-02T14:04:02.353 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.353 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.353 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.353 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:02.353 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.353 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.354 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.354 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-02T14:04:02.354 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.354 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.354 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.354 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.354 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_crash": { 2024-04-02T14:04:02.354 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_crash", 2024-04-02T14:04:02.355 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.355 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.355 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.355 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:02.355 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.355 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.355 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.355 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-02T14:04:02.355 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.355 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.356 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.356 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.356 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_device": { 2024-04-02T14:04:02.356 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_device", 2024-04-02T14:04:02.356 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.356 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.356 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.356 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:02.356 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.357 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.357 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.357 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-02T14:04:02.357 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.357 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.357 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.357 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.357 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_ident": { 2024-04-02T14:04:02.357 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_ident", 2024-04-02T14:04:02.358 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.358 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.358 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.358 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.358 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.358 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.358 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.358 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-02T14:04:02.359 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.359 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.359 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.359 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.359 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_perf": { 2024-04-02T14:04:02.359 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_perf", 2024-04-02T14:04:02.359 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.359 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.359 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.359 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.360 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.360 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.360 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.360 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-02T14:04:02.360 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.360 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.360 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.360 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.361 INFO:teuthology.orchestra.run.smithi052.stdout: "contact": { 2024-04-02T14:04:02.361 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "contact", 2024-04-02T14:04:02.361 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.361 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.361 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.361 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.361 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.361 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.361 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.361 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.362 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.362 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.362 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.362 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.362 INFO:teuthology.orchestra.run.smithi052.stdout: "description": { 2024-04-02T14:04:02.362 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "description", 2024-04-02T14:04:02.362 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.362 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.363 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.363 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.363 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.363 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.363 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.363 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.363 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.363 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.363 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.364 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.364 INFO:teuthology.orchestra.run.smithi052.stdout: "device_url": { 2024-04-02T14:04:02.364 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_url", 2024-04-02T14:04:02.364 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.364 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.364 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.364 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-02T14:04:02.364 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.365 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.365 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.365 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.365 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.365 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.365 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.365 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.365 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2024-04-02T14:04:02.366 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2024-04-02T14:04:02.366 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.366 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.366 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.366 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.366 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.366 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.366 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.366 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.367 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.367 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.367 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.367 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.367 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-04-02T14:04:02.367 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-04-02T14:04:02.367 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:02.367 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.367 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.367 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "24", 2024-04-02T14:04:02.368 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "8", 2024-04-02T14:04:02.368 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.368 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.368 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.368 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.368 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.368 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.368 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.368 INFO:teuthology.orchestra.run.smithi052.stdout: "last_opt_revision": { 2024-04-02T14:04:02.369 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "last_opt_revision", 2024-04-02T14:04:02.369 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:02.369 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.369 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.369 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1", 2024-04-02T14:04:02.369 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.369 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.369 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.369 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.370 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.370 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.370 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.370 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.370 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard": { 2024-04-02T14:04:02.370 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard", 2024-04-02T14:04:02.370 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.370 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.371 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.371 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.371 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.371 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.371 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.371 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.371 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.371 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.371 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.371 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.372 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard_description": { 2024-04-02T14:04:02.372 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard_description", 2024-04-02T14:04:02.372 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.372 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.372 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.372 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.372 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.372 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.373 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.373 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.373 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.373 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.373 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.373 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.373 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.373 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.373 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.374 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.374 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.374 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.374 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.374 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.374 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.374 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.374 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.375 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.375 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.375 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.375 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.375 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.375 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.375 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.375 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.375 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.376 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.376 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.376 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.376 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.376 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.376 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.376 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.376 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.376 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.377 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.377 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.377 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.377 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.377 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.377 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.377 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.377 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.377 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.378 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.378 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.378 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.378 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.378 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.378 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.378 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.378 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.378 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.379 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.379 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.379 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.379 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.379 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.379 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.379 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.379 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.379 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.380 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.380 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.380 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.380 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.380 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.380 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.380 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.380 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.381 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.381 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.381 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.381 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.381 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.381 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.381 INFO:teuthology.orchestra.run.smithi052.stdout: "organization": { 2024-04-02T14:04:02.381 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "organization", 2024-04-02T14:04:02.382 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.382 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.382 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.382 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.382 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.382 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.382 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.382 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.382 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.383 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.383 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.383 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.383 INFO:teuthology.orchestra.run.smithi052.stdout: "proxy": { 2024-04-02T14:04:02.383 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "proxy", 2024-04-02T14:04:02.383 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.383 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.383 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.383 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.384 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.384 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.384 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.384 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.384 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.384 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.384 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.384 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.385 INFO:teuthology.orchestra.run.smithi052.stdout: "url": { 2024-04-02T14:04:02.385 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url", 2024-04-02T14:04:02.385 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.385 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.385 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.385 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-02T14:04:02.385 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.385 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.386 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.386 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.386 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.386 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.386 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.386 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.386 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.386 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.386 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.387 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "test_orchestrator", 2024-04-02T14:04:02.387 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.387 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.387 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.387 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.387 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.387 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.387 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.387 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.388 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.388 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.388 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.388 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.388 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.388 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.388 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.388 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.389 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.389 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.389 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.389 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.389 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.389 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.389 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.389 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.389 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.390 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.390 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.390 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.390 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.390 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.390 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.390 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.390 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.391 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.391 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.391 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.391 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.391 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.391 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.391 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.391 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.391 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.392 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.392 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.392 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.392 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.392 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.392 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.392 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.392 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.392 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.393 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.393 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.393 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.393 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.393 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.393 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.393 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.393 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.393 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.394 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.394 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.394 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.394 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.394 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.394 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.394 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.394 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.395 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.395 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.395 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.395 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.395 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.395 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.395 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.395 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.396 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "volumes", 2024-04-02T14:04:02.396 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.396 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.396 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.396 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.396 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.396 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.396 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.397 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.397 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.397 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.397 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.397 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.397 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.397 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.397 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.397 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.398 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.398 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.398 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.398 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.398 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.398 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.398 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.398 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.398 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.399 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.399 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.399 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.399 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.399 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.399 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.399 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.399 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.399 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.400 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.400 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.400 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.400 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.400 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.400 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.400 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.400 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.400 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.400 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.401 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.401 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.401 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.401 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.401 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.401 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.401 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.401 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.402 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.402 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.402 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.402 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.402 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.402 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.402 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.402 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.402 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.403 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.403 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.403 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.403 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.403 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.403 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.403 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.403 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.403 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.404 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.404 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.404 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.404 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_clones": { 2024-04-02T14:04:02.404 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_clones", 2024-04-02T14:04:02.404 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:02.404 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.404 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.405 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4", 2024-04-02T14:04:02.405 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.405 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.405 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.405 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-02T14:04:02.405 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.405 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.405 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.406 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.406 INFO:teuthology.orchestra.run.smithi052.stdout: "periodic_async_work": { 2024-04-02T14:04:02.406 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "periodic_async_work", 2024-04-02T14:04:02.406 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.406 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.406 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.406 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.406 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.407 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.407 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.407 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Periodically check for async work", 2024-04-02T14:04:02.407 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.407 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.407 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.407 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.407 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_delay": { 2024-04-02T14:04:02.407 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_delay", 2024-04-02T14:04:02.408 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:02.408 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.408 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.408 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-04-02T14:04:02.408 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.408 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.408 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.408 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-02T14:04:02.408 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.409 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.409 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.409 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.409 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_no_wait": { 2024-04-02T14:04:02.409 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_no_wait", 2024-04-02T14:04:02.409 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.409 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.409 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.409 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-04-02T14:04:02.410 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.410 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.410 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.410 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-02T14:04:02.410 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.410 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.410 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.410 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.410 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.411 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.411 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.411 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix", 2024-04-02T14:04:02.411 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-04-02T14:04:02.411 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-04-02T14:04:02.411 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-04-02T14:04:02.411 INFO:teuthology.orchestra.run.smithi052.stdout: "discovery_interval": { 2024-04-02T14:04:02.411 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "discovery_interval", 2024-04-02T14:04:02.411 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-04-02T14:04:02.412 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.412 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.412 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "100", 2024-04-02T14:04:02.412 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.412 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.412 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.412 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.412 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.412 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.413 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.413 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.413 INFO:teuthology.orchestra.run.smithi052.stdout: "identifier": { 2024-04-02T14:04:02.413 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "identifier", 2024-04-02T14:04:02.413 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.413 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.413 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.413 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.414 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.414 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.414 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.414 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.414 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.414 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.414 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.415 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.415 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-04-02T14:04:02.415 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-04-02T14:04:02.415 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-04-02T14:04:02.415 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.415 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.415 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-04-02T14:04:02.415 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.416 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.416 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.416 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.416 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.416 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.416 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.416 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.416 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-04-02T14:04:02.417 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-04-02T14:04:02.417 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.417 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.417 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.417 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.417 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.417 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.418 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.418 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.418 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.418 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.418 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.418 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.418 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.418 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.419 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.419 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.419 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.419 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.419 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.419 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-04-02T14:04:02.419 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-04-02T14:04:02.419 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.420 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.420 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.420 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.420 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.420 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.420 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.420 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.420 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.421 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.421 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.421 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.421 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-04-02T14:04:02.421 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-04-02T14:04:02.421 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.421 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.422 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.422 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-04-02T14:04:02.422 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.422 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.422 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-04-02T14:04:02.422 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-04-02T14:04:02.422 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-04-02T14:04:02.422 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-04-02T14:04:02.423 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-04-02T14:04:02.423 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-04-02T14:04:02.423 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-04-02T14:04:02.423 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.423 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.423 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.423 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.423 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.424 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.424 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-04-02T14:04:02.424 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-04-02T14:04:02.424 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-04-02T14:04:02.424 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.424 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-04-02T14:04:02.424 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-04-02T14:04:02.424 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.425 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.425 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.425 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.425 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.425 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.425 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.425 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.425 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_host": { 2024-04-02T14:04:02.426 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_host", 2024-04-02T14:04:02.426 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.426 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.426 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.426 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-04-02T14:04:02.426 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.426 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.426 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.427 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.427 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.427 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.427 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.427 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.427 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_port": { 2024-04-02T14:04:02.427 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_port", 2024-04-02T14:04:02.427 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-04-02T14:04:02.427 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.427 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.428 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10051", 2024-04-02T14:04:02.428 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.428 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.428 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.428 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.428 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.428 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.428 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.428 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.429 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_sender": { 2024-04-02T14:04:02.429 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_sender", 2024-04-02T14:04:02.429 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-04-02T14:04:02.429 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-04-02T14:04:02.429 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-04-02T14:04:02.429 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-02T14:04:02.429 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-04-02T14:04:02.429 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-04-02T14:04:02.429 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-04-02T14:04:02.430 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-04-02T14:04:02.430 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-04-02T14:04:02.430 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-04-02T14:04:02.430 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-04-02T14:04:02.430 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.430 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.430 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.430 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.431 INFO:teuthology.orchestra.run.smithi052.stdout: "services": { 2024-04-02T14:04:02.431 INFO:teuthology.orchestra.run.smithi052.stdout: "dashboard": "https://172.21.15.52:8443/" 2024-04-02T14:04:02.431 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.431 INFO:teuthology.orchestra.run.smithi052.stdout: "always_on_modules": { 2024-04-02T14:04:02.431 INFO:teuthology.orchestra.run.smithi052.stdout: "octopus": [ 2024-04-02T14:04:02.431 INFO:teuthology.orchestra.run.smithi052.stdout: "balancer", 2024-04-02T14:04:02.431 INFO:teuthology.orchestra.run.smithi052.stdout: "crash", 2024-04-02T14:04:02.431 INFO:teuthology.orchestra.run.smithi052.stdout: "devicehealth", 2024-04-02T14:04:02.431 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator", 2024-04-02T14:04:02.431 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_autoscaler", 2024-04-02T14:04:02.432 INFO:teuthology.orchestra.run.smithi052.stdout: "progress", 2024-04-02T14:04:02.432 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_support", 2024-04-02T14:04:02.432 INFO:teuthology.orchestra.run.smithi052.stdout: "status", 2024-04-02T14:04:02.432 INFO:teuthology.orchestra.run.smithi052.stdout: "telemetry", 2024-04-02T14:04:02.432 INFO:teuthology.orchestra.run.smithi052.stdout: "volumes" 2024-04-02T14:04:02.432 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.432 INFO:teuthology.orchestra.run.smithi052.stdout: "pacific": [ 2024-04-02T14:04:02.432 INFO:teuthology.orchestra.run.smithi052.stdout: "balancer", 2024-04-02T14:04:02.433 INFO:teuthology.orchestra.run.smithi052.stdout: "crash", 2024-04-02T14:04:02.433 INFO:teuthology.orchestra.run.smithi052.stdout: "devicehealth", 2024-04-02T14:04:02.433 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator", 2024-04-02T14:04:02.433 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_autoscaler", 2024-04-02T14:04:02.433 INFO:teuthology.orchestra.run.smithi052.stdout: "progress", 2024-04-02T14:04:02.433 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_support", 2024-04-02T14:04:02.433 INFO:teuthology.orchestra.run.smithi052.stdout: "status", 2024-04-02T14:04:02.433 INFO:teuthology.orchestra.run.smithi052.stdout: "telemetry", 2024-04-02T14:04:02.433 INFO:teuthology.orchestra.run.smithi052.stdout: "volumes" 2024-04-02T14:04:02.434 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.434 INFO:teuthology.orchestra.run.smithi052.stdout: "quincy": [ 2024-04-02T14:04:02.434 INFO:teuthology.orchestra.run.smithi052.stdout: "balancer", 2024-04-02T14:04:02.434 INFO:teuthology.orchestra.run.smithi052.stdout: "crash", 2024-04-02T14:04:02.434 INFO:teuthology.orchestra.run.smithi052.stdout: "devicehealth", 2024-04-02T14:04:02.434 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator", 2024-04-02T14:04:02.434 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_autoscaler", 2024-04-02T14:04:02.434 INFO:teuthology.orchestra.run.smithi052.stdout: "progress", 2024-04-02T14:04:02.435 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_support", 2024-04-02T14:04:02.435 INFO:teuthology.orchestra.run.smithi052.stdout: "status", 2024-04-02T14:04:02.435 INFO:teuthology.orchestra.run.smithi052.stdout: "telemetry", 2024-04-02T14:04:02.435 INFO:teuthology.orchestra.run.smithi052.stdout: "volumes" 2024-04-02T14:04:02.435 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.435 INFO:teuthology.orchestra.run.smithi052.stdout: "reef": [ 2024-04-02T14:04:02.435 INFO:teuthology.orchestra.run.smithi052.stdout: "balancer", 2024-04-02T14:04:02.435 INFO:teuthology.orchestra.run.smithi052.stdout: "crash", 2024-04-02T14:04:02.436 INFO:teuthology.orchestra.run.smithi052.stdout: "devicehealth", 2024-04-02T14:04:02.436 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator", 2024-04-02T14:04:02.436 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_autoscaler", 2024-04-02T14:04:02.436 INFO:teuthology.orchestra.run.smithi052.stdout: "progress", 2024-04-02T14:04:02.436 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_support", 2024-04-02T14:04:02.436 INFO:teuthology.orchestra.run.smithi052.stdout: "status", 2024-04-02T14:04:02.436 INFO:teuthology.orchestra.run.smithi052.stdout: "telemetry", 2024-04-02T14:04:02.436 INFO:teuthology.orchestra.run.smithi052.stdout: "volumes" 2024-04-02T14:04:02.436 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-04-02T14:04:02.437 INFO:teuthology.orchestra.run.smithi052.stdout: "squid": [ 2024-04-02T14:04:02.437 INFO:teuthology.orchestra.run.smithi052.stdout: "balancer", 2024-04-02T14:04:02.437 INFO:teuthology.orchestra.run.smithi052.stdout: "crash", 2024-04-02T14:04:02.437 INFO:teuthology.orchestra.run.smithi052.stdout: "devicehealth", 2024-04-02T14:04:02.437 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator", 2024-04-02T14:04:02.437 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_autoscaler", 2024-04-02T14:04:02.437 INFO:teuthology.orchestra.run.smithi052.stdout: "progress", 2024-04-02T14:04:02.437 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_support", 2024-04-02T14:04:02.438 INFO:teuthology.orchestra.run.smithi052.stdout: "status", 2024-04-02T14:04:02.438 INFO:teuthology.orchestra.run.smithi052.stdout: "telemetry", 2024-04-02T14:04:02.438 INFO:teuthology.orchestra.run.smithi052.stdout: "volumes" 2024-04-02T14:04:02.438 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-04-02T14:04:02.438 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.438 INFO:teuthology.orchestra.run.smithi052.stdout: "last_failure_osd_epoch": 93, 2024-04-02T14:04:02.438 INFO:teuthology.orchestra.run.smithi052.stdout: "active_clients": [ 2024-04-02T14:04:02.438 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.438 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "devicehealth", 2024-04-02T14:04:02.439 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2024-04-02T14:04:02.439 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.439 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2024-04-02T14:04:02.439 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2024-04-02T14:04:02.439 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 346519428 2024-04-02T14:04:02.439 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.439 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-04-02T14:04:02.439 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.440 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.440 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "libcephsqlite", 2024-04-02T14:04:02.440 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2024-04-02T14:04:02.440 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.440 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2024-04-02T14:04:02.440 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2024-04-02T14:04:02.440 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 1256648647 2024-04-02T14:04:02.440 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.441 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-04-02T14:04:02.441 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.441 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.441 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_support", 2024-04-02T14:04:02.441 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2024-04-02T14:04:02.441 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.441 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2024-04-02T14:04:02.441 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2024-04-02T14:04:02.442 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 390814167 2024-04-02T14:04:02.442 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.442 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-04-02T14:04:02.442 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.442 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.442 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snap_schedule", 2024-04-02T14:04:02.442 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2024-04-02T14:04:02.442 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.443 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2024-04-02T14:04:02.443 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2024-04-02T14:04:02.443 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 932564830 2024-04-02T14:04:02.443 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.443 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-04-02T14:04:02.443 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.443 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.443 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snap_schedule", 2024-04-02T14:04:02.444 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2024-04-02T14:04:02.444 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.444 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2024-04-02T14:04:02.444 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2024-04-02T14:04:02.444 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 2729174362 2024-04-02T14:04:02.444 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.444 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-04-02T14:04:02.444 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:02.445 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.445 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "volumes", 2024-04-02T14:04:02.445 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2024-04-02T14:04:02.445 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-04-02T14:04:02.445 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2024-04-02T14:04:02.445 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2024-04-02T14:04:02.445 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 364040453 2024-04-02T14:04:02.446 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.446 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-04-02T14:04:02.446 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:02.446 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-04-02T14:04:02.446 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:04:02.448 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:00 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/4005001766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T14:04:02.448 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:00 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1200357950' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-02T14:04:02.448 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:01 smithi088 ceph-mon[36528]: pgmap v326: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:04:02.448 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:00 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/4005001766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T14:04:02.448 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:00 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1200357950' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-02T14:04:02.449 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:01 smithi189 ceph-mon[36683]: pgmap v326: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:04:02.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:00.080+0000 7f81d6689640 1 -- 172.21.15.52:0/1200357950 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f81b80a3d20 msgr2=0x7f81b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:02.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:00.080+0000 7f81d6689640 1 --2- 172.21.15.52:0/1200357950 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f81b80a3d20 0x7f81b80a61e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f81d00780c0 tx=0x7f81c002cf90 comp rx=0 tx=0).stop 2024-04-02T14:04:02.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:00.080+0000 7f81d6689640 1 -- 172.21.15.52:0/1200357950 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81d006a1b0 msgr2=0x7f81d01984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:02.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:00.080+0000 7f81d6689640 1 --2- 172.21.15.52:0/1200357950 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81d006a1b0 0x7f81d01984a0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f81c400b4a0 tx=0x7f81c400b960 comp rx=0 tx=0).stop 2024-04-02T14:04:02.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:00.080+0000 7f81d6689640 1 -- 172.21.15.52:0/1200357950 shutdown_connections 2024-04-02T14:04:02.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:00.080+0000 7f81d6689640 1 --2- 172.21.15.52:0/1200357950 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f81b80a3d20 0x7f81b80a61e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:02.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:00.080+0000 7f81d6689640 1 --2- 172.21.15.52:0/1200357950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81d0107b90 0x7f81d019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:02.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:00.080+0000 7f81d6689640 1 --2- 172.21.15.52:0/1200357950 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d006ab50 0x7f81d01989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:02.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:00.080+0000 7f81d6689640 1 --2- 172.21.15.52:0/1200357950 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81d006a1b0 0x7f81d01984a0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:02.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:00.080+0000 7f81d6689640 1 -- 172.21.15.52:0/1200357950 >> 172.21.15.52:0/1200357950 conn(0x7f81d0077aa0 msgr2=0x7f81d00ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:02.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:00.080+0000 7f81d6689640 1 -- 172.21.15.52:0/1200357950 shutdown_connections 2024-04-02T14:04:02.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:00.080+0000 7f81d6689640 1 -- 172.21.15.52:0/1200357950 wait complete. 2024-04-02T14:04:02.461 DEBUG:tasks.check_counter:Getting stats from a 2024-04-02T14:04:02.462 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph daemon mds.a perf dump 2024-04-02T14:04:02.463 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:00 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/4005001766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T14:04:02.463 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:00 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1200357950' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-02T14:04:02.463 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:01 smithi052 ceph-mon[32104]: pgmap v326: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-02T14:04:02.718 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:04:03.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:03 smithi189 ceph-mon[36683]: pgmap v327: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:04:03.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:03 smithi052 ceph-mon[32104]: pgmap v327: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:04:03.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:03 smithi088 ceph-mon[36528]: pgmap v327: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:04:04.532 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:04:04.532 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T14:04:04.532 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 1512, 2024-04-02T14:04:04.532 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 1514, 2024-04-02T14:04:04.532 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 453339, 2024-04-02T14:04:04.533 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 480817, 2024-04-02T14:04:04.533 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 6, 2024-04-02T14:04:04.533 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 2, 2024-04-02T14:04:04.533 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.151293729, 2024-04-02T14:04:04.533 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.050021451, 2024-04-02T14:04:04.533 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.039381300, 2024-04-02T14:04:04.533 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.048227233, 2024-04-02T14:04:04.533 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:04.533 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 1512, 2024-04-02T14:04:04.534 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.051315271, 2024-04-02T14:04:04.534 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000033938 2024-04-02T14:04:04.534 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.534 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:04.534 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.534 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.534 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.534 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.535 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T14:04:04.535 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T14:04:04.535 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.535 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T14:04:04.535 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 673, 2024-04-02T14:04:04.535 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 833, 2024-04-02T14:04:04.535 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 166038, 2024-04-02T14:04:04.535 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 657030, 2024-04-02T14:04:04.536 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 7, 2024-04-02T14:04:04.536 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 5, 2024-04-02T14:04:04.536 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.084593834, 2024-04-02T14:04:04.536 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.028891243, 2024-04-02T14:04:04.536 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.018430056, 2024-04-02T14:04:04.536 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.028652152, 2024-04-02T14:04:04.536 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:04.536 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 830, 2024-04-02T14:04:04.537 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.036383280, 2024-04-02T14:04:04.537 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000043835 2024-04-02T14:04:04.537 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.537 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:04.537 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.537 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.537 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.537 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.537 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-02T14:04:04.538 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 443584 2024-04-02T14:04:04.538 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.538 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T14:04:04.538 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 249, 2024-04-02T14:04:04.538 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 233, 2024-04-02T14:04:04.538 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 1639708, 2024-04-02T14:04:04.538 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 134798, 2024-04-02T14:04:04.539 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 3, 2024-04-02T14:04:04.539 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 2, 2024-04-02T14:04:04.539 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.045994573, 2024-04-02T14:04:04.539 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.015630546, 2024-04-02T14:04:04.539 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.035704937, 2024-04-02T14:04:04.539 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.004509349, 2024-04-02T14:04:04.539 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:04.539 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 233, 2024-04-02T14:04:04.539 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.012280744, 2024-04-02T14:04:04.540 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000052707 2024-04-02T14:04:04.540 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.540 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:04.540 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.540 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.540 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.540 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.540 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 1639440, 2024-04-02T14:04:04.541 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 134480 2024-04-02T14:04:04.541 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.541 INFO:teuthology.orchestra.run.smithi052.stdout: "cct": { 2024-04-02T14:04:04.541 INFO:teuthology.orchestra.run.smithi052.stdout: "total_workers": 1, 2024-04-02T14:04:04.541 INFO:teuthology.orchestra.run.smithi052.stdout: "unhealthy_workers": 0 2024-04-02T14:04:04.541 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.541 INFO:teuthology.orchestra.run.smithi052.stdout: "finisher-MDSRank": { 2024-04-02T14:04:04.541 INFO:teuthology.orchestra.run.smithi052.stdout: "queue_len": 0, 2024-04-02T14:04:04.541 INFO:teuthology.orchestra.run.smithi052.stdout: "complete_latency": { 2024-04-02T14:04:04.542 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 3503, 2024-04-02T14:04:04.542 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.108799053, 2024-04-02T14:04:04.542 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000031058 2024-04-02T14:04:04.542 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:04.542 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.542 INFO:teuthology.orchestra.run.smithi052.stdout: "finisher-PurgeQueue": { 2024-04-02T14:04:04.542 INFO:teuthology.orchestra.run.smithi052.stdout: "queue_len": 0, 2024-04-02T14:04:04.543 INFO:teuthology.orchestra.run.smithi052.stdout: "complete_latency": { 2024-04-02T14:04:04.543 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.543 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.543 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.543 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:04.543 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.543 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": { 2024-04-02T14:04:04.543 INFO:teuthology.orchestra.run.smithi052.stdout: "request": 0, 2024-04-02T14:04:04.543 INFO:teuthology.orchestra.run.smithi052.stdout: "reply": 0, 2024-04-02T14:04:04.544 INFO:teuthology.orchestra.run.smithi052.stdout: "reply_latency": { 2024-04-02T14:04:04.544 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.544 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.544 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.544 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.544 INFO:teuthology.orchestra.run.smithi052.stdout: "slow_reply": 0, 2024-04-02T14:04:04.544 INFO:teuthology.orchestra.run.smithi052.stdout: "forward": 0, 2024-04-02T14:04:04.544 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_fetch_complete": 0, 2024-04-02T14:04:04.544 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_fetch_keys": 0, 2024-04-02T14:04:04.545 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_commit": 0, 2024-04-02T14:04:04.545 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_split": 0, 2024-04-02T14:04:04.545 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_merge": 0, 2024-04-02T14:04:04.545 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes": 0, 2024-04-02T14:04:04.545 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_top": 0, 2024-04-02T14:04:04.545 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_bottom": 0, 2024-04-02T14:04:04.545 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_pin_tail": 0, 2024-04-02T14:04:04.545 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_pinned": 0, 2024-04-02T14:04:04.545 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_expired": 0, 2024-04-02T14:04:04.546 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_with_caps": 0, 2024-04-02T14:04:04.546 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": 0, 2024-04-02T14:04:04.546 INFO:teuthology.orchestra.run.smithi052.stdout: "subtrees": 1, 2024-04-02T14:04:04.546 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse": 0, 2024-04-02T14:04:04.546 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_hit": 0, 2024-04-02T14:04:04.546 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_forward": 0, 2024-04-02T14:04:04.546 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_discover": 0, 2024-04-02T14:04:04.546 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_dir_fetch": 0, 2024-04-02T14:04:04.547 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_remote_ino": 0, 2024-04-02T14:04:04.547 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_lock": 0, 2024-04-02T14:04:04.547 INFO:teuthology.orchestra.run.smithi052.stdout: "load_cent": 0, 2024-04-02T14:04:04.547 INFO:teuthology.orchestra.run.smithi052.stdout: "q": 0, 2024-04-02T14:04:04.547 INFO:teuthology.orchestra.run.smithi052.stdout: "exported": 0, 2024-04-02T14:04:04.547 INFO:teuthology.orchestra.run.smithi052.stdout: "exported_inodes": 0, 2024-04-02T14:04:04.547 INFO:teuthology.orchestra.run.smithi052.stdout: "imported": 0, 2024-04-02T14:04:04.547 INFO:teuthology.orchestra.run.smithi052.stdout: "imported_inodes": 0, 2024-04-02T14:04:04.547 INFO:teuthology.orchestra.run.smithi052.stdout: "openino_dir_fetch": 0, 2024-04-02T14:04:04.548 INFO:teuthology.orchestra.run.smithi052.stdout: "openino_backtrace_fetch": 0, 2024-04-02T14:04:04.548 INFO:teuthology.orchestra.run.smithi052.stdout: "openino_peer_discover": 0, 2024-04-02T14:04:04.548 INFO:teuthology.orchestra.run.smithi052.stdout: "root_rfiles": 0, 2024-04-02T14:04:04.548 INFO:teuthology.orchestra.run.smithi052.stdout: "root_rbytes": 0, 2024-04-02T14:04:04.548 INFO:teuthology.orchestra.run.smithi052.stdout: "root_rsnaps": 0, 2024-04-02T14:04:04.548 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T14:04:04.548 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_set_tag": 0, 2024-04-02T14:04:04.548 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T14:04:04.548 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_inotable_repaired": 0, 2024-04-02T14:04:04.549 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_dir_inodes": 0, 2024-04-02T14:04:04.549 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T14:04:04.549 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T14:04:04.549 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_file_inodes": 0, 2024-04-02T14:04:04.549 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_inode_file_caps": 0, 2024-04-02T14:04:04.549 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T14:04:04.549 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_grant": 0, 2024-04-02T14:04:04.549 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T14:04:04.549 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T14:04:04.550 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T14:04:04.550 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_caps": 0, 2024-04-02T14:04:04.550 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_caps_dirty": 0, 2024-04-02T14:04:04.550 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_cap_release": 0, 2024-04-02T14:04:04.550 INFO:teuthology.orchestra.run.smithi052.stdout: "process_request_cap_release": 0 2024-04-02T14:04:04.550 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.550 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_cache": { 2024-04-02T14:04:04.550 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_update": 0, 2024-04-02T14:04:04.550 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_update_receipt": 0, 2024-04-02T14:04:04.551 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_try_discover": 0, 2024-04-02T14:04:04.551 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_send_discover": 0, 2024-04-02T14:04:04.551 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_handle_discover": 0, 2024-04-02T14:04:04.551 INFO:teuthology.orchestra.run.smithi052.stdout: "num_strays": 0, 2024-04-02T14:04:04.551 INFO:teuthology.orchestra.run.smithi052.stdout: "num_strays_delayed": 0, 2024-04-02T14:04:04.551 INFO:teuthology.orchestra.run.smithi052.stdout: "num_strays_enqueuing": 0, 2024-04-02T14:04:04.551 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_created": 0, 2024-04-02T14:04:04.551 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_enqueued": 0, 2024-04-02T14:04:04.551 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_reintegrated": 0, 2024-04-02T14:04:04.552 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_migrated": 0, 2024-04-02T14:04:04.552 INFO:teuthology.orchestra.run.smithi052.stdout: "num_recovering_processing": 0, 2024-04-02T14:04:04.552 INFO:teuthology.orchestra.run.smithi052.stdout: "num_recovering_enqueued": 0, 2024-04-02T14:04:04.552 INFO:teuthology.orchestra.run.smithi052.stdout: "num_recovering_prioritized": 0, 2024-04-02T14:04:04.552 INFO:teuthology.orchestra.run.smithi052.stdout: "recovery_started": 0, 2024-04-02T14:04:04.552 INFO:teuthology.orchestra.run.smithi052.stdout: "recovery_completed": 0, 2024-04-02T14:04:04.552 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_quiesce_path": 0, 2024-04-02T14:04:04.552 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_quiesce_inode": 0, 2024-04-02T14:04:04.552 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T14:04:04.553 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_exportdir": 0, 2024-04-02T14:04:04.553 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_flush": 0, 2024-04-02T14:04:04.553 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_fragmentdir": 0, 2024-04-02T14:04:04.553 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_fragstats": 0, 2024-04-02T14:04:04.553 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_inodestats": 0 2024-04-02T14:04:04.553 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.553 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_log": { 2024-04-02T14:04:04.553 INFO:teuthology.orchestra.run.smithi052.stdout: "evlrg": 0, 2024-04-02T14:04:04.553 INFO:teuthology.orchestra.run.smithi052.stdout: "evadd": 0, 2024-04-02T14:04:04.554 INFO:teuthology.orchestra.run.smithi052.stdout: "evex": 0, 2024-04-02T14:04:04.554 INFO:teuthology.orchestra.run.smithi052.stdout: "evtrm": 0, 2024-04-02T14:04:04.554 INFO:teuthology.orchestra.run.smithi052.stdout: "ev": 5, 2024-04-02T14:04:04.554 INFO:teuthology.orchestra.run.smithi052.stdout: "evexg": 0, 2024-04-02T14:04:04.554 INFO:teuthology.orchestra.run.smithi052.stdout: "evexd": 0, 2024-04-02T14:04:04.554 INFO:teuthology.orchestra.run.smithi052.stdout: "segadd": 0, 2024-04-02T14:04:04.554 INFO:teuthology.orchestra.run.smithi052.stdout: "segex": 0, 2024-04-02T14:04:04.554 INFO:teuthology.orchestra.run.smithi052.stdout: "segtrm": 0, 2024-04-02T14:04:04.554 INFO:teuthology.orchestra.run.smithi052.stdout: "seg": 2, 2024-04-02T14:04:04.555 INFO:teuthology.orchestra.run.smithi052.stdout: "segmjr": 2, 2024-04-02T14:04:04.555 INFO:teuthology.orchestra.run.smithi052.stdout: "segexg": 0, 2024-04-02T14:04:04.555 INFO:teuthology.orchestra.run.smithi052.stdout: "segexd": 0, 2024-04-02T14:04:04.555 INFO:teuthology.orchestra.run.smithi052.stdout: "expos": 4194304, 2024-04-02T14:04:04.555 INFO:teuthology.orchestra.run.smithi052.stdout: "wrpos": 4195732, 2024-04-02T14:04:04.555 INFO:teuthology.orchestra.run.smithi052.stdout: "rdpos": 4195424, 2024-04-02T14:04:04.555 INFO:teuthology.orchestra.run.smithi052.stdout: "jlat": { 2024-04-02T14:04:04.555 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.555 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.556 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.556 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.556 INFO:teuthology.orchestra.run.smithi052.stdout: "replayed": 5 2024-04-02T14:04:04.556 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.556 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_mem": { 2024-04-02T14:04:04.556 INFO:teuthology.orchestra.run.smithi052.stdout: "ino": 3, 2024-04-02T14:04:04.556 INFO:teuthology.orchestra.run.smithi052.stdout: "ino+": 3, 2024-04-02T14:04:04.556 INFO:teuthology.orchestra.run.smithi052.stdout: "ino-": 0, 2024-04-02T14:04:04.556 INFO:teuthology.orchestra.run.smithi052.stdout: "dir": 1, 2024-04-02T14:04:04.557 INFO:teuthology.orchestra.run.smithi052.stdout: "dir+": 1, 2024-04-02T14:04:04.557 INFO:teuthology.orchestra.run.smithi052.stdout: "dir-": 0, 2024-04-02T14:04:04.557 INFO:teuthology.orchestra.run.smithi052.stdout: "dn": 0, 2024-04-02T14:04:04.557 INFO:teuthology.orchestra.run.smithi052.stdout: "dn+": 0, 2024-04-02T14:04:04.557 INFO:teuthology.orchestra.run.smithi052.stdout: "dn-": 0, 2024-04-02T14:04:04.557 INFO:teuthology.orchestra.run.smithi052.stdout: "cap": 0, 2024-04-02T14:04:04.557 INFO:teuthology.orchestra.run.smithi052.stdout: "cap+": 0, 2024-04-02T14:04:04.557 INFO:teuthology.orchestra.run.smithi052.stdout: "cap-": 0, 2024-04-02T14:04:04.558 INFO:teuthology.orchestra.run.smithi052.stdout: "rss": 52676, 2024-04-02T14:04:04.558 INFO:teuthology.orchestra.run.smithi052.stdout: "heap": 215416 2024-04-02T14:04:04.558 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.558 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_server": { 2024-04-02T14:04:04.558 INFO:teuthology.orchestra.run.smithi052.stdout: "dispatch_client_request": 0, 2024-04-02T14:04:04.558 INFO:teuthology.orchestra.run.smithi052.stdout: "dispatch_server_request": 0, 2024-04-02T14:04:04.558 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_request": 0, 2024-04-02T14:04:04.558 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_session": 0, 2024-04-02T14:04:04.558 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_peer_request": 0, 2024-04-02T14:04:04.559 INFO:teuthology.orchestra.run.smithi052.stdout: "req_create_latency": { 2024-04-02T14:04:04.559 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.559 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.559 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.559 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.559 INFO:teuthology.orchestra.run.smithi052.stdout: "req_getattr_latency": { 2024-04-02T14:04:04.559 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.559 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.560 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.560 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.560 INFO:teuthology.orchestra.run.smithi052.stdout: "req_getfilelock_latency": { 2024-04-02T14:04:04.560 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.560 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.560 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.560 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.560 INFO:teuthology.orchestra.run.smithi052.stdout: "req_link_latency": { 2024-04-02T14:04:04.560 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.561 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.561 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.561 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.561 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookup_latency": { 2024-04-02T14:04:04.561 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.561 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.561 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.561 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.562 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookuphash_latency": { 2024-04-02T14:04:04.562 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.562 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.562 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.562 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.562 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupino_latency": { 2024-04-02T14:04:04.562 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.562 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.563 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.563 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.563 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupname_latency": { 2024-04-02T14:04:04.563 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.563 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.563 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.563 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.563 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupparent_latency": { 2024-04-02T14:04:04.563 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.564 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.564 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.564 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.564 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupsnap_latency": { 2024-04-02T14:04:04.564 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.564 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.564 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.564 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.565 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lssnap_latency": { 2024-04-02T14:04:04.565 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.565 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.565 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.565 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.565 INFO:teuthology.orchestra.run.smithi052.stdout: "req_mkdir_latency": { 2024-04-02T14:04:04.565 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.566 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.566 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.566 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.566 INFO:teuthology.orchestra.run.smithi052.stdout: "req_mknod_latency": { 2024-04-02T14:04:04.566 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.567 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.567 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.567 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.567 INFO:teuthology.orchestra.run.smithi052.stdout: "req_mksnap_latency": { 2024-04-02T14:04:04.567 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.567 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.567 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.567 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.567 INFO:teuthology.orchestra.run.smithi052.stdout: "req_open_latency": { 2024-04-02T14:04:04.568 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.568 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.568 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.568 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.568 INFO:teuthology.orchestra.run.smithi052.stdout: "req_readdir_latency": { 2024-04-02T14:04:04.568 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.568 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.568 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.569 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.569 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rename_latency": { 2024-04-02T14:04:04.569 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.569 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.569 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.569 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.569 INFO:teuthology.orchestra.run.smithi052.stdout: "req_renamesnap_latency": { 2024-04-02T14:04:04.569 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.569 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.570 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.570 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.570 INFO:teuthology.orchestra.run.smithi052.stdout: "req_snapdiff_latency": { 2024-04-02T14:04:04.570 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.570 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.570 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.570 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.570 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rmdir_latency": { 2024-04-02T14:04:04.571 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.571 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.571 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.571 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.571 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rmsnap_latency": { 2024-04-02T14:04:04.571 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.571 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.571 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.571 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.572 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rmxattr_latency": { 2024-04-02T14:04:04.572 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.572 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.572 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.572 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.572 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setattr_latency": { 2024-04-02T14:04:04.572 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.572 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.572 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.572 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.573 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setdirlayout_latency": { 2024-04-02T14:04:04.573 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.573 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.573 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.573 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.573 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setfilelock_latency": { 2024-04-02T14:04:04.573 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.573 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.573 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.574 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.574 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setlayout_latency": { 2024-04-02T14:04:04.574 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.574 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.574 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.574 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.574 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setxattr_latency": { 2024-04-02T14:04:04.574 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.575 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.575 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.575 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.575 INFO:teuthology.orchestra.run.smithi052.stdout: "req_symlink_latency": { 2024-04-02T14:04:04.575 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.575 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.575 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.575 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.575 INFO:teuthology.orchestra.run.smithi052.stdout: "req_unlink_latency": { 2024-04-02T14:04:04.575 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.576 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.576 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.576 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.576 INFO:teuthology.orchestra.run.smithi052.stdout: "cap_revoke_eviction": 0, 2024-04-02T14:04:04.576 INFO:teuthology.orchestra.run.smithi052.stdout: "cap_acquisition_throttle": 0, 2024-04-02T14:04:04.576 INFO:teuthology.orchestra.run.smithi052.stdout: "req_getvxattr_latency": { 2024-04-02T14:04:04.576 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.576 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.576 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.577 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:04.577 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.577 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_sessions": { 2024-04-02T14:04:04.577 INFO:teuthology.orchestra.run.smithi052.stdout: "session_count": 1, 2024-04-02T14:04:04.577 INFO:teuthology.orchestra.run.smithi052.stdout: "session_add": 2, 2024-04-02T14:04:04.577 INFO:teuthology.orchestra.run.smithi052.stdout: "session_remove": 1, 2024-04-02T14:04:04.577 INFO:teuthology.orchestra.run.smithi052.stdout: "sessions_open": 1, 2024-04-02T14:04:04.577 INFO:teuthology.orchestra.run.smithi052.stdout: "sessions_stale": 0, 2024-04-02T14:04:04.577 INFO:teuthology.orchestra.run.smithi052.stdout: "total_load": 0, 2024-04-02T14:04:04.578 INFO:teuthology.orchestra.run.smithi052.stdout: "average_load": 0, 2024-04-02T14:04:04.578 INFO:teuthology.orchestra.run.smithi052.stdout: "avg_session_uptime": 1783, 2024-04-02T14:04:04.578 INFO:teuthology.orchestra.run.smithi052.stdout: "mdthresh_evicted": 0 2024-04-02T14:04:04.578 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.578 INFO:teuthology.orchestra.run.smithi052.stdout: "mempool": { 2024-04-02T14:04:04.578 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_bytes": 0, 2024-04-02T14:04:04.578 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_items": 0, 2024-04-02T14:04:04.578 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T14:04:04.579 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_items": 0, 2024-04-02T14:04:04.579 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T14:04:04.579 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_items": 0, 2024-04-02T14:04:04.579 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T14:04:04.579 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T14:04:04.579 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T14:04:04.579 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T14:04:04.579 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T14:04:04.579 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_items": 0, 2024-04-02T14:04:04.579 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T14:04:04.580 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T14:04:04.580 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_bytes": 0, 2024-04-02T14:04:04.580 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_items": 0, 2024-04-02T14:04:04.580 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_bytes": 0, 2024-04-02T14:04:04.580 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_items": 0, 2024-04-02T14:04:04.580 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T14:04:04.580 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T14:04:04.580 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T14:04:04.580 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T14:04:04.581 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T14:04:04.581 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_items": 0, 2024-04-02T14:04:04.581 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_bytes": 0, 2024-04-02T14:04:04.581 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_items": 0, 2024-04-02T14:04:04.581 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T14:04:04.581 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T14:04:04.581 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_bytes": 0, 2024-04-02T14:04:04.581 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_items": 0, 2024-04-02T14:04:04.581 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_bytes": 0, 2024-04-02T14:04:04.582 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_items": 0, 2024-04-02T14:04:04.582 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T14:04:04.582 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_items": 0, 2024-04-02T14:04:04.582 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T14:04:04.582 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_items": 0, 2024-04-02T14:04:04.582 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_bytes": 189371, 2024-04-02T14:04:04.582 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_items": 51, 2024-04-02T14:04:04.582 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_bytes": 0, 2024-04-02T14:04:04.582 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_items": 0, 2024-04-02T14:04:04.583 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_bytes": 0, 2024-04-02T14:04:04.583 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_items": 0, 2024-04-02T14:04:04.583 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_bytes": 0, 2024-04-02T14:04:04.583 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_items": 0, 2024-04-02T14:04:04.583 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_bytes": 0, 2024-04-02T14:04:04.583 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_items": 0, 2024-04-02T14:04:04.583 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_bytes": 9320, 2024-04-02T14:04:04.583 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_items": 178, 2024-04-02T14:04:04.583 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T14:04:04.584 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_items": 0, 2024-04-02T14:04:04.584 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_bytes": 0, 2024-04-02T14:04:04.584 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_items": 0, 2024-04-02T14:04:04.584 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_bytes": 8464, 2024-04-02T14:04:04.584 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_items": 13, 2024-04-02T14:04:04.584 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_bytes": 0, 2024-04-02T14:04:04.584 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_items": 0, 2024-04-02T14:04:04.584 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_bytes": 0, 2024-04-02T14:04:04.584 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_items": 0 2024-04-02T14:04:04.585 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.585 INFO:teuthology.orchestra.run.smithi052.stdout: "objecter": { 2024-04-02T14:04:04.585 INFO:teuthology.orchestra.run.smithi052.stdout: "op_active": 0, 2024-04-02T14:04:04.585 INFO:teuthology.orchestra.run.smithi052.stdout: "op_laggy": 0, 2024-04-02T14:04:04.585 INFO:teuthology.orchestra.run.smithi052.stdout: "op_send": 2184, 2024-04-02T14:04:04.585 INFO:teuthology.orchestra.run.smithi052.stdout: "op_send_bytes": 21, 2024-04-02T14:04:04.585 INFO:teuthology.orchestra.run.smithi052.stdout: "op_resend": 3, 2024-04-02T14:04:04.585 INFO:teuthology.orchestra.run.smithi052.stdout: "op_reply": 2184, 2024-04-02T14:04:04.585 INFO:teuthology.orchestra.run.smithi052.stdout: "op_latency": { 2024-04-02T14:04:04.585 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 2184, 2024-04-02T14:04:04.586 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 1.475981594, 2024-04-02T14:04:04.586 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000675815 2024-04-02T14:04:04.586 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.586 INFO:teuthology.orchestra.run.smithi052.stdout: "op_inflight": 0, 2024-04-02T14:04:04.586 INFO:teuthology.orchestra.run.smithi052.stdout: "oplen_avg": { 2024-04-02T14:04:04.586 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 2184, 2024-04-02T14:04:04.586 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 2185 2024-04-02T14:04:04.586 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.587 INFO:teuthology.orchestra.run.smithi052.stdout: "op": 2184, 2024-04-02T14:04:04.587 INFO:teuthology.orchestra.run.smithi052.stdout: "op_r": 2184, 2024-04-02T14:04:04.587 INFO:teuthology.orchestra.run.smithi052.stdout: "op_w": 0, 2024-04-02T14:04:04.587 INFO:teuthology.orchestra.run.smithi052.stdout: "op_rmw": 0, 2024-04-02T14:04:04.587 INFO:teuthology.orchestra.run.smithi052.stdout: "op_pg": 0, 2024-04-02T14:04:04.587 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_stat": 1422, 2024-04-02T14:04:04.587 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_create": 0, 2024-04-02T14:04:04.587 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_read": 760, 2024-04-02T14:04:04.587 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_write": 0, 2024-04-02T14:04:04.588 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_writefull": 0, 2024-04-02T14:04:04.588 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_writesame": 0, 2024-04-02T14:04:04.588 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_append": 0, 2024-04-02T14:04:04.588 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_zero": 0, 2024-04-02T14:04:04.588 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_truncate": 0, 2024-04-02T14:04:04.588 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_delete": 0, 2024-04-02T14:04:04.588 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_mapext": 0, 2024-04-02T14:04:04.588 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_sparse_read": 0, 2024-04-02T14:04:04.588 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_clonerange": 0, 2024-04-02T14:04:04.589 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_getxattr": 1, 2024-04-02T14:04:04.589 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_setxattr": 0, 2024-04-02T14:04:04.589 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_cmpxattr": 0, 2024-04-02T14:04:04.589 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_rmxattr": 0, 2024-04-02T14:04:04.589 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_resetxattrs": 0, 2024-04-02T14:04:04.589 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_call": 0, 2024-04-02T14:04:04.589 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_watch": 0, 2024-04-02T14:04:04.589 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_notify": 0, 2024-04-02T14:04:04.589 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T14:04:04.589 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_pgls": 0, 2024-04-02T14:04:04.590 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_pgls_filter": 0, 2024-04-02T14:04:04.590 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_other": 0, 2024-04-02T14:04:04.590 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_active": 0, 2024-04-02T14:04:04.590 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_send": 0, 2024-04-02T14:04:04.590 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_resend": 0, 2024-04-02T14:04:04.590 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_ping": 0, 2024-04-02T14:04:04.590 INFO:teuthology.orchestra.run.smithi052.stdout: "poolop_active": 0, 2024-04-02T14:04:04.590 INFO:teuthology.orchestra.run.smithi052.stdout: "poolop_send": 0, 2024-04-02T14:04:04.591 INFO:teuthology.orchestra.run.smithi052.stdout: "poolop_resend": 0, 2024-04-02T14:04:04.591 INFO:teuthology.orchestra.run.smithi052.stdout: "poolstat_active": 0, 2024-04-02T14:04:04.591 INFO:teuthology.orchestra.run.smithi052.stdout: "poolstat_send": 0, 2024-04-02T14:04:04.591 INFO:teuthology.orchestra.run.smithi052.stdout: "poolstat_resend": 0, 2024-04-02T14:04:04.591 INFO:teuthology.orchestra.run.smithi052.stdout: "statfs_active": 0, 2024-04-02T14:04:04.591 INFO:teuthology.orchestra.run.smithi052.stdout: "statfs_send": 0, 2024-04-02T14:04:04.591 INFO:teuthology.orchestra.run.smithi052.stdout: "statfs_resend": 0, 2024-04-02T14:04:04.591 INFO:teuthology.orchestra.run.smithi052.stdout: "command_active": 0, 2024-04-02T14:04:04.591 INFO:teuthology.orchestra.run.smithi052.stdout: "command_send": 0, 2024-04-02T14:04:04.592 INFO:teuthology.orchestra.run.smithi052.stdout: "command_resend": 0, 2024-04-02T14:04:04.592 INFO:teuthology.orchestra.run.smithi052.stdout: "map_epoch": 101, 2024-04-02T14:04:04.592 INFO:teuthology.orchestra.run.smithi052.stdout: "map_full": 0, 2024-04-02T14:04:04.592 INFO:teuthology.orchestra.run.smithi052.stdout: "map_inc": 11, 2024-04-02T14:04:04.592 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_sessions": 5, 2024-04-02T14:04:04.592 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_session_open": 5, 2024-04-02T14:04:04.592 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_session_close": 0, 2024-04-02T14:04:04.592 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_laggy": 0, 2024-04-02T14:04:04.592 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_wr": 0, 2024-04-02T14:04:04.593 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_rd": 2, 2024-04-02T14:04:04.593 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_del": 0 2024-04-02T14:04:04.593 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.593 INFO:teuthology.orchestra.run.smithi052.stdout: "oft": { 2024-04-02T14:04:04.593 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_objs": 0, 2024-04-02T14:04:04.593 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_kv_pairs": 0, 2024-04-02T14:04:04.593 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_updates": 0, 2024-04-02T14:04:04.593 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_removes": 0 2024-04-02T14:04:04.593 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.593 INFO:teuthology.orchestra.run.smithi052.stdout: "osd-slow-ops": { 2024-04-02T14:04:04.594 INFO:teuthology.orchestra.run.smithi052.stdout: "slow_ops_count": 0 2024-04-02T14:04:04.594 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.594 INFO:teuthology.orchestra.run.smithi052.stdout: "purge_queue": { 2024-04-02T14:04:04.594 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing_ops": 0, 2024-04-02T14:04:04.594 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T14:04:04.594 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing": 0, 2024-04-02T14:04:04.594 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing_high_water": 0, 2024-04-02T14:04:04.594 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executed_ops": 0, 2024-04-02T14:04:04.595 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executed": 0, 2024-04-02T14:04:04.595 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_item_in_journal": 0 2024-04-02T14:04:04.595 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.595 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T14:04:04.595 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-04-02T14:04:04.595 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-04-02T14:04:04.595 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-04-02T14:04:04.595 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 2430, 2024-04-02T14:04:04.596 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 2024795, 2024-04-02T14:04:04.596 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:04.596 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 2430, 2024-04-02T14:04:04.596 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-04-02T14:04:04.596 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-04-02T14:04:04.596 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 2430, 2024-04-02T14:04:04.596 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 2024795, 2024-04-02T14:04:04.596 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-04-02T14:04:04.596 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.596 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.597 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.597 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:04.597 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.597 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-objecter_bytes": { 2024-04-02T14:04:04.597 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-04-02T14:04:04.597 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-04-02T14:04:04.597 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-04-02T14:04:04.597 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-04-02T14:04:04.597 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-04-02T14:04:04.598 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:04.598 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:04.598 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 2184, 2024-04-02T14:04:04.598 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 1433, 2024-04-02T14:04:04.598 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 2184, 2024-04-02T14:04:04.598 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 1433, 2024-04-02T14:04:04.598 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-04-02T14:04:04.598 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.599 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.599 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.599 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:04.599 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.599 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-objecter_ops": { 2024-04-02T14:04:04.599 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-04-02T14:04:04.599 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 1024, 2024-04-02T14:04:04.599 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-04-02T14:04:04.600 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-04-02T14:04:04.600 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-04-02T14:04:04.600 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:04.600 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:04.600 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 2184, 2024-04-02T14:04:04.600 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 2184, 2024-04-02T14:04:04.600 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 2184, 2024-04-02T14:04:04.600 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 2184, 2024-04-02T14:04:04.600 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-04-02T14:04:04.600 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.601 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.601 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.601 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:04.601 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.601 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-write_buf_throttle": { 2024-04-02T14:04:04.601 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-04-02T14:04:04.601 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 3758096384, 2024-04-02T14:04:04.601 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-04-02T14:04:04.601 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-04-02T14:04:04.602 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-04-02T14:04:04.602 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:04.602 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:04.602 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-04-02T14:04:04.602 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-04-02T14:04:04.602 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 0, 2024-04-02T14:04:04.602 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 0, 2024-04-02T14:04:04.602 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-04-02T14:04:04.603 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.603 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.603 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.603 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:04.603 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:04.603 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-write_buf_throttle-0x5645abb2c6e0": { 2024-04-02T14:04:04.603 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-04-02T14:04:04.603 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 3758096384, 2024-04-02T14:04:04.603 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-04-02T14:04:04.604 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-04-02T14:04:04.604 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-04-02T14:04:04.604 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:04.604 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:04.604 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-04-02T14:04:04.604 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-04-02T14:04:04.604 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 0, 2024-04-02T14:04:04.604 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 0, 2024-04-02T14:04:04.604 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-04-02T14:04:04.605 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:04.605 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:04.605 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:04.605 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:04.605 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:04.605 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:04:05.435 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-02T14:04:05.436 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-02T14:04:05.436 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-04-02T14:04:05.436 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-04-02T14:04:05.436 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-04-02T14:04:05.436 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-02T14:04:05.436 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T14:04:05.436 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-04-02T14:04:05.436 DEBUG:tasks.check_counter:Getting stats from d 2024-04-02T14:04:05.437 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph daemon mds.d perf dump 2024-04-02T14:04:05.682 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:04:05.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:05 smithi189 ceph-mon[36683]: pgmap v328: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:04:05.803 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:05 smithi052 ceph-mon[32104]: pgmap v328: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:04:05.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:05 smithi088 ceph-mon[36528]: pgmap v328: 129 pgs: 129 active+clean; 73 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:04:07.250 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:04:07.250 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T14:04:07.251 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 4, 2024-04-02T14:04:07.251 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 4, 2024-04-02T14:04:07.251 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 3072, 2024-04-02T14:04:07.251 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 448, 2024-04-02T14:04:07.251 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 4, 2024-04-02T14:04:07.251 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 0, 2024-04-02T14:04:07.251 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.001480801, 2024-04-02T14:04:07.251 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.000033227, 2024-04-02T14:04:07.252 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.000644916, 2024-04-02T14:04:07.252 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T14:04:07.252 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:07.252 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 3, 2024-04-02T14:04:07.252 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000785294, 2024-04-02T14:04:07.252 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000261764 2024-04-02T14:04:07.252 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:07.252 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:07.252 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:07.253 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:07.253 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:07.253 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:07.253 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-04-02T14:04:07.253 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 448 2024-04-02T14:04:07.253 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:07.253 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T14:04:07.253 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 230, 2024-04-02T14:04:07.254 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 219, 2024-04-02T14:04:07.254 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 1592192, 2024-04-02T14:04:07.254 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 134464, 2024-04-02T14:04:07.254 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 2, 2024-04-02T14:04:07.254 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 1, 2024-04-02T14:04:07.254 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.042282714, 2024-04-02T14:04:07.254 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.014340133, 2024-04-02T14:04:07.254 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.032391450, 2024-04-02T14:04:07.255 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.004435618, 2024-04-02T14:04:07.255 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:07.255 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 218, 2024-04-02T14:04:07.255 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.010346549, 2024-04-02T14:04:07.255 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000047461 2024-04-02T14:04:07.255 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:07.255 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:07.255 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:07.255 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:07.256 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:07.256 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:07.256 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 1592192, 2024-04-02T14:04:07.256 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 134464 2024-04-02T14:04:07.256 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:07.256 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T14:04:07.256 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 2, 2024-04-02T14:04:07.256 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 167, 2024-04-02T14:04:07.257 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 320, 2024-04-02T14:04:07.257 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 185120, 2024-04-02T14:04:07.257 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 5, 2024-04-02T14:04:07.257 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 3, 2024-04-02T14:04:07.257 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.015185080, 2024-04-02T14:04:07.257 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.010969941, 2024-04-02T14:04:07.257 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.001632741, 2024-04-02T14:04:07.257 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T14:04:07.257 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:07.258 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 164, 2024-04-02T14:04:07.258 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.012915344, 2024-04-02T14:04:07.258 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000078752 2024-04-02T14:04:07.258 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:07.258 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:07.258 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:07.258 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:07.258 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:07.259 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:07.259 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-02T14:04:07.259 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 185120 2024-04-02T14:04:07.259 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:07.259 INFO:teuthology.orchestra.run.smithi052.stdout: "cct": { 2024-04-02T14:04:07.259 INFO:teuthology.orchestra.run.smithi052.stdout: "total_workers": 0, 2024-04-02T14:04:07.259 INFO:teuthology.orchestra.run.smithi052.stdout: "unhealthy_workers": 0 2024-04-02T14:04:07.259 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:07.259 INFO:teuthology.orchestra.run.smithi052.stdout: "mempool": { 2024-04-02T14:04:07.260 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_bytes": 0, 2024-04-02T14:04:07.260 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_items": 0, 2024-04-02T14:04:07.260 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T14:04:07.260 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_items": 0, 2024-04-02T14:04:07.260 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T14:04:07.260 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_items": 0, 2024-04-02T14:04:07.260 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T14:04:07.260 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T14:04:07.260 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T14:04:07.261 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T14:04:07.261 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T14:04:07.261 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_items": 0, 2024-04-02T14:04:07.261 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T14:04:07.261 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T14:04:07.261 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_bytes": 0, 2024-04-02T14:04:07.261 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_items": 0, 2024-04-02T14:04:07.261 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_bytes": 0, 2024-04-02T14:04:07.261 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_items": 0, 2024-04-02T14:04:07.262 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T14:04:07.262 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T14:04:07.262 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T14:04:07.262 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T14:04:07.262 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T14:04:07.262 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_items": 0, 2024-04-02T14:04:07.262 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_bytes": 0, 2024-04-02T14:04:07.263 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_items": 0, 2024-04-02T14:04:07.263 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T14:04:07.263 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T14:04:07.263 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_bytes": 0, 2024-04-02T14:04:07.263 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_items": 0, 2024-04-02T14:04:07.263 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_bytes": 0, 2024-04-02T14:04:07.263 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_items": 0, 2024-04-02T14:04:07.263 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T14:04:07.263 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_items": 0, 2024-04-02T14:04:07.264 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T14:04:07.264 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_items": 0, 2024-04-02T14:04:07.264 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_bytes": 148995, 2024-04-02T14:04:07.264 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_items": 31, 2024-04-02T14:04:07.264 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_bytes": 0, 2024-04-02T14:04:07.264 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_items": 0, 2024-04-02T14:04:07.264 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_bytes": 0, 2024-04-02T14:04:07.264 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_items": 0, 2024-04-02T14:04:07.264 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_bytes": 0, 2024-04-02T14:04:07.265 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_items": 0, 2024-04-02T14:04:07.265 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_bytes": 0, 2024-04-02T14:04:07.265 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_items": 0, 2024-04-02T14:04:07.265 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_bytes": 0, 2024-04-02T14:04:07.265 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_items": 0, 2024-04-02T14:04:07.265 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T14:04:07.265 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_items": 0, 2024-04-02T14:04:07.265 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_bytes": 0, 2024-04-02T14:04:07.265 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_items": 0, 2024-04-02T14:04:07.266 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_bytes": 928, 2024-04-02T14:04:07.266 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_items": 2, 2024-04-02T14:04:07.266 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_bytes": 0, 2024-04-02T14:04:07.266 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_items": 0, 2024-04-02T14:04:07.266 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_bytes": 0, 2024-04-02T14:04:07.266 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_items": 0 2024-04-02T14:04:07.266 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:07.267 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T14:04:07.267 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-04-02T14:04:07.267 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-04-02T14:04:07.267 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-04-02T14:04:07.267 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 232, 2024-04-02T14:04:07.267 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 1559746, 2024-04-02T14:04:07.267 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:07.267 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 232, 2024-04-02T14:04:07.267 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-04-02T14:04:07.268 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-04-02T14:04:07.268 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 232, 2024-04-02T14:04:07.268 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 1559746, 2024-04-02T14:04:07.268 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-04-02T14:04:07.268 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:07.268 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:07.268 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:07.268 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:07.269 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:07.269 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:04:07.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:07 smithi052 ceph-mon[32104]: pgmap v329: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:04:07.722 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:07 smithi088 ceph-mon[36528]: pgmap v329: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:04:07.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:07 smithi189 ceph-mon[36683]: pgmap v329: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-02T14:04:08.091 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-02T14:04:08.092 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-02T14:04:08.092 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-02T14:04:08.092 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-02T14:04:08.092 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-02T14:04:08.092 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-02T14:04:08.092 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-02T14:04:08.092 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-02T14:04:08.093 DEBUG:tasks.check_counter:Getting stats from g 2024-04-02T14:04:08.093 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph daemon mds.g perf dump 2024-04-02T14:04:08.344 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:04:09.708 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:09 smithi052 ceph-mon[32104]: pgmap v330: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:04:09.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:09 smithi189 ceph-mon[36683]: pgmap v330: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:04:09.919 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:09 smithi088 ceph-mon[36528]: pgmap v330: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:04:10.102 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:04:10.102 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T14:04:10.102 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 1005, 2024-04-02T14:04:10.102 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 991, 2024-04-02T14:04:10.102 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 1891936, 2024-04-02T14:04:10.103 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 376674, 2024-04-02T14:04:10.103 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 5, 2024-04-02T14:04:10.103 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 1, 2024-04-02T14:04:10.103 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.130271530, 2024-04-02T14:04:10.103 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.052315514, 2024-04-02T14:04:10.103 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.052489266, 2024-04-02T14:04:10.104 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.026289882, 2024-04-02T14:04:10.104 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:10.104 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 990, 2024-04-02T14:04:10.104 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.049144393, 2024-04-02T14:04:10.104 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000049640 2024-04-02T14:04:10.104 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.104 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:10.105 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.105 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.105 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.105 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.105 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 1636912, 2024-04-02T14:04:10.105 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 135312 2024-04-02T14:04:10.105 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.105 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T14:04:10.106 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 1409, 2024-04-02T14:04:10.106 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 1569, 2024-04-02T14:04:10.106 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 362698, 2024-04-02T14:04:10.106 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 893827, 2024-04-02T14:04:10.106 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 7, 2024-04-02T14:04:10.106 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 5, 2024-04-02T14:04:10.106 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.106030395, 2024-04-02T14:04:10.106 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.035334925, 2024-04-02T14:04:10.107 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.026779579, 2024-04-02T14:04:10.107 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.035280305, 2024-04-02T14:04:10.107 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:10.107 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 1567, 2024-04-02T14:04:10.107 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.055467706, 2024-04-02T14:04:10.107 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000035397 2024-04-02T14:04:10.107 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.107 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:10.108 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.108 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.108 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.108 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.108 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-04-02T14:04:10.108 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 447616 2024-04-02T14:04:10.109 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.109 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T14:04:10.109 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 4, 2024-04-02T14:04:10.109 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 6, 2024-04-02T14:04:10.110 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 1629, 2024-04-02T14:04:10.110 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 1284, 2024-04-02T14:04:10.110 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 5, 2024-04-02T14:04:10.110 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 4, 2024-04-02T14:04:10.110 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.001391497, 2024-04-02T14:04:10.110 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.000064154, 2024-04-02T14:04:10.110 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.001162793, 2024-04-02T14:04:10.111 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000074957, 2024-04-02T14:04:10.111 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:10.111 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 4, 2024-04-02T14:04:10.111 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.002345193, 2024-04-02T14:04:10.111 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000586298 2024-04-02T14:04:10.112 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.112 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:10.112 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.112 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.112 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.112 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.113 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T14:04:10.113 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T14:04:10.113 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.113 INFO:teuthology.orchestra.run.smithi052.stdout: "cct": { 2024-04-02T14:04:10.113 INFO:teuthology.orchestra.run.smithi052.stdout: "total_workers": 1, 2024-04-02T14:04:10.113 INFO:teuthology.orchestra.run.smithi052.stdout: "unhealthy_workers": 0 2024-04-02T14:04:10.113 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.113 INFO:teuthology.orchestra.run.smithi052.stdout: "finisher-MDSRank": { 2024-04-02T14:04:10.114 INFO:teuthology.orchestra.run.smithi052.stdout: "queue_len": 0, 2024-04-02T14:04:10.114 INFO:teuthology.orchestra.run.smithi052.stdout: "complete_latency": { 2024-04-02T14:04:10.114 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 3671, 2024-04-02T14:04:10.114 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.089953858, 2024-04-02T14:04:10.114 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000024503 2024-04-02T14:04:10.114 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:10.114 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.114 INFO:teuthology.orchestra.run.smithi052.stdout: "finisher-PurgeQueue": { 2024-04-02T14:04:10.115 INFO:teuthology.orchestra.run.smithi052.stdout: "queue_len": 0, 2024-04-02T14:04:10.115 INFO:teuthology.orchestra.run.smithi052.stdout: "complete_latency": { 2024-04-02T14:04:10.115 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.115 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.115 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.115 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:10.115 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.116 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": { 2024-04-02T14:04:10.116 INFO:teuthology.orchestra.run.smithi052.stdout: "request": 0, 2024-04-02T14:04:10.116 INFO:teuthology.orchestra.run.smithi052.stdout: "reply": 0, 2024-04-02T14:04:10.116 INFO:teuthology.orchestra.run.smithi052.stdout: "reply_latency": { 2024-04-02T14:04:10.116 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.116 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.116 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.116 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.117 INFO:teuthology.orchestra.run.smithi052.stdout: "slow_reply": 0, 2024-04-02T14:04:10.117 INFO:teuthology.orchestra.run.smithi052.stdout: "forward": 0, 2024-04-02T14:04:10.117 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_fetch_complete": 0, 2024-04-02T14:04:10.117 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_fetch_keys": 0, 2024-04-02T14:04:10.117 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_commit": 0, 2024-04-02T14:04:10.117 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_split": 0, 2024-04-02T14:04:10.117 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_merge": 0, 2024-04-02T14:04:10.117 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes": 0, 2024-04-02T14:04:10.118 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_top": 0, 2024-04-02T14:04:10.118 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_bottom": 0, 2024-04-02T14:04:10.118 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_pin_tail": 0, 2024-04-02T14:04:10.118 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_pinned": 0, 2024-04-02T14:04:10.118 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_expired": 0, 2024-04-02T14:04:10.118 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_with_caps": 0, 2024-04-02T14:04:10.118 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": 0, 2024-04-02T14:04:10.119 INFO:teuthology.orchestra.run.smithi052.stdout: "subtrees": 1, 2024-04-02T14:04:10.119 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse": 0, 2024-04-02T14:04:10.119 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_hit": 0, 2024-04-02T14:04:10.119 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_forward": 0, 2024-04-02T14:04:10.119 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_discover": 0, 2024-04-02T14:04:10.119 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_dir_fetch": 0, 2024-04-02T14:04:10.119 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_remote_ino": 0, 2024-04-02T14:04:10.119 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_lock": 0, 2024-04-02T14:04:10.119 INFO:teuthology.orchestra.run.smithi052.stdout: "load_cent": 0, 2024-04-02T14:04:10.120 INFO:teuthology.orchestra.run.smithi052.stdout: "q": 0, 2024-04-02T14:04:10.120 INFO:teuthology.orchestra.run.smithi052.stdout: "exported": 0, 2024-04-02T14:04:10.120 INFO:teuthology.orchestra.run.smithi052.stdout: "exported_inodes": 0, 2024-04-02T14:04:10.120 INFO:teuthology.orchestra.run.smithi052.stdout: "imported": 0, 2024-04-02T14:04:10.120 INFO:teuthology.orchestra.run.smithi052.stdout: "imported_inodes": 0, 2024-04-02T14:04:10.120 INFO:teuthology.orchestra.run.smithi052.stdout: "openino_dir_fetch": 0, 2024-04-02T14:04:10.120 INFO:teuthology.orchestra.run.smithi052.stdout: "openino_backtrace_fetch": 0, 2024-04-02T14:04:10.120 INFO:teuthology.orchestra.run.smithi052.stdout: "openino_peer_discover": 0, 2024-04-02T14:04:10.121 INFO:teuthology.orchestra.run.smithi052.stdout: "root_rfiles": 0, 2024-04-02T14:04:10.121 INFO:teuthology.orchestra.run.smithi052.stdout: "root_rbytes": 0, 2024-04-02T14:04:10.121 INFO:teuthology.orchestra.run.smithi052.stdout: "root_rsnaps": 0, 2024-04-02T14:04:10.121 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T14:04:10.121 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_set_tag": 0, 2024-04-02T14:04:10.121 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T14:04:10.121 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_inotable_repaired": 0, 2024-04-02T14:04:10.122 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_dir_inodes": 0, 2024-04-02T14:04:10.122 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T14:04:10.122 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T14:04:10.122 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_file_inodes": 0, 2024-04-02T14:04:10.122 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_inode_file_caps": 0, 2024-04-02T14:04:10.122 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T14:04:10.122 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_grant": 0, 2024-04-02T14:04:10.123 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T14:04:10.123 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T14:04:10.123 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T14:04:10.123 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_caps": 0, 2024-04-02T14:04:10.123 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_caps_dirty": 0, 2024-04-02T14:04:10.123 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_cap_release": 0, 2024-04-02T14:04:10.123 INFO:teuthology.orchestra.run.smithi052.stdout: "process_request_cap_release": 0 2024-04-02T14:04:10.123 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.123 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_cache": { 2024-04-02T14:04:10.124 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_update": 0, 2024-04-02T14:04:10.124 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_update_receipt": 0, 2024-04-02T14:04:10.124 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_try_discover": 0, 2024-04-02T14:04:10.124 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_send_discover": 0, 2024-04-02T14:04:10.124 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_handle_discover": 0, 2024-04-02T14:04:10.124 INFO:teuthology.orchestra.run.smithi052.stdout: "num_strays": 0, 2024-04-02T14:04:10.124 INFO:teuthology.orchestra.run.smithi052.stdout: "num_strays_delayed": 0, 2024-04-02T14:04:10.124 INFO:teuthology.orchestra.run.smithi052.stdout: "num_strays_enqueuing": 0, 2024-04-02T14:04:10.124 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_created": 0, 2024-04-02T14:04:10.125 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_enqueued": 0, 2024-04-02T14:04:10.125 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_reintegrated": 0, 2024-04-02T14:04:10.125 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_migrated": 0, 2024-04-02T14:04:10.125 INFO:teuthology.orchestra.run.smithi052.stdout: "num_recovering_processing": 0, 2024-04-02T14:04:10.125 INFO:teuthology.orchestra.run.smithi052.stdout: "num_recovering_enqueued": 0, 2024-04-02T14:04:10.125 INFO:teuthology.orchestra.run.smithi052.stdout: "num_recovering_prioritized": 0, 2024-04-02T14:04:10.125 INFO:teuthology.orchestra.run.smithi052.stdout: "recovery_started": 0, 2024-04-02T14:04:10.125 INFO:teuthology.orchestra.run.smithi052.stdout: "recovery_completed": 0, 2024-04-02T14:04:10.125 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_quiesce_path": 0, 2024-04-02T14:04:10.126 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_quiesce_inode": 0, 2024-04-02T14:04:10.126 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T14:04:10.126 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_exportdir": 0, 2024-04-02T14:04:10.126 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_flush": 0, 2024-04-02T14:04:10.126 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_fragmentdir": 0, 2024-04-02T14:04:10.126 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_fragstats": 0, 2024-04-02T14:04:10.126 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_inodestats": 0 2024-04-02T14:04:10.126 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.127 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_log": { 2024-04-02T14:04:10.127 INFO:teuthology.orchestra.run.smithi052.stdout: "evlrg": 0, 2024-04-02T14:04:10.127 INFO:teuthology.orchestra.run.smithi052.stdout: "evadd": 0, 2024-04-02T14:04:10.127 INFO:teuthology.orchestra.run.smithi052.stdout: "evex": 0, 2024-04-02T14:04:10.127 INFO:teuthology.orchestra.run.smithi052.stdout: "evtrm": 0, 2024-04-02T14:04:10.127 INFO:teuthology.orchestra.run.smithi052.stdout: "ev": 7, 2024-04-02T14:04:10.127 INFO:teuthology.orchestra.run.smithi052.stdout: "evexg": 0, 2024-04-02T14:04:10.127 INFO:teuthology.orchestra.run.smithi052.stdout: "evexd": 0, 2024-04-02T14:04:10.127 INFO:teuthology.orchestra.run.smithi052.stdout: "segadd": 0, 2024-04-02T14:04:10.128 INFO:teuthology.orchestra.run.smithi052.stdout: "segex": 0, 2024-04-02T14:04:10.128 INFO:teuthology.orchestra.run.smithi052.stdout: "segtrm": 0, 2024-04-02T14:04:10.128 INFO:teuthology.orchestra.run.smithi052.stdout: "seg": 2, 2024-04-02T14:04:10.128 INFO:teuthology.orchestra.run.smithi052.stdout: "segmjr": 2, 2024-04-02T14:04:10.128 INFO:teuthology.orchestra.run.smithi052.stdout: "segexg": 0, 2024-04-02T14:04:10.128 INFO:teuthology.orchestra.run.smithi052.stdout: "segexd": 0, 2024-04-02T14:04:10.128 INFO:teuthology.orchestra.run.smithi052.stdout: "expos": 4194304, 2024-04-02T14:04:10.128 INFO:teuthology.orchestra.run.smithi052.stdout: "wrpos": 4196264, 2024-04-02T14:04:10.128 INFO:teuthology.orchestra.run.smithi052.stdout: "rdpos": 4196123, 2024-04-02T14:04:10.129 INFO:teuthology.orchestra.run.smithi052.stdout: "jlat": { 2024-04-02T14:04:10.129 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.129 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.129 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.129 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.129 INFO:teuthology.orchestra.run.smithi052.stdout: "replayed": 7 2024-04-02T14:04:10.129 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.130 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_mem": { 2024-04-02T14:04:10.130 INFO:teuthology.orchestra.run.smithi052.stdout: "ino": 3, 2024-04-02T14:04:10.130 INFO:teuthology.orchestra.run.smithi052.stdout: "ino+": 3, 2024-04-02T14:04:10.130 INFO:teuthology.orchestra.run.smithi052.stdout: "ino-": 0, 2024-04-02T14:04:10.130 INFO:teuthology.orchestra.run.smithi052.stdout: "dir": 1, 2024-04-02T14:04:10.130 INFO:teuthology.orchestra.run.smithi052.stdout: "dir+": 1, 2024-04-02T14:04:10.130 INFO:teuthology.orchestra.run.smithi052.stdout: "dir-": 0, 2024-04-02T14:04:10.131 INFO:teuthology.orchestra.run.smithi052.stdout: "dn": 0, 2024-04-02T14:04:10.131 INFO:teuthology.orchestra.run.smithi052.stdout: "dn+": 0, 2024-04-02T14:04:10.131 INFO:teuthology.orchestra.run.smithi052.stdout: "dn-": 0, 2024-04-02T14:04:10.131 INFO:teuthology.orchestra.run.smithi052.stdout: "cap": 0, 2024-04-02T14:04:10.131 INFO:teuthology.orchestra.run.smithi052.stdout: "cap+": 0, 2024-04-02T14:04:10.131 INFO:teuthology.orchestra.run.smithi052.stdout: "cap-": 0, 2024-04-02T14:04:10.131 INFO:teuthology.orchestra.run.smithi052.stdout: "rss": 49928, 2024-04-02T14:04:10.131 INFO:teuthology.orchestra.run.smithi052.stdout: "heap": 215416 2024-04-02T14:04:10.132 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.132 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_server": { 2024-04-02T14:04:10.132 INFO:teuthology.orchestra.run.smithi052.stdout: "dispatch_client_request": 0, 2024-04-02T14:04:10.132 INFO:teuthology.orchestra.run.smithi052.stdout: "dispatch_server_request": 0, 2024-04-02T14:04:10.132 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_request": 0, 2024-04-02T14:04:10.132 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_session": 0, 2024-04-02T14:04:10.132 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_peer_request": 0, 2024-04-02T14:04:10.132 INFO:teuthology.orchestra.run.smithi052.stdout: "req_create_latency": { 2024-04-02T14:04:10.133 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.133 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.133 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.133 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.133 INFO:teuthology.orchestra.run.smithi052.stdout: "req_getattr_latency": { 2024-04-02T14:04:10.133 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.133 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.134 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.134 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.134 INFO:teuthology.orchestra.run.smithi052.stdout: "req_getfilelock_latency": { 2024-04-02T14:04:10.134 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.134 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.134 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.134 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.135 INFO:teuthology.orchestra.run.smithi052.stdout: "req_link_latency": { 2024-04-02T14:04:10.135 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.135 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.135 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.135 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.135 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookup_latency": { 2024-04-02T14:04:10.135 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.136 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.136 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.136 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.136 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookuphash_latency": { 2024-04-02T14:04:10.136 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.136 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.136 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.136 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.137 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupino_latency": { 2024-04-02T14:04:10.137 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.137 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.137 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.137 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.137 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupname_latency": { 2024-04-02T14:04:10.138 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.138 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.138 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.138 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.138 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupparent_latency": { 2024-04-02T14:04:10.138 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.138 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.139 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.139 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.139 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupsnap_latency": { 2024-04-02T14:04:10.139 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.139 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.139 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.139 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.139 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lssnap_latency": { 2024-04-02T14:04:10.140 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.140 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.140 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.140 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.140 INFO:teuthology.orchestra.run.smithi052.stdout: "req_mkdir_latency": { 2024-04-02T14:04:10.140 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.141 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.141 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.141 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.142 INFO:teuthology.orchestra.run.smithi052.stdout: "req_mknod_latency": { 2024-04-02T14:04:10.142 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.142 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.142 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.142 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.142 INFO:teuthology.orchestra.run.smithi052.stdout: "req_mksnap_latency": { 2024-04-02T14:04:10.142 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.142 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.143 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.143 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.143 INFO:teuthology.orchestra.run.smithi052.stdout: "req_open_latency": { 2024-04-02T14:04:10.143 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.143 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.143 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.143 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.143 INFO:teuthology.orchestra.run.smithi052.stdout: "req_readdir_latency": { 2024-04-02T14:04:10.144 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.144 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.144 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.144 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.144 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rename_latency": { 2024-04-02T14:04:10.144 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.144 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.145 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.145 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.145 INFO:teuthology.orchestra.run.smithi052.stdout: "req_renamesnap_latency": { 2024-04-02T14:04:10.145 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.145 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.145 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.145 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.145 INFO:teuthology.orchestra.run.smithi052.stdout: "req_snapdiff_latency": { 2024-04-02T14:04:10.145 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.146 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.146 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.146 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.146 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rmdir_latency": { 2024-04-02T14:04:10.146 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.146 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.146 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.146 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.147 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rmsnap_latency": { 2024-04-02T14:04:10.147 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.147 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.147 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.147 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.147 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rmxattr_latency": { 2024-04-02T14:04:10.147 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.147 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.148 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.148 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.148 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setattr_latency": { 2024-04-02T14:04:10.148 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.148 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.148 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.148 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.148 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setdirlayout_latency": { 2024-04-02T14:04:10.148 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.149 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.149 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.149 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.149 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setfilelock_latency": { 2024-04-02T14:04:10.149 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.149 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.149 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.149 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.150 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setlayout_latency": { 2024-04-02T14:04:10.150 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.150 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.150 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.150 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.150 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setxattr_latency": { 2024-04-02T14:04:10.150 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.150 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.151 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.151 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.151 INFO:teuthology.orchestra.run.smithi052.stdout: "req_symlink_latency": { 2024-04-02T14:04:10.151 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.151 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.151 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.151 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.151 INFO:teuthology.orchestra.run.smithi052.stdout: "req_unlink_latency": { 2024-04-02T14:04:10.152 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.152 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.152 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.152 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.152 INFO:teuthology.orchestra.run.smithi052.stdout: "cap_revoke_eviction": 0, 2024-04-02T14:04:10.152 INFO:teuthology.orchestra.run.smithi052.stdout: "cap_acquisition_throttle": 0, 2024-04-02T14:04:10.152 INFO:teuthology.orchestra.run.smithi052.stdout: "req_getvxattr_latency": { 2024-04-02T14:04:10.152 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.153 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.153 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.153 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:10.153 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.153 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_sessions": { 2024-04-02T14:04:10.153 INFO:teuthology.orchestra.run.smithi052.stdout: "session_count": 1, 2024-04-02T14:04:10.153 INFO:teuthology.orchestra.run.smithi052.stdout: "session_add": 3, 2024-04-02T14:04:10.153 INFO:teuthology.orchestra.run.smithi052.stdout: "session_remove": 2, 2024-04-02T14:04:10.154 INFO:teuthology.orchestra.run.smithi052.stdout: "sessions_open": 3, 2024-04-02T14:04:10.154 INFO:teuthology.orchestra.run.smithi052.stdout: "sessions_stale": 0, 2024-04-02T14:04:10.154 INFO:teuthology.orchestra.run.smithi052.stdout: "total_load": 0, 2024-04-02T14:04:10.154 INFO:teuthology.orchestra.run.smithi052.stdout: "average_load": 0, 2024-04-02T14:04:10.154 INFO:teuthology.orchestra.run.smithi052.stdout: "avg_session_uptime": 4095, 2024-04-02T14:04:10.154 INFO:teuthology.orchestra.run.smithi052.stdout: "mdthresh_evicted": 0 2024-04-02T14:04:10.154 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.154 INFO:teuthology.orchestra.run.smithi052.stdout: "mempool": { 2024-04-02T14:04:10.155 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_bytes": 0, 2024-04-02T14:04:10.155 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_items": 0, 2024-04-02T14:04:10.155 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T14:04:10.155 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_items": 0, 2024-04-02T14:04:10.155 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T14:04:10.155 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_items": 0, 2024-04-02T14:04:10.155 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T14:04:10.155 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T14:04:10.156 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T14:04:10.156 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T14:04:10.156 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T14:04:10.156 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_items": 0, 2024-04-02T14:04:10.156 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T14:04:10.156 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T14:04:10.156 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_bytes": 0, 2024-04-02T14:04:10.156 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_items": 0, 2024-04-02T14:04:10.156 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_bytes": 0, 2024-04-02T14:04:10.157 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_items": 0, 2024-04-02T14:04:10.157 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T14:04:10.157 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T14:04:10.157 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T14:04:10.157 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T14:04:10.157 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T14:04:10.157 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_items": 0, 2024-04-02T14:04:10.157 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_bytes": 0, 2024-04-02T14:04:10.158 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_items": 0, 2024-04-02T14:04:10.158 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T14:04:10.158 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T14:04:10.158 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_bytes": 0, 2024-04-02T14:04:10.158 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_items": 0, 2024-04-02T14:04:10.158 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_bytes": 0, 2024-04-02T14:04:10.158 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_items": 0, 2024-04-02T14:04:10.158 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T14:04:10.159 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_items": 0, 2024-04-02T14:04:10.159 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T14:04:10.159 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_items": 0, 2024-04-02T14:04:10.159 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_bytes": 197553, 2024-04-02T14:04:10.159 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_items": 57, 2024-04-02T14:04:10.159 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_bytes": 0, 2024-04-02T14:04:10.159 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_items": 0, 2024-04-02T14:04:10.159 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_bytes": 0, 2024-04-02T14:04:10.159 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_items": 0, 2024-04-02T14:04:10.160 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_bytes": 0, 2024-04-02T14:04:10.160 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_items": 0, 2024-04-02T14:04:10.160 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_bytes": 0, 2024-04-02T14:04:10.160 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_items": 0, 2024-04-02T14:04:10.160 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_bytes": 9320, 2024-04-02T14:04:10.160 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_items": 178, 2024-04-02T14:04:10.160 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T14:04:10.160 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_items": 0, 2024-04-02T14:04:10.161 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_bytes": 0, 2024-04-02T14:04:10.161 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_items": 0, 2024-04-02T14:04:10.161 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_bytes": 8464, 2024-04-02T14:04:10.161 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_items": 13, 2024-04-02T14:04:10.161 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_bytes": 0, 2024-04-02T14:04:10.161 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_items": 0, 2024-04-02T14:04:10.161 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_bytes": 0, 2024-04-02T14:04:10.161 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_items": 0 2024-04-02T14:04:10.162 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.162 INFO:teuthology.orchestra.run.smithi052.stdout: "objecter": { 2024-04-02T14:04:10.162 INFO:teuthology.orchestra.run.smithi052.stdout: "op_active": 0, 2024-04-02T14:04:10.162 INFO:teuthology.orchestra.run.smithi052.stdout: "op_laggy": 0, 2024-04-02T14:04:10.162 INFO:teuthology.orchestra.run.smithi052.stdout: "op_send": 2166, 2024-04-02T14:04:10.162 INFO:teuthology.orchestra.run.smithi052.stdout: "op_send_bytes": 21, 2024-04-02T14:04:10.162 INFO:teuthology.orchestra.run.smithi052.stdout: "op_resend": 3, 2024-04-02T14:04:10.162 INFO:teuthology.orchestra.run.smithi052.stdout: "op_reply": 2166, 2024-04-02T14:04:10.163 INFO:teuthology.orchestra.run.smithi052.stdout: "op_latency": { 2024-04-02T14:04:10.163 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 2166, 2024-04-02T14:04:10.163 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 1.960975567, 2024-04-02T14:04:10.163 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000905344 2024-04-02T14:04:10.163 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.163 INFO:teuthology.orchestra.run.smithi052.stdout: "op_inflight": 0, 2024-04-02T14:04:10.163 INFO:teuthology.orchestra.run.smithi052.stdout: "oplen_avg": { 2024-04-02T14:04:10.163 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 2166, 2024-04-02T14:04:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 2167 2024-04-02T14:04:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "op": 2166, 2024-04-02T14:04:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "op_r": 2166, 2024-04-02T14:04:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "op_w": 0, 2024-04-02T14:04:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "op_rmw": 0, 2024-04-02T14:04:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "op_pg": 0, 2024-04-02T14:04:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_stat": 1397, 2024-04-02T14:04:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_create": 0, 2024-04-02T14:04:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_read": 767, 2024-04-02T14:04:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_write": 0, 2024-04-02T14:04:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_writefull": 0, 2024-04-02T14:04:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_writesame": 0, 2024-04-02T14:04:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_append": 0, 2024-04-02T14:04:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_zero": 0, 2024-04-02T14:04:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_truncate": 0, 2024-04-02T14:04:10.166 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_delete": 0, 2024-04-02T14:04:10.166 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_mapext": 0, 2024-04-02T14:04:10.166 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_sparse_read": 0, 2024-04-02T14:04:10.166 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_clonerange": 0, 2024-04-02T14:04:10.166 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_getxattr": 1, 2024-04-02T14:04:10.166 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_setxattr": 0, 2024-04-02T14:04:10.166 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_cmpxattr": 0, 2024-04-02T14:04:10.167 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_rmxattr": 0, 2024-04-02T14:04:10.167 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_resetxattrs": 0, 2024-04-02T14:04:10.167 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_call": 0, 2024-04-02T14:04:10.167 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_watch": 0, 2024-04-02T14:04:10.167 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_notify": 0, 2024-04-02T14:04:10.167 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T14:04:10.167 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_pgls": 0, 2024-04-02T14:04:10.167 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_pgls_filter": 0, 2024-04-02T14:04:10.168 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_other": 0, 2024-04-02T14:04:10.168 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_active": 0, 2024-04-02T14:04:10.168 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_send": 0, 2024-04-02T14:04:10.168 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_resend": 0, 2024-04-02T14:04:10.168 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_ping": 0, 2024-04-02T14:04:10.168 INFO:teuthology.orchestra.run.smithi052.stdout: "poolop_active": 0, 2024-04-02T14:04:10.168 INFO:teuthology.orchestra.run.smithi052.stdout: "poolop_send": 0, 2024-04-02T14:04:10.168 INFO:teuthology.orchestra.run.smithi052.stdout: "poolop_resend": 0, 2024-04-02T14:04:10.169 INFO:teuthology.orchestra.run.smithi052.stdout: "poolstat_active": 0, 2024-04-02T14:04:10.169 INFO:teuthology.orchestra.run.smithi052.stdout: "poolstat_send": 0, 2024-04-02T14:04:10.169 INFO:teuthology.orchestra.run.smithi052.stdout: "poolstat_resend": 0, 2024-04-02T14:04:10.169 INFO:teuthology.orchestra.run.smithi052.stdout: "statfs_active": 0, 2024-04-02T14:04:10.169 INFO:teuthology.orchestra.run.smithi052.stdout: "statfs_send": 0, 2024-04-02T14:04:10.169 INFO:teuthology.orchestra.run.smithi052.stdout: "statfs_resend": 0, 2024-04-02T14:04:10.169 INFO:teuthology.orchestra.run.smithi052.stdout: "command_active": 0, 2024-04-02T14:04:10.169 INFO:teuthology.orchestra.run.smithi052.stdout: "command_send": 0, 2024-04-02T14:04:10.170 INFO:teuthology.orchestra.run.smithi052.stdout: "command_resend": 0, 2024-04-02T14:04:10.170 INFO:teuthology.orchestra.run.smithi052.stdout: "map_epoch": 101, 2024-04-02T14:04:10.170 INFO:teuthology.orchestra.run.smithi052.stdout: "map_full": 0, 2024-04-02T14:04:10.170 INFO:teuthology.orchestra.run.smithi052.stdout: "map_inc": 11, 2024-04-02T14:04:10.170 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_sessions": 6, 2024-04-02T14:04:10.170 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_session_open": 6, 2024-04-02T14:04:10.170 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_session_close": 0, 2024-04-02T14:04:10.170 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_laggy": 0, 2024-04-02T14:04:10.171 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_wr": 0, 2024-04-02T14:04:10.171 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_rd": 2, 2024-04-02T14:04:10.171 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_del": 0 2024-04-02T14:04:10.171 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.171 INFO:teuthology.orchestra.run.smithi052.stdout: "oft": { 2024-04-02T14:04:10.171 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_objs": 0, 2024-04-02T14:04:10.171 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_kv_pairs": 0, 2024-04-02T14:04:10.171 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_updates": 0, 2024-04-02T14:04:10.172 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_removes": 0 2024-04-02T14:04:10.172 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.172 INFO:teuthology.orchestra.run.smithi052.stdout: "osd-slow-ops": { 2024-04-02T14:04:10.172 INFO:teuthology.orchestra.run.smithi052.stdout: "slow_ops_count": 0 2024-04-02T14:04:10.172 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.172 INFO:teuthology.orchestra.run.smithi052.stdout: "purge_queue": { 2024-04-02T14:04:10.172 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing_ops": 0, 2024-04-02T14:04:10.172 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T14:04:10.173 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing": 0, 2024-04-02T14:04:10.173 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing_high_water": 0, 2024-04-02T14:04:10.173 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executed_ops": 0, 2024-04-02T14:04:10.173 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executed": 0, 2024-04-02T14:04:10.173 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_item_in_journal": 0 2024-04-02T14:04:10.173 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.173 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T14:04:10.173 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-04-02T14:04:10.174 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-04-02T14:04:10.174 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-04-02T14:04:10.174 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 2414, 2024-04-02T14:04:10.174 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 2023309, 2024-04-02T14:04:10.174 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:10.174 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 2414, 2024-04-02T14:04:10.174 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-04-02T14:04:10.174 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-04-02T14:04:10.175 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 2414, 2024-04-02T14:04:10.175 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 2023309, 2024-04-02T14:04:10.175 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-04-02T14:04:10.175 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.175 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.175 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.175 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:10.175 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.175 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-objecter_bytes": { 2024-04-02T14:04:10.176 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-04-02T14:04:10.176 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-04-02T14:04:10.176 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-04-02T14:04:10.176 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-04-02T14:04:10.176 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-04-02T14:04:10.176 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:10.176 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:10.176 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 2166, 2024-04-02T14:04:10.177 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 1965, 2024-04-02T14:04:10.177 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 2166, 2024-04-02T14:04:10.177 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 1965, 2024-04-02T14:04:10.177 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-04-02T14:04:10.177 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.177 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.177 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.177 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:10.177 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.178 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-objecter_ops": { 2024-04-02T14:04:10.178 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-04-02T14:04:10.178 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 1024, 2024-04-02T14:04:10.178 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-04-02T14:04:10.178 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-04-02T14:04:10.178 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-04-02T14:04:10.178 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:10.178 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:10.179 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 2166, 2024-04-02T14:04:10.179 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 2166, 2024-04-02T14:04:10.179 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 2166, 2024-04-02T14:04:10.179 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 2166, 2024-04-02T14:04:10.179 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-04-02T14:04:10.179 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.179 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.179 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.180 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:10.180 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.180 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-write_buf_throttle": { 2024-04-02T14:04:10.180 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-04-02T14:04:10.180 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 3758096384, 2024-04-02T14:04:10.180 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-04-02T14:04:10.180 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-04-02T14:04:10.180 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-04-02T14:04:10.181 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:10.181 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:10.181 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-04-02T14:04:10.181 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-04-02T14:04:10.181 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 0, 2024-04-02T14:04:10.181 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 0, 2024-04-02T14:04:10.181 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-04-02T14:04:10.181 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.181 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.182 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.182 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:10.182 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:10.182 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-write_buf_throttle-0x55bc0299a6e0": { 2024-04-02T14:04:10.182 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-04-02T14:04:10.182 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 3758096384, 2024-04-02T14:04:10.182 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-04-02T14:04:10.182 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-04-02T14:04:10.183 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-04-02T14:04:10.183 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:10.183 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:10.183 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-04-02T14:04:10.183 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-04-02T14:04:10.183 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 0, 2024-04-02T14:04:10.183 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 0, 2024-04-02T14:04:10.183 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-04-02T14:04:10.184 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:10.184 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:10.184 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:10.184 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:10.184 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:10.184 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:04:10.877 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-02T14:04:10.877 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-02T14:04:10.877 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-04-02T14:04:10.877 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-04-02T14:04:10.878 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-04-02T14:04:10.878 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-02T14:04:10.878 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T14:04:10.878 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-04-02T14:04:10.878 DEBUG:tasks.check_counter:Getting stats from j 2024-04-02T14:04:10.878 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph daemon mds.j perf dump 2024-04-02T14:04:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T14:04:11.547 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:11 smithi052 ceph-mon[32104]: pgmap v331: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:04:11.547 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:11 smithi052 ceph-mon[32104]: osdmap e102: 12 total, 12 up, 12 in 2024-04-02T14:04:11.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:11 smithi189 ceph-mon[36683]: pgmap v331: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:04:11.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:11 smithi189 ceph-mon[36683]: osdmap e102: 12 total, 12 up, 12 in 2024-04-02T14:04:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:11 smithi088 ceph-mon[36528]: pgmap v331: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-02T14:04:11.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:11 smithi088 ceph-mon[36528]: osdmap e102: 12 total, 12 up, 12 in 2024-04-02T14:04:12.910 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-04-02T14:04:12.910 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T14:04:12.910 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 232, 2024-04-02T14:04:12.910 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 221, 2024-04-02T14:04:12.911 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 1592736, 2024-04-02T14:04:12.911 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 135744, 2024-04-02T14:04:12.911 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 4, 2024-04-02T14:04:12.911 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 0, 2024-04-02T14:04:12.911 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.044817157, 2024-04-02T14:04:12.911 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.014518321, 2024-04-02T14:04:12.911 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.033925719, 2024-04-02T14:04:12.911 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.004596203, 2024-04-02T14:04:12.911 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:12.912 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 220, 2024-04-02T14:04:12.912 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.012257287, 2024-04-02T14:04:12.912 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000055714 2024-04-02T14:04:12.912 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:12.912 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:12.912 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:12.912 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:12.912 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:12.913 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:12.913 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 1592736, 2024-04-02T14:04:12.913 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 135744 2024-04-02T14:04:12.913 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:12.913 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T14:04:12.913 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 6, 2024-04-02T14:04:12.913 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 171, 2024-04-02T14:04:12.913 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 3392, 2024-04-02T14:04:12.914 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 185920, 2024-04-02T14:04:12.914 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 5, 2024-04-02T14:04:12.914 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 3, 2024-04-02T14:04:12.914 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.015335252, 2024-04-02T14:04:12.914 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.010895237, 2024-04-02T14:04:12.914 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.002036863, 2024-04-02T14:04:12.914 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T14:04:12.914 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:12.915 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 169, 2024-04-02T14:04:12.915 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.015366410, 2024-04-02T14:04:12.915 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000090925 2024-04-02T14:04:12.915 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:12.915 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:12.915 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:12.915 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:12.915 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:12.915 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:12.916 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-04-02T14:04:12.916 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 185920 2024-04-02T14:04:12.916 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:12.916 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T14:04:12.916 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 0, 2024-04-02T14:04:12.916 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 2, 2024-04-02T14:04:12.916 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 0, 2024-04-02T14:04:12.916 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 0, 2024-04-02T14:04:12.917 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 2, 2024-04-02T14:04:12.917 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 1, 2024-04-02T14:04:12.917 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.000302715, 2024-04-02T14:04:12.917 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.000000000, 2024-04-02T14:04:12.917 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.000242479, 2024-04-02T14:04:12.917 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T14:04:12.917 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:12.917 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:12.917 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:12.917 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:12.918 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:12.918 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:12.918 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:12.918 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:12.918 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:12.918 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:12.918 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T14:04:12.918 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T14:04:12.919 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:12.919 INFO:teuthology.orchestra.run.smithi052.stdout: "cct": { 2024-04-02T14:04:12.919 INFO:teuthology.orchestra.run.smithi052.stdout: "total_workers": 0, 2024-04-02T14:04:12.919 INFO:teuthology.orchestra.run.smithi052.stdout: "unhealthy_workers": 0 2024-04-02T14:04:12.919 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:12.919 INFO:teuthology.orchestra.run.smithi052.stdout: "mempool": { 2024-04-02T14:04:12.919 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_bytes": 0, 2024-04-02T14:04:12.919 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_items": 0, 2024-04-02T14:04:12.920 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T14:04:12.920 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_items": 0, 2024-04-02T14:04:12.920 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T14:04:12.920 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_items": 0, 2024-04-02T14:04:12.920 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T14:04:12.920 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T14:04:12.920 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T14:04:12.920 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T14:04:12.920 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T14:04:12.921 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_items": 0, 2024-04-02T14:04:12.921 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T14:04:12.921 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T14:04:12.921 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_bytes": 0, 2024-04-02T14:04:12.921 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_items": 0, 2024-04-02T14:04:12.921 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_bytes": 0, 2024-04-02T14:04:12.921 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_items": 0, 2024-04-02T14:04:12.921 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T14:04:12.922 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T14:04:12.922 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T14:04:12.922 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T14:04:12.922 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T14:04:12.922 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_items": 0, 2024-04-02T14:04:12.922 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_bytes": 0, 2024-04-02T14:04:12.922 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_items": 0, 2024-04-02T14:04:12.922 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T14:04:12.923 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T14:04:12.923 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_bytes": 0, 2024-04-02T14:04:12.923 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_items": 0, 2024-04-02T14:04:12.923 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_bytes": 0, 2024-04-02T14:04:12.923 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_items": 0, 2024-04-02T14:04:12.923 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T14:04:12.923 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_items": 0, 2024-04-02T14:04:12.923 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T14:04:12.923 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_items": 0, 2024-04-02T14:04:12.924 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_bytes": 148714, 2024-04-02T14:04:12.924 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_items": 30, 2024-04-02T14:04:12.924 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_bytes": 0, 2024-04-02T14:04:12.924 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_items": 0, 2024-04-02T14:04:12.924 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_bytes": 0, 2024-04-02T14:04:12.924 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_items": 0, 2024-04-02T14:04:12.924 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_bytes": 0, 2024-04-02T14:04:12.924 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_items": 0, 2024-04-02T14:04:12.924 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_bytes": 0, 2024-04-02T14:04:12.925 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_items": 0, 2024-04-02T14:04:12.925 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_bytes": 0, 2024-04-02T14:04:12.925 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_items": 0, 2024-04-02T14:04:12.925 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T14:04:12.925 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_items": 0, 2024-04-02T14:04:12.925 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_bytes": 0, 2024-04-02T14:04:12.925 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_items": 0, 2024-04-02T14:04:12.925 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_bytes": 928, 2024-04-02T14:04:12.925 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_items": 2, 2024-04-02T14:04:12.925 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_bytes": 0, 2024-04-02T14:04:12.926 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_items": 0, 2024-04-02T14:04:12.926 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_bytes": 0, 2024-04-02T14:04:12.926 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_items": 0 2024-04-02T14:04:12.926 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-04-02T14:04:12.926 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T14:04:12.926 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-04-02T14:04:12.926 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-04-02T14:04:12.926 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-04-02T14:04:12.927 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 234, 2024-04-02T14:04:12.927 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 1560006, 2024-04-02T14:04:12.927 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:12.927 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 234, 2024-04-02T14:04:12.927 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-04-02T14:04:12.927 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-04-02T14:04:12.927 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 234, 2024-04-02T14:04:12.927 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 1560006, 2024-04-02T14:04:12.927 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-04-02T14:04:12.928 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-04-02T14:04:12.928 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-04-02T14:04:12.928 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-04-02T14:04:12.928 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:12.928 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-04-02T14:04:12.928 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-04-02T14:04:13.573 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-02T14:04:13.574 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-02T14:04:13.574 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-02T14:04:13.574 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-02T14:04:13.574 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-02T14:04:13.574 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-02T14:04:13.574 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-02T14:04:13.575 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-04-02T14:04:13.575 DEBUG:tasks.check_counter:Getting stats from b 2024-04-02T14:04:13.575 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph daemon mds.b perf dump 2024-04-02T14:04:13.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:13 smithi189 ceph-mon[36683]: pgmap v333: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:04:13.828 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T14:04:13.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:13 smithi052 ceph-mon[32104]: pgmap v333: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:04:13.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:13 smithi088 ceph-mon[36528]: pgmap v333: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:04:15.574 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:15 smithi088 ceph-mon[36528]: pgmap v334: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:04:15.703 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-02T14:04:15.704 INFO:teuthology.orchestra.run.smithi088.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T14:04:15.704 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_messages": 25114, 2024-04-02T14:04:15.704 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages": 25093, 2024-04-02T14:04:15.704 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_bytes": 6416073, 2024-04-02T14:04:15.704 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_bytes": 23751371, 2024-04-02T14:04:15.705 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_created_connections": 169, 2024-04-02T14:04:15.705 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_active_connections": 8, 2024-04-02T14:04:15.705 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_total_time": 1.217340416, 2024-04-02T14:04:15.705 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_send_time": 0.376992297, 2024-04-02T14:04:15.705 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_recv_time": 0.318189429, 2024-04-02T14:04:15.705 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_fast_dispatch_time": 0.418309768, 2024-04-02T14:04:15.705 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:15.706 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 25081, 2024-04-02T14:04:15.706 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 1.286855751, 2024-04-02T14:04:15.706 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000051307 2024-04-02T14:04:15.706 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.706 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:15.706 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 448, 2024-04-02T14:04:15.706 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000061875, 2024-04-02T14:04:15.707 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000138 2024-04-02T14:04:15.707 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.707 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_encrypted_bytes": 100064, 2024-04-02T14:04:15.707 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_encrypted_bytes": 354832 2024-04-02T14:04:15.707 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.707 INFO:teuthology.orchestra.run.smithi088.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T14:04:15.707 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_messages": 35902, 2024-04-02T14:04:15.707 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages": 54971, 2024-04-02T14:04:15.708 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_bytes": 10681156, 2024-04-02T14:04:15.708 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_bytes": 37885938, 2024-04-02T14:04:15.708 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_created_connections": 11, 2024-04-02T14:04:15.708 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_active_connections": 5, 2024-04-02T14:04:15.708 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_total_time": 1.969496096, 2024-04-02T14:04:15.708 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_send_time": 0.833983882, 2024-04-02T14:04:15.708 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_recv_time": 1.158918028, 2024-04-02T14:04:15.709 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_fast_dispatch_time": 0.359996008, 2024-04-02T14:04:15.709 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:15.709 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 54975, 2024-04-02T14:04:15.709 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 1.625552763, 2024-04-02T14:04:15.709 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000029568 2024-04-02T14:04:15.709 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.709 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:15.709 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 38470, 2024-04-02T14:04:15.709 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.007177631, 2024-04-02T14:04:15.710 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000186 2024-04-02T14:04:15.710 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.710 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_encrypted_bytes": 42720, 2024-04-02T14:04:15.710 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_encrypted_bytes": 140512 2024-04-02T14:04:15.710 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.710 INFO:teuthology.orchestra.run.smithi088.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T14:04:15.710 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_messages": 24909, 2024-04-02T14:04:15.711 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages": 24767, 2024-04-02T14:04:15.711 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_bytes": 8317569, 2024-04-02T14:04:15.711 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_bytes": 20464334, 2024-04-02T14:04:15.711 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_created_connections": 16, 2024-04-02T14:04:15.711 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_active_connections": 6, 2024-04-02T14:04:15.711 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_total_time": 1.489208414, 2024-04-02T14:04:15.711 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_send_time": 0.535264536, 2024-04-02T14:04:15.711 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_recv_time": 0.421561106, 2024-04-02T14:04:15.711 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_fast_dispatch_time": 0.463812651, 2024-04-02T14:04:15.712 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:15.712 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 24770, 2024-04-02T14:04:15.712 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.685215562, 2024-04-02T14:04:15.712 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000027663 2024-04-02T14:04:15.712 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.712 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:15.712 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 5025, 2024-04-02T14:04:15.712 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000835213, 2024-04-02T14:04:15.712 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000166 2024-04-02T14:04:15.712 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.713 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_encrypted_bytes": 2055168, 2024-04-02T14:04:15.713 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_encrypted_bytes": 265328 2024-04-02T14:04:15.713 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.713 INFO:teuthology.orchestra.run.smithi088.stdout: "cct": { 2024-04-02T14:04:15.713 INFO:teuthology.orchestra.run.smithi088.stdout: "total_workers": 1, 2024-04-02T14:04:15.713 INFO:teuthology.orchestra.run.smithi088.stdout: "unhealthy_workers": 0 2024-04-02T14:04:15.713 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.713 INFO:teuthology.orchestra.run.smithi088.stdout: "finisher-MDSRank": { 2024-04-02T14:04:15.713 INFO:teuthology.orchestra.run.smithi088.stdout: "queue_len": 0, 2024-04-02T14:04:15.714 INFO:teuthology.orchestra.run.smithi088.stdout: "complete_latency": { 2024-04-02T14:04:15.714 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 36769, 2024-04-02T14:04:15.714 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 19.766052901, 2024-04-02T14:04:15.714 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000537573 2024-04-02T14:04:15.714 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:15.714 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.714 INFO:teuthology.orchestra.run.smithi088.stdout: "finisher-PurgeQueue": { 2024-04-02T14:04:15.714 INFO:teuthology.orchestra.run.smithi088.stdout: "queue_len": 0, 2024-04-02T14:04:15.715 INFO:teuthology.orchestra.run.smithi088.stdout: "complete_latency": { 2024-04-02T14:04:15.715 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 5710, 2024-04-02T14:04:15.715 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 2.014101115, 2024-04-02T14:04:15.715 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000352732 2024-04-02T14:04:15.715 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:15.715 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.715 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-04-02T14:04:15.715 INFO:teuthology.orchestra.run.smithi088.stdout: "request": 10867, 2024-04-02T14:04:15.716 INFO:teuthology.orchestra.run.smithi088.stdout: "reply": 10867, 2024-04-02T14:04:15.716 INFO:teuthology.orchestra.run.smithi088.stdout: "reply_latency": { 2024-04-02T14:04:15.716 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 10867, 2024-04-02T14:04:15.716 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 36.533995592, 2024-04-02T14:04:15.716 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.003361921 2024-04-02T14:04:15.716 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.716 INFO:teuthology.orchestra.run.smithi088.stdout: "slow_reply": 0, 2024-04-02T14:04:15.716 INFO:teuthology.orchestra.run.smithi088.stdout: "forward": 0, 2024-04-02T14:04:15.716 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_fetch_complete": 0, 2024-04-02T14:04:15.717 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_fetch_keys": 0, 2024-04-02T14:04:15.717 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_commit": 194, 2024-04-02T14:04:15.717 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_split": 22, 2024-04-02T14:04:15.717 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_merge": 0, 2024-04-02T14:04:15.717 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes": 5065, 2024-04-02T14:04:15.717 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_top": 238, 2024-04-02T14:04:15.717 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_bottom": 4777, 2024-04-02T14:04:15.717 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_pin_tail": 50, 2024-04-02T14:04:15.717 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_pinned": 4724, 2024-04-02T14:04:15.718 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_expired": 16, 2024-04-02T14:04:15.718 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_with_caps": 3511, 2024-04-02T14:04:15.718 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": 3511, 2024-04-02T14:04:15.718 INFO:teuthology.orchestra.run.smithi088.stdout: "subtrees": 5, 2024-04-02T14:04:15.718 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse": 11211, 2024-04-02T14:04:15.718 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_hit": 10554, 2024-04-02T14:04:15.718 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_forward": 0, 2024-04-02T14:04:15.718 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_discover": 0, 2024-04-02T14:04:15.719 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_dir_fetch": 0, 2024-04-02T14:04:15.719 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_remote_ino": 0, 2024-04-02T14:04:15.719 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_lock": 1, 2024-04-02T14:04:15.719 INFO:teuthology.orchestra.run.smithi088.stdout: "load_cent": 0, 2024-04-02T14:04:15.719 INFO:teuthology.orchestra.run.smithi088.stdout: "q": 0, 2024-04-02T14:04:15.719 INFO:teuthology.orchestra.run.smithi088.stdout: "exported": 2, 2024-04-02T14:04:15.719 INFO:teuthology.orchestra.run.smithi088.stdout: "exported_inodes": 12, 2024-04-02T14:04:15.719 INFO:teuthology.orchestra.run.smithi088.stdout: "imported": 0, 2024-04-02T14:04:15.720 INFO:teuthology.orchestra.run.smithi088.stdout: "imported_inodes": 0, 2024-04-02T14:04:15.720 INFO:teuthology.orchestra.run.smithi088.stdout: "openino_dir_fetch": 0, 2024-04-02T14:04:15.720 INFO:teuthology.orchestra.run.smithi088.stdout: "openino_backtrace_fetch": 53435, 2024-04-02T14:04:15.720 INFO:teuthology.orchestra.run.smithi088.stdout: "openino_peer_discover": 0, 2024-04-02T14:04:15.720 INFO:teuthology.orchestra.run.smithi088.stdout: "root_rfiles": 3452, 2024-04-02T14:04:15.720 INFO:teuthology.orchestra.run.smithi088.stdout: "root_rbytes": 26601218, 2024-04-02T14:04:15.720 INFO:teuthology.orchestra.run.smithi088.stdout: "root_rsnaps": 6, 2024-04-02T14:04:15.720 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_backtrace_fetch": 53435, 2024-04-02T14:04:15.721 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_set_tag": 0, 2024-04-02T14:04:15.721 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T14:04:15.721 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_inotable_repaired": 0, 2024-04-02T14:04:15.721 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_dir_inodes": 1174, 2024-04-02T14:04:15.721 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_dir_base_inodes": 19, 2024-04-02T14:04:15.721 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_dirfrag_rstats": 1155, 2024-04-02T14:04:15.721 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_file_inodes": 52261, 2024-04-02T14:04:15.721 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_inode_file_caps": 1, 2024-04-02T14:04:15.722 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_revoke": 49, 2024-04-02T14:04:15.722 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_grant": 10369, 2024-04-02T14:04:15.722 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_trunc": 8, 2024-04-02T14:04:15.722 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T14:04:15.722 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T14:04:15.722 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_caps": 7598, 2024-04-02T14:04:15.722 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_caps_dirty": 6623, 2024-04-02T14:04:15.722 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_cap_release": 353, 2024-04-02T14:04:15.723 INFO:teuthology.orchestra.run.smithi088.stdout: "process_request_cap_release": 3546 2024-04-02T14:04:15.723 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.723 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_cache": { 2024-04-02T14:04:15.723 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_update": 101, 2024-04-02T14:04:15.723 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_update_receipt": 0, 2024-04-02T14:04:15.723 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_try_discover": 0, 2024-04-02T14:04:15.723 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_send_discover": 0, 2024-04-02T14:04:15.724 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_handle_discover": 185, 2024-04-02T14:04:15.724 INFO:teuthology.orchestra.run.smithi088.stdout: "num_strays": 1, 2024-04-02T14:04:15.724 INFO:teuthology.orchestra.run.smithi088.stdout: "num_strays_delayed": 0, 2024-04-02T14:04:15.724 INFO:teuthology.orchestra.run.smithi088.stdout: "num_strays_enqueuing": 0, 2024-04-02T14:04:15.724 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_created": 3236, 2024-04-02T14:04:15.724 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_enqueued": 3233, 2024-04-02T14:04:15.724 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_reintegrated": 2, 2024-04-02T14:04:15.724 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_migrated": 0, 2024-04-02T14:04:15.725 INFO:teuthology.orchestra.run.smithi088.stdout: "num_recovering_processing": 0, 2024-04-02T14:04:15.725 INFO:teuthology.orchestra.run.smithi088.stdout: "num_recovering_enqueued": 0, 2024-04-02T14:04:15.725 INFO:teuthology.orchestra.run.smithi088.stdout: "num_recovering_prioritized": 0, 2024-04-02T14:04:15.725 INFO:teuthology.orchestra.run.smithi088.stdout: "recovery_started": 0, 2024-04-02T14:04:15.725 INFO:teuthology.orchestra.run.smithi088.stdout: "recovery_completed": 0, 2024-04-02T14:04:15.725 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_quiesce_path": 0, 2024-04-02T14:04:15.725 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_quiesce_inode": 0, 2024-04-02T14:04:15.725 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_enqueue_scrub": 19, 2024-04-02T14:04:15.726 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_exportdir": 2, 2024-04-02T14:04:15.726 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_flush": 0, 2024-04-02T14:04:15.726 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_fragmentdir": 22, 2024-04-02T14:04:15.726 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_fragstats": 0, 2024-04-02T14:04:15.726 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_inodestats": 0 2024-04-02T14:04:15.726 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.726 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_log": { 2024-04-02T14:04:15.727 INFO:teuthology.orchestra.run.smithi088.stdout: "evlrg": 0, 2024-04-02T14:04:15.727 INFO:teuthology.orchestra.run.smithi088.stdout: "evadd": 22725, 2024-04-02T14:04:15.727 INFO:teuthology.orchestra.run.smithi088.stdout: "evex": 0, 2024-04-02T14:04:15.727 INFO:teuthology.orchestra.run.smithi088.stdout: "evtrm": 0, 2024-04-02T14:04:15.727 INFO:teuthology.orchestra.run.smithi088.stdout: "ev": 22725, 2024-04-02T14:04:15.727 INFO:teuthology.orchestra.run.smithi088.stdout: "evexg": 0, 2024-04-02T14:04:15.727 INFO:teuthology.orchestra.run.smithi088.stdout: "evexd": 0, 2024-04-02T14:04:15.727 INFO:teuthology.orchestra.run.smithi088.stdout: "segadd": 27, 2024-04-02T14:04:15.728 INFO:teuthology.orchestra.run.smithi088.stdout: "segex": 0, 2024-04-02T14:04:15.728 INFO:teuthology.orchestra.run.smithi088.stdout: "segtrm": 0, 2024-04-02T14:04:15.728 INFO:teuthology.orchestra.run.smithi088.stdout: "seg": 27, 2024-04-02T14:04:15.728 INFO:teuthology.orchestra.run.smithi088.stdout: "segmjr": 3, 2024-04-02T14:04:15.728 INFO:teuthology.orchestra.run.smithi088.stdout: "segexg": 0, 2024-04-02T14:04:15.728 INFO:teuthology.orchestra.run.smithi088.stdout: "segexd": 0, 2024-04-02T14:04:15.728 INFO:teuthology.orchestra.run.smithi088.stdout: "expos": 4194304, 2024-04-02T14:04:15.728 INFO:teuthology.orchestra.run.smithi088.stdout: "wrpos": 52262247, 2024-04-02T14:04:15.728 INFO:teuthology.orchestra.run.smithi088.stdout: "rdpos": 0, 2024-04-02T14:04:15.729 INFO:teuthology.orchestra.run.smithi088.stdout: "jlat": { 2024-04-02T14:04:15.729 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 427, 2024-04-02T14:04:15.729 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 19.048538226, 2024-04-02T14:04:15.729 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.044610159 2024-04-02T14:04:15.729 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.729 INFO:teuthology.orchestra.run.smithi088.stdout: "replayed": 0 2024-04-02T14:04:15.730 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.730 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_mem": { 2024-04-02T14:04:15.730 INFO:teuthology.orchestra.run.smithi088.stdout: "ino": 3582, 2024-04-02T14:04:15.730 INFO:teuthology.orchestra.run.smithi088.stdout: "ino+": 6842, 2024-04-02T14:04:15.730 INFO:teuthology.orchestra.run.smithi088.stdout: "ino-": 3260, 2024-04-02T14:04:15.730 INFO:teuthology.orchestra.run.smithi088.stdout: "dir": 240, 2024-04-02T14:04:15.730 INFO:teuthology.orchestra.run.smithi088.stdout: "dir+": 273, 2024-04-02T14:04:15.731 INFO:teuthology.orchestra.run.smithi088.stdout: "dir-": 33, 2024-04-02T14:04:15.731 INFO:teuthology.orchestra.run.smithi088.stdout: "dn": 5065, 2024-04-02T14:04:15.731 INFO:teuthology.orchestra.run.smithi088.stdout: "dn+": 8486, 2024-04-02T14:04:15.731 INFO:teuthology.orchestra.run.smithi088.stdout: "dn-": 3421, 2024-04-02T14:04:15.731 INFO:teuthology.orchestra.run.smithi088.stdout: "cap": 3511, 2024-04-02T14:04:15.731 INFO:teuthology.orchestra.run.smithi088.stdout: "cap+": 6768, 2024-04-02T14:04:15.731 INFO:teuthology.orchestra.run.smithi088.stdout: "cap-": 3257, 2024-04-02T14:04:15.732 INFO:teuthology.orchestra.run.smithi088.stdout: "rss": 96856, 2024-04-02T14:04:15.732 INFO:teuthology.orchestra.run.smithi088.stdout: "heap": 231800 2024-04-02T14:04:15.732 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.732 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_server": { 2024-04-02T14:04:15.732 INFO:teuthology.orchestra.run.smithi088.stdout: "dispatch_client_request": 11199, 2024-04-02T14:04:15.732 INFO:teuthology.orchestra.run.smithi088.stdout: "dispatch_server_request": 1, 2024-04-02T14:04:15.732 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_request": 10867, 2024-04-02T14:04:15.732 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_session": 43, 2024-04-02T14:04:15.733 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_peer_request": 2, 2024-04-02T14:04:15.733 INFO:teuthology.orchestra.run.smithi088.stdout: "req_create_latency": { 2024-04-02T14:04:15.733 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 6669, 2024-04-02T14:04:15.733 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 24.564366956, 2024-04-02T14:04:15.733 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.003683365 2024-04-02T14:04:15.733 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.733 INFO:teuthology.orchestra.run.smithi088.stdout: "req_getattr_latency": { 2024-04-02T14:04:15.733 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 123, 2024-04-02T14:04:15.733 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.032229944, 2024-04-02T14:04:15.734 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000262032 2024-04-02T14:04:15.734 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.734 INFO:teuthology.orchestra.run.smithi088.stdout: "req_getfilelock_latency": { 2024-04-02T14:04:15.734 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:15.734 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:15.734 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:15.734 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.735 INFO:teuthology.orchestra.run.smithi088.stdout: "req_link_latency": { 2024-04-02T14:04:15.735 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 2, 2024-04-02T14:04:15.735 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.002558914, 2024-04-02T14:04:15.735 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.001279457 2024-04-02T14:04:15.735 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.735 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookup_latency": { 2024-04-02T14:04:15.735 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 98, 2024-04-02T14:04:15.735 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.033852339, 2024-04-02T14:04:15.735 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000345432 2024-04-02T14:04:15.736 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.736 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookuphash_latency": { 2024-04-02T14:04:15.736 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:15.736 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:15.736 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:15.736 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.736 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupino_latency": { 2024-04-02T14:04:15.737 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:15.737 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:15.737 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:15.737 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.737 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupname_latency": { 2024-04-02T14:04:15.737 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:15.737 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:15.737 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:15.738 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.738 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupparent_latency": { 2024-04-02T14:04:15.738 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:15.738 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:15.738 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:15.738 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.738 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupsnap_latency": { 2024-04-02T14:04:15.738 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:15.739 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:15.739 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:15.739 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.739 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lssnap_latency": { 2024-04-02T14:04:15.739 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 11, 2024-04-02T14:04:15.740 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.001766074, 2024-04-02T14:04:15.740 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000160552 2024-04-02T14:04:15.740 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.740 INFO:teuthology.orchestra.run.smithi088.stdout: "req_mkdir_latency": { 2024-04-02T14:04:15.740 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 85, 2024-04-02T14:04:15.740 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 1.082533900, 2024-04-02T14:04:15.740 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.012735692 2024-04-02T14:04:15.740 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.740 INFO:teuthology.orchestra.run.smithi088.stdout: "req_mknod_latency": { 2024-04-02T14:04:15.741 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:15.741 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:15.741 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:15.741 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.741 INFO:teuthology.orchestra.run.smithi088.stdout: "req_mksnap_latency": { 2024-04-02T14:04:15.741 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 11, 2024-04-02T14:04:15.741 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.057177894, 2024-04-02T14:04:15.741 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.005197990 2024-04-02T14:04:15.742 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.742 INFO:teuthology.orchestra.run.smithi088.stdout: "req_open_latency": { 2024-04-02T14:04:15.742 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 442, 2024-04-02T14:04:15.742 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.099899361, 2024-04-02T14:04:15.742 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000226016 2024-04-02T14:04:15.742 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.742 INFO:teuthology.orchestra.run.smithi088.stdout: "req_readdir_latency": { 2024-04-02T14:04:15.742 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 107, 2024-04-02T14:04:15.743 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.809482775, 2024-04-02T14:04:15.743 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.007565259 2024-04-02T14:04:15.743 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.743 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rename_latency": { 2024-04-02T14:04:15.743 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 81, 2024-04-02T14:04:15.743 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.167423524, 2024-04-02T14:04:15.743 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.002066957 2024-04-02T14:04:15.743 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.743 INFO:teuthology.orchestra.run.smithi088.stdout: "req_renamesnap_latency": { 2024-04-02T14:04:15.744 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:15.744 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:15.744 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:15.744 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.744 INFO:teuthology.orchestra.run.smithi088.stdout: "req_snapdiff_latency": { 2024-04-02T14:04:15.744 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:15.744 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:15.744 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:15.745 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.745 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rmdir_latency": { 2024-04-02T14:04:15.745 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 22, 2024-04-02T14:04:15.745 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.854716166, 2024-04-02T14:04:15.745 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.038850734 2024-04-02T14:04:15.745 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.745 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rmsnap_latency": { 2024-04-02T14:04:15.745 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 5, 2024-04-02T14:04:15.745 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.022290747, 2024-04-02T14:04:15.746 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.004458149 2024-04-02T14:04:15.746 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.746 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rmxattr_latency": { 2024-04-02T14:04:15.746 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:15.746 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:15.746 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:15.746 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.746 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setattr_latency": { 2024-04-02T14:04:15.747 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 10, 2024-04-02T14:04:15.747 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.064076286, 2024-04-02T14:04:15.747 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.006407628 2024-04-02T14:04:15.747 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.747 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setdirlayout_latency": { 2024-04-02T14:04:15.747 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:15.747 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:15.747 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:15.748 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.748 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setfilelock_latency": { 2024-04-02T14:04:15.748 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 8, 2024-04-02T14:04:15.748 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.003106442, 2024-04-02T14:04:15.748 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000388305 2024-04-02T14:04:15.748 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.748 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setlayout_latency": { 2024-04-02T14:04:15.749 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:15.749 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:15.749 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:15.749 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.749 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setxattr_latency": { 2024-04-02T14:04:15.749 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 9, 2024-04-02T14:04:15.749 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.021305162, 2024-04-02T14:04:15.749 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.002367240 2024-04-02T14:04:15.750 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.750 INFO:teuthology.orchestra.run.smithi088.stdout: "req_symlink_latency": { 2024-04-02T14:04:15.750 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 7, 2024-04-02T14:04:15.750 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.004925093, 2024-04-02T14:04:15.750 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000703584 2024-04-02T14:04:15.750 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.750 INFO:teuthology.orchestra.run.smithi088.stdout: "req_unlink_latency": { 2024-04-02T14:04:15.750 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 3177, 2024-04-02T14:04:15.751 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 8.712284015, 2024-04-02T14:04:15.751 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.002742299 2024-04-02T14:04:15.751 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.751 INFO:teuthology.orchestra.run.smithi088.stdout: "cap_revoke_eviction": 0, 2024-04-02T14:04:15.751 INFO:teuthology.orchestra.run.smithi088.stdout: "cap_acquisition_throttle": 0, 2024-04-02T14:04:15.751 INFO:teuthology.orchestra.run.smithi088.stdout: "req_getvxattr_latency": { 2024-04-02T14:04:15.751 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:15.751 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:15.751 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:15.752 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:15.752 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.752 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_sessions": { 2024-04-02T14:04:15.752 INFO:teuthology.orchestra.run.smithi088.stdout: "session_count": 2, 2024-04-02T14:04:15.752 INFO:teuthology.orchestra.run.smithi088.stdout: "session_add": 7, 2024-04-02T14:04:15.752 INFO:teuthology.orchestra.run.smithi088.stdout: "session_remove": 5, 2024-04-02T14:04:15.752 INFO:teuthology.orchestra.run.smithi088.stdout: "sessions_open": 2, 2024-04-02T14:04:15.752 INFO:teuthology.orchestra.run.smithi088.stdout: "sessions_stale": 0, 2024-04-02T14:04:15.753 INFO:teuthology.orchestra.run.smithi088.stdout: "total_load": 58, 2024-04-02T14:04:15.753 INFO:teuthology.orchestra.run.smithi088.stdout: "average_load": 29, 2024-04-02T14:04:15.753 INFO:teuthology.orchestra.run.smithi088.stdout: "avg_session_uptime": 630, 2024-04-02T14:04:15.753 INFO:teuthology.orchestra.run.smithi088.stdout: "mdthresh_evicted": 0 2024-04-02T14:04:15.753 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.753 INFO:teuthology.orchestra.run.smithi088.stdout: "mempool": { 2024-04-02T14:04:15.753 INFO:teuthology.orchestra.run.smithi088.stdout: "bloom_filter_bytes": 0, 2024-04-02T14:04:15.753 INFO:teuthology.orchestra.run.smithi088.stdout: "bloom_filter_items": 0, 2024-04-02T14:04:15.754 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T14:04:15.754 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_alloc_items": 0, 2024-04-02T14:04:15.754 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T14:04:15.754 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_data_items": 0, 2024-04-02T14:04:15.754 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T14:04:15.754 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T14:04:15.754 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T14:04:15.754 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T14:04:15.755 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T14:04:15.755 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_other_items": 0, 2024-04-02T14:04:15.755 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T14:04:15.755 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T14:04:15.755 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_extent_bytes": 0, 2024-04-02T14:04:15.755 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_extent_items": 0, 2024-04-02T14:04:15.755 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_blob_bytes": 0, 2024-04-02T14:04:15.755 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_blob_items": 0, 2024-04-02T14:04:15.756 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T14:04:15.756 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T14:04:15.756 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T14:04:15.756 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T14:04:15.756 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T14:04:15.756 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_fsck_items": 0, 2024-04-02T14:04:15.756 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_txc_bytes": 0, 2024-04-02T14:04:15.756 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_txc_items": 0, 2024-04-02T14:04:15.756 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T14:04:15.757 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T14:04:15.757 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_bytes": 0, 2024-04-02T14:04:15.757 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_items": 0, 2024-04-02T14:04:15.757 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_bytes": 0, 2024-04-02T14:04:15.757 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_items": 0, 2024-04-02T14:04:15.757 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T14:04:15.757 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_reader_items": 0, 2024-04-02T14:04:15.757 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T14:04:15.758 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_writer_items": 0, 2024-04-02T14:04:15.758 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_anon_bytes": 444343, 2024-04-02T14:04:15.758 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_anon_items": 3647, 2024-04-02T14:04:15.758 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_meta_bytes": 1320, 2024-04-02T14:04:15.758 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_meta_items": 15, 2024-04-02T14:04:15.758 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_bytes": 0, 2024-04-02T14:04:15.758 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_items": 0, 2024-04-02T14:04:15.758 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_mapbl_bytes": 0, 2024-04-02T14:04:15.759 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_mapbl_items": 0, 2024-04-02T14:04:15.759 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_pglog_bytes": 0, 2024-04-02T14:04:15.759 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_pglog_items": 0, 2024-04-02T14:04:15.759 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_bytes": 9560, 2024-04-02T14:04:15.759 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_items": 184, 2024-04-02T14:04:15.759 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T14:04:15.759 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_mapping_items": 0, 2024-04-02T14:04:15.759 INFO:teuthology.orchestra.run.smithi088.stdout: "pgmap_bytes": 0, 2024-04-02T14:04:15.760 INFO:teuthology.orchestra.run.smithi088.stdout: "pgmap_items": 0, 2024-04-02T14:04:15.760 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_co_bytes": 13857227, 2024-04-02T14:04:15.760 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_co_items": 159385, 2024-04-02T14:04:15.760 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_1_bytes": 0, 2024-04-02T14:04:15.760 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_1_items": 0, 2024-04-02T14:04:15.760 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_2_bytes": 0, 2024-04-02T14:04:15.760 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_2_items": 0 2024-04-02T14:04:15.761 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.761 INFO:teuthology.orchestra.run.smithi088.stdout: "objecter": { 2024-04-02T14:04:15.761 INFO:teuthology.orchestra.run.smithi088.stdout: "op_active": 0, 2024-04-02T14:04:15.761 INFO:teuthology.orchestra.run.smithi088.stdout: "op_laggy": 0, 2024-04-02T14:04:15.761 INFO:teuthology.orchestra.run.smithi088.stdout: "op_send": 61330, 2024-04-02T14:04:15.761 INFO:teuthology.orchestra.run.smithi088.stdout: "op_send_bytes": 51767556, 2024-04-02T14:04:15.761 INFO:teuthology.orchestra.run.smithi088.stdout: "op_resend": 31, 2024-04-02T14:04:15.761 INFO:teuthology.orchestra.run.smithi088.stdout: "op_reply": 61330, 2024-04-02T14:04:15.761 INFO:teuthology.orchestra.run.smithi088.stdout: "op_latency": { 2024-04-02T14:04:15.762 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 61330, 2024-04-02T14:04:15.762 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 60.859785539, 2024-04-02T14:04:15.762 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000992333 2024-04-02T14:04:15.762 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.762 INFO:teuthology.orchestra.run.smithi088.stdout: "op_inflight": 0, 2024-04-02T14:04:15.762 INFO:teuthology.orchestra.run.smithi088.stdout: "oplen_avg": { 2024-04-02T14:04:15.762 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 61330, 2024-04-02T14:04:15.762 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 61617 2024-04-02T14:04:15.762 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.763 INFO:teuthology.orchestra.run.smithi088.stdout: "op": 61330, 2024-04-02T14:04:15.763 INFO:teuthology.orchestra.run.smithi088.stdout: "op_r": 53526, 2024-04-02T14:04:15.763 INFO:teuthology.orchestra.run.smithi088.stdout: "op_w": 7804, 2024-04-02T14:04:15.763 INFO:teuthology.orchestra.run.smithi088.stdout: "op_rmw": 0, 2024-04-02T14:04:15.763 INFO:teuthology.orchestra.run.smithi088.stdout: "op_pg": 0, 2024-04-02T14:04:15.763 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_stat": 0, 2024-04-02T14:04:15.763 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_create": 11, 2024-04-02T14:04:15.763 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_read": 72, 2024-04-02T14:04:15.763 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_write": 480, 2024-04-02T14:04:15.764 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_writefull": 53, 2024-04-02T14:04:15.764 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_writesame": 0, 2024-04-02T14:04:15.764 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_append": 0, 2024-04-02T14:04:15.764 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_zero": 0, 2024-04-02T14:04:15.764 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_truncate": 6, 2024-04-02T14:04:15.764 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_delete": 6991, 2024-04-02T14:04:15.764 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_mapext": 0, 2024-04-02T14:04:15.764 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_sparse_read": 0, 2024-04-02T14:04:15.764 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_clonerange": 0, 2024-04-02T14:04:15.765 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_getxattr": 53454, 2024-04-02T14:04:15.765 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_setxattr": 22, 2024-04-02T14:04:15.765 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_cmpxattr": 0, 2024-04-02T14:04:15.765 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_rmxattr": 0, 2024-04-02T14:04:15.765 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_resetxattrs": 0, 2024-04-02T14:04:15.765 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_call": 0, 2024-04-02T14:04:15.765 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_watch": 0, 2024-04-02T14:04:15.765 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_notify": 0, 2024-04-02T14:04:15.766 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T14:04:15.766 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_pgls": 0, 2024-04-02T14:04:15.766 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_pgls_filter": 0, 2024-04-02T14:04:15.766 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_other": 8, 2024-04-02T14:04:15.766 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_active": 0, 2024-04-02T14:04:15.766 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_send": 0, 2024-04-02T14:04:15.766 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_resend": 0, 2024-04-02T14:04:15.767 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_ping": 0, 2024-04-02T14:04:15.767 INFO:teuthology.orchestra.run.smithi088.stdout: "poolop_active": 0, 2024-04-02T14:04:15.767 INFO:teuthology.orchestra.run.smithi088.stdout: "poolop_send": 0, 2024-04-02T14:04:15.767 INFO:teuthology.orchestra.run.smithi088.stdout: "poolop_resend": 0, 2024-04-02T14:04:15.767 INFO:teuthology.orchestra.run.smithi088.stdout: "poolstat_active": 0, 2024-04-02T14:04:15.767 INFO:teuthology.orchestra.run.smithi088.stdout: "poolstat_send": 0, 2024-04-02T14:04:15.767 INFO:teuthology.orchestra.run.smithi088.stdout: "poolstat_resend": 0, 2024-04-02T14:04:15.767 INFO:teuthology.orchestra.run.smithi088.stdout: "statfs_active": 0, 2024-04-02T14:04:15.767 INFO:teuthology.orchestra.run.smithi088.stdout: "statfs_send": 0, 2024-04-02T14:04:15.768 INFO:teuthology.orchestra.run.smithi088.stdout: "statfs_resend": 0, 2024-04-02T14:04:15.768 INFO:teuthology.orchestra.run.smithi088.stdout: "command_active": 0, 2024-04-02T14:04:15.768 INFO:teuthology.orchestra.run.smithi088.stdout: "command_send": 0, 2024-04-02T14:04:15.768 INFO:teuthology.orchestra.run.smithi088.stdout: "command_resend": 0, 2024-04-02T14:04:15.768 INFO:teuthology.orchestra.run.smithi088.stdout: "map_epoch": 103, 2024-04-02T14:04:15.768 INFO:teuthology.orchestra.run.smithi088.stdout: "map_full": 0, 2024-04-02T14:04:15.768 INFO:teuthology.orchestra.run.smithi088.stdout: "map_inc": 18, 2024-04-02T14:04:15.768 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_sessions": 12, 2024-04-02T14:04:15.768 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_session_open": 12, 2024-04-02T14:04:15.769 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_session_close": 0, 2024-04-02T14:04:15.769 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_laggy": 0, 2024-04-02T14:04:15.769 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_wr": 486, 2024-04-02T14:04:15.769 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_rd": 0, 2024-04-02T14:04:15.769 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_del": 34 2024-04-02T14:04:15.769 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.769 INFO:teuthology.orchestra.run.smithi088.stdout: "oft": { 2024-04-02T14:04:15.769 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_objs": 1, 2024-04-02T14:04:15.769 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_kv_pairs": 0, 2024-04-02T14:04:15.770 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_updates": 0, 2024-04-02T14:04:15.770 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_removes": 0 2024-04-02T14:04:15.770 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.770 INFO:teuthology.orchestra.run.smithi088.stdout: "osd-slow-ops": { 2024-04-02T14:04:15.770 INFO:teuthology.orchestra.run.smithi088.stdout: "slow_ops_count": 0 2024-04-02T14:04:15.770 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.770 INFO:teuthology.orchestra.run.smithi088.stdout: "purge_queue": { 2024-04-02T14:04:15.770 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing_ops": 0, 2024-04-02T14:04:15.770 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing_ops_high_water": 10, 2024-04-02T14:04:15.771 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing": 0, 2024-04-02T14:04:15.771 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing_high_water": 6, 2024-04-02T14:04:15.771 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executed_ops": 6452, 2024-04-02T14:04:15.771 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executed": 3233, 2024-04-02T14:04:15.771 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_item_in_journal": 0 2024-04-02T14:04:15.771 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.771 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T14:04:15.771 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-02T14:04:15.771 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 104857600, 2024-04-02T14:04:15.772 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-02T14:04:15.772 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 85921, 2024-04-02T14:04:15.772 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 17917369, 2024-04-02T14:04:15.772 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:15.772 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 85921, 2024-04-02T14:04:15.772 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 0, 2024-04-02T14:04:15.772 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 0, 2024-04-02T14:04:15.773 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 85921, 2024-04-02T14:04:15.773 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 17917369, 2024-04-02T14:04:15.773 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-02T14:04:15.773 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:15.773 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:15.773 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:15.773 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:15.773 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.773 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-objecter_bytes": { 2024-04-02T14:04:15.774 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-02T14:04:15.774 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 104857600, 2024-04-02T14:04:15.774 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-02T14:04:15.774 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 0, 2024-04-02T14:04:15.774 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 0, 2024-04-02T14:04:15.774 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:15.774 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:15.774 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 61330, 2024-04-02T14:04:15.775 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 52119841, 2024-04-02T14:04:15.775 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 61330, 2024-04-02T14:04:15.775 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 52119841, 2024-04-02T14:04:15.775 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-02T14:04:15.775 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:15.775 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:15.775 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:15.775 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:15.775 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.776 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-objecter_ops": { 2024-04-02T14:04:15.776 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-02T14:04:15.776 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 1024, 2024-04-02T14:04:15.776 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-02T14:04:15.776 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 0, 2024-04-02T14:04:15.776 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 0, 2024-04-02T14:04:15.776 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:15.776 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:15.777 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 61330, 2024-04-02T14:04:15.777 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 61330, 2024-04-02T14:04:15.777 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 61330, 2024-04-02T14:04:15.777 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 61330, 2024-04-02T14:04:15.777 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-02T14:04:15.777 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:15.777 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:15.777 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:15.777 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:15.777 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.778 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-write_buf_throttle": { 2024-04-02T14:04:15.778 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-02T14:04:15.778 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 3758096384, 2024-04-02T14:04:15.778 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-02T14:04:15.778 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 3233, 2024-04-02T14:04:15.778 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 352285, 2024-04-02T14:04:15.778 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:15.778 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 3233, 2024-04-02T14:04:15.779 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 0, 2024-04-02T14:04:15.779 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 0, 2024-04-02T14:04:15.779 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 53, 2024-04-02T14:04:15.779 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 352285, 2024-04-02T14:04:15.779 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-02T14:04:15.779 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:15.779 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:15.779 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:15.780 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:15.780 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:15.780 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-write_buf_throttle-0x56453e25e500": { 2024-04-02T14:04:15.780 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-02T14:04:15.780 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 3758096384, 2024-04-02T14:04:15.780 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-02T14:04:15.780 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 22725, 2024-04-02T14:04:15.780 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 48067943, 2024-04-02T14:04:15.780 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:15.781 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 22725, 2024-04-02T14:04:15.781 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 0, 2024-04-02T14:04:15.781 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 0, 2024-04-02T14:04:15.781 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 427, 2024-04-02T14:04:15.781 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 48067943, 2024-04-02T14:04:15.781 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-02T14:04:15.781 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:15.781 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:15.782 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:15.782 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:15.782 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:15.782 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-02T14:04:15.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:15 smithi189 ceph-mon[36683]: pgmap v334: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:04:15.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:15 smithi052 ceph-mon[32104]: pgmap v334: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-02T14:04:16.521 INFO:tasks.check_counter:Daemon mds.b mds.exported=2 2024-04-02T14:04:16.521 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-02T14:04:16.521 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=101 2024-04-02T14:04:16.521 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-04-02T14:04:16.521 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-02T14:04:16.521 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-04-02T14:04:16.521 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-04-02T14:04:16.522 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=22 2024-04-02T14:04:16.522 DEBUG:tasks.check_counter:Getting stats from e 2024-04-02T14:04:16.522 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph daemon mds.e perf dump 2024-04-02T14:04:16.761 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T14:04:16.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:16 smithi189 ceph-mon[36683]: osdmap e103: 12 total, 12 up, 12 in 2024-04-02T14:04:16.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:16 smithi052 ceph-mon[32104]: osdmap e103: 12 total, 12 up, 12 in 2024-04-02T14:04:16.882 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:16 smithi088 ceph-mon[36528]: osdmap e103: 12 total, 12 up, 12 in 2024-04-02T14:04:17.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:17 smithi189 ceph-mon[36683]: pgmap v336: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T14:04:17.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:17 smithi052 ceph-mon[32104]: pgmap v336: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T14:04:17.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:17 smithi088 ceph-mon[36528]: pgmap v336: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T14:04:18.529 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-02T14:04:18.530 INFO:teuthology.orchestra.run.smithi088.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T14:04:18.530 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_messages": 1908, 2024-04-02T14:04:18.530 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages": 1894, 2024-04-02T14:04:18.530 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_bytes": 20840767, 2024-04-02T14:04:18.530 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_bytes": 664984, 2024-04-02T14:04:18.530 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_created_connections": 6, 2024-04-02T14:04:18.530 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_active_connections": 2, 2024-04-02T14:04:18.531 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_total_time": 0.201415096, 2024-04-02T14:04:18.531 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_send_time": 0.072349964, 2024-04-02T14:04:18.531 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_recv_time": 0.077569889, 2024-04-02T14:04:18.531 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_fast_dispatch_time": 0.046971210, 2024-04-02T14:04:18.531 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:18.531 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 1893, 2024-04-02T14:04:18.531 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.070539641, 2024-04-02T14:04:18.532 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000037263 2024-04-02T14:04:18.532 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.532 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:18.532 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.532 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.532 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.532 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.532 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_encrypted_bytes": 1640256, 2024-04-02T14:04:18.533 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_encrypted_bytes": 139008 2024-04-02T14:04:18.533 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.533 INFO:teuthology.orchestra.run.smithi088.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T14:04:18.533 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_messages": 534, 2024-04-02T14:04:18.533 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages": 699, 2024-04-02T14:04:18.533 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_bytes": 21110400, 2024-04-02T14:04:18.533 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_bytes": 619698, 2024-04-02T14:04:18.533 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_created_connections": 8, 2024-04-02T14:04:18.533 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_active_connections": 6, 2024-04-02T14:04:18.534 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_total_time": 0.067232865, 2024-04-02T14:04:18.534 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_send_time": 0.019109948, 2024-04-02T14:04:18.534 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_recv_time": 0.025267972, 2024-04-02T14:04:18.534 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_fast_dispatch_time": 0.017083653, 2024-04-02T14:04:18.534 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:18.534 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 697, 2024-04-02T14:04:18.534 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.030401233, 2024-04-02T14:04:18.534 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000043617 2024-04-02T14:04:18.535 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.535 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:18.535 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.535 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.535 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.535 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.535 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-04-02T14:04:18.535 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_encrypted_bytes": 451728 2024-04-02T14:04:18.535 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.536 INFO:teuthology.orchestra.run.smithi088.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T14:04:18.536 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_messages": 274, 2024-04-02T14:04:18.536 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages": 276, 2024-04-02T14:04:18.536 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_bytes": 8461338, 2024-04-02T14:04:18.536 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_bytes": 87139, 2024-04-02T14:04:18.536 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_created_connections": 5, 2024-04-02T14:04:18.536 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_active_connections": 4, 2024-04-02T14:04:18.536 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_total_time": 0.027800968, 2024-04-02T14:04:18.537 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_send_time": 0.006683105, 2024-04-02T14:04:18.537 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_recv_time": 0.010493008, 2024-04-02T14:04:18.537 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_fast_dispatch_time": 0.008472629, 2024-04-02T14:04:18.537 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:18.537 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 274, 2024-04-02T14:04:18.537 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.014308312, 2024-04-02T14:04:18.537 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000052220 2024-04-02T14:04:18.537 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.538 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:18.538 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.538 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.538 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.538 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.538 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T14:04:18.538 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T14:04:18.538 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.539 INFO:teuthology.orchestra.run.smithi088.stdout: "cct": { 2024-04-02T14:04:18.539 INFO:teuthology.orchestra.run.smithi088.stdout: "total_workers": 1, 2024-04-02T14:04:18.539 INFO:teuthology.orchestra.run.smithi088.stdout: "unhealthy_workers": 0 2024-04-02T14:04:18.539 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.539 INFO:teuthology.orchestra.run.smithi088.stdout: "finisher-MDSRank": { 2024-04-02T14:04:18.539 INFO:teuthology.orchestra.run.smithi088.stdout: "queue_len": 0, 2024-04-02T14:04:18.539 INFO:teuthology.orchestra.run.smithi088.stdout: "complete_latency": { 2024-04-02T14:04:18.539 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 4066, 2024-04-02T14:04:18.539 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.098799614, 2024-04-02T14:04:18.540 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000024298 2024-04-02T14:04:18.540 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:18.540 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.540 INFO:teuthology.orchestra.run.smithi088.stdout: "finisher-PurgeQueue": { 2024-04-02T14:04:18.540 INFO:teuthology.orchestra.run.smithi088.stdout: "queue_len": 0, 2024-04-02T14:04:18.540 INFO:teuthology.orchestra.run.smithi088.stdout: "complete_latency": { 2024-04-02T14:04:18.540 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.540 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.540 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.540 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:18.541 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.541 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-04-02T14:04:18.541 INFO:teuthology.orchestra.run.smithi088.stdout: "request": 0, 2024-04-02T14:04:18.541 INFO:teuthology.orchestra.run.smithi088.stdout: "reply": 0, 2024-04-02T14:04:18.541 INFO:teuthology.orchestra.run.smithi088.stdout: "reply_latency": { 2024-04-02T14:04:18.541 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.541 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.541 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.542 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.542 INFO:teuthology.orchestra.run.smithi088.stdout: "slow_reply": 0, 2024-04-02T14:04:18.542 INFO:teuthology.orchestra.run.smithi088.stdout: "forward": 0, 2024-04-02T14:04:18.542 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_fetch_complete": 0, 2024-04-02T14:04:18.542 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_fetch_keys": 0, 2024-04-02T14:04:18.542 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_commit": 0, 2024-04-02T14:04:18.542 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_split": 0, 2024-04-02T14:04:18.542 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_merge": 0, 2024-04-02T14:04:18.543 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes": 7959, 2024-04-02T14:04:18.543 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_top": 0, 2024-04-02T14:04:18.543 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_bottom": 7959, 2024-04-02T14:04:18.543 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_pin_tail": 0, 2024-04-02T14:04:18.543 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_pinned": 7959, 2024-04-02T14:04:18.543 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_expired": 0, 2024-04-02T14:04:18.543 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_with_caps": 0, 2024-04-02T14:04:18.543 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": 0, 2024-04-02T14:04:18.544 INFO:teuthology.orchestra.run.smithi088.stdout: "subtrees": 4, 2024-04-02T14:04:18.544 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse": 0, 2024-04-02T14:04:18.544 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_hit": 0, 2024-04-02T14:04:18.544 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_forward": 0, 2024-04-02T14:04:18.544 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_discover": 0, 2024-04-02T14:04:18.544 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_dir_fetch": 0, 2024-04-02T14:04:18.544 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_remote_ino": 0, 2024-04-02T14:04:18.544 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_lock": 0, 2024-04-02T14:04:18.545 INFO:teuthology.orchestra.run.smithi088.stdout: "load_cent": 0, 2024-04-02T14:04:18.545 INFO:teuthology.orchestra.run.smithi088.stdout: "q": 0, 2024-04-02T14:04:18.545 INFO:teuthology.orchestra.run.smithi088.stdout: "exported": 0, 2024-04-02T14:04:18.545 INFO:teuthology.orchestra.run.smithi088.stdout: "exported_inodes": 0, 2024-04-02T14:04:18.545 INFO:teuthology.orchestra.run.smithi088.stdout: "imported": 0, 2024-04-02T14:04:18.545 INFO:teuthology.orchestra.run.smithi088.stdout: "imported_inodes": 0, 2024-04-02T14:04:18.545 INFO:teuthology.orchestra.run.smithi088.stdout: "openino_dir_fetch": 0, 2024-04-02T14:04:18.545 INFO:teuthology.orchestra.run.smithi088.stdout: "openino_backtrace_fetch": 0, 2024-04-02T14:04:18.546 INFO:teuthology.orchestra.run.smithi088.stdout: "openino_peer_discover": 0, 2024-04-02T14:04:18.546 INFO:teuthology.orchestra.run.smithi088.stdout: "root_rfiles": 3452, 2024-04-02T14:04:18.546 INFO:teuthology.orchestra.run.smithi088.stdout: "root_rbytes": 26601218, 2024-04-02T14:04:18.546 INFO:teuthology.orchestra.run.smithi088.stdout: "root_rsnaps": 6, 2024-04-02T14:04:18.546 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T14:04:18.546 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_set_tag": 0, 2024-04-02T14:04:18.546 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T14:04:18.546 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_inotable_repaired": 0, 2024-04-02T14:04:18.547 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_dir_inodes": 0, 2024-04-02T14:04:18.547 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T14:04:18.547 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T14:04:18.547 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_file_inodes": 0, 2024-04-02T14:04:18.547 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_inode_file_caps": 0, 2024-04-02T14:04:18.547 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T14:04:18.547 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_grant": 0, 2024-04-02T14:04:18.547 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T14:04:18.547 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T14:04:18.548 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T14:04:18.548 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_caps": 0, 2024-04-02T14:04:18.548 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_caps_dirty": 0, 2024-04-02T14:04:18.548 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_cap_release": 0, 2024-04-02T14:04:18.548 INFO:teuthology.orchestra.run.smithi088.stdout: "process_request_cap_release": 0 2024-04-02T14:04:18.548 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.548 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_cache": { 2024-04-02T14:04:18.548 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_update": 0, 2024-04-02T14:04:18.549 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_update_receipt": 0, 2024-04-02T14:04:18.549 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_try_discover": 0, 2024-04-02T14:04:18.549 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_send_discover": 0, 2024-04-02T14:04:18.549 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_handle_discover": 0, 2024-04-02T14:04:18.549 INFO:teuthology.orchestra.run.smithi088.stdout: "num_strays": 0, 2024-04-02T14:04:18.549 INFO:teuthology.orchestra.run.smithi088.stdout: "num_strays_delayed": 0, 2024-04-02T14:04:18.549 INFO:teuthology.orchestra.run.smithi088.stdout: "num_strays_enqueuing": 0, 2024-04-02T14:04:18.549 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_created": 0, 2024-04-02T14:04:18.550 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_enqueued": 0, 2024-04-02T14:04:18.550 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_reintegrated": 0, 2024-04-02T14:04:18.550 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_migrated": 0, 2024-04-02T14:04:18.550 INFO:teuthology.orchestra.run.smithi088.stdout: "num_recovering_processing": 0, 2024-04-02T14:04:18.550 INFO:teuthology.orchestra.run.smithi088.stdout: "num_recovering_enqueued": 0, 2024-04-02T14:04:18.550 INFO:teuthology.orchestra.run.smithi088.stdout: "num_recovering_prioritized": 0, 2024-04-02T14:04:18.550 INFO:teuthology.orchestra.run.smithi088.stdout: "recovery_started": 0, 2024-04-02T14:04:18.551 INFO:teuthology.orchestra.run.smithi088.stdout: "recovery_completed": 0, 2024-04-02T14:04:18.551 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_quiesce_path": 0, 2024-04-02T14:04:18.551 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_quiesce_inode": 0, 2024-04-02T14:04:18.551 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T14:04:18.551 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_exportdir": 0, 2024-04-02T14:04:18.551 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_flush": 0, 2024-04-02T14:04:18.551 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_fragmentdir": 0, 2024-04-02T14:04:18.551 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_fragstats": 0, 2024-04-02T14:04:18.551 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_inodestats": 0 2024-04-02T14:04:18.552 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.552 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_log": { 2024-04-02T14:04:18.552 INFO:teuthology.orchestra.run.smithi088.stdout: "evlrg": 0, 2024-04-02T14:04:18.552 INFO:teuthology.orchestra.run.smithi088.stdout: "evadd": 0, 2024-04-02T14:04:18.552 INFO:teuthology.orchestra.run.smithi088.stdout: "evex": 0, 2024-04-02T14:04:18.552 INFO:teuthology.orchestra.run.smithi088.stdout: "evtrm": 0, 2024-04-02T14:04:18.552 INFO:teuthology.orchestra.run.smithi088.stdout: "ev": 22725, 2024-04-02T14:04:18.552 INFO:teuthology.orchestra.run.smithi088.stdout: "evexg": 0, 2024-04-02T14:04:18.553 INFO:teuthology.orchestra.run.smithi088.stdout: "evexd": 0, 2024-04-02T14:04:18.553 INFO:teuthology.orchestra.run.smithi088.stdout: "segadd": 0, 2024-04-02T14:04:18.553 INFO:teuthology.orchestra.run.smithi088.stdout: "segex": 0, 2024-04-02T14:04:18.553 INFO:teuthology.orchestra.run.smithi088.stdout: "segtrm": 0, 2024-04-02T14:04:18.553 INFO:teuthology.orchestra.run.smithi088.stdout: "seg": 27, 2024-04-02T14:04:18.553 INFO:teuthology.orchestra.run.smithi088.stdout: "segmjr": 3, 2024-04-02T14:04:18.553 INFO:teuthology.orchestra.run.smithi088.stdout: "segexg": 0, 2024-04-02T14:04:18.553 INFO:teuthology.orchestra.run.smithi088.stdout: "segexd": 0, 2024-04-02T14:04:18.554 INFO:teuthology.orchestra.run.smithi088.stdout: "expos": 4194304, 2024-04-02T14:04:18.554 INFO:teuthology.orchestra.run.smithi088.stdout: "wrpos": 52262247, 2024-04-02T14:04:18.554 INFO:teuthology.orchestra.run.smithi088.stdout: "rdpos": 52262111, 2024-04-02T14:04:18.554 INFO:teuthology.orchestra.run.smithi088.stdout: "jlat": { 2024-04-02T14:04:18.554 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.554 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.554 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.554 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.555 INFO:teuthology.orchestra.run.smithi088.stdout: "replayed": 22725 2024-04-02T14:04:18.555 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.555 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_mem": { 2024-04-02T14:04:18.555 INFO:teuthology.orchestra.run.smithi088.stdout: "ino": 3582, 2024-04-02T14:04:18.555 INFO:teuthology.orchestra.run.smithi088.stdout: "ino+": 6823, 2024-04-02T14:04:18.555 INFO:teuthology.orchestra.run.smithi088.stdout: "ino-": 3241, 2024-04-02T14:04:18.555 INFO:teuthology.orchestra.run.smithi088.stdout: "dir": 240, 2024-04-02T14:04:18.555 INFO:teuthology.orchestra.run.smithi088.stdout: "dir+": 273, 2024-04-02T14:04:18.556 INFO:teuthology.orchestra.run.smithi088.stdout: "dir-": 33, 2024-04-02T14:04:18.556 INFO:teuthology.orchestra.run.smithi088.stdout: "dn": 7959, 2024-04-02T14:04:18.556 INFO:teuthology.orchestra.run.smithi088.stdout: "dn+": 8018, 2024-04-02T14:04:18.556 INFO:teuthology.orchestra.run.smithi088.stdout: "dn-": 59, 2024-04-02T14:04:18.556 INFO:teuthology.orchestra.run.smithi088.stdout: "cap": 0, 2024-04-02T14:04:18.556 INFO:teuthology.orchestra.run.smithi088.stdout: "cap+": 0, 2024-04-02T14:04:18.556 INFO:teuthology.orchestra.run.smithi088.stdout: "cap-": 0, 2024-04-02T14:04:18.556 INFO:teuthology.orchestra.run.smithi088.stdout: "rss": 67796, 2024-04-02T14:04:18.556 INFO:teuthology.orchestra.run.smithi088.stdout: "heap": 215416 2024-04-02T14:04:18.557 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.557 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_server": { 2024-04-02T14:04:18.557 INFO:teuthology.orchestra.run.smithi088.stdout: "dispatch_client_request": 0, 2024-04-02T14:04:18.557 INFO:teuthology.orchestra.run.smithi088.stdout: "dispatch_server_request": 0, 2024-04-02T14:04:18.557 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_request": 0, 2024-04-02T14:04:18.557 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_session": 0, 2024-04-02T14:04:18.557 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_peer_request": 0, 2024-04-02T14:04:18.557 INFO:teuthology.orchestra.run.smithi088.stdout: "req_create_latency": { 2024-04-02T14:04:18.557 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.558 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.558 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.558 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.558 INFO:teuthology.orchestra.run.smithi088.stdout: "req_getattr_latency": { 2024-04-02T14:04:18.558 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.558 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.558 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.559 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.559 INFO:teuthology.orchestra.run.smithi088.stdout: "req_getfilelock_latency": { 2024-04-02T14:04:18.559 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.559 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.559 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.559 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.559 INFO:teuthology.orchestra.run.smithi088.stdout: "req_link_latency": { 2024-04-02T14:04:18.559 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.559 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.560 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.560 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.560 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookup_latency": { 2024-04-02T14:04:18.560 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.560 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.560 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.560 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.560 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookuphash_latency": { 2024-04-02T14:04:18.561 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.561 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.561 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.561 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.561 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupino_latency": { 2024-04-02T14:04:18.561 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.561 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.561 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.562 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.562 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupname_latency": { 2024-04-02T14:04:18.562 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.562 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.562 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.562 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.562 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupparent_latency": { 2024-04-02T14:04:18.562 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.563 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.563 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.563 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.563 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupsnap_latency": { 2024-04-02T14:04:18.563 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.563 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.563 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.564 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.564 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lssnap_latency": { 2024-04-02T14:04:18.564 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.564 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.564 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.564 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.565 INFO:teuthology.orchestra.run.smithi088.stdout: "req_mkdir_latency": { 2024-04-02T14:04:18.565 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.565 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.565 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.565 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.566 INFO:teuthology.orchestra.run.smithi088.stdout: "req_mknod_latency": { 2024-04-02T14:04:18.566 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.566 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.566 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.566 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.566 INFO:teuthology.orchestra.run.smithi088.stdout: "req_mksnap_latency": { 2024-04-02T14:04:18.566 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.567 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.567 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.567 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.567 INFO:teuthology.orchestra.run.smithi088.stdout: "req_open_latency": { 2024-04-02T14:04:18.567 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.567 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.567 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.567 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.568 INFO:teuthology.orchestra.run.smithi088.stdout: "req_readdir_latency": { 2024-04-02T14:04:18.568 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.568 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.568 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.568 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.568 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rename_latency": { 2024-04-02T14:04:18.568 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.568 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.569 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.569 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.569 INFO:teuthology.orchestra.run.smithi088.stdout: "req_renamesnap_latency": { 2024-04-02T14:04:18.569 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.569 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.569 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.569 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.569 INFO:teuthology.orchestra.run.smithi088.stdout: "req_snapdiff_latency": { 2024-04-02T14:04:18.569 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.570 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.570 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.570 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.570 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rmdir_latency": { 2024-04-02T14:04:18.570 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.570 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.570 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.570 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.571 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rmsnap_latency": { 2024-04-02T14:04:18.571 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.571 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.571 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.571 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.571 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rmxattr_latency": { 2024-04-02T14:04:18.571 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.571 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.572 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.572 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.572 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setattr_latency": { 2024-04-02T14:04:18.572 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.572 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.572 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.572 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.572 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setdirlayout_latency": { 2024-04-02T14:04:18.573 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.573 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.573 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.573 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.573 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setfilelock_latency": { 2024-04-02T14:04:18.573 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.573 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.573 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.573 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.574 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setlayout_latency": { 2024-04-02T14:04:18.574 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.574 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.574 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.574 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.574 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setxattr_latency": { 2024-04-02T14:04:18.574 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.574 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.575 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.575 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.575 INFO:teuthology.orchestra.run.smithi088.stdout: "req_symlink_latency": { 2024-04-02T14:04:18.575 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.575 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.575 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.575 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.575 INFO:teuthology.orchestra.run.smithi088.stdout: "req_unlink_latency": { 2024-04-02T14:04:18.576 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.576 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.576 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.576 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.576 INFO:teuthology.orchestra.run.smithi088.stdout: "cap_revoke_eviction": 0, 2024-04-02T14:04:18.576 INFO:teuthology.orchestra.run.smithi088.stdout: "cap_acquisition_throttle": 0, 2024-04-02T14:04:18.576 INFO:teuthology.orchestra.run.smithi088.stdout: "req_getvxattr_latency": { 2024-04-02T14:04:18.576 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.577 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.577 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.577 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:18.577 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.577 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_sessions": { 2024-04-02T14:04:18.577 INFO:teuthology.orchestra.run.smithi088.stdout: "session_count": 2, 2024-04-02T14:04:18.577 INFO:teuthology.orchestra.run.smithi088.stdout: "session_add": 7, 2024-04-02T14:04:18.577 INFO:teuthology.orchestra.run.smithi088.stdout: "session_remove": 5, 2024-04-02T14:04:18.578 INFO:teuthology.orchestra.run.smithi088.stdout: "sessions_open": 5, 2024-04-02T14:04:18.578 INFO:teuthology.orchestra.run.smithi088.stdout: "sessions_stale": 0, 2024-04-02T14:04:18.578 INFO:teuthology.orchestra.run.smithi088.stdout: "total_load": 0, 2024-04-02T14:04:18.578 INFO:teuthology.orchestra.run.smithi088.stdout: "average_load": 0, 2024-04-02T14:04:18.578 INFO:teuthology.orchestra.run.smithi088.stdout: "avg_session_uptime": 4072, 2024-04-02T14:04:18.578 INFO:teuthology.orchestra.run.smithi088.stdout: "mdthresh_evicted": 0 2024-04-02T14:04:18.578 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.578 INFO:teuthology.orchestra.run.smithi088.stdout: "mempool": { 2024-04-02T14:04:18.579 INFO:teuthology.orchestra.run.smithi088.stdout: "bloom_filter_bytes": 0, 2024-04-02T14:04:18.579 INFO:teuthology.orchestra.run.smithi088.stdout: "bloom_filter_items": 0, 2024-04-02T14:04:18.579 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T14:04:18.579 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_alloc_items": 0, 2024-04-02T14:04:18.579 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T14:04:18.579 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_data_items": 0, 2024-04-02T14:04:18.579 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T14:04:18.579 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T14:04:18.580 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T14:04:18.580 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T14:04:18.580 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T14:04:18.580 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_other_items": 0, 2024-04-02T14:04:18.580 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T14:04:18.580 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T14:04:18.580 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_extent_bytes": 0, 2024-04-02T14:04:18.580 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_extent_items": 0, 2024-04-02T14:04:18.581 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_blob_bytes": 0, 2024-04-02T14:04:18.581 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_blob_items": 0, 2024-04-02T14:04:18.581 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T14:04:18.581 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T14:04:18.581 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T14:04:18.581 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T14:04:18.581 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T14:04:18.581 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_fsck_items": 0, 2024-04-02T14:04:18.581 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_txc_bytes": 0, 2024-04-02T14:04:18.582 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_txc_items": 0, 2024-04-02T14:04:18.582 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T14:04:18.582 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T14:04:18.582 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_bytes": 0, 2024-04-02T14:04:18.582 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_items": 0, 2024-04-02T14:04:18.582 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_bytes": 0, 2024-04-02T14:04:18.582 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_items": 0, 2024-04-02T14:04:18.582 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T14:04:18.583 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_reader_items": 0, 2024-04-02T14:04:18.583 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T14:04:18.583 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_writer_items": 0, 2024-04-02T14:04:18.583 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_anon_bytes": 347432, 2024-04-02T14:04:18.583 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_anon_items": 3660, 2024-04-02T14:04:18.583 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_meta_bytes": 0, 2024-04-02T14:04:18.583 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_meta_items": 0, 2024-04-02T14:04:18.583 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_bytes": 0, 2024-04-02T14:04:18.583 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_items": 0, 2024-04-02T14:04:18.584 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_mapbl_bytes": 0, 2024-04-02T14:04:18.584 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_mapbl_items": 0, 2024-04-02T14:04:18.584 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_pglog_bytes": 0, 2024-04-02T14:04:18.584 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_pglog_items": 0, 2024-04-02T14:04:18.584 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_bytes": 9560, 2024-04-02T14:04:18.584 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_items": 184, 2024-04-02T14:04:18.584 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T14:04:18.585 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_mapping_items": 0, 2024-04-02T14:04:18.585 INFO:teuthology.orchestra.run.smithi088.stdout: "pgmap_bytes": 0, 2024-04-02T14:04:18.585 INFO:teuthology.orchestra.run.smithi088.stdout: "pgmap_items": 0, 2024-04-02T14:04:18.585 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_co_bytes": 14253889, 2024-04-02T14:04:18.585 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_co_items": 123244, 2024-04-02T14:04:18.585 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_1_bytes": 0, 2024-04-02T14:04:18.585 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_1_items": 0, 2024-04-02T14:04:18.585 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_2_bytes": 0, 2024-04-02T14:04:18.586 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_2_items": 0 2024-04-02T14:04:18.586 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.586 INFO:teuthology.orchestra.run.smithi088.stdout: "objecter": { 2024-04-02T14:04:18.586 INFO:teuthology.orchestra.run.smithi088.stdout: "op_active": 0, 2024-04-02T14:04:18.586 INFO:teuthology.orchestra.run.smithi088.stdout: "op_laggy": 0, 2024-04-02T14:04:18.586 INFO:teuthology.orchestra.run.smithi088.stdout: "op_send": 2456, 2024-04-02T14:04:18.586 INFO:teuthology.orchestra.run.smithi088.stdout: "op_send_bytes": 26, 2024-04-02T14:04:18.586 INFO:teuthology.orchestra.run.smithi088.stdout: "op_resend": 4, 2024-04-02T14:04:18.587 INFO:teuthology.orchestra.run.smithi088.stdout: "op_reply": 2456, 2024-04-02T14:04:18.587 INFO:teuthology.orchestra.run.smithi088.stdout: "op_latency": { 2024-04-02T14:04:18.587 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 2456, 2024-04-02T14:04:18.587 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 1.440971206, 2024-04-02T14:04:18.587 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000586714 2024-04-02T14:04:18.587 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.587 INFO:teuthology.orchestra.run.smithi088.stdout: "op_inflight": 0, 2024-04-02T14:04:18.587 INFO:teuthology.orchestra.run.smithi088.stdout: "oplen_avg": { 2024-04-02T14:04:18.587 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 2456, 2024-04-02T14:04:18.588 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 2457 2024-04-02T14:04:18.588 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.588 INFO:teuthology.orchestra.run.smithi088.stdout: "op": 2456, 2024-04-02T14:04:18.588 INFO:teuthology.orchestra.run.smithi088.stdout: "op_r": 2456, 2024-04-02T14:04:18.588 INFO:teuthology.orchestra.run.smithi088.stdout: "op_w": 0, 2024-04-02T14:04:18.588 INFO:teuthology.orchestra.run.smithi088.stdout: "op_rmw": 0, 2024-04-02T14:04:18.588 INFO:teuthology.orchestra.run.smithi088.stdout: "op_pg": 0, 2024-04-02T14:04:18.588 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_stat": 1577, 2024-04-02T14:04:18.588 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_create": 0, 2024-04-02T14:04:18.589 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_read": 876, 2024-04-02T14:04:18.589 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_write": 0, 2024-04-02T14:04:18.589 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_writefull": 0, 2024-04-02T14:04:18.589 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_writesame": 0, 2024-04-02T14:04:18.589 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_append": 0, 2024-04-02T14:04:18.589 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_zero": 0, 2024-04-02T14:04:18.589 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_truncate": 0, 2024-04-02T14:04:18.589 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_delete": 0, 2024-04-02T14:04:18.589 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_mapext": 0, 2024-04-02T14:04:18.590 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_sparse_read": 0, 2024-04-02T14:04:18.590 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_clonerange": 0, 2024-04-02T14:04:18.590 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_getxattr": 2, 2024-04-02T14:04:18.590 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_setxattr": 0, 2024-04-02T14:04:18.590 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_cmpxattr": 0, 2024-04-02T14:04:18.590 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_rmxattr": 0, 2024-04-02T14:04:18.590 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_resetxattrs": 0, 2024-04-02T14:04:18.590 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_call": 0, 2024-04-02T14:04:18.591 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_watch": 0, 2024-04-02T14:04:18.591 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_notify": 0, 2024-04-02T14:04:18.591 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T14:04:18.591 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_pgls": 0, 2024-04-02T14:04:18.591 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_pgls_filter": 0, 2024-04-02T14:04:18.591 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_other": 0, 2024-04-02T14:04:18.591 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_active": 0, 2024-04-02T14:04:18.591 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_send": 0, 2024-04-02T14:04:18.592 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_resend": 0, 2024-04-02T14:04:18.592 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_ping": 0, 2024-04-02T14:04:18.592 INFO:teuthology.orchestra.run.smithi088.stdout: "poolop_active": 0, 2024-04-02T14:04:18.592 INFO:teuthology.orchestra.run.smithi088.stdout: "poolop_send": 0, 2024-04-02T14:04:18.592 INFO:teuthology.orchestra.run.smithi088.stdout: "poolop_resend": 0, 2024-04-02T14:04:18.592 INFO:teuthology.orchestra.run.smithi088.stdout: "poolstat_active": 0, 2024-04-02T14:04:18.592 INFO:teuthology.orchestra.run.smithi088.stdout: "poolstat_send": 0, 2024-04-02T14:04:18.592 INFO:teuthology.orchestra.run.smithi088.stdout: "poolstat_resend": 0, 2024-04-02T14:04:18.593 INFO:teuthology.orchestra.run.smithi088.stdout: "statfs_active": 0, 2024-04-02T14:04:18.593 INFO:teuthology.orchestra.run.smithi088.stdout: "statfs_send": 0, 2024-04-02T14:04:18.593 INFO:teuthology.orchestra.run.smithi088.stdout: "statfs_resend": 0, 2024-04-02T14:04:18.593 INFO:teuthology.orchestra.run.smithi088.stdout: "command_active": 0, 2024-04-02T14:04:18.593 INFO:teuthology.orchestra.run.smithi088.stdout: "command_send": 0, 2024-04-02T14:04:18.593 INFO:teuthology.orchestra.run.smithi088.stdout: "command_resend": 0, 2024-04-02T14:04:18.593 INFO:teuthology.orchestra.run.smithi088.stdout: "map_epoch": 103, 2024-04-02T14:04:18.593 INFO:teuthology.orchestra.run.smithi088.stdout: "map_full": 0, 2024-04-02T14:04:18.593 INFO:teuthology.orchestra.run.smithi088.stdout: "map_inc": 13, 2024-04-02T14:04:18.594 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_sessions": 8, 2024-04-02T14:04:18.594 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_session_open": 8, 2024-04-02T14:04:18.594 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_session_close": 0, 2024-04-02T14:04:18.594 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_laggy": 0, 2024-04-02T14:04:18.594 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_wr": 0, 2024-04-02T14:04:18.594 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_rd": 2, 2024-04-02T14:04:18.594 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_del": 0 2024-04-02T14:04:18.595 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.595 INFO:teuthology.orchestra.run.smithi088.stdout: "oft": { 2024-04-02T14:04:18.595 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_objs": 0, 2024-04-02T14:04:18.595 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_kv_pairs": 0, 2024-04-02T14:04:18.595 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_updates": 0, 2024-04-02T14:04:18.595 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_removes": 0 2024-04-02T14:04:18.595 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.595 INFO:teuthology.orchestra.run.smithi088.stdout: "osd-slow-ops": { 2024-04-02T14:04:18.595 INFO:teuthology.orchestra.run.smithi088.stdout: "slow_ops_count": 0 2024-04-02T14:04:18.596 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.596 INFO:teuthology.orchestra.run.smithi088.stdout: "purge_queue": { 2024-04-02T14:04:18.596 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing_ops": 0, 2024-04-02T14:04:18.596 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T14:04:18.596 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing": 0, 2024-04-02T14:04:18.596 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing_high_water": 0, 2024-04-02T14:04:18.596 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executed_ops": 0, 2024-04-02T14:04:18.596 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executed": 0, 2024-04-02T14:04:18.596 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_item_in_journal": 0 2024-04-02T14:04:18.597 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.597 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T14:04:18.597 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-02T14:04:18.597 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 104857600, 2024-04-02T14:04:18.597 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-02T14:04:18.597 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 2712, 2024-04-02T14:04:18.597 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 50152329, 2024-04-02T14:04:18.598 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:18.598 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 2712, 2024-04-02T14:04:18.598 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 0, 2024-04-02T14:04:18.598 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 0, 2024-04-02T14:04:18.598 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 2712, 2024-04-02T14:04:18.598 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 50152329, 2024-04-02T14:04:18.598 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-02T14:04:18.599 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.599 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.599 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.599 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:18.599 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.599 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-objecter_bytes": { 2024-04-02T14:04:18.599 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-02T14:04:18.599 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 104857600, 2024-04-02T14:04:18.599 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-02T14:04:18.600 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 0, 2024-04-02T14:04:18.600 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 0, 2024-04-02T14:04:18.600 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:18.600 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:18.600 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 2456, 2024-04-02T14:04:18.600 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 48080691, 2024-04-02T14:04:18.600 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 2456, 2024-04-02T14:04:18.600 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 48080691, 2024-04-02T14:04:18.600 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-02T14:04:18.601 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.601 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.601 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.601 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:18.601 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.601 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-objecter_ops": { 2024-04-02T14:04:18.602 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-02T14:04:18.602 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 1024, 2024-04-02T14:04:18.602 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-02T14:04:18.602 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 0, 2024-04-02T14:04:18.602 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 0, 2024-04-02T14:04:18.602 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:18.602 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:18.602 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 2456, 2024-04-02T14:04:18.603 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 2456, 2024-04-02T14:04:18.603 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 2456, 2024-04-02T14:04:18.603 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 2456, 2024-04-02T14:04:18.603 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-02T14:04:18.603 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.603 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.603 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.603 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:18.604 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.604 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-write_buf_throttle": { 2024-04-02T14:04:18.604 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-02T14:04:18.604 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 3758096384, 2024-04-02T14:04:18.604 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-02T14:04:18.604 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 0, 2024-04-02T14:04:18.604 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 0, 2024-04-02T14:04:18.604 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:18.605 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:18.605 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 0, 2024-04-02T14:04:18.605 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 0, 2024-04-02T14:04:18.605 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 0, 2024-04-02T14:04:18.605 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 0, 2024-04-02T14:04:18.605 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-02T14:04:18.605 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.605 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.605 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.606 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:18.606 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:18.606 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-write_buf_throttle-0x55cc2cc566e0": { 2024-04-02T14:04:18.606 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-02T14:04:18.606 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 3758096384, 2024-04-02T14:04:18.606 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-02T14:04:18.606 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 0, 2024-04-02T14:04:18.606 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 0, 2024-04-02T14:04:18.607 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:18.607 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:18.607 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 0, 2024-04-02T14:04:18.607 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 0, 2024-04-02T14:04:18.607 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 0, 2024-04-02T14:04:18.607 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 0, 2024-04-02T14:04:18.607 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-02T14:04:18.607 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:18.608 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:18.608 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:18.608 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:18.608 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:18.608 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-02T14:04:19.337 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-02T14:04:19.337 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-02T14:04:19.337 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-02T14:04:19.338 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-04-02T14:04:19.338 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-04-02T14:04:19.338 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-02T14:04:19.338 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T14:04:19.338 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-04-02T14:04:19.338 DEBUG:tasks.check_counter:Getting stats from h 2024-04-02T14:04:19.338 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph daemon mds.h perf dump 2024-04-02T14:04:19.577 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T14:04:19.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:19 smithi189 ceph-mon[36683]: pgmap v337: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T14:04:19.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:19 smithi052 ceph-mon[32104]: pgmap v337: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T14:04:19.849 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:19 smithi088 ceph-mon[36528]: pgmap v337: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T14:04:21.302 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-02T14:04:21.303 INFO:teuthology.orchestra.run.smithi088.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T14:04:21.303 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_messages": 163, 2024-04-02T14:04:21.303 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages": 196, 2024-04-02T14:04:21.303 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_bytes": 36039, 2024-04-02T14:04:21.303 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_bytes": 426538, 2024-04-02T14:04:21.303 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_created_connections": 43, 2024-04-02T14:04:21.304 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_active_connections": 4, 2024-04-02T14:04:21.304 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_total_time": 0.029206588, 2024-04-02T14:04:21.304 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_send_time": 0.006041865, 2024-04-02T14:04:21.304 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_recv_time": 0.018863960, 2024-04-02T14:04:21.304 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_fast_dispatch_time": 0.000398101, 2024-04-02T14:04:21.304 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:21.304 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 194, 2024-04-02T14:04:21.304 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.032815810, 2024-04-02T14:04:21.305 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000169153 2024-04-02T14:04:21.305 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.305 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:21.305 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 165, 2024-04-02T14:04:21.305 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000062310, 2024-04-02T14:04:21.305 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000377 2024-04-02T14:04:21.305 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.305 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T14:04:21.305 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T14:04:21.306 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.306 INFO:teuthology.orchestra.run.smithi088.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T14:04:21.306 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_messages": 30, 2024-04-02T14:04:21.306 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages": 196, 2024-04-02T14:04:21.306 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_bytes": 13774, 2024-04-02T14:04:21.306 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_bytes": 537272, 2024-04-02T14:04:21.306 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_created_connections": 16, 2024-04-02T14:04:21.306 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_active_connections": 5, 2024-04-02T14:04:21.307 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_total_time": 0.014504810, 2024-04-02T14:04:21.307 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_send_time": 0.006981013, 2024-04-02T14:04:21.307 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_recv_time": 0.005413997, 2024-04-02T14:04:21.307 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_fast_dispatch_time": 0.000238907, 2024-04-02T14:04:21.307 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:21.307 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 193, 2024-04-02T14:04:21.307 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.025114020, 2024-04-02T14:04:21.308 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000130124 2024-04-02T14:04:21.308 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.308 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:21.308 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 24, 2024-04-02T14:04:21.308 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000002639, 2024-04-02T14:04:21.308 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000109 2024-04-02T14:04:21.308 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.308 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_encrypted_bytes": 2256, 2024-04-02T14:04:21.308 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_encrypted_bytes": 454304 2024-04-02T14:04:21.309 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.309 INFO:teuthology.orchestra.run.smithi088.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T14:04:21.309 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_messages": 1457, 2024-04-02T14:04:21.309 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages": 1157, 2024-04-02T14:04:21.309 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_bytes": 2046793, 2024-04-02T14:04:21.309 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_bytes": 458536, 2024-04-02T14:04:21.309 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_created_connections": 8, 2024-04-02T14:04:21.309 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_active_connections": 5, 2024-04-02T14:04:21.310 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_total_time": 0.139688638, 2024-04-02T14:04:21.310 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_send_time": 0.056715988, 2024-04-02T14:04:21.310 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_recv_time": 0.091289600, 2024-04-02T14:04:21.310 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_fast_dispatch_time": 0.009305145, 2024-04-02T14:04:21.310 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:21.310 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 1157, 2024-04-02T14:04:21.310 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.050710804, 2024-04-02T14:04:21.310 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000043829 2024-04-02T14:04:21.310 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.311 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:21.311 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 2014, 2024-04-02T14:04:21.311 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000342090, 2024-04-02T14:04:21.311 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000169 2024-04-02T14:04:21.311 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.311 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_encrypted_bytes": 1661472, 2024-04-02T14:04:21.311 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_encrypted_bytes": 143152 2024-04-02T14:04:21.311 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.311 INFO:teuthology.orchestra.run.smithi088.stdout: "cct": { 2024-04-02T14:04:21.312 INFO:teuthology.orchestra.run.smithi088.stdout: "total_workers": 1, 2024-04-02T14:04:21.312 INFO:teuthology.orchestra.run.smithi088.stdout: "unhealthy_workers": 0 2024-04-02T14:04:21.312 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.312 INFO:teuthology.orchestra.run.smithi088.stdout: "finisher-MDSRank": { 2024-04-02T14:04:21.312 INFO:teuthology.orchestra.run.smithi088.stdout: "queue_len": 0, 2024-04-02T14:04:21.312 INFO:teuthology.orchestra.run.smithi088.stdout: "complete_latency": { 2024-04-02T14:04:21.312 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 55, 2024-04-02T14:04:21.312 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.001075545, 2024-04-02T14:04:21.312 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000019555 2024-04-02T14:04:21.312 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:21.313 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.313 INFO:teuthology.orchestra.run.smithi088.stdout: "finisher-PurgeQueue": { 2024-04-02T14:04:21.313 INFO:teuthology.orchestra.run.smithi088.stdout: "queue_len": 0, 2024-04-02T14:04:21.313 INFO:teuthology.orchestra.run.smithi088.stdout: "complete_latency": { 2024-04-02T14:04:21.313 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 2, 2024-04-02T14:04:21.313 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000006706, 2024-04-02T14:04:21.313 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000003353 2024-04-02T14:04:21.313 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:21.314 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.314 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-04-02T14:04:21.314 INFO:teuthology.orchestra.run.smithi088.stdout: "request": 99, 2024-04-02T14:04:21.314 INFO:teuthology.orchestra.run.smithi088.stdout: "reply": 3, 2024-04-02T14:04:21.314 INFO:teuthology.orchestra.run.smithi088.stdout: "reply_latency": { 2024-04-02T14:04:21.314 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 3, 2024-04-02T14:04:21.314 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 4.050847645, 2024-04-02T14:04:21.314 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 1.350282548 2024-04-02T14:04:21.315 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.315 INFO:teuthology.orchestra.run.smithi088.stdout: "slow_reply": 0, 2024-04-02T14:04:21.315 INFO:teuthology.orchestra.run.smithi088.stdout: "forward": 96, 2024-04-02T14:04:21.315 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_fetch_complete": 0, 2024-04-02T14:04:21.315 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_fetch_keys": 0, 2024-04-02T14:04:21.315 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_commit": 11, 2024-04-02T14:04:21.315 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_split": 0, 2024-04-02T14:04:21.315 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_merge": 0, 2024-04-02T14:04:21.315 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes": 37, 2024-04-02T14:04:21.316 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_top": 10, 2024-04-02T14:04:21.316 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_bottom": 27, 2024-04-02T14:04:21.316 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_pin_tail": 0, 2024-04-02T14:04:21.316 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_pinned": 21, 2024-04-02T14:04:21.316 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_expired": 8, 2024-04-02T14:04:21.316 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_with_caps": 2, 2024-04-02T14:04:21.316 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": 2, 2024-04-02T14:04:21.316 INFO:teuthology.orchestra.run.smithi088.stdout: "subtrees": 3, 2024-04-02T14:04:21.316 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse": 146, 2024-04-02T14:04:21.317 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_hit": 28, 2024-04-02T14:04:21.317 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_forward": 94, 2024-04-02T14:04:21.317 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_discover": 19, 2024-04-02T14:04:21.317 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_dir_fetch": 0, 2024-04-02T14:04:21.317 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_remote_ino": 0, 2024-04-02T14:04:21.317 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_lock": 1, 2024-04-02T14:04:21.317 INFO:teuthology.orchestra.run.smithi088.stdout: "load_cent": 0, 2024-04-02T14:04:21.317 INFO:teuthology.orchestra.run.smithi088.stdout: "q": 0, 2024-04-02T14:04:21.317 INFO:teuthology.orchestra.run.smithi088.stdout: "exported": 0, 2024-04-02T14:04:21.318 INFO:teuthology.orchestra.run.smithi088.stdout: "exported_inodes": 0, 2024-04-02T14:04:21.318 INFO:teuthology.orchestra.run.smithi088.stdout: "imported": 0, 2024-04-02T14:04:21.318 INFO:teuthology.orchestra.run.smithi088.stdout: "imported_inodes": 0, 2024-04-02T14:04:21.318 INFO:teuthology.orchestra.run.smithi088.stdout: "openino_dir_fetch": 0, 2024-04-02T14:04:21.318 INFO:teuthology.orchestra.run.smithi088.stdout: "openino_backtrace_fetch": 0, 2024-04-02T14:04:21.318 INFO:teuthology.orchestra.run.smithi088.stdout: "openino_peer_discover": 0, 2024-04-02T14:04:21.318 INFO:teuthology.orchestra.run.smithi088.stdout: "root_rfiles": 3452, 2024-04-02T14:04:21.318 INFO:teuthology.orchestra.run.smithi088.stdout: "root_rbytes": 26601218, 2024-04-02T14:04:21.319 INFO:teuthology.orchestra.run.smithi088.stdout: "root_rsnaps": 6, 2024-04-02T14:04:21.319 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T14:04:21.319 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_set_tag": 0, 2024-04-02T14:04:21.319 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T14:04:21.319 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_inotable_repaired": 0, 2024-04-02T14:04:21.319 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_dir_inodes": 0, 2024-04-02T14:04:21.319 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T14:04:21.319 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T14:04:21.319 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_file_inodes": 0, 2024-04-02T14:04:21.320 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_inode_file_caps": 0, 2024-04-02T14:04:21.320 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T14:04:21.320 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_grant": 0, 2024-04-02T14:04:21.320 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T14:04:21.320 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T14:04:21.320 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T14:04:21.320 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_caps": 1, 2024-04-02T14:04:21.320 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_caps_dirty": 0, 2024-04-02T14:04:21.320 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_cap_release": 0, 2024-04-02T14:04:21.321 INFO:teuthology.orchestra.run.smithi088.stdout: "process_request_cap_release": 2 2024-04-02T14:04:21.321 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.321 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_cache": { 2024-04-02T14:04:21.321 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_update": 0, 2024-04-02T14:04:21.321 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_update_receipt": 71, 2024-04-02T14:04:21.322 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_try_discover": 45, 2024-04-02T14:04:21.322 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_send_discover": 47, 2024-04-02T14:04:21.322 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_handle_discover": 0, 2024-04-02T14:04:21.322 INFO:teuthology.orchestra.run.smithi088.stdout: "num_strays": 0, 2024-04-02T14:04:21.322 INFO:teuthology.orchestra.run.smithi088.stdout: "num_strays_delayed": 0, 2024-04-02T14:04:21.322 INFO:teuthology.orchestra.run.smithi088.stdout: "num_strays_enqueuing": 0, 2024-04-02T14:04:21.322 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_created": 0, 2024-04-02T14:04:21.322 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_enqueued": 0, 2024-04-02T14:04:21.323 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_reintegrated": 0, 2024-04-02T14:04:21.323 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_migrated": 0, 2024-04-02T14:04:21.323 INFO:teuthology.orchestra.run.smithi088.stdout: "num_recovering_processing": 0, 2024-04-02T14:04:21.323 INFO:teuthology.orchestra.run.smithi088.stdout: "num_recovering_enqueued": 0, 2024-04-02T14:04:21.323 INFO:teuthology.orchestra.run.smithi088.stdout: "num_recovering_prioritized": 0, 2024-04-02T14:04:21.323 INFO:teuthology.orchestra.run.smithi088.stdout: "recovery_started": 0, 2024-04-02T14:04:21.323 INFO:teuthology.orchestra.run.smithi088.stdout: "recovery_completed": 0, 2024-04-02T14:04:21.323 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_quiesce_path": 0, 2024-04-02T14:04:21.324 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_quiesce_inode": 0, 2024-04-02T14:04:21.324 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T14:04:21.324 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_exportdir": 0, 2024-04-02T14:04:21.324 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_flush": 0, 2024-04-02T14:04:21.324 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_fragmentdir": 0, 2024-04-02T14:04:21.324 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_fragstats": 0, 2024-04-02T14:04:21.324 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_inodestats": 0 2024-04-02T14:04:21.324 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.324 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_log": { 2024-04-02T14:04:21.325 INFO:teuthology.orchestra.run.smithi088.stdout: "evlrg": 0, 2024-04-02T14:04:21.325 INFO:teuthology.orchestra.run.smithi088.stdout: "evadd": 5, 2024-04-02T14:04:21.325 INFO:teuthology.orchestra.run.smithi088.stdout: "evex": 0, 2024-04-02T14:04:21.325 INFO:teuthology.orchestra.run.smithi088.stdout: "evtrm": 0, 2024-04-02T14:04:21.325 INFO:teuthology.orchestra.run.smithi088.stdout: "ev": 5, 2024-04-02T14:04:21.325 INFO:teuthology.orchestra.run.smithi088.stdout: "evexg": 0, 2024-04-02T14:04:21.325 INFO:teuthology.orchestra.run.smithi088.stdout: "evexd": 0, 2024-04-02T14:04:21.325 INFO:teuthology.orchestra.run.smithi088.stdout: "segadd": 2, 2024-04-02T14:04:21.325 INFO:teuthology.orchestra.run.smithi088.stdout: "segex": 0, 2024-04-02T14:04:21.326 INFO:teuthology.orchestra.run.smithi088.stdout: "segtrm": 0, 2024-04-02T14:04:21.326 INFO:teuthology.orchestra.run.smithi088.stdout: "seg": 2, 2024-04-02T14:04:21.326 INFO:teuthology.orchestra.run.smithi088.stdout: "segmjr": 2, 2024-04-02T14:04:21.326 INFO:teuthology.orchestra.run.smithi088.stdout: "segexg": 0, 2024-04-02T14:04:21.326 INFO:teuthology.orchestra.run.smithi088.stdout: "segexd": 0, 2024-04-02T14:04:21.326 INFO:teuthology.orchestra.run.smithi088.stdout: "expos": 4194304, 2024-04-02T14:04:21.326 INFO:teuthology.orchestra.run.smithi088.stdout: "wrpos": 4195732, 2024-04-02T14:04:21.326 INFO:teuthology.orchestra.run.smithi088.stdout: "rdpos": 0, 2024-04-02T14:04:21.327 INFO:teuthology.orchestra.run.smithi088.stdout: "jlat": { 2024-04-02T14:04:21.327 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 4, 2024-04-02T14:04:21.327 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.008419055, 2024-04-02T14:04:21.327 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.002104763 2024-04-02T14:04:21.327 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.327 INFO:teuthology.orchestra.run.smithi088.stdout: "replayed": 0 2024-04-02T14:04:21.327 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.327 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_mem": { 2024-04-02T14:04:21.327 INFO:teuthology.orchestra.run.smithi088.stdout: "ino": 41, 2024-04-02T14:04:21.328 INFO:teuthology.orchestra.run.smithi088.stdout: "ino+": 46, 2024-04-02T14:04:21.328 INFO:teuthology.orchestra.run.smithi088.stdout: "ino-": 5, 2024-04-02T14:04:21.328 INFO:teuthology.orchestra.run.smithi088.stdout: "dir": 46, 2024-04-02T14:04:21.328 INFO:teuthology.orchestra.run.smithi088.stdout: "dir+": 50, 2024-04-02T14:04:21.328 INFO:teuthology.orchestra.run.smithi088.stdout: "dir-": 4, 2024-04-02T14:04:21.328 INFO:teuthology.orchestra.run.smithi088.stdout: "dn": 37, 2024-04-02T14:04:21.328 INFO:teuthology.orchestra.run.smithi088.stdout: "dn+": 45, 2024-04-02T14:04:21.328 INFO:teuthology.orchestra.run.smithi088.stdout: "dn-": 8, 2024-04-02T14:04:21.328 INFO:teuthology.orchestra.run.smithi088.stdout: "cap": 2, 2024-04-02T14:04:21.329 INFO:teuthology.orchestra.run.smithi088.stdout: "cap+": 3, 2024-04-02T14:04:21.329 INFO:teuthology.orchestra.run.smithi088.stdout: "cap-": 1, 2024-04-02T14:04:21.329 INFO:teuthology.orchestra.run.smithi088.stdout: "rss": 50488, 2024-04-02T14:04:21.329 INFO:teuthology.orchestra.run.smithi088.stdout: "heap": 223608 2024-04-02T14:04:21.329 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.329 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_server": { 2024-04-02T14:04:21.329 INFO:teuthology.orchestra.run.smithi088.stdout: "dispatch_client_request": 100, 2024-04-02T14:04:21.329 INFO:teuthology.orchestra.run.smithi088.stdout: "dispatch_server_request": 0, 2024-04-02T14:04:21.329 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_request": 99, 2024-04-02T14:04:21.330 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_session": 13, 2024-04-02T14:04:21.330 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_peer_request": 0, 2024-04-02T14:04:21.330 INFO:teuthology.orchestra.run.smithi088.stdout: "req_create_latency": { 2024-04-02T14:04:21.330 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.330 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.330 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.330 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.330 INFO:teuthology.orchestra.run.smithi088.stdout: "req_getattr_latency": { 2024-04-02T14:04:21.331 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 1, 2024-04-02T14:04:21.331 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000170058, 2024-04-02T14:04:21.331 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000170058 2024-04-02T14:04:21.331 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.331 INFO:teuthology.orchestra.run.smithi088.stdout: "req_getfilelock_latency": { 2024-04-02T14:04:21.331 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.331 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.331 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.331 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.331 INFO:teuthology.orchestra.run.smithi088.stdout: "req_link_latency": { 2024-04-02T14:04:21.332 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.332 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.332 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.332 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.332 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookup_latency": { 2024-04-02T14:04:21.332 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 1, 2024-04-02T14:04:21.332 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 4.050558031, 2024-04-02T14:04:21.332 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 4.050558031 2024-04-02T14:04:21.332 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.333 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookuphash_latency": { 2024-04-02T14:04:21.333 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.333 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.333 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.333 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.333 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupino_latency": { 2024-04-02T14:04:21.333 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 1, 2024-04-02T14:04:21.333 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000119556, 2024-04-02T14:04:21.334 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000119556 2024-04-02T14:04:21.334 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.334 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupname_latency": { 2024-04-02T14:04:21.334 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.334 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.334 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.334 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.334 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupparent_latency": { 2024-04-02T14:04:21.335 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.335 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.335 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.335 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.335 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupsnap_latency": { 2024-04-02T14:04:21.335 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.335 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.335 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.335 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.336 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lssnap_latency": { 2024-04-02T14:04:21.336 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.336 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.336 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.336 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.336 INFO:teuthology.orchestra.run.smithi088.stdout: "req_mkdir_latency": { 2024-04-02T14:04:21.336 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.336 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.337 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.337 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.337 INFO:teuthology.orchestra.run.smithi088.stdout: "req_mknod_latency": { 2024-04-02T14:04:21.337 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.337 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.337 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.337 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.337 INFO:teuthology.orchestra.run.smithi088.stdout: "req_mksnap_latency": { 2024-04-02T14:04:21.338 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.338 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.338 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.338 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.338 INFO:teuthology.orchestra.run.smithi088.stdout: "req_open_latency": { 2024-04-02T14:04:21.338 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.338 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.339 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.339 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.339 INFO:teuthology.orchestra.run.smithi088.stdout: "req_readdir_latency": { 2024-04-02T14:04:21.339 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.339 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.339 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.339 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.339 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rename_latency": { 2024-04-02T14:04:21.340 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.340 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.340 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.340 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.340 INFO:teuthology.orchestra.run.smithi088.stdout: "req_renamesnap_latency": { 2024-04-02T14:04:21.340 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.340 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.340 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.340 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.341 INFO:teuthology.orchestra.run.smithi088.stdout: "req_snapdiff_latency": { 2024-04-02T14:04:21.341 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.341 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.341 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.341 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.341 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rmdir_latency": { 2024-04-02T14:04:21.341 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.341 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.342 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.342 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.342 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rmsnap_latency": { 2024-04-02T14:04:21.342 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.342 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.343 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.343 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.343 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rmxattr_latency": { 2024-04-02T14:04:21.343 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.343 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.343 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.343 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.343 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setattr_latency": { 2024-04-02T14:04:21.343 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.344 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.344 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.344 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.344 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setdirlayout_latency": { 2024-04-02T14:04:21.344 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.344 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.344 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.344 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.345 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setfilelock_latency": { 2024-04-02T14:04:21.345 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.345 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.345 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.345 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.345 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setlayout_latency": { 2024-04-02T14:04:21.345 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.346 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.346 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.346 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.346 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setxattr_latency": { 2024-04-02T14:04:21.346 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.346 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.346 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.346 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.346 INFO:teuthology.orchestra.run.smithi088.stdout: "req_symlink_latency": { 2024-04-02T14:04:21.347 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.347 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.347 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.347 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.347 INFO:teuthology.orchestra.run.smithi088.stdout: "req_unlink_latency": { 2024-04-02T14:04:21.347 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.347 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.347 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.348 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.348 INFO:teuthology.orchestra.run.smithi088.stdout: "cap_revoke_eviction": 0, 2024-04-02T14:04:21.348 INFO:teuthology.orchestra.run.smithi088.stdout: "cap_acquisition_throttle": 0, 2024-04-02T14:04:21.348 INFO:teuthology.orchestra.run.smithi088.stdout: "req_getvxattr_latency": { 2024-04-02T14:04:21.348 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.348 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.348 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.349 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:21.349 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.349 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_sessions": { 2024-04-02T14:04:21.349 INFO:teuthology.orchestra.run.smithi088.stdout: "session_count": 1, 2024-04-02T14:04:21.349 INFO:teuthology.orchestra.run.smithi088.stdout: "session_add": 2, 2024-04-02T14:04:21.349 INFO:teuthology.orchestra.run.smithi088.stdout: "session_remove": 1, 2024-04-02T14:04:21.349 INFO:teuthology.orchestra.run.smithi088.stdout: "sessions_open": 1, 2024-04-02T14:04:21.349 INFO:teuthology.orchestra.run.smithi088.stdout: "sessions_stale": 0, 2024-04-02T14:04:21.350 INFO:teuthology.orchestra.run.smithi088.stdout: "total_load": 1, 2024-04-02T14:04:21.350 INFO:teuthology.orchestra.run.smithi088.stdout: "average_load": 1, 2024-04-02T14:04:21.350 INFO:teuthology.orchestra.run.smithi088.stdout: "avg_session_uptime": 639, 2024-04-02T14:04:21.350 INFO:teuthology.orchestra.run.smithi088.stdout: "mdthresh_evicted": 0 2024-04-02T14:04:21.350 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.350 INFO:teuthology.orchestra.run.smithi088.stdout: "mempool": { 2024-04-02T14:04:21.350 INFO:teuthology.orchestra.run.smithi088.stdout: "bloom_filter_bytes": 0, 2024-04-02T14:04:21.350 INFO:teuthology.orchestra.run.smithi088.stdout: "bloom_filter_items": 0, 2024-04-02T14:04:21.351 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T14:04:21.351 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_alloc_items": 0, 2024-04-02T14:04:21.351 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T14:04:21.351 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_data_items": 0, 2024-04-02T14:04:21.351 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T14:04:21.351 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T14:04:21.351 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T14:04:21.351 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T14:04:21.352 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T14:04:21.352 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_other_items": 0, 2024-04-02T14:04:21.352 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T14:04:21.352 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T14:04:21.352 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_extent_bytes": 0, 2024-04-02T14:04:21.352 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_extent_items": 0, 2024-04-02T14:04:21.352 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_blob_bytes": 0, 2024-04-02T14:04:21.352 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_blob_items": 0, 2024-04-02T14:04:21.353 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T14:04:21.353 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T14:04:21.353 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T14:04:21.353 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T14:04:21.353 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T14:04:21.353 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_fsck_items": 0, 2024-04-02T14:04:21.353 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_txc_bytes": 0, 2024-04-02T14:04:21.353 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_txc_items": 0, 2024-04-02T14:04:21.353 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T14:04:21.354 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T14:04:21.354 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_bytes": 0, 2024-04-02T14:04:21.354 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_items": 0, 2024-04-02T14:04:21.354 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_bytes": 0, 2024-04-02T14:04:21.354 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_items": 0, 2024-04-02T14:04:21.354 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T14:04:21.354 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_reader_items": 0, 2024-04-02T14:04:21.354 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T14:04:21.355 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_writer_items": 0, 2024-04-02T14:04:21.355 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_anon_bytes": 275836, 2024-04-02T14:04:21.355 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_anon_items": 134, 2024-04-02T14:04:21.355 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_meta_bytes": 0, 2024-04-02T14:04:21.355 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_meta_items": 0, 2024-04-02T14:04:21.355 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_bytes": 0, 2024-04-02T14:04:21.355 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_items": 0, 2024-04-02T14:04:21.355 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_mapbl_bytes": 0, 2024-04-02T14:04:21.355 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_mapbl_items": 0, 2024-04-02T14:04:21.356 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_pglog_bytes": 0, 2024-04-02T14:04:21.356 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_pglog_items": 0, 2024-04-02T14:04:21.356 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_bytes": 9320, 2024-04-02T14:04:21.356 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_items": 178, 2024-04-02T14:04:21.356 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T14:04:21.356 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_mapping_items": 0, 2024-04-02T14:04:21.356 INFO:teuthology.orchestra.run.smithi088.stdout: "pgmap_bytes": 0, 2024-04-02T14:04:21.356 INFO:teuthology.orchestra.run.smithi088.stdout: "pgmap_items": 0, 2024-04-02T14:04:21.356 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_co_bytes": 187922, 2024-04-02T14:04:21.357 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_co_items": 998, 2024-04-02T14:04:21.357 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_1_bytes": 0, 2024-04-02T14:04:21.357 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_1_items": 0, 2024-04-02T14:04:21.357 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_2_bytes": 0, 2024-04-02T14:04:21.357 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_2_items": 0 2024-04-02T14:04:21.357 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.357 INFO:teuthology.orchestra.run.smithi088.stdout: "objecter": { 2024-04-02T14:04:21.357 INFO:teuthology.orchestra.run.smithi088.stdout: "op_active": 0, 2024-04-02T14:04:21.358 INFO:teuthology.orchestra.run.smithi088.stdout: "op_laggy": 0, 2024-04-02T14:04:21.358 INFO:teuthology.orchestra.run.smithi088.stdout: "op_send": 42, 2024-04-02T14:04:21.358 INFO:teuthology.orchestra.run.smithi088.stdout: "op_send_bytes": 12320, 2024-04-02T14:04:21.358 INFO:teuthology.orchestra.run.smithi088.stdout: "op_resend": 31, 2024-04-02T14:04:21.358 INFO:teuthology.orchestra.run.smithi088.stdout: "op_reply": 42, 2024-04-02T14:04:21.358 INFO:teuthology.orchestra.run.smithi088.stdout: "op_latency": { 2024-04-02T14:04:21.358 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 42, 2024-04-02T14:04:21.358 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.208995841, 2024-04-02T14:04:21.359 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.004976091 2024-04-02T14:04:21.359 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.359 INFO:teuthology.orchestra.run.smithi088.stdout: "op_inflight": 0, 2024-04-02T14:04:21.359 INFO:teuthology.orchestra.run.smithi088.stdout: "oplen_avg": { 2024-04-02T14:04:21.359 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 42, 2024-04-02T14:04:21.360 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 66 2024-04-02T14:04:21.360 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.360 INFO:teuthology.orchestra.run.smithi088.stdout: "op": 42, 2024-04-02T14:04:21.360 INFO:teuthology.orchestra.run.smithi088.stdout: "op_r": 0, 2024-04-02T14:04:21.360 INFO:teuthology.orchestra.run.smithi088.stdout: "op_w": 42, 2024-04-02T14:04:21.360 INFO:teuthology.orchestra.run.smithi088.stdout: "op_rmw": 0, 2024-04-02T14:04:21.360 INFO:teuthology.orchestra.run.smithi088.stdout: "op_pg": 0, 2024-04-02T14:04:21.360 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_stat": 0, 2024-04-02T14:04:21.360 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_create": 10, 2024-04-02T14:04:21.361 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_read": 0, 2024-04-02T14:04:21.361 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_write": 4, 2024-04-02T14:04:21.361 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_writefull": 7, 2024-04-02T14:04:21.361 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_writesame": 0, 2024-04-02T14:04:21.361 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_append": 0, 2024-04-02T14:04:21.361 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_zero": 0, 2024-04-02T14:04:21.361 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_truncate": 0, 2024-04-02T14:04:21.361 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_delete": 6, 2024-04-02T14:04:21.361 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_mapext": 0, 2024-04-02T14:04:21.362 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_sparse_read": 0, 2024-04-02T14:04:21.362 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_clonerange": 0, 2024-04-02T14:04:21.362 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_getxattr": 0, 2024-04-02T14:04:21.362 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_setxattr": 20, 2024-04-02T14:04:21.362 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_cmpxattr": 0, 2024-04-02T14:04:21.362 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_rmxattr": 0, 2024-04-02T14:04:21.362 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_resetxattrs": 0, 2024-04-02T14:04:21.362 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_call": 0, 2024-04-02T14:04:21.362 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_watch": 0, 2024-04-02T14:04:21.363 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_notify": 0, 2024-04-02T14:04:21.363 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T14:04:21.363 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_pgls": 0, 2024-04-02T14:04:21.363 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_pgls_filter": 0, 2024-04-02T14:04:21.363 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_other": 0, 2024-04-02T14:04:21.363 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_active": 0, 2024-04-02T14:04:21.363 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_send": 0, 2024-04-02T14:04:21.363 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_resend": 0, 2024-04-02T14:04:21.364 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_ping": 0, 2024-04-02T14:04:21.364 INFO:teuthology.orchestra.run.smithi088.stdout: "poolop_active": 0, 2024-04-02T14:04:21.364 INFO:teuthology.orchestra.run.smithi088.stdout: "poolop_send": 0, 2024-04-02T14:04:21.364 INFO:teuthology.orchestra.run.smithi088.stdout: "poolop_resend": 0, 2024-04-02T14:04:21.364 INFO:teuthology.orchestra.run.smithi088.stdout: "poolstat_active": 0, 2024-04-02T14:04:21.364 INFO:teuthology.orchestra.run.smithi088.stdout: "poolstat_send": 0, 2024-04-02T14:04:21.364 INFO:teuthology.orchestra.run.smithi088.stdout: "poolstat_resend": 0, 2024-04-02T14:04:21.364 INFO:teuthology.orchestra.run.smithi088.stdout: "statfs_active": 0, 2024-04-02T14:04:21.364 INFO:teuthology.orchestra.run.smithi088.stdout: "statfs_send": 0, 2024-04-02T14:04:21.365 INFO:teuthology.orchestra.run.smithi088.stdout: "statfs_resend": 0, 2024-04-02T14:04:21.365 INFO:teuthology.orchestra.run.smithi088.stdout: "command_active": 0, 2024-04-02T14:04:21.365 INFO:teuthology.orchestra.run.smithi088.stdout: "command_send": 0, 2024-04-02T14:04:21.365 INFO:teuthology.orchestra.run.smithi088.stdout: "command_resend": 0, 2024-04-02T14:04:21.365 INFO:teuthology.orchestra.run.smithi088.stdout: "map_epoch": 103, 2024-04-02T14:04:21.365 INFO:teuthology.orchestra.run.smithi088.stdout: "map_full": 0, 2024-04-02T14:04:21.365 INFO:teuthology.orchestra.run.smithi088.stdout: "map_inc": 13, 2024-04-02T14:04:21.365 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_sessions": 9, 2024-04-02T14:04:21.365 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_session_open": 9, 2024-04-02T14:04:21.366 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_session_close": 0, 2024-04-02T14:04:21.366 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_laggy": 0, 2024-04-02T14:04:21.366 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_wr": 18, 2024-04-02T14:04:21.366 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_rd": 0, 2024-04-02T14:04:21.366 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_del": 1 2024-04-02T14:04:21.366 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.366 INFO:teuthology.orchestra.run.smithi088.stdout: "oft": { 2024-04-02T14:04:21.366 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_objs": 1, 2024-04-02T14:04:21.367 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_kv_pairs": 0, 2024-04-02T14:04:21.367 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_updates": 0, 2024-04-02T14:04:21.367 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_removes": 0 2024-04-02T14:04:21.367 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.367 INFO:teuthology.orchestra.run.smithi088.stdout: "osd-slow-ops": { 2024-04-02T14:04:21.367 INFO:teuthology.orchestra.run.smithi088.stdout: "slow_ops_count": 0 2024-04-02T14:04:21.367 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.367 INFO:teuthology.orchestra.run.smithi088.stdout: "purge_queue": { 2024-04-02T14:04:21.368 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing_ops": 0, 2024-04-02T14:04:21.368 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T14:04:21.368 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing": 0, 2024-04-02T14:04:21.368 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing_high_water": 0, 2024-04-02T14:04:21.368 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executed_ops": 0, 2024-04-02T14:04:21.368 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executed": 0, 2024-04-02T14:04:21.368 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_item_in_journal": 0 2024-04-02T14:04:21.368 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.368 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T14:04:21.369 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-02T14:04:21.369 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 104857600, 2024-04-02T14:04:21.369 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-02T14:04:21.369 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 1646, 2024-04-02T14:04:21.369 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 1932410, 2024-04-02T14:04:21.369 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:21.369 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 1646, 2024-04-02T14:04:21.369 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 0, 2024-04-02T14:04:21.369 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 0, 2024-04-02T14:04:21.370 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 1646, 2024-04-02T14:04:21.370 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 1932410, 2024-04-02T14:04:21.370 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-02T14:04:21.370 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.370 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.370 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.371 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:21.371 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.371 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-objecter_bytes": { 2024-04-02T14:04:21.371 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-02T14:04:21.371 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 104857600, 2024-04-02T14:04:21.371 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-02T14:04:21.371 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 0, 2024-04-02T14:04:21.371 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 0, 2024-04-02T14:04:21.372 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:21.372 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:21.372 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 42, 2024-04-02T14:04:21.372 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 12320, 2024-04-02T14:04:21.372 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 42, 2024-04-02T14:04:21.372 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 12320, 2024-04-02T14:04:21.372 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-02T14:04:21.373 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.373 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.373 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.373 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:21.373 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.374 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-objecter_ops": { 2024-04-02T14:04:21.374 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-02T14:04:21.374 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 1024, 2024-04-02T14:04:21.374 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-02T14:04:21.374 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 0, 2024-04-02T14:04:21.374 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 0, 2024-04-02T14:04:21.374 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:21.374 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:21.374 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 42, 2024-04-02T14:04:21.375 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 42, 2024-04-02T14:04:21.375 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 42, 2024-04-02T14:04:21.375 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 42, 2024-04-02T14:04:21.375 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-02T14:04:21.375 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.375 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.375 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.375 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:21.376 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.376 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-write_buf_throttle": { 2024-04-02T14:04:21.376 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-02T14:04:21.376 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 3758096384, 2024-04-02T14:04:21.376 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-02T14:04:21.376 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 0, 2024-04-02T14:04:21.376 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 0, 2024-04-02T14:04:21.376 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:21.377 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:21.377 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 0, 2024-04-02T14:04:21.377 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 0, 2024-04-02T14:04:21.377 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 0, 2024-04-02T14:04:21.377 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 0, 2024-04-02T14:04:21.377 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-02T14:04:21.377 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.377 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.378 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.378 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:21.378 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:21.378 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-write_buf_throttle-0x558918be4500": { 2024-04-02T14:04:21.378 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-02T14:04:21.378 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 3758096384, 2024-04-02T14:04:21.378 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-02T14:04:21.378 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 5, 2024-04-02T14:04:21.378 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 1428, 2024-04-02T14:04:21.379 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:21.379 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 5, 2024-04-02T14:04:21.379 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 0, 2024-04-02T14:04:21.379 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 0, 2024-04-02T14:04:21.379 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 4, 2024-04-02T14:04:21.379 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 1428, 2024-04-02T14:04:21.379 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-02T14:04:21.379 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:21.379 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:21.379 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:21.380 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:21.380 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:21.380 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-02T14:04:21.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:21 smithi088 ceph-mon[36528]: pgmap v338: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 212 B/s wr, 10 op/s 2024-04-02T14:04:21.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:21 smithi189 ceph-mon[36683]: pgmap v338: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 212 B/s wr, 10 op/s 2024-04-02T14:04:21.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:21 smithi052 ceph-mon[32104]: pgmap v338: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 212 B/s wr, 10 op/s 2024-04-02T14:04:22.081 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-02T14:04:22.081 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-02T14:04:22.081 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-02T14:04:22.082 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=71 2024-04-02T14:04:22.082 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-02T14:04:22.082 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-02T14:04:22.082 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T14:04:22.082 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-04-02T14:04:22.082 DEBUG:tasks.check_counter:Getting stats from k 2024-04-02T14:04:22.083 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph daemon mds.k perf dump 2024-04-02T14:04:22.320 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T14:04:23.739 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:23 smithi088 ceph-mon[36528]: pgmap v339: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:04:23.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:23 smithi189 ceph-mon[36683]: pgmap v339: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:04:23.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:23 smithi052 ceph-mon[32104]: pgmap v339: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:04:24.080 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-02T14:04:24.080 INFO:teuthology.orchestra.run.smithi088.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T14:04:24.081 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_messages": 790, 2024-04-02T14:04:24.081 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages": 792, 2024-04-02T14:04:24.081 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_bytes": 261951, 2024-04-02T14:04:24.081 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_bytes": 251284, 2024-04-02T14:04:24.081 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_created_connections": 6, 2024-04-02T14:04:24.081 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_active_connections": 2, 2024-04-02T14:04:24.081 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_total_time": 0.073039834, 2024-04-02T14:04:24.081 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_send_time": 0.029428562, 2024-04-02T14:04:24.082 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_recv_time": 0.016929328, 2024-04-02T14:04:24.082 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_fast_dispatch_time": 0.018546070, 2024-04-02T14:04:24.082 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:24.082 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 790, 2024-04-02T14:04:24.082 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.030995114, 2024-04-02T14:04:24.082 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000039234 2024-04-02T14:04:24.082 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.082 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:24.083 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.083 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.083 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.083 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.083 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T14:04:24.083 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T14:04:24.083 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.084 INFO:teuthology.orchestra.run.smithi088.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T14:04:24.084 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_messages": 911, 2024-04-02T14:04:24.084 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages": 897, 2024-04-02T14:04:24.084 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_bytes": 1802090, 2024-04-02T14:04:24.084 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_bytes": 349401, 2024-04-02T14:04:24.084 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_created_connections": 4, 2024-04-02T14:04:24.084 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_active_connections": 3, 2024-04-02T14:04:24.084 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_total_time": 0.107566678, 2024-04-02T14:04:24.085 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_send_time": 0.032984423, 2024-04-02T14:04:24.085 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_recv_time": 0.051031422, 2024-04-02T14:04:24.085 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_fast_dispatch_time": 0.027524787, 2024-04-02T14:04:24.085 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:24.085 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 896, 2024-04-02T14:04:24.085 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.037578259, 2024-04-02T14:04:24.085 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000041940 2024-04-02T14:04:24.085 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.086 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:24.086 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.086 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.086 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.086 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.086 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_encrypted_bytes": 1641184, 2024-04-02T14:04:24.086 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_encrypted_bytes": 141424 2024-04-02T14:04:24.086 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.087 INFO:teuthology.orchestra.run.smithi088.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T14:04:24.087 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_messages": 780, 2024-04-02T14:04:24.087 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages": 948, 2024-04-02T14:04:24.087 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_bytes": 263456, 2024-04-02T14:04:24.087 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_bytes": 700004, 2024-04-02T14:04:24.087 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_created_connections": 6, 2024-04-02T14:04:24.087 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_active_connections": 4, 2024-04-02T14:04:24.087 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_total_time": 0.103188784, 2024-04-02T14:04:24.088 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_send_time": 0.035541733, 2024-04-02T14:04:24.088 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_recv_time": 0.025722695, 2024-04-02T14:04:24.088 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_fast_dispatch_time": 0.030541804, 2024-04-02T14:04:24.088 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:24.088 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 946, 2024-04-02T14:04:24.088 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.059884464, 2024-04-02T14:04:24.088 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000063302 2024-04-02T14:04:24.088 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.089 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:24.089 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.089 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.089 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.089 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.089 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-04-02T14:04:24.089 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_encrypted_bytes": 453872 2024-04-02T14:04:24.089 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.089 INFO:teuthology.orchestra.run.smithi088.stdout: "cct": { 2024-04-02T14:04:24.090 INFO:teuthology.orchestra.run.smithi088.stdout: "total_workers": 1, 2024-04-02T14:04:24.090 INFO:teuthology.orchestra.run.smithi088.stdout: "unhealthy_workers": 0 2024-04-02T14:04:24.090 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.090 INFO:teuthology.orchestra.run.smithi088.stdout: "finisher-MDSRank": { 2024-04-02T14:04:24.090 INFO:teuthology.orchestra.run.smithi088.stdout: "queue_len": 0, 2024-04-02T14:04:24.090 INFO:teuthology.orchestra.run.smithi088.stdout: "complete_latency": { 2024-04-02T14:04:24.090 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 3696, 2024-04-02T14:04:24.090 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.109783049, 2024-04-02T14:04:24.091 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000029703 2024-04-02T14:04:24.091 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:24.091 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.091 INFO:teuthology.orchestra.run.smithi088.stdout: "finisher-PurgeQueue": { 2024-04-02T14:04:24.091 INFO:teuthology.orchestra.run.smithi088.stdout: "queue_len": 0, 2024-04-02T14:04:24.091 INFO:teuthology.orchestra.run.smithi088.stdout: "complete_latency": { 2024-04-02T14:04:24.091 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.091 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.091 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.092 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:24.092 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.092 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-04-02T14:04:24.092 INFO:teuthology.orchestra.run.smithi088.stdout: "request": 0, 2024-04-02T14:04:24.092 INFO:teuthology.orchestra.run.smithi088.stdout: "reply": 0, 2024-04-02T14:04:24.092 INFO:teuthology.orchestra.run.smithi088.stdout: "reply_latency": { 2024-04-02T14:04:24.092 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.092 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.093 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.093 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.093 INFO:teuthology.orchestra.run.smithi088.stdout: "slow_reply": 0, 2024-04-02T14:04:24.093 INFO:teuthology.orchestra.run.smithi088.stdout: "forward": 0, 2024-04-02T14:04:24.093 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_fetch_complete": 0, 2024-04-02T14:04:24.093 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_fetch_keys": 0, 2024-04-02T14:04:24.093 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_commit": 0, 2024-04-02T14:04:24.093 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_split": 0, 2024-04-02T14:04:24.094 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_merge": 0, 2024-04-02T14:04:24.094 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes": 26, 2024-04-02T14:04:24.094 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_top": 2, 2024-04-02T14:04:24.094 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_bottom": 24, 2024-04-02T14:04:24.094 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_pin_tail": 0, 2024-04-02T14:04:24.095 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_pinned": 22, 2024-04-02T14:04:24.095 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_expired": 0, 2024-04-02T14:04:24.095 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_with_caps": 0, 2024-04-02T14:04:24.095 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": 0, 2024-04-02T14:04:24.095 INFO:teuthology.orchestra.run.smithi088.stdout: "subtrees": 4, 2024-04-02T14:04:24.095 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse": 0, 2024-04-02T14:04:24.095 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_hit": 0, 2024-04-02T14:04:24.095 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_forward": 0, 2024-04-02T14:04:24.095 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_discover": 0, 2024-04-02T14:04:24.095 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_dir_fetch": 0, 2024-04-02T14:04:24.096 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_remote_ino": 0, 2024-04-02T14:04:24.096 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_lock": 0, 2024-04-02T14:04:24.096 INFO:teuthology.orchestra.run.smithi088.stdout: "load_cent": 0, 2024-04-02T14:04:24.096 INFO:teuthology.orchestra.run.smithi088.stdout: "q": 0, 2024-04-02T14:04:24.096 INFO:teuthology.orchestra.run.smithi088.stdout: "exported": 0, 2024-04-02T14:04:24.096 INFO:teuthology.orchestra.run.smithi088.stdout: "exported_inodes": 0, 2024-04-02T14:04:24.096 INFO:teuthology.orchestra.run.smithi088.stdout: "imported": 0, 2024-04-02T14:04:24.096 INFO:teuthology.orchestra.run.smithi088.stdout: "imported_inodes": 0, 2024-04-02T14:04:24.096 INFO:teuthology.orchestra.run.smithi088.stdout: "openino_dir_fetch": 0, 2024-04-02T14:04:24.097 INFO:teuthology.orchestra.run.smithi088.stdout: "openino_backtrace_fetch": 0, 2024-04-02T14:04:24.097 INFO:teuthology.orchestra.run.smithi088.stdout: "openino_peer_discover": 0, 2024-04-02T14:04:24.097 INFO:teuthology.orchestra.run.smithi088.stdout: "root_rfiles": 0, 2024-04-02T14:04:24.097 INFO:teuthology.orchestra.run.smithi088.stdout: "root_rbytes": 0, 2024-04-02T14:04:24.097 INFO:teuthology.orchestra.run.smithi088.stdout: "root_rsnaps": 0, 2024-04-02T14:04:24.097 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T14:04:24.097 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_set_tag": 0, 2024-04-02T14:04:24.097 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T14:04:24.098 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_inotable_repaired": 0, 2024-04-02T14:04:24.098 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_dir_inodes": 0, 2024-04-02T14:04:24.098 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T14:04:24.098 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T14:04:24.098 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_file_inodes": 0, 2024-04-02T14:04:24.098 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_inode_file_caps": 0, 2024-04-02T14:04:24.098 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T14:04:24.098 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_grant": 0, 2024-04-02T14:04:24.099 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T14:04:24.099 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T14:04:24.099 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T14:04:24.099 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_caps": 0, 2024-04-02T14:04:24.099 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_caps_dirty": 0, 2024-04-02T14:04:24.099 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_cap_release": 0, 2024-04-02T14:04:24.099 INFO:teuthology.orchestra.run.smithi088.stdout: "process_request_cap_release": 0 2024-04-02T14:04:24.099 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.099 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_cache": { 2024-04-02T14:04:24.100 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_update": 0, 2024-04-02T14:04:24.100 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_update_receipt": 0, 2024-04-02T14:04:24.100 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_try_discover": 0, 2024-04-02T14:04:24.100 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_send_discover": 0, 2024-04-02T14:04:24.100 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_handle_discover": 0, 2024-04-02T14:04:24.100 INFO:teuthology.orchestra.run.smithi088.stdout: "num_strays": 0, 2024-04-02T14:04:24.100 INFO:teuthology.orchestra.run.smithi088.stdout: "num_strays_delayed": 0, 2024-04-02T14:04:24.100 INFO:teuthology.orchestra.run.smithi088.stdout: "num_strays_enqueuing": 0, 2024-04-02T14:04:24.101 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_created": 0, 2024-04-02T14:04:24.101 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_enqueued": 0, 2024-04-02T14:04:24.101 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_reintegrated": 0, 2024-04-02T14:04:24.101 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_migrated": 0, 2024-04-02T14:04:24.101 INFO:teuthology.orchestra.run.smithi088.stdout: "num_recovering_processing": 0, 2024-04-02T14:04:24.101 INFO:teuthology.orchestra.run.smithi088.stdout: "num_recovering_enqueued": 0, 2024-04-02T14:04:24.101 INFO:teuthology.orchestra.run.smithi088.stdout: "num_recovering_prioritized": 0, 2024-04-02T14:04:24.101 INFO:teuthology.orchestra.run.smithi088.stdout: "recovery_started": 0, 2024-04-02T14:04:24.102 INFO:teuthology.orchestra.run.smithi088.stdout: "recovery_completed": 0, 2024-04-02T14:04:24.102 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_quiesce_path": 0, 2024-04-02T14:04:24.102 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_quiesce_inode": 0, 2024-04-02T14:04:24.102 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T14:04:24.102 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_exportdir": 0, 2024-04-02T14:04:24.102 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_flush": 0, 2024-04-02T14:04:24.102 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_fragmentdir": 0, 2024-04-02T14:04:24.102 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_fragstats": 0, 2024-04-02T14:04:24.103 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_inodestats": 0 2024-04-02T14:04:24.103 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.103 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_log": { 2024-04-02T14:04:24.103 INFO:teuthology.orchestra.run.smithi088.stdout: "evlrg": 0, 2024-04-02T14:04:24.103 INFO:teuthology.orchestra.run.smithi088.stdout: "evadd": 0, 2024-04-02T14:04:24.103 INFO:teuthology.orchestra.run.smithi088.stdout: "evex": 0, 2024-04-02T14:04:24.103 INFO:teuthology.orchestra.run.smithi088.stdout: "evtrm": 0, 2024-04-02T14:04:24.104 INFO:teuthology.orchestra.run.smithi088.stdout: "ev": 24, 2024-04-02T14:04:24.104 INFO:teuthology.orchestra.run.smithi088.stdout: "evexg": 0, 2024-04-02T14:04:24.104 INFO:teuthology.orchestra.run.smithi088.stdout: "evexd": 0, 2024-04-02T14:04:24.104 INFO:teuthology.orchestra.run.smithi088.stdout: "segadd": 0, 2024-04-02T14:04:24.104 INFO:teuthology.orchestra.run.smithi088.stdout: "segex": 0, 2024-04-02T14:04:24.104 INFO:teuthology.orchestra.run.smithi088.stdout: "segtrm": 0, 2024-04-02T14:04:24.104 INFO:teuthology.orchestra.run.smithi088.stdout: "seg": 2, 2024-04-02T14:04:24.104 INFO:teuthology.orchestra.run.smithi088.stdout: "segmjr": 2, 2024-04-02T14:04:24.105 INFO:teuthology.orchestra.run.smithi088.stdout: "segexg": 0, 2024-04-02T14:04:24.105 INFO:teuthology.orchestra.run.smithi088.stdout: "segexd": 0, 2024-04-02T14:04:24.105 INFO:teuthology.orchestra.run.smithi088.stdout: "expos": 4194304, 2024-04-02T14:04:24.105 INFO:teuthology.orchestra.run.smithi088.stdout: "wrpos": 4248844, 2024-04-02T14:04:24.105 INFO:teuthology.orchestra.run.smithi088.stdout: "rdpos": 4248270, 2024-04-02T14:04:24.105 INFO:teuthology.orchestra.run.smithi088.stdout: "jlat": { 2024-04-02T14:04:24.105 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.105 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.106 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.106 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.106 INFO:teuthology.orchestra.run.smithi088.stdout: "replayed": 24 2024-04-02T14:04:24.106 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.106 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_mem": { 2024-04-02T14:04:24.106 INFO:teuthology.orchestra.run.smithi088.stdout: "ino": 24, 2024-04-02T14:04:24.106 INFO:teuthology.orchestra.run.smithi088.stdout: "ino+": 25, 2024-04-02T14:04:24.106 INFO:teuthology.orchestra.run.smithi088.stdout: "ino-": 1, 2024-04-02T14:04:24.106 INFO:teuthology.orchestra.run.smithi088.stdout: "dir": 17, 2024-04-02T14:04:24.107 INFO:teuthology.orchestra.run.smithi088.stdout: "dir+": 17, 2024-04-02T14:04:24.107 INFO:teuthology.orchestra.run.smithi088.stdout: "dir-": 0, 2024-04-02T14:04:24.107 INFO:teuthology.orchestra.run.smithi088.stdout: "dn": 26, 2024-04-02T14:04:24.107 INFO:teuthology.orchestra.run.smithi088.stdout: "dn+": 26, 2024-04-02T14:04:24.107 INFO:teuthology.orchestra.run.smithi088.stdout: "dn-": 0, 2024-04-02T14:04:24.107 INFO:teuthology.orchestra.run.smithi088.stdout: "cap": 0, 2024-04-02T14:04:24.107 INFO:teuthology.orchestra.run.smithi088.stdout: "cap+": 0, 2024-04-02T14:04:24.107 INFO:teuthology.orchestra.run.smithi088.stdout: "cap-": 0, 2024-04-02T14:04:24.107 INFO:teuthology.orchestra.run.smithi088.stdout: "rss": 50340, 2024-04-02T14:04:24.108 INFO:teuthology.orchestra.run.smithi088.stdout: "heap": 215416 2024-04-02T14:04:24.108 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.108 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_server": { 2024-04-02T14:04:24.108 INFO:teuthology.orchestra.run.smithi088.stdout: "dispatch_client_request": 0, 2024-04-02T14:04:24.108 INFO:teuthology.orchestra.run.smithi088.stdout: "dispatch_server_request": 0, 2024-04-02T14:04:24.108 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_request": 0, 2024-04-02T14:04:24.108 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_session": 0, 2024-04-02T14:04:24.108 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_peer_request": 0, 2024-04-02T14:04:24.109 INFO:teuthology.orchestra.run.smithi088.stdout: "req_create_latency": { 2024-04-02T14:04:24.109 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.109 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.109 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.109 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.109 INFO:teuthology.orchestra.run.smithi088.stdout: "req_getattr_latency": { 2024-04-02T14:04:24.109 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.109 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.110 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.110 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.110 INFO:teuthology.orchestra.run.smithi088.stdout: "req_getfilelock_latency": { 2024-04-02T14:04:24.110 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.110 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.110 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.110 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.111 INFO:teuthology.orchestra.run.smithi088.stdout: "req_link_latency": { 2024-04-02T14:04:24.111 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.111 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.111 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.111 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.111 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookup_latency": { 2024-04-02T14:04:24.111 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.111 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.112 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.112 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.112 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookuphash_latency": { 2024-04-02T14:04:24.112 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.112 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.112 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.112 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.112 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupino_latency": { 2024-04-02T14:04:24.113 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.113 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.113 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.113 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.113 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupname_latency": { 2024-04-02T14:04:24.113 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.113 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.113 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.114 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.114 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupparent_latency": { 2024-04-02T14:04:24.114 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.114 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.114 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.114 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.114 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupsnap_latency": { 2024-04-02T14:04:24.115 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.115 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.115 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.115 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.115 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lssnap_latency": { 2024-04-02T14:04:24.115 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.115 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.115 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.116 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.116 INFO:teuthology.orchestra.run.smithi088.stdout: "req_mkdir_latency": { 2024-04-02T14:04:24.116 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.117 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.117 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.117 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.117 INFO:teuthology.orchestra.run.smithi088.stdout: "req_mknod_latency": { 2024-04-02T14:04:24.117 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.118 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.118 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.118 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.118 INFO:teuthology.orchestra.run.smithi088.stdout: "req_mksnap_latency": { 2024-04-02T14:04:24.118 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.118 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.118 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.118 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.119 INFO:teuthology.orchestra.run.smithi088.stdout: "req_open_latency": { 2024-04-02T14:04:24.119 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.119 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.119 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.119 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.119 INFO:teuthology.orchestra.run.smithi088.stdout: "req_readdir_latency": { 2024-04-02T14:04:24.119 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.119 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.119 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.120 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.120 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rename_latency": { 2024-04-02T14:04:24.120 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.120 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.120 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.120 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.120 INFO:teuthology.orchestra.run.smithi088.stdout: "req_renamesnap_latency": { 2024-04-02T14:04:24.120 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.120 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.120 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.121 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.121 INFO:teuthology.orchestra.run.smithi088.stdout: "req_snapdiff_latency": { 2024-04-02T14:04:24.121 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.121 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.121 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.121 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.121 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rmdir_latency": { 2024-04-02T14:04:24.121 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.121 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.122 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.122 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.122 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rmsnap_latency": { 2024-04-02T14:04:24.122 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.122 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.122 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.122 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.122 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rmxattr_latency": { 2024-04-02T14:04:24.122 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.123 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.123 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.123 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.123 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setattr_latency": { 2024-04-02T14:04:24.123 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.123 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.123 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.123 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.123 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setdirlayout_latency": { 2024-04-02T14:04:24.124 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.124 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.124 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.124 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.124 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setfilelock_latency": { 2024-04-02T14:04:24.124 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.124 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.124 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.125 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.125 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setlayout_latency": { 2024-04-02T14:04:24.125 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.125 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.125 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.125 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.125 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setxattr_latency": { 2024-04-02T14:04:24.125 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.126 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.126 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.126 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.126 INFO:teuthology.orchestra.run.smithi088.stdout: "req_symlink_latency": { 2024-04-02T14:04:24.126 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.126 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.126 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.126 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.126 INFO:teuthology.orchestra.run.smithi088.stdout: "req_unlink_latency": { 2024-04-02T14:04:24.127 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.127 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.127 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.127 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.127 INFO:teuthology.orchestra.run.smithi088.stdout: "cap_revoke_eviction": 0, 2024-04-02T14:04:24.127 INFO:teuthology.orchestra.run.smithi088.stdout: "cap_acquisition_throttle": 0, 2024-04-02T14:04:24.127 INFO:teuthology.orchestra.run.smithi088.stdout: "req_getvxattr_latency": { 2024-04-02T14:04:24.127 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.127 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.128 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.128 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:24.128 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.128 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_sessions": { 2024-04-02T14:04:24.128 INFO:teuthology.orchestra.run.smithi088.stdout: "session_count": 1, 2024-04-02T14:04:24.128 INFO:teuthology.orchestra.run.smithi088.stdout: "session_add": 2, 2024-04-02T14:04:24.128 INFO:teuthology.orchestra.run.smithi088.stdout: "session_remove": 1, 2024-04-02T14:04:24.128 INFO:teuthology.orchestra.run.smithi088.stdout: "sessions_open": 1, 2024-04-02T14:04:24.129 INFO:teuthology.orchestra.run.smithi088.stdout: "sessions_stale": 0, 2024-04-02T14:04:24.129 INFO:teuthology.orchestra.run.smithi088.stdout: "total_load": 0, 2024-04-02T14:04:24.129 INFO:teuthology.orchestra.run.smithi088.stdout: "average_load": 0, 2024-04-02T14:04:24.129 INFO:teuthology.orchestra.run.smithi088.stdout: "avg_session_uptime": 2021, 2024-04-02T14:04:24.129 INFO:teuthology.orchestra.run.smithi088.stdout: "mdthresh_evicted": 0 2024-04-02T14:04:24.129 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.129 INFO:teuthology.orchestra.run.smithi088.stdout: "mempool": { 2024-04-02T14:04:24.129 INFO:teuthology.orchestra.run.smithi088.stdout: "bloom_filter_bytes": 0, 2024-04-02T14:04:24.130 INFO:teuthology.orchestra.run.smithi088.stdout: "bloom_filter_items": 0, 2024-04-02T14:04:24.130 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T14:04:24.130 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_alloc_items": 0, 2024-04-02T14:04:24.130 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T14:04:24.130 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_data_items": 0, 2024-04-02T14:04:24.130 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T14:04:24.130 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T14:04:24.130 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T14:04:24.131 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T14:04:24.131 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T14:04:24.131 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_other_items": 0, 2024-04-02T14:04:24.131 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T14:04:24.131 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T14:04:24.131 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_extent_bytes": 0, 2024-04-02T14:04:24.131 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_extent_items": 0, 2024-04-02T14:04:24.132 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_blob_bytes": 0, 2024-04-02T14:04:24.132 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_blob_items": 0, 2024-04-02T14:04:24.132 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T14:04:24.132 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T14:04:24.132 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T14:04:24.132 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T14:04:24.132 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T14:04:24.132 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_fsck_items": 0, 2024-04-02T14:04:24.133 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_txc_bytes": 0, 2024-04-02T14:04:24.133 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_txc_items": 0, 2024-04-02T14:04:24.133 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T14:04:24.133 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T14:04:24.133 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_bytes": 0, 2024-04-02T14:04:24.133 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_items": 0, 2024-04-02T14:04:24.133 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_bytes": 0, 2024-04-02T14:04:24.133 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_items": 0, 2024-04-02T14:04:24.134 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T14:04:24.134 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_reader_items": 0, 2024-04-02T14:04:24.134 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T14:04:24.134 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_writer_items": 0, 2024-04-02T14:04:24.134 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_anon_bytes": 190860, 2024-04-02T14:04:24.134 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_anon_items": 72, 2024-04-02T14:04:24.134 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_meta_bytes": 0, 2024-04-02T14:04:24.135 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_meta_items": 0, 2024-04-02T14:04:24.135 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_bytes": 0, 2024-04-02T14:04:24.135 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_items": 0, 2024-04-02T14:04:24.135 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_mapbl_bytes": 0, 2024-04-02T14:04:24.135 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_mapbl_items": 0, 2024-04-02T14:04:24.135 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_pglog_bytes": 0, 2024-04-02T14:04:24.135 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_pglog_items": 0, 2024-04-02T14:04:24.136 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_bytes": 9320, 2024-04-02T14:04:24.136 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_items": 178, 2024-04-02T14:04:24.136 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T14:04:24.136 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_mapping_items": 0, 2024-04-02T14:04:24.136 INFO:teuthology.orchestra.run.smithi088.stdout: "pgmap_bytes": 0, 2024-04-02T14:04:24.136 INFO:teuthology.orchestra.run.smithi088.stdout: "pgmap_items": 0, 2024-04-02T14:04:24.136 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_co_bytes": 101072, 2024-04-02T14:04:24.136 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_co_items": 661, 2024-04-02T14:04:24.136 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_1_bytes": 0, 2024-04-02T14:04:24.137 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_1_items": 0, 2024-04-02T14:04:24.137 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_2_bytes": 0, 2024-04-02T14:04:24.137 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_2_items": 0 2024-04-02T14:04:24.137 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.137 INFO:teuthology.orchestra.run.smithi088.stdout: "objecter": { 2024-04-02T14:04:24.137 INFO:teuthology.orchestra.run.smithi088.stdout: "op_active": 0, 2024-04-02T14:04:24.137 INFO:teuthology.orchestra.run.smithi088.stdout: "op_laggy": 0, 2024-04-02T14:04:24.138 INFO:teuthology.orchestra.run.smithi088.stdout: "op_send": 2218, 2024-04-02T14:04:24.138 INFO:teuthology.orchestra.run.smithi088.stdout: "op_send_bytes": 21, 2024-04-02T14:04:24.138 INFO:teuthology.orchestra.run.smithi088.stdout: "op_resend": 3, 2024-04-02T14:04:24.138 INFO:teuthology.orchestra.run.smithi088.stdout: "op_reply": 2218, 2024-04-02T14:04:24.138 INFO:teuthology.orchestra.run.smithi088.stdout: "op_latency": { 2024-04-02T14:04:24.138 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 2218, 2024-04-02T14:04:24.138 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 1.141977152, 2024-04-02T14:04:24.139 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000514867 2024-04-02T14:04:24.139 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.139 INFO:teuthology.orchestra.run.smithi088.stdout: "op_inflight": 0, 2024-04-02T14:04:24.139 INFO:teuthology.orchestra.run.smithi088.stdout: "oplen_avg": { 2024-04-02T14:04:24.139 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 2218, 2024-04-02T14:04:24.139 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 2219 2024-04-02T14:04:24.139 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.139 INFO:teuthology.orchestra.run.smithi088.stdout: "op": 2218, 2024-04-02T14:04:24.139 INFO:teuthology.orchestra.run.smithi088.stdout: "op_r": 2218, 2024-04-02T14:04:24.140 INFO:teuthology.orchestra.run.smithi088.stdout: "op_w": 0, 2024-04-02T14:04:24.140 INFO:teuthology.orchestra.run.smithi088.stdout: "op_rmw": 0, 2024-04-02T14:04:24.140 INFO:teuthology.orchestra.run.smithi088.stdout: "op_pg": 0, 2024-04-02T14:04:24.140 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_stat": 1426, 2024-04-02T14:04:24.140 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_create": 0, 2024-04-02T14:04:24.140 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_read": 790, 2024-04-02T14:04:24.140 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_write": 0, 2024-04-02T14:04:24.140 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_writefull": 0, 2024-04-02T14:04:24.141 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_writesame": 0, 2024-04-02T14:04:24.141 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_append": 0, 2024-04-02T14:04:24.141 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_zero": 0, 2024-04-02T14:04:24.141 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_truncate": 0, 2024-04-02T14:04:24.141 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_delete": 0, 2024-04-02T14:04:24.141 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_mapext": 0, 2024-04-02T14:04:24.141 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_sparse_read": 0, 2024-04-02T14:04:24.142 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_clonerange": 0, 2024-04-02T14:04:24.142 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_getxattr": 1, 2024-04-02T14:04:24.142 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_setxattr": 0, 2024-04-02T14:04:24.142 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_cmpxattr": 0, 2024-04-02T14:04:24.142 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_rmxattr": 0, 2024-04-02T14:04:24.142 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_resetxattrs": 0, 2024-04-02T14:04:24.142 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_call": 0, 2024-04-02T14:04:24.143 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_watch": 0, 2024-04-02T14:04:24.143 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_notify": 0, 2024-04-02T14:04:24.143 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T14:04:24.143 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_pgls": 0, 2024-04-02T14:04:24.143 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_pgls_filter": 0, 2024-04-02T14:04:24.143 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_other": 0, 2024-04-02T14:04:24.143 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_active": 0, 2024-04-02T14:04:24.143 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_send": 0, 2024-04-02T14:04:24.144 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_resend": 0, 2024-04-02T14:04:24.144 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_ping": 0, 2024-04-02T14:04:24.144 INFO:teuthology.orchestra.run.smithi088.stdout: "poolop_active": 0, 2024-04-02T14:04:24.144 INFO:teuthology.orchestra.run.smithi088.stdout: "poolop_send": 0, 2024-04-02T14:04:24.144 INFO:teuthology.orchestra.run.smithi088.stdout: "poolop_resend": 0, 2024-04-02T14:04:24.144 INFO:teuthology.orchestra.run.smithi088.stdout: "poolstat_active": 0, 2024-04-02T14:04:24.144 INFO:teuthology.orchestra.run.smithi088.stdout: "poolstat_send": 0, 2024-04-02T14:04:24.144 INFO:teuthology.orchestra.run.smithi088.stdout: "poolstat_resend": 0, 2024-04-02T14:04:24.145 INFO:teuthology.orchestra.run.smithi088.stdout: "statfs_active": 0, 2024-04-02T14:04:24.145 INFO:teuthology.orchestra.run.smithi088.stdout: "statfs_send": 0, 2024-04-02T14:04:24.145 INFO:teuthology.orchestra.run.smithi088.stdout: "statfs_resend": 0, 2024-04-02T14:04:24.145 INFO:teuthology.orchestra.run.smithi088.stdout: "command_active": 0, 2024-04-02T14:04:24.145 INFO:teuthology.orchestra.run.smithi088.stdout: "command_send": 0, 2024-04-02T14:04:24.145 INFO:teuthology.orchestra.run.smithi088.stdout: "command_resend": 0, 2024-04-02T14:04:24.145 INFO:teuthology.orchestra.run.smithi088.stdout: "map_epoch": 103, 2024-04-02T14:04:24.145 INFO:teuthology.orchestra.run.smithi088.stdout: "map_full": 0, 2024-04-02T14:04:24.146 INFO:teuthology.orchestra.run.smithi088.stdout: "map_inc": 13, 2024-04-02T14:04:24.146 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_sessions": 5, 2024-04-02T14:04:24.146 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_session_open": 5, 2024-04-02T14:04:24.146 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_session_close": 0, 2024-04-02T14:04:24.146 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_laggy": 0, 2024-04-02T14:04:24.146 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_wr": 0, 2024-04-02T14:04:24.146 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_rd": 2, 2024-04-02T14:04:24.146 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_del": 0 2024-04-02T14:04:24.146 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.147 INFO:teuthology.orchestra.run.smithi088.stdout: "oft": { 2024-04-02T14:04:24.147 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_objs": 0, 2024-04-02T14:04:24.147 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_kv_pairs": 0, 2024-04-02T14:04:24.147 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_updates": 0, 2024-04-02T14:04:24.147 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_removes": 0 2024-04-02T14:04:24.147 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.147 INFO:teuthology.orchestra.run.smithi088.stdout: "osd-slow-ops": { 2024-04-02T14:04:24.147 INFO:teuthology.orchestra.run.smithi088.stdout: "slow_ops_count": 0 2024-04-02T14:04:24.148 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.148 INFO:teuthology.orchestra.run.smithi088.stdout: "purge_queue": { 2024-04-02T14:04:24.148 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing_ops": 0, 2024-04-02T14:04:24.148 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T14:04:24.148 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing": 0, 2024-04-02T14:04:24.148 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing_high_water": 0, 2024-04-02T14:04:24.148 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executed_ops": 0, 2024-04-02T14:04:24.149 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executed": 0, 2024-04-02T14:04:24.149 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_item_in_journal": 0 2024-04-02T14:04:24.149 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.149 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T14:04:24.149 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-02T14:04:24.149 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 104857600, 2024-04-02T14:04:24.149 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-02T14:04:24.149 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 2477, 2024-04-02T14:04:24.150 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 2088309, 2024-04-02T14:04:24.150 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:24.150 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 2477, 2024-04-02T14:04:24.150 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 0, 2024-04-02T14:04:24.150 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 0, 2024-04-02T14:04:24.150 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 2477, 2024-04-02T14:04:24.150 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 2088309, 2024-04-02T14:04:24.151 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-02T14:04:24.151 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.151 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.151 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.151 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:24.151 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.151 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-objecter_bytes": { 2024-04-02T14:04:24.151 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-02T14:04:24.152 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 104857600, 2024-04-02T14:04:24.152 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-02T14:04:24.152 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 0, 2024-04-02T14:04:24.152 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 0, 2024-04-02T14:04:24.152 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:24.152 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:24.152 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 2218, 2024-04-02T14:04:24.152 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 54545, 2024-04-02T14:04:24.153 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 2218, 2024-04-02T14:04:24.153 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 54545, 2024-04-02T14:04:24.153 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-02T14:04:24.153 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.153 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.153 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.153 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:24.153 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.154 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-objecter_ops": { 2024-04-02T14:04:24.154 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-02T14:04:24.154 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 1024, 2024-04-02T14:04:24.154 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-02T14:04:24.154 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 0, 2024-04-02T14:04:24.154 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 0, 2024-04-02T14:04:24.154 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:24.154 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:24.155 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 2218, 2024-04-02T14:04:24.155 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 2218, 2024-04-02T14:04:24.155 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 2218, 2024-04-02T14:04:24.155 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 2218, 2024-04-02T14:04:24.155 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-02T14:04:24.155 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.155 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.156 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.156 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:24.156 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.156 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-write_buf_throttle": { 2024-04-02T14:04:24.156 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-02T14:04:24.156 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 3758096384, 2024-04-02T14:04:24.156 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-02T14:04:24.156 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 0, 2024-04-02T14:04:24.157 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 0, 2024-04-02T14:04:24.157 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:24.157 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:24.157 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 0, 2024-04-02T14:04:24.157 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 0, 2024-04-02T14:04:24.157 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 0, 2024-04-02T14:04:24.157 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 0, 2024-04-02T14:04:24.157 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-02T14:04:24.158 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.158 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.158 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.162 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:24.162 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-02T14:04:24.162 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-write_buf_throttle-0x55bb2f3326e0": { 2024-04-02T14:04:24.162 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-02T14:04:24.162 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 3758096384, 2024-04-02T14:04:24.162 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-02T14:04:24.162 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 0, 2024-04-02T14:04:24.163 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 0, 2024-04-02T14:04:24.163 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:24.163 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:24.163 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 0, 2024-04-02T14:04:24.163 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 0, 2024-04-02T14:04:24.163 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 0, 2024-04-02T14:04:24.163 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 0, 2024-04-02T14:04:24.163 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-02T14:04:24.164 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-02T14:04:24.164 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-02T14:04:24.164 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-02T14:04:24.164 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:24.164 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-02T14:04:24.164 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-02T14:04:24.995 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-02T14:04:24.996 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-02T14:04:24.996 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-04-02T14:04:24.996 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-04-02T14:04:24.996 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-04-02T14:04:24.996 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-04-02T14:04:24.996 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T14:04:24.996 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-04-02T14:04:24.996 DEBUG:tasks.check_counter:Getting stats from c 2024-04-02T14:04:24.997 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph daemon mds.c perf dump 2024-04-02T14:04:25.251 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T14:04:25.641 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:25 smithi189 ceph-mon[36683]: pgmap v340: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:04:25.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:25 smithi052 ceph-mon[32104]: pgmap v340: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:04:25.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:25 smithi088 ceph-mon[36528]: pgmap v340: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-02T14:04:27.121 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-04-02T14:04:27.121 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T14:04:27.121 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 239, 2024-04-02T14:04:27.121 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 279, 2024-04-02T14:04:27.121 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 61845, 2024-04-02T14:04:27.121 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 544427, 2024-04-02T14:04:27.122 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 47, 2024-04-02T14:04:27.122 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 5, 2024-04-02T14:04:27.122 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.037572295, 2024-04-02T14:04:27.122 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.008777594, 2024-04-02T14:04:27.122 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.023726265, 2024-04-02T14:04:27.122 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.002033932, 2024-04-02T14:04:27.122 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:27.122 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 277, 2024-04-02T14:04:27.123 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.021747128, 2024-04-02T14:04:27.123 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000078509 2024-04-02T14:04:27.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.123 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:27.123 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 188, 2024-04-02T14:04:27.123 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000111160, 2024-04-02T14:04:27.123 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000591 2024-04-02T14:04:27.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.124 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 2176, 2024-04-02T14:04:27.124 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 3808 2024-04-02T14:04:27.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.124 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T14:04:27.124 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 108, 2024-04-02T14:04:27.124 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 274, 2024-04-02T14:04:27.124 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 35425, 2024-04-02T14:04:27.124 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 576261, 2024-04-02T14:04:27.124 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 16, 2024-04-02T14:04:27.125 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 6, 2024-04-02T14:04:27.125 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.021594225, 2024-04-02T14:04:27.125 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.009329662, 2024-04-02T14:04:27.125 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.007365491, 2024-04-02T14:04:27.125 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.002513229, 2024-04-02T14:04:27.125 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:27.125 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 272, 2024-04-02T14:04:27.125 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.026323939, 2024-04-02T14:04:27.126 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000096779 2024-04-02T14:04:27.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.126 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:27.126 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 11, 2024-04-02T14:04:27.126 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000001609, 2024-04-02T14:04:27.126 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000146 2024-04-02T14:04:27.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.126 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-04-02T14:04:27.127 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 457520 2024-04-02T14:04:27.127 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.127 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T14:04:27.127 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 1668, 2024-04-02T14:04:27.127 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 1314, 2024-04-02T14:04:27.127 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 2117899, 2024-04-02T14:04:27.127 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 475277, 2024-04-02T14:04:27.127 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 7, 2024-04-02T14:04:27.128 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 6, 2024-04-02T14:04:27.128 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.162571130, 2024-04-02T14:04:27.128 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.059993292, 2024-04-02T14:04:27.128 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.115999416, 2024-04-02T14:04:27.128 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.011245522, 2024-04-02T14:04:27.128 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:27.128 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1313, 2024-04-02T14:04:27.128 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.061475655, 2024-04-02T14:04:27.128 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000046820 2024-04-02T14:04:27.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.129 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:27.129 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2269, 2024-04-02T14:04:27.129 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000492598, 2024-04-02T14:04:27.129 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000217 2024-04-02T14:04:27.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.129 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 1670832, 2024-04-02T14:04:27.129 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 164208 2024-04-02T14:04:27.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.130 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-04-02T14:04:27.130 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 1, 2024-04-02T14:04:27.130 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-04-02T14:04:27.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.130 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-MDSRank": { 2024-04-02T14:04:27.130 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-04-02T14:04:27.131 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-04-02T14:04:27.131 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 192, 2024-04-02T14:04:27.131 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.020011219, 2024-04-02T14:04:27.131 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000104225 2024-04-02T14:04:27.131 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:27.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.131 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-PurgeQueue": { 2024-04-02T14:04:27.131 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-04-02T14:04:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-04-02T14:04:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 15, 2024-04-02T14:04:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000275919, 2024-04-02T14:04:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000018394 2024-04-02T14:04:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: "mds": { 2024-04-02T14:04:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: "request": 95, 2024-04-02T14:04:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: "reply": 9, 2024-04-02T14:04:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: "reply_latency": { 2024-04-02T14:04:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 9, 2024-04-02T14:04:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.007025432, 2024-04-02T14:04:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000780603 2024-04-02T14:04:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_reply": 0, 2024-04-02T14:04:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: "forward": 86, 2024-04-02T14:04:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_complete": 0, 2024-04-02T14:04:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_keys": 0, 2024-04-02T14:04:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_commit": 11, 2024-04-02T14:04:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_split": 0, 2024-04-02T14:04:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_merge": 0, 2024-04-02T14:04:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes": 53, 2024-04-02T14:04:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_top": 12, 2024-04-02T14:04:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_bottom": 41, 2024-04-02T14:04:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pin_tail": 0, 2024-04-02T14:04:27.135 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pinned": 34, 2024-04-02T14:04:27.135 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_expired": 8, 2024-04-02T14:04:27.135 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_with_caps": 8, 2024-04-02T14:04:27.135 INFO:teuthology.orchestra.run.smithi189.stdout: "caps": 8, 2024-04-02T14:04:27.135 INFO:teuthology.orchestra.run.smithi189.stdout: "subtrees": 5, 2024-04-02T14:04:27.135 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse": 147, 2024-04-02T14:04:27.135 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_hit": 40, 2024-04-02T14:04:27.135 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_forward": 85, 2024-04-02T14:04:27.135 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_discover": 20, 2024-04-02T14:04:27.136 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_dir_fetch": 0, 2024-04-02T14:04:27.136 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_remote_ino": 0, 2024-04-02T14:04:27.136 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_lock": 0, 2024-04-02T14:04:27.136 INFO:teuthology.orchestra.run.smithi189.stdout: "load_cent": 0, 2024-04-02T14:04:27.136 INFO:teuthology.orchestra.run.smithi189.stdout: "q": 0, 2024-04-02T14:04:27.136 INFO:teuthology.orchestra.run.smithi189.stdout: "exported": 0, 2024-04-02T14:04:27.136 INFO:teuthology.orchestra.run.smithi189.stdout: "exported_inodes": 0, 2024-04-02T14:04:27.136 INFO:teuthology.orchestra.run.smithi189.stdout: "imported": 2, 2024-04-02T14:04:27.137 INFO:teuthology.orchestra.run.smithi189.stdout: "imported_inodes": 12, 2024-04-02T14:04:27.137 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_dir_fetch": 0, 2024-04-02T14:04:27.137 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_backtrace_fetch": 127, 2024-04-02T14:04:27.137 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_peer_discover": 0, 2024-04-02T14:04:27.137 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rfiles": 3452, 2024-04-02T14:04:27.137 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rbytes": 26601218, 2024-04-02T14:04:27.137 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rsnaps": 6, 2024-04-02T14:04:27.137 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_fetch": 127, 2024-04-02T14:04:27.137 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_set_tag": 0, 2024-04-02T14:04:27.138 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T14:04:27.138 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_inotable_repaired": 0, 2024-04-02T14:04:27.138 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_inodes": 32, 2024-04-02T14:04:27.138 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T14:04:27.138 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dirfrag_rstats": 32, 2024-04-02T14:04:27.138 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_file_inodes": 95, 2024-04-02T14:04:27.138 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_inode_file_caps": 0, 2024-04-02T14:04:27.138 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T14:04:27.139 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_grant": 1, 2024-04-02T14:04:27.139 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T14:04:27.139 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T14:04:27.139 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T14:04:27.139 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps": 7, 2024-04-02T14:04:27.139 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps_dirty": 7, 2024-04-02T14:04:27.139 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_cap_release": 2, 2024-04-02T14:04:27.139 INFO:teuthology.orchestra.run.smithi189.stdout: "process_request_cap_release": 1 2024-04-02T14:04:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_cache": { 2024-04-02T14:04:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update": 3, 2024-04-02T14:04:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update_receipt": 70, 2024-04-02T14:04:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_try_discover": 44, 2024-04-02T14:04:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_send_discover": 47, 2024-04-02T14:04:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_handle_discover": 3, 2024-04-02T14:04:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays": 0, 2024-04-02T14:04:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_delayed": 0, 2024-04-02T14:04:27.141 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_enqueuing": 0, 2024-04-02T14:04:27.141 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_created": 1, 2024-04-02T14:04:27.141 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_enqueued": 1, 2024-04-02T14:04:27.141 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_reintegrated": 0, 2024-04-02T14:04:27.141 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_migrated": 0, 2024-04-02T14:04:27.141 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_processing": 0, 2024-04-02T14:04:27.141 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_enqueued": 0, 2024-04-02T14:04:27.141 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_prioritized": 0, 2024-04-02T14:04:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_started": 0, 2024-04-02T14:04:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_completed": 0, 2024-04-02T14:04:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_path": 0, 2024-04-02T14:04:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_inode": 0, 2024-04-02T14:04:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T14:04:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_exportdir": 0, 2024-04-02T14:04:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_flush": 0, 2024-04-02T14:04:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragmentdir": 0, 2024-04-02T14:04:27.143 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragstats": 0, 2024-04-02T14:04:27.143 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_inodestats": 0 2024-04-02T14:04:27.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.143 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_log": { 2024-04-02T14:04:27.143 INFO:teuthology.orchestra.run.smithi189.stdout: "evlrg": 0, 2024-04-02T14:04:27.143 INFO:teuthology.orchestra.run.smithi189.stdout: "evadd": 24, 2024-04-02T14:04:27.143 INFO:teuthology.orchestra.run.smithi189.stdout: "evex": 0, 2024-04-02T14:04:27.143 INFO:teuthology.orchestra.run.smithi189.stdout: "evtrm": 0, 2024-04-02T14:04:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: "ev": 24, 2024-04-02T14:04:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: "evexg": 0, 2024-04-02T14:04:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: "evexd": 0, 2024-04-02T14:04:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: "segadd": 2, 2024-04-02T14:04:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: "segex": 0, 2024-04-02T14:04:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: "segtrm": 0, 2024-04-02T14:04:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: "seg": 2, 2024-04-02T14:04:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: "segmjr": 2, 2024-04-02T14:04:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: "segexg": 0, 2024-04-02T14:04:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: "segexd": 0, 2024-04-02T14:04:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: "expos": 4194304, 2024-04-02T14:04:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: "wrpos": 4248844, 2024-04-02T14:04:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: "rdpos": 0, 2024-04-02T14:04:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: "jlat": { 2024-04-02T14:04:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 14, 2024-04-02T14:04:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.225762624, 2024-04-02T14:04:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.016125901 2024-04-02T14:04:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: "replayed": 0 2024-04-02T14:04:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_mem": { 2024-04-02T14:04:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: "ino": 53, 2024-04-02T14:04:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: "ino+": 59, 2024-04-02T14:04:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: "ino-": 6, 2024-04-02T14:04:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: "dir": 53, 2024-04-02T14:04:27.147 INFO:teuthology.orchestra.run.smithi189.stdout: "dir+": 57, 2024-04-02T14:04:27.147 INFO:teuthology.orchestra.run.smithi189.stdout: "dir-": 4, 2024-04-02T14:04:27.147 INFO:teuthology.orchestra.run.smithi189.stdout: "dn": 53, 2024-04-02T14:04:27.147 INFO:teuthology.orchestra.run.smithi189.stdout: "dn+": 62, 2024-04-02T14:04:27.147 INFO:teuthology.orchestra.run.smithi189.stdout: "dn-": 9, 2024-04-02T14:04:27.147 INFO:teuthology.orchestra.run.smithi189.stdout: "cap": 8, 2024-04-02T14:04:27.147 INFO:teuthology.orchestra.run.smithi189.stdout: "cap+": 13, 2024-04-02T14:04:27.147 INFO:teuthology.orchestra.run.smithi189.stdout: "cap-": 5, 2024-04-02T14:04:27.148 INFO:teuthology.orchestra.run.smithi189.stdout: "rss": 52604, 2024-04-02T14:04:27.148 INFO:teuthology.orchestra.run.smithi189.stdout: "heap": 223608 2024-04-02T14:04:27.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.148 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_server": { 2024-04-02T14:04:27.148 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_client_request": 95, 2024-04-02T14:04:27.148 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_server_request": 0, 2024-04-02T14:04:27.148 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_request": 95, 2024-04-02T14:04:27.148 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_session": 12, 2024-04-02T14:04:27.148 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_peer_request": 0, 2024-04-02T14:04:27.149 INFO:teuthology.orchestra.run.smithi189.stdout: "req_create_latency": { 2024-04-02T14:04:27.149 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1, 2024-04-02T14:04:27.149 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.003427447, 2024-04-02T14:04:27.149 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.003427447 2024-04-02T14:04:27.149 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.149 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getattr_latency": { 2024-04-02T14:04:27.149 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1, 2024-04-02T14:04:27.149 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000202616, 2024-04-02T14:04:27.150 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000202616 2024-04-02T14:04:27.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.150 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getfilelock_latency": { 2024-04-02T14:04:27.150 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.150 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.150 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.150 INFO:teuthology.orchestra.run.smithi189.stdout: "req_link_latency": { 2024-04-02T14:04:27.151 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.151 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.151 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.151 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.151 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookup_latency": { 2024-04-02T14:04:27.151 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 6, 2024-04-02T14:04:27.151 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.002073923, 2024-04-02T14:04:27.151 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000345653 2024-04-02T14:04:27.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.152 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookuphash_latency": { 2024-04-02T14:04:27.152 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.152 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.152 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.152 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupino_latency": { 2024-04-02T14:04:27.152 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.153 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.153 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.153 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupname_latency": { 2024-04-02T14:04:27.153 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.153 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.153 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.154 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupparent_latency": { 2024-04-02T14:04:27.154 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.154 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.154 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.154 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupsnap_latency": { 2024-04-02T14:04:27.154 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.154 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.155 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.155 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lssnap_latency": { 2024-04-02T14:04:27.155 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.155 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.155 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.156 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mkdir_latency": { 2024-04-02T14:04:27.157 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.157 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.157 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.157 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.157 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mknod_latency": { 2024-04-02T14:04:27.157 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.157 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.157 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.158 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mksnap_latency": { 2024-04-02T14:04:27.158 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.158 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.158 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.158 INFO:teuthology.orchestra.run.smithi189.stdout: "req_open_latency": { 2024-04-02T14:04:27.159 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.159 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.159 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.159 INFO:teuthology.orchestra.run.smithi189.stdout: "req_readdir_latency": { 2024-04-02T14:04:27.159 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.159 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.159 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.160 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rename_latency": { 2024-04-02T14:04:27.160 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.160 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.160 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.160 INFO:teuthology.orchestra.run.smithi189.stdout: "req_renamesnap_latency": { 2024-04-02T14:04:27.160 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.161 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.161 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.161 INFO:teuthology.orchestra.run.smithi189.stdout: "req_snapdiff_latency": { 2024-04-02T14:04:27.161 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.161 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.161 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.162 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmdir_latency": { 2024-04-02T14:04:27.162 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.162 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.162 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.162 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.162 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmsnap_latency": { 2024-04-02T14:04:27.162 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.162 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.162 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.163 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.163 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmxattr_latency": { 2024-04-02T14:04:27.163 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.163 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.163 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.163 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.163 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setattr_latency": { 2024-04-02T14:04:27.163 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.164 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.164 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.164 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setdirlayout_latency": { 2024-04-02T14:04:27.164 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.164 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.164 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.165 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setfilelock_latency": { 2024-04-02T14:04:27.165 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.165 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.165 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.165 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setlayout_latency": { 2024-04-02T14:04:27.165 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.165 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.165 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.166 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setxattr_latency": { 2024-04-02T14:04:27.166 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.166 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.166 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.166 INFO:teuthology.orchestra.run.smithi189.stdout: "req_symlink_latency": { 2024-04-02T14:04:27.166 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.167 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.167 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.167 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.167 INFO:teuthology.orchestra.run.smithi189.stdout: "req_unlink_latency": { 2024-04-02T14:04:27.167 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1, 2024-04-02T14:04:27.167 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.001321446, 2024-04-02T14:04:27.167 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.001321446 2024-04-02T14:04:27.167 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.168 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_revoke_eviction": 0, 2024-04-02T14:04:27.168 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_acquisition_throttle": 0, 2024-04-02T14:04:27.168 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getvxattr_latency": { 2024-04-02T14:04:27.168 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.168 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.168 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.168 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:27.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.169 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_sessions": { 2024-04-02T14:04:27.169 INFO:teuthology.orchestra.run.smithi189.stdout: "session_count": 1, 2024-04-02T14:04:27.169 INFO:teuthology.orchestra.run.smithi189.stdout: "session_add": 2, 2024-04-02T14:04:27.169 INFO:teuthology.orchestra.run.smithi189.stdout: "session_remove": 1, 2024-04-02T14:04:27.169 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_open": 1, 2024-04-02T14:04:27.169 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_stale": 0, 2024-04-02T14:04:27.169 INFO:teuthology.orchestra.run.smithi189.stdout: "total_load": 6, 2024-04-02T14:04:27.169 INFO:teuthology.orchestra.run.smithi189.stdout: "average_load": 6, 2024-04-02T14:04:27.170 INFO:teuthology.orchestra.run.smithi189.stdout: "avg_session_uptime": 558, 2024-04-02T14:04:27.170 INFO:teuthology.orchestra.run.smithi189.stdout: "mdthresh_evicted": 0 2024-04-02T14:04:27.170 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.170 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-04-02T14:04:27.170 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-04-02T14:04:27.170 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-04-02T14:04:27.170 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T14:04:27.170 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-04-02T14:04:27.170 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T14:04:27.171 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-04-02T14:04:27.171 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T14:04:27.171 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T14:04:27.171 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T14:04:27.171 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T14:04:27.171 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T14:04:27.171 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-04-02T14:04:27.171 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T14:04:27.172 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T14:04:27.172 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-04-02T14:04:27.172 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-04-02T14:04:27.172 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-04-02T14:04:27.172 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-04-02T14:04:27.172 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T14:04:27.172 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T14:04:27.173 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T14:04:27.173 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T14:04:27.173 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T14:04:27.173 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-04-02T14:04:27.173 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-04-02T14:04:27.173 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-04-02T14:04:27.173 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T14:04:27.173 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T14:04:27.174 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-04-02T14:04:27.174 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-04-02T14:04:27.174 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-04-02T14:04:27.174 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-04-02T14:04:27.174 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T14:04:27.174 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-04-02T14:04:27.174 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T14:04:27.174 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-04-02T14:04:27.175 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 308338, 2024-04-02T14:04:27.175 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 159, 2024-04-02T14:04:27.175 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 88, 2024-04-02T14:04:27.175 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 1, 2024-04-02T14:04:27.175 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-04-02T14:04:27.175 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-04-02T14:04:27.175 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-04-02T14:04:27.175 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-04-02T14:04:27.175 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-04-02T14:04:27.176 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-04-02T14:04:27.176 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 9320, 2024-04-02T14:04:27.176 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 178, 2024-04-02T14:04:27.176 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T14:04:27.176 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-04-02T14:04:27.176 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-04-02T14:04:27.176 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-04-02T14:04:27.176 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 241887, 2024-04-02T14:04:27.177 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 1537, 2024-04-02T14:04:27.177 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-04-02T14:04:27.177 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-04-02T14:04:27.177 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-04-02T14:04:27.177 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-04-02T14:04:27.177 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.177 INFO:teuthology.orchestra.run.smithi189.stdout: "objecter": { 2024-04-02T14:04:27.177 INFO:teuthology.orchestra.run.smithi189.stdout: "op_active": 0, 2024-04-02T14:04:27.177 INFO:teuthology.orchestra.run.smithi189.stdout: "op_laggy": 0, 2024-04-02T14:04:27.178 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send": 202, 2024-04-02T14:04:27.178 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send_bytes": 68479, 2024-04-02T14:04:27.178 INFO:teuthology.orchestra.run.smithi189.stdout: "op_resend": 33, 2024-04-02T14:04:27.178 INFO:teuthology.orchestra.run.smithi189.stdout: "op_reply": 202, 2024-04-02T14:04:27.178 INFO:teuthology.orchestra.run.smithi189.stdout: "op_latency": { 2024-04-02T14:04:27.178 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 202, 2024-04-02T14:04:27.178 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.627988051, 2024-04-02T14:04:27.178 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.003108851 2024-04-02T14:04:27.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.179 INFO:teuthology.orchestra.run.smithi189.stdout: "op_inflight": 0, 2024-04-02T14:04:27.179 INFO:teuthology.orchestra.run.smithi189.stdout: "oplen_avg": { 2024-04-02T14:04:27.179 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 202, 2024-04-02T14:04:27.179 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 234 2024-04-02T14:04:27.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.179 INFO:teuthology.orchestra.run.smithi189.stdout: "op": 202, 2024-04-02T14:04:27.179 INFO:teuthology.orchestra.run.smithi189.stdout: "op_r": 128, 2024-04-02T14:04:27.179 INFO:teuthology.orchestra.run.smithi189.stdout: "op_w": 74, 2024-04-02T14:04:27.180 INFO:teuthology.orchestra.run.smithi189.stdout: "op_rmw": 0, 2024-04-02T14:04:27.180 INFO:teuthology.orchestra.run.smithi189.stdout: "op_pg": 0, 2024-04-02T14:04:27.180 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_stat": 0, 2024-04-02T14:04:27.180 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_create": 10, 2024-04-02T14:04:27.180 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_read": 1, 2024-04-02T14:04:27.180 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_write": 15, 2024-04-02T14:04:27.180 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writefull": 12, 2024-04-02T14:04:27.180 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writesame": 0, 2024-04-02T14:04:27.181 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_append": 0, 2024-04-02T14:04:27.181 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_zero": 0, 2024-04-02T14:04:27.181 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_truncate": 0, 2024-04-02T14:04:27.181 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_delete": 14, 2024-04-02T14:04:27.181 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_mapext": 0, 2024-04-02T14:04:27.181 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_sparse_read": 0, 2024-04-02T14:04:27.181 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_clonerange": 0, 2024-04-02T14:04:27.181 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_getxattr": 127, 2024-04-02T14:04:27.181 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_setxattr": 20, 2024-04-02T14:04:27.182 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_cmpxattr": 0, 2024-04-02T14:04:27.182 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_rmxattr": 0, 2024-04-02T14:04:27.182 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_resetxattrs": 0, 2024-04-02T14:04:27.182 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_call": 0, 2024-04-02T14:04:27.182 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_watch": 0, 2024-04-02T14:04:27.182 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_notify": 0, 2024-04-02T14:04:27.182 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T14:04:27.182 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls": 0, 2024-04-02T14:04:27.183 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls_filter": 0, 2024-04-02T14:04:27.183 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_other": 0, 2024-04-02T14:04:27.183 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_active": 0, 2024-04-02T14:04:27.183 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_send": 0, 2024-04-02T14:04:27.183 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_resend": 0, 2024-04-02T14:04:27.183 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_ping": 0, 2024-04-02T14:04:27.183 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_active": 0, 2024-04-02T14:04:27.183 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_send": 0, 2024-04-02T14:04:27.184 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_resend": 0, 2024-04-02T14:04:27.184 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_active": 0, 2024-04-02T14:04:27.184 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_send": 0, 2024-04-02T14:04:27.184 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_resend": 0, 2024-04-02T14:04:27.184 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_active": 0, 2024-04-02T14:04:27.184 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_send": 0, 2024-04-02T14:04:27.184 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_resend": 0, 2024-04-02T14:04:27.184 INFO:teuthology.orchestra.run.smithi189.stdout: "command_active": 0, 2024-04-02T14:04:27.184 INFO:teuthology.orchestra.run.smithi189.stdout: "command_send": 0, 2024-04-02T14:04:27.185 INFO:teuthology.orchestra.run.smithi189.stdout: "command_resend": 0, 2024-04-02T14:04:27.185 INFO:teuthology.orchestra.run.smithi189.stdout: "map_epoch": 103, 2024-04-02T14:04:27.185 INFO:teuthology.orchestra.run.smithi189.stdout: "map_full": 0, 2024-04-02T14:04:27.185 INFO:teuthology.orchestra.run.smithi189.stdout: "map_inc": 13, 2024-04-02T14:04:27.185 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_sessions": 10, 2024-04-02T14:04:27.185 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_open": 10, 2024-04-02T14:04:27.185 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_close": 0, 2024-04-02T14:04:27.185 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_laggy": 0, 2024-04-02T14:04:27.186 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_wr": 29, 2024-04-02T14:04:27.186 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_rd": 0, 2024-04-02T14:04:27.186 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_del": 6 2024-04-02T14:04:27.186 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.186 INFO:teuthology.orchestra.run.smithi189.stdout: "oft": { 2024-04-02T14:04:27.186 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_objs": 1, 2024-04-02T14:04:27.186 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_kv_pairs": 0, 2024-04-02T14:04:27.186 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_updates": 0, 2024-04-02T14:04:27.187 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_removes": 0 2024-04-02T14:04:27.187 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.187 INFO:teuthology.orchestra.run.smithi189.stdout: "osd-slow-ops": { 2024-04-02T14:04:27.187 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_ops_count": 0 2024-04-02T14:04:27.187 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.187 INFO:teuthology.orchestra.run.smithi189.stdout: "purge_queue": { 2024-04-02T14:04:27.187 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops": 0, 2024-04-02T14:04:27.187 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops_high_water": 2, 2024-04-02T14:04:27.188 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing": 0, 2024-04-02T14:04:27.188 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_high_water": 1, 2024-04-02T14:04:27.188 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed_ops": 2, 2024-04-02T14:04:27.188 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed": 1, 2024-04-02T14:04:27.188 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_item_in_journal": 0 2024-04-02T14:04:27.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.188 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T14:04:27.188 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-02T14:04:27.189 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-04-02T14:04:27.189 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-02T14:04:27.189 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 2011, 2024-04-02T14:04:27.189 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 2016146, 2024-04-02T14:04:27.189 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:27.189 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 2011, 2024-04-02T14:04:27.189 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-02T14:04:27.189 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-02T14:04:27.189 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 2011, 2024-04-02T14:04:27.190 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 2016146, 2024-04-02T14:04:27.190 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-02T14:04:27.190 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.190 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.190 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.190 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:27.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.190 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_bytes": { 2024-04-02T14:04:27.191 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-02T14:04:27.191 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-04-02T14:04:27.191 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-02T14:04:27.191 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-02T14:04:27.191 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-02T14:04:27.191 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:27.191 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:27.191 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 202, 2024-04-02T14:04:27.192 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 68588, 2024-04-02T14:04:27.192 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 202, 2024-04-02T14:04:27.192 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 68588, 2024-04-02T14:04:27.192 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-02T14:04:27.192 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.192 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.192 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.192 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:27.193 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.193 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_ops": { 2024-04-02T14:04:27.193 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-02T14:04:27.193 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 1024, 2024-04-02T14:04:27.193 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-02T14:04:27.193 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-02T14:04:27.193 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-02T14:04:27.193 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:27.193 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:27.194 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 202, 2024-04-02T14:04:27.194 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 202, 2024-04-02T14:04:27.194 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 202, 2024-04-02T14:04:27.194 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 202, 2024-04-02T14:04:27.194 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-02T14:04:27.194 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.194 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.194 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.195 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:27.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.195 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle": { 2024-04-02T14:04:27.195 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-02T14:04:27.195 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-04-02T14:04:27.195 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-02T14:04:27.195 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 1, 2024-04-02T14:04:27.195 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 109, 2024-04-02T14:04:27.196 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:27.196 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 1, 2024-04-02T14:04:27.196 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-02T14:04:27.196 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-02T14:04:27.196 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 1, 2024-04-02T14:04:27.196 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 109, 2024-04-02T14:04:27.196 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-02T14:04:27.196 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.196 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.197 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.197 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:27.197 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:27.197 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle-0x5587712c0500": { 2024-04-02T14:04:27.197 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-02T14:04:27.197 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-04-02T14:04:27.197 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-02T14:04:27.197 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 24, 2024-04-02T14:04:27.198 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 54540, 2024-04-02T14:04:27.198 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:27.198 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 24, 2024-04-02T14:04:27.198 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-02T14:04:27.198 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-02T14:04:27.198 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 14, 2024-04-02T14:04:27.198 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 54540, 2024-04-02T14:04:27.198 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-02T14:04:27.199 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:27.199 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:27.199 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:27.199 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:27.199 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:27.199 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-04-02T14:04:27.726 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:27 smithi189 ceph-mon[36683]: pgmap v341: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 96 B/s wr, 14 op/s 2024-04-02T14:04:27.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:27 smithi052 ceph-mon[32104]: pgmap v341: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 96 B/s wr, 14 op/s 2024-04-02T14:04:27.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:27 smithi088 ceph-mon[36528]: pgmap v341: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 96 B/s wr, 14 op/s 2024-04-02T14:04:27.977 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-02T14:04:27.977 INFO:tasks.check_counter:Daemon mds.c mds.imported=2 2024-04-02T14:04:27.977 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=3 2024-04-02T14:04:27.977 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=70 2024-04-02T14:04:27.977 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-02T14:04:27.977 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-02T14:04:27.977 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T14:04:27.978 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-04-02T14:04:27.978 DEBUG:tasks.check_counter:Getting stats from f 2024-04-02T14:04:27.978 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph daemon mds.f perf dump 2024-04-02T14:04:28.230 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T14:04:29.613 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:29 smithi189 ceph-mon[36683]: pgmap v342: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:04:29.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:29 smithi052 ceph-mon[32104]: pgmap v342: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:04:29.891 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-04-02T14:04:29.892 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T14:04:29.892 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 1178, 2024-04-02T14:04:29.892 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 935, 2024-04-02T14:04:29.892 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 382256, 2024-04-02T14:04:29.892 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 312310, 2024-04-02T14:04:29.892 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 9, 2024-04-02T14:04:29.893 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 4, 2024-04-02T14:04:29.893 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.114092297, 2024-04-02T14:04:29.893 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.043503533, 2024-04-02T14:04:29.893 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.074374659, 2024-04-02T14:04:29.893 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.005293649, 2024-04-02T14:04:29.893 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:29.893 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 933, 2024-04-02T14:04:29.893 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.045943909, 2024-04-02T14:04:29.893 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000049243 2024-04-02T14:04:29.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.894 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:29.894 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2030, 2024-04-02T14:04:29.894 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000506622, 2024-04-02T14:04:29.894 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000249 2024-04-02T14:04:29.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.894 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T14:04:29.894 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T14:04:29.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.895 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T14:04:29.895 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 32, 2024-04-02T14:04:29.895 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 199, 2024-04-02T14:04:29.895 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 15628, 2024-04-02T14:04:29.895 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 590734, 2024-04-02T14:04:29.895 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 21, 2024-04-02T14:04:29.895 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 4, 2024-04-02T14:04:29.896 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.017255233, 2024-04-02T14:04:29.896 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.007682101, 2024-04-02T14:04:29.896 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.007525205, 2024-04-02T14:04:29.896 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000222081, 2024-04-02T14:04:29.896 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:29.896 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 197, 2024-04-02T14:04:29.896 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.022487965, 2024-04-02T14:04:29.896 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000114152 2024-04-02T14:04:29.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.897 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:29.897 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 23, 2024-04-02T14:04:29.897 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000003198, 2024-04-02T14:04:29.897 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000139 2024-04-02T14:04:29.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.897 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 4320, 2024-04-02T14:04:29.897 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 456112 2024-04-02T14:04:29.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.898 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T14:04:29.898 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 340, 2024-04-02T14:04:29.898 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 325, 2024-04-02T14:04:29.898 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 1663301, 2024-04-02T14:04:29.898 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 529616, 2024-04-02T14:04:29.898 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 38, 2024-04-02T14:04:29.898 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 7, 2024-04-02T14:04:29.898 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.066901630, 2024-04-02T14:04:29.899 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.016880359, 2024-04-02T14:04:29.899 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.055955055, 2024-04-02T14:04:29.899 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.006118532, 2024-04-02T14:04:29.899 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:29.899 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 324, 2024-04-02T14:04:29.899 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.031259064, 2024-04-02T14:04:29.899 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000096478 2024-04-02T14:04:29.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.900 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:29.900 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 40, 2024-04-02T14:04:29.900 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000004387, 2024-04-02T14:04:29.900 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000109 2024-04-02T14:04:29.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.900 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 1654304, 2024-04-02T14:04:29.900 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 146560 2024-04-02T14:04:29.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.900 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-04-02T14:04:29.901 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 1, 2024-04-02T14:04:29.901 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-04-02T14:04:29.901 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.901 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-MDSRank": { 2024-04-02T14:04:29.901 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-04-02T14:04:29.901 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-04-02T14:04:29.901 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 44, 2024-04-02T14:04:29.901 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.002694757, 2024-04-02T14:04:29.902 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000061244 2024-04-02T14:04:29.902 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:29.902 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.902 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-PurgeQueue": { 2024-04-02T14:04:29.902 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-04-02T14:04:29.902 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-04-02T14:04:29.902 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2, 2024-04-02T14:04:29.902 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000008771, 2024-04-02T14:04:29.903 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000004385 2024-04-02T14:04:29.903 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:29.903 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.903 INFO:teuthology.orchestra.run.smithi189.stdout: "mds": { 2024-04-02T14:04:29.903 INFO:teuthology.orchestra.run.smithi189.stdout: "request": 1, 2024-04-02T14:04:29.903 INFO:teuthology.orchestra.run.smithi189.stdout: "reply": 1, 2024-04-02T14:04:29.903 INFO:teuthology.orchestra.run.smithi189.stdout: "reply_latency": { 2024-04-02T14:04:29.903 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1, 2024-04-02T14:04:29.904 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000123661, 2024-04-02T14:04:29.904 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000123661 2024-04-02T14:04:29.904 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.904 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_reply": 0, 2024-04-02T14:04:29.904 INFO:teuthology.orchestra.run.smithi189.stdout: "forward": 0, 2024-04-02T14:04:29.904 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_complete": 0, 2024-04-02T14:04:29.904 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_keys": 0, 2024-04-02T14:04:29.904 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_commit": 11, 2024-04-02T14:04:29.904 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_split": 0, 2024-04-02T14:04:29.905 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_merge": 0, 2024-04-02T14:04:29.905 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes": 36, 2024-04-02T14:04:29.905 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_top": 9, 2024-04-02T14:04:29.905 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_bottom": 27, 2024-04-02T14:04:29.905 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pin_tail": 0, 2024-04-02T14:04:29.905 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pinned": 21, 2024-04-02T14:04:29.905 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_expired": 8, 2024-04-02T14:04:29.905 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_with_caps": 1, 2024-04-02T14:04:29.905 INFO:teuthology.orchestra.run.smithi189.stdout: "caps": 1, 2024-04-02T14:04:29.906 INFO:teuthology.orchestra.run.smithi189.stdout: "subtrees": 3, 2024-04-02T14:04:29.906 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse": 43, 2024-04-02T14:04:29.906 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_hit": 25, 2024-04-02T14:04:29.906 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_forward": 0, 2024-04-02T14:04:29.906 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_discover": 18, 2024-04-02T14:04:29.906 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_dir_fetch": 0, 2024-04-02T14:04:29.906 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_remote_ino": 0, 2024-04-02T14:04:29.906 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_lock": 0, 2024-04-02T14:04:29.907 INFO:teuthology.orchestra.run.smithi189.stdout: "load_cent": 0, 2024-04-02T14:04:29.907 INFO:teuthology.orchestra.run.smithi189.stdout: "q": 0, 2024-04-02T14:04:29.907 INFO:teuthology.orchestra.run.smithi189.stdout: "exported": 0, 2024-04-02T14:04:29.907 INFO:teuthology.orchestra.run.smithi189.stdout: "exported_inodes": 0, 2024-04-02T14:04:29.907 INFO:teuthology.orchestra.run.smithi189.stdout: "imported": 0, 2024-04-02T14:04:29.907 INFO:teuthology.orchestra.run.smithi189.stdout: "imported_inodes": 0, 2024-04-02T14:04:29.907 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_dir_fetch": 0, 2024-04-02T14:04:29.907 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_backtrace_fetch": 0, 2024-04-02T14:04:29.908 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_peer_discover": 0, 2024-04-02T14:04:29.908 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rfiles": 3452, 2024-04-02T14:04:29.908 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rbytes": 26601218, 2024-04-02T14:04:29.908 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rsnaps": 6, 2024-04-02T14:04:29.908 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T14:04:29.908 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_set_tag": 0, 2024-04-02T14:04:29.908 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T14:04:29.908 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_inotable_repaired": 0, 2024-04-02T14:04:29.908 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_inodes": 0, 2024-04-02T14:04:29.909 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T14:04:29.909 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T14:04:29.909 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_file_inodes": 0, 2024-04-02T14:04:29.909 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_inode_file_caps": 0, 2024-04-02T14:04:29.909 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T14:04:29.909 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_grant": 0, 2024-04-02T14:04:29.909 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T14:04:29.909 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T14:04:29.909 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T14:04:29.910 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps": 0, 2024-04-02T14:04:29.910 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps_dirty": 0, 2024-04-02T14:04:29.910 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_cap_release": 0, 2024-04-02T14:04:29.910 INFO:teuthology.orchestra.run.smithi189.stdout: "process_request_cap_release": 0 2024-04-02T14:04:29.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.910 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_cache": { 2024-04-02T14:04:29.910 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update": 0, 2024-04-02T14:04:29.910 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update_receipt": 68, 2024-04-02T14:04:29.911 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_try_discover": 43, 2024-04-02T14:04:29.911 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_send_discover": 45, 2024-04-02T14:04:29.911 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_handle_discover": 0, 2024-04-02T14:04:29.911 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays": 0, 2024-04-02T14:04:29.911 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_delayed": 0, 2024-04-02T14:04:29.911 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_enqueuing": 0, 2024-04-02T14:04:29.911 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_created": 0, 2024-04-02T14:04:29.911 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_enqueued": 0, 2024-04-02T14:04:29.912 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_reintegrated": 0, 2024-04-02T14:04:29.912 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_migrated": 0, 2024-04-02T14:04:29.912 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_processing": 0, 2024-04-02T14:04:29.912 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_enqueued": 0, 2024-04-02T14:04:29.912 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_prioritized": 0, 2024-04-02T14:04:29.912 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_started": 0, 2024-04-02T14:04:29.912 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_completed": 0, 2024-04-02T14:04:29.912 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_path": 0, 2024-04-02T14:04:29.912 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_inode": 0, 2024-04-02T14:04:29.913 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T14:04:29.913 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_exportdir": 0, 2024-04-02T14:04:29.913 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_flush": 0, 2024-04-02T14:04:29.913 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragmentdir": 0, 2024-04-02T14:04:29.913 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragstats": 0, 2024-04-02T14:04:29.913 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_inodestats": 0 2024-04-02T14:04:29.913 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.913 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_log": { 2024-04-02T14:04:29.914 INFO:teuthology.orchestra.run.smithi189.stdout: "evlrg": 0, 2024-04-02T14:04:29.914 INFO:teuthology.orchestra.run.smithi189.stdout: "evadd": 7, 2024-04-02T14:04:29.914 INFO:teuthology.orchestra.run.smithi189.stdout: "evex": 0, 2024-04-02T14:04:29.914 INFO:teuthology.orchestra.run.smithi189.stdout: "evtrm": 0, 2024-04-02T14:04:29.914 INFO:teuthology.orchestra.run.smithi189.stdout: "ev": 7, 2024-04-02T14:04:29.914 INFO:teuthology.orchestra.run.smithi189.stdout: "evexg": 0, 2024-04-02T14:04:29.914 INFO:teuthology.orchestra.run.smithi189.stdout: "evexd": 0, 2024-04-02T14:04:29.914 INFO:teuthology.orchestra.run.smithi189.stdout: "segadd": 2, 2024-04-02T14:04:29.915 INFO:teuthology.orchestra.run.smithi189.stdout: "segex": 0, 2024-04-02T14:04:29.915 INFO:teuthology.orchestra.run.smithi189.stdout: "segtrm": 0, 2024-04-02T14:04:29.915 INFO:teuthology.orchestra.run.smithi189.stdout: "seg": 2, 2024-04-02T14:04:29.915 INFO:teuthology.orchestra.run.smithi189.stdout: "segmjr": 2, 2024-04-02T14:04:29.915 INFO:teuthology.orchestra.run.smithi189.stdout: "segexg": 0, 2024-04-02T14:04:29.915 INFO:teuthology.orchestra.run.smithi189.stdout: "segexd": 0, 2024-04-02T14:04:29.915 INFO:teuthology.orchestra.run.smithi189.stdout: "expos": 4194304, 2024-04-02T14:04:29.915 INFO:teuthology.orchestra.run.smithi189.stdout: "wrpos": 4196264, 2024-04-02T14:04:29.916 INFO:teuthology.orchestra.run.smithi189.stdout: "rdpos": 0, 2024-04-02T14:04:29.916 INFO:teuthology.orchestra.run.smithi189.stdout: "jlat": { 2024-04-02T14:04:29.916 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 6, 2024-04-02T14:04:29.916 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.011537434, 2024-04-02T14:04:29.916 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.001922905 2024-04-02T14:04:29.916 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.916 INFO:teuthology.orchestra.run.smithi189.stdout: "replayed": 0 2024-04-02T14:04:29.917 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.917 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_mem": { 2024-04-02T14:04:29.917 INFO:teuthology.orchestra.run.smithi189.stdout: "ino": 40, 2024-04-02T14:04:29.917 INFO:teuthology.orchestra.run.smithi189.stdout: "ino+": 45, 2024-04-02T14:04:29.917 INFO:teuthology.orchestra.run.smithi189.stdout: "ino-": 5, 2024-04-02T14:04:29.917 INFO:teuthology.orchestra.run.smithi189.stdout: "dir": 45, 2024-04-02T14:04:29.917 INFO:teuthology.orchestra.run.smithi189.stdout: "dir+": 50, 2024-04-02T14:04:29.917 INFO:teuthology.orchestra.run.smithi189.stdout: "dir-": 5, 2024-04-02T14:04:29.917 INFO:teuthology.orchestra.run.smithi189.stdout: "dn": 36, 2024-04-02T14:04:29.918 INFO:teuthology.orchestra.run.smithi189.stdout: "dn+": 44, 2024-04-02T14:04:29.918 INFO:teuthology.orchestra.run.smithi189.stdout: "dn-": 8, 2024-04-02T14:04:29.918 INFO:teuthology.orchestra.run.smithi189.stdout: "cap": 1, 2024-04-02T14:04:29.918 INFO:teuthology.orchestra.run.smithi189.stdout: "cap+": 1, 2024-04-02T14:04:29.918 INFO:teuthology.orchestra.run.smithi189.stdout: "cap-": 0, 2024-04-02T14:04:29.918 INFO:teuthology.orchestra.run.smithi189.stdout: "rss": 51488, 2024-04-02T14:04:29.918 INFO:teuthology.orchestra.run.smithi189.stdout: "heap": 223608 2024-04-02T14:04:29.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.919 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_server": { 2024-04-02T14:04:29.919 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_client_request": 1, 2024-04-02T14:04:29.919 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_server_request": 0, 2024-04-02T14:04:29.919 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_request": 1, 2024-04-02T14:04:29.919 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_session": 17, 2024-04-02T14:04:29.919 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_peer_request": 0, 2024-04-02T14:04:29.919 INFO:teuthology.orchestra.run.smithi189.stdout: "req_create_latency": { 2024-04-02T14:04:29.920 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.920 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.920 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.920 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.921 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getattr_latency": { 2024-04-02T14:04:29.921 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.921 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.921 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.921 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.921 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getfilelock_latency": { 2024-04-02T14:04:29.921 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.921 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.921 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.922 INFO:teuthology.orchestra.run.smithi189.stdout: "req_link_latency": { 2024-04-02T14:04:29.922 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.922 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.922 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.922 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookup_latency": { 2024-04-02T14:04:29.922 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.923 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.923 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.923 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookuphash_latency": { 2024-04-02T14:04:29.923 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.923 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.923 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.924 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupino_latency": { 2024-04-02T14:04:29.924 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1, 2024-04-02T14:04:29.924 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000123661, 2024-04-02T14:04:29.924 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000123661 2024-04-02T14:04:29.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.924 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupname_latency": { 2024-04-02T14:04:29.925 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.925 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.925 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.925 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.925 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupparent_latency": { 2024-04-02T14:04:29.925 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.925 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.925 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.925 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.926 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupsnap_latency": { 2024-04-02T14:04:29.926 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.926 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.926 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.926 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lssnap_latency": { 2024-04-02T14:04:29.927 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.927 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.927 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.927 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mkdir_latency": { 2024-04-02T14:04:29.928 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:29 smithi088 ceph-mon[36528]: pgmap v342: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:04:29.928 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.928 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.928 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.929 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.929 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mknod_latency": { 2024-04-02T14:04:29.929 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.929 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.929 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.929 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.929 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mksnap_latency": { 2024-04-02T14:04:29.929 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.929 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.930 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.930 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.930 INFO:teuthology.orchestra.run.smithi189.stdout: "req_open_latency": { 2024-04-02T14:04:29.930 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.930 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.930 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.930 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.931 INFO:teuthology.orchestra.run.smithi189.stdout: "req_readdir_latency": { 2024-04-02T14:04:29.931 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.931 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.931 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.931 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rename_latency": { 2024-04-02T14:04:29.931 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.931 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.931 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.932 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.932 INFO:teuthology.orchestra.run.smithi189.stdout: "req_renamesnap_latency": { 2024-04-02T14:04:29.932 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.932 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.932 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.932 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.932 INFO:teuthology.orchestra.run.smithi189.stdout: "req_snapdiff_latency": { 2024-04-02T14:04:29.932 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.933 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.933 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.933 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmdir_latency": { 2024-04-02T14:04:29.933 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.933 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.933 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.933 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmsnap_latency": { 2024-04-02T14:04:29.934 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.934 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.934 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.934 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.934 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmxattr_latency": { 2024-04-02T14:04:29.934 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.934 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.934 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.935 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.935 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setattr_latency": { 2024-04-02T14:04:29.935 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.935 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.935 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.935 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.935 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setdirlayout_latency": { 2024-04-02T14:04:29.935 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.936 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.936 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.936 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setfilelock_latency": { 2024-04-02T14:04:29.936 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.936 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.936 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.936 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setlayout_latency": { 2024-04-02T14:04:29.937 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.937 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.937 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.937 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.937 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setxattr_latency": { 2024-04-02T14:04:29.937 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.937 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.937 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.937 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.938 INFO:teuthology.orchestra.run.smithi189.stdout: "req_symlink_latency": { 2024-04-02T14:04:29.938 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.938 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.938 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.938 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.938 INFO:teuthology.orchestra.run.smithi189.stdout: "req_unlink_latency": { 2024-04-02T14:04:29.938 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.938 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.939 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.939 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_revoke_eviction": 0, 2024-04-02T14:04:29.939 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_acquisition_throttle": 0, 2024-04-02T14:04:29.939 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getvxattr_latency": { 2024-04-02T14:04:29.939 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.939 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.939 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.939 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:29.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.940 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_sessions": { 2024-04-02T14:04:29.940 INFO:teuthology.orchestra.run.smithi189.stdout: "session_count": 1, 2024-04-02T14:04:29.940 INFO:teuthology.orchestra.run.smithi189.stdout: "session_add": 3, 2024-04-02T14:04:29.940 INFO:teuthology.orchestra.run.smithi189.stdout: "session_remove": 2, 2024-04-02T14:04:29.940 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_open": 1, 2024-04-02T14:04:29.940 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_stale": 0, 2024-04-02T14:04:29.940 INFO:teuthology.orchestra.run.smithi189.stdout: "total_load": 1, 2024-04-02T14:04:29.941 INFO:teuthology.orchestra.run.smithi189.stdout: "average_load": 0, 2024-04-02T14:04:29.941 INFO:teuthology.orchestra.run.smithi189.stdout: "avg_session_uptime": 648, 2024-04-02T14:04:29.941 INFO:teuthology.orchestra.run.smithi189.stdout: "mdthresh_evicted": 0 2024-04-02T14:04:29.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.941 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-04-02T14:04:29.941 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-04-02T14:04:29.941 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-04-02T14:04:29.941 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T14:04:29.941 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-04-02T14:04:29.942 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T14:04:29.942 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-04-02T14:04:29.942 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T14:04:29.942 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T14:04:29.942 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T14:04:29.942 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T14:04:29.942 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T14:04:29.942 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-04-02T14:04:29.943 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T14:04:29.943 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T14:04:29.943 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-04-02T14:04:29.943 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-04-02T14:04:29.943 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-04-02T14:04:29.943 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-04-02T14:04:29.943 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T14:04:29.943 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T14:04:29.944 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T14:04:29.944 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T14:04:29.944 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T14:04:29.944 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-04-02T14:04:29.944 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-04-02T14:04:29.944 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-04-02T14:04:29.944 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T14:04:29.944 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T14:04:29.944 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-04-02T14:04:29.945 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-04-02T14:04:29.945 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-04-02T14:04:29.945 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-04-02T14:04:29.945 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T14:04:29.945 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-04-02T14:04:29.945 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T14:04:29.945 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-04-02T14:04:29.945 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 271630, 2024-04-02T14:04:29.945 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 113, 2024-04-02T14:04:29.946 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 0, 2024-04-02T14:04:29.946 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 0, 2024-04-02T14:04:29.946 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-04-02T14:04:29.946 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-04-02T14:04:29.946 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-04-02T14:04:29.946 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-04-02T14:04:29.946 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-04-02T14:04:29.946 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-04-02T14:04:29.947 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 9320, 2024-04-02T14:04:29.947 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 178, 2024-04-02T14:04:29.947 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T14:04:29.947 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-04-02T14:04:29.947 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-04-02T14:04:29.947 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-04-02T14:04:29.947 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 183330, 2024-04-02T14:04:29.947 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 977, 2024-04-02T14:04:29.947 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-04-02T14:04:29.948 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-04-02T14:04:29.948 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-04-02T14:04:29.948 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-04-02T14:04:29.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.948 INFO:teuthology.orchestra.run.smithi189.stdout: "objecter": { 2024-04-02T14:04:29.948 INFO:teuthology.orchestra.run.smithi189.stdout: "op_active": 0, 2024-04-02T14:04:29.948 INFO:teuthology.orchestra.run.smithi189.stdout: "op_laggy": 0, 2024-04-02T14:04:29.948 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send": 43, 2024-04-02T14:04:29.948 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send_bytes": 12473, 2024-04-02T14:04:29.949 INFO:teuthology.orchestra.run.smithi189.stdout: "op_resend": 33, 2024-04-02T14:04:29.949 INFO:teuthology.orchestra.run.smithi189.stdout: "op_reply": 43, 2024-04-02T14:04:29.949 INFO:teuthology.orchestra.run.smithi189.stdout: "op_latency": { 2024-04-02T14:04:29.949 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 43, 2024-04-02T14:04:29.949 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.213995936, 2024-04-02T14:04:29.949 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.004976649 2024-04-02T14:04:29.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.949 INFO:teuthology.orchestra.run.smithi189.stdout: "op_inflight": 0, 2024-04-02T14:04:29.950 INFO:teuthology.orchestra.run.smithi189.stdout: "oplen_avg": { 2024-04-02T14:04:29.950 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 43, 2024-04-02T14:04:29.950 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 66 2024-04-02T14:04:29.950 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.950 INFO:teuthology.orchestra.run.smithi189.stdout: "op": 43, 2024-04-02T14:04:29.950 INFO:teuthology.orchestra.run.smithi189.stdout: "op_r": 0, 2024-04-02T14:04:29.950 INFO:teuthology.orchestra.run.smithi189.stdout: "op_w": 43, 2024-04-02T14:04:29.950 INFO:teuthology.orchestra.run.smithi189.stdout: "op_rmw": 0, 2024-04-02T14:04:29.950 INFO:teuthology.orchestra.run.smithi189.stdout: "op_pg": 0, 2024-04-02T14:04:29.951 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_stat": 0, 2024-04-02T14:04:29.951 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_create": 10, 2024-04-02T14:04:29.951 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_read": 0, 2024-04-02T14:04:29.951 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_write": 6, 2024-04-02T14:04:29.951 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writefull": 7, 2024-04-02T14:04:29.951 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writesame": 0, 2024-04-02T14:04:29.951 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_append": 0, 2024-04-02T14:04:29.951 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_zero": 0, 2024-04-02T14:04:29.952 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_truncate": 0, 2024-04-02T14:04:29.952 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_delete": 6, 2024-04-02T14:04:29.952 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_mapext": 0, 2024-04-02T14:04:29.952 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_sparse_read": 0, 2024-04-02T14:04:29.952 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_clonerange": 0, 2024-04-02T14:04:29.952 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_getxattr": 0, 2024-04-02T14:04:29.952 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_setxattr": 20, 2024-04-02T14:04:29.952 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_cmpxattr": 0, 2024-04-02T14:04:29.952 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_rmxattr": 0, 2024-04-02T14:04:29.953 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_resetxattrs": 0, 2024-04-02T14:04:29.953 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_call": 0, 2024-04-02T14:04:29.953 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_watch": 0, 2024-04-02T14:04:29.953 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_notify": 0, 2024-04-02T14:04:29.953 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T14:04:29.953 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls": 0, 2024-04-02T14:04:29.953 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls_filter": 0, 2024-04-02T14:04:29.953 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_other": 0, 2024-04-02T14:04:29.953 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_active": 0, 2024-04-02T14:04:29.954 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_send": 0, 2024-04-02T14:04:29.954 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_resend": 0, 2024-04-02T14:04:29.954 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_ping": 0, 2024-04-02T14:04:29.954 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_active": 0, 2024-04-02T14:04:29.954 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_send": 0, 2024-04-02T14:04:29.954 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_resend": 0, 2024-04-02T14:04:29.954 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_active": 0, 2024-04-02T14:04:29.954 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_send": 0, 2024-04-02T14:04:29.955 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_resend": 0, 2024-04-02T14:04:29.955 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_active": 0, 2024-04-02T14:04:29.955 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_send": 0, 2024-04-02T14:04:29.955 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_resend": 0, 2024-04-02T14:04:29.955 INFO:teuthology.orchestra.run.smithi189.stdout: "command_active": 0, 2024-04-02T14:04:29.955 INFO:teuthology.orchestra.run.smithi189.stdout: "command_send": 0, 2024-04-02T14:04:29.955 INFO:teuthology.orchestra.run.smithi189.stdout: "command_resend": 0, 2024-04-02T14:04:29.955 INFO:teuthology.orchestra.run.smithi189.stdout: "map_epoch": 103, 2024-04-02T14:04:29.955 INFO:teuthology.orchestra.run.smithi189.stdout: "map_full": 0, 2024-04-02T14:04:29.956 INFO:teuthology.orchestra.run.smithi189.stdout: "map_inc": 13, 2024-04-02T14:04:29.956 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_sessions": 9, 2024-04-02T14:04:29.956 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_open": 9, 2024-04-02T14:04:29.956 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_close": 0, 2024-04-02T14:04:29.956 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_laggy": 0, 2024-04-02T14:04:29.956 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_wr": 16, 2024-04-02T14:04:29.956 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_rd": 0, 2024-04-02T14:04:29.956 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_del": 1 2024-04-02T14:04:29.957 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.957 INFO:teuthology.orchestra.run.smithi189.stdout: "oft": { 2024-04-02T14:04:29.957 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_objs": 1, 2024-04-02T14:04:29.957 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_kv_pairs": 0, 2024-04-02T14:04:29.957 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_updates": 0, 2024-04-02T14:04:29.957 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_removes": 0 2024-04-02T14:04:29.957 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.957 INFO:teuthology.orchestra.run.smithi189.stdout: "osd-slow-ops": { 2024-04-02T14:04:29.957 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_ops_count": 0 2024-04-02T14:04:29.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.958 INFO:teuthology.orchestra.run.smithi189.stdout: "purge_queue": { 2024-04-02T14:04:29.958 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops": 0, 2024-04-02T14:04:29.958 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T14:04:29.958 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing": 0, 2024-04-02T14:04:29.958 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_high_water": 0, 2024-04-02T14:04:29.958 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed_ops": 0, 2024-04-02T14:04:29.958 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed": 0, 2024-04-02T14:04:29.959 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_item_in_journal": 0 2024-04-02T14:04:29.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.959 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T14:04:29.959 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-02T14:04:29.959 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-04-02T14:04:29.959 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-02T14:04:29.959 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 1546, 2024-04-02T14:04:29.959 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 1904183, 2024-04-02T14:04:29.960 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:29.960 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 1546, 2024-04-02T14:04:29.960 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-02T14:04:29.960 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-02T14:04:29.960 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 1546, 2024-04-02T14:04:29.960 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 1904183, 2024-04-02T14:04:29.960 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-02T14:04:29.960 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.960 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.961 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.961 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:29.961 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.961 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_bytes": { 2024-04-02T14:04:29.961 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-02T14:04:29.961 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-04-02T14:04:29.961 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-02T14:04:29.961 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-02T14:04:29.962 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-02T14:04:29.962 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:29.962 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:29.962 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 43, 2024-04-02T14:04:29.962 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 12473, 2024-04-02T14:04:29.962 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 43, 2024-04-02T14:04:29.962 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 12473, 2024-04-02T14:04:29.962 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-02T14:04:29.962 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.963 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.963 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.963 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:29.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.963 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_ops": { 2024-04-02T14:04:29.963 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-02T14:04:29.963 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 1024, 2024-04-02T14:04:29.963 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-02T14:04:29.964 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-02T14:04:29.964 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-02T14:04:29.964 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:29.964 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:29.964 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 43, 2024-04-02T14:04:29.964 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 43, 2024-04-02T14:04:29.964 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 43, 2024-04-02T14:04:29.964 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 43, 2024-04-02T14:04:29.964 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-02T14:04:29.965 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.965 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.965 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.965 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:29.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.965 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle": { 2024-04-02T14:04:29.965 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-02T14:04:29.965 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-04-02T14:04:29.966 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-02T14:04:29.966 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-02T14:04:29.966 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-02T14:04:29.966 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:29.966 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:29.966 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-02T14:04:29.966 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-02T14:04:29.966 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-04-02T14:04:29.966 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-04-02T14:04:29.967 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-02T14:04:29.967 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.967 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.967 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.967 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:29.967 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:29.967 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle-0x5603bc492500": { 2024-04-02T14:04:29.967 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-02T14:04:29.968 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-04-02T14:04:29.968 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-02T14:04:29.968 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 7, 2024-04-02T14:04:29.968 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 1960, 2024-04-02T14:04:29.968 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:29.968 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 7, 2024-04-02T14:04:29.968 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-02T14:04:29.968 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-02T14:04:29.968 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 6, 2024-04-02T14:04:29.969 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 1960, 2024-04-02T14:04:29.969 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-02T14:04:29.969 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:29.969 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:29.969 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:29.969 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:29.969 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:29.969 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-04-02T14:04:30.729 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-02T14:04:30.729 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-02T14:04:30.729 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-02T14:04:30.729 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=68 2024-04-02T14:04:30.729 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-04-02T14:04:30.729 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-02T14:04:30.729 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T14:04:30.729 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-04-02T14:04:30.730 DEBUG:tasks.check_counter:Getting stats from i 2024-04-02T14:04:30.730 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph daemon mds.i perf dump 2024-04-02T14:04:30.970 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T14:04:31.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:31 smithi189 ceph-mon[36683]: pgmap v343: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:04:31.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:31 smithi052 ceph-mon[32104]: pgmap v343: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:04:31.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:31 smithi088 ceph-mon[36528]: pgmap v343: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:04:32.741 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-04-02T14:04:32.742 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T14:04:32.742 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 54, 2024-04-02T14:04:32.742 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 194, 2024-04-02T14:04:32.742 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 21438, 2024-04-02T14:04:32.742 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 468908, 2024-04-02T14:04:32.743 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 22, 2024-04-02T14:04:32.743 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 5, 2024-04-02T14:04:32.743 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.019848373, 2024-04-02T14:04:32.743 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.006446453, 2024-04-02T14:04:32.743 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.007180748, 2024-04-02T14:04:32.743 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000693691, 2024-04-02T14:04:32.743 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:32.743 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 188, 2024-04-02T14:04:32.744 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.023974376, 2024-04-02T14:04:32.744 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000127523 2024-04-02T14:04:32.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.744 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:32.744 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 31, 2024-04-02T14:04:32.744 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000004290, 2024-04-02T14:04:32.744 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000138 2024-04-02T14:04:32.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.745 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 5408, 2024-04-02T14:04:32.745 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 329392 2024-04-02T14:04:32.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.745 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T14:04:32.745 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 1306, 2024-04-02T14:04:32.745 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 1083, 2024-04-02T14:04:32.745 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 412959, 2024-04-02T14:04:32.745 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 930996, 2024-04-02T14:04:32.746 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 42, 2024-04-02T14:04:32.746 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 8, 2024-04-02T14:04:32.746 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.129900798, 2024-04-02T14:04:32.746 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.048278567, 2024-04-02T14:04:32.746 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.089421618, 2024-04-02T14:04:32.746 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.005327675, 2024-04-02T14:04:32.746 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:32.746 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1085, 2024-04-02T14:04:32.747 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.054230479, 2024-04-02T14:04:32.747 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000049982 2024-04-02T14:04:32.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.747 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:32.747 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2192, 2024-04-02T14:04:32.747 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000577315, 2024-04-02T14:04:32.747 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000263 2024-04-02T14:04:32.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.748 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 2016, 2024-04-02T14:04:32.748 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 133952 2024-04-02T14:04:32.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.748 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T14:04:32.748 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 313, 2024-04-02T14:04:32.748 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 323, 2024-04-02T14:04:32.748 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 1674091, 2024-04-02T14:04:32.748 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 165940, 2024-04-02T14:04:32.748 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 7, 2024-04-02T14:04:32.749 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 6, 2024-04-02T14:04:32.749 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.053690464, 2024-04-02T14:04:32.749 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.016811220, 2024-04-02T14:04:32.749 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.040881050, 2024-04-02T14:04:32.749 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.005865652, 2024-04-02T14:04:32.749 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:32.749 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 322, 2024-04-02T14:04:32.749 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.024758568, 2024-04-02T14:04:32.749 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000076889 2024-04-02T14:04:32.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.750 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:32.750 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 59, 2024-04-02T14:04:32.750 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000012614, 2024-04-02T14:04:32.750 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000213 2024-04-02T14:04:32.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.750 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 1671280, 2024-04-02T14:04:32.750 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 156016 2024-04-02T14:04:32.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.751 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-04-02T14:04:32.751 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 1, 2024-04-02T14:04:32.751 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-04-02T14:04:32.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.751 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-MDSRank": { 2024-04-02T14:04:32.751 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-04-02T14:04:32.752 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-04-02T14:04:32.752 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 53, 2024-04-02T14:04:32.752 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.002691626, 2024-04-02T14:04:32.752 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000050785 2024-04-02T14:04:32.752 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:32.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.752 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-PurgeQueue": { 2024-04-02T14:04:32.752 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-04-02T14:04:32.752 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-04-02T14:04:32.753 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2, 2024-04-02T14:04:32.753 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000008985, 2024-04-02T14:04:32.753 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000004492 2024-04-02T14:04:32.753 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:32.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.753 INFO:teuthology.orchestra.run.smithi189.stdout: "mds": { 2024-04-02T14:04:32.753 INFO:teuthology.orchestra.run.smithi189.stdout: "request": 85, 2024-04-02T14:04:32.753 INFO:teuthology.orchestra.run.smithi189.stdout: "reply": 6, 2024-04-02T14:04:32.754 INFO:teuthology.orchestra.run.smithi189.stdout: "reply_latency": { 2024-04-02T14:04:32.754 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 6, 2024-04-02T14:04:32.754 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.002971868, 2024-04-02T14:04:32.754 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000495311 2024-04-02T14:04:32.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.754 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_reply": 0, 2024-04-02T14:04:32.754 INFO:teuthology.orchestra.run.smithi189.stdout: "forward": 79, 2024-04-02T14:04:32.754 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_complete": 0, 2024-04-02T14:04:32.755 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_keys": 0, 2024-04-02T14:04:32.755 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_commit": 11, 2024-04-02T14:04:32.755 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_split": 0, 2024-04-02T14:04:32.755 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_merge": 0, 2024-04-02T14:04:32.755 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes": 37, 2024-04-02T14:04:32.755 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_top": 10, 2024-04-02T14:04:32.755 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_bottom": 27, 2024-04-02T14:04:32.755 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pin_tail": 0, 2024-04-02T14:04:32.756 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pinned": 21, 2024-04-02T14:04:32.756 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_expired": 9, 2024-04-02T14:04:32.756 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_with_caps": 1, 2024-04-02T14:04:32.756 INFO:teuthology.orchestra.run.smithi189.stdout: "caps": 1, 2024-04-02T14:04:32.756 INFO:teuthology.orchestra.run.smithi189.stdout: "subtrees": 3, 2024-04-02T14:04:32.756 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse": 137, 2024-04-02T14:04:32.756 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_hit": 37, 2024-04-02T14:04:32.756 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_forward": 77, 2024-04-02T14:04:32.757 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_discover": 20, 2024-04-02T14:04:32.757 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_dir_fetch": 0, 2024-04-02T14:04:32.757 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_remote_ino": 0, 2024-04-02T14:04:32.757 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_lock": 0, 2024-04-02T14:04:32.757 INFO:teuthology.orchestra.run.smithi189.stdout: "load_cent": 0, 2024-04-02T14:04:32.757 INFO:teuthology.orchestra.run.smithi189.stdout: "q": 0, 2024-04-02T14:04:32.757 INFO:teuthology.orchestra.run.smithi189.stdout: "exported": 0, 2024-04-02T14:04:32.757 INFO:teuthology.orchestra.run.smithi189.stdout: "exported_inodes": 0, 2024-04-02T14:04:32.757 INFO:teuthology.orchestra.run.smithi189.stdout: "imported": 0, 2024-04-02T14:04:32.758 INFO:teuthology.orchestra.run.smithi189.stdout: "imported_inodes": 0, 2024-04-02T14:04:32.758 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_dir_fetch": 0, 2024-04-02T14:04:32.758 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_backtrace_fetch": 0, 2024-04-02T14:04:32.758 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_peer_discover": 0, 2024-04-02T14:04:32.758 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rfiles": 3452, 2024-04-02T14:04:32.758 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rbytes": 26601218, 2024-04-02T14:04:32.758 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rsnaps": 6, 2024-04-02T14:04:32.758 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T14:04:32.759 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_set_tag": 0, 2024-04-02T14:04:32.759 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T14:04:32.759 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_inotable_repaired": 0, 2024-04-02T14:04:32.759 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_inodes": 0, 2024-04-02T14:04:32.759 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T14:04:32.759 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T14:04:32.759 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_file_inodes": 0, 2024-04-02T14:04:32.759 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_inode_file_caps": 0, 2024-04-02T14:04:32.760 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T14:04:32.760 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_grant": 0, 2024-04-02T14:04:32.760 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T14:04:32.760 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T14:04:32.760 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T14:04:32.760 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps": 0, 2024-04-02T14:04:32.760 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps_dirty": 0, 2024-04-02T14:04:32.760 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_cap_release": 0, 2024-04-02T14:04:32.761 INFO:teuthology.orchestra.run.smithi189.stdout: "process_request_cap_release": 0 2024-04-02T14:04:32.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.761 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_cache": { 2024-04-02T14:04:32.761 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update": 0, 2024-04-02T14:04:32.761 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update_receipt": 73, 2024-04-02T14:04:32.761 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_try_discover": 46, 2024-04-02T14:04:32.761 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_send_discover": 49, 2024-04-02T14:04:32.761 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_handle_discover": 0, 2024-04-02T14:04:32.762 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays": 0, 2024-04-02T14:04:32.762 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_delayed": 0, 2024-04-02T14:04:32.762 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_enqueuing": 0, 2024-04-02T14:04:32.762 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_created": 0, 2024-04-02T14:04:32.762 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_enqueued": 0, 2024-04-02T14:04:32.762 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_reintegrated": 0, 2024-04-02T14:04:32.762 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_migrated": 0, 2024-04-02T14:04:32.762 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_processing": 0, 2024-04-02T14:04:32.763 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_enqueued": 0, 2024-04-02T14:04:32.763 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_prioritized": 0, 2024-04-02T14:04:32.763 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_started": 0, 2024-04-02T14:04:32.763 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_completed": 0, 2024-04-02T14:04:32.763 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_path": 0, 2024-04-02T14:04:32.763 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_inode": 0, 2024-04-02T14:04:32.763 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T14:04:32.763 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_exportdir": 0, 2024-04-02T14:04:32.764 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_flush": 0, 2024-04-02T14:04:32.764 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragmentdir": 0, 2024-04-02T14:04:32.764 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragstats": 0, 2024-04-02T14:04:32.764 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_inodestats": 0 2024-04-02T14:04:32.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.764 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_log": { 2024-04-02T14:04:32.764 INFO:teuthology.orchestra.run.smithi189.stdout: "evlrg": 0, 2024-04-02T14:04:32.764 INFO:teuthology.orchestra.run.smithi189.stdout: "evadd": 10, 2024-04-02T14:04:32.764 INFO:teuthology.orchestra.run.smithi189.stdout: "evex": 0, 2024-04-02T14:04:32.765 INFO:teuthology.orchestra.run.smithi189.stdout: "evtrm": 0, 2024-04-02T14:04:32.765 INFO:teuthology.orchestra.run.smithi189.stdout: "ev": 10, 2024-04-02T14:04:32.765 INFO:teuthology.orchestra.run.smithi189.stdout: "evexg": 0, 2024-04-02T14:04:32.765 INFO:teuthology.orchestra.run.smithi189.stdout: "evexd": 0, 2024-04-02T14:04:32.765 INFO:teuthology.orchestra.run.smithi189.stdout: "segadd": 2, 2024-04-02T14:04:32.765 INFO:teuthology.orchestra.run.smithi189.stdout: "segex": 0, 2024-04-02T14:04:32.765 INFO:teuthology.orchestra.run.smithi189.stdout: "segtrm": 0, 2024-04-02T14:04:32.765 INFO:teuthology.orchestra.run.smithi189.stdout: "seg": 2, 2024-04-02T14:04:32.766 INFO:teuthology.orchestra.run.smithi189.stdout: "segmjr": 2, 2024-04-02T14:04:32.766 INFO:teuthology.orchestra.run.smithi189.stdout: "segexg": 0, 2024-04-02T14:04:32.766 INFO:teuthology.orchestra.run.smithi189.stdout: "segexd": 0, 2024-04-02T14:04:32.766 INFO:teuthology.orchestra.run.smithi189.stdout: "expos": 4194304, 2024-04-02T14:04:32.766 INFO:teuthology.orchestra.run.smithi189.stdout: "wrpos": 4197187, 2024-04-02T14:04:32.766 INFO:teuthology.orchestra.run.smithi189.stdout: "rdpos": 0, 2024-04-02T14:04:32.766 INFO:teuthology.orchestra.run.smithi189.stdout: "jlat": { 2024-04-02T14:04:32.766 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 9, 2024-04-02T14:04:32.766 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.015619952, 2024-04-02T14:04:32.767 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.001735550 2024-04-02T14:04:32.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.767 INFO:teuthology.orchestra.run.smithi189.stdout: "replayed": 0 2024-04-02T14:04:32.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.767 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_mem": { 2024-04-02T14:04:32.767 INFO:teuthology.orchestra.run.smithi189.stdout: "ino": 41, 2024-04-02T14:04:32.767 INFO:teuthology.orchestra.run.smithi189.stdout: "ino+": 46, 2024-04-02T14:04:32.767 INFO:teuthology.orchestra.run.smithi189.stdout: "ino-": 5, 2024-04-02T14:04:32.768 INFO:teuthology.orchestra.run.smithi189.stdout: "dir": 46, 2024-04-02T14:04:32.768 INFO:teuthology.orchestra.run.smithi189.stdout: "dir+": 51, 2024-04-02T14:04:32.768 INFO:teuthology.orchestra.run.smithi189.stdout: "dir-": 5, 2024-04-02T14:04:32.768 INFO:teuthology.orchestra.run.smithi189.stdout: "dn": 37, 2024-04-02T14:04:32.768 INFO:teuthology.orchestra.run.smithi189.stdout: "dn+": 46, 2024-04-02T14:04:32.768 INFO:teuthology.orchestra.run.smithi189.stdout: "dn-": 9, 2024-04-02T14:04:32.768 INFO:teuthology.orchestra.run.smithi189.stdout: "cap": 1, 2024-04-02T14:04:32.768 INFO:teuthology.orchestra.run.smithi189.stdout: "cap+": 3, 2024-04-02T14:04:32.769 INFO:teuthology.orchestra.run.smithi189.stdout: "cap-": 2, 2024-04-02T14:04:32.769 INFO:teuthology.orchestra.run.smithi189.stdout: "rss": 51672, 2024-04-02T14:04:32.769 INFO:teuthology.orchestra.run.smithi189.stdout: "heap": 223608 2024-04-02T14:04:32.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.769 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_server": { 2024-04-02T14:04:32.769 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_client_request": 89, 2024-04-02T14:04:32.769 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_server_request": 0, 2024-04-02T14:04:32.769 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_request": 85, 2024-04-02T14:04:32.769 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_session": 29, 2024-04-02T14:04:32.770 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_peer_request": 1, 2024-04-02T14:04:32.770 INFO:teuthology.orchestra.run.smithi189.stdout: "req_create_latency": { 2024-04-02T14:04:32.770 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.770 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.770 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.770 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getattr_latency": { 2024-04-02T14:04:32.770 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 4, 2024-04-02T14:04:32.771 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.002396134, 2024-04-02T14:04:32.771 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000599033 2024-04-02T14:04:32.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.771 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getfilelock_latency": { 2024-04-02T14:04:32.771 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.771 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.771 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.771 INFO:teuthology.orchestra.run.smithi189.stdout: "req_link_latency": { 2024-04-02T14:04:32.772 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.772 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.772 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.772 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookup_latency": { 2024-04-02T14:04:32.772 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2, 2024-04-02T14:04:32.772 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000575734, 2024-04-02T14:04:32.772 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000287867 2024-04-02T14:04:32.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.773 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookuphash_latency": { 2024-04-02T14:04:32.773 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.773 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.773 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.773 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupino_latency": { 2024-04-02T14:04:32.773 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.773 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.774 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.774 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupname_latency": { 2024-04-02T14:04:32.774 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.774 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.774 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.774 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupparent_latency": { 2024-04-02T14:04:32.775 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.775 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.775 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.775 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupsnap_latency": { 2024-04-02T14:04:32.775 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.775 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.775 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.776 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lssnap_latency": { 2024-04-02T14:04:32.776 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.776 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.776 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.776 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mkdir_latency": { 2024-04-02T14:04:32.777 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.777 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.777 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.778 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mknod_latency": { 2024-04-02T14:04:32.778 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.778 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.778 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.778 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mksnap_latency": { 2024-04-02T14:04:32.778 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.779 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.779 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.779 INFO:teuthology.orchestra.run.smithi189.stdout: "req_open_latency": { 2024-04-02T14:04:32.779 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.779 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.779 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.779 INFO:teuthology.orchestra.run.smithi189.stdout: "req_readdir_latency": { 2024-04-02T14:04:32.780 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.780 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.780 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.780 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rename_latency": { 2024-04-02T14:04:32.780 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.780 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.780 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.781 INFO:teuthology.orchestra.run.smithi189.stdout: "req_renamesnap_latency": { 2024-04-02T14:04:32.781 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.781 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.781 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.781 INFO:teuthology.orchestra.run.smithi189.stdout: "req_snapdiff_latency": { 2024-04-02T14:04:32.781 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.781 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.782 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.782 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmdir_latency": { 2024-04-02T14:04:32.782 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.782 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.782 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.782 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmsnap_latency": { 2024-04-02T14:04:32.783 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.783 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.783 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.783 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmxattr_latency": { 2024-04-02T14:04:32.783 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.783 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.783 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.784 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setattr_latency": { 2024-04-02T14:04:32.784 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.784 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.784 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.784 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setdirlayout_latency": { 2024-04-02T14:04:32.784 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.784 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.785 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.785 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setfilelock_latency": { 2024-04-02T14:04:32.785 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.785 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.785 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.785 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setlayout_latency": { 2024-04-02T14:04:32.785 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.786 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.786 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.786 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setxattr_latency": { 2024-04-02T14:04:32.786 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.786 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.786 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.787 INFO:teuthology.orchestra.run.smithi189.stdout: "req_symlink_latency": { 2024-04-02T14:04:32.787 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.787 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.787 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.787 INFO:teuthology.orchestra.run.smithi189.stdout: "req_unlink_latency": { 2024-04-02T14:04:32.787 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.787 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.788 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.788 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_revoke_eviction": 0, 2024-04-02T14:04:32.788 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_acquisition_throttle": 0, 2024-04-02T14:04:32.788 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getvxattr_latency": { 2024-04-02T14:04:32.788 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.788 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.788 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.789 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:32.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.789 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_sessions": { 2024-04-02T14:04:32.789 INFO:teuthology.orchestra.run.smithi189.stdout: "session_count": 2, 2024-04-02T14:04:32.789 INFO:teuthology.orchestra.run.smithi189.stdout: "session_add": 5, 2024-04-02T14:04:32.789 INFO:teuthology.orchestra.run.smithi189.stdout: "session_remove": 3, 2024-04-02T14:04:32.789 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_open": 2, 2024-04-02T14:04:32.789 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_stale": 0, 2024-04-02T14:04:32.789 INFO:teuthology.orchestra.run.smithi189.stdout: "total_load": 5, 2024-04-02T14:04:32.790 INFO:teuthology.orchestra.run.smithi189.stdout: "average_load": 1, 2024-04-02T14:04:32.790 INFO:teuthology.orchestra.run.smithi189.stdout: "avg_session_uptime": 597, 2024-04-02T14:04:32.790 INFO:teuthology.orchestra.run.smithi189.stdout: "mdthresh_evicted": 0 2024-04-02T14:04:32.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.790 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-04-02T14:04:32.790 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-04-02T14:04:32.790 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-04-02T14:04:32.790 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T14:04:32.791 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-04-02T14:04:32.791 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T14:04:32.791 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-04-02T14:04:32.791 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T14:04:32.791 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T14:04:32.791 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T14:04:32.791 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T14:04:32.791 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T14:04:32.791 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-04-02T14:04:32.792 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T14:04:32.792 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T14:04:32.792 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-04-02T14:04:32.792 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-04-02T14:04:32.792 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-04-02T14:04:32.792 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-04-02T14:04:32.792 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T14:04:32.792 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T14:04:32.792 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T14:04:32.793 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T14:04:32.793 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T14:04:32.793 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-04-02T14:04:32.793 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-04-02T14:04:32.793 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-04-02T14:04:32.793 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T14:04:32.793 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T14:04:32.793 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-04-02T14:04:32.793 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-04-02T14:04:32.794 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-04-02T14:04:32.794 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-04-02T14:04:32.794 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T14:04:32.794 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-04-02T14:04:32.794 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T14:04:32.794 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-04-02T14:04:32.794 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 259544, 2024-04-02T14:04:32.794 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 135, 2024-04-02T14:04:32.794 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 0, 2024-04-02T14:04:32.795 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 0, 2024-04-02T14:04:32.795 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-04-02T14:04:32.795 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-04-02T14:04:32.795 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-04-02T14:04:32.795 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-04-02T14:04:32.795 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-04-02T14:04:32.795 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-04-02T14:04:32.795 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 9320, 2024-04-02T14:04:32.796 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 178, 2024-04-02T14:04:32.796 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T14:04:32.796 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-04-02T14:04:32.796 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-04-02T14:04:32.796 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-04-02T14:04:32.796 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 187634, 2024-04-02T14:04:32.796 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 1006, 2024-04-02T14:04:32.796 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-04-02T14:04:32.796 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-04-02T14:04:32.797 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-04-02T14:04:32.797 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-04-02T14:04:32.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.797 INFO:teuthology.orchestra.run.smithi189.stdout: "objecter": { 2024-04-02T14:04:32.797 INFO:teuthology.orchestra.run.smithi189.stdout: "op_active": 0, 2024-04-02T14:04:32.797 INFO:teuthology.orchestra.run.smithi189.stdout: "op_laggy": 0, 2024-04-02T14:04:32.797 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send": 48, 2024-04-02T14:04:32.797 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send_bytes": 13420, 2024-04-02T14:04:32.798 INFO:teuthology.orchestra.run.smithi189.stdout: "op_resend": 30, 2024-04-02T14:04:32.798 INFO:teuthology.orchestra.run.smithi189.stdout: "op_reply": 48, 2024-04-02T14:04:32.798 INFO:teuthology.orchestra.run.smithi189.stdout: "op_latency": { 2024-04-02T14:04:32.798 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 48, 2024-04-02T14:04:32.798 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.233995570, 2024-04-02T14:04:32.798 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.004874907 2024-04-02T14:04:32.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.798 INFO:teuthology.orchestra.run.smithi189.stdout: "op_inflight": 0, 2024-04-02T14:04:32.799 INFO:teuthology.orchestra.run.smithi189.stdout: "oplen_avg": { 2024-04-02T14:04:32.799 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 48, 2024-04-02T14:04:32.799 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 72 2024-04-02T14:04:32.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.799 INFO:teuthology.orchestra.run.smithi189.stdout: "op": 48, 2024-04-02T14:04:32.799 INFO:teuthology.orchestra.run.smithi189.stdout: "op_r": 0, 2024-04-02T14:04:32.799 INFO:teuthology.orchestra.run.smithi189.stdout: "op_w": 48, 2024-04-02T14:04:32.799 INFO:teuthology.orchestra.run.smithi189.stdout: "op_rmw": 0, 2024-04-02T14:04:32.800 INFO:teuthology.orchestra.run.smithi189.stdout: "op_pg": 0, 2024-04-02T14:04:32.800 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_stat": 0, 2024-04-02T14:04:32.800 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_create": 10, 2024-04-02T14:04:32.800 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_read": 0, 2024-04-02T14:04:32.800 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_write": 9, 2024-04-02T14:04:32.800 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writefull": 8, 2024-04-02T14:04:32.800 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writesame": 0, 2024-04-02T14:04:32.800 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_append": 0, 2024-04-02T14:04:32.800 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_zero": 0, 2024-04-02T14:04:32.801 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_truncate": 0, 2024-04-02T14:04:32.801 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_delete": 6, 2024-04-02T14:04:32.801 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_mapext": 0, 2024-04-02T14:04:32.801 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_sparse_read": 0, 2024-04-02T14:04:32.801 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_clonerange": 0, 2024-04-02T14:04:32.801 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_getxattr": 0, 2024-04-02T14:04:32.801 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_setxattr": 20, 2024-04-02T14:04:32.801 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_cmpxattr": 0, 2024-04-02T14:04:32.801 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_rmxattr": 0, 2024-04-02T14:04:32.802 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_resetxattrs": 0, 2024-04-02T14:04:32.802 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_call": 0, 2024-04-02T14:04:32.802 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_watch": 0, 2024-04-02T14:04:32.802 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_notify": 0, 2024-04-02T14:04:32.802 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T14:04:32.802 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls": 0, 2024-04-02T14:04:32.802 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls_filter": 0, 2024-04-02T14:04:32.802 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_other": 0, 2024-04-02T14:04:32.802 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_active": 0, 2024-04-02T14:04:32.803 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_send": 0, 2024-04-02T14:04:32.803 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_resend": 0, 2024-04-02T14:04:32.803 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_ping": 0, 2024-04-02T14:04:32.803 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_active": 0, 2024-04-02T14:04:32.803 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_send": 0, 2024-04-02T14:04:32.803 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_resend": 0, 2024-04-02T14:04:32.803 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_active": 0, 2024-04-02T14:04:32.803 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_send": 0, 2024-04-02T14:04:32.804 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_resend": 0, 2024-04-02T14:04:32.804 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_active": 0, 2024-04-02T14:04:32.804 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_send": 0, 2024-04-02T14:04:32.804 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_resend": 0, 2024-04-02T14:04:32.804 INFO:teuthology.orchestra.run.smithi189.stdout: "command_active": 0, 2024-04-02T14:04:32.804 INFO:teuthology.orchestra.run.smithi189.stdout: "command_send": 0, 2024-04-02T14:04:32.804 INFO:teuthology.orchestra.run.smithi189.stdout: "command_resend": 0, 2024-04-02T14:04:32.804 INFO:teuthology.orchestra.run.smithi189.stdout: "map_epoch": 103, 2024-04-02T14:04:32.804 INFO:teuthology.orchestra.run.smithi189.stdout: "map_full": 0, 2024-04-02T14:04:32.805 INFO:teuthology.orchestra.run.smithi189.stdout: "map_inc": 13, 2024-04-02T14:04:32.805 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_sessions": 10, 2024-04-02T14:04:32.805 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_open": 10, 2024-04-02T14:04:32.805 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_close": 0, 2024-04-02T14:04:32.805 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_laggy": 0, 2024-04-02T14:04:32.805 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_wr": 17, 2024-04-02T14:04:32.805 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_rd": 0, 2024-04-02T14:04:32.805 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_del": 2 2024-04-02T14:04:32.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.805 INFO:teuthology.orchestra.run.smithi189.stdout: "oft": { 2024-04-02T14:04:32.806 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_objs": 1, 2024-04-02T14:04:32.806 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_kv_pairs": 0, 2024-04-02T14:04:32.806 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_updates": 0, 2024-04-02T14:04:32.806 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_removes": 0 2024-04-02T14:04:32.806 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.806 INFO:teuthology.orchestra.run.smithi189.stdout: "osd-slow-ops": { 2024-04-02T14:04:32.806 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_ops_count": 0 2024-04-02T14:04:32.806 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.806 INFO:teuthology.orchestra.run.smithi189.stdout: "purge_queue": { 2024-04-02T14:04:32.807 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops": 0, 2024-04-02T14:04:32.807 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T14:04:32.807 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing": 0, 2024-04-02T14:04:32.807 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_high_water": 0, 2024-04-02T14:04:32.807 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed_ops": 0, 2024-04-02T14:04:32.807 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed": 0, 2024-04-02T14:04:32.807 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_item_in_journal": 0 2024-04-02T14:04:32.807 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.807 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T14:04:32.808 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-02T14:04:32.808 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-04-02T14:04:32.808 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-02T14:04:32.808 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 1669, 2024-04-02T14:04:32.808 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 1940591, 2024-04-02T14:04:32.808 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:32.808 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 1669, 2024-04-02T14:04:32.808 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-02T14:04:32.808 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-02T14:04:32.809 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 1669, 2024-04-02T14:04:32.809 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 1940591, 2024-04-02T14:04:32.809 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-02T14:04:32.809 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.809 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.809 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.809 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:32.809 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.809 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_bytes": { 2024-04-02T14:04:32.810 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-02T14:04:32.810 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-04-02T14:04:32.810 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-02T14:04:32.810 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-02T14:04:32.810 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-02T14:04:32.810 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:32.810 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:32.810 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 48, 2024-04-02T14:04:32.810 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 13420, 2024-04-02T14:04:32.811 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 48, 2024-04-02T14:04:32.811 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 13420, 2024-04-02T14:04:32.811 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-02T14:04:32.811 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.811 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.811 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.811 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:32.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.811 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_ops": { 2024-04-02T14:04:32.812 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-02T14:04:32.812 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 1024, 2024-04-02T14:04:32.812 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-02T14:04:32.812 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-02T14:04:32.812 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-02T14:04:32.812 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:32.812 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:32.812 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 48, 2024-04-02T14:04:32.812 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 48, 2024-04-02T14:04:32.813 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 48, 2024-04-02T14:04:32.813 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 48, 2024-04-02T14:04:32.813 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-02T14:04:32.813 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.813 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.813 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.813 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:32.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.813 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle": { 2024-04-02T14:04:32.813 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-02T14:04:32.814 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-04-02T14:04:32.814 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-02T14:04:32.814 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-02T14:04:32.814 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-02T14:04:32.814 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:32.814 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:32.814 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-02T14:04:32.814 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-02T14:04:32.814 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-04-02T14:04:32.815 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-04-02T14:04:32.815 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-02T14:04:32.815 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.815 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.815 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.815 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:32.815 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:32.815 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle-0x5587234b4500": { 2024-04-02T14:04:32.815 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-02T14:04:32.816 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-04-02T14:04:32.816 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-02T14:04:32.816 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 10, 2024-04-02T14:04:32.816 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 2883, 2024-04-02T14:04:32.816 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:32.816 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 10, 2024-04-02T14:04:32.816 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-02T14:04:32.816 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-02T14:04:32.817 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 9, 2024-04-02T14:04:32.817 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 2883, 2024-04-02T14:04:32.817 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-02T14:04:32.817 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:32.817 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:32.817 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:32.817 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:32.817 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:32.817 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-04-02T14:04:33.567 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-02T14:04:33.567 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-02T14:04:33.567 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-04-02T14:04:33.567 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=73 2024-04-02T14:04:33.567 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-02T14:04:33.567 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-02T14:04:33.568 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T14:04:33.568 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-04-02T14:04:33.568 DEBUG:tasks.check_counter:Getting stats from l 2024-04-02T14:04:33.568 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 -- ceph daemon mds.l perf dump 2024-04-02T14:04:33.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:33 smithi189 ceph-mon[36683]: pgmap v344: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-04-02T14:04:33.810 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T14:04:33.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:33 smithi052 ceph-mon[32104]: pgmap v344: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-04-02T14:04:33.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:33 smithi088 ceph-mon[36528]: pgmap v344: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-04-02T14:04:35.496 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-04-02T14:04:35.497 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T14:04:35.497 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 2246, 2024-04-02T14:04:35.497 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 2246, 2024-04-02T14:04:35.497 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 641320, 2024-04-02T14:04:35.497 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 713411, 2024-04-02T14:04:35.497 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 6, 2024-04-02T14:04:35.498 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 2, 2024-04-02T14:04:35.498 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.200746001, 2024-04-02T14:04:35.498 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.067198758, 2024-04-02T14:04:35.498 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.049013420, 2024-04-02T14:04:35.498 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.068763396, 2024-04-02T14:04:35.498 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:35.498 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2245, 2024-04-02T14:04:35.498 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.089281206, 2024-04-02T14:04:35.499 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000039768 2024-04-02T14:04:35.499 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.499 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:35.499 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.499 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.499 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.499 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.499 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-04-02T14:04:35.500 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 448 2024-04-02T14:04:35.500 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.500 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T14:04:35.500 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 4, 2024-04-02T14:04:35.500 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 176, 2024-04-02T14:04:35.500 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 876, 2024-04-02T14:04:35.500 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 456090, 2024-04-02T14:04:35.500 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 6, 2024-04-02T14:04:35.500 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 4, 2024-04-02T14:04:35.501 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.011373271, 2024-04-02T14:04:35.501 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.007071766, 2024-04-02T14:04:35.501 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.002481951, 2024-04-02T14:04:35.501 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000058598, 2024-04-02T14:04:35.501 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:35.501 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 173, 2024-04-02T14:04:35.501 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.014128572, 2024-04-02T14:04:35.501 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000081668 2024-04-02T14:04:35.502 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.502 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:35.502 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.502 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.502 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.502 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.502 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-02T14:04:35.502 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 455392 2024-04-02T14:04:35.503 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.503 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T14:04:35.503 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 261, 2024-04-02T14:04:35.503 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 247, 2024-04-02T14:04:35.503 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 1642268, 2024-04-02T14:04:35.503 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 143662, 2024-04-02T14:04:35.503 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 3, 2024-04-02T14:04:35.504 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 2, 2024-04-02T14:04:35.504 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.050473595, 2024-04-02T14:04:35.504 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.015773770, 2024-04-02T14:04:35.504 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.038926804, 2024-04-02T14:04:35.504 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.005927863, 2024-04-02T14:04:35.504 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T14:04:35.504 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 246, 2024-04-02T14:04:35.505 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.014300725, 2024-04-02T14:04:35.505 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000058133 2024-04-02T14:04:35.505 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.505 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-02T14:04:35.505 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.505 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.505 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.505 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.505 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 1642000, 2024-04-02T14:04:35.506 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 143344 2024-04-02T14:04:35.506 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.506 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-04-02T14:04:35.506 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 1, 2024-04-02T14:04:35.506 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-04-02T14:04:35.506 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.506 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-MDSRank": { 2024-04-02T14:04:35.506 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-04-02T14:04:35.507 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-04-02T14:04:35.507 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 3817, 2024-04-02T14:04:35.507 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.108413017, 2024-04-02T14:04:35.507 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000028402 2024-04-02T14:04:35.507 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:35.507 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.507 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-PurgeQueue": { 2024-04-02T14:04:35.507 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-04-02T14:04:35.507 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-04-02T14:04:35.508 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.508 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.508 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.508 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:35.508 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.508 INFO:teuthology.orchestra.run.smithi189.stdout: "mds": { 2024-04-02T14:04:35.508 INFO:teuthology.orchestra.run.smithi189.stdout: "request": 0, 2024-04-02T14:04:35.508 INFO:teuthology.orchestra.run.smithi189.stdout: "reply": 0, 2024-04-02T14:04:35.509 INFO:teuthology.orchestra.run.smithi189.stdout: "reply_latency": { 2024-04-02T14:04:35.509 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.509 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.509 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.509 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.509 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_reply": 0, 2024-04-02T14:04:35.509 INFO:teuthology.orchestra.run.smithi189.stdout: "forward": 0, 2024-04-02T14:04:35.509 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_complete": 0, 2024-04-02T14:04:35.509 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_keys": 0, 2024-04-02T14:04:35.510 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_commit": 0, 2024-04-02T14:04:35.510 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_split": 0, 2024-04-02T14:04:35.510 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_merge": 0, 2024-04-02T14:04:35.510 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes": 0, 2024-04-02T14:04:35.510 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_top": 0, 2024-04-02T14:04:35.510 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_bottom": 0, 2024-04-02T14:04:35.510 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pin_tail": 0, 2024-04-02T14:04:35.510 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pinned": 0, 2024-04-02T14:04:35.511 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_expired": 0, 2024-04-02T14:04:35.511 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_with_caps": 0, 2024-04-02T14:04:35.511 INFO:teuthology.orchestra.run.smithi189.stdout: "caps": 0, 2024-04-02T14:04:35.511 INFO:teuthology.orchestra.run.smithi189.stdout: "subtrees": 1, 2024-04-02T14:04:35.511 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse": 0, 2024-04-02T14:04:35.511 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_hit": 0, 2024-04-02T14:04:35.511 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_forward": 0, 2024-04-02T14:04:35.511 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_discover": 0, 2024-04-02T14:04:35.511 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_dir_fetch": 0, 2024-04-02T14:04:35.512 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_remote_ino": 0, 2024-04-02T14:04:35.512 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_lock": 0, 2024-04-02T14:04:35.512 INFO:teuthology.orchestra.run.smithi189.stdout: "load_cent": 0, 2024-04-02T14:04:35.512 INFO:teuthology.orchestra.run.smithi189.stdout: "q": 0, 2024-04-02T14:04:35.512 INFO:teuthology.orchestra.run.smithi189.stdout: "exported": 0, 2024-04-02T14:04:35.512 INFO:teuthology.orchestra.run.smithi189.stdout: "exported_inodes": 0, 2024-04-02T14:04:35.512 INFO:teuthology.orchestra.run.smithi189.stdout: "imported": 0, 2024-04-02T14:04:35.512 INFO:teuthology.orchestra.run.smithi189.stdout: "imported_inodes": 0, 2024-04-02T14:04:35.512 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_dir_fetch": 0, 2024-04-02T14:04:35.513 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_backtrace_fetch": 0, 2024-04-02T14:04:35.513 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_peer_discover": 0, 2024-04-02T14:04:35.513 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rfiles": 0, 2024-04-02T14:04:35.513 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rbytes": 0, 2024-04-02T14:04:35.513 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rsnaps": 0, 2024-04-02T14:04:35.513 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T14:04:35.513 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_set_tag": 0, 2024-04-02T14:04:35.513 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T14:04:35.513 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_inotable_repaired": 0, 2024-04-02T14:04:35.514 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_inodes": 0, 2024-04-02T14:04:35.514 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T14:04:35.514 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T14:04:35.514 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_file_inodes": 0, 2024-04-02T14:04:35.514 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_inode_file_caps": 0, 2024-04-02T14:04:35.514 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T14:04:35.514 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_grant": 0, 2024-04-02T14:04:35.515 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T14:04:35.515 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T14:04:35.515 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T14:04:35.515 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps": 0, 2024-04-02T14:04:35.515 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps_dirty": 0, 2024-04-02T14:04:35.515 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_cap_release": 0, 2024-04-02T14:04:35.515 INFO:teuthology.orchestra.run.smithi189.stdout: "process_request_cap_release": 0 2024-04-02T14:04:35.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.515 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_cache": { 2024-04-02T14:04:35.516 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update": 0, 2024-04-02T14:04:35.516 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update_receipt": 0, 2024-04-02T14:04:35.516 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_try_discover": 0, 2024-04-02T14:04:35.516 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_send_discover": 0, 2024-04-02T14:04:35.516 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_handle_discover": 0, 2024-04-02T14:04:35.516 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays": 0, 2024-04-02T14:04:35.516 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_delayed": 0, 2024-04-02T14:04:35.516 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_enqueuing": 0, 2024-04-02T14:04:35.517 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_created": 0, 2024-04-02T14:04:35.517 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_enqueued": 0, 2024-04-02T14:04:35.517 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_reintegrated": 0, 2024-04-02T14:04:35.517 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_migrated": 0, 2024-04-02T14:04:35.517 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_processing": 0, 2024-04-02T14:04:35.517 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_enqueued": 0, 2024-04-02T14:04:35.517 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_prioritized": 0, 2024-04-02T14:04:35.517 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_started": 0, 2024-04-02T14:04:35.518 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_completed": 0, 2024-04-02T14:04:35.518 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_path": 0, 2024-04-02T14:04:35.518 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_inode": 0, 2024-04-02T14:04:35.518 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T14:04:35.518 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_exportdir": 0, 2024-04-02T14:04:35.518 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_flush": 0, 2024-04-02T14:04:35.518 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragmentdir": 0, 2024-04-02T14:04:35.519 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragstats": 0, 2024-04-02T14:04:35.519 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_inodestats": 0 2024-04-02T14:04:35.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.519 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_log": { 2024-04-02T14:04:35.519 INFO:teuthology.orchestra.run.smithi189.stdout: "evlrg": 0, 2024-04-02T14:04:35.519 INFO:teuthology.orchestra.run.smithi189.stdout: "evadd": 0, 2024-04-02T14:04:35.519 INFO:teuthology.orchestra.run.smithi189.stdout: "evex": 0, 2024-04-02T14:04:35.519 INFO:teuthology.orchestra.run.smithi189.stdout: "evtrm": 0, 2024-04-02T14:04:35.520 INFO:teuthology.orchestra.run.smithi189.stdout: "ev": 10, 2024-04-02T14:04:35.520 INFO:teuthology.orchestra.run.smithi189.stdout: "evexg": 0, 2024-04-02T14:04:35.520 INFO:teuthology.orchestra.run.smithi189.stdout: "evexd": 0, 2024-04-02T14:04:35.520 INFO:teuthology.orchestra.run.smithi189.stdout: "segadd": 0, 2024-04-02T14:04:35.520 INFO:teuthology.orchestra.run.smithi189.stdout: "segex": 0, 2024-04-02T14:04:35.520 INFO:teuthology.orchestra.run.smithi189.stdout: "segtrm": 0, 2024-04-02T14:04:35.520 INFO:teuthology.orchestra.run.smithi189.stdout: "seg": 2, 2024-04-02T14:04:35.520 INFO:teuthology.orchestra.run.smithi189.stdout: "segmjr": 2, 2024-04-02T14:04:35.520 INFO:teuthology.orchestra.run.smithi189.stdout: "segexg": 0, 2024-04-02T14:04:35.521 INFO:teuthology.orchestra.run.smithi189.stdout: "segexd": 0, 2024-04-02T14:04:35.521 INFO:teuthology.orchestra.run.smithi189.stdout: "expos": 4194304, 2024-04-02T14:04:35.521 INFO:teuthology.orchestra.run.smithi189.stdout: "wrpos": 4197187, 2024-04-02T14:04:35.521 INFO:teuthology.orchestra.run.smithi189.stdout: "rdpos": 4196879, 2024-04-02T14:04:35.521 INFO:teuthology.orchestra.run.smithi189.stdout: "jlat": { 2024-04-02T14:04:35.521 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.521 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.521 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.522 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.522 INFO:teuthology.orchestra.run.smithi189.stdout: "replayed": 10 2024-04-02T14:04:35.522 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.522 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_mem": { 2024-04-02T14:04:35.522 INFO:teuthology.orchestra.run.smithi189.stdout: "ino": 3, 2024-04-02T14:04:35.522 INFO:teuthology.orchestra.run.smithi189.stdout: "ino+": 3, 2024-04-02T14:04:35.522 INFO:teuthology.orchestra.run.smithi189.stdout: "ino-": 0, 2024-04-02T14:04:35.522 INFO:teuthology.orchestra.run.smithi189.stdout: "dir": 1, 2024-04-02T14:04:35.523 INFO:teuthology.orchestra.run.smithi189.stdout: "dir+": 1, 2024-04-02T14:04:35.523 INFO:teuthology.orchestra.run.smithi189.stdout: "dir-": 0, 2024-04-02T14:04:35.523 INFO:teuthology.orchestra.run.smithi189.stdout: "dn": 0, 2024-04-02T14:04:35.523 INFO:teuthology.orchestra.run.smithi189.stdout: "dn+": 0, 2024-04-02T14:04:35.523 INFO:teuthology.orchestra.run.smithi189.stdout: "dn-": 0, 2024-04-02T14:04:35.523 INFO:teuthology.orchestra.run.smithi189.stdout: "cap": 0, 2024-04-02T14:04:35.523 INFO:teuthology.orchestra.run.smithi189.stdout: "cap+": 0, 2024-04-02T14:04:35.523 INFO:teuthology.orchestra.run.smithi189.stdout: "cap-": 0, 2024-04-02T14:04:35.524 INFO:teuthology.orchestra.run.smithi189.stdout: "rss": 49716, 2024-04-02T14:04:35.524 INFO:teuthology.orchestra.run.smithi189.stdout: "heap": 215416 2024-04-02T14:04:35.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.524 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_server": { 2024-04-02T14:04:35.524 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_client_request": 0, 2024-04-02T14:04:35.524 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_server_request": 0, 2024-04-02T14:04:35.524 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_request": 0, 2024-04-02T14:04:35.524 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_session": 0, 2024-04-02T14:04:35.525 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_peer_request": 0, 2024-04-02T14:04:35.525 INFO:teuthology.orchestra.run.smithi189.stdout: "req_create_latency": { 2024-04-02T14:04:35.525 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.525 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.525 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.525 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getattr_latency": { 2024-04-02T14:04:35.525 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.525 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.526 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.526 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.526 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getfilelock_latency": { 2024-04-02T14:04:35.526 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.526 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.526 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.526 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.526 INFO:teuthology.orchestra.run.smithi189.stdout: "req_link_latency": { 2024-04-02T14:04:35.527 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.527 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.527 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.527 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookup_latency": { 2024-04-02T14:04:35.527 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.527 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.527 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.528 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookuphash_latency": { 2024-04-02T14:04:35.528 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.528 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.528 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.528 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupino_latency": { 2024-04-02T14:04:35.528 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.528 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.529 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.529 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupname_latency": { 2024-04-02T14:04:35.529 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.529 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.529 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.529 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupparent_latency": { 2024-04-02T14:04:35.529 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.530 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.530 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.530 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupsnap_latency": { 2024-04-02T14:04:35.530 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.530 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.530 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.531 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lssnap_latency": { 2024-04-02T14:04:35.531 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.531 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.531 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.531 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.531 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mkdir_latency": { 2024-04-02T14:04:35.531 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.532 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.532 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.532 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.532 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mknod_latency": { 2024-04-02T14:04:35.533 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.533 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.533 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.533 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mksnap_latency": { 2024-04-02T14:04:35.533 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.533 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.533 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.534 INFO:teuthology.orchestra.run.smithi189.stdout: "req_open_latency": { 2024-04-02T14:04:35.534 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.534 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.534 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.534 INFO:teuthology.orchestra.run.smithi189.stdout: "req_readdir_latency": { 2024-04-02T14:04:35.535 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.535 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.535 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.535 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rename_latency": { 2024-04-02T14:04:35.535 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.535 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.535 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.536 INFO:teuthology.orchestra.run.smithi189.stdout: "req_renamesnap_latency": { 2024-04-02T14:04:35.536 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.536 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.536 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.536 INFO:teuthology.orchestra.run.smithi189.stdout: "req_snapdiff_latency": { 2024-04-02T14:04:35.536 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.537 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.537 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.537 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmdir_latency": { 2024-04-02T14:04:35.537 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.537 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.537 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.537 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmsnap_latency": { 2024-04-02T14:04:35.538 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.538 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.538 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.538 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmxattr_latency": { 2024-04-02T14:04:35.538 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.538 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.538 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.539 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setattr_latency": { 2024-04-02T14:04:35.539 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.539 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.539 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.539 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setdirlayout_latency": { 2024-04-02T14:04:35.539 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.540 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.540 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.540 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setfilelock_latency": { 2024-04-02T14:04:35.540 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.540 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.540 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.541 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setlayout_latency": { 2024-04-02T14:04:35.541 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.541 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.541 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.541 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setxattr_latency": { 2024-04-02T14:04:35.541 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.541 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.542 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.542 INFO:teuthology.orchestra.run.smithi189.stdout: "req_symlink_latency": { 2024-04-02T14:04:35.542 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.542 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.542 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.542 INFO:teuthology.orchestra.run.smithi189.stdout: "req_unlink_latency": { 2024-04-02T14:04:35.543 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.543 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.543 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.543 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_revoke_eviction": 0, 2024-04-02T14:04:35.543 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_acquisition_throttle": 0, 2024-04-02T14:04:35.543 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getvxattr_latency": { 2024-04-02T14:04:35.543 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.544 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.544 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.544 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:35.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.544 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_sessions": { 2024-04-02T14:04:35.544 INFO:teuthology.orchestra.run.smithi189.stdout: "session_count": 2, 2024-04-02T14:04:35.544 INFO:teuthology.orchestra.run.smithi189.stdout: "session_add": 5, 2024-04-02T14:04:35.544 INFO:teuthology.orchestra.run.smithi189.stdout: "session_remove": 3, 2024-04-02T14:04:35.545 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_open": 2, 2024-04-02T14:04:35.545 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_stale": 0, 2024-04-02T14:04:35.545 INFO:teuthology.orchestra.run.smithi189.stdout: "total_load": 0, 2024-04-02T14:04:35.545 INFO:teuthology.orchestra.run.smithi189.stdout: "average_load": 0, 2024-04-02T14:04:35.545 INFO:teuthology.orchestra.run.smithi189.stdout: "avg_session_uptime": 6243, 2024-04-02T14:04:35.545 INFO:teuthology.orchestra.run.smithi189.stdout: "mdthresh_evicted": 0 2024-04-02T14:04:35.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.545 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-04-02T14:04:35.545 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-04-02T14:04:35.546 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-04-02T14:04:35.546 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T14:04:35.546 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-04-02T14:04:35.546 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T14:04:35.546 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-04-02T14:04:35.546 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T14:04:35.546 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T14:04:35.546 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T14:04:35.547 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T14:04:35.547 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T14:04:35.547 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-04-02T14:04:35.547 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T14:04:35.547 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T14:04:35.547 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-04-02T14:04:35.547 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-04-02T14:04:35.547 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-04-02T14:04:35.547 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-04-02T14:04:35.548 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T14:04:35.548 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T14:04:35.548 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T14:04:35.548 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T14:04:35.548 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T14:04:35.548 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-04-02T14:04:35.548 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-04-02T14:04:35.548 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-04-02T14:04:35.549 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T14:04:35.549 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T14:04:35.549 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-04-02T14:04:35.549 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-04-02T14:04:35.549 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-04-02T14:04:35.549 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-04-02T14:04:35.549 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T14:04:35.549 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-04-02T14:04:35.549 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T14:04:35.550 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-04-02T14:04:35.550 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 182076, 2024-04-02T14:04:35.550 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 49, 2024-04-02T14:04:35.550 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 0, 2024-04-02T14:04:35.550 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 0, 2024-04-02T14:04:35.550 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-04-02T14:04:35.550 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-04-02T14:04:35.550 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-04-02T14:04:35.551 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-04-02T14:04:35.551 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-04-02T14:04:35.551 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-04-02T14:04:35.551 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 9320, 2024-04-02T14:04:35.551 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 178, 2024-04-02T14:04:35.551 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T14:04:35.551 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-04-02T14:04:35.551 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-04-02T14:04:35.552 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-04-02T14:04:35.552 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 8464, 2024-04-02T14:04:35.552 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 13, 2024-04-02T14:04:35.552 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-04-02T14:04:35.552 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-04-02T14:04:35.552 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-04-02T14:04:35.552 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-04-02T14:04:35.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.552 INFO:teuthology.orchestra.run.smithi189.stdout: "objecter": { 2024-04-02T14:04:35.553 INFO:teuthology.orchestra.run.smithi189.stdout: "op_active": 0, 2024-04-02T14:04:35.553 INFO:teuthology.orchestra.run.smithi189.stdout: "op_laggy": 0, 2024-04-02T14:04:35.553 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send": 2245, 2024-04-02T14:04:35.553 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send_bytes": 21, 2024-04-02T14:04:35.553 INFO:teuthology.orchestra.run.smithi189.stdout: "op_resend": 3, 2024-04-02T14:04:35.553 INFO:teuthology.orchestra.run.smithi189.stdout: "op_reply": 2245, 2024-04-02T14:04:35.553 INFO:teuthology.orchestra.run.smithi189.stdout: "op_latency": { 2024-04-02T14:04:35.553 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2245, 2024-04-02T14:04:35.554 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 1.153978033, 2024-04-02T14:04:35.554 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000514021 2024-04-02T14:04:35.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.554 INFO:teuthology.orchestra.run.smithi189.stdout: "op_inflight": 0, 2024-04-02T14:04:35.554 INFO:teuthology.orchestra.run.smithi189.stdout: "oplen_avg": { 2024-04-02T14:04:35.554 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2245, 2024-04-02T14:04:35.554 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 2246 2024-04-02T14:04:35.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.555 INFO:teuthology.orchestra.run.smithi189.stdout: "op": 2245, 2024-04-02T14:04:35.555 INFO:teuthology.orchestra.run.smithi189.stdout: "op_r": 2245, 2024-04-02T14:04:35.555 INFO:teuthology.orchestra.run.smithi189.stdout: "op_w": 0, 2024-04-02T14:04:35.555 INFO:teuthology.orchestra.run.smithi189.stdout: "op_rmw": 0, 2024-04-02T14:04:35.555 INFO:teuthology.orchestra.run.smithi189.stdout: "op_pg": 0, 2024-04-02T14:04:35.555 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_stat": 1449, 2024-04-02T14:04:35.555 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_create": 0, 2024-04-02T14:04:35.555 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_read": 794, 2024-04-02T14:04:35.555 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_write": 0, 2024-04-02T14:04:35.556 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writefull": 0, 2024-04-02T14:04:35.556 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writesame": 0, 2024-04-02T14:04:35.556 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_append": 0, 2024-04-02T14:04:35.556 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_zero": 0, 2024-04-02T14:04:35.556 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_truncate": 0, 2024-04-02T14:04:35.556 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_delete": 0, 2024-04-02T14:04:35.556 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_mapext": 0, 2024-04-02T14:04:35.556 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_sparse_read": 0, 2024-04-02T14:04:35.557 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_clonerange": 0, 2024-04-02T14:04:35.557 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_getxattr": 1, 2024-04-02T14:04:35.557 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_setxattr": 0, 2024-04-02T14:04:35.557 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_cmpxattr": 0, 2024-04-02T14:04:35.557 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_rmxattr": 0, 2024-04-02T14:04:35.557 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_resetxattrs": 0, 2024-04-02T14:04:35.557 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_call": 0, 2024-04-02T14:04:35.557 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_watch": 0, 2024-04-02T14:04:35.557 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_notify": 0, 2024-04-02T14:04:35.558 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T14:04:35.558 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls": 0, 2024-04-02T14:04:35.558 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls_filter": 0, 2024-04-02T14:04:35.558 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_other": 0, 2024-04-02T14:04:35.558 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_active": 0, 2024-04-02T14:04:35.558 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_send": 0, 2024-04-02T14:04:35.558 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_resend": 0, 2024-04-02T14:04:35.558 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_ping": 0, 2024-04-02T14:04:35.559 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_active": 0, 2024-04-02T14:04:35.559 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_send": 0, 2024-04-02T14:04:35.559 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_resend": 0, 2024-04-02T14:04:35.559 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_active": 0, 2024-04-02T14:04:35.559 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_send": 0, 2024-04-02T14:04:35.559 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_resend": 0, 2024-04-02T14:04:35.559 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_active": 0, 2024-04-02T14:04:35.559 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_send": 0, 2024-04-02T14:04:35.560 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_resend": 0, 2024-04-02T14:04:35.560 INFO:teuthology.orchestra.run.smithi189.stdout: "command_active": 0, 2024-04-02T14:04:35.560 INFO:teuthology.orchestra.run.smithi189.stdout: "command_send": 0, 2024-04-02T14:04:35.560 INFO:teuthology.orchestra.run.smithi189.stdout: "command_resend": 0, 2024-04-02T14:04:35.560 INFO:teuthology.orchestra.run.smithi189.stdout: "map_epoch": 103, 2024-04-02T14:04:35.560 INFO:teuthology.orchestra.run.smithi189.stdout: "map_full": 0, 2024-04-02T14:04:35.560 INFO:teuthology.orchestra.run.smithi189.stdout: "map_inc": 13, 2024-04-02T14:04:35.560 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_sessions": 4, 2024-04-02T14:04:35.561 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_open": 4, 2024-04-02T14:04:35.561 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_close": 0, 2024-04-02T14:04:35.561 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_laggy": 0, 2024-04-02T14:04:35.561 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_wr": 0, 2024-04-02T14:04:35.561 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_rd": 2, 2024-04-02T14:04:35.561 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_del": 0 2024-04-02T14:04:35.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.561 INFO:teuthology.orchestra.run.smithi189.stdout: "oft": { 2024-04-02T14:04:35.562 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_objs": 0, 2024-04-02T14:04:35.562 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_kv_pairs": 0, 2024-04-02T14:04:35.562 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_updates": 0, 2024-04-02T14:04:35.562 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_removes": 0 2024-04-02T14:04:35.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.562 INFO:teuthology.orchestra.run.smithi189.stdout: "osd-slow-ops": { 2024-04-02T14:04:35.562 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_ops_count": 0 2024-04-02T14:04:35.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.563 INFO:teuthology.orchestra.run.smithi189.stdout: "purge_queue": { 2024-04-02T14:04:35.563 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops": 0, 2024-04-02T14:04:35.563 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T14:04:35.563 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing": 0, 2024-04-02T14:04:35.563 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_high_water": 0, 2024-04-02T14:04:35.563 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed_ops": 0, 2024-04-02T14:04:35.564 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed": 0, 2024-04-02T14:04:35.564 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_item_in_journal": 0 2024-04-02T14:04:35.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.564 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T14:04:35.564 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-02T14:04:35.564 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-04-02T14:04:35.564 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-02T14:04:35.564 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 2507, 2024-04-02T14:04:35.565 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 2042420, 2024-04-02T14:04:35.565 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:35.565 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 2507, 2024-04-02T14:04:35.565 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-02T14:04:35.565 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-02T14:04:35.565 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 2507, 2024-04-02T14:04:35.565 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 2042420, 2024-04-02T14:04:35.565 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-02T14:04:35.565 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.566 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.566 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.566 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:35.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.566 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_bytes": { 2024-04-02T14:04:35.566 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-02T14:04:35.566 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-04-02T14:04:35.566 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-02T14:04:35.567 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-02T14:04:35.567 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-02T14:04:35.567 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:35.567 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:35.567 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 2245, 2024-04-02T14:04:35.567 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 2888, 2024-04-02T14:04:35.567 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 2245, 2024-04-02T14:04:35.568 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 2888, 2024-04-02T14:04:35.568 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-02T14:04:35.568 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.568 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.568 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.568 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:35.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.568 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_ops": { 2024-04-02T14:04:35.569 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-02T14:04:35.569 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 1024, 2024-04-02T14:04:35.569 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-02T14:04:35.569 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-02T14:04:35.569 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-02T14:04:35.569 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:35.569 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:35.569 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 2245, 2024-04-02T14:04:35.569 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 2245, 2024-04-02T14:04:35.570 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 2245, 2024-04-02T14:04:35.570 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 2245, 2024-04-02T14:04:35.570 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-02T14:04:35.570 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.570 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.570 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.570 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:35.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.571 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle": { 2024-04-02T14:04:35.571 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-02T14:04:35.571 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-04-02T14:04:35.571 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-02T14:04:35.571 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-02T14:04:35.571 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-02T14:04:35.571 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:35.572 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:35.572 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-02T14:04:35.572 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-02T14:04:35.572 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-04-02T14:04:35.572 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-04-02T14:04:35.572 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-02T14:04:35.572 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.572 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.573 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.573 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:35.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-02T14:04:35.573 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle-0x55f34a9586e0": { 2024-04-02T14:04:35.573 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-02T14:04:35.573 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-04-02T14:04:35.573 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-02T14:04:35.574 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-02T14:04:35.574 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-02T14:04:35.574 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-02T14:04:35.574 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-02T14:04:35.574 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-02T14:04:35.574 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-02T14:04:35.577 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-04-02T14:04:35.577 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-04-02T14:04:35.577 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-02T14:04:35.577 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-02T14:04:35.577 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-02T14:04:35.577 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-02T14:04:35.578 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:35.578 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-02T14:04:35.578 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-04-02T14:04:35.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:35 smithi189 ceph-mon[36683]: pgmap v345: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:04:35.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:35 smithi052 ceph-mon[32104]: pgmap v345: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:04:35.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:35 smithi088 ceph-mon[36528]: pgmap v345: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:04:36.298 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-02T14:04:36.298 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-02T14:04:36.298 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-04-02T14:04:36.298 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-04-02T14:04:36.298 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-04-02T14:04:36.298 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-04-02T14:04:36.299 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T14:04:36.299 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-04-02T14:04:36.299 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-02T14:04:36.310 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-02T14:04:36.311 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-04-02T14:04:36.311 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T14:04:36.331 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-04-02T14:04:36.331 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T14:04:36.390 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T14:04:36.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.540+0000 7efe227e9640 1 -- 172.21.15.52:0/3920272035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe1c106e80 msgr2=0x7efe1c10ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:36.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.540+0000 7efe227e9640 1 --2- 172.21.15.52:0/3920272035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe1c106e80 0x7efe1c10ea00 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7efe14009ca0 tx=0x7efe1402c0f0 comp rx=0 tx=0).stop 2024-04-02T14:04:36.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.540+0000 7efe227e9640 1 -- 172.21.15.52:0/3920272035 shutdown_connections 2024-04-02T14:04:36.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.540+0000 7efe227e9640 1 --2- 172.21.15.52:0/3920272035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe1c106e80 0x7efe1c10ea00 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:36.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.540+0000 7efe227e9640 1 --2- 172.21.15.52:0/3920272035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe1c100c20 0x7efe1c106750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:36.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.540+0000 7efe227e9640 1 --2- 172.21.15.52:0/3920272035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe1c100270 0x7efe1c100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:36.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.540+0000 7efe227e9640 1 -- 172.21.15.52:0/3920272035 >> 172.21.15.52:0/3920272035 conn(0x7efe1c076f90 msgr2=0x7efe1c0fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:36.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.540+0000 7efe227e9640 1 -- 172.21.15.52:0/3920272035 shutdown_connections 2024-04-02T14:04:36.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.540+0000 7efe227e9640 1 -- 172.21.15.52:0/3920272035 wait complete. 2024-04-02T14:04:36.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.541+0000 7efe227e9640 1 Processor -- start 2024-04-02T14:04:36.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.541+0000 7efe227e9640 1 -- start start 2024-04-02T14:04:36.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.541+0000 7efe227e9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe1c100270 0x7efe1c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:36.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.541+0000 7efe227e9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe1c106e80 0x7efe1c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:36.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.541+0000 7efe227e9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe1c19eea0 0x7efe1c19f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:36.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.541+0000 7efe227e9640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c111ee0 con 0x7efe1c100270 2024-04-02T14:04:36.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.541+0000 7efe227e9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c111d60 con 0x7efe1c19eea0 2024-04-02T14:04:36.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.541+0000 7efe227e9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c112060 con 0x7efe1c106e80 2024-04-02T14:04:36.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.541+0000 7efe1bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe1c100270 0x7efe1c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:36.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.541+0000 7efe1bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe1c100270 0x7efe1c19a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51376/0 (socket says 172.21.15.52:51376) 2024-04-02T14:04:36.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.541+0000 7efe1bfff640 1 -- 172.21.15.52:0/1565504556 learned_addr learned my addr 172.21.15.52:0/1565504556 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:04:36.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.541+0000 7efe20d86640 1 --2- 172.21.15.52:0/1565504556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe1c19eea0 0x7efe1c19f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:36.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.541+0000 7efe1b7fe640 1 --2- 172.21.15.52:0/1565504556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe1c106e80 0x7efe1c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:36.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.542+0000 7efe1bfff640 1 -- 172.21.15.52:0/1565504556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe1c106e80 msgr2=0x7efe1c19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:36.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.542+0000 7efe1bfff640 1 --2- 172.21.15.52:0/1565504556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe1c106e80 0x7efe1c19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:36.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.542+0000 7efe1bfff640 1 -- 172.21.15.52:0/1565504556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe1c19eea0 msgr2=0x7efe1c19f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:36.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.542+0000 7efe1bfff640 1 --2- 172.21.15.52:0/1565504556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe1c19eea0 0x7efe1c19f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:36.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.542+0000 7efe1bfff640 1 -- 172.21.15.52:0/1565504556 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1c19f990 con 0x7efe1c100270 2024-04-02T14:04:36.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.542+0000 7efe1bfff640 1 --2- 172.21.15.52:0/1565504556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe1c100270 0x7efe1c19a5e0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7efe1000e9f0 tx=0x7efe1000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:36.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.542+0000 7efe197fa640 1 -- 172.21.15.52:0/1565504556 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1000cd80 con 0x7efe1c100270 2024-04-02T14:04:36.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.542+0000 7efe197fa640 1 -- 172.21.15.52:0/1565504556 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe10005500 con 0x7efe1c100270 2024-04-02T14:04:36.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.542+0000 7efe227e9640 1 -- 172.21.15.52:0/1565504556 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe1c19fc80 con 0x7efe1c100270 2024-04-02T14:04:36.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.542+0000 7efe197fa640 1 -- 172.21.15.52:0/1565504556 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10010690 con 0x7efe1c100270 2024-04-02T14:04:36.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.542+0000 7efe227e9640 1 -- 172.21.15.52:0/1565504556 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe1c1a7420 con 0x7efe1c100270 2024-04-02T14:04:36.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.543+0000 7efe227e9640 1 -- 172.21.15.52:0/1565504556 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe1c103320 con 0x7efe1c100270 2024-04-02T14:04:36.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.544+0000 7efe197fa640 1 -- 172.21.15.52:0/1565504556 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efe100046c0 con 0x7efe1c100270 2024-04-02T14:04:36.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.545+0000 7efe197fa640 1 --2- 172.21.15.52:0/1565504556 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efe040a3d20 0x7efe040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:36.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.545+0000 7efe1b7fe640 1 --2- 172.21.15.52:0/1565504556 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efe040a3d20 0x7efe040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:36.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.545+0000 7efe197fa640 1 -- 172.21.15.52:0/1565504556 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7efe100d1370 con 0x7efe1c100270 2024-04-02T14:04:36.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.545+0000 7efe1b7fe640 1 --2- 172.21.15.52:0/1565504556 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efe040a3d20 0x7efe040a61e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efe0c005e00 tx=0x7efe0c005c50 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:36.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.548+0000 7efe197fa640 1 -- 172.21.15.52:0/1565504556 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7efe10053f20 con 0x7efe1c100270 2024-04-02T14:04:36.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.736+0000 7efe227e9640 1 -- 172.21.15.52:0/1565504556 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe1c068d40 con 0x7efe1c100270 2024-04-02T14:04:36.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.738+0000 7efe197fa640 1 -- 172.21.15.52:0/1565504556 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+25112 (secure 0 0 0) 0x7efe1001f340 con 0x7efe1c100270 2024-04-02T14:04:36.742 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T14:04:36.743 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":103,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T14:04:15.482659+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:50:58.778690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:50:59.561527+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:51:03.343487+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/713755826":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/1937276046":"2024-04-03T13:53:29.889159+0000","172.21.15.52:6800/3836085837":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/2957262850":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/3836085837":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/2985814329":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T14:04:36.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.740+0000 7efe227e9640 1 -- 172.21.15.52:0/1565504556 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efe040a3d20 msgr2=0x7efe040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:36.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.740+0000 7efe227e9640 1 --2- 172.21.15.52:0/1565504556 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efe040a3d20 0x7efe040a61e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efe0c005e00 tx=0x7efe0c005c50 comp rx=0 tx=0).stop 2024-04-02T14:04:36.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.740+0000 7efe227e9640 1 -- 172.21.15.52:0/1565504556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe1c100270 msgr2=0x7efe1c19a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:36.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.740+0000 7efe227e9640 1 --2- 172.21.15.52:0/1565504556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe1c100270 0x7efe1c19a5e0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7efe1000e9f0 tx=0x7efe1000edf0 comp rx=0 tx=0).stop 2024-04-02T14:04:36.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.740+0000 7efe227e9640 1 -- 172.21.15.52:0/1565504556 shutdown_connections 2024-04-02T14:04:36.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.740+0000 7efe227e9640 1 --2- 172.21.15.52:0/1565504556 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7efe040a3d20 0x7efe040a61e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:36.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.741+0000 7efe227e9640 1 --2- 172.21.15.52:0/1565504556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe1c19eea0 0x7efe1c19f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:36.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.741+0000 7efe227e9640 1 --2- 172.21.15.52:0/1565504556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe1c106e80 0x7efe1c19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:36.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.741+0000 7efe227e9640 1 --2- 172.21.15.52:0/1565504556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe1c100270 0x7efe1c19a5e0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:36.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.741+0000 7efe227e9640 1 -- 172.21.15.52:0/1565504556 >> 172.21.15.52:0/1565504556 conn(0x7efe1c076f90 msgr2=0x7efe1c105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:36.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.741+0000 7efe227e9640 1 -- 172.21.15.52:0/1565504556 shutdown_connections 2024-04-02T14:04:36.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.741+0000 7efe227e9640 1 -- 172.21.15.52:0/1565504556 wait complete. 2024-04-02T14:04:36.755 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:46:20.746053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:50:58.778690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:50:59.561527+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:51:03.343487+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T14:04:36.756 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T14:04:36.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.858+0000 7f2384d9f640 1 -- 172.21.15.52:0/3080418474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2380106870 msgr2=0x7f2380106c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:36.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.858+0000 7f2384d9f640 1 --2- 172.21.15.52:0/3080418474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2380106870 0x7f2380106c50 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f2370009930 tx=0x7f237002bc30 comp rx=0 tx=0).stop 2024-04-02T14:04:36.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.858+0000 7f2384d9f640 1 -- 172.21.15.52:0/3080418474 shutdown_connections 2024-04-02T14:04:36.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.858+0000 7f2384d9f640 1 --2- 172.21.15.52:0/3080418474 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2380101220 0x7f238010d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:36.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.858+0000 7f2384d9f640 1 --2- 172.21.15.52:0/3080418474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2380100860 0x7f2380100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:36.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.858+0000 7f2384d9f640 1 --2- 172.21.15.52:0/3080418474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2380106870 0x7f2380106c50 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:36.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.858+0000 7f2384d9f640 1 -- 172.21.15.52:0/3080418474 >> 172.21.15.52:0/3080418474 conn(0x7f23800fc330 msgr2=0x7f23800fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:36.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.858+0000 7f2384d9f640 1 -- 172.21.15.52:0/3080418474 shutdown_connections 2024-04-02T14:04:36.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.858+0000 7f2384d9f640 1 -- 172.21.15.52:0/3080418474 wait complete. 2024-04-02T14:04:36.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.859+0000 7f2384d9f640 1 Processor -- start 2024-04-02T14:04:36.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.859+0000 7f2384d9f640 1 -- start start 2024-04-02T14:04:36.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.859+0000 7f2384d9f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2380100860 0x7f23800fc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:36.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.859+0000 7f2384d9f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2380101220 0x7f23800fceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:36.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.859+0000 7f2384d9f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2380106870 0x7f23800fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:36.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.859+0000 7f2384d9f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2380110c40 con 0x7f2380101220 2024-04-02T14:04:36.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.859+0000 7f2384d9f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2380110ac0 con 0x7f2380106870 2024-04-02T14:04:36.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.859+0000 7f2384d9f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2380110dc0 con 0x7f2380100860 2024-04-02T14:04:36.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.859+0000 7f237effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2380101220 0x7f23800fceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:36.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.859+0000 7f237effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2380101220 0x7f23800fceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51404/0 (socket says 172.21.15.52:51404) 2024-04-02T14:04:36.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.859+0000 7f237effd640 1 -- 172.21.15.52:0/1116436582 learned_addr learned my addr 172.21.15.52:0/1116436582 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:04:36.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.860+0000 7f237f7fe640 1 --2- 172.21.15.52:0/1116436582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2380100860 0x7f23800fc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:36.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.860+0000 7f237ffff640 1 --2- 172.21.15.52:0/1116436582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2380106870 0x7f23800fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:36.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.860+0000 7f237effd640 1 -- 172.21.15.52:0/1116436582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2380100860 msgr2=0x7f23800fc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:36.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.860+0000 7f237effd640 1 --2- 172.21.15.52:0/1116436582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2380100860 0x7f23800fc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:36.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.860+0000 7f237effd640 1 -- 172.21.15.52:0/1116436582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2380106870 msgr2=0x7f23800fd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:36.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.860+0000 7f237effd640 1 --2- 172.21.15.52:0/1116436582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2380106870 0x7f23800fd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:36.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.860+0000 7f237effd640 1 -- 172.21.15.52:0/1116436582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23801a6770 con 0x7f2380101220 2024-04-02T14:04:36.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.860+0000 7f237f7fe640 1 --2- 172.21.15.52:0/1116436582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2380100860 0x7f23800fc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:04:36.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.860+0000 7f237ffff640 1 --2- 172.21.15.52:0/1116436582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2380106870 0x7f23800fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:04:36.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.860+0000 7f237effd640 1 --2- 172.21.15.52:0/1116436582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2380101220 0x7f23800fceb0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f237400c890 tx=0x7f237400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:36.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.860+0000 7f237cff9640 1 -- 172.21.15.52:0/1116436582 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2374004220 con 0x7f2380101220 2024-04-02T14:04:36.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.860+0000 7f2384d9f640 1 -- 172.21.15.52:0/1116436582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23801a6a60 con 0x7f2380101220 2024-04-02T14:04:36.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.860+0000 7f237cff9640 1 -- 172.21.15.52:0/1116436582 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23740043c0 con 0x7f2380101220 2024-04-02T14:04:36.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.860+0000 7f2384d9f640 1 -- 172.21.15.52:0/1116436582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23801a7000 con 0x7f2380101220 2024-04-02T14:04:36.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.860+0000 7f237cff9640 1 -- 172.21.15.52:0/1116436582 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237401d060 con 0x7f2380101220 2024-04-02T14:04:36.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.861+0000 7f2384d9f640 1 -- 172.21.15.52:0/1116436582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2380068c60 con 0x7f2380101220 2024-04-02T14:04:36.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.862+0000 7f237cff9640 1 -- 172.21.15.52:0/1116436582 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f23740079b0 con 0x7f2380101220 2024-04-02T14:04:36.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.863+0000 7f237cff9640 1 --2- 172.21.15.52:0/1116436582 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f23680a3d20 0x7f23680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:36.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.863+0000 7f237f7fe640 1 --2- 172.21.15.52:0/1116436582 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f23680a3d20 0x7f23680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:36.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.863+0000 7f237cff9640 1 -- 172.21.15.52:0/1116436582 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f23740d08c0 con 0x7f2380101220 2024-04-02T14:04:36.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.863+0000 7f237f7fe640 1 --2- 172.21.15.52:0/1116436582 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f23680a3d20 0x7f23680a61e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2370006010 tx=0x7f237002c7d0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:36.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:36.867+0000 7f237cff9640 1 -- 172.21.15.52:0/1116436582 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2374014030 con 0x7f2380101220 2024-04-02T14:04:37.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.050+0000 7f2384d9f640 1 -- 172.21.15.52:0/1116436582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2380109de0 con 0x7f2380101220 2024-04-02T14:04:37.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.050+0000 7f237cff9640 1 -- 172.21.15.52:0/1116436582 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) ==== 93+0+10 (secure 0 0 0) 0x7f237409cbd0 con 0x7f2380101220 2024-04-02T14:04:37.055 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-04-02T14:04:37.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.052+0000 7f2384d9f640 1 -- 172.21.15.52:0/1116436582 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f23680a3d20 msgr2=0x7f23680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:37.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.052+0000 7f2384d9f640 1 --2- 172.21.15.52:0/1116436582 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f23680a3d20 0x7f23680a61e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2370006010 tx=0x7f237002c7d0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.052+0000 7f2384d9f640 1 -- 172.21.15.52:0/1116436582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2380101220 msgr2=0x7f23800fceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:37.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.052+0000 7f2384d9f640 1 --2- 172.21.15.52:0/1116436582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2380101220 0x7f23800fceb0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f237400c890 tx=0x7f237400cd50 comp rx=0 tx=0).stop 2024-04-02T14:04:37.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.052+0000 7f2384d9f640 1 -- 172.21.15.52:0/1116436582 shutdown_connections 2024-04-02T14:04:37.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.052+0000 7f2384d9f640 1 --2- 172.21.15.52:0/1116436582 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f23680a3d20 0x7f23680a61e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.052+0000 7f2384d9f640 1 --2- 172.21.15.52:0/1116436582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2380106870 0x7f23800fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.052+0000 7f2384d9f640 1 --2- 172.21.15.52:0/1116436582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2380101220 0x7f23800fceb0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.052+0000 7f2384d9f640 1 --2- 172.21.15.52:0/1116436582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2380100860 0x7f23800fc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.052+0000 7f2384d9f640 1 -- 172.21.15.52:0/1116436582 >> 172.21.15.52:0/1116436582 conn(0x7f23800fc330 msgr2=0x7f238006a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:37.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.053+0000 7f2384d9f640 1 -- 172.21.15.52:0/1116436582 shutdown_connections 2024-04-02T14:04:37.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.053+0000 7f2384d9f640 1 -- 172.21.15.52:0/1116436582 wait complete. 2024-04-02T14:04:37.067 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T14:04:37.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.168+0000 7f6cd72c8640 1 -- 172.21.15.52:0/2875673511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cd0100d60 msgr2=0x7f6cd010d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:37.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.168+0000 7f6cd72c8640 1 --2- 172.21.15.52:0/2875673511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cd0100d60 0x7f6cd010d030 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0009900 tx=0x7f6cc002bc30 comp rx=0 tx=0).stop 2024-04-02T14:04:37.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.169+0000 7f6cd72c8640 1 -- 172.21.15.52:0/2875673511 shutdown_connections 2024-04-02T14:04:37.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.169+0000 7f6cd72c8640 1 --2- 172.21.15.52:0/2875673511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cd0100d60 0x7f6cd010d030 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.169+0000 7f6cd72c8640 1 --2- 172.21.15.52:0/2875673511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cd01003c0 0x7f6cd0100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.169+0000 7f6cd72c8640 1 --2- 172.21.15.52:0/2875673511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cd0105130 0x7f6cd0105510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.169+0000 7f6cd72c8640 1 -- 172.21.15.52:0/2875673511 >> 172.21.15.52:0/2875673511 conn(0x7f6cd0075d40 msgr2=0x7f6cd0076150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:37.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.169+0000 7f6cd72c8640 1 -- 172.21.15.52:0/2875673511 shutdown_connections 2024-04-02T14:04:37.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.169+0000 7f6cd72c8640 1 -- 172.21.15.52:0/2875673511 wait complete. 2024-04-02T14:04:37.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.169+0000 7f6cd72c8640 1 Processor -- start 2024-04-02T14:04:37.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.169+0000 7f6cd72c8640 1 -- start start 2024-04-02T14:04:37.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.169+0000 7f6cd72c8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cd01003c0 0x7f6cd0196180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:37.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.169+0000 7f6cd72c8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cd0100d60 0x7f6cd01966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:37.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.169+0000 7f6cd72c8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cd0105130 0x7f6cd019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:37.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.169+0000 7f6cd72c8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd010fcc0 con 0x7f6cd01003c0 2024-04-02T14:04:37.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.169+0000 7f6cd72c8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd010fb40 con 0x7f6cd0105130 2024-04-02T14:04:37.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.169+0000 7f6cd72c8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd010fe40 con 0x7f6cd0100d60 2024-04-02T14:04:37.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.169+0000 7f6cd5064640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cd01003c0 0x7f6cd0196180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:37.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.169+0000 7f6cd5064640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cd01003c0 0x7f6cd0196180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51424/0 (socket says 172.21.15.52:51424) 2024-04-02T14:04:37.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.170+0000 7f6cd5064640 1 -- 172.21.15.52:0/3162808752 learned_addr learned my addr 172.21.15.52:0/3162808752 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:04:37.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.170+0000 7f6cd5865640 1 --2- 172.21.15.52:0/3162808752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cd0105130 0x7f6cd019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:37.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.170+0000 7f6cd4863640 1 --2- 172.21.15.52:0/3162808752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cd0100d60 0x7f6cd01966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:37.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.170+0000 7f6cd5064640 1 -- 172.21.15.52:0/3162808752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cd0100d60 msgr2=0x7f6cd01966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:37.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.170+0000 7f6cd5064640 1 --2- 172.21.15.52:0/3162808752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cd0100d60 0x7f6cd01966c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.170+0000 7f6cd5064640 1 -- 172.21.15.52:0/3162808752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cd0105130 msgr2=0x7f6cd019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:37.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.170+0000 7f6cd5064640 1 --2- 172.21.15.52:0/3162808752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cd0105130 0x7f6cd019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.170+0000 7f6cd5064640 1 -- 172.21.15.52:0/3162808752 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd019b120 con 0x7f6cd01003c0 2024-04-02T14:04:37.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.170+0000 7f6cd5064640 1 --2- 172.21.15.52:0/3162808752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cd01003c0 0x7f6cd0196180 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800eab0 tx=0x7f6cc800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:37.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.170+0000 7f6cce7fc640 1 -- 172.21.15.52:0/3162808752 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc800ce40 con 0x7f6cd01003c0 2024-04-02T14:04:37.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.170+0000 7f6cce7fc640 1 -- 172.21.15.52:0/3162808752 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cc8011950 con 0x7f6cd01003c0 2024-04-02T14:04:37.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.170+0000 7f6cd72c8640 1 -- 172.21.15.52:0/3162808752 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd019b410 con 0x7f6cd01003c0 2024-04-02T14:04:37.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.170+0000 7f6cce7fc640 1 -- 172.21.15.52:0/3162808752 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8010ae0 con 0x7f6cd01003c0 2024-04-02T14:04:37.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.170+0000 7f6cd72c8640 1 -- 172.21.15.52:0/3162808752 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd01a2ca0 con 0x7f6cd01003c0 2024-04-02T14:04:37.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.171+0000 7f6cd72c8640 1 -- 172.21.15.52:0/3162808752 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd0068d40 con 0x7f6cd01003c0 2024-04-02T14:04:37.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.172+0000 7f6cce7fc640 1 -- 172.21.15.52:0/3162808752 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6cc8011b00 con 0x7f6cd01003c0 2024-04-02T14:04:37.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.173+0000 7f6cce7fc640 1 --2- 172.21.15.52:0/3162808752 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6cb80a3da0 0x7f6cb80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:37.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.173+0000 7f6cd4863640 1 --2- 172.21.15.52:0/3162808752 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6cb80a3da0 0x7f6cb80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:37.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.173+0000 7f6cce7fc640 1 -- 172.21.15.52:0/3162808752 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f6cc80d1120 con 0x7f6cd01003c0 2024-04-02T14:04:37.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.173+0000 7f6cd4863640 1 --2- 172.21.15.52:0/3162808752 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6cb80a3da0 0x7f6cb80a6260 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0076340 tx=0x7f6cc400c6a0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:37.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.176+0000 7f6cce7fc640 1 -- 172.21.15.52:0/3162808752 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6cc8016030 con 0x7f6cd01003c0 2024-04-02T14:04:37.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.381+0000 7f6cd72c8640 1 -- 172.21.15.52:0/3162808752 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6cd0197480 con 0x7f6cd01003c0 2024-04-02T14:04:37.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.382+0000 7f6cce7fc640 1 -- 172.21.15.52:0/3162808752 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) ==== 104+0+11 (secure 0 0 0) 0x7f6cc809d4b0 con 0x7f6cd01003c0 2024-04-02T14:04:37.386 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-04-02T14:04:37.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.384+0000 7f6cd72c8640 1 -- 172.21.15.52:0/3162808752 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6cb80a3da0 msgr2=0x7f6cb80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:37.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.384+0000 7f6cd72c8640 1 --2- 172.21.15.52:0/3162808752 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6cb80a3da0 0x7f6cb80a6260 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0076340 tx=0x7f6cc400c6a0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.384+0000 7f6cd72c8640 1 -- 172.21.15.52:0/3162808752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cd01003c0 msgr2=0x7f6cd0196180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:37.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.384+0000 7f6cd72c8640 1 --2- 172.21.15.52:0/3162808752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cd01003c0 0x7f6cd0196180 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800eab0 tx=0x7f6cc800ef70 comp rx=0 tx=0).stop 2024-04-02T14:04:37.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.384+0000 7f6cd72c8640 1 -- 172.21.15.52:0/3162808752 shutdown_connections 2024-04-02T14:04:37.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.384+0000 7f6cd72c8640 1 --2- 172.21.15.52:0/3162808752 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f6cb80a3da0 0x7f6cb80a6260 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.384+0000 7f6cd72c8640 1 --2- 172.21.15.52:0/3162808752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cd0105130 0x7f6cd019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.384+0000 7f6cd72c8640 1 --2- 172.21.15.52:0/3162808752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cd0100d60 0x7f6cd01966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.384+0000 7f6cd72c8640 1 --2- 172.21.15.52:0/3162808752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cd01003c0 0x7f6cd0196180 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.384+0000 7f6cd72c8640 1 -- 172.21.15.52:0/3162808752 >> 172.21.15.52:0/3162808752 conn(0x7f6cd0075d40 msgr2=0x7f6cd00fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:37.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.384+0000 7f6cd72c8640 1 -- 172.21.15.52:0/3162808752 shutdown_connections 2024-04-02T14:04:37.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.384+0000 7f6cd72c8640 1 -- 172.21.15.52:0/3162808752 wait complete. 2024-04-02T14:04:37.398 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T14:04:37.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.500+0000 7f1e3b58d640 1 -- 172.21.15.52:0/3975022971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e34102d40 msgr2=0x7f1e34103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:37.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.500+0000 7f1e3b58d640 1 --2- 172.21.15.52:0/3975022971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e34102d40 0x7f1e34103120 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f1e30009900 tx=0x7f1e3002bcd0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.501+0000 7f1e3b58d640 1 -- 172.21.15.52:0/3975022971 shutdown_connections 2024-04-02T14:04:37.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.501+0000 7f1e3b58d640 1 --2- 172.21.15.52:0/3975022971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e3410c110 0x7f1e3410e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.501+0000 7f1e3b58d640 1 --2- 172.21.15.52:0/3975022971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e34103660 0x7f1e3410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.501+0000 7f1e3b58d640 1 --2- 172.21.15.52:0/3975022971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e34102d40 0x7f1e34103120 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.501+0000 7f1e3b58d640 1 -- 172.21.15.52:0/3975022971 >> 172.21.15.52:0/3975022971 conn(0x7f1e34076d80 msgr2=0x7f1e34077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:37.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.501+0000 7f1e3b58d640 1 -- 172.21.15.52:0/3975022971 shutdown_connections 2024-04-02T14:04:37.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.501+0000 7f1e3b58d640 1 -- 172.21.15.52:0/3975022971 wait complete. 2024-04-02T14:04:37.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.501+0000 7f1e3b58d640 1 Processor -- start 2024-04-02T14:04:37.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.501+0000 7f1e3b58d640 1 -- start start 2024-04-02T14:04:37.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.501+0000 7f1e3b58d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e34102d40 0x7f1e34100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:37.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.501+0000 7f1e3b58d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e34103660 0x7f1e34100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:37.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.501+0000 7f1e3b58d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e3410c110 0x7f1e34101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:37.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.501+0000 7f1e3b58d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1e34111f00 con 0x7f1e3410c110 2024-04-02T14:04:37.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.501+0000 7f1e3b58d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1e34111d80 con 0x7f1e34103660 2024-04-02T14:04:37.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.501+0000 7f1e3b58d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1e34112080 con 0x7f1e34102d40 2024-04-02T14:04:37.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.501+0000 7f1e39b2a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e3410c110 0x7f1e34101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:37.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.502+0000 7f1e39b2a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e3410c110 0x7f1e34101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51442/0 (socket says 172.21.15.52:51442) 2024-04-02T14:04:37.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.502+0000 7f1e39b2a640 1 -- 172.21.15.52:0/1482344927 learned_addr learned my addr 172.21.15.52:0/1482344927 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:04:37.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.502+0000 7f1e38b28640 1 --2- 172.21.15.52:0/1482344927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e34103660 0x7f1e34100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:37.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.502+0000 7f1e39329640 1 --2- 172.21.15.52:0/1482344927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e34102d40 0x7f1e34100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:37.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.502+0000 7f1e39b2a640 1 -- 172.21.15.52:0/1482344927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e34102d40 msgr2=0x7f1e34100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:37.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.502+0000 7f1e39b2a640 1 --2- 172.21.15.52:0/1482344927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e34102d40 0x7f1e34100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.502+0000 7f1e39b2a640 1 -- 172.21.15.52:0/1482344927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e34103660 msgr2=0x7f1e34100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:37.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.502+0000 7f1e39b2a640 1 --2- 172.21.15.52:0/1482344927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e34103660 0x7f1e34100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.502+0000 7f1e39b2a640 1 -- 172.21.15.52:0/1482344927 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e341a68c0 con 0x7f1e3410c110 2024-04-02T14:04:37.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.502+0000 7f1e38b28640 1 --2- 172.21.15.52:0/1482344927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e34103660 0x7f1e34100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:04:37.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.502+0000 7f1e39b2a640 1 --2- 172.21.15.52:0/1482344927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e3410c110 0x7f1e34101410 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c00b9e0 tx=0x7f1e2c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:37.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.502+0000 7f1e2a7fc640 1 -- 172.21.15.52:0/1482344927 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c019070 con 0x7f1e3410c110 2024-04-02T14:04:37.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.502+0000 7f1e2a7fc640 1 -- 172.21.15.52:0/1482344927 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e2c014440 con 0x7f1e3410c110 2024-04-02T14:04:37.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.502+0000 7f1e3b58d640 1 -- 172.21.15.52:0/1482344927 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e341a6bb0 con 0x7f1e3410c110 2024-04-02T14:04:37.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.502+0000 7f1e2a7fc640 1 -- 172.21.15.52:0/1482344927 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c0125e0 con 0x7f1e3410c110 2024-04-02T14:04:37.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.502+0000 7f1e3b58d640 1 -- 172.21.15.52:0/1482344927 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e341a7100 con 0x7f1e3410c110 2024-04-02T14:04:37.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.503+0000 7f1e3b58d640 1 -- 172.21.15.52:0/1482344927 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e3406a8f0 con 0x7f1e3410c110 2024-04-02T14:04:37.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.504+0000 7f1e2a7fc640 1 -- 172.21.15.52:0/1482344927 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1e2c01c070 con 0x7f1e3410c110 2024-04-02T14:04:37.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.505+0000 7f1e2a7fc640 1 --2- 172.21.15.52:0/1482344927 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1e200a3dd0 0x7f1e200a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:37.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.505+0000 7f1e39329640 1 --2- 172.21.15.52:0/1482344927 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1e200a3dd0 0x7f1e200a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:37.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.505+0000 7f1e2a7fc640 1 -- 172.21.15.52:0/1482344927 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f1e2c0d0620 con 0x7f1e3410c110 2024-04-02T14:04:37.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.506+0000 7f1e39329640 1 --2- 172.21.15.52:0/1482344927 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1e200a3dd0 0x7f1e200a6290 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1e3002efd0 tx=0x7f1e30000bc0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:37.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.509+0000 7f1e2a7fc640 1 -- 172.21.15.52:0/1482344927 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1e2c09ca40 con 0x7f1e3410c110 2024-04-02T14:04:37.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.694+0000 7f1e3b58d640 1 -- 172.21.15.52:0/1482344927 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1e3402d370 con 0x7f1e3410c110 2024-04-02T14:04:37.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.695+0000 7f1e2a7fc640 1 -- 172.21.15.52:0/1482344927 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) ==== 100+0+11 (secure 0 0 0) 0x7f1e2c09c340 con 0x7f1e3410c110 2024-04-02T14:04:37.699 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-04-02T14:04:37.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.697+0000 7f1e3b58d640 1 -- 172.21.15.52:0/1482344927 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1e200a3dd0 msgr2=0x7f1e200a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:37.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.697+0000 7f1e3b58d640 1 --2- 172.21.15.52:0/1482344927 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1e200a3dd0 0x7f1e200a6290 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1e3002efd0 tx=0x7f1e30000bc0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.697+0000 7f1e3b58d640 1 -- 172.21.15.52:0/1482344927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e3410c110 msgr2=0x7f1e34101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:37.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.697+0000 7f1e3b58d640 1 --2- 172.21.15.52:0/1482344927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e3410c110 0x7f1e34101410 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c00b9e0 tx=0x7f1e2c00bea0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.697+0000 7f1e3b58d640 1 -- 172.21.15.52:0/1482344927 shutdown_connections 2024-04-02T14:04:37.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.697+0000 7f1e3b58d640 1 --2- 172.21.15.52:0/1482344927 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f1e200a3dd0 0x7f1e200a6290 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.697+0000 7f1e3b58d640 1 --2- 172.21.15.52:0/1482344927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e3410c110 0x7f1e34101410 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.697+0000 7f1e3b58d640 1 --2- 172.21.15.52:0/1482344927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e34103660 0x7f1e34100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.697+0000 7f1e3b58d640 1 --2- 172.21.15.52:0/1482344927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e34102d40 0x7f1e34100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.697+0000 7f1e3b58d640 1 -- 172.21.15.52:0/1482344927 >> 172.21.15.52:0/1482344927 conn(0x7f1e34076d80 msgr2=0x7f1e340fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:37.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.697+0000 7f1e3b58d640 1 -- 172.21.15.52:0/1482344927 shutdown_connections 2024-04-02T14:04:37.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.697+0000 7f1e3b58d640 1 -- 172.21.15.52:0/1482344927 wait complete. 2024-04-02T14:04:37.710 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:37 smithi052 ceph-mon[32104]: pgmap v346: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:04:37.710 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:37 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1565504556' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:04:37.710 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:37 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1116436582' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:04:37.711 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:37 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3162808752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:04:37.712 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T14:04:37.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:37 smithi189 ceph-mon[36683]: pgmap v346: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:04:37.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:37 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1565504556' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:04:37.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:37 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1116436582' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:04:37.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:37 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3162808752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:04:37.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.814+0000 7fddf7eb1640 1 -- 172.21.15.52:0/509651459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fddf0107b50 msgr2=0x7fddf0107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:37.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.814+0000 7fddf7eb1640 1 --2- 172.21.15.52:0/509651459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fddf0107b50 0x7fddf0107f30 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fdde8009930 tx=0x7fdde802bc30 comp rx=0 tx=0).stop 2024-04-02T14:04:37.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.814+0000 7fddf7eb1640 1 -- 172.21.15.52:0/509651459 shutdown_connections 2024-04-02T14:04:37.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.814+0000 7fddf7eb1640 1 --2- 172.21.15.52:0/509651459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fddf00696c0 0x7fddf010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.814+0000 7fddf7eb1640 1 --2- 172.21.15.52:0/509651459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddf0068d00 0x7fddf0069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.814+0000 7fddf7eb1640 1 --2- 172.21.15.52:0/509651459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fddf0107b50 0x7fddf0107f30 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.814+0000 7fddf7eb1640 1 -- 172.21.15.52:0/509651459 >> 172.21.15.52:0/509651459 conn(0x7fddf0077b00 msgr2=0x7fddf0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:37.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.815+0000 7fddf7eb1640 1 -- 172.21.15.52:0/509651459 shutdown_connections 2024-04-02T14:04:37.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.815+0000 7fddf7eb1640 1 -- 172.21.15.52:0/509651459 wait complete. 2024-04-02T14:04:37.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.815+0000 7fddf7eb1640 1 Processor -- start 2024-04-02T14:04:37.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.815+0000 7fddf7eb1640 1 -- start start 2024-04-02T14:04:37.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.815+0000 7fddf7eb1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fddf0068d00 0x7fddf01983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:37.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.815+0000 7fddf7eb1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddf00696c0 0x7fddf01988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:37.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.815+0000 7fddf7eb1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fddf0107b50 0x7fddf019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:37.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.815+0000 7fddf7eb1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fddf0111ef0 con 0x7fddf0068d00 2024-04-02T14:04:37.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.815+0000 7fddf7eb1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fddf0111d70 con 0x7fddf00696c0 2024-04-02T14:04:37.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.815+0000 7fddf7eb1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fddf0112070 con 0x7fddf0107b50 2024-04-02T14:04:37.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.815+0000 7fddf5c4d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fddf0068d00 0x7fddf01983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:37.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.815+0000 7fddf5c4d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fddf0068d00 0x7fddf01983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51462/0 (socket says 172.21.15.52:51462) 2024-04-02T14:04:37.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.815+0000 7fddf5c4d640 1 -- 172.21.15.52:0/2280616005 learned_addr learned my addr 172.21.15.52:0/2280616005 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:04:37.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.815+0000 7fddf544c640 1 --2- 172.21.15.52:0/2280616005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddf00696c0 0x7fddf01988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:37.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.816+0000 7fddf644e640 1 --2- 172.21.15.52:0/2280616005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fddf0107b50 0x7fddf019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:37.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.816+0000 7fddf5c4d640 1 -- 172.21.15.52:0/2280616005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fddf0107b50 msgr2=0x7fddf019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:37.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.816+0000 7fddf5c4d640 1 --2- 172.21.15.52:0/2280616005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fddf0107b50 0x7fddf019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.816+0000 7fddf5c4d640 1 -- 172.21.15.52:0/2280616005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddf00696c0 msgr2=0x7fddf01988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:37.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.816+0000 7fddf5c4d640 1 --2- 172.21.15.52:0/2280616005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddf00696c0 0x7fddf01988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:37.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.816+0000 7fddf5c4d640 1 -- 172.21.15.52:0/2280616005 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf019d350 con 0x7fddf0068d00 2024-04-02T14:04:37.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.816+0000 7fddf644e640 1 --2- 172.21.15.52:0/2280616005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fddf0107b50 0x7fddf019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:04:37.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.816+0000 7fddf5c4d640 1 --2- 172.21.15.52:0/2280616005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fddf0068d00 0x7fddf01983b0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fdde8009900 tx=0x7fdde8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:37.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.816+0000 7fdde6ffd640 1 -- 172.21.15.52:0/2280616005 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde8041070 con 0x7fddf0068d00 2024-04-02T14:04:37.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.816+0000 7fdde6ffd640 1 -- 172.21.15.52:0/2280616005 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdde80054e0 con 0x7fddf0068d00 2024-04-02T14:04:37.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.816+0000 7fddf7eb1640 1 -- 172.21.15.52:0/2280616005 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf019d5e0 con 0x7fddf0068d00 2024-04-02T14:04:37.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.816+0000 7fdde6ffd640 1 -- 172.21.15.52:0/2280616005 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde8003c20 con 0x7fddf0068d00 2024-04-02T14:04:37.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.816+0000 7fddf7eb1640 1 -- 172.21.15.52:0/2280616005 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf01a4ed0 con 0x7fddf0068d00 2024-04-02T14:04:37.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.817+0000 7fddf7eb1640 1 -- 172.21.15.52:0/2280616005 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddf01996b0 con 0x7fddf0068d00 2024-04-02T14:04:37.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.818+0000 7fdde6ffd640 1 -- 172.21.15.52:0/2280616005 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdde8044070 con 0x7fddf0068d00 2024-04-02T14:04:37.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.819+0000 7fdde6ffd640 1 --2- 172.21.15.52:0/2280616005 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdddc0a3d20 0x7fdddc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:37.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.819+0000 7fddf544c640 1 --2- 172.21.15.52:0/2280616005 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdddc0a3d20 0x7fdddc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:37.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.819+0000 7fdde6ffd640 1 -- 172.21.15.52:0/2280616005 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7fdde80f2ac0 con 0x7fddf0068d00 2024-04-02T14:04:37.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.819+0000 7fddf544c640 1 --2- 172.21.15.52:0/2280616005 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdddc0a3d20 0x7fdddc0a61e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fddf00fdb60 tx=0x7fdde0005f90 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:37.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:37.822+0000 7fdde6ffd640 1 -- 172.21.15.52:0/2280616005 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fdde80beee0 con 0x7fddf0068d00 2024-04-02T14:04:37.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:37 smithi088 ceph-mon[36528]: pgmap v346: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T14:04:37.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:37 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1565504556' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:04:37.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:37 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1116436582' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T14:04:37.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:37 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3162808752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T14:04:38.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.008+0000 7fddf7eb1640 1 -- 172.21.15.52:0/2280616005 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fddf002d370 con 0x7fddf0068d00 2024-04-02T14:04:38.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.009+0000 7fdde6ffd640 1 -- 172.21.15.52:0/2280616005 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v103) ==== 103+0+11 (secure 0 0 0) 0x7fdde80be7e0 con 0x7fddf0068d00 2024-04-02T14:04:38.013 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-04-02T14:04:38.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.010+0000 7fddf7eb1640 1 -- 172.21.15.52:0/2280616005 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdddc0a3d20 msgr2=0x7fdddc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:38.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.010+0000 7fddf7eb1640 1 --2- 172.21.15.52:0/2280616005 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdddc0a3d20 0x7fdddc0a61e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fddf00fdb60 tx=0x7fdde0005f90 comp rx=0 tx=0).stop 2024-04-02T14:04:38.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.010+0000 7fddf7eb1640 1 -- 172.21.15.52:0/2280616005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fddf0068d00 msgr2=0x7fddf01983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:38.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.010+0000 7fddf7eb1640 1 --2- 172.21.15.52:0/2280616005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fddf0068d00 0x7fddf01983b0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fdde8009900 tx=0x7fdde8004270 comp rx=0 tx=0).stop 2024-04-02T14:04:38.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.010+0000 7fddf7eb1640 1 -- 172.21.15.52:0/2280616005 shutdown_connections 2024-04-02T14:04:38.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.011+0000 7fddf7eb1640 1 --2- 172.21.15.52:0/2280616005 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fdddc0a3d20 0x7fdddc0a61e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:38.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.011+0000 7fddf7eb1640 1 --2- 172.21.15.52:0/2280616005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fddf0107b50 0x7fddf019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:38.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.011+0000 7fddf7eb1640 1 --2- 172.21.15.52:0/2280616005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddf00696c0 0x7fddf01988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:38.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.011+0000 7fddf7eb1640 1 --2- 172.21.15.52:0/2280616005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fddf0068d00 0x7fddf01983b0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:38.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.011+0000 7fddf7eb1640 1 -- 172.21.15.52:0/2280616005 >> 172.21.15.52:0/2280616005 conn(0x7fddf0077b00 msgr2=0x7fddf00fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:38.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.011+0000 7fddf7eb1640 1 -- 172.21.15.52:0/2280616005 shutdown_connections 2024-04-02T14:04:38.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.011+0000 7fddf7eb1640 1 -- 172.21.15.52:0/2280616005 wait complete. 2024-04-02T14:04:38.025 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T14:04:38.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.125+0000 7fb061a77640 1 -- 172.21.15.52:0/3330887611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb05c102d70 msgr2=0x7fb05c103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:38.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.125+0000 7fb061a77640 1 --2- 172.21.15.52:0/3330887611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb05c102d70 0x7fb05c103150 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009900 tx=0x7fb04c02bcd0 comp rx=0 tx=0).stop 2024-04-02T14:04:38.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.125+0000 7fb061a77640 1 -- 172.21.15.52:0/3330887611 shutdown_connections 2024-04-02T14:04:38.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.125+0000 7fb061a77640 1 --2- 172.21.15.52:0/3330887611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb05c10c140 0x7fb05c10e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:38.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.125+0000 7fb061a77640 1 --2- 172.21.15.52:0/3330887611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb05c103690 0x7fb05c10bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:38.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.125+0000 7fb061a77640 1 --2- 172.21.15.52:0/3330887611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb05c102d70 0x7fb05c103150 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:38.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.125+0000 7fb061a77640 1 -- 172.21.15.52:0/3330887611 >> 172.21.15.52:0/3330887611 conn(0x7fb05c077b60 msgr2=0x7fb05c077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:38.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.125+0000 7fb061a77640 1 -- 172.21.15.52:0/3330887611 shutdown_connections 2024-04-02T14:04:38.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.125+0000 7fb061a77640 1 -- 172.21.15.52:0/3330887611 wait complete. 2024-04-02T14:04:38.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.126+0000 7fb061a77640 1 Processor -- start 2024-04-02T14:04:38.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.126+0000 7fb061a77640 1 -- start start 2024-04-02T14:04:38.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.126+0000 7fb061a77640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb05c102d70 0x7fb05c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:38.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.126+0000 7fb061a77640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb05c103690 0x7fb05c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:38.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.126+0000 7fb061a77640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb05c10c140 0x7fb05c19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:38.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.126+0000 7fb061a77640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c111f10 con 0x7fb05c102d70 2024-04-02T14:04:38.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.126+0000 7fb061a77640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c111d90 con 0x7fb05c103690 2024-04-02T14:04:38.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.126+0000 7fb061a77640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c112090 con 0x7fb05c10c140 2024-04-02T14:04:38.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.126+0000 7fb05b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb05c102d70 0x7fb05c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:38.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.126+0000 7fb05b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb05c102d70 0x7fb05c198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51474/0 (socket says 172.21.15.52:51474) 2024-04-02T14:04:38.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.126+0000 7fb05b7fe640 1 -- 172.21.15.52:0/892095523 learned_addr learned my addr 172.21.15.52:0/892095523 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:04:38.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.126+0000 7fb05affd640 1 --2- 172.21.15.52:0/892095523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb05c103690 0x7fb05c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:38.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.127+0000 7fb05bfff640 1 --2- 172.21.15.52:0/892095523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb05c10c140 0x7fb05c19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:38.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.127+0000 7fb05b7fe640 1 -- 172.21.15.52:0/892095523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb05c10c140 msgr2=0x7fb05c19cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:38.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.127+0000 7fb05b7fe640 1 --2- 172.21.15.52:0/892095523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb05c10c140 0x7fb05c19cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:38.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.127+0000 7fb05b7fe640 1 -- 172.21.15.52:0/892095523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb05c103690 msgr2=0x7fb05c198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:38.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.127+0000 7fb05b7fe640 1 --2- 172.21.15.52:0/892095523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb05c103690 0x7fb05c198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:38.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.127+0000 7fb05b7fe640 1 -- 172.21.15.52:0/892095523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05c19d2d0 con 0x7fb05c102d70 2024-04-02T14:04:38.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.127+0000 7fb05b7fe640 1 --2- 172.21.15.52:0/892095523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb05c102d70 0x7fb05c198330 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fb04c02efd0 tx=0x7fb04c02ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:38.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.127+0000 7fb058ff9640 1 -- 172.21.15.52:0/892095523 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c041070 con 0x7fb05c102d70 2024-04-02T14:04:38.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.127+0000 7fb061a77640 1 -- 172.21.15.52:0/892095523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb05c19d560 con 0x7fb05c102d70 2024-04-02T14:04:38.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.127+0000 7fb058ff9640 1 -- 172.21.15.52:0/892095523 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb04c0054e0 con 0x7fb05c102d70 2024-04-02T14:04:38.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.127+0000 7fb058ff9640 1 -- 172.21.15.52:0/892095523 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c034440 con 0x7fb05c102d70 2024-04-02T14:04:38.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.127+0000 7fb061a77640 1 -- 172.21.15.52:0/892095523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb05c1a4e50 con 0x7fb05c102d70 2024-04-02T14:04:38.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.128+0000 7fb061a77640 1 -- 172.21.15.52:0/892095523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb05c068d40 con 0x7fb05c102d70 2024-04-02T14:04:38.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.129+0000 7fb058ff9640 1 -- 172.21.15.52:0/892095523 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb04c044070 con 0x7fb05c102d70 2024-04-02T14:04:38.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.130+0000 7fb058ff9640 1 --2- 172.21.15.52:0/892095523 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb0440a3d20 0x7fb0440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:38.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.130+0000 7fb05affd640 1 --2- 172.21.15.52:0/892095523 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb0440a3d20 0x7fb0440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:38.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.130+0000 7fb058ff9640 1 -- 172.21.15.52:0/892095523 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7fb04c0f2890 con 0x7fb05c102d70 2024-04-02T14:04:38.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.130+0000 7fb05affd640 1 --2- 172.21.15.52:0/892095523 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb0440a3d20 0x7fb0440a61e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb05c10dca0 tx=0x7fb05000a2d0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:38.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.134+0000 7fb058ff9640 1 -- 172.21.15.52:0/892095523 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb04c0becb0 con 0x7fb05c102d70 2024-04-02T14:04:38.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.330+0000 7fb061a77640 1 -- 172.21.15.52:0/892095523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb05c199630 con 0x7fb05c102d70 2024-04-02T14:04:38.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.331+0000 7fb058ff9640 1 -- 172.21.15.52:0/892095523 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v103) ==== 81+0+940 (secure 0 0 0) 0x7fb05c199630 con 0x7fb05c102d70 2024-04-02T14:04:38.335 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/713755826 2024-04-03T13:53:29.889159+0000 2024-04-02T14:04:38.335 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/1937276046 2024-04-03T13:53:29.889159+0000 2024-04-02T14:04:38.335 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/3836085837 2024-04-03T13:53:29.889159+0000 2024-04-02T14:04:38.336 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2957262850 2024-04-03T13:53:29.889159+0000 2024-04-02T14:04:38.336 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2475099387 2024-04-03T13:43:54.653361+0000 2024-04-02T14:04:38.336 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/2368921152 2024-04-03T13:43:27.309404+0000 2024-04-02T14:04:38.336 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/673843171 2024-04-03T13:43:54.653361+0000 2024-04-02T14:04:38.336 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/2368921152 2024-04-03T13:43:27.309404+0000 2024-04-02T14:04:38.336 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2609300748 2024-04-03T13:43:27.309404+0000 2024-04-02T14:04:38.336 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/3836085837 2024-04-03T13:53:29.889159+0000 2024-04-02T14:04:38.336 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2985814329 2024-04-03T13:53:29.889159+0000 2024-04-02T14:04:38.337 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/85571395 2024-04-03T13:43:27.309404+0000 2024-04-02T14:04:38.337 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/293852652 2024-04-03T13:43:54.653361+0000 2024-04-02T14:04:38.337 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/1829428070 2024-04-03T13:43:27.309404+0000 2024-04-02T14:04:38.337 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/673843171 2024-04-03T13:43:54.653361+0000 2024-04-02T14:04:38.337 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/3257395293 2024-04-03T13:43:54.653361+0000 2024-04-02T14:04:38.338 INFO:teuthology.orchestra.run.smithi052.stderr:listed 16 entries 2024-04-02T14:04:38.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.333+0000 7fb061a77640 1 -- 172.21.15.52:0/892095523 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb0440a3d20 msgr2=0x7fb0440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:38.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.333+0000 7fb061a77640 1 --2- 172.21.15.52:0/892095523 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb0440a3d20 0x7fb0440a61e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb05c10dca0 tx=0x7fb05000a2d0 comp rx=0 tx=0).stop 2024-04-02T14:04:38.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.333+0000 7fb061a77640 1 -- 172.21.15.52:0/892095523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb05c102d70 msgr2=0x7fb05c198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:38.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.333+0000 7fb061a77640 1 --2- 172.21.15.52:0/892095523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb05c102d70 0x7fb05c198330 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fb04c02efd0 tx=0x7fb04c02ce50 comp rx=0 tx=0).stop 2024-04-02T14:04:38.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.333+0000 7fb061a77640 1 -- 172.21.15.52:0/892095523 shutdown_connections 2024-04-02T14:04:38.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.333+0000 7fb061a77640 1 --2- 172.21.15.52:0/892095523 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fb0440a3d20 0x7fb0440a61e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:38.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.333+0000 7fb061a77640 1 --2- 172.21.15.52:0/892095523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb05c10c140 0x7fb05c19cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:38.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.333+0000 7fb061a77640 1 --2- 172.21.15.52:0/892095523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb05c103690 0x7fb05c198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:38.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.333+0000 7fb061a77640 1 --2- 172.21.15.52:0/892095523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb05c102d70 0x7fb05c198330 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:38.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.333+0000 7fb061a77640 1 -- 172.21.15.52:0/892095523 >> 172.21.15.52:0/892095523 conn(0x7fb05c077b60 msgr2=0x7fb05c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:38.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.333+0000 7fb061a77640 1 -- 172.21.15.52:0/892095523 shutdown_connections 2024-04-02T14:04:38.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.333+0000 7fb061a77640 1 -- 172.21.15.52:0/892095523 wait complete. 2024-04-02T14:04:38.348 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-02T14:04:38.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:04:38.348 DEBUG:teuthology.orchestra.run.smithi052:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-02T14:04:38.455 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-04-02T14:04:38.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:04:38.455 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-02T14:04:38.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:04:38.470 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-04-02T14:04:38.526 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-02T14:04:38.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:04:38.526 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T14:04:38.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T14:04:38.593 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link delete ceph-brx 2024-04-02T14:04:38.602 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:38 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1482344927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:04:38.602 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:38 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2280616005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T14:04:38.602 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:38 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/892095523' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T14:04:38.663 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-02T14:04:38.674 INFO:tasks.cephadm:Teardown begin 2024-04-02T14:04:38.674 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-02T14:04:38.675 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:04:38.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.776+0000 7fe2646c5640 1 -- 172.21.15.52:0/3005137454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe25c107ab0 msgr2=0x7fe25c107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:38.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.776+0000 7fe2646c5640 1 --2- 172.21.15.52:0/3005137454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe25c107ab0 0x7fe25c107e90 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fe254009930 tx=0x7fe25402bc30 comp rx=0 tx=0).stop 2024-04-02T14:04:38.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.776+0000 7fe2646c5640 1 -- 172.21.15.52:0/3005137454 shutdown_connections 2024-04-02T14:04:38.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.776+0000 7fe2646c5640 1 --2- 172.21.15.52:0/3005137454 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe25c102460 0x7fe25c10e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:38.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.776+0000 7fe2646c5640 1 --2- 172.21.15.52:0/3005137454 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe25c101aa0 0x7fe25c101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:38.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.776+0000 7fe2646c5640 1 --2- 172.21.15.52:0/3005137454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe25c107ab0 0x7fe25c107e90 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:38.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.776+0000 7fe2646c5640 1 -- 172.21.15.52:0/3005137454 >> 172.21.15.52:0/3005137454 conn(0x7fe25c0fd590 msgr2=0x7fe25c0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:38.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.776+0000 7fe2646c5640 1 -- 172.21.15.52:0/3005137454 shutdown_connections 2024-04-02T14:04:38.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.776+0000 7fe2646c5640 1 -- 172.21.15.52:0/3005137454 wait complete. 2024-04-02T14:04:38.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.776+0000 7fe2646c5640 1 Processor -- start 2024-04-02T14:04:38.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.776+0000 7fe2646c5640 1 -- start start 2024-04-02T14:04:38.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.776+0000 7fe2646c5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe25c101aa0 0x7fe25c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:38.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.776+0000 7fe2646c5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe25c102460 0x7fe25c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:38.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.776+0000 7fe2646c5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe25c107ab0 0x7fe25c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:38.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.776+0000 7fe2646c5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c111e90 con 0x7fe25c102460 2024-04-02T14:04:38.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.776+0000 7fe2646c5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c111d10 con 0x7fe25c101aa0 2024-04-02T14:04:38.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.776+0000 7fe2646c5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c112010 con 0x7fe25c107ab0 2024-04-02T14:04:38.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.777+0000 7fe261c60640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe25c102460 0x7fe25c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:38.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.777+0000 7fe261c60640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe25c102460 0x7fe25c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51492/0 (socket says 172.21.15.52:51492) 2024-04-02T14:04:38.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.777+0000 7fe261c60640 1 -- 172.21.15.52:0/3562291405 learned_addr learned my addr 172.21.15.52:0/3562291405 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:04:38.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.777+0000 7fe262461640 1 --2- 172.21.15.52:0/3562291405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe25c101aa0 0x7fe25c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:38.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.777+0000 7fe262c62640 1 --2- 172.21.15.52:0/3562291405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe25c107ab0 0x7fe25c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:38.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.777+0000 7fe261c60640 1 -- 172.21.15.52:0/3562291405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe25c107ab0 msgr2=0x7fe25c19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.777+0000 7fe261c60640 1 --2- 172.21.15.52:0/3562291405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe25c107ab0 0x7fe25c19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.777+0000 7fe261c60640 1 -- 172.21.15.52:0/3562291405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe25c101aa0 msgr2=0x7fe25c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.777+0000 7fe261c60640 1 --2- 172.21.15.52:0/3562291405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe25c101aa0 0x7fe25c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.777+0000 7fe261c60640 1 -- 172.21.15.52:0/3562291405 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe25c19d3c0 con 0x7fe25c102460 2024-04-02T14:04:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.777+0000 7fe262c62640 1 --2- 172.21.15.52:0/3562291405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe25c107ab0 0x7fe25c19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T14:04:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.777+0000 7fe261c60640 1 --2- 172.21.15.52:0/3562291405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe25c102460 0x7fe25c198910 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00d950 tx=0x7fe24c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.777+0000 7fe2537fe640 1 -- 172.21.15.52:0/3562291405 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c017070 con 0x7fe25c102460 2024-04-02T14:04:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.777+0000 7fe2646c5640 1 -- 172.21.15.52:0/3562291405 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe25c19d6b0 con 0x7fe25c102460 2024-04-02T14:04:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.777+0000 7fe2537fe640 1 -- 172.21.15.52:0/3562291405 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe24c009c30 con 0x7fe25c102460 2024-04-02T14:04:38.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.777+0000 7fe2537fe640 1 -- 172.21.15.52:0/3562291405 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c00f6b0 con 0x7fe25c102460 2024-04-02T14:04:38.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.777+0000 7fe2646c5640 1 -- 172.21.15.52:0/3562291405 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe25c1a4f40 con 0x7fe25c102460 2024-04-02T14:04:38.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.778+0000 7fe2646c5640 1 -- 172.21.15.52:0/3562291405 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe25c0688f0 con 0x7fe25c102460 2024-04-02T14:04:38.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.779+0000 7fe2537fe640 1 -- 172.21.15.52:0/3562291405 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe24c0109d0 con 0x7fe25c102460 2024-04-02T14:04:38.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.780+0000 7fe2537fe640 1 --2- 172.21.15.52:0/3562291405 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe2480a3d20 0x7fe2480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:38.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.780+0000 7fe262461640 1 --2- 172.21.15.52:0/3562291405 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe2480a3d20 0x7fe2480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:38.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.780+0000 7fe2537fe640 1 -- 172.21.15.52:0/3562291405 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7fe24c0d07c0 con 0x7fe25c102460 2024-04-02T14:04:38.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.780+0000 7fe262461640 1 --2- 172.21.15.52:0/3562291405 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe2480a3d20 0x7fe2480a61e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe254005ab0 tx=0x7fe254005fd0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:38.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:38.783+0000 7fe2537fe640 1 -- 172.21.15.52:0/3562291405 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe24c009de0 con 0x7fe25c102460 2024-04-02T14:04:38.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:38 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1482344927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:04:38.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:38 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2280616005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T14:04:38.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:38 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/892095523' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T14:04:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:38 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1482344927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T14:04:38.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:38 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2280616005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T14:04:38.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:38 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/892095523' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T14:04:39.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.009+0000 7fe2646c5640 1 -- 172.21.15.52:0/3562291405 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe25c02d370 con 0x7fe25c102460 2024-04-02T14:04:39.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.010+0000 7fe2537fe640 1 -- 172.21.15.52:0/3562291405 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11923 (secure 0 0 0) 0x7fe24c09cad0 con 0x7fe25c102460 2024-04-02T14:04:39.014 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T14:04:39.014 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:57:45.240902+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24467,24457,24490,24475,24503]},"id":1}]} 2024-04-02T14:04:39.015 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-04-02T14:04:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.012+0000 7fe2646c5640 1 -- 172.21.15.52:0/3562291405 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe2480a3d20 msgr2=0x7fe2480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.012+0000 7fe2646c5640 1 --2- 172.21.15.52:0/3562291405 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe2480a3d20 0x7fe2480a61e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe254005ab0 tx=0x7fe254005fd0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.012+0000 7fe2646c5640 1 -- 172.21.15.52:0/3562291405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe25c102460 msgr2=0x7fe25c198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.012+0000 7fe2646c5640 1 --2- 172.21.15.52:0/3562291405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe25c102460 0x7fe25c198910 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00d950 tx=0x7fe24c00de10 comp rx=0 tx=0).stop 2024-04-02T14:04:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.012+0000 7fe2646c5640 1 -- 172.21.15.52:0/3562291405 shutdown_connections 2024-04-02T14:04:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.012+0000 7fe2646c5640 1 --2- 172.21.15.52:0/3562291405 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7fe2480a3d20 0x7fe2480a61e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.012+0000 7fe2646c5640 1 --2- 172.21.15.52:0/3562291405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe25c107ab0 0x7fe25c19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.012+0000 7fe2646c5640 1 --2- 172.21.15.52:0/3562291405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe25c102460 0x7fe25c198910 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.012+0000 7fe2646c5640 1 --2- 172.21.15.52:0/3562291405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe25c101aa0 0x7fe25c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.012+0000 7fe2646c5640 1 -- 172.21.15.52:0/3562291405 >> 172.21.15.52:0/3562291405 conn(0x7fe25c0fd590 msgr2=0x7fe25c0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.012+0000 7fe2646c5640 1 -- 172.21.15.52:0/3562291405 shutdown_connections 2024-04-02T14:04:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.012+0000 7fe2646c5640 1 -- 172.21.15.52:0/3562291405 wait complete. 2024-04-02T14:04:39.027 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T14:04:39.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.127+0000 7f11a4663640 1 -- 172.21.15.52:0/4286435537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f119c100a80 msgr2=0x7f119c100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:39.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.127+0000 7f11a4663640 1 --2- 172.21.15.52:0/4286435537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f119c100a80 0x7f119c100f00 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f118c009930 tx=0x7f118c02bc30 comp rx=0 tx=0).stop 2024-04-02T14:04:39.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.127+0000 7f11a4663640 1 -- 172.21.15.52:0/4286435537 shutdown_connections 2024-04-02T14:04:39.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.127+0000 7f11a4663640 1 --2- 172.21.15.52:0/4286435537 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f119c101440 0x7f119c10d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.127+0000 7f11a4663640 1 --2- 172.21.15.52:0/4286435537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f119c100a80 0x7f119c100f00 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.127+0000 7f11a4663640 1 --2- 172.21.15.52:0/4286435537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f119c106a40 0x7f119c106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.127+0000 7f11a4663640 1 -- 172.21.15.52:0/4286435537 >> 172.21.15.52:0/4286435537 conn(0x7f119c077b60 msgr2=0x7f119c077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:39.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.127+0000 7f11a4663640 1 -- 172.21.15.52:0/4286435537 shutdown_connections 2024-04-02T14:04:39.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.127+0000 7f11a4663640 1 -- 172.21.15.52:0/4286435537 wait complete. 2024-04-02T14:04:39.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.128+0000 7f11a4663640 1 Processor -- start 2024-04-02T14:04:39.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.128+0000 7f11a4663640 1 -- start start 2024-04-02T14:04:39.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.128+0000 7f11a4663640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f119c100a80 0x7f119c1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:39.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.128+0000 7f11a4663640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f119c101440 0x7f119c199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:39.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.128+0000 7f11a4663640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f119c106a40 0x7f119c1a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:39.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.128+0000 7f11a4663640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f119c110e20 con 0x7f119c101440 2024-04-02T14:04:39.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.128+0000 7f11a4663640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f119c110ca0 con 0x7f119c106a40 2024-04-02T14:04:39.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.128+0000 7f11a4663640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f119c110fa0 con 0x7f119c100a80 2024-04-02T14:04:39.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.128+0000 7f11a1bfe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f119c101440 0x7f119c199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:39.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.128+0000 7f11a1bfe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f119c101440 0x7f119c199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51520/0 (socket says 172.21.15.52:51520) 2024-04-02T14:04:39.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.128+0000 7f11a1bfe640 1 -- 172.21.15.52:0/2057896414 learned_addr learned my addr 172.21.15.52:0/2057896414 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:04:39.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.128+0000 7f11a2c00640 1 --2- 172.21.15.52:0/2057896414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f119c106a40 0x7f119c1a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:39.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.128+0000 7f11a23ff640 1 --2- 172.21.15.52:0/2057896414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f119c100a80 0x7f119c1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:39.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.128+0000 7f11a1bfe640 1 -- 172.21.15.52:0/2057896414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f119c100a80 msgr2=0x7f119c1992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:39.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.128+0000 7f11a1bfe640 1 --2- 172.21.15.52:0/2057896414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f119c100a80 0x7f119c1992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.128+0000 7f11a1bfe640 1 -- 172.21.15.52:0/2057896414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f119c106a40 msgr2=0x7f119c1a4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:39.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.128+0000 7f11a1bfe640 1 --2- 172.21.15.52:0/2057896414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f119c106a40 0x7f119c1a4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.128+0000 7f11a1bfe640 1 -- 172.21.15.52:0/2057896414 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f119c1a48a0 con 0x7f119c101440 2024-04-02T14:04:39.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.128+0000 7f11a1bfe640 1 --2- 172.21.15.52:0/2057896414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f119c101440 0x7f119c199820 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f118c009230 tx=0x7f118c02cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:39.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.129+0000 7f11937fe640 1 -- 172.21.15.52:0/2057896414 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c041070 con 0x7f119c101440 2024-04-02T14:04:39.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.129+0000 7f11937fe640 1 -- 172.21.15.52:0/2057896414 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f118c004450 con 0x7f119c101440 2024-04-02T14:04:39.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.129+0000 7f11a4663640 1 -- 172.21.15.52:0/2057896414 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f119c1a4a70 con 0x7f119c101440 2024-04-02T14:04:39.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.129+0000 7f11937fe640 1 -- 172.21.15.52:0/2057896414 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c034440 con 0x7f119c101440 2024-04-02T14:04:39.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.129+0000 7f11a4663640 1 -- 172.21.15.52:0/2057896414 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f119c1a4ed0 con 0x7f119c101440 2024-04-02T14:04:39.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.130+0000 7f11a4663640 1 -- 172.21.15.52:0/2057896414 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f119c068d40 con 0x7f119c101440 2024-04-02T14:04:39.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.131+0000 7f11937fe640 1 -- 172.21.15.52:0/2057896414 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f118c044070 con 0x7f119c101440 2024-04-02T14:04:39.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.131+0000 7f11937fe640 1 --2- 172.21.15.52:0/2057896414 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f11880a3d20 0x7f11880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:39.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.132+0000 7f11a23ff640 1 --2- 172.21.15.52:0/2057896414 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f11880a3d20 0x7f11880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:39.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.132+0000 7f11937fe640 1 -- 172.21.15.52:0/2057896414 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f118c0f24d0 con 0x7f119c101440 2024-04-02T14:04:39.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.132+0000 7f11a23ff640 1 --2- 172.21.15.52:0/2057896414 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f11880a3d20 0x7f11880a61e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1198005e00 tx=0x7f1198005c50 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:39.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.136+0000 7f11937fe640 1 -- 172.21.15.52:0/2057896414 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f118c0be8f0 con 0x7f119c101440 2024-04-02T14:04:39.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.361+0000 7f11a4663640 1 -- 172.21.15.52:0/2057896414 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f119c02d370 con 0x7f119c101440 2024-04-02T14:04:39.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.361+0000 7f11937fe640 1 -- 172.21.15.52:0/2057896414 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11923 (secure 0 0 0) 0x7f118c0be1f0 con 0x7f119c101440 2024-04-02T14:04:39.365 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T14:04:39.366 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1683593044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1683593044},{"type":"v1","addr":"172.21.15.52:6835","nonce":1683593044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/361596788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":361596788},{"type":"v1","addr":"172.21.15.52:6837","nonce":361596788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:51:00.603246+0000","modified":"2024-04-02T13:57:45.240902+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24503,"mds_1":24490,"mds_2":24475,"mds_3":24467,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2213112521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2213112521},{"type":"v1","addr":"172.21.15.52:6839","nonce":2213112521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/2689882080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2689882080},{"type":"v1","addr":"172.21.15.52:6841","nonce":2689882080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6833/4248342942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":4248342942},{"type":"v1","addr":"172.21.15.88:6833","nonce":4248342942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/3157031038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3157031038},{"type":"v1","addr":"172.21.15.189:6833","nonce":3157031038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24437":{"gid":24437,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/5019107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":5019107},{"type":"v1","addr":"172.21.15.88:6835","nonce":5019107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.189:6835/1647960400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":1647960400},{"type":"v1","addr":"172.21.15.189:6835","nonce":1647960400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1091039373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1091039373},{"type":"v1","addr":"172.21.15.88:6837","nonce":1091039373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.189:6837/2295541152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2295541152},{"type":"v1","addr":"172.21.15.189:6837","nonce":2295541152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/729001189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":729001189},{"type":"v1","addr":"172.21.15.189:6839","nonce":729001189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/2513527493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":2513527493},{"type":"v1","addr":"172.21.15.88:6839","nonce":2513527493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24467,24457,24490,24475,24503]},"id":1}]} 2024-04-02T14:04:39.366 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-04-02T14:04:39.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.363+0000 7f11a4663640 1 -- 172.21.15.52:0/2057896414 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f11880a3d20 msgr2=0x7f11880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:39.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.363+0000 7f11a4663640 1 --2- 172.21.15.52:0/2057896414 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f11880a3d20 0x7f11880a61e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1198005e00 tx=0x7f1198005c50 comp rx=0 tx=0).stop 2024-04-02T14:04:39.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.363+0000 7f11a4663640 1 -- 172.21.15.52:0/2057896414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f119c101440 msgr2=0x7f119c199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:39.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.363+0000 7f11a4663640 1 --2- 172.21.15.52:0/2057896414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f119c101440 0x7f119c199820 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f118c009230 tx=0x7f118c02cba0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.363+0000 7f11a4663640 1 -- 172.21.15.52:0/2057896414 shutdown_connections 2024-04-02T14:04:39.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.363+0000 7f11a4663640 1 --2- 172.21.15.52:0/2057896414 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f11880a3d20 0x7f11880a61e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.363+0000 7f11a4663640 1 --2- 172.21.15.52:0/2057896414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f119c106a40 0x7f119c1a4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.363+0000 7f11a4663640 1 --2- 172.21.15.52:0/2057896414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f119c101440 0x7f119c199820 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.363+0000 7f11a4663640 1 --2- 172.21.15.52:0/2057896414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f119c100a80 0x7f119c1992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.363+0000 7f11a4663640 1 -- 172.21.15.52:0/2057896414 >> 172.21.15.52:0/2057896414 conn(0x7f119c077b60 msgr2=0x7f119c103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:39.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.363+0000 7f11a4663640 1 -- 172.21.15.52:0/2057896414 shutdown_connections 2024-04-02T14:04:39.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.363+0000 7f11a4663640 1 -- 172.21.15.52:0/2057896414 wait complete. 2024-04-02T14:04:39.379 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T14:04:39.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.478+0000 7feaff590640 1 -- 172.21.15.52:0/2439459125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feaf8102450 msgr2=0x7feaf810e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:39.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.478+0000 7feaff590640 1 --2- 172.21.15.52:0/2439459125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feaf8102450 0x7feaf810e960 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009cd0 tx=0x7feaf002c0f0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.478+0000 7feaff590640 1 -- 172.21.15.52:0/2439459125 shutdown_connections 2024-04-02T14:04:39.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.478+0000 7feaff590640 1 --2- 172.21.15.52:0/2439459125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feaf8102450 0x7feaf810e960 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.478+0000 7feaff590640 1 --2- 172.21.15.52:0/2439459125 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaf8101a90 0x7feaf8101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.478+0000 7feaff590640 1 --2- 172.21.15.52:0/2439459125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feaf8107aa0 0x7feaf8107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.478+0000 7feaff590640 1 -- 172.21.15.52:0/2439459125 >> 172.21.15.52:0/2439459125 conn(0x7feaf80fd5a0 msgr2=0x7feaf80ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:39.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.478+0000 7feaff590640 1 -- 172.21.15.52:0/2439459125 shutdown_connections 2024-04-02T14:04:39.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.478+0000 7feaff590640 1 -- 172.21.15.52:0/2439459125 wait complete. 2024-04-02T14:04:39.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.479+0000 7feaff590640 1 Processor -- start 2024-04-02T14:04:39.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.479+0000 7feaff590640 1 -- start start 2024-04-02T14:04:39.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.479+0000 7feaff590640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaf8101a90 0x7feaf81982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:39.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.479+0000 7feaff590640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feaf8102450 0x7feaf8198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:39.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.479+0000 7feaff590640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feaf8107aa0 0x7feaf819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:39.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.479+0000 7feaff590640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feaf8111e80 con 0x7feaf8102450 2024-04-02T14:04:39.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.479+0000 7feaff590640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feaf8111d00 con 0x7feaf8101a90 2024-04-02T14:04:39.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.479+0000 7feaff590640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feaf8112000 con 0x7feaf8107aa0 2024-04-02T14:04:39.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.479+0000 7feafcb2b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feaf8102450 0x7feaf8198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:39.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.479+0000 7feafcb2b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feaf8102450 0x7feaf8198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51534/0 (socket says 172.21.15.52:51534) 2024-04-02T14:04:39.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.479+0000 7feafcb2b640 1 -- 172.21.15.52:0/1751445348 learned_addr learned my addr 172.21.15.52:0/1751445348 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:04:39.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.479+0000 7feafd32c640 1 --2- 172.21.15.52:0/1751445348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaf8101a90 0x7feaf81982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:39.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.479+0000 7feafdb2d640 1 --2- 172.21.15.52:0/1751445348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feaf8107aa0 0x7feaf819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:39.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.480+0000 7feafcb2b640 1 -- 172.21.15.52:0/1751445348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feaf8107aa0 msgr2=0x7feaf819cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:39.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.480+0000 7feafcb2b640 1 --2- 172.21.15.52:0/1751445348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feaf8107aa0 0x7feaf819cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.480+0000 7feafcb2b640 1 -- 172.21.15.52:0/1751445348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaf8101a90 msgr2=0x7feaf81982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:39.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.480+0000 7feafcb2b640 1 --2- 172.21.15.52:0/1751445348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaf8101a90 0x7feaf81982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.480+0000 7feafcb2b640 1 -- 172.21.15.52:0/1751445348 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf819d2d0 con 0x7feaf8102450 2024-04-02T14:04:39.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.480+0000 7feafcb2b640 1 --2- 172.21.15.52:0/1751445348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feaf8102450 0x7feaf8198820 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7feaf400e9f0 tx=0x7feaf400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:39.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.480+0000 7feaee7fc640 1 -- 172.21.15.52:0/1751445348 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf400ccc0 con 0x7feaf8102450 2024-04-02T14:04:39.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.480+0000 7feaee7fc640 1 -- 172.21.15.52:0/1751445348 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf4010910 con 0x7feaf8102450 2024-04-02T14:04:39.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.480+0000 7feaff590640 1 -- 172.21.15.52:0/1751445348 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf819d5c0 con 0x7feaf8102450 2024-04-02T14:04:39.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.480+0000 7feaee7fc640 1 -- 172.21.15.52:0/1751445348 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf40188a0 con 0x7feaf8102450 2024-04-02T14:04:39.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.480+0000 7feaff590640 1 -- 172.21.15.52:0/1751445348 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf81a4e50 con 0x7feaf8102450 2024-04-02T14:04:39.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.481+0000 7feaff590640 1 -- 172.21.15.52:0/1751445348 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf8068d40 con 0x7feaf8102450 2024-04-02T14:04:39.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.482+0000 7feaee7fc640 1 -- 172.21.15.52:0/1751445348 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feaf4010ac0 con 0x7feaf8102450 2024-04-02T14:04:39.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.482+0000 7feaee7fc640 1 --2- 172.21.15.52:0/1751445348 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7feae40a3da0 0x7feae40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:39.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.483+0000 7feafd32c640 1 --2- 172.21.15.52:0/1751445348 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7feae40a3da0 0x7feae40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:39.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.483+0000 7feaee7fc640 1 -- 172.21.15.52:0/1751445348 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7feaf40d1510 con 0x7feaf8102450 2024-04-02T14:04:39.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.483+0000 7feafd32c640 1 --2- 172.21.15.52:0/1751445348 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7feae40a3da0 0x7feae40a6260 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7feae800ab60 tx=0x7feae800a2d0 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:39.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.486+0000 7feaee7fc640 1 -- 172.21.15.52:0/1751445348 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7feaf409d7c0 con 0x7feaf8102450 2024-04-02T14:04:39.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.677+0000 7feaff590640 1 -- 172.21.15.52:0/1751445348 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feaf802d370 con 0x7feaf8102450 2024-04-02T14:04:39.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.679+0000 7feaee7fc640 1 -- 172.21.15.52:0/1751445348 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+25112 (secure 0 0 0) 0x7feaf409d0c0 con 0x7feaf8102450 2024-04-02T14:04:39.683 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-04-02T14:04:39.683 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":103,"fsid":"bfa00e6e-f0f6-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:42:59.639077+0000","modified":"2024-04-02T14:04:15.482659+0000","last_up_change":"2024-04-02T13:50:10.804387+0000","last_in_change":"2024-04-02T13:49:50.070800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:46:20.746053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:50:58.778690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:50:59.561527+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:51:03.343487+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4fe9c5ef-4edb-4b75-8e4f-5615bae725c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6803","nonce":1314806472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6805","nonce":1314806472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6809","nonce":1314806472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1314806472},{"type":"v1","addr":"172.21.15.52:6807","nonce":1314806472}]},"public_addr":"172.21.15.52:6803/1314806472","cluster_addr":"172.21.15.52:6805/1314806472","heartbeat_back_addr":"172.21.15.52:6809/1314806472","heartbeat_front_addr":"172.21.15.52:6807/1314806472","state":["exists","up"]},{"osd":1,"uuid":"2608f75a-4c4b-4a33-9e5b-3fd14d5ed4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6801","nonce":288986438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6803","nonce":288986438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6807","nonce":288986438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":288986438},{"type":"v1","addr":"172.21.15.88:6805","nonce":288986438}]},"public_addr":"172.21.15.88:6801/288986438","cluster_addr":"172.21.15.88:6803/288986438","heartbeat_back_addr":"172.21.15.88:6807/288986438","heartbeat_front_addr":"172.21.15.88:6805/288986438","state":["exists","up"]},{"osd":2,"uuid":"142d5d34-737f-44bf-a17c-c2ad6d0be723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6801","nonce":2416423625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6803","nonce":2416423625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6807","nonce":2416423625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2416423625},{"type":"v1","addr":"172.21.15.189:6805","nonce":2416423625}]},"public_addr":"172.21.15.189:6801/2416423625","cluster_addr":"172.21.15.189:6803/2416423625","heartbeat_back_addr":"172.21.15.189:6807/2416423625","heartbeat_front_addr":"172.21.15.189:6805/2416423625","state":["exists","up"]},{"osd":3,"uuid":"0960a00e-5147-4ff9-9ac2-d16b6ceb8ec4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6811","nonce":1799783980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6813","nonce":1799783980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6817","nonce":1799783980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":1799783980},{"type":"v1","addr":"172.21.15.52:6815","nonce":1799783980}]},"public_addr":"172.21.15.52:6811/1799783980","cluster_addr":"172.21.15.52:6813/1799783980","heartbeat_back_addr":"172.21.15.52:6817/1799783980","heartbeat_front_addr":"172.21.15.52:6815/1799783980","state":["exists","up"]},{"osd":4,"uuid":"1fbfe997-a0e7-4b89-8e6b-754a9bee10a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6809","nonce":2077759625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6811","nonce":2077759625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6815","nonce":2077759625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2077759625},{"type":"v1","addr":"172.21.15.88:6813","nonce":2077759625}]},"public_addr":"172.21.15.88:6809/2077759625","cluster_addr":"172.21.15.88:6811/2077759625","heartbeat_back_addr":"172.21.15.88:6815/2077759625","heartbeat_front_addr":"172.21.15.88:6813/2077759625","state":["exists","up"]},{"osd":5,"uuid":"77eeb2e8-2420-4dcc-8774-beed04c6d827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6809","nonce":3106153812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6811","nonce":3106153812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6815","nonce":3106153812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3106153812},{"type":"v1","addr":"172.21.15.189:6813","nonce":3106153812}]},"public_addr":"172.21.15.189:6809/3106153812","cluster_addr":"172.21.15.189:6811/3106153812","heartbeat_back_addr":"172.21.15.189:6815/3106153812","heartbeat_front_addr":"172.21.15.189:6813/3106153812","state":["exists","up"]},{"osd":6,"uuid":"17099b55-abab-4f7b-8cab-c9573cd9b2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6819","nonce":2542655509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6821","nonce":2542655509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6825","nonce":2542655509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2542655509},{"type":"v1","addr":"172.21.15.52:6823","nonce":2542655509}]},"public_addr":"172.21.15.52:6819/2542655509","cluster_addr":"172.21.15.52:6821/2542655509","heartbeat_back_addr":"172.21.15.52:6825/2542655509","heartbeat_front_addr":"172.21.15.52:6823/2542655509","state":["exists","up"]},{"osd":7,"uuid":"0fe3487a-3c01-483b-95ca-71ce42e74bcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6817","nonce":714628313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6819","nonce":714628313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6823","nonce":714628313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":714628313},{"type":"v1","addr":"172.21.15.88:6821","nonce":714628313}]},"public_addr":"172.21.15.88:6817/714628313","cluster_addr":"172.21.15.88:6819/714628313","heartbeat_back_addr":"172.21.15.88:6823/714628313","heartbeat_front_addr":"172.21.15.88:6821/714628313","state":["exists","up"]},{"osd":8,"uuid":"f93ee6a0-9a7b-4090-9035-a169686c4b9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6817","nonce":1602944902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6819","nonce":1602944902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6823","nonce":1602944902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1602944902},{"type":"v1","addr":"172.21.15.189:6821","nonce":1602944902}]},"public_addr":"172.21.15.189:6817/1602944902","cluster_addr":"172.21.15.189:6819/1602944902","heartbeat_back_addr":"172.21.15.189:6823/1602944902","heartbeat_front_addr":"172.21.15.189:6821/1602944902","state":["exists","up"]},{"osd":9,"uuid":"8aaad86a-c313-467b-8998-13f154bda92a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6827","nonce":2350236811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6829","nonce":2350236811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6833","nonce":2350236811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2350236811},{"type":"v1","addr":"172.21.15.52:6831","nonce":2350236811}]},"public_addr":"172.21.15.52:6827/2350236811","cluster_addr":"172.21.15.52:6829/2350236811","heartbeat_back_addr":"172.21.15.52:6833/2350236811","heartbeat_front_addr":"172.21.15.52:6831/2350236811","state":["exists","up"]},{"osd":10,"uuid":"0d33f6f2-3a01-4753-b2ef-b8c42454198e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6825","nonce":3425831447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6827","nonce":3425831447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6831","nonce":3425831447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3425831447},{"type":"v1","addr":"172.21.15.88:6829","nonce":3425831447}]},"public_addr":"172.21.15.88:6825/3425831447","cluster_addr":"172.21.15.88:6827/3425831447","heartbeat_back_addr":"172.21.15.88:6831/3425831447","heartbeat_front_addr":"172.21.15.88:6829/3425831447","state":["exists","up"]},{"osd":11,"uuid":"4369f21d-d344-4e2f-b556-010faefd182c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6825","nonce":383422941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6827","nonce":383422941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6831","nonce":383422941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":383422941},{"type":"v1","addr":"172.21.15.189:6829","nonce":383422941}]},"public_addr":"172.21.15.189:6825/383422941","cluster_addr":"172.21.15.189:6827/383422941","heartbeat_back_addr":"172.21.15.189:6831/383422941","heartbeat_front_addr":"172.21.15.189:6829/383422941","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:30.241521+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:45:53.556193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:17.339542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:46:42.842419+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:07.757698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:32.713721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:47:58.234113+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:23.506477+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:48:48.838982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:15.957992+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:49:41.904479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:50:08.680595+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/713755826":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/1937276046":"2024-04-03T13:53:29.889159+0000","172.21.15.52:6800/3836085837":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/2957262850":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/2475099387":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6800/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:6801/2368921152":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/2609300748":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6801/3836085837":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/2985814329":"2024-04-03T13:53:29.889159+0000","172.21.15.52:0/85571395":"2024-04-03T13:43:27.309404+0000","172.21.15.52:0/293852652":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/1829428070":"2024-04-03T13:43:27.309404+0000","172.21.15.52:6800/673843171":"2024-04-03T13:43:54.653361+0000","172.21.15.52:0/3257395293":"2024-04-03T13:43:54.653361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T14:04:39.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.681+0000 7feaff590640 1 -- 172.21.15.52:0/1751445348 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7feae40a3da0 msgr2=0x7feae40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:39.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.681+0000 7feaff590640 1 --2- 172.21.15.52:0/1751445348 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7feae40a3da0 0x7feae40a6260 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7feae800ab60 tx=0x7feae800a2d0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.681+0000 7feaff590640 1 -- 172.21.15.52:0/1751445348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feaf8102450 msgr2=0x7feaf8198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:39.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.681+0000 7feaff590640 1 --2- 172.21.15.52:0/1751445348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feaf8102450 0x7feaf8198820 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7feaf400e9f0 tx=0x7feaf400edf0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.681+0000 7feaff590640 1 -- 172.21.15.52:0/1751445348 shutdown_connections 2024-04-02T14:04:39.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.681+0000 7feaff590640 1 --2- 172.21.15.52:0/1751445348 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7feae40a3da0 0x7feae40a6260 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.681+0000 7feaff590640 1 --2- 172.21.15.52:0/1751445348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feaf8107aa0 0x7feaf819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.681+0000 7feaff590640 1 --2- 172.21.15.52:0/1751445348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feaf8102450 0x7feaf8198820 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.681+0000 7feaff590640 1 --2- 172.21.15.52:0/1751445348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaf8101a90 0x7feaf81982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.681+0000 7feaff590640 1 -- 172.21.15.52:0/1751445348 >> 172.21.15.52:0/1751445348 conn(0x7feaf80fd5a0 msgr2=0x7feaf80fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:39.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.681+0000 7feaff590640 1 -- 172.21.15.52:0/1751445348 shutdown_connections 2024-04-02T14:04:39.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.681+0000 7feaff590640 1 -- 172.21.15.52:0/1751445348 wait complete. 2024-04-02T14:04:39.694 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:39 smithi052 ceph-mon[32104]: pgmap v347: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:04:39.695 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:39 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3562291405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:04:39.695 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:39 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2057896414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:04:39.696 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-02T14:04:39.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:39 smithi189 ceph-mon[36683]: pgmap v347: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:04:39.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:39 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3562291405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:04:39.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:39 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2057896414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:04:39.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.796+0000 7f34fa10e640 1 -- 172.21.15.52:0/2872225581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34f40696a0 msgr2=0x7f34f410e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:39.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.796+0000 7f34fa10e640 1 --2- 172.21.15.52:0/2872225581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34f40696a0 0x7f34f410e940 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009cd0 tx=0x7f34ec02c0f0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.796+0000 7f34fa10e640 1 -- 172.21.15.52:0/2872225581 shutdown_connections 2024-04-02T14:04:39.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.796+0000 7f34fa10e640 1 --2- 172.21.15.52:0/2872225581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34f40696a0 0x7f34f410e940 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.796+0000 7f34fa10e640 1 --2- 172.21.15.52:0/2872225581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34f4068ce0 0x7f34f4069160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.796+0000 7f34fa10e640 1 --2- 172.21.15.52:0/2872225581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34f4107a80 0x7f34f4107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.796+0000 7f34fa10e640 1 -- 172.21.15.52:0/2872225581 >> 172.21.15.52:0/2872225581 conn(0x7f34f4077b00 msgr2=0x7f34f4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:39.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.796+0000 7f34fa10e640 1 -- 172.21.15.52:0/2872225581 shutdown_connections 2024-04-02T14:04:39.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.796+0000 7f34fa10e640 1 -- 172.21.15.52:0/2872225581 wait complete. 2024-04-02T14:04:39.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.797+0000 7f34fa10e640 1 Processor -- start 2024-04-02T14:04:39.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.797+0000 7f34fa10e640 1 -- start start 2024-04-02T14:04:39.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.797+0000 7f34fa10e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34f4068ce0 0x7f34f4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:39.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.797+0000 7f34fa10e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34f40696a0 0x7f34f41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:39.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.797+0000 7f34fa10e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34f4107a80 0x7f34f419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:39.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.797+0000 7f34fa10e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f34f4111e90 con 0x7f34f40696a0 2024-04-02T14:04:39.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.797+0000 7f34fa10e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f34f4111d10 con 0x7f34f4107a80 2024-04-02T14:04:39.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.797+0000 7f34fa10e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f34f4112010 con 0x7f34f4068ce0 2024-04-02T14:04:39.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.797+0000 7f34f2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34f40696a0 0x7f34f41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:39.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.797+0000 7f34f2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34f40696a0 0x7f34f41988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51554/0 (socket says 172.21.15.52:51554) 2024-04-02T14:04:39.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.797+0000 7f34f2ffd640 1 -- 172.21.15.52:0/2752012206 learned_addr learned my addr 172.21.15.52:0/2752012206 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:04:39.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.798+0000 7f34f37fe640 1 --2- 172.21.15.52:0/2752012206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34f4068ce0 0x7f34f4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:39.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.798+0000 7f34f3fff640 1 --2- 172.21.15.52:0/2752012206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34f4107a80 0x7f34f419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:39.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.798+0000 7f34f2ffd640 1 -- 172.21.15.52:0/2752012206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34f4068ce0 msgr2=0x7f34f4198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:39.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.798+0000 7f34f2ffd640 1 --2- 172.21.15.52:0/2752012206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34f4068ce0 0x7f34f4198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.798+0000 7f34f2ffd640 1 -- 172.21.15.52:0/2752012206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34f4107a80 msgr2=0x7f34f419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:39.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.798+0000 7f34f2ffd640 1 --2- 172.21.15.52:0/2752012206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34f4107a80 0x7f34f419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:39.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.798+0000 7f34f2ffd640 1 -- 172.21.15.52:0/2752012206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f419d360 con 0x7f34f40696a0 2024-04-02T14:04:39.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.798+0000 7f34f2ffd640 1 --2- 172.21.15.52:0/2752012206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34f40696a0 0x7f34f41988b0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f34e400d950 tx=0x7f34e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:39.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.798+0000 7f34f0ff9640 1 -- 172.21.15.52:0/2752012206 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e4017070 con 0x7f34f40696a0 2024-04-02T14:04:39.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.798+0000 7f34f0ff9640 1 -- 172.21.15.52:0/2752012206 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34e4009c30 con 0x7f34f40696a0 2024-04-02T14:04:39.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.798+0000 7f34fa10e640 1 -- 172.21.15.52:0/2752012206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f419d650 con 0x7f34f40696a0 2024-04-02T14:04:39.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.798+0000 7f34f0ff9640 1 -- 172.21.15.52:0/2752012206 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e400f6b0 con 0x7f34f40696a0 2024-04-02T14:04:39.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.798+0000 7f34fa10e640 1 -- 172.21.15.52:0/2752012206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f41a4ee0 con 0x7f34f40696a0 2024-04-02T14:04:39.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.799+0000 7f34fa10e640 1 -- 172.21.15.52:0/2752012206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f40688f0 con 0x7f34f40696a0 2024-04-02T14:04:39.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.800+0000 7f34f0ff9640 1 -- 172.21.15.52:0/2752012206 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f34e40109d0 con 0x7f34f40696a0 2024-04-02T14:04:39.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.801+0000 7f34f0ff9640 1 --2- 172.21.15.52:0/2752012206 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f34dc0a3d20 0x7f34dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:39.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.801+0000 7f34f37fe640 1 --2- 172.21.15.52:0/2752012206 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f34dc0a3d20 0x7f34dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:39.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.801+0000 7f34f0ff9640 1 -- 172.21.15.52:0/2752012206 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f34e40d0770 con 0x7f34f40696a0 2024-04-02T14:04:39.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.801+0000 7f34f37fe640 1 --2- 172.21.15.52:0/2752012206 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f34dc0a3d20 0x7f34dc0a61e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f34e800bb60 tx=0x7f34e800b390 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:39.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:39.805+0000 7f34f0ff9640 1 -- 172.21.15.52:0/2752012206 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f34e4009de0 con 0x7f34f40696a0 2024-04-02T14:04:39.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:39 smithi088 ceph-mon[36528]: pgmap v347: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T14:04:39.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:39 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3562291405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:04:39.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:39 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2057896414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T14:04:40.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.031+0000 7f34fa10e640 1 -- 172.21.15.52:0/2752012206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f34f4109a50 con 0x7f34f40696a0 2024-04-02T14:04:40.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.033+0000 7f34f0ff9640 1 -- 172.21.15.52:0/2752012206 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) ==== 161+0+0 (secure 0 0 0) 0x7f34e409c3d0 con 0x7f34f40696a0 2024-04-02T14:04:40.037 INFO:teuthology.orchestra.run.smithi052.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-02T14:04:40.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.037+0000 7f34fa10e640 1 -- 172.21.15.52:0/2752012206 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f34dc0a3d20 msgr2=0x7f34dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:40.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.037+0000 7f34fa10e640 1 --2- 172.21.15.52:0/2752012206 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f34dc0a3d20 0x7f34dc0a61e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f34e800bb60 tx=0x7f34e800b390 comp rx=0 tx=0).stop 2024-04-02T14:04:40.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.037+0000 7f34fa10e640 1 -- 172.21.15.52:0/2752012206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34f40696a0 msgr2=0x7f34f41988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:40.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.037+0000 7f34fa10e640 1 --2- 172.21.15.52:0/2752012206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34f40696a0 0x7f34f41988b0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f34e400d950 tx=0x7f34e400de10 comp rx=0 tx=0).stop 2024-04-02T14:04:40.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.037+0000 7f34fa10e640 1 -- 172.21.15.52:0/2752012206 shutdown_connections 2024-04-02T14:04:40.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.037+0000 7f34fa10e640 1 --2- 172.21.15.52:0/2752012206 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f34dc0a3d20 0x7f34dc0a61e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:40.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.037+0000 7f34fa10e640 1 --2- 172.21.15.52:0/2752012206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34f4107a80 0x7f34f419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:40.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.037+0000 7f34fa10e640 1 --2- 172.21.15.52:0/2752012206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34f40696a0 0x7f34f41988b0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:40.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.037+0000 7f34fa10e640 1 --2- 172.21.15.52:0/2752012206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34f4068ce0 0x7f34f4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:40.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.037+0000 7f34fa10e640 1 -- 172.21.15.52:0/2752012206 >> 172.21.15.52:0/2752012206 conn(0x7f34f4077b00 msgr2=0x7f34f40fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:40.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.037+0000 7f34fa10e640 1 -- 172.21.15.52:0/2752012206 shutdown_connections 2024-04-02T14:04:40.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.037+0000 7f34fa10e640 1 -- 172.21.15.52:0/2752012206 wait complete. 2024-04-02T14:04:40.053 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-02T14:04:40.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.166+0000 7f2cf5263640 1 -- 172.21.15.52:0/791423009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cf0068ce0 msgr2=0x7f2cf0069160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:40.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.166+0000 7f2cf5263640 1 --2- 172.21.15.52:0/791423009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cf0068ce0 0x7f2cf0069160 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009930 tx=0x7f2ce402bc30 comp rx=0 tx=0).stop 2024-04-02T14:04:40.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.166+0000 7f2cf5263640 1 -- 172.21.15.52:0/791423009 shutdown_connections 2024-04-02T14:04:40.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.166+0000 7f2cf5263640 1 --2- 172.21.15.52:0/791423009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cf00696a0 0x7f2cf010e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:40.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.166+0000 7f2cf5263640 1 --2- 172.21.15.52:0/791423009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cf0068ce0 0x7f2cf0069160 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:40.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.166+0000 7f2cf5263640 1 --2- 172.21.15.52:0/791423009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2cf0107a80 0x7f2cf0107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:40.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.166+0000 7f2cf5263640 1 -- 172.21.15.52:0/791423009 >> 172.21.15.52:0/791423009 conn(0x7f2cf0077b00 msgr2=0x7f2cf0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:40.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.167+0000 7f2cf5263640 1 -- 172.21.15.52:0/791423009 shutdown_connections 2024-04-02T14:04:40.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.167+0000 7f2cf5263640 1 -- 172.21.15.52:0/791423009 wait complete. 2024-04-02T14:04:40.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.167+0000 7f2cf5263640 1 Processor -- start 2024-04-02T14:04:40.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.167+0000 7f2cf5263640 1 -- start start 2024-04-02T14:04:40.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.167+0000 7f2cf5263640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2cf0068ce0 0x7f2cf01009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:40.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.167+0000 7f2cf5263640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cf00696a0 0x7f2cf0100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:40.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.168+0000 7f2cf5263640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cf0107a80 0x7f2cf0101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:40.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.168+0000 7f2cf5263640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf0111e90 con 0x7f2cf0107a80 2024-04-02T14:04:40.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.168+0000 7f2cf5263640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf0111d10 con 0x7f2cf0068ce0 2024-04-02T14:04:40.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.168+0000 7f2cf5263640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf0112010 con 0x7f2cf00696a0 2024-04-02T14:04:40.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.168+0000 7f2cef7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cf0107a80 0x7f2cf0101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:40.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.168+0000 7f2cee7fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cf00696a0 0x7f2cf0100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:40.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.168+0000 7f2cef7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cf0107a80 0x7f2cf0101440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51610/0 (socket says 172.21.15.52:51610) 2024-04-02T14:04:40.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.168+0000 7f2cef7fe640 1 -- 172.21.15.52:0/3046786299 learned_addr learned my addr 172.21.15.52:0/3046786299 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:04:40.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.168+0000 7f2ceeffd640 1 --2- 172.21.15.52:0/3046786299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2cf0068ce0 0x7f2cf01009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:40.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.168+0000 7f2cee7fc640 1 -- 172.21.15.52:0/3046786299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2cf0068ce0 msgr2=0x7f2cf01009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:40.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.168+0000 7f2cee7fc640 1 --2- 172.21.15.52:0/3046786299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2cf0068ce0 0x7f2cf01009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:40.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.168+0000 7f2cee7fc640 1 -- 172.21.15.52:0/3046786299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cf0107a80 msgr2=0x7f2cf0101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:40.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.168+0000 7f2cee7fc640 1 --2- 172.21.15.52:0/3046786299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cf0107a80 0x7f2cf0101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:40.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.168+0000 7f2cee7fc640 1 -- 172.21.15.52:0/3046786299 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf01a68f0 con 0x7f2cf00696a0 2024-04-02T14:04:40.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.168+0000 7f2cef7fe640 1 --2- 172.21.15.52:0/3046786299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cf0107a80 0x7f2cf0101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T14:04:40.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.168+0000 7f2cee7fc640 1 --2- 172.21.15.52:0/3046786299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cf00696a0 0x7f2cf0100f00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009230 tx=0x7f2ce402cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:40.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.168+0000 7f2cd7fff640 1 -- 172.21.15.52:0/3046786299 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4041070 con 0x7f2cf00696a0 2024-04-02T14:04:40.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.168+0000 7f2cf5263640 1 -- 172.21.15.52:0/3046786299 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cf01a6b80 con 0x7f2cf00696a0 2024-04-02T14:04:40.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.169+0000 7f2cd7fff640 1 -- 172.21.15.52:0/3046786299 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ce4004450 con 0x7f2cf00696a0 2024-04-02T14:04:40.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.169+0000 7f2cd7fff640 1 -- 172.21.15.52:0/3046786299 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4003df0 con 0x7f2cf00696a0 2024-04-02T14:04:40.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.169+0000 7f2cf5263640 1 -- 172.21.15.52:0/3046786299 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cf01a7090 con 0x7f2cf00696a0 2024-04-02T14:04:40.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.169+0000 7f2cf5263640 1 -- 172.21.15.52:0/3046786299 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cf00688f0 con 0x7f2cf00696a0 2024-04-02T14:04:40.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.171+0000 7f2cd7fff640 1 -- 172.21.15.52:0/3046786299 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2ce4044070 con 0x7f2cf00696a0 2024-04-02T14:04:40.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.171+0000 7f2cd7fff640 1 --2- 172.21.15.52:0/3046786299 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2cd80a3d20 0x7f2cd80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:40.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.171+0000 7f2cd7fff640 1 -- 172.21.15.52:0/3046786299 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9780+0+0 (secure 0 0 0) 0x7f2ce40f24e0 con 0x7f2cf00696a0 2024-04-02T14:04:40.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.171+0000 7f2ceeffd640 1 --2- 172.21.15.52:0/3046786299 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2cd80a3d20 0x7f2cd80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:40.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.172+0000 7f2ceeffd640 1 --2- 172.21.15.52:0/3046786299 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2cd80a3d20 0x7f2cd80a61e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0005fd0 tx=0x7f2ce0009500 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:40.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.175+0000 7f2cd7fff640 1 -- 172.21.15.52:0/3046786299 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2ce40be150 con 0x7f2cf00696a0 2024-04-02T14:04:40.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:40 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-a[32100]: 2024-04-02T14:04:40.032+0000 7f1bf6f77700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T14:04:40.340 INFO:journalctl@ceph.mds.a.smithi052.stdout:Apr 02 14:04:40 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-a[71342]: ignoring --setuser ceph since I am not root 2024-04-02T14:04:40.341 INFO:journalctl@ceph.mds.a.smithi052.stdout:Apr 02 14:04:40 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-a[71342]: ignoring --setgroup ceph since I am not root 2024-04-02T14:04:40.341 INFO:journalctl@ceph.mds.a.smithi052.stdout:Apr 02 14:04:40 smithi052 ceph-mds[71346]: -- 172.21.15.52:0/1887537194 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564ace135a40 con 0x564ace108800 2024-04-02T14:04:40.341 INFO:journalctl@ceph.mds.a.smithi052.stdout:Apr 02 14:04:40 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-a[71342]: starting mds.a at 2024-04-02T14:04:40.341 INFO:journalctl@ceph.mds.g.smithi052.stdout:Apr 02 14:04:40 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-g[70987]: ignoring --setuser ceph since I am not root 2024-04-02T14:04:40.341 INFO:journalctl@ceph.mds.g.smithi052.stdout:Apr 02 14:04:40 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-g[70987]: ignoring --setgroup ceph since I am not root 2024-04-02T14:04:40.341 INFO:journalctl@ceph.mds.g.smithi052.stdout:Apr 02 14:04:40 smithi052 ceph-mds[70991]: -- 172.21.15.52:0/857326715 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5639545cba40 con 0x56395459f000 2024-04-02T14:04:40.342 INFO:journalctl@ceph.mds.g.smithi052.stdout:Apr 02 14:04:40 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-g[70987]: starting mds.g at 2024-04-02T14:04:40.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:40.405+0000 7f2cf5263640 1 -- 172.21.15.52:0/3046786299 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2cf0109fe0 con 0x7f2cf00696a0 2024-04-02T14:04:40.420 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-e[66042]: ignoring --setuser ceph since I am not root 2024-04-02T14:04:40.420 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-e[66042]: ignoring --setgroup ceph since I am not root 2024-04-02T14:04:40.420 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-mds[66046]: -- 172.21.15.88:0/3137012227 <== mon.2 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56281d8cba40 con 0x56281d89f000 2024-04-02T14:04:40.421 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-e[66042]: starting mds.e at 2024-04-02T14:04:40.421 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-b[66653]: ignoring --setuser ceph since I am not root 2024-04-02T14:04:40.421 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-b[66653]: ignoring --setgroup ceph since I am not root 2024-04-02T14:04:40.421 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-mds[66657]: -- 172.21.15.88:0/140491979 <== mon.2 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55adecc7ba40 con 0x55adecc4ec00 2024-04-02T14:04:40.421 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-b[66653]: starting mds.b at 2024-04-02T14:04:40.422 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-h[66349]: ignoring --setuser ceph since I am not root 2024-04-02T14:04:40.422 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-h[66349]: ignoring --setgroup ceph since I am not root 2024-04-02T14:04:40.422 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-mds[66353]: -- 172.21.15.88:0/2097207882 <== mon.2 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557376bdba40 con 0x557376baec00 2024-04-02T14:04:40.423 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-h[66349]: starting mds.h at 2024-04-02T14:04:40.423 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-h[66349]: 2024-04-02T14:04:40.081+0000 7f8453d9d700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-04-02T14:04:40.423 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-k[65717]: ignoring --setuser ceph since I am not root 2024-04-02T14:04:40.424 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-k[65717]: ignoring --setgroup ceph since I am not root 2024-04-02T14:04:40.424 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-mds[65721]: -- 172.21.15.88:0/3704858387 <== mon.2 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5600e5627a40 con 0x5600e55fac00 2024-04-02T14:04:40.424 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-k[65717]: starting mds.k at 2024-04-02T14:04:40.532 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-f[66881]: ignoring --setuser ceph since I am not root 2024-04-02T14:04:40.532 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-f[66881]: ignoring --setgroup ceph since I am not root 2024-04-02T14:04:40.533 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-mds[66885]: -- 172.21.15.189:0/4250203127 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556344c0fa40 con 0x556344be2c00 2024-04-02T14:04:40.533 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-f[66881]: starting mds.f at 2024-04-02T14:04:40.533 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-l[66560]: ignoring --setuser ceph since I am not root 2024-04-02T14:04:40.533 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-l[66560]: ignoring --setgroup ceph since I am not root 2024-04-02T14:04:40.533 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-mds[66564]: -- 172.21.15.189:0/1567189555 <== mon.2 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b498af9a40 con 0x55b498accc00 2024-04-02T14:04:40.534 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-l[66560]: starting mds.l at 2024-04-02T14:04:40.534 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-i[67499]: ignoring --setuser ceph since I am not root 2024-04-02T14:04:40.534 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-i[67499]: ignoring --setgroup ceph since I am not root 2024-04-02T14:04:40.534 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-mds[67503]: -- 172.21.15.189:0/3490819977 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a80e8584e0 con 0x55a80e82ec00 2024-04-02T14:04:40.535 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-mds[67503]: -- 172.21.15.189:0/3490819977 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a80e85ba40 con 0x55a80e82ec00 2024-04-02T14:04:40.535 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-i[67499]: starting mds.i at 2024-04-02T14:04:40.535 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-c[67191]: ignoring --setuser ceph since I am not root 2024-04-02T14:04:40.535 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-c[67191]: ignoring --setgroup ceph since I am not root 2024-04-02T14:04:40.535 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-mds[67195]: -- 172.21.15.189:0/2653079012 <== mon.2 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5640bd729a40 con 0x5640bd6fc800 2024-04-02T14:04:40.535 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-c[67191]: starting mds.c at 2024-04-02T14:04:40.788 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1751445348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:04:40.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2752012206' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-02T14:04:40.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-mon[36683]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T14:04:40.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-mon[36683]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T14:04:40.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2752012206' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-02T14:04:40.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-mon[36683]: osdmap e104: 12 total, 12 up, 12 in 2024-04-02T14:04:40.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-mon[36683]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-02T14:04:40.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/3046786299' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T14:04:40.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:40 smithi189 ceph-mon[36683]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T14:04:40.840 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:40 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1751445348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:04:40.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:40 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2752012206' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-02T14:04:40.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:40 smithi052 ceph-mon[32104]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T14:04:40.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:40 smithi052 ceph-mon[32104]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T14:04:40.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:40 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2752012206' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-02T14:04:40.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:40 smithi052 ceph-mon[32104]: osdmap e104: 12 total, 12 up, 12 in 2024-04-02T14:04:40.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:40 smithi052 ceph-mon[32104]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-02T14:04:40.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:40 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/3046786299' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T14:04:40.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:40 smithi052 ceph-mon[32104]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T14:04:40.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1751445348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T14:04:40.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2752012206' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-02T14:04:40.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-mon[36528]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T14:04:40.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-mon[36528]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T14:04:40.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2752012206' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-02T14:04:40.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-mon[36528]: osdmap e104: 12 total, 12 up, 12 in 2024-04-02T14:04:40.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-mon[36528]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-02T14:04:40.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/3046786299' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T14:04:40.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:40 smithi088 ceph-mon[36528]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T14:04:41.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.044+0000 7f2cd7fff640 1 -- 172.21.15.52:0/3046786299 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) ==== 104+0+0 (secure 0 0 0) 0x7f2ce40c2450 con 0x7f2cf00696a0 2024-04-02T14:04:41.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.048+0000 7f2cf5263640 1 -- 172.21.15.52:0/3046786299 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2cd80a3d20 msgr2=0x7f2cd80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:41.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.048+0000 7f2cf5263640 1 --2- 172.21.15.52:0/3046786299 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2cd80a3d20 0x7f2cd80a61e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0005fd0 tx=0x7f2ce0009500 comp rx=0 tx=0).stop 2024-04-02T14:04:41.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.048+0000 7f2cf5263640 1 -- 172.21.15.52:0/3046786299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cf00696a0 msgr2=0x7f2cf0100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:41.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.048+0000 7f2cf5263640 1 --2- 172.21.15.52:0/3046786299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cf00696a0 0x7f2cf0100f00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009230 tx=0x7f2ce402cbf0 comp rx=0 tx=0).stop 2024-04-02T14:04:41.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.048+0000 7f2cf5263640 1 -- 172.21.15.52:0/3046786299 shutdown_connections 2024-04-02T14:04:41.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.048+0000 7f2cf5263640 1 --2- 172.21.15.52:0/3046786299 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f2cd80a3d20 0x7f2cd80a61e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:41.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.048+0000 7f2cf5263640 1 --2- 172.21.15.52:0/3046786299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cf0107a80 0x7f2cf0101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:41.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.048+0000 7f2cf5263640 1 --2- 172.21.15.52:0/3046786299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cf00696a0 0x7f2cf0100f00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:41.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.048+0000 7f2cf5263640 1 --2- 172.21.15.52:0/3046786299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2cf0068ce0 0x7f2cf01009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:41.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.048+0000 7f2cf5263640 1 -- 172.21.15.52:0/3046786299 >> 172.21.15.52:0/3046786299 conn(0x7f2cf0077b00 msgr2=0x7f2cf00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:41.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.049+0000 7f2cf5263640 1 -- 172.21.15.52:0/3046786299 shutdown_connections 2024-04-02T14:04:41.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.049+0000 7f2cf5263640 1 -- 172.21.15.52:0/3046786299 wait complete. 2024-04-02T14:04:41.065 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-02T14:04:41.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.169+0000 7f9ba7b1c640 1 -- 172.21.15.52:0/1700777422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ba0101330 msgr2=0x7f9ba010d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:41.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.169+0000 7f9ba7b1c640 1 --2- 172.21.15.52:0/1700777422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ba0101330 0x7f9ba010d840 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f9b98009cd0 tx=0x7f9b9802c0f0 comp rx=0 tx=0).stop 2024-04-02T14:04:41.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.169+0000 7f9ba7b1c640 1 -- 172.21.15.52:0/1700777422 shutdown_connections 2024-04-02T14:04:41.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.170+0000 7f9ba7b1c640 1 --2- 172.21.15.52:0/1700777422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ba0101330 0x7f9ba010d840 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:41.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.170+0000 7f9ba7b1c640 1 --2- 172.21.15.52:0/1700777422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ba0100970 0x7f9ba0100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:41.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.170+0000 7f9ba7b1c640 1 --2- 172.21.15.52:0/1700777422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ba0106980 0x7f9ba0106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:41.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.170+0000 7f9ba7b1c640 1 -- 172.21.15.52:0/1700777422 >> 172.21.15.52:0/1700777422 conn(0x7f9ba0077b00 msgr2=0x7f9ba0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:41.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.170+0000 7f9ba7b1c640 1 -- 172.21.15.52:0/1700777422 shutdown_connections 2024-04-02T14:04:41.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.170+0000 7f9ba7b1c640 1 -- 172.21.15.52:0/1700777422 wait complete. 2024-04-02T14:04:41.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.170+0000 7f9ba7b1c640 1 Processor -- start 2024-04-02T14:04:41.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.170+0000 7f9ba7b1c640 1 -- start start 2024-04-02T14:04:41.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.170+0000 7f9ba7b1c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ba0100970 0x7f9ba0199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:41.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.170+0000 7f9ba7b1c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ba0101330 0x7f9ba01996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:41.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.170+0000 7f9ba7b1c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ba0106980 0x7f9ba01a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:41.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.170+0000 7f9ba7b1c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba0110d60 con 0x7f9ba0101330 2024-04-02T14:04:41.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.170+0000 7f9ba7b1c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba0110be0 con 0x7f9ba0100970 2024-04-02T14:04:41.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.170+0000 7f9ba7b1c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba0110ee0 con 0x7f9ba0106980 2024-04-02T14:04:41.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.171+0000 7f9ba50b7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ba0101330 0x7f9ba01996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:41.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.171+0000 7f9ba50b7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ba0101330 0x7f9ba01996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51628/0 (socket says 172.21.15.52:51628) 2024-04-02T14:04:41.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.171+0000 7f9ba50b7640 1 -- 172.21.15.52:0/1381069808 learned_addr learned my addr 172.21.15.52:0/1381069808 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:04:41.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.171+0000 7f9ba60b9640 1 --2- 172.21.15.52:0/1381069808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ba0106980 0x7f9ba01a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:41.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.171+0000 7f9ba58b8640 1 --2- 172.21.15.52:0/1381069808 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ba0100970 0x7f9ba0199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:41.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.171+0000 7f9ba50b7640 1 -- 172.21.15.52:0/1381069808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ba0106980 msgr2=0x7f9ba01a4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:41.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.171+0000 7f9ba50b7640 1 --2- 172.21.15.52:0/1381069808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ba0106980 0x7f9ba01a4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:41.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.171+0000 7f9ba50b7640 1 -- 172.21.15.52:0/1381069808 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ba0100970 msgr2=0x7f9ba0199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:41.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.171+0000 7f9ba50b7640 1 --2- 172.21.15.52:0/1381069808 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ba0100970 0x7f9ba0199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:41.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.171+0000 7f9ba50b7640 1 -- 172.21.15.52:0/1381069808 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba01a4740 con 0x7f9ba0101330 2024-04-02T14:04:41.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.171+0000 7f9ba50b7640 1 --2- 172.21.15.52:0/1381069808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ba0101330 0x7f9ba01996c0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000d950 tx=0x7f9b9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:41.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.171+0000 7f9b96ffd640 1 -- 172.21.15.52:0/1381069808 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b90017070 con 0x7f9ba0101330 2024-04-02T14:04:41.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.171+0000 7f9b96ffd640 1 -- 172.21.15.52:0/1381069808 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b90009c30 con 0x7f9ba0101330 2024-04-02T14:04:41.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.171+0000 7f9ba7b1c640 1 -- 172.21.15.52:0/1381069808 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba01a4910 con 0x7f9ba0101330 2024-04-02T14:04:41.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.171+0000 7f9b96ffd640 1 -- 172.21.15.52:0/1381069808 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9000f6b0 con 0x7f9ba0101330 2024-04-02T14:04:41.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.171+0000 7f9ba7b1c640 1 -- 172.21.15.52:0/1381069808 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba01a4eb0 con 0x7f9ba0101330 2024-04-02T14:04:41.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.172+0000 7f9ba7b1c640 1 -- 172.21.15.52:0/1381069808 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ba00688f0 con 0x7f9ba0101330 2024-04-02T14:04:41.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.173+0000 7f9b96ffd640 1 -- 172.21.15.52:0/1381069808 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9b900109d0 con 0x7f9ba0101330 2024-04-02T14:04:41.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.174+0000 7f9b96ffd640 1 --2- 172.21.15.52:0/1381069808 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9b8c0a3da0 0x7f9b8c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:41.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.174+0000 7f9ba58b8640 1 --2- 172.21.15.52:0/1381069808 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9b8c0a3da0 0x7f9b8c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:41.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.174+0000 7f9b96ffd640 1 -- 172.21.15.52:0/1381069808 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9780+0+0 (secure 0 0 0) 0x7f9b900d0900 con 0x7f9ba0101330 2024-04-02T14:04:41.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.174+0000 7f9ba58b8640 1 --2- 172.21.15.52:0/1381069808 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9b8c0a3da0 0x7f9b8c0a6260 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c005e00 tx=0x7f9b9c005c50 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:41.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.179+0000 7f9b96ffd640 1 -- 172.21.15.52:0/1381069808 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9b90009de0 con 0x7f9ba0101330 2024-04-02T14:04:41.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:41.366+0000 7f9ba7b1c640 1 -- 172.21.15.52:0/1381069808 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9ba0108f00 con 0x7f9ba0101330 2024-04-02T14:04:41.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: pgmap v349: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T14:04:41.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T14:04:41.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T14:04:41.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: Cluster is now healthy 2024-04-02T14:04:41.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-02T14:04:41.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.88:6833/1835972451,v1:172.21.15.88:6835/1835972451] up:boot 2024-04-02T14:04:41.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.52:6838/630069796,v1:172.21.15.52:6840/630069796] up:boot 2024-04-02T14:04:41.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.52:6839/2438488985,v1:172.21.15.52:6841/2438488985] up:boot 2024-04-02T14:04:41.591 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.189:6836/336524067,v1:172.21.15.189:6838/336524067] up:boot 2024-04-02T14:04:41.592 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.88:6832/4046820324,v1:172.21.15.88:6834/4046820324] up:boot 2024-04-02T14:04:41.592 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.189:6837/820130827,v1:172.21.15.189:6839/820130827] up:boot 2024-04-02T14:04:41.592 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.189:6832/3945126957,v1:172.21.15.189:6834/3945126957] up:boot 2024-04-02T14:04:41.592 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.88:6838/1297991543,v1:172.21.15.88:6839/1297991543] up:boot 2024-04-02T14:04:41.592 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.189:6833/1913108199,v1:172.21.15.189:6835/1913108199] up:boot 2024-04-02T14:04:41.592 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: mds.? [v2:172.21.15.88:6836/1051563524,v1:172.21.15.88:6837/1051563524] up:boot 2024-04-02T14:04:41.592 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: fsmap 12 up:standby 2024-04-02T14:04:41.592 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T14:04:41.593 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T14:04:41.593 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T14:04:41.593 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T14:04:41.593 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T14:04:41.593 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T14:04:41.593 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T14:04:41.593 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T14:04:41.593 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T14:04:41.594 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T14:04:41.594 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:41 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1381069808' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T14:04:41.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: pgmap v349: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T14:04:41.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T14:04:41.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T14:04:41.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: Cluster is now healthy 2024-04-02T14:04:41.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-02T14:04:41.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.88:6833/1835972451,v1:172.21.15.88:6835/1835972451] up:boot 2024-04-02T14:04:41.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.52:6838/630069796,v1:172.21.15.52:6840/630069796] up:boot 2024-04-02T14:04:41.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.52:6839/2438488985,v1:172.21.15.52:6841/2438488985] up:boot 2024-04-02T14:04:41.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.189:6836/336524067,v1:172.21.15.189:6838/336524067] up:boot 2024-04-02T14:04:41.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.88:6832/4046820324,v1:172.21.15.88:6834/4046820324] up:boot 2024-04-02T14:04:41.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.189:6837/820130827,v1:172.21.15.189:6839/820130827] up:boot 2024-04-02T14:04:41.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.189:6832/3945126957,v1:172.21.15.189:6834/3945126957] up:boot 2024-04-02T14:04:41.790 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.88:6838/1297991543,v1:172.21.15.88:6839/1297991543] up:boot 2024-04-02T14:04:41.791 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.189:6833/1913108199,v1:172.21.15.189:6835/1913108199] up:boot 2024-04-02T14:04:41.791 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: mds.? [v2:172.21.15.88:6836/1051563524,v1:172.21.15.88:6837/1051563524] up:boot 2024-04-02T14:04:41.791 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: fsmap 12 up:standby 2024-04-02T14:04:41.791 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T14:04:41.791 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T14:04:41.791 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T14:04:41.792 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T14:04:41.792 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T14:04:41.792 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T14:04:41.792 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T14:04:41.792 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T14:04:41.792 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T14:04:41.792 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T14:04:41.792 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:41 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1381069808' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T14:04:41.920 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: pgmap v349: 129 pgs: 129 active+clean; 73 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T14:04:41.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T14:04:41.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T14:04:41.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: Cluster is now healthy 2024-04-02T14:04:41.921 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-02T14:04:41.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.88:6833/1835972451,v1:172.21.15.88:6835/1835972451] up:boot 2024-04-02T14:04:41.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.52:6838/630069796,v1:172.21.15.52:6840/630069796] up:boot 2024-04-02T14:04:41.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.52:6839/2438488985,v1:172.21.15.52:6841/2438488985] up:boot 2024-04-02T14:04:41.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.189:6836/336524067,v1:172.21.15.189:6838/336524067] up:boot 2024-04-02T14:04:41.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.88:6832/4046820324,v1:172.21.15.88:6834/4046820324] up:boot 2024-04-02T14:04:41.922 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.189:6837/820130827,v1:172.21.15.189:6839/820130827] up:boot 2024-04-02T14:04:41.923 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.189:6832/3945126957,v1:172.21.15.189:6834/3945126957] up:boot 2024-04-02T14:04:41.923 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.88:6838/1297991543,v1:172.21.15.88:6839/1297991543] up:boot 2024-04-02T14:04:41.923 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.189:6833/1913108199,v1:172.21.15.189:6835/1913108199] up:boot 2024-04-02T14:04:41.923 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: mds.? [v2:172.21.15.88:6836/1051563524,v1:172.21.15.88:6837/1051563524] up:boot 2024-04-02T14:04:41.924 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: fsmap 12 up:standby 2024-04-02T14:04:41.924 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T14:04:41.924 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T14:04:41.924 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T14:04:41.925 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T14:04:41.925 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T14:04:41.925 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T14:04:41.925 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T14:04:41.926 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T14:04:41.926 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T14:04:41.926 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: from='mgr.25160 172.21.15.52:0/497442129' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T14:04:41.926 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:41 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1381069808' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T14:04:42.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.042+0000 7f9b96ffd640 1 -- 172.21.15.52:0/1381069808 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) ==== 181+0+0 (secure 0 0 0) 0x7f9b9009ca50 con 0x7f9ba0101330 2024-04-02T14:04:42.047 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_metadata' removed 2024-04-02T14:04:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.046+0000 7f9ba7b1c640 1 -- 172.21.15.52:0/1381069808 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9b8c0a3da0 msgr2=0x7f9b8c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.046+0000 7f9ba7b1c640 1 --2- 172.21.15.52:0/1381069808 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9b8c0a3da0 0x7f9b8c0a6260 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c005e00 tx=0x7f9b9c005c50 comp rx=0 tx=0).stop 2024-04-02T14:04:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.046+0000 7f9ba7b1c640 1 -- 172.21.15.52:0/1381069808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ba0101330 msgr2=0x7f9ba01996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.046+0000 7f9ba7b1c640 1 --2- 172.21.15.52:0/1381069808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ba0101330 0x7f9ba01996c0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000d950 tx=0x7f9b9000de10 comp rx=0 tx=0).stop 2024-04-02T14:04:42.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.046+0000 7f9ba7b1c640 1 -- 172.21.15.52:0/1381069808 shutdown_connections 2024-04-02T14:04:42.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.046+0000 7f9ba7b1c640 1 --2- 172.21.15.52:0/1381069808 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f9b8c0a3da0 0x7f9b8c0a6260 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:42.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.046+0000 7f9ba7b1c640 1 --2- 172.21.15.52:0/1381069808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ba0106980 0x7f9ba01a4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:42.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.046+0000 7f9ba7b1c640 1 --2- 172.21.15.52:0/1381069808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ba0101330 0x7f9ba01996c0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:42.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.046+0000 7f9ba7b1c640 1 --2- 172.21.15.52:0/1381069808 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ba0100970 0x7f9ba0199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:42.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.046+0000 7f9ba7b1c640 1 -- 172.21.15.52:0/1381069808 >> 172.21.15.52:0/1381069808 conn(0x7f9ba0077b00 msgr2=0x7f9ba0103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:42.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.047+0000 7f9ba7b1c640 1 -- 172.21.15.52:0/1381069808 shutdown_connections 2024-04-02T14:04:42.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.047+0000 7f9ba7b1c640 1 -- 172.21.15.52:0/1381069808 wait complete. 2024-04-02T14:04:42.066 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-02T14:04:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.177+0000 7f313a43a640 1 -- 172.21.15.52:0/699310998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3134100970 msgr2=0x7f3134100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.177+0000 7f313a43a640 1 --2- 172.21.15.52:0/699310998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3134100970 0x7f3134100df0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f3128009930 tx=0x7f312802bc30 comp rx=0 tx=0).stop 2024-04-02T14:04:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.177+0000 7f313a43a640 1 -- 172.21.15.52:0/699310998 shutdown_connections 2024-04-02T14:04:42.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.177+0000 7f313a43a640 1 --2- 172.21.15.52:0/699310998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3134101330 0x7f313410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.177+0000 7f313a43a640 1 --2- 172.21.15.52:0/699310998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3134100970 0x7f3134100df0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.177+0000 7f313a43a640 1 --2- 172.21.15.52:0/699310998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3134106980 0x7f3134106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.177+0000 7f313a43a640 1 -- 172.21.15.52:0/699310998 >> 172.21.15.52:0/699310998 conn(0x7f3134077b00 msgr2=0x7f3134077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.177+0000 7f313a43a640 1 -- 172.21.15.52:0/699310998 shutdown_connections 2024-04-02T14:04:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.177+0000 7f313a43a640 1 -- 172.21.15.52:0/699310998 wait complete. 2024-04-02T14:04:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.177+0000 7f313a43a640 1 Processor -- start 2024-04-02T14:04:42.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.177+0000 7f313a43a640 1 -- start start 2024-04-02T14:04:42.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.178+0000 7f313a43a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3134100970 0x7f3134197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:42.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.178+0000 7f313a43a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3134101330 0x7f3134197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:42.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.178+0000 7f313a43a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3134106980 0x7f313419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:42.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.178+0000 7f313a43a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3134110d90 con 0x7f3134101330 2024-04-02T14:04:42.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.178+0000 7f313a43a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3134110c10 con 0x7f3134106980 2024-04-02T14:04:42.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.178+0000 7f313a43a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3134110f10 con 0x7f3134100970 2024-04-02T14:04:42.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.178+0000 7f31337fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3134101330 0x7f3134197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:42.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.178+0000 7f31337fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3134101330 0x7f3134197850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48450/0 (socket says 172.21.15.52:48450) 2024-04-02T14:04:42.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.178+0000 7f31337fe640 1 -- 172.21.15.52:0/4216884275 learned_addr learned my addr 172.21.15.52:0/4216884275 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:04:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.178+0000 7f31389d7640 1 --2- 172.21.15.52:0/4216884275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3134106980 0x7f313419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.178+0000 7f3133fff640 1 --2- 172.21.15.52:0/4216884275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3134100970 0x7f3134197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.178+0000 7f31337fe640 1 -- 172.21.15.52:0/4216884275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3134100970 msgr2=0x7f3134197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.178+0000 7f31337fe640 1 --2- 172.21.15.52:0/4216884275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3134100970 0x7f3134197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.178+0000 7f31337fe640 1 -- 172.21.15.52:0/4216884275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3134106980 msgr2=0x7f313419bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.178+0000 7f31337fe640 1 --2- 172.21.15.52:0/4216884275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3134106980 0x7f313419bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.178+0000 7f31337fe640 1 -- 172.21.15.52:0/4216884275 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313419c300 con 0x7f3134101330 2024-04-02T14:04:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.178+0000 7f31389d7640 1 --2- 172.21.15.52:0/4216884275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3134106980 0x7f313419bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T14:04:42.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.178+0000 7f31337fe640 1 --2- 172.21.15.52:0/4216884275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3134101330 0x7f3134197850 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f3128009230 tx=0x7f312802cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:42.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.178+0000 7f31317fa640 1 -- 172.21.15.52:0/4216884275 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128041070 con 0x7f3134101330 2024-04-02T14:04:42.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.178+0000 7f31317fa640 1 -- 172.21.15.52:0/4216884275 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3128004450 con 0x7f3134101330 2024-04-02T14:04:42.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.178+0000 7f313a43a640 1 -- 172.21.15.52:0/4216884275 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313419c590 con 0x7f3134101330 2024-04-02T14:04:42.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.178+0000 7f31317fa640 1 -- 172.21.15.52:0/4216884275 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128034440 con 0x7f3134101330 2024-04-02T14:04:42.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.178+0000 7f313a43a640 1 -- 172.21.15.52:0/4216884275 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31341a3e80 con 0x7f3134101330 2024-04-02T14:04:42.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.179+0000 7f313a43a640 1 -- 172.21.15.52:0/4216884275 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31340688f0 con 0x7f3134101330 2024-04-02T14:04:42.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.180+0000 7f31317fa640 1 -- 172.21.15.52:0/4216884275 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3128044070 con 0x7f3134101330 2024-04-02T14:04:42.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.181+0000 7f31317fa640 1 --2- 172.21.15.52:0/4216884275 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f311c0a3d20 0x7f311c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:42.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.181+0000 7f3133fff640 1 --2- 172.21.15.52:0/4216884275 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f311c0a3d20 0x7f311c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:42.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.181+0000 7f31317fa640 1 -- 172.21.15.52:0/4216884275 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9284+0+0 (secure 0 0 0) 0x7f31280f25f0 con 0x7f3134101330 2024-04-02T14:04:42.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.182+0000 7f3133fff640 1 --2- 172.21.15.52:0/4216884275 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f311c0a3d20 0x7f311c0a61e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3124009890 tx=0x7f3124008040 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:42.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.186+0000 7f31317fa640 1 -- 172.21.15.52:0/4216884275 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f31280bea40 con 0x7f3134101330 2024-04-02T14:04:42.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:42.381+0000 7f313a43a640 1 -- 172.21.15.52:0/4216884275 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3134198660 con 0x7f3134101330 2024-04-02T14:04:43.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.048+0000 7f31317fa640 1 -- 172.21.15.52:0/4216884275 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v106) ==== 169+0+0 (secure 0 0 0) 0x7f31280be340 con 0x7f3134101330 2024-04-02T14:04:43.052 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data' removed 2024-04-02T14:04:43.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.052+0000 7f313a43a640 1 -- 172.21.15.52:0/4216884275 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f311c0a3d20 msgr2=0x7f311c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:43.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.052+0000 7f313a43a640 1 --2- 172.21.15.52:0/4216884275 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f311c0a3d20 0x7f311c0a61e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3124009890 tx=0x7f3124008040 comp rx=0 tx=0).stop 2024-04-02T14:04:43.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.052+0000 7f313a43a640 1 -- 172.21.15.52:0/4216884275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3134101330 msgr2=0x7f3134197850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:43.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.052+0000 7f313a43a640 1 --2- 172.21.15.52:0/4216884275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3134101330 0x7f3134197850 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f3128009230 tx=0x7f312802cc70 comp rx=0 tx=0).stop 2024-04-02T14:04:43.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.053+0000 7f313a43a640 1 -- 172.21.15.52:0/4216884275 shutdown_connections 2024-04-02T14:04:43.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.053+0000 7f313a43a640 1 --2- 172.21.15.52:0/4216884275 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7f311c0a3d20 0x7f311c0a61e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:43.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.053+0000 7f313a43a640 1 --2- 172.21.15.52:0/4216884275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3134106980 0x7f313419bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:43.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.053+0000 7f313a43a640 1 --2- 172.21.15.52:0/4216884275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3134101330 0x7f3134197850 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:43.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.053+0000 7f313a43a640 1 --2- 172.21.15.52:0/4216884275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3134100970 0x7f3134197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:43.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.053+0000 7f313a43a640 1 -- 172.21.15.52:0/4216884275 >> 172.21.15.52:0/4216884275 conn(0x7f3134077b00 msgr2=0x7f31341030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:43.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.053+0000 7f313a43a640 1 -- 172.21.15.52:0/4216884275 shutdown_connections 2024-04-02T14:04:43.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.053+0000 7f313a43a640 1 -- 172.21.15.52:0/4216884275 wait complete. 2024-04-02T14:04:43.072 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-02T14:04:43.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.182+0000 7ff16d488640 1 -- 172.21.15.52:0/390719009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff168102d70 msgr2=0x7ff168103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:43.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.182+0000 7ff16d488640 1 --2- 172.21.15.52:0/390719009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff168102d70 0x7ff168103150 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7ff160009900 tx=0x7ff16002bcd0 comp rx=0 tx=0).stop 2024-04-02T14:04:43.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.182+0000 7ff16d488640 1 -- 172.21.15.52:0/390719009 shutdown_connections 2024-04-02T14:04:43.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.182+0000 7ff16d488640 1 --2- 172.21.15.52:0/390719009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff16810c140 0x7ff16810e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:43.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.182+0000 7ff16d488640 1 --2- 172.21.15.52:0/390719009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff168103690 0x7ff16810bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:43.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.182+0000 7ff16d488640 1 --2- 172.21.15.52:0/390719009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff168102d70 0x7ff168103150 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:43.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.182+0000 7ff16d488640 1 -- 172.21.15.52:0/390719009 >> 172.21.15.52:0/390719009 conn(0x7ff168077b60 msgr2=0x7ff168077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:43.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.182+0000 7ff16d488640 1 -- 172.21.15.52:0/390719009 shutdown_connections 2024-04-02T14:04:43.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.182+0000 7ff16d488640 1 -- 172.21.15.52:0/390719009 wait complete. 2024-04-02T14:04:43.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.182+0000 7ff16d488640 1 Processor -- start 2024-04-02T14:04:43.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.183+0000 7ff16d488640 1 -- start start 2024-04-02T14:04:43.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.183+0000 7ff16d488640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff168102d70 0x7ff1681983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:43.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.183+0000 7ff16d488640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff168103690 0x7ff168198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:43.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.183+0000 7ff16d488640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff16810c140 0x7ff16819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:43.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.183+0000 7ff16d488640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff168111f50 con 0x7ff168102d70 2024-04-02T14:04:43.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.183+0000 7ff16d488640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff168111dd0 con 0x7ff16810c140 2024-04-02T14:04:43.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.183+0000 7ff16d488640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681120d0 con 0x7ff168103690 2024-04-02T14:04:43.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.183+0000 7ff166ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff168102d70 0x7ff1681983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:43.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.183+0000 7ff166ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff168102d70 0x7ff1681983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48458/0 (socket says 172.21.15.52:48458) 2024-04-02T14:04:43.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.183+0000 7ff166ffd640 1 -- 172.21.15.52:0/2516761601 learned_addr learned my addr 172.21.15.52:0/2516761601 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-04-02T14:04:43.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.183+0000 7ff1667fc640 1 --2- 172.21.15.52:0/2516761601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff168103690 0x7ff168198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:43.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.183+0000 7ff1677fe640 1 --2- 172.21.15.52:0/2516761601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff16810c140 0x7ff16819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:43.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.183+0000 7ff166ffd640 1 -- 172.21.15.52:0/2516761601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff168103690 msgr2=0x7ff168198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:43.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.183+0000 7ff166ffd640 1 --2- 172.21.15.52:0/2516761601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff168103690 0x7ff168198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:43.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.183+0000 7ff166ffd640 1 -- 172.21.15.52:0/2516761601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff16810c140 msgr2=0x7ff16819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:43.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.183+0000 7ff166ffd640 1 --2- 172.21.15.52:0/2516761601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff16810c140 0x7ff16819cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:43.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.183+0000 7ff166ffd640 1 -- 172.21.15.52:0/2516761601 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff16819d3c0 con 0x7ff168102d70 2024-04-02T14:04:43.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.183+0000 7ff166ffd640 1 --2- 172.21.15.52:0/2516761601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff168102d70 0x7ff1681983d0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7ff16002efd0 tx=0x7ff16002ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:43.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.184+0000 7ff14ffff640 1 -- 172.21.15.52:0/2516761601 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff160041070 con 0x7ff168102d70 2024-04-02T14:04:43.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.184+0000 7ff14ffff640 1 -- 172.21.15.52:0/2516761601 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1600054e0 con 0x7ff168102d70 2024-04-02T14:04:43.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.184+0000 7ff16d488640 1 -- 172.21.15.52:0/2516761601 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff16819d650 con 0x7ff168102d70 2024-04-02T14:04:43.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.184+0000 7ff14ffff640 1 -- 172.21.15.52:0/2516761601 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff160034440 con 0x7ff168102d70 2024-04-02T14:04:43.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.184+0000 7ff16d488640 1 -- 172.21.15.52:0/2516761601 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1681a4f40 con 0x7ff168102d70 2024-04-02T14:04:43.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.185+0000 7ff16d488640 1 -- 172.21.15.52:0/2516761601 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff168068d40 con 0x7ff168102d70 2024-04-02T14:04:43.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.186+0000 7ff14ffff640 1 -- 172.21.15.52:0/2516761601 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff160044070 con 0x7ff168102d70 2024-04-02T14:04:43.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.186+0000 7ff14ffff640 1 --2- 172.21.15.52:0/2516761601 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff1500a3d20 0x7ff1500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T14:04:43.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.187+0000 7ff1667fc640 1 --2- 172.21.15.52:0/2516761601 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff1500a3d20 0x7ff1500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T14:04:43.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.187+0000 7ff14ffff640 1 -- 172.21.15.52:0/2516761601 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8754+0+0 (secure 0 0 0) 0x7ff1600f2760 con 0x7ff168102d70 2024-04-02T14:04:43.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.187+0000 7ff1667fc640 1 --2- 172.21.15.52:0/2516761601 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff1500a3d20 0x7ff1500a61e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff16810ddf0 tx=0x7ff15c008040 comp rx=0 tx=0).ready entity=mgr.25160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T14:04:43.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.190+0000 7ff14ffff640 1 -- 172.21.15.52:0/2516761601 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff1600bee20 con 0x7ff168102d70 2024-04-02T14:04:43.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:43 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/1381069808' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T14:04:43.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:43 smithi052 ceph-mon[32104]: osdmap e105: 12 total, 12 up, 12 in 2024-04-02T14:04:43.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:43 smithi052 ceph-mon[32104]: pgmap v351: 97 pgs: 97 active+clean; 26 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:04:43.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:43 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/4216884275' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T14:04:43.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:43.378+0000 7ff16d488640 1 -- 172.21.15.52:0/2516761601 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff1681a5170 con 0x7ff168102d70 2024-04-02T14:04:43.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:43 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/1381069808' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T14:04:43.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:43 smithi088 ceph-mon[36528]: osdmap e105: 12 total, 12 up, 12 in 2024-04-02T14:04:43.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:43 smithi088 ceph-mon[36528]: pgmap v351: 97 pgs: 97 active+clean; 26 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:04:43.421 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:43 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/4216884275' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T14:04:43.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:43 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/1381069808' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T14:04:43.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:43 smithi189 ceph-mon[36683]: osdmap e105: 12 total, 12 up, 12 in 2024-04-02T14:04:43.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:43 smithi189 ceph-mon[36683]: pgmap v351: 97 pgs: 97 active+clean; 26 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T14:04:43.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:43 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/4216884275' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T14:04:44.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:44.054+0000 7ff14ffff640 1 -- 172.21.15.52:0/2516761601 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v107) ==== 178+0+0 (secure 0 0 0) 0x7ff1600be720 con 0x7ff168102d70 2024-04-02T14:04:44.058 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data_ec' removed 2024-04-02T14:04:44.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:44.058+0000 7ff16d488640 1 -- 172.21.15.52:0/2516761601 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff1500a3d20 msgr2=0x7ff1500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:44.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:44.058+0000 7ff16d488640 1 --2- 172.21.15.52:0/2516761601 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff1500a3d20 0x7ff1500a61e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff16810ddf0 tx=0x7ff15c008040 comp rx=0 tx=0).stop 2024-04-02T14:04:44.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:44.058+0000 7ff16d488640 1 -- 172.21.15.52:0/2516761601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff168102d70 msgr2=0x7ff1681983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T14:04:44.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:44.058+0000 7ff16d488640 1 --2- 172.21.15.52:0/2516761601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff168102d70 0x7ff1681983d0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7ff16002efd0 tx=0x7ff16002ce50 comp rx=0 tx=0).stop 2024-04-02T14:04:44.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:44.058+0000 7ff16d488640 1 -- 172.21.15.52:0/2516761601 shutdown_connections 2024-04-02T14:04:44.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:44.058+0000 7ff16d488640 1 --2- 172.21.15.52:0/2516761601 >> [v2:172.21.15.52:6800/1019901931,v1:172.21.15.52:6801/1019901931] conn(0x7ff1500a3d20 0x7ff1500a61e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:44.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:44.058+0000 7ff16d488640 1 --2- 172.21.15.52:0/2516761601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff16810c140 0x7ff16819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:44.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:44.058+0000 7ff16d488640 1 --2- 172.21.15.52:0/2516761601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff168103690 0x7ff168198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:44.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:44.058+0000 7ff16d488640 1 --2- 172.21.15.52:0/2516761601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff168102d70 0x7ff1681983d0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T14:04:44.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:44.058+0000 7ff16d488640 1 -- 172.21.15.52:0/2516761601 >> 172.21.15.52:0/2516761601 conn(0x7ff168077b60 msgr2=0x7ff1680fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T14:04:44.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:44.058+0000 7ff16d488640 1 -- 172.21.15.52:0/2516761601 shutdown_connections 2024-04-02T14:04:44.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-02T14:04:44.058+0000 7ff16d488640 1 -- 172.21.15.52:0/2516761601 wait complete. 2024-04-02T14:04:44.079 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T14:04:44.121 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T14:04:44.167 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T14:04:44.198 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-02T14:04:44.199 DEBUG:teuthology.orchestra.run.smithi052:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-02T14:04:44.214 DEBUG:teuthology.orchestra.run.smithi088:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-02T14:04:44.229 DEBUG:teuthology.orchestra.run.smithi189:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-02T14:04:44.253 INFO:tasks.cephadm:Stopping all daemons... 2024-04-02T14:04:44.254 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-02T14:04:44.254 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mon.a 2024-04-02T14:04:44.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:44 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/4216884275' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T14:04:44.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:44 smithi052 ceph-mon[32104]: osdmap e106: 12 total, 12 up, 12 in 2024-04-02T14:04:44.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:44 smithi052 ceph-mon[32104]: from='client.? 172.21.15.52:0/2516761601' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T14:04:44.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:44 smithi052 systemd[1]: Stopping Ceph mon.a for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:04:44.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:44 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/4216884275' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T14:04:44.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:44 smithi088 ceph-mon[36528]: osdmap e106: 12 total, 12 up, 12 in 2024-04-02T14:04:44.420 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:44 smithi088 ceph-mon[36528]: from='client.? 172.21.15.52:0/2516761601' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T14:04:44.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:44 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/4216884275' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T14:04:44.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:44 smithi189 ceph-mon[36683]: osdmap e106: 12 total, 12 up, 12 in 2024-04-02T14:04:44.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:44 smithi189 ceph-mon[36683]: from='client.? 172.21.15.52:0/2516761601' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T14:04:44.969 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:44 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-a[32100]: 2024-04-02T14:04:44.597+0000 7f1bfc782700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:04:44.970 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:44 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-a[32100]: 2024-04-02T14:04:44.597+0000 7f1bfc782700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-02T14:04:44.970 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:44 smithi052 podman[107342]: 2024-04-02 14:04:44.658800145 +0000 UTC m=+0.204597308 container died bcb19f26fc2831378bb2651f24afeb62329f351e7576f203ccbddca409ea7310 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-a, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-02T14:04:45.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:45 smithi052 podman[107342]: 2024-04-02 14:04:45.065228022 +0000 UTC m=+0.611025167 container cleanup bcb19f26fc2831378bb2651f24afeb62329f351e7576f203ccbddca409ea7310 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T14:04:45.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Apr 02 14:04:45 smithi052 bash[107342]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-a 2024-04-02T14:04:45.505 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mon.a.service' 2024-04-02T14:04:45.560 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:04:45.560 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-02T14:04:45.561 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-02T14:04:45.561 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mon.b 2024-04-02T14:04:45.869 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:45 smithi088 systemd[1]: Stopping Ceph mon.b for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:04:46.170 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:45 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-b[36524]: 2024-04-02T14:04:45.864+0000 7fba07f2f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:04:46.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:45 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-b[36524]: 2024-04-02T14:04:45.864+0000 7fba07f2f700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-02T14:04:46.171 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:45 smithi088 podman[72199]: 2024-04-02 14:04:45.957838797 +0000 UTC m=+0.220327723 container died 9e66ff83e3f91ec2ad55240248270b19efa411f900cf66bcffde5c481ea87ea5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-04-02T14:04:46.613 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:46 smithi088 podman[72199]: 2024-04-02 14:04:46.316442674 +0000 UTC m=+0.578931587 container cleanup 9e66ff83e3f91ec2ad55240248270b19efa411f900cf66bcffde5c481ea87ea5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-b, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T14:04:46.613 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 02 14:04:46 smithi088 bash[72199]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-b 2024-04-02T14:04:46.748 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mon.b.service' 2024-04-02T14:04:46.792 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:04:46.793 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-02T14:04:46.793 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-02T14:04:46.793 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mon.c 2024-04-02T14:04:47.107 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:46 smithi189 systemd[1]: Stopping Ceph mon.c for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:04:47.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:47 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-c[36664]: 2024-04-02T14:04:47.103+0000 7f60b3fd1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:04:47.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:47 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-c[36664]: 2024-04-02T14:04:47.103+0000 7f60b3fd1700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-02T14:04:47.539 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:47 smithi189 podman[72974]: 2024-04-02 14:04:47.318219779 +0000 UTC m=+0.333271099 container died 7cca3de296374be9c1ea7f654e4fe920e7cf944418d590cf03f1dc20beecdbaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-c, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-02T14:04:47.866 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:47 smithi189 podman[72974]: 2024-04-02 14:04:47.695866165 +0000 UTC m=+0.710917475 container cleanup 7cca3de296374be9c1ea7f654e4fe920e7cf944418d590cf03f1dc20beecdbaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-02T14:04:47.866 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:47 smithi189 bash[72974]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-c 2024-04-02T14:04:48.261 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 02 14:04:48 smithi189 podman[72988]: 2024-04-02 14:04:48.064875647 +0000 UTC m=+0.743923956 container remove 7cca3de296374be9c1ea7f654e4fe920e7cf944418d590cf03f1dc20beecdbaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mon-c, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=) 2024-04-02T14:04:48.284 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mon.c.service' 2024-04-02T14:04:48.367 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:04:48.367 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-02T14:04:48.367 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-02T14:04:48.368 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mgr.x 2024-04-02T14:04:48.682 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 14:04:48 smithi052 systemd[1]: Stopping Ceph mgr.x for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:04:49.024 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 14:04:48 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T14:04:48.677+0000 7ff8d5d27700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:04:49.025 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 14:04:48 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x[32338]: 2024-04-02T14:04:48.677+0000 7ff8d5d27700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-02T14:04:49.025 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 14:04:48 smithi052 podman[107552]: 2024-04-02 14:04:48.709412356 +0000 UTC m=+0.154814032 container died c88f5d3e94020556dac8920fd178408575b666be184f1106a35aff0e380c6a39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326) 2024-04-02T14:04:49.340 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 14:04:49 smithi052 podman[107552]: 2024-04-02 14:04:49.121400184 +0000 UTC m=+0.566801849 container cleanup c88f5d3e94020556dac8920fd178408575b666be184f1106a35aff0e380c6a39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-02T14:04:49.340 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Apr 02 14:04:49 smithi052 bash[107552]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-x 2024-04-02T14:04:49.481 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mgr.x.service' 2024-04-02T14:04:49.520 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:04:49.520 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-02T14:04:49.520 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-02T14:04:49.521 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mgr.y 2024-04-02T14:04:49.837 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 14:04:49 smithi088 systemd[1]: Stopping Ceph mgr.y for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:04:50.117 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 14:04:49 smithi088 podman[72320]: 2024-04-02 14:04:49.833550113 +0000 UTC m=+0.121948248 container died 671f18422f67908b2704866b3611d4dae0df462e1c0469ad1ad1b20f461c4406 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240326) 2024-04-02T14:04:50.420 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 14:04:50 smithi088 podman[72320]: 2024-04-02 14:04:50.213255938 +0000 UTC m=+0.501654071 container cleanup 671f18422f67908b2704866b3611d4dae0df462e1c0469ad1ad1b20f461c4406 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-02T14:04:50.420 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 02 14:04:50 smithi088 bash[72320]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-y 2024-04-02T14:04:50.595 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mgr.y.service' 2024-04-02T14:04:50.632 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:04:50.632 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-02T14:04:50.632 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-02T14:04:50.633 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mgr.z 2024-04-02T14:04:50.967 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 14:04:50 smithi189 systemd[1]: Stopping Ceph mgr.z for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:04:51.257 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 14:04:50 smithi189 podman[73097]: 2024-04-02 14:04:50.96402922 +0000 UTC m=+0.141036894 container died a5b65e91ec3698da1475d13340fb6028f782eb1945cdcb6836450e95e458e194 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_CLEAN=True) 2024-04-02T14:04:51.538 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 14:04:51 smithi189 podman[73097]: 2024-04-02 14:04:51.344710708 +0000 UTC m=+0.521718374 container cleanup a5b65e91ec3698da1475d13340fb6028f782eb1945cdcb6836450e95e458e194 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, io.buildah.version=1.33.5) 2024-04-02T14:04:51.539 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 02 14:04:51 smithi189 bash[73097]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mgr-z 2024-04-02T14:04:51.771 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mgr.z.service' 2024-04-02T14:04:51.809 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:04:51.809 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-02T14:04:51.810 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-02T14:04:51.810 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.0 2024-04-02T14:04:52.090 INFO:journalctl@ceph.osd.0.smithi052.stdout:Apr 02 14:04:51 smithi052 systemd[1]: Stopping Ceph osd.0 for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:04:52.590 INFO:journalctl@ceph.osd.0.smithi052.stdout:Apr 02 14:04:52 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-0[44682]: 2024-04-02T14:04:52.126+0000 7fdf43a53700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:04:52.591 INFO:journalctl@ceph.osd.0.smithi052.stdout:Apr 02 14:04:52 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-0[44682]: 2024-04-02T14:04:52.126+0000 7fdf43a53700 -1 osd.0 107 *** Got signal Terminated *** 2024-04-02T14:04:52.591 INFO:journalctl@ceph.osd.0.smithi052.stdout:Apr 02 14:04:52 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-0[44682]: 2024-04-02T14:04:52.126+0000 7fdf43a53700 -1 osd.0 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T14:04:57.966 INFO:journalctl@ceph.osd.0.smithi052.stdout:Apr 02 14:04:57 smithi052 podman[107673]: 2024-04-02 14:04:57.657808387 +0000 UTC m=+5.656884356 container died ca8f74146fef6b809701d3519c57e9b9f62cbf4010a637221c01542d19c240bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T14:04:58.249 INFO:journalctl@ceph.osd.0.smithi052.stdout:Apr 02 14:04:58 smithi052 podman[107673]: 2024-04-02 14:04:58.061002671 +0000 UTC m=+6.060078649 container cleanup ca8f74146fef6b809701d3519c57e9b9f62cbf4010a637221c01542d19c240bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T14:04:58.249 INFO:journalctl@ceph.osd.0.smithi052.stdout:Apr 02 14:04:58 smithi052 bash[107673]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-0 2024-04-02T14:04:58.590 INFO:journalctl@ceph.osd.0.smithi052.stdout:Apr 02 14:04:58 smithi052 podman[107690]: 2024-04-02 14:04:58.24593764 +0000 UTC m=+0.588869867 container remove ca8f74146fef6b809701d3519c57e9b9f62cbf4010a637221c01542d19c240bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T14:04:58.986 INFO:journalctl@ceph.osd.0.smithi052.stdout:Apr 02 14:04:58 smithi052 podman[107783]: 2024-04-02 14:04:58.576765253 +0000 UTC m=+0.017607083 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:04:58.987 INFO:journalctl@ceph.osd.0.smithi052.stdout:Apr 02 14:04:58 smithi052 podman[107783]: 2024-04-02 14:04:58.697073022 +0000 UTC m=+0.137914826 container create a4393d77699fc86701bb0f9391d712021157128db5c51c0008c3a689e9cd6785 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-0-deactivate, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-02T14:04:58.987 INFO:journalctl@ceph.osd.0.smithi052.stdout:Apr 02 14:04:58 smithi052 podman[107783]: 2024-04-02 14:04:58.98324589 +0000 UTC m=+0.424087694 container init a4393d77699fc86701bb0f9391d712021157128db5c51c0008c3a689e9cd6785 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-02T14:04:59.287 INFO:journalctl@ceph.osd.0.smithi052.stdout:Apr 02 14:04:58 smithi052 podman[107783]: 2024-04-02 14:04:58.98659154 +0000 UTC m=+0.427433352 container start a4393d77699fc86701bb0f9391d712021157128db5c51c0008c3a689e9cd6785 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-0-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326) 2024-04-02T14:04:59.288 INFO:journalctl@ceph.osd.0.smithi052.stdout:Apr 02 14:04:59 smithi052 podman[107783]: 2024-04-02 14:04:59.066664971 +0000 UTC m=+0.507506775 container attach a4393d77699fc86701bb0f9391d712021157128db5c51c0008c3a689e9cd6785 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-0-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-02T14:04:59.568 INFO:journalctl@ceph.osd.0.smithi052.stdout:Apr 02 14:04:59 smithi052 podman[107783]: 2024-04-02 14:04:59.284698591 +0000 UTC m=+0.725540408 container died a4393d77699fc86701bb0f9391d712021157128db5c51c0008c3a689e9cd6785 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5) 2024-04-02T14:04:59.998 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.0.service' 2024-04-02T14:05:00.039 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:05:00.039 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-02T14:05:00.040 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-02T14:05:00.040 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.1 2024-04-02T14:05:00.345 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 02 14:05:00 smithi088 systemd[1]: Stopping Ceph osd.1 for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:05:00.670 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 02 14:05:00 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-1[41137]: 2024-04-02T14:05:00.341+0000 7fcbd594b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:05:00.670 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 02 14:05:00 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-1[41137]: 2024-04-02T14:05:00.341+0000 7fcbd594b700 -1 osd.1 107 *** Got signal Terminated *** 2024-04-02T14:05:00.671 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 02 14:05:00 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-1[41137]: 2024-04-02T14:05:00.341+0000 7fcbd594b700 -1 osd.1 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T14:05:06.112 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 02 14:05:05 smithi088 podman[72445]: 2024-04-02 14:05:05.811528158 +0000 UTC m=+5.579457304 container died a59bbc6312dd552d0d1d22082836f02e6c4c7adbdf0894735e4abcd67cd52bb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-02T14:05:06.420 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 02 14:05:06 smithi088 podman[72445]: 2024-04-02 14:05:06.208633387 +0000 UTC m=+5.976562531 container cleanup a59bbc6312dd552d0d1d22082836f02e6c4c7adbdf0894735e4abcd67cd52bb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-1, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-02T14:05:06.421 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 02 14:05:06 smithi088 bash[72445]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-1 2024-04-02T14:05:06.775 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 02 14:05:06 smithi088 podman[72461]: 2024-04-02 14:05:06.468733773 +0000 UTC m=+0.656897651 container remove a59bbc6312dd552d0d1d22082836f02e6c4c7adbdf0894735e4abcd67cd52bb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-1, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-02T14:05:07.025 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 02 14:05:06 smithi088 podman[72553]: 2024-04-02 14:05:06.674964991 +0000 UTC m=+0.017532598 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:05:07.026 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 02 14:05:06 smithi088 podman[72553]: 2024-04-02 14:05:06.811459499 +0000 UTC m=+0.154027093 container create 2bd346f5dc1cd69d9e185db7ee0e776c28dbdd7bc73f16aa24677361048140e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-1-deactivate, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T14:05:07.316 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 02 14:05:07 smithi088 podman[72553]: 2024-04-02 14:05:07.02255279 +0000 UTC m=+0.365120393 container init 2bd346f5dc1cd69d9e185db7ee0e776c28dbdd7bc73f16aa24677361048140e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-02T14:05:07.316 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 02 14:05:07 smithi088 podman[72553]: 2024-04-02 14:05:07.026062676 +0000 UTC m=+0.368630274 container start 2bd346f5dc1cd69d9e185db7ee0e776c28dbdd7bc73f16aa24677361048140e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T14:05:07.317 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 02 14:05:07 smithi088 podman[72553]: 2024-04-02 14:05:07.122651423 +0000 UTC m=+0.465219048 container attach 2bd346f5dc1cd69d9e185db7ee0e776c28dbdd7bc73f16aa24677361048140e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-1-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T14:05:07.621 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 02 14:05:07 smithi088 podman[72553]: 2024-04-02 14:05:07.313406855 +0000 UTC m=+0.655974474 container died 2bd346f5dc1cd69d9e185db7ee0e776c28dbdd7bc73f16aa24677361048140e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-02T14:05:07.961 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.1.service' 2024-04-02T14:05:08.003 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:05:08.003 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-02T14:05:08.004 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-02T14:05:08.004 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.2 2024-04-02T14:05:08.288 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 02 14:05:08 smithi189 systemd[1]: Stopping Ceph osd.2 for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:05:08.789 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 02 14:05:08 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-2[41830]: 2024-04-02T14:05:08.441+0000 7f89cd364700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:05:08.789 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 02 14:05:08 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-2[41830]: 2024-04-02T14:05:08.441+0000 7f89cd364700 -1 osd.2 107 *** Got signal Terminated *** 2024-04-02T14:05:08.789 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 02 14:05:08 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-2[41830]: 2024-04-02T14:05:08.441+0000 7f89cd364700 -1 osd.2 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T14:05:14.289 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 02 14:05:13 smithi189 podman[73224]: 2024-04-02 14:05:13.967938427 +0000 UTC m=+5.766062729 container died 1825481ddb9787f7474a984f835f477edbf728dbd8da618e056bf96fc861d802 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-2, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-02T14:05:14.788 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 02 14:05:14 smithi189 podman[73224]: 2024-04-02 14:05:14.291719934 +0000 UTC m=+6.089844220 container cleanup 1825481ddb9787f7474a984f835f477edbf728dbd8da618e056bf96fc861d802 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-02T14:05:14.789 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 02 14:05:14 smithi189 bash[73224]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-2 2024-04-02T14:05:14.789 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 02 14:05:14 smithi189 podman[73253]: 2024-04-02 14:05:14.519261202 +0000 UTC m=+0.548710918 container remove 1825481ddb9787f7474a984f835f477edbf728dbd8da618e056bf96fc861d802 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T14:05:15.259 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 02 14:05:14 smithi189 podman[73344]: 2024-04-02 14:05:14.856780877 +0000 UTC m=+0.018042798 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:05:15.259 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 02 14:05:14 smithi189 podman[73344]: 2024-04-02 14:05:14.995546747 +0000 UTC m=+0.156808666 container create ef329d3a3d8fc3d69aa91f2877a3172cc35aef9d5fa2bae19ddc3034dcd3fd59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326) 2024-04-02T14:05:15.539 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 02 14:05:15 smithi189 podman[73344]: 2024-04-02 14:05:15.256398753 +0000 UTC m=+0.417660660 container init ef329d3a3d8fc3d69aa91f2877a3172cc35aef9d5fa2bae19ddc3034dcd3fd59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-2-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-02T14:05:15.540 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 02 14:05:15 smithi189 podman[73344]: 2024-04-02 14:05:15.2596048 +0000 UTC m=+0.420866707 container start ef329d3a3d8fc3d69aa91f2877a3172cc35aef9d5fa2bae19ddc3034dcd3fd59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-02T14:05:15.540 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 02 14:05:15 smithi189 podman[73344]: 2024-04-02 14:05:15.35652744 +0000 UTC m=+0.517789360 container attach ef329d3a3d8fc3d69aa91f2877a3172cc35aef9d5fa2bae19ddc3034dcd3fd59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-2-deactivate, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-02T14:05:15.913 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 02 14:05:15 smithi189 podman[73344]: 2024-04-02 14:05:15.548068606 +0000 UTC m=+0.709330514 container died ef329d3a3d8fc3d69aa91f2877a3172cc35aef9d5fa2bae19ddc3034dcd3fd59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-2-deactivate, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5) 2024-04-02T14:05:16.287 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.2.service' 2024-04-02T14:05:16.332 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:05:16.332 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-02T14:05:16.333 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-02T14:05:16.333 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.3 2024-04-02T14:05:16.661 INFO:journalctl@ceph.osd.3.smithi052.stdout:Apr 02 14:05:16 smithi052 systemd[1]: Stopping Ceph osd.3 for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:05:17.090 INFO:journalctl@ceph.osd.3.smithi052.stdout:Apr 02 14:05:16 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-3[51356]: 2024-04-02T14:05:16.657+0000 7f3830323700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:05:17.090 INFO:journalctl@ceph.osd.3.smithi052.stdout:Apr 02 14:05:16 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-3[51356]: 2024-04-02T14:05:16.657+0000 7f3830323700 -1 osd.3 107 *** Got signal Terminated *** 2024-04-02T14:05:17.091 INFO:journalctl@ceph.osd.3.smithi052.stdout:Apr 02 14:05:16 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-3[51356]: 2024-04-02T14:05:16.657+0000 7f3830323700 -1 osd.3 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T14:05:18.489 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:18 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:18.135+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.076720+0000 front 2024-04-02T14:04:53.076994+0000 (oldest deadline 2024-04-02T14:05:17.776793+0000) 2024-04-02T14:05:18.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:18 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:18.486+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:19.517 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:19 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:19.093+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.076720+0000 front 2024-04-02T14:04:53.076994+0000 (oldest deadline 2024-04-02T14:05:17.776793+0000) 2024-04-02T14:05:19.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:19 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:19.513+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:20.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:19 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:19.894+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:20.538 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:20 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:20.060+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.076720+0000 front 2024-04-02T14:04:53.076994+0000 (oldest deadline 2024-04-02T14:05:17.776793+0000) 2024-04-02T14:05:20.590 INFO:journalctl@ceph.osd.3.smithi052.stdout:Apr 02 14:05:20 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-3[51356]: 2024-04-02T14:05:20.307+0000 7f382890f700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:54.741262+0000 front 2024-04-02T14:04:54.741303+0000 (oldest deadline 2024-04-02T14:05:20.041488+0000) 2024-04-02T14:05:21.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:20 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:20.557+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:21.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:20 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:20.865+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:21.286 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:20 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:20.905+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:21.521 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:21 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:21.089+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.076720+0000 front 2024-04-02T14:04:53.076994+0000 (oldest deadline 2024-04-02T14:05:17.776793+0000) 2024-04-02T14:05:21.590 INFO:journalctl@ceph.osd.3.smithi052.stdout:Apr 02 14:05:21 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-3[51356]: 2024-04-02T14:05:21.282+0000 7f382890f700 -1 osd.3 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:54.741262+0000 front 2024-04-02T14:04:54.741303+0000 (oldest deadline 2024-04-02T14:05:20.041488+0000) 2024-04-02T14:05:21.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:21 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:21.518+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:22.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:21 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:21.848+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:22.270 INFO:journalctl@ceph.osd.3.smithi052.stdout:Apr 02 14:05:21 smithi052 podman[107900]: 2024-04-02 14:05:21.989823717 +0000 UTC m=+5.461819862 container died dc6b29dc4c94f0e52b6bda5ff86ae271fa1f064fd1618452c0d60d3c67335bd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-3, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-02T14:05:22.271 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:21 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:21.900+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:22.502 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:22 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:22.113+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.076720+0000 front 2024-04-02T14:04:53.076994+0000 (oldest deadline 2024-04-02T14:05:17.776793+0000) 2024-04-02T14:05:22.590 INFO:journalctl@ceph.osd.3.smithi052.stdout:Apr 02 14:05:22 smithi052 podman[107900]: 2024-04-02 14:05:22.374630881 +0000 UTC m=+5.846627032 container cleanup dc6b29dc4c94f0e52b6bda5ff86ae271fa1f064fd1618452c0d60d3c67335bd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-02T14:05:22.590 INFO:journalctl@ceph.osd.3.smithi052.stdout:Apr 02 14:05:22 smithi052 bash[107900]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-3 2024-04-02T14:05:22.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:22 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:22.498+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:22.904 INFO:journalctl@ceph.osd.3.smithi052.stdout:Apr 02 14:05:22 smithi052 podman[107918]: 2024-04-02 14:05:22.626202159 +0000 UTC m=+0.630594009 container remove dc6b29dc4c94f0e52b6bda5ff86ae271fa1f064fd1618452c0d60d3c67335bd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-02T14:05:23.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:22 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:22.856+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:23.249 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:22 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:22.900+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:23.250 INFO:journalctl@ceph.osd.3.smithi052.stdout:Apr 02 14:05:22 smithi052 podman[108010]: 2024-04-02 14:05:22.855524758 +0000 UTC m=+0.017674792 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:05:23.250 INFO:journalctl@ceph.osd.3.smithi052.stdout:Apr 02 14:05:22 smithi052 podman[108010]: 2024-04-02 14:05:22.977138504 +0000 UTC m=+0.139288522 container create de2c5175d675687203786673353bb969a89b38d27a7a6ce363058d119b1abc3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-02T14:05:23.250 INFO:journalctl@ceph.osd.3.smithi052.stdout:Apr 02 14:05:23 smithi052 podman[108010]: 2024-04-02 14:05:23.246595783 +0000 UTC m=+0.408745817 container init de2c5175d675687203786673353bb969a89b38d27a7a6ce363058d119b1abc3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-3-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-02T14:05:23.455 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:23 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:23.140+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.076720+0000 front 2024-04-02T14:04:53.076994+0000 (oldest deadline 2024-04-02T14:05:17.776793+0000) 2024-04-02T14:05:23.557 INFO:journalctl@ceph.osd.3.smithi052.stdout:Apr 02 14:05:23 smithi052 podman[108010]: 2024-04-02 14:05:23.249898395 +0000 UTC m=+0.412048422 container start de2c5175d675687203786673353bb969a89b38d27a7a6ce363058d119b1abc3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-3-deactivate, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T14:05:23.557 INFO:journalctl@ceph.osd.3.smithi052.stdout:Apr 02 14:05:23 smithi052 podman[108010]: 2024-04-02 14:05:23.330139573 +0000 UTC m=+0.492289607 container attach de2c5175d675687203786673353bb969a89b38d27a7a6ce363058d119b1abc3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-04-02T14:05:23.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:23 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:23.452+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:23.840 INFO:journalctl@ceph.osd.3.smithi052.stdout:Apr 02 14:05:23 smithi052 podman[108010]: 2024-04-02 14:05:23.554038766 +0000 UTC m=+0.716188805 container died de2c5175d675687203786673353bb969a89b38d27a7a6ce363058d119b1abc3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:05:24.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:23 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:23.875+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:24.218 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:23 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:23.863+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:24.242 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.3.service' 2024-04-02T14:05:24.287 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:05:24.288 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-02T14:05:24.288 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-02T14:05:24.288 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.4 2024-04-02T14:05:24.498 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:24 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:24.145+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.076720+0000 front 2024-04-02T14:04:53.076994+0000 (oldest deadline 2024-04-02T14:05:17.776793+0000) 2024-04-02T14:05:24.590 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 02 14:05:24 smithi088 systemd[1]: Stopping Ceph osd.4 for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:05:24.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:24 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:24.495+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:24.849 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 02 14:05:24 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-4[47689]: 2024-04-02T14:05:24.586+0000 7f8655f14700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:05:24.850 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 02 14:05:24 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-4[47689]: 2024-04-02T14:05:24.586+0000 7f8655f14700 -1 osd.4 107 *** Got signal Terminated *** 2024-04-02T14:05:24.850 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 02 14:05:24 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-4[47689]: 2024-04-02T14:05:24.586+0000 7f8655f14700 -1 osd.4 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T14:05:25.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:24 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:24.845+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:25.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:24 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:24.863+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:25.496 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:25 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:25.155+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.076720+0000 front 2024-04-02T14:04:53.076994+0000 (oldest deadline 2024-04-02T14:05:17.776793+0000) 2024-04-02T14:05:25.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:25 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:25.492+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:26.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:25 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:25.852+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:26.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:25 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:25.861+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:26.469 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:26 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:26.129+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.076720+0000 front 2024-04-02T14:04:53.076994+0000 (oldest deadline 2024-04-02T14:05:17.776793+0000) 2024-04-02T14:05:26.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:26 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:26.465+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:27.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:26 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:26.833+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:27.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:26 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:26.886+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:27.500 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:27 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:27.163+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.076720+0000 front 2024-04-02T14:04:53.076994+0000 (oldest deadline 2024-04-02T14:05:17.776793+0000) 2024-04-02T14:05:27.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:27 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:27.496+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:28.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:27 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:27.849+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:28.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:27 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:27.871+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:28.520 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:28 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:28.165+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.076720+0000 front 2024-04-02T14:04:53.076994+0000 (oldest deadline 2024-04-02T14:05:17.776793+0000) 2024-04-02T14:05:28.520 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:28 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:28.165+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.478184+0000 front 2024-04-02T14:05:02.478035+0000 (oldest deadline 2024-04-02T14:05:27.178017+0000) 2024-04-02T14:05:28.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:28 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:28.515+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:29.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:28 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:28.877+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:29.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:28 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:28.877+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:29.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:28 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:28.879+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:29.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:28 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:28.879+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:29.517 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:29 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:29.174+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.076720+0000 front 2024-04-02T14:04:53.076994+0000 (oldest deadline 2024-04-02T14:05:17.776793+0000) 2024-04-02T14:05:29.518 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:29 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:29.174+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.478184+0000 front 2024-04-02T14:05:02.478035+0000 (oldest deadline 2024-04-02T14:05:27.178017+0000) 2024-04-02T14:05:29.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:29 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:29.588+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:29.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:29 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:29.513+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:30.116 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:29 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:29.927+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:30.116 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:29 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:29.927+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:30.117 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:29 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:29.745+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.678847+0000 front 2024-04-02T14:05:03.678915+0000 (oldest deadline 2024-04-02T14:05:28.978958+0000) 2024-04-02T14:05:30.340 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:29 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:29.878+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.508702+0000 front 2024-04-02T14:05:01.508650+0000 (oldest deadline 2024-04-02T14:05:29.709226+0000) 2024-04-02T14:05:30.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:29 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:29.874+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:30.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:29 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:29.874+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:30.420 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 02 14:05:30 smithi088 podman[72672]: 2024-04-02 14:05:30.112835895 +0000 UTC m=+5.632571716 container died 9fbce6bec9399fdbef27d3d5f73bcaf48f3fbf7a5885961364f0066298c3a289 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T14:05:30.538 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:30 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:30.202+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.076720+0000 front 2024-04-02T14:04:53.076994+0000 (oldest deadline 2024-04-02T14:05:17.776793+0000) 2024-04-02T14:05:30.539 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:30 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:30.202+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.478184+0000 front 2024-04-02T14:05:02.478035+0000 (oldest deadline 2024-04-02T14:05:27.178017+0000) 2024-04-02T14:05:30.733 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 02 14:05:30 smithi088 podman[72672]: 2024-04-02 14:05:30.439990444 +0000 UTC m=+5.959726254 container cleanup 9fbce6bec9399fdbef27d3d5f73bcaf48f3fbf7a5885961364f0066298c3a289 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:05:30.733 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 02 14:05:30 smithi088 bash[72672]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-4 2024-04-02T14:05:30.733 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 02 14:05:30 smithi088 podman[72689]: 2024-04-02 14:05:30.680123622 +0000 UTC m=+0.561792765 container remove 9fbce6bec9399fdbef27d3d5f73bcaf48f3fbf7a5885961364f0066298c3a289 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-04-02T14:05:31.038 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:30 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:30.588+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:31.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:30 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:30.560+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:31.100 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:30 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:30.728+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.678847+0000 front 2024-04-02T14:05:03.678915+0000 (oldest deadline 2024-04-02T14:05:28.978958+0000) 2024-04-02T14:05:31.101 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:30 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:30.965+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:31.101 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:30 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:30.965+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:31.340 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:30 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:30.890+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.508702+0000 front 2024-04-02T14:05:01.508650+0000 (oldest deadline 2024-04-02T14:05:29.709226+0000) 2024-04-02T14:05:31.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:30 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:30.849+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:31.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:30 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:30.849+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:31.353 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 02 14:05:31 smithi088 podman[72780]: 2024-04-02 14:05:31.097669412 +0000 UTC m=+0.106894913 container create f527137c6a3f40d0ae910e4aa152b76f9938f24d64823467d3f3e35ef3e79287 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=) 2024-04-02T14:05:31.353 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 02 14:05:31 smithi088 podman[72780]: 2024-04-02 14:05:31.008738815 +0000 UTC m=+0.017964327 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:05:31.354 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 02 14:05:31 smithi088 podman[72780]: 2024-04-02 14:05:31.350566129 +0000 UTC m=+0.359791637 container init f527137c6a3f40d0ae910e4aa152b76f9938f24d64823467d3f3e35ef3e79287 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, GIT_CLEAN=True) 2024-04-02T14:05:31.539 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:31 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:31.209+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.076720+0000 front 2024-04-02T14:04:53.076994+0000 (oldest deadline 2024-04-02T14:05:17.776793+0000) 2024-04-02T14:05:31.539 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:31 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:31.209+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.478184+0000 front 2024-04-02T14:05:02.478035+0000 (oldest deadline 2024-04-02T14:05:27.178017+0000) 2024-04-02T14:05:31.643 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 02 14:05:31 smithi088 podman[72780]: 2024-04-02 14:05:31.353740544 +0000 UTC m=+0.362966051 container start f527137c6a3f40d0ae910e4aa152b76f9938f24d64823467d3f3e35ef3e79287 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-4-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=) 2024-04-02T14:05:31.644 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 02 14:05:31 smithi088 podman[72780]: 2024-04-02 14:05:31.458856757 +0000 UTC m=+0.468082279 container attach f527137c6a3f40d0ae910e4aa152b76f9938f24d64823467d3f3e35ef3e79287 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-02T14:05:31.920 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 02 14:05:31 smithi088 podman[72780]: 2024-04-02 14:05:31.640898314 +0000 UTC m=+0.650123828 container died f527137c6a3f40d0ae910e4aa152b76f9938f24d64823467d3f3e35ef3e79287 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T14:05:31.920 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:31 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:31.685+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.678847+0000 front 2024-04-02T14:05:03.678915+0000 (oldest deadline 2024-04-02T14:05:28.978958+0000) 2024-04-02T14:05:32.038 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:31 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:31.620+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:32.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:31 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:31.555+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:32.258 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:31 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:31.967+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:32.258 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:31 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:31.967+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:32.285 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.4.service' 2024-04-02T14:05:32.325 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:05:32.325 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-02T14:05:32.325 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-02T14:05:32.326 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.5 2024-04-02T14:05:32.340 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:31 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:31.843+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.508702+0000 front 2024-04-02T14:05:01.508650+0000 (oldest deadline 2024-04-02T14:05:29.709226+0000) 2024-04-02T14:05:32.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:31 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:31.866+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:32.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:31 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:31.866+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:32.535 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:32 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:32.255+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.076720+0000 front 2024-04-02T14:04:53.076994+0000 (oldest deadline 2024-04-02T14:05:17.776793+0000) 2024-04-02T14:05:32.535 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:32 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:32.255+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.478184+0000 front 2024-04-02T14:05:02.478035+0000 (oldest deadline 2024-04-02T14:05:27.178017+0000) 2024-04-02T14:05:32.536 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:32 smithi189 systemd[1]: Stopping Ceph osd.5 for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:05:32.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:32 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:32.584+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:32.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:32 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:32.531+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:32.790 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:32 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:32.621+0000 7f2e42012700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:05:32.790 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:32 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:32.621+0000 7f2e42012700 -1 osd.5 107 *** Got signal Terminated *** 2024-04-02T14:05:32.790 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:32 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:32.621+0000 7f2e42012700 -1 osd.5 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T14:05:33.171 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:32 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:32.673+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.678847+0000 front 2024-04-02T14:05:03.678915+0000 (oldest deadline 2024-04-02T14:05:28.978958+0000) 2024-04-02T14:05:33.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:32 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:32.921+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:33.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:32 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:32.921+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:33.340 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:32 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:32.877+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.508702+0000 front 2024-04-02T14:05:01.508650+0000 (oldest deadline 2024-04-02T14:05:29.709226+0000) 2024-04-02T14:05:33.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:32 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:32.853+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:33.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:32 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:32.853+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:33.552 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:33 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:33.287+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.076720+0000 front 2024-04-02T14:04:53.076994+0000 (oldest deadline 2024-04-02T14:05:17.776793+0000) 2024-04-02T14:05:33.552 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:33 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:33.287+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.478184+0000 front 2024-04-02T14:05:02.478035+0000 (oldest deadline 2024-04-02T14:05:27.178017+0000) 2024-04-02T14:05:34.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:33 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:33.581+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:34.039 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:33 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:33.548+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:34.170 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:33 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:33.691+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.678847+0000 front 2024-04-02T14:05:03.678915+0000 (oldest deadline 2024-04-02T14:05:28.978958+0000) 2024-04-02T14:05:34.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:33 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:33.901+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:34.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:33 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:33.901+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:34.340 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:33 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:33.920+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.508702+0000 front 2024-04-02T14:05:01.508650+0000 (oldest deadline 2024-04-02T14:05:29.709226+0000) 2024-04-02T14:05:34.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:33 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:33.837+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:34.341 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:33 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:33.837+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:34.558 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:34 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:34.286+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.076720+0000 front 2024-04-02T14:04:53.076994+0000 (oldest deadline 2024-04-02T14:05:17.776793+0000) 2024-04-02T14:05:34.558 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:34 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:34.286+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.478184+0000 front 2024-04-02T14:05:02.478035+0000 (oldest deadline 2024-04-02T14:05:27.178017+0000) 2024-04-02T14:05:34.920 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:34 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:34.661+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.678847+0000 front 2024-04-02T14:05:03.678915+0000 (oldest deadline 2024-04-02T14:05:28.978958+0000) 2024-04-02T14:05:34.920 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:34 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:34.876+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:34.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:34 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:34.876+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:35.038 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:34 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:34.554+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:35.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:34 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:34.592+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:35.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:34 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:34.592+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:35.090 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:34 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:34.886+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.508702+0000 front 2024-04-02T14:05:01.508650+0000 (oldest deadline 2024-04-02T14:05:29.709226+0000) 2024-04-02T14:05:35.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:34 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:34.828+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:35.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:34 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:34.828+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:35.558 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:35 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:35.302+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.076720+0000 front 2024-04-02T14:04:53.076994+0000 (oldest deadline 2024-04-02T14:05:17.776793+0000) 2024-04-02T14:05:35.559 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:35 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:35.302+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.478184+0000 front 2024-04-02T14:05:02.478035+0000 (oldest deadline 2024-04-02T14:05:27.178017+0000) 2024-04-02T14:05:35.559 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:35 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:35.302+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:12.479154+0000 front 2024-04-02T14:05:12.479177+0000 (oldest deadline 2024-04-02T14:05:34.779328+0000) 2024-04-02T14:05:36.038 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:35 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:35.554+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:36.039 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:35 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:35.554+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:05:36.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:35 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:35.610+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:36.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:35 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:35.610+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:36.170 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:35 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:35.701+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.678847+0000 front 2024-04-02T14:05:03.678915+0000 (oldest deadline 2024-04-02T14:05:28.978958+0000) 2024-04-02T14:05:36.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:35 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:35.922+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:36.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:35 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:35.922+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:36.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:35 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:35.922+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:05:36.340 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:35 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:35.878+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.508702+0000 front 2024-04-02T14:05:01.508650+0000 (oldest deadline 2024-04-02T14:05:29.709226+0000) 2024-04-02T14:05:36.340 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:35 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:35.878+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:12.610521+0000 front 2024-04-02T14:05:12.610606+0000 (oldest deadline 2024-04-02T14:05:34.910522+0000) 2024-04-02T14:05:36.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:35 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:35.840+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:36.341 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:35 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:35.840+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:36.533 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:36 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:36.272+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.076720+0000 front 2024-04-02T14:04:53.076994+0000 (oldest deadline 2024-04-02T14:05:17.776793+0000) 2024-04-02T14:05:36.533 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:36 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:36.272+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.478184+0000 front 2024-04-02T14:05:02.478035+0000 (oldest deadline 2024-04-02T14:05:27.178017+0000) 2024-04-02T14:05:36.534 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:36 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:36.272+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:12.479154+0000 front 2024-04-02T14:05:12.479177+0000 (oldest deadline 2024-04-02T14:05:34.779328+0000) 2024-04-02T14:05:36.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:36 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:36.529+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:36.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:36 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:36.529+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:05:36.790 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:36 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:36.596+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:36.790 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:36 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:36.596+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:37.072 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:36 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:36.743+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.678847+0000 front 2024-04-02T14:05:03.678915+0000 (oldest deadline 2024-04-02T14:05:28.978958+0000) 2024-04-02T14:05:37.072 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:36 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:36.743+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:11.279985+0000 front 2024-04-02T14:05:11.280008+0000 (oldest deadline 2024-04-02T14:05:35.880657+0000) 2024-04-02T14:05:37.072 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:36 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:36.904+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:37.072 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:36 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:36.904+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:37.072 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:36 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:36.904+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:05:37.090 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:36 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:36.849+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.508702+0000 front 2024-04-02T14:05:01.508650+0000 (oldest deadline 2024-04-02T14:05:29.709226+0000) 2024-04-02T14:05:37.090 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:36 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:36.849+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:12.610521+0000 front 2024-04-02T14:05:12.610606+0000 (oldest deadline 2024-04-02T14:05:34.910522+0000) 2024-04-02T14:05:37.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:36 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:36.814+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:37.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:36 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:36.814+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:37.615 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:37 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:37.528+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:37.616 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:37 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:37.528+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:05:37.616 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:37 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:37.290+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.076720+0000 front 2024-04-02T14:04:53.076994+0000 (oldest deadline 2024-04-02T14:05:17.776793+0000) 2024-04-02T14:05:37.616 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:37 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:37.290+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.478184+0000 front 2024-04-02T14:05:02.478035+0000 (oldest deadline 2024-04-02T14:05:27.178017+0000) 2024-04-02T14:05:37.616 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:37 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5[48341]: 2024-04-02T14:05:37.290+0000 7f2e3e606700 -1 osd.5 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:12.479154+0000 front 2024-04-02T14:05:12.479177+0000 (oldest deadline 2024-04-02T14:05:34.779328+0000) 2024-04-02T14:05:38.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:37 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:37.611+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:38.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:37 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:37.611+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:38.084 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:37 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:37.702+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.678847+0000 front 2024-04-02T14:05:03.678915+0000 (oldest deadline 2024-04-02T14:05:28.978958+0000) 2024-04-02T14:05:38.085 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:37 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:37.702+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:11.279985+0000 front 2024-04-02T14:05:11.280008+0000 (oldest deadline 2024-04-02T14:05:35.880657+0000) 2024-04-02T14:05:38.085 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:37 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:37.911+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:38.085 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:37 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:37.911+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:38.085 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:37 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:37.911+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:05:38.090 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:37 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:37.826+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.508702+0000 front 2024-04-02T14:05:01.508650+0000 (oldest deadline 2024-04-02T14:05:29.709226+0000) 2024-04-02T14:05:38.090 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:37 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:37.826+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:12.610521+0000 front 2024-04-02T14:05:12.610606+0000 (oldest deadline 2024-04-02T14:05:34.910522+0000) 2024-04-02T14:05:38.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:37 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:37.782+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:38.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:37 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:37.782+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:38.417 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:38 smithi189 podman[73449]: 2024-04-02 14:05:38.138220572 +0000 UTC m=+5.620052994 container died 68e83f11dec8e79819d5d4cb6696ac249854cc4d6ef37e14c8b9449fabd64c64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-02T14:05:38.784 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:38 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:38.526+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:38.785 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:38 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:38.527+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:05:38.785 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:38 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:38.613+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:38.785 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:38 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:38.613+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:38.786 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:38 smithi189 podman[73449]: 2024-04-02 14:05:38.522115975 +0000 UTC m=+6.003948365 container cleanup 68e83f11dec8e79819d5d4cb6696ac249854cc4d6ef37e14c8b9449fabd64c64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-02T14:05:38.786 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:38 smithi189 bash[73449]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5 2024-04-02T14:05:38.920 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:38 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:38.895+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:38.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:38 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:38.895+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:38.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:38 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:38.895+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:05:38.921 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:38 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:38.664+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.678847+0000 front 2024-04-02T14:05:03.678915+0000 (oldest deadline 2024-04-02T14:05:28.978958+0000) 2024-04-02T14:05:38.922 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:38 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:38.664+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:11.279985+0000 front 2024-04-02T14:05:11.280008+0000 (oldest deadline 2024-04-02T14:05:35.880657+0000) 2024-04-02T14:05:39.038 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:38 smithi189 podman[73467]: 2024-04-02 14:05:38.781888847 +0000 UTC m=+0.640571886 container remove 68e83f11dec8e79819d5d4cb6696ac249854cc4d6ef37e14c8b9449fabd64c64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2) 2024-04-02T14:05:39.090 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:38 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:38.797+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.508702+0000 front 2024-04-02T14:05:01.508650+0000 (oldest deadline 2024-04-02T14:05:29.709226+0000) 2024-04-02T14:05:39.090 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:38 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:38.797+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:12.610521+0000 front 2024-04-02T14:05:12.610606+0000 (oldest deadline 2024-04-02T14:05:34.910522+0000) 2024-04-02T14:05:39.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:38 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:38.781+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:39.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:38 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:38.781+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:39.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:38 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:38.781+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:05:39.388 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:39 smithi189 podman[73553]: 2024-04-02 14:05:39.033279048 +0000 UTC m=+0.017617391 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:05:39.388 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:39 smithi189 podman[73553]: 2024-04-02 14:05:39.14957399 +0000 UTC m=+0.133912327 container create d300125fc5976c006285f76df48dd49d5d3e748474a296d8d72c5541cb9be3b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-02T14:05:39.388 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:39 smithi189 podman[73553]: 2024-04-02 14:05:39.3853755 +0000 UTC m=+0.369713833 container init d300125fc5976c006285f76df48dd49d5d3e748474a296d8d72c5541cb9be3b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-04-02T14:05:39.634 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:39 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:39.629+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.678847+0000 front 2024-04-02T14:05:03.678915+0000 (oldest deadline 2024-04-02T14:05:28.978958+0000) 2024-04-02T14:05:39.634 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:39 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:39.629+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:11.279985+0000 front 2024-04-02T14:05:11.280008+0000 (oldest deadline 2024-04-02T14:05:35.880657+0000) 2024-04-02T14:05:39.646 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:39 smithi189 podman[73553]: 2024-04-02 14:05:39.388513068 +0000 UTC m=+0.372851403 container start d300125fc5976c006285f76df48dd49d5d3e748474a296d8d72c5541cb9be3b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-02T14:05:39.646 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:39 smithi189 podman[73553]: 2024-04-02 14:05:39.485451285 +0000 UTC m=+0.469789625 container attach d300125fc5976c006285f76df48dd49d5d3e748474a296d8d72c5541cb9be3b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-02T14:05:39.646 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:39 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:39.557+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:39.647 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:39 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:39.557+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:05:39.991 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:39 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:39.642+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:39.991 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:39 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:39.642+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:39.992 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 02 14:05:39 smithi189 podman[73553]: 2024-04-02 14:05:39.681159662 +0000 UTC m=+0.665497987 container died d300125fc5976c006285f76df48dd49d5d3e748474a296d8d72c5541cb9be3b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-5-deactivate, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T14:05:40.090 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:39 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:39.833+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.508702+0000 front 2024-04-02T14:05:01.508650+0000 (oldest deadline 2024-04-02T14:05:29.709226+0000) 2024-04-02T14:05:40.090 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:39 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:39.833+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:12.610521+0000 front 2024-04-02T14:05:12.610606+0000 (oldest deadline 2024-04-02T14:05:34.910522+0000) 2024-04-02T14:05:40.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:39 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:39.790+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:40.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:39 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:39.790+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:40.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:39 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:39.790+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:05:40.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:39 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:39.909+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:40.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:39 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:39.909+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:40.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:39 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:39.909+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:05:40.327 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.5.service' 2024-04-02T14:05:40.373 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:05:40.373 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-02T14:05:40.373 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-02T14:05:40.373 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.6 2024-04-02T14:05:40.683 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:40 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:40.520+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:40.683 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:40 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:40.520+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:05:40.696 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:40 smithi052 systemd[1]: Stopping Ceph osd.6 for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:05:40.864 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:40 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:40.583+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.678847+0000 front 2024-04-02T14:05:03.678915+0000 (oldest deadline 2024-04-02T14:05:28.978958+0000) 2024-04-02T14:05:40.865 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:40 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:40.583+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:11.279985+0000 front 2024-04-02T14:05:11.280008+0000 (oldest deadline 2024-04-02T14:05:35.880657+0000) 2024-04-02T14:05:41.038 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:40 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:40.679+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:41.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:40 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:40.679+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:41.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:40 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:40.823+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:41.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:40 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:40.823+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:41.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:40 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:40.823+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:05:41.091 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:40 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:40.692+0000 7f50a44f7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:05:41.091 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:40 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:40.692+0000 7f50a44f7700 -1 osd.6 107 *** Got signal Terminated *** 2024-04-02T14:05:41.091 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:40 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:40.692+0000 7f50a44f7700 -1 osd.6 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T14:05:41.092 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:40 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:40.840+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.508702+0000 front 2024-04-02T14:05:01.508650+0000 (oldest deadline 2024-04-02T14:05:29.709226+0000) 2024-04-02T14:05:41.092 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:40 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:40.840+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:12.610521+0000 front 2024-04-02T14:05:12.610606+0000 (oldest deadline 2024-04-02T14:05:34.910522+0000) 2024-04-02T14:05:41.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:40 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:40.860+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:41.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:40 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:40.860+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:41.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:40 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:40.860+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:05:41.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:41 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:41.661+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:41.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:41 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:41.661+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:41.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:41 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:41.519+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:41.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:41 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:41.519+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:05:41.817 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:41 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:41.534+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.678847+0000 front 2024-04-02T14:05:03.678915+0000 (oldest deadline 2024-04-02T14:05:28.978958+0000) 2024-04-02T14:05:41.817 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:41 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:41.534+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:11.279985+0000 front 2024-04-02T14:05:11.280008+0000 (oldest deadline 2024-04-02T14:05:35.880657+0000) 2024-04-02T14:05:42.090 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:41 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:41.811+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.508702+0000 front 2024-04-02T14:05:01.508650+0000 (oldest deadline 2024-04-02T14:05:29.709226+0000) 2024-04-02T14:05:42.090 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:41 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:41.811+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:12.610521+0000 front 2024-04-02T14:05:12.610606+0000 (oldest deadline 2024-04-02T14:05:34.910522+0000) 2024-04-02T14:05:42.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:41 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:41.819+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:42.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:41 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:41.819+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:42.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:41 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:41.819+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:05:42.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:41 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:41.814+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:42.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:41 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:41.814+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:42.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:41 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:41.814+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:05:42.788 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:42 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:42.524+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:42.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:42 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:42.524+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:05:42.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:42 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:42.623+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:42.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:42 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:42.623+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:42.796 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:42 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:42.529+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.678847+0000 front 2024-04-02T14:05:03.678915+0000 (oldest deadline 2024-04-02T14:05:28.978958+0000) 2024-04-02T14:05:42.796 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:42 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:42.529+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:11.279985+0000 front 2024-04-02T14:05:11.280008+0000 (oldest deadline 2024-04-02T14:05:35.880657+0000) 2024-04-02T14:05:43.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:42 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:42.797+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:43.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:42 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:42.797+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:43.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:42 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:42.797+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:05:43.091 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:42 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:42.809+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.508702+0000 front 2024-04-02T14:05:01.508650+0000 (oldest deadline 2024-04-02T14:05:29.709226+0000) 2024-04-02T14:05:43.091 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:42 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:42.809+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:12.610521+0000 front 2024-04-02T14:05:12.610606+0000 (oldest deadline 2024-04-02T14:05:34.910522+0000) 2024-04-02T14:05:43.091 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:42 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:42.809+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.012352+0000 front 2024-04-02T14:05:17.012400+0000 (oldest deadline 2024-04-02T14:05:42.312596+0000) 2024-04-02T14:05:43.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:42 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:42.792+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:43.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:42 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:42.792+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:43.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:42 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:42.792+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:05:43.770 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:43 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:43.500+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.678847+0000 front 2024-04-02T14:05:03.678915+0000 (oldest deadline 2024-04-02T14:05:28.978958+0000) 2024-04-02T14:05:43.770 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:43 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:43.500+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:11.279985+0000 front 2024-04-02T14:05:11.280008+0000 (oldest deadline 2024-04-02T14:05:35.880657+0000) 2024-04-02T14:05:43.788 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:43 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:43.508+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:43.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:43 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:43.508+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:05:43.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:43 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:43.648+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:43.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:43 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:43.648+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:44.090 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:43 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:43.766+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.508702+0000 front 2024-04-02T14:05:01.508650+0000 (oldest deadline 2024-04-02T14:05:29.709226+0000) 2024-04-02T14:05:44.090 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:43 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:43.766+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:12.610521+0000 front 2024-04-02T14:05:12.610606+0000 (oldest deadline 2024-04-02T14:05:34.910522+0000) 2024-04-02T14:05:44.090 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:43 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:43.766+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.012352+0000 front 2024-04-02T14:05:17.012400+0000 (oldest deadline 2024-04-02T14:05:42.312596+0000) 2024-04-02T14:05:44.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:43 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:43.748+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:44.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:43 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:43.748+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:44.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:43 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:43.748+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:05:44.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:43 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:43.765+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:44.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:43 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:43.765+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:44.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:43 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:43.765+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:05:44.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:43 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:43.765+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:05:44.788 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:44 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:44.496+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:44.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:44 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:44.496+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:05:44.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:44 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:44.656+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:44.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:44 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:44.656+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:44.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:44 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:44.656+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:05:44.920 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:44 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:44.532+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.678847+0000 front 2024-04-02T14:05:03.678915+0000 (oldest deadline 2024-04-02T14:05:28.978958+0000) 2024-04-02T14:05:44.920 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:44 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:44.532+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:11.279985+0000 front 2024-04-02T14:05:11.280008+0000 (oldest deadline 2024-04-02T14:05:35.880657+0000) 2024-04-02T14:05:44.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:44 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:44.753+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:44.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:44 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:44.753+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:44.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:44 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:44.753+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:05:44.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:44 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:44.753+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:05:45.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:44 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:44.766+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:45.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:44 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:44.766+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:45.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:44 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:44.766+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:05:45.090 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:44 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:44.775+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.508702+0000 front 2024-04-02T14:05:01.508650+0000 (oldest deadline 2024-04-02T14:05:29.709226+0000) 2024-04-02T14:05:45.091 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:44 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:44.775+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:12.610521+0000 front 2024-04-02T14:05:12.610606+0000 (oldest deadline 2024-04-02T14:05:34.910522+0000) 2024-04-02T14:05:45.091 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:44 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6[57898]: 2024-04-02T14:05:44.775+0000 7f509cae3700 -1 osd.6 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.012352+0000 front 2024-04-02T14:05:17.012400+0000 (oldest deadline 2024-04-02T14:05:42.312596+0000) 2024-04-02T14:05:45.788 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:45 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:45.528+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:45.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:45 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:45.528+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:05:45.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:45 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:45.665+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:45.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:45 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:45.665+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:45.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:45 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:45.665+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:05:45.920 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:45 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:45.546+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.678847+0000 front 2024-04-02T14:05:03.678915+0000 (oldest deadline 2024-04-02T14:05:28.978958+0000) 2024-04-02T14:05:45.920 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:45 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:45.546+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:11.279985+0000 front 2024-04-02T14:05:11.280008+0000 (oldest deadline 2024-04-02T14:05:35.880657+0000) 2024-04-02T14:05:45.920 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:45 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:45.546+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:15.881092+0000 front 2024-04-02T14:05:15.881190+0000 (oldest deadline 2024-04-02T14:05:45.281715+0000) 2024-04-02T14:05:45.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:45 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:45.714+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:45.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:45 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:45.714+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:45.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:45 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:45.714+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:05:45.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:45 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:45.714+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:05:46.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:45 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:45.744+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:46.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:45 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:45.744+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:46.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:45 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:45.744+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:05:46.557 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:46 smithi052 podman[108128]: 2024-04-02 14:05:46.241689285 +0000 UTC m=+5.675795524 container died 569101ebe81f85ec2fdd884cafa28be6fc1dfbcd45046d6d05ed0a2f76c4613d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-02T14:05:46.840 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:46 smithi052 podman[108128]: 2024-04-02 14:05:46.761666627 +0000 UTC m=+6.195772887 container cleanup 569101ebe81f85ec2fdd884cafa28be6fc1dfbcd45046d6d05ed0a2f76c4613d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T14:05:46.840 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:46 smithi052 bash[108128]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6 2024-04-02T14:05:46.840 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:46 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:46.706+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:46.840 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:46 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:46.706+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:46.841 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:46 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:46.706+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:05:46.841 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:46 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:46.707+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.280638+0000 front 2024-04-02T14:05:20.280658+0000 (oldest deadline 2024-04-02T14:05:46.081454+0000) 2024-04-02T14:05:46.920 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:46 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:46.535+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.678847+0000 front 2024-04-02T14:05:03.678915+0000 (oldest deadline 2024-04-02T14:05:28.978958+0000) 2024-04-02T14:05:46.920 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:46 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:46.535+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:11.279985+0000 front 2024-04-02T14:05:11.280008+0000 (oldest deadline 2024-04-02T14:05:35.880657+0000) 2024-04-02T14:05:46.920 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:46 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:46.535+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:15.881092+0000 front 2024-04-02T14:05:15.881190+0000 (oldest deadline 2024-04-02T14:05:45.281715+0000) 2024-04-02T14:05:46.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:46 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:46.743+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:46.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:46 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:46.743+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:46.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:46 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:46.743+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:05:46.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:46 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:46.743+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:05:47.038 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:46 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:46.576+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:47.039 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:46 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:46.576+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:05:47.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:46 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:46.616+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:47.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:46 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:46.616+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:47.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:46 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:46.616+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:05:47.340 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:47 smithi052 podman[108144]: 2024-04-02 14:05:47.004749444 +0000 UTC m=+0.759523818 container remove 569101ebe81f85ec2fdd884cafa28be6fc1dfbcd45046d6d05ed0a2f76c4613d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2) 2024-04-02T14:05:47.703 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:47 smithi052 podman[108238]: 2024-04-02 14:05:47.414020838 +0000 UTC m=+0.102099951 container create a2cf4d719b64a48cbea46fcdca10e4a63dfd4a8769120ebcebc20f0bc0969456 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T14:05:47.703 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:47 smithi052 podman[108238]: 2024-04-02 14:05:47.330199244 +0000 UTC m=+0.018278384 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:05:47.703 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:47 smithi052 podman[108238]: 2024-04-02 14:05:47.599654448 +0000 UTC m=+0.287733561 container init a2cf4d719b64a48cbea46fcdca10e4a63dfd4a8769120ebcebc20f0bc0969456 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-04-02T14:05:47.703 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:47 smithi052 podman[108238]: 2024-04-02 14:05:47.602662618 +0000 UTC m=+0.290741730 container start a2cf4d719b64a48cbea46fcdca10e4a63dfd4a8769120ebcebc20f0bc0969456 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6-deactivate, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-04-02T14:05:47.704 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:47 smithi052 podman[108238]: 2024-04-02 14:05:47.699781765 +0000 UTC m=+0.387860893 container attach a2cf4d719b64a48cbea46fcdca10e4a63dfd4a8769120ebcebc20f0bc0969456 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T14:05:47.920 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:47 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:47.577+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.678847+0000 front 2024-04-02T14:05:03.678915+0000 (oldest deadline 2024-04-02T14:05:28.978958+0000) 2024-04-02T14:05:47.920 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:47 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:47.577+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:11.279985+0000 front 2024-04-02T14:05:11.280008+0000 (oldest deadline 2024-04-02T14:05:35.880657+0000) 2024-04-02T14:05:47.920 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:47 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:47.577+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:15.881092+0000 front 2024-04-02T14:05:15.881190+0000 (oldest deadline 2024-04-02T14:05:45.281715+0000) 2024-04-02T14:05:47.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:47 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:47.735+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:47.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:47 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:47.735+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:47.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:47 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:47.735+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:05:47.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:47 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:47.735+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:05:48.038 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:47 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:47.571+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:48.038 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:47 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:47.571+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:48.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:47 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:47.571+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:05:48.039 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:47 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:47.580+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:48.039 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:47 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:47.580+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:05:48.039 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:47 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:47.580+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:21.001302+0000 front 2024-04-02T14:05:21.001382+0000 (oldest deadline 2024-04-02T14:05:46.901419+0000) 2024-04-02T14:05:48.090 INFO:journalctl@ceph.osd.6.smithi052.stdout:Apr 02 14:05:47 smithi052 podman[108238]: 2024-04-02 14:05:47.903177111 +0000 UTC m=+0.591256237 container died a2cf4d719b64a48cbea46fcdca10e4a63dfd4a8769120ebcebc20f0bc0969456 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T14:05:48.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:47 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:47.709+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:48.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:47 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:47.709+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:48.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:47 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:47.709+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:05:48.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:47 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:47.709+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.280638+0000 front 2024-04-02T14:05:20.280658+0000 (oldest deadline 2024-04-02T14:05:46.081454+0000) 2024-04-02T14:05:48.495 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.6.service' 2024-04-02T14:05:48.539 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:05:48.539 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-02T14:05:48.540 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-02T14:05:48.540 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.7 2024-04-02T14:05:48.840 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:48 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:48.711+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:48.840 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:48 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:48.711+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:48.840 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:48 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:48.711+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:05:48.840 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:48 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:48.711+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.280638+0000 front 2024-04-02T14:05:20.280658+0000 (oldest deadline 2024-04-02T14:05:46.081454+0000) 2024-04-02T14:05:48.920 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:48 smithi088 systemd[1]: Stopping Ceph osd.7 for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:05:48.921 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:48 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:48.613+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.678847+0000 front 2024-04-02T14:05:03.678915+0000 (oldest deadline 2024-04-02T14:05:28.978958+0000) 2024-04-02T14:05:48.921 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:48 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:48.613+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:11.279985+0000 front 2024-04-02T14:05:11.280008+0000 (oldest deadline 2024-04-02T14:05:35.880657+0000) 2024-04-02T14:05:48.921 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:48 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:48.613+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:15.881092+0000 front 2024-04-02T14:05:15.881190+0000 (oldest deadline 2024-04-02T14:05:45.281715+0000) 2024-04-02T14:05:48.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:48 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:48.748+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:48.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:48 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:48.748+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:48.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:48 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:48.748+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:05:48.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:48 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:48.748+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:05:49.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:48 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:48.588+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:49.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:48 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:48.588+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:49.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:48 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:48.588+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:05:49.039 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:48 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:48.618+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:49.039 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:48 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:48.618+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:05:49.040 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:48 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:48.618+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:21.001302+0000 front 2024-04-02T14:05:21.001382+0000 (oldest deadline 2024-04-02T14:05:46.901419+0000) 2024-04-02T14:05:49.317 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:48 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:48.955+0000 7f9d44c0d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:05:49.317 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:48 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:48.955+0000 7f9d44c0d700 -1 osd.7 107 *** Got signal Terminated *** 2024-04-02T14:05:49.317 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:48 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:48.955+0000 7f9d44c0d700 -1 osd.7 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T14:05:49.920 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:49 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:49.565+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.678847+0000 front 2024-04-02T14:05:03.678915+0000 (oldest deadline 2024-04-02T14:05:28.978958+0000) 2024-04-02T14:05:49.920 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:49 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:49.565+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:11.279985+0000 front 2024-04-02T14:05:11.280008+0000 (oldest deadline 2024-04-02T14:05:35.880657+0000) 2024-04-02T14:05:49.921 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:49 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:49.565+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:15.881092+0000 front 2024-04-02T14:05:15.881190+0000 (oldest deadline 2024-04-02T14:05:45.281715+0000) 2024-04-02T14:05:49.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:49 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:49.779+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:49.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:49 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:49.779+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:49.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:49 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:49.779+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:05:49.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:49 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:49.779+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:05:50.038 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:49 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:49.571+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:50.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:49 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:49.571+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:50.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:49 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:49.571+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:05:50.039 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:49 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:49.568+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:50.039 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:49 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:49.568+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:05:50.039 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:49 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:49.568+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:21.001302+0000 front 2024-04-02T14:05:21.001382+0000 (oldest deadline 2024-04-02T14:05:46.901419+0000) 2024-04-02T14:05:50.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:49 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:49.708+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:50.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:49 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:49.708+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:50.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:49 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:49.708+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:05:50.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:49 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:49.708+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.280638+0000 front 2024-04-02T14:05:20.280658+0000 (oldest deadline 2024-04-02T14:05:46.081454+0000) 2024-04-02T14:05:50.827 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:50 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:50.553+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.678847+0000 front 2024-04-02T14:05:03.678915+0000 (oldest deadline 2024-04-02T14:05:28.978958+0000) 2024-04-02T14:05:50.828 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:50 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:50.553+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:11.279985+0000 front 2024-04-02T14:05:11.280008+0000 (oldest deadline 2024-04-02T14:05:35.880657+0000) 2024-04-02T14:05:50.828 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:50 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:50.553+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:15.881092+0000 front 2024-04-02T14:05:15.881190+0000 (oldest deadline 2024-04-02T14:05:45.281715+0000) 2024-04-02T14:05:51.038 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:50 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:50.538+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:51.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:50 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:50.538+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:51.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:50 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:50.538+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:05:51.039 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:50 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:50.582+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:51.039 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:50 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:50.582+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:05:51.039 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:50 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:50.582+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:21.001302+0000 front 2024-04-02T14:05:21.001382+0000 (oldest deadline 2024-04-02T14:05:46.901419+0000) 2024-04-02T14:05:51.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:50 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:50.669+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:51.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:50 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:50.669+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:51.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:50 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:50.669+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:05:51.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:50 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:50.669+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.280638+0000 front 2024-04-02T14:05:20.280658+0000 (oldest deadline 2024-04-02T14:05:46.081454+0000) 2024-04-02T14:05:51.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:50 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:50.823+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:51.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:50 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:50.823+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:51.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:50 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:50.823+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:05:51.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:50 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:50.823+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:05:51.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:51 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:51.493+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:51.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:51 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:51.493+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:51.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:51 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:51.493+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:05:51.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:51 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:51.586+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:51.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:51 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:51.586+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:05:51.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:51 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:51.586+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:21.001302+0000 front 2024-04-02T14:05:21.001382+0000 (oldest deadline 2024-04-02T14:05:46.901419+0000) 2024-04-02T14:05:51.832 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:51 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:51.559+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.678847+0000 front 2024-04-02T14:05:03.678915+0000 (oldest deadline 2024-04-02T14:05:28.978958+0000) 2024-04-02T14:05:51.833 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:51 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:51.559+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:11.279985+0000 front 2024-04-02T14:05:11.280008+0000 (oldest deadline 2024-04-02T14:05:35.880657+0000) 2024-04-02T14:05:51.833 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:51 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:51.559+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:15.881092+0000 front 2024-04-02T14:05:15.881190+0000 (oldest deadline 2024-04-02T14:05:45.281715+0000) 2024-04-02T14:05:51.833 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:51 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:51.559+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:26.382656+0000 front 2024-04-02T14:05:26.382742+0000 (oldest deadline 2024-04-02T14:05:51.082724+0000) 2024-04-02T14:05:52.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:51 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:51.714+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:52.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:51 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:51.714+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:52.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:51 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:51.714+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:05:52.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:51 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:51.714+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.280638+0000 front 2024-04-02T14:05:20.280658+0000 (oldest deadline 2024-04-02T14:05:46.081454+0000) 2024-04-02T14:05:52.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:51 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:51.829+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:52.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:51 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:51.829+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:52.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:51 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:51.829+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:05:52.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:51 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:51.829+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:05:52.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:51 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:51.829+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:05:52.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:52 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:52.489+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:52.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:52 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:52.489+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:52.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:52 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:52.489+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:05:52.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:52 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:52.571+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:52.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:52 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:52.571+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:05:52.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:52 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:52.571+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:21.001302+0000 front 2024-04-02T14:05:21.001382+0000 (oldest deadline 2024-04-02T14:05:46.901419+0000) 2024-04-02T14:05:52.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:52 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:52.571+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:26.902164+0000 front 2024-04-02T14:05:26.902253+0000 (oldest deadline 2024-04-02T14:05:51.602092+0000) 2024-04-02T14:05:52.920 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:52 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:52.555+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.678847+0000 front 2024-04-02T14:05:03.678915+0000 (oldest deadline 2024-04-02T14:05:28.978958+0000) 2024-04-02T14:05:52.920 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:52 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:52.555+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:11.279985+0000 front 2024-04-02T14:05:11.280008+0000 (oldest deadline 2024-04-02T14:05:35.880657+0000) 2024-04-02T14:05:52.920 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:52 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:52.555+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:15.881092+0000 front 2024-04-02T14:05:15.881190+0000 (oldest deadline 2024-04-02T14:05:45.281715+0000) 2024-04-02T14:05:52.920 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:52 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:52.555+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:26.382656+0000 front 2024-04-02T14:05:26.382742+0000 (oldest deadline 2024-04-02T14:05:51.082724+0000) 2024-04-02T14:05:52.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:52 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:52.784+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:52.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:52 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:52.784+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:52.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:52 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:52.784+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:05:52.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:52 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:52.784+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:05:52.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:52 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:52.784+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:05:53.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:52 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:52.728+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:53.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:52 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:52.728+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:53.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:52 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:52.728+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:05:53.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:52 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:52.728+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.280638+0000 front 2024-04-02T14:05:20.280658+0000 (oldest deadline 2024-04-02T14:05:46.081454+0000) 2024-04-02T14:05:53.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:53 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:53.535+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:53.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:53 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:53.535+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:05:53.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:53 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:53.535+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:21.001302+0000 front 2024-04-02T14:05:21.001382+0000 (oldest deadline 2024-04-02T14:05:46.901419+0000) 2024-04-02T14:05:53.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:53 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:53.535+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:26.902164+0000 front 2024-04-02T14:05:26.902253+0000 (oldest deadline 2024-04-02T14:05:51.602092+0000) 2024-04-02T14:05:53.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:53 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:53.537+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:53.790 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:53 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:53.537+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:53.790 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:53 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:53.537+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:05:53.790 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:53 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:53.537+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:05:53.920 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:53 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:53.590+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.678847+0000 front 2024-04-02T14:05:03.678915+0000 (oldest deadline 2024-04-02T14:05:28.978958+0000) 2024-04-02T14:05:53.920 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:53 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:53.590+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:11.279985+0000 front 2024-04-02T14:05:11.280008+0000 (oldest deadline 2024-04-02T14:05:35.880657+0000) 2024-04-02T14:05:53.921 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:53 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:53.590+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:15.881092+0000 front 2024-04-02T14:05:15.881190+0000 (oldest deadline 2024-04-02T14:05:45.281715+0000) 2024-04-02T14:05:53.921 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:53 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7[54744]: 2024-04-02T14:05:53.590+0000 7f9d3d1f9700 -1 osd.7 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:26.382656+0000 front 2024-04-02T14:05:26.382742+0000 (oldest deadline 2024-04-02T14:05:51.082724+0000) 2024-04-02T14:05:53.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:53 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:53.832+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:53.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:53 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:53.832+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:53.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:53 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:53.832+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:05:53.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:53 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:53.832+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:05:53.922 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:53 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:53.832+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:05:54.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:53 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:53.683+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:54.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:53 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:53.683+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:54.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:53 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:53.683+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:05:54.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:53 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:53.683+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.280638+0000 front 2024-04-02T14:05:20.280658+0000 (oldest deadline 2024-04-02T14:05:46.081454+0000) 2024-04-02T14:05:54.601 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:54 smithi088 podman[72898]: 2024-04-02 14:05:54.322350293 +0000 UTC m=+5.588734068 container died 37961c3b431707b383acfe96cab59359eff7566ab99fa3d5d921ce72d20da45f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-02T14:05:54.920 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:54 smithi088 podman[72898]: 2024-04-02 14:05:54.688829185 +0000 UTC m=+5.955212961 container cleanup 37961c3b431707b383acfe96cab59359eff7566ab99fa3d5d921ce72d20da45f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7, ceph=True, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-02T14:05:54.920 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:54 smithi088 bash[72898]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7 2024-04-02T14:05:54.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:54 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:54.832+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:54.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:54 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:54.832+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:54.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:54 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:54.832+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:05:54.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:54 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:54.832+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:05:54.921 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:54 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:54.832+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:05:55.038 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:54 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:54.583+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:55.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:54 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:54.583+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:55.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:54 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:54.583+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:05:55.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:54 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:54.583+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:05:55.039 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:54 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:54.555+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:55.039 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:54 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:54.555+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:05:55.039 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:54 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:54.555+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:21.001302+0000 front 2024-04-02T14:05:21.001382+0000 (oldest deadline 2024-04-02T14:05:46.901419+0000) 2024-04-02T14:05:55.040 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:54 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:54.555+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:26.902164+0000 front 2024-04-02T14:05:26.902253+0000 (oldest deadline 2024-04-02T14:05:51.602092+0000) 2024-04-02T14:05:55.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:54 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:54.690+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:55.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:54 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:54.690+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:55.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:54 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:54.690+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:05:55.093 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:54 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:54.690+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.280638+0000 front 2024-04-02T14:05:20.280658+0000 (oldest deadline 2024-04-02T14:05:46.081454+0000) 2024-04-02T14:05:55.301 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:54 smithi088 podman[72915]: 2024-04-02 14:05:54.956954374 +0000 UTC m=+0.635515492 container remove 37961c3b431707b383acfe96cab59359eff7566ab99fa3d5d921ce72d20da45f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-02T14:05:55.613 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:55 smithi088 podman[73007]: 2024-04-02 14:05:55.200970706 +0000 UTC m=+0.018732298 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:05:55.613 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:55 smithi088 podman[73007]: 2024-04-02 14:05:55.316065008 +0000 UTC m=+0.133826580 container create b2369c0a267b5a164704e4c1ee8758562c6864b319b73932c02acec04a49f288 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-02T14:05:55.614 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:55 smithi088 podman[73007]: 2024-04-02 14:05:55.610642815 +0000 UTC m=+0.428404397 container init b2369c0a267b5a164704e4c1ee8758562c6864b319b73932c02acec04a49f288 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7-deactivate, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS) 2024-04-02T14:05:55.876 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:55 smithi088 podman[73007]: 2024-04-02 14:05:55.613819459 +0000 UTC m=+0.431581059 container start b2369c0a267b5a164704e4c1ee8758562c6864b319b73932c02acec04a49f288 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-02T14:05:55.876 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:55 smithi088 podman[73007]: 2024-04-02 14:05:55.710954744 +0000 UTC m=+0.528716343 container attach b2369c0a267b5a164704e4c1ee8758562c6864b319b73932c02acec04a49f288 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7-deactivate, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T14:05:56.038 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:55 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:55.553+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:56.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:55 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:55.553+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:56.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:55 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:55.553+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:05:56.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:55 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:55.553+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:05:56.039 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:55 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:55.546+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:56.039 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:55 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:55.546+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:05:56.039 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:55 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:55.546+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:21.001302+0000 front 2024-04-02T14:05:21.001382+0000 (oldest deadline 2024-04-02T14:05:46.901419+0000) 2024-04-02T14:05:56.040 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:55 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:55.546+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:26.902164+0000 front 2024-04-02T14:05:26.902253+0000 (oldest deadline 2024-04-02T14:05:51.602092+0000) 2024-04-02T14:05:56.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:55 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:55.670+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:56.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:55 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:55.670+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:56.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:55 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:55.670+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:05:56.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:55 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:55.670+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.280638+0000 front 2024-04-02T14:05:20.280658+0000 (oldest deadline 2024-04-02T14:05:46.081454+0000) 2024-04-02T14:05:56.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:55 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:55.670+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:28.982578+0000 front 2024-04-02T14:05:28.982687+0000 (oldest deadline 2024-04-02T14:05:54.882440+0000) 2024-04-02T14:05:56.170 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:55 smithi088 podman[73007]: 2024-04-02 14:05:55.896203042 +0000 UTC m=+0.713964636 container died b2369c0a267b5a164704e4c1ee8758562c6864b319b73932c02acec04a49f288 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-02T14:05:56.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:55 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:55.872+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:56.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:55 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:55.872+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:56.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:55 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:55.872+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:05:56.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:55 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:55.872+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:05:56.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:55 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:55.872+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:05:56.465 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 02 14:05:56 smithi088 podman[73033]: 2024-04-02 14:05:56.447877049 +0000 UTC m=+0.546393169 container remove b2369c0a267b5a164704e4c1ee8758562c6864b319b73932c02acec04a49f288 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-7-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-04-02T14:05:56.470 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.7.service' 2024-04-02T14:05:56.512 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:05:56.512 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-02T14:05:56.513 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-02T14:05:56.513 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.8 2024-04-02T14:05:56.845 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:56 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:56.562+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:56.846 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:56 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:56.562+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:56.846 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:56 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:56.562+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:05:56.846 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:56 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:56.562+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:05:56.846 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:56 smithi189 systemd[1]: Stopping Ceph osd.8 for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:05:56.846 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:56 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:56.580+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:56.847 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:56 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:56.580+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:05:56.847 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:56 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:56.580+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:21.001302+0000 front 2024-04-02T14:05:21.001382+0000 (oldest deadline 2024-04-02T14:05:46.901419+0000) 2024-04-02T14:05:56.847 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:56 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:56.580+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:26.902164+0000 front 2024-04-02T14:05:26.902253+0000 (oldest deadline 2024-04-02T14:05:51.602092+0000) 2024-04-02T14:05:56.847 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:56 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:56.841+0000 7f32c5b31700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:05:56.847 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:56 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:56.841+0000 7f32c5b31700 -1 osd.8 107 *** Got signal Terminated *** 2024-04-02T14:05:56.847 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:56 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:56.841+0000 7f32c5b31700 -1 osd.8 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T14:05:57.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:56 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:56.672+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:57.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:56 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:56.672+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:57.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:56 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:56.672+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:05:57.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:56 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:56.672+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.280638+0000 front 2024-04-02T14:05:20.280658+0000 (oldest deadline 2024-04-02T14:05:46.081454+0000) 2024-04-02T14:05:57.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:56 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:56.672+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:28.982578+0000 front 2024-04-02T14:05:28.982687+0000 (oldest deadline 2024-04-02T14:05:54.882440+0000) 2024-04-02T14:05:57.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:56 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:56.899+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:57.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:56 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:56.899+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:57.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:56 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:56.899+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:05:57.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:56 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:56.899+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:05:57.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:56 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:56.899+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:05:57.992 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:57 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:57.649+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:57.992 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:57 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:57.649+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:57.992 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:57 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:57.649+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:05:57.992 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:57 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:57.649+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.280638+0000 front 2024-04-02T14:05:20.280658+0000 (oldest deadline 2024-04-02T14:05:46.081454+0000) 2024-04-02T14:05:57.992 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:57 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:57.649+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:28.982578+0000 front 2024-04-02T14:05:28.982687+0000 (oldest deadline 2024-04-02T14:05:54.882440+0000) 2024-04-02T14:05:58.038 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:57 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:57.537+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:58.038 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:57 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:57.537+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:58.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:57 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:57.537+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:05:58.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:57 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:57.537+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:05:58.039 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:57 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:57.568+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:58.039 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:57 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:57.568+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:05:58.039 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:57 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:57.568+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:21.001302+0000 front 2024-04-02T14:05:21.001382+0000 (oldest deadline 2024-04-02T14:05:46.901419+0000) 2024-04-02T14:05:58.039 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:57 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:57.568+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:26.902164+0000 front 2024-04-02T14:05:26.902253+0000 (oldest deadline 2024-04-02T14:05:51.602092+0000) 2024-04-02T14:05:58.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:57 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:57.926+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:58.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:57 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:57.926+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:58.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:57 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:57.926+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:05:58.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:57 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:57.926+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:05:58.421 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:57 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:57.926+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:05:58.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:58 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:58.505+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:58.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:58 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:58.505+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:58.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:58 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:58.505+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:05:58.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:58 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:58.505+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:05:58.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:58 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:58.547+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:58.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:58 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:58.547+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:05:58.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:58 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:58.547+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:21.001302+0000 front 2024-04-02T14:05:21.001382+0000 (oldest deadline 2024-04-02T14:05:46.901419+0000) 2024-04-02T14:05:58.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:58 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:58.547+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:26.902164+0000 front 2024-04-02T14:05:26.902253+0000 (oldest deadline 2024-04-02T14:05:51.602092+0000) 2024-04-02T14:05:59.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:58 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:58.667+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:05:59.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:58 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:58.667+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:05:59.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:58 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:58.667+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:05:59.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:58 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:58.667+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.280638+0000 front 2024-04-02T14:05:20.280658+0000 (oldest deadline 2024-04-02T14:05:46.081454+0000) 2024-04-02T14:05:59.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:58 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:58.667+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:28.982578+0000 front 2024-04-02T14:05:28.982687+0000 (oldest deadline 2024-04-02T14:05:54.882440+0000) 2024-04-02T14:05:59.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:58 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:58.949+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:05:59.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:58 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:58.949+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:05:59.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:58 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:58.949+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:05:59.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:58 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:58.949+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:05:59.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:58 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:58.949+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:05:59.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:59 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:59.569+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:05:59.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:59 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:59.569+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:05:59.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:59 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:59.569+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:21.001302+0000 front 2024-04-02T14:05:21.001382+0000 (oldest deadline 2024-04-02T14:05:46.901419+0000) 2024-04-02T14:05:59.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:05:59 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:05:59.569+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:26.902164+0000 front 2024-04-02T14:05:26.902253+0000 (oldest deadline 2024-04-02T14:05:51.602092+0000) 2024-04-02T14:05:59.790 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:59 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:59.474+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:05:59.790 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:59 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:59.474+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:05:59.790 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:59 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:59.474+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:05:59.790 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:59 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:59.474+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:05:59.790 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:05:59 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:05:59.474+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:00.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:59 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:59.669+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:06:00.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:59 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:59.669+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:06:00.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:59 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:59.669+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:06:00.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:59 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:59.669+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.280638+0000 front 2024-04-02T14:05:20.280658+0000 (oldest deadline 2024-04-02T14:05:46.081454+0000) 2024-04-02T14:06:00.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:05:59 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:05:59.669+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:28.982578+0000 front 2024-04-02T14:05:28.982687+0000 (oldest deadline 2024-04-02T14:05:54.882440+0000) 2024-04-02T14:06:00.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:59 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:59.910+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:06:00.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:59 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:59.910+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:06:00.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:59 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:59.910+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:06:00.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:59 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:59.910+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:06:00.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:59 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:59.910+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:06:00.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:05:59 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:05:59.910+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:36.744629+0000 front 2024-04-02T14:05:36.744702+0000 (oldest deadline 2024-04-02T14:05:59.544909+0000) 2024-04-02T14:06:00.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:06:00 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:06:00.560+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:06:00.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:06:00 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:06:00.560+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:06:00.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:06:00 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:06:00.560+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:21.001302+0000 front 2024-04-02T14:05:21.001382+0000 (oldest deadline 2024-04-02T14:05:46.901419+0000) 2024-04-02T14:06:00.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:06:00 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:06:00.560+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:26.902164+0000 front 2024-04-02T14:05:26.902253+0000 (oldest deadline 2024-04-02T14:05:51.602092+0000) 2024-04-02T14:06:00.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:00 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:00.447+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:00.790 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:00 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:00.447+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:00.790 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:00 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:00.447+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:00.790 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:00 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:00.447+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:00.790 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:00 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:00.447+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:01.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:00 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:00.714+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:06:01.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:00 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:00.714+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:06:01.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:00 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:00.714+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:06:01.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:00 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:00.714+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.280638+0000 front 2024-04-02T14:05:20.280658+0000 (oldest deadline 2024-04-02T14:05:46.081454+0000) 2024-04-02T14:06:01.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:00 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:00.714+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:28.982578+0000 front 2024-04-02T14:05:28.982687+0000 (oldest deadline 2024-04-02T14:05:54.882440+0000) 2024-04-02T14:06:01.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:00 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:00.908+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:06:01.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:00 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:00.908+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:06:01.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:00 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:00.908+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:06:01.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:00 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:00.908+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:06:01.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:00 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:00.908+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:06:01.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:00 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:00.908+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:36.744629+0000 front 2024-04-02T14:05:36.744702+0000 (oldest deadline 2024-04-02T14:05:59.544909+0000) 2024-04-02T14:06:01.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:06:01 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:06:01.546+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:01.098650+0000 front 2024-04-02T14:05:01.098476+0000 (oldest deadline 2024-04-02T14:05:29.299248+0000) 2024-04-02T14:06:01.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:06:01 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:06:01.546+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:09.299657+0000 front 2024-04-02T14:05:09.299681+0000 (oldest deadline 2024-04-02T14:05:34.599800+0000) 2024-04-02T14:06:01.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:06:01 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:06:01.546+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:21.001302+0000 front 2024-04-02T14:05:21.001382+0000 (oldest deadline 2024-04-02T14:05:46.901419+0000) 2024-04-02T14:06:01.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:06:01 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:06:01.546+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:26.902164+0000 front 2024-04-02T14:05:26.902253+0000 (oldest deadline 2024-04-02T14:05:51.602092+0000) 2024-04-02T14:06:01.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:06:01 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8[55371]: 2024-04-02T14:06:01.546+0000 7f32be11d700 -1 osd.8 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:36.803450+0000 front 2024-04-02T14:05:36.803383+0000 (oldest deadline 2024-04-02T14:06:00.804091+0000) 2024-04-02T14:06:01.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:01 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:01.427+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:01.790 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:01 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:01.427+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:01.790 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:01 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:01.428+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:01.790 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:01 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:01.428+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:01.793 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:01 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:01.428+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:02.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:01 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:01.732+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:06:02.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:01 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:01.732+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:06:02.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:01 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:01.732+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:06:02.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:01 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:01.732+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.280638+0000 front 2024-04-02T14:05:20.280658+0000 (oldest deadline 2024-04-02T14:05:46.081454+0000) 2024-04-02T14:06:02.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:01 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:01.732+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:28.982578+0000 front 2024-04-02T14:05:28.982687+0000 (oldest deadline 2024-04-02T14:05:54.882440+0000) 2024-04-02T14:06:02.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:01 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:01.942+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:06:02.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:01 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:01.942+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:06:02.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:01 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:01.942+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:06:02.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:01 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:01.942+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:06:02.421 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:01 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:01.942+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:06:02.421 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:01 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:01.942+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:36.744629+0000 front 2024-04-02T14:05:36.744702+0000 (oldest deadline 2024-04-02T14:05:59.544909+0000) 2024-04-02T14:06:02.595 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:06:02 smithi189 podman[73659]: 2024-04-02 14:06:02.303040681 +0000 UTC m=+5.588309733 container died 34c22a60508195e9e617541d4a1ee026e9270a8e35d62baee8ba2b8b803e0c5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, io.buildah.version=1.33.5) 2024-04-02T14:06:02.596 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:02 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:02.428+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:02.596 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:02 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:02.428+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:02.596 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:02 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:02.428+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:02.596 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:02 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:02.428+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:02.596 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:02 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:02.428+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:02.929 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:06:02 smithi189 podman[73659]: 2024-04-02 14:06:02.683234104 +0000 UTC m=+5.968503138 container cleanup 34c22a60508195e9e617541d4a1ee026e9270a8e35d62baee8ba2b8b803e0c5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:06:02.929 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:06:02 smithi189 bash[73659]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8 2024-04-02T14:06:03.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:02 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:02.734+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:06:03.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:02 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:02.734+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:06:03.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:02 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:02.734+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:06:03.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:02 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:02.734+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.280638+0000 front 2024-04-02T14:05:20.280658+0000 (oldest deadline 2024-04-02T14:05:46.081454+0000) 2024-04-02T14:06:03.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:02 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:02.734+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:28.982578+0000 front 2024-04-02T14:05:28.982687+0000 (oldest deadline 2024-04-02T14:05:54.882440+0000) 2024-04-02T14:06:03.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:02 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:02.734+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:34.883036+0000 front 2024-04-02T14:05:34.883108+0000 (oldest deadline 2024-04-02T14:06:02.483325+0000) 2024-04-02T14:06:03.249 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:06:02 smithi189 podman[73677]: 2024-04-02 14:06:02.926145036 +0000 UTC m=+0.617303312 container remove 34c22a60508195e9e617541d4a1ee026e9270a8e35d62baee8ba2b8b803e0c5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:06:03.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:02 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:02.931+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:06:03.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:02 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:02.931+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:06:03.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:02 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:02.931+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:06:03.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:02 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:02.931+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:06:03.421 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:02 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:02.931+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:06:03.421 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:02 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:02.931+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:36.744629+0000 front 2024-04-02T14:05:36.744702+0000 (oldest deadline 2024-04-02T14:05:59.544909+0000) 2024-04-02T14:06:03.507 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:06:03 smithi189 podman[73769]: 2024-04-02 14:06:03.149819984 +0000 UTC m=+0.017979424 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:06:03.508 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:06:03 smithi189 podman[73769]: 2024-04-02 14:06:03.26050435 +0000 UTC m=+0.128663833 container create c090f6464ab27bfd93459c7302fdcd188f3921ee5a247500dfe2055f835368cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True) 2024-04-02T14:06:03.508 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:06:03 smithi189 podman[73769]: 2024-04-02 14:06:03.504788258 +0000 UTC m=+0.372947697 container init c090f6464ab27bfd93459c7302fdcd188f3921ee5a247500dfe2055f835368cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-02T14:06:03.508 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:03 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:03.455+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:03.508 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:03 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:03.455+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:03.508 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:03 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:03.455+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:03.508 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:03 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:03.455+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:03.509 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:03 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:03.455+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:03.788 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:06:03 smithi189 podman[73769]: 2024-04-02 14:06:03.508165352 +0000 UTC m=+0.376324789 container start c090f6464ab27bfd93459c7302fdcd188f3921ee5a247500dfe2055f835368cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-02T14:06:03.789 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:06:03 smithi189 podman[73769]: 2024-04-02 14:06:03.604951486 +0000 UTC m=+0.473110942 container attach c090f6464ab27bfd93459c7302fdcd188f3921ee5a247500dfe2055f835368cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T14:06:04.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:03 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:03.781+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:06:04.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:03 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:03.781+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:06:04.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:03 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:03.781+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:06:04.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:03 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:03.781+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.280638+0000 front 2024-04-02T14:05:20.280658+0000 (oldest deadline 2024-04-02T14:05:46.081454+0000) 2024-04-02T14:06:04.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:03 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:03.781+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:28.982578+0000 front 2024-04-02T14:05:28.982687+0000 (oldest deadline 2024-04-02T14:05:54.882440+0000) 2024-04-02T14:06:04.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:03 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:03.781+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:34.883036+0000 front 2024-04-02T14:05:34.883108+0000 (oldest deadline 2024-04-02T14:06:02.483325+0000) 2024-04-02T14:06:04.160 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 02 14:06:03 smithi189 podman[73769]: 2024-04-02 14:06:03.798197082 +0000 UTC m=+0.666356540 container died c090f6464ab27bfd93459c7302fdcd188f3921ee5a247500dfe2055f835368cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-02T14:06:04.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:03 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:03.888+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:06:04.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:03 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:03.888+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:06:04.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:03 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:03.888+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:06:04.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:03 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:03.888+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:06:04.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:03 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:03.888+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:06:04.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:03 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:03.888+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:36.744629+0000 front 2024-04-02T14:05:36.744702+0000 (oldest deadline 2024-04-02T14:05:59.544909+0000) 2024-04-02T14:06:04.483 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.8.service' 2024-04-02T14:06:04.527 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:06:04.527 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-02T14:06:04.527 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-02T14:06:04.528 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.9 2024-04-02T14:06:04.739 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:04 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:04.444+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:04.739 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:04 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:04.444+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:04.739 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:04 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:04.444+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:04.740 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:04 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:04.444+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:04.740 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:04 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:04.444+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:04.810 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:04 smithi052 systemd[1]: Stopping Ceph osd.9 for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:06:05.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:04 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:04.806+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:06:05.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:04 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:04.806+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:06:05.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:04 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:04.806+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:06:05.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:04 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:04.806+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.280638+0000 front 2024-04-02T14:05:20.280658+0000 (oldest deadline 2024-04-02T14:05:46.081454+0000) 2024-04-02T14:06:05.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:04 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:04.806+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:28.982578+0000 front 2024-04-02T14:05:28.982687+0000 (oldest deadline 2024-04-02T14:05:54.882440+0000) 2024-04-02T14:06:05.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:04 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:04.806+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:34.883036+0000 front 2024-04-02T14:05:34.883108+0000 (oldest deadline 2024-04-02T14:06:02.483325+0000) 2024-04-02T14:06:05.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:04 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:04.856+0000 7fb16ea33700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:06:05.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:04 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:04.856+0000 7fb16ea33700 -1 osd.9 107 *** Got signal Terminated *** 2024-04-02T14:06:05.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:04 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:04.856+0000 7fb16ea33700 -1 osd.9 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T14:06:05.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:04 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:04.851+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:06:05.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:04 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:04.851+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:06:05.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:04 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:04.851+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:06:05.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:04 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:04.851+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:06:05.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:04 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:04.851+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:06:05.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:04 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:04.851+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:36.744629+0000 front 2024-04-02T14:05:36.744702+0000 (oldest deadline 2024-04-02T14:05:59.544909+0000) 2024-04-02T14:06:05.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:05 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:05.474+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:05.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:05 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:05.474+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:05.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:05 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:05.474+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:05.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:05 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:05.474+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:05.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:05 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:05.474+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:06.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:05 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:05.856+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:06:06.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:05 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:05.856+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:06:06.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:05 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:05.856+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:06:06.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:05 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:05.856+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:06:06.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:05 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:05.856+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:06:06.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:05 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:05.856+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:36.744629+0000 front 2024-04-02T14:05:36.744702+0000 (oldest deadline 2024-04-02T14:05:59.544909+0000) 2024-04-02T14:06:06.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:05 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:05.838+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:06:06.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:05 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:05.838+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:06:06.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:05 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:05.838+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:06:06.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:05 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:05.838+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.280638+0000 front 2024-04-02T14:05:20.280658+0000 (oldest deadline 2024-04-02T14:05:46.081454+0000) 2024-04-02T14:06:06.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:05 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:05.838+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:28.982578+0000 front 2024-04-02T14:05:28.982687+0000 (oldest deadline 2024-04-02T14:05:54.882440+0000) 2024-04-02T14:06:06.341 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:05 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:05.838+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:34.883036+0000 front 2024-04-02T14:05:34.883108+0000 (oldest deadline 2024-04-02T14:06:02.483325+0000) 2024-04-02T14:06:06.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:06 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:06.501+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:06.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:06 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:06.501+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:06.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:06 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:06.501+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:06.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:06 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:06.501+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:06.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:06 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:06.501+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:07.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:06 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:06.903+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:06:07.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:06 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:06.903+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:06:07.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:06 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:06.903+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:06:07.173 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:06 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:06.903+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:06:07.173 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:06 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:06.903+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:06:07.174 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:06 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:06.903+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:36.744629+0000 front 2024-04-02T14:05:36.744702+0000 (oldest deadline 2024-04-02T14:05:59.544909+0000) 2024-04-02T14:06:07.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:06 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:06.861+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:06:07.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:06 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:06.861+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:06:07.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:06 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:06.861+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:06:07.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:06 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:06.861+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.280638+0000 front 2024-04-02T14:05:20.280658+0000 (oldest deadline 2024-04-02T14:05:46.081454+0000) 2024-04-02T14:06:07.341 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:06 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:06.861+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:28.982578+0000 front 2024-04-02T14:05:28.982687+0000 (oldest deadline 2024-04-02T14:05:54.882440+0000) 2024-04-02T14:06:07.341 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:06 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:06.861+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:34.883036+0000 front 2024-04-02T14:05:34.883108+0000 (oldest deadline 2024-04-02T14:06:02.483325+0000) 2024-04-02T14:06:07.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:07 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:07.485+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:07.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:07 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:07.485+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:07.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:07 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:07.485+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:07.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:07 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:07.485+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:07.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:07 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:07.485+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:08.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:07 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:07.877+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:06:08.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:07 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:07.877+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:06:08.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:07 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:07.877+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:06:08.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:07 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:07.877+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:06:08.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:07 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:07.877+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:06:08.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:07 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:07.877+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:36.744629+0000 front 2024-04-02T14:05:36.744702+0000 (oldest deadline 2024-04-02T14:05:59.544909+0000) 2024-04-02T14:06:08.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:07 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:07.886+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:06:08.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:07 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:07.886+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:06:08.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:07 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:07.886+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:06:08.340 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:07 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:07.886+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.280638+0000 front 2024-04-02T14:05:20.280658+0000 (oldest deadline 2024-04-02T14:05:46.081454+0000) 2024-04-02T14:06:08.341 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:07 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:07.886+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:28.982578+0000 front 2024-04-02T14:05:28.982687+0000 (oldest deadline 2024-04-02T14:05:54.882440+0000) 2024-04-02T14:06:08.341 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:07 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:07.886+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:34.883036+0000 front 2024-04-02T14:05:34.883108+0000 (oldest deadline 2024-04-02T14:06:02.483325+0000) 2024-04-02T14:06:08.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:08 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:08.533+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:08.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:08 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:08.533+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:08.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:08 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:08.533+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:08.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:08 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:08.533+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:08.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:08 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:08.533+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:09.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:08 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:08.839+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:06:09.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:08 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:08.839+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:06:09.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:08 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:08.839+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:06:09.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:08 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:08.839+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.280638+0000 front 2024-04-02T14:05:20.280658+0000 (oldest deadline 2024-04-02T14:05:46.081454+0000) 2024-04-02T14:06:09.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:08 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:08.839+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:28.982578+0000 front 2024-04-02T14:05:28.982687+0000 (oldest deadline 2024-04-02T14:05:54.882440+0000) 2024-04-02T14:06:09.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:08 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:08.839+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:34.883036+0000 front 2024-04-02T14:05:34.883108+0000 (oldest deadline 2024-04-02T14:06:02.483325+0000) 2024-04-02T14:06:09.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:08 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:08.899+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:06:09.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:08 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:08.899+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:06:09.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:08 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:08.899+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:06:09.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:08 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:08.899+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:06:09.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:08 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:08.899+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:06:09.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:08 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:08.899+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:36.744629+0000 front 2024-04-02T14:05:36.744702+0000 (oldest deadline 2024-04-02T14:05:59.544909+0000) 2024-04-02T14:06:09.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:09 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:09.500+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:09.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:09 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:09.500+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:09.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:09 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:09.500+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:09.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:09 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:09.500+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:09.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:09 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:09.500+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:09.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:09 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:09.500+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-04-02T14:05:44.670379+0000 front 2024-04-02T14:05:44.670402+0000 (oldest deadline 2024-04-02T14:06:09.370328+0000) 2024-04-02T14:06:10.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:09 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:09.826+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.376823+0000 front 2024-04-02T14:04:56.377128+0000 (oldest deadline 2024-04-02T14:05:20.476942+0000) 2024-04-02T14:06:10.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:09 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:09.826+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:03.278736+0000 front 2024-04-02T14:05:03.278869+0000 (oldest deadline 2024-04-02T14:05:28.578489+0000) 2024-04-02T14:06:10.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:09 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:09.826+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:13.279716+0000 front 2024-04-02T14:05:13.279687+0000 (oldest deadline 2024-04-02T14:05:37.979584+0000) 2024-04-02T14:06:10.090 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:09 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:09.826+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.280638+0000 front 2024-04-02T14:05:20.280658+0000 (oldest deadline 2024-04-02T14:05:46.081454+0000) 2024-04-02T14:06:10.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:09 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:09.826+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:28.982578+0000 front 2024-04-02T14:05:28.982687+0000 (oldest deadline 2024-04-02T14:05:54.882440+0000) 2024-04-02T14:06:10.091 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:09 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9[64515]: 2024-04-02T14:06:09.826+0000 7fb16b027700 -1 osd.9 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:34.883036+0000 front 2024-04-02T14:05:34.883108+0000 (oldest deadline 2024-04-02T14:06:02.483325+0000) 2024-04-02T14:06:10.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:09 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:09.882+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:06:10.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:09 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:09.882+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:06:10.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:09 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:09.882+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:06:10.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:09 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:09.882+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:06:10.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:09 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:09.882+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:06:10.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:09 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:09.882+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:36.744629+0000 front 2024-04-02T14:05:36.744702+0000 (oldest deadline 2024-04-02T14:05:59.544909+0000) 2024-04-02T14:06:10.552 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:10 smithi052 podman[108354]: 2024-04-02 14:06:10.287813382 +0000 UTC m=+5.560744998 container died 3f40d09a3764c0deec83439dc0f22816dccf758203cadd1df7c377924a90953c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T14:06:10.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:10 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:10.458+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:10.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:10 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:10.458+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:10.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:10 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:10.458+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:10.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:10 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:10.458+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:10.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:10 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:10.458+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:10.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:10 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:10.458+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-04-02T14:05:44.670379+0000 front 2024-04-02T14:05:44.670402+0000 (oldest deadline 2024-04-02T14:06:09.370328+0000) 2024-04-02T14:06:10.840 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:10 smithi052 podman[108354]: 2024-04-02 14:06:10.648335187 +0000 UTC m=+5.921266803 container cleanup 3f40d09a3764c0deec83439dc0f22816dccf758203cadd1df7c377924a90953c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-02T14:06:10.840 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:10 smithi052 bash[108354]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9 2024-04-02T14:06:11.272 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:10 smithi052 podman[108371]: 2024-04-02 14:06:10.933139458 +0000 UTC m=+0.647964334 container remove 3f40d09a3764c0deec83439dc0f22816dccf758203cadd1df7c377924a90953c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T14:06:11.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:10 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:10.917+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:06:11.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:10 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:10.917+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:06:11.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:10 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:10.917+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:06:11.421 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:10 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:10.917+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:06:11.421 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:10 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:10.917+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:06:11.421 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:10 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:10.917+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:36.744629+0000 front 2024-04-02T14:05:36.744702+0000 (oldest deadline 2024-04-02T14:05:59.544909+0000) 2024-04-02T14:06:11.580 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:11 smithi052 podman[108461]: 2024-04-02 14:06:11.172437251 +0000 UTC m=+0.017663773 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:06:11.581 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:11 smithi052 podman[108461]: 2024-04-02 14:06:11.30066693 +0000 UTC m=+0.145893480 container create 015502e179b63a1c84e77011c293c8c02bb6ae0cddd977ecaac3f8ee2bca2203 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-02T14:06:11.581 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:11 smithi052 podman[108461]: 2024-04-02 14:06:11.503096098 +0000 UTC m=+0.348322607 container init 015502e179b63a1c84e77011c293c8c02bb6ae0cddd977ecaac3f8ee2bca2203 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:06:11.581 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:11 smithi052 podman[108461]: 2024-04-02 14:06:11.506946421 +0000 UTC m=+0.352172930 container start 015502e179b63a1c84e77011c293c8c02bb6ae0cddd977ecaac3f8ee2bca2203 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0) 2024-04-02T14:06:11.581 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:11 smithi052 podman[108461]: 2024-04-02 14:06:11.57783476 +0000 UTC m=+0.423061272 container attach 015502e179b63a1c84e77011c293c8c02bb6ae0cddd977ecaac3f8ee2bca2203 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:06:11.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:11 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:11.462+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:11.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:11 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:11.462+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:11.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:11 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:11.462+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:11.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:11 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:11.462+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:11.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:11 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:11.462+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:11.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:11 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:11.462+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-04-02T14:05:44.670379+0000 front 2024-04-02T14:05:44.670402+0000 (oldest deadline 2024-04-02T14:06:09.370328+0000) 2024-04-02T14:06:11.840 INFO:journalctl@ceph.osd.9.smithi052.stdout:Apr 02 14:06:11 smithi052 podman[108461]: 2024-04-02 14:06:11.80477846 +0000 UTC m=+0.650004981 container died 015502e179b63a1c84e77011c293c8c02bb6ae0cddd977ecaac3f8ee2bca2203 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240326) 2024-04-02T14:06:12.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:11 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:11.894+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:06:12.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:11 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:11.894+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:06:12.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:11 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:11.894+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:06:12.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:11 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:11.894+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:06:12.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:11 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:11.894+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:06:12.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:11 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:11.894+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:36.744629+0000 front 2024-04-02T14:05:36.744702+0000 (oldest deadline 2024-04-02T14:05:59.544909+0000) 2024-04-02T14:06:12.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:11 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:11.894+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-04-02T14:05:45.246942+0000 front 2024-04-02T14:05:45.246977+0000 (oldest deadline 2024-04-02T14:06:11.146870+0000) 2024-04-02T14:06:12.370 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.9.service' 2024-04-02T14:06:12.413 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:06:12.414 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-02T14:06:12.414 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-02T14:06:12.414 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.10 2024-04-02T14:06:12.736 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:12 smithi088 systemd[1]: Stopping Ceph osd.10 for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:06:12.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:12 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:12.500+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:12.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:12 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:12.500+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:12.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:12 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:12.500+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:12.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:12 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:12.500+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:12.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:12 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:12.500+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:12.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:12 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:12.500+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-04-02T14:05:44.670379+0000 front 2024-04-02T14:05:44.670402+0000 (oldest deadline 2024-04-02T14:06:09.370328+0000) 2024-04-02T14:06:13.170 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:12 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:12.732+0000 7f492da86700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:06:13.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:12 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:12.732+0000 7f492da86700 -1 osd.10 107 *** Got signal Terminated *** 2024-04-02T14:06:13.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:12 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:12.732+0000 7f492da86700 -1 osd.10 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T14:06:13.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:12 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:12.891+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:06:13.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:12 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:12.891+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:06:13.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:12 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:12.891+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:06:13.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:12 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:12.891+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:06:13.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:12 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:12.891+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:06:13.171 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:12 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:12.891+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:36.744629+0000 front 2024-04-02T14:05:36.744702+0000 (oldest deadline 2024-04-02T14:05:59.544909+0000) 2024-04-02T14:06:13.172 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:12 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:12.891+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-04-02T14:05:45.246942+0000 front 2024-04-02T14:05:45.246977+0000 (oldest deadline 2024-04-02T14:06:11.146870+0000) 2024-04-02T14:06:14.038 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:13 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:13.540+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:14.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:13 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:13.540+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:14.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:13 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:13.540+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:14.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:13 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:13.540+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:14.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:13 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:13.540+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:14.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:13 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:13.540+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-04-02T14:05:44.670379+0000 front 2024-04-02T14:05:44.670402+0000 (oldest deadline 2024-04-02T14:06:09.370328+0000) 2024-04-02T14:06:14.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:13 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:13.933+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:06:14.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:13 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:13.933+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:06:14.421 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:13 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:13.933+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:06:14.421 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:13 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:13.933+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:06:14.421 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:13 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:13.933+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:06:14.421 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:13 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:13.933+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:36.744629+0000 front 2024-04-02T14:05:36.744702+0000 (oldest deadline 2024-04-02T14:05:59.544909+0000) 2024-04-02T14:06:14.421 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:13 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:13.933+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-04-02T14:05:45.246942+0000 front 2024-04-02T14:05:45.246977+0000 (oldest deadline 2024-04-02T14:06:11.146870+0000) 2024-04-02T14:06:14.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:14 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:14.507+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:14.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:14 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:14.507+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:14.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:14 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:14.507+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:14.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:14 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:14.507+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:14.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:14 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:14.507+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:14.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:14 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:14.507+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-04-02T14:05:44.670379+0000 front 2024-04-02T14:05:44.670402+0000 (oldest deadline 2024-04-02T14:06:09.370328+0000) 2024-04-02T14:06:15.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:14 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:14.965+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:06:15.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:14 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:14.965+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:06:15.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:14 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:14.965+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:06:15.421 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:14 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:14.965+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:06:15.421 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:14 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:14.965+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:06:15.421 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:14 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:14.965+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:36.744629+0000 front 2024-04-02T14:05:36.744702+0000 (oldest deadline 2024-04-02T14:05:59.544909+0000) 2024-04-02T14:06:15.421 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:14 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:14.965+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-04-02T14:05:45.246942+0000 front 2024-04-02T14:05:45.246977+0000 (oldest deadline 2024-04-02T14:06:11.146870+0000) 2024-04-02T14:06:15.743 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:15 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:15.485+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:15.743 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:15 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:15.485+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:15.743 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:15 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:15.485+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:15.743 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:15 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:15.485+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:15.744 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:15 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:15.485+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:15.744 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:15 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:15.485+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-04-02T14:05:44.670379+0000 front 2024-04-02T14:05:44.670402+0000 (oldest deadline 2024-04-02T14:06:09.370328+0000) 2024-04-02T14:06:15.744 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:15 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:15.485+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6820 osd.7 since back 2024-04-02T14:05:52.871465+0000 front 2024-04-02T14:05:52.871420+0000 (oldest deadline 2024-04-02T14:06:14.571388+0000) 2024-04-02T14:06:16.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:15 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:15.978+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:06:16.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:15 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:15.978+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:06:16.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:15 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:15.978+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:06:16.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:15 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:15.978+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:06:16.421 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:15 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:15.978+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:06:16.421 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:15 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:15.978+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:36.744629+0000 front 2024-04-02T14:05:36.744702+0000 (oldest deadline 2024-04-02T14:05:59.544909+0000) 2024-04-02T14:06:16.421 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:15 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:15.978+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-04-02T14:05:45.246942+0000 front 2024-04-02T14:05:45.246977+0000 (oldest deadline 2024-04-02T14:06:11.146870+0000) 2024-04-02T14:06:16.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:16 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:16.438+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:16.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:16 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:16.438+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:16.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:16 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:16.438+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:16.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:16 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:16.438+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:16.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:16 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:16.438+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:16.790 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:16 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:16.438+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-04-02T14:05:44.670379+0000 front 2024-04-02T14:05:44.670402+0000 (oldest deadline 2024-04-02T14:06:09.370328+0000) 2024-04-02T14:06:16.790 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:16 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:16.438+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6820 osd.7 since back 2024-04-02T14:05:52.871465+0000 front 2024-04-02T14:05:52.871420+0000 (oldest deadline 2024-04-02T14:06:14.571388+0000) 2024-04-02T14:06:17.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:16 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:16.948+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:53.836011+0000 front 2024-04-02T14:04:53.836031+0000 (oldest deadline 2024-04-02T14:05:19.136092+0000) 2024-04-02T14:06:17.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:16 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:16.948+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-02T14:05:02.637469+0000 front 2024-04-02T14:05:02.637414+0000 (oldest deadline 2024-04-02T14:05:28.537677+0000) 2024-04-02T14:06:17.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:16 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:16.948+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.538196+0000 front 2024-04-02T14:05:08.538208+0000 (oldest deadline 2024-04-02T14:05:34.938746+0000) 2024-04-02T14:06:17.420 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:16 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:16.948+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:17.740419+0000 front 2024-04-02T14:05:17.740623+0000 (oldest deadline 2024-04-02T14:05:43.540864+0000) 2024-04-02T14:06:17.421 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:16 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:16.948+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.441636+0000 front 2024-04-02T14:05:29.441687+0000 (oldest deadline 2024-04-02T14:05:51.741823+0000) 2024-04-02T14:06:17.421 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:16 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:16.948+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:36.744629+0000 front 2024-04-02T14:05:36.744702+0000 (oldest deadline 2024-04-02T14:05:59.544909+0000) 2024-04-02T14:06:17.421 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:16 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10[61382]: 2024-04-02T14:06:16.948+0000 7f492a07a700 -1 osd.10 107 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-04-02T14:05:45.246942+0000 front 2024-04-02T14:05:45.246977+0000 (oldest deadline 2024-04-02T14:06:11.146870+0000) 2024-04-02T14:06:17.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:17 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:17.408+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:17.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:17 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:17.408+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:17.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:17 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:17.408+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:17.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:17 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:17.408+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:17.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:17 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:17.408+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:17.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:17 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:17.408+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-04-02T14:05:44.670379+0000 front 2024-04-02T14:05:44.670402+0000 (oldest deadline 2024-04-02T14:06:09.370328+0000) 2024-04-02T14:06:17.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:17 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:17.408+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6820 osd.7 since back 2024-04-02T14:05:52.871465+0000 front 2024-04-02T14:05:52.871420+0000 (oldest deadline 2024-04-02T14:06:14.571388+0000) 2024-04-02T14:06:18.460 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:18 smithi088 podman[73124]: 2024-04-02 14:06:18.190229246 +0000 UTC m=+5.570205600 container died aced5a2df9805c5ad2ae7524a2e3e942481feaa36370e01cf78a4f81ee227146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-02T14:06:18.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:18 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:18.417+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:18.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:18 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:18.417+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:18.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:18 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:18.417+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:18.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:18 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:18.417+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:18.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:18 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:18.417+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:18.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:18 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:18.417+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-04-02T14:05:44.670379+0000 front 2024-04-02T14:05:44.670402+0000 (oldest deadline 2024-04-02T14:06:09.370328+0000) 2024-04-02T14:06:18.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:18 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:18.417+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6820 osd.7 since back 2024-04-02T14:05:52.871465+0000 front 2024-04-02T14:05:52.871420+0000 (oldest deadline 2024-04-02T14:06:14.571388+0000) 2024-04-02T14:06:18.920 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:18 smithi088 podman[73124]: 2024-04-02 14:06:18.539875567 +0000 UTC m=+5.919851908 container cleanup aced5a2df9805c5ad2ae7524a2e3e942481feaa36370e01cf78a4f81ee227146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-02T14:06:18.920 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:18 smithi088 bash[73124]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10 2024-04-02T14:06:18.920 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:18 smithi088 podman[73143]: 2024-04-02 14:06:18.691070702 +0000 UTC m=+0.495436910 container remove aced5a2df9805c5ad2ae7524a2e3e942481feaa36370e01cf78a4f81ee227146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-02T14:06:19.413 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:19 smithi088 podman[73233]: 2024-04-02 14:06:18.992826541 +0000 UTC m=+0.017670012 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:06:19.414 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:19 smithi088 podman[73233]: 2024-04-02 14:06:19.100265777 +0000 UTC m=+0.125109287 container create 3abc56daf6b596a494a9a63cd5297601c7ca64d3b990e26e2958b23d4111d5f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T14:06:19.414 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:19 smithi088 podman[73233]: 2024-04-02 14:06:19.32747636 +0000 UTC m=+0.352319811 container init 3abc56daf6b596a494a9a63cd5297601c7ca64d3b990e26e2958b23d4111d5f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10-deactivate, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:06:19.414 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:19 smithi088 podman[73233]: 2024-04-02 14:06:19.330572032 +0000 UTC m=+0.355415478 container start 3abc56daf6b596a494a9a63cd5297601c7ca64d3b990e26e2958b23d4111d5f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T14:06:19.414 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:19 smithi088 podman[73233]: 2024-04-02 14:06:19.410666686 +0000 UTC m=+0.435510128 container attach 3abc56daf6b596a494a9a63cd5297601c7ca64d3b990e26e2958b23d4111d5f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD) 2024-04-02T14:06:19.670 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 02 14:06:19 smithi088 podman[73233]: 2024-04-02 14:06:19.616115089 +0000 UTC m=+0.640958552 container died 3abc56daf6b596a494a9a63cd5297601c7ca64d3b990e26e2958b23d4111d5f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T14:06:19.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:19 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:19.462+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:19.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:19 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:19.462+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:19.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:19 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:19.462+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:19.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:19 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:19.462+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:19.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:19 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:19.462+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:19.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:19 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:19.462+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-04-02T14:05:44.670379+0000 front 2024-04-02T14:05:44.670402+0000 (oldest deadline 2024-04-02T14:06:09.370328+0000) 2024-04-02T14:06:19.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:19 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:19.462+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6820 osd.7 since back 2024-04-02T14:05:52.871465+0000 front 2024-04-02T14:05:52.871420+0000 (oldest deadline 2024-04-02T14:06:14.571388+0000) 2024-04-02T14:06:20.164 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.10.service' 2024-04-02T14:06:20.204 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:06:20.204 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-02T14:06:20.204 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-02T14:06:20.204 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.11 2024-04-02T14:06:20.538 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:20 smithi189 systemd[1]: Stopping Ceph osd.11 for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:06:20.539 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:20 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:20.444+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:20.539 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:20 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:20.444+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:20.539 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:20 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:20.444+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:20.539 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:20 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:20.444+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:20.539 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:20 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:20.444+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:20.539 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:20 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:20.444+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-04-02T14:05:44.670379+0000 front 2024-04-02T14:05:44.670402+0000 (oldest deadline 2024-04-02T14:06:09.370328+0000) 2024-04-02T14:06:20.539 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:20 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:20.444+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6820 osd.7 since back 2024-04-02T14:05:52.871465+0000 front 2024-04-02T14:05:52.871420+0000 (oldest deadline 2024-04-02T14:06:14.571388+0000) 2024-04-02T14:06:21.038 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:20 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:20.570+0000 7fa7a21bb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:06:21.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:20 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:20.570+0000 7fa7a21bb700 -1 osd.11 107 *** Got signal Terminated *** 2024-04-02T14:06:21.039 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:20 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:20.570+0000 7fa7a21bb700 -1 osd.11 107 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T14:06:21.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:21 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:21.437+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:21.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:21 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:21.437+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:21.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:21 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:21.437+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:21.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:21 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:21.437+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:21.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:21 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:21.437+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:21.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:21 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:21.437+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-04-02T14:05:44.670379+0000 front 2024-04-02T14:05:44.670402+0000 (oldest deadline 2024-04-02T14:06:09.370328+0000) 2024-04-02T14:06:21.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:21 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:21.437+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6820 osd.7 since back 2024-04-02T14:05:52.871465+0000 front 2024-04-02T14:05:52.871420+0000 (oldest deadline 2024-04-02T14:06:14.571388+0000) 2024-04-02T14:06:22.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:22 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:22.478+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:22.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:22 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:22.478+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:22.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:22 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:22.478+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:22.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:22 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:22.478+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:22.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:22 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:22.478+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:22.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:22 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:22.478+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-04-02T14:05:44.670379+0000 front 2024-04-02T14:05:44.670402+0000 (oldest deadline 2024-04-02T14:06:09.370328+0000) 2024-04-02T14:06:22.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:22 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:22.478+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6820 osd.7 since back 2024-04-02T14:05:52.871465+0000 front 2024-04-02T14:05:52.871420+0000 (oldest deadline 2024-04-02T14:06:14.571388+0000) 2024-04-02T14:06:23.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:23 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:23.483+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:23.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:23 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:23.483+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:23.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:23 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:23.483+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:23.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:23 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:23.483+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:23.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:23 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:23.483+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:23.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:23 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:23.483+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-04-02T14:05:44.670379+0000 front 2024-04-02T14:05:44.670402+0000 (oldest deadline 2024-04-02T14:06:09.370328+0000) 2024-04-02T14:06:23.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:23 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:23.483+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6820 osd.7 since back 2024-04-02T14:05:52.871465+0000 front 2024-04-02T14:05:52.871420+0000 (oldest deadline 2024-04-02T14:06:14.571388+0000) 2024-04-02T14:06:24.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:24 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:24.444+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:24.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:24 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:24.444+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:24.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:24 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:24.444+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:24.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:24 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:24.444+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:24.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:24 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:24.444+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:24.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:24 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:24.444+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-04-02T14:05:44.670379+0000 front 2024-04-02T14:05:44.670402+0000 (oldest deadline 2024-04-02T14:06:09.370328+0000) 2024-04-02T14:06:24.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:24 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:24.444+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6820 osd.7 since back 2024-04-02T14:05:52.871465+0000 front 2024-04-02T14:05:52.871420+0000 (oldest deadline 2024-04-02T14:06:14.571388+0000) 2024-04-02T14:06:25.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:25 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:25.461+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-04-02T14:04:56.163002+0000 front 2024-04-02T14:04:56.162986+0000 (oldest deadline 2024-04-02T14:05:18.463078+0000) 2024-04-02T14:06:25.788 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:25 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:25.461+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-02T14:05:08.464729+0000 front 2024-04-02T14:05:08.464547+0000 (oldest deadline 2024-04-02T14:05:34.364881+0000) 2024-04-02T14:06:25.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:25 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:25.461+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-04-02T14:05:20.166533+0000 front 2024-04-02T14:05:20.166572+0000 (oldest deadline 2024-04-02T14:05:43.666714+0000) 2024-04-02T14:06:25.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:25 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:25.461+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-02T14:05:29.468582+0000 front 2024-04-02T14:05:29.468506+0000 (oldest deadline 2024-04-02T14:05:52.968466+0000) 2024-04-02T14:06:25.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:25 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:25.461+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-02T14:05:32.968992+0000 front 2024-04-02T14:05:32.969071+0000 (oldest deadline 2024-04-02T14:05:58.869094+0000) 2024-04-02T14:06:25.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:25 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:25.461+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-04-02T14:05:44.670379+0000 front 2024-04-02T14:05:44.670402+0000 (oldest deadline 2024-04-02T14:06:09.370328+0000) 2024-04-02T14:06:25.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:25 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:25.461+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.88:6820 osd.7 since back 2024-04-02T14:05:52.871465+0000 front 2024-04-02T14:05:52.871420+0000 (oldest deadline 2024-04-02T14:06:14.571388+0000) 2024-04-02T14:06:25.789 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:25 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11[62150]: 2024-04-02T14:06:25.461+0000 7fa79a7a7700 -1 osd.11 107 heartbeat_check: no reply from 172.21.15.189:6820 osd.8 since back 2024-04-02T14:06:01.572497+0000 front 2024-04-02T14:06:01.572549+0000 (oldest deadline 2024-04-02T14:06:24.472496+0000) 2024-04-02T14:06:26.280 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:25 smithi189 podman[73885]: 2024-04-02 14:06:25.959746675 +0000 UTC m=+5.554623863 container died d07d621d0db0721d2ad2b72e71f7e6e8048c28a954d9b16873862af6eb62f761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:06:26.280 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:26 smithi189 podman[73885]: 2024-04-02 14:06:26.277379994 +0000 UTC m=+5.872257131 container cleanup d07d621d0db0721d2ad2b72e71f7e6e8048c28a954d9b16873862af6eb62f761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T14:06:26.538 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:26 smithi189 bash[73885]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11 2024-04-02T14:06:26.539 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:26 smithi189 podman[73902]: 2024-04-02 14:06:26.486925437 +0000 UTC m=+0.527047324 container remove d07d621d0db0721d2ad2b72e71f7e6e8048c28a954d9b16873862af6eb62f761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-02T14:06:27.176 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:26 smithi189 podman[73997]: 2024-04-02 14:06:26.879403922 +0000 UTC m=+0.107107154 container create 0ac3b6d5cc306e6fd5fafdb133c1402e6345b22b3ec36444bbbfb39e367690d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux ) 2024-04-02T14:06:27.176 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:26 smithi189 podman[73997]: 2024-04-02 14:06:26.789344536 +0000 UTC m=+0.017047758 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T14:06:27.177 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:27 smithi189 podman[73997]: 2024-04-02 14:06:27.106977655 +0000 UTC m=+0.334680859 container init 0ac3b6d5cc306e6fd5fafdb133c1402e6345b22b3ec36444bbbfb39e367690d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS) 2024-04-02T14:06:27.177 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:27 smithi189 podman[73997]: 2024-04-02 14:06:27.109936694 +0000 UTC m=+0.337639906 container start 0ac3b6d5cc306e6fd5fafdb133c1402e6345b22b3ec36444bbbfb39e367690d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2) 2024-04-02T14:06:27.177 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:27 smithi189 podman[73997]: 2024-04-02 14:06:27.173342495 +0000 UTC m=+0.401045709 container attach 0ac3b6d5cc306e6fd5fafdb133c1402e6345b22b3ec36444bbbfb39e367690d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326) 2024-04-02T14:06:27.538 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:27 smithi189 podman[73997]: 2024-04-02 14:06:27.398735366 +0000 UTC m=+0.626438606 container died 0ac3b6d5cc306e6fd5fafdb133c1402e6345b22b3ec36444bbbfb39e367690d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-04-02T14:06:27.932 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.11.service' 2024-04-02T14:06:27.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:27 smithi189 podman[74023]: 2024-04-02 14:06:27.910128367 +0000 UTC m=+0.502881595 container remove 0ac3b6d5cc306e6fd5fafdb133c1402e6345b22b3ec36444bbbfb39e367690d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-osd-11-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:06:27.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:27 smithi189 systemd[1]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.11.service: Deactivated successfully. 2024-04-02T14:06:27.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:27 smithi189 systemd[1]: Stopped Ceph osd.11 for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4. 2024-04-02T14:06:27.958 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 02 14:06:27 smithi189 systemd[1]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@osd.11.service: Consumed 17.168s CPU time. 2024-04-02T14:06:27.973 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:06:27.973 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-02T14:06:27.973 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-02T14:06:27.974 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.a 2024-04-02T14:06:28.340 INFO:journalctl@ceph.mds.a.smithi052.stdout:Apr 02 14:06:28 smithi052 systemd[1]: Stopping Ceph mds.a for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:06:28.608 INFO:journalctl@ceph.mds.a.smithi052.stdout:Apr 02 14:06:28 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-a[71342]: 2024-04-02T14:06:28.383+0000 7fab091ee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:06:28.609 INFO:journalctl@ceph.mds.a.smithi052.stdout:Apr 02 14:06:28 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-a[71342]: 2024-04-02T14:06:28.383+0000 7fab091ee700 -1 mds.a *** got signal Terminated *** 2024-04-02T14:06:32.404 INFO:journalctl@ceph.mds.a.smithi052.stdout:Apr 02 14:06:32 smithi052 podman[108577]: 2024-04-02 14:06:32.106229875 +0000 UTC m=+3.932924913 container died c4bf472e5cd1e0f975093bad60c016da28e48deca3d36752d6fb22e8cda9bb32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-02T14:06:32.754 INFO:journalctl@ceph.mds.a.smithi052.stdout:Apr 02 14:06:32 smithi052 podman[108577]: 2024-04-02 14:06:32.500283883 +0000 UTC m=+4.326978924 container cleanup c4bf472e5cd1e0f975093bad60c016da28e48deca3d36752d6fb22e8cda9bb32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-a, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True) 2024-04-02T14:06:32.754 INFO:journalctl@ceph.mds.a.smithi052.stdout:Apr 02 14:06:32 smithi052 bash[108577]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-a 2024-04-02T14:06:32.880 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.a.service' 2024-04-02T14:06:32.918 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:06:32.918 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-02T14:06:32.918 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-02T14:06:32.919 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.d 2024-04-02T14:06:33.089 INFO:journalctl@ceph.mds.d.smithi052.stdout:Apr 02 14:06:32 smithi052 systemd[1]: Stopping Ceph mds.d for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:06:33.589 INFO:journalctl@ceph.mds.d.smithi052.stdout:Apr 02 14:06:33 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-d[70389]: 2024-04-02T14:06:33.236+0000 7f97f441d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:06:33.590 INFO:journalctl@ceph.mds.d.smithi052.stdout:Apr 02 14:06:33 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-d[70389]: 2024-04-02T14:06:33.236+0000 7f97f441d700 -1 mds.d *** got signal Terminated *** 2024-04-02T14:06:36.740 INFO:journalctl@ceph.mds.d.smithi052.stdout:Apr 02 14:06:36 smithi052 podman[108709]: 2024-04-02 14:06:36.467642157 +0000 UTC m=+3.330278912 container died ef988063f2ad61f1a4c1276706137e26765b5aedc40deac23f94a83fb3ba2df2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-02T14:06:37.089 INFO:journalctl@ceph.mds.d.smithi052.stdout:Apr 02 14:06:36 smithi052 podman[108709]: 2024-04-02 14:06:36.953969261 +0000 UTC m=+3.816606014 container cleanup ef988063f2ad61f1a4c1276706137e26765b5aedc40deac23f94a83fb3ba2df2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-04-02T14:06:37.090 INFO:journalctl@ceph.mds.d.smithi052.stdout:Apr 02 14:06:36 smithi052 bash[108709]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-d 2024-04-02T14:06:37.332 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.d.service' 2024-04-02T14:06:37.367 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:06:37.368 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-02T14:06:37.368 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-02T14:06:37.368 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.g 2024-04-02T14:06:37.839 INFO:journalctl@ceph.mds.g.smithi052.stdout:Apr 02 14:06:37 smithi052 systemd[1]: Stopping Ceph mds.g for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:06:37.840 INFO:journalctl@ceph.mds.g.smithi052.stdout:Apr 02 14:06:37 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-g[70987]: 2024-04-02T14:06:37.683+0000 7ff54e5c8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:06:37.840 INFO:journalctl@ceph.mds.g.smithi052.stdout:Apr 02 14:06:37 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-g[70987]: 2024-04-02T14:06:37.683+0000 7ff54e5c8700 -1 mds.g *** got signal Terminated *** 2024-04-02T14:06:40.386 INFO:journalctl@ceph.mds.g.smithi052.stdout:Apr 02 14:06:40 smithi052 podman[108828]: 2024-04-02 14:06:40.108013653 +0000 UTC m=+2.527792770 container died 111cee9802159fd512aa896a7153636618f6a8545ecbed040746a9b19daf2d5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-g, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2) 2024-04-02T14:06:40.745 INFO:journalctl@ceph.mds.g.smithi052.stdout:Apr 02 14:06:40 smithi052 podman[108828]: 2024-04-02 14:06:40.483166907 +0000 UTC m=+2.902946003 container cleanup 111cee9802159fd512aa896a7153636618f6a8545ecbed040746a9b19daf2d5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-g, ceph=True, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:06:40.746 INFO:journalctl@ceph.mds.g.smithi052.stdout:Apr 02 14:06:40 smithi052 bash[108828]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-g 2024-04-02T14:06:40.921 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.g.service' 2024-04-02T14:06:40.956 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:06:40.956 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-02T14:06:40.957 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-02T14:06:40.957 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.j 2024-04-02T14:06:41.281 INFO:journalctl@ceph.mds.j.smithi052.stdout:Apr 02 14:06:41 smithi052 systemd[1]: Stopping Ceph mds.j for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:06:41.589 INFO:journalctl@ceph.mds.j.smithi052.stdout:Apr 02 14:06:41 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-j[70039]: 2024-04-02T14:06:41.277+0000 7ff33714d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:06:41.590 INFO:journalctl@ceph.mds.j.smithi052.stdout:Apr 02 14:06:41 smithi052 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-j[70039]: 2024-04-02T14:06:41.277+0000 7ff33714d700 -1 mds.j *** got signal Terminated *** 2024-04-02T14:06:45.590 INFO:journalctl@ceph.mds.j.smithi052.stdout:Apr 02 14:06:45 smithi052 podman[108948]: 2024-04-02 14:06:45.286772086 +0000 UTC m=+4.113056195 container died 566dd4642c00b4e72bdc0276ac4e2cebedaafa561d46ac39a2b9d591655307a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-j, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux ) 2024-04-02T14:06:45.915 INFO:journalctl@ceph.mds.j.smithi052.stdout:Apr 02 14:06:45 smithi052 podman[108948]: 2024-04-02 14:06:45.728534101 +0000 UTC m=+4.554818201 container cleanup 566dd4642c00b4e72bdc0276ac4e2cebedaafa561d46ac39a2b9d591655307a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-j, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux ) 2024-04-02T14:06:45.915 INFO:journalctl@ceph.mds.j.smithi052.stdout:Apr 02 14:06:45 smithi052 bash[108948]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-j 2024-04-02T14:06:46.151 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.j.service' 2024-04-02T14:06:46.174 INFO:journalctl@ceph.mds.j.smithi052.stdout:Apr 02 14:06:45 smithi052 podman[108962]: 2024-04-02 14:06:45.913042834 +0000 UTC m=+0.623200568 container remove 566dd4642c00b4e72bdc0276ac4e2cebedaafa561d46ac39a2b9d591655307a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-j, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T14:06:46.175 INFO:journalctl@ceph.mds.j.smithi052.stdout:Apr 02 14:06:46 smithi052 systemd[1]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.j.service: Deactivated successfully. 2024-04-02T14:06:46.175 INFO:journalctl@ceph.mds.j.smithi052.stdout:Apr 02 14:06:46 smithi052 systemd[1]: Stopped Ceph mds.j for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4. 2024-04-02T14:06:46.175 INFO:journalctl@ceph.mds.j.smithi052.stdout:Apr 02 14:06:46 smithi052 systemd[1]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.j.service: Consumed 1.085s CPU time. 2024-04-02T14:06:46.185 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:06:46.185 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-02T14:06:46.185 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-02T14:06:46.185 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.b 2024-04-02T14:06:46.499 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 02 14:06:46 smithi088 systemd[1]: Stopping Ceph mds.b for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:06:46.756 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 02 14:06:46 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-b[66653]: 2024-04-02T14:06:46.496+0000 7fbba73b3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:06:46.756 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 02 14:06:46 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-b[66653]: 2024-04-02T14:06:46.496+0000 7fbba73b3700 -1 mds.b *** got signal Terminated *** 2024-04-02T14:06:48.392 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 02 14:06:48 smithi088 podman[73339]: 2024-04-02 14:06:48.125968889 +0000 UTC m=+1.743842388 container died 3189802bb9564724045d58037a77c863dc732ee697d952000a5dd65597afaf40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-b, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-02T14:06:48.670 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 02 14:06:48 smithi088 podman[73339]: 2024-04-02 14:06:48.478798932 +0000 UTC m=+2.096672430 container cleanup 3189802bb9564724045d58037a77c863dc732ee697d952000a5dd65597afaf40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-02T14:06:48.670 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 02 14:06:48 smithi088 bash[73339]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-b 2024-04-02T14:06:48.854 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.b.service' 2024-04-02T14:06:48.892 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:06:48.893 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-02T14:06:48.893 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-02T14:06:48.893 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.e 2024-04-02T14:06:49.169 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 02 14:06:48 smithi088 systemd[1]: Stopping Ceph mds.e for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:06:49.669 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 02 14:06:49 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-e[66042]: 2024-04-02T14:06:49.232+0000 7f293f819700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:06:49.670 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 02 14:06:49 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-e[66042]: 2024-04-02T14:06:49.232+0000 7f293f819700 -1 mds.e *** got signal Terminated *** 2024-04-02T14:06:52.386 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 02 14:06:52 smithi088 podman[73473]: 2024-04-02 14:06:52.122196466 +0000 UTC m=+3.015930165 container died ee6638eee1800dafee1270fe768726f9206f2a2fc3271aed2dcb507fa9462e4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326) 2024-04-02T14:06:52.670 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 02 14:06:52 smithi088 podman[73473]: 2024-04-02 14:06:52.474674312 +0000 UTC m=+3.368408010 container cleanup ee6638eee1800dafee1270fe768726f9206f2a2fc3271aed2dcb507fa9462e4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-e, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=) 2024-04-02T14:06:52.670 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 02 14:06:52 smithi088 bash[73473]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-e 2024-04-02T14:06:52.896 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.e.service' 2024-04-02T14:06:52.931 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:06:52.931 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-02T14:06:52.932 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-02T14:06:52.932 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.h 2024-04-02T14:06:53.169 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 02 14:06:53 smithi088 systemd[1]: Stopping Ceph mds.h for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:06:53.670 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 02 14:06:53 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-h[66349]: 2024-04-02T14:06:53.279+0000 7f8451d99700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:06:53.670 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 02 14:06:53 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-h[66349]: 2024-04-02T14:06:53.279+0000 7f8451d99700 -1 mds.h *** got signal Terminated *** 2024-04-02T14:06:56.365 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 02 14:06:56 smithi088 podman[73593]: 2024-04-02 14:06:56.0982165 +0000 UTC m=+2.954837775 container died db0696c6a3761338e4df3142be444c25e55f75cfce89b076402d6daf086e02fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-h, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-02T14:06:56.670 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 02 14:06:56 smithi088 podman[73593]: 2024-04-02 14:06:56.453518893 +0000 UTC m=+3.310140158 container cleanup db0696c6a3761338e4df3142be444c25e55f75cfce89b076402d6daf086e02fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-02T14:06:56.670 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 02 14:06:56 smithi088 bash[73593]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-h 2024-04-02T14:06:56.835 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.h.service' 2024-04-02T14:06:56.869 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:06:56.870 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-02T14:06:56.870 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-02T14:06:56.870 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.k 2024-04-02T14:06:57.169 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 02 14:06:56 smithi088 systemd[1]: Stopping Ceph mds.k for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:06:57.670 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 02 14:06:57 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-k[65717]: 2024-04-02T14:06:57.195+0000 7f50aea47700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:06:57.670 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 02 14:06:57 smithi088 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-k[65717]: 2024-04-02T14:06:57.195+0000 7f50aea47700 -1 mds.k *** got signal Terminated *** 2024-04-02T14:07:00.363 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 02 14:07:00 smithi088 podman[73713]: 2024-04-02 14:07:00.095491394 +0000 UTC m=+3.016658288 container died 0f8f26af25837ecb1c9d02660d2e7da65c239b5d208c8d454b993b823e1841af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-02T14:07:00.670 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 02 14:07:00 smithi088 podman[73713]: 2024-04-02 14:07:00.459750154 +0000 UTC m=+3.380917027 container cleanup 0f8f26af25837ecb1c9d02660d2e7da65c239b5d208c8d454b993b823e1841af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-k, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0) 2024-04-02T14:07:00.670 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 02 14:07:00 smithi088 bash[73713]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-k 2024-04-02T14:07:00.888 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.k.service' 2024-04-02T14:07:00.922 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:07:00.923 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-02T14:07:00.923 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-02T14:07:00.923 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.c 2024-04-02T14:07:01.212 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 02 14:07:00 smithi189 systemd[1]: Stopping Ceph mds.c for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:07:01.483 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 02 14:07:01 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-c[67191]: 2024-04-02T14:07:01.209+0000 7f543836b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:07:01.483 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 02 14:07:01 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-c[67191]: 2024-04-02T14:07:01.209+0000 7f543836b700 -1 mds.c *** got signal Terminated *** 2024-04-02T14:07:04.421 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 02 14:07:04 smithi189 podman[74101]: 2024-04-02 14:07:04.109540594 +0000 UTC m=+2.991498942 container died da47ce00bf484ed57db0f880790ad833af4740507668108a2c0d25785d57b7f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2) 2024-04-02T14:07:04.746 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 02 14:07:04 smithi189 podman[74101]: 2024-04-02 14:07:04.517388661 +0000 UTC m=+3.399347018 container cleanup da47ce00bf484ed57db0f880790ad833af4740507668108a2c0d25785d57b7f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-c, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-02T14:07:04.746 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 02 14:07:04 smithi189 bash[74101]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-c 2024-04-02T14:07:04.890 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.c.service' 2024-04-02T14:07:04.927 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:07:04.928 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-02T14:07:04.928 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-02T14:07:04.928 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.f 2024-04-02T14:07:05.288 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 02 14:07:05 smithi189 systemd[1]: Stopping Ceph mds.f for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:07:05.288 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 02 14:07:05 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-f[66881]: 2024-04-02T14:07:05.245+0000 7fecc0059700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:07:05.288 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 02 14:07:05 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-f[66881]: 2024-04-02T14:07:05.245+0000 7fecc0059700 -1 mds.f *** got signal Terminated *** 2024-04-02T14:07:08.374 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 02 14:07:08 smithi189 podman[74232]: 2024-04-02 14:07:08.106593884 +0000 UTC m=+2.963059443 container died 67fc9458e9f5e1b2a3bee3c98f549973deb66238ef10de0c3f48f3187bb8d9c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-f, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-02T14:07:08.725 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 02 14:07:08 smithi189 podman[74232]: 2024-04-02 14:07:08.463058731 +0000 UTC m=+3.319524273 container cleanup 67fc9458e9f5e1b2a3bee3c98f549973deb66238ef10de0c3f48f3187bb8d9c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-f, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=) 2024-04-02T14:07:08.725 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 02 14:07:08 smithi189 bash[74232]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-f 2024-04-02T14:07:08.889 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.f.service' 2024-04-02T14:07:08.924 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:07:08.925 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-02T14:07:08.925 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-02T14:07:08.925 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.i 2024-04-02T14:07:09.253 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 02 14:07:09 smithi189 systemd[1]: Stopping Ceph mds.i for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:07:09.538 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 02 14:07:09 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-i[67499]: 2024-04-02T14:07:09.250+0000 7fc86828d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:07:09.538 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 02 14:07:09 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-i[67499]: 2024-04-02T14:07:09.250+0000 7fc86828d700 -1 mds.i *** got signal Terminated *** 2024-04-02T14:07:12.288 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 02 14:07:12 smithi189 podman[74353]: 2024-04-02 14:07:12.108696616 +0000 UTC m=+2.962032763 container died ad384f043f4af8f88ad183060e5f3b8fcf81b84313bbcce02973f4cb455e0f01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-i, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0) 2024-04-02T14:07:12.653 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 02 14:07:12 smithi189 podman[74353]: 2024-04-02 14:07:12.383235184 +0000 UTC m=+3.236571325 container cleanup ad384f043f4af8f88ad183060e5f3b8fcf81b84313bbcce02973f4cb455e0f01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-i, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:07:12.654 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 02 14:07:12 smithi189 bash[74353]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-i 2024-04-02T14:07:12.815 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.i.service' 2024-04-02T14:07:12.850 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:07:12.850 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-02T14:07:12.850 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-02T14:07:12.850 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.l 2024-04-02T14:07:13.164 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 02 14:07:12 smithi189 systemd[1]: Stopping Ceph mds.l for bfa00e6e-f0f6-11ee-b647-cb9ed24678a4... 2024-04-02T14:07:13.538 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 02 14:07:13 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-l[66560]: 2024-04-02T14:07:13.160+0000 7f6713f63700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:07:13.538 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 02 14:07:13 smithi189 ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-l[66560]: 2024-04-02T14:07:13.160+0000 7f6713f63700 -1 mds.l *** got signal Terminated *** 2024-04-02T14:07:16.431 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 02 14:07:16 smithi189 podman[74477]: 2024-04-02 14:07:16.113512036 +0000 UTC m=+3.052602737 container died 82ee271879cb8553682581db677f3947819ec73532c8ba08bc5d49896b5831c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-l, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-04-02T14:07:16.731 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 02 14:07:16 smithi189 podman[74477]: 2024-04-02 14:07:16.511634974 +0000 UTC m=+3.450725647 container cleanup 82ee271879cb8553682581db677f3947819ec73532c8ba08bc5d49896b5831c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-l, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5) 2024-04-02T14:07:16.732 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 02 14:07:16 smithi189 bash[74477]: ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4-mds-l 2024-04-02T14:07:16.870 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bfa00e6e-f0f6-11ee-b647-cb9ed24678a4@mds.l.service' 2024-04-02T14:07:16.904 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:07:16.904 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-02T14:07:16.905 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-04-02T14:07:17.103 INFO:teuthology.orchestra.run.smithi052.stdout:Deleting cluster with fsid: bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T14:07:19.521 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-04-02T14:07:19.710 INFO:teuthology.orchestra.run.smithi088.stdout:Deleting cluster with fsid: bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T14:07:22.147 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-04-02T14:07:22.339 INFO:teuthology.orchestra.run.smithi189.stdout:Deleting cluster with fsid: bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T14:07:24.819 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T14:07:24.856 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T14:07:24.889 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T14:07:24.918 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-02T14:07:24.920 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635602/remote/smithi052/crash 2024-04-02T14:07:24.921 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/crash -- . 2024-04-02T14:07:24.961 INFO:teuthology.orchestra.run.smithi052.stderr:tar: /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-04-02T14:07:24.961 INFO:teuthology.orchestra.run.smithi052.stderr:tar: Error is not recoverable: exiting now 2024-04-02T14:07:24.963 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635602/remote/smithi088/crash 2024-04-02T14:07:24.964 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/crash -- . 2024-04-02T14:07:25.015 INFO:teuthology.orchestra.run.smithi088.stderr:tar: /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-04-02T14:07:25.017 INFO:teuthology.orchestra.run.smithi088.stderr:tar: Error is not recoverable: exiting now 2024-04-02T14:07:25.018 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635602/remote/smithi189/crash 2024-04-02T14:07:25.019 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/crash -- . 2024-04-02T14:07:25.107 INFO:teuthology.orchestra.run.smithi189.stderr:tar: /var/lib/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-04-02T14:07:25.107 INFO:teuthology.orchestra.run.smithi189.stderr:tar: Error is not recoverable: exiting now 2024-04-02T14:07:25.108 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-02T14:07:25.108 DEBUG:teuthology.orchestra.run.smithi052:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-02T14:07:25.145 INFO:tasks.cephadm:Compressing logs... 2024-04-02T14:07:25.146 DEBUG:teuthology.orchestra.run.smithi052:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-02T14:07:25.188 DEBUG:teuthology.orchestra.run.smithi088:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-02T14:07:25.191 DEBUG:teuthology.orchestra.run.smithi189:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-02T14:07:25.215 INFO:teuthology.orchestra.run.smithi052.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-02T14:07:25.215 INFO:teuthology.orchestra.run.smithi052.stderr:: No such file or directory 2024-04-02T14:07:25.215 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.6.log 2024-04-02T14:07:25.215 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mon.a.log 2024-04-02T14:07:25.217 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.audit.log 2024-04-02T14:07:25.217 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.6.log: /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.g.log 2024-04-02T14:07:25.217 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.j.log 2024-04-02T14:07:25.217 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.audit.log: /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.3.log 2024-04-02T14:07:25.224 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mgr.x.log 2024-04-02T14:07:25.224 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.0.log 2024-04-02T14:07:25.224 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.cephadm.log 2024-04-02T14:07:25.224 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mgr.x.log: /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.0.log: 90.2% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.audit.log.gz 2024-04-02T14:07:25.224 INFO:teuthology.orchestra.run.smithi052.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-02T14:07:25.229 INFO:teuthology.orchestra.run.smithi052.stderr: 94.7% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.j.log.gz 2024-04-02T14:07:25.231 INFO:teuthology.orchestra.run.smithi189.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-02T14:07:25.231 INFO:teuthology.orchestra.run.smithi189.stderr:: No such file or directory 2024-04-02T14:07:25.231 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.l.log 2024-04-02T14:07:25.231 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.audit.log 2024-04-02T14:07:25.231 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.i.log 2024-04-02T14:07:25.232 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.11.log 2024-04-02T14:07:25.232 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mgr.z.log 2024-04-02T14:07:25.232 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.i.log: /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.8.log 2024-04-02T14:07:25.232 INFO:teuthology.orchestra.run.smithi189.stderr: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-02T14:07:25.232 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.5.log 2024-04-02T14:07:25.232 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.f.log 2024-04-02T14:07:25.232 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.cephadm.log 2024-04-02T14:07:25.232 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.f.log: 90.3% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.audit.log.gz 2024-04-02T14:07:25.232 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.log 2024-04-02T14:07:25.233 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.cephadm.log: 83.4% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.cephadm.log.gz 2024-04-02T14:07:25.233 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.2.log 2024-04-02T14:07:25.233 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.log: 87.5% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.log.gz 2024-04-02T14:07:25.233 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.c.log 2024-04-02T14:07:25.233 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.2.log: 91.9% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mgr.z.log.gz 2024-04-02T14:07:25.233 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-volume.log 2024-04-02T14:07:25.234 INFO:teuthology.orchestra.run.smithi088.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-02T14:07:25.234 INFO:teuthology.orchestra.run.smithi088.stderr:: No such file or directory 2024-04-02T14:07:25.234 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.10.log 2024-04-02T14:07:25.234 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.audit.log 2024-04-02T14:07:25.234 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.b.log 2024-04-02T14:07:25.235 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.4.log 2024-04-02T14:07:25.235 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.1.log 2024-04-02T14:07:25.235 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.b.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-02T14:07:25.235 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.k.log 2024-04-02T14:07:25.235 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.h.log 2024-04-02T14:07:25.235 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.7.log 2024-04-02T14:07:25.235 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.e.log 2024-04-02T14:07:25.235 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.7.log: 90.3% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.audit.log.gz 2024-04-02T14:07:25.235 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mgr.y.log 2024-04-02T14:07:25.238 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.log 2024-04-02T14:07:25.239 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-volume.log 2024-04-02T14:07:25.239 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.log: 83.9% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.cephadm.log.gz 2024-04-02T14:07:25.242 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mon.c.log 2024-04-02T14:07:25.242 INFO:teuthology.orchestra.run.smithi052.stderr: 87.5% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.log.gz 2024-04-02T14:07:25.242 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.cephadm.log 2024-04-02T14:07:25.252 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.9.log 2024-04-02T14:07:25.252 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.d.log 2024-04-02T14:07:25.256 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mgr.y.log: 91.9%gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.log 2024-04-02T14:07:25.256 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mgr.y.log.gz 2024-04-02T14:07:25.256 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.cephadm.log: 83.4% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.cephadm.log.gz 2024-04-02T14:07:25.262 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-volume.log 2024-04-02T14:07:25.265 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.a.log 2024-04-02T14:07:25.265 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.d.log: 91.0% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-volume.log.gz 2024-04-02T14:07:25.266 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-02T14:07:25.268 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-volume.log: /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mon.c.log: 90.9% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-volume.log.gz 2024-04-02T14:07:25.270 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.a.log: 94.5% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.d.log.gz 2024-04-02T14:07:25.275 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.log: gzip -5 --verbose -- /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mon.b.log 2024-04-02T14:07:25.278 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-volume.log: 87.5% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph.log.gz 2024-04-02T14:07:25.284 INFO:teuthology.orchestra.run.smithi189.stderr: 94.6% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.l.log.gz 2024-04-02T14:07:25.285 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.log: 94.5% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.g.log.gz 2024-04-02T14:07:25.287 INFO:teuthology.orchestra.run.smithi052.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-02T14:07:25.292 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mon.b.log: 94.3% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.k.log.gz 2024-04-02T14:07:25.299 INFO:teuthology.orchestra.run.smithi088.stderr: 91.0% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-volume.log.gz 2024-04-02T14:07:25.317 INFO:teuthology.orchestra.run.smithi189.stderr: 93.1% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.i.log.gz 2024-04-02T14:07:25.324 INFO:teuthology.orchestra.run.smithi189.stderr: 93.0% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.f.log.gz 2024-04-02T14:07:25.325 INFO:teuthology.orchestra.run.smithi088.stderr: 93.2% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.h.log.gz 2024-04-02T14:07:25.336 INFO:teuthology.orchestra.run.smithi052.stderr: 94.5% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.a.log.gz 2024-04-02T14:07:25.370 INFO:teuthology.orchestra.run.smithi189.stderr: 93.0% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.c.log.gz 2024-04-02T14:07:25.507 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mgr.x.log.gz 2024-04-02T14:07:25.608 INFO:teuthology.orchestra.run.smithi189.stderr: 92.1% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mon.c.log.gz 2024-04-02T14:07:25.739 INFO:teuthology.orchestra.run.smithi088.stderr: 92.1% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mon.b.log.gz 2024-04-02T14:07:26.548 INFO:teuthology.orchestra.run.smithi052.stderr: 90.6% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mon.a.log.gz 2024-04-02T14:07:26.819 INFO:teuthology.orchestra.run.smithi088.stderr: 95.4% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.e.log.gz 2024-04-02T14:07:27.594 INFO:teuthology.orchestra.run.smithi088.stderr: 92.8% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.1.log.gz 2024-04-02T14:07:27.693 INFO:teuthology.orchestra.run.smithi088.stderr: 92.4% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.10.log.gz 2024-04-02T14:07:27.711 INFO:teuthology.orchestra.run.smithi189.stderr: 92.9% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.8.log.gz 2024-04-02T14:07:27.950 INFO:teuthology.orchestra.run.smithi052.stderr: 92.9% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.0.log.gz 2024-04-02T14:07:28.001 INFO:teuthology.orchestra.run.smithi189.stderr: 93.1% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.11.log.gz 2024-04-02T14:07:28.444 INFO:teuthology.orchestra.run.smithi189.stderr: 93.0% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.2.log.gz 2024-04-02T14:07:28.469 INFO:teuthology.orchestra.run.smithi052.stderr: 92.9% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.6.log.gz 2024-04-02T14:07:28.527 INFO:teuthology.orchestra.run.smithi088.stderr: 92.6% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.7.log.gz 2024-04-02T14:07:28.605 INFO:teuthology.orchestra.run.smithi189.stderr: 92.4% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.5.log.gz 2024-04-02T14:07:28.607 INFO:teuthology.orchestra.run.smithi189.stderr: 2024-04-02T14:07:28.607 INFO:teuthology.orchestra.run.smithi189.stderr:real 0m3.402s 2024-04-02T14:07:28.607 INFO:teuthology.orchestra.run.smithi189.stderr:user 0m12.159s 2024-04-02T14:07:28.608 INFO:teuthology.orchestra.run.smithi189.stderr:sys 0m0.477s 2024-04-02T14:07:28.643 INFO:teuthology.orchestra.run.smithi052.stderr: 92.5% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.9.log.gz 2024-04-02T14:07:28.836 INFO:teuthology.orchestra.run.smithi052.stderr: 92.8% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.3.log.gz 2024-04-02T14:07:28.837 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-04-02T14:07:28.838 INFO:teuthology.orchestra.run.smithi052.stderr:real 0m3.636s 2024-04-02T14:07:28.838 INFO:teuthology.orchestra.run.smithi052.stderr:user 0m14.300s 2024-04-02T14:07:28.838 INFO:teuthology.orchestra.run.smithi052.stderr:sys 0m0.498s 2024-04-02T14:07:29.305 INFO:teuthology.orchestra.run.smithi088.stderr: 93.1% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-osd.4.log.gz 2024-04-02T14:07:42.540 INFO:teuthology.orchestra.run.smithi088.stderr: 93.6% -- replaced with /var/log/ceph/bfa00e6e-f0f6-11ee-b647-cb9ed24678a4/ceph-mds.b.log.gz 2024-04-02T14:07:42.541 INFO:teuthology.orchestra.run.smithi088.stderr: 2024-04-02T14:07:42.541 INFO:teuthology.orchestra.run.smithi088.stderr:real 0m17.336s 2024-04-02T14:07:42.542 INFO:teuthology.orchestra.run.smithi088.stderr:user 0m30.696s 2024-04-02T14:07:42.542 INFO:teuthology.orchestra.run.smithi088.stderr:sys 0m1.042s 2024-04-02T14:07:42.542 INFO:tasks.cephadm:Archiving logs... 2024-04-02T14:07:42.542 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635602/remote/smithi052/log 2024-04-02T14:07:42.544 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-02T14:07:44.605 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635602/remote/smithi088/log 2024-04-02T14:07:44.606 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-02T14:07:48.660 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635602/remote/smithi189/log 2024-04-02T14:07:48.661 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-02T14:07:50.606 INFO:tasks.cephadm:Removing cluster... 2024-04-02T14:07:50.606 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 --force 2024-04-02T14:07:50.807 INFO:teuthology.orchestra.run.smithi052.stdout:Deleting cluster with fsid: bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T14:07:51.082 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 --force 2024-04-02T14:07:51.274 INFO:teuthology.orchestra.run.smithi088.stdout:Deleting cluster with fsid: bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T14:07:51.559 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 --force 2024-04-02T14:07:51.756 INFO:teuthology.orchestra.run.smithi189.stdout:Deleting cluster with fsid: bfa00e6e-f0f6-11ee-b647-cb9ed24678a4 2024-04-02T14:07:52.092 INFO:tasks.cephadm:Removing cephadm ... 2024-04-02T14:07:52.092 DEBUG:teuthology.orchestra.run.smithi052:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-02T14:07:52.110 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-02T14:07:52.126 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-02T14:07:52.141 INFO:tasks.cephadm:Teardown complete 2024-04-02T14:07:52.142 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-02T14:07:52.155 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-02T14:07:52.155 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-02T14:07:52.157 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-02T14:07:52.168 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-02T14:07:52.286 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-02T14:07:52.296 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-02T14:07:52.296 DEBUG:teuthology.orchestra.run.smithi052:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T14:07:52.299 DEBUG:teuthology.orchestra.run.smithi088:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T14:07:52.302 DEBUG:teuthology.orchestra.run.smithi189:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T14:07:52.313 INFO:teuthology.orchestra.run.smithi052.stderr:bash: line 1: ntpq: command not found 2024-04-02T14:07:52.313 INFO:teuthology.orchestra.run.smithi088.stderr:bash: line 1: ntpq: command not found 2024-04-02T14:07:52.315 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-04-02T14:07:52.318 INFO:teuthology.orchestra.run.smithi088.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T14:07:52.318 INFO:teuthology.orchestra.run.smithi088.stdout:=============================================================================== 2024-04-02T14:07:52.318 INFO:teuthology.orchestra.run.smithi088.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 60 -691us[ -691us] +/- 83ms 2024-04-02T14:07:52.318 INFO:teuthology.orchestra.run.smithi088.stdout:^* hv02.front.sepia.ceph.com 3 7 377 120 -1464us[-1352us] +/- 65ms 2024-04-02T14:07:52.318 INFO:teuthology.orchestra.run.smithi088.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 59 +2396us[+2396us] +/- 79ms 2024-04-02T14:07:52.318 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T14:07:52.320 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T14:07:52.320 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-04-02T14:07:52.320 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 57 -680us[ -680us] +/- 83ms 2024-04-02T14:07:52.320 INFO:teuthology.orchestra.run.smithi189.stdout:^* hv02.front.sepia.ceph.com 3 6 377 61 -1466us[-1456us] +/- 66ms 2024-04-02T14:07:52.320 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 56 +2411us[+2411us] +/- 79ms 2024-04-02T14:07:52.320 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T14:07:52.320 INFO:teuthology.orchestra.run.smithi052.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T14:07:52.320 INFO:teuthology.orchestra.run.smithi052.stdout:=============================================================================== 2024-04-02T14:07:52.321 INFO:teuthology.orchestra.run.smithi052.stdout:^+ hv01.front.sepia.ceph.com 2 8 377 58 -571us[ -579us] +/- 84ms 2024-04-02T14:07:52.321 INFO:teuthology.orchestra.run.smithi052.stdout:^* hv02.front.sepia.ceph.com 3 6 377 56 -1402us[-1410us] +/- 66ms 2024-04-02T14:07:52.321 INFO:teuthology.orchestra.run.smithi052.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 56 +2478us[+2478us] +/- 80ms 2024-04-02T14:07:52.321 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T14:07:52.321 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-02T14:07:52.331 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-02T14:07:52.331 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-02T14:07:52.348 DEBUG:teuthology.orchestra.run.smithi052:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-02T14:07:52.451 DEBUG:teuthology.orchestra.run.smithi088:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-02T14:07:52.522 DEBUG:teuthology.orchestra.run.smithi189:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-02T14:07:52.593 DEBUG:teuthology.orchestra.run.smithi052:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T14:07:52.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:07:52.627 DEBUG:teuthology.orchestra.run.smithi088:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T14:07:52.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:07:52.657 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T14:07:52.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:07:52.687 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-02T14:07:52.701 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-02T14:07:52.716 INFO:teuthology.task.internal:Duration was 1956.581769 seconds 2024-04-02T14:07:52.716 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-02T14:07:52.730 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-02T14:07:52.730 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-02T14:07:52.733 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-02T14:07:52.735 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-02T14:07:52.776 INFO:teuthology.orchestra.run.smithi052.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T14:07:52.776 INFO:teuthology.orchestra.run.smithi088.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T14:07:52.779 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T14:07:53.200 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-02T14:07:53.201 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi052.front.sepia.ceph.com 2024-04-02T14:07:53.201 DEBUG:teuthology.orchestra.run.smithi052:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-02T14:07:53.223 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi088.front.sepia.ceph.com 2024-04-02T14:07:53.223 DEBUG:teuthology.orchestra.run.smithi088:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-02T14:07:53.267 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi189.front.sepia.ceph.com 2024-04-02T14:07:53.267 DEBUG:teuthology.orchestra.run.smithi189:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-02T14:07:53.299 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-02T14:07:53.299 DEBUG:teuthology.orchestra.run.smithi052:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-02T14:07:53.302 DEBUG:teuthology.orchestra.run.smithi088:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-02T14:07:53.309 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-02T14:07:53.368 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-02T14:07:53.368 DEBUG:teuthology.orchestra.run.smithi052:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-02T14:07:53.373 DEBUG:teuthology.orchestra.run.smithi088:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-02T14:07:53.377 DEBUG:teuthology.orchestra.run.smithi189:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-02T14:07:53.787 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-02T14:07:53.799 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-02T14:07:53.799 DEBUG:teuthology.orchestra.run.smithi052:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-02T14:07:53.826 DEBUG:teuthology.orchestra.run.smithi088:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-02T14:07:53.854 DEBUG:teuthology.orchestra.run.smithi189:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-02T14:07:53.881 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-02T14:07:53.894 DEBUG:teuthology.orchestra.run.smithi052:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-02T14:07:53.897 DEBUG:teuthology.orchestra.run.smithi088:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-02T14:07:53.899 DEBUG:teuthology.orchestra.run.smithi189:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-02T14:07:53.920 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern = core 2024-04-02T14:07:53.922 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern = core 2024-04-02T14:07:53.947 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = core 2024-04-02T14:07:54.000 DEBUG:teuthology.orchestra.run.smithi052:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-02T14:07:54.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:07:54.014 DEBUG:teuthology.orchestra.run.smithi088:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-02T14:07:54.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:07:54.028 DEBUG:teuthology.orchestra.run.smithi189:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-02T14:07:54.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:07:54.054 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-02T14:07:54.066 INFO:teuthology.task.internal:Transferring archived files... 2024-04-02T14:07:54.067 DEBUG:teuthology.misc:Transferring archived files from smithi052:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635602/remote/smithi052 2024-04-02T14:07:54.067 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-02T14:07:54.111 DEBUG:teuthology.misc:Transferring archived files from smithi088:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635602/remote/smithi088 2024-04-02T14:07:54.111 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-02T14:07:54.149 DEBUG:teuthology.misc:Transferring archived files from smithi189:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635602/remote/smithi189 2024-04-02T14:07:54.149 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-02T14:07:54.188 INFO:teuthology.task.internal:Removing archive directory... 2024-04-02T14:07:54.188 DEBUG:teuthology.orchestra.run.smithi052:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-02T14:07:54.190 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-02T14:07:54.193 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-02T14:07:54.236 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-02T14:07:54.249 INFO:teuthology.task.internal:Not uploading archives. 2024-04-02T14:07:54.249 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-02T14:07:54.261 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-02T14:07:54.261 DEBUG:teuthology.orchestra.run.smithi052:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-02T14:07:54.263 DEBUG:teuthology.orchestra.run.smithi088:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-02T14:07:54.266 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-02T14:07:54.275 INFO:teuthology.orchestra.run.smithi052.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 2 14:07 /home/ubuntu/cephtest 2024-04-02T14:07:54.277 INFO:teuthology.orchestra.run.smithi088.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 2 14:07 /home/ubuntu/cephtest 2024-04-02T14:07:54.291 INFO:teuthology.orchestra.run.smithi189.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 2 14:07 /home/ubuntu/cephtest 2024-04-02T14:07:54.292 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-02T14:07:54.304 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-02T14:07:54.329 INFO:teuthology.nuke:Checking targets against current locks 2024-04-02T14:07:54.356 DEBUG:teuthology.nuke:shortname: smithi052 2024-04-02T14:07:54.356 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T14:07:54.380 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi052.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635602', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 13:25:59.065404', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ4hvD4dzn+gihLLou7tjNJeN+pPC+08Djcuvx2kkrTbIFRynm6+soIhDSKaNFH5WTOXNg0msf8ZUHbDhabFDLM='} 2024-04-02T14:07:54.384 DEBUG:teuthology.nuke:shortname: smithi088 2024-04-02T14:07:54.384 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T14:07:54.410 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi088.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635602', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 13:25:59.066622', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLEnBstAMH77Vw5uhL3R5WQH+sBW3hvPvnK1HFA2D4RUwPKYRw1e9a4xZzv2/fShvfc5EeYAVjQtNzAYZaTZxHs='} 2024-04-02T14:07:54.411 DEBUG:teuthology.nuke:shortname: smithi189 2024-04-02T14:07:54.412 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T14:07:54.443 INFO:teuthology.orchestra.console.smithi052:Power off 2024-04-02T14:07:54.444 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T14:07:54.455 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi189.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635602', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 13:25:59.063604', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPIdCMVgKBubA6WSEoJW9Dl19J6G5iL0VTTqWlyAu+Agou+bdmHcqVF5rdr2e5CSJ5YWxSQVBMBPWbIz1mnjZZM='} 2024-04-02T14:07:54.468 DEBUG:teuthology.orchestra.console.smithi052:power off output: Chassis Power Control: Down/Off 2024-04-02T14:07:54.468 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:07:54.484 INFO:teuthology.orchestra.console.smithi088:Power off 2024-04-02T14:07:54.485 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T14:07:54.503 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is on 2024-04-02T14:07:54.512 DEBUG:teuthology.orchestra.console.smithi088:power off output: Chassis Power Control: Down/Off 2024-04-02T14:07:54.512 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:07:54.533 INFO:teuthology.orchestra.console.smithi189:Power off 2024-04-02T14:07:54.534 DEBUG:teuthology.orchestra.console.smithi189:pexpect command: ipmitool -H smithi189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T14:07:54.543 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on 2024-04-02T14:07:54.556 DEBUG:teuthology.orchestra.console.smithi189:power off output: Chassis Power Control: Down/Off 2024-04-02T14:07:54.556 DEBUG:teuthology.orchestra.console.smithi189:pexpect command: ipmitool -H smithi189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:07:54.582 DEBUG:teuthology.orchestra.console.smithi189:check power output: Chassis Power is on 2024-04-02T14:07:58.504 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:07:58.544 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:07:58.583 DEBUG:teuthology.orchestra.console.smithi189:pexpect command: ipmitool -H smithi189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:07:58.614 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is on 2024-04-02T14:07:58.654 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on 2024-04-02T14:07:58.693 DEBUG:teuthology.orchestra.console.smithi189:check power output: Chassis Power is on 2024-04-02T14:08:02.619 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:08:02.655 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:08:02.694 DEBUG:teuthology.orchestra.console.smithi189:pexpect command: ipmitool -H smithi189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:08:02.730 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is off 2024-04-02T14:08:02.764 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is off 2024-04-02T14:08:02.804 DEBUG:teuthology.orchestra.console.smithi189:check power output: Chassis Power is on 2024-04-02T14:08:02.831 INFO:teuthology.orchestra.console.smithi052:Power off completed 2024-04-02T14:08:02.865 INFO:teuthology.orchestra.console.smithi088:Power off completed 2024-04-02T14:08:06.807 DEBUG:teuthology.orchestra.console.smithi189:pexpect command: ipmitool -H smithi189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:08:06.916 DEBUG:teuthology.orchestra.console.smithi189:check power output: Chassis Power is off 2024-04-02T14:08:07.017 INFO:teuthology.orchestra.console.smithi189:Power off completed 2024-04-02T14:08:07.124 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsx}} duration: 1956.5817692279816 failure_reason: 'Command failed (workunit test suites/fsx.sh) on smithi052 with status 2: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=191d963ac9de7322da8aff20b567b3fa458e2ef6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh''' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=526d8653bc7b422e89150358f99eb5b5 status: fail success: false 2024-04-02T14:08:07.124 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T14:08:07.213 INFO:teuthology.run:FAIL